This commit is contained in:
DoTheEvo 2023-03-05 10:50:50 +01:00
parent 142879cfc2
commit f27dda894b
10 changed files with 440 additions and 32 deletions

View File

@ -43,9 +43,9 @@ Repo documents self hosted apps in similar format and also uses caddy for revers
- `docker-compose.yml` do not need any editing to get started,
changes are to be done in the `.env` file.
- Not using `ports` directive if theres only web traffic in a container.<br>
- Not using `ports` directive if theres only web traffic for a container.<br>
Theres an expectation of running a reverse proxy which makes mapping ports
on docker host unnecessary. Instead `expose` is used which is basically
to a docker host unnecessary. Instead `expose` is used which is basically
just documentation.<br>
- For persistent storage bind mount `./whatever_data` is used.
No volumes, nor static path somewhere... just relative path next to compose file.
@ -60,13 +60,31 @@ your first time running a docker container.
# Some extra info
### Caddy
Kinda the core of the setup is Caddy reverse proxy.</br>
It's described in most details, it's really amazingly simple but robust software.
### Compose
All guides have reverse proxy section with Caddyfile config for them.
When making changes use `docker-compose down` and `docker-compose up -d`,
not just restart or stop/start.
---
### Docker network
You really want to create a custom docker bridge network and use it.
`docker network create caddy_net`
It can be named whatever, but what it does over default is that it provides
[automatic DNS resolution](https://docs.docker.com/network/bridge/)
between containers. Meaning one can exec in to caddy container and ping another
container on that custom docker network by its hostname.
So config files can just use hostnames and they will work.
---
### .env
Often the `.env` file is used as `env_file`,
which can be a bit difficult concept at a first glance.
@ -138,8 +156,25 @@ It is absofuckinglutely amazing in how simple yet effective it is.
* detailed info on a container, it's IP, published and exposed ports, when it was created,..
* quick management, quick exec in to a container, check logs, stop it,...
Written in Go, so its super fast and installation is trivial when it is a single binary,
as likely your distro does not have it in repos. If you use arch, like I do, its on AUR.
Written in Go, so its super fast and installation is trivial when it is a single binary.<br>
download `linux-amd64` version; make it executable with chmod +x; move it to `/usr/bin/`;
now you can ctop anywhere.
---
### Sendinblue
Services often need ability to send emails, for notification, registration,
password reset and such... Sendinblue is free, offers 300 mails a day
and is easy to setup.
```
EMAIL_HOST=smtp-relay.sendinblue.com
EMAIL_PORT=587
EMAIL_HOST_USER=whoever_example@gmail.com>
EMAIL_HOST_PASSWORD=xcmpwik-c31d9eykwewf2342df2fwfj04-FKLzpHgMjGqP23
EMAIL_USE_TLS=1
```
---
@ -160,30 +195,41 @@ so it might be worth the time to check out the concept to setup own ansible scri
The best aspect of having such repo is that it is a dedicated place where
one can write solution to issues encountered,
or enable freshly discovered feature for all deployments.
or enable freshly discovered feature for all future deployments.
---
### Sendinblue
Services often need ability to send emails, for registration, password reset and such...
Sendinblue offers 300 mails a day and is easy to setup.
```
EMAIL_HOST=smtp-relay.sendinblue.com
EMAIL_PORT=587
EMAIL_HOST_USER=<registration_email@gmail.com>
EMAIL_HOST_PASSWORD=xs...... S1Rzp
EMAIL_USE_TLS=1
```
---
### other guides
### Other guides
* [StarWhiz/docker_deployment_notes](https://github.com/StarWhiz/docker_deployment_notes)
- got inspired and wrote in similar way setup for various services
* [BaptisteBdn/docker-selfhosted-apps](https://github.com/BaptisteBdn/docker-selfhosted-apps)
- many services using traefik for reverse proxy
* [Awesome Docker Compose Examples](https://github.com/Haxxnet/Compose-Examples)
---
### For docker noobs
First, docker is easy. Like really.<br>
Second, there are two main uses.
* A developer who daily works on an app and docker eases everything about it, from testing to deployment
* A self-hosting / devops approach, where you just want to get a service running
This whole repo is obviously about the second use. So be careful that you wont
spend time on resources used to educate the developers. I mean if you get through
that you will know docker better. But theres always danger that after sinking
4 hours in to learning it, one cant even get nginx web server up and working.
And my personal preference in learning is getting something up as fast as possible
and then tinker with it and try to understand it.
So to solve this, when googling for guides, look for *docker compose*
rather than just docker tutorials and notice if they are talking fundamentals or
deployment.
* [This](https://youtu.be/DM65_JyGxCo) one is pretty good. That entire channel
has good stuff worth looking.
Will add stuff as I encounter it.

View File

@ -18,3 +18,5 @@ Most of the stuff around here is about deployment, how to deploy others people w
# Basics
tag - not just version number, tags are not even present on local docker..
ports quotes - always in quotes because in low number it might try to interpret ports as base 60

View File

@ -24,9 +24,10 @@ Instant notifications if email feels old timey and crowded
Support for multiple user, supports ios.
* **signal-cli-rest-api** - no gui, need a sim card phone number registred,
notification are just send to phone numbers.
The wider spread of it might make it a winner since no need for another app.
Signal wide spread might make it a winner, since you are not asking people
to install an another app.
Afte few weeks of tinkering with these... ntfy is the winner for me, for now.<br>
Afte few weeks of tinkering with these... **ntfy is the winner for me**, for now.<br>
Compose files for the other two are at the end.
# docker-compose for ntfy
@ -141,6 +142,47 @@ Type=simple
ExecStart=/opt/borg_backup.sh
```
# Grafana to ntfy
Alerting in grafana to ntfy [works](https://github.com/DoTheEvo/selfhosted-apps-docker/tree/master/prometheus_grafana#alertmanager),
but its ugly with just json shown.
To solve this
* deploy container [grafana-to-ntfy](https://github.com/kittyandrew/grafana-to-ntfy).
Should be on the same network with grafana.
Set in `.env` ntfy url of your ntfy server and specific topic
* in grafana set contact point webhook aimed at `http://grafana-to-ntfy:8080`,
with credentials from the `.env`
`docker-compose.yml`
```yml
services:
grafana-to-ntfy:
container_name: grafana-to-ntfy
hostname: grafana-to-ntfy
image: kittyandrew/grafana-to-ntfy
restart: unless-stopped
env_file:
- .env
networks:
default:
name: $DOCKER_MY_NETWORK
external: true
```
`.env`
```php
# GENERAL
DOCKER_MY_NETWORK=caddy_net
TZ=Europe/Bratislava
NTFY_URL=https://ntfy.example.com/grafana
BAUTH_USER=admin
BAUTH_PASS=test
```
<details>
<summary><h1>gotify and signal compose</h1></summary>

View File

@ -0,0 +1,11 @@
$REPOSITORY_PATH = 'C:\Backup'
$BACKUP_THIS = 'C:\Users'
$KOPIA_PASSWORD='aaa'
kopia policy set $BACKUP_THIS --before-folder-action "powershell -WindowStyle Hidden C:\win_vss_before.ps1"
kopia policy set $BACKUP_THIS --after-folder-action "powershell -WindowStyle Hidden C:\win_vss_after.ps1"
kopia repository connect filesystem --path $REPOSITORY_PATH --password $KOPIA_PASSWORD
kopia snapshot create $BACKUP_THIS
kopia repository disconnect

View File

@ -26,8 +26,10 @@ but general use and concepts are universal.</br>
# Some aspects of Kopia
* Kopia configuraiton uses term policies to apply to various
- global policy, from which repos inherit settings
- repos policy created on repo creation
* Backup configuration is stored in a repository where backups are stored.<br>
This includes global policy, that is global in sense of a repo, not all of kopia.
* You connect to a repository before using it, and disconnect afterwards.<br>
Only one repository can be connected at the time(at least for cli version).
* Currently to ignore a folder - `CACHEDIR.TAG` file can be placed inside,
@ -104,9 +106,9 @@ use of sudo so that kopia has access everywhere<br>
REPOSITORY_PATH='/mnt/mirror/KOPIA/docker_host_kopia'
BACKUP_THIS='/home /etc'
export KOPIA_PASSWORD='aaa'
KOPIA_PASSWORD='aaa'
kopia repository connect filesystem --path $REPOSITORY_PATH
kopia repository connect filesystem --path $REPOSITORY_PATH --password $KOPIA_PASSWORD
kopia snapshot create $BACKUP_THIS
kopia repository disconnect
```
@ -200,5 +202,28 @@ WantedBy=multi-user.target
# Remote backup
...
... some day ...
# Kopia in Windows
While GUI version seems like a way to go.. well its not there yet.
The schedule it uses is running only under a user, theres no certainty it will run.
So here goes cli version
* [download](https://github.com/kopia/kopia/releases/) latest named kopia-X.XX.X-windows-x64.zip
, \~11MB
* extract, move to `C:\kopia`
* download `win_vss_before.ps1` and `win_vss_after.ps1` from this repo,
or crete them from
[here](https://kopia.io/docs/advanced/actions/#windows-shadow-copy)
* kopia-backup-home-etc.sh
* powershell as as administrator
* --enable-actions
* in tray, right click on the icon - `Launch At Startup`<br>
this creates registry entry - *HKCU\Software\Microsoft\Windows\CurrentVersion\Run\KopiaUI*
*
kopia policy set <target_dir> --before-folder-action "powershell -WindowStyle Hidden <path_to_script>\before.ps1"
kopia policy set <target_dir> --after-folder-action "powershell -WindowStyle Hidden <path_to_script>\after.ps1"

View File

@ -0,0 +1,18 @@
if ($args.Length -eq 0) {
$kopiaSnapshotId = $env:KOPIA_SNAPSHOT_ID
} else {
$kopiaSnapshotId = $args[0]
}
if (([Security.Principal.WindowsPrincipal][Security.Principal.WindowsIdentity]::GetCurrent()).IsInRole([Security.Principal.WindowsBuiltInRole] 'Administrator')) {
$mountPoint = Get-Item "${PSScriptRoot}\${kopiaSnapshotId}"
$mountedVolume = $mountPoint.Target
cmd /c rmdir $mountPoint
Get-CimInstance -ClassName Win32_ShadowCopy | Where-Object { "$($_.DeviceObject)\" -eq "\\?\${mountedVolume}" } | Remove-CimInstance
} else {
Start-Process 'powershell' '-f', $MyInvocation.MyCommand.Path, $kopiaSnapshotId -Verb RunAs -WindowStyle Hidden -Wait
if ($proc.ExitCode) {
exit $proc.ExitCode
}
}

View File

@ -0,0 +1,30 @@
if ($args.Length -eq 0) {
$kopiaSnapshotId = $env:KOPIA_SNAPSHOT_ID
$kopiaSourcePath = $env:KOPIA_SOURCE_PATH
} else {
$kopiaSnapshotId = $args[0]
$kopiaSourcePath = $args[1]
}
$sourceDrive = Split-Path -Qualifier $kopiaSourcePath
$sourcePath = Split-Path -NoQualifier $kopiaSourcePath
# use Kopia snapshot ID as mount point name for extra caution for duplication
$mountPoint = "${PSScriptRoot}\${kopiaSnapshotId}"
if (([Security.Principal.WindowsPrincipal][Security.Principal.WindowsIdentity]::GetCurrent()).IsInRole([Security.Principal.WindowsBuiltInRole] 'Administrator')) {
$shadowId = (Invoke-CimMethod -ClassName Win32_ShadowCopy -MethodName Create -Arguments @{ Volume = "${sourceDrive}\" }).ShadowID
$shadowDevice = (Get-CimInstance -ClassName Win32_ShadowCopy | Where-Object { $_.ID -eq $shadowId }).DeviceObject
if (-not $shadowDevice) {
# fail the Kopia snapshot early if shadow copy was not created
exit 1
}
cmd /c mklink /d $mountPoint "${shadowDevice}\"
} else {
$proc = Start-Process 'powershell' '-f', $MyInvocation.MyCommand.Path, $kopiaSnapshotId, $kopiaSourcePath -PassThru -Verb RunAs -WindowStyle Hidden -Wait
if ($proc.ExitCode) {
exit $proc.ExitCode
}
}
Write-Output "KOPIA_SNAPSHOT_PATH=${mountPoint}${sourcePath}"

230
mealie/readme.md Normal file
View File

@ -0,0 +1,230 @@
a# Mealie in docker
###### guide-by-example
![logo](https://i.imgur.com/qDXwqaU.png)
# Purpose & Overview
Documentation and notes.
* [Official site](https://www.bookstackapp.com/)
* [Github](https://github.com/BookStackApp/BookStack)
* [DockerHub](https://hub.docker.com/r/linuxserver/bookstack)
BookStack is a modern, open source, good looking wiki platform
for storing and organizing information.
Written in PHP, using Laravel framework, with MySQL database for the user data.</br>
There is no official Dockerhub image so the one maintained by
[linuxserver.io](https://www.linuxserver.io/) is used,
which uses nginx as a web server.
# Files and directory structure
```
/home/
└── ~/
└── docker/
└── bookstack/
├── 🗁 bookstack_data/
├── 🗁 bookstack_db_data/
├── 🗋 .env
├── 🗋 docker-compose.yml
└── 🗋 bookstack-backup-script.sh
```
* `bookstack_data/` - a directory with bookstacks web app data
* `bookstack_db_data/` - a directory with database data
* `.env` - a file containing environment variables for docker compose
* `docker-compose.yml` - a docker compose file, telling docker how to run the containers
* `bookstack-backup-script.sh` - a backup script, to be run daily
Only the files are required. The directories are created on the first run.
# docker-compose
Dockerhub linuxserver/bookstack
[example compose.](https://hub.docker.com/r/linuxserver/bookstack)
`docker-compose.yml`
```yml
services:
mealie:
image: hkotel/mealie
container_name: mealie
hostname: mealie
restart: unless-stopped
env_file: .env
volumes:
- ./mealie_data:/app/data
expose:
- 80:80
networks:
default:
name: $DOCKER_MY_NETWORK
external: true
```
`.env`
```bash
# GENERAL
DOCKER_MY_NETWORK=caddy_net
TZ=Europe/Bratislava
# MEALIE
PUID=1000
PGID=1000
RECIPE_PUBLIC=true
RECIPE_SHOW_NUTRITION=true
RECIPE_SHOW_ASSETS=true
RECIPE_LANDSCAPE_VIEW=true
RECIPE_DISABLE_COMMENTS=false
RECIPE_DISABLE_AMOUNT=false
# USING SENDINBLUE FOR SENDING EMAILS
MAIL_DRIVER=smtp
MAIL_ENCRYPTION=tls
MAIL_HOST=smtp-relay.sendinblue.com
MAIL_PORT=587
MAIL_FROM=book@example.com
MAIL_USERNAME=<registration-email@gmail.com>
MAIL_PASSWORD=<sendinblue-smtp-key-goes-here>
```
**All containers must be on the same network**.</br>
Which is named in the `.env` file.</br>
If one does not exist yet: `docker network create caddy_net`
`APP_URL` in the `.env` **must be set** for bookstack to work.<br>
`MAIL_` stuff must be set for password reset and new registrations.
# Reverse proxy
Caddy v2 is used, details
[here](https://github.com/DoTheEvo/selfhosted-apps-docker/tree/master/caddy_v2).</br>
`Caddyfile`
```php
book.{$MY_DOMAIN} {
reverse_proxy bookstack:80
}
```
# First run
Default login: `admin@admin.com` // `password`
---
![interface-pic](https://i.imgur.com/cN1GUZw.png)
# Trouble shooting
* It did not start.<br>
Ctrl+f in `.env` file for word `example` to be replaced with actual domain
name. `APP_URL` has to be set correctly for bookstack to work.
* After update cant see edit tools.<br>
Clear browsers cookies/cache.
* The test email button in preferences throws error.<br>
Exec in to the container and `printenv` to see.
Check [mail.php](https://github.com/BookStackApp/BookStack/blob/development/app/Config/mail.php)
to see exact `MAIL_` env variables names and default values.
Test in Thunderbird your smtp server working or not.
# Update
Manual image update:
- `docker-compose pull`</br>
- `docker-compose up -d`</br>
- `docker image prune`
It is **strongly recommended** to now add current **tags** to the images in the compose.<br>
Tags will allow you to easily return to a working state if an update goes wrong.
If there was a **major version jump**, and bookstack does not work,
exec in to the app container and run php artisan migrate</br>
`docker container exec -it bookstack /bin/bash`</br>
`cd /app/www`</br>
`php artisan migrate`
# Backup and restore
#### Backup
Using [kopia](https://github.com/DoTheEvo/selfhosted-apps-docker/tree/master/kopia_backup)
or [borg](https://github.com/DoTheEvo/selfhosted-apps-docker/tree/master/borg_backup)
to make daily snapshot of the entire docker directory.
#### Restore
* down the containers `docker-compose down`</br>
* delete/move/rename the entire project directory</br>
* from the backups copy back the entire project directory</br>
* start the containers `docker-compose up -d`
# Backup of just user data
Users data daily export using the
[official procedure.](https://www.bookstackapp.com/docs/admin/backup-restore/)</br>
For bookstack it means database dump and backing up several directories
containing user uploaded files.
Daily kopia/borg backup run takes care of backing up the directories.
So only database dump is needed and done with the script.</br>
The created backup sql file is overwritten on every run of the script,
but that's ok since kopia/borg are keeping daily snapshots.
#### Backup script
Placed inside `bookstack` directory on the host
`bookstack-backup-script.sh`
```bash
#!/bin/bash
# CREATE DATABASE DUMP, bash -c '...' IS USED OTHERWISE OUTPUT > WOULD TRY TO GO TO THE HOST
docker container exec bookstack-db bash -c 'mysqldump -u $MYSQL_USER -p$MYSQL_PASSWORD $MYSQL_DATABASE > $MYSQL_DIR/BACKUP.bookstack.database.sql'
```
The script must be **executable** - `chmod +x bookstack-backup-script.sh`
#### Cronjob - scheduled backup
Running on the host
* `su` - switch to root
* `crontab -e` - add new cron job</br>
* `0 22 * * * /home/bastard/docker/bookstack/bookstack-backup-script.sh`</br>
runs it every day [at 22:00](https://crontab.guru/#0_22_*_*_*)
* `crontab -l` - list cronjobs to check
# Restore the user data
Assuming clean start and latest images.<br>
Will need `BACKUP.bookstack.database.sql` and content of `bookstack_data/www/`<br>
Note that database restore must happen before bookstack app is first run.
* start only the database container: `docker-compose up -d bookstack-db`
* copy `BACKUP.bookstack.database.sql` in `bookstack/bookstack_db_data/`
* restore the database inside the container</br>
`docker container exec --workdir /config bookstack-db bash -c 'mysql -u $MYSQL_USER -p$MYSQL_PASSWORD $MYSQL_DATABASE < BACKUP.bookstack.database.sql'`
* now start the app container: `docker-compose up -d`
* let it run so it creates its file structure
* down the containers `docker-compose down`
* in `bookstack/bookstack_data/www/`</br>
replace directories `files`,`images`,`uploads` and the file `.env`</br>
with the ones from the BorgBackup repository
* start the containers: `docker-compose up -d`
* if there was a major version jump, exec in to the app container and run `php artisan migrate`</br>
`docker container exec -it bookstack /bin/bash`</br>
`cd /app/www`</br>
`php artisan migrate`
Again, the above steps are based on the
[official procedure](https://www.bookstackapp.com/docs/admin/backup-restore/)
at the time of writing this.

View File

@ -276,7 +276,9 @@ switch to sata ssd seems to prevent any more occurancies of this high disk usage
# Comamnds & settings
* `/gamerule playersSleepingPercentage 1` - use bed whenever, sleep not dependant on other players
* to import a map, download, extract, copy the directory to minecraft-data<br>
`mv import <directory-name> normal`, there should be no spaces in the name
* `/mv setspawn` sets spawn point in the current world
# Update

View File

@ -32,6 +32,8 @@ Lot of the prometheus stuff here is based off the magnificent
* **[Loki](#Loki)** - all of the above but for log files
* **[Caddy monitoring](#Caddy_monitoring)** - monitoring a reverse proxy
![dashboards_pic](https://i.imgur.com/ac9Qj1F.png)
# Overview
[Good youtube overview](https://youtu.be/h4Sl21AKiDg) of Prometheus.</br>