diff --git a/README.md b/README.md index 050bd69..8d7c3c8 100644 --- a/README.md +++ b/README.md @@ -43,9 +43,9 @@ Repo documents self hosted apps in similar format and also uses caddy for revers - `docker-compose.yml` do not need any editing to get started, changes are to be done in the `.env` file. -- Not using `ports` directive if theres only web traffic in a container.
+- Not using `ports` directive if theres only web traffic for a container.
Theres an expectation of running a reverse proxy which makes mapping ports - on docker host unnecessary. Instead `expose` is used which is basically + to a docker host unnecessary. Instead `expose` is used which is basically just documentation.
- For persistent storage bind mount `./whatever_data` is used. No volumes, nor static path somewhere... just relative path next to compose file. @@ -60,13 +60,31 @@ your first time running a docker container. # Some extra info +### Caddy + Kinda the core of the setup is Caddy reverse proxy.
It's described in most details, it's really amazingly simple but robust software. -### Compose +All guides have reverse proxy section with Caddyfile config for them. -When making changes use `docker-compose down` and `docker-compose up -d`, -not just restart or stop/start. +--- + +### Docker network + +You really want to create a custom docker bridge network and use it. + +`docker network create caddy_net` + +It can be named whatever, but what it does over default is that it provides +[automatic DNS resolution](https://docs.docker.com/network/bridge/) +between containers. Meaning one can exec in to caddy container and ping another +container on that custom docker network by its hostname. + +So config files can just use hostnames and they will work. + +--- + +### .env Often the `.env` file is used as `env_file`, which can be a bit difficult concept at a first glance. @@ -138,8 +156,25 @@ It is absofuckinglutely amazing in how simple yet effective it is. * detailed info on a container, it's IP, published and exposed ports, when it was created,.. * quick management, quick exec in to a container, check logs, stop it,... -Written in Go, so its super fast and installation is trivial when it is a single binary, -as likely your distro does not have it in repos. If you use arch, like I do, its on AUR. +Written in Go, so its super fast and installation is trivial when it is a single binary.
+download `linux-amd64` version; make it executable with chmod +x; move it to `/usr/bin/`; +now you can ctop anywhere. + +--- + +### Sendinblue + +Services often need ability to send emails, for notification, registration, +password reset and such... Sendinblue is free, offers 300 mails a day +and is easy to setup. + +``` +EMAIL_HOST=smtp-relay.sendinblue.com +EMAIL_PORT=587 +EMAIL_HOST_USER=whoever_example@gmail.com> +EMAIL_HOST_PASSWORD=xcmpwik-c31d9eykwewf2342df2fwfj04-FKLzpHgMjGqP23 +EMAIL_USE_TLS=1 +``` --- @@ -160,30 +195,41 @@ so it might be worth the time to check out the concept to setup own ansible scri The best aspect of having such repo is that it is a dedicated place where one can write solution to issues encountered, -or enable freshly discovered feature for all deployments. +or enable freshly discovered feature for all future deployments. --- -### Sendinblue - -Services often need ability to send emails, for registration, password reset and such... - -Sendinblue offers 300 mails a day and is easy to setup. - -``` -EMAIL_HOST=smtp-relay.sendinblue.com -EMAIL_PORT=587 -EMAIL_HOST_USER= -EMAIL_HOST_PASSWORD=xs...... S1Rzp -EMAIL_USE_TLS=1 -``` - ---- - -### other guides +### Other guides * [StarWhiz/docker_deployment_notes](https://github.com/StarWhiz/docker_deployment_notes) - got inspired and wrote in similar way setup for various services * [BaptisteBdn/docker-selfhosted-apps](https://github.com/BaptisteBdn/docker-selfhosted-apps) - many services using traefik for reverse proxy +* [Awesome Docker Compose Examples](https://github.com/Haxxnet/Compose-Examples) +--- + +### For docker noobs + +First, docker is easy. Like really.
+ +Second, there are two main uses. + +* A developer who daily works on an app and docker eases everything about it, from testing to deployment +* A self-hosting / devops approach, where you just want to get a service running + +This whole repo is obviously about the second use. So be careful that you wont +spend time on resources used to educate the developers. I mean if you get through +that you will know docker better. But theres always danger that after sinking +4 hours in to learning it, one cant even get nginx web server up and working. +And my personal preference in learning is getting something up as fast as possible +and then tinker with it and try to understand it. + +So to solve this, when googling for guides, look for *docker compose* +rather than just docker tutorials and notice if they are talking fundamentals or +deployment. + +* [This](https://youtu.be/DM65_JyGxCo) one is pretty good. That entire channel +has good stuff worth looking. + +Will add stuff as I encounter it. diff --git a/docker-basics-and-troubleshoot/readme.md b/docker-basics-and-troubleshoot/readme.md index 734212d..6fb957e 100644 --- a/docker-basics-and-troubleshoot/readme.md +++ b/docker-basics-and-troubleshoot/readme.md @@ -18,3 +18,5 @@ Most of the stuff around here is about deployment, how to deploy others people w # Basics tag - not just version number, tags are not even present on local docker.. + +ports quotes - always in quotes because in low number it might try to interpret ports as base 60 diff --git a/gotify-ntfy-signal/readme.md b/gotify-ntfy-signal/readme.md index 64ba5e4..cd23508 100644 --- a/gotify-ntfy-signal/readme.md +++ b/gotify-ntfy-signal/readme.md @@ -24,9 +24,10 @@ Instant notifications if email feels old timey and crowded Support for multiple user, supports ios. * **signal-cli-rest-api** - no gui, need a sim card phone number registred, notification are just send to phone numbers. - The wider spread of it might make it a winner since no need for another app. + Signal wide spread might make it a winner, since you are not asking people + to install an another app. -Afte few weeks of tinkering with these... ntfy is the winner for me, for now.
+Afte few weeks of tinkering with these... **ntfy is the winner for me**, for now.
Compose files for the other two are at the end. # docker-compose for ntfy @@ -141,6 +142,47 @@ Type=simple ExecStart=/opt/borg_backup.sh ``` +# Grafana to ntfy + +Alerting in grafana to ntfy [works](https://github.com/DoTheEvo/selfhosted-apps-docker/tree/master/prometheus_grafana#alertmanager), +but its ugly with just json shown. + +To solve this + +* deploy container [grafana-to-ntfy](https://github.com/kittyandrew/grafana-to-ntfy). + Should be on the same network with grafana. + Set in `.env` ntfy url of your ntfy server and specific topic +* in grafana set contact point webhook aimed at `http://grafana-to-ntfy:8080`, + with credentials from the `.env` + +`docker-compose.yml` +```yml +services: + grafana-to-ntfy: + container_name: grafana-to-ntfy + hostname: grafana-to-ntfy + image: kittyandrew/grafana-to-ntfy + restart: unless-stopped + env_file: + - .env + +networks: + default: + name: $DOCKER_MY_NETWORK + external: true +``` + +`.env` +```php +# GENERAL +DOCKER_MY_NETWORK=caddy_net +TZ=Europe/Bratislava + +NTFY_URL=https://ntfy.example.com/grafana +BAUTH_USER=admin +BAUTH_PASS=test +``` +

gotify and signal compose

 diff --git a/kopia_backup/kopia-backup-scipt.ps1 b/kopia_backup/kopia-backup-scipt.ps1 new file mode 100644 index 0000000..7198699 --- /dev/null +++ b/kopia_backup/kopia-backup-scipt.ps1 @@ -0,0 +1,11 @@ +$REPOSITORY_PATH = 'C:\Backup' +$BACKUP_THIS = 'C:\Users' +$KOPIA_PASSWORD='aaa' + +kopia policy set $BACKUP_THIS --before-folder-action "powershell -WindowStyle Hidden C:\win_vss_before.ps1" +kopia policy set $BACKUP_THIS --after-folder-action "powershell -WindowStyle Hidden C:\win_vss_after.ps1" + +kopia repository connect filesystem --path $REPOSITORY_PATH --password $KOPIA_PASSWORD +kopia snapshot create $BACKUP_THIS +kopia repository disconnect + diff --git a/kopia_backup/readme.md b/kopia_backup/readme.md index 14f59bd..f1c2631 100644 --- a/kopia_backup/readme.md +++ b/kopia_backup/readme.md @@ -26,8 +26,10 @@ but general use and concepts are universal.
# Some aspects of Kopia +* Kopia configuraiton uses term policies to apply to various + - global policy, from which repos inherit settings + - repos policy created on repo creation * Backup configuration is stored in a repository where backups are stored.
- This includes global policy, that is global in sense of a repo, not all of kopia. * You connect to a repository before using it, and disconnect afterwards.
Only one repository can be connected at the time(at least for cli version). * Currently to ignore a folder - `CACHEDIR.TAG` file can be placed inside, @@ -104,9 +106,9 @@ use of sudo so that kopia has access everywhere
REPOSITORY_PATH='/mnt/mirror/KOPIA/docker_host_kopia' BACKUP_THIS='/home /etc' -export KOPIA_PASSWORD='aaa' +KOPIA_PASSWORD='aaa' -kopia repository connect filesystem --path $REPOSITORY_PATH +kopia repository connect filesystem --path $REPOSITORY_PATH --password $KOPIA_PASSWORD kopia snapshot create $BACKUP_THIS kopia repository disconnect ``` @@ -200,5 +202,28 @@ WantedBy=multi-user.target # Remote backup -... +... some day ... +# Kopia in Windows + +While GUI version seems like a way to go.. well its not there yet. +The schedule it uses is running only under a user, theres no certainty it will run. + +So here goes cli version + +* [download](https://github.com/kopia/kopia/releases/) latest named kopia-X.XX.X-windows-x64.zip +, \~11MB +* extract, move to `C:\kopia` +* download `win_vss_before.ps1` and `win_vss_after.ps1` from this repo, + or crete them from + [here](https://kopia.io/docs/advanced/actions/#windows-shadow-copy) +* kopia-backup-home-etc.sh + +* powershell as as administrator +* --enable-actions +* in tray, right click on the icon - `Launch At Startup`
+ this creates registry entry - *HKCU\Software\Microsoft\Windows\CurrentVersion\Run\KopiaUI* +* + +kopia policy set --before-folder-action "powershell -WindowStyle Hidden \before.ps1" +kopia policy set --after-folder-action "powershell -WindowStyle Hidden \after.ps1" diff --git a/kopia_backup/win_vss_after.ps1 b/kopia_backup/win_vss_after.ps1 new file mode 100644 index 0000000..01c9fb8 --- /dev/null +++ b/kopia_backup/win_vss_after.ps1 @@ -0,0 +1,18 @@ +if ($args.Length -eq 0) { + $kopiaSnapshotId = $env:KOPIA_SNAPSHOT_ID +} else { + $kopiaSnapshotId = $args[0] +} + +if (([Security.Principal.WindowsPrincipal][Security.Principal.WindowsIdentity]::GetCurrent()).IsInRole([Security.Principal.WindowsBuiltInRole] 'Administrator')) { + $mountPoint = Get-Item "${PSScriptRoot}\${kopiaSnapshotId}" + $mountedVolume = $mountPoint.Target + + cmd /c rmdir $mountPoint + Get-CimInstance -ClassName Win32_ShadowCopy | Where-Object { "$($_.DeviceObject)\" -eq "\\?\${mountedVolume}" } | Remove-CimInstance +} else { + Start-Process 'powershell' '-f', $MyInvocation.MyCommand.Path, $kopiaSnapshotId -Verb RunAs -WindowStyle Hidden -Wait + if ($proc.ExitCode) { + exit $proc.ExitCode + } +} diff --git a/kopia_backup/win_vss_before.ps1 b/kopia_backup/win_vss_before.ps1 new file mode 100644 index 0000000..e3692b1 --- /dev/null +++ b/kopia_backup/win_vss_before.ps1 @@ -0,0 +1,30 @@ +if ($args.Length -eq 0) { + $kopiaSnapshotId = $env:KOPIA_SNAPSHOT_ID + $kopiaSourcePath = $env:KOPIA_SOURCE_PATH +} else { + $kopiaSnapshotId = $args[0] + $kopiaSourcePath = $args[1] +} + +$sourceDrive = Split-Path -Qualifier $kopiaSourcePath +$sourcePath = Split-Path -NoQualifier $kopiaSourcePath +# use Kopia snapshot ID as mount point name for extra caution for duplication +$mountPoint = "${PSScriptRoot}\${kopiaSnapshotId}" + +if (([Security.Principal.WindowsPrincipal][Security.Principal.WindowsIdentity]::GetCurrent()).IsInRole([Security.Principal.WindowsBuiltInRole] 'Administrator')) { + $shadowId = (Invoke-CimMethod -ClassName Win32_ShadowCopy -MethodName Create -Arguments @{ Volume = "${sourceDrive}\" }).ShadowID + $shadowDevice = (Get-CimInstance -ClassName Win32_ShadowCopy | Where-Object { $_.ID -eq $shadowId }).DeviceObject + if (-not $shadowDevice) { + # fail the Kopia snapshot early if shadow copy was not created + exit 1 + } + + cmd /c mklink /d $mountPoint "${shadowDevice}\" +} else { + $proc = Start-Process 'powershell' '-f', $MyInvocation.MyCommand.Path, $kopiaSnapshotId, $kopiaSourcePath -PassThru -Verb RunAs -WindowStyle Hidden -Wait + if ($proc.ExitCode) { + exit $proc.ExitCode + } +} + +Write-Output "KOPIA_SNAPSHOT_PATH=${mountPoint}${sourcePath}" diff --git a/mealie/readme.md b/mealie/readme.md new file mode 100644 index 0000000..3945f0e --- /dev/null +++ b/mealie/readme.md @@ -0,0 +1,230 @@ +a# Mealie in docker + +###### guide-by-example + +![logo](https://i.imgur.com/qDXwqaU.png) + +# Purpose & Overview + +Documentation and notes. + +* [Official site](https://www.bookstackapp.com/) +* [Github](https://github.com/BookStackApp/BookStack) +* [DockerHub](https://hub.docker.com/r/linuxserver/bookstack) + +BookStack is a modern, open source, good looking wiki platform +for storing and organizing information. + +Written in PHP, using Laravel framework, with MySQL database for the user data.
+There is no official Dockerhub image so the one maintained by +[linuxserver.io](https://www.linuxserver.io/) is used, +which uses nginx as a web server. + +# Files and directory structure + +``` +/home/ +└── ~/ + └── docker/ + └── bookstack/ + ├── 🗁 bookstack_data/ + ├── 🗁 bookstack_db_data/ + ├── 🗋 .env + ├── 🗋 docker-compose.yml + └── 🗋 bookstack-backup-script.sh +``` + +* `bookstack_data/` - a directory with bookstacks web app data +* `bookstack_db_data/` - a directory with database data +* `.env` - a file containing environment variables for docker compose +* `docker-compose.yml` - a docker compose file, telling docker how to run the containers +* `bookstack-backup-script.sh` - a backup script, to be run daily + +Only the files are required. The directories are created on the first run. + +# docker-compose + +Dockerhub linuxserver/bookstack +[example compose.](https://hub.docker.com/r/linuxserver/bookstack) + +`docker-compose.yml` +```yml +services: + + mealie: + image: hkotel/mealie + container_name: mealie + hostname: mealie + restart: unless-stopped + env_file: .env + volumes: + - ./mealie_data:/app/data + expose: + - 80:80 + +networks: + default: + name: $DOCKER_MY_NETWORK + external: true +``` + +`.env` +```bash +# GENERAL +DOCKER_MY_NETWORK=caddy_net +TZ=Europe/Bratislava + + +# MEALIE +PUID=1000 +PGID=1000 +RECIPE_PUBLIC=true +RECIPE_SHOW_NUTRITION=true +RECIPE_SHOW_ASSETS=true +RECIPE_LANDSCAPE_VIEW=true +RECIPE_DISABLE_COMMENTS=false +RECIPE_DISABLE_AMOUNT=false + +# USING SENDINBLUE FOR SENDING EMAILS +MAIL_DRIVER=smtp +MAIL_ENCRYPTION=tls +MAIL_HOST=smtp-relay.sendinblue.com +MAIL_PORT=587 +MAIL_FROM=book@example.com +MAIL_USERNAME= +MAIL_PASSWORD= +``` + +**All containers must be on the same network**.
+Which is named in the `.env` file.
+If one does not exist yet: `docker network create caddy_net` + +`APP_URL` in the `.env` **must be set** for bookstack to work.
+`MAIL_` stuff must be set for password reset and new registrations. + +# Reverse proxy + +Caddy v2 is used, details +[here](https://github.com/DoTheEvo/selfhosted-apps-docker/tree/master/caddy_v2).
+ +`Caddyfile` +```php +book.{$MY_DOMAIN} { + reverse_proxy bookstack:80 +} +``` + +# First run + +Default login: `admin@admin.com` // `password` + +--- + +![interface-pic](https://i.imgur.com/cN1GUZw.png) + +# Trouble shooting + +* It did not start.
+ Ctrl+f in `.env` file for word `example` to be replaced with actual domain + name. `APP_URL` has to be set correctly for bookstack to work. +* After update cant see edit tools.
+ Clear browsers cookies/cache. +* The test email button in preferences throws error.
+ Exec in to the container and `printenv` to see. + Check [mail.php](https://github.com/BookStackApp/BookStack/blob/development/app/Config/mail.php) + to see exact `MAIL_` env variables names and default values. + Test in Thunderbird your smtp server working or not. + +# Update + +Manual image update: + +- `docker-compose pull`
+- `docker-compose up -d`
+- `docker image prune` + +It is **strongly recommended** to now add current **tags** to the images in the compose.
+Tags will allow you to easily return to a working state if an update goes wrong. + +If there was a **major version jump**, and bookstack does not work, +exec in to the app container and run php artisan migrate
+`docker container exec -it bookstack /bin/bash`
+`cd /app/www`
+`php artisan migrate` + +# Backup and restore + +#### Backup + +Using [kopia](https://github.com/DoTheEvo/selfhosted-apps-docker/tree/master/kopia_backup) +or [borg](https://github.com/DoTheEvo/selfhosted-apps-docker/tree/master/borg_backup) +to make daily snapshot of the entire docker directory. + +#### Restore + +* down the containers `docker-compose down`
+* delete/move/rename the entire project directory
+* from the backups copy back the entire project directory
+* start the containers `docker-compose up -d` + +# Backup of just user data + +Users data daily export using the +[official procedure.](https://www.bookstackapp.com/docs/admin/backup-restore/)
+For bookstack it means database dump and backing up several directories +containing user uploaded files. + +Daily kopia/borg backup run takes care of backing up the directories. +So only database dump is needed and done with the script.
+The created backup sql file is overwritten on every run of the script, +but that's ok since kopia/borg are keeping daily snapshots. + +#### Backup script + +Placed inside `bookstack` directory on the host + +`bookstack-backup-script.sh` +```bash +#!/bin/bash + +# CREATE DATABASE DUMP, bash -c '...' IS USED OTHERWISE OUTPUT > WOULD TRY TO GO TO THE HOST +docker container exec bookstack-db bash -c 'mysqldump -u $MYSQL_USER -p$MYSQL_PASSWORD $MYSQL_DATABASE > $MYSQL_DIR/BACKUP.bookstack.database.sql' +``` + +The script must be **executable** - `chmod +x bookstack-backup-script.sh` + +#### Cronjob - scheduled backup + +Running on the host + +* `su` - switch to root +* `crontab -e` - add new cron job
+* `0 22 * * * /home/bastard/docker/bookstack/bookstack-backup-script.sh`
+ runs it every day [at 22:00](https://crontab.guru/#0_22_*_*_*) +* `crontab -l` - list cronjobs to check + +# Restore the user data + +Assuming clean start and latest images.
+Will need `BACKUP.bookstack.database.sql` and content of `bookstack_data/www/`
+Note that database restore must happen before bookstack app is first run. + +* start only the database container: `docker-compose up -d bookstack-db` +* copy `BACKUP.bookstack.database.sql` in `bookstack/bookstack_db_data/` +* restore the database inside the container
+ `docker container exec --workdir /config bookstack-db bash -c 'mysql -u $MYSQL_USER -p$MYSQL_PASSWORD $MYSQL_DATABASE < BACKUP.bookstack.database.sql'` +* now start the app container: `docker-compose up -d` +* let it run so it creates its file structure +* down the containers `docker-compose down` +* in `bookstack/bookstack_data/www/`
+ replace directories `files`,`images`,`uploads` and the file `.env`
+ with the ones from the BorgBackup repository +* start the containers: `docker-compose up -d` +* if there was a major version jump, exec in to the app container and run `php artisan migrate`
+ `docker container exec -it bookstack /bin/bash`
+ `cd /app/www`
+ `php artisan migrate` + +Again, the above steps are based on the +[official procedure](https://www.bookstackapp.com/docs/admin/backup-restore/) +at the time of writing this. diff --git a/minecraft/readme.md b/minecraft/readme.md index e487de6..1d8cbc3 100644 --- a/minecraft/readme.md +++ b/minecraft/readme.md @@ -276,7 +276,9 @@ switch to sata ssd seems to prevent any more occurancies of this high disk usage # Comamnds & settings * `/gamerule playersSleepingPercentage 1` - use bed whenever, sleep not dependant on other players - +* to import a map, download, extract, copy the directory to minecraft-data
+ `mv import normal`, there should be no spaces in the name +* `/mv setspawn` sets spawn point in the current world # Update diff --git a/prometheus_grafana/readme.md b/prometheus_grafana/readme.md index f353a42..f731dc9 100644 --- a/prometheus_grafana/readme.md +++ b/prometheus_grafana/readme.md @@ -32,6 +32,8 @@ Lot of the prometheus stuff here is based off the magnificent * **[Loki](#Loki)** - all of the above but for log files * **[Caddy monitoring](#Caddy_monitoring)** - monitoring a reverse proxy +![dashboards_pic](https://i.imgur.com/ac9Qj1F.png) + # Overview [Good youtube overview](https://youtu.be/h4Sl21AKiDg) of Prometheus.