selfhosted-apps-docker/README.md

243 lines
9.4 KiB
Markdown
Raw Normal View History

2020-05-01 09:38:43 +00:00
# Selfhosted-Apps-Docker
2020-04-09 22:52:11 +00:00
2020-05-18 22:49:18 +00:00
###### guide-by-example
2020-04-09 22:52:11 +00:00
2020-05-01 09:51:20 +00:00
---
2020-05-01 09:38:43 +00:00
![logo](https://i.imgur.com/u5LH0jI.png)
2020-04-09 22:52:11 +00:00
2020-05-01 09:51:20 +00:00
---
2020-05-01 09:55:39 +00:00
* [caddy_v2](caddy_v2/) - reverse proxy
2023-02-12 18:45:01 +00:00
* [vaultwarden](vaultwarden/) - password manager
2020-05-01 09:40:53 +00:00
* [bookstack](bookstack/) - notes and documentation
* [borg_backup](borg_backup/) - backup utility
* [ddclient](ddclient/) - automatic DNS update
2020-05-10 21:48:51 +00:00
* [dnsmasq](dnsmasq/) - DNS and DHCP server
2023-02-05 18:26:14 +00:00
* [gotify / ntfy / signal](gotify-ntfy-signal/) - instant notifications apps
2020-05-01 09:40:53 +00:00
* [homer](homer/) - homepage
2023-02-05 18:26:14 +00:00
* [jellyfin](jellyfin/) - video and music streaming
* [kopia](kopia_backup/) - backup utility replacing borg
2022-08-29 15:13:53 +00:00
* [minecraft](minecraft/) - game server
2023-02-05 18:26:14 +00:00
* [meshcrentral](meshcrentral/) - web based remote desktop, like teamviewer or anydesk
* [rustdesk](rustdesk/) - remote desktop, like teamviewer or anydesk
2020-05-01 09:40:53 +00:00
* [nextcloud](nextcloud/) - file share & sync
2023-02-05 18:26:14 +00:00
* [opnsense](opnsense/) - a firewall, enterprise level
2022-09-13 07:02:53 +00:00
* [qbittorrent](qbittorrent/) - video and music streaming
2020-05-12 17:59:50 +00:00
* [portainer](portainer/) - docker management
2020-05-01 09:40:53 +00:00
* [prometheus_grafana](prometheus_grafana/) - monitoring
2023-02-05 18:26:14 +00:00
* [unifi](unifi/) - management utility for ubiquiti devices
* [snipeit](snipeit/) - IT inventory management
* [trueNAS scale](trueNASscale/) - network file sharing
2020-05-01 09:40:53 +00:00
* [watchtower](watchtower/) - automatic docker images update
2020-07-19 16:39:31 +00:00
* [wireguard](wireguard/) - the one and only VPN to ever consider
2023-02-05 18:26:14 +00:00
* [zammad](zammad/) - ticketing system
2020-05-10 21:48:51 +00:00
* [arch_linux_host_install](arch_linux_host_install)
2020-04-09 22:52:11 +00:00
2023-02-05 18:26:14 +00:00
Can also just check the directories listed at the top for work in progress
2023-01-22 10:24:43 +00:00
2022-11-19 23:06:37 +00:00
Check also [StarWhiz / docker_deployment_notes](https://github.com/StarWhiz/docker_deployment_notes/blob/master/README.md)<br>
2023-02-05 18:26:14 +00:00
Repo documents self hosted apps in similar format and also uses caddy for reverse proxy
2022-11-19 23:06:37 +00:00
2023-02-05 18:26:14 +00:00
# Core concepts
2020-06-24 19:00:04 +00:00
2023-02-05 18:26:14 +00:00
- `docker-compose.yml` do not need any editing to get started,
changes are to be done in the `.env` file.
2023-03-05 09:50:50 +00:00
- Not using `ports` directive if theres only web traffic for a container.<br>
2023-02-05 18:26:14 +00:00
Theres an expectation of running a reverse proxy which makes mapping ports
2023-03-05 09:50:50 +00:00
to a docker host unnecessary. Instead `expose` is used which is basically
2023-02-05 18:26:14 +00:00
just documentation.<br>
- For persistent storage bind mount `./whatever_data` is used.
No volumes, nor static path somewhere... just relative path next to compose file.
2023-02-12 18:45:01 +00:00
- no version is declared in compose, as the practice was
[deprecated](https://nickjanetakis.com/blog/docker-tip-51-which-docker-compose-api-version-should-you-use)
2020-06-24 19:02:49 +00:00
2023-03-05 10:25:09 +00:00
---
### Requirements
2020-05-22 16:05:03 +00:00
2023-02-05 18:26:14 +00:00
**Basic linux and basic docker-compose knowledge.**
The shit here is pretty hand holding and detailed, but it still should not be
your first time running a docker container.
2020-05-22 16:05:03 +00:00
2023-03-05 10:25:09 +00:00
---
2020-05-22 16:05:03 +00:00
2023-03-05 10:25:09 +00:00
### Caddy reverse proxy
2023-03-05 09:50:50 +00:00
2023-03-05 10:25:09 +00:00
Kinda the heart of the setup is [Caddy reverse proxy](https://github.com/DoTheEvo/selfhosted-apps-docker/tree/master/caddy_v2).</br>
It's described in most details and all guides have reverse proxy section
with Caddyfile config specific for them.</br>
Caddy is really great at simplifying the mess of https certificates, where
you don't really have to deal with anything, while having a one simple,
readable config file.
2023-02-05 18:26:14 +00:00
2023-03-05 10:25:09 +00:00
But no big issue if using [traefik](https://github.com/DoTheEvo/Traefik-v2-examples)
or nginx proxy manager. You just have to deal with reverse settings,
and 90% of the time its just sending traffic to port 80 and nothing else.
2023-03-05 09:50:50 +00:00
---
### Docker network
2023-03-05 10:25:09 +00:00
You really want to create a custom named docker network and use it.
2023-03-05 09:50:50 +00:00
`docker network create caddy_net`
It can be named whatever, but what it does over default is that it provides
[automatic DNS resolution](https://docs.docker.com/network/bridge/)
2023-03-05 10:25:09 +00:00
between containers. Meaning one can exec in to a container and ping another
container by its hostname.<br>
This makes config files simpler and cleaner.
2023-03-05 09:50:50 +00:00
---
2020-05-12 18:25:19 +00:00
2023-03-05 09:50:50 +00:00
### .env
2020-05-20 17:20:01 +00:00
2021-01-05 12:33:56 +00:00
Often the `.env` file is used as `env_file`,
which can be a bit difficult concept at a first glance.
2020-05-12 17:55:28 +00:00
2020-05-20 18:29:12 +00:00
`env_file: .env`
* `.env` - actual name of a file that is used only by compose.</br>
2020-05-12 17:59:50 +00:00
It is used automatically just by being in the directory
2020-05-12 17:55:28 +00:00
with the `docker-compose.yml`</br>
2023-02-05 18:26:14 +00:00
Variables in it are available during the building of a container,
2020-05-20 18:38:44 +00:00
but unless named in the `environment:` option, they are not available
2023-02-05 18:26:14 +00:00
once the container is running.
2020-05-13 20:23:40 +00:00
* `env_file` - an option in compose that defines an existing external file.</br>
Variables in this file will be available in the running container,
but not during building of the container.
2020-05-12 17:55:28 +00:00
2021-01-05 12:33:56 +00:00
So a compose file having `env_file: .env` mixes these two together.
2020-06-24 19:07:10 +00:00
2021-01-05 12:33:56 +00:00
Benefit is that you do not need to make changes at multiple places.
Adding variables or changing a name in `.env` does not require you
to also go in to compose to add/change it there... also the compose file
looks much cleaner, less cramped.
2020-05-12 17:55:28 +00:00
2020-06-24 19:12:31 +00:00
Only issue is that **all** variables from the `.env` file are available in
all containers that use this `env_file: .env` method.</br>
2023-02-05 18:26:14 +00:00
That can lead to potential issues if a container picks up environment
variable that is intended for a different container of the stack.
2020-06-24 19:12:31 +00:00
2020-06-24 19:18:47 +00:00
In the setups here it works and is tested, but if you start to use this
everywhere without understanding it, you can encounter issues.
2020-06-24 19:12:31 +00:00
So first troubleshooting step should be abandoning `.env` and write out
2021-01-05 12:33:56 +00:00
the variables directly in the compose file only under containers that want them.
2020-05-12 17:55:28 +00:00
2020-05-12 18:25:19 +00:00
---
2021-01-08 20:46:27 +00:00
### Docker images latest tag
2020-05-12 18:25:19 +00:00
2023-02-05 18:26:14 +00:00
Most of the time the images are without any tag,
which defaults to `latest` tag being used.</br>
2020-05-12 19:11:15 +00:00
This is [frowned upon](https://vsupalov.com/docker-latest-tag/),
2023-02-12 18:45:01 +00:00
and you should put there the current tags once things are going.
It will make updates easier when you know you can go back to a working version
with backups and knowing image version.<br>
2020-06-24 19:18:47 +00:00
2020-05-13 20:13:43 +00:00
---
2020-05-13 20:11:30 +00:00
### Cloudflare
2023-02-05 18:26:14 +00:00
For managing DNS records. The free tier provides lot of management options and
2021-01-08 20:46:27 +00:00
benefits. Like proxy between your domain and your server, so no one
2020-05-13 20:11:30 +00:00
can get your public IP just from your domain name. Or 5 firewall rules that allow
you to geoblock whole world except your country.
[How to move to cloudflare.](https://support.cloudflare.com/hc/en-us/articles/205195708-Changing-your-domain-nameservers-to-Cloudflare)
2020-07-19 16:39:31 +00:00
---
### ctop
[official site](https://github.com/bcicen/ctop)
![ctop-look](https://i.imgur.com/nGAd1MQ.png)
2023-02-05 18:26:14 +00:00
htop like utility for quick containers management.
2020-07-19 16:39:31 +00:00
It is absofuckinglutely amazing in how simple yet effective it is.
2023-02-05 18:26:14 +00:00
* hardware use overview, so you know which container uses how much cpu, ram, bandwidth, IO,...
2020-07-19 16:46:46 +00:00
* detailed info on a container, it's IP, published and exposed ports, when it was created,..
2023-02-05 18:26:14 +00:00
* quick management, quick exec in to a container, check logs, stop it,...
2020-07-19 16:39:31 +00:00
2023-03-05 09:50:50 +00:00
Written in Go, so its super fast and installation is trivial when it is a single binary.<br>
download `linux-amd64` version; make it executable with chmod +x; move it to `/usr/bin/`;
now you can ctop anywhere.
---
### Sendinblue
Services often need ability to send emails, for notification, registration,
password reset and such... Sendinblue is free, offers 300 mails a day
and is easy to setup.
```
EMAIL_HOST=smtp-relay.sendinblue.com
EMAIL_PORT=587
EMAIL_HOST_USER=whoever_example@gmail.com>
EMAIL_HOST_PASSWORD=xcmpwik-c31d9eykwewf2342df2fwfj04-FKLzpHgMjGqP23
EMAIL_USE_TLS=1
```
2021-01-08 20:46:27 +00:00
2023-02-05 18:26:14 +00:00
---
### Archlinux as a docker host
My go-to is archlinux as I know it the best.
Usually in a virtual machine with snapshots before updates.
For Arch installation I had [this notes](arch_linux_host_install/)
on how to install and what to do afterwards.<br>
But after [archinstall script](https://wiki.archlinux.org/title/archinstall)
started to be included with arch ISO I switched to that.<br>
For after the install setup I created
[Ansible-Arch repo](https://github.com/DoTheEvo/ansible-arch) that gets shit
done in few minutes without danger of forgetting something.<br>
Ansible is really easy to use and very easy to read and understand playbooks,
so it might be worth the time to check out the concept to setup own ansible scripts.
The best aspect of having such repo is that it is a dedicated place where
one can write solution to issues encountered,
2023-03-05 09:50:50 +00:00
or enable freshly discovered feature for all future deployments.
2023-02-05 18:26:14 +00:00
---
2023-03-05 09:50:50 +00:00
### Other guides
2023-02-05 18:26:14 +00:00
2023-03-05 09:50:50 +00:00
* [StarWhiz/docker_deployment_notes](https://github.com/StarWhiz/docker_deployment_notes)
- got inspired and wrote in similar way setup for various services
* [BaptisteBdn/docker-selfhosted-apps](https://github.com/BaptisteBdn/docker-selfhosted-apps)
- many services using traefik for reverse proxy
* [Awesome Docker Compose Examples](https://github.com/Haxxnet/Compose-Examples)
2023-02-05 18:26:14 +00:00
2023-03-05 09:50:50 +00:00
---
2023-02-05 18:26:14 +00:00
2023-03-05 09:50:50 +00:00
### For docker noobs
2021-01-08 20:46:27 +00:00
2023-03-05 09:50:50 +00:00
First, docker is easy. Like really.<br>
2021-01-08 20:47:08 +00:00
2023-03-05 09:50:50 +00:00
Second, there are two main uses.
2021-01-08 20:46:27 +00:00
2023-03-05 09:50:50 +00:00
* A developer who daily works on an app and docker eases everything about it, from testing to deployment
* A self-hosting / devops approach, where you just want to get a service running
This whole repo is obviously about the second use. So be careful that you wont
2023-03-05 10:28:20 +00:00
spend time on resources used to educate developers. I mean if you get through
that you will know docker better. But theres always the danger that after sinking
4 hours learning, one still cant get plain nginx web server up and working.
2023-03-05 09:50:50 +00:00
And my personal preference in learning is getting something up as fast as possible
and then tinker with it and try to understand it.
2023-03-05 10:25:09 +00:00
So to solve this, when googling for guides, look for **docker compose**
rather than just **docker** tutorials and notice if they are talking
fundamentals or deployment.
2023-03-05 09:50:50 +00:00
* [This](https://youtu.be/DM65_JyGxCo) one is pretty good. That entire channel
has good stuff worth looking.
2021-01-08 20:46:27 +00:00
2023-03-05 09:50:50 +00:00
Will add stuff as I encounter it.