Docker Swarm: Difference between revisions

From Wildsong
Jump to navigationJump to search
Brian Wilson (talk | contribs)
Created page with "Docker Swarm is an orchestrator and so is Kubernetes. Kubernetes is breathing down my neck too and today I am thinking, "What the hell, go for it! Why NOT run a single node with Kubernetes?" So jump off to that page for the latest on my explorations. I was going to use Dart as a worker but I decided I did not want to pay the electric bill to spin all those drives up anymore, so I sold Dart and now it lives in Newberg. I am going to use Tern instead. Shoul..."
 
Brian Wilson (talk | contribs)
mNo edit summary
Line 1: Line 1:
Docker Swarm is an orchestrator and so is Kubernetes.  
Docker Swarm is an orchestrator and so is Kubernetes.  


[[Kubernetes]] is breathing down my neck too and today I am thinking, "What the hell, go for it! Why NOT run a single node with Kubernetes?" So jump off to that page for the latest on my explorations.
[[Kubernetes]] is breathing down my neck too and today I am thinking, "What the hell, go for it! Why NOT run a single node with Kubernetes?"
I spent an hour looking at it and it just adds more complexity.


I was going to use [[Dart]] as a worker but I decided I did not want to pay the electric bill to spin all those drives up anymore, so
Some day I will spin up some IoT and Edge nodes for buzzword compliance but for now it's just one node. [[Bellman]]
I sold Dart and now it lives in Newberg. I am going to use [[Tern]] instead. Should be fine for testing.


  bellman> '''docker swarm init'''
Note, it made me pick an ethernet address.
 
  bellman> '''docker swarm init --advertise-addr 192.168.123.2'''
  Swarm initialized: current node (isk0jocx0rb37yonoafstyvoj) is now a manager.
  Swarm initialized: current node (isk0jocx0rb37yonoafstyvoj) is now a manager.
   
   
Line 15: Line 17:
  To add a manager to this swarm, run 'docker swarm join-token manager' and follow the instructions.
  To add a manager to this swarm, run 'docker swarm join-token manager' and follow the instructions.


  tern> docker swarm join --token SWMTKN-1-5b81dywl9xkis6769fxnsvjahfy361w2kxkz69nc35bz3nxt6s-43jxeopl6inw8xur1vpcl23w7 192.168.123.2:2377
I can add a node on another machine using that token. I won't be doing this today. It would look like this.
 
  '''tern>''' docker swarm join --token SWMTKN-1-5b81dywl9xkis6769fxnsvjahfy361w2kxkz69nc35bz3nxt6s-43jxeopl6inw8xur1vpcl23w7 192.168.123.2:2377
  This node joined a swarm as a worker.
  This node joined a swarm as a worker.


Line 23: Line 27:
  vjbx2h8n8280ecib2btzkwcxw    tern                Ready              Active                                  18.09.1
  vjbx2h8n8280ecib2btzkwcxw    tern                Ready              Active                                  18.09.1


  bellman> '''docker network create -d overlay --attachable proxy_net'''
  bellman> '''docker network create -d overlay --attachable testing'''
shaboxhgakqer14j1ve7zyysj
 
The "attachable" option is for containers not yet running in swarm. I will need that soon,
when I run bash in Debian for tests.
 
Now ordinarily I'd use Docker Compose to start a reverse proxy (my favorite today is Varnish).
 
Before worrying about Varnish though I will spin up simple web server on just the one existing node.
If I add tern it will spread them over the two nodes. I want it to use that "testing" network.
 
docker service create --name web --replicas 4 -p 80:80 --network testing --detach nginx:latest


The "attachable" option is for containers not yet running in swarm.
Now I have 4 copies of nginx running. I can see that they were published on port 80 but that's inside the funny swarm network,
how to see them? They are on localhost, I can do this "curl http://localhost". I can get the id (or just use the name "web")
and then kill them off,


Now ordinarily I'd use Docker Compose to start a proxy running consisting of two components.
docker service ls
One is the nginx reverse proxy and one is the letsencrypt docker. Here are the rules
docker service rm pmbrvm6wow7q
to start them swarm style using Docker Stack.
curl http://localhost


How about before trying all that I spin up simple web server? This works, and the proxy works normally too.
When I do the "curl" with the nginx replicas shut down, I see the page served by Varnish (still running in Compose), it's the Home Assistant instance.
It creates 4 replicas.
So I guess the swarm takes precedence over whatever is running in Compose.  


docker service create --name web --replicas=4 \
I skipped the "--network testing" parameter and everything still worked. I think maybe that's just so I can attach more services later?? Like this.
    -p 80:80 -e NETWORK_ACCESS=internal \
This looks the way I expected.
    -e VIRTUAL_HOST=solr.wildsong.biz -e VIRTUAL_PORT=80 \
    -e LETSENCRYPT_HOST=solr.wildsong.biz -e LETSENCRYPT_MAIL=brian@wildsong.biz \
    nginx:latest


This did not work, it's just something I tried. Possibly it's the volume settings?
docker run -it --rm --network testing debian:bullseye bash
How do volumes work in a swarm?
# apt update
# apt install -y bind9-dnsutils
# nslookup web
Server:        127.0.0.11
Address:        127.0.0.11#53
Non-authoritative answer:
Name:  web
Address: 10.0.1.27
# apt install -y curl
# curl http://10.0.1.27
# curl http://web/
 
The standard nginx page is returned from curl both times, so I know it's hitting a replica and running under the name "web", which is what I assigned.
Inside the container I can see my local LAN too, for example from the Debian instance I can "curl http://bellman.wildsong.biz:8123/" and get the Home Assistant page.
So far, easy easy.
 
== That healthcheck thing ==
 
It's not too soon to think about it. ;-)
 
With nginx I can create my own Docker image and build the healthcheck right into the image.
 
Yes I know "curl" is not the answer. https://blog.sixeyed.com/docker-healthchecks-why-not-to-use-curl-or-iwr/
but for now it's what I am using! In my Dockerfile,
 
FROM nginx:latest
HEALTHCHECK CMD curl --fail http://localhost || exit 1
 
$ docker build -t wildsong/nginx .
$ docker service rm web
$ docker service create --name web --replicas 1 -p 80:80 --network testing --detach wildsong/nginx
$ docker ps | grep web
01a3f36f7580  wildsong/nginx:latest                                      "/docker-entrypoint.…"  About a minute ago  Up About a minute (healthy)  80/tcp  web.1.ssssfmwmwp8je7g1dnsabevew
 
When I create the service, I will get a warning because I have not pushed that image (wildsong/nginx) to a registry, but it still works
because I am running only one node for now. When I do the "docker ps" I can see that the container is marked as "healthy".
 
LUNCH BREAK NOW.
 
=== How do volumes work in a swarm? ===


  docker service create --name proxy \
  docker service create --name proxy \
Line 53: Line 107:
   -v proxy_certs:/etc/nginx/certs:ro \
   -v proxy_certs:/etc/nginx/certs:ro \
   jwilder/nginx-proxy:alpine
   jwilder/nginx-proxy:alpine
== Bring in Compose ==
By that I mean I want to deploy a stack of containers using a docker-compose.yml file as the configuration.
So far I have not needed it, if I start just one container per project then "docker service" commands are fine.

Revision as of 19:44, 21 April 2023

Docker Swarm is an orchestrator and so is Kubernetes.

Kubernetes is breathing down my neck too and today I am thinking, "What the hell, go for it! Why NOT run a single node with Kubernetes?" I spent an hour looking at it and it just adds more complexity.

Some day I will spin up some IoT and Edge nodes for buzzword compliance but for now it's just one node. Bellman

Note, it made me pick an ethernet address.

bellman> docker swarm init --advertise-addr 192.168.123.2
Swarm initialized: current node (isk0jocx0rb37yonoafstyvoj) is now a manager.

To add a worker to this swarm, run the following command:

   docker swarm join --token SWMTKN-1-5b81dywl9xkis6769fxnsvjahfy361w2kxkz69nc35bz3nxt6s-43jxeopl6inw8xur1vpcl23w7 192.168.123.2:2377

To add a manager to this swarm, run 'docker swarm join-token manager' and follow the instructions.

I can add a node on another machine using that token. I won't be doing this today. It would look like this.

tern> docker swarm join --token SWMTKN-1-5b81dywl9xkis6769fxnsvjahfy361w2kxkz69nc35bz3nxt6s-43jxeopl6inw8xur1vpcl23w7 192.168.123.2:2377
This node joined a swarm as a worker.
bellman> docker node ls
ID                            HOSTNAME            STATUS              AVAILABILITY        MANAGER STATUS      ENGINE  VERSION
isk0jocx0rb37yonoafstyvoj *   bellman             Ready               Active              Leader              19.03.5
vjbx2h8n8280ecib2btzkwcxw     tern                Ready               Active                                  18.09.1
bellman> docker network create -d overlay --attachable testing
shaboxhgakqer14j1ve7zyysj

The "attachable" option is for containers not yet running in swarm. I will need that soon, when I run bash in Debian for tests.

Now ordinarily I'd use Docker Compose to start a reverse proxy (my favorite today is Varnish).

Before worrying about Varnish though I will spin up simple web server on just the one existing node. If I add tern it will spread them over the two nodes. I want it to use that "testing" network.

docker service create --name web --replicas 4 -p 80:80 --network testing --detach nginx:latest

Now I have 4 copies of nginx running. I can see that they were published on port 80 but that's inside the funny swarm network, how to see them? They are on localhost, I can do this "curl http://localhost". I can get the id (or just use the name "web") and then kill them off,

docker service ls
docker service rm pmbrvm6wow7q
curl http://localhost

When I do the "curl" with the nginx replicas shut down, I see the page served by Varnish (still running in Compose), it's the Home Assistant instance. So I guess the swarm takes precedence over whatever is running in Compose.

I skipped the "--network testing" parameter and everything still worked. I think maybe that's just so I can attach more services later?? Like this. This looks the way I expected.

docker run -it --rm --network testing debian:bullseye bash
# apt update
# apt install -y bind9-dnsutils
# nslookup web
Server:         127.0.0.11
Address:        127.0.0.11#53

Non-authoritative answer:
Name:   web
Address: 10.0.1.27
# apt install -y curl
# curl http://10.0.1.27
# curl http://web/

The standard nginx page is returned from curl both times, so I know it's hitting a replica and running under the name "web", which is what I assigned. Inside the container I can see my local LAN too, for example from the Debian instance I can "curl http://bellman.wildsong.biz:8123/" and get the Home Assistant page. So far, easy easy.

That healthcheck thing

It's not too soon to think about it. ;-)

With nginx I can create my own Docker image and build the healthcheck right into the image.

Yes I know "curl" is not the answer. https://blog.sixeyed.com/docker-healthchecks-why-not-to-use-curl-or-iwr/ but for now it's what I am using! In my Dockerfile,

FROM nginx:latest
HEALTHCHECK CMD curl --fail http://localhost || exit 1
$ docker build -t wildsong/nginx .
$ docker service rm web
$ docker service create --name web --replicas 1 -p 80:80 --network testing --detach wildsong/nginx
$ docker ps | grep web
01a3f36f7580   wildsong/nginx:latest                                      "/docker-entrypoint.…"   About a minute ago   Up About a minute (healthy)   80/tcp   web.1.ssssfmwmwp8je7g1dnsabevew

When I create the service, I will get a warning because I have not pushed that image (wildsong/nginx) to a registry, but it still works because I am running only one node for now. When I do the "docker ps" I can see that the container is marked as "healthy".

LUNCH BREAK NOW.

How do volumes work in a swarm?

docker service create --name proxy \
 -p 80:80 -p 443:443 \
 -e DHPARAM_GENERATION="false" \
 -v /var/run/docker.sock:/tmp/docker.sock:ro \
 -v ./network_internal.conf:/etc/nginx/network_internal.conf \
 -v ./vhost.d:/etc/nginx/vhost.d \
 -v proxy_html:/usr/share/nginx/html \
 -v proxy_dhparam:/etc/nginx/dhparam \
 -v proxy_certs:/etc/nginx/certs:ro \
 jwilder/nginx-proxy:alpine

Bring in Compose

By that I mean I want to deploy a stack of containers using a docker-compose.yml file as the configuration. So far I have not needed it, if I start just one container per project then "docker service" commands are fine.