Sharing the Traefik Proxy ​

Use case ​

Brewblox uses a Traefik gateway to let multiple services be publicly accessible on the same host port.

This works fine, but if you decide you want a secondary Traefik container to do the same for other non-Brewblox services on your computer, you run into problems.

Traefik service discovery is an issue, and port sharing is (again) an issue.

Requirements ​

• System includes an active and accessible Brewblox system.
• System includes one or more non-Brewblox containers.
• Both Brewblox and non-Brewblox containers are accessible on port 80/443.
• Top-level routing is based on hostname.
• Brewblox has a dedicated hostname.
• Hostname routing logic is extensible: 2..N hostnames must be recognized as routing rules.

Brewblox routing ​

We'll start with the routing rules required for Brewblox, and then extend to include other hosts.

Referenced compose configuration should be implemented in docker-compose.yml . It will override the default configuration in docker-compose.shared.yml .

Traefik service configuration ​

If you check brewblox/docker-compose.shared.yml , you'll find the traefik service configuration:

  traefik:
    image: traefik:2.10
    restart: unless-stopped
    labels:
      - traefik.http.routers.api.rule=PathPrefix(`/api`) || PathPrefix(`/dashboard`)
      - traefik.http.routers.api.service=api@internal
      - traefik.http.middlewares.prefix-strip.stripprefixregex.regex=/[^/]+
      - traefik.http.middlewares.auth.forwardauth.address=http://auth:5000/auth/verify
      - traefik.http.middlewares.cors.headers.AccessControlAllowMethods=CONNECT,HEAD,GET,DELETE,OPTIONS,PATCH,POST,PUT,TRACE
      - traefik.http.middlewares.cors.headers.accessControlAllowOriginListRegex=.*
      - traefik.http.middlewares.cors.headers.AccessControlAllowCredentials=true
      - traefik.http.middlewares.cors.headers.AccessControlAllowHeaders=Origin,X-Requested-With,Content-Type,Accept
    volumes:
      - type: bind
        source: ./traefik
        target: /config
        read_only: true
      - type: bind
        source: /var/run/docker.sock
        target: /var/run/docker.sock
      - type: bind
        source: /etc/localtime
        target: /etc/localtime
        read_only: true
    ports:
      - "${BREWBLOX_PORT_HTTP}:${BREWBLOX_PORT_HTTP}"
      - "${BREWBLOX_PORT_HTTPS}:${BREWBLOX_PORT_HTTPS}"
      - "${BREWBLOX_PORT_MQTT}:${BREWBLOX_PORT_MQTT}"
      - "${BREWBLOX_PORT_MQTTS}:${BREWBLOX_PORT_MQTTS}"
      - "127.0.0.1:${BREWBLOX_PORT_ADMIN}:${BREWBLOX_PORT_ADMIN}"
    environment:
      - TRAEFIK_API_DASHBOARD=true
      - TRAEFIK_PROVIDERS_DOCKER=true
      - TRAEFIK_PROVIDERS_DOCKER_CONSTRAINTS=LabelRegex(`com.docker.compose.project`, `${COMPOSE_PROJECT_NAME}`)
      - TRAEFIK_PROVIDERS_DOCKER_DEFAULTRULE=PathPrefix(`/{{ index .Labels "com.docker.compose.service" }}`)
      - TRAEFIK_PROVIDERS_FILE_DIRECTORY=/config
      - TRAEFIK_ENTRYPOINTS_WEBSECURE_ADDRESS=:${BREWBLOX_PORT_HTTPS}
      - TRAEFIK_ENTRYPOINTS_WEBSECURE_HTTP_TLS=true
      - TRAEFIK_ENTRYPOINTS_WEBSECURE_HTTP_MIDDLEWARES=cors,auth
      - TRAEFIK_ENTRYPOINTS_WEB_ADDRESS=:${BREWBLOX_PORT_HTTP}
      - TRAEFIK_ENTRYPOINTS_WEB_HTTP_REDIRECTIONS_ENTRYPOINT_TO=websecure
      - TRAEFIK_ENTRYPOINTS_ADMIN_ADDRESS=:${BREWBLOX_PORT_ADMIN}
      - TRAEFIK_ENTRYPOINTS_ADMIN_HTTP_MIDDLEWARES=cors
      - TRAEFIK_ENTRYPOINTS_MQTT_ADDRESS=:${BREWBLOX_PORT_MQTT}/tcp
      - TRAEFIK_ENTRYPOINTS_MQTTS_ADDRESS=:${BREWBLOX_PORT_MQTTS}/tcp

This is a big blob of configuration at once, so we'll go through it section by section.

    labels:
      - traefik.http.routers.api.rule=PathPrefix(`/api`) || PathPrefix(`/dashboard`)
      - traefik.http.routers.api.service=api@internal
      - traefik.http.middlewares.prefix-strip.stripprefixregex.regex=/[^/]+
      - traefik.http.middlewares.auth.forwardauth.address=http://auth:5000/auth/verify
      - traefik.http.middlewares.cors.headers.AccessControlAllowMethods=CONNECT,HEAD,GET,DELETE,OPTIONS,PATCH,POST,PUT,TRACE
      - traefik.http.middlewares.cors.headers.AccessControlAllowOriginListRegex=.*
      - traefik.http.middlewares.cors.headers.AccessControlAllowCredentials=true
      - traefik.http.middlewares.cors.headers.AccessControlAllowHeaders=Origin,X-Requested-With,Content-Type,Accept

The two labels starting with traefik.http.routers.api are for the Traefik dashboard, hosted at <ADDRESS>/dashboard/ .

traefik.http.middlewares.prefix-strip.stripprefixregex.regex=/[^/]+ is a reusable middleware for routing a public path with a prefix to a private path without a prefix.

traefik.http.middlewares.cors.(...) headers contain access control and Cross-Origin Resource Sharing (CORS) configuration. For Brewblox this is set very permissive, as the origin (server address) is different for every installation.

    volumes:
      - type: bind
        source: ./traefik
        target: /config
        read_only: true
      - type: bind
        source: /var/run/docker.sock
        target: /var/run/docker.sock
      - type: bind
        source: /etc/localtime
        target: /etc/localtime
        read_only: true

The used SSL certs are placed in ./traefik , along with a traefik-cert.yaml configuration file.

/var/run/docker.sock:/var/run/docker.sock allows access to the Docker socket. This is required in order to autodetect active Docker containers.

/etc/localtime is mounted to make sure the container uses the same time and timezone settings as the host.

    ports:
      - "${BREWBLOX_PORT_HTTP}:${BREWBLOX_PORT_HTTP}"
      - "${BREWBLOX_PORT_HTTPS}:${BREWBLOX_PORT_HTTPS}"
      - "${BREWBLOX_PORT_MQTT}:${BREWBLOX_PORT_MQTT}"
      - "${BREWBLOX_PORT_MQTTS}:${BREWBLOX_PORT_MQTTS}"
      - "127.0.0.1:${BREWBLOX_PORT_ADMIN}:${BREWBLOX_PORT_ADMIN}"

The BREWBLOX_PORT_XXXX variables are defined in the brewblox/.env file. The default values are set during installation.

The admin port is special: it is a non-authenticated HTTP port that is only accessible from the server itself. This port is used by brewblox-ctl during installation and updates.

There are quite a few arguments in the environment: section. We'll look at it a few lines at a time.

      - TRAEFIK_API_DASHBOARD=true

This enables the Traefik dashboard at /dashboard/ (the trailing / is required).

      - TRAEFIK_PROVIDERS_DOCKER=true
      - TRAEFIK_PROVIDERS_DOCKER_CONSTRAINTS=LabelRegex(`com.docker.compose.project`, `${COMPOSE_PROJECT_NAME}`)
      - TRAEFIK_PROVIDERS_DOCKER_DEFAULTRULE=PathPrefix(`/{{ index .Labels "com.docker.compose.service" }}`)

TRAEFIK_PROVIDERS_DOCKER enables Traefik scanning the Docker socket for active containers.

To avoid trying to route to any and all containers on the host, we add constraints. Docker-compose sets the com.docker.compose.project label on managed containers. The value equals that of the COMPOSE_PROJECT_NAME that is set in the brewblox/.env file.

The default routing rule for Brewblox services is to use the service name as prefix. eg. <ADDRESS>/spark-one/blocks should be routed to the spark-one service. We can get service name from another container label set by docker-compose: com.docker.compose.service .

      - TRAEFIK_PROVIDERS_FILE_DIRECTORY=/config

/config is a mounted volume that leads to brewblox/traefik . There's a configuration file and SSL certificates in there.

      - TRAEFIK_ENTRYPOINTS_WEBSECURE_ADDRESS=:${BREWBLOX_PORT_HTTPS}
      - TRAEFIK_ENTRYPOINTS_WEBSECURE_HTTP_TLS=true
      - TRAEFIK_ENTRYPOINTS_WEBSECURE_HTTP_MIDDLEWARES=cors,auth
      - TRAEFIK_ENTRYPOINTS_WEB_ADDRESS=:${BREWBLOX_PORT_HTTP}
      - TRAEFIK_ENTRYPOINTS_WEB_HTTP_REDIRECTIONS_ENTRYPOINT_TO=websecure
      - TRAEFIK_ENTRYPOINTS_ADMIN_ADDRESS=:${BREWBLOX_PORT_ADMIN}
      - TRAEFIK_ENTRYPOINTS_ADMIN_HTTP_MIDDLEWARES=cors
      - TRAEFIK_ENTRYPOINTS_MQTT_ADDRESS=:${BREWBLOX_PORT_MQTT}/tcp
      - TRAEFIK_ENTRYPOINTS_MQTTS_ADDRESS=:${BREWBLOX_PORT_MQTTS}/tcp

There are five entrypoints:

• websecure
• web
• admin
• mqtt
• mqtts

websecure is the primary user-facing entrypoint. HTTPS and authentication are both enabled.

web is a HTTP entrypoint that immediately redirects requests to websecure . Without it, you would get an error when visiting http://{address} .

admin is a HTTP entrypoint used by brewblox-ctl . To keep it safe, it is only accessible from the server itself (ie. bound to 127.0.0.1 in ports: ).

mqtt and mqtts are directly forwarded to the MQTT eventbus.

Traefik dashboard ​

You can check the Traefik dashboard by navigating to <ADDRESS>/dashboard/ . The trailing / is required.

Here you'll see all Routers, Services, and Middlewares that are currently active.

For a detailed explanation of how these interact, you can check this Traefik documentation page .

Other Brewblox services ​

Traefik allows for a number of shortcuts:

• A Traefik Router is automatically added if you declare a Traefik Service.
• You only need to specify a target port on your service if multiple ports are exposed.

The traefik service includes a default setting for the routing rule.

If you check docker-compose.shared.yml , you'll find:

  eventbus:
    labels:
      # MQTT
      - traefik.tcp.routers.mqtt.entrypoints=mqtt
      - traefik.tcp.routers.mqtt.rule=HostSNI(`*`)
      - traefik.tcp.routers.mqtt.tls=false
      - traefik.tcp.routers.mqtt.service=mqtt
      - traefik.tcp.services.mqtt.loadBalancer.server.port=1883
      # MQTTS with TLS termination by traefik
      - traefik.tcp.routers.mqtts.entrypoints=mqtts
      - traefik.tcp.routers.mqtts.rule=HostSNI(`*`)
      - traefik.tcp.routers.mqtts.tls=true
      - traefik.tcp.routers.mqtts.service=mqtts
      - traefik.tcp.services.mqtts.loadBalancer.server.port=1884
      # MQTT over websockets
      - traefik.http.services.eventbus.loadbalancer.server.port=15675
  victoria:
    labels:
      - traefik.http.services.victoria.loadbalancer.server.port=8428
  redis:
    labels:
      - traefik.enable=false
    labels:
      - traefik.http.routers.ui.rule=PathPrefix(`/ui`) || PathPrefix(`/static`) || Path(`/`)

Notes:

• eventbus and victoria have multiple published ports, so we need to be specific.
• redis is not accessible through the gateway.
• ui has a custom routing rule: if you go to <Address>:<Port> , you will be routed to the UI service.

So much for the default settings. Time for change!

Adding DNS hostnames ​

Let's add another service: webby . We want to be routed to webby if we navigate to webby.local .

First, we want the webby.local address to point to our host. A permanent solution is to add an entry in the router DNS records, but for now we can use avahi-publish to temporarily add DNS-SD records.

Open two terminal windows, and run (one command in each):

avahi-publish -a -R brewblox.local $(hostname -I | cut -d' ' -f1)

avahi-publish -a -R webby.local $(hostname -I | cut -d' ' -f1)

Leave the terminal windows open. The records disappear when you close the running process.

Now, if we navigate to either https://webby.local or https://brewblox.local , we end up in the Brewblox UI.

Host-based routing ​

Now, add the actual webby service to docker-compose.yml. We'll keep it simple, and use the default Nginx image.

We want webby.local to go to webby , and brewblox.local to go to the Brewblox UI.

  webby:
    image: nginx
    labels:
      - traefik.http.routers.webby.rule=Host(`webby.local`)
      - traefik.http.routers.webby.priority=9001

Now, if you go to brewblox.local , you'll end up at https://brewblox.local/ui/dashboard/dashboard-home , and if you go to webby.local , you get the Nginx welcome message.

Multi-project routing ​

If you don't want webby to be defined in brewblox/docker-compose.yml, you will need to set up a shared Docker network so the Traefik router can reach webby .

See the docker-compose documentation for how to set up custom networks.

You will also need to change or remove the provider constraint for the traefik service.

Currently, it is:

  traefik:
    environment: