LetsEncrypt TLS NGINX Proxy Container

We will be using a prebuilt NGINX reverse proxy and certbot image, maintained by a community organization called linuxserver.io.
The main benefit of this image is that there is minimal configuration to setup TLS, and the TLS cert will also renew automatically every 60 days as long as you retrain ownership of your domain.

Since NGINX is bundled, we are able to also use this container to proxy all traffic to an arbitrary number of backing services through a technique utilizing URL rewrite. This will allow all members of your group to launch their own backing webapp, even if they all run on port 80 and 443. This works due to only the proxy listening on those ports; the actual backing webapps will not be mapped at all as their traffic will be routed through the proxy via URL rewrite.

This will be explained lower in the page during configuration of backing apps.

To start, we need to install podman machine and podman-plugins. The latter is CNI plugins for internal DNS by container name to function properly. Unlike docker, podman ships this in two parts and both are required.

dnf update -y
dnf install -y podman podman-plugins
podman pull docker.io/linuxserver/swag:2.8.0

podman network create container-network
podman volume create letsencrypt

Copy and paste the below herenow doc to create the service, which we will name “letsencrypt.service”.
Notice the %N specifier, this will use the name of what you name the unit file. In this case, it is “letsencrypt.service”

cat << EOF > /etc/systemd/system/letsencrypt.service
[Unit]
Description=CSCI 4830 Container (letsencrypt.service)
Requires=network-online.target
After=network-online.target

[Service]
TimeoutStartSec=0
Restart=always
ExecStartPre=-/usr/bin/podman stop %N
ExecStartPre=-/usr/bin/podman rm %N
ExecStart=/usr/bin/podman run -t --rm --name %N \\
                                 --network container-network \\
                                 --cap-add=NET_ADMIN \\
                                 --env PUID=1000 \\
                                 --env PGID=1000 \\
                                 --env TZ=America/Denver \\
                                 --env URL=<yourAssignedName>.cs.colorado.edu \\
                                 --env SUBDOMAINS= \\
                                 --env EXTRA_DOMAINS= \\
                                 --env VALIDATION=http \\
                                 --env EMAIL=chbu1234@colorado.edu \\
                                 --env DHLEVEL=2048 \\
                                 --env STAGING=true \\
                                 --publish 80:80 \\
                                 --publish 443:443 \\
                                 --volume %N:/config \\
                                 docker.io/linuxserver/swag:amd64-latest
ExecStop=/usr/bin/podman stop %N

[Install]
EOF

Modify this file to reflect your group’s assigned A record, (the –env URL) Once you are confident that it is working by running podman logs -f letsencrypt; stop the service to destroy the container. Change STAGING variable to false in order to obtain a production TLS certificate that will renew every 60 days. The usage of vim is recommended as a continued practice for this course throughout the semester. If you would like a refresher visit the Module 2 review videos on Canvas.

# Open the unit file to modify as instructed
vim /etc/systemd/system/letsencrypt.service

# Start the service
systemctl daemon-reload
systemctl start letsencrypt

# Tail the logs of this container to ensure it was successful
podman logs -f letsencrypt

# If successful, stop the service
systemctl stop letsencrypt

# Open the unit file and change the staging env to FALSE
# This will obtain a production certificate without counting towards your group's rate limit
vim /etc/systemd/system/letsencrypt.service

# Enable and start the service, the --now flag will also start the service
systemctl daemon-reload
systemctl enable --now letsencrypt

Your container should now be running, if you visit your site you will see a blank landing page that is using your new TLS certificate. Our next step is to create proxy configurations.

# Navigate to the podman volume on the host that is currently mapped into the container
# Notice there is a slight variation vs docker, in which /container/storage replaces /docker
cd /var/lib/containers/storage/volumes/letsencrypt/_data/

# Feel free to preview your TLS cert as well, the below file contains the private key, cert, and chain
# You can technically also copy this out to another app
cat /var/lib/containers/storage/volumes/letsencrypt/_data/keys/letsencrypt/priv-fullchain-bundle.pem

Example below is to create a simple configuration to forward all traffic to a specific backing “test-tripwire” container based on the URL. Note that this will not work until the actual backing application is running. Take special note at the proxy_pass line, this is where you are referencing another container via its podman dns name. In this case, it is proxying all traffic to a container called “devuni-pathfinder” on port 80. Notice as well that the backing container has no TLS; as the termination is occuring at the proxy container and not the backing container.

The backing containers can actually all be the same port due to them not being mapped to the host. Only the proxy is mapping its container ports 80 and 443, to the host ports, in this case also 80 and 443. In order to differentiate which backing container to run from, we will be using URL matching. That way everyone’s backing app can forward to just port 80 and 443, but with different URLs.

This is done with a technique with location matching (based on matching a string in a URL).
In order for the backing application to understand the new URL format that is its new base URL, we will use a rewrite rule.

You will need to note that whatever location URL’s your group defines below will completely overwrite your root webapp. For example, if it is wordpress, the URL will instead map to the backing application instead of a wordpress page. Keep this in mind so you do not create conflicting URL’s which may cause confusion.

Should you want a landing page for your group, use “/” as your location and remove the rewrite rule. In this example the root url will proxy to the wordpress container (https://dev-uni.cs.colorado.edu), and the /test-tripwire (https://dev-uni.cs.colorado.edu/test-tripwire) location will proxy to the tripwire container.

The first rewrite rule will append a trailing slash if a user does not include it, and the second rewrite rule will add /test-tripwire to the URL and also allowing the backing app (in this case “tripwire”) to make use of it as its baseurl. Working examples of your unit files will need to be submitted to Canvas; I will actually test them so make sure they actually work.

cat << EOF > /var/lib/containers/storage/volumes/letsencrypt/_data/nginx/proxy-confs/uni-example-proxy.conf
server {
    listen 443 ssl;
    listen [::]:443 ssl;

    server_name dev-uni.cs.colorado.edu;

    include /config/nginx/ssl.conf;

    client_max_body_size 0;

    location / {
        include /config/nginx/proxy.conf;
        include /config/nginx/resolver.conf;
        proxy_pass http://devuni-wordpress:80;
    }

    location /test-tripwire {
        include /config/nginx/proxy.conf;
        include /config/nginx/resolver.conf;
        proxy_pass http://tripwire-nginx:80;

        rewrite ^([^.]*[^/])$ $1/ permanent;
        rewrite ^/test-tripwire/?(.*)$ /$1 break;
    }
}
EOF