Today I’m setting up a simple nginx proxy, so I can store updates used by my many Linux systems. Most of them run a derivative of Debian, so this guide focuses mostly on caching apt repositories (Debian, Ubuntu, Proxmox, and more), but the same approach should work with any distro.

Install nginx⌗

I’m using a Debian 12 (Bookworm) unprivilaged LXC container, but this is basic nginx which should be in every distro ever. I pointed my local DNS address deb.palnet.net (using a DNS override in Unbound on OPNsense, in my case) to the IP of the container.

So:

#Ensure system is up to date
apt update
apt upgrade -y
#Install nginx
apt install nginx -y
#Set it to autostart
systemctl enable nginx
#Remove defaults in sites-enabled
cd /etc/nginx
rm sites-enabled/*

Now we’re ready to setup our cache!

Nginx Config⌗

Here’s my nginx config (/etc/nginx/sites-available/debcache.conf):

# Global Cache settings
proxy_cache_path /var/debcache/cache levels=2:2 keys_zone=generic:500m inactive=3650d max_size=1000g min_free=5g loader_files=1000 loader_sleep=50ms loader_threshold=300ms use_temp_path=off;
# Log with cache status
log_format cachelog '$remote_addr [$time_local] "$request" $status "$http_user_agent" "$upstream_cache_status"';

# URI paths to avoid cache
# These paths will change to indicate new release contents
# All other .deb files can be cached nearly indefinitely, as the
# version number is coded into the file name.
map $request_uri $nocache {
    ~InRelease 1;
    ~Release 1;
    ~Packages 1;
}

# Deb Server
server {
    # IF you need legacy IP, enable this one
    #listen 80 reuseport;

    # Everyone needs modern IP
    listen [::]:80 reuseport;


    #Log settings
    access_log /var/debcache/access.log cachelog;
    error_log /var/debcache/error.log;


    # Cache Location
    slice 1m;
    proxy_cache generic;
    proxy_ignore_headers Expires Cache-Control;
    proxy_cache_valid 200 206 3650d;
    proxy_cache_valid 301 302 0;
    proxy_set_header  Range $slice_range;
    proxy_cache_lock on;
    proxy_cache_lock_age 2m;
    proxy_cache_lock_timeout 1h;
    proxy_cache_use_stale error timeout invalid_header updating http_500 http_502 http_503 http_504;
    proxy_cache_revalidate on;
    #Nocache for those entries
    proxy_cache_bypass $nocache;
    proxy_no_cache $nocache;
    # 1G max file
    proxy_max_temp_file_size 1024m;
    # Cache key
    proxy_cache_key      $http_host$uri$slice_range;
    # Upstream Configuration
    proxy_next_upstream error timeout http_404;
    # Cache status
    add_header X-Cache-Status $upstream_cache_status;
    proxy_redirect off;
    proxy_ignore_client_abort on;
    # Upstream request headers
    proxy_ssl_server_name on;

    # Redirect Locations
    # Must include trailing slash!

    # Debian
    location /debian/ {
        proxy_pass http://deb.debian.org/debian/;
        proxy_set_header Host "deb.debian.org";
    }
    location /debsec/ {
        proxy_pass http://deb.debian.org/debian-security/;
        proxy_set_header Host "deb.debian.org";
    }

    # Ubuntu
    location /ubuntu/ {
        proxy_pass http://us.archive.ubuntu.com/ubuntu/;
        proxy_set_header Host "us.archive.ubuntu.com";
    }
    location /ubusec/ {
        proxy_pass http://security.ubuntu.com/ubuntu/;
        proxy_set_header Host "security.ubuntu.com";
    }

    # Kali
    location /kali/ {
        proxy_pass http://http.kali.org/kali/;
        proxy_set_header Host "http.kali.org";
    }

    # Proxmox (non-enterprise)
    location /proxmox/ {
        proxy_pass http://download.proxmox.com/debian/;
        proxy_set_header Host "download.proxmox.com";
    }

    # Caddy Server
    location /caddy/ {
        proxy_pass https://dl.cloudsmith.io/public/caddy/stable/deb/debian/;
        proxy_set_header Host "dl.cloudsmith.io";
    }

    # Nodesource NodeJS
    location /node/ {
        proxy_pass http://deb.nodesource.com/;
        proxy_set_header Host "deb.nodesource.com";
    }

    # Authelia
    location /authelia/ {
        proxy_pass https://apt.authelia.com/stable/debian/debian/;
        proxy_set_header Host "apt.authelia.com";
    }

    # Influx
    location /influx/ {
        proxy_pass http://repos.influxdata.com/debian/;
        proxy_set_header Host "repos.influxdata.com";
    }

    # Stats endpoint
    location = /nginx_status {
        stub_status;
    }

    # Static Files (conversion scripts)
    root /var/debcache/static/;
    autoindex on;
}

Once you are done creating the file, create a link to it in sites-enabled so it is loaded by nginx:

cd /etc/nginx
ln -s ../sites-available/debcache.conf sites-enabled/debcache.conf

And restart nginx (systemctl restart nginx)

Replacement Script⌗

I went through all of my Debian systems (to see which repos they are using), and wrote a simple script which uses sed to replace everything to point to my cache. I chose this approach instead of setting up apt to use a proxy, since this replaces https requests with http requests (to the cache - the cache -> upstream is still using HTTPS). This avoids complications in having apt trust a self-signed proxy cert. The lack of TLS is not significant here since deb packages are GPG signed.

Anyway, here’s the script (put it in /var/debcache/static/rewrite.sh)

#!/bin/bash

# My mirror location
export mirror=http://deb.palnet.net

# Load OS release info into variables
. /etc/os-release

#Function to rewrite a single file
rewrite() {
    echo Rewriting $1
    #Debian repos
    sed -i s#http://deb.debian.org/debian#$mirror/debian#g $1
    sed -i s#http://ftp.us.debian.org/debian#$mirror/debian#g $1
    sed -i s#http://security.debian.org#$mirror/debsec#g $1
    sed -i s#http://deb.debian.org/debian-security#$mirror/debsec#g $1
    #Ubuntu repos
    sed -i s#http://us.archive.ubuntu.com/ubuntu#$mirror/ubuntu#g $1
    sed -i s#http://security.ubuntu.com/ubuntu#$mirror/ubusec#g $1
    #Kali repos
    sed -i s#http://http.kali.org/kali#$mirror/kali#g $1
    #Proxmox repos (this catches all of them)
    sed -i s#http://download.proxmox.com/debian#$mirror/proxmox#g $1
    #Caddy server
    sed -i s#https://dl.cloudsmith.io/public/caddy/stable/deb/debian#$mirror/caddy#g $1
    #NodeJS from Nodesource
    sed -i s#https://deb.nodesource.com#$mirror/node#g $1
    #Authelia
    sed -i s#https://apt.authelia.com/stable/debian/debian#$mirror/authelia#g $1
    #Influxdata
    sed -i s#https://repos.influxdata.com/debian#$mirror/influx#g $1
}

#Rewrite sources.list itself
rewrite '/etc/apt/sources.list'

#Rewrite everything in sources.list.d
export -f rewrite
find '/etc/apt/sources.list.d/' -name "*.list" -type f -exec bash -c 'rewrite "$0"' {} \;

# If this system is using the 'new style' deb mirrors file (/etc/apt/sources.list.d/debian.sources)
# Replace it with equivalent sources.list entries
if test -f "/etc/apt/sources.list.d/debian.sources"; then
    echo "Rewriting /etc/apt/sources.list.d/debian.sources to sources.list format"
    echo "deb $mirror/debian $VERSION_CODENAME main contrib" >> /etc/apt/sources.list
    echo "deb-src $mirror/debian $VERSION_CODENAME main contrib" >> /etc/apt/sources.list
    echo "deb $mirror/debian $VERSION_CODENAME-updates main contrib" >> /etc/apt/sources.list
    echo "deb-src $mirror/debian $VERSION_CODENAME-updates main contrib" >> /etc/apt/sources.list
    echo "deb $mirror/debsec $VERSION_CODENAME-security main contrib" >> /etc/apt/sources.list
    echo "deb-src $mirror/debsec $VERSION_CODENAME-security main contrib" >> /etc/apt/sources.list
    rm /etc/apt/sources.list.d/debian.sources
fi

Then, on any system you have, you can run curl deb.palnet.net/rewrite.sh | bash and it will replace all of the deb references it finds. If you don’t have curl, you can do the same thing with wget -q -O - deb.palnet.net/rewrite.sh | bash (you may need to sudo bash depending on your current privilege)

Upgrade Distribution⌗

Now that I have this place to put deb upgrade scripts, I wrote another one to replace bookworm with trixie to help upgrade deb systems easily.

This one goes in /var/debcache/static/bookworm2trixie.sh

#!/bin/bash
#Only do sources.list
sed -i 's/bookworm/trixie/g' /etc/apt/sources.list

and you can again get to it at deb.palnet.net/bookworm2trixie.sh and pipe it into bash.

Monitoring⌗

You can access the logs of your cache from the debcache system, to see if you are getting hits or misses. The first system to update will miss everything, but after that the hit rate should be decent.

tail -f /var/debcache/access.log
#Or for the error file
tail -f /var/debcache/error.log