I’ve played with network booting before (and I even tried to netboot Windows, what a nightmare that was), but now I want to get serious about it. I want to netboot my VDI Clients. I’ve been working on a thin client series, and the next step is to get rid of the installation entirely. I could use something like Linux Terminal Server Project, but that’s a bit overkill for this, and I wanted to learn Alpine anyway, so I’ve chosen to use Alpine Linux for the client operating system. It’s extremely basic. However, even using Alpine is still a huge setup, so this article will focus on setting up a netboot server (also using Alpine Linux), to prepare for the next video where I create the Proxmox VDI Client using this netboot system.

Contents⌗

Here’s a table of contents:

Video⌗

Click on the thumbnail to watch the video for this! Video Thumbnail

The Basics⌗

The boot process looks something like this:

  1. BIOS/UEFI on victim machine decides it should network boot, either because that’s the highest priority boot order or it can’t find any other options.
  2. BIOS/UEFI loads the PXE ROM from the network card and executes it.
  3. PXE code does a DHCP DISCOVER, finds the DHCP server, and receives a next-server. It may also receive one or more boot filenames for UEFI/BIOS.
  4. PXE code downloads the boot filename from the next-server address via TFTP and executes it. If filename is not provided, it first does a DHCP request to the address of next-server (instead of broadcast like normal) to get the filename. This method is called ‘Proxy DHCP’. We will be using that method today.
  5. We’ve compiled a special version of iPXE with an embedded script, and that is the boot code being executed at this point. iPXE is a ‘better’ PXE ROM. Here we have the option of hardcoding the boot commands into an embedded script. The script also has the option of requesting more commands over HTTP, so you can compile a single version of iPXE and use the HTTP server along with CGI/PHP/… to decide what boot commands a specific client needs, since the HTTP request can include things like the MAC address formatted in the URL. We will not be doing this today, but it’s a future enhancement.
  6. The script includes commands to download and load the kernel and initrd over HTTP instead of TFTP, so using iPXE saves us a ton of time over using TFTP and gives us more control over what arguments to pass to the kernel.
  7. For Alpine we can add a modloop argument which specifies the HTTP address to download the modloop file. If you provide it, that file will contain all of the additional kernel modules that may be needed after the early boot process.
  8. For Alpine, we can pass the URL of an Alpine mirror as a kernel command line argument, so the system has a working package manager by default. Normal systems would use the CD/USB IMG as the mirror until you configure networking, so this can get a fully functional working system without an image at all.
  9. For Alpine, we can pass the URL to an APKOVL file which will be applied on top of the initramfs root image. This usually includes /etc but can include more. It also includes the /etc/apk/world file, which is the list of packages that should be installed. Essentially, given an apkovl file, Alpine will rebuild the installation that was saved previously, using a combination of downloading and installing packages from the mirror configured in the /etc/apk/repositories file and the configuration and user data in the APKOVL file.
  10. Due to the modloop and apkovl, we have a completely running system using only 4 files on the server (vmlinuz, initramfs, modloop, apkovl) which will then configure itself exactly as we want. We can export the configuration of a running system using lbu to generate the apkovl file, and the other 3 files are provided by Alpine for netbooting. If we include /home in our apkovl, we can include all of the configuration we need for the thin client without having to mount anything off NFS.
  11. (Optional) If we are doing this in a big organization, we shouldn’t hammer the Alpine package servers every time a thin client boots, so you should setup your own local mirror. That’s beyond the scope of this tutorial. Alpine provides a public rsync server for mirrors to synchronize from, and you can configure your mirror to only mirror specific Alpine versions and architectures to save space and pass requests to the primary mirrors otherwise.

Netboot Server Setup⌗

For the server, we can choose whatever OS we want. But, I’m into Alpine now, so I’m going to use Alpine for the server as well. I’m installing it on Proxmox VE as a VM. I used the full extended ISO, but I don’t think that’s necessary. I ran setup-alpine to install it to disk using sys mode, which is the full system installed on disk and not running from RAM.

Alpine is pretty darn minimal. We need a text editor, it doesn’t come with nano.

#Nano is a text editor that's useful
apk add nano

Now that we have nano we can add the community repository (nano /etc/apk/repositories) and uncomment the line ending in community, but not the ones with edge in them.

And finally, enable root SSH login for now so we can setup the system (disable this once you are done): nano /etc/ssh/sshd_config, find the line starting with #PermitRootLogin and change the line to PermitRootLogin yes. Restart the server using rc-service sshd restart. From now on you can use ssh for the rest of the commands.

Setup HTTP Server⌗

Going to use nginx here, since we need PHP support in the future. You could also use lighttpd.

#Install it
apk add nginx
#Add www user for nginx to run as
adduser -D -g 'www' www
mkdir /srv/www
chown -R www:www /var/lib/nginx
chown -R www:www /srv/www
#Backup old nginx config since we will rewrite it
mv /etc/nginx/nginx.conf /etc/nginx/nginx.conf.bak

Now we need to create an nginx config as simple as possible, since the default one will 404 on everything. So nano /etc/nginx/nginx.conf and paste this in:

user www;
worker_processes auto;
error_log /var/log/nginx/error.log warn;
pid /var/run/nginx/nginx.pid;

events {
    worker_connections 1024;
}

http {
    include /etc/nginx/mime.types;
    default_type application/octet-stream;
    sendfile on;
    access_log /var/log/nginx/access.log;
    keepalive_timeout 3000;
    server {
        listen 80;
        root /srv/www;
        index index.html index.htm;
        server_name localhost;
        client_max_body_size 32m;
        error_page 500 502 503 504 /50x.html;
        location = /50x.html {
            root /var/lib/nginx/html;
        }
    }
}

And finally configure it to start on boot:

#Start on boot
rc-update add nginx
#Start now
rc-service nginx start

Download Alpine Netboot⌗

Alpine provides netboot images in a .tar.gz format. We need to put these in our /srv/www directory.

Go to the Alpine download page and copy the link to the netboot x86_64. As of the writing of this, the file is called alpine-netboot-3.15.4-x86_64.tar.gz.

Now download the file:

#Move to web directory
cd /srv/www
#Download alpine link
wget <paste the link here>
#Untar it
tar -xzf alpine*
#Delete the tar
rm alpine-netboot*
#Files are now in boot folder

There are two versions of the files, virt and lts. Virt is stripped down to only include the drivers needed to run on common hypervisors, while lts should include more driver support.

Setup TFTP Server⌗

We’re gonna need a TFTP server for the ipxe image, since PXE normally requires TFTP to be used. Let’s get that setup now.

#Install it
apk add tftp-hpa
#Add it to run as a service
rc-update add in.tftpd
#Start service now
rc-service in.tftpd start
#Default directory is /var/tftpboot/

Since tftp-hpa creates a chroot for itself, it can’t follow symlinks up the directory tree, so we will need to copy our files into /var/tftpboot/ instead of symlinking them. We will need to be aware of this when we install iPXE later.

Download & Configure iPXE⌗

Now that we have a functional server, we need to write our embedded ipxe script to download vmlinuz and friends over HTTP, then compile this into an ipxe binary and put it on our tftp server.

First we need to install git then clone the ipxe repo:

#Install git, make, gcc, perl, all the stuff ipxe will need
apk add git make binutils mtools perl xz-dev libc-dev gcc
#Going to put ipxe in the srv folder
cd /srv
#Clone ipxe
git clone https://github.com/ipxe/ipxe.git
#cd into it
cd ipxe/src

Now we can create an embedded script (nano netboot.ipxe):

#!ipxe

#Init networking
dhcp

#Networking info we got from the DHCP server
echo next-server is ${next-server}
echo filaneme is ${filename}
echo MAC address is ${net0/mac}
echo IP address is ${ip}

#Set flavor to lts
set flavor lts
echo flavor is ${flavor}

#Set command line 
set cmdline modules=loop,squashfs quiet
echo cmdline is ${cmdline}

#Server address
set server http://${next-server}
echo server is ${server}

#Kernel file
set vmlinuz ${server}/boot/vmlinuz-${flavor}
echo vmlinuz is ${vmlinuz}
set initramfs ${server}/boot/initramfs-${flavor}
echo initramfs is ${initramfs}

#Modloop file
set modloop ${server}/boot/modloop-${flavor}
echo modloop is ${modloop}

#Repository for apk
#Update this if you'd like a newer version of Alpine
#Alternatively, set branch to edge for the absolutel latest
set mirror http://dl-cdn.alpinelinux.org/alpine
set branch v3.15
set repo ${mirror}/${branch}/main
echo repo is ${repo}

#apkovl file - set this if you want to apply
#an apkovl file to configure the Alpne instance
set apkovl ${server}/thinclient.apkovl.tar.gz
echo apkovl is ${apkovl}

#Uncomment this if you want to see the information before continuing
#prompt Press any key to continue

#Kernel, initrd
#For EFI, we need to tell the kernel the initrd filename. For BIOS it doens't hurt to leave the initrd argument.
#If you want to use Alpine bare, use this line:
#kernel ${vmlinuz} ${cmdline} alpine_repo=${repo} modloop=${modloop} initrd=initramfs-${flavor}
#If you want to use Alpine with an apkovl, use this line:
kernel ${vmlinuz} ${cmdline} modloop=${modloop} apkovl=${apkovl} initrd=initramfs-${flavor}
initrd ${initramfs}

#Boot
boot

#Pause if errors
prompt Some error occurred, press any key to continue

Then we build ipxe (make a script nano build.sh then chmod +x build.sh to automate this):

#Build BIOS version (x86 but should boot into x64 environment)
make bin-i386-pcbios/undionly.kpxe EMBED=netboot.ipxe
#Build EFI version (x86)
make bin-x86_64-efi/ipxe.efi EMBED=netboot.ipxe
#Copy files to tftp root
cp bin-i386-pcbios/undionly.kpxe /var/tftpboot/
cp bin-x86_64-efi/ipxe.efi /var/tftpboot/ipxe64.efi
#The APKOVL we are using is for x64, so not building ipxe32.efi right now
#Also not building arm variants for this project

After this, we should have the binaries in our tftp directory and can enable netbooting in our DHCP server

GCC 12 Resolution⌗

Error-checking in GCC 12 results in many compile errors for iPXE, as referenced by this issue. Until the issue is resolved on their end, use this script instead to treat errors as warnings:

#Build BIOS version (x86 but should boot into x64 environment)
NO_WERROR=1 make bin-i386-pcbios/undionly.kpxe EMBED=netboot.ipxe
#Build EFI version (x86)
NO_WERROR=1 make bin-x86_64-efi/ipxe.efi EMBED=netboot.ipxe
#Copy files to tftp root
cp bin-i386-pcbios/undionly.kpxe /var/tftpboot/
cp bin-x86_64-efi/ipxe.efi /var/tftpboot/ipxe64.efi
#The APKOVL we are using is for x64, so not building ipxe32.efi right now
#Also not building arm variants for this project

Configure Netboot in OPNsense⌗

I use OPNsense as my firewall, so here’s how I added the next-server and filenames to OPNsense’s DHCP server.

Testing⌗

If all goes well, you should be able to boot your client, and as long as PXE is enabled higher than any other bootable devices in the boot order, it should launch into our thin client. You can even configure a VM in Proxmox with no disk and no DVD drive and it should netboot.