I consider myself a fairly skilled Linux user but I'm also a human being, so one late evening while upgrading my server I inadvertently ended up wiping it entirely.
That's when I decided that I'll never waste my time installing another server, and instead build something maintenance-free that lets me focus on what matters.
Lightwhale
Ready, Boot, Docker!
Lightwhale is a free, single-purpose operating system streamlined to effortlessly run Docker containers on bare-metal servers.
No installation or configuration is required, Lightwhale boots directly into an operational Docker Engine and SSH server, and all necessary software is immediately available.
The operating system is immutable, which enhances security and inherently makes it maintenance-free.
Custom configuration and data is stored on a separate device and is therefore never entangled with the core system files. This ensures absolute data segregation, and allow for seamless backup, restore, and provisioning.
The versatility allows using it for a minimum-headache home server with focus on the content, or an Edge server to take back control of sensitive data, or even move out of the cloud completely and run an on‑premises cluster.
The simple design of Lightwhale provides complete control over both system and data, minimal cognitive burden, and makes it incredibly easy to get started.
Getting Started
The concise expert's guide is next, followed by the more explanatory beginner's guide.
TL;DR
- 1. Download the latest x86 image
-
curl -JOL https://lightwhale.asklandd.dk/download/lightwhale-2.1.4-x86.iso
- 2. Write the image to a USB flash drive
-
lsblk
sudo umount /dev/sdx?
sudo dd bs=4M conv=fsync status=progress \ if=lightwhale-2.1.4-x86.iso of=/dev/sdx
- 3. Boot Lightwhale off the USB flash drive
- 4. Enable persistence (optional)
-
Write the magic header to the storage device to be used as persistence device. Note that all data will be erased from this device when Lightwhale claims it for persistence.
lsblk
echo "lightwhale-please-format-me" |\ sudo dd conv=notrunc of=/dev/sdy
Now, reboot to have Lightwhale automatically claim the persistence device.
- 5. Setup WiFi (optional)
-
sudo setup-wifi --ssid="my wifi" --password="my secret" --country=DK
sudo /etc/init.d/S40network start
- 6. Run a docker container
-
This requires an internet connection to pull the image, either using ethernet cable or wifi.
docker run hello-world
- 1. Install
-
Install QEMU on your workstation, or host:
sudo pacman -S qemu-system-x86 # On Arch/Endavour
sudo apt-get install qemu-system-x86 # On Debian/Ubuntu
- 2. Boot
-
Boot Lightwhale in QEMU, setup port-forwarding from the host to QEMU port
10022
and10080
that serve Lightwhale's SSH and (eventually) HTTP services respectively:qemu-system-x86_64 \ -enable-kvm \ -m 2G \ -drive file=lightwhale-2.1.4-x86.iso,format=raw,if=virtio \ -device virtio-rng-pci \ -net nic,model=virtio \ -net user,hostfwd=tcp::10022-:22,hostfwd=tcp::10080-:80
- 1. Insert
-
Insert the USB flash drive and identify what device it was assigned, e.g.
/dev/sdx
.lsblk
- 2. Write
-
First unmount the device for safety, then write the image to it:
sudo umount /dev/sdx?
sudo dd bs=4M conv=fsync status=progress \ if=lightwhale-2.1.4-x86.iso of=/dev/sdx
- 3. Boot
-
Boot the computer off the USB flash drive, adjust computer boot order as necessary.
- 1. Generate SSH key pair on host machine
-
Chances are these already exist, but otherwise run:
ssh-keygen -t ed25519 -b 4096 \ -C "This Key belongs to $USER@$HOSTNAME"
- 2. Copy SSH key from host machine to Lightwhale
-
ssh-copy-id -p 10022 op@localhost
This will prompt for a password for the login.
If
ssh
warns aboutremote host identification changed
, it is most likely because Lightwhale was rebooted and its SSH server auto-generated a new key during startup, which the host machine does not know about. Simply delete the old identification entry on the host in~/.ssh/known_hosts
and then retryssh-copy-id
:ssh-keygen -R [localhost]:10022
- 3. Log into Lightwhale using host SSH key
-
This should not (and never again) prompt for password:
ssh -p 10022 op@localhost
- 4. Reconfigure Lightwhale SSH service to disable password logins
-
Now, on Lightwhale, modify
/etc/ssh/sshd_config
to disable SSH password logins by addingPasswordAuthentication no
to the configuration.sudo sed '/^#PasswordAuthentication/a\ PasswordAuthentication no # Disable, ie. only allow pubkey to auth.' \ -i.orig /etc/ssh/sshd_config
nohup sudo /etc/init.d/S50sshd restart
When restarting
sshd
for the changes to take effect, notice how this cuts the branch you're sitting on ;) - 1. Create some simple content to serve
-
echo "Hello, World! This is Lightwhale!" | \ sudo tee /var/www/index.html
- 2. Start the web server container
-
d run -d -p 80:80 -v /var/www:/public \ --restart unless-stopped iorivur/darkhttpd
In the event that Docker fails to lookup the registry, simply retry the command. For some reason, this happens regularly within QEMU.
- 3. Query the web server
-
Either do the query locally, or like here from the host machine against Lightwhale in QEMU:
curl -v http://localhost:10080/
- Zero installation
-
Since the immutable rootfs cannot have anything added on it, all necessary software must be baked into it with sensible defaults from the start, primarily docker and sshd. This results in a fully self-contained image that can simply be written to a boot media and then live-boot, similar to a video game cartridge in principle.
Gone are the tedious and error-prone installation sittings that involve deciding on a partition scheme, formatting file systems, selecting and install software, the bootloader too, and finally rebooting into a system that still requires configuration before it can be used. And that is just the installation…
- Zero maintenance
-
With a fully functioning rootfs, there is no need to install any extra software, nor is there any need to update what already works.
No more package managers, package dependencies, or race of staying up to date. The dreadful operation of reinstalling is effectively accomplished by a simple reboot.
Docker, SSH, and the Linux kernel are the most critical moving part of Lightwhale. And in all fairness, they could eventually need updating. But it is only very rarely required. When that happens, just write a new Lightwhale image on the boot device.
- Reduced file system attack surface
-
The rootfs is inherintly resilient to both unintentional and malicious modification. One cannot accidentally delete something, nor can a virus modify any files here.
In contrast, on a traditional system all files are exposed including every little part of the underlying operating system, e.g
/bin/sh
,/lib/libc.so.6
, and even/usr/bin/[
.As an extreme but interesting example, this command harmless to Lightwhale:
[ -f /etc/lightwhale-release ] && sudo rm -frv /bin /lib /usr /boot
- No junk
-
A common issue with long-living installations is the accumulation of unused files. Junk take up disk space, pollute backups, and may even slow down the system, but is fortunately impossible on this rootfs.
- It's just a copy
-
Remember that the entire system is just a static image written to a boot device. That device and system holds no information itself. So if the boot device breaks, toss it and write a new one.
- Reduced power usage
-
A sustainable system and architecture starts with a sustainable platform. And Lightwhale has taken measures to be more environmentally friendly.
First if all, it only runs a bare minimum of system services, which reduces CPU load and thereby power consumption.
Futhermore, once the rootfs has been loaded into memory, the boot device is never accessed again until next reboot. This greatly reduces wear and tear of the physical boot device, especially because no writes ever occur. But it also allows the boot device to spin down, enter low-power mode, even power off, or be completely removed. If many Lightwhale instances are network-booted, it may lower power consumption even further because no boot media is required in any of these servers.
- Free to experiment
- Don't hold back, feel free to explore and experiment, have fun, you can't break it.
- 1. Scan for magic header
-
Scan all available block devices for a magic header, specifically this sequence of bytes at the very start of the device:
lightwhale-please-format-me
If found, it is consider a magic device.
- 2. Create persistence partition
-
If a magic device was found, it will be partitioned, formatted with ext4, and labeled
lightwhale-data
.This becomes the actual persistence partition.
- 3. Mount persistence partition or fallback tmpfs
-
Scan for the persistence partition by its label:
lightwhale‑data
.If found, mount it read-write under
/mnt/lightwhale‑data
.If not found, employ the fallback tmpfs-writability strategy and mount a tmpfs instead. Note that the tmpfs is mounted at the same mount point:
/mnt/lightwhale‑data
.At this point,
/mnt/lightwhale‑data
is a read-writable directory intended to essentially contain all customizations. If persistence is active, any files saved here will be retained during reboot. - 4. Create overlay file system
-
If the persistence partition (or tmpfs) was mounted on
/mnt/lightwhale‑data
, then merge it into an overlay file system, and thereby ensure persistence (or tmpfs writability) of selected directories. /etc/
- All custom system configuration.
The persisted
/etc
is overlaid early in the boot process for custom configuration and init scripts to take effect. /home/
- All user home directories.
/var/www/
- Standard web root. Not strictly necessary, but convenient for running an ad-hoc web server.
/var/lib/docker/
-
All Docker runtime files including downloaded images, containers and their state, configured networks, and managed volumes.
There is a technically that prevents this directory to be overlaid like the others, because Docker requires it for an internal overlay itself. So instead, Lightwhale automatically reconfigures Docker to use
/mnt/lightwhale-data/docker
for its data root. - 1. Create a persistence device file
-
Create an empty file for the virtual persistence device:
dd if=/dev/zero of=persistence.img bs=1M count=700
- 2. Write magic header
-
Write magic header to persistence device file so it will automatically be claimed for persistence at startup:
echo "lightwhale-please-format-me" |\ dd conv=notrunc of=persistence.img
- 3. Start QEMU
-
Start QEMU with the virtual persistence device file attached:
qemu-system-x86_64 \ -enable-kvm \ -m 2G \ -drive file=persistence.img,index=0,format=raw,media=disk \ -drive file=lightwhale-2.1.4-x86.iso,index=2,format=raw,media=cdrom -boot d \ -device virtio-rng-pci \ -net nic,model=virtio \ -net user,hostfwd=tcp::10022-:22,hostfwd=tcp::10080-:80
During startup, notice how Lightwhale claims the device and creates the
lightwhale-data
persistence partition, and overlays the essential directories. - 4. Copy SSH key from host machine to Lightwhale
-
Copy SSH user key from host to Lightwhale in QEMU and enter password, after removing any old offending SSH key:
ssh-keygen -R [localhost]:10022
ssh-copy-id -p 10022 op@localhost
- 5. Log into Lightwhale using host SSH key
-
This should not prompt for password:
ssh -p 10022 op@localhost
- 6. Start a Web Server
-
Start a web server and test it locally:
echo "Hello, World! Lightwhale has persistence enabled!" | \ sudo tee /var/www/index.html
d run -d --restart unless-stopped -p 80:80 -v /var/www:/public iorivur/darkhttpd
curl -v http://localhost
- 7. Reboot
-
sudo reboot
Now, normally this would completely reset Lightwhale, and the SSH server host keys, the authorized SSH user key, and the web server would be erased from memory. But we have persistence enabled…
- 8. Test Web Server
-
Query the web server from the host:
curl -v http://localhost:10080/
This returns the
index.html
, which confirms that both data and container were persistend. - 9. Test SSH
-
Connect with SSH from the host:
ssh -p 10022 op@localhost
This logs in with out password prompt, which confirms that the SSH server host keys and user SSH key were persisted.
- How do I login on Lightwhale?
-
Please refer to the guide.
- Wait, what? Does Lightwhale save data or not?
-
By default, Lightwhale only saves changes to memory, which means you will lose all your work at shutdown. To preserve configuration, containers, and data across reboots and have Lightwhale restart as you intend, you must first enable persistence. This is a very simple process that requires a dedicated persistence device.
- How can I connect Lightwhale to wifi?
-
Please refer to the guide.
- What hardware is supported?
-
Only x86-64, both BIOS and EFI.
- Can I run Lightwhale on Raspberry Pi?
-
No, not currently. It's on the backlog.
- Can I run Lightwhale on Macintosh M1 or M2?
-
No.
- Can I run Lightwhale in the cloud/Proxmox/Ventoy/etc?
-
Probably. Lightwhale is just Linux, so everything is possible. However, Lightwhale is intended to run as the main OS on your own, real, physical computer, and that is currently what this guide covers.
- Does Lightwhale come with a firewall/anti-virus?
-
No, nothing out of the box. Always take adequate security measures. Don't put servers directly on the internet, but run them behind a firewall. Change the default password, and disable password login.
Lightwhale does come with
iptables
so it is possible to manually configure firewall rules. - How am I covered if Lightwhale crashes, breaks something, makes me loose money/customers/reputation/hair/sleep/etc?
-
Think, take responsibility of your own actions, use at your own risk. There's no warrenty.
- Should I use Lightwhale at home, school, or work?
-
Yes.
- How do I install software on Lightwhale?
-
Only containers can be installed. Installing software directly on the file system is not possbile and would defeat the very purpose of Lightwhale.
Remember that containers don't necessarily have to be services, though;
git
,wget
, etc. can also be containerized, effectively allowing these types of apps to be installed too. - How do I update Lightwhale?
-
The underlying concept is that you don't.
This is based on the premise that critically necessary updates of the OS are in fact extremely rare. Lightwhale primarily consists of Docker, sshd, and Linux, running off an immutable rootfs using containerized execution. This significantly reduces the likelihood of requiring such updates.
That said, updated Lightwhale image will occur, either due to a critical bug-fix, must-have features, or simple convenience. In that case, you have two options:
- First and foremost: You can choose not to update! If the update doesn't include anything important to you, just leave your system running as it is.
-
Or, if you must: Simply download the image, write it to a boot device, and reboot into an updated Lightwhale.
Remember to read the CHANGELOG first, as it may contain important information.
A running Lightwhale instance can conveniently be updated in-place from itself. Because the boot device is not in use at this point, simply use the x86 permalink to download the most recently released x86 image, and directly overwrite the old version on the boot device, e.g.
/dev/sdx
, and reboot into the updated Lightwhale system:curl -JL https://lightwhale.asklandd.dk/download/latest-x86 | \ sudo dd bs=4M conv=fsync of=/dev/sdx
- What's the TL;DR on how to run Lightwhale off a USB flash drive with persistence enabled?
-
Fair enough. Assuming your USB flash drive for the boot device is
/dev/sdx
, and your harddisk for the persistence device is/dev/sdy
:curl -sL https://lightwhale.asklandd.dk/download/latest-x86 | \ sudo dd bs=4M conv=fsync of=/dev/sdx
echo "lightwhale-please-format-me" | \ sudo dd conv=notrunc of=/dev/sdy
- Can you please add
wget
,nano
,$my_fav_app_omg_i_love_it
to the rootfs? -
No, not likely.
I understand that it's disappointing not to have all the tools at your disposal, that you're accustomed to on mainstream Linux distros. Maybe you're used to
nano
, and now you feel forced to learnvi
. But remember, this is a minimalistic purpose-specific server OS, and with that comes some compromises and limitations; you get one editor, one http client, and other preselected essentials. Everything needed is there, but perhaps not in the shapes and sizes you prefer. If every person's preferred apps were added, Lightwhale would quickly turn into Bloatwhale, and that is not the goal of this project.
Beginner's Guide
The fastest way to get started is to simply boot Lightwhale off a USB flash drive on the actual server hardware.
The easiest way to experiment with Lightwhale is to use a virtual machine like QEMU on your workstation. Because this gives faster turnaround when rebooting while testing e.g. immutability and persistence.
This guide uses QEMU and provides Linux commands that can be copied and pasted directly into the shell for convenience. Alternative instructions are available elsewhere.
Download Lightwhale
Start by downloading a Lightwhale image from the download page. For x86 machines and QEMU, just get the latest x86 image:
curl -JOL https://lightwhale.asklandd.dk/download/lightwhale-2.1.4-x86.iso
Booting into QEMU
Booting from USB Flash Drive
To instead boot on real hardware, write the Lightwhale image file to a USB flash drive and boot it. Almost any drive will suffice since Lightwhale is small, and once booted, no further reads or writes will occur with that device. Note that, as always, any existing data on the devices will be permanently lost when writing the image.
Congratulations!
Now your Lightwhale server is up and ready for use.
Using Lightwhale
The default Lightwhale account is op
with password opsecret
.
After logging in, it is Docker business as usual;
docker
and docker compose
are
conveniently aliased by d
and d-c
respectively
— for your immersive Docker-centric pleasure.
Note that at this point Lightwhale is immutable, allowing safe experimentation; nothing is stored permanently, and rebooting will completely restore the system to factory settings.
Following are some recommended first steps and simple use-cases.
Again, the commands are intended to be copy and pasted into a Linux shell on the host, targeting a Lightwhale server running in QEMU, as described.
Log in using SSH
Using QEMU port-forwarding, connect to the host port 10022 which forwards to Lightwhale port 22 where SSH is served:
ssh -p 10022 op@localhost
For security reasons, it is important to change the op
password
before connecting Lightwhale to the internet,
because by now
everyone
knows your password.
Or even better…
Disable SSH Password Login
To harden security and increase convenience, disable SSH password logins and only allow authentication using SSH keys.
Configure console keymap and font
To configure the Linux console for the keyboard and display the characters of your language,
edit /etc/default/console
.
Most keymaps and fonts are built-in and supported by Lightwhale, e.g.:
cat <<EOF | sudo tee /etc/default/console
KEYMAP=dk-latin1
FONT=lat1-16
EOF
Or do it from the boot loader prompt:
lightwhale keymap=dk-latin1 font=lat1-16
Connect to a WiFi Network
Lightwhale automatically attempts to connect to wired LAN using DHCP, but wifi requires manual setup.
Use setup-wifi
to connect to a wifi network given its SSID and password, e.g.:
sudo setup-wifi --ssid="my wifi" --password="my secret" --country=DK
Alternatively, do it from the boot loader prompt, or press F1
for detailed help:
lightwhale wifi="my wifi;my secret;DK"
Start a Web Server
A nice way to take the Docker Engine of Lightwhale for a quick spin, is to start a container with a web server and request its contents.
Reboot
This concludes our tour.
Now would be a good time to reboot your system and experience firsthand how the immutable system reverts all changes and starts up fully restored — factory reset, if you will.
Read on to learn why Lightwhale is immutable, how it works, and later how to enable persistence.
Immutability by Design
Ah, yes! This is what distinguishes Lightwhale from other server operating systems.
A server without the ability to any install software or retain data across reboots is useless. This is why in practice, one always combines the two core principles of Lightwhale: immutability and persistence. They are quite contradicting concepts, but work wonders when combined properly — like ☯.
Advantages of Immutability
Immutability is implemented by using a read-only root file system, or rootfs. And this alone instantly brings a number of advantages into the system:
Booting into Immutability
When Lightwhale is booted off the boot device, first the Linux kernel loaded and started, then the rootfs is loaded and mounted into memory, and everything continues to initialize and eventually run off here. The rootfs is a squashfs which is compressed to save memory, and files are decompresses at run-time when read and cached.
Fallback tmpfs Writability Strategy
The immutable rootfs clearly has some advantages, but in order to be useful some means of writability is required. Because without it, the SSH server has nowhere to save its auto-generated host keys, network configuration and logs cannot be saved, and no containers can be installed.
If no persistence is active, Lightwhale employs its fallback tmpfs-writability strategy that uses tmpfs to add an in‑memory overlay on top of the immutable rootfs. This provides the writable file system, that allows the system to function as expected.
The overlaying of tmpfs works exactly like the overlaying of persistence, and is infact dynamically interchangeable during startup and will be explained in detail in the next chapter.
Lightwhale can now seamlessly both read and write, but with the tmpfs living in memory, all changes will unfortunately be lost at reboot. This is where Lightwhale persistence comes into play.
Persistence
Lightwhale handles persistence in its own special way. In order for custom configuration, containers, and data to be retained across reboots, a physical persistence device is required. The persistence device cannot be the same as the boot device, rather it must be a separate and dedicated device that Lightwhale has full disposal of.
In general, persistence should always be enabled to make Lightwhale useful.
The main reason behind persistence not being enforced, is because Lightwhale, being a headless zero-installation live-bootable server OS, has no natural way to pause during startup and prompt the user to prepare the persistence device, as traditional installers do. Additionally, it provides a convenient way to try out Lightwhale before commiting to using a persistence device.
When persistence is active, it completely sets aside the fallback tmpfs-writablity strategy.
Lightwhale fully automates persistence device detection, partitioning, formatting, mounting, and making sure only data and modified configuration is persisted.
The only manual step required is to actually enable persistence.
Enable Persistence
Simply write the magic header to the desired storage device
to be used as persistence device.
Writing the magic header is easy, and can be done from Lightwhale itself
once the device has been identified, e.g. /dev/sdx
:
lsblk
echo "lightwhale-please-format-me" |\
sudo dd conv=notrunc of=/dev/sdx
Note that we're writing to the root device itself, and not a partition on it, e.g. /dev/sdx1
).
Also note that the naming scheme for NVME devices differs from block devices,
e.g. the NVME root device is named /dev/nvme0n1
and partitions are named e.g. nvme0n1p1
.
Now reboot, and Lightwhale will automatically claim the device for persistence.
Claim Persistence
Lightwhale attempts to claims persistence early during system startup,
and this happens fully automatically without any user interaction.
It is implemented through a series of steps in /lib/lightwhale/setup-persistence
.
If no persistence devices is mounted, Lightwhale instead uses fallback tmpfs for writability in step 3.
Overlay File System
Lightwhale adds persistence (or tmpfs) on a per-directory basis by mounting an overlay file system on top of the rootfs. The overlay virtually merges layers of existing directories, while keeping their individual content separate.
Effectively, the overlay mirrors the original immutable rootfs, while modifications, new files, and even deletions made to it, will override the original content, but be kept isolated on the persistence file system (or tmpfs).
Selected Per‑Directory Persistence
Persistence (or tmpfs) does not cover the entire rootfs. Instead it is limited to the following essential directories only:
Directories excluded
Directories like /bin
, /usr
, and /lib
are deliberately not persisted.
Installing software directly on the rootfs
is considered "abuse" of the system
because it would defeat the very purpose
of immutability and Lightwhale altogether.
One should add software though Docker containers,
which can easily include system tools like an updated
git
client.
Absolute Data Segregation
The strict separation of the system on the boot device and custom configuration, containers, and data on the persistence device, virtually merged by an overlay, gives Lightwhale its amazing perk of absolute data segregation.
It also ensure that it is safe to rewrite Lightwhale or replace the boot device anytime.
A more classic approach would be to mount,
for example /etc
,
on a different disk to separate config from system.
However, remember that this requires a fully populated directory structure to be available
on the that mount point, including files that were never changed from the original system. Lightwhale only saves the modifications.
Testing Persistence in QEMU
This is easy, interesting, heck, you might even find it fun!
FAQ
Frequently asked questions, or future question that someone could be asking a some point.