systemd-nspawn
Related articles
systemd-nspawn is like the chroot command, but it is a chroot on steroids.
systemd-nspawn may be used to run a command or OS in a light-weight namespace container. It is more powerful than chroot since it fully virtualizes the file system hierarchy, as well as the process tree, the various IPC subsystems and the host and domain name.
systemd-nspawn limits access to various kernel interfaces in the container to read-only, such as /sys
, /proc/sys
or /sys/fs/selinux
. Network interfaces and the system clock may not be changed from within the container. Device nodes may not be created. The host system cannot be rebooted and kernel modules may not be loaded from within the container.
This mechanism differs from Lxc-systemd or Libvirt-lxc, as it is a much simpler tool to configure.
Installation
systemd-nspawn is part of and packaged with systemd.
Examples
Create and boot a minimal Arch Linux distribution in a container
First install arch-install-scripts.
Next, create a directory to hold the container. In this example we will use ~/MyContainer
.
Next, we use pacstrap to install a basic arch-system into the container. At minimum we need to install the base group.
# pacstrap -i -c -d ~/MyContainer base [additional pkgs/groups]
Once your installation is finished, boot into the container:
# systemd-nspawn -b -D ~/MyContainer -n
The -b
option will boot the container (i.e. run systemd
as PID=1), instead of just running a shell. -D
specifies the directory that becomes the container's root directory and -n
will set up a private network between host and container.
After the container starts, log in as "root" with no password.
The container can be powered off by running poweroff
from within the container. From the host, containers can be controlled by the machinectl tool.
Enable Container on boot
If you want to use a container frequently, you can have systemd start it on boot. First you have to enable the systemd
target machines.target
# systemctl enable machines.target
then you can
# mv ~/MyContainer /var/lib/machines/MyContainer # systemctl enable systemd-nspawn@MyContainer.service # systemctl start systemd-nspawn@MyContainer.service
Building and Testing packages
Management
machinectl
Managing your containers is essentially done with the machinectl
command. See machinectl(1)
for more detail then listed here.
Examples:
- Spawn a new shell inside a running container:
$ machinectl login MyContainer
- Show detailed information about a container:
$ machinectl status MyContainer
- Reboot a container:
$ machinectl reboot MyContainer
- Poweroff a container:
$ machinectl poweroff MyContainer
- Download an image:
# machinectl pull-tar URL name
systemd toolchain
Much of the core systemd toolchain has been updated to work with containers. Tools that do usually provide a -M, --machine=
option which will take a container name as argument.
Examples:
- See journal logs for a particular machine:
$ journalctl -M MyContainer
- Show control group contents:
$ systemd-cgls -M MyContainer
- See startup time of container:
$ systemd-analyze -M MyContainer
- For an overview of resource usage:
$ systemd-cgtop
Tips
X environment
See Xhost and Change root#Run graphical applications from chroot.
You will need to set the DISPLAY
environment variable inside your container session to connect to the external X server.
X stores some required files in the /tmp
directory. In order for your container to display anything, it needs access to those files. To do so, append the --bind=/tmp/.X11-unix:/tmp/.X11-unix
option when starting the container.
Run Firefox inside an nspawn container
See Firefox tweaks.
Networking
Note the canonical systemd-networkd host and container .network files are from https://github.com/systemd/systemd/tree/master/network
You need to set up the container .network manually after pacstrapping and # systemctl enable systemd-networkd
(your dhcp client) with systemd-nspawn's -n switch to ensure a virtual Ethernet link is setup. Don't forget to set up DNS, e.g. by either 1) edit your container's /etc/resolv.conf
by adding your DNS server's IP address, or have 2) systemd-resolved manage /etc/resolv.conf
for you.
See systemd-networkd#Usage with containers for more complex examples.
nsswitch.conf
To make it easier to connect to a container from the host, you can enable local DNS resolution for container names. In /etc/nsswitch.conf
, add mymachines
to the hosts:
section, e.g.
hosts: files mymachines dns myhostname
Then, any DNS lookup for hostname foo
on the host will first consult /etc/hosts
, then the names of local containers, then upstream DNS etc.
use host networking
To disable private networking used by containers started with machinectl start MyContainer
, edit the configuration of systemd-nspawn@.service
with systemctl edit systemd-nspawn@.service
and set the ExecStart=
option without the --network-veth
parameter unlike the original service:
/etc/systemd/system/systemd-nspawn@.service.d/override.conf
[Service] ExecStart= ExecStart=/usr/bin/systemd-nspawn --quiet --keep-unit --boot --link-journal=try-guest --machine=%I
The newly started containers will use the hosts networking.
Virtual Ethernet interfaces
If a container is started with systemd-nspawn ... -n
, systemd will automatically create one virtual Ethernet interface on the host, and one in the container, connected by a virtual Ethernet cable.
If the name of the container is foo
, the name of the virtual Ethernet interface on the host is ve-foo
. The name of the virtual Ethernet interface in the container is always host0
.
When examining the interfaces with ip link
, interface names will be shown with a suffix, such as ve-foo@if2
and host0@if9
. The @ifN
is not actually part of the name of the interface; instead, ip link
appends this information to indicate which "slot" the virtual Ethernet cable connects to on the other end.
For example, a host virtual Ethernet interface shown as ve-foo@if2
will connect to container foo
, and inside the container to the second network interface -- the one shown with index 2 when running ip link
inside the container. Similarly, in the container, the interface named host0@if9
will connect to the 9th slot on the host.
Running on a non-systemd system
See Init#systemd-nspawn.
Troubleshooting
root login fails
If you get the following error when you try to login (i.e. using machinectl login <name>
):
arch-nspawn login: root Login incorrect
And journalctl
shows:
pam_securetty(login:auth): access denied: tty 'pts/0' is not secureĀ !
Add pts/0
to the list of terminal names in /etc/securetty
on the container filesystem, see [1]. You can also opt to delete /etc/securetty
on the container to allow root to login to any tty, see [2].
unable to upgrade some packages on the container
It can sometimes be impossible to upgrade some packages on the container, filesystem being a perfect example. The issue is due to /sys
being mounted as Read Only. The workaround is to remount the directory in Read Write when running mount -o remount,rw -t sysfs sysfs /sys
, do the upgrade then reboot the container.