This article will describe installing LXD/LXC and running containers.
Table of Contents
1 Install packages for LXD/LXC and ZFS
Install packages for LXD/LXC and ZFS with apt.
$ sudo apt install -y lxd lxd-client lxd-tools criu $ sudo apt install -y zfsutils-linux $ sudo modprobe zfs # or reboot machine
A criu package is for live migration. ZFS is used for containers store. ZFS should be used when there is a block device/volume for ZFS. If not using ZFS, a directory on filesystem is used.
2 Initialize LXD
Initialize LXD and lxd-bridge with interactive mode. "Address to bind LXD to" means which NIC is listened by LXD daemon.
$ sudo lxd init Name of the storage backend to use (dir or zfs): zfs Create a new ZFS pool (yes/no)? yes Name of the new ZFS pool: tank Would you like to use an existing block device (yes/no)? no Size in GB of the new loop device (1GB minimum): 20 Would you like LXD to be available over the network (yes/no)? yes Address to bind LXD to (not including port): 0.0.0.0 Port to bind LXD to (8443 recommended): 8443 Trust password for new clients: Again: Do you want to configure the LXD bridge (yes/no)? yes Warning: Stopping lxd.service, but it can still be activated by: lxd.socket LXD has been successfully configured.
Using –auto option can initialize LXD without interactive mode.
$ sudo lxd init --auto \ --storage-backend zfs \ --storage-pool tank \ --storage-create-loop 20 \ --network-address 0.0.0.0 \ --network-port 8443 \ --trust-password password
A case of dir is as below.
$sudo lxd init --auto \ --storage-backend dir \ --network-address 0.0.0.0 \ --network-port 8443 \ --trust-password password
lxd-bridge can be initialized without interactive mode.
$ sudo systemctl stop lxd-bridge $ sudo systemctl --system daemon-reload $ sudo su -c 'cat <<EOF > /etc/default/lxd-bridge USE_LXD_BRIDGE="true" LXD_BRIDGE="lxdbr0" UPDATE_PROFILE="true" LXD_CONFILE="" LXD_DOMAIN="lxd" LXD_IPV4_ADDR="10.202.80.1" LXD_IPV4_NETMASK="255.255.255.0" LXD_IPV4_NETWORK="10.202.80.1/24" LXD_IPV4_DHCP_RANGE="10.202.80.2,10.202.80.254" LXD_IPV4_DHCP_MAX="252" LXD_IPV4_NAT="true" LXD_IPV6_ADDR="" LXD_IPV6_MASK="" LXD_IPV6_NETWORK="" LXD_IPV6_NAT="false" LXD_IPV6_PROXY="false" EOF ' $ sudo systemctl enable lxd-bridge $ sudo systemctl start lxd-bridge
3 Create containers
Create containers with lxc launch. '.' and '_' cannot be used to containers name.
$ # lxc launch <image> <countainer> $ lxc launch ubuntu:16.04 ubuntu-16-04
A image of Debian 8 in images.linuxcontainers.org is as below.
images:debian/jessie/amd64
You can get image list with below command.
$ lxc image list <remote>:
The remote servers are images, ubuntu and ubuntu-daily by default. Image size is about 100MByte.
$ lxc image list images: $ lxc image list ubuntu: $ lxc image list ubuntu-daily:
3.1 Fedora 22/23 dnf causes 'Inappropriate ioctl for device'
A gpgcheck of dnf on Fedora 22/23 containers does not work.
# dnf install -y <pkg> gpgme.GpgmeError: (7, 32870, 'Inappropriate ioctl for device')
A –nogpgcheck option can avoid this error.
# dnf install --nogpgcheck -y <pkg>
The tee command will fix this error according to RedHat bugzilla.
$ lxc exec images-fedora-23-amd64 -- dnf reinstall -y gzip | tee
Or run below command on containers.
$ lxc exec images-fedora-23-amd64 -- /bin/bash # dnf reinstall -y gzip | tee
3.2 Debian jessie's installation of policykit-1 will be freezed
Because LXC or Debian jessie has a problem, installation of policykit-1 will be freezed.
# apt install -y policykit-1 <snip> Setting up policykit-1 (0.105-8) ... <hung>
Debian strech has no this problem. Replacing systemd to Debian stretch's systemd will avoid this problem temporarily but some package dependencies will be broken. So Debian jessie cannot be used till this problem is fixed.
4 Network configration
A default profile is used for containers network by default.
$ lxc profile list default docker $ lxc profile show default devices: eth0: name: eth0 nictype: bridged parent: lxdbr0 type: nic
4.1 bridged (Not recommended)
A bridged will construct private network. A lxd-bridge provides NAT with dnsmasq. If you use multiple nameservers for private network and internet, dnsmasq will be problem.
4.1.1 Multiple nameserver cannot resolve name
A below /etc/resolv.conf provide nameserver 192.168.11.2 for private network and nameserver 192.168.11.1 for internet.
$ cat /etc/resolv.conf # Dynamic resolv.conf(5) file for glibc resolver(3) generated by # resolvconf(8) # DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE # OVERWRITTEN nameserver 192.168.11.2 nameserver 192.168.11.1 search hiroom2.com
The dnsmasq will use /etc/resolv.conf, but one nameserver return REFUSE which means cannot resolve name, dnsmasq will not request next nameserver.
For avoiding this problem, server directive and resolv-file directive must be used in dnsmasq configuration.
$ cat /var/lib/lxd-bridge/dnsmasq.conf server=/hiroom2.com/192.168.11.2 resolv-file=/var/lib/lxd-bridge/resolv.conf
A nameserver 192.168.11.1 for internet is in resolv.conf.
$ cat /var/lib/lxd-bridge/resolv.conf nameserver 192.168.11.1
This dnsmasq.conf is set to LXD_CONFILE.
$ diff -uprN /etc/default/lxd-bridge{.org,} --- /etc/default/lxd-bridge.org 2016-06-18 09:16:05.533082034 +0900 +++ /etc/default/lxd-bridge 2016-06-18 09:16:18.037062209 +0900 @@ -1,7 +1,7 @@ USE_LXD_BRIDGE="true" LXD_BRIDGE="lxdbr0" UPDATE_PROFILE="true" -LXD_CONFILE="" +LXD_CONFILE="/var/lib/lxd-bridge/dnsmasq.conf" LXD_DOMAIN="lxd" LXD_IPV4_ADDR="10.202.80.1" LXD_IPV4_NETMASK="255.255.255.0"
If /etc/resolv.conf is generated by dhcp server information, you need to change dnsmasq.conf and resolv.conf when dhcp server information is changed.
4.1.2 Applying patch for auto generating LXD_CONFILE
I create patch. Sorry but this is not enough. After lxd init, recreate deb file with below command.
Install packages for building lxd.
$ sudo apt install -y devscripts $ sudo apt-get build-dep -y lxd
Download lxd source code. If you could not download source code, please erase # before deb-src in /etc/apt/sources.list and run "sudo apt update".
$ mkdir lxd.ubuntu-16.04 $ cd lxd.ubuntu-16.04/ $ apt source lxd $ cd lxd-2.0.2
Apply patch.
$ GITHUB=https://github.com/hiroom2/lxd-pkg-ubuntu/commit $ wget ${GITHUB}/f9c429c8c7177ad1484cb2f6b3a17e5eeacf2442.patch $ patch -p1 < f9c429c8c7177ad1484cb2f6b3a17e5eeacf2442.patch $ echo "a.patch" | EDITOR=true dpkg-source --commit
Build and install lxd.
$ dpkg-buildpackage -us -uc $ sudo dpkg -i ../*.deb
Commant out LXD_CONFILE and add LXD_GEN_CONFILE="true" in /etc/default/lxd-bridge. /var/run/lxd-bridge/dnsmasq.conf and /var/run/lxd-bridge/resolv.conf will be generate and used by lxd-bridge.
$ sudo sed -i 's/^LXD_CONFILE/#LXD_CONFILE/g' /etc/default/lxd-bridge $ sudo su -c 'echo LXD_GEN_CONFILE="true" >> /etc/default/lxd-bridge' $ sudo systemctl restart lxd-bridge
4.2 macvlan (Not recommended)
macvlan provides external access. But the connection between host machine and containers cannot be enabled.
$ lxc profile create vlan $ cat <<EOF | lxc profile edit vlan name: vlan config: {} description: VLAN profile devices: eth0: name: eth0 nictype: macvlan parent: ens3 type: nic EOF
Apply this profile to containers which names ubuntu-16-04.
$ lxc profile apply ubuntu-16-04 vlan
MAC address can be checkd by lxc config show.
$ lxc config show ubuntu-16-04 | \ grep volatile.eth0.hwaddr: | \ awk '{ print $2 }' 00:16:3e:0e:f6:38
DHCP server can provide IP address to this MAC address.
$ lxc exec ubuntu-16-04 -- ifconfig eth0 eth0 Link encap:Ethernet HWaddr 00:16:3e:0e:f6:38 inet addr:192.168.11.86 Bcast:192.168.11.255 Mask:255.255.255.0 inet6 addr: fe80::216:3eff:fe0e:f638/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:70 errors:0 dropped:0 overruns:0 frame:0 TX packets:77 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1 RX bytes:7192 (7.1 KB) TX bytes:9830 (9.8 KB)
4.3 bridged (Recommended)
Create bridge interface before creating profile. This provides external access. And the connection between host machine and containers can be enabled.
Create profile which using br0.
$ lxc profile create bridge $ cat <<EOF | lxc profile edit bridge name: bridge config: {} description: Bridge profile devices: eth0: name: eth0 nictype: bridged parent: br0 type: nic EOF
5 Export containers to image
Stop containers and export to image.
$ lxc stop ubuntu-16-04 $ lxc publish ubuntu-16-04 --alias ubuntu-16-04 Container published with fingerprint: fdd934c336bbd506c2f4f068a872ccd7504564b50d27d3a168a4835b6cfe4e96
Image can be export to file.
$ lxc image export fdd934c336bbd506c2f4f068a872ccd7504564b50d27d3a168a4835b6cfe4e96
Exported image can be import on other machine.
$ lxc import fdd934c336bbd506c2f4f068a872ccd7504564b50d27d3a168a4835b6cfe4e96.tar.gz
6 Remote server configuration
NIC and port for LXD daemon listening is set when sudo lxc init. You can change NIC and port with below command.
$ lxc config set core.https_address 0.0.0.0:8443
You can change password with below command.
$ lxc config set core.trust_password password
Other machine can add this machine as remote server.
$ lxc remote add myimages 192.168.11.77:8443 Certificate fingerprint: c2e967e5d788db7fbbd19e1f22b789b74313083094bf5585334fe3d15192ab6a ok (y/n)? y Admin password for myimages: # type password Client certificate stored at server: myimages
6.1 Download image from remote server
Download image from remote server like images, ubuntu, ubuntu-daily.
$ lxc image list myimages: <snip> ... b39919dbaf79 ... <snip>
$ lxc launch myimages:b39919dbaf79 container-from-myimages Creating container-from-myimages
6.2 Container migration
Install criu package local machine and remote server.
$ lxc move images-centos-7-amd64 myimages: $ lxc list myimages: <snip> ... images-centos-7-amd64 ... <snip>