Docker’s LinuxKit
Asyou saw in my recent DockerCon recap post, Docker released two new interesting pieces of software during a weeklong container-rama in Austin, Texas.
- The Moby Project: is an open source push to advance the software containerization movement by providing a set of components, frameworks, reference assemblies, and place for “container enthusiasts to experiment and exchange ideas.”
- LinuxKit: a purpose built toolkit that enables engineers to create secure, immutable, and minimal Linux distributions.
In this post I’ll show you how to create a mini-OS of your very own, using the magic of the
moby tool to create images, and the linuxkit tool for pushing and running VM images.
Let’s jump to it.
Prerequisites
In order to get started, you’re going to need Go and Docker for Macinstalled. I’d recommend just getting the installer for Golang if you’d like to make it easier.
Open up a terminal and grab the source code for the
linuxkit software.$ git clone https://github.com/linuxkit/linuxkit
Option 1: use the Makefile. Change into the
linuxkitdirectory, and then run make to install the moby tool and add it to your path. We’ll detail that in a moment.
Option 2: If you already have go installed you can run:
go get -u github.com/linuxkit/linuxkit/src/cmd/moby
to install the
moby tool, this command to get linuxkit.go get -u github.com/linuxkit/linuxkit/src/cmd/linuxkit
Let’s use the Makefile.
Once you’ve clone the code, switch into the directory and run
make.
This does a few nice things for us, including sticking our tools into
bin/, though it’s up to us to add infrakit-instance-hyperkit, linuxkit, and moby to our path.$ sudo cp bin/* /usr/local/bin/
Alternatively, you can simply run the
make install command.$ sudo make install
Once we have the tool, let’s build the example in the main repository: linuxkit.yml. Let’s build the example configuration and then run it.
$ moby build linuxkit.yml Extract kernel image: linuxkit/kernel:4.9.x Add init containers: Process init image: linuxkit/init:42fe8cb1508b3afed39eb89821906e3cc7a70551 Process init image: linuxkit/runc:b0fb122e10dbb7e4e45115177a61a3f8d68c19a9 Process init image: linuxkit/containerd:60e2486a74c665ba4df57e561729aec20758daed Process init image: linuxkit/ca-certificates:eabc5a6e59f05aa91529d80e9a595b85b046f935 Add onboot containers: Create OCI config for linuxkit/sysctl:2cf2f9d5b4d314ba1bfc22b2fe931924af666d8c Create OCI config for linuxkit/binfmt:8881283ac627be1542811bd25c85e7782aebc692 Create OCI config for linuxkit/dhcpcd:48e249ebef6a521eed886b3bce032db69fbb4afa Add service containers: Create OCI config for linuxkit/rngd:3dad6dd43270fa632ac031e99d1947f20b22eec9 Create OCI config for nginx:alpine Add files: etc/docker/daemon.json Create outputs: linuxkit-bzImage linuxkit-initrd.img linuxkit-cmdline linuxkit.iso linuxkit-efi.iso linuxkit $
This will create a number of OS formats that you can run on various platforms. When we execute the
moby run command, linuxkit will automatically select the correct backend depending on whether you’re running OSX or Linux.Create outputs: linuxkit-bzImage linuxkit-initrd.img linuxkit-cmdline linuxkit.iso linuxkit-efi.iso
Here’s how we can check what backends we have available.
$ linuxkit run — help USAGE: moby run [backend] [options] [prefix]
‘backend’ specifies the run backend. If not specified the platform specific default will be used Supported backends are (default platform in brackets): gcp hyperkit [macOS] qemu [linux] vmware packet
‘options’ are the backend specific options. See ‘moby run [backend] — help’ for details.
‘prefix’ specifies the path to the VM image. It defaults to ‘./moby’. $
Once we’ve built the configuration, go ahead and run it.
$ linuxkit run linuxkit virtio-net-vpnkit: initialising, opts=”path=/Users/jesse/Library/Containers/com.docker.docker/Data/s50" virtio-net-vpnkit: magic=VMN3T version=1 commit=0123456789012345678901234567890123456789 Connection established with MAC=02:50:00:00:00:02 and MTU 1500 early console in extract_kernel input_data: 0x0000000001f2c3b4 input_len: 0x000000000067b5ac output: 0x0000000001000000 output_len: 0x0000000001595288 kernel_total_size: 0x000000000118a000 booted via startup_32() Physical KASLR using RDRAND RDTSC… Virtual KASLR using RDRAND RDTSC… Decompressing Linux… Parsing ELF… Performing relocations… done. Booting the kernel. ... <<<A LOT MORE OUTPUT>>> <<<A LOT MORE OUTPUT>>> ...
Welcome to LinuxKit
## .
## ## ## ==
## ## ## ## ## ===
/"""""""""""""""""\___/ ===
~~~ {~~ ~~~~ ~~~ ~~~~ ~~~ ~ / ===- ~~~
\______ o __/
\ \ __/
\____\_______/
/ # [ 2.409832] IPVS: Creating netns size=2104 id=1 [ 2.410248] IPVS: ftp: loaded support on port[0] = 21 [ 2.439048] tsc: Refined TSC clocksource calibration: 2492.430 MHz [ 2.439438] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x23ed4aa1ffc, max_idle_ns: 440795280344 ns [ 2.641612] IPVS: Creating netns size=2104 id=2 [ 2.642177] IPVS: ftp: loaded support on port[0] = 21 [ 3.452517] clocksource: Switched to clocksource tsc
/ #
Great! Now you’re running a bare bones Linux ISO with the 4.9 kernel as described in the
linuxkit.yml file.kernel: image: “linuxkit/kernel:4.9.x” cmdline: “console=ttyS0 console=tty0 page_poison=1”
You can also check the architecture from the command line. You’ll need to hit the enter key to see the command line prompt.
/ # uname -a Linux moby-025000000003 4.9.22-moby #1 SMP Fri Apr 14 12:32:33 UTC 2017 x86_64 Linux
Great, it matches up. To escape command prompt, issue a `halt` command.
/ # halt Terminated / # [ 60.996724] reboot: System halted
Now you’re free to copy the
linuxkit.yml example and create your own images. The yaml format builds the image according to a set of a parameters, according to the documentation:kernelspecifies a kernel Docker image, containing a kernel and a filesystem tarball, eg containing modules. The example kernels are built fromkernel/
initis the baseinitprocess Docker image, which is unpacked as the base system, containinginit,containerd,runcand a few tools. Built frompkg/init/
onboot are the system containers, executed sequentially in order. They should terminate quickly when done.
services is the system services, which normally run for the whole time the system is up
files are additional files to add to the image
outputs are descriptions of what to build, such as ISOs.
Let’s turn things up a bit and build the Kubernetes LinuxKit demo from DockerCon.
Change first to the
Kubernetes/ project directory.$ cd projects/kubernetes/
You’ll need to add your public ssh key to the
kube-master.yml file in the files section.$ vim kube-master.ymlfiles:
— path: root/.ssh/authorized_keys
contents: <your key here>
- {path: etc/cni, directory: true}
- {path: opt/cni, directory: true}
Once that’s set, build the OS images with
make build-vm-images.
After those images are complete, you could use the
hyperkit software on OSX to boot the Kubernetes master image.$ ./boot-master.sh
After a few moments, you’ll be presented with a command prompt for a running Kubernetes master.
ps -ef |grep [k]ube
784 root 0:00 /usr/bin/runc run --bundle /containers/services/kubelet --pid-file /run/kubelet.pid kubelet
841 root 0:00 {kubelet.sh} /bin/sh /usr/bin/kubelet.sh
1528 root 0:00 kubelet --kubeconfig=/var/lib/kubeadm/kubelet.conf --require-kubeconfig=true --pod-manifest-path=/var/lib/kubeadm
/manifests --allow-privileged=true --cluster-dns=10.96.0.10 --cluster-domain=cluster.local --cgroups-per-qos=false --enforce-node-allo$
atable= --network-plugin=cni --cni-conf-dir=/etc/cni/net.d --cni-bin-dir=/opt/cni/bin
On your new linuxkit powered machine, grab the IP address of the master, so we can log into the kubelete container.
/ # ip addr show dev eth0
4: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 02:50:00:00:00:03 brd ff:ff:ff:ff:ff:ff
inet 192.168.65.4/24 brd 192.168.65.255 scope global eth0
valid_lft forever preferred_lft forever
inet6 fe80::600d:f07b:66b5:43bb/64 scope link
valid_lft forever preferred_lft forever
Back on your local machine, run the connection script to log into the kubelete
$ ./ssh_into_kubelet.sh 192.168.65.5 + ./ssh.sh -t 192.168.65.5 nsenter --mount --target 1 runc exec --tty kubelet ash -l + docker run -ti -v /Users/jesse/.ssh/:/root/.ssh jdeathe/centos-ssh ssh -o Compression=yes -o LogLevel=FATAL -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o IdentitiesOnly=yes -t 192.168.65.5 nsenter --mount --target 1 runc exec --tty kubelet ash -l moby-025000000004:/#
and then manually initialize the master with
kubeadm-init.sh.
Once the
kubeadm command completes, you’ve have the beginning of a bootstrapped cluster. You’ll need to grab the kubeadm join arguments from the output so we can spin up three other nodes with hyperkit and add them to the cluster as nodes.You can now join any number of machines by running the following on each node moby-025000000004:/# tmux --help
kubeadm join --token 814e8c.110a38974eb9e54e 192.168.65.5:6443
To start, you’ll have a single node, so let’s add three more.
moby-025000000004:/# kubectl get nodes NAME STATUS AGE VERSION moby-025000000004 Ready 45m v1.6.1 moby-025000000004:/#
Back on your laptop, we’ll use the
boot-node.sh script to launch three more hyperkit powered machines. In three separate shells, use your output to run the boot-node.sh script three times with the aforementioned arguments.
Here’s what it looked like on my machine.
$ ./boot-node.sh 1 --token 814e8c.110a38974eb9e54e 192.168.65.5:6443 + rm -f kube-node-1-disk.img + ../../bin/linuxkit run -cpus 2 -mem 4096 -disk-size 4096 -disk kube-node-1-disk.img -data '--token 814e8c.110a38974eb9e54e 192.168.65 .5:6443' kube-node virtio-net-vpnkit: initialising, opts="path=/Users/jesse/Library/Containers/com.docker.docker/Data/s50" virtio-net-vpnkit: magic=VMN3T version=1 commit=0123456789012345678901234567890123456789 Connection established with MAC=02:50:00:00:00:08 and MTU 1500 early console in extract_kernel input_data: 0x0000000001ef83b4 input_len: 0x00000000006faaf1 output: 0x0000000001000000 output_len: 0x00000000015e1278 kernel_total_size: 0x00000000011c6000 booted via startup_32() Physical KASLR using RDRAND RDTSC... Virtual KASLR using RDRAND RDTSC... <SNIPPED>
The other commands will yield similar output.
shell2> ./boot-node.sh 2 --token bb38c6.117e66eabbbce07d 192.168.65.22:6443
shell3> ./boot-node.sh 3 --token bb38c6.117e66eabbbce07d 192.168.65.22:6443
After those boot up, you’ll be able to see your new nodes via
kubectl on the master.moby-025000000004:/# kubectl get nodes NAME STATUS AGE VERSION moby-025000000004 Ready 45m v1.6.1 moby-025000000005 Ready 14m v1.6.1 moby-025000000006 Ready 13m v1.6.1 moby-025000000007 Ready 13m v1.6.1
Now you’d got a very lean, secure image from which to deploy Kubernetes!
I hope you’ve enjoyed this post. If you’d like to learn more about Linuxkit, be sure to check out the repository, and read my post on the announcement of The Moby Project and Linuxkit. If you’ve got any questions feel free to leave a comment here or on Twitter.
If you have questions or would like to get involved with the project, join the discussion in the Docker Slack community.
Thanks for reading, and as ever — click the heart below if this helped!
reference: https://medium.com/contino-io/dockers-linuxkit-2c460769f522
留言
張貼留言