KubeVirt is an application add-on to Kubernetes. KubeVirt adds additional virtualization resource types (virtual machines) via Kubernetes’ CRD API. KubeVirt lets the virtual machines (VMs) run concurrently with the Pods within Kubernetes, which allows them proper access to items that the Pods have access to — and allows the items in the Pods be controlled by the Kubernetes command-line interface. In this article, I will provide a high-level overview of the architecture of KubeVirt, going over the stack, the components, and what each component does.
KubeVirt is built using a service architecture and orchestration pattern. Virtualization services speak to the virtualization API, which in turn is speaking to the Kubernetes cluster to schedule requested virtual machine interfaces (VMIs). Scheduling, networking, and storage are all delegated to Kubernetes while KubeVirt provides the virtualization functionality.
The main CRD for KubeVirt is the VirtualMachine resource. This resource defines the properties of the virtual machine, like the number and type of NICs, CPU, and RAM.
When new VM objects are created, the virt-controller makes the Pod run within the virtual machine. When that happens, the Pod is automatically scheduled to a certain node; the virt-controller changes the VM object with the node name it is scheduled on, and gives control to the virt-handler node that is running on each node within the cluster.
KubeVirt creates one Pod for every VM object that’s created. Then the main container in the Pod runs the virt-launcher. This component gives the namespaces and cgroups needed for the VM process. The virt-handler passes on the VM’s CRD to the virt-launcher, and that makes use of a local libvirtd instance in the container to start the virtual machine. Virt-launcher monitors the VM process and will terminate once the virtual machine has exited.
There are moments where the Kubernetes runtime could try to shut down the virt-launcher Pod before the VM process has finished. Once this happens, the virt-launcher will give the kill signal to the VM process and try to slow the termination of the Pod until the virtual machine is properly shut down.
An instance of libvirtd is deployed with each VM Pod. The virt-launcher manages the lifecycle of the VM process from libvirtd.
KubeVirt Virtual Machine can be configured with disks that are backed by volumes. Persistent Volume Claim volumes make Kubernetes persistent volumes available as disks to the VM are deployed from KubeVirt. However, there is a limitation at the time of this writing (the persistent volumes need to be Internet Small Computer Systems Interface block devices), but the good news is that progress is being made to allow file-based persistent volume disks.
Ephemeral Volumes are supported as well, and are dynamically generated. They are connected with the VM when the VM starts, and are terminated when the VM is stopped. At the moment, these are backed by Persistent Volume Claim volumes.
Registry Disk volumes are supported as well. These volumes reference a Docker image that embeds a qcow or raw disk. They are then pulled from a container registry. Similar to ephemeral images, the data in the disks only function for the container’s life cycle.
CloudInit NoCloud volumes give VMs a user-data source which is added as a disk to give configuration information to users with cloud-init installed. The details are provided in clear text, base64-encoded UserData files or via Kubernetes Secrets.
Registry Disk configuration