Tuesday, December 4, 2012

qemu-kvm.git has unforked back into qemu.git!

With the QEMU 1.3.0 release the qemu-kvm.git fork has now been merged back. The qemu.git source tree now contains equivalent code - it is no longer necessary to use qemu-kvm.git.

This is great news and has taken a lot of work from folks in the community. The qemu-kvm.git source tree had a number of differences compared to qemu.git. Over time, these changes were cleaned up and merged into qemu.git so that there is no longer a need to maintain a separate qemu-kvm.git source tree.

Many distros had both qemu and qemu-kvm or kvm packages. This sometimes led to confusion when people were unsure which package to install - both packages supported KVM to some degree. Now they are equivalent and distros will be able to simplify QEMU packaging.

For full details of the QEMU 1.3.0 release, see the announcement.

Friday, November 9, 2012

GlusterFS for KVM Users and Developers at KVM Forum 2012

At KVM Forum 2012 I gave an overview of GlusterFS for KVM users and developers. If you're looking for an introduction to the GlusterFS distributed storage system, check out the slides:

GlusterFS for KVM Users and Developers [PDF]

EDIT: Vijay Bellur provided a link to his presentation on extending GlusterFS. Check it out for deeper information on writing xlators:

Extending GlusterFS - The Translator Way [PDF]

Hopefully slides from other talks will become available online at the KVM Forum 2012 website.

Friday, September 21, 2012

Thoughts on Linux Containers (LXC)

I have been using Linux Containers (LXC) to run lightweight virtual machines for about a year now. I revisited LXC this week because I temporarily lack access to a machine with virtualization extensions for KVM. So I need an alternative for running Linux virtual machines with good performance. Here are my thoughts on LXC and its current status.

What is Linux Containers (LXC)?

The Linux Containers (LXC) project provides container virtualization for Linux, which has been described as "chroot(8) on steroids". The idea of containers is that resources like process identifiers (pids), user identifiers (uids), or memory are no longer managed globally by the kernel. Instead, the kernel accounts each resource against a container and gives them separate namespaces. This way multiple containers can run an init(8) process with pid 1 without conflicting, for example.

LXC can run multiple Linux systems simultaneously on one kernel. Containers are isolated from each other and do not conflict because each resource belongs to a container. The lxc 0.8.0~rc1-8+deb7u1 package I used on Debian comes with "templates" for Fedora, OpenSuSE, Ubuntu, Debian, Arch Linux, and altlinux. It takes about 10 minutes to create a new container pre-installed with one of these Linux distributions.

It's also possible to share resources, like file systems, between containers. It's even possible to run just a single process in a container without booting up a full Linux distribution - this really is "chroot(8) on steroids". But I think these features are useful to fewer users because they require understanding the exact resource dependencies of the application being containerized. It's easier to boot up a full Linux distribution to run the application.

In case you're wondering, Linux Containers (LXC) is purely a software kernel feature and therefore works fine inside KVM, Xen, or VMware. I use it to slice up my virtual server hosted on KVM and it works fine on Amazon EC2, Linode, etc.

How to create a container

First you need to install the lxc package:

sudo aptitude install lxc

There are now a bunch of utilities called lxc(1), lxc-start(1), lxc-create(1), etc available. The lxc(1) utility is a wrapper for the others, similar to git(1) versus git-branch(1).

Create a Debian container from its template:

sudo lxc create mydebian -t debian

There is a configuration file in /var/lib/lxc/mydebian/config that defines resources available to the container. You need to edit this file to set memory limits, set up networking, change the hostname, and more. The specific configuration options are out of scope for this post. Refer to your lxc package documentation, for example, /usr/share/doc/lxc/ on Debian.

Once installation is complete, launch the container:

sudo lxc start mydebian

To shut down the container:

sudo lxc shutdown mydebian

The rough edges

Use virt-manager, if possible

LXC has a few hurdles to get going. They remind me of issues first-time qemu users face. And the answer to configuration complexity is to use a high-level tool like virt-manager. Here are the advantages to using virt-manager:

  • Configure the container from a graphical interface. Skip learning configuration file syntax for something you will rarely edit.
  • Sets up environment dependencies for cgroups and networking. If you're not familiar with network bridges or IP forwarding this can save time.
  • Libvirt API support if you wish to automate or write custom tooling. This is actually a libvirt benefit and not specific to virt-manager itself.

Unfortunately, I wasn't able to use virt-manager because libvirt requires cgroups controllers that are not available on my machine (without recompiling the kernel). Libvirt assumes that you want to fully isolate the container and relies on several cgroups controllers. In my case I don't care about limiting memory usage and it just so happens that the distro kernel I run doesn't have that controller built. I decided to use lxc(1) manually and deal with the configuration file and networking setup.

Templates should indicate what to do after installation

After installing the Debian template, the container booted without networking and no login console. The boot process also spits out a number of errors that are not common on baremetal systems, because the unmodified Debian distro is trying to do things that are not allowed inside the container. Without much in the way of getting started documentation, it took a little while to figure out what was missing.

I ensured the ttys were running getty in /etc/inittab and created /dev/ttyX devices in the container. That took care of login consoles and made lxc console mydebian happy.

For networking I chose to use the veth type together with IP forwarding for external network access. This is fairly standard stuff if you've been playing with KVM or Xen.

Anyway, what is missing from LXC is some information between lxc create and lxc start that gives hints what to do in order to successfully boot and access the container.


Linux Containers (LXC) allows you to run lightweight Linux virtual machines with good performance. Once they are set up they boot quickly and feel like baremetal. As of September 2012, the container setup experience requires patience and some experience with troubleshooting Linux boot and networking issues. Hopefully the libvirt lxc support will reach the level where it's possible to create containers in virt-manager's wizard without getting familiar with low-level details.

Are you using LXC or have you tried it? Share your comments below.