Do you need to recompile your Linux kernel in order to make a change to a module? What if you just want to try to fix a small bug while running a distro kernel package?
The Linux kernel source tree is large and rebuilding from scratch is a productivity killer. This article covers how to make changes to kernel modules without rebuilding the entire kernel.
I need to preface this by saying that I don't know if this is a "best practice". Maybe there are better ways but here is what I've been using recently.
Step by step
In most cases you can safely modify one or more kernel modules without rebuilding the whole kernel. Follow these steps:
1. Get the kernel sources
Download the kernel source tree corresponding to your current kernel version. How to get the kernel sources for the exact kernel package version you are currently running depends on your Linux distribution. On Fedora do the following:
$ dnf download --source kernel # or specify the exact kernel-X.Y.Z-R package you need
kernel-4.14.14-300.fc27.src.rpm 1.9 MB/s | 98 MB 00:50
$ rpmbuild -rp kernel-4.14.14-300.fc27.src.rpm
$ cd ~/rpmbuild/BUILD/kernel-4.14.fc27/linux-4.14.14-300.fc27.x86_64/
If you can't figure out how to get the corresponding kernel sources, use uname -r to find the kernel version and grab the vanilla sources from git. This will work as long as the kernel package you are running hasn't been patched too heavily by the package maintainers:
$ git clone git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git
$ cd linux-stable
$ uname -r
4.14.14-300.fc27.x86_64
$ git checkout v4.14.14 # let's hope this is close to what we're running!
2. Get the kernel config file
It is critical that you use the same .config file as your distro as some configuration options will build incompatible kernel modules. All will be good if your .config file matches your kernel's configuration, so grab it from /boot:
$ cp /boot/config-$(uname -r) .config
$ make oldconfig # this shouldn't produce much output
3. Set the version string
Kernel module versioning relies on a version string that is compiled into each kernel module. If the version string does not match your kernel's version then the module cannot be loaded. Be sure to set CONFIG_LOCALVERSION to match uname -r in the .config file:
$ uname -r # we only want the stuff after the X.Y.Z version number
4.14.14-300.fc27.x86_6
$ sed -i 's/^CONFIG_LOCALVERSION=.*$/CONFIG_LOCALVERSION="-300.fc27.x86_64"' .config
4. Build your modules
Use the out-of-tree build syntax to compile just the modules you need. In this example let's rebuild drivers/virtio modules:
$ make modules_prepare
$ make -j4 M=drivers/virtio modules # or whatever directory you want
5. Install and copy your modules
It can be useful to install the modules in a staging directory so they can copied to remote machines or installed locally:
$ mkdir /tmp/staging
$ make M=drivers/virtio INSTALL_MOD_PATH=/tmp/staging modules_install
$ scp /tmp/staging/lib/modules/4.14.14-300.fc27.x86_64/extra/* root@remote-host:/lib/modules/4.14.14-300.fc27.x86_64/kernel/drivers/virtio/
Beware that some distros ship compressed kernel modules. Set CONFIG_MODULE_COMPRESS_XZ=y in the .config file to get .ko.xz files, for example.
6. Reload modules or reboot the test machine
Now that the new modules are in /lib/modules/... it's time to load them. If the old modules are currently loaded you may be able to rmmod them after terminating processes that rely on those modules. Then load the new modules using modprobe. If the old modules cannot be unloaded because the system depends on them, you need to reboot.
If the modules you modified are loaded during early boot, you'll need to rebuild the initramfs. Make sure you have a backup initramfs in case the system fails to boot!
Caveats
This approach has limitations that mean it's mostly useful for
debugging and development. For quality assurance testing it is better
to follow a full build process that produces the same output that end
users will install.
Here are some things to be aware of:
- Don't make .config changes unless you are sure they are compatible with the running kernel.
- Do not introduce new module dependencies since this approach doesn't rebuild dependency information.
- Do not change exported symbols if other kernel modules depend on the code you are changing, unless you also rebuild the modules that depend on yours.
- Your modified modules will not be cryptographically signed and will taint the kernel if your distro kernel package is signed.
What happens if things go wrong? Either you'll get an error when attempting to load the kernel module. Or you might just get an oops when there is a crash due to ABI breakage.
Conclusion
This may seem like a long process but it's faster than recompiling a full kernel from scratch. Once you've got it working you can keep modifying code and rebuilding from Step 3.
