Kernel compilation howto

By: Frank Rysanek of FCC prumyslove systemy s.r.o. [rysanek (AT) fccps.cz]
Last update: August 2019 (Debian 9/10, kernel 5.2.x)

Contents

Prerequisites

For the compilation itself, you may need several packages. Historically I seem to recall packages such as:

gcc
make (gmake, GNU make)
binutils
bc
kernel-headers (perhaps not necessary)
bison
byacc
flex
ncurses-dev
libssl-dev
libelf-dev
bzip2 (perhaps not necessary)

If unsure/lazy, just try the following steps, and see if you can get by without checking the required packages. Modern distros may have them all in a typical install.
(Don't expect them in the bare Debian "base system" though.)

Get a copy of the kernel source

Download and unpack a kernel tarball from kernel.org. Feel free to use a browser to select a version that suits you.

cd /usr/src
wget --no-check-certificate https://cdn.kernel.org/pub/linux/kernel/v5.x/linux-5.0.13.tar.xz
tar xvf linux-5.0.13.tar.xz
cd linux-5.0.13

Alternatively, you can install the kernel source for your distro kernel from RPM/DEB depending on distribution... (using RPM, YUM, APT, whatever)

Configure your kernel before compilation

Tell the newly unpacked kernel sources to get comfortable in your system:

make mrproper
Nominally, this is a "thorough make clean", but it may also prepare some symlinks and whatnot (or so they used to say in the old days).

Configure the kernel, using one of the following:

make oldconfig
or
cp /boot/config-<your_choice> ./.config
make menuconfig

The point is that the kernel config is stored in a text file called .config, that needs to be created in the kernel source tree root directory (root of the tarball just unpacked). The pristine kernel tarball does not contain one. If it does, such .config probably doesn't match your distro/hardware. How you arrive at a plausible .config, that's your choice.

You can either use the traditional "make oldconfig", which will try to find a backup of the running kernel's config somewhere in the system.
Or, you can find the backed-up copy in the /boot directory of your distro. The backup copy is not mandatory for a proper boot of the distro, it's there merely for your convenience.
If you copy the .config from a backup by hand, and the kernel version is not a precise match (likely it is not), menuconfig can sort that out - it'll drop options it doesn't know for the kernel version being configured. The .config that you save on exit from menuconfig will be valid.

You can also create a new .config from scratch, by running make menuconfig in a pristine kernel source tree.

Generic pitfalls in the kernel config

Beware, the new kernel may add options (and defaults for them) that were not present in the old .config. Such as, recent kernels seem to add EXPERT and DEBUG_KERNEL:

 General setup -> Configure standard kernel features (expert users)
 Kernel hacking -> Kernel debugging
DEBUG_KERNEL gets auto-selected by EXPERT. And, DEBUG_KERNEL leaves debugging symbols in the binaries, which means that the kernel and all the modules are HUUUGE. For normal use, you definitely want to avoid these two options.

Beware of kernel module versioning. Depending on how advanced you are, it may help you avoid goofed-up kernel reinstalls, or it may present a hassle if you DO NOT want to recompile+reinstall the modules every time your main monolithic binary is recompiled. For least hassle, turn off kernel module versioning *and signing*. That's in menuconfig

  Enable loadable module support -> Module versioning support
  Enable loadable module support -> Module signature verification
Note that this makes your system less secure against malicious module injection (a local attack, probably requiring root privileges anyway)

Run make menuconfig, make your choices.
Upon leaving menuconfig, elect to save the configuration, which results in a fresh .config, valid for the kernel version at hand.

Historical snippet of advice:
You may want to compile the driver for your bootable disk controller into the main kernel binary monolithically (not as a module), later on this will get you slightly further down the boot process, even if you goof up when creating the initrd.
As a downside, you may expect some error messages from the HWconf/udev subsystems during boot, as they don't find the disk controller driver module :-)
On modern distroes, this should be a non-issue, as the updateinitramfs should find all the necessary drivers. = just keep all the HDD controller drivers as modules.

Compile and link

Next, you need to produce a kernel binary, and a set of modules. Modules will be created for components, that you tagged with "M" in menuconfig. (Note that you can skip these two commands, make install and make modules_install will call these automatically if omitted.)

make bzImage
make modules

The resulting bootable kernel binary will be left at
./arch/x86/boot/bzImage
The modules will be interspersed with source files throughout the whole source tree.

Performance hint: in a modern PC, you likely have multiple CPU cores. Especially the compilation and linking phase are bounded by CPU horsepower, and the individual translation units can be handled in parallel. The "make" program can handle this parallelization automagically. All you need to do is tell it, how many parallel processes to start. You can even count on SMT/HT to squeeze the last bit of juice out of your CPU (feel free to benchmark and compare).
The sweet parameter here is -j <number> . Meaning the number of threads desired. I typically put that number equal to the number of my CPU cores, but other people recommend N+1, perhaps to keep the machine busy even during periods while the system is waiting idle for the start of a new thread or something.
Example:

make -j 4 bzImage
make -j4 modules

That's right, I believe you can use -j N with or without a space between -j and N.

What do I do with the binaries

Beware: "make install" may end up updating your initrd and grub configuration!
See the following two chapters and maybe prepare their config for that eagerness on part of "make install". That said, don't panic, either - you will have a chance to change the config of initrd and grub after "make install".
Such as, if this kernel is newer than your current default entry, "update-grub" will try to boot your new kernel by default. What a pesky side-effect.

You can copy the monolithic kernel by hand from arch/i386/boot/bzImage to wherever you want it, probably /boot/SOME_NICE_NAME.

Or you can call "make install" to have your distro select a predictable name for your new kernel and do the copy to /boot.

You definitely want to perform "make modules_install", which will collect the resulting binary modules (*.ko) into /lib/modules/kernel_version/, which is where you want to leave them (unless you're going to copy them to another bootable disk / image, and obviously you know what you're doing)

make modules_install
make install

This has put the monolithic binary and the modules into place, automagically.

Let's take a look into the boot directory.

cd /boot
ls
ls -l

Create a new initrd

Most modern distro's are using a temporary "initial ramdisk", for various reasons. It used to be required for loading hardware-specific disk controller drivers as modules, nowadays there are further dependencies (LVM, udev, ...).

You can often get the distro to boot with a monolithic kernel, but it takes time to pick the right options that need to be made monolithic.

It's usually a safe bet to do something like "make oldconfig" and then build your own initial ramdisk with all the modules. The distro usually contains a script called something like mkinitrd, which requires a kernel version / path to the modules to be copied to the ramdisk image.

A particular initrd image typically only contains modules for a single kernel version = to support multiple kernel versions, you need a dedicated initrd file for each kernel version. (Lest some modules fail to load at initrd startup.) That's why, in /boot/, you can see a corresponding versioned initrd for every kernel image in there.

Debian/Ubuntu:

update /etc/initramfs-tools/initramfs.conf (for a smaller size, specify MODULES=dep)

update /etc/initramfs-tools/modules - you may wish to add some of:

raid1
sd_mod
crc16
crc32c_intel
crc32c_generic

Auto-update or manually create from scratch an initial ramdisk for your new kernel:

update-initramfs -u -k <kernel_version>     # = this is a higher level automagical tool
OR maybe
mkinitramfs [kernel_version]   # a lower level tool

If this kernel version is new on your system, skip the -u option to update-initramfs (or it will complain that there's nothing to update) and instead use the -c option (= create a new initrd for this kernel).

Configure and update Grub; reboot

Check /etc/default/grub , you may want to add the following at the kernel cmdline:

To disable any Spectre/Meltdown safeguards that eat CPU horsepower:

 The old way:
  pti=off nospectre_v1 spectre_v2=off l1tf=off nospec_store_bypass_disable no_stf_barrier noretpoline

 The new way (since Kernel 5.2):
  mitigations=off

If you want to let hwmon (lm_sensors) to access the SuperIO HW monitor chip:

acpi_enforce_resources=lax

To force a particular video mode (e.g. if a KVM switch is giving you trouble):

video=1280x1024@60
Note: the frame rate after @ is important / required. 60 Hz is a good value for most LCD's.

To work around some bugs in Intel ATOM BayTrail/Braswell IGP power saving, that cause freezes when running X-Windows with or even without video playback, try the following curse:

intel_idle.max_cstate=1

Return to the good old Ethernet netdevice names such as eth0, eth1 etc.:

net.ifnames=0

To make the system more talkative at boot time, try removing "quiet" and "splash".

Note that you can test the individual kernel cmdline curses at runtime, without clobbering your grub.conf forever - choose "extended options" in the Grub menu and pres "e" to edit the entry. Whatever options you add or erase, they won't get stored into grub.conf .

Still in /etc/default/grub, you may want to change the "default" entry. If you've just compiled a very new kernel, chances are that it's newer than your current running kernel, and update-grub will default to moving this new kernel to the formost position in the boot menu. The "default" in /etc/default/grub is a zero-based ordinal number, referring to the entries in /boot/grub/grub.cfg (menu.lst in the old days).

NOTE THAT SO FAR, YOU HAVEN'T SPOILED YOUR BOOT CONFIGURATION.
YOU HAVE MERELY COPIED A FEW NEW FILES SOMEPLACE HARMLESS.

ON TO THE HARMFUL STUFF.

Then finally, tell grub to update its config:

update-grub

Now the harm has been done, potentially :-)

reboot

Side notes on the build process

Further updates to the compiled and installed kernel:

At the compilation stage, if you want to erase any previously compiled object files (binary snippets comprising the final monolith), just call 'make clean' before, after or without 'make menuconfig'. Note that 'make mrproper' does an even more thorough cleanup, specifically it deletes any existing .config .

make clean
make menuconfig
make bzImage
make modules
...etc

If you turn off kernel module versioning (often a good thing), but you still want to keep a few different boot profiles with different kernel configurations, you need to have a few different module subdirs under /lib/modules, for the same kernel version. Some global configuration options can change your monolith and modules to the point that they're incompatible with a different configuration of the same version kernel source tree. So, to achieve the multiple copies of the module set under your /lib/modules subdir, consider explicitly modifying the top-level Makefile in the kernel source tree, the appropriate line is called

EXTRAVERSION
(somwehere at the top of the file)
Such as,
VERSION = 2
PATCHLEVEL = 6
SUBLEVEL = 21
EXTRAVERSION = .7-HACKED-v001
You should edit the Makefile before "make bzImage" and "make modules", i.e. before you start compiling the binaries. If you do it even before "make menuconfig", the new EXTRAVERSION gets reflected in .config (just in the comments on top, doesn't affect the compiled binaries)

The following invocation of "make modules_install" will automatically create a separate directory under /lib/modules, named accordingly.

At the very least, you can use a custom extraversion to distinguish your own cut of the modules from the distro-default modules (if you're recompiling the distro's default kernel version from source).

In general, use a different extraversion if:

You need to recompile and reinstall from scratch (start with "make clean") and maybe modify the EXTRAVERSION if:

OTOH, you can avoid "make clean" if:

If you're building some modules out of tree (which appears to be turning into a no-no recently) you need to have "modversions.h" already created in the tree you're building against.
I.e., your tree already needs to be past "make modules" (if not past "make modules-install").
There used to be an officially supported way (Makefile structure etc) for compiling your own drivers out of tree.
That said, it's sadly true that modern kernels tend to punish you for compiling things "out of tree", by tainting the kernel on insmod of such a module. The morale is: learn to implant your source code straight into the source tree :-(

Make a debian package

Debian kind of has a notorious stock package called kernel-package that's supposed to facilitate the packaging of a kernel.
Except that this "facilitation aid" is a behemoth, about 1.5 GB in size, and has some bug that made the maintainers withdraw that package from Stretch and Buster...

Which has prompted me to roll my own bash script called package_kernel.sh to slurp the kernel that's installed on the current system and turn that into a debian package, for installation on other systems.

You can put the script anywhere: in your home directory, in some working directory, or even in some system-wide directory with your custom executables - its location is not important.
The current working directory is your choice too. You can just use /usr/src/ if you want - which might be convenient, if this is where you unpack your kernel source trees... The script will create a subdirectory and a .deb file, both named after the kernel version and arch.

Usage (insert your own kernel version and CPU architecture):

package_kernel.sh 5.2.7 amd64

On the target system, apply using:

dpkg -i linux-image-VERSION-ARCH.deb

A note on versioning:
if you want the Debianese long version name, such as 4.9.0-8-amd64 instead of just 4.9.0, on /lib/modules/VERSION and on the files in /boot/*, you probably need to insert that version into the EXTRAVERSION variable in the Makefile = before compilation. As the package_kernel.sh script does not handle compilation, it cannot arrange that naming nuance...
On my part, call it a design decision = I don't need more rope to hang myself with...