Extending the Kernel with Built-in Kernel Headers

Note: this article is a followup to Zack Brown's "Android Low Memory Killer—In or Out?"

Linux kernel headers are the unstable, constantly-changing, internal API of the kernel. This includes internal kernel structures (for example, task_struct) as well as helper macros and functions. Unlike the UAPI headers used to build userspace programs that are stable and backward-compatible, the internal kernel headers can change at any time and any release. While this allows the kernel unlimited flexibility to evolve and change, it presents some difficulties for code that needs to be loaded into the kernel at runtime and executed in kernel context.

Kernel modules are a prime example of such code code. They execute in kernel context and depend on this same unstable API that can change at any time. A module has to be built for the kernel it is running on and may not load on another because of an internal API change could break it. Another example is eBPF tracing programs. These programs are dynamically compiled from C to eBPF, loaded into the kernel and execute in kernel space in an in-kernel BPF virtual machine. Since these programs trace the kernel, they need to use the in kernel API at times, and they have the same challenges as kernel modules as far as internal API changes go. They may need to understand what data structures in the kernel look like or call kernel helper functions.

Kernel headers are usually unavailable on the target where these BPF tracing programs need to be dynamically compiled and run. That is certainly the case with Android, which runs on billions of devices. It is not practical to ship custom kernel headers for every device. My solution to the problem is to embed the kernel headers within the kernel image itself and make it available through the sysfs virtual filesystem (usually mounted at /sys) as a compressed archive file (/sys/kernel/kheaders.tar.xz). This archive can be uncompressed as needed to a temporary directory. This simple change guarantees that the headers are always shipped with the running kernel.

Several kernel developers disagreed with the solution; however, kernel maintainer Greg Kroah-Hartman was supportive of the solution as were many others. Greg argued that the solution is simple and just works as did other kernel developers. Linus pulled the patches in v5.2 of the kernel release.

To enable the embedded kernel headers, build your kernel with CONFIG_KHEADERS=y kernel option, or =m if you want to save some memory.

The rest of this article looks at challenges with kernel headers, solutions and the limitations.

Challenges with Kernel Headers

Filesystem or Archive?