One of piece of infrastructure that's come in as part of the Kernel Self Protection Project (KSPP) is support for gcc plugins. I touched on this briefly in my DevConf talk but I wanted to discuss a few more of the 'practicalities' of dealing with compiler plugins.
At an incredibly abstract level, a compiler transforms a program from some form
A to another form
A might be C, C++ and you expect
A' to be
a binary file you can run. Modern compilers like gcc produce the final result
by transforming your program several passes, so you end up with
A''' etc. The gcc plugin architecture allows you to hook in at
various points to make changes to the intermediate state of the program.
gcc has a number of
internal representations so depending on where you are hooking you may need
to use a different representation.
Kernel development gets a (not undeserved) reputation for being poorly
documented and difficult to get into. To write even a self-contained kernel
module requires some knowledge about the rest of the code base. If you have
some familiarity with the code base it makes things much easier. I've found
compiler plugins to be similarly difficult. I'm not working with the gcc
code base on a regular basis so figuring out how to do something practical
with the internal structures feels like an uphill battle. I played around
with writing a toy plugin to look at the representation and it took me
forever to figure out how to get the root of the tree so I could do something
as simple as call
walk_tree. Once I figured that out, I spent more time
figuring out how to actually do a switch on the node to see what type it was.
Basically, I'm a beginner in an unfamiliar code base so it takes me a while
to do anything.
Continuing the parallels between kernels and compilers, the internal ABI of
gcc may change between versions, similar how the kernel provides no stable
internal ABI. If you want to support multiple compiler versions in your plugin,
this results in an explosion of
#ifdef VERSION >= BLAH all throughout the
code. Arguably, external kernel modules have the same problem but I'd argue
the problem is slightly worse for compiler plugins. Kernel modules can be
built and shipped for particular kernel versions but it's harder to require
specific compiler versions.
With all this talk about how hard it is to use compiler plugins, there might be some questions about if it's really worth it to support them at all. My useless answer is "it depends" and "isn't that the ultimate question of any feature". If you have a plugin that can eliminate bug classes, is it worth the maintenance burden? I say yes. One long term option is to get features merged into the main gcc trunk so they don't have to be carried as plugins. Some of the tweaks are kernel specific though, so we're probably stuck carrying the more useful plugins. There is interest in new compiler flags and features so we'll have to see what happens in the future.