GSOC 2016 Summary

As Google Summer of Code is almost over I will create a summary of my contribution to The Linux Foundation during this period.

Firstly, I will describe again my project in order for the further work description to be more comprehensible.

This project aims to add support for Linux Kernel backports in already existing configuration option, localmodconfig. The goal is to automatically determine the available functionalities not only from the drivers provided by the current kernel version but also from all the backported ones. This project is not limited to backports but aims to build a better kernel configuration generator. Current solutions are based on heuristics so this project’s ultimate goal is to provide a deterministic algorithm of generating the most suitable kernel configuration for a target machine.

I have documented my entire work in blog posts so, for additional information about the project’s purpose you can consult this post.

This project consists of research so it had multiple phases / approaches:

1. The first approach was to use hardware interrogation tools to determine what actual devices are available on a particular system and create a configuration which includes the drivers for these devices. During my research I have created this document gathering all information encountered. This approach is not fiable: firstly, the list is not exhaustive for the existing hardware interfaces and, secondly, some tools do not provide driver information about the connected devices so they are not relevant. Finally, I have found a generic hardware interrogation tool ‘udevadm’ which determines all connected devices and driver information for those which have an available driver in the current kernel.

Other documents created during phase 1: Short summary based on the ‘Attack Surface Metrics and Automated Compile-Time OS Kernel Tailoring’ paper which targets same goal as this project.

2. I modified the already existing scripts/kconfig/streamline_config.pl which creates a configuration starting from the output of ‘lsmod’ (comand which returns the current loaded drivers) to apply same algorithm for drivers found with ‘udevadm’

Phases 1 & 2 are documented in this post including conclusions and results.

3. Since not all devices expose the driver name my next task was to extend entries from virtual file system /sys to display by default the CONFIG_* symbol that needs to be set at configuration time in order to add the corresponding driver to the kernel. I extended the kernel ‘struct kobject’ to contain and display this information.

Here you can find the patch for this implementation.

Documentation, conclusions and results: this blog post – section Approach 1.

4. Modifying the kobject structure did not provide the required results so next step was to extend the kernel ‘struct module’ to display the desired information.

Documentation, conclusions and results: this blog post – section Approach 2

The series of patches:

Cover letter

[RFC PATCH 1/3] Add kconfig_symbol attribute to struct module

[RFC PATCH 2/3] Add generation of Module.ksymb file in streamline_config.pl

[RFC PATCH 3/3] Add dynamic pegging of Kconfig symbol

5.  Starting from the previous series of patches I have implemented dynamic pegging of CONFIG_* symbol, namely, CONFIG_* symbol is determined at compilation time by the Makefile and passed to the kernel modules as compilation parameter. This implementation also solves few matching problems between module names – CONFIG_* symbol and optimizes it.

Documentation, conclusions and results: this blog post – section Approach 3

The series of patches:

Cover letter

[RFC PATCH 1/5] Add generation of Module.symb in streamline_config

[RFC PATCH 2/5] Add CONFIG symbol to module as compilation parameter

[RFC PATCH 3/5] Trigger Module.ksymb generation in Makefile

[RFC PATCH 4/5] Set KCONFIG_KSYMB as value for kconfig_ksymb module attribute

[RFC PATCH 5/5] Add kconf_symb as kernel module attribute

 

What is done by now ?

The result of the project currently is that for each module instance from /sys  there is a kconfig_symbol atribute which displays the CONFIG_* option for that module.

The project’s flow:

  1. Generate associations (module name – CONFIG_* symbol) into file scripts/Module.ksymb
  2. Update Makefile to load file scripts/Module.ksymb and access the matchings.
  3. Update the Makfile to dynamically add the corresponding CONFIG_FOO symbol to the compilation command of kernel module ‘foo’ as macro passed as parameter
  4. Add extra attribute to kernel ‘struct module’ for kconfig_symbol
  5. Update modpost to add that macro as content of kconfig_symbol attribute

The project provides the infrastructure of  loading the associations from a file scripts/Module.ksymb, adding them to the kernel module and update the kernel module’s code to have an extra attribute in order to display this information. Therefore, if the generator will be upgraded / replaced with another one which outputs data in the same format, the other components of the project are perfectly reusable.

The solution offers a good matching (module-name <-> CONFIG_* symbol) correctness percentage but still has some issues. For the tested system (x86_64, configuration generated with ‘make localmodconfig’), from a total of 58 loaded modules, 54 has a correct kconfig_symbol attribute while the other 4 had an empty kconfig_symbol attribute.

example-gsoc

Note: This post contains brief descriptions of each phase. More details can be found in each corresponding blog post.

What will be done ?

  1. Implement a better algorithm for generating the associations (module-name <–> CONFIG_* symbol).
  2. Extend kernel structures to display the CONFIG_* symbol for all devices not only the ones for which drivers are loaded
  3. Extend udevadm to extract CONFIG_* symbols from /sys along with other collected information

 

Regards,

Cristina-Gabriela Moraru

Advertisements

One thought on “GSOC 2016 Summary

  1. Good job!

    I’d like to note that although an alternative build system approach was proposed to avoid the reverse engineering approach I am personally not convinced yet we have enough expressed motivation for a change. As such I’d encourage to continue on with your approach without affecting upstream much. I think striving for a ‘make udevconfig’ as you had proposed once is a good decent next step. As time goes by we can revisit a change, but tools still need to be developed in order to be able to take complete advantage over a reverse mapping between CONFIG symbols and modules or built-in objects. When more tools mature I think we’ll have to revisit this.

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s