diff --git a/index.html b/index.html index 2c6681d..31a8575 100644 --- a/index.html +++ b/index.html @@ -1414,14 +1414,15 @@ using the f_pos specific lock, which makes the file position update to become the mutual exclusion. So, we can safely implement those operations without unnecessary locking. -

Since Linux v5.6, the proc_ops +

Additionally, since Linux v5.6, the proc_ops structure was introduced to replace the use of the file_operations - structure when registering proc handlers. -

+ structure when registering proc handlers. See more information in the 7.1 +section. +

6.2 The file structure

-

Each device is represented in the kernel by a file structure, which is defined +

Each device is represented in the kernel by a file structure, which is defined in include/linux/fs.h. Be aware that a file is a kernel level structure and never appears in a user space program. It is not the same thing as a FILE @@ -1433,31 +1434,31 @@ function. Also, its name is a bit misleading; it represents an abstract open -

An instance of struct file is commonly named +

An instance of struct file is commonly named filp . You’ll also see it referred to as a struct file object. Resist the temptation. -

Go ahead and look at the definition of file. Most of the entries you see, like struct +

Go ahead and look at the definition of file. Most of the entries you see, like struct dentry are not used by device drivers, and you can ignore them. This is because drivers do not fill file directly; they only use structures contained in file which are created elsewhere. -

+

6.3 Registering A Device

-

As discussed earlier, char devices are accessed through device files, usually located in +

As discussed earlier, char devices are accessed through device files, usually located in /dev. This is by convention. When writing a driver, it is OK to put the device file in your current directory. Just make sure you place it in /dev for a production driver. The major number tells you which driver handles which device file. The minor number is used only by the driver itself to differentiate which device it is operating on, just in case the driver handles more than one device. -

Adding a driver to your system means registering it with the kernel. This is synonymous +

Adding a driver to your system means registering it with the kernel. This is synonymous with assigning it a major number during the module’s initialization. You do this by using the register_chrdev function, defined by include/linux/fs.h.

1int register_chrdev(unsigned int major, const char *name, struct file_operations *fops);
-

Where unsigned int major is the major number you want to request, +

Where unsigned int major is the major number you want to request, const char *name is the name of the device as it will appear in /proc/devices and struct file_operations *fops @@ -1467,13 +1468,13 @@ registration failed. Note that we didn’t pass the minor number to register_chrdev . That is because the kernel doesn’t care about the minor number; only our driver uses it. -

Now the question is, how do you get a major number without hijacking +

Now the question is, how do you get a major number without hijacking one that’s already in use? The easiest way would be to look through Documentation/admin-guide/devices.txt and pick an unused one. That is a bad way of doing things because you will never be sure if the number you picked will be assigned later. The answer is that you can ask the kernel to assign you a dynamic major number. -

If you pass a major number of 0 to register_chrdev +

If you pass a major number of 0 to register_chrdev , the return value will be the dynamically allocated major number. The downside is that you can not make a device file in advance, since you do not @@ -1490,11 +1491,11 @@ third method is that we can have our driver make the device file using the device_destroy during the call to cleanup_module . -

However, register_chrdev() +

However, register_chrdev() would occupy a range of minor numbers associated with the given major. The recommended way to reduce waste for char device registration is using cdev interface. -

The newer interface completes the char device registration in two distinct steps. +

The newer interface completes the char device registration in two distinct steps. First, we should register a range of device numbers, which can be completed with register_chrdev_region or alloc_chrdev_region @@ -1503,12 +1504,12 @@ First, we should register a range of device numbers, which can be completed with

1int register_chrdev_region(dev_t from, unsigned count, const char *name); 
 2int alloc_chrdev_region(dev_t *dev, unsigned baseminor, unsigned count, const char *name);
-

The choose of two different functions depend on whether you know the major numbers for your +

The choose of two different functions depend on whether you know the major numbers for your device. Using register_chrdev_region if you know the device major number and alloc_chrdev_region if you would like to allocate a dynamicly-allocated major number. -

Second, we should initialize the data structure +

Second, we should initialize the data structure struct cdev for our char device and associate it with the device numbers. To initialize the struct cdev @@ -1517,7 +1518,7 @@ device. Using register_chrdev_region

1struct cdev *my_dev = cdev_alloc(); 
 2my_cdev->ops = &my_fops;
-

However, the common usage pattern will embed the +

However, the common usage pattern will embed the struct cdev within a device-specific structure of your own. In this case, we’ll need cdev_init @@ -1528,18 +1529,18 @@ device. Using register_chrdev_region -

Once we finish the initialization, we can add the char device to the system by using +

Once we finish the initialization, we can add the char device to the system by using the cdev_add .

1int cdev_add(struct cdev *p, dev_t dev, unsigned count);
-

To find a example using the interface, you can see ioctl.c described in section +

To find a example using the interface, you can see ioctl.c described in section 9. -

+

6.4 Unregistering A Device

-

We can not allow the kernel module to be +

We can not allow the kernel module to be rmmod ’ed whenever root feels like it. If the device file is opened by a process and then we remove the kernel module, using the file would cause a call to the memory location @@ -1549,7 +1550,7 @@ unlucky, another kernel module was loaded into the same location, which means a jump into the middle of another function within the kernel. The results of this would be impossible to predict, but they can not be very positive. -

Normally, when you do not want to allow something, you return an error code +

Normally, when you do not want to allow something, you return an error code (a negative number) from the function which is supposed to do it. With cleanup_module that’s impossible because it is a void function. However, there is a counter @@ -1575,26 +1576,26 @@ decrease and display this counter:

  • module_refcount(THIS_MODULE) : Return the value of reference count of current module.
  • -

    It is important to keep the counter accurate; if you ever do lose track of the +

    It is important to keep the counter accurate; if you ever do lose track of the correct usage count, you will never be able to unload the module; it’s now reboot time, boys and girls. This is bound to happen to you sooner or later during a module’s development. -

    +

    6.5 chardev.c

    -

    The next code sample creates a char driver named chardev. You can dump its device +

    The next code sample creates a char driver named chardev. You can dump its device file.

    1cat /proc/devices
    -

    (or open the file with a program) and the driver will put the number of times the +

    (or open the file with a program) and the driver will put the number of times the device file has been read from into the file. We do not support writing to the file (like echo "hi" > /dev/hello ), but catch these attempts and tell the user that the operation is not supported. Don’t worry if you don’t see what we do with the data we read into the buffer; we don’t do much with it. We simply read in the data and print a message acknowledging that we received it. -

    In the multiple-threaded environment, without any protection, concurrent access +

    In the multiple-threaded environment, without any protection, concurrent access to the same memory may lead to the race condition, and will not preserve the performance. In the kernel module, this problem may happen due to multiple instances accessing the shared resources. Therefore, a solution is to enforce the @@ -1768,32 +1769,32 @@ concurrency details in the 12

    +

    6.6 Writing Modules for Multiple Kernel Versions

    -

    The system calls, which are the major interface the kernel shows to the processes, +

    The system calls, which are the major interface the kernel shows to the processes, generally stay the same across versions. A new system call may be added, but usually the old ones will behave exactly like they used to. This is necessary for backward compatibility – a new kernel version is not supposed to break regular processes. In most cases, the device files will also remain the same. On the other hand, the internal interfaces within the kernel can and do change between versions. -

    There are differences between different kernel versions, and if you want +

    There are differences between different kernel versions, and if you want to support multiple kernel versions, you will find yourself having to code conditional compilation directives. The way to do this to compare the macro LINUX_VERSION_CODE to the macro KERNEL_VERSION . In version a.b.c of the kernel, the value of this macro would be 216a+ 28b+ c  . -

    +

    7 The /proc File System

    -

    In Linux, there is an additional mechanism for the kernel and kernel modules to send +

    In Linux, there is an additional mechanism for the kernel and kernel modules to send information to processes — the /proc file system. Originally designed to allow easy access to information about processes (hence the name), it is now used by every bit of the kernel which has something interesting to report, such as /proc/modules which provides the list of modules and /proc/meminfo which gathers memory usage statistics. -

    The method to use the proc file system is very similar to the one used with device +

    The method to use the proc file system is very similar to the one used with device drivers — a structure is created with all the information needed for the /proc file, including pointers to any handler functions (in our case there is only one, the one called when somebody attempts to read from the /proc file). Then, @@ -1804,18 +1805,18 @@ one called when somebody attempts to read from the

    Normal file systems are located on a disk, rather than just in memory (which is +

    Normal file systems are located on a disk, rather than just in memory (which is where /proc is), and in that case the index-node (inode for short) number is a pointer to a disk location where the file’s inode is located. The inode contains information about the file, for example the file’s permissions, together with a pointer to the disk location or locations where the file’s data can be found. -

    Because we don’t get called when the file is opened or closed, there’s nowhere for +

    Because we don’t get called when the file is opened or closed, there’s nowhere for us to put try_module_get and module_put in this module, and if the file is opened and then the module is removed, there’s no way to avoid the consequences. -

    Here a simple example showing how to use a /proc file. This is the HelloWorld for +

    Here a simple example showing how to use a /proc file. This is the HelloWorld for the /proc filesystem. There are three parts: create the file /proc/helloworld in the function init_module , return a value (and a buffer) when the file /proc/helloworld is read in the callback @@ -1823,12 +1824,12 @@ function procfile_read , and delete the file /proc/helloworld in the function cleanup_module . -

    The /proc/helloworld is created when the module is loaded with the function +

    The /proc/helloworld is created when the module is loaded with the function proc_create . The return value is a struct proc_dir_entry , and it will be used to configure the file /proc/helloworld (for example, the owner of this file). A null return value means that the creation has failed. -

    Every time the file /proc/helloworld is read, the function +

    Every time the file /proc/helloworld is read, the function procfile_read is called. Two parameters of this function are very important: the buffer (the second parameter) and the offset (the fourth one). The content of the @@ -1845,7 +1846,7 @@ function, if it never returns zero, the read function is called endlessly. $ cat /proc/helloworld HelloWorld! -

    +

    1/* 
    @@ -1917,10 +1918,10 @@ HelloWorld!
     67module_exit(procfs1_exit); 
     68 
     69MODULE_LICENSE("GPL");
    -

    +

    7.1 The proc_ops Structure

    -

    The proc_ops +

    The proc_ops structure is defined in include/linux/proc_fs.h in Linux v5.6+. In older kernels, it used file_operations for custom hooks in /proc file system, but it contains some @@ -1932,10 +1933,10 @@ performance. For example, the file which never disappears in proc_flag as PROC_ENTRY_PERMANENT to save 2 atomic ops, 1 allocation, 1 free in per open/read/close sequence. -

    +

    7.2 Read and Write a /proc File

    -

    We have seen a very simple example for a /proc file where we only read +

    We have seen a very simple example for a /proc file where we only read the file /proc/helloworld. It is also possible to write in a /proc file. It works the same way as read, a function is called when the /proc file is written. But there is a little difference with read, data comes from @@ -1943,7 +1944,7 @@ user, so you have to import data from user space to kernel space (with copy_from_user or get_user ) -

    The reason for copy_from_user +

    The reason for copy_from_user or get_user is that Linux memory (on Intel architecture, it may be different under some @@ -1954,7 +1955,7 @@ not reference a unique location in memory, only a location in a memory segment, and you need to know which memory segment it is to be able to use it. There is one memory segment for the kernel, and one for each of the processes. -

    The only memory segment accessible to a process is its own, so when +

    The only memory segment accessible to a process is its own, so when writing regular programs to run as processes, there is no need to worry about segments. When you write a kernel module, normally you want to access the kernel memory segment, which is handled automatically by the system. @@ -2069,22 +2070,22 @@ because data is already in kernel space. 95module_exit(procfs2_exit); 96 97MODULE_LICENSE("GPL"); -

    +

    7.3 Manage /proc file with standard filesystem

    -

    We have seen how to read and write a /proc file with the /proc interface. But it is +

    We have seen how to read and write a /proc file with the /proc interface. But it is also possible to manage /proc file with inodes. The main concern is to use advanced functions, like permissions. -

    In Linux, there is a standard mechanism for file system registration. +

    In Linux, there is a standard mechanism for file system registration. Since every file system has to have its own functions to handle inode and file operations, there is a special structure to hold pointers to all those functions, struct inode_operations , which includes a pointer to struct proc_ops . -

    The difference between file and inode operations is that file operations deal with +

    The difference between file and inode operations is that file operations deal with the file itself whereas inode operations deal with ways of referencing the file, such as creating links to it. -

    In /proc, whenever we register a new file, we’re allowed to specify which +

    In /proc, whenever we register a new file, we’re allowed to specify which struct inode_operations will be used to access to it. This is the mechanism we use, a struct inode_operations @@ -2095,7 +2096,7 @@ creating links to it. which includes pointers to our procf_read and procfs_write functions. -

    Another interesting point here is the +

    Another interesting point here is the module_permission function. This function is called whenever a process tries to do something with the /proc file, and it can decide whether to allow access or not. Right now it is only @@ -2104,7 +2105,7 @@ pointer to a structure which includes information on the currently running process), but it could be based on anything we like, such as what other processes are doing with the same file, the time of day, or the last input we received. -

    It is important to note that the standard roles of read and write are reversed in +

    It is important to note that the standard roles of read and write are reversed in the kernel. Read functions are used for output, whereas write functions are used for input. The reason for that is that read and write refer to the user’s point of view — if a process reads something from the kernel, then the kernel needs to output it, and @@ -2219,14 +2220,14 @@ input. 105module_exit(procfs3_exit); 106 107MODULE_LICENSE("GPL"); -

    Still hungry for procfs examples? Well, first of all keep in mind, there are rumors +

    Still hungry for procfs examples? Well, first of all keep in mind, there are rumors around, claiming that procfs is on its way out, consider using sysfs instead. Consider using this mechanism, in case you want to document something kernel related yourself. -

    +

    7.4 Manage /proc file with seq_file

    -

    As we have seen, writing a /proc file may be quite “complex”. +

    As we have seen, writing a /proc file may be quite “complex”. So to help people writting /proc file, there is an API named seq_file that helps formating a /proc file for output. It is based on sequence, which is composed of @@ -2235,7 +2236,7 @@ So to help people writting , and stop() . The seq_file API starts a sequence when a user read the /proc file. -

    A sequence begins with the call of the function +

    A sequence begins with the call of the function start() . If the return is a non NULL value, the function next() @@ -2252,7 +2253,7 @@ time next() returns NULL , then the function stop() is called. -

    BE CAREFUL: when a sequence is finished, another one starts. That means that at the end +

    BE CAREFUL: when a sequence is finished, another one starts. That means that at the end of function stop() , the function start() is called again. This loop finishes when the function @@ -2269,14 +2270,14 @@ of function stop() -

    srYrsNNYtaeenetoooertusetupstrxr((ntn))( tis)istrr teeaNreNatUaUtmLtLmeLmLen?e?ntntt  +

    srYrsNNYtaeenetoooertusetupstrxr((ntn))( tis)istrr teeaNreNatUaUtmLtLmeLmLen?e?ntntt

    Figure 1:How seq_file works
    -

    The seq_file +

    The seq_file provides basic functions for proc_ops , such as seq_read , seq_lseek @@ -2401,26 +2402,26 @@ the same way as in the previous example. 116module_exit(procfs4_exit); 117 118MODULE_LICENSE("GPL"); -

    If you want more information, you can read this web page: +

    If you want more information, you can read this web page:

    -

    You can also read the code of fs/seq_file.c in the linux kernel. +

    You can also read the code of fs/seq_file.c in the linux kernel.

    8 sysfs: Interacting with your module

    -

    sysfs allows you to interact with the running kernel from userspace by reading or +

    sysfs allows you to interact with the running kernel from userspace by reading or setting variables inside of modules. This can be useful for debugging purposes, or just as an interface for applications or scripts. You can find sysfs directories and files under the /sys directory on your system.

    1ls -l /sys
    -

    Attributes can be exported for kobjects in the form of regular files in the +

    Attributes can be exported for kobjects in the form of regular files in the filesystem. Sysfs forwards file I/O operations to methods defined for the attributes, providing a means to read and write kernel attributes. -

    An attribute definition in simply: +

    An attribute definition in simply:

    1struct attribute { 
    @@ -2431,7 +2432,7 @@ providing a means to read and write kernel attributes.
     6 
     7int sysfs_create_file(struct kobject * kobj, const struct attribute * attr); 
     8void sysfs_remove_file(struct kobject * kobj, const struct attribute * attr);
    -

    For example, the driver model defines +

    For example, the driver model defines struct device_attribute like:

    @@ -2449,7 +2450,7 @@ providing a means to read and write kernel attributes. 8 9int device_create_file(struct device *, const struct device_attribute *); 10void device_remove_file(struct device *, const struct device_attribute *); -

    To read or write attributes, show() +

    To read or write attributes, show() or store() method must be specified when declaring the attribute. For the common cases include/linux/sysfs.h provides convenience macros @@ -2458,7 +2459,7 @@ common cases __ATTR_WO , etc.) to make defining attributes easier as well as making code more concise and readable. -

    An example of a hello world module which includes the creation of a variable +

    An example of a hello world module which includes the creation of a variable accessible via sysfs is given below.

    @@ -2523,34 +2524,34 @@ accessible via sysfs is given below. 59module_exit(mymodule_exit); 60 61MODULE_LICENSE("GPL"); -

    Make and install the module: +

    Make and install the module:

    1make 
     2sudo insmod hello-sysfs.ko
    -

    Check that it exists: +

    Check that it exists:

    1sudo lsmod | grep hello_sysfs
    -

    What is the current value of myvariable +

    What is the current value of myvariable ?

    1cat /sys/kernel/mymodule/myvariable
    -

    Set the value of myvariable +

    Set the value of myvariable and check that it changed.

    1echo "32" > /sys/kernel/mymodule/myvariable 
     2cat /sys/kernel/mymodule/myvariable
    -

    Finally, remove the test module: +

    Finally, remove the test module:

    1sudo rmmod hello_sysfs
    -

    In the above case, we use a simple kobject to create a directory under +

    In the above case, we use a simple kobject to create a directory under sysfs, and communicate with its attributes. Since Linux v2.6.0, the kobject structure made its appearance. It was initially meant as a simple way of @@ -2559,17 +2560,17 @@ bit of mission creep, it is now the glue that holds much of the device model and its sysfs interface together. For more information about kobject and sysfs, see Documentation/driver-api/driver-model/driver.rst and https://lwn.net/Articles/51437/. -

    +

    9 Talking To Device Files

    -

    Device files are supposed to represent physical devices. Most physical devices are +

    Device files are supposed to represent physical devices. Most physical devices are used for output as well as input, so there has to be some mechanism for device drivers in the kernel to get the output to send to the device from processes. This is done by opening the device file for output and writing to it, just like writing to a file. In the following example, this is implemented by device_write . -

    This is not always enough. Imagine you had a serial port connected to a modem +

    This is not always enough. Imagine you had a serial port connected to a modem (even if you have an internal modem, it is still implemented from the CPU’s perspective as a serial port connected to a modem, so you don’t have to tax your imagination too hard). The natural thing to do would be to use the @@ -2579,7 +2580,7 @@ responses for commands or the data received through the phone line). However, this leaves open the question of what to do when you need to talk to the serial port itself, for example to configure the rate at which data is sent and received. -

    The answer in Unix is to use a special function called +

    The answer in Unix is to use a special function called ioctl (short for Input Output ConTroL). Every device can have its own ioctl @@ -2588,7 +2589,7 @@ kernel), write ioctl’s (to return information to a process), both or neither. here the roles of read and write are reversed again, so in ioctl’s read is to send information to the kernel and write is to receive information from the kernel. -

    The ioctl function is called with three parameters: the file descriptor of the +

    The ioctl function is called with three parameters: the file descriptor of the appropriate device file, the ioctl number, and a parameter, which is of type long so you can use a cast to use it to pass anything. You will not be able to pass a structure this way, but you will be able to pass a pointer to the structure. Here is an @@ -2788,7 +2789,7 @@ example: 188 189MODULE_LICENSE("GPL"); 190MODULE_DESCRIPTION("This is test_ioctl module"); -

    You can see there is an argument called +

    You can see there is an argument called cmd in test_ioctl_ioctl() function. It is the ioctl number. The ioctl number encodes the major @@ -2803,11 +2804,11 @@ included both by the programs which will use ioctl (so they can generate the appropriate ioctl’s) and by the kernel module (so it can understand it). In the example below, the header file is chardev.h and the program which uses it is userspace_ioctl.c. -

    If you want to use ioctls in your own kernel modules, it is best to receive an +

    If you want to use ioctls in your own kernel modules, it is best to receive an official ioctl assignment, so if you accidentally get somebody else’s ioctls, or if they get yours, you’ll know something is wrong. For more information, consult the kernel source tree at Documentation/userspace-api/ioctl/ioctl-number.rst. -

    Also, we need to be careful that concurrent access to the shared resources will +

    Also, we need to be careful that concurrent access to the shared resources will lead to the race condition. The solution is using atomic Compare-And-Swap (CAS), which we mentioned at 6.5 section, to enforce the exclusive access.

    @@ -3197,10 +3198,10 @@ which we mentioned at 6.5 101    close(file_desc); 102    exit(EXIT_FAILURE); 103} -

    +

    10 System Calls

    -

    So far, the only thing we’ve done was to use well defined kernel mechanisms to +

    So far, the only thing we’ve done was to use well defined kernel mechanisms to register /proc files and device handlers. This is fine if you want to do something the kernel programmers thought you’d want, such as write a device driver. But what if @@ -3208,7 +3209,7 @@ kernel programmers thought you’d want, such as write a device driver. But what you want to do something unusual, to change the behavior of the system in some way? Then, you are mostly on your own. -

    If you are not being sensible and using a virtual machine then this is where kernel +

    If you are not being sensible and using a virtual machine then this is where kernel programming can become hazardous. While writing the example below, I killed the open() system call. This meant I could not open any files, I could not run any @@ -3220,7 +3221,7 @@ ensure you do not lose any files, even within a test environment, please run right before you do the insmod and the rmmod . -

    Forget about /proc files, forget about device files. They are just minor details. +

    Forget about /proc files, forget about device files. They are just minor details. Minutiae in the vast expanse of the universe. The real process to kernel communication mechanism, the one used by all processes, is system calls. When a process requests a service from the kernel (such as opening a file, forking to a new @@ -3229,11 +3230,11 @@ change the behaviour of the kernel in interesting ways, this is the place to do it. By the way, if you want to see which system calls a program uses, run strace <arguments> . -

    In general, a process is not supposed to be able to access the kernel. It can not +

    In general, a process is not supposed to be able to access the kernel. It can not access kernel memory and it can’t call kernel functions. The hardware of the CPU enforces this (that is the reason why it is called “protected mode” or “page protection”). -

    System calls are an exception to this general rule. What happens is that the +

    System calls are an exception to this general rule. What happens is that the process fills the registers with the appropriate values and then calls a special instruction which jumps to a previously defined location in the kernel (of course, that location is readable by user processes, it is not writable by them). Under Intel CPUs, @@ -3241,7 +3242,7 @@ this is done by means of interrupt 0x80. The hardware knows that once you jump t this location, you are no longer running in restricted user mode, but as the operating system kernel — and therefore you’re allowed to do whatever you want. -

    The location in the kernel a process can jump to is called system_call. The +

    The location in the kernel a process can jump to is called system_call. The procedure at that location checks the system call number, which tells the kernel what service the process requested. Then, it looks at the table of system calls ( sys_call_table @@ -3251,7 +3252,7 @@ different process, if the process time ran out). If you want to read this code, at the source file arch/$(architecture)/kernel/entry.S, after the line ENTRY(system_call) . -

    So, if we want to change the way a certain system call works, what we need to do +

    So, if we want to change the way a certain system call works, what we need to do @@ -3262,7 +3263,7 @@ code, and then calling the original function) and then change the pointer at don’t want to leave the system in an unstable state, it’s important for cleanup_module to restore the table to its original state. -

    To modify the content of sys_call_table +

    To modify the content of sys_call_table , we need to consider the control register. A control register is a processor register that changes or controls the general behavior of the CPU. For x86 architecture, the cr0 register has various control flags that modify the basic @@ -3275,11 +3276,11 @@ read-only sections Therefore, we must disable the

    However, sys_call_table +

    However, sys_call_table symbol is unexported to prevent misuse. But there have few ways to get the symbol, manual symbol lookup and kallsyms_lookup_name . Here we use both depend on the kernel version. -

    Because of the control-flow integrity, which is a technique to prevent the redirect +

    Because of the control-flow integrity, which is a technique to prevent the redirect execution code from the attacker, for making sure that the indirect calls go to the expected addresses and the return addresses are not changed. Since Linux v5.7, the kernel patched the series of control-flow enforcement (CET) for x86, and some @@ -3304,10 +3305,10 @@ COLLECT_GCC_OPTIONS='-v' '-Q' '-O2' '--help=target' '-mtune=generic' '-marc  GNU C17 (Ubuntu 9.3.0-17ubuntu1~20.04) version 9.3.0 (x86_64-linux-gnu) ... -

    But CET should not be enabled in the kernel, it may break the Kprobes and bpf. +

    But CET should not be enabled in the kernel, it may break the Kprobes and bpf. Consequently, CET is disabled since v5.11. To guarantee the manual symbol lookup worked, we only use up to v5.4. -

    Unfortunately, since Linux v5.7 kallsyms_lookup_name +

    Unfortunately, since Linux v5.7 kallsyms_lookup_name is also unexported, it needs certain trick to get the address of kallsyms_lookup_name . If CONFIG_KPROBES @@ -3319,7 +3320,7 @@ passes the addresses of the saved registers and the Kprobe struct to the handler you defined, then executes it. Kprobes can be registered by symbol name or address. Within the symbol name, the address will be handled by the kernel. -

    Otherwise, specify the address of sys_call_table +

    Otherwise, specify the address of sys_call_table from /proc/kallsyms and /boot/System.map into sym parameter. Following is the sample usage for /proc/kallsyms: @@ -3334,8 +3335,8 @@ ffffffff820013a0 R sys_call_table ffffffff820023e0 R ia32_sys_call_table $ sudo insmod syscall.ko sym=0xffffffff820013a0 -

    -

    Using the address from /boot/System.map, be careful about KASLR (Kernel +

    +

    Using the address from /boot/System.map, be careful about KASLR (Kernel Address Space Layout Randomization). KASLR may randomize the address of kernel code and data at every boot time, such as the static address listed in /boot/System.map will offset by some entropy. The purpose of KASLR is to protect @@ -3364,7 +3365,7 @@ ffffffff82000300 R sys_call_table $ sudo grep sys_call_table /proc/kallsyms ffffffff86400300 R sys_call_table -

    If KASLR is enabled, we have to take care of the address from /proc/kallsyms each +

    If KASLR is enabled, we have to take care of the address from /proc/kallsyms each time we reboot the machine. In order to use the address from /boot/System.map, make sure that KASLR is disabled. You can add the nokaslr for disabling KASLR in next booting time: @@ -3380,8 +3381,8 @@ $ grep quiet /etc/default/grub GRUB_CMDLINE_LINUX_DEFAULT="quiet nokaslr splash" $ sudo update-grub -

    -

    For more information, check out the following: +

    +

    For more information, check out the following:

    -

    The source code here is an example of such a kernel module. We want to “spy” on a certain +

    The source code here is an example of such a kernel module. We want to “spy” on a certain user, and to pr_info() a message whenever that user opens a file. Towards this end, we replace the system call to open a file with our own function, called @@ -3408,7 +3409,7 @@ spy on, it calls pr_info() to display the name of the file to be opened. Then, either way, it calls the original open() function with the same parameters, to actually open the file. -

    The init_module +

    The init_module function replaces the appropriate location in sys_call_table and keeps the original pointer in a variable. The @@ -3426,7 +3427,7 @@ with B_open , which will call what it thinks is the original system call, A_open , when it’s done. -

    Now, if B is removed first, everything will be well — it will simply restore the system +

    Now, if B is removed first, everything will be well — it will simply restore the system call to A_open , which calls the original. However, if A is removed and then B is removed, the system will crash. A’s removal will restore the system call to the original, @@ -3446,7 +3447,7 @@ problem. When A is removed, it sees that the system call was changed to will still try to call A_open which is no longer there, so that even without removing B the system would crash. -

    Note that all the related problems make syscall stealing unfeasible for +

    Note that all the related problems make syscall stealing unfeasible for production use. In order to keep people from doing potential harmful things sys_call_table is no longer exported. This means, if you want to do something more than a mere @@ -3687,13 +3688,13 @@ dry run of this example, you will have to patch your current kernel in order to 227module_exit(syscall_end); 228 229MODULE_LICENSE("GPL"); -

    +

    11 Blocking Processes and threads

    -

    +

    11.1 Sleep

    -

    What do you do when somebody asks you for something you can not do right +

    What do you do when somebody asks you for something you can not do right away? If you are a human being and you are bothered by a human being, the only thing you can say is: "Not right now, I’m busy. Go away!". But if you are a kernel module and you are bothered by a process, you have another @@ -3701,21 +3702,21 @@ possibility. You can put the process to sleep until you can service it. After al processes are being put to sleep by the kernel and woken up all the time (that is the way multiple processes appear to run on the same time on a single CPU). -

    This kernel module is an example of this. The file (called /proc/sleep) can only +

    This kernel module is an example of this. The file (called /proc/sleep) can only be opened by a single process at a time. If the file is already open, the kernel module calls wait_event_interruptible . The easiest way to keep a file open is to open it with:

    1tail -f
    -

    This function changes the status of the task (a task is the kernel data structure +

    This function changes the status of the task (a task is the kernel data structure which holds information about a process and the system call it is in, if any) to TASK_INTERRUPTIBLE , which means that the task will not run until it is woken up somehow, and adds it to WaitQ, the queue of tasks waiting to access the file. Then, the function calls the scheduler to context switch to a different process, one which has some use for the CPU. -

    When a process is done with the file, it closes it, and +

    When a process is done with the file, it closes it, and module_close is called. That function wakes up all the processes in the queue (there’s no mechanism to only wake up one of them). It then returns and the process which just @@ -3728,31 +3729,31 @@ Eventually, one of the processes which was in the queue will be given control of the CPU by the scheduler. It starts at the point right after the call to module_interruptible_sleep_on . -

    This means that the process is still in kernel mode - as far as the process +

    This means that the process is still in kernel mode - as far as the process is concerned, it issued the open system call and the system call has not returned yet. The process does not know somebody else used the CPU for most of the time between the moment it issued the call and the moment it returned. -

    It can then proceed to set a global variable to tell all the other processes that the +

    It can then proceed to set a global variable to tell all the other processes that the file is still open and go on with its life. When the other processes get a piece of the CPU, they’ll see that global variable and go back to sleep. -

    So we will use tail -f +

    So we will use tail -f to keep the file open in the background, while trying to access it with another process (again in the background, so that we need not switch to a different vt). As soon as the first background process is killed with kill %1 , the second is woken up, is able to access the file and finally terminates. -

    To make our life more interesting, module_close +

    To make our life more interesting, module_close does not have a monopoly on waking up the processes which wait to access the file. A signal, such as Ctrl +c (SIGINT) can also wake up a process. This is because we used module_interruptible_sleep_on . We could have used module_sleep_on instead, but that would have resulted in extremely angry users whose Ctrl+c’s are ignored. -

    In that case, we want to return with +

    In that case, we want to return with -EINTR immediately. This is important so users can, for example, kill the process before it receives the file. -

    There is one more point to remember. Some times processes don’t want to sleep, they want +

    There is one more point to remember. Some times processes don’t want to sleep, they want either to get what they want immediately, or to be told it cannot be done. Such processes use the O_NONBLOCK flag when opening the file. The kernel is supposed to respond by returning with the error @@ -3788,7 +3789,7 @@ $ cat_nonblock /proc/sleep Last input: $ -

    +

    1/* 
    @@ -4067,14 +4068,14 @@ $
     57 
     58    return 0; 
     59}
    -

    +

    11.2 Completions

    -

    Sometimes one thing should happen before another within a module having multiple threads. +

    Sometimes one thing should happen before another within a module having multiple threads. Rather than using /bin/sleep commands, the kernel has another way to do this which allows timeouts or interrupts to also happen. -

    In the following example two threads are started, but one needs to start before +

    In the following example two threads are started, but one needs to start before another.

    @@ -4157,31 +4158,31 @@ another. 74 75MODULE_DESCRIPTION("Completions example"); 76MODULE_LICENSE("GPL"); -

    The machine +

    The machine structure stores the completion states for the two threads. At the exit point of each thread the respective completion state is updated, and wait_for_completion is used by the flywheel thread to ensure that it does not begin prematurely. -

    So even though flywheel_thread +

    So even though flywheel_thread is started first you should notice if you load this module and run dmesg that turning the crank always happens first because the flywheel thread waits for it to complete. -

    There are other variations upon the +

    There are other variations upon the wait_for_completion function, which include timeouts or being interrupted, but this basic mechanism is enough for many common situations without adding a lot of complexity. -

    +

    12 Avoiding Collisions and Deadlocks

    -

    If processes running on different CPUs or in different threads try to access the same +

    If processes running on different CPUs or in different threads try to access the same memory, then it is possible that strange things can happen or your system can lock up. To avoid this, various types of mutual exclusion kernel functions are available. These indicate if a section of code is "locked" or "unlocked" so that simultaneous attempts to run it can not happen.

    12.1 Mutex

    -

    You can use kernel mutexes (mutual exclusions) in much the same manner that you +

    You can use kernel mutexes (mutual exclusions) in much the same manner that you might deploy them in userland. This may be all that is needed to avoid collisions in most cases.

    @@ -4227,10 +4228,10 @@ most cases. 39 40MODULE_DESCRIPTION("Mutex example"); 41MODULE_LICENSE("GPL"); -

    +

    12.2 Spinlocks

    -

    As the name suggests, spinlocks lock up the CPU that the code is running on, +

    As the name suggests, spinlocks lock up the CPU that the code is running on, taking 100% of its resources. Because of this you should only use the spinlock @@ -4238,7 +4239,7 @@ taking 100% of its resources. Because of this you should only use the spinlock mechanism around code which is likely to take no more than a few milliseconds to run and so will not noticeably slow anything down from the user’s point of view. -

    The example here is "irq safe" in that if interrupts happen during the lock then +

    The example here is "irq safe" in that if interrupts happen during the lock then they will not be forgotten and will activate when the unlock happens, using the flags variable to retain their state. @@ -4307,10 +4308,10 @@ they will not be forgotten and will activate when the unlock happens, using the 61 62MODULE_DESCRIPTION("Spinlock example"); 63MODULE_LICENSE("GPL"); -

    +

    12.3 Read and write locks

    -

    Read and write locks are specialised kinds of spinlocks so that you can exclusively +

    Read and write locks are specialised kinds of spinlocks so that you can exclusively read from something or write to something. Like the earlier spinlocks example, the one below shows an "irq safe" situation in which if other functions were triggered from irqs which might also read and write to whatever you are concerned with @@ -4375,14 +4376,14 @@ module. 53 54MODULE_DESCRIPTION("Read/Write locks example"); 55MODULE_LICENSE("GPL"); -

    Of course, if you know for sure that there are no functions triggered by irqs +

    Of course, if you know for sure that there are no functions triggered by irqs which could possibly interfere with your logic then you can use the simpler read_lock(&myrwlock) and read_unlock(&myrwlock) or the corresponding write functions.

    12.4 Atomic operations

    -

    If you are doing simple arithmetic: adding, subtracting or bitwise operations, then +

    If you are doing simple arithmetic: adding, subtracting or bitwise operations, then there is another way in the multi-CPU and multi-hyperthreaded world to stop other parts of the system from messing with your mojo. By using atomic operations you can be confident that your addition, subtraction or bit flip did actually happen @@ -4467,7 +4468,7 @@ below. -

    Before the C11 standard adopts the built-in atomic types, the kernel already +

    Before the C11 standard adopts the built-in atomic types, the kernel already provided a small set of atomic types by using a bunch of tricky architecture-specific codes. Implementing the atomic types by C11 atomics may allow the kernel to throw away the architecture-specific codes and letting the kernel code be more friendly to @@ -4480,21 +4481,21 @@ For further details, see:

  • Time to move to C11 atomics?
  • Atomic usage patterns in the kernel
  • -

    +

    13 Replacing Print Macros

    -

    +

    13.1 Replacement

    -

    In Section 2, I said that X Window System and kernel module programming do not +

    In Section 2, I said that X Window System and kernel module programming do not mix. That is true for developing kernel modules. But in actual use, you want to be able to send messages to whichever tty the command to load the module came from. -

    "tty" is an abbreviation of teletype: originally a combination keyboard-printer +

    "tty" is an abbreviation of teletype: originally a combination keyboard-printer used to communicate with a Unix system, and today an abstraction for the text stream used for a Unix program, whether it is a physical terminal, an xterm on an X display, a network connection used with ssh, etc. -

    The way this is done is by using current, a pointer to the currently running task, +

    The way this is done is by using current, a pointer to the currently running task, to get the current task’s tty structure. Then, we look inside that tty structure to find a pointer to a string write function, which we use to write a string to the tty. @@ -4577,16 +4578,16 @@ tty. -

    +

    13.2 Flashing keyboard LEDs

    -

    In certain conditions, you may desire a simpler and more direct way to communicate +

    In certain conditions, you may desire a simpler and more direct way to communicate to the external world. Flashing keyboard LEDs can be such a solution: It is an immediate way to attract attention or to display a status condition. Keyboard LEDs are present on every hardware, they are always visible, they do not need any setup, and their use is rather simple and non-intrusive, compared to writing to a tty or a file. -

    From v4.14 to v4.15, the timer API made a series of changes +

    From v4.14 to v4.15, the timer API made a series of changes to improve memory safety. A buffer overflow in the area of a timer_list structure may be able to overwrite the @@ -4609,7 +4610,7 @@ Thus, it is better to use a unique prototype to separate from the cluster that t container_of macro instead of the unsigned long value. For more information see: Improving the kernel timers API. -

    Before Linux v4.14, setup_timer +

    Before Linux v4.14, setup_timer was used to initialize the timer and the timer_list structure looked like: @@ -4624,7 +4625,7 @@ Thus, it is better to use a unique prototype to separate from the cluster that t 8 9void setup_timer(struct timer_list *timer, void (*callback)(unsigned long), 10                 unsigned long data); -

    Since Linux v4.14, timer_setup +

    Since Linux v4.14, timer_setup is adopted and the kernel step by step converting to timer_setup from setup_timer @@ -4638,7 +4639,7 @@ Moreover, the timer_setup

    1void timer_setup(struct timer_list *timer, 
     2                 void (*callback)(struct timer_list *), unsigned int flags);
    -

    The setup_timer +

    The setup_timer was then removed since v4.15. As a result, the timer_list structure had changed to the following. @@ -4649,7 +4650,7 @@ Moreover, the timer_setup 4    u32 flags; 5    /* ... */ 6}; -

    The following source code illustrates a minimal kernel module which, when +

    The following source code illustrates a minimal kernel module which, when loaded, starts blinking the keyboard LEDs until it is unloaded.

    @@ -4738,7 +4739,7 @@ loaded, starts blinking the keyboard LEDs until it is unloaded. 83module_exit(kbleds_cleanup); 84 85MODULE_LICENSE("GPL"); -

    If none of the examples in this chapter fit your debugging needs, +

    If none of the examples in this chapter fit your debugging needs, there might yet be some other tricks to try. Ever wondered what CONFIG_LL_DEBUG in make menuconfig @@ -4749,25 +4750,25 @@ everything what your code does over a serial line. If you find yourself porting kernel to some new and former unsupported architecture, this is usually amongst the first things that should be implemented. Logging over a netconsole might also be worth a try. -

    While you have seen lots of stuff that can be used to aid debugging here, there are +

    While you have seen lots of stuff that can be used to aid debugging here, there are some things to be aware of. Debugging is almost always intrusive. Adding debug code can change the situation enough to make the bug seem to disappear. Thus, you should keep debug code to a minimum and make sure it does not show up in production code. -

    +

    14 Scheduling Tasks

    -

    There are two main ways of running tasks: tasklets and work queues. Tasklets are a +

    There are two main ways of running tasks: tasklets and work queues. Tasklets are a quick and easy way of scheduling a single function to be run. For example, when triggered from an interrupt, whereas work queues are more complicated but also better suited to running multiple things in a sequence. -

    +

    14.1 Tasklets

    -

    Here is an example tasklet module. The +

    Here is an example tasklet module. The tasklet_fn function runs for a few seconds. In the meantime, execution of the example_tasklet_init @@ -4819,7 +4820,7 @@ better suited to running multiple things in a sequence. 42 43MODULE_DESCRIPTION("Tasklet example"); 44MODULE_LICENSE("GPL"); -

    So with this example loaded dmesg +

    So with this example loaded dmesg should show: @@ -4831,23 +4832,23 @@ Example tasklet starts Example tasklet init continues... Example tasklet ends -

    Although tasklet is easy to use, it comes with several defators, and developers are +

    Although tasklet is easy to use, it comes with several defators, and developers are discussing about getting rid of tasklet in linux kernel. The tasklet callback runs in atomic context, inside a software interrupt, meaning that it cannot sleep or access user-space data, so not all work can be done in a tasklet handler. Also, the kernel only allows one instance of any given tasklet to be running at any given time; multiple different tasklet callbacks can run in parallel. -

    In recent kernels, tasklets can be replaced by workqueues, timers, or threaded +

    In recent kernels, tasklets can be replaced by workqueues, timers, or threaded interrupts.1 While the removal of tasklets remains a longer-term goal, the current kernel contains more than a hundred uses of tasklets. Now developers are proceeding with the API changes and the macro DECLARE_TASKLET_OLD exists for compatibility. For further information, see https://lwn.net/Articles/830964/. -

    +

    14.2 Work queues

    -

    To add a task to the scheduler we can use a workqueue. The kernel then uses the +

    To add a task to the scheduler we can use a workqueue. The kernel then uses the Completely Fair Scheduler (CFS) to execute work within the queue.

    @@ -4884,36 +4885,36 @@ Completely Fair Scheduler (CFS) to execute work within the queue. 31 32MODULE_LICENSE("GPL"); 33MODULE_DESCRIPTION("Workqueue example"); -

    +

    15 Interrupt Handlers

    -

    +

    15.1 Interrupt Handlers

    -

    Except for the last chapter, everything we did in the kernel so far we have done as a +

    Except for the last chapter, everything we did in the kernel so far we have done as a response to a process asking for it, either by dealing with a special file, sending an ioctl() , or issuing a system call. But the job of the kernel is not just to respond to process requests. Another job, which is every bit as important, is to speak to the hardware connected to the machine. -

    There are two types of interaction between the CPU and the rest of the +

    There are two types of interaction between the CPU and the rest of the computer’s hardware. The first type is when the CPU gives orders to the hardware, the order is when the hardware needs to tell the CPU something. The second, called interrupts, is much harder to implement because it has to be dealt with when convenient for the hardware, not the CPU. Hardware devices typically have a very small amount of RAM, and if you do not read their information when available, it is lost. -

    Under Linux, hardware interrupts are called IRQ’s (Interrupt ReQuests). There +

    Under Linux, hardware interrupts are called IRQ’s (Interrupt ReQuests). There are two types of IRQ’s, short and long. A short IRQ is one which is expected to take a very short period of time, during which the rest of the machine will be blocked and no other interrupts will be handled. A long IRQ is one which can take longer, and during which other interrupts may occur (but not interrupts from the same device). If at all possible, it is better to declare an interrupt handler to be long. -

    When the CPU receives an interrupt, it stops whatever it is doing (unless it is +

    When the CPU receives an interrupt, it stops whatever it is doing (unless it is processing a more important interrupt, in which case it will deal with this one only when the more important one is done), saves certain parameters on the stack and calls the interrupt handler. This means that certain things are not allowed in the @@ -4925,10 +4926,10 @@ heavy work deferred from an interrupt handler. Historically, BH (Linux naming for Bottom Halves) statistically book-keeps the deferred functions. Softirq and its higher level abstraction, Tasklet, replace BH since Linux 2.3. -

    The way to implement this is to call +

    The way to implement this is to call request_irq() to get your interrupt handler called when the relevant IRQ is received. -

    In practice IRQ handling can be a bit more complex. Hardware is often +

    In practice IRQ handling can be a bit more complex. Hardware is often designed in a way that chains two interrupt controllers, so that all the IRQs from interrupt controller B are cascaded to a certain IRQ from interrupt controller A. Of course, that requires that the kernel finds out which IRQ it @@ -4945,7 +4946,7 @@ need to solve another truckload of problems. It is not enough to know if a certain IRQs has happened, it’s also important to know what CPU(s) it was for. People still interested in more details, might want to refer to "APIC" now. -

    This function receives the IRQ number, the name of the function, +

    This function receives the IRQ number, the name of the function, flags, a name for /proc/interrupts and a parameter to be passed to the interrupt handler. Usually there is a certain number of IRQs available. How many IRQs there are is hardware-dependent. The flags can include @@ -4955,16 +4956,16 @@ How many IRQs there are is hardware-dependent. The flags can include SA_INTERRUPT to indicate this is a fast interrupt. This function will only succeed if there is not already a handler on this IRQ, or if you are both willing to share. -

    +

    15.2 Detecting button presses

    -

    Many popular single board computers, such as Raspberry Pi or Beagleboards, have a +

    Many popular single board computers, such as Raspberry Pi or Beagleboards, have a bunch of GPIO pins. Attaching buttons to those and then having a button press do something is a classic case in which you might need to use interrupts, so that instead of having the CPU waste time and battery power polling for a change in input state, it is better for the input to trigger the CPU to then run a particular handling function. -

    Here is an example where buttons are connected to GPIO numbers 17 and 18 and +

    Here is an example where buttons are connected to GPIO numbers 17 and 18 and an LED is connected to GPIO 4. You can change those numbers to whatever is appropriate for your board.

    @@ -5113,14 +5114,14 @@ appropriate for your board. 142 143MODULE_LICENSE("GPL"); 144MODULE_DESCRIPTION("Handle some GPIO interrupts"); -

    +

    15.3 Bottom Half

    -

    Suppose you want to do a bunch of stuff inside of an interrupt routine. A common +

    Suppose you want to do a bunch of stuff inside of an interrupt routine. A common way to do that without rendering the interrupt unavailable for a significant duration is to combine it with a tasklet. This pushes the bulk of the work off into the scheduler. -

    The example below modifies the previous example to also run an additional task +

    The example below modifies the previous example to also run an additional task when an interrupt is triggered.

    @@ -5294,19 +5295,19 @@ when an interrupt is triggered. 165 166MODULE_LICENSE("GPL"); 167MODULE_DESCRIPTION("Interrupt with top and bottom half"); -

    +

    16 Crypto

    -

    At the dawn of the internet, everybody trusted everybody completely…but that did +

    At the dawn of the internet, everybody trusted everybody completely…but that did not work out so well. When this guide was originally written, it was a more innocent era in which almost nobody actually gave a damn about crypto - least of all kernel developers. That is certainly no longer the case now. To handle crypto stuff, the kernel has its own API enabling common methods of encryption, decryption and your favourite hash functions. -

    +

    16.1 Hash functions

    -

    Calculating and checking the hashes of things is a common operation. Here is a +

    Calculating and checking the hashes of things is a common operation. Here is a demonstration of how to calculate a sha256 hash within a kernel module.

    @@ -5374,20 +5375,20 @@ demonstration of how to calculate a sha256 hash within a kernel module. 62 63MODULE_DESCRIPTION("sha256 hash test"); 64MODULE_LICENSE("GPL"); -

    Install the module: +

    Install the module:

    1sudo insmod cryptosha256.ko 
     2sudo dmesg
    -

    And you should see that the hash was calculated for the test string. -

    Finally, remove the test module: +

    And you should see that the hash was calculated for the test string. +

    Finally, remove the test module:

    1sudo rmmod cryptosha256
    -

    +

    16.2 Symmetric key encryption

    -

    Here is an example of symmetrically encrypting a string using the AES algorithm +

    Here is an example of symmetrically encrypting a string using the AES algorithm and a password.

    @@ -5592,10 +5593,10 @@ and a password. 196 197MODULE_DESCRIPTION("Symmetric key encryption example"); 198MODULE_LICENSE("GPL"); -

    +

    17 Virtual Input Device Driver

    -

    The input device driver is a module that provides a way to communicate +

    The input device driver is a module that provides a way to communicate with the interaction device via the event. For example, the keyboard can send the press or release event to tell the kernel what we want to do. The input device driver will allocate a new input structure with @@ -5603,7 +5604,7 @@ do. The input device driver will allocate a new input structure with and sets up input bitfields, device id, version, etc. After that, registers it by calling input_register_device() . -

    Here is an example, vinput, It is an API to allow easy +

    Here is an example, vinput, It is an API to allow easy development of virtual input drivers. The drivers needs to export a vinput_device() that contains the virtual device name and @@ -5619,7 +5620,7 @@ development of virtual input drivers. The drivers needs to export a

  • the readback function: read()
  • -

    Then using vinput_register_device() +

    Then using vinput_register_device() and vinput_unregister_device() will add a new device to the list of support virtual input devices.

    @@ -5628,7 +5629,7 @@ development of virtual input drivers. The drivers needs to export a

    1int init(struct vinput *);
    -

    This function is passed a struct vinput +

    This function is passed a struct vinput already initialized with an allocated struct input_dev . The init() function is responsible for initializing the capabilities of the input device and register @@ -5636,20 +5637,20 @@ it.

    1int send(struct vinput *, char *, int);
    -

    This function will receive a user string to interpret and inject the event using the +

    This function will receive a user string to interpret and inject the event using the input_report_XXXX or input_event call. The string is already copied from user.

    1int read(struct vinput *, char *, int);
    -

    This function is used for debugging and should fill the buffer parameter with the +

    This function is used for debugging and should fill the buffer parameter with the last event sent in the virtual input device format. The buffer will then be copied to user. -

    vinput devices are created and destroyed using sysfs. And, event injection is done +

    vinput devices are created and destroyed using sysfs. And, event injection is done through a /dev node. The device name will be used by the userland to export a new virtual input device. -

    The class_attribute +

    The class_attribute structure is similar to other attribute types we talked about in section 8:

    @@ -5660,7 +5661,7 @@ virtual input device. 5    ssize_t (*store)(struct class *class, struct class_attribute *attr, 6                    const char *buf, size_t count); 7}; -

    In vinput.c, the macro CLASS_ATTR_WO(export/unexport) +

    In vinput.c, the macro CLASS_ATTR_WO(export/unexport) defined in include/linux/device.h (in this case, device.h is included in include/linux/input.h) will generate the class_attribute structures which are named class_attr_export/unexport. Then, put them into @@ -5670,14 +5671,14 @@ will generate the class_attribute that should be assigned in vinput_class . Finally, call class_register(&vinput_class) to create attributes in sysfs. -

    To create a vinputX sysfs entry and /dev node. +

    To create a vinputX sysfs entry and /dev node.

    1echo "vkbd" | sudo tee /sys/class/vinput/export
    -

    To unexport the device, just echo its id in unexport: +

    To unexport the device, just echo its id in unexport:

    1echo "0" | sudo tee /sys/class/vinput/unexport
    @@ -6138,7 +6139,7 @@ will generate the class_attribute 400 401MODULE_LICENSE("GPL"); 402MODULE_DESCRIPTION("Emulate input events"); -

    Here the virtual keyboard is one of example to use vinput. It supports all +

    Here the virtual keyboard is one of example to use vinput. It supports all KEY_MAX keycodes. The injection format is the KEY_CODE such as defined in include/linux/input.h. A positive value means @@ -6146,12 +6147,12 @@ will generate the class_attribute while a negative value is a KEY_RELEASE . The keyboard supports repetition when the key stays pressed for too long. The following demonstrates how simulation work. -

    Simulate a key press on "g" ( KEY_G +

    Simulate a key press on "g" ( KEY_G = 34):

    1echo "+34" | sudo tee /dev/vinput0
    -

    Simulate a key release on "g" ( KEY_G +

    Simulate a key release on "g" ( KEY_G = 34):

    @@ -6269,13 +6270,13 @@ following demonstrates how simulation work. 108 109MODULE_LICENSE("GPL"); 110MODULE_DESCRIPTION("Emulate keyboard input events through /dev/vinput"); -

    +

    18 Standardizing the interfaces: The Device Model

    -

    Up to this point we have seen all kinds of modules doing all kinds of things, but there +

    Up to this point we have seen all kinds of modules doing all kinds of things, but there was no consistency in their interfaces with the rest of the kernel. To impose some consistency such that there is at minimum a standardized way to start, suspend and resume a device a device model was added. An example is shown below, and you can @@ -6382,13 +6383,13 @@ functions. 97 98MODULE_LICENSE("GPL"); 99MODULE_DESCRIPTION("Linux Device Model example"); -

    +

    19 Optimizations

    -

    +

    19.1 Likely and Unlikely conditions

    -

    Sometimes you might want your code to run as quickly as possible, +

    Sometimes you might want your code to run as quickly as possible, especially if it is handling an interrupt or doing something which might cause noticeable latency. If your code contains boolean conditions and if you know that the conditions are almost always likely to evaluate as either @@ -6407,7 +6408,7 @@ to succeed. 4    bio = NULL; 5    goto out; 6} -

    When the unlikely +

    When the unlikely macro is used, the compiler alters its machine instruction output, so that it continues along the false branch and only jumps if the condition is true. That avoids flushing the processor pipeline. The opposite happens if you use the @@ -6416,34 +6417,34 @@ avoids flushing the processor pipeline. The opposite happens if you use the -

    +

    20 Common Pitfalls

    -

    +

    20.1 Using standard libraries

    -

    You can not do that. In a kernel module, you can only use kernel functions which are +

    You can not do that. In a kernel module, you can only use kernel functions which are the functions you can see in /proc/kallsyms. -

    +

    20.2 Disabling interrupts

    -

    You might need to do this for a short time and that is OK, but if you do not enable +

    You might need to do this for a short time and that is OK, but if you do not enable them afterwards, your system will be stuck and you will have to power it off. -

    +

    21 Where To Go From Here?

    -

    For people seriously interested in kernel programming, I recommend kernelnewbies.org +

    For people seriously interested in kernel programming, I recommend kernelnewbies.org and the Documentation subdirectory within the kernel source code which is not always easy to understand but can be a starting point for further investigation. Also, as Linus Torvalds said, the best way to learn the kernel is to read the source code yourself. -

    If you would like to contribute to this guide or notice anything glaringly wrong, +

    If you would like to contribute to this guide or notice anything glaringly wrong, please create an issue at https://github.com/sysprog21/lkmpg. Your pull requests will be appreciated. -

    Happy hacking! +

    Happy hacking!

    -

    1The goal of threaded interrupts is to push more of the work to separate threads, so that the +

    1The goal of threaded interrupts is to push more of the work to separate threads, so that the minimum needed for acknowledging an interrupt is reduced, and therefore the time spent handling the interrupt (where it can’t handle any other interrupts at the same time) is reduced. See https://lwn.net/Articles/302043/.

    diff --git a/lkmpg-for-ht.html b/lkmpg-for-ht.html index 2c6681d..31a8575 100644 --- a/lkmpg-for-ht.html +++ b/lkmpg-for-ht.html @@ -1414,14 +1414,15 @@ using the f_pos specific lock, which makes the file position update to become the mutual exclusion. So, we can safely implement those operations without unnecessary locking. -

    Since Linux v5.6, the proc_ops +

    Additionally, since Linux v5.6, the proc_ops structure was introduced to replace the use of the file_operations - structure when registering proc handlers. -

    + structure when registering proc handlers. See more information in the 7.1 +section. +

    6.2 The file structure

    -

    Each device is represented in the kernel by a file structure, which is defined +

    Each device is represented in the kernel by a file structure, which is defined in include/linux/fs.h. Be aware that a file is a kernel level structure and never appears in a user space program. It is not the same thing as a FILE @@ -1433,31 +1434,31 @@ function. Also, its name is a bit misleading; it represents an abstract open -

    An instance of struct file is commonly named +

    An instance of struct file is commonly named filp . You’ll also see it referred to as a struct file object. Resist the temptation. -

    Go ahead and look at the definition of file. Most of the entries you see, like struct +

    Go ahead and look at the definition of file. Most of the entries you see, like struct dentry are not used by device drivers, and you can ignore them. This is because drivers do not fill file directly; they only use structures contained in file which are created elsewhere. -

    +

    6.3 Registering A Device

    -

    As discussed earlier, char devices are accessed through device files, usually located in +

    As discussed earlier, char devices are accessed through device files, usually located in /dev. This is by convention. When writing a driver, it is OK to put the device file in your current directory. Just make sure you place it in /dev for a production driver. The major number tells you which driver handles which device file. The minor number is used only by the driver itself to differentiate which device it is operating on, just in case the driver handles more than one device. -

    Adding a driver to your system means registering it with the kernel. This is synonymous +

    Adding a driver to your system means registering it with the kernel. This is synonymous with assigning it a major number during the module’s initialization. You do this by using the register_chrdev function, defined by include/linux/fs.h.

    1int register_chrdev(unsigned int major, const char *name, struct file_operations *fops);
    -

    Where unsigned int major is the major number you want to request, +

    Where unsigned int major is the major number you want to request, const char *name is the name of the device as it will appear in /proc/devices and struct file_operations *fops @@ -1467,13 +1468,13 @@ registration failed. Note that we didn’t pass the minor number to register_chrdev . That is because the kernel doesn’t care about the minor number; only our driver uses it. -

    Now the question is, how do you get a major number without hijacking +

    Now the question is, how do you get a major number without hijacking one that’s already in use? The easiest way would be to look through Documentation/admin-guide/devices.txt and pick an unused one. That is a bad way of doing things because you will never be sure if the number you picked will be assigned later. The answer is that you can ask the kernel to assign you a dynamic major number. -

    If you pass a major number of 0 to register_chrdev +

    If you pass a major number of 0 to register_chrdev , the return value will be the dynamically allocated major number. The downside is that you can not make a device file in advance, since you do not @@ -1490,11 +1491,11 @@ third method is that we can have our driver make the device file using the device_destroy during the call to cleanup_module . -

    However, register_chrdev() +

    However, register_chrdev() would occupy a range of minor numbers associated with the given major. The recommended way to reduce waste for char device registration is using cdev interface. -

    The newer interface completes the char device registration in two distinct steps. +

    The newer interface completes the char device registration in two distinct steps. First, we should register a range of device numbers, which can be completed with register_chrdev_region or alloc_chrdev_region @@ -1503,12 +1504,12 @@ First, we should register a range of device numbers, which can be completed with

    1int register_chrdev_region(dev_t from, unsigned count, const char *name); 
     2int alloc_chrdev_region(dev_t *dev, unsigned baseminor, unsigned count, const char *name);
    -

    The choose of two different functions depend on whether you know the major numbers for your +

    The choose of two different functions depend on whether you know the major numbers for your device. Using register_chrdev_region if you know the device major number and alloc_chrdev_region if you would like to allocate a dynamicly-allocated major number. -

    Second, we should initialize the data structure +

    Second, we should initialize the data structure struct cdev for our char device and associate it with the device numbers. To initialize the struct cdev @@ -1517,7 +1518,7 @@ device. Using register_chrdev_region

    1struct cdev *my_dev = cdev_alloc(); 
     2my_cdev->ops = &my_fops;
    -

    However, the common usage pattern will embed the +

    However, the common usage pattern will embed the struct cdev within a device-specific structure of your own. In this case, we’ll need cdev_init @@ -1528,18 +1529,18 @@ device. Using register_chrdev_region -

    Once we finish the initialization, we can add the char device to the system by using +

    Once we finish the initialization, we can add the char device to the system by using the cdev_add .

    1int cdev_add(struct cdev *p, dev_t dev, unsigned count);
    -

    To find a example using the interface, you can see ioctl.c described in section +

    To find a example using the interface, you can see ioctl.c described in section 9. -

    +

    6.4 Unregistering A Device

    -

    We can not allow the kernel module to be +

    We can not allow the kernel module to be rmmod ’ed whenever root feels like it. If the device file is opened by a process and then we remove the kernel module, using the file would cause a call to the memory location @@ -1549,7 +1550,7 @@ unlucky, another kernel module was loaded into the same location, which means a jump into the middle of another function within the kernel. The results of this would be impossible to predict, but they can not be very positive. -

    Normally, when you do not want to allow something, you return an error code +

    Normally, when you do not want to allow something, you return an error code (a negative number) from the function which is supposed to do it. With cleanup_module that’s impossible because it is a void function. However, there is a counter @@ -1575,26 +1576,26 @@ decrease and display this counter:

  • module_refcount(THIS_MODULE) : Return the value of reference count of current module.
  • -

    It is important to keep the counter accurate; if you ever do lose track of the +

    It is important to keep the counter accurate; if you ever do lose track of the correct usage count, you will never be able to unload the module; it’s now reboot time, boys and girls. This is bound to happen to you sooner or later during a module’s development. -

    +

    6.5 chardev.c

    -

    The next code sample creates a char driver named chardev. You can dump its device +

    The next code sample creates a char driver named chardev. You can dump its device file.

    1cat /proc/devices
    -

    (or open the file with a program) and the driver will put the number of times the +

    (or open the file with a program) and the driver will put the number of times the device file has been read from into the file. We do not support writing to the file (like echo "hi" > /dev/hello ), but catch these attempts and tell the user that the operation is not supported. Don’t worry if you don’t see what we do with the data we read into the buffer; we don’t do much with it. We simply read in the data and print a message acknowledging that we received it. -

    In the multiple-threaded environment, without any protection, concurrent access +

    In the multiple-threaded environment, without any protection, concurrent access to the same memory may lead to the race condition, and will not preserve the performance. In the kernel module, this problem may happen due to multiple instances accessing the shared resources. Therefore, a solution is to enforce the @@ -1768,32 +1769,32 @@ concurrency details in the 12

    +

    6.6 Writing Modules for Multiple Kernel Versions

    -

    The system calls, which are the major interface the kernel shows to the processes, +

    The system calls, which are the major interface the kernel shows to the processes, generally stay the same across versions. A new system call may be added, but usually the old ones will behave exactly like they used to. This is necessary for backward compatibility – a new kernel version is not supposed to break regular processes. In most cases, the device files will also remain the same. On the other hand, the internal interfaces within the kernel can and do change between versions. -

    There are differences between different kernel versions, and if you want +

    There are differences between different kernel versions, and if you want to support multiple kernel versions, you will find yourself having to code conditional compilation directives. The way to do this to compare the macro LINUX_VERSION_CODE to the macro KERNEL_VERSION . In version a.b.c of the kernel, the value of this macro would be 216a+ 28b+ c  . -

    +

    7 The /proc File System

    -

    In Linux, there is an additional mechanism for the kernel and kernel modules to send +

    In Linux, there is an additional mechanism for the kernel and kernel modules to send information to processes — the /proc file system. Originally designed to allow easy access to information about processes (hence the name), it is now used by every bit of the kernel which has something interesting to report, such as /proc/modules which provides the list of modules and /proc/meminfo which gathers memory usage statistics. -

    The method to use the proc file system is very similar to the one used with device +

    The method to use the proc file system is very similar to the one used with device drivers — a structure is created with all the information needed for the /proc file, including pointers to any handler functions (in our case there is only one, the one called when somebody attempts to read from the /proc file). Then, @@ -1804,18 +1805,18 @@ one called when somebody attempts to read from the

    Normal file systems are located on a disk, rather than just in memory (which is +

    Normal file systems are located on a disk, rather than just in memory (which is where /proc is), and in that case the index-node (inode for short) number is a pointer to a disk location where the file’s inode is located. The inode contains information about the file, for example the file’s permissions, together with a pointer to the disk location or locations where the file’s data can be found. -

    Because we don’t get called when the file is opened or closed, there’s nowhere for +

    Because we don’t get called when the file is opened or closed, there’s nowhere for us to put try_module_get and module_put in this module, and if the file is opened and then the module is removed, there’s no way to avoid the consequences. -

    Here a simple example showing how to use a /proc file. This is the HelloWorld for +

    Here a simple example showing how to use a /proc file. This is the HelloWorld for the /proc filesystem. There are three parts: create the file /proc/helloworld in the function init_module , return a value (and a buffer) when the file /proc/helloworld is read in the callback @@ -1823,12 +1824,12 @@ function procfile_read , and delete the file /proc/helloworld in the function cleanup_module . -

    The /proc/helloworld is created when the module is loaded with the function +

    The /proc/helloworld is created when the module is loaded with the function proc_create . The return value is a struct proc_dir_entry , and it will be used to configure the file /proc/helloworld (for example, the owner of this file). A null return value means that the creation has failed. -

    Every time the file /proc/helloworld is read, the function +

    Every time the file /proc/helloworld is read, the function procfile_read is called. Two parameters of this function are very important: the buffer (the second parameter) and the offset (the fourth one). The content of the @@ -1845,7 +1846,7 @@ function, if it never returns zero, the read function is called endlessly. $ cat /proc/helloworld HelloWorld! -

    +

    1/* 
    @@ -1917,10 +1918,10 @@ HelloWorld!
     67module_exit(procfs1_exit); 
     68 
     69MODULE_LICENSE("GPL");
    -

    +

    7.1 The proc_ops Structure

    -

    The proc_ops +

    The proc_ops structure is defined in include/linux/proc_fs.h in Linux v5.6+. In older kernels, it used file_operations for custom hooks in /proc file system, but it contains some @@ -1932,10 +1933,10 @@ performance. For example, the file which never disappears in proc_flag as PROC_ENTRY_PERMANENT to save 2 atomic ops, 1 allocation, 1 free in per open/read/close sequence. -

    +

    7.2 Read and Write a /proc File

    -

    We have seen a very simple example for a /proc file where we only read +

    We have seen a very simple example for a /proc file where we only read the file /proc/helloworld. It is also possible to write in a /proc file. It works the same way as read, a function is called when the /proc file is written. But there is a little difference with read, data comes from @@ -1943,7 +1944,7 @@ user, so you have to import data from user space to kernel space (with copy_from_user or get_user ) -

    The reason for copy_from_user +

    The reason for copy_from_user or get_user is that Linux memory (on Intel architecture, it may be different under some @@ -1954,7 +1955,7 @@ not reference a unique location in memory, only a location in a memory segment, and you need to know which memory segment it is to be able to use it. There is one memory segment for the kernel, and one for each of the processes. -

    The only memory segment accessible to a process is its own, so when +

    The only memory segment accessible to a process is its own, so when writing regular programs to run as processes, there is no need to worry about segments. When you write a kernel module, normally you want to access the kernel memory segment, which is handled automatically by the system. @@ -2069,22 +2070,22 @@ because data is already in kernel space. 95module_exit(procfs2_exit); 96 97MODULE_LICENSE("GPL"); -

    +

    7.3 Manage /proc file with standard filesystem

    -

    We have seen how to read and write a /proc file with the /proc interface. But it is +

    We have seen how to read and write a /proc file with the /proc interface. But it is also possible to manage /proc file with inodes. The main concern is to use advanced functions, like permissions. -

    In Linux, there is a standard mechanism for file system registration. +

    In Linux, there is a standard mechanism for file system registration. Since every file system has to have its own functions to handle inode and file operations, there is a special structure to hold pointers to all those functions, struct inode_operations , which includes a pointer to struct proc_ops . -

    The difference between file and inode operations is that file operations deal with +

    The difference between file and inode operations is that file operations deal with the file itself whereas inode operations deal with ways of referencing the file, such as creating links to it. -

    In /proc, whenever we register a new file, we’re allowed to specify which +

    In /proc, whenever we register a new file, we’re allowed to specify which struct inode_operations will be used to access to it. This is the mechanism we use, a struct inode_operations @@ -2095,7 +2096,7 @@ creating links to it. which includes pointers to our procf_read and procfs_write functions. -

    Another interesting point here is the +

    Another interesting point here is the module_permission function. This function is called whenever a process tries to do something with the /proc file, and it can decide whether to allow access or not. Right now it is only @@ -2104,7 +2105,7 @@ pointer to a structure which includes information on the currently running process), but it could be based on anything we like, such as what other processes are doing with the same file, the time of day, or the last input we received. -

    It is important to note that the standard roles of read and write are reversed in +

    It is important to note that the standard roles of read and write are reversed in the kernel. Read functions are used for output, whereas write functions are used for input. The reason for that is that read and write refer to the user’s point of view — if a process reads something from the kernel, then the kernel needs to output it, and @@ -2219,14 +2220,14 @@ input. 105module_exit(procfs3_exit); 106 107MODULE_LICENSE("GPL"); -

    Still hungry for procfs examples? Well, first of all keep in mind, there are rumors +

    Still hungry for procfs examples? Well, first of all keep in mind, there are rumors around, claiming that procfs is on its way out, consider using sysfs instead. Consider using this mechanism, in case you want to document something kernel related yourself. -

    +

    7.4 Manage /proc file with seq_file

    -

    As we have seen, writing a /proc file may be quite “complex”. +

    As we have seen, writing a /proc file may be quite “complex”. So to help people writting /proc file, there is an API named seq_file that helps formating a /proc file for output. It is based on sequence, which is composed of @@ -2235,7 +2236,7 @@ So to help people writting , and stop() . The seq_file API starts a sequence when a user read the /proc file. -

    A sequence begins with the call of the function +

    A sequence begins with the call of the function start() . If the return is a non NULL value, the function next() @@ -2252,7 +2253,7 @@ time next() returns NULL , then the function stop() is called. -

    BE CAREFUL: when a sequence is finished, another one starts. That means that at the end +

    BE CAREFUL: when a sequence is finished, another one starts. That means that at the end of function stop() , the function start() is called again. This loop finishes when the function @@ -2269,14 +2270,14 @@ of function stop() -

    srYrsNNYtaeenetoooertusetupstrxr((ntn))( tis)istrr teeaNreNatUaUtmLtLmeLmLen?e?ntntt  +

    srYrsNNYtaeenetoooertusetupstrxr((ntn))( tis)istrr teeaNreNatUaUtmLtLmeLmLen?e?ntntt

    Figure 1:How seq_file works
    -

    The seq_file +

    The seq_file provides basic functions for proc_ops , such as seq_read , seq_lseek @@ -2401,26 +2402,26 @@ the same way as in the previous example. 116module_exit(procfs4_exit); 117 118MODULE_LICENSE("GPL"); -

    If you want more information, you can read this web page: +

    If you want more information, you can read this web page:

    -

    You can also read the code of fs/seq_file.c in the linux kernel. +

    You can also read the code of fs/seq_file.c in the linux kernel.

    8 sysfs: Interacting with your module

    -

    sysfs allows you to interact with the running kernel from userspace by reading or +

    sysfs allows you to interact with the running kernel from userspace by reading or setting variables inside of modules. This can be useful for debugging purposes, or just as an interface for applications or scripts. You can find sysfs directories and files under the /sys directory on your system.

    1ls -l /sys
    -

    Attributes can be exported for kobjects in the form of regular files in the +

    Attributes can be exported for kobjects in the form of regular files in the filesystem. Sysfs forwards file I/O operations to methods defined for the attributes, providing a means to read and write kernel attributes. -

    An attribute definition in simply: +

    An attribute definition in simply:

    1struct attribute { 
    @@ -2431,7 +2432,7 @@ providing a means to read and write kernel attributes.
     6 
     7int sysfs_create_file(struct kobject * kobj, const struct attribute * attr); 
     8void sysfs_remove_file(struct kobject * kobj, const struct attribute * attr);
    -

    For example, the driver model defines +

    For example, the driver model defines struct device_attribute like:

    @@ -2449,7 +2450,7 @@ providing a means to read and write kernel attributes. 8 9int device_create_file(struct device *, const struct device_attribute *); 10void device_remove_file(struct device *, const struct device_attribute *); -

    To read or write attributes, show() +

    To read or write attributes, show() or store() method must be specified when declaring the attribute. For the common cases include/linux/sysfs.h provides convenience macros @@ -2458,7 +2459,7 @@ common cases __ATTR_WO , etc.) to make defining attributes easier as well as making code more concise and readable. -

    An example of a hello world module which includes the creation of a variable +

    An example of a hello world module which includes the creation of a variable accessible via sysfs is given below.

    @@ -2523,34 +2524,34 @@ accessible via sysfs is given below. 59module_exit(mymodule_exit); 60 61MODULE_LICENSE("GPL"); -

    Make and install the module: +

    Make and install the module:

    1make 
     2sudo insmod hello-sysfs.ko
    -

    Check that it exists: +

    Check that it exists:

    1sudo lsmod | grep hello_sysfs
    -

    What is the current value of myvariable +

    What is the current value of myvariable ?

    1cat /sys/kernel/mymodule/myvariable
    -

    Set the value of myvariable +

    Set the value of myvariable and check that it changed.

    1echo "32" > /sys/kernel/mymodule/myvariable 
     2cat /sys/kernel/mymodule/myvariable
    -

    Finally, remove the test module: +

    Finally, remove the test module:

    1sudo rmmod hello_sysfs
    -

    In the above case, we use a simple kobject to create a directory under +

    In the above case, we use a simple kobject to create a directory under sysfs, and communicate with its attributes. Since Linux v2.6.0, the kobject structure made its appearance. It was initially meant as a simple way of @@ -2559,17 +2560,17 @@ bit of mission creep, it is now the glue that holds much of the device model and its sysfs interface together. For more information about kobject and sysfs, see Documentation/driver-api/driver-model/driver.rst and https://lwn.net/Articles/51437/. -

    +

    9 Talking To Device Files

    -

    Device files are supposed to represent physical devices. Most physical devices are +

    Device files are supposed to represent physical devices. Most physical devices are used for output as well as input, so there has to be some mechanism for device drivers in the kernel to get the output to send to the device from processes. This is done by opening the device file for output and writing to it, just like writing to a file. In the following example, this is implemented by device_write . -

    This is not always enough. Imagine you had a serial port connected to a modem +

    This is not always enough. Imagine you had a serial port connected to a modem (even if you have an internal modem, it is still implemented from the CPU’s perspective as a serial port connected to a modem, so you don’t have to tax your imagination too hard). The natural thing to do would be to use the @@ -2579,7 +2580,7 @@ responses for commands or the data received through the phone line). However, this leaves open the question of what to do when you need to talk to the serial port itself, for example to configure the rate at which data is sent and received. -

    The answer in Unix is to use a special function called +

    The answer in Unix is to use a special function called ioctl (short for Input Output ConTroL). Every device can have its own ioctl @@ -2588,7 +2589,7 @@ kernel), write ioctl’s (to return information to a process), both or neither. here the roles of read and write are reversed again, so in ioctl’s read is to send information to the kernel and write is to receive information from the kernel. -

    The ioctl function is called with three parameters: the file descriptor of the +

    The ioctl function is called with three parameters: the file descriptor of the appropriate device file, the ioctl number, and a parameter, which is of type long so you can use a cast to use it to pass anything. You will not be able to pass a structure this way, but you will be able to pass a pointer to the structure. Here is an @@ -2788,7 +2789,7 @@ example: 188 189MODULE_LICENSE("GPL"); 190MODULE_DESCRIPTION("This is test_ioctl module"); -

    You can see there is an argument called +

    You can see there is an argument called cmd in test_ioctl_ioctl() function. It is the ioctl number. The ioctl number encodes the major @@ -2803,11 +2804,11 @@ included both by the programs which will use ioctl (so they can generate the appropriate ioctl’s) and by the kernel module (so it can understand it). In the example below, the header file is chardev.h and the program which uses it is userspace_ioctl.c. -

    If you want to use ioctls in your own kernel modules, it is best to receive an +

    If you want to use ioctls in your own kernel modules, it is best to receive an official ioctl assignment, so if you accidentally get somebody else’s ioctls, or if they get yours, you’ll know something is wrong. For more information, consult the kernel source tree at Documentation/userspace-api/ioctl/ioctl-number.rst. -

    Also, we need to be careful that concurrent access to the shared resources will +

    Also, we need to be careful that concurrent access to the shared resources will lead to the race condition. The solution is using atomic Compare-And-Swap (CAS), which we mentioned at 6.5 section, to enforce the exclusive access.

    @@ -3197,10 +3198,10 @@ which we mentioned at 6.5 101    close(file_desc); 102    exit(EXIT_FAILURE); 103} -

    +

    10 System Calls

    -

    So far, the only thing we’ve done was to use well defined kernel mechanisms to +

    So far, the only thing we’ve done was to use well defined kernel mechanisms to register /proc files and device handlers. This is fine if you want to do something the kernel programmers thought you’d want, such as write a device driver. But what if @@ -3208,7 +3209,7 @@ kernel programmers thought you’d want, such as write a device driver. But what you want to do something unusual, to change the behavior of the system in some way? Then, you are mostly on your own. -

    If you are not being sensible and using a virtual machine then this is where kernel +

    If you are not being sensible and using a virtual machine then this is where kernel programming can become hazardous. While writing the example below, I killed the open() system call. This meant I could not open any files, I could not run any @@ -3220,7 +3221,7 @@ ensure you do not lose any files, even within a test environment, please run right before you do the insmod and the rmmod . -

    Forget about /proc files, forget about device files. They are just minor details. +

    Forget about /proc files, forget about device files. They are just minor details. Minutiae in the vast expanse of the universe. The real process to kernel communication mechanism, the one used by all processes, is system calls. When a process requests a service from the kernel (such as opening a file, forking to a new @@ -3229,11 +3230,11 @@ change the behaviour of the kernel in interesting ways, this is the place to do it. By the way, if you want to see which system calls a program uses, run strace <arguments> . -

    In general, a process is not supposed to be able to access the kernel. It can not +

    In general, a process is not supposed to be able to access the kernel. It can not access kernel memory and it can’t call kernel functions. The hardware of the CPU enforces this (that is the reason why it is called “protected mode” or “page protection”). -

    System calls are an exception to this general rule. What happens is that the +

    System calls are an exception to this general rule. What happens is that the process fills the registers with the appropriate values and then calls a special instruction which jumps to a previously defined location in the kernel (of course, that location is readable by user processes, it is not writable by them). Under Intel CPUs, @@ -3241,7 +3242,7 @@ this is done by means of interrupt 0x80. The hardware knows that once you jump t this location, you are no longer running in restricted user mode, but as the operating system kernel — and therefore you’re allowed to do whatever you want. -

    The location in the kernel a process can jump to is called system_call. The +

    The location in the kernel a process can jump to is called system_call. The procedure at that location checks the system call number, which tells the kernel what service the process requested. Then, it looks at the table of system calls ( sys_call_table @@ -3251,7 +3252,7 @@ different process, if the process time ran out). If you want to read this code, at the source file arch/$(architecture)/kernel/entry.S, after the line ENTRY(system_call) . -

    So, if we want to change the way a certain system call works, what we need to do +

    So, if we want to change the way a certain system call works, what we need to do @@ -3262,7 +3263,7 @@ code, and then calling the original function) and then change the pointer at don’t want to leave the system in an unstable state, it’s important for cleanup_module to restore the table to its original state. -

    To modify the content of sys_call_table +

    To modify the content of sys_call_table , we need to consider the control register. A control register is a processor register that changes or controls the general behavior of the CPU. For x86 architecture, the cr0 register has various control flags that modify the basic @@ -3275,11 +3276,11 @@ read-only sections Therefore, we must disable the

    However, sys_call_table +

    However, sys_call_table symbol is unexported to prevent misuse. But there have few ways to get the symbol, manual symbol lookup and kallsyms_lookup_name . Here we use both depend on the kernel version. -

    Because of the control-flow integrity, which is a technique to prevent the redirect +

    Because of the control-flow integrity, which is a technique to prevent the redirect execution code from the attacker, for making sure that the indirect calls go to the expected addresses and the return addresses are not changed. Since Linux v5.7, the kernel patched the series of control-flow enforcement (CET) for x86, and some @@ -3304,10 +3305,10 @@ COLLECT_GCC_OPTIONS='-v' '-Q' '-O2' '--help=target' '-mtune=generic' '-marc  GNU C17 (Ubuntu 9.3.0-17ubuntu1~20.04) version 9.3.0 (x86_64-linux-gnu) ... -

    But CET should not be enabled in the kernel, it may break the Kprobes and bpf. +

    But CET should not be enabled in the kernel, it may break the Kprobes and bpf. Consequently, CET is disabled since v5.11. To guarantee the manual symbol lookup worked, we only use up to v5.4. -

    Unfortunately, since Linux v5.7 kallsyms_lookup_name +

    Unfortunately, since Linux v5.7 kallsyms_lookup_name is also unexported, it needs certain trick to get the address of kallsyms_lookup_name . If CONFIG_KPROBES @@ -3319,7 +3320,7 @@ passes the addresses of the saved registers and the Kprobe struct to the handler you defined, then executes it. Kprobes can be registered by symbol name or address. Within the symbol name, the address will be handled by the kernel. -

    Otherwise, specify the address of sys_call_table +

    Otherwise, specify the address of sys_call_table from /proc/kallsyms and /boot/System.map into sym parameter. Following is the sample usage for /proc/kallsyms: @@ -3334,8 +3335,8 @@ ffffffff820013a0 R sys_call_table ffffffff820023e0 R ia32_sys_call_table $ sudo insmod syscall.ko sym=0xffffffff820013a0 -

    -

    Using the address from /boot/System.map, be careful about KASLR (Kernel +

    +

    Using the address from /boot/System.map, be careful about KASLR (Kernel Address Space Layout Randomization). KASLR may randomize the address of kernel code and data at every boot time, such as the static address listed in /boot/System.map will offset by some entropy. The purpose of KASLR is to protect @@ -3364,7 +3365,7 @@ ffffffff82000300 R sys_call_table $ sudo grep sys_call_table /proc/kallsyms ffffffff86400300 R sys_call_table -

    If KASLR is enabled, we have to take care of the address from /proc/kallsyms each +

    If KASLR is enabled, we have to take care of the address from /proc/kallsyms each time we reboot the machine. In order to use the address from /boot/System.map, make sure that KASLR is disabled. You can add the nokaslr for disabling KASLR in next booting time: @@ -3380,8 +3381,8 @@ $ grep quiet /etc/default/grub GRUB_CMDLINE_LINUX_DEFAULT="quiet nokaslr splash" $ sudo update-grub -

    -

    For more information, check out the following: +

    +

    For more information, check out the following:

    -

    The source code here is an example of such a kernel module. We want to “spy” on a certain +

    The source code here is an example of such a kernel module. We want to “spy” on a certain user, and to pr_info() a message whenever that user opens a file. Towards this end, we replace the system call to open a file with our own function, called @@ -3408,7 +3409,7 @@ spy on, it calls pr_info() to display the name of the file to be opened. Then, either way, it calls the original open() function with the same parameters, to actually open the file. -

    The init_module +

    The init_module function replaces the appropriate location in sys_call_table and keeps the original pointer in a variable. The @@ -3426,7 +3427,7 @@ with B_open , which will call what it thinks is the original system call, A_open , when it’s done. -

    Now, if B is removed first, everything will be well — it will simply restore the system +

    Now, if B is removed first, everything will be well — it will simply restore the system call to A_open , which calls the original. However, if A is removed and then B is removed, the system will crash. A’s removal will restore the system call to the original, @@ -3446,7 +3447,7 @@ problem. When A is removed, it sees that the system call was changed to will still try to call A_open which is no longer there, so that even without removing B the system would crash. -

    Note that all the related problems make syscall stealing unfeasible for +

    Note that all the related problems make syscall stealing unfeasible for production use. In order to keep people from doing potential harmful things sys_call_table is no longer exported. This means, if you want to do something more than a mere @@ -3687,13 +3688,13 @@ dry run of this example, you will have to patch your current kernel in order to 227module_exit(syscall_end); 228 229MODULE_LICENSE("GPL"); -

    +

    11 Blocking Processes and threads

    -

    +

    11.1 Sleep

    -

    What do you do when somebody asks you for something you can not do right +

    What do you do when somebody asks you for something you can not do right away? If you are a human being and you are bothered by a human being, the only thing you can say is: "Not right now, I’m busy. Go away!". But if you are a kernel module and you are bothered by a process, you have another @@ -3701,21 +3702,21 @@ possibility. You can put the process to sleep until you can service it. After al processes are being put to sleep by the kernel and woken up all the time (that is the way multiple processes appear to run on the same time on a single CPU). -

    This kernel module is an example of this. The file (called /proc/sleep) can only +

    This kernel module is an example of this. The file (called /proc/sleep) can only be opened by a single process at a time. If the file is already open, the kernel module calls wait_event_interruptible . The easiest way to keep a file open is to open it with:

    1tail -f
    -

    This function changes the status of the task (a task is the kernel data structure +

    This function changes the status of the task (a task is the kernel data structure which holds information about a process and the system call it is in, if any) to TASK_INTERRUPTIBLE , which means that the task will not run until it is woken up somehow, and adds it to WaitQ, the queue of tasks waiting to access the file. Then, the function calls the scheduler to context switch to a different process, one which has some use for the CPU. -

    When a process is done with the file, it closes it, and +

    When a process is done with the file, it closes it, and module_close is called. That function wakes up all the processes in the queue (there’s no mechanism to only wake up one of them). It then returns and the process which just @@ -3728,31 +3729,31 @@ Eventually, one of the processes which was in the queue will be given control of the CPU by the scheduler. It starts at the point right after the call to module_interruptible_sleep_on . -

    This means that the process is still in kernel mode - as far as the process +

    This means that the process is still in kernel mode - as far as the process is concerned, it issued the open system call and the system call has not returned yet. The process does not know somebody else used the CPU for most of the time between the moment it issued the call and the moment it returned. -

    It can then proceed to set a global variable to tell all the other processes that the +

    It can then proceed to set a global variable to tell all the other processes that the file is still open and go on with its life. When the other processes get a piece of the CPU, they’ll see that global variable and go back to sleep. -

    So we will use tail -f +

    So we will use tail -f to keep the file open in the background, while trying to access it with another process (again in the background, so that we need not switch to a different vt). As soon as the first background process is killed with kill %1 , the second is woken up, is able to access the file and finally terminates. -

    To make our life more interesting, module_close +

    To make our life more interesting, module_close does not have a monopoly on waking up the processes which wait to access the file. A signal, such as Ctrl +c (SIGINT) can also wake up a process. This is because we used module_interruptible_sleep_on . We could have used module_sleep_on instead, but that would have resulted in extremely angry users whose Ctrl+c’s are ignored. -

    In that case, we want to return with +

    In that case, we want to return with -EINTR immediately. This is important so users can, for example, kill the process before it receives the file. -

    There is one more point to remember. Some times processes don’t want to sleep, they want +

    There is one more point to remember. Some times processes don’t want to sleep, they want either to get what they want immediately, or to be told it cannot be done. Such processes use the O_NONBLOCK flag when opening the file. The kernel is supposed to respond by returning with the error @@ -3788,7 +3789,7 @@ $ cat_nonblock /proc/sleep Last input: $ -

    +

    1/* 
    @@ -4067,14 +4068,14 @@ $
     57 
     58    return 0; 
     59}
    -

    +

    11.2 Completions

    -

    Sometimes one thing should happen before another within a module having multiple threads. +

    Sometimes one thing should happen before another within a module having multiple threads. Rather than using /bin/sleep commands, the kernel has another way to do this which allows timeouts or interrupts to also happen. -

    In the following example two threads are started, but one needs to start before +

    In the following example two threads are started, but one needs to start before another.

    @@ -4157,31 +4158,31 @@ another. 74 75MODULE_DESCRIPTION("Completions example"); 76MODULE_LICENSE("GPL"); -

    The machine +

    The machine structure stores the completion states for the two threads. At the exit point of each thread the respective completion state is updated, and wait_for_completion is used by the flywheel thread to ensure that it does not begin prematurely. -

    So even though flywheel_thread +

    So even though flywheel_thread is started first you should notice if you load this module and run dmesg that turning the crank always happens first because the flywheel thread waits for it to complete. -

    There are other variations upon the +

    There are other variations upon the wait_for_completion function, which include timeouts or being interrupted, but this basic mechanism is enough for many common situations without adding a lot of complexity. -

    +

    12 Avoiding Collisions and Deadlocks

    -

    If processes running on different CPUs or in different threads try to access the same +

    If processes running on different CPUs or in different threads try to access the same memory, then it is possible that strange things can happen or your system can lock up. To avoid this, various types of mutual exclusion kernel functions are available. These indicate if a section of code is "locked" or "unlocked" so that simultaneous attempts to run it can not happen.

    12.1 Mutex

    -

    You can use kernel mutexes (mutual exclusions) in much the same manner that you +

    You can use kernel mutexes (mutual exclusions) in much the same manner that you might deploy them in userland. This may be all that is needed to avoid collisions in most cases.

    @@ -4227,10 +4228,10 @@ most cases. 39 40MODULE_DESCRIPTION("Mutex example"); 41MODULE_LICENSE("GPL"); -

    +

    12.2 Spinlocks

    -

    As the name suggests, spinlocks lock up the CPU that the code is running on, +

    As the name suggests, spinlocks lock up the CPU that the code is running on, taking 100% of its resources. Because of this you should only use the spinlock @@ -4238,7 +4239,7 @@ taking 100% of its resources. Because of this you should only use the spinlock mechanism around code which is likely to take no more than a few milliseconds to run and so will not noticeably slow anything down from the user’s point of view. -

    The example here is "irq safe" in that if interrupts happen during the lock then +

    The example here is "irq safe" in that if interrupts happen during the lock then they will not be forgotten and will activate when the unlock happens, using the flags variable to retain their state. @@ -4307,10 +4308,10 @@ they will not be forgotten and will activate when the unlock happens, using the 61 62MODULE_DESCRIPTION("Spinlock example"); 63MODULE_LICENSE("GPL"); -

    +

    12.3 Read and write locks

    -

    Read and write locks are specialised kinds of spinlocks so that you can exclusively +

    Read and write locks are specialised kinds of spinlocks so that you can exclusively read from something or write to something. Like the earlier spinlocks example, the one below shows an "irq safe" situation in which if other functions were triggered from irqs which might also read and write to whatever you are concerned with @@ -4375,14 +4376,14 @@ module. 53 54MODULE_DESCRIPTION("Read/Write locks example"); 55MODULE_LICENSE("GPL"); -

    Of course, if you know for sure that there are no functions triggered by irqs +

    Of course, if you know for sure that there are no functions triggered by irqs which could possibly interfere with your logic then you can use the simpler read_lock(&myrwlock) and read_unlock(&myrwlock) or the corresponding write functions.

    12.4 Atomic operations

    -

    If you are doing simple arithmetic: adding, subtracting or bitwise operations, then +

    If you are doing simple arithmetic: adding, subtracting or bitwise operations, then there is another way in the multi-CPU and multi-hyperthreaded world to stop other parts of the system from messing with your mojo. By using atomic operations you can be confident that your addition, subtraction or bit flip did actually happen @@ -4467,7 +4468,7 @@ below. -

    Before the C11 standard adopts the built-in atomic types, the kernel already +

    Before the C11 standard adopts the built-in atomic types, the kernel already provided a small set of atomic types by using a bunch of tricky architecture-specific codes. Implementing the atomic types by C11 atomics may allow the kernel to throw away the architecture-specific codes and letting the kernel code be more friendly to @@ -4480,21 +4481,21 @@ For further details, see:

  • Time to move to C11 atomics?
  • Atomic usage patterns in the kernel
  • -

    +

    13 Replacing Print Macros

    -

    +

    13.1 Replacement

    -

    In Section 2, I said that X Window System and kernel module programming do not +

    In Section 2, I said that X Window System and kernel module programming do not mix. That is true for developing kernel modules. But in actual use, you want to be able to send messages to whichever tty the command to load the module came from. -

    "tty" is an abbreviation of teletype: originally a combination keyboard-printer +

    "tty" is an abbreviation of teletype: originally a combination keyboard-printer used to communicate with a Unix system, and today an abstraction for the text stream used for a Unix program, whether it is a physical terminal, an xterm on an X display, a network connection used with ssh, etc. -

    The way this is done is by using current, a pointer to the currently running task, +

    The way this is done is by using current, a pointer to the currently running task, to get the current task’s tty structure. Then, we look inside that tty structure to find a pointer to a string write function, which we use to write a string to the tty. @@ -4577,16 +4578,16 @@ tty. -

    +

    13.2 Flashing keyboard LEDs

    -

    In certain conditions, you may desire a simpler and more direct way to communicate +

    In certain conditions, you may desire a simpler and more direct way to communicate to the external world. Flashing keyboard LEDs can be such a solution: It is an immediate way to attract attention or to display a status condition. Keyboard LEDs are present on every hardware, they are always visible, they do not need any setup, and their use is rather simple and non-intrusive, compared to writing to a tty or a file. -

    From v4.14 to v4.15, the timer API made a series of changes +

    From v4.14 to v4.15, the timer API made a series of changes to improve memory safety. A buffer overflow in the area of a timer_list structure may be able to overwrite the @@ -4609,7 +4610,7 @@ Thus, it is better to use a unique prototype to separate from the cluster that t container_of macro instead of the unsigned long value. For more information see: Improving the kernel timers API. -

    Before Linux v4.14, setup_timer +

    Before Linux v4.14, setup_timer was used to initialize the timer and the timer_list structure looked like: @@ -4624,7 +4625,7 @@ Thus, it is better to use a unique prototype to separate from the cluster that t 8 9void setup_timer(struct timer_list *timer, void (*callback)(unsigned long), 10                 unsigned long data); -

    Since Linux v4.14, timer_setup +

    Since Linux v4.14, timer_setup is adopted and the kernel step by step converting to timer_setup from setup_timer @@ -4638,7 +4639,7 @@ Moreover, the timer_setup

    1void timer_setup(struct timer_list *timer, 
     2                 void (*callback)(struct timer_list *), unsigned int flags);
    -

    The setup_timer +

    The setup_timer was then removed since v4.15. As a result, the timer_list structure had changed to the following. @@ -4649,7 +4650,7 @@ Moreover, the timer_setup 4    u32 flags; 5    /* ... */ 6}; -

    The following source code illustrates a minimal kernel module which, when +

    The following source code illustrates a minimal kernel module which, when loaded, starts blinking the keyboard LEDs until it is unloaded.

    @@ -4738,7 +4739,7 @@ loaded, starts blinking the keyboard LEDs until it is unloaded. 83module_exit(kbleds_cleanup); 84 85MODULE_LICENSE("GPL"); -

    If none of the examples in this chapter fit your debugging needs, +

    If none of the examples in this chapter fit your debugging needs, there might yet be some other tricks to try. Ever wondered what CONFIG_LL_DEBUG in make menuconfig @@ -4749,25 +4750,25 @@ everything what your code does over a serial line. If you find yourself porting kernel to some new and former unsupported architecture, this is usually amongst the first things that should be implemented. Logging over a netconsole might also be worth a try. -

    While you have seen lots of stuff that can be used to aid debugging here, there are +

    While you have seen lots of stuff that can be used to aid debugging here, there are some things to be aware of. Debugging is almost always intrusive. Adding debug code can change the situation enough to make the bug seem to disappear. Thus, you should keep debug code to a minimum and make sure it does not show up in production code. -

    +

    14 Scheduling Tasks

    -

    There are two main ways of running tasks: tasklets and work queues. Tasklets are a +

    There are two main ways of running tasks: tasklets and work queues. Tasklets are a quick and easy way of scheduling a single function to be run. For example, when triggered from an interrupt, whereas work queues are more complicated but also better suited to running multiple things in a sequence. -

    +

    14.1 Tasklets

    -

    Here is an example tasklet module. The +

    Here is an example tasklet module. The tasklet_fn function runs for a few seconds. In the meantime, execution of the example_tasklet_init @@ -4819,7 +4820,7 @@ better suited to running multiple things in a sequence. 42 43MODULE_DESCRIPTION("Tasklet example"); 44MODULE_LICENSE("GPL"); -

    So with this example loaded dmesg +

    So with this example loaded dmesg should show: @@ -4831,23 +4832,23 @@ Example tasklet starts Example tasklet init continues... Example tasklet ends -

    Although tasklet is easy to use, it comes with several defators, and developers are +

    Although tasklet is easy to use, it comes with several defators, and developers are discussing about getting rid of tasklet in linux kernel. The tasklet callback runs in atomic context, inside a software interrupt, meaning that it cannot sleep or access user-space data, so not all work can be done in a tasklet handler. Also, the kernel only allows one instance of any given tasklet to be running at any given time; multiple different tasklet callbacks can run in parallel. -

    In recent kernels, tasklets can be replaced by workqueues, timers, or threaded +

    In recent kernels, tasklets can be replaced by workqueues, timers, or threaded interrupts.1 While the removal of tasklets remains a longer-term goal, the current kernel contains more than a hundred uses of tasklets. Now developers are proceeding with the API changes and the macro DECLARE_TASKLET_OLD exists for compatibility. For further information, see https://lwn.net/Articles/830964/. -

    +

    14.2 Work queues

    -

    To add a task to the scheduler we can use a workqueue. The kernel then uses the +

    To add a task to the scheduler we can use a workqueue. The kernel then uses the Completely Fair Scheduler (CFS) to execute work within the queue.

    @@ -4884,36 +4885,36 @@ Completely Fair Scheduler (CFS) to execute work within the queue. 31 32MODULE_LICENSE("GPL"); 33MODULE_DESCRIPTION("Workqueue example"); -

    +

    15 Interrupt Handlers

    -

    +

    15.1 Interrupt Handlers

    -

    Except for the last chapter, everything we did in the kernel so far we have done as a +

    Except for the last chapter, everything we did in the kernel so far we have done as a response to a process asking for it, either by dealing with a special file, sending an ioctl() , or issuing a system call. But the job of the kernel is not just to respond to process requests. Another job, which is every bit as important, is to speak to the hardware connected to the machine. -

    There are two types of interaction between the CPU and the rest of the +

    There are two types of interaction between the CPU and the rest of the computer’s hardware. The first type is when the CPU gives orders to the hardware, the order is when the hardware needs to tell the CPU something. The second, called interrupts, is much harder to implement because it has to be dealt with when convenient for the hardware, not the CPU. Hardware devices typically have a very small amount of RAM, and if you do not read their information when available, it is lost. -

    Under Linux, hardware interrupts are called IRQ’s (Interrupt ReQuests). There +

    Under Linux, hardware interrupts are called IRQ’s (Interrupt ReQuests). There are two types of IRQ’s, short and long. A short IRQ is one which is expected to take a very short period of time, during which the rest of the machine will be blocked and no other interrupts will be handled. A long IRQ is one which can take longer, and during which other interrupts may occur (but not interrupts from the same device). If at all possible, it is better to declare an interrupt handler to be long. -

    When the CPU receives an interrupt, it stops whatever it is doing (unless it is +

    When the CPU receives an interrupt, it stops whatever it is doing (unless it is processing a more important interrupt, in which case it will deal with this one only when the more important one is done), saves certain parameters on the stack and calls the interrupt handler. This means that certain things are not allowed in the @@ -4925,10 +4926,10 @@ heavy work deferred from an interrupt handler. Historically, BH (Linux naming for Bottom Halves) statistically book-keeps the deferred functions. Softirq and its higher level abstraction, Tasklet, replace BH since Linux 2.3. -

    The way to implement this is to call +

    The way to implement this is to call request_irq() to get your interrupt handler called when the relevant IRQ is received. -

    In practice IRQ handling can be a bit more complex. Hardware is often +

    In practice IRQ handling can be a bit more complex. Hardware is often designed in a way that chains two interrupt controllers, so that all the IRQs from interrupt controller B are cascaded to a certain IRQ from interrupt controller A. Of course, that requires that the kernel finds out which IRQ it @@ -4945,7 +4946,7 @@ need to solve another truckload of problems. It is not enough to know if a certain IRQs has happened, it’s also important to know what CPU(s) it was for. People still interested in more details, might want to refer to "APIC" now. -

    This function receives the IRQ number, the name of the function, +

    This function receives the IRQ number, the name of the function, flags, a name for /proc/interrupts and a parameter to be passed to the interrupt handler. Usually there is a certain number of IRQs available. How many IRQs there are is hardware-dependent. The flags can include @@ -4955,16 +4956,16 @@ How many IRQs there are is hardware-dependent. The flags can include SA_INTERRUPT to indicate this is a fast interrupt. This function will only succeed if there is not already a handler on this IRQ, or if you are both willing to share. -

    +

    15.2 Detecting button presses

    -

    Many popular single board computers, such as Raspberry Pi or Beagleboards, have a +

    Many popular single board computers, such as Raspberry Pi or Beagleboards, have a bunch of GPIO pins. Attaching buttons to those and then having a button press do something is a classic case in which you might need to use interrupts, so that instead of having the CPU waste time and battery power polling for a change in input state, it is better for the input to trigger the CPU to then run a particular handling function. -

    Here is an example where buttons are connected to GPIO numbers 17 and 18 and +

    Here is an example where buttons are connected to GPIO numbers 17 and 18 and an LED is connected to GPIO 4. You can change those numbers to whatever is appropriate for your board.

    @@ -5113,14 +5114,14 @@ appropriate for your board. 142 143MODULE_LICENSE("GPL"); 144MODULE_DESCRIPTION("Handle some GPIO interrupts"); -

    +

    15.3 Bottom Half

    -

    Suppose you want to do a bunch of stuff inside of an interrupt routine. A common +

    Suppose you want to do a bunch of stuff inside of an interrupt routine. A common way to do that without rendering the interrupt unavailable for a significant duration is to combine it with a tasklet. This pushes the bulk of the work off into the scheduler. -

    The example below modifies the previous example to also run an additional task +

    The example below modifies the previous example to also run an additional task when an interrupt is triggered.

    @@ -5294,19 +5295,19 @@ when an interrupt is triggered. 165 166MODULE_LICENSE("GPL"); 167MODULE_DESCRIPTION("Interrupt with top and bottom half"); -

    +

    16 Crypto

    -

    At the dawn of the internet, everybody trusted everybody completely…but that did +

    At the dawn of the internet, everybody trusted everybody completely…but that did not work out so well. When this guide was originally written, it was a more innocent era in which almost nobody actually gave a damn about crypto - least of all kernel developers. That is certainly no longer the case now. To handle crypto stuff, the kernel has its own API enabling common methods of encryption, decryption and your favourite hash functions. -

    +

    16.1 Hash functions

    -

    Calculating and checking the hashes of things is a common operation. Here is a +

    Calculating and checking the hashes of things is a common operation. Here is a demonstration of how to calculate a sha256 hash within a kernel module.

    @@ -5374,20 +5375,20 @@ demonstration of how to calculate a sha256 hash within a kernel module. 62 63MODULE_DESCRIPTION("sha256 hash test"); 64MODULE_LICENSE("GPL"); -

    Install the module: +

    Install the module:

    1sudo insmod cryptosha256.ko 
     2sudo dmesg
    -

    And you should see that the hash was calculated for the test string. -

    Finally, remove the test module: +

    And you should see that the hash was calculated for the test string. +

    Finally, remove the test module:

    1sudo rmmod cryptosha256
    -

    +

    16.2 Symmetric key encryption

    -

    Here is an example of symmetrically encrypting a string using the AES algorithm +

    Here is an example of symmetrically encrypting a string using the AES algorithm and a password.

    @@ -5592,10 +5593,10 @@ and a password. 196 197MODULE_DESCRIPTION("Symmetric key encryption example"); 198MODULE_LICENSE("GPL"); -

    +

    17 Virtual Input Device Driver

    -

    The input device driver is a module that provides a way to communicate +

    The input device driver is a module that provides a way to communicate with the interaction device via the event. For example, the keyboard can send the press or release event to tell the kernel what we want to do. The input device driver will allocate a new input structure with @@ -5603,7 +5604,7 @@ do. The input device driver will allocate a new input structure with and sets up input bitfields, device id, version, etc. After that, registers it by calling input_register_device() . -

    Here is an example, vinput, It is an API to allow easy +

    Here is an example, vinput, It is an API to allow easy development of virtual input drivers. The drivers needs to export a vinput_device() that contains the virtual device name and @@ -5619,7 +5620,7 @@ development of virtual input drivers. The drivers needs to export a

  • the readback function: read()
  • -

    Then using vinput_register_device() +

    Then using vinput_register_device() and vinput_unregister_device() will add a new device to the list of support virtual input devices.

    @@ -5628,7 +5629,7 @@ development of virtual input drivers. The drivers needs to export a

    1int init(struct vinput *);
    -

    This function is passed a struct vinput +

    This function is passed a struct vinput already initialized with an allocated struct input_dev . The init() function is responsible for initializing the capabilities of the input device and register @@ -5636,20 +5637,20 @@ it.

    1int send(struct vinput *, char *, int);
    -

    This function will receive a user string to interpret and inject the event using the +

    This function will receive a user string to interpret and inject the event using the input_report_XXXX or input_event call. The string is already copied from user.

    1int read(struct vinput *, char *, int);
    -

    This function is used for debugging and should fill the buffer parameter with the +

    This function is used for debugging and should fill the buffer parameter with the last event sent in the virtual input device format. The buffer will then be copied to user. -

    vinput devices are created and destroyed using sysfs. And, event injection is done +

    vinput devices are created and destroyed using sysfs. And, event injection is done through a /dev node. The device name will be used by the userland to export a new virtual input device. -

    The class_attribute +

    The class_attribute structure is similar to other attribute types we talked about in section 8:

    @@ -5660,7 +5661,7 @@ virtual input device. 5    ssize_t (*store)(struct class *class, struct class_attribute *attr, 6                    const char *buf, size_t count); 7}; -

    In vinput.c, the macro CLASS_ATTR_WO(export/unexport) +

    In vinput.c, the macro CLASS_ATTR_WO(export/unexport) defined in include/linux/device.h (in this case, device.h is included in include/linux/input.h) will generate the class_attribute structures which are named class_attr_export/unexport. Then, put them into @@ -5670,14 +5671,14 @@ will generate the class_attribute that should be assigned in vinput_class . Finally, call class_register(&vinput_class) to create attributes in sysfs. -

    To create a vinputX sysfs entry and /dev node. +

    To create a vinputX sysfs entry and /dev node.

    1echo "vkbd" | sudo tee /sys/class/vinput/export
    -

    To unexport the device, just echo its id in unexport: +

    To unexport the device, just echo its id in unexport:

    1echo "0" | sudo tee /sys/class/vinput/unexport
    @@ -6138,7 +6139,7 @@ will generate the class_attribute 400 401MODULE_LICENSE("GPL"); 402MODULE_DESCRIPTION("Emulate input events"); -

    Here the virtual keyboard is one of example to use vinput. It supports all +

    Here the virtual keyboard is one of example to use vinput. It supports all KEY_MAX keycodes. The injection format is the KEY_CODE such as defined in include/linux/input.h. A positive value means @@ -6146,12 +6147,12 @@ will generate the class_attribute while a negative value is a KEY_RELEASE . The keyboard supports repetition when the key stays pressed for too long. The following demonstrates how simulation work. -

    Simulate a key press on "g" ( KEY_G +

    Simulate a key press on "g" ( KEY_G = 34):

    1echo "+34" | sudo tee /dev/vinput0
    -

    Simulate a key release on "g" ( KEY_G +

    Simulate a key release on "g" ( KEY_G = 34):

    @@ -6269,13 +6270,13 @@ following demonstrates how simulation work. 108 109MODULE_LICENSE("GPL"); 110MODULE_DESCRIPTION("Emulate keyboard input events through /dev/vinput"); -

    +

    18 Standardizing the interfaces: The Device Model

    -

    Up to this point we have seen all kinds of modules doing all kinds of things, but there +

    Up to this point we have seen all kinds of modules doing all kinds of things, but there was no consistency in their interfaces with the rest of the kernel. To impose some consistency such that there is at minimum a standardized way to start, suspend and resume a device a device model was added. An example is shown below, and you can @@ -6382,13 +6383,13 @@ functions. 97 98MODULE_LICENSE("GPL"); 99MODULE_DESCRIPTION("Linux Device Model example"); -

    +

    19 Optimizations

    -

    +

    19.1 Likely and Unlikely conditions

    -

    Sometimes you might want your code to run as quickly as possible, +

    Sometimes you might want your code to run as quickly as possible, especially if it is handling an interrupt or doing something which might cause noticeable latency. If your code contains boolean conditions and if you know that the conditions are almost always likely to evaluate as either @@ -6407,7 +6408,7 @@ to succeed. 4    bio = NULL; 5    goto out; 6} -

    When the unlikely +

    When the unlikely macro is used, the compiler alters its machine instruction output, so that it continues along the false branch and only jumps if the condition is true. That avoids flushing the processor pipeline. The opposite happens if you use the @@ -6416,34 +6417,34 @@ avoids flushing the processor pipeline. The opposite happens if you use the -

    +

    20 Common Pitfalls

    -

    +

    20.1 Using standard libraries

    -

    You can not do that. In a kernel module, you can only use kernel functions which are +

    You can not do that. In a kernel module, you can only use kernel functions which are the functions you can see in /proc/kallsyms. -

    +

    20.2 Disabling interrupts

    -

    You might need to do this for a short time and that is OK, but if you do not enable +

    You might need to do this for a short time and that is OK, but if you do not enable them afterwards, your system will be stuck and you will have to power it off. -

    +

    21 Where To Go From Here?

    -

    For people seriously interested in kernel programming, I recommend kernelnewbies.org +

    For people seriously interested in kernel programming, I recommend kernelnewbies.org and the Documentation subdirectory within the kernel source code which is not always easy to understand but can be a starting point for further investigation. Also, as Linus Torvalds said, the best way to learn the kernel is to read the source code yourself. -

    If you would like to contribute to this guide or notice anything glaringly wrong, +

    If you would like to contribute to this guide or notice anything glaringly wrong, please create an issue at https://github.com/sysprog21/lkmpg. Your pull requests will be appreciated. -

    Happy hacking! +

    Happy hacking!

    -

    1The goal of threaded interrupts is to push more of the work to separate threads, so that the +

    1The goal of threaded interrupts is to push more of the work to separate threads, so that the minimum needed for acknowledging an interrupt is reduced, and therefore the time spent handling the interrupt (where it can’t handle any other interrupts at the same time) is reduced. See https://lwn.net/Articles/302043/.