Linux Kernel scatterlist API introduction

1. Preface

We often see the shadow of scatterlist in subsystems that need to interact with a large amount of data in the user space (for example, Mmc[1], Video, audio, and so on). For those of us who are "non-native English", the first word is the first, the head is instantly circled. Scatter can be translated into "scatter, scatter," and list is the meaning of "lists", thus scatterlist can be translated as "hash list". What is a "hash table"? It's too abstract!

Abstract because this word omits the subject----physical memory (physical memories), plus, it is good to understand more, both: the physical memory of the hash list. More popular, is to put some scattered physical memory, organized in the form of a list. Well, perhaps you would ask, what is the use of it?

Of course, it is useful to refer to the following introduction of this article.

2. Background of scatterlist generation

I didn't go to the Scatterlist API was introduced in which kernel version (the age is too old), by guessing, I think it should be related to the MMU. Since the introduction of the MMU, the software in the Linux system will have to face a problem (the following will be illustrated in Figure 1 as an example of the system architecture):

Suppose in a system (see picture 1 below) There are three modules that can access the MEMORY:CPU, DMA controller, and a peripheral. The CPU accesses the MEMORY;DMA as a virtual address (VA) through the MMU and accesses the memory directly as a physical address (PA) in the form of a device address (DA) through its own iommu.

Then, a "software entity" allocates and uses a piece of storage space (see Figure 2 below). The storage space is contiguous in the CPU view (virtual space) and the starting address is VA1 (in fact, it is mapped to 3 discrete physical memory, which we represent as PA1,PA2,PA3).

Then, if the software accesses this space (Operation VA1) purely from a CPU perspective, then there is no problem at all, as the MMU implements a continuous VA-to-discontinuous PA mapping.

However, if the software after a series of operations, to the DMA controller to the storage space, and eventually by the DMA controller to move the data to a peripheral, because the DMA controller can only access the physical address, only "discontinuous physical memory block" Submitted for the unit (not the virtual address we are familiar with).

At this time, Scatterlist was born: For convenience, we need to use a data structure to describe the "discontinuous physical memory block" (starting address, length and other information), this data structure is scatterlist (refer to the following 3rd chapter of the description). and multiple scatterlist together to form a table (can be a struct scatterlist type array, also can be kernel help abstraction out of the struct sg_table), you can complete the description of the virtual address.

Finally, essentially: thescatterlist (array) is a medium for a variety of different address mapping spaces (PA, VA, DA, and so on) (because the physical address is real, it exists, and thus can be used as a common language), These mapping spaces can be converted to each other (for example, from VA to da).

Image 1 Cpu_dma_device_memory

Image 2 Cpu_view_memory

3. Scatterlist API Introduction 3.1 struct SCATTERLIST

struct Scatterlist is used to describe a contiguous block of memory (in page) on a physical address, which is defined in "Include/linux/scatterlist.h", as follows:

struct Scatterlist { #ifdef Config_debug_sg         unsigned long   sg_magic; #endif         unsigned long   page_link;         unsigned int    offset;         unsigned int    length;         dma_addr_t      dma_address; #ifdef config_need_sg_dma_length         unsigned int    Dma_length; #endif };

A page_link that indicates the page on which the memory block resides. Bit0 and bit1 have a special purpose (refer to the following introduction), so the page minimum of 4 bytes is required to align.
Offset that indicates the offset (starting position) of the memory block in the page.
Length of the memory block.

Dma_address, the actual starting address of the memory block (PA, which is closer to our human language than the page).
Dma_length, the corresponding length information.

3.2 struct sg_table

In a real-world scenario, a single scatterlist is meaningless, and we need multiple scatterlist to form an array to represent the virtual address space that is physically discontinuous. Typically, a module that uses the Scatterlist function maintains the array (pointers and lengths) itself, such as the struct mmc_data mentioned in [2]:

struct Mmc_data { ... unsigned int sg_len; /* Size of scatter list */ struct Scatterlist *sg; /* I/O scatter list */ S32 Host_cookie; /* Host Private Data */ };

However, for the user can be lazy, kernel abstract out a simple data structure: struct sg_table, help to save the scatterlist array pointer and length:

struct Sg_table { struct Scatterlist *sgl; /* the list */ unsigned int nents; /* Number of mapped entries */ unsigned int orig_nents; /* Original size of list */ };

Where SGL is the first address of an array of memory blocks, Orig_nents is a memory block array of size,nents is a valid number of memory blocks (may be less than orig_nents).

The above thoughts are more direct, but a little, we have to carefully understand:

How many valid blocks of memory are there in the scatterlist array? This is not a very intuitive thing, mainly the following 2 rules determine:

1) If the bit0 of a scatterlist page_link in the scatterlist array is 1, it means that the scatterlist is not a valid block of memory, but a chain (hinge), pointing to another scatterlist array. With this mechanism, different scatterlist arrays can be chained together, because Scatterlist is also known as chain Scatterlist.

2) If the bit1 of a scatterlist page_link in the scatterlist array is 1, the scatterlist is the last valid block of memory in the Scatterlist array (which is ignored later).

3.3 API Introduction

After understanding the meaning of scatterlist, it is much easier to look at the API in "Include/linux/scatterlist.h", for example (brief introduction, no more detailed analysis):

#define SG_DMA_ADDRESS (SG) ((SG)->dma_address) #ifdef Config_need_sg_dma_length #define SG_DMA_LEN (SG) ((SG)->dma_length) #else #define SG_DMA_LEN (SG) ((SG)->length) #endif

Sg_dma_address, Sg_dma_len, gets the physical address and length of a scatterlist.

#define SG_IS_CHAIN (SG) ((SG)->page_link & 0x01) #define SG_IS_LAST (SG) ((SG)->page_link & 0x02) #define SG_CHAIN_PTR (SG) \ (struct scatterlist *) ((SG)->page_link & ~0x03))

Sg_is_chain can be used to determine whether a scatterlist is a chain,sg_is_last available to determine whether a scatterlist is the last sg_table in Scatterlist.

Sg_chain_ptr can get the scatterlist that chain scatterlist points to.

static inline void Sg_assign_page (struct scatterlist *sg, struct page *page) static inline void Sg_set_page (struct scatterlist *sg, struct page *page, unsigned int len, unsigned int offset) Static inline struct page *sg_page (struct scatterlist *sg) static inline void Sg_set_buf (struct scatterlist *sg, const void *BUF, unsigned int buflen) #define FOR_EACH_SG (sglist, SG, NR, __i) \ for (__i = 0, sg = (sglist), __i < (NR), __i++, sg = sg_next (SG)) static inline void Sg_chain (struct scatterlist *prv, unsigned int prv_nents, struct Scatterlist *SGL) static inline void Sg_mark_end (struct scatterlist *sg) static inline void Sg_unmark_end (struct scatterlist *sg) Static inline dma_addr_t Sg_phys (struct scatterlist *sg) static inline void *sg_virt (struct scatterlist *sg)

Sg_assign_page, assigns the page to the specified scatterlist (sets the Page_link field).
Sg_set_page, assigns the specified offset in the page, and specifies the length of memory to the specified scatterlist (set Page_link, offset, len fields).
Sg_page, gets the page pointer that corresponds to the scatterlist.
SG_SET_BUF, assigns the specified length of buffer to scatterlist (gets the page pointer from the virtual address, after offset in the page, and then calls Sg_set_page).

FOR_EACH_SG, traversing all valid scatterlist in a scatterlist array (sglist) (consider Sg_is_chain and sg_is_last).

Sg_chain, bundling two scatterlist arrays together.

Sg_mark_end, Sg_unmark_end, the last one is marked (or not marked) by a scatterlist.

Sg_phys, Sg_virt, gets the physical or virtual address of a scatterlist.

And so on (no longer listed, interested students directly to see the code on the line).

Linux Kernel scatterlist API introduction