Flashcache memory and disk, disk and disk Io

In flashcache, disk-related reads and writes are divided into the following two types: 1) interaction between disk and memory 2) interaction between disk and disk, for example, when a read Miss occurs, it is directly read from the disk, in 1st cases, what about read hits? This is also the case in 1st cases, but it is read from SSD at this time. The interaction between a disk and a disk is used to write dirty data and copy the dirty cache block in the SSD to the disk. Now we will introduce the interface functions used in the next two cases, so that the two functions will be very friendly when you look at the read/write process, and clearly know where the data flows. In Case 1, two functions are dm_io_async_bvec and flashcache_dm_io_async_vm. 

int dm_io_async_bvec(unsigned int num_regions, #if LINUX_VERSION_CODE >= KERNEL_VERSION(2,6,26)                struct dm_io_region *where, #else                struct io_region *where, #endif                int rw,                 struct bio_vec *bvec, io_notify_fn fn,                 void *context){    struct dm_io_request iorq;    iorq.bi_rw = rw;    iorq.mem.type = DM_IO_BVEC;    iorq.mem.ptr.bvec = bvec;    iorq.notify.fn = fn;    iorq.notify.context = context;    iorq.client = flashcache_io_client;    return dm_io(&iorq, num_regions, where, NULL);}#endif

 

int flashcache_dm_io_async_vm(struct cache_c *dmc, unsigned int num_regions, #if LINUX_VERSION_CODE < KERNEL_VERSION(2,6,26)              struct io_region *where, #else              struct dm_io_region *where, #endif              int rw,              void *data, io_notify_fn fn, void *context){    unsigned long error_bits = 0;    int error;    struct dm_io_request io_req = {        .bi_rw = rw,        .mem.type = DM_IO_VMA,        .mem.ptr.vma = data,        .mem.offset = 0,        .notify.fn = fn,        .notify.context = context,        .client = flashcache_io_client,    };    error = dm_io(&io_req, 1, where, &error_bits);    if (error)        return error;    if (error_bits)        return error_bits;    return 0;}#endif

The above two functions use struct dm_io_request to encapsulate the request. Only two types of requests are different. The first function correspondsDm_io_bvec,The second function isDm_io_vma.

As a matter of fact, I have never understood why I want to use these two functions to deal with the hard disk and memory, but later I saw dm_io and found that there are multiple Io service types, the two functions correspond to different Io types respectively. Next, let's take a look at the dm_io-related data structure.

Dm_io

DM-io provides synchronous or asynchronous Io services for device mapper.

When using DM-Io, you must set the dm_io_region structure (io_region earlier than 2.6.26). This structure defines the region of the IO operation. Reading is generally targeted at a dm_io_region zone, write can be targeted at a group of dm_io_region zones.

struct dm_io_region {    struct block_device *bdev;    sector_t sector;    sector_t count;         /* If this is zero the region is ignored. */};

For earlier kernel versions, you must set an io_region structure to describe the expected I/O location. Each io_region indicates a block device with a start position and length in the region.

struct io_region {      struct block_device *bdev;      sector_t sector;      sector_t count;   };

DM-io can be read from or written to one or more io_region instances. An io_region structure array specifies that data is written to multiple regions.

DM-io has four dm_io_mem_type types (the earlier kernel version has only the first three types, and flashcache mainly uses dm_io_bvec ):

enum dm_io_mem_type {    DM_IO_PAGE_LIST,/* Page list */    DM_IO_BVEC,     /* Bio vector */    DM_IO_VMA,      /* Virtual memory area */    DM_IO_KMEM,     /* Kernel memory */};struct dm_io_memory {    enum dm_io_mem_type type;    union {            struct page_list *pl;            struct bio_vec *bvec;            void *vma;            void *addr;    } ptr;    unsigned offset;};

DM-io provides synchronous and asynchronous I/O services. The old Kernel provides three I/O services, each of which has a synchronous and asynchronous version. Dm_io_page_listThe first I/O service type uses a string of memory pages as the buffer, along with an offset on the home page.

    struct page_list {      struct page_list *next;      struct page *page;   }; int dm_io_sync(unsigned int num_regions, struct io_region *where, int rw,                  struct page_list *pl, unsigned int offset,                  unsigned long *error_bits);   int dm_io_async(unsigned int num_regions, struct io_region *where, int rw,                   struct page_list *pl, unsigned int offset,                   io_notify_fn fn, void *context);

 

Dm_io_bvecThe second type of I/O service caches a group of bio carriers in front of I/O data. If the caller assembles bio in advance, this service can be completed smoothly. However, you need to direct different bio pages to different devices.

    int dm_io_sync_bvec(unsigned int num_regions, struct io_region *where,                       int rw, struct bio_vec *bvec,                       unsigned long *error_bits);   int dm_io_async_bvec(unsigned int num_regions, struct io_region *where,                        int rw, struct bio_vec *bvec,                        io_notify_fn fn, void *context); 

Dm_io_vmaThe third type of I/O service uses a pointer to the virtual dynamic memory buffer as the data buffer of I/O. If the caller needs to perform I/O operations on a large block device and does not want to allocate a large number of personal memory pages, the service is competent.

 int dm_io_sync_vm(unsigned int num_regions, struct io_region *where, int rw,                     void *data, unsigned long *error_bits);   int dm_io_async_vm(unsigned int num_regions, struct io_region *where, int rw,                      void *data, io_notify_fn fn, void *context);

The caller of the asynchronous I/O service must include a complete callback function and a pointer to some of the I/O content data. Typedef void (* io_policy_fn) (unsigned long error, void * context); this "error" parameter is like this "* error" parameter in any synchronization version, in this callback function, it is like a set of single digits (rather than a simple error value ).
When I/O is written to multiple target regions, this bit set allows DM-Io to indicate the success or failure of each region. Before using any DM-io service, you must call dm_io_get () and specify the number of pages they want to execute I/O. DM-Io will try to change the size of its memory pool to confirm that there are enough pages for the execution of I/O to avoid unnecessary waiting.
When the user completes using the I/O service, they will call dm_io_put () and specify the same number of pages to be assigned to dm_io_get.

DM-io encapsulates the request type through the dm_io_request structure. If dm_io_policy.fn is set, it is asynchronous IO; otherwise, it is synchronous Io.

struct dm_io_request {    int bi_rw;                      /* READ|WRITE - not READA */    struct dm_io_memory mem;        /* Memory to use for io */    struct dm_io_notify notify;     /* Synchronous if notify.fn is NULL */    struct dm_io_client *client;    /* Client memory handler */};

Before using the dm_io service, you must use the dm_io_client_create function (dm_io_get before 2.6.22) to create the dm_io_client structure and allocate a memory pool for DM-io execution. After the DM-io service is used, call the dm_io_client_destroy function (dm_io_put before 2.6.22) to release the memory pool.

struct dm_io_client {    mempool_t *pool;    struct bio_set *bios;};

 

The DM-io function executes specific IO requests.

int dm_io(struct dm_io_request *io_req, unsigned num_regions,      struct dm_io_region *where, unsigned long *sync_error_bits){    int r;    struct dpages dp;    r = dp_init(io_req, &dp);    if (r)            return r;    if (!io_req->notify.fn)            return sync_io(io_req->client, num_regions, where,                           io_req->bi_rw, &dp, sync_error_bits);    return async_io(io_req->client, num_regions, where, io_req->bi_rw,                    &dp, io_req->notify.fn, io_req->notify.context);}

In the second case, the interaction between the disk and the disk. This case is only used to write the dirty blocks in the SSD into the disk.

int dm_kcopyd_copy(struct dm_kcopyd_client *kc, struct dm_io_region *from,                   unsigned int num_dests, struct dm_io_region *dests,                   unsigned int flags, dm_kcopyd_notify_fn fn, void *context)

The first parameter is dm_kcopyd_client. When using the kcopyd asynchronous copy service, you must first create a corresponding client. First, you must allocate the "kcopyd client" structure and call the function as follows:

Kcopyd_client_create (flashcache_copy_pages, & flashcache_kcp_client );

Create the dm_kcopyd_client structure.

The second parameter dm_io_region is the source address, and the fourth parameter is the destination address. The definition of struct dm_io_region {
Struct block_device * bdev;
Sector_t sector;
Sector_t count;/* If this is zero the region is ignored .*/
}; Dm_kcopyd_policy_fn is the callback function of kcopyd after processing the request. context is the callback function parameter, and the corresponding kcached_job is set in flashcache.