Linux kernel structure--KFIFO ring buffer

1. Preface

A ring buffer was used in the recent project, and the code was changed by the Kfifo of the Linux kernel. Buffers are often used in the file system to alleviate the CPU's read and write memory and the speed of reading and writing disks through buffers. For example, when a process a produces data to another process B, process B needs to process the data passed on by processes and writes to the file, and if B is not finished, a delays sending. To ensure that process a reduces the wait time, a buffer can be used between A and B, and a each time the data is stored in the buffer and B is taken in each buffer. This is a typical producer and consumer model in which data in the buffer satisfies the FIFO feature and can therefore be implemented in a queue. The Linux kernel's kfifo is exactly a ring queue that can be used as a ring buffer. Producers and consumers use buffers as shown:

The detailed introduction of the ring buffer and the implementation method can refer to Http://en.wikipedia.org/wiki/Circular_buffer, introduced in very detailed, listed the implementation of the ring queue of several methods. The inconvenience of the ring queue is how to determine whether the queue is empty or full. There are three ways to implement this on Wikipedia.

2. Linux kernel Kfifo

The Kfifo design is very ingenious, the code is very concise, for the queue and out of the handling of the unexpected. First look at the data structure of KFIFO:

struct Kfifo {    unsigned char *buffer;     /* The buffer holding the data */    unsigned int size;         /* The size of the allocated buffer */    unsigned int in;           /* data is added at offset (in% size) */    unsigned int out;          /* data is extracted from off. (out% size) */    spinlock_t *lock;          /* Protects concurrent modifications */};

KFIFO provides the following methods:

Creates a kfifostruct Kfifo *kfifo_init (unsigned char *buffer, unsigned int size,                gfp_t gfp_mask, spinlock_t *) based on the given buffer lock);//The given size allocates buffer and kfifostruct Kfifo *kfifo_alloc (unsigned int size, gfp_t gfp_mask,                 spinlock_t *lock);// Free kfifo space void Kfifo_free (struct kfifo *fifo)//Add data kfifo int unsigned to kfifo_put (struct KFIFO *fifo,                const unsigned char *buffer, unsigned int len)//Fetch data from Kfifo unsigned int kfifo_put (struct KFIFO *fifo,                const unsigned char *buffer, uns igned int len)//Gets buffer size of data in Kfifo unsigned int kfifo_len (struct Kfifo *fifo)

Defines the purpose of the spin lock in order to prevent multi-process/thread concurrency using KFIFO. Because in and out change at each get and out. The source code for initializing and creating Kfifo is as follows:

struct Kfifo *kfifo_init (unsigned char *buffer, unsigned int size,             gfp_t gfp_mask, spinlock_t *lock) {    struct KFIF o *fifo;    /* Size must be a power of 2 */    bug_on (!is_power_of_2 (size));    FIFO = Kmalloc (sizeof (struct kfifo), gfp_mask);    if (!FIFO)        return err_ptr (-ENOMEM);    Fifo->buffer = buffer;    fifo->size = size;    Fifo->in = fifo->out = 0;    Fifo->lock = lock;    return FIFO;} struct KFIFO *kfifo_alloc (unsigned int size, gfp_t gfp_mask, spinlock_t *lock) {    unsigned char *buffer;    struct Kfifo *ret;    if (!is_power_of_2 (size)) {        bug_on (Size > 0x80000000);        Size = roundup_pow_of_two (size);    }    Buffer = Kmalloc (size, gfp_mask);    if (!buffer)        return err_ptr (-ENOMEM);    RET = kfifo_init (buffer, size, gfp_mask, lock);    if (Is_err (ret))        kfree (buffer);    return ret;}

In Kfifo_init and Kfifo_calloc, the value of Kfifo->size is always extended to a power of 2 on the basis of the size parameter passed in by the caller, which is the kernel's usual practice. The benefits are self-evident--the kfifo->size modulo operation can be translated into operations, such as: kfifo->in% kfifo->size can be converted to Kfifo->in & (kfifo->size–1)

The clever thing about Kfifo is that in and out are defined as unsigned types, both in and out are incremented when put and get, and overflow when the maximum is reached, so that it is recycled from 0 onwards. The put and get codes are as follows:

static inline unsigned int kfifo_put (struct Kfifo *fifo, const unsigned char *buffer, unsigned int len) {    unsigned long flags;    unsigned int ret;    Spin_lock_irqsave (Fifo->lock, flags);    ret = __kfifo_put (FIFO, buffer, Len);    Spin_unlock_irqrestore (Fifo->lock, flags); return ret;}    static inline unsigned int kfifo_get (struct Kfifo *fifo, unsigned char *buffer, unsigned int len) {    unsigned long flags;    unsigned int ret;    Spin_lock_irqsave (Fifo->lock, flags);        ret = __kfifo_get (FIFO, buffer, Len);    When fifo->in = = Fifo->out, Buufer is empty if (fifo->in = = fifo->out) fifo->in = fifo->out = 0;    Spin_unlock_irqrestore (Fifo->lock, flags); return ret;}        unsigned int __kfifo_put (struct Kfifo *fifo, const unsigned char *buffer, unsigned int len) {unsigned int l;    Buffer Hollow Length len = min (len, fifo->size-fifo->in + fifo->out); /* * Ensure that we sample the fifo-≫out index-before-we * Start putting bytes into the Kfifo.    */SMP_MB (); /* First put the data starting from Fifo->in to buffer end */L = min (len, Fifo->size-(Fifo->in & (fifo-    (>size-1)));    memcpy (Fifo->buffer + (Fifo->in & (fifo->size-1)), buffer, L);    /* Then put the rest (if any) at the beginning of the buffer */memcpy (fifo->buffer, buffer + L, len-l);     /* * Ensure that we add the bytes to the kfifo-before-* We update the fifo->in index.    */SMP_WMB ();  Fifo->in + = Len; Each time it accumulates, it overflows after reaching its maximum value and automatically turns to 0 return Len;        unsigned int __kfifo_get (struct Kfifo *fifo, unsigned char *buffer, unsigned int len) {unsigned int l;    The length of the buffer with data len = min (len, fifo->in-fifo->out);     /* * Ensure that we are sample the Fifo->in index-before-we * Start removing bytes from the Kfifo.    */SMP_RMB (); /* First get the data from Fifo->out until the end of the buffer */    L = min (len, Fifo->size-(Fifo->out & (Fifo->size-1)));    memcpy (buffer, Fifo->buffer + (Fifo->out & (Fifo->size-1)), L);    /* Then get the rest (if any) from the beginning of the buffer */memcpy (buffer + L, Fifo->buffer, len-l);     /* * Ensure that we remove the bytes from the kfifo-before-* We update the fifo->out index.    */SMP_MB (); Fifo->out + = Len; Each time it accumulates, it overflows after reaching its maximum value and automatically turns to 0 return Len;

Both put and get are locked during calls to the __put and __get procedures to prevent concurrency. You can see from the code that both put and get call two times memcpy, which is for boundary conditions. For example: Blue indicates idle, red indicates occupancy.

(1) Empty Kfifo,

(2) after put a buffer

(3) Get a buffer after

(4) When the buffer length of the put is beyond the length of the in to the end, then move the rest to the head

3. Test procedure

After writing a ring_buffer in Kfifo, the existing thread mutex is concurrency controlled. The ring_buffer of the design are as follows:

/** @brief The ring buffer * @atuher Anker date:2013-12-18* ring_buffer.h * * */#ifndef kfifo_header_h #define KFI written in Linux Kfifo Fo_header_h#include <inttypes.h> #include <string.h> #include <stdlib.h> #include <stdio.h># Include <errno.h> #include <assert.h>//determine if X is a 2-square # # is_power_of_2 (x) ((x)! = 0 && (((x) & (( x)-1)) = = 0)//Take the minimum value of A and B # # min (A, B) ((a) < (b))?     (a): (b) struct ring_buffer{void *buffer;       Buffer uint32_t size;         Size uint32_t in;        Inlet position uint32_t out;    Exit position pthread_mutex_t *f_lock; Mutex};//initialization buffer struct ring_buffer* ring_buffer_init (void *buffer, uint32_t size, pthread_mutex_t *f_lock) {assert (buff    ER);    struct Ring_buffer *ring_buf = NULL;        if (!is_power_of_2 (size)) {fprintf (stderr, "size must be power of 2.\n");    return ring_buf;    } ring_buf = (struct Ring_buffer *) malloc (sizeof (struct ring_buffer)); if (!ring_buf) {FPRintf (stderr, "Failed to malloc memory,errno:%u,reason:%s", errno, Strerror (errno));    return ring_buf;    } memset (ring_buf, 0, sizeof (struct ring_buffer));    Ring_buf->buffer = buffer;    ring_buf->size = size;    ring_buf->in = 0;        ring_buf->out = 0;    Ring_buf->f_lock = F_lock; return ring_buf;}        Release buffer void Ring_buffer_free (struct ring_buffer *ring_buf) {if (ring_buf) {if (Ring_buf->buffer) {        Free (ring_buf->buffer);    Ring_buf->buffer = NULL;    } free (RING_BUF);    Ring_buf = NULL; The length of the}}//buffer is uint32_t __ring_buffer_len (const struct Ring_buffer *ring_buf) {return (ring_buf->in-ring_buf->out); }//fetching data from buffer uint32_t __ring_buffer_get (struct ring_buffer *ring_buf, void * buffer, uint32_t size) {assert (Ring_buf | | b    Uffer);    uint32_t len = 0;            size = min (size, ring_buf->in-ring_buf->out); /* First get the data from Fifo->out until the end of the buffer */len = min (sIze, Ring_buf->size-(Ring_buf->out & (Ring_buf->size-1)));    memcpy (buffer, Ring_buf->buffer + (Ring_buf->out & (Ring_buf->size-1)), Len); /* Then get the rest (if any) from the beginning of the buffer */memcpy (buffer + len, Ring_buf->buffer, Size-len)    ;    Ring_buf->out + = size; return size;} Hold data to buffer uint32_t __ring_buffer_put (struct ring_buffer *ring_buf, void *buffer, uint32_t size) {assert (Ring_buf | | bu    Ffer);    uint32_t len = 0;    size = min (size, ring_buf->size-ring_buf->in + ring_buf->out); /* First put the data starting from Fifo->in to buffer end */len = min (Size, ring_buf->size-(ring_buf->in    & (Ring_buf->size-1)));    memcpy (Ring_buf->buffer + (Ring_buf->in & (ring_buf->size-1)), buffer, Len);    /* Then put the rest (if any) at the beginning of the buffer */memcpy (ring_buf->buffer, buffer + len, Size-len);    Ring_buf->in + = size; return size;} Uint32_T Ring_buffer_len (const struct Ring_buffer *ring_buf) {uint32_t len = 0;    Pthread_mutex_lock (Ring_buf->f_lock);    Len = __ring_buffer_len (RING_BUF);    Pthread_mutex_unlock (Ring_buf->f_lock); return Len;}    uint32_t ring_buffer_get (struct ring_buffer *ring_buf, void *buffer, uint32_t size) {uint32_t ret;    Pthread_mutex_lock (Ring_buf->f_lock);    ret = __ring_buffer_get (ring_buf, buffer, size);    There is no data in buffer if (ring_buf->in = = ring_buf->out) ring_buf->in = ring_buf->out = 0;    Pthread_mutex_unlock (Ring_buf->f_lock); return ret;}    uint32_t ring_buffer_put (struct ring_buffer *ring_buf, void *buffer, uint32_t size) {uint32_t ret;    Pthread_mutex_lock (Ring_buf->f_lock);    ret = __ring_buffer_put (ring_buf, buffer, size);    Pthread_mutex_unlock (Ring_buf->f_lock); return ret;} #endif

Use multithreaded simulation for producers and consumers to write test programs as follows:

/** @brief Ring Buffer test program, creating two threads, one producer, one consumer. * Producers input data to buffer every 1 seconds, and consumers fetch data every 2 seconds. * @atuher Anker date:2013-12-18 * */#include "ring_buffer.h" #include <pthread.h> #include <time.h> #define    Buffer_size 1024x768 * 1024typedef struct student_info{uint64_t stu_id;    uint32_t age; uint32_t score;}    Student_info;void print_student_info (const student_info *stu_info) {assert (Stu_info);    printf ("id:%lu\t", stu_info->stu_id);    printf ("age:%u\t", stu_info->age); printf ("score:%u\n", Stu_info->score);}    Student_info * Get_student_info (time_t timer) {Student_info *stu_info = (Student_info *) malloc (sizeof (student_info));    if (!stu_info) {fprintf (stderr, "Failed to malloc memory.\n");    return NULL;    } srand (timer);    STU_INFO-&GT;STU_ID = 10000 + rand ()% 9999;    Stu_info->age = rand ()% 30;    Stu_info->score = rand ()% 101;    Print_student_info (Stu_info); return stu_info;} void * Consumer_proc (void *arg) {struct Ring_buffer *ring_buf = (strUCT Ring_buffer *) arg;     Student_info Stu_info;    while (1) {sleep (2);    printf ("------------------------------------------\ n");    printf ("Get a student info from ring buffer.\n");    Ring_buffer_get (Ring_buf, (void *) &stu_info, sizeof (Student_info));    printf ("Ring buffer length:%u\n", Ring_buffer_len (RING_BUF));    Print_student_info (&stu_info);    printf ("------------------------------------------\ n"); } return (void *) ring_buf;}    void * Producer_proc (void *arg) {time_t cur_time;    struct Ring_buffer *ring_buf = (struct ring_buffer *) arg;    while (1) {time (&cur_time);    Srand (Cur_time);    int seed = rand ()% 11111;    printf ("******************************************\n");    Student_info *stu_info = get_student_info (cur_time + seed);    printf ("Put a student info to ring buffer.\n");    Ring_buffer_put (Ring_buf, (void *) Stu_info, sizeof (Student_info));    printf ("Ring buffer length:%u\n", Ring_buffer_len (RING_BUF)); printf ("************\ n ");    Sleep (1); } return (void *) ring_buf;}    int consumer_thread (void *arg) {int err;    pthread_t Tid;    Err = Pthread_create (&tid, NULL, Consumer_proc, ARG); if (err! = 0) {fprintf (stderr, "Failed to create consumer thread.errno:%u, reason:%s\n", errno, Strerror (err    NO));    return-1; } return to Tid;}    int producer_thread (void *arg) {int err;    pthread_t Tid;    Err = Pthread_create (&tid, NULL, Producer_proc, ARG); if (err! = 0) {fprintf (stderr, "Failed to create consumer thread.errno:%u, reason:%s\n", errno, Strerror (err    NO));    return-1; } return to Tid;}    int main () {void * buffer = NULL;    uint32_t size = 0;    struct Ring_buffer *ring_buf = NULL;    pthread_t Consume_pid, Produce_pid;    pthread_mutex_t *f_lock = (pthread_mutex_t *) malloc (sizeof (pthread_mutex_t)); if (Pthread_mutex_init (F_lock, NULL)! = 0) {fprintf (stderr, "Failed init mutex,errno:%u,reason:%s\n", ErrnO, Strerror (errno));    return-1;    Buffer = (void *) malloc (buffer_size);    if (!buffer) {fprintf (stderr, "Failed to malloc memory.\n");    return-1;    } size = Buffer_size;    Ring_buf = ring_buffer_init (buffer, size, f_lock);    if (!ring_buf) {fprintf (stderr, "Failed to init ring buffer.\n");    return-1;    } #if 0 Student_info *stu_info = get_student_info (638946124);    Ring_buffer_put (Ring_buf, (void *) Stu_info, sizeof (Student_info));    Stu_info = Get_student_info (976686464);    Ring_buffer_put (Ring_buf, (void *) Stu_info, sizeof (Student_info));    Ring_buffer_get (Ring_buf, (void *) Stu_info, sizeof (Student_info));    Print_student_info (stu_info); #endif printf ("Multi thread test.......\n");    Produce_pid = Producer_thread ((void*) ring_buf);    Consume_pid = Consumer_thread ((void*) ring_buf);    Pthread_join (Produce_pid, NULL);    Pthread_join (Consume_pid, NULL);    Ring_buffer_free (RING_BUF);    Free (f_lock); return 0;}

Summarize:

len = min (len, fifo->size-fifo->in + fifo->out);
InLenAnd(fifo->size-fifo->in + fifo->out)A smaller value is assigned to Len. Note that when(fifo->in = = fifo->out+fifo->size), the buffer is full, and the resulting smaller value must be 0, and the number of bytes that are actually written after it is all 0.
Another boundary condition is whenLenVery large (because Len is unsigned, negative numbers are a big positive number for it), this sentence also guarantees that Len takes a smaller value becauseFifo->inAlways greater than or equal toFifo->out, so the expression that followsl = min (len, Fifo->size-(Fifo->in & (Fifo->size-1)));Does not exceed the value offifo->sizeThe size.
SMP_MB (); SMP_WMB ();Is the memory barrier, this is not the scope of our discussion, you can ignore it.
l = min (len, Fifo->size-(Fifo->in & (Fifo->size-1)));Is the number of bytes to write in the previous stepLen"Cut", and here is a technique used. Note: The actual assignment toFifo->bufferThe number of bytesfifo->size, it must be a power of 2, otherwise there will be an error. Sincefifo->sizeis a power of 2, then(fifo->size-1)That is, a few 1 of the number in the back, can guarantee(Fifo->in & (fifo->size-1))The total is not more than(fifo->size-1)That part of, and(fifo->in)% (fifo->size-1)The same effect.
So the code behind is not difficult to understand,It writes the data to the fifo->in to the end of the buffer, and if it is not finished, writes the remainder at the beginning of the buffer header, thus implementing a cyclic buffer. Finally, move the write pointer back to Len Byte and return len.
As can be seen from the above, the value of Fifo->in can change from 0 to more than fifo->size values, fifo->out, but their difference will not exceed fifo->size.

Linux kernel structure--KFIFO ring buffer