Circular buffer (circular buffers) implementation

usage

A useful feature of a circular buffer is that when a data element is used, the remaining data elements do not need to move their storage location. Conversely, a non-circular buffer (for example, a normal queue) after a data element is used, the remaining data elements need to be moved forward. In other words, a round buffer is suitable for FIFO buffers, rather than round buffers for last-in-first-out buffers.
A round buffer is suitable for situations in which the maximum capacity of the buffer is defined beforehand. Expands the capacity of a circular buffer, which requires data to be moved. Therefore, a buffer is more appropriate if it needs to adjust its capacity frequently.

Write operations that overwrite data that is not processed in a circular buffer are allowed in some cases. Especially when it comes to multimedia processing. For example, producers of audio can overwrite audio data that has not yet been processed by the sound card.

Work Process

A circular buffer is initially empty and has a predetermined length. For example, this is a circular buffer with seven element spaces, where a single line and an arrow at the bottom indicate "end to end" forming a circular address space:

Assume that 1 is written to the middle of the buffer (where the initial write position is irrelevant for a circular buffer):

Write 2 more elements, 2 & 3, respectively-appended to the 1:

If two elements are processed, then the oldest two elements in the buffer are removed. In this example, 1 & 2 are removed and only 3 are left in the buffer:

If there are 7 elements in the buffer, it is full:

If the buffer is full and new data is written, one strategy is to overwrite the oldest data. In this example, 2 new data-A & B-write, covering 3 & 4:

You can also take other policies that prohibit overwriting buffer data, take an error code to return, or throw an exception.

Finally, if you remove 2 data from the buffer, not 3 & 4 but 5 & 6. Because A & B already covers 3 & 4:

Circular buffer working mechanismBecause computer memory is a linear address space, circular buffers require a special design to be logically implemented.

read pointers and write pointers

In general, a circular buffer requires 4 pointers:

• The actual starting position in memory;
• The actual end position in memory can also be replaced by the buffer length;
• The starting position of valid data stored in the buffer (read pointer);
• The end position of the valid data stored in the buffer (write pointer).

Read pointers and write pointers can be represented by integer values.

The following example reads and writes a read-write pointer to a buffer that is not full:

The following example reads and writes pointers to a full buffer:

distinguish between full or empty buffersThe buffer is full, or empty, and it is possible that the read pointer is pointing to the same position as the write pointer.

There are several strategies for detecting whether a buffer is full or empty:

always keep a storage unit empty

There is always a storage unit in the buffer that remains unused. Buffers are stored up to a maximum of data. If the read-write pointer points to the same location, the buffer is empty. If the write pointer is in the next position adjacent to the read pointer, the buffer is full. The advantages of this strategy are simple and robust, the disadvantage is that the actual amount of data stored in the semantics is inconsistent with the buffer capacity, and the remainder calculation is required to test whether the buffer is full.

Use data Count

Instead of using explicit write pointers, this strategy keeps a count of the data stored in the buffer. So testing the buffer is empty is very simple, and the performance impact can be ignored. The disadvantage is that both read and write operations need to modify this storage data count, and concurrency control is required for multithreaded access buffers.

Mirror Indication Bit

If the length of the buffer is n and the logical address space is 0 to n-1, then the specified n to 2n-1 is the mirrored logical address space. This policy specifies that the address space for read-write pointers is 0 to 2n-1, where the lower half corresponds to the regular logical address space, and the higher half corresponds to the mirrored logical address space. When the pointer value is greater than or equal to 2n, make it foldback (wrapped) to ptr-2n. Use one to indicate whether the write pointer or read pointer has entered the virtual Mirror store: The set represents the entry, and the pail bit indicates that no entry is still in the base store.

When the value of a read-write pointer is the same, the buffer is empty if the indicated bit is the same, or if the indicated bit is different, the buffer is full. The advantage of this method is that the test buffer full/empty is simple, there is no need to do the remainder operation, and the read-write thread can design a special algorithm strategy, which can achieve sophisticated concurrency control. The disadvantage is that the read-write pointers require an additional one as the indicator bit.

If the buffer length is a power of 2, this method can omit the mirror indication bit. If the value of the read-write pointer is equal, the buffer is empty, and if the read-write pointer is different than n, the buffer is full, which can be judged by the conditional expression (write pointer = = (read pointer xor or buffer length).

/* This approach adds one bit to end and start pointers */* Circular Buffer object */typedef struct {int siz   E  /* Maximum number of elements */int start;    /* Index of oldest element */int end;  /* index at which to write new element */Elemtype *elems; /* Vector of elements */} Circularbuffer;    void Cbinit (Circularbuffer *cb, int size) {cb->size = size;    Cb->start = 0;    cb->end = 0; Cb->elems = (Elemtype *) calloc (cb->size, sizeof (Elemtype));}  void Cbprint (Circularbuffer *cb) {printf ("size=0x%x, start=%d, end=%d\n", Cb->size, Cb->start, cb->end);} int  Cbisfull (Circularbuffer *cb) {return cb->end = = (cb->start ^ cb->size);/* This inverts the most significant Bit of start before comparison */} int cbisempty (Circularbuffer *cb) {return cb->end = cb->start;} int cbinc R (Circularbuffer *cb, int p) {return (P + 1) & (2*cb->size-1); /* Start and end pointers incrementation is do modulo 2*size */} void Cbwrite (Circularbuffer *cb, Elemtype *elem) {C    b->elems[cb->end& (cb->size-1)] = *elem;    if (Cbisfull (CB))/* Full, Overwrite moves start pointer */Cb->start = CBINCR (CB, Cb->start); Cb->end = CBINCR (CB, cb->end);}    void Cbread (Circularbuffer *cb, Elemtype *elem) {*elem = cb->elems[cb->start& (cb->size-1)]; Cb->start = CBINCR (CB, Cb->start);}

read/write Count

Saves the number of data written and read buffers, respectively, with two signed integer variables. The difference is the number of valid data that has not yet been processed in the buffer. The advantage of this approach is that the read thread and the write thread do not interfere; the disadvantage is that it requires an additional two variables.

record the last action

Use a single record to read or write the last operation. If the last operation is written, the buffer is full if the read-write pointer value is equal, or the buffer is empty if the last operation is a readout. The disadvantage of this strategy is that read and write operations share a flag bit and concurrency control is required for multiple threads.

posix optimization implementation

#include <sys/mman.h> #include <stdlib.h> #include <unistd.h> #define Report_exceptional_condition (   ) abort () struct ring_buffer{void *address;  unsigned long count_bytes;  unsigned long write_offset_bytes; unsigned long read_offset_bytes;};  Warning order should is at least-linuxvoidring_buffer_create (struct ring_buffer *buffer, unsigned long order) {  Char path[] = "/dev/shm/ring-buffer-xxxxxx";  int file_descriptor;  void *address;   int status;  File_descriptor = mkstemp (path);   if (File_descriptor < 0) report_exceptional_condition ();  Status = Unlink (path);   if (status) report_exceptional_condition ();  Buffer->count_bytes = 1UL << order;  buffer->write_offset_bytes = 0;   buffer->read_offset_bytes = 0;  Status = Ftruncate (File_descriptor, buffer->count_bytes);   if (status) report_exceptional_condition (); buffer->address = Mmap (NULL, buffer->count_bytes << 1, Prot_none, Map_anonymous |   Map_private,-1, 0);   if (buffer->address = = map_failed) report_exceptional_condition (); Address = Mmap (buffer->address, buffer->count_bytes, Prot_read | Prot_write, map_fixed |   map_shared, File_descriptor, 0);   if (address! = buffer->address) report_exceptional_condition (); Address = mmap (buffer->address + buffer->count_bytes, buffer->count_bytes, Prot_read | Prot_write, map_fixed |   map_shared, File_descriptor, 0);   if (address! = buffer->address + buffer->count_bytes) report_exceptional_condition ();  Status = Close (File_descriptor); if (status) Report_exceptional_condition ();}   Voidring_buffer_free (struct Ring_buffer *buffer) {int status;  Status = Munmap (buffer->address, buffer->count_bytes << 1); if (status) Report_exceptional_condition ();} void *ring_buffer_write_address (struct ring_buffer *buffer) {/*** void pointer arithmetic is a constraint ViolatiOn. /return buffer->address + buffer->write_offset_bytes;} Voidring_buffer_write_advance (struct Ring_buffer *buffer, unsigned long count_bytes) {buffer-& Gt;write_offset_bytes + = Count_bytes;} void *ring_buffer_read_address (struct ring_buffer *buffer) {return buffer->address + buffer->read_offset_bytes;} Voidring_buffer_read_advance (struct Ring_buffer *buffer, unsigned long count_bytes) {Buffer-&gt   ; Read_offset_bytes + = Count_bytes;      if (buffer->read_offset_bytes >= buffer->count_bytes) {/* If the read pointer is greater than or equal to the buffer length, those read-write pointers are simultaneously returned in the [0, Buffer_size] range */      Buffer->read_offset_bytes-= buffer->count_bytes;    Buffer->write_offset_bytes-= buffer->count_bytes; }} unsigned longring_buffer_count_bytes (struct Ring_buffer *buffer) {return buffer->write_offset_bytes-buffer- >read_offset_bytes;} unsigned longring_buffer_count_free_bytes (struct Ring_buffer *buffer) {return buffer->count_bytes -Ring_buffer_count_bytes (buffer);}  Voidring_buffer_clear (struct Ring_buffer *buffer) {buffer->write_offset_bytes = 0; buffer->read_offset_bytes = 0;} /*note, that initial anonymous mmap () can is Avoided-after initial mmap () for descriptor Fd,you can try Mmap () with hint Ed address as (Buffer->address + buffer->count_bytes) and if it fails-another one with hinted address as (BUFFER-&G T;address-buffer->count_bytes). Make sure map_fixed are not used in such case, as under certain situations it could end with Segfault.  The advantage of such approach is, then it avoids requirement to map twice the amount you need initially (especially useful e.g. if want to use HUGETLBFS and the allowed amount are limited) and in context of gcc/glibc-you can avoid certain f Eature macros (map_anonymous usually requires one of: _bsd_source, _svid_source or _gnu_source). */

Kfifo of the Linux kernel

In the Linux kernel files kfifo.h and kfifo.c, a FIFO round buffer implementation is defined. If there is only one read thread, one write thread, and there are no shared modified control variables, then you can prove that concurrency control is not required in this case. Kfifo meet the above conditions. Kfifo requires that the buffer length be a power of 2. Read and write pointers are unsigned integer variables, respectively. Converting a read-write pointer to an index value within a buffer requires only the bitwise-and action: (The pointer value is bitwise AND (buffer length-1)). This avoids the computationally expensive "redundancy" operation. And the following relationship is always established:

read pointer + buffer stored data length = = Write pointer

Even if the write pointer reaches the upper bound of the unsigned integer, the value of the write pointer after overflow is less than the value of the read pointer, the relationship remains the same (because of the nature of unsigned integer addition). Kfifo writes, the data length of the current writable storage space in the buffer is calculated first:

len = min{data length to write, buffer length-(write pointer-read pointer)}

The data is then written in two segments. The first paragraph starts at the end of the buffer from the write pointer, and the second paragraph writes the remaining writable data from the beginning of the buffer, which may have a data length of 0 and no actual data is written.

Circular buffer (circular buffers) implementation