Precautions for using lock-free queue (ring buffer), queue Loops

Precautions for using lock-free queue (ring buffer), queue Loops

The circular buffer is a common data structure in the producer and consumer models. The producer puts the data into the end of the array, and the consumer removes the data from the other end of the array. when the end of the array is reached, the producer returns to the header of the array. If there is only one producer and one consumer, the Ring Buffer can be accessed without a lock ). Writing an index only allows the producer to access and modify it. As long as the writer saves the new value to the buffer before updating the index, the reader will always see the consistent data structure. Similarly, read indexes can only be accessed and modified by consumers.

Implementation principle of ring Buffer

When the reader and the writer pointer are equal, it indicates that the buffer zone is empty. If the write pointer is behind the read pointer, it indicates that the buffer zone is full.

List9. 2.6.10Circular buffer implementation code

/*

* _ K1_o_put-puts some data into the FIFO, no locking version

* Note that with only one concurrent reader and one concurrent

* Writer, you don't need extra locking to use these functions.

*/

Unsigned int _ k1_o_put (struct kfifo * fifo,

Unsigned char * buffer, unsigned int len)

{

Unsigned int l;

Len = min (len, fifo-> size-fifo-> in + fifo-> out );

/* First put the data starting from fifo-> in to buffer end */

L = min (len, fifo-> size-(fifo-> in & (fifo-> size-1 )));

Memcpy (fifo-> buffer + (fifo-> in & (fifo-> size-1), buffer, l );

/* Then put the rest (if any) at the beginning of the buffer */

Memcpy (fifo-> buffer, buffer + l, len-l );

Fifo-> in + = len;

Return len;

}

 

/*

* _ K1_o_get-gets some data from the FIFO, no locking version

* Note that with only one concurrent reader and one concurrent

* Writer, you don't need extra locking to use these functions.

*/

Unsigned int _ k1_o_get (struct kfifo * fifo,

Unsigned char * buffer, unsigned int len)

{

Unsigned int l;

Len = min (len, fifo-> in-fifo-> out );

/* First get the data from fifo-> out until the end of the buffer */

L = min (len, fifo-> size-(fifo-> out & (fifo-> size-1 )));

Memcpy (buffer, fifo-> buffer + (fifo-> out & (fifo-> size-1), l );

/* Then get the rest (if any) from the beginning of the buffer */

Memcpy (buffer + l, fifo-> buffer, len-l );

Fifo-> out + = len;

Return len;

}

 

Note that

When using ring_buffer_get (kw.o_get) or ring_buffer_put (kw.o_put), if the returned parameter is not equal to the input parameter len, the operation fails.

 

We define

// Note that student_info occupies 24 bytes in memory in 17 bytes.

Typedef struct student_info

{

Uint64_t stu_id; // 8 bytes

Uint32_t age; // 4 bytes

Uint32_t score; // 4 bytes

Char sex; // 1 byte

} Student_info;

 

We createRing buffer, which only contains64Byte size(Although we actually use big, small, and far greater), it is saved24BytesStudent_info. Check the response.

 

// Print Student Information

Void print_student_info (const student_info * stu_info)

{

Assert (stu_info );

Printf ("id: % lu \ t", stu_info-> stu_id );

Printf ("age: % u \ t", stu_info-> age );

Printf ("sex: % d \ t", stu_info-> sex );

Printf ("score: % u \ n", stu_info-> score );

}

 

Student_info * get_student_info (time_t timer)

{

Student_info * stu_info = (student_info *) malloc (sizeof (student_info ));

Srand (timer );

Stu_info-> stu_id = 10000 + rand () % 9999;

Stu_info-> age = rand () % 30;

Stu_info-> score = rand () % 101;

Stu_info-> sex = rand () % 2;

Print_student_info (stu_info );

Return stu_info;

}

Void print_ring_buffer_len (struct ring_buffer * ring_buf)

{

// Print the buffer Length

Uint32_t ring_buf_len = 0;

// Gets the length of the buffer that has been used. The size-ring_buf_len is the length of the buffer that is not used.

Ring_buf_len = ring_buffer_len (ring_buf );

Printf ("no use ring_buf_len: % d \ n", (ring_buf-> size-ring_buf_len ));

}

Int main (int argc, char * argv [])

{

Uint32_t size = 0;

// Determine the number of bytes of stored or acquired data

Uint32_t oklen = 0;

Struct ring_buffer * ring_buf = NULL;

// 64 bytes

Size = BUFFER_SIZE;

Ring_buf = ring_buffer_alloc (size );

Printf ("input student \ n ");

{

Student_info * stu_info;

Student_info stu_temp;

Uint32_t student_len = sizeof (student_info );

Printf ("ring_buf_len: % d \ n", ring_buf-> size );

Printf ("student_len: % d \ n", student_len );

// At this time, there is no data in the ring buffer. Of course, the data to be retrieved is empty.

Memset (& stu_temp, 0, student_len );

Oklen = ring_buffer_get (ring_buf, (void *) (& stu_temp), student_len );

If (oklen = student_len)

{

Printf ("get student data \ n ");

}

Else

{

Printf ("no student data \ n ");

}

Printf ("\ n ");

// There are 64-24 = 40 bytes after the first call.

Stu_info = get_student_info (976686458 );

Oklen = ring_buffer_put (ring_buf, (void *) stu_info, student_len );

If (oklen = student_len)

{

Printf ("1 put student data success \ n ");

}

Else

{

Printf ("1 put student data failure \ n ");

}

Print_ring_buffer_len (ring_buf );

 

Printf ("\ n ");

// For the second call, there are still 64-48 = 16 bytes after the use of bytes

Stu_info = get_student_info (976686464 );

Oklen = ring_buffer_put (ring_buf, (void *) stu_info, student_len );

If (oklen = student_len)

{

Printf ("2 put student data success \ n ");

}

Else

{

Printf ("2 put student data failure \ n ");

}

Print_ring_buffer_len (ring_buf );

 

Printf ("\ n ");

// Bytes are required for the third call, but only bytes fail.

// Fill all the bytes

// Verify that when the _ k1_o_put function or _ k1_o_get function is called, the operation fails if the returned parameter is not the same as the input parameter len.

Stu_info = get_student_info (976686445 );

Oklen = ring_buffer_put (ring_buf, (void *) stu_info, student_len );

If (oklen = student_len)

{

Printf ("3 put student data success \ n ");

}

Else

{

Printf ("3 put student data failure \ n ");

}

Print_ring_buffer_len (ring_buf );

Printf ("\ n ");

// Bytes are required for the fourth call, but no Bytes are required.

//// Verify that when the _ k1_o_put function or _ k1_o_get function is called, the operation fails if the returned parameter is not the same as the input parameter len.

Stu_info = get_student_info (976686421 );

Oklen = ring_buffer_put (ring_buf, (void *) stu_info, student_len );

If (oklen = student_len)

{

Printf ("4 put student data success \ n ");

}

Else

{

Printf ("4 put student data failure \ n ");

}

Print_ring_buffer_len (ring_buf );

 

Printf ("\ n ");

// Now, the student data is saved in the student data. We can take the data three times to see the effect.

Printf ("output student \ n ");

 

Printf ("\ n ");

// Obtain and print data for the first time

Memset (stu_info, 0, student_len );

Oklen = ring_buffer_get (ring_buf, (void *) stu_info, student_len );

If (oklen = student_len)

{

Print_student_info (stu_info );

Printf ("1 get student data success \ n ");

}

Else

{

Printf ("1 get student data failure \ n ");

}

Print_ring_buffer_len (ring_buf );

 

Printf ("\ n ");

//// Obtain and print data for the second time

Memset (stu_info, 0, student_len );

Oklen = ring_buffer_get (ring_buf, (void *) stu_info, student_len );

If (oklen = student_len)

{

Print_student_info (stu_info );

Printf ("2 get student data success \ n ");

}

Else

{

Printf ("2 get student data failure \ n ");

}

Print_ring_buffer_len (ring_buf );

 

Printf ("\ n ");

// Data retrieval fails for the third time.

Memset (stu_info, 0, student_len );

Oklen = ring_buffer_get (ring_buf, (void *) stu_info, student_len );

If (oklen = student_len)

{

Print_student_info (stu_info );

Printf ("3 get student data success \ n ");

}

Else

{

Printf ("3 get student data failure \ n ");

}

Print_ring_buffer_len (ring_buf );

 

}

 

Return 1;

}

 

Conclusion: when using ring_buffer_get (kw.o_get) or ring_buffer_put (kmeano_put), if the returned parameter is not the same as the input parameter len, the operation fails. Code download: tessc.rar (http://files.cnblogs.com/dragonsuc/TDDOWNLOAD.zip)

Note:

1. Data is thread-safe when only one thread is responsible for reading and the other thread is responsible for writing. The above implementation is implemented based on this principle. When multiple threads read or write data, data correctness is not guaranteed.
So when used, one thread writes and one thread reads. Commonly used in network applications, data is written into a queue by opening a thread interface. Start a scheduling thread to read network data and distribute it to the processing thread.

2. The data length macro defines a length by default. When the length is exceeded, subsequent data writing will fail.

References:

Http://blog.csdn.net/mergerly/article/details/39009473

Http://www.cnblogs.com/Anker/p/3481373.html

 

 

What kind of buffer is the ring buffer?

Should belong to the queue, cyclic queue

Because the system buffer space is insufficient or the queue is full, you cannot perform operations ON the socket (1005), on api \ 'connenct

Set the maximum disk cache address.
Increase the memory size if enough