"Go" buffer design--ring queue

Original link: http://blog.csdn.net/billow_zhang/article/details/4420789

When communicating between the two modules of a program, the buffer becomes a frequently used mechanism.

For example, the Write module writes information to the buffer, and the read-out module reads the information out of the buffer. This makes:

• The program is clearly divided into modules to establish a good modular architecture, so that the write and read out into a high-aggregation, low-coupling module.
• For the processing of the write and readout can be produced by the uneven speed, so that the entire processing velocity tends to smooth uniform state, to avoid the readout module processing slow speed so that the write module waiting to make the response slow down the situation, also avoid writing the module is not uniform, so that the readout module busy and idle situation.
• You can increase the concurrency of the processing. Due to the good design and partitioning of the write and read-out modules, they can be isolated and isolated from each other, thus using threads and processes to produce different concurrent processing, and through the buffer size adjustment, so that the processing to achieve a good match and running state. For example, a write module can have n threads or processes, a read-out module can have m threads and processes, and a cache flush can configure the size of L. N, M, L can be set by simulation test to adapt to the value of the specific application. It is also possible to set up a mechanism for automatic adjustment, which, of course, creates the complexity of the design.

The buffer is obviously not suitable for the following scenario:

• The reception and processing of data is closely related and difficult to partition modules.
• There is clearly no case of uneven processing between the modules in the processing, or not the main problem.
• A situation in which a synchronous response is required. Obviously, the write-side simply push the information into the queue, and cannot get the read-out processing response information, only suitable for asynchronous information transmission situation.

Design of the buffer:

• The buffer is a FIFO queue. The Write module inserts the information into the queue, and the readout module pops the message out of the queue.
• The writing module and the readout module need to coordinate and synchronize the information.
• For multi-threaded and multi-process write or readout modules, critical section processing is required between the writing module and the readout module.

Queues use ring queues, such as. The ring queue is characterized by the need for dynamic memory release and allocation, using a fixed-size memory space for reuse. In the actual queue insert and eject operation, the head will increase when the push operation is continuous, and the tail will increase when the pop operation. Push speed, it is possible to catch up with tail, this time that the queue is full, can no longer push operation, need to wait for the pop operation to free up the queue space. When the operation of the pop fast, so that tail chase head, this time that the queue is empty, can no longer do the pop operation, need to wait for push to come in the data.

The following is a list of the source programs for the data structure of a ring queue class.

[CPP]View Plaincopy

4. /* LoopQue.h
6. Author:zhangtao
8. Date:july 26, 2009
10. */
14. # ifndef Loopque_h
16. # define Loopque_h
20. # include <new>
24. namespace XTL {
28. Template<typename _tp>
30. Class Loopque_impl {
32. Public
34. static int addsize (int max_size) {
36. return max_size * sizeof (_TP);
38. }
42. Loopque_impl (int msize): max_size (Msize), _front (0), _rear (0), _size (0) {}
46. _tp& Front () {return data[_front];}
50. void push (const _tp& value) {
52. Data[_rear] = value;
54. _rear = (_rear + 1)% Max_size;
56. _size++;
58. }
62. void Pop () {
64. _front = (_front + 1)% Max_size;
66. _size--;
68. }
72. int Check_pop (_tp& TV) {
74. if (empty ())
76. return-1;
80. TV = Front ();
82. Pop ();
84. }
88. int Check_push (const _tp& value) {
90. if (full ())
92. return-1;
96. push (value);
98. }
102. BOOL Full () const {return _size = = max_size;}
104. BOOL Empty () const {return _size = = 0;}
106. int size () const {return _size;}
108. int capacity () const {return max_size;}
112. Private
114. int32_t _front; Front Index
116. int32_t _rear; Rear Index
118. int32_t _size; Queue data Record number
122. Const int32_t max_size; Queue capacity
126. _TP Data[0]; Data record occupy symbol
128. };
132. Template<typename _tp>
134. struct Loopque_allocate {
136. loopque_impl<_tp>& allocate (int msize) {
138. Char *p = new Char[sizeof (loopque_impl<_tp>) +
140. Loopque_impl<_tp>::addsize (msize)];
142. return * (New (P) loopque_impl<_tp> (msize));
144. }
148. void deallocate (void *p) {
150. Delete [] (char *) p;
152. }
154. };
158. template< TypeName _tp, typename Alloc = loopque_allocate<_tp> >
160. Class Loopque {
162. Public
164. typedef _TP VALUE_TYPE;
168. Loopque (int msize): Impl (Alloc.allocate (msize)) {}
170. ~loopque () {alloc.deallocate ((void *) &impl);}
174. value_type& Front () {return Impl.front ();}
176. Const value_type& Front () const {return impl.front;}
180. void push (const value_type& value) {Impl.push (value);}
182. void Pop () {Impl.pop ();}
186. int Check_pop (value_type& TV) {return Impl.check_pop (TV);}
188. int Check_push (const value_type& value) {return Impl.check_push (value);}
192. BOOL Full () const {return impl.full ();}
194. BOOL Empty () const {return impl.empty ();}
196. int size () const {return impl.size ();}
200. Private
202. Alloc Alloc;
204. loopque_impl<_tp>& Impl;
206. };
210. }//End of < namespace STL >
214. # endif//end of <ifndef loopque_h>

The program defines two classes of Loopque_impl and Loopqueue. The former defines the basic data structure and implementation of the ring queue, and the latter carries out a memory allocation wrapper.
The constructor for the Loopque_impl of 21 rows is created with the space size of the queue as a parameter for this class. That is, when the class is created, it determines the size of the queue.

The array space for the queue defined in 63 rows is _TP data[0]. It seems to be a strange thing. In fact, this space size should be the max_size array space. But since Max_size is determined at the time of class creation, here, data[0] only acts as a placeholder. Therefore, loopque_impl this class is not directly used, need to properly allocate memory size, in order to use, which is also required to design a class loopqueue one of the important reasons. Perhaps you wonder, why use this? Wouldn't it be easy to define a pointer, for example: _tp *data, and then use data = new _tp[max_size in the constructor? But, don't forget, our ring queue class is likely to be an inter-process shared class. For example, one process push operation, another process pop operation. In this way, the class needs to be built into shared memory. A member of a class in a shared memory, if it contains pointers or references to such types, will cause a lot of trouble in memory allocation. And the way we design this class with this placeholder will reduce this hassle and complexity.

The Addsize class function of line 17 determines the size of additional memory space required by the Loopque_impl.

Loopqueue shows how to use Loopque_impl to solve memory allocation problems. From the 79-row template parameter, we see that there is a alloc type in addition to the parameters of the buffer data storage type _TP. This is the template class used to allocate loopque_impl memory space.

In the member of Loopqueue, a reference impl (102 rows) of LOOPQUE_IMPL is defined. This reference is the space that points to the Loopque_impl used to allocate space using Alloc.

The alloc template parameter has a DEFAULT definition value of Loopque_allocate. From this default allocated memory class, we can see a sample of the implementation of the allocation Loopque_impl, see 69 lines:

Char *p = new Char[sizeof (loopque_impl<_tp>) + loopque_impl<_tp>::addsize (msize)];
return * (New (P) loopque_impl<_tp> (msize));

Here, we first allocate enough memory based on Msize to the data[msize], and then use the location new operation to create the Loopque_impl in this memory area. In this way, the Loopque_impl class can use the members of its _TP data[0]. In fact, this member already has _TP Data[msize] such a space. Here, if we design another class that allocates memory, for example, Loopque_shmalloc. This class uses shared memory and establishes the Loopque_impl class in it. So we can use:
LOOPQUE<_TP, loopque_shmalloc>

To create a ring queue class that can be used to communicate between processes.

At this point, we can summarize:

• The data structure and basic operations of the ring queue are fairly simple. The main operation is push and get.
• The ring queue may require a thread or inter-process sharing operation. In particular, the sharing between processes makes memory allocation a relatively complex issue. For this problem, we define the basic data structure as no pointers and reference members, suitable for memory management classes, and then design a proxy for the two-time wrapper class, the memory allocation function as a template can be customized parameters.

Here, we design and implement the circular queue data structure, and also for this structure can be customized storage to shared memory design policy strategy. But the following work will be more challenging:

• For push operations, wait for the queue to have space before the operation, which means that the queue is not full state. The push operation is not resumed until this state has occurred, otherwise the push action hangs.
• For get operations, you need to wait for the queue to have data before the operation, which means that the queue is not empty. The get operation is not resumed until this state has occurred, otherwise the get action hangs.
• The above push operation/get operations are generally different processes and threads. There are also processes and threads that may have multiple push operations and get operations.

These work will be carried out in the subsequent design. and listen to tell.

Attachment: Loopque Test procedure:

[CPP]View Plaincopy

4. /* Tst-loopque.cpp
6. Test program for <LoopQue> class
8. Author:zhangtao
10. Date:july 27, 2009
12. */
16. # include <iostream>
18. # include <cstdlib>//For <atol> function
20. # include "Xtl/loopque.h"
24. Int
26. Main (int argc, char **argv)
28. {
30. int qsize = 0;
34. if (argc > 1)
36. Qsize = Atol (argv[1]);
40. if (Qsize < 1)
42. Qsize = 5;
46. Xtl::loopque<int> Queue (qsize);
48. for (int i = 0; i < (qsize-1); i++) {
50. Queue.push (i);
52. Std::cout << "Loop push:" << i << "/n";
54. }
58. Queue.check_push (1000);
60. Std::cout << "full:" << queue.full () << "Size:"
62. << queue.size () << "/n/n";
66. for (int i = 0; i < qsize; i++) {
68. int val = Queue.front ();
70. Std::cout << "Loop pop:" << val << "/n";
72. Queue.pop ();
74. }
78. Std::cout << "/nempty:" << queue.empty () << "Size:"
80. << queue.size () << "/n";
82. return 0;
86. }

"Go" buffer design--ring queue