Buffer Technology II: The simple realization of buffer pool Bufferpool

In the article buffering technology mentioned whether it is a single buffer, double buffering or circular buffering is only a process-specific buffer, and once the time-sharing parallelism of the operating system is taken into account, there is only one process buffer system at any given moment, while the buffer system for other processes is not working (or migrating to swap Cache peripheral disks either occupy physical memory and significantly reduce the amount of available physical memory, so using a common buffer pool to provide buffering services for multiple concurrent processes is the current mainstream means (operating system, database).

Due to the complete provision of operating system-level buffer pool involving more kernel knowledge, high complexity, for demonstration purposes, so this article through a single process of multithreading to provide a buffer service program demo analysis.

0. Extensible array implementation of the queue template
For the buffer pool, it consists of 3 different types of buffers: idle buffers, buffers filled with input data, and buffers filled with output data, managed through 3 queues: Empty buffer queue emq, buffer queue inq filled with input data, buffer queue OUTQ with output data filled. Therefore, the reasonable queue implementation is the basis of the buffer pool management, where the queue template is implemented through an array of scalable capacity.

queue.h

#pragma once #include <cassert> #include <cstring> namespace _tool {Template <typename t> S queue {Protected:int mfront,mend,mcapacity;//define the header of the queue (the position of the first valid data buffer), the tail (the next idle position to be inserted), mcapacity (current Maximum capacity of the queue) T *mpdata;  The array pointer to a contiguous array of buffer blocks/* Expands the current buffer block array capacity twice times * * virtual void double_resize (void) {T *tmp
            = new t[this->mcapacity*2];
            int size = This->size (); for (int i=0; i<size; i++) {Tmp[i] = this->mpdata[(this->mfront+i)%this->mcapacit
            Y];
            } delete[] Mpdata;
            This->mpdata = tmp;
            this->mcapacity*=2;
            This->mend = size;
        This->mfront = 0;
            Public:/* Default initialization function/queue (void) {const int init_size = 10;
            this->mcapacity = init_size;
            This->mfront = This->mend = 0; This->mpdata = new t[this->mcapacity];
            }/* "Reference delivery" initialization function/queue (const queue &from) {this->mcapacity = from.mcapacity;
            This->mfront = From.mfront;
            This->mend = From.mend;
            This->mpdata = new t[this->mcapacity]; for (int i=0; i<this->mcapacity; i++) {This->mpdata[i] = from.mpdata[i];//Whether a deep copy is still taken
            Specific implementation of the T data type} virtual ~queue (void) {if (this->mpdata)
            {delete[] this->mpdata;
        } this->mpdata = 0; /* * Press the new data into the queue/virtual void push_back (T value) {if (This->size () >= this->m
            Capacity-1) {this->double_resize ();
            } this->mpdata[this->mend++] = value;
        This->mend%= this->mcapacity;
/* * Remove the header element from the queue        Virtual const T Pop_front (void) {assert (!this->empty ());//output assertion, if the queue is empty, eject the assertion and, if not null, eject the queue header
            T tmp = this->mpdata[this->mfront++];
            This->mfront%= this->mcapacity;
        return TMP; /* Get the header element of the queue without ejecting the/virtual const T &get_front (void) const {ASSERT (!this->emp
            Ty ());
        Return this->mpdata[this->mfront]; /* Get queue when valid data size size*/virtual int size (void) const {return (this->mend+this->m
        Capacity-this->mfront)% this->mcapacity;
        virtual bool Empty (void) const {return this->size () ==0;
            Const queue &operator = (Const queue &from)//reference Pass {if (this->mpdata)
            {delete[] this->mpdata;
            } this->mpdata = 0;
        this->mcapacity = from.mcapacity;    This->mfront = From.mfront;
            This->mend = From.mend;
            This->mpdata = new t[this->mcapacity]; for (int i=0; i<this->mcapacity; i++) {this->mpdata[i] = from.mpdata[i];//Here you can see the queue's
        A deep copy is also a shallow copy of the operator function implemented by the child element itself, the implementation decision of the return *this;

}
    }; /*fixed_queue is a subclass inherited from the queue, and the Fixed_queue subclass does not support the dynamic extensible/template <typename T> class Fixed_queue:publ
            IC queue<t> {public:fixed_queue (int size = ten) {this->mcapacity = size+1;
            This->mfront = This->mend = 0;
        This->mpdata = new t[this->mcapacity]; }/* Reference delivery initialization function/* Fixed_queue (const fixed_queue &from) {this->mcapacity = from.
            mcapacity;
            This->mfront = From.mfront;
            This->mend = From.mend;
            This->mpdata = new t[this->mcapacity]; for (iNT I=0; i<this->mcapacity;
            i++) {This->mpdata[i] = From.mpdata[i]; Virtual ~fixed_queue (void) {if (this->mpdata) {D
            Elete[] this->mpdata;
        } this->mpdata = 0;
            virtual void push_back (T value) {assert (!this->full ());
            this->mpdata[this->mend++] = value;
        This->mend%= this->mcapacity;
        virtual bool Full (void) {return (This->size () >= this->mcapacity-1);
            } const Fixed_queue &operator = (const fixed_queue &right) {if (this->mpdata)
            {delete[] this->mpdata;
            } this->mpdata = 0;
            this->mcapacity = right.mcapacity;
            This->mfront = Right.mfront;
       This->mend = Right.mend;     This->mpdata = new t[this->mcapacity];
            for (int i=0; i<this->mcapacity; i++) {this->mpdata[i] = Right.mpdata[i];
        return *this; }/* If the fixed_queue does not have sufficient capacity, you must explicitly indicate the new size parameter/void resize (int size) {if This->mpdat
            A) {delete[] this->mpdata;
            } this->mpdata = 0;
            this->mcapacity = size+1;
            This->mfront = This->mend = 0;

        This->mpdata = new t[this->mcapacity]; /* Modify the Double_resize () dynamic extension function that inherits from the parent class \ */Protected:void double_resize (void) {assert (!" Don ' t call me!! ");}}

1. bufferpool External API and concrete implementation
The main components of the buffer pool are three types of queues: Free buffer block queue em, buffer block queues filled with input data in, buffer block queues filled with output data to be processed
The working mode of buffer pool is mainly 4 kinds: receiving input, extracting input, receiving output and extracting output;
BufferPool.h

#pragma once #include "queue.h" #include <windows.h> class Iobuffer {public:/* To declare three different queues in advance by enumerating the data structures, namely, the free buffer block queues, the number of inputs

    According to buffer block queue, output data buffer block Queue */enum Ebufftype {bt_em = 0, bt_in = 1, bt_out = 2,};
    Iobuffer (int buffer_pool_capacity, int each_buffer_size);

    ~iobuffer (void); 4 external functions int accept_in (void); For ease of implementation, only a buffer is returned, and a fill_in load input buffer queue is required, which together with fill_in constitutes the hosting input int accept_out (void);//For ease of implementation, only Returns a buffer that requires the fill_out to mount the output buffer queue, together with Fill_out to form the hosting output void fill_in (int number);//load buffer block buffer into input buffer queue void fill_out (i NT number);//buffer buffers load the data to be output and then put into the output buffer queue//2 the function void distill_in (void) of the buffer block that fills the data, and/or the corresponding fetch input void distill_out ( void);//corresponds to fetch output _tool::fixed_queue<char> &thebuffer (int number);

    Returns the reference Private:iobuffer (const Iobuffer &from) of the buffer for the specified index; int get_buff (int type);/has handled mutex problem void put_buf (int type, int work_buf),//Mutex issues handled/* To simplify operations, declare a piece directly in the heap of the current process space buffer_p
Ool_capacity * each_buffer_size size of space    Use as the physical space of the buffer pool and point the mpbuffers pointer to the first address of the space * * _tool::fixed_queue<char> *mpbuffers; 3 queues, the reason that the template class uses int is image, because and the subsequent design together, we are through the array of buffer space,//so each buffer block corresponding buffer Space array index index, for simplified management, the reason for the management of each buffer block index can _ 
    Tool::queue<int> Mqem;
    _tool::queue<int> Mqin;

    _tool::queue<int> mqout; HANDLE mhem;//Queue capacity semaphore HANDLE mhin;//queue capacity semaphore HANDLE mhout;//queue capacity semaphore HANDLE mhoem;//queue access semaphore HANDLE Mhoi
    n;//queue access to the semaphore HANDLE mhoout;//queue access to the semaphore int mbufferid;
static int mstatotalbuffers;
 };

BufferPool.cpp

#include "BufferPool.h" #include <sstream> #include <iostream> using namespace _tool;
extern HANDLE Output_mutex;

int iobuffer::mstatotalbuffers = 0; Iobuffer::iobuffer (int buffer_pool_capacity, int each_buffer_size) {/* Initializes the buffer pool, the buffer buffer block number in total has Buffer_pool_ Capacity parameter specifies that each buffer block buffer contains each_buffer_size bytes */this->mpbuffers = new Fixed_queue<char>[buffer_pool_capa
    City];
    for (int i=0; i<buffer_pool_capacity; i++) {this->mpbuffers[i].resize (each_buffer_size);
    //Record the buffer pool number This->mbufferid = ++iobuffer::mstatotalbuffers;

    Std::stringstream ss;//sets a String object that is used to provide the kernel's unique naming ss << "Bufferpool NO." << iobuffer::mstatotalbuffers;
    /* Set the following 3 mutexes, 3 of each semaphore, each queue corresponds to a mutex and semaphore, where the mutex corresponding to the queue's specific operational permissions, such as the pop-up queue head, or press into the new element, it is clear that each time only one thread to obtain specific operational permissions. and semaphore corresponding to the specific queue of the current available buffer block number, it should be first to obtain semaphore, and then or access to operational rights, dual-layer kernel lock to ensure the final thread mutex/This->mhem = CreateSemaphore (NULL, Buffer_capacity, Buffer_capacity, (SS.STR () + "EM"). c_sTR ());
    This->mhin = CreateSemaphore (NULL, 0, Buffer_capacity, (SS.STR () + "in"). C_STR ());
    This->mhout = CreateSemaphore (NULL, 0, Buffer_capacity, (SS.STR () + "Out"). C_str ());
    This->mhoem = CreateMutex (NULL, False, (SS.STR () + "OEM"). C_STR ());
    This->mhoin = CreateMutex (NULL, False, (SS.STR () + "OIN"). C_STR ());

    This->mhoout = CreateMutex (NULL, False, (SS.STR () + "Oout"). C_STR ()); Initializes the idle buffer queue, adding the index of all current free buffer blocks to the idle queue sequentially, which is also for (int i=0; i<buffer_capacity; i++) {This->mqem.pus
    H_back (i); }//* Reference-passing initialization function is designed to be iobuffer::iobuffer, and you can do it yourself, but it's not very meaningful here. * * (const iobuffer &from) {} iobuffer::~iobuffer (
    void) {//delete buffer delete[] this->mpbuffers;
    Clear the Semaphore CloseHandle (This->mhem);
    CloseHandle (This->mhin);
    CloseHandle (this->mhout);
    CloseHandle (THIS-&GT;MHOEM);
    CloseHandle (This->mhoin);
CloseHandle (this->mhoout); /* Returns a reference to the buffer block buffer for the specified index/* fixed_queue<char> &iobuFfer::thebuffer (int number) {return this->mpbuffers[number];} int iobuffer::get_buff (int type) {queue<int> *pq;//temporary queue pointer pointing to the first address of the buffer block to be manipulated HANDLE handle=null,ohandle=null;/ /temporary kernel object, which is used to obtain the mutex and semaphore of the corresponding queue///////Temp-Select different queue and control handle switch (type) {Case BT_EM:PQ = &this-&gt
        ; mqem; Handle = this->mhem;

    Gets the access permission for the idle queue Ohandle = this->mhoem;//Gets the operation permission of the idle queue, break;
        Case BT_IN:PQ = &this->mqIN;
        Handle = this->mhin;
        Ohandle = this->mhoin;

    Break
        Case BT_OUT:PQ = &this->mqOUT;
        Handle = this->mhout;
        Ohandle = this->mhoout;

    Break Default:assert (!)
        Unknown type ");
    Break //apply for a space, wait for the signal volume WaitForSingleObject (handle, INFINITE);
    Get access to the specific queue, which will reduce the number of available resources for that queue by 1 int tmp; Queue operations are mutually exclusive WaitForSingleObject (Ohandle, INFINITE); Get action permissions for a specific queue tmp = Pq->pop_front ()//To eject the index of the available buffer block for the queueEleasemutex (Ohandle)//free operation permission for specific queues return TMP;
    } void Iobuffer::p ut_buf (int type, int work_buf) {queue<int> *pq;
    HANDLE Handle=null,ohandle=null;
        Select different queues and control handles by type {case BT_EM:PQ = &this->mqEM;
        Handle = this->mhem;
        Ohandle = this->mhoem;

    Break
        Case BT_IN:PQ = &this->mqIN;
        Handle = this->mhin;
        Ohandle = this->mhoin;

    Break
        Case BT_OUT:PQ = &this->mqOUT;
        Handle = this->mhout;
        Ohandle = this->mhoout;

    Break Default:assert (!)
        Unknown type ");
    Break //Free up a space, release semaphore WaitForSingleObject (Ohandle, INFINITE);//Get the operation permissions for the queue Pq->push_back (number), or add the buffer block
    into the tail of the queue ReleaseMutex (Ohandle); ReleaseSemaphore (handle, 1, NULL);//Add the available resource count for the queue plus 1}/* To load the new data for the INQ queue, first extract an available free buffer block from the free buffer block queue/int iobuffer::accept_in (
void) {return this->get_buff (BT_EM);the int iobuffer::accept_out (void) {return This->get_buff (BT_EM),}/* joins the input buffer block with valid data loaded with number number into the INQ input buffer block waiting queue, waiting for Dist

Ill_in () function extraction processing */void iobuffer::fill_in (int number) {This->put_buf (bt_in, number);}

void Iobuffer::fill_out (int number) {This->put_buf (bt_out, number);} * * Extract input data buffer queue function/void Iobuffer::d istill_in (void) {//Get a full input buffer int number = This->get_buff (bt_in);//From input The first buffer block is extracted in the buffer block queue, and the Get_buff () function has designed the mutual exclusion mechanism WaitForSingleObject (Output_mutex, INFINITE); Get output permissions for stdout standard output std::cout << "*****flushing buffer" << number << "to input*****" << std:
    : Endl;
    Std::cout << "# #INPUT >"; Sequentially outputs the byte sequence data stored in the number buffer block sequentially to stdout while (!this->mpbuffers[number].empty ()) {Std::cout << this
    ->mpbuffers[number].pop_front ();
    } std::cout << Std::endl;
    ReleaseMutex (Output_mutex); Release the buffer this->put_buf (bt_em, number); Adding the number buffer block to the idle queue, the Put_buff () function has been designed with a mutexMechanism}//* Extract output Data buffer queue function/void Iobuffer::d istill_out (void) {//Get a full output buffer int number = This->get_buff (bt_out
    );
    WaitForSingleObject (Output_mutex, INFINITE);
    Std::cout << "Flushing buffer" << number << "to output" << Std::endl;
    Std::cout << "# #OUTPUT";
    while (!this->mpbuffers[number].empty ()) {std::cout << this->mpbuffers[number].pop_front ();
    } std::cout << Std::endl;
    ReleaseMutex (Output_mutex);
Release the buffer this->put_buf (bt_em, number); }

2. Bufferpool test program
test_main.cpp