Fqueue Message Queue (1)

Fqueue is an open source Message Queue System developed by Chinese developers using Java. Message queues can be used to handle high-concurrency database read/write operations, reducing database load. The fqueue communication layer uses the netty framework, while the data storage adopts the file queue mode.

The main classes at the storage layer are as follows:

Fqueue: The main queue implements the main class. All messages are stored and read through this class.

Fsqueue: underlying implementation of fqueue, mainly for read/write and management control at the file Queue System Level

Logentity: operation class for a single data storage file

Logindex: index file control class

Key to the entire storage queueCodeAnd functions are as follows:

1. There are dedicated read and write operation handles responsible for the operations on the current stored files.

// The currently written position pointer <br/> private logentity writerhandle = NULL; <br/> // The currently read position pointer <br/> private logentity readerhandle = NULL;

Writerhandle = createlogentity (path + fileseparator + fileprefix + "Data _" + writerindex + ". IDB ", DB, <br/> writerindex); <br/> If (readerindex = writerindex) {<br/> readerhandle = writerhandle; <br/>} else {<br/> readerhandle = createlogentity (path + fileseparator + fileprefix + "Data _" + readerindex + ". IDB ", DB, <br/> readerindex); </P> <p>}

2. Dedicated index management

Public class logindex {</P> <p>/** <br/> * record the write position <br/> * @ Param POS <br/> * /<br/> Public void putwriterposition (int pos ); </P> <p>/** <br/> * record location <br/> * @ Param POS <br/> */<br /> Public void putreaderposition (int pos ); </P> <p>/** <br/> * record Write File index <br/> * @ Param index <br/> */<br /> Public void putwriterindex (INT index ); </P> <p>/** <br/> * record file index reading <br/> * @ Param index <br/> */<br /> Public void putreaderindex (INT index ); </P> <p>3. Use memory ing files to process stored files. The default value is 150 MB,

<textarea readonly name="code" class="java">Private file; <br/> private randomaccessfile rafile; <br/> private filechannel FC; <br/> Public mappedbytebuffer; </P> <p> rafile = new randomaccessfile (file, "RWD"); <br/> fc = rafile. getchannel (); <br/> mappedbytebuffer = FC. map (mapmode. read_write, 0, this. filelimitlength); </P> <p></textarea>
<textarea readonly name="code" class="java">Public fsqueue (string path) throws exception {<br/> This (path, 1024*1024*150); <br/>}</textarea> 4. the actual data storage start from 20 bytes. the first 8 bytes are marked as "fqueuefs", then 4 bytes is the version, 4 bytes is the next file number, and 4 bytes is the current storage location.
<textarea readonly name="code" class="java">Public class logentity {<br/> ..... <br/> Public static final string magic = "fqueuefs"; <br/> Public static int messagestartposition = 20; </P> <p> mappedbytebuffer. put (magic. getbytes (); <br/> mappedbytebuffer. putint (version); // 8 version <br/> mappedbytebuffer. putint (nextfile); // 12 next fileindex <br/> mappedbytebuffer. putint (endposition); // 16 <br/></textarea>

The basic process for adding, deleting, and deleting data is as follows:

Added fqueue. offer () ---> fsqueue. add () -----> logentity. write () ---- [writefull] ---> Generate the next file ID, create a new storage file and index rotatenextlogwriter () ----> logentity. write ()

Read fqueue. poll () ----> fsqueue. readnextandremove () -----> logentity. readnextandremove () ------ [the current file has been read to the end] ---> get the next file and add the current file to the delete queue filerunner. adddeletefile () ----> Read logentity again. readnextandremove () and update the index location.