A lightweight and efficient multithreaded C++stream style asynchronous log (ii)

Directory

• A lightweight and efficient multithreaded C++stream style asynchronous log (ii)
  • Objective
  • LogFile Class
  • Asynclogging class
  • Asynclogging implementation
  • Add Alternate Cache
  • Conclusion

A lightweight and efficient multithreaded C++stream style asynchronous log (II.) preface

This article immediately follows the previous article: Describes how the previous sections of the log are imported asynchronously into a local file.
First, we will briefly introduce the next logfile class, and then we will explain the double buffering mechanism in asynclogging.
Structure diagram of the entire log module,

LogFile Class

The logfile log file class completes the management of the log file.
Rollfile (): Scrolls a new log file out when scrolling past the m_rollsize size of the file.
Getlogfilename (): When used with scrolling logs, name the log file with scrolling time as the suffix.
M_mutex: For append () data, lock the file.
Append (): Sticky log.
Flush (): Flushes the buffer.

LogFile has a AppendFile class, which is the class that is ultimately used to manipulate local files.
Append (): The system function fwrite () is called to write to the local file.
Flush (): Flushes the buffer.
Writtenbytes (): Gets the number of write bytes.

Asynclogging class

Asynclogging the asynchronous log class to complete the asynchronous write operation of the log.
Before introducing its interface, describe its working logic.

Asynclogging has the following types of caches.
M_currentbuffer: A cache that points to a log that is currently receiving append from other threads.
M_buffers: A pointer container used to hold the log cache that is currently full or past the flushing cycle.
M_nextbuffer: Points to the cache that is used to replace M_currentbuffer when M_currentbuffer is full.

BackupBuffer1: Alternate Cache.
BackupBuffer2: Alternate Cache.
Bufferstowrite: And m_buffers by swapping swap () after append () to LogFile's pointer container.

The double buffering mechanism used by asynclogging has two cache containers: M_buffers, Bufferstowrite alternately. We are referred to as a and B briefly.
A is used to receive logs that other threads append () come in.
B is used to write the currently accepted cache to the log file. When B is finished, clean () b, swap A, a, and so forth.

优点: The new log does not have to wait for disk operations, and each new log triggers a log thread, but instead sends a large buffer of multiple log threads to the log thread to write to the file. Equivalent to batch processing, reducing the frequency of thread wakes and reducing overhead.
In addition, in order to write log messages in a timely manner, that is, buffer A does not have a push in the log will be executed every three seconds above write operations.

Asynclogging uses a larger logbuffer to keep a log of logger sent over.
Mutex: Used to control the writing of multiple threads.
Condition: Used to wait for data in the buffer.
Thread: Uses a thread to handle cached interchanges, as well as log writes.

Asynclogging implementation

The

below gives a simple implementation of asynclogging.
There are actually several alternate caches, which are not added to facilitate understanding of the program; the fallback cache is primarily intended to reduce the overhead of repeated new operations,

#ifndef _async_logging_hh#define _async_logging_hh#include "mutexlock.hh" #include "thread.hh" #include "logstream.hh "#include" ptr_vector.hh "#include" condition.hh "#include <string>class asynclogging{public:asynclogging (    Const std::string FilePath, off_t rollsize, int flushinterval = 3);    ~asynclogging ();        void Start () {m_isrunning = true;    M_thread.start ();        } void Stop () {m_isrunning = false;    M_cond.notify ();    } void Append (const char *logline, int len);p rivate:asynclogging (const asynclogging&);    asynclogging& operator= (const asynclogging&);    void Threadroutine ();    typedef logbuffer<klargebuffer> Buffer;    typedef oneself::p tr_vector<buffer> buffervector;    typedef oneself::auto_ptr<buffer> BUFFERPTR;    const int M_flushinterval;    BOOL m_isrunning;    off_t m_rollsize;    Std::string M_filepath;    Thread M_thread;    Mutexlock M_mutex;    Condition M_cond;    Bufferptr M_currentbuffer; BUffervector m_buffers;}; #endif//asynclogging.cpp#include "asynclogging.hh" #include "logfile.hh" #include <assert.h> #include < stdio.h>asynclogging::asynclogging (const std::string FilePath, off_t rollsize, int flushinterval): M_filepath ( FilePath), M_rollsize (2048), M_flushinterval (Flushinterval), m_isrunning (false), M_thread (Std::bind (&a Synclogging::threadroutine, this)), M_mutex (), M_cond (M_mutex), M_currentbuffer (new Buffer), M_buffers () {} Asynclogging::~asynclogging () {if (m_isrunning) stop ();}    void Asynclogging::append (const char* logline, int len) {Mutexlockguard lock (M_mutex);    if (M_currentbuffer->avail () > Len) {m_currentbuffer->append (logline, Len);                } else{M_buffers.push_back (M_currentbuffer.release ());        M_currentbuffer.reset (new Buffer);        M_currentbuffer->append (Logline, Len);    M_cond.notify (); }}void Asynclogging::threadroutine () {assert (m_isrunning = = true);   LogFile output (M_filepath, m_rollsize, false);    Buffervector Bufferstowrite;    Bufferstowrite.reserve (8);        while (m_isrunning) {assert (Bufferstowrite.empty ());            {Mutexlockguard lock (M_mutex);            if (M_buffers.empty ()) {m_cond.waitforseconds (m_flushinterval);            } m_buffers.push_back (M_currentbuffer.release ());            M_currentbuffer.reset (new Buffer);        M_buffers.swap (Bufferstowrite);        } assert (!bufferstowrite.empty ()); for (size_t i = 0; i < bufferstowrite.size (); ++i) {Output.append (Bufferstowrite[i]->data (), Bufferstowrit        E[i]->length ());        } bufferstowrite.clear ();    Output.flush (); } output.flush ();}

Add Alternate Cache

To increase the backup cache optimization above program, the above program performed a total of new operations in two places.
When the 1.m_currentbuffer is filled, it needs to be filled into the container.
2. By the time you need to write the contents of the M_currentbuffer into a local file, it will move the current content out, it will need a new cache to M_currentbuffer.

So we're going to have a m_nextbuffer to do the M_currentbuffer cache. At the same time, add two backupbuffer in the thread to M_nextbuffer as the standby cache; When the log volume is not enough, consider using new Action to dynamically add the cache.

#ifndef _async_logging_hh#define _async_logging_hh#include "mutexlock.hh" #include "thread.hh" #include "logstream.hh "#include" ptr_vector.hh "#include" condition.hh "#include <memory> #include <string>class asynclogging{    public:asynclogging (const std::string FilePath, off_t rollsize, int flushinterval = 3);    ~asynclogging ();        void Start () {m_isrunning = true;    M_thread.start ();        } void Stop () {m_isrunning = false;    M_cond.notify ();    } void Append (const char *logline, int len);p rivate:asynclogging (const asynclogging&);    asynclogging& operator= (const asynclogging&);    void Threadroutine ();    typedef logbuffer<klargebuffer> Buffer;    typedef myself::p tr_vector<buffer> buffervector;    typedef std::unique_ptr<buffer> BUFFERPTR;    const int M_flushinterval;    BOOL m_isrunning;    off_t m_rollsize;    Std::string M_filepath;    Thread M_thread;    Mutexlock M_mutex;    Condition M_cond; Bufferptr m_currentbuffer;    Bufferptr M_nextbuffer; Buffervector m_buffers;}; #endif//asynvlogging.cpp#include "asynclogging.hh" #include "logfile.hh" #include <assert.h> #include < stdio.h>asynclogging::asynclogging (const std::string FilePath, off_t rollsize, int flushinterval): M_filepath ( FilePath), M_rollsize (rollsize), M_flushinterval (Flushinterval), m_isrunning (false), M_thread (Std::bind (&A mp Asynclogging::threadroutine, this)), M_mutex (), M_cond (M_mutex), M_currentbuffer (new Buffer), M_nextbuffer (new Buffer), M_buffers () {}asynclogging::~asynclogging () {if (m_isrunning) stop ();}    void Asynclogging::append (const char* logline, int len) {Mutexlockguard lock (M_mutex);    if (M_currentbuffer->avail () > Len) {m_currentbuffer->append (logline, Len);                } else{M_buffers.push_back (M_currentbuffer.release ());        if (m_nextbuffer) {m_currentbuffer = Std::move (M_nextbuffer);       } else{     M_currentbuffer.reset (new Buffer);        } m_currentbuffer->append (Logline, Len);    M_cond.notify ();    }}void Asynclogging::threadroutine () {assert (m_isrunning = = true);    LogFile output (M_filepath, m_rollsize, false);    Bufferptr BackupBuffer1 (new Buffer);    Bufferptr BackupBuffer2 (new Buffer);    Buffervector Bufferstowrite;    Bufferstowrite.reserve (8);        while (m_isrunning) {assert (Bufferstowrite.empty ());            {Mutexlockguard lock (M_mutex);            if (M_buffers.empty ()) {m_cond.waitforseconds (m_flushinterval);            } m_buffers.push_back (M_currentbuffer.release ());            M_currentbuffer = Std::move (backupBuffer1);            M_buffers.swap (Bufferstowrite);        if (!m_nextbuffer) M_nextbuffer = Std::move (BackupBuffer2);        } assert (!bufferstowrite.empty ()); for (size_t i = 0; i < bufferstowrite.size (); ++i) {Output.append (bufferstowrite[i]->Data (), bufferstowrite[i]->length ()); } if (Bufferstowrite.size () > 2) {//Drop non-bzero-ed buffers, avoid trashing buff        Erstowrite.resize (2);            } if (!backupbuffer1) {assert (!bufferstowrite.empty ());            BackupBuffer1 = Std::move (Bufferstowrite.pop_back ());        Backupbuffer1->reset ();            } if (!backupbuffer2) {assert (!bufferstowrite.empty ());            BackupBuffer2 = Std::move (Bufferstowrite.pop_back ());        Backupbuffer2->reset ();        } bufferstowrite.clear ();    Output.flush (); } output.flush ();}

Conclusion

This paper mainly introduces the implementation of Asynclogging class in Muduo, and the double caching mechanism.
The LogFile class and the AppendFile class are the basic operation classes of the log file management class and the local file respectively. Not difficult to understand, interested in the words can see Muduo source code, this article no longer write down, if you want all the source can leave a message.