C # Ultra High performance write log code open source

1. Demand

The requirement is simple, that is, high-level sketches of the logs in C # development. For example, in high-concurrency, high-traffic places need to write logs. We know that the program is time-consuming to operate the disk, so we write the log to the disk for a while, which is not something we want to see.

2. Solution 2.1, Simple principle explanation

The queue is cached to memory, and then we have a thread that writes to the disk from the queue, so that we can write the log at high speed and high performance. Because the speed of the place we separated out, that is, the program after the log thrown into the queue, the program of the log part of the completion, the later operation of the disk time-consuming part of the program is not required to care, by another thread operation.

As the saying goes, the fish and bear can not have both, so there will be a problem, that is, if the log has been queued up this time the program crashes or the computer power loss will cause the log part is lost, but some places for high-performance log, whether you can ignore some circumstances, please according to the situation.

2.2. Example diagram

3. Key Code section

Here the log part of the LZ chose the more common log4net, of course, you can choose other log components, such as Nlog and so on.

3.1. Log to queue section

The first step is to put the logs in a queue before we can write them to disk from the queue.

        public void Enqueuemessage (String message, flashloglevel level, Exception ex = null)        {            if (level = = Flashloglevel . Debug && _log. isdebugenabled)             | | (level = = Flashloglevel.error && _log. iserrorenabled)             | | (level = = Flashloglevel.fatal && _log. isfatalenabled)             | | (level = = Flashloglevel.info && _log. isinfoenabled)             | | (level = = Flashloglevel.warn && _log. iswarnenabled))            {                _que. Enqueue (New flashlogmessage                {                    Message = "[" + DateTime.Now.ToString ("Yyyy-mm-dd hh:mm:ss,fff") + "]\r\n" + message, Level                    = level,                    Exception = ex                });                Notifies the thread to write log _mre to disk                . Set ();            }        }

_log is the ilog of the Log4net log component, which contains the write log, judge the log level and other functions, the first part of the code if the judge is the level of judgment and the current level of the log to do a comparison, to see if it is necessary to write a queue, this can effectively improve the performance of the log.

One of the _que is the Concurrentqueue queue. _mre is the ManualResetEvent signal, ManualResetEvent is used to notify the thread queue that there are new logs that can be written to disk from the queue. After the log is written from the queue, reset the signal and wait for the next new log to arrive.

3.2. Queue to disk

From queue to disk we need a thread to write to the disk from the queue, that is, we will load the thread when the program starts, such as ASP. Application_Start in global .

       <summary>/////Another thread logs the log only once when the program is initialized.///</summary> public void Register ()            {Thread T = new Thread (new ThreadStart (Writelog));            T.isbackground = false;        T.start ();            }///<summary>//write logs from the queue to disk///</summary> private void Writelog () { while (true) {//waits for signal notification _mre.                WaitOne (); Determine if there is a need for content such as disk while (_que.                    Count > 0) {flashlogmessage msg; if (_que.                        Trydequeue (out msg))//Get the content from the queue and delete the contents of the queue {//judge the log level, then write the log Switch (MSG. Level) {case FLASHLOGLEVEL.DEBUG: _LOG.D Ebug (Msg. Message, Msg.                                Exception);                            Break Case Flashloglevel.Info: _log. Info (Msg. Message, Msg.                                Exception);                            Break Case Flashloglevel.error: _log. Error (Msg. Message, Msg.                                Exception);                            Break Case Flashloglevel.warn: _log. Warn (Msg. Message, Msg.                                Exception);                            Break Case Flashloglevel.fatal: _log. Fatal (Msg. Message, Msg.                                Exception);                        Break }}}//Reset signal _mre.            Reset (); }        }

3.3. Complete code

Using Log4net;using log4net. Config;using system;using system.collections.concurrent;using system.io;using system.threading;namespace emrys.flashlog{public sealed class Flashlogger {//<summary>///Record Message queue///&LT;/SU        mmary> private readonly concurrentqueue<flashlogmessage> _que;        <summary>///signal///</summary> private readonly ManualResetEvent _mre;        <summary>///journal///</summary> private readonly ILog _log; <summary>///Log///</summary> private static Flashlogger _flashlog = new Flashlogge        R (); Private Flashlogger () {//Set Log profile path Xmlconfigurator.configure (New FileInfo (Path.Combine (App            Domain.CurrentDomain.BaseDirectory, "Log4net.config")));            _que = new concurrentqueue<flashlogmessage> ();            _mre = new ManualResetEvent (false); _log = Logmanager.getlogger (System.Reflection.MethodBase.GetCurrentMethod ().        DeclaringType);        }//<summary>//To implement a single case///</summary>//<returns></returns>        public static Flashlogger Instance () {return _flashlog;        }///<summary>/////Another thread logging, only one call at the time of program initialization///</summary> public void Register ()            {Thread T = new Thread (new ThreadStart (Writelog));            T.isbackground = false;        T.start ();            }///<summary>//write logs from the queue to disk///</summary> private void Writelog () { while (true) {//waits for signal notification _mre.                WaitOne (); Determine if there is a need for content such as disk while (_que.                    Count > 0) {flashlogmessage msg; if (_que.             Trydequeue (out msg))//Get the content from the queue and delete the contents of the queue {           Determine the log level, and then write the log switch (MSG. Level) {case FLASHLOGLEVEL.DEBUG: _LOG.D Ebug (Msg. Message, Msg.                                Exception);                            Break Case Flashloglevel.info: _log. Info (Msg. Message, Msg.                                Exception);                            Break Case Flashloglevel.error: _log. Error (Msg. Message, Msg.                                Exception);                            Break Case Flashloglevel.warn: _log. Warn (Msg. Message, Msg.                                Exception);                            Break Case Flashloglevel.fatal: _log. Fatal (Msg. Message, Msg.                                Exception);                        Break             }}}//Reset signal   _mre.            Reset (); }}///<summary>//write Log///</summary>//<param name= "message" > Log Text </param>//<param Name= "level" > Grade </param>//<param name= "ex" >exception</par am> public void Enqueuemessage (String message, flashloglevel level, Exception ex = null) {if (level = = Flashloglevel.debug && _log. isdebugenabled) | | (level = = Flashloglevel.error && _log. iserrorenabled) | | (level = = Flashloglevel.fatal && _log. isfatalenabled) | | (level = = Flashloglevel.info && _log. isinfoenabled) | | (level = = Flashloglevel.warn && _log. iswarnenabled)) {_que. Enqueue (new Flashlogmessage {Message = "[" + DateTime.Now.ToString ("Yyyy-mm-dd HH:mm:ss , FFF ") +"]\r\n "+ message, level = level, ExCeption = ex}); Notifies the thread to write log _mre to disk.            Set (); }} public static void Debug (String msg, Exception ex = null) {Instance ().        Enqueuemessage (MSG, Flashloglevel.debug, ex); public static void Error (String msg, Exception ex = null) {Instance ().        Enqueuemessage (MSG, Flashloglevel.error, ex); public static void Fatal (String msg, Exception ex = null) {Instance ().        Enqueuemessage (MSG, Flashloglevel.fatal, ex); public static void Info (String msg, Exception ex = null) {Instance ().        Enqueuemessage (MSG, Flashloglevel.info, ex); public static void Warn (String msg, Exception ex = null) {Instance ().        Enqueuemessage (MSG, Flashloglevel.warn, ex); }}///<summary>///log level///</summary> public enum Flashloglevel {Debug, I NFO, Error,        Warn, Fatal}///<summary>//log content///</summary> public class Flashlogmessage        {public string Message {get; set;}        Public Flashloglevel level {get; set;}    Public Exception Exception {get; set;} } }

4, performance comparison and application 4.1, performance comparison

Tested and found

Using the original log4net to write to the log 100,000 data requires:19104 milliseconds.

It only takes 251 milliseconds for the data to be queued in the same way.

4.2, application 4.2.1, need to register at the start of the program, such as the Global.asax in the ASP. Application_Start registration.

    public class MvcApplication:System.Web.HttpApplication    {        protected void Application_Start ()        {            Arearegistration.registerallareas ();            Filterconfig.registerglobalfilters (globalfilters.filters);            Routeconfig.registerroutes (routetable.routes);            Bundleconfig.registerbundles (bundletable.bundles);           flashlogger.instance (). Register ();        }    }

4.2.2, call Flashlogger's static method directly where you want to write the log.

            Flashlogger.debug ("Debug");            Flashlogger.debug ("Debug", New Exception ("TestException"));            Flashlogger.info ("Info");            Flashlogger.fatal ("Fatal");            Flashlogger.error ("Error");            Flashlogger.warn ("Warn", New Exception ("TestException"));

C # Ultra High performance write log code open source