C # high-speed, high-performance, open-source log writing code,

C # high-speed, high-performance, open-source log writing code,
1. Requirements

The requirement is very simple, that is, writing logs at high speed in C # development. For example, in high concurrency and high traffic, logs must be written. We know that the program is time-consuming when operating the disk. Therefore, writing logs to the disk consumes a certain amount of time, which is not what we want to see.

 

2. Solution 2.1. simple principles

UseQueueFirst cache to the memory, then we always haveThreadWrite logs to the disk from the queue so that logs can be written at high speed and high performance. Because of the slow speed, we have separated it. That is to say, after the program throws the log to the queue, the log part of the program is completed, and the time-consuming part of the program that operates the disk later does not need to be concerned, operated by another thread.

As the saying goes, the fish and the bear's paw cannot have both sides. This will cause a problem, that is, if the log has reached the queue, when the program crashes or the computer loses power, it will cause part of the log to be lost, however, in some cases, whether or not logs can be written with high performance can be ignored, depending on the situation.

 

2.2. Example

 

3. key code

The LZ used to write logs here selects the commonly used log4net. Of course, you can also select other log components, such as nlog.

3.1 log to the queue Section

First, we need to put the logs in the queue before writing them to the disk.

Public void EnqueueMessage (string message, FlashLogLevel level, Exception ex = null) {if (level = FlashLogLevel. debug & _ log. isDebugEnabled) | (level = FlashLogLevel. error & _ log. isErrorEnabled) | (level = FlashLogLevel. fatal & _ log. isFatalEnabled) | (level = FlashLogLevel. info & _ log. isInfoEnabled) | (level = FlashLogLevel. warn & _ log. isWarnEnabled) {_ que. enqueue (new FlashLogMessage {Message = "[" + DateTime. now. toString ("yyyy-MM-dd HH: mm: ss, fff") + "] \ r \ n" + message, Level = level, Exception = ex }); // The Notification thread writes the log _ mre to the disk. set ();}}

_ Log is the ILog of the log4net log component, which includes functions such as writing logs and determining the log level. if judgment at the beginning of the Code is a comparison between the judgment level and the current log level, to check whether a queue needs to be written, this can effectively improve the log performance.

The _ que is a ConcurrentQueue queue. _ Mre is the ManualResetEvent signal, and ManualResetEvent is used to notify the thread queue of New logs, which can be written to the disk from the queue. After writing logs from the queue, reset the signal and wait for a new log to arrive.

 

3.2 queue to disk

From the queue to the disk, we need a thread to write data from the queue to the disk. That is to say, we need to load this thread when the program starts. For example, in asp.net, we needGlobalInApplication_StartLoad.

/// <Summary> /// log recorded by another thread, called only once during program initialization /// </summary> public void Register () {Thread t = new Thread (new ThreadStart (WriteLog); t. isBackground = false; t. start () ;}//< summary> /// write logs from the queue to the disk /// </summary> private void WriteLog () {while (true) {// wait for the signal to notify _ mre. waitOne (); // determine whether content exists such as disk while (_ que. count> 0) {FlashLogMessage msg; if (_ que. tryDequeue (out msg) // obtain the content from the queue, delete the content in the queue {// determine the log level, and then write the log switch (msg. level) {case FlashLogLevel. debug: _ log. debug (msg. message, msg. exception); break; case FlashLogLevel. info: _ log. info (msg. message, msg. exception); break; case FlashLogLevel. error: _ log. error (msg. message, msg. exception); break; case FlashLogLevel. warn: _ log. warn (msg. message, msg. exception); break; case FlashLogLevel. fatal: _ log. fatal (msg. message, msg. exception); break ;}}// reset the signal _ mre. reset ();}}

 

3.3 complete code: using log4net; using log4net. config; using System. collections. concurrent; using System. IO; using System. threading; namespace Emrys. flashLog {public sealed class FlashLogger {// <summary> // record the Message Queue /// </summary> private readonly ConcurrentQueue <FlashLogMessage> _ que; /// <summary> /// signal /// </summary> private readonly ManualResetEvent _ mre; /// <summary> /// log /// </summary> private readonly ILog _ log; /// <summary> /// log /// </summary> private static FlashLogger _ flashLog = new FlashLogger (); private FlashLogger () {// set the log configuration file path XmlConfigurator. configure (new FileInfo (Path. combine (AppDomain. currentDomain. baseDirectory, "log4net. config "); _ que = new ConcurrentQueue <FlashLogMessage> (); _ mre = new ManualResetEvent (false); _ log = LogManager. getLogger (System. reflection. methodBase. getCurrentMethod (). declaringType) ;}/// <summary> /// implement Singleton /// </summary> /// <returns> </returns> public static FlashLogger Instance () {return _ flashLog;} // <summary> // logs recorded by another thread, called only once during program initialization /// </summary> public void Register () {Thread t = new Thread (new ThreadStart (WriteLog); t. isBackground = false; t. start () ;}//< summary> /// write logs from the queue to the disk /// </summary> private void WriteLog () {while (true) {// wait for the signal to notify _ mre. waitOne (); // determine whether content exists such as disk while (_ que. count> 0) {FlashLogMessage msg; if (_ que. tryDequeue (out msg) // obtain the content from the queue, delete the content in the queue {// determine the log level, and then write the log switch (msg. level) {case FlashLogLevel. debug: _ log. debug (msg. message, msg. exception); break; case FlashLogLevel. info: _ log. info (msg. message, msg. exception); break; case FlashLogLevel. error: _ log. error (msg. message, msg. exception); break; case FlashLogLevel. warn: _ log. warn (msg. message, msg. exception); break; case FlashLogLevel. fatal: _ log. fatal (msg. message, msg. exception); break ;}}// reset the signal _ mre. reset ();}} /// <summary> /// write logs /// </summary> /// <param name = "message"> log text </param> /// <param name = "level"> level </param> // <param name = "ex"> Exception </param> public void EnqueueMessage (string message, flashLogLevel level, Exception ex = null) {if (level = FlashLogLevel. debug & _ log. isDebugEnabled) | (level = FlashLogLevel. error & _ log. isErrorEnabled) | (level = FlashLogLevel. fatal & _ log. isFatalEnabled) | (level = FlashLogLevel. info & _ log. isInfoEnabled) | (level = FlashLogLevel. warn & _ log. isWarnEnabled) {_ que. enqueue (new FlashLogMessage {Message = "[" + DateTime. now. toString ("yyyy-MM-dd HH: mm: ss, fff") + "] \ r \ n" + message, Level = level, Exception = ex }); // The Notification thread writes the log _ mre to the disk. set () ;}} public static void Debug (string msg, Exception ex = null) {Instance (). enqueueMessage (msg, FlashLogLevel. debug, ex);} public static void Error (string msg, Exception ex = null) {Instance (). enqueueMessage (msg, FlashLogLevel. error, ex);} public static void Fatal (string msg, Exception ex = null) {Instance (). enqueueMessage (msg, FlashLogLevel. fatal, ex);} public static void Info (string msg, Exception ex = null) {Instance (). enqueueMessage (msg, FlashLogLevel. info, ex);} public static void Warn (string msg, Exception ex = null) {Instance (). enqueueMessage (msg, FlashLogLevel. warn, ex) ;}/// <summary> /// Log Level /// </summary> public enum FlashLogLevel {Debug, Info, Error, Warn, fatal} /// <summary> /// log Content /// </summary> public class FlashLogMessage {public string Message {get; set;} public FlashLogLevel Level {get; set ;}public Exception {get; set ;}}}View Code

 

4. Performance Comparison and application 4.1 and Performance Comparison

Tested and found

To write 100000 entries of log data using the original log4net, You need:29178 Ms.

Similarly, data queuing takes only 261 milliseconds.

 

 

4.2. The application 4.2.1 needs to be registered when the program starts, for example, Application_Start registration in Global. asax in asp.net.

    public class MvcApplication : System.Web.HttpApplication    {        protected void Application_Start()        {            AreaRegistration.RegisterAllAreas();            FilterConfig.RegisterGlobalFilters(GlobalFilters.Filters);            RouteConfig.RegisterRoutes(RouteTable.Routes);            BundleConfig.RegisterBundles(BundleTable.Bundles);           FlashLogger.Instance().Register();        }    }

 

4.2.2. directly call the static method of FlashLogger where logs need to be written.

            FlashLogger.Debug("Debug");            FlashLogger.Debug("Debug", new Exception("testexception"));            FlashLogger.Info("Info");            FlashLogger.Fatal("Fatal");            FlashLogger.Error("Error");            FlashLogger.Warn("Warn", new Exception("testexception"));

 

5. Open-source code

Https://github.com/Emrys5/Emrys.FlashLog

 

I hope it will be helpful to you. This article is original and you are welcome to make a brick andRecommendation.