GO Logger Log Practice

This is a creation in Article, where the information may have evolved or changed.

Analyze the log packet of the used print log

1. Go standard library comes with log, this log has less func, does not distinguish level, but simple enough, has prefix function, can set flag to control the time format, caller file name and line number, other standard package such as Net/http Database/sql This package is used as well.
2. Pack your own log, add level, color. such as Ngaut/log, this log star is not many, or from a recent fire in a project PINGCAP/TIDB see, a little fresh feeling, but this log may be used for TIDB, the lack of some of the methods of log, resulting in the inability to use in some third-party libraries such as Gorm that can be customized logger. So I fork the https://github.com/hanjm/log and add some methods so that it can be used for Gorm.
3. Completely self-fulfilling log, structured output, usually key=value or JSON, famous for having Logrus, ZAP, etc. The first time to see Logrus feel beautiful, so a lot of use, until the attention TIDB when the issues mail with the log, the inside of the log with caller, feel very useful, so to search Logrus issue see if there is this function, Found a issuehttps://github.com/sirupsen/logrus/issues/63, discussed for three years this feature has not yet added, had to give up the beautiful Logrus, find a replacement zap, Zap's design is very good, custom strong. Log is the code that is often called, and each invocation inevitably allocates memory, and the number of allocations and the amount of memory allocated each time will affect performance. The processing of log content is also a point of performance, like log. printf parameter is interface{}, Logrus field is map[stirng]interface{}, print interface{} can only rely on reflect, go is static strong type language, with reflection overhead is relatively large, So zap uses a manually specified type of way, from the benchmark provided by ZAP to see the performance improvement is quite large, although compared to Logrus used more cumbersome, but for performance, it is worthwhile.

Summarize

1. in order to facilitate log analysis, the uniform use of JSON row logs, so that the use of elk can be avoided by custom regular expression stored in the Elasticsearch field.
2. Net/http Database/sql and some third-party packages may use the log of the label library directly, and a trick can change all the behavior of using the standard package log through log. Setoutput (w io). Writer) To change the position, W is an IO that implements the Write (p []byte) (n int, err Error) method. Writer can.
3. Runtime. Caller can get the caller's PC, filename, number of file lines, runtime. FUNCFORPC (PC). Name () can get the function name of the PC, which is very helpful for debug. However, there is a performance overhead, so the scenario is: for HTTP server access log, there is no need to use the log with caller, and for the specific implementation of the HTTP API log in the function, it is necessary to record caller, and the light has a file name and the number of rows is not enough, After all, change the number of lines of code changed, and function names generally do not change, with the functions of the name is more intuitive.
4. GitHub searched a lap, many companies will customize their own log, such as Tidb Ngaut/log, seven cattle qiniu/log, hungry Eleme/log, Mailgun mailgun/log, yes, I also built a small wheel zaplog.zaplog is packaged with ZAP, with caller func name, compatible with Logrus Stdlog's log output tool.

Package Zaplogimport ("bytes" "FMT" "Go.uber.org/zap" "Go.uber.org/zap/zapcore" "Log" "Runtime" "strings")// Callerencoder'll add caller to log. Format is "Filename:lineNum:funcName", e.g: "zaplog/zaplog_test.go:15:zaplog. Testnewlogger "func callerencoder (caller Zapcore. Entrycaller, enc zapcore. Primitivearrayencoder) {Enc. AppendString (Strings. Join ([]string{caller. Trimmedpath (), runtime. FUNCFORPC (caller. PC). Name ()}, ":"))}func Newloggerconfig (debuglevel bool) (Loggerconfig Zap. Config) {loggerconfig = zap. Newproductionconfig () LoggerConfig.EncoderConfig.EncodeTime = Zapcore. ISO8601TimeEncoderloggerConfig.EncoderConfig.EncodeCaller = Callerencoderif DebugLevel {loggerconfig.level = zap. Newatomiclevelat (Zap. DebugLevel)}return}//Newcustomloggers is a shortcut to get normal logger, Nocallerlogger.func newcustomloggers ( debuglevel bool) (logger, Nocallerlogger *zap. Logger) {loggerconfig: = Newloggerconfig (debuglevel) Logger, err: = Loggerconfig.build () if err! = Nil {panic (err)} Loggerconfig.disablecalLer = Truenocallerlogger, err = Loggerconfig.build () if err! = Nil {panic (err)}return}//Newlogger return a normal LOGGERFU NC newlogger (debuglevel bool) (Logger *zap. Logger) {loggerconfig: = Newloggerconfig (debuglevel) Logger, err: = Loggerconfig.build () if err! = Nil {panic (err)}return} Newnocallerlogger return a no caller key value, 'll be Fasterfunc Newnocallerlogger (debuglevel bool) (Nocallerlogger * Zap. Logger) {loggerconfig: = Newloggerconfig (debuglevel) Loggerconfig.disablecaller = Truenocallerlogger, err: = Loggerconfig.build () if err! = Nil {panic (err)}return}//Compatiblelogger is a logger which compatible to LOGRUS/STD LOG/PR ometheus.//it implements Print () Println () Printf () Dbug () Debugln () DEBUGF () Info () Infoln () Infof () Warn () Warnln () War  NF ()//Error () Errorln () Errorf () Fatal () Fataln () fatalf () Panic () Panicln () PANICF () with () Withfield () withfields () type Compatiblelogger struct {_log *zap. logger}//Newcompatiblelogger return Compatiblelogger with caller Fieldfunc NewCompatiblelogger (debuglevel bool) *compatiblelogger {return &compatiblelogger{newlogger (debuglevel). Withoptions (Zap. Addcallerskip (1))}}//Print logs a message at level Info on the Compatiblelogger.func (L compatiblelogger) print (args ... i nterface{}) {L._log. Info (FMT. Sprint (args ...)} PRINTLN logs a message at level Info on the Compatiblelogger.func (L compatiblelogger) Println (args ... interface{}) {L. _log. Info (FMT. Sprint (args ...)} printf logs a message at level Info on the Compatiblelogger.func (L compatiblelogger) Printf (format string, args ... int erface{}) {L._log. Info (FMT. Sprintf (format, args ...)} Debug logs a message at level debug on the Compatiblelogger.func (L compatiblelogger) debug (args ... interface{}) {L._lo G.debug (FMT. Sprint (args ...)} DEBUGLN logs a message at level Debug on the Compatiblelogger.func (L compatiblelogger) debugln (args ... interface{}) {L . _log. Debug (FMT. Sprint (args ...)} DEBUGF logs a message at level Debug on the Compatiblelogger.func (L CompatIblelogger) DEBUGF (format string, args ... interface{}) {L._log. Debug (FMT. Sprintf (format, args ...)} Info logs a message at level Info on the Compatiblelogger.func (L compatiblelogger) Info (args ... interface{}) {L._LOG.I NFO (FMT. Sprint (args ...)} INFOLN logs a message at level Info on the Compatiblelogger.func (L compatiblelogger) infoln (args ... interface{}) {l._l og. Info (FMT. Sprint (args ...)} INFOF logs a message at level Info on the Compatiblelogger.func (L compatiblelogger) infof (format string, args ... inter face{}) {L._log. Info (FMT. Sprintf (format, args ...)} Warn logs a message at level Warn on the Compatiblelogger.func (L compatiblelogger) Warn (args ... interface{}) {L._LOG.W ARN (FMT. Sprint (args ...)} WARNLN logs a message at level Warn on the Compatiblelogger.func (L compatiblelogger) warnln (args ... interface{}) {l._l og. Warn (FMT. Sprint (args ...)} WARNF logs a message at level Warn on the Compatiblelogger.func (L compatiblelogger) warnf (format string, args ... inter face{}) {L. _log. Warn (FMT. Sprintf (format, args ...)} Error logs a message at level error on the Compatiblelogger.func (L compatiblelogger) error (args ... interface{}) {L._lo G.error (FMT. Sprint (args ...)} ERRORLN logs a message at level Error on the Compatiblelogger.func (L compatiblelogger) errorln (args ... interface{}) {L . _log. Error (FMT. Sprint (args ...)} Errorf logs a message at level Error on the Compatiblelogger.func (L compatiblelogger) Errorf (format string, args ... in terface{}) {L._log. Error (FMT. Sprintf (format, args ...)} Fatal logs a message at level Fatal on the Compatiblelogger.func (L compatiblelogger) Fatal (args ... interface{}) {L._lo G.fatal (FMT. Sprint (args ...)} Fatalln logs a message at level Fatal on the Compatiblelogger.func (L compatiblelogger) Fatalln (args ... interface{}) {L . _log. Fatal (FMT. Sprint (args ...)} Fatalf logs a message at level Fatal on the Compatiblelogger.func (L compatiblelogger) fatalf (format string, args ... in terface{}) {L._log. Fatal (FMT. Sprintf (format, args ...))} Panic logs a message at level Painc on the Compatiblelogger.func (L compatiblelogger) Panic (args ... interface{}) {L._lo G.panic (FMT. Sprint (args ...)} PANICLN logs a message at level Painc on the Compatiblelogger.func (L compatiblelogger) panicln (args ... interface{}) {L . _log. Panic (FMT. Sprint (args ...)} PANICF logs a message at level Painc on the Compatiblelogger.func (L compatiblelogger) PANICF (format string, args ... in terface{}) {L._log. Panic (FMT. Sprintf (format, args ...)} With return a logger with a extra field.func (L-*compatiblelogger) with (key string, Value interface{}) *compatiblelogg ER {return &compatiblelogger{l._log. With (Zap. Any (key, value))}}//Withfield return a logger with an extra Field.func (l *compatiblelogger) Withfield (key string, value interface{}) *compatiblelogger {return &compatiblelogger{l._log. With (Zap. Any (key, value))}}//Withfields return a logger with extra Fields.func (L-*compatiblelogger) withfields (Fields map[string] interface{}) *compAtiblelogger {i: = 0var clog *compatibleloggerfor k, V: = Range fields {if i = = 0 {clog = L.withfield (k, v)} else {clog = Clog. Withfield (k, v)}i++}return clog}//Formatstdlog Set the output of stand package log to Zaplogfunc Formatstdlog () {log. SetFlags (log. Llongfile) log. Setoutput (&logwriter{newnocallerlogger (False)})}type logwriter struct {logger *zap. logger}//Write implement IO. Writer, as STD log ' s Outputfunc (w logwriter) Write (P []byte) (n int, err error) {i: = bytes. Index (P, []byte (":")) + 1j: = bytes. Index (p[i:], []byte (":")) + 1 + icaller: = bytes. TrimRight (P[:j], ":")//Find last Index of/i = bytes. LastIndex (caller, []byte ("/"))//Find penultimate Index of/i = bytes. LastIndex (Caller[:i], []byte ("/")) W.logger.info ("Stdlog", Zap. ByteString ("Caller", Caller[i+1:]), Zap. ByteString ("Log", bytes. Trimspace (p[j:])) return Len (P), nil}