Nginx log analysis-mark specific content with sed color

Nginx log analysis-mark specific content with sed color
Filter logs in real time

Only the log lines that return status code 500 in nginx access logs are displayed:

tail -f access_log.log | grep 500 --color

Note: After tail-f, only pipelines can be used once. The following command will not output any

tail -f access_log.log | grep 500 | grep 500

 

Mark specific content in the log by color

For example, the nginx log format is:

 log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '                         '$status $body_bytes_sent "$http_referer" '                         '"$http_user_agent" "$http_x_forwarded_for"';

The log Content is:

192.168.1.181 - - [13/Apr/2011:15:19:10 +0800] "GET /tomcat.png HTTP/1.1" 304 0 "http://192.168.1.9/" "Mozilla/5.0 (Windows NT 6.1; rv:2.0) Gecko/20100101 Firefox/4.0" "-"192.168.1.181 - - [13/Apr/2011:15:19:10 +0800] "GET /favicon.ico HTTP/1.1" 304 0 "-" "Mozilla/5.0 (Windows NT 6.1; rv:2.0) Gecko/20100101 Firefox/4.0" "-"192.168.1.181 - - [13/Apr/2011:15:19:10 +0800] "GET /bg-nav.png HTTP/1.1" 304 0 "http://192.168.1.9/tomcat.css" "Mozilla/5.0 (Windows NT 6.1; rv:2.0) Gecko/20100101 Firefox/4.0" "-"192.168.1.181 - - [13/Apr/2011:15:19:10 +0800] "GET /bg-upper.png HTTP/1.1" 304 0 "http://192.168.1.9/tomcat.css" "Mozilla/5.0 (Windows NT 6.1; rv:2.0) Gecko/20100101 Firefox/4.0" "-"192.168.1.181 - - [13/Apr/2011:15:19:10 +0800] "GET /bg-middle.png HTTP/1.1" 304 0 "http://192.168.1.9/tomcat.css" "Mozilla/5.0 (Windows NT 6.1; rv:2.0) Gecko/20100101 Firefox/4.0" "-"192.168.1.181 - - [13/Apr/2011:15:19:10 +0800] "GET /bg-button.png HTTP/1.1" 304 0 "http://192.168.1.9/tomcat.css" "Mozilla/5.0 (Windows NT 6.1; rv:2.0) Gecko/20100101 Firefox/4.0" "-"192.168.1.114 - - [13/Apr/2011:15:19:37 +0800] "GET / HTTP/1.0" 200 12220 "-" "-" "-"192.168.1.114 - - [13/Apr/2011:15:20:22 +0800] "GET / HTTP/1.0" 200 12220 "-" "-" "-"

Requirement: Mark requests whose return status code is not 200

If you use grep only, use the following command:

grep -v "200" access_log.log

Use sed to mark a non-200 status code in color:

To spell out the correct Regular Expression of sed, we start from marking 200 as green.

sed 's/200/\x1b[32m&\x1b[0m/g' access_log.log

Note: echo uses the octal character \ 033 to print color characters, but sed does not support octal characters. hexadecimal: \ x1b must be used.

Next, mark status code 3XX as yellow:

sed 's/3[0-9][0-9]/\x1b[33m&\x1b[0m/g' access_log.log

However, note that there are numbers in other parts of the nginx log line. The above matching is not accurate enough.

Next, mark the three-digit color after HTTP/1.0 "or HTTP/1.1:

sed 's/\(HTTP\/1\.[01]" \)\(3[0-9][0-9]\)/\1\x1b[33m\2\x1b[0m/g' access_log.log

Next, if the returned data volume after the status code is greater than 1 K, it is marked with red:

sed 's/\(HTTP\/1\.[01]" [0-9][0-9][0-9] \)\([0-9]\+\)[0-9][0-9][0-9]/\1\x1b[31m[\2KB]\x1b[0m/g' access_log.log

Research on Nginx configuration and deployment, Upstream load balancing Module

Deployment of Nginx + MySQL + PHP in CentOS 6.2

Build a WEB server using Nginx

Build a Web server based on Linux6.3 + Nginx1.2 + PHP5 + MySQL5.5

Performance Tuning for Nginx in CentOS 6.3

Configure Nginx to load the ngx_pagespeed module in CentOS 6.3

Install and configure Nginx + Pcre + php-fpm in CentOS 6.4

Nginx installation and configuration instructions

Nginx log filtering using ngx_log_if does not record specific logs

Nginx details: click here
Nginx: click here

This article permanently updates the link address: