Rotten mud: Cut nginx logs, rotten nginx logs

Rotten mud: Cut nginx logs, rotten nginx logs

This article is sponsored by ilanniweb and first published onThe world

For more articles, follow my ilanniweb

I read the blog's nginx logs over the past few days and found that the log files are getting bigger and bigger.

Because nginx does not cut log files by itself, it is intended to cut logs in other ways. In order to analyze the data recorded in the nginx log files in the future, therefore, we plan to cut it by day.

PS: In this article, we will perform the following operations on centos 6.5 OS 64bit.

To cut nginx logs, we can use two different methods: Through logrotate and through shell scripts.

If nginx is installed in yum mode, the system automatically uses the logrotate log management software by day. However, after multiple tests, it was found that logrotate cut logs were not separated by their own time points, which was a bit disappointing.

Therefore, we recommend that you use shell scripts to cut logs for installation in yum or source code.

To learn more about the linux system, we will also introduce logrotate-related knowledge points.

The two methods are described as follows:

1. Use logrotate to cut

As mentioned above, nginx installed in the yum or apt-get method will automatically use the logrotate log management software for cutting. Therefore, this section describes the logrotate-related knowledge points.

1.1 logrotate Introduction

What is logrotate? It is a log management tool for linux systems. It can cut, compress and other software's log file software.

Logrotate runs Based On crontab, so this time point is controlled by crontab. You can query the configuration file/etc/anacrontab of crontab.

Run the following command to install logrotate:

Yum-y install logrotate

For ubuntu, run the following command:

Apt-get-y install logrotate

To view the logrotate configuration file, run the following command:

Rpm-ql logrotate

We can see that the logrotate configuration file is/etc/logrotate. conf, and/etc/logrotate. d/is the directory used to store other configuration files. All files in this directory will be actively read into/etc/logrotate. conf for execution.

1.2 logrotate Configuration file details

Logrotate has fewer configuration options. To meet this article, we will take the nginx configuration file as an example:

Cat/etc/logrotate. d/nginx

/Var/log/nginx/*. log {

Daily

Missingok

Rotate 7

Dateext

# Compress

Delaycompress

Notifempty

Create 640 nginx adm

Sharedscripts

Postrotate

[-F/var/run/nginx. pid] & kill-USR1 'cat/var/run/nginx. pid'

Endscript

}

In this configuration file, each parameter has the following functions:

/Var/log/nginx/is the directory where nginx logs are stored. You can modify the directory according to the actual situation.

Daily: log files are rotated by day.

Weekly: log files are cyclically distributed.

Monthly: log files are collected on a monthly basis.

Missingok: During log rotation, any errors will be ignored, such as errors such as "file not found.

Rotate 7: stores 7 log files at a time. For 8th log files, the log file with the longest time will be deleted.

Dateext: defines the log file suffix as the date format, that is, after the cut file is: xxx.log-20160402.gz such format. If this parameter is commented out, the split is incremental by number, that is, the format of xxx. log-1 mentioned above.

Compress: after the round robin task is completed, the archive that has been Round Robin will be compressed using gzip.

Delaycompress: always used with the compress option. The delaycompress option instructs logrotate not to compress the latest archive. Compression will be performed in the next cycle. This is useful when you or any software still needs to read the latest archive.

Receivempty: if the file is empty, no dumping is performed.

Create 640 nginx adm: Creates a new log file with the specified permissions and book attributes, and logrotate also renames the original log file.

Postrotate/endscript: after all other commands are completed, the specified commands in postrotate and endscript will be executed. In this case, the rsyslogd process immediately reads its configuration and continues running. Note: These two keywords must be entered separately.

1.3 View the default logrotate execution time

By default, logrotate is cut at every day. This time point can be viewed in the crontab configuration file. As follows:

Cat/etc/anacrontab

SHELL =/bin/sh

PATH =/sbin:/bin:/usr/sbin:/usr/bin

MAILTO = root

# The maximal random delay added to the base delay of the jobs

RANDOM_DELAY = 45

# The jobs will be started during the following hours only

START_HOURS_RANGE = 3-22

1 5 cron. daily nice run-parts/etc/cron. daily

7 25 cron. weekly nice run-parts/etc/cron. weekly

@ Monthly 45 cron. monthly nice run-parts/etc/cron. monthly

The START_HOURS_RANGE parameter indicates the time point when logrotate is configured.

If you want to cut logs by logrotate at a specified time point, you can modify the START_HOURS_RANGE parameter of this configuration file.

However, after many experiments, I found that logrotate did not cut logs at the specified time.

Finally, I chose to use the shell script to cut nginx logs.

Ii. Use shell scripts to cut

It is easy to use shell scripts to cut nginx logs. The shell script content is as follows:

Vim/usr/local/cut_del_logs.sh

#! /Bin/bash

# Initialization

LOGS_PATH =/var/log/nginx

YESTERDAY = $ (date-d "yesterday" + % Y % m % d)

# Daily log Cutting

Mv $ {LOGS_PATH}/ilanni.com. log $ {LOGS_PATH}/ilanni.com _ $ {YESTERDAY}. log

# Send the USR1 signal to the nginx main process and re-open the log file. Otherwise, data will continue to be written to the mv file. The reason is: in linux, the kernel finds files based on file descriptors. Otherwise, log splitting fails.

Kill-USR1 'ps axu | grep "nginx: master process" | grep-v grep | awk '{print $2 }''

# Delete logs 7 days ago

Cd $ {LOGS_PATH}

Find.-mtime + 7-name "* 20 [1-9] [3-9] *" | xargs rm-f

# Or

# Find.-mtime + 7-name "ilanni.com _ *" | xargs rm-f

Exit 0

The shell script has two functions: the first is to cut nginx logs, and the second is to delete nginx logs seven days ago.

In the nginx log cutting function, we should note that the shell script named the cut log is named after the previous day when the log is cut.

Therefore, when we put the shell script in crontab for execution, we recommend that you execute it at 00:00 every day. As follows:

Vim/etc/crontab

0 0 *** root/bin/sh/usr/local/cut_del_logs.sh

View the log files cut by nginx as follows:

Ll -- full-time/var/log/nginx/

Through this, we can clearly see that the cut shell script does cut and name the log file as we imagined.