Nginx Automatic Cutting Access log method one

The Web Access log (Access_log) records the access behavior of all external clients to the Web server, including important information such as the client IP, the access date, the URL resource accessed, the HTTP status code returned by the server, and so on.
A typical Web Access log is as follows:

112.97.37.90--[14/sep/2013:14:37:39 +0800] "get/http/1.1" 301 5 "-" "mozilla/5.0 (Linux; U Android 2.3.6; ZH-CN; Lenovo A326 build/grk39f) applewebkit/533.1 (khtml, like Gecko) version/4.0 Mobile safari/533.1 micromessenger/4.5.1.259 " -

Planning:

1, to solve the problem:

When the amount of website access is large, the log data will be many, if all write to a log file, the file will become more and more large. The file will slow down at a large speed, such as a file hundreds of trillion. When the log is written, it affects the speed of operation. Also, if I want to see an access log, a hundreds of trillion file, download it down and open it very slowly. Use the third party free Log analysis tool-log treasure, you can upload Nginx, Apache, IIS log files, they help analyze the site security aspects. After all, specialization, more professional. Log treasure to upload the file is also made a size limit, not more than 50m.

2, Nignx does not automatically separate the file storage log mechanism. Because Nginx it will not help you automatically save files. Therefore, you need to write your own script to implement.

The shell script file nginx_log_division.sh content is as follows:

#/bin/bash

Logs_path= "/data/wwwlogs/"

#以前的日志文件.

Log_name= "Xxx.log"

Pid_path= "/usr/local/nginx/logs/nginx.pid"

MV ${logs_path}${log_name} ${logs_path}${log_name}_$ (Date--date= "last WEEK" + "%y-%m-d"). Log

KILL-USR1 ' Cat ${pid_path} '

The above shell script principle is: The previous log file move renamed to one, the purpose is to backup.

According to the date of last Monday, when the script was run "2013-09-16", the generated file name is "Xxx.log_ 20130909.log".

Before KILL-USR1 ' Cat ${pid_path} ' was executed, the file name was changed to the newly named file "Xxx.log_ 20130909" as usual, even though the MV command was performed on the file. The reason: In Linux systems, the kernel is based on file descriptors to find files.

----------------understanding of Linux file descriptors

A file descriptor is an integer identifier that is named by the Linux kernel for each open file.

The Linux kernel generates (or maintains) a " file descriptor " for each process, which records "files opened by this process (identified)".

In this environment, Nginx is a running process, this process has already opened a log file, the file descriptor is recorded in the file.

Even if the path to the log file changes, it can be found (based on the file descriptor table).

----------------------------------------------

When executing the command "KILL-USR1 ' Cat ${pid_path} '", the Nginx.pid file is actually a number (you can open a look, I am here 894), Nginx will its main process PID (process number) is written to the Nginx.pid file, so you can directly manipulate the specified process number by using the Cat command to get its main process number.

KILL-USR1 ' Cat ${pid_path} ' is equivalent to

KILL–USR1 894 #指定发信号 (USR1) signal to the process number.

In Linux systems, Linux communicates with "running processes" through signals. Linux systems, there are many predefined good signals, like sighup. USR1 is a user-defined signal. It can be understood that the process defines itself to this signal what to do (that is, the process writer himself is sure to get the signal to do nothing, it is entirely up to the developer to decide). And in Nginx, it wrote its own code processing when I received the USR1 signal, let Nginx reopen the log file. The specific principle is as follows:

1, nginx main process received USR1 signal, will reopen the log file (named after the log name in the Nginx configuration file, is the value set by the Access_log item in the configuration file, if the file does not exist, a new file Xxx.log will be created automatically).

2. Then change the owner of the log file to "worker process" so that the worker process has read and write access to the log file (master and worker are usually run with different users, so the owner needs to be changed).

3. The Nginx main process closes the log file with the same name (that is, the file that was renamed to Xxx.log_ 20130909.log using the MV command) and notifies the worker process to use the newly opened log file (the file xxx.log that the main process opened just now). The specific implementation of the more granular point is that the main process sends the USR1 signal to worker,worker received this signal, will reopen the log file (that is, the Xxx.log in the configuration file)

=================================== timed Script execution

Set the shell script file above to add to the scheduled task. Crontab is a scheduled task process under Linux. Boot This process will start, and it will go to its own list every once in a while to see if there are tasks that need to be performed.

Crontab-e

* * * * 1/data/wwwlogs/nginx_log_division.sh

will open a file and add the code above

The format is "time-of-day day of the week to execute the shell file path." Use * can be understood as "every", every minute, every hour, every month and so on.

I set the nginx_log_division.sh script to run on Monday 4 o'clock in the morning, and the script's content is to regenerate a new log file.

Attached: setting the nginx Log Configuration Method

Log_format site ' $remote _addr-$remote _user [$time _local] "$request" '

' $status $body _bytes_sent ' $http _referer '

' "$http _user_agent" $http _x_forwarded_for ';

Access_log/data/wwwlogs/xxxx.com.log site

#第二个参数表示使用那个日志格式, a name is identified for each log format, and site corresponds to the name in Log_format

Nginx Automatic Cutting Access log method one