Tracking and debugging of nginx source code

Tracking and debugging of source code is indispensable when learning excellent source code. It is not only an effective way for us to solve program bugs, but also an effective way for us to understand and learn excellent source code.

This article mainly introduces some source code debugging methods and uses nginx source code for examples.

1. debug with GDB

A. First, you should be familiar with some basic commands for GDB debugging (unfamiliar steps
Use GDB to debug the program and be familiar with commands such as step, run, break, list, info, and continue ).

B, download nginx source code, here use nginx-1.0.14, unzip the file. The auto folder contains various command sets during configure running, and SRC is the source code. To use

To debug nginx through GDB, you must enable the-G compilation option when generating the nginx program. We need to modify
In the Auto/CC/CONF file, add the-G option to ngx_compile_opt = "-c ".

Change to ngx_compile_opt = "-C-G", execute the configure command: sudo./configure next, and run the command: Vim objs/makefile to confirm that the-G parameter is

No. The result is as follows.

-G compilation option is enabled, and then run the command: sudo make. (If make has been executed before, make sure that it can be re-compiled for the second time.

Refresh the timestamps of all source files to indirectly recompile a new nginx executable program. The command is: Find. -Name "*. C "| xargs touch.

C. Start nginx and run the following command in the objs Directory: sudo. /nginx: After nginx is successfully run, run the command: PS-Ef | grep nginx to view the nginx master and Worker Process PID,

If you perform GDB debugging on nginx worker 2177, you can use the-P option of GDB. Command:
GDB-P 2177.

Debugging is troublesome when multiple working processes exist. You can modify the configuration item and the nginx. conf file. Add master_process off; single-process mode will monitor processes and work

Process logic is all in one process. At this time, there is only one process that can be debugged by GDB.

We know that the worker process will stay at epoll_wait and wait for the corresponding event to occur. This function call is encapsulated in ngx_process_events_and_timers. So we set a breakpoint in this function:

B ngx_event.c: ngx_process_events_and_timers, combined with the gdb command C, S, N:

Run the C command to keep nginx running until the first breakpoint is reached. The event processing method is ngx_process_events. Therefore, we use command s to track this function. when the epoll_wait function is executed

The current process stays here and cannot be executed downward. This proves that the worker sub-process is blocked at the epoll_wait function call.

In this case, run the following command on another terminal to send a message to nginx: curl-I localhost. You can see that the request has been sent and is waiting for a response.

In this case, you can continue to execute command C to get a response from another terminal. In this case, you can run the BT command to view the function call process in single-process mode ..

Curl

GDB BT

2. debug nginx using strace and pstack

A. strace is often used to track system calls and received signals during process execution. In the Linux World, processes cannot directly access hardware devices. When a process needs to access hardware devices (such as reading disk files,

When receiving network data, you must switch from user mode to kernel mode to access the hardware device through system calls. Strace can trace the system calls generated by a process, including parameters,

Returned value: the time consumed by the execution. In addition, the ltrace command is used to view function calls in a dynamic library.

B. Which function is called? The answer cannot be found in the strace output result, because the output shows that all of them are system calls. to display the function call stack information in the program, it is pstack's turn.

Pstack is a script tool. Its core implementation is to use the GDB and thread apply all BT commands.

Run man to view the specific usage of strace, ltrace, and pstack in man manual.

Here we modify the nginx configuration file nginx. conf so that it only has one master process and one worker process. Add the modified master_process off in 1 #

Add worker_processes 1 to the configuration file;

Disable nginx and restart nginx. Run the "PS-Ef | grep nginx" command to view the current nginx process, and use the "-P" option of the strace command to track the nginx working process ,.

We can see that the working process is blocked on the epoll_wait system call, because there is no client request at this time, nginx is blocked here, And the curl command is executed on another terminal: curl-I localhost,

Let's take a look at the strace output.

Each line output by strace records a system call. The system call name and call parameters are on the left of the equal sign, and the return value of the system call is on the right of the equal sign.

Through the above system calls, we can make the following analysis:

(1). epoll_wait returns 1, indicating that one descriptor has a read/write event. Here, of course, it is a read event.
(2). accept4 accepts this request. The number 3 Returned indicates the file descriptor of the socket.
(3). epoll_ctl adds the Socket socket established by accept4 (note parameter 3) to the event listening mechanism.
(4). Recv reads data from the socket file descriptor where a readable event occurs. The read data contains 79 bytes in the second parameter.
Pipeline. stat64 checks whether the HTML file requested by the client exists. If the returned value is 0, the file exists.
Opening. Open/fstat64 open and get the file status information. The file descriptor returned by the open file is 9, which is used in subsequent system calls.
Response. writev sends the response header to the client through the socket Socket socket represented by file descriptor 3.
Response. sendfile64 sends the response body represented by file descriptor 9 to the client through the socket Socket socket represented by file descriptor 3.
Into. Write a log to the log file represented by file descriptor 4.
Response. Recv check whether the client has sent other pending requests/information.
Finally, close the Socket socket represented by file descriptor 3.
Strace can provide the internal information during nginx execution, so there are some strange phenomena, such as nginx startup failure, inconsistent file data in the response and expected data, inexplicable segment fault segment error,

There is a performance bottleneck (using the-T option to track the time consumed by each function), and using strace may provide some help. Finally, to exit the strace trace, press Ctrl + C.

The strace command traces system calls and cannot provide more clear information about the function call relationship of nginx. If we find that nginx is not running properly, want to know what function is being executed inside nginx,

The command pstack is a very convenient and practical tool. Pstack is also very simple to use, followed by the process ID. For example, if there is no client request, nginx is blocked at the epoll_wait system call.

The following figure shows the relationship between the nginx function call stack and pstack:

The Calling relationship between the main () function and epoll_wait () function is clear at a glance, and the stack information shown in GDB is the same. We can use this for analysis and optimization.

In addition to method 1 and method 2, methods such as piling debugging are not described here. I will have the opportunity to introduce the debugging of special application logic in the future.

Okay. Thank you for reading.

References:

Http://code.google.com/p/nginxsrp/wiki/NginxCodeReview

<In-depth analysis of nginx>