Nginx Architecture Detailed

Http://tengine.taobao.org/book/chapter_09.html

Nginx Architecture Detailed (50%)

Nginx's next article will be more in-depth introduction to the implementation of nginx principle. In the previous chapter, we learned how to design a high-performance server, and that chapter will begin to explain how nginx is implementing a high-performance server step-by-step. Nginx Source directory structure (100%)

Nginx Excellent In addition to the program structure and code style, Nginx source organization is also concise and clear, the structure of the directory hierarchy is clear, it is worth learning. Nginx Source directory and Nginx's modular and functional division is closely integrated, which also makes it easy for us to find the relevant features of the code. This section first introduce Nginx source directory structure, first to Nginx source has a general understanding, the next section will explain how to compile nginx.

The following is the Nginx source directory structure:

. ├──auto automatic detection of the system environment and compilation-related scripts │├──cc detection scripts for compiler-related compilation options │├──lib some of the libraries required for Nginx compilation │├─ ─os some of the system parameters associated with the platform are related to system calls │└──types some auxiliary scripts related to data types ├──conf store the default profile, which is copied to the installation after make install              Directory to ├──contrib to store some utilities, such as the GEO Configuration Generation tool (geo2nginx.pl) ├──html to store the default Web page files, after make install, will be copied to the installation directory to ├──man Nginx Man Handbook └──src The source code of Nginx ├──core Nginx, including the definition of common data structure, and the core code of Nginx initialization operation such as Mai         The N function ├──event the encapsulation of the system event handling mechanism and the implementation code of the timer │└──modules different event handling modes such as SELECT, poll, Epoll, Kqueue, etc. ├──http        Nginx as HTTP server-related code │└──modules contains the various functions of HTTP ├──mail nginx as the Mail proxy server related code ├──MISC Some auxiliary code, test the compatibility of C + + header, and support for Google_perftools └──os is mainly the encapsulation of system functions provided by various system architectures, providing a unified system calling interface.

the Configure principle of Nginx (100%)

Nginx's compilation journey will begin with configure, and the Configure script will generate the required files (including source files and makefile files) based on the options we enter and the system environment involved. Configure invokes a series of auto scripts to implement the initialization of the compilation environment. Auto Script

The auto script is made up of a series of scripts, some of which are implemented with some common functionality called by another script (if they have one), and some are doing certain functions (such as option). The main execution sequence and invocation relationship between scripts are shown in the following figure (top to bottom, which indicates the execution of the main process):

Next, we combine the code to analyze the principle of the next configure: initialization

. Auto/options
. Auto/init
. auto/sources

This is the first three lines of the Configure source code, followed by option, INIT, sources under the Auto directory. The Auto/options Master is the Configure option that handles user input, as well as the output help information. Readers can be combined with Nginx source code to read the contents of this chapter. Because of the space relationship, the structure of this file is outlined here:

# #1. Set the shell variables corresponding to the options and their initial values Help=no ngx_prefix= ngx_sbin_path= ngx_conf_prefix= ngx_conf_path= ngx_error_log_path= NGX_PID _path= ngx_lock_path= ngx_user= ngx_group= ... # 2, process each option value and set to the corresponding global variable in the For option do opt= "$opt ' echo $option | SED-E \ "s/\ (--[^=]*=\) \ (. *. *\)/\1 ' \2 '/\ ' '" # Gets the value of this option case "$option" in-*=*) value= ' echo ' $option " |
            Sed-e ' s/[-_a-za-z0-9]*=//';;
    *) value= "";                   ESAC # matches according to the option content and sets the appropriate options case "$option" in--help) Help=yes
        ;;
        --prefix=) ngx_prefix= "!";;
        --prefix=*) ngx_prefix= "$value";;
        --sbin-path=*) ngx_sbin_path= "$value";;
        --conf-path=*) ngx_conf_path= "$value";;
        --error-log-path=*) ngx_error_log_path= "$value";;
        --pid-path=*) ngx_pid_path= "$value";; --lock-path=*) ngx_lock_path= "$value";;
        --user=*) ngx_user= "$value";;

        --group=*) ngx_group= "$value";;
        ... *) # No matching options found echo "$0:error:invalid option \" $option \ "" Exit 1
    ;; Esac Done # 3. Processing the option # If there is a--help, the output help message if [$HELP = yes];                      Then cat << END--help print this message--prefix=path Set installation prefix--sbin-path=path set nginx binary pathname--conf-pa
        Th=path set nginx.conf pathname--error-log-path=path set error log pathname --pid-path=path set Nginx.pid pathname--lock-path=path set Nginx.lock path
  Name--user=user set non-privileged user for worker processes      --group=group set non-privileged group for worker processes EN D Exit 1 Fi # Default file path ngx_conf_path=${ngx_conf_path:-conf/nginx.conf} ngx_conf_prefix= ' dirname $NGX _conf_path ' Ngx_pi D_path=${ngx_pid_path:-logs/nginx.pid} Ngx_lock_path=${ngx_lock_path:-logs/nginx.lock} ...

In the above code, we have selected some of the code in the file to illustrate. We can combine the source code and then analyze. The purpose of auto/options is primarily to handle user options, and to generate some global variable values from the options that are used in other files. The file will also output configure help information. Auto/init

The directory for this file is to initialize the path to some temporary files, check echo compatibility, and create makefile.

# Generate the MAKEFILE file path that will eventually execute the compilation ngx_makefile= $NGX _objs/makefile # Dynamically generate the path to the Nginx module list, because some of Nginx's modules can be compiled, and can add their own modules, So the module list is dynamically generated ngx_modules_c= $NGX _objs/ngx_modules.c ngx_auto_headers_h= $NGX _objs/ngx_auto_headers.h NGX_AUTO_ config_h= $NGX _objs/ngx_auto_config.h # Automatic test directory and log output file ngx_autotest= $NGX _objs/autotest # If configure error, use to find the cause of the error ngx_ autoconf_err= $NGX _objs/autoconf.err ngx_err= $NGX _objs/autoconf.err makefile= $NGX _objs/makefile NGX_PCH= NGX_USE_ Pch= # Check if Echo supports-N or \c # Check the Echo ' s "-n" option and "\c" capability if echo "test\c" | grep c >/dev/null; Then # does not support the-C method, check if the-n method is supported if Echo-n test | grep n >/dev/null; Then ngx_n= ngx_c= Else ngx_n=-n ngx_c= fi else ngx_n= ngx_c= ' \c ' fi # Create the most initial makefile file # Default indicates that the current compilation object # Clean indicates that the makefile file and the OBJS directory will need to be deleted when the clean work is performed, and only makefile files and OBJS directories are generated during the entire process.

All other temporary files are under the Objs directory, so after clean, the entire directory is restored to its original state # to perform the compilation again, you need to re-execute the Configure command # Create Makefile cat << END > Makefile DEfault:build clean:rm-rf Makefile $NGX _objs END 

Auto/sources

It can be seen from the file name that its main function is related to the source file. Its main role is to define variables for different functions or files required by the system. According to the function, divided into core/regex/event/unix/freebsd/http and so on. Each function will be composed of four variables, "_modules" means that the module related to the function, will eventually output to the ngx_modules.c file, that is, dynamically generate the module that needs to be compiled into Nginx, "INCS" means that this feature depends on the source directory, to find the header file will be used, In the Compile option, it appears in "-I", "DEPS" shows the name of the file that is required to be relied on in makefile, that is, when compiling, you need to check the update time of these files; "SRCs" means that the required source files need to be compiled for this feature.

Take the core:

core_modules= "Ngx_core_module ngx_errlog_module ngx_conf_module ngx_emp_server_module ngx_emp_server_core_module"
         core_incs= "Src/core" core_deps= "src/core/nginx.h \ src/core/ngx_config.h \ src/core/ngx_core.h \
         src/core/ngx_log.h \ src/core/ngx_palloc.h \ src/core/ngx_array.h \ src/core/ngx_list.h \ SRC/CORE/NGX_HASH.H \ src/core/ngx_buf.h \ src/core/ngx_queue.h \ Src/core/ngx_string.h \ src/core/ngx_parse.h \ src/core/ngx_inet.h \ src/core/ngx_file.h \ SRC/CORE/NGX_CRC. h \ src/core/ngx_crc32.h \ src/core/ngx_murmurhash.h \ src/core/ngx_md5.h \ src/core/n
         Gx_sha1.h \ src/core/ngx_rbtree.h \ src/core/ngx_radix_tree.h \ src/core/ngx_slab.h \ src/core/ngx_times.h \ src/core/ngx_shmtx.h \ src/core/ngx_connection.h \ src/core/ngx_cycle.h \ Src/core/ngx_conf_file.h \ src/core/ngx_resolver.h \ src/core/ngx_open_file_cache.h \ src/core/nginx_emp_server.h \ Src/core/emp_server.h \ src/core/task_thread.h \ src/core/standard.h \ Src/core/dprin t.h \ Src/core/ngx_crypt.h "core_srcs=" src/core/nginx.c \ src/core/ngx_log.c \ SRC/CORE/NGX_PA LLOC.C \ src/core/ngx_array.c \ src/core/ngx_list.c \ src/core/ngx_hash.c \ src/core/n
         GX_BUF.C \ src/core/ngx_queue.c \ src/core/ngx_output_chain.c \ src/core/ngx_string.c \
         SRC/CORE/NGX_PARSE.C \ src/core/ngx_inet.c \ src/core/ngx_file.c \ src/core/ngx_crc32.c \ SRC/CORE/NGX_MURMURHASH.C \ src/core/ngx_md5.c \ src/core/ngx_rbtree.c \ Src/core/ngx_rad IX_TREE.C \ src/core/ngx_slab.c \ src/core/ngx_times.c \ src/core/ngx_shmtx.c \ Src/co
  RE/NGX_CONNECTION.C \       SRC/CORE/NGX_CYCLE.C \ src/core/ngx_spinlock.c \ src/core/ngx_cpuinfo.c \ Src/core/ngx_c ONF_FILE.C \ src/core/ngx_resolver.c \ src/core/ngx_open_file_cache.c \ Src/core/nginx_emp_serv ER.C \ src/core/emp_server.c \ src/core/standard.c \ src/core/task_thread.c \ Src/core
 /DPRINT.C \ src/core/ngx_crypt.c "

If we write a third-party module ourselves, we may refer to the values of these variables, or modify them, such as adding our own modules, or adding our own header file to find the directory (in the config of the third-party module), and later we will see how it is framed by third-party modules. Before proceeding with the analysis of the execution process, let's introduce some tool scripts. Auto/have

Cat << End >> $NGX _auto_config_h

#ifndef $have
#define $have  1
#endif

End

From the code, we can see that the function of this tool is to $have the value of the variable, the macro is defined as 1, and output to the Auto_config file. Usually we use this tool to control whether an attribute is turned on. Before using this tool, you need to define the name of the macro, which is the $have variable. Then go back to the Configure file:

# Ngx_debug is processed in the Auto/options file, if there is a--with-debug option, the value is Yes
if [$NGX _DEBUG = YES]; then
    # when there is a DEBUG option, the ngx_ is defined DEBUG macro
    have=ngx_debug. Auto/have
fi

In this code, you can see how configure defines an attribute: the macro definition, output to the config header file, and then in the program can determine whether the macro is defined, to achieve different characteristics.

Continue down in the Configure file:

# compiler option
. auto/cc/conf

# header file supports macros defined
if ["$NGX _platform"! = Win32]; then
    . auto/headers
fi

# operating system phase Check the configuration of the
auto/os/conf

# UNIX System for common configuration detection
if ["$NGX _platform"! = Win32]; then
    . Auto/unix
fi

Configure will invoke several other files in turn to detect the environment, including compilers, operating system-related. Auto/feature

Nginx Configure will automatically detect the characteristics of different platforms, the magic is the implementation of Auto/feature, before continuing down the analysis, we first look at the implementation of this tool principle. The core idea of this tool is to output a small section of Representative C programs, then set up the compilation options, then compile the connection to run, and then analyze the results. For example, if you want to detect the existence of a library, in the small section of the C program to call a function inside the library, and then compile the link, if the error, the library environment is not normal, if the compilation is successful, and run normally, the library environment detection is normal. When we write nginx third-party modules, we often use this tool to detect the environment, so this tool works throughout the configure process.

Let's look at a short example of use:

Ngx_feature= "poll ()"
ngx_feature_name=
ngx_feature_run=no
ngx_feature_incs= "#include <poll.h> "
ngx_feature_path=
ngx_feature_libs=
ngx_feature_test=" int  n; struct pollfd  pl;
                  PL.FD = 0;
                  pl.events = 0;
                  pl.revents = 0;
                  n = Poll (&PL, 1, 0);
                  if (n = =-1) return 1 "
. auto/feature if

[$ngx _found = no];
    then # Sets the value of the variable if POLL is not found
    Event_poll=non E
fi

This code is implemented in Auto/unix to detect whether the current operating system supports poll function calls. Before calling Auto/feature, you need to set the value of several input parameter variables, then the result will be inside the $ngx_found variable, and output the macro definition to indicate support for this feature:

$ngx _feature      attribute name
$NGX the macro definition name of the _feature_name attribute, if the attribute test succeeds, the macro definition is defined
$ngx _feature_path to find the header file directory at compile time
Ngx_feature_test the test code to execute
$ngx _feature_incs the header file that you want to include in your code
$NGX _feature_libs The library file option that requires link when compiling
$ngx _ Feature_run  After the successful compilation, the binary files need to do the action, can be Yes value bug other

#ngx_found if found, and test success, its value is yes, otherwise its value is no

Look at the key code for Ngx_feature:

# Initialize the output to no ngx_found=no #将特性名称小写转换成大写 if test-n "$ngx _feature_name"; Then # lowercase to uppercase ngx_have_feature= ' echo $ngx _feature_name \ | TR abcdefghijklmnopqrstuvwxyz abcdefghijklmnopqrstuvwxyz ' Fi # converts all include directories to compile options if Test-n "$ngx _feature_path"; Then for ngx_temp in $ngx _feature_path;
Do ngx_feature_inc_path= ' $ngx _feature_inc_path-i $ngx _temp "done fi # generates temporary small segment C program code. # $NGX _feature_incs variable is the header file that the program needs to include # $NGX _feature_test is the test code cat << END > $NGX _autotest.c #include <sys/ty
    pes.h> $NGX _include_unistd_h $ngx _feature_incs int main () {$NGX _feature_test;
return 0; } END # Compile command # After compiling the target file is $NGX _autotest, will determine whether the file exists to determine whether the compilation succeeded ngx_test= "$CC $CC _test_flags $CC _aux_flags $ngx _feature_i Nc_path \ o $NGX _autotest $NGX _autotest.c $NGX _test_ld_opt $ngx _feature_libs "# executes the compilation process # after the compilation succeeds, the $ngx_autotest named is generated. File eval "/bin/sh-c \" $ngx _test\ ">> $NGX _autoconf_err 2>&1" # If the file exists, the compilation succeeds if [-X $NGX _autotest]; ThEn case "The $ngx _feature_run" in # needs to be run to determine if the feature # test program can execute normally (i.e. if the status code after the program exits is 0), and if it exits normally, the feature test succeeds, setting Ngx_foun
            D is yes and adds a macro definition named Ngx_feature_name, with a macro value of 1 Yes) # # If the program exits normally and the exit code is 0, the program executes successfully and we can manually return non-zero in the test code to indicate a program error #/bin/sh is used to intercept "killed" or "Abort trap" messages if/bin/sh-c $NGX _autotest >> $NGX _ Autoconf_err 2>&1;  Then echo "Found" Ngx_found=yes # Add macro definition, the value of the macro is 1 if test-n "$ngx _feature_name"; Then have= $ngx _have_feature. 

        Auto/have fi else echo "found but was not working" FI; # need to run the program to determine whether the feature is supported, if supported, the result of the program standard output as the value of the macro) #/bin/sh is used to intercept "killed" or "Abort Trap "Messages If/bin/sh-c $NGX _autotest >> $NGX _autoconf_err 2>&1; Then echo "Found" Ngx_found=yes # is not the same as YesYes, value will be set to the value that the program prints from the standard output to the NGX_FEATURE_NAME macro variable # In this case, the program needs to set the Ngx_feature_name variable name cat <<
            End >> $NGX _auto_config_h #ifndef $ngx _feature_name #define $NGX _feature_name ' $NGX _autotest ' #endif END

        else echo "found but was not working" FI;; # is the opposite of Yes the bug) #/bin/sh is used to intercept "killed" or "Abort trap" messages IF/BIN/S H-c $NGX _autotest >> $NGX _autoconf_err 2>&1;

                Then echo "Not found" else echo "found" Ngx_found=yes If Test-n "$ngx _feature_name"; Then have= $ngx _have_feature.

        Auto/have fi fi;; # no need to run the program, and finally define the macro variable *) echo "found" Ngx_found=yes if Test-n "$ngx _feature_nam E "; Then have= $ngx _have_feature.
     Auto/have fi   ;;
    Esac Else # failed to compile echo "not Found" # compilation failed, will save information to log file echo "----------" >> $NGX _autoconf_err  # Keep the contents of the compiled file cat $NGX _autotest.c >> $NGX _autoconf_err echo "----------" >> $NGX _autoconf_err # Options for preserving compiled files echo $ngx _test >> $NGX _autoconf_err echo "----------" >> $NGX _autoconf_err fi # most After you delete the generated temporary file RM $NGX _autotest*

Auto/cc/conf

After understanding the tool auto/feature, continue our mainstream, auto/cc/conf code is very good understanding, this step is mainly to detect the compiler, and set compiler-related options. It calls Auto/cc/name to get the name of the compiler, and then sets some compiler-related options based on the compiler's choice of executing different compiler-related files such as gcc execution AUTO/CC/GCC. Auto/include

This tool is used to detect if a header file is supported. The header file that needs to be detected is placed inside the $ngx_include, and if supported, the value of the $ngx_found variable is yes and the macro definition of Ngx_have_{ngx_include} is generated. Auto/headers

Generates a macro definition for the header file. The resulting definition is placed inside the objs/ngx_auto_headers.h: