Linux Server performance optimization

1. The throughput rate here refers specifically to requests processed within the Web server unit time. 2. Premise of stress test:1> number of concurrent users 2> total requests 3> Request Resource Description 3. Average user request Wait time the primary user measures the server quality for a single user in the case of a certain number of concurrent users, while the average server request processing time is compared to the former.  is used to measure the overall service quality of the server, which is actually the reciprocal of the throughput rate. 4. Open the long connection support for the Web server for requests marked connection:keep-alive in the HTTP header.      Reduce the number of times the system calls accept, which reduces the overhead of establishing a connection. 5. Processes, kernel-level threads and user-level threads in different situations. IO model, mmap (NEI-cun mapping), direct IO, such as Sendfile syscall and asynchronous IO. Multi-Channel IO multiplexing (SELECT, Poll,epoll and Kqueue etc) 6. Server concurrency policy 1> a process handles one connection, non-blocking IO.  Stability is strong, but the overhead of the context switch grows rapidly with HTTP request increments. 2> a kernel-level thread that handles a single connection, non-blocking IO, multi-process multithreaded hybrid approach. The problem with Context switch still exists.        In theory, more concurrent connections can be supported. 3> a process handles multiple connections, non-blocking IO. (Epoll, Kqueue) lighttpd, Nginx. Strong concurrency performance support. Throughput RateThe throughput rate here refers specifically to requests processed within the Web server unit time. Throughput rate: A metric that measures the performance of Web services, characterizing the number of requests processed per second. The indicator is affected by 3 factors: the number of concurrent users, the total number of requests, and the type of resource requested. Sometimes, the higher the number of concurrent users, the higher the throughput rate, and the more time it takes to request a few KB of files and a few m of files, and the final processing times are obviously different. Therefore, the throughput rate is a relatively comprehensive indicator and does not refer to the concurrency capability. CPU Concurrency CalculationMulti-process, multi-threaded selection and scheduling: both process switching and thread switching require a certain amount of overhead, and Web server software that typically uses multithreaded models has better performance than multi-process use. The essence of a process being suspended is to take its data out of the CPU register in the kernel stack, and the essence of a process recovery is to reload its data into the CPU register, which is called the hardware context, and the data that is loaded and moved out is what the server expects to support a large number of concurrent numbers. To minimize the number of context switches, the simplest approach is to reduce process counts, use threads as much as possible, and design concurrency policies with other I/O models. system CallsSystem calls: Some function calls that need to switch from user mode to kernel mode can be called system calls, such as opening files. System calls can be somewhat overhead, and reducing system calls is a programming detail that can speed up processing. The process usually runs in the user state, can use CPU and memory to accomplish some tasks, and when the process needs to operate on the hardware peripherals (read disk files, send network data), it must switch to the kernel state because the system calls the design process from the user state to the kernel state switch, resulting in a certain amount of memory space exchange, This is also a certain level of context switching, so the overhead of system calls is often considered more expensive Memory allocation

Nginx multi-threaded to handle requests, multiple threads can share memory resources, a phased memory allocation policy, on demand, timely release, so that the overall memory usage in a small number of ranges, 10,000 inactive HTTP persistent connection only requires 2.5MB of memory

Persistent ConnectionsTCP link hold: You can reduce the creation and shutdown of TCP links between the server and the client by maintaining a TCP link. The connection:keep-alive in HTTP has this function I/O modelIO Model: Because the CPU is much faster than IO, io latency is often a performance bottleneck, so the IO model is important.

Synchronous blocking I/O: for the process, some system calls in order to synchronize IO, the process will be blocked to varying degrees, such as accept, send, read and so on.

synchronous non-blocking I/O:For a process, some system calls can be returned immediately after the call, informing the process that IO is ready to avoid blocking the process. Multi-Channel I/O readiness Notification:For the way that non-blocking I/O is synchronized, the process still needs to poll the file descriptor (handle) to know which IO is ready, and the multipath I/O readiness notification changes the process to a callback notification. In particular, the Web server, while processing a large number of file descriptors is essential, but the use of synchronous non-blocking I/O is obviously not the best choice, in this model, we know that if the server wants to receive data from multiple TCP connections at the same time, it must take turns to receive data for each socket call method, such as recv (). Regardless of whether these sockets have data to be received, ask again, if most sockets do not have the data to receive, then the process will waste a lot of CPU time to check these sockets, which is obviously not what we want to see. The advent of multiple I/O readiness Notifications provides a high-performance scenario for a large number of file descriptor readiness checks, which allows a process to monitor all file descriptors at the same time, and to quickly obtain all the ready file descriptors, and then access the data only for those file descriptors. The epoll can support both horizontal and edge triggering, theoretically with a higher performance of edge triggering, but the code implementation is quite complex because any accidental loss of events can cause request processing errors. By default, Epoll uses a horizontal trigger, and if you want to use edge triggering, you need to add the Epollet option when registering the event. Instead of a value that represents the number of ready descriptors, you only need to go to the Epoll specified array to get the appropriate number of file descriptors, and memory mapping (MMAP) technology is used, which eliminates the overhead of copying these file descriptors on system calls. The Linux kernel provides a special way to access disk files, which can associate an address space in memory with a disk file that we want to specify, thus transforming our access to this memory into Access to disk files, a technique known as memory Mapping (memorymapping). In most cases, using memory maps can improve the performance of disk I/O, it does not need to use a system call such as read () or write () to access files, but rather to establish the association of Memory and disk files through MMAP () system calls, and then access the files as freely as the memory accesses. Memory Mapping:The file is matched to an address space in memory so that you can write the file as if it were in memory. Of course, this approach is essentially no different from writing files. Direct I/o:In the user process address space and in the middle of the disk usually there is a kernel buffer that the operating system governs, when writing to the file, it is generally written to this buffer, and then by some delay policy to write to disk. Doing so can improve write efficiency. But for applications such as databases, it is often desirable to manage read and write caches to avoid the fearless memory waste of kernel buffers. The Linux open function supports the O_direct parameter for direct IO. Sendfile:If the Web server wants to send a file, it will go through the process of opening the file, reading the contents of the file from the disk (this usually involves copying the kernel buffer data to the user process), and then sending the file content through the socket (which is usually designed to replicate the user process data to the NIC kernel buffer). You can see that duplicate data replication is avoidable. The sendfile can support copying directly from the file kernel buffer to the NIC kernel buffer. Server concurrency PolicyPHP script, the worker process is usually just responsible for forwarding the request to the standalone FASTCGI process or forwarding the request to the back-end server as a reverse proxy server, when the worker process does not depend on too many local resources, so in order to increase the number of concurrent connections, the number of worker processes can be increased appropriately. In general, however, the throughput rate of the dynamic content itself is quite limited, and because of the overhead of the scripting interpreter, the throughput rate of the 2000req/s is usually quite high, so the worker process is not very stressful. However, if multiple back-end servers expand dynamic content computing capabilities as a reverse proxy-based load balancer, the number of woker processes will gradually become a bottleneck for overall performance. Of course, too many worker processes lead to more context switching overhead and memory overhead, resulting in longer response times for all connections as a whole. The scope of the above situation cannot be fits, and this refers to single-machine concurrency, which needs to be selected according to the actual number of concurrency. In general, what concurrency policy is used by Web servers in the case of a large number of concurrent users is the key to affecting the maximum number of concurrency.

Linux Server performance optimization