Long response times possible causes:
Hardware, application server, network, load machine, middleware thread pool queuing, database connection pool queuing, SQL statements, JVMGC, code logic
How to analyze:
Analyzing System Architecture Nodes
Developing the time that the Print method in the log consumes when it is called
Web container: Nginx Apache Tomcat resin WebLogic WebSphere
Apache: Handling static resource performance is better, performance of dynamic requests such as servlet processing will be worse
Tomcat: Handling dynamic request performance is better
Apache:
1. Channel configuration file httpd.conf
1) Location:
Yum Installation:/etc/httpd/conf/
Compile and install:/opt/lampp/etc/
Etc/extra directory has multiple conf files, in the channel configuration file, if you want to use the Conf file in extra, you need to include the extra file in the channel configuration file in the corresponding conf, or the contents of the configuration file that will be used to copy into the channel configuration file.
2) channel configuration file related configuration explanation:
ServerRoot boot path for "/etc/httpd" →apache
Pidfile Run/httpd.pid→apache will generate a PID when it starts.
Timeout 60→ receive and send time-out is 60s
KeepAlive off→ Long Connection (build handshake, channel always in), OFF here to identify the short connection, long connection performance is good, but the disadvantage is also obvious, the thread and threads pool between the existence of a connection thread is not used when the connection is still there, then other threads, Unable to connect to the thread pool, causing the system to be unresponsive.
The use keepalive long connection is used by default in JMeter, and short connections are used by default in LR, and the effect of TPS on performance testing may vary greatly. The TPS that is tested with long connections can be much larger.
Maxkeepaliverequests 100→ Maximum number of long connections used
KeepAliveTimeout 15→ Long Connection maximum wait time, if this time exceeds the configured time, this connection does not take effect
# prefork mpm→ Process working mode. The multi-Path processing module (MPM) implements a non-threaded, pre-derived Web server that is suitable for systems that do not have a thread-safe library and need to avoid thread-compatibility issues. It is the best mpm to require each request to be independent of each other, so that if a request has a problem it will not affect the other request.
Startservers 8→ Initialize the number of boot processes
Minspareservers 5→ Minimum number of server idle processes (pre-derived)
Maxspareservers 20→ Maximum number of server idle processes
Serverlimit 256→apache simultaneous processing of service processes
MaxClients 256→apache can process the number of requests at the same time, its default value of 150 is not enough, if the total number of requests reached this value (can be confirmed by Ps-ef|grep Http|wc-l), then the subsequent request will be queued until a processed request is complete. This is the main reason why the system resources are still a lot left and HTTP access is slow. The default limit for Apache cannot be greater than 256.
Maxrequestsperchild 4000→ a process that processes up to 4,000 requests during its lifetime, if the process is destroyed after processing 4,000 requests, avoids the zombie process, prevents accidental memory leaks, and automatically reduces the number of processes when the server load drops
# worker mpm→ Mixed multi-process multithreaded working mode. Multiple processes are used, and each process has multiple threads to obtain the stability of the process-based MPM. The number of threads that each process can have is fixed. The server increases or decreases the number of processes depending on the load.
Startservers 4→ Initialize the number of boot processes
MaxClients 300→ Maximum number of processes started
Minsparethreads 25→ Minimum number of idle threads
Maxsparethreads 75→ Maximum number of idle threads
Threadsperchild 25→ 25 threads per process
Maxrequestsperchild 0→ process is always alive
Listen 80→ Monitor Port number
loadmodule→ loading the dynamic library. So identity Dynamic Library
Include conf.d/*.conf→ to load the configuration files in the conf.d/*.conf
DocumentRoot "/var/www/html" →apache working path, storing application code
Errorlog logs/error_log→ Error log path
LogLevel warn→ Log level, general configuration info level
Adddefaultcharset utf-8→ Default Character Set
# Some Examples:
#ErrorDocument "The server made a boo boo."
#ErrorDocument 404/missing.html
#ErrorDocument 404 "/cgi-bin/missing_handler.pl"
#ErrorDocument 402 http://www.example.com/subscription_info.html
Configuring monitoring:
<Location/server-status>
SetHandler Server-status
Order Deny,allow
Deny from all
# Allow from. example.com
</Location>
<Location/server-info>
SetHandler Server-info
Order Deny,allow
Deny from all
# Allow from. example.com
</Location>
After modifying the channel configuration file, restart Apache:
Service httpd stop/service httpd start
2, Apache monitoring
1) Monitor Server-status
In the channel configuration file httpd.conf, configure the Allow from. example.com for monitoring <Location/server-status> to be commented out, Access http://192.168.20.129/ Server-status, the page tip does not allow access, as follows:
Configure the allow for monitoring <Location/server-status> from all to access http://192.168.20.129/server-status, as shown in page access:
Compare the contents of the configuration file page to explain:
It is convenient for us to monitor the status of the process and then analyze the system performance.
2) Monitor Server-info
In the pass-through configuration file, configure the allow for listening <Location/server-info> to access Http://192.168.20.129/server-info, as shown in page access:
3. Switch Apache to working mode of the thread
By default, Apache is the working mode of the process, so how do I switch Apache to thread working mode?
1) Enter the/usr/sbin directory
[Email protected] sbin]# cd/usr/sbin/
[[email protected] sbin]# ls http*
httpd httpd.event Httpd.worker
2) Rename the httpd file
[Email protected] sbin]# MV httpd httpd_prefork
[Email protected] sbin]# MV Httpd.worker httpd
[[email protected] sbin]# httpd-l→ view Apache's working mode
Compiled in Modules:
Core.c
worker.c→ Modified, the working mode becomes WORKER.C
Http_core.c
Mod_so.c
3) Restart Apache
[[Email protected] sbin]# service httpd stop
Stop httpd: [OK]
[[Email protected] sbin]# service httpd start
Starting httpd: [OK]
4) Visit the Http://192.168.20.129/server-status page again
The configuration in the channel configuration file is as follows:
# worker MPM
# startservers:initial number of server processes to start
# maxclients:maximum number of simultaneous client connections
# minsparethreads:minimum number of worker threads which is kept spare
# maxsparethreads:maximum number of worker threads which is kept spare
# threadsperchild:constant number of worker threads in each server process
# maxrequestsperchild:maximum number of requests a server process serves
<ifmodule worker.c>
Startservers 4
MaxClients 300
Minsparethreads 25
Maxsparethreads 75
Threadsperchild 25
Maxrequestsperchild 0
</IfModule>
First visit page:
1 requests currently being processed, workers
Second third time ... Nth Visit page:
1 requests currently being processed, $ idle workers
Depending on the configuration in the pass-through configuration file, Apache starts with 4 processes and 25 threads in each process, starting with a total of 100 threads and the configuration of Minsparethreads and Maxsparethreads,apache Kill 1 Processes, then in the page display, 1 busy threads and 74 idle threads are displayed.
After the nth access page, the final stabilization is 50 threads.
5) page interpretation, similar process work mode
Apache Server Status for 192.168.20.129
1 requests currently being processed, $ idle workers
_____________w___________ ....... .......................
................................................................
_________________________.......................................
................................................................
................................................................
................................................................
................................................................
................................................................
................................................................
................................................................
................................................................
................................................................
................................................................
................................................................
................................................................
................................................................
Scoreboard Key:
"_" Waiting for Connection, "S" starting Up, "R" Reading Request,
"W" Sending Reply, "K" Keepalive (Read), "D" DNS Lookup,
"C" Closing connection, "L" Logging, "G" Gracefully finishing,
"I" Idle cleanup of worker, "." Open slot with no current process
PID Key:
2555 in state: _, 2555 in state: _, 2555 in state: _
2555 in state: _, 2555 in state: _, 2555 in state: _
2555 in state: _, 2555 in state: _, 2555 in state: _
2555 in state: _, 2555 in state: _, 2555 in state: _
2555 in state: _, 2555 in State:w, 2555 in state: _
2555 in state: _, 2555 in state: _, 2555 in state: _
2555 in state: _, 2555 in state: _, 2555 in state: _
2555 in state: _, 2555 in state: _, 2555 in state: _
2555 in state: _, 2445 in state: _, 2445 in state: _
2445 in state: _, 2445 in state: _, 2445 in state: _
2445 in state: _, 2445 in state: _, 2445 in state: _
2445 in state: _, 2445 in state: _, 2445 in state: _
2445 in state: _, 2445 in state: _, 2445 in state: _
2445 in state: _, 2445 in state: _, 2445 in state: _
2445 in state: _, 2445 in state: _, 2445 in state: _
2445 in state: _, 2445 in state: _, 2445 in state: _
2445 in state: _, 2445 in state: _,
6) The meaning of the process and thread monitoring page
Example:
max:75 Threads
70 or so threads are working (send, receive, write log, parse, disconnect, reconnect)
tps:3000 distributed load 1 seconds to send 200 requests wait1
Web container: 75 Threads Wait2
16 CPUs, processing 16 requests at a time wait3
Slow response time, see time-consuming places, see if thread pools are queued (thread pool working state, mostly working)
Fewer requests are eventually sent to the database
The better the load machine, the better the performance may be
You need to detect if a thread pool request is queued, and if it exists, you need to increase the thread pool
See where the thread pool status takes time
4. Apache Log
1) Location
[[email protected] httpd]# ls
Conf CONF.D logs modules run
[Email protected] httpd]# pwd
/etc/httpd
[Email protected] httpd]# CD logs/
[[email protected] logs]# ls
Access_log access_log-20170702 error_log-20170618 error_log-20170702
access_log-20170618 Error_log error_log-20170626
2) Log
error_log→ error Log
access_log→ Growth Log
You can see the features on the line. Access heat, heat: functional points for performance testing
Maximum number of concurrent functions per second, key features, user-focused features
5. Apache Features
Like Nginx, you can do both Web container and load Balancer server
Apache+tomcat do load balancing, front-end Apache back-end Tomcat
Web container: Apache Configuration Monitoring