Linux Server Performance View Analytics tuning

Source: Internet
Author: User
Tags gpg gpg encryption ibm db2 syslog system log cpu usage

A Linux server Performance View 1.1 CPU Performance View 1, view the number of physical CPUs:
cat /proc/cpuinfo |grep "physical id"|sort|uniq|wc -l
2. View the number of cores in each physical CPU:
cat /proc/cpuinfo |grep "cpu cores"|wc -l
3, the number of logical CPUs:
cat /proc/cpuinfo |grep "processor"|wc -l

Number of physical CPUs * Number of cores = number of logical CPUs (in case of Hyper-Threading technology not supported)

1.2 Memory View 1, view memory usage:
#free -m             total       used       free     shared    buffers     cachedMem:          3949       2519       1430          0        189       1619-/+ buffers/cache:        710       3239Swap:         3576          0       3576total:内存总数used:已经使用的内存数free:空闲内存数shared:多个进程共享的内存总额- buffers/cache:(已用)的内存数,即used-buffers-cached+ buffers/cache:(可用)的内存数,即free+buffers+cachedBuffer Cache用于针对磁盘块的读写;Page Cache用于针对文件inode的读写,这些Cache能有效地缩短I/O系统调用的时间。对操作系统来说free/used是系统可用/占用的内存;对应用程序来说-/+ buffers/cache是可用/占用内存,因为buffers/cache很快就会被使用。

We should look at the application from the perspective of our work.

1.3 Hard Drive View 1, view hard disk and partition information:
fdisk -l
2. Check the disk space usage of the file system:
df -h
3. Check the I/O performance of the hard disk (show once every second, show 5 times):
iostat -x 1 5

The Iostat is included in the package Systat and can be installed with the Yum-y install Systat.

Parameters that are commonly followed:

4. Check the size of a directory on your Linux system:
du -sh /root

If you find that a partition is nearly exhausted, you can go to the mount point of the partition, use the following command to find the file or directory that occupies the most space , and then find the top 10 files or directories in the system that occupy the most space in the order of large to small:

du -cksh *|sort -rn|head -n 10
1.4 Viewing Average load

Sometimes the system responds slowly, but cannot find the reason, then it is necessary to look at the average load, see if it has a large number of processes in the queue waiting.

The simplest of commands:


and a dynamic command top.
We only care about the following sections:

top - 21:33:09 up  1:00,  1 user,  load average: 0.00, 0.01, 0.05如果每个逻辑cpu当前的活动进程不大于3,则系统性能良好;如果每个逻辑cpu当前的活动进程不大于4,表示可以接受;如果每个逻辑cpu当前的活动进程大于5,则系统性能问题严重。

General calculation method: Load value/number of logical CPUs

You can also combine the Vmstat command to determine whether the system is busy, where:

1.5 Other parameters
查看内核版本号:uname -a简化命令:uname -r查看系统是32位还是64位的:file /sbin/init查看发行版:cat /etc/issue或lsb_release -a查看系统已载入的相关模块:lsmod查看pci设置:lspci
Two Linux server performance evaluation 2.1.1 Factors that affect Linux server performance 1. Operating system level
2. Program application level 2.1.2 System performance evaluation standard
Impact Performance Factors Good Bad Bad
Cpu user% + sys%< 70% user% + sys%= 85% user% + sys% >=90%
Memory Swap in (SI) =0 swap out (SO) =0 Per CPU with ten page/s More swaps in & swaps out
Disk Iowait% < 20% Iowait% =35% Iowait% >= 50%


%user:表示CPU处在用户模式下的时间百分比。%sys:表示CPU处在系统模式下的时间百分比。%iowait:表示CPU等待输入输出完成时间的百分比。swap in:即si,表示虚拟内存的页导入,即从SWAP DISK交换到RAMswap out:即so,表示虚拟内存的页导出,即从RAM交换到SWAP DISK
2.1.3 System performance analysis tools 1. Common system commands

Vmstat, SAR, Iostat, netstat, free, PS, top, etc.

2. Common Combination method
2.1.4 Linux performance Evaluation and optimization

Overall system performance evaluation (uptime command)

16:38:00 up 118 days, 3:01, 5 users,load average:1.22, 1.02, 0.91


    • Load average three-value size is generally not larger than the number of system CPUs.

      The system has 8 CPUs, such as load average three long-term greater than 8, indicating that the CPU is busy, the load is high, may affect system performance.

    • But occasionally greater than 8, generally does not affect system performance.

    • If the load average output value is less than the number of CPUs, it means that the CPU has idle time slices, such as the output in this example, the CPU is very idle

2.2.1 CPU Performance Evaluation 1. Monitor the system CPU with the Vmstat command

Display the system's various resources related performance brief information, mainly look at the CPU load situation.

The following is the output of the Vmstat command on a system:

[[email protected] ~]#vmstat 2 3procs ———–memory———- —swap– —–io—- –system– —–cpu——r  b swpd freebuff  cache si so bi bo incs us sy idwa st0  0 0 162240 8304 67032 0 0 13 21 1007 23 0 1 98 0 00  0 0 162240 8304 67032 0 0 1 0 1010 20 0 1 100 0 00  0 0 162240 8304 67032 0 0 1 1 1009 18 0 1 99 0 0Procs

r--the number of processes running and waiting for CPU time slices, this value if longer than the number of system CPUs, indicating that the CPU is not enough, need to increase the CPU

b--the number of processes waiting on the resource, such as waiting for I/O, or memory exchange.


Percentage of CPU time consumed by the user process.
When the value of us is higher, the user process consumes more CPU time, but if the long-term is greater than 50%, the optimizer or algorithm needs to be considered.


Percentage of CPU time consumed by the kernel process. When the value of SY is high, it indicates that the kernel consumes a lot of CPU resources.

Based on experience, the reference value for Us+sy is 80%, and if Us+sy is greater than 80%, there may be insufficient CPU resources.

2. Monitor system CPU with SAR command

The SAR makes separate statistics on each aspect of the system, but increases the overhead, but the overhead can be evaluated without much impact on the system's statistical results.

The following is the CPU statistics output of a system by the SAR command:

[[email protected] ~]# sar -u 3 5Linux 2.6.9-42.ELsmp (webserver) 11/28/2008_i686_ (8 CPU)11:41:24 AM CPU %user %nice%system %iowait %steal %idle11:41:27 AM all 0.88 0.00 0.29 0.00 0.00 98.8311:41:30 AM all 0.13 0.00 0.17 0.21 0.00 99.5011:41:33 AM all 0.04 0.00 0.04 0.00 0.00 99.9211:41:36 AM all 90.08 0.00 0.13 0.16 0.00 9.6311:41:39 AM all 0.38 0.00 0.17 0.04 0.00 99.41Average: all 0.34 0.00 0.16 0.05 0.00 99.45

The output is interpreted as follows:

%user列显示了用户进程消耗的CPU 时间百分比。%nice列显示了运行正常进程所消耗的CPU 时间百分比。%system列显示了系统进程消耗的CPU时间百分比。%iowait列显示了IO等待所占用的CPU时间百分比%steal列显示了在内存相对紧张的环境下pagein强制对不同的页面进行的steal操作 。%idle列显示了CPU处在空闲状态的时间百分比。问题

Have you ever experienced a low overall CPU utilization of the system and slow application?

In a multi-CPU system, if the program uses a single thread, there will be a phenomenon, the overall CPU utilization is not high, but the system application is slow response, this may be because the program uses a single thread, the single thread uses only one CPU, resulting in a CPU utilization of 100%, unable to process other requests , while the rest of the CPU is idle, which results in a low overall CPU usage, and the application of slow phenomenon occurs.

2.3.1 Memory Performance Evaluation 1. Use the free command to monitor memory

Free is the most commonly used instruction for monitoring Linux memory usage, as shown in the following output:

[[email protected] ~]# free -mtotal used freeshared buffers cachedMem: 8111 7185 926 0 243 6299 -/+ buffers/cache: 643 7468Swap: 8189 0 8189

Empirical formula:

2. Monitor memory with the Vmstat command
[[email protected] ~]# vmstat 2 3procs ———–memory———- —swap– —–io—- –system– —–cpu——r b swpd freebuff cache si so bi bo incs us sy idwa st0 0 0 162240 8304 67032 0 0 13 21 1007 23 0 1 98 0 00 0 0 162240 8304 67032 0 0 1 0 1010 20 0 1 100 0 00 0 0 162240 8304 67032 0 0 1 1 1009 18 0 1 99 0 0


swpd--切换到内存交换区的内存数量(k为单位)。如swpd值偶尔非0,不影响系统性能free--当前空闲的物理内存数量(k为单位)buff--buffers cache的内存数量,一般对块设备的读写才需要缓冲cache--page cached的内存数量

Generally as a file system cached, frequently accessed files will be cached, such as the cache value is large, indicating that the number of cached files, if at this time the bi in IO is relatively small, the file system efficiency is better.



The value of Si and so is not 0 for a long time, indicating insufficient system memory. system memory needs to be increased.

2.4.1 Disk I/O performance evaluation 1. Disk Storage Fundamentals

frequently accessed files or data with memory read-write instead of direct disk I/O, is highly efficient.

Separate the frequently read-write files from the long-term unchanging files and place them on different disk devices.

For data that is frequently written, consider using a bare device instead of a file system.

Bare Device Advantages:


The disadvantages of using a bare device are:

Data management, space management is not flexible, requires a very professional people to operate.

2. Evaluating disk Performance with Iostat
[[email protected] ~]# iostat -d 2 3Linux 2.6.9-42.ELsmp (webserver) 12/01/2008_i686_ (8 CPU) Device: tps Blk_read/sBlk_wrtn/sBlk_read Blk_wrtnsda 1.87 2.58 114.12 6479462 286537372 Device: tps Blk_read/sBlk_wrtn/sBlk_read Blk_wrtnsda 0.00 0.00 0.00 0 0 Device: tps Blk_read/sBlk_wrtn/sBlk_read Blk_wrtnsda 1.00 0.00 12.00 0 24

The explanations are as follows:


A basic understanding of disk read and write performance is available through the blk_read/s and BLK_WRTN/S values.
If the BLK_WRTN/S value is large, indicating that disk writes are frequent, consider optimizing the disk or program,
If the blk_read/s value is large, it means that there are many direct disk read operations, which can put the read data into memory

Rules follow:

Long-term, ultra-large data read and write, is certainly not normal, this situation will certainly affect the performance of the system.

3. Using SAR to evaluate disk performance

With the "sar–d" combination, you can make a basic statistic of the system's disk IO, see one of the following outputs:

[[email protected] ~]# sar -d 2 3Linux 2.6.9-42.ELsmp (webserver) 11/30/2008_i686_ (8 CPU)11:09:33 PM DEV tps rd_sec/swr_sec/savgrq-sz avgqu-sz await svctm %util11:09:35 PM dev8-0 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.0011:09:35 PM DEV tps rd_sec/swr_sec/savgrq-sz avgqu-sz await svctm %util11:09:37 PM dev8-0 1.00 0.00 12.00 12.00 0.00 0.00 0.00 0.0011:09:37 PM DEV tps rd_sec/swr_sec/savgrq-sz avgqu-sz await svctm %util11:09:39 PM dev8-0 1.99 0.00 47.76 24.00 0.00 0.50 0.25 0.05Average: DEV tps rd_sec/swr_sec/savgrq-sz avgqu-sz await svctm %utilAverage: dev8-0 1.00 0.00 19.97 20.00 0.00 0.33 0.17 0.02

Parameter meaning:


Evaluation criteria for disk IO performance:

The normal SVCTM should be less than the await value, and SVCTM and disk performance, CPU, memory load will also affect the SVCTM value, too many requests will indirectly lead to the increase of SVCTM value.


%util--measures disk I/O important metrics,

If the%util is close to 100%, indicating that the disk generates too many I/O requests and that the I/O system is fully operational, the disk may have bottlenecks.

The program can be optimized or replaced by a higher, faster disk.

2.5.1. Network performance evaluation
(1)通过ping命令检测网络的连通性(2)通过netstat –i组合检测网络接口状况(3)通过netstat –r组合检测系统的路由表信息(4)通过sar –n组合显示系统的网络运行状态
Three Linux server performance Tuning 1. Tuning the Linux kernel elevator algorithm for disk I/O

When the file system is selected, the algorithm balances low latency requirements, collects enough data, and effectively organizes requests for disk reads and writes.

2. Disable unnecessary daemons, save memory and CPU resources

Some Linux daemons that should be disabled are automatically run by default:

Serial Number Daemon Description
1 APMD Advanced Power Management Daemon
2 Nfslock for NFS file locking
3 ISDN ISDN Moderm support
4 Autofs automatically mount the file system in the background (e.g. Mount CD-ROM automatically)
5 Sendmail Mail Transfer agent
6 Xfs X Window font server

3. Turn off GUI4, clean up unnecessary modules or functions

Too many features or modules that are launched in a server package are actually unwanted (like many feature modules in Apache), disabling helps increase the amount of system memory available, freeing up resources for the software that really needs it to run faster.

5. Disable the Control Panel

In Linux, there are many popular control panels, such as cpanel,plesk,webmin and phpMyAdmin, which disable the release of approximately 120MB of memory and memory usage by approximately 30-40%.

6. Improve Linux Exim server performance

With the DNS cache daemon, you can reduce the bandwidth and CPU time required to resolve DNS records, and the DNS cache improves network performance by eliminating the need to find DNS records every time from the root node.

Djbdns is a very powerful DNS server with DNS caching capabilities, DJBDNS is more secure than the BIND DNS server, performs better, can be downloaded directly through, or is available through Red Hat packages.

7, using AES256 enhanced GPG file encryption security

To improve the security of backup files or sensitive information, many Linux system administrators use GPG encryption, in the use of GPG, it is best to specify GPG using AES256 encryption algorithm, AES256 use 256-bit key, it is an open encryption algorithm, the United States National Security Agency (NSA) Use it to protect top secret information.

8. Remote Backup service Security

Security is the most important factor in choosing a remote backup service, and most system administrators are afraid of two things: (hackers) can delete backup files and not restore the system from backup.

To ensure the security of backup files 100%, backup service companies provide remote backup servers, use SCP scripts or rsync to transfer data over SSH so that no one can access and access remote systems directly, so no one can delete data from the backup service. When choosing a remote backup service provider, it's a good idea to understand its service robustness in many ways, and if you can, you can test it yourself.

9. Update the default kernel parameter settings

To successfully and successfully run an enterprise application, such as a database server, you may need to update some of the default kernel parameter settings, for example, 2.4.x series kernel Message Queuing parameter Msgmni has a default value (for example, shared memory, or Shmmax in red The hat system has a default of only 33554432 bytes, it only allows limited database concurrent connections, and below provides some recommended values for better database server operation (from the IBM DB2 Support site):

kernel.shmmax=268435456 (32-bit)
kernel.shmmax=1073741824 (64-bit)
Kernel.sem= "250 32000 32 1024″

10. Optimize TCP

Optimizing the TCP protocol helps improve network throughput, the larger the bandwidth used across the WAN, and the longer the latency, the more TCP Linux size is recommended to increase the data transfer rate, and the TCP Linux size determines whether the sending host will not receive a data transfer acknowledgement How much data can be sent to the receiving host.

11. Select the correct file system

Replace ext3 with the Ext4 file system

EXT4 is an enhanced version of the Ext3 file system, extending storage limits

Log capability to ensure high levels of data integrity (in abnormal shutdown events)

It does not need to check the disk (this is a very time-consuming action) when it is not properly shut down and restarted

Faster write speed, EXT4 log optimizes drive head motion

12. Using the Noatime file system Mount option

Using the Noatime option in the File system boot profile fstab, this mount option can effectively improve performance if external storage is used.

13. Adjust Linux file descriptor limits

Linux limits the number of file descriptors that any process can open, with the default limit of 1024 per process, which may hinder benchmarking clients (such as Httperf and Apachebench) and the Web server itself from getting the best performance, with one process per connection for Apache, So it is not affected, but a single-process Web server, such as Zeus, uses one file descriptor per connection and is therefore susceptible to the default restrictions.

Opening a file limit is a restriction that can be adjusted with the Ulimit command, the ulimit-as command displays the current limit, and the Ulimit-ah command displays a hard limit (you cannot increase the limit until the kernel parameters in/proc are not adjusted).

Linux third-party application performance Tips

As with third-party applications running on Linux, there are many performance tuning techniques that can help you improve the performance of your Linux servers and lower your operating costs.

14. Correct configuration of MySQL

To allocate more memory to MySQL, you can set the MySQL cache size, and if the MySQL server instance uses more memory, reduce the cache size and increase the MySQL cache if MySQL stalls when the request grows.

15. Proper configuration of Apache

Checking how much memory Apache uses, and then adjusting the startservers and minspareservers parameters to free up more memory, will help you save 30-40% of memory.

16. Analyzing Linux Server performance

The best way to improve system efficiency is to find the bottleneck that causes the overall speed drop and solve it, here are some basic tips for finding the key bottlenecks in your system:

When a large application, such as OpenOffice and Firefox, runs simultaneously, the computer may start to slow down and the chances of running out of memory are higher.

If it is really slow to start, it may be that the initial startup of the application takes a long time to load, and it is normal to run once it is started, or the hard drive is probably too slow.

CPU load continues to be high, memory is sufficient, but CPU utilization is low, and the load time can be monitored using the CPU load analysis tool.

17. Learn 5 Linux performance commands

You can manage the performance of your Linux system with a few commands, listed below are the 5 most common Linux performance commands, including
top, Vmstat, Iostat, free, and SAR, which help system administrators quickly resolve performance issues.

(1) Top

The tasks of the current kernel service also display statistics for many host states, which are automatically updated every 5 seconds by default.
such as: current uptime, system load, number of processes and memory utilization,

In addition, this command shows the processes that use the most CPU time (including the various information for each process, such as running users, executing commands, and so on).

(2) Vmstat

The Vmstat command provides a snapshot of the current CPU, IO, process, and memory utilization, which, like the top command, automatically updates data such as:

$ vmstat 10

(3) Iostat

Iostat provides three reports: CPU utilization, device utilization, and network file system utilization, and three reports can be displayed independently using the-c,-d and-H parameters.

(4) Free

Displays the main memory and swap space memory statistics, specifying the-t parameter to display total memory, specifying the-b parameter in bytes, using-m in megabytes, and by default in kilobytes.

The free command can also be run continuously using the-s parameter plus a delay time (in seconds), such as:

$ free-s 5

(5) SAR

Collect, view, and record performance data, a command that is older than the previous commands, and collects and displays longer periods of data.


Here are some of the other performance techniques that are categorized as:

18. Transfer log files to memory

When a machine is running, it is best to put the system log in memory, when the system shuts down and then copy it to the hard disk, when you run a syslog-enabled laptop or mobile device, Ramlog can help you improve the system battery or mobile device flash drive life, One of the benefits of using Ramlog is that there is no need to worry about a daemon sending a message to the syslog every 30 seconds, before the hard drive must be kept running, which is bad for hard drives and batteries.

19. Package first, then write

In memory to partition a fixed size of the space to save the log file, which means that the laptop hard drive does not have to keep running, only when a daemon needs to write to the log to run, note that the Ramlog use of memory space is fixed, otherwise the system memory will be used up quickly, if the notebook uses solid-state drive, 50-80MB memory can be allocated to ramlog use, Ramlog can reduce many write cycles, greatly improve the life of solid-state drives.

20, general tuning skills

Use static content instead of dynamic content whenever possible, and if you are generating weather forecasts or other data that must be updated every 1 hours, it is best to write a program that generates a static file every 1 hours instead of letting the user run a CGI to generate the report dynamically.

Choosing the fastest and most suitable api,cgi for a dynamic application can be the easiest to program, but it will produce a process for each request, which is usually a costly and unnecessary process, fastcgi is a better choice, like Apache Mod_perl, Can greatly improve the performance of your application.

Linux Server Performance View Analytics tuning

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.