Linux Performance Tuning Overview

Linux Performance Tuning Overview

-What is performance tuning? (What)
-Why performance optimization? (Why)
-When do I need performance optimization? (When)
-Where do I need performance tuning? (Where)
-Who will perform performance optimization? (Who)
-How to optimize the performance? (How)
-Conclusion

# What is performance tuning? (What )#

! [] (Http:// I .imgur.com/RkFKTz6.jpg)

Before talking about performance tuning, let's talk about the computer architecture.

For example, there are three parts: hardware, operating system, and application. In fact, performance tuning is to adjust this content, including hardware, operating systems, and applications. Among them, these three aspects contain a number of content.

Hardware includes: CPU, memory, disk, network card, other ......,

Operating systems include processes, virtual memory, file systems, networks, and others ......,

I don't need to talk about the application. We all know that common applications include Apache, MySQL, Nginx, and Memcahed.

So what is performance tuning?

Performance Tuning is to have a deep understanding of computer hardware, operating systems, and applications, adjust the relationship between the three, and maximize the performance of the entire system (including hardware, operating systems, and applications, and can continuously meet existing business needs. This is what we call Performance Tuning. What do you know about the customer service?

# Why Performance Tuning? (Why )#

Next, let's talk about why performance optimization is required. In the end, there are two reasons: first, to achieve better system performance (that is, your existing system runs well, but it can run better ). Second, performance tuning is used to meet increasing business needs. To help you better understand why performance tuning is required? In three aspects:

Hardware Selection (select servers based on the server application type) <br/>
Operating system release version (select release version) <br/>
Applications (Nginx, MySQL, etc.) <br/>

#1. hardware selection ##

Whether you are renting a server or buying a server by yourself, you have to have a problem. What kind of hardware configuration server should we choose.

Generally, we select servers based on the application type, because you cannot configure hardware to meet all application requirements, because the specific requirements of each application are different. Next, let's take a look at the application types in project implementation:

Server Load balancer: the performance requirement is relatively low, because it is only responsible for data forwarding, but you must select a network card with outstanding performance. (Recommended Configuration: CPU E5620 x 1 memory 8G hard disk 500G (RAID5 ))

Web servers: Generally, they only process some static pages or images. Therefore, they are not very demanding and can be used by mainstream servers. (Recommended Configuration: CPU E5620 x 1 memory 16G hard disk 500G (RAID5 ))

Application Server: Generally, an application server is responsible for implementing Website Functions and occupies a relatively important position in the architecture. In particular, there is only one application server in the website architecture, high requirements on CPU, memory, and disks. (Recommended Configuration: CPU E5620 x 2 memory 32G hard disk 500G (RAID10 ))

Cache servers: they are divided into front-end page cache and back-end data cache. Their typical applications are Varnish and Memcached, which have high memory requirements. Generally, we use large memory when configuring the server. (Recommended Configuration: CPU E5620 x 1 memory 32G hard disk 500G (RAID10 ))

Database Server: The data server has high requirements on CPU, memory, and disk. Once a hardware is short, performance problems may occur. (Recommended Configuration: CPU E5620 x 2 memory 64 gb ssd 500 GB (RAID10 ))

Backup Server: Generally, the backup server has no requirements, but it must have enough hard disk space. (Recommended Configuration: CPU E5620 x 1 memory 4 GB hard disk 2 TB (RAID5 ))

Monitoring Server: Generally, there is no need for a monitoring server. (Recommended Configuration: CPU E5620 x 1 memory 4G hard drive 500 (RAID5 ))

Other servers: for other servers, you can see the specific needs of specific analysis.

Now you know what is the performance tuning of the hardware. According to your specific application, perform specific analysis, especially for servers like MySQL, which have high requirements on CPU, memory, and disk.

Therefore, to optimize the performance of hardware, We must select the appropriate hardware configuration. This is the first problem to be solved for website architecture or project implementation!

#2. Operating System ##

This book is called "Linux Performance Tuning" (Linux Performance Tuning), written by Fernando Apesteguia.

Why do we need performance tuning? He came to the conclusion that:

"When a release is packaged and sent to the customer, it is designed to be fully compatible with most computers in the market. This is a collection of rather mixed hardware (hard disks, video cards, network cards, and so on ).

Therefore, Red Hat, SUSE, Mandriva, Ubuntu, and some other release vendors have chosen some conservative settings to ensure installation is successful ."

Simply put, your operating system is running well, but you can adjust it to achieve higher performance. For example, if you have a high-performance disk, however, if some option parameters in your operating system are not enabled by default, these advanced features cannot be implemented to improve hard disk performance.

Also, I want to talk about the selection of operating system releases. RedHat or CentOS are used in many projects or website architectures, the operating system is mainly developed for enterprise applications.

Ubuntu and other operating systems have better desktop support, so pay attention to the release version. (CentOS is widely used in enterprises.) then, we generally do not select the latest release version, because the released version has many bugs, don't be a "Mouse" first,

For example, if CentOS 7 is stable for a period of time, you can select CentOS 6.4 or 6.5.

(However, the new version also has many benefits. The new version adds many new features and removes known bugs. You can try to use the new operating system for some unimportant applications)

#3. Application ##

Finally, let's talk about the application. Let's take a look at the Apache MPM configuration file:


Prefork model:

<IfModule prefork. c>
StartServers 8
MinSpareServers5
MaxSpareServers 20
ServerLimit 256
MaxClients 256
MaxRequestsPerChild 4000
</IfModule>
You can see from the configuration file that eight processes are enabled When apache starts to start, with a minimum of five processes and a maximum of 20 processes. Each process has a maximum of 256 requests, A maximum of 4000 requests can be accepted. If this limit is exceeded, the request is automatically destroyed.

Worker Model:

<IfModule worker. c>
StartServers 2
MaxClients 150
MinSpareThreads 25
MaxSpareThreads 75
ThreadsPerChild 25
MaxRequestsPerChild 0
</IfModule>
Take a look at the configuration file of the worker model. By default, two processes are started, each of which can accept 150 requests, and each process has a minimum of 25 threads, the maximum number of threads is 75. By default, the number of threads is 25. Each thread can accept a request with no limit of 0.

Now, after reading the configuration file above, we can see that the default Apache configuration file is relatively conservative and only suitable for some small and medium websites, to obtain a high-performance Apache server, performance tuning is also required, including apache compilation options and configuration file optimization. We will not elaborate on the specific optimization here.

Through the above explanation, we will start from three aspects: hardware, operating system, and application. We will discuss with you why performance tuning is required. I believe you know and understand this, I believe everyone can't wait.

Hey hey, let's talk about a lot of problems without making it clear. Let's talk to you next, when do we need performance tuning?

# When do I need performance optimization? (When )#

It is generally divided into two time periods:

-Before going online (basic optimization)
-Post-launch (continuous optimization)

In this case, we usually need to prepare hardware servers, install the operating system, and set up the environment during the project implementation period to launch the project. Another important issue is performance optimization, including operating system optimization and application environment optimization. I call the optimization before going online as basic optimization, also known as experience optimization.

Based on your project and work experience, you can optimize the performance of the server or architecture before the launch to meet your business needs.

There is also the optimization after the project goes online. Before going online, we have gone through basic performance optimization to solve most of the performance problems, after all, the tests before the launch are simulated and related performance optimization, which is quite different from the real environment after the launch.

The first thing we need to do is to monitor the performance of the launched projects, including server performance monitoring and service performance monitoring.

Server performance monitoring includes:

CPU usage, CPU load, memory usage, disk I/O, disk space usage, network traffic, and system processes. Service Performance Monitoring includes apache, nginx, and mysql.

Performance monitoring is required for all the services in the above architecture. If any problem is found, performance optimization is required. In this process, continuous optimization is also called monitoring optimization. Next, let's take a look at the specific points where performance tuning is required?

# Where do I need performance tuning? (Where )#


In the above section, we only talk about some major aspects of performance tuning, including hardware, operating systems, and applications.

In fact, another part is the optimization of the program itself. The program developed by developers according to their needs Needs needs performance optimization, but there is little access to our O & M personnel. Let's take a look at these three major blocks:

-Hardware (CPU, memory, disk, and nic)
-Operating System (process, file system, kernel ......)
-Applications (Nginx, MySQL ......)

#1. Hardware ##

Hardware optimization generally includes two parts:

-Pre-release (hardware selection)
-After launch (hardware expansion)

Generally, you need to select hardware configurations based on specific applications during project construction. In this regard, you need some O & M experience and new contacts may be somewhat lacking, however, after one or two projects, you can select the hardware configuration.

But with an unwritten experience, the higher the hardware configuration, the better (not to mention what I said ).

Why do we need to select a type based on specific applications? What kind of applications do we need? What kind of hardware configurations do we need? The important thing is to save costs, we can't spend anything we should not spend on cutting edge. It also saves costs for the company and maximizes resource utilization.

The above is the initial stage of project construction. You are lucky to be here at the beginning of the project.

Generally, experienced O & M engineers do not have any problems in hardware selection. Therefore, we do not consider hardware during performance optimization, theoretically, the hardware configuration of our servers will not occur in this performance issue.

However, as our business is getting better and better, we didn't take into account such a high performance requirement (access volume) at the initial stage of project creation. Currently, some hardware cannot meet business requirements, in this case, we need to replace a better CPU, a larger memory, and a faster disk.

As for how to identify hardware as a performance bottleneck, we will not elaborate on it here. We will elaborate on it later. Finally, let's look at a hardware architecture diagram to help you better understand hardware optimization, for example (Dell R 710 architecture ):

! [] (Http:// I .imgur.com/pHb7XaA.jpg)

#2. Operating System ##

Next we will talk about the operating system. In fact, most of the optimizations are in the optimization of the operating system and applications. Apart from the hardware selection before and after the launch, next, let's take a look at what operating system optimization includes:

-Operating System installation optimization
-System initialization
-Process Optimization
-Memory Optimization
-IO Optimization
-File System Optimization
-Network Adjustment

Let's take a look at a picture to help us understand it more intuitively, such:

! [] (Http:// I .imgur.com/6qcc4WU.jpg)

#3. Application ##

Finally, let's talk about application optimization. Here we will take a MySQL optimization example to give you a more intuitive understanding.

-MySQL compilation and installation optimization
-MySQL configuration file Optimization
-INDEX OPTIMIZATION
-MySQL Engine Optimization
-Query Cache Optimization
-SQL statement Optimization
-Optimize the table type (MyISAM or InnoDB)
-Lock Mechanism Optimization
-MySQL Server optimization (for SSD)

I believe that you have a deeper understanding of Performance Optimization in terms of hardware, operating systems, and applications. Next we will discuss an important issue, who will optimize performance?

# Who performs performance optimization? (Who )#

When talking about performance optimization, we first think of O & M engineers who will optimize the performance.

In fact, I want to say that one-sided performance optimization is not just about O & M engineers.

In fact, performance optimization is a task of the team. Why do I say this?

Let's take a look at the following. If a company needs to create a project, let's take the most common e-commerce mall project as an example, the company confirmed that due to business needs, we need to create a building materials mall online. What is the specific process of the project? It may not be very detailed, but the general process is as follows:

-Operational Requirements
-Product sorting requirements
-Develop and develop specific business applications
-O & M to build a Development Environment
-QA project testing
-Launch the O & M Project
-Monitoring for project monitoring
-To develop a specific application, the Operation Department, product department, Development Department, O & M department, QA (test), and monitoring department are required.

Performance problems exist in the same project (business), not only the O & M department needs performance optimization, but the Department solves this performance problem together, which is indispensable.

It may occur in products, programs (*. php), business needs, or O & M environments. However, most of the performance tuning activities include development, O & M, testing, and monitoring.

# How to optimize performance? (How )#

The following describes how to perform Performance Tuning. The specific steps are as follows:

-Performance indicator-> confirm measurement criteria
-Performance Test-> verify performance indicators
-Performance Analysis-> identify performance bottlenecks
-Performance Tuning-> solving performance problems
-Performance Monitoring-> test the optimization effect

#1. Performance indicators ##

As we mentioned above, our optimization aims to achieve better performance. What are the performance indicators? How can we measure it? Generally, there are three indicators to measure a project (the website here:

-Throughput> indicates the number of user or system requests completed per unit time.
-Concurrency-> the number of user access requests that can be accepted at the same time
-Response Time-> the interval between the time when the user sends a request and receives the response.

#2. Performance Testing ##

The purpose of our products or projects (more simply websites) is to allow users to use them. We need to analyze what performance users need to pay attention to from the user's perspective.

When a user clicks a button, link, or sends an Operation Command, the system sends the request to the user and displays it as a webpage, the time spent in this process is a visual impression of the website's performance.

That is to say, the response time. When the response time is small, the user experience will be relatively good. Of course, the response time for the user experience includes personal subjective factors and objective response time.

When developing and building websites, we need to consider how to better combine these two parts to achieve the best user experience. Users focus on the corresponding time of user operations.

Second, we should consider the performance points that need to be concerned from the O & M perspective. Once again, we have to consider website performance from the perspective of developers (design. Finally, QA tests and feedbacks on our website performance.

After the above description, we need to collect the throughput, concurrency, and response time of the system to test the system performance. The procedure is as follows:

Confirm the throughput, concurrency, and response time values.
Find or develop corresponding performance testing tools
Perform Performance Testing
Feedback results and submit test report
As a result, there are two ways to achieve our expected performance goals, so that we do not need to complete the performance optimization task to be handed over to O & M for release, but only need to perform relevant performance monitoring, it facilitates Performance Optimization after the product is launched.

The other is that we have not achieved our expected goal. We need to find performance bottlenecks and optimize the performance.

#3. Performance analysis ##

Through the above performance test, we found that the website did not meet our expected performance goal. What we need to do is to monitor the existing system (server), including hardware and software monitoring, it provides effective performance monitoring data for performance tuning.

Next we will focus on what tools can be used to identify performance bottlenecks:

Hardware:

-Use vmstat, sar, and iostat to check whether a CPU bottleneck exists.
-Use free and vmstat to detect memory bottlenecks
-Use iostat to check whether it is a disk I/O bottleneck
-Use netstat to check whether it is a network bandwidth bottle

Operating System:

-Process
-File System
-SWAP Partition
-Kernel Parameter Adjustment
-Applications (MySQL, etc ):

Mysqlreport Performance Analysis Report
Mysqlsla slow query log analysis

#4. Performance Tuning ##

-Determine the optimization target
-Specific optimization steps
-Detection and optimization results
-Determine the optimization target

The goal of our performance optimization is to improve the website performance by 10% or 20%. The boss cannot say that if you optimize the website performance for me today, you will be able to double the website performance.

First, you need to ask him what we need to achieve.

Next, let's take a look at the entire environment (Architecture) including code (of course, you need to understand the business logic and the general understanding, which is certainly no harm). We have time to communicate with developers, it is important to ask how many pitfalls are required in the code.

It is often better to query SQL statements in the code than how many days your system has been optimized. Haha.

Optimization steps

If you do not understand system parameters, do not modify them at will. Otherwise, you will regret it.
Only one system resource is debugged at a time, such as CPU, memory, or disk.
We recommend that you modify the parameter values as few as possible for each change.
When analyzing a system resource, multiple tools often have unexpected results.
Better than others (I would rather do less than overhead it. If the performance has reached the requirement, do not do anything at will and do your monitoring well ).
Detection Optimization Result

After each performance optimization, you must check the performance process. For example, the AB tool of the Web server is a good detection tool. After each optimization, you can see the specific changes.
#5. Performance monitoring ##

Performance monitoring is very important, including server performance monitoring and performance monitoring of specific services. The following describes the specific performance metrics:

Server performance monitoring

-CPU usage
-CPU load
-Memory usage
-Disk I/O
-Network Traffic
-Disk space
-System Process
-Service Performance Monitoring (MySQL)

-MySQL query throughput, including Change DB, Select, Insert, Update, and Delete
-MySQL persistent connection Utilization
-MySQL queries cache space usage
-MySQL query cache hit rate
-MySQL cache Query Count
-MySQL index cache hit rate
-MySQL index read statistics
-MySQL connection Throughput
-MySQL connection cache hit rate
-MySQL concurrent connections, including the maximum number of allowed connections, the actual maximum number of connections, the current number of connections, the number of active connections, and the number of cached connections
-MySQL traffic statistics
-MySQL table statistics locking

# Conclusion #

In this "performance optimization overview" blog, I just want to explain the specific optimization ideas and help you understand performance optimization. This makes it easier for you to understand, let everyone know that performance optimization is not as difficult as it is to be done. As long as we have a good grasp of the method, we can solve any problem.

 

For more information, visit the Li xingli blog.