How to double the disk I/O performance of the CentOS Server

How to double the disk I/O performance of the CentOS Server

In this issue, let's look at some ways to reduce file fragments in linux. It is mainly used for scenarios where disks are running at full capacity for a long time (such as http proxy servers). There is also a small trick to improve I/O performance by several times for Internet image servers. If a dedicated file system is customized for the server, the file fragmentation problem can be completely solved, and the disk io performance can reach the limit. For our proxy server, I/O performance is increased to 3-5 times.

The Linux kernel and various file systems are optimized in the existing file system to improve disk access speed. However, these optimization schemes can be fully utilized only when they are cooperated in our server design.

The Linux kernel caches most of the idle memory in the file system to the Virtual File System as a File Cache called pagecache. When the memory is insufficient, this part of the memory will be eliminated using the lru algorithm. Run the free command to view the memory. The cached part is the file cache.

If you can find the Statistical Features of file access in the current use scenario and write an elimination algorithm, the File Cache hit rate can be greatly improved. For http forward proxy, a good elimination algorithm can use 1 GB of memory to achieve the cache Effect of GB of lru algorithm. If you do not want to write a new elimination algorithm, you do not need to set up another File cache program at the application layer for caching.

Minimum allocation

The minimal allocation will waste some disk space (allocated but not used)

If a large number of small files are used in the current scenario, changing the pre-allocation size will waste a lot of disk space. Therefore, this value must be set based on the current scenario. It seems that the source code needs to be changed to take effect. I don't remember it. I changed it in. If you are interested, google it yourself.

Io access Scheduling

Targeted optimization: io access scheduling can greatly improve io performance, provided that the application layer initiates sufficient io access for linux to schedule at the same time. How can I simultaneously issue multiple I/O accesses from the application layer? Solution 1: Use aio_read to initiate multiple file read/write requests asynchronously.

Tip: when the file handle is set to non-blocking, the process will still wait for disk io sleep. Non-blocking does not take effect for file read/write. Under normal circumstances, reading a file will only introduce more than 10 milliseconds of sleep, so it is not obvious. When the disk io is very large, reading a file will cause more than 10 seconds of sleep. For details, see the kernel source code do_generic_file_read. lock_page_killable is called to sleep, but the non-blocking mark of the handle is not determined.

Pre-reading the Linux kernel predicts "future read requests" and reads data in advance. Pre-reading can reduce the number of read io and reduce the latency of read requests.

When files are expanded and disk space needs to be allocated, they can be stored in the memory instead of being allocated immediately. After multiple disk space allocation requests are aggregated, one-time allocation is performed.

There are several side effects of latency allocation: 1. If the application forcibly refreshes data after each write, latency allocation will not work. 2. latency allocation may introduce a large disk IO latency intermittently (because a large amount of data needs to be written to the disk at a time)

Targeted optimization:

Interested students can refer to http://jsmylinux.no-ip.org/applications/using-e4defrag/

"Storing files in each directory continuously" is a very valuable feature. Suppose there are 10 images on a webpage. Although these 10 images exist in 10 files, they are almost simultaneously accessed by users. If these 10 images can be stored in a continuous disk space, I/O performance can be improved by 10 times (10 files can be read at one seek) the traditional method is to splice the 10 images into a large image, and then cut the large image into 10 small images at the front end. With e4defrag, you can place the files that require continuous access in the same folder, and then regularly use e4defrag for disk sorting.

To implement our own file system, we once wrote a dedicated File System, which improves the disk I/O performance to 3-5 times for the proxy server. On most servers, you do not need to support the "Modify file" function. Once a file is created, it cannot be modified. It can only be read or deleted. On this premise, we can eliminate all file fragments and increase the disk I/O efficiency to the theoretical limit.

For Files larger than 16 MB, the file system is notified to allocate 16 MB disk space when the server creates a file. Each time the file is expanded to an hour later, the file size is either 16 MB or the end of the file. It is not allowed to allocate non-16 MB space when the file is not terminated. When reading and writing a file, read and write 16 MB each time or until the end of the file.

In our file system, small files are completely free of fragments, and a file can be obtained in a single seek, achieving the best theoretical performance. Each time a large file head locates and reads/writes 16 MB, the performance does not reach 100%, but it is already quite good. There is a formula to measure the disk I/O efficiency: disk utilization = transmission time/(average track time + transmission time) for the disk we used at that time (1T7200 to sata ), 16 MB continuous read/write can reach more than 98% disk utilization.