Impact of storage methods and media on Performance

The data storage method has a great impact on the overall performance of the application. Is Data Access smooth or random? Store data on disks and flash cards.

The data storage method has a great impact on the overall performance of the application. Is Data Access smooth or random? Store data on disks and flash cards.

Summary

The data storage method has a great impact on the overall performance of the application. Is Data Access smooth or random? Are data stored on disks and stored on flash cards? Does multi-threaded read/write affect performance? How can we choose a variety of data storage methods? This article provides you with a performance test data in different storage modes, so that you can use this data to select an appropriate data storage mode during future program development.

TAG

Storage performance, innodb performance, storage medium

Directory

Directory... 1

Introduction... 1

Storage performance analysis... 2

Test procedure description :... 2

Store test data :... 2

Mysql innodb performance testing... 4

Mysql (InnoDB) disk flushing policy... 6

C/s mode communication performance... 6

Direct file storage... 7

File IO method... 7

Completely Random write or skip, 5 times performance gap... 8

Multithread random read, processing speed, response time... 9

System cache... 10

Kernel Parameters related to system cache... 10

Write back. 10 on the dirty page

Summary... 11

Introduction

The data storage method has a great impact on the overall performance of the application. Is Data Access smooth or random? Are data stored on disks and stored on flash cards? Does multi-threaded read/write affect performance? How can we choose a variety of data storage methods?

This article will conduct detailed performance tests on different storage methods to provide you with performance test data for different storage methods. It will also briefly introduce the performance differences between different storage methods.

Storage Performance Analysis

There are various reasons that affect the storage speed, including storage media, read/write disk methods, and the impact of the hardware environment on read/write disks. Here we will mainly share some research results on storage speed.

The hardware environment is as follows:

CPU: INTER Nehalem E5620 2.4GHZx2

Memory: PC-8500 4 GB * 8

Hard Disk: 300G 10 k * 2, RAID: 1

Flash: SSD 160GB_MLC X25-M G2 x 6

NIC: Gigabit

Data volume: 117 GB

Test procedure description:

The test is divided into two sets of programs:

A. Storage Test

A) the storage test program uses pread/pwrite for storage testing. The speed of the blockchain traversal is based on the block chain library developed by Fr.

B) to reduce the impact of system cache on random read/write

I. Increase the data volume to 117 GB

Ii. Each data is tested only once.

Iii. Clear memory at the program entrance

C) read and write all data at a time during sequential read/write tests.

D) during random read/write tests, 4 kb is read each time and 381 MB is read.

B. Network Performance Testing

A) Use ub + ubrpc to implement the server and client of the Stress tool.

B) UBSVR_NODELAY

C) Common Idl specifications

D) Test Two packet requests of different sizes.

Store test data:

Disk

Sequential read: 145.59 MB/s

Random read: 0.91 MB/s (4 kb read each time, 381 MB read)

Sequential write: 83.1 MB/s

Random write: 0.34 MB/s (4 kb for each write and 381 MB for each write)

Flash

Sequential read: 61.5 MB/s

Random read: 14.9 MB/s (4 kb read each time, 381 MB read)

Sequential write: 59.8 MB/s

Random write: 1.93 MB/s (4 kb for each write and 381 MB for each write)

Memory

Sequential write: 1655 MB/s

Random write: 1496 MB/s

Eg: block chain traversal speed: 10 million RMB, 565582 us

Sequential read/write Performance Comparison Between disks and flash cards (unit: MB ):

Comparison of random read/write performance between disks and flash cards (unit: MB ):

Compared with the random read/write performance of disks and flash cards, we can see that for write operations, the performance difference between disks and flash cards is small, in fact, the performance difference varies with the amount of data written at random write and other factors such as the flash block size. At the same time, in the flash with write optimization, when writing data to a flash card, the data is first written to a buffer. when certain conditions (such as buffer full) are met, the buffer data is flushed into flash, blocking write, this causes performance jitter. Therefore, when most of the application's operations are write operations, you can barely put data on the disk without a flash card.

However, from the test results, the random read performance on the disk and flash card is 8 times or more different, so when the program reads a considerable number of disk operations, putting data on the flash card is a good choice. For example, for a random query of a large number of database applications, we can consider putting the database storage files on the flash card.

On the other hand, we can intuitively see that the speed on the disk is much higher than the speed on the flash card, regardless of sequential reading or sequential writing. Therefore, if the data required by the program is loaded from the disk at a time, the modifications to the data after the program is loaded are all memory operations and do not directly write the disk. When you need to write the disk, it is also a time to dump data in the memory to the disk. We should all put data on disks, rather than flash cards.