Exploration of AWS and azure in cloud computing (5)
-- EC2 and azure VM disk Performance Analysis
After the virtual machine is created, the CPU and memory configurations are basically clear at a glance. If you do not consider the performance of the video card, the most important performance bottleneck of a machine is the hard disk. Because both EC2 and azure VMS use virtual machines, storage disks are stored in disk arrays or NAS devices in some form, therefore, the Read/Write Performance of disks has become the most important consideration for cloud computing virtual servers. In this section, we will not consider the specific implementation of Elastic Block Store in EC2 or azure drive in azure, but use the free HD tuner, analyzes the performance of Virtual Machine disks in EC2/azure.
EC2 EBS Definition
First, in EC2, we created a large server that runs Windows Server 2012. Four disks are attached to the server as follows:
|
Size |
Type |
Remarks |
C |
30 GB |
Standard |
Auto-enable volume IO = on, System Disk |
D |
50 GB |
Standard |
Auto-enable volume IO = off |
E |
100 GB |
Provisioned iops |
Iops = 1000 |
F |
200 GB |
Provisioned iops |
Iops = 2000 |
The reason for selecting large here is that, as mentioned in the EC2 documentation, if you use a provisioned iops volume, you 'd better use an EBS-optimized instance. EBS optimized instances Use optimized configurations to provide dedicated EBS throughput capabilities. This optimization minimizes the content between EBS Io and other traffic of your EC2 instance to achieve optimal performance. EBS optimization allows the instance to fully use the iops on the EBS volume. The traffic is between 1000 Mbps and Mbps, depending on the instance type you are using. The iops volume is designed to provide performance of less than 99.9% at 10% of the year.
EBS optimized instances only support M1 large, M1 extra large, and high memory capacity of up to four times (m2.4xlarge). Therefore, the large type is selected here.
Another concept to be understood is the EBS type. EBS has two types: Standard and provisioned iops:
Standard volumes provide lightweight or separated I/O requirements. These volumes provide approximately 100 iops on average and about several hundred iops at the time of outbreak, suitable for boot disks.
Provisioned iops volumes are designed for applications with high Io requirements, especially databases. They are sensitive to storage performance and consistency during random I/O access. You can define the iops value when creating a volume. Currently, up to 2000 iops is supported for each volume. You can create multiple volumes for the ApplicationProgramTens of thousands of iops. The provisioned iops volume is at least 10 Gb, And the iops value is at least 10 times the volume size. For example, the minimum volume of 1000 iops is 100 GB.
Note that some conditions are required to meet the iops volume SLA.
1. This volume is attached to an EBS-optimized instance.
2. The average queue length is at least 1/200 iops.
3. The read/write operation block is 16 kb or smaller. For example, 1000 iops can send 1000 write records of 16 KB, 500 write records of 32 KB, and 250 write records of 64 KB.
Like a standard volume, the first time you access data, there is a maximum reduction in iops by 50%. After the first access, the performance will be restored. Therefore, to maximize the performance, it is best to access all the blocks before accessing the data. Snapshots also reduce iops performance. For more details, see the EBS documentation. Http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/AmazonEBS.html
Another thing to note is auto-enable volume Io, which is not related to disk performance. This means that when the disk status changes"Damage"(Impaired), so do not prohibit Io operations, automatically allow the disk to still perform Io operations.
Azure VM Definition
In azure Vm, we will create two azure drives, and two drives that come with Azure VM. There are also four drives on one machine:
|
Size |
Cache |
Remarks |
C |
30 GB |
Readwrite |
System Disk |
D |
140 GB |
Readwrite |
Built-in storage |
E |
100 GB |
None |
|
F |
100 GB |
Readwrite |
|
Azure VM is still preview, so there is no way to compare the document on disk type and Performance Analysis with AWS. What we can know Is that C, E, and F disks are stored in the azure storage account in the form of vhd and attached to virtual machines. Disk D is a disk that is added by default when a server creates a virtual machine and can be used for temporary data storage. Among them, C, E, F are all persistent storage. You can create snapshots and back up data across regions, while D disks do not have these advantages. Of course, d disks are delivered in vain, and no such d disks can be used on EC2.
Next, we will use the HD tune tool to compare the disk performance of azure and EC2, including disk read/write performance, read/write of files of different sizes, and random read/write. First, compare different types of EC2 disks, then compare the disk performance of azure Vm, and finally compare EC2 and azure disks horizontally.
EC2 disk performance comparison 1, Benchmark
Disk |
Minimum transmission rate (MB/s) |
Maximum Transmission Rate (MB/s) |
Average rate (MB/s) |
Access time (MS) |
Outbreak rate (MB/s) |
Legend |
C |
42.0 |
56.0 |
50.9 |
6.62 |
57.0 |
Figure 1 |
D |
12.2 |
58.9 |
52.8 |
8.00 |
51.2 |
Figure 2 |
E |
31.7 |
35.2 |
33.5 |
20.3 |
40.8 |
Figure 3 |
F |
32.7 |
40.8 |
33.6 |
19.4 |
40.8 |
Figure 4 |
Figure 1 Benchmark-EC2-C-System
Figure 2 benchmark-EC2-D-Standard
Figure 3 benchmark-ec2-e-1000 iops
Figure 4 benchmark-ec2-f-2000 iops
According to the performance of the four drivers in the table, the transmission rate of the standard drive is much higher than that of the provisioned iops drive. The average transmission rate of the standard drive is about 50 MB/S while that of the iops drive is only 33 Mb/s. Therefore, it is better to use the standard type for data that needs to be quickly transmitted for a short time.
2. File Benchmark
disk |
sequential read (kb/s) |
sequential write (kb/s) |
4 kb random read iops |
4 kb random write iops |
32 4 kb random read iops |
32 4 kb random write iops |
legend |
C |
102176 |
42056 |
1828 |
1046 |
16409 |
4316 |
Figure 5 |
D |
98794 |
38097 |
2142 |
1301 |
15569 |
4440 |
Figure 6 |
E |
41631 |
40365 |
1020 |
1020 |
1019 |
1019 |
Figure 7 |
F |
41890 |
34581 |
2040 |
1869 |
2057 |
2040 |
Figure 8 |
Figure 5 file benchmark-EC2-C-System
Figure 6 file benchmark-EC2-D-Standard
Figure 7 file benchmark-ec2-e-1000 iops
Figure 8 file benchmark-EC2-F-2000 iops
From the above table, we can see that the reading performance of a standard disk is far better than that of an iops disk. For random read/write of 4 kb, the performance is also far better than that of the iops type, however, its read/write performance is greatly affected by external influences and interference. In addition, the sequential read Speed of standard data is nearly twice the sequential write speed, and the read speed is about 100 Mb/s, but the write speed is only about 35 Mb/s, this is basically the same as the test results just now. However, the standard disk claims 100 iops, but the test results are far better than the data he claims. For an iops disk, its read/write performance is relatively average and the speed is around 40 MB/s, but the iops is very good enough to meet our requirements. For disks with 1000 iops, the read and write operations are basically 1020 iops. For disks with 2000 iops, the read and write operations are basically 2040. Although the speed cannot be compared with the peak value of the standard type of disk, but due to stable performance, many requirements such as databases can be implemented reliably.
Azure VM disk performance comparison 1, Benchmark
Disk |
Minimum transmission rate (MB/s) |
Maximum Transmission Rate (MB/s) |
Average rate (MB/s) |
Access time (MS) |
Outbreak rate (MB/s) |
Legend |
C |
8.3 |
36.1 |
15.2 |
30.7 |
208.1 |
Figure 9 |
D |
304.9 |
1012.9 |
922.6 |
0.059 |
744.5 |
Figure 10 |
E |
9.7 |
15.8 |
15.0 |
3.44 |
10.9 |
Figure 11 |
F |
26.0 |
40.5 |
31.5 |
0.962 |
77.4 |
Figure 12 |
Figure 9 benchmark-azure-C-System
Figure 10 benchmark-azure-D-Temp
Figure 11 benchmark-azure-e-No Cache
Figure 12 benchmark-azure-F-readwrite Cache
One of the highlights of azure's disks is the complimentary D disk. The minimum transmission rate of this disk is 304.9 Mb/s, and the maximum speed is even 1012.9 Mb/s. The average speed is 922.6 Mb/s, this is the speed at which two ssd raid 0 Arrays can be achieved. In this case, EC2 is completely killed in seconds. Unfortunately, the data on this disk is not persistent and can only be used as temporary storage. The other three disks are a little confusing. The first is the c disk used as the operating system. The minimum transmission rate is only 8.3 Mb/s, the maximum is 36.1 Mb/s, and the average speed is 15.2 Mb/s. This is consistent with the speed of the E disk that does not open the cache. That is to say, the access performance of vhd files in azure storage account is only 15 Mb/s. In addition, the access time of drive C has reached 30.7 ms, which in many cases cannot meet the application requirements. Even if the read/write cache is enabled on the drive, the speed is only 31.5 Mb/s.
2. File Benchmark
disk |
sequential read (kb/s) |
sequential write (kb/s) |
4 kb random read iops |
4 kb random write iops |
32 4 kb random read iops |
32 4 kb random write iops |
legend |
C |
24003 |
12249 |
2944 |
5910 |
0 |
2041 |
Figure 13 |
D |
2353760 |
1911305 |
18938 |
17802 |
70883 |
67622 |
Figure 14 |
E |
58760 |
39056 |
188 |
164 |
4688 |
1519 |
Figure 15 |
F |
21846 |
13701 |
1059 |
156 |
1227 |
1470 |
Figure 16 |
Figure 13 file benchmark-azure-C-System
Figure 14 file benchmark-azure-D-Temp
Figure 15 file benchmark-azure-e-No Cache
Figure 16 file benchmark-azure-F-readwrite Cache
Basically, the performance is not very different from that of benchmark. The fastest sequential read and write on disk D reaches the astonishing 2353760kb/s and 1911305kb/s, I have seen the fastest hard disk read/write speed of around 2 Gb/s. If it is not an SSD, it must be similar to the memory cache technology. However, the speed of the data disks used for persistent storage is very low. The system disk only reads 23 Mb/s and writes 11 Mb/s, and iops data is also integrated, however, it is surprising that 32 random 4 kb read iops is actually 0. I tested it several times and it was all 0. That is to say, when we perform 32 random 4 kb read/write operations, the disk should be hang. I don't know whether this is because of the operating system or the software test, but the fact is that the disk does not work. This State can easily cause the operating system to lose the response and the program to be dragged to death. It is hoped that this terrorism will not occur after the feature of Azure is officially release.
In addition, for Disks without cache, the sequential read and write speeds are acceptable, which is about 50 Mb/s and 40 MB/s, however, iops only has a poor performance of 188 and 164. For disks with cache, the read/write speed is reduced to about 20 mb and 10 MB because cache verification is required.
Disk Performance Comparison Between EBS and azure drive
Based on the above data, we get the following summary table:
The so-called success and failure. Among all disks, the best performance is the temporary disk in azure. In addition, including system disks, Disks without cache and disks with cache, azure disks are weaker than EBS, and even disks become stuck with zero iops.
To let the azure VM exit the preview stage, the disk performance must be solved. Otherwise, a large number of problems will occur in practical applications.