How to perform a performance calibration test on Exadata I/O

some time ago, I saw MacLean Liu share a good article on Exadata I/O performance calibration: Calibrate Test Exadata IO, this article basically covers the exadata I/O performance calibration method. Just recently someone has asked me this question, so I will try to make some additions to the fish, there is duplication of places please ignore it.

If I/O performance calibration or testing is required on exadata, there are usually several methods:

1. Oracle I/O performance calibration Tools Orion;

2. Oracle Database 11g introduced the Dbms_resource_manager. Calibrate_io package;

3. Calibrate command in the Exadata cell end cellsrv;

Of course, it's said that testing with DD also allows you to get IOPS and Mbps of the disk. In the absence of any tools, of course, DD can also be used to make rough estimates, but the results are usually unreliable, so there is no dd such as this category. The following are a few simple introduction and comparison of these methods. Orion Orion is a database-independent I/O performance calibration and testing tool that almost supports all operating system platforms. Orion is free, and users can download their media to Otn downloads link: http://www.oracle.com/technetwork/topics/index-089595.html. I/O calibration with Orion is also simple, including the following steps: (for example, Linux x86_64) 1. Download the Orion media, upload it on a server that requires I/O calibration, and then unzip it to get a binary file; 2. Create a file called Orion.lun , which records the name of the device (LUN) that requires I/O calibration, for example;

/dev/sdb
/dev/sdc
/dev/sdd
/DEV/SDE 3. Perform I/O calibration operations;

./orion_x86_64 0 2 0 2–run <workload_mode>

Where workload mode (load mode) consists of 5 kinds, for the database, commonly used in DSS mode and OLTP mode, for more complex requirements can use advanced mode.

 Simple-tests random 8K small ios in various loads, then random 1M large iOS at various loads.
Normal -tests combinations of random 8K small iOS and random 1M large iOS
advanced -run the Worklo Ad specified by the user using optional Parameters
DSS -run and random 1M large IOs at increasing loads to Determine the maximum throughput
OLTP -run with random 8K small IOs in increasing loads to determine the max Imum IOPS

4. View I/O calibration report Orion_<date>_summary.txt, you can use Excel or 0 2gnuplot to generate a chart. It is not possible to provide a report on the Orion Test Exadata because there is currently no test data. The advantages of Orion are as follows: not dependent on the operating system and database version, no need to install the database before I/O calibration, simple, free, and by default provide a variety of calibration modes to choose from, but at the same time there are some disadvantages: The tool is too simple, There are some deficiencies compared with the professional performance testing software such as LoadRunner; Unable to automatically perceive the Oracle RAC architecture, the resulting calibration data is only obtained from a particular host, and this tool cannot be used to test NAS storage devices. Calibrate_ioDbms_resource_manager. Calibrate_io is an I/O calibration function introduced by 11g. The following information needs to be checked before testing: Timed_statistics 0 2 must be set to true; asynchronous I/O is enabled; If you use a file system, Filesystemio_options 0 2 needs to be set to SetAll. You can query whether the asynchronous I/O of the data file is enabled by using the following command:

COL NAME FORMAT A50
SELECT name,asynch_io from V$datafile f,v$iostat_file I
WHERE  f.file#=i.file_no
and    filetype_name= ' DATA FILE ';

0 2dbms_resource_manager. Calibrate_io This function needs to provide 2 parameters as input: 0 2num_physical_disks (number of physical disks, default 1), and max_latency (maximum latency for physical disks, default 20ms), After the test is completed, 3 results are generated 3 results: Max_iops: Maximum ioPS, that is, the maximum number of I/O requests per second can be obtained. Its test I/O request is based on random distribution, database block size read operations; max_mbps: Maximum Mbps, that is, the maximum throughput per second, in megabytes. The requested I/O request for its test is based on a randomly distributed read operation of 1M size; Actual_latency: Actual latency, in the case of maximum IOPS, the average latency of I/O requests based on the size of the database block, in milliseconds.

SET serveroutput on
DECLARE
lat INTEGER;
ioPS INTEGER;
Mbps INTEGER;
BEGIN
Dbms_resource_manager. Calibrate_io (1, IOPS,
Mbps, LAT);
Dbms_output. Put_Line (' max_iops = ' | | iops); Dbms_output. Put_Line (' latency = ' | | lat); Dbms_output. Put_Line (' max_mbps = ' | | mbps ');
End;
/

After the test is complete, the results of the test are displayed below. Of course, Oracle also provides the corresponding dynamic performance view for query:

SELECT * from Gv$io_calibration_status;
SELECT * from Dba_rsrc_io_calibrate;

The advantage of Calibrate_io is that it automatically senses the RAC architecture, not the results from a single node test, and that the test takes the data file of the database as the benchmark, and is the closest to the database. When you do a exadata POC, you can use it to compare I/O to test , comparing how much I/O performance can be improved after migrating to Exadata. Of course, there are some problems: the need for a database version of more than 11g, the test needs to install the database, the results of the test only the overall performance data, can not see the information of a single physical disk. CELLCLI Calibrate

In the case of stopping CELLSRV services, use the root user to run calibrate in CELLCLI. If you need to use online mode, you must add the Force parameter later. The following references the data to the blog from MacLean Liu:

cellcli> calibrate Force;
Calibration would take a few minutes ... Aggregate Random read throughput across all hard disk luns:1921 MBPS Aggregate Random read throughput across all Flash di SK luns:4164.33 MBPS Aggregate Random Read IOs per second (IOPS) across all hard disk luns:4971 Aggregate Random Read IO S/second (IOPS) across all flash disk luns:145195 Controller read throughput:1919.64 MBPS calibrating hard disks (re
Ad only) ... LUN 0_0 on drive [28:0] Random Read throughput:168.12 MBPS, and 430 IOPS LUNs 0_1 on drive [28:1] Random Read throughput  : 164.23 MBPS, and 423 iops LUN 0_10 on drive [28:10] Random Read throughput:170.80 MBPS, and 433 ioPS LUNs 0_11 on drive [28:11] Random Read throughput:168.32 Mbps, and 421 IOPS LUN 0_2 on drive [28:2] Random Read throughput:170.07 MBPS and 431 ioPS LUN 0_3 on drive [28:3] Random Read throughput:169.82 MBPS, and 421 IOPS LUNs 0_4 on drive [28:4] random RE Ad throughput:165.17 MBPS, and 417 IOPS LUN 0_5 on drive [28:5] RaNdom read throughput:166.82 Mbps, and 429 ioPS LUN 0_6 on drive [28:6] Random Read throughput:170.85 Mbps, and 430 IOPS LUN 0_7 on drive [28:7] Random Read throughput:168.42 MBPS, and 429 IOPS LUNs 0_8 on drive [28:8] Random Read Throughpu t:169.78 MBPS, and 428 ioPS LUN 0_9 on drive [28:9] Random Read throughput:168.77 MBPS, and 430 IOPS calibrating flash
Disks (Read only, note that writes would be significantly slower) ... LUN 1_0 on drive [FLASH_1_0] Random Read throughput:271.01 MBPS, and 19808 IOPS LUNs 1_1 on drive [flash_1_1] Random Read throughput:270.24 MBPS, and 19821 ioPS LUN 1_2 on drive [flash_1_2] Random Read throughput:270.41 MBPS, and 19844 ioPS L UN 1_3 on drive [flash_1_3] Random Read throughput:270.37 MBPS, and 19812 IOPS LUNs 2_0 on drive [flash_2_0] Random Read t hroughput:272.32 MBPS, and 20634 ioPS LUN 2_1 on drive [flash_2_1] Random Read throughput:272.12 MBPS, and 20635 ioPS LU
N 2_2 on drive [flash_2_2] Random Read throughput:272.28 MBPS, and 20676 IOPSLUN 2_3 on drive [flash_2_3] Random Read throughput:272.43 MBPS, and 20669 IOPS LUNs 4_0 on drive [FLASH_4_0] Random Read throughput:271.13 MBPS, and 19802 ioPS LUN 4_1 on drive [flash_4_1] Random Read throughput:271.90 MBPS, and 19799 ioPS L UN 4_2 on drive [flash_4_2] Random Read throughput:271.42 MBPS, and 19798 IOPS LUNs 4_3 on drive [flash_4_3] Random Read t hroughput:272.25 MBPS, and 19808 ioPS LUN 5_0 on drive [flash_5_0] Random Read throughput:272.22 MBPS, and 19824 ioPS LU N 5_1 on drive [Flash_5_1] Random Read throughput:272.44 MBPS, and 19823 IOPS LUNs 5_2 on drive [flash_5_2] Random Read th roughput:271.83 MBPS, and 19808 ioPS LUN 5_3 on drive [Flash_5_3] Random Read throughput:271.73 MBPS, and 19837 ioPS CAL
Ibrate results are within an acceptable range. Calibration has finished.

The calibrate here is a test for a single cell node, and the IOPS and Mbps data given on Exadata Datasheet are test results from this command. (first measure the individual cell node's indicators, and then multiply the number of cell nodes) at the same time in the test ioPS, the I/O request used is 8K of random read; When you test Mbps, you use random reads of 1M size. At the same time, the above tests give both IOPS and Mbps of the entire cell node, as well as IOPS and Mbps of individual LUNs in the cell node. So why not test all the cell nodes together and get a total test result? exadata Because different cell nodes need to be accessed through the InfiniBand network, if the test is carried out at the same time, the bottleneck may be on the InfiniBand, resulting in inaccurate test results.

Calibrate is designed for exadata, so it can only run on the Exadata cell side. Using it has the following benefits: Both the performance data of a cell node and the performance data of a single disk or flash disk can be obtained, while installation Exadata can be 0 2 to detect the I/O performance of the disk/flash disk, eliminate the possible defective disk/flash disk, and when I/O performance problem occurs , you can use it to calibrate the I/O of the cell to determine whether it is a hardware-level problem.

Above