GIS image cutting performance tuning and GIS image Performance Tuning

GIS image cutting performance tuning and GIS image Performance Tuning
Status Quo and objectives

DellR920 Server, virtualized vmware vsphere 5.5 system, Guest System Windows Server 2012 R2. Deploy software ARCGIS 10. Image graph cutting test: the original cut takes 70 hours, And now it takes 17 hours. There is still room for further optimization. Debugging objectives:

 

1. Verify the optimal graph cutting parameter configuration in the virtualization environment, and issue the optimal performance configuration method for ArcGIS10 In the virtualization environment.

2. verify the effectiveness of SSD applications, and test the optimal performance ratio of ssd srt, slice cache, ESXi host cache, and SAS 12 GB, it is concluded that the slice and the optimal SSD Configuration Optimization Method in the R920 server environment.

3. Verify the optimal Optimization Method of ARCGIS system. Best experience in ARCGIS optimization and debugging.

4. Shorten the optimization time again and try to reach the shortest time by using different technical means.

 

Optimization content A. Virtualization

1. Select VSPHERE5.5 Updata 2 with VMtools for the vsphere server version.

2. When the host physical memory is insufficient, the ESXi host cache technology is used.

3. hardware optimization

4. Virtual Machine Configuration

5. optimized the VMFS file system and enabled the AlwaysWrite Back write buffer function for SAS and raid cards. Because graph cutting is a continuous write operation, this operation is required.

B. Windows 2012 R2

1. Modify the Registry to change the performance of the TCP network protocol stack.

2. Enable the MAX value of the default CPU performance for Windows 2012. Windows 2012 disables some CPU cores for power-saving considerations.

3. Optimize page file, it is recommended to set to the minimum value = physical memory * 2 + 50 M, maximum value = physical memory * 4 + 50 M, can refer to the MS official documentation: http://support.microsoft.com/kb/2860880

4. IO optimization. Modify the default number of IO requests for windows in the registry.

5. NTFS file system optimization

C. SSD Optimization

SSD-based optimization solves the write amplification problem caused by erase-before-write. Different types of IO separation reduce the performance impact of write operations.

1. Change sequential logging to In-page logging to avoid repeated writes at the same location.

2. A large number of random in-place update writes are merged into a small number of sequential writes through cache writing.

3. Taking advantage of the high random read/write capability of SSD, we can reduce the number of writes and increase the read performance.

4. control Panel-management tools-service, disable Superfetch, Disk Defragmenter, and Windows Search Services: For SSD, due to the change in storage mode, the original Disk fragmentation cleaning becomes redundant, it is even harmful, so we need to disable disk fragment cleaning. However, the Superfetch pre-reading and Windows Search indexing for traditional mechanical hard disks do not need to be disabled now.

5. Use the memory with faster read/write speeds as the SSD high-speed cache. The principle is that the system first stores data in the virtual cache of the memory when reading and writing SSD. Some common data can be read and written repeatedly in the memory, thus reducing the number of SSD reads and writes. The advantage of this is that the read/write speed is faster, and the SSD Service Life can be prolonged.

D. ARCGIS Optimization

1. Maplex advanced note. Map documents edited using the Maplex advanced note function may cause low cut efficiency. The easiest way to determine whether the cropping speed is slow due to Maplex is to disable the Maplex engine, cut the graph, and compare the size of the generated slicing file within the unit of time with the original one, if the speed varies greatly, you can try to delete all the layers in the map document except the current cut graph scale, and then split the map hierarchically.

2. Format of the published file. Most of the time, we make a cut chart on the server. Citing ESRI's after-sales technical engineers, the Services released through mxd on multi-core servers may have exceptions. esri acknowledges that it is a bug, we recommend that you use the msd-based map service. Therefore, to slice on the server, you must save the map document as published in msd format and then cut the map.

3. Set the maximum number of instances. In general, we will set the number of instances of the graph service to the number of CPU cores + 1, because there is a SOC process dedicated for recycling and is not involved in actual work, therefore, this setting can maximize the performance of the slice server. However, in our actual operation, we found that this method will cause the SOC process to be recycled after being created on some models of servers, and only one process is splitting the graph. At this time, we can only find a proper value by constantly trying to set different instance numbers.

4. Cropping mode. As we all know, there are two ways to cut a graph. One is to directly execute an update slice in the map service attribute, or use the server tool provided by the ArcToolbox tool to cut the graph. In principle, both methods call the same GP process. There is no difference, but in actual use, we often encounter ArcMap program shutdown exceptions when updating slice through service properties. Therefore, we recommend that you use a tool to cut the graph.

5. Compact & loose. ArcGIS10 introduces a new cache format, which saves original independent cached images as bundle binary files. each bundle File Stores 128*128 cached images. Compared with thousands of tile files, this method is easier to manage cache, reduce disk space usage, and improve cache Read efficiency. Therefore, we should also use a compact format to create Slices During graph cutting.

6. image format. Different image formats directly affect the size of cached files. Generally, the system uses PNG32 by default to save tile files. However, if we want to reduce the occupied space, you can use JPG, PNG8, and PNG24 formats. Although JPG has the largest compression ratio, it is not recommended because the generated slice file has serious noise. Through multiple attempts, the PNG8 format not only can greatly compress the size of the cached file (about 60% of the PNG32 format), but also has little difference in the effect of slicing, therefore, it is a good alternative to the PNG32 format.

Next step

1. Replace the SAS12GB interface to test the eMLC particles.

Upgrade the firmware of the master chip to the latest version. The firmware version A3AE and A00 updated on April 9, June 3, 2015.

• NAND: 24nm eMLC
• Interface: SAS 12 Gb/s
• Form Factor: 2.5 "SFF x 7mm High (15mm for 1.6 TB model)
• Performance
• Sequential Read (Sustained): 900 MB/s
• Sequential Write (Sustained): 400 MB/s
• Random Read 4 k (IOPS): 120,000
• Random Write 4 k (IOPS): 30,000

 

2. Use ARCGIS to optimize the cropping.

3. Try multiple scenarios and test targets to cut the graph and test its stability and speed.