Test SHP file index acceleration in gdal1.11

In the gdal library, you can set some filters for reading vector data to filter vector images. For the shapefile format, if the data volume is too large, it is simply intolerable to set this filter to be slow. Fortunately, gdal1.10 supports reading the spatial index file (. SBN/. sbx) of the shapefile for acceleration. The following code uses the same data to test the gdal1.9.0 and gdal1.11.0 versions, and compare the speed (you will surely see the results in a big mouth ~~).

The first is to test the code. The function is very simple. There are two SHP files, one vertex file and one surface file. The area file is very large. You need to query the corresponding area file graphics based on the points in the point file. We would like to thank the data provided by midnight wind for testing. Figure 1 shows how to use ArcMap to display two data items. Figure 2 shows the data volume and number of elements of two data items.

Figure 1 Effect of opening with ArcMap

Figure 2 data volume and number of elements

The following is the test code. Only the key functions are posted.

Void searchsampledatafromshp_liml () {const char * pszpoints = "C: \ Users \ liminlu \ Desktop \ SHP \ c5pointnew. SHP "; const char * pszpolygs =" C: \ Users \ liminlu \ Desktop \ SHP \ c5.shp "; // register the driver and the configuration item cplsetconfigoption (" gdal_filename_is_utf8 ", "No"); cplsetconfigoption ("shape_encoding", ""); ogrregisterall (); // open two data ogrsfdriver * podriver = ogrsfdriverregistrar: getregistrar () -> getdriverbyname ("ESRI shapefile"); ogrdatasource * ppntds = podriver-> open (pszpoints, false); If (ppntds = NULL) {cout <"open the file: "<pszpoints <" failed "<Endl; return;} ogrdatasource * pplgds = podriver-> open (pszpolygs, false); If (pplgds = NULL) {cout <"Open File:" <pszpolygs <"failed" <Endl; return;} ogrfeature * ppntfeature = NULL, * pplgfeature = NULL; ogrlayer * ppntlayer = ppntds-> getlayer (0); ppntlayer-> resetreading (); ppntfeature = ppntlayer-> getfeature (0); int nfeildcount = ppntfeature-> getfieldcount (); int nfeaturecount = ppntlayer-> getfeaturecount (); ogrlayer * pplglayer = pplgds-> getlayer (0); pplglayer-> resetreading (); pplgfeature = pplglayer-> getfeature (0 ); int nplgfeildcount = pplgfeature-> getfieldcount (); For (INT I = 0; I <nfeaturecount; I ++) {ppntfeature = ppntlayer-> getfeature (I ); double dvalue = ppntfeature-> getfieldasdouble (nFeildCount-1); ogrpoint * Ppoint = (ogrpoint *) ppntfeature-> getgeometryref (); // set the filtering attribute pplglayer-> resetreading (); pplglayer-> setspatialfilter (ogrgeometry *) Ppoint); pplgfeature = pplglayer-> getnextfeature (); if (pplgfeature = NULL) {ogrfeature: temperature (ppntfeature); continue;} ogrfeature: temperature (ppntfeature); ogrfeature: destroyfeature (pplgfeature);} ogrdatasource :: destroydatasource (ppntds); ogrdatasource: destroydatasource (pplgds );}

The following is a small function of the main function and the output time.

void ShowTime(){time_t t = time(0); char tmp[64]; strftime( tmp, sizeof(tmp), "%Y/%m/%d %X", localtime(&t) ); puts( tmp ); }int _tmain(int argc, _TCHAR* argv[]){ShowTime();SearchSampleDataFromSHP_liml();ShowTime();system("pause");return 0;}

First, let's take a look at the time when gdal1.9.0 was used, as shown in processing time 3. (Note: all of the following test times are obtained using the release version)

Figure 3 time spent using gdal1.9.0

Let's take a look at the gdal1.11.0 time, as shown in processing time 4.

Figure 4 time spent using gdal1.11.0

From the above two processing times, we can see that the processing time in gdal1.11.0 has greatly increased (100 times), So if you use the spatial index, please update the gdal version.

We know that the shapefile generally requires three files with the suffix SHP, DBF, and shx. If the data has projection information, add a prj file. This standard SHP file is commonly used. If you create it using gdal, several files will be generated. However, when you open it with ArcMAP, several files will be automatically added, with the suffix SBN and sbx. There may be an XML file. 2. These two files are the space index files automatically generated by ArcMap (ESRI spatial index files ).

According to the gdal official documentation, currently, the gdal database only supports reading spatial index files and does not support creation. Therefore, to process SHP files with large data volumes, you can use ArcMap to open a space index file and use gdal to process it. In addition, gdal supports reading and writing the quad-tree index file (. qix) used by umn mapserver ). For details, see the shapefile format page on the gdal official website (URL: http://www.gdal.org/drv_shapefile.html ).

Test SHP file index acceleration in gdal1.11