Use the map SDK to generate a Mars coordinate correction Database

As we all know, our domestic electronic maps such as Google AMAP are using the encrypted offset of the Mars coordinate (GCJ-02), but the mobile phone GPS acquisition is the earth coordinate (WGS-84 ), how to correct the Earth to Mars is a must for Chinese programmers. Generally, there are the following methods: correction algorithms, correction interfaces, and correction databases. Some things are actually public secrets, so here I will write down the process of generating a correction database, it should not be a big problem. Note that the premise of this article is that there is no correction algorithm, and you can use the SDK correction interface to generate a correction library.

Correction data can be found online, but it is not easy to find for free. Most of the data that can be found online is incomplete, or the accuracy is low. Someone on X Bao sells correction databases or correction interfaces, the price is not cheap, and so arrogant data is not shared by ordinary people. So I came up with the idea of generating a correction database through my own correction interface.

Autonavi and Baidu provide online correction interfaces, but after all, they are online network requests. If there are many coordinates, the correction speed is slow. There are about 10 million correction data records with 0.01 Accuracy in China. The size of a 40-byte correction data is 1 GB, and the size of the data after compression is 100 or 200 MB. If there are 10 corrections per second and 10 million corrections may need to be corrected for a month. Generally, the map mobile SDK (such as AMAP) also provides the coordinate correction function, but most of them call the online correction interface, which is slow. However, when I used the Baidu map SDK, I found that Baidu map also had a correction interface which was not published in the document, but existed, and its correction was completed instantly and directly, you do not need to connect to the Internet. Therefore, you can use the built-in Correction Algorithm to generate a correction database.

Baidu map SDK has Android and IOS versions. I chose IOS for the simple reason: IOS supports x86 native command mode, and the speed is naturally much faster than Android arm's Java virtual machine. The code is not complex. Basically, it is to traverse all the longitude and latitude ranges in China cyclically, convert coordinates, and calculate the offset. Baidu himself on the basis of the Mars coordinate made a secondary encryption to form their own Baidu coordinate system (BD-09), support from the earth coordinate (WGS-84) to Baidu coordinate (BD-09 ), but it also supports switching from the Mars coordinate (GCJ-02) to the Baidu coordinate (BD-09), so we can calculate the Mars coordinate (GCJ-02) through the subtraction of the two ).

As a result, I wrote a correction library Generation Program that supports generating correction data of different precision types. The main code is as follows:

-(Void) Dogen {If (isrunning) // prevent repeated access to return; isrunning = yes; // obtain the file storage path and generate the file name nsarray * documentspaths = nssearchpathfordirectoriesindomains, yes); nsstring * fn = [nsstring stringwithformat: @ "coordoffset_db_mongod_mongod.txt", gentype, stepdis]; fn = [[documentspaths objectatindex: 0] stringbyappendingpathcomponent: FN] nslog (@ "gen filename: % @", FN); // open the file using the C function to write a large file * H File = fopen ([FN utf8string], "W +"); If (hfile = NULL) {isrunning = no; return ;}// use a custom memory pool, regularly release the memory to prevent the memory in a large loop from being insufficient. The * Pool = [[NSAID utoreleasepool alloc] init]; curcount = 0; int cchcount = 5000; Response coor1, coor2, coor3, coor4; char * Buf = (char *) malloc (101*1024); // apply for a 101kb memory as the write cache int idx = 0; For (int ix = startlon; IX <= endlon; ix + = stepdis) {for (INT Iy = startlat; Iy <= endlat; Iy + = stepdi S) {coor1.longpolling = IX/1000000.0; coor1.latitude = Iy/1000000.0; If (gentype = 0) {// determine the Conversion Type // Baidu, directly convert nsdictionary * Dist = bmkbaiducoorforwgs84 (coor1); coor2 = bmkcoordictionarydecode (DIST);} else {// Google autonavi, first convert WGS to Baidu, then Google autonavi to Baidu, then calculate the approximate offset and re-calculate nsdictionary * Dist = Hangzhou (coor1); coor2 = bmkcoordictionarydecode (DIST); Dist = Hangzhou (coor1); coor3 = bmkcoordictionarydecode (Dist ); double dx1 = coo R3.longbench-coor1.longbench; double dy1 = coor3.latitude-coor1.latitude; double dx2 = coor2.longbench-coor1.longbench; double dy2 = coor2.latitude-coor1.latitude; double dx3 = dx2-dx1; double dy3 = dy2-dy1; // calculate the gaode coordinate which may be closest to the original coordinate correction result. recalculate coor3.long1_= coor1.long1_+ dx3; coor3.latitude = coor1.latitude + dy3; Dist = bmkbaiducoorforgcj (coor3 ); coor4 = bmkcoordictionarydecode (DIST); dx1 = coor4.longbench-coor3.longit Ude; dy1 = coor4.latitude-coor3.latitude; dx3 = dx2-dx1; dy3 = dy2-dy1; coor2.longbench = coor1.longbench + dx3; coor2.latitude = coor1.latitude + dy3 ;} // write the buffer int Len = sprintf (& Buf [idx], "% 0.3f, % 0.3f, % 0.6f, % 0.6f \ n", coor1.long1_, coor1.latitude, coor2.long1_- coor1.long1, coor2.latitude-coor1.latitude); idx + = Len; cchcount ++; curcount ++; If (cchcount> 5000 | idx> = 100*1024 | curcount> = reccount) {// every 5000 coordinates, or buffer When KB is used or the last record is reached, write the fwrite (BUF, idx, 1, hfile) file; idx = 0; cchcount = 0; // release the memory pool, and re-create the memory pool [pool release]; pool = [[NSAID utoreleasepool alloc] init]; // record the current coordinates for refreshing progress using curlon = IX/1000000.0; curlat = Iy/1000000.0;} If (! Isrunning) break;} If (! Isrunning) break;} Free (BUF); [pool release]; fclose (hfile); isrunning = no ;}

The preceding code is executed in the thread. To prevent the memory pool from being released in a large loop from being insufficient, you need to create a memory pool and release the memory regularly. To support writing large files, we need to use the file operation function of C to replace nsdata. To ensure the write speed, I use the write buffer, which is written every 5000 rows. I thought it would take half a day to execute the generation process. I also wrote a timer to refresh the progress to estimate the completion time. After such optimization, the generation speed is faster, and 30 million records can be generated in about ten minutes, this is far beyond my expectation.

Considering the actual needs, I generated about 0.025 pieces of data with a precision of 5 million. Run the task directly in the simulator. After running the task, find the generated file in the finder and copy it to the database.

I use Oracle for databases. I use SQL loader for import. Before import, I delete all the index constraints and expand the tablespace restrictions. Every 5000 commit entries are used. The local machine is connected to the server Gigabit, the import speed is about 1500 entries per second, and it takes more than half an hour to complete the import.

Create an index after import. Considering that floating point index efficiency may not be high, I converted the longitude and latitude into an integer, for example, 113.05 to 113050000, 23.05 to 23050000, and then created a joint index. It takes 6 or 7 seconds to query a coordinate after the index is created. Then, take six digits for the longitude and latitude to merge them into a 12-bit unique primary key index field, for example, 113.05 and 23.05 to 113050023050. After the primary key index is created, the query SQL will return instantly.

Now, the correction database has been created, and you can directly call the SQL query.