GDAL2.1 increased support for MongoDB storage space Big Data

As spatial data permeates every aspect of social life, the ability to provide services for big data needs to be enhanced. such as National Geographic conditions census data, only space vector data a province data volume in the 30GB, the image is more terabytes, the national data is even more terrible, traditional centralized storage mode if used in several departments, in more than 10 departments, thousands of service requests in the case of concurrency, unsustainable. For this reason, it is urgent to use Internet thinking to deal with spatial data in the mode of cluster concurrent processing and provide necessary statistic analysis. This article discusses how to use the MongoDB storage space Big data, starting from the most basic data warehousing storage.

1 Catalog

1.1 gdal2.1 added support for MongoDB spatial database connection

1.2 Necessary Module compilation

1.3 Using virtual machines and hosts to build a virtual LAN to build a distributed database as an experiment

1.1 GDAL2.1 characteristics

Gdal, as a bridge of spatial data in many formats, is used by many mature commercial spatial databases. A highly abstract data-driven model that makes it easier to add new data-format drivers and block data physical storage, allowing system design developers to do the necessary things for the data logic. GDAL2.1 added the MongoDB data Space module support, very good to the MongoDB spatial data storage features, distributed storage, automatic load balancing together, so that through MongoDB to expose the spatial data to the system design and development personnel, to achieve greater design freedom.

1.2

Gdal supports MongoDB compilation parameters
1, SCons
2, boost1.55
3. gdal2.1 Trunk SVN

4. Mongo-cxx-driver C + + client legacy-1.0.2 code download

1) Compile the command line:
C:\users\zjz\desktop\mongo-cxx-driver\legacy-1.0.2>scons--32--cpppath=c:\local
\boost_1_55_0--libpath=c:\local\boost_1_55_0\lib32-msvc-10.0--dynamic-windows
--sharedclient--prefix=c:\local\mongo-client-install Install

2) Compile GDAL2.1, look for comment lines in nmake.opt, as follows; use non-plug-in mode (that is, DLL dynamic call mode)

# Uncomment for MongoDB support
# This configuration was valid for a libmongoclient built as a DLL with:scons.bat- -32--ssl--dynamic-windows--sharedclient --prefix=c:\users\even\dev\mongo-client-install--cpppath=c:\users \even\dev\boost_1_55_0_32bit--libpath=c:\users\even\dev\boost_1_55_0_32bit\lib32-msvc-10.0 Install
MONGODB_ PLUGIN = NO
Mongodb_inc = C:\mongo-client-install\include
# Boost library names must be edited to reflect the ACTU Al MSVC and Boost versions
Boost_inc = C:\local\boost_1_55_0
boost_lib_path= C:\local\boost_1_55_0\ lib32-msvc-10.0
Mongodb_libs = c:/mongo-client-install/lib/mongoclient.lib  $ (BOOST_LIB_PATH) \libboost_ thread-vc100-mt-1_55.lib   $ (Boost_lib_path) \libboost_system-vc100-mt-1_55.lib  $ (BOOST_LIB_PATH) \libboost_date_time-vc100-mt-1_55.lib   $ (boost_lib_path) \libboost_chrono-vc100-mt-1_55.lib

Install VirtualBox virtual Linux system on 1.3 win7, install ubuntu32 bit
Virtual MongoDB cross-system cluster with host
Slice Storage Geodatabase Database
Read and write geodata with gdal2.1 MongoDB driver Drive
Storage of big data, such as 30GB Hubei Province geographical conditions of the census database, stored as follows, the connection in Mongovue is shown as follows:

The next chapter describes how to configure database storage in a cluster.

Related articles

1) ArcGIS plugin supports MONGODB database storage space Big Data

2) GDAL1.11.1 support PostGIS, ArcGIS module

3) Constructing a query system based on PostGIS spatial information

4) Build a distributed spatial data storage environment in a virtual machine-installed Linux system and host

GDAL2.1 increased support for MongoDB storage space Big Data