Using MongoDB to develop LBS application practice "turn"

Recently, as a commando, with colleagues in a blitz to build a simple lbs system. The current mainstream approach is to use MongoDB, because it has encapsulated the common LBS basic operations (such as looking for nearby people), the function is very powerful, for the development cycle only a week of projects, MongoDB really is the savior, the most important needs to complete, thank goodness!

MongoDB is the more famous NoSQL db, want to know the classmate may ask Google or degree Niang, for professional problems I tend to ask Google. The article also references this article. We are using the current version of mongodb-2.4.9.

First, from the LBS feature provided by MongoDB: MongoDB supports the following types of queries:

1. Intra-region search: the so-called intra-region search, which lists all records in a certain area nearby, such as "nearby" in the Baidu map. MongoDB implements such queries with the $geowithin command, which is powerful in supporting rectangular areas ($box), circular regions ($center), and Polygon regions ($polygon) queries! As an example of a circular area query, you only need to give the latitude and longitude of the center and the query radius, you can take all the records in that region, but the results are unordered.

2. Find nearby: Find records near a nearby location that are returned in order from near to far, such as many social apps looking for people nearby. MongoDB uses the $near command to enter the coordinates of the center point to return results, and the result is sorted by distance from small to large. $near interface is a step more than $geowithin, but the query efficiency is basically the same for the 2-d coordinate index.

Another command is $geonear, which can be considered an upgraded version of $near that returns distances and other diagnostic information in addition to returning records. Our application needs to calculate the distance and want to use this command from the beginning, but its use is very inconvenient. Strangely, this command cannot be integrated with the find command like $near and $geowithin, and it is not flexible enough to meet our application needs.

Second, the distance calculation.

There are two common types of MongoDB geolocation indexes: 1) 2d planar coordinate index, which is suitable for plane-based distance calculation. 2) 2dsphere Geometric sphere index, suitable for spherical geometry distance operation. The pursuit of absolute precision, should choose 2dsphere. However, when the coordinate span is not too large (for example, within 2000 km), the distance difference calculated by these two indexes is almost negligible. About MONGODB implementation of the 2-dimensional geographical index, you can refer to this article, write easy to understand.

Next talk about the use of composite indexes encountered in practice. Because MongoDB is an open source project, there will always be some imperfections, and these problems can only be encountered when used.

A composite index is an index of multiple fields, assuming that records in a MONGO collection (collection) contain a and B two fields, we want to query a record with a greater than 0 and b greater than 100, obviously a, B compound index {"A": 1, "B": 1} can effectively improve the efficiency of such queries , MongoDB has done it. The query is described in MONGO's language as: Db.posts.find ({"A": {$gt: 0}, "B": {$gt: 100}}). But if we're going to get a record of a greater than 0 and sort by B in ascending order, the federated index you just created won't work. This query is described in MONGO's language as: Db.posts.find ({"A": {$gt: 0}}). Sort ({"B": 1}).

And even limiting the total number of records to be taken will not improve query efficiency. For example, I assume that there are 2000 records for the compound condition, and I only fetch 100, which is Db.posts.find ({"A": {$gt: 0}}). Sort ({"B": 1}). Limit (100). As I understand it, if the composite index works, either the limit or not should be fast. But the test found that the time of the query and the number of records matching the conditions are proportional, that is, 2000 fetch 100 to 50ms, and 1000 take 100 is as long as 30ms, it seems that no matter how much limit, MONGO will all meet the Find condition record all read to memory, Sort according to the criteria of sort. This sort is precisely the bottleneck of performance, if the removal of the sorting, 2000 take 100 also as long as about 10ms, query A is greater than 0 and B is more than 100 records are as long as 100ms.

According to the analysis of the foreigner, this is Mogodb current bug, should be in version 2.6 fix off, hope so. There is no way to optimize the problem at the moment, but only when find has more constraints to reduce the number of records being sorted, but it is often not possible to do so in the face of actual demand.

So, the lesson learned from this project is that open source projects do have very powerful features that help us solve most of the underlying problems and often give us a lot of surprises (such as MONGO's strong and comprehensive support for geo-information queries). However, the so-called "no free Lunch", the use of open source projects not only need to learn how to use, the use of the process will also encounter some pits, the need to pay tuition fees, so, always maintain a cautious and critical attitude.

Using MongoDB to develop LBS application practice "turn"