In the previous two articles, the basic aggregation functions count, distinct, group, and MapReduce for data aggregation in Mongodb, we have provided two implementation methods for data aggregation, today, in this article, we will talk about another data aggregation implementation method in Mongodb-aggregate pipeline aggregate. Face
In the previous two articles, the basic aggregation functions count, distinct, group, and MapReduce for data aggregation in Mongodb, we have provided two implementation methods for data aggregation, today, in this article, we will talk about another data aggregation implementation method in Mongodb-aggregate pipeline aggregate. Face
In the previous two articles And We have already provided two implementation methods for data aggregation. today, in this article, we will talk about another data aggregation implementation method in Mongodb-aggregate pipeline aggregate.
In the face of the needs of users for data statistics, Mongodb has introduced a new function aggregation framework (aggregation framework) since version 2.2. it is a new framework for data aggregation, this concept is similar to pipelines in data processing. Each document uses a pipeline composed of multiple nodes. each node has its own special functions (grouping, filtering, etc ), the output result is obtained after the document passes through a pipeline composed of multiple nodes. The basic functions of MPs queue are as follows: (1) filter documents and filter documents that meet the conditions; (2) modify documents to change the output structure of documents.
Usage of aggregate pipelines: db. collection. aggregate ();
You can use the following pipeline operators for multiple nodes in the pipeline to describe the functions of various pipeline operators:
$ Project: modify the structure of a document (rename, add, or delete a domain). It can also be used to create computing results and nested documents.
$ Match: filters data and outputs only documents that meet the conditions.
$ Limit: limit the number of documents returned by the MongoDB aggregation pipeline.
$ Skip: skip a specified number of documents in an aggregation pipeline.
$ Unwind: splits an array field in the document into multiple entries, each containing a value in the array.
$ Group: groups documents in the set for statistical results.
$ Sort: document sorting output.
$ GeoNear: outputs ordered documents close to a certain geographical location.
Here are two simple examples:
db.article.aggregate( { $project : { title : 1 , author : 1 , }} );
db.articles.aggregate( [ { $match : { score : { $gt : 70, $lte : 90 } } }, { $group: { _id: user, count: { $sum: 1 } } } ] );
The following sections describe the notes for using an aggregation pipeline:
(1) pipelines are sequential.
(2) $ group operations are currently processed in the memory. Therefore, you cannot use this method to group a large number of documents;
(3) when using $ unwind to split the field values in the array, note that you must not forget to write the $ symbol, for example, {$ unwind: "$ tags "}, there is a $ sign before the tags field;
(4) MongoDB 24. the memory is optimized. if $ sort appears before $ limit, $ sort will only operate on the previous $ limit documents, and only the previous $ limit documents will be retained in the memory, memory saved
(5) $ sort is performed in the memory. if the memory occupied exceeds 10% of the physical memory, the program will produce an error.
(6) the output result of the MPs queue cannot exceed 16 MB. an error occurs when the output result exceeds 16 MB.
(7) If the memory occupied by an MPs queue operator exceeds 10% of the system memory capacity during execution, an error is reported;
(8) aggregation pipelines provide good performance and consistent interfaces. they are easy to use and can be used for some simple and fixed aggregation operations, however, MapReduce is still used for aggregation tasks of complex and massive datasets.
So far, a simple description of data aggregation operations in the Mongodb database is over. if you want to learn more, I think the official website is the best teaching material,