Aggregate pipelines aggregate for data aggregation in Mongodb

Source: Internet
Author: User
In the previous two articles, the basic aggregation functions count, distinct, group, and MapReduce for data aggregation in Mongodb, we have provided two implementation methods for data aggregation, today, in this article, we will talk about another data aggregation implementation method in Mongodb-aggregate pipeline aggregate. Face

In the previous two articles, the basic aggregation functions count, distinct, group, and MapReduce for data aggregation in Mongodb, we have provided two implementation methods for data aggregation, today, in this article, we will talk about another data aggregation implementation method in Mongodb-aggregate pipeline aggregate. Face

In the previous two articles And We have already provided two implementation methods for data aggregation. today, in this article, we will talk about another data aggregation implementation method in Mongodb-aggregate pipeline aggregate.


In the face of the needs of users for data statistics, Mongodb has introduced a new function aggregation framework (aggregation framework) since version 2.2. it is a new framework for data aggregation, this concept is similar to pipelines in data processing. Each document uses a pipeline composed of multiple nodes. each node has its own special functions (grouping, filtering, etc ), the output result is obtained after the document passes through a pipeline composed of multiple nodes. The basic functions of MPs queue are as follows: (1) filter documents and filter documents that meet the conditions; (2) modify documents to change the output structure of documents.


Usage of aggregate pipelines: db. collection. aggregate ();

You can use the following pipeline operators for multiple nodes in the pipeline to describe the functions of various pipeline operators:

$ Project: modify the structure of a document (rename, add, or delete a domain). It can also be used to create computing results and nested documents.
$ Match: filters data and outputs only documents that meet the conditions.
$ Limit: limit the number of documents returned by the MongoDB aggregation pipeline.
$ Skip: skip a specified number of documents in an aggregation pipeline.
$ Unwind: splits an array field in the document into multiple entries, each containing a value in the array.
$ Group: groups documents in the set for statistical results.
$ Sort: document sorting output.
$ GeoNear: outputs ordered documents close to a certain geographical location.


Here are two simple examples:

db.article.aggregate(    { $project : {        title : 1 ,        author : 1 ,    }} );

db.articles.aggregate( [                        { $match : { score : { $gt : 70, $lte : 90 } } },                        { $group: { _id: user, count: { $sum: 1 } } }                       ] );

The following sections describe the notes for using an aggregation pipeline:

(1) pipelines are sequential.
(2) $ group operations are currently processed in the memory. Therefore, you cannot use this method to group a large number of documents;
(3) when using $ unwind to split the field values in the array, note that you must not forget to write the $ symbol, for example, {$ unwind: "$ tags "}, there is a $ sign before the tags field;
(4) MongoDB 24. the memory is optimized. if $ sort appears before $ limit, $ sort will only operate on the previous $ limit documents, and only the previous $ limit documents will be retained in the memory, memory saved
(5) $ sort is performed in the memory. if the memory occupied exceeds 10% of the physical memory, the program will produce an error.

(6) the output result of the MPs queue cannot exceed 16 MB. an error occurs when the output result exceeds 16 MB.
(7) If the memory occupied by an MPs queue operator exceeds 10% of the system memory capacity during execution, an error is reported;
(8) aggregation pipelines provide good performance and consistent interfaces. they are easy to use and can be used for some simple and fixed aggregation operations, however, MapReduce is still used for aggregation tasks of complex and massive datasets.

So far, a simple description of data aggregation operations in the Mongodb database is over. if you want to learn more, I think the official website is the best teaching material,

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.