MongoDB Study Notes-aggregation

MongoDB Study Notes-aggregation

The latest new project to use mysql has not forgotten yet. The summary records the following MongoDB aggregation.

AggregationIt refers to various operations that can process batch records and return computing results. MongoDB provides a wide range of aggregate operations for computing datasets. InMongodPerforming aggregation on instances can greatly simplify application code and reduce resource consumption.

In MongoDB, aggregation operations such as query are all usedSetAs input, the final result will output one or more documents.

Aggregate MPs queue

An aggregate pipeline is a framework based on the data processing pipeline concept. Use a multi-stage pipeline to convert a set of documents to the final aggregate results. (Aggregate details)

An aggregate pipeline is an alternative to map simplification. In addition, for aggregate tasks, aggregate pipelines are a preferred solution, because the complexity of ing simplification may not be guaranteed.

Each step of the aggregation pipeline can use a maximum of 100 MB of memory. If a step exceeds this limit, MongoDB reports an error. To process a large amount of data, you can useAllowDiskUseIn this case, the MPs queue writes data to a temporary file.

Some MPs queue stages can use MPs queue expressions as operators. The MPs queue expression can be used to convert the input file. Pipeline expressions use a document struct and can contain other expressions.

Pipeline expressions can only operate documents in the current pipeline, but cannot access other documents: expression operations can complete the conversion of documents in the memory.

In general, expressions are stateless and only in the computing state during aggregation, except for the cumulative operator expressions.

Use$ GroupYou must maintain the state of the operator (such as the total number, maximum value, minimum value, and related data) when the pipeline processes the document ).

Ing simplification

Ing simplification is a data processing method that converts a large amount of data into valuable * Aggregated * results. In MongoDB, useMapReduceCommand to perform the ing simplification operation.

See the following ing simplification operation:

In this ing simplification operation, MongoDB performs the * map * operation on each input document (for example, the document that meets the query conditions in the collection. The ing operation outputs the key-Value Pair results. For those keywords with multiple values, MongoDB executes the * reduce * operation to collect and compress the final aggregation result. Then, MongoDB saves the result to a collection. The simplification function can also output the result to the * finalize * function to further process the aggregation result. Of course, this step is optional.

In MongoDB, all ing simplification functions are written in JavaScript and run inMongodIn process. The ing simplification operation uses a collection of documents as * input * and can execute any sorting and limiting operations before the ing phase.MapReduceThe command can return the result as a document or write the result to the set. The input and output sets can be sharded.

If you select ing simplification to return results instantly, these documents must be inBSON document sizeThe current limit is 16 MB.

Single-purpose Aggregation

Aggregation refers to a type of operations performed on datasets. These methods perform specific steps on the input data to calculate a result. MongoDB provides a set of aggregation methods to perform specific operations on a dataset.

Although they are used in a limited range compared with aggregate pipelines and ing simplification, the names of these methods intuitively express their functions and are very easy to understand and use.

1) Total

MongoDB returns the total number of documents that meet the query conditions. BesidesCountCommand,MongoIn the script programCount ()Method andCursor. count ()You can obtain the total number of documents.

Example

Now there isRecords.OnlyThe following documents:

{ a: 1, b: 0 }{ a: 1, b: 1 }{ a: 1, b: 4 }{ a: 2, b: 2 }

The following Operation counts the number of documents in the set and returns a number.4:

db.records.count()

The following operations will count FieldsAThe value is1Number of documents, and finally return3:

db.records.count({a: 1})

2) remove duplicates

Remove duplicatesThe operation returns records with no repeated queried field values. InMongoIn the script program, useDistinctCommand orDb. collection. distinct ()Method to remove duplicates. See the following example to remove duplicates:

Example

Now there isRecords.OnlyThe following documents:

{ a: 1, b: 0 }{ a: 1, b: 1 }{ a: 1, b: 1 }{ a: 1, b: 4 }{ a: 2, b: 2 }{ a: 2, b: 2 }

See the followingDb. collection. distinct ()Method pair FieldBTo remove duplicates:

db.records.distinct("b")

The result of this operation is:

[ 0, 1, 4, 2 ]

3) Group

GroupThe operation groups the queried documents according to the specified field values. The grouping operation returns an array of documents, each of which contains the computing results of a group of documents.

You canMongoIn the script programGroupCommand orDb. collection. group ()To use the grouping function.

GroupThe command cannot be run on the shard set. Note that,GroupThe size of the result set cannot exceed 16 MB.

Example

Now there isRecordsContains the following documents:

{ a: 1, count: 4 }{ a: 1, count: 2 }{ a: 1, count: 4 }{ a: 2, count: 3 }{ a: 2, count: 1 }{ a: 1, count: 5 }{ a: 4, count: 4 }

Please consider usingGroupCommand to group the documents in the set. The grouping condition is FieldAValue less3For each groupCountField count:

db.records.group({   key: {a: 1},   cond: {a: {$lt: 3}},   reduce: function(cur, result) { result.count += cur.count },   initial: {count: 0}})

The result of this grouping operation is:

[  {a: 1, count: 15},  {a: 2, count: 4}] 

For more information about MongoDB, see the following links:

MongoDB 3.0 official version released and downloaded

CentOS compilation and installation of MongoDB

CentOS compilation and installation of php extensions for MongoDB and mongoDB

CentOS 6 install MongoDB and server configuration using yum

Install MongoDB2.4.3 in Ubuntu 13.04

MongoDB beginners must read (both concepts and practices)

MongoDB Installation Guide for Ubunu 14.04

MongoDB authoritative Guide (The Definitive Guide) in English [PDF]

Nagios monitoring MongoDB sharded cluster service practice

Build MongoDB Service Based on CentOS 6.5 Operating System

MongoDB details: click here
MongoDB: click here

This article permanently updates the link address: