MongoDB Learning and Finishing (next)

Aggregation Aggregate

• Aggregation (aggregate) is primarily used to calculate data, similar to sum (), AVG () in SQL
• Grammar

Db. Collection name. Aggregate ([{pipe: {expression}}])

Piping

• Pipelines are typically used in UNIX and Linux to input the output of the current command as the next command

PS grep MONGO

• In MongoDB, pipelines have the same effect, and after the document is processed, the next processing is done through the pipeline
• Common piping
• $group: Grouping documents in a collection for statistical results
• $match: Filter data to only output documents that match the criteria
• $project: Modify the structure of the input document, such as renaming, adding, deleting fields, creating calculation results
• $sort: Sort the input document after output
• $limit: Limit the number of documents returned by the aggregation pipeline
• $skip: Skips a specified number of documents and returns the remaining documents
• $unwind: Splitting a field of an array type

An expression

• Process the input document and output
• Grammar

Expression: ' $ column name '

• Common expressions
• $sum: Calculate sum, $sum: 1 count with Count
• $avg: Calculate average
• $min: Get the minimum value
• $max: Get maximum value
• $push: Inserting values into an array in the resulting document
• $first: Get the first document data based on the ordering of the resource documents
• $last: Get the last document data based on the ordering of the resource documents

$group

• Group documents in a collection for statistical results
• _ID represents the basis for grouping, using a field in the format ' $ field '
• Example 1: Statistics on the total number of boys and girls

db.stu.aggregate ([    {$group:        {            _id:'$gender',            counter:{ $sum:1}        }    ])

Group by null

• Divides all the documents in the collection into a group
• Example 2: Total number of students, average age

db.stu.aggregate ([    {$group:        {            _id:null,            counter:{$sum: 1 },            avgage:{$avg:'$age'}}    ])

Pivot Data

• Example 3: Statistics on students ' gender and student names

db.stu.aggregate ([    {$group:        {            _id:'$gender',            name:{$ Push:'$name'}}    ])

• Use the $ $ROOT to add the contents of the document to the array of result sets, as shown in the following code

Db.stu.aggregate ([    {$group:        {            _id: ' $gender ',            name:{$push: ' $ $ROOT '}        }    } ])

$match

• For filtering data, only output documents that match the criteria
• Standard query operations using MONGODB
• Example 1: Querying for students older than 20

Db.stu.aggregate ([    {$match: {age:{$gt:}}])

• Example 2: Query the number of boys and girls older than 20

Db.stu.aggregate ([    {$match: {age:{$gt: $}}},    {$group: {_id: ' $gender ', counter:{$sum: 1}}} ])

$project

• Modify the structure of the input document, such as renaming, adding, deleting fields, creating calculation results
• Example 1: Query the student's name, age

Db.stu.aggregate ([    {$project: {_id:0,name:1,age:1}}])

• Example 2: Query the number of boys and girls, the number of output

Db.stu.aggregate ([    {$group: {_id: ' $gender ', counter:{$sum: 1}},    {$project: {_id:0,counter:1} }])

$sort

• Sort the input document after output
• Example 1: Querying student information, ascending by age

B.stu.aggregate ([{$sort: {age:1}}])

• Example 2: Query the number of boys and girls, descending by number

Db.stu.aggregate ([    {$group: {_id: ' $gender ', counter:{$sum: 1}}},    {$sort: {counter:-1}} ])

$limit

• Limit the number of documents returned by the aggregation pipeline
• Example 1: Query 2 Student Information

Db.stu.aggregate ([{$limit: 2}])

$skip

• Skips a specified number of documents and returns the remaining documents
• Example 2: Querying student information starting with 3rd

Db.stu.aggregate ([{$skip: 2}])

• Example 3: Statistics on the number of boys and girls, according to the number of ascending, take the second data

Db.stu.aggregate ([    {$group: {_id: ' $gender ', counter:{$sum: 1}},    {$sort: {counter:1}},    {$skip: 1},    {$limit: 1} ])

• Note The order: write skip first, then write limit

$unwind

• Splits one of the array type fields in a document into multiple bars, each containing a value from the array

Grammar 1

• To split a field value

Db. Collection name. Aggregate ([{$unwind: ' $ field name '}])

• Construct data

Db.t2. Insert ({_id:1, item:'t-shirt', size:[' S ', ' M ', ' L '] })

• Inquire

Db.t2.aggregate ([{$unwind: ' $size '}])

Grammar 2

• To split a field value
• Handling empty arrays, non-arrays, no fields, NULL cases

Db.inventory.aggregate ([{    $unwind: {        path: ' $ field name ',        preservenullandemptyarrays:< Boolean> #防止数据丢失    }}])

• Construct data

Db.t3. Insert ([{"_id": 1, "Item": "A", "Size": ["S", "M", "L"]2[]
   
    3
    
    4
    
    5
    null 
     }])
   

• Querying using Syntax 1

Db.t3.aggregate ([{$unwind: ' $size '}])

• View the results of the query and find that the empty array, field-free, null documents are discarded
• Q: How can I not discard it?
• A: Query using Syntax 2

Db.t3.aggregate ([{$unwind: {path: ' $sizes ', Preservenullandemptyarrays:true}}])

Super Admin

• For more secure access to MongoDB, visitors are required to provide a user name and password, so users need to be created in MongoDB
• Using the role-user-database security management method
• Common system roles are as follows:
• Root: Only available in the Admin database, super account, super privilege
• READ: Allows the user to read the specified database
• ReadWrite: Allows the user to read and write to the specified database
• Create a Super Admin user

 Use Admindb.createuser ({    user:'admin',    pwd:'  123',    roles:[{role: ' Root ', db: ' admin '}] })

Enable secure authentication

• Modifying a configuration file

sudo VI /etc/mongod.conf

• Enable authentication
• Note: Keys and Values Be sure to add a space between , Otherwise the resolution will error

Security

Authorization:enabled

• Restart Service

sudo Service Mongod Stop sudo service mongod start

• Terminal connection

' Admin ' ' 123 ' ' Admin '

General user Management

• Log in with Super Administrator and then go to user management action
• View users of the current database

 Use Test1show Users

• Create a normal user

Db.createuser ({    user:'t1',    pwd:' 123 ' ,    roles:[{role: ' ReadWrite ', db: ' Test1 '}]})

• Terminal connection

123 --authenticationdatabase test1

• Switch the database, execute the command to see the effect
• Modify User: Can modify PWD, roles properties

Db.updateuser ('t1', {pwd:'456'})

replication (replica set) What is replication

• Replication provides redundant backups of data, stores copies of data on multiple servers, improves data availability, and ensures data security
• Replication also allows recovery of data from hardware failures and service outages

Why do you want to copy

• Data backup
• Data disaster recovery
• Read/write separation
• High (24* 7) data availability
• No Downtime Maintenance
• Replica sets are transparent to applications

How replication works

• Replication requires a minimum of two nodes A, B ...
• A is the master node responsible for processing client requests
• The rest is from the node, which is responsible for replicating the data on the master node
• The common collocation methods of nodes are: one master, one from, one main and many from
• The master node records all operations on it, obtains these operations from the node periodically polling the master node, and then performs these operations on its own copy of the data, ensuring that the data from the node is consistent with the primary node
• Data interaction between primary node and slave node ensures data consistency

Features of replication

• N-node clusters
• Any node can be used as the master node
• All write operations are on the primary node
• Auto Fail-Over
• Automatic recovery

Setting Up Replication Nodes

• The next operation will need to open multiple terminal windows, and may be connected to more than one Ubuntu host, it will appear a bit messy, it is recommended to implement in Xshell
• Step1: Create database directory T1, t2
• Demo in Desktop directory, other directories can also, pay attention to the permissions can be

mkdir T1 mkdir T2

• Step2: Start Mongod with the following format, note that the name of Replset is consistent

192.168. 196.128 27017 --dbpath ~/desktop/t1--192.168. 196.128 27018 --dbpath ~/desktop/t2--replset rs0

• Step3: Connect to master server, set 192.168.196.128:27017 as primary server here

192.168. 196.128 27017

• STEP4: Initialization

Rs.initiate ()

• After initialization is complete, the prompt is as follows:

• Step5: View current status

Rs.status ()

• Current state such as:

• STEP6: Adding replica Sets

Rs.add ('192.168.196.128:27018')

• STEP7: After the replica set has been added successfully, the current state is as follows:

• STEP8: Connecting a second MONGO service

192.168. 196.128 27018

• After the connection is successful, the prompt is as follows:

• STEP9: Inserting data into the primary server

Use test1  for (i=0;i<; i++) {Db.t1.insert ({_id:i})}db.t1. Find ()

• STEP10: Inserting queries from the server
• Note: If you are reading from a server, you need to set Rs.slaveok ()

Rs.slaveok () db.t1. Find ()

Other Instructions

• Remove from node

Rs.remove ('192.168.196.128:27018')

• After shutting down the primary server and restarting, you will find that the original slave server has changed from the server, the newly started server (the original slave server) to the slave server

Backup

• Grammar

Mongodump-h dbhost-d Dbname-o dbdirectory

• -H: Server address, or port number can be specified
• -D: The name of the database that needs to be backed up
• -O: The data storage location of the backup, which contains the data backed up
• Example 1

sudo mkdir Test1bak

sudo mongodump-h 192.168.196.128:27017-d test1-o ~/desktop/test1bak

Recovery

• Grammar

Mongorestore-h dbhost-d dbname--dir dbdirectory

• -H: Server address
• -D: DB instance that needs to be recovered
• --dir: Where to back up your data
• Example 2

192.168. 196.128:27017 -D test2--dir ~/desktop/test1bak/test1

Interacting with Python

• Click to view official documents
• Install Python Packages
• Introducing Package Pymongo
• Connect, create Client
• Get Database Test1
• Get Set Stu
• Add a document
• Find a document
• Finding multiple Documents 1
• Finding Multiple Documents 2
• Get the number of documents

进入虚拟环境

sudo Install Pymongo

或源码安装

Python setup.py

Importpymongoclient=pymongo. Mongoclient ("localhost", 27017) DB=Client.test1stu=Db.stus1={name:'GJ', age:18}s1_id=Stu.insert_one (S1). Inserted_ids2=Stu.find_one () forCurinchstu.find ():PrintCurcur=Stu.find () Cur.next () Cur.next () Cur.next ( )PrintStu.count ()

MongoDB Learning and Finishing (next)