MongoDB Learning Notes

A.
MongoDB features, features, usage scenarios
MongoDB Installation and Configuration
MongoDB Architecture
Database start, stop, connect
Json/bson data format
Mongodb
1. Features
Data is in memory, written to disk through a background thread
Non-modal structure for data storage----> such as product data, different product descriptions are not the same, then if the relational database is difficult to handle
can be inserted in real time (write performance is very good)
Automatic Fragment Processing
The stored data format is in binary JSON format ===>bson
Supports binary data and large objects (picture)
2. Usage Scenarios
Website Data Mongodb is very suitable for real-time insertion, updating, querying, and with the site of real-time data storage required for replication and high scalability
Caching: Caching layers for the role information data
Large-size, low-value data
High scalability, suitable for hundreds of servers composed of database clusters have, MapReduce engine built-in support.
3. Scenario Not applicable
. Highly transactional systems such as banking, accounting
Traditional Business intelligence applications
Unable to control the order in which data is written to disk
Installation of 4.Mongodb
Windows edition: Unzip, go to the Bin directory (if you set the environment variable does not matter that directory)
Set up the Data folder in the MongoDB directory, specify the directory---> can be a different directory
Start command Mongod--dbpath=d:\mongodb\data
Client Connection: MONGO
Linux version:
TAR-ZXVF mongodb-linux-i686-2.6.3.tgz
Create a log, Data folder
Start command:./mongod-dbpath=/usr/nosqlsoft/mongodb/db-logpath=/usr/nosqlsoft/mongodb/logs/mongo.log--fork
Client connections:./mongo

MongoDB boot can have some other parameters
DBPath
LogPath error log file
Logappend error log with append mode
BIND_IP external service bound IP, generally set to null
Port Ports for external service (do not want to use the default)
Fork the platform to run the service in a daemon manner
Maxconns
Other.......

5. Architecture
MONGODB is a portable database that is exactly the same on different platforms, the logical structure of data and the way data is stored, and so on.
Relational database is the concept of DB in database MongoDB
relational database--table Mongodb--->collection (collection)
relational database---> Record Mongodb--->document (document)

Data storage structure
If there is a database foo database file There will be foo.ns,foo.0,foo.1,foo.2 and so on

MongoDB has a mechanism for pre-allocated space, each pre-allocated file is populated with 0-in
MongoDB will always keep extra space and spare data files, to avoid the data explosion caused by the disk pressure to write too much problem
data file every new assignment, It will be twice times the size of the previous data file. Maximum 2G
Database each table (collection) corresponds to a namespace, which is concentrated in the. ns file
6. data format binary JSON = = "Bson format
{" 300001 ": {name:" Shanghai Composite ", Yes:3000.0,tod : 2990.1,now:3002.73}, "000001": {name: "Pune Bank", yes:23.22,tod:23.5,now:23.0}}
JSON format type performs very fast and is not expensive
object represented by JS
can pass information across domains

------------------------------------------------------------------------------------------- ------------------
B.
Bson
MongoDB Additions and deletions
MongoDB client GUI tool
MongoDB basic Operation
1.bson binary JSON
2. Basic Operation
Close service: can be directly CTRL + C shutdown
or access the Admin database use admin
Db.shutdownserver ()
2.1 Insert record operation
J = {name: "Zhangsan" , age:30};
Db.mydb.save (j)
T = {name: "Lisi", age:32},
Db.mydb.insert (j)
Note: You do not need to create a collection in advance, which is created automatically the first time you insert data.
The document can store any structure of data, the actual application of the majority of the same document collection, this is more flexible, the structure is not okay
each time the data is inserted in the collection will have an ID, the property name is _id
Write a loop insert data:
for (var i = 1; I < 10;i++) Db.mydb.save ({x:3,j:i})

Db.mydb.find ()
If the screen is not fully displayed, you can display the next screen using it commands
2.2 "_id"
Each document stored in MongoDB will have a default primary key _id, with a fixed name
Can be of any type, default is Objectid, must be unique
If you do not use this type, you must display the write Db.mydb.insert ({"_id": "0001", Name: "Zhangsan"})

2.3 Query Operations
Db.mydb.find ()
var cursor = Db.mydb.find ();
while (Cursor.hasnext ()) Printjson (Cursor.next ())
===============================
Similar to JS script
Db.mydb.find (). ForEach (Printjson);

================
In the MongoDB script
var cursor = Db.mydb.find ()
can manipulate the cursor as an array to operate
Printjson (Cursor[0])
Printjson (cursor[1]);
======================
var arr = Db.mydb.find (). ToArray ();
Arr[0];
Arr[1];
Note: These features are only in MongoDB Shell, and not all other applications support it.
The Mongodb cursor object is not without a snapshot, if there are other users in the collection for the first time the latter call next (), you may not get the data in the cursor, so to explicitly lock the cursor you query
2.4 Simple conditional Query
Db.mydb.find ({x : 4}). foreach (Printjson)
Db.mydb.find ({x:4,j:90}). foreach (Printjson);
2.5 findOne () syntax
returns only the first data in the cursor, or return null is the null data
Printjson (Db.mydb.findOne ({x:4}));
Printjson (Db.mydb.findOne ({x:1000})); 1000 does not exist return null
2.6 Limits the number of result sets by limit
Db.mydb.find (). limit (3);
2.7 Modify record
Db.mydb.update ({name: "Zhangsan"},{$set: {name: " Zhangsan_new "}})

Db.mydb.update ({attribute 1: "Value", Property 2: "Value" ...},{$set: {property: New Value, property: New Value}})
3. Common tool Sets
Bsondump: Dump files in Bson format into JSON format data
MONGO: The client command line tool, in fact, is also a JS interpreter, support JS syntax
Mongod: Database Server
Mongodump/mongorestore database backup and recovery tools
Mongoexport/mongoimport Data import and Export tool
Mongofiles:gridfs management tool for binary file access
MONGOs: Shard Routing, if the sharding feature is used, the application connects MONGOs instead of Mongod
Mongostat: Real-time performance monitoring tool
4. Client GUI
Mongovue http://www.mongovue.com Visualization Tool
PHP Writing Tools Rockmongo http://code.google.com/p/rock-php/
Mongohub Graphics Management interface specifically for Mac systems

--------------------------------------------------------------------------------------------------------------- ----------------
C.
Mongodb Advanced Query
1. Conditional operators
<, <=,>,>=
Db.collection.find ({"field": {$gt: Value}})//greater Than
Db.mydb.find ({"Age": {$gt: 30}});
Db.collection.find ({"field": {$lt: Value}})//less than
Db.collection.find ({"field": {$gte: Value}})//greater than or equal to
Db.collection.find ({"field": {$lte: Value}})//less than or equal to
Multi-criteria
Db.collection.find ({"field1": {$gt: value1, $lt: value2}})
Db.mydb.find ({"Age": {$gte: +, $lte: 32}});
Db.mydb.find ({"Age": {$gte: +, $lte: +}, "name": "Lisi"});
2. $all Match All
This operation conforms to the SQL syntax in similar, except that in only one of the () can be satisfied,
The $all must satisfy all values within []:
Db.mydb.insert ({name: "Zhaoliu", Age:[30,32,34]});
Db.mydb.find ({"Age": {$all: [30,32]}});
3. $exists determine if a field exists
Querying documents that contain the age attribute
Db.mydb.find ({age:{$exists: true}})
Querying documents that do not contain the age attribute
Db.mydb.find ({age:{$exists: false}})
4.NULL Value Processing
Db.mydb.find ({"Age": {$in: [null], $exists: true}}); The query contains the age property, and the value is null

Querying for documents with no age attribute and null with age property value
Db.mydb.find ({"Age": {$in: [null]}});
5. $mod modulo operation
The age attribute divided by 10 remainder is 2 of the document taken out
Db.mydb.find ({age:{$mod: [10,2]}}); The Age attribute%10 ==2
6. $ne Not equal to
Age is not 30, all queries come with documents that do not contain the ages attribute, and also contain documents with the aging property being null
Db.mydb.find ({age:{$ne: 30}})
The query has an age attribute and the value is not 30.
Db.mydb.find ({age:{$ne: +, $exists: true}});
7. $in is the same as the purpose of SQL standard syntax, which is to query a range of enumerations.
Db.mydb.find ({age:{$in: [30,23]}})

8. $nin does not contain
Db.mydb.find ({age:{$nin: [30,23]}})
Db.mydb.find ({age:{$nin: [30,23], $exists: true}})
9. $size number of array elements
Db.mydb.find ({age:{$size: 3}})
Db.mydb.find ({age:{$size: 1}})
Db.mydb.insert ({age:[100]});
Db.mydb.find ({age:{$size: 1}})
10. Regular expression Matching
Query mismatch name=z* lead record
Db.mydb.find ({name:{$not:/^z.*/}})
Db.mydb.find ({name:{$not:/^z.*/, $exists: true}});
11.JavaScript queries and $where queries
Query J for data greater than 20
Db.mydb.find ({j:{$gt: 20}})
Db.mydb.find ({$where: "this.j>20"});
Db.mydb.find ("this.j>20")
=======================
F=function () {return this.j>20;}
Db.mydb.find (f);
12.count Query Record Bar number
Db.mydb.find (). Count ()
Db.mydb.find ({j:{$gt: *}}). Count ()
13.skip limit returns the starting point of a document
Starting from 3rd record, return 5 records
Db.mydb.find (). Skip (3). Limit (5);
14.sort
Db.mydb.find ({age:{$exists: true}}). sort ({age:1}) ascending
Db.mydb.find ({j:{$exists: true}}). sort ({j:-1});
----------------------------------------------------------------------------------
D:
MongoDB Advanced Applications
Stored Procedures
Capped collection use
Gridfs for massive data storage
The MapReduce implementation of MongoDB
1. Cursors

1>
For (var a=db.person.find (); A.hasnext ();) {
Printjson (A.next ());
}
2>
var a = Db.person.find ();
while (A.hasnext ()) {
Printjson (A.next ());
}
3>
Db.person.find (). ForEach (function (u) {Printjson (U)});

2. Stored Procedures
MongoDB's stored procedures are written using JavaScript
SQL Custom Functions
function AddNumbers (x, y) {
return x+y;
}
SQL custom functions are converted into Stored procedure db.system.js.save ();
1) Defining Stored procedures
Db.system.js.save ({"_id": "AddNumbers", Value:function (x, y) {return x+y;});
--->_id: corresponds to a fairly functional name value: corresponding function body
This forms the MongoDB stored procedure.
2) View stored procedures:
Db.system.js.find ();
3) Execute the stored procedure
Db.eval (' statement ');
Such as:
Db.eval (' addnumbers (2,3) ');

Note: eval () is a special function that does not define stored procedures, and is directly nested inside functions using
Db.eval (function () {return 3+3;});
-----------------------------------------------------
Db.system.js.save ({"_id": "GetCount", Value:function () {return Db.person.count ();}});
Call
Db.eval (' GetCount () ');

Note: The syntax of JS and the basic operation of MongoDB

3.capped Collection
Better performance
1)-A special set, fixed-size collection, the data is full after the LRU (least recently used) and in the order of insertion to remove aging data
2) Build is pre-specified size, data is full, then add data, replace the oldest data
3) can be inserted and updated, cannot check the size of the collection otherwise the update fails
4) cannot be deleted, all rows in the collection can be deleted by drop, after deletion, the collection needs to be rebuilt
5) on 32-bit machine, one capped collection Max 483.5m
Unlimited on 64-bit machines

Use
(1) loggging log, write system log, can reduce personnel maintenance (automatically clean LRU), while capped collection can view the most recent insert data (Automatic Maintenance query order), capped collection has the maximum capacity limit
(2) Cached
(3) Auto archiving records the intermediate results of the statistics into the capped collection, which facilitates the statistical extraction
Maximize performance
Write features, write more, read less, do not index at least, otherwise affect the insertion speed
Use natural sort to maintain insertion order
Precautions when using
Set the MAXSIZE, and the initial size, because capped collection use, always check the size before checking max
You can use validate to see how much is being used in order to decide how much to set
Create cappped Collection syntax
Db.createcollection ("Capcol", {cappped:true,size:100000,max:100});
Create Normal: Collection
Db.createcollection ("Mu");
To delete an element in a collection
Db.collectionName.drop ();

Attention:
By default, the system automatically creates an index for collection, but does not create an index on capped collection


4.GridFs
is a specification for MongoDB that stores large files
Internal provides storage of large files, dividing files into small pieces (small documents for storage)
can store pictures or video files
Note: In the Internet generally use a targeted distributed file system to store files, videos and other HDFs gfs TFS
Gridfs Storing and viewing files
Gridfs uses two collections to store files with the default prefix FS
That
1) files Store metadata objects
2) Chunks Store related system binary block
Fs.files Fs.chunks
Command
Mongodbfiles put D:\\filename
Mongodbfiles List
Db.fs.files.find ();
Such as
Mongofiles put D:\\ddd.pdf
Mongofiles List

Get file
Mongofile get filename
Such as
Mongofiles Get D:\\ddd.pdf
Client view:
Show collections;
can display Fs.chunks,fs.files
Db.fs.files.find (); View field Description
FileName: File Store Name
ChunkSize: chunking size
Uploaddate: Storage Time
MD5: Encryption method
Connected to:127.0.0.1
Added file: {_id:objectid (' 54f48b837ea57845ede86935 '), FileName: "D:\\ddd.pdf"
, chunksize:261120, Uploaddate:new Date (1425312644357), MD5: "21c7b1d4d246f779
20f45f5ed855f23c ", length:49307054}
============================================
E:MONGODB Management
1.MapReduce
Much like a grouping operation in a relational database
MongoDB uses map/reduce for parallel statistics
MongoDB uses two functions to use MapReduce, and one map function is the reduce function.
The map function iterates through all the records in the collection by calling emit (Key,value), and then passes the key to value
Processing for the reduce function. Both of these functions can be implemented using JS
Perform an operation with Db.runcommand or the MapReduce command

Map function-----> decomposition generates key values
Reduce function---> Merge accept key values parameter to operate
A. Preparing data
Db.students.insert ({classid:1,age:14,name: "Tom"});
Db.students.insert ({classid:1,age:14,name: "Tom"});
Db.students.insert ({classid:1,age:14,name: "Tom"});
Db.students.insert ({classid:2,age:9,name: "Tony"});
Db.students.insert ({classid:2,age:13,name: "Harry"});
Db.students.insert ({classid:2,age:19,name: "Vincent"});
Db.students.insert ({classid:1,age:14,name: "Bill"});
Db.students.insert ({classid:2,age:17,name: "Bruce"});
B.map function
Values that are equivalent to This.classid as key,1 as objects
m = function () {Emit (this.classid,1)};
C.reduce function
R = function (key,values) {
var x = 0;
Values.foreach (function (v) {x+=v});
return x;
}
D. Implementation
res = Db.runcommand ({
MapReduce: "Students",
Map:m,
Reduce:r,
Out: "Students_res"
});

As a function of further processing
f = function (Key,value) {return {classid:key,count:value};}
res = Db.runcommand ({
MapReduce: "Students",
Map:m,
Reduce:r,
Out: "Students_res",
Finalize:f
});

Add some conditions
res = Db.runcommand ({
MapReduce: "Students",
Map:m,
Reduce:r,
Out: "Students_res",
Finalize:f,
query:{age:{$lt: 10}});

Another case
Book1 = {Name: "Understanding Java", pages:100};
Book2 = {Name: "Understanding Json", pages:200};
BOOK3 = {Name: "Understanding XML", pages:300};
book= {Name: "Understanding WebService", pages:400};
Book={name: "Understanding Axis2", pages:150};

Map function
var map = function () {
if (this.pages>=250)
Category = "Big Books";
Else
Category = "Small Books";
Emit (Category,this.name);
};
var reduce = function (key,values) {
var sum = 0;
Values.foreach (function (DOC) {
sum + = 1;
});
return {books:sum};
};
Perform
Db.books.mapReduce (map,reduce,{out: "Book_results"});
View Db.book_results.find ();

2. Data Export Mongoexport
mongoexport-d test-c books-o d:\\books.dat---->json format
-D Specify Database-C collection-o file

mongoexport-d test-c books-csv-o d:\\books_csv.dat---->csv format

3. Import
mongoimport-d test-c Books d:\\books.dat--->json format
mongoimport-d test-c Books--type csv--headerline--file d:\\books_csv.dat---> Import CSV format data
------------------------------------------------------------------------------------------

F.
osql 18th Talk-MONGODB Server Management
1. Backup and Recovery
How to create a database use database name if the database exists directly open if it does not exist create and open
Use TestDB
Backing Up the mongodump-d database
mongodump-d TestDB
The given dump directory is created directly under the current directory for the backed up files
You can specify the directory where the backup resides
mongodump-d Testdb-o Mydump

Recovery
Delete database into database use database name Db.dropdatabase ();
MONGORESTORE-D database name dump/database name
mongorestore-d TestDB Dump/testdb

Mongorestore-d TestDB The name of the custom folder/testdb

You can delete a table and insert data into a table without first deleting the database and specifying the-drop parameter.
Mongorestore-d Testdb-drop Dump/testdb

2. Access control
Before MongoDB boot, can be directly operated, and is arbitrary operation. This can lead to insecurity.
We can:
Bind IP Intranet address to access MongoDB service
Setting the Listening port
Use user name and password

Binding IP
Add parameter--bind_ip when starting
such as: Add parameters at startup--bind 192.168.0.3---> Only this IP access
Mongod--bind_ip 127.0.0.1--dbpath=d:\mongodb\data can only be accessed by this machine.

Set port default port is 27017
Mongod--bind_ip 127.0.0.1--port 28018--dbpath=d:\mongodb\data
MONGO 127.0.0.1:28018

User name and password
By default, all libraries have root privileges and all operations can be performed.
Can be set by specifying parameters.
Only need to start--auth---> Start the Login verification module The operation can also be performed after the default startup.
At first, MongoDB has an admin database (empty) by default.
More information is saved in admin.system.users than the user permissions set in other databases.
When no user is added in admin.system.users, even if MongoDB initiates the validation
The module can still perform any operation. Therefore, you need to add the document to the collection.

Establish the system root user
Db.adduser ("root", "1234");

Db.auth ("root", "1234");

Mongo-u root-p
Enter your password 1234
Connection Successful
At a later start, if the--auth connection must pass the user name password, otherwise can connect, can not do other operations

MongoDB also supports setting up users for a particular database.
Add a read-only user to test
Db.adduser ("User name", "password", true)
Use this user login to read-only operation

3. Command operation
Db.user.count ();
MONGO Test--eval "Printjson (Db.user.count ())"

MongoDB Learning Notes