Analyzing MongoDB data using Hadoop mapreduce

Use Hadoop MapReduce analyzes MongoDB data

(Many internet crawlers now store the data in Mongdb, so they study it and write this document)

Copyright NOTICE: This article is Yunshuxueyuan original article.
If you want to reprint please indicate the source: http://www.cnblogs.com/sxt-zkys/
QQ Technology Group: 299142667

First, the installation and use of Mongdb

1, official website download mongodb-linux-x86_64-rhel70-3.2.9.tgz

2, decompression (you can configure the environment variable)

3. Start the service side

./mongod--dbpath=/opt/local/mongodb/data--logpath=/opt/local/mongodb/logs--logappend--fork (background boot)

The first type: without AUTH certification

The second type: Requires auth authentication (that is, user name and password required)

When you specify the user name and password to view the data, the discovery can be seen

4. Start the client

./mongo

5. Client Shell command

Show DBS shows which databases are in MongoDB

DB Displays the database that is currently in use

Use DB name of the database you are using

(Note: If database does not exist, a will be created, and MongoDB will delete the database if it exits without any action)

Db.auth (Username,password) Username for username, password for password login to the database you want to use

Db.getcollectionnames () See what tables are in the current database

Db.  [Collectionname].insert ({...}) Add a document record to the specified database

Db. [Collectionname].findone () finds the first piece of data in a document

Db. [Collectionname].find () Find all records for a document

Db. [Collection].update ({query condition},{$set: {update content}}) updates a document record

Db. [Collection].drop () delete a collection from the database

Db.dropdatabase () Deleting a database

Data Example of the Mapreduce analysis MongoDB

1, write The MapReduce code, need to add two additional jar package, and need (jdk1.7 above)

2. Requirement Introduction and realization

Original data:

Result data:

Code writing:

Job:

Mapper:

Reduce:

The final result data:

Third, the last to recommend a MongoDB database management tools, very useful

Copyright NOTICE: This article is Yunshuxueyuan original article.
If you want to reprint please indicate the source: http://www.cnblogs.com/sxt-zkys/
QQ Technology Group: 299142667

Analyzing MongoDB data using Hadoop mapreduce