How MySQL data is migrated to MongoDB scripts in Python

about MongoDB

MongoDB is a database based on distributed file storage. Written by the C + + language. Designed to provide scalable, high-performance data storage solutions for WEB applications.

MongoDB is a product between a relational database and a non-relational database, and is the most versatile and most like relational database in a non-relational database.

MongoDB is a document database that has a natural advantage in storing small files. As the business changes, you need to import the line records from the MySQL database into the MongoDB documentation.

One, the scene : online MySQL database a table migrated to MongoDB, the field has no change.

Second, the Python module:

Use Python's Torndb,pymongo and time modules.

* Note: Install SETUP.PY,PIP,MYSQLDB first

Execute the following command:

Pip Install Torndb
Pip Install Pymongo

Third, the script content is as follows:

[Root ~] #cat nmytomongo.py

#!/usr/bin/env python#fielname:mytomongo.py#author:xkops#coding:utf-8import torndb,pymongo,time# Connect to MySQL Databasemysql = torndb. Connection (host= ' 127.0.0.1 ', database= ' database ', user= ' username ', password= ' password ') #connect to MongoDB and obtain Total lines in Mysqlmongo = Pymongo. Mongoclient (' mongodb://ip '). Databasemongo.authenticate (' username ', password= ' password ') Countlines = Mysql.query (' SELECT Max (Table_field) from table_name ') Count = countlines[0][' max (table_field) ' #count = 300print Counti = 0 J = 100star T_time = Time.time () #select from MySQL to insert MongoDB by lines.for I in range (0,count,100): #print A, b #print i #pr int ' SELECT * from quiz_submission where quiz_submission_id >%d and quiz_submission_id <=%d '% (i,j) submission = m Ysql.query (' SELECT * from table_name where Table_field >%d and table_field <=%d '% (i,j)) #print submission if SUBM Ission: #collection_name like MySQL table_name mongo.collection_name.insert_many (submission) else:i +=100J +=100 Continue I +=100 j +=100end_time = Time.time () deltatime = end_time-start_timetotalhour = Int (deltatime/3600) to Talminute = Int ((deltatime-totalhour * 3600)/each) Totalsecond = Int (Deltatime-totalhour * 3600-totalminute *) #pri NT Migrate data total time consuming.print "data migrate finished,total time consuming:%d Hour%d Minute%d Seconds"% (to Talhour,totalminute,totalsecond)

* Note: Change the database address, user, password, library name, table name, and field name in the above code according to your own needs.

Iv. Execute the Migration script:

[Root ~] #python nmytomongo.py &>/tmp/migratelog.txt &

Review the time that the/tmp/migratelog.txt data migration was consumed after the script execution was completed.

