How to migrate MySQL Data to MongoDB script in Python

MongoDB is a product between relational databases and non-relational databases. It has the most abundant functions and features like relational databases. This article describes how to migrate MySQL Data to MongoDB scripts in Python. For more information, see MongoDB Introduction

MongoDB is a distributed file storage-based database. Written in C ++. It is designed to provide scalable, high-performance data storage solutions for WEB applications.

MongoDB is a product between relational databases and non-relational databases. It has the most abundant functions and features like relational databases.

MongoDB is a document database with a natural advantage in storing small files. As the business requirements change, you need to import the Row Records in the online MySQL database to the document records in MongoDB.

I. Scenarios: An online MySQL database table is migrated to MongoDB, and the fields remain unchanged.

Ii. Python module:

Use Python's torndb, pymongo, and time modules.

* Note: first install setup. py, pip, MySQLdb

Run the following command:

Pip install torndb
Pip install pymongo

Iii. Script content:

[Root ~] # Cat nmytomongo. py

#! /Usr/bin/env python # fielName: mytomongo. py # Author: xkops # coding: utf-8import torndb, pymongo, time # connect to mysql databasemysql = torndb. connection (host = '2014. 1. 0.0.1 ', database = 'database', user = 'username', password = 'Password') # connect to mongodb and obtain total lines in mysqlmongo = pymongo. mongoClient ('mongodb: // ip '). databasemongo. authenticate ('username', password = 'Password') countlines = mysql. query ('select max (table_field) FROM table_name ') count = countlines [0] ['max (table_field)'] # count = 300 print counti = 0 j = 100start_time = time. time () # select from mysql to insert mongodb by 100 lines. for I in range (0, count, 100): # print, B # print I # print 'select * FROM quiz_submission where quiz_submission_id> % d and quiz_submission_id <= % d' % (I, j) submission = mysql. query ('select * FROM table_name where table_field> % d and table_field <= % d' % (I, j) # print submission if submission: # collection_name like mysql table_name mongo. collection_name.insert_sion (submission) else: I + = 100 j + = 100 continue I + = 100 j + = 100end_time = time. time () deltatime = end_time-start_timetotalhour = int (deltatime/3600) totalminute = int (deltatime-totalhour * 3600)/60) totalsecond = int (deltatime-totalhour * 3600-totalminute * 60) # print migrate data total time consuming. print "Data Migrate Finished, Total Time Consuming: % d Hour % d Minute % d Seconds" % (totalhour, totalminute, totalsecond)

* Note: Change the database address, user, password, database name, table name, and field name in the above Code as needed.

4. Execute the migration script:

[Root ~] # Python nmytomongo. py &>/tmp/migratelog.txt &

After the script is executed, view the time consumed for migration of/tmp/migratelog.txt data.