MongoDB backup and recovery

A powerful database like MongoDB naturally provides a comprehensive, rich, and easy-to-use backup and recovery mechanism.

1. Full-database backup and recovery

1.1 tool description

Mongodb provides mongodump and mongorestore tools to conveniently back up and restore the entire database. For how to use the tool, you can use the -- help option following the command to view the help documents of the two tools. As follows:

$ mongodump --helpExport MongoDB data to BSON files.options:  --help                   produce help message  -v [ --verbose ]         be more verbose (include multiple times for more                            verbosity e.g. -vvvvv)  --version                print the program's version and exit  -h [ --host ] arg        mongo host to connect to ( <set name>/s1,s2 for                            sets)  --port arg               server port. Can also use --host hostname:port  --ipv6                   enable IPv6 support (disabled by default)  -u [ --username ] arg    username  -p [ --password ] arg    password  --dbpath arg             directly access mongod database files in the given                            path, instead of connecting to a mongod  server -                            needs to lock the data directory, so cannot be used                            if a mongod is currently accessing the same path  --directoryperdb         if dbpath specified, each db is in a separate                            directory  --journal                enable journaling  -d [ --db ] arg          database to use  -c [ --collection ] arg  collection to use (some commands)  -o [ --out ] arg (=dump) output directory or "-" for stdout  -q [ --query ] arg       json query  --oplog                  Use oplog for point-in-time snapshotting  --repair                 try to recover a crashed database  --forceTableScan         force a table scan (do not use $snapshot)$ mongorestore --helpusage: mongorestore [options] [directory or filename to restore from]options:  --help                  produce help message  -v [ --verbose ]        be more verbose (include multiple times for more                           verbosity e.g. -vvvvv)  --version               print the program's version and exit  -h [ --host ] arg       mongo host to connect to ( <set name>/s1,s2 for sets)  --port arg              server port. Can also use --host hostname:port  --ipv6                  enable IPv6 support (disabled by default)  -u [ --username ] arg   username  -p [ --password ] arg   password  --dbpath arg            directly access mongod database files in the given                           path, instead of connecting to a mongod  server -                           needs to lock the data directory, so cannot be used                           if a mongod is currently accessing the same path  --directoryperdb        if dbpath specified, each db is in a separate                           directory  --journal               enable journaling  -d [ --db ] arg         database to use  -c [ --collection ] arg collection to use (some commands)  --objcheck              validate object before inserting  --filter arg            filter to apply before inserting  --drop                  drop each collection before import  --oplogReplay           replay oplog for point-in-time restore  --keepIndexVersion      don't upgrade indexes to newest version

Common options are as follows:

Mongodump-h host-d dbname-o directory
-H: the address of the server where MongDB is located, such as 127.0.0.1. You can also specify the port number 127.0.0.1: 27017.
-D: name of the database to be backed up, for example, db_test
-O: storage location of backup data, for example :~ \ Dump. Of course, this directory must be created in advance. After the backup is complete, the system automatically creates a db_test directory under the dump directory, which stores the backup data of the database instance.

Mongorestore-h host-d dbname -- directoryperdb dbdirectory
-H: Address of the MongoDB Server
-D: name of the database to be restored, for example, db_test. Of course, this name can be different from that of the backup, such as new_db.
-- Directoryperdb: Location of the backup data file, for example :~ \ Dump \ db_test (here we need to add the db_test sub-directory and read "Every db is in a separate directory" from -- directoryperdb in mongoretore's help ".)

1.2 tool use instance

Now there is a test database on a remote mongo database server. Two Collections (book and user) are created for testing. The contents of the collection can be viewed through the simple mongo command line, as follows:

book collection：{    _id: ObjectId("52441b86d8947f5302000000"),    name: "C++ Primer",    author: "Not Known",    type: "good book"}{    _id: ObjectId("52441bd0d8947fb601000000"),    name: "C++ Model Exploration",    author: "A Foreigner",    type: "Should Be Read Before Interview"}user collection：{    _id: ObjectId("52442736d8947fb501000001"),    name: "lfqy",    gender: "male"}

Use mongodump to back up the database:

$ mongodump -h 10.77.20.xx -d test -o ~/dumpconnected to: 10.77.20.xxDATABASE: test to /home/lfqy/dump/testtest.system.indexes to /home/lfqy/dump/test/system.indexes.bson 2 objectstest.book to /home/lfqy/dump/test/book.bson 2 objectstest.user to /home/lfqy/dump/test/user.bson 1 objects

Next we will delete the test database on the server (you can use various methods, commands, and tools... Here we use a mongo visual management tool .), Then run the command to restore the database.

$ mongorestore -h 10.77.20.xx -d test --directoryperdb ~/dump/testconnected to: 10.77.20.xxThu Sep 26 20:37:14 /home/lfqy/dump/test/book.bsonThu Sep 26 20:37:14  going into namespace [test.book]2 objects foundThu Sep 26 20:37:14 /home/lfqy/dump/test/user.bsonThu Sep 26 20:37:14  going into namespace [test.user]1 objects foundThu Sep 26 20:37:14 /home/lfqy/dump/test/system.indexes.bsonThu Sep 26 20:37:14  going into namespace [test.system.indexes]Thu Sep 26 20:37:14 { key: { _id: 1 }, ns: "test.book", name: "_id_" }Thu Sep 26 20:37:14 { key: { _id: 1 }, ns: "test.user", name: "_id_" }2 objects found

In this way, the test database is restored.

In addition, using the mongorestore tool to restore the database will not affect the records of newly inserted databases after the backup. Specifically, assume that there are two records A and B in the book set of the test database before the backup. After the backup, C is newly inserted in the book set. In this way, after the mongorestore tool is used to restore the database, the backup data will overwrite the records A and B, without affecting C. Finally, there will still be three records A, B, and C in the book collection. You can also specify the -- drop option to show that the original database is dropped before recovery.

2. Export data of a single collection

In addition to full-database backup and recovery, mongo also provides convenient Export and Import tools for a single collection of data, including export and Export Import, it can be used to back up and recover data after disaster when performing risky operations on data.

2.1 tool description

The export tool can export a collection to a file in JSON or CSV format. The export import tool can import the content of a specific file to a specified collection (JSON data can be imported, you can also import CSV format data ). You can also use the command + -- help option to view the help of the two tools, as shown below:

$ mongoexport --helpExport MongoDB data to CSV, TSV or JSON files.options:  --help                    produce help message  -v [ --verbose ]          be more verbose (include multiple times for more                             verbosity e.g. -vvvvv)  --version                 print the program's version and exit  -h [ --host ] arg         mongo host to connect to ( <set name>/s1,s2 for                             sets)  --port arg                server port. Can also use --host hostname:port  --ipv6                    enable IPv6 support (disabled by default)  -u [ --username ] arg     username  -p [ --password ] arg     password  --dbpath arg              directly access mongod database files in the given                             path, instead of connecting to a mongod  server -                             needs to lock the data directory, so cannot be used                            if a mongod is currently accessing the same path  --directoryperdb          if dbpath specified, each db is in a separate                             directory  --journal                 enable journaling  -d [ --db ] arg           database to use  -c [ --collection ] arg   collection to use (some commands)  -f [ --fields ] arg       comma separated list of field names e.g. -f                             name,age  --fieldFile arg           file with fields names - 1 per line  -q [ --query ] arg        query filter, as a JSON string  --csv                     export to csv instead of json  -o [ --out ] arg          output file; if not specified, stdout is used  --jsonArray               output to a json array rather than one object per                             line  -k [ --slaveOk ] arg (=1) use secondaries for export if available, default                             true$ mongoimport --helpoptions:  --help                  produce help message  -v [ --verbose ]        be more verbose (include multiple times for more                           verbosity e.g. -vvvvv)  --version               print the program's version and exit  -h [ --host ] arg       mongo host to connect to ( <set name>/s1,s2 for sets)  --port arg              server port. Can also use --host hostname:port  --ipv6                  enable IPv6 support (disabled by default)  -u [ --username ] arg   username  -p [ --password ] arg   password  --dbpath arg            directly access mongod database files in the given                           path, instead of connecting to a mongod  server -                           needs to lock the data directory, so cannot be used                           if a mongod is currently accessing the same path  --directoryperdb        if dbpath specified, each db is in a separate                           directory  --journal               enable journaling  -d [ --db ] arg         database to use  -c [ --collection ] arg collection to use (some commands)  -f [ --fields ] arg     comma separated list of field names e.g. -f name,age  --fieldFile arg         file with fields names - 1 per line  --ignoreBlanks          if given, empty fields in csv and tsv will be ignored  --type arg              type of file to import.  default: json (json,csv,tsv)  --file arg              file to import from; if not specified stdin is used  --drop                  drop collection first   --headerline            CSV,TSV only - use first line as headers  --upsert                insert or update objects that already exist  --upsertFields arg      comma-separated fields for the query part of the                           upsert. You should make sure this is indexed  --stopOnError           stop importing at first error rather than continuing  --jsonArray             load a json array, not one item per line. Currently                           limited to 4MB.

The common options are as follows:

Export export-h host-d dbname-c collection_name-o filename (export data from the collection named collection_name in the dbname database on the host database server to the file named filename .)
-H: Specifies the IP address of the database server.
-U: Specifies the database user name.
-P: Specifies the Database Password.
-D: Specifies the database name.
-C: Specifies the collection name.
-F: Specifies the columns to export.
-O: Specifies the file name to be exported.
-Q: specify the criteria for filtering the exported data.
Collect import-h host-d dbname-c collection_name filename (recover the data in the file named filename to the collection of collection_name in the database named dbname on the database server whose address is host .)
-H: Specifies the IP address of the database server.
-U: Specifies the database user name.
-P: Specifies the Database Password.
-D: Specifies the database name.
-C: Specifies the collection name.
-F: Specifies the columns to import.
2.2 tool use instance

Here we also use the test database mentioned above as an example to first export the book set.

$ mongoexport -h 10.77.20.xx -d test -c book -o book.jsonconnected to: 10.77.20.xxexported 2 records

Next, delete the book set (the previous method is used here) and restore it from the file (that is, import the data in the file into the database ).

$ mongoimport -h 10.77.20.xx -d test -c book book.jsonconnected to: 10.77.20.xximported 2 objects

In this way, the data in book. json is imported into the book collection in test.

Similarly, after the book. json file is imported to the database, it will not be affected. After the data is backed up, it will be added to the database. Specifically, assume that there are two records A and B in the book collection before the backup. After the backup, C is newly inserted in the book collection. In this way, after the backup data is imported into the book set, the AB record will be overwritten, without affecting C. In the end, there will still be three records A, B, and C in the book set. Similar to mongorestore, you can specify the -- drop option to show that the original set is dropped before restoration.
3. Differences between the two

I don't know much about this. First, it is not difficult to see from the above that mongorestore and mongodump provide the restoration and backup of the entire mongo database, volume import and volume export provide more fine-grained collection-level data import and export. The two have different granularities. the granularity of volume import and volume export is finer and more flexible. Second, Export Import and Export export only export and import data in the set, but do not back up other components in the database (such as indexes ), mongorestore and mongodump are used to restore and back up all the components in the database (including indexes and other components. However, this also results in a long time-consuming file exported by mongorestore and mongodump, while the exported files by Export Import and Export export are relatively small, fast, and flexible in format. As for the usage scenarios, let's think about it as needed.