MongoDB mistakenly deleted table recovery

First, the scene description

An engineer of the company executes Db.giveget_card.drop () and mistakenly deletes the online table.

Fortunately every day to do a backup, this time reflects the importance of backup, hahaha ...

Second, simulate the failure process:

Backup Data size:

Rs_test01:primary> use ycsbswitched to DB ycsbrs_test01:primary> db.giveget_card.count (); 3173391

This table is updated before you delete it.

Rs_test01:primary> Db.giveget_card.insert ({id:1}); Writeresult ({"ninserted": 1}) rs_test01:primary> Db.giveget_card.insert ({id:2}); Writeresult ({"ninserted": 1}) rs_test01:primary> Db.giveget_card.insert ({id:3}); Writeresult ({"ninserted": 1}) rs_test01:primary> Db.giveget_card.insert ({id:4}); Writeresult ({"ninserted": 1})

Other tables also have update operations.

Rs_test01:primary> Db.tab.find (); {"_id": ObjectId ("59354ba202d9a99ab2f879c6"), "name": "A"} {"_id": ObjectId ("59354ba602d9a99ab2f879c7"), "name": "B"} {"_id": ObjectId ("59354ba802d9a99ab2f879c8"), "name": "C"} {"_id": ObjectId ("59354baa02d9a99ab2f879c9"), "name": "D"}

This table and other tables are updated after the delete operation.

Rs_test01:primary> Db.giveget_card.find (); {"_id": ObjectId ("59354c28d905432aeaccd53c"), "id": 5} {"_id": ObjectId ("59354c2bd905432aeaccd53d"), "id": 6}rs_test01:primary> db.tab.find (); {"_id": ObjectId ("59354ba202d9a99ab2f879c6"), "name": "A"} {"_id": ObjectId ("59354ba602d9a99ab2f879c7"), "name": "B"} {"_id": ObjectId ("59354ba802d9a99ab2f879c8"), "name": "C"} {"_id": ObjectId ("59354baa02d9a99ab2f879c9"), "name": "D"} {"_id": ObjectId ("59354ccfd905432aeaccd542"), "name": "E"} {"_id": ObjectId ("59354cd2d905432aeaccd543"), "name": "F"}

Third, recovery steps

1. Copy the Giveget_card.bson and Giveget_card.metadata.json files from the tab table in the backup to the/TMP/RESTORE/YCSB directory (self-built directory), YCSB the name of the library.

# cp/data/backup/rs07/ycsb/giveget_card.*/TMP/RESTORE/YCSB

2, after the backup time, delete the Oplog before the operation of the export, for the recovery of the table

# mongodump--port 2203-d local-c oplog.rs-q ' {"ts": {$gte: Timestamp (1496664480, 10430), $lte: Timestamp (1496665113 , 10430)}} '-o/tmp/oplog

--the timestamp is the result of conversion using the conversion tool.

3. Use Bsondump to view the Oplog log and find the timestamp of the drop operation 1496665069

# bsondump /tmp/oplog/local/oplog.rs.bson {"ts": {"$timestamp": {"T": 1496664760, "I": 1}}, "T": {"$ Numberlong ":" "{"}, "H": {"$numberLong": "7079172056815894727"}, "V": 2, "OP": "I", "ns": "Ycsb.giveget_card", "O": {"_id" : {"$oid": "59354ab8c5308d8c7a9da8b5"}, "id": 1.0}}{"ts": {"$timestamp": {"T": 1496664762, "I": 1}}, "T": {"$numberLong": "" "}," H ": {" $numberLong ":" -1797107728294067016 "}," V ": 2," OP ":" I "," ns ":" Ycsb.giveget_card "," O ": {" _id ": {" $oid ":" 59354abac5308d8c7a9da8b6 "}," id ": 2.0}}{" ts ": {" $timestamp ": {" T ": 1496664765," I ": 1}}," T ": {" $numberLong ":" ""}, "H": {"$numberLong": "8604646791509150392"}, "V": 2, "OP": "I", "ns": "Ycsb.giveget_card", "O": {"_id": {"$oid": " 59354abdc5308d8c7a9da8b7 "}," id ": 3.0}}{" ts ": {" $timestamp ": {" T ": 1496664768," I ": 1}}," T ": {" $numberLong ":" ""}, "H": {"$numberLong": "9018614066505371436"}, "V": 2, "OP": "I", "ns": "Ycsb.giveget_card", "O": {"_id": {"$oid": " 59354ac0c5308d8c7a9da8b8 "}," id ": 4.0}}{" ts ": {" $timestamp ": {" T ": 1496664994," I ": 1}}," T ": {" $numberLong ":" ""}, "H": {"$numberLong": "4471524661347063602"}, "V": 2, "OP": "C", "ns": "YCSB. $cmd", "O": {"Create": "tab"}}{"ts": {"$timestamp": {"T": 1496664994, "I": 2}}, "T": {"$ Numberlong ":" "{"}, "H": {"$numberLong": " -4215905958456607246"}, "V": 2, "OP": "I", "ns": "Ycsb.tab", "O": {"_id": {"$oid" : "59354ba202d9a99ab2f879c6"}, "name": "A"}}{"ts": {"$timestamp": {"T": 1496664998, "I": 1}}, "T": {"$numberLong": "12"}, "H": {"$numberLong": "6170506962401844481"}, "V": 2, "OP": "I", "ns": "Ycsb.tab", "O": {"_id": {"$oid": " 59354ba602d9a99ab2f879c7 "}," name ":" B "}}{" ts ": {" $timestamp ": {" T ": 1496665000," I ": 1}}," T ": {" $numberLong ":" ""}, "H ": {" $numberLong ":" -8071456063660489895 "}," V ": 2," OP ":" I "," ns ":" Ycsb.tab "," O ": {" _id ": {" $oid ":" 59354ba802d9a99ab2f879c8 "}," name ":" C "}}{" ts ": {" $timestamp ": {" T ": 1496665002," I ": 1}}," T ": {" $numberLong ":" ""}, "H ": {" $numberLong ":" 4387884836668659146 "}," V ": 2," OP ":" I "," ns ":" Ycsb.tab "," O ": {" _id ": {" $oid ":" 59354baa02d9a99ab2f879c9 "}," name ":" D "}}{" ts ": {" $timestamp ": {" T ": 1496665069," I ": 1}}," T ": {" $numberLong ":" ""}, "H ": {" $numberLong ":" -6913449254950935781 "}," V ": 2," OP ":" C "," ns ":" ycsb.$Cmd "," O ": {" Drop ":" Giveget_card "}}2017-06-05t20:27:25.552+080010 objects found 

4, copy the Oplog Bson file to the corresponding directory

# Cp/tmp/oplog/local/oplog.rs.bson/tmp/restore/oplog.bson

The directory structure to restore at this time:

# pwd/tmp/restore# Lsoplog.bson YCSB

5, to this point, all the preparation operations have been done to restore data.

[[email protected] restore]# mongorestore --port 2203 --oplogreplay -- oploglimit=1496665069:1 /tmp/restore2017-06-05t20:36:45.361+0800building a list of  dbs and collections to restore from /tmp/restore dir2017-06-05t20:36:45.364 +0800reading metadata for ycsb.giveget_card from /tmp/restore/ycsb/giveget_ Card.metadata.json2017-06-05t20:36:45.364+0800restoring ycsb.giveget_card from /tmp/restore/ Ycsb/giveget_card.bson2017-06-05t20:36:48.362+0800[........ ..........]   ycsb.giveget_card  15.4MB/475MB   (3.2%) 2017-06-05t20:36:51.362+0800[# .......................]   ycsb.giveget_card  31.1MB/475MB   (6.6%) 2017-06-05t20:36:54.362+0800[## ......................]   ycsb.giveget_card  46.6MB/475MB   (9.8%) 2017-06-05t20:36:57.362+0800[### .....................]   ycsb.giveget_card  62.1mb/475mb   (13.1%) 2017-06-05t20:37:00.362+0800[### ............ ...]   ycsb.giveget_card  76.4MB/475MB   (16.1%) 2017-06-05t20:37:03.362+0800[#### ....................]   ycsb.giveget_card  90.7MB/475MB   (19.1%) 2017-06-05t20:37:06.362+0800[##### ...................]   ycsb.giveget_card  105MB/475MB   (22.0%) 2017-06-05t20:37:09.362+0800[###### ..................]   ycsb.giveget_card  120MB/475MB   (25.2%) 2017-06-05t20:37:12.362+0800[###### ..................]   ycsb.giveget_card  133MB/475MB   (28.0%) 2017-06-05t20:37:15.362+0800[####### .................]   ycsb.giveget_card  146MB/475MB   (30.8%) 2017-06-05t20:37:18.363+0800[######## ................]   ycsb.giveget_card  163MB/475MB   (34.3%) 2017-06-05t20:37:21.362+0800[######## ................]   ycsb.giveget_card  178mb/475mb   (37.4%) 2017-06-05t20:37:24.362+0800[######### ....... ...]   ycsb.giveget_card  196MB/475MB   (41.3%) 2017-06-05t20:37:27.362+0800[########## ..............]   ycsb.giveget_card  214MB/475MB   (45.0%) 2017-06-05t20:37:30.362+0800[########## #.............]   ycsb.giveget_card  231MB/475MB   (48.6%) 2017-06-05t20:37:33.362+0800[########## ##............]   ycsb.giveget_card  245MB/475MB   (51.5%) 2017-06-05t20:37:36.362+0800[########## ###...........]   ycsb.giveget_card  261MB/475MB   (54.8%) 2017-06-05t20:37:39.362+0800[########## ####..........]   ycsb.giveget_card  279MB/475MB   (58.7%) 2017-06-05t20:37:42.362+0800[########## #####.........]   ycsb.giveget_card  297MB/475MB   (62.5%) 2017-06-05t20:37:45.362+0800[########## #####.........]   ycsb.giveget_card  312mb/475mb   (65.8%) 2017-06-05t20:37:48.362+0800[################ ...]   ycsb.giveget_card  328MB/475MB   (69.0%) 2017-06-05t20:37:51.362+0800[########## #######.......]   ycsb.giveget_card  341MB/475MB   (71.8%) 2017-06-05t20:37:54.362+0800[########## #######.......]   ycsb.giveget_card  356MB/475MB   (74.9%) 2017-06-05t20:37:57.362+0800[########## ########......]   ycsb.giveget_card  373MB/475MB   (78.5%) 2017-06-05t20:38:00.362+0800[########## #########.....]   ycsb.giveget_card  388MB/475MB   (81.7%) 2017-06-05t20:38:03.362+0800[########## ##########....]   ycsb.giveget_card  405MB/475MB   (85.2%) 2017-06-05t20:38:06.362+0800[########## ###########...]   ycsb.giveget_card  419MB/475MB   (88.2%) 2017-06-05t20:38:09.362+0800[########## ###########...]   ycsb.giveget_card  434mb/475mb   (91.4%) 2017-06-05t20:38:12.362+0800[###################### ...]   ycsb.giveget_card  442MB/475MB   (93.1%) 2017-06-05t20:38:15.362+0800[########## #############.]   ycsb.giveget_card  459MB/475MB   (96.6%) 2017-06-05t20:38:18.362+0800[########## #############.]   ycsb.giveget_card  475MB/475MB   (99.9%) 2017-06-05t20:38:18.427+0800[########## ##############]  ycsb.giveget_card  475mb/475mb   (100.0%) 2017-06-05T20:38:18.427 +0800restoring indexes for collection ycsb.giveget_card from  metadata2017-06-05t20:38:44.680+0800finished restoring ycsb.giveget_card  (3173391  Documents) 2017-06-05t20:38:44.680+0800replaying oplog2017-06-05t20:38:44.739+0800done

6. View the results of the recovery

Rs_test01:primary> Db.giveget_card.find ({id: {$gte: 1}}); {"_id": ObjectId ("59354cb9d905432aeaccd540"), "id": 5} {"_id": ObjectId ("59354cc0d905432aeaccd541"), "id": 6} {"_id": ObjectId ("59354ab8c5308d8c7a9da8b5"), "id": 1} {"_id": ObjectId ("59354abac5308d8c7a9da8b6"), "id": 2} {"_id": ObjectId ("59354abdc5308d8c7a9da8b7"), "id": 3} {"_id": ObjectId ("59354ac0c5308d8c7a9da8b8"), "id": 4}

The data content is the same, but the order of storage is different from the order in which the previous data was stored.

Rs_test01:primary> Db.giveget_card.count (); 3173397

Results count= Backup table data after 3173391+ update data 6.

7, because the dump out of the Oplog also contains the operation of other tables. See if there are any effects on other tables during the recovery process.

Rs_test01:primary> Db.tab.find (); {"_id": ObjectId ("59354ba202d9a99ab2f879c6"), "name": "A"} {"_id": ObjectId ("59354ba602d9a99ab2f879c7"), "name": "B"} {"_id": ObjectId ("59354ba802d9a99ab2f879c8"), "name": "C"} {"_id": ObjectId ("59354baa02d9a99ab2f879c9"), "name": "D"} {"_id": ObjectId ("59354ccfd905432aeaccd542"), "name": "E"} {"_id": ObjectId ("59354cd2d905432aeaccd543"), "name": "F"}

--View tab table data is the same as the original table data, there is no effect, indicating that the other table log in the empty run.

The above is the backup combined with oplog recovery operation.

Backup is very important!!! Backup is very important!!! Backup is very important!!!

Important things to say three times ~ ~ ~

This article is from the "11570138" blog, please be sure to keep this source http://11580138.blog.51cto.com/11570138/1932502

MongoDB mistakenly deleted table recovery