MongoDB secondary replset error rs102 too stale to catch up

1Http: // 101.130.92.32: 28018/_ replsetMonitoring site error: replset error rs102 too stale to catch up
Tue Aug 13 05:18:06 [rssync] replset see

Http://www.mongodb.org/display/DOCS/Resyncing+a+Very+Stale+Replica+Set+Member

05:18:06 [rssync] replset error rs102 too stale to catch up
05:20:07 [rssync] replset syncing to: 101.130.92.323: 27018
05:20:07 [rssync] replset error rs102 too stale to catch up, at lbgst from 101.130.92.323: 27018
05:20:07 [rssync] replset our last optime: Aug 6 19:18:05 52014bed: 2
05:20:07 [rssync] replset oldest at 101.130.92.323: 27018: Aug 10 10:58:26 52061cd2: 17
05:20:07 [rssync] replset see

Http://www.mongodb.org/display/DOCS/Resyncing+a+Very+Stale+Replica+Set+Member

05:20:07 [rssync] replset error rs102 too stale to catch up
05:22:08 [rssync] replset syncing to: 101.130.92.323: 27018
05:22:08 [rssync] replset error rs102 too stale to catch up, at lbgst from 101.130.92.323: 27018
05:22:08 [rssync] replset our last optime: Aug 6 19:18:05 52014bed: 2
05:22:08 [rssync] replset oldest at 101.130.92.323: 27018: Aug 10 10:59:17 52061d05: 3E

 

2. Check the status and find the recovring status.
[Bguser @ 346437-bgstore_db1 ~] $/DB/MongoDB/bin/Mongo -- Port 27018
MongoDB shell version: 2.0.1
Connecting to: 127.0.0.1: 27018/test
Recovering> dB. printreplicationinfo ();
Configured oplog size: 90000 MB
Log length start to end: 553610 secs (153.78hrs)
Oplog first event time: Wed Jul 31 2013 09:31:15 GMT + 0000 (UTC)
Oplog last event time: Tue Aug 06 2013 19:18:05 GMT + 0000 (UTC)
Now: Tue Aug 13 2013 06:03:40 GMT + 0000 (UTC)
Recovering> Rs. Status ();
{
"Set": "RPL ",
"Date": isodate ("2013-08-13t06: 03: 49z "),
"Mystate": 3,
"Syncingto": "101.130.92.323: 27018 ",
"Members ":[
{
"_ Id": 0,
"Name": "101.130.92.32: 27017 ",
"Hbglth": 1,
"State": 2,
"Statestr": "secondary ",
"Uptime": 8433,
"Optime ":{
"T": 1376373827000,
"I": 16
},
"Optimedate": isodate ("2013-08-13t06: 03: 47z "),
"Lasthbgrtbbgt": isodate ("2013-08-13t06: 03: 47z "),
"Pingms": 0
},
{
"_ Id": 4,
"Name": "101.130.92.32: 27019 ",
"Hbglth": 1,
"State": 7,
"Statestr": "arbiter ",
"Uptime": 8435,
"Optime ":{
"T": 0,
"I": 0
},
"Optimedate": isodate ("1970-01-01t00: 00: 00Z "),
"Lasthbgrtbbgt": isodate ("2013-08-13t06: 03: 47z "),
"Pingms": 0
},
{
"_ Id": 5,
"Name": "101.130.92.32: 27018 ",
"Hbglth": 1,
"State": 3,
"Statestr": "recovering ",
"Optime ":{
"T": 1375816685000,
"I": 2
},
"Optimedate": isodate ("2013-08-06t19: 18: 05z"), -- the synchronization time is incorrect, either on July 22, August 6. Today and the 13th day of China
"Self": True
},
{
"_ Id": 6,
"Name": "Maid: 27017 ",
"Hbglth": 1,
"State": 1,
"Statestr": "primary ",
"Uptime": 8433,
"Optime ":{
"T": 1376373827000,
"I": 17
},
"Optimedate": isodate ("2013-08-13t06: 03: 47z "),
"Lasthbgrtbbgt": isodate ("2013-08-13t06: 03: 47z "),
"Pingms": 0
},
{
"_ Id": 7,
"Name": "Maid: 27018 ",
"Hbglth": 1,
"State": 2,
"Statestr": "secondary ",
"Uptime": 8435,
"Optime ":{
"T": 1376373827000,
"I": 17
},
"Optimedate": isodate ("2013-08-13t06: 03: 47z "),
"Lasthbgrtbbgt": isodate ("2013-08-13t06: 03: 47z "),
"Pingms": 0
}
],
"OK": 1
}
Recovering>

 

3. Go to the official website to view the information and find the following:
Http://docs.mongodb.org/manual/tutorial/resync-replica-set-member/#replica-set-resync-by-copying

• Restart the specified d with an empty data directory And let MongoDB's normal initial syncing fbgture restore the data.
This is the more simple option, but may take longer to replace the data.

The restart procedure is as follows:
(1) Stop the failed mongod to stop the running MongoDB service with synchronization exceptions
(2) Delete all data in the dbpath (including subdirectories) delete all data files in the MongoDB directory including other subdirectories
(3) start it and it will automatically resynchronize itself from primary to start the MongoDB Service. The Service will automatically re-synchronize data from the primary.
 

4. After Re-Sync, the status of recovering has not changed to secondary, but the data has been synchronized.
Recovering> Rs. Status ();
{
"Set": "RPL ",
"Date": isodate ("2013-08-13t07: 58: 41z "),
"Mystate": 3,
"Syncingto": "101.130.92.323: 27018 ",
"Members ":[
{
"_ Id": 0,
"Name": "Maid: 27017 ",
"Health": 1,
"State": 2,
"Statestr": "secondary ",
"Uptime": 5629,
"Optime ":{
"T": 1376380719000,
"I": 84
},
"Optimedate": isodate ("2013-08-13t07: 58: 39z "),
"Lastheartbeat": isodate ("2013-08-13t07: 58: 39z "),
"Pingms": 0
},
{
"_ Id": 4,
"Name": "Maid: 27019 ",
"Health": 1,
"State": 7,
"Statestr": "arbiter ",
"Uptime": 5629,
"Optime ":{
"T": 0,
"I": 0
},
"Optimedate": isodate ("1970-01-01t00: 00: 00Z "),
"Lastheartbeat": isodate ("2013-08-13t07: 58: 39z "),
"Pingms": 0
},
{
"_ Id": 5,
"Name": "Maid: 27018 ",
"Health": 1,
"State": 3,
"Statestr": "recovering ",
"Optime ":{
"T": 1376380721000,
"I": 31
},
"Optimedate": isodate ("2013-08-13t07: 58: 41z "),
"Self": True
},
{
"_ Id": 6,
"Name": "Maid: 27017 ",
"Health": 1,
"State": 1,
"Statestr": "primary ",
"Uptime": 5629,
"Optime ":{
"T": 1376380719000,
"I": 66
},
"Optimedate": isodate ("2013-08-13t07: 58: 39z "),
"Lastheartbeat": isodate ("2013-08-13t07: 58: 39z "),
"Pingms": 0
},
{
"_ Id": 7,
"Name": "Maid: 27018 ",
"Health": 1,
"State": 2,
"Statestr": "secondary ",
"Uptime": 5629,
"Optime ":{
"T": 1376380719000,
"I": 66
},
"Optimedate": isodate ("2013-08-13t07: 58: 39z "),
"Lastheartbeat": isodate ("2013-08-13t07: 58: 39z "),
"Pingms": 0
}
],
"OK": 1
}
Recovering>

You still need to observe for a while to see if it will eventually change from recovering to secondary.

 

After one day of observation, it is still recovring, and no exception information is reported in the log.So I tried to restart it.

Killed 12292

/DB/MongoDB/bin/mongod-F/etc/MongoDB/27018. conf

 

The status is changed to secondary.