MongoDB Abnormal shutdown cannot start

Just contact MongoDB, encountered a small problem, the information found on the Internet seems to be not very full, the summary record, I hope to help everyone.

First, the problem description

When reconfiguring the master and slave of MongoDB, the database "kill Mongod process" was shut down gracefully, and the result throws an error at reboot.

PS: Normal should be: Mongod-f/mvnobss/users/mgousr01/mongodb/conf/master.conf--shutdown

[Email protected]:/mvnobss/users/mgousr01/mongodb/bin>./start_mongodb.sh about-to-fork child process, waiting until Connections.forked Process:30717error:child process failed, exited with error number 100

Second, problem solving

To view logs:

2017-05-11t16:01:42.370+0800 i control  ***** server restarted ***** 2017-05-11t16:01:42.404+0800 i journal  [initandlisten] journal dir=/mvnobss/ users/mgousr01/mongodb/dbdata/rs0/journal2017-05-11t16:01:42.404+0800 i journal  [ initandlisten] recover begin2017-05-11t16:01:42.404+0800 i journal  [ initandlisten] recover lsn: 757156372017-05-11t16:01:42.404+0800 i journal   [initandlisten] recover /mvnobss/users/mgousr01/mongodb/dbdata/rs0/journal/j._232017-05-11t16 : 01:42.408+0800 i journal  [initandlisten] recover skipping application  of section seq:59812763 < lsn:757156372017-05-11T16:01:43.187+0800 I  Journal  [initandlisten] recover skipping application of section seq : 59872073 < lsn:757156372017-05-11t16:01: 43.517+0800 i journal  [initandlisten] recover /mvnobss/users/mgousr01/mongodb/ dbdata/rs0/journal/j._242017-05-11t16:01:44.633+0800 i journal  [initandlisten]  recover /mvnobss/users/mgousr01/mongodb/dbdata/rs0/journal/j._252017-05-11t16:01:45.762+0800 i  Journal  [initandlisten] recover /mvnobss/users/mgousr01/mongodb/dbdata/rs0/journal/j._ 262017-05-11t16:01:45.765+0800 i journal  [initandlisten] recover skipping  application of section seq:60050033 < lsn:757156372017-05-11t16:01:45.766+0800  F JOURNAL  [initandlisten] dbexception during recovery: 15923  Couldn ' t get file length when opening mapping /mvnobss/users/mgousr01/ Mongodb/dbdata/rs0/lxvomgodb1/lxvomgodb1.19 boost::filesystem::file_size: no such file  or directory:  "/Mvnobss/users/mgousr01/mongodb/dbdata/rs0/lxvomgodb1/lxvomgodb1.19 "2017-05-11t16:01:45.766+0800 i  Storage  [initandlisten] exception in initandlisten: 15923 couldn ' t  get file length when opening mapping /mvnobss/users/mgousr01/mongodb/dbdata/rs0/ Lxvomgodb1/lxvomgodb1.19 boost::filesystem::file_size: no such file or directory :  "/mvnobss/users/mgousr01/mongodb/dbdata/rs0/lxvomgodb1/lxvomgodb1.19",  terminating2017-05-11t16:0 1:45.766+0800 i control  [initandlisten] now exiting2017-05-11t16:01:45.766+0800  i network  [initandlisten] shutdown: going to close listening  sockets ... 2017-05-11t16:01:45.767+0800 i network  [initandlisten] shutdown: going to  flush diaglog ... 2017-05-11t16:01:45.767+0800 i network  [initandlisten] shutdoWn: going to close sockets ... 2017-05-11t16:01:45.767+0800 i storage  [initandlisten] shutdown: waiting  For fs preallocator ... 2017-05-11t16:01:45.767+0800 i storage  [initandlisten] shutdown: final  Commit ... 2017-05-11t16:01:45.767+0800 i storage  [initandlisten] shutdown: closing  All files ... 2017-05-11t16:01:45.767+0800 i storage  [initandlisten] closeallfiles ()   Finished2017-05-11t16:01:45.767+0800 i control  [initandlisten] dbexit:  rc :  100

The log says/mvnobss/users/mgousr01/mongodb/dbdata/rs0/lxvomgodb1/can't be found. lxvomgodb1.19 file, this is normal, because I stopped the database before backing up the database deleted, the general situation can be started up.

Baidu Online said it is necessary to delete the abnormal shutdown after the resulting mongod.lock file to restart, good, delete to go again:

[email protected]:/mvnobss/users/mgousr01/mongodb/dbdata/rs0> lltotal 20drwxrwxr-x 3 mgousr01 mongodb 4096 2015-10-10 17:46  Admindrwxrwxr-x 2 mgousr01 mongodb 4096 2017-05-11 11:12 journaldrwxrwxr-x  3 mgousr01 mongodb 4096 2015-10-10 17:43 localdrwxrwxr-x 3  Mgousr01 mongodb 4096 2017-05-11 15:47 lxvomgodb1-rwxrwxr-x 1 mgousr01  mongodb    0 2017-05-11 16:02 mongod.lock-rw-rw-r-- 1  mgousr01 mongodb   69 2015-10-10 17:09 storage.bson[email protected]:/ Mvnobss/users/mgousr01/mongodb/dbdata/rs0>[email protected]:/mvnobss/users/mgousr01/mongodb/dbdata/rs0 >[email protected]:/mvnobss/users/mgousr01/mongodb/dbdata/rs0>rm -rf mongod.lock 

The

Reboot still error, and the Mongod.lock file is generated:

[Email protected]:/mvnobss/users/mgousr01/mongodb/bin>./start_mongodb.sh about-to-fork child process, waiting until Connections.forked Process:30717error:child process failed, exited with error number 100[email Prote Cted]:/mvnobss/users/mgousr01/mongodb/dbdata/rs0>lltotal 20drwxrwxr-x 3 Mgousr01 mongodb 4096 2015-10-10 17:46 Admindrwxrwxr-x 2 Mgousr01 mongodb 4096 2017-05-11 11:12 journaldrwxrwxr-x 3 Mgousr01 mongodb 4096 2015-10-10 17:43 Locald Rwxrwxr-x 3 Mgousr01 mongodb 4096 2017-05-11 15:47 lxvomgodb1-rwxrwxr-x 1 mgousr01 mongodb 0 2017-05-11 16:15 Mongod.lo Ck

Tried a lot of times or not, continue to read the log:

2017-05-11t16:15:42.379+0800 i control  ***** server restarted ***** 2017-05-11t16:15:42.413+0800 i storage  [initandlisten] **************2017-05-11t16 : 15:42.413+0800 i storage  [initandlisten] error: journal files are  present in journal directory, yet starting without journaling  enabled.2017-05-11t16:15:42.413+0800 i storage  [initandlisten] it is  recommended that you start with journaling enabled so that  recovery may occur.2017-05-11t16:15:42.413+0800 i storage  [initandlisten]  2017-05-11t16:15:42.413+0800 i storage  [initandlisten] exception  In initandlisten: 13597 can ' T start without --journal enabled when  journal/ files are p resent, terminating2017-05-11t16:15:42.413+0800 i control  [initandlisten]  dbexit:  rc: 100

The journal file currently exists in the journal directory, and the boot process does not start journaling, and the error is thrown because journaling is started when the database is started.

Then go to the journal directory and delete the journal file for the non-normal shutdown time point:

[Email protected]:/mvnobss/users/mgousr01/mongodb/dbdata/rs0/journal>ls-lrttotal 3179664-rw-------1 Mgousr01 MongoDB 1076690944 2017-05-11 11:11 j._23-rw-------1 mgousr01 mongodb 1081876480 2017-05-11 11:11 j._24-rw-------1 mgous R01 MongoDB 1093656576 2017-05-11 11:12 j._25-rw-------1 mgousr01 mongodb 3735552 2017-05-11 15:38 j._26-rw-------1 m Gousr01 MongoDB 2017-05-11 15:38 lsn[email protected]:/mvnobss/users/mgousr01/mongodb/dbdata/rs0/journal>rm -RF j._26

Startup success:

[Email protected]:/mvnobss/users/mgousr01/mongodb/bin>lltotal 4-rwxrwxr-x 1 Mgousr01 mongodb 60 2015-08-11 17:44 Start_mongodb.sh[email protected]:/mvnobss/users/mgousr01/mongodb/bin>./start_mongodb.sh about to fork child Process, waiting until server is a ready for connections.forked Process:30927child process started successfully, parent Exi Ting

Because not very understand the role of this file directory, online access to a variety of information just know, this journal is Oracle's redo Log,mysql Binlog, for fault recovery.

The specific operation mechanism needs further study.

The environment above me is a test environment, not afraid to lose data so delete does not matter, production environment should be cautious.

"Personally, it should be possible to specify the disable journaling in the configuration file, and then go to start MongoDB, start the success and then enable the restart of the database, have not tried ~"

MongoDB Abnormal shutdown cannot start