Read and analyze the KFS source code of the Distributed File System (II): metaserver metadata persistence

The metaserver metadata persistence of the KFS file system adopts the checkpoint + Log method,The following uses the source code to analyze the persistence mechanism and implementation details of metadata in KFS metaserver.

1. Related source code files

KFS metaserver metadata persistenceCodeThe directory is KFS-[version]/src/CC/meta, where,KFSThe source code of metadata persistence is as follows:

(1) Meta/statup. CC:ResponsibleKFSIn the startup process.CheckpointAndLog.

(2) Meta/checkpoint. CC: ResponsibleMetadataOfCheckpointingOperation.

(3) Meta/restore. CC: From savedCheckpointRebuildingIncluree(Organized in the form of B + tree ).

(4)Meta/logger. CC:IsMetadata.

(5)Meta/replay. CC:InCheckpointAfter recovery, redoLogAll operations in the log file.

(6) Meta/kfstree. h: Build a newInclureeThat is, the initialization root directory is"/".

2. Checkpoint and log mechanisms

A log is usually a record of a completed processing operation by the system or software for future use for system recovery. It is generally in text format. The checkpoint mechanism synchronizes the modified data blocks in the memory with the data files on the disk.

In the metadata persistence of KFS, allCheckpoint + Log method,I think it is mainly due to the following considerations:

(1) record necessary checkpoint, ensure that the file system metadata is follow serialization requirements, permanent persistently store data on disks, ensure data synchronization and consistency between memory and hard disk. when the system recovers next time, the system restores the database directly according to the deserialization requirements, quickly re-build the metadata tree of KFS tree .

(2)Eckpoint and log)To shorten the KFS system startup and recovery time. When the system recovers,FirstRestore the system to the last checkpoint state (that is, re-build the B + tree). Then, you only needYou can redo the operations in the log after the checkpoint, instead of redo all the operations in the log,This effectively shortens the system recovery time.

3. Metadata persistence

In KFS , log operations are performed by logger. CC is automatically completed, by default, logs are performed every 10 minutes (write switching); the checkpoint operation is performed by checkpoint. CC is implemented, but it is manually completed through offline operations of logcompactor_main.cc , the main function works as follows:

1. LoadLogAndCheckpointFile directory;
2. Restore the lastCheckpointFile;
3. Last redoCheckpointAllLogs;
4. SetInclureeAll leaf nodes write newCheckpointFile.

4. Metadata Loading Process

When the KFS file system is stopped,Metaserver metadata is persistently stored on a physical disk (checkpoint and log files);When KFS is started next time, the persistent data willStarted by KFS metaserverProgramThe attached method is shown in the call relationship of the related method:

 

Where,

(1) After KFS is started, pass the logdir and checkpoint directory cpdir information to the KFS: kfs_startup () function, which first calls KFS: logger_setup_paths () set the KFS log directory;

(2) then, the KFS: kfs_startup () function will continue to call KFS: checkpointer_setup_paths () to set the checkpoint directory of KFS;

(3) Next, the KFS: kfs_startup () function will continue to call the KFS: setup_initial_tree () function to initialize the B + tree primary REE of metaserver, which is divided into the following two situations:

(4.1) if the latest checkpoint file exists, call the restorer: rebuild () function. Based on the load checkpoint file, construct the three tree after initialization;

(4.2) Otherwise, call the KFS: incluree. new_tree () function to initialize a new incluree tree.,Contains only the root directory"/", And set the "." and "..." link items associated with it;

(5) returnIn the KFS: kfs_startup () function, call the replay: playalllogs () function to redo the operations in all logs after the last Checkpointing;

(6) In the KFS: kfs_startup () function, call the KFS: logger_init () function to start logging and set the log rotation interval;

(7) In the KFS: kfs_startup () function, call the KFS: checkpointer_init () function to initialize the checkpoint.

So far, the restoration process of persistent metadata in the KFS file system has been completed.