The secondary of Hadoop NameNode

Namenode storage File system changes as log appended to a local file: This file is edits. When a namenode starts, it reads the status of HDFS from an image file: Fsimage, using edits from the edits log file. It then writes the new HDFs state to Fsimage and begins the usual operation, at which point edits is an empty file. Because Namenode merges fsimage and edits only at startup, edits log files become large over a period of time and run on busy clusters, a side effect is that a larger edits file will take a longer time to namenode on the next reboot.

Secondary Namenode periodically merges fsimage and edits log files and keeps the edits log file size at a certain limit. It usually runs on different machines with Namenode, and the memory requirements are the same as Namenode.

The checkpoint process begins to be controlled by the secondary namenode two configuration parameters.

Dfs.namenode.checkpoint.period, which is set to 1 hours by default, specifies the maximum delay between two checkpoints in a row
Dfs.namenode.checkpoint.txns, which is set to 1M by default, defines the number of uncheckpointed transactions in Namenode, which will force an emergency checkpoint even if the checkpoint interval has not been reached.

Secondary NameNode stores the latest checkpoints in a directory in the same way as the NameNode directory. To facilitate checkpoint mirroring is always ready to be namenode read, if necessary.

For security, how to store secondary NameNode with NameNode on the same machine?

Hadoop 1.x version:
Under the Hadoop node, modify the Masters content

[Email protected] conf]# VI Masters

For example, change the inside of localhost to HADOOP1
The effect is as follows:

Hadoop:
[Email protected] conf]# JPS
7008JobTracker
6836NameNode
7105 Jps

HADOOP1:
[Email protected] local]# JPS
6315DataNode
6470TaskTracker
6561 Jps
6384SecondaryNameNode

hadoop2.x version:

Need to add Masters file

VI Masters
Write the host name or IP address in the file that you want secondary namenode to store

The configuration file needs to be modified:
Modify Hdfs-site.xml:

<property ><name>Dfs.http.address</name><value>master:50070</value><description>The address and the base port where the Dfs Namenode Web UI would listen on. If the port is 0 then the server would start on a free port.</Description></Property ><property ><name>Dfs.namenode.secondary.http-address</name><value>slave1:50090</value>The main name here is to change the hostname</Property >

To modify the Core-site.xml file:

<property >  <name>Fs.checkpoint.period</name>  <value>3600</value>  <description>The number of seconds between and periodic checkpoints.</Description></Property ><property >  <name>Fs.checkpoint.size</name>Backup interval at log size interval<value>67108864</value></Property >

The above modification is complete, the corresponding node also makes the same modification.

separate start and stop of Secondarynamenode
Start:

bin/hadoop-daemons.sh--configconf/--hostsmastersstartsecondarynamenode

Stop it:

bin/hadoop-daemons.sh--configconf/--hostsmastersstopsecondarynamenode

hadoop2.x Secondarynamenode always in the execution start-dfs.sh after automatically up, also did not configure Secondarynamenode, how to automatically start Secondarynamenode?

Startup commands for Hadoop:
start-dfs.sh
Once the script is executed, it will start Master,slave and then Secondarynamenode.
This means that start-dfs.sh defines them, and you can modify the scripts inside them.

sudo nano start-dfs.sh

#---------------------------------------------------------# Secondary Namenodes (if any)secondary_namenodes=$ ($HADOOP _prefix/bin/hdfs Getconf-secondarynamenodes2>/dev/null)if[-N"$SECONDARY _namenodes"]; ThenEcho "Starting secondary namenodes [$SECONDARY _namenodes]""$HADOOP _prefix/sbin/hadoop-daemons.sh"--config"$HADOOP _conf_dir"--hostnames"$SECONDARY _namenodes"--script"$bin/hdfs"Start Secondarynamenodefi

We see the above code section, comment out Secondarynamenode will not get up (before doing this operation, you first back up a copy).

The secondary of Hadoop NameNode