Shell script----Install Hadoop process Summary

Start the script under $hadoop_home/bin or $hadoop_home/sbin, the shell terminal will display the output information;

Based on the output information, the information of Bash-x [Script-name] and the script itself can be used to locate where the error began.

The following are the problems in these processes, and how they are addressed;

1. No NodeManager to stop occurs when calling the Stop-yarn.sh script, closing ResourceManager and Nodemanger;

But SSH to their own slave node, JPS still show a nodemanager process;

Resolved as follows:

Point one: Hadoop_pid_dir, configuring the environment variable will generate the PID file for the Namenode process in the master node (the format of the file name is specified by the script), which holds the PID number of the Namenode process The PID file of the Datanode process is generated under the directory of each slave node, and the Secondarynamenode node;

Yarn_pid_dir, the environment variable is configured, the PID file of the ResourceManager process is generated under the directory of the master node, and the PID file of NodeManager process is generated under the directory of each slave node.

The default value for both is:/tmp;

Point two: When the kill process, depending on the signal value emitted, the operation of the process is different, such as the Kill PID and kill 9 pid, in the yarn-deamon.sh:

(stop)if[-F $pid]; ThenTarget_pid=`Cat$pid 'if Kill-0$TARGET _pid >/dev/NULL 2>&1;then #kill-0 should be to determine if the process exists         Echostopping $commandKill$TARGET _pid #此处kill默认信号选项为15, term signalSleep$YARN _stop_timeoutif Kill-0$TARGET _pid >/dev/NULL 2>&1; then #发送term信号之后, and after sleep, see if the process still exists           Echo "$command did not stop gracefully after $YARN _stop_timeout seconds:killing with kill-9"          Kill-9$TARGET _pid #发送kill信号, to the process, to forcibly kill the process;fi      Else        Echono $command to stopfi      RM-F $pidElse      Echono $command to stopfi    ;;

Point three: Show no nodemanager to stop; One is the/tmp directory of the Nodemanger PID file does not exist, one is NodeManager pid file exists, but there is no nodemanager start, excluding the latter case, because the slave node still has the NodeManager process ; You can specify the directory where the PID is generated by changing these two environment variables;

2. In one script, there are several ways to invoke another script:

1# does not produce a sub-shell (child shell), is loaded in the current process execution, after execution completes, its generated variables are still valid, after execution, will return the previous script called the script after the command
2 . $path/yarn-config.  sh #相当于source, note. There are spaces between the paths and the path; 3 4 # generates Sub-shell, the script executes in the new process, equivalent to fork (), exec (), and its change to the variable does not reflect the parent shell;
5 # The parent shell continues to execute 6 $path/yarn-daemon after the child shell finishes executing .  SH   
#不会产生sub-shell, the script exits after executing in the current process and does not return to the previous script to continue with the following command
9 #可以做个试验: exec ls; the shell exits when execution is complete;  exec java-cp $path [classname]  

3. bash-x [Script-name]

#!/usr/bin/env Bash
#hello. Class in the $home directory

Build_command () {
JAVA-CP $HOME Hello
Return 3


Var= ' Build_command '   + build_command
                      + + Java-cp/home/mickeysun Hello
                      + + return 3
                      + var= ' Hello world! '

echo $?               + Echo 3
                      3
echo $var             + echo Hello ' world! '
                      Hello world!  

The results appear as follows:
3
Hello world!

Analysis:
echo $? The return value of the Build_command is displayed;
The echo $var shows what the Build_command writes to the standard output;

The right side is the use of bash-x [Script-name], showing the results as shown on the left;

4. cmd= (): About the use of CMD

6. IFS

7. While Condition;do

Cmd

...

Cmd

Done < < ()

8. Read-d "-R ARG

9. $# [email protected] Shift $ $? ( Explained in 3)

#!/usr/bin/EnvBash #[email protected]:~$./test.sh Mickey Sun name Echo[email protected] # Mickey Sun name
Echo$# # 3Echo$0 #./test.sh
Echo$1 # mickey Echo$2 # Sun ShiftEcho$1 # Sun Echo[Email protected] # Sun Name
Echo$# # 2

Shell script----Install Hadoop process Summary