Using shell scripts to filter inaccessible nodes in Hadoop

Recently used a cluster HP1, because the maintenance of the cluster of people do not give force, the node always over a period of time dropped one or two. When Hadoop was restarted today, HDFs was in protected mode.

decided to filter out all the inaccessible nodes in the slaves node, so I wrote a small script, which is recorded here, and it is convenient to use it directly.

PS: Written in C shell

The code is as follows:

#!/bin/cshif ($ #argv < 1) thenecho "Usage: $ host_file" exit 1endifset NODES = ' cat $ ' foreach NODE ($NODES) ping-q -C 1 $NODE >/dev/nullif ($ = = 0) Thenecho $NODEendifend

Incidentally, a script with an SSH check is included:

#!/bin/cshset nodes = ' Cat slaves ' foreach I ($nodes) echo $issh $i exitend

Using shell scripts to filter inaccessible nodes in Hadoop