Hadoop deployment script sharing

Recently, the Hadoop cluster deployment method that is not connected via ssh has been abandoned, and the method of ssh key verification has been returned. This is a little troublesome. Each machine must upload a public key. I am a very lazy person, so I can write a few small scripts to complete the distribution of public keys on a machine.

First, generate an ssh key script

#! /Bin/sh
Ssh-keygen-t rsa-p'-f ~ /. Ssh/id_rsa
Cp ~ /. Ssh/id_rsa.pub ~ /. Ssh/authorized_keys

Ssh-keygen generally requires passphrase input, but usually three carriage returns are passed. I am lazy input, and I don't need to add-P.

Then add the public key to the slave node script.

#! /Bin/sh
Read-p "Enter the IP address of the remote server:" ip
Ssh-copy-id-o StrictHostKeyChecking = no-I ~ /. Ssh/id_rsa.pub root @ $ ip
Ssh root @ $ ip 'sed-I "s/^ # RSAAuthentication \ yes/g"/etc/ssh/sshd_config'
Ssh root @ $ ip 'sed-I "s/^ # PubkeyAuthentication \ yes/PubkeyAuthentication yes/g"/etc/ssh/sshd_config'
Ssh root @ $ ip 'sed-I "s/^ # PermitRootLogin \ yes/g"/etc/ssh/sshd_config'
Ssh root @ $ ip 'service sshd restart'
Hostname = 'ssh root @ $ {ip} 'hostname''
Echo "add the Host Name and IP address to the local/etc/hosts file"
Echo "$ ip $ hostname">/etc/hosts
Echo "the remote host name is called $ hostname. Check/etc/hosts to ensure that the host name and IP address are added to the host list file"
Echo "host public key copied"

Then the third script reads the host list and copies/etc/hosts to all hosts.
#! /Bin/sh
Cat/etc/hosts | while read LINE
Do
Ip = 'echo $ LINE | awk' {print $1} '| grep-v ":" | grep-v "127.0.0.1 "'
Echo "Copying/etc/hosts to $ {ip }"
Scp-o StrictHostKeyChecking = no/etc/hosts root $ {ip}:/etc/
Done

No explanation

Build a Hadoop environment on Ubuntu 13.04

Cluster configuration for Ubuntu 12.10 + Hadoop 1.2.1

Build a Hadoop environment on Ubuntu (standalone mode + pseudo Distribution Mode)

Configuration of Hadoop environment in Ubuntu

Detailed tutorial on creating a Hadoop environment for standalone Edition

Build a Hadoop environment (using virtual machines to build two Ubuntu systems in a Winodws environment)