Shell script Practice Notes (6) -- Cluster Environment configuration Detection

 

1). Background:
During cluster deployment, consistent configuration and environment settings are required. for Virtual Machine clusters, you can copy, copy, and restore cluster machines by using images. different from physical machine clusters, if there are more than one machine and many people operate and configure it, it is good for mature and competent teams and small teams who are not familiar with the environment, due to varying levels, this may lead to inconsistent environments. therefore, it is always the best practice to write scripts for automated configuration and environment verification.

2). Assuming the Application Scenario:
*) System environment:
Install cdh5, the cluster size is 16 machines, each machine has 16 CPUs, 16 GB memory, 2 SATA disks totaling 500 GB, and the operating system is centos 6.4.
Cluster machine, IP Range: 192.168.1.101 ~ 192.168.1.116.

*) Basic Requirements
When installing cdh5, you must meet the following basic requirements:
#) You need to configure the/etc/hosts file for each machine so that each machine has the domain name of all machines in the cluster.
#) You need to disable the firewall and disable enabling it.
#) Local Yum source needs to be configured
#) Try to satisfy the/mnt/disk {n} format of Disk Partitions
#) Basic synchronization of Machine Time

3). Specific implementation
*) Configure the cluster SSH without logon
Select a central control machine (stepping stone) or a machine in the cluster as the initiator of the automated script, as the system administrator, and use the root user.
#) Create an RSA key locally to generate an RSA public/private key pair
Mkdir-P ~ /. SSH
Cd ~ /. SSH
Ssh-keygen-t rsa-p''
Id_rsa (Private Key) and id_rsa.pub (Public Key) files are generated by default.

#) Edit the script
Ssh-copy-ID-I id_rsa [email protected] <target_ip>

 
#! /Bin/bashusername = "root" server_ips = ("192.168.1.101" "192.168.1.102" "192.168.1.103" 192.168.1.104 "" 192.168.1.105 "" 192.168.1.106 "" 192.168.1.107 "192.168.1.108" "192.168.1.109" "192.168.1.110 """ 192.168.1.111 "" 192.168.1.112 "" 192.168.1.113 "" 192.168.1.114 "" 192.168.1.115 "" 192.168.1.116 ") for (I = 0; I <$ {# server_ips [*]}; I ++); Do ssh-copy-ID-I ~ /. Ssh/id_rsa [email protected] $ {server_ips [I]} done
 

This step is still painful. You need to manually enter 16 passwords (16 machines ).

#) Enable RSA Verification
Edit/etc/ssh/sshd_conf

 
Rsaauthentication yespubkeyauthentication yesauthorizedkeysfile. Ssh/authorized_keysgssapiauthentication nousedns No
 

The first three items are used to enable the RSA service, and the last two items are used to solve the problem of slow response to the initial SSH connection.

#) Restart the SSH service
Service sshd restart

*) Write automated scripts

 
#! /Bin/bashusername = "root" server_ips = ("192.168.1.101" "192.168.1.102" "192.168.1.103" 192.168.1.104 "" 192.168.1.105 "" 192.168.1.106 "" 192.168.1.107 "192.168.1.108" "192.168.1.109" "192.168.1.110 """ 192.168.1.111 "" 192.168.1.112 "" 192.168.1.113 "" 192.168.1.114 "" 192.168.1.115 "" 192.168.1.116 ") # Description: # execute commands on each node and summarize the execution results to a file for comparison # Params: #$1 => command, the command to be executed #$2 => filename, the file to save the output result, used to compare the result with execute_all_servers () {ssh_command = $1 result_file = $2 Echo "Start execute... "> $ result_file for (I = 0; I <$ {# server_ips [*]}; I ++); Do echo" server_ip: $ {server_ips [I]}, execute command: '$ ssh_command' ">>$ result_file SSH [email protected] $ {server_ips [I]}" $ ssh_command ">>$ result_file echo" ====== ====================================== ">>$ result_file done}
 

#) Detect/etc/hosts files
Execute_all_servers "cat/etc/hosts" "check_hosts_result.log"

#) Disk partition and mount Detection
Execute_all_servers "DF-H; fdisk-L" "check_fdisk_result.log"

#) Firewall shutdown Detection
Execute_all_servers "service iptables status" "check_iptable_result.log"

#) Firewall close command
Execute_all_servers "service iptables stop; chkconfig -- levels 235 iptables off" "stop_iptables_result.log"

Other environment detection and environment configurations that need to be added can be implemented in a similar way. This is not perfect, but provides a solution.