Cluster Expansion: Hadoop environment Building

Hadoop Installation steps:

2. Packages that the system needs to install
   Ssh,rpm,gcc,make
   Yum-y Install Openssh-server openssh-clients

4. Detect SSH installation and RPM installation
   A. SSH localhost
   B. Yum install rpm

6. Installing Java
   A. Installing the Java-7
   RPM-UVH Java

   B. Adding and configuring a new JDK system command

   Update-alternatives–install "/usr/bin/java" "java" "/usr/java/jdk1.7.0_45/bin/java" 16001
   Update-alternatives–install "/usr/bin/java" "Javac" "/usr/java/jdk1.7.0_45/bin/javac" 16001
   Alternatives–config Java
   Alternatives–config Javac
   C. Check and test
   Update-alternatives–display Java
   Java-version

8. Add HDUser User
   A. Groupadd Hdgroup
   B. useradd-g Hdgroup HDUser
   C. PASSWC HDUser
   D. Chown-r HDUser.
   E. chmod 755 HDUser
   F. Chgrp-r Hdgroup HDUser

10. Copy the public key on the Da-master to the new machine
    Ssh-copy-id-i id_rsa.pub [email protected]

12. Modify host name and Hosts file
    127.0.0.1 localhost
    172.21.36.4 da-slave05
    172.21.101.7 WEB2
    172.21.101.30 Da-master
    172.21.101.31 da-slave01
    172.21.101.32 DA-SLAVE02
    172.21.101.33 da-slave03
    172.21.101.34 da-slave04
    :: 1 localhost6.localdomain6 Localhost6

14. Enter HDUser user Environment
    A. Su-hduser
    B. TAR-ZXF hadoop.2.2.0.tar.gz
    C. ln-s hadoop-2.2.0/

16. Editing environment variables
    Vim ~/.BAHSRC

18. modifying system parameters
    A. Turn off the firewall
    Service Iptables Stop
    Chkconfig iptables off
    Vim/etc/selinux/config

    Change into disabled

    Setenforce 0
    Service Iptables Status

    B. Modifying the maximum number of open files
    1) vim/etc/security/limits.conf

    • • Nofile 204800
        2) vim/etc/security/limits.d/90-nproc.conf
      • Soft Nproc New-max-user-processes-value

    C. Adjusting the swap partition size
    Vim/etc/sysctl.conf
    vm.swappiness = 0
    Vm.dirty_ratio = 5

    1. Installing the Lzo Package
       CD lzo-2.06
       ./configure–enable-shared
       Make
       Make install
       #编辑/etc/ld.so.conf, after joining/usr/local/lib/, execute/sbin/ldconfig
       or cp/usr/local/lib/liblzo2.*/usr/lib64/.

    10. Copy the database configuration parameter file on master to the machine

    11. Sync Time: http://www.cnblogs.com/panfeng412/p/drop-caches-under-linux-system.html
    Http://blog.sina.com.cn/s/blog_79fded950101gfc7.html

Introduction is relatively brief, to be time to post a presentation, please look forward to!

Cluster Expansion: Hadoop environment Building