0. Modify IP, host name, Hosts fileSetupModify network card IPService Network RestartRestart the Network service for IP to take effect
Vim/etc/sysconfig/network Modify Host name vim/etc/hosts/Modify Hosts file 192.168.126.128 hadoop001192.168.126.129 hadoop002192.168.126.130 hadoop003192.168.126.131 hadoop0041. Installing the JDK environment ALT + P Open remote Connection SECURECRT Remote Transport tool SFTP put c:/........tar.gz The win on the software to linux & nbsp Mkdir-p/java/jdk Build multi-level folders TAR-ZXVF jdk----. tar.gz &NB Sp;-c /java/jdk Unzip JDK to specified path vim/etc/profile &NBS P Modify Global environment variables Export Java_home=/java/jdk/jdk1.7.0_65export path= $JAVA _home/bin: $PATHsource/etc/profile Reload profile echo$java_home print variable java_homejava-version see if environment variable configuration is installed successfully Protobuftar -ZXVF PROTOBUF-2.5.0.TAR.GZ&NBSP;CD protobuf-2.5.0./configure--prefix=/opt Configuring environment variables join/opt/binprotoc-–version 2. Installing HADOOPTAR-ZXVF hadoop-2.4.1.tar.gz-c/java/decompression hadoopls lib/native/See what files are in the extracted directory CD etc/hadoop/into the profile directory vim hadoop-env.sh Modify Profile environment variable (export java_home=/java/jdk/jdk1.7.0_65) *-site.xml*vim core-site.xml Modify configuration file (go to official website for parameter meaning) < configuration><property><name>fs.defaultfs</name><value>hdfs://hadoop001:9000< value></property><property><name>hadoop.tmp.dir</name><value>/java/ Hadoop-2.4.1/tmp<value></property></configuration> vim Hdfs-site.xml modifying HDFs Profiles < configuration><property><!--number of replicas, default 3--><name>dfs.replication</name><value>2 </VALUE></PROPERTY></CONFIGURATION>&NBSP;CP mapred-site.xml.template Mapred-site.xml Copy a sample configuration file vim Mapred-site.xml modify MapReduce profile <configuration><property><name> Mapreduce.framework.name</name><value>yarn</value></property></configuration> vim yarn-site.xml Configure the Yarn profile, or you can configure the hostname if you modify the HOSTs file mapping <configuration><!--Site specific YARN configuration Properties--><!--Master node--><property ><name>yarn.resourcemanager.hostname</name><value>hadoop001</value></property ><!--from Node--><property><name>yarn.nodemanager.aux-services</name><value> Mapreduce_shuffle</value></property></configuration> vim Slaves configuration from node (can be host name) 192.168.126.128192.168.126.129192.168.126.130&NBSP;SCP/JAVA/HADOOP-2.4.1/[email protected]:/java/ HADOOP-2.4.1/copied to other machines cd/java/hadoop-2.4.1/bin./hadoop Namenode-format formatted HDFs file system (built files and folders) Success: INFO Common. Storage:storage Directory/java/hadoop-2.4.1/tmp/dfs/name has been successfully formatted 3. Start JPS look at the Java process, There's nothing at this time CD sbin/to Sbin to find the start command./start-dfs.sh start hdfs[[email protected] sbin]# jps2488 NameNode2746 SecondaryNameNode2872 Jps2601 datanode [[email protected] ~]# jps2422 DataNode2485 jps 4. Start HDFs Client CD/ Java/hadoop-2.4.1/bin into BIN./HDFs dfs-ls/view HSFS/lower content./hdfs dfs-put/root/install.log hdfs://hadoop001:9000/files to the HDFs file system (128M per block)./hdfs dfs-ls/View Hsfs /under Content ls/java/hadoop-2.4.1/tmp/dfs/View File save location (Namenode,datanode) ls/java/hadoop-2.4.1/tmp/dfs/(datanode) only Data folder CD /java/hadoop-2.4.1/tmp/dfs/data/current/bp-343820156-192.168.126.128-1443868868316/current/finalized view files are divided into several pieces. HDFs dfs-get hdfs://hadoop001:9000/install.log download files from HDFS (specify downloaded directories after get) principle: Touch hadoop.tar.gz build an empty file cat Block1 >> hadoop.tar.gz add block to Hadoop.tar.gzcat Block2 >> hadoop.tar.gz Append Block2 to Hadoop.tar.gz ./hdfs dfshdfs other commands./hdfs dfs-chown tangwan:hadoop/install.log Change the HDFs file user owners and groups./hdfs Dfs-ls/install.log View the post-change status./hdfs dfs-chmod 600/install.log Change permissions./hdfs dfs-ls/View permissions Change permissions after users can still download but if you create a folder to give permissions, You can only specify permissions for the user to upload./hdfs DFS-DF/See how much space is left in HDFs Kb./hdfs dfs-df-h/See how much space is left in HDFs m,g./hdfs dfs-mkdir-p/aaa/bbb Create a folder in HDFs ./Hadoop fs-du-s/#查看hdfs占用空间Stop Hdfs:/java/hadoop-2.4.1/sbin Enter sbin./stop-dfs.sh stop hdfshadoop001:50070 browser interface (can download not can upload)
From for notes (Wiz)
29.Hadoop of HDFs cluster build notes
