Hadoop Standalone Mode deployment

One, Hadoop deployment model

Stand-alone mode: Run as a standalone Java process on a separate machine by default, primarily for debugging environments

Pseudo-Distribution mode: Simulates a distributed multi-node environment on a single machine, with each Hadoop daemon running as a standalone Java process

Fully distributed mode: a real production environment, built in a fully distributed cluster environment

Ii. adding users and Groups

$ sudo adduser Hadoop # #创建用户hadoop $ sudo usermod-g sudo hadoop # #将hadoop用户添加进su Do user groups

Third, install the relevant software

$ sudo apt-get update$ sudo apt-get-y install openssh-server rsync openjdk-7-jdk$ sudo service ssh restart$ java-version

650) this.width=650; "src=" Http://s5.51cto.com/wyfs02/M02/77/A4/wKiom1ZqdkLT79muAAAYyRv7iTA099.png "title=" QQ picture 20151211150733.png "alt=" Wkiom1zqdklt79muaaayyrv7ita099.png "/>

Four, configure SSH password-free login

$ su-l Hadoop # #切换至hadoop用户 $ ssh-keygen-t rsa-p "" # #配置SSH免密登录 $ cat ~/.SS H/id_rsa.pub >> ~/.ssh/authorized_keys # #将公钥追加到authorized_keys中 $ ssh localhost # #验证

V. Configuration of Hadoop

① Download and install Hadoop

$ su-l hadoop$ wget http://apache.fayea.com/hadoop/common/hadoop-2.7.1/hadoop-2.7.1.tar.gz $ tar-zxvf hadoop-2.7.1.tar.gz$ sudo mv Hadoop-2.7.1/usr/local/hadoop

650) this.width=650; "src=" Http://s3.51cto.com/wyfs02/M00/77/A5/wKiom1ZqfYHjL61cAAATfLrD9O4929.png "title=" QQ picture 20151211153831.png "alt=" Wkiom1zqfyhjl61caaatflrd9o4929.png "/>

② Configuring Hadoop

A. Adding environment variables

$ vi/home/hadoop/.bashrc#hadoop startexport Java_home=/usr/lib/jvm/java-7-openjdk-amd64export HADOOP_INSTALL=/usr/ Local/hadoop/export path= $PATH: $HADOOP _install/binexport path= $PATH: $HADOOP _install/sbinexport hadoop_mapred_home = $HADOOP _installexport hadoop_common_home= $HADOOP _installexport hadoop_hdfs_home= $HADOOP _installexport YARN_HOME= $HADOOP _installexport hadoop_common_lib_native_dir= $HADOOP _install/lib/nativeexport hadoop_opts= "-  Djava.library.path= $HADOOP _install/lib "#HADOOP end$. /HOME/HADOOP/.BASHRC # #重载使其生效

650) this.width=650; "src=" Http://s1.51cto.com/wyfs02/M02/77/A5/wKiom1ZqfQSQy-7EAAA_xMvga78946.png "title=" QQ picture 20151211153629.png "alt=" Wkiom1zqfqsqy-7eaaa_xmvga78946.png "/>

VI. Testing

① Creating input data

$ su-l hadoop$ cd/usr/local/hadoop$ sudo mkdir input$ sudo cp/etc/passwd input/

② performing Hadoop wordcount applications (Word frequency statistics)

$ bin/hadoop Jar Share/hadoop/mapreduce/sources/hadoop-mapreduce-examples-2.7.1-sources.jar Org.apache.hadoop.examples.WordCount Input Output

650) this.width=650; "src=" Http://s2.51cto.com/wyfs02/M02/77/A6/wKiom1ZqhXmw05xGAABwmLjxay4549.png "title=" QQ picture 20151211161232.png "alt=" Wkiom1zqhxmw05xgaabwmljxay4549.png "/>

③ to view generated word statistics

$ Hadoop dfs-ls output-rw-r--r--1 hadoop hadoop 0 2015-12-18 13:18 output/_success-rw-r--r--1 Hadoop hadoo P 2128 2015-12-18 13:18 output/part-r-00000$ Hadoop dfs-cat output/*

attached :

To configure a single-node Hadoop document: http://hadoop.apache.org/docs/r2.7.1/hadoop-project-dist/hadoop-common/SingleCluster.html# Standalone_operation

This article is from the Notepad blog, so be sure to keep this source http://wangzhijian.blog.51cto.com/6427016/1765696

Hadoop Standalone Mode deployment