"hadoop2.6.0" Installation + example run

Source: Internet
Author: User
Tags hdfs dfs

As the latest version of Hadoop is downloaded, the various assorted on the web are not applicable. Fortunately, the official website is clear enough. If anyone read this article, the biggest advice is crossing net .

Official website 2.6.0 Installation Tutorial: http://hadoop.apache.org/docs/r2.6.0/hadoop-project-dist/hadoop-common/SingleCluster.html

HDFS directive: http://hadoop.apache.org/docs/r2.6.0/hadoop-project-dist/hadoop-common/FileSystemShell.html

Note: 2.6.0 default is 64-bit system, if the use of 32-bit machine will always appear the following warning: This can be ignored, will not affect the results

Java HotSpot (TM) Client VM warning:you has loaded library/home/software/hadoop-2.6.0/lib/native/libhadoop.so.1.0.0 Which might has disabled stack guard. The VM would try to fix the stack guard now.
It's highly recommended that's the library with ' execstack-c <libfile> ', or link it with '-Z noexecstack '.
14/12/04 21:52:59 WARN util. nativecodeloader:unable to load Native-hadoop library for your platform ... using Builtin-java classes where applicable

1. Installing the JDK I wrote it in another article. http://www.cnblogs.com/dplearning/p/4140334.html

2. SSH password-free login http://www.cnblogs.com/dplearning/p/4140352.html

3. Configuration

Etc/hadoop/core-site.xml:

<configuration>    <property>        <name>fs.defaultFS</name>        <value>hdfs:// Localhost:9000</value>    </property></configuration>

Etc/hadoop/hdfs-site.xml:

<configuration>    <property>        <name>dfs.replication</name>        <value>1< /value>    </property></configuration>

4. $ bin/hdfs namenode-format//format

5. $ sbin/start-dfs.sh//Open process

If successful, the JPS viewing process should be

If there is no Datanode check the log

If there is an error

Java.io.IOException:Incompatible clusterids In/tmp/hadoop-root/dfs/data:namenode Clusterid = cid-2b67ec7b-5edc-4911-bb22-1bb8092a7613; Datanode Clusterid = cid-aa4ac802-100d-4d29-813d-c6b92dd78f02

Then, should be the/tmp/hadoop-root folder and the previous remnants of the file, all empty after the format, restart the program should be good.

Running Examples:

1. First create a folder on HDFs Bin/hdfs dfs-mkdir-p/user/kzy/input

Bin/hdfs dfs-mkdir-p/user/kzy/output

2. Upload some files first: Bin/hdfs dfs-put etc/hadoop//user/kzy/input upload the Etc/hadoop file to the/user/kzy/input of HDFs

3. Execution instructions

Bin/hadoop jar Share/hadoop/mapreduce/hadoop-mapreduce-examples-2.6.0.jar grep/user/kzy/input/hadoop/user/kzy/ Output/o ' Dfs[a-z. +

Note that/user/kzy/output/o is a folder that has not been created and has a warning if you use an existing folder

4. View Results

Bin/hdfs dfs-cat/user/kzy/output/o/*

Then run some wordcount, the official website in http://hadoop.apache.org/docs/r2.6.0/hadoop-mapreduce-client/hadoop-mapreduce-client-core/ mapreducetutorial.html#example:_wordcount_v1.0

Run

Bin/hadoop jar Share/hadoop/mapreduce/hadoop-mapreduce-examples-2.6.0.jar wordcount/user/kzy/input/hadoop/user/ Kzy/output/wordcount

Use

Bin/hdfs dfs-cat/user/kzy/output/wordcount/* See results

"hadoop2.6.0" Installation + example run

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.