Spark Tutorial - Building a Spark Cluster - Configuring Hadoop Standalone Mode and Running Wordcount (2)

Source: Internet
Author: User
Keywords Running stand-alone installation
Tags aliyun configuration default directory file hadoop html http

Previous: http://www.aliyun.com/zixun/aggregation/13383.html "> Spark Tutorial - Building a Spark Cluster - Configuring Hadoop Standalone Mode and Running Wordcount (1)

Install rsync

Our version of Ubuntu 12.10 installed rsync acquiescence, we can through the following order to install or update rsync

3. Install hadoop, home forest to download the latest stable version of hadoop save the computer in the following location:

Create / usr / local / hadoop directory and extract the downloaded Hadoop / usr / local / hadoop:

Configure the Java installation information in hadoop-env.sh.

Enter /usr/local/Hadoop/Hadoop-1.2.1/conf

Open hadoop-env.sh:

Press Enter to enter the configuration file:

In front of us when installing Java Java installed in "/usr/lib/java/jdk1.7.0_60", then we hadoop-env.sh configuration file to add the following configuration information

Save and exit.

Use the source command to make hadoop-env.sh configuration information take effect:

At this point Hadoop stand-alone mode configuration success!

In order to facilitate our boot after the boot can also use the Hadoop bin directory immediately related commands, the bin directory can be configured to "~ / .bashrc" file, then start the system automatically reads "~ / .bashrc" The contents of the file, I can always use the Hadoop command:

Modified file content:

Save and exit, use the following command to make the configuration take effect:

5. Verify Hadoop version information, using the "hadoop version" command:

6. Running Hadoop comes WordCount example:

First in the Hadoop directory to create an input directory and copy all the files under conf to the directory, you need the following command:

Use the hadoop command to run the native wordcount program and output the result to output:

Running process is as follows:

Always run know completion (because the operation is a bit long, the middle omitted some running process, only the beginning and the end section):

View the results of the operation:

After the command is executed, Wordcount displays the word statistics of several files in the input of input, as follows (a part of it is taken straight):

At this point, stand-alone mode of construction, configuration and operation of the test completely successful!

Related Article

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.