Spark Tutorial - Building a Spark Cluster - Configuring Hadoop Standalone Mode and Running Wordcount (2)

Previous: http://www.aliyun.com/zixun/aggregation/13383.html "> Spark Tutorial - Building a Spark Cluster - Configuring Hadoop Standalone Mode and Running Wordcount (1)

Install rsync

Our version of Ubuntu 12.10 installed rsync acquiescence, we can through the following order to install or update rsync

3. Install hadoop, home forest to download the latest stable version of hadoop save the computer in the following location:

Create / usr / local / hadoop directory and extract the downloaded Hadoop / usr / local / hadoop:

Configure the Java installation information in hadoop-env.sh.

Enter /usr/local/Hadoop/Hadoop-1.2.1/conf

Open hadoop-env.sh:

Press Enter to enter the configuration file:

In front of us when installing Java Java installed in "/usr/lib/java/jdk1.7.0_60", then we hadoop-env.sh configuration file to add the following configuration information

Save and exit.

Use the source command to make hadoop-env.sh configuration information take effect:

At this point Hadoop stand-alone mode configuration success!

In order to facilitate our boot after the boot can also use the Hadoop bin directory immediately related commands, the bin directory can be configured to "~ / .bashrc" file, then start the system automatically reads "~ / .bashrc" The contents of the file, I can always use the Hadoop command:

Modified file content:

Save and exit, use the following command to make the configuration take effect:

5. Verify Hadoop version information, using the "hadoop version" command:

6. Running Hadoop comes WordCount example:

First in the Hadoop directory to create an input directory and copy all the files under conf to the directory, you need the following command:

Use the hadoop command to run the native wordcount program and output the result to output:

Running process is as follows:

Always run know completion (because the operation is a bit long, the middle omitted some running process, only the beginning and the end section):

View the results of the operation:

After the command is executed, Wordcount displays the word statistics of several files in the input of input, as follows (a part of it is taken straight):

At this point, stand-alone mode of construction, configuration and operation of the test completely successful!