Hadoop Learning Notes (2)-building Hadoop native mode

0. Preface

There are three ways to run Hadoop. Local (Standalone) mode, pseudo-distributed (pseudo-distributed mode), distributed (fully-distributed mode). Behind the foot of the building local and pseudo-distributed, distributed readers to build their own.
References (official website, web-based materials for the shop):
Http://hadoop.apache.org/docs/r2.6.4/hadoop-project-dist/hadoop-common/SingleCluster.html#Standalone_Operation

Test environment: UBUNTU14 (readers can install their own virtual machines in VMware running Ubuntu)

1. Installing the JDK

Hadoop is installed to run on the JVM, so install the JDK. So you have to follow the JVM.

1.1 Download JDK

Download URL:
Http://www.oracle.com/technetwork/java/javase/downloads/jdk8-downloads-2133151.html
Select Linux 64-bit system, the compression suffix is tar.gz

Medium-time 8.73 version. I actually downloaded the time 8.71 version.

1.2 Decompression

I downloaded in the Downloads folder, into the Downloads folder, and then extracted directly into the/USR/LOCAL/JDK, of course, first in the/usr/local/folder to create a new JDK folder:
The command is as follows:

1.3 Configuring JDK,JRE Environment variables

I am using the vim command to modify the configuration file, so first install the Vim editing tool. The code is as follows:

Modify the BASHRC file to add environment variables

Add the following to the end of the file: (add only the contents of the red box)

Then use the Soure command to make it effective:

1.4 Verification

Input: Java-version
The ability to display the version number indicates that the JDK configuration was successful.

3. Installing Hadoop

Install local mode here

3.1 Downloads

：
Http://hadoop.apache.org/releases.html
I use the 2.6.4 version, click Binary to enter the download page.

Come down and downloads in the system.

3.2 Decompression

Create a Hadoop folder in the local folder. To store hadoop-2.6.4 after decompression

Go back to the Downloads folder to extract Hadoop:

3.3 Configuring the Hadoop environment variables

Before configuring, go to/usr/local to do a permission setting, use command: sudo chmod–r 777 Hadoop

The Hadoop folder is set to the highest read and write permissions. This makes it convenient to modify the configuration file and create folders in the directory later.

Configuring the Hadoop environment is configuring the hadoop-env.sh file. Commands such as:

Modify the Java_home path and add the Hadoop_home path (the path matches your actual location). Content such as:

To verify that the configuration was successful, enter Bin/hadoop version to view Hadoop 2.6. Version 4:

3.4 Configuring the Bin configuration environment variable for Hadoop

Configure the Bin directory in the environment variable. Edit the. bashrc file:

Add the following red block diagram.

Use the source command to make the configuration effective:

At this point, the commands under the Bin directory of Hadoop can be used in all directories, as tested below, using Hadoop version in the root directory.

The local mode is now well-equipped.

4. Test Local mode

Create the input folder as the party to be tested for the import file.
Copy all the. XML-terminated files in the Etc/hadoop directory in the Hadoop directory into input

Here you test 2 mapreduce programs, one grep, one wordcount.

4.1 Example One: grep

The example is to search all the files in the input folder, find the word with the letter A-Z followed by the start of DFS and output it.

Enter the code as follows:
Bin/hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples-2.6.4.jar grep input Output ' dfs[a-z. +

Use the cat command to view all the files in the output folder.
Input: Cat output/*

4.2 Example Two: WordCount

This example searches all the files in the input folder, counts the occurrences of all the words, and outputs them in the Output/wordcount folder.

View with the Cat command

The results are as follows:

To this Hadoop native mode configuration is complete.

Xianming

Hadoop Learning Notes (2)-building Hadoop native mode