Installing the Hadoop tutorial on Windows

Installing the Hadoop tutorial on Windows

See 2010.1.6 www.hadoopor.com/[email protected]

1. Installing the JDK

Installing the JRE is not recommended, but it is recommended to install the JDK directly because the JRE can be installed at the same time when the JDK is installed. The development of the MapReduce program and the compilation of Hadoop depend on the JDK, which is not enough.

jre:http://www.java.com/zh_cn/download/manual.jsp

jdk:http://java.sun.com/javase/downloads/index.jsp, download Java SE.

2. Installing Cygwin

Before installing Cygwin, you have to download the Cygwin installer setup.exe first.

Cygwin Installer Download Address: Http://www.cygwin.com/setup.exe, of course, you can download Cygwin installer from Http://www.cygwin.cn/setup.exe, but if During installation, you will only be able to download from Http://www.cygwin.com/setup.exe, as shown in the error, and this tutorial downloads the Cygwin 1.7.1 version.

The Cygwin installer Setup.exe's storage directory is free of any request. When the Setup.exe download succeeds, run

Setup.exe, pop up the dialog box as shown:

In the dialog shown, click "Next" and go to the dialog box as shown:

In the dialog box shown, select Install from the Internet, and then click Next to enter the dialog box as shown:

In the dialog box shown, set the installation directory of Cygwin, install for select "All Users", the Default Text File Type Select "Unix/binary", and then click "Next" to enter the dialog as shown:

In the dialog box shown, set the Cygwin installation package to the directory, and then click "Next" to enter the dialog box as shown:

In the dialog box shown, select "Direct Connection" and then click "Next" to enter the dialog as shown:

In the dialog box shown, click "Next" to enter the dialog as shown:

In the dialog box shown, you may be prompted to click "OK" as shown in the "Setup Alert" dialog box.

After entering the "Select Packages" dialog box, you must ensure that "OpenSSL" under "Net Category" is installed, as shown in:

If you also want to compile Hadoop on eclipse, you must also install sed under Base Category, as shown in:

In addition, it is recommended that the "Editors category" under the Vim installation to facilitate the Cygwin directly modify the configuration file, "Devel category" under the Subversion recommended installation, as shown in:

When this is done, click "Next" in the "Select Packages" dialog box and go to the Cygwin installation package download process as shown in the following:

Wait for the installation package to be downloaded, and when the download is complete, it automatically enters the dialog box as shown:

In the dialog box shown, select "Create icon on desktop" to make it easy to launch Cygwin directly from the desktop, then click the "Finish" button. At this point, the Cgywin has been installed and the contents of the installation directory are as follows:

3. Configure Environment variables

Environment variables that need to be configured include path and Java_home:java_home to the JRE installation directory, the bin directory of the JDK, the bin directory of Cygwin, and the Usr\bin directory for Cygwin must be added to the PATH environment variable , as shown in:

4. Installing the SSHD Service

Click on the Cygwin icon on the desktop to launch Cygwin, execute the ssh-host-config command as shown:

When executing ssh-host-config, when the input yes/no is required, select Enter No, as shown in:

If it is a version prior to Cygwin 1.7, the Ssh-host-config display interface looks like this:

When you see "having fun", it generally indicates that the SSHD service was installed successfully, as shown in. Next, you need to start the sshd service.

5. Start the SSHD service

Right -click on the " My Computer " icon on the desktop and click on the " manage " menu to go to Windows Computer Management, as shown in:

In the dialog box shown, select "CYGWINsshd", eject right -click, and start the CYGWIN sshd service, after success, as shown in:

When the status of CYGWIN sshd is " started ", the next step is to configure SSH login.

6. Configure SSH Login

Execute the ssh-keygen command to generate the key file as shown in:

In the dialog box shown, if you need to enter, press the ENTER key directly, if you do not make an error, you should press ENTER three times. Next, generate the authorized_keys file, as shown in the following:

As shown, the authorized_keys file can be generated in just two steps:

CD ~/. ssh/

CP Id_rsa.pub Authorized_keys

After you do this, the Exit command exits the Cygwin window, and if you do not do this, the following operation may encounter an error. Next, rerun Cygwin, execute the ssh localhost command, and when you first execute SSH localhost, you will see a hint like the one shown, enter Yes, and then enter:

If you are a Windows domain user, you may experience problems with the following error message:.

This error has no workaround, the problem is resolved, you can focus on the Hadoop technology Forum paste: http://bbs.hadoopor.com/thread-348-1-1.htmL (cygwin< /c5>1.7.1 version ssh issue ). Otherwise, if successful, when executing the WHO command, you can see the information as shown:

Now that the SSH login is successfully configured, you can start installing Hadoop.

7. Download the Hadoop installation package

Hadoop installation package:

Http://labs.xiaonei.com/apache-mirror/hadoop/core/hadoop-0.20.1/hadoop-0.20.1.tar.gz

8. Installing Hadoop

Extract the Hadoop installation package hadoop-0.20.1.tar.gz to the D:\hadoop\run directory ( can be modified to another directory ) as shown in:

Next, you need to modify the Hadoop configuration file, which is located in the Conf subdirectory, which is a total of four files, hadoop-env.sh, Core-site.xml, Hdfs-site.xml, and Mapred-site.xml, respectively. In the Cygwin environment, masters and slaves two files do not need to be modified.

? Modify Hadoop-env.sh

Just modify the Java_home to the JDK's installation directory, and note that the JDK must be 1.6 or later.

? Modify Core-site.xml

To simplify the Core-site.xml configuration, copy the Core-default.xml file under the D:\hadoop\run\src\core directory to the D:\hadoop\run\conf directory and Core-default.xml The file name is changed to Core-site.xml. Modify the value of the Fs.default.name as follows:

The port number in the 8888, can be changed to other unoccupied ports.

? Modify Hdfs-site.xml

To simplify the Hdfs-site.xml configuration, copy the Hdfs-default.xml files under the D:\hadoop\run\src\hdfs directory to the D:\hadoop\run\conf directory and hdfs-default.xml the file name Change to Hdfs-site.xml. There is no need to make any other changes.

? Modify Mapred-site.xml

To simplify the Mapred-site.xml configuration, the Mapred-default.xml in the D:\hadoop\run\src\mapred directory is

The file is copied to the D:\hadoop\run\conf directory, and the Mapred-default.xml file name is changed to Mapred-site.xml.

The port number in the 9999, can be changed to other unoccupied ports. Here, Hadoop announces that installation is complete and you can start to experience Hadoop!

9. Start Hadoop

In Cygwin, go to the bin directory of Hadoop and run./start-all.sh start Hadoop, you can execute the./hadoop fs-ls/Command after successful startup, and view the root directory of Hadoop as shown in:

If you are running MapReduce, refer to the other documentation, which concludes this tutorial.

Install the Hadoop tutorial on Windows (GO)