Installation configuration for Hadoop under Windows

Because I recently recently have been learning Hadoop, exposure to a lot of theory, but want to go deep into the whole Hadoop platform, it must combat training, first step, of course, first build good one Hadoop platform first. But the pit daddy is, Hadoop is required to install in the Linux environment, under Windows is not directly run. So you can only get a Cygwin under windows and throw the Hadoop installer inside. My impression of Cygwin has not been very good, previously used this simulation of the Linux environment, and later found that there will always be a variety of environmental problems, very time-consuming. So I guess, now that I'm building a Hadoop platform on top, it's definitely not a simple event. It turned out that I did encounter a lot of problems.

The first problem encountered first is the difference on the configuration file. Before I downloaded the Hadoop installation version file on the Internet Hadoop2.0 then, my installation tutorial will be 1.0, the configuration file inside the corresponding configuration file is not found. Like what In Hadoop2.0 there is no mapred-site.xml, instead of mapred-site.xml.template file, where the directory page is not called Conf directory, so, 1th, build a Hadoop platform, to the corresponding platform to install.

2. Before configuring the Core-site,hdfs-site,mapreduce-site file, first install the SSH service, because Hadoop communication will require SSH authentication, will also produce a key file, with a key file, you can achieve a future password-free login, This step is necessary, and if you do not perform this pre-operation, the following command will be executed in error.

JDK installation and path configuration under 3.Cygwin because the Hadoop platform requires the Java environment and requires the JDK to be installed in the Cygwin environment, but our JDK is installed under the Windows disk. The first place to locate the address to your installed address, by cgwdrive+ your actual installation location, or you directly find the original installation directory, directly copied to the Cygwin directory, and then the path settings. The JDK settings are important, and many of the subsequent commands are based on this.

4. The last one is the configuration of the 3 large configuration files, if you do not intend to configure, that is the default stand-alone mode, the configuration of the operation is equivalent to have Datanode, NameNode, HDFs and so on, but all on this machine, a pseudo-distributed mode, this is very simple, is to define the port number, and some descriptive information.

5. Finally, before running the entire service, the HDFS should be formatted, Hadoop Hdfs-format, In the final side is start-all.sh, in different versions of Hadoop, start-all.sh in different directories, version 1.2 is in the Bin directory, version 2.0 is under Sbin, CD to which directory to perform the operation needs attention.

The above operation is I in the process of building a platform encountered problems, the above is I think it is easier to make mistakes, the other problems in the web search is a bunch of tutorials, similar.

Installation configuration for Hadoop under Windows