Hadoop realizes remote login and debugging

Configuring Remote Logins

1) Set up Hadoop on your own Linux machine, please refer to the detailed procedure: http://www.cnblogs.com/stardjyeah/p/4641554.html

2) Modify the Hosts file for Linux

# vim/etc/hosts

Added in the bottom line of the Hosts file, in the following format:

The first part: Network IP address.

Part two: hostname. Domain name, note that there is a half-width point between the host name and the domain name.

The second part: hostname (hostname alias), is actually the host name.

For example: 192.168.1.109 hadoop-master.localdomain hadoop-master

Of course, each line can also be two parts, that is, the host IP address and host name;

For example: 192.168.1.109 hadoop-master

3) Modify the Core-site.xml,yarn-site.xml,mapred-site.xml,hdfs-site.xml under Hadoop

Change all of the localhost into hadoop-master.

4) Restart the Linux system, then open all the HDFs and yarn processes, enter 192.168.1.109:8088 192.168.1.109:50070 on the browser of another machine 192.168.1.109:50090 see if you can get the corresponding page Surface (PS: Correct page in the installation configuration of Hadoop), if there is, in the cmd with telnet command to connect the above three ports, if there is no problem, the configuration is successful.

5) If the above 4 steps are passed, then congratulations, you can start remote access to your Hadoop, you can use SECURECRT to emulate the terminal, you can use the FileZilla file transfer and so on.

Configuring Remote Debugging

1. First prepare a hadoop2.x eclipse plugin, please refer to the following steps: http://blog.csdn.net/qingmu0803/article/details/39665147, this blogger uses Eclipse. Kepler, and I use MyEclipse 10, and its plugin is also available.

2. Unzip the Hadoop into the corresponding directory and configure the Hadoop_home and path (similar to the JDK configuration) to replace the bin folder under the Hadoop root directory (the bin folder download link to replace: http://pan.baidu.com/s/ 1sjupu3n) then enter Hadoop in CMD if the following display indicates success

3. Errors that may occur

The following error may be reported during the debugging run:

org.apache.hadoop.io.nativeio.nativeio$windows. Access (nativeio.java:557)

This should be a bug in Hadoop, let's start by looking at the source code

If after replacing the local Hadoop/bin folder also reported this error, then we can only modify the source code, the Red box section of the picture is commented out, changed to return True, and in the project root directory to create the package shown, and the modified class file into the test

I've changed this file. Download Link: Http://pan.baidu.com/s/18Xdgi

4. Formally enter the commissioning phase

Remote Debugging Hadoop-hdfs Demo:

Http://www.cnblogs.com/stardjyeah/p/4643618.html

Remote Debugging Hadoop-mapreduce Demo:

Http://www.cnblogs.com/stardjyeah/p/4643628.html

Hadoop realizes remote login and debugging