Remote debugging of hadoop2.0 in eclipse

In the article "Compiling hadoop2.0 source code under Eclipse", I introduced in detail how to compile hadoop2.0 source code in the eclipse environment and command line environment, and briefly introduced how to build the hadoop environment, this article focuses on how to debug the source code in eclipse.

In hadoop2.0, commonly used modules include namenode, datanode, ResourceManager, nodemanager, and fsshell. The following uses the fsshell and namenode modules for debugging as an example.

Fsshell command debugging:

The fsshell command is the most basic way for hadoop clients (any node in a hadoop cluster can be used as a client) to access the hadoop cluster. For example, the bin/hadoop FS command is the command set used by the client to access the hadoop file system. Take bin/hadoop FS-ls/(displays files in the root directory of the hadoop file system) as an example.

• Find the file bin/hadoop in the hadoop directory on the client node. Open this script and find the location, for example

• As shown in:

• NOTE: If suspent is set to Y, it indicates that the wait is paused. If you know that the debugger (debugging in eclipse) is connected, the execution continues. Address = 8000 indicates listening for requests on port 8000, you can also switch to another non-conflicting port.

Because the debugging fsshell requires the hadoop runtime environment, the hadoop cluster should be started first. The startup process is not described here. After the startup is successful, we enter the command on the client, and the information shown in is displayed:

This indicates that the debugging environment has been established and listening for debugging requests starts on port 8000. The following describes how to start debugging in Eclipse:

• Open eclipse and find hadoop-common (the previous blog introduced how to import hadoop into eclipse)
• Find the fsshell. Java file in hadoop-common and open it.
• Find the main function in fsshell and set a breakpoint in the first line of main function.
• Right-click the file and choose debug as> debug configurations.... The configuration dialog box is displayed.
• Find and double-click Remote Application on the left. A new configuration page is created. Set the host to the IP address of the client, and set the port to 8000, as shown in figure

• Click Debug. Now, you can debug the command you just entered and track the execution process of the command.

Namenode debugging:

Namenode debugging is similar to the above debugging steps. The following is a brief introduction.

• To debug namenode, You need to modify the bin/HDFS file of the node where namenode is located. The modification is as follows:

 

Change

Suspend = n. If it is set to y, you can debug the Startup Process of namenode.

• Start namenode
• The namenode debugging portal locates the main function in the namenode. Java file of the hadoop-HDFS project and sets the breakpoint.
• Just like fsshell, create a remote application. Enter the host as the IP address of the namenode node, and the port is 8000.
• Click debug to start debugging.

This section describes how to debug hadoop2.0 in eclipse. Now we can use debugging to track the hadoop execution process and analyze the hadoop source code in depth. Finally, you can modify the hadoop source code and add the required functions.

This article is from the blog, If you view the original, please click http://www.cnblogs.com/meibenjin/p/3175679.html