Hadoop state Transition Graph generation

Event-driven and state-shifted, to a large extent, to ensure the smooth advancement of large-scale systems. State transitions have the following benefits:

1. A state transition diagram can be generated. When discussing a function, you can discuss the diagram directly.
2. A state transfer diagram, through the lock, producer/consumer mode, etc., can ensure that a point in time only one state transfer, and thus ensure the correctness of the state
3. In the UT (unit test), you just need to test each transfer path.

Hadoop also employs this approach, such as the state transfer of MapReduce, the management of resources and tasks in NM and RM.

Build method

The state transition diagram inside Hadoop can be generated through a program. Method: Use class Visualizestatemachine to specify three parameters at run time: the name of the diagram, the class that contains the event transfer (separated by "," between classes), the generated file name (. gv format), as

$ hadoop org.apache.hadoop.yarn.state.VisualizeStateMachine TaskStateChange org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl TaskImpl.gv

The resulting diagram is as follows

All Hadoop state transition graph generation Methods MVN command generation

Reference MAPREDUCE-2930

$mvn-Pvisualize compile$dot-Tpng NodeManager.> NodeManager.-Tpng ResourceManager.> ResourceManager.-Tpng MapReduce.> MapReduce.png;

Generated by Visualizestatemachine

There are roughly three types of state transitions used in Hadoop: MapReduce, NodeManager, ResourceManager, and the script to generate these three classes of transfer graphs

MapReduce related

Hadoop org. Apache. Hadoop. Yarn. State. VisualizestatemachineMapReduce org. Apache. Hadoop. MapReduce. V2. App. Job. Impl. Jobimpl, org. Apache. Hadoop. MapReduce. V2. App. Job. Impl. Taskimpl, org. Apache. Hadoop. MapReduce. V2. App. Job. Impl. TaskattemptimplMapreduce. GV

NodeManager related

Hadoop org. Apache. Hadoop. Yarn. State. VisualizestatemachineNodeManager org. Apache. Hadoop. Yarn. Server. NodeManager. Containermanager. Application. Applicationimpl, org. Apache. Hadoop. Yarn. Server. NodeManager. Containermanager. Container. Containerimpl, org. Apache. Hadoop. Yarn. Server. NodeManager. Containermanager. Localizer. LocalizedresourceNodeManager. GV

ResourceManager related

Hadoop org. Apache. Hadoop. Yarn. State. VisualizestatemachineResourceManager org. Apache. Hadoop. Yarn. Server. ResourceManager. Rmapp. Attempt. Rmappattemptimpl, org. Apache. Hadoop. Yarn. Server. ResourceManager. Rmapp. Rmappimpl, org. Apache. Hadoop. Yarn. Server. ResourceManager. Rmcontainer. Rmcontainerimpl, org. Apache. Hadoop. Yarn. Server. ResourceManager. Rmnode. RmnodeimplResourceManager. GV

Hadoop state Transition Graph generation