Pig System Analysis (7) Pig Utility class

Explain

Explain is a pig-provided debugging tool that uses explain to output pig lation execution plans. It is worth mentioning that explain supports the-DOT option to output the execution plan in DOT format (dot is a graphical description language, refer to http://zh.wikipedia.org/zh/DOT%E8%AF%AD%E8%A8%80)

Code implementation detailed in Org.apache.pig.impl.plan.DotPlanDumper, this part of the implementation of our design implementation plan visualization provides a reference. The following diagram section intercepts the graphs that were generated after the physical execution plan dot file was opened using Graphviz. (Graphviz Introduction Please refer to Http://zh.wikipedia.org/wiki/Graphviz)

Illustrate

Illustrate is further than explain, it will automatically generate the sample data suitable for execution plan according to the specific statement expression conditions, and simulate execution, and the results of each step are output according to the table to simulate a complete data processing process. Code implementation details see Org.apache.pig.pen.Illustrator.

Pigcontext

The Pigcontext class contains contextual basic information that is required at various stages of the pig execution process, pigcontext from the front end to the back end, all the time until the Hadoop job phase is available. In the Map&reduce initialization method, get the Pigcontext from the Hadoop configuration.

pigcontext= (Pigcontext) objectserializer.deserialize (Job.get ("Pig.pigcontext"));

The more important information in Pigcontext is:

Exectype: Execution type, LOCAL/MR.

Hexecutionengine: Back-end execution engine.

Script files/various jar packs and so on.

Mapping of Definedfunctions:funcname and Funcspec.

Datastorage: File System interface, Dfs&lfs.

See more highlights of this column: http://www.bianceng.cnhttp://www.bianceng.cn/database/storage/