Pig System Analysis (5) from logical plan to physical plan

Physical plan generation process

The optimized logical execution plan is logtophytranslationvisitor processed to generate a physical execution plan.

This is a classic Vistor design pattern application scenario.

Where the Logtophytranslationvisitor visit () is the entry method, through Dependencyorderwalker traversal handles each logicalrelationaloperator in the logical execution plan. Dependencyorderwalker traverses the nodes in the DAG in a dependent order, ensuring that it is accessed only if all the predecessors of the node are accessed. The core logic is as follows, doallpredecessors recursively calls itself, adding nodes that meet no precursors to the FIFO queue, and the resulting effect is equivalent to sequentially accessing the graph topology.

public void Walk (planvisitorvisitor) throws Frontendexception {     
        list<operator> FIFO = new arraylist< Operator> ();     
        Set<operator> seen = new hashset<operator> ();     
        List<operator> leaves = plan.getsinks ();     
        if (leaves = null) return;     
        for (Operator op:leaves) {     
            doallpredecessors (OP, seen, FIFO);     
        }     
        for (Operator Op:fifo) {     
            op.accept (visitor);     
        }     
}

Next, each logicalrelationaloperator in turn calls the Logtophytranslationvisitor corresponding visit method to process itself, translates into physicaloperator. Finally, the complete logical execution plan is generated. The following figure is the visit operator method in Logtophytranslationvisitor.

Physical plan Structure

Analyze the execution plan for code generation in Pig System Analysis (3) before parsing, as shown in the figure: