Pig and Python (pig and Python)

Python is really everywhere in the country.

Pig 0.9 after python as an embedded voice, using the Jython interpreter to use the python2.5 feature, this interface is the topmost org.apache.pig.scripting.Pig
First the Python script translates some pig latin. Then pass the variable defined in Python to it and run it last.

1) Pig.compile or compilefromfile to pre-compile the code
2) The Bind method binds a variable in the control flow to a variable in the Pig Latin script and returns a Boundscript object
3) for the Boundscript object, the ability to call the Runsingle method to run him, return a Pigstat object, assuming that during the binding process the Pig object is bound to a set of map including the number of parameters, it is called the Run method, the same return a Pigstats object.

A separate instance of a user-written UDF is built and runs in each map or reduce task, and the constructor parameter is a way of passing information to the user UDF.

Python corresponds to the type of pig

int number
Long number
Float number
Double number
Chararray string
ByteArray string
Map Dictionary
Tuple tuple
Bag List Oftuples

Pig's load function is created from Hadoop-based InputFormat, and the base class is Loadfunc,loadfunc's default implementation is for HDFs, and Pig provides a way for the load function to initialize itself by providing the Preparetoread method. Once the user's load function implements the GetSchema method, the LOAD statement no longer needs to define their schema.

Similarly. The storage function is built on Hadoop-based Outoutformat. A tuple of pig is accepted, then a healthy-value pair is created based on the output of a good thing, and the base class is Storefunc,pig will call the stored function Preparetowrite method on each map or reduce task. Putnext It is the core method of storage functionality.

Copyright notice: This article Bo Master original article. Blog, not reproduced without consent.

Pig and Python (pig and Python)