Pig installation and configuration tutorial

From the original station, the technical content is ≈ 0, translated from hadoop-the definitive guide.

Pig is a project donated by Yahoo to Apache. It is an SQL-like language and an advanced query language built on mapreduce, compile some operations into map and reduce of the mapreduce model, and you can define your own functions. This is another clone Google project developed by Yahoo: sawzall.

Pig is a client applicationProgramEven if you wantHadoopClusterRunPig does not need to attach anything to the cluster. The installation of pig is usually simple:

1. Install java6 (cygwin should be installed in Windows) and set java_home.

% Export java_home =/home/Tom/JDK1.6

2, to http://hadoop.apache.org/pig/releases.html download a stable release version (currently 0.5.0, can run on the Hadoop-0.20. *), decompress to your workspace:

% Tar xzf pig-x.y.z.tar.gz

3. For convenience, you can put the pig program directory in the command line path, for example:

% Export pig_install =/home/Tom/pig-x.y.z

% Export Path = $ path: $ pig_install/bin

After logging out or restarting, you can use pig-help to check the help. Is the installation simple enough? :)

Pig has two modes:

One is the local mode, that is, the local mode. In this mode, pig runs in a JVM and accesses the local file system. It is only suitable for small-scale datasets and is generally used to experience pig. In addition, it does not use hadoop's local runner. Pig converts the query to a physical plan and then executes it on its own.

Enter

% Pig-x local

You can enter the local mode.

Another mode is hadoop. In this mode, pig converts queries to corresponding mapreduce jobs and submits them to the hadoop cluster for running, the cluster can be real distributed or pseudo distributed. If you want pig to know hadoop, you need to tell it hadoop'sVersionAnd some key daemon information (that is, the address and port of namenode and jobtracker ). For example, the following can allow pig to connect to any hadoop0.20:

% Export pig_hadoop_version = 20

Next, specify the cluster namenode and jobtracker. There are two ways to add your hadoop conf address to the pig classpath:

% Export pig_classpath = $ hadoop_install/CONF/

Another method is to create a pig. properties file in the conf folder of the pig directory (which may need to be created by yourself), and then add the cluster namenode and jobtracker information to it:

FS. Default. Name =HDFS: // Localhost/

Mapred. Job. Tracker = localhost: 8021

Then, run the following command on the terminal:

% Pig

You will see the following information:

21:22:20, 489 [main] info org.apache.pig.backend.hadoop.exe cutionengine.

Hexecutionengine-connecting to hadoop file system at: HDFS: // localhost/

21:22:20, 760 [main] info org.apache.pig.backend.hadoop.exe cutionengine.

Hexecutionengine-connecting to map-reduce job tracker at: localhost: 8021

Grunt>

As you can see, the pig report has been connected to hadoop's namenode and jobtracker, isn't it easy?

at this point, the installation and configuration of pig are complete. Is it easy? What are you waiting for? Please try it and have fun ~