Install and use Pig 0.12.1

Install and use Pig 0.12.1

Install and use Pig 0.12.1

1: Installation

Decompress the package, configure environment variables, and verify that pig is successfully installed.

[Bkjia @ jifeng02 ~] $ Tar zxf pig-0.12.0.tar.gz
[Bkjia @ jifeng02 ~] $ Vi. bash_profile
#. Bash_profile

# Get the aliases and functions
.~ /. Bashrc
Fi

# User specific environment and startup programs

PATH = $ PATH: $ HOME/bin

Export PATH
Export JAVA_HOME = $ HOME/jdk1.7.0 _ 45
Export PATH = $ JAVA_HOME/bin: $ PATH
Export CLASSPATH =.: $ JAVA_HOME/lib/dt. jar: $ JAVA_HOME/lib/tools. jar
Export Hadoop_HOME = $ HOME/hadoop/hadoop-1.2.1.
Export ANT_HOME = $ HOME/apache-ant-1.9.4

Export HIVE_HOME = $ HOME/hadoop/hive-0.12.0-bin
Export HBASE_HOME = $ HOME/hbase-0.94.21
Export PIG_HOME =$ HOME/pig-0.12.1

Export PATH = $ PATH: $ ANT_HOME/bin: $ HIVE_HOME/bin: $ HBASE_HOME/bin: $ PIG_HOME/bin
~
~
~
". Bash_profile" 23L, 591C written
[Bkjia @ jifeng02 ~] $ Source. bash_profile

[Bkjia @ jifeng02 ~] $ Pig-helpwhich: no hadoop in (/home/bkjia/jdk1.7.0 _ 45/bin:/home/bkjia/jdk1.7.0 _ 45/bin: /home/bkjia/jdk1.7.0 _ 45/bin:/usr/lib/qt-3.3/bin:/usr/local/bin:/usr/bin: /usr/local/sbin:/usr/sbin:/home/bkjia/bin:/home/bkjia/apache-ant-1.9.4/bin: /home/bkjia/hadoop/hive-0.12.0-bin/bin:/home/bkjia/hbase-0.94.21/bin:/home/bkjia/bin: /home/bkjia/apache-ant-1.9.4/bin:/home/bkjia/hadoop/hive-0.12.0-bin/bin:/home/bkjia/hbase-0.94.21/bin: /home/bkjia/pig-0.12.1/bin:/home/bkjia/apache-ant-1.9.4/bin: /home/bkjia/hadoop/hive-0.12.0-bin/bin:/home/bkjia/hbase-0.94.21/bin:/home/bkjia/pig-0.12.0/bin) Warning: $ HADOOP_HOME is deprecated. apache Pig version 0.12.0 (r1529718) compiled Oct 07 2013, 12:20:14

2. Pig Execution Mode
Pig has two execution modes:
1) Local)
In local mode, Pig runs in a single JVM and can access local files. This mode is suitable for processing small-scale data or learning.
Run the following naming settings to set the Local Mode:

[Bkjia @ jifeng02 ~] $ Pig-x local
Which: no hadoop in (/home/bkjia/jdk1.7.0 _ 45/bin:/home/bkjia/jdk1.7.0 _ 45/bin:/home/bkjia/jdk1.7.0 _ 45/bin: /usr/lib/qt-3.3/bin:/usr/local/sbin:/usr/sbin: /home/bkjia/bin:/home/bkjia/apache-ant-1.9.4/bin:/home/bkjia/hadoop/hive-0.12.0-bin/bin :: /home/bkjia/hbase-0.94.21/bin:/home/bkjia/apache-ant-1.9.4/bin: /home/bkjia/hadoop/hive-0.12.0-bin/bin:/home/bkjia/hbase-0.94.21/bin:/home/bkjia/pig-0.12.1/bin: /home/bkjia/bin:/home/bkjia/apache-ant-1.9.4/bin:/home/bkjia/hadoop/hive-0.12.0-bin/bin :: /home/bkjia/hbase-0.94.21/bin:/home/bkjia/pig-0.12.0/bin)
Warning: $ HADOOP_HOME is deprecated.

22:57:09, 716 [main] INFO org. apache. pig. Main-Apache Pig version 0.12.0 (r1529718) compiled Oct 07 2013, 12:20:14
22:57:09, 717 [main] INFO org. apache. pig. Main-Logging error messages to:/home/bkjia/pig_141_37029715.log
22:57:09, 735 [main] INFO org. apache. pig. impl. util. Utils-Default bootup file/home/bkjia/. pigbootup not found
22:57:09, 828 [main] INFO org.apache.pig.backend.hadoop.exe cutionengine. HExecutionEngine-Connecting to hadoop file system at: file :///
Grunt>

2) MapReduce Mode

In MapReduce mode, Pig converts a query to a MapReduce job and submits it to Hadoop (Clustering or pseudo-distributed ).

Check whether the current Pig version supports your current Hadoop version. Pig of a certain version only supports Hadoop of a specific version. You can visit the Pig official website to obtain version support information.

Pig uses the HADOOP_HOME environment variable. If this variable is not set, Pig can also use its own Hadoop library, but this will not ensure that its built-in validation library is compatible with the HADOOP version you actually use, therefore, we recommend that you explicitly set the HADOOP_HOME variable. You also need to set the PIG_CLASSPATH variable:

[Bkjia @ jifeng02 ~] $ Vi. bash_profile

#. Bash_profile

# Get the aliases and functions
If [-f ~ /. Bashrc]; then
.~ /. Bashrc
Fi

# User specific environment and startup programs

PATH = $ PATH: $ HOME/bin

Export PATH
Export JAVA_HOME = $ HOME/jdk1.7.0 _ 45
Export PATH = $ JAVA_HOME/bin: $ PATH
Export CLASSPATH =.: $ JAVA_HOME/lib/dt. jar: $ JAVA_HOME/lib/tools. jar
Export HADOOP_HOME = $ HOME/hadoop/hadoop-1.2.1.
Export ANT_HOME = $ HOME/apache-ant-1.9.4

Export HIVE_HOME = $ HOME/hadoop/hive-0.12.0-bin
Export HBASE_HOME = $ HOME/hbase-0.94.21
Export PIG_HOME =$ HOME/pig-0.12.0
Export PIG_CLASSPATH = $ HOME/hadoop/hadoop-1.2.1/conf/

Export PATH = $ PATH: $ ANT_HOME/bin: $ HIVE_HOME/bin: $ HBASE_HOME/bin: $ PIG_HOME/bin: $ HADOOP_HOME/bin
~
~
~
". Bash_profile" 24L, 661C written
[Bkjia @ jifeng02 ~] $ Source. bash_profile
[Bkjia @ jifeng02 ~] $ Pig
Warning: $ HADOOP_HOME is deprecated.

23:06:24, 122 [main] INFO org. apache. pig. Main-Apache Pig version 0.12.0 (r1529718) compiled Oct 07 2013, 12:20:14
23:06:24, 123 [main] INFO org. apache. pig. Main-Logging error messages to:/home/bkjia/pig_141_37584121.log
23:06:24, 142 [main] INFO org. apache. pig. impl. util. Utils-Default bootup file/home/bkjia/. pigbootup not found
23:06:24, 281 [main] INFO org.apache.pig.backend.hadoop.exe cutionengine. HExecutionEngine-Connecting to hadoop file system at: hdfs: // bkjia01: 9000
23:06:24, 463 [main] INFO org.apache.pig.backend.hadoop.exe cutionengine. HExecutionEngine-Connecting to map-reduce job tracker at: bkjia01: 9001
Grunt>

3. Run the Pig Program
There are three Pig program execution methods:

1) Script Mode
Directly run the file containing the Pig script. For example, the following command will run all the commands in the local scripts. pig file:
Pig scripts. pig

2) Grunt Mode
Grunt provides an interactive running environment where you can edit and execute commands on the command line.
Grund also supports command history and access through the up and down arrow keys.
Grund supports automatic command completion. For example, when you enter a = foreach B g and press the Tab key, the command line automatically becomes a = foreach B generate. You can even customize the detailed method of auto-complementing full functions of commands. For more information, see related documents.

3) embedded Mode
You can run Pig programs in java, similar to running SQL programs using JDBC.
Pig Latin Editor
PigPen is an Ecliipse plug-in that provides common functions for developing and running Pig programs in Eclipse, such as script editing and running. : Http://wiki.apache.org/pig/PigPen
Other Editors also provide the ability to edit Pig scripts, such as vi.

Installation and testing of Pig

Pig installation and configuration tutorial

Pig installation and deployment and testing in MapReduce Mode

Install Pig and test in local mode.

Installation configuration and basic use of Pig

Hadoop Pig advanced syntax

This article permanently updates the link address: