Introduction and installation of Chao Wu teacher course--pig

1.Pig is a data processing framework based on Hadoop.  
MapReduce is developed using Java, and Pig has its own data processing language, and the pig's processing process is converted to Mr to run.

The Data Processing language of 2.Pig is the way of traffic, similar to the maths problem in junior middle School. Step-by-step, and SQL is not the same,SQL one step to get the answer ,


3.Pig Basic data types: int, long, float, double, chararry, ByteArray
    
composite data types:Map, Tuple, Bag
the type of bag such as {(' age ', +), (' name ', ' Zhang San ')}


4. How to install pig
4.1 Copy the pig-0.11.1.tar.gz to the/usr/local .
4.2 Using the command TAR-ZXVF pig-0.11.1.tar.gz decompression
4.3 using the command MV pig-0.11.1 Pig to rename
4.4 Editing a file Vi/etc/profile setting environment variables 
Export $PIG _home=/usr/local/bin 
Export PATH = ... $PIG _home/bin ....  
save, and then execute Source/etc/profile
4.5 Edit File $pig_home/conf/pig.properties, add two lines as follows   
fs.default.name=hdfs://hadoop0:9000 
mapred.job.tracker=hadoop0:9001


5. How the WLAN data is analyzed with pig
5.1 uploading pending data to HDFs
5.2 Convert data in HDFs to a pattern that pig can handle  
A = LOAD '/wlan ' as (T0:long, Msisdn:chararray, T2:chararray, T3:chararray, T4:chararray, T5:chararray, T6: Long, T7:long, T8:long,t9:long,t10:chararray);

5.3 Extract the useful fields from the inside 
B = FOREACH A GENERATE MSISDN, T6, T7, T8, T9;

5.4 Grouping Data
C = GROUP B by MSISDN;

5.5 Traffic Summary
D = FOREACH C GENERATE Group, sum (B.T6), sum (B.T7), sum (B.T8), sum (B.T9);

5.6 Store in HDFs
STORE D into '/wlan_result ';

Introduction and installation of Chao Wu teacher course--pig