Reprint Please specify source: http://blog.csdn.net/l1028386804/article/details/46491773
1.Pig is a data processing framework based on Hadoop. MapReduce is developed using Java, and Pig has its own data processing language, and the pig's processing process is converted to Mr to run.
The data processing language of 2.Pig is the way of traffic, similar to the maths problem in junior middle school. 3.Pig basic Data type: int, long, float, double, chararry, ByteArray composite data type: MAP, Tuple, Bag
The type of bag such as {(' age ', +), (' name ', ' Zhang San ')}
4. How to install Pig4.1 copy pig-0.11.1.tar.gz to/usr/local 4.2 use command tar-zxvf pig-0.11.1.tar.gz extract 4.3 use command mv pig-0.11.1 Pig for renaming 4.4 series File vi/etc/profile setting environment variables export $PIG _home=/usr/local/bin
Export PATH = ... $PIG _home/bin ....
Save, and then execute Source/etc/profile
4.5 Edit File $pig_home/conf/pig.properties
Add two lines to the following:
fs.default.name=hdfs://hadoop0:9000
mapred.job.tracker=hadoop0:9001
5. How WLAN data is analyzed using Pig 5.1 upload pending data to HDFs 5.2 convert the data in HDFs to the pattern that pig can process a = LOAD '/wlan ' as (T0:long, Msisdn:chararray, T2: Chararray, T3:chararray, T4:chararray, T5:chararray, T6:long, T7:long, T8:long, T9:long, T10:chararray);
5.3 Extract the useful fields from the inside B = FOREACH A GENERATE msisdn, T6, T7, T8, T9;
5.4 Packet Data C = group B by MSISDN;
5.5 Traffic Summary D = FOREACH C GENERATE Group, sum (B.T6), sum (B.T7), sum (B.T8), sum (B.T9);
5.6 Stored in HDFs store D into '/wlan_result ';
The--pig framework for Hadoop
