Some instance syntax used in Pig

Some instance syntax used in Pig

Some instance syntax used in pig

1: Use a regular expression to load the name:
LOAD '/user/wizad/data/wizad/raw/2014-0 {6, 7---7--} */3_1/adwords *'

2: simple usage of filter:

Filter by value

FILTER clickDate_all BY log_type = '2 ';
FILTER mapping_table BY mapping_ad_network_id = '3' AND mapping_type = '5 ';
Test = FILTER allRow BY (ad_id = '20160901' OR ad_id = '20160901' OR ad_id = '20160901') AND log_type = 2;
Test = FILTER allRow BY (INDEXOF (ad_id, '20140901') = 0 or indexof (ad_id, '20140901') = 0 or indexof (ad_id, '20140901 ') = 0) AND log_type = 2;

Combined with the size function

FILTER count_imei BY (SIZE (cimei)> 14 and size (cimei) <17 );

Regular Expression

FILTER cimei2 by not cimei MATCHES '^ [0-9] * $ ';
FILTER cmac2 BY cmac MATCHES '/[A-F \ d] {2}: [A-F \ d] {2}: [A-F \ d] {2 }: [A-F \ d] {2}: [A-F \ d] {2}: [A-F \ d] {2 }/';

3: Sorting

ORDER province_count BY $2 DESC;

4: Use of the CONCAT function. It can be used to generate an independent column. For example, if a number is counted, add a column name.
FOREACH origin_cleaned_data generate concat ('<-_', '->') AS cou, guid, log_type;
Read_social_14 = FOREACH metadata_social_14 generate concat ('14', '='), guid_social;
All_id = FOREACH allRow GENERATE id, CONCAT ('_', '-') as cc;

5: replace values: filter NULL values and change them to unknown values.
Origin_historical = FOREACH origin_cleaned_data GENERATE wizad_ad_id, guid, log_type,
(Province_region_id = '')? 'Unknown ': province_region_id)

6: split into different subsets by value:
SPLIT geelyTuiGuang INTO Android IF OS _id = 1, ios IF OS _id = 2;
SPLIT ios INTO ios6 IF (INDEXOF (OS _version, '7 ')! = 0), ios7 if indexof (OS _version, '7') = 0;

7: replace function replacement value
FOREACH ios6 GENERATE imei, mac_address as cmac, REPLACE (idfa, 'null ','');

8: en_guid = STREAM duimei THROUGH 'awk-F "," '{if ($3 = "null") print $1 "," $2 ","; else print $0 }'';

Installation and testing of Pig

Pig installation and configuration tutorial

Pig installation and deployment and testing in MapReduce Mode

Install Pig and test in local mode.

Installation configuration and basic use of Pig

Hadoop Pig advanced syntax

This article permanently updates the link address: