Hive uses PHP scripts to complete map/reduce

One of the more powerful features of hive SQL is the ability to use external scripting to complete map/reduce. Use the following:
TRANSFORM (...) USING ' ... ' As (...).
Using the inside can use PHP script to handle, the specific use of the following example.

①, build a table:

`member`(    >   `id`int,    >   `user_name`string,    >   `passwd`string    > )    format delimited    "\t"//这句很必要，不然下面的文件导入会导致所有的值都变成null     > stored as textfile;

②, prepare the following data/tmp/member.dat

1     zhangsan     zs10242     lisi     ls19913     wangwu     ww20014     liumang     lm12345     linxing     lx1990

③, importing data

loadlocal'/tmp/member.dat'table member;

can see:

selectfrom member;OK1       zhangsan        zs10242       lisi    ls19913       wangwu  ww20014       liumang lm12345       linxing lx1990

The data is ready, now we do a MD5 encryption in the third column. We use PHP scripts to process the PHP script (/tmp/changepasswd.php) code as follows:

    
     while(!feof(STDIN)){     $line = rtrim(fgets(STDIN), "\n");      //逐行读取if (empty($line)) continue;     //空，跳出当次循环$data = explode("\t", $line);    //将切分出来的数组保存起来，下面判断使用foreach($dataas &$val){          $val = md5($val);     }     unset($val);     echo implode("\t", $data) . "\n";}

④, add the php script to hive.

add file /tmp/changePasswd.php;

⑤, using PHP scripts to execute Map/reduce

hive> insert overwrite table member    > select TRANSFORM(`id`,`user_name`,`passwd`)"/usr/bin/php changePasswd.php"    > as (`id`,`user_name`,`passwd`) from member;

Finally, we can see that the data for the passwd column has changed:

selectfrom member;OK1       zhangsan        d03eed89429cc3006cc279322c2800c52       lisi    063401506c9d9f0e49a706e3779b74283       wangwu  ac5a8109dbbb46c9f69ffd5fc93c11f84       liumang fda8b97fd723bdbf6a754812b5ecec275       4035378ace8936e93d95aa77e7e224d4

Copyright NOTICE: This article is the original blogger articles, reproduced please indicate the source.

The above describes the use of Hive PHP script to complete the map/reduce, including aspects of the content, I hope that the PHP tutorial interested in a friend helpful.

