Apche Pig calculation data import hbase

Source: Internet
Author: User

1.pig import or load data into HBase using the Hbasestorage UDF function, address: http://pig.apache.org/docs/r0.11.0/func.html#HBaseStorage
In the shell case:

Pig-dpig.additional.jars=/home/pps_recommend/pig-0.11.0/lib/protobuf-java-2.4.0a.jar-x $varPigMode << EOF
Recommend_data = LOAD ' $recommend _output ' as (Id:chararray,items:chararray);
STORE recommend_data into ' hbase://short_video_recommend ' USING org.apache.pig.backend.hadoop.hbase.HBaseStorage (' Info:items ');
Eof

(Hint: EOF is the output of other applications in the shell, remember: especially in the shell if you have an IF judgment statement eof to shelf write)

2.STORE [Load Pig DataSet alias] into ' hbase://[hbase table name] ' USING org.apache.pig.backend.hadoop.hbase.HBaseStorage (' [hbase column]: Pig field name ');
3. Note: 1. In the pseudo-distributed under the boot, in the Zookeeper,hbase Normal startup situation can be imported,
2. Formal environment configuration: 2.1 in $PIG _home/conf/pig.properties join: #----------------------------------------------------------- ------------------------------------------------------------------------------

#pig. load.default.statements=

hbase.zookeeper.quorum=10.11.8.161,10.11.8.162,10.11.8.163,10.11.8.164,10.11.8.165,10.11.8.166,10.11.8.167

hbase.master.info.bindaddress=10.11.5.222

hbase.master.port=60000

# -------------------------------------------------------------------------------------------------------------- ------------------------------

#hbase.zookeeper.quorum= zookeeper cluster address

#hbase.master.info.bindaddress=hbase Master Address

#hbase.master.port=hbase Master Port

2.2 Copy 3 packages in $pig_home/lib:

Protobuf-java-2.4.0a.jar Hbase-0.94.1.jar Zookeeper-3.4.3.jar
These packages can be obtained from $hbase_home/lib and $HBASE _home

Apche Pig calculation data import hbase

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.