Develop hive using thrift in php

Overview: Because hadoop and hive are both written in java, script languages like php are used (in fact, thrift supports many other languages, most of which are included in mainstream languages) developing hive requires a cross-language Connection Bridge. So far, I have learned two ways to achieve this: 1. thriftThrift described later is F

Overview: Because hadoop and hive are both written in java, script languages like php are used (in fact, thrift supports many other languages, most of which are included in mainstream languages) developing hive requires a cross-language Connection Bridge. So far, I have learned two ways to achieve this: 1. thrift described later is F

Overview:

Because both hadoop and hive are written in java, we need to use scripting languages like php (in fact, thrift supports many other languages, most of which are included in mainstream languages) developing hive requires a cross-language Connection Bridge. So far, I have known two methods to achieve this:

1. Thrift described later

Thrift is an open-source Facebook project. It is also a middleware mentioned on the hadoop and hive official websites. This development is to use Thrift.

2. php java bridge (this has never been used and I don't know much about it)

Zookeeper -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Thrift-related thrift

Installation:

After hadoop and hive are installed and mysql is configured as the hive metabase

1. Download thrift (Official apache thrift website)

2. decompress: tar-zxvf compressed package name

3. Install the dependent libraries libevent and libevent-devel.

4. Execute:./configure in the thrift path [For details, see./configure -- help]; then make & make install

Specific instructions can view the official site information: http://thrift.apache.org/

Usage:

The thrift directory contains packages, protocol, server, and transport folders, and provides the following hive functions:

Interface thrithiveif extends thrithivemetastoreif {public function execute ($ query); // run $ query public function fetchOne (); // get a result public function fetchN ($ numRows ); // get $ numRows results public function fetchAll (); // obtain all results public function getSchema (); // obtain metadata public function getThriftSchema (); // public function getClusterStatus (); // obtain the cluster status public function getQueryPlan (); // obtain the execution plan}

Add the following content to the PHP file during the call:

$ GLOBALS ['thrift _ root'] = 'thrift /'; // thrift ROOT directory // load the required files for connecting to Hive require_once $ GLOBALS ['thrift _ root']. 'packages/hive_service/thrithive. php '; require_once $ GLOBALS ['thrift _ root']. 'Transport/TSocket. php '; require_once $ GLOBALS ['thrift _ root']. 'protocol/ TBinaryProtocol. php '; // Set up the transport/protocol/client $ transport = new TSocket (host_ip, 10000); $ protocol = new TBinaryProtocol ($ transport ); $ client = new thrithiveclient ($ protocol); $ transport-> open ();

Then, the hql statement is executed through $ client. For example:

1. Obtain the database:

$ Client-> execute ("show databases"); $ result = $ client-> fetchAll (); // $ result receives the execution result

Note: The hql to be executed is used as the parameter of the execute () function and the result is obtained using the fetchAll () function;

2. Get metadata:

To obtain hive metadata, you can use the getSchema () function to obtain the metadata with the execution of a query. The example is as follows:

$ Client-> execute ("select id from hivetest "); // In the hivetest table, only the id and name fields are int and string types. $ schema = $ client-> getSchema (); $ result = $ client-> fetchAll ();

The printed $ schema value is:

Metastore_Schema Object (
[FieldSchemas] => Array (
[0] => metastore_FieldSchema Object (
[Name] => id
[Type] => int
[Comment] =>)
)
[Properties] =>
)

To obtain the data, you can use the following function to convert $ schema:

function objtoarr($obj){    $ret = array();    foreach($obj as $key =>$value){        if(gettype($value) == 'array' || gettype($value) == 'object'){            $ret[$key] = objtoarr($value);        }        else{            $ret[$key] = $value;        }    }    return $ret;}

Execute (), fetchaAll (), and getSchema () are mainly used. You can try the other functions again.