Python Method for operating hbase data, pythonhbase

Python Method for operating hbase data, pythonhbase

Configure thrift

Python package thrift

The python compiler is pycharm community edition. in project Settings, locate the project interpreter, locate the package under the corresponding project, select "+" to add, and search for hbase-thrift (Python client for HBase Thrift interface ), then install the package.

Install thrift on the server.

Refer to the official website, and you can also install it on the local machine for use on the terminal.

Thrift Getting Started

You can also refer to the installation methodExample of calling HBase using python

First, install thrift

Download thrift. here, we use thrift-0.7.0-dev.tar.gz.

Tar xzf thrift-0.7.0-dev.tar.gz
Cd thrift-0.7.0-dev
Sudo./configure-with-cpp = no-with-ruby = no
Sudo make
Sudo make install

Then, go to the HBase source code package and find

Src/main/resources/org/apache/hadoop/hbase/thrift/

Run

Thrift-gen py Hbase. thrift
Mv gen-py/hbase // usr/lib/python2.4/site-packages/(may vary with python versions)

Example 1

# coding:utf-8from thrift import Thriftfrom thrift.transport import TSocketfrom thrift.transport import TTransportfrom thrift.protocol import TBinaryProtocolfrom hbase import Hbase# from hbase.ttypes import ColumnDescriptor, Mutation, BatchMutationfrom hbase.ttypes import *import csvdef client_conn(): # Make socket transport = TSocket.TSocket('hostname,like:localhost', port) # Buffering is critical. Raw sockets are very slow transport = TTransport.TBufferedTransport(transport) # Wrap in a protocol protocol = TBinaryProtocol.TBinaryProtocol(transport) # Create a client to use the protocol encoder client = Hbase.Client(protocol) # Connect! transport.open() return clientif __name__ == "__main__": client = client_conn() # r = client.getRowWithColumns('table name', 'row name', ['column name']) # print(r[0].columns.get('column name')), type((r[0].columns.get('column name'))) result = client.getRow("table name","row name") data_simple =[] # print result[0].columns.items() for k, v in result[0].columns.items(): #.keys()  #data.append((k,v))  # print type(k),type(v),v.value,,v.timestamp  data_simple.append((v.timestamp, v.value)) writer.writerows(data) csvfile.close() csvfile_simple = open("data_xy_simple.csv", "wb") writer_simple = csv.writer(csvfile_simple) writer_simple.writerow(["timestamp", "value"]) writer_simple.writerows(data_simple) csvfile_simple.close() print "finished"

The basic python should know that result is a list, and result [0]. columns. items () is a dict key-value pair. You can query relevant information. You can also observe the value and type of the variable by outputting the variable.

Note:In the above program, transport. open () is linked. After execution, you also need to disconnect transport. close ()

Currently, only read data is involved, and other dbase operations will be updated later.

The above python Method for operating hbase data is all the content that I have shared with you. I hope you can give us a reference and support the help house.