Python通過Thrift插入Hbase實驗

標籤：python   hbase   thrift網關   

#python
from thrift import Thrift
from thrift.transport import TSocket
from thrift.transport import TTransport
from thrift.protocol import TBinaryProtocol

from hbase import Hbase
from hbase.ttypes import *

transport = TSocket.TSocket(‘192.168.137.101‘,9090)

transport = TTransport.TBufferedTransport(transport)

protocol = TBinaryProtocol.TBinaryProtocol(transport)

client = Hbase.Client(protocol)
transport.open()

mutations = [Mutation(column="tuser:b",value="1")]

#client.mutateRow(‘ttable0‘,‘rowkey1‘, mutations, None)

for l1 in range(10):
for l2 in range(10):
for l3 in range(10):
for l4 in range(10):
client.mutateRow(‘ttable0‘,‘%d‘%l1+‘%d‘%l2+‘%d‘%l3+‘%d‘%l4, mutations, None)

print ‘finished‘

4台centos虛擬機器

2.2Ghz CPU × 1

1.5GB記憶體

共用單硬碟

插入1000條用時93秒，平均一秒插入107條

複製0.py為1.py, 2.py ……8.py

開啟9個進程分別插入ttable0, ttable1, ttable2 ……ttable8

其它條件不變，總共用時約220秒，平均一秒插入409條

Python通過Thrift插入Hbase實驗