Python Access secured Hadoop Cluster through Thrift APIApache Thrift Python Kerberos Support typical way to connect Kerberos secured Thrift server example-hive example-hbase
Apache Thrift Python Kerberos Support
Both supports are only avaliable in Linux platform Native Support
Dependency: Kerberos (Python package) >> PURE-SASL (python package) >> Thrift (Python package)
Source: https://github.com/apache/thrift/blob/0.9.3/lib/py/src/transport/TTransport.py
Class Tsaslclienttransport (Ttransportbase, Creadabletransport): "" "
SASL Transport
" ""
START = 1
OK = 2
Bad = 3
ERROR = 4
COMPLETE = 5
def __init__ (self, transport, host, service, mechanism= ' GSSAPI ',
**sas L_kwargs): "" "
Cloudera ' s new API
Dependency: cyrus-sasl-*, Saslwrapper, Python-saslwrapper (Linux Libs) >> SASL (Python package) >> THRIFT-SASL (Python package) >> Thrift (Python package)
Source: init.py ">https://github.com/cloudera/thrift_sasl/blob/master/thrift_sasl/init. py
Class Tsaslclienttransport (Ttransportbase, Creadabletransport):
START = 1
OK = 2 Bad
= 3
ERROR = 4
COMPLETE = 5
def __init__ (self, sasl_client_factory, mechanism, trans): "" "
@param sasl_client_factory:a Callable that returns a new SASL. Client Object
@param mechanism:the SASL mechanism (e.g. "GSSAPI")
@param trans:the underlying transport over W Hich to communicate.
typical way to connect thrift servernon-secured Thrift Server
Transport = Tsocket.tsocket (Thrift_server_url, thrift_server_port)
transport = Ttransport.tbufferedtransport ( Transport)
protocol = Tbinaryprotocol.tbinaryprotocol (transport)
Secured Thrift Server
Use transport
Tsaslclienttransport
Transport = Tsocket.tsocket (Thrift_server_url, thrift_server_port)
transport = Ttransport.tsaslclienttransport (
Transport,
host= ' krb5_server ' service= ' service_name ' mechanism= ' GSSAPI '
)
protocol = Tbinaryprotocol.tbinaryprotocol (transport)
Example-server Information
/etc/krb5.conf
[Libdefaults]
Default_realm = Cloudera
Dns_lookup_kdc = False
Dns_lookup_realm = False
Ticket_lifetime = 86400
Renew _lifetime = 604800
forwardable = True
default_tgs_enctypes = Rc4-hmac default_tkt_enctypes
= Rc4-hmac
Permitted_enctypes = Rc4-hmac
udp_preference_limit = 1
kdc_timeout = 3000
[Realms]
Cloudera = {
KDC = Quickstart.cloudera
admin_server = Quickstart.cloudera
}
Hive Thrift server:quickstart.cloudera:10000
HBase Thrift server:quickstart.cloudera:9090 example-hive
Import sys from
hive import thrifthive to
hive.ttypes import hiveserverexception from
Thrift Import Thrift From
thrift.transport import Tsocket to
thrift.transport import ttransport from
thrift.protocol Import Tbinaryprotocol
transport = tsocket.tsocket (' Quickstart.cloudera ', 10000)
transport = Ttransport.tsaslclienttransport (
Transport,
host= ' Quickstart.cloudera '
service=
' hive ') Mechanism= ' GSSAPI '
)
Transport.open ()
protocol = Tbinaryprotocol.tbinaryprotocol (transport)
Client = thrifthive.client (protocol)
Example-hbase
from thrift.transport.TSocket import tsocket from thrift.transport import Ttransport from Thrift.protocol Import Tbinaryprotocol from hbase import hbase transport = Tsocket.tsocket (' Quickstart.cloudera ', 9090) t Ransport = Ttransport.tsaslclienttransport (transport, host= ' Quickstart.cloudera ' service= ' hbase ') Mechanis m= ' GSSAPI ') transport.open () protocol = TBINARYPROTOCOL.TBINARYPROTOCOL (transport) client = hbase.client (protocol)