Python # Pykafka-producer and consume

###

Python version: 2.7.13

Pykafka version: 2.6.0

Note: Python 3.6. Version 2 will give an error.

Note: This is a production data via Pykafka module to Kafka

GitHub Address: Https://github.com/Parsely/pykafka

Pykafka doc:http://pykafka.readthedocs.io/en/latest/usage.html

producer.py

#-*-Coding:utf-8-*-from pykafka Import kafkaclientclient = Kafkaclient (hosts= "192.168.0.100:9092") # can accept multiple Clientprint client.topics# View All Topictopic = client.topics[' Test ']# Select a topicmessage = "test message Test Message" # When you have topic, You can create a producer to send messages that produce Kafka data, in string form with Topic.get_sync_producer () as producer: for    I in range (4):            Producer.produce (' test message ' + str (i * * 2))

###

consume.py

#-*-coding:utf-8-*- fromPykafkaImportkafkaclientclient= Kafkaclient (hosts='192.168.0.100:9092') Topic=client.topics['Test']balanced_consumer=Topic.get_balanced_consumer (Consumer_group='Test_kafka_group', Auto_commit_enable=False,#when set to false do not need to add consumer_group, directly connected to topic can be taken to the messagezookeeper_connect='192.168.0.100:2181'#Here you can connect multiple ZK) forMessageinchBalanced_consumer:#Print Message    ifMessage is  notNone:PrintMessage.offset, Message.value#print the number of offsets and values of the received message body

###

Python # Pykafka-producer and consume