JDK:1.6.0_25 64-bit
kafka:2.9.2-0.8.2.1
Kafka official Http://apache.fayea.com/kafka/0.8.2.1/kafka_2.9.2-0.8.2.1.tgz
Tar-ZXVF kafka_2.9.2-0.8.2.1. tgz-c/usr/local/&&MVKafka_2.9.2-0.8.2.1Kafka
Cd/usr/local/kafka
VIConfig/zookeeper.properties
Datadir=/usr/local/kafka/zookeeper
VIConfig/server.properties
Broker.ID=0
port=9092
hostname=192.168.194.110
Log.dirs=/usr/local/kafka/kafka-logs
zookeeper.connect=192.168.194.110:2181
Start Zookeeper
Bin/zookeeper-server-start. SH config/zookeeper.properties
Start Kafka Broker
Bin/kafka-server-start. SH Config/server.properties &
View startup status
JPs
14867Quorumpeermain # # #存在代表zookeeper服务启动正常
14919 Kafka # # #代表kafka broker starts successfully
Shutting down the firewall
Service Iptables Stop
Standalone Kafka producer Consumer message send and receive test
Bin/kafka-console-producer.SH--broker-list192.168.194.110:9092--topic Test # # # #启动producer
Bin/kafka-console-consumer.SH--zookeeper192.168.194.110:2181--topic Test--from-beginning
Then enter the MSG content to be sent at the producer end to see if the consumer end receives the message
[2015-09-11 13:58:00,470] Error error in handling batch of 1 events (Kafka.producer.async.ProducerSendThread)
kafka.common.FailedToSendMessageException:Failed to send messages after 3 tries.
At Kafka.producer.async.DefaultEventHandler.handle (defaulteventhandler.scala:90)
At Kafka.producer.async.ProducerSendThread.tryToHandle (producersendthread.scala:105)
At kafka.producer.async.producersendthread$ $anonfun $processevents$3.apply (producersendthread.scala:88)
At kafka.producer.async.producersendthread$ $anonfun $processevents$3.apply (producersendthread.scala:68)
At Scala.collection.immutable.Stream.foreach (stream.scala:526)
At Kafka.producer.async.ProducerSendThread.processEvents (producersendthread.scala:67)At Kafka.producer.async.ProducerSendThread.run (producersendthread.scala:45)
This exception occurs on the producer side, please use the IP address of the producer listening broker instead of localhost
To this stand-alone version of Kafka has been built and tested successfully
Use Java to invoke Kafka's producer messages and consumer to send messages
Maven Add Dependency1<Dependency>
2<groupId>Org.apache.kafka</groupId>
3<Artifactid>kafka_2.10</Artifactid>
4<version>0.8.2.0</version>
5</Dependency>
Kafka message producer Kafkaproducer
ImportJava.util.Date;
ImportJava.util.Properties;
ImportJava.util.Random;
ImportKafka.javaapi.producer.Producer;
ImportKafka.producer.KeyedMessage;
ImportKafka.producer.ProducerConfig;
PublicclassKafkaproducerextendsThread {
PublicStaticvoidMain (string[] args) {
Properties props =NewProperties ();
Props.put ("Metadata.broker.list", "192.168.194.110:9092");
Props.put ("Serializer.class", "Kafka.serializer.StringEncoder");
Props.put ("Key.serializer.class", "Kafka.serializer.StringEncoder");
Props.put ("Request.required.acks", "1");
//Configuring the serialization class for key
Producerconfig config =NewProducerconfig (props);
producer<string, string> Producer =Newproducer<string, string> (config);
Random rnd =NewRandom ();
Longruntime =NewDate (). GetTime ();
String IP = "192.168.2." + rnd.nextint (255);
String msg = runtime + ", www.example.com," + IP;
keyedmessage<string, string> data =NewKeyedmessage<string, string> ("Test", "Test-key", msg);
Producer.send (data);
}
}
Kafka Message Consumer kafkaconsumer
ImportJava.util.HashMap;
ImportJava.util.List;
ImportJava.util.Map;
ImportJava.util.Properties;
ImportKafka.consumer.ConsumerConfig;
ImportKafka.consumer.ConsumerIterator;
ImportKafka.consumer.KafkaStream;
ImportKafka.javaapi.consumer.ConsumerConnector;
ImportKafka.serializer.StringDecoder;
ImportKafka.utils.VerifiableProperties;
Publicclasskafkaconsumer{
PrivateStaticConsumerconnector consumer =NULL;
PublicStaticvoidMain (string[] args) {
Properties props =NewProperties ();
//Zookeeper Configuration
Props.put ("Zookeeper.connect", "192.168.194.110:2181");
//Group represents a consumer group
Props.put ("Group.id", "Jd-group");
//ZK Connection timed out
Props.put ("zookeeper.session.timeout.ms", "4000");
Props.put ("zookeeper.sync.time.ms", "200");
Props.put ("auto.commit.interval.ms", "1000");
Props.put ("Auto.offset.reset", "smallest");
//Serialization Classes
Props.put ("Serializer.class", "Kafka.serializer.StringEncoder");
Consumerconfig config =NewConsumerconfig (props);
Consumer = kafka.consumer.Consumer.createJavaConsumerConnector (config);
map<string, integer> topiccountmap =NewHashmap<string, integer> ();
Topiccountmap.put ("Test",NewInteger (1));
Stringdecoder Keydecoder =NewStringdecoder (NewVerifiableproperties ());
Stringdecoder Valuedecoder =NewStringdecoder (NewVerifiableproperties ());
Map<string, list<kafkastream<string, string>>> consumermap =
Consumer.createmessagestreams (Topiccountmap,keydecoder,valuedecoder);
kafkastream<string, string> stream = Consumermap.get ("Test"). Get (0);
Consumeriterator<string, string> it = Stream.iterator ();
while(It.hasnext ())
System.out.println (It.next (). message ());
}
}
Start producer and consumer separately for simple message sending and receiving
Results:
Log4j:warn No appenders could be found forLogger (kafka.utils.VerifiableProperties).
Log4j:warn Initialize the log4j system properly.
Log4j:warn See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
1441952197141,www.example.com,192.168.2.86
Kafka installation and Getting Started demo
