First, for the server-side construction can refer to the previous article: Kafka stand-alone version of the environment to build and test
Server-Side ip:10.0.30.221
The directory of the running environment is as follows:
You need to change the following two properties in Server.properties under the Config folder
Zookeeper.connect=localhost:2181 changed into zookeeper.connect=10.0.30.221:2181
And the default commented out
#host. Name=localhost converted to host.name=10.0.30.221
Host.name does not change causes the client to report the error as follows
Exception in Thread "Thread-0" kafka.common.FailedToSendMessageException:Failed to send messages after 3 tries.at KAFKA.P Roducer.async.DefaultEventHandler.handle (defaulteventhandler.scala:90) at Kafka.producer.Producer.send ( producer.scala:76) at Kafka.javaapi.producer.Producer.send (producer.scala:33) at Its.kafka.Producer.run ( PRODUCER.JAVA:46)
After completion of the above steps, according to the "Kafka stand-alone version of the environment set up and test" method to start Zookeeper-server and Kafka-server can
Second, the client building
The Win7 system used by the client to connect to the server in eclipse
1. Under Eclipse, create a new project Kafka_producer with the following directory:
Note: The log4j.properties file under the Config folder is placed under SRC so that the log information can be observed
The code for producer is as follows:
Import Java.util.properties;import Kafka.producer.keyedmessage;import Kafka.producer.producerconfig;public class Producer extends thread{private final Kafka.javaapi.producer.producer<integer, string> Producer; Private final String topic; Private final String name; private final int numsofmessage; Private Final Properties Props = new properties (); Public Producer (String name,string topic,int numsofmessage) {props.put ("Serializer.class", " Kafka.serializer.StringEncoder ");p rops.put (" Metadata.broker.list "," 10.0.30.221:9092 ");//Asynchronous Send//props.put (" Producer.type "," async "), or//How many//props.put (" batch.num.messages "," 100 ") are sent each time; Producer = new Kafka.javaapi.producer.producer<integer, string> (new Producerconfig (props)); this.topic = topic; THIS.name = name; This.numsofmessage = Numsofmessage; } public void Run () {int messageno = 1; while (Messageno <= numsofmessage) {//per producer, the number of messages produced; string message = new String (name+ "' Smessage_" + Messageno+ "******"); Keyedmessage<integer, string> messageforsend = new Keyedmessage<integer, string> (topic, message); Producer.send (Messageforsend); messageno++; } producer.close (); }}
The code to start producer is as follows:
Import Java.util.concurrent.timeunit;public class Kafkaproducerdemo implements kafkaproperties{public static void main (string[] args) {Startthread (1, "Testtopic", 10);} /** * @param numsofproducer number of Producers * @param topic the subject of the message * @param numsofmessage the message tree produced by each producer * @return */public static void Startthread (int numsofproducer,string topic,int numsofmessage) {for (int i = 1; I <= numsofproducer; i + +) {String Name = "Producer" + i;new Producer (name,topic,numsofmessage). Start ();}}
2. Create a new Kafka_consumer project under Eclipse, with the following directory:
The consumer code is as follows:
Import Java.util.hashmap;import java.util.list;import Java.util.map;import Java.util.properties;import Kafka.consumer.consumerconfig;import Kafka.consumer.consumeriterator;import Kafka.consumer.kafkastream;import Kafka.javaapi.consumer.consumerconnector;public class Consumer extends Thread {private final Consumerconnector Consumer Private final String topic; Private final String name; Public Consumer (String name,string topic) {Consumer = Kafka.consumer.Consumer.createJavaConsumerConnector (c Reateconsumerconfig ()); this.topic = topic; THIS.name = name; } private static Consumerconfig Createconsumerconfig () {Properties props = new Properties (); Props.put ("Zookeeper.connect", kafkaproperties.zkconnect); Props.put ("Group.id", kafkaproperties.groupid); Props.put ("zookeeper.session.timeout.ms", "60000"); Props.put ("zookeeper.sync.time.ms", "200"); Props.put ("auto.commit.interval.ms", "1000"); The minimum number of bytes received at a time, by default, is 1//props.put ("Fetch.min.bytes"," 1024 "); The minimum time to wait, the default is//props.put ("fetch.wait.max.ms", "600000"); return new Consumerconfig (props); } public void Run () {map<string, integer> topiccountmap = new hashmap<string, integer> (); Topiccountmap.put (topic, New Integer (1)); Map<string, list<kafkastream<byte[], byte[]>>> consumermap = Consumer.createmessagestreams ( TOPICCOUNTMAP); Kafkastream<byte[], byte[]> stream = Consumermap.get (topic). get (0); Consumeriterator<byte[], byte[]> it = Stream.iterator (); while (It.hasnext ()) {System.out.println ("************" +name+ "gets" +new String (It.next (). Message ())); } }}
To start the consumer code:
public class Kafkaconsumerdemo implements kafkaproperties{public static void Main (string[] args) { //consumer1 Consumer consumerThread1 = new Consumer ("Consumer1", kafkaproperties.topic); Consumerthread1.start (); }}
The properties of the code (in order to pass the property value, of course, it can also be XML to provide property values):
Public interface kafkaproperties{ final static String Zkconnect = "10.0.30.221:2181"; Final static String groupId = "group1"; Final static String topic = "Testtopic"; Final static String Kafkaserverurl = "10.0.30.221"; Final static int kafkaserverport = 9092; Final static int kafkaproducerbuffersize = 64*1024; Final static int connectiontimeout = 100000; Final static int reconnectinterval = 10000; Final static String clientId = "Simpleconsumerdemoclient";}
3. Start consumer and then start producer, which is observed in the console window of Eclipse: