Kafka environment to build two---Windows Client Linux server

First, for the server-side construction can refer to the previous article: Kafka stand-alone version of the environment to build and test

Server-Side ip:10.0.30.221

The directory of the running environment is as follows:

You need to change the following two properties in Server.properties under the Config folder

Zookeeper.connect=localhost:2181 changed into zookeeper.connect=10.0.30.221:2181

And the default commented out

#host. Name=localhost converted to host.name=10.0.30.221

Host.name does not change causes the client to report the error as follows

Exception in Thread "Thread-0" kafka.common.FailedToSendMessageException:Failed to send messages after 3 tries.at KAFKA.P Roducer.async.DefaultEventHandler.handle (defaulteventhandler.scala:90) at Kafka.producer.Producer.send ( producer.scala:76) at Kafka.javaapi.producer.Producer.send (producer.scala:33) at Its.kafka.Producer.run ( PRODUCER.JAVA:46)

After completion of the above steps, according to the "Kafka stand-alone version of the environment set up and test" method to start Zookeeper-server and Kafka-server can

Second, the client building

The Win7 system used by the client to connect to the server in eclipse

1. Under Eclipse, create a new project Kafka_producer with the following directory:

Note: The log4j.properties file under the Config folder is placed under SRC so that the log information can be observed

The code for producer is as follows:

Import Java.util.properties;import Kafka.producer.keyedmessage;import Kafka.producer.producerconfig;public class    Producer extends thread{private final Kafka.javaapi.producer.producer<integer, string> Producer;    Private final String topic;    Private final String name;    private final int numsofmessage;    Private Final Properties Props = new properties (); Public Producer (String name,string topic,int numsofmessage) {props.put ("Serializer.class", " Kafka.serializer.StringEncoder ");p rops.put (" Metadata.broker.list "," 10.0.30.221:9092 ");//Asynchronous Send//props.put ("    Producer.type "," async "), or//How many//props.put (" batch.num.messages "," 100 ") are sent each time;    Producer = new Kafka.javaapi.producer.producer<integer, string> (new Producerconfig (props));    this.topic = topic;    THIS.name = name;    This.numsofmessage = Numsofmessage;      } public void Run () {int messageno = 1; while (Messageno <= numsofmessage) {//per producer, the number of messages produced; string message = new String (name+ "' Smessage_" + Messageno+ "******");      Keyedmessage<integer, string> messageforsend = new Keyedmessage<integer, string> (topic, message);      Producer.send (Messageforsend);      messageno++; } producer.close (); }}

The code to start producer is as follows:

Import Java.util.concurrent.timeunit;public class Kafkaproducerdemo implements kafkaproperties{public static void main (string[] args) {Startthread (1, "Testtopic", 10);}  /** * @param numsofproducer  number of Producers * @param topic the subject of the message * @param numsofmessage the message tree produced by each producer * @return  */public static void Startthread (int numsofproducer,string topic,int numsofmessage) {for (int i = 1; I <= numsofproducer; i + +) {String Name = "Producer" + i;new Producer (name,topic,numsofmessage). Start ();}}

2. Create a new Kafka_consumer project under Eclipse, with the following directory:

The consumer code is as follows:

Import Java.util.hashmap;import java.util.list;import Java.util.map;import Java.util.properties;import Kafka.consumer.consumerconfig;import Kafka.consumer.consumeriterator;import Kafka.consumer.kafkastream;import Kafka.javaapi.consumer.consumerconnector;public class Consumer extends Thread {private final Consumerconnector  Consumer  Private final String topic;    Private final String name; Public Consumer (String name,string topic) {Consumer = Kafka.consumer.Consumer.createJavaConsumerConnector (c    Reateconsumerconfig ());    this.topic = topic;  THIS.name = name;    } private static Consumerconfig Createconsumerconfig () {Properties props = new Properties ();    Props.put ("Zookeeper.connect", kafkaproperties.zkconnect);    Props.put ("Group.id", kafkaproperties.groupid);    Props.put ("zookeeper.session.timeout.ms", "60000");    Props.put ("zookeeper.sync.time.ms", "200");    Props.put ("auto.commit.interval.ms", "1000"); The minimum number of bytes received at a time, by default, is 1//props.put ("Fetch.min.bytes"," 1024 ");    The minimum time to wait, the default is//props.put ("fetch.wait.max.ms", "600000");  return new Consumerconfig (props);    } public void Run () {map<string, integer> topiccountmap = new hashmap<string, integer> ();    Topiccountmap.put (topic, New Integer (1)); Map<string, list<kafkastream<byte[], byte[]>>> consumermap = Consumer.createmessagestreams (    TOPICCOUNTMAP);    Kafkastream<byte[], byte[]> stream = Consumermap.get (topic). get (0);    Consumeriterator<byte[], byte[]> it = Stream.iterator ();    while (It.hasnext ()) {System.out.println ("************" +name+ "gets" +new String (It.next (). Message ())); }  }}

To start the consumer code:

public class Kafkaconsumerdemo implements kafkaproperties{public  static void Main (string[] args) {    //consumer1    Consumer consumerThread1 = new Consumer ("Consumer1", kafkaproperties.topic);    Consumerthread1.start ();  }}

The properties of the code (in order to pass the property value, of course, it can also be XML to provide property values):

Public interface kafkaproperties{  final static String Zkconnect = "10.0.30.221:2181";    Final static  String groupId = "group1";  Final static String topic = "Testtopic";  Final static String Kafkaserverurl = "10.0.30.221";  Final static int kafkaserverport = 9092;  Final static int kafkaproducerbuffersize = 64*1024;  Final static int connectiontimeout = 100000;  Final static int reconnectinterval = 10000;  Final static String clientId = "Simpleconsumerdemoclient";}

3. Start consumer and then start producer, which is observed in the console window of Eclipse: