Kafka producer Java Client

Producer

Contains a buffer pool that holds messages to be sent, messages in the buffer pool that have not yet been transmitted to the Kafka cluster.
The Kafka I/O thread at the bottom is responsible for translating messages from the buffer pool into requests sent to the cluster. If the close () method is not called when the produce is ended, the resources are compromised.
　

Common configuration

Bootstrap.servers

ACKs

The producer needs the server side after receiving the message, the scale of the feedback confirmation, mainly for the reliability transmission of the message; Acks=0 indicates that the producer does not need confirmation from the server;
Acks=1 indicates that the server sends an ACK after the message is saved, rather than waiting for the message to be received by the other follower roles, Acks=all (or acks=-1) means that the server side waits for all replicas to be received before sending an acknowledgment.

Retries

Batch.size

linger.ms

By default, the buffer's message is immediately sent to the server, even though the buffer space is not exhausted.

Batch.size

Buffer.memory

Key.serializer,value.serializer

Describes the serialization method used to serialize the user-supplied key and Vaule values into bytes.

Producer

　　

 Public classProducer {Private Static FinalLogger Logger = Loggerfactory.getlogger (Producer.class); PrivateKafkaproducer<string, string>Kafkaproducer; PrivateRandom random =NewRandom (); PrivateString topic; Private intretry;  PublicProducer () { This("My_init"); }         PublicProducer (String topic) { This(topic,3); }         PublicProducer (String topic,intretry) {          This. Topic =topic;  This. Retry =retry; if(NULL==kafkaproducer) {Properties props=NewProperties (); InputStream instream=NULL; Try{instream= This. GetClass (). getClassLoader (). getResourceAsStream ("Kafka-producer.properties");                   Props.load (instream); Kafkaproducer=NewKafkaproducer<string, string>(props); } Catch(IOException e) {logger.error ("Kafkaproducer initialization failed:" +e.getmessage (), E); } finally {                   if(NULL!=instream) {                       Try{instream.close (); } Catch(IOException e) {logger.error ("Kafkaproducer initialization failed:" +e.getmessage (), E); }                   }               }           }      }        /*** Send messages via Kafkaproducer *@paramtopic message Receive subject *@paramPartitionnum which partition *@paramRetry retry times *@parammessage value for specific messages*/       PublicRecordmetadata Sendkafkamessage (FinalString message) {    Producerrecord<string, string> record =NewProducerrecord<string, string> (topic, Random.nextint (3), "", message); future<recordmetadata> meta = kafkaproducer.send (record,NewCallback () {
            //The Send method is asynchronous, adding a message to the cache waiting to be sent and returning immediately, which enables the producer to increase efficiency by sending messages in bulk             Public voidoncompletion (recordmetadata recordmetadata,exception Exception) {if(NULL!=exception) {Logger.error ("Kafka Send Message failed:" +exception.getmessage (), exception);                  Retrykakfamessage (message);          }              }          }); Recordmetadata metadata=NULL; Try{metadata=Meta.get (); } Catch(Interruptedexception e) {}Catch(Executionexception e) {}returnmetadata; }          /*** When the Kafka message fails to send, retry*/      Private voidRetrykakfamessage (FinalString Retrymessage) {Producerrecord<string, string> record =NewProducerrecord<string, string> (topic, Random.nextint (3), "", Retrymessage);          for(inti = 1; I <= retry; i++) {              Try{kafkaproducer.send (record); return; } Catch(Exception e) {logger.error ("Kafka Send Message failed:" +e.getmessage (), E);              Retrykakfamessage (Retrymessage); }          }      }      /*** Kafka Instance Destruction*/       Public voidClose () {if(NULL!=kafkaproducer)            {Kafkaproducer.flush ();          Kafkaproducer.close (); }      }         PublicString gettopic () {returntopic; }         Public voidsettopic (String topic) { This. Topic =topic; }         Public intGetretry () {returnretry; }         Public voidSetretry (intretry) {           This. Retry =retry; }  }

Testproducer

　　

 Public classTestproducer {Private Static FinalLogger Logger = Loggerfactory.getlogger (testproducer.class);  Public Static voidMain (string[] args) {Executorservice executor= Executors.newfixedthreadpool (3);  for(inti=0;i<3;i++) {Executor.submit (NewRunnable () {@Override Public voidrun () {String topic= "2017-11-6-test"; Producer P=NewProducer (topic);  for(intn=1;n<=5;n++) {String str= "Hello World =" +N; Recordmetadata message=p.sendkafkamessage (str); Logger.info ("Send message:" +message.topic () + "---" +message.partition () + "---" +Message.offset ());                } p.close ();        }            }); } System.out.println ("This is main"); Executor.shutdown ();//This indicates that the thread automatically exits after executionSystem.out.println ("Hello World"); }}

Kafka producer Java Client