Kafka producer Java Client

Source: Internet
Author: User
Tags getmessage

Producer

Contains a buffer pool that holds messages to be sent, messages in the buffer pool that have not yet been transmitted to the Kafka cluster.
The Kafka I/O thread at the bottom is responsible for translating messages from the buffer pool into requests sent to the cluster. If the close () method is not called when the produce is ended, the resources are compromised.
 

Common configuration

Bootstrap.servers

ACKs

The producer needs the server side after receiving the message, the scale of the feedback confirmation, mainly for the reliability transmission of the message; Acks=0 indicates that the producer does not need confirmation from the server;
Acks=1 indicates that the server sends an ACK after the message is saved, rather than waiting for the message to be received by the other follower roles, Acks=all (or acks=-1) means that the server side waits for all replicas to be received before sending an acknowledgment.

Retries


Batch.size


linger.ms

By default, the buffer's message is immediately sent to the server, even though the buffer space is not exhausted.

Batch.size


Buffer.memory

Key.serializer,value.serializer

Describes the serialization method used to serialize the user-supplied key and Vaule values into bytes.

Producer

  

 Public classProducer {Private Static FinalLogger Logger = Loggerfactory.getlogger (Producer.class); PrivateKafkaproducer<string, string>Kafkaproducer; PrivateRandom random =NewRandom (); PrivateString topic; Private intretry;  PublicProducer () { This("My_init"); }         PublicProducer (String topic) { This(topic,3); }         PublicProducer (String topic,intretry) {          This. Topic =topic;  This. Retry =retry; if(NULL==kafkaproducer) {Properties props=NewProperties (); InputStream instream=NULL; Try{instream= This. GetClass (). getClassLoader (). getResourceAsStream ("Kafka-producer.properties");                   Props.load (instream); Kafkaproducer=NewKafkaproducer<string, string>(props); } Catch(IOException e) {logger.error ("Kafkaproducer initialization failed:" +e.getmessage (), E); } finally {                   if(NULL!=instream) {                       Try{instream.close (); } Catch(IOException e) {logger.error ("Kafkaproducer initialization failed:" +e.getmessage (), E); }                   }               }           }      }        /*** Send messages via Kafkaproducer *@paramtopic message Receive subject *@paramPartitionnum which partition *@paramRetry retry times *@parammessage value for specific messages*/       PublicRecordmetadata Sendkafkamessage (FinalString message) {    Producerrecord<string, string> record =NewProducerrecord<string, string> (topic, Random.nextint (3), "", message); future<recordmetadata> meta = kafkaproducer.send (record,NewCallback () {
//The Send method is asynchronous, adding a message to the cache waiting to be sent and returning immediately, which enables the producer to increase efficiency by sending messages in bulk Public voidoncompletion (recordmetadata recordmetadata,exception Exception) {if(NULL!=exception) {Logger.error ("Kafka Send Message failed:" +exception.getmessage (), exception); Retrykakfamessage (message); } } }); Recordmetadata metadata=NULL; Try{metadata=Meta.get (); } Catch(Interruptedexception e) {}Catch(Executionexception e) {}returnmetadata; } /*** When the Kafka message fails to send, retry*/ Private voidRetrykakfamessage (FinalString Retrymessage) {Producerrecord<string, string> record =NewProducerrecord<string, string> (topic, Random.nextint (3), "", Retrymessage); for(inti = 1; I <= retry; i++) { Try{kafkaproducer.send (record); return; } Catch(Exception e) {logger.error ("Kafka Send Message failed:" +e.getmessage (), E); Retrykakfamessage (Retrymessage); } } } /*** Kafka Instance Destruction*/ Public voidClose () {if(NULL!=kafkaproducer) {Kafkaproducer.flush (); Kafkaproducer.close (); } } PublicString gettopic () {returntopic; } Public voidsettopic (String topic) { This. Topic =topic; } Public intGetretry () {returnretry; } Public voidSetretry (intretry) { This. Retry =retry; } }

Testproducer

  

 Public classTestproducer {Private Static FinalLogger Logger = Loggerfactory.getlogger (testproducer.class);  Public Static voidMain (string[] args) {Executorservice executor= Executors.newfixedthreadpool (3);  for(inti=0;i<3;i++) {Executor.submit (NewRunnable () {@Override Public voidrun () {String topic= "2017-11-6-test"; Producer P=NewProducer (topic);  for(intn=1;n<=5;n++) {String str= "Hello World =" +N; Recordmetadata message=p.sendkafkamessage (str); Logger.info ("Send message:" +message.topic () + "---" +message.partition () + "---" +Message.offset ());                } p.close ();        }            }); } System.out.println ("This is main"); Executor.shutdown ();//This indicates that the thread automatically exits after executionSystem.out.println ("Hello World"); }}

Kafka producer Java Client

Related Article

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.