Using JMX to monitor Kafka

Kafka can configure monitoring using JMX for running status, either through the JDK's own jconsole to observe the results, or through the Java API. For a description of the monitoring indicator, refer to: http://kafka.apache.org/ Documentation.html#monitoring
Turn on the JMX port to modify bin/kafka-server-start.sh, add the Jmx_port parameter, add the following look

If ["x$kafka_heap_opts" = "x"]; Then    export kafka_heap_opts= "-xmx1g-xms1g"    export jmx_port= "9999" fi

can be connected via jconsole test



Access via JAVAAPI
Get target values in the following ways

public class kafkadataprovider{protected final Logger Logger = Loggerfactory.getlogger (GetClass ());    private static final String message_in_per_sec = "Kafka.server:type=brokertopicmetrics,name=messagesinpersec";    private static final String bytes_in_per_sec = "Kafka.server:type=brokertopicmetrics,name=bytesinpersec";    private static final String bytes_out_per_sec = "Kafka.server:type=brokertopicmetrics,name=bytesoutpersec"; private static final String produce_request_per_sec = "Kafka.network:type=requestmetrics,name=requestspersec,request    =produce "; private static final String consumer_request_per_sec = "Kafka.network:type=requestmetrics,name=requestspersec,    Request=fetchconsumer "; private static final String flower_request_per_sec = "kafka.network:type=requestmetrics,name=requestspersec,request=    Fetchfollower "; private static final String Active_controller_count = "Kafka.controller:type=kafkacontroller,name=    Activecontrollercount "; Private static final String part_count = "Kafka.server:type=replicamanager,name=partitioncount"; Public String Extractmonitordata () {//todo obtains IP and parameter Kafkaroleinfo by calling API Monitordatapoint = new Kafkarolein        Fo ();        String Jmxurl = "Service:jmx:rmi:///jndi/rmi://192.168.40.242:9999/jmxrmi";            try {mbeanserverconnection jmxconnection = metricdatautils.getmbeanserverconnection (Jmxurl);            ObjectName messagecountobj = new ObjectName (MESSAGE_IN_PER_SEC);            ObjectName bytesinpersecobj = new ObjectName (BYTES_IN_PER_SEC);            ObjectName bytesoutpersecobj = new ObjectName (BYTES_OUT_PER_SEC);            ObjectName producerequestspersecobj = new ObjectName (PRODUCE_REQUEST_PER_SEC);            ObjectName consumerrequestspersecobj = new ObjectName (CONSUMER_REQUEST_PER_SEC);            ObjectName flowerrequestspersecobj = new ObjectName (FLOWER_REQUEST_PER_SEC);            ObjectName activecontrollercountobj = new ObjectName (active_controller_count); ObjectName Partcountobj = new ObjectName (part_count);            Long messagesinpersec = (long) jmxconnection.getattribute (Messagecountobj, "Count");            Long bytesinpersec = (long) jmxconnection.getattribute (Bytesinpersecobj, "Count");            Long bytesoutpersec = (long) jmxconnection.getattribute (Bytesoutpersecobj, "Count");            Long producerequestcountpersec = (long) jmxconnection.getattribute (Producerequestspersecobj, "Count");            Long consumerrequestcountpersec = (long) jmxconnection.getattribute (Consumerrequestspersecobj, "Count");            Long flowerrequestcountpersec = (long) jmxconnection.getattribute (Flowerrequestspersecobj, "Count");            Integer activecontrollercount = (integer) jmxconnection.getattribute (activecontrollercountobj, "Value");            Integer partcount = (integer) jmxconnection.getattribute (partcountobj, "Value");            Monitordatapoint.setmessagesinpersec (MESSAGESINPERSEC); Monitordatapoint.setbytesinpersec (BYTESINPERSEC);            Monitordatapoint.setbytesoutpersec (BYTESOUTPERSEC);            Monitordatapoint.setproducerequestcountpersec (PRODUCEREQUESTCOUNTPERSEC);            Monitordatapoint.setconsumerrequestcountpersec (CONSUMERREQUESTCOUNTPERSEC);            Monitordatapoint.setflowerrequestcountpersec (FLOWERREQUESTCOUNTPERSEC);            Monitordatapoint.setactivecontrollercount (Activecontrollercount);        Monitordatapoint.setpartcount (Partcount);        } catch (IOException e) {e.printstacktrace ();        } catch (Malformedobjectnameexception e) {e.printstacktrace ();        } catch (Attributenotfoundexception e) {e.printstacktrace ();        } catch (Mbeanexception e) {e.printstacktrace ();        } catch (Reflectionexception e) {e.printstacktrace ();        } catch (Instancenotfoundexception e) {e.printstacktrace ();    } return monitordatapoint.tostring (); } public static void Main (string[] args) {System.out.println (New Kafkadataprovider (). Extractmonitordata ()); /** * Get mbeanserver Connection * * @param jmxurl * @return * @throws IOException */Public MBeanS Erverconnection getmbeanserverconnection (String jmxurl) throws IOException {jmxserviceurl url = new Jmxserviceurl (        Jmxurl);        Jmxconnector jmxc = jmxconnectorfactory.connect (URL, null);        Mbeanserverconnection mbsc = Jmxc.getmbeanserverconnection ();    return MBSC; }}

Other Tools

In addition to writing customized monitoring programs

Kafka-web-consolehttps://github.com/claudemamo/kafka-web-console
Deploy SBT:
Http://www.scala-sbt.org/0.13/tutorial/Manual-Installation.html
Http://www.scala-sbt.org/release/tutorial/zh-cn/Installing-sbt-on-Linux.html

kafkaoffsetmonitorhttps://github.com/quantifind/kafkaoffsetmonitor/releases/tag/v0.2.0
JAVA-CP Kafkaoffsetmonitor-assembly-0.2.0.jar com.quantifind.kafka.offsetapp.OffsetGetterWeb--zk localhost:12181- -port 8080--refresh 5.minutes--retain 1.day

Mx4jloader

Using JMX to monitor Kafka