Using confluent ' s JDBC Connector without installing the entire platform

Transferred from: https://prefrontaldump.wordpress.com/2016/05/02/using-confluents-jdbc-connector-without-installing-the-entire-platform/

I was interested in trying out Confluent's JDBC connector without installing their entire platform (I ' d like to stick to V  Anilla Kafka as much as possible). Here is the steps I followed to get it working with SQL Server.

Download Kafka 0.9, Untar the archive, and create a directory named Connect_libs in the Kafka root ( kafka_2.10-0.9.0.1/connect_libs ).

Download the Confluent platform and extract the following jars (you should also is able to pull these from confluent ' s MAV En repo, though I was unsuccessful):

• Common-config-2.0.1.jar
• Common-metrics-2.0.1.jar
• Common-utils-2.0.1.jar
• Kafka-connect-jdbc-2.0.1.jar

*place these jars along with the SQL Server driver in kafka_2.10-0.9.0.1/connect_libs . Update in with the broker list and create with the settings-to- bootstrap.servers kafka_2.10-0.9.0.1/config/connect-standalone.properties try out kafka_2.10-0.9.0.1/config/connect-jdbc.properties :

Name=sqlserver-feedconnector.class=io.confluent.connect.jdbc.jdbcsourceconnectortasks.max=1  connection.url=jdbc:sqlserver://xxx.xxx.xxx.xxx:1433;databasename=feeddb;user=user;password= Passwordtable.whitelist=tblfeedip,tblfeedurlmode=timestamp+incrementingtimestamp.column.name= locallastupdatedincrementing.column.name=idtopic.prefix=stg-    

Create the topics and on the stg-tblFeedIP stg-tblFeedURL cluster.

ADD the Connect_libs directory to the classpath:
export CLASSPATH="connect_libs/*"

And finally, run the connector in standalone mode with (make sure is in the root Kafka directory):bin/connect-standalone.sh config/connect-standalone.properties config/connect-jdbc.properties

Then, tail your topics to verify that messages is being produced by the connector.

* If you don ' ts about cluttering up the default libs directory ( kafka_2.10-0.9.0.1/libs ), you can also just dump the jars there and not There are to worry about setting the classpath.

Using confluent ' s JDBC Connector without installing the entire platform