. Net windows Kafka installation and usage (Getting Started),. netkafka

. Net windows Kafka installation and usage (Getting Started),. netkafka
For complete solutions, see:

Setting Up and Running Apache Kafka on Windows OS

 

Two problems encountered during Environment setup are listed here to facilitate query: 1. \ Java \ jre7 \ lib \ ext \ QTJava.zip was unexpected at this time. Process exited Solution:1.1 right-click "my computer"-> "Advanced System settings"-> "environment variable" 1.2. The classpathvalue contains no path containing qtjava.zip. If yes, delete the corresponding path, the problem is solved. 2. Missing 'server' JVM (Java \ jre7 \ bin \ server \ jvm. dll .) Solution:2.1. Copy C: \ Program Files \ Java \ jdk1.6.0 \ jre \ bin \ server2.2 and paste it to C: \ Program Files \ Java \ jre1.6.0 \ bin

Build Environment

1. Install JDK

1.1 installation file: http://www.oracle.com/technetwork/java/javase/downloads/jre8-downloads-%3155.htmldownload Server JRE.
1.2 after the installation is complete, add the following environment variables (right-click "my computer"-> "Advanced System settings"-> "environment variables "):

• • JAVA_HOME: C: \ Program Files (x86) \ Java \ jre1.8.0 _ 60 (this is the default installation path. If the installation directory is changed during installation, fill in the changed path)

• • PATH: Add "; % JAVA_HOME % \ bin" after the existing value"

1.3 open cmd and run "java-version" to view the Java version of the current system:

2. Install Zookeeper

Kafka depends on Zookeeper. Therefore, we need to install and run Zookeeper before running Kafka.

2.1 Download Installation File: http://zookeeper.apache.org/releases.html2.2 unzip file (this article decompress to G: \ zookeeper-3.4.8) 2.3 open G: \ zookeeper-3.4.8 \ conf, rename zoo_sample.cfg to zoo. 4.1.2.4 open zoo from the text editor. convert 2.5 change the value of dataDir to ": \ zookeeper-3.4.8 \ data" 2.6 Add the following system variables:

• • ZOOKEEPER_HOME: G: \ zookeeper-3.4.8
  • Path: Add "; % ZOOKEEPER_HOME % \ bin;" to the end of the existing value ;"

2.7 run Zookeeper: Open cmd and then execute

zkserver

3. install and run Kafka3.1 download Installation File: http://kafka.apache.org/downloads.html3.2 unzip file (this article decompress to G: \ kafka_2.11-0.10.0.1) 3.3 open G: \ kafka_2.11-0.10.0.1 \ config3.4 Open server from text editor. properties3.5. dirs value changed to "G: \ kafka_2.11-0.10.0.1 \ kafka-logs" 3.6 open tranquility 3.7 to the kafka file directory: cd/d G: \ kafka_2.11-0.10.0.1 \ 3.8 input and execute to open kafka:

.\bin\windows\kafka-server-start.bat .\config\server.properties

4. Create topics4.1 open cmd and go to G: \ kafka_2.11-0.10.0.1 \ bin \ windows4.2 to create a topic:

kafka-topics.bat --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic test

5. Open a Producer:

cd /d G:\kafka_2.11-0.10.0.1\bin\windowskafka-console-producer.bat --broker-list localhost:9092 --topic test

 

6. Open a Consumer:

cd /d G:\kafka_2.11-0.10.0.1\bin\windows

kafka-console-consumer.bat --zookeeper localhost:2181 --topic test

Then you can enter a message in the Producer Console window. After the message is entered, the Consumer window will soon display the message sent by the Producer:

 

So far, the establishment of the Kafka runtime environment has been completed :-)

Web Management (formal O & M deployment ):

Some people say that kafka manage

. Net sdk:

Someone suggested: https://github.com/Microsoft/CSharpClient-for-Kafka (Microsoft provides the sdk, looked at the source code I feel a little complicated)

And
Kafka-net is much simpler (I don't like writing some tasks after reading the source code)

Recommendation: kafka-net is easier to use.

Address: https://github.com/Jroland/kafka-net

 

The above sdk can search for nuget to view the open source address and the latest release package. Of course, there are many other options for. net sdk.

Personal development tools (supported by windows), used in windows Development Environment

KafkaTool

: Http://www.kafkatool.com/download.html

 

 

Kafka-net sdk demo

Reference Source: http://www.cnblogs.com/Wulex/p/5619425.html

For more information, see the official open source address of demo: https://github.com/Jroland/kafka-net

KafkaProducer program:

class Program
    {
        static void Main(string[] args)
        {
            do
            {
                Produce(GetKafkaBroker(), getTopicName());
                System.Threading.Thread.Sleep(3000);
            } while (true);
        }

        private static void Produce(string broker, string topic)
        {
            var options = new KafkaOptions(new Uri(broker));
            var router = new BrokerRouter(options);
            var client = new Producer(router);

            var currentDatetime =DateTime.Now;
            var key = currentDatetime.Second.ToString();
            var events = new[] { new Message("Hello World " + currentDatetime, key) };
            client.SendMessageAsync(topic, events).Wait(1500);
            Console.WriteLine("Produced: Key: {0}. Message: {1}", key, events[0].Value.ToUtf8String());

            using (client) { }
        }

        private static string GetKafkaBroker()
        {
            string KafkaBroker = string.Empty;
            const string kafkaBrokerKeyName = "KafkaBroker";

            if (!ConfigurationManager.AppSettings.AllKeys.Contains(kafkaBrokerKeyName))
            {
                KafkaBroker = "http://localhost:9092";
            }
            else
            {
                KafkaBroker = ConfigurationManager.AppSettings[kafkaBrokerKeyName];
            }
            return KafkaBroker;
        }
        private static string getTopicName()
        {
            string TopicName = string.Empty;
            const string topicNameKeyName = "Topic";

            if (!ConfigurationManager.AppSettings.AllKeys.Contains(topicNameKeyName))
            {
                throw new Exception("Key \"" + topicNameKeyName + "\" not found in Config file -> configuration/AppSettings");
            }
            else
            {
                TopicName = ConfigurationManager.AppSettings[topicNameKeyName];
            }
            return TopicName;
        }
    }

4. KafkaConsumer program:

class Program
    {
        static void Main(string[] args)
        {
            Consume(getKafkaBroker(), getTopicName());
            
        }

        private static void Consume(string broker, string topic)
        {   
            var options = new KafkaOptions(new Uri(broker));
            var router = new BrokerRouter(options);
            var consumer = new Consumer(new ConsumerOptions(topic, router));

            //Consume returns a blocking IEnumerable (ie: never ending stream)
            foreach (var message in consumer.Consume())
            {
                Console.WriteLine("Response: Partition {0},Offset {1} : {2}",
                    message.Meta.PartitionId, message.Meta.Offset, message.Value.ToUtf8String());
            }
        }

        private static string getKafkaBroker()
        {
            string KafkaBroker = string.Empty;
            var KafkaBrokerKeyName = "KafkaBroker";

            if (!ConfigurationManager.AppSettings.AllKeys.Contains(KafkaBrokerKeyName))
            {
                KafkaBroker = "http://localhost:9092";
            }
            else
            {
                KafkaBroker = ConfigurationManager.AppSettings[KafkaBrokerKeyName];
            }
            return KafkaBroker;
        }

        private static string getTopicName()
        {
            string TopicName = string.Empty;
            var TopicNameKeyName = "Topic";

            if (!ConfigurationManager.AppSettings.AllKeys.Contains(TopicNameKeyName))
            {
                throw new Exception("Key \"" + TopicNameKeyName + "\" not found in Config file -> configuration/AppSettings");
            }
            else
            {
                TopicName = ConfigurationManager.AppSettings[TopicNameKeyName];
            }
            return TopicName;
        }
    }

5. Consumer result: