PHP Kafka use

This article mainly introduces PHP Kafka use, has a certain reference value, now share to everyone, the need for friends can refer to

Install and use Shell command Terminal Operations Kafka
Environment configuration
1, download the latest version of KAFKA:KAFKA_2.11-1.0.0.TGZ
/HTTP/ Mirrors.shu.edu.cn/apache/kafka/1.0.0/kafka_2.11-1.0.0.tgz

2, configuration, after decompression into the Config directory
2.1, Configure Zookeeper.properties
Default listener port 2181

2.2, configure Server.properties
to turn on listening ports, such as listening locally
listeners=plaintext:// 10.168.1.99:9092

3, start
3.1, Start zookeeper
./bin/zookeeper-server-start.sh config/zookeeper.properties

3.2, start server
./bin/kafka-server-start.sh config/server.properties

here to start server times wrong memory, The solution either increases the system memory, or the Kafka needs the memory
I'm turning small Kafka boot the required memory size
Vim kafka-server-start.sh
put it: Export kafka_heap_opts= "- XMX1G-XMS1G "
instead: Export kafka_heap_opts="-xmx512m-xms512m "

4, start the test consumer, and listen for topic test
./bin/ kafka-console-consumer.sh--zookeeper 10.168.1.99:2181--topic test--from-beginning

5, start the test producer, and listen for topic's Test
./bin/kafka-console-producer.sh--broker-list 10.168.1.99:9092--topic test

At this point, after entering data in the producer, return to the consumer terminal can see the information generated by the producer

Second, the use of PHP operation Kafka

1, install the Kafka extension Php-rdkafka
1.1, before the installation of Php-rdkafka, you need to first install Librdkafka
git clone https://github.com/edenhill/librdkafka.git
./configure
Make && make install

1.2, installation Php-rdkafka
git clone https://github.com/arnaud-lb/php-rdkafka.git
CD Php-rdkafka
Phpize
./configure
Make && make install

2. Write Kafka producer and consumer category

<?php/** * kafka.php. * USER:LVFK * DATE:2018/2/7 0007 * time:11:04 * Desc:kafka service */namespace app\models;use yii\base\invalidconfigexceptio    N;class kafka{public $broker _list = ' 10.168.1.99:9092 ';//configuration Kafka, you can separate multiple Kafka public with commas $topic = ' topic ';    Public $partition = 0;    protected $producer = null;    protected $consumer = null; Public Function __construct () {if (Empty ($this->broker_list)) {throw new Invalidconfigexception (        "Broker not config");        } $rk = new \rdkafka\producer ();        if (empty ($rk)) {throw new Invalidconfigexception ("producer Error");        } $rk->setloglevel (Log_debug);        if (! $rk->addbrokers ($this->broker_list)) {throw new Invalidconfigexception ("producer Error");    } $this->producer = $rk;        }/** * producer * @param array $messages * @return Mixed */Public function send ($messages = []) { $topic = $this->producer-> Newtopic ($this->topic);    Return $topic->produce (Rd_kafka_partition_ua, $this->partition, Json_encode ($messages));        }/** * Consumer * * Public Function consumer ($object, $callback) {$conf = new \rdkafka\conf ();        $conf->set (' Group.id ', 0);        $conf->set (' metadata.broker.list ', $this->broker_list);        $topicConf = new \rdkafka\topicconf ();        $topicConf->set (' auto.offset.reset ', ' smallest ');        $conf->setdefaulttopicconf ($topicConf);        $consumer = new \rdkafka\kafkaconsumer ($conf);        $consumer->subscribe ([$this->topic]);        echo "Waiting for messages.....\n";            while (true) {$message = $consumer->consume (120*1000);                     Switch ($message->err) {case Rd_kafka_resp_err_no_error:echo "message payload ...";                    $object-$callback ($message->payload);            Break        } sleep (1);    }}} 

3. Configuration

Configuration Kafka producer ' Asynclog ' and [    ' class ' = ' \\app\\models\\Kafka ',    ' broker_list ' = ' 10.168.1.99:9092 ',    ' topic ' = ' asynclog ']

4. Production of messages in business code
\yii:: $app->asynclog->send ([' This is Indexcontroller, '. Date (' Y-md h:i:s ', Time ())]);

5, in the command of Yii consumption

5.1, write kafkacontroller.php

<?php/** * @link http://www.yiiframework.com/* @copyright Copyright (c) Yii software LLC * @license Http://www.yi iframework.com/license/*/namespace app\commands;use yii\console\controller;/** * This command echoes the first argument That's entered. * * This command is provided as a example for the Learn how to create console commands. * * @author Qiang Xue <qiang.xue@gmail.com> * @since 2.0 */class Kafkacontroller extends controller{    /**     * T His command echoes the entered as the message.     * @param string $message The message to be echoed.     */Public    function Actionconsume ()    {        \yii:: $app->asynclog->consumer ($this, ' callback ');    }    Public function callback ($message)    {        \yii::info ($message, ' Testkafka ');        \yii:: $app->log->setflushinterval (1);}    }

5.2. Operation:
./yii Kafka/consume


6, when the 4th step in the production of data, at the end of the 5th step can consume data