version, through the Yun install Clustershell installation, will be prompted no package, the source of the Yum in the long-term no update, so use to Epel-release
installation command:
sudo yum install epel-release
Then the Yum install Clustershell can be installed by Epel.
1.2.2: Configuring Cluster groups
Vim/etc/clustershell/groups
Add a group name: server IP or Host
kafka:192.168.17.129 192.168.17.130
Introduction
Cluster installation:
I. preparations:
1. Version introduction:
Currently we are using a version of kafka_2.9.2-0.8.1 (scala-2.9.2 is officially recommended for Kafka, in addition to 2.8.2 and 2.10.2 available)
2. Environment preparation:
Install JDK 6. The current version is 1.6 and java_home is configured.
3. Configuration modification:
1) copy the online configuration to the local
Kafka is a distributed publish-subscribe messaging system. It is originally developed at LinkedIn and became a Apache project in July, 2011. Today, Kafka is used by LinkedIn, Twitter, and Square for applications including log aggregation, queuing, and real time m Onitoring and event processing.In the upcoming version 0.8 release, Kafka'll support intra-cluster re
row, and the following list shows the details).Leader: A node that reads and writes messages from a partition. Each node becomes leader random.Replicas: The node that replicates the log, whether it is leader or not, regardless of whether it is still available.ISR: The replica set of the synchronization state. This replica set includes nodes that may later become leader for the active person.Now look at the theme test that was created on a single node (if there is no better zookeeper, the previo
is a brief introduction to the Kafka cluster construction process:
Prep environment: At least 3 Linux servers (the author is a 5 redhat version of cloud server)
First step: Install Jdk/jre
Step Two: Install Zookeeper (Kafka comes with zookeeper service, but it is recommended that you build a zookeeper cluster separate
Reference:Kafka cluster--3 broker 3 Zookeeper Create a real combat kafka_kafka introduction and installation _v1.3 http://www.docin.com/p-1291437890.htmlI. Preparatory work:1. Prepare 3 machines with IP addresses of: 192.168.3.230 (233,234) 2 respectively. Download Kafka stable version, my version is: Scala 2.11-kafka_2.11-0.9.0.0.tgz http://kafka.apache.org/downloads.html 3. respectively extracted into the
First, cluster installation1. Kafka Download:Can be found on the official website of Kafka (http://kafka.apache.org), and then wgetwget http://mirrors.cnnic.cn/apache/kafka/0.8.2.2/kafka_2.10-0.8.2.2.tgzUnzip the file:Tar zxvf kafka_2.10-0.8.2.2.tgzNote that Kafka relies on
Kafka is to store consumer information on the client rather than the MQ server, so that the server does not need to record the message delivery process, each client knows where to read the message next time. The message delivery process also uses the client's active PULL model, which greatly reduces the burden on the server.
Kafka also emphasizes reducing the serialization and copy overhead of data. It org
Hello:
650) This. width = 650; "src =" http://s3.51cto.com/wyfs02/M02/49/1A/wKiom1QO9ljRwwGzAAEPTr3xscE724.jpg "Title =" 10.png" alt = "wkiom1qo9ljrwwgzaaeptr3xsce724.jpg"/>
The command on the slave2 host is:
#kafka-console-consumer.sh--zookeepermaster:2181--topictest--from-beginning
The result is as follows:
650) This. width = 650; "src =" http://s3.51cto.com/wyfs02/M00/49/1A/wKiom1QO9p3jYxAMAADGg8WbYnU114.jpg "Title =" 11.png" alt = "wkiom1qo9p3jyx
The previous log has been set up zookeeper cluster, see: http://www.cnblogs.com/lianliang/p/6533670.html, then continue to build Kafka cluster1, first download Kafka GZ package: Http://kafka.apache.org/downloadsUnzip to/opt/soft/kafka/directory, unzip and create folder logs for Kaf
Zookeeper + kafka cluster installation 2
This is the continuation of the previous article. The installation of kafka depends on zookeeper. Both this article and the previous article are true distributed installation configurations and can be directly used in the production environment.
For zookeeper installation, refer:
Http://blog.csdn.net/ubuntu64fan/article/de
The environment of this article is as follows:Operating system: CentOS 6 32-bitJDK version: 1.8.0_77 32-bitKafka version: 0.9.0.1 (Scala 2.11)
Connected with the Kafka stand-alone, cluster mode installation detailed (a)6. Single-node multi-broker modeKafka can be used in a variety of modes, including single-node single-broker, single-node multi-broker, multi-node multi-broker.Here we simply distinguish
Install a Kafka Cluster
Assume that four brokers need to be configured in the cluster to form the Kafka cluster in the following chart.
2.1 configuration file
Configure all Kafka producer files. The brokerpid value is a unique
Three machines: Vm1, VM2, vm31. Deploy the zookeeper cluster assuming there is already a deployed zookeeper cluster: ZK1, ZK2, ZK32, download, decompression Kafka
1
tar -xzvfkafka_2.10-0.8.2.1.tgz
3, modify the vm1 above the Config/server.properties
1234
broker.id=1log.dirs=/usr/local/bigdata/tmp/kafka_logshost
Zookeeper + kafka cluster Installation
Prepare three VMS. The system is RHEL64.1) the configuration of each machine is as follows:
$ Cat/etc/hosts
...# zookeeper hostnames:192.168.8.182 zk1192.168.8.183 zk2192.168.8.184 zk3
2) install jdk, zookeeper, and kafka on each machine. The configuration is as follows:
$ Vi/etc/profile
...# jdk, zookeeper
Environmental requirements: Three installed Centos6.4 virtual machine, need to have java1.7 above environment, need zookeeper environment.1) Download Kafka installation package from Kafka websiteDownload Kafka2) Unpack the installation package3) Since the name is too long, change to Kafka:MV kafka_2.10-0.9.0.1 Kafka4) Enter the Config directory below Kafka and mo
Go from: Kafka cluster expansion and redistribution of partitions
We add the machine to the already deployed Kafka cluster is the most normal demand, and add it is very convenient, we need to do is to copy the corresponding configuration file from the deployed Kafka node, a
The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion;
products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the
content of the page makes you feel confusing, please write us an email, we will handle the problem
within 5 days after receiving your email.
If you find any instances of plagiarism from the community, please send an email to:
info-contact@alibabacloud.com
and provide relevant evidence. A staff member will contact you within 5 working days.