because too many Java objects are stored in the memory. If the executor has too many cache objects in the JVM, for example, 80 GB up, how many times of full GC occurs at this time, and you will wonder why my program does not move? You can check the GC log. It was originally in GC.
3. Implement Fault Tolerant tachyon Cluster Based on zookeeper
3.0 prerequisites
Hadoop version: 2.2.0.2.0.6.0-101
1, Decompression Zookeeper2. Create the Zoo.cfg file under $zookeeper_home/conf (refer to the configuration file: zoo_sample.cfg) with the content:# CS Communication Heartbeat time, between Zookeeper servers or between the client and server to maintain the heartbeat interval, that is, each ticktime time will send a heartbeat. Ticktime the maximum number of heartbeats (ticktime) that can be tolerated by the initial connection between the follower serve
2081Issue seven (multi-Problem collection):Start Error:Solution:Indicates that myid this file does not exist into the/export/servers/data/zookeeper/directory created myID this file (not a folder)Solution:In the myID file, enter the Coo.cfg profile that corresponds to the current IP address myID if the current node is configured in the Coo.cfg profile server.1=192.168.1.110:2888:3888 then in the myID file enter 1, save exitSolution:Turn off the firewa
installation of Zookeeper (standalone version) in Linux has introduced how to build a standalone version of Zookeeper in Linux, this article will continue to build the cluster version of zookeeper based on the previous article.On the original basis, two more virtual machines are prepared:My virtual machine IP is: 192.1
1. Download Zookeeper http://mirror.bit.edu.cn/apache/zookeeper/After decompression, the directory renamed to Zookeeper1, into the Conf directory, the ZOO_SAMPLE.CFG copy, renamed to Zoo.cfgEdit Zoo_simple.cfgConfiguration Data directory and log directory:First, in the Zookeeper1 directory, create a new directory data, logs#------------Log directory------------#dataDir=/tmp/zookeeperdatadir=d:/develop-env/
The previous article built the zookeeper clusterWell, today we can build a cluster of SOLR search service, which is different from Redis cluster, it needs ZK management, as an agent layerInstall four Tomcat and modify its port number to not conflict. 8080~8083If it is a formal environment, use 4 Linux nodes respectivelyModify the Server.xml file to modify the por
In the previous blog we explained the "zookeeper" stand-alone configuration, this blog will continue to introduce the Zookeeper cluster deployment and configuration. Environment
The environment of the cluster configuration is the same as that of the stand-alone configuration, the only difference is that the
Zookeeper cluster Installation
Zookeeper is a highly reliable coordination system for large-scale distributed systems. Therefore, we can see that zookeeper is a coordination system, and its role is distributed systems. So why do distributed systems need a coordination system? The reason is as follows:
Developing a dist
Deploy three Services
Zoo-node1.cfg
# The number of milliseconds of each tick ticktime=2000 # The number of ticks, the initial # synchronization phase CA N Take initlimit=10 # The number of ticks so can pass between # Sending a request and getting an acknowledgement syncli
Mit=5 # The directory where the snapshot is stored.
# do not use/tmp for storage,/tmp here is just # example sakes. Datadir=g:/zookeeper-3.4.9/data/node1 Datalogdir=g:/
1: Perform these operations on the 192.168.80.200 server and upload the zookeeper-3.4.5.tar.gz file to the/usr/local directory on the server2: Unzip: TAR-ZXVF zookeeper-3.4.5.tar.gz3: Rename: MV zookeeper-3.4.5 zookeeper4:CD zookeeper/conf5: Modify the configuration file name: MV Zoo_sample.cfg zoo.cfg6:vi zoo.cfgModif
Build ActiveMQ clusters based on ZooKeeper and LevelDBOfficial Document: Http://activemq.apache.org/replicated-leveldb-store.htmlCluster schematic diagram:650) this.width=650; "Src=" http://activemq.apache.org/replicated-leveldb-store.data/replicated-leveldb-store.png "alt=" Replicated-leveldb-store.png "/>The principle of high availability:Use Zookeeper (cluster
Zookeeper + kafka cluster Installation
Prepare three VMS. The system is RHEL64.1) the configuration of each machine is as follows:
$ Cat/etc/hosts
...# zookeeper hostnames:192.168.8.182 zk1192.168.8.183 zk2192.168.8.184 zk3
2) install jdk, zookeeper, and kafka on each machine. The configuration is as
Zookeeper configuration: Zookeeper has three configurations: stand-alone/pseudo-distributed/clustered where pseudo-distribution simulates distributed scenarios on a single computer through different ports, requires n-copies of configuration files and startup programs, and clusters are multiple zookeeper programs. First get the tar package through SFTP, extract to
=2000initlimit=10Synclimit=5Datadir=/soft/tmp/zk1/dataDatalogdir=/soft/tmp/zk1/logclientport=2181server.1=localhost:2287:3387server.2=localhost:2288:3388server.3=localhost:2289:3389/soft/zookeeper-2/conf/zoo.cfg content is as follows:ticktime=2000initlimit=10Synclimit=5Datadir=/soft/tmp/zk2/dataDatalogdir=/soft/tmp/zk2/logclientport=2182server.1=localhost:2287:3387server.2=localhost:2288:3388server.3=localhost:2289:3389/soft/
What is zookeeper?Apache ZooKeeper is a high-performance coordination server for distributed applications. It exposes public services (such as naming and configuration management, synchronization, and group services) using a simple interface, so that users do not have to start programming from scratch. It provides ready-made support for consensus-building, group management, leader elections, and presence ag
availability
disadvantage: stability relies on database performance dependency database
here is the Activemq+zookeeper cluster integration
Zookeeper in the Linux cluster build, please refer to: http://blog.csdn.net/xiaojin21cen/article/details/79056578 II, server description
There are 3 servers, the server's operating
version, through the Yun install Clustershell installation, will be prompted no package, the source of the Yum in the long-term no update, so use to Epel-release
installation command:
sudo yum install epel-release
Then the Yum install Clustershell can be installed by Epel.
1.2.2: Configuring Cluster groups
Vim/etc/clustershell/groups
Add a group name: server IP or Host
kafka:192.168.17.129 192.168.17.130 192.168.17.131 II:
The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion;
products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the
content of the page makes you feel confusing, please write us an email, we will handle the problem
within 5 days after receiving your email.
If you find any instances of plagiarism from the community, please send an email to:
info-contact@alibabacloud.com
and provide relevant evidence. A staff member will contact you within 5 working days.