Origins
The earliest contact with zookeeper, when learning the authoritative guide to Hadoop, was a sub-project of the Hadoop project.
Recently, a "distributed lock" is required in the project.
Before, in the development of peer-network loan system, the use of "distributed lock", the concept sounds very high-end, is actually more than one machine, while running the project under the "lock."
Previously, a "distributed lock" was implemented with Redis, but there was a recurring problem. Can only be inferred, program exits unexpectedly, or a set of redis for local development and test environments,
Local thread timed tasks, which are often forced to close, are not successfully released.
Listen boss said, Memcache comes with a distributed lock, is not used.
According to Boss, 2005 ago, the implementation of distributed lock is mainly used Memcache and Oracle. Later, there was a nosql redis,oracle basically replaced by MySQL,
A "distributed lock" is a problem. As for how others have solved it, it may be that memcache is used as a cache, and there may be other wits or solutions.
To explore the technical concept of "distributed lock", the real implementation must be to consider "business scenarios".
Separate technology, rarely out of business scenarios.
Later, Hadoop appeared, there are zookeeper, I do not know when the use of zookeeper to achieve distributed lock of people become more.
Where do you see that coming from? Baidu search "Zookeeper", related articles more, search zookeeper+ distributed lock people have become more.
See a technology fire not fire, see how many people in the study, how many people in summing up related articles, will know.
Zookeeper Basic Introduction
Zookeeper is a distributed, open source distributed application Coordination Service that is an open source implementation of Google's chubby and an important component of Hadoop and HBase.
It is a software that provides consistent services for distributed applications, including configuration maintenance, name services, distributed synchronization, group services, and so on.
The goal of zookeeper is to encapsulate complex and error-prone services that provide users with easy-to-use interfaces and performance-efficient, robust systems.
The zookeeper contains a simple set of primitives, [1] providing interfaces to Java and C.
The Zookeeper code version provides a distributed, exclusive lock, election, queue interface, and the code is zookeeper-3.4.3\src\recipes. Where the distribution locks and queues have Java and C two versions, the election is only Java version.
Zookeeper is based on the fast Paxos algorithm, the Paxos algorithm has a live lock problem, that is, when there are multiple proposer interleaved submissions, it is possible to repel each other so that no proposer can commit success,
And fast Paxos made some optimizations, through the election to produce a leader, only leader to submit proposer, the specific algorithm visible fast Paxos. Therefore, to understand zookeeper first need to know about fast Paxos. [3]
Basic operating procedure of zookeeper:
1, election leader.
2. Synchronize data.
3. There are many algorithms in the process of election leader, but the election standards to be met are consistent.
4, leader to have the highest zxid.
5, most of the machines in the cluster get response and follow selected leader.
official website
http://zookeeper.apache.org/
Zookeeper start
In the actual project development, the Linux version is used.
#Zookeeper启动
./zkserver.sh Start &
./zkcli.sh-server 127.0.0.1:2181 &
No big problem.
Learn by yourself, using the version of Windows.
#Zookeeper启动
CD/D E:\Mongodb-Redis-Nginx\zookeeper-3.5.1-alpha\bin
Zkserver Start &
Zkcli-server 127.0.0.1:2181 &
E:\mongodb-redis-nginx\zookeeper-3.5.1-alpha\bin>zkserver.cmd start
The system could not find the path specified.
Error:java_home is incorrectly set.
E:\mongodb-redis-nginx\zookeeper-3.5.1-alpha\bin>call "-dzookeeper.log.dir=e:\m
Ongodb-redis-nginx\zookeeper-3.5.1-alpha\bin\. \logs ""-dzookeeper.root.logger=i
Nfo,console ""-dzookeeper.log.file=zookeeper-administrator-server-xiaolei.log ""
-xx:+heapdumponoutofmemoryerror ""-xx:onoutofmemoryerror=cmd/c taskkill/pid
p/t/F "-CP" E:\Mongodb-Redis-Nginx\zookeeper-3.5.1-alpha\bin\. \build\classes;
E:\Mongodb-Redis-Nginx\zookeeper-3.5.1-alpha\bin\. \build\lib\*; E:\Mongodb-Redis
-nginx\zookeeper-3.5.1-alpha\bin\. \*; E:\Mongodb-Redis-Nginx\zookeeper-3.5.1-alp
Ha\bin\. \lib\*; E:\Mongodb-Redis-Nginx\zookeeper-3.5.1-alpha\bin\. \conf "Org.ap
Ache.zookeeper.server.quorum.QuorumPeerMain "E:\Mongodb-Redis-Nginx\zookeeper-3.
5.1-alpha\bin\. \conf\zoo.cfg "Start
The file name, directory name, or volume label method is incorrect.
E:\mongodb-redis-nginx\zookeeper-3.5.1-alpha\bin>endlocal
#打印Java_HOME, clearly has been configured, so the reason for excluding java_home
E:\mongodb-redis-nginx\zookeeper-3.5.1-alpha\bin>echo%java_home%
C:\Program files\java\jdk1.7.0_17;
Even if it is to change the Java_home "C:\Program files\java\jdk1.7.0_17" or not.
Online search, may be the JDK version of the problem, I use JDK1.7.
So, I'm probably judging the JDK and zookeeper version matching issues.
Re-download 3.4.6 version of Zookeeper
CD/D E:\Mongodb-Redis-Nginx\zookeeper-3.4.6\bin
Zkserver Start &
E:\mongodb-redis-nginx\zookeeper-3.4.6\bin>zkserver.cmd start
E:\mongodb-redis-nginx\zookeeper-3.4.6\bin>java "-dzookeeper.log.dir=e:\mongodb-
Redis-nginx\zookeeper-3.4.6\bin\. " "-dzookeeper.root.logger=info,console"-CP "
E:\Mongodb-Redis-Nginx\zookeeper-3.4.6\bin\. \build\classes; E:\Mongodb-Redis-Ngi
Nx\zookeeper-3.4.6\bin\. \build\lib\*; E:\Mongodb-Redis-Nginx\zookeeper-3.4.6\bin
\.. \*; E:\Mongodb-Redis-Nginx\zookeeper-3.4.6\bin\. \lib\*; E:\Mongodb-Redis-Nginx
\zookeeper-3.4.6\bin\. \conf "Org.apache.zookeeper.server.quorum.QuorumPeerMain
"E:\Mongodb-Redis-Nginx\zookeeper-3.4.6\bin\. \conf\zoo.cfg "Start
Error: The main class could not be found or could not be loaded Org.apache.zookeeper.server.quorum.QuorumPeerMain
E:\mongodb-redis-nginx\zookeeper-3.4.6\bin>endlocal
Configuring the Zoo.cfg File
In the Conf directory, copy zoo_sample.cfg, duplicate name zoo.cfg, restart
Another error.
E:\mongodb-redis-nginx\zookeeper-3.4.6\bin>zkserver.cmd start
E:\mongodb-redis-nginx\zookeeper-3.4.6\bin>java "-dzookeeper.log.dir=e:\mongodb-
Redis-nginx\zookeeper-3.4.6\bin\. " "-dzookeeper.root.logger=info,console"-CP "
E:\Mongodb-Redis-Nginx\zookeeper-3.4.6\bin\. \build\classes; E:\Mongodb-Redis-Ngi
Nx\zookeeper-3.4.6\bin\. \build\lib\*; E:\Mongodb-Redis-Nginx\zookeeper-3.4.6\bin
\.. \*; E:\Mongodb-Redis-Nginx\zookeeper-3.4.6\bin\. \lib\*; E:\Mongodb-Redis-Nginx
\zookeeper-3.4.6\bin\. \conf "Org.apache.zookeeper.server.quorum.QuorumPeerMain
"E:\Mongodb-Redis-Nginx\zookeeper-3.4.6\bin\. \conf\zoo.cfg "Start
2015-12-01 10:14:27,347 [myID:]-INFO [main:[email protected]]-Autopu
Rge.snapretaincount set to 3
2015-12-01 10:14:27,350 [myID:]-INFO [main:[email protected]]-Autopu
Rge.purgeinterval set to 0
2015-12-01 10:14:27,351 [myID:]-INFO [main:[email protected]]-Purge
Task is not scheduled.
2015-12-01 10:14:27,351 [myID:]-WARN [main:[email protected]]-either no CO
Nfig or no quorum defined in config, running in standalone mode
2015-12-01 10:14:27,411 [myID:]-ERROR [main:[email protected]]-Invalid
arguments, exiting abnormally
Java.lang.NumberFormatException:For input string: "E:\Mongodb-Redis-Nginx\zooke
Eper-3.4.6\bin\. \conf\zoo.cfg "
At java.lang.NumberFormatException.forInputString (Unknown Source)
At Java.lang.Integer.parseInt (Unknown Source)
At Java.lang.Integer.parseInt (Unknown Source)
At Org.apache.zookeeper.server.ServerConfig.parse (serverconfig.java:60)
At Org.apache.zookeeper.server.ZooKeeperServerMain.initializeAndRun (ZooK
EEPERSERVERMAIN.JAVA:83)
At Org.apache.zookeeper.server.ZooKeeperServerMain.main (zookeeperserverm
AIN.JAVA:52)
At Org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun (Qu
orumpeermain.java:116)
At Org.apache.zookeeper.server.quorum.QuorumPeerMain.main (quorumpeermain
. java:78)
2015-12-01 10:14:27,415 [myID:]-INFO [main:[email protected]]-usage:z
Ookeeperservermain ConfigFile | Port DataDir [Ticktime] [Maxcnxns]
Usage:zookeeperservermain ConfigFile | Port DataDir [Ticktime] [Maxcnxns]
Mom eggs Ah, the internet to find a lot of information, nothing is available.
Later, 1 answers were found based on the error message "Windows Zookeeper startup Java.lang.NumberFormatException".
Zkserver.cmd not the "start" on the back of the normal start.
CD/D E:\Mongodb-Redis-Nginx\zookeeper-3.4.6\bin
Zkcli.cmd-server 127.0.0.1:2181
Through a few difficult steps, the normal start up.
Zookeeper Launch Summary
1. The actual project is Linux, the problem is not big, local development learning with windows, a lot of problems.
2.zookeeper3.5.1-alpha, and local JDK1.7, there is a conflict, unable to start normally.
3.Zookeeper boot, you need to configure the ZOO.CFG in the Conf directory. Copy-paste-rename once.
4.Windows start, do not need to take "start" parameter, direct zkserver.cmd, really enough pit.
Zoo.cfg
# The number of milliseconds of each tickticktime=2000# the number of ticks, the initial # synchronization phase can T akeinitlimit=10# the number of ticks so can pass between # Sending a request and getting an acknowledgementsynclimit=5# The directory where the snapshot is stored.# does not use/tmp for storage,/tmp here is just # example Sakes.datadir=c:/zoo keeper# the port at which the clients would connectclientport=2181# the maximum number of client connections.# increase thi s if you need to handle more clients#maxclientcnxns=60## is sure to read the maintenance sections of the # Administrator Gu IDE before turning on autopurge.## http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance## the Number of snapshots to retain in datadir#autopurge.snapretaincount=3# Purge task interval in hours# Set to "0" to disable Auto Purge feature#autopurge.purgeinterval=1
DataDir and clientport should be necessary.
a little feeling: escape or face
In the case of Windows, I really want to go to Linux immediately, there is not so much fart.
However, I have always realized that "people" prefer to avoid problems, just endure to solve the problem positively.
In addition, it is to solve every problem encountered, their ability to solve problems, it is certainly greatly improved.
References
Zookeeper Baidu Encyclopedia
Http://baike.baidu.com/link?url=MsOQSGlqQA1BKF8v9OlB7k_jRi6lZm4fU9JeyP_pwA8yFa8mJopj3B7INfVVLRCIKkkEo2osXfBqnnSvuTq0p_
Zookeeper Exception connectionlossexception resolution
Http://www.sjsjw.com/kf_cloud/article/022572ABA018042.asp
E:\Mongodb-Redis-Nginx\zookeeper-3.5.1-alpha\bin
Throw "Invalid arguments, exiting abnormally" error message when starting zookeeper
Http://www.bug315.com/article/156.htm
Zookeeper Getting Started: Basic concepts, 5 configuration, starting
