The meaning of Spark.deploy.zookeeper.url in the Spark HA configuration

Source: Internet
Author: User
Tags zookeeper

There's a lot of spark ha configuration online, and recently I was looking at Wang Lin's spark video to pay for it. That person cow b blows very big, the ability should be some, but has the ability, not necessarily is the good teacher. First blowing China, blowing on the first to become the world. Even if you really are the first in the World, video (2. The 12th lesson in the Spark kernel decryption (11-43) is the wrong word about spark.deploy.zookeeper.url. He said that the address of the master of Spark should be configured, and then he started the spark master and zookeeper on several of the configured machines. In fact, the URL here refers to the zookeeper URL. For example, the following configuration:

Spark_daemon_java_opts= "-dspark.deploy.recoverymode=zookeeper-dspark.deploy.zookeeper.url=ubuntu3:2181,ubuntu4 : 2181,ubuntu5:2181 "

We need to start zookeeper in Ubuntu3~ubuntu5 and we can start master separately on UBUNTU1 and UBUNTU2. The same can be achieved with high availability. It shows that what he said is wrong.

The meaning of Spark.deploy.zookeeper.url in the Spark HA configuration

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.