Spark3000 disciple 12th class spark ha Combat summary

Listen to the 12th lesson of the Liaoliang teacher tonight Spark ha combat, class notes and assignments are as follows:

Spark ha requires the installation of zookeeper, and a stable version of 3.4.6 is recommended.

1. Download zookeeper3.4.6,
2. Configure Environment variables
3. Create data logs
4.VI conf/zoo.cfg
Create myID in the 5 data directory

spark-env.sh Configuring HA:

Export spark_daemon_java_opts= "-dspark.deploy.recoverymode=zookeeper-dspark.deploy.zookeeper.url=master:2181, work1:2181,work2:2181-Dspark.deploy.zookeepr.dir=/spark "

STANDBY Master, step aside meaning, wait for the service

Test: ./stop-master.sh

Failover recovery will not be restored immediately, the general minute level.

Wait a moment to see that master automatically switches successfully.

Follow-up courses can be referred to Sina Weibo Liaoliang _dt Big Data Dream Factory: Http://weibo.com/ilovepains

Liaoliang China Spark First person, public number Dt_spark

Spark3000 disciple 12th class spark ha Combat summary