Spark-shell Unable to start the network problem

Due to the need to manually install SBT for the first time, networking is required, so the network adapter mode of the virtual machine is set to "bridge mode" so that it can be connected to the Internet.

However, when the "Spark-shell--master yarn--deploy-mode Client" command is executed, it cannot be started and remains in the middle State,

As follows:

[Email protected] test_code]# Spark-shell--master yarn--deploy-mode Client
Setting default log level to "WARN".
To adjust logging level use Sc.setloglevel (Newlevel). For Sparkr, use Setloglevel (Newlevel).
17/05/07 18:07:37 WARN util. nativecodeloader:unable to load Native-hadoop library for your platfor

M. ... using Builtin-java classes where applicable

It suddenly occurred to me that the network adapter mode of the virtual machine did not change over and was reset to "host mode only" (this is due to the installation of the Hadoop cluster

, a few VMware VMs are using "host-only Mode"), Spark-shell starts normally, as follows:

[Email protected] master]# Spark-shell--master yarn--deploy-mode Client
Setting default log level to "WARN".
To adjust logging level use Sc.setloglevel (Newlevel). For Sparkr, use Setloglevel (Newlevel).
17/05/07 18:30:12 WARN util. nativecodeloader:unable to load Native-hadoop library for your platform ... using Builtin-java classes where applicable
17/05/07 18:30:28 WARN yarn. Client:neither Spark.yarn.jars nor Spark.yarn.archive is set, falling back to uploading libraries under spark_home.
17/05/07 18:31:15 WARN Metastore. objectstore:failed to get Database global_temp, returning nosuchobjectexception
Spark context Web UI available at http://192.168.1.200:4040
Spark context available as ' sc ' (master = yarn, App id = application_1494142860645_0001).
Spark session available as ' spark '.
Welcome to
____ __
/ __/__ ___ _____/ /__
_\ \ _/_ '/__/' _/
/___/. __/\_,_/_//_/\_\ version 2.1.0
/_/

Using Scala version 2.11.8 (Java HotSpot (TM) 64-bit Server VM, Java 1.8.0_121)
Type in expressions to has them evaluated.
Type:help for more information.

Scala>

BTW:

By viewing the active NameNode status, you can see that the cluster is not functioning properly and cannot communicate with other nodes, causing the block to be lost. Shown on the following webui:

There is missing blocks. The following files may be corrupted:

blk_1073741947/opt/hadoop/out_wordcount4/part-r-00000
blk_1073741946/opt/hadoop/out_wordcount2/part-r-00000
blk_1073741945/opt/hadoop/out_wordcount/part-r-00000
Blk_1073741933/opt/hadoop/input/readme.txt
Blk_1073741931/hbase/masterprocwals/state-00000000000000000018.log
Blk_1073741930/hbase/oldwals/slave4%2c16020%2c1489814171196.meta.1489821379742.meta
blk_1073741929/hbase/oldwals/slave3%2c16020%2c1489814171011.1489821375778
blk_1073741928/hbase/oldwals/slave4%2c16020%2c1489814171196.1489821375970
blk_1073741927/hbase/oldwals/slave5%2c16020%2c1489814170009.1489821374298
Blk_1073741920/hbase/data/hbase/meta/1588230740/info/8e011b40156f4eeab4e83caf63ee1d23
blk_1073741847/hbase/data/hbase/namespace/3792ee8c4881d96201d73a19d76aa598/info/ Cc6c271a546248419df4c0988d191b4d
Blk_1073741846/hbase/data/hbase/namespace/3792ee8c4881d96201d73a19d76aa598/.regioninfo
blk_1073741845/hbase/data/hbase/namespace/.tabledesc/.tableinfo.0000000001
Blk_1073741968/linkage/block_9.csv
Blk_1073741967/linkage/block_8.csv
blk_1073741839/hbase/data/hbase/meta/.tabledesc/.tableinfo.0000000001
Blk_1073741966/linkage/block_7.csv
Blk_1073741838/hbase/data/hbase/meta/1588230740/.regioninfo
Blk_1073741837/hbase/hbase.id
Blk_1073741965/linkage/block_6.csv
Blk_1073741836/hbase/hbase.version
Blk_1073741964/linkage/block_5.csv
Blk_1073741963/linkage/block_4.csv
Blk_1073741962/linkage/block_3.csv
Blk_1073741961/linkage/block_2.csv
Blk_1073741833/tmp/hadoop-yarn/staging/history/done_intermediate/root/job_1489568197327_0001.summary
Blk_1073741960/linkage/block_1.csv
blk_1073741832/out/part-r-00000
Blk_1073741959/linkage/block_10.csv
Blk_1073741958/sogou/sogouq1.txt
Blk_1073741957/tmp/hadoop-yarn/staging/history/done_intermediate/root/job_1492443686126_0001_conf.xml
Blk_1073741956/tmp/hadoop-yarn/staging/history/done_intermediate/root/job_1492443686126_0001-1492443717631- Root-word+count-1492443749993-1-1-succeeded-default-1492443728809.jhist
Blk_1073741955/tmp/hadoop-yarn/staging/history/done_intermediate/root/job_1492443686126_0001.summary
blk_1073741954/opt/hadoop/out_1/part-r-00000
Blk_1073741825/word

Please check the logs or run fsck in order to identify the missing blocks. See the Hadoop FAQs for common causes and potential solutions.

Spark-shell Unable to start the network problem