Bug that starts Spark-shell--master yarn

The error is as follows

 -/ ./ .  the: -: toERROR cluster. Yarnclientschedulerbackend:yarn application have already exited with state finished! -/ ./ .  the: -: toERROR client. transportclient:failed to send RPC8494797597505892687To/192.168.1.21:44578: Java.nio.channels.ClosedChannelExceptionjava.nio.channels.ClosedChannelException at Io.netty.channel.AbstractChannel$AbstractUnsafe. Write (...) (Unknown Source) -/ ./ .  the: -: toERROR cluster. Yarnschedulerbackend$YarnSchedulerEndpoint: Sending Requestexecutors (0,0, Map (), Set ()) to AM is unsuccessfuljava.io.IOException:Failed to send RPC8494797597505892687To/192.168.1.21:44578: Java.nio.channels.ClosedChannelException at Org.apache.spark.network.client.TransportClient.lambda$sendRpc$2(Transportclient.java:237) at Io.netty.util.concurrent.DefaultPromise.notifyListener0 (Defaultpromise.java:507) at Io.netty.util.concurrent.DefaultPromise.notifyListenersNow (Defaultpromise.java:481) at Io.netty.util.concurrent.DefaultPromise.notifyListeners (Defaultpromise.java:420) at Io.netty.util.concurrent.DefaultPromise.tryFailure (Defaultpromise.java:122) at Io.netty.channel.AbstractChannel$AbstractUnsafe. Safesetfailure (Abstractchannel.java:852) at Io.netty.channel.AbstractChannel$AbstractUnsafe. Write (Abstractchannel.java:738) at Io.netty.channel.DefaultChannelPipeline$HeadContext. Write (Defaultchannelpipeline.java:1251) at Io.netty.channel.AbstractChannelHandlerContext.invokeWrite0 (Abstractchannelhandlercontext.java:733) at Io.netty.channel.AbstractChannelHandlerContext.invokeWrite (Abstractchannelhandlercontext.java:725)

You need to add the following configuration in Yarn-site.xml

    <property>        <name>yarn.nodemanager.pmem-check-enabled</name>        <value>false</value>    </property>    <property>        <name>yarn.nodemanager.vmem-check-enabled</name>        <value>false</value>    </property>

Distribute yarn-site.xml and restart yarn , and try again spark-shell --master yarn .

FIN

Bug that starts Spark-shell--master yarn