Spark Project startup report memory overflow cannot create thread error

1. The error message is as follows:

Java.lang.OutOfMemoryError:unable to create new native thread
At Java.lang.Thread.start0 (Native Method)
At Java.lang.Thread.start (thread.java:714)
At Java.util.concurrent.ThreadPoolExecutor.addWorker (threadpoolexecutor.java:950)
At Java.util.concurrent.ThreadPoolExecutor.execute (threadpoolexecutor.java:1368)
At Org.jboss.netty.util.internal.DeadLockProofWorker.start (deadlockproofworker.java:38)
At Org.jboss.netty.channel.socket.nio.AbstractNioSelector.openSelector (abstractnioselector.java:349)
At Org.jboss.netty.channel.socket.nio.abstractnioselector.<init> (abstractnioselector.java:100)
At Org.jboss.netty.channel.socket.nio.nioclientboss.<init> (nioclientboss.java:63)
At Org.jboss.netty.channel.socket.nio.NioClientBossPool.newBoss (nioclientbosspool.java:61)
At Org.jboss.netty.channel.socket.nio.NioClientBossPool.newBoss (nioclientbosspool.java:27)
At Org.jboss.netty.channel.socket.nio.AbstractNioBossPool.init (abstractniobosspool.java:75)
At Org.jboss.netty.channel.socket.nio.nioclientbosspool.<init> (nioclientbosspool.java:45)
At Org.jboss.netty.channel.socket.nio.nioclientsocketchannelfactory.<init> ( nioclientsocketchannelfactory.java:185)
At Akka.remote.transport.netty.nettytransport.<init> (nettytransport.scala:283)
At Akka.remote.transport.netty.nettytransport.<init> (nettytransport.scala:240)
At Sun.reflect.NativeConstructorAccessorImpl.newInstance0 (Native Method)
At Sun.reflect.NativeConstructorAccessorImpl.newInstance (nativeconstructoraccessorimpl.java:62)
At Sun.reflect.DelegatingConstructorAccessorImpl.newInstance (delegatingconstructoraccessorimpl.java:45)
At Java.lang.reflect.Constructor.newInstance (constructor.java:423)
At akka.actor.reflectivedynamicaccess$ $anonfun $createinstancefor$2.apply (dynamicaccess.scala:78)
At Scala.util.try$.apply (try.scala:161)
At Akka.actor.ReflectiveDynamicAccess.createInstanceFor (dynamicaccess.scala:73)
At akka.actor.reflectivedynamicaccess$ $anonfun $createinstancefor$3.apply (dynamicaccess.scala:84)
At akka.actor.reflectivedynamicaccess$ $anonfun $createinstancefor$3.apply (dynamicaccess.scala:84)
At Scala.util.Success.flatMap (try.scala:200)
At Akka.actor.ReflectiveDynamicAccess.createInstanceFor (dynamicaccess.scala:84)
At akka.remote.endpointmanager$ $anonfun $9.apply (remoting.scala:711)
At akka.remote.endpointmanager$ $anonfun $9.apply (remoting.scala:703)
At scala.collection.traversablelike$withfilter$ $anonfun $map$2.apply (traversablelike.scala:722)
At Scala.collection.iterator$class.foreach (iterator.scala:727)
At Scala.collection.AbstractIterator.foreach (iterator.scala:1157)
At Scala.collection.iterablelike$class.foreach (iterablelike.scala:72)
At Scala.collection.AbstractIterable.foreach (iterable.scala:54)
At Scala.collection.traversablelike$withfilter.map (traversablelike.scala:721)
At akka.remote.endpointmanager.akka$remote$endpointmanager$ $listens (remoting.scala:703)
At akka.remote.endpointmanager$ $anonfun $receive$2.applyorelse (remoting.scala:491)
At Akka.actor.actor$class.aroundreceive (actor.scala:467)
At Akka.remote.EndpointManager.aroundReceive (remoting.scala:394)
At Akka.actor.ActorCell.receiveMessage (actorcell.scala:516)
At Akka.actor.ActorCell.invoke (actorcell.scala:487)
At Akka.dispatch.Mailbox.processMailbox (mailbox.scala:238)
At Akka.dispatch.Mailbox.run (mailbox.scala:220)
At Akka.dispatch.forkjoinexecutorconfigurator$akkaforkjointask.exec (abstractdispatcher.scala:397)
At Scala.concurrent.forkjoin.ForkJoinTask.doExec (forkjointask.java:260)
At Scala.concurrent.forkjoin.forkjoinpool$workqueue.runtask (forkjoinpool.java:1339)
At Scala.concurrent.forkjoin.ForkJoinPool.runWorker (forkjoinpool.java:1979)
At Scala.concurrent.forkjoin.ForkJoinWorkerThread.run (forkjoinworkerthread.java:107)
17/04/01 19:20:02 error actorsystemimpl:uncaught fatal error from thread [sparkdriver-akka.actor.default-dispatcher-3] Shutting down Actorsystem [Sparkdriver]

2. Solution:

Modify the maximum number of threads allowed by a user under a Linux server
Modify files under/etc/security/limits.d/90-nproc.conf ( the file name may differ from one system to another )
* Soft Nproc 4096
Root Soft Nproc Unlimited
The maximum number of threads other than the root user is 4096

3. Common commands

Ulimit-u This command to view the maximum number of threads for the current user
Command parameters
-A Show all restrictions
The upper limit of the-C core file size
-D upper limit for process data segment size
The maximum file size that the-f shell can create
The upper limit of-m resident memory size
-s stack size upper limit
-T maximum CPU time per second
-P Pipe Size
-N Maximum number of open files
The upper limit of the-U process number
-V Upper limit of virtual memory

Spark Project startup report memory overflow cannot create thread error