How to open spark with Jupyter notebook

The computer has Anaconda Python installed and then downloaded the spark2.1.0. Because the version is too new, some content on the web and in the book is no longer applicable. For example, on how to use Ipython and Jupyter, the tutorial gives you the option to open spark with the following statement into Ipython or Ipython Notebook:

Ipython=1./bin/pysparkipython_opts= "Notebook"./bin/pyspark

Then the goose runs the above statement and I get the following:

in 2.0 +. Remove these from the environment and set Pyspark_driver_python and pyspark_driver_python_opts instead.

So the Setup method for using Jupyter to open Spark is:

1 export pyspark_driver_python=jupyter2 export pyspark_driver_python_opts=' Notebook ' Pyspark

This will automatically open spark with Jupyter notebook when you open spark again.

How to open spark with Jupyter notebook