Jupyter Spark Environment Configuration (online, offline can be achieved) _jupyter

Application scenarios in order to be able to develop spark programs in Jupyter, Bowen records the process of configuring spark development environment in Jupyter. Many blogs can not effectively build jupyter spark development environment. Prerequisite Components Spark Downloads
Spark-2.3.0-bin-hadoop2.7.tgz Apache Toree
Apache Toree has one main goal:provide the foundation for interactive applications to connect and use Apache Spark. Download Address
Apache/incubator-toree GitHub Source code incubator/toree tar package download Callout:
System environment is not installed Scala, Hadoop

[Root@localhost bin]# cat/etc/redhat-release
CentOS Linux release 7.3.1611 (Core)

installation Command Online install
The Anaconda environment variable is already in the Linux PATH environment variable, is not configured, or is switched to the Anaconda Bin directory and is installed and configured using the PIP command.

# Your-spark-home:spark installation package path
pip install toree
jupyter toree Install--spark_home=your-spark-home

Offline install
Download GitHub source code, TAR packets can be offline installation. Source Code Installation
/root/anaconda2/bin/python setup.py Install jupyter toree install--spark_home=your-spark-home Test Code test environment is built successfully

Import Org.apache.spark.sql.SparkSession

object Sparksqldemo {
    val sparksession = Sparksession.builder ().
        Master ("local[1]")
        . AppName ("Spark Session Example")
        . Getorcreate ()

    def main (args:array[string]) {
        val input = SparkSession.read.json ("Cars1.json")
        Input.createorreplacetempview ("Cars1")
        val result = Sparksession.sql ("SELECT * from Cars1")
        result.show ()
    }
}

Sparksqldemo.main (Array ()) # call method

Execution results
Extensions: Installing multiple cores installing multiple kernels
Options--interpreters=<unicode> (toreeinstall.interpreters)
Default: ' Scala '
A Comma separated list of the interpreters to install. The names of the
Interpreters are case sensitive.

Jupyter Toree Install--interpreters=scala,pyspark,sparkr,sql

References jupyter Notebook toree Configuration Pyspark development environment and its working principle. Jupyter notebook installation based on Pyspark and Scala Spark is recommended. Recommended Apache Spark in Python:beginner ' s Guide. Recommended toree-0.2.0.dev1.tar.gz Apache Toree hadoop-common-2.2.0-bin FAQs How do I visualize data? Only one sparkcontext maybe running into this JVM