Spark builds demo code reading environment under Eclipse v2-

Source: Internet
Author: User

Https://files.cnblogs.com/files/wifi0/spark2.1.1example_api_sql_streaming_eclipseProject.zip
Https://files.cnblogs.com/files/wifi0/runconfig.zip

Building Code Reading Environment

Download spark-2.1.1-bin-hadoop2.7.tgz

Http://spark.apache.org/downloads.html

Decompression spark-2.1.1-bin-hadoop2.7.tgz

Note: The jar package under the examples directory and all the jar packages under Examples\jars will be used to create a eclipes userlibrary. Name is: Spark2.1.1jars

Download Winutils

Https://github.com/steveloughran/winutils

Figure 6 After decompression:

Create Spark2.1.1jars

Eclipse->window->preferences->java->build Path->user liberary->new

Enter Spark2.1.1jars. Add External Jars.

Select all the jar files in the Spark/jars and spark/examples/jars two directories.

Create Hadoop_home

Eclipse->window->preferences->run/debug->string Substitution,hadoop_home Point

f:/tddownload/winutils-master/winutils-master/hadoop-2.7.1

Set up eclipse to compile with jdk8 and use Jdk8 lib.

Run eclipse as an administrator

Import Example

File->import->existing Projects into Workspace->select Archive File

Select Spark2.1.1example_api_sql_streaming_eclipseproject.zip

Are you sure.

This is what it looks like after importing. When you then run Spark, the program adds a directory to the inside, paying attention to cleanup.

Install a MySQL server on this machine, Root,root. or modify the corresponding code DataSource.

Spark builds demo code reading environment under Eclipse v2-

Related Article

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.