Quick StartThis article describes how to use scala, java, and python to compile a spark click Mode Program. First, you only need to successfully build Spark on a machine. Practice: Enter the Spark root directory and enter the command: $ sbt/sbt package(Because of the Great Firewall of tianchao, the mainland China cannot succeed, unless you can smoothly flip the wall). If you do not want to climb the wall, you can download the pre-compiled Spark, spark
1. PreparationThis article focuses on how to build the Spark 2.11 stand-alone development environment in Ubuntu 16.04, which is divided into 3 parts: JDK installation, Scala installation, and spark installation.
JDK 1.8:jdk-8u171-linux-x64.tar.gz
Scala 11.12:scala 2.11.12
Spark 2.2.1:spark-2.2.1-bin-hadoop2.7.tgz
It is important to note that
Note: The Scala version I used is 2.11.8
First, the Hello World code, as follows:
Object helloscala{
def main (args:array[string]): Unit = {
println ("Hello World ...")}
}
Here is a brief description of some of Scala's knowledge Points: first, Scala data types
Scala's data types include: Byte, Char, short, Int, Long, Float, Double, Boolean
Note: Scala
Scala entry-Eclipse Development Environment setup
Due to the need for deep learning, I recently started to learn Scala, an artistic programming language.
Scala: a java-like programming. Integrates the features of Object-Oriented (Oriented) programming and Functional (Functional programming) programming, which facilitates the development of multi-core concurrency
To use Spark, learn Scala first.Reference Tutorial: Http://meetfp.com/zh/scala-basicOnline compilation: Http://meetfp.com/tryoutOther information: HTTP://SCALACHINA.COM/NODE/16 http://blog.csdn.net/mapdigit/article/details/21878083Characteristics:
Scala is a purely object-oriented language, all objects: Unify primitive types and classes, and unify functi
Array/ ing/tuples
I. Array
1. Fixed Length Array
Two forms of declaring an array:
Declare the Val array name of the specified length = new array [type] (array length)
Provides an array of initial values without the new keyword.
Scala declares an array with an array class name and uses () to specify the length or provide an initial value sequence.
In JVM, scala array is implemented in Java array
checksumaccumulator { private var sum = 0 = { + = b } = { return ~ (sum 0xFF) + 1 }}Any parameters passed to the method can be used inside the method. An important feature of the method parameters in Scala is that they are all Val, not var. If you want to reassign a parameter to a method, the result is a compilation failure:Scala> def Add (b:byte): Unit = { | // compile however, because B is Val | sum + = b| } a membe
Description, since Spark was written in Scala. Therefore, it is best to use Scala whether you are looking at the source code or writing spark-related codes. Then, as a programmer, the first thing to do is to sharpen the sword in hand. That's creating a code environment for writing Scala. Here because I am personally familiar with eclipse (although I think he is a
Background Notes
Recent research spark need Scala to prepare a Scala environment for documenting the configuration notes.The Scala language is based on the Java language, so you have to install the JDK first. Experimental Environment: Win7 under VMware Virtual machine operating system: CentOS 6.5 x64 java:jdk-8u161-linux-x64.tar.gz scala:scala-2.12.4.tgz Install
Reprinted from Http://www.tuicool.com/articles/jYzuAv and http://www.importnew.com/1537.htmlIn a keynote speech that I collaborated with Martin Fowler, he made an insightful point:The legacy of Java will be the platform, not the programming language.The original engineers of Java technology made a wise decision to separate the programming language from the runtime environment, which ultimately enabled more than 200 languages to run on the Java platform. This architecture is critical to the long-
01. Introduction and installation of Scala 1.1 Scala introductionScala is the Java language scripting, the feature is that the Java language is not scripted to use the script, so that it has a simple script language, WYSIWYG features, and high programming efficiency, fast implementation and other characteristics. In the big Data world, the famous frameworks for Spark and Kafka are written in the
Functional Object
Advantages and disadvantages of immutable object
Adv 1. Immutable objects are more likely to be inferred because they do not change values over time.
Adv 2. immutable objects can be automatically transmitted without changes. To transfer a mutable object, you must first back up the mutable object to prevent modification to the object state in other processes.
Adv 3. The thread cannot change the state of an immutable object (corruption, corruption). It is destined to be thread-sa
1.scala InterpreterThe installation process is similar to a JDK installation, you need to install the JDK first, and then download and install Scala to configure the PATH environment variable.2. Run cmd in the win environment using WIN+R, bring up the command window, execute the callout command①scala Support TAB Command Auto-completion②
As we all know, Scala is a functional programming language of the Geek type, and the features supported include:
The function has a "class citizen" status;
Support for anonymous functions (volume of function numbers)
Supports higher order functions
Support closures
Partial application functions
Currying
The first point to note is that there are two forms of methods and function objects in
Scala List class Faq:how do I create a list in Scala?You can create a Scala List in several different ways, including these approaches:
Lisp style
Java style
Using the List class range method
Using the List class fill method
Using the List class tabulate method
In this Scala List tutorial,
Scala REPLRes1,res2, ... res5 and so on, respectively, indicate the output result. If the result of a computed output is identified as Res2, then the res2 can be used directly in subsequent calls.Cases:Scala Worksheek in the Scala IDEThis is a very useful feature provided by the Scala IDE. Its function is the equivalent of Sc
Once an object is created, it will no longer change its contents, so that the object is immutable. This can be done without worrying about competition management when accessing objects in multiple threads, and Java's string is an example of an immutable object. Based on this, when you use Scala to create multithreaded applications, you can write unlocked code with the invariant state (immutable states) to write concise multithreaded code without worry
This chapter describes Scala's support for XML, including the main contents of this article at the level of headings.
A description of the concept and XML format of semi-structured data is skipped, directly into Scala's XML-related syntax and usage. First, XML syntax
The Scala command line can be entered directly at the beginning of a tag, and the compiler automatically enters the XML input pattern until it encounters a closing flag that matches the t
the system is windows. The installation configuration Scala is as follows:
One, install Scala
1,java6 above (recommended Java 7 +) installation
Address: http://www.oracle.com/technetwork/java/javase/downloads/index.html
Configuring environment variables after installation
2,scala 2.12.2
Address: http://www.scala-lang.org/download/
3. Download the post-installat
Come from:http://lidrema.blog.163.com/blog/static/209702148201461145859142/Scala: A Java-like programming. Integrates object-oriented (object-oriented) programming and functional (functional programming) programming features to develop multi-core concurrency models and cloud computing (parallelism for Mulicore and cloud Comp uting). Bibliography: "Learn Scala", Author:horstmannbefore installing the
The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion;
products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the
content of the page makes you feel confusing, please write us an email, we will handle the problem
within 5 days after receiving your email.
If you find any instances of plagiarism from the community, please send an email to:
info-contact@alibabacloud.com
and provide relevant evidence. A staff member will contact you within 5 working days.