"Scala" Scala technology stack

Get a quick look at the Scala technology stack

I was hopelessly a huge fan of Scala. After I used the Scala development project and wrote the framework, it seemed to condense into a huge black hole, and the attraction made me fly to it so that I started to deviate from Java. Although Java 8 for the Java camp added a glimmer of light, but it is wistfully, awaited began to come out. Scala programmers, however, have long enjoyed the benefits of lambda, higher-order functions, trait, implicit conversions, and so forth.

Java is like a prehistoric beast, it almost reached the extreme in the direction of Oo, hard to pull it into the FP camp, indeed some imposition. Scala does not, because it is the birth of OO and FP-hybrid-perfect gene fusion.

"Object-oriented meets Functional", which is the banner that flies on the official Scala language website. This is also the ambition of Scala, and of course, the ambition of Martin Odersky.

Related Vendor Content

Go all the way technology people's entrepreneurial story future the role of intelligent hardware in the internet of things AI technical map Global Architect Summit Shenzhen station, July 15-16th, South China's largest technology event!

Related Sponsors

Archsummit Shenzhen 2016 will be held in July 15-16 in Oct Intercontinental Hotel, March 20 before the reduction of 2040 yuan!

the development of the Scala community

However, a language cannot exist in isolation, it must provide a platform for attachment, and a biosphere built around it. Not so, the language is not enough to grow. Ruby is excellent, but without the push of Ruby on Rails, it's hard to get to the point where it is today. The same is true for Scala. Conversely, when we use a language, we also choose a technology stack that conforms to the language, and find a framework or tool that fits the specific scenario throughout the biosphere.

Of course, when we use Scala for software development, we can also look for large Java community support, but if you choose to invoke Java-developed libraries, you sacrifice the benefits that Scala brings to us. Fortunately, in today's case, you don't have to. The Scala community, which is evolving with the Scala language, has begun to slowly form a relatively complete Scala technology stack. Whether it's enterprise development, automated testing, or big data, these frameworks or tools are already fully present in Scala's ecosystem.

get a quick look at the Scala technology stack

A good way to learn about the Scala technology stack and learn these frameworks quickly is to download the activator that Typesafe launches. It provides a relatively affluent development template based on Scala and Scala's mainstream framework, which in fact implies the best practices and guidance that Typesafe provides for Scala development. Is the activator template:

So, is there a way to get an overall picture of what frameworks or tools are included in the Scala technology stack, and what are their features and usage scenarios? Thanks to Lauris Dzilums and other contributors on GitHub. On GitHub, Lauris Dzilums, he built a repository called Awesome-scala, which covers the main areas of Scala-based development frameworks and tools, including:

• Database
• Web Frameworks
• i18n
• Authentication
• Testing
• JSON manipulation
• Serialization
• Science and Data analysis
• Big Data
• Functional Reactive Programming
• Modularization and Dependency Injection
• Distributed Systems
• Extensions
• Android
• HTTP
• Semantic Web
• Metrics and Monitoring
• SBT Plugins

Does it feel like a "flower-and-eye" charm? Not too little, but too much! So let me Shanfanjiujian, in my experience, introduce some frameworks or tools, from the persistence, distributed system, HTTP, web framework, big data, testing six aspects, to make a dragonfly water-like view.

Persistence of

Ultimately, the persistence of data is primarily through JDBC access to the database. However, we need better API interfaces, better fit with Scala, or more natural orm. If you want to execute SQL statements to manipulate the database, then using a relatively broad framework SCALIKEJDBC, it provides a very simple API interface, and even provides the SQL DSL syntax. For example:

  Val Alice:option[member] = withsql {    select.from (Member as M). Where.eq (m.name, name)  }.map (rs = Member (RS)) . Single.apply ()

If you want to use an ORM framework, Squeryl should be a good choice. My colleague Yang Yun used the framework in the project, and it was a good experience. The current version of the framework is 0.9.5, which is already relatively mature. Squeryl supports the convention mapping of objects to relational tables, which is equivalent to defining a Poso (Plain old Scala object), thereby reducing the intrusion of the framework. If the mapping violates the Convention, you can define the mapping using the framework-defined annotation such as @column. The framework provides ORG.SQUERYL.TABLE[T] to accomplish this mapping relationship.

Because Scala's higher-order functions, biased functions, and so on can be used to make Squeryl's syntax very natural, such as updating tables based on conditions:

Update (songs) (s = =  where (s.title = = = "Watermelon Man")  set (s.title: = "The Watermelon Man",      s.year  : = s.year.~ + 1))

Distributed Systems

I gave up introductions such as modular management and dependency injection because their value in the Scala community is not as big as the Java community. For example, we have the flexibility to use trait combined with cake pattern to implement dependency injection features. Therefore, I skip these directly to introduce a more influential framework that supports distributed systems.

Finagle's bloodline is noble, from the past was originally poor, now the high gate Han Twitter. Twitter, an internet company that was developed earlier using Scala as a server, has built up a lot of Scala experience and has launched some influential frameworks based on these experiences. Due to the high requirements of Twitter for scalability, performance, and concurrency, these quality attributes are also of great concern to these frameworks. Finagle is one of them. It is an extended RPC system to support the construction of high concurrent servers. I didn't really use finagle in the project, so you can get more information on its official website.

For distributed support, the absolutely akka frame is still the same. It has so much influence that even when the Scala language starts at 2.10, it abandons its actor model and incorporates the Akka actor into the 2.10 version of the language feature. Many frameworks also choose to use Akka in distributed processing, such as Spark, Spray. Akka's Actor model references the Erlang language, provides a proprietary mailbox for each actor, and encapsulates the implementation details of message processing in a good package, making concurrent programming easier. Akka nicely unifies the local actor and the remote actor, providing an almost consistent API interface. Akka also provides a good support for message fault tolerance, in addition to providing a complete set of monitoring mechanisms, as well as the processing of dead letter.

Akka is born to support EDA (Event-driven Architecture). When we model the domain, we can consider modeling events. In Akka, these event models can be defined as Scala's case class and passed as a message to the actor. Borrow The example from Vaughn Vernon in "Implementing Domain driven Design" for the following stream of events:

We can use Akka to simply implement:

Case Class allphonenumberlisted (Phonenumbers:list[int]) Case class phonenumbermatched (Phonenumbers:list[int]) case Class Allphonenumberread (Filename:string) class Phonenumberspublisher (actor:actorref) extends Actorref {def receive = { Case readphonenumbers =>//list Phone numbersactor! Allphonenumberlisted (List (1110))}}class Phonenumberfinder (actor:actorref) extends Actorref {def receive = {case allphonenumberlisted (numbers) =//matchactor! phonenumbermatched ()}}val finder = System.actorof (Prop (New Phonenumberfinder (...)) Val publisher = System.actorof (Prop (New Phonenumberspublisher (Finder))) Publisher! Readphonenumbers ("Callinfo.txt")

We can easily deploy actors such as Phonenumberspublisher, Phonenumberfinder, etc. to the remote Actor if the number of phone numbers to be processed is large. At this point, you just need to change the way the client obtains the actor.

The Finagle implemented by Twitter is for RPC communication, Akka provides internal Message Queuing (MailBox), and Kafka, developed by LinkedIn, provides a distributed Message Queuing middleware that supports high throughput. This message queue, with a writer's hat, can support efficient Publisher-subscriber mode for message processing, and quickly, stably, and scalable features are quickly attracting developers ' attention and are supported by a number of frameworks that are listed as candidate message queues, for example, Spark Streaming supports Kafka as the input Source for streaming data.

HTTP

Strictly speaking, spray is not a simple HTTP framework, it also supports rest, JSON, Caching, Routing, IO and other functions. The spray module and the relationship between them are as follows:

I mainly use spray as a rest framework in my project and work with Akka to deal with domain logic. The schema for the spray processing HTTP requests is as follows:

Spray provides a DSL-style path syntax that makes it very easy to write requests that support various HTTP verbs, such as:

Trait Httpservicebase extends directives with Json4ssupport {implicit Val system:actorsystem implicit def json4s           Formats:formats = Defaultformats def route:route}trait CustomerService extends Httpservicebase {val route = Path ("Customer"/"groups") {get {parameters (' Groupids.?)} {(Groupids) = complete {Group IDs match {case Some (groupids) = Viewusergroup.queryusergr                                   OUP (Groupids.split (","). ToList) Case None = Viewusergroup.queryusergroup ()          }                              }                    }               }          } ~ Path ("Customers"/"VIP"/"Failureinfo") {post {entity (As[failurevipcustomerrequest])       {Request = =                        Complete {vipcustomer.failureinfo (request) }                    }               }          }}

I personally think that in web development, you can completely abandon the web framework, the direct selection of Angularjs combined with spray and Akka, the same can be good to meet web development needs.

Spray supports rest, and spray itself provides the service container Spray-can, allowing standalone deployment (and of course, deployment to application servers such as jetty and Tomcat). Spray's internal processing mechanism for HTTP requests is actually based on Akka-io, which sends a BIND message to HTTP via the IO actor. For example:

IO (Http)! Http.bind (Service, interface = "0.0.0.0", port = 8889)

We can write different boot objects to bind different host hosts and ports. All of these features enable spray to support the more popular Micro service architecture style in the present.

Web Framework

As we said earlier, when we choose spray as the rest framework, it is entirely possible to choose JavaScript frameworks such as ANGULARJS or backbone to develop Web clients. The client can handle its own logic and then send the request to the rest server in JSON format. At this point, we treat the model as a resource (Resource), and the view is completely on the client. JS Controller is responsible for controlling the interface logic of the client, and the controller of the service side is responsible for processing the business logic, so the traditional MVC changes to vc+r+c mode. Here R refers to the resource, while the server and the client communicate through the JSON-formatted resource.

To use a proprietary web framework, the most popular is the play framework, a standard MVC framework, under the Scala technology stack. Another relatively niche web framework is lift. Unlike most web frameworks such as Ror, Struts, Django, and spring MVC, play, it does not use the MVC pattern, but instead uses the so-called view first. It drives developers to create a "separation of concerns" between content generation and content presentation (Markup).

Lift focuses on the view, because in some Web applications, there may be multiple pages facing the same model action. Using a controller in MVC can make control very complex. Lift proposed a kind of so-called View-snippet-model (referred to as VSM) mode.

View is mainly in response to the page request HTML content, divided into template views and generated views. Snippet's responsibilities are used to generate dynamic content and to reconcile model and view when models change.

Big Data

The biggest star of the big Data frame is the non-spark mo genus. Unlike many proprietary big data processing platforms, spark builds on a unified, abstract rdd that allows it to respond in a fundamentally consistent way to different big data processing scenarios, including mapreduce,streaming,sql,machine learning and graph. This is what Matei Zaharia called "Designing a Generic programming abstraction (Unified programming abstraction).

Because Spark has an advanced Dag execution engine, cyclic data flow and memory calculations are supported. Therefore, compared to Hadoop, performance is more excellent. It runs at 100 times times the speed of Hadoop mapreduce in memory and 10 times times on disk.

Because of the Scala language, the total code volume of spark is surprisingly low by using Scala's language features efficiently, and performance has a certain advantage in most aspects (only in streaming, less than Storm). is for the spark 0.9 version of benchmark:

With Scala, the functional nature of the language has been best exploited. In fact, many features of functional language include invariance, no side effects, sub-groups, etc., which are inherently matched with data processing. So, for WordCount, we can do this in such a simple way:

File = Spark.textfile ("hdfs://...") file.flatmap (line = Line.split (""))    . Map (Word = = (Word, 1))    . Reducebykey (_ + _)

If you use Hadoop, it's not so convenient. Fortunately, one of Twitter's open source framework scalding provides abstraction and packaging for Hadoop MapReduce. It allows us to execute the MapReduce job in Scala's way:

Class Wordcountjob (Args:args) extends Job (args) {  TextLine (args ("input"))    . FlatMap (' line-word ') {line: String = Tokenize (line)}    . GroupBy (' word ' {_.size}    . Write (Tsv (args ("output")))  //Split a piece of text into individual words.  def tokenize (text:string): array[string] = {    //lowercase each word and remove punctuation.    Text.toLowerCase.replaceAll ("[^a-za-z0-9\\s]", "" "). Split (" \\s+ ")  }}

Test

While we can write unit tests for Scala project development using, for example, JUnit, testng, use a BDD framework such as Cocumber to write acceptance tests. In most cases, however, we prefer to use scalatest or SPECS2. In some Java development projects, we also started experimenting with scalatest to write acceptance tests and even unit tests.

If I were to choose scalatest or SPECS2, I would prefer to scalatest, because Scalatest supports a variety of styles that cater to different needs, such as the traditional JUnit style, functional style, and spec mode. One of my blogs, "Scalatest's Test style," details the respective syntax.

A widely used test tool is Gatling, which is a performance testing and stress testing tool developed based on Scala, Akka, and Netty. Liu, a colleague of mine, wrote a detailed in-depth introduction to Gatling in the article "New Generation server performance testing Tool Gatling" published in Infoq.

Scalameter is also a very good performance testing tool. We can write Scalameter performance test cases in a style like writing scalatest tests, and can quickly generate performance test data. These features are very useful for benchmark testing for code or software products. We used Scalameter to write performance tests for Scala collections, such as comparing vectors, ArrayBuffer, Listbuffer, and list collections, so that we could better use Scala collections. The following code shows how to write a performance test using Scalameter:

Import Org.scalameter.api._object rangebenchmarkextends Performancetest.microbenchmark {  val ranges = for {    Size <-gen.range ("size") (300000, 1500000, 300000)  } yield 0 until size  measure method "map" in {    using (ran GES) Curve ("Range") in {      _.map (_ + 1)}}  

Select a frame or tool based on the scene

The Scala technology stack is almost bucket compared to the vast community of Java and the huge stack of technology it offers. However, the perfectly formed, not to mention the Scala and Scala technology stack, is still on the way to maturity. For Scala programmers, because the project is different, it may not be able to dabble in all technology stacks, and there are multiple options for different aspects. When selecting these frameworks or tools, you should make judgments based on the actual scenario. To be on the safe side, it is best to use a technical matrix approach to design tradeoffs and decisions for multiple scenarios.

We cannot be complacent, depending on the Java community. After all, the Java framework has gone through thoroughly tempered and has many successful cases as evidence. Focus on Scala technology stack, but do not limit their own vision, possibilities and for, choose the appropriate technical solutions, is the right path to design and development.

Resources:

Get a quick look at the Scala technology stack: http://www.infoq.com/cn/articles/scala-technology/

What is the framework behind Twitter's use of Scala: Http://zhidao.baidu.com/link?url=9Ld-XxLLV9rXrE6i_E_ Irs7mxupqugdplnqyx85xiyc9sn80bmkgrnvtoqnrzjy6u1ha8qwkk_x7yr3qvfxjh9x498bj2pz_c2vgxnwkfiu

Four Scala web frameworks: http://www.jdon.com/idea/scala/scala-web-frameworks-web-developers.html

Spark Tutorial Download: https://www.douban.com/group/topic/65879846/

"Scala" Scala technology stack