Geotrellis using (vi) Scala concurrent (parallel) programming

This article is mainly about Scala concurrent (parallel) programming, then why the topic is called Geotrellis use (vi), mainly because this series explains how to use Geotrellis, the specific previous several blog has been introduced. I think the basis of doing any one thing is very important, like the college entrance examination or a variety of examinations, teachers will emphasize the basis, which is very reasonable. The basics of using the Geotrellis framework are Scala and spark, so this article first introduces the Scala programming language, and it's also important to understand Scala's concurrency (parallel) programming, the Scala Foundation, Without the Scala language foundation there is no more Scala concurrent programming than the use of geotrellis or spark, this article is a brief introduction to the basics of Scala, this is a lot of books or articles, we can search online.

First, the Scala foundation

The most important thing about Scala is pattern matching, which makes the whole Scala language flexible and convenient, the popular saying pattern matching is the switch case in other languages, but the function is far more complex, involving the sample class (case Class), Unapply functions and other specific online have a lot of introduction. Second, there are powerful for expressions, partial functions, implicit conversions, and so on, the following mainly introduces Scala concurrent (parallel) programming.

ii. introduction of SBT

With the Scala language programming, it's best to use the SBT framework, which automatically helps you with package management and so on, equivalent to Maven in Java, so let's start with a brief introduction to the SBT Foundation.

      Install SBT First, it's simple, just download the installation package (http://www.scala-sbt.org/release/docs/ installing-sbt-on-windows.html), the specific installation process and configuration, etc., we can also find on the Internet. After the installation is complete, install the SBT plugin in idea and choose the Create SBT project, the main difference with the common Scala language is to create a BUILD.SBT file, which mainly records the dependencies of the project, and add the following two lines of code to add dependencies:

 librarydependencies + =  "  % "  akka-actor_2.11  % "  2.4.4   resolvers  + =  " akka Snapshot Repository  "  at Span style= "color: #800000;" > " http://repo.akka.io/snapshots/ " 

Actually, the BUILD.SBT file is a Scala source file that is managed directly by SBT, and the statements are all in accordance with Scala syntax, where librarydependencies and resolvers are defined key,+=% At and all are good ways to write. Librarydependencies is the storage system-dependent key, which adds a ModuleID object,"com.typesafe.akka" to GroupID," akka-actor_2.11" For Artifactid,2.4.4" is the revision,% method that eventually creates a ModuleID object, where you need to be aware that _2.11 represents the current Scala version. Resolvers indicates how the system can find the above Librarydependencies,at method by creating a Resolver object with two strings, which is the name and the latter is the address. In general Lib's official website will have to write their own above statements for users to easily add their lib dependency.

Third, concurrent programming

Here's how you can use Scala for concurrent programming.

1. Native support

The Scala language natively supports concurrent programming, just to make the class inherit scala.actors.Actor, to replicate the parent's Act method, or to create an anonymous class directly using actor{}, where receive is a partial function, Used to receive and process messages sent by other actors, where pattern matching is used, which can be handled differently depending on the message type, which is equivalent to routing.

1Object ActorTest2extendsApp {2Val Actor_a:actor =actor{3      while(true){4 Receive {5          Casemsg = println ("actor_a" +msg)6       }7     }8   }9 TenVal Actor_b =actor{ One      while(true){ A Receive { -          Casemsg = { -println ("Actor_b" +msg) theActor_a! "b---->>> a" -Sender! "Receive" +msg -         } -       } +     } -   } +  AActor_a! "WSF" atActor_b!Math.PI -}

The above code defines two actor actor_a,actor_b, in which case the actor automatically start, then sends a message to each actor in the main thread, and the actor receives the message and makes a simple print operation. Since Scala has deprecated this approach for concurrent programming, here's a brief introduction, let's look at how to do concurrent programming by using Akka.

2, Akka

      Akka is an easy-to-use Scala concurrency programming framework (URL: http://akka.io/) with the purpose of "Build powerful concurrent & Distributed Applications more easily. Introducing Akka only needs to add the code described in the SBT Operations section in the BUILD.SBT file, but it will be modified according to your own Scala version and the version of Akka you want to use. Once added, idea will automatically download Akka's actor package. Its use is essentially the same as the native actor, which also creates a class that inherits Akka.actor. Actor, the replication of its receive method. The code is as follows:

1 class extends actor{2   override def receive={3case      message:string=> println (message) 4      Case _ = = unhandled ()5  }6 }

Unlike the native actor, Akka adds the concept of path to its actor, that is, each actor has an absolute path, so that the system first creates one and then creates the actor under system, with the following code:

Val system = Actorsystem ("Akkatest")

Val actor = system.actorof (Props (Classof[wsf.akkascala.myactor]), "Akkaactor")

where Actorsystem ("Akkatest") creates a akka system for managing actors, the second sentence is to create a Myactor instance above in system. By printing Actor.path You can get akka://akkatest/user/akkaactor, you can see that the actor is actually under System, where user represents the user custom actor.

The actor instance is created without start and will start automatically and can use actor! "Hello actor" Statement to send a message to the actor, the Myactor receive method receives the statement and then makes a pattern match if it can match on the row to handle accordingly.

Since the actor has a path, it is also able to create its own actor instance, just add the following code to the current actor class:

Val otheractor = context.actorof (Props (Classof[otheractor]), "Otheractor")

Where Otheractor is another actor defined, the print Otheractor.path can get the following effect: Akka://akkatest/user/akkaactor/otheractor, This indicates that a child actor is indeed created in Myactor. Myactor can manage instances of otheractor.

The above introduces the concurrent programming of Akka, and its parallel programming should be modified slightly.

First set up a Remoteactor project to change the reference of the project in BUILD.SBT to Librarydependencies ++=Seq ("Com.typesafe.akka"% "akka-actor_2.11"% " 2.4.4","Com.typesafe.akka"% "Akka-remote_ 2.11 "% " 2.4.4 ") , you can see that compared to the normal actor project just adds aAkka-remoteReference. A new Remoteactor class is then inherited from the actor, no different from the normal actor. You then create a main class to start the actor. The only thing to be aware of is to create a new application.conf file in the Resources folder, which is the system's configuration file, which adds the following code:

1 Akka {2loglevel = "INFO"3 actor {4Provider = "Akka.remote.RemoteActorRefProvider"5   }6 Remote {7Enabled-transports = ["Akka.remote.netty.tcp"]8 netty.tcp {9hostname = "127.0.0.1"TenPort = 5150 One     } ALog-sent-messages = on -Log-received-messages = on -   } the}

The main definition is to use the TCP protocol for data transmission, the port is 5150. This completes the definition of remoteactor.

Then create a new Localactor project, Similarly modify the contents of the BUILD.SBT file as above, and then create a new Localactor class, because here need to send a message to Remoteactor, so you must establish a Remoteactor child actor, the specific command is as follows:

Val remoteactor = context.actorselection ("Akka.tcp://[email protected]:5150/user/remoteactor")

Where Akka://remotesys/user/remoteactor is the path created by Remoteactor through system, which differs from Akka after adding. TCP is created by TCP and then remotesys through @ 127.0.0.1:5150 Specifies the IP address and port of the remote actor. This allows you to create an instance of Remoteactor that can send messages to remoteactor through this instance.

Localactor also need to add the application.conf file, but only need to add the following statement:

1 Akka {2  actor {3     Provider = "Akka.remote.RemoteActorRefProvider"4    }5 }

Iv. Summary

This article briefly introduces the Scala Foundation, the SBT simple operation, the native actor, the concurrency of the Akka and the parallel mode actor, these are some of the lessons I learned in the process of learning geotrellis, and it is only by playing a good foundation that we can develop my knowledge better. Know it and know why. Understand that this can be very helpful for reading Geotrellis source code and spark source code.

V. Reference links

A preliminary study on the use of Geotrellis
Second, geotrellis use (b) Geotrellis-chatta-demo and Geotrellis frame data reading method
Iii. Geotrellis Use (iii) analysis of GEOTRELLIS data processing process
Iv. use of Geotrellis (iv) Geotrellis Data Processing Section detailsV. Geotrellis use (v) Use of Scala operations AccumuloVi. Geotrellis Use (vi) Scala concurrent (parallel) programming

Geotrellis using (vi) Scala concurrent (parallel) programming