Spark: Learning the--37 of essays

"Fast Learning Scala" the fourth after class exercises:

1. Set a map that contains some of the equipment you want, along with their price. Then build another map with the same set of keys, but the price hits 90 percent.

2. Write a program to read the word from the file. Use a mutable map to count how often each word appears. Reading these words can be done using Java.util.Scanner:

3. Repeat the previous exercise, this time with an immutable mapping

4. Repeat the previous exercise, this time using sorted mappings so that the words can be printed sequentially

5. Repeat the previous exercise, this time using JAVA.UTIL.TREEMAP and making it suitable for the Scala API

6. Define a chain hash map, map "Monday" to Java.util.Calendar.MONDAY, and then add other dates in turn. Presentation elements are accessed in the order in which they are inserted

7. Print out a table of all Java System Properties

8. Write a function Minmax (Values:array[int]) to return the duality of the minimum and maximum values in the array

9. Write a function iteqgt (Values:array[int],v:int) that returns the number of less than V, equal to V, and greater than V in the array, requiring three values to be returned together

10. When you zip the two strings together, such as "Hello". Zip ("world"), what will be the result. Come up with a well-spoken use case

/////////////////////////////////////////////////////////////////////////////////////////////////////////////// /////////////////////////////////////////////////////////////////////////////////////

Package SK Import Java.util.Calendar import scala.collection.JavaConversions.propertiesAsScalaMap Import Org.apache.spark.
{sparkcontext, sparkconf} import org.apache.spark.sparkcontext._ Import Scala.collection.mutable.LinkedHashMap Import Scala.collection. {mutable, sortedmap} import scala.collection.immutable.
 {TREEMAP, HashMap} import scala.io.Source/** * Created by Sendoh on 2015/5/4. */Object Answerx {def main (args:array[string]): Unit ={//1. Hit 90 percent val Map1 = Map ("Pen"-5, "book"-20
    , "iphone"--(k, v) <-map1) yield (k, V * 0.9)//print words and occurrences val conf = new sparkconf ()  Val sc = new Sparkcontext (conf) val datawords = Sc.textfile ("Example.txt"). FlatMap (_.split (")"). Map (Word = 1). Reducebykey (_ + _). Collect () println (datawords)//2-val in = new Java.util.Scanner (New Java.io.File ("myfile.t 
    XT ") while (In.hasnext ()) handles In.next () or turns to the 9th chapter to see a more scala approach.
    Finally, print out all the words and the number of times they appear. Val Source2 = Source.fromfiLe ("Example.txt"). Mkstring.split ("\\s+") val map2 = new hashmap[string, Int] for (key <-source2) {map2 (k EY) = Map2.getorelse (key, 0) + 1} println (Map2.mkstring (","))//3-the difference between immutable mappings and mutable mappings is that each time an element is added, it returns a new mapping Val s Ource3 = Source.fromfile ("Example.txt"). Mkstring.split ("\\s+") var map3 = new hashmap[string, Int] for (key <- Source3) {map3 + = (Key-(Map3.getorelse (key, 0) + 1)} println (Map3.mkstring (","))//4-and the above code has no area No, just modify the mapping to SortedMap val source4 = Source.fromfile ("Example.txt"). Mkstring.split ("\\s+") var MAP4 = Sortedmap[strin G, Int] () for (key <-Source4) {MAP4 + = (Key---(Map4.getorelse (key, 0) + 1)} println (Map4.mkstr  ING (","))//5 val source5 = Source.fromfile ("Example.txt"). Mkstring.split ("\\s+") Val map5:map[string, Int] = New treemap[string, Int] for (key <-Source5) {MAP5 (key) = Map5.getorelse (key, 0) + 1} println (MAP5
. mkstring (","))//6    Val map6 = new linkedhashmap[string, Int] map6 + = ("Monday", calendar.monday) Map6 + = ("Tuesday" Ca Lendar. Tuesday) Map6 + = ("Wednesday"-calendar.wednesday) Map6 + = ("Thursday"-calendar.thursday) Map6 + = ( "Friday", calendar.friday) Map6 + = ("Saturday", calendar.saturday) Map6 + = ("Sunday" Calendar.sund  AY)//7-attribute to Scala map using Val props:scala.collection.map[string, String] = System.getproperties () Val keys = Props.keyset val keylengths = for (key <-keys) yield key.length val maxkeylength = Keylengths.max for (
      Key <-keys) {print (key) print ("* (maxkeylength-key.length)) Print (" | ") 
    println (Props (key))}//8 def maxmin (Values:array[int]): Unit ={(Values.max, values.min)}//9 def iteqgt (Values:array[int], v:int): Unit ={val buf = Values.tobuffer (Values.count (_ < v), values . Count (_ = = v), Values.count (_ &Gt V)}//10 "Hello". Zip ("World")//res0:scala.collection.immutable.indexedseq[(char, char)] = Vector ((h,w)
 , (E,o), (L,r), (l,l), (O,d))}}

/////////////////////////////////////////////////////////////////////////////////////////////////////////////// //////////////////////////////////////////////////////////

Simpleskewedgroupbytext:spark Example

Package SK import Org.apache.spark. {sparkcontext, sparkconf} import org.apache.spark.sparkcontext._ import scala.util.Random/** * Created by Sendoh on 201
 5/5/4. */Object Simpleskewedgroupbytext {def main (args:array[string]): Unit ={val conf = new sparkconf (). Setappname ("Si Mpleskewedgroupbytext ") Val sc = new Sparkcontext (conf) var nummappers = if (args.length > 0) args (0). ToInt Els E 2 var numkvpairs = if (Args.length > 1) args (1). ToInt else, var valsize = if (Args.length > 2) args (2) . ToInt else, var numreducers = if (Args.length > 3) args (3). ToInt else nummappers var ratio = if (Args.leng
      Th > 4) args (4). ToInt Else 5.0//Val pairs1 = sc.parallelize (0 until nummappers, nummappers). flatmap{p = Val Rangen = new Random val result = new array[(Int, Array[byte])] (numkvpairs) for (i <-0 until NUMK Vpairs) {val Bytearr = new Array[byte] (valsize) rangen.nextbytes (Bytearr) ValOffset = Rangen.nextint (+) * numreducers if (Rangen.nextdouble < ratio/(numreducers + ratio-1)) {
          Result (i) = (offset, Bytearr)} else {val key = 1 + rangen.nextint (numReducers-1) + offset Result (i) = (key, Bytearr)}} result}.cache pairs1.count println ("Result:" + pairs1.
 Groupbykey (numreducers). Count) Sc.stop ()}}