Functional Programming (38)-Functional stream Io:io Process in action

In the previous sections of the discussion we finally came to a generalized and generic IO process type process[f[_],o]. This type can also represent both a data source and a data terminal (Sink). In this discussion we will demonstrate its combination (composibility) and the functional integrity of the entire process from the data source to the receiving terminal IO through a number of application examples for the characteristics of process[f,o].

We have investigated and experimented with the various functions of IO process in the previous discussion, and now let's explore the data source design: In order to achieve the security of resource use and the composable nature of IO programs, we must ensure that the data source can be released regardless of the resource usage or the occurrence of an exception. At the same time, all side effects must be deferred until interpreter begins the arithmetic IO program.

Let's try a component that reads the file character content:

  Import java.io. {BufferedReader, FileReader}  def readFile (filename:string): process[io,string] =    await[io,bufferedreader,string] (io{new bufferedreader (new FileReader (FileName)}) {Case left    (err) = Halt (err) Case Right    (r) = {    lazy val Next:process[io, String] = await (Io{r.readline}) {case left    (ERR2) = await (Io{r.close}) {_ = = Halt[io,string] (ERR2)} ()    Case right (line) = Emit (line, Next)    } ()    next    }    ()

Note the following questions: First, all IO actions are implemented by the await function, and then all statements that produce side effects are wrapped in io{}, which is a typical delay operation. Let's take a look at this await function:

  def Await[f[_],a,o] (Req:f[a]) (     rcvfn:either[throwable,a] = process[f,o] = (A:either[throwable,a]) = = Halt[ F,o] (end))     (Fallback:process[f,o] = Halt[f,o] (end),     onerror:process[f,o] = Halt[f,o] (end)): process[f,o] = Await (Req,rcvfn,fallback,onerror)

Req is a f[a], in the example is Io[a]. The concise representation of this function is: await (IO (iorequest)) (IOResult = = Process). From the style of this function, we can see that there is no side effect when calling the await function, because the statement that produces the side effect is nested inside io{}. We need a interpreter to compute this io{...} Cause side effects. Another input parameter for await is iorequest = = Process:iorequest is the result of the operation Iorequest return: either A or string or an exception exception. So we can use Partialfunction to express:

{Case Left (err) = Process[io,???]; Case Right (a) = Process[io,???]}

We use Partialfunction to describe the processing methods that return normal results or anomalies after the operation Iorequest.

The above example ReadFile is to use this await function to open the file: IO {new BufferedReader (new FileReader (FileName)})

Read one line: IO {r.readline}, if read succeeds send emit out: Case Right (line) = Emit (Line,next),

If an exception is present Close File: Case Left (err) = = IO {R.close}, note that we will use the exception end to represent the normal completion of the read:

  Case Object End extends Exception case  Object Kill extends Exception

The above kill is a forced stop signal. As I said earlier, we need to use a interpreter to compute readfile to really produce the desired side effects such as reading and writing files. Now let's get to know a interpreter:

  def Collect[o] (Src:process[io,o]): indexedseq[o] =  {     val E = Java.util.concurrent.Executors.newFixedThreadPool (4)     def go (Cur:process[io,o], Acc:indexedseq[o]): Indexedseq [O] = cur match {case        Halt (e) = + ACC case        Emit (os,ns) = Go (NS, ACC + + OS) case        Halt (err) = throw E RR case        Await (RQ,RF,FB,FL) = +            val Next =              Try RF (right (Unsafeperformio (RQ) (E)))              Catch {case ERR: Throwable = RF (Left (ERR))}            Go (Next, ACC)             }     try Go (src, indexedseq ())     finally E.shutdown  }

The first thing to note is this sentence: Unsafeperformio (RQ) (E). It can really produce side effects. IO is performed when the current state of Process[io,o] SRC is await. The result of the operation IO is the input parameter for the RF function of await, as we described above. So, the operation Io{iorequest} is to build an await structure to put the iorequest and the transition state function RF into it like this:

Await (Iorequest) (RF) (FB,FL) = await (IOREQUES,RF,FB,FL),

Then return to Collect,collect to see that the SRC state is await and will operate the iorequest and then run RF.

Our next question is how to read the contents of a file one line at a time rather than moving it all into memory at once, so that we can read a line, handle a row, and consume the least memory. Let's take a closer look at this part of ReadFile:

  def readFile (filename:string): process[io,string] =    await[io,bufferedreader,string] (io{new bufferedreader (new FileReader (FileName)}) {Case left    (err) = Halt (err) Case Right    (r) = {    lazy val Next:process[io, String] = await (Io{r.readline}) {case left    (ERR2) = halt[io,string] (ERR2)//await (Io{r.close}) {_ = = Halt[io, String] (ERR2)} () Case right    (line) = Emit (line, Next)    } ()    next    }    ()

If BufferedReader is created successfully, the operation IO produces the right (r) result; Operation Io{r.readline} returns the final result next. Next may be halt (err) or emit (Line,next). If this analysis is done then the entire ReadFile function will be read into the first line of the file and then emit output. In addition to the recursive algorithm, it is remembered that functional programming features program operations in State machines. In the above example the results of the operation in addition to the output value of line there is a next state next. Take a look at the following component:

  Status carry, output PROCESS[F,O2]  final def Drain[o2]: Process[f,o2] = This match {case  Halt (e) = = Halt (e)      //termination    case Emit (os,ns) = Ns.drain  //operation next state NS, output cases  await (RQ,RF,FB,CL) = await (RQ, RF andthen (_.drain) )//Still output await  }

this drain component actually acts as a moving state. If we write this: ReadFile ("MyFile.txt"). Drain so in our example above ReadFile return emit (line,next), drain then readFile output state will operate next, This procedure goes back to ReadFile Next's Io{r.readline} operation. If we add a few more components before the drain component:

ReadFile ("Farenheit.txt"). Filter (line =!line.startswith ("#"). Map (line + line.touppercase). Drain

Then we'll get to read a line of characters, filter the lines starting with #, turn into uppercase characters, and return to the effect of reading another line of alternating loops.

Obviously ReadFile is too targeted. The function type Variant has a low complexity of readability. We need a more generalized form to implement the concise and fluent expression of functional programming languages.

We should first redefine the IO operation. The await function seems too complex:

  The streamlined expression of await  def Eval[f[_],a] (Fa:f[a]): process[f,a] =//Operation F[a]    Await[f,a,a] (FA) {case left    (err) = Halt (Err) Case right    (a) = Emit (A, Halt (End))  } ()  def Evalio[a] (ioa:io[a]) = Eval[io,a] (IOA)   // Operation Io[a]  //Determining the Operation  def Eval_[f[_],a,b] (Fa:f[a]): process[f,b] = Eval[f,a] (FA). Drain[b]//operation F[a] until terminated

so the arithmetic IO only needs to write this: eval (iorequest), is not much more streamlined.

One more generic secure IO resource uses the component function:

def Resource[r,o] (//   general Purpose IO program operation function      Acquire:io[r]) (  //Get IO resource. Open file      use:r = Process[io,o]) (  //io operation function  readLine      release:r = Process[io,o]): Process[io,o ] =//release resource function close file    eval (acquire) flatMap {r = = use (r). OnComplete (Release (R))}    def Resource_[r,o] (
   
    //and resource, just the arithmetic realease until the end      Acquire:io[r]) (  //Get IO resources. Open file      use:r = Process[io,o]) (  //io operation function  readLine      release:r = Io[unit]): Process[io,o] =//Release resource function Close file    resource (acquire) (use) (Release andthen (Eval_[io,unit,o]))
   

The following is an example of applying a resource component: Reading from a file, and actively releasing resources when a read or exception is completed:

  def lines (filename:string): process[io,string] =//Read     resource      {io {io} from FileName. Source.fromfile (FileName)}}  //occupies resource     {src =  //use resource. Progressive Read       lazy val iter = src.getlines        def nextline = if (iter.hasnext) Some (iter.next) Else None//Next line       lazy val ge Tlines:process[io,string] =  //Read         eval (io{nextline}) FlatMap {  //Op IO case         None + = Halt (End)   //cannot Continue reading: Complete or exception case         Some (line) = Emit (line, getlines)//Read and send         }       getlines     }     { src = Eval_ (io{src.close})}//Release resources

So easy to read and easy to solve.

Now we should be able to describe an IO program in a concise but yet clear and detailed manner:

Open File Fahrenheit.txt

Reads a line of characters

Filter blank lines or lines that start with #, lines that can be used to represent the Heinz temperature number

Convert the Heinz temperature to a temperature of Celsius

The temperature conversion function in this is as follows:

  def FahrenheitToCelsius (f:double): Double =    (f-32) * 5.0/9.0

So the whole program can be written like this:

      Lines ("Fahrenheit.txt").      Filter (line =!line.startswith ("#") &&!line.trim.isempty).      Map (line = FahrenheitToCelsius (line.todouble). toString).      Drain

This code is not very clear description of what it represents the function, good!

Now it's time to understand the other end of the IO process: sink. If we need to implement the output function through process, that is, Source[o] This O send output to a sink. In fact we can also use process to express sink, first look at a simple version of the sink as follows:

  Type Simplesink[f[_],o] = Process[f,o = F[unit]]        

Simplesink is an IO Process, and its output is an O + f[unit] function, explained by an example:

  def simplewritefile (filename:string, Append:boolean = False): Simplesink[io, string] =    Resource[filewriter, string = = Io[unit]]     {IO {new FileWriter (Filename,append)}}   //acquire    {w = io{(s:string) = io{ W.write (s)}}}  //use    {w = io{w.close}}  //release

Here is a sink to use:

    Type Sink[f[_],o] = process[f, O = Process[f,unit]]    import java.io.FileWriter    def filew (file:string, append : Boolean = False): sink[io,string] =      resource[filewriter, String = Process[io,unit]]        {IO {new FileWriter (fi Le, append)}}        {w = stepwrite {(s:string) = Eval[io,unit] (IO (W.write (s))}}//Repeat loop line by row        {w = = Eval_ ( Io (w.close))}/    * An infinite loop constant stream. */    def Stepwrite[a] (A:A): process[io,a] =      eval (IO (a)). flatMap {a = E MIT (A, Stepwrite (a))} repeats the Operation Io (a) via the next state of the emit

We need to implement progressive output, so we use this stepwrite to compute IO. The Stepwrite is implemented by returning emit to the infinite loop.

The next step is to connect the sink with the process. We can use the following to components to connect:

    def To[o2] (Sink:sink[f,o]): process[f,unit] =      

The join component is the standard monadic component, because we need to flatten Process[f,process[f,unit]] to Process[f,unit]:

    def Join[f[_],a] (P:process[f,process[f,a]): process[f,a] =       p.flatmap (PA = pa)

Now we can add a component to write celsius.txt in the process of the previous example:

    Val Converter:process[io,unit] =      lines ("Fahrenheit.txt").//Read      filter (line =!line.startswith ("#") & &!line.trim.isempty). Filter      map (line = FahrenheitToCelsius (line.todouble). toString).  Temperature conversion      Pipe (intersperse ("\ n")).  Add end of line to      (Filew ("Celsius.txt")).  Write      drain     //Resume Loop

The above sink type operation IO does not return any results (Unit). But sometimes we want the IO operation to return something, such as a result set after the arithmetic database query, and we need a new type:

    Type Channel[f[_],i,o] = process[f, I = Process[f,o]]

The channel and the sink are very similar, and differ only in process[f,o] and Process[f,unit].

We use channel to describe a database query:

    Import java.sql. {Connection, PreparedStatement, ResultSet} def query (Conn:io[connection]): Channel[io, Connection = Prepare        Dstatement, Map[string,any]] =//map = = = Row Resource_//i >>> Connection = PreparedStatement {conn}//Open Connection {conn = constant {(Q:connection = preparedstatement) =//Circular Query Res Ource_ {IO {//Run query val rs = q (conn). executeQuery val ncols = Rs.getmet            Adata.getcolumncount val cols = (1 to ncols). Map (Rs.getMetaData.getColumnName) (RS, cols)                  }} {case (rs, cols) =//read record row def step = if (!rs.next) None Else Some (Cols.map (c = (c, Rs.getobject (c): any)). Tomap) Lazy Val rows:process[io,map[                  String,any]] =//Loop read eval (step). flatMap {case None = = Halt (End)  Case Some (row) = Emit (row, rows)//loop operation rows function} rows} {p =& Gt IO {P._1.close}}//Close the ResultSet}} {c = = IO (c.close)}

More application demonstrations are provided below:

From a file to read the file name of the Heinz temperature after the temperature conversion and put into the Celsius.txt

    Val Convertall:process[io,unit] = (For {out      <-filew ("Celsius.txt").  the type of once//out is String = Process[io, Unit]      file <-lines ("Fahrenheits.txt")//fahrenheits.txt saved a string of file names      _ <-lines (file).  Dynamic open file read temperature record           map (line = FahrenheitToCelsius (line.todouble)).//Temperature System conversion           FLATMAP (Celsius = celsius.tostring)//Output    } yield ()) drain  //Resume Loop

Output to multiple. Celsius files:

    Val Convertmultisink:process[io,unit] = (For {file      <-lines ("Fahrenheits.txt")//Read file name      _ <-lines (file) . Open file read temperature data           map (line = FahrenheitToCelsius (line.todouble)).//Temperature System Conversion           Map (_ toString).            to (Filew (file + ". Celsius"))//write to File    } yield ()) drain

We can add processing components to the process as needed:

    Val Convertmultisink2:process[io,unit] = (For {      file <-lines ("Fahrenheits.txt")      _ <-lines (file).           Filter (!_.startswith ("#")). Filter # Start string           map (line = FahrenheitToCelsius (line.todouble)).           Filter (_ > 0). Filter 0 degrees below the temperature           map (_ toString).           to (Filew (file + ". Celsius"))    

Copyright NOTICE: This article for Bo Master original article, without Bo Master permission not reproduced.

Functional Programming (38)-Functional stream Io:io Process in action