Akka (in): Http:file streaming-File Exchange

the so-called file Exchange refers to the HTTP protocol between the server and the client file upload and download. Akka-http as a system integration tool should have efficient data exchange methods including file Exchange and database table row upload and download. The Akka-http data exchange pattern supports streaming operations: represents an element that can be an infinite length stream for exchanging data. This mode first solves the need of the data segmentation operation and complex message property setting which the pure HTTP big data must carry through the multipart transmission, moreover the user can also conveniently use the Akka-stream to carry on the deep processing to the data, eliminates the data conversion the trouble. More importantly: Akka-http also supports Reactive-stream, which avoids the problems caused by the transfer rate. In this article we discuss the two-way transfer of files using Akka-http.

The content storage format of any file is a bunch of bytes on the hard drive, memory, or data line. The file exchange process consists of reading the bytes in the file, transmitting the bytes, and finally writing the bytes to the file. We see that each link here is bytes, so you may not need any data conversion process in the program. Akka provides a set of file read and write functions, as follows:

def frompath (F:path, chunksize:int =8192): Source[bytestring, Future[ioresult]] =Frompath (F, chunkSize, StartPosition=0) def frompath (F:path, Chunksize:int, Startposition:long): Source[bytestring, Future[ioresult]=Source.fromgraph (NewFilesource (f, ChunkSize, StartPosition, Defaultattributes.filesource, Sourceshape ("Filesource")) def topath (F:path, options:set[openoption]= Set (WRITE, truncate_existing, CREATE)): Sink[bytestring, Future[ioresult]] =Topath (F, Options, StartPosition=0) def topath (F:path, options:set[openoption], Startposition:long): Sink[bytestring, Future[ioresult]=Sink.fromgraph (NewFilesink (f, startposition, Options, Defaultattributes.filesink, Sinkshape ("Filesink")))

We see: The Frompath type is the Source[bytesgtring,_],topath type is sink[bytestring,_], is directly the flow pattern, should be directly placed in the HTTP message entity, as follows:

  def fileStream (filepath:string, Chunksize:int): source[bytestring,any] = {    = {      ///    implicit val ec = httpSys.dispatchers.lookup ("Akka.http.blocking-ops-dispatcher")      val file = Paths. Get (FilePath)      Fileio.frompath (file, chunkSize)        . Withattributes (Actorattributes.dispatcher ("  Akka.http.blocking-ops-dispatcher")    }    Limitablebytesource (loadFile)  } 

FileStream is source[bytestring,_] can be placed directly into the entity:

  Val uploadtext = HttpRequest (Httpmethods.post,uri = s"http://localhost:8011/file/text" )  = httpentity (    contenttypes. ' Application/octet-stream ',    fileStream ( " /users/tiger-macpro/downloads/a4. TIF",()  )

We put the FileStream into the HttpRequest. For HttpResponse, you can use the following method:

  val route = Pathprefix ( " file  "  ) {( get  & Path ("  text           "/Remaining) {fp => Withoutsizelimit { Complete (httpentity (contenttypes. ' Application /octet-stre Am ', FileStream (  " /users/tiger-macpro/
      " + FP, 256    

Note: Complete was built with HttpResponse. Because Entity.databyes is source[bytestring,_], so we can directly import it into the sink:

          Entity.dataBytes.runWith (Fileio.topath (Paths.  Get(destpath)            ) Case _ = println (S"Download file saved to: $destPath  ") }

We mentioned above that Fileio.topath is a sink. Since our aim is large file exchange, Withoutsizelimit is used regardless of the upload download:

Val route = Pathprefix ("file") {    (Get& Path ("Exchange"/Remaining)) {fp =withoutsizelimit {Complete (httpentity (contenttypes. ' Application/octet-Stream ', FileStream ("/users/tiger-macpro/"+ FP, the))        )      }    } ~(Post& Path ("Exchange") {withoutsizelimit {extractdatabytes {bytes=Val fut= Bytes.runwith (Fileio.topath (Paths.Get(DestPath))) OnComplete (fut) {_=Complete (S"Save upload file to: $destPath")            }          }        }      } 

Okay, here's the sample code that shows the exchange of character or binary files:

Service side:

import akka.actor._import akka.stream._import akka.stream.scaladsl._import Akka.http.scaladsl.Httpimport Akka.http.scaladsl.server.directives._import Akka.http.scaladsl.model._import Akka.http.scaladsl.model.httpentity._import java.nio.file._ObjectFileserver extends App {ImplicitVal Httpsys = Actorsystem ("Httpsystem")  ImplicitVal Httpmat =Actormaterializer ()ImplicitVal Httpec =httpsys.dispatcher def fileStream (filepath:string, Chunksize:int)={def loadFile= {       //implicit val ec = httpSys.dispatchers.lookup ("Akka.http.blocking-ops-dispatcher")Val file = Paths.Get(FilePath) fileio.frompath (file, chunkSize). Withattributes (Actorattributes.dispatcher ("Akka.http.blocking-ops-dispatcher")} limitablebytesource (LoadFile)} Val destpath="/users/tiger-macpro/downloads/a4-1. TIF"Val Route= Pathprefix ("file") {    (Get& Path ("Exchange"/Remaining)) {fp =withoutsizelimit {Complete (httpentity (contenttypes. ' Application/octet-Stream ', FileStream ("/users/tiger-macpro/"+ FP, the))        )      }    } ~(Post& Path ("Exchange") {withoutsizelimit {extractdatabytes {bytes=Val fut= Bytes.runwith (Fileio.topath (Paths.Get(DestPath))) OnComplete (fut) {_=Complete (S"Save upload file to: $destPath")}}}} Val (port, host)= (8011,"localhost") Val bindingfuture=Http (). Bindandhandle (Route,host,port) println (s"Server running at $host $port. Press any key to exit ...") Scala.io.StdIn.readLine () Bindingfuture.flatmap (_.unbind ()). OnComplete (_=httpsys.terminate ())}

Client:

import akka.actor._import akka.stream._import akka.stream.scaladsl._import Akka.http.scaladsl.Httpimport Akka.http.scaladsl.model.HttpEntity.limitableByteSourceimport Akka.http.scaladsl.model._import Java.nio.file._ Import Akka.util.ByteStringimport scala.util._ObjectFileclient extends App {ImplicitVal sys = Actorsystem ("Clientsys")  ImplicitVal Mat =Actormaterializer ()ImplicitVal EC =sys.dispatcher def downloadfileto (Request:httprequest, destpath:string)={val Futresp=Http (SYS). Singlerequest (Request) Futresp. andthen { CaseSuccess ([email protected] (Statuscodes.ok, _, Entity, _)) =Entity.dataBytes.runWith (Fileio.topath (Paths.Get(DestPath))) . oncomplete { Case_ = = println (s"Download file saved to: $destPath") }         CaseSuccess ([email protected] (code, _, _, _)) = =println (S"Download request failed, response code: $code") r.discardentitybytes () CaseSuccess (_) = println ("Unable to download file!")         CaseFailure (Err) = println (s"Download failed: ${err.getmessage}")}} val dlfile="Downloads/readme.txt"Val Downloadtext= HttpRequest (URI = s"http://localhost:8011/file/exchange/"+dlfile) Downloadfileto (Downloadtext,"/users/tiger-macpro/downloads/sample.txt") Scala.io.StdIn.readLine () Val dlFile2="Downloads/image.png"Val downloadText2= HttpRequest (URI = s"http://localhost:8011/file/exchange/"+dlFile2) Downloadfileto (DOWNLOADTEXT2,"/users/tiger-macpro/downloads/sample.png") Scala.io.StdIn.readLine () def uploadfile (Request:httprequest, dataentity:requestentity)={val Futresp=Http (SYS). Singlerequest (Request.copy (Entity=dataentity)) Futresp. andthen { CaseSuccess ([email protected] (Statuscodes.ok, _, Entity, _)) =Entity.dataBytes.map (_.utf8string). Runforeach (println) CaseSuccess ([email protected] (code, _, _, _)) = =println (S"Upload request failed, response code: $code") r.discardentitybytes () CaseSuccess (_) = println ("Unable to Upload file!")         CaseFailure (Err) = println (s"Upload failed: ${err.getmessage}")}} def fileStream (filepath:string, Chunksize:int): Source[bytestring,any]={def loadFile= {      //implicit val ec = httpSys.dispatchers.lookup ("Akka.http.blocking-ops-dispatcher")Val file = Paths.Get(FilePath) fileio.frompath (file, chunkSize). Withattributes (Actorattributes.dispatcher ("Akka.http.blocking-ops-dispatcher")} limitablebytesource (LoadFile)} Val uploadtext= HttpRequest (Httpmethods.post,uri = s"Http://localhost:8011/file/exchange") Val TextData=httpentity (contenttypes. ' Application/octet-Stream ', FileStream ("/users/tiger-macpro/downloads/readme.txt", the) ) UploadFile (uploadtext,textdata) scala.io.StdIn.readLine () sys.terminate ()}

Akka (in): Http:file streaming-File Exchange