Datasnap stream transfers Big Data

Datasnap can directly pass and return parameters of the tstream type, which is very convenient. However, many people find that the job is abnormal when the size is slightly larger.

Datasnap's default cache size is 32 K. Therefore, if the stream size exceeds this size, it will be automatically divided into multiple packages, which is the basis for passing a large amount of data, if one-time sending is performed, memory restrictions may apply.

When a large amount of data is transferred, the obtained data size is-1. Therefore, if you still read the data of the stream in the normal way, the problem may occur.

Because the data in the stream is sent by the original data packet, the transmission speed is different from other methods without data packet compression and encryption.

0102030405060708091011121314151617181920212223

// FS is a file streamfunction TMyDSServer.PutFile(Stream: TStream): Boolean;const  BufSize = $F000;var  Buffer: TBytes;  ReadCount: Integer;begin  if Stream.Size = -1 then// If the size is unknown, data is read until no data is available.  begin    SetLength(Buffer, BufSize);    repeat      ReadCount := Stream.Read(Buffer[0], BufSize);      if ReadCount > 0 then        FS.WriteBuffer(Buffer[0], ReadCount);      if ReadCount < BufSize then        break;    until ReadCount < BufSize;  end  else // If the size is known, the data is copied directly.    FS.CopyFrom(Stream, 0);  Result := True;end;

 

Function tdmcommonfun. downloadfile (const filename: string): Boolean;
VaR
A: tservermethods1client;
INI: Tinifile;
Stream, Ms: tstream;
Buffer: tbytes;
Readcount: integer;
Const
Bufsize = $ f000;
Begin
Result: = false;
If (not tryconnectappserver) or (filename = '') then
Exit;
A: = tservermethods1client. Create (sqlconnection1.dbxconnection );
MS: = tmemorystream. Create;
Try
Stream: = A. downloadfile (filename );
If stream. size =-1 then
Begin
Setlength (buffer, bufsize );
Repeat
Readcount: = stream. Read (buffer [0], bufsize );
If readcount> 0 then
Ms. writebuffer (buffer [0], readcount );
If readcount <bufsize then
Break;
Until readcount <bufsize;
End
Else
Begin
Ms. copyfrom (stream, 0 );
End;
// Delete BAK files
If fileexists (extractfilepath (application. exename) + filename + 'bak') then
Deletefile (pwidechar (extractfilepath (application. exename) + filename
+ 'Bak '));
// Rename an existing file
If fileexists (extractfilepath (application. exename) + filename) then
Begin
Renamefile (extractfilepath (application. exename) + filename,
Extractfilepath (application. exename) + filename + 'bak ');
End;
// Download the latest file
Tmemorystream (MS). savetofile (extractfilepath (application. exename) +
Filename );
// Update the local version
INI: = Tinifile. Create (extractfilepath (application. exename) + 'client. ini ');
Try
INI. writeinteger (filename, 'ver ', getver (filename ));
Finally
INI. Free;
End;
Finally
A. Free;
Ms. Free;
End;
Result: = true;
End;