Serialization | Iot framework ServerSuperIO tutorial-10. Two methods (such as file) for continuous transmission of large data streams, serversuperio

Serialization | Iot framework ServerSuperIO tutorial-10. Two methods (such as file) for continuous transmission of large data streams, serversuperio

1. C # Introduction to cross-platform Iot communication framework ServerSuperIO (SSIO)

Serialization | Iot framework ServerSuperIO tutorial 1.4 communication modes and mechanisms.

Serialization | Iot framework ServerSuperIO tutorial 2. service instance configuration parameters

Serialization | Iot framework ServerSuperIO tutorial-3. Device Driver Introduction

Serialization | Iot framework ServerSuperIO tutorial-4. For example, you can develop a device driver that supports both serial and network communication.

Serialization | Iot framework ServerSuperIO tutorial-5. Polling communication mode development and precautions.

Serialization | Iot framework ServerSuperIO tutorial-6. concurrent communication mode development and precautions

Serialization | Iot framework ServerSuperIO tutorial-7. Self-control communication mode development and precautions

Serialization | Iot framework ServerSuperIO tutorial-8. single-instance communication mode development and precautions

Serialization | Iot framework ServerSuperIO tutorial-9. protocol filter to solve multiple packet-related, sticky packet, and redundant data

 

Contents

10. Two Methods for continuous transmission of large data streams (such as files)... 2

10.1 overview... 2

10.2 Two transmission modes for large data streams... 2

10.2.1 Protocol packet method... 2

10.2.2 request length, confirmation method... 3

10.3 achieve continuous transmission of large data blocks... 4

10.3.1 design request sending data protocol... 4

10.3.2 client code implementation... 4

10.3.3 implementation of the ServerSuperIO framework and precautions... 7

10.4 running results... 11

10. Two Methods for continuous transmission of large data streams (for example, files) 10.1 Overview

The ability to integrate multiple data sources is essential to the current situation of the Internet of Things or the perception of the Internet of Things, especially the Industrial Internet of Things. There are two types of data sources: hardware generation and Software Generation. For example:

 

Based on the actual situation, the IOT framework must have the ability to integrate various types of data and various application scenarios. Take the data size as an example. The data size is as small as the data within the capacity range of one receiving cache and as large as the data that exceeds the capacity range of one receiving cache. It is possible as long as the network permits it. Previous articles on serialization are based on small data packets. This article introduces the transmission mode of large data streams.

10.2 Two transmission modes for large data streams 10.2.1 protocol data packets

This method defines the beginning and end of the packet protocol, and splits large data blocks into small data packets of a certain length, data is transmitted in batches in a combination of protocol headers, small data packets, and Protocol tails. After receiving each batch of data, perform data verification and assemble the data to restore the complete data. As follows:

 

This method has the following problems:

(1) If there is a problem with the data in each package, resend the data. The protocol should be designed to complete the re-issuance mechanism.

(2) data sources are diverse, such as compressed files, serialized files, and encrypted files, therefore, the data of each small data packet may be consistent with the protocol header or the Protocol end, or even the CRC check, which leads to the failure of normal data verification and parsing. At this time, the data is re-sent, A high probability event may occur in similar cases.

This method of transmitting large data streams should be selected based on the actual situation on the site to avoid possible risks and improve the overall stability of the project and product.

If you choose this method, you can implement it according to the previous article. You can implement it yourself. This article mainly introduces the following method.

10.2.2 Request Length and Confirmation Method

The client sends the request to send the data command, and marks the length of the data to be sent. After receiving the request command, the server determines whether the request data length is within the permitted range, and then returns the same command data or other validation data to the client, indicates whether data of this length can be sent. If it can be sent, the client continues to send data streams, and the server also continues to receive data streams. As follows:

 

For this data transmission method, ServerSuperIO provides a dedicated interface. The following is a detailed introduction.

10.3 implement continuous transmission of large data blocks 10.3.1 design request sending data protocol

The request sends the 0x62 command, a total of 10 bytes. The Checksum is the sum starting from the "server address", excluding the "Data Header", "checksum", and "Protocol end ".

The data frame of the Request instruction is as follows:

Private void btSendFile_Click (object sender, EventArgs e) {try {if (this. _ tcpClient = null) {return;} if (! File.Exists(this.txt FilePath. text) {WriteLog ("select a file"); return;} byte [] backData = new byte [10]; backData [0] = 0x55; backData [1] = 0xaa; // protocol header backData [2] = byte. parse (this. numericUpDown1.Value. toString (); // The slave address backData [3] = 0x62; // command int count = (int) (new FileInfo(this.txt FilePath. text )). length; byte [] countBytes = BitConverter. getBytes (count); backData [4] = countBytes [0]; backData [5] = countBytes [1]; backData [6] = countBytes [2]; backData [7] = countBytes [3]; byte [] checkSum = new byte [6]; Buffer. blockCopy (backData, 2, checkSum, 0, checkSum. length); backData [8] = (byte) checkSum. sum (B => B); // calculate the checksum and backData [9] = 0x0d; this. _ tcpClient. client. send (backData, 0, backData. length, SocketFlags. none);} catch (SocketException ex) {Disconnect (); WriteLog (ex. message );}}

After receiving confirmation information from the server, the code for sending persistent data is as follows:

Private void SendFile () {FileStream fs = null; try {if (this. _ tcpClient = null) {return;} string fileName = this.txt FilePath. Text; if (! File. exists (fileName) {WriteLog ("select file"); return;} WriteLog ("Start Transmission>"); byte [] sendBuffer = new byte [1024]; fs = new FileStream (fileName, FileMode. open, FileAccess. read, FileShare. read); long length = fs. length; int count = 0; Stopwatch watch = new Stopwatch (); watch. start (); while (length> 0) {int sendNum = fs. read (sendBuffer, 0, sendBuffer. length); sendNum = _ tcpClient. client. send (sendBuffer, 0, sendNum, SocketFlags. none); length-= sendNum; count + = sendNum; float percent = (fs. length-length)/(float) fs. length) * 10010000f; WriteLog ("passed:" + percent. toString ("0.00") + "%");} watch. stop (); WriteLog ("transfer completed! Total: "+ count. toString () + ", time consumed:" + watch. elapsed. totalSeconds. toString (CultureInfo. invariantCulture);} catch (SocketException ex) {this. disconnect (); WriteLog (ex. message);} catch (Exception ex) {WriteLog (ex. message);} finally {if (fs! = Null) {fs. Close (); fs. Dispose ();}}}

10.3.3 implementation of the ServerSuperIO framework and precautions

There is basically nothing to talk about the code implementation of the client. It mainly introduces how to implement the Code based on the ServerSuperIO framework in a device-driven manner. Note: The following automatic control mode is implemented.

DeviceProtocol: The ProtocolDriver interface has a GetPackageLength (byte [] data, IChannel channel, ref int readTimeout) function interface. The data parameter is the command for sending data requests, the channel parameter is an instance of the current IO channel, and readTimeout is the time used to customize the length of data to be returned. If the returned value is 0, it is considered that the data is not continuously received. You can use the channel parameter to directly return the confirmation information. The specific code is as follows:

Public override int GetPackageLength (byte [] data, IChannel channel, ref int readTimeout) {if (data = null | data. length <= 0) return 0; readTimeout = 2000; if (CheckData (data) {try {if (data [3] = 0x62) // send the file request {int length = BitConverter. toInt32 (new byte [] {data [4], data [5], data [6], data [7]}, 0); if (length <= 1024*1024) // limit {int num = channel. write (data); if (num> 0) {Console. writeLine ("Returned file request confirmation data"); return length;} else {return 0 ;}} else {return 0 ;}} else {return 0 ;}} catch (Exception) {return 0 ;}} else {Console. writeLine ("verification error"); return 0 ;}}

 2.Implementation of protocol commands

To process large data blocks, a special protocol command is added for parsing and saving data. The Code is as follows:

internal class DeviceFileCommand:ProtocolCommand    {        public override string Name        {            get { return CommandArray.FileData.ToString(); }        }        public override dynamic Analysis<T>(byte[] data, T t)        {            if (t != null)            {                string path = AppDomain.CurrentDomain.BaseDirectory + DateTime.Now.ToString("yyyyMMddHHmmss") + ".txt";                File.WriteAllBytes(path, t as byte[]);                return path;            }            else            {                return null;            }        }}

When receiving a large Data stream, all Data information is returned to the device-driven Communicate interface. The info parameter Data is the current request Data command, and BigData is the information of continuously receiving Data, by calling this. protocol. driverAnalysis protocol interface driver protocol command DeviceFileCommand. The Code is as follows:

Public override void Communicate (ServerSuperIO. communicate. IRequestInfo info) {string hexs = BinaryUtil. byteToHex (info. data); OnDeviceRuningLog ("receipt>" + hexs); byte [] cmds = this. protocol. getCommand (info. data); CommandArray cr = (CommandArray) cmds [0]; dynamic obj = this. protocol. driverAnalysis <byte []> (cr. toString (), info. data, info. bigData); if (obj! = Null) {if (cr = CommandArray. realTimeData) {_ deviceDyn. dyn = (Dyn) obj;} else if (cr = CommandArray. fileData) {OnDeviceRuningLog ("file storage path:" + obj. toString () ;}} OnDeviceRuningLog ("normal communication ");}

It is mainly used to configure StartCheckPackageLength = true in the configuration parameters. During the data connection process, it will detect the Protocol Interface GetPackageLength of the corresponding device driver.

Static void Main (string [] args) {DeviceSelfDriver dev2 = new DeviceSelfDriver (); dev2.DeviceParameter. deviceName = "network device"; dev2.DeviceParameter. deviceAddr = 1; dev2.DeviceParameter. deviceID = "1"; dev2.DeviceDynamic. deviceID = "1"; dev2.DeviceParameter. deviceCode = "1"; dev2.DeviceParameter. NET. remoteIP = "127.0.0.1"; dev2.DeviceParameter. NET. remotePort = 9600; dev2.CommunicateType = CommunicateType. NET; dev2.Initialize ("1"); IServer server = new ServerManager (). createServer (new ServerConfig () {ServerName = "Service 1", ComReadTimeout = 1000, ComWriteTimeout = 1000, NetReceiveTimeout = 1000, NetSendTimeout = 1000, ControlMode = ControlMode. self, SocketMode = SocketMode. tcp, StartReceiveDataFliter = true, ClearSocketSession = false, StartCheckPackageLength = true, CheckSameSocketSession = false, DeliveryMode = DeliveryMode. deviceIP,}); server. addDeviceCompleted + = server_AddDeviceCompleted; server. deleteDeviceCompleted + = server_DeleteDeviceCompleted; server. start (); server. addDevice (dev2); while ("exit" = Console. readLine () {server. stop ();}}

10.4 Running Effect

Image

Video

 

1. [serialization] C # communication (Serial Port and network) Framework Design and Implementation

2. [Open Source] C # cross-platform Iot communication framework ServerSuperIO (SSIO) Introduction

2. Overall system construction solution using SuperIO and open-source cross-platform Iot framework ServerSuperIO

3. C # technical route of industrial IoT and integrated system solutions (data sources, data collection, data upload and receiving, ActiveMQ, Mongodb, WebApi, and mobile App)

5. ServerSuperIO Open Source Address: https://github.com/wxzz/ServerSuperIO

Internet of Things & integrated technology (. NET) QQ Group:54256083