Esframework 4.0 Quick Start (11)-use a compact serializer to improve performance several times

In a distributed communication system, the network transmits binary streams, while the memory is a variety of objects built based on the object model, when we need to pass an object to another node over the network, we first need to serialize it into a byte stream, and then send it to the target node through the network. After receiving the object, then deserialize it into an object instance. The esframework system follows the same rules.

Esframework calls these objects that need to pass through the network as contract. Protocol classes are generally a simple data structure encapsulation and are a dummy class used to save the status (excluding any methods, except for those inherited from objects), it is a bit similar to the anemia entity mapped to the table in the database. (For more information about contract, seeEsframework 4.0 advanced (01) -- message). Esframework-based distributed systems use these protocol-class instances for data exchange. If the skeleton process is self-assembled and implemented using the icontracthelper provided by esplus, The esframework underlying layer will automatically help us complete protocol Object serialization and deserialization.

However, if we use the rapid engine, we often use esplus. application. customizeinfo. passive. icustomizeinfooutter interface send binary custom information (in fact, it is automatically completed by esframework when the underlying layer is converted to message and then to byte stream ), this binary information is usually the serialization result of a service object in the system, and this serialization process must be completed by ourselves.

 

I. Two Methods of serialization

To convert a business object to a binary stream, you can use either of the following methods. net binary serializer, Or, first convert the business object to a string (such as XML), and then encode the result with UTF-8 to get the byte stream. Both solutions have defects.

1. Binary serializer of. net

Using the binary serializer that comes with. NET is the most convenient for us, because we only need to call the API directly and do not need to do any action by ourselves. However, the defects are also obvious:

(1) first, the serializable label must be added to the class that can be serialized, And the binary serializer of. NET must be bound to the namespace andProgramSet. For example, deserialization cannot be completed if the result of serializing a protocol object in Windows is passed to Silverlight, even if Silverlight has the same protocol class definition. This also indicates that if the communication client is developed using Silverlight, the binary serializer of. Net cannot be used.

(2) the serialization result is bloated and its size is huge. In a distributed communication system with high concurrency and performance, this will reduce communication messages and waste a lot of bandwidth, which is intolerable.

(3) low efficiency .. The built-in binary serializer of net works based on the reflection mechanism, such as reading type information and reading/setting field values. If you only serialize 100,000 objects per second, you may still be able to cope with it. However, if you need to serialize tens of thousands or even hundreds of thousands of objects per second, you will be overwhelmed.

(4) encryption is difficult. Because. net binary serializer will reflect the internal Member (private) of the read object when serializing the Protocol object. If the DLL of the definition protocol is encrypted, the name of a private member is often obfuscated into a random name, which requires that all parties involved in the communication use the same encrypted DLL. Otherwise, deserialization of the protocol object may fail.

 

2. Use string to transfer data

To solve the serialization unification problem similar to the communication between the Silverlight client and the server, we can use string as the transit, and use the same format (such as utf8) for the same string regardless of whether the Silverlight environment is used) the byte stream is certainly the same. In turn, the same byte stream uses the same decoder for decoding in different environments to obtain the same string. This is a feasible solution, but there are also some disadvantages:

(1) We need to build our own conversion between Protocol objects and strings-this may be a time-consuming and error-prone job.

(2) The size of the serialized result depends on the rules used when the Protocol objects and strings are converted to each other. It also depends on our patience. To make the size of the result smaller, we may need more brains.

Of course, there are also ready-made tools for converting objects and strings, such as JSON and XML. In general, the result of using the JSON conversion protocol object is much smaller than that of using XML.

(3) similar to the binary serializer of. net, reading/setting of object property values and object creation are usually based on reflection. Therefore, there are also problems with efficiency.

 

Ii. solution 3

Esplus provides a more compact binary serializer that does not need to be bound to namespaces and assembly: esplus. serialization.Compactpropertyserializer.

(1) compactpropertyserializer is built based on emit and cache technologies, and avoids the overhead caused by reflection, so the efficiency is greatly improved.

(2) compactpropertyserializer uses concise and compact serialization formats and rules to make the serialization result size shorter.

(3) The serialized class does not need to add any labels and does not depend on the namespace and Assembly. As long as the class definition is completely consistent, compactpropertyserializer can work normally. Real weak intrusion.

(4) esframework. SL also provides the same compactpropertyserializer. Therefore, based on the 3rd point, the server and the Silverlight client can work together.

(5) As long as the serialization format of compactpropertyserializer is used, the. NET Server can work with clients built in any other language (such as C ++ and Java.

Compactpropertyserializer solves the preceding problems. Of course, it is not all-powerful and has some restrictions. We will discuss it later.

 

Iii. How to Use compactpropertyserializer

Esplus. serialization.CompactpropertyserializerAnd esframework. SL. serialization.CompactpropertyserializerIs a binary serializer provided by esframework for desktop applications and sivlerlight clients. Their implementations are almost the same, so you should pay attention to the same aspects during use:

(1) compactpropertyserializer supports serialization of classes and structures, but the serialized classes or structures must have Default constructors ).

(2) compactpropertyserializer only serializes thoseRead/writeIf a property is read-only or write-only, the property will not be serialized. This is also compact.PropertyDescription of the property in the serializer name.

(3) types supported by compactpropertyserializer: Basic Data Types (such as int, long, bool, etc.), String, byte [], and class or struct composed of these types.

(4) multi-layer Nesting is supported-The serialized class can contain other types of objects, as long as each embedded object is finally composed of the basic data type.

(5) Except for byte []/list <>/dictionary <,> generic, other set types are not supported.

(6) Circular references are not supported. If a circular reference exists, the serialization will lead to an endless loop.

As mentioned in this Article, the Protocol classes used in communication systems are some of the simplest "dumb classes" that only contain data. Therefore, the above restrictions do not apply to the design of protocol classes. Use a simple data type as much as possible, and then expose the fields to be serialized through readable and writable attributes.

Compactpropertyserializer includes two basic methods of serialization and deserialization:

Byte [] Serialize < T > (T obj );
T deserialize < T > ( Byte [] Buff, Int Startindex) Where T: New ();

The method has obvious meaning and is not explained. In addition, compactpropertyserializer uses singleton. We can directly use this Singleton in the program and use compactpropertyserializer.DefaultProperty to obtain the reference of the singleton instance.

 

4. Try compactpropertyserializer's performance and efficiency

Take the beginsendfilecontract protocol class to be used in file transfer as an example. beginsendfilecontract is defined as follows:

View code

[Serializable] Public   Class Beginsendfilecontract
{
# Region Originfilepath
Private   String Originfilepath;
///   <Summary>
/// The full path of the file sent by the sender.
///   </Summary>
Public   String Originfilepath
{
Get { Return Originfilepath ;}
Set {Originfilepath = Value ;}
}
# Endregion

# Region Filelength
Private   Long Filelength =   0 ;
Public   Long Filelength
{
Get
{
Return   This . Filelength;
}
Set
{
This . Filelength = Value;
}
}
# Endregion

# Region Lastupdatetime
Private   Datetime Lastupdatetime;
///   <Summary>
/// The last update time of the file to be sent.
///   </Summary>
Public   Datetime Lastupdatetime
{
Get { Return Lastupdatetime ;}
Set {Lastupdatetime = Value ;}
}
# Endregion

# Region Fileid
Private   String Fileid =   "" ;
Public   String Fileid
{
Get
{
Return   This . Fileid;
}
Set
{
This . Fileid = Value;
}
}
# Endregion

# Region Comment
Private   String Comment =   Null ;
///   <Summary>
/// Comment is used to store additional information related to this file sending. For example, in a file upload service similar to FTP, the Comment content can be the path where the server saves the uploaded file.
///   </Summary>
Public   String Comment
{
Get { Return Comment ;}
Set {Comment = Value ;}
}
# Endregion
}

The [serializable] label is added because the binary serializer that comes with. NET is required in the following test. The formal beginsendfilecontract definition does not contain this label.

Like most protocol classes, this class only contains several simple type attributes. Now let's compare the performance of the binary serializer that comes with. Net with compactpropertyserializer.

(1) Compare the serialization result size:

Beginsendfilecontract Contract =   New   Beginsendfilecontract ();
Byte [] Result1 =   Compactpropertyserializer . Default. serialize < Beginsendfilecontract > (Contract );
Byte [] Result2 = Esbasic. helpers. Serializehelper . Serializeobject (contract );

Esbasic. helpers. serializehelper is a simplified encapsulation of the binary serializer that comes with. net. The execution result is as follows:

The result1 length is:32

The result2 length is:242

. Net built-in serializer results are 7-8 times the compactpropertyserializer results. If you assign meaningful values to some string-type fields of contract, this multiple will be slightly lower; if the contract definition contains more attributes to be serialized, the multiples will continue to increase. In any case, this ratio is very scary, so in high-frequency communication systems, compared to using the. NET built-in serializer, using compactpropertyserializer can save a lot of bandwidth.

(2) test performance:

We run two serialization 1 million times respectively, depending on the desired time:

Beginsendfilecontract Contract =   New   Beginsendfilecontract ();
Stopwatch Stopwatch =   New   Stopwatch ();
Stopwatch. Start ();
For ( Int I =   0 ; I <   1000000 ; I ++ )
{
Byte [] Result1 =   Compactpropertyserializer . Default. serialize < Beginsendfilecontract > (Contract );
}
Stopwatch. Stop ();
Double Span1 = Stopwatch. elapsedmilliseconds;

Stopwatch. Reset ();
Stopwatch. Start ();
For ( Int I =   0 ; I <   1000000 ; I ++ )
{
Byte [] Result2 = Esbasic. helpers. Serializehelper . Serializeobject (contract );
}
Stopwatch. Stop ();
Double Span2 = Stopwatch. elapsedmilliseconds;

The running result is as follows:

Span1:5324MS

Span2:17249MS

Compactpropertyserializer is more than three times faster than the serialization tool provided by. net. Its advantage is self-evident.

You can refer to the demo above to write more test programs to test more content, including their comparison in deserialization.

 

For general communication applications, use. net's built-in binary serializers may be enough and won't have much impact. However, if MMORPG, video/audio conferencing, and so on require high-frequency and high-performance communication systems ,. net's built-in binary serializer is not the best choice. If you use esframework to build your distributed communication application, you can get more help from compactpropertyserializer.

 

Esframework 4.0 Overview

What are the advantages of esframework 4.0?

Esframework 4.0 upgrade description (continuous update)

Esframework 4.0 Quick StartArticle

All articles in esframework 4.0 advanced edition Series