Google protocol buffers for. Net Applications

Google protocol buffers for. Net Applications

Wu jian2011-2-22

OriginalArticle, Reprint must indicate the source: http://www.cnblogs.com/wu-jian/

 

Preface

I recently received a cross-platform test project, Server Linux, which is a series of socket interfaces developed by Java and the client windows. Therefore, I plan to use. net. I thought this cross-mainstream platform socket communication should not be a problem, butCodeProcess, with repeated debugging, I found that I was wrong. So far, it has taken a week for the two companies to see their North China Network and China Southern Telecom.

However, there is always an unexpected surprise. Protocol buffer was recognized in the process, which is more powerful than XML, JSON, language-independent, platform-independent, smaller storage, less ambiguity, and higher performance, in fact, Google has been contributing. Whether it's copy left or copy right, let's look back at our Baidu, copy im, copy Mall, copy game, copy video, copy real estate, there is also a Baidu Library that is always ranked first in search but cannot be opened. The impression is that after jquery became popular for a long time, Baidu opened up a JS library, which is also the only contribution of Baidu to Chinese Internet technology, the responsibility of a large company is okay. Let's say it is off the subject.

Protocal buffer official site:Http://code.google.com/p/protobuf/Unfortunately,. NET is not supported,CommunityThe power of the Community version can not be ignored. MySQL has recently launched a community version. From this link, we can see the community camp of protobuf:Http://code.google.com/p/protobuf/wiki/ThirdPartyAddOns

OK. This article mainly describes some of my experiences using protobuf on the application layer in. net. I will share my learning notes with you. I have limited personal abilities. please correct me if you have any shortcomings.

 

Requirement

Java is the server,. NET is the client, socket communication, data encapsulation and transmission using protobuf, such:

The demo constructs three simple. proto files for each client:

Message myrequest {
// Version Number
Required int32 version = 1 ;
// Name
Required string name = 2 ;
// Personal website
Optional string website = 3 [ Default = " Http://www.paotiao.com/ " ];
// Additional data
Optional bytes data = 4 ;
} Message myresponse {
// Version Number
Required int32 version = 1 ;
// Response Results
Required int32 result = 2 ;
} Message mydata {
// Personal Profile
Optional string resume = 1 [ Default = " I'm Goodman " ];
}

Myrequest is a client request, myresponse is the server response, and mydata is attached to the data field of myrequest as an attribute. Note that this byte data field is used, therefore, it takes a maximum of time and finally leads to abandoning protobuf-net for cross-platform applications.

 

Protobuf-Net

Official site:Http://code.google.com/p/protobuf-net/

Protobuf-net is one of the most powerful and widely used third-party systems. It supports. net, C #, WCF, and VB, and has a wide range of demos. It can also find the most information on the Internet.

Generate a. CS File

After installation, use protogen.exe to generate the. CS file from the. proto file (in the demo, I encapsulate the command in/tools/getcs. BAT ):

  Echo on
Protogen-I: protomyrequest. proto-O: protomyrequest. CS
Protogen-I: protomyresponse. proto-O: protomyresponse. CS
Protogen-I: protomydata. proto-O: protomydata. CS

The generated 3. CS files are then included in the project and the protobuf-net.dll is referenced in the project

Sample Code (server and client)

Using system; using system. io; using system. text; using system. threading; using system. net; using system. net. sockets; using system. runtime. serialization; using system. runtime. serialization. formatters. binary; using protobuf; // custom using protomydata; using protomyrequest; using protomyresponse; namespace protobuf_net {class program {Private Static manualresetevent alldone = new manualresetevent (false); static Vo Id main (string [] ARGs) {begindemo ();} Private Static void begindemo () {// start tcplistener Server = new tcplistener (IPaddress. parse ("127.0.0.1"), 9527); server. start (); server. beginaccepttcpclient (clientconnected, server); console. writeline ("server: Waiting for data ---"); // start the threadpool of the client. queueuserworkitem (runclient); alldone. waitone (); console. writeline ("server: Exit ---"); server. stop ();} // server processes PRI Vate static void clientconnected (iasyncresult result) {try {tcplistener Server = (tcplistener) result. asyncstate; using (tcpclient client = server. endaccepttcpclient (result) using (networkstream stream = client. getstream () {// obtain the console. writeline ("server: Client Connected, read data ---"); // proto-Buf uses base128 varints encoding myrequest = serializer. deserializewithlengthprefix <myrequest> (stream, prefix Style. base128); // use C # binaryformatter iformatter formatter = new binaryformatter (); mydata = (mydata) formatter. deserialize (New memorystream (myrequest. data); // mydata. mydata = serializer. deserializewithlengthprefix <mydata. mydata> (New memorystream (request. data), prefixstyle. base128); console. writeline ("server: retrieved successfully, myrequest. version = {0}, myrequest. name = {1}, myrequest. website = {2}, m Ydata. resume = {3} ", myrequest. version, myrequest. name, myrequest. website, mydata. resume); // response (myresponse) myresponse = new myresponse (); myresponse. version = myrequest. version; myresponse. result = 99; serializer. serializewithlengthprefix (stream, myresponse, prefixstyle. base128); console. writeline ("server: Response succeeded ---"); // debug // int final = stream. readbyte (); // If (final = 123) // {// C Onsole. writeline ("server: Got client-happy marker"); //} // else // {// console. writeline ("server: Oops! Something went wrong "); //} console. writeline ("server: close connection ---"); stream. close (); client. close () ;}finally {alldone. set () ;}/// client request private static void runclient (object state) {try {// construct mydata = new mydata (); mydata. resume = "my personal profile"; // construct myrequest = new myrequest (); myrequest. version = 1; myrequest. name = "WU Jian"; myrequest. website = "www.paotiao.com"; // use C # binaryformatter using (memorystream MS = new memorystream () {iformatter formatter = new binaryformatter (); formatter. serialize (MS, mydata); // serializer. serialize (MS, mydata); myrequest. data = Ms. getbuffer (); Ms. close ();} console. writeline ("client: the object has been constructed... "); Using (tcpclient client = new tcpclient () {client. connect (New ipendpoint (IPaddress. parse ("127.0.0.1"), 9527); console. writeline ("client: socket connection successful... "); Using (networkstream stream = client. getstream () {// send console. writeline ("client: send data... "); protobuf. serializer. serializewithlengthprefix (stream, myrequest, prefixstyle. base128); // receives the console. writeline ("client: Waiting for response... "); myresponse = protobuf. serializer. deserializewithlengthprefix <myresponse> (stream, prefixstyle. base128); console. writeline ("client: result obtained successfully, version = {0}, result = {1}", myresponse. version, myresponse. result); // debug client-happy marker // stream. writebyte (123); // close stream. close ();} client. close (); console. writeline ("client: close... ") ;}} catch (Exception error) {console. writeline ("client error: {0}", error. tostring () ;}}// end class}

From the code, we can find that protobuf-net has been very considerate. Whether the client sends an object or the server receives the object, you only need one line of code to achieve it:

// Client sending object
Protobuf. serializer. serializewithlengthprefix (stream, myrequest, prefixstyle. base128 );
// Receiving objects on the server
Myrequest = Serializer. deserializewithlengthprefix < Myrequest > (Stream, prefixstyle. base128 );

Therefore, if both the server and client use. net, protobuf-net will be the ideal choice.

However, my project needs to be cross-platform and the byte field is used in the project. After repeated debugging and comparison, I found a key problem: If the proto script and object property values are identical, but as long as it contains byte fields, the binary serialized by Java is different from the binary serialized by C. Google's native support for Java in protobuf makes it almost clear that protobuf-net has flaws in protobuf support.

After protobuf-CSHARP-sport is used, protobuf-net uses byte [] of C # To implement bytes, while Java and protobuf-CSHARP-port both use bytestring, the former is unsigned, the latter is signed, and the basic difference in language causes the two to be incompatible, so in the end I can only give up protobuf-net.

 

Protobuf-CSHARP-Port

Official site:Http://code.google.com/p/protobuf-csharp-port/

Protobuf-CSHARP-port does not have the same documentation, demo, and application scope as protobuf-Net. However, protobuf-CSHARP-port does not comply with Google's protobuf, and even applications and code are almost the same, therefore, protobuf-CSHARP-port is the best choice across platforms.

Generate a. CS File

Use Google protoc.exe to generate a binary file.

Then, use protogen.exe to generate a class C # binary file (in the demo, I encapsulate the command in/tools/getcs. BAT ):

Echo on
Protoc -- descriptor_set_out = protomyrequest. protobin -- include_imports protomyrequest. proto
Protoc -- descriptor_set_out = protomyresponse. protobin -- include_imports protomyresponse. proto
Protoc -- descriptor_set_out = protomydata. protobin -- include_imports protomydata. proto

Protogen protomyrequest. protobin
Protogen protomyresponse. protobin
Protogen protomydata. protobin

The generated three. CS files are included in the project, and Google. protocolbuffers. dll is referenced in the project.

Sample Code (server and client)

Using system; using system. io; using system. net; using system. net. sockets; using system. threading; using Google. protocolbuffers; namespace protobuf_csharp_sport {class program {Private Static manualresetevent alldone = new manualresetevent (false); static void main (string [] ARGs) {begindemo ();} Private Static void begindemo () {// start the tcplistener Server = new tcplistener (IPaddress. parse ("127.0.0.1"), 9528); server. start (); server. beginaccepttcpclient (clientconnected, server); console. writeline ("server: Waiting for data ---"); // start the threadpool of the client. queueuserworkitem (runclient); alldone. waitone (); console. writeline ("server: Exit ---"); server. stop () ;}// the server processes Private Static void clientconnected (iasyncresult result) {try {tcplistener Server = (tcplistener) result. asyncstate; using (tcpclient client = server. endaccepttcpclient (result) {using (networkstream stream = client. getstream () {// obtain the console. writeline ("server: Client Connected, Data Reading ---"); byte [] myrequestbuffer = new byte [49]; int myrequestlength = 0; do {myrequestlength = stream. read (myrequestbuffer, 0, myrequestbuffer. length);} while (stream. dataavailable); myrequest = myrequest. parsefrom (myrequestbuffer); mydata = mydata. parsefrom (myrequest. data); console. writeline ("server: retrieved successfully, myrequest. version = {0}, myrequest. name = {1}, myrequest. website = {2}, mydata. resume = {3} ", myrequest. version, myrequest. name, myrequest. website, mydata. resume); // response (myresponse) myresponse. builder myresponsebuilder = myresponse. createbuilder (); myresponsebuilder. version = myrequest. version; myresponsebuilder. result = 99; myresponse = myresponsebuilder. build (); myresponse. writeto (Stream); console. writeline ("server: Response succeeded ---"); console. writeline ("server: close connection ---"); stream. close ();} client. close () ;}finally {alldone. set () ;}/// client requests Private Static void runclient (object state) {try {// construct mydata. builder mydatabuilder = mydata. createbuilder (); mydatabuilder. resume = "my profile"; mydata = mydatabuilder. build (); // construct myrequest. builder myrequestbuilder = myrequest. createbuilder (); myrequestbuilder. version = 1; myrequestbuilder. name = "WU Jian"; myrequestbuilder. website = "www.paotiao.com"; // Note: The bytestring type myrequestbuilder is directly supported. data = mydata. tobytestring (); myrequest = myrequestbuilder. build (); console. writeline ("client: the object has been constructed... "); Using (tcpclient client = new tcpclient () {client. connect (New ipendpoint (IPaddress. parse ("127.0.0.1"), 9528); console. writeline ("client: socket connection successful... "); Using (networkstream stream = client. getstream () {// send console. writeline ("client: send data... "); myrequest. writeto (Stream); // receives the console. writeline ("client: Waiting for response... "); byte [] myresponsebuffer = new byte [4]; int myresponselength = 0; do {myresponselength = stream. read (myresponsebuffer, 0, myresponsebuffer. length);} while (stream. dataavailable); myresponse = myresponse. parsefrom (myresponsebuffer); console. writeline ("client: the result is obtained successfully, myresponse. version = {0}, myresponse. result = {1} ", myresponse. version, myresponse. result); // close stream. close ();} client. close (); console. writeline ("client: close... ") ;}} catch (Exception error) {console. writeline ("client error: {0}", error. tostring () ;}}// end class}

 

Protobuf #

Official site:Http://code.google.com/p/protosharp/

Protobuf not tested yet #

 

Conclusion

It took a week to understand and learn about Google protobuf's applications in. Net and the protobuf cross-platform solution.Java nettyI think it may be a platform difference, but I know little about Java. If you have any knowledge, please give me some advice.

 

Demo

Download Demo:Http://files.cnblogs.com/wu-jian/ProtobufDemo.rar

Demo running environment:. NET Framework 4.0, vs2010

 

<Full text>

 

Author: Wu Jian
Source: http://www.cnblogs.com/wu-jian/
The copyright of this article is shared by the author and the blog Park. You are welcome to repost it, but you must specify the source and provide the original article connection clearly on the article page. Otherwise, you will be held legally liable.