A kind of automatic reflection message type of Google PROTOBUF network transmission scheme

The question to be addressed in this article is how to automatically create a specific Protobuf message object and deserialize it after receiving the PROTOBUF data. "Automatic" means that when a new Protobuf message type is added to a program, this part of the code does not need to be modified and does not need to register the type of messages itself. In fact, Google PROTOBUF itself has a strong reflection (reflection) function, can be based on type name to create a specific type of message object, we directly use.

This article assumes that readers understand what Google Protocol buffers is, and this is not a protobuf introductory tutorial.

In this paper, the C + + language For example, other language estimates have a similar solution, welcome to Add.

The sample code for this article is: https://github.com/chenshuo/recipes/tree/master/protobuf

Two problems of using PROTOBUF in network programming

Google Protocol Buffers (PROTOBUF) is a very good library that defines a compact, extensible binary message format, especially for network data transfer. It provides binding for many languages, greatly facilitates the development of distributed programs, so that the system is no longer limited to the use of a language to write.

The use of PROTOBUF in network programming requires two issues to be addressed:

The length, the PROTOBUF package data does not have the length information or the non-terminal, needs to make the correct segmentation by the application itself when occurs and receives;

Type, PROTOBUF packaged data does not have its own type information, it needs to be transmitted by the sender to the receiver, the receiver creates a specific Protobuf message object, and then deserializes it.

The first is a good solution, usually by adding a fixed length header to each message, such as the one I implemented in the Muduo network programming Example: Boost.asio's chat server Lengthheadercodec, code see http:// Code.google.com/p/muduo/source/browse/trunk/examples/asio/chat/codec.h

The second problem is actually very well solved, PROTOBUF has built support for it. But the strange thing is, from the simple search on the internet, I found a lot of cottage practice.

Cottage practices

The following are included with int length and type information before PROTOBUF data plus header,header. There are two main types of cottage practices for type information:

put int typeid in header, receiver use Switch-case to select corresponding message type and processing function;

With string TypeName in the header, the receiver uses the look-up table to select the corresponding message type and handler function.

Both of these approaches are problematic.

The first approach requires that the uniqueness of the typeid be maintained, which corresponds to the PROTOBUF message type one by one. If the use of PROTOBUF message is not wide, such as the receiver and the sender are their own maintenance of the program, then the uniqueness of typeid is not difficult to ensure that the use of version management tools. If the use of PROTOBUF message is large, such as the whole company is used, and distributed programs developed by different departments may communicate with each other, then need a global organization within the company to allocate typeID, each add new message type to register, more trouble.

The second approach is a little bit better. The uniqueness of the typeName is better, because the package name (that is, the fully qualified type name of the message) can be added to each department in advance namespace without conflict and repetition. But every time you add a message type, you have to manually modify the initialization code for the look-up table, which is more cumbersome.

In fact, no need to reinvent the wheel itself, PROTOBUF itself has brought the solution.

Automatically create a Message object based on type name reflection

Google Protobuf itself has a strong reflection (reflection) feature that creates a specific type of message object based on type name. But strangely, the official tutorial does not explicitly mention this usage, I guess many people do not know this usage, so I think it is worth writing this blog to talk about.

The following is the Protobuf class diagram drawn by Chen Yu.