Analysis of Google Protocol Buffers

Analysis of Google Protocol Buffers

This article mainly tends to introduce how to use Google Protocol Buffer technology to compress and parse your data files, for more detailed information, please refer to Google open Developer Web documentation, address: http://code.google.com/apis/protocolbuffers/docs/overview.html.

Application and Analysis of Protocol Buffers

The sample code SerialProtocolBuffer in this article:

------------------------------------------ Split line ------------------------------------------

FTP address: ftp://ftp1.bkjia.com

Username: ftp1.bkjia.com

Password: www.bkjia.com

Analysis of LinuxIDC.com \ 2014 \ Google Protocol Buffers in September

For the download method, see

------------------------------------------ Split line ------------------------------------------

I. Brief Introduction

Of course, before continuing this article, readers still need to have some basic knowledge about Google Protocol Buffers. Protocol buffers is a technology used to serialize structured data. It supports multiple languages such as C ++, Java, and Python. It can be used to persist data or serialize data transmitted over the network. Compared with other XML technologies, this technology is more space-saving (stored in binary stream), faster and more flexible.

Generally, writing a protocol buffers application involves the following three steps:

1. Define a message format file. It is best to use proto as the suffix name

2. Use the protocol buffers compiler provided by Google to generate code files, which are generally. h and. cc files. It mainly describes the Message format in a specific language.

3. Use the API provided by the protocol buffers library to write applications

Ii. Define the Proto File

The proto file is the prototype definition file of the Message Protocol. In this file, we can use descriptive language to define the data format required by our program. First, we can use the example of a phone book provided in the Google online document to understand it, but a slight change is added.

Message Person {
Required string name = 1;
Required int32 id = 2;
Optional string email = 3;

Enum PhoneType {
MOBILE = 0;
HOME = 1;
WORK = 2;
}

Message PhoneNumber {
Required string number = 1;
Optional PhoneType type = 2 [default = HOME];
}

Repeated PhoneNumber phone = 4;

Required bytes unsure = 5; // Add byte array here
}

Message AddressBook {
Repeated Person person = 1;
}

 

As you can see, the message format definition is very simple. For each field, there is a modifier (required/repeated/optional), field type (bool/string/bytes/int32, etc) and the field Tag.

The three modifiers can be clearly understood in terms of meaning,

1) for the required field, the initial value must be provided; otherwise, the field is not initialized. Compilation in the buffer library in Debug mode may fail during serialization, And the parsing of this field will always fail during deserialization. Therefore, initialize fields whose modifier is required during serialization.

2) If optional is not initialized, a default value is assigned to this field. Of course, you can also specify the default value, as shown in the PhoneType field type in the preceding proto definition.

3) for repeated fields, this field can be repeated. This addressbook example provided by google has a good application scenario for this modifier, that is, each person may have multiple phone numbers. In the advanced language, we can implement this through arrays, while in the proto definition file, we can use repeated for modification to achieve the same purpose. Of course, 0 occurrences are included.

The field Tag indicates the location where the field is stored in the binary stream. This is required, and the Tag value of the same field during serialization and deserialization must match, otherwise, deserialization may cause unexpected problems.

3. Compile the proto file to generate data definition code for specific language data

After defining the proto file, you can use the file as the input file of the protocol buffers compiler and compile the data definition code file in a specific language. This article is mainly for the C ++ language, so after using the compiler, the generated code files are. h and. cc. For C ++, Java and Python have their own compiler: http://code.google.com/p/protobuf/downloads/list

After downloading the corresponding compiler binary file, you can use the following command to complete the compilation process:

Protoc.exe-proto_path = SRC -- cpp_out = dst src/addressbook. proto

-- Proto_path indicates the directory where the proto file is located, -- cpp_out indicates the directory where the generated code file is to be placed, and the last parameter indicates the path of the proto file. As shown in the above command, you can see that the addressbook under the SRC directory. after proto is compiled and put in the DST directory, addressbook should be generated. pb. h and addressbook. pb. cc file (/Files/royenhome/addressbook.rar ).

By viewing the header file, we can find that the following functions are generated for each field, taking number as an example:

// Required string number = 1;
Inline bool has_number () const;
Inline void clear_number ();
Inline const: std: string & number () const;
Inline void set_number (const: std: string & value );
Inline void set_number (const char * value );
Inline: std: string * mutable_number ();

It can be seen that an has function (has_number), clear clearing function (clear_number), set function (set_number), get function (number and mutable_number) will be generated for each field ). The difference between the two functions in the get function is explained here. For the get function with the prototype const std: string & number () const, the return value is a constant field, the value cannot be modified. However, in some cases, it is necessary to modify the field. Therefore, a mutable get function is provided to obtain the pointer of the field variable to change its value.

The function generated by the field modifier repeated is slightly different. For example, if the phone field is used, the compiler generates the following code for it:

// Repeated. Person. PhoneNumber phone = 4;
Inline int phone_size () const;
Inline void clear_phone ();
Inline const: google: protobuf: RepeatedPtrField <: Person_PhoneNumber> & phone () const;
Inline: google: protobuf: RepeatedPtrField <: Person_PhoneNumber> * mutable_phone ();
Inline const: Person_PhoneNumber & phone (int index) const;
Inline: Person_PhoneNumber * mutable_phone (int index );
Inline: Person_PhoneNumber * add_phone ();

It can be seen that the set function has become the add function, which is very understandable. As mentioned above, the implementation of the repeated field in the advanced language may be an array or a dynamic array, so of course you can add a new field value by adding it. The get function also changes a lot, so you don't need to talk about it.

Well, this article mainly gives a brief introduction to understanding protocol buffer. Of course, you can refer to the official documentation for more details. The next article will introduce how to use protocol buffers to complete serialization and deserialization of data.

For more details, please continue to read the highlights on the next page:

1 2 3 4 Next Page