Protobuf Learning Notes __protobuf

What is Protobuf
A data storage and transmission format, used for structured data, has the advantages of compact structure, small footprint, fast structure, good expansibility, and platform and language independent.
Use steps
1. Define the proto file (structure description) 2. Generate the implementation of the corresponding language (Java, etc.) 3. The data stream is generated into the proto format data Flow 4. Receiver-side Analytic data Example: http://blog.csdn.net/wanyanxgf/article/details/8817925
proto File Definition
Option java_package: Package option java_outer_classname for generating Java files: generated java filename message: Defines a message, equivalent to Java class Enum: Defines an enumeration that is equivalent to the Java enumeration required: Indicates that this field must exist, that is, the field must exist within a structured stream, or optional: This field represents an optional, can or may not be repeated: represents this field can appear 0~ n times, you can think of a list default in Java: Represents the default value that is not reflected in the stream, that is, the default value after the resolution of the receiver is the same as defined in the class on the receiving end, and the number following the unrelated field of the sender: The number representation of the field, which represents the field in the stream, Cannot be changed after definition (because of the storage-mode relationship, try to use 15 because)
commonly-asked segment types (corresponding to Java):Double==>double float==>float bool==>bool string==>string bytes==>byte[] int32/uint32/sint32==> Int (UInt32 is unsigned, because the storage mode of int32 result in the processing of negative numbers is more resources, so the introduction of Sint32, good negative treatment) Int64/uint64/sint64==>long (IBID.) Fixed*==>int, Long (with fixed length to store, in a larger number when the use of dynamic processing more resources than the use of better)

Option Java_package = "Com.meituan.service.mobile.protobuf.hotel";
Option Java_outer_classname = "Hotelcommentproto";

Message hotelcommentlist{
        repeated hotelcomment comments=1;
}


Message hotelcomment{
        required Int64 id=1;
        Optional int64 poi_id=2;
        Optional int64 did=3;
        Optional int64 user_id=4;
        Optional string user_name=5;
        Optional Int32 score=6 [default=5];
        Optional string score_text=7;
        Optional string comment=8;
        Optional int64 feedback_time=9;
        Optional Hoteltype ht=10;
}
 Enum Hoteltype {
    NORMAL = 0;
    EXP = 1;
  }

Encoding
The first explanation for the next varints is a way to express integers by one to multiple byte 1. Converts a number to 2, from low to 7 bits per group, such as: 150=000001 0010110 2. Take the bit group from low to high, and if there are bit groups, add 1 in the first place, or 0 if not. "1" 000001 "0" 0010110

Example One

Define
required Int32 a = 1;
assigning value
a=150;

encode results: 08 96 01

Binary is: 00001000 10010110 0000001

Analysis: According to the varints rule, the above results are 2 groups of byte

The byte group 1:00001000 represents the definition of the field, and the rule is: (Field_number << 3) | Wire_type by this rule: the first bit "0" represents no subsequent byte group, and the three byte "000" represents wire type 0 (wire type), and the middle number representing the field represents "0001" for 1 Note: This description can explain the number of fields described above to use as much as 15, if more than 15, then the definition of the field can not be expressed in a byte, need multiple byte.

byte group 2:10010110 0000001 According to the first bit bit "1", "0" of the rule two byte to represent the back and the back. Because the number is low before the generation, the restore needs to put the byte back, "000001 "0010110", 10010110=128+16+4+2=150

The different wire types represent different structured ways:
Type=0: That is, using the varints rule type=1,5 described above: The data length has been explicitly specified and stored in the previous way type=2: by string example

Define
Required String b = 2;
Assign Value
b=testing

The result is: 67 6e

Byte group 1:12=00010010 represents type=2, field number =2

byte group 2:07=00000111 indicates that 7 bytes are followed by the value of this field.

Encoded value of byte group 3:74 6e -utf-8 value

Storage of repeated

Optional Int64 user_id=4;

If this field has more than one value, there will be multiple 4:value1,4:value2 in the final stream (not necessarily the same), which are grouped together at parse time

Optional Int64 user_id=4[packed=true];

Storage is similar to string, such as: 8e 02, where "22" is the field definition, "06" is the subsequent data length, followed by the data.
Impact of Changes
1. The number of all fields indicates that 2 is not to be changed. The new field is required and cannot be resolved if the producer and the parser versions are inconsistent. 3. The new fields are optional or repeated. If the production party uses the new version, the old version of the parser discards the value of the new field. If the production side uses the old version, the new version of the parser has no relevant data to fill 4. Non-required fields are deleted, affecting the same 3 (not recommended) 5. Modify field types, Int32, UInt32, Int64, UInt64, BOOL can resolve each other, but data truncation may occur 6. To modify the field type, string, bytes can resolve each other (bytes are utf-8 resolvable) 7. Modifies the field modifier, which is repeated by the producer, and the parser is optional, which can be resolved. If it is a base type or string, the last value is used when parsing (each of which is the previous one), and if the other type (message, etc.) executes the merge, that is, multiple message property merges, whichever is the later.
Java code generation rules
Generates a *orbuider interface for each message and a message's *orbuider interface is a class implementation of the message property's acquisition method message. The Orbuilder interface inherits the Generatedmessage,message class and cannot be modified directly. Must be built using its internal builder. The Write method that contains the message class in the internal builder