Flatbuffers vs Protocol Buffers

Flatbuffers released last year, recently looked at, and the same is the hand of Google protocol buffers very similar. On the official website, Flatbuffers (FB) mainly for the game development and performance requirements of the application. Compared to protocol buffers (PB), FB does not need parsing, only through the serialized binary buffer can complete data access.

The main features of FB are:

1) data access does not need to be resolved

The data is serialized into a binary buffer, and then the data access is read directly to the buffer, so the read efficiency is high.

2) high memory efficiency and fast speed

-Data access is only in binary buffer after serialization, no additional memory allocations are required.

-Data access speed close to native struct, only one more dereference (based on start address and offset, then value)

3) elasticity

are optional fields that support forward and backward compatibility

4) Small amount of code generated

Rely on a single header file

5) Strong type

Compile-time detection error

6) Easy to use

Schema and JSON text can be parsed

7) Cross-platform

C++/java/go no need for additional reliance

The implementation principle of FB:

The data after serialization via FB is a binary buffer, which contains various objects (Structs,tables,vectors), which are organized by the offset. So the data can be accessed In-place (starting address + offset), just like an array or struct.

The table in FB represents a data structure that corresponds to a vtable that indicates the offset of each field in buffer. Next, when you access a field, you can find the field directly based on the offset. because FB accesses raw buffer directly, it gets dirty data when the data is destroyed. FB ensures that the non-trusted buffer data is correct by verifier.

Below tested PB and FB, do performance comparison, test scenario is very simple, construct n objects, serialize, write files, and then deserialize. The object structure of the test is as follows:

Message Sample {    required int32 id = 1;    Required String name = 2;    Required Int32 IP = 3;    repeated Int32 addr_list = 4;}

The test results are as follows:

Number of records	Flatbuffers	Protocol buffers
10000	0.197/4.4m/0.02	0.166/3.0m/0.08
20000	0.383/8.8m/0.03	0.248/6.4m/0.168
30000	0.534/13m/0.05	0.378/10m/0.26
100000	1.937/44m/0.144	1.314/38m/0.8
200000	3.9/88m/0.276	2.66/67m/1.542
1000000	17.59/439m/1.396	13.355/397m/8.012

X/y/z three data, corresponding to the total time of serialization/file size/reverse sequence time, including the time to read and write files. Can be seen, PB in the time and size of serialization is better than FB, serialization is about 30% faster, the size of the serialized data is about 20% small. and FB because do not need to parse, deserialization abnormally fast.

Although the anti-serialization of FB super fast, but the size of the data is quite large (probably because there is no need to parse, direct access to binary buffer, so the data can not be compressed), the transfer or storage overhead, may be offset by a portion of the deserialization time. Feeling, FB is still very limited in the use of the scene, on the mobile side may be an application scenario.

See GitHub for specific test code

Flatbuffers vs Protocol Buffers