"Android" Flatbuffer with JSON

You might ask, why use new tools if we already have very standard JSON and conversion libraries such as Gson and Jackson?

Try Flatbuffers First, and then you'll find it much faster than JSON.

What is Flatbuffers?

Flatbuffers is an efficient cross-platform serialization class library that can be used in C + +, C #, C, Go, Java, JavaScript, PHP, and Python. It was developed by Google for use in game development and other performance-focused applications.

Why use Flatbuffers?

• no parsing/unpacking is required to access the serialized data -flatbuffers differs from other libraries in that it uses binary buffer files to represent hierarchical data so that they can be accessed directly without parsing and unpacking, while supporting data structure evolution (forward, backward compatibility).

• memory is fast and efficient -access to the data only requires access to the in-memory buffers. It does not require extra memory allocations (at least in C + +, which may change in other languages). Flatbuffers is also suitable for use with mmap or data streams and requires only a portion of the buffer to be stored in memory. Access speed is close to the original structure access, with only a bit of delay (a virtual function table vtable), is to allow format upgrades and optional fields. Flatbuffers is suitable for projects that spend a lot of time and space (memory allocation) to access and build serialized data, such as games and other performance-sensitive applications. You can refer to the benchmark here.

• flexible -because of the optional fields, you have strong upgrade and fallback compatibility (especially important for historic games, without having to upgrade all the data for each version), and are also free to choose what data to store and design the data structure.

• The Lightweight code footprint -flatbuffers requires only a very small amount of generated code, and a tiny header file that represents a minimal dependency, which is easy to integrate. The above reference page can be seen in detail.

• strongly typed -compile times are wrong without having to write the repeated error-prone run-time checks yourself. It can automatically generate useful code.

• Ease of Use -generated C + + code allows for thin access and build code. There are also optional methods for implementing chart parsing, JSON-like run-time string presentation, and so on. (the latter is faster and more memory efficient than JSON parsing libraries)

• The code is cross-platform and does not rely on-c++ code to run on any modern Gcc/clang and VS2010. There are also build files for testing and example (. mk Files on Android, other platforms are cmake files).

Who uses flatbuffers?

• Bobbleapp, the first map app in India. We have significantly enhanced the performance of our app after using Flatbuffers in Bobbleapp.

• Cocos2d-x, the first open source mobile game engine, uses Flatbuffers to serialize all the game data.

• Facebook uses flatbuffers to communicate on the client side of the Android app. They wrote an article describing how flatbuffers accelerates the loading of content.

• Google's Fun propulsion Labs uses flatbuffers extensively in all of their libraries and games.

How much does app performance improve?

• parsing Speed parsing a 20KB JSON stream (which is almost Bobbleapp's return size) requires 35MS, which exceeds the UI refresh interval, which is 16.6ms. If we parse the JSON, we will slide the cache from the disk to cause the drop frame (visual lag).

• The parser initializes a JSON parser that needs to build the field mapping before parsing, which can take 100ms to 200ms, and obviously slow down the app startup time.

• Garbage Collection creates a lot of small objects when parsing json, and in our experiment, when parsing a JSON stream of 20kb, we allocate about 100KB of instantaneous storage, which is very stressful for Java memory recycling.

Flatbuffers vs JSON

I tried to parse 4MB JSON files using Flatbuffers and JSON.

Flatbuffers spent a 1-5ms,json of about 2000ms. There was no GC in the Android app while using Flatbuffers, and a lot of GC occurred when using JSON. The UI is fully stuck when using JSON, so it can only be parsed in the background thread when it is used.

GitHub Source Address: Https://github.com/amitshekhariitbhu/FlatBuffer

Transferred from Oschina Open source China Community: Http://www.oschina.net/news/75092/android-flatbuffers-json

You might ask, why use new tools if we already have very standard JSON and conversion libraries such as Gson and Jackson?

Try Flatbuffers First, and then you'll find it much faster than JSON.

What is Flatbuffers?

Flatbuffers is an efficient cross-platform serialization class library that can be used in C + +, C #, C, Go, Java, JavaScript, PHP, and Python. It was developed by Google for use in game development and other performance-focused applications.

Why use Flatbuffers?

• no parsing/unpacking is required to access the serialized data -flatbuffers differs from other libraries in that it uses binary buffer files to represent hierarchical data so that they can be accessed directly without parsing and unpacking, while supporting data structure evolution (forward, backward compatibility).

• memory is fast and efficient -access to the data only requires access to the in-memory buffers. It does not require extra memory allocations (at least in C + +, which may change in other languages). Flatbuffers is also suitable for use with mmap or data streams and requires only a portion of the buffer to be stored in memory. Access speed is close to the original structure access, with only a bit of delay (a virtual function table vtable), is to allow format upgrades and optional fields. Flatbuffers is suitable for projects that spend a lot of time and space (memory allocation) to access and build serialized data, such as games and other performance-sensitive applications. You can refer to the benchmark here.

• flexible -because of the optional fields, you have strong upgrade and fallback compatibility (especially important for historic games, without having to upgrade all the data for each version), and are also free to choose what data to store and design the data structure.

• The Lightweight code footprint -flatbuffers requires only a very small amount of generated code, and a tiny header file that represents a minimal dependency, which is easy to integrate. The above reference page can be seen in detail.

• strongly typed -compile times are wrong without having to write the repeated error-prone run-time checks yourself. It can automatically generate useful code.

• Ease of Use -generated C + + code allows for thin access and build code. There are also optional methods for implementing chart parsing, JSON-like run-time string presentation, and so on. (the latter is faster and more memory efficient than JSON parsing libraries)

• The code is cross-platform and does not rely on-c++ code to run on any modern Gcc/clang and VS2010. There are also build files for testing and example (. mk Files on Android, other platforms are cmake files).

Who uses flatbuffers?

• Bobbleapp, the first map app in India. We have significantly enhanced the performance of our app after using Flatbuffers in Bobbleapp.

• Cocos2d-x, the first open source mobile game engine, uses Flatbuffers to serialize all the game data.

• Facebook uses flatbuffers to communicate on the client side of the Android app. They wrote an article describing how flatbuffers accelerates the loading of content.

• Google's Fun propulsion Labs uses flatbuffers extensively in all of their libraries and games.

How much does app performance improve?

• parsing Speed parsing a 20KB JSON stream (which is almost Bobbleapp's return size) requires 35MS, which exceeds the UI refresh interval, which is 16.6ms. If we parse the JSON, we will slide the cache from the disk to cause the drop frame (visual lag).

• The parser initializes a JSON parser that needs to build the field mapping before parsing, which can take 100ms to 200ms, and obviously slow down the app startup time.

• Garbage Collection creates a lot of small objects when parsing json, and in our experiment, when parsing a JSON stream of 20kb, we allocate about 100KB of instantaneous storage, which is very stressful for Java memory recycling.

Flatbuffers vs JSON

I tried to parse 4MB JSON files using Flatbuffers and JSON.

Flatbuffers spent a 1-5ms,json of about 2000ms. There was no GC in the Android app while using Flatbuffers, and a lot of GC occurred when using JSON. The UI is fully stuck when using JSON, so it can only be parsed in the background thread when it is used.

"Android" Flatbuffer and JSON