Converged Rocksdb, Pregel, Fault-tolerent Foxx & satellite collections How to improve database performance by 35%?

Abstract: After months of research and development, the open source multi-model database Arangodb finally released its 3.2 full version, which eliminates two major hurdles, adds a long-awaited feature, and integrates an interesting feature. In addition, the official team said the new version increased the performance of Arangodb by an average of 35%. Also, the memory footprint is reduced compared to version 3.1. There have also been significant improvements in cluster management. Specific as follows:

rocksdb Storage Engine:

Integration with Facebook's ROCKSDB storage engine makes it the first pluggable storage engine in the architecture, and users can now use as much data as the disk. Plus rocksdb better locking behavior (document-level locks), dense-write applications will significantly improve performance. There is no memory limit, and only document-level locking, which eliminates two major hurdles for many users. If you select Rocksdb as the storage engine, all content, including indexes, is persisted on disk, which greatly reduces the time to start. For more information, see "Comparing new Rocksdb and Mmfiles engines" to test the new engine for the operating system and use cases.

Pregel Distributed Graphics Processing:

Distributed graph processing is a missing feature in Arangodb's graphical toolbox. However, Arangodb satisfies this requirement by implementing the Pregel computational model.

Through PageRank, community detection, vertex center metrics, and further algorithms, ARANGODB can now be used to gain advanced insight into the features of graphic shadowing. For example, you can use the graphics processing feature to detect a community. You can then use the results to effectively distribute the data to the cluster, allowing the use of smartgraph to reach its full potential. We believe that by integrating distributed graph processing, users will now have one of the most complete set of graphical toolsets in a single database.

Use the Pregal community testing tutorials to test new pregal combinations and further improve advanced graph skills with new tutorials on using smartgraphs in Arangodb.

Foxx fault-tolerant mechanism:

The Foxx service with fault tolerant mechanism can expand the database according to your requirements in cluster mode.

Many developers prefer to use Arangodb's Foxx JavaScript framework to implement data-centric microservices. Define your own highly configurable HTTP routes, and full access to the C + + level of the Arangodb kernel can be easily implemented. In version 3.2, Arangodb's Foxx team completely rewritten the management of internal parts to support Foxx services with fault-tolerant mechanisms. This ensures that the multi-coordinator cluster will always keep its services synchronized, even if all existing coordinators are unavailable, and the new coordinator will be fully initialized.

Test the new fault-tolerant Foxx or learning Foxx yourself by following the new Foxx tutorial.

Powerful graphical visualization capabilities:

Graphical data can be easily processed using ARANGODB 3.2.

You can export data by using the Open-source option Arangoexport and then import it into Cytoscape (see Tutorial).

Or you can insert the new Keylines 3.5 via Foxx and install the on-demand connection. With this option, you will always show the latest data in keylines neatly, without any export/import hassles. Just follow this tutorial to get started with Arangodb and Keylines.

Read-Only users:

In order to strengthen the basic user management in Arangodb, it added the read-only user function. The permissions of these users can be defined at the database and collection level. At the database level, users can gain administrator privileges, read access, or deny access. At the collection level, in the database, the user can be given read/write, read-only, or denied access. If the user does not have access to the database or collection, the user will not display the databases and collections. Refer to the tutorials for new user management.

Geo-Query Geo Index Cursor:

Geo-discovery is becoming more and more important to our community. With Geo_cursor, you can now arrange documents by distance to a point in space (see Tutorial). This makes the query easier, like "where is the vegetarian restaurant half a mile radius around Times Square?" "We plan to increase support for other geospatial features (such as polygons) in the next iteration."

Satellite Collection Satellite Collections:

The Satellite collection (satellite collections) is an exciting outcome of this collaboration. It is designed to enable faster connection operations when using a shard dataset. To avoid expensive network hops during connection processing between machines, only one solution can be found to enable local connectivity.

With satellite collections, you can define collections to spread to the cluster, and set the collection to replicate to each computer. The ARANGODB query optimizer knows where each shard is and sends a request to the relevant dbservers, and then executes the query locally. Dbservers then sends some of the results back to the coordinator, which summarizes the final results. This way, you can avoid network jumps during the connection operation of the Shard collection, which increases query performance and reduces network traffic. This can be more easily understood by an example. In the following pattern, the set C is divided into multiple machines, while the smaller satellites (i.e., S1-S5) are copied to each machine and run around the debris track of C.

The use cases for satellite collections are abundant. In this more in-depth blog post, we used an IoT case. Individualized patient treatment based on genome sequencing analysis is another good example of how effective joint operations involving a large number of datasets can help improve patient care and save on infrastructure costs.

Leisure Data encryption:

With Rocksdb, you can encrypt data stored on disk using a highly secure AES algorithm. Even if someone steals one of your disks, they won't be able to access the data. With the upgrade, Arangodb a significant step towards HIPAA compliance.

Enhanced authentication through LDAP:

Typically, users are defined and managed through ARANGODB. With LDAP, you can use an external server to manage users. Arangodb implements a common pattern that can be extended. If you have special requirements that do not conform to this pattern, you can contact Arangodb.

3.2 Official version download link: https://docs.arangodb.com/3.2 ...

Converged Rocksdb, Pregel, Fault-tolerent Foxx & satellite collections How to improve database performance by 35%?