Concurrent key-value repositories up to 160 million operations per second FASTER born

FASTER

Over the past decade, data-intensive applications and services in the cloud have grown dramatically. data is created on a variety of edge facilities (for example, devices, browsers, and servers) and is processed by cloud applications to gain data value or make decisions. applications and services can process the collected data, or they can monitor the data in real time. These applications are typically update-intensive and involve a large number of states, beyond the processing power appropriate for the primary storage. However, they show significant time locality in their access patterns (time-localized interpretation of Wikipedia =>en.wikipedia.org/wiki/locality_of_reference). A new key-value store for point operations. faster combines a cache-optimized concurrent hash index with a "mixed log": A record store that spans both main memory and stored concurrent log structures, and supports fast in-place updates of in-memory "hot swap". faster extends the standard key-value storage interface to handle read-modify-write, blind-update, and Crdt-based updates. experiments show that the faster achieves higher throughput on a single machine (up to 160 million operations per second) compared to the currently widely deployed repositories, and when the workload size fits the memory size, His performance will be much better than the performance of pure memory data structures.

background

The Microsoft Research team announced a new Key-value repository, called Faster, in Sigmod in June 2018. the faster supports fast and frequent data lookups. It also helps address the issue of updating large amounts of state information in today's cloud-era applications.

Let's use the Internet of things as a scenario . billions of devices report and update status, such as performance counters for each device. This causes the application to not take full advantage of resources such as repositories and networks on the machine. He can help solve this problem faster because it uses the time position in these applications to control the amount of system memory.

according to Microsoft, "faster is a single-node shared memory key-value repository." Key-value storage is a NoSQL database that uses a simple key/value method for data storage. It contains two important innovations:

• Cache-friendly, concurrent, and Latch-free (latch-release, interpreted Link: www.dbtan.com/2010/05/latch-free.html) hash index. It maintains a logical pointer recorded in the log. the faster hash index is a hash buckets array that points to a cache row size, each with a 8-byte entry to hold the hash label. It also contains a logical pointer to the store record:
  
• a new concurrency and hybrid log recorder allocator . This helps to support indexes that include fast storage (such as cloud storage and SSD) and primary storage.

What makes faster different?

traditional key-value storage uses the log structure to record data. However, faster is different because it has a hybrid log that combines the log structure with read replica updates (for external storage) and in-place updates (for higher-performance memory). Therefore, the head of the hybrid log that is located in memory uses the read-copy-update, while the hybrid log trailer in the main storage uses an in-place update. There is a read-only area in memory that is located between the two regions. It provides another opportunity for the core record to be copied back to the trailer. This captures the temporary location of the update and allows automatic collection of hot records in memory.

As a result, faster can even go beyond pure memory data structures such as Intel TBB hash map. Microsoft says its performance is much better than today's popular cache system for key-value storage such as Rocksdb and Redis.

In addition to this, faster provides support for failback because it includes a recovery strategy that helps restore the system to a recent consistent state at a low cost. This differs from the recovery mechanism in traditional database systems because it does not involve blocking or creating a separate "pre-written log".

For more information, please see the Official research report.

FASTER Project GitHub Address: Github.com/microsoft/faster

Value:

Some people will ask, with this faster, since he speed so fast is not able to replace Redis or MongoDB, the answer is currently not, because faster is the kernel level, and Google's leveldb almost level, faster speed is very fast undoubtedly, But the lack of support for a more complex business scenario, such as clustering, data consistency, and so on, is compelling enough to attract more people into the faster to develop products like Redis and MongoDB, which will be more powerful. Maybe Redis later on the bottom of the faster rewrite.

Translation reference:

hub.packtpub.com/say-hello-to-faster-a-new-key-value-store-for-large-state-management-by-microsoft/

www.microsoft.com/en-us/research/publication/faster-concurrent-key-value-store-place-updates/