Lightcloud: Distributed key-value database

Plurk.com on February 28 this year open source Lightcloud this distributed key-value database, according to the official website information, has the following characteristics

Based on Tokyo Tyrant (this project appears to be another network interface for a database system called Tokyo Cabinet). Tokyo Tyrant is one of the fastest key-value databases and has been developed for several years and is applied in n sites.

Very good performance (can be compared with memcached)

Millions data can be stored on a small number of servers

Simple to extend by adding nodes

Nodes can be backed up by Master-master replication. Easy to add automatic error recovery and load Balancing features

Scaling using the Lua language

No shutdown required to back up and restore data

The Lightcloud Manager tool makes it easy to manage nodes, back up nodes, and view status

Very small (client approximately 500 lines, manager about 400 lines)

Pure python development, but easy to migrate to other languages

And the difference between memcached and MySQL.

Memcached is a caching server, while the data stored in Lightcloud is persistent (in the end, the database).

MySQL and other relational database server storage key-value pairs are inefficient (certainly not lightcloud this dedicated high).

and memcached Performance Comparison

Make 10,000 sets and get for Lightcloud and memcached

Elapsed　for　10000　gets:　1.74538516998　seconds　 [memcache]
Elapsed　for　10000　gets:　3.57339096069　seconds　[lightcloud]

Elapsed　for　10000　sets:　1.88236999512　seconds　[memcache]
Elapsed　for　10000　sets:　9.23674893379　seconds　[lightcloud]

It is worth noting that memcached is fully operational in memory, while Lightcloud is read and write to the hard drive.

The data from Tokyo Tyrant's official website was even more alarming (Lightcloud increased network and Python language execution overhead on Tokyo tyrant)

1 million GETS in < 0.5 seconds

1 million SETS in < 0.5 seconds

Using Lua for Scaling

The LUA extension appears to be provided by Tokyo Tyrant

Here is a simple extension that expands out a INCR command

function　incr(key,　value)
　　value　=　tonumber(value)
　　if　not　value　then
　　　return　nil
　　end
　　local　old　=　tonumber(_get(key))
　　if　old　then
　　　value　=　value　+　old
　　end
　　if　not　_put(key,　value)　then
　　　return　nil
　　end
　　return　value
end