Key-value Database __ Database

In traditional file system, it is necessary to maintain the hierarchical structure of the directory, to use complex structure such as dentry,inode,directory to save the metadata information, and more and more system consider using Key-value form to save the metadata information in the file system in the face of the requirement of more custom file system. The key-value of using a database to hold these metadata is a good choice, compared to the traditional relational database, the Key-value database is closer to the application requirements, because the database that holds the metadata often does not have the complex relational operation, only needs to provide the efficient read-write performance, Reliability and persistence.

Berkeley DB

The classic Key-value database, C language development, can provide a high read and write performance, support for mass storage applications, the database itself to achieve the backup mechanism, support two backup development mechanism, interface simple. Open source, but no one seems to be defending it now, and is said to be not doing too well in the long run.

SQLite

1. Acid Services
2.0 Configuration – No installation and management configuration required
3. A complete database stored in a single disk file
4. Database files can be shared freely between machines in different byte order
5. Support Database size to 2TB
6. Small enough, roughly 30,000 lines C code, 250K
7. Faster than some popular databases operate in most ordinary databases
8. Simple, Easy API
9. Includes TCL binding, while supporting other languages via wrapper
10. The source code of good annotation, and has more than 90% test coverage
11. Independence: No additional dependencies
Source full open, you can use it for any purpose, including selling it
13. Support for a variety of development languages, C, PHP, Perl, Java, ASP. Net,python

Related article: http://www.sqlite.com.cn/

Redis
Redis is a very new project. Redis is essentially a key-value type of memory database, much like memcached, the entire database load in memory to operate, periodically through asynchronous operation of the database data flush to the hard disk to save. Because it is a pure memory operation, Redis has excellent performance and can handle more than 100,000 read and write operations per second. Redis's outstanding feature is not only performance, Redis's greatest charm is to support the preservation of the list and set set of data structure, but also support the list for various operations, its value can be string,list,sets, or ordered sets. For example, from both ends of the list push and pop data, take the list interval, sorting, and so on, the set support for the sets of the set intersection operation, in addition to the maximum limit of single value is 1GB, unlike memcached can only save 1MB of data, So Redis can be used to implement a number of useful functions, such as using his list to do FIFO two-way linked list, to achieve a lightweight high-performance Message Queuing service, with his set can do high-performance tag system and so on. In addition, Redis can also set the expire time for the deposited key-value, so it can also be used as a function-enhanced version of the memcached.

The main disadvantage of Redis is that the database capacity is limited by physical memory, can not be used for high performance reading and writing of massive data , and it has no native extensible mechanism, no scale (extensible) capability, and relies on client to realize distributed reading and writing. Therefore, the Redis suitable scenarios are mainly confined to the high-performance operations and operations of smaller data volumes . According to Redis's official website test report, 50 concurrent requests, 10w access, write speed is 11x10e4/s, read speed of 8,100 times/s. The site currently using Redis has Github,engine yard.

Introduction to basic data types: http://www.cnblogs.com/xhan/archive/2011/02/02/1948891.html

http://timyang.net/data/redis-misunderstanding/

Interactive command: Http://redis.io/commands

Tokyo Cabinet and Tokoy tyrant
TC and TT developers are Japanese Mikio Hirabayashi, mainly used in Japan's largest SNS website mixi.jp, TC Development of the earliest time, is now a very mature project, is also the Kye-value database field, the largest hot spot, It is now widely used on many websites. TC is a high-performance storage engine, and TT provides multi-threaded high concurrency server, performance is excellent, can handle 450,000 times per second read and write operations. In addition to supporting Key-value storage, TC also supports saving Hashtable data types, so it is much like a simple database table, and also supports conditional queries based on column, paging query and sorting functions, essentially supporting Tanku's basic query functionality So you can simply replace many of the operations of relational databases. Tc/tt in the practical application of Mixi, stored more than 20 million data, while supporting tens of thousands of concurrent connections, is a time-tested project. TC, while ensuring high concurrent read and write performance, has a reliable data persistence mechanism while supporting Hashtable of relational database table structures as well as simple conditions, paging and sorting operations, is a great NoSQL database.

The main disadvantage of TC is that after the amount of data reached billion levels, concurrent write data performance will be significantly reduced, nosql:if only It is easy to mention, they found that in the TC insert 160 million 2-20KB data, write performance began to drop dramatically. It seems that when the amount of data on the billion, TC performance began to decline significantly, from the TC author's own Mixi data, at least thousands the amount of data has not encountered such a significant write performance bottlenecks. Flare, which is an improvement on TC and TT, mainly supports scalability.

Mongodb
To meet the mass storage requirements, MONGO mainly solve the problem of the access efficiency of massive data, according to official documents, when the amount of data reaches 50GB or more, MONGO database access speed is 10 times times more than MySQL. MONGO's concurrent read and write efficiency is not particularly good, according to official performance tests that can handle 5,000-1.5 read-write requests per second

Couchdb
Support for mass storage, COUCHDB only provides an interface based on HTTP rest, so couchdb is very bad in terms of concurrent read and write performance alone.

Cassandra

Distributed computing-oriented databases that meet high scalability and availability are seen as a big table of open source Google. Facebook,twitter and digg.com are using Cassandra. The main characteristic of Cassandra is that it is not a database, but a distributed network service composed of a bunch of database nodes, and a write operation to Cassandra will be replicated to other nodes, and the Cassandra read operation will also be routed to a node to read. For a Cassandra cluster, extended performance is a relatively simple thing to do, just add nodes to the cluster. See an article that says Facebook's Cassandra Cluster has a database cluster of more than 100 servers. Cassandra with a single node, the concurrent read and write performance of its nodes is not particularly good, there are articles that the evaluation down Cassandra about 10,000 times per second read and write requests, its concurrency performance depends on the total number of nodes of the system, routing efficiency, not just a single node of the concurrent load capacity.

Voldemort

Similar to Cassandra, it is a distributed computing-oriented database that provides high scalability and availability. Voldemort official Voldemort Read and write performance is also very good, more than 15,000 times per second read and write.

Leveldb

Google developed the database, LEVELDB is an embedded key-value database. Its key and association values can be arbitrary byte arrays, and sorted by key values, the sorting mechanism can be overloaded. The data storage mechanism is simple enough to support only the Put,get and delete commands, followed by forward and back iterative traversal. The data automatically uses snappy compression, a compressed library that Google uses for Bigtable,mapreduce and RPC, and announces open source. LEVELDB also has some limitations: it does not support SQL queries and indexes, supports multithreaded single process access, and can be used for embedded devices. Leveldb optimizes the bulk write operation. It caches more than one modification request in memory, and writes to disk after the threshold is accumulated to the configuration file's preset settings. for sequential and Random writes, as well as sequential read operations, it is very good performance , according to Google's performance benchmark test, it can score in some test projects leading SQLite two orders of magnitude. SQLite is slightly better than leveldb in random read operations and is twice times faster than leveldb when writing larger data. Leveldb also performed better than Kyoto cabinet, Kyoto cabinet is also a key-value database, but Google did not compare it to all test projects, as SQLite did. Similarly, Riak conducted some tests comparing LEVELDB and InnoDB, in some test projects, Google's leveldb is better than InnoDB or can achieve the same performance. Leveldb is written in C + +, and some external dependency libraries have been successfully ported to Windows, Mac OS X, Android, and various Unix. In practical applications, LEVELDB has been used in some of the experimental versions of Chrome as an implementation of the Indexdb API. The Riak is used for node-level storage.

Nosql

Regular relational databases support only thousands of or tens of thousands of accesses per second, and memory databases can reach tens of thousands of to Brovants, enabling high concurrency, mass storage, high scalability, and high availability

Resources:

http://blog.csdn.net/21aspnet/article/details/6614013

Http://blog.evanweaver.com/articles/2009/07/06/up-and-running-with-cassandra

Http://hb.qq.com/a/20110823/000019.htm

Http://www.docin.com/p-115838909.html

http://www.iteye.com/topic/617156

http://blog.s135.com/dtcc/