Get you in-depth knowledge of the distributed storage of Web site databases

Author: Finalbsd
Original load: http://www.sanotes.net/html/y2009/358.html

In the Web 2.0 era, the site will often face a rapid increase in traffic, but our application how to meet the user's access requirements, and basically we see the situation is the performance bottleneck is in the database, this does not blame the database, after all, to meet a large number of traffic is really a great pressure on any database, Whether it is a business database Oracle, MS SQL Server, DB2, or open source MySQL, PostgreSQL, is a big challenge, the solution is very simple, is to spread the data on different databases (can be hardware, or can be logical), This article is the main discussion of how the database distributed storage problem.

At present, the main distribution of storage is in accordance with a certain way of slicing, mainly vertical segmentation (portrait) and horizontal segmentation (horizontal) Two ways, of course, there are two ways to achieve more appropriate segmentation granularity.

1. Vertical segmentation (Portrait) data is a database segmentation according to the website business, product segmentation, such as user data, blog post data, photo data, label data, group data and so on each business a separate database or database server.

2. Horizontal slicing (horizontal) data is all data as a large product, but all the flat data according to some key (such as user name) scattered in different databases or database server, the decentralized data access pressure, this is the main discussion in this paper.

This article focuses on the Mysql/postgresql class of open source databases, while the platform is in Linux/freebsd, using Php/perl/ruby/python and other scripting languages, with APACHE/LIGHTTPD and other Web servers Platform below the Web application, do not discuss the storage of static files, such as video, pictures, CSS, JS, that is another topic.

Note: The following will be repeatedly mentioned in a place "node", refers to a database node, may be a physical database server, or a database, the general situation refers to a database server, and is a database server with Master/slave structure, Let's look at one piece to understand the architecture of such a node:

First, hash-based distribution mode

1. Introduction of hashing method

Hash (hash) based distributed storage, mainly rely on the main key and hash algorithm, such as the user-oriented application of the main role is the user, then the key can be a user ID or user name, email address and so on (the value must be passed anywhere in the site), using this unique value as key, By hashing the key, the different user data is scattered over different database nodes (node).

We use simple examples to describe the problem: for example, there is an application, key is a user ID, has 10 database nodes, the simplest hashing algorithm is our user ID number module with all of our nodes, the remainder is the corresponding node machine, algorithm: node = User ID% Total node number, then, The User ID 125 node: 125 10 = 5, then it should be on the node with the name 5. Similarly, a more powerful and reasonable hash algorithm can be constructed to distribute the user to different nodes more evenly.

2. Expansion of the hash distribution storage mode

We know that since a hashing algorithm has been defined, these keys will be scattered over the specified nodes in a step-by-step manner, but what if all the current nodes are not enough to meet the requirements? There is a problem of capacity expansion, the first is to modify the hash algorithm, while the data will be based on the hashing algorithm for migration or modification.

(1) Expansion of the migration mode: After modifying the hashing algorithm, such as 10 nodes before, now increase to 20 nodes, then the hash algorithm is [modulo 20], corresponding to the existence of a previous node is allocated data will be more, but the new join node data less unbalanced state, Then consider using the data in the previous data according to key using the new hash algorithm to calculate the new node, the data migration to the new node, but the cost is relatively large, the instability increases, the advantage is that the data is more uniform, and can make full use of the old and new nodes.

(2) Make full use of the new node: After adding new nodes, the hash algorithm hashes all newly added data to the new node and no longer allocates data to the old node, so there is no cost of migrating the data. The advantage is that only need to modify the hash algorithm, without migrating the data can be simple to increase the node, but in the query data, you must use the old key to use the old hash algorithm, the new added key using the new hash algorithm, or can not find the data node. The disadvantage is obvious, one is the hash algorithm complexity increases, if frequently add new nodes, the algorithm will be very complex, can not be maintained, another aspect is the old node can not make full use of resources, because the old node is simply to retain the old key data, of course, this also has the appropriate solution.

In summary, it is difficult and cumbersome to add nodes to the distribution of data, but there are many suitable occasions, especially for applications that can anticipate the size of future data, but universal Web2.0 sites cannot predict the amount of data.

Second, based on the global node allocation method

1. Introduction to Global node assignment methods

is to record all key information and Database node mapping relationship, save to the global table, when the need to access a node, the first to go to the global table to find, find and then locate the corresponding node. Global tables are generally stored in two ways:

(1) Using the node database itself (Mysql/postgresql) to store node information, remote access, in order to ensure performance, while using the HEAP memory table, or use the Memcached cache mode to cache, speed up the node lookup

(2) using BDB (BerkeleyDB), dbm/gdbm/ndbm, such as local file database, based on the Key=>value hash database, the lookup performance is relatively high, combined with the APC, Memcached and other cache acceleration.

The first storage method is easy to query (including remote query), the disadvantage is that performance is not very good (this is the common problem of all relational databases), the second way is a bit local query speed (especially hash database, time complexity is O (1), relatively fast), the disadvantage is not remote use, And can not be in the middle of multiple machines to share data, there is a consistent data situation.

Let's describe the implementation of the approximate structure: if we have 10 database nodes, a global database is used to store key-to-node mapping information, assuming the global database has a table called Allnode, containing two fields, Key and NodeID, assuming we continue to follow the above case, the user ID is Key , and there is a user ID of 125 user, it corresponds to the node, our query table gets:

Key NodeID
13 2
148 5
22 9
125 6

You can confirm that the user ID is 125 users, where the node is 6, then you can quickly locate the node, the data processing.

Let's take a look at the distributed storage structure diagram:

2. Expansion of global node distribution mode

Global node allocation also has the problem of capacity expansion, but it has long been taken into account of this problem, and so designed to facilitate the expansion, the main way of expansion is two kinds:

(1) Allocate key-to-node mapping expansion by natural increment of nodes

This is the most typical, the simplest, the most economical way to expand the machine resources, roughly, according to each node allocation of the specified amount of data, such as a node to store 100,000 of user data, the first node stores 0-10w user data, the second node stores 10w-20w user data, The third node stores 20w-30w user information, and so on, the user increases to a certain amount of data to increase the node server, while the key is assigned to the newly added node, mapping relationship records to the global table, so that the node can be infinitely increased. The problem is that if the early node user access frequency is relatively low, the latter increase the frequency of the node user access is relatively high, there is a node server load imbalance phenomenon, this is also possible to solve the solution.

(2) Mapping key-to-node expansion through probabilistic algorithms

This approach is based on the node, given the probability of assigning a key to each node, and then assigning a key, according to each node is assigned the probability of allocation, if the average data capacity per node exceeds the specified percentage, such as 50%, then consider adding new nodes, Then the probability of the new node increasing key is greater than the old node.

In general, the probability of the allocation of the node is also recorded in the database, for example, we put all the probability of 100, a total of 10 nodes, then set the probability of each node allocated data is 10, we look at the data table structure:

NodeID Weight
1 10
2 10
3 10

Now join a new node, the newly added node, the probability of being assigned key is greater than the old node, then the new node must be added to the probability of calculation, the formula: 10х+у=100,у>х, concluded: У{10 ... 90},х{1 ... 9},x is the probability of a single old node, the probability of each node of the old node is the same, y is the probability of a new node, according to this formula, calculate the probability of the new node Y range, in accordance with the specific application of the probability formula to calculate.

Iii. Problems in existence

Now let's analyze and solve the problem of the two types of distributed storage, so that we can avoid or fuse some problems and shortcomings when we consider the architecture in practice.

1. Problems with hashing and global allocation methods

(1) The hashing method is not very convenient to expand the expansion, it is necessary to modify the hashing algorithm, and may also need to migrate data, it is the advantage of locating a node from key very fast, O (1) time complexity, and basically do not need to query the database, save response time.

(2) The problem of global distribution is the most obvious single point of failure, the global database down will affect all applications. Another problem is that the query volume is large, the operation of each key node must go through the global database, the pressure is very large, the advantage is easy to expand, increase the node simple.

2. Search and statistics issues from distributed storage

(1) General search or statistics are all data processing, but because after splitting, the data scattered on the different nodes of the machine, can not be global search and statistics. Solution one is to store the primary underlying data in a global table for easy lookup and statistics, but this kind of data should not be too much, part of the core data.

(2) using the site search engine to index and record all the data, such as using Lucene and other open source index system to index all data, easy to search. For statistical operations can be used in the background of non-real-time statistics, can be used to traverse all nodes, but inefficient.

3. Performance optimization Issues

(1) hash algorithm, node probability and allocation, etc. in order to improve performance can be developed using the compilation language, Lib or all forms of PHP extension.

(2) for the use of MySQL, you can use a custom database connection pool, the Apache Module in the form of loading, can be freely customized using a variety of connection methods.

(3) For global data or frequently accessed data, can be used in APC, Memcache, DBM, BDB, shared memory, file system and other ways to cache, reduce the database access pressure.

(4) Adopt the powerful processing mechanism of the data itself, such as MySQL5 table partition or MySQL5 cluster. In addition, it is suggested that the InnoDB table engine should be used as the main storage engine in the actual architecture, and MyISAM as some logs, statistics and so on, whether in security, reliability, speed is guaranteed.

Get you in-depth knowledge of the distributed storage of Web site databases