Web Data storage

The storage of data is necessarily the thing that any website must go through, we can store the data in different place, the database, the file, the memory, the program itself, the cookie,session can, but as long as need persist the data, then certainly still fall on the disk, Our disks can be formatted into various types of file systems, or they can be directly called by the application to bare disks

One. Storage type

1. Block storage
2. File storage
3. Object storage

Block storage, as the name implies, is a simple block device, which does not include the file system, generally our newly bought disk mounted to the server, called/dev/sdb This is a block storage, such as iSCSI, shared over you find him directly mount can not hang, prompt to format, because it is a bare device, You can format it as any file system to mount it, and that means to give you a hard drive.

File storage is formatted block device, now we generally use the XFS format, formerly EXT4, such as the most commonly used NFS, it is the embodiment of the share we can directly mount the uninstall, but the above file system is the share of the side of the grid good, you can not change, is to give you a directory meaning

Object storage is mostly distributed, it is to solve the block storage is not easy to share file storage is not fast enough to appear, if the object storage provides fuse, then object storage can also be conveniently mounted use, which is also the advantage of glusterfs, otherwise it is based on metadata to access the corresponding data, Ceph follows the RESTFULAPI mode and you need access to the corresponding URI

Two. Storage connection mode

Das Storage (direct-attached storage direct-attached Storage), directly connected to the storage on the server, the first reaction is the disk

NAS Storage (Network access storage network-attached Storage), storage created over the network, SAB,NFS is

SAN Storage (storage Area network), dedicated fiber-optic connection to the server to use storage, more than large applications or database systems, the disadvantage is high cost, very high ....

So with the growing business, the use of Distributed file systems is an over-weighed

Distributed File system Now there are many, recommended that you learn to use Ceph and glusterfs,glusterfs can be used in production, Ceph is now a distributed file system, he integrated block storage, file storage, object storage three ways, And OpenStack can be a good combination, the bottom is an object repository, in essence, is also an object storage

Three. Database-related

Mention of the database, everyone's first reaction is, performance bottlenecks, slow, afraid of accidents, the accident can resign, if there is a DBA is how wonderful things

There are several schemas for storing data within a database:

1. A MySQL to save the world. This is generally the beginning of the enterprise, test environment use, because the accident is also harmless
2. One master one from, high availability. Read and write are the main library, and then sync to from the library, the main library problem manually switch to from the library, the benefits are the simplest, but also guarantee the backup of data, now use a lot of people, the disadvantage is that you need to manually switch, there will be a delay, but also the need for human participation in the delay, If he goes to the bathroom or the phone shuts down, it's bi. Poi, and then the pressure is in the main library, read and write synchronization are required
3. One master one from, read and write separate. Write the main library, synchronize to from the library, read the data read from the library, but must have no delay or read the main library, compared to the last one to reduce the majority of reading pressure; The disadvantage is that developers need to work together, because only he knows what data can delay what is not delayed, but also need to remember more IP, developers do not cooperate that is

4. A master multi-slave, read and write separation, reading pressure is more dispersed, because the general frequency of reading data is much larger than writing; The disadvantage is that each slave node needs to synchronize data from the primary, the more nodes, the longer it takes to complete the synchronization completely

5. One master multi-slave, load balanced. Use LVS for reading from the node to do the cluster, write only the main node, the advantage is that the original code control of the read IP unified to the LVS processing, only need to have two IP one read from the IP one write the main IP; The disadvantage is the same as before, the master still needs to synchronize to each

6. Primary master copy double write. Previously optimized as long as the point is read operations, write and not targeted optimization, the architecture benefits is divided into two full-scale database, each can operate, the task is reduced by half. The disadvantage is that you need to develop mates, there are two additional libraries that need to allocate odd even numbers

7. Primary master copy write. This is the same as a master from, read and write separation is very similar, the difference lies in the process of failure

　　　　

8. Double Master double Slave. In fact, the master copy is added to a library to ensure the redundancy of the data

　　　　

9.MHA. This is the solution provided by MySQL, more data, through a management node to control the large cluster

　　

　　

Access to the database:

1. Direct access, not explained
2. By proxy access, the agent-differentiated operation connects to the corresponding machine, comparing the famous Mycat

　

　

But as time goes by, the volume of our business becomes huge, so distributed databases are the ultimate means

If a website has developed into such a situation, then it is not a waste of life, a huge amount of business constantly crushing us, we need to further classify data, fast query, business separation, service decoupling

Finally, the figure is "large Web site Technology architecture", interested can see, a good book, he will not have any rapid improvement of your technology, but will have a very important impact on your thoughts

Web Data storage