Database technology in past and present life

Recently read some articles about the development of database technology, basically revolve around the discussion of SQL and NoSQL to unfold. Remember when the NoSQL movement just started, "Dour No to SQL" slogan, of course, the slogan did not become a reality, people eventually accepted the not-only of SQL. But really to give NoSQL the next definition, it is not easy, people used to MongoDB, Redis such products as a general definition of NoSQL, can be attributed to the "four major categories." But for me, the last interview with a large data architect was even more pertinent, with no use of SQL being nosql, even in traditional relational databases.

In other words, you can think of them as a pair of good base friends, you have me, I have you. It is very apt to describe the words described in an article by the software engineer Martin Fowler: Polyglot persistence. It is translated into Chinese can be called mixed persistence or multi-language persistence, so in the future of the database technology field, it will be very likely to break the so-called relationship between the boundaries between the NoSQL, with different tools to achieve different purposes, a variety of database technology in a set of systems, each of their respective roles. So, as Fowler says, no matter how the NoSQL is defined, it is useless work.

relational database (SQL) has become the mainstream technology for more than 20 years, which has its rationale. It stores data on disk, and people can perform various operations on the data through the most standardized language SQL. Its transactional (transactional) ability to effectively provide user concurrent access control and provide consistency for application data invocation. And because relational database mainly stores structured data, its data model and standardization are more applicable to the generation of reports (report).

But the biggest problem with relational databases is that they are designed to run on a single server, so it is likely that there will be a huge bottleneck in scale-out. Scale-up is you buy a larger machine to run the database, Scale-out is to use a number of common servers to form a cluster, so that the database distributed in the node of this cluster. The concept of cluster (Cluster) is to use more machines to do one thing, in which if a machine is down, other machines can continue to operate, so the entire cluster can work properly. In this regard, some of the big internet companies in the front, like Google and Amazon have run a very large cluster, in the gradual derivation of the concept of cloud computing. They are aware that the SQL database in response to the cluster has a relatively large performance bottlenecks, so they abandon the SQL, they began to develop such as bigtable and Dynamo storage platform, they should be the earliest NoSQL database.

As a result, people began to notice the problem, and a variety of nosql products emerged. Adewale the characteristics of such a database: No relational data model, no Structured query language, often for large cluster design, no fixed schema, a record can store arbitrary data, often open source. One of the major problems that it solves is large dataset storage, where large clusters using NoSQL databases can store PB-level data and process large amounts of analysis data. and a variety of data models can be used to deal with different types of data, such as documents, pictures, videos and other unstructured data.

This does not mean abandoning relational databases, of course, but it is still the most efficient when it comes to dealing with certain types of data. As with acid and Third-party tool support, NoSQL databases are far less than traditional relational databases. So Fowler suggests that people consider the so-called hybrid persistence principle, which uses a variety of data storage technologies based on the need for data in different applications. For example, fast read and write access, no persistent applications can use Redis, financial and reporting system, can use relational database, cluster of large-scale analysis data, you can use Cassandra, while many read operations, write operations are very few, you can use MongoDB. People can choose the right database technology, without the need for a whole brain to use the SQL database.

This article is mainly about the development of SQL and NoSQL database and their pros and cons, the next article will be more discussion of the principle of mixed persistence, how to use good polyglot persistence.

(Responsible editor: Lu Guang)