Explore the architecture of the NUODB database: What is NuoDB?

Introduced

The traditional relational database is suitable for the vertical scaling (scale-up) architecture. In other words, to handle more load, you need to switch to a more powerful computer. A few years ago, this meant that in order to support a horizontal scaling (scale-out) architecture, you had to give up SQL, or use zoning, active-passive replication and other techniques. It is not possible to implement a true acid programming model on a flexible, logical database. It was this situation that led to the Newsql movement, and the emergence of NuoDB was also to solve the problem.

NuoDB is a relational database designed for cloud scaling. How do you understand it? NuoDB is a real SQL service: It has all the properties of acid transactions, supports standard SQL languages, and has true relational logic. And from the start, NuoDB's design allows it to scale in a way that the cloud services can scale.

"Cloud scaling" is no longer defined here. If you are really interested, you can go to our technical blog to learn what cloud scaling is, and the article has a detailed definition. Simply put, you need a horizontal scaling model, but I think it should also be agile, easy-to-use, automated, secure, and highly available. This article is from this point of view.

It should be noted that NuoDB is "just" a software. This means that NuoDB is running on Linux, Mac, Windows, or Solaris on either a laptop, a private cloud, or a public cloud. You can use NuoDB in Amazon Web Services or Google Compute engine, either by integrating it with OpenStack or by running as a local Windows service on a laptop computer. NuoDB is flexible, you can test, develop, and deploy to anywhere in your own way.

This article will describe what NUODB is and what architecture it can do to meet some of today's challenges and what it can do to solve your problems. By the end of this article, you will be able to understand the differences between the key concepts and architectures of NUODB. You can also understand some of the actual deployment and management functions, scaling your own NuoDB database.

Three-tier architecture

The easiest way to learn about NuoDB is to understand its three-tier architecture. NuoDB includes management, transaction, and storage tiers. Let's go back to the management layer and look at the transaction and storage tiers.

To make a relational system scalable, splitting transactions and storage is the key. The traditional SQL database synchronizes the Data Representation (page) on the hard disk with the B-tree structure in memory. This tight coupling is effective, but has a very large impact on IOPS (I/O operations per second), making it difficult to scale horizontally. The architecture that separates these roles can scale horizontally, with little impact on data such as disk throughput.

NuoDB persistence and transaction processing are two completely separate tasks, which means that you can scale and handle failures separately for these layers. Scale the transaction throughput horizontally? You can do it without adding a disk. Want to have an independent archive to prevent data center failure? Achieving the goal does not affect transactional performance at the same time. This separation is not only conducive to the expansion of the system, but also more conducive to scaling on demand, according to your needs for deployment.

The transaction layer is responsible for atomicity, consistency, and isolation, but does not care about persistence. This also means that the transaction layer is in memory, it runs fast, anything can fail, and no data or consistency is lost at any time. The transaction layer is also a cache layer (especially if it can be cached on demand, as we'll introduce later), and you don't need to add other caching logic on top of the NuoDB database.

Obviously, the storage layer is responsible for the D (persistence) in acid. The storage layer is always active and consistent with all data. It is responsible for persisting data at the time of submission, and the transaction accessing the data in the event of a failed hit in the cache.

It should be noted that the "submission" in the NuoDB is adjustable. You can sacrifice performance and high availability to some extent because they are in different layers. To understand the approach and reason for adjusting the submission agreement, I need to explain what these layers are like.

Peer-to-peer Coordination

These two database tiers consist of multiple processes that can run across any number of hosts. Each individually executable process runs in one of two modes, either the transaction engine (Transaction engine,te) or the Storage Manager (Storage MANAGER,SM). All processes are peer-to-peer, without a single coordinator or point of failure, and there is no host-specific configuration. By default, all peer processes authenticate and communicate with each other through encrypted sessions.

Each TE is responsible for accepting connections from the SQL client and processing the query. The cache is stored in the TE process space. SM and Te communicate with each other through simple peer-to-peer coordination protocols. When TE fails the local cache hit, it can get the desired object from any peer process, which usually means that if there is an object in another Te cache, go to that te, which is much faster than SM querying data from persistent storage.

This simple, flexible process model simplifies startup, horizontal scaling, and migration. For example, suppose you want a NuoDB database that is as simple as possible. You start a separate TE and SM on the same host. Then you have a running, fully acid database, but they are all on the same host. You can test this on your laptop, but it's not very suitable for a real deployment environment.

Next you can install the software on the second host and send a message for it to start a new te. The new TE will authenticate with the existing processes, load some of the root elements into its own cache, and report that it is ready to handle the transaction load. The whole process takes less than 100 milliseconds to get ready for work from messages to TE. Running TE on two separate hosts can double the transaction throughput of the database and increase the recovery capability of the failure.

The drawback is that we have only dealt with one aspect of permanence. Next, you can build a third host and send a message for it to start the second SM. The second SM is automatically synchronized with the running system, ready to participate actively in the database processing. So far, another aspect of database persistence has been dealt with. is as simple as that.

Finally, let's look at how to build the first TE and SM on different hosts. You can guess, online fourth host, on this new host to start a te or a SM, when it is ready to turn off the original TE or SM (regardless of what you started on the new host). What you have done is a real-time migration of database components without loss of availability. Because the entire service has not been closed. This is also configured with a fully redundant database, because any host may fail, but you still have a complete data archive and can handle transactions.

All work is done easily and quickly, because NuoDB is built on a simple model based on process and equivalence. In addition, there are simple on-demand caching schemas and data formats that are truly cached and shared. But the data format is not a real SQL structure. We call it an atom (atom).