ORACLE DATABASE 11G RAC knowledge of the cluster concept of a detailed tutorial (i)

Introduction to cluster concept Cluster Terminology

Service hardware: Refers to the hardware that provides computing services, such as PC, PC server.

Service entities: Service entities usually refer to service software and service hardware.

Node: A stand-alone host running the Heartbeat process is called a node, and the node is the core component of HA, with operating systems and Heartbeat software services running on each node.

Resource (Resource): A resource is an entity that can be controlled by a node, and when a node fails, these resources can be taken over by other nodes. such as: Disk partitions, file systems, IP addresses, application services, shared storage

Event: An event is something that can happen in a cluster, such as a node system failure, network connectivity failure, NIC failure, and application failure. These events can result in the transfer of resources from the node, and the HA test is based on these events.

What is a cluster

A cluster (cluster) is a group of computers that, as a whole, provide users with a set of network resources, and these individual computer systems are nodes of the cluster (node). Clustering provides the following key features.

(i) scalability. Cluster performance is not limited to a single service entity, new service entities can dynamically join the cluster, thereby enhancing the performance of the cluster.

(ii) High availability. Clustering enables clients to avoid the "out of service" warning from being easily encountered through service entity redundancy. When a node server fails, the application running on this server is automatically taken over on another node server. Eliminating a single point of failure is important to enhance data availability, accessibility, and reliability.

(iii) Load balancing. Load balancing can distribute tasks more evenly to compute and network resources in the cluster environment in order to improve data throughput.

(iv) Error recovery. If one of the servers in the cluster is unavailable due to a failure or maintenance need, resources and applications will be transferred to the available cluster nodes. This is because the resources in one node cannot work, and the process of another available node can transparently take over and continue to complete the task is called error recovery.

The links and differences between distributed and cluster are as follows:

(a) distributed refers to the distribution of different businesses in different places.

(b) While clustering refers to the pooling of several servers to achieve the same business.

(c) Each node of the distributed, can do the cluster, and the cluster is not necessarily distributed. and distributed, from the narrow sense, but also similar to the cluster, but its organization is relatively loose, unlike the cluster, there is a certain organizational, a server down, other servers can top up. Each node of the distributed, complete different business, a node is down, this business is inaccessible.

Clusters are divided into three major categories:

HA: Highly available cluster (high availability Cluster).

LBC: Load Balancing cluster/Load Balancing system (load Balance Cluster)

HPC: Cluster of scientific computing clusters (high performance Computing Cluster)/high-performance Computing (HI performance Computing). 

Why build a database cluster

With the fast development of economy, the rapid expansion of enterprise scale, the explosive growth of the number of enterprise users and the quantity of data have put forward a severe test to the database. For all databases, in addition to documenting the correct processing results, there are several challenges to face. How to improve the processing speed and realize the balanced load of the database. • How to ensure database availability, data security, and how to achieve data clustering scalability. How to solve these problems comprehensively has become the focus of many enterprises.

In the database, the formation of clusters is the same reason, mainly for the following reasons:

(a) with the growth of enterprises, increased business volume, database access and the rapid growth of data, its processing capacity and calculation speed correspondingly increased, so that a single device can not afford.

(b) In the above cases, if you throw away the existing equipment, do a lot of hardware upgrades, will cause the waste of existing resources, and the next time the volume of business increases, and then face again a high level of hardware upgrades investment. Therefore, people hope that through a few small and medium-sized servers to build clusters, to achieve the load balance and continuous expansion of the database; When a higher database processing speed is required, a simple addition to the database server can be extended.

(iii) database as the core of information system, plays a very important role, a single device can not guarantee the continuous operation of the system, if a system failure, will seriously affect the normal operation of the system, and even bring huge economic losses. Therefore, people hope that through the formation of database clusters, to achieve high availability of the database, when a node failure, the system will automatically detect and transfer the application of the fault node to ensure the continuous work of the database.

(iv) The enterprise's database holds the important information of the enterprise, some core data even related to the lifeblood of the enterprise, a single device can not guarantee the security of the database, once lost, it is difficult to find back. Therefore, it is hoped that through the establishment of database clusters, the redundancy of the data set, the backup data to ensure security.

  Classification of database clusters

Database cluster technology is a technology that combines multiple servers to form a cluster to achieve comprehensive performance better than a single large server, this technology can not only meet the need of application, but also save the investment cost greatly. Database cluster technology belongs to two kinds of systems: Cluster technology based on database engine and cluster technology based on database Gateway (middleware). In the area of database cluster products, it mainly includes Oracle RAC, Microsoft MSCS, IBM db2udb, Sybase ASE, and Icx-uds of cluster technology based on database Gateway (middleware), which is based on the database engine cluster technology.

Generally speaking, database cluster software focus on the direction and try to solve the problem divided into three categories: L load Balancing cluster (load Balance CLUSTER,LBC) focus on the horizontal expansion of the database, improve the performance of the database. • High-availability clusters (availability Cluster,hac) focus on ensuring database applications continue. Most of the database clusters focus on this. • High-security cluster (CLUSTER,HSC) focuses on disaster tolerance.

Only Oracle RAC can achieve the three scalable

  distributed database architectures

(i) Oracle RAC:

The biggest feature of its architecture is the shared storage architecture (Shared-storage), where the entire RAC cluster is built on a shared storage device, with high-speed network interconnection between nodes. Oraclerac provides very good high-availability features, such as load balancing and application transparent dice (TAF), with the greatest advantage of being fully transparent to applications and switching to a RAC cluster without modification. But the scalability of RAC is limited, first, because the entire cluster relies on the underlying shared storage, the I/O capabilities and availability of shared storage determine the capabilities that the entire cluster can provide, and for I/O intensive applications, this mechanism determines that subsequent expansions can only be Scale up type, For hardware costs, developer requirements, maintenance costs are relatively high. Oracle is clearly aware of this problem, in Oracle's MAA (Maximum availability Architecture) architecture, using ASM to consolidate the capabilities of multiple storage devices, so that the RAC-bottom shared storage devices have the ability to scale linearly, The entire cluster is no longer dependent on the processing power and availability of large storage.

Another problem with RAC is that as the number of nodes increases, the cost of communication between nodes increases, and when a certain limit is added, adding nodes may not bring performance improvements or even performance degradation. The main reason for this problem is that Oracle RAC is transparent to the application, the application can connect to any node in the cluster, and when the application contention resources on different nodes, the communication overhead between the RAC nodes can seriously affect the processing capability of the cluster. So there are two recommendations for using ORACLE RAC: L Communication between nodes using high-speed Internet; L distribute different applications to different nodes as much as possible.

For this reason, Oracle RAC is typically extensible in the DSS environment (decision support system decision Support systems, short of DSS) because the DSS environment can easily distribute different tasks across different compute nodes, and for OLTP applications (on -line Transaction processing online transaction processing systems, Oracle RAC is more often used to increase availability than to increase scalability.

(ii) MySQL Cluster

The MySQL cluster is completely different from the Oracle RAC, and it takes a shared nothing architecture with a unshared schema. The entire cluster consists of a management node (NDB_MGMD), a processing node (MYSQLD), and a storage node (NDBD) that does not have a shared storage device. The MySQL cluster mainly utilizes the NDB storage engine, the NDB storage engine is a memory storage engine that requires the data to be loaded into memory. Data is automatically distributed across different storage nodes in the cluster, and each storage node holds only one fragment of the complete data (fragment). At the same time, users can set the same data to be stored on multiple different storage nodes to ensure that a single point of failure does not result in data loss. MySQL cluster mainly consists of 3 parts: L SQL server node L NDB data Storage node L monitor and manage nodes

Such hierarchies are also related to the architecture in which MySQL itself separates SQL processing from storage. The advantage of MySQL cluster is that it is a distributed database cluster, processing nodes and storage nodes can be linearly increased, the whole cluster has no single point of failure, availability and scalability can be high, more suitable for OLTP applications. But its problem is: L NDB ("NDB" is an "in-memory" storage engine with high availability and good data consistency. The storage engine must require that the data be loaded into memory, which is more restrictive, but is now being improved by the new version of NDB, allowing only the indexed data to be loaded in memory, and the data can be saved on disk. L Current MySQL cluster performance is not ideal, because the data is distributed to different storage nodes according to the primary key hash, if the application is not through the primary key to obtain data, we must scan all the storage nodes, return the results to the processing node to deal with. In addition, the write operation needs to write multiple data to different storage nodes at the same time, and the network requirements between nodes are very high.

Although the current performance of MySQL cluster is not ideal, but the architecture of share nothing must be the future trend, Oracle took over MySQL, also in the development of MySQL cluster, I have great expectations of the future of MySQL cluster.

(iii) Distributed database architecture

MySQL 5 after the data table partitioning function (Sharding), Sharding is not a particular database software ancillary functions, but the specific technical details above the abstract processing is horizontal expansion (Scale out, or horizontally extended, outward expansion) solution, The main purpose of this paper is to break the I/O capability limit of single node database server and solve the problem of database extensibility. For example, Oracle's RAC is a shared storage mechanism, for I/O-intensive applications, bottlenecks easily fall on the storage, such a mechanism determines that subsequent expansion can only be Scale up (upward expansion) type, for hardware costs, developer requirements, maintenance costs are relatively high. Sharding is basically an extensibility solution for open source databases, and few have heard of the Sharding of business databases. The current trend of the industry is basically to embrace Scale out, gradually liberated from the Scale up.

The advantage of the Sharding architecture is that the cluster expansion capability is very strong, almost can be linear expansion, and the entire cluster is very high availability, some node failure, does not affect other nodes to provide services. Sharding principle is simple, easy to implement, is a very good solution to the expansion of the database program. Sharding is not the silver bullet of the database expansion scheme, but also has its unsuitable scenario, such as processing transaction type application It may cause the application architecture complex or limit the function of the system, this is also its flaw.read-write separation is an important idea of architecture distributed system。 Many of the system's overall ability to deal with the growth of the business can not keep pace, so it is bound to bring bottlenecks, simple upgrade hardware can not be once and for all. According to the characteristics of the business type, we need to make a series of adjustments from the architecture mode, such as Business module segmentation, database splitting and so on. Centralized and distributed are two opposing modes, and the application characteristics of the same industry also determine the structure of ideas. For example, some portals in the Internet industry, for technical and cost considerations, more use of open source database products (such as  mysql), because most of the typical read more write less request, so for MYSQL and its replication technology to provide a great way to the conditions. And compared to some traditional intensive trade, such as telecommunications, finance and so on, considering the single point of processing capacity and reliability, stability and other issues, may be more use of commercial databases, such as  DB2, oracle  and so on. On the database level, most of the traditional industry core library using a centralized structure of ideas, using a highly equipped minicomputer to do host carrier, because the database itself and the host powerful processing capacity, the database can generally support the operation of the business, therefore, the Oracle read-write separation architecture relative to MySQL, relatively less. The read-Write separation architecture utilizes the replication technology of the database to distribute the reading and writing on different processing nodes to achieve the goal of increasing usability and extensibility. The most common approach is to take advantage of MySQL Replication technology, Master DB assumes write operations, replicates data changes to multiple Slave DB, and undertakes read operations. This architecture is suitable for read-intensive types of applications, and read performance can grow linearly by increasing the number of Slave DB. To avoid a single point of failure in Master DB, the cluster typically uses two Master db for dual-machine hot standby, so the entire cluster has very high levels of read and write availability. In addition to providing Active Standby functionality from 11g , mysql,oracle  has the basis for implementing a read-write separation architecture. The flaw in the read-write separation architecture is that each node must have a complete set of data, either Master or Slave, and if the data volume is large, the scalability of the cluster is limited by the storage capacity of a single node, and for write-intensive types of applications, The read-write detach schema is not appropriate.

Using Oracle read and write separation of ideas, Writer DB and Reader db using log replication software to achieve real-time synchronization; Writer DB is responsible for transaction-related real-time query and transaction processing, Reader DB is responsible for read-only access, processing some non-real-time transaction details, summary query of report class, etc. 。 At the same time, in order to meet the requirements of high availability and scalability, the read-write end of the appropriate extension, such as Writer DB using HA or RAC architecture model, at present, in addition to the database manufacturers cluster products, the solution to the database expansion capacity of the main two: data fragmentation and read-write separation The principle of data slicing (Sharding) is to do horizontal segmentation of data, similar to the principle of hash partitioning, through the application of architecture to solve access routing and Reader DB can use multiple sets, through load balancing or business separation, effectively share the pressure of reading library.

For shared-nothing database schema, one of the core problems is the real-time synchronization of read-write library, and although Reader DB is only responsible for business queries, it does not mean that the database is functionally read-only. Read-only is from the application point of view, in order to ensure data consistency and conflict considerations, because the query business module may need to involve some intermediate processing, if you need to process in the database (depending on the application requirements and design), so the reader DB still need to be writable on the functional. Here is a discussion of data synchronization technology selection problem:

There are many technologies for real-time data synchronization, based on the OS layer (such as VERITAS VVR), based on storage replication (mostly supported in high-end storage), based on application distribution or database tier based technology. Because data synchronization may not be a single DB synchronization, involving issues such as business data selection and multi-source consolidation, the majority of OS replication and storage replication is not appropriate for the technology of read-write separation. Based on the log Oracle replication technology, Oracle's own components can be implemented, but also have sophisticated business software. The choice of a commercial stand-alone product or Oracle's own component functionality depends on a variety of factors. such as the team's corresponding technical operational capacity, project input cost, business system load level.

With Oracle's own component functionality, no outside of Logical Standby, stream, and 11g physical Standby (Active Data Guard), the stream is the most flexible, but most unstable, 11g Physica L Standby supports recovery and read-only parallelism, but because it is not a logical application mechanism for logging, it is most limited in a read-write separation scenario. If the technical team is sufficient for the relevant technology, and the processing capacity of the selection scheme can support the requirements of data synchronization, it is feasible to adopt Oracle's own components. Choose commercial products, more for the stability, processing capacity, and other considerations. There are also a number of mature Oracle replication software available, whether it's a veteran shareplex or a DDS from a native DSG company, or a realsync of the nine Bridge Company, or an Oracle upstart goldengate. As Goldengate is being acquired and promoted by Oracle, it is believed that Goldengate will do well in disaster tolerance, data distribution and synchronization. Of course, the architecture is a reliable distributed read and write separation system, but also need to do a lot of application design, not in the scope of this article.

(iv) CAP and BASE theory

Distributed domain CAP theory: L Consistency (consistency), consistent data updates, all data changes are synchronized l availability (availability), good response performance L Partition tolerance (partitioning fault-tolerant) reliability

Theorem: Any distributed system can satisfy only two points at the same time, and cannot be balanced by three.

Advice: Architects should not waste their energy on how to design a perfect distributed system that satisfies the three, but rather a trade-off.

The ACID model of a relational database has high consistency + availability difficult to partition: L atomicity atomicity: All operations in a transaction must be completed, or all are not completed. L Consistency consistency. The database should be in a consistent state at the start or end of a transaction. L Isolation isolation layer. The transaction will assume that only it is operating the database itself, not knowing each other. L Durability. Once the transaction is complete, it cannot be returned.

(v) Cross-database transactions

2PC (two-phase commit), 2PC is the anti-scalability patterns (Pat Helland) is an anti-scalable mode, which means that traditional relational databases are very difficult to implement a distributed database cluster, Relational databases are very limited in their ability to scale. In recent years, the NoSQL (non relational database) movement has been developed, by sacrificing strong consistency, using BASE model, using the idea of final consistency to design distributed system, so that the system can achieve high availability and scalability. So, is it possible to implement a set of distributed database clusters that guarantee availability and consistency and provide a good scalability.

The main realization of base thought is to divide database by function sharding Fragmentation base idea focuses on basic usability, and if you need high availability, which is pure performance, then you have to sacrifice consistency or fault tolerance, and the base idea's solution has the potential to be tapped for performance. L Common: All the alternatives to relational database SQL, logic with data distribution, any model can be persisted by itself, separating data processing and data storage, separating read and write, storage can be asynchronous or synchronous, depending on the degree of conformance requirements. L Different points: NOSQL Key-value Storage products are the product BOX that meets the header of the relational database, which can be used in fields such as PHP Ruby, and is a kind of product that can be taken, while domain model + distributed cache + storage is a complex architecture solution, not a product , but this approach is more flexible and should be mastered by architects.

At present, there are many distributed database products, but most of them are for the DSS type of applications, because compared to OLTP applications, DSS applications are more easily distributed expansion, such as PostgreSQL based on the development of Greenplum, it is very good to solve the problem of usability and scalability, And it provides a powerful parallel computing capability. For OLTP applications, business characteristics determine their requirements: high availability, consistency, short response time, support for transactions and joins, and so on. Databases and NoSQL as more and more NoSQL products emerge, they have features that are not available in relational databases and can be done well in terms of usability and scalability.

First, NoSQL's application scenario is very limited, and a type of NoSQL is designed only for a specific type of application scenario. Relational databases are much more versatile, and the use of NoSQL has to be clear about whether their application scenarios are appropriate.

Second, using relational database and application architecture, such as Sharding and read-write separation technology, can also build a distributed database cluster with high availability and extensibility.

Third, relational database vendors are still very strong, the world has a large number of users, the demand will inevitably promote the advent of new products.

Finally, the development of hardware, such as the technology of flash memory maturing, the future flash memory can be used as a cache between the disk and memory, or a complete replacement of the disk. But the memory price is getting lower, the capacity is more and more big, the in-memory cache or the database application is more and more widespread, may bring the quantity level performance enhancement to the application. The IO problem facing the database will be greatly improved.