SMP, NUMA, MPP architecture

From the system architecture, the current commercial server can be broadly divided into three categories, that is, symmetric multiprocessor architecture (Smp:symmetric multi-processor), non-uniform storage access structure (numa:non-uniform memory access), And a massive parallel processing structure (mpp:massive Parallel processing).

SMP (Symmetric multi-processor )

SMP (symmetric multi-processor), a symmetric multiprocessor system has many tightly coupled multiprocessor systems in which all CPUs share all resources, such as bus, memory, and I/O system, or only one copy of the operating system or the management database, one of the biggest features of this system is the sharing of all resources. There is no difference between multiple CPUs, equal access to memory, peripherals, an operating system. The operating system manages a queue in which each processor sequentially processes the process in the queue. if two processors simultaneously request access to a resource (for example, the same memory address), the hardware and software lock mechanism is used to solve the resource contention problem.

The

Symmetric multiprocessor architecture refers to multiple CPUs in a server that work symmetrically, without primary or secondary relationships. Each CPU shares the same physical memory, and the time required for each CPU to access any address in memory is the same, so an SMP server expands by increasing memory, using a faster CPU, increasing the CPU, expanding I/O (number of slots and buses), and adding more external devices (usually disk storage).

　　SMP The main feature of the server is sharing, all resources in the system (CPU , memory, I/O and so on) are shared. because of this feature, the main problem with the SMP Server is that it has a very limited ability to scale. For SMP servers, each shared link can cause bottlenecks in the SMP server expansion, and the most restricted memory. Experiments have shown that the best CPU utilization for SMP servers is between 2 and 4 CPUs.

NUMA (non-uniform Memory Access)

Numa is one of the results of this effort as people begin to explore how to effectively scale up and build large-scale systems due to the limitations of the ability of SMP to expand. Use NUMA technology that can put dozens of CPUs (even hundreds of CPUs ) is combined within a single server. Its CPU module structure is shown in 2:

numa module, each CPU slots, etc. Because of the connection and information interaction between its nodes through the Interconnect module (such as crossbar Switch Span style= "text-decoration:underline;" > You can access the entire system's memory (this is NUMA system and MPP obviously, accessing local memory is much faster than accessing remote memory (the memory of other nodes in the system), which is the origin of non-uniform storage access to NUMA. Due to this feature, in order to better perform the system performance, development applications need to minimize the different CPU

With NUMA technology, it is possible to solve the problem of extension of the original SMP system, which can support hundreds of CPUs in a physical server. Examples of typical NUMA servers include HP Superdome, sun15k, IBMp690, and more.

However, NUMA technology also has some drawbacks, because the latency of accessing remote memory far exceeds the local memory, so when the number of CPUs increases, system performance does not increase linearly . when HP released the Superdome server, it published its relative performance values with other HP Nuix servers, and found that the relative performance value of the 64-way CPU Superdome (NUMA fabric) was 20, while the 8-way N4000 (shared SMP structure) The relative performance value is 6.3. From this result see, 8 times times the number of CPUs in exchange for only 3 times times the performance of the promotion.

MPP (Massive Parallel processing)

Unlike Numa,MPP provides another way to extend the system by connecting multiple SMP servers through a certain node internetwork, working together and accomplishing the same tasks. From the user's point of view is a server system . The basic characteristic is that multiple SMP servers (each SMP server is called node) are connected through the node internetwork, each node only accesses its own local resources (memory, storage, etc.), is a completely non-shared (Share nothing) structure, and thus the best expansion capability, theoretically is the extension of unlimited, The current technology can achieve 512 nodes interconnection, thousands of CPUs. At present, the industry has no standard on the node internetwork, such as NCR's BYNET,IBM Spswitch, they all adopt different internal realization mechanism. However, the node Internet is only used internally by the MPP server and is transparent to the user.

　　in the MPP in the system, each SMP nodes can also run their own operating systems, databases, and so on. But Unlike NUMA, it does not have a problem with offsite memory access. In other words, the CPU within each node cannot access the memory of the other node. The information between nodes is implemented interactively through the node internetwork, which is generally referred to as data redistribution (redistribution).

　　but MPP The server needs a complex mechanism to dispatch and balance the load and parallel processes of each node. Currently, some MPP-based servers tend to mask this complexity through system-level software such as databases. For example, NCR's Teradata is a relational database software based on MPP technology, when developing an application based on this database, regardless of how many nodes the backend server consists of, the developer is faced with the same database system, without having to consider how to dispatch the load of one of the nodes.

The MPP (massively Parallel processing), massively parallel processing system, is made up of a number of loosely coupled processing units, and it is important to note that this refers to the processing unit rather than the processor. The CPU in each unit has its own private resources, such as bus, memory, hard disk, etc. Each unit has an operating system and an instance replica of the management database. The biggest feature of this structure is the non-sharing of resources.

Differences between the three architectures

SMP System and MPP System Comparison

Typically, the MPP system is less efficient than SMP because it wants to transfer information between different processing units, but this is not absolute because the MPP system does not share resources because it has more resources than SMP, and when the transactions that need to be processed reach a certain scale, MPP is more efficient than SMP.

　　This is determined by the proportion of time that the communication takes to occupy the computation time, and if the communication time is more, the MPP The system is not dominant, conversely, if the communication time is relatively small, the MPP the system can give full play to the advantages of resources to achieve high efficiency.

　　OTLP currently in use program, the user accesses a central database, if using the SMP system structure, it is more efficient than using MPP The structure is much faster. The MPP system shows advantages in decision support and data mining . It can be said that if the operation is not related to each other, there is less communication between the processing units, then the use of the MPP system is better, the opposite is not appropriate.

For SMP, a key factor that restricts its speed is the shared bus, so for the DSS program, only the MPP can be selected, but not the SMP, when the large program processing requirements than the shared bus, the bus will not be able to process, then the SMP system is not. Of course, two structures have their advantages and disadvantages, if they can be combined to learn from each other, of course, the best.

NUMA and MPP the Difference

In terms of architecture, NUMA and MPP have many similarities: they are made up of multiple nodes, each with its own CPU, memory, I/O, and the nodes can interact with each other through the node interconnection mechanism. So where are the differences? It is not difficult to find the difference by analyzing the internal architecture and working principles of the following NUMA and MPP servers.

　　first, its node interconnection mechanism is different, NUMA node interconnection mechanism is implemented within the same physical server, when a CPU when remote memory access is required, it must wait, which is also NUMA server cannot implement CPU The main reason for linear scaling of performance increases. While the node interconnection mechanism of MPP is implemented by I/O on different SMP servers , each node accesses only local memory and storage, and the information interaction between nodes is carried out in parallel with the processing of the node itself. Therefore , the performance of MPP can be linearly extended when adding nodes.

　　the second is a different memory access mechanism. within a NUMA server, any CPU can access the entire system's memory, but the performance of remote access is much lower than local memory access, so you should try to avoid remote memory access when developing your application. In the MPP Server, each node accesses only local memory, and there is no problem with remote memory access .

Choice of Data Warehouse

What kind of server is better suited to the data Warehouse environment? This requires starting with the load characteristics of the Data Warehouse environment itself. As we all know, the typical data warehouse environment has a large number of complex processing and comprehensive analysis, requiring the system to have very high I/O processing capability, and the storage system needs to provide sufficient I/O bandwidth to match. While a typical OLTP system is based on online transaction processing, each exchange involves little data, requiring the system to be highly transactional and able to handle as many transactions as possible in a unit of time. Obviously, the load characteristics of these two environments are completely different.

From a NUMA architecture, it can integrate many CPUs within a physical server, making the system highly transactional, and minimizing data interaction between different CPU modules due to remote memory access Shiyan longer than local memory access. Obviously, the NUMA architecture is more suitable for the OLTP transaction processing environment, and when used in the Data Warehouse environment, the CPU utilization will be greatly reduced because of the large amount of complex data processing which inevitably leads to a lot of interaction.

In contrast, the MPP server architecture is more capable of parallel processing, which is more suitable for complex data synthesis analysis and processing environment. Of course, it needs to use a relational database system that supports MPP technology to mask the complexity of load balancing and scheduling between nodes. In addition, this parallel processing ability also has a great relationship with the node internetwork. Obviously, the MPP server, which adapts to the data Warehouse environment, should be very prominent in the I/O performance of the node internetwork in order to realize the performance of the whole system.

Excerpt from Baidu Library, their own later access to convenient

SMP, NUMA, MPP architecture