Detailed explanation of the network architecture VL2 of the data center, and detailed explanation of the vl2 of the data center

Detailed explanation of the network architecture VL2 of the data center, and detailed explanation of the vl2 of the data center

I. background

With the development of network technology, data centers have become the infrastructure for providing IT network services and distributed parallel computing. The data center is widely used, and application requirements are constantly increasing. The business data volume reaches TB/P level or above. In addition, video and financial service data have strict requirements on service quality, latency, and bandwidth. Therefore, when constructing a data center network, the performance of the data center network is very high.

1. Data Center cost

Table 1 shows the cost of the data center, most of which comes from the server. However, the service resource utilization rate of the data center is not high, and the server utilization rate is usually below 30%. In addition to low utilization, long supply cycle, uncertain demand changes, fast demand updates, and high management risks, redundant resources are required to ensure fault tolerance and other reasons, leading to high costs for data centers.

Table 1. Costs of data centers

2. Data Center performance requirements

Data center performance requirements include flexibility, scalability, multi-path transmission, low latency, high bandwidth, modular design, flat network design, low cost, green and energy-saving. The most important thing is flexibility, that is, the data center services can be allocated to any server. This improves service development efficiency and reduces costs. Three important aspects of flexibility:

• Workload management: You can quickly deploy service code on the server.
• Storage Management: in a distributed storage system, servers can quickly access stored data.
• Network: the server can communicate with other servers in the data center.

2. tree-like Data Center Network Architecture

The most commonly used tree-like architecture in traditional data centers is shown in 1. The traditional data center network is a three-tier architecture with the lowest layer as the server for processing services, the second layer as the switch for data exchange, and the third layer as the Access Router and border router for routing.

1. process the request

Multiple applications run in the data center at the same time. Each application generally runs on a specific server set and within the data center, requests from the Internet are distributed to the corresponding service pool of the application through Server Load balancer for processing. The IP address that receives external requests is called a virtual IP address (VIP). The server set that processes the requests is a direct IP address (DIP ). Requests from the Internet are routed to the layer-2 domain through the layer-3 border router (BR) and Access Router (AR, the VIP address of the application is configured on the Load balancer (LB) of the switch connected to Figure 1. For each VIP, LB is configured with a dip list, which includes server () according to this list, the Server Load balancer allocates the received requests to the server pool corresponding to the DIP for processing.

Figure 1. Network Architecture of traditional data centers

2. Problems with tree architecture

Due to the limited addressing capability of a single switch, the data center needs a layer-3 router to add more switches to expand services. However, this hierarchical tree architecture has a series of problems, resulting in low resource utilization in the data center.

(1) limited bandwidth between servers

In a data center, the link rate between the server and the switch is usually 1 Gb, And the link bandwidth between the switch domain switches is 10 Gb. As shown in figure 2, if each vswitch has 50 servers, the total bandwidth from the server to the vswitch is 50 Gb, far greater than the bandwidth between vswitches. The over-subscribe ratio between the server and the switch is. The higher the level, the more serious the excess subscription, the more the excess subscription ratio of the server to the router even reached. In this case, communication between a server in a subnetwork and a server in another subnet is limited by the bandwidth of the Upper-layer link. servers that fail to seize the bandwidth can only wait, wasting server resources.

Figure 2. oversubscription of tree architecture

(2) scattered resources

3. If the server of the same service is not deployed in the same subnet, the communication between the servers will be affected by the above-mentioned oversubscription phenomenon, thus increasing the communication overhead. To avoid this problem, servers of the same service should be deployed in a server set as much as possible. To ensure service scalability and stability, some redundant resources need to be added, however, when these resources are not used, they cannot be used by other services, resulting in a waste of resources.

Figure 3. Resource Dispersion

(3) low resource utilization, affecting services

Excessive subscription and scattered resources will lead to low service resource utilization. In addition, different services in the same subnet will also affect each other. For example, when a service suffers a flood of traffic, other servers in the same subtree will also be affected.

Iii. VL2 data center network architecture

The VL2 data center network architecture was proposed by Microsoft. after observing the traffic in multiple actual data centers, it summarized the traffic characteristics of the data center and designed a virtual 2-layer network architecture. Using the CLOS architecture, the new internal addressing mode of the data center and the flexible, high-performance, and high-utilization data center network architecture implemented by algorithms such as VLB and ECMP.

1. VL2 Architecture

4. The VL2 architecture is divided into two layers: The underlying server and the upper-layer switch. The two layers are connected by a rack switch (ToR. The switch layer includes the aggregation switch (Aggregate Switches) and the Intermediate switch (Intermediate Switches). The link connection between the aggregation switch and the relay switch forms a two-byte diagram. The network adopts the CLOS architecture, expand the link bandwidth. Each aggregation switch can be connected to other aggregation switches through a relay switch. If one of the n relay switches fails, only one/n of Bidirectional bandwidth is reduced. This design increases the number of paths and the robustness of the network.

Figure 4. VL2 Architecture

2. VL2 addressing

VL2 uses two types of Addresses in the data center. The underlying server uses AAs (Application Addresses) and the upper-layer switch uses LAs (Locator Addresses ). In this address allocation method, the server considers that it is in the same subnet as other servers because they use the same AAs address prefix. VL2 implements addressing by adding the shim sub-layer, ToR switch tunnel, and Directory System in the server protocol stack. Specific addressing Method 5: The Application Server S communicates with the other server D. During the first communication, S sends the physical address of the ARP packet request D, at this time, the shim layer in the protocol stack will intercept this ARP packet, that is, no ARP broadcast occurs, but send data to the directory system and request the LAs address of D. The directory system records the AAs ing relationship between AAs-LAs, where AAs is the server address and LAs is the ToR switch address connected to the server. Therefore, after the directory server receives the S request, the returned address is the ToR server address of D. After receiving a response from the directory system, the shim layer encapsulates the data packet. the destination address is the ToR address of D, that is, the LAs address. Then, send the packet to the ToR server. This ToR sends data packets to the ToR switch of D through the aggregation switch and relay switch. D. After the ToR switch receives data, it unpacks the data, obtains the real destination address of the data packet, and forwards it to the server. 3.3 VL2 load balancing and multi-path Transmission

VL2 uses VLB for load balancing and ECMP for multi-path transmission. As shown in figure 5, all the relay switches use the same LAs address. The distance between any switch and the relay switch is three hops, and there is no distance. Each aggregation switch can communicate with the server. After the data arrives at the aggregation switch, the aggregation switch randomly selects a path for data transmission. Because the address of the relay switch is the same, you only need to select a path in the link status for data transmission, to achieve multi-path transmission.

Figure 5. VL2 addressing mode

3. VL2 directory update mechanism

The Directory System of VL2 consists of RSM (Replicated State Machine) and DS (Directory System ). RSM is used to ensure consistency among multiple directory servers. The reliability of LAs-AAs ing is mainly used to write ing. DS is mainly used for read ing and user ing requests. Each DS caches all AAs-LAs mappings in RSM and synchronizes them with RSM every 30 seconds. When an update occurs on the server, such as migration of a virtual machine, the system sends an update message to the DS server and the DS sends the UPDATE message to the RSM server. After receiving the message, the RSM server updates its own ing relationship and copies the update to all other RSM instances for ing backup and redundancy. Then, reply to the DS server ACK message to confirm that the ing has been updated. Then, the DS server replies to the server ACK to confirm that the ing has been updated. Finally, all DS instances are notified to update the DS ing. In addition, VL2 uses a passive update mechanism. If a DS server receives an old ing request, the ing does not exist but the DS is not updated yet. Without knowing it, DS will still use this old ing request to respond, but the ToR switch of the receiver finds that the target server is not in its own domain, the system will forward information to DS to notify DS that the ing has expired and trigger DS to update the ing.

Figure 6. VL2 directory update mechanism

Iv. Summary

Vl2 uses a new network architecture to address the problems of excessive subscription, low resource utilization, and high data center costs in traditional data centers. Increase the bandwidth in the data center and use a new addressing Method to Solve the resource segmentation problem. It meets the needs of data center flexibility. It also uses VLB, ECMP and other algorithms to achieve load balancing and multi-path transmission, increase resource utilization, and improve network stability. However, the VL2 architecture needs to change the server's host protocol stack and require a high-performance, low-latency directory system to provide the ing search service, which brings additional overhead to the data center.