TRILL technology and its networking model

TRILL (Transparent Interconnection of lots of links, multi-link Transparent Interconnection) is a standard developed by IETF to implement layer-2 extension of data centers. Currently, some protocols have been standardized, such as RFC6325, 6326,6327 and so on. The core idea of this Protocol is to introduce mature layer-3 routing control algorithms into layer-2 switching, and add a new encapsulation (tunnel encapsulation) to the original L2 packets ), switch to the new address space for forwarding. The new address has a routing attribute similar to the IP address, and has many advantages such as large-scale networking, shortest path forwarding, equivalent multi-path, fast convergence, and easy scalability, this avoids STP, MSTP, and other technical defects and achieves robust large-scale L2 networking.

1. TRILL-implement layer-2 multi-path forwarding

The TRILL standard involves several important concepts:

Routing Bridge: A Routing Bridge, or RBridge or RB, is a collectively referred to as a network node that supports the TRILL function, similar to an IP Router;

VLAN x Forwarder: a vlan x Forwarder, similar to a PE role in a VPN, used for VLAN-based election. The main function is to encapsulate the user-side packets and send the TRILL header to the TRILL network for forwarding. Or, the TRILL network packet is unencapsulated and restored to the user-side message and sent to the user;

Nickname: 16-bit long, similar to an IP address, is the basis for RB Node Route computing. Nickname evolved from the Mac address because the Mac address has 48 bits. If it is directly used for encoding, the overhead is too large and the size is too large, so it is reduced from 48 bits to 16 bits and 64 K. Each node has different nick names. The nick names can be automatically selected or manually configured. Each RB can have multiple nick names.

The concept of "multi-path" was previously used only for IP forwarding. When two routers have multiple equal-cost or non-equivalent forwarding paths, the router can calculate the IP packet along the shortest path and measure the value according to the path, stream-based shared forwarding allows you to make full use of Bandwidth Resources. If we think about the most frequently used L2 forwarding table in our vswitch, that is, the MAC table, we can see that for a unicast table item, the outbound port can only be a unique physical port or aggregation port, and cannot have multiple independent physical ports at the same time. If so, the table item becomes a multicast table item. That is to say, MAC forwarding tables do not have L2 multi-path capabilities. The emergence of TRILL technology does not change this situation. Instead, it uses tunnel encapsulation to convert the original layer-2 MAC forwarding into a layer-3 route forwarding of a class IP address, that is, TRILL technology applies the IP packet forwarding idea to Ethernet frame forwarding. The ethernet switch supporting TRILL technology is called "RBridge (Routing Bridge )".

Because RB needs to encapsulate and unencapsulate user-side packets, we can call the end-point device responsible for packet addition, encapsulation, and encapsulation as Vlan X Forwarder, similar to the PE in VPN.

The router can calculate the shortest path and ECMP between each other through the link status routing protocol, and update the forwarding path when the topology changes. The RBridge uses the link state control protocol trill is-IS similar to the IS-IS routing protocol to calculate the shortest path and the equivalent multi-path. Trill is-IS only calculates the topology between RBridge, regardless of the topology between the two hosts in the network (in fact, the most common topology between the two rb is the direct connection mode ).

To implement the preceding routing control function, you need to define a globally unique identifier for each RB in the network. Because the Router ID has been used by the IP address and its form is similar to the IP address, considering that trill is-IS still a L2 service, TRILL selects a new ID named Nickname to identify each RB device. The final result of trill is-IS calculation IS to form unicast and multicast forwarding tables with different nick names.

Figure 1. RBridge forwarding to known unicast

1. When the unicast Ethernet frame enters the TRILL network through the Ingress RBridge at the edge of the TRILL network, an additional "TRILL Header" (similar to the IP packet header) is added before the original frame header ), it includes Ingress RBridge Nickname and Egress RBridge Nickname, just like the source IP address and destination IP address in the IP header. Add the "Next-Hop Header" before the "TRILL Header" (like the MAC header or PPP header before the IP packet) to complete the TRILL frame encapsulation. Since then, the forwarding process of TRILL frames between RBridge is like that of IP packets between routers. RBridge performs one-to-one jump Based on the Egress nick name in the TRILL header. The Next-hop header must be modified in each hop, while the TRILL header only changes the TTL value. RBridge implements Shortest Path forwarding and equivalent path sharing for TRILL frames, avoiding the link blocking problem caused by STP operation in traditional L2 Networks. The TRILL frame is eventually restored to a standard Ethernet frame in the Egress RBridge on the edge of the TRILL network and sent to the TRILL network.

RBridge only needs to know the Optimal Path to the next hop RBridge, and does not need to know how to reach the target host. Therefore, only Ingress/Egress RBridge needs to enable traditional MAC address learning (the MAC address learned from the local port and the MAC address learned from the remote Egress RBridge ), the core RBridge on the TRILL network does not need to maintain host-related MAC tables. In addition, RBridge can use a traditional Ethernet switch to connect to each other, and the BRridge and the interconnected switch can run the STP protocol, but RBridge will terminate the STP instance and will not spread the BPDU through RBridge.

Figure 2. RBridge multi-target Frame Forwarding

2. Processing of multi-destination Ethernet frames (broadcast, multicast, and unknown unicast, RBridge IS required to generate multiple distribution trees with different roots based on the calculation result of trill is-IS. The multi-destination frame enters the TRILL network, and Ingress RBridge selects a distribution tree for the frame to be forwarded on the TRILL network, and uses the root RBridge Nickname as the Egress RBridge Nickname in the "TRILL Header. The subsequent processing process is similar to the forwarding of IP multicast packets between multicast routers. Each RBridge selects TRILL replication and forwarding policies only based on the distribution tree identified by the root RBridge.

It should be noted that because the TRILL technology defines a new frame format, Traditional Ethernet switches cannot support this feature through upgrading software, only Ethernet switches using the new ASIC/NP chip can support TRILL forwarding.

Ii. TRILL limitations

Although TRILL has obvious characteristics, it also has some problems to solve. So far, TRILL is still in the process of standardization, and there are still a large number of drafts under discussion. The Protocol itself has the following problems:

VLAN extensions larger than 4 K are not supported. For virtualized multi-tenant cloud computing data centers, VLAN isolation is usually required for 4 K, while TRILL's support capability is still limited to 4 K, which is difficult to meet the needs;

Weak OAM support;

Because TRILL is mostly used in data centers, most of the RB is a direct connection network that does not span the Traditional Ethernet network. For such a network, TRILL's outer Ethernet header encapsulation is redundant and can be streamlined and optimized.

Only Level0 is supported, and the Multi Level mechanism is not supported;

I did not consider how to carry the FCoE business.

3. TRILL Application

TRILL's application in China is still in its infancy. Some operators, financial companies, large enterprises, and Internet companies are already paying attention to or considering TRILL technology.

The following factors must be taken into account for TRILL networking:

Convergence Ratio

L3 gateway deployment location

L3 gateway Load Balancing

The size of the MAC/ARP table of the device.

For specific networking applications, the following network types are listed Based on the deployment scenario:

1. networking model 1: existing networking expansion TRILL domain

Leaf + aggregation + spine three-layer networking environment, L3 gateway on the aggregation layer, centralized L3 Gateway

Figure 3. Expanding the TRILL domain of an existing network

Networking description (3 ):

Based on the existing POD, the new trill pod domain is expanded horizontally. the L3 gateway of the TRILL domain is located at the aggregation layer and connected to the core layer through the routing protocol;

To solve the problem of dual-active access on the leaf layer, the leaf node supports N: 1 virtualization, such as the IRF of H3C;

The two aggregation nodes also perform N: 1 virtualization to avoid VRRP configuration, so that the L3 forwarding traffic can be evenly shared in the gateway;

Since the existing commercial ASIC does not support TRILL + L3 processing in a Pipeline, how can the aggregation node implement TRILL + L3 forwarding?

Board proxy: TRILL and L3 on the device are separated to two different boards, and then the proxy is enabled between them, in essence, a pipeline flow of the original chip is divided into two chips for separate execution, reducing the requirements on the chip. Typical M1/F1 board combinations on Cisco N7K;

1: N is used to virtualize a device into two devices. One is TRILL and the other is L3. The two virtual devices are connected through external connections, just like the two completely independent devices are interconnected (4 ). For example, H3C MDC technology or Cisco VDC technology.

Figure 4. TRILL + L3 through device 1: N

This networking model can smoothly expand the TRILL Domain Based on the Data Center Network established by the existing traditional technology, and realize VLAN cross-rack L2 connection in the TRILL domain. STPs-free, high link full-active utilization, and high reliability. However, the TRILL domain is limited in size. VLANs can only be connected to the POD Layer 2 and cannot be connected across pods. In addition, the centralized L3 gateway limits the forwarding performance of L3.

2. networking model 2: Create a TRILL core to connect VLANs across pods

Create a TRILL core, which is in parallel with the L3 core

Figure 5. Create a TRILL core to enable cross-pod vlan connection

Networking description (5 ):

Based on the networking model 1, it can meet the needs of L2 interconnection between VLANs and a wider range of pods;

In order not to affect the existing networking, add a dedicated TRILL core, which is in parallel with the existing L3 core;

VLAN is divided into local VLANs in the POD, such as VLAN10, 20, 30, 40, and cross-pod vlan, such as VLAN1000;

Layer-3 forwarding traffic between the local VLAN in the POD, for example, between VLAN10 and VLAN20 or between VLAN30 and VLAN40, is directly forwarded on the local L3 Gateway (5 indicates traffic 1 );

L2 intercommunication across POD VLANs is performed through TRILL, and TRILL core is bypassed (traffic 2 in step 5 );

L3 forwarding across PODVLAN and local VLANs must be performed by bypassing the L3 core (Traffic 3 in step 5 ).

This networking model can be smoothly evolved on the basis of existing networking to achieve cross-pod vlan connection. However, the cross-pod vlan L3 gateway can only be deployed on the aggregation node of a POD, which has a performance bottleneck.

3. networking model 3: create a two-tier TRILL Network

Completely new TRILL is created. The leaf + spine two-layer structure is used, and the gateway is concentrated on the Spine node.

Figure 6. Create a two-tier TRILL Network

Networking description (6 ):

Cancel the aggregation layer, and the entire network is reduced to two layers;

The gateway distributes the load among multiple gateways on the spine node. because the number of gateways is greater than two, vrreps can be used to share the load among multiple gateways and back up the load. Figure 6 uses the blue arrow to show that the traffic sent by different hosts is forwarded to different gateway nodes for sharing using different gateway MAC addresses.

The two-layer architecture of this networking model is more streamlined and has low latency. VLANs can be deployed anywhere in the data center. L3 gateway supports load balancing and backup. L2 forwarding can achieve horizontal non-convergence and good scalability. However, for large networking scenarios such as thousands or even tens of thousands of virtual servers, the centralized gateway has a high requirement on ARP table items. The VRRPE sharing method cannot share the traffic of the same host in the domain.

4. networking model 4: L3 gateway enabling on leaf nodes

Figure 7. L3 gateway on the leaf node

Networking description (7 ):

For scenarios with low L3 forwarding requirements, the L3 gateway function can be centralized at the leaf edge;

The Spine node no longer functions as the L3 gateway. It only performs the TRILL forwarding function and does not learn users' MAC addresses;

Nodes that execute the leaf function can perform N: 1 virtualization to achieve VRRP-free configuration and gateway active-active forwarding.

In this networking model, the traffic at the spine node is evenly shared and backed up. Turning the leaf node up is equivalent to deploying a TRILL (8) layer-3 device group, with the core concentrated on a logical device. For spine nodes, the L3 forwarding traffic is fully shared according to the TRILL multi-path. Even if different traffic is sent from the same host, the traffic can be shared among spine nodes, more balanced sharing. However, the limitation is that the network is only suitable for scenarios with low L3 forwarding performance requirements.

Figure 8. layer-3 device deployment TRILL Network

5. networking model 5: distributed L3 gateway Mode Based on Model 3

Figure 9. Distributed L3 gateway networking

Networking description (9 ):

The leaf node and core node enable the L3 gateway function at the same time;

To ensure the connectivity between VLAN of the leaf node, a common interconnection VLAN is configured between all gateway devices of the entire network. It is equivalent to connecting all gateway nodes to a broadcast network;

Static Routing or dynamic routing protocols can be configured on each gateway device to form a route forwarding table;

The gateway of the local VLAN is deployed on each leaf node, and the layer-3 network of the VLAN of the Cross-leaf node is locked on the spine node;

Layer-3 forwarding traffic between local VLANs of the same leaf is shown in blue lines in 9. forwarding between local VLANs of different leaf nodes needs to be transferred through intercommunication VLAN, as shown in red lines in 9;

The local VLAN of the Leaf node and the VLAN of the Cross-leaf node need to be forwarded to the spine node, as shown in the pink line in 9;

This networking model moves the layer-3 interface of the leaf local VLAN originally forwarded by the spine node to the leaf node, which reduces the burden on the spine node and the ARP table entry requirements. However, the configuration is relatively complex, you need to introduce an Internet VLAN to implement route accessibility for each leaf node.

Iv. Conclusion

Based on the analysis of the advantages and disadvantages of the above networking models, we can see that TRILL has obvious advantages over traditional STP technology in terms of meeting the needs of large L2 expansion, Link Utilization, and stability and reliability.

The technology is constantly evolving, and TRILL is also constantly evolving. This includes both the improvement of the Protocol and the continuous improvement of ASIC functions. The current TRILL solution is not perfect, and there is room for improvement and optimization. We expect a more perfect TRILL Networking Solution in the future to better meet user needs.