What is the control plane/data plane/network topology

What is the control plane?

In network routing, the control plane is the part of the router architecture that is concerned with drawing the network topology, or the information in a routing table that defines what to do with incoming packets. Control plane functions, such as participating in routing protocols, run in the architectural control element. In most cases, the routing table contains a list of destination addresses and the outgoing interface(s) associated with each. Control plane logic also can identify certain packets to be discarded, as well as preferential treatment of certain packets for which a high quality of service is defined by such mechanisms as differentiated services.

What is the data plane? 

The data plane (or forwarding plane) is the high speed path through the router/switch. Packets that pass through the device use the data plane, as opposed to packets directed to the device. For this reason, the data plane is also called the forwarding plane.

What is network topology?

Network topology refers to the way data flows in a network. The control plane establishes and changes network topology. Again, think of the stoplights that function at the intersections of a city. Network topology is like the way that the roads are arranged, and the computing devices within the network are like the destinations that those roads lead to.

Solving the problem of scale mismatch

The biggest challenge with this architecture is scale mismatch. The control plane fleet is badly outnumbered by the data plane fleet. We looked to our storage services for help. When it comes to serving content at scale. Instead of exposing APIs directly to the data plane, the control plane can periodically write updated configuration into an bucket. Data plane servers then poll this bucket for updated configuration and cache it locally. Similarly, to stay up to date on the data plane’s operational state, the control plane can poll an bucket into which data plane servers periodically write that information. This architecture is illustrated in the following diagram.
This architecture has several advantages. It’s simple to implement, and is scaled to support even the largest client fleets. Additionally, as the size of the data plane fleet grows, the control plane fleet can still stay relatively small. And if the control plane has an outage, the data plane can continue running with the last known configuration, even as servers come in or out of service. This property, called static stability, is a desirable attribute in distributed systems.

An example of a system using this architecture is Hyperplane, the internal network function virtualization system behind services and resources like Network Load Balancer, NAT Gateway, and PrivateLink. The Hyperplane data plane contains devices that process customer traffic, and that need to know about configuration of individual Network Load Balancers, NAT Gateways, and PrivateLink connections. A periodic task within the Hyperplane control plane scans its DynamoDB tables containing customer configuration and writes that configuration into several files. The data plane then periodically downloads these files and uses their content to update internal routing configuration.


In other systems, it’s important for changes in the control plane configuration to be reflected in the data plane in single digit seconds or faster. An example of this situation is a container service that notifies a server about a new container it needs to run. In such systems, polling periodically updated files can make it impossible to achieve acceptable propagation latencies.

When faced with such scenarios, we look for other approaches where the small fleet can be in control of the pace at which requests flow through the system. One such approach is to reverse the flow of API calls and have the smaller control plane fleet push configuration changes to the larger data plane fleet. This architecture is illustrated in the following diagram.