Linux 3.10 kernel bridge forwarding Logic
Linux 3.10 kernel bridge forwarding logic-lvyilong316
I have previously analyzed the bridge forwarding logic of linux kernel 2.6.32. Next I will analyze the bridge forwarding logic of linux kernel 3.10. This is exactly the kernel corresponding to CentOS 5 and CentOS 7. 3.10 The greatest change in the bridge logic in the kernel is the addition of vlan processing logic and brdige entry function settings.
1. netdev_rx_handler_register
Before analysis, we should first introduce an important function: netdev_rx_handler_register, which is not available in the 2.6 kernel.
Lnetdev_rx_handler_register
- /*
- * Dev: register the dev of the receiving function.
- * Rx_handler: The receiving function to be registered
- * Rx_handler_data: indicates the data used by rx_handler_data.
- */
- Int netdev_rx_handler_register (struct net_device * dev,
- Rx_handler_func_t * rx_handler,
- Void * rx_handler_data)
- {
- ASSERT_RTNL ();
- If (dev-> rx_handler)
- Return-EBUSY;
- /* Note: rx_handler_data must be set before rx_handler */
- Rcu_assign_pointer (dev-> rx_handler_data, rx_handler_data );
- Rcu_assign_pointer (dev-> rx_handler, rx_handler );
- Return 0;
- }
This function registers the receiving function for the device (net_device), and then calls the registered receiving function based on the device interface that receives the skb In the _ netif_receive_skb function. For example, register the br_handle_frame function for the interface under the bridge, and register the bond_handle_frame function for the bonding interface. This is more flexible than the old-fashioned bridge processing. With this mechanism, you can also register processing functions in the module. For example, openvswitch in 3.10 (OpenvSwitch has already been integrated into the kernel in 3.10) creates the netdev_create function of netdev vport.
Lnetdev_create
- static struct vport *netdev_create(const struct vport_parms *parms)
- {
- struct vport *vport;
- /....../
- err = netdev_rx_handler_register(netdev_vport->dev, netdev_frame_hook,vport);
- /....../
- }
This function sets the reception function of the device as the netdev_frame_hook function when creating the netdev vport, which is also the entry function of the entire openvswitch, if you check the OpenvSwitch source code, you can see that when installed in the 2.6 kernel, The br_handle_frame_hook function of bridge is replaced here, and the bridge logic enters the OpenvSwitch logic.
2. Bridge forwarding Logic Analysis
Analyze the netif_receive_skb function first. This function is the entry to the protocol stack.
Lnetif_receive_skb
- Int netif_receive_skb (struct sk_buff * skb)
- {
- Int ret;
- If (skb_defer_rx_timestamp (skb ))
- Return NET_RX_SUCCESS;
- Rcu_read_lock ();
- /* RPS logic processing. Now the RPS mechanism is used in the kernel to distribute packets to the receiving queues of various CPUs for load balancing */
- # Ifdef CONFIG_RPS
- If (static_key_false (& rps_needed )){
- Struct rps_dev_flow voidflow, * rflow = & voidflow;
- Int cpu = get_rps_cpu (skb-> dev, skb, & rflow );
- If (cpu> = 0 ){
- Ret = enqueue_to_backlog (skb, cpu, & rflow-> last_qtail );
- Rcu_read_unlock ();
- Return ret;
- }
- }
- # Endif
- Ret = _ netif_receive_skb (skb );
- Rcu_read_unlock ();
- Return ret;
- }
Netif_receive_skb only processes the data packet RPS and then calls _ netif_receive_skb.
_ Netif_receive_skb does not have any additional processing logic. It mainly calls _ netif_receive_skb_core, which is actually equivalent to netif_receive_skb of the 2.6 kernel. The following code omits the logic unrelated to bridge.
L _ netif_receive_skb_core
- Static int _ netif_receive_skb_core (struct sk_buff * skb, bool pfmemalloc)
- {
- Struct packet_type * ptype, * pt_prev;
- Rx_handler_func_t * rx_handler;
- Struct net_device * orig_dev;
- Struct net_device * null_or_dev;
- Bool deliver_exact = false;
- Int ret = NET_RX_DROP;
- _ Be16 type;
- /*......*/
- Orig_dev = skb-> dev;
- Skb_reset_network_header (skb );
- Pt_prev = NULL;
- Skb-> skb_iif = skb-> dev-> ifindex;
- /* Ptype_all protocol processing, tcpdump packet capture here */
- List_for_each_entry_rcu (ptype, & ptype_all, list ){
- If (! Ptype-> dev | ptype-> dev = skb-> dev ){
- If (pt_prev)
- Ret = deliver_skb (skb, pt_prev, orig_dev );
- Pt_prev = ptype;
- }
- }
- /* Call the rx_handler of the receiving device */
- Rx_handler = rcu_dereference (skb-> dev-> rx_handler );
- If (rx_handler ){
- If (pt_prev ){
- Ret = deliver_skb (skb, pt_prev, orig_dev );
- Pt_prev = NULL;
- }
- Switch (rx_handler (& skb )){
- Case RX_HANDLER_CONSUMED:
- Ret = NET_RX_SUCCESS;
- Goto out;
- Case RX_HANDLER_ANOTHER:
- Goto another_round;
- Case RX_HANDLER_EXACT:
- Deliver_exact = true;
- Case RX_HANDLER_PASS:
- Break;
- Default:
- BUG ();
- }
- }
- /* Pass to the upper-layer protocol according to skb-> protocol */
- Type = skb-> protocol;
- List_for_each_entry_rcu (ptype, & ptype_base [ntohs (type) & PTYPE_HASH_MASK], list ){
- If (ptype-> type = type & (ptype-> dev = null_or_dev | ptype-> dev = skb-> dev | ptype-> dev = orig_dev )) {
- If (pt_prev)
- Ret = deliver_skb (skb, pt_prev, orig_dev );
- Pt_prev = ptype;
- }
- }
- If (pt_prev ){
- If (unlikely (skb_orphan_frags (skb, GFP_ATOMIC )))
- Goto drop;
- Else
- Ret = pt_prev-> func (skb, skb-> dev, pt_prev, orig_dev );
- } Else {
- Drop:
- Atomic_long_inc (& skb-> dev-> rx_dropped );
- Kfree_skb (skb );
- Ret = NET_RX_DROP;
- }
- Out:
- Return ret;
- }
If a dev is added to a bridge (as an interface of the bridge), The rx_handler of this interface device is set to the br_handle_frame function, which is set in the br_add_if function, br_add_if (net/bridge/br_if.c) is set when an interface is added to the bridge device. Entering br_handle_frame also enters the bridge logic code.
Lbr_add_if
- int br_add_if(struct net_bridge *br, struct net_device *dev)
- {
- /*......*/
- err = netdev_rx_handler_register(dev, br_handle_frame, p);
- /*......*/
- }
Lbr_handle_frame
- Rx_handler_result_t br_handle_frame (struct sk_buff ** pskb)
- {
- Struct net_bridge_port * p;
- Struct sk_buff * skb = * pskb;
- Const unsigned char * dest = eth_hdr (skb)-> h_dest;
- Br_should_route_hook_t * fig;
- If (unlikely (skb-> pkt_type = PACKET_LOOPBACK ))
- Return RX_HANDLER_PASS;
- If (! Is_valid_ether_addr (eth_hdr (skb)-> h_source ))
- Goto drop;
- Skb = skb_share_check (skb, GFP_ATOMIC );
- If (! Skb)
- Return RX_HANDLER_CONSUMED;
- /* Get the bridge port corresponding to dev */
- P = br_port_get_rcu (skb-> dev );
- /* Special target mac address processing */
- If (unlikely (is_link_local_ether_addr (dest ))){
- /*
- * See IEEE 802.1D Table 7-10 Reserved addresses
- *
- * Assignment Value
- * Bridge Group Address 01-80-C2-00-00-00
- * (MAC Control) 802.3 01-80-C2-00-00-01
- * (Link Aggregation) 802.3 01-80-C2-00-00-02
- * 802.1X PAE address 01-80-C2-00-00-03
- *
- * 802.1AB LLDP 01-80-C2-00-00-0E
- *
- * Others reserved for future standardization
- */
- Switch (dest [5]) {
- Case 0x00:/* Bridge Group Address */
- /* If STP is turned off, then must forward to keep loop detection */
- If (p-> br-> stp_enabled = BR_NO_STP)
- Goto forward;
- Break;
- Case 0x01:/* ieee mac (Pause )*/
- Goto drop;
- Default:
- /* Allow selective forwarding for most other protocols */
- If (p-> br-> group_fwd_mask & (1u <dest [5])
- Goto forward;
- }
- /* LOCAL_IN hook Point, note that after this hook point does not mean to be sent to the host protocol stack (only the special purpose mac01-80-C2 will come here )*/
- If (NF_HOOK (NFPROTO_BRIDGE, NF_BR_LOCAL_IN, skb, skb-> dev,
- NULL, br_handle_local_finish )){
- Return RX_HANDLER_CONSUMED;/* consumed by filter */
- } Else {
- * Pskb = skb;
- Return RX_HANDLER_PASS;/* continue processing */
- }
- }
- /* Forwarding logic */
- Forward:
- Switch (p-> state ){
- Case BR_STATE_FORWARDING:
- Fig = rcu_dereference (br_should_route_hook );
- If (FIG ){
- If (* fig) (skb )){
- * Pskb = skb;
- Return RX_HANDLER_PASS;
- }
- Dest = eth_hdr (skb)-> h_dest;
- }
- /* Fall through */
- Case BR_STATE_LEARNING:
- /* If the target mac of the skb is the same as that of the mac of the bridge, the skb is sent to the local protocol stack */
- If (ether_addr_equal (p-> br-> dev-> dev_addr, dest ))
- Skb-> pkt_type = PACKET_HOST;
- /* NF_BR_PRE_ROUTING hook Point */
- NF_HOOK (NFPROTO_BRIDGE, NF_BR_PRE_ROUTING, skb, skb-> dev, NULL, br_handle_frame_finish );
- Break;
- Default:
- Drop:
- Kfree_skb (skb );
- }
- Return RX_HANDLER_CONSUMED;
- }
After the NF_BR_LOCAL_IN hook point, the br_handle_local_finish function is executed.
Lbr_handle_local_finish
- Static int br_handle_local_finish (struct sk_buff * skb)
- {
- Struct net_bridge_port * p = br_port_get_rcu (skb-> dev );
- 2010vid = 0;
- /* Obtain the skb vlan id (3.10 of the bridge Support vlan )*/
- Br_vlan_get_tag (skb, & vid );
- /* Update the mac table of bridge. Note that the vlan id is also a parameter, indicating that each vlan has an independent mac table */
- Br_fdb_update (p-> br, p, eth_hdr (skb)-> h_source, vid );
- Return 0;/* process further */
- }
The br_handle_frame_finish function is executed after the NF_BR_PRE_ROUTING hook Point.
Lbr_handle_frame_finish
- Int br_handle_frame_finish (struct sk_buff * skb)
- {
- Const unsigned char * dest = eth_hdr (skb)-> h_dest;
- Struct net_bridge_port * p = br_port_get_rcu (skb-> dev );
- Struct net_bridge * br;
- Struct net_bridge_fdb_entry * dst;
- Struct net_bridge_mdb_entry * mdst;
- Struct sk_buff * skb2;
- 2010vid = 0;
- If (! P | p-> state = BR_STATE_DISABLED)
- Goto drop;
- /* This check mainly involves vlan-related checks, for example, whether the vlan is the same as the vlan configured for the receiving interface */
- If (! Br_allowed_ingress (p-> br, nbp_get_vlan_info (p), skb, & vid ))
- Goto out;
- /* Insert into forwarding database after filtering to avoid spoofing */
- Br = p-> br;
- /* Update the forwarding database */
- Br_fdb_update (br, p, eth_hdr (skb)-> h_source, vid );
- /* Multicast mac Processing */
- If (! Is_broadcast_ether_addr (dest) & is_multicast_ether_addr (dest )&&
- Br_multicast_rcv (br, p, skb ))
- Goto drop;
- If (p-> state = BR_STATE_LEARNING)
- Goto drop;
- BR_INPUT_SKB_CB (skb)-> brdev = br-> dev;
- /* The packet skb2 goes to the local host (NULL to skip ).*/
- Skb2 = NULL;
- /* If the bridge is set to the hybrid mode */
- If (br-> dev-> flags & IFF_PROMISC)
- Skb2 = skb;
- Dst = NULL;
- /* If the target mac of skb is broadcast */
- If (is_broadcast_ether_addr (dest ))
- Skb2 = skb;
- Else if (is_multicast_ether_addr (dest) {/* multicast */
- Mdst = br_mdb_get (br, skb, vid );
- If (mdst | BR_INPUT_SKB_CB_MROUTERS_ONLY (skb )){
- If (mdst & mdst-> mglist) |
- Br_multicast_is_router (br ))
- Skb2 = skb;
- Br_multicast_forward (mdst, skb, skb2 );
- Skb = NULL;
- If (! Skb2)
- Goto out;
- } Else
- Skb2 = skb;
- Br-> dev-> stats. multicast ++;
- } Else if (dst = _ br_fdb_get (br, dest, vid) & dst-> is_local) {/* the destination address is the mac address of the local machine, then it is sent to the local protocol stack */
- Skb2 = skb;
- /* Do not forward the packet since it's local .*/
- Skb = NULL;
- }
- If (skb ){
- If (dst ){
- Dst-> used = jiffies;
- Br_forward (dst-> dst, skb, skb2); // forward it to the Target Interface
- } Else
- Br_flood_forward (br, skb, skb2); // broadcast if the target interface is not found
- }
- If (skb2)
- Return br_pass_frame_up (skb2); // send it to the local protocol stack
- Out:
- Return 0;
- Drop:
- Kfree_skb (skb );
- Goto out;
- }
Let's first look at the br_pass_frame_up function sent to the local protocol stack.
Lbr_pass_frame_up
- Static int br_pass_frame_up (struct sk_buff * skb)
- {
- Struct net_device * indev, * brdev = BR_INPUT_SKB_CB (skb)-> brdev;
- Struct net_bridge * br = netdev_priv (brdev );
- // Update the statistics (omitted)
- /* Bridge is just like any other port. Make sure
- * Packet is allowed within T in promisc modue when someone
- * May be running packet capture.
- */
- If (! (Brdev-> flags & IFF_PROMISC )&&! Br_allowed_egress (br, br_get_vlan_info (br), skb )){
- Kfree_skb (skb); // if it is not in hybrid mode and the vlan processing is not required, it is discarded.
- Return NET_RX_DROP;
- }
- // Vlan processing logic
- Skb = br_handle_vlan (br, br_get_vlan_info (br), skb );
- If (! Skb)
- Return NET_RX_DROP;
- Indev = skb-> dev;
- Skb-> dev = brdev; // key point. Here, skb-> dev is changed to bridge.
- // Go to the protocol stack again after NF_BR_LOCAL_IN
- Return NF_HOOK (NFPROTO_BRIDGE, NF_BR_LOCAL_IN, skb, indev, NULL,
- Netif_receive_skb );
- }
Enter netif_receive_skb again, because skb-dev is set to bridge, and the rx_handler function of the bridge device is not set, so it will not enter the bridge logic again, directly enters the upper-layer protocol stack of the host.
The forwarding logic is mainly used in the br_forward function, while br_forward mainly calls the _ br_forward function.
L _ br_forward
- Static void _ br_forward (const struct net_bridge_port * to, struct sk_buff * skb)
- {
- Struct net_device * indev;
- // Vlan Processing
- Skb = br_handle_vlan (to-> br, nbp_get_vlan_info (to), skb );
- If (! Skb)
- Return;
- Indev = skb-> dev;
- Skb-> dev = to-> dev; // skb-> dev is set as the exit device dev
- Skb_forward_csum (skb );
- // Call br_forward_finish after the NF_BR_FORWARD hook Point
- NF_HOOK (NFPROTO_BRIDGE, NF_BR_FORWARD, skb, indev, skb-> dev,
- Br_forward_finish );
- }
Lbr_forward_finish
- Int br_forward_finish (struct sk_buff * skb)
- {
- // Call br_dev_queue_push_xmit after the NF_BR_POST_ROUTING hook Point
- Return NF_HOOK (NFPROTO_BRIDGE, NF_BR_POST_ROUTING, skb, NULL, skb-> dev, br_dev_queue_push_xmit );
- }
Lbr_dev_queue_push_xmit
- Int br_dev_queue_push_xmit (struct sk_buff * skb)
- {
- /* Ip_fragment doesn't copy the MAC header */
- If (nf_bridge_maybe_copy_header (skb) | (packet_length (skb)> skb-> dev-> mtu &&! Skb_is_gso (skb ))){
- Kfree_skb (skb );
- } Else {
- Skb_push (skb, ETH_HLEN );
- Br_drop_fake_rtable (skb );
- Dev_queue_xmit (skb); // send it to the Link Layer
- }
- Return 0;
- }
When Skb enters dev_queue_xmit, it calls the sending function of the corresponding device driver. That is, the bridge logic. As a result, the bridge forwarding logic of 3.10kernel is shown in:
Note: Like 2.6kernel, bridge OUTPUT hook points are in the send function of bridge dev, which is not analyzed and listed here.