Neutron Understanding (9): How OpenStack implements the Neutron network and the Nova VM firewall [how Nova Implements Security Group and how Neutron Implements Vi Rtual Firewall]

Learn Neutron Series articles:

(1) Virtualization network implemented by Neutron

(2) Neutron Openvswitch + VLAN Virtual network

(3) Neutron Openvswitch + gre/vxlan Virtual network

(4) Neutron OVS OpenFlow flow table and L2 Population

(5) Neutron DHCP Agent

(6) Neutron L3 Agent

(7) Neutron LBaas

(8) Neutron Security Group

(9) Neutron FWaas and Nova Security Group

(Ten) Neutron Vpnaas

The basics of this article and the techniques and implementations used and the Neutron Understanding (8): How Neutron implements a virtual machine firewall is very similar to [how Netruon Implements Security Group], so it saves the same part.

1. Nova Security Group 1.1 configuration

Node	Configuration file	Configuration Items	Description
Controller	/etc/nova/nova.conf	Security_group_api = Nova	Yes, the Nova secgroup* command is using the Nova security group API.
	/etc/neutron/plugins/ml2/ml2_conf.ini	Enable_security_group = False	Prohibit Neutron security group
Nova-compute	/etc/nova/nova.conf /etc/nova/nova-compute.conf	Firewall_driver = Nova.virt.firewall.IptablesFirewallDriver	Specifies the driver for the Nova security group, either Iptablesfirewalldriver or Nwfilterfirewall. The default is Iptablesfirewalldriver. See below for instructions.
	/etc/neutron/plugins/ml2/ml2_conf.ini	Enable_security_group = False	Prohibit Neutron security group
Network	/etc/neutron/plugins/ml2/ml2_conf.ini	Enable_security_group = False	Prohibit Neutron security group

Nova offers two implementations: using the Libvirt ' s nwfilter implementation and using the Linux iptables implementation, the default is to use the Linux iptables. You can specify the value by setting the Firewall_driver of the configuration item. It is important to note that some nwfilter functions are still used even with iptables. See https://ask.openstack.org/en/question/19456/how-security-group-is-implemented/

• Firewall_driver=nova.virt.libvirt.firewall.iptablesfirewalldriver
• Firewall_driver=nova.virt.libvirt.firewall.nwfilterfirewall

1.2 CLI

List security groups:
[Email protected]:~$ nova secgroup-list-rules novasg1+-------------+-----------+---------+-----------+------------ --+| IP Protocol | From Port | to Port | IP Range  | Source Group |+-------------+-----------+---------+-----------+--------------+| TCP | |        0.0.0.0/0 |              | +-------------+-----------+---------+-----------+--------------+
Create a security group rule:

[Email protected]:~$ nova secgroup-add-rule novasg1 UDP 53 53 100.1.100.0/24
+-------------+-----------+---------+----------------+--------------+
| IP Protocol | From Port | to Port | IP Range | Source Group |
+-------------+-----------+---------+----------------+--------------+
| UDP | 53 | 53 | 100.1.100.0/24 | |
+-------------+-----------+---------+----------------+--------------+

To delete a virtual machine's security group:

[Email protected]:~$ nova Remove-secgroup 2c59a875-bc23-4605-ad70-5315d7a3f8e2 NOVASG1

To add a security group to a virtual machine:

[Email protected]:~$ nova Add-secgroup 2c59a875-bc23-4605-ad70-5315d7a3f8e2 NOVASG1

Create a second security group:

[Email protected]:~$ Nova Secgroup-add-rule NOVASG2

To add a rule:

[Email protected]:~$ nova secgroup-add-rule novasg2 icmp-1-1 0.0.0.0/0
+-------------+-----------+---------+-----------+--------------+
| IP Protocol | From Port | to Port | IP Range | Source Group |
+-------------+-----------+---------+-----------+--------------+
| ICMP | -1 | -1 | 0.0.0.0/0 | |
+-------------+-----------+---------+-----------+--------------+

Add the security group to the virtual machine again:

[Email protected]:~$ nova Add-secgroup 2c59a875-bc23-4605-ad70-5315d7a3f8e2 NOVASG2

1.3 Iptables Chain

Nova-compute adds the input,output and FORWARD chains to the filter table:

-N Nova-compute-forward
-N Nova-compute-input
-N Nova-compute-output
-N nova-compute-inst-122#每个虚机一个链, the naming convention is "Inst"-<instance in the database id>
-N nova-compute-local
-N Nova-compute-provider
-N Nova-compute-sg-fallback
-N Nova-filter-top
-A input-j Nova-compute-input
-A forward-j Nova-filter-top
-A forward-j Nova-compute-forward
-A output-j Nova-filter-top
-A output-j Nova-compute-output
-A nova-compute-forward-s 0.0.0.0/32-d 255.255.255.255/32-p udp-m UDP--sport,--dport 67-j ACCEPT#允许本机上的虚机发出 DHCP Broadcast
-A nova-compute-input-s 0.0.0.0/32-d 255.255.255.255/32-p udp-m UDP--sport,--dport 67-j ACCEPT#允许本机接受 DHCP Broadcast packets
-A nova-compute-inst-122-m state--state invalid-j DROP
-A nova-compute-inst-122-m state--state related,established-j ACCEPT
-A nova-compute-inst-122-j Nova-compute-provider
-A nova-compute-inst-122-s 91.1.180.2/32-p udp-m UDP--sport--dport 68-j ACCEPT#接受该虚机所在子网的 The package returned by DHCP Server
-A nova-compute-inst-122-s 91.1.180.0/24-j ACCEPT#在配置项 allow_same_net_traffic = True to accept the same network segment virtual machine's visiting package
-A nova-compute-inst-122-p tcp-m tcp--dport 22-j ACCEPT#用户安全组规则指定的来访包
-A nova-compute-inst-122-s 100.1.100.0/24-p udp-m UDP--dport 53-j ACCEPT#用户安全组规则指定的来访包
-A nova-compute-inst-122-p icmp-j ACCEPT#用户安全组规则指定的来防爆
-A nova-compute-inst-122-j Nova-compute-sg-fallback#没被上面规则处理的其它来访包
-A nova-compute-local-d 91.1.180.14/32-j nova-compute-inst-122# "-D" determines that the Nova security group checks only the network packets that enter the virtual machine
-A nova-compute-sg-fallback-j DROP#丢弃其它包, only network access specified by the above rules is allowed
-A nova-filter-top-j nova-compute-local

2. FWaas2.1 Concept

Starting with the Havana version, Neutron provides a network four-layer firewall virtualization reference implementation Firewall-as-a-service, referred to as FWaas, based on the Neutron L3 Agent. The analysis in this paper is based on the OpenStack Juno version. In the Juno version, FWaas is a sub-tenant, but can be shared among multiple tenants. Only one firewall is allowed per tenant. Similar to a physical firewall, FWaas has three main concepts:

(1) Rule: Allows the user to specify the name to match, description, protocol (TCP, UDP, ICMP, any), behavior (allow,deny), source/destination IP address/subnet and port number/port number range.

The difference from the rules in the Neutron security group is that there is a need to specify that the processing behavior of the packets to be matched is through (allow) and not passed (DENY), but the network direction cannot be specified. The FWaas applies the rules to network packets that go into and out of the network at the same time.

(2) Policy: A logical collection of rules. The Policy can be shared and audited (audited). Currently, FWaas just saves "audited" to DB, and does not do anything with it.

(3) Firewall (Firewall): A logical collection of policies. See above on the right. In the Juno version, each tenant can have up to one Firewall. The firewall can be shared.

What needs to be explained here is the difference between the FWaas and security group. Security group rules are implemented on Linux Bridge QBR on compute nodes connected to an instance, and firewall rules created by FWaaS are implemented on virtual routers implemented at the edge of the tenant network. FWaaS is not intended to replace the functionality of a security group, and it provides a more supplemental security group, especially in its current implementation state. FWaaS currently lacks some of the functionality provided by security groups, including the inability to specify the direction of communication. In contrast, security groups also require FWaas as a supplement because they lack the ability to create specific rules to deny all traffic.

2.2 Configuration

Node	Configuration and operation
On the control node	Modify/etc/neutron/neutron.conf:[default]service_plugins = Router,lbaas,firewall[service_providers]service_provider = FIREWALL:Iptables:neutron.agent.linux.iptables_firewall. Ovshybridiptablesfirewalldriver:default Service Neutron-server Restart modify/usr/share/openstack-dashboard/openstack_ dashboard/local/local_settings.py: ' Enable_firewall ': True
On the network node	Modify/etc/neutron/fwaas_driver.ini:[fwaas] Driver = Neutron.services.firewall.drivers.linux.iptables_fwaas. Iptablesfwaasdriver Enabled = True Service neutron-l3-agent restart

2.3 Implementation

In the current implementation, FWaas is controlling the network packets to and from the data network by adding iptbales rules to all Virtual Router in its tenant.

Code implementation:

On the control node (class Firewallplugin):

(1) Create rule: A pure database operation that saves the rule to the data.

(2) Create policy: Pure database operation, save policy to data.

(3) If rule is added to a policy that has already been added to Firwall, or if a policy is added to an existing firewall, after the database operation, the RPC fanout to all L3 Agent host:

{' args ': {' firewall ': {' status ': ' Pending_update ', ' name ': U ' fw-for-tcp ', ' Shared ': None, ' firewall_policy_id ': U ' D14e23a3-2ee6-411d-b678-e6db3dac45f5 ', ' tenant_id ': U ' 74c8ada23a3449f888d9e19b76d13aab ', ' admin_state_up ': True, ' Id ': U ' aa85bd66-dc4c-4d1b-909e-6f5736c279c7 ', ' firewall_rule_list ': [{' Protocol ': U ' ICMP ', ' description ': U ', ' Source_port ': None, ' source_ip_address ': None, ' destination_ip_address ': None, ' firewall_policy_id ': U ' D14e23a3-2ee6-411d-b678-e6db3dac45f5 ', ' position ': 1, ' destination_port ': None, ' id ': U ' 8658229d-6e34-4069-b091-e560f9e54dc9 ', ' name ': U ' rule-allow-icmp ', ' tenant_id ': U ' 74c8ada23a3449f888d9e19b76d13aab ', ' enabled ': True, ' action ': U ' Allow ', ' ip_version ': 4L, ' Shared ': False}, {' Protocol ': U ' tcp ', ' description ': U ', ' sour Ce_port ': None, ' source_ip_address ': None, ' destination_ip_address ': None, ' firewall_policy_id ': U ' D14e23a3-2ee6-411d-b678-e6db3dac45f5 ', ' position ': 2, ' destination_port ': ' + ', ' id ': U ' 00b5bad2-dd14-48d6-9a5c-7b65e6e8c480 ', ' name ': U ' fule-allow-tcp-80 ', ' tenant_id ': U ' 74c8ada23a3449f888d9e19b76d13aab ', ' enabled ': True, ' action ': U ' Allow ', ' ip_ Version ': 4L, ' Shared ': False}], ' description ': U '}, ' host ': ' Controller '}, ' namespace ': None, ' method ': ' Update_firewal L '}

• Insert_rule/remove_rule/update_firewall_rule/update_firewall_policy-> firewall_db_mixin.insert _rule/remove_rule/update_firewall_rule/update_firewall_policy-> _rpc_update_firewall_policy->  _rpc_update_firewall-(if policy has a firewall) Firewallagentapi.update_firewall
• Create_firewall->& nbsp Firewall_db_mixin.create_firewall-> firewallagentapi.create_firewall
• update_firewall - > firewall_db_mixin.update_firewall, Firewallagentapi.update_firewall
• delete_firewall  -> firewall_db_mixin.delete_firewall, Firewallagentapi.delete_firewall
• Firewallagentapi.create/update/delete_firewall-> fanout_cast ("Create/update/delete_firewall", topics. L3_agent, "Controller", firewall)-----> fwaasl3agentrpccallback.create/update/delete_firewall->  Fwaasl3agentrpccallback._invoke_driver_for_plugin_api

On the network node (class Fwaasl3agentrpccallback):

(1) Get all the router through RPC, and get the routers on the tenant where firewall is located

(2) Call Iptablesfwaasdriver.update_firewall, update the iptables rule for each router in turn

(3) First delete the existing rule and then regenerate the rule according to the configured rules

[Email protected]:/var/cache# IP netns exec qrouter-e438bebe-6795-4b68-a613-ec0df38d3064 iptables-t Filter-S-P INPUT accept-P FORWARD accept-P OUTPUT accept-n neutron-filter-Top-n neutron-l3-agent-Forward-n neutron-l3-agent-Input-n neutron-l3-agent-Output-n Neutron-l3-agent-fwaas-defau#新增的 Firewall Chain-N Neutron-l3-agent-iv4aa85bd66#新增的 Firewall Chain-N neutron-l3-agent-Local-n Neutron-l3-agent-ov4aa85bd66#for Firewall-A input-j neutron-l3-agent-input-a forward-j neutron-filter-Top-a forward-j Neutron-l3-agent-forward#将 forward go to Neutron's chain-A output-j neutron-filter-Top-a output-j neutron-l3-agent-Output-a neutron-filter-top-j neutron-l3-agent-Local-a neutron-l3-agent-forward-o qr-+-j neutron-l3-agent-iv4aa85bd66#进数据网络的包-A neutron-l3-agent-forward-i qr-+-j neutron-l3-agent-ov4aa85bd66#出数据网络的包
-A neutron-l3-agent-forward-o qr-+-j Neutron-l3-agent-fwaas-defau#进数据网络的包的默认处理 Chain-A neutron-l3-agent-forward-i qr-+-j Neutron-l3-agent-fwaas-defau#出数据网络的包的默认处理 Chain-A neutron-l3-agent-input-d 127.0.0.1/32-p tcp-m tcp--dport 9697-J accept-a neutron-l3-agent-fwaas-defau-j DROP #默认丢弃没有被以上规则处理的所有包
-A neutron-l3-agent-iv4aa85bd66-m state--state invalid-j drop-a neutron-l3-agent-iv4aa85bd66-m State--state RELATED,E Stablished-j ACCEPT #接受状态为 Related, established (established connection) package -A neutron-l3-agent-iv4aa85bd66-p tcp-m TCP--dport 80-j Accept #根据定义的 FWaas rule, accepts TCP packets with destination port 80
-A neutron-l3-agent-ov4aa85bd66-m state--state invalid-j drop-a neutron-l3-agent-ov4aa85bd66-m State--state RELATED,E Stablished-j accept-a neutron-l3-agent-ov4aa85bd66-p tcp-m TCP--dport 80-j accept #根据 FWaas rule, receiving TCP with destination port 80 Package 

Neutron Understanding (9): How OpenStack implements the Neutron network and the Nova VM firewall [how Nova Implements Security Group and how Neutron Implements Vi Rtual Firewall]