FortiOS 6.0 VPN: VXLan over IPsec using VTEP

This scenario is intended for network engineers who are familiar with the FortiGate platform and are looking for an example FortiOS 6.0 configuration. It does not include all of the required configuration steps but the intention is to provide the information you need to implement VXLAN over IPsec.

Here we will show a very specific configuration, using Hub-And-Spoke Topology, however the same logic can be applied to static VPN. In this example,  DIALUP VPN is convenient in a Hub-and-Spoke topology as it uses a single phase 1 dialup definition on the hub FortiGates with additional spoke tunnels being added, without any changes to the hubs beyond that of adding additional user accounts for each additional spoke. 

1. IPsec VPN Configuration

1.1. Phase 1 & Phase 2

For details about Hub-and-Spoke configuration using DIALUP, you can refer to https://cookbook.fortinet.com/hub-and-spoke-vpn-using-quick-mode-selectors/ as it is the same configuration uses here. However for readability VPN configuration are show here:

FortiGate HUB:

config vpn ipsec phase1-interface
   edit "SPOKES"
      set type dynamic
      set interface "port2"
      set mode aggressive
      set peertype one
      set proposal aes256-sha256
      set xauthtype auto
      set authusrgrp "SPOKES"
      set peerid "SPOKES"
      set psksecret <SECRET>
   next
end

config vpn ipsec phase2-interface
   edit "SPOKES"
      set phase1name "SPOKES"
      set proposal aes128-sha1 aes256-sha1 aes128-sha256 aes256-sha256 aes128gcm aes256gcm chacha20poly1305
   next
end

FortiGate Spoke:

config vpn ipsec phase1-interface
   edit "HUB"
      set interface "port2"
      set mode aggressive
      set peertype any
      set proposal aes256-sha256
      set localid "SPOKES"
      set xauthtype client
      set authusr "SPOKE1"
      set authpasswd <SECRET>
      set remote-gw <HUB_PUBLIC_IP>
      set psksecret <SECRET>
   next
end

config vpn ipsec phase2-interface
   edit "HUB"
      set phase1name "HUB"
      set proposal aes128-sha1 aes256-sha1 aes128-sha256 aes256-sha256 aes128gcm aes256gcm chacha20poly1305
      set auto-negotiate enable
      set src-subnet 192.168.255.2 255.255.255.255
   next
end

Note: The hub FortiGates each insert a reverse route pointing to newly established tunnel interfaces, for any of the subnets provided by the spoke FortiGate’s source quick mode selectors. That is why we are setting the tunnel IP address here. 

1.2. IPsec VPN Policy

FortiGate Hub:

This policy allow VXLAN traffic between Spokes, as Spoke to Spoke traffic is done through Hub.

config firewall policy
   edit 1
      set name "VXLAN_SPOKE_to_SPOKE"
      set srcintf "SPOKES"
      set dstintf "SPOKES"
      set srcaddr "NET_192.168.255.0"
      set dstaddr "NET_192.168.255.0"
      set action accept
      set schedule "always"
      set service "UDP_4789"
      set logtraffic all
      set fsso disable
   next
end

FortiGate Spoke:

For the VPN IPsec to establish a policy refering to tunnel interface is needed. In this example, VXLAN tunnel is issued by FortiGate and ending at the remote site FortiGate’s tunnel interface, thus explicitely allowing VXLAN traffic is not needed That is why we can create such a policy:

config firewall policy
   edit 1
      set name "FICTIVE_IPSEC_POLICY"
      set srcintf "HUB"
      set dstintf "HUB"
      set srcaddr "none"
      set dstaddr "none"
      set action accept
      set schedule "always"
      set service "PING"
      set logtraffic disable
      set fsso disable
   next
end

1.3. IPsec Tunnel Interface

IPsec tunnel interface are used to support VXLan tunnel termination, thus we need to set IP address on them. Allowing PING is convenient for troubleshooting purposes.

FortiGate Hub:

config system interface
   edit "SPOKES"
      set vdom "root"
      set ip 192.168.255.1 255.255.255.255
      set allowaccess ping
      set type tunnel
      set remote-ip 192.168.255.254 255.255.255.0
      set snmp-index 12
      set interface "port2"
   next
end

Note: Remote IP is not used but needed in configuration. 

FortiGate Spoke:

config system interface
   edit "HUB"
      set vdom "root"
      set ip 192.168.255.2 255.255.255.255
      set allowaccess ping
      set type tunnel
      set remote-ip 192.168.255.1 255.255.255.0
      set snmp-index 12
      set interface "port2"
   next
end

2. VXLAN Interface

We need to create VXLan interface and bind it to ipsec tunnel one. All VXLan interface share the same VNI.

FortiGate Hub:

config system vxlan
   edit "SPOKES_VXLAN"
      set interface "SPOKES"
      set vni 1
      set remote-ip "192.168.255.2" "192.168.255.3"
   next
end

Notes: Remotes IPs are Spokes tunnel interfaces IP addresses.

FortiGate Spoke:

config system vxlan
   edit "HUB_VXLAN"
      set interface "HUB"
      set vni 1
      set remote-ip "192.168.255.1" "192.168.255.3"
   next
end

Note: Remote IP is the Hub tunnel interface IP. We could add other Spokes tunnel IPs to establish VXLan tunnel between Spokes (ex: 

set remote-ip "192.168.255.1" "192.168.255.3").

Note: To add more “remote IP” to a VXLAN interface, interface cannot be in use (ex: Policy). So you might want to provision future Spokes remotes IPs here at the beginning to avoid trafic disruption (otherwise you will have to delete the reference (ex: the policy) before adding remote ips).

3. Binding VXLan Interface to Ethernet port

VXlan encapsulates OSI layer 2 Ethernet frames within layer 3 IP packets that is why we have to bind internal port and VXLAN interface, devices behind port1 will have direct layer 2 access to remote peer over VXLAN tunnel. This can be done:

• using Switch Interface
• using Virtual Wire Pair

For this exemple we will use both methods:

Switch Interface – FortiGate Hub (can be created from GUI):

config system switch-interface
   edit "SW"
      set vdom "root"
      set member "port1" "SPOKES_VXLAN"
   next
end

Note: According to switch interface configuration, allowing intra switch traffic is implicitely allowed (default) or needs an explicit policy: 

set intra-switch-policy explicit

Virtual Wire Pair – FortiGate Spoke (can be created from GUI):

config system virtual-wire-pair
   edit "VWP"
      set member "HUB_VXLAN" "port1"
   next
end

Virtual-Wire-Pair needs explicit policy to allow traffic between interfaces:

4. Testing:

Ping From Spoke Lan to Hub:

user@pc-spoke1:~$ ping 192.168.1.1 -c 3
PING 192.168.1.1 (192.168.1.1) 56(84) bytes of data.
64 bytes from 192.168.1.1: icmp_seq=1 ttl=64 time=1.24 ms
64 bytes from 192.168.1.1: icmp_seq=2 ttl=64 time=0.672 ms
64 bytes from 192.168.1.1: icmp_seq=3 ttl=64 time=0.855 ms

--- 192.168.1.1 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2002ms
rtt min/avg/max/mdev = 0.672/0.923/1.243/0.239 ms

Sniffing traffic on Hub:

FGT-HUB # diagnose sniffer packet any 'icmp or (udp and port 4789)' 4 0 a
interfaces=[any]
filters=[icmp or (udp and port 4789)]
15:00:01.438230 SPOKES in 192.168.255.2.4790 -> 192.168.255.1.4789: udp 106

<<<< 1

15:00:01.438256 SPOKES_VXLAN in 192.168.1.2 -> 192.168.1.1: icmp: echo request

<<<< 2

15:00:01.438260 port1 out 192.168.1.2 -> 192.168.1.1: icmp: echo request

<<<< 3

15:00:01.438532 port1 in 192.168.1.1 -> 192.168.1.2: icmp: echo reply
15:00:01.438536 SPOKES_VXLAN out 192.168.1.1 -> 192.168.1.2: icmp: echo reply
15:00:01.438546 SPOKES out 192.168.255.1.4851 -> 192.168.255.2.4789: udp 106

NOTES:

1. VXLAN traffic (UDP/4789) entering at IPsec Tunnel Interface
2. Ingress ICMP Request to VXLan interface
3. Egress ICMP Request from port1