FortiOS 5.2 Expert Recipe: Hub-and-spoke VPN using quick mode selectors

In this expert cookbook article and an included example recipe, we will explore a scalable approach to setting up a large number of spoke VPNs by using quick mode selector source definitions on the spoke FortiGates and the dialup VPN configurations on the hub FortiGates.

We will also explore how redundant spoke VPN tunnels can be configured in order to offer maximum redundancy for environments with critical availability requirements. We will be authenticating the VPN tunnels using X-Auth in order to ensure separate credentials for each spoke.

This recipe is based on FortiOS firmware version 5.2, so some of the steps shown  may not be the same as with other versions of the firmware.

This topology consists of 2 hub networks and 2 spoke networks, using private IP ranges, separated by a simulated Internet, with 100.64.0.0/16 representing the Internet. Each FortiGate also has a loopback  interface that is routable across the VPN.

The diagram topology shows the VPN tunnels along with their redundant links:

The red dotted line showing the VPN tunnel connection between the  primary and backup data centers; in this case, our two hubs.

The blue dotted line showing the VPN tunnel connection between the primary datacenter and the branch offices; the spokes in the scenario.

The orange dotted line shows the VPN tunnel connection between the backup datacenter and the branch offices.

While the topology shown in the diagram can be built using individual static tunnels between each site, this would not scale well if addition spokes grow to a significant number. There would also be limited support for dynamically addressed sites. This strategy put forth by this article offers a solution to these issues by using a single phase 1 dialup definition on the hub FortiGates with additional spoke tunnels being added, without any changes to the hubs beyond that of adding additional user accounts for each additional spoke.

Spoke authentication is maintained by with X-Auth, which keeps the authentication of the individual tunnels separate in such a way that the use of a Pre-Shared Key alone is insufficient to authenticate a tunnel. A Public Key Infrastructure can also be used, provided that separate key-pairs are used for each VPN tunnel to maintain the segregation of the spokes.

The key points of this design are:

Each hub FortiGate is configured with a dialup interface-mode Phase1 using X-Auth.

Each spoke has its own user account on the hub FortiGates. In this example, local accounts are used on each hub, but a RADIUS or LDAP authentication server could be used on the back end, eliminating the need to managed the accounts on the FortiGates.

Spoke FortiGates are configured to propagate their local subnets using quick mode selectors (specifically, a source object).

When a new spoke tunnel is connected, the hub FortiGate validates the shared secret along with the X-Auth credentials provided by the spoke FortiGate.

Spokes FortiGates can have dynamically assigned IP addresses such as those given out by DSL or cable ISPs.

The hub FortiGates each insert a reverse route pointing to newly established tunnel interfaces, for any of the  subnets provided by the spoke FortiGate’s source quick mode selectors.

Each spoke FortiGate uses configured static routes  to direct traffic that needs to go to the datacenter(s) through the VPN tunnels destined for the hubs. The static route to the backup hub is set to a higher priority number value, making it the less preferred route. There is also an option where you can send all of your traffic from the spokes through the VPN tunnel by default. This can be done by configuring the WAN interface to route all traffic through the public IP address of the hub FortiGate. This is what our example configuration is set to do.

We need to aware of any potential points where asymmetrical routing could occur as it relates to traffic returning to the spokes (This is essentially the response to a request coming back through a different route than it took to get there). This can be a potential problem especially when communicating to hosts that are connected to both data centers and we happen to be redistributing spoke routes using a dynamic routing protocol with hub sites using OSI Layer 3 networking devices. In this case, we would ensure that the backup hub’s redistributed routes are less preferred than the primary hub’s routes. In all cases, it is important to have a clear view of the routing flows between each endpoint and to keep “diag debug flow” in our toolbox to diagnose those potential asymmetric routing issues. In our example, we would want to route traffic destined to resources in each respective hub directly to that hub, rather than have it cross the inter-datacenter VPN tunnel, and have default routing flow to the primary hub under normal circumstances.

The Hub FortiGates

Let’s look at the relevant configuration points of the hub FortiGates (These will be identical on each hub FortiGate:

While the GUI can be used for these steps, we are going to use the CLI to keep things simple and avoid potential confusion that may be caused by changes in the GUI’s layout.

Create the IPsec tunnel:

config vpn ipsec phase1-interface
   edit "SPOKES"
      set type dynamic
      set interface "port1"
      set mode aggressive
      set peertype one
      set proposal aes256-sha256
      set xauthtype auto
      set authusrgrp "SPOKE-GRP"
      set peerid "SPOKES"
      set psksecret SuperSecretSpokeSecret
   next
end
config vpn ipsec phase2-interface
   edit "SPOKES-P2"
      set phase1name "SPOKES"
      set proposal aes256-sha256
      set keepalive enable
   next
end

Create a user for each of the spokes:

config user local
   edit "SPOKE1"
      set type password
      set passwd Spoke1SuperSecret
   next
   edit "SPOKE2"
      set type password
      set passwd Spoke2SuperSecret
   next
end

Create a user group and include the spoke members:

config user group
   edit "SPOKE-GRP"
      set member "SPOKE1" "SPOKE2"
   next
end

Create the firewall policies

config firewall policy
   edit 1
      set srcintf "port2" "loop0"
      set dstintf "SPOKES"
      set srcaddr "all"
      set dstaddr "all"
      set action accept
      set schedule "always"
      set service "ALL"
   next
   edit 2
      set srcintf "SPOKES"
      set dstintf "port2" "loop0"
      set srcaddr "all"
      set dstaddr "all"
      set action accept
      set schedule "always"
      set service "ALL"
   next
end

A few of the above configuration aspects require further explanation:

• Aggressive mode: We are using this mode in order to ensure that these dialup spokes are terminated on the right dialup phase1. If the hub unit has other dialup phase1 (for FortiClient VPN users, for instance), the hub would otherwise be unable to distinguish between each dialup phase1.A few of the above configuration aspects require further explanation:

• X-Auth: As previously stated, this allows us to authenticate each connecting spoke unit to a local group, which is defined in the above configuration as currently containing two user accounts (our example has two spokes). Provisioning additional spokes on the hub would simply involve adding additional user accounts.

• Policies: As usual, we must always configure policies in order for traffic to flow. IPsec Phase1 follows a special rule in which tunnels will not even attempt to come up unless they have at least one policy referring to them (this happens to be a good trick to know when you want to disable an IPsec VPN tunnel without deleting its configuration).

• The Spoke FortiGates

With the hub FortiGates configured and ready for incoming connections, the spoke FortiGates can be configured. Below is the steps for configuring SPOKE1. To configure additional spoke FortiGates change the unit specific information.

Create the IPsec tunnel

config vpn ipsec phase1-interface
   edit "HUB-PRIMARY"
      set interface "port1"
      set mode aggressive
      set proposal aes256-sha256
      set localid "SPOKES"
      set xauthtype client
      set authusr "SPOKE1"
      set authpasswd Spoke1SuperSecret
      set mesh-selector-type subnet
      set remote-gw 100.64.10.2
      set psksecret SuperSecretSpokeSecret
   next
   edit "HUB-BACKUP"
      set interface "port1"
      set mode aggressive
      set proposal aes256-sha256
      set localid "SPOKES"
      set xauthtype client
      set authusr "SPOKE1"
      set authpasswd Spoke1SuperSecret
      set mesh-selector-type subnet
      set remote-gw 100.64.11.2
      set psksecret SuperSecretSpokeSecret
   next
end
config vpn ipsec phase2-interface
   edit "PRIMARY-P2"
      set phase1name "HUB-PRIMARY"
      set proposal aes256-sha256
      set keepalive enable
      set src-addr-type name
      set dst-addr-type name
      set src-name "VPN_SUBNETS"
      set dst-name "all"
   next
   edit "BACKUP-P2"
      set phase1name "HUB-BACKUP"
      set proposal aes256-sha256
      set keepalive enable
      set src-addr-type name
      set dst-addr-type name
      set src-name "VPN_SUBNETS"
      set dst-name "all"
   next
end

Creating addresses for the subnets

config firewall address
   edit "NET_192.168.12.0/24"
      set subnet 192.168.12.0 255.255.255.0
   next
   edit "NET_100.64.254.12/32"
      set subnet 100.64.254.12 255.255.255.255
   next
end

Creating an address group for the subnets

config firewall addrgrp
   edit "VPN_SUBNETS"
      set member "NET_100.64.254.12/32" "NET_192.168.12.0/24"
   next
end

Configuring static routes

Use edit 0 to create a route with the next unused number.

config router static
   edit 0
      set dst 100.64.11.2 255.255.255.255
      set device "port1"
   next
    edit 0
      set dst 100.64.10.2 255.255.255.255
      set device "port1"
   next
   edit 0
      set device "HUB-PRIMARY"
   next
   edit 0
      set device "HUB-BACKUP"
      set priority 20
   next
end

Configuring the firewall policies

Use edit 0 to create a policy with the next unused number.

config firewall policy
   edit 0
      set srcintf "port2" "loop0"
      set dstintf "HUB-PRIMARY"
      set srcaddr "all"
      set dstaddr "all"
      set action accept
      set schedule "always"
      set service "ALL"
   next
   edit 0
      set srcintf "HUB-PRIMARY"
      set dstintf "port2" "loop0"
      set srcaddr "all"
      set dstaddr "all"
      set action accept
      set schedule "always"
      set service "ALL"
   next
   edit 0
      set srcintf "port2" "loop0"
      set dstintf "HUB-BACKUP"
      set srcaddr "all"
      set dstaddr "all"
      set action accept
      set schedule "always"
      set service "ALL"
   next
   edit 0
      set srcintf "HUB-BACKUP"
      set dstintf "port2" "loop0"
      set srcaddr "all"
      set dstaddr "all"
      set action accept
      set schedule "always"
      set service "ALL"
   next
end

Each spoke configuration calls for similar Phase1 parameters, but differs for the rest of the configuration in a few keys areas:

• Aggressive mode: As the hub is validating the inbound ID, we have configured our peer ID to the matching string “SPOKES”.

• X-Auth: Our spokes are acting as X-auth clients, and each of our unit is using distinct credentials passed to the hub device during IKE phase1 negotiation.

• Phase 2 quick mode selectors: As the title of this recipe suggests, this is where the spoke provisioning routing automation happens. We’ve defined address objects, added them to a group, and performed the configuration found in Phase2. There is however a peculiarity where if we have more than one subnet behind our spoke unit, the “set mesh-selector-type subnet” command must be configured to ensure multiple Phase2 SAs are negotiated for each subnet listed in our group.

• Routing: As previously expressed, we have configured our default routing to flow through the primary hub (blue links) and failover routing to the backup hub (orange links, using route priority adjustment). Notice that we are explicitly routing each hub’s public IP through the public Internet to ensure that traffic will not flow through the VPN tunnel (and result in flapping).

Where the spoke configurations will be different

As explained earlier, the spoke FortiGate configurations will be slightly different on each individual spoke. The settings will be similar on all of the spoke with the following exceptions:

X-Auth: Our spokes are acting as X-auth clients, and each of our unit is using distinct credentials passed to the hub device during IKE phase1 negotiation.

   config vpn ipsec phase1-interface
      edit "HUB-PRIMARY"
         set authusr <The account will be the one associated with the specific spoke>
         set authpasswd <The password will be the one associated with the specific spoke>
      next 
      edit "HUB-BACKUP"
         set authusr <The account will be the one associated with the specific spoke>
         set authpasswd <The password will be the one associated with the specific spoke>
      next
   end

Phase 2 quick mode selectors: This is where the spoke  routing automation happens. We’ve defined address objects, added them to a group, and performed the configuration found in Phase2. There is however a peculiarity where if we have more than one subnet behind our spoke unit, the following  setting must be used to ensure multiple Phase2 SAs are negotiated for each subnet listed in our group:

   config vpn ipsec phase1-interface
      edit <ipsec tunnel name>
         set mesh-selector-type subnet
      end
   end

Routing: This wont necessarily be different between the different spoke FortiGates, but as previously mentioned, in this example recipe we have configured our default routing to flow through the primary hub and failover routing to the backup hub. Notice that we are explicitly routing each hub’s public IP through the public Internet to ensure that traffic will not flow through the VPN tunnel (and result in flapping).

Results

And this concludes our VPN configuration! But this recipe would not be complete without a very important verification step. Lets look at the routing table on the hub:

HUB # get router info routing-table all
Codes: K - kernel, C - connected, S - static, R - RIP, B - BGP
       O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, ia - IS-IS inter area
       * - candidate default< br/> S*      0.0.0.0/0 [10/0] via 192.168.56.2, port1
S       100.64.254.12/32 [15/0] is directly connected, HUB_0
S       100.64.254.13/24 [15/0] is directly connected, HUB_1
C       192.168.11.0/24 is directly connected, port2
S       192.168.12.0/24 [15/0] is directly connected, HUB_0
S       192.168.13.0/24 [15/0] is directly connected, HUB_1
C       192.168.56.0/24 is directly connected, port1

As can be seen above, our spoke subnets have been automatically injected into the hub’s routing tables. A closer look at the VPN details of one spoke confirms that the hub received the negotiated subnets during quick mode negotiation and inserted distinct SAs for each SA.

FGT1 # get vpn ipsec tunnel details 
gateway
  name: 'HUB_0'
  type: route-based
  local-gateway: 192.168.56.11:0 (static)
  remote-gateway: 192.168.56.12:0 (dynamic)
  mode: ike-v1
  interface: 'port1' (2)
  rx  packets: 56  bytes: 8736  errors: 0
  tx  packets: 41  bytes: 3444  errors: 0
  dpd: enabled/negotiated  idle: 5000ms  retry: 3  count: 0
  selectors
    name: 'HUB-P2'
    auto-negotiate: disable
    mode: tunnel
    src: 0:0.0.0.0-255.255.255.255:0
    dst: 0:192.168.12.0-192.168.12.255:0
       --------OUTPUT TRUNCATED--------
  selectors
    name: 'HUB-P2'
    auto-negotiate: disable
    mode: tunnel
    src: 0:0.0.0.0-255.255.255.255:0
    dst: 0:100.64.254.12-100.64.254.12:0
       --------OUTPUT TRUNCATED--------

If you require communication between the spokes, this can be routed through the hub FortiGates. The only change to the example recipe’s configuration is an addition policy on each of the hub FortiGates which defines the both the Incoming Interface and the Outgoing Interface as the VPN Dialup Interface (in this example, SPOKES)

On the Spoke FortiGates, once the poke tunnels have been established, you can see the default route to the primary datacenter  and the alternate though less preferred route to the backup datacenter by running the command get router info routing-table all

FGT-SPOKE-1 # get router info routing-table all
Codes: K - kernel, C - connected, S - static, R - RIP, B - BGP
O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2
i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, ia - IS-IS inter area
* - candidate default
S* 0.0.0.0/0 [10/0] is directly connected, HUB-PRIMARY
             [10/0] is directly connected, HUB-BACKUP, [20/0]
S 100.64.10.2/32 [10/0] is directly connected, port1
S 100.64.11.2/32 [10/0] is directly connected, port1
C 100.64.12.0/24 is directly connected, port1
C 100.64.254.12/32 is directly connected, lo0
C 192.168.12.0/24 is directly connected, port2

We can test the failover function by shutting down the port1 interface on the primary hub. This will bring down the VPN between the primary hub and the spokes. Once the DPD detects the fault, traffic switches over to the backup hub as shown here:

FGT-SPOKE-1 # get router info routing-table all
Codes: K - kernel, C - connected, S - static, R - RIP, B - BGP
O - OSPF, IA - OSPF inter area
N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
E1 - OSPF external type 1, E2 - OSPF external type 2
i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, ia - IS-IS inter area
* - candidate default
S* 0.0.0.0/0 [10/0] is directly connected, HUB-BACKUP, [20/0]
S 100.64.10.2/32 [10/0] is directly connected, port1
S 100.64.11.2/32 [10/0] is directly connected, port1
C 100.64.12.0/24 is directly connected, port1
C 100.64.254.12/32 is directly connected, lo0
C 192.168.12.0/24 is directly connected, port2

Final notes

• The technique shown here does not involve dynamic routing so this configuration and its very straight forward template can be easily used to scale up the topology to include thousands of spoke sites.

• To make it even easier, this configuration can be entirely built and automated with FortiManager, which has support for provisioning hub-and-spoke dialup topologies.