Dynamic BGP on loopback dual hub example

This example shows how to configure SD-WAN for a dual hub topology using dynamic BGP on loopback. The steps assume the spoke and hub FortiGates have configured LAN & WAN interfaces and default routing in place.

The FortiGates will require further SD-WAN configuration relevant to your business, such as SD-WAN rules for various applications, and firewall policies to permit and deny traffic as required. Similarly, security profiles are not discussed but should be implemented in accordance to the business’s security requirements.

Note that both hubs share the same IP address for HUB-Lo. This is one of the benefits of BGP on loopback.

Spoke

To configure the spoke:

1. Configure the Loopback interface to be used for BGP peering.

   config system interface 
       edit "Branch-Lo"
           set vdom "root"
           set ip 172.16.0.1 255.255.255.255
           set allowaccess ping
           set type loopback
       next
   end

2. Configure a unique location ID.

   config system settings
       set location-id 172.16.0.1
   end

3. Configure IKE to inject a static route to reach the loopback on all phase1-interfaces toward the hubs. Note, additional VPN configuration required. See Site-to-site VPN.

   config vpn ipsec phase1-interface
       edit HUB1_VPN1
           set exchange-ip-addr4 172.16.0.1
       next
       edit HUB1_VPN2
           set exchange-ip-addr4 172.16.0.1
       next
       edit HUB2_VPN1
           set exchange-ip-addr4 172.16.0.1
       next
       edit HUB2_VPN2
           set exchange-ip-addr4 172.16.0.1
       next
   end

4. Configure BGP:

   config router bgp
       set as 65000
       set router-id 172.16.0.1
       set ibgp-multipath enable
       set recursive-next-hop enable
       set graceful-restart enable
       config neighbor
           edit 172.16.255.250
               set soft-reconfiguration enable
               set capability-graceful-restart enable
               set advertisement-interval 1
               set interface " Branch-Lo"
               set update-source " Branch-Lo"
               set connect-timer 1
               set remote-as 65000
           next
           edit 172.16.255.252
               set soft-reconfiguration enable
               set capability-graceful-restart enable
               set advertisement-interval 1
               set interface " Branch-Lo"
               set update-source " Branch-Lo"
               set connect-timer 1
               set remote-as 65000
           next
       end
       config network
           edit 1
               set prefix 10.1.1.0/24
           next
       end
   end

5. Configure SD-WAN members to utilize the loopback as the source for health checks. Extra configuration provided to define SD-WAN interfaces referenced in later steps. This is required as the VPN tunnel interfaces do not have IP addresses assigned.

   config system sdwan
       config zone
           edit "virtual-wan-link"
           next
           edit "WAN1"
           next
           edit "WAN2"
           next
           edit "HUB1"
           next
           edit "HUB2"
           next
       end
       config members
           edit 1
               set interface "port1"
               set zone "WAN1"
           next
           edit 2
               set interface "port2"
               set zone "WAN2"
           next
           edit 3
               set interface "HUB1-VPN1"
               set zone "HUB1"
               set source 172.16.0.1
           next
           edit 4
               set interface "HUB1-VPN2"
               set zone "HUB1"
               set source 172.16.0.1
           next
           edit 5
               set interface "HUB2-VPN1"
               set zone "HUB2"
               set source 172.16.0.1
           next
           edit 6
               set interface "HUB2-VPN2"
               set zone "HUB2"
               set source 172.16.0.1
           next
       end
   end

6. Configure a firewall policy to permit health check probes.

   config firewall policy
       edit 4
           set name "Health Check Access"
           set srcintf "Branch-Lo"
           set dstintf "HUB1” “HUB2"
           set action accept
           set srcaddr "all"
           set dstaddr "all"
           set schedule "always"
           set service "PING BGP"
       next
   end

Hub

To configure the hub:

1. Configure the Loopback interfaces to be used for BGP peering and health checks.

   config system interface
       edit "HUB-Lo"
           set vdom "root"
           set ip 172.16.255.251 255.255.255.255
           set allowaccess ping
           set type loopback
       next
       edit "BGP-Lo"
           set vdom "root"
           set ip 172.16.255.252 255.255.255.255
           set allowaccess ping
           set type loopback
       next
   end

   You should never share one loopback interface for both BGP peering and HUB health checks (done by spokes). See HUB and BGP loopback interfaces.

2. Configure a unique location ID.

   config system settings
       set location-id 172.16.255.252
   end

3. Configure IKE to inject a static route to reach the loopback on all phase1-interfaces toward the spokes.

   config vpn ipsec phase1-interface
       edit VPN1
           set exchange-ip-addr4 172.16.255.252
       next
       edit VPN2
           set exchange-ip-addr4 172.16.255.252
       next    
   end

4. Configure BGP:

   config router bgp
       set as 65000
       set router-id 172.16.0.1
       set ibgp-multipath enable
       set recursive-next-hop enable
       set graceful-restart enable
       config neighbor group
           edit “EDGE”
               set soft-reconfiguration enable
               set capability-graceful-restart enable
               set advertisement-interval 1
               set next-hop-self enable
               set interface " BGP-Lo"
               set update-source " BGP-Lo"
               set remote-as 65000
           next
       end
   	config neighbor-range
           edit 1
               set prefix 172.16.0.0 255.255.224.0
               set neighbor-group "DYN_BRANCH"
       next
       end
       config network
           edit 1
               set prefix 172.16.0.0 255.255.0.0
               set route-map "LOCAL_REGION"
           next
           edit 10
               set prefix 10.1.0.0 255.255.0.0
           next
       end
   end

5. Configure VPN tunnels to be SD-WAN members:

   config system sdwan
       set status enable
       config zone
           edit "OVERLAYS"
           next
       end
       config members
           edit 1
               set interface "VPN1"
               set zone "OVERLAYS"
           next
           edit 2
               set interface "VPN2"
               set zone "OVERLAYS"
           next
       end
   end

6. Configure a firewall policy to permit health check probes and BGP peering.

   config firewall policy
       edit 4
           set name "Health Check Access"
           set srcintf "VPN1" "VPN2"
           set dstintf "HUB-Lo"
           set action accept
           set srcaddr "all"
           set dstaddr "all"
           set schedule "always"
           set service "ALL"
       next
       edit 5
           set name "Peering"
           set srcintf "VPN1" "VPN2"
           set dstintf "BGP-Lo"
           set action accept
           set srcaddr "all"
           set dstaddr "all"
           set schedule "always"
           set service "PING" "BGP"
       next
   end

HUB and BGP loopback interfaces

The reason to configure separate interfaces on the hub for BGP peering and hub health checks (done by spokes, and/or other hubs) is that, when a health check is configured, a kernel route is installed for that IP address. These kernel routes exist to support the health check and will not be removed even when the health check fails.

If the BGP neighbor configuration incorrectly references the same IP address as the health check, the spoke will continue to send BGP traffic over the tunnel that the peering was initially established over, even if the health check for this tunnel is failing.

Example spoke health-check configuration:

config health-check
    edit "Hub_HC"
        set server "172.16.255.251"
        set members 1 2
    next
end

Spoke kernel route lookup:

get router info kernel | grep 172.16.255.251

tab=254 vf=0 scope=0 type=1 proto=18 prio=0 172.16.0.1/255.255.255.255/0->172.16.255.251/32 pref=0.0.0.0 gwy=20.0.0.1 dev=20(advpn)
tab=254 vf=0 scope=0 type=1 proto=18 prio=0 172.16.0.1/255.255.255.255/0->172.16.255.251/32 pref=0.0.0.0 gwy=30.0.0.1 dev=21(advpn_b)