Map SD-WAN member priorities to BGP MED attribute when spoke advertises routes using iBGP to hub

When a spoke advertises routes using iBGP to a hub, SD-WAN member priorities are mapped into the BGP multiple exit discriminator (MED) attribute using the following CLI commands:

config system sdwan
    config neighbor
        edit <bgp-peer-IP>
            set member <num_1> ... <num_n>
            set route-metric {preferable | priority}
            set health-check <health-check-name>
        next
    end
end

Option	Description
preferable	Select neighbor based on its HC to match BGP preferable/unpreferable route_map.
priority	Select neighbor based on its members' priority-in-sla/priority-out-sla value.

Routes to prefixes behind spokes are advertised by the SD-WAN hub to eBGP peers on an external network. The relative values of the BGP MED attribute for each hub are used to indicate to eBGP peers the more preferred paths, that is, the preferred hub used to route to spoke prefixes.

Example

This example includes one spoke and two hubs:

• The spoke has two overlays to Hub-1 and Hub-2 respectively.

• BGP neighbors between the spoke and hubs are over loopback IP addresses

• SD-WAN is configured on the spoke.

To configure and test the example:

1. Configure Hub-1:

   The Hub-2 configuration is identical except for the router-id, which reflects the different loopback IP address.

   1. Configure SD-WAN:

      config system sdwan
          set status enable
          config zone
              edit "overlay"
              next
          end
          config members
              edit 1
                  set interface "EDGE_T1"
                  set zone "overlay"
              next
              edit 2
                  set interface "EDGE_T2"
                  set zone "overlay"
              next
          end
          config health-check
              edit "Remote_HC"
                  set detect-mode remote
                  set sla-id-redistribute 1
                  set members 1 2
                  config sla
                      edit 1
                          set link-cost-factor remote                
                          set priority-in-sla 100
                          set priority-out-sla 200
                      next
                  end
              next
          end
      end

      The priority-[in|out]-sla settings are used if a spoke does not send a priority. This is used in legacy configurations where spokes only send in or out for the SLA status.

   2. Configure BGP:

      config router bgp
          set as 65001
          set router-id 172.31.0.1        //172.31.0.2 for Hub-2
          set recursive-inherit-priority enable
          ...
          config neighbor-group
              edit "EDGE"
                  set remote-as 65001
                  set update-source "Loopback0"
                  set route-reflector-client enable
              next
          end
          config neighbor-range
              edit 1
                  set prefix 172.31.0.64 255.255.255.192
                  set neighbor-group "EDGE"
              next
          end
          ...
      end

2. Configure the spoke:

   1. Configure SD-WAN:

      config system sdwan
          set status enable
          config zone
              edit "overlay"
              next
          end
          config members
              edit 4
                  set interface "H1_T11"
                  set zone "overlay"
                  set source 172.31.0.65
                  set priority-in-sla 50
                  set priority-out-sla 100
              next
              edit 5
                  set interface "H1_T22"
                  set zone "overlay"
                  set source 172.31.0.65
                  set priority-in-sla 70
                  set priority-out-sla 120
              next
              edit 7
                  set interface "H2_T11"
                  set zone "overlay"
                  set source 172.31.0.65
                  set priority-in-sla 60
                  set priority-out-sla 110
              next
              edit 8
                  set interface "H2_T22"
                  set zone "overlay"
                  set source 172.31.0.65
                  set priority-in-sla 80
                  set priority-out-sla 130
              next
          end
          config health-check
              edit "HUB"
                  set server "172.31.100.100"
                  set embed-measured-health enable
                  set sla-id-redistribute 1
                  set sla-fail-log-period 10
                  set sla-pass-log-period 10
                  set members 4 5 7 8
                  config sla
                      edit 1
                          set link-cost-factor latency
                          set latency-threshold 100
                      next
                  end
              next
          end
          config neighbor
              edit "172.31.0.1"
                  set member 4 5
                  set route-metric priority
                  set health-check "HUB"
              next
              edit "172.31.0.2"
                  set member 7 8
                  set route-metric priority    
                  set health-check "HUB"
              next
          end
      end

   2. Configure BGP:

      config router bgp
          set as 65001
          set router-id 172.31.0.65
          ...
          config neighbor
              edit "172.31.0.1"
                  ......
                  set remote-as 65001
                  set update-source "Loopback0"
              next
              edit "172.31.0.2"
                  ...
                  set remote-as 65001
                  set update-source "Loopback0"
              next
      
          end
          config network
              edit 1
                  set prefix 10.0.3.0 255.255.255.0
              next
          end
          ...
      end

      The routes with MEDs are advertised to a router on the external network that establishes a BGP neighbor relationship with Hub-1 and Hub-2. When sending traffic destined for 10.0.3.0/24, the router on the external network will prefer to send traffic to the hub with the lower MED.

3. All overlays are in SLA:

   Spoke # diagnose sys sdwan health-check
   Health Check(HUB):
   Seq(4 H1_T11): state(alive), packet-loss(0.000%), latency(0.225), jitter(0.035), mos(4.404), bandwidth-up(999998), bandwidth-dw(999997), bandwidth-bi(1999995), sla_map=0x1
   Seq(5 H1_T22): state(alive), packet-loss(0.000%), latency(0.203), jitter(0.016), mos(4.404), bandwidth-up(999998), bandwidth-dw(999998), bandwidth-bi(1999996), sla_map=0x1
   Seq(7 H2_T11): state(alive), packet-loss(0.000%), latency(0.260), jitter(0.035), mos(4.404), bandwidth-up(999997), bandwidth-dw(999997), bandwidth-bi(1999994), sla_map=0x1
   Seq(8 H2_T22): state(alive), packet-loss(0.000%), latency(0.201), jitter(0.018), mos(4.404), bandwidth-up(999998), bandwidth-dw(999998), bandwidth-bi(1999996), sla_map=0x1

   When sending traffic destined for 10.0.3.0/24, the router on the external network will prefer to send traffic to Hub-1 with lower MED 50 over Hub-2 with higher MED 60.

   1. Check Hub-1 metric from the external network router:

      External_Network_Router # get router info bgp neighbors 172.31.0.1 advertised-routes
      VRF 0 BGP table version is 31, local router ID is 172.31.0.65
      Status codes: s suppressed, d damped, h history, * valid, > best, i - internal
      Origin codes: i - IGP, e - EGP, ? - incomplete
      
         Network             Next Hop         Metric     LocPrf Weight RouteTag Path
      *>i10.0.3.0/24      172.31.0.65     50            100  32768        0 i <-/->
      
      Total number of prefixes 1

   2. Check Hub-2 metric from the external network router:

      External_Network_Router # get router info bgp neighbors 172.31.0.2 advertised-routes
      VRF 0 BGP table version is 31, local router ID is 172.31.0.65
      Status codes: s suppressed, d damped, h history, * valid, > best, i - internal
      Origin codes: i - IGP, e - EGP, ? - incomplete
      
         Network             Next Hop        Metric     LocPrf Weight RouteTag Path
      *>i10.0.3.0/24      172.31.0.65     60            100  32768        0 i <-/->
      
      Total number of prefixes 1

4. Verify that the metric updates when a link becomes out of SLA.

   When sending traffic destined for 10.0.3.0/24, the router on the external network will prefer to send traffic to Hub-2 with lower MED 60 over Hub-1 with higher MED 70.

   # diagnose sys sdwan health-check
   Health Check(HUB):
   Seq(4 H1_T11): state(alive), packet-loss(0.000%), latency(120.225), jitter(0.037), mos(4.338), bandwidth-up(999997), bandwidth-dw(999996), bandwidth-bi(1999993), sla_map=0x0
   Seq(5 H1_T22): state(alive), packet-loss(0.000%), latency(0.203), jitter(0.015), mos(4.404), bandwidth-up(999998), bandwidth-dw(999997), bandwidth-bi(1999995), sla_map=0x1
   Seq(7 H2_T11): state(alive), packet-loss(0.000%), latency(0.249), jitter(0.026), mos(4.404), bandwidth-up(999998), bandwidth-dw(999996), bandwidth-bi(1999994), sla_map=0x1
   Seq(8 H2_T22): state(alive), packet-loss(0.000%), latency(0.205), jitter(0.018), mos(4.404), bandwidth-up(999998), bandwidth-dw(999997), bandwidth-bi(1999995), sla_map=0x1
   

   1. Check Hub-1 metric from the external network router:

      External_Network_Router # get router info bgp neighbors 172.31.0.1 advertised-routes
      VRF 0 BGP table version is 31, local router ID is 172.31.0.65
      Status codes: s suppressed, d damped, h history, * valid, > best, i - internal
      Origin codes: i - IGP, e - EGP, ? - incomplete
      
         Network             Next Hop        Metric     LocPrf Weight RouteTag Path
      *>i10.0.3.0/24      172.31.0.65     70            100  32768        0 i <-/->
      
      Total number of prefixes 1

   2. Check Hub-2 metric from the external network router:

      External_Network_Router # get router info bgp neighbors 172.31.0.2 advertised-routes
      VRF 0 BGP table version is 31, local router ID is 172.31.0.65
      Status codes: s suppressed, d damped, h history, * valid, > best, i - internal
      Origin codes: i - IGP, e - EGP, ? - incomplete
      
         Network             Next Hop        Metric     LocPrf Weight RouteTag Path
      *>i10.0.3.0/24      172.31.0.65     60            100  32768        0 i <-/->
      
      Total number of prefixes 1

5. H1_T11 and H2_T11 are out of SLA.

   When sending traffic destined for 10.0.3.0/24, the router on the external network will prefer to send traffic to Hub-1 with lower MED 70 over Hub-2 with higher MED 80.

   Spoke # diagnose sys sdwan  health-check
   Health Check(HUB):
   Seq(4 H1_T11): state(alive), packet-loss(0.000%), latency(120.194), jitter(0.011), mos(4.338), bandwidth-up(999997), bandwidth-dw(999997), bandwidth-bi(1999994), sla_map=0x0
   Seq(5 H1_T22): state(alive), packet-loss(0.000%), latency(0.191), jitter(0.008), mos(4.404), bandwidth-up(999998), bandwidth-dw(999998), bandwidth-bi(1999996), sla_map=0x1
   Seq(7 H2_T11): state(alive), packet-loss(0.000%), latency(120.235), jitter(0.029), mos(4.338), bandwidth-up(999997), bandwidth-dw(999997), bandwidth-bi(1999994), sla_map=0x0
   Seq(8 H2_T22): state(alive), packet-loss(0.000%), latency(0.215), jitter(0.016), mos(4.404), bandwidth-up(999998), bandwidth-dw(999998), bandwidth-bi(1999996), sla_map=0x1
   

   1. Check Hub-1 metric from the external network router:

      External_Network_Router # get router info bgp neighbors 172.31.0.1 advertised-routes
      VRF 0 BGP table version is 31, local router ID is 172.31.0.65
      Status codes: s suppressed, d damped, h history, * valid, > best, i - internal
      Origin codes: i - IGP, e - EGP, ? - incomplete
      
         Network             Next Hop        Metric     LocPrf Weight RouteTag Path
      *>i10.0.3.0/24      172.31.0.65     70            100  32768        0 i <-/->
      
      Total number of prefixes 1

   2. Check Hub-2 metric from the external network router:

      External_Network_Router # get router info bgp neighbors 172.31.0.2 advertised-routes
      VRF 0 BGP table version is 31, local router ID is 172.31.0.65
      Status codes: s suppressed, d damped, h history, * valid, > best, i - internal
      Origin codes: i - IGP, e - EGP, ? - incomplete
      
         Network             Next Hop        Metric     LocPrf Weight RouteTag Path
      *>i10.0.3.0/24      172.31.0.65     80            100  32768        0 i <-/->
      
      Total number of prefixes 1

6. H1_T11, H2_T11, and H1_T22 are out of SLA.

   When sending traffic destined for 10.0.3.0/24, the router on the external network will prefer to send traffic to Hub-2 with lower MED 80 over Hub-1 with higher MED 100.

   Spoke # diagnose sys sdwan  health-check
   Health Check(HUB):
   Seq(4 H1_T11): state(alive), packet-loss(0.000%), latency(120.199), jitter(0.016), mos(4.338), bandwidth-up(999997), bandwidth-dw(999996), bandwidth-bi(1999993), sla_map=0x0
   Seq(5 H1_T22): state(alive), packet-loss(0.000%), latency(120.190), jitter(0.012), mos(4.338), bandwidth-up(999998), bandwidth-dw(999997), bandwidth-bi(1999995), sla_map=0x0
   Seq(7 H2_T11): state(alive), packet-loss(0.000%), latency(120.251), jitter(0.022), mos(4.338), bandwidth-up(999997), bandwidth-dw(999996), bandwidth-bi(1999993), sla_map=0x0
   Seq(8 H2_T22): state(alive), packet-loss(0.000%), latency(0.211), jitter(0.014), mos(4.404), bandwidth-up(999998), bandwidth-dw(999997), bandwidth-bi(1999995), sla_map=0x1
   

   1. Check Hub-1 metric from the external network router:

      External_Network_Router # get router info bgp neighbors 172.31.0.1 advertised-routes
      VRF 0 BGP table version is 31, local router ID is 172.31.0.65
      Status codes: s suppressed, d damped, h history, * valid, > best, i - internal
      Origin codes: i - IGP, e - EGP, ? - incomplete
      
         Network             Next Hop        Metric     LocPrf Weight RouteTag Path
      *>i10.0.3.0/24      172.31.0.65     100           100  32768        0 i <-/->
      
      Total number of prefixes 1
      

   2. Check Hub-2 metric from the external network router:

      External_Network_Router # get router info bgp neighbors 172.31.0.2 advertised-routes
      VRF 0 BGP table version is 31, local router ID is 172.31.0.65
      Status codes: s suppressed, d damped, h history, * valid, > best, i - internal
      Origin codes: i - IGP, e - EGP, ? - incomplete
      
         Network             Next Hop        Metric     LocPrf Weight RouteTag Path
      *>i10.0.3.0/24      172.31.0.65     80            100  32768        0 i <-/->
      
      Total number of prefixes 1

7. All of the overlays are out of SLA.

   When sending traffic destined for 10.0.3.0/24, the router on the external network will prefer to send traffic to Hub-1 with lower MED 100 over Hub-2 with higher MED 110.

   Spoke # diagnose sys sdwan  health-check
   Health Check(HUB):
   Seq(4 H1_T11): state(alive), packet-loss(0.000%), latency(120.201), jitter(0.013), mos(4.338), bandwidth-up(999997), bandwidth-dw(999997), bandwidth-bi(1999994), sla_map=0x0
   Seq(5 H1_T22): state(alive), packet-loss(0.000%), latency(120.187), jitter(0.008), mos(4.338), bandwidth-up(999998), bandwidth-dw(999998), bandwidth-bi(1999996), sla_map=0x0
   Seq(7 H2_T11): state(alive), packet-loss(0.000%), latency(120.251), jitter(0.023), mos(4.338), bandwidth-up(999997), bandwidth-dw(999997), bandwidth-bi(1999994), sla_map=0x0
   Seq(8 H2_T22): state(alive), packet-loss(0.000%), latency(120.194), jitter(0.010), mos(4.338), bandwidth-up(999998), bandwidth-dw(999998), bandwidth-bi(1999996), sla_map=0x0

   1. Check Hub-1 metric from the external network router:

      External_Network_Router # get router info bgp neighbors 172.31.0.1 advertised-routes
      VRF 0 BGP table version is 31, local router ID is 172.31.0.65
      Status codes: s suppressed, d damped, h history, * valid, > best, i - internal
      Origin codes: i - IGP, e - EGP, ? - incomplete
      
         Network             Next Hop        Metric     LocPrf Weight RouteTag Path
      *>i10.0.3.0/24      172.31.0.65     100           100  32768        0 i <-/->
      
      Total number of prefixes 1
      

   2. Check Hub-2 metric from the external network router:

      External_Network_Router # get router info bgp neighbors 172.31.0.2 advertised-routes
      VRF 0 BGP table version is 31, local router ID is 172.31.0.65
      Status codes: s suppressed, d damped, h history, * valid, > best, i - internal
      Origin codes: i - IGP, e - EGP, ? - incomplete
      
         Network             Next Hop        Metric     LocPrf Weight RouteTag Path
      *>i10.0.3.0/24      172.31.0.65     110           100  32768        0 i <-/->
      
      Total number of prefixes 1