Running speed tests from the hub to the spokes in dial-up IPsec tunnels
In a hub and spoke SD-WAN topology that uses dial-up VPN overlays, QoS can be applied on individual tunnels based on the measured bandwidth between the hub and spokes. The FortiGate can use the built in speed test to dynamically populate the egress bandwidth to individual dial-up tunnels from the hub.
SD-WAN members on a spoke can switch routes when the speed test is running from the hub to the spoke. The speed test results can be cached for reuse when a tunnel comes back after going down.
CLI commands
Allow upload speed tests to be run from the hub to spokes on demand for dial-up IPsec tunnel:
config system speed-test-schedule
edit <interface>
set dynamic-server {enable | disable}
next
end
|
<interface> |
The dial-up IPsec tunnel interface on the hub. |
|
dynamic-server {enable | disable} |
Enable/disable the dynamic speed test server (default = disable). |
|
To limit the maximum bandwidth used in the speed test, enable |
config system global
set speedtest-server {enable | disable}
end
|
speedtest-server {enable | disable} |
Enable/disable the speed test server on the spoke (default = disable). This setting must be enabled on spoke FortiGates. This enables iPerf in server mode, which listens on the default iPerf TCP port 5201. |
Allow an SD-WAN member on the spoke to switch routes when it is on speed test from the hub to spokes:
config system sdwan
set speedtest-bypass-routing {enable | disable}
config neighbor
edit <bgp neighbor>
set mode speedtest
next
end
end
|
speedtest-bypass-routing {enable | disable} |
Enable/disable bypass routing when doing a speed test on an SD-WAN member (default = disable). |
|
set mode speedtest |
Use the speed test to select the neighbor. |
Manually run uploading speed test on the physical interfaces of each tunnel of an dial-up IPsec interface:
execute speed-test-dynamic <interface> <tunnel_name> <'y'/'n'> <max-out> <min-out>
|
<interface> |
IPsec phase1 interface name. |
|
<tunnel_name> |
The tunnel name, or |
|
<'y'/'n'> |
Apply the result to the tunnels' shaper or not. |
|
<max-out> |
The maximum speed used in a speed test, in kbps. |
|
<min-out> |
The minimum speed used in a speed test, in kbps. |
Manually run a non-blocking uploading speed test:
diagnose netlink interface speed-test-tunnel <interface> <tunnel_name>
Debug and test commands:
|
diagnose debug application speedtest <int> |
Enable debug of the speed test module in the forticron daemon. |
|
diagnose debug application speedtestd <int> |
Enable debug of the speed test server daemon. |
|
diagnose test application forticron 9 |
List the scheduled speed tests. |
|
diagnose test application forticron 10 |
Show the cached speed test results. |
|
diagnose test application forticron 11 |
Write the cached speed test results to disk. |
|
diagnose test application forticron 12 |
Load the speed test results from disk. |
|
diagnose test application forticron 99 |
Cancel all pending speed tests. |
Example
In this example, the hub is configured as a VPN dial-up server and both of the spokes are connected to the hub. It is assumed that the VPN configuration is already done, with a dynamic gateway type and kernel device creation (net-device) disabled. Only one SD-WAN interface is used, so there is only one VPN overlay member in the SD-WAN zone. Multiple WAN interfaces and VPN overlays could be used.
The VPN interfaces and IP addresses are:
|
FortiGate |
Interface |
IP Address |
|---|---|---|
|
FGT_A (Hub) |
hub-phase1 |
10.10.100.254 |
|
FGT_B (Spoke) |
spoke11-p1 |
10.10.100.2 |
|
FGT_D (Spoke) |
spoke21-p1 |
10.10.100.3 |
A recurring speed test is configured that runs on the hub over the dial-up interfaces. The speed tests are performed over the underlay interface from the hub to the spoke. Each spoke is configured to operate as a speed test server and to allow the speed test to run on its underlay interface. The spokes establish BGP peering with the hub over the VPN interface, and advertises its loopback network to the hub. The specific configuration is only shown for FGT_B.
When the speed test is running, routing through the VPN overlay can be bypassed, and route maps are used to filter the routes that are advertised to peers. The spoke's route map does not advertise any routes to the peer, forcing the hub to use others paths to reach the spoke's network.
When no speed tests are running, the spoke's route map allows its network to be advertised on the hub.
When the speed test is complete, the measured egress bandwidth is dynamically applied to the VPN tunnel on the hub, and the result is cached for future use, in case the tunnel is disconnected and reconnected again.
To configure the hub FortiGate (FGT_A):
-
Configure a shaping profile:
config firewall shaping-profile edit "profile_1" config shaping-entries edit 1 set class-id 2 set priority low set guaranteed-bandwidth-percentage 10 set maximum-bandwidth-percentage 10 next end set default-class-id 2 next endThree classes are used in the profile for low, medium, and high priority traffic. Each class is assigned a guaranteed and maximum bandwidth as a percentage of the measured bandwidth from the speed test.
-
Use the shaping profile in the interface:
config system interface edit "hub-phase1" set egress-shaping-profile "profile_1" next end -
Configure a schedule to use for the speed tests:
config firewall schedule recurring edit "speedtest_recurring" set start 01:00 set end 23:00 set day monday tuesday wednesday thursday friday saturday next end -
Configure the speed test schedule:
config system speed-test-schedule edit "hub-phase1" set schedules "speedtest_recurring" set dynamic-server enable next end
To configure the spoke FortiGates (FGT_B and FGT_D):
-
Enable the speed test daemon:
config system global set speedtest-server enable end -
Allow speed tests on the interface:
config system interface edit "port1" append allowaccess speed-test next end -
Configure SD-WAN with bypass routing enabled for speed tests on member spoke11-p1:
config system sdwan set speedtest-bypass-routing enable config members edit 1 set interface "spoke11-p1" next end config neighbor edit "10.10.100.254" set member 1 set mode speedtest next end end -
Configure BGP routing:
config router route-map edit "No_Speed-Test" config rule edit 1 set action permit next end next edit "Start_Speed-Test" config rule edit 1 set action deny next end next end config router bgp set as 65412 config neighbor edit "10.10.100.254" set advertisement-interval 1 set remote-as 65412 set route-map-out "Start_Speed-Test" set route-map-out-preferable "No_Speed-Test" next end config network edit 1 set prefix 2.2.2.2 255.255.255.255 next edit 2 set prefix 10.1.100.0 255.255.255.0 next end end
To manually run the speed test:
# execute speed-test-dynamic hub-phase1 all y 1000 100
Start testing the speed of each tunnel of hub-phase1
[6400d9] hub-phase1_0: physical_intf=port1, local_ip=172.16.200.1, server_ip=172.16.200.2
Wait for test 6400d9 to finish...
Speed-test result for test ID 6400d9:
Completed
measured upload bandwidth is 1002 kbps
measured time Sun Jun 20 15:56:34 2021
The tested out-bandwidth is more than the set maximum accepted value 1000. Will update the tunnel's shaper by the set update-outbandwidth-maximum.
Apply shaping profile 'profile_1' with bandwidth 1000 to tunnel hub-phase1_0 of interface hub-phase1
[6400e0] hub-phase1_1: physical_intf=port1, local_ip=172.16.200.1, server_ip=172.16.200.4
Wait for test 6400e0 to finish...
Speed-test result for test ID 6400e0:
Completed
measured upload bandwidth is 1002 kbps
measured time Sun Jun 20 15:56:39 2021
The tested out-bandwidth is more than the set maximum accepted value 1000. Will update the tunnel's shaper by the set update-outbandwidth-maximum.
Apply shaping profile 'profile_1' with bandwidth 1000 to tunnel hub-phase1_1 of interface hub-phase1# diagnose netlink interface speed-test-tunnel hub-phase1 all send speed test request for tunnel 'hub-phase1_0' of 'hub-phase1': 172.16.200.1 -> 172.16.200.2 send speed test request for tunnel 'hub-phase1_1' of 'hub-phase1': 172.16.200.1 -> 172.16.200.4
Results
-
Before the speed test starts, FGT_A can receive the route from FGT_B by BGP:
# get router info routing-table bgp Routing table for VRF=0 B 2.2.2.2/32 [200/0] via 10.10.100.2 (recursive via 172.16.200.2, hub-phase1), 00:00:10 B 10.1.100.0/24 [200/0] via 10.10.100.2 (recursive via 172.16.200.2, hub-phase1), 00:00:10
-
At the scheduled time, the speed test starts for the hub-phase1 interface from hub to spoke:
# diagnose test application forticron 9 Speed test schedules: Interface Server Update Up/Down-limit (kbps) Days H:M TOS Schedule --------------------------------------------------------------------------------------------------------------------------- hub-phase1 dynamic 1111111 14:41 0x00 speedtest_recurring Active schedules: 64002f: hub-phase1(port1) 172.16.200.2 hub-phase1_1 64002e: hub-phase1(port1) 172.16.200.4 hub-phase1_0The
diagnose debug application speedtest -1command can be used on both the hub and spokes to check the speed test execution. -
While the speed test is running, FGT_A does not receive the route from FGT_B by BGP:
# get router info routing-table bgp Routing table for VRF=0
-
Speed tests results can be dynamically applied to the dial-up tunnel for egress traffic shaping:
# diagnose vpn tunnel list ------------------------------------------------------ name=hub-phase1_0 ver=2 serial=c 172.16.200.1:0->172.16.200.4:0 tun_id=172.16.200.4 dst_mtu=1500 dpd-link=on remote_location=0.0.0.0 weight=1 ... egress traffic control: bandwidth=737210(kbps) lock_hit=0 default_class=2 n_active_class=3 class-id=2 allocated-bandwidth=73720(kbps) guaranteed-bandwidth=73720(kbps) max-bandwidth=73720(kbps) current-bandwidth=0(kbps) priority=low forwarded_bytes=52 dropped_packets=0 dropped_bytes=0 class-id=3 allocated-bandwidth=221163(kbps) guaranteed-bandwidth=221162(kbps) max-bandwidth=294883(kbps) current-bandwidth=0(kbps) priority=medium forwarded_bytes=0 dropped_packets=0 dropped_bytes=0 class-id=4 allocated-bandwidth=442325(kbps) guaranteed-bandwidth=147441(kbps) max-bandwidth=442325(kbps) current-bandwidth=0(kbps) priority=high forwarded_bytes=0 dropped_packets=0 dropped_bytes=0 ------------------------------------------------------ name=hub-phase1_1 ver=2 serial=d 172.16.200.1:0->172.16.200.2:0 tun_id=172.16.200.2 dst_mtu=1500 dpd-link=on remote_location=0.0.0.0 weight=1 ... egress traffic control: bandwidth=726813(kbps) lock_hit=0 default_class=2 n_active_class=3 class-id=2 allocated-bandwidth=72681(kbps) guaranteed-bandwidth=72681(kbps) max-bandwidth=72681(kbps) current-bandwidth=0(kbps) priority=low forwarded_bytes=123 dropped_packets=0 dropped_bytes=0 class-id=3 allocated-bandwidth=218044(kbps) guaranteed-bandwidth=218043(kbps) max-bandwidth=290725(kbps) current-bandwidth=0(kbps) priority=medium forwarded_bytes=0 dropped_packets=0 dropped_bytes=0 class-id=4 allocated-bandwidth=436087(kbps) guaranteed-bandwidth=145362(kbps) max-bandwidth=436087(kbps) current-bandwidth=0(kbps) priority=high forwarded_bytes=0 dropped_packets=0 dropped_bytes=0 -
Speed test results can be cached, indexed, and written to disk:
# diagnose test application forticron 10 Speed test results: 1: vdom=root, phase1intf=hub-phase1, peer-id='spoke11-p1', bandwidth=737210, last_log=1624226603 2: vdom=root, phase1intf=hub-phase1, peer-id='spoke21-p1', bandwidth=726813, last_log=1624226614
# diagnose test application forticron 11 Write 2 logs to disk.
# diagnose test application forticron 12 load 2 results.
Disable then reenable the IPsec VPN tunnel and the cached speed test results can be applied to the tunnel again:
# diagnose vpn tunnel list ------------------------------------------------------ name=hub-phase1_0 ver=2 serial=c 172.16.200.1:0->172.16.200.4:0 tun_id=172.16.200.4 dst_mtu=1500 dpd-link=on remote_location=0.0.0.0 weight=1 ... egress traffic control: bandwidth=737210(kbps) lock_hit=0 default_class=2 n_active_class=3 ------------------------------------------------------ name=hub-phase1_1 ver=2 serial=d 172.16.200.1:0->172.16.200.2:0 tun_id=172.16.200.2 dst_mtu=1500 dpd-link=on remote_location=0.0.0.0 weight=1 ... egress traffic control: bandwidth=726813(kbps) lock_hit=0 default_class=2 n_active_class=3