Network virtual functions
You must create network virtual functions (VFs) to use with the FortiGate-VM. As confirmed in NIC versions, the particular combination of NIC and driver/module supports SR-IOV. You can further check this by ensuring that the module has settings for VFs.
[root@esxi-tiger-14-7:~] esxcli system module parameters list -m i40en Name Type Value Description ------------- ------------ ----- ----------- DRSS array of int Enable/disable the DefQueue RSS(default = 0 ) EEE array of int Energy Efficient Ethernet feature (EEE): 0 = disable, 1 = enable, (default = 1) LLDP array of int Link Layer Discovery Protocol (LLDP) agent: 0 = disable, 1 = enable, (default = 1) RSS array of int Enable/disable the NetQueue RSS( default = 1 ) RxITR int Default RX interrupt interval (0..0xFFF), in microseconds (default = 50) TxITR int Default TX interrupt interval (0..0xFFF), in microseconds, (default = 100) VMDQ array of int Number of Virtual Machine Device Queues: 0/1 = disable, 2-16 enable (default =8) max_vfs array of int Maximum number of VFs to be enabled (0..128) trust_all_vfs array of int Always set all VFs to trusted mode 0 = disable (default), other = enable
Leaving the max_vfs as-is is fine. This just limits the number that you can define. This leaves the number of VFs configurable in the hands of the mixture of ESXi and NIC and driver. In the documented example, only two VFs are needed per physical interface. Configuring eight allows a greater degree of flexibility without the need to reboot the host.
The trust_all_vfs is an important setting. It ensures that spoof check is disabled and that the VF is trusted.
[root@esxi-tiger-14-7:~] esxcli system module parameters set -m i40en -p "max_vfs=0,0,8,8,8,8 trust_all_vfs=0,0,1,1,1,1" [root@esxi-tiger-14-7:~] esxcli system module parameters list -m i40en Name Type Value Description ------------- ------------ ----------- ----------- DRSS array of int Enable/disable the DefQueue RSS(default = 0 ) EEE array of int Energy Efficient Ethernet feature (EEE): 0 = disable, 1 = enable, (default = 1) LLDP array of int Link Layer Discovery Protocol (LLDP) agent: 0 = disable, 1 = enable, (default = 1) RSS array of int Enable/disable the NetQueue RSS( default = 1 ) RxITR int Default RX interrupt interval (0..0xFFF), in microseconds (default = 50) TxITR int Default TX interrupt interval (0..0xFFF), in microseconds, (default = 100) VMDQ array of int Number of Virtual Machine Device Queues: 0/1 = disable, 2-16 enable (default =8) max_vfs array of int 0,0,8,8,8,8 Maximum number of VFs to be enabled (0..128) trust_all_vfs array of int 0,0,1,1,1,1 Always set all VFs to trusted mode 0 = disable (default), other = enable [root@esxi-tiger-14-7:~] reboot
Why “0,0,8,8,8,8” and “0,0,1,1,1,1”? This is an array of values, which references each NIC using the i40en driver. If you compare this to the earlier esxcli network nic list output, you see that six NICs are using the i40en driver: vmnic0 and vmnic1 in addition to the four that are of larger interest. vmnic0 and vmnic1 are Dell OEM devices on the mainboard and are therefore not recommended for this use case. So the array references all six NICs in order. You must diligently check any changes made to the hardware after this setup.
|
Disabling spoof check allows the VM to define the MAC addresses it associates to interfaces rather than those that the host set. This is important when considering the deployment of LAGs and for FortiGate Clustering Protocol vMAC operation. |
|
|
Setting the VF to trusted is important to ensure that the VF tracks and follows the status of the PF. Allowing the VM to detect interface down accordingly. This setting is also mandatory for LAG. |
To make further checks around this area, installing the vendor toolset is highly recommended, if available. For this example, the Intel plugin is installed:
[root@esxi-tiger-14-7:/vmfs/volumes/62248617-84a2aac8-cef7-e4434b314530/Tiger] unzip Intel-intnetcli_1.6.5.0__esx7.0.zip Archive: Intel-intnetcli_1.6.5.0__esx7.0.zip inflating: Intel-intnetcli_intnetcli.1.6.5.0-700.15843807_18728558.zip inflating: doc/README.txt [root@esxi-tiger-14-7:/vmfs/volumes/62248617-84a2aac8-cef7-e4434b314530/Tiger] unzip Intel-intnetcli_intnetcli.1.6.5.0-700.15843807_18728558.zip Archive: Intel-intnetcli_intnetcli.1.6.5.0-700.15843807_18728558.zip inflating: index.xml inflating: vendor-index.xml inflating: metadata.zip inflating: vib20/int-esx-intnetcli/INT_bootbank_int-esx-intnetcli_700.1.6.5.0-15843807.vib [root@esxi-tiger-14-7:/vmfs/volumes/62248617-84a2aac8-cef7-e4434b314530/Tiger] cd [root@esxi-tiger-14-7:~] esxcli software vib install -v /vmfs/volumes/ESXI-TIGER-14-7/Tiger/vib20/int-esx-intnetcli/INT_bootbank_int-esx-intnetcli_700.1.6.5.0-15843807.vib Installation Result Message: The update completed successfully, but the system needs to be rebooted for the changes to be effective. Reboot Required: true VIBs Installed: INT_bootbank_int-esx-intnetcli_700.1.6.5.0-15843807 VIBs Removed: VIBs Skipped: [root@esxi-tiger-14-7:~] reboot [root@esxi-tiger-14-7:~] esxcli intnet sriovnic vf get -n vmnic4 VF ID Trusted Spoof Check ----- ------- ----------- 0 true false 1 true false 2 true false 3 true false 4 true false 5 true false 6 true false 7 true false
|
|
Adding the configuration in this way automatically created the eight VFs. You can also create the VFs in vCentre, in case you executed the build process differently. |
[root@esxi-tiger-14-7:~] esxcli network sriovnic list
Name PCI Device Driver Link Speed Duplex MAC Address MTU Description
------ ------------ ------ ---- ----- ------ ----------------- ---- -----------
vmnic4 0000:3b:00.0 i40en Up 25000 Full 3c:fd:fe:c3:8a:c8 1500 Intel(R) Ethernet Controller XXV710 for 25GbE SFP28
vmnic5 0000:3b:00.1 i40en Up 25000 Full 3c:fd:fe:c3:8a:c9 1500 Intel(R) Ethernet Controller XXV710 for 25GbE SFP28
vmnic6 0000:5e:00.0 i40en Up 25000 Full 3c:fd:fe:c3:94:1c 1500 Intel(R) Ethernet Controller XXV710 for 25GbE SFP28
vmnic7 0000:5e:00.1 i40en Up 25000 Full 3c:fd:fe:c3:94:1d 1500 Intel(R) Ethernet Controller XXV710 for 25GbE SFP28
[root@esxi-tiger-14-7:~] esxcli network sriovnic vf list -n vmnic4
VF ID Active PCI Address Owner World ID
----- ------ -------------- --------------
0 false 00000:059:02.0 -
1 false 00000:059:02.1 -
2 false 00000:059:02.2 -
3 false 00000:059:02.3 -
4 false 00000:059:02.4 -
5 false 00000:059:02.5 -
6 false 00000:059:02.6 -
7 false 00000:059:02.7 -