Network virtual functions
You must create network virtual functions (VFs) to use with the FortiGate-VM.
To determine which NICs are capable of running VFs:
[root@rhel-tiger-14-6 ~]# lspci -d ::0x200 -vv | egrep "Ethernet controller|SR-IOV" 01:00.0 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01) 01:00.1 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01) 19:00.0 Ethernet controller: Intel Corporation Ethernet Controller X710 for 10GbE SFP+ (rev 01) Capabilities: [160] Single Root I/O Virtualization (SR-IOV) 19:00.1 Ethernet controller: Intel Corporation Ethernet Controller X710 for 10GbE SFP+ (rev 01) Capabilities: [160] Single Root I/O Virtualization (SR-IOV) 3b:00.0 Ethernet controller: Intel Corporation Ethernet Controller E810-C for QSFP (rev 02) Capabilities: [160] Single Root I/O Virtualization (SR-IOV) 3b:00.1 Ethernet controller: Intel Corporation Ethernet Controller E810-C for QSFP (rev 02) Capabilities: [160] Single Root I/O Virtualization (SR-IOV) 5e:00.0 Ethernet controller: Intel Corporation Ethernet Controller E810-C for QSFP (rev 02) Capabilities: [160] Single Root I/O Virtualization (SR-IOV) 5e:00.1 Ethernet controller: Intel Corporation Ethernet Controller E810-C for QSFP (rev 02) Capabilities: [160] Single Root I/O Virtualization (SR-IOV) 86:00.0 Ethernet controller: Intel Corporation Ethernet Controller XL710 for 40GbE QSFP+ (rev 02) Capabilities: [160] Single Root I/O Virtualization (SR-IOV) 86:00.1 Ethernet controller: Intel Corporation Ethernet Controller XL710 for 40GbE QSFP+ (rev 02) Capabilities: [160] Single Root I/O Virtualization (SR-IOV) 86:00.2 Ethernet controller: Intel Corporation Ethernet Controller XL710 for 40GbE QSFP+ (rev 02) Capabilities: [160] Single Root I/O Virtualization (SR-IOV) 86:00.3 Ethernet controller: Intel Corporation Ethernet Controller XL710 for 40GbE QSFP+ (rev 02) Capabilities: [160] Single Root I/O Virtualization (SR-IOV) af:00.0 Ethernet controller: Intel Corporation Ethernet Controller XL710 for 40GbE QSFP+ (rev 02) Capabilities: [160] Single Root I/O Virtualization (SR-IOV) af:00.1 Ethernet controller: Intel Corporation Ethernet Controller XL710 for 40GbE QSFP+ (rev 02) Capabilities: [160] Single Root I/O Virtualization (SR-IOV) af:00.2 Ethernet controller: Intel Corporation Ethernet Controller XL710 for 40GbE QSFP+ (rev 02) Capabilities: [160] Single Root I/O Virtualization (SR-IOV) af:00.3 Ethernet controller: Intel Corporation Ethernet Controller XL710 for 40GbE QSFP+ (rev 02) Capabilities: [160] Single Root I/O Virtualization (SR-IOV)
To determine the maximum amount of VFs that a PF can run:
[root@rhel-tiger-14-6 ~]# cat /sys/class/net/ens4f0/device/sriov_totalvfs 32
A useful way to see available NICs, the device name, and PCIe bus address is as follows:
[root@rhel-tiger-14-6 ~]# lshw -c network -businfo Bus info Device Class Description ======================================================== pci@0000:01:00.0 eno3 network I350 Gigabit Network Connection pci@0000:01:00.1 eno4 network I350 Gigabit Network Connection pci@0000:19:00.0 eno1 network Ethernet Controller X710 for 10GbE SFP+ pci@0000:19:00.1 eno2 network Ethernet Controller X710 for 10GbE SFP+ pci@0000:3b:00.0 ens1f0 network Ethernet Controller E810-C for QSFP pci@0000:3b:00.1 ens1f1 network Ethernet Controller E810-C for QSFP pci@0000:5e:00.0 ens2f0 network Ethernet Controller E810-C for QSFP pci@0000:5e:00.1 ens2f1 network Ethernet Controller E810-C for QSFP pci@0000:86:00.0 ens5f0 network Ethernet Controller XL710 for 40GbE QSFP+ pci@0000:86:00.1 ens5f1 network Ethernet Controller XL710 for 40GbE QSFP+ pci@0000:86:00.2 ens5f2 network Ethernet Controller XL710 for 40GbE QSFP+ pci@0000:86:00.3 ens5f3 network Ethernet Controller XL710 for 40GbE QSFP+ pci@0000:af:00.0 ens4f0 network Ethernet Controller XL710 for 40GbE QSFP+ pci@0000:af:00.1 ens4f1 network Ethernet Controller XL710 for 40GbE QSFP+ pci@0000:af:00.2 ens4f2 network Ethernet Controller XL710 for 40GbE QSFP+ pci@0000:af:00.3 ens4f3 network Ethernet Controller XL710 for 40GbE QSFP+
There are standard ways to persist this across system restart. The following is an example script that is installed as a system service:
[root@rhel-tiger-14-6 ~]# touch /usr/local/bin/tiger-nic-vf-handler [root@rhel-tiger-14-6 ~]# chown root:root /usr/local/bin/tiger-nic-vf-handler [root@rhel-tiger-14-6 ~]# chmod 755 /usr/local/bin/tiger-nic-vf-handler [root@rhel-tiger-14-6 ~]# cat /usr/local/bin/tiger-nic-vf-handler #!/bin/bash numvfs=2 lookup() { if [[ -z $1 ]] ; then echo "" else awk -v "id=$1" 'BEGIN { FS = "=" } $1 == id { print $2 ; exit }' $2 fi } start() { for _ethdev in ${ethlist//;/ } ; do ethdev=${_ethdev#*:} echo $numvfs > /sys/class/net/${ethdev}/device/sriov_numvfs sleep 2 for ((vfnum=0;vfnum<${numvfs};vfnum++)); do ip link set $ethdev vf $vfnum spoofchk off trust on for iface in $(ls /sys/class/net) ; do if [[ $(readlink /sys/class/net/${iface}) =~ $(readlink /sys/class/net/${ethdev}/device/virtfn${vfnum} xargs -n 1 basename) ]] ; then nmcli device set $iface managed no fi done done ip link set dev $ethdev up done } stop() { for _ethdev in ${ethlist//;/ } ; do ethdev=${_ethdev#*:} echo 0 > /sys/class/net/${ethdev}/device/sriov_numvfs done } ethlist=$(lookup unmanaged-devices /etc/NetworkManager/conf.d/10-tiger.conf case $1 in startstop) "$1" ;; esac [root@rhel-tiger-14-6 ~]# touch /etc/systemd/system/tiger-nic-vf-handler.service [root@rhel-tiger-14-6 ~]# chown root:root /etc/systemd/system/tiger-nic-vf-handler.service [root@rhel-tiger-14-6 ~]# chmod 644 /etc/systemd/system/tiger-nic-vf-handler.service [root@rhel-tiger-14-6 ~]# cat /etc/systemd/system/tiger-nic-vf-handler.service [Unit] Description=Create/Destroy unmanaged virtual functions and NetworkManager Before=libvirtd.service [Service] Type=oneshot ExecStart=/usr/local/bin/tiger-nic-vf-handler start ExecStop=/usr/local/bin/tiger-nic-vf-handler stop RemainAfterExit=yes [Install] WantedBy=multi-user.target [root@rhel-tiger-14-6 ~]# systemctl enable --now tiger-nic-vf-handler Created symlink /etc/systemd/system/multi-user.target.wants/tiger-nic-vf-handler.service → /etc/systemd/system/tiger-nic-vf-handler.service.
The script simply sources the list of unmanaged interfaces from the NetworkManager configuration and sets up the VFs accordingly. The filename in the script must match the filename created earlier. You may want to update the number of VFs created with this script by updating the script variable accordingly. The script is set to run before the libvirtd service is started to allow dependent configuration to be persisted.
The The |
You may create the VFs as follows. Note the PCI addresses:
[root@rhel-tiger-14-6 ~]# lspci egrep "Eth.+Virt" 86:02.0 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02) 86:02.1 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02) 86:06.0 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02) 86:06.1 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02) 86:0a.0 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02) 86:0a.1 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02) 86:0e.0 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02) 86:0e.1 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02) af:02.0 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02) af:02.1 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02) af:06.0 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02) af:06.1 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02) af:0a.0 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02) af:0a.1 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02) af:0e.0 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02) af:0e.1 Ethernet controller: Intel Corporation Ethernet Virtual Function 700 Series (rev 02
Note that you can see the mapping of PF to VF as follows:
[root@rhel-tiger-14-6 ~]# ls -l /sys/class/net/ens4f0/device/virtfn* lrwxrwxrwx 1 root root 0 Nov 17 22:44 /sys/class/net/ens4f0/device/virtfn0 -> ../0000:af:02.0 lrwxrwxrwx 1 root root 0 Nov 17 22:44 /sys/class/net/ens4f0/device/virtfn1 -> ../0000:af:02.1