NIC versions
NICs generally have three components to consider when considering SR-IOV:
- Firmware (or NVM)
- PF driver
- VF driver
|
Care should be taken to ensure the drivers and firmware/non-volatile memory image are aligned as per manufacturer’s recommendations. For example, the Intel X700 series recommendations are detailed in the Feature Support Matrix. |
The VF driver is part of the FortiGate-VM instance. The driver versions are documented here. There is only one VF driver version per FortiOS version. This offers the least flexibility when aligning versions, making it the starting point.
As an example, the system used in this document has Intel XXV710 NIC cards and is running FortiOS 7.0.5. Consulting the two resources above:
- VF driver: IAVF 4.1.1
- Software Release Version: 26.2
- PF driver: i40en 1.13.1.0
- NVM version: 8.30
|
|
Any deviation from this alignment would need to be diligently tested and may still cause later supportability issues. |
Instructions for Mellanox public repositories can be found at Mellanox Technologies Ltd. Public Repository. You can find firmware update instructions at Firmware Update Instructions.
Having said that, the example system in this document breaks this rule. As it is not carrying production traffic, then it allows future versions to be considered without any negative impact. The example prints are based upon Intel’s Software Release Version 26.6 which importantly contains some fixes for issues observed on other hypervisors.
[root@esxi-tiger-14-7:~] esxcli network nic get -n vmnic4
Advertised Auto Negotiation: true
Advertised Link Modes: Auto, 10000BaseSR/Full, 25000BaseSR/Full
Auto Negotiation: true
Cable Type: FIBRE
Current Message Level: 0
Driver Info:
Bus Info: 0000:3b:00:0
Driver: i40en
Firmware Version: 8.50 0x8000b703 1.3082.0
Version: 2.1.5.0
Link Detected: true
Link Status: Up
Name: vmnic4
PHYAddress: 0
Pause Autonegotiate: false
Pause RX: false
Pause TX: false
Supported Ports: FIBRE
Supports Auto Negotiation: true
Supports Pause: true
Supports Wakeon: false
Transceiver:
Virtual Address: 00:50:56:54:6f:71
Wakeon: None
|
|
ESXi 7.0U2 packages the i40enu driver version. This is not helpful and is something that VMware has retracted. |
Search, download, and transfer to the versions desired to the server:
-
Google: intel xl710 nvm 8.50
-
VMware Compatibility Guide
Compile and install the driver, following the vendor's instructions:
Note the following:
- The NIC features are listed. This includes SR-IOV.
- The driver version shown is 2.1.5.0, which does not align with the Intel matrix shown earlier, but is the closest available to download.
- The firmware versions can be poorly handled. 8.5 is 8.50.
- There is a more recent version available which you could consider. The problems that we previously experienced are reported fixed in the version chosen. Furthering to the latest may cause issue with VF driver compatibility.
This further reinforces the need to align NIC versions and diligently test the solution.
Remove the i40enu driver and install the i40en driver, following the vendor's instructions:
[root@esxi-tiger-14-7:/vmfs/volumes/6216bfd4-deba7d2c-3ae7-e4434b314530/Tiger] unzip Intel-i40en_2.1.5.0-1OEM.700.1.0.15843807_18631754-package.zip Archive: Intel-i40en_2.1.5.0-1OEM.700.1.0.15843807_18631754-package.zip inflating: Intel-i40en_2.1.5.0-1OEM.700.1.0.15843807_18631754.zip inflating: doc/README.txt inflating: doc/release_note_i40en_2.1.5.0-1OEM.700.1.0.15843807.txt [root@esxi-tiger-14-7:/vmfs/volumes/6216bfd4-deba7d2c-3ae7-e4434b314530/Tiger] unzip Intel-i40en_2.1.5.0-1OEM.700.1.0.15843807_18631754.zip Archive: Intel-i40en_2.1.5.0-1OEM.700.1.0.15843807_18631754.zip inflating: index.xml inflating: vendor-index.xml inflating: metadata.zip inflating: vib20/i40en/INT_bootbank_i40en_2.1.5.0-1OEM.700.1.0.15843807.vib [root@esxi-tiger-14-7:~] cd [root@esxi-tiger-14-7:~] esxcli software vib list grep i40enu i40enu 1.8.1.137-1vmw.702.0.20.18426014 VMW VMwareCertified 2022-03-03 [root@esxi-tiger-14-7:~] esxcli software vib remove -n i40enu Removal Result Message: The update completed successfully, but the system needs to be rebooted for the changes to be effective. Reboot Required: true VIBs Installed: VIBs Removed: VMW_bootbank_i40enu_1.8.1.137-1vmw.702.0.20.18426014 VIBs Skipped [root@esxi-tiger-14-7:~] esxcli software vib install -v /vmfs/volumes/ESXI-TIGER-14-7/Tiger/vib20/i40en/INT_bootbank_i40en_2.1.5.0-1OEM.700.1.0.15843807.vib Installation Result Message: The update completed successfully, but the system needs to be rebooted for the changes to be effective. Reboot Required: true VIBs Installed: INT_bootbank_i40en_2.1.5.0-1OEM.700.1.0.15843807 VIBs Removed: VIBs Skipped: [root@esxi-tiger-14-7:~] reboot
Upgrade the firmware, following the vendor's instructions:
[root@esxi-tiger-14-7:/vmfs/volumes/6216bfd4-deba7d2c-3ae7-e4434b314530/Tiger] unzip 700Series_NVMUpdatePackage_v8_50.zip Archive: 700Series_NVMUpdatePackage_v8_50.zip inflating: 700Series_NVMUpdatePackage_v8_50_EFI.zip inflating: 700Series_NVMUpdatePackage_v8_50_ESX.tar.gz inflating: 700Series_NVMUpdatePackage_v8_50_FreeBSD.tar.gz inflating: 700Series_NVMUpdatePackage_v8_50_Linux.tar.gz inflating: 700Series_NVMUpdatePackage_v8_50_Windows.zip [root@esxi-tiger-14-7:/vmfs/volumes/6216bfd4-deba7d2c-3ae7-e4434b314530/Tiger] tar xzf 700Series_NVMUpdatePackage_v8_50_ESX.tar.gz [root@esxi-tiger-14-7:/vmfs/volumes/6216bfd4-deba7d2c-3ae7-e4434b314530/Tiger] cd 700Series/ESXi_x64/ [root@esxi-tiger-14-7:/vmfs/volumes/6216bfd4-deba7d2c-3ae7-e4434b314530/Tiger/700Series/ESXi_x64] ./nvmupdaten64e <output omitted for brevity> [root@esxi-tiger-14-7:/vmfs/volumes/6216bfd4-deba7d2c-3ae7-e4434b314530/Tiger/700Series/ESXi_x64] reboot
You can check the results via the previous esxcli network nic get command or via esxcli system module get:
[root@esxi-tiger-14-7:~] esxcli system module get -m i40en Module: i40en Module File: /usr/lib/vmware/vmkmod/i40en License: ThirdParty:Intel Version: 2.1.5.0-1OEM.700.1.0.15843807 Build Type: release Provided Namespaces: Required Namespaces: com.vmware.vmkapi@v2_6_0_0 Containing VIB: i40en VIB Acceptance Level: certified