Oracle Exadata Database Machine with respect CVE-2017-5753 (Spectre v1), CVE-2017-5715 (Spectre v2), and CVE-2017-5754 (Meltdown) Intel processor vulnerabilities Status:
Note 1: Install updated ACFS drivers before updating Exadata database servers to >= 18.1.5.0.0 or >= 12.2.1.1.7, and use the most recent patchmgr/dbserver.patch (Patch 21634633 version 18.1.5.0.0 / 5.180529 or later) to perform the database server updates. Updated ACFS drivers are needed whether or not ACFS is in use in order for the most efficient CVE-2017-5715 mitigation to be in place once >= 18.1.5.0.0 or >= 12.2.1.1.7 is installed. See section Required ACFS Driver Updates below for details.
Note 2: The InfiniBand Switch component (NM2-36P) is not currently believed to be impacted by these vulnerabilities.
Required ACFS Driver Updates
Install updated ACFS drivers before updating Exadata database servers to >= 18.1.5.0.0 or >= 12.2.1.1.7. Note the following:- Updated ACFS drivers are needed for the most efficient CVE-2017-5715 mitigation to be in place once >= 18.1.5.0.0 or >= 12.2.1.1.7 is installed.
- Updated ACFS drivers are needed whether or not ACFS is in use.
- In an OVM configuration dom0 may be updated to >= 18.1.5.0.0 or >= 12.2.1.1.7 before the ACFS drivers are updated in domU.
- Grid infrastructure 18.3.0.0.180717 and later version 18 quarterly updates
- Grid infrastructure 12.2.0.1.180717 and later version 12.2.0.1 quarterly updates
- Grid infrastructure 12.1.0.2.180831 and later version 12.1.0.2 quarterly updates
- Grid infrastructure 12.1.0.2.180717 plus patch 23312691
To verify your systems have the proper mitigations in place after the ACFS driver and Exadata updates are performed, run the following command:
$ grep . /sys/devices/system/cpu/vulnerabilities/*
The expected output depends on the hardware version, server type, and system configuration. If the output shown on your system does not match X7 hardware - storage server and database server (non-OVM and domU)
Expected vulnerability mitigation output:
$ grep . /sys/devices/system/cpu/vulnerabilities/*
/sys/devices/system/cpu/vulnerabilities/meltdown:Mitigation: PTI
/sys/devices/system/cpu/vulnerabilities/spectre_v1:Mitigation: lfence
/sys/devices/system/cpu/vulnerabilities/spectre_v2:Mitigation: IBRS, IBRS_FW, IBPB
/sys/devices/system/cpu/vulnerabilities/meltdown:Mitigation: PTI
/sys/devices/system/cpu/vulnerabilities/spectre_v1:Mitigation: lfence
/sys/devices/system/cpu/vulnerabilities/spectre_v2:Mitigation: IBRS, IBRS_FW, IBPB
X6 and earlier hardware - storage server and database server (non-OVM and domU)
Expected vulnerability mitigation output:
$ grep . /sys/devices/system/cpu/vulnerabilities/*
/sys/devices/system/cpu/vulnerabilities/meltdown:Mitigation: PTI
/sys/devices/system/cpu/vulnerabilities/spectre_v1:Mitigation: lfence
/sys/devices/system/cpu/vulnerabilities/spectre_v2:Mitigation: Full generic retpoline, IBRS_FW, IBPB
/sys/devices/system/cpu/vulnerabilities/meltdown:Mitigation: PTI
/sys/devices/system/cpu/vulnerabilities/spectre_v1:Mitigation: lfence
/sys/devices/system/cpu/vulnerabilities/spectre_v2:Mitigation: Full generic retpoline, IBRS_FW, IBPB
All hardware - database server (dom0)
Expected vulnerability mitigation output:
# grep . /sys/devices/system/cpu/vulnerabilities/*
/sys/devices/system/cpu/vulnerabilities/meltdown:Vulnerable
/sys/devices/system/cpu/vulnerabilities/spectre_v1:Mitigation: lfence
/sys/devices/system/cpu/vulnerabilities/spectre_v2:Mitigation: Full generic retpoline
Troubleshooting unexpected mitigation
If Spectre v2 mitigation does not match the expected output shown above, then confirm "imageinfo -ver" indicates the installed Exadata versions is >= 18.1.5.0.0 or >= 12.2.1.1.7. If the proper Exadata version is installed, then follow these guidelines:Storage servers
This situation should never occur on storage servers. Contact Oracle Support.
Database servers (non-OVM and domU)
Perform the following verification steps on each database server:
- Verify the patch containing the updated ACFS drivers has been installed in the grid infrastructure home. If the following OPatch command does not return output then the ACFS patch is not installed. $ <gihome>/grid/OPatch/opatch lsinventory -bugs_fixed | grep ^27463879
27463879 27463879 Tue May 15 20:31:10 UTC 2018 TRACKING BUG FOR RECOMPILING USM DRIVERS WITH
If the patch is not installed, perform the following steps: 1) install the ACFS patch, as indicated in the "Required ACFS Driver Updates" section above; 2) reboot (required for the kernel to re-enable proper mitigation); 3) verify proper mitigation. This may be done in a rolling manner.
If the patch is installed, but it was installed after Exadata software was upgraded to >= 18.1.5.0.0 or >= 12.2.1.1.7, then reboot is required for the kernel to re-enable proper mitigation.
Otherwise, proceed to the next troubleshooting step. - Verify the state of the ACFS driver using the acfsdriverstate command.$ <gihome>/bin/acfsdriverstate version
ACFS-9325: Driver OS kernel version = 4.1.12-94.8.2.el6uek.x86_64.
If Driver OS kernel version = 4.1.12-32, then the updated ACFS driver is not loaded. This scenario may occur if the ACFS patch was installed before database server update, but the database server update was performed with an older patchmgr/dbserver.patch. To load the proper ACFS driver perform the following steps as root: 1) stop clusterware; 2) run "<gihome>/bin/acfsroot install" to load the proper drivers; 3) reboot (required for the kernel to re-enable proper mitigation); 4) verify proper mitigation. This may be done in a rolling manner.
If Driver OS kernel version = 4.1.12-94.8.2 or a higher kernel version (note that this will not necessarily match the installed kernel version), then the correct driver is loaded. Proceed to the next troubleshooting step. - Review /var/log/messages for output referring to "Spectre V2" logged shortly after the time for the current system boot, indicating a module is loaded "not compiled with retpoline compiler", similar to the following:kernel: badmodule: loading module not compiled with retpoline compiler.
kernel: Spectre V2 : Disabling Spectre v2 mitigation retpoline.
kernel: Spectre V2 : Spectre v2 mitigation set to IBRS.
If /var/log/messages does not contain any message referring to "Spectre V2" then one of the following conditions exist: 1) /var/log/messages file was rotated - review an older messages file; or 2) Exadata software has not been upgraded to >= 18.1.5.0.0 or >= 12.2.1.1.7.
If the message reported is "oracleoks: loading module not compiled with retpoline compiler", then it indicates the proper ACFS driver is not in place. Review the previous troubleshooting steps.
If the message refers to any other module, then it is likely caused by user-installed software that supplies a module that has not been compiled with a retpoline-aware compiler. Contact the vendor of that kernel module to obtain an update.
For retpoline mitigation to be active, kernel modules/drivers that contain code needing retpolines must be compiled with a retpoline-aware compiler. Loading a module needing retpolines that was not compiled with a retpoline-aware compiler (e.g. an older ACFS driver, or a third-party module) will cause the kernel to disable retpoline mitigation systemwide, and fallback to a different mitigation (e.g. IBRS), which may have higher than expected performance impact on some systems. Review /var/log/messages, as shown above, for output showing the kernel disabling retpoline because a module was not compiled with a retpoline-aware compiler. All kernel modules delivered with Exadata 18.1.5.0.0 and 12.2.1.1.7 have been compiled with a retpoline-aware compiler. The updated ACFS drivers discussed above have been compiled with a retpoline-aware compiler. - If the previous troubleshooting steps do not resolve the issue, then Contact Oracle Support.
No comments:
Post a Comment