Security Bulletin: NVIDIA Container Toolkit - 13 January 2025

Updated 01/28/2025 02:50 PM

NVIDIA has released a software update for NVIDIA® Container Toolkit and NVIDIA GPU Operator. To protect your system, install the software update as described in the installation section of the NVIDIA Container Toolkit documentation and the NVIDIA GPU Operator documentation.

Go to NVIDIA Product Security.


Details

This section provides a summary of potential vulnerabilities that this security update addresses and their impact. Descriptions use CWE™, and base scores and vectors use CVSS v3.1 standards.

CVE ID Description Vector Base Score Severity CWE Impacts
CVE-2024-0135 NVIDIA Container Toolkit contains an improper isolation vulnerability where a specially crafted container image could lead to modification of a host binary. A successful exploit of this vulnerability may lead to code execution, denial of service, escalation of privileges, information disclosure, and data tampering. AV:N/AC:H/PR:H/UI:R/S:C/C:H/I:H/A:H 7.6 High CWE‑653 Code execution, denial of service, escalation of privileges, information disclosure, data tampering
CVE-2024-0136 NVIDIA Container Toolkit contains an improper isolation vulnerability where a specially crafted container image could lead to untrusted code obtaining read and write access to host devices. This vulnerability is present only when the NVIDIA Container Toolkit is configured in a nondefault way. A successful exploit of this vulnerability may lead to code execution, denial of service, escalation of privileges, information disclosure, and data tampering. AV:N/AC:H/PR:H/UI:R/S:C/C:H/I:H/A:H 7.6 High CWE‑653 Code execution, denial of service, escalation of privileges, information disclosure, data tampering
CVE-2024-0137 NVIDIA Container Toolkit contains an improper isolation vulnerability where a specially crafted container image could lead to untrusted code running in the host’s network namespace. This vulnerability is present only when the NVIDIA Container Toolkit is configured in a nondefault way. A successful exploit of this vulnerability may lead to denial of service and escalation of privileges. AV:N/AC:H/PR:L/UI:R/S:C/C:L/I:L/A:L 5.5 Medium CWE‑653 Denial of service, escalation of privileges

The NVIDIA risk assessment is based on an average of risk across a diverse set of installed systems and may not represent the true risk to your local installation. NVIDIA recommends evaluating the risk to your specific configuration.

Security Updates

The following table lists the software products and versions affected, and the updated version available from nvidia.com that includes this security update.

To protect your system, install the software update as described in the installation section of the NVIDIA Container Toolkit documentation and the NVIDIA GPU Operator documentation.

CVE IDs Addressed Affected Products Platform or OS Affected Versions Updated Version
CVE‑2024-0135
CVE‑2024-0136
CVE‑2024-0137
NVIDIA Container Toolkit Linux All versions up to and including v1.17.2 v1.17.3
NVIDIA GPU Operator Linux All versions up to and including 24.9.0 24.9.1

Mitigations

Customer Required Mitigations for CVE-2024-0136 and CVE-2024-0137

To mitigate CVE-2024-0136 and CVE-2024-0137, ensure that the NVIDIA Container Toolkit is configured in the default way. Specifically, ensure that in the /etc/nvidia-container-runtime/config.toml file, the NVIDIA Container Toolkit is configured as follows to use the host’s ldconfig binary:

[nvidia-container-cli]
ldconfig = "@/sbin/ldconfig"

Note the @ in the ldconfig binary path. The @ prefix indicates that the path is relative to the host’s file system and not the container’s file system. 

Configuring the NVIDIA Container Toolkit this way prevents the execution of untrusted code and protects systems from CVE-2024-0136 and CVE-2024-0137.

Systems are vulnerable to CVE-2024-0136 and CVE-2024-0137 only if the NVIDIA Container Toolkit is configured in a non-default way, specifically to run the ldconfig binary in the container’s file system.

To reduce the likelihood of the NVIDIA Container Toolkit being inadvertently configured in an unsafe manner, the v1.17.3 release of the NVIDIA Container Toolkit will change the default behavior of the NVIDIA Container Toolkit to disallow  ldconfig paths that are not relative to the host’s file system. An explicit opt-in feature flag will be added to allow the execution of possibly unsafe code from a container if required. 

Preventing Unprivileged Containers from Bypassing the Kubernetes Device Plugin API

The default installation of NVIDIA GPU Operator might allow unprivileged containers to bypass the Kubernetes Device Plugin API and gain access to GPUs on a node if the NVIDIA_VISIBLE_DEVICES environment variable is set. Unprivileged containers might gain access to GPUs without specifying resource requests or limits, or gain access to more GPUs than were originally specified through resource requests or limits.

To prevent unprivileged containers from bypassing the Kubernetes Device Plugin API, configure the NVIDIA GPU Operator in the clusterpolicy custom resource definition as follows:

toolkit:
  env:
  - name: ACCEPT_NVIDIA_VISIBLE_DEVICES_ENVVAR_WHEN_UNPRIVILEGED
    value: "false"
  - name: ACCEPT_NVIDIA_VISIBLE_DEVICES_AS_VOLUME_MOUNTS
    value: "true"
devicePlugin:
   env:
   - name: DEVICE_LIST_STRATEGY
     value: volume-mounts
This configuration ensures that: 
  • The NVIDIA Container Toolkit ignores the NVIDIA_VISIBLE_DEVICES environment variable for unprivileged containers.
  • The NVIDIA Kubernetes Device Plugin produces a list of GPU devices as volume mounts, according to resource requests or limits, and the NVIDIA Container Toolkit accepts only this list. 

Notes

Earlier software releases of this product are also affected. If you are using an earlier release, upgrade to the latest release version.
 
To protect your system, install the software update as described in the installation section of the NVIDIA Container Toolkit documentation and the NVIDIA GPU Operator documentation.

Acknowledgements

NVIDIA thanks Andres Riancho, Ronen Shustin, and Shir Tamari from Wiz Research for reporting issues CVE-2024-0135, CVE-2024-0136, and CVE-2024-0137.

Get the Most Up-to-Date Product Security Information

Visit the NVIDIA Product Security page to

  • Subscribe to security bulletin notifications
  • See the current list of NVIDIA security bulletins
  • Report a potential security issue in any NVIDIA supported product
  • Learn more about the vulnerability management process followed by the NVIDIA Product Security Incident Response Team (PSIRT)

Revision History

Revision Date Description
1.0 13 January 2025 Initial release

Support

If you have any questions about this security bulletin, contact NVIDIA Support.

Disclaimer

ALL NVIDIA INFORMATION, DESIGN SPECIFICATIONS, REFERENCE BOARDS, FILES, DRAWINGS, DIAGNOSTICS, LISTS, AND OTHER DOCUMENTS (TOGETHER AND SEPARATELY, “MATERIALS”) ARE BEING PROVIDED “AS IS.” NVIDIA MAKES NO WARRANTIES, EXPRESS, IMPLIED, STATUTORY, OR OTHERWISE WITH RESPECT TO THE MATERIALS, AND ALL EXPRESS OR IMPLIED CONDITIONS, REPRESENTATIONS AND WARRANTIES, INCLUDING ANY IMPLIED WARRANTY OR CONDITION OF TITLE, MERCHANTABILITY, SATISFACTORY QUALITY, FITNESS FOR A PARTICULAR PURPOSE AND NON-INFRINGEMENT, ARE HEREBY EXCLUDED TO THE MAXIMUM EXTENT PERMITTED BY LAW.

Information is believed to be accurate and reliable at the time it is furnished. However, NVIDIA Corporation assumes no responsibility for the consequences of use of such information or for any infringement of patents or other rights of third parties that may result from its use. No license is granted by implication or otherwise under any patent or patent rights of NVIDIA Corporation. Specifications mentioned in this publication are subject to change without notice. This publication supersedes and replaces all information previously supplied. NVIDIA Corporation products are not authorized for use as critical components in life support devices or systems without express written approval of NVIDIA Corporation.

Is this answer helpful?

Live Chat

Chat online with one of our support agents

CHAT NOW

ASK US A QUESTION

Contact Support for assistance

800.797.6530

Ask a Question