Skip to content

fix(cuda.core): make IPC imports wait for a pending release of the same descriptor - #3450

Merged
rparolin merged 5 commits into
NVIDIA:mainfrom
rparolin:fix/ipc-import-release-window
Nov 16, 2026
Merged

rparolin merged 5 commits into
NVIDIA:mainfrom
rparolin:fix/ipc-import-release-window

Conversation

@rparolin

@rparolin rparolin commented Nov 12, 2026 •

Copy link
Copy Markdown
Collaborator

Description

deviceptr_import_ipc() shares one handle per descriptor, because the driver's first free of an imported pointer unmaps every import of it. The deleter took ipc_import_mutex only after the handle's last reference was gone. In that gap, an import on another thread could:

  1. find the cache entry expired,
  2. re-import the still-mapped pointer, and
  3. register a new entry.

The deleter then erased the new entry and freed the pointer, which unmapped it under the new Buffer.

Fix (memory.cpp):

  • The cache entry outlives the last reference. Only its own deleter erases it, under the mutex, together with the free.
  • An import that finds an expired entry waits on a condition variable until that release is done, then imports fresh.
  • cleanup_in_context always calls the after_cleanup hook, and the hook notifies the waiters. Every exit path therefore wakes them, including activation and driver failures.

Validation

Test Without the fix With the fix
Deterministic window test Fails 50/50 Passes 50/50
8-thread stress test Re-imports a mapped pointer every run 0 failures, plain and TSan builds, no TSan reports
  • New test tests/memory_ipc/test_ipc_import_release_race.py: threads in one child import, use and close one descriptor in a loop. It catches a hang (missed wakeup) and a lost mapping (failed copy). On real hardware the race did not trigger on the unpatched build either (about 6M cycles on an RTX 5880 Ada, driver 595.99), so this test guards the new waiting path rather than reproducing the original bug.

Performance: no measurable regression. Each cell is the median ns per import, n=20, before → after, measured on the final commit. The GPU runs alternate between builds.

Case Fake driver, -O2 GPU, optimized wheel
Live entry, 1 thread 57 → 52 491 → 480
Live entry, 4–8 threads 477 → 354 3993 → 4085
Import + release, 1 thread 604 → 607 1329 → 1323
Import + release, 4–8 threads 752 → 587 3904 → 4088

🤖 Generated with Claude Code

…me descriptor

deviceptr_import_ipc() shares one handle per descriptor because the
driver's first free of an imported pointer unmaps every import of it
(nvbug 5570902). The handle's deleter took ipc_import_mutex only after
the last reference was gone. In that gap an import found the cache entry
expired, re-imported the still-mapped pointer and registered a new entry.
The deleter then erased that new entry and freed the pointer, which
unmapped it under the new Buffer.

The cache entry now outlives the last reference and only its own deleter
erases it, under the mutex, together with the free. An import that finds
an expired entry waits on a condition variable until that release is
done, then imports fresh.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@rparolin rparolin added this to the cuda.core next milestone Nov 12, 2026
@rparolin rparolin added bug Something isn't working cuda.core Everything related to the cuda.core module labels Nov 12, 2026
@rparolin rparolin self-assigned this Nov 12, 2026
@copy-pr-bot

copy-pr-bot Bot commented Nov 12, 2026

Copy link
Copy Markdown
Contributor

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@rparolin
rparolin marked this pull request as ready for review November 12, 2026 19:17
@rparolin
rparolin enabled auto-merge (squash) November 12, 2026 19:18
@rparolin
rparolin requested a review from Andy-Jost November 12, 2026 19:18
@coderabbitai

coderabbitai Bot commented Nov 12, 2026 •

Copy link
Copy Markdown

Explain Change →

Important

Review skipped

Review was skipped as selected files did not have any reviewable changes.

Run configuration

Commits

Reviewing files that changed from the base of the PR and between 43f6ea4 and 9711882.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • Trigger review
Summary

Summary by CodeRabbit

  • Bug Fixes
    • Improved reliability when repeated imports of the same shared GPU memory descriptor overlap with the final handle release. Imports now share a handle, and any import arriving during release waits for it to finish, helping prevent hangs and invalid memory mappings.
  • Tests
    • Added stress coverage for concurrent imports, data copies, and releases, including checks for stalled threads and processes.

The IPC import cache now coordinates imports with release of the last handle. A new multiprocessing stress test exercises concurrent imports and releases.

Changes: IPC Import Release Synchronization

Layer / File(s) Summary
IPC import cache synchronization
cuda_core/cuda/core/_cpp/rt/api.hpp, cuda_core/cuda/core/_cpp/rt/memory.cpp
The comments document shared handles and waiting for an in-progress free. The cache tracks weak handles and their owning boxes. Importers wait on expired entries until the deleter frees the mapping, erases the entry, and notifies waiters.
Concurrent import and release stress test
cuda_core/tests/memory_ipc/test_ipc_import_release_race.py
A spawned child runs four workers that repeatedly import the descriptor, copy data, and close buffers. The parent checks worker completion, child exit status, and join timeout.

Change: Bug fix

Merge Risk: Low · previous review 43f6e

The new IPC import/release synchronization has no established production defect. A new stress test could leave a child process running when the test exits abnormally. This is a small test-hygiene fix and is safe to follow up on.

Security architecture risk: Low · up to 43f6e

The change closes a mapping-lifetime race without expanding access permissions. A narrow reentrant-warning path may still stall final release and subsequent imports of the same descriptor.

Retained concerns

  • Low · reliability · inferred: The new wait protocol can self-deadlock if final release on a different host thread emits the per-thread-default-stream warning and a synchronous warning handler imports the same descriptor. Context resolution invokes the warning before the deleter removes its expired cache entry; the nested import waits for that removal, while the deleter cannot proceed until the handler returns. This can strand mapping cleanup and other importers of that descriptor. It requires application-side reentrancy and has not been reproduced at runtime.
Security review details

Security Blast Radius

  • inferred — The directly affected ownership domain is callers sharing a descriptor in one importing process. Cache coordination does not add privileges or create a new cross-service entrypoint. The inferred reentrancy stall affects the releasing call and imports waiting on that descriptor; no broader tenant or environment exposure is established.

Trust Boundaries and Controls

  • observed — The new test's descriptor and allocation size originate from its own parent allocation and travel over a queue to its own child. Its process-kill helper receives that child process object. These routed test entrypoints do not introduce an external descriptor source or general process-control interface.

Resilience and Maintainability Implications

  • observed — Context activation and driver failures still reach notification and unlock, preventing those completed failure paths from leaving waiters asleep. Cleanup can nevertheless leak the mapping on failure, as before; notification is a progress mechanism rather than a successful-cleanup guarantee.
- [ ] Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Review info

Run configuration

Commits

Reviewing files that changed from the base of the PR and between e119c31 and 595dc19.

Files selected for processing: 3

  • cuda_core/cuda/core/_cpp/rt/api.hpp
  • cuda_core/cuda/core/_cpp/rt/memory.cpp
  • cuda_core/tests/memory_ipc/test_ipc_import_release_race.py

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 11 remain after this review.

@github-actions

github-actions Bot commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor
Doc Preview CI
Preview removed because the pull request was closed or merged.

MSVC 14.44 cannot deduce std::unique_lock's template argument for a lock
declared inside the deleter lambda and used from the nested after_cleanup
lambda (C2955, C2662). The import path keeps CTAD.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@rparolin

Copy link
Copy Markdown
Collaborator Author

/ok to test

Wrap the child-process checks in try/finally so a failed assertion no
longer leaks the exported buffer (CodeRabbit review note).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@rparolin

Copy link
Copy Markdown
Collaborator Author

/ok to test

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Review info

Run configuration

Commits

Reviewing files that changed from the base of the PR and between af2b502 and 43f6ea4.

Files selected for processing: 1

  • cuda_core/tests/memory_ipc/test_ipc_import_release_race.py

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 11 remain after this review.

Comment on lines +112 to +113
finally:
buffer.close(stream)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor · Stability & Availability · Quick win

important: Stop the child before closing the exported buffer.

If queue population raises or process.join() receives KeyboardInterrupt, execution skips kill_subprocesses. This finally then closes the exported buffer while the child can remain blocked or continue using its imports. Terminate and join any started child in finally before closing buffer.

As per coding guidelines, use “track_child_processes for fixture cleanup so a stuck child cannot block IPC resource teardown.”

Source: Coding guidelines

@Andy-Jost Andy-Jost left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving. I went through the fix and found no problem with it: an import can't get past an expired entry until that entry's own deleter has erased it and freed the pointer, and every exit path wakes the waiters. Thanks for finding this.

Please add a release note for the fix.

@rparolin
rparolin merged commit 2f1185e into NVIDIA:main Nov 16, 2026
117 checks passed
rparolin added a commit to rparolin/cuda-python that referenced this pull request Nov 16, 2026
1.3.0 was tagged before NVIDIA#3450 merged, so the fix ships in 1.3.1.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Andy-Jost pushed a commit to Andy-Jost/cuda-python that referenced this pull request Nov 17, 2026
…me descriptor (NVIDIA#3450)

* fix(cuda.core): make IPC imports wait for a pending release of the same descriptor

deviceptr_import_ipc() shares one handle per descriptor because the
driver's first free of an imported pointer unmaps every import of it
(nvbug 5570902). The handle's deleter took ipc_import_mutex only after
the last reference was gone. In that gap an import found the cache entry
expired, re-imported the still-mapped pointer and registered a new entry.
The deleter then erased that new entry and freed the pointer, which
unmapped it under the new Buffer.

The cache entry now outlives the last reference and only its own deleter
erases it, under the mutex, together with the free. An import that finds
an expired entry waits on a condition variable until that release is
done, then imports fresh.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Use CTAD for the ipc_import_mutex locks

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 2f1185e)
rparolin added a commit that referenced this pull request Nov 17, 2026
…ce fix (#3479)

* docs(cuda.core): add 1.3.0 release note for the IPC import/release race fix

Follow-up to #3450, which merged without a release note.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(cuda.core): scope the IPC race release note to memory pool buffers

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Andy-Jost pushed a commit to Andy-Jost/cuda-python that referenced this pull request Nov 17, 2026
…ce fix (NVIDIA#3479)

* docs(cuda.core): add 1.3.0 release note for the IPC import/release race fix

Follow-up to NVIDIA#3450, which merged without a release note.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(cuda.core): scope the IPC race release note to memory pool buffers

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit de4c60d)
github-actions Bot pushed a commit that referenced this pull request Nov 17, 2026
Removed preview folders for the following PRs:
- PR #3359
- PR #3446
- PR #3448
- PR #3450
- PR #3464
- PR #3478
- PR #3479
Andy-Jost added a commit that referenced this pull request Nov 18, 2026
* [doc-only] docs(cuda.core): fix the floor-move wording; advise against pinning cuda-bindings (#3441)

* docs(cuda.core): drop the Breaking Changes section name from the floor policy

The release notes no longer have a Breaking Changes section for a floor
move; 1.3.0 lists it under Highlights. Say only that the release notes
list every move.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(cuda.core): advise against pinning cuda-bindings next to cuda-core

Users who pin cuda-bindings themselves can conflict with the floor of a
later cuda-core release. Tell them to depend on cuda-core with the cu12
or cu13 extra and let cuda-core choose the cuda-bindings version.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 83a3422)

* Mark cuda-gdb source-line test thread_unsafe (#3433)

Signed-off-by: Ralf Juengling <rjuengling@nvidia.com>
(cherry picked from commit 1a22eec)

* fix(cuda.core): serialize pinned-memory kernel-access test on Windows (#3436)

* fix(cuda.core): serialize pinned-memory kernel-access test on Windows

closes #3426

Add @thread_unsafe_on_windows to the pinned-memory kernel-access test.
The test accesses host memory while parallel workers may execute GPU
work; synchronizing its own stream does not coordinate those workers.

This follows the guard used by similar tests on Windows and WSL.
Windows CI is needed to confirm it resolves the reported crashes.
The separate OOM failures remain under investigation.

Validation: Ruff and git diff --check pass.

* fix(cuda.core): report CUDA synchronization errors during test teardown

Check the cuCtxSynchronize result in init_cuda so asynchronous errors
surface during the triggering test's teardown instead of going unnoticed
until a later test. Always attempt context cleanup via finally.

Related to #3426.

* fix(cuda.core): synchronize the bound context during test teardown

Use device.sync() because the test wrapper may pop the current context
before init_cuda teardown. This avoids spurious INVALID_CONTEXT errors
while preserving synchronization error reporting and context cleanup.

Add regression coverage for a missing current context and cleanup after
synchronization failures.

Related to #3426.

* fix(cuda.core): serialize memory-pool watermark reset tests

Mark test_mempool_watermark_reset thread_unsafe, matching test_mempool_trim.
Both exercise the pool teardown and synchronization sequence associated
with the suspected driver race in #3350.

* fix(cuda.core): bypass access revocation when destroying host pools

Skip peer-access revocation for locally created HOST and HOST_NUMA pools
to investigate pinned-pool teardown OOMs and crashes. Continue destroying
the pools and preserve cleanup for device, managed, and imported pools.

This is a diagnostic workaround pending Windows CI validation.

Related to #3426.

* fix(cuda.core): preserve the first exception during fixture cleanup

Keep setup or synchronization failures primary and attach subsequent
cleanup failures as exception notes. Always attempt to pop the context.

Add regression coverage for combined failures, including interrupts.

* fix(cuda.core): limit host-pool cleanup workaround to pinned allocations

HOST and HOST_NUMA also describe managed-pool preferences. Check the allocation type so the pinned-memory workaround preserves managed-pool peer-access revocation.

Signed-off-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>

* test(cuda.core): expose pinned allocation OOM and retain its traceback

Signed-off-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>

* ci: diagnose PR3436 pinned pool teardown on Windows

Signed-off-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>

* ci: move watermark investigation out of PR3436

Signed-off-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>

* test(cuda.core): restore expected pinned allocation OOM handling

Signed-off-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>

* revert(cuda.core): defer pinned host pool cleanup workaround

Signed-off-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>

---------

Signed-off-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
Co-authored-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
(cherry picked from commit b715078)

* [no-ci] ci: exclude unavailable historical GitHub references (#3471)

* ci: exclude unavailable historical GitHub references

Signed-off-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>

* ci: require manual copy-pr-bot approval

Signed-off-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>

---------

Signed-off-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
(cherry picked from commit 3991a69)

* cuda.core: map ml_dtypes bfloat16 to kDLBfloat in DLPack export (#3359)

_smv_dtype_numpy_to_dlpack dispatched on numpy dtype kind, but
ml_dtypes.bfloat16 has kind "V", so bf16 views fell through to the
unsupported-dtype BufferError and could not be re-exported, even though
the DLPack import path already accepts kDLBfloat and _tensor_map.pyx
maps bfloat16. Detect bfloat16 up front and export it as kDLBfloat
(16 bits).

The new round-trip test goes through StridedMemoryView.from_dlpack so
it cannot skip on stacks where NumPy's own DLPack support for extended
dtypes is unavailable, unlike the existing parametrized round-trip
test's NumPy-consumer probe.

Signed-off-by: Sainath-Reddy7 <180950590+Sainath-Reddy7@users.noreply.github.com>
Co-authored-by: Sainath-Reddy7 <180950590+Sainath-Reddy7@users.noreply.github.com>
Co-authored-by: Sebastian Berg <sebastianb@nvidia.com>
(cherry picked from commit d1d1457)

* fix(cuda.core): make IPC imports wait for a pending release of the same descriptor (#3450)

* fix(cuda.core): make IPC imports wait for a pending release of the same descriptor

deviceptr_import_ipc() shares one handle per descriptor because the
driver's first free of an imported pointer unmaps every import of it
(nvbug 5570902). The handle's deleter took ipc_import_mutex only after
the last reference was gone. In that gap an import found the cache entry
expired, re-imported the still-mapped pointer and registered a new entry.
The deleter then erased that new entry and freed the pointer, which
unmapped it under the new Buffer.

The cache entry now outlives the last reference and only its own deleter
erases it, under the mutex, together with the free. An import that finds
an expired entry waits on a condition variable until that release is
done, then imports fresh.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Use CTAD for the ipc_import_mutex locks

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 2f1185e)

* test(cuda.core): skip HOST graph allocation before CUDA 13 (#3446)

(cherry picked from commit 11cc666)

* test(cuda.core): query green stream contexts with v2 (#3448)

Signed-off-by: Rui Luo <ruluo@nvidia.com>
(cherry picked from commit 7076a67)

* test(cuda.core): skip the VMM RDMA policy test on WSL (#3478)

GPUDirect RDMA is not supported on WSL, but some drivers report the
capability there, and the RDMA-enabled VMM allocation in
test_vmm_allocator_policy_configuration then fails. Skip the test on
WSL; the capability checks stay as they are.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 36bc530)

* docs(cuda.core): add 1.3.0 release note for the IPC import/release race fix (#3479)

* docs(cuda.core): add 1.3.0 release note for the IPC import/release race fix

Follow-up to #3450, which merged without a release note.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(cuda.core): scope the IPC race release note to memory pool buffers

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit de4c60d)

* Fix conditional graph lifetime tests to use supported nodes (#3469)

* Fix conditional graph lifetime tests to use supported nodes

Signed-off-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>

* ci: require manual copy-pr-bot approval

Signed-off-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>

---------

Signed-off-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
Co-authored-by: Andy Jost <ajost@nvidia.com>
(cherry picked from commit b80866b)

* fix(cuda.core): run cuMemPoolSetAccess in relaxed mode in PinnedMemoryResource (#3475)

Since #3405, PinnedMemoryResource.__init__ without options calls
cuMemPoolSetAccess to grant the current device read/write access to
the driver's default pool.  The driver refuses that call, and
invalidates the capture, while a global-mode capture is active on any
thread.  Users of PyTorch's torch.cuda.graph (which defaults to global
mode) therefore see CUDA_ERROR_STREAM_CAPTURE_UNSUPPORTED on
construction and CUDA_ERROR_STREAM_CAPTURE_INVALIDATED on end_building.

Fix: introduce MP_set_access in _memory_pool.pyx, wrapping
cuMemPoolSetAccess in the existing _enter_relaxed_capture_mode /
_exit_relaxed_capture_mode helpers, following the identical pattern
used by pool attribute reads and writes and trim() since #2834.
Call it from _PMR_init instead of the bare with-nogil block.

The with-options path (cuMemPoolCreate) is unaffected; that call is
still refused under global capture as documented in the 1.3.0 notes.

Add tests for the three capture combinations (global+same_thread,
global+other_thread, thread_local+same_thread) and a capture-mode
restore check modelled on test_dmr_init_restores_capture_mode.

Closes #3451

Co-authored-by: Andy Jost <ajost@nvidia.com>
(cherry picked from commit d94f19c)

* cuda.core: carry the cuda-bindings floors through minimal-sysctk12/13 extras (#3460)

* cuda.core: carry the cuda-bindings floors through minimal-sysctk extras

The floors lived only in the cu12/cu13 extras, so a package that depends
on plain cuda-core never passed them to pip or uv, and an older
cuda-bindings failed only at import (#3459).

Declare the floors in new minimal-sysctk12/13 extras, include them from
cu12/cu13, and depend on cuda-bindings without a version. The floor
parser, the CI wheel-floor reader, the pre-commit hook and the docs read
the new extras.

* cuda.core: read cu<N> floors through the minimal-sysctk self-reference

Drop the unversioned cuda-bindings from the base dependencies, so a
plain cuda-core install keeps asking for an extra.

Keep the floor readers on the cu<N> extras, as before, and teach them
to follow a cuda-core[...] self-reference and to skip an unversioned
cuda-bindings[all]. The cu<N> layout then yields the floor written in
minimal-sysctk<N>, and the existing tests of the cu<N> form stay as
they are. New tests cover the self-reference.

---------

Co-authored-by: Andy Jost <ajost@nvidia.com>
(cherry picked from commit ad38141)

---------

Signed-off-by: Ralf Juengling <rjuengling@nvidia.com>
Signed-off-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
Signed-off-by: Sainath-Reddy7 <180950590+Sainath-Reddy7@users.noreply.github.com>
Signed-off-by: Rui Luo <ruluo@nvidia.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Ralf Juengling <rjuengling@nvidia.com>
Co-authored-by: jpascucci-nv <jpascucci@nvidia.com>
Co-authored-by: Ralf W. Grosse-Kunstleve <rgrossekunst@nvidia.com>
Co-authored-by: Ralf W. Grosse-Kunstleve <rwgkio@gmail.com>
Co-authored-by: B Sainath Reddy <bsainathreddy4@gmail.com>
Co-authored-by: Sainath-Reddy7 <180950590+Sainath-Reddy7@users.noreply.github.com>
Co-authored-by: Sebastian Berg <sebastianb@nvidia.com>
Co-authored-by: Rob Parolin <rparolin@users.noreply.github.com>
Co-authored-by: Rui Luo <ruluo@nvidia.com>
Co-authored-by: Leo Fang <leof@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working cuda.core Everything related to the cuda.core module

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants