fix(cold): track stream producer tasks for graceful shutdown#40
fix(cold): track stream producer tasks for graceful shutdown#40
Conversation
Stream producers were spawned via bare `tokio::spawn`, escaping the task runner's `TaskTracker`. This meant they were not awaited during graceful shutdown and ran outside the read/write serialization barrier. Add a separate `stream_tracker` on `ColdStorageTaskInner` for long-lived stream producers. Streams are tracked for shutdown but not drained before writes — backends provide their own read isolation (MDBX snapshots, PostgreSQL REPEATABLE READ, anchor-hash detection). Strengthen `ColdStorage::produce_log_stream` docs to make the concurrency contract explicit. Closes ENG-1988 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
| // Graceful shutdown: drain reads first (short-lived), then streams | ||
| // (bounded by deadline). Reads must drain first because StreamLogs |
There was a problem hiding this comment.
Should we apply a tokio::time::timeout to the stream tracker shutdown here rather than relying on the backend impls doing it?
There was a problem hiding this comment.
we already rely on backends to provide read isolation. we could add some safety here, but i'm not sure it's necessary? do you have a strong opinion we should?
There was a problem hiding this comment.
No, not a strong opinion. It just seems like a lightweight failsafe we could add, but ultimately this will more than likely be run in an environment where it will be forcefully killed if it doesn't stop within a reasonable time, so not a big deal.
There was a problem hiding this comment.
let's keep as is for now. can always revisit. any other concerns?
There was a problem hiding this comment.
Nope. Sorry - I thought I already approved!
Summary
tokio::spawn, escaping the task runner'sTaskTracker— not awaited during shutdown, not visible to the read/write barrierstream_tracker: TaskTrackertoColdStorageTaskInnerfor long-lived stream producersColdStorage::produce_log_streamdocs to make the concurrency contract explicitCloses ENG-1988
Test plan
cargo clippy -p signet-cold --all-features --all-targetscargo clippy -p signet-cold --no-default-features --all-targetscargo +nightly fmtcargo t -p signet-cold— conformance suite passes🤖 Generated with Claude Code