[Bug]: Spanner Change Streams reading have potential data loss when the dataflow worker has high CPU utilization

[jiangzzhu]

What happened?

Spanner Change Streams reading query may early terminate and introduce data loss when dataflow worker has CPU utilization.

This was caused by how timestamps were being tracked when queries completed. Specifically, an unbounded end timestamp is no longer permitted for the change stream query RPC. However, if a query reached its calculated artificial end timestamp, the code wasn't careful enough and could accidentally advance the restriction tracker to the full, potentially unbounded end timestamp of the original range.

The issue was fixed by #37326

Issue Priority

Priority: 2 (default / most bugs should be filed as P2)

Issue Components

• Component: Python SDK
• Component: Java SDK
• Component: Go SDK
• Component: Typescript SDK
• Component: IO connector
• Component: Beam YAML
• Component: Beam examples
• Component: Beam playground
• Component: Beam katas
• Component: Website
• Component: Infrastructure
• Component: Spark Runner
• Component: Flink Runner
• Component: Prism Runner
• Component: Twister2 Runner
• Component: Hazelcast Jet Runner
• Component: Google Cloud Dataflow Runner

[jiangzzhu]

The issue was fixed by #37326

[jiangzzhu]

cc: @tianz101 , @chenxuesdu , @scwhittle