Skip to content

Crashes under runnerv2 due to project tracking semantics #676

Description

@dpcollins-google

Under dataflow RunnerV2, after trySplit returns successfully, the runner does not immediately check the progress accounted for by testing the RestrictionTracker. Instead, it destroys the RestrictionTracker and tries to construct a new one using the already claimed restriction instead. The runner will not do this if the SDF implements the @Getsize annotated method, in which case it will be used instead. As far as I can tell, this particular interplay is undocumented, but it should remain true for this public API.

Our I/O will crash when constructed with an already completed restriction to avoid constructing a restriction tracker in an unusable state. We should implement GetSize so that we don't need to allow for partially unusable RestrictionTrackers.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    🚨This issue needs some love.api: pubsubliteIssues related to the googleapis/java-pubsublite API.triage meI really want to be triaged.

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions