Skip to content

Add CSM for batch write flow control #2685

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
mutianf merged 1 commit into from
Dec 15, 2025
Merged

Add CSM for batch write flow control #2685

mutianf merged 1 commit into from
Dec 15, 2025

Conversation

@kongweihan
Copy link
Contributor

@kongweihan kongweihan commented Oct 6, 2025

Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly:

  • Make sure to open an issue as a bug/issue before writing your code! That way we can discuss the change, evaluate designs, and agree on the general idea
  • Ensure the tests and linter pass
  • Code coverage does not decrease (if any source code was changed)
  • Appropriate docs were updated (if necessary)
  • Rollback plan is reviewed and LGTMed
  • All new data plane features have a completed end to end testing plan

Fixes #<issue_number_goes_here> ☕️

If you write sample code, please follow the samples format.

@kongweihan kongweihan requested a review from a team as a code owner October 6, 2025 16:18
@kongweihan kongweihan requested a review from a team October 6, 2025 16:18
@product-auto-label product-auto-label bot added size: m Pull request size is medium. api: bigtable Issues related to the googleapis/java-bigtable API. labels Oct 6, 2025
@kongweihan kongweihan force-pushed the csm-batch-write-flow-control branch 2 times, most recently from 2ca884e to b4006a2 Compare October 6, 2025 16:54
mutianf
mutianf reviewed Oct 6, 2025
View reviewed changes
@kongweihan kongweihan force-pushed the csm-batch-write-flow-control branch from b4006a2 to e8a246f Compare October 7, 2025 03:03
mutianf
mutianf reviewed Oct 8, 2025
View reviewed changes
@kongweihan kongweihan force-pushed the csm-batch-write-flow-control branch from e8a246f to ae0ed5a Compare October 8, 2025 14:27
@kongweihan kongweihan force-pushed the csm-batch-write-flow-control branch 7 times, most recently from 7d3fdd2 to 5f07efc Compare December 10, 2025 00:37
@product-auto-label product-auto-label bot added size: l Pull request size is large. and removed size: m Pull request size is medium. labels Dec 10, 2025
mutianf
mutianf reviewed Dec 10, 2025
View reviewed changes
...e/src/test/java/com/google/cloud/bigtable/data/v2/stub/metrics/BuiltinMetricsTracerTest.java Outdated
}
builder.addEntriesBuilder().setIndex(i);

com.google.protobuf.Duration duration = builder.getRateLimitInfoBuilder().getPeriodBuilder().setSeconds(10).build();
Copy link
Contributor

@mutianf mutianf Dec 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: can you import com.google.bigtable.v2.RateLimitInfo so we don't need to write the full path here?

Copy link
Contributor Author

@kongweihan kongweihan Dec 15, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You meant Duration? I didn't see a full path here for RateLimitInfo?

@kongweihan kongweihan force-pushed the csm-batch-write-flow-control branch 3 times, most recently from 8e84e9f to 7981240 Compare December 11, 2025 22:36
mutianf
mutianf reviewed Dec 12, 2025
View reviewed changes
Copy link
Contributor

@mutianf mutianf left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for adding the tests, left some more nits but otherwise looks good!

...e/src/test/java/com/google/cloud/bigtable/data/v2/stub/metrics/BuiltinMetricsTracerTest.java Outdated
batcher.add(
RowMutationEntry.create("batch-write-flow-control-success-08").setCell("f", "q", "v"));

// closing the batcher to trigger the third flush
Copy link
Contributor

@mutianf mutianf Dec 12, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit:

Suggested change
// closing the batcher to trigger the third flush
// closing the batcher to trigger the flush

...e/src/test/java/com/google/cloud/bigtable/data/v2/stub/metrics/BuiltinMetricsTracerTest.java Outdated
batcher.add(
RowMutationEntry.create("batch-write-flow-control-success-05").setCell("f", "q", "v"));

// closing the batcher to trigger the third flush
Copy link
Contributor

@mutianf mutianf Dec 12, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// closing the batcher to trigger the third flush
// closing the batcher to trigger the flush

@kongweihan kongweihan force-pushed the csm-batch-write-flow-control branch from 7981240 to caf5a50 Compare December 15, 2025 17:36
@mutianf mutianf merged commit 62ffd1b into googleapis:main Dec 15, 2025
32 of 34 checks passed
@release-please release-please bot mentioned this pull request Dec 15, 2025
Merged
blakeli0 added a commit that referenced this pull request Feb 3, 2026
@blakeli0 @diegomarquezp
@cloud-java-bot @renovate-bot @release-please @sushanb @kongweihan @trollyxia @mutianf
chore: Merge release v2.71.0 into protobuf-3.x (#2771)
2e10f0b 
* tests: Remove unnecessary null check for protobuf lists (#2723)

ErrorProne will complain ImpossibleNullComparison once protobuf runtime is upgraded to 4.x. See the downstream tests in sdk-platform-java.

* chore: add release-please config for protobuf-4.x (#2722)

* chore: add release-please config for protobuf-4.x

* chore: generate libraries at Wed Dec  3 18:40:37 UTC 2025

* chore: enable manifest option in release config

---------

Co-authored-by: cloud-java-bot <[email protected]>

* deps: update shared dependencies (#2734)

* chore(main): release 2.70.1-SNAPSHOT (#2715)

* chore(main): release 2.70.1-SNAPSHOT

* chore: generate libraries at Fri Dec 12 14:04:07 UTC 2025

---------

Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>
Co-authored-by: cloud-java-bot <[email protected]>

* chore(main): release 2.70.1 (#2735)

* chore(main): release 2.70.1

* chore: generate libraries at Fri Dec 12 15:48:36 UTC 2025

---------

Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>
Co-authored-by: cloud-java-bot <[email protected]>

* fix(bigtable): add handling for gauge metrics (#2719)

* fix(bigtable): add handling for gauge metrics

* add it

* chore: generate libraries at Tue Nov 25 20:15:38 UTC 2025

---------

Co-authored-by: cloud-java-bot <[email protected]>

* chore(main): release 2.70.2-SNAPSHOT (#2736)

Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>

* feat: Add CSM for batch write flow control (#2685)

* test: Add missing test table cleanup for MaterializedViewIT (#2721)

Change-Id: I1281568894ac2d04f0aafe0aa7f4efa7570ae58a

Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly:
- [ ] Make sure to open an issue as a [bug/issue](https://togithub.com/googleapis/java-bigtable/issues/new/choose) before writing your code!  That way we can discuss the change, evaluate designs, and agree on the general idea
- [ ] Ensure the tests and linter pass
- [ ] Code coverage does not decrease (if any source code was changed)
- [ ] Appropriate docs were updated (if necessary)
- [ ] Rollback plan is reviewed and LGTMed
- [ ] All new data plane features have a completed end to end testing plan

Fixes #<issue_number_goes_here> ☕️

If you write sample code, please follow the [samples format](
https://togithub.com/GoogleCloudPlatform/java-docs-samples/blob/main/SAMPLE_FORMAT.md).

* tests: Migrate usage of GeneratedMessageV3 to Message (#2745)

* tests: Migrate usage of GeneratedMessageV3 to Message

* chore: generate libraries at Mon Dec 22 22:29:53 UTC 2025

---------

Co-authored-by: cloud-java-bot <[email protected]>

* fix: create stub with BigtableClientContext so otels are closed (#2747)

* fix: create stub with BigtableClientContext so otels are closed

* address comment

* test: Reduce ExecuteQueryIT flakiness with a retry loop (#2748)

Change-Id: I4171e5b11d2278a8b3dc19d2dec3f03d784759ce

* fix: use the same background executor in otel reader and monitoring c… (#2746)

* fix: use the same background executor in otel reader and monitoring client

* add public api doc

* fix tests

* close background executor

* fix test

* address comments

* fix

* deps: update shared dependencies (#2752)

* fix: update BigtableChannelPool to use the background executor (#2753)

* deps: update dependency com.google.cloud:gapic-libraries-bom to v1.76.0 (#2754)

* chore(main): release 2.71.0 (#2737)

* chore(main): release 2.71.0

* chore: generate libraries at Thu Jan 15 16:06:19 UTC 2026

---------

Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>
Co-authored-by: cloud-java-bot <[email protected]>

---------

Co-authored-by: Diego Marquez <[email protected]>
Co-authored-by: cloud-java-bot <[email protected]>
Co-authored-by: Mend Renovate <[email protected]>
Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>
Co-authored-by: Sushan Bhattarai <[email protected]>
Co-authored-by: Weihan Kong <[email protected]>
Co-authored-by: Lixia Chen <[email protected]>
Co-authored-by: Mattie Fu <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@mutianf mutianf mutianf approved these changes

Assignees

No one assigned

Labels

api: bigtable Issues related to the googleapis/java-bigtable API. size: l Pull request size is large.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@kongweihan @mutianf