-
Notifications
You must be signed in to change notification settings - Fork 84
test: introduce new StorageITRunner #1785
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Split enclosed classes into their own respective classes ITBlobReadMaskTest and ITBucketReadMaskTest
Refactor RetryTestFixture to extend TestWatcher to allow for manual registration
frankyn
approved these changes
Nov 30, 2022
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the discussion, PR LGTM, I added an item to add documentation to the contributing guide for using this infrastructure / linking to javadocs.
This was referenced Dec 9, 2022
This was referenced Dec 20, 2022
gcf-owl-bot bot
added a commit
that referenced
this pull request
Mar 31, 2023
Source-Link: googleapis/synthtool@43c709a Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-java:latest@sha256:3387f93d4577788512112ff69ddab746ae9192ddd9f13cfd175ef310d62d7d30
BenWhitehead
pushed a commit
that referenced
this pull request
Mar 31, 2023
Source-Link: googleapis/synthtool@43c709a Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-java:latest@sha256:3387f93d4577788512112ff69ddab746ae9192ddd9f13cfd175ef310d62d7d30 Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Labels
api: storage
Issues related to the googleapis/java-storage API.
size: xl
Pull request size is extra large.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
A note to the reviewer:
This PR refactors the hole integration test suite and touches many files. However, the
majority of the files touched are not logically changing, simply changing how they declare
their test environment dependencies.
This PR is intentionally several commits, each commit is a small logical grouping of a
specific change. Take each commit in turn rather than the hole PR as a single change.
First read the rest of this description, then take a look at each individual commit.
The later majority of commits are simply migrating an individual test class to using
the new runners.
New StorageITRunner
Add two new JUnit runner to provide support for Cross Running test in a class across multiple
backends and transports.
When a class is annotated
@CrossRun(transports = {HTTP, GRPC}, backends = {PROD, TEST_BENCH})
each test defined in the class will be ran four times, once for each pair from the permutation.
Each permutation will add a new entry to the test suite of the form
[Transport][Backend]
.This name form can be thought of as keys to access a location in an adjacency matrix.
If only a single backed is needed
@SingleBackend()
can be used to denote the backend.If a class is annotated
@ParallelFriendly
the test suite may attempt to run the tests inparallel. Currently parallelism factor is set to 2 * core count.
Field
@Inject
ionWhen defining the test several "globally scoped" items are available for injection to the test
class by annotating a public non-final field with
@Inject
.Instances of Storage, along with BucketInfo, TestBench, etc. are registered as injectable values.
If a field isn't able to be injected an InitializationError will be raised with details as to why.
CrossRun.Ignore
&CrossRun.Exclude
Not all tests in a class will apply to all intersections, to allow filtering out tests for
these cases there are two new annotations which can be used. If a test is both Ignored and
Excluded, the Ignore will take precedence.
@Ignor
ed functions similar to the standard@org.junit.Ignore
where the test will be loadedbut not ran.
@Exclud
ed tests will not be loaded into the suite at all, thereby not being reported at all.@Parameterized
runningWhen a class is annotated
@Parameterized(ParameterProvider)
in addition to and Cross Runpreviously resolved, each parameter will result in another dimension added to the suite.
The name of the suite will be extended with another dimension below any cross name,
for example
[HTTP][PROD][myParam]
.Runner Registry
A new class
Registry
has been added which is responsible for managing "globally scoped"instances of resources associated with running integration tests. All injectable instances
are registered with the Registry and will be lazily initialized, reused and cleaned up.
Instances like Storage clients, Backend Buckets, standard objects etc are all managed per
backend.
When a BucketInfo is loaded into the registry it's name is also made available to the
provider for Storage instances. If a test attempts to mutate the state of a bucket the
operation will be interrupted with a vetoing exception causing the test to fail.
Refactoring Done
testbench binds ports it is not a multi instance thing anyway.
@ParallelFriendly
TODO
Items to still perform in later PRs
an InitializationError is raised stating the fact they are unsafe and will be ran
multiple times (once for each sub-suite generated when cross running).
annotations at their own discretion