Skip to content

Conversation

@ayousufi
Copy link
Collaborator

@ayousufi ayousufi commented Mar 26, 2025

This is a follow up to the schema change in #355. While this PR mostly updates the unit, integration, and acceptance tests, there is a key implementation change in the filter interpreter such that all_of filters will now build a new bool sub-filter for each subexpression and push it onto the parent’s :filter array. This ensures an ANDing across the subexpressions.

Performance benchmarking
Given the above change will result in an extra bool filter term, I performed benchmarking using the test_es_query.rb script adapted to an all_of query with nested any_satisfy blocks.

Results from current schema:
Average took value: 504ms
Median took value: 454ms
P99 took value: 1190ms

vs PR schema:
Average took value: 505ms
Median took value: 448ms
P99 took value: 1054ms

We can see there is no significant impact to performance with the additional bool filter terms.

@ayousufi ayousufi force-pushed the abdullah/all-of-schema branch from f56c519 to 7bb7667 Compare March 28, 2025 17:56
@ayousufi ayousufi force-pushed the abdullah/all-of-schema-2 branch from 863bc53 to 2ba0a6f Compare March 28, 2025 17:57
@ayousufi ayousufi marked this pull request as ready for review March 28, 2025 18:41
@ayousufi ayousufi force-pushed the abdullah/all-of-schema-2 branch from 2ba0a6f to 5e096fe Compare March 31, 2025 22:55
@CLAassistant
Copy link

CLAassistant commented Mar 31, 2025

CLA assistant check
All committers have signed the CLA.

@ayousufi ayousufi force-pushed the abdullah/all-of-schema branch 2 times, most recently from b5ba782 to b39b987 Compare April 1, 2025 18:53
@ayousufi ayousufi force-pushed the abdullah/all-of-schema-2 branch from a0cccb4 to fa00135 Compare April 1, 2025 18:54
@ayousufi ayousufi force-pushed the abdullah/all-of-schema branch from b39b987 to ce08cb9 Compare April 1, 2025 20:36
@ayousufi ayousufi force-pushed the abdullah/all-of-schema-2 branch from fa00135 to 583817a Compare April 1, 2025 20:38
@ayousufi ayousufi force-pushed the abdullah/all-of-schema branch from ce08cb9 to d5adb32 Compare April 1, 2025 22:17
@ayousufi ayousufi force-pushed the abdullah/all-of-schema-2 branch from 583817a to 3477d47 Compare April 1, 2025 22:19
Copy link
Collaborator

@myronmarston myronmarston left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good! Left some feedback.

Copy link
Collaborator

@myronmarston myronmarston left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, can we add some coverage for how all_of interacts with not and empty filters?

@ayousufi ayousufi force-pushed the abdullah/all-of-schema branch from d5adb32 to d2e7b15 Compare April 4, 2025 13:25
@ayousufi ayousufi force-pushed the abdullah/all-of-schema-2 branch 2 times, most recently from 1f27653 to a5d6764 Compare April 4, 2025 13:30
@ayousufi ayousufi requested a review from myronmarston April 4, 2025 13:38
Copy link
Collaborator

@myronmarston myronmarston left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is coming along nicely! Left some more feedback. Almost ready to merge...

@ayousufi ayousufi force-pushed the abdullah/all-of-schema-2 branch 2 times, most recently from 38f96fe to ce975cc Compare April 4, 2025 21:13
@ayousufi ayousufi requested a review from myronmarston April 4, 2025 21:16
@ayousufi ayousufi force-pushed the abdullah/all-of-schema branch from d2e7b15 to 0f76b93 Compare April 4, 2025 21:16
@ayousufi ayousufi force-pushed the abdullah/all-of-schema-2 branch from ce975cc to 9b222f1 Compare April 4, 2025 21:17
Copy link
Collaborator

@myronmarston myronmarston left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@ayousufi ayousufi force-pushed the abdullah/all-of-schema branch from 0f76b93 to f651f46 Compare April 4, 2025 21:57
@ayousufi ayousufi force-pushed the abdullah/all-of-schema-2 branch from 9b222f1 to 6411b5e Compare April 4, 2025 22:22
Base automatically changed from abdullah/all-of-schema to main April 4, 2025 22:23
@ayousufi ayousufi force-pushed the abdullah/all-of-schema-2 branch from 6411b5e to 9d53588 Compare April 4, 2025 22:24
@ayousufi ayousufi merged commit 56d1620 into main Apr 4, 2025
21 checks passed
@ayousufi ayousufi deleted the abdullah/all-of-schema-2 branch April 4, 2025 22:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants