Repository navigation
test: cover the memory accounting paths of the grouped aggregate streams that no test reaches - #26562
Open
jayzhan211 wants to merge 1 commit into
Open
jayzhan211 wants to merge 1 commit into
jayzhan211 wants to merge 1 commit into
Conversation
…ams that no test reaches
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
Rationale for this change
The grouped aggregate streams decide how much memory to reserve in several places: when they spill, when they replay their spill files, when they hand out a materialized batch in slices, and when the pool refuses a request. To find out which of these decisions the existing tests protect, I broke them one at a time on main and ran the physical-plan tests, the
memory_limitand aggregate fuzz integration tests, and the sqllogictests. Of 26 such changes, 14 broke no test:A pool refusal that is wrongly ignored or wrongly returned is a query that runs past its memory limit or fails when it should succeed, so these paths deserve tests before anyone changes them.
What changes are included in this PR?
One test file,
aggregates/memory_call_sequence_tests.rs, with 11 tests. Each runs oneAggregateExecunder a pool that records every call made on behalf of the aggregate's consumers (register with itscan_spill, grow, try_grow and whether it was refused, shrink, unregister), and snapshots that sequence together with the query result. Each test also asserts through metrics or output batch sizes that it really took its path (it spilled, or emitted whole batches, and so on).Refused output materialization needs an aggregate whose output is larger than the memory it accounts for, which no built-in aggregate has, so the file defines a small test UDAF for it.
The early emit of the unclustered Partial and PartialReduce streams is left out: their own unit tests already catch every change I made there.
What is the testing strategy for this PR?
The PR is the tests. Each of the 14 changes above makes at least one of them fail, and all of them pass on main.
Are there any user-facing changes?
No.