Repository navigation
Read all log files overlapping a partitioned restore window - #14382
Open
mpatou-openai wants to merge 2 commits into
Open
mpatou-openai wants to merge 2 commits into
mpatou-openai wants to merge 2 commits into
Conversation
mpatou-openai
force-pushed
the
dev/mpatou/partitioned-restore-log-pagination
branch
from
October 31, 2026 02:00
ea10d2d to
7de529d
Compare
mpatou-openai
marked this pull request as ready for review
November 2, 2026 17:07
mpatou-openai
force-pushed
the
dev/mpatou/partitioned-restore-log-pagination
branch
from
November 3, 2026 17:56
7de529d to
77611d6
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A partitioned restore can finish successfully while omitting mutations. The dispatcher scanned at most 1,000 entries from the beginning of the log manifest for each version window, then filtered that page for overlap. If the first 1,000 files had expired, it could dispatch an empty log task even when later files covered the window.
The log task now reads all pages covering its half-open version window. It advances past the full metadata key so files with the same start version are preserved. Newly initialized restores record the longest log-file span to bound the lookup. Existing restores can scan from the start. Each page uses its own read transaction, and the matching file list no longer has to fit in one task parameter. New log tasks use version 2. Upgraded agents also re-read the manifest for queued version 1 tasks.
Added a regression with a matching file on each side of the 1,000-row boundary, multiple tags at one start version, and files touching both ends of the window. It exercises manifests with and without the span metadata. Built fdbclient_test and ran the focused test on this revision. One test passed, zero failed.