Repository navigation
perf(storage): parallelize sharded history last-shard reads (v2.8.0 backport) - #29
Closed
BrianBland wants to merge 2 commits into
Closed
BrianBland wants to merge 2 commits into
BrianBland wants to merge 2 commits into
Conversation
…ry last-shard reads Squashed paradigmxyz#26766 at f47fd63, rebased onto v2.8.0 from base-v2.7.0.2 (de52d82). v2.8.0 includes paradigmxyz#27602, which rebuilds pruned history from empty. The backport already carried equivalent rebuild-from-empty handling for its chunked execution, so the index history stages keep the backport's logic. Both the upstream rollback test and the backport's chunked prune test are kept. Co-authored-by: Toshi <toshi-noreply@coinbase.com>
Co-authored-by: Toshi <toshi-noreply@coinbase.com>
Author
|
Not carrying this for v2.8.0: base/base now pins upstream v2.8.0 with no patches. Upstream closed paradigmxyz#26766 as stale, and paradigmxyz#27602 fixes the empty-rebuild restore case that motivated it. If benchmarks later show incremental or live persistence needs parallel tail reads, re-carry from the upstream head dc38531, which also has the 16-job cap and synced clear. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Rebases the Base carry of paradigmxyz#26766 (snapshot f47fd63) from base-v2.7.0.2 (de52d82) onto v2.8.0.
Every other patch in base-v2.7.0.2 (paradigmxyz#27498, paradigmxyz#27669, paradigmxyz#27665, paradigmxyz#27939, paradigmxyz#27940, paradigmxyz#27677, #28) is already in v2.8.0, so this is the only carried patch.
Conflict: v2.8.0 adds paradigmxyz#27602 (rebuild pruned history from empty). The backport already carried equivalent rebuild-from-empty handling for its chunked execution, so
IndexAccountHistoryStage/IndexStorageHistoryStage::executekeep the backport's logic (identical to base-v2.7.0.2). Both upstream'sprune_rebuild_retries_after_checkpoint_rollbacktest and the backport'sprune_bump_rebuilds_empty_table_then_merges_later_chunkstest are kept and pass.Consumed by
just reth-prepare-releasein base/base to cut base-v2.8.0.1.Tests:
cargo test -p reth-stages --lib -- index_(46 passed),cargo test -p reth-provider --lib -- history(61 passed).