Repository navigation
DDIR explain: demand is a set, a min demands what may decide it, and the restricted run reads demanded rows whole - #986
Merged
frankmcsherry merged 3 commits intoOct 21, 2026
Conversation
A collection maps data to diffs, and an explanation names the data whose diffs must be accurate, so demand's counts mean nothing. Explain still combined demand by its counts. A lookup multiplied demand by the count of the host-form row it found, and the host forms of a collection below a negation, or inside a loop (whose records hold each round's changes), have negative counts. So demand reaching one collection by two paths could come back as +1 and -1, cancel in the site's concat, and leave a row's explanation empty; a lone negative contribution was dropped as well. Demand and every host form, witness and forward, sites and sources, are now read as sets (`Sb::presence`, which was `distinct_full`): each row whose count is not zero, once. Pair tables then carry no negative counts, and demand never cancels. tests/explain.rs `negated_fanout_explanations_sufficient` reads a negated collection directly and through a map; it fails without this. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019XsakwFGwtESwZ1gSYwG4o
A min's output at a key is decided by the inputs at or below it. The keyed lookup narrowed a demanded min row to the inputs whose value equals it, which is right for a row the reference produced and the restricted run agrees with. It misses both ways the restricted run can disagree: - the restricted run raised the value, its lower inputs left out: the narrowing demanded the restricted run's achieving inputs and never the reference's, so the key's output was never made as in the reference. A label that rose to the queried label then joined a later count's group, and that count's explanation was insufficient; - the restricted run lowered the value: the inputs that lowered it were never demanded, so they were never corrected. The narrowing now keeps the inputs at or below the demanded value (`ky_data_le`), from both runs as before. The value contract test covers the least value and a higher one. tests/explain.rs `min_then_count_explanations_sufficient`, label propagation by `min` and then a count per label, fails without this. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019XsakwFGwtESwZ1gSYwG4o
The restricted run's inputs are the sources restricted to the demand sets. They were restricted by a semijoin on the key alone, so demanding one row fed the restricted run every source row at that key, each once per demanded row at it. For an input with several values at a key, the restricted run then computed on rows and counts the demand did not name: a count over the demanded rows came out as a count over the whole key, and the restricted run could agree with the reference by accident, so the inputs that make it agree were never demanded. Each demand set's closure kept demanded rows by key alone the same way. Both now match whole rows (`Sb::restrict`), and keep the source's counts. tests/explain.rs `restricted_rows_keep_their_counts`, a count over an input with several values at a key, its difference with the input, and a min, fails without this. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019XsakwFGwtESwZ1gSYwG4o
This was referenced Oct 21, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Three explain soundness fixes, found by the differential testing (findings 5 and 6, plus one found while sweeping for them). With them, the explain oracle passes with the negation and non-monotone skips removed. Each commit has a regression test in
tests/explain.rsthat fails without it. +93/−25, all inexplain/and its tests.1. Demand is a set (e6b9824).
Sb::presence, renamed fromdistinct_full.negated_fanout_explanations_sufficient.2. A
mindemands the inputs at or below its demanded value (50fafd8).min's keyed lookup kept only the inputs whose value equals the demanded value. When the restricted run raised a key's minimum (the inputs achieving the lower value left out), demand asked for the inputs achieving the wrong, raised value, and never the reference's lower ones. The minimal case is label propagation, then a count per label.min_then_count_explanations_sufficient, plus the updated value-contract unit test.3. The restricted run reads demanded rows whole (5179498).
Sb::restrict).restricted_rows_keep_their_counts.Validation
Cost: release explain suite, instructions retired: 423.1G on master-next, 567.4G here (+34%). This was accepted for correctness.
Follow-up once #980 lands: remove the harness's shape skip for these findings, so
case.explainable = !g.timed && !exploded;, and update theCase::explainabledoc.🤖 Generated with Claude Code
https://claude.ai/code/session_019XsakwFGwtESwZ1gSYwG4o