Skip to content

Go: stabilize build constraint evaluation - #22839

Open
owen-mc wants to merge 1 commit into
github:mainfrom
owen-mc:go/fix-dca-tuple-sum-noise
Open

owen-mc wants to merge 1 commit into
github:mainfrom
owen-mc:go/fix-dca-tuple-sum-noise

Conversation

@owen-mc

@owen-mc owen-mc commented Oct 16, 2026

Copy link
Copy Markdown
Contributor

Evaluate the small set of files containing Go build constraints bottom-up, while keeping ExtractedOrExternalFile.hasBuildConstraints() demand-specialized.

This avoids propagating bound-file demands through the location predicates used to find build-constraint comments.

Motivation

On its own, this change avoids an unnecessarily expensive evaluation plan.

There are only 24 build-constrained files in the database used for testing (aws/aws-sdk-go), but the existing demand-specialized plan starts from demanded call targets and function bodies, passes through several location joins, and produces roughly 15–19 million intermediate tuples before joining with the small set of build-constraint comments.

The new private pragma[nomagic] helper computes the build-constrained files directly. The public predicate remains magic-enabled, so demand propagation in callers is preserved. Its bound variant now joins the demanded files against the small precomputed relation, reducing its tuple sum to roughly 40,000.

DCA issue

DCA repeatedly reported large tuple-sum changes for:

Files::ExtractedOrExternalFile.hasBuildConstraints/0#4828fb61#b

These reports occurred even when both variants analyzed the same source commit with effectively unchanged CodeQL code.

The old plan contains a large intermediate location join evaluated with fine-grained parallelism. The number of tuples recorded for that join depends on worker scheduling and duplicate handling, so repeated evaluations of the same query and database produced materially different tuple sums while returning identical results.

Because this predicate is shared through the recursive mayHaveSideEffects analysis, it can also be requested by different queries or stages depending on parallel evaluation order. This can change its queryCausingWork and stage attribution in DCA. Stage-key differences can make the resulting report more confusing, but the underlying problem is the large schedule-dependent intermediate join.

By computing the constrained-file relation bottom-up, this PR removes that join from the bound predicate. The remaining work is small and stable, so it should no longer cross DCA's tuple-sum reporting threshold or produce recurring spurious alerts.

Evaluate the small set of build-constrained files bottom-up while keeping the public predicate demand-specialized. This avoids a large, schedule-dependent location join without changing query results.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@owen-mc
owen-mc requested a review from a team October 16, 2026 19:27
@owen-mc owen-mc added the no-change-note-required This PR does not need a change note label Oct 16, 2026
@github-actions github-actions Bot added the Go label Oct 16, 2026
@owen-mc

owen-mc commented Oct 17, 2026

Copy link
Copy Markdown
Contributor Author

I ran DCA. Tuple sums for that predicate seem much improved. There is a result for Log ERROR lines, per source (v1) that I need to look into.

@owen-mc
owen-mc marked this pull request as ready for review October 17, 2026 09:24
@owen-mc
owen-mc requested a review from a team as a code owner October 17, 2026 09:24
Copilot AI max review requested due to automatic review settings October 17, 2026 09:24

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The localized refactor preserves predicate semantics while establishing the intended evaluation boundary.

No open issues 🎉

What changed in this PR

Stabilizes Go build-constraint evaluation to reduce expensive location joins and recurring DCA noise without changing results.

Changes:

  • Adds a private bottom-up helper using pragma[nomagic].
  • Preserves demand specialization on the public hasBuildConstraints() predicate.
File Description
go/​ql/​lib/​semmle/​go/​Files.qll Separates build-constraint discovery from caller demand propagation.

Review effort: Max

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Go no-change-note-required This PR does not need a change note

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants