Enforce oracle-first work: oracle lint rules, CI checks, and agent feedback hooks #2234

Why

Agents keep taking the shortcut path when they change behavior in an area that
an oracle owns. They change production first, then make the oracle pass with a
relaxed comparison, an exclusion, a model rule that matches the new output, or
one pinned example. AGENTS.md and the oracle guide already forbid this. The
prose did not stop it, because nothing made the shortcut visible or slow.

PR #2233 adds the scoring section in AGENTS.md ("How oracle work is scored"),
ORC-015 declared reach, and createOracleReach in
packages/db/tests/oracle-reach.ts. This plan adds the mechanical checks.

The evidence comes from PR #2168 (source-health channel). A production-first
rewrite of reload handling there needed four more review rounds and about ten
follow-up commits. Each round found a defect in the previous fix. These
shortcuts appeared in that work:

• Production changed before the oracle could see the defect.
• A truncate step could never land while a reload was in flight, because the
  driver's delayed markReady() always ran within the same step. The oracle
  was green in a state that it never reached.
• A direct-subscriber observation existed only on eager sources. The
  subscription lifecycle oracle subscribed without initial state, so its
  replay flush published nothing.
• Model rules changed to match production traces. Two of them (an automatic
  query restart, and "a key requested again drops a replay failure") were later
  found wrong and removed.
• Comparison exclusions (replayOpen, a narrowed identity check) widened
  without a fresh maintainer decision.
• Most edits went through shell commands (Python and sed), so an edit hook
  that matches only the harness's own edit tool would not fire.

Order of work

1. Install the agent feedback hooks with a small starter rule set, so that the
   session that does the cleanup gets feedback as it works.
2. Add the custom ESLint oracle rules. Run them over the repository and record
   every existing violation.
3. Clean up the violations owner by owner, each with the oracle-first order
   from AGENTS.md. Expect to find real gaps: a conditional expect often
   hides an unreached state.
4. Add the CI jobs (RED replay, owner coupling, model tripwire).
5. Adopt createOracleReach in packages/db/tests/source-health-oracle.test.ts
   (PR feat(db): split Collection status from sync errors; add reload() and cleanup admission #2168, or a follow-up after it merges).

Static checks: custom ESLint rules

Apply these rules to files whose names contain oracle. Keep them syntactic
and fast, so the edit hook can run them on one file.

Diff checks in CI

• RED replay. The PR's evidence block names the oracle tests. CI runs those
  tests from the PR head against the merge-base production src/. They must
  fail there and pass on the head. This checks the oracle-first order
  mechanically: an oracle that cannot see the defect cannot produce RED.
• Owner coupling. A production change in an area that the coverage map
  assigns to an owner, without a change to that owner and without an evidence
  block, fails. This needs a machine-readable table of owners to production
  paths, which the coverage map does not have yet.
• Model tripwire. A PR that changes both production and a model region gets
  a flag for the auditor. It does not fail the build. It makes the pair hard to
  miss.

Run-time checks

• Declared reach (createOracleReach, ORC-015).
• Mutation testing (for example Stryker) on the production files that each
  oracle owns, as a nightly job. A low kill rate for an owner shows an oracle
  that accepts wrong implementations. It is too slow for each PR.

Immediate agent feedback across harnesses

Contributors use several agent tools. Most now have a post-edit hook that can
return lint output to the model. Each has its own configuration path, tool
names, and feedback protocol. Research from 2026-10-23:

Sources: code.claude.com/docs/en/hooks, developers.openai.com/codex/hooks,
cursor.com/docs/hooks, geminicli.com/docs/hooks/reference,
docs.github.com/en/copilot/reference/hooks-reference,
docs.windsurf.com/windsurf/cascade/hooks, the Cline repository's
.clinerules/hooks/README.md, opencode.ai/docs/plugins, and
aider.chat/docs/usage/lint-test.html. Check each one again before you build:
these APIs change often.

Design

1. One shared script with thin adapters. scripts/agent-lint-feedback.mjs
   reads the hook's JSON from stdin, finds the edited paths (each harness uses
   a different field, and Codex sends the patch text of apply_patch), runs
   ESLint on those files only, and writes the response in the format the
   harness expects (--format=claude|codex|gemini|copilot|cline|windsurf).
   Each harness gets a small committed configuration that calls the script.
2. An end-of-turn hook for shell edits. A post-edit hook matches only the
   harness's own edit tool. Where a harness has an end-of-turn hook, run lint on
   git diff --name-only there. That catches shell edits too, and it can force
   another turn. For the other harnesses, the pre-commit hook is the fallback.
3. Speed. A hook that takes seconds on each edit gets turned off. Use
   eslint --cache on single files, or a daemon such as eslint_d. Keep the
   oracle rules syntactic, and keep type-aware rules out of the edit hook.
4. Fallbacks. Some harnesses need setup by each contributor (Codex trust,
   the Cline toggle, the Copilot default branch). Keep these layers:
   • an AGENTS.md instruction to run pnpm lint:changed after edits.
   • editor diagnostics, which IDE-based agents read.
   • the pre-commit hook.
   • CI.

Pre-commit

lint-staged already runs eslint --fix on staged files. The oracle rules run
there once they are in the ESLint configuration. On a commit of more than about
100 files, ESLint ran out of memory in PR #2168. Lint in chunks, or raise the
Node heap in the hook.

This issue was written by Isaac.