Research: refactoring-aware metamorphic regression testing for ownership analysis (RefactoringMiner prior art) #432

Classification

• Type: bounded research / analyzer-regression testing spike
• Priority: P2 / backlog; no production integration by default
• Area: C# Roslyn source mappings → OwnIR ownership facts → interprocedural reasoning → diagnostic correspondence across revisions
• Prior art: tsantalis/RefactoringMiner; Alikhanifard & Tsantalis, A Novel Refactoring and Semantic Aware Abstract Syntax Tree Differencing Tool and a Benchmark for Evaluating the Accuracy of Diff Tools (DOI)
• Related Own.NET: Research: semantic mutation testing harness for Own.NET analyzer coverage (bounded pilot) #405 (semantic mutation harness and negative controls), Research: SootUp-inspired OwnIR/ICFG semantic-fidelity and incomplete-call audit (bounded spike) #430 (source→OwnIR/ICFG fidelity), P-022 diagnostics: normalized AnalysisTrace and first-divergence minimizer for Python↔Rust shadow mode #269 (first-divergence trace), Research: evaluate Bifrost for C# code intelligence, agent navigation, and evidence contracts #426 (evidence/completeness), Canonical first-party call facts: preserve interprocedural calls instead of collapsing them into frontend release/use/escape guesses #382 (canonical call facts), ConsumesParam flattens guarded release into unconditional handoff, causing false OWN009 and false must summaries #380 (guarded disposal)
• Default decision: do not port/adopt RefactoringMiner, do not replace OwnIR/Roslyn, and do not claim behavioral equivalence from a recognized refactoring.

Why investigate

Most regression tests exercise one syntactic form of an ownership scenario. Yet real code is routinely renamed, extracted into helpers, inlined, moved between files/partial declarations, or structurally rewritten without intentionally changing ownership behavior. Those transformations can expose losses at Roslyn extraction, OwnIR lowering, interprocedural analysis, summary/call mapping, or diagnostic location mapping.

Research question: Can a small set of independently reviewed, behavior-preserving C# refactorings expose previously untracked Own.NET false negatives, false positives, or diagnostic-evidence regressions? Can we trace the first divergence without creating another generic AST-diff engine?

The evaluation target is incremental, reproducible analyst value, not a better-looking diff, a mutation score, or a new proof layer. In particular, do not mistake a structurally plausible AST mapping for an ownership-sound semantic equivalence argument (relevant after S5's STOP-SOUNDNESS).

What RefactoringMiner contributes (and what it does not)

Its important architectural contribution is refactoring-aware AST differencing, not a replacement for AST diff:

1. Match declarations/program elements first, using language-level clues (signatures, call sites, field references, etc.).
2. Align statements and recognize higher-level transformations (Extract/Inline/Move/Rename and related operations).
3. Run localized tree matching (including GumTree 3.0 Simple-based steps), constrained/corrected by the previously found correspondences.
4. Represent cross-file moves and one-to-many / many-to-one / many-to-many mappings where ordinary one-to-one tree matching loses intentional correspondences.

The research benchmark includes 800 bug-fixing commits and 188 refactoring commits; its reported mapping accuracy is evidence for the approach on that benchmark, not evidence of correctness for C# or Own.NET. “Semantic-aware” here means better matching of syntactic nodes using language semantics, not proof of program equivalence.

RefactoringMiner also has a Purity Checker, AST diff API, and MCP/agent-facing tooling. An identified “pure” refactoring means the tool did not detect overlapping behavior-changing edits under its model; it cannot discharge C# ownership, exception, aliasing, or observational-equivalence obligations. Do not enable its agent tooling against C# on the assumption that it is supported.

C# support caveat: inspect, do not depend on the unofficial port

The upstream project does not currently list C# as a supported language. An unofficial C# fork exists, with a documented C# → srcML/XML/CPatMiner → Java-model adaptation. On inspected fork revision 48871150, there are concrete source-level red flags:

• CSharpGitHistoryRefactoringMiner.createModel(...) is declared static, while the parent GitHistoryRefactoringMinerImpl.detectRefactorings(...) calls its own static createModel(...). Java static method hiding is not virtual dispatch: the subclass's C# routing is not invoked by those parent calls, even when the overridden detectRefactorings delegates via super. This is a static source-path finding, not a runtime reproduction.
• CSharpRefactoringMiner CLI explicitly throws UnsupportedOperationException for diff: C# AST diff is not implemented there.
• In that CLI, detectAll(String[] args) calls detector.detectAtCommit(repo, args[2], ...) rather than detectAll(...); inspect as another potential CLI wiring defect, but do not assert reproduced breakage without running it.

Treat the fork as unverified/incomplete research material, not a C# dependency or validated baseline. The best first C# solution is likely Roslyn-based, with ideas borrowed from the paper. Any further fork inspection is strictly optional and timeboxed.

Bounded pilot

0. Pre-register and reuse existing infrastructure

• Pin the Own.NET commit, extractor/analysis mode, runtime/toolchain, fixture IDs, commands, reference-paper/tool revisions.
• Read and reuse Research: semantic mutation testing harness for Own.NET analyzer coverage (bounded pilot) #405's harness/corpus and Research: SootUp-inspired OwnIR/ICFG semantic-fidelity and incomplete-call audit (bounded spike) #430's fact-preservation ledger wherever applicable. Do not create a parallel test runner, IR representation, or new evidence schema.
• Before executing transforms, independently record expected ownership state and diagnostics for each original/modified variant. Include clean cases and confirmed-positive cases; missing warnings are only meaningful for supported bug families.

1. Curate 12–20 reviewed source-pair fixtures, not a giant generator

Start with 5 kinds of C# changes, at least two reviewed cases for each applicable kind:

1. Rename / symbol-preserving reshaping: local/method rename and parameter rename when references/dispatch/reflection are unaffected.
2. Extract / inline variable: include aliasing and evaluation-order traps; accept only validated equivalent instances.
3. Extract / inline method: especially resource acquire/use/release crossing the call boundary, including a guarded branch.
4. Move code / declarations: move implementations between partial files or helper methods while checking symbol identity, accessibility, binding, and initialization order.
5. Control-flow-preserving reshaping: equivalent branching/guard normalization with using / try/finally cases only when all normal and exceptional paths remain equivalent.

Suggested coverage: allocation/acquire, alias, use, escape/return/transfer, release/Dispose, exceptional cleanup, partial Dispose, and a first-party call boundary. Use existing Own.NET samples where possible.

Include 2–4 deliberately non-equivalent near-neighbors, such as moving a release outside finally, changing its guard, disposing a different alias/resource, or altering a consuming callee. These are positive controls for catching an overconfident matching/purity oracle. They are not additional generic mutation operators; #405 owns such operators.

Reject/refuse to score transforms that change overload resolution, virtual dispatch, closure/async lifetimes, resource identity, disposal timing, exception behavior, conditional compilation, reflection-observed names, or side-effect evaluation order unless the changed semantics are the explicit test target. Every scored pair must compile and pass Roslyn extraction.

2. Prototype source correspondence, not a new AST-diff engine

Use Roslyn SyntaxTree, SemanticModel/ISymbol, and IOperation where useful, plus manually curated transformations. Create a minimal offline mapping ledger:

• old ↔ new declaration/symbol and statement/operation spans; support cross-file moves;
• optional one-to-many / many-to-one relation for extract/inline (never force a bogus 1:1 match);
• what is mapped / ambiguous / unsupported, with reasons;
• optionally an offline ownership-event trace (Acquire, Alias, Use, Transfer, Dispose, Escape) derived from existing OwnIR facts, not a new normative IR.

The objective is to map comparable source behavior/diagnostics despite changed syntax. Distinguish mapping failure from analyzer failure. Do not infer ownership correctness from AST similarity or merely matching event counts.

3. Differential/metamorphic runs and first-divergence audit

For each independently validated semantics-preserving pair:

• Expect existing confirmed findings (family, affected resource, relevant path) to persist modulo mapped location and evidence, not identical line numbers/text or arbitrary exact count.
• Expect previously clean behavior to remain free of newly introduced applicable warnings.
• Compare source semantics → Roslyn facts → OwnIR/call/summary → analyzer state/diagnostic → report locations. Where a contradiction occurs, locate the first divergence and determine whether it is extraction/lowering, CFG/ICFG, modeling, transfer/join, reporting, oracle, mapping ambiguity, or an already known limitation.
• For deliberately non-equivalent near-neighbors, verify that the comparison does not label the change pure/equivalent simply because most AST nodes correspond. Where Research: semantic mutation testing harness for Own.NET analyzer coverage (bounded pilot) #405 supplies expected injected defects, reuse its oracle.

Do not equate absent findings with proven safety, a changed warning location with a regression, or a nonzero exit with successful detection.

4. Optional PR-review experiment, only if justified

If the corpus reveals a real mapping/diagnostic regression, apply the same offline approach to one small real C# refactoring PR with manually confirmed ownership-relevant before/after behavior. Assess whether refactoring-aware review isolates the meaningful change better than normal git diff. No public action, always-on bot, GitHub Action, PR annotations, or MCP integration without a separate decision.

Result classes / artifacts

For each pair record: stable ID, source refs/diff, transformation, compile/extraction status, manually reviewed semantic preconditions, expected result, correspondences with unmapped/ambiguous nodes, baseline/new diagnostics and evidence, earliest substantiated divergence, reproduction command, run times (if measured), and disposition.

Classify at least:

• Preserved: expected matched findings / clean behavior survived a validated equivalent change.
• Confirmed regression (FN/FP/evidence): externally reviewed source equivalence + reproducible analyzer difference.
• Expected semantic change: deliberately non-equivalent control with justified changed outcome.
• Invalid/non-equivalent: transformation failed its purported equivalence/compilation/extraction requirement.
• Mapping ambiguous / oracle unresolved: no sound comparison possible; exclude from scored denominators.
• Infrastructure error / timeout / known limitation: record separately, never count as success.

A count of Preserved examples is not an accuracy proof. A single true analyzer bug with a clean, independent oracle is more valuable than 100 cosmetic matches.

Acceptance / stop gate

• A pinned and independently runnable 12–20-pair C# fixture ledger spanning the five transformation families where applicable (or documented exclusions).
• All scored cases compile/extract; semantic assumptions are independently reviewed, including exceptional/alias/call boundaries as relevant.
• At least 2 non-equivalent near-neighbors demonstrate the oracle does not blindly trust recognized refactorings.
• Reuses Research: semantic mutation testing harness for Own.NET analyzer coverage (bounded pilot) #405/Research: SootUp-inspired OwnIR/ICFG semantic-fidelity and incomplete-call audit (bounded spike) #430 facilities where available; produces mapping/evidence records without shipping another general-purpose diff engine.
• Every discrepancy has mapped diagnostic context and a first-divergence hypothesis backed by source/OwnIR excerpts; ambiguous cases are not called Own.NET defects.
• A short comparison of structural-only vs symbol/refactoring-aware matching: where does additional complexity actually remove ambiguity?
• Explicit findings and STOP / RESEARCH ONLY / NARROW FOLLOW-UP decision with running/maintenance costs.
• Promote only for ≥1 independently reproduced, previously untracked ownership-related regression/gap or substantial incremental coverage that existing Research: semantic mutation testing harness for Own.NET analyzer coverage (bounded pilot) #405 tests demonstrably miss at reasonable cost.

Out of scope: full RefactoringMiner port, fixing the unofficial fork, unbounded source-mining/fuzzing, semantic-equivalence proof claims, production safety/proof badges, new diagnostics, and a mandatory CI gate in the first pilot.

References

• Official RefactoringMiner
• Refactoring-aware AST diff paper, benchmark: 800 bug fixes + 188 refactoring commits
• Official AST-diff API
• Purity Checker
• Official MCP review tooling
• Unofficial C# fork and README, not validated as working

Success means finding and explaining a concrete ownership-analysis inconsistency that ordinary fixtures miss, or credibly ruling out enough benefit to STOP.