Research: evaluate Bifrost for C# code intelligence, agent navigation, and evidence contracts #426

Classification

• Type: bounded research / external-tool evaluation
• Priority: P2 / backlog
• Area: C# symbol resolution, call/usage graph, evidence provenance, agent navigation
• Default decision: no production integration, no OwnIR rewrite, no change to diagnostic semantics
• External project: BrokkAi/bifrost (Apache-2.0); documentation

Motivation

Bifrost is a Rust-based multi-language code-intelligence/static-analysis engine exposing structural queries, resolved call/reference relationships, demand-driven data-flow, and MCP/LSP/CLI interfaces. Its most relevant ideas for Own.NET are:

1. A two-axis evidence contract: whether a returned relation is proven, and whether the requested search scope is complete. Proven positive evidence is not an exhaustive search; an empty partial search is not proof of absence.
2. Stable semantic identity and source provenance across queries/adapters.
3. Demand-driven graph work and reusable repository indices.
4. Potentially better agent navigation over nontrivial C# code, rather than another speculative ownership diagnostic.

Do not infer that Bifrost already solves Own.NET's ownership/disposal problem. The documented bifrost.correctness.resource-lifecycle policy does not claim C# support: its reviewed close binding is incomplete for C#. Documentation also describes C# taint discovery as potentially inconclusive. The shipped v0.13 policy catalog is not automatically populated merely because a policy is documented.

This spike responds to the disappointing practical utility and soundness limitations seen in prior speculative analysis work (including S5): require empirical benefit and explicit uncertainty before adding another analytical layer.

Hypotheses to test (not assumed facts)

• H1, code navigation: on a fixed C# fixture set, Bifrost can return useful symbol/call/reference evidence not readily or reliably obtained from the current agent workflow, without silently claiming resolution when it is unknown.
• H2, uncertainty contract: separating relation proof from search completeness would prevent specific classes of misleading Own.NET output (e.g. "no findings" after an incomplete traversal). This is a design audit first, not automatic adoption.
• H3, marginal agent utility: a Bifrost-assisted agent locates the correct ownership-related implementation paths with fewer false leads and/or less effort than the existing workflow.
• H4, runtime cost: indexing, query latency, cache invalidation and memory are acceptable for an optional local tool. No assumed performance win.

Bounded investigation

A. Pin and reproduce

• Pin Bifrost version/commit, CLI invocation, fixture snapshot, OS/runtime, resource limits, and exact commands. Use an isolated workspace; no edits to the Own.NET production pipeline.
• Check actual supported capabilities and result fields before designing assertions. Distinguish unsupported, not run, inconclusive/partial, ambiguous, budget exceeded and true complete answers.
• Preserve raw machine-readable outputs, stderr, exit codes and command timings. Do not equate a zero-result scan with clean unless scope/completeness justify it.

B. C# semantic-resolution micro-corpus

Build 12–20 small, labeled cases, reusing existing Own.NET tests where possible:

• overloads, generic methods, extension methods;
• interface/virtual dispatch, overridden methods;
• partial classes / partial Dispose implementations across files;
• event += / -= subscriptions and teardown via helper calls;
• using, try/finally, conditional cleanup, exception paths;
• ownership transfer, escape, alias to the wrong resource;
• async/await and callback boundaries;
• conditional compilation (#if), generated code, or unresolved external calls.

For each case, document the exact question, manually reviewed expected answer, the Roslyn/OwnIR-known facts (where available), and Bifrost's actual query/output. Compare correct resolved facts, missing facts, false relationships, and honestly unknown cases; do not score an unsupported case as a true negative. Prefer a small adversarial corpus to a large unverified dump.

Important: Bifrost's stock C# resource-lifecycle diagnostic is not a head-to-head Own.NET benchmark. If a custom C# policy is attempted, record the semantic model and supported obligations explicitly; do not weaken or silently redefine soundness to make it pass.

C. Agent workflow A/B

Select 5–8 pre-registered real Own.NET navigation tasks (e.g., identify a Dispose/ownership rule's actual entry point, locate an effective interprocedural call chain, find all relevant partial declarations or overload call sites).

Compare under the same repository snapshot, task instructions, model/budget and answer rubric:

• Baseline: existing agent workflow with source search and available Roslyn/compiler evidence.
• Treatment: same workflow plus Bifrost CLI/MCP.

Capture correctness against manually verified ground truth, false leads, abstentions, elapsed time, tool calls/tokens where measurable. Report individual cases, not only an aggregate "accuracy" percentage. Keep the tool from self-grading its own findings.

D. Evidence contract audit

Independently assess whether Own.NET's current result/diagnostic representation can distinguish:

• a proven-positive witness vs an unproven candidate;
• complete bounded absence vs no result from unsupported/incomplete traversal;
• call-target uncertainty / dispatch frontier;
• source snapshot, analysis scope, model and budget provenance;
• actual solver failure vs transport/tool failure.

Write 2–3 concrete counterexamples where conflation would change the user-visible conclusion, with proposed minimal schema/wording changes only if a real gap is reproduced. Make no proven promise unsupported by a soundness argument and tests.

E. Operational profile

On a pinned subset of Own.NET measure cold index, warm query, small-edit reindex, peak RSS and disk cache size. Record query families, scope and repeat count. If unsupported or too slow, record that result without workarounds that obscure the actual cost.

Stop / decision gates

Stop and mark NO-GO if the relevant C# capability is unsupported, unsupported states are disguised as clean, or the setup cost outweighs the bounded navigation benefit.

Treat the following as distinct outcomes:

• NO-GO / no measured gain: not worth integrating, with reusable fixtures/evidence retained.
• RESEARCH ONLY: useful design concepts (e.g. evidence/completeness schema) but no demonstrated operational value.
• FOLLOW-UP CANDIDATE: reproducible nontrivial benefit in a named task with no material regression or unsoundness; open a separate narrowly scoped implementation issue.

No production dependency, permanent MCP configuration, generic engine migration, or new diagnostics from this issue. No claim of improved detection until measured against known-positive and known-negative examples.

Deliverables / acceptance criteria

• Reproducible report with version/commit, commands, environment, fixtures, expected outputs and raw evidence.
• Per-case resolution/confidence/completeness table with failures and abstentions visible.
• Paired agent-task results (baseline vs Bifrost), including unsuccessful cases.
• Bounded index/latency/memory measurements or a documented reason they could not be run.
• Evidence-contract gap assessment with concrete counterexamples, or explicit no actionable gap found.
• Final NO-GO / RESEARCH ONLY / FOLLOW-UP CANDIDATE verdict; any implementation split into a new issue.

References and related work

• Bifrost source
• Bifrost: evidence and result contract
• Bifrost: C# and resource-lifecycle policy limitations
• Bifrost: data flow and typestate
• DataFlowBench (possible independent fixture/reference source; verify what is actually runnable)
• Own.NET Research: OpenTaint-style contract tests and engine-gap triage for method-effect models #425 (OpenTaint-style method-effect contract tests; keep this spike focused on Bifrost navigation/evidence)
• Own.NET Audit foundation: finding identity v2 (pattern, occurrence, lineage) with provenance #266 (finding identity/provenance; assess overlap rather than inventing a parallel schema)