diff --git a/docs/designs/saiph/README.md b/docs/designs/saiph/README.md new file mode 100644 index 0000000..4823995 --- /dev/null +++ b/docs/designs/saiph/README.md @@ -0,0 +1,432 @@ +# Saiph + +Design records for Saiph, the version control system Rigel sells. Its code +will live in `RigelBuild/saiph`, which does not exist yet. Records go in +this directory. Decisions go in `decisions//DL-NNN.md`; the areas +are `overview`, `store`, `client`, `review`, `ci-cd`, `merge-queue` and +`git-bridge`, one per record. + +This page is the overview of the set. + +Tracker: RIG-5665 + +> Freezes on merge; later changes supersede by citation, never rewrite +> (convention: the `docs/designs/agent/compass-agent-config-delivery/design.md` +> header in the Compass designs). Unless a path says otherwise, every code +> path is in the `RigelBuild/saiph` repo. A Compass path says "in +> Compass". + +Ledger-impact: DL-580 files D1. The set replaces the deleted Compass +record `internal-vcs.md`; DL-578, DL-579 and DL-581 point at +[store](store.md#decisions). The sibling records file DL-590..DL-626. The +RIG-5558 decisions (DL-535..DL-538, DL-561) and DL-090 and DL-326 stay +whole: they govern the git backend. + +**Named decisions this record introduces:** D1 in [Decisions](#decisions). + +## Open Questions + +Every record's questions, in one index. Matt reads this first. Each +record is written against its recommendation. "LB" marks a load-bearing +question: a different answer reshapes the record. + +| Id | Question | Recommendation | LB | +| -- | -- | -- | -- | +| overview OQ-1 | Which license does the saiph repo take? | AGPL-3.0-only, as Compass | n | +| overview OQ-2 | Do review, queue, CI and git bridge ship in the saiph repo? | Yes, as Saiph services | y | +| overview OQ-3 | Where do conflicts live? | Never stored; typed results; open conflicts stay in the working copy | n | +| overview OQ-4 | Tenant model in a single-tenant install? | One tenant row at boot; RLS stays on | n | +| [store](store.md#open-questions) OQ-1 | Which metadata database? | YugabyteDB YSQL (core only); one Postgres ≥16 allowed on one node | y | +| store OQ-2 | Which bytes are hashed? | Deterministic CBOR (RFC 8949 §4.2.1) | y | +| store OQ-3 | May a reader fetch an object by id alone? | Only with read on the whole repo | n | +| store OQ-4 | Retention of unpinned changesets? | 7 days | n | +| store OQ-5 | Does the store keep conflicted changesets? | No | n | +| store OQ-6 | How does `saiph-server` know the caller? | JWTs from configured issuers; Compass is one | y | +| [client](client.md#open-questions) OQ-1 | Checkout mode for agent sessions? | Virtual on Podman, materialized on microVM | y | +| client OQ-2 | `saiphd` upgrade without breaking mounts? | FUSE takeover; restart as fallback | n | +| client OQ-3 | Does `saiphd` start from a snapshot? | No; node disk plus a warm fetch | n | +| client OQ-4 | Mixed git and Saiph session keeps GitHub egress? | Yes, while any repo is on git | n | +| client OQ-5 | Does the agent hold a Saiph token? | No; the Runner relays to `saiphd` | n | +| client OQ-6 | How is a Saiph-only repo granted to an agent? | Accept it by its cutover state | n | +| client OQ-7 | Virtual checkouts on macOS? | No; materialized only | n | +| client OQ-8 | How does a human get a token from Compass? | OAuth 2.0 device grant | n | +| [review](review.md#open-questions) OQ-1 | Who may upload a version of another owner's change? | Any account with write | n | +| review OQ-2 | How are change classes detected? | Lexically, from `Diff` hunks | n | +| review OQ-3 | May a read-narrowed reader use review? | No | n | +| review OQ-4 | How long do closed changes' versions stay readable? | Re-pin for 90 days | n | +| review OQ-5 | What does a repo without `.saiphreview` require? | One human LGTM, owners and CI | n | +| review OQ-6 | How does an agent learn it is its turn? | A home-channel message from Compass | n | +| [ci-cd](ci-cd.md#open-questions) OQ-1 | One CI config file or one per directory? | One `.saiph/ci.yaml` | n | +| ci-cd OQ-2 | How does moon see a Saiph checkout? | A Saiph provider for moon's pluggable VCS | y | +| ci-cd OQ-3 | May the deploy controller move `release/*` refs? | Yes, by a ref-prefix grant | n | +| ci-cd OQ-4 | Does CI revert a confirmed culprit? | No; it comments | n | +| ci-cd OQ-5 | Compass jobs Kubernetes cannot run? | Keep them on GitHub Actions against the mirror | n | +| ci-cd OQ-6 | Is a flaky target quarantined? | Yes | y | +| [merge-queue](merge-queue.md#open-questions) OQ-1 | Pre-merge or post-merge gating? | Per-repo mode, default `pre` | y | +| merge-queue OQ-2 | Does a conflict-fix version need a new approval? | The review policy decides | y | +| merge-queue OQ-3 | Which model does the fixer call, and how? | One OpenAI-compatible chat call | n | +| merge-queue OQ-4 | What is trivial by default? | Conservative: small text content conflicts | n | +| merge-queue OQ-5 | Where does queue policy live? | `.saiphqueue` at the trunk head | n | +| merge-queue OQ-6 | What does the queue do with `INFRA`? | Hold the batch | n | +| merge-queue OQ-7 | Land-by-merge mode? | Yes, per owner | n | +| [git-bridge](git-bridge.md#open-questions) OQ-1 | GitHub PRs after cutover? | Import allowlisted PRs; land by merge | y | +| git-bridge OQ-2 | PRs from authors not on the allowlist? | Comment and close | n | +| git-bridge OQ-3 | History Saiph cannot hold? | Refuse; the repo stays on git | y | +| git-bridge OQ-4 | Open GitHub PRs at the switch? | Import them | n | + +Dependencies: git-bridge OQ-2 and OQ-4 depend on git-bridge OQ-1; +client OQ-6 depends on git-bridge's cutover state; merge-queue OQ-6 and +ci-cd OQ-6 are siblings. The overview's own questions follow. + +### OQ-1 — Which license does the saiph repo take? + +Compass is AGPL-3.0-only. Saiph is sold, self-hosted, and used through +Compass. + +- (a) AGPL-3.0-only, as Compass. Customers who self-host get the source; + a vendor that runs a modified copy as a service must share it. +- (b) A source-available license. It restricts competitors more, but + customers lose the freedoms Compass gives them. + +**Recommendation: (a).** One license across Rigel's products. Either way, +no code from `facebook/sapling` enters the repo: it is GPL-2.0-only, and +GPL-2.0-only code cannot be copied into AGPL-3.0 code (ruling 12). + +### OQ-2 — Do review, the merge queue, CI and the git bridge ship in the saiph repo? + +Matt placed the schema, proto, binaries and deploy manifests in the saiph +repo, and kept the agent `vcs_*` tools, the Runner backend pick and the +web UI hooks in Compass. The workflow services are not named. + +- (a) In the saiph repo, as Saiph services. Scion says such a system + "requires its own code review, code search, and CI", so a customer who + self-hosts Saiph without Compass still needs them. +- (b) In Compass. Saiph without Compass would then have no review or + merge queue. + +**Recommendation: (a).** Compass keeps only its UI hooks and agent tools. +Each sibling record designs its service on this assumption. + +### OQ-3 — Where do conflicts live? + +Shared by store, review and merge-queue. **Recommendation:** the store +never keeps a conflicted changeset; conflicts are typed results, and the +`saiph` client keeps open conflicts in the working copy, as Sapling does +([store](store.md#oq-5--does-the-store-keep-conflicted-changesets-shared-with-review-and-merge-queue)). + +### OQ-4 — Which tenant model in a single-tenant install? + +**Recommendation:** one tenant row, created at boot. The tenant column and +RLS stay on, so an install can add tenants with no migration +([store](store.md#tenancy)). + +## Resolved decisions + +- **Split and name (RIG-5821 R2-1, R2-2).** Matt ruled on 2026-10-19: + Saiph is its own product in its own repo, `RigelBuild/saiph`. Rigel + sells it; customers self-host it or use it through Compass, and the + Compass repo is its first user. The proto package is `saiph.v1`, the CLI + is `saiph`, the node daemon is `saiphd`, and the server is + `saiph-server`. +- **Client.** Matt ruled on 2026-10-18: Saiph has its own client, not jj. + It is inspired by jj and Sapling and built Scion's way: a node daemon, a + bounded object cache, sparse and virtual checkouts, and server-side + mutations ([client](client.md)). +- **Scion to the letter.** Matt ruled on 2026-10-18: the core (server and + client) follows the Scion post. Where the post is silent, the + `facebook/sapling` repo (Mononoke, EdenFS, the Sapling client) is the + reference. Store departures are listed in + [Deviations from Scion](store.md#deviations-from-scion). + +## Rulings + +Matt, 2026-10-18 and 2026-10-19 (docs#48 threads, RIG-5821). These are +fixed. + +> 1. A full Scion implementation: many Servers and many Runners. The single +> Server is only for dogfooding. Mirror Scion as closely as possible. +> 2. The git mirror carries trunk only. Changes under review are never +> mirrored. No "PR" and no branches unless a reason exists. Scion is the +> reference; Twigg, Piper, Critique and jj inspire the UI and workflow. +> 3. Review is Critique/Twigg/jj: changes stack freely into DAGs, and CI +> uses the graph. +> 4. No GitHub egress for agents on a fully native repo. GitHub mirroring is +> for migrating repos and for public visibility. The Compass repo stays +> fully mirrored. +> 5. Two backends. RIG-5558 (`agent-vcs-tools.md`) ships whole for repos on +> git and GitHub. Saiph is a second backend, picked per repo behind the +> agent `vcs_*` tools. +> 6. No length cap; split into files as needed. +> 7. CI/CD is designed alongside, on Kubernetes. +> 8. The product may split out with its own name (RIG-5821 R2-1, R2-2). +> 9. A merge queue (2026-10-18): batching, bisection, stacks as units, +> per-affected-target lanes instead of a global lock, server-side +> auto-rebase, and a cheap-model fixer for trivial conflicts; a +> non-trivial conflict ejects the change. +> 10. The core VCS, server and client, follows Scion to the letter. Where +> the Scion post is silent, the eden/sapling repo (Mononoke, EdenFS, +> the Sapling client) is the reference. +> 11. Our own client, not jj: inspired by jj and Sapling, built Scion's +> way (node daemon, bounded cache, sparse and virtual checkouts, +> server-side mutations). A 16-byte change id stays, as our own +> concept. +> 12. Sapling is GPL-2.0-only and Compass is AGPL-3.0-only: borrow designs +> and ideas only, never code. Running an unmodified EdenFS as a +> separate program is not in scope. +> 13. The product is named Saiph (2026-10-19). It is sold: customers +> self-host it or use it through Compass, and the Compass repo is its +> first user. Proto `saiph.v1`; CLI `saiph`; daemon `saiphd`. +> 14. Saiph lives in its own repo, `RigelBuild/saiph` (2026-10-19). Its +> schema, proto, binaries and deploy manifests belong there. Compass is +> a client: the agent `vcs_*` tools, the Runner backend pick and the +> web UI hooks stay in Compass. + +## Problem / Intent + +Compass agents keep history in git and land work through GitHub. That ties +every agent to a forge, a clone and a push credential, and it scales the +way git does. Saiph is a Scion-style version control system that Rigel +sells and Compass uses first. This record maps the set: goals, terms, +components, how Saiph sits beside the git backend, and the order of work. + +### Goals + +- A server that owns history: it stores changesets, builds trees, commits, + rebases, merges and answers history queries + ([Scion](https://www.anthropic.com/engineering/redesigning-source-control): + "the server answers questions about file history and blame, creates + commits, performs merges and rebases"). +- Many stateless servers and strict cache tiers over immutable objects. +- Our own client that fetches only what it reads: sparse and virtual + checkouts through a node daemon. +- Review over a DAG of changes with versions, not branches. +- CI that reads the graph and the diff API, and a merge queue without a + global lock. +- Trunk exported to GitHub as derived git commits, and git history + imported. +- Tenancy, per-directory ACLs and enforced caps from the first release, + because Saiph is sold. + +### Non-goals + +- Replacing the git backend. RIG-5558 ships unchanged (ruling 5). +- Git wire compatibility. No clone, fetch or push of a Saiph repo over the + git protocol; the GitHub mirror is the git view. +- Mirroring changes under review (ruling 2). +- jj as the client (ruling 11), and running EdenFS (ruling 12). +- Managed-service operations. In Compass's terms, the boundary test is the + [OSS-core boundary record](../compass/meta/oss-core-managed-boundary/design.md#boundary-test). + +## Approach + +### Glossary + +| Term | Meaning | Follows | +| -- | -- | -- | +| changeset | An immutable entry in history, created only by the server: ordered parents, a change id, and the files changed against the first parent with content ids. | Scion ("an entry in a write-ahead log"); Mononoke Bonsai | +| blob | File content, named by BLAKE2b-256 of its bytes keyed `saiph.content`, stored compressed in object storage. | Scion ("compressed file contents in blob storage"); Mononoke `ContentId` | +| tree | A derived, content-addressed directory node whose entries carry sizes; never uploaded. | Scion (trees fetched recursively); Mononoke fsnodes | +| ref | A name and a target changeset, moved only by compare-and-swap. Few exist. | Scion ("a compare-and-swap on a single Spanner row") | +| trunk | The repo's main ref. It moves forward only, and each changeset on it has a dense `trunk_seq`. | Scion's main ref; Piper head | +| change | A unit of work across rewrites, named by a 16-byte change id that Saiph assigns. Not an object. | Saiph's own concept, inspired by jj ("a commit as it evolves over time") | +| version | One state of a change; a new version on each rewrite. | Twigg ("Increased by 1 on rebases, amends, msg changes and submits") | +| stack | The DAG of changes a change depends on, reviewed and landed as a graph. | Critique, Twigg, jj | +| workspace | One working copy on a node, laid out by `saiphd`, sparse and optionally virtual. | Scion ("sparse checkouts" and "virtualized checkouts"); EdenFS | +| derived data | A value computed from a changeset and its parents' values, written back to the database. | Scion ("defined recursively"); Mononoke `DerivableType` | +| pin | A holder's claim that keeps a changeset alive through gc, in place of a ref. | Saiph's own; Scion discourages refs | + +### Component map + +```mermaid +flowchart TB + subgraph compass[Compass repo: clients of Saiph] + T[agent vcs_* tools and Runner backend pick] + W[Compass web app: Saiph UI hooks, review proxy] + M[token minter and cutover row] + end + subgraph node[Each node: DaemonSet] + CLI[saiph CLI in each session] + D[saiphd: bounded object cache, workspaces, CSI driver] + end + subgraph region[Each region: Deployment] + RC[saiph-cache: tree and blob cache] + end + subgraph core[saiph-server: Deployment, replicas N] + S[saiph.v1: repos, objects, history, mutation, refs] + DW[derive and gc workers: lease rows] + end + subgraph flows[Saiph workflow services: Deployments] + RV[saiph-review: schema saiph_review] + MQ[saiph-queue: schema saiph_queue] + CI[saiph-ci: schema saiph_ci, Kubernetes Jobs] + GB[saiph-bridge: schema saiph_bridge] + end + DB[(metadata DB: YugabyteDB YSQL, or one Postgres; schemas saiph, saiph_*)] + OS[(object storage: file contents)] + GH[(GitHub mirror: trunk only)] + T --> CLI + CLI -->|socket| D + D --> RC --> S + CLI -->|Commit, Rebase, history| S + W --> RV + W --> S + M -.->|tokens| CLI + RV --> S + MQ -->|Rebase, Diff, UpdateRef chain| S + MQ --> RV + MQ --> CI + CI -->|WatchRefs, Diff| S + CI -->|WatchEvents| RV + CI -->|CSI volume| D + GB -->|WatchRefs, CommitBatch| S + GB --> GH + S --> DB + S --> OS + RV --> DB + MQ --> DB + CI --> DB + GB --> DB + DW --> DB +``` + +- **Server tier.** `saiph-server` runs as a Deployment with `replicas: N` + behind one Service. The metadata database (YugabyteDB core, or one + Postgres) and object storage are operator-provided. Manifests live in + the saiph repo (`deploy/`). +- **Cache tier.** `saiph-cache`, one Deployment per region, answers reads + by id and asks the servers on a miss. Scion: "When a cache is missing an + object, it asks the authoritative scion-servers". +- **Daemon tier.** `saiphd` runs once per node (DaemonSet). It keeps a + bounded object cache for every job on the node, lays out workspaces, + and serves CI pods through its CSI driver. Scion: "the client talks over + a socket to a daemon, scion-daemon, that stores objects for the whole + node". +- **Client tier.** The `saiph` CLI runs in the session and talks to + `saiphd` for objects and to `saiph-server` for mutations and history. + The daemon, cache and server answer the same object RPCs, so the client + does not know the topology. +- **Workflow tier.** `saiph-review`, `saiph-queue`, `saiph-ci` (with its + `saiph-ci-step` Job wrapper) and `saiph-bridge` are Saiph services + (OQ-2). Each keeps its rows in its own schema (`saiph_review`, + `saiph_queue`, `saiph_ci`, `saiph_bridge`) in the store's database, + under the store's tenant GUC, and calls `saiph.v1` with its own service + token. None holds history of its own. +- **Compass.** The agent `vcs_*` tools, the Runner backend pick, the web + UI hooks, the review proxy, the token minter and the `repo_cutovers` + row. Compass mints the tokens its agents and users present + ([store](store.md) OQ-6, [client](client.md)). + +### Two backends behind the `vcs_*` tools + +The agent `vcs_*` tools (RIG-5558, `createVcsTools` in the agent's +`vcs.ts` in Compass) stay the one tool surface. Each repo has a backend: + +- **git** (default): the RIG-5558 path, unchanged. Local jj over git, the + Compass Server's mirror, GitHub PRs, `forge_*` tools. +- **saiph**: the tools drive the `saiph` CLI. `vcs_submit` creates a new + version of each change on the server; review, CI and the merge queue + run in Saiph; trunk reaches GitHub only through the git bridge. + +The backend is a property of the repo, which Compass reads when it grants +a repo to an agent (`AgentRepositoryService` in Compass). The Runner picks +the backend when it prepares the session: for a Saiph repo it mounts the +node's `saiphd` socket and a Saiph token, not a git credential. One +session can hold repos on both backends; each tool call routes by the repo +it names. An agent on a Saiph repo has no `github.com` egress (ruling 4). +A repo moves to Saiph once: import, a freeze of the git backend for that +repo, then the switch ([git-bridge](git-bridge.md)). The tool names and arguments are +kept, so the prompt and skill change only where the review words differ. + +### Record index and work order + +| Record | Owns | Depends on | +| -- | -- | -- | +| [store](store.md) | Changeset format, blobs, metadata database and ref CAS, derived data, server-side commit, rebase and merge, the `saiph.v1` API, auth seam, tenancy, ACLs, scaling, gc, caps, deviations from Scion | — | +| [client](client.md) | The `saiph` CLI, `saiphd`, `saiph-cache`, the object protocol, sparse and virtual checkout, token claims and Compass issuance, the `vcs_*` tools and Runner pick on Saiph | store | +| [review](review.md) | Change, version, stack DAG, threads, approvals, attention, landing, web UI | store | +| [ci-cd](ci-cd.md) | Triggers from ref and version events, presubmit, postsubmit milestones, culprit finding, affected targets, engine, deploy | store, review | +| [merge-queue](merge-queue.md) | Batching, bisection, stacks as units, per-target lanes, auto-rebase, conflict fixer and ejection | store, review, ci-cd | +| [git-bridge](git-bridge.md) | Trunk export to GitHub, git history import, per-repo cutover | store | + +Work order, by binary: + +1. `saiph-server`: store S1–S8. Every record reads its entities. +2. `saiph`, `saiphd` and `saiph-cache` ([client](client.md) C1–C8), and + `saiph-bridge` ([git-bridge](git-bridge.md) B1–B7), in parallel. The + bridge registers the git-commit and git-tree derivers (store S9). +3. `saiph-review` ([review](review.md)), after client. +4. `saiph-ci` and `saiph-ci-step` ([ci-cd](ci-cd.md)), after review. +5. `saiph-queue` ([merge-queue](merge-queue.md)) last: it composes review + state, CI results, and the store's `Rebase` with resolutions and + chained `UpdateRef`. +6. Compass tasks run beside their binary: token issuance and the agent + backend ([client](client.md) C9–C10), the review proxy and UI + ([review](review.md) R12), and the cutover row + ([git-bridge](git-bridge.md) B8). + +### Decisions + +- **D1 — DL-580.** Two backends sit behind one agent `vcs_*` tool surface, + chosen per repo: the RIG-5558 git backend ships unchanged for repos on + GitHub, and Saiph serves Saiph repos through the `saiph` CLI, where + agents have no GitHub egress and only trunk is mirrored to GitHub as + derived git commits; the tools, the Runner backend pick and the web UI + hooks stay in Compass, and Saiph stays in its own repo. + +## Plan + +### Global Constraints + +- **Public repo.** No operator names, private product detail, or links to + private records. Tracker ids stay plain text. +- **Names.** Saiph; proto package `saiph.v1`; CLI `saiph`; node daemon + `saiphd`; regional cache `saiph-cache`; server `saiph-server`; services + `saiph-review`, `saiph-queue`, `saiph-ci` and `saiph-bridge`; schemas + `saiph`, `saiph_review`, `saiph_queue`, `saiph_ci` and `saiph_bridge`. + No other use of the name is mentioned. +- **Repo boundary.** Saiph code lives in `RigelBuild/saiph` and imports + nothing from Compass. Compass reaches Saiph only through `saiph.v1` and + the `saiph` CLI, with tokens it issues. +- **Scion first.** Core choices follow the Scion post; gaps follow + `facebook/sapling`; each departure is listed in the record that makes + it. +- **Sapling and EdenFS: designs and ideas only, never code.** No file, + function, test or data table from `facebook/sapling` is copied, + translated or pasted. Running an unmodified EdenFS is not in scope. +- **RIG-5558 is frozen.** No change to its proto fields, its mirror, its + migrations or its decisions. +- **Caps.** Every usage-billed component states its cap per layer, a + budget alert that pages Matt, and spend-cap rows, in the record that + enables it. + +### O1 — Sibling records + +Done: [client](client.md), [review](review.md), [ci-cd](ci-cd.md), +[merge-queue](merge-queue.md) and [git-bridge](git-bridge.md), written +against [store](store.md#api-the-saiphv1-connect-package). Their store +additions are folded into store. A later addition to `saiph.v1` +supersedes store. + +Interfaces: `saiph.v1` services `RepoService`, `ObjectService`, +`HistoryService`, `MutationService`, `RefService`, and `auth.Principal`, +as frozen in store. + +### O2 — Backend selection in Compass + +A per-repo backend field on the Compass side, read by the agent tools and +the Runner. Designed in client and git-bridge; this record fixes only the +rule (D1). + +Interfaces: + +```proto +// compass.v1, in Compass (derived, never stored; see client.md and git-bridge.md) +enum RepoBackend { REPO_BACKEND_UNSPECIFIED = 0; REPO_BACKEND_GIT = 1; REPO_BACKEND_SAIPH = 2; } +``` + +## Tasks + +- [x] O1 — client, review, ci-cd, merge-queue and git-bridge records +- [ ] O2 — per-repo backend selection behind the `vcs_*` tools and in the Runner +- [ ] Store S1–S11 ([store](store.md#tasks)) +- [ ] Client C1–C11, review R1–R12, ci-cd CI1–CI11, merge-queue Q1–Q11, git-bridge B1–B9 (each record's Tasks) diff --git a/docs/designs/saiph/ci-cd.md b/docs/designs/saiph/ci-cd.md new file mode 100644 index 0000000..0ca0ce5 --- /dev/null +++ b/docs/designs/saiph/ci-cd.md @@ -0,0 +1,1048 @@ +# Saiph: CI/CD on Kubernetes + +Tracker: RIG-5665 + +> Freezes on merge; later changes supersede by citation, never rewrite +> (convention: the `docs/designs/agent/compass-agent-config-delivery/design.md` +> header). Unless a path says otherwise, every code path is in the +> `RigelBuild/saiph` repo, which does not exist yet. A Compass path says +> "in Compass". + +Ledger-impact: D1–D10 are filed as DL-602..DL-611, and no filed decision +is superseded. Decisions about Compass's GitHub Actions CI stay whole: +they govern repos on the git backend +([overview](README.md#rulings), ruling 5). + +**Named decisions this record introduces:** D1–D10 in [Decisions](#decisions). + +Length: Matt lifted the length cap for this set (ruling 6 in the +[overview](README.md#rulings)). The prose sits mostly in Approach. + +This record designs Saiph's CI and deploy service, `saiph-ci`. It builds +on the entities and RPCs of [store](store.md) and on the review contract +in [review](review.md), and it gives [merge-queue](merge-queue.md) the calls the queue runs on. +The [overview](README.md) holds the glossary and the component map. + +## Open Questions + +None depends on another unless it says so. + +### OQ-1 — One CI config file, or one per directory? + +Twigg lets a repo "commit a `CI.json` and/or `CD.json` file anywhere in +your repository". Each file owns its folder, and a change below that +folder triggers its jobs ([Twigg CI/CD](https://twigg.vc/docs/v/2/ci-cd/)). + +- (a) Per-directory `CI.json` and `CD.json` files, as Twigg. +- (b) One root file, `.saiph/ci.yaml`. Directory scope comes from the + build graph: a moon project is a directory with its own `moon.yml`. A + repo with no build graph lists directory jobs in the same file. + +**Recommendation: (b).** Saiph reads targets from the build graph, so the +file holds only policy: classes, timeouts, resources, milestones +and deploy stages. One file is one place to review, and one `.saiphacl` on +`.saiph/` protects it ([store](store.md#per-directory-acls)). Per-directory +files let anyone with write in a directory change the CI policy of that +directory. Either way, CI reads the config at the run's base, never from +the change under test (D4). + +### OQ-2 — How does moon see a Saiph checkout? (load-bearing for the cache) + +moon reads repo state only through git. Its check is +`self.worktree.git_dir.exists() && find_command_on_path("git").is_some()` +(`is_enabled` in +[`crates/vcs/src/git/git_client.rs`](https://github.com/moonrepo/moon/blob/v2.6.2/crates/vcs/src/git/git_client.rs)), +and the task runner turns its cache off without it: "If the VCS root does +not exist (like in a Docker container), we should avoid failing and simply +disable caching" (`is_cache_enabled` in +[`crates/task-runner/src/task_runner.rs`](https://github.com/moonrepo/moon/blob/v2.6.2/crates/task-runner/src/task_runner.rs)). +A Saiph job has no `.git` and no `git`. So moon runs, but it neither reads +nor writes its local or remote cache, and it rebuilds its project graph on +every run ("No VCS to hash with, so abort caching", `new_with_cache` in +`crates/workspace/src/workspace_builder.rs` in moon). Affected detection +is not hit: moon takes changed files from stdin before it asks git +([Targets and affected detection](#targets-and-affected-detection)). + +- (a) Write a Saiph provider for moon's proposed pluggable VCS and + contribute it upstream + ([moon#2620](https://github.com/moonrepo/moon/issues/2620), open, + "[feature] Pluggable VCS support"). Until it ships, Saiph jobs run moon + without its cache. +- (b) Put a `git` shim and an empty `.git` in the job image. The shim + answers the subcommands moon runs (`rev-parse`, `branch --show-current`, + `ls-files`, `remote get-url`, `--version`). +- (c) Fork moon. + +**Recommendation: (a).** (b) puts a git-shaped lie in every job. The moon +issue itself warns that treating another VCS as git "can produce an empty +branch, a stale revision, or changed files calculated from a different +repository state". (c) is a permanent maintenance cost. The cost of (a) is +small for the first user: Compass's `.moon/workspace.yml` (in Compass) has +no `remote:` block, so its CI shares no remote cache today. Bazel repos +use the cache from the start, because Bazel needs no VCS to hash. + +### OQ-3 — May the deploy controller move `release/*` refs? (security; store addition) + +Store lets only a `land` subject move a protected ref, and a repo role +covers every ref. `REPO_ROLE_LAND` for CD would let CD move trunk too. + +- (a) Add a ref prefix to grants (`SetGrantRequest.ref_prefix`, see + [Store additions](#store-additions)). CD holds `land` on `release/` only. +- (b) CD never moves refs. A person with `land` cuts releases and lands + picks with the `saiph` CLI, and CD only runs stages. + +**Recommendation: (a).** Rapid automates the cut and the approved picks; +(b) keeps a manual step in every release. (a) is one field and one check +in the store. + +### OQ-4 — Does CI revert a confirmed culprit? + +- (a) No. CI comments on the culprit change and adds its owner to the + attention set. The breakage keeps the target red at trunk until a later + trunk changeset passes it (D6). +- (b) Yes. CI opens a revert change and sends it through the merge queue + when the culprit's posterior is at least 0.95. + +**Recommendation: (a)** for the first release. Revert policy is product +behavior that needs breakage data from the first user. (b) can be added +later on the same `Breakage` rows. + +### OQ-5 — Compass jobs that Kubernetes cannot run + +Compass's CI selects a macOS suite (`darwin_affected` in `outputLines`, +`tools/ci-matrix/index.ts` in Compass). It needs a macOS host. + +- (a) Keep it as a GitHub Actions job on the trunk-only mirror. It runs + after landing and reports on GitHub only. +- (b) Add a pull-based `saiph-ci-agent` for hosts outside Kubernetes. It + takes shards from the controller as job pods do. + +**Recommendation: (a)** at Compass's cutover. Presubmit coverage for that +suite is lost, and the cutover issue names it. Build (b) when a customer +needs it. Depends on the cutover plan in [git-bridge](git-bridge.md). + +### OQ-6 — Is a flaky target quarantined? (load-bearing for the merge queue) + +A result never changes, and CI never retries (D5). When two results for +one target disagree on the same inputs, CI opens a flake breakage +([Flakes and quarantine](#flakes-and-quarantine)). The question is what +the flaky target does to gating while that breakage is open. + +- (a) Quarantine. The target still runs and reports, but it is not + required: `TargetResult.quarantined` is true, review gets + `required=false`, and the merge queue drops it from a batch's required + set until the breakage closes. +- (b) No quarantine. The target stays required. Every lane that touches + it ejects until the owner fixes the test. + +**Recommendation: (a).** A flaky test is a bug, and the breakage routes it +to its owner. (b) stops every unrelated change that touches the target. +(a) does not hide the bug: the breakage stays open and listed until a fix +lands. Recorded as D7. + +## Problem / Intent + +Saiph replaces the forge, so it must also replace the forge's CI. Matt +ruled that CI/CD is designed alongside Saiph, on Kubernetes (ruling 7), +that review stacks are DAGs which CI uses (ruling 3), and that the merge +queue runs per-target lanes (ruling 9). This record designs that service: +triggers from the store's ref log and from review's version events, +presubmit on versions, postsubmit on trunk in milestones with culprit +finding, affected targets from the store's `Diff`, the engine, job source +with no git, result storage, the CI side of the merge queue, deploy +stages on release refs, a remote build cache, and compute caps. + +Scion takes the position this record builds on: "we've invested in +post-merge and pre-deploy automated testing", and "a CI job that needs +the diff between a commit and its merge base … can ask the server for it" +([Redesigning source control](https://www.anthropic.com/engineering/redesigning-source-control)). +The post does not say how its CI runs. Google's published TAP, culprit +finding and Rapid fill that gap, as Mononoke fills the store's. + +## Approach + +### Shape + +`saiph-ci` is one Go service, `cmd/saiph-ci`, with four parts: the +trigger consumers, the planner, the controller and the deploy +controller. It reads the store and review only through `saiph.v1`. It +keeps its own rows in the schema `saiph_ci`, under the store's tenant GUC +`saiph.tenant_id` and the same row-level security. It runs work as +Kubernetes Jobs whose source comes from `saiphd` through CSI. The store +answers every history question; this service owns only orchestration. + +```mermaid +flowchart LR + S[saiph-server] -->|WatchRefs| T[triggers] + R[review] -->|WatchEvents| T + T --> C[controller] + Q[merge queue] -->|AffectedTargets, RequestRun| C + C --> P[planner] + P -->|Diff, ReadTree, ReadFile| S + P -->|paths on stdin| M[moon query tasks] + C -->|one Job per shard| K[Kubernetes] + K -->|CSI volume| D[saiphd] + K -->|FinishShard, UploadLog| C + C -->|ReportChecks, Publish| R + C --> DB[(saiph_ci)] +``` + +### Triggers + +- **Trunk and release refs.** One `WatchRefs` stream per repo, resumed at + the stored `after_log_seq`. Store task S8 tests that "a stream resumed + at `after_log_seq` misses nothing" ([store](store.md#tasks)). The + consumer keeps entries for the trunk ref and `release/*`. Each entry + carries the landed `chain` and its `trunk_seqs`. +- **Versions.** One `ReviewService.WatchEvents` stream per repo, resumed + at the stored `after_seq`, for `VERSION_UPLOADED`, `ABANDONED` and + `LANDED`. On `OUT_OF_RANGE` the consumer rescans with `ListChanges` and + resumes at its `event_seq`. +- **Exactly once.** The consumer writes the new cursor and the runs it + creates in one `saiph_ci` transaction. A restart replays nothing twice + and skips nothing. One replica holds each repo's stream under a lease, + in the store's `Lease` pattern. +- **Reconnects.** Resuming a stream at its cursor after a server restart + is the stream protocol, not a retry. Each reconnect is counted, and the + trigger-lag alert fires when a cursor falls five minutes behind. + +### Targets and affected detection + +A target is a build-graph task id; for moon it is `project:task`. The +config puts each target in a class. `gate` targets run on presubmit and +in the merge queue; `post` targets run only on trunk. This is the split +Google measured: presubmit runs small and medium tests, and postsubmit +runs the large and enormous ones +([Who Broke the Build?](https://research.google.com/pubs/archive/45794.pdf)). + +`AffectedTargets(base, head)` has five steps: + +1. **Paths.** `Diff(from=base, to=head)` with no hunks, paged. It compares + two root trees and needs no ancestry, so a speculative merge-queue base + works. `Diff` leaves out paths the caller cannot read, so CI's service + subject needs read on the whole repo. +2. **Graph inputs.** The config lists the globs the build graph reads: + `.moon/**`, every `moon.yml`, toolchain manifests and lockfiles, and + `.saiph/ci.yaml`. `graph_changed` is true when a diff path matches + one. `graph_id` is the SHA-256 of the sorted (path, blob id) list of + graph-input files at head. The planner keeps that manifest per + changeset in `saiph_ci.graph_manifests`. It derives it from the base's + manifest plus the diff, or from one `ReadTree` walk the first time. +3. **Skeleton.** The planner writes the graph-input files of a `graph_id` + into a local directory with `ReadFile`, once, and keeps the directory + in an LRU bounded by `--skeleton-bytes`. moon builds its project and + task graph from these files alone, and matching changed paths against + input globs needs no other file. A toolchain plugin that reads a file + outside the list fails the plan loudly; the fix is a glob in the + config. +4. **moon.** The planner runs + `moon query tasks --affected --downstream deep` in the skeleton, with + the diff paths on stdin, one per line. moon reads changed files from + stdin when stdin is not a terminal + (`crates/app/src/queries/changed_files.rs` in moon), so it never asks + git. Compass selects its CI targets the same way today + (`taskTargetsForPaths` in `tools/ci-matrix/index.ts`, in Compass). +5. **Barrier.** `all=true` when moon cannot build the graph, when the + diff has more than `max_affected_paths` paths (default 20,000), or + when it touches workspace or toolchain config (`.moon/workspace.yml`, + `.moon/toolchain*.yml`, `.prototools`). `all` overrides `graph_id` and + `graph_changed`. + +`AffectedTargets(X, X)` is an empty diff and returns at once. A run +filters the result by the classes its kind runs. A target that affected +detection drops is left out of the required set; it is never reported as +skipped. A repo with no build graph lists targets with path globs in the +config, and the planner matches them directly. Both are one `Adapter` +interface. A later Bazel adapter uses the same skeleton, with `BUILD`, +`*.bzl` and `MODULE.bazel` as graph inputs. + +The planner also gives each target an **input key**: the SHA-256 of +`graph_id`, the target, the job image digest, the store tree id of each +directory its inputs cover (the blob id for a single file), and the input +keys of the tasks it depends on. A tree id changes when any file below it +changes, so equal keys mean equal inputs. A coarse key can miss a flake; +it cannot invent one. The key never skips a run. + +The config is read at the run's base, never from the change under test +(D4), so a change cannot weaken the policy that checks it. When the diff +touches `.saiph/ci.yaml`, the planner adds the built-in gate target +`saiph:ci-config`, which validates the head's file. + +```yaml +# .saiph/ci.yaml (read at the run's base) +graph_inputs: [".moon/**", "**/moon.yml", "**/package.json", "bun.lock", "go.mod", "go.sum"] +classes: # first match wins; an unmatched target is gate + - { targets: "e2e:*", class: post } +timeouts: { default: 30m, "e2e:*": 2h } +resources: { default: { cpu: "2", memory: 4Gi }, "*:build": { cpu: "8", memory: 16Gi } } +checkout: virtual # or materialized +milestone: { minutes: 45, changesets: 50 } +owners: { "web:*": "account:acct_web" } +deploy: + cut: { cron: "0 9 * * MON" } + stages: + - { name: staging, target: "deploy:staging", service_account: deploy-staging } + - { name: prod, target: "deploy:prod", service_account: deploy-prod, approval: true } +``` + +### Presubmit on versions + +On `VERSION_UPLOADED`, CI requests a `RUN_KIND_PRESUBMIT` run of the +version's changeset against the event's `base`, the trunk merge base that +review computes. Review stacks are DAGs (ruling 3). A version's changeset +holds every change it depends on, so its tree is the stack up to that +change, and its affected set from the trunk base covers its parents' +changes too. A green child means the stack below it passes on that tree. + +CI reports through `ReportChecks`: one `CheckRun` per target, named +`ci/`, `required` unless the target is quarantined, with +`details_url` pointing at the log. It sends `complete=true` once the plan +is known, because no check is added after that. PASSED maps to `SUCCESS`; +FAILED and INFRA map to `FAILURE`; a cancelled target maps to +`CANCELLED`. A red target that is `broken_at_base` says so in its +summary: the author did not break it, and the change lands after trunk is +fixed and the change is rebased. + +A new version cancels the targets of older versions that have not +started. `ABANDONED` cancels the change's open runs. A version whose +changeset already has results reuses them. + +Presubmit runs code that nobody has reviewed yet: agents' changes and, +later, imported pull requests ([git-bridge](git-bridge.md) OQ-2). Its Jobs run under +the Restricted Pod Security profile with `hostUsers: false`. They mount +no Secret, hold only a read token for the repo and read access to the +cache, and a NetworkPolicy lets them reach only `saiph-ci`, the cache and +`saiphd`. + +### Postsubmit milestones + +TAP does not test every commit. It tests "a number of consecutive code +commits together", and a milestone is "typically cut every 45 minutes" +([TAP](https://research.google.com/pubs/archive/45861.pdf)). Saiph does +the same on trunk. A milestone is cut at the trunk head when +`milestone.minutes` (default 45) have passed or `milestone.changesets` +(default 50) have landed since the last one. It is a +`RUN_KIND_POSTSUBMIT` run of every affected `gate` and `post` target, with +the previous milestone as base. + +A target that is red at a milestone opens a breakage when its last trunk +result was green. The breakage holds the range from the last green +`trunk_seq` to the first red one, and culprit finding runs over it. A +target that stays red adds no breakage. A breakage closes when the target +passes on a later trunk changeset (D6). Until then, every run whose base +is at or after the first red `trunk_seq` marks that target +`broken_at_base`. + +### Culprit finding + +The candidates are the trunk changesets in the range whose own diff +affects the target, by `AffectedTargets(parent, candidate)`. No other +changeset can change its result. The search is Google's flake-aware +culprit finding, which "is based on Bayesian inference and noisy binary +search, utilizing prior information" +([paper](https://research.google/pubs/flake-aware-culprit-finding/)). The +prior is uniform over the candidates, and the noise is the target's flake +rate from `GetTargetStats`. Each step probes the candidate that splits +the posterior mass most evenly. A probe is a `RUN_KIND_CULPRIT` run of +every target broken in that range. + +No changeset runs a target twice. A probe at a changeset that already has +a result, such as a merge-queue batch head, uses that result for free. +The search stops when one candidate's posterior reaches 0.95, when every +candidate has a result, or after `max_culprit_probes` probes (default +16). A confirmed culprit gets one change-level comment through +`ReviewService.Publish`, with `RESOLUTION_UNRESOLVED` and its owner in +`attention_add`. CI does not revert in the first release (OQ-4). An +unconfirmed search lists its best candidates and their posteriors on the +breakage. + +### Flakes and quarantine + +Each (repo, changeset, target) has one result, and it never changes. CI +never retries a target, and a Job never retries its pod. A flaky test is +a bug, so CI records the evidence and routes it to an owner. + +- **Detection.** When a new result disagrees with a stored result that + has the same input key, CI opens a `BREAKAGE_KIND_FLAKE` breakage for + the target, or adds to the open one. Neither result changes. A + merge-queue batch that fails a target, followed by a successor batch + that passes it with the same key, is this case. +- **Quarantine** (OQ-6). While the flake breakage is open, each new + result of that target has `quarantined=true`. The target still runs and + reports, so its results keep feeding the statistics. Review gets + `required=false`, and the merge queue drops it from a batch's required + set. +- **Close.** The breakage closes when the target's input key changes on + trunk and its next 20 results all pass, or when a `land` subject closes + it with `CloseBreakage` after a fix lands. +- **Routing.** The config's `owners` map names the account that owns a + target. `ListBreakages` lists each open flake with that owner. + +`GetTargetStats.flaky` counts results that are part of a disagreement. + +**INFRA** is a pod that ends without its step wrapper's report: an +eviction, a lost node, a failed image pull. Each unfinished target of the +shard gets INFRA on the run. INFRA is red and fires the CI infra alert, +which CI's lane owns. It is stored on the run, not as the changeset's +result, so a person may request a new run after the cause is fixed. +Nothing re-requests it automatically. A target that hits its own timeout +or memory limit is FAILED, because those limits are part of the target's +contract. + +### Engine: a first-party controller and Jobs + +The controller splits a run into shards and runs each shard as one +`batch/v1` Job. A shard holds about `shard_seconds` (default 600) of +targets by their p50 duration. Pending shards wait in `saiph_ci`, ordered +by priority and then age. The controller creates a Job only while the +repo is under `max_running_shards`. Pods use a PriorityClass with +`preemptionPolicy: Never`, because a preempted pod would be an INFRA +result. [Alternatives considered](#alternatives-considered) compares the +engines this replaces. + +```yaml +apiVersion: batch/v1 +kind: Job +metadata: { name: "ci--", namespace: "saiph-ci-" } +spec: + backoffLimit: 0 # never retry a pod (D5) + activeDeadlineSeconds: 1800 # the sum of the shard's target timeouts + ttlSecondsAfterFinished: 3600 + template: + spec: + restartPolicy: Never + hostUsers: false + priorityClassName: saiph-ci-presubmit + serviceAccountName: saiph-ci-pre + tolerations: + - { key: saiph.rigel.build/ci, operator: Equal, value: "true", effect: NoSchedule } + securityContext: { runAsNonRoot: true, seccompProfile: { type: RuntimeDefault } } + containers: + - name: step + image: "" + command: [saiph-ci-step, --run, "", --shard, ""] + workingDir: /src + resources: { requests: { cpu: "2", memory: 4Gi }, limits: { cpu: "2", memory: 4Gi } } + volumeMounts: [{ name: src, mountPath: /src }] + volumes: + - name: src + csi: + driver: saiph.rigel.build + readOnly: false + volumeAttributes: { repo: owner/name, changeset: "<64 hex>", sparse: "", mode: virtual } +``` + +The step wrapper, `saiph-ci-step`, runs `moon run` with the shard's +targets, as Compass's GitHub Actions jobs do today. It reads moon's run +report in `.moon/cache/`, sends one outcome per target with +`StepService.FinishShard`, and streams the log with `UploadLog`. Logs are +stored zstd-compressed under +`//ci//.log.zst`. A shard can report only +its own targets: `FinishShard` checks a random per-shard token that the +controller puts in the Job's environment and stores hashed. + +### Job source: a CSI volume from `saiphd` + +Jobs have no git. Their source is a CSI inline ephemeral volume served by +`saiphd`, as agreed in [client](client.md#ci-source-the-csi-driver). Pod +Security forbids hostPath at Baseline and allows `csi` at Restricted. The +volume is a copy-on-write overlay over the changeset, deleted when the +pod ends. The mode defaults to `virtual` with an empty sparse set, so the +FUSE root fetches only what the job reads. A repo whose tools walk the +whole tree sets `checkout: materialized`. The `CSIDriver` object's +`tokenRequests` gives the driver a token with audience +`saiph.rigel.build`. The pod's service account maps to +`service:k8s//`, which CI grants +`REPO_ROLE_READ` when a repo enables CI. A job that needs history uses +the `saiph` CLI in API mode. + +### Results + +```sql +-- internal/ci/migrations/0001_saiph_ci.sql; every table has ENABLE and +-- FORCE ROW LEVEL SECURITY and a tenant_isolation policy on saiph.tenant_id. +CREATE SCHEMA saiph_ci; +CREATE TABLE saiph_ci.runs ( + tenant_id TEXT NOT NULL DEFAULT current_setting('saiph.tenant_id', TRUE), + repo_id BIGINT NOT NULL, + run_id TEXT NOT NULL, + kind SMALLINT NOT NULL, + changeset BYTEA NOT NULL CHECK (length(changeset) = 32), + base BYTEA, + change_id BYTEA, -- presubmit: the review change + version INT, + idempotency_key TEXT NOT NULL, + priority SMALLINT NOT NULL, + state SMALLINT NOT NULL, + created_unix_ms BIGINT NOT NULL, + PRIMARY KEY (tenant_id, run_id), + UNIQUE (tenant_id, repo_id, idempotency_key) +); +CREATE TABLE saiph_ci.run_targets ( -- per-run status; INFRA and CANCELLED live only here + tenant_id TEXT NOT NULL DEFAULT current_setting('saiph.tenant_id', TRUE), + run_id TEXT NOT NULL, + target TEXT NOT NULL, + shard INT, + status SMALLINT NOT NULL, + PRIMARY KEY (tenant_id, run_id, target) +); +CREATE TABLE saiph_ci.results ( -- one immutable outcome per changeset and target + tenant_id TEXT NOT NULL DEFAULT current_setting('saiph.tenant_id', TRUE), + repo_id BIGINT NOT NULL, + changeset BYTEA NOT NULL CHECK (length(changeset) = 32), + target TEXT NOT NULL, + passed BOOLEAN NOT NULL, + input_key BYTEA NOT NULL CHECK (length(input_key) = 32), + run_id TEXT NOT NULL, + duration_ms BIGINT NOT NULL, + trunk_seq BIGINT, -- set for trunk changesets + quarantined BOOLEAN NOT NULL, + finished_unix_ms BIGINT NOT NULL, + PRIMARY KEY (tenant_id, repo_id, changeset, target) +); +CREATE INDEX results_by_input ON saiph_ci.results (tenant_id, repo_id, target, input_key); +CREATE TABLE saiph_ci.breakages ( + tenant_id TEXT NOT NULL DEFAULT current_setting('saiph.tenant_id', TRUE), + repo_id BIGINT NOT NULL, + id TEXT NOT NULL, + target TEXT NOT NULL, + kind SMALLINT NOT NULL, -- 1 break, 2 flake + last_green_trunk_seq BIGINT, + first_red_trunk_seq BIGINT, + culprit BYTEA, + posterior DOUBLE PRECISION, + owner TEXT, + opened_unix_ms BIGINT NOT NULL, + closed_unix_ms BIGINT, + closed_by TEXT, + PRIMARY KEY (tenant_id, repo_id, id) +); +CREATE TABLE saiph_ci.cursors ( -- one per repo and stream ("refs", "review") + tenant_id TEXT NOT NULL DEFAULT current_setting('saiph.tenant_id', TRUE), + repo_id BIGINT NOT NULL, + stream TEXT NOT NULL, + seq BIGINT NOT NULL, + lease_token TEXT, + lease_until_unix_ms BIGINT, + PRIMARY KEY (tenant_id, repo_id, stream) +); +-- Also: graph_manifests (changeset, graph_id, zstd CBOR manifest), +-- repo_settings (caps), releases and release_items. +``` + +Results are keyed by changeset, because a changeset id names the exact +tree and history. Review maps a version to its changeset, and `runs` +records `change_id` and `version`, so the UI lists results per version. +Trunk results also carry `trunk_seq`. A result is inserted with +`ON CONFLICT DO NOTHING`, so the first outcome stands. CI holds no pins. +A run reads changesets that a ref, a review version pin or the queue's +`merge-queue:` pin keeps, or that are younger than the repo's +`retention_days`. + +### API: `proto/saiph/v1/ci.proto` + +CI calls the store and review as `service:saiph-ci`. When a repo enables +CI, that subject gets `REPO_ROLE_WRITE` on the repo, for `ReportChecks`, +`Publish` and whole-repo reads. Deploy uses the same subject, with a +second grant of `REPO_ROLE_LAND` on the ref prefix `release/` (OQ-3). + +CI forwards the caller's token to the store's `GetRepo` and reads +`caller_role`. Reads need READ. `RequestRun` needs WRITE, and +`RUN_KIND_MERGE_QUEUE` and `RUN_KIND_CULPRIT` need LAND. `CloseBreakage` +and `Approve` need LAND, and `Approve` refuses a service subject. +`StepService` takes no issuer token; its `shard_token` authenticates it. + +```proto +syntax = "proto3"; +package saiph.v1; + +service TargetService { + rpc AffectedTargets(AffectedTargetsRequest) returns (AffectedTargetsResponse) { option idempotency_level = NO_SIDE_EFFECTS; } + rpc GetResults(GetResultsRequest) returns (GetResultsResponse) { option idempotency_level = NO_SIDE_EFFECTS; } + rpc ListBreakages(ListBreakagesRequest) returns (ListBreakagesResponse) { option idempotency_level = NO_SIDE_EFFECTS; } + rpc CloseBreakage(CloseBreakageRequest) returns (CloseBreakageResponse); + rpc GetTargetStats(GetTargetStatsRequest) returns (GetTargetStatsResponse) { option idempotency_level = NO_SIDE_EFFECTS; } +} +message AffectedTargetsRequest { string repo = 1; bytes base = 2; bytes head = 3; } +// all = the barrier; it overrides graph_id and graph_changed. A false graph_changed always means unchanged. +message AffectedTargetsResponse { repeated string targets = 1; bool all = 2; bytes graph_id = 3; bool graph_changed = 4; } +message GetResultsRequest { string repo = 1; bytes changeset = 2; repeated string targets = 3; } +message GetResultsResponse { repeated TargetResult results = 1; } +enum BreakageKind { BREAKAGE_KIND_UNSPECIFIED = 0; BREAKAGE_KIND_BREAK = 1; BREAKAGE_KIND_FLAKE = 2; } +message Breakage { + string id = 1; string target = 2; BreakageKind kind = 3; + uint64 last_green_trunk_seq = 4; uint64 first_red_trunk_seq = 5; + bytes culprit = 6; double posterior = 7; string owner = 8; + bool open = 9; int64 opened_unix_ms = 10; int64 closed_unix_ms = 11; +} +message ListBreakagesRequest { string repo = 1; bool include_closed = 2; uint32 page_size = 3; string page_token = 4; } +message ListBreakagesResponse { repeated Breakage breakages = 1; string next_page_token = 2; } +message CloseBreakageRequest { string repo = 1; string id = 2; string reason = 3; } +message CloseBreakageResponse { Breakage breakage = 1; } +// Empty targets = all; window_days defaults to 7, at most 30. Flake rate = flaky / runs. +message GetTargetStatsRequest { string repo = 1; repeated string targets = 2; uint32 window_days = 3; } +message TargetStats { string target = 1; uint64 runs = 2; uint64 passed = 3; uint64 failed = 4; uint64 flaky = 5; uint64 infra = 6; int64 p50_duration_ms = 7; } +message GetTargetStatsResponse { repeated TargetStats stats = 1; } + +service RunService { + rpc RequestRun(RequestRunRequest) returns (RequestRunResponse); + rpc GetRun(GetRunRequest) returns (GetRunResponse) { option idempotency_level = NO_SIDE_EFFECTS; } + rpc WatchRun(WatchRunRequest) returns (stream WatchRunResponse); + rpc CancelRun(CancelRunRequest) returns (CancelRunResponse); +} +enum RunKind { RUN_KIND_UNSPECIFIED = 0; RUN_KIND_PRESUBMIT = 1; RUN_KIND_POSTSUBMIT = 2; RUN_KIND_MERGE_QUEUE = 3; RUN_KIND_CULPRIT = 4; RUN_KIND_DEPLOY = 5; } +enum RunState { RUN_STATE_UNSPECIFIED = 0; RUN_STATE_QUEUED = 1; RUN_STATE_RUNNING = 2; RUN_STATE_DONE = 3; RUN_STATE_CANCELLED = 4; } +// No FLAKY status: a flake is a breakage (D7). +enum TargetStatus { + TARGET_STATUS_UNSPECIFIED = 0; TARGET_STATUS_PENDING = 1; TARGET_STATUS_RUNNING = 2; + TARGET_STATUS_PASSED = 3; TARGET_STATUS_FAILED = 4; TARGET_STATUS_INFRA = 5; TARGET_STATUS_CANCELLED = 6; +} +message TargetResult { + string target = 1; TargetStatus status = 2; + reserved 3; // attempts: CI never retries + bool reused = 4; bool broken_at_base = 5; string details_url = 6; + bool quarantined = 7; bytes input_key = 8; int64 duration_ms = 9; +} +// Empty targets = affected(base, changeset) for the kind's classes. priority 0 = the kind's default. +message RequestRunRequest { + string repo = 1; bytes changeset = 2; bytes base = 3; RunKind kind = 4; + repeated string targets = 5; string idempotency_key = 6; uint32 priority = 7; + bytes change_id = 8; uint32 version = 9; +} +message RequestRunResponse { string run_id = 1; bool reused = 2; } +message Run { + string run_id = 1; string repo = 2; RunKind kind = 3; RunState state = 4; + bytes changeset = 5; bytes base = 6; bool all = 7; repeated TargetResult results = 8; + int64 created_unix_ms = 9; +} +message GetRunRequest { string run_id = 1; } +message GetRunResponse { Run run = 1; } +message WatchRunRequest { string run_id = 1; } +// One event per target state change; the last event carries done. +message RunEvent { oneof event { TargetResult result = 1; Run done = 2; } } +message WatchRunResponse { RunEvent event = 1; } +message CancelRunRequest { string run_id = 1; string reason = 2; } +message CancelRunResponse { Run run = 1; } + +// Called only by saiph-ci-step inside a Job. +service StepService { + rpc FinishShard(FinishShardRequest) returns (FinishShardResponse); + rpc UploadLog(stream UploadLogRequest) returns (UploadLogResponse); +} +message StepOutcome { string target = 1; bool passed = 2; int64 duration_ms = 3; string summary = 4; } +message FinishShardRequest { string run_id = 1; uint32 shard = 2; string shard_token = 3; repeated StepOutcome outcomes = 4; } +message FinishShardResponse {} +message UploadLogRequest { string run_id = 1; uint32 shard = 2; string shard_token = 3; bytes chunk = 4; } +message UploadLogResponse { uint64 bytes = 1; } + +service DeployService { + rpc CutRelease(CutReleaseRequest) returns (CutReleaseResponse); + rpc ProposePick(ProposePickRequest) returns (ProposePickResponse); + rpc Approve(ApproveRequest) returns (ApproveResponse); + rpc GetRelease(GetReleaseRequest) returns (GetReleaseResponse) { option idempotency_level = NO_SIDE_EFFECTS; } +} +enum StageState { STAGE_STATE_UNSPECIFIED = 0; STAGE_STATE_WAITING = 1; STAGE_STATE_RUNNING = 2; STAGE_STATE_DONE = 3; STAGE_STATE_FAILED = 4; } +message Stage { string name = 1; StageState state = 2; string run_id = 3; repeated string approved_by = 4; } +// Item 0 is the cut; later items are picks. +message ReleaseItem { string id = 1; bytes trunk_changeset = 2; bytes release_changeset = 3; string run_id = 4; repeated string approved_by = 5; bool landed = 6; } +message Release { string name = 1; bytes head = 2; repeated ReleaseItem items = 3; repeated Stage stages = 4; } +// Empty trunk_changeset = the newest milestone where every target passed. +message CutReleaseRequest { string repo = 1; string name = 2; bytes trunk_changeset = 3; } +message CutReleaseResponse { Release release = 1; } +message ProposePickRequest { string repo = 1; string release = 2; bytes trunk_changeset = 3; } +message ProposePickResponse { ReleaseItem item = 1; } +// Exactly one of item_id and stage. +message ApproveRequest { string repo = 1; string release = 2; string item_id = 3; string stage = 4; } +message ApproveResponse { Release release = 1; } +message GetReleaseRequest { string repo = 1; string name = 2; } +message GetReleaseResponse { Release release = 1; } +``` + +`RequestRun` with the same `idempotency_key` returns the same run. A new +key gets a new run, but a target that already has a result at that +changeset returns it with `reused=true`. `priority` is 0–100, and higher +runs sooner. 0 means the kind's default: presubmit 30, postsubmit 20, +culprit 40, merge queue 50. The queue uses 60 for bisection probes and +land retests. + +### What the merge queue gets + +- `AffectedTargets` as a snapshot diff, cheap on equal inputs, with `all` + as the barrier and a `graph_changed` that is never falsely false. +- `RequestRun` with `RUN_KIND_MERGE_QUEUE`, which runs `gate` targets + only. +- Results that never change. FAILED, INFRA and `broken_at_base` are red; + a `quarantined` target is not required. +- Successor evidence instead of reruns: a later batch's result on the + same input key is the second result that shows a flake. +- `GetTargetStats` and `ListBreakages`. CI holds no pins. + +### Deploy: releases, stages and approvals + +Google's Rapid branches a release "from the mainline at a specific +revision" and cherry-picks fixes into it +([Release engineering](https://sre.google/sre-book/release-engineering/)). +In Saiph a release is the protected ref `release/`. It moves forward +only, by `land` only, in a single-row CAS. + +- **Cut.** On the config's `deploy.cut` schedule or a `CutRelease` call, + CD takes the newest milestone where every `gate` and `post` target + passed and creates `release/` there with `UpdateRef`. CD holds + `land` on `release/` only (OQ-3). +- **Picks.** `ProposePick` names a trunk changeset. CD rebases it onto + the release head with the store's `Rebase`, runs its affected targets + as a `RUN_KIND_DEPLOY` run, and waits for approval. A conflict refuses + the pick and names the paths. An approved, green pick moves the release + ref with one `UpdateRef`. +- **Stages.** Each stage runs its deploy target as a Job in the stage's + own namespace, under a service account the operator grants. Deploy + credentials live only there, never in a presubmit namespace. A stage + with `approval: true` waits for `Approve` from a human `land` account. + A failed stage stops the release; it never rolls back on its own. +- **Audit.** Every cut and pick is a ref-log row with its subject and + reason. + +### Remote build cache + +The cache speaks REAPI with the `SHA256` digest function. It is Buildbarn +`bb-storage` (Apache-2.0), one Deployment per region from the saiph +repo's `deploy/`. + +- **Auth.** bb-storage verifies the pods' projected service-account JWTs + against a JWKS file, which its `sync_jwks_to_configmap` keeps current. + An `instance_name_prefix` authorizer scopes each repo to the instance + `/`. `get_authorizer` admits every CI service account of + the tenant. `put_authorizer` admits only `saiph-ci-post`, the + postsubmit and deploy account, so a presubmit can read the cache but + never poison it. +- **Digests.** A source file's REAPI digest is the SHA-256 alias in the + store's content meta. `saiphd` sets the xattr `user.saiph.sha256` from + `ObjectService.GetContentMeta` ([Store additions](#store-additions)), + and Bazel reads it with `--unix_digest_hash_attribute_name`, so Bazel + hashes no source file in a virtual checkout. Store tree ids are not + REAPI `Directory` digests, so trees are not shared. +- **moon.** moon speaks the same protocol (`remote.host`, + `cache.instanceName`), but its cache stays off until OQ-2 is solved. + moon retries a transient cache error up to three times and only warns + on a failed write. Against Saiph's own cache that hides a defect, so + the cache error rate has its own alert, and task CI10 turns the retry + off or files the upstream change. + +### Caps and spend + +| Layer | Cap | Enforced by | +| -- | -- | -- | +| Tenant namespace | ResourceQuota on `requests.cpu`, `requests.memory`, `limits.cpu`, `limits.memory`, `pods` and `count/jobs.batch` in `saiph-ci-` | the API server refuses a Job or pod over quota; the shard stays queued and the quota alert fires | +| Container | LimitRange default and maximum | the API server | +| Repo | `max_running_shards` (default 20) | the controller creates no Job past it | +| Run | `max_shards_per_run` (default 200) | the planner refuses the run with `RESOURCE_EXHAUSTED` | +| Job | `activeDeadlineSeconds`, the sum of its targets' timeouts, at most 4 h | the Job controller ends the pod; its targets are FAILED | +| Finished Job | `ttlSecondsAfterFinished: 3600` | the TTL controller | +| Culprit search | `max_culprit_probes` (default 16) per breakage | the controller stops and leaves the search unconfirmed | +| Node pool | the cluster autoscaler's maximum for the pool tainted `saiph.rigel.build/ci` | the autoscaler adds no node past it | +| Cache | bb-storage block storage on a fixed-size volume | bb-storage evicts the oldest blocks | +| Logs | the bucket quota and a 30-day lifecycle rule on the `ci/` prefix | the object store | + +`saiph_ci_pod_seconds_total{tenant,repo,kind}` measures compute. A +deployment that enables CI on a usage-billed vendor also adds a budget +alert that pages Matt through the billing paging route, and a spend-cap +row per layer with evidence that it enforces. Until then CI stays off in +that deployment. + +### Store additions + +Folded into [store](store.md#api-the-saiphv1-connect-package): +`SetGrantRequest.ref_prefix` and the grants key change +([Refs and the ref log](store.md#refs-and-the-ref-log), task S2), and +`ObjectService.GetContentMeta` (task S9). + +### Alternatives considered + +- **Tekton Pipelines and Triggers.** Kubernetes-native, but triggers are + an `EventListener` with `TriggerBinding`s for webhooks, and each + `PipelineRun` keeps its own state beside CI's results. Every + self-hosted install would carry its CRDs and controllers. Rejected. +- **Argo Workflows and Argo Events.** Argo Events delivers events + asynchronously and unordered, and CI needs ordered, resumable ref and + version streams. Rejected. +- **Buildkite agent-stack-k8s.** Its control plane is Buildkite's hosted + service, so a self-hosted Saiph would depend on a vendor and egress. + Rejected. +- **Woodpecker.** The org knows it: an existing Woodpecker server runs + another monorepo's `moon ci`. Its Kubernetes backend runs pods with a + temporary PVC per pipeline and clones with git. Its forges are git + forges, and "Addon forges are still experimental". Results would be + keyed by pipeline number, not changeset. Rejected for Saiph; that + server is not affected. +- **Rerun a red target to test for a flake.** Trunk's merge queue advises + "Retry limit should be 1-3 max". Rejected: a rerun hides a flaky test, + and successor evidence with quarantine shows it instead. +- **Results keyed by run.** Rejected: a second run could replace a red + result. +- **bazel-remote.** It has no JWT auth (htpasswd or mTLS only), so it + cannot scope puts per repo by service account. Rejected. +- **A git clone or a git shim in the job.** Rejected: OQ-2 (b). +- **Per-directory config files.** OQ-1 (a). + +### Decisions + +- **D1 — DL-602.** `saiph-ci` is a first-party Go service that runs each shard of + targets as one Kubernetes Job with `backoffLimit: 0`. It adopts no CI + engine. +- **D2 — DL-603.** CI triggers from the store's `WatchRefs` and review's + `WatchEvents`, each resumed at a cursor stored in the same transaction + as the runs it creates. +- **D3 — DL-604.** A target is a build-graph task id. Affected targets come from + the store's snapshot `Diff`, fed to moon on stdin in a skeleton of + graph-input files, with `all` as the barrier. +- **D4 — DL-605.** CI policy is one `.saiph/ci.yaml`, read at the run's base and + never from the change under test. +- **D5 — DL-606.** Each (repo, changeset, target) has one immutable result. CI + never retries a target, and a Job never retries a pod. INFRA is red, + alerts, and is stored on the run only. +- **D6 — DL-607.** Postsubmit runs in TAP-style milestones. A red target opens a + breakage that keeps it red at trunk until a later trunk changeset passes + it. Culprit finding is flake-aware Bayesian search that runs each + (target, changeset) at most once, and it comments, never reverts. +- **D7 — DL-608.** A flake is two results that disagree on one input key. It opens + a flake breakage, and the target is quarantined, not required, until + the breakage closes. +- **D8 — DL-609.** Jobs get their source from a `saiphd` CSI inline ephemeral + volume and have no git. +- **D9 — DL-610.** CD follows Rapid: a release is a protected `release/*` ref cut + from a green milestone, a pick is a rebased trunk changeset, and stages + run deploy targets behind human `land` approvals. +- **D10 — DL-611.** The remote cache is REAPI with SHA-256 on bb-storage, scoped + per repo by instance prefix and written only by postsubmit. CI compute + is capped by a namespace ResourceQuota, a per-repo shard limit and a + budget alert that pages Matt. + +## Plan + +### Global Constraints + +- **Inherit [store](store.md#global-constraints):** public repo, Go 1.26, + `connectrpc.com/connect/v2`, Compass's lint rules, red/green tests, + `buf lint` STANDARD, SQL that YugabyteDB YSQL and Postgres 16 both run. +- **Layout.** `internal/ci/` holds `trigger`, `plan`, `adapter/moon`, + `control`, `step`, `results`, `culprit`, `deploy` and `api`. + `cmd/saiph-ci` is the service and `cmd/saiph-ci-step` the step wrapper. + Migrations live in `internal/ci/migrations/`, numbered from `0001`. +- **Kubernetes ≥ 1.33**, for `hostUsers: false`. `k8s.io/client-go` only; + no CRDs. +- **No retries.** Nothing reruns a target, a pod or a shard. Resuming a + stream at its cursor is allowed. A call to the store or review that + fails ends the step loudly, and the work stays queued. +- **moon v2.6.2 or later** as the first adapter, called as a binary. +- **Tests.** Controller tests use a fake clientset. One end-to-end test + per task runs a kind cluster with the CSI driver from client task C8. + +### Order + +```text +CI1 proto ─┬─ CI2 schema ─┬─ CI3 triggers ─┬─ CI6 presubmit ─┐ + │ ├─ CI4 planner ──┼─ CI7 postsubmit ┼─ CI11 deploy+caps + │ └─ CI5 controller┘ │ +CI9 store adds ─ CI8 CD ─────────────────────────────────────┤ +CI10 cache ──────────────────────────────────────────────────┘ +``` + +### CI1 — Proto and generated code + +`proto/saiph/v1/ci.proto` as in [API](#api-protosaiphv1ciproto). Tests: +`buf lint` passes; `buf breaking` against the frozen file passes. + +Interfaces: + +```go +// gen/saiph/v1/saiphv1connect +type TargetServiceHandler interface{ /* AffectedTargets, GetResults, ListBreakages, CloseBreakage, GetTargetStats */ } +type RunServiceHandler interface{ /* RequestRun, GetRun, WatchRun, CancelRun */ } +type StepServiceHandler interface{ /* FinishShard, UploadLog */ } +type DeployServiceHandler interface{ /* CutRelease, ProposePick, Approve, GetRelease */ } +``` + +### CI2 — The `saiph_ci` schema and the results store + +The tables in [Results](#results) with RLS, and the result insert. Tests: +a second insert for one changeset and target leaves the first; +another tenant's rows are invisible; `ResultsByInputKey` returns both +sides of a disagreement. + +Interfaces: + +```go +// internal/ci/results +type Result struct{ Changeset object.ID; Target string; Passed bool; InputKey [32]byte; RunID string; DurationMs int64; TrunkSeq *int64; Quarantined bool } +func (s *Store) Insert(ctx context.Context, tenant string, repo meta.RepoID, r Result) (inserted bool, err error) +func (s *Store) Get(ctx context.Context, tenant string, repo meta.RepoID, cs object.ID, targets []string) ([]Result, error) +func (s *Store) ResultsByInputKey(ctx context.Context, tenant string, repo meta.RepoID, target string, key [32]byte) ([]Result, error) +``` + +### CI3 — Triggers + +The `WatchRefs` and `WatchEvents` consumers, the cursor lease and the +`ListChanges` rescan. Tests: a consumer killed between two events +creates each run once; `OUT_OF_RANGE` rescans and resumes at +`event_seq`; a second replica waits for the lease. + +Interfaces: + +```go +// internal/ci/trigger +type Sink interface{ OnRef(ctx context.Context, e *saiphv1.WatchRefsResponse, tx pgx.Tx) error; OnReview(ctx context.Context, e *saiphv1.ReviewEvent, tx pgx.Tx) error } +func RunRefs(ctx context.Context, env *Env, repo string, sink Sink) error +func RunReview(ctx context.Context, env *Env, repo string, sink Sink) error +``` + +### CI4 — Planner and the moon adapter + +`AffectedTargets`, the graph manifest, the skeleton cache, the moon +adapter, classes and the input key. Tests: a diff under one project +returns its tasks and their dependents; a `moon.yml` change sets +`graph_changed`; a `.moon/workspace.yml` change sets `all`; equal base +and head returns no targets without running moon; a speculative base +with no common ancestor works; an edit outside a target's inputs keeps +its input key. + +Interfaces: + +```go +// internal/ci/plan +type Affected struct{ Targets []string; All bool; GraphID [32]byte; GraphChanged bool } +type Adapter interface { + GraphInputs(cfg *Config) []string + Affected(ctx context.Context, skeleton string, paths []string) ([]string, error) + InputDirs(ctx context.Context, skeleton, target string) ([]string, error) +} +func (p *Planner) AffectedTargets(ctx context.Context, repo string, base, head object.ID) (Affected, error) +func (p *Planner) InputKey(ctx context.Context, repo string, cs object.ID, target string) ([32]byte, error) +func LoadConfig(ctx context.Context, c saiphv1connect.ObjectServiceClient, repo string, base object.ID) (*Config, error) +``` + +### CI5 — Controller, Jobs and the step wrapper + +`RequestRun`, `GetRun`, `WatchRun`, `CancelRun`, sharding, the pending +queue, Job creation, `StepService`, logs and INFRA. Tests: the same +idempotency key returns the same run; a target with a result returns +`reused=true`; the repo never exceeds `max_running_shards`; an evicted +pod yields INFRA, fires the alert and creates no new Job; a Job has +`backoffLimit: 0`; a wrong shard token is `UNAUTHENTICATED`. + +Interfaces: + +```go +// internal/ci/control +type Shard struct{ RunID string; N uint32; Targets []string; Priority uint32 } +func (c *Controller) RequestRun(ctx context.Context, req *saiphv1.RequestRunRequest) (*saiphv1.RequestRunResponse, error) +func (c *Controller) JobFor(s Shard, cfg *plan.Config, src CSISource) *batchv1.Job +type CSISource struct{ Repo string; Changeset object.ID; Sparse []string; Mode string } +// cmd/saiph-ci-step +func runShard(ctx context.Context, step saiphv1connect.StepServiceClient, runID string, shard uint32, targets []string) error +``` + +### CI6 — Presubmit + +Runs on `VERSION_UPLOADED`, `ReportChecks`, cancellation on a new version +or `ABANDONED`, the Restricted profile and the NetworkPolicy. Tests: a +stack child's run covers its parents' changes; a failed target reports +`FAILURE` and `required=true`; a quarantined one `required=false`; a +presubmit pod cannot reach the internet or put to the cache. + +Interfaces: + +```go +// internal/ci/control +func (c *Controller) OnVersion(ctx context.Context, e *saiphv1.ReviewEvent, tx pgx.Tx) error +func checkRuns(results []*saiphv1.TargetResult) []*saiphv1.CheckRun +``` + +### CI7 — Postsubmit, breakages, culprits and flakes + +Milestones, breakages, `broken_at_base`, culprit search, flake +detection, quarantine, `ListBreakages`, `CloseBreakage` and +`GetTargetStats`. Tests: a red milestone opens one breakage; with a +uniform prior and no flakes the search finds the culprit in +⌈log₂ n⌉ probes; with a 10% flake rate it reaches 0.95 on a seeded +simulation; no (target, changeset) runs twice; a confirmed culprit gets +one comment and an attention entry; a disagreement on one input key +quarantines the target. + +Interfaces: + +```go +// internal/ci/culprit +type Candidate struct{ Changeset object.ID; TrunkSeq int64 } +type Search struct{ Candidates []Candidate; FlakeRate float64; Observed map[object.ID]bool } +func (s *Search) Posterior() []float64 +func (s *Search) Next() (Candidate, bool) +func (s *Search) Confirmed(threshold float64) (Candidate, bool) +``` + +### CI8 — Deploy + +`DeployService`, release cuts and picks, stage Jobs and approvals. +Depends on CI9. Tests: CD's grant cannot move trunk; a conflicting pick +is refused with paths; a stage with `approval: true` waits for a human +`land` account and refuses a service subject; a failed stage stops the +release. + +Interfaces: + +```go +// internal/ci/deploy +func (d *Deployer) Cut(ctx context.Context, repo, name string, at object.ID) (*saiphv1.Release, error) +func (d *Deployer) Pick(ctx context.Context, repo, release string, trunkCS object.ID) (*saiphv1.ReleaseItem, error) +func (d *Deployer) Approve(ctx context.Context, p auth.Principal, req *saiphv1.ApproveRequest) (*saiphv1.Release, error) +``` + +### CI9 — Store additions in `saiph-server` + +Folded into store tasks S2 and S9 +([store](store.md#s2--metadata-schema-and-the-ref-cas)), which carry the +tests and interfaces. + +### CI10 — Remote build cache + +bb-storage manifests, the JWKS sync, the instance-prefix and put +authorizers, and moon's cache retry turned off or an upstream change +filed. Tests: a presubmit token reads and cannot put; a repo cannot read +another repo's instance; a Bazel build on a virtual checkout hashes no +source file. + +Interfaces: + +```go +// deploy/cache: bb-storage config (Jsonnet), JWKS ConfigMap, Deployment, Service +``` + +### CI11 — Deployment, caps and alerts + +`deploy/ci/` manifests: the `saiph-ci` Deployment, the namespaces per +tenant with ResourceQuota, LimitRange and NetworkPolicy, the +PriorityClasses, the service accounts and grants. The infra, trigger-lag, +quota and cache-error alerts. The caps rows and the budget alert. Tests: +a Job over quota stays queued and fires the quota alert; each alert fires +on a forced fault and stays quiet on a healthy run. + +Interfaces: + +```go +// cmd/saiph-ci +type ciFlags struct{ DSN, SaiphURL, Issuers, Namespace string; SkeletonBytes int64; MaxRunningShards int } +func registerFlags(fs *flag.FlagSet) *ciFlags +``` + +## Tasks + +- [ ] CI1 — `proto/saiph/v1/ci.proto` and generated code +- [ ] CI2 — `saiph_ci` schema, RLS and results store +- [ ] CI3 — triggers: `WatchRefs`, `WatchEvents`, cursors and lease +- [ ] CI4 — planner, moon adapter, graph manifest, skeleton, input key +- [ ] CI5 — controller, Jobs, `saiph-ci-step`, logs, INFRA +- [ ] CI6 — presubmit, `ReportChecks`, sandbox +- [ ] CI7 — postsubmit milestones, breakages, culprit search, flakes and quarantine +- [ ] CI8 — deploy: releases, picks, stages, approvals +- [ ] CI9 — store additions (done in store S2 and S9) +- [ ] CI10 — remote build cache on bb-storage +- [ ] CI11 — deploy manifests, caps, alerts and budget alert diff --git a/docs/designs/saiph/client.md b/docs/designs/saiph/client.md new file mode 100644 index 0000000..c3493fd --- /dev/null +++ b/docs/designs/saiph/client.md @@ -0,0 +1,1105 @@ +# Saiph: client, daemon, and cache tiers + +Tracker: RIG-5665 + +> Freezes on merge; later changes supersede by citation, never rewrite +> (convention: the `docs/designs/agent/compass-agent-config-delivery/design.md` +> header). Unless a path says otherwise, every code path is in the +> `RigelBuild/saiph` repo, which does not exist yet. A Compass path says +> "in Compass". + +Ledger-impact: D1–D7 are filed as DL-590..DL-596. The store additions +this record asked for are folded into [store](store.md#api-the-saiphv1-connect-package) +(tasks S6 and S11). + +**Named decisions this record introduces:** D1–D7 in [Decisions](#decisions). + +This record designs Saiph's client side: the `saiph` CLI, the node daemon +`saiphd`, the regional cache `saiph-cache`, the object protocol between +them, sparse and virtual checkouts, the CSI driver that gives CI its +source, the token model that tells `saiph-server` who calls it, and the +agent `vcs_*` tools on the Saiph backend. It builds on the entities and +RPCs of [store](store.md) and the rulings in the +[overview](README.md#rulings). Its additions to `saiph.v1`, listed in +[Store additions](#store-additions), are folded into store. + +## Open Questions + +Matt ruled that Saiph has its own client, built Scion's way (ruling 11), +and that the core follows Scion to the letter, with `facebook/sapling` +(EdenFS, the Sapling client) as the reference where the post is silent +(ruling 10). Settled by those rulings, so not open: jj as the client or +backend (rejected, see [Alternatives considered](#alternatives-considered)), +and running EdenFS (ruling 12). The record is written against each +recommendation below. None depends on another, except OQ-6, which depends +on [git-bridge](git-bridge.md). + +### OQ-1 — Which checkout mode do agent sessions get? (load-bearing) + +A virtual checkout is a FUSE filesystem served by `saiphd`. Compass's +microVM tier shares exactly one read-write host directory into the guest +through virtiofsd (`workspaceShare` in +`go/internal/runtime/microvm_lifecycle.go` in Compass: "the backend +expresses exactly one read-write workspace share"). Nobody has tested +virtiofsd over a FUSE mount on the host. + +- (a) Virtual on the Podman tier. Materialized on the microVM tier until + task C6 shows virtiofsd serves the FUSE root correctly; then virtual + there too. +- (b) Materialized everywhere. Simple, but a large repo costs a full + sparse fetch and disk per session, which is what Scion avoids: "when a + job does need files on disk, the client only pays for what it actually + reads". +- (c) Virtual everywhere now. Risks a broken microVM tier. + +**Recommendation: (a).** The mode is a per-session field +(`CreateSessionRequest.mode`), so the switch is a Runner flag, not a +migration. + +### OQ-2 — How does `saiphd` upgrade without breaking virtual mounts? + +A FUSE mount dies with the process that serves it. + +- (a) Takeover, as EdenFS does: "a new EdenFS daemon to seamlessly take + over mount points from an existing running daemon without unmounting + the filesystems" + ([Takeover.md](https://github.com/facebook/sapling/blob/main/eden/fs/docs/Takeover.md)). + The DaemonSet uses `maxSurge: 1`, so the new pod starts beside the old + one and receives the FUSE file descriptors and the session table over a + unix socket. +- (b) Restart: on upgrade the Runner restarts every session that has a + virtual root. Materialized roots survive any restart. + +**Recommendation: (a), with (b) as the fallback.** go-fuse opens a +mount from an inherited descriptor (the `/dev/fd/N` mount point in +`fuse/mount_linux.go` of `hanwen/go-fuse`), but a resumed server must not +repeat the FUSE `INIT` handshake. Task C6 proves that first; if go-fuse +cannot skip it, C6 carries a patch (go-fuse is BSD-licensed), and until +it lands the upgrade uses (b). + +### OQ-3 — Does `saiphd` start from a snapshot? + +Scion: "Each daemon starts from a periodically refreshed snapshot". The +post does not say what a snapshot holds or how it reaches a node. + +- (a) No snapshot file. The daemon's object store is on node disk and + survives restarts. A new node prefetches the trunk head of the repos + and sparse prefixes in its `--prefetch` list from `saiph-cache`. +- (b) A snapshot job packs trunk's objects into one file in object + storage, and each new daemon downloads it. That adds a second + distribution path, object-storage credentials on every node, and a + job to run. + +**Recommendation: (a).** It gives a warm start with the RPCs the tiers +already serve. [Deviations from Scion](#deviations-from-scion) lists it. + +### OQ-4 — Does a session with git and Saiph repos keep GitHub egress? + +Ruling 4: "No GitHub egress for agents on a fully native repo." Egress is +per session (one network namespace), not per repo. + +- (a) A session keeps the forge hosts when any granted repo is on the git + backend, and loses them when every granted repo is on Saiph. +- (b) Compass refuses to grant an agent repos on both backends. + +**Recommendation: (a).** The GitHub credential still covers only the git +repos, so a mixed session gains nothing it lacks today. (b) blocks the +migration, which moves one repo at a time ([git-bridge](git-bridge.md)). + +### OQ-5 — Does the agent hold a Saiph token? (security) + +- (a) No. The Runner relays the agent's Saiph calls to `saiphd`, and the + daemon attaches the session's token upstream, as the Runner relays + forge calls today (`Gateway.Forge` in + `go/internal/runner/gateway/forge.go` in Compass: "forwards one + agent-initiated forge call to the Server's RelayForgeCall under the + session bound to this container"). +- (b) The token is a 0600 file in the agent's `$HOME`, as the GitHub + credential is (`gitCredentialSecretName` in + `go/server/git_credential.go` in Compass). + +**Recommendation: (a).** The agent gains no credential, which is the +posture of `agent-vcs-tools.md` ("The agent gains no credential"). It +also needs no network egress for Saiph at all, so the microVM tier needs +no new device. + +### OQ-6 — How is a repo that exists only in Saiph granted to an agent? (depends on git-bridge) + +`AgentRepositoryService.AddAgentRepository` in Compass refuses a repo +with no GitHub App installation ("FAILED_PRECONDITION when no GitHub App +is configured … the App has no installation for the org"). + +- (a) It accepts the repo when `repoCutoverState` (in + `go/server/vcs_backend.go` in Compass, designed in [git-bridge](git-bridge.md)) + says it is not on the git backend, skips the GitHub checks, and stores + the grant row as today. The git credential broker skips every repo + whose state is not `GIT`, as the cutover already requires. +- (b) A new forge provider value `saiph` for such rows. + +**Recommendation: (a).** One grant table, and the backend comes from one +function, as agreed with [git-bridge](git-bridge.md). + +### OQ-7 — Does macOS get virtual checkouts? + +EdenFS serves macOS over NFSv3 ("On macOS, EdenFS uses NFSv3", +[Overview.md](https://github.com/facebook/sapling/blob/main/eden/fs/docs/Overview.md)). + +- (a) macOS gets materialized checkouts only in this record. +- (b) An NFSv3 server inside `saiphd`. + +**Recommendation: (a).** Only humans run macOS, and a sparse +materialized checkout serves them. (b) is a new record when a large repo +needs it. + +### OQ-8 — How does a human get a Saiph token from Compass? + +- (a) `saiph auth login ` runs the OAuth 2.0 device grant + (RFC 8628) against Compass. Compass returns a one-hour token and a + refresh token, which the CLI keeps in a 0600 file. +- (b) Compass's settings page mints a 30-day token that the human pastes + in. A JWT cannot be revoked before it expires, so a leaked token lives + for 30 days. + +**Recommendation: (a).** Tokens stay short-lived. A refresh token is an +opaque row in Compass that Compass can revoke. A standalone install uses +Saiph's own revocable `saiph.tokens` rows instead ([store](store.md) OQ-6). + +## Problem / Intent + +Scion's server does the work and its client stays small: "The client +maintains a bounded local object cache, lays out the working copy files +on disk, monitors for filesystem changes, and gives users an ergonomic +way to work with the server" +([Scion](https://www.anthropic.com/engineering/redesigning-source-control)). +This record designs that client for Saiph, with the tiers Scion puts +between the client and the server, and plugs it into the places that use +it: Compass agent sessions, CI pods on Kubernetes, and a human at a +terminal. The git backend (RIG-5558) stays as it is (ruling 5). + +## Approach + +### Ground truth + +- **Saiph's repo** is `RigelBuild/saiph`. It does not exist yet, so every + Saiph path here is a plan. `cmd/saiph`, `cmd/saiphd` and + `cmd/saiph-cache` belong to this record ([store](store.md#global-constraints) + layout). +- **The store contract.** Every server call here is a `saiph.v1` RPC from + [store](store.md#api-the-saiphv1-connect-package): `GetTrees`, + `GetBlobs`, `GetChangesets`, `FindMissingBlobs`, `PutBlobs`, + `PutChunkedBlob`, `ReadTree`, `ReadFile`, `Commit`, `Rebase`, `Merge`, + `Log`, `MergeBase`, `Diff`, `Blame`, `GetRef`, `Pin`, `Unpin`, + `GetRepo`, `SetGrant`. Id reads need read on the whole repo; path reads + check each path ([store](store.md) OQ-3). +- **The review contract** ([review](review.md)): package `saiph.v1`, file + `proto/saiph/v1/review.proto`, service `ReviewService`, served by its + own binary `saiph-review`, with + `UploadVersions`, `AbandonChange`, `GetChange` and `GetStack`. The + caller must be the agent's account (`account:`), never a service. +- **The cutover contract** ([git-bridge](git-bridge.md)): Compass reads a repo's state + with `repoCutoverState` in `go/server/vcs_backend.go` in Compass, which + returns `GIT`, `IMPORTING`, `NATIVE_MIRRORED` or `NATIVE_ONLY`. +- **The CI contract** ([ci-cd](ci-cd.md)): CI pods get source through the CSI + driver below, with the attributes agreed there. CI nodes carry the + taint `saiph.rigel.build/ci=true:NoSchedule`. +- **Compass today.** An agent reaches its Runner over one unix socket per + container, and the Runner relays forge calls to the Server under the + session binding (`Gateway.Forge` in + `go/internal/runner/gateway/forge.go` in Compass). In a microVM, guestd + forwards `/run/compass/agent.sock` to the Runner over vsock + (`go/internal/guestd/supervisor.go` in Compass). The microVM tier boots + "exactly one read-write workspace share" at `/workspace` + (`workspaceShare` in `go/internal/runtime/microvm_lifecycle.go` in + Compass). `runtime.SourceVolume` is "a checkout dir backed by a writable + session-volume mount" (`go/internal/runtime/workspace.go` in Compass). + The Runner's egress policy is one static allowlist + (`SpecDefaults.Egress` in `go/internal/runner/spec.go` in Compass, from + `--egress-allow`). The GitHub credential reaches the agent as the file + secret `GITHUB_APP_TOKEN` (`brokeredSecretResolver.ResolveFor` in + `go/server/git_credential.go` in Compass). +- **Compass already verifies JWTs** from Kubernetes issuers with + `go-jose/go-jose/v4`: `RunnerVerifier.Verify` in + `go/internal/auth/workload_identity.go` in Compass refreshes a JWKS + every hour (`jwksRefreshEvery`), re-fetches on an unknown key id at + most every five minutes (`kidRefetchCooldown`), keeps the last good keys + for a day (`staleKeysAfter`), and requires `exp` and `iat` because + "go-jose skips an absent time claim". Saiph's verifier follows the same + rules, written fresh in the saiph repo. + +### Four tiers + +Scion: "In the fleet, the client talks over a socket to a daemon, +scion-daemon, that stores objects for the whole node … talks only to the +tree and blob cache servers in its region, called scion-cache … When a +cache is missing an object, it asks the authoritative scion-servers". +Saiph has the same four tiers. + +```mermaid +flowchart LR + subgraph node[Node] + CLI[saiph CLI] -->|unix socket| D[saiphd: object store, workspaces, views] + CSI[kubelet] -->|CSI| D + end + D -->|GetTrees, GetBlobs, GetChangesets| C[saiph-cache: memory LRU] + C -->|miss| S[saiph-server] + D -->|every other call| S + D -->|ReviewService| R[saiph-review] +``` + +- **`saiph`** is the CLI. It holds no repo data. It asks `saiphd` for + objects and workspace state, and sends mutations through the daemon to + the server. +- **`saiphd`** runs once per node. It keeps the bounded object store for + every job on the node, lays out workspaces, watches them for changes, + and holds each workspace's view and op log. It forwards every call it + does not answer to its upstreams with the caller's own bearer token. +- **`saiph-cache`** runs once per region. It answers id reads from memory + and asks the servers on a miss. +- **`saiph-server`** is the store's server ([store](store.md)). + +Tiering is strict, as Scion's is: "we strictly enforce the tiering to +isolate failure domains". A daemon sends id reads only to `--cache`. A +cache sends them only to `--server`. When the cache is down, the daemon +fails the read; it never falls back to the server. Calls that are not id +reads (`Commit`, `Log`, `Diff`, path reads, refs, pins, review) go from +the daemon to their service directly, because they are the client's work +with the server, not the fetch protocol. + +### Deviations from Scion + +| Scion | Saiph | Why | +| -- | -- | -- | +| "Each daemon starts from a periodically refreshed snapshot" | No snapshot file. The daemon's store survives restarts on node disk, and a new node prefetches its `--prefetch` list from the cache (OQ-3) | The post does not say what a snapshot is. A second distribution path needs object-storage credentials on every node. | +| One client process on each machine | On a laptop, `saiph` starts a per-user `saiphd` whose `--cache` is the server URL | Scion's tiers are "in the fleet". A laptop has no regional cache. | + +### Gaps filled from EdenFS and Sapling + +The post leaves these open. Each row names the reference and marks where +Saiph departs. Paths are in `facebook/sapling`. Designs only, never code +(ruling 12). + +| Gap | Reference | Saiph | +| -- | -- | -- | +| Virtual checkout | EdenFS: FUSE on Linux, "lazily fetching file data" (`eden/fs/docs/Overview.md`) | Follows: FUSE through `hanwen/go-fuse/v2` | +| Local edits in a virtual checkout | EdenFS materializes a changed inode and its ancestors into the overlay ("Materialized files are stored in the overlay", `eden/fs/docs/Glossary.md`) | Follows | +| Change tracking | EdenFS Journal, used "to implement APIs like `getFilesChangedSince()`" (`Glossary.md`); Sapling uses Watchman so status runs "in O(files-you-have-changed) time" (`website/docs/scale/axes.md`) | Follows: the FUSE journal in virtual mode, inotify in materialized mode | +| Kernel caching | EdenFS "returns an infinite expiry" and invalidates explicitly, because it "tracks all writes" (`eden/fs/docs/Caching.md`) | Follows | +| In-memory blob cache | EdenFS keeps "a capped-size, minimum-entry, in-memory LRU cache for recently-accessed blobs" (`Caching.md`) | Follows | +| Disk store eviction | EdenFS: "Local store eviction is an unsolved problem" (`Caching.md`) | **Departs:** a byte cap with LRU eviction, because Scion's cache is "bounded" | +| Daemon restart | EdenFS takeover hands mounts to a new daemon "without unmounting the filesystems" (`eden/fs/docs/Takeover.md`) | Follows (OQ-2) | +| Undo | Sapling MetaLog "makes atomic updates possible, and allows viewing past states for debugging and undoing" (`website/docs/dev/internals/metalog.md`) | Follows the idea: a per-view op log on the node | +| Graph view | Sapling Smartlog shows "just the ones relevant to you" (`website/docs/overview/smartlog.md`) | Follows | +| Change identity | Sapling mutation records | **Departs:** the store's change id ([store](store.md#changeset)) | +| Open conflicts | Sapling refuses to commit unresolved conflicts ([store](store.md) OQ-5) | Follows | + +### The object protocol: want-only fetch + +Scion: "Scion's negotiation doesn't tell the server what the client +already has; instead, it operates exclusively on trees and blobs the +client wants. Starting from a root tree, the client checks its local +store for missing trees and requests them recursively, then repeats the +process for blobs." Saiph's fetch is exactly that, over the store's id +reads: + +1. Read the changeset (`GetChangesets`) for its `root_tree`. +2. Walk trees from the root. At each level, collect the child tree ids + that the sparse set includes and the local store lacks. Ask for them + with `GetTrees`, at most 1,000 ids per call, and repeat level by level. +3. Collect the blob ids under the sparse set that the store lacks. Ask + for them with `GetBlobs`. Each tree entry carries a size, so the + fetcher packs each call under the 16 MiB response limit. A blob larger + than 4 MiB comes back as a chunk list; the fetcher then asks for the + chunks. + +There is no have-list and no pack. "Every request is for a specific +object that never changes, so any cache can answer it", and a fetch that +stops resumes from what the store already holds. A virtual checkout stops +after step 1: it fetches a tree when a directory is first looked up and a +blob when a file is first read. Scion: "Sparse and even virtualized +checkouts don't need special support, because the client simply doesn't +ask for trees it doesn't want." + +- **Same RPCs at every tier.** `saiphd` and `saiph-cache` implement the + read half of `saiphv1connect.ObjectServiceHandler` (`GetTrees`, + `GetBlobs`, `GetChangesets`) with the store's messages. Scion: they + "all respond to identical RPCs for this protocol, so the client doesn't + need to know the cache topology". +- **Wire form is the disk form.** Scion's caches "stream objects directly + from memory, in the disk format that the daemon will consume". A tier + asks with `GetBlobsRequest.stored_form` and gets `Blob.stored`: the + store's object body (a format byte, then zstd). Tree and changeset + objects travel as their `encoded` CBOR bytes, with `omit_decoded` set. + Both fields are [Store additions](#store-additions). +- **Every object is verified** before a tier keeps it. A tree or changeset + decodes with `object.DecodeTree` or `object.DecodeChangeset` (store S1) + and must hash to the asked id. A blob must hash to its id after + decompression (`object.BlobID`). A tier never serves bytes it has not + verified, so a faulty tier cannot corrupt a checkout. +- **Access before cache.** Store's caches "authenticate each caller for + the repo before they answer from cache". A tier calls `GetRepo` upstream + with the caller's bearer and keeps the verdict for 60 seconds, keyed by + the SHA-256 of the bearer and the repo, and never past the token's + `exp`. An id read is served only when the verdict says `id_reads` (the + caller reads the whole repo). A caller with partial read gets + `PERMISSION_DENIED` on id reads and uses path reads (`ReadTree`, + `ReadFile`), which every tier passes through uncached. +- **Cache keys** are `(tenant, repo, id)`, as in `saiph-server`. The tenant + comes from the verdict (`GetRepoResponse.tenant`), never from the + request. + +### `saiph-cache` + +- A Deployment per region, from the saiph repo's `deploy/`, with + `replicas: N` and a Service with `trafficDistribution: PreferSameZone`. + Each added region adds one Deployment ([store](store.md#deviations-from-scion)). +- An in-memory LRU of stored bodies, bounded by `--memory-bytes` (default + 8 GiB). No disk: Scion's tiers do "not require long-lived state". +- A miss asks the server once per key, however many callers wait + (`golang.org/x/sync/singleflight`). +- Replicas do not share entries; each holds what its callers asked for. + Scion does not say how its caches shard, so Saiph starts with the + simplest form and measures the hit rate. +- It serves only `GetTrees`, `GetBlobs` and `GetChangesets` from cache, and + passes `GetRepo`, `ReadTree` and `ReadFile` through. Every other method + answers `UNIMPLEMENTED`. + +### `saiphd`: the node daemon + +Scion's client "maintains a bounded local object cache, lays out the +working copy files on disk, monitors for filesystem changes". On a node +those jobs are in `saiphd`, shared by every job on the node. + +- **Where it runs.** On a Kubernetes node it is a DaemonSet (privileged, + for `/dev/fuse` and mount propagation). On a Compass Runner host it is a + host service beside `compass-runner`, on the socket the Compass + deployment names: `/run/compass/vcs/daemon.sock` (agreed with + [ci-cd](ci-cd.md); only the Runner reaches it). On a laptop, `saiph` starts a + per-user `saiphd` on `$XDG_RUNTIME_DIR/saiph/daemon.sock`. +- **Object store.** Stored-form objects under `--store-dir`, one file per + object at `/////`, bounded by + `--store-bytes` (default 50 GiB) with LRU eviction. A read touches the + file's mtime at most once an hour; startup rebuilds the LRU from mtimes. + An evicted object is fetched again on its next read. +- **Access.** The store is shared across sessions and tenants, so a hit is + served to a session only after the access verdict for that session's + token and the repo says `id_reads` (the same cache as the tiers). A + session with partial read gets path reads, uncached. +- **Sessions.** A session binds a token and a checkout mode to a set of + workspaces. The Runner creates one per agent session, the CSI driver + one per volume, and the laptop daemon has one named `local`. + `SetSessionToken` replaces the token before it expires. Deleting a + session unmounts and deletes its workspaces. +- **Proxy.** `saiphd` serves `saiph.v1`, including `ReviewService`, on its + socket and forwards every call it does not answer (writes, history, + refs, review) to `--server` or, for `ReviewService`, `--review` (the + `saiph-review` endpoint) with the session's token. Id reads go to + `--cache` only. +- **Daemon API.** The CLI and the Runner talk to `saiphd` over + `saiphd.v1` (task C5). It is local, not part of `saiph.v1`. + +### Checkouts: sparse, virtual and materialized + +A workspace is a directory, a repo, a sparse set of path prefixes (empty = +whole tree), a mode and a view. `.saiph/workspace` at its root names the +workspace id and repo; the CLI finds it by walking up from the working +directory. + +- **Materialized.** `saiphd` fetches the sparse set (the want-only walk) + and writes the files. Changing the sparse set fetches or deletes only + the difference. This mode works everywhere, including macOS (OQ-7) and + the microVM share (OQ-1). +- **Virtual.** `saiphd` mounts a FUSE filesystem with + `hanwen/go-fuse/v2`, `allow_other`, at the workspace directory. A + lookup fetches a tree; an open or read fetches the blob. Kernel entry + and attribute timeouts are infinite, and `saiphd` invalidates entries + itself on checkout, because it sees every write (as EdenFS does). The + sparse set still applies: a path outside it does not exist. +- **Local edits.** A write to an unmodified file copies it into the + session's overlay directory, with its parent directories, as EdenFS + materializes inodes. The overlay is bounded by `--overlay-bytes` per + session (default 20 GiB); past it, writes fail with `ENOSPC`. +- **Read-only.** A CSI volume with `readOnly: true` refuses every write + in both modes. +- **Podman tier.** The Runner bind-mounts the workspace directory into + the container with `rslave` propagation, so a virtual mount made by + `saiphd` shows through. +- **MicroVM tier.** The workspace is the `/workspace` share's host + directory. Materialized works now; virtual waits on task C6 (OQ-1). + +### Filesystem monitor + +Sapling uses a filesystem monitor so status runs "in O(files-you-have-changed) +time". Saiph's monitor lives in `saiphd`, never in the CLI. + +- **Virtual mode:** the FUSE server sees every write. It appends the path + to an in-memory journal with a sequence number, as EdenFS's Journal + does. Status reads the journal since the last snapshot and the overlay. +- **Materialized mode:** an inotify watch per directory (through + `fsnotify`). An overflow (`IN_Q_OVERFLOW`) marks the workspace for one + full rescan, which compares size and mtime against the last snapshot and + hashes only the candidates. On the microVM tier, virtiofsd writes to the + host directory with ordinary syscalls, so a host watch sees guest writes + ([INFERENCE]; task C6 tests it). +- **Ignores.** `.gitignore` files are honored, so mirrored repos behave as + they do in git. + +### Working copy, commit and conflicts + +The working copy is the files on top of one changeset, `wc_parent`, as in +Sapling. Nothing is committed until the user commits. + +- **Commit.** `saiph commit` takes the changed paths from the monitor, + hashes each file (`object.BlobID`), asks `FindMissingBlobs`, uploads the + missing ones with `PutBlobs` (or chunks and `PutChunkedBlob` over 4 MiB), + and calls `Commit` with `parents = [wc_parent]`. The server builds the + tree ([store](store.md#server-side-commit-rebase-and-merge)). New blobs + also go into the node store. The new changeset becomes `wc_parent`. +- **Amend** commits with the old changeset's parents and `change_id`, then + calls `Rebase` for the view's descendants of the old changeset onto the + new one, as jj rebases descendants automatically. +- **Conflicts** stay in the working copy (store OQ-5 (a)). When `Rebase` + or `Merge` returns a `Conflict` for the changeset being checked out, + `saiphd` writes conflict markers from its `ConflictHunk`s and records the + conflicted paths. `saiph commit` refuses while any path is unresolved, + as Sapling does; `saiph resolve ` marks one resolved. A rebase of + a stack stops at the first conflict; `rewrites` names what moved. + +### Change ids, views, the op log and undo + +These come from jj and Sapling, built inside Saiph's own client. + +- **Change ids.** The store gives each changeset a 16-byte change id that + survives amend and rebase ([store](store.md#changeset)). The CLI prints + it as jj does: "hexadecimal numbers that use 'digits' z-k instead of + 0-9a-f" (jj glossary). Changeset ids print as lowercase hex. The two + alphabets cannot be confused, and either works as a revision. The CLI + prints the shortest prefix unique within the view. +- **View.** Each workspace has a view: `wc_parent`, the visible heads, + local bookmarks (a name to a change id) and the trunk head last seen. + Bookmarks never reach the server; Saiph's only refs are trunk and + `release/*` ([store](store.md#refs-and-the-ref-log)). Review keys a + stack by change id, not by name. +- **Op log.** Every command that changes a view writes an operation: id, + parent op, time, the command line and the new view, as jj's + `Operation{view_id, parents, metadata}` and Sapling's MetaLog do. The + log is one append-only file per workspace in `saiphd`, fsynced per op. + `PutView` is a compare-and-swap on the head op, so two commands racing + on one workspace cannot both win. +- **Undo.** `saiph undo` writes a new op whose view is the view before the + last op; `saiph op restore ` writes the view of any op. Undo is + itself an op, so it can be undone. Undo changes the view only; it never + deletes a changeset, and the store keeps unpinned changesets for the + repo's retention window. +- **Keeping heads alive.** `saiphd` pins each view's heads with + `Pin(holder = "workspace:", ttl_seconds = 2592000)`, refreshed daily, + and unpins heads that leave every op in the last 30 days. +- **Smartlog.** `saiph log` with no arguments shows the view's heads, + their ancestors down to their merge base with trunk, and trunk's head, + as Sapling's smartlog shows "just the ones relevant to you". It reads + `HistoryService.Log(heads = view heads + trunk, stop = merge bases)` and + marks each change with its review state from `ReviewService.GetStack`. + +### Commands, and how Matt uses them + +```text +saiph auth login https://compass.example.com # OQ-8 (a) +saiph clone owner/name ~/src/name --sparse go/ --sparse docs/ +cd ~/src/name +saiph log # smartlog +saiph new trunk # start on trunk's head +saiph commit -m "fix: ..." # upload + Commit +saiph amend # keep the change id, restack descendants +saiph sync # fetch trunk, Rebase the view's stacks onto it +saiph submit --ready # ReviewService.UploadVersions +saiph undo # back one op +saiph op log +saiph sparse add web/ +``` + +The full set: `clone`, `status`, `diff`, `log`, `show`, `new`, `commit`, +`amend`, `rebase -s -d `, `sync`, `resolve`, `bookmark +set|delete`, `submit`, `abandon`, `undo`, `op log`, `op restore`, +`sparse add|remove|list`, `auth login|token`, `cat`, `ls`. Every command +takes `--json`. + +### Jobs that skip the checkout + +Scion: "many write workloads can also skip the checkout and function +entirely via API". Every query the CLI makes is a `saiph.v1` call, so a +job can make it without a workspace. + +- `saiph cat`, `ls`, `log`, `diff` and `show` take `--repo owner/name` and + a revision, and call `ReadFile`, `ReadTree`, `Log` and `Diff` directly. +- `saiph commit --repo owner/name --parent --write =@ + --delete ` uploads and calls `Commit` with no checkout. +- The merge queue and review call `saiph.v1` directly + ([merge-queue](merge-queue.md), [review](review.md)). + +### Tokens and Compass's issuance + +Store OQ-6 (a): `saiph-server` verifies JWTs from configured issuers. +This section fixes the claims and how Compass mints them. + +- **Claims.** `iss` (equals `Issuer.URL`), `aud` (equals + `Issuer.Audience`, default `saiph`), `sub` (`account:` or + `service:`, which sets `Principal.Kind` and `Principal.ID`), + `tenant` (`Principal.Tenant`), `scope` (space-separated, as RFC 8693 + §4.2 defines it), `exp` and `iat`, plus `agent` (true on agent-session + tokens; [review](review.md) adds it as `Claims.Agent` and `Principal.Agent`). + `exp` and `iat` are required, as Compass requires them + (`RunnerVerifier.Verify`: "go-jose skips an absent time claim"). The + lifetime `exp - iat` is at most one hour. Algorithms are ES256 and + RS256. +- **Scopes.** `read`, `write`, `land`, `provision` and `import` (store + S11). A scope caps what the token can do; the repo grant (`SetGrant`) + still decides per repo. `EnsureTenant` needs `provision`; `CommitBatch` + and an import `Commit` need `import`. +- **Keys.** An issuer with no static `Keys` is discovered from its URL: + `/.well-known/openid-configuration`, then its `jwks_uri`. The keys + refresh hourly, on an unknown `kid` at most every five minutes, and the + last good set is kept for 24 hours, as Compass's runner verifier does. +- **Kubernetes issuers.** An issuer with `Kubernetes: true` maps + `sub = system:serviceaccount::` to + `Principal{Kind: service, ID: "k8s//", Tenant: Issuer.Tenant, + Scopes: ["read"]}`, the subject agreed with [ci-cd](ci-cd.md). The cluster's + service-account issuer must be https and serve discovery; Kubernetes + documents that it is "OIDC compatible, but not strictly OIDC + compliant", which is enough for `jwks_uri`. +- **`Issuer.Tenant`** is required for a Kubernetes issuer (its tokens + carry no `tenant`) and, when set on any issuer, must equal the `tenant` + claim if one is present. +- **Compass mints** with an ES256 key it loads from a file. Its issuer URL + is `https:///saiph`; it serves + `/saiph/.well-known/openid-configuration` and `/saiph/jwks.json`. It + mints for three callers: + - an agent session: `sub = account:`, `agent: true`, + scopes `read write`, 15 minutes, through the Runner (below); + - a human: through the device grant (OQ-8); + - Compass itself: `sub = service:compass`, scopes `provision`, for + `EnsureTenant` and `SetGrant`. +- **Grants.** When `AddAgentRepository` adds a repo that is not on git + (OQ-6), Compass calls `SetGrant(repo, "account:", + REPO_ROLE_WRITE)`; removing the grant sets `REPO_ROLE_UNSPECIFIED`. + `saiph-review` verifies with the same `internal/auth` Verifier, so an + agent's review calls carry its account, as [review](review.md) requires. Review + reads `agent` into `Principal.Agent` ([review](review.md) Store additions). + +### Agent tools on the Saiph backend + +The `vcs_*` tools keep their names and arguments. Each call picks its +backend from the workspace it runs in: a `.saiph/` directory selects +Saiph, a `.jj/` directory selects the RIG-5558 git backend, whose +behavior does not change (ruling 5). + +| Tool | Git backend (RIG-5558, unchanged) | Saiph backend | +| -- | -- | -- | +| `vcs_status` | jj in the clone | `saiphd.v1` `Status` | +| `vcs_commit` | jj commit, bookmark moved | `Snapshot`, then the bookmark set in the view | +| `vcs_workspace` `add`, `forget`, `update-stale` | jj workspace | `CreateWorkspace`, `DeleteWorkspace`, `Checkout` of the view's `wc_parent` | +| `vcs_sync` | fetch and rebase | `GetRef(trunk)`, `Rebase` of the bookmark's stack onto it | +| `vcs_submit` | forge `submit_stack` | `ReviewService.UploadVersions(heads, VERSION_REASON_UPLOAD, request_land = ready)` | +| `vcs_retire` | forge `retire_head` | `AbandonChange` | +| `vcs_pr_status` | forge `stack_status` | `GetChange`, `GetStack` | + +`vcs_submit`'s `issues` go into the message as trailer lines, one +`Issue: ` line per issue with the id verbatim (for example +`Issue: RIG-5821`), because `UploadVersions` has no issue field. + +- **The relay.** The agent holds no token (OQ-5). The Runner's gateway + already serves the container's `agent.sock`; it mounts the Connect + paths of `saiph.v1` (including `ReviewService`) and `saiphd.v1` on the + same socket and forwards them to `saiphd` with the session id set, + refusing a call that + names another session. In a microVM, guestd already forwards + `agent.sock` over vsock, so nothing new crosses the VM boundary. The + agent's `saiph` CLI uses `SAIPH_SOCKET=/run/compass/agent.sock`. +- **The token.** At session start, and five minutes before each token + expires, the Runner calls the new `RunnerService.MintSaiphToken` and + passes the result to `saiphd` with `CreateSession` or + `SetSessionToken`. The Server mints only for a session with at least + one granted repo not on git. +- **The Runner's backend pick.** The Server derives each granted repo's + backend from `repoCutoverState`: `GIT` and `IMPORTING` are + `REPO_BACKEND_GIT`; `NATIVE_MIRRORED` and `NATIVE_ONLY` are + `REPO_BACKEND_SAIPH`; a repo with no cutover row is `REPO_BACKEND_GIT`. + This keeps README O2's `RepoBackend` enum as a derived API value, never + a stored column (agreed with [git-bridge](git-bridge.md)). The session start the + Runner receives lists the repos with their backends. The Runner uses it + to create the `saiphd` session, to decide the egress below, and the + git credential broker skips non-git repos. + +### Egress + +Ruling 4: no GitHub egress for agents on a native repo. + +- A session whose granted repos are all `REPO_BACKEND_SAIPH` loses the + forge hosts: the Runner removes the hosts in its new `--forge-hosts` + flag (default `github.com,api.github.com`) from that session's egress + policy. It gets no `GITHUB_APP_TOKEN` either. A mixed session keeps + them (OQ-4). +- A session needs no egress for Saiph at all: every Saiph call goes over + the socket to `saiphd`, and `saiphd` reaches its tiers from the host. +- A tier that cannot enforce egress (`egressUnenforcer` in + `go/internal/runner/spec.go` in Compass) keeps today's behavior: the + policy is not enforced there, so the rule holds only on enforcing tiers. + +### CI source: the CSI driver + +Agreed with [ci-cd](ci-cd.md). A CI pod gets its source as a CSI inline +ephemeral volume, because Pod Security baseline forbids hostPath +("HostPath volumes must be forbidden") and allows `csi`. + +```yaml +volumes: + - name: src + csi: + driver: saiph.rigel.build + readOnly: false # a writable copy-on-write workspace + volumeAttributes: + repo: owner/name + changeset: "<64 hex>" + sparse: "go/\ndocs/" # newline-separated prefixes; empty = whole tree + mode: virtual # or materialized +``` + +- `saiphd` serves the CSI Node service on + `/var/lib/kubelet/plugins/saiph.rigel.build/csi.sock`, beside a + `node-driver-registrar` sidecar. The `CSIDriver` sets + `attachRequired: false`, `podInfoOnMount: true`, + `volumeLifecycleModes: [Ephemeral]`, `requiresRepublish: true` and + `tokenRequests: [{audience: saiph.rigel.build, + expirationSeconds: 3600}]`. The DaemonSet tolerates + `saiph.rigel.build/ci=true:NoSchedule`. +- Writes go to the session overlay and are deleted at unpublish. A volume + with `readOnly: true` refuses them. +- `NodePublishVolume` reads the pod's token from + `csi.storage.k8s.io/serviceAccount.tokens`, creates a session with it + and a workspace at the target path, and returns when the root is ready + (virtual) or the sparse set is written (materialized). Republish only + replaces the token, since it "should only change the contents rather + than the mount". `NodeUnpublishVolume` deletes the session and its + workspace. +- The server maps the token to `service:k8s//` + (above). CI grants that subject `REPO_ROLE_READ` on the repo. +- **Bazel digests.** `saiphd` sets the xattr `user.saiph.sha256` on each + file from the SHA-256 alias in the store's content meta, so Bazel skips + hashing (`--unix_digest_hash_attribute_name`). The read is + `ObjectService.GetContentMeta`, which [ci-cd](ci-cd.md) adds to the store. + +### Caps and spend + +| Layer | Cap | Enforced by | +| -- | -- | -- | +| Node store | `saiphd --store-bytes` (default 50 GiB) | LRU eviction before each write | +| Session overlay | `saiphd --overlay-bytes` (default 20 GiB) | writes fail with `ENOSPC` | +| Session fetch | `saiphd --session-fetch-bytes-per-hour` (default 20 GiB) | `RESOURCE_EXHAUSTED` to the session | +| Node upstream | `saiphd --upstream-bytes-per-sec` (default 100 MiB/s) | token bucket; callers wait | +| Cache memory | `saiph-cache --memory-bytes` (default 8 GiB), container limit 1.25× that | LRU eviction; the kernel OOM-kills past the limit | +| Cache upstream | `saiph-cache --upstream-bytes-per-sec` (default 200 MiB/s) | token bucket | +| Cache replicas | fixed `replicas`, no autoscaler | the Deployment | +| Request | 16 MiB per call | Connect read limits, as [store](store.md#caps-and-spend) | + +Cross-region traffic is the billed egress: a daemon talks only to its +region's cache, and only a cache miss crosses to the servers. A deployment +that runs `saiph-cache` or `saiphd` on a usage-billed vendor also adds the +vendor's egress cap where one exists, a budget alert that pages Matt +through the billing paging route, and a spend-cap row per layer with +evidence that it enforces. Until then the tier stays off in that +deployment, as the store's caps say. + +### Alternatives considered + +- **jj as the backend or the client** (an earlier draft's question). + Rejected, + and ruling 11 settles it. jj is local-first: its `Backend` trait writes + trees locally (`write_tree`, `write_file` in `lib/src/backend.rs` of + `jj-vcs/jj`), while a Saiph tree "is derived, never uploaded" + ([store](store.md#trees)). Its op log is a repo-wide store of views and + op heads (`OpStore`, `op_heads_store.rs`), which assumes one client sees + every ref; Saiph's views are per workspace and its refs are on the + server. It commits conflicts, which store OQ-5 (a) rejects. And it + carries its git backend and colocation. Saiph takes the ideas (change + ids, op log, undo, auto-rebase) and builds them in its own client. +- **The Sapling client against a Saiph EdenAPI.** Rejected by ruling 12 + and the license constraint. +- **Git clients through the git bridge.** The overview's non-goal: no git + wire compatibility. +- **A Compass proxy for every call.** Store OQ-6 (b): it breaks the tiers. +- **A hostPath socket in CI pods.** Pod Security baseline forbids it. +- **The agent holds its token.** OQ-5 (b). + +### Store additions + +Folded into [store](store.md#api-the-saiphv1-connect-package): +`stored_form`, `Blob.stored`, `GetTreesRequest.omit_decoded`, +`GetRepoResponse.id_reads` and `tenant` (store S6), and the +`internal/auth` issuer, claims, scopes, `MaxTokenLifetime`, +`discoverKeys` and `principal` (store S11). + +### Decisions + +- **D1 — DL-590.** Saiph's client follows Scion's tiers: the `saiph` CLI, one + `saiphd` per node, and a regional `saiph-cache`. They answer the same + id-read RPCs as `saiph-server`, tiering is strict, objects travel in + their stored form, and every tier verifies each object's id before it + keeps it. +- **D2 — DL-591.** Fetch is want-only: trees recursively, then blobs, with no have + list. Checkouts are sparse, and either virtual (FUSE through go-fuse, + with an overlay) or materialized. The filesystem monitor is the FUSE + journal or inotify, in `saiphd`. EdenFS is a design reference only. +- **D3 — DL-592.** The working copy sits on one changeset and is committed through + server-side `Commit`. The CLI shows change ids in jj's reverse hex, keeps + a per-workspace op log with undo, keeps bookmarks local, and shows a + smartlog. jj as backend or client is rejected. +- **D4 — DL-593.** Tokens are JWTs with `iss`, `sub`, `aud`, `tenant`, `scope`, + `exp` and `iat`, at most one hour long. Keys are discovered from the + issuer URL, and Kubernetes service-account tokens map to + `service:k8s//` in `Issuer.Tenant`. Compass mints ES256 tokens + for agents, humans (device grant) and itself. +- **D5 — DL-594.** Agents hold no Saiph token. The `vcs_*` tools on a Saiph + workspace go through the Runner's relay to `saiphd`, which adds the + session's token. Each repo's backend is derived from `repoCutoverState` + and never stored. The git backend is unchanged. +- **D6 — DL-595.** A session whose repos are all on Saiph has no GitHub egress and + no GitHub credential. +- **D7 — DL-596.** CI gets source through the CSI driver `saiph.rigel.build` + as an inline ephemeral volume, with the pod's own token. Every tier + ships with a cap per layer, a paging budget alert and spend-cap rows. + +## Plan + +### Global Constraints + +- **Inherit [store](store.md#global-constraints):** public repo, Go 1.26, + `connectrpc.com/connect/v2`, Compass's lint rules, red/green tests, + `buf lint` STANDARD, Scion first, and Sapling and EdenFS as designs only, + never code. +- **Layout.** `cmd/saiph`, `cmd/saiphd`, `cmd/saiph-cache`; + `internal/store`, `internal/fetch`, `internal/tier`, `internal/daemon`, + `internal/wc`, `internal/oplog`, `internal/vfs`, `internal/csi`, + `internal/cli`; `proto/saiphd/v1/daemon.proto`. +- **Dependencies:** `github.com/hanwen/go-fuse/v2` (BSD-3-Clause), + `github.com/fsnotify/fsnotify` (BSD-3-Clause), + `github.com/go-jose/go-jose/v4` (Apache-2.0), + `github.com/go-git/go-git/v5/plumbing/format/gitignore` (Apache-2.0), + `golang.org/x/sync/singleflight`, and for CSI only + `github.com/container-storage-interface/spec` with + `google.golang.org/grpc`, because kubelet speaks gRPC. +- **Compass tasks** (C9, C10) follow Compass's conventions and leave + RIG-5558's proto fields, mirror, migrations and decisions untouched. +- **No Saiph code imports Compass.** Compass reaches Saiph through + `saiph.v1`, `saiphd.v1` and the CLI. + +### Order + +```text +C1 store+fetch ─┬─ C2 tiers+cache ─┬─ C5 saiphd ─┬─ C6 virtual ─┐ +C3 store adds ──┘ │ ├─ C7 CLI ─────┼─ C11 deploy+caps +C4 auth ───────────────────────────┘ ├─ C8 CSI ─────┤ + C9 Compass tokens ─ C10 agent backend ─┘ +``` + +### C1 — Node object store and the want-only fetcher + +A bounded stored-form store on disk and the tree-then-blob fetcher. +Tests: eviction keeps the store under its cap; a fetch of a sparse set asks +only for missing ids under the set; an interrupted fetch resumes without +asking again for stored objects; an object whose bytes do not hash to its +id is refused and not stored; a call never exceeds 1,000 ids or 16 MiB. + +Interfaces: + +```go +// internal/store +type ObjKind uint8 +const (ObjBlob ObjKind = iota + 1; ObjTree; ObjChangeset; ObjChunk) +type Key struct{ Tenant, Repo string; Kind ObjKind; ID object.ID } +func Open(dir string, capBytes int64) (*Store, error) +func (s *Store) Get(k Key) (stored []byte, ok bool, err error) +func (s *Store) Put(k Key, stored []byte) error // caller has verified +func (s *Store) Size() int64 +// internal/fetch +type Access struct{ Bearer, Tenant, Repo string } +type Upstream interface { + Read(ctx context.Context, a Access, kind store.ObjKind, ids []object.ID) (map[object.ID][]byte, error) +} +func New(s *store.Store, up Upstream) *Fetcher +func (f *Fetcher) Changeset(ctx context.Context, a Access, id object.ID) (object.Changeset, object.ID, error) // root tree +func (f *Fetcher) Tree(ctx context.Context, a Access, id object.ID) ([]object.TreeEntry, error) +func (f *Fetcher) Blob(ctx context.Context, a Access, id object.ID) ([]byte, error) +func (f *Fetcher) Prefetch(ctx context.Context, a Access, root object.ID, sparse []string) error +``` + +### C2 — Tier handler, access verdicts and `saiph-cache` + +The read half of `ObjectService` shared by `saiphd` and `saiph-cache`, +the access-verdict cache, and the cache binary. Tests: a caller without +`id_reads` gets `PERMISSION_DENIED` on id reads even for a cached object; +a verdict never outlives the token's `exp`; ten concurrent misses for one +id make one upstream call; a cache with no reachable server fails the read +rather than skipping a tier; methods other than the three id reads and +the pass-through reads answer `UNIMPLEMENTED`. + +Interfaces: + +```go +// internal/tier +type Verdict struct{ Tenant string; IDReads bool; Expires time.Time } +func NewAccessCache(up saiphv1connect.RepoServiceClient, ttl time.Duration) *AccessCache +func (c *AccessCache) Check(ctx context.Context, bearer, repo string) (Verdict, error) +func NewHandler(src fetch.Upstream, local *store.Store, access *AccessCache, pass saiphv1connect.ObjectServiceClient) *Handler +// cmd/saiph-cache +type cacheFlags struct{ Listen, Server string; MemoryBytes, UpstreamBytesPerSec int64 } +``` + +### C3 — Store additions in `saiph-server` + +Folded into store task S6 +([store](store.md#s6--intake-commit-and-the-read-services)), which +carries the tests and interfaces. + +### C4 — Issuer additions in `internal/auth` + +Folded into store task S11 +([store](store.md#s11--server-binary-auth-deployment-and-caps)), which +carries the tests and interfaces. + +### C5 — `saiphd`: sessions, workspaces, monitor, commit and op log + +The daemon binary, `saiphd.v1`, materialized checkouts, the inotify +monitor, `Snapshot` (upload and `Commit`), the op log and pins. Tests: +status after editing two files in a 100,000-file checkout reads two +paths; an inotify overflow triggers one rescan; amend keeps the change id +and restacks descendants; two racing `PutView` calls on one op head leave +one winner; undo restores the prior view; a session cannot name another +session's workspace. + +Interfaces: + +```proto +// proto/saiphd/v1/daemon.proto +syntax = "proto3"; +package saiphd.v1; +import "saiph/v1/saiph.proto"; + +enum CheckoutMode { CHECKOUT_MODE_UNSPECIFIED = 0; CHECKOUT_MODE_MATERIALIZED = 1; CHECKOUT_MODE_VIRTUAL = 2; } +enum FileState { FILE_STATE_UNSPECIFIED = 0; FILE_STATE_ADDED = 1; FILE_STATE_MODIFIED = 2; FILE_STATE_DELETED = 3; FILE_STATE_CONFLICTED = 4; } +message Bookmark { string name = 1; bytes change_id = 2; } +message View { bytes wc_parent = 1; repeated bytes heads = 2; repeated Bookmark bookmarks = 3; bytes trunk = 4; } +message Op { bytes id = 1; bytes parent = 2; int64 time_unix_ms = 3; string command = 4; View view = 5; } +message FileStatus { string path = 1; FileState state = 2; } + +service DaemonService { + rpc CreateSession(CreateSessionRequest) returns (CreateSessionResponse); + rpc SetSessionToken(SetSessionTokenRequest) returns (SetSessionTokenResponse); + rpc DeleteSession(DeleteSessionRequest) returns (DeleteSessionResponse); + rpc CreateWorkspace(CreateWorkspaceRequest) returns (CreateWorkspaceResponse); + rpc DeleteWorkspace(DeleteWorkspaceRequest) returns (DeleteWorkspaceResponse); + rpc Status(StatusRequest) returns (StatusResponse); + rpc Snapshot(SnapshotRequest) returns (SnapshotResponse); + rpc Checkout(CheckoutRequest) returns (CheckoutResponse); + rpc GetView(GetViewRequest) returns (GetViewResponse); + rpc PutView(PutViewRequest) returns (PutViewResponse); + rpc ListOps(ListOpsRequest) returns (ListOpsResponse); +} +message CreateSessionRequest { string session_id = 1; string token = 2; CheckoutMode mode = 3; string root_dir = 4; } +message CreateSessionResponse {} +message SetSessionTokenRequest { string session_id = 1; string token = 2; } +message SetSessionTokenResponse {} +message DeleteSessionRequest { string session_id = 1; } +message DeleteSessionResponse {} +// changeset empty = trunk head; path is under the session's root_dir. +message CreateWorkspaceRequest { string session_id = 1; string repo = 2; string path = 3; bytes changeset = 4; repeated string sparse = 5; bool read_only = 6; } +message CreateWorkspaceResponse { string workspace = 1; View view = 2; } +message DeleteWorkspaceRequest { string session_id = 1; string workspace = 2; } +message DeleteWorkspaceResponse {} +message StatusRequest { string session_id = 1; string workspace = 2; } +message StatusResponse { bytes wc_parent = 1; repeated FileStatus files = 2; } +// amend: keep wc_parent's change id and parents, then Rebase its descendants in the view. +message SnapshotRequest { string session_id = 1; string workspace = 2; string message = 3; bool amend = 4; repeated string paths = 5; } +message SnapshotResponse { bytes changeset = 1; bytes op = 2; repeated saiph.v1.Rewrite rewrites = 3; } +message CheckoutRequest { string session_id = 1; string workspace = 2; bytes changeset = 3; repeated string sparse = 4; } +message CheckoutResponse { bytes op = 1; } +message GetViewRequest { string session_id = 1; string workspace = 2; } +message GetViewResponse { bytes op = 1; View view = 2; } +// FAILED_PRECONDITION when the head op is not expected_op. +message PutViewRequest { string session_id = 1; string workspace = 2; bytes expected_op = 3; View view = 4; string command = 5; } +message PutViewResponse { bytes op = 1; } +message ListOpsRequest { string session_id = 1; string workspace = 2; uint32 limit = 3; } +message ListOpsResponse { repeated Op ops = 1; } +``` + +```go +// internal/daemon +type Config struct { + StoreDir, Socket, CSISocket, Cache, Server, Review string + StoreBytes, OverlayBytes, SessionFetchBytesPerHour, UpstreamBytesPerSec int64 + Prefetch []Prefetch // OQ-3 (a) +} +type Prefetch struct{ Repo string; Sparse []string } +func New(cfg Config) (*Daemon, error) +func (d *Daemon) Serve(ctx context.Context) error +// internal/wc +type Monitor interface { + Changed(since uint64) (paths []string, seq uint64, rescan bool, err error) + Close() error +} +func NewInotify(root string, ignore gitignore.Matcher) (Monitor, error) +// internal/oplog +func Open(path string) (*Log, error) +func (l *Log) Head() (*daemonv1.Op, error) +func (l *Log) Put(expected []byte, v *daemonv1.View, command string) (*daemonv1.Op, error) +func (l *Log) List(limit int) ([]*daemonv1.Op, error) +``` + +### C6 — Virtual checkouts, takeover and the microVM proof + +The FUSE root, the overlay, the journal, kernel-cache invalidation, and +takeover (OQ-2). It also proves virtiofsd serving a FUSE-backed directory +into a microVM guest, and host inotify seeing guest writes (OQ-1). Tests: +reading one file in an unfetched tree fetches one path of trees and one +blob; a write materializes the file into the overlay; the overlay cap +returns `ENOSPC`; a takeover keeps an open file readable across a daemon +upgrade; a guest read of a virtual root through virtiofsd returns the +committed bytes. + +Interfaces: + +```go +// internal/vfs +type Root struct{ /* go-fuse server, workspace, overlay */ } +func Mount(dir string, f *fetch.Fetcher, a fetch.Access, root object.ID, sparse []string, overlayDir string, overlayBytes int64, readOnly bool) (*Root, error) +func (r *Root) Journal() wc.Monitor +func (r *Root) Checkout(root object.ID, sparse []string) error // invalidates kernel entries +// internal/daemon/takeover +func Handoff(ctx context.Context, sock string, roots []*vfs.Root) error // sends FUSE fds and session table +func Resume(ctx context.Context, sock string) ([]*vfs.Root, error) // skips FUSE INIT +``` + +### C7 — The `saiph` CLI + +Commands, revision parsing, reverse-hex change ids, smartlog, undo and +`op` commands, API-only `--repo` mode, `auth login`, and starting the +per-user `saiphd` on a laptop. Tests: a change id and a changeset id with +the same leading bytes parse to different revisions; the shortest unique +prefix is unique in the view; `undo` after `amend` restores the old head; +`commit --repo` with no workspace creates a changeset; smartlog shows only +view heads, their ancestors to trunk, and trunk. + +Interfaces: + +```go +// internal/cli +type Rev struct{ Changeset object.ID; ChangeID [16]byte } +func FormatChangeID(id [16]byte) string // 32 letters, z-k for 0-f +func ParseRev(s string, v *daemonv1.View, lookup func(ctx context.Context, prefix string) ([]Rev, error)) (Rev, error) +func ShortestUnique(ids [][16]byte, id [16]byte) int +func Main(ctx context.Context, args []string, stdout, stderr io.Writer) int +``` + +### C8 — The CSI driver + +The CSI Node service inside `saiphd`, the `CSIDriver` object and the +DaemonSet changes. Tests: publish with each mode yields the requested +tree at the target path; writes land in the overlay and vanish at +unpublish; `readOnly` refuses writes; republish replaces the +token without remounting; unpublish deletes the workspace; a pod whose +service account has no grant gets a publish error and no files. + +Interfaces: + +```go +// internal/csi +const DriverName = "saiph.rigel.build" +const TokenAttr = "csi.storage.k8s.io/serviceAccount.tokens" +type NodeServer struct{ csi.UnimplementedNodeServer /* *daemon.Daemon */ } +func (n *NodeServer) NodePublishVolume(ctx context.Context, r *csi.NodePublishVolumeRequest) (*csi.NodePublishVolumeResponse, error) +func (n *NodeServer) NodeUnpublishVolume(ctx context.Context, r *csi.NodeUnpublishVolumeRequest) (*csi.NodeUnpublishVolumeResponse, error) +func (n *NodeServer) NodeGetInfo(ctx context.Context, r *csi.NodeGetInfoRequest) (*csi.NodeGetInfoResponse, error) +``` + +### C9 — Compass token issuance (in Compass) + +The ES256 minter, the discovery endpoints, `MintSaiphToken`, the device +grant (OQ-8), `EnsureTenant` per Compass tenant, and `SetGrant` from +`AddAgentRepository` (OQ-6). Tests: a minted token verifies with C4's +verifier through discovery; a session with only git repos gets +`FAILED_PRECONDITION`; the token's `sub` is the session's agent account +and its `agent` claim is true; +removing a repo grant calls `SetGrant` with `REPO_ROLE_UNSPECIFIED`. + +Interfaces: + +```go +// go/internal/saiphtoken (in Compass) +func NewMinter(keyFile, issuerURL, audience string, now func() time.Time) (*Minter, error) +// agent sets the agent claim (review.md Store additions); true for agent-session tokens. +func (m *Minter) Mint(tenant, subject string, scopes []string, agent bool, ttl time.Duration) (token string, expires time.Time, err error) +func (m *Minter) DiscoveryHandler() http.Handler // /saiph/.well-known/openid-configuration, /saiph/jwks.json +``` + +```proto +// proto/compass/v1/runner.proto (in Compass) — added to RunnerService +rpc MintSaiphToken(MintSaiphTokenRequest) returns (MintSaiphTokenResponse); +message MintSaiphTokenRequest { string session_id = 1; } +message MintSaiphTokenResponse { string token = 1; int64 expires_unix_ms = 2; } +``` + +### C10 — The Saiph backend for agents (in Compass) + +The derived `RepoBackend`, the repo list in the Runner's session start, +the relay on `agent.sock`, the token refresh, the per-session egress +subtraction, and the Saiph branch of the `vcs_*` tools. Tests: a `.jj/` +workspace runs today's git path unchanged; a `.saiph/` workspace maps each +tool as the table says; an all-Saiph session's policy has no forge hosts +and no `GITHUB_APP_TOKEN`; a mixed session keeps both; a relayed call +naming another session is refused. + +Interfaces: + +```go +// go/server/vcs_backend.go (in Compass) +func repoBackend(state cutoverState, found bool) compassv1.RepoBackend +// go/internal/runner/saiphrelay (in Compass) +func Mount(mux *http.ServeMux, sessionID string, daemonSock string) // saiph.v1 (incl. ReviewService) and saiphd.v1 paths +``` + +```proto +// proto/compass/v1/compass.proto (in Compass): README O2's enum, a derived value +enum RepoBackend { REPO_BACKEND_UNSPECIFIED = 0; REPO_BACKEND_GIT = 1; REPO_BACKEND_SAIPH = 2; } +message SessionRepo { string repo = 1; RepoBackend backend = 2; } +``` + +```ts +// packages/compass-agent/src/vcs-saiph.ts (in Compass) +export type VcsBackend = "git" | "saiph"; +export function selectBackend(cwd: string): Promise; +export function createSaiphVcsBackend(socketPath: string): VcsBackendImpl; +``` + +### C11 — Deployment, caps and alerts + +`deploy/` manifests for the `saiphd` DaemonSet (privileged, `maxSurge: 1`, +the CSI registrar), the `saiph-cache` Deployment and Service +(`trafficDistribution: PreferSameZone`), the caps table's flags, metrics +(`saiphd_store_bytes`, `saiph_cache_memory_bytes`, +`saiph_cache_upstream_bytes_total`), and the budget alert and spend-cap +rows for a usage-billed deployment. Tests: a store at its cap evicts and +stays under it; the upstream rate limit holds under load; the alert fires +on a synthetic breach and stays quiet on a healthy deployment. + +Interfaces: the flags in C2 and C5 and the manifests above. + +## Tasks + +- [ ] C1 — node object store and want-only fetcher +- [ ] C2 — tier handler, access verdicts, `saiph-cache` +- [ ] C3 — store additions (done in store S6) +- [ ] C4 — issuer claims, discovery and Kubernetes mapping (done in store S11) +- [ ] C5 — `saiphd`: sessions, materialized checkouts, monitor, commit, op log +- [ ] C6 — virtual checkouts, takeover, microVM proof +- [ ] C7 — `saiph` CLI: commands, change ids, smartlog, undo, API mode +- [ ] C8 — CSI driver +- [ ] C9 — Compass token issuance (in Compass) +- [ ] C10 — Saiph backend for agent tools, relay and egress (in Compass) +- [ ] C11 — deployment, caps, budget alert and spend-cap rows diff --git a/docs/designs/saiph/decisions/ci-cd/DL-602.md b/docs/designs/saiph/decisions/ci-cd/DL-602.md new file mode 100644 index 0000000..cf6e971 --- /dev/null +++ b/docs/designs/saiph/decisions/ci-cd/DL-602.md @@ -0,0 +1,6 @@ +--- +id: DL-602 +decision: "saiph-ci is a first-party Go service that runs each shard of targets as one Kubernetes Job with backoffLimit 0 and adopts no CI engine." +status: "Active (design, 2026-10-19)" +record: ../../ci-cd.md#decisions +--- diff --git a/docs/designs/saiph/decisions/ci-cd/DL-603.md b/docs/designs/saiph/decisions/ci-cd/DL-603.md new file mode 100644 index 0000000..4073107 --- /dev/null +++ b/docs/designs/saiph/decisions/ci-cd/DL-603.md @@ -0,0 +1,6 @@ +--- +id: DL-603 +decision: "Saiph CI triggers from the store's WatchRefs and review's WatchEvents, each resumed at a cursor stored in the same transaction as the runs it creates." +status: "Active (design, 2026-10-19)" +record: ../../ci-cd.md#decisions +--- diff --git a/docs/designs/saiph/decisions/ci-cd/DL-604.md b/docs/designs/saiph/decisions/ci-cd/DL-604.md new file mode 100644 index 0000000..0df4b6c --- /dev/null +++ b/docs/designs/saiph/decisions/ci-cd/DL-604.md @@ -0,0 +1,6 @@ +--- +id: DL-604 +decision: "A Saiph CI target is a build-graph task id; affected targets come from the store's snapshot Diff, fed to moon on stdin in a skeleton of graph-input files, with all as the barrier." +status: "Active (design, 2026-10-19)" +record: ../../ci-cd.md#decisions +--- diff --git a/docs/designs/saiph/decisions/ci-cd/DL-605.md b/docs/designs/saiph/decisions/ci-cd/DL-605.md new file mode 100644 index 0000000..304213b --- /dev/null +++ b/docs/designs/saiph/decisions/ci-cd/DL-605.md @@ -0,0 +1,6 @@ +--- +id: DL-605 +decision: "Saiph CI policy is one .saiph/ci.yaml, read at the run's base and never from the change under test." +status: "Active (design, 2026-10-19)" +record: ../../ci-cd.md#decisions +--- diff --git a/docs/designs/saiph/decisions/ci-cd/DL-606.md b/docs/designs/saiph/decisions/ci-cd/DL-606.md new file mode 100644 index 0000000..c82eca1 --- /dev/null +++ b/docs/designs/saiph/decisions/ci-cd/DL-606.md @@ -0,0 +1,6 @@ +--- +id: DL-606 +decision: "Each (repo, changeset, target) has one immutable Saiph CI result; CI never retries a target and a Job never retries a pod; INFRA is red, alerts, and is stored on the run only." +status: "Active (design, 2026-10-19)" +record: ../../ci-cd.md#decisions +--- diff --git a/docs/designs/saiph/decisions/ci-cd/DL-607.md b/docs/designs/saiph/decisions/ci-cd/DL-607.md new file mode 100644 index 0000000..433a837 --- /dev/null +++ b/docs/designs/saiph/decisions/ci-cd/DL-607.md @@ -0,0 +1,6 @@ +--- +id: DL-607 +decision: "Saiph postsubmit runs in TAP-style milestones; a red target opens a breakage that keeps it red at trunk until a later trunk changeset passes it; culprit finding is flake-aware Bayesian search that runs each (target, changeset) at most once, and it comments, never reverts." +status: "Active (design, 2026-10-19)" +record: ../../ci-cd.md#decisions +--- diff --git a/docs/designs/saiph/decisions/ci-cd/DL-608.md b/docs/designs/saiph/decisions/ci-cd/DL-608.md new file mode 100644 index 0000000..6717c58 --- /dev/null +++ b/docs/designs/saiph/decisions/ci-cd/DL-608.md @@ -0,0 +1,6 @@ +--- +id: DL-608 +decision: "A Saiph CI flake is two results that disagree on one input key; it opens a flake breakage, and the target is quarantined, not required, until the breakage closes." +status: "Active (design, 2026-10-19)" +record: ../../ci-cd.md#decisions +--- diff --git a/docs/designs/saiph/decisions/ci-cd/DL-609.md b/docs/designs/saiph/decisions/ci-cd/DL-609.md new file mode 100644 index 0000000..9701808 --- /dev/null +++ b/docs/designs/saiph/decisions/ci-cd/DL-609.md @@ -0,0 +1,6 @@ +--- +id: DL-609 +decision: "Saiph CI Jobs get their source from a saiphd CSI inline ephemeral volume and have no git." +status: "Active (design, 2026-10-19)" +record: ../../ci-cd.md#decisions +--- diff --git a/docs/designs/saiph/decisions/ci-cd/DL-610.md b/docs/designs/saiph/decisions/ci-cd/DL-610.md new file mode 100644 index 0000000..a0f2896 --- /dev/null +++ b/docs/designs/saiph/decisions/ci-cd/DL-610.md @@ -0,0 +1,6 @@ +--- +id: DL-610 +decision: "Saiph CD follows Rapid: a release is a protected release/* ref cut from a green milestone, a pick is a rebased trunk changeset, and stages run deploy targets behind human land approvals." +status: "Active (design, 2026-10-19)" +record: ../../ci-cd.md#decisions +--- diff --git a/docs/designs/saiph/decisions/ci-cd/DL-611.md b/docs/designs/saiph/decisions/ci-cd/DL-611.md new file mode 100644 index 0000000..b392562 --- /dev/null +++ b/docs/designs/saiph/decisions/ci-cd/DL-611.md @@ -0,0 +1,6 @@ +--- +id: DL-611 +decision: "Saiph's remote build cache is REAPI with SHA-256 on bb-storage, scoped per repo by instance prefix and written only by postsubmit; CI compute is capped by a namespace ResourceQuota, a per-repo shard limit and a budget alert that pages Matt." +status: "Active (design, 2026-10-19)" +record: ../../ci-cd.md#decisions +--- diff --git a/docs/designs/saiph/decisions/client/DL-590.md b/docs/designs/saiph/decisions/client/DL-590.md new file mode 100644 index 0000000..f8b2138 --- /dev/null +++ b/docs/designs/saiph/decisions/client/DL-590.md @@ -0,0 +1,6 @@ +--- +id: DL-590 +decision: "Saiph's client follows Scion's tiers: the saiph CLI, one saiphd per node and a regional saiph-cache answer the same id-read RPCs as saiph-server, tiering is strict, objects travel in their stored form, and every tier verifies each object's id before it keeps it." +status: "Active (design, 2026-10-19)" +record: ../../client.md#decisions +--- diff --git a/docs/designs/saiph/decisions/client/DL-591.md b/docs/designs/saiph/decisions/client/DL-591.md new file mode 100644 index 0000000..e381ae4 --- /dev/null +++ b/docs/designs/saiph/decisions/client/DL-591.md @@ -0,0 +1,6 @@ +--- +id: DL-591 +decision: "Saiph fetch is want-only (trees recursively, then blobs, with no have list); checkouts are sparse and either virtual (FUSE through go-fuse, with an overlay) or materialized; the filesystem monitor is the FUSE journal or inotify in saiphd; EdenFS is a design reference only." +status: "Active (design, 2026-10-19)" +record: ../../client.md#decisions +--- diff --git a/docs/designs/saiph/decisions/client/DL-592.md b/docs/designs/saiph/decisions/client/DL-592.md new file mode 100644 index 0000000..0e3e155 --- /dev/null +++ b/docs/designs/saiph/decisions/client/DL-592.md @@ -0,0 +1,6 @@ +--- +id: DL-592 +decision: "A Saiph working copy sits on one changeset and is committed through server-side Commit; the saiph CLI shows change ids in jj's reverse hex, keeps a per-workspace op log with undo, keeps bookmarks local and shows a smartlog; jj as backend or client is rejected." +status: "Active (design, 2026-10-19)" +record: ../../client.md#decisions +--- diff --git a/docs/designs/saiph/decisions/client/DL-593.md b/docs/designs/saiph/decisions/client/DL-593.md new file mode 100644 index 0000000..b52bb27 --- /dev/null +++ b/docs/designs/saiph/decisions/client/DL-593.md @@ -0,0 +1,6 @@ +--- +id: DL-593 +decision: "Saiph tokens are JWTs with iss, sub, aud, tenant, scope, exp and iat, at most one hour long; keys are discovered from the issuer URL; Kubernetes service-account tokens map to service:k8s// in Issuer.Tenant; Compass mints ES256 tokens for agents, humans (device grant) and itself." +status: "Active (design, 2026-10-19)" +record: ../../client.md#decisions +--- diff --git a/docs/designs/saiph/decisions/client/DL-594.md b/docs/designs/saiph/decisions/client/DL-594.md new file mode 100644 index 0000000..a971af1 --- /dev/null +++ b/docs/designs/saiph/decisions/client/DL-594.md @@ -0,0 +1,6 @@ +--- +id: DL-594 +decision: "Agents hold no Saiph token: the vcs_* tools on a Saiph workspace go through the Runner's relay to saiphd, which adds the session's token; each repo's backend is derived from repoCutoverState and never stored; the git backend is unchanged." +status: "Active (design, 2026-10-19)" +record: ../../client.md#decisions +--- diff --git a/docs/designs/saiph/decisions/client/DL-595.md b/docs/designs/saiph/decisions/client/DL-595.md new file mode 100644 index 0000000..bf92c9b --- /dev/null +++ b/docs/designs/saiph/decisions/client/DL-595.md @@ -0,0 +1,6 @@ +--- +id: DL-595 +decision: "A Compass session whose repos are all on Saiph has no GitHub egress and no GitHub credential." +status: "Active (design, 2026-10-19)" +record: ../../client.md#decisions +--- diff --git a/docs/designs/saiph/decisions/client/DL-596.md b/docs/designs/saiph/decisions/client/DL-596.md new file mode 100644 index 0000000..68cd69e --- /dev/null +++ b/docs/designs/saiph/decisions/client/DL-596.md @@ -0,0 +1,6 @@ +--- +id: DL-596 +decision: "Saiph CI gets source through the CSI driver saiph.rigel.build as an inline ephemeral volume with the pod's own token, and every Saiph client tier ships with a cap per layer, a paging budget alert and spend-cap rows." +status: "Active (design, 2026-10-19)" +record: ../../client.md#decisions +--- diff --git a/docs/designs/saiph/decisions/git-bridge/DL-621.md b/docs/designs/saiph/decisions/git-bridge/DL-621.md new file mode 100644 index 0000000..c32ad11 --- /dev/null +++ b/docs/designs/saiph/decisions/git-bridge/DL-621.md @@ -0,0 +1,6 @@ +--- +id: DL-621 +decision: "Saiph's git export is a pure function and import round-trips: every imported git commit exports byte-identical, and the bridge proves it per batch and never force-pushes." +status: "Active (design, 2026-10-19)" +record: ../../git-bridge.md#decisions +--- diff --git a/docs/designs/saiph/decisions/git-bridge/DL-622.md b/docs/designs/saiph/decisions/git-bridge/DL-622.md new file mode 100644 index 0000000..0fb09e1 --- /dev/null +++ b/docs/designs/saiph/decisions/git-bridge/DL-622.md @@ -0,0 +1,6 @@ +--- +id: DL-622 +decision: "After a repo's switch to Saiph, GitHub main has one writer: a ruleset whose only bypass actor is the bridge's own App enforces it, and the bridge checks it before switching." +status: "Active (design, 2026-10-19)" +record: ../../git-bridge.md#decisions +--- diff --git a/docs/designs/saiph/decisions/git-bridge/DL-623.md b/docs/designs/saiph/decisions/git-bridge/DL-623.md new file mode 100644 index 0000000..b494124 --- /dev/null +++ b/docs/designs/saiph/decisions/git-bridge/DL-623.md @@ -0,0 +1,6 @@ +--- +id: DL-623 +decision: "Saiph git import is trunk-only, first-parent diffed and refusing: it goes through store intake with CommitBatch, and unrepresentable history keeps the repo on git." +status: "Active (design, 2026-10-19)" +record: ../../git-bridge.md#decisions +--- diff --git a/docs/designs/saiph/decisions/git-bridge/DL-624.md b/docs/designs/saiph/decisions/git-bridge/DL-624.md new file mode 100644 index 0000000..f426391 --- /dev/null +++ b/docs/designs/saiph/decisions/git-bridge/DL-624.md @@ -0,0 +1,6 @@ +--- +id: DL-624 +decision: "Saiph cutover is a per-repo state machine in Compass (GIT, IMPORTING, NATIVE_MIRRORED, NATIVE_ONLY); the backend is derived, and rollback from NATIVE_MIRRORED loses no landed history." +status: "Active (design, 2026-10-19)" +record: ../../git-bridge.md#decisions +--- diff --git a/docs/designs/saiph/decisions/git-bridge/DL-625.md b/docs/designs/saiph/decisions/git-bridge/DL-625.md new file mode 100644 index 0000000..acf73ca --- /dev/null +++ b/docs/designs/saiph/decisions/git-bridge/DL-625.md @@ -0,0 +1,6 @@ +--- +id: DL-625 +decision: "After cutover, allowlisted same-repo GitHub PRs become external Saiph review changes and land by merge; other PRs are closed with a comment." +status: "Active (design, 2026-10-19)" +record: ../../git-bridge.md#decisions +--- diff --git a/docs/designs/saiph/decisions/git-bridge/DL-626.md b/docs/designs/saiph/decisions/git-bridge/DL-626.md new file mode 100644 index 0000000..99ffc5a --- /dev/null +++ b/docs/designs/saiph/decisions/git-bridge/DL-626.md @@ -0,0 +1,6 @@ +--- +id: DL-626 +decision: "The Saiph git bridge is its own binary, schema and GitHub App; it mints its own tokens and leaves RIG-5558 unchanged." +status: "Active (design, 2026-10-19)" +record: ../../git-bridge.md#decisions +--- diff --git a/docs/designs/saiph/decisions/merge-queue/DL-612.md b/docs/designs/saiph/decisions/merge-queue/DL-612.md new file mode 100644 index 0000000..28da52f --- /dev/null +++ b/docs/designs/saiph/decisions/merge-queue/DL-612.md @@ -0,0 +1,6 @@ +--- +id: DL-612 +decision: "saiph-queue is its own service with the schema saiph_queue and one planner per repo under a lease; it reads review and is never called by it." +status: "Active (design, 2026-10-19)" +record: ../../merge-queue.md#decisions +--- diff --git a/docs/designs/saiph/decisions/merge-queue/DL-613.md b/docs/designs/saiph/decisions/merge-queue/DL-613.md new file mode 100644 index 0000000..1892dab --- /dev/null +++ b/docs/designs/saiph/decisions/merge-queue/DL-613.md @@ -0,0 +1,6 @@ +--- +id: DL-613 +decision: "Saiph merge-queue lanes are not configured: two entries share a lane when their affected targets or changed paths meet; all, graph_changed and a .saiphqueue change are barriers." +status: "Active (design, 2026-10-19)" +record: ../../merge-queue.md#decisions +--- diff --git a/docs/designs/saiph/decisions/merge-queue/DL-614.md b/docs/designs/saiph/decisions/merge-queue/DL-614.md new file mode 100644 index 0000000..2240f34 --- /dev/null +++ b/docs/designs/saiph/decisions/merge-queue/DL-614.md @@ -0,0 +1,6 @@ +--- +id: DL-614 +decision: "A Saiph merge-queue batch is tested on a speculative projection of its lane, with a Zuul window per lane and at most batch_max entries." +status: "Active (design, 2026-10-19)" +record: ../../merge-queue.md#decisions +--- diff --git a/docs/designs/saiph/decisions/merge-queue/DL-615.md b/docs/designs/saiph/decisions/merge-queue/DL-615.md new file mode 100644 index 0000000..3e63705 --- /dev/null +++ b/docs/designs/saiph/decisions/merge-queue/DL-615.md @@ -0,0 +1,6 @@ +--- +id: DL-615 +decision: "The Saiph merge queue never reruns a target; a failure is judged by CI's flake quarantine, successor probes with flake rates as priors, and prefix bisection on changesets CI has not tested; INFRA holds a batch until Requeue." +status: "Active (design, 2026-10-19)" +record: ../../merge-queue.md#decisions +--- diff --git a/docs/designs/saiph/decisions/merge-queue/DL-616.md b/docs/designs/saiph/decisions/merge-queue/DL-616.md new file mode 100644 index 0000000..97841ef --- /dev/null +++ b/docs/designs/saiph/decisions/merge-queue/DL-616.md @@ -0,0 +1,6 @@ +--- +id: DL-616 +decision: "A Saiph merge-queue batch lands by one UpdateRef with chain; a lost CAS rebases the batch onto the new trunk and retests only targets the trunk delta touches, with no retest when the graph is unchanged and they do not meet; at most 100 attempts." +status: "Active (design, 2026-10-19)" +record: ../../merge-queue.md#decisions +--- diff --git a/docs/designs/saiph/decisions/merge-queue/DL-617.md b/docs/designs/saiph/decisions/merge-queue/DL-617.md new file mode 100644 index 0000000..3f2ec6e --- /dev/null +++ b/docs/designs/saiph/decisions/merge-queue/DL-617.md @@ -0,0 +1,6 @@ +--- +id: DL-617 +decision: "Saiph merge-queue auto-rebase is Rebase with drop_empty; the queue fixes only trunk conflicts, with a deterministic classifier first and then one chat-completion call whose answer may only choose lines the two sides wrote, written by Rebase with resolutions and uploaded as a CONFLICT_FIX version that passes the same CI." +status: "Active (design, 2026-10-19)" +record: ../../merge-queue.md#decisions +--- diff --git a/docs/designs/saiph/decisions/merge-queue/DL-618.md b/docs/designs/saiph/decisions/merge-queue/DL-618.md new file mode 100644 index 0000000..751094f --- /dev/null +++ b/docs/designs/saiph/decisions/merge-queue/DL-618.md @@ -0,0 +1,6 @@ +--- +id: DL-618 +decision: "A Saiph merge-queue eject is one unresolved change-level comment with a typed reason and the conflict or failed targets attached, with the owner in attention." +status: "Active (design, 2026-10-19)" +record: ../../merge-queue.md#decisions +--- diff --git a/docs/designs/saiph/decisions/merge-queue/DL-619.md b/docs/designs/saiph/decisions/merge-queue/DL-619.md new file mode 100644 index 0000000..fb0e0a8 --- /dev/null +++ b/docs/designs/saiph/decisions/merge-queue/DL-619.md @@ -0,0 +1,6 @@ +--- +id: DL-619 +decision: "Saiph merge-queue fixer spend has a per-call token cap and per-repo and per-tenant daily micro-USD caps, reserved before each call, plus a budget alert that pages Matt; the fixer stays off until they exist." +status: "Active (design, 2026-10-19)" +record: ../../merge-queue.md#decisions +--- diff --git a/docs/designs/saiph/decisions/merge-queue/DL-620.md b/docs/designs/saiph/decisions/merge-queue/DL-620.md new file mode 100644 index 0000000..d30de2e --- /dev/null +++ b/docs/designs/saiph/decisions/merge-queue/DL-620.md @@ -0,0 +1,6 @@ +--- +id: DL-620 +decision: "Saiph merge-queue policy is .saiphqueue at the trunk head, with a per-repo gate of pre (default) or post; operator caps can only be lowered." +status: "Active (design, 2026-10-19)" +record: ../../merge-queue.md#decisions +--- diff --git a/docs/designs/saiph/decisions/overview/DL-580.md b/docs/designs/saiph/decisions/overview/DL-580.md new file mode 100644 index 0000000..cc4752b --- /dev/null +++ b/docs/designs/saiph/decisions/overview/DL-580.md @@ -0,0 +1,6 @@ +--- +id: DL-580 +decision: "Two backends sit behind one agent vcs_* tool surface, chosen per repo: the RIG-5558 git backend ships unchanged for repos on GitHub, and Saiph serves Saiph repos through the saiph CLI, where agents have no GitHub egress and only trunk is mirrored to GitHub as derived git commits; the tools, the Runner backend pick and the web UI hooks stay in Compass, and Saiph stays in its own repo." +status: "Active (design, 2026-10-18)" +record: ../../README.md#decisions +--- diff --git a/docs/designs/saiph/decisions/review/DL-597.md b/docs/designs/saiph/decisions/review/DL-597.md new file mode 100644 index 0000000..aac1bed --- /dev/null +++ b/docs/designs/saiph/decisions/review/DL-597.md @@ -0,0 +1,6 @@ +--- +id: DL-597 +decision: "A Saiph review change is named by the store's 16-byte change_id; a version is an immutable changeset plus its base, numbered densely from 1; review owns versions and the store keeps none." +status: "Active (design, 2026-10-19)" +record: ../../review.md#decisions +--- diff --git a/docs/designs/saiph/decisions/review/DL-598.md b/docs/designs/saiph/decisions/review/DL-598.md new file mode 100644 index 0000000..abe5e61 --- /dev/null +++ b/docs/designs/saiph/decisions/review/DL-598.md @@ -0,0 +1,6 @@ +--- +id: DL-598 +decision: "The Saiph stack graph is a DAG of open-change dependencies taken from changeset parents and kept apart from version history; each change is reviewed and approved alone and lands with or after what it sits on." +status: "Active (design, 2026-10-19)" +record: ../../review.md#decisions +--- diff --git a/docs/designs/saiph/decisions/review/DL-599.md b/docs/designs/saiph/decisions/review/DL-599.md new file mode 100644 index 0000000..9297a14 --- /dev/null +++ b/docs/designs/saiph/decisions/review/DL-599.md @@ -0,0 +1,6 @@ +--- +id: DL-599 +decision: "Saiph approval is four gates (LGTM, owner coverage from the owners list in .saiphacl, complete required checks, and .saiphreview rules); approvals carry only across message-only versions, trivial rebases and conflict fixes the policy allows." +status: "Active (design, 2026-10-19)" +record: ../../review.md#decisions +--- diff --git a/docs/designs/saiph/decisions/review/DL-600.md b/docs/designs/saiph/decisions/review/DL-600.md new file mode 100644 index 0000000..2841dd9 --- /dev/null +++ b/docs/designs/saiph/decisions/review/DL-600.md @@ -0,0 +1,6 @@ +--- +id: DL-600 +decision: "Saiph review is its own service, saiph-review, serving saiph.v1.ReviewService from schema saiph_review; it learns of landings from the trunk ref log and never calls the queue, which pulls land units." +status: "Active (design, 2026-10-19)" +record: ../../review.md#decisions +--- diff --git a/docs/designs/saiph/decisions/review/DL-601.md b/docs/designs/saiph/decisions/review/DL-601.md new file mode 100644 index 0000000..dd5b2a7 --- /dev/null +++ b/docs/designs/saiph/decisions/review/DL-601.md @@ -0,0 +1,6 @@ +--- +id: DL-601 +decision: "Compass keeps the Saiph review UI and agent tools, proxies ReviewService for the browser with a token it mints for the user, and tells an agent it is its turn with a home-channel message." +status: "Active (design, 2026-10-19)" +record: ../../review.md#decisions +--- diff --git a/docs/designs/saiph/decisions/store/DL-578.md b/docs/designs/saiph/decisions/store/DL-578.md new file mode 100644 index 0000000..17cc849 --- /dev/null +++ b/docs/designs/saiph/decisions/store/DL-578.md @@ -0,0 +1,6 @@ +--- +id: DL-578 +decision: "Saiph's store follows Scion: changesets hold ordered parents, a 16-byte change id and the files changed against the first parent, are created only by the server, and are encoded as deterministic CBOR with BLAKE2b-256 ids keyed per type; file contents live zstd-compressed in object storage; changesets, derived data, refs, the ref log and pins live in a metadata database (YugabyteDB YSQL for many servers, Postgres 16 or later on one node, one shared schema saiph); a ref moves only by a single-row compare-and-swap." +status: "Active (design, 2026-10-18)" +record: ../../store.md#decisions +--- diff --git a/docs/designs/saiph/decisions/store/DL-579.md b/docs/designs/saiph/decisions/store/DL-579.md new file mode 100644 index 0000000..41bdd29 --- /dev/null +++ b/docs/designs/saiph/decisions/store/DL-579.md @@ -0,0 +1,6 @@ +--- +id: DL-579 +decision: "Saiph's server creates commits, rebases and merges; a conflict is a typed result (paths, jj-ordered terms, hunks) and is never stored; UpdateRef advances a ref by an ordered chain of changesets in one compare-and-swap; derived data (trees, commit graph, ACL manifest eager; path index and git commits on trunk; blame lazy) is a pure function of a changeset and its parents, written back to the database." +status: "Active (design, 2026-10-18)" +record: ../../store.md#decisions +--- diff --git a/docs/designs/saiph/decisions/store/DL-581.md b/docs/designs/saiph/decisions/store/DL-581.md new file mode 100644 index 0000000..a40d085 --- /dev/null +++ b/docs/designs/saiph/decisions/store/DL-581.md @@ -0,0 +1,6 @@ +--- +id: DL-581 +decision: "Saiph is its own repo and product, with its own Connect package saiph.v1, its own schema and tenant GUC, and per-tenant object keys; it reads the caller from an auth.Principal built from a verified issuer token (Compass is one issuer); per-directory ACLs are in-tree .saiphacl files, and a denied read answers NOT_FOUND." +status: "Active (design, 2026-10-18)" +record: ../../store.md#decisions +--- diff --git a/docs/designs/saiph/git-bridge.md b/docs/designs/saiph/git-bridge.md new file mode 100644 index 0000000..86d1865 --- /dev/null +++ b/docs/designs/saiph/git-bridge.md @@ -0,0 +1,801 @@ +# Saiph: git bridge + +Tracker: RIG-5665 + +> Freezes on merge; later changes supersede by citation, never rewrite +> (convention: the `docs/designs/agent/compass-agent-config-delivery/design.md` +> header). Unless a path says otherwise, every code path is in the +> `RigelBuild/saiph` repo, which does not exist yet. A Compass path says +> "in Compass". + +Ledger-impact: D1–D6 are filed as DL-621..DL-626. The RIG-5558 decisions +(DL-535..DL-538, DL-561) and DL-052 stay whole: they govern the git +backend, which this record does not change. + +**Named decisions this record introduces:** D1–D6 in [Decisions](#decisions). + +Length: Matt lifted the length cap for this set (ruling 6 in the +[overview](README.md#rulings)). The prose sits mostly in Approach, which +fixes the export mapping, the push rules, the import mapping and the +cutover steps. + +This record designs Saiph's git interop: trunk export to GitHub, history +import from GitHub, and the per-repo move from the RIG-5558 git backend to +Saiph. It builds on the entities and RPCs of [store](store.md) and the +rulings in the [overview](README.md#rulings). + +## Open Questions + +The record is written against each recommendation below. OQ-2 and OQ-4 +depend on OQ-1; the others stand alone. + +### OQ-1 — What happens to GitHub PRs after cutover? (load-bearing) + +After cutover, GitHub's default branch takes writes only from the bridge +([Cutover](#per-repo-cutover)). Bots still open PRs on the mirror. The +Compass repo has two that matter. release-please opens a Release PR on +each push to main (`release-pr` in `.github/workflows/release.yml` in +Compass), and it tags a release only when that PR is merged +([release-please README](https://github.com/googleapis/release-please#whats-a-release-pr): +"When the Release PR is merged, release-please takes the following +steps", then "Tags the commit with the version number"). Renovate PRs come +back when closed: +["This PR will be recreated if closed unmerged"](https://github.com/renovatebot/renovate/blob/main/docs/usage/key-concepts/pull-requests.md#immortal-prs). + +- (a) **Refuse all.** The bridge comments on every PR with the Saiph path + and closes it. Release automation and dependency updates must be + rebuilt on Saiph before the repo moves. Renovate has no Saiph platform, + so updates stop. +- (b) **Import allowlisted PRs, land by rebase.** The bridge imports a PR + from an allowlisted author as a review change. The merge queue lands it + as usual, with new ids, and the bridge closes the PR with the landed + commit. Renovate works. release-please does not: its PR closes unmerged, + so it never tags. +- (c) **Import allowlisted PRs, land by merge.** As (b), but the queue + lands the change as a two-parent merge whose second parent is the PR + head, exported with its original id. GitHub then marks the PR merged: + "A pull request can be marked as merged if its head branch commits + become reachable from the base branch outside that pull request" + ([GitHub docs](https://docs.github.com/en/pull-requests/reference/pull-request-merges#indirect-merges)). + release-please accepts merges ("Both squash-merge and merge commits work + with Release PRs"). It needs a land-by-merge mode in the merge queue + ([Contracts with sibling records](#contracts-with-sibling-records)). + +**Recommendation: (c).** Both bots keep working with no change to their +setup, and GitHub main still has one writer. Every imported PR goes +through Saiph review, CI and the queue like any change. (b) is the +fallback if the merge queue does not take the land-by-merge mode. + +One risk is open. release-please finds a release by the merged PR's +merge commit (`mergeCommitOid: pullRequest.mergeCommit?.oid` in +`src/github.ts`, and `src/manifest.ts` skips a commit with a "missing +merged pull request"). GitHub's docs do not say whether an indirect merge +sets `mergeCommit`. Task B6 tests it on a scratch repo before the Compass +repo leaves `GIT`. If it is not set, release-please never tags, (c) gives +it nothing over (b), and this question reopens. + +### OQ-2 — What happens to PRs from authors not on the allowlist? (depends on OQ-1) + +The Compass mirror is public, so strangers can open PRs. + +- (a) The bridge comments once with the contribution path and closes the + PR. +- (b) The bridge imports it as a review change with no owner, and a human + adopts it. CI then runs a stranger's code, which [ci-cd](ci-cd.md) must sandbox + first. +- (c) The bridge ignores it. The PR stays open and can never merge. + +**Recommendation: (a) now, (b) when [ci-cd](ci-cd.md) runs untrusted +presubmits.** A PR that can never merge should not look open. + +### OQ-3 — What does import do with history Saiph cannot hold? (load-bearing) + +Three git shapes have no Saiph form. A gitlink (mode `160000`, a +submodule) has no entry kind ([store](store.md#changeset): file, +executable, symlink). A path that is not UTF-8 fails the store's path +rule. A non-canonical tree, such as a zero-padded mode, derives a +different tree id, so its commit cannot round-trip (D1). + +- (a) **Refuse.** Import stops at the first such commit and reports every + one it finds, with commit and path. That repo stays on the git backend. +- (b) **Represent.** Add a gitlink kind (Mononoke has one: `GitSubmodule` + in `eden/mononoke/mononoke_types/src/file_change.rs`), raw path bytes, + and raw tree bytes as store additions. +- (c) **Rewrite.** Import a cleaned history and force-push it to GitHub at + cutover. Every SHA, tag, fork and signature breaks. + +**Recommendation: (a).** The Compass checkout has no `.gitmodules` file. +(b) can come later as new `FileChange` kinds, which do not change old +ids. (c) breaks D1. + +### OQ-4 — What happens to open GitHub PRs at the switch? (depends on OQ-1) + +At the switch, agents and humans may have open PRs on the git backend. + +- (a) **Import.** The bridge imports each open PR whose head is in the same + repo through the OQ-1 PR path, and closes it with a link to the new + change. Compass names the owner from its PR ownership rows + (`forge_authored_artifacts`, the RIG-5558 table). +- (b) **Close and resubmit**, as RIG-5558 did for jj-vine PRs + (`agent-vcs-tools.md` OQ-5). Each owner re-creates the work on Saiph. + +**Recommendation: (a).** It reuses the inbound path, and no work is lost. +PRs from forks get (b). + +## Problem / Intent + +Repos that move to Saiph already live on GitHub, with years of history and +people who read them there. Matt ruled that GitHub keeps a trunk-only +mirror for migration and public visibility, and that the Compass repo stays +mirrored (rulings 2 and 4). This record designs that bridge: deterministic +git commits from changesets, a push of trunk to GitHub, a full import of +git history into changesets, and a reversible, per-repo cutover from the +RIG-5558 git backend, which ships unchanged (ruling 5). + +## Approach + +### Ground truth + +- `RigelBuild/saiph` does not exist yet, so every Saiph path is a plan. A + Compass path says "in Compass". +- The RIG-5558 git backend ships unchanged and is frozen (ruling 5). The + bridge is new code in the saiph repo. It does not call, wrap or edit + the `Mirror` interface (`go/internal/forge/mirror.go` in Compass) or + any table it uses. +- Saiph imports nothing from Compass. The bridge mints its own GitHub + tokens; Compass reaches the bridge only through `saiph.v1`. + +### Shape + +```mermaid +flowchart LR + GH[(GitHub repo)] -->|fetch, webhooks| BR[saiph-bridge] + BR -->|PutBlobs, CommitBatch, UpdateRef while importing| S[saiph-server] + S -->|KindGitCommit deriver| GC[(saiph.git_commits)] + BR -->|WatchRefs, GetGitCommits, reads| S + BR -->|receive-pack, trunk only| GH + BR -->|UploadVersions IMPORT, AbandonChange| RV[saiph-review] + C[Compass] -->|BridgeService| BR +``` + +The bridge has three parts: + +- **The git-commit deriver** (`internal/gitexport`) runs inside + `saiph-server`. It fills the store's `KindGitCommit` slot + (`derive.Register`, store S9). It is pure, so every replica derives the + same bytes. +- **`saiph-bridge`** (`cmd/saiph-bridge`) is its own binary, like + `saiph-review`, `saiph-queue` and `saiph-ci`. Its rows live in schema + `saiph_bridge`, with migrations in `internal/bridge/migrations/` + numbered from `0001`. Each repo has one worker, held by a lease row + that any replica can take, as the store's derive workers do. The worker + imports, catches up, exports and handles inbound PRs for that repo. +- **Compass's cutover row** (in Compass) says which backend agents use. + +Import builds changesets through the store's one intake path +([Intake](store.md#intake): "Import ([git-bridge](git-bridge.md)) runs inside +`saiph-server`"). The bridge reads git objects and sends structured +inputs with `CommitBatch`. The server checks, encodes and hashes them. +No client-encoded changeset crosses the wire. + +### Identity and tokens + +- **Saiph side.** The bridge is `service:saiph-bridge`. It signs its own + five-minute tokens per tenant, as review does: its issuer is in + `saiph-server`'s `--issuers` file and may mint only that subject. + Scopes are `read write land import` (`import` is a store addition). + Compass grants roles per repo with `SetGrant`: `REPO_ROLE_LAND` while + the repo is `IMPORTING`, then `REPO_ROLE_WRITE` from the switch on. +- **GitHub side.** The bridge is its own GitHub App, not Compass's agent + App and not the CI App. Its permissions are Contents write, Workflows + write, Pull requests write and Metadata read. A separate App lets a + ruleset name exactly one bypass actor (D2). Registering an App is a + manual step with no IaC path (the comment beside the second + `create-github-app-token` step in `.github/workflows/release.yml` in + Compass says the same), so B9 files a human-action issue for it. +- **Minting.** `internal/githubapp` signs an RS256 App JWT and calls + `POST /app/installations/{id}/access_tokens` with the repo and the + permissions above. It is new code; it does not import + `ScopedAppMinter` (`go/internal/forge/githubapp.go` in Compass). + Tokens are cached until five minutes before they expire. +- **Git transport.** The token is the HTTP password with user + `x-access-token`, as GitHub documents: "use the installation access + token as the HTTP password" + ([GitHub docs](https://docs.github.com/en/apps/creating-github-apps/authenticating-with-a-github-app/authenticating-as-a-github-app-installation)). + +### Export mapping + +Export is a pure function of a changeset and its parents' git ids (D1). +The store owns the slot; this record owns the function +([Derived data](store.md#derived-data): "git trees and commits from +changesets, with `extra` keys for round-trip fields"). + +| Git field | From | +| -- | -- | +| blob id | content meta `git_sha1` (lazy, `saiph.derived`) | +| tree entry mode | `ENTRY_KIND_FILE` `100644`, `EXECUTABLE` `100755`, `SYMLINK` `120000`, `TREE` `40000` | +| tree | entries in git order (a directory name sorts as if it ended in `/`); one git tree per Saiph tree node, memoized as lazy kind `KindGitTree` | +| `parent` lines | `parents` in order, each mapped through `saiph.git_commits` | +| `author`, `committer` | `name seconds tz`; seconds are `time_unix_ms / 1000` rounded down; `tz` is `tz_offset_min` as `±hhmm`; `extra["git.author"]` or `extra["git.committer"]` replaces the line when present | +| extra headers | `extra["git.headers"]` verbatim when present; otherwise one `change-id` header | +| message | `message` bytes | + +- **Recursion.** A commit needs its parents' git ids, so deriving a merge + derives its side parents first. A side parent gets a + `saiph.git_commits` row even though it has no `trunk_seq`. +- **`change-id`.** A changeset born in Saiph exports one header, + `change-id `, in jj's form: jj writes `change_id.reverse_hex()` under + `CHANGE_ID_COMMIT_HEADER: &str = "change-id"` + ([`lib/src/git_backend.rs`](https://github.com/jj-vcs/jj/blob/9ff73e7f/lib/src/git_backend.rs)). + A jj user who fetches the mirror sees Saiph's change ids. +- **Round-trip keys.** Import sets `git.headers` on every imported + changeset, empty when the commit had no extra headers. It holds every + header after `committer` (`gpgsig`, `gpgsig-sha256`, `mergetag`, + `encoding`, `change-id` and unknown ones) as raw bytes, in order. Import + sets `git.author` or `git.committer` only when the parsed `Signature` + does not print back to the same line. So an imported commit exports + byte-identical, signatures included, and a Saiph-born one never carries + a `gpgsig`. +- **Mononoke** stores the same round-trip data as typed fields + (`git_extra_headers` in `bonsai.thrift`). Saiph keeps them in `extra`, + which the store does not read. + +### Export push + +Each mirrored repo's worker tails `WatchRefs(names = [trunk_ref])` from +`saiph_bridge.repos.export_log_seq`. + +1. It coalesces trunk moves: at most one push per repo per 10 seconds. + GitHub's "recommended maximum limit is 6 pushes per minute per + repository" + ([repository limits](https://docs.github.com/en/repositories/creating-and-managing-repositories/repository-limits)). +2. It reads `GetGitCommits(changesets = [head])`. It builds the same + commit with `internal/gitexport` from `GetChangesets`, `GetTrees` and + `GetBlobs`, and checks that the ids agree. A disagreement is a bug: the + worker stops and pages. +3. It builds a pack of the objects that are new since the last exported + head: the commits from `Log(heads = [head], stop = [last])` without + `first_parent`, the trees on their changed paths and their new blobs. + It encodes the pack with go-git's `packfile.NewEncoder`. GitHub's push + limit is "enforced at 2GB" (same page); a pack above 1 GiB is split + into several pushes along the first-parent chain. +4. It opens `transport.NewReceivePackSession` and reads + `AdvertisedReferencesContext`. The remote branch must equal the last + exported head. Then it sends one command + `{Name: refs/heads/, Old: last, New: head}` with + `report-status` and reads `ReportStatus`. +5. On success it stores the new head and `export_log_seq` in one + transaction. + +The push is always a fast-forward. The bridge never force-pushes (D1). + +**When GitHub rejects a push**, the worker classifies the report: + +| Rejection | Action | +| -- | -- | +| Remote branch is not the last exported head | Someone else wrote GitHub main. Stop the mirror, page, keep Saiph trunk moving. A human finds the writer; the bridge never overwrites it. | +| Workflows permission (a change under `.github/workflows/`) | The App lacks Workflows write. Stop and page: a config error, not data. | +| Push protection, `GH009: Secrets detected! This push failed.` | Stop at that changeset and page with the rejected commit. Trunk is immutable, so the commit cannot be dropped. A human rotates the secret, then uses GitHub's bypass, which "anyone with write access to the repository" can use "by specifying a bypass reason" ([push protection](https://docs.github.com/en/code-security/secret-scanning/introduction/about-push-protection)). The worker retries every 10 minutes. | +| A ruleset or branch protection rule | Stop and page. The ruleset must name the bridge App as a bypass actor (D2). | +| Network, 5xx, timeout | Retry with backoff from 10 seconds to 10 minutes. | + +A file above 100 MiB cannot reach a push: a mirrored repo cannot raise +`max_file_bytes` above 100 MiB (store, Caps). The bridge sets +`git_mirrored` on the repo, and the store enforces it. + +**Changes under review are never mirrored** (README non-goals, ruling +2). The worker pushes only the trunk branch. Off-trunk changesets reach +GitHub only as side parents of a landed merge (OQ-1 (c)), and by then +they are on trunk. + +Tags are not mirrored either. Saiph has no tags. Tags made on GitHub, +such as release-please's, stay on GitHub and point at mirrored commits. + +### Import mapping + +Import reads one branch, the GitHub default branch, and its full history +(D3). + +1. **Fetch.** `git.PlainCloneContext(ctx, dir, true, &git.CloneOptions{ + URL, Auth, ReferenceName, SingleBranch: true, Tags: git.NoTags})` into a + scratch volume. Catch-up uses `(*Repository).FetchContext`. +2. **Order.** Commits reachable from the branch head, parents first. + Commits already in `saiph.git_commits` (`GetGitCommits(git_commits = + ...)`) are skipped. +3. **Check.** Each commit must fit Saiph (OQ-3). The worker re-encodes + every tree and the commit with `internal/gitobj`; bytes that differ + mean non-canonical. It also refuses a gitlink, a path that is not + UTF-8 or fails `object.CheckPath`, a message that is not UTF-8, and a + file above `max_file_bytes`. The worker records every refusal with its + commit and path and keeps going, so one run lists them all. +4. **Changes.** `changes` is the diff from `tree(parents[0])` to the + commit's tree. A root commit lists every path. A merge is diffed + against its first parent only: the store's tree rule says "A merge + lists every path whose merged result differs from `parents[0]`". Git + records no copies, so `copy_from` stays empty. A path that changes from + a file to a directory lists the file as deleted, which the store's + prefix rule allows. +5. **Blobs.** `FindMissingBlobs`, then `PutBlobs` (at most 4 MiB each) + or `PutChunkedBlob`. Content ids come from `object.BlobID`. +6. **Signatures.** Seconds become `time_unix_ms = seconds * 1000`; the + `±hhmm` offset becomes `tz_offset_min`. `git.author` and + `git.committer` keep a line that does not print back the same. +7. **Change id.** A `change-id` header is read with jj's + `ChangeId::try_from_reverse_hex`. Otherwise the id is jj's + `synthetic_change_id_from_git_commit_id`: commit bytes `[4..20]`, in + reverse order, each byte's bits reversed ("We reverse the bits of the + commit id to create the change id"). A jj user's ids survive, and + every other commit gets the id jj would give it. +8. **Write.** `CommitBatch` with up to 1,000 commits, parents first, then + `UpdateRef(trunk_ref, expected, target, chain)` along the first-parent + chain. A trunk move derives `saiph.git_commits`. +9. **Verify.** For each batch head, the worker waits for + `GetGitCommits` and checks it equals the git commit it came from. A + mismatch is a `ROUND_TRIP` refusal. This proves D1 on every import. + +A refusal stops the import before the refused commit. The repo stays +`IMPORTING` with its refusals in `GetBridgeRepo`, and it cannot switch. + +**Catch-up.** While the repo is `IMPORTING`, GitHub main is still the +writer. The worker fetches on each `push` webhook and every 60 seconds +with a conditional request (a 304 "does not count against your primary +rate limit"). It imports the new commits through steps 2–9. A branch +that moved to a commit that does not descend from the last imported head +(a force-push) is a refusal: the operator restarts the import. + +### Inbound PRs + +After the switch, GitHub PRs follow OQ-1 (c) and OQ-2 (a). + +1. **Webhook.** The bridge serves `POST /github/webhook` and checks + `X-Hub-Signature-256` with the App's webhook secret. It handles + `pull_request` events: `opened`, `reopened`, `synchronize` and + `closed`. +2. **Filter.** A PR whose head is in another repo, or whose author is not + on the repo's `inbound_authors` list, gets one comment with the + contribution path and is closed (OQ-2 (a)). +3. **Import.** The worker fetches `refs/pull//head`. The PR's commits + from the merge base with the mirror to the head go through the import + steps above, off trunk. The k-th commit from the base gets change id + `BLAKE2b-128("saiph.bridge.pr" ‖ repo ‖ PR number ‖ k)` unless it has + a `change-id` header. So a force-push of the PR becomes a new version + of the same change, not a new change. +4. **Upload.** `UploadVersions(heads = [PR head changeset], reason = + VERSION_REASON_IMPORT, owner = "external:github:")`. The + bridge comments once on the PR with the change's review link. +5. **Land.** A human with `REPO_ROLE_WRITE` requests the land, as review + requires. The queue lands the change by merge + ([Contracts with sibling records](#contracts-with-sibling-records)). + The PR head keeps its changeset, so it exports to its original git id. + The next trunk push makes it reachable from GitHub main, and GitHub + marks the PR merged ("A pull request can be marked as merged if its + head branch commits become reachable from the base branch outside that + pull request", + [GitHub docs](https://docs.github.com/en/pull-requests/reference/pull-request-merges#indirect-merges)). +6. **After.** On `EVENT_KIND_LANDED` the bridge waits for the export of + `landed_trunk_seq`. If the PR is not marked merged 5 minutes after + that, it comments with the landed commit and closes the PR. A PR + closed on GitHub, or by its author, gets `AbandonChange`. A change + abandoned in Saiph closes its PR. + +The bridge keeps `(repo, pr number) → change id` in +`saiph_bridge.inbound_prs`. It reads `GetChanges` to reconcile after a +restart. + +### Per-repo cutover + +```mermaid +stateDiagram-v2 + [*] --> GIT + GIT --> IMPORTING: StartImport + IMPORTING --> GIT: CancelImport + IMPORTING --> NATIVE_MIRRORED: Switch + NATIVE_MIRRORED --> GIT: Rollback + NATIVE_MIRRORED --> NATIVE_ONLY: StopMirror + NATIVE_ONLY --> NATIVE_MIRRORED: StartMirror +``` + +| State | Agents' backend | GitHub credential and egress for the repo | Where work lands | Writer of GitHub main | Bridge's Saiph role | +| -- | -- | -- | -- | -- | -- | +| `GIT` (or no row) | git (RIG-5558) | yes | GitHub PRs | GitHub merges | none | +| `IMPORTING` | git (RIG-5558) | yes | GitHub PRs | GitHub merges; the bridge imports | `LAND` | +| `NATIVE_MIRRORED` | Saiph | no | Saiph review and queue | the bridge only | `WRITE` | +| `NATIVE_ONLY` | Saiph | no | Saiph review and queue | nobody | `WRITE` | + +Compass keeps the state in its own table and derives the backend with +`repoCutoverState` and `repoBackend` (`go/server/vcs_backend.go` in +Compass), as agreed with [client](client.md). A session picks its +backend at start, so a session that started on git finishes on git. + +**Switch.** Compass's admin RPC runs these steps in order: + +1. **Precheck.** The bridge is caught up, has no refusal, and the + branch head carries the repo's Saiph policy files. A repo with + `.github/workflows/` needs `.github/workflows/.saiphacl` granting + write to named humans and `service:saiph-bridge` (store + [ACLs](store.md#per-directory-acls)), plus `.saiphreview` and + `.saiphqueue`. These files land on GitHub through the git path before + the switch, so import brings them. +2. **Freeze.** An IaC change adds a GitHub ruleset on the branch with + "Restrict updates" ("Only allow users with bypass permission to update + matching refs") and the bridge App as its only bypass actor. For the + Compass repo, the same change takes the Trunk merge queue's apps off + the bypass list. Agents' GitHub merges now fail; their PRs wait. +3. **Final catch-up.** The bridge fetches once more. `Switch` takes the + expected head and fails if GitHub moved. +4. **`Switch`.** The bridge reads the ruleset with + `GET /repos/{owner}/{repo}/rulesets/{id}` and needs + `current_user_can_bypass = "always"`. It sets + `last exported = last imported`, turns on `git_mirrored` and enters + `NATIVE_MIRRORED`. Exports start. +5. **Compass** grants the bridge `REPO_ROLE_WRITE`, enables the queue + (`EnableRepo`), writes `NATIVE_MIRRORED`, and runs OQ-4 (a) for open + PRs. New sessions get the Saiph backend and no GitHub credential for + the repo (client D6). + +The freeze lasts from step 2 to step 5, minutes for a caught-up repo. + +**Rollback** works from `NATIVE_MIRRORED` while the mirror is exact: +Compass disables the queue (`DisableRepo`), the bridge exports the last +trunk head and checks GitHub main equals it, Compass writes `GIT`, and +an IaC change removes the ruleset. Every landed change is on GitHub +already, so no history is lost. Open Saiph changes stay in Saiph; their +owners re-create them as PRs. A later `StartImport` resumes from +`saiph.git_commits` and imports only what GitHub gained. + +**`NATIVE_ONLY`** stops exports. The Compass repo never takes it +(ruling 4); the bridge refuses `StopMirror` for any repo on its +`--always-mirror` list. `StartMirror` exports the gap, which works +because export is deterministic and nobody else wrote GitHub main. + +**Drift.** Compass compares its row with `GetBridgeRepo` every 5 +minutes and pages on a mismatch. + +### Caps and spend + +| Layer | Usage-billed part | Cap | +| -- | -- | -- | +| Store | blobs and rows from import | the repo's `quota_bytes`; import stops with `RESOURCE_EXHAUSTED` | +| Node | `saiph-bridge` pods | CPU and memory limits in the Deployment | +| Cluster | import scratch volumes | a namespace `ResourceQuota` on `requests.storage`; `--scratch-bytes` (default 20 GiB) per repo | +| Vendor account | cloud disks and compute | the store's existing account cap and billing alert (store S11); B9 adds the bridge's spend-cap rows beside it | +| GitHub | API and pushes | no bill; the bridge caps itself at one push per 10 seconds per repo and one poll per 60 seconds | + +Alerts that page Matt: export lag above 15 minutes, a stopped mirror, +an import refusal on a repo meant to switch, and the store's budget +alert. + +### Contracts with sibling records + +- **Merge queue: land by merge.** For an entry whose change owner is + `external:github:*` on a repo with `git_mirrored`, the queue lands with + `Merge(parents = [projection head, member])` instead of `Rebase`. The + member keeps its changeset and git id. The merge is on the chain; the + member is a side parent. A merge conflict ejects the entry as any + trunk conflict does. [Merge queue](merge-queue.md) has no such mode + today; it is the condition for OQ-1 (c). The queue needs no pause: it + is not enabled until step 5 of the switch, and rollback disables it + first. +- **Review.** As agreed: `UploadVersions` with `VERSION_REASON_IMPORT` + and an `external:github:` owner, `AbandonChange` on an external + change, `GetChanges` fields 1, 3, 10 and 11, and + `WatchEvents(kinds = [EVENT_KIND_LANDED])`. Review's + [landing detection](review.md#landing-detection) already marks a side + member of a landed merge `LANDED`. +- **Client.** `repoCutoverState` returns the four states above; no row + means `GIT`; the backend is derived, never stored. Client OQ-6 (a) + reads the same function. +- **CI.** ci-cd OQ-5 (a) keeps the macOS suite on GitHub Actions on the + mirror. That works because a push with an App token starts workflow + runs, and a `GITHUB_TOKEN` push would not. OQ-2 (b) waits for CI's + untrusted presubmit sandbox. A presubmit secret scan would stop most + `GH009` stalls before they reach trunk; that is CI's call. + +### Store additions + +Folded into [store](store.md#api-the-saiphv1-connect-package): +`CommitBatch`, the `import` scope, `SetGitMirrored` and +`Repo.git_mirrored` (task S6), and `GetGitCommits` and the `KindGitTree` +slot ([Derived data](store.md#derived-data), task S9). + +### Alternatives considered + +- **Reuse the RIG-5558 `Mirror` and its mirror dir.** Rejected: ruling 5 + freezes it, and Saiph imports nothing from Compass. +- **Export only a snapshot, not history.** Rejected: GitHub readers and + forks keep their SHAs only if import and export round-trip. +- **Import with `Commit` one call per commit.** Works, but costs a round + trip per commit. `CommitBatch` is the same intake. +- **Rewrite history at cutover** (OQ-3 (c)). Rejected: it breaks every + SHA. +- **Let GitHub keep a second writer** (a GitHub merge queue beside + Saiph's). Rejected: two writers need a merge on the mirror, and the + mirror would stop being a function of Saiph trunk. +- **Compass mints the bridge's GitHub tokens.** Rejected: the saiph repo + would need a Compass contract for a self-hosted Saiph to mirror. + +### Decisions + +- **D1 (DL-621) — Export is a pure function, and import round-trips.** Every + imported git commit exports byte-identical; the bridge proves it per + batch and never force-pushes. +- **D2 (DL-622) — GitHub main has one writer after the switch.** A ruleset whose + only bypass actor is the bridge's own App enforces it, and the bridge + checks it before switching. +- **D3 (DL-623) — Import is trunk-only, first-parent diffed and refusing.** It + goes through store intake with `CommitBatch`, and unrepresentable + history keeps the repo on git (OQ-3 (a)). +- **D4 (DL-624) — Cutover is a per-repo state machine in Compass.** `GIT`, + `IMPORTING`, `NATIVE_MIRRORED`, `NATIVE_ONLY`; the backend is derived; + rollback from `NATIVE_MIRRORED` loses no landed history. +- **D5 (DL-625) — Allowlisted same-repo GitHub PRs become external review changes + and land by merge** (OQ-1 (c)); other PRs are closed with a comment + (OQ-2 (a)). +- **D6 (DL-626) — The bridge is its own binary, schema and GitHub App.** It mints + its own tokens and leaves RIG-5558 unchanged. + +## Plan + +### Global Constraints + +- Every store Global Constraint applies: public repo, Go 1.26, + `connectrpc.com/connect/v2`, the Compass lint rules, red/green tests on + Postgres 16 and YugabyteDB, `buf lint` STANDARD. +- **No Compass import** in the saiph repo. Compass code changes are only + B8, and they live in Compass. +- **RIG-5558 unchanged.** No task edits `go/internal/forge/mirror.go`, + `forge_mirrors` or `agent-vcs-tools.md` in Compass. +- **No sapling code** (ruling 12). Mononoke and jj are cited for ideas; + jj is Apache-2.0, and the change-id algorithm is written fresh. +- **New dependency:** `github.com/go-git/go-git/v5` (Apache-2.0) at + v5.19.3 or later, for fetch, pack encoding and receive-pack. +- **Migrations.** `saiph_bridge` numbers from `0001`. The Compass table + takes the next free number at implementation time. +- **Never** force-push, push a non-trunk ref, or push without a + `report-status` check. + +### Order + +```text +B1 gitobj+gitexport ─┬─ B2 store additions ─┬─ B4 import ─┬─ B6 inbound PRs ─┐ + │ └─ B5 export ─┤ ├─ B9 deploy+caps +B3 bridge skeleton ──┴────────────────────────────────────┴─ B7 cutover ─────┤ +B8 Compass cutover row (after B3's proto) ────────────────────────────────────┘ +``` + +### B1 — Git objects and the export function + +`internal/gitobj` encodes and decodes git commits and trees and checks +canonical form. `internal/gitexport` maps changesets to commits and +registers `KindGitCommit` and `KindGitTree`. Tests: vectors of real +commits (signed, `mergetag`, octopus merge, `change-id` header, an +odd ident line) round-trip byte-identical; a zero-padded mode is not +canonical; the synthetic change id matches jj's for ten fixed commits; +a Saiph-born changeset exports one `change-id` header. + +Interfaces: + +```go +// internal/gitobj +type Hash [20]byte +type TreeEntry struct{ Mode uint32; Name []byte; ID Hash } +type Commit struct { + Tree Hash + Parents []Hash + Author []byte // raw ident line value + Committer []byte + Headers []byte // raw header lines after committer, in order + Message []byte +} +func ObjectID(kind string, body []byte) Hash +func EncodeTree(entries []TreeEntry) []byte // sorts in git order +func DecodeTree(body []byte) ([]TreeEntry, error) +func EncodeCommit(c Commit) []byte +func DecodeCommit(body []byte) (Commit, error) +func CanonicalTree(body []byte) bool + +// internal/gitexport +const ( + ExtraHeaders = "git.headers" + ExtraAuthor = "git.author" + ExtraCommitter = "git.committer" +) +func CommitFor(cs *object.Changeset, tree gitobj.Hash, parents []gitobj.Hash) (gitobj.Commit, error) +func TreeFor(entries []object.TreeEntry, id func(object.TreeEntry) (gitobj.Hash, error)) ([]gitobj.TreeEntry, error) +func ReverseHex(id [16]byte) string +func ParseReverseHex(s string) ([16]byte, error) +func SyntheticChangeID(commit gitobj.Hash) [16]byte +type CommitDeriver struct{} +func (CommitDeriver) Kind() derive.Kind // derive.KindGitCommit +func (CommitDeriver) Derive(ctx context.Context, env *derive.Env, id object.ID) ([]byte, error) +type TreeDeriver struct{} +func (TreeDeriver) Kind() derive.Kind // derive.KindGitTree +func (TreeDeriver) Derive(ctx context.Context, env *derive.Env, id object.ID) ([]byte, error) +``` + +### B2 — Store additions + +Folded into store tasks S6 and S9 +([store](store.md#s6--intake-commit-and-the-read-services)), which carry +the tests and interfaces. + +### B3 — `saiph-bridge` skeleton + +The binary, flags, schema and migrations, the per-repo lease, the token +issuer, the GitHub App minter and `BridgeService`. Tests: two replicas +hold one repo's lease; a token minted for another subject is refused by +`saiph-server`; a cached GitHub token is reused until five minutes +before expiry. + +Interfaces: + +```go +// cmd/saiph-bridge +type bridgeFlags struct { + DSN, SaiphURL, Issuers, Listen string + IssuerKeyFile, AppKeyFile, WebhookSecretFile, ScratchDir string + AppID, ScratchBytes int64 + AlwaysMirror []string +} +func registerFlags(fs *flag.FlagSet) *bridgeFlags +// internal/githubapp +type Minter struct{ /* app id, key, http client, cache */ } +func NewMinter(appID int64, key *rsa.PrivateKey, hc *http.Client) *Minter +func (m *Minter) Token(ctx context.Context, owner, repo string) (token string, expires time.Time, err error) +// internal/bridge +type State int16 // StateGit = 1, StateImporting = 2, StateNativeMirrored = 3, StateNativeOnly = 4 +func (b *Bridge) Run(ctx context.Context) error // leases repos and runs one worker each +``` + +```proto +// proto/saiph/v1/bridge.proto +service BridgeService { + rpc StartImport(StartImportRequest) returns (StartImportResponse); + rpc CancelImport(CancelImportRequest) returns (CancelImportResponse); + rpc GetBridgeRepo(GetBridgeRepoRequest) returns (GetBridgeRepoResponse) { option idempotency_level = NO_SIDE_EFFECTS; } + rpc Switch(SwitchRequest) returns (SwitchResponse); + rpc Rollback(RollbackRequest) returns (RollbackResponse); + rpc StopMirror(StopMirrorRequest) returns (StopMirrorResponse); + rpc StartMirror(StartMirrorRequest) returns (StartMirrorResponse); +} +enum CutoverState { CUTOVER_STATE_UNSPECIFIED = 0; CUTOVER_STATE_GIT = 1; CUTOVER_STATE_IMPORTING = 2; CUTOVER_STATE_NATIVE_MIRRORED = 3; CUTOVER_STATE_NATIVE_ONLY = 4; } +enum RefusalReason { REFUSAL_REASON_UNSPECIFIED = 0; REFUSAL_REASON_GITLINK = 1; REFUSAL_REASON_PATH_NOT_UTF8 = 2; REFUSAL_REASON_NONCANONICAL = 3; REFUSAL_REASON_MESSAGE_NOT_UTF8 = 4; REFUSAL_REASON_BAD_PATH = 5; REFUSAL_REASON_FILE_TOO_LARGE = 6; REFUSAL_REASON_ROUND_TRIP = 7; REFUSAL_REASON_FORCE_PUSH = 8; } +message Refusal { bytes git_commit = 1; string path = 2; RefusalReason reason = 3; } +message BridgeRepo { string repo = 1; string github_repo = 2; string branch = 3; CutoverState state = 4; bytes imported_git_commit = 5; bytes exported_git_commit = 6; int64 export_lag_ms = 7; repeated Refusal refusals = 8; string last_error = 9; repeated string inbound_authors = 10; } +message StartImportRequest { string repo = 1; string github_repo = 2; string branch = 3; repeated string inbound_authors = 4; } +message StartImportResponse { BridgeRepo repo = 1; } +message CancelImportRequest { string repo = 1; } +message CancelImportResponse { BridgeRepo repo = 1; } +message GetBridgeRepoRequest { string repo = 1; } +message GetBridgeRepoResponse { BridgeRepo repo = 1; } +message SwitchRequest { string repo = 1; bytes expected_git_commit = 2; } +message SwitchResponse { BridgeRepo repo = 1; } +message RollbackRequest { string repo = 1; } +message RollbackResponse { BridgeRepo repo = 1; } +message StopMirrorRequest { string repo = 1; } +message StopMirrorResponse { BridgeRepo repo = 1; } +message StartMirrorRequest { string repo = 1; } +message StartMirrorResponse { BridgeRepo repo = 1; } +``` + +`BridgeService` callers need the `provision` scope (`service:compass`). + +### B4 — Import and catch-up + +Steps 1–9 of [Import mapping](#import-mapping) and catch-up. Tests: a +fixture repo with a merge imports with each merge's `changes` equal to +its first-parent diff; a gitlink, a non-UTF-8 path and a zero-padded +mode each give one refusal naming commit and path; a re-run imports +nothing; a force-push on the branch gives `FORCE_PUSH`; every imported +head's `GetGitCommits` equals its source. + +Interfaces: + +```go +// internal/gitimport +type Source interface { + Commit(h gitobj.Hash) (gitobj.Commit, error) + Tree(h gitobj.Hash) ([]byte, error) + Blob(h gitobj.Hash) (io.ReadCloser, int64, error) +} +func Convert(ctx context.Context, src Source, h gitobj.Hash, maxFileBytes int64) (*saiphv1.CommitRequest, []*saiphv1.Refusal, error) +type Importer struct{ /* saiphv1connect clients, Source */ } +func (im *Importer) Run(ctx context.Context, repo, branch string, head gitobj.Hash) (imported int, refusals []*saiphv1.Refusal, err error) +``` + +### B5 — Export push + +[Export push](#export-push), with the rejection table. Tests against a +local `git http-backend` fixture: a trunk move pushes one fast-forward; +a remote that moved stops the mirror without a push; three moves in 10 +seconds give one push; each rejection kind maps to its action (fixture +hooks print the GitHub messages). + +Interfaces: + +```go +// internal/bridge +type PushErrorKind int // PushRemoteMoved, PushWorkflows, PushSecret, PushRule, PushTransient +type PushError struct{ Kind PushErrorKind; Message string } +func (e *PushError) Error() string +func ClassifyReport(rs *packp.ReportStatus, remote, expected gitobj.Hash) error +func (w *worker) exportOnce(ctx context.Context) error +``` + +### B6 — Inbound PRs + +[Inbound PRs](#inbound-prs) and the OQ-4 (a) import of open PRs at the +switch. Tests: an allowlisted same-repo PR becomes an external change; +a force-push adds a version of the same change; a fork PR is commented +on and closed; a PR closed on GitHub abandons its change; a landed merge +makes a scratch GitHub repo mark the PR merged, and the test records +whether `mergeCommit` is set (OQ-1 risk). + +Interfaces: + +```go +// internal/bridge +func (b *Bridge) ServeWebhook(w http.ResponseWriter, r *http.Request) +func VerifySignature(secret, body []byte, header string) error +func PRChangeID(repo string, pr int64, k int) [16]byte +func (w *worker) importPR(ctx context.Context, pr int64) (changeID [16]byte, err error) +``` + +### B7 — Cutover RPCs + +`Switch`, `Rollback`, `StopMirror`, `StartMirror` and `CancelImport` +with the prechecks of [Per-repo cutover](#per-repo-cutover). Tests: +`Switch` fails without the ruleset bypass, with a refusal, with a missing +`.github/workflows/.saiphacl`, or when GitHub moved; `Rollback` fails +while GitHub main differs from the export; `StopMirror` fails for an +`--always-mirror` repo. + +Interfaces: the `BridgeService` handlers, each of the shape +`func (s *bridgeService) Switch(ctx context.Context, req *saiphv1.SwitchRequest) (*saiphv1.SwitchResponse, error)`. + +### B8 — The cutover row (in Compass) + +A `repo_cutovers` table, `repoCutoverState`, an admin RPC that runs the +switch and rollback steps against `BridgeService`, `SetGrant` and the +queue's `EnableRepo`/`DisableRepo`, and the 5-minute drift check. +Tests: no row is `GIT`; `IMPORTING` gives `REPO_BACKEND_GIT`; a failed +`Switch` leaves the row unchanged; drift pages. + +Interfaces: + +```go +// go/server/vcs_backend.go (in Compass) +type cutoverState int16 // cutoverGit = 1, cutoverImporting = 2, cutoverNativeMirrored = 3, cutoverNativeOnly = 4 +func (s *Server) repoCutoverState(ctx context.Context, repo string) (state cutoverState, found bool, err error) +``` + +```sql +-- next free migration number (in Compass) +CREATE TABLE repo_cutovers ( + repo TEXT PRIMARY KEY, + state SMALLINT NOT NULL CHECK (state IN (1, 2, 3, 4)), + actor_id TEXT NOT NULL, + updated_at TIMESTAMPTZ NOT NULL DEFAULT now() +); +``` + +### B9 — Deploy, caps and the App + +Deploy manifests in `deploy/` (Deployment, Service for the webhook, +`ResourceQuota`, migration Job), the alerts and the spend-cap rows in +[Caps and spend](#caps-and-spend), a runbook for the switch, and the +human-action issue to register the bridge App. Tests: the manifests +validate; each alert fires on an injected failure and stays quiet on a +healthy fixture. + +## Tasks + +- [ ] B1 — `internal/gitobj`, `internal/gitexport`, the two derivers +- [ ] B2 — store additions (done in store S6 and S9) +- [ ] B3 — `saiph-bridge` binary, `saiph_bridge` schema, lease, tokens, GitHub App minter, `BridgeService` proto +- [ ] B4 — import and catch-up with refusals +- [ ] B5 — trunk export push and rejection handling +- [ ] B6 — inbound PRs and the open-PR import at the switch +- [ ] B7 — cutover RPCs and prechecks +- [ ] B8 — `repo_cutovers`, `repoCutoverState`, admin flow, drift check (in Compass) +- [ ] B9 — deploy manifests, caps, alerts, runbook, App registration issue diff --git a/docs/designs/saiph/merge-queue.md b/docs/designs/saiph/merge-queue.md new file mode 100644 index 0000000..cbe92c0 --- /dev/null +++ b/docs/designs/saiph/merge-queue.md @@ -0,0 +1,1118 @@ +# Saiph: merge queue + +Tracker: RIG-5665 + +> Freezes on merge; later changes supersede by citation, never rewrite +> (convention: the `docs/designs/agent/compass-agent-config-delivery/design.md` +> header). Unless a path says otherwise, every code path is in the +> `RigelBuild/saiph` repo, which does not exist yet. A Compass path says +> "in Compass". + +Ledger-impact: D1–D9 are filed as DL-612..DL-620. The store addition this +record asked for (`Rebase` resolutions) is folded into +[store](store.md#server-side-commit-rebase-and-merge) (task S7). + +**Named decisions this record introduces:** D1–D9 in [Decisions](#decisions). + +Length: Matt lifted the length cap for this set (ruling 6 in the +[overview](README.md#rulings)). The prose sits mostly in Approach. + +This record designs the merge queue that ruling 9 of the +[overview](README.md#rulings) asks for. It builds on the +[store](store.md#what-the-merge-queue-gets) calls, on review's land units +([review](review.md)) and on CI's affected targets and runs ([ci-cd](ci-cd.md)). It reuses +their entity and RPC names. The store call it added is listed in +[Store additions](#store-additions) and folded into store. + +## Open Questions + +OQ-1 and OQ-2 are load-bearing. OQ-6 has a sibling: ci-cd OQ-6 (flake +quarantine) changes how often OQ-6's hold fires, not its answer. The record +is written against each recommendation. + +### OQ-1 — Pre-merge or post-merge gating? (load-bearing) + +Scion: "We haven't been able to effectively replicate the industry standard +of a global repository lock on the merge queue, so instead, we've invested +in post-merge and pre-deploy automated testing. This does mean that +concurrent changes can conflict, but Claude can typically fix these +conflicts faster than humans can even identify them" +([Scion](https://www.anthropic.com/engineering/redesigning-source-control)). +Ruling 9 asks for batching and bisection, which is pre-merge gating. The +question is whether a repo may also take Scion's stance. + +- (a) A per-repo `gate` mode in `.saiphqueue` (OQ-5). `pre`, the default: + a batch lands only after a passing `RUN_KIND_MERGE_QUEUE` run on its + projected state. `post`: the queue rebases, fixes and lands by CAS with + no CI wait, and CI postsubmit, culprit finding and the pre-deploy gate + ([ci-cd](ci-cd.md)) catch a breakage. In both modes, an entry that holds a + conflict-fix version waits for a passing run. +- (b) `pre` only. Simple, but a repo with slow gate tests cannot trade + green-trunk proof for throughput. +- (c) `post` only, as Scion. It drops the batching and bisection that + ruling 9 asks for. + +**Recommendation: (a), default `pre`.** Scion could not scale a global +lock. Per-target lanes remove the lock; they keep pre-merge testing. A +repo moves to `post` only when the queue's CI is its bottleneck. The +Compass repo starts on `pre`. + +### OQ-2 — Does a conflict-fix version need a new approval? (load-bearing for review) + +Matt asked that the author or the review policy decide. + +- (a) The review policy decides by default: `carry_on_fix` in + `.saiphreview`, true unless the repo sets it false. The author can + require approval for one change with `fix_needs_approval`, which review + owns (`RequestLandRequest` field 4) and returns as + `LandEntry.fix_needs_approval` (field 6). +- (b) A fix always needs a new approval. Safe, but a one-line changelog + merge then waits for a human, which defeats the fixer at agent scale. +- (c) A fix never needs one. The author loses control of their change. + +**Recommendation: (a).** In every case the fix passes the same queue CI, +and review runs its semantic constraints again on the fix diff +([review](review.md)). + +### OQ-3 — Which model does the fixer call, and through what? + +Matt named "Luna or similar". Compass's model registry lists `gpt-6-luna` +for the implementer role (`docs/model-registry/README.md` in Compass: +"GPT-6 Luna implements"). + +- (a) An OpenAI-compatible `POST /v1/chat/completions` endpoint, set per + install (`--fixer-url`, `--fixer-model`). Through Compass, this is the + Compass LLM gateway, which serves that route + ([gateway design](../compass/server/compass-server-llm-gateway/design.md)), with + the stable name `gpt-6-luna`. A self-hosted Saiph points at any vendor. + One call per conflict, with no tools. +- (b) A Compass agent session on a Runner, with tools. It can run builds, + but Saiph then needs Compass to land a change, and a fix costs a whole + session. + +**Recommendation: (a).** The classifier limits the input to a few small +hunks, so tools buy nothing, and the queue's CI run is the build check. + +### OQ-4 — What is trivial by default? + +- (a) Conservative. Only `CONFLICT_KIND_CONTENT` in text files; at most 3 + paths, 5 hunks, 20 lines in any one term of a hunk and 60 conflicted + lines in all; no path under a deny glob (default `**/.saiph*`). +- (b) A size limit only, for any kind. A `MODIFY_DELETE` or + `FILE_DIRECTORY` conflict turns on intent, which a model cannot read + from hunks. + +**Recommendation: (a).** Mononoke's server-side merge resolution is also +bounded by a count: `collect_overlap_states` in +`eden/mononoke/features/pushrebase/src/lib.rs` returns +`MergeResolutionError::TooManyConflicts` past the +`scm/mononoke:pushrebase_max_merge_conflicts` knob. A model alone is not a +gate: MergeBERT reports "63-68% accuracy for merge resolution synthesis" +([MergeBERT](https://arxiv.org/abs/2109.00084)). + +### OQ-5 — Where does queue policy live? + +- (a) A repo-root `.saiphqueue` JSON file, read at the trunk head, as + review reads `.saiphreview`. A policy change goes through review and the + queue like any other change. A change cannot loosen the policy it lands + under, because the queue reads trunk, not the candidate. +- (b) An admin RPC and a table. Faster to change, but not reviewed and not + in history. + +**Recommendation: (a).** Spend caps are not in the tree. They are operator +flags, and `.saiphqueue` can only lower them. + +### OQ-6 — What does the queue do with an `INFRA` result? + +CI keeps one result per (changeset, target) and never retries +([ci-cd](ci-cd.md), "Flakes and quarantine"). `INFRA` means the job never +reached a verdict: an eviction, a lost node, a failed image pull. It is +red, and CI's infra alert fires. CI stores `INFRA` on the run, not as the +changeset's result, "so a person may request a new run after the cause +is fixed. Nothing re-requests it automatically." + +- (a) Request a new run at once, or rebuild the projection so its + changesets are new and CI runs them again. Either is a retry, and the + house rule forbids retrying until green. +- (b) Hold the batch. Its entries stay queued, with no bisection and no + eject, until a holder of `REPO_ROLE_LAND` calls `Requeue` after the + cause is fixed. That is the person CI's rule names. A trunk move does + not release the hold: a new projection would run the same inputs + again with no one asking, which is the automatic re-request CI forbids. +- (c) Eject the entries. That blames the author for CI's failure. + +**Recommendation: (b).** A held batch blocks only its own lane; other +lanes go on. CI's infra alert already pages the CI lane, and its fix +ends with one `Requeue`. + +### OQ-7 — Does the queue take a land-by-merge mode? + +[git-bridge](git-bridge.md) OQ-1 (c) asks for one, so that +release-please, which accepts merges, keeps tagging. + +- (a) Yes, per owner. `.saiphqueue` `merge_owners` lists change owners, + for example `"external:github:release-please[bot]"`. Such an entry is + batched alone. It lands as `Merge(trunk, head)`, with a `chain` of the + one merge changeset; its members keep their ids. Review's "Landing + detection" already marks side members of a merge as landed. +- (b) No. The bridge takes its fallback, OQ-1 (b): the queue lands + imported PRs by rebase, and release-please stops tagging. + +**Recommendation: (a).** It costs one branch in landing, no new store +RPC, and keeps both bots working. + +## Problem / Intent + +Agents land work at a rate no human queue was built for. Scion went from +"around 2,500 code changes merged per day" to "over 30,000 per day", and +could not keep "a global repository lock on the merge queue" +([Scion](https://www.anthropic.com/engineering/redesigning-source-control)). +Saiph needs a queue that lands review's ready stacks onto trunk with no +global lock: it batches, bisects failures, takes stacks as units, rebases +on the server, has a cheap model fix trivial conflicts, and ejects every +other conflict (ruling 9). + +## Approach + +### Ground truth + +- **Saiph's repo** is `RigelBuild/saiph`. It does not exist yet, so every + Saiph path here is a plan. +- **Store.** The queue uses only the calls listed in + [What the merge queue gets](store.md#what-the-merge-queue-gets): + `Rebase` with `drop_empty`, `Diff`, `MergeBase`, `IsAncestor`, `UpdateRef` + with `chain`, `Pin`, `Unpin` and `WatchRefs`, plus the reads `GetRef`, + `GetChangesets`, `ReadFile`, `Log` and `PutBlobs`. Trunk moves only for + a subject with `REPO_ROLE_LAND`. Nothing conflicted is stored + ([OQ-3](README.md#open-questions)). +- **Review** ([review](review.md), `saiph.v1.ReviewService`). The queue reads + review; review never calls the queue. A land unit is a change with + `land_requested` plus its open dependencies. `WatchEvents` with + `EVENT_KIND_READINESS_CHANGED` carries `land_ready`, and + `GetLandUnit` returns the unit's `LandEntry` rows in dependency order. + Review records a landing from the trunk ref log; it has no callback. +- **CI** ([ci-cd](ci-cd.md), `saiph.v1.TargetService` and + `saiph.v1.RunService`). `AffectedTargets(base, head)` is a snapshot diff + of two root trees with no ancestry rule, so a speculative base works. + It returns moon task ids `project:task`, `all` (the barrier) and + `graph_changed`. A false `graph_changed` "always means it is + unchanged". `RequestRun` with `RUN_KIND_MERGE_QUEUE` runs only `gate` + tasks. `GetTargetStats` gives per-target flake counts. CI holds no pins. +- **Model gateway.** Through Compass, the fixer calls the Compass LLM + gateway's `POST /v1/chat/completions` + ([gateway design](../compass/server/compass-server-llm-gateway/design.md)). + The gateway design has no spend cap, so the queue enforces its own + (see [Model spend caps](#model-spend-caps)). + +### Research and what Saiph takes from it + +| Source | What it does | Saiph takes | +| -- | -- | -- | +| Scion | No global lock; "post-merge and pre-deploy automated testing"; "Claude can typically fix these conflicts faster than humans can even identify them" | No global lock; the `post` gate mode (OQ-1); a model fixer | +| Trunk, predictive testing | "each PR is tested as if all the PRs ahead of it have already been merged" ([docs](https://docs.trunk.io/merge-queue/optimizations/predictive-testing)) | Projected state | +| Trunk, parallel queues | "C tests predictively against both A and B"; "ALL impact is a serialization point" ([docs](https://docs.trunk.io/merge-queue/optimizations/parallel-queues)) | Lanes from affected targets; the `all` barrier | +| Trunk, pending failure depth | waits on "levels of successor test runs" before it fails or bisects ([docs](https://docs.trunk.io/merge-queue/optimizations/pending-failure-depth)) | Successor evidence | +| Trunk, batching | "It bisects into 2 sub-batches, retests, and recurses on whichever sub-batch failed"; "Only failures are reused" ([docs](https://docs.trunk.io/merge-queue/optimizations/batching)) | Bisection with its own concurrency, by prefixes, on new changesets | +| Trunk, anti-flake | "Retry limit should be 1-3 max" ([docs](https://docs.trunk.io/merge-queue/optimizations/anti-flake-protection)) | Nothing: no reruns (the house rule). Successor evidence and CI's flake quarantine replace them | +| Trunk, stacks | "tests and lands that Pull Request and every Pull Request beneath it atomically" ([docs](https://docs.trunk.io/merge-queue/using-the-queue/stacked-pull-requests)) | A stack is one entry | +| Graphite | "can't resolve any legitimate merge conflicts as a result of racing PRs on trunk that require human intervention" ([docs](https://graphite.com/docs/merge-pull-requests)) | Structural rebase is automatic; a real conflict needs a fixer or an eject | +| Uber SubmitQueue | "if the build graph remains unchanged, we just need to use the intersection of changed targets"; "only 7.9% of changes actually cause a change to the build graph" ([paper](https://www.masoud.io/docs/eurosys19.pdf)) | The land-time retest skip; a graph change is a barrier | +| Zuul | "Each time a change successfully merges, the window is increased by one. Each time a change fails, the window is halved" ([docs](https://zuul-ci.org/docs/zuul/latest/gating.html)) | An adaptive window of runs in flight | +| Google | Flake Aware Culprit Finding "is based on Bayesian inference and noisy binary search, utilizing prior information" ([paper](https://research.google/pubs/flake-aware-culprit-finding/)) | Flake rates as priors: they decide whether a failure waits for successor evidence or is bisected at once | +| Mononoke pushrebase | `MAX_REBASE_ATTEMPTS: usize = 100`; a server-side merge bounded by `scm/mononoke:pushrebase_max_merge_conflicts` (`eden/mononoke/features/pushrebase/src/lib.rs`) | The CAS retry bound; a size-bounded trivial class | + +Chromium's commit queue names the same root problem: "The main problem is +test flakiness" +([design](https://www.chromium.org/developers/testing/commit-queue/design/)). +No primary source found describes a queue-side model conflict fixer. +GitHub's Copilot agent resolves a conflict, then "requests your review". +So the fixer here is Saiph's own design, held to the same CI as any +change. + +### Shape + +`saiph-queue` is its own binary in the saiph repo (OQ-2 of the +[overview](README.md#open-questions)). Its replicas keep no state of +their own; state is in the schema `saiph_queue`. Each repo has one +**planner**, held by a lease row that any replica can take, as the store's +derive workers do. The planner makes decisions in milliseconds and holds +nothing while CI runs. Batches in disjoint lanes test and land in +parallel, and they meet only at the trunk CAS. + +```mermaid +flowchart LR + RV[review: WatchEvents, GetLandUnit] --> P[planner, one per repo] + P -->|Rebase, Diff| S[saiph-server] + P -->|AffectedTargets, RequestRun| CI[saiph-ci] + P --> FX[fixer] -->|chat completion| M[model endpoint] + FX -->|Rebase with resolutions| S + FX -->|UploadVersions CONFLICT_FIX| RV + P -->|UpdateRef with chain| S + P -->|Publish eject comment| RV +``` + +The queue's words: + +| Term | Meaning | +| -- | -- | +| entry | One land unit in the queue: a root change id and its members' `(change_id, version, changeset)`. A stack DAG is one entry. | +| seq | The entry's place in queue order, given at admission. | +| overlap | Two entries overlap when their targets meet, their changed paths meet, or either is a barrier. | +| barrier | An entry with `all=true` or `graph_changed=true`. It overlaps every entry. | +| ahead set | Every earlier active entry that an entry overlaps, closed over overlap. | +| lane | The entries that share an ahead set chain. A lane is not stored; it falls out of the ahead sets. | +| projection | Trunk head with the ahead set rebased on it in seq order: the state an entry is tested on. | +| batch | A run of entries tested by one CI run. It lands by one CAS, alone or with the batches before it in its lane. | +| fix | A conflict resolution the queue makes, recorded as a `VERSION_REASON_CONFLICT_FIX` version. | +| eject | Removal from the queue with a typed reason, a review comment, and the owner in attention. | +| required set | The targets a batch must pass: its run's targets, less quarantined targets and targets broken at the base. | +| probe | A CI run on a prefix of a failed batch, or on a successor's head for one target. Its changeset is new, so it is not a rerun. | +| hold | An entry or batch that waits with no verdict: for an ahead entry to land, or after `INFRA` (OQ-6). | + +### Who calls whom + +- **The queue's identity** is the subject `service:merge-queue`, with + scopes `read write land` (see [Auth](#auth)). The queue serves a repo + only while that subject holds `REPO_ROLE_LAND` on it (`GetRepo` returns + `caller_role`) and the repo is enabled. `EnableRepo` does both in one + call; `DisableRepo` undoes both ([API](#api-protosaiphv1queueproto)). +- **Review to queue.** The queue reads review. It needs review's land + units and three writes: `UploadVersions` for a fix, `Publish` for an + eject, and nothing else. +- **Queue to store.** `Rebase`, `Diff`, `UpdateRef`, `Pin`, `Unpin`, + `WatchRefs` and the reads. It writes trunk and nothing else. +- **Queue to CI.** `AffectedTargets`, `RequestRun`, `WatchRun`, + `CancelRun`, `GetTargetStats` and `ListBreakages`. CI forwards the + queue's token to the store and reads `caller_role`; + `RUN_KIND_MERGE_QUEUE` needs `REPO_ROLE_LAND`, which the queue holds. + +### Admission + +The planner tails `WatchEvents(repo, after_seq, kinds = +[EVENT_KIND_READINESS_CHANGED])` from its cursor +`saiph_queue.repos.review_seq`. For an event with `land_ready = true` +it calls `GetLandUnit(repo, change_id)`: + +- `ready = false`: nothing happens. The blockers are review's. +- `ready = true`: the planner records one entry: the root change, the + `LandEntry` rows in dependency order, and the next `seq`. A change is in + at most one active entry. A new unit that holds every member of a + queued entry replaces it and keeps its `seq`. When a member is in a + batch already, the new unit waits until that batch lands or leaves, and + then the planner reads `GetLandUnit` again; it returns only the members + still open. +- `land_ready = false` withdraws the entry. A batch that holds it + dissolves (see [Failure handling](#failure-handling)). + +Then the planner places the entry against trunk head H: + +1. `Rebase(changesets = members, onto = H, drop_empty = true)`. A + conflict here is a trunk conflict + ([Conflicts](#conflicts-auto-rebase-and-the-fixer)). +2. **Changed paths.** `Diff(from = empty, to = member, hunks = false)` for + each member gives the member's own `FileDiff.path` and `copied_from`, + with no trunk in it. The union is the entry's paths. +3. **Targets.** `AffectedTargets(base = H, head = the rebased head)`. + `all = true` or `graph_changed = true` makes the entry a barrier. So + does any change to `.saiphqueue` (OQ-5). + +Targets measured from H hold on any projection while the build graph is +unchanged. That is Uber's rule: "if the build graph remains unchanged, we +just need to use the intersection of changed targets" +([SubmitQueue](https://www.masoud.io/docs/eurosys19.pdf)). On each trunk +move from `WatchRefs`, the planner calls `AffectedTargets(old head, new +head)`; when `all` or `graph_changed` is true, it measures every queued +entry's targets again. + +### Lanes, ahead sets and the window + +Two entries overlap when their targets meet, their paths meet, or either +is a barrier. Targets catch a change that breaks another through a +dependency. Paths catch two edits to one file that no target owns, such as +docs, which conflict as text. An entry's ahead set is every earlier active +entry it overlaps, closed over overlap, so a lane is a connected group of +entries in `seq` order. Lanes test and land in parallel and never wait for +each other. Trunk names the same idea: "C tests predictively against both +A and B" and "ALL impact is a serialization point" +([parallel queues](https://docs.trunk.io/merge-queue/optimizations/parallel-queues)). + +Each lane has a window: the most entries it may have in testing at once. +It starts at `window` (default 20, Zuul's default), grows by one for each +entry that lands and halves for each batch that fails, with a floor of 2 +and a ceiling of `window_max` (default 64). Zuul: "Each time a change +successfully merges, the window is increased by one. Each time a change +fails, the window is halved" +([Zuul](https://zuul-ci.org/docs/zuul/latest/gating.html)). Windows live +in the planner's memory, keyed by a lane's lowest `seq`. When lanes join +the smaller window wins; when one splits each part keeps it; a new planner +starts every lane at `window`. + +### Projections and batches + +A lane is a straight line of batches in `seq` order. The planner cuts +them as the window allows: + +- A batch is up to `batch_max` (default 8) consecutive entries of one lane + that are not yet in testing. A barrier is a batch on its own: its run + covers every gate task, and a mixed batch would bisect against all of + them. +- The batch's base is the head of the batch before it in the lane, or H + when none is in testing. +- The planner builds the projection with one `Rebase(changesets = members, + onto = previous head, drop_empty = true)` per entry, in order. The last + result is the batch head. The first-parent path from the base to the + head is the batch's `chain`, stored with the batch. +- An entry that rebased cleanly onto H but conflicts in the projection has + a speculative conflict. It leaves the batch and holds until the batches + ahead of it land or leave; then it is planned again. The queue never + fixes a conflict against a state that has not landed. +- The planner pins the chain with `Pin(holder = "merge-queue:b", ttl_seconds = 86400)`, renews it hourly while the batch lives and + calls `Unpin(holder)` when it lands or dissolves. The holder follows the + store's `":"` form; the key is the batch id, not the + store's example `merge-queue:`, because lanes are not stored. + Members are pinned already: review pins every version of an open + change ([review](review.md)). + +A member that `drop_empty` drops is already on trunk. The rest of the +entry goes on, because the store rebases the dropped member's descendants +onto its parent. The dropped change never lands, so review cannot see it +close; the queue publishes a comment on it ("already on trunk at +``") and puts its owner in attention. + +### CI runs + +For each batch the planner calls `RequestRun(repo, changeset = batch head, +base = batch base, kind = RUN_KIND_MERGE_QUEUE, targets = [], +idempotency_key = "mq:b", priority = 50)`. Empty `targets` is +`affected(base, head)`: exactly the batch's own delta. A batch further +down the lane tests only what it adds; what it does not touch passed in +the batches before it, by Uber's rule. Then it reads `WatchRun(run_id)`. + +A batch's **required set** is its run's targets, less: + +- results with `quarantined = true`, and targets with an open + `BREAKAGE_KIND_FLAKE` breakage in `ListBreakages` at the time of the + verdict (CI's flake quarantine, ci-cd OQ-6); +- results with `broken_at_base = true`. Trunk was red on that target + before this batch, CI blocks only the targets a breakage broke, and the + culprit is CI's to find ([ci-cd](ci-cd.md)). If the queue held such a batch, + the fix for the breakage could not land either. + +The batch passes when every required result is `PASSED`. `reused = true` +results count as they are. `INFRA` holds the batch (OQ-6). `CANCELLED` +comes only from the queue's own `CancelRun` and gives no verdict. Every +other required result is a failure. + +Probes, land retests and successor probes run at priority 60, so a lane +that is deciding a failure moves before new batches start. + +### Failure handling + +A failed batch has a set F of required targets that did not pass. The +queue never runs a (changeset, target) twice; every probe below is on a +changeset CI has not tested for that target. + +1. **Quarantine first.** The planner reads `ListBreakages` once. A + target in F with an open `BREAKAGE_KIND_FLAKE` breakage leaves the + required set, and the verdict is taken again. If F is now empty, the + batch passes. +2. **Successor evidence.** Trunk's pending failure depth waits on + "levels of successor test runs". Saiph asks for them. For each target + T in F, the planner takes the nearest batch behind this one in the lane + whose projection holds it, and calls `RequestRun(changeset = that + head, base = that base, targets = [T], priority = 60)`, up to + `pending_failure_depth` (default 2) batches deep. A successor pass on + the same `input_key` is the disagreement CI turns into a flake + breakage, and step 1 then drops T (ci-cd "Flakes and quarantine"). A + successor fail on the same `input_key` confirms T. A result on a + different `input_key` is not evidence. +3. **When there is no successor.** The prior is T's flake rate, + `flaky / runs` from `GetTargetStats(window_days = 7)`, as Google's + flake-aware search uses "prior information". At or above + `flake_prior` (default 0.01), the batch waits up to `pfd_wait` + (default 10 minutes) for a successor to be cut and probed. Below it, + the failure is taken as real at once. +4. **Bisection.** A confirmed failure in a batch of one ejects that + entry. In a batch of n, the projection already holds a head after each + entry, so a prefix needs no new rebase. The planner probes up to + `probe_parallel` (default 3) evenly spaced prefix heads per round, with + `targets = F`, and keeps the shortest failing prefix. That takes + ⌈log₄ n⌉ rounds; Trunk's "bisects into 2 sub-batches" is the same + search with `probe_parallel = 1`. A prefix that fails and a longer one + that passes on the same `input_key` is a flake, and step 1 applies. +5. **After the culprit.** The culprit entry is ejected + ([Eject](#eject)). Entries before it form a new batch on the prefix + head; `RequestRun` with empty targets reuses the probe results and + runs only the rest. Entries after it go back to their lane and are + projected again, so their changesets are new. + +When ci-cd OQ-6 is ruled no, step 1 never fires, and a flaky target ejects +the entry that drew it until the target's owner fixes it. The window +halves for each failed batch. + +### Landing + +A batch lands when it passed (`gate = pre`) or when it is cut +(`gate = post`, OQ-1), and when every batch ahead of it in its lane has +landed. Landing runs these steps, in order: + +1. **Review.** `GetLandUnit` for each entry. An entry that is not ready, + or whose `(change_id, version)` set changed, leaves the batch, and the + batch is cut again without it ([Land hand-off](review.md) in review). +2. **Trunk.** `GetRef(trunk)` gives T. When T is the batch base, go to + step 4. +3. **Recompute.** `Rebase(changesets = the batch's members, onto = T, + drop_empty = true)` gives a new head and chain. A conflict here is a + trunk conflict ([Conflicts](#conflicts-auto-rebase-and-the-fixer)). + Then `AffectedTargets(base = batch base, head = T)` measures the + trunk delta. When `all` and `graph_changed` are false and its targets + do not meet the batch's required set, the batch lands with no retest: + Uber's rule. Otherwise the planner requests a land retest of the + intersection on the new head, at priority 60, and lands on a pass. +4. **CAS.** `UpdateRef(name = trunk, expected = T, target = head, chain = + chain, reason = "merge-queue b")`. One CAS lands the whole + batch. `updated = false` returns the current target; the planner goes + back to step 3 with it. +5. **After.** `Unpin(holder)`, the entries are `LANDED` in + `saiph_queue.entries`, and the window grows by one per entry. Review + learns of the landing from the ref log, never from the queue. + +Each pass through step 3 is new work on a new trunk, not a retry. It is +still bounded: after `max_land_attempts` (100, Mononoke's +`MAX_REBASE_ATTEMPTS`), the batch holds and the queue's lost-CAS alert +fires. A busy repo loses a CAS only to a batch in another lane, so the +retest skip makes most recomputes free. + +### Conflicts, auto-rebase and the fixer + +Auto-rebase is the store's `Rebase` with `drop_empty`, run as +`service:merge-queue`. A conflict returns as a typed `Conflict` with each +`PathConflict`'s kind, terms and hunks; nothing conflicted is stored. The +queue fixes only a **trunk conflict**: one against a trunk head that has +landed. A speculative conflict holds, as in +[Projections and batches](#projections-and-batches), because the state it +conflicts with may never land. + +**Classifier.** Deterministic, and first. A conflict is trivial when +every rule in `.saiphqueue` `trivial` holds (defaults in OQ-4): + +- every `PathConflict` is `CONFLICT_KIND_CONTENT` and no term is binary; +- at most `max_paths` paths, `max_hunks` hunks, `max_term_lines` lines in + any one term and `max_lines` conflicted lines in all; +- no path matches a `deny` glob (default `**/.saiph*`, so no ACL, review + or queue policy file); +- the fixer is on and the spend caps allow the call + ([Model spend caps](#model-spend-caps)). + +Anything else is ejected with the conflict attached. + +**Fixer.** One `POST /v1/chat/completions` per trivial conflict, with +`model = --fixer-model`, `temperature = 0`, `response_format = {"type": +"json_object"}`, and `max_tokens` set from the per-fix cap. The system +prompt is fixed in the binary. The user message is JSON: for each hunk, +the path, the base, trunk and change terms as lines, and three lines of +context; plus the trunk changeset's message and the change's message. The +answer is `{"hunks": [{"path": "…", "index": 0, "lines": ["…"]}], +"resolvable": true}`. The prompt tells the model to answer +`"resolvable": false` when the two sides change meaning. + +**Validation.** The fixer refuses the answer, and the entry is ejected, +when `resolvable` is false, when a hunk is missing or extra, when a +resolved line appears in neither the trunk term nor the change term of its +hunk, or when a line looks like a conflict marker. So the model may only +choose and order lines that the two sides wrote. This is the safe part of +git's `union` driver, which keeps both sides "in random order and the user +should verify the result"; here the order is the model's and the +verification is CI. + +**Writing the fix.** The fixer builds each file from the merge with the +resolved hunks, stores it with `PutBlobs`, and calls `Rebase` again with +`resolutions` ([Store additions](#store-additions)). The store checks that +each resolution lands on a path that still conflicts, so a stale fix +fails loudly. Then `UploadVersions(heads = the rebased heads, reason = +VERSION_REASON_CONFLICT_FIX)` makes the fix a version. Members that +rebased cleanly come out `TRIVIAL_REBASE` and keep their approvals. The +fixed member's approvals follow review: they carry when `carry_on_fix` +holds and the change has no `fix_needs_approval`, and review checks its +`require` rules again on the fix diff (OQ-2). The planner then reads +`GetLandUnit` again. A ready unit is placed as a new entry with the same +`seq`; an unready one waits for its approvers, out of the queue. Every fix +passes a queue CI run before it lands, in both gate modes. + +### Eject + +An eject removes the entry and calls review once, on the change that +caused it: `Publish(change_id, version = current, comments = [one +NewComment{thread_id = 0, anchor{path = ""}, body, resolution = +RESOLUTION_UNRESOLVED}], attention_add = [owner])`. The owner comes from +`GetChange(...).change.owner`. The unresolved thread keeps the change +unready until the owner resolves it. Other members of the entry go back to +the queue only when review says they are ready again. + +| Reason | When | The comment carries | +| -- | -- | -- | +| `CONFLICT` | A trunk conflict the classifier refuses | Each path, its kind and its hunks with base, trunk and change terms | +| `CONFLICT_UNFIXED` | The fixer answered unresolvable, or validation refused it | The same, plus the refusal | +| `FIX_BUDGET` | A trivial conflict, but a spend cap refused the call | The same, plus the cap that refused | +| `FAILED_TARGETS` | Bisection named this entry | Each failed target and its `details_url` | +| `POLICY_INVALID` | The entry changes `.saiphqueue` to something the queue cannot parse | The parse error | +| `MANUAL` | A `REPO_ROLE_LAND` holder called `Eject` | The caller and the reason text | + +A body over review's 64 KiB comment limit is cut at a hunk boundary and +ends with a count of what was left out. + +### Gate modes + +`gate = pre` lands a batch after a passing `RUN_KIND_MERGE_QUEUE` run on +its projection. `gate = post` cuts and lands batches with no CI wait: the +queue still rebases, fixes and ejects conflicts, and CI postsubmit, +culprit finding and the pre-deploy gate catch a breakage ([ci-cd](ci-cd.md)). +That is Scion's stance, per repo. In `post`, an entry that holds a +`CONFLICT_FIX` version is batched alone and waits for a passing run, since +no human has read the fix. The repo's mode is read from `.saiphqueue` at +the trunk head when a batch is cut. + +### Policy: `.saiphqueue` + +A JSON file at the repo root, read at the trunk head. A missing file or +key means the default. An entry that changes the file is a barrier, and +the queue parses the new file at admission; a file it cannot parse ejects +the entry with `POLICY_INVALID`. + +```json +{ + "gate": "pre", + "window": 20, "window_max": 64, "batch_max": 8, + "pending_failure_depth": 2, "flake_prior": 0.01, "pfd_wait_seconds": 600, + "probe_parallel": 3, + "fixer": true, + "merge_owners": [], + "trivial": { "max_paths": 3, "max_hunks": 5, "max_term_lines": 20, "max_lines": 60, "deny": ["**/.saiph*"] }, + "fix_max_tokens": 16000, + "fix_repo_daily_micro_usd": 5000000 +} +``` + +The last two can only lower the operator's flags of the same name. + +### Model spend caps + +Money is integer micro-USD, never a float, as Compass's +`ModelMetadata.InputCostMicroUSD` and `OutputCostMicroUSD` are +(`go/internal/store/model_registry.go` in Compass). The operator sets the +fixer's prices as `--fixer-input-micro-usd-per-mtok` and +`--fixer-output-micro-usd-per-mtok`. + +- **Reserve, then call.** Before a call the fixer estimates input tokens + as the request's bytes divided by 3, rounded up, and reserves the + worst case: that estimate plus `max_tokens`, at the prices. The + reservation is one `saiph_queue.fix_spend` update that succeeds only + while the repo's and the tenant's day stay under their caps. A refused + reservation makes no call and ejects with `FIX_BUDGET`. +- **Settle.** After the call, the fixer replaces the reservation with the + response's `usage.prompt_tokens` and `usage.completion_tokens` at the + prices. A call that fails keeps its reservation: the vendor may have + billed it. A failed call is never repeated; the entry is ejected with + `CONFLICT_UNFIXED`. + +| Layer | Cap | Enforced by | +| -- | -- | -- | +| Fix call | `--fix-max-tokens` (default 16,000), input estimate plus output | the fixer refuses a larger input and sends the rest as `max_tokens` | +| Repo per day | `--fix-repo-daily-micro-usd` (default 5,000,000) | the `fix_spend` reservation; no call past it | +| Tenant per day | `--fix-tenant-daily-micro-usd` (default 20,000,000) | the same reservation, summed over the tenant's repos | +| Calls in flight | `--fix-max-in-flight` per repo (default 4) | the planner | +| Queue CI | CI's own caps | `saiph-ci` ([ci-cd](ci-cd.md), "Caps and spend") | +| Queue size | `--max-entries` per repo (default 5,000) | admission leaves new units with review until there is room | +| Vendor account | the vendor's usage limit, or the gateway's when it has one | outside Saiph; the deployment's spend-cap row names it, or names the gap | + +`saiph_queue_fix_spend_micro_usd_total{tenant,repo}` measures spend. A +deployment that enables the fixer on a usage-billed vendor also adds a +budget alert that pages Matt through the billing paging route, and a +spend-cap row per layer with evidence that it enforces. Until then +`--fixer-url` stays empty in that deployment, the fixer is off, and every +conflict ejects. + +### Auth + +- **The queue's token.** As review does, `saiph-queue` signs its own + five-minute tokens per tenant, `sub = service:merge-queue`, scopes + `read write land`, with a key whose public half is an issuer in + `saiph-server`'s `--issuers` file. That issuer may mint only + `service:merge-queue` (`Issuer.Subjects`, review's store addition). + The same token works at review and CI, which verify with the store's + `Verifier`. +- **Repo access.** `EnableRepo` forwards the caller's bearer to + `SetGrant(repo, "service:merge-queue", REPO_ROLE_LAND)`, so the store + decides who may turn the queue on. `DisableRepo` removes the grant. +- **The fixer's credential.** A bearer read from `--fixer-token-file`. + Through Compass it is a gateway token, which belongs to an agent + account (`GatewayCaller.AgentAccountID`, + `go/internal/store/gateway_tokens.go` in Compass), so the fixer's model + spend shows in that account's token usage. Saiph imports nothing from + Compass. + +### State: the `saiph_queue` schema + +```sql +-- internal/queue/migrations/0001_saiph_queue.sql; every table has ENABLE and +-- FORCE ROW LEVEL SECURITY and a tenant_isolation policy on saiph.tenant_id. +CREATE SCHEMA saiph_queue; +CREATE TABLE saiph_queue.repos ( -- one planner lease per repo + tenant_id TEXT NOT NULL DEFAULT current_setting('saiph.tenant_id', TRUE), + repo_id BIGINT NOT NULL, + repo TEXT NOT NULL, -- "owner/name" + enabled BOOLEAN NOT NULL, + review_seq BIGINT NOT NULL DEFAULT 0, -- WatchEvents cursor + log_seq BIGINT NOT NULL DEFAULT 0, -- WatchRefs cursor + next_seq BIGINT NOT NULL DEFAULT 1, + lease_token TEXT, + lease_until_unix_ms BIGINT, + PRIMARY KEY (tenant_id, repo_id) +); +CREATE TABLE saiph_queue.entries ( + tenant_id TEXT NOT NULL DEFAULT current_setting('saiph.tenant_id', TRUE), + repo_id BIGINT NOT NULL, + entry_id BIGINT GENERATED ALWAYS AS IDENTITY, + seq BIGINT NOT NULL, + root_change_id BYTEA NOT NULL CHECK (length(root_change_id) = 16), + state SMALLINT NOT NULL, -- queued, testing, held, landed, ejected, withdrawn + barrier BOOLEAN NOT NULL, + targets TEXT[] NOT NULL, + paths TEXT[] NOT NULL, + batch_id BIGINT, + eject_reason SMALLINT, + created_unix_ms BIGINT NOT NULL, + updated_unix_ms BIGINT NOT NULL, + PRIMARY KEY (tenant_id, repo_id, entry_id) +); +CREATE TABLE saiph_queue.members ( -- LandEntry rows, dependency order + tenant_id TEXT NOT NULL DEFAULT current_setting('saiph.tenant_id', TRUE), + repo_id BIGINT NOT NULL, + entry_id BIGINT NOT NULL, + ord INT NOT NULL, + change_id BYTEA NOT NULL CHECK (length(change_id) = 16), + version INT NOT NULL, + changeset BYTEA NOT NULL CHECK (length(changeset) = 32), + PRIMARY KEY (tenant_id, repo_id, entry_id, ord) +); +CREATE TABLE saiph_queue.batches ( + tenant_id TEXT NOT NULL DEFAULT current_setting('saiph.tenant_id', TRUE), + repo_id BIGINT NOT NULL, + batch_id BIGINT GENERATED ALWAYS AS IDENTITY, + base BYTEA NOT NULL, + head BYTEA NOT NULL, + chain BYTEA[] NOT NULL, + prefix_heads BYTEA[] NOT NULL, -- head after each entry + run_id TEXT, + state SMALLINT NOT NULL, -- testing, passed, failed, held, landed, dissolved + land_attempts INT NOT NULL DEFAULT 0, + landed_log_seq BIGINT, + created_unix_ms BIGINT NOT NULL, + PRIMARY KEY (tenant_id, repo_id, batch_id) +); +CREATE TABLE saiph_queue.probes ( + tenant_id TEXT NOT NULL DEFAULT current_setting('saiph.tenant_id', TRUE), + repo_id BIGINT NOT NULL, + batch_id BIGINT NOT NULL, + changeset BYTEA NOT NULL, + kind SMALLINT NOT NULL, -- successor, prefix, land retest + targets TEXT[] NOT NULL, + run_id TEXT NOT NULL, + PRIMARY KEY (tenant_id, repo_id, batch_id, changeset, kind) +); +CREATE TABLE saiph_queue.fixes ( + tenant_id TEXT NOT NULL DEFAULT current_setting('saiph.tenant_id', TRUE), + repo_id BIGINT NOT NULL, + fix_id BIGINT GENERATED ALWAYS AS IDENTITY, + entry_id BIGINT NOT NULL, + change_id BYTEA NOT NULL, + conflict_changeset BYTEA NOT NULL, + onto BYTEA NOT NULL, + outcome SMALLINT NOT NULL, -- uploaded, unresolvable, refused, budget, call_failed + input_tokens BIGINT, output_tokens BIGINT, + cost_micro_usd BIGINT NOT NULL, + created_unix_ms BIGINT NOT NULL, + PRIMARY KEY (tenant_id, repo_id, fix_id) +); +CREATE TABLE saiph_queue.fix_spend ( + tenant_id TEXT NOT NULL DEFAULT current_setting('saiph.tenant_id', TRUE), + repo_id BIGINT NOT NULL, + day INT NOT NULL, -- days since the Unix epoch, UTC + micro_usd BIGINT NOT NULL, -- settled plus reserved + PRIMARY KEY (tenant_id, repo_id, day) +); +CREATE TABLE saiph_queue.events ( -- WatchQueue, kept 30 days + tenant_id TEXT NOT NULL DEFAULT current_setting('saiph.tenant_id', TRUE), + repo_id BIGINT NOT NULL, + seq BIGINT NOT NULL, + entry_id BIGINT NOT NULL, + kind SMALLINT NOT NULL, + detail BYTEA NOT NULL, -- the QueueEvent, proto-encoded + at_unix_ms BIGINT NOT NULL, + PRIMARY KEY (tenant_id, repo_id, seq) +); +-- saiph_queue.schema_migrations, under its own advisory lock key, as saiph_ci's. +``` + +The planner writes its cursors and the rows they cause in one transaction, +so a crash replays nothing twice and skips nothing. The lease follows the +store's `Meta.Lease` pattern; any replica takes an expired one. + +### API: `proto/saiph/v1/queue.proto` + +The queue forwards the caller's bearer to `GetRepo` and reads +`caller_role`. Reads need `REPO_ROLE_READ`. `Eject`, `Requeue`, +`EnableRepo` and `DisableRepo` need `REPO_ROLE_LAND`. A change's owner +leaves the queue through review (`RequestLand` with `cancel = true`). + +```proto +syntax = "proto3"; +package saiph.v1; + +service QueueService { + rpc EnableRepo(EnableRepoRequest) returns (EnableRepoResponse); + rpc DisableRepo(DisableRepoRequest) returns (DisableRepoResponse); + rpc ListQueue(ListQueueRequest) returns (ListQueueResponse) { option idempotency_level = NO_SIDE_EFFECTS; } + rpc GetEntry(GetEntryRequest) returns (GetEntryResponse) { option idempotency_level = NO_SIDE_EFFECTS; } + rpc WatchQueue(WatchQueueRequest) returns (stream QueueEvent); + rpc Eject(EjectRequest) returns (EjectResponse); + rpc Requeue(RequeueRequest) returns (RequeueResponse); +} +enum EntryState { ENTRY_STATE_UNSPECIFIED = 0; ENTRY_STATE_QUEUED = 1; ENTRY_STATE_TESTING = 2; ENTRY_STATE_HELD = 3; ENTRY_STATE_LANDED = 4; ENTRY_STATE_EJECTED = 5; ENTRY_STATE_WITHDRAWN = 6; } +enum HoldReason { HOLD_REASON_UNSPECIFIED = 0; HOLD_REASON_AHEAD = 1; HOLD_REASON_INFRA = 2; HOLD_REASON_LAND_ATTEMPTS = 3; HOLD_REASON_PFD_WAIT = 4; } +enum EjectReason { EJECT_REASON_UNSPECIFIED = 0; EJECT_REASON_CONFLICT = 1; EJECT_REASON_CONFLICT_UNFIXED = 2; EJECT_REASON_FIX_BUDGET = 3; EJECT_REASON_FAILED_TARGETS = 4; EJECT_REASON_POLICY_INVALID = 5; EJECT_REASON_MANUAL = 6; } +message QueueMember { bytes change_id = 1; uint32 version = 2; bytes changeset = 3; } +message QueueEntry { + uint64 entry_id = 1; uint64 seq = 2; bytes root_change_id = 3; repeated QueueMember members = 4; + EntryState state = 5; HoldReason hold = 6; EjectReason eject = 7; bool barrier = 8; + repeated string targets = 9; uint64 batch_id = 10; string run_id = 11; uint64 lane_root_seq = 12; + int64 created_unix_ms = 13; int64 updated_unix_ms = 14; +} +message EnableRepoRequest { string repo = 1; } +message EnableRepoResponse {} +message DisableRepoRequest { string repo = 1; } +message DisableRepoResponse {} +message ListQueueRequest { string repo = 1; bool include_closed = 2; uint32 page_size = 3; string page_token = 4; } +message ListQueueResponse { repeated QueueEntry entries = 1; string next_page_token = 2; uint64 event_seq = 3; } +message GetEntryRequest { string repo = 1; bytes change_id = 2; } +message GetEntryResponse { QueueEntry entry = 1; } +message WatchQueueRequest { string repo = 1; uint64 after_seq = 2; } +message QueueEvent { uint64 seq = 1; QueueEntry entry = 2; int64 at_unix_ms = 3; } +message EjectRequest { string repo = 1; bytes change_id = 2; string reason = 3; } +message EjectResponse { QueueEntry entry = 1; } +// Releases a hold of kind INFRA or LAND_ATTEMPTS. +message RequeueRequest { string repo = 1; uint64 batch_id = 2; } +message RequeueResponse {} +``` + +`WatchQueue` answers `OUT_OF_RANGE` for an `after_seq` older than the kept +events; the consumer rescans with `ListQueue` and resumes at its +`event_seq`, as review's `WatchEvents` does. + +### Hand-offs + +- **In from review.** Readiness through `WatchEvents` and `GetLandUnit`; + out to review only `UploadVersions` with `VERSION_REASON_CONFLICT_FIX` + and the eject `Publish` ([Land hand-off](review.md) in review). Review + marks a landing from the ref log. A land unit may hold a merge whose + side member is not on the first-parent `chain`; review's "Landing + detection" walks `Log` without `first_parent` and gives a side member + the `trunk_seq` of the oldest `chain` entry that reaches it. +- **Out to CI.** `RUN_KIND_MERGE_QUEUE` runs on projections, probes and + land retests; CI keeps the results, the flake breakages and the + quarantine. A landed batch reaches CI as a `WatchRefs` entry with its + `chain`, which feeds postsubmit milestones and culprit finding. In + `gate = post`, that is the only test of a batch. +- **Out to CD.** None directly. A release is cut from a green trunk + milestone ([ci-cd](ci-cd.md)); the queue never moves `release/*`. +- **Git bridge.** Its OQ-1 (c) asks for a land-by-merge mode; OQ-7 here. + +### Store additions + +Folded into [store](store.md#server-side-commit-rebase-and-merge): +`RebaseRequest.resolutions` and `PathResolution` (task S7). The pin +holder `merge-queue:b` follows the store's +`":"` form. + +### Alternatives considered + +- **A global repo lock.** Rejected: Scion could not keep one at 30,000 + merges a day. +- **Configured lanes**, as Trunk's parallel queues take impacted targets + from the client. Rejected: CI already derives targets from the tree, and + a client-sent list can be wrong. +- **Reruns to test for a flake.** Rejected by the house rule, as in + ci-cd; successor probes and quarantine show a flake instead of hiding it. +- **Fixing speculative conflicts.** Rejected: the model would pay for + states that may never land. +- **The fixer writes with `Commit`.** Rejected: `Commit` needs path write, + and the queue holds only `land`. `Rebase` with `resolutions` keeps the + fix inside the conflicted paths. +- **git's `union` driver as the fixer.** Rejected: it keeps both sides + "in random order and the user should verify the result". +- **Storing conflicts so a human resolves them in the store.** The store + never keeps them ([overview](README.md#open-questions) OQ-3). +- **A Compass agent session as the fixer.** OQ-3 (b). + +### Decisions + +- **D1 — DL-612.** `saiph-queue` is its own service with the schema `saiph_queue` + and one planner per repo under a lease. It reads review and never is + called by it. +- **D2 — DL-613.** Lanes are not configured: two entries share a lane when their + affected targets or changed paths meet. `all`, `graph_changed` and a + `.saiphqueue` change are barriers. +- **D3 — DL-614.** A batch is tested on a speculative projection of its lane, with + a Zuul window per lane and at most `batch_max` entries. +- **D4 — DL-615.** The queue never reruns a target. A failure is judged by CI's + flake quarantine, successor probes with flake rates as priors, and + prefix bisection on changesets CI has not tested. `INFRA` holds a batch + until `Requeue`. +- **D5 — DL-616.** A batch lands by one `UpdateRef` with `chain`. A lost CAS + rebases the batch onto the new trunk and retests only targets the trunk + delta touches, with no retest when the graph is unchanged and they do + not meet; at most 100 attempts. +- **D6 — DL-617.** Auto-rebase is `Rebase` with `drop_empty`. The queue fixes only + trunk conflicts: a deterministic classifier first, then one + chat-completion call whose answer may only choose lines the two sides + wrote, written by `Rebase` with `resolutions` and uploaded as a + `CONFLICT_FIX` version that passes the same CI. +- **D7 — DL-618.** An eject is one unresolved change-level comment with a typed + reason and the conflict or failed targets attached, and the owner in + attention. +- **D8 — DL-619.** Fixer spend has a per-call token cap and per-repo and + per-tenant daily micro-USD caps, reserved before each call, and a + budget alert that pages Matt; the fixer stays off until they exist. +- **D9 — DL-620.** Queue policy is `.saiphqueue` at the trunk head, with a per-repo + `gate` of `pre` (default) or `post`; operator caps can only be lowered. + +## Plan + +### Global Constraints + +- **Inherit [store](store.md#global-constraints):** public repo, Go 1.26, + `connectrpc.com/connect/v2`, Compass's lint rules (`errors.AsType[T]`, + no `time.Sleep`, no `fmt.Print*`, no `context.Background`/`TODO` + outside `main` and tests), red/green tests, `buf lint` STANDARD, SQL + that YugabyteDB YSQL and Postgres 16 both run, no diff library. +- **Layout.** `internal/queue/` holds `plan`, `lane`, `run`, `land`, + `fix`, `eject`, `policy`, `spend` and `api`. `cmd/saiph-queue` is the + service. Migrations live in `internal/queue/migrations/`, numbered from + `0001`. +- **No retries.** Nothing reruns a target or repeats a model call. + Resuming a stream at its cursor is allowed. A landing's rebase after a + lost CAS is new work, bounded at 100. +- **Money** is `int64` micro-USD. +- **The fixer** speaks the OpenAI-compatible chat-completions schema over + `net/http`; no vendor SDK. +- **Tests.** Time-dependent logic (leases, windows, `pfd_wait`) uses + `testing/synctest`; tests wait on the event, never a sleep. Store, + review and CI are in-process Connect fakes; database tests run on + Postgres 16 and YugabyteDB. + +### Order + +```text +Q1 proto ─┬─ Q2 schema ─ Q4 admission ─ Q5 lanes+batches ─ Q6 runs ─┬─ Q7 failures ─┐ +Q3 store resolutions ─────────────────────────────── Q9 fixer ──────┼─ Q8 landing ──┼─ Q11 wiring+caps + Q10 eject+API ──┘ ┘ +``` + +### Q1 — Proto and generated code + +`proto/saiph/v1/queue.proto` as in [API](#api-protosaiphv1queueproto). +Tests: `buf lint` passes; `buf breaking` against the frozen file passes. + +Interfaces: + +```go +// gen/saiph/v1/saiphv1connect +type QueueServiceHandler interface{ /* EnableRepo, DisableRepo, ListQueue, GetEntry, WatchQueue, Eject, Requeue */ } +``` + +### Q2 — The `saiph_queue` schema and the lease + +The tables in [State](#state-the-saiph_queue-schema) with RLS, the planner +lease and the cursors. Tests: another tenant's rows are invisible; a +second replica does not take a live lease and takes an expired one; a +cursor and the rows it causes commit together. + +Interfaces: + +```go +// internal/queue/plan +type Store struct{ /* pgxpool */ } +func (s *Store) Lease(ctx context.Context, tenant string, repo meta.RepoID, token string, ttl time.Duration) (Cursors, bool, error) +type Cursors struct{ ReviewSeq, LogSeq, NextSeq int64 } +``` + +### Q3 — Store addition: `Rebase` resolutions + +Folded into store task S7 +([store](store.md#s7--text-merge-rebase-merge-and-diff)), which carries +the tests and interfaces. + +### Q4 — Admission and placement + +`WatchEvents`, `GetLandUnit`, entries, the `Rebase` onto trunk, paths +from `Diff`, targets from `AffectedTargets`, barriers. Tests: a stack +DAG is one entry in parents-first order; a unit that holds a queued +entry replaces it with the same `seq`; `land_ready = false` withdraws; a +`.saiphqueue` change is a barrier; a replayed event is a no-op. + +Interfaces: + +```go +// internal/queue/plan +type Entry struct{ ID, Seq int64; Root [16]byte; Members []Member; Barrier bool; Targets, Paths []string } +type Member struct{ ChangeID [16]byte; Version uint32; Changeset object.ID } +func (p *Planner) Admit(ctx context.Context, tx pgx.Tx, e *saiphv1.ReviewEvent) error +func (p *Planner) Place(ctx context.Context, e *Entry, trunk object.ID) error +``` + +### Q5 — Lanes, windows, batches and pins + +Overlap, ahead sets, the window, cutting batches, projections and pins. +Tests: entries with disjoint targets and paths land in parallel; a +barrier waits for every earlier entry; the window grows by one per land +and halves per failure within its floor and ceiling; a speculative +conflict holds; a batch's chain is pinned and unpinned once. + +Interfaces: + +```go +// internal/queue/lane +func Overlaps(a, b *plan.Entry) bool +func AheadSets(entries []*plan.Entry) map[int64][]int64 +type Window struct{ Size, Floor, Max int } +func (w *Window) Landed(n int) +func (w *Window) Failed() +type Batch struct{ ID int64; Entries []int64; Base, Head object.ID; Chain, PrefixHeads []object.ID } +func (l *Lanes) Cut(ctx context.Context, trunk object.ID) ([]*Batch, error) +``` + +### Q6 — CI runs and verdicts + +`RequestRun`, `WatchRun`, the required set, `INFRA` and `CANCELLED`. +Tests: a `quarantined` result is not required; `broken_at_base` is not +required; `INFRA` holds and requests nothing; a batch's second +`RequestRun` uses the same key and gets the same run. + +Interfaces: + +```go +// internal/queue/run +type Verdict struct{ Passed bool; Failed []string; Infra bool } +func Required(results []*saiphv1.TargetResult, flaky map[string]bool) []string +func Judge(results []*saiphv1.TargetResult, flaky map[string]bool) Verdict +``` + +### Q7 — Failure handling + +Successor probes, flake priors, `pfd_wait`, prefix bisection and the +split after a culprit. Tests: a successor pass on the same `input_key` +lands the batch once CI quarantines the target; a different `input_key` +is not evidence; a batch of 8 with one culprit finds it in 2 rounds with +`probe_parallel = 3`; no (changeset, target) is requested twice across +a seeded simulation; entries before the culprit land. + +Interfaces: + +```go +// internal/queue/run +type Bisect struct{ PrefixHeads []object.ID; Parallel int; Failing map[int]bool } +func (b *Bisect) Next() []int +func (b *Bisect) Culprit() (int, bool) +func FlakePrior(s *saiphv1.TargetStats) float64 +``` + +### Q8 — Landing + +`GetLandUnit` before the CAS, recompute, the retest skip, `UpdateRef` +with `chain`, the attempt bound. Tests: a batch on an unchanged trunk +lands with one CAS; a lost CAS against a disjoint trunk delta lands with +no new run; an intersecting delta runs only the intersection; a changed +version leaves the batch; attempt 101 holds and fires the alert. + +Interfaces: + +```go +// internal/queue/land +type Outcome struct{ Landed bool; LogSeq int64; Retest []string; Hold saiphv1.HoldReason } +func (l *Lander) Land(ctx context.Context, b *lane.Batch) (Outcome, error) +func SkipRetest(delta *saiphv1.AffectedTargetsResponse, required []string) bool +``` + +### Q9 — Classifier, fixer and spend + +The classifier, the chat-completion call, validation, `PutBlobs`, +`Rebase` with `resolutions`, `UploadVersions`, the spend ledger. Tests: +a `MODIFY_DELETE` is not trivial; a deny-glob path is not trivial; an +answer with an invented line is refused; a valid answer becomes a +`CONFLICT_FIX` version; a reservation past the repo or tenant cap makes +no call; a failed call keeps its reservation and is not repeated; usage +settles the ledger to the cent. + +Interfaces: + +```go +// internal/queue/fix +type Limits struct{ MaxPaths, MaxHunks, MaxTermLines, MaxLines int; Deny []string } +func Trivial(c *saiphv1.Conflict, l Limits) (bool, string) +type Fixer struct{ URL, Model string; Token []byte; HTTP *http.Client; Spend *spend.Ledger } +func (f *Fixer) Resolve(ctx context.Context, c *saiphv1.Conflict, msgs [2]string) ([]*saiphv1.PathResolution, error) +func Validate(c *saiphv1.Conflict, answer Answer) error +// internal/queue/spend +func (l *Ledger) Reserve(ctx context.Context, tenant string, repo meta.RepoID, microUSD int64) (Reservation, error) +func (r Reservation) Settle(ctx context.Context, microUSD int64) error +``` + +### Q10 — Eject and the queue API + +The eject comment, the typed reasons, `EnableRepo`, `DisableRepo`, +`ListQueue`, `GetEntry`, `WatchQueue`, `Eject`, `Requeue`. Tests: an eject +publishes one unresolved change-level comment and adds the owner; a body +over 64 KiB is cut at a hunk; `Eject` by a `REPO_ROLE_WRITE` caller is +`PERMISSION_DENIED`; `Requeue` releases an `INFRA` hold; `WatchQueue` +resumes at its cursor. + +Interfaces: + +```go +// internal/queue/eject +func Body(reason saiphv1.EjectReason, c *saiphv1.Conflict, failed []*saiphv1.TargetResult, limit int) string +// internal/queue/api +func NewHandler(d *Deps) saiphv1connect.QueueServiceHandler +``` + +### Q11 — Binary, auth, deployment and caps + +`cmd/saiph-queue` with `--dsn`, `--server`, `--review`, `--ci`, +`--signing-key`, `--fixer-url`, `--fixer-model`, `--fixer-token-file`, +the price and cap flags; the issuer entry with `Subjects`; metrics; the +lost-CAS alert; the fixer spend alert and spend-cap rows for each +deployment that turns the fixer on. Tests: the queue's token carries +`service:merge-queue` and `land`; an empty `--fixer-url` ejects every +conflict; the spend metric matches the ledger. + +Interfaces: + +```go +// cmd/saiph-queue +type flags struct{ DSN, Server, Review, CI, SigningKey, FixerURL, FixerModel, FixerTokenFile string; InputMicroUSDPerMTok, OutputMicroUSDPerMTok, RepoDailyMicroUSD, TenantDailyMicroUSD int64; FixMaxTokens, FixMaxInFlight, MaxEntries int } +``` + +## Tasks + +- [ ] Q1 — `proto/saiph/v1/queue.proto` and generated code +- [ ] Q2 — `saiph_queue` schema, RLS, lease and cursors +- [ ] Q3 — store addition: `Rebase` resolutions (done in store S7) +- [ ] Q4 — admission and placement +- [ ] Q5 — lanes, windows, batches and pins +- [ ] Q6 — CI runs and verdicts +- [ ] Q7 — failure handling: successor probes, priors, bisection +- [ ] Q8 — landing: CAS, recompute, retest skip +- [ ] Q9 — classifier, fixer and spend ledger +- [ ] Q10 — eject and `QueueService` +- [ ] Q11 — `cmd/saiph-queue`, auth, deployment, caps and alerts diff --git a/docs/designs/saiph/review.md b/docs/designs/saiph/review.md new file mode 100644 index 0000000..5085162 --- /dev/null +++ b/docs/designs/saiph/review.md @@ -0,0 +1,1171 @@ +# Saiph: review + +Tracker: RIG-5665 + +> Freezes on merge; later changes supersede by citation, never rewrite +> (convention: the `docs/designs/agent/compass-agent-config-delivery/design.md` +> header). Unless a path says otherwise, every code path is in the +> `RigelBuild/saiph` repo, which does not exist yet. A Compass path says +> "in Compass". + +Ledger-impact: D1–D5 are filed as DL-597..DL-601. No existing decision +changes. The RIG-5558 decisions (DL-535..DL-538, DL-561) stay whole: +repos on the git backend keep GitHub review. + +**Named decisions this record introduces:** D1–D5 in [Decisions](#decisions). + +Length: Matt lifted the length cap for this set (ruling 6 in the +[overview](README.md#rulings)). The prose sits mostly in Approach, which +freezes the review entities, the `ReviewService` API and the land hand-off +that [ci-cd](ci-cd.md) and [merge-queue](merge-queue.md) build on. + +This record designs review for Saiph: changes, versions, the stack graph, +threads, approvals, the attention set, what "ready to land" means, and the +Compass web UI hooks. It builds on the entities and RPCs of +[store](store.md) and adds the `saiph.v1.ReviewService`. + +## Open Questions + +Each question has a recommendation; the record is written against it. None +depends on another. + +### OQ-1 — Who may upload a new version of another owner's change? + +- (a) Any account with `REPO_ROLE_WRITE`. The version records its + uploader; the owner does not change. Agents fix each other's changes, + and a human can push a fix to an agent's change. +- (b) The owner only, as Critique works. Another author must make a new + change. + +**Recommendation: (a).** Fleets of agents hand work over often. The +uploader is visible on each version, and a version that changes code +drops every approval (see [Approvals](#approvals)), so (a) cannot smuggle +code past review. + +### OQ-2 — How are change classes detected? + +Scion's review tool "can express complex semantic constraints on +approvals, which lets us differentiate simple things like comment-only +changes from sensitive logical changes" +([Scion](https://www.anthropic.com/engineering/redesigning-source-control)). +The post does not publish how. + +- (a) Lexical, from the store's `Diff` hunks: whitespace-only, comment-only + (every changed line starts with a comment prefix for its file + extension), else logic. No parser. +- (b) Syntactic, with a parser per language. It needs a parser library, + which the global constraints forbid, and a grammar per language. + +**Recommendation: (a).** A lexical class can be wrong in one direction: a +changed line inside a multi-line string that starts with `//` reads as a +comment. So a class only lowers the LGTM count (a `waive` rule). It never +waives owner approval, CI or a `require` rule. A parser can replace the +classifier later without changing the API. + +### OQ-3 — May a reader whose read a `.saiphacl` narrows use review? + +- (a) No. Review reads need read on the whole repo, as the store's id + reads do ([store](store.md) OQ-3). A narrowed reader gets `NOT_FOUND`. +- (b) Yes, with every thread, diff and summary filtered per path. + +**Recommendation: (a).** Comments quote code, titles name paths, and a +filter on every review read is a leak waiting for one missed call. (b) +can come later as a supersession. + +### OQ-4 — How long do versions of closed changes stay readable? + +Open changes pin every version. A landed or abandoned change no longer +needs its old versions. + +- (a) Re-pin every version with a 90-day expiry when the change closes. +- (b) Keep them forever. +- (c) Unpin at once; gc keeps them for the repo's `retention_days`. + +**Recommendation: (a).** Interdiffs of recently landed changes stay +readable for audits and culprit finding; the bytes still count against +`saiph.repos.quota_bytes`, and they stop growing after 90 days. + +### OQ-5 — What does a repo without `.saiphreview` require? + +- (a) One LGTM from a human, plus owner coverage and CI. +- (b) One LGTM from anyone, agents included. + +**Recommendation: (a).** Critique: "every change requires an LGTM +regardless of approval status, ensuring that at least two pairs of eyes +viewed the change". With agents as authors, the second pair of eyes +defaults to a human. A repo lowers it with a `waive` rule. + +### OQ-6 — How does a Compass agent learn it is its turn? + +- (a) Compass posts a one-line message into the agent's home channel, + through the existing message delivery path. +- (b) A new `AgentControl` variant with its own ack cursor, as forge + notifications have. + +**Recommendation: (a).** The delivery path already turns a message into a +turn, with cursors, acks and retries. (b) adds proto fields, a cursor +table and an ack arm for the same effect. + +## Problem / Intent + +Saiph repos land code without GitHub, so they need their own review. +Matt ruled that review follows Critique, Twigg and jj: changes stack +freely into DAGs, and CI uses the graph (ruling 3). This record defines +the review entities, the `ReviewService` API, the readiness contract the +merge queue consumes, and the Compass UI and agent hooks. + +## Approach + +### Ground truth + +- **Saiph's repo** is `RigelBuild/saiph`. It does not exist yet, so every + Saiph path here is a plan. +- **Store.** Review builds on the store's changeset and its 16-byte + `change_id`, which a rewrite keeps ([Changeset](store.md#changeset)), + and on `GetRepo`, `GetRef`, `GetChangesets`, `Log`, `Diff`, `Rebase`, + `ReadFile`, `Pin`, `Unpin` and `WatchRefs`, plus the additions + in [Store additions](#store-additions). The store keeps no versions; + review groups changesets into versions by change id. +- **Workflow tier.** Review ships in the saiph repo as a Saiph service, as + the merge queue and CI do ([overview](README.md#open-questions) OQ-2 + (a)). If Matt rules (b), the design holds and only the paths move. +- **Conflicts** are never stored ([overview](README.md#open-questions) + OQ-3), so every version is a clean changeset. +- **Compass** keeps the web UI hooks and the agent tools. Saiph knows + Compass only as a token issuer and a caller. + +### Research and what Saiph takes from it + +| Source | What it does | Saiph takes | +| -- | -- | -- | +| Critique ([SWE book ch. 19](https://abseil.io/resources/swe-book/html/ch19.html)) | The author "uploads a snapshot (showing a patch at a particular point in time)"; reviewers "look at diffs between any pairs of snapshots"; "Comments are by default marked as unresolved"; a reply lets "a reviewer to provide a complete thought on a change atomically"; LGTM and approval are separate bits | Versions, interdiff, unresolved by default, atomic publish, LGTM apart from owner approval | +| Gerrit ([patch sets](https://gerrit-review.googlesource.com/Documentation/concept-patch-sets.html), [labels](https://gerrit-review.googlesource.com/Documentation/config-labels.html), [attention set](https://gerrit-review.googlesource.com/Documentation/user-attention-set.html)) | "Multiple git commits can share the same Change-Id"; "You can diff between other patch sets"; `copyCondition` keeps votes by `changekind:`; the attention set | Version kinds that keep approvals; the attention rules | +| Twigg ([`aa_public.go`](https://github.com/twigg-vc/monorepo/blob/a5d7461c78948f474c1d54716ed33f5d575e7d54/twigg/commit/aa_public.go), [`submit.go`](https://github.com/twigg-vc/monorepo/blob/a5d7461c78948f474c1d54716ed33f5d575e7d54/twigg/server/submit.go)) | A version number "Increased by 1 on rebases, amends, msg changes and submits"; "OWNERS files, with ownership cascading down directories"; submit checks `latestVersionOfParent.IsSubmitted` | Dense version numbers; owners that cascade; a change lands with or after its parents | +| jj ([glossary](https://docs.jj-vcs.dev/latest/glossary/)) | Changes "don't exist as an object in the data model; only the change ID does"; the "change ID generally remains the same" | The change id as the one stable name | +| Scion ([post](https://www.anthropic.com/engineering/redesigning-source-control)) | Review "can express complex semantic constraints on approvals" | Change classes and `.saiphreview` rules | +| ReviewStack ([docs](https://sapling-scm.com/docs/addons/reviewstack/)) | A "dropdown for navigating the commits in the stack", "approving each change independently" | Each change in a stack is reviewed alone; the stack view navigates | + +Twigg is AGPL-3.0; review takes its ideas, not its code. + +### Shape + +`saiph-review` (`cmd/saiph-review`) is one Go service. Its replicas share +no memory. Per-repo background work (the trunk watcher, the pin refresh +and the event sweep) runs under a lease row that any replica can take, as +the store's derive workers do. It keeps its rows in the schema +`saiph_review` and reads history only through `saiph.v1`. + +```mermaid +flowchart LR + A[saiph CLI, vcs_* tools] -->|UploadVersions, Publish| RV[saiph-review] + UI[Compass web UI] -->|ReviewService, through Compass| RV + RV -->|Rebase, CoverOwners| S[saiph-server] + RV -->|GetChangesets, Log, Diff, Pin| S + S -->|WatchRefs on trunk| RV + CI[saiph-ci] -->|ReportChecks, Publish| RV + Q[saiph-queue] -->|GetLandUnit, UploadVersions, Publish| RV + RV -->|WatchEvents| CI + RV -->|WatchEvents| Q + RV --> DB[(saiph_review)] +``` + +### Callers and tokens + +Review verifies bearer tokens with the store's `internal/auth` `Verifier` +and the same `--issuers` file, so a caller is one `auth.Principal` +([Tenancy](store.md#tenancy)). + +- **On the caller's behalf.** Review forwards the caller's bearer to + `saiph-server` for `GetRepo` and for every history read, so the store's + ACLs decide what the caller sees. Every review call needs + `caller_role` of at least `REPO_ROLE_READ` and `id_reads = true` (the + `GetRepoResponse` field [client](client.md) adds; OQ-3); otherwise it answers + `NOT_FOUND`. Writes need + `REPO_ROLE_WRITE`. +- **On its own behalf.** Pins, version-kind rebases, interdiff rebases and the + trunk watcher run as `service:saiph-review`. Review signs its own + five-minute tokens per tenant, scopes `read write`, with a key whose + public half is an issuer in `saiph-server`'s `--issuers` file. That + issuer may mint only `service:saiph-review` (`Issuer.Subjects`, a store + addition). The subject gets `REPO_ROLE_WRITE` on a repo through + `SetGrant` when review is enabled for it. It never holds `land`. + +| Subject | Caller | May | +| -- | -- | -- | +| `account:` | A human | Everything below `land`; approves | +| `account:`, `Principal.Agent` | A Compass agent session ([client](client.md)) | Author and review; its LGTM counts only where `.saiphreview` sets `lgtm_from_agents` (OQ-5); never `ReportChecks` | +| `service:merge-queue` | `saiph-queue` | `GetLandUnit`, `UploadVersions` with `VERSION_REASON_CONFLICT_FIX`, comments | +| `service:saiph-ci` | `saiph-ci` | `ReportChecks`, comments | +| `service:saiph-bridge` | The git bridge | `UploadVersions` with `VERSION_REASON_IMPORT`, `AbandonChange` of an external change | +| `service:compass` | Compass | Reads and `WatchEvents`, for attention routing (OQ-6) | + +A service never approves and never enters the attention set. Gerrit says +the same of its "Service Users": they "will never be added to the +attention set" +([attention set](https://gerrit-review.googlesource.com/Documentation/user-attention-set.html)). + +### Data model + +| Entity | Key | What it is | +| -- | -- | -- | +| Change | `(repo, change_id)` | The unit of review. Its name is the store's 16-byte `change_id`, which every rewrite keeps. It has an owner, a status (`OPEN`, `LANDED`, `ABANDONED`), reviewers, an attention set and a `land_requested` bit. | +| Version | `(repo, change_id, number)` | An immutable snapshot: one changeset plus its base, the changeset's `parents[0]`. Numbers are dense from 1. A version records its uploader, a `VersionReason` and a `VersionKind`. | +| Stack edge | `(repo, change_id, version, depends_on)` | "This version of the change sits on that open change." One row per parent of the version's changeset whose change id is an open change. | +| Thread | `(repo, change_id, thread_id)` | Comments on one anchor: a version, a path, a side and a line range. `path = ""` is a change-level thread. A thread is `UNRESOLVED` or `RESOLVED`. | +| Comment | `(repo, change_id, comment_id)` | One message in a thread: a draft that only its author sees, or published. | +| Approval | `(repo, change_id, approver, kind)` | `LGTM` or `OWNER`, valid for one version (see [Approvals](#approvals)). | +| Check | `(repo, changeset, name)` | One check result, reported by a service for a version's changeset. | + +Two graphs exist and they are different: + +- **Version history** is per change: version 1, 2, 3 and so on. A + version may rebase, amend or reword the one before it. It is a list, + not a graph, and it belongs to review alone. +- **The stack graph** joins changes. Change B depends on change A when + B's current version has a parent whose change id is A, and A is open. + The edges are a DAG: a change may depend on two changes, and two + changes may depend on one (ruling 3). Edges belong to a version, so a + rebase of B onto trunk drops the edge in B's next version without + touching A. + +A landed or abandoned change is never a dependency. The stack of a +change is the connected part of the open-change graph that holds it. + +### Uploading versions + +`UploadVersions(repo, heads, reason, reviewers, request_land, owner)` +turns changesets into versions. The client has already pushed the +changesets through the store's intake ([Intake](store.md#intake)). + +1. Review checks the caller (see [Callers and tokens](#callers-and-tokens)) + and reads the trunk head with `GetRef`. +2. It walks `Log(heads, stop = [trunk head])` with the caller's bearer. + The result is every changeset between trunk and the heads, at most 100 + (`INVALID_ARGUMENT` above). A head already on trunk is + `FAILED_PRECONDITION`. +3. For each changeset, parents first, it reads the `change_id`. Two + changesets with one change id in one upload are a divergence: + `INVALID_ARGUMENT` naming the id. +4. A changeset whose change id is new creates a change. Its owner is the + caller's account, or the request's `owner` for an import (below). +5. A changeset that equals the change's current version is skipped, so + a retry is a no-op. Any other changeset becomes the next version. An + upload to an abandoned change restores it. An upload to a landed change + is `FAILED_PRECONDITION`. +6. Review pins the new changesets (see [Pins](#pins)) and then, in one + transaction, writes the versions, their stack edges, the version kind, + the carried approvals, the attention changes and the events. +7. `reviewers` join every uploaded change's reviewer list and attention + set. `request_land = true` sets `land_requested` on each head's change. + +An upload by a non-owner is allowed (OQ-1 (a)); the version records the +uploader. The first account to upload a version of a change owned by +`external:*` becomes its owner. + +**Imports.** Only a service caller may set `owner`, and a service caller +must send `reason = VERSION_REASON_IMPORT` with `owner = +"external:github:"`. The external author stays in the changeset's +`author`. An external owner can neither approve nor request a land; a +human with `REPO_ROLE_WRITE` requests the land +([git-bridge](git-bridge.md) OQ-1). A service may abandon only a +change whose owner is `external:*`. + +**Version kind.** Review compares the new version N with the one before +it, P, as Gerrit's `changekind:` does: + +| Kind | When | Effect on approvals | +| -- | -- | -- | +| `FIRST` | N is version 1 | None yet | +| `MESSAGE_ONLY` | Same base and same root tree; only the message or author differs | Carried | +| `TRIVIAL_REBASE` | Base differs; `Rebase(P, onto = N.base)` is clean and its root tree equals N's, and the messages match | Carried | +| `CODE` | Anything else, or the rebase conflicts | Dropped, unless a conflict fix carries them | + +The rebase runs as `service:saiph-review`. It writes an unpinned +changeset that gc removes after `retention_days`; review keeps the id in +`saiph_review.interdiffs` and reuses it for the interdiff below. + +**Issue trailer.** A change names a tracker issue with an `Issue:` line +in the message trailer, one line per issue, the value verbatim (for +example `Issue: RIG-5821`). Review parses only that key and returns the +values in `ChangeSummary.issues`; Compass reads them with `refs.Parse`. +`vcs_submit` writes its `issues` there ([client](client.md)). + +### Interdiff + +`Interdiff(change_id, from_version, to_version)` diffs any two versions +of a change, as Critique lets reviewers "look at diffs between any pairs +of snapshots". `from_version = 0` means the base of `to_version`, which is +the plain diff of one version. + +- **Same base.** Review calls the store's `Diff(from = A, to = B)` with + the caller's bearer, so `.saiphacl` read rules apply as usual. +- **Different bases.** Review rebases A onto B's base (cached in + `saiph_review.interdiffs`) and diffs the result against B. The answer + sets `base_moved`, and the UI says so above the file list. If the + rebase conflicts, review diffs A against B directly and sets + `rebase_conflicted`; the diff then includes the base's changes, and the + UI labels it. + +Review does no text diffing itself. Every hunk comes from the store's +`Diff`, which uses `internal/textdiff`. + +### Pins + +Review keeps every version of an open change alive with +`Pin(holder = "review:", changesets, ttl_seconds = +2592000)`, the holder form the store's `PinRequest` comment gives. The +pin worker renews every open change's pins once a day, so a pin never +lapses while the change is open. + +When a change lands or is abandoned, review pins its versions again with +a 90-day expiry (OQ-4) and stops renewing them. After that, gc decides. +Pinned bytes count against `saiph.repos.quota_bytes`, so the store's quota +caps them ([Caps and spend](store.md#caps-and-spend)). + +### Approvals + +Approval is four gates, kept apart as Critique keeps LGTM apart from +approval. Critique's LGTM means "I have reviewed this change, believe +that it meets our standards, and I think it is okay to commit it after +addressing unresolved comments"; its approval means "as a gatekeeper, I +allow this change to be committed to the codebase" +([SWE book ch. 19](https://abseil.io/resources/swe-book/html/ch19.html)). + +| Gate | Satisfied when | Blocker | +| -- | -- | -- | +| LGTM | The policy's LGTM count is met by accounts other than the owner and the current version's uploader | `NEEDS_LGTM` | +| Owner | Every changed path is covered by an `OWNER` approval from an owner of its scope | `NEEDS_OWNER`, with the uncovered paths and suggested owners | +| Checks | The checks for the version's changeset are `complete`, and every required check is `SUCCESS` | `CHECKS_PENDING` or `CHECKS_FAILED` | +| Semantic rules | Every `require` rule in `.saiphreview` that matches the change holds | `NEEDS_LGTM` with the rule's paths | + +Only an account with `REPO_ROLE_WRITE` approves, never a service. An +agent's LGTM counts only where the policy sets `lgtm_from_agents` +(OQ-5). An agent's `OWNER` approval counts only where an `owners` entry +names its account directly, never through a `role:` entry. The owner may +give the `OWNER` approval on its own change, as in Critique; it never +gives the LGTM. + +**Owners.** Owners live in `.saiphacl`, so one file sets who reads, +who writes and who approves a directory. The file gains an optional +`owners` list ([Store additions](#store-additions)). A missing `owners` +key inherits it from the enclosing scope, the way `read` and `write` do; +at the top, the owners are the scope's `write` list. An owner entry +counts only if it also passes the scope's `write` check, so `owners` can +narrow `write` but never widen it. Coverage is checked under the +version's own ACL manifest and under the trunk head's, as the store +checks writes, so a change cannot make its author an owner of its own +paths. Review asks the store with `CoverOwners`; the store matches +`role:` entries against its grants. + +**Policy file.** `.saiphreview` sits at the repo root and is read at the +trunk head, as the queue reads `.saiphqueue`. It is JSON: + +```json +{ + "lgtm": 1, + "lgtm_from_agents": false, + "checks": true, + "carry_on_fix": true, + "rules": [ + { "waive": "lgtm", "class": "comment", "paths": ["docs/"] }, + { "require": "lgtm", "count": 2, "paths": ["internal/auth/"] }, + { "require": "lgtm_from", "subjects": ["account:acct_1"], "paths": [".saiphreview"] } + ] +} +``` + +- Without the file, the defaults above apply (OQ-5 (a)). +- `checks: false` drops the check gate, for a repo with no CI. +- `carry_on_fix` is the merge queue's switch (see + [Land hand-off](#land-hand-off)). +- A `waive` rule only lowers the LGTM count to zero, and only when the + change's class is at or below `class` and every changed path is under + `paths` (empty means every path). It never waives owner coverage, + checks or a `require` rule (OQ-2). +- A `require` rule matches when any changed path is under `paths`. + `lgtm` raises the count; `lgtm_from` needs one LGTM from a listed + subject (`account:` or `role:`). +- A version whose tree holds a malformed `.saiphreview` gets the blocker + `POLICY_INVALID`, so a malformed file never reaches trunk. + +**Change classes.** Review classifies each version's diff against its +base from the store's `Diff` hunks, with no parser (OQ-2 (a)). The class +of a file is the highest of its changed lines; the class of a change is +the highest of its files. + +| Class | A changed line is in it when | +| -- | -- | +| `WHITESPACE` | It differs from its pair only in whitespace | +| `COMMENT` | It is blank or starts, after indentation, with the comment prefix of its file type (`//` for `.go`, `.ts`, `.tsx`, `.js`, `.rs`, `.proto`; `#` for `.py`, `.sh`, `.yaml`, `.yml`, `.toml`, `.nix`; `--` for `.sql`), or its file is `.md` | +| `LOGIC` | Anything else, every binary file, every added, deleted or copied file, and every unknown file type | + +**Stickiness.** An approval belongs to the version it was given on. +It carries to a new version of kind `MESSAGE_ONLY` or `TRIVIAL_REBASE`, +and to a `CONFLICT_FIX` version under the rules in +[Land hand-off](#land-hand-off). A `CODE` version drops every approval, +and the approvers enter the attention set. + +### Threads and publishing + +A thread anchors to one version, one path, one side and a line range. +`path = ""` makes it a change-level thread. Critique notes that +"Comments are by default marked as unresolved"; Saiph does the same, and +an unresolved thread blocks readiness (`UNRESOLVED_THREADS`). A thread +stays on its version; the UI lists threads from earlier versions with +their version number. + +Drafts are private. `SaveDraft` stores one; `Publish` sends a whole +reply at once, so "a reviewer [can] provide a complete thought on a +change atomically". One `Publish` call, in one transaction: + +1. turns the named drafts and the inline `comments` into comments; +2. sets each touched thread's resolution; +3. adds `approve` approvals for the version and removes `revoke` ones; +4. adds `reviewers_add` and applies the attention rules below; +5. writes the events. + +`client_request_id` makes a retry return the first result, as Compass's +`AppendMessage` does with its own key. `version` must be the change's +current version for `approve`; comments may anchor to any version. + +### Attention set + +The attention set names who must act next. The rules follow Gerrit's: +"If reviewers are added to a change, then they are added to the +attention set"; "Replying (commenting, voting or just writing a change +message) removes the replying user from the attention set"; "If a +reviewer replies, then the change owner (and uploader) are added to the +attention set" +([attention set](https://gerrit-review.googlesource.com/Documentation/user-attention-set.html)). +Saiph adds: + +- When the owner replies, the authors of the threads it replied to are + added. +- A `CODE` version adds every approver whose approval it dropped. +- A complete check set with a failed required check adds the owner. +- `attention_add` and `attention_remove` on `Publish` set it by hand. +- A landed or abandoned change has an empty set. +- A service and an `external:*` owner never enter it. + +### Readiness + +A change is **ready** when it is open, its current version passes every +gate in [Approvals](#approvals), no thread is unresolved, and every parent +of its changeset is landable. A parent is landable when it is on trunk, +is the current version of an open change, or is a version of a landed +change. Any other parent (an old version of an open change, or a version +of an abandoned change) is the blocker `STALE_PARENT`. + +A change is **land ready** when it has `land_requested`, it is ready, and +every open change it depends on, transitively, is ready. A dependency +does not need `land_requested`: landing a change lands what it sits on. +Review recomputes readiness for a change and every change that depends +on it in the transaction that changes any input, and writes a +`READINESS_CHANGED` event when `ready` or `land_ready` flips. + +### Land hand-off + +This is the whole contract with [merge-queue](merge-queue.md). The queue pulls; review +never calls the queue. + +- **Request.** The owner, or any account with `REPO_ROLE_WRITE`, calls + `RequestLand(change_id)`, or uploads with `request_land = true`. + `cancel = true` clears it. `fix_needs_approval = true` makes any + conflict fix of this change need new approvals (merge queue OQ-2 (a)). +- **Admission.** The queue tails `WatchEvents(kinds = + [EVENT_KIND_READINESS_CHANGED])`. On `land_ready = true` it calls + `GetLandUnit(change_id)`. The land unit is the change plus its open + dependencies. The answer is `ready` and the unit's `LandEntry` rows in + dependency order, parents first, or the blockers. +- **Before the trunk move.** The queue reads the unit again just before + its trunk CAS and lands only if it is still ready with the same + `(change_id, version)` set. +- **Conflict fix.** The queue's fixer uploads with `reason = + VERSION_REASON_CONFLICT_FIX` as `service:merge-queue`. The fix takes + the check results of the version it fixes; the queue's own CI run tests + the fix before it lands. If `carry_on_fix` holds and the change has no + `fix_needs_approval`, LGTM and `OWNER` approvals carry, and review runs + the semantic rules again on the fix diff (the interdiff from the fixed + version): a `require` rule that matches a `LOGIC` path in the fix diff + needs a new approval on the fix. Otherwise no approval carries, the + change is no longer ready, and the prior approvers enter the attention + set. +- **Eject.** The queue `Publish`es a change-level comment (`path = ""`, + `RESOLUTION_UNRESOLVED`) with `attention_add = [""]`. The + unresolved thread makes the change not ready until the owner resolves + it. CI does the same for a culprit ([ci-cd](ci-cd.md)). +- **Landing.** Review learns of a landing from the trunk ref log, never + from the queue (below). + +### Landing detection + +One trunk watcher per repo, under a lease row in `saiph_review.cursors`, +tails `WatchRefs(names = [trunk_ref])` from its stored `log_seq`. For +each entry it walks `Log(heads = [new_target], stop = [old_target])` +without `first_parent`, because a land unit may hold a merge: `chain` and +its `trunk_seqs` cover only the first-parent path, and a side member is +reachable from the merge but is not in `chain`. A changeset on the +first-parent path takes its own `trunk_seq`. A side changeset takes the +`trunk_seq` of the first `chain` entry, oldest first, whose history +reaches it. For each changeset whose change id names an open change: + +- the changeset is the current version: the change becomes `LANDED`; +- otherwise (the queue rebased it): review adds a version with + `VERSION_REASON_LANDED` for that changeset, then marks the change + `LANDED`. + +The change records `landed_changeset` and `landed_trunk_seq`, its pins +move to the 90-day expiry, and review writes `EVENT_KIND_LANDED`. The +cursor advances in the same transaction, so a crash replays the entry +and the replay is a no-op. + +### Events + +`WatchEvents(repo, after_seq, kinds)` streams `ReviewEvent`s in `seq` +order. `seq` is per repo, taken from `saiph_review.repo_state` in the +transaction that writes the event, so the order is the commit order. The +stream is a poller over the table every 250 ms, like the store's +`WatchRefs`. Events are kept 30 days; an `after_seq` older than the +oldest kept event answers `OUT_OF_RANGE`, and the consumer rescans with +`ListChanges`, whose `event_seq` is the cursor to resume from. + +| Kind | When | Extra fields | +| -- | -- | -- | +| `VERSION_UPLOADED` | A version is written | `version`, `changeset`, `base`, `depends_on` | +| `ABANDONED`, `RESTORED` | Status changes | — | +| `LANDED` | The trunk watcher marks it | `landed_changeset`, `trunk_seq` | +| `READINESS_CHANGED` | `ready` or `land_ready` flips | `land_ready` | +| `ATTENTION_CHANGED` | Someone enters the attention set | `attention_added` | + +### Tables + +Review keeps its rows in its own schema, `saiph_review`, in the store's +database. It follows the store's tenancy exactly: a `tenant_id` column +defaulted from `saiph.tenant_id`, `ENABLE` and `FORCE ROW LEVEL +SECURITY`, and a `tenant_isolation` policy on every table. Request +handlers run in `InTenant`; the watcher, pin and sweep workers use a +BYPASSRLS role, as the store's gc and derive workers do. The SQL runs on +YugabyteDB YSQL and on Postgres 16. + +Why a schema of its own and not tables in `saiph`: review is its own +binary with its own migrations (`internal/review/migrations/`), as CI is +with `saiph_ci`. A review migration never touches the store's schema. +Review keys rows by the store's `repo_id`, read from `saiph.repos` with a +`SELECT`-only grant; a sweep drops rows whose repo is gone. This rests on +the overview's OQ-2 (a): if the workflow services move out of the saiph +repo, they also move to their own database. + +```sql +-- internal/review/migrations/0001_saiph_review.sql; every table has +-- tenant_id TEXT NOT NULL DEFAULT current_setting('saiph.tenant_id', TRUE), +-- ENABLE and FORCE ROW LEVEL SECURITY and a tenant_isolation policy. +CREATE SCHEMA saiph_review; +CREATE TABLE saiph_review.repo_state (tenant_id TEXT, repo_id BIGINT, + event_seq BIGINT NOT NULL, PRIMARY KEY (tenant_id, repo_id)); +CREATE TABLE saiph_review.changes (tenant_id TEXT, repo_id BIGINT, + change_id BYTEA CHECK (length(change_id) = 16), owner TEXT NOT NULL, + status SMALLINT NOT NULL, version INT NOT NULL, title TEXT NOT NULL, + issues TEXT[], land_requested BOOLEAN NOT NULL, fix_needs_approval BOOLEAN NOT NULL, + ready BOOLEAN NOT NULL, land_ready BOOLEAN NOT NULL, next_id BIGINT NOT NULL, + landed_changeset BYTEA, landed_trunk_seq BIGINT, + created_unix_ms BIGINT NOT NULL, updated_unix_ms BIGINT NOT NULL, closed_unix_ms BIGINT, + PRIMARY KEY (tenant_id, repo_id, change_id)); +CREATE INDEX changes_by_owner ON saiph_review.changes (tenant_id, owner, status); +CREATE TABLE saiph_review.versions (tenant_id TEXT, repo_id BIGINT, change_id BYTEA, + version INT, changeset BYTEA NOT NULL CHECK (length(changeset) = 32), base BYTEA, + uploader TEXT NOT NULL, reason SMALLINT NOT NULL, kind SMALLINT NOT NULL, + class SMALLINT NOT NULL, created_unix_ms BIGINT NOT NULL, + PRIMARY KEY (tenant_id, repo_id, change_id, version), + UNIQUE (tenant_id, repo_id, changeset)); +CREATE TABLE saiph_review.stack_edges (tenant_id TEXT, repo_id BIGINT, change_id BYTEA, + version INT, depends_on BYTEA, parent BYTEA NOT NULL, + PRIMARY KEY (tenant_id, repo_id, change_id, version, depends_on)); +CREATE INDEX stack_edges_by_dep ON saiph_review.stack_edges (tenant_id, repo_id, depends_on); +CREATE TABLE saiph_review.reviewers (tenant_id TEXT, repo_id BIGINT, change_id BYTEA, + subject TEXT, in_attention BOOLEAN NOT NULL, attention_since_unix_ms BIGINT, + PRIMARY KEY (tenant_id, repo_id, change_id, subject)); +CREATE INDEX attention_by_subject ON saiph_review.reviewers (tenant_id, subject) WHERE in_attention; +CREATE TABLE saiph_review.threads (tenant_id TEXT, repo_id BIGINT, change_id BYTEA, + thread_id BIGINT, version INT NOT NULL, path TEXT NOT NULL, side SMALLINT NOT NULL, + start_line INT NOT NULL, line_count INT NOT NULL, resolved BOOLEAN NOT NULL, + author TEXT NOT NULL, PRIMARY KEY (tenant_id, repo_id, change_id, thread_id)); +CREATE TABLE saiph_review.comments (tenant_id TEXT, repo_id BIGINT, change_id BYTEA, + comment_id BIGINT, thread_id BIGINT NOT NULL, author TEXT NOT NULL, body TEXT NOT NULL, + suggestion TEXT, published_unix_ms BIGINT NOT NULL, + PRIMARY KEY (tenant_id, repo_id, change_id, comment_id)); +CREATE TABLE saiph_review.drafts (tenant_id TEXT, repo_id BIGINT, change_id BYTEA, + draft_id BIGINT, author TEXT NOT NULL, thread_id BIGINT NOT NULL, version INT NOT NULL, + path TEXT NOT NULL, side SMALLINT NOT NULL, start_line INT NOT NULL, line_count INT NOT NULL, + body TEXT NOT NULL, suggestion TEXT, resolution SMALLINT NOT NULL, updated_unix_ms BIGINT NOT NULL, + PRIMARY KEY (tenant_id, repo_id, change_id, draft_id)); +CREATE TABLE saiph_review.publishes (tenant_id TEXT, author TEXT, client_request_id TEXT, + repo_id BIGINT NOT NULL, response BYTEA NOT NULL, PRIMARY KEY (tenant_id, author, client_request_id)); +CREATE TABLE saiph_review.approvals (tenant_id TEXT, repo_id BIGINT, change_id BYTEA, + approver TEXT, kind SMALLINT, version INT NOT NULL, given_on INT NOT NULL, + PRIMARY KEY (tenant_id, repo_id, change_id, approver, kind)); +CREATE TABLE saiph_review.checks (tenant_id TEXT, repo_id BIGINT, changeset BYTEA, + name TEXT, reporter TEXT NOT NULL, state SMALLINT NOT NULL, required BOOLEAN NOT NULL, + details_url TEXT, summary TEXT, updated_unix_ms BIGINT NOT NULL, + PRIMARY KEY (tenant_id, repo_id, changeset, name)); +CREATE TABLE saiph_review.check_sets (tenant_id TEXT, repo_id BIGINT, changeset BYTEA, + reporter TEXT, complete BOOLEAN NOT NULL, PRIMARY KEY (tenant_id, repo_id, changeset, reporter)); +CREATE TABLE saiph_review.events (tenant_id TEXT, repo_id BIGINT, seq BIGINT, + change_id BYTEA NOT NULL, kind SMALLINT NOT NULL, event BYTEA NOT NULL, at_unix_ms BIGINT NOT NULL, + PRIMARY KEY (tenant_id, repo_id, seq)); +CREATE TABLE saiph_review.interdiffs (tenant_id TEXT, repo_id BIGINT, from_changeset BYTEA, + onto BYTEA, result BYTEA, conflicted BOOLEAN NOT NULL, + PRIMARY KEY (tenant_id, repo_id, from_changeset, onto)); +CREATE TABLE saiph_review.cursors (tenant_id TEXT, repo_id BIGINT, stream TEXT, + seq BIGINT NOT NULL, lease_token TEXT, lease_until_unix_ms BIGINT, + PRIMARY KEY (tenant_id, repo_id, stream)); -- streams: "trunk", "pins", "sweep" +``` + +`events.event` is the encoded `ReviewEvent`. A check name has one +reporter: a second reporter for the same name and changeset is +`ALREADY_EXISTS`. + +### API: `proto/saiph/v1/review.proto` + +The package is `saiph.v1`, beside the store's `saiph.proto`, CI's +`ci.proto` and the queue's `queue.proto`. Reads carry `idempotency_level += NO_SIDE_EFFECTS`. Review messages carry no prefix. + +```proto +// proto/saiph/v1/review.proto +service ReviewService { + rpc UploadVersions(UploadVersionsRequest) returns (UploadVersionsResponse); + rpc AbandonChange(AbandonChangeRequest) returns (AbandonChangeResponse); + rpc RestoreChange(RestoreChangeRequest) returns (RestoreChangeResponse); + rpc GetChange(GetChangeRequest) returns (GetChangeResponse) { option idempotency_level = NO_SIDE_EFFECTS; } + rpc GetChanges(GetChangesRequest) returns (GetChangesResponse) { option idempotency_level = NO_SIDE_EFFECTS; } + rpc ListChanges(ListChangesRequest) returns (ListChangesResponse) { option idempotency_level = NO_SIDE_EFFECTS; } + rpc GetStack(GetStackRequest) returns (GetStackResponse) { option idempotency_level = NO_SIDE_EFFECTS; } + rpc Interdiff(InterdiffRequest) returns (InterdiffResponse) { option idempotency_level = NO_SIDE_EFFECTS; } + rpc ListThreads(ListThreadsRequest) returns (ListThreadsResponse) { option idempotency_level = NO_SIDE_EFFECTS; } + rpc SaveDraft(SaveDraftRequest) returns (SaveDraftResponse); + rpc DeleteDraft(DeleteDraftRequest) returns (DeleteDraftResponse); + rpc ListDrafts(ListDraftsRequest) returns (ListDraftsResponse) { option idempotency_level = NO_SIDE_EFFECTS; } + rpc Publish(PublishRequest) returns (PublishResponse); + rpc RequestLand(RequestLandRequest) returns (RequestLandResponse); + rpc ReportChecks(ReportChecksRequest) returns (ReportChecksResponse); + rpc GetLandUnit(GetLandUnitRequest) returns (GetLandUnitResponse) { option idempotency_level = NO_SIDE_EFFECTS; } + rpc WatchEvents(WatchEventsRequest) returns (stream ReviewEvent); +} +enum ChangeStatus { CHANGE_STATUS_UNSPECIFIED = 0; CHANGE_STATUS_OPEN = 1; CHANGE_STATUS_LANDED = 2; CHANGE_STATUS_ABANDONED = 3; } +enum VersionReason { VERSION_REASON_UNSPECIFIED = 0; VERSION_REASON_UPLOAD = 1; VERSION_REASON_CONFLICT_FIX = 2; VERSION_REASON_LANDED = 3; VERSION_REASON_IMPORT = 4; } +enum VersionKind { VERSION_KIND_UNSPECIFIED = 0; VERSION_KIND_FIRST = 1; VERSION_KIND_MESSAGE_ONLY = 2; VERSION_KIND_TRIVIAL_REBASE = 3; VERSION_KIND_CODE = 4; } +enum ChangeClass { CHANGE_CLASS_UNSPECIFIED = 0; CHANGE_CLASS_WHITESPACE = 1; CHANGE_CLASS_COMMENT = 2; CHANGE_CLASS_LOGIC = 3; } +enum ApprovalKind { APPROVAL_KIND_UNSPECIFIED = 0; APPROVAL_KIND_LGTM = 1; APPROVAL_KIND_OWNER = 2; } +enum Side { SIDE_UNSPECIFIED = 0; SIDE_BASE = 1; SIDE_VERSION = 2; } +enum Resolution { RESOLUTION_UNSPECIFIED = 0; RESOLUTION_UNRESOLVED = 1; RESOLUTION_RESOLVED = 2; } +enum CheckState { CHECK_STATE_UNSPECIFIED = 0; CHECK_STATE_PENDING = 1; CHECK_STATE_RUNNING = 2; CHECK_STATE_SUCCESS = 3; CHECK_STATE_FAILURE = 4; CHECK_STATE_CANCELLED = 5; CHECK_STATE_SKIPPED = 6; } +enum BlockerKind { BLOCKER_KIND_UNSPECIFIED = 0; BLOCKER_KIND_NOT_OPEN = 1; BLOCKER_KIND_NEEDS_LGTM = 2; BLOCKER_KIND_NEEDS_OWNER = 3; BLOCKER_KIND_CHECKS_PENDING = 4; BLOCKER_KIND_CHECKS_FAILED = 5; BLOCKER_KIND_UNRESOLVED_THREADS = 6; BLOCKER_KIND_STALE_PARENT = 7; BLOCKER_KIND_DEPENDENCY_NOT_READY = 8; BLOCKER_KIND_POLICY_INVALID = 9; } +enum EventKind { EVENT_KIND_UNSPECIFIED = 0; EVENT_KIND_VERSION_UPLOADED = 1; EVENT_KIND_ABANDONED = 2; EVENT_KIND_RESTORED = 3; EVENT_KIND_LANDED = 4; EVENT_KIND_READINESS_CHANGED = 5; EVENT_KIND_ATTENTION_CHANGED = 6; } + +// owner: "account:" or "external:github:". +message Change { bytes change_id = 1; string owner = 2; ChangeStatus status = 3; string title = 4; repeated string reviewers = 5; repeated string attention = 6; bool land_requested = 7; bool fix_needs_approval = 8; repeated string issues = 9; bytes landed_changeset = 10; uint64 landed_trunk_seq = 11; int64 created_unix_ms = 12; int64 updated_unix_ms = 13; } +message Version { uint32 number = 1; bytes changeset = 2; bytes base = 3; string uploader = 4; VersionReason reason = 5; VersionKind kind = 6; ChangeClass class = 7; repeated bytes depends_on = 8; int64 created_unix_ms = 9; } +message VersionRef { bytes change_id = 1; uint32 version = 2; bytes changeset = 3; } +message Approval { string approver = 1; ApprovalKind kind = 2; uint32 version = 3; uint32 given_on = 4; } +message CheckRun { string name = 1; CheckState state = 2; bool required = 3; string details_url = 4; string summary = 5; } +// paths: uncovered paths for NEEDS_OWNER; suggested: owners or subjects that would clear it. +message Blocker { BlockerKind kind = 1; bytes change_id = 2; string detail = 3; repeated string paths = 4; repeated string suggested = 5; } +message Readiness { bool ready = 1; bool land_ready = 2; repeated Approval approvals = 3; repeated CheckRun checks = 4; bool checks_complete = 5; repeated Blocker blockers = 6; uint32 unresolved_threads = 7; } +// Fields 1, 3, 10 and 11 are the git bridge's contract. +message ChangeSummary { bytes change_id = 1; string title = 2; ChangeStatus status = 3; string owner = 4; uint32 version = 5; bool land_requested = 6; bool ready = 7; bool land_ready = 8; repeated string attention = 9; bytes landed_changeset = 10; uint64 landed_trunk_seq = 11; repeated string issues = 12; int64 updated_unix_ms = 13; bytes changeset = 14; repeated bytes depends_on = 15; string repo = 16; } + +// owner: service callers only, with VERSION_REASON_IMPORT. +message UploadVersionsRequest { string repo = 1; repeated bytes heads = 2; VersionReason reason = 3; repeated string reviewers = 4; bool request_land = 5; string owner = 6; } +message UploadVersionsResponse { repeated VersionRef versions = 1; } +message AbandonChangeRequest { string repo = 1; bytes change_id = 2; string reason = 3; } +message AbandonChangeResponse {} +message RestoreChangeRequest { string repo = 1; bytes change_id = 2; } +message RestoreChangeResponse {} +message GetChangeRequest { string repo = 1; bytes change_id = 2; } +message GetChangeResponse { Change change = 1; repeated Version versions = 2; Readiness readiness = 3; } +// At most 1,000 ids; an unknown or unreadable id is left out, never an error. +message GetChangesRequest { string repo = 1; repeated bytes change_ids = 2; } +message GetChangesResponse { repeated ChangeSummary changes = 1; } +// repo empty: every readable repo of the tenant; allowed only when attention or owner is the caller. +message ListChangesRequest { string repo = 1; ChangeStatus status = 2; bool land_requested = 3; string attention = 4; string owner = 5; uint32 page_size = 6; string page_token = 7; } +message ListChangesResponse { repeated ChangeSummary changes = 1; string next_page_token = 2; uint64 event_seq = 3; } +message GetStackRequest { string repo = 1; bytes change_id = 2; } +message StackEdge { bytes change_id = 1; bytes depends_on = 2; } +// At most 500 changes; truncated says more exist. +message GetStackResponse { repeated ChangeSummary changes = 1; repeated StackEdge edges = 2; bool truncated = 3; } +// from_version 0 = the base of to_version. +message InterdiffRequest { string repo = 1; bytes change_id = 2; uint32 from_version = 3; uint32 to_version = 4; repeated string paths = 5; uint32 context_lines = 6; uint32 page_size = 7; string page_token = 8; } +message InterdiffResponse { repeated FileDiff files = 1; string next_page_token = 2; bool base_moved = 3; bool rebase_conflicted = 4; bytes from_changeset = 5; bytes to_changeset = 6; } +// path "" = change-level. +message Anchor { string path = 1; Side side = 2; uint32 start_line = 3; uint32 line_count = 4; } +message Comment { uint64 comment_id = 1; string author = 2; string body = 3; string suggestion = 4; int64 published_unix_ms = 5; } +message Thread { uint64 thread_id = 1; uint32 version = 2; Anchor anchor = 3; bool resolved = 4; repeated Comment comments = 5; } +message ListThreadsRequest { string repo = 1; bytes change_id = 2; uint32 version = 3; bool unresolved_only = 4; uint32 page_size = 5; string page_token = 6; } +message ListThreadsResponse { repeated Thread threads = 1; string next_page_token = 2; } +// thread_id 0 = a new thread at anchor. +message NewComment { uint64 thread_id = 1; Anchor anchor = 2; string body = 3; Resolution resolution = 4; string suggestion = 5; } +message Draft { uint64 draft_id = 1; uint32 version = 2; NewComment comment = 3; int64 updated_unix_ms = 4; } +// draft_id 0 = a new draft. +message SaveDraftRequest { string repo = 1; bytes change_id = 2; Draft draft = 3; } +message SaveDraftResponse { Draft draft = 1; } +message DeleteDraftRequest { string repo = 1; bytes change_id = 2; uint64 draft_id = 3; } +message DeleteDraftResponse {} +message ListDraftsRequest { string repo = 1; bytes change_id = 2; } +message ListDraftsResponse { repeated Draft drafts = 1; } +message PublishRequest { string repo = 1; bytes change_id = 2; uint32 version = 3; repeated uint64 draft_ids = 4; repeated NewComment comments = 5; repeated ApprovalKind approve = 6; repeated ApprovalKind revoke = 7; repeated string attention_add = 8; repeated string attention_remove = 9; repeated string reviewers_add = 10; string client_request_id = 11; } +message PublishResponse { repeated Thread threads = 1; Readiness readiness = 2; } +message RequestLandRequest { string repo = 1; bytes change_id = 2; bool cancel = 3; bool fix_needs_approval = 4; } +message RequestLandResponse { bool ready = 1; repeated Blocker blockers = 2; } +// complete = the required set for this changeset is final. +message ReportChecksRequest { string repo = 1; bytes change_id = 2; uint32 version = 3; bytes changeset = 4; repeated CheckRun checks = 5; bool complete = 6; } +message ReportChecksResponse {} +message GetLandUnitRequest { string repo = 1; bytes change_id = 2; } +message LandEntry { bytes change_id = 1; uint32 version = 2; bytes changeset = 3; repeated bytes parents = 4; repeated bytes depends_on = 5; bool fix_needs_approval = 6; } +// entries in dependency order, parents first. +message GetLandUnitResponse { bool ready = 1; repeated LandEntry entries = 2; repeated Blocker blockers = 3; } +message WatchEventsRequest { string repo = 1; uint64 after_seq = 2; repeated EventKind kinds = 3; } +message ReviewEvent { uint64 seq = 1; bytes change_id = 2; uint32 version = 3; bytes changeset = 4; bytes base = 5; repeated bytes depends_on = 6; EventKind kind = 7; int64 at_unix_ms = 8; bytes landed_changeset = 9; uint64 trunk_seq = 10; bool land_ready = 11; repeated string attention_added = 12; } +``` + +Rules the types do not show: + +- **Checks.** A required check passes only on `SUCCESS`; `SKIPPED` on a + required check blocks. A target that affected detection drops is left + out of the set, not reported `SKIPPED`. A report for a changeset that is + no longer the current version is stored and changes nothing. Review + keeps summaries only; logs stay with the reporter. +- **`ReportChecks`** needs a service caller with `REPO_ROLE_WRITE`. +- **`GetLandUnit`** needs `REPO_ROLE_LAND`. +- **Errors.** An unknown change, or one the caller cannot read, is + `NOT_FOUND`. A write to a closed change is `FAILED_PRECONDITION`. + `Publish` with `approve` on an old version is `FAILED_PRECONDITION` + naming the current one. + +### Store additions + +Folded into [store](store.md#api-the-saiphv1-connect-package): the +`owners` list in `.saiphacl` and `CoverOwners` (store +[Per-directory ACLs](store.md#per-directory-acls), tasks S4 and S6), and +`Issuer.Subjects`, `Claims.Agent` and `Principal.Agent` (store S11). +Compass's `Minter` sets `agent: true` on agent-session tokens +([client](client.md) C9). + +### Compass web UI hooks + +The web UI lives in Compass and stays there. It follows the locked +aesthetic: Night Owl surfaces, one purple mark (the brand mark, never in +review), Departure Mono for display and Space Mono for UI, 1-bit +dot-matrix shapes, and only `--cx-*` and `--rigel-*` tokens from +`apps/ui/src/design/tokens.css` (in Compass), never a literal colour. + +- **Transport.** The browser holds a Compass session, not a Saiph token. + The Compass server mounts `saiph.v1.ReviewService` and the store's + `HistoryService` read calls under its own origin and forwards each call + with a 15-minute token it mints for the user (`sub = + account:`, scopes `read write`). The UI's + `createSaiphClients(conn)` sits beside `createLiveClients` in + `apps/ui/src/live/client.ts` and reuses its gRPC-Web transport and + bearer interceptor. Queries use `createConnectQuery`, refetching every + 15 s and on focus. +- **Routes.** `RouteMatch` in `apps/ui/src/view-route.ts` gains + `{ view: "changes" }` at `/changes` and `{ view: "change", repo, + changeId }` at `/change//`. The compared versions ride + in the query string (`?from=1&v=3`), so moving the slider never pushes + a history entry. +- **Attention dashboard** (`/changes`). Three lists from `ListChanges` + with `repo` empty: "Your turn" (`attention` is me), "Waiting" (`owner` + is me, open, not my turn) and "Ready to land" (mine, `land_ready`). + Gerrit calls its first section "Your turn"; Saiph keeps the name. +- **Change view.** A header with the title, the owner, the readiness + badge and the blockers; the stack graph; the version slider; the file + list; the diff with threads inline. +- **Readiness badge.** It follows `prLifecycle` in `board-render.ts`: + `landed`, `abandoned`, `ready` (`land_ready`), `in_review` (any + approval or thread) and `in_progress`. Check pips use a new + `reviewCheckPip(state)` beside `checkPip`: `SUCCESS` is success, + `FAILURE` and `CANCELLED` are failure, anything else is pending. +- **Stack graph.** An SVG on the dot grid with `shape-rendering: + crispEdges`. Each change is a dot; each edge is a run of dots. Layers + are depth from trunk (longest path); order inside a layer is the + change id, so the layout is stable without a layout library. The + current change is `--cx-accent`, a ready one `--cx-ok`, a blocked one + `--cx-warn`, a failed check `--cx-error`. A click opens that change. +- **Version slider.** A dot track with one tick per version and two + thumbs, `from` and `to`. A tick's glyph shows its kind: solid for + `CODE`, hollow for `TRIVIAL_REBASE` and `MESSAGE_ONLY`, a cross for + `CONFLICT_FIX`. Arrow keys move the focused thumb; the move uses + `--cx-motion-fast`, which the tokens zero for reduced motion. The + label reads "v2 → v5" in `--cx-text-sm`. When `base_moved` is set, a + one-line notice sits above the files. +- **Diff.** The file list and hunks come from `Interdiff`. The rows reuse + `SessionTrace`'s diff classes: `block-diff`, `diff-path`, + `diff-line[data-kind]`, `diff-gutter` and `diff-body`; `data-kind` + gains `ctx` for context lines. A click on a gutter opens a draft at + that line; the reply bar publishes every draft at once. + +### Agents as authors and reviewers + +Agents act as their own accounts (`account:`, `Principal.Agent`), +through the Runner relay [client](client.md) defines. The `vcs_*` tools keep +their names; on a Saiph repo they call review: + +| Tool | Review call | +| -- | -- | +| `vcs_submit` | `UploadVersions(heads, VERSION_REASON_UPLOAD, request_land = ready)`; `issues` become `Issue:` trailers | +| `vcs_retire` | `AbandonChange` | +| `vcs_pr_status` | `GetChange` and `GetStack` | + +New review tools sit in `createReviewTools(broker)` in +`packages/compass-agent/src/review.ts` (in Compass), beside +`createForgeTools`: + +| Tool | Approval | Call | +| -- | -- | -- | +| `review_my_turn` | read | `ListChanges(attention = me)` | +| `review_get_change` | read | `GetChange`, `ListThreads` | +| `review_interdiff` | read | `Interdiff` | +| `review_publish` | write | `Publish` (comments, resolutions, `LGTM`, `OWNER`, attention) | +| `review_request_land` | write | `RequestLand` | + +**Your turn** (OQ-6 (a)). A Compass worker tails `WatchEvents(kinds = +[EVENT_KIND_ATTENTION_CHANGED])` per Saiph repo as `service:compass` +under a lease, like `LeaseForgeMirror`. For each `attention_added` that +is an agent account, it calls `AppendMessage` into that agent's home +channel as the system account, with `clientRequestID = +"saiph-attention:::"`, so a replay inserts +nothing. The message is one line: the change title, its short id and +"your turn". + +An agent cannot LGTM its own change or one it uploaded the current +version of. Its LGTM counts only where `lgtm_from_agents` is set. + +### Caps and spend + +Review adds no new usage-billed meter. Its rows live in the store's +database, and the bytes it pins count against `saiph.repos.quota_bytes`. +The limits below cap row growth per call and per repo. + +| Layer | Cap | Enforced by | +| -- | -- | -- | +| Upload | 100 changesets per `UploadVersions` | `INVALID_ARGUMENT` | +| Repo | 10,000 open changes | `RESOURCE_EXHAUSTED` on the change that would be one more | +| Change | 1,000 versions, 2,000 threads, 10,000 comments, 50 reviewers | `RESOURCE_EXHAUSTED` | +| Comment | 64 KiB body and suggestion | `INVALID_ARGUMENT` | +| Checks | 500 checks per changeset | `INVALID_ARGUMENT` | +| Events | 30 days kept | the sweep worker | +| Pins | 90 days after a change closes (OQ-4) | the store's gc | +| Reads | `page_size` at most 1,000 (default 100); 16 MiB bodies | the handlers, as in the store | + +### Decisions + +- **D1 — DL-597.** A change is named by the store's 16-byte `change_id`. A + version is an immutable changeset plus its base, numbered densely from + 1. Review owns versions; the store keeps none. +- **D2 — DL-598.** The stack graph is a DAG of open-change dependencies, taken + from changeset parents and kept apart from version history. Each change + is reviewed and approved alone; it lands with or after what it sits on. +- **D3 — DL-599.** Approval is four gates: LGTM, owner coverage from the `owners` + list in `.saiphacl`, complete required checks, and `.saiphreview` + rules. Approvals carry only across message-only versions, trivial + rebases and conflict fixes the policy allows. +- **D4 — DL-600.** Review is its own service, `saiph-review`, serving + `saiph.v1.ReviewService` from schema `saiph_review`. It learns of + landings from the trunk ref log and never calls the queue; the queue + pulls land units. +- **D5 — DL-601.** Compass keeps the review UI and the agent tools. It proxies + `ReviewService` for the browser with a token it mints for the user, + and it tells an agent it is its turn with a home-channel message. + +## Plan + +### Global Constraints + +- The repo is public. No task names the private monorepo or links a + private tracker. +- Go 1.26 and `connectrpc.com/connect/v2` only. Dependencies are the + store's list (`fxamacker/cbor/v2`, `x/crypto/blake2b`, `pgx/v5`, + `minio-go/v7`, `klauspost/compress`). No diff library and no parser + library: every hunk comes from the store's `Diff`. +- Lint bans `errors.As` (use `errors.AsType[T]`), `time.Sleep`, + `fmt.Print*`, and `context.Background`/`context.TODO` outside `main` + and tests. +- SQL runs on YugabyteDB YSQL and Postgres 16. Migrations live in + `internal/review/migrations/`. Every table has `tenant_id`, `ENABLE` + and `FORCE ROW LEVEL SECURITY` and a `tenant_isolation` policy. +- Review reads history only through `saiph.v1` with the caller's bearer, + or as `service:saiph-review`; it never reads `saiph.*` tables except + `saiph.repos`. +- Sapling and EdenFS code is GPL-2.0-only and Twigg is AGPL-3.0: take + designs, copy no code. +- Compass tasks use Compass's own conventions and the UI tokens only; + no hex colour, and purple only in the brand mark. +- No git vocabulary in names: change, version, land; never "PR", + "branch" or "merge request". + +### Order + +R1 → R2 → (R3, R4) → R5 → R6 → R7 → R8; R9 and R10 after R1; R11 after +R5; R12 after R7. + +### R1 — Proto and generated code + +`proto/saiph/v1/review.proto` as in +[API](#api-protosaiphv1reviewproto), generated into the store's +`saiphv1` and `saiphv1connect` packages and its TS client. Tests: `buf +lint`, a drift check, a round-trip of each message. + +Interfaces: the proto above; `saiphv1connect.ReviewServiceHandler` and +`saiphv1connect.ReviewServiceClient`. + +### R2 — Schema and the meta layer + +The `saiph_review` migration and typed queries. Tests: RLS hides another +tenant's rows; `NextEventSeq` is gap-free under concurrent writers; a +second `Lease` holder gets `ok = false` until the first expires. + +Interfaces: + +```go +// internal/review/rdb +type DB struct{ /* *pgxpool.Pool, system pool */ } +func Open(ctx context.Context, dsn, systemDSN string) (*DB, error) +func (d *DB) InTenant(ctx context.Context, tenant string, fn func(tx pgx.Tx) error) error +func (d *DB) AsSystem(ctx context.Context, fn func(tx pgx.Tx) error) error +func (d *DB) RepoID(ctx context.Context, tenant, repo string) (int64, error) +func NextEventSeq(ctx context.Context, tx pgx.Tx, repo int64) (uint64, error) +func AppendEvent(ctx context.Context, tx pgx.Tx, repo int64, e *saiphv1.ReviewEvent) error +func (d *DB) Lease(ctx context.Context, tenant string, repo int64, stream, token string, ttl time.Duration) (seq int64, ok bool, err error) +``` + +### R3 — Store additions in `saiph-server` + +Folded into store tasks S4, S6 and S11 +([store](store.md#s4--derived-framework-trees-commit-graph-and-acls)), +which carry the tests and interfaces. + +### R4 — Version kinds and change classes + +The kind of a new version against its previous one, and the lexical +class of a diff. Tests: a reworded version is `MESSAGE_ONLY`; a clean +rebase is `TRIVIAL_REBASE`; a rebase that conflicts is `CODE`; a `//` +line in a `.go` file is `COMMENT`; an added file is `LOGIC`; a +whitespace-only hunk is `WHITESPACE`. + +Interfaces: + +```go +// internal/review/classify +func Class(files []*saiphv1.FileDiff) saiphv1.ChangeClass +func LineClass(path string, oldLine, newLine []byte) saiphv1.ChangeClass + +// internal/review/kind +type Store interface { + GetChangesets(ctx context.Context, ids [][]byte) ([]*saiphv1.ChangesetObject, error) + Rebase(ctx context.Context, changeset, onto []byte) (rewrite []byte, conflicted bool, err error) +} +func Of(ctx context.Context, s Store, prev, next *saiphv1.ChangesetObject) (saiphv1.VersionKind, error) +``` + +### R5 — Upload, abandon and the stack graph + +`UploadVersions`, `AbandonChange`, `RestoreChange`, `GetChange`, +`GetChanges`, `ListChanges` and `GetStack`, with pins and the `Issue:` +trailer. Tests: a retry adds no version; a divergent upload is +`INVALID_ARGUMENT`; an edge appears for an open parent change and not +for a trunk parent; a rebase onto trunk drops the edge; a service upload +without `VERSION_REASON_IMPORT` is `PERMISSION_DENIED`; a service may +abandon only an `external:*` change; a narrowed reader gets `NOT_FOUND`. + +Interfaces: + +```go +// internal/review/core +type Deps struct { + DB *rdb.DB + Store StoreClients // saiphv1connect clients, per-call bearer + Self *SelfToken + Policy *policy.Cache + Now func() time.Time +} +type StoreClients struct { + Repo saiphv1connect.RepoServiceClient + Object saiphv1connect.ObjectServiceClient + History saiphv1connect.HistoryServiceClient + Mutate saiphv1connect.MutationServiceClient + Ref saiphv1connect.RefServiceClient +} +type Access struct{ Principal auth.Principal; RepoID int64; Role saiphv1.RepoRole; Bearer string } +func (d *Deps) Authorize(ctx context.Context, repo string, need saiphv1.RepoRole) (Access, error) +func (d *Deps) Upload(ctx context.Context, a Access, req *saiphv1.UploadVersionsRequest) ([]*saiphv1.VersionRef, error) +func (d *Deps) Stack(ctx context.Context, a Access, change [16]byte) (*saiphv1.GetStackResponse, error) +func ParseIssues(message string) []string +type SelfToken struct{ /* signer, issuer URL */ } +func (t *SelfToken) Bearer(tenant string) (string, error) +``` + +### R6 — Threads, drafts and publish + +`SaveDraft`, `DeleteDraft`, `ListDrafts`, `ListThreads`, `Publish` and +the attention rules. Tests: a published thread starts unresolved; a +reply removes the replier and adds the owner; a reply by the owner adds +the thread authors; a retry with the same `client_request_id` returns +the stored response; a service is never added to attention; another +account's drafts are invisible. + +Interfaces: + +```go +// internal/review/core +func (d *Deps) Publish(ctx context.Context, a Access, req *saiphv1.PublishRequest) (*saiphv1.PublishResponse, error) +func Attention(change *saiphv1.Change, actor string, replyTo []string, added []string) (add, remove []string) +``` + +### R7 — Policy, approvals, checks and readiness + +The `.saiphreview` parser, the four gates, stickiness, `ReportChecks`, +`RequestLand` and readiness propagation. Tests: no file means one human +LGTM; a `waive` rule never waives owners; `SKIPPED` on a required check +blocks; a `CODE` version drops approvals and adds the approvers to +attention; a `TRIVIAL_REBASE` keeps them; a dependency that loses an +approval flips `land_ready` on every change above it. + +Interfaces: + +```go +// internal/review/policy +type Rule struct{ Waive, Require, Class string; Count int; Subjects, Paths []string } +type Policy struct{ LGTM int; LGTMFromAgents, Checks, CarryOnFix bool; Rules []Rule } +func Parse(b []byte) (Policy, error) +func Default() Policy +type Cache struct{ /* keyed by (tenant, repo, trunk head) */ } +func (c *Cache) At(ctx context.Context, a core.Access, trunkHead []byte) (Policy, error) + +// internal/review/ready +type Input struct { + Change *saiphv1.Change + Version *saiphv1.Version + Approvals []*saiphv1.Approval + Checks []*saiphv1.CheckRun + Complete bool + Uncovered []*saiphv1.UncoveredPath + Threads uint32 + Parents []ParentState + Policy policy.Policy + Changed []string // paths changed against the base +} +type ParentState struct{ Changeset []byte; OnTrunk, CurrentOpen, Landed bool } +func Evaluate(in Input) *saiphv1.Readiness +func Carry(kind saiphv1.VersionKind, reason saiphv1.VersionReason, p policy.Policy, fixNeedsApproval bool) bool +``` + +### R8 — Land hand-off and landing detection + +`GetLandUnit`, the conflict-fix path, and the trunk watcher. Tests: a +unit lists parents first; a unit with a not-ready dependency is not +ready; a fix with `carry_on_fix` keeps approvals and re-runs `require` +rules on the fix diff; a fix with `fix_needs_approval` drops them; a +landed rebased changeset adds a `VERSION_REASON_LANDED` version; the side +member of a landed merge becomes `LANDED` with the merge's `trunk_seq`; a +replayed ref-log entry is a no-op. + +Interfaces: + +```go +// internal/review/land +func LandUnit(ctx context.Context, d *core.Deps, a core.Access, change [16]byte) (*saiphv1.GetLandUnitResponse, error) +type Watcher struct{ /* *core.Deps, lease token */ } +func (w *Watcher) Run(ctx context.Context, tenant string, repo int64) error +func (w *Watcher) Apply(ctx context.Context, tx pgx.Tx, e *saiphv1.WatchRefsResponse) error +``` + +### R9 — Interdiff + +`Interdiff` with the rebase cache. Tests: same base passes through to +`Diff`; a moved base sets `base_moved` and hides the base's changes; a +conflicted rebase sets `rebase_conflicted`; a path the caller cannot read +is left out. + +Interfaces: + +```go +// internal/review/interdiff +func Compute(ctx context.Context, d *core.Deps, a core.Access, req *saiphv1.InterdiffRequest) (*saiphv1.InterdiffResponse, error) +``` + +### R10 — Events, pins worker and sweep + +`WatchEvents`, the 250 ms poller, the daily pin renewal, the 90-day +re-pin on close, the 30-day event sweep and the repo-gone sweep. Tests: +an old `after_seq` is `OUT_OF_RANGE`; `ListChanges.event_seq` resumes +with no gap; a closed change's pins carry a 90-day TTL. + +Interfaces: + +```go +// internal/review/events +func Watch(ctx context.Context, db *rdb.DB, a core.Access, req *saiphv1.WatchEventsRequest, send func(*saiphv1.ReviewEvent) error) error +// internal/review/workers +func RenewPins(ctx context.Context, d *core.Deps, tenant string, repo int64) error +func Sweep(ctx context.Context, d *core.Deps, now time.Time) error +``` + +### R11 — `cmd/saiph-review`, wiring and caps + +The binary, its flags, the Connect handlers, the caps in +[Caps and spend](#caps-and-spend), and the deploy manifest. Tests: each +cap returns its code; a token from an unknown issuer is `UNAUTHENTICATED`. + +Interfaces: + +```go +// cmd/saiph-review +type flags struct{ Listen, Server, DSN, SystemDSN, Issuers, SigningKey, IssuerURL string } +// internal/review/api +func NewHandler(d *core.Deps, v *auth.Verifier) (string, http.Handler) +``` + +### R12 — Compass: proxy, UI and agent tools (in Compass) + +The `ReviewService` proxy with a user token; `createSaiphClients`; the +`changes` and `change` routes; the attention dashboard, the stack graph, +the version slider and the diff; `createReviewTools`; the `vcs_*` +mapping; the attention worker. Tests: the proxy refuses a caller with no +session; the route round-trips through `parseRoute` and `routePath`; the +stack layout is stable for a fixed input; the slider moves by keyboard; +the attention worker's replay inserts no second message; `vcs_submit` +writes `Issue:` trailers. + +Interfaces: + +```ts +// apps/ui/src/live/client.ts (in Compass) +export function createSaiphClients(conn: ResolvedConnection): SaiphClients; +// apps/ui/src/view-route.ts (in Compass) +type RouteMatch = /* existing */ | { view: "changes" } | { view: "change"; repo: string; changeId: string }; +// apps/ui/src/board-render.ts (in Compass) +export function reviewCheckPip(state: CheckState): "success" | "failure" | "pending"; +export function changeLifecycle(s: ChangeSummary): "landed" | "abandoned" | "ready" | "in_review" | "in_progress"; +// packages/compass-agent/src/review.ts (in Compass) +export function createReviewTools(broker: ReviewBroker): Tool[]; +``` + +```go +// go/internal/saiphreview (in Compass) +func NewProxy(minter *saiphtoken.Minter, upstream string) http.Handler +func RunAttention(ctx context.Context, st *store.Store, c saiphv1connect.ReviewServiceClient, repo string) error +``` + +## Tasks + +- [ ] R1 — `review.proto` and generated Go and TS +- [ ] R2 — `saiph_review` schema, RLS, event seq, leases +- [ ] R3 — store additions (done in store S4, S6 and S11) +- [ ] R4 — version kinds and change classes +- [ ] R5 — upload, abandon, restore, reads, stack graph, pins, `Issue:` trailer +- [ ] R6 — threads, drafts, publish, attention +- [ ] R7 — `.saiphreview`, approvals, checks, readiness, `RequestLand` +- [ ] R8 — `GetLandUnit`, conflict fixes, trunk watcher +- [ ] R9 — interdiff +- [ ] R10 — `WatchEvents`, pin renewal, sweep +- [ ] R11 — `cmd/saiph-review`, wiring, caps +- [ ] R12 — Compass: proxy, UI views, agent tools, attention messages diff --git a/docs/designs/saiph/store.md b/docs/designs/saiph/store.md new file mode 100644 index 0000000..bf53397 --- /dev/null +++ b/docs/designs/saiph/store.md @@ -0,0 +1,1519 @@ +# Saiph: store and server + +Tracker: RIG-5665 + +> Freezes on merge; later changes supersede by citation, never rewrite +> (convention: the `docs/designs/agent/compass-agent-config-delivery/design.md` +> header). Unless a path says otherwise, every code path is in the +> `RigelBuild/saiph` repo, which does not exist yet. A Compass path says +> "in Compass". + +Ledger-impact: DL-578, DL-579 and DL-581 file D1–D3. They replace the +decisions of the deleted `internal-vcs.md` record, which made a git mirror +the store. DL-535, DL-090, DL-326 and DL-561 stay whole: they govern the git +backend, and this record designs Saiph only +([overview](README.md#rulings), ruling 5). The store additions that +[client](client.md), [review](review.md), [ci-cd](ci-cd.md), +[merge-queue](merge-queue.md) and [git-bridge](git-bridge.md) asked for +are folded into this record, and each of those records points here. + +**Named decisions this record introduces:** D1–D3 in [Decisions](#decisions). + +Length: Matt lifted the length cap for this set (ruling 6 in the +[overview](README.md#rulings)). The prose sits mostly in Approach, which +freezes the formats every sibling record builds on. + +Saiph is the version control system Rigel sells. Customers self-host it or +use it through Compass, and the Compass repo is its first user. This record +owns the entities and RPCs the sibling records share: [client](client.md), +[review](review.md), [ci-cd](ci-cd.md), [git-bridge](git-bridge.md) and [merge-queue](merge-queue.md) in this +directory. The [overview](README.md) holds the glossary and the component +map. + +## Open Questions + +Matt ruled on 2026-10-18 that Saiph's core follows Scion to the letter. +Where the Scion post is silent, Mononoke, Sapling and EdenFS +(`facebook/sapling`) are the reference. Every departure is listed in +[Deviations from Scion](#deviations-from-scion) or +[Gaps filled from Mononoke and Sapling](#gaps-filled-from-mononoke-and-sapling). +The record is written against each recommendation below. + +Settled by those rulings, so no longer open: the content hash is +BLAKE2b-256 keyed per object type, as Mononoke's (was OQ-1); the server is +its own `saiph-server` binary in the saiph repo (was OQ-5). + +### OQ-1 — Which metadata database? (load-bearing) + +Scion keeps changesets, refs and derived data in Spanner. A ref move "is a +compare-and-swap on a single Spanner row", and servers "compute derived +data on the fly and write the results back to Spanner". Spanner is a +Google Cloud service, so a self-hosted product cannot ship on it. Scion +also holds that "Everything must be horizontally scalable." + +- (a) **YugabyteDB (YSQL).** The closest self-hosted equivalent of + Spanner. "The transaction design is based on the Google Spanner + architecture" + ([yugabyte-db README](https://github.com/yugabyte/yugabyte-db)). YSQL + "reuses the PostgreSQL query layer" and supports row-level security + ([docs](https://docs.yugabyte.com/preview/secure/authorization/row-level-security/)). + Writes scale out across nodes. The repo "contains two differently + licensed components": the core database is Apache-2.0, and YugabyteDB + Anywhere, the management platform, is under the Polyform Free Trial + License 1.0.0. Saiph runs the core database only. +- (b) **Postgres ≥16 with one primary** (CloudNativePG on Kubernetes). + Familiar, and Compass already runs it. One primary bounds write + throughput, which fails Scion's scaling principle. +- (c) **CockroachDB.** Spanner-inspired and write-scalable, but its free + tier under the CockroachDB Software License is + ["throttled after 7 days without sending telemetry"](https://www.cockroachlabs.com/docs/stable/licensing-faqs), + which a self-hosted product cannot accept. +- (d) **FoundationDB** (Apache-2.0). Scales, but has no SQL, + ["Transaction size cannot exceed 10,000,000 bytes of affected data"](https://apple.github.io/foundationdb/known-limitations.html), + and transactions end after five seconds. + +**Recommendation: (a), with (b) allowed on one node.** The schema and +queries use only what YSQL and Postgres 16 both run, and CI runs the store +suite on both (S2). A deployment with many servers (ruling 1) runs +YugabyteDB. Dogfood and small self-hosted installs may run one Postgres. +Postgres does not win alone: its edge is familiarity, and it fails the +scaling principle. The design uses no LISTEN/NOTIFY, so behavior is the +same on both. + +### OQ-2 — Which bytes are hashed? (load-bearing) + +Scion does not say. Mononoke hashes the Thrift serialization of each +object. Saiph creates every changeset on the server, as Scion does +("Changesets are created server-side"), so only `saiph-server` encodes; +clients decode and may verify. + +- (a) **Deterministic CBOR** ([RFC 8949 §4.2.1](https://www.rfc-editor.org/rfc/rfc8949#section-4.2.1), + "Core Deterministic Encoding Requirements") with integer keys, via + `fxamacker/cbor` (MIT). The determinism rule is in the RFC, so id + stability does not rest on one library's behavior. +- (b) **Thrift**, as Mononoke. It follows the reference exactly, but it + adds the Thrift compiler beside buf. +- (c) **Protobuf.** Not an option for hashing + ([protobuf.dev](https://protobuf.dev/programming-guides/serialization-not-canonical/): + "Protobuf serialization is not (and cannot be) canonical"). + +**Recommendation: (a).** It departs from the gap-fill reference, so the +gaps table lists it. Test vectors in the repo (S1) bind every decoder. + +### OQ-3 — May a reader fetch an object by id alone? (security) + +Scion's caches answer by id: "Every request is for a specific object that +never changes, so any cache can answer it". A blob id names content, not a +path, so a reader without access to a restricted directory could ask for a +guessed id and learn that the content exists. + +- (a) Id reads (`GetBlobs`, `GetTrees`, `GetChangesets`) need read on the + whole repo. Readers with partial access use path reads (`ReadTree`, + `ReadFile`), which check the ACL for each path. +- (b) All reads go by path. Caches lose cheap id lookups. +- (c) Key blobs per ACL scope. Restricted content stops deduplicating, and + moving a file across scopes copies it. + +**Recommendation: (a).** Most repos have no read-narrowing ACL, so most +readers keep id reads. A partial reader never gets an id oracle. + +### OQ-4 — Retention defaults + +Trunk history is kept forever. The question is how long an unpinned +changeset lives (an abandoned commit, a dropped rebase candidate). + +- (a) 7 days after creation. +- (b) 30 days. + +**Recommendation: (a) 7 days**, set per deployment. Review versions and +workspaces hold pins, so a live change never depends on this window. + +### OQ-5 — Does the store keep conflicted changesets? (shared with review and merge-queue) + +- (a) No. The store refuses a conflicted tree. `Rebase` and `Merge` return + a typed conflict and store nothing conflicted. The `saiph` client keeps + conflict state in its working copy, as Sapling does: Sapling refuses to + commit while conflicts are open (`checkunresolved` in + `eden/scm/sapling/mergeutil.py`: "unresolved merge conflicts"). +- (b) Yes, as jj does: a tree is a `Merge`, and conflicts can be + committed and shared. + +**Recommendation: (a).** It follows the reference. Trunk and the git +mirror can never hold a conflict, and every reader sees plain trees. (b) +can be added later as a new changeset field without changing old ids. + +### OQ-6 — How does `saiph-server` know the caller? (load-bearing for client) + +Saiph is its own product, so Compass's interceptors no longer sit in front +of it. The Scion post does not cover authentication. + +- (a) `saiph-server` verifies signed bearer tokens (JWT) from configured + issuers. Claims carry the tenant, the subject kind (account or service), + the subject id and scopes. Compass is one issuer: it mints short-lived + tokens for its agents and services. An install without Compass uses + tokens Saiph issues itself, stored hashed in `saiph.tokens`. +- (b) Compass proxies every call. That adds a hop to every object read and + breaks Scion's cache tiering. + +**Recommendation: (a).** [client](client.md) designs the claims, key +discovery and Compass's issuance; the types are folded into S11 here. A +Kubernetes service-account issuer has a fixed tenant; Compass's issuer +sets the `tenant` claim. + +## Problem / Intent + +Saiph is the version control system Rigel sells; the Compass repo is its +first user. Matt ruled for a full Scion implementation with many servers +(ruling 1), with the core following Scion to the letter (ruling 10). This +record designs the store and server: the changeset format, the blob and +metadata stores, the ref compare-and-swap, derived data, server-side +commit, rebase and merge, and the Connect API that every client and +sibling record calls. + +The model is Scion +([Redesigning source control](https://www.anthropic.com/engineering/redesigning-source-control)): +"Client-server asymmetry. In git, every client is capable of being a +full-fledged server. In Scion, the server answers questions about file +history and blame, creates commits, performs merges and rebases". + +## Approach + +### Ground truth + +- **Saiph's repo** is `RigelBuild/saiph`. It does not exist yet, so every + Saiph path in this record is a plan, not a citation. +- **Compass as a model.** Compass serves Connect v2 + (`connectrpc.com/connect/v2`). Its tenant tables carry `tenant_id` with + `FORCE ROW LEVEL SECURITY` and a `tenant_isolation` policy on the GUC + `compass.tenant_id`, set with `SET LOCAL` (`go/internal/store/tenant_tx.go` + in Compass); `WithSystemRole` switches to a BYPASSRLS role. Its object + store wraps minio-go (`NewS3ObjectStore` in + `go/internal/store/objectstore.go` in Compass) and runs against Garage, + R2, MinIO and AWS S3. Saiph reuses these patterns, not the packages. +- **Compass's deployment.** The Kubernetes record runs one Compass Server + with `replicas: 1` and `Recreate` + ([server-on-kubernetes](../compass/server-on-kubernetes.md#design)). Saiph runs + its own Deployments and does not change that record. +- **License.** `facebook/sapling` is GPL-2.0-only (its file headers: "GNU + General Public License version 2"). Compass is AGPL-3.0-only. This + record borrows designs from Sapling, never code (Global Constraints). + +### Shape: immutable content, transactional coordination + +The store has two halves, as Scion has. File contents are content-addressed +and compressed in object storage: a changeset holds "content-addressed +pointers to the compressed file contents in blob storage". Changesets, +derived data and every mutable row (refs, pins, the ref log, ACL grants, +quotas) live in the metadata database, as Scion keeps them in Spanner. +Any tier can cache an immutable object forever: "Every request is for a +specific object that never changes, so any cache can answer it". + +Creating a changeset never takes a lock and never touches a ref. Moving a +ref is a compare-and-swap on one row. Scion: "no packfiles to accept, no +locking or cross-server coordination, and no refs to move around… moving +one is a compare-and-swap on a single Spanner row that happens separately +from creating changesets." The object store is never the ref authority. + +```mermaid +flowchart LR + C[saiph clients and caches] -->|Connect| S1[saiph-server 1] + C -->|Connect| S2[saiph-server N] + S1 --> O[(object storage: file contents)] + S2 --> O + S1 --> P[(metadata DB: changesets, derived data, refs, ref log, pins, grants)] + S2 --> P +``` + +### Deviations from Scion + +Each row is a place where Saiph does not do what the Scion post says. The +only accepted reasons are a fact Scion relies on that Saiph lacks, or a +detail the post leaves open. + +| Scion | Saiph | Why | +| -- | -- | -- | +| Changesets, refs and derived data in Spanner ("a single Spanner row"; "write the results back to Spanner") | The same data in YugabyteDB YSQL; one Postgres ≥16 allowed on a single node (OQ-1) | Saiph lacks Spanner: it is a Google Cloud service, and customers self-host. YugabyteDB's transactions are "based on the Google Spanner architecture". | +| Cache servers in every region ("the tree and blob cache servers in its region") | One region by default; each added region adds a `saiph-cache` Deployment with the same RPCs ([client](client.md)) | Scion relies on a multi-region fleet. A self-hosted install usually has one cluster. The tiers and RPCs are unchanged. | + +### Gaps filled from Mononoke and Sapling + +The post leaves these open. Each row names the reference Saiph follows and +marks where Saiph departs from it. Every path is in `facebook/sapling`. + +| Gap | Reference | Saiph | +| -- | -- | -- | +| Content hash | Mononoke ids are BLAKE2b-256, keyed per type (`context_key => "changeset"`, `"content"`, `"chunk"` in `eden/mononoke/mononoke_types/src/typed_hash.rs`) | Follows: BLAKE2b-256 keyed per type | +| Hashed bytes | Mononoke hashes its Thrift serialization | **Departs:** deterministic CBOR, because RFC 8949 states the determinism rule (OQ-2) | +| Changeset fields | Bonsai ([`bonsai.thrift`](https://github.com/facebook/sapling/blob/79071c2a3ea20c36ed71332608535e5801101bba/eden/mononoke/mononoke_types/serialization/bonsai.thrift)): ordered parents, changed paths, no tree | Follows, plus a change id | +| Change identity across rewrites | Sapling records mutations from predecessor to successor (`eden/scm/lib/mutationstore`: "a store for recording records of commit mutations") | **Departs** by Matt's ruling: a 16-byte change id, Saiph's own concept, inspired by jj | +| Write-ahead log mechanics | Neither says how the log is kept | An immutable row per changeset with an `intake_seq` from a database sequence, which orders intake without a lock | +| Large files | Mononoke `FileContents::Chunked` (`eden/mononoke/mononoke_types/src/file_contents.rs`) | Follows: 4 MiB chunks | +| Per-repo storage | Mononoke `PrefixBlobstore` (`eden/mononoke/blobstore/prefixblob`) | Follows: a `//` key prefix | +| Upload negotiation | Sapling's EdenAPI `lookup_batch` and `process_files_upload` (`eden/scm/lib/edenapi/trait/src/api.rs`) | Follows: `FindMissingBlobs`, then `PutBlobs` | +| Tree entries with sizes | Mononoke fsnodes: "summary information about the files in the manifest that is useful in the implementation of filesystems" (`eden/mononoke/mononoke_types/src/fsnode.rs`) | Follows: each entry carries a size | +| Content aliases | Mononoke `ContentMetadataV2` holds `sha256` and `git_sha1` (`eden/mononoke/mononoke_types/serialization/content.thrift`) | Follows: lazy content meta holds both | +| Ref log | Mononoke `bookmarks_update_log` (repo, name, from, to, reason, timestamp) | Follows: `saiph.ref_log` | +| Commit graph skip lists | Mononoke `commit_graph_edges` keeps `p1_linear_depth` and skew ancestors | Follows the idea: `p1_depth` and a Bitcoin skip pointer | +| Directory ACLs | Mononoke `.slacl` files and `AclManifests` | Follows: `.saiphacl` and an ACL manifest | +| Conflicts | Sapling refuses to commit unresolved conflicts (OQ-5) | Follows | +| Tenancy | None: Mononoke and Scion serve one organization | Adds a tenant id and RLS on every row, because Saiph is sold | +| Retention | Neither covers it | Adds gc of unpinned, non-trunk changesets after 7 days (OQ-4), because storage is usage-billed | + +### Ids and encoding (OQ-2) + +- An **id** is 32 bytes: BLAKE2b-256 of the object's bytes, keyed by the + object's type, as Mononoke's typed hashes are. The keys are + `saiph.changeset`, `saiph.tree`, `saiph.acl`, `saiph.content` and + `saiph.chunk`. The key separates types, so a blob that holds a valid + changeset encoding never shares a changeset's id. Go has keyed BLAKE2b + in `golang.org/x/crypto/blake2b`. +- A **blob id** is the keyed hash of the raw file bytes. SHA-256 and the + git SHA-1 are aliases in lazy content meta, as in Mononoke's + `ContentMetadataV2`, so a build cache that keys by SHA-256 (REAPI + `SHA256 = 1`) maps to Saiph through the alias. +- Changesets and tree nodes are encoded as deterministic CBOR (RFC 8949 + §4.2.1) with small integer map keys (OQ-2). Only `saiph-server` + encodes. A decoder re-encodes and rejects input that is not + byte-identical, so every client that verifies an id gets the same + answer. +- Wire form: `bytes` of length 32. Text form: lowercase hex. Every id + travels with its size where one exists (`size_bytes`), as REAPI digests + do. + +### Changeset + +The changeset is the unit of history, as in Scion ("the unit of work is a +“changeset,” an entry in a write-ahead log") and Mononoke Bonsai +([bonsai.thrift](https://github.com/facebook/sapling/blob/79071c2a3ea20c36ed71332608535e5801101bba/eden/mononoke/mononoke_types/serialization/bonsai.thrift): +"Bonsai Changeset is the fundamental commit object of Mononoke's internal +representation"). Scion: "where a git commit stores a pointer to the tree, +a changeset stores a list of the files that changed relative to its first +parent". It never stores a tree. Changesets are created on the server only +(see [Intake](#intake)). + +```text +Changeset (CBOR map, keys in this order; id = BLAKE2b-256 keyed "saiph.changeset") + 1 parents [id] ordered; parents[0] is the diff base; [] is a root + 2 change_id bytes(16) Saiph's change id; kept across rewrites + 3 author Signature + 4 committer Signature + 5 message text UTF-8 + 6 changes map sorted by path bytes; null = delete + 7 extra map optional; absent when empty +Signature { 1 name: text, 2 email: text, 3 time_unix_ms: int, 4 tz_offset_min: int } +FileChange { 1 kind: 0 file | 1 executable | 2 symlink, 2 content: id, 3 size: uint, + 4 copy_from: { 1 path: text, 2 parent: uint } (optional) } +``` + +Rules, all checked at intake: + +- **Tree rule.** `tree(cs) = apply(tree(parents[0]), changes)`. A root + changeset applies to the empty tree. A merge lists every path whose + merged result differs from `parents[0]`. So a tree is always derivable + from one parent and the changes, with no merge logic. +- **Parent order is hashed.** Bonsai: "Parents are stored and hashed + ordered. This matches Git". Two orders are two changesets. +- **Paths.** UTF-8, `/`-separated, no empty, `.` or `..` component, no NUL, + at most 4,096 bytes. Bonsai's prefix rule holds: "There is no distinction + between added and modified files", and "A deleted path may be a prefix of + changed paths and means a file was replaced by a directory. Otherwise, + path conflicts are not allowed". +- **Content exists.** Each `content` id names a blob already stored in the + repo, with the stated `size`. +- **No conflicts** are stored (OQ-5). A changeset's tree is plain. +- **`extra`** carries keys the git bridge needs to round-trip an imported + commit (for example a GPG signature). Keys are namespaced (`git.gpgsig`). + The store does not interpret them. + +A changeset has no tree id, no version and no review state. The root tree +id is derived (below) and kept on the changeset's row. + +**The change id** is Saiph's own concept (ruling 11). Neither Scion nor +Bonsai has one; Sapling links rewrites through mutation records instead. +The idea comes from jj, where a change is "a commit as it evolves over +time". The server picks 16 random bytes on a change's first `Commit`, and +`Rebase` keeps them. So review groups versions by change id +([review](review.md)), and the merge queue follows a change across server-side +rebases with no walk over a rewrite log. + +### Blobs + +A blob is file content. Object keys are per repo, as Mononoke's +`PrefixBlobstore` prefixes keys: `//b/`. Per-repo keys +give up dedup across repos. They buy a simple repo delete (one prefix) and +no cross-repo existence oracle. + +- A blob of at most 4 MiB is one object. +- A larger blob is stored as 4 MiB chunks, as Mononoke's + `FileContents::Chunked`. Each chunk is a blob keyed `saiph.chunk`, and a + chunk list sits at `//b/.chunks`. The client uploads + missing chunks, then calls `PutChunkedBlob`. The server streams the + chunks, hashes them in order, and stores the list only if the whole hash + matches. An interrupted upload resumes by asking `FindMissingBlobs` for + the chunks. +- Every object body is zstd-compressed (`klauspost/compress`), as Scion + keeps "compressed file contents". The body starts with one format byte + (`1` = zstd), so a later codec needs no migration. Ids are always of the + raw bytes. +- Uploads negotiate as Sapling's EdenAPI does (`lookup_batch`, then + `process_files_upload`): `FindMissingBlobs` answers from `saiph.blobs` + rows, not object HEADs, then `PutBlobs` sends only the missing content. + A row is written only after the object PUT succeeds. +- `max_file_bytes` is a per-repo setting, default 100 MiB. A repo with a + git mirror cannot raise it, because "GitHub blocks files larger than + 100 MiB" + ([GitHub docs](https://docs.github.com/en/repositories/working-with-files/managing-large-files/about-large-files-on-github)). + +### Trees + +A tree node is derived, never uploaded. It is a sorted list of entries: + +```text +TreeNode (CBOR array of entries, sorted by name bytes; id = BLAKE2b-256 keyed "saiph.tree") +Entry { 1 name: text, 2 kind: 0 file | 1 executable | 2 symlink | 3 tree, + 3 id: id, 4 size: uint } size of a tree = sum of its file sizes +``` + +Nodes are rows in `saiph.nodes`, as Scion writes derived data back to its +database. A changeset writes only the nodes on the paths it changes, so a +commit costs O(changed paths × depth), never the repo size. Each entry +carries a size, as Mononoke's fsnodes do, so a virtual checkout lists a +directory without fetching blobs. Clients fetch trees by id from the root +down, as Scion's do: "Starting from a root tree, the client checks its +local store for missing trees and requests them recursively". + +### Metadata database (OQ-1) + +The metadata database holds changesets, derived data and all mutable +state, in its own schema `saiph`, with its own migrations and its own +tenant GUC. File contents never enter it. SQL is limited to what +YugabyteDB YSQL and Postgres 16 both run. Each tenant table carries +`tenant_id` with `ENABLE` and `FORCE ROW LEVEL SECURITY` and a +`tenant_isolation` policy on the GUC `saiph.tenant_id`, the pattern +Compass uses. + +```sql +-- internal/meta/migrations/0001_saiph.sql (sketch; every tenant table gets RLS) +CREATE SCHEMA saiph; +CREATE SEQUENCE saiph.intake_seq; +CREATE TABLE saiph.tenants (id TEXT PRIMARY KEY, quota_bytes BIGINT NOT NULL DEFAULT 0); +CREATE TABLE saiph.repos ( + tenant_id TEXT NOT NULL DEFAULT current_setting('saiph.tenant_id', TRUE) + REFERENCES saiph.tenants (id) ON DELETE RESTRICT, + id BIGINT GENERATED ALWAYS AS IDENTITY, + name TEXT NOT NULL, -- "owner/name" + trunk_ref TEXT NOT NULL DEFAULT 'trunk', + max_file_bytes BIGINT NOT NULL DEFAULT 104857600, + quota_bytes BIGINT NOT NULL, + retention_days INT NOT NULL DEFAULT 7, + git_mirrored BOOLEAN NOT NULL DEFAULT false, -- KindGitCommit runs only here + stored_bytes BIGINT NOT NULL DEFAULT 0, + log_seq BIGINT NOT NULL DEFAULT 0, -- last ref-log entry + trunk_seq BIGINT NOT NULL DEFAULT 0, -- last trunk position + deleted_unix_ms BIGINT, + created_unix_ms BIGINT NOT NULL, + PRIMARY KEY (tenant_id, id), UNIQUE (tenant_id, name)); +CREATE TABLE saiph.changesets ( + tenant_id TEXT NOT NULL DEFAULT current_setting('saiph.tenant_id', TRUE), + repo_id BIGINT NOT NULL, + id BYTEA NOT NULL CHECK (length(id) = 32), + encoded BYTEA NOT NULL, -- the hashed CBOR bytes + intake_seq BIGINT NOT NULL DEFAULT nextval('saiph.intake_seq'), + change_id BYTEA NOT NULL CHECK (length(change_id) = 16), + p1 BYTEA, -- parents[0]; NULL for a root + parent_count SMALLINT NOT NULL, + generation BIGINT NOT NULL, -- 1 + max over all parents + p1_depth BIGINT NOT NULL, -- length of the first-parent chain + skip BYTEA, -- first-parent ancestor at GetSkipHeight(p1_depth) + root_tree BYTEA NOT NULL, + acl_root BYTEA, -- ACL manifest; NULL = no ACL files + trunk_seq BIGINT, -- position on trunk; NULL off trunk + created_unix_ms BIGINT NOT NULL, + PRIMARY KEY (tenant_id, repo_id, id)); +CREATE INDEX changesets_by_change ON saiph.changesets (tenant_id, repo_id, change_id); +CREATE TABLE saiph.changeset_parents (tenant_id TEXT, repo_id BIGINT, child BYTEA, + ord SMALLINT, parent BYTEA, PRIMARY KEY (tenant_id, repo_id, child, ord)); +CREATE INDEX changeset_children ON saiph.changeset_parents (tenant_id, repo_id, parent); +CREATE TABLE saiph.blobs (tenant_id TEXT, repo_id BIGINT, id BYTEA, size_bytes BIGINT, + chunks INT NOT NULL DEFAULT 0, stored_bytes BIGINT, last_seen_unix_ms BIGINT, + PRIMARY KEY (tenant_id, repo_id, id)); +CREATE TABLE saiph.nodes (tenant_id TEXT, repo_id BIGINT, id BYTEA, kind SMALLINT, + encoded BYTEA, created_unix_ms BIGINT, PRIMARY KEY (tenant_id, repo_id, id)); -- tree and ACL nodes +CREATE TABLE saiph.refs (tenant_id TEXT, repo_id BIGINT, name TEXT, target BYTEA NOT NULL, + version BIGINT NOT NULL, updated_unix_ms BIGINT, PRIMARY KEY (tenant_id, repo_id, name)); +CREATE TABLE saiph.ref_log (tenant_id TEXT, repo_id BIGINT, log_seq BIGINT, name TEXT, + old_target BYTEA, new_target BYTEA, chain BYTEA[], subject TEXT, reason TEXT, + at_unix_ms BIGINT, PRIMARY KEY (tenant_id, repo_id, log_seq)); +CREATE TABLE saiph.pins (tenant_id TEXT, repo_id BIGINT, holder TEXT, changeset BYTEA, + expires_unix_ms BIGINT, PRIMARY KEY (tenant_id, repo_id, holder, changeset)); +CREATE TABLE saiph.grants (tenant_id TEXT, repo_id BIGINT, subject TEXT, role SMALLINT, + ref_prefix TEXT NOT NULL DEFAULT '', -- '' = repo-wide; else UpdateRef and DeleteRef under it only + PRIMARY KEY (tenant_id, repo_id, subject, ref_prefix)); +-- object = the changeset, or the blob for per-blob kinds; path = '' unless per path +CREATE TABLE saiph.derived (tenant_id TEXT, repo_id BIGINT, kind SMALLINT, object BYTEA, + path TEXT NOT NULL DEFAULT '', value BYTEA, PRIMARY KEY (tenant_id, repo_id, kind, object, path)); +CREATE TABLE saiph.derive_cursors (tenant_id TEXT, repo_id BIGINT, kind SMALLINT, + log_seq BIGINT, lease_token TEXT, lease_until_unix_ms BIGINT, + PRIMARY KEY (tenant_id, repo_id, kind)); +CREATE TABLE saiph.path_touches (tenant_id TEXT, repo_id BIGINT, path TEXT, trunk_seq BIGINT, + PRIMARY KEY (tenant_id, repo_id, path, trunk_seq)); +CREATE TABLE saiph.git_commits (tenant_id TEXT, repo_id BIGINT, changeset BYTEA, + git_commit BYTEA CHECK (length(git_commit) = 20), + PRIMARY KEY (tenant_id, repo_id, changeset), UNIQUE (tenant_id, repo_id, git_commit)); +-- OQ-6 (a): Saiph-issued tokens; read before the tenant is known, so no RLS +CREATE TABLE saiph.tokens (hash BYTEA PRIMARY KEY, tenant_id TEXT NOT NULL, + kind SMALLINT NOT NULL, subject TEXT NOT NULL, scopes TEXT[], expires_unix_ms BIGINT); +``` + +The sketch elides `NOT NULL`, the `tenant_id` default and the foreign keys +on the later tables; the migration has them all. Repos have a numeric id, +so a rename moves no object. Object keys use the numeric id. + +### Refs and the ref log + +A ref is a name and a target changeset. Scion "discourages refs (pointers +to commits, such as branches and tags), except for the main ref or release +branches, because *global* state that every server has to agree on is a +looming scaling problem". So does Saiph. A repo has its trunk ref and +optional `release/*` refs. Review heads, workspaces and merge-queue +candidates are not refs; they hold **pins** instead, so gc keeps their +changesets. + +- **Names** match `^[a-z0-9][a-z0-9._/-]{0,199}$`. +- **Protected refs** are the trunk ref and `release/*`. They move only + forward along the first-parent chain, and only for a subject with the + `land` role (the merge queue, admins). A grant may carry a + `ref_prefix`: it then gives its role only for `UpdateRef` and + `DeleteRef` on refs whose names start with it, so a deploy controller + can hold `land` on `release/` and not on trunk + ([ci-cd](ci-cd.md) OQ-3). Every other check, and + `GetRepoResponse.caller_role`, uses the repo-wide grant. +- **The CAS.** `UpdateRef` names the expected target. One statement + decides it: + +```sql +UPDATE saiph.refs SET target = $new, version = version + 1, updated_unix_ms = $now + WHERE tenant_id = $t AND repo_id = $r AND name = $name AND target = $expected + RETURNING version; +``` + + Exactly one of two racing writers wins. On Postgres under READ + COMMITTED, the second writer blocks on the row, then "The search + condition of the command (the WHERE clause) is re-evaluated to see if + the updated version of the row still matches" + ([Postgres docs](https://www.postgresql.org/docs/current/transaction-iso.html)). + If either database reports a serialization failure instead, the server + retries once from a fresh read. The loser gets zero rows and a typed + `current` result, not an error. +- **Batch advance.** `UpdateRef` may carry a `chain`: an ordered list of + changesets where `chain[0].parents[0]` is the expected target, each next + entry's `parents[0]` is the one before it, and the last is the new + target. The server checks the chain from immutable rows before the + transaction, then moves the ref once. The merge queue lands a whole batch + with one CAS. Without a chain, the server walks the first-parent chain + from the new target back to the expected one (at most 10,000 steps, else + `INVALID_ARGUMENT` asking for a chain). +- **In the same transaction** the server bumps `saiph.repos.log_seq`, + writes one `saiph.ref_log` row, and, for the trunk ref, gives each chain + entry the next `trunk_seq`. The repo row lock orders ref-log entries, so + the log is gapless. The log follows Mononoke's `bookmarks_update_log`. +- **`trunk_seq`** is a dense position on trunk: 1 for the first trunk + changeset, then +1 for each. CI milestones, bisection and the git bridge + use it as "the n-th trunk changeset". +- **`WatchRefs`** streams ref-log entries after a given `log_seq`. Each + replica runs one poller per watched repo; it reads `saiph.ref_log` rows + after its last seq every 250 ms and fans them out to that replica's + streams. A reconnecting consumer resumes from its last seq and misses + nothing. + +### Derived data + +Derived data follows Scion: "Each type of derived data is defined +recursively, so its value for a changeset is computed from its values for +that changeset’s parents." Scion's servers "compute derived data on the fly +and write the results back" to the database, and so do Saiph's: values +live in `saiph.nodes` and `saiph.derived`. Mononoke has the same model +([`derivable_type.rs`](https://github.com/facebook/sapling/blob/main/eden/mononoke/mononoke_types/src/derivable_type.rs), +"Enum which consolidates all available derived data types"). A derived +value is a pure function of immutable inputs, so two servers that derive +the same value write the same bytes. No lease guards a derivation; an +in-process singleflight stops duplicate work in one replica, and +`INSERT … ON CONFLICT DO NOTHING` absorbs a cross-replica race. + +Each kind has a mode, as Scion chooses per type "whether to compute it +eagerly or defer the work": + +- **Eager** kinds are derived at intake, in the changeset's transaction. A + changeset without them is never visible. +- **On-trunk** kinds are derived when trunk advances. A worker per + (repo, kind) reads the ref log from `saiph.derive_cursors.log_seq`, under + a lease row with a token and a TTL (the pattern of `LeaseForgeMirror` in + `go/internal/store/forge_mirrors.go` in Compass). Any replica can hold + the lease. +- **Lazy** kinds are derived on first read and memoized. + +| Kind | Mode | Value | Used by | +| -- | -- | -- | -- | +| Root tree | Eager | `saiph.nodes` rows; root id in `saiph.changesets.root_tree` | every read, `Diff`, client checkout | +| Commit graph | Eager | `generation`, `p1_depth`, `skip` columns | `MergeBase`, `Log`, ancestry checks | +| ACL manifest | Eager | `saiph.nodes` rows; root id in `acl_root` | every ACL check | +| Path index | On-trunk | `saiph.path_touches` rows | `PathHistory` on trunk | +| Git commit | On-trunk, repos with `git_mirrored` only | `saiph.git_commits` rows | [git-bridge](git-bridge.md) export, `GetGitCommits` | +| Git tree | Lazy, per tree node | the 20-byte git tree id in `saiph.derived` | [git-bridge](git-bridge.md) export | +| Content meta | Lazy, per blob | `{is_binary, is_utf8, lines, sha256, git_sha1}` in `saiph.derived` | `Diff`, `Blame`, `GetContentMeta`, git bridge, build-cache aliases | +| Blame | Lazy, per (changeset, path) | range list in `saiph.derived` | `Blame` | + +- **Trees.** `tree(cs)` applies `cs.changes` to `tree(parents[0])` and + writes only the nodes on changed paths. +- **Commit graph.** Scion builds "skip lists for the commit graph". + `generation` prunes ancestry walks. `skip` is the Bitcoin skip pointer on + the first-parent chain + ([`chain.cpp`](https://raw.githubusercontent.com/bitcoin/bitcoin/master/src/chain.cpp) + `GetSkipHeight`: `return (height & 1) ? InvertLowestOne(InvertLowestOne(height - 1)) + 1 : InvertLowestOne(height);`), + so "ancestor at depth d" takes O(log n) steps. Mononoke's + `commit_graph_edges` keeps the same idea as `p1_linear_depth` with skew + ancestors. +- **ACL manifest.** Mononoke's `AclManifests` is the model + ([`acl_manifest/derive.rs`](https://raw.githubusercontent.com/facebook/sapling/main/eden/mononoke/derived_data/acl_manifest/derive.rs): + "Only `.slacl` file changes (explicit + implicit deletes) are fed to + `derive_manifest`; unchanged subtrees are reused automatically"). A + changeset that touches no ACL file copies its parent's `acl_root`. +- **Path index.** For each trunk changeset, one row per changed path and + per ancestor directory of it. `PathHistory` on a trunk changeset is one + index range scan. Off trunk, it walks the first-parent chain until it + reaches a trunk changeset, then uses the index. +- **Git commit and git tree.** The store owns the slots and the table; + the git-commit worker runs only for repos with `git_mirrored`. + [git-bridge](git-bridge.md) owns both derivations (git trees and + commits from changesets, with `extra` keys for round-trip fields). + `GetGitCommits` maps changesets and git commits both ways. +- **Blame.** Lazy, as Mononoke's blame: `blame(cs, path)` is + `blame(parents[0], path)` moved through the line diff, with new lines + assigned to `cs`. The value is a `saiph.derived` row keyed by + (changeset, path). +- **Adding a kind** later is one `Deriver`, as Scion has "added new types + of derived data many times as the system has grown". A backfill runs it + over trunk from the oldest unrecorded `trunk_seq`. + +### Intake + +Scion: "Changesets are created server-side and are available everywhere, +instantly." Saiph has no RPC that takes a client-encoded changeset. +`Commit`, `Rebase`, `Merge` and history import all build changesets +through one intake path. Import ([git-bridge](git-bridge.md)) runs inside +`saiph-server`, as Scion's "git-to-Scion ingestion pipeline" is +server-side. For each changeset, in order: + +1. Check path, size and parent-order rules on the structured input. +2. Check that every parent is stored or earlier in the batch. Otherwise + `FAILED_PRECONDITION`, with the missing ids in a `MissingObjects` + detail. +3. Check that every blob is stored, with `SELECT … FOR SHARE` on its row, + and touch `last_seen_unix_ms`. Otherwise `FAILED_PRECONDITION` with the + missing digests. +4. Check write ACLs for every changed path (below). +5. Encode, hash, and derive the eager kinds. +6. In one transaction, insert the changeset row (with its encoded bytes), + its parent rows and its new nodes, all `ON CONFLICT DO NOTHING`. + +Intake is idempotent: the same input gives the same id and no new row. +It takes no lock beyond its own inserts and the blob share locks, so +commits stay "cheap, incremental, and lock-free". Nothing reaches object +storage during intake, so a crash leaves no orphan. + +### Server-side commit, rebase and merge + +Scion's server "creates commits, performs merges and rebases". So does +Saiph's. These calls let a thin client (the `saiph` CLI, an agent, the web +app, the merge queue) change history with no local repo. + +- **`Commit`** takes parents, file changes whose content is already + stored, a message and an optional change id. The committer is the + caller at server time. An admin, or a token with the `import` scope + (history import), may also set the committer and `extra`; the scope + gives nothing else. The server builds the changeset through intake. +- **`CommitBatch`** runs intake once over up to 1,000 `CommitRequest`s, + so a parent may be an earlier entry. Import uses it + ([git-bridge](git-bridge.md)). +- **`Rebase`** takes a set of changesets (any shape, up to 1,000) and an + `onto`. Every parent outside the set is replaced by `onto`. Each + changeset is rewritten in topological order: the new changes are a + three-way merge of each path, with the old `parents[0]` tree as the base, + the new `parents[0]` tree as the destination, and the old changeset's + tree as the source. Change id, author and message are kept. The + committer is the caller, at server time. With `drop_empty`, a rewritten + changeset with no changes is dropped and its children take its parent. + This drops a change that trunk already holds. +- **Resolutions.** `Rebase` may carry `resolutions`: content for a path + whose three-way merge, while a named changeset is rewritten, is a + `CONFLICT_KIND_CONTENT` conflict. A non-empty list needs + `REPO_ROLE_LAND`. A resolution for any other path, or for a + `.saiphacl`, is `INVALID_ARGUMENT` naming the path, so a fix made + against an old trunk fails instead of overwriting. Content not stored + is `FAILED_PRECONDITION`. A conflict the resolutions do not cover still + returns `Conflict`. The merge queue's fixer uses it + ([merge-queue](merge-queue.md)). +- **`Merge`** takes two or more parents and merges each path against the + merge base. With more than one best merge base, every path that differs + between the bases is a `MULTIPLE_BASES` conflict. The store does not + merge criss-cross histories implicitly. +- **Conflicts are results, not errors.** A conflict is + `Conflict{changeset, paths[]}`. Each `PathConflict` has a kind, the + file versions as jj orders them, and hunks. jj's `Merge` + ([`lib/src/merge.rs`](https://github.com/jj-vcs/jj/blob/9ff73e7f/lib/src/merge.rs)) + "Alternates between positive and negative terms, starting with + positive". So `adds` has n entries and `removes` has n−1. For a rebase, + `adds = [destination, source]` and `removes = [base]`. Each hunk carries + the destination line range and the text of each term. A binary file has + no hunks. +- **Nothing conflicted is stored** (OQ-5). On a conflict, `Rebase` + returns the rewrites that came before it. Those are stored, unpinned + changesets; the caller pins the ones it keeps. `Merge` writes nothing. +- **The text merge** is in-house (`internal/textdiff`): a histogram line + diff and a three-way line merge, written fresh (Sapling's diff code is + GPL-2.0-only; see Global Constraints). The conflict shape above is + Saiph's own. A file is binary when its first 8 KiB holds a NUL byte, as + git decides. + +### Per-directory ACLs + +ACLs live in the tree, as Mononoke's `.slacl` files do. A file named +`.saiphacl` in a directory sets the ACL for that directory and everything +under it, down to the next `.saiphacl`. It is JSON: + +```json +{ "read": ["role:read"], "write": ["account:acct_1", "role:admin"] } +``` + +- A missing `read`, `write` or `owners` key inherits that list from the + enclosing scope. The repo grants (`saiph.grants`) are the outermost + scope. +- An entry is `account:`, `service:`, or `role:`, which + matches any subject holding that repo role or a higher one. `admin` + always passes. A malformed list fails intake with `INVALID_ARGUMENT`. +- **Owners.** The optional `owners` list names who must approve a change + to paths in the scope ([review](review.md)). `CoverOwners` returns each + path whose scope has no owner among the given approvers, checked under + the changeset's ACL manifest and the trunk head's, so a version that + adds itself to `owners` is not covered by the new entry. An owner + without write in the scope does not cover it. +- **Write.** Every changed path needs write in its scope under + `parents[0]`'s ACL manifest and under the trunk head's. Changing, + adding or deleting `/.saiphacl` needs write in the scope that encloses + ``, never the file's own scope. A `land` subject passes write checks + for `Rebase` and `Merge`, because those reapply changes already checked + at intake. +- **Read.** A path read (`ReadTree`, `ReadFile`, `Diff`, `Blame`, + `PathHistory`) needs read in the path's scope under the changeset's ACL + manifest and under the trunk head's. A denied read answers `NOT_FOUND`, + never `PERMISSION_DENIED`, so it does not confirm the path exists. + `Diff` and `ReadTree` leave out denied entries. +- **Id reads** need read on the whole repo: no `.saiphacl` narrows read for + the caller (OQ-3). A malformed `.saiphacl` fails intake with + `INVALID_ARGUMENT` naming the path. +- Protected paths, such as a mirrored repo's `.github/workflows/`, use + this same mechanism: a `.saiphacl` there grants write to named humans only. + +### Tenancy + +Saiph is sold to customers who self-host it, and Compass tenants use it +through Compass, so tenancy is core, not an add-on. In Compass's terms it +passes the +[boundary record](../compass/meta/oss-core-managed-boundary/design.md#boundary-test) +test (the public single-tenant product compiles and runs it), and +Compass's product rule is +[self-host and managed](https://github.com/RigelBuild/compass/blob/main/docs/concepts/self-host-and-managed.md). +This record restates neither. + +- Every `saiph` row carries `tenant_id`, under RLS on `saiph.tenant_id`. + `meta.(*Meta).InTenant` sets it with `SET LOCAL`. GC, derive workers and + token lookup use a BYPASSRLS role. +- Every object key starts with the tenant id: `//…`. +- The tenant comes from the verified token (OQ-6), never from a request + field. Handlers read one `auth.Principal{Kind, ID, Tenant, Scopes, + Agent}` from the context; `Agent` marks an agent's account. +- A self-hosted install has one tenant row, created at boot. Compass + creates a Saiph tenant per Compass tenant with `EnsureTenant`, called + with a token Compass mints for that tenant. + +### Horizontal scaling + +Ruling 1 asks for many servers, and Scion says "Everything must be +horizontally scalable". `saiph-server` keeps no state of its own. Every +write goes to the metadata database or object storage, so any replica +answers any call. + +- **Deployment.** `saiph-server` runs as a Deployment with `replicas: N` + and `RollingUpdate`. Its manifests live in the saiph repo (`deploy/`). A + single replica is for dogfooding only (ruling 1). +- **Object cache.** Scion's servers "are essentially large caches". Each + replica keeps an in-process LRU of immutable objects keyed by + `(tenant, repo, id)`, bounded by `--cache-bytes` (default 1 GiB). An + entry is never invalidated, because its key names its content. +- **HTTP caching.** Every read by id is marked + `option idempotency_level = NO_SIDE_EFFECTS;`, so Connect serves it over + GET ([Connect docs](https://connectrpc.com/docs/go/get-requests-and-caching/): + "Handlers will automatically support GET requests using this option"). + Those responses carry + `Cache-Control: private, max-age=31536000, immutable`. +- **Tiers.** Scion: "The scion-daemon, scion-cache, and scion-server all + respond to identical RPCs for this protocol". `saiphd` (the node daemon) + and `saiph-cache` (the regional cache) serve the same `ObjectService` + reads as `saiph-server`, and tiering is strict: a daemon talks only to + its cache tier, a cache only to servers. Both are designed in + [client](client.md); they authenticate each caller for the repo before they + answer from cache. +- **Workers.** Derive and gc workers hold lease rows with a TTL and a + token. Any replica can take any lease. +- **Write ceiling.** Trunk advances serialize on one ref row per repo. + Landing a batch with one `UpdateRef` keeps that row cold + ([merge-queue](merge-queue.md)). Changeset intake scales with the database. + +### GC and retention + +- **Roots** are every ref target, every unexpired pin, and every changeset + younger than the repo's `retention_days` (OQ-4). Trunk history is always + reachable from the trunk ref, so it is kept forever. +- **Changesets.** A daily gc worker per repo marks every changeset reachable + from a root through parent rows. It deletes the unmarked changeset rows + and their derived rows in one transaction. +- **Blobs.** A blob row is deleted when no live changeset names it and its + `last_seen_unix_ms` is older than the retention window. The `DELETE` + re-checks `last_seen_unix_ms` in its `WHERE`. An intake that touched the + row (step 3) holds a share lock, so the delete waits and then skips the + row. The object is deleted after the row's transaction commits. +- **Nodes** (tree and ACL) are swept weekly: keep every node reachable + from a live changeset's `root_tree` or `acl_root`, or newer than the + retention window, and delete the rest. +- **Repo delete** sets `deleted_unix_ms`. After 30 days a worker deletes + the repo's rows and its object prefix. `CreateRepo` refuses the name + until then. +- **Ref log** rows are kept forever. They are small, and they are the + audit trail of every trunk move. + +### Caps and spend + +Object storage and the database are usage-billed when an operator runs them +on a cloud vendor. Saiph enforces its own caps, and the deployment that +enables it adds the vendor layer. + +| Layer | Cap | Enforced by | +| -- | -- | -- | +| Repo | `saiph.repos.quota_bytes` (required at `CreateRepo`) | intake and `PutBlobs` refuse with `RESOURCE_EXHAUSTED` before any write | +| Tenant | `saiph.tenants.quota_bytes` (0 = no tenant cap) | the same check, summed over the tenant's repos | +| File | `saiph.repos.max_file_bytes` | `PutBlobs` and `PutChunkedBlob` | +| Request | 16 MiB per call (`connect.WithReadMaxBytes`) | Connect handler options | +| Bucket | Garage `garage bucket set-quotas --max-size --max-objects `, or the vendor's bucket quota | object store refuses uploads past the quota | +| Database | the operator's volume size (CloudNativePG `storage.size`, or the YugabyteDB volume claims) | the database stops writing at a full volume; ref and changeset writes fail closed | + +`stored_bytes` counts blob bytes as stored (after compression), changeset +encodings, nodes and derived rows. A deployment that enables Saiph on a +usage-billed vendor also adds a budget alert that pages Matt through the +billing paging route, and a spend-cap row per layer with evidence that it +enforces. Until then Saiph stays off in that deployment. + +### API: the `saiph.v1` Connect package + +The API is its own proto package, `saiph.v1`, in +`proto/saiph/v1/saiph.proto` in the saiph repo. Go code generates to +`github.com/RigelBuild/saiph/gen/saiph/v1` (`saiphv1`, `saiphv1connect`); +TS generates to a client package that the Compass web app consumes. It is +a public API: the `saiph` CLI, `saiphd`, agents, the web app, the merge +queue, CI and customer tooling call it. Scion: "all of the source control +queries the client uses are also exposed via API". + +Every request names a repo as `"owner/name"`. Ids are 32-byte `bytes`. +An empty `bytes` field means "not set". Errors use Connect codes: +`NOT_FOUND` (also for a read the ACL denies), `PERMISSION_DENIED` (writes +only), `FAILED_PRECONDITION` with a `MissingObjects` detail, +`RESOURCE_EXHAUSTED` (quota), `INVALID_ARGUMENT` (encoding, paths, limits). +A lost CAS and a conflict are not errors; they are typed results. + +```proto +// proto/saiph/v1/saiph.proto +syntax = "proto3"; +package saiph.v1; + +message Digest { bytes id = 1; uint64 size_bytes = 2; } +message Signature { string name = 1; string email = 2; int64 time_unix_ms = 3; int32 tz_offset_min = 4; } +enum EntryKind { ENTRY_KIND_UNSPECIFIED = 0; ENTRY_KIND_FILE = 1; ENTRY_KIND_EXECUTABLE = 2; ENTRY_KIND_SYMLINK = 3; ENTRY_KIND_TREE = 4; } +message Entry { string name = 1; EntryKind kind = 2; bytes id = 3; uint64 size_bytes = 4; } +message CopyFrom { string path = 1; uint32 parent = 2; } +message FileChange { EntryKind kind = 1; Digest content = 2; CopyFrom copy_from = 3; } +message PathChange { string path = 1; oneof change { FileChange file = 2; bool delete = 3; } } +message Changeset { + repeated bytes parents = 1; + bytes change_id = 2; + Signature author = 3; + Signature committer = 4; + string message = 5; + repeated PathChange changes = 6; // sorted by path, as encoded + map extra = 7; +} +// FAILED_PRECONDITION detail: what the server lacks. +message MissingObjects { repeated Digest blobs = 1; repeated bytes changesets = 2; } + +service RepoService { + rpc EnsureTenant(EnsureTenantRequest) returns (EnsureTenantResponse); + rpc CreateRepo(CreateRepoRequest) returns (CreateRepoResponse); + rpc GetRepo(GetRepoRequest) returns (GetRepoResponse); + rpc ListRepos(ListReposRequest) returns (ListReposResponse); + rpc DeleteRepo(DeleteRepoRequest) returns (DeleteRepoResponse); + rpc SetGrant(SetGrantRequest) returns (SetGrantResponse); + rpc SetGitMirrored(SetGitMirroredRequest) returns (SetGitMirroredResponse); +} +enum RepoRole { REPO_ROLE_UNSPECIFIED = 0; REPO_ROLE_READ = 1; REPO_ROLE_WRITE = 2; REPO_ROLE_LAND = 3; REPO_ROLE_ADMIN = 4; } +message Repo { string name = 1; string trunk_ref = 2; uint64 quota_bytes = 3; uint64 stored_bytes = 4; uint64 max_file_bytes = 5; uint32 retention_days = 6; bool git_mirrored = 7; } +message CreateRepoRequest { string repo = 1; uint64 quota_bytes = 2; uint64 max_file_bytes = 3; uint32 retention_days = 4; } +message CreateRepoResponse { Repo repo = 1; } +message GetRepoRequest { string repo = 1; } +// id_reads: no .saiphacl narrows read for the caller (OQ-3). tenant: the caller's, from its token. +message GetRepoResponse { Repo repo = 1; RepoRole caller_role = 2; bool id_reads = 3; string tenant = 4; } +message ListReposRequest { string prefix = 1; uint32 page_size = 2; string page_token = 3; } +message ListReposResponse { repeated Repo repos = 1; string next_page_token = 2; } +message DeleteRepoRequest { string repo = 1; } +message DeleteRepoResponse {} +// subject is "account:" or "service:"; UNSPECIFIED removes the grant. +// ref_prefix set: the role applies only to UpdateRef and DeleteRef under it. +message SetGrantRequest { string repo = 1; string subject = 2; RepoRole role = 3; string ref_prefix = 4; } +message SetGrantResponse {} +// The tenant comes from the caller's token, which needs the provision scope. +message EnsureTenantRequest { uint64 quota_bytes = 1; } +message EnsureTenantResponse { bool created = 1; } +// Admin only. Enabling fails while max_file_bytes > 100 MiB. +message SetGitMirroredRequest { string repo = 1; bool git_mirrored = 2; } +message SetGitMirroredResponse { Repo repo = 1; } + +service ObjectService { + rpc FindMissingBlobs(FindMissingBlobsRequest) returns (FindMissingBlobsResponse); + rpc PutBlobs(PutBlobsRequest) returns (PutBlobsResponse); + rpc PutChunkedBlob(PutChunkedBlobRequest) returns (PutChunkedBlobResponse); + rpc GetBlobs(GetBlobsRequest) returns (GetBlobsResponse) { option idempotency_level = NO_SIDE_EFFECTS; } + rpc GetChangesets(GetChangesetsRequest) returns (GetChangesetsResponse) { option idempotency_level = NO_SIDE_EFFECTS; } + rpc GetTrees(GetTreesRequest) returns (GetTreesResponse) { option idempotency_level = NO_SIDE_EFFECTS; } + rpc GetContentMeta(GetContentMetaRequest) returns (GetContentMetaResponse) { option idempotency_level = NO_SIDE_EFFECTS; } + rpc ReadTree(ReadTreeRequest) returns (ReadTreeResponse); + rpc ReadFile(ReadFileRequest) returns (stream ReadFileResponse); +} +message FindMissingBlobsRequest { string repo = 1; repeated Digest digests = 2; } +message FindMissingBlobsResponse { repeated Digest missing = 1; } +// Each content at most 4 MiB; the server computes the ids. +message PutBlobsRequest { string repo = 1; repeated bytes contents = 2; } +message PutBlobsResponse { repeated Digest digests = 1; } +message PutChunkedBlobRequest { string repo = 1; Digest digest = 2; repeated Digest chunks = 3; } +message PutChunkedBlobResponse {} +// chunks set: content empty. stored set (stored_form): content and chunks +// empty; the body as the store keeps it (format byte, then zstd). +message Blob { bytes id = 1; bytes content = 2; repeated Digest chunks = 3; bytes stored = 4; } +message GetBlobsRequest { string repo = 1; repeated bytes ids = 2; bool stored_form = 3; } +message GetBlobsResponse { repeated Blob blobs = 1; } +message ChangesetObject { bytes id = 1; bytes encoded = 2; Changeset changeset = 3; bytes root_tree = 4; uint64 generation = 5; } +message GetChangesetsRequest { string repo = 1; repeated bytes ids = 2; bool omit_decoded = 3; } +message GetChangesetsResponse { repeated ChangesetObject changesets = 1; } +message TreeObject { bytes id = 1; bytes encoded = 2; repeated Entry entries = 3; } +// omit_decoded: entries empty; encoded only. +message GetTreesRequest { string repo = 1; repeated bytes ids = 2; bool omit_decoded = 3; } +message GetTreesResponse { repeated TreeObject trees = 1; } +// Id read on blob ids, at most 1,000; derived lazily; unknown ids are left out. +message GetContentMetaRequest { string repo = 1; repeated bytes ids = 2; } +message ContentMeta { bytes id = 1; bool is_binary = 2; bool is_utf8 = 3; uint64 lines = 4; bytes sha256 = 5; bytes git_sha1 = 6; } +message GetContentMetaResponse { repeated ContentMeta metas = 1; } +// Path read, ACL-checked per entry; depth 0 = one level. +message ReadTreeRequest { string repo = 1; bytes changeset = 2; string path = 3; uint32 depth = 4; } +message ReadTreeResponse { repeated PathEntry entries = 1; } +message PathEntry { string path = 1; Entry entry = 2; } +message ReadFileRequest { string repo = 1; bytes changeset = 2; string path = 3; } +message ReadFileResponse { oneof frame { Entry header = 1; bytes chunk = 2; } } + +service HistoryService { + rpc Log(LogRequest) returns (LogResponse); + rpc MergeBase(MergeBaseRequest) returns (MergeBaseResponse) { option idempotency_level = NO_SIDE_EFFECTS; } + rpc IsAncestor(IsAncestorRequest) returns (IsAncestorResponse) { option idempotency_level = NO_SIDE_EFFECTS; } + rpc Diff(DiffRequest) returns (DiffResponse) { option idempotency_level = NO_SIDE_EFFECTS; } + rpc PathHistory(PathHistoryRequest) returns (PathHistoryResponse); + rpc Blame(BlameRequest) returns (BlameResponse) { option idempotency_level = NO_SIDE_EFFECTS; } + rpc CoverOwners(CoverOwnersRequest) returns (CoverOwnersResponse) { option idempotency_level = NO_SIDE_EFFECTS; } + rpc GetGitCommits(GetGitCommitsRequest) returns (GetGitCommitsResponse) { option idempotency_level = NO_SIDE_EFFECTS; } +} +// approvers: subjects with an OWNER approval. Checked under the changeset's +// ACL manifest and the trunk head's. At most 1,000 paths. Needs id reads. +message CoverOwnersRequest { string repo = 1; bytes changeset = 2; repeated string paths = 3; repeated string approvers = 4; } +message UncoveredPath { string path = 1; string scope = 2; repeated string owners = 3; } +message CoverOwnersResponse { repeated UncoveredPath uncovered = 1; } +// Either list may be empty; unknown ids are left out. +message GetGitCommitsRequest { string repo = 1; repeated bytes changesets = 2; repeated bytes git_commits = 3; } +message GitCommitMapping { bytes changeset = 1; bytes git_commit = 2; } +message GetGitCommitsResponse { repeated GitCommitMapping mappings = 1; } +// Ancestors of heads, newest generation first, excluding ancestors of stop. +message LogRequest { string repo = 1; repeated bytes heads = 2; repeated bytes stop = 3; repeated string paths = 4; bool first_parent = 5; uint32 page_size = 6; string page_token = 7; } +message LogEntry { bytes id = 1; Changeset changeset = 2; uint64 trunk_seq = 3; } +message LogResponse { repeated LogEntry entries = 1; string next_page_token = 2; } +// All best common ancestors; empty when the heads share no history. +message MergeBaseRequest { string repo = 1; repeated bytes heads = 2; } +message MergeBaseResponse { repeated bytes bases = 1; } +message IsAncestorRequest { string repo = 1; bytes ancestor = 2; bytes descendant = 3; } +message IsAncestorResponse { bool ancestor = 1; } +// from empty = to's parents[0]; paths are prefixes; empty = all. +message DiffRequest { string repo = 1; bytes from = 2; bytes to = 3; repeated string paths = 4; bool hunks = 5; uint32 context_lines = 6; uint32 page_size = 7; string page_token = 8; } +enum ChangeKind { CHANGE_KIND_UNSPECIFIED = 0; CHANGE_KIND_ADDED = 1; CHANGE_KIND_MODIFIED = 2; CHANGE_KIND_DELETED = 3; CHANGE_KIND_COPIED = 4; } +message DiffHunk { uint32 old_start = 1; uint32 old_lines = 2; uint32 new_start = 3; uint32 new_lines = 4; bytes text = 5; } +message FileDiff { string path = 1; ChangeKind kind = 2; Entry old = 3; Entry new = 4; string copied_from = 5; bool binary = 6; repeated DiffHunk hunks = 7; } +message DiffResponse { repeated FileDiff files = 1; string next_page_token = 2; } +message PathHistoryRequest { string repo = 1; bytes from = 2; string path = 3; uint32 page_size = 4; string page_token = 5; } +message PathHistoryResponse { repeated bytes changesets = 1; string next_page_token = 2; } +message BlameRequest { string repo = 1; bytes changeset = 2; string path = 3; } +message BlameRange { uint32 start_line = 1; uint32 line_count = 2; bytes changeset = 3; string original_path = 4; uint32 original_start_line = 5; } +message BlameResponse { repeated BlameRange ranges = 1; } + +service MutationService { + rpc Commit(CommitRequest) returns (CommitResponse); + rpc Rebase(RebaseRequest) returns (RebaseResponse); + rpc Merge(MergeRequest) returns (MergeResponse); + rpc CommitBatch(CommitBatchRequest) returns (CommitBatchResponse); +} +// committer = caller at server time unless an admin or the import scope sets it; +// change_id empty = new random id; extra is honored for admins and the import scope only. +message CommitRequest { string repo = 1; repeated bytes parents = 2; repeated PathChange changes = 3; string message = 4; Signature author = 5; bytes change_id = 6; Signature committer = 7; map extra = 8; } +message CommitResponse { bytes changeset = 1; bytes root_tree = 2; } +// A parent may be an earlier entry. At most 1,000 commits and 16 MiB. +message CommitBatchRequest { string repo = 1; repeated CommitRequest commits = 2; } +message CommitBatchResponse { repeated bytes changesets = 1; } +// Content for a path that conflicts while rewriting changeset (an old id); kind FILE or EXECUTABLE. +message PathResolution { bytes changeset = 1; string path = 2; Digest content = 3; EntryKind kind = 4; } +message RebaseRequest { string repo = 1; repeated bytes changesets = 2; bytes onto = 3; bool drop_empty = 4; repeated PathResolution resolutions = 5; } +message Rewrite { bytes old = 1; bytes new = 2; bool dropped = 3; } +// conflict set: rewrites hold only the changesets rewritten before it. +message RebaseResponse { repeated Rewrite rewrites = 1; Conflict conflict = 2; } +message MergeRequest { string repo = 1; repeated bytes parents = 2; string message = 3; bytes change_id = 4; } +message MergeResponse { oneof outcome { bytes changeset = 1; Conflict conflict = 2; } } +enum ConflictKind { + CONFLICT_KIND_UNSPECIFIED = 0; CONFLICT_KIND_CONTENT = 1; CONFLICT_KIND_MODIFY_DELETE = 2; + CONFLICT_KIND_ADD_ADD = 3; CONFLICT_KIND_FILE_DIRECTORY = 4; CONFLICT_KIND_MODE = 5; + CONFLICT_KIND_BINARY = 6; CONFLICT_KIND_MULTIPLE_BASES = 7; +} +// One side of a conflict; entry unset = the path is absent on that side. +message ConflictTerm { bytes changeset = 1; Entry entry = 2; } +// jj Merge order: adds has n terms, removes n-1. +message ConflictHunk { uint32 dest_start_line = 1; uint32 dest_line_count = 2; repeated bytes adds = 3; repeated bytes removes = 4; } +message PathConflict { string path = 1; ConflictKind kind = 2; repeated ConflictTerm adds = 3; repeated ConflictTerm removes = 4; repeated ConflictHunk hunks = 5; } +message Conflict { bytes changeset = 1; repeated PathConflict paths = 2; } + +service RefService { + rpc GetRef(GetRefRequest) returns (GetRefResponse); + rpc ListRefs(ListRefsRequest) returns (ListRefsResponse); + rpc UpdateRef(UpdateRefRequest) returns (UpdateRefResponse); + rpc DeleteRef(DeleteRefRequest) returns (DeleteRefResponse); + rpc WatchRefs(WatchRefsRequest) returns (stream WatchRefsResponse); + rpc Pin(PinRequest) returns (PinResponse); + rpc Unpin(UnpinRequest) returns (UnpinResponse); +} +message Ref { string name = 1; bytes target = 2; uint64 version = 3; int64 updated_unix_ms = 4; } +message GetRefRequest { string repo = 1; string name = 2; } +message GetRefResponse { Ref ref = 1; } +message ListRefsRequest { string repo = 1; string prefix = 2; } +message ListRefsResponse { repeated Ref refs = 1; } +// expected empty = create; chain[0].parents[0] == expected, last == target. +message UpdateRefRequest { string repo = 1; string name = 2; bytes expected = 3; bytes target = 4; repeated bytes chain = 5; string reason = 6; } +message UpdateRefResponse { bool updated = 1; Ref current = 2; uint64 log_seq = 3; repeated uint64 trunk_seqs = 4; } +message DeleteRefRequest { string repo = 1; string name = 2; bytes expected = 3; } +message DeleteRefResponse { bool deleted = 1; Ref current = 2; } +message WatchRefsRequest { string repo = 1; uint64 after_log_seq = 2; repeated string names = 3; } +message WatchRefsResponse { uint64 log_seq = 1; string name = 2; bytes old_target = 3; bytes new_target = 4; repeated bytes chain = 5; repeated uint64 trunk_seqs = 6; string subject = 7; string reason = 8; int64 at_unix_ms = 9; } +// holder is ":", for example "review:" or "merge-queue:b". +message PinRequest { string repo = 1; string holder = 2; repeated bytes changesets = 3; uint32 ttl_seconds = 4; } +message PinResponse {} +// changesets empty = every pin of the holder. +message UnpinRequest { string repo = 1; string holder = 2; repeated bytes changesets = 3; } +message UnpinResponse {} +``` + +Limits on every call: at most 1,000 ids or changesets per request, 16 MiB +per request and response body, and `page_size` at most 1,000 (default +100). `UpdateRef` on a protected ref needs `REPO_ROLE_LAND`. `Pin` and +`Unpin` need `REPO_ROLE_WRITE`. + +### What the merge queue gets + +[merge-queue](merge-queue.md) builds on these calls and on nothing else in +the store: + +- **Auto-rebase** is `Rebase` with `drop_empty`. A conflict comes back as + `Conflict{changeset, paths[]}`, with hunks, so the conflict fixer gets + the exact terms and the queue can eject without parsing text. +- **Affected paths** come from `Diff` with `paths` and no `hunks`, and + from `MergeBase` and `IsAncestor` to place a stack against trunk. + Per-target lanes map those paths to targets ([ci-cd](ci-cd.md)). +- **Landing a batch** is one `UpdateRef` with `chain`. A lost CAS returns + the current target; the lane rebases onto it and retries. No global lock + is held while CI runs. +- **Candidates** survive gc through `Pin` with the holder + `merge-queue:b`, and are dropped with `Unpin`. +- **Conflict fixes** are written by `Rebase` with `resolutions`. +- **Trunk events** stream from `WatchRefs`, which resumes from a + `log_seq` after a restart. + +### Alternatives considered + +- **A git repo per repo as the store** (the deleted `internal-vcs.md`). + Rejected by ruling 1: git packs, refs and locks are what Scion removed. + Git stays a derived export ([git-bridge](git-bridge.md)). +- **Client-uploaded changesets**, as Sapling's EdenAPI + `upload_changesets`. Rejected: Scion creates changesets on the server, + and one encoder keeps ids stable. +- **File contents in the database.** Rejected: Scion keeps contents in + blob storage, and contents dominate the bytes. +- **Object-store conditional writes as the ref authority.** Rejected: a + ref move must also order the ref log and assign `trunk_seq` in the same + transaction. +- **Store trees in each changeset, as git does.** Rejected: Scion's + changeset holds changed files, and the server builds trees once. +- **Store conflicts (jj `Merge`).** Deferred (OQ-5). + +### Decisions + +- **D1 — DL-578.** Saiph's store follows Scion: changesets hold ordered + parents, a 16-byte change id and the files changed against the first + parent, are created only by the server, and are encoded as deterministic + CBOR with BLAKE2b-256 ids keyed per type; file contents live + zstd-compressed in object storage; changesets, derived data, refs, the + ref log and pins live in a metadata database (YugabyteDB YSQL for many + servers, Postgres ≥16 on one node, one shared schema `saiph`); a ref + moves only by a single-row compare-and-swap. +- **D2 — DL-579.** The server creates commits, rebases and merges; + conflicts are typed results (paths, jj-ordered terms, hunks) and are + never stored; `UpdateRef` advances a ref by an ordered chain of + changesets in one CAS; derived data (trees, commit graph, ACL manifest + eager; path index and git commits on trunk; blame lazy) is a pure + function of a changeset and its parents, written back to the database. +- **D3 — DL-581.** Saiph is its own repo and product, with its own Connect + package `saiph.v1`, its own schema and tenant GUC, and per-tenant object + keys; it reads the caller from an `auth.Principal` built from a verified + issuer token (Compass is one issuer); per-directory ACLs are in-tree + `.saiphacl` files, and a denied read answers `NOT_FOUND`. + +## Plan + +### Global Constraints + +- **Public repo.** No operator names, hosts or private product detail. + Cite code by symbol and path. +- **Repo and language.** Code lives in `RigelBuild/saiph` (Go module + `github.com/RigelBuild/saiph`, Go 1.26, the toolchain Compass uses). + `connectrpc.com/connect/v2` only. Lint starts from Compass's + `go/.golangci.yml` rules: no `errors.As` (use `errors.AsType[T]`), no + `time.Sleep`, no `fmt.Print*`, no `context.Background`/`TODO` outside + `main` and tests. +- **Sapling and EdenFS: designs and ideas only, never code.** + `facebook/sapling` is GPL-2.0-only, and GPL-2.0-only code cannot be + copied into AGPL-3.0 code. No file, function, test or data table from + that repo is copied, translated or pasted, whatever license Saiph takes + ([overview](README.md#open-questions) OQ-1). Running an unmodified + EdenFS as a separate program is not in scope. +- **Scion first.** Every store choice follows the Scion post or appears in + [Deviations from Scion](#deviations-from-scion) or + [Gaps filled from Mononoke and Sapling](#gaps-filled-from-mononoke-and-sapling). + A task that departs further supersedes this record first. +- **Dependencies:** `github.com/fxamacker/cbor/v2` (MIT), + `golang.org/x/crypto/blake2b`, `github.com/jackc/pgx/v5`, + `github.com/minio/minio-go/v7` and `github.com/klauspost/compress`, the + last three as Compass uses them. No diff library. +- **Layout.** `internal/` holds `object`, `meta`, `blob`, `derive`, `acl`, + `textdiff`, `intake`, `gc`, `auth` and `api`; `cmd/saiph-server` is the + server binary. `cmd/saiph` and `cmd/saiphd` belong to [client](client.md). +- **Auth dependency:** `github.com/go-jose/go-jose/v4` (Apache-2.0), as + [client](client.md) uses it. +- **Database.** SQL uses only what YugabyteDB YSQL and Postgres 16 both + run. The `saiph` schema has its own migrations in + `internal/meta/migrations/`, numbered from `0001`. +- **Tests.** Every task is red/green. Database tests run on both Postgres + 16 and YugabyteDB; object-store tests use a Garage fixture or an + in-memory `Objects`. +- **Proto.** `buf lint` STANDARD with no `ignore_only` entry. + +### Order + +```text +S1 ids+encoding ─┬─ S2 meta ──┬─ S4 derive+acl ─ S6 intake+Commit ─┬─ S7 rebase/merge/diff ─┐ + └─ S3 blobs ─┘ ├─ S8 refs+pins+watch ───┼─ S10 gc ─ S11 wiring+caps +S5 proto (after S1) ──────────────────────────────────────────────┘ S9 trunk derive ───────┘ +``` + +### S1 — Ids and canonical encoding + +Encode and strictly decode changesets and tree nodes; keyed BLAKE2b-256 +ids; path rules; test vectors shared with every client. Tests: round-trip; +a reordered map, a non-shortest integer, an unsorted path and a bad path +each fail decode; the same bytes under two type keys give two ids; the +vectors file matches. + +Interfaces: + +```go +// internal/object +type ID [32]byte +type Digest struct{ ID ID; Size uint64 } +type Kind uint8 // KindFile, KindExecutable, KindSymlink, KindTree +type Signature struct{ Name, Email string; TimeUnixMs int64; TZOffsetMin int32 } +type CopyFrom struct{ Path string; Parent uint32 } +type FileChange struct{ Kind Kind; Content ID; Size uint64; CopyFrom *CopyFrom } +type Changeset struct { + Parents []ID + ChangeID [16]byte + Author Signature + Committer Signature + Message string + Changes map[string]*FileChange // nil value = delete + Extra map[string][]byte +} +type TreeEntry struct{ Name string; Kind Kind; ID ID; Size uint64 } +type HashKey string // KeyChangeset "saiph.changeset", KeyTree "saiph.tree", KeyACL "saiph.acl", KeyContent "saiph.content", KeyChunk "saiph.chunk" +func Hash(key HashKey, b []byte) ID // keyed BLAKE2b-256 +func ParseID(s string) (ID, error) +func (id ID) String() string +func BlobID(content []byte) ID // Hash(KeyContent, content) +func EncodeChangeset(cs *Changeset) ([]byte, ID, error) +func DecodeChangeset(b []byte) (*Changeset, ID, error) +func EncodeTree(entries []TreeEntry) ([]byte, ID, error) +func DecodeTree(b []byte) ([]TreeEntry, ID, error) +func CheckPath(p string) error +// testdata: internal/object/testdata/vectors.json +``` + +### S2 — Metadata schema and the ref CAS + +The `saiph` schema with RLS on every table, sqlc queries in +`internal/meta/queries/`, tenant transactions, ref-prefix grants, and +`UpdateRef` with its chain check and ref-log write. Tests, each on +Postgres 16 and on YugabyteDB: cross-tenant reads return no rows; two +racing `UpdateRef` calls give one winner and one `Current`; the ref log +is gapless under concurrency; `trunk_seq` is dense; a `release/` grant +moves `release/x` and not trunk; `caller_role` ignores prefix grants. + +Interfaces: + +```go +// internal/meta +type RepoID int64 +type Meta struct{ /* pgxpool on Postgres or YugabyteDB YSQL */ } +func Open(ctx context.Context, dsn string) (*Meta, error) +func (m *Meta) Migrate(ctx context.Context) error +func (m *Meta) EnsureTenant(ctx context.Context, tenant string) error +func (m *Meta) InTenant(ctx context.Context, tenant string, fn func(q *db.Queries) error) error +func (m *Meta) AsSystem(ctx context.Context, fn func(q *db.Queries) error) error +type RefUpdate struct { + Repo RepoID + Name string + Expected object.ID // zero = create + Target object.ID + Chain []object.ID // checked by the caller from immutable rows + Subject string + Reason string + Trunk bool +} +type RefResult struct { + Updated bool + Version int64 + Current object.ID + LogSeq int64 + TrunkSeqs []int64 +} +func (m *Meta) UpdateRef(ctx context.Context, tenant string, u RefUpdate) (RefResult, error) +func (m *Meta) Lease(ctx context.Context, tenant string, repo RepoID, kind int16, token string, ttl time.Duration) (cursor int64, ok bool, err error) +type Role int16 // RoleRead, RoleWrite, RoleLand, RoleAdmin +func (m *Meta) RefRole(ctx context.Context, tenant string, repo RepoID, subject, ref string) (Role, error) +type GitCommitRow struct{ Changeset object.ID; GitCommit [20]byte } +func (m *Meta) GitCommits(ctx context.Context, tenant string, repo RepoID, changesets []object.ID, gits [][20]byte) ([]GitCommitRow, error) +``` + +### S3 — Blob store + +Put, chunked put, get, find-missing, zstd framing, per-repo keys and quota +accounting, over an object store that wraps minio-go. Tests: same bytes +give the same id and one row; a chunk list with a wrong total hash is +refused; a quota overrun refuses before any PUT; `FindMissing` answers +from rows. + +Interfaces: + +```go +// internal/blob +type Objects interface { + Put(ctx context.Context, key string, body []byte) error + Get(ctx context.Context, key string) ([]byte, error) + Delete(ctx context.Context, key string) error + ListPrefix(ctx context.Context, prefix string, fn func(key string) error) error +} +type S3Config struct{ Endpoint, Bucket, AccessKey, SecretKey string; Secure bool } +func NewS3Objects(cfg S3Config) (Objects, error) +type Store struct{ /* Objects, *meta.Meta */ } +func New(objects Objects, m *meta.Meta) *Store +func Key(tenant string, repo meta.RepoID, id object.ID) string // //b/ +func (s *Store) FindMissing(ctx context.Context, tenant string, repo meta.RepoID, digests []object.Digest) ([]object.Digest, error) +func (s *Store) Put(ctx context.Context, tenant string, repo meta.RepoID, contents [][]byte) ([]object.Digest, error) +func (s *Store) PutChunked(ctx context.Context, tenant string, repo meta.RepoID, whole object.Digest, chunks []object.Digest) error +func (s *Store) Get(ctx context.Context, tenant string, repo meta.RepoID, id object.ID) (content []byte, chunks []object.Digest, err error) +``` + +### S4 — Derived framework, trees, commit graph and ACLs + +Eager derivation (root tree, commit graph columns, ACL manifest), the +skip pointer, `.saiphacl` parsing and the ACL checker. Tests: a tree from +changes equals a tree built from scratch; `SkipHeight` matches Bitcoin's +values for 0..1,024; an ancestor at depth d takes at most 2·log2(n) steps; +an ACL file change needs write in the enclosing scope; a denied read is +`NotFound`. + +Owner tests: an inherited `owners` list covers a child path; an owner +without write in the scope does not cover; a version that adds itself to +`owners` is not covered by the new entry. + +Interfaces: + +```go +// internal/derive +type Kind int16 // KindPathIndex, KindGitCommit, KindGitTree, KindContentMeta, KindBlame +type Node struct{ ID object.ID; Kind int16; Encoded []byte } // a tree or ACL node +type EagerResult struct { + RootTree object.ID + ACLRoot *object.ID + Generation int64 + P1Depth int64 + Skip *object.ID + Nodes []Node // inserted in the changeset's transaction +} +func SkipHeight(height int64) int64 +func Eager(ctx context.Context, env *Env, cs *object.Changeset, id object.ID) (EagerResult, error) +func Ancestor(ctx context.Context, env *Env, from object.ID, depth int64) (object.ID, error) +type Deriver interface { + Kind() Kind + Derive(ctx context.Context, env *Env, id object.ID) ([]byte, error) +} + +// internal/acl +type Rule struct{ Read, Write, Owners []string } +func ParseRule(b []byte) (Rule, error) +type Checker struct{ /* *derive.Env, grants */ } +func (c *Checker) CanRead(ctx context.Context, p auth.Principal, cs object.ID, path string) (bool, error) +func (c *Checker) CanWrite(ctx context.Context, p auth.Principal, cs object.ID, path string) (bool, error) +func (c *Checker) NarrowsRead(ctx context.Context, p auth.Principal, cs object.ID) (bool, error) +type UncoveredPath struct{ Path, Scope string; Owners []string } +func (c *Checker) CoverOwners(ctx context.Context, cs object.ID, paths, approvers []string) ([]UncoveredPath, error) +``` + +### S5 — Proto and generated code + +`proto/saiph/v1/saiph.proto` as in [API](#api-the-saiphv1-connect-package), +with the saiph repo's own `buf.yaml` and `buf.gen.yaml` (Go and TS). +Tests: `buf lint`, a drift check, and a round-trip of each message. + +Interfaces: the proto above; generated packages +`github.com/RigelBuild/saiph/gen/saiph/v1` (`saiphv1`) and +`github.com/RigelBuild/saiph/gen/saiph/v1/saiphv1connect`; a TS client +package for the Compass web app. + +### S6 — Intake, Commit and the read services + +Intake (`internal/intake`, steps 1–6), `Commit`, `CommitBatch`, +`RepoService` (with `SetGitMirrored`), `ObjectService` (with +`stored_form` and `omit_decoded`), `Log`, `MergeBase`, `IsAncestor` and +`CoverOwners`. Tests: intake is idempotent; a missing blob returns +`MissingObjects`; a write outside the caller's ACL scope is +`PERMISSION_DENIED`; a GET to `GetChangesets` works; a batch whose second +commit names the first as parent is stored; a caller without admin or +`import` cannot set `extra`; enabling `git_mirrored` with +`max_file_bytes` above 100 MiB is `FAILED_PRECONDITION`; `stored` bytes +decompress to bytes that hash to the id; `id_reads` is false for a caller +a `.saiphacl` narrows; `GetRepoResponse.tenant` is the token's tenant. + +Interfaces (every unary handler has this shape): + +```go +// internal/auth +type PrincipalKind uint8 // PrincipalAccount, PrincipalService +type Principal struct{ Kind PrincipalKind; ID, Tenant string; Scopes []string; Agent bool } +func WithPrincipal(ctx context.Context, p Principal) context.Context +func PrincipalFrom(ctx context.Context) (Principal, bool) + +// internal/intake +type Input struct { + Parents []object.ID + ChangeID [16]byte // zero = new random id + Author object.Signature + Committer object.Signature + Message string + Changes map[string]*object.FileChange + Extra map[string][]byte +} +type Intake struct{ /* *meta.Meta, *blob.Store, *derive.Env, *acl.Checker */ } +func (in *Intake) Create(ctx context.Context, p auth.Principal, repo meta.RepoID, inputs []Input) ([]object.ID, error) + +// internal/api +type Deps struct{ Meta *meta.Meta; Blobs *blob.Store; Env *derive.Env; ACL *acl.Checker; Intake *intake.Intake } +func Mount(mux *http.ServeMux, deps Deps, opts ...connect.HandlerOption) + +func (s *mutationService) Commit(ctx context.Context, req *saiphv1.CommitRequest) (*saiphv1.CommitResponse, error) +func (s *objectService) GetChangesets(ctx context.Context, req *saiphv1.GetChangesetsRequest) (*saiphv1.GetChangesetsResponse, error) +func (s *objectService) ReadFile(ctx context.Context, req *saiphv1.ReadFileRequest, stream saiphv1connect.ObjectServiceReadFileServerStream) error +func (s *historyService) MergeBase(ctx context.Context, req *saiphv1.MergeBaseRequest) (*saiphv1.MergeBaseResponse, error) +func (s *mutationService) CommitBatch(ctx context.Context, req *saiphv1.CommitBatchRequest) (*saiphv1.CommitBatchResponse, error) +func (s *repoService) SetGitMirrored(ctx context.Context, req *saiphv1.SetGitMirroredRequest) (*saiphv1.SetGitMirroredResponse, error) +func (s *historyService) CoverOwners(ctx context.Context, req *saiphv1.CoverOwnersRequest) (*saiphv1.CoverOwnersResponse, error) +``` + +### S7 — Text merge, Rebase, Merge and Diff + +The histogram diff and three-way merge, `Rebase`, `Merge` and `Diff`. +Tests: a clean rebase keeps change ids; an overlapping edit returns a +`CONTENT` conflict with `adds = [dest, src]`, `removes = [base]` and the +right line range; `drop_empty` drops a change trunk holds; criss-cross +merge bases give `MULTIPLE_BASES`; `Diff` with `paths` returns only those +prefixes. + +Resolution tests: a resolution on a conflicted content path lands its +content; one on a clean path or a `.saiphacl` is `INVALID_ARGUMENT`; a +`REPO_ROLE_WRITE` caller is `PERMISSION_DENIED`; missing content is +`FAILED_PRECONDITION`; an uncovered conflict still returns `Conflict`. + +Interfaces: + +```go +// internal/textdiff +type LineRange struct{ Start, Count int } // Start is 1-based +type Edit struct{ Old, New LineRange } +type ConflictHunk struct{ Dest LineRange; Adds, Removes [][]byte } +func IsBinary(b []byte) bool +func Diff(old, new []byte) []Edit +func Merge3(base, dest, src []byte) (merged []byte, conflicts []ConflictHunk) + +// internal/intake +type Resolution struct{ Changeset object.ID; Path string; Content object.ID; Kind object.Kind } + +func (s *mutationService) Rebase(ctx context.Context, req *saiphv1.RebaseRequest) (*saiphv1.RebaseResponse, error) +func (s *mutationService) Merge(ctx context.Context, req *saiphv1.MergeRequest) (*saiphv1.MergeResponse, error) +func (s *historyService) Diff(ctx context.Context, req *saiphv1.DiffRequest) (*saiphv1.DiffResponse, error) +``` + +### S8 — Ref service, pins and the ref stream + +`GetRef`, `ListRefs`, `UpdateRef` (chain check, protected refs), +`DeleteRef`, `WatchRefs` (a ref-log poller, rows as truth), `Pin`, `Unpin`. +Tests: a chain whose links do not connect is `INVALID_ARGUMENT`; a +non-`land` caller cannot move trunk; a lost CAS returns `updated=false` +and the current ref; a stream resumed at `after_log_seq` misses nothing. + +Interfaces: + +```go +func (s *refService) UpdateRef(ctx context.Context, req *saiphv1.UpdateRefRequest) (*saiphv1.UpdateRefResponse, error) +func (s *refService) WatchRefs(ctx context.Context, req *saiphv1.WatchRefsRequest, stream saiphv1connect.RefServiceWatchRefsServerStream) error +func (s *refService) Pin(ctx context.Context, req *saiphv1.PinRequest) (*saiphv1.PinResponse, error) +type RefLogEntry struct{ LogSeq int64; Name string; Old, New object.ID; Chain []object.ID; TrunkSeqs []int64; Subject, Reason string; AtUnixMs int64 } +func (m *Meta) RefLogAfter(ctx context.Context, tenant string, repo RepoID, after int64, limit int) ([]RefLogEntry, error) +// internal/api: one poller per watched repo per replica, every 250 ms +func newRefPoller(m *meta.Meta, every time.Duration) *refPoller +``` + +### S9 — On-trunk and lazy derivation + +Derive workers for the path index and the git-commit slot (the git-bridge +`Deriver` plugs in, for repos with `git_mirrored`), the lazy git-tree +slot, lazy content meta and blame, `PathHistory`, `Blame`, +`GetContentMeta` and `GetGitCommits`. Tests: a worker resumes from its +cursor after a lease loss; `PathHistory` on trunk is one range scan; +blame of a moved line names the original changeset; `GetContentMeta` +returns the SHA-256 of the blob's bytes and leaves out an unknown id; +`GetGitCommits` answers both directions. + +Interfaces: + +```go +// internal/derive +func RunTrunkWorker(ctx context.Context, env *Env, d Deriver) error +func Register(d Deriver) // git-bridge registers KindGitCommit and KindGitTree +func (s *historyService) PathHistory(ctx context.Context, req *saiphv1.PathHistoryRequest) (*saiphv1.PathHistoryResponse, error) +func (s *historyService) Blame(ctx context.Context, req *saiphv1.BlameRequest) (*saiphv1.BlameResponse, error) +func (s *objectService) GetContentMeta(ctx context.Context, req *saiphv1.GetContentMetaRequest) (*saiphv1.GetContentMetaResponse, error) +func (s *historyService) GetGitCommits(ctx context.Context, req *saiphv1.GetGitCommitsRequest) (*saiphv1.GetGitCommitsResponse, error) +``` + +### S10 — GC and retention + +Changeset mark-sweep, the blob delete race guard, the weekly node sweep and +repo delete. Tests: a pinned changeset survives; an expired pin does not; a +blob touched by an intake in flight survives the sweep; repo delete frees +the name after 30 days. + +Interfaces: + +```go +// internal/gc +type Config struct{ Every, NodeSweepEvery, RepoDeleteAfter time.Duration } +func Run(ctx context.Context, m *meta.Meta, b *blob.Store, cfg Config) error +func CollectRepo(ctx context.Context, m *meta.Meta, b *blob.Store, tenant string, repo meta.RepoID, now time.Time) (Stats, error) +type Stats struct{ Changesets, Blobs, Nodes int64; Bytes int64 } +``` + +### S11 — Server binary, auth, deployment and caps + +`cmd/saiph-server` with `--dsn`, `--bucket`, `--cache-bytes` and +`--issuers` (a file of trusted issuers). `internal/auth` verifies issuer +JWTs, with keys from the file or discovered from the issuer URL, and +Saiph's own hashed tokens, into an `auth.Principal` (OQ-6). Deploy +manifests in `deploy/` (a Deployment with `replicas: N`, a Service, a +migration Job) and the caps rows. Tests: a token from an unknown issuer +is `UNAUTHENTICATED`; the token's tenant, not a request field, scopes +every row; a token without `exp` or `iat`, with a lifetime over one hour, +or with a wrong `aud` is refused; an unknown `kid` refetches keys at most +once per five minutes; a Kubernetes token maps to `service:k8s//` +in the issuer's tenant with scope `read`; a `tenant` claim that differs +from `Issuer.Tenant` is refused; an issuer with `Subjects` rejects any +other `sub`; `agent: true` sets `Principal.Agent`; two replicas serve one +repo with one consistent trunk; a quota refusal end to end against +Garage with a bucket quota set. + +Interfaces: + +```go +// internal/auth +type Issuer struct { + Name, URL string + Keys []crypto.PublicKey // empty = discover from URL + Provision bool + Audience string // default "saiph" + Tenant string // required when Kubernetes; else must match the tenant claim if set + CAFile string // trust roots for discovery; empty = system roots + Kubernetes bool // sub system:serviceaccount:: -> service:k8s//, scope read + Subjects []string // non-empty: the only sub values this issuer may mint +} +type Claims struct { + jwt.Claims // iss, sub, aud, exp, iat (go-jose/v4/jwt); exp and iat required + Tenant string `json:"tenant,omitempty"` + Scope string `json:"scope,omitempty"` // space-separated + Agent bool `json:"agent,omitempty"` // the subject is an agent's account +} +const MaxTokenLifetime = time.Hour +var Scopes = []string{"read", "write", "land", "provision", "import"} +type Verifier struct{ /* issuers, *meta.Meta for saiph.tokens */ } +func NewVerifier(issuers []Issuer, m *meta.Meta) *Verifier +func (v *Verifier) Verify(ctx context.Context, bearer string) (Principal, error) +func Interceptor(v *Verifier) connect.ServerInterceptor +func discoverKeys(ctx context.Context, hc *http.Client, issuerURL string) (jose.JSONWebKeySet, error) +func (i Issuer) principal(c Claims) (Principal, error) +// cmd/saiph-server +type serverFlags struct{ DSN, Bucket, Issuers string; CacheBytes int64 } +func registerFlags(fs *flag.FlagSet) *serverFlags +``` + +## Tasks + +- [ ] S1 — ids and canonical encoding (`internal/object`, test vectors) +- [ ] S2 — `saiph` schema, RLS, `Meta`, ref CAS, ref log and ref-prefix grants +- [ ] S3 — blob store, chunking, zstd framing, quotas, minio-go objects +- [ ] S4 — eager derivation: trees, commit graph and skip list, ACL manifest, owners and checker +- [ ] S5 — `proto/saiph/v1/saiph.proto` and generated code +- [ ] S6 — intake, `Commit`, `CommitBatch`, repo, object and read history services, `CoverOwners` +- [ ] S7 — text merge, `Rebase` with resolutions, `Merge`, `Diff` +- [ ] S8 — ref service, chain CAS, pins, `WatchRefs` +- [ ] S9 — on-trunk and lazy derivation, `PathHistory`, `Blame`, `GetContentMeta`, `GetGitCommits` +- [ ] S10 — gc and retention +- [ ] S11 — `saiph-server` binary, issuer auth and discovery, deploy manifests, caps rows and budget alert