docs(compass): design the Compass Server on Kubernetes (RIG-5559) by rigel-mintaka · Pull Request #38 · RigelBuild/docs

rigel-mintaka · 2026-10-17T07:18:57Z

Summary

Design record docs/designs/compass/server-on-kubernetes.md: the OSS product changes Compass needs to run its Server and Runner on Kubernetes. Decisions D1–D10 are ledgered as DL-539 to DL-547 and DL-560.

OQ-13 resolved (Matt: option 1, DL-577). P7 ships an arm64 build gate, and the Runner stays amd64-only until an arm64 KVM boot passes. No open questions remain.

Decisions

D1 topology: one pod per container child, a 1-replica Recreate Server, one Runner per Server.
D2 Server image: a nix closure on distroless.
D3 Postgres is user-provided, never rendered. Migrations run at boot. The owner is a superuser, or a CREATEROLE + BYPASSRLS role on PG ≥16.
D4 secrets by name, and a boot-only master key.
D5 NATS with password auth.
D6 the Server terminates TLS.
D7 Runner serverCA and egressAllow.
D8 paired :main-<n>-<sha12> tags, with only the digest written to pod specs.
D9 all images are public, and registry reads are anonymous.
D10 every image is multi-arch (amd64 + arm64).

Tasks

Task	Change	State
P1	Server image and publish lane	done (#2355)
P2	`tools/server-manifests` renderer: no Postgres or init container, digest-only images	follow-up to merged #2363, after P4
P3	Runner `serverCA` and `egressAllow`	done (#2358, #2360)
P4	Admin token read from a Secret volume (Go)	todo
P5	Anonymous registry reads in every lane	todo
P6	Paired tags on the head of every main push	todo
P7	Multi-arch images, index-aware retag and tagger, per-arch guest kernel and console	todo
P8	Runner volume root on a quota subdirectory of the session mount	todo
P9	Runner `--server-name` sets the TLS server name; renderer `serverName`	todo

Spec-impact: P4 changes the admin-token requirement in docs/specs/product/compass.md ("SHALL name a private regular file") and the --admin-token-file help text.
Ledger-impact: DL-539..547 and DL-560.

Refs RIG-5559

linear-code · 2026-10-17T07:19:03Z

RIG-5559

rigel-mintaka · 2026-10-17T08:44:16Z

Review result at `880e093`

Open: High 0 / Medium 0 / Low 0.

Pre-review: a design-critic pass raised 24 findings. All are addressed, and the forks are recorded as OQs.
Round 1 (3c1c44a): 4 high and 4 medium across correctness and design. Fixed in 3052ece.
Round 2: 2 medium. Fixed in 1fc4dfb.
Round 3: 1 medium. Fixed in 880e093.
Round 4: clean.

Open decisions for Matt: OQ-4 and OQ-8 through OQ-11, each with a recommendation. OQ-5 gates T7.

github-actions · 2026-10-17T22:27:29Z

Rigel public design docs preview: https://compass-managed-rig-5559-ser.docs-2ui.pages.dev

Deployed from compass-managed/rig-5559-server-on-k8s at d0abefc.

rigel-mintaka · 2026-10-18T01:42:42Z

Review result at `aea68e8`

Open: High 0 / Medium 0 / Low 0.

At c39c164 (first round after the narrowing): 1 high (deployment detail in the title, body and commit messages) and 4 medium: no PG16 floor, the Flux setter target not digest-only, a P4 open that blocks on a FIFO, and DL-547 claiming more than P5 delivered. There were also 3 low.
At c88bafd: 1 high (arm64 guest kernel name and console) and 1 medium (reads still authenticated through REGISTRY_AUTH_FILE).
At 4e68a8f: 1 medium (an anonymous pre-push probe blocks a new package) and 1 low (the PR body).
At aea68e8: clean.

Open for Matt: OQ-13, the arm64 microVM boot proof. Recommendation: option 1.

rigel-mintaka · 2026-10-18T04:42:07Z

Review result at `f8e01a3`

Open: High 0 / Medium 0 / Low 0.

This round adds P8. The Runner renderer points COMPASS_MICROVM_VOLUME_ROOT at the mount point itself, so the quota preflight (readVolumeQuota) fails on every rendered Runner. P8 moves the volume root to /var/lib/compass/sessions/volumes. P2 is reworded as a follow-up to the merged #2363. The review checked P8 against compass main 049a1af5 and found nothing. OQ-13 is still open for Matt.

Server image, a public server-manifests renderer, and Runner egress and CA values. Ledgers D1-D9 as DL-539 to DL-547. Co-authored-by: Matt Wilkinson <matt@rigel.build>

…r key (RIG-5559) Review fixes: forge args and App secrets, one Runner pod per Server, and a boot-only master key. Co-authored-by: Matt Wilkinson <matt@rigel.build>

…ewer App optional (RIG-5559) Review fixes: OQ-5 lists owner-role attributes per option, and brokered push needs only the primary App. Refs RIG-5559 Co-authored-by: Matt Wilkinson <matt@rigel.build>

…G-5559) Review fix: the token covers the agent grants plus its owner grants. Refs RIG-5559 Co-authored-by: Matt Wilkinson <matt@rigel.build>

…IG-5559) The record now covers only what the OSS product needs to run on Kubernetes: the admin token read from a Secret volume, public images, user-provided Postgres, and paired orderable image tags. Co-authored-by: Matt Wilkinson <matt@rigel.build>

Postgres 16 floor, digest-only image updates, non-blocking admin token open, anonymous registry reads everywhere, and multi-arch images as D10 and P7. Co-authored-by: Matt Wilkinson <matt@rigel.build>

… reads (RIG-5559) Co-authored-by: Matt Wilkinson <matt@rigel.build>

…-5559) Co-authored-by: Matt Wilkinson <matt@rigel.build>

…RIG-5559) Adds P8. P2 is now a follow-up to the merged first renderer. Co-authored-by: Matt Wilkinson <matt@rigel.build>

…(RIG-5559) Records the arm64 build-gate ruling as DL-577 and adds P9, a Runner --server-name that sets tls.Config.ServerName. Co-authored-by: Matt Wilkinson <matt@rigel.build>

rigel-mintaka · 2026-10-18T15:42:45Z

Review result at `d0abefc`

Open: High 0 / Medium 0 / Low 0.

Rebased onto main.
OQ-13 is resolved as option 1 (DL-577), and the record has no open questions left.
New P9: the Runner gets --server-name, which sets tls.Config.ServerName, plus a renderer value serverName. This lets a Runner dial the in-cluster Service and still verify a certificate that names only the public host. The private deployment record needs it for its Managed tier.
The review checked P9 against compass main 62fd1b29 and the Go 1.26 stdlib and found nothing.

rigel-mintaka changed the title ~~docs(compass): design the Compass Server on Kubernetes, k3s first then EKS (RIG-5559)~~ docs(compass): design the Compass Server on Kubernetes, k3s first then EKS Oct 17, 2026

rigel-mintaka marked this pull request as ready for review October 17, 2026 08:45

rigel-mintaka requested a review from mattwilkinsonn as a code owner October 17, 2026 08:45

mattwilkinsonn reviewed Oct 17, 2026

View reviewed changes

rigel-mintaka force-pushed the compass-managed/rig-5559-server-on-k8s branch from c39c164 to c88bafd Compare October 17, 2026 22:25

rigel-mintaka changed the title ~~docs(compass): design the Compass Server on Kubernetes, k3s first then EKS~~ docs(compass): design the Compass Server on Kubernetes (RIG-5559) Oct 17, 2026

mattwilkinsonn reviewed Oct 18, 2026

View reviewed changes

Comment thread docs/designs/compass/server-on-kubernetes.md Outdated

rigel-mintaka and others added 10 commits October 18, 2026 11:40

docs(compass): design the Compass Server on Kubernetes (RIG-5559)

e297d24

Server image, a public server-manifests renderer, and Runner egress and CA values. Ledgers D1-D9 as DL-539 to DL-547. Co-authored-by: Matt Wilkinson <matt@rigel.build>

docs(compass): wire the forge App, pin one Runner and guard the maste…

564d77a

…r key (RIG-5559) Review fixes: forge args and App secrets, one Runner pod per Server, and a boot-only master key. Co-authored-by: Matt Wilkinson <matt@rigel.build>

docs(compass): scope Postgres role needs per option and make the revi…

0d3123a

…ewer App optional (RIG-5559) Review fixes: OQ-5 lists owner-role attributes per option, and brokered push needs only the primary App. Refs RIG-5559 Co-authored-by: Matt Wilkinson <matt@rigel.build>

docs(compass): scope the brokered token to effective forge grants (RI…

b216392

…G-5559) Review fix: the token covers the agent grants plus its owner grants. Refs RIG-5559 Co-authored-by: Matt Wilkinson <matt@rigel.build>

docs(compass): fix the Kubernetes record review findings (RIG-5559)

7b687d7

Postgres 16 floor, digest-only image updates, non-blocking admin token open, anonymous registry reads everywhere, and multi-arch images as D10 and P7. Co-authored-by: Matt Wilkinson <matt@rigel.build>

docs(compass): cover the arm64 guest boot path and explicit anonymous…

8872622

… reads (RIG-5559) Co-authored-by: Matt Wilkinson <matt@rigel.build>

docs(compass): keep the pre-push destination probe authenticated (RIG…

3f8c10b

…-5559) Co-authored-by: Matt Wilkinson <matt@rigel.build>

docs(compass): point the Runner volume root at a quota subdirectory (…

7a78ca0

…RIG-5559) Adds P8. P2 is now a follow-up to the merged first renderer. Co-authored-by: Matt Wilkinson <matt@rigel.build>

docs(compass): resolve OQ-13 and add the Runner TLS server name task …

d0abefc

…(RIG-5559) Records the arm64 build-gate ruling as DL-577 and adds P9, a Runner --server-name that sets tls.Config.ServerName. Co-authored-by: Matt Wilkinson <matt@rigel.build>

rigel-mintaka force-pushed the compass-managed/rig-5559-server-on-k8s branch from f8e01a3 to d0abefc Compare October 18, 2026 15:40

mattwilkinsonn merged commit dfb1b54 into main Oct 18, 2026
2 checks passed

mattwilkinsonn deleted the compass-managed/rig-5559-server-on-k8s branch October 18, 2026 18:22

Navigation Menu

Repository navigation

docs(compass): design the Compass Server on Kubernetes (RIG-5559) - #38

docs(compass): design the Compass Server on Kubernetes (RIG-5559)#38
mattwilkinsonn merged 10 commits into
mainfrom
compass-managed/rig-5559-server-on-k8s

rigel-mintaka commented Oct 17, 2026 •

edited

Loading

Uh oh!

linear-code Bot commented Oct 17, 2026

Uh oh!

rigel-mintaka commented Oct 17, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions Bot commented Oct 17, 2026 •

edited

Loading

Uh oh!

rigel-mintaka commented Oct 18, 2026

Uh oh!

rigel-mintaka commented Oct 18, 2026

Uh oh!

Uh oh!

rigel-mintaka commented Oct 18, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

rigel-mintaka commented Oct 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Decisions

Tasks

Uh oh!

linear-code Bot commented Oct 17, 2026

Uh oh!

rigel-mintaka commented Oct 17, 2026

Review result at 880e093

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions Bot commented Oct 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rigel-mintaka commented Oct 18, 2026

Review result at aea68e8

Uh oh!

rigel-mintaka commented Oct 18, 2026

Review result at f8e01a3

Uh oh!

Uh oh!

rigel-mintaka commented Oct 18, 2026

Review result at d0abefc

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

rigel-mintaka commented Oct 17, 2026 •

edited

Loading

Review result at `880e093`

github-actions Bot commented Oct 17, 2026 •

edited

Loading

Review result at `aea68e8`

Review result at `f8e01a3`

Review result at `d0abefc`