From 1feac85c40d63091170e35b9c0ec63b8a98802df Mon Sep 17 00:00:00 2001 From: Cursor Agent Date: Thu, 15 Oct 2026 08:08:32 +0000 Subject: [PATCH] V3: BlueField feature path and CPU production hooks Add an empty dpu-bf3 slot, a simulated storage-queue contract that stays NOT_RUN_HARDWARE, and CPU-testable authz, degrade, cold-ops, and paired-gate scaffolding. Default tests do not link DOCA, SPDK, or libnixl. Co-authored-by: RustFS --- .github/workflows/ci.yml | 10 + AGENTS.md | 12 +- COMPATIBILITY.md | 19 +- Cargo.lock | 7 + Cargo.toml | 2 + README.md | 42 ++- THIRD_PARTY_NOTICES.md | 12 +- crates/bench/Cargo.toml | 4 + crates/bench/src/bin/v3_paired.rs | 16 + crates/bench/src/lib.rs | 66 +++- crates/cold/src/lib.rs | 9 +- crates/cold/src/ops.rs | 479 +++++++++++++++++++++++++ crates/cold/src/pack.rs | 43 ++- crates/cold/src/sink.rs | 6 +- crates/core-types/src/error.rs | 2 + crates/dpu/Cargo.toml | 24 ++ crates/dpu/src/ablation.rs | 112 ++++++ crates/dpu/src/bin/dpu_demo.rs | 24 ++ crates/dpu/src/degrade.rs | 68 ++++ crates/dpu/src/demo.rs | 116 ++++++ crates/dpu/src/lib.rs | 27 ++ crates/dpu/src/queue.rs | 441 +++++++++++++++++++++++ crates/dpu/src/reactor.rs | 56 +++ crates/dpu/src/target.rs | 83 +++++ crates/protocol/src/harden.rs | 478 ++++++++++++++++++++++++ crates/protocol/src/lib.rs | 8 +- crates/protocol/src/reply.rs | 2 + crates/transport/src/directory.rs | 6 + crates/transport/src/lib.rs | 4 + crates/transport/src/plane.rs | 295 +++++++++++++++ crates/transport/src/rate.rs | 123 +++++++ docs/ARCHITECTURE.md | 23 +- docs/BENCHMARKS.md | 22 ++ docs/OPERATIONS.md | 42 ++- docs/RISKS.md | 11 +- docs/SECURITY.md | 18 +- docs/V3_DPU_TARGETS.md | 21 ++ docs/V3_FAILURES.md | 20 ++ docs/V3_TEST_MATRIX.md | 35 ++ docs/adr/0001-integration-route.md | 4 + docs/adr/0002-no-erasure-coding.md | 1 + docs/adr/0006-storage-queue-offload.md | 43 +++ docs/adr/0007-trusted-fabric.md | 37 ++ docs/adr/0008-second-engine.md | 22 ++ 44 files changed, 2861 insertions(+), 34 deletions(-) create mode 100644 crates/bench/src/bin/v3_paired.rs create mode 100644 crates/cold/src/ops.rs create mode 100644 crates/dpu/Cargo.toml create mode 100644 crates/dpu/src/ablation.rs create mode 100644 crates/dpu/src/bin/dpu_demo.rs create mode 100644 crates/dpu/src/degrade.rs create mode 100644 crates/dpu/src/demo.rs create mode 100644 crates/dpu/src/lib.rs create mode 100644 crates/dpu/src/queue.rs create mode 100644 crates/dpu/src/reactor.rs create mode 100644 crates/dpu/src/target.rs create mode 100644 crates/protocol/src/harden.rs create mode 100644 crates/transport/src/plane.rs create mode 100644 crates/transport/src/rate.rs create mode 100644 docs/V3_DPU_TARGETS.md create mode 100644 docs/V3_FAILURES.md create mode 100644 docs/V3_TEST_MATRIX.md create mode 100644 docs/adr/0006-storage-queue-offload.md create mode 100644 docs/adr/0007-trusted-fabric.md create mode 100644 docs/adr/0008-second-engine.md diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index b6c31b6..60303af 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -28,8 +28,14 @@ jobs: run: | cargo run -p gpucache-transport --bin shard-demo --quiet cargo run -p gpucache-cold --bin cold-demo --quiet + - name: V3 DPU contract and paired gate + run: | + cargo run -p gpucache-dpu --bin dpu-demo --quiet + cargo run -p gpucache-bench --bin v3-paired --quiet - name: Optional RDMA feature still skips hardware run: cargo test -p gpucache-transport --all-targets --features rdma-nixl + - name: Optional DPU feature still skips hardware + run: cargo test -p gpucache-dpu --all-targets --features dpu-bf3 - name: Reject accelerator linkage and erasure coding run: | python3 - <<'PY' @@ -62,6 +68,10 @@ jobs: echo "RustFS cold tier must not be a hot-path dependency" exit 1 fi + if grep -n 'gpucache-dpu' crates/storage/Cargo.toml crates/transport/Cargo.toml crates/protocol/Cargo.toml crates/core-types/Cargo.toml crates/cold/Cargo.toml; then + echo "DPU crate must stay off the hot path and off the cold crate" + exit 1 + fi test-aarch64: name: test (aarch64 runner) diff --git a/AGENTS.md b/AGENTS.md index c942fb6..e30b15b 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -4,14 +4,15 @@ GPUCache is the G3.5 shared KV tier. RustFS is the colder G4 backend. Read [READ ## Phase -The active phase is the **V2 first slice** (explicitly authorized). V1 A/B/C stay in tree and must keep passing. Do not start V3 DPU, DOCA, SPDK, or GDS work. Do not publish crates, cut releases, deploy, or edit a website unless asked. +The active phase is **V3** (explicitly authorized). V1 A/B/C and the V2 first slice stay in tree and must keep passing. Do not start work past V3. Do not purchase hardware. Do not publish crates, cut releases, deploy, or edit a website unless asked. ## Honesty rules -- Do not claim DPU, DOCA, RDMA, SPDK, or CUDA acceleration. The default build is CPU and TCP. Feature `rdma-nixl` does not link a device. An aarch64 build is not a BlueField test. GDS and BlueField stay in development / `NOT_RUN_HARDWARE`. -- Do not invent vLLM, LMCache, or NIXL results. A real engine KV export/import needs a GPU and stays `NOT_RUN_HARDWARE` until that run exists. +- Do not claim DPU, DOCA, RDMA, SPDK, GDS, or CUDA acceleration. The default build is CPU and TCP. Feature `rdma-nixl` does not link a device. Feature `dpu-bf3` does not link DOCA, SPDK, or an NVMe device. An aarch64 build is not a BlueField test. Public wording for BlueField-3, GDS, and DPU stays in development / `NOT_RUN_HARDWARE` / optional feature. +- Do not invent vLLM, LMCache, NIXL, TTFT, or TCO results. A real engine KV export/import needs a GPU and stays `NOT_RUN_HARDWARE` until that run exists. Cost stays `TCO_UNKNOWN` until a paired NIC-RDMA vs DPU run exists. TCP is not the DPU win baseline. - Do not treat a KV block as model-agnostic long-term memory. A miss falls back to recompute. A wrong-tenant or wrong-layout hit is a defect. - Do not add erasure coding, parity, or durability replicas. A lost or failed hot block is a clean miss, then recompute and repopulate. Cold durability belongs to RustFS. An extra hot-prefix copy, if it is ever added, is a latency hint and not a durability promise. +- Do not call the tree PB-scale verified or production-ready. Arm process start is not a completed offload. Host GPUNetIO NIC-mode samples are not a DPU Arm storage daemon. - Docs in this repo are English. ## Crates @@ -23,9 +24,10 @@ The active phase is the **V2 first slice** (explicitly authorized). V1 A/B/C sta | `gpucache-storage` | the two crates above | | `gpucache-transport` | the three crates above. TCP correctness path. Optional feature `rdma-nixl` must add no crate | | `gpucache-cold` | `gpucache-core-types` and `sha2`. Not a dependency of storage or transport | +| `gpucache-dpu` | `gpucache-core-types` only. Optional feature `dpu-bf3` must add no crate. Not a dependency of storage, transport, protocol, or cold | | `gpucache-bench` | nothing. Prints `NOT_RUN_HARDWARE` baseline rows and must not invent timings | -No crate may add a CUDA, DOCA, NIXL, RDMA, SPDK, erasure-coding, or replica-placement dependency. The feature name `rdma-nixl` is a slot only. `unsafe` is forbidden. Registered buffers whose completion is unknown stay quarantined until drain or a newer epoch; do not free them only because a lease timer fired. The cold crate stays off the hot path. +No crate may add a CUDA, DOCA, NIXL, RDMA, SPDK, erasure-coding, or replica-placement dependency. The feature names `rdma-nixl` and `dpu-bf3` are slots only. `unsafe` is forbidden. Registered buffers whose completion is unknown stay quarantined until drain or a newer epoch; do not free them only because a lease timer fired. The cold crate stays off the hot path. The DPU crate stays off the hot path until a measured offload exists. ## Checks @@ -35,4 +37,4 @@ cargo clippy --workspace --all-targets -- -D warnings cargo test --workspace --all-targets ``` -`cargo test --workspace --all-targets` is the one-command CPU repro. +`cargo test --workspace --all-targets` is the one-command CPU repro. It must pass with default features and without DOCA or libnixl. diff --git a/COMPATIBILITY.md b/COMPATIBILITY.md index e6a80b8..5785cdd 100644 --- a/COMPATIBILITY.md +++ b/COMPATIBILITY.md @@ -1,6 +1,6 @@ # Compatibility manifest -Recorded for V1 milestones A, B, and C. This file is not a performance acceptance report and not a hardware inventory of a BlueField node. +Recorded for V1 milestones A, B, and C, the V2 first slice, and the V3 CPU slice. This file is not a performance acceptance report and not a hardware inventory of a BlueField node. | Component | Value | Result | | --- | --- | --- | @@ -14,12 +14,23 @@ Recorded for V1 milestones A, B, and C. This file is not a performance acceptanc | vLLM | Not pinned; no engine commit | `NOT_RUN_HARDWARE` | | LMCache | Not a dependency | `NOT_RUN_HARDWARE` (route undecided) | | NIXL / UCX / RDMA | Not linked. Candidate recorded: `nixl-sys` 1.3.1 (crates.io, updated 2026-07-07; docs.rs also lists 1.5.0), Apache-2.0, repo `ai-dynamo/nixl`. Feature `rdma-nixl` has no crate dependency. `stub-api` aborts on call, so it is not used. Mooncake Transfer Engine is Apache-2.0 and was not linked (in-tree `unsafe` FFI, not a pinned crates.io crate) | Live RDMA `NOT_RUN_HARDWARE`. `NixlTransport::open` returns `rdma not linked` with and without the feature | -| DOCA / BlueField BFB / firmware / GDS | Not present. Wording stays in development / `NOT_RUN_HARDWARE` | `NOT_RUN_HARDWARE` | +| DOCA / BlueField BFB / firmware / GDS | Not present. Feature `dpu-bf3` is empty. Planned mode, BFB, OS, DOCA, firmware, and IOMMU values are unset targets in `docs/V3_DPU_TARGETS.md`. Wording stays in development / `NOT_RUN_HARDWARE` / optional feature | `NOT_RUN_HARDWARE` | | RustFS server / client | No endpoint authorized in this environment. Contract mock covers Put, Head, RangeGet, Delete, pack, manifest, async sink, and cost-bounded refill | Live probe `NOT_RUN_HARDWARE`. Mock `PASS` | | TCP / file store | Loopback TCP and a temp directory on the build host | `PASS` for `cargo test --workspace --all-targets` on x86_64. Not an NVMe format and not a DPU run | -| Direct Rust deps | `sha2` 0.10.8; dev `proptest` 1.4.0 | Apache-2.0 OR MIT; see `Cargo.lock`. The V2 slice adds no direct crate. `gpucache-bench` has an empty dependency list. Optional feature `rdma-nixl` adds none | +| Direct Rust deps | `sha2` 0.10.8; dev `proptest` 1.4.0 | Apache-2.0 OR MIT; see `Cargo.lock`. V2 and V3 add no direct crate. `gpucache-bench` and `gpucache-dpu` add no third-party crate. Optional features `rdma-nixl` and `dpu-bf3` add none | +| SPDK | Not linked. `AffineQueue` is a reactor-id check with no thread | `NOT_RUN_HARDWARE` | +| mTLS | Not linked. Boundary is the trusted fabric in ADR 0007 | `NOT_RUN` (library hooks for ACL, replay, and seals are CPU tests) | | License of this repo | Apache-2.0 | See `LICENSE` | CI (see `.github/workflows/ci.yml`) builds and runs the same tests on x86_64 and on aarch64 (GitHub ARM runner, plus qemu-user cross execution). An aarch64 result is a CPU result. It does not certify BlueField-3, DOCA, or NVMe-on-DPU. -The integration decision between a native vLLM KV connector and LMCache is pending. No engine commit is pinned because the spike did not run. +The integration decision between a native vLLM KV connector and LMCache is pending. No engine commit is pinned because the spike did not run. SGLang / HiCache is an evaluation note only (ADR 0008) and is not a dependency. + +| Path | Default build | Optional feature | Hardware | +| --- | --- | --- | --- | +| TCP + file store | CPU tests | none | CPU `PASS` on the V1/V2 suite. Not an NVMe format | +| `rdma-nixl` | `open` refuses | empty feature, still refuses | `NOT_RUN_HARDWARE` | +| `dpu-bf3` | stubs refuse | empty feature, still refuses | `NOT_RUN_HARDWARE` | +| aarch64 | same Rust core | none | CPU only. Not a BlueField test | +| RustFS | mock | `GPUCACHE_RUSTFS_*` | Live `NOT_RUN_HARDWARE` | +| NIC RDMA vs DPU paired gate | `v3-paired` text | none | `NOT_RUN_HARDWARE`, `TCO_UNKNOWN` | diff --git a/Cargo.lock b/Cargo.lock index e6261d6..d54a575 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -147,6 +147,13 @@ dependencies = [ "sha2", ] +[[package]] +name = "gpucache-dpu" +version = "0.1.0" +dependencies = [ + "gpucache-core-types", +] + [[package]] name = "gpucache-protocol" version = "0.1.0" diff --git a/Cargo.toml b/Cargo.toml index d132f7d..f97bfed 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -6,6 +6,7 @@ members = [ "crates/storage", "crates/cold", "crates/transport", + "crates/dpu", "crates/bench", ] @@ -24,6 +25,7 @@ gpucache-protocol = { path = "crates/protocol" } gpucache-storage = { path = "crates/storage" } gpucache-cold = { path = "crates/cold" } gpucache-transport = { path = "crates/transport" } +gpucache-dpu = { path = "crates/dpu" } [workspace.lints.rust] unsafe_code = "forbid" diff --git a/README.md b/README.md index 3b806e8..2e63185 100644 --- a/README.md +++ b/README.md @@ -8,9 +8,9 @@ GPUCache does not do erasure coding and does not keep durability replicas. A los ## Status -**V1 milestones A, B, and C are implemented. The V2 first slice is in progress. DPU acceleration is not.** +**V1 milestones A, B, and C are implemented. The V2 first slice is in this tree. V3 is in progress: a CPU contract for a storage-side BlueField path, plus production hooks that do not need a DPU. DPU acceleration is not measured.** -V1 runs the cache contract on CPU. V2 adds a transport trait, a simulated registered-buffer pool, a shard-local directory, and a stricter G4 cold contract. The long-term deployment target is a storage-side BlueField-3, but this repository does not link DOCA, CUDA, RDMA, or SPDK, and it does not claim that any of those paths exist. Feature `rdma-nixl` does not link libnixl. An aarch64 build is a CPU build. It is not a BlueField functional test. GDS and BlueField offload are in development / `NOT_RUN_HARDWARE`. +V1 runs the cache contract on CPU. V2 adds a transport trait, a simulated registered-buffer pool, a shard-local directory, and a stricter G4 cold contract. V3 adds an optional `dpu-bf3` feature that does not link DOCA or SPDK, a simulated DRAM-slab storage queue, and CPU-side authz, replay, quota, and cold-ops hooks. The long-term deployment target is a storage-side BlueField-3. The GPU client keeps an ordinary ConnectX. This repository does not link DOCA, CUDA, RDMA, or SPDK, and it does not claim that any of those paths ran. Feature `rdma-nixl` does not link libnixl. Feature `dpu-bf3` does not link a device. An aarch64 build is a CPU build. It is not a BlueField functional test. GDS, DOCA, and BlueField offload are in development / `NOT_RUN_HARDWARE` / optional feature. They are not "already implemented" and not "deployed". | Milestone | Status | | --- | --- | @@ -25,10 +25,17 @@ V1 runs the cache contract on CPU. V2 adds a transport trait, a simulated regist | V2: G4 pack/manifest, async sink, cost-bounded refill | Implemented against the mock. Live RustFS is `NOT_RUN_HARDWARE` | | V2: P/D, G3.5, and G4 hop list | Documented in [docs/V2_DATA_PATHS.md](docs/V2_DATA_PATHS.md). Not a zero-copy claim | | V2: paired TTFT/TPOT against native NIXL | `NOT_RUN_HARDWARE` | -| V3: BlueField-3 data-path offload and production hardening | Not started; not purchased and not measured. Not an erasure-coding milestone | +| V3: `dpu-bf3` feature slot, simulated DRAM slab + storage queue, SPDK affinity check | In progress. Devices are not linked. `open_doca` / `open_nvme_device` / `open_spdk` refuse. `dpu-demo` prints `NOT_RUN_HARDWARE` | +| V3: one candidate offload (storage queue) and ablation stubs | Interface only. P2P, DPA, compression, and crypto offload stay `NOT_RUN_HARDWARE` when enabled. Hot-tier EC cannot be enabled | +| V3: authz, replay window, integrity seal, rate limit, secret rotation, audit | CPU hooks in this tree. Not mTLS. Trusted-fabric boundary is [docs/adr/0007-trusted-fabric.md](docs/adr/0007-trusted-fabric.md) | +| V3: rebalance, rolling upgrade, fault-domain isolation | In-process stubs on the V2 shard directory. They fence fills. They do not copy a replica | +| V3: degrade matrix (DPU restart, GPU reset, NIC flap, storage loss, NVMe timeout, control-plane loss, RustFS slow/down) | Decisions are clean misses. No payload, no wrong hit, no permanent block. Not a hardware fault injection | +| V3: G4 retention, manifest version, reclaim, delete verification, tenant cold keys, backfill budget | Implemented against the mock. Live RustFS is `NOT_RUN_HARDWARE` | +| V3: paired NIC-RDMA baseline vs DPU | `NOT_RUN_HARDWARE`. `tco=TCO_UNKNOWN`. `go_gate=NOT_EVALUATED`. TCP is not the DPU baseline | +| V3: second engine (SGLang / HiCache) | Not added. Evaluation note only. See [docs/adr/0008-second-engine.md](docs/adr/0008-second-engine.md) | | Erasure coding and durability replicas | Will not be built. A lost or failed hot block is a clean miss, then recompute and repopulate. Cold durability is RustFS G4 | -The integration route (native vLLM KV connector or LMCache) is **not chosen**. See [docs/adr/0001-integration-route.md](docs/adr/0001-integration-route.md). Hot-tier failure policy is [docs/adr/0002-no-erasure-coding.md](docs/adr/0002-no-erasure-coding.md). +The integration route (native vLLM KV connector or LMCache) is **not chosen**. See [docs/adr/0001-integration-route.md](docs/adr/0001-integration-route.md). Hot-tier failure policy is [docs/adr/0002-no-erasure-coding.md](docs/adr/0002-no-erasure-coding.md). The V3 offload candidate is [docs/adr/0006-storage-queue-offload.md](docs/adr/0006-storage-queue-offload.md). It is a hypothesis pending a paired measurement, not a measured bottleneck. ## What V1 A does @@ -108,6 +115,26 @@ cargo run -p gpucache-cold --bin cold-demo Success is `status=COLD_CONTRACT_OK` and `gds=NOT_RUN_HARDWARE`. `rustfs_live=NOT_RUN_HARDWARE` when no endpoint is configured. +In-process DPU contract. This does not open a device: + +```bash +cargo run -p gpucache-dpu --bin dpu-demo +``` + +Success is `status=DPU_CONTRACT_OK`, `offload_active=false`, and `hardware=NOT_RUN_HARDWARE`. `doca=NOT_LINKED`. Arm process start is not an offload. + +Optional feature, still with no SDK: + +```bash +cargo test -p gpucache-dpu --all-targets --features dpu-bf3 +``` + +V3 paired gate. No timings, no TCO, TCP is not the baseline: + +```bash +cargo run -p gpucache-bench --bin v3-paired +``` + `kv-block-xfer import ` reads one export frame on stdin and imports it into a fresh `MemoryStore`. It does not talk to an engine. ## Layout @@ -117,13 +144,14 @@ crates/core-types identity, keys, lifecycle, errors crates/protocol batch messages, replies, version negotiation crates/storage MemoryStore, FileStore, kv-block-xfer crates/transport TCP server, client, transport trait, shard directory, share-demo, fault-demo, shard-demo -crates/cold RustFS contract, pack/manifest, async sink, refill, cold-demo -crates/bench paired-baseline stub (no timings) +crates/cold RustFS contract, pack/manifest, async sink, refill, retention, cold-demo +crates/dpu BlueField feature slot, simulated storage queue, dpu-demo. Not linked by transport +crates/bench paired-baseline and v3-paired stubs (no timings) docs/adr decision records COMPATIBILITY.md environment manifest ``` -Design notes: [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md), [docs/PROTOCOL.md](docs/PROTOCOL.md), [docs/SECURITY.md](docs/SECURITY.md), [docs/OPERATIONS.md](docs/OPERATIONS.md), [docs/BENCHMARKS.md](docs/BENCHMARKS.md), [docs/CORRECTNESS.md](docs/CORRECTNESS.md), [docs/RISKS.md](docs/RISKS.md), [docs/V1C_TEST_MATRIX.md](docs/V1C_TEST_MATRIX.md), [docs/V2_DATA_PATHS.md](docs/V2_DATA_PATHS.md), [docs/V2_TEST_MATRIX.md](docs/V2_TEST_MATRIX.md). +Design notes: [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md), [docs/PROTOCOL.md](docs/PROTOCOL.md), [docs/SECURITY.md](docs/SECURITY.md), [docs/OPERATIONS.md](docs/OPERATIONS.md), [docs/BENCHMARKS.md](docs/BENCHMARKS.md), [docs/CORRECTNESS.md](docs/CORRECTNESS.md), [docs/RISKS.md](docs/RISKS.md), [docs/V1C_TEST_MATRIX.md](docs/V1C_TEST_MATRIX.md), [docs/V2_DATA_PATHS.md](docs/V2_DATA_PATHS.md), [docs/V2_TEST_MATRIX.md](docs/V2_TEST_MATRIX.md), [docs/V3_DPU_TARGETS.md](docs/V3_DPU_TARGETS.md), [docs/V3_TEST_MATRIX.md](docs/V3_TEST_MATRIX.md), [docs/V3_FAILURES.md](docs/V3_FAILURES.md). ## License diff --git a/THIRD_PARTY_NOTICES.md b/THIRD_PARTY_NOTICES.md index aa820ba..cc6c7e9 100644 --- a/THIRD_PARTY_NOTICES.md +++ b/THIRD_PARTY_NOTICES.md @@ -1,6 +1,16 @@ # Third-party notices -GPUCache is Apache-2.0. See [LICENSE](LICENSE). The V2 first slice adds no direct dependency. NIXL (`nixl-sys`, Apache-2.0) and Mooncake (Apache-2.0) were audited and not linked. See [docs/adr/0004-nixl-transport.md](docs/adr/0004-nixl-transport.md). +GPUCache is Apache-2.0. See [LICENSE](LICENSE). The V2 first slice and the V3 CPU slice add no direct dependency. NIXL (`nixl-sys`, Apache-2.0) and Mooncake (Apache-2.0) were audited and not linked. See [docs/adr/0004-nixl-transport.md](docs/adr/0004-nixl-transport.md). + +## DOCA, SPDK, and GDS (not linked) + +This tree does not vendor or link DOCA, SPDK, or GDS. Feature `dpu-bf3` has an empty dependency list. No DOCA binary ships with this source tree. + +DOCA is NVIDIA software. Its license is NVIDIA's, not Apache-2.0. This change did **not** review the DOCA 3.5 EULA or any performance-disclosure clause, and it does not state a legal conclusion. Before anyone links DOCA or publishes a number from a DOCA run, a human has to read the exact license for the version they installed. Do not assume every NVIDIA hardware result is unpublishable, and do not publish a restricted result. That review is `NOT_RUN_LICENSE` here. + +SPDK is a candidate for a future reactor-backed NVMe queue. It is not a dependency. This notice does not pin an SPDX identifier for a version we did not vendor. + +No model weights are included. Engine licenses stay out of scope until ADR 0001 selects a route. Direct dependencies: diff --git a/crates/bench/Cargo.toml b/crates/bench/Cargo.toml index 02fa5e8..2c7fd92 100644 --- a/crates/bench/Cargo.toml +++ b/crates/bench/Cargo.toml @@ -12,5 +12,9 @@ publish = false name = "paired-baseline" path = "src/bin/paired_baseline.rs" +[[bin]] +name = "v3-paired" +path = "src/bin/v3_paired.rs" + [lints] workspace = true diff --git a/crates/bench/src/bin/v3_paired.rs b/crates/bench/src/bin/v3_paired.rs new file mode 100644 index 0000000..0a79a28 --- /dev/null +++ b/crates/bench/src/bin/v3_paired.rs @@ -0,0 +1,16 @@ +// Copyright 2024-2026 RustFS, Inc. +// SPDX-License-Identifier: Apache-2.0 +//! V3 paired-experiment placeholder. +//! +//! The required baseline is a normal NIC RDMA path, not TCP. This process +//! prints `NOT_RUN_HARDWARE`, `TCO_UNKNOWN`, and `go_gate=NOT_EVALUATED`. +//! It does not measure TTFT, watts, or token goodput. + +use std::process::ExitCode; + +use gpucache_bench::v3_paired_report; + +fn main() -> ExitCode { + print!("{}", v3_paired_report()); + ExitCode::SUCCESS +} diff --git a/crates/bench/src/lib.rs b/crates/bench/src/lib.rs index 7d623d7..730a5ee 100644 --- a/crates/bench/src/lib.rs +++ b/crates/bench/src/lib.rs @@ -39,9 +39,52 @@ pub fn paired_baseline_report() -> String { out } +/// V3 paired gate. The required baseline is NIC RDMA, not a weak TCP path. +/// Every measured row stays `NOT_RUN_HARDWARE`. Cost stays `TCO_UNKNOWN`. +/// The report has no digits, so it cannot publish a TTFT, a watt, or a token rate. +pub fn v3_paired_report() -> String { + let lines = [ + "status=NOT_RUN_HARDWARE", + "comparison=NIC_RDMA_BASELINE_VS_DPU", + "nic_rdma_baseline=NOT_RUN_HARDWARE", + "dpu_storage_queue=NOT_RUN_HARDWARE", + "tcp_used_as_dpu_baseline=FORBIDDEN", + "same_gpu=REQUIRED_NOT_RUN", + "same_model=REQUIRED_NOT_RUN", + "same_trace=REQUIRED_NOT_RUN", + "same_slo=REQUIRED_NOT_RUN", + "same_capacity=REQUIRED_NOT_RUN", + "same_auth=REQUIRED_NOT_RUN", + "ttft=NOT_RUN_HARDWARE", + "tpot=NOT_RUN_HARDWARE", + "itl=NOT_RUN_HARDWARE", + "goodput=NOT_RUN_HARDWARE", + "host_core_seconds=NOT_RUN_HARDWARE", + "dpu_cpu=NOT_RUN_HARDWARE", + "dpu_dram=NOT_RUN_HARDWARE", + "gpu_interference=NOT_RUN_HARDWARE", + "wire_bytes=NOT_RUN_HARDWARE", + "ssd_write_amp=NOT_RUN_HARDWARE", + "wall_power=NOT_RUN_HARDWARE", + "tco=TCO_UNKNOWN", + "go_gate=NOT_EVALUATED", + "no_go_keeps_portable_backend=YES", + "soak=NOT_RUN_HARDWARE", + "pb_scale=NOT_CLAIMED", + "production_ready=NOT_CLAIMED", + "soft_roce=NOT_A_PERFORMANCE_RESULT", + ]; + let mut out = String::new(); + for line in lines { + out.push_str(line); + out.push('\n'); + } + out +} + #[cfg(test)] mod tests { - use super::paired_baseline_report; + use super::{paired_baseline_report, v3_paired_report}; #[test] fn report_has_no_invented_numbers() { @@ -71,4 +114,25 @@ mod tests { assert!(!report.contains("tokens/s")); assert!(!report.contains("PASS")); } + + #[test] + fn v3_report_requires_nic_rdma_and_invents_nothing() { + let report = v3_paired_report(); + assert!(report.chars().all(|c| !c.is_ascii_digit())); + for key in [ + "comparison=NIC_RDMA_BASELINE_VS_DPU", + "nic_rdma_baseline=NOT_RUN_HARDWARE", + "dpu_storage_queue=NOT_RUN_HARDWARE", + "tcp_used_as_dpu_baseline=FORBIDDEN", + "tco=TCO_UNKNOWN", + "go_gate=NOT_EVALUATED", + "pb_scale=NOT_CLAIMED", + "production_ready=NOT_CLAIMED", + "soft_roce=NOT_A_PERFORMANCE_RESULT", + ] { + assert!(report.contains(key), "missing {key}"); + } + assert!(!report.contains("tokens/s")); + assert!(!report.contains("PASS")); + } } diff --git a/crates/cold/src/lib.rs b/crates/cold/src/lib.rs index 7548fc1..3fcf8d1 100644 --- a/crates/cold/src/lib.rs +++ b/crates/cold/src/lib.rs @@ -12,6 +12,7 @@ mod demo; mod http; mod mock; +mod ops; mod pack; mod sigv4; mod sink; @@ -19,8 +20,12 @@ mod sink; pub use demo::{run_cold_demo, ColdReport}; pub use http::{live_probe, ProbeReport, ProbeStatus}; pub use mock::MockObjectStore; +pub use ops::{ + enqueue_backfill, refill_degraded, verify_deleted, BackfillLimit, ColdAvailability, ColdKeys, + ColdOps, ObjectRecord, ReclaimStep, Retention, +}; pub use pack::{ - contract_smoke, pack_blocks, read_manifest, Manifest, ObjectMeta, ObjectStore, PackedBlock, - PayloadRange, + contract_smoke, manifest_version, pack_blocks, read_manifest, Manifest, ObjectMeta, + ObjectStore, PackedBlock, PayloadRange, MANIFEST_VERSION, }; pub use sink::{refill_generation, AsyncSink, ColdJob, RefillBudget, RefillOutcome}; diff --git a/crates/cold/src/ops.rs b/crates/cold/src/ops.rs new file mode 100644 index 0000000..d5d6548 --- /dev/null +++ b/crates/cold/src/ops.rs @@ -0,0 +1,479 @@ +// Copyright 2024-2026 RustFS, Inc. +// SPDX-License-Identifier: Apache-2.0 + +//! G4 retention, manifest checks, async reclaim, tenant key generations, +//! and a backfill byte budget. +//! +//! Live RustFS stays outside this module. A slow or unavailable cold tier +//! returns [`RefillOutcome::Missing`] without reading the store, so the +//! caller recomputes and is not blocked. Expired objects are deleted only +//! after the following head shows them gone. + +use std::collections::{BTreeMap, VecDeque}; + +use gpucache_core_types::{CacheError, Checksum, QuotaKind}; + +use crate::pack::{key_ok, ObjectStore, MANIFEST_VERSION}; +use crate::sink::{refill_generation, AsyncSink, ColdJob, RefillBudget, RefillOutcome}; + +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub struct Retention { + max_age_ms: u64, +} + +impl Retention { + pub fn new(max_age_ms: u64) -> Result { + if max_age_ms == 0 { + return Err(CacheError::Malformed { + reason: "retention", + }); + } + Ok(Self { max_age_ms }) + } + + pub fn expired(&self, written_ms: u64, now_ms: u64) -> bool { + now_ms.saturating_sub(written_ms) >= self.max_age_ms + } +} + +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub enum ColdAvailability { + Ready, + Slow, + Unavailable, +} + +pub fn refill_degraded( + store: &dyn ObjectStore, + manifest_key: &str, + generation: u64, + budget: &RefillBudget, + availability: ColdAvailability, +) -> Result { + match availability { + ColdAvailability::Slow | ColdAvailability::Unavailable => Ok(RefillOutcome::Missing), + ColdAvailability::Ready => refill_generation(store, manifest_key, generation, budget), + } +} + +pub fn verify_deleted(store: &dyn ObjectStore, key: &str) -> Result<(), CacheError> { + if !key_ok(key) { + return Err(CacheError::Malformed { reason: "cold key" }); + } + match store.head(key) { + Err(CacheError::Storage { + reason: "object missing", + }) => Ok(()), + Ok(_) => Err(CacheError::Storage { + reason: "delete unverified", + }), + Err(err) => Err(err), + } +} + +#[derive(Clone, Debug, PartialEq, Eq)] +pub struct ObjectRecord { + pub key: String, + pub written_ms: u64, + pub checksum: Checksum, + pub tenant: Vec, + pub key_generation: u64, + pub manifest_version: u16, +} + +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub enum ReclaimStep { + Idle, + Verified, +} + +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +enum KeyState { + Current, + Overlap, + Retired, +} + +struct ColdKey { + tenant: Vec, + generation: u64, + state: KeyState, +} + +pub struct ColdKeys { + items: Vec, + next: BTreeMap, u64>, +} + +impl ColdKeys { + pub fn new() -> Self { + Self { + items: Vec::new(), + next: BTreeMap::new(), + } + } + + pub fn issue(&mut self, tenant: &[u8]) -> Result { + self.ensure_tenant(tenant)?; + if self.next.get(tenant).copied().unwrap_or(0) != 0 { + return Err(CacheError::IdempotencyConflict); + } + Ok(self.insert(tenant, KeyState::Current)) + } + + pub fn rotate(&mut self, tenant: &[u8]) -> Result { + self.ensure_tenant(tenant)?; + if !self + .items + .iter() + .any(|item| item.tenant.as_slice() == tenant && item.state == KeyState::Current) + { + return Err(CacheError::UnknownGeneration); + } + for item in &mut self.items { + if item.tenant.as_slice() == tenant && item.state == KeyState::Current { + item.state = KeyState::Overlap; + } + } + Ok(self.insert(tenant, KeyState::Current)) + } + + pub fn retire(&mut self, tenant: &[u8], generation: u64) -> Result<(), CacheError> { + let Some(item) = self + .items + .iter_mut() + .find(|item| item.tenant.as_slice() == tenant && item.generation == generation) + else { + return Err(CacheError::UnknownGeneration); + }; + if item.state != KeyState::Overlap { + return Err(CacheError::FencingRejected { reason: "secret" }); + } + item.state = KeyState::Retired; + Ok(()) + } + + pub fn allow_write(&self, tenant: &[u8], generation: u64) -> Result<(), CacheError> { + self.allow(tenant, generation, true) + } + + pub fn allow_read(&self, tenant: &[u8], generation: u64) -> Result<(), CacheError> { + self.allow(tenant, generation, false) + } + + fn allow(&self, tenant: &[u8], generation: u64, write: bool) -> Result<(), CacheError> { + let Some(item) = self + .items + .iter() + .find(|item| item.tenant.as_slice() == tenant && item.generation == generation) + else { + return Err(CacheError::Unauthorized); + }; + let ok = match item.state { + KeyState::Current => true, + KeyState::Overlap => !write, + KeyState::Retired => false, + }; + if ok { + Ok(()) + } else { + Err(CacheError::Unauthorized) + } + } + + fn ensure_tenant(&self, tenant: &[u8]) -> Result<(), CacheError> { + if tenant.is_empty() || tenant.len() > 128 { + return Err(CacheError::Malformed { reason: "cold key" }); + } + Ok(()) + } + + fn insert(&mut self, tenant: &[u8], state: KeyState) -> u64 { + let generation = self + .next + .get(tenant) + .copied() + .unwrap_or(0) + .saturating_add(1); + self.next.insert(tenant.to_vec(), generation); + self.items.push(ColdKey { + tenant: tenant.to_vec(), + generation, + state, + }); + generation + } +} + +impl Default for ColdKeys { + fn default() -> Self { + Self::new() + } +} + +pub struct ColdOps { + retention: Retention, + records: BTreeMap, + reclaim: VecDeque, + keys: ColdKeys, +} + +impl ColdOps { + pub fn new(retention: Retention) -> Self { + Self { + retention, + records: BTreeMap::new(), + reclaim: VecDeque::new(), + keys: ColdKeys::new(), + } + } + + pub fn keys(&self) -> &ColdKeys { + &self.keys + } + + pub fn keys_mut(&mut self) -> &mut ColdKeys { + &mut self.keys + } + + pub fn record_put(&mut self, record: ObjectRecord) -> Result<(), CacheError> { + if !key_ok(&record.key) || record.tenant.is_empty() { + return Err(CacheError::Malformed { reason: "cold key" }); + } + if record.manifest_version != MANIFEST_VERSION { + return Err(CacheError::Storage { + reason: "store corrupt", + }); + } + self.keys + .allow_write(&record.tenant, record.key_generation)?; + self.records.insert(record.key.clone(), record); + Ok(()) + } + + pub fn allow_read(&self, tenant: &[u8], generation: u64) -> Result<(), CacheError> { + self.keys.allow_read(tenant, generation) + } + + pub fn schedule_expired(&mut self, now_ms: u64) -> usize { + let mut added = 0usize; + let due: Vec = self + .records + .iter() + .filter(|(_, record)| self.retention.expired(record.written_ms, now_ms)) + .map(|(key, _)| key.clone()) + .filter(|key| !self.reclaim.contains(key)) + .collect(); + for key in due { + self.reclaim.push_back(key); + added = added.saturating_add(1); + } + added + } + + pub fn reclaim_one(&mut self, store: &mut dyn ObjectStore) -> Result { + let Some(key) = self.reclaim.pop_front() else { + return Ok(ReclaimStep::Idle); + }; + let Some(record) = self.records.get(&key).cloned() else { + return Ok(ReclaimStep::Idle); + }; + match store.head(&key) { + Err(CacheError::Storage { + reason: "object missing", + }) => { + self.records.remove(&key); + return Ok(ReclaimStep::Verified); + } + Err(err) => return Err(err), + Ok(meta) if meta.checksum != record.checksum => { + store.delete(&key)?; + verify_deleted(store, &key)?; + self.records.remove(&key); + return Err(CacheError::ChecksumMismatch); + } + Ok(_) => {} + } + store.delete(&key)?; + if let Err(err) = verify_deleted(store, &key) { + self.reclaim.push_front(key); + return Err(err); + } + self.records.remove(&key); + Ok(ReclaimStep::Verified) + } +} + +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub struct BackfillLimit { + max_bytes: u64, + used: u64, +} + +impl BackfillLimit { + pub fn new(max_bytes: u64) -> Result { + if max_bytes == 0 { + return Err(CacheError::Malformed { reason: "backfill" }); + } + Ok(Self { max_bytes, used: 0 }) + } + + pub fn remaining(&self) -> u64 { + self.max_bytes.saturating_sub(self.used) + } + + pub fn used(&self) -> u64 { + self.used + } +} + +pub fn enqueue_backfill( + sink: &mut AsyncSink, + job: ColdJob, + limit: &mut BackfillLimit, +) -> Result<(), CacheError> { + let weight = job.byte_weight(); + if weight == 0 { + return Err(CacheError::Malformed { reason: "backfill" }); + } + if weight > limit.remaining() { + return Err(CacheError::QuotaExceeded { + kind: QuotaKind::InFlightBytes, + }); + } + sink.enqueue(job)?; + limit.used = limit.used.saturating_add(weight); + Ok(()) +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::pack::{pack_blocks, PackedBlock}; + use crate::MockObjectStore; + use gpucache_core_types::{content_checksum, synthetic_identity}; + + struct StickyStore { + inner: MockObjectStore, + } + + impl ObjectStore for StickyStore { + fn put(&mut self, key: &str, body: &[u8]) -> Result<(), CacheError> { + self.inner.put(key, body) + } + fn head(&self, key: &str) -> Result { + self.inner.head(key) + } + fn range_get(&self, key: &str, offset: u64, length: u64) -> Result, CacheError> { + self.inner.range_get(key, offset, length) + } + fn delete(&mut self, _key: &str) -> Result<(), CacheError> { + Ok(()) + } + } + + fn record(key: &str, generation: u64, written_ms: u64) -> ObjectRecord { + ObjectRecord { + key: key.to_string(), + written_ms, + checksum: content_checksum(b"body"), + tenant: b"tenant-a".to_vec(), + key_generation: generation, + manifest_version: MANIFEST_VERSION, + } + } + + #[test] + fn retention_reclaim_verifies_delete_and_a_lie_is_not_success() { + let retention = Retention::new(100).unwrap(); + assert!(!retention.expired(0, 99)); + assert!(retention.expired(0, 100)); + assert!(!retention.expired(50, 40)); + let mut ops = ColdOps::new(retention); + let generation = ops.keys_mut().issue(b"tenant-a").unwrap(); + let mut store = MockObjectStore::default(); + store.put("ns/object", b"body").unwrap(); + ops.record_put(record("ns/object", generation, 0)).unwrap(); + assert_eq!(ops.schedule_expired(50), 0); + assert_eq!(ops.reclaim_one(&mut store).unwrap(), ReclaimStep::Idle); + assert_eq!(ops.schedule_expired(100), 1); + assert_eq!(ops.reclaim_one(&mut store).unwrap(), ReclaimStep::Verified); + assert!(store.head("ns/object").is_err()); + + let mut sticky = StickyStore { + inner: MockObjectStore::default(), + }; + sticky.put("ns/stuck", b"body").unwrap(); + ops.record_put(record("ns/stuck", generation, 0)).unwrap(); + ops.schedule_expired(200); + assert_eq!( + ops.reclaim_one(&mut sticky).unwrap_err(), + CacheError::Storage { + reason: "delete unverified" + } + ); + assert!(sticky.head("ns/stuck").is_ok()); + } + + #[test] + fn retired_cold_key_cannot_read_or_write() { + let mut keys = ColdKeys::new(); + let first = keys.issue(b"tenant-a").unwrap(); + let second = keys.rotate(b"tenant-a").unwrap(); + keys.allow_read(b"tenant-a", first).unwrap(); + assert_eq!( + keys.allow_write(b"tenant-a", first).unwrap_err(), + CacheError::Unauthorized + ); + keys.retire(b"tenant-a", first).unwrap(); + assert_eq!( + keys.allow_read(b"tenant-a", first).unwrap_err(), + CacheError::Unauthorized + ); + keys.allow_write(b"tenant-a", second).unwrap(); + let mut ops = ColdOps::new(Retention::new(10).unwrap()); + let generation = ops.keys_mut().issue(b"tenant-b").unwrap(); + let mut bad = record("ns/object", generation, 0); + bad.manifest_version = 2; + assert!(ops.record_put(bad).is_err()); + } + + #[test] + fn backfill_stops_at_the_byte_budget() { + let payload = vec![1u8; 8]; + let mut identity = synthetic_identity(1, 1); + identity.prefix_chain[0].0[0] = 1; + let block = PackedBlock { + generation: 1, + identity, + checksum: content_checksum(&payload), + payload, + }; + let pack = pack_blocks("bf", &[block]).unwrap(); + let job = ColdJob::from_pack(&pack); + let weight = job.byte_weight(); + let mut sink = AsyncSink::new(4, weight.saturating_mul(4)).unwrap(); + let mut limit = BackfillLimit::new(weight).unwrap(); + enqueue_backfill(&mut sink, job.clone(), &mut limit).unwrap(); + assert_eq!( + enqueue_backfill(&mut sink, job, &mut limit).unwrap_err(), + CacheError::QuotaExceeded { + kind: QuotaKind::InFlightBytes, + } + ); + assert_eq!(sink.queued(), 1); + assert_eq!(limit.used(), weight); + } + + #[test] + fn slow_or_down_rustfs_is_a_miss_and_does_not_call_the_store() { + let mut store = MockObjectStore::default(); + store.put("ns/manifest", b"not-a-manifest").unwrap(); + let budget = RefillBudget::new(1024, 1).unwrap(); + for availability in [ColdAvailability::Slow, ColdAvailability::Unavailable] { + let outcome = refill_degraded(&store, "ns/manifest", 1, &budget, availability).unwrap(); + assert_eq!(outcome, RefillOutcome::Missing); + } + assert!(store.head("ns/manifest").is_ok()); + } +} diff --git a/crates/cold/src/pack.rs b/crates/cold/src/pack.rs index 884722c..ad3eb3b 100644 --- a/crates/cold/src/pack.rs +++ b/crates/cold/src/pack.rs @@ -8,7 +8,9 @@ use gpucache_core_types::{ const OBJECT_MAGIC: &[u8; 4] = b"GPKG"; const MANIFEST_MAGIC: &[u8; 4] = b"GPMF"; -const VERSION: u16 = 1; +const OBJECT_VERSION: u16 = 1; +/// Manifest layout this crate will read. Any other version is a miss. +pub const MANIFEST_VERSION: u16 = 1; #[derive(Clone, Debug, PartialEq, Eq)] pub struct ObjectMeta { @@ -77,7 +79,7 @@ pub fn pack_blocks(namespace: &str, blocks: &[PackedBlock]) -> Result Result Result, CacheError> { Ok(out) } +/// Read the manifest version. A version other than [`MANIFEST_VERSION`] is +/// not a hit; [`read_manifest`] rejects it. +pub fn manifest_version(bytes: &[u8]) -> Result { + let mut r = ByteReader::new(bytes); + let magic = r.raw(4)?; + if magic != MANIFEST_MAGIC { + return Err(CacheError::Storage { + reason: "store corrupt", + }); + } + r.u16() +} + pub fn read_manifest(bytes: &[u8]) -> Result { let mut r = ByteReader::new(bytes); let magic = r.raw(4)?; - if magic != MANIFEST_MAGIC || r.u16()? != VERSION { + if magic != MANIFEST_MAGIC || r.u16()? != MANIFEST_VERSION { return Err(CacheError::Storage { reason: "store corrupt", }); @@ -237,4 +252,24 @@ mod tests { let mut store = MockObjectStore::default(); contract_smoke(&mut store).unwrap(); } + + #[test] + fn unknown_manifest_version_is_not_a_hit() { + let blocks = sample_blocks().unwrap(); + let pack = pack_blocks("ver", &blocks).unwrap(); + assert_eq!( + manifest_version(&pack.manifest_bytes).unwrap(), + MANIFEST_VERSION + ); + let mut bad = pack.manifest_bytes.clone(); + bad[4] = 2; + bad[5] = 0; + assert_eq!(manifest_version(&bad).unwrap(), 2); + assert!(matches!( + read_manifest(&bad), + Err(CacheError::Storage { + reason: "store corrupt" + }) + )); + } } diff --git a/crates/cold/src/sink.rs b/crates/cold/src/sink.rs index d7dbe68..942a233 100644 --- a/crates/cold/src/sink.rs +++ b/crates/cold/src/sink.rs @@ -32,9 +32,13 @@ impl ColdJob { } } - fn weight(&self) -> u64 { + pub fn byte_weight(&self) -> u64 { self.object_bytes.len() as u64 + self.manifest_bytes.len() as u64 } + + fn weight(&self) -> u64 { + self.byte_weight() + } } #[derive(Clone, Copy, Debug, PartialEq, Eq)] diff --git a/crates/core-types/src/error.rs b/crates/core-types/src/error.rs index 6ac185d..45632cb 100644 --- a/crates/core-types/src/error.rs +++ b/crates/core-types/src/error.rs @@ -16,6 +16,8 @@ pub enum QuotaKind { BatchItems, IdempotencyTable, PrefixLinks, + /// Per-tenant admission rate. Not a durability quota. + RateLimit, } /// Recoverable contract failure. Payloads and prompts are never embedded. diff --git a/crates/dpu/Cargo.toml b/crates/dpu/Cargo.toml new file mode 100644 index 0000000..39b5e85 --- /dev/null +++ b/crates/dpu/Cargo.toml @@ -0,0 +1,24 @@ +[package] +name = "gpucache-dpu" +version.workspace = true +edition.workspace = true +license.workspace = true +rust-version.workspace = true +authors.workspace = true +description = "BlueField-3 feature slot and CPU-simulated storage-queue contract. Does not link DOCA, SPDK, or a DPU." +publish = false + +[features] +default = [] +# Slot only. Does not pull DOCA, SPDK, or a hardware SDK. See docs/adr/0006-storage-queue-offload.md. +dpu-bf3 = [] + +[dependencies] +gpucache-core-types.workspace = true + +[[bin]] +name = "dpu-demo" +path = "src/bin/dpu_demo.rs" + +[lints] +workspace = true diff --git a/crates/dpu/src/ablation.rs b/crates/dpu/src/ablation.rs new file mode 100644 index 0000000..40770b7 --- /dev/null +++ b/crates/dpu/src/ablation.rs @@ -0,0 +1,112 @@ +// Copyright 2024-2026 RustFS, Inc. +// SPDX-License-Identifier: Apache-2.0 + +//! On/off switches for offloads that this slice does not run. +//! +//! The selected candidate is the storage queue, not one of these knobs. +//! Enabling a knob does not execute it. Hot-tier erasure coding cannot be +//! enabled. See docs/adr/0002-no-erasure-coding.md. + +use gpucache_core_types::CacheError; + +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub enum Knob { + P2p, + Dpa, + Compression, + Crypto, + ErasureCoding, +} + +const KNOBS: [Knob; 5] = [ + Knob::P2p, + Knob::Dpa, + Knob::Compression, + Knob::Crypto, + Knob::ErasureCoding, +]; + +#[derive(Clone, Debug, PartialEq, Eq)] +pub struct Ablation { + enabled: [bool; 5], +} + +impl Default for Ablation { + fn default() -> Self { + Self::new() + } +} + +impl Ablation { + pub fn new() -> Self { + Self { + enabled: [false; 5], + } + } + + pub fn knobs() -> &'static [Knob] { + &KNOBS + } + + pub fn set(&mut self, knob: Knob, on: bool) -> Result<(), CacheError> { + if knob == Knob::ErasureCoding && on { + return Err(CacheError::Malformed { + reason: "hot tier ec", + }); + } + self.enabled[index(knob)] = on; + Ok(()) + } + + pub fn enabled(&self, knob: Knob) -> bool { + self.enabled[index(knob)] + } + + /// `OFF` when the knob is disabled. `NOT_RUN_HARDWARE` when it is enabled. + /// Neither arm changes payload bytes. + pub fn probe(&self, knob: Knob) -> &'static str { + if self.enabled(knob) { + "NOT_RUN_HARDWARE" + } else { + "OFF" + } + } +} + +fn index(knob: Knob) -> usize { + match knob { + Knob::P2p => 0, + Knob::Dpa => 1, + Knob::Compression => 2, + Knob::Crypto => 3, + Knob::ErasureCoding => 4, + } +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn defaults_are_off_and_ec_stays_off() { + let mut ablation = Ablation::new(); + for knob in Ablation::knobs() { + assert!(!ablation.enabled(*knob)); + assert_eq!(ablation.probe(*knob), "OFF"); + } + let err = ablation.set(Knob::ErasureCoding, true).unwrap_err(); + assert_eq!( + err, + CacheError::Malformed { + reason: "hot tier ec" + } + ); + assert!(!ablation.enabled(Knob::ErasureCoding)); + ablation.set(Knob::P2p, true).unwrap(); + ablation.set(Knob::Crypto, true).unwrap(); + assert_eq!(ablation.probe(Knob::P2p), "NOT_RUN_HARDWARE"); + assert_eq!(ablation.probe(Knob::Crypto), "NOT_RUN_HARDWARE"); + assert_eq!(ablation.probe(Knob::Dpa), "OFF"); + assert_eq!(ablation.probe(Knob::ErasureCoding), "OFF"); + } +} diff --git a/crates/dpu/src/bin/dpu_demo.rs b/crates/dpu/src/bin/dpu_demo.rs new file mode 100644 index 0000000..e2c76b4 --- /dev/null +++ b/crates/dpu/src/bin/dpu_demo.rs @@ -0,0 +1,24 @@ +// Copyright 2024-2026 RustFS, Inc. +// SPDX-License-Identifier: Apache-2.0 +//! In-process storage-queue contract. Hardware stays NOT_RUN_HARDWARE. +//! +//! `status=DPU_CONTRACT_OK` means the simulated submit/complete path matched +//! bytes. It is not a BlueField run, a DOCA run, or a performance result. + +use std::process::ExitCode; + +use gpucache_dpu::run_dpu_demo; + +fn main() -> ExitCode { + match run_dpu_demo() { + Ok(report) => { + print!("{report}"); + ExitCode::SUCCESS + } + Err(err) => { + eprintln!("status=DPU_FAILED"); + eprintln!("error={err}"); + ExitCode::from(2) + } + } +} diff --git a/crates/dpu/src/degrade.rs b/crates/dpu/src/degrade.rs new file mode 100644 index 0000000..859ed89 --- /dev/null +++ b/crates/dpu/src/degrade.rs @@ -0,0 +1,68 @@ +// Copyright 2024-2026 RustFS, Inc. +// SPDX-License-Identifier: Apache-2.0 + +//! Failure outcomes for the V3 degrade matrix. +//! +//! Every listed fault is a clean miss. The engine recomputes. The decision +//! carries no payload, does not report a wrong hit, and does not stick in a +//! permanent block. Device-side effects live on `StorageQueue::apply_fault`. + +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub enum Fault { + DpuRestart, + GpuReset, + NicFlap, + StorageLoss, + NvmeTimeout, + ControlPlaneLoss, + RustFsSlow, + RustFsUnavailable, +} + +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub struct DegradeDecision { + pub fault: Fault, + pub outcome: &'static str, + pub recompute: bool, + pub wrong_hit: bool, + pub blocked: bool, +} + +pub fn decide(fault: Fault) -> DegradeDecision { + DegradeDecision { + fault, + outcome: "CLEAN_MISS", + recompute: true, + wrong_hit: false, + blocked: false, + } +} + +pub fn all_faults() -> &'static [Fault] { + &[ + Fault::DpuRestart, + Fault::GpuReset, + Fault::NicFlap, + Fault::StorageLoss, + Fault::NvmeTimeout, + Fault::ControlPlaneLoss, + Fault::RustFsSlow, + Fault::RustFsUnavailable, + ] +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn every_fault_is_a_clean_miss() { + for fault in all_faults() { + let decision = decide(*fault); + assert_eq!(decision.outcome, "CLEAN_MISS"); + assert!(decision.recompute); + assert!(!decision.wrong_hit); + assert!(!decision.blocked); + } + } +} diff --git a/crates/dpu/src/demo.rs b/crates/dpu/src/demo.rs new file mode 100644 index 0000000..b03c0fd --- /dev/null +++ b/crates/dpu/src/demo.rs @@ -0,0 +1,116 @@ +// Copyright 2024-2026 RustFS, Inc. +// SPDX-License-Identifier: Apache-2.0 + +//! CPU contract for the storage-queue interface. +//! +//! Success is `status=DPU_CONTRACT_OK`. That line means the in-process +//! simulation matched. It does not mean a BlueField, DOCA, or NVMe device ran. + +use core::fmt; + +use gpucache_core_types::CacheError; + +use crate::ablation::{Ablation, Knob}; +use crate::queue::{ + dpu_feature_selected, open_doca, open_nvme_device, open_spdk, ArmAgent, StorageQueue, +}; +use crate::target::planned_targets; + +#[derive(Clone, Debug, PartialEq, Eq)] +pub struct DpuReport { + pub feature: &'static str, +} + +impl fmt::Display for DpuReport { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + writeln!(f, "status=DPU_CONTRACT_OK")?; + writeln!(f, "arm_process=STARTED")?; + writeln!(f, "offload_active=false")?; + writeln!(f, "offload=STORAGE_QUEUE")?; + writeln!(f, "hardware=NOT_RUN_HARDWARE")?; + writeln!(f, "simulated_complete=INTERFACE_ONLY")?; + writeln!(f, "nvme=NOT_LINKED")?; + writeln!(f, "doca=NOT_LINKED")?; + writeln!(f, "spdk=NOT_LINKED")?; + writeln!(f, "gds=NOT_RUN_HARDWARE")?; + writeln!(f, "ec_hot_tier=OFF")?; + writeln!(f, "p2p=NOT_RUN_HARDWARE")?; + writeln!(f, "dpa=OFF")?; + writeln!(f, "compression=OFF")?; + writeln!(f, "crypto_offload=NOT_RUN_HARDWARE")?; + writeln!(f, "feature_dpu_bf3={}", self.feature)?; + writeln!(f, "topology=STORAGE_SIDE_BF3_TARGET")?; + writeln!(f, "client_nic=CONNECTX_ORDINARY_TARGET")?; + writeln!(f, "gpunetio_nic_mode=NOT_THIS_DAEMON")?; + for row in planned_targets() { + writeln!(f, "target_{}=NOT_RUN_HARDWARE", row.field)?; + } + Ok(()) + } +} + +pub fn run_dpu_demo() -> Result { + let agent = ArmAgent::start(); + if !agent.process_started() || agent.offload_active() { + return Err(CacheError::Storage { + reason: "dpu agent", + }); + } + let mut queue = StorageQueue::open_simulated(4096, 4)?; + let write = queue.submit_write(b"tenant-a", b"kv-demo")?; + let wrote = queue.complete(write)?; + if !wrote.simulated || wrote.offload_active || wrote.hardware != "NOT_RUN_HARDWARE" { + return Err(CacheError::Storage { + reason: "dpu agent", + }); + } + let slot = wrote.slot.ok_or(CacheError::Storage { + reason: "dpu agent", + })?; + let read = queue.submit_read(b"tenant-a", slot)?; + let got = queue.complete(read)?; + if got.payload.as_deref() != Some(b"kv-demo".as_slice()) || got.offload_active { + return Err(CacheError::ChecksumMismatch); + } + if open_nvme_device().is_ok() || open_doca().is_ok() || open_spdk().is_ok() { + return Err(CacheError::Storage { + reason: "doca not linked", + }); + } + let mut ablation = Ablation::new(); + ablation.set(Knob::P2p, true)?; + ablation.set(Knob::Crypto, true)?; + if ablation.probe(Knob::P2p) != "NOT_RUN_HARDWARE" + || ablation.probe(Knob::ErasureCoding) != "OFF" + || ablation.set(Knob::ErasureCoding, true).is_ok() + { + return Err(CacheError::Malformed { + reason: "hot tier ec", + }); + } + let feature = if dpu_feature_selected() { + "SELECTED_NOT_LINKED" + } else { + "UNSELECTED" + }; + Ok(DpuReport { feature }) +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn demo_does_not_claim_a_device() { + let text = run_dpu_demo().unwrap().to_string(); + assert!(text.contains("status=DPU_CONTRACT_OK")); + assert!(text.contains("offload_active=false")); + assert!(text.contains("hardware=NOT_RUN_HARDWARE")); + assert!(text.contains("nvme=NOT_LINKED")); + assert!(text.contains("doca=NOT_LINKED")); + assert!(text.contains("ec_hot_tier=OFF")); + assert!(text.contains("gpunetio_nic_mode=NOT_THIS_DAEMON")); + assert!(!text.contains("production ready")); + assert!(!text.contains("PB-scale")); + } +} diff --git a/crates/dpu/src/lib.rs b/crates/dpu/src/lib.rs new file mode 100644 index 0000000..6acc48a --- /dev/null +++ b/crates/dpu/src/lib.rs @@ -0,0 +1,27 @@ +// Copyright 2024-2026 RustFS, Inc. +// SPDX-License-Identifier: Apache-2.0 +//! Storage-side BlueField-3 slot for GPUCache. +//! +//! The default build and `--features dpu-bf3` both refuse to open DOCA, SPDK, +//! or an NVMe device. `dpu-demo` prints `NOT_RUN_HARDWARE`. An aarch64 build +//! of this crate is a CPU build, not a BlueField test. Starting the Arm +//! process flag is not an offload. + +#![forbid(unsafe_code)] + +mod ablation; +mod degrade; +mod demo; +mod queue; +mod reactor; +mod target; + +pub use ablation::{Ablation, Knob}; +pub use degrade::{all_faults, decide, DegradeDecision, Fault}; +pub use demo::{run_dpu_demo, DpuReport}; +pub use queue::{ + dpu_feature_selected, dpu_probe, open_doca, open_nvme_device, open_spdk, ArmAgent, Completion, + StorageQueue, TaskId, +}; +pub use reactor::{AffineQueue, ReactorId}; +pub use target::{planned_targets, PlannedTarget}; diff --git a/crates/dpu/src/queue.rs b/crates/dpu/src/queue.rs new file mode 100644 index 0000000..a363869 --- /dev/null +++ b/crates/dpu/src/queue.rs @@ -0,0 +1,441 @@ +// Copyright 2024-2026 RustFS, Inc. +// SPDX-License-Identifier: Apache-2.0 + +//! Simulated DPU DRAM slab and a storage-queue submit/complete interface. +//! +//! `open_simulated` copies bytes inside this process and labels the completion +//! `NOT_RUN_HARDWARE`. `open_nvme_device`, `open_doca`, and `open_spdk` fail +//! in every build, including `--features dpu-bf3`. Arm process start does not +//! flip `offload_active`. + +use std::collections::BTreeMap; + +use gpucache_core_types::{ + content_checksum, verify_checksum, CacheError, Checksum, HARD_BYTES_CAP, +}; + +use crate::degrade::{decide, DegradeDecision, Fault}; + +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub struct TaskId(pub u64); + +#[derive(Clone, Debug, PartialEq, Eq)] +pub struct Completion { + pub task_id: u64, + pub hardware: &'static str, + pub simulated: bool, + pub offload_active: bool, + pub slot: Option, + pub payload: Option>, +} + +/// Arm agent flag. `start` means the process record exists. It does not submit +/// a storage task and it does not activate offload. +#[derive(Debug)] +pub struct ArmAgent { + started: bool, +} + +impl ArmAgent { + pub fn start() -> Self { + Self { started: true } + } + + pub fn process_started(&self) -> bool { + self.started + } + + pub fn offload_active(&self) -> bool { + false + } +} + +struct Slot { + tenant: Vec, + bytes: Vec, + checksum: Checksum, + exposed: bool, +} + +struct Slab { + cap: u64, + used: u64, + next: u64, + slots: BTreeMap, +} + +impl Slab { + fn new(cap: u64) -> Result { + if cap == 0 || cap > HARD_BYTES_CAP as u64 { + return Err(CacheError::Malformed { reason: "dpu slab" }); + } + Ok(Self { + cap, + used: 0, + next: 1, + slots: BTreeMap::new(), + }) + } + + fn insert(&mut self, tenant: &[u8], bytes: &[u8]) -> Result { + let len = bytes.len() as u64; + let next = self + .used + .checked_add(len) + .ok_or(CacheError::QuotaExceeded { + kind: gpucache_core_types::QuotaKind::RetainedBytes, + })?; + if next > self.cap || self.next == u64::MAX { + return Err(CacheError::QuotaExceeded { + kind: gpucache_core_types::QuotaKind::RetainedBytes, + }); + } + let id = self.next; + self.next += 1; + self.used = next; + self.slots.insert( + id, + Slot { + tenant: tenant.to_vec(), + bytes: bytes.to_vec(), + checksum: content_checksum(bytes), + exposed: true, + }, + ); + Ok(id) + } + + fn read(&mut self, tenant: &[u8], id: u64) -> Result, CacheError> { + let Some(slot) = self.slots.get_mut(&id) else { + return Err(CacheError::Storage { + reason: "clean miss", + }); + }; + if slot.tenant.as_slice() != tenant { + return Err(CacheError::Unauthorized); + } + if !slot.exposed { + return Err(CacheError::Storage { + reason: "clean miss", + }); + } + if verify_checksum(&slot.bytes, slot.checksum).is_err() { + slot.exposed = false; + return Err(CacheError::ChecksumMismatch); + } + Ok(slot.bytes.clone()) + } + + fn hide_all(&mut self) { + for slot in self.slots.values_mut() { + slot.exposed = false; + } + } + + fn restart(&mut self) { + self.slots.clear(); + self.used = 0; + } +} + +enum Pending { + Write { tenant: Vec, payload: Vec }, + Read { tenant: Vec, slot: u64 }, +} + +pub struct StorageQueue { + slab: Slab, + pending: BTreeMap, + next_task: u64, + max_inflight: usize, + control_up: bool, +} + +impl StorageQueue { + pub fn open_simulated(cap_bytes: u64, max_inflight: usize) -> Result { + if max_inflight == 0 { + return Err(CacheError::Malformed { reason: "dpu slab" }); + } + Ok(Self { + slab: Slab::new(cap_bytes)?, + pending: BTreeMap::new(), + next_task: 1, + max_inflight, + control_up: true, + }) + } + + pub fn control_up(&self) -> bool { + self.control_up + } + + pub fn inflight(&self) -> usize { + self.pending.len() + } + + pub fn submit_write(&mut self, tenant: &[u8], payload: &[u8]) -> Result { + self.admit(tenant, payload.len())?; + self.enqueue(Pending::Write { + tenant: tenant.to_vec(), + payload: payload.to_vec(), + }) + } + + pub fn submit_read(&mut self, tenant: &[u8], slot: u64) -> Result { + self.admit(tenant, 1)?; + self.enqueue(Pending::Read { + tenant: tenant.to_vec(), + slot, + }) + } + + pub fn complete(&mut self, task: TaskId) -> Result { + let Some(pending) = self.pending.remove(&task.0) else { + return Err(CacheError::UnknownGeneration); + }; + let (slot, payload) = match pending { + Pending::Write { tenant, payload } => { + let id = self.slab.insert(&tenant, &payload)?; + (Some(id), None) + } + Pending::Read { tenant, slot } => { + let bytes = self.slab.read(&tenant, slot)?; + (Some(slot), Some(bytes)) + } + }; + Ok(self.receipt(task, slot, payload)) + } + + pub fn fail(&mut self, task: TaskId) -> Result<(), CacheError> { + if self.pending.remove(&task.0).is_none() { + return Err(CacheError::UnknownGeneration); + } + Ok(()) + } + + pub fn apply_fault(&mut self, fault: Fault) -> DegradeDecision { + match fault { + Fault::DpuRestart => { + self.pending.clear(); + self.slab.restart(); + self.control_up = true; + } + Fault::GpuReset | Fault::NicFlap | Fault::StorageLoss => { + self.pending.clear(); + self.slab.hide_all(); + self.control_up = true; + } + Fault::NvmeTimeout => { + if let Some(id) = self.pending.keys().next().copied() { + self.pending.remove(&id); + } + } + Fault::ControlPlaneLoss => { + self.pending.clear(); + self.slab.hide_all(); + self.control_up = false; + } + Fault::RustFsSlow | Fault::RustFsUnavailable => {} + } + decide(fault) + } + + pub fn restore_control(&mut self) { + self.control_up = true; + } + + fn admit(&self, tenant: &[u8], len: usize) -> Result<(), CacheError> { + if !self.control_up { + return Err(CacheError::Cancelled); + } + if tenant.is_empty() || tenant.len() > 128 || len == 0 { + return Err(CacheError::Malformed { reason: "dpu task" }); + } + if len > HARD_BYTES_CAP || len as u64 > self.slab.cap { + return Err(CacheError::ObjectTooLarge { + limit: self.slab.cap, + actual: len as u64, + }); + } + if self.pending.len() >= self.max_inflight || self.next_task == u64::MAX { + return Err(CacheError::QuotaExceeded { + kind: gpucache_core_types::QuotaKind::InFlightRequests, + }); + } + Ok(()) + } + + fn enqueue(&mut self, pending: Pending) -> Result { + let id = self.next_task; + self.next_task += 1; + self.pending.insert(id, pending); + Ok(TaskId(id)) + } + + fn receipt(&self, task: TaskId, slot: Option, payload: Option>) -> Completion { + Completion { + task_id: task.0, + hardware: dpu_probe(), + simulated: true, + offload_active: false, + slot, + payload, + } + } +} + +pub fn dpu_feature_selected() -> bool { + cfg!(feature = "dpu-bf3") +} + +pub fn dpu_probe() -> &'static str { + "NOT_RUN_HARDWARE" +} + +pub fn open_nvme_device() -> Result<(), CacheError> { + let _selected = dpu_feature_selected(); + Err(CacheError::Storage { + reason: "nvme not linked", + }) +} + +pub fn open_doca() -> Result<(), CacheError> { + let _selected = dpu_feature_selected(); + Err(CacheError::Storage { + reason: "doca not linked", + }) +} + +pub fn open_spdk() -> Result<(), CacheError> { + let _selected = dpu_feature_selected(); + Err(CacheError::Storage { + reason: "spdk not linked", + }) +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::degrade::all_faults; + + #[test] + fn arm_start_is_not_offload_and_devices_stay_closed() { + let agent = ArmAgent::start(); + assert!(agent.process_started()); + assert!(!agent.offload_active()); + assert_eq!(dpu_probe(), "NOT_RUN_HARDWARE"); + for open in [open_nvme_device, open_doca, open_spdk] { + assert!(matches!(open(), Err(CacheError::Storage { .. }))); + } + if cfg!(feature = "dpu-bf3") { + assert!(dpu_feature_selected()); + } else { + assert!(!dpu_feature_selected()); + } + assert!(matches!( + open_nvme_device(), + Err(CacheError::Storage { + reason: "nvme not linked" + }) + )); + } + + #[test] + fn simulated_write_then_read_roundtrips_without_claiming_offload() { + let mut queue = StorageQueue::open_simulated(64, 4).unwrap(); + let write = queue.submit_write(b"tenant-a", b"kv-block").unwrap(); + assert_eq!(queue.inflight(), 1); + let wrote = queue.complete(write).unwrap(); + assert!(wrote.simulated); + assert!(!wrote.offload_active); + assert_eq!(wrote.hardware, "NOT_RUN_HARDWARE"); + assert!(wrote.payload.is_none()); + let slot = wrote.slot.unwrap(); + let read = queue.submit_read(b"tenant-a", slot).unwrap(); + let got = queue.complete(read).unwrap(); + assert_eq!(got.payload.as_deref(), Some(b"kv-block".as_slice())); + let foreign = queue.submit_read(b"tenant-b", slot).unwrap(); + assert_eq!( + queue.complete(foreign).unwrap_err(), + CacheError::Unauthorized + ); + } + + #[test] + fn failed_write_is_not_readable() { + let mut queue = StorageQueue::open_simulated(64, 2).unwrap(); + let write = queue.submit_write(b"tenant-a", b"secret-bytes").unwrap(); + queue.fail(write).unwrap(); + assert_eq!(queue.inflight(), 0); + let read = queue.submit_read(b"tenant-a", 1).unwrap(); + assert_eq!( + queue.complete(read).unwrap_err(), + CacheError::Storage { + reason: "clean miss" + } + ); + } + + #[test] + fn faults_miss_and_then_accept_new_work() { + let mut queue = StorageQueue::open_simulated(128, 4).unwrap(); + let write = queue.submit_write(b"tenant-a", b"block").unwrap(); + let wrote = queue.complete(write).unwrap(); + let slot = wrote.slot.unwrap(); + for fault in all_faults() { + if matches!(fault, Fault::RustFsSlow | Fault::RustFsUnavailable) { + let before = queue.inflight(); + let decision = queue.apply_fault(*fault); + assert!(!decision.blocked); + assert!(!decision.wrong_hit); + assert_eq!(decision.outcome, "CLEAN_MISS"); + assert_eq!(queue.inflight(), before); + continue; + } + let _ = queue.submit_write(b"tenant-a", b"inflight"); + let decision = queue.apply_fault(*fault); + assert_eq!(decision.outcome, "CLEAN_MISS"); + assert!(decision.recompute); + assert!(!decision.wrong_hit); + assert!(!decision.blocked); + assert_eq!(queue.inflight(), 0); + if *fault == Fault::ControlPlaneLoss { + assert!(!queue.control_up()); + assert_eq!( + queue.submit_write(b"tenant-a", b"next").unwrap_err(), + CacheError::Cancelled + ); + queue.restore_control(); + } + let read = queue.submit_read(b"tenant-a", slot).unwrap(); + assert!(queue.complete(read).is_err()); + let again = queue.submit_write(b"tenant-a", b"fresh").unwrap(); + let done = queue.complete(again).unwrap(); + assert!(done.payload.is_none()); + assert!(!done.offload_active); + } + } + + #[test] + fn checksum_flip_hides_the_slot() { + let mut queue = StorageQueue::open_simulated(64, 2).unwrap(); + let write = queue.submit_write(b"tenant-a", b"stable").unwrap(); + let slot = queue.complete(write).unwrap().slot.unwrap(); + { + let stored = queue.slab.slots.get_mut(&slot).unwrap(); + stored.bytes[0] ^= 0xff; + } + let read = queue.submit_read(b"tenant-a", slot).unwrap(); + assert_eq!( + queue.complete(read).unwrap_err(), + CacheError::ChecksumMismatch + ); + let again = queue.submit_read(b"tenant-a", slot).unwrap(); + assert_eq!( + queue.complete(again).unwrap_err(), + CacheError::Storage { + reason: "clean miss" + } + ); + } +} diff --git a/crates/dpu/src/reactor.rs b/crates/dpu/src/reactor.rs new file mode 100644 index 0000000..3ae0240 --- /dev/null +++ b/crates/dpu/src/reactor.rs @@ -0,0 +1,56 @@ +// Copyright 2024-2026 RustFS, Inc. +// SPDX-License-Identifier: Apache-2.0 + +//! Thread-affinity contract for a future SPDK reactor. +//! +//! SPDK queue pairs and other reactor-affine objects must be created on the +//! reactor thread and must stay there. They are not Tokio tasks and must not +//! move across Tokio workers. This module does not start SPDK, Tokio, or a +//! thread. `submit_on` only checks that the caller names the bound reactor. + +use gpucache_core_types::CacheError; + +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub struct ReactorId(pub u32); + +pub struct AffineQueue { + bound: ReactorId, +} + +impl AffineQueue { + pub fn bind(bound: ReactorId) -> Self { + Self { bound } + } + + pub fn bound(&self) -> ReactorId { + self.bound + } + + pub fn submit_on(&self, current: ReactorId) -> Result<(), CacheError> { + if current != self.bound { + return Err(CacheError::Storage { + reason: "reactor affine", + }); + } + Ok(()) + } +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn queue_rejects_a_different_reactor() { + let queue = AffineQueue::bind(ReactorId(3)); + queue.submit_on(ReactorId(3)).unwrap(); + let err = queue.submit_on(ReactorId(4)).unwrap_err(); + assert_eq!( + err, + CacheError::Storage { + reason: "reactor affine" + } + ); + assert_eq!(queue.bound(), ReactorId(3)); + } +} diff --git a/crates/dpu/src/target.rs b/crates/dpu/src/target.rs new file mode 100644 index 0000000..b5f6090 --- /dev/null +++ b/crates/dpu/src/target.rs @@ -0,0 +1,83 @@ +// Copyright 2024-2026 RustFS, Inc. +// SPDX-License-Identifier: Apache-2.0 + +//! Planned BlueField-3 install targets. None of these versions were measured +//! or flashed by this tree. Host GPUNetIO NIC-mode samples are not this daemon. + +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub struct PlannedTarget { + pub field: &'static str, + pub value: &'static str, + pub measured: &'static str, +} + +pub fn planned_targets() -> &'static [PlannedTarget] { + const NOT_RUN: &str = "NOT_RUN_HARDWARE"; + &[ + PlannedTarget { + field: "topology", + value: "storage-side BlueField-3; the GPU client keeps an ordinary ConnectX", + measured: NOT_RUN, + }, + PlannedTarget { + field: "not_this_daemon", + value: "host GPUNetIO NIC-mode samples are not a DPU Arm storage daemon", + measured: NOT_RUN, + }, + PlannedTarget { + field: "dpu_mode", + value: "target: DPU mode so the Arm complex owns the NVMe namespaces; mode unset", + measured: NOT_RUN, + }, + PlannedTarget { + field: "bfb", + value: "target: BlueField BFB matched to that mode; version unset", + measured: NOT_RUN, + }, + PlannedTarget { + field: "os", + value: "target: Arm OS image from that BFB; version unset", + measured: NOT_RUN, + }, + PlannedTarget { + field: "doca", + value: "target: DOCA SDK matched to that BFB; version unset; not linked", + measured: NOT_RUN, + }, + PlannedTarget { + field: "firmware", + value: "target: BF3 firmware matched to that BFB; version unset", + measured: NOT_RUN, + }, + PlannedTarget { + field: "iommu", + value: "target: record IOMMU or SMMU, BAR, and registration limits at install; this tree does not change them", + measured: NOT_RUN, + }, + PlannedTarget { + field: "data_path", + value: "first honest path: DPU DRAM slab plus NVMe I/O; simulated in-process here", + measured: NOT_RUN, + }, + ] +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn targets_are_unmeasured() { + let rows = planned_targets(); + assert!(rows.len() >= 8); + assert!(rows.iter().all(|row| row.measured == "NOT_RUN_HARDWARE")); + assert!(rows.iter().any(|row| row.field == "not_this_daemon")); + let text = rows + .iter() + .map(|row| row.value) + .collect::>() + .join(" "); + assert!(!text.contains("already implemented")); + assert!(!text.contains("deployed")); + } +} diff --git a/crates/protocol/src/harden.rs b/crates/protocol/src/harden.rs new file mode 100644 index 0000000..9786486 --- /dev/null +++ b/crates/protocol/src/harden.rs @@ -0,0 +1,478 @@ +// Copyright 2024-2026 RustFS, Inc. +// SPDX-License-Identifier: Apache-2.0 + +//! CPU hooks for tenant authorization, replay windows, integrity seals, +//! and secret rotation. +//! +//! This is not mTLS. The TCP listener still relies on a trusted fabric when +//! it leaves loopback. See docs/adr/0007-trusted-fabric.md. Seals are SHA-256 +//! over a length-delimited encoding. They are not an HMAC and not a device MAC. +//! Audit events carry generation ids and kinds, never fingerprints or payloads. + +use std::collections::VecDeque; + +use gpucache_core_types::{ + content_checksum, CacheError, Checksum, TenantId, TrustDomain, HARD_BYTES_CAP, +}; + +use crate::auth::{fingerprints_equal, AuthContext}; + +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub enum AuditEvent { + Rotate { generation: u64 }, + Retire { generation: u64 }, + Deny, + Replay, +} + +#[derive(Clone, Debug, PartialEq, Eq)] +pub struct SecurityLog { + events: VecDeque, + cap: usize, +} + +impl SecurityLog { + pub fn new(cap: usize) -> Result { + if cap == 0 || cap > 1024 { + return Err(CacheError::Malformed { reason: "audit" }); + } + Ok(Self { + events: VecDeque::new(), + cap, + }) + } + + pub fn push(&mut self, event: AuditEvent) { + if self.events.len() == self.cap { + self.events.pop_front(); + } + self.events.push_back(event); + } + + pub fn events(&self) -> impl Iterator { + self.events.iter() + } + + pub fn len(&self) -> usize { + self.events.len() + } + + pub fn is_empty(&self) -> bool { + self.events.is_empty() + } +} + +struct AclEntry { + tenant: TenantId, + trust: TrustDomain, + fingerprint: [u8; 32], + write: bool, +} + +pub struct Acl { + entries: Vec, + max: usize, +} + +impl Acl { + pub fn new(max: usize) -> Result { + if max == 0 || max > 1024 { + return Err(CacheError::Malformed { reason: "authz" }); + } + Ok(Self { + entries: Vec::new(), + max, + }) + } + + pub fn grant(&mut self, ctx: AuthContext, write: bool) -> Result<(), CacheError> { + if let Some(entry) = self.entries.iter_mut().find(|entry| { + entry.tenant == ctx.tenant + && entry.trust == ctx.trust_domain + && fingerprints_equal(&entry.fingerprint, &ctx.credential_fingerprint) + }) { + entry.write = write; + return Ok(()); + } + if self.entries.len() >= self.max { + return Err(CacheError::QuotaExceeded { + kind: gpucache_core_types::QuotaKind::InFlightRequests, + }); + } + self.entries.push(AclEntry { + tenant: ctx.tenant, + trust: ctx.trust_domain, + fingerprint: ctx.credential_fingerprint, + write, + }); + Ok(()) + } + + pub fn check(&self, ctx: &AuthContext, write: bool) -> Result<(), CacheError> { + let mut allowed = false; + for entry in &self.entries { + let same = entry.tenant == ctx.tenant + && entry.trust == ctx.trust_domain + && fingerprints_equal(&entry.fingerprint, &ctx.credential_fingerprint); + if same && (!write || entry.write) { + allowed = true; + } + } + if allowed { + Ok(()) + } else { + Err(CacheError::Unauthorized) + } + } +} + +pub struct ReplayGuard { + skew_ms: u64, + capacity: usize, + seen: VecDeque<([u8; 32], [u8; 16])>, +} + +impl ReplayGuard { + pub fn new(skew_ms: u64, capacity: usize) -> Result { + if capacity == 0 || capacity > 4096 { + return Err(CacheError::Malformed { reason: "replay" }); + } + Ok(Self { + skew_ms, + capacity, + seen: VecDeque::new(), + }) + } + + /// Accept `nonce` once inside the skew window. + /// + /// The window is bounded. After `capacity` newer nonces, the oldest nonce + /// can be presented again. That is a window, not a permanent replay cache. + pub fn observe( + &mut self, + tenant: &[u8], + nonce: &[u8; 16], + now_ms: u64, + issued_ms: u64, + ) -> Result<(), CacheError> { + if tenant.is_empty() || *nonce == [0u8; 16] { + return Err(CacheError::Malformed { reason: "replay" }); + } + if issued_ms > now_ms || now_ms - issued_ms > self.skew_ms { + return Err(CacheError::Unauthorized); + } + let tenant_key = content_checksum(tenant).0; + if self + .seen + .iter() + .any(|(seen_tenant, seen_nonce)| *seen_tenant == tenant_key && seen_nonce == nonce) + { + return Err(CacheError::IdempotencyConflict); + } + if self.seen.len() == self.capacity { + self.seen.pop_front(); + } + self.seen.push_back((tenant_key, *nonce)); + Ok(()) + } +} + +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub struct IntegritySeal { + pub key_id: u64, + pub generation: u64, + pub checksum: Checksum, +} + +pub fn seal(key_id: u64, generation: u64, payload: &[u8]) -> Result { + if payload.is_empty() || payload.len() > HARD_BYTES_CAP { + return Err(CacheError::Malformed { + reason: "integrity", + }); + } + Ok(IntegritySeal { + key_id, + generation, + checksum: tag(key_id, generation, payload), + }) +} + +pub fn open_seal(seal: &IntegritySeal, payload: &[u8]) -> Result<(), CacheError> { + let actual = tag(seal.key_id, seal.generation, payload); + if actual != seal.checksum { + return Err(CacheError::ChecksumMismatch); + } + Ok(()) +} + +fn tag(key_id: u64, generation: u64, payload: &[u8]) -> Checksum { + let mut buf = Vec::with_capacity(20 + payload.len()); + buf.extend_from_slice(&key_id.to_le_bytes()); + buf.extend_from_slice(&generation.to_le_bytes()); + let len = u32::try_from(payload.len()).unwrap_or(u32::MAX); + buf.extend_from_slice(&len.to_le_bytes()); + buf.extend_from_slice(payload); + content_checksum(&buf) +} + +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +enum GenState { + Current, + Overlap, + Retired, +} + +struct Gen { + id: u64, + fingerprint: [u8; 32], + state: GenState, +} + +pub struct SecretRing { + items: Vec, + next_id: u64, + log: SecurityLog, +} + +impl SecretRing { + pub fn open(fingerprint: [u8; 32], audit_cap: usize) -> Result { + if fingerprint == [0u8; 32] { + return Err(CacheError::Unauthorized); + } + let mut ring = Self { + items: Vec::new(), + next_id: 1, + log: SecurityLog::new(audit_cap)?, + }; + let id = ring.push(fingerprint, GenState::Current); + ring.log.push(AuditEvent::Rotate { generation: id }); + Ok(ring) + } + + pub fn rotate(&mut self, fingerprint: [u8; 32]) -> Result { + if fingerprint == [0u8; 32] { + return Err(CacheError::Unauthorized); + } + if self + .items + .iter() + .any(|item| item.fingerprint == fingerprint && item.state != GenState::Retired) + { + return Err(CacheError::IdempotencyConflict); + } + for item in &mut self.items { + if item.state == GenState::Current { + item.state = GenState::Overlap; + } + } + let id = self.push(fingerprint, GenState::Current); + self.log.push(AuditEvent::Rotate { generation: id }); + Ok(id) + } + + pub fn retire(&mut self, id: u64) -> Result<(), CacheError> { + let Some(item) = self.items.iter_mut().find(|item| item.id == id) else { + return Err(CacheError::UnknownGeneration); + }; + if item.state != GenState::Overlap { + return Err(CacheError::FencingRejected { reason: "secret" }); + } + item.state = GenState::Retired; + self.log.push(AuditEvent::Retire { generation: id }); + Ok(()) + } + + pub fn allow_read(&self, fingerprint: &[u8; 32]) -> Result { + self.allow(fingerprint, false) + } + + pub fn allow_write(&self, fingerprint: &[u8; 32]) -> Result { + self.allow(fingerprint, true) + } + + pub fn log(&self) -> &SecurityLog { + &self.log + } + + fn allow(&self, fingerprint: &[u8; 32], write: bool) -> Result { + for item in &self.items { + if !fingerprints_equal(&item.fingerprint, fingerprint) { + continue; + } + let ok = match item.state { + GenState::Current => true, + GenState::Overlap => !write, + GenState::Retired => false, + }; + if ok { + return Ok(item.id); + } + } + Err(CacheError::Unauthorized) + } + + fn push(&mut self, fingerprint: [u8; 32], state: GenState) -> u64 { + let id = self.next_id; + self.next_id = self.next_id.saturating_add(1); + self.items.push(Gen { + id, + fingerprint, + state, + }); + id + } +} + +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub struct RequestStamp { + pub write: bool, + pub nonce: [u8; 16], + pub now_ms: u64, + pub issued_ms: u64, +} + +pub fn authorize_request( + acl: &Acl, + replay: &mut ReplayGuard, + log: &mut SecurityLog, + ctx: &AuthContext, + stamp: RequestStamp, +) -> Result<(), CacheError> { + if let Err(err) = acl.check(ctx, stamp.write) { + log.push(AuditEvent::Deny); + return Err(err); + } + if let Err(err) = replay.observe( + ctx.tenant.as_bytes(), + &stamp.nonce, + stamp.now_ms, + stamp.issued_ms, + ) { + if err != (CacheError::Malformed { reason: "replay" }) { + log.push(AuditEvent::Replay); + } + return Err(err); + } + Ok(()) +} + +#[cfg(test)] +mod tests { + use super::*; + use gpucache_core_types::TenantId; + + fn ctx(tenant: &[u8], fingerprint: u8) -> AuthContext { + AuthContext::new( + TenantId::new(tenant.to_vec()).unwrap(), + TrustDomain::new(b"td-local".to_vec()).unwrap(), + [fingerprint; 32], + ) + .unwrap() + } + + #[test] + fn acl_fails_closed_and_read_does_not_grant_write() { + let mut acl = Acl::new(4).unwrap(); + assert_eq!( + acl.check(&ctx(b"tenant-a", 1), false).unwrap_err(), + CacheError::Unauthorized + ); + acl.grant(ctx(b"tenant-a", 1), false).unwrap(); + acl.check(&ctx(b"tenant-a", 1), false).unwrap(); + assert_eq!( + acl.check(&ctx(b"tenant-a", 1), true).unwrap_err(), + CacheError::Unauthorized + ); + assert_eq!( + acl.check(&ctx(b"tenant-b", 1), false).unwrap_err(), + CacheError::Unauthorized + ); + acl.grant(ctx(b"tenant-a", 1), true).unwrap(); + acl.check(&ctx(b"tenant-a", 1), true).unwrap(); + } + + #[test] + fn replay_rejects_a_second_nonce_and_a_stale_stamp() { + let mut guard = ReplayGuard::new(1000, 2).unwrap(); + let nonce = [2u8; 16]; + guard.observe(b"tenant-a", &nonce, 5_000, 4_500).unwrap(); + assert_eq!( + guard + .observe(b"tenant-a", &nonce, 5_100, 4_600) + .unwrap_err(), + CacheError::IdempotencyConflict + ); + guard.observe(b"tenant-b", &nonce, 5_100, 4_600).unwrap(); + assert_eq!( + guard + .observe(b"tenant-a", &[3u8; 16], 9_000, 1_000) + .unwrap_err(), + CacheError::Unauthorized + ); + } + + #[test] + fn seal_detects_a_bit_flip_and_returns_no_match_on_success_path() { + let payload = b"kv-bytes"; + let seal = seal(7, 3, payload).unwrap(); + open_seal(&seal, payload).unwrap(); + let mut bad = *payload; + bad[0] ^= 1; + assert_eq!( + open_seal(&seal, &bad).unwrap_err(), + CacheError::ChecksumMismatch + ); + } + + #[test] + fn rotation_keeps_overlap_reads_and_audits_without_the_fingerprint() { + let mut ring = SecretRing::open([4u8; 32], 8).unwrap(); + let first = ring.allow_write(&[4u8; 32]).unwrap(); + let second = ring.rotate([5u8; 32]).unwrap(); + assert!(ring.allow_read(&[4u8; 32]).is_ok()); + assert_eq!( + ring.allow_write(&[4u8; 32]).unwrap_err(), + CacheError::Unauthorized + ); + ring.allow_write(&[5u8; 32]).unwrap(); + ring.retire(first).unwrap(); + assert_eq!( + ring.allow_read(&[4u8; 32]).unwrap_err(), + CacheError::Unauthorized + ); + assert_eq!( + ring.retire(second).unwrap_err(), + CacheError::FencingRejected { reason: "secret" } + ); + let text = format!("{:?}", ring.log().events().collect::>()); + assert!(text.contains("Rotate")); + assert!(text.contains("Retire")); + assert!(!text.contains("4, 4, 4")); + assert!(!text.contains("5, 5, 5")); + } + + #[test] + fn authorize_request_audits_deny_and_replay() { + let mut acl = Acl::new(4).unwrap(); + acl.grant(ctx(b"tenant-a", 9), true).unwrap(); + let mut replay = ReplayGuard::new(50, 4).unwrap(); + let mut log = SecurityLog::new(8).unwrap(); + let stamp = RequestStamp { + write: true, + nonce: [9u8; 16], + now_ms: 100, + issued_ms: 90, + }; + authorize_request(&acl, &mut replay, &mut log, &ctx(b"tenant-a", 9), stamp).unwrap(); + authorize_request(&acl, &mut replay, &mut log, &ctx(b"tenant-a", 9), stamp).unwrap_err(); + let denied = RequestStamp { + write: false, + nonce: [1u8; 16], + now_ms: 100, + issued_ms: 90, + }; + authorize_request(&acl, &mut replay, &mut log, &ctx(b"tenant-a", 8), denied).unwrap_err(); + let events: Vec<_> = log.events().copied().collect(); + assert_eq!(events, vec![AuditEvent::Replay, AuditEvent::Deny]); + } +} diff --git a/crates/protocol/src/lib.rs b/crates/protocol/src/lib.rs index 5df515e..dcc320f 100644 --- a/crates/protocol/src/lib.rs +++ b/crates/protocol/src/lib.rs @@ -3,13 +3,15 @@ //! Batch messages, deadlines, and wire-version negotiation for GPUCache. //! //! V1 A defined the request frames. V1 B adds reply frames and a TCP listener -//! in `gpucache-transport`. Nothing here depends on a GPU, RDMA stack, or DPU. +//! in `gpucache-transport`. V3 adds CPU authz, replay, seal, and rotation +//! hooks. Nothing here depends on a GPU, RDMA stack, or DPU. #![forbid(unsafe_code)] mod api; mod auth; mod codec; +mod harden; mod negotiate; mod reply; @@ -28,6 +30,10 @@ pub use codec::{ OP_COMMIT, OP_GET, OP_HELLO, OP_HELLO_ACK, OP_INVALIDATE, OP_LOOKUP, OP_RELEASE, OP_RESERVE, WIRE_MAGIC, }; +pub use harden::{ + authorize_request, open_seal, seal, Acl, AuditEvent, IntegritySeal, ReplayGuard, RequestStamp, + SecretRing, SecurityLog, +}; pub use negotiate::{negotiate, HELLO_OP, WIRE_VERSION}; pub use reply::{decode_reply, encode_reply, ReplyBody, REPLY_MAGIC}; diff --git a/crates/protocol/src/reply.rs b/crates/protocol/src/reply.rs index 4983321..cbb4ba9 100644 --- a/crates/protocol/src/reply.rs +++ b/crates/protocol/src/reply.rs @@ -490,6 +490,7 @@ fn quota_byte(kind: QuotaKind) -> u8 { QuotaKind::BatchItems => 5, QuotaKind::IdempotencyTable => 6, QuotaKind::PrefixLinks => 7, + QuotaKind::RateLimit => 8, } } @@ -503,6 +504,7 @@ fn read_quota(value: u8) -> Result { 5 => QuotaKind::BatchItems, 6 => QuotaKind::IdempotencyTable, 7 => QuotaKind::PrefixLinks, + 8 => QuotaKind::RateLimit, _ => { return Err(CacheError::Malformed { reason: "quota kind", diff --git a/crates/transport/src/directory.rs b/crates/transport/src/directory.rs index e77ae21..dd28b64 100644 --- a/crates/transport/src/directory.rs +++ b/crates/transport/src/directory.rs @@ -145,6 +145,12 @@ impl ShardDirectory { self.flights.len() } + /// Drop in-flight fills and bump the epoch. Credits return to the pool. + /// Callers must treat the dropped fills as misses. This is not a copy. + pub fn fence_epoch(&mut self) -> u64 { + self.fence_inflight() + } + pub fn check_epoch(&self, request_epoch: u64) -> Result<(), CacheError> { if request_epoch != self.epoch { return Err(CacheError::EpochMismatch { diff --git a/crates/transport/src/lib.rs b/crates/transport/src/lib.rs index 104dc50..eca1b5d 100644 --- a/crates/transport/src/lib.rs +++ b/crates/transport/src/lib.rs @@ -16,7 +16,9 @@ mod directory; mod fault_demo; mod frame; mod iface; +mod plane; mod pool; +mod rate; mod registered; mod server; mod shard_demo; @@ -30,6 +32,8 @@ pub use iface::{ rdma_feature_selected, rdma_probe, HardwarePath, KvTransport, NixlTransport, TcpTransport, TransferDesc, TransferReceipt, TransportKind, }; +pub use plane::{ControlPlane, RebalancePlan, UpgradeStamp}; +pub use rate::RateGate; pub use registered::{CheckedBatch, PoolStats, RegisteredPool, SimCompletion, SlotId}; pub use server::{serve, ServerConfig, ServerHandle}; pub use shard_demo::{run_shard_demo, ShardReport}; diff --git a/crates/transport/src/plane.rs b/crates/transport/src/plane.rs new file mode 100644 index 0000000..2488677 --- /dev/null +++ b/crates/transport/src/plane.rs @@ -0,0 +1,295 @@ +// Copyright 2024-2026 RustFS, Inc. +// SPDX-License-Identifier: Apache-2.0 + +//! Rebalance, rolling upgrade, and fault-domain isolation on top of one shard. +//! +//! These paths fence in-flight fills. They do not copy payload bytes and they +//! do not add a replica. A fenced fill is a miss. The engine recomputes. + +use std::collections::BTreeMap; + +use gpucache_core_types::CacheError; + +use crate::directory::{Admit, FillRequest, ShardDirectory, ShardParams}; + +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +enum Phase { + Serving, + Draining, +} + +pub struct RebalancePlan { + pub from_shard: u64, + pub to_shard: u64, + pub source_epoch: u64, + pub destination: ShardDirectory, +} + +pub struct UpgradeStamp { + pub generation: u64, + pub epoch: u64, + pub fenced: bool, +} + +pub struct ControlPlane { + dir: ShardDirectory, + params: ShardParams, + domains: BTreeMap, + phase: Phase, + software_generation: u64, + upgraded: bool, +} + +impl ControlPlane { + pub fn single_node(params: ShardParams) -> Self { + Self { + dir: ShardDirectory::single_node(params.clone()), + params, + domains: BTreeMap::new(), + phase: Phase::Serving, + software_generation: 1, + upgraded: false, + } + } + + pub fn epoch(&self) -> u64 { + self.dir.epoch() + } + + pub fn inflight(&self) -> usize { + self.dir.inflight() + } + + pub fn credits_used(&self) -> u64 { + self.dir.credits_used() + } + + pub fn software_generation(&self) -> u64 { + self.software_generation + } + + pub fn serving(&self) -> bool { + self.phase == Phase::Serving + } + + pub fn begin_or_join(&mut self, req: &FillRequest) -> Result { + if self.phase != Phase::Serving { + return Err(CacheError::Cancelled); + } + self.dir.begin_or_join(req) + } + + pub fn complete_flight( + &mut self, + flight_id: u64, + request_epoch: u64, + request: [u8; 16], + ) -> Result<(), CacheError> { + self.dir.complete_flight(flight_id, request_epoch, request) + } + + pub fn assign_domain(&mut self, node_id: u64, domain: u16) -> Result<(), CacheError> { + if !self + .dir + .members() + .iter() + .any(|member| member.node_id == node_id) + { + return Err(CacheError::Malformed { + reason: "fault domain", + }); + } + self.domains.insert(node_id, domain); + Ok(()) + } + + /// Remove every member in `domain`. The last remaining member stays. + /// In-flight fills are fenced and become misses. No payload is copied. + pub fn isolate_domain(&mut self, domain: u16) -> Result { + let victims: Vec = self + .dir + .members() + .iter() + .filter(|member| self.domains.get(&member.node_id) == Some(&domain)) + .map(|member| member.node_id) + .collect(); + if victims.is_empty() { + return Err(CacheError::Malformed { + reason: "fault domain", + }); + } + if victims.len() >= self.dir.members().len() { + return Err(CacheError::FencingRejected { reason: "member" }); + } + let mut epoch = self.dir.epoch(); + for node_id in victims { + epoch = self.dir.remove_member(node_id)?; + self.domains.remove(&node_id); + } + Ok(epoch) + } + + /// Fence this shard and hand back an empty directory for `to_shard`. + /// The destination has no flights. That is a miss, not a moved hit. + pub fn rebalance_to(&mut self, to_shard: u64) -> Result { + if self.phase != Phase::Serving { + return Err(CacheError::Cancelled); + } + if to_shard == self.dir.shard_id() { + return Err(CacheError::Malformed { + reason: "rebalance", + }); + } + let source_epoch = self.dir.fence_epoch(); + let destination = ShardDirectory::open(to_shard, self.params.clone()); + Ok(RebalancePlan { + from_shard: self.dir.shard_id(), + to_shard, + source_epoch, + destination, + }) + } + + pub fn begin_upgrade(&mut self) -> Result<(), CacheError> { + if self.phase != Phase::Serving { + return Err(CacheError::Malformed { reason: "upgrade" }); + } + self.phase = Phase::Draining; + Ok(()) + } + + pub fn finish_upgrade(&mut self) -> Result { + if self.phase != Phase::Draining { + return Err(CacheError::Malformed { reason: "upgrade" }); + } + let epoch = self.dir.fence_epoch(); + self.software_generation = self.software_generation.saturating_add(1); + self.upgraded = true; + self.phase = Phase::Serving; + Ok(UpgradeStamp { + generation: self.software_generation, + epoch, + fenced: true, + }) + } + + /// From `Draining`, resume serving without a fence so an in-flight fill + /// can still complete. After `finish_upgrade`, fence again so a writer + /// on the old epoch misses. The generation counter does not go backwards. + pub fn rollback(&mut self) -> Result { + if self.phase == Phase::Draining { + self.phase = Phase::Serving; + return Ok(UpgradeStamp { + generation: self.software_generation, + epoch: self.dir.epoch(), + fenced: false, + }); + } + if !self.upgraded { + return Err(CacheError::Malformed { reason: "upgrade" }); + } + let epoch = self.dir.fence_epoch(); + Ok(UpgradeStamp { + generation: self.software_generation, + epoch, + fenced: true, + }) + } +} + +#[cfg(test)] +mod tests { + use super::*; + + fn params() -> ShardParams { + ShardParams::new(100, 80, 4, 2, 4).unwrap() + } + + fn fill(epoch: u64) -> FillRequest { + FillRequest { + physical_key: b"phys-a".to_vec(), + tenant: b"tenant-a".to_vec(), + bytes: 10, + request: 1u128.to_le_bytes(), + epoch, + } + } + + #[test] + fn rebalance_fences_the_source_and_starts_an_empty_destination() { + let mut plane = ControlPlane::single_node(params()); + let admit = plane.begin_or_join(&fill(1)).unwrap(); + let Admit::Leader { flight_id, epoch } = admit else { + panic!("leader"); + }; + let plan = plane.rebalance_to(2).unwrap(); + assert_eq!(plan.from_shard, 0); + assert_eq!(plan.to_shard, 2); + assert_eq!(plan.destination.shard_id(), 2); + assert_eq!(plan.destination.inflight(), 0); + assert_eq!(plane.inflight(), 0); + assert_eq!(plane.credits_used(), 0); + let err = plane + .complete_flight(flight_id, epoch, 1u128.to_le_bytes()) + .unwrap_err(); + assert!(matches!(err, CacheError::EpochMismatch { .. })); + assert_eq!(plan.source_epoch, plane.epoch()); + } + + #[test] + fn rolling_upgrade_rejects_new_work_and_rollback_before_finish_keeps_the_fill() { + let mut plane = ControlPlane::single_node(params()); + let admit = plane.begin_or_join(&fill(1)).unwrap(); + let Admit::Leader { flight_id, .. } = admit else { + panic!("leader"); + }; + plane.begin_upgrade().unwrap(); + assert_eq!( + plane.begin_or_join(&fill(1)).unwrap_err(), + CacheError::Cancelled + ); + let stamp = plane.rollback().unwrap(); + assert!(!stamp.fenced); + assert!(plane.serving()); + plane + .complete_flight(flight_id, 1, 1u128.to_le_bytes()) + .unwrap(); + assert_eq!(plane.inflight(), 0); + } + + #[test] + fn finish_upgrade_fences_and_a_later_rollback_fences_again() { + let mut plane = ControlPlane::single_node(params()); + plane.begin_or_join(&fill(1)).unwrap(); + plane.begin_upgrade().unwrap(); + let finished = plane.finish_upgrade().unwrap(); + assert!(finished.fenced); + assert_eq!(finished.generation, 2); + assert_eq!(plane.inflight(), 0); + plane.begin_or_join(&fill(plane.epoch())).unwrap(); + let rolled = plane.rollback().unwrap(); + assert!(rolled.fenced); + assert_eq!(rolled.generation, 2); + assert_eq!(plane.inflight(), 0); + assert_eq!(plane.credits_used(), 0); + } + + #[test] + fn isolate_domain_refuses_to_remove_the_last_member() { + let mut plane = ControlPlane::single_node(params()); + let node = plane.dir.members()[0].node_id; + plane.assign_domain(node, 7).unwrap(); + plane.begin_or_join(&fill(1)).unwrap(); + let err = plane.isolate_domain(7).unwrap_err(); + assert_eq!(err, CacheError::FencingRejected { reason: "member" }); + assert_eq!(plane.inflight(), 1); + assert_eq!(plane.epoch(), 1); + plane.assign_domain(node, 1).unwrap(); + plane.dir.note_member(9, 1).unwrap(); + plane.assign_domain(9, 7).unwrap(); + let epoch = plane.isolate_domain(7).unwrap(); + assert!(epoch > 1); + assert_eq!(plane.inflight(), 0); + assert!(plane.dir.members().iter().all(|member| member.node_id != 9)); + } +} diff --git a/crates/transport/src/rate.rs b/crates/transport/src/rate.rs new file mode 100644 index 0000000..010ec92 --- /dev/null +++ b/crates/transport/src/rate.rs @@ -0,0 +1,123 @@ +// Copyright 2024-2026 RustFS, Inc. +// SPDX-License-Identifier: Apache-2.0 + +//! Per-tenant admission window. The caller passes `now_ms`. This module does +//! not sleep and does not read a device clock, so it stays deterministic under +//! qemu-user. + +use std::collections::BTreeMap; + +use gpucache_core_types::{CacheError, QuotaKind}; + +struct Window { + start_ms: u64, + used: u32, +} + +pub struct RateGate { + default_limit: u32, + window_ms: u64, + max_tenants: usize, + limits: BTreeMap, u32>, + windows: BTreeMap, Window>, +} + +impl RateGate { + pub fn new(default_limit: u32, window_ms: u64, max_tenants: usize) -> Result { + if default_limit == 0 || window_ms == 0 || max_tenants == 0 { + return Err(CacheError::Malformed { reason: "rate" }); + } + Ok(Self { + default_limit, + window_ms, + max_tenants, + limits: BTreeMap::new(), + windows: BTreeMap::new(), + }) + } + + pub fn set_limit(&mut self, tenant: &[u8], limit: u32) -> Result<(), CacheError> { + if tenant.is_empty() || tenant.len() > 128 || limit == 0 { + return Err(CacheError::Malformed { reason: "rate" }); + } + if !self.limits.contains_key(tenant) && self.limits.len() >= self.max_tenants { + return Err(CacheError::QuotaExceeded { + kind: QuotaKind::RateLimit, + }); + } + self.limits.insert(tenant.to_vec(), limit); + Ok(()) + } + + pub fn admit(&mut self, tenant: &[u8], now_ms: u64) -> Result<(), CacheError> { + if tenant.is_empty() || tenant.len() > 128 { + return Err(CacheError::Malformed { reason: "rate" }); + } + let limit = self + .limits + .get(tenant) + .copied() + .unwrap_or(self.default_limit); + if !self.windows.contains_key(tenant) && self.windows.len() >= self.max_tenants { + return Err(CacheError::QuotaExceeded { + kind: QuotaKind::RateLimit, + }); + } + let window = self.windows.entry(tenant.to_vec()).or_insert(Window { + start_ms: now_ms, + used: 0, + }); + if now_ms.saturating_sub(window.start_ms) >= self.window_ms { + window.start_ms = now_ms; + window.used = 0; + } + if window.used >= limit { + return Err(CacheError::QuotaExceeded { + kind: QuotaKind::RateLimit, + }); + } + window.used = window.used.saturating_add(1); + Ok(()) + } +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn noisy_neighbor_does_not_spend_the_quiet_tenant_budget() { + let mut gate = RateGate::new(2, 1_000, 8).unwrap(); + gate.set_limit(b"noisy", 1).unwrap(); + gate.admit(b"noisy", 10).unwrap(); + assert_eq!( + gate.admit(b"noisy", 20).unwrap_err(), + CacheError::QuotaExceeded { + kind: QuotaKind::RateLimit, + } + ); + gate.admit(b"quiet", 20).unwrap(); + gate.admit(b"quiet", 20).unwrap(); + assert_eq!( + gate.admit(b"quiet", 20).unwrap_err(), + CacheError::QuotaExceeded { + kind: QuotaKind::RateLimit, + } + ); + gate.admit(b"quiet", 1_020).unwrap(); + gate.admit(b"noisy", 5).unwrap_err(); + } + + #[test] + fn clock_rewind_does_not_refresh_the_window() { + let mut gate = RateGate::new(1, 1_000, 2).unwrap(); + gate.admit(b"tenant-a", 500).unwrap(); + assert!(gate.admit(b"tenant-a", 100).is_err()); + assert_eq!( + gate.admit(b"tenant-a", 500).unwrap_err(), + CacheError::QuotaExceeded { + kind: QuotaKind::RateLimit, + } + ); + } +} diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index 4136056..eda9483 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -1,4 +1,4 @@ -# Architecture (V1 C, V2 first slice) +# Architecture (V1 C, V2 first slice, V3 in progress) GPUCache is a shared KV data tier. The hot tier is G3.5. RustFS is an optional colder G4 capacity tier. It is not on the prefill-to-decode path. The inference engine still owns tokenization, attention, and scheduling. This repo does not replace them. @@ -18,13 +18,18 @@ gpucache-storage::MemoryStore v gpucache-core-types (identity, keys, lifecycle, errors) -gpucache-cold optional G4 pack/manifest, async sink, budgeted refill. - Not linked by transport. +gpucache-cold optional G4 pack/manifest, async sink, budgeted refill, + retention and reclaim. Not linked by transport. V2 control plane (same process, not a new wire protocol) KvTransport TCP submit is a host copy. NIXL open refuses. RegisteredPool simulated pin, reuse, quarantine. Not a NIC. ShardDirectory one shard per process. Epoch fence, single-flight, credits. + +V3 (not on the TCP hot path) + gpucache-dpu simulated DRAM slab and storage queue. DOCA/SPDK/NVMe open refuses. + ControlPlane rebalance, rolling upgrade, fault domain. Fences fills. No replica. + protocol harden ACL, replay window, integrity seal, secret rotation. ``` `MemoryStore` remains the in-process contract. `FileStore` is what the TCP service uses. `gpucache-bench` only names the paired baselines and prints `NOT_RUN_HARDWARE`. Planned crates still absent: a standalone metadata service, an engine adapter, a DPU agent, and metrics. Their contracts stay out of `core-types` so GPU and DPU types do not leak in. @@ -81,3 +86,15 @@ ABSENT -> RESERVED -> WRITING -> COMMITTED -> EVICTING -> ABSENT `ShardDirectory` is local to one shard. Shard 0 is the single-node case and uses the same methods. A membership change bumps the epoch, drops in-flight fills, and releases credits. Single-flight joins do not charge twice and do not return payload bytes. There is no per-token central directory and no per-block quorum. See [adr/0005-shard-fencing.md](adr/0005-shard-fencing.md). The cold sink queues a pack and drains it with Put. Refill Heads the manifest first. Over budget returns `OverBudget` and no payload. That outcome is a recompute, not a hit. + +## V3 offload and hardening + +`gpucache-dpu` is a separate crate. The hot path does not call it. The candidate offload is the storage queue: submit, then complete, against a bounded in-process slab. Completions are labeled `NOT_RUN_HARDWARE`. `ArmAgent::start` does not set `offload_active`. See [adr/0006-storage-queue-offload.md](adr/0006-storage-queue-offload.md) and [V3_DPU_TARGETS.md](V3_DPU_TARGETS.md). + +`AffineQueue` records the reactor id a future SPDK queue would be bound to. A mismatched id is an error. No reactor thread is started. + +`ControlPlane` wraps one `ShardDirectory`. Rebalance fences the source and returns an empty destination directory. Rolling upgrade rejects new admits while draining. Finish fences. Rollback before finish lets the in-flight fill complete. Fault-domain isolation removes members in that domain and refuses to remove the last member. None of these copy payload bytes. + +G4 `ColdOps` records retention, checks `MANIFEST_VERSION`, reclaims through delete plus a following head, and refuses a retired tenant key. `refill_degraded` turns a slow or unavailable RustFS into `Missing` without touching the store. Backfill enqueue stops at a byte budget. Live HTTP is unchanged and still `NOT_RUN_HARDWARE` without `GPUCACHE_RUSTFS_*`. + +The degrade table is [V3_FAILURES.md](V3_FAILURES.md). Every row is a clean miss. diff --git a/docs/BENCHMARKS.md b/docs/BENCHMARKS.md index e74af99..154add9 100644 --- a/docs/BENCHMARKS.md +++ b/docs/BENCHMARKS.md @@ -24,4 +24,26 @@ Paired baselines use the same prefix-cache idea on both sides when a GPU run exi There is no `FAIL_PERFORMANCE` figure because there is no paired measurement to fail. Do not add a benchmark that prints a success banner without raw samples, a manifest, and the hardware it ran on. The V2 CPU tests do not fill this gap. +## V3 paired gate + +The harness is `cargo run -p gpucache-bench --bin v3-paired`. + +The comparison it names is **NIC RDMA baseline vs DPU storage queue**, under the same GPU, model, trace, service behavior, capacity, policy, auth semantics, and SLO. TCP is forbidden as that baseline (`tcp_used_as_dpu_baseline=FORBIDDEN`). A weak TCP number would not be a DPU result. Soft-RoCE, if someone later smokes it on a hypervisor, is not a performance result and must not be copied into this table. + +| Row | Result | +| --- | --- | +| NIC RDMA baseline | `NOT_RUN_HARDWARE` | +| DPU storage queue | `NOT_RUN_HARDWARE` | +| TTFT / TPOT / ITL / SLO goodput | `NOT_RUN_HARDWARE` | +| Host core-seconds, DPU CPU, DPU DRAM, GPU interference | `NOT_RUN_HARDWARE` | +| Wire bytes, SSD write amplification, wall power | `NOT_RUN_HARDWARE` | +| TCO | `TCO_UNKNOWN` | +| Go gate | `NOT_EVALUATED` | +| PB-scale | `NOT_CLAIMED` | +| Production ready | `NOT_CLAIMED` | + +Engineering targets, not measurements, and not yet confirmed as the go gate: full cost per million useful output tokens improves by at least ten percent with correctness and SLO held, or an isolation goal that removes a whole x86 box is met without a higher total cost. A twenty percent drop in host core-seconds is only a mechanism signal. It does not by itself prove the cost target. Until a paired run exists the gate stays `NOT_EVALUATED`. A no-go keeps the portable TCP and file backends. This tree does not buy hardware to chase the gate. + +Soak of a day or longer, with the fault matrix on real devices, was not run. + Correctness criteria for the CPU path are in [CORRECTNESS.md](CORRECTNESS.md). diff --git a/docs/OPERATIONS.md b/docs/OPERATIONS.md index dc5b19a..fa8ca9d 100644 --- a/docs/OPERATIONS.md +++ b/docs/OPERATIONS.md @@ -91,6 +91,46 @@ There is no on-disk config file. Flags are the schema. Leave `GPUCACHE_RUSTFS_ENDPOINT`, `GPUCACHE_RUSTFS_BUCKET`, `GPUCACHE_RUSTFS_PREFIX`, `GPUCACHE_RUSTFS_ACCESS_KEY`, and `GPUCACHE_RUSTFS_SECRET_KEY` unset unless an operator has authorized a test bucket. Unset means `NOT_RUN_HARDWARE`. The mock smoke still runs inside `cargo test`. `https://` is `NOT_RUN_HARDWARE` in this milestone because no TLS stack is linked. Do not invent credentials. +## V3 CPU commands + +```bash +cargo run -p gpucache-dpu --bin dpu-demo +cargo run -p gpucache-bench --bin v3-paired +cargo test -p gpucache-dpu --all-targets --features dpu-bf3 +``` + +`dpu-demo` prints the contract result and `NOT_RUN_HARDWARE`. `v3-paired` prints `TCO_UNKNOWN` and `go_gate=NOT_EVALUATED`. The feature test still refuses DOCA, SPDK, and NVMe open. + +## Diagnostics + +`dpu-demo` is the diagnostics sketch. It prints whether `dpu-bf3` was selected, that DOCA/SPDK/NVMe are not linked, that hot-tier EC is off, and that each planned target is `NOT_RUN_HARDWARE`. It does not print payloads, fingerprints, or prompts. There is no device health counter because no device is open. + +## Capacity + +Hot-tier limits (`ResourceLimits`, shard credits, the simulated slab cap, cold retention, the backfill byte budget) are admission ceilings for recomputable KV. They are not a measured cluster size. A lost hot block is a miss. Do not size the hot tier as the only copy of user data, and do not describe this tree as PB-scale. Cold retention deletes expired mock objects after a verified delete. It is not a second durability replica of the hot tier. + +## Upgrade and rollback + +`ControlPlane::begin_upgrade` stops new fills. In-flight fills can still complete. `rollback` before finish returns to serving without a fence. `finish_upgrade` bumps a software generation and fences, so old epochs miss and the engine recomputes. `rollback` after finish fences again. The generation counter does not decrease. No BlueField firmware, BFB, or DOCA package is part of this rollback. The TCP wire format is unchanged apart from quota kind byte 8 (`RateLimit`) on error replies. + +## Minimal deploy example + +Development only, loopback, ordinary files, no DPU mode change: + +```bash +cargo run -p gpucache-transport --bin gpucache-tcp -- serve \ + --data-dir /tmp/gpucache-dev \ + --tenant tenant-a \ + --trust td-local \ + --fingerprint <64 hex chars of a credential hash> +``` + +Then, in another shell, `cargo run -p gpucache-dpu --bin dpu-demo` shows the offload interface is still `NOT_RUN_HARDWARE`. Do not point `--data-dir` at a device node. Do not pass a live secret as the fingerprint; pass the hash. + +## Compatibility + +See [../COMPATIBILITY.md](../COMPATIBILITY.md) and [V3_DPU_TARGETS.md](V3_DPU_TARGETS.md). TCP remains the correctness path. `rdma-nixl` and `dpu-bf3` are empty features. aarch64 CI is a CPU run. + ## Not operated here -No NVMe format, no DPU mode change, no firmware, no IOMMU or BIOS change, no production network change. Those need a separate authorization. There is no website deploy and no crates.io publish in this milestone. +No NVMe format, no DPU mode change, no firmware, no IOMMU or BIOS change, no production network change, no hardware purchase. Those need a separate authorization. There is no website deploy and no crates.io publish in this milestone. G4 live calls stay env-gated. diff --git a/docs/RISKS.md b/docs/RISKS.md index 51491d3..d4c42fc 100644 --- a/docs/RISKS.md +++ b/docs/RISKS.md @@ -23,6 +23,15 @@ | Short write or disk full reported as a durable hit | Mitigated | The in-memory commit is abandoned and the request id is forgotten. See `docs/V1C_TEST_MATRIX.md` | | Hostile protocol bytes | Mitigated by decoder fuzz | `proptest` inside `cargo test`. Not a multi-hour libFuzzer run | | Wrong model or tokenizer hit | Mitigated | Those identities miss. Comparison of synthetic tokens is exact. `tolerance=NONE` | -| Invented baseline numbers | Closed in this tree | `paired-baseline` prints `NOT_RUN_HARDWARE` and contains no digits | +| Invented baseline numbers | Closed in this tree | `paired-baseline` and `v3-paired` print `NOT_RUN_HARDWARE` and contain no digits. TCO is `TCO_UNKNOWN` | +| Simulated DPU queue treated as BlueField | Mitigated by labels | `dpu-demo` prints `offload_active=false` and `NOT_RUN_HARDWARE`. Feature `dpu-bf3` links nothing | +| Hot-tier EC switched on through an ablation knob | Mitigated | `Ablation::set` rejects erasure coding. ADR 0002 | +| Integrity seal treated as a MAC | Documented | SHA-256 of a length-delimited buffer. Not an HMAC. ADR 0007 | +| Replay window eviction | Accepted | The oldest nonce can be reused after the cap. Not a permanent cache | +| Secret audit leaking a credential | Mitigated in tests | Events store kinds and generation ids, not fingerprints | +| Control-plane loss returning a stale hit | Mitigated on the simulated queue | Slots are hidden and new submits fail until `restore_control`. Old slots stay hidden | +| RustFS slowness blocking prefill | Mitigated in the hook | `refill_degraded` returns `Missing` and does not call the store | +| DOCA license and performance disclosure | `NOT_RUN_LICENSE` | No DOCA binary is linked. Review the actual EULA before linking or publishing numbers. See THIRD_PARTY_NOTICES.md | +| Calling this tree production-ready or PB-scale | Closed | Both labels are `NOT_CLAIMED` | Minimum reproduction of the CPU contract is `cargo test --workspace --all-targets`. There is no hardware failure log because the hardware was not present. diff --git a/docs/SECURITY.md b/docs/SECURITY.md index f720bb8..dc07248 100644 --- a/docs/SECURITY.md +++ b/docs/SECURITY.md @@ -1,4 +1,4 @@ -# Security (V1 C) +# Security (V1 C, V3 hooks) The library calls still require an `AuthContext`. The TCP service does too. It binds `127.0.0.1` unless the operator sets `--allow-remote`, and it refuses to start with an empty tenant policy. A data request with an unknown tenant, a mismatched trust domain, or a wrong fingerprint is `Unauthorized` and returns no payload. Hello carries versions and the store epoch only. @@ -14,13 +14,25 @@ The fingerprint is a hash of a credential, never the credential. Do not pass a r - Only `COMMITTED`, checksum-valid, same-identity blocks are returned. - Quarantine plus fencing is the rule for buffers whose completion is unknown. A timer is not treated as proof that a copy has stopped. +## V3 hooks + +These sit beside the V1 gate. They do not replace it and they do not add mTLS. See [adr/0007-trusted-fabric.md](adr/0007-trusted-fabric.md). + +- `Acl` fails closed. A grant is read or write. Write is not implied by read. Unknown tenants and unknown fingerprints are `Unauthorized` and return no payload. +- `ReplayGuard` rejects a repeated nonce inside a bounded window and rejects a stamp outside the caller-supplied skew. The window forgets the oldest nonce when it is full. That is not permanent replay protection. +- `seal` / `open_seal` mix a key id and generation into a SHA-256. A mismatch is `ChecksumMismatch` and returns no bytes. This is not an HMAC. Anyone who can write the bytes can recompute it. It is an integrity hook for a trusted fabric, not a claim of cryptographic authenticity against a hostile store. +- `SecretRing` rotates fingerprints. The previous fingerprint stays readable until `retire`. Writes use the current fingerprint only. Audit events store `Rotate`, `Retire`, `Deny`, and `Replay` plus a generation id. They do not store the fingerprint, the payload, or a prompt. +- `RateGate` is a per-tenant admit window. A noisy tenant does not consume another tenant's budget. Refusal is `QuotaExceeded { RateLimit }`. +- Control-plane loss on the simulated DPU queue hides slots and refuses new submits. It does not return a hit while membership is unknown. + ## What is not enforced -- mTLS, replay protection beyond the idempotency window, secret rotation, and audit shipping are later work (the product note puts production hardening in V3; V1 still requires this minimum gate before any network exposure). +- mTLS is not linked. The TCP service does not call `Acl` or `ReplayGuard` yet; those are library hooks a caller can use beside the V1 fingerprint. `--allow-remote` does not add them and does not terminate TLS. Treat a non-loopback bind as a trusted fabric, or do not bind it. +- Replay protection beyond the idempotency window and the bounded nonce window is not a permanent cache. - `Attestation::Drained` is accepted from any caller who already passed the tenant gate. It is not a device measurement. A buggy client inside the tenant can reclaim its own in-flight buffer early. - The idempotency table forgets entries after `idempotency_ttl_ms` or when it is full (new ids are then rejected until expiry). Reusing an id after expiry can run the operation again. - This tree does not encrypt KV bytes at rest. `MemoryStore` keeps them in process memory. `FileStore` writes them as ordinary files in the directory you name. That directory must not be a production disk or a device node. The open path rejects `/dev`, `/proc`, `/sys`, `/`, and `..`. -- TCP has no mTLS and no replay window beyond idempotency. Do not expose it past a trusted host. `--allow-remote` does not add authentication; the fingerprint check is still the only gate. +- TCP has no mTLS. The V3 nonce window is a separate hook; the TCP server does not call it yet. Do not expose the listener past a trusted fabric. `--allow-remote` does not add authentication; the fingerprint check is still required. - `PrefixHit` bytes cover only `matched_prefix_blocks` of the query chain. A client that ignores that field and decodes them as the full query will mis-align. The server will not do that alignment for the client. ## Fallback diff --git a/docs/V3_DPU_TARGETS.md b/docs/V3_DPU_TARGETS.md new file mode 100644 index 0000000..5083d5f --- /dev/null +++ b/docs/V3_DPU_TARGETS.md @@ -0,0 +1,21 @@ +# V3 DPU targets + +These rows are install targets for a future storage-side BlueField-3. They are not measurements. This repository did not purchase a DPU, flash a BFB, change IOMMU or ACS, format NVMe, or link DOCA. `planned_targets()` in `gpucache-dpu` prints the same fields with `measured=NOT_RUN_HARDWARE`. + +| Field | Target | Recorded value | +| --- | --- | --- | +| Topology | Storage-side BlueField-3 next to the NVMe. The GPU host keeps an ordinary ConnectX. | Not assembled. `NOT_RUN_HARDWARE` | +| What this daemon is not | An Arm process on the DPU that serves the DRAM slab and the NVMe queue | Host GPUNetIO NIC-mode samples are not this process. Do not label them as the storage daemon | +| DPU mode | DPU mode, so the embedded CPU owns the NVMe namespaces used by the cache | Unset. This tree does not switch modes | +| BFB | The BlueField boot image paired with that mode | Version unset | +| OS | The Arm OS from that BFB | Version unset | +| DOCA | The DOCA SDK paired with that BFB | Version unset. Not linked. Feature `dpu-bf3` is an empty Cargo feature | +| Firmware | The BF3 firmware paired with that BFB | Version unset | +| IOMMU / BAR / registration | Limits written down at install time | Unset. This tree does not change BIOS, IOMMU, or ACS | +| First data path | DRAM slab plus NVMe submit/complete | In-process simulation only. `open_nvme_device` returns `nvme not linked` | +| SPDK, if it is ever linked | One reactor thread per affine queue. The queue is not a Tokio task and is not moved across workers | Not linked. `AffineQueue::submit_on` checks the reactor id only | +| Soft-RoCE | Functional smoke on a developer hypervisor, if someone runs it later | Not run here. Not a performance result. Not the V3 baseline | + +An aarch64 `cargo test` is a CPU build of the same Rust core. It does not execute this table. + +Public wording: BlueField-3, GDS, and DOCA are **in development / `NOT_RUN_HARDWARE` / optional feature**. They are not already implemented and not deployed. diff --git a/docs/V3_FAILURES.md b/docs/V3_FAILURES.md new file mode 100644 index 0000000..b5bede1 --- /dev/null +++ b/docs/V3_FAILURES.md @@ -0,0 +1,20 @@ +# V3 failure matrix + +Each row is a clean miss. The engine recomputes and may fill again. The hook returns no payload, does not report a wrong hit, and does not stay blocked. `StorageQueue::apply_fault` applies the DPU-local rows to the simulated slab. RustFS rows do not wipe the hot slab; `refill_degraded` returns `Missing` without calling the store. + +| Fault | What the hook does | Result | +| --- | --- | --- | +| DPU restart | Drop in-flight tasks and drop slab slots | Clean miss. A new write can complete. Old slot ids are not readable | +| GPU reset | Hide every slot and drop in-flight tasks. Client completion is unknown, so bytes stay unreadable | Clean miss. The queue still accepts a new write | +| NIC flap | Same hide. An in-flight transfer is not a hit | Clean miss. Not a permanent block | +| Storage loss | Hide slots. Hidden bytes are not a durability replica and are not returned | Clean miss, then recompute | +| NVMe timeout | Drop one in-flight task. Do not expose it | Clean miss. Later submits still run | +| Control-plane loss | Hide slots and refuse new submits with `Cancelled` | No read while membership is unknown. `restore_control` allows new writes. Old slots stay hidden | +| RustFS slow | Do not wait and do not call the object store | `RefillOutcome::Missing` | +| RustFS unavailable | Same as slow | `RefillOutcome::Missing` | + +Checksum mismatch hides the simulated slot after the failing read. A wrong tenant is `Unauthorized` and receives no bytes. + +Lease expiry is still not proof that a copy stopped. The simulated slab does not free a slot only because a timer fired. Restart and the other faults above are explicit fencing events, not a lease callback. + +This matrix is a CPU contract. It is not a fault campaign on BlueField, GPU, NIC, or RustFS hardware. diff --git a/docs/V3_TEST_MATRIX.md b/docs/V3_TEST_MATRIX.md new file mode 100644 index 0000000..b9f7fcc --- /dev/null +++ b/docs/V3_TEST_MATRIX.md @@ -0,0 +1,35 @@ +# V3 test matrix + +Command for the CPU rows: + +```bash +cargo test --workspace --all-targets +cargo test -p gpucache-dpu --all-targets --features dpu-bf3 +cargo run -p gpucache-dpu --bin dpu-demo +cargo run -p gpucache-bench --bin v3-paired +``` + +Default features do not need DOCA, SPDK, libnixl, or a DPU. `--features dpu-bf3` still refuses `open_doca`, `open_spdk`, and `open_nvme_device`. + +`test-aarch64` and `test-aarch64-qemu` in CI remain CPU runs. This slice adds no sleeps and no threads, so it does not add a qemu-user timer race. An aarch64 pass is not a BlueField pass. Qemu-user is not a performance run. + +| Check | Result in this tree | Label | +| --- | --- | --- | +| `dpu-demo` simulated round trip, devices closed, EC off | Expected `DPU_CONTRACT_OK` with `offload_active=false` | CPU contract. Hardware `NOT_RUN_HARDWARE` | +| Storage-queue submit, complete, fail, wrong tenant, checksum flip | Unit tests in `gpucache-dpu` | CPU contract | +| Arm `start` does not set offload | Unit test | CPU contract | +| Reactor id mismatch | `AffineQueue` unit test | Contract only. SPDK `NOT_RUN_HARDWARE` | +| Ablation knobs and hot-tier EC refusal | Unit test | EC stays off | +| Fault matrix | `StorageQueue::apply_fault` tests | CPU contract. Hardware faults `NOT_RUN_HARDWARE` | +| ACL, replay, seal, secret rotation, audit contents | `gpucache-protocol` tests | CPU contract. mTLS `NOT_RUN` (not linked) | +| Rate limit and noisy neighbor | `RateGate` tests | CPU contract. Caller clock, no sleep | +| Rebalance, rolling upgrade, rollback, fault domain | `ControlPlane` tests | CPU contract. No payload copy | +| G4 retention, verified delete, cold key retire, backfill cap, slow RustFS | `gpucache-cold` tests | Mock `PASS`. Live RustFS `NOT_RUN_HARDWARE` | +| Unknown manifest version | `read_manifest` test | Rejected. Not a hit | +| V3 paired report | No digits. NIC RDMA required. `TCO_UNKNOWN`. `go_gate=NOT_EVALUATED` | `NOT_RUN_HARDWARE` | +| Real BF3 / DOCA / SPDK / NVMe device | Not present | `NOT_RUN_HARDWARE` | +| DOCA license review before any linkage or perf publication | Not done | `NOT_RUN_LICENSE` | +| 24 hour soak, wall power, PB-scale | Not run | `NOT_RUN_HARDWARE` / `NOT_RUN_BUDGET` | +| vLLM or SGLang KV round trip | Not run | `NOT_RUN_HARDWARE` | + +`status=DPU_CONTRACT_OK` means the simulation matched. It is not a device pass and not `production ready`. diff --git a/docs/adr/0001-integration-route.md b/docs/adr/0001-integration-route.md index 71d3aaa..092df9d 100644 --- a/docs/adr/0001-integration-route.md +++ b/docs/adr/0001-integration-route.md @@ -33,6 +33,10 @@ Prefer the hook that already exists in the pinned revision. Switch only if that The fault-matrix work did not run the GPU spike. This ADR stays **Pending**. No engine adapter was added. +## V3 + +The V3 CPU slice did not run the spike and did not add SGLang or HiCache. See [0008-second-engine.md](0008-second-engine.md). This ADR stays **Pending**. + ## Consequences - Callers must not assume GPUCache can read an arbitrary engine's KV bytes. diff --git a/docs/adr/0002-no-erasure-coding.md b/docs/adr/0002-no-erasure-coding.md index a266f99..b09eff5 100644 --- a/docs/adr/0002-no-erasure-coding.md +++ b/docs/adr/0002-no-erasure-coding.md @@ -26,4 +26,5 @@ DOCA EC, SPDK RAID targets, and similar offloads are not GPUCache components. Th - The V1 crates have no EC types, no parity buffers, and no replica placement. The file backend deletes or skips a bad file; it does not rebuild one. V1 C did not add parity. A short write or a full disk is a miss. - RustFS remains responsible for any durable cold object it is asked to store. The cold client is off the hot path and is a contract mock unless an authorized endpoint is configured. - V2 shard membership fences in-flight fills. It does not copy a block onto another node and it is not a durability replica. +- V3 rebalance and fault-domain isolation fence the same way. They do not copy a block. The DPU ablation refuses to enable hot-tier erasure coding. - A cache miss after a node loss is correct behavior. The KV block is not the user's only copy. diff --git a/docs/adr/0006-storage-queue-offload.md b/docs/adr/0006-storage-queue-offload.md new file mode 100644 index 0000000..b0a4a47 --- /dev/null +++ b/docs/adr/0006-storage-queue-offload.md @@ -0,0 +1,43 @@ +# ADR 0006: The first offload candidate is the storage queue + +- Status: **Accepted as a candidate, not as a measured win** +- Date: 2026-10-15 +- Milestone: V3 + +## Context + +V3 asks for one offload, with a real submit and complete path, on a storage-side BlueField-3. The GPU client keeps an ordinary ConnectX. Host GPUNetIO NIC-mode samples are a different topology and are not this daemon. + +V2 did not produce host CPU, power, or GPU-interference numbers. Those rows are `NOT_RUN_HARDWARE`. The spec says not to widen a hardware design without bottleneck evidence. This change was explicitly authorized as a shippable slice without a BlueField, so the choice below is a hypothesis the paired experiment can accept or reject. It is not evidence that the bottleneck was measured. + +Candidates: + +| Candidate | Why it was not selected as the first offload | +| --- | --- | +| NIC / network processing | Ordinary ConnectX RDMA already bypasses host staging once NIXL is linked. Calling that a DPU win would mix two different claims. V2 left NIXL unlinked on purpose. | +| Local directory and quota | Real CPU work, and it stays on the portable control plane so a DPU restart cannot become the only copy of membership. Fencing already lives in `ShardDirectory`. | +| Crypto | Integrity seals in this tree are SHA-256 checksums, not a cipher the DPU must terminate. Crypto offload is an ablation knob, default off. | +| Compression, P2P, DPA | No measured byte savings or path. Ablation knobs only. | +| Erasure coding | Rejected for the hot tier. See [0002-no-erasure-coding.md](0002-no-erasure-coding.md). The knob cannot be turned on. | + +The V2 hop list shows the storage side as a host copy into `FileStore` plus the kernel page cache. The spec's first honest DPU data path is a DRAM slab plus NVMe I/O, owned by the Arm complex next to the drives. + +## Decision + +The single candidate offload is the **storage queue**: submit a read or write against a bounded DRAM slab, then complete it. + +- `ArmAgent::start` records that a process exists. `offload_active` stays false. Process start is not the offload. +- `StorageQueue::submit_write` / `submit_read` and `complete` are the task interface. A failed or timed-out task does not expose bytes. +- `open_simulated` copies inside this process and labels the completion `NOT_RUN_HARDWARE` and `simulated=true`. +- `open_nvme_device`, `open_doca`, and `open_spdk` return a storage error in every build, including `--features dpu-bf3`. The feature has no crate dependencies. +- A future SPDK backend must use an independent reactor. `AffineQueue` refuses a submit attributed to a different reactor id. Affine objects are not Tokio tasks and must not move across Tokio workers. This tree does not start SPDK or Tokio. +- P2P, DPA, compression, and crypto can be marked on. Their probe is `NOT_RUN_HARDWARE` and they do not change bytes. Hot-tier EC stays off. + +Planned DPU mode, BFB, OS, DOCA, firmware, and IOMMU limits are listed in [../V3_DPU_TARGETS.md](../V3_DPU_TARGETS.md) as targets. Versions are unset. Nothing was flashed. + +## Consequences + +- `gpucache-dpu` depends only on `gpucache-core-types`. Transport, storage, protocol, and cold do not depend on it. The TCP hot path is unchanged. +- `dpu-demo` prints `status=DPU_CONTRACT_OK` only after the simulated round trip. The same output says `hardware=NOT_RUN_HARDWARE` and `offload_active=false`. +- A later measurement may reject this candidate. The portable TCP and file backends stay. A no-go does not upgrade hardware. +- BF4 and DOCA features beyond this queue are not claimed. diff --git a/docs/adr/0007-trusted-fabric.md b/docs/adr/0007-trusted-fabric.md new file mode 100644 index 0000000..ed398f7 --- /dev/null +++ b/docs/adr/0007-trusted-fabric.md @@ -0,0 +1,37 @@ +# ADR 0007: Trusted fabric, not a linked mTLS stack + +- Status: **Accepted** +- Date: 2026-10-15 +- Milestone: V3 + +## Context + +V1 already requires a tenant, a trust domain, and a credential fingerprint on every data call. The TCP listener binds loopback unless the operator sets `--allow-remote`. V3 asks for tenant authorization, mTLS or an explicit trusted-fabric boundary, integrity, anti-replay, quota and rate limits, noisy-neighbor tests, secret rotation, and audit events. + +Linking a TLS stack (for example rustls) would add a new dependency and would still not be a measured mutual-auth deployment. This tree has no certificate authority, no production network, and no operator policy for which peers are allowed. Pretending the socket is mTLS would be a false claim. + +## Decision + +The V3 boundary is a **trusted fabric**, documented here, plus CPU hooks. mTLS is not linked. + +Operators: + +- Leave the listener on `127.0.0.1` unless the host network is already restricted to cache peers. +- `--allow-remote` does not add authentication and does not start TLS. The fingerprint and the V3 ACL still apply. +- Do not place this TCP service on an open network and call that mTLS. + +In-process hooks, all caller-clocked and bounded: + +- `Acl` maps a fingerprint to read or write. An empty ACL fails closed. A read grant is not a write grant. The fingerprint compare scans every entry. +- `ReplayGuard` accepts a nonce once inside a caller-supplied skew. The window is bounded. Evicting the oldest nonce means that nonce can appear again later. This is not an infinite replay cache. +- `seal` / `open_seal` bind a key id and a generation into a SHA-256 of a length-delimited buffer. A mismatch is `ChecksumMismatch` and returns no bytes. This is not an HMAC and not a transport MAC. On a trusted fabric it catches accidental corruption. It does not stop a caller who can recompute SHA-256. +- `RateGate` limits admits per tenant per window. A noisy tenant spends only its own budget. +- `SecretRing::rotate` moves the previous fingerprint to overlap (read, not write) and audits the generation id. `retire` applies to overlap only. The audit log stores event kinds and generation ids, not fingerprints, payloads, or prompts. + +Shard membership, epoch fencing, rebalance, rolling upgrade, and fault-domain isolation extend the V2 directory. They fence in-flight fills. They do not copy payloads and they are not replicas. See [0005-shard-fencing.md](0005-shard-fencing.md). + +## Consequences + +- `cargo test --workspace --all-targets` covers these hooks without certificates. +- A later milestone may add mTLS behind a feature flag. Until that lands, security docs must keep saying the fabric is the boundary. +- Wrong-tenant and replay failures are not hits. Callers recompute. diff --git a/docs/adr/0008-second-engine.md b/docs/adr/0008-second-engine.md new file mode 100644 index 0000000..0b92de3 --- /dev/null +++ b/docs/adr/0008-second-engine.md @@ -0,0 +1,22 @@ +# ADR 0008: SGLang / HiCache stays an evaluation note + +- Status: **Pending** +- Date: 2026-10-15 +- Milestone: V3 + +## Context + +V3 asks for a second engine after the primary interface is stable, with SGLang / HiCache as the preferred evaluation. Each layout has to be tested on its own. A KV block is not a universal tensor. + +The primary route is still undecided. [0001-integration-route.md](0001-integration-route.md) is **Pending** because the vLLM spike was not run (`NOT_RUN_HARDWARE`). Adding a second engine now would be a second half-built path. + +## Decision + +Do not add SGLang, HiCache, or any other engine adapter in this change. + +When ADR 0001 is Accepted and one vLLM path round-trips a real block, a follow-up may evaluate SGLang / HiCache against that same identity and checksum contract. The evaluation has to use that engine's layout, dtype, and shard placement. A miss on mismatch stays a miss. This note is not an integration and not a claim that formats are interchangeable. + +## Consequences + +- No new crate, dependency, or Python package was added for a second engine. +- Docs must not say GPUCache speaks SGLang or HiCache today.