V1 C: fault matrix, exact synthetic tokens, and baseline stubs

.github/workflows/ci.yml

@@ -20,12 +20,20 @@ jobs:
         run: cargo clippy --workspace --all-targets -- -D warnings
       - name: Test
         run: cargo test --workspace --all-targets
+      - name: Fallback demo
+        run: cargo run -p gpucache-transport --bin fault-demo --quiet
+      - name: Paired baseline stub
+        run: cargo run -p gpucache-bench --bin paired-baseline --quiet
       - name: Reject accelerator crates in V1
         run: |
           if grep -RInE 'cuda|doca|nixl|rdma|spdk' crates/*/Cargo.toml; then
             echo "V1 crates must not depend on GPU or DPU stacks"
             exit 1
           fi
+          if grep -RInE 'reed_solomon|erasure_coding|parity_shard' crates --include '*.rs'; then
+            echo "V1 must not add erasure coding"
+            exit 1
+          fi
       - name: Keep the cold tier off the hot path
         run: |
           if grep -n 'gpucache-cold' crates/storage/Cargo.toml crates/transport/Cargo.toml crates/protocol/Cargo.toml crates/core-types/Cargo.toml; then
@@ -41,7 +49,9 @@ jobs:
       - uses: actions/checkout@v4
       - uses: dtolnay/rust-toolchain@stable
       - name: Test
-        run: cargo test --workspace --all-targets
+        run: |
+          echo "aarch64 cargo test is a CPU build. It is not a BlueField or DOCA run."
+          cargo test --workspace --all-targets
 
   test-aarch64-qemu:
     name: test (aarch64 qemu-user)

AGENTS.md

@@ -4,7 +4,7 @@ GPUCache is the G3.5 shared KV tier. RustFS is the colder G4 backend. Read [READ
 
 ## Phase
 
-The active phase is **V1 milestone B**. Do not implement V1 C, V2 RDMA, or V3 DPU work in a V1 B change. Do not publish crates, cut releases, deploy, or edit a website unless asked.
+The active phase is **V1 milestone C**. Do not implement V2 RDMA or V3 DPU work in a V1 C change. Do not publish crates, cut releases, deploy, or edit a website unless asked.
 
 ## Honesty rules
 
@@ -23,6 +23,7 @@ The active phase is **V1 milestone B**. Do not implement V1 C, V2 RDMA, or V3 DP
 | `gpucache-storage` | the two crates above |
 | `gpucache-transport` | the three crates above. TCP only |
 | `gpucache-cold` | `gpucache-core-types` and `sha2`. Not a dependency of storage or transport |
+| `gpucache-bench` | nothing. Prints `NOT_RUN_HARDWARE` baseline rows and must not invent timings |
 
 No crate may add CUDA, DOCA, NIXL, RDMA, SPDK, erasure coding, or replica placement. `unsafe` is forbidden. Registered buffers whose completion is unknown stay quarantined until drain or a newer epoch; do not free them only because a lease timer fired. The cold crate stays off the hot path.
 

COMPATIBILITY.md

@@ -1,12 +1,12 @@
 # Compatibility manifest
 
-Recorded for V1 milestones A and B. This file is not a performance acceptance report and not a hardware inventory of a BlueField node.
+Recorded for V1 milestones A, B, and C. This file is not a performance acceptance report and not a hardware inventory of a BlueField node.
 
 | Component | Value | Result |
 | --- | --- | --- |
 | OS | Ubuntu 24.04.5 LTS | Recorded on the V1 A build host |
 | Kernel | Linux 6.12.94+ | Recorded on the V1 A build host |
-| Architecture exercised locally | x86_64 native, and aarch64 via qemu-user (`qemu-aarch64-static`) | `PASS` for `cargo test --workspace --all-targets` on both. This is not a BlueField run. |
+| Architecture exercised locally for V1 C | x86_64 native | `PASS` for `cargo test --workspace --all-targets`, `fault-demo`, and `paired-baseline`. aarch64 was not executed on this host (`NOT_RUN` here). CI still runs `test-aarch64` and `test-aarch64-qemu`. Neither is a BlueField run. |
 | Rust | stable 1.99.0 (b940084d7 2026-09-28); MSRV declared as 1.85 | `PASS` |
 | Python | 3.12.3 present on the build host | Not used by V1 A |
 | CUDA | Not installed | `NOT_RUN_HARDWARE` |
@@ -17,7 +17,7 @@ Recorded for V1 milestones A and B. This file is not a performance acceptance re
 | DOCA / BlueField BFB / firmware | Not present | `NOT_RUN_HARDWARE` |
 | RustFS server / client | No endpoint authorized in this environment. Contract mock covers Put, Head, RangeGet, Delete, pack, and manifest | Live probe `NOT_RUN_HARDWARE`. Mock `PASS` |
 | TCP / file store | Loopback TCP and a temp directory on the build host | `PASS` for `cargo test --workspace --all-targets` on x86_64. Not an NVMe format and not a DPU run |
-| Direct Rust deps | `sha2` 0.10.8; dev `proptest` 1.4.0 | Apache-2.0 OR MIT; see `Cargo.lock`. No new direct crate in V1 B |
+| Direct Rust deps | `sha2` 0.10.8; dev `proptest` 1.4.0 | Apache-2.0 OR MIT; see `Cargo.lock`. V1 C adds no direct crate. `gpucache-bench` has an empty dependency list |
 | License of this repo | Apache-2.0 | See `LICENSE` |
 
 CI (see `.github/workflows/ci.yml`) builds and runs the same tests on x86_64 and on aarch64 (GitHub ARM runner, plus qemu-user cross execution). An aarch64 result is a CPU result. It does not certify BlueField-3, DOCA, or NVMe-on-DPU.

Cargo.toml

@@ -6,6 +6,7 @@ members = [
     "crates/storage",
     "crates/cold",
     "crates/transport",
+    "crates/bench",
 ]
 
 [workspace.package]

README.md

@@ -8,7 +8,7 @@ GPUCache does not do erasure coding and does not keep durability replicas. A los
 
 ## Status
 
-**V1 milestones A and B are implemented. DPU acceleration is not.**
+**V1 milestones A, B, and C are implemented. DPU acceleration is not.**
 
 V1 runs the cache contract on CPU. The long-term deployment target is a storage-side BlueField-3, but this repository does not link DOCA, CUDA, RDMA, or SPDK, and it does not claim that any of those paths exist. An aarch64 build is a CPU build. It is not a BlueField functional test.
 
@@ -18,7 +18,7 @@ V1 runs the cache contract on CPU. The long-term deployment target is a storage-
 | V1 A: real vLLM KV block export/import | `NOT_RUN_HARDWARE` (no GPU in this environment) |
 | V1 B: TCP service, file backend, LRU admission, tenant quotas, two-client reuse | Implemented in this tree |
 | V1 B: RustFS live Put/Head/RangeGet/Delete | `NOT_RUN_HARDWARE` (no authorized endpoint). Contract mock passes |
-| V1 C: fuzz, fault injection, engine output comparison, paired baselines | Not started |
+| V1 C: fuzz, fault injection, synthetic token checks, paired-baseline stubs | Implemented in this tree. Real engine output is `NOT_RUN_HARDWARE` |
 | V2: NIC RDMA (NIXL or Mooncake) plus a real RustFS cold path | Not started; blocked on V1 |
 | V3: BlueField-3 data-path offload and production hardening | Not started; not purchased and not measured. Not an erasure-coding milestone |
 | Erasure coding and durability replicas | Will not be built. A lost or failed hot block is a clean miss, then recompute and repopulate. Cold durability is RustFS G4 |
@@ -63,7 +63,7 @@ From a clean checkout, with Rust stable (MSRV 1.85):
 cargo test --workspace --all-targets
 ```
 
-That command is the CPU contract repro for V1 A and V1 B. It includes unit tests, property tests, timeout and cancel tests, a single-process import of a synthetic block, a cross-process import through `kv-block-xfer`, file-backend quota and restart tests, and the TCP reuse tests. The bytes are synthetic. They are not a vLLM KV tensor.
+That command is the CPU contract repro for V1 A, V1 B, and V1 C. It includes unit tests, property tests, protocol fuzz, timeout and cancel tests, a single-process import of a synthetic block, a cross-process import through `kv-block-xfer`, file-backend quota and restart tests, the TCP reuse tests, and the V1 C fault matrix. The bytes are synthetic. They are not a vLLM KV tensor.
 
 Two localhost clients sharing one file-backed service:
 
@@ -73,6 +73,20 @@ cargo run -p gpucache-transport --bin share-demo
 
 A successful run prints `status=REUSE_OK` and `partial_prefix=WHOLE_BLOCK_PREFIX`. It does not print a sliced-block hit.
 
+Hit, miss, and recompute fallback from a clean temp directory:
+
+```bash
+cargo run -p gpucache-transport --bin fault-demo
+```
+
+Success is `status=FALLBACK_OK` plus `criteria=EXACT_TOKEN_IDS` and `tolerance=NONE`. The process prints that banner only after the checks pass. A wrong tenant or wrong model is not a hit. This is not a vLLM run.
+
+Paired GPU baselines stay `NOT_RUN_HARDWARE` and print no timings:
+
+```bash
+cargo run -p gpucache-bench --bin paired-baseline
+```
+
 `kv-block-xfer import <credential-hex>` reads one export frame on stdin and imports it into a fresh `MemoryStore`. It does not talk to an engine.
 
 ## Layout
@@ -81,13 +95,14 @@ A successful run prints `status=REUSE_OK` and `partial_prefix=WHOLE_BLOCK_PREFIX
 crates/core-types    identity, keys, lifecycle, errors
 crates/protocol      batch messages, replies, version negotiation
 crates/storage       MemoryStore, FileStore, kv-block-xfer
-crates/transport     TCP server, client, share-demo
+crates/transport     TCP server, client, share-demo, fault-demo
 crates/cold          RustFS contract, pack/manifest, live probe
+crates/bench         paired-baseline stub (no timings)
 docs/adr             decision records
 COMPATIBILITY.md     environment manifest
 ```
 
-Design notes: [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md), [docs/PROTOCOL.md](docs/PROTOCOL.md), [docs/SECURITY.md](docs/SECURITY.md), [docs/OPERATIONS.md](docs/OPERATIONS.md), [docs/BENCHMARKS.md](docs/BENCHMARKS.md), [docs/RISKS.md](docs/RISKS.md).
+Design notes: [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md), [docs/PROTOCOL.md](docs/PROTOCOL.md), [docs/SECURITY.md](docs/SECURITY.md), [docs/OPERATIONS.md](docs/OPERATIONS.md), [docs/BENCHMARKS.md](docs/BENCHMARKS.md), [docs/CORRECTNESS.md](docs/CORRECTNESS.md), [docs/RISKS.md](docs/RISKS.md), [docs/V1C_TEST_MATRIX.md](docs/V1C_TEST_MATRIX.md).
 
 ## License
 

THIRD_PARTY_NOTICES.md

@@ -1,6 +1,6 @@
 # Third-party notices
 
-GPUCache V1 A and V1 B are Apache-2.0. See [LICENSE](LICENSE).
+GPUCache V1 A, V1 B, and V1 C are Apache-2.0. See [LICENSE](LICENSE). V1 C adds no direct dependency.
 
 Direct dependencies:
 

crates/bench/Cargo.toml

@@ -0,0 +1,16 @@
+[package]
+name = "gpucache-bench"
+version.workspace = true
+edition.workspace = true
+license.workspace = true
+rust-version.workspace = true
+authors.workspace = true
+description = "Paired-baseline harness stubs. GPU rows stay NOT_RUN_HARDWARE. No accelerator dependencies."
+publish = false
+
+[[bin]]
+name = "paired-baseline"
+path = "src/bin/paired_baseline.rs"
+
+[lints]
+workspace = true

crates/bench/src/bin/paired_baseline.rs

@@ -0,0 +1,14 @@
+// Copyright 2024-2026 RustFS, Inc.
+// SPDX-License-Identifier: Apache-2.0
+//! Print the paired-baseline matrix. Every GPU row is NOT_RUN_HARDWARE.
+//!
+//! This process does not measure TTFT, TPOT, or token goodput.
+
+use std::process::ExitCode;
+
+use gpucache_bench::paired_baseline_report;
+
+fn main() -> ExitCode {
+    print!("{}", paired_baseline_report());
+    ExitCode::SUCCESS
+}

crates/bench/src/lib.rs

@@ -0,0 +1,60 @@
+// Copyright 2024-2026 RustFS, Inc.
+// SPDX-License-Identifier: Apache-2.0
+//! Paired baseline names for the V1 exit criteria.
+//!
+//! This crate does not run a GPU, an engine, or a timer. It prints
+//! `NOT_RUN_HARDWARE` for every row that needs token goodput. Filling in a
+//! number here would be a fabricated result.
+
+/// Text report. It contains no digits, so it cannot smuggle a latency or a token rate.
+pub fn paired_baseline_report() -> String {
+    let lines = [
+        "status=NOT_RUN_HARDWARE",
+        "engine_native_prefix_cache=NOT_RUN_HARDWARE",
+        "local_offload=NOT_RUN_HARDWARE",
+        "gpucache_cpu_tcp_goodput=NOT_RUN_HARDWARE",
+        "scenario_cold=NOT_RUN_HARDWARE",
+        "scenario_warm=NOT_RUN_HARDWARE",
+        "scenario_no_reuse=NOT_RUN_HARDWARE",
+        "scenario_multi_turn=NOT_RUN_HARDWARE",
+        "scenario_hbm_within=NOT_RUN_HARDWARE",
+        "scenario_hbm_cliff=NOT_RUN_HARDWARE",
+        "write_cost=NOT_RUN_HARDWARE",
+        "samples=NONE",
+        "numeric_tolerance=NOT_APPLICABLE",
+        "bypass_when_no_reuse=MISS_RECOMPUTE",
+    ];
+    let mut out = String::new();
+    for line in lines {
+        out.push_str(line);
+        out.push('\n');
+    }
+    out
+}
+
+#[cfg(test)]
+mod tests {
+    use super::paired_baseline_report;
+
+    #[test]
+    fn report_has_no_invented_numbers() {
+        let report = paired_baseline_report();
+        assert!(report.chars().all(|c| !c.is_ascii_digit()));
+        for key in [
+            "engine_native_prefix_cache=NOT_RUN_HARDWARE",
+            "local_offload=NOT_RUN_HARDWARE",
+            "gpucache_cpu_tcp_goodput=NOT_RUN_HARDWARE",
+            "scenario_cold=NOT_RUN_HARDWARE",
+            "scenario_warm=NOT_RUN_HARDWARE",
+            "scenario_no_reuse=NOT_RUN_HARDWARE",
+            "scenario_multi_turn=NOT_RUN_HARDWARE",
+            "scenario_hbm_within=NOT_RUN_HARDWARE",
+            "scenario_hbm_cliff=NOT_RUN_HARDWARE",
+            "samples=NONE",
+        ] {
+            assert!(report.contains(key), "missing {key}");
+        }
+        assert!(!report.contains("tokens/s"));
+        assert!(!report.contains("PASS"));
+    }
+}

crates/protocol/src/fuzz.rs

@@ -0,0 +1,110 @@
+// Copyright 2024-2026 RustFS, Inc.
+// SPDX-License-Identifier: Apache-2.0
+//! Protocol decoder fuzz. Random, truncated, duplicated, and reordered frames
+//! must not panic and must not decode as a different successful request.
+
+use gpucache_core_types::{
+    content_checksum, synthetic_identity, CacheError, RequestId, HARD_BYTES_CAP,
+};
+use proptest::prelude::*;
+
+use crate::{
+    decode_export, decode_hello, decode_hello_ack, decode_reply, decode_request, encode_export,
+    encode_hello, encode_request, AuthContext, CommittedExport, ItemHeader, LookupItem,
+    RequestBody, WIRE_VERSION,
+};
+
+fn sample_request() -> (AuthContext, RequestBody) {
+    let identity = synthetic_identity(1, 1);
+    let auth = AuthContext::new(
+        identity.tenant.clone(),
+        identity.trust_domain.clone(),
+        [9u8; 32],
+    )
+    .unwrap();
+    let body = RequestBody::Lookup(vec![LookupItem {
+        header: ItemHeader {
+            request_id: RequestId::from_counter(1).unwrap(),
+            epoch: 1,
+            deadline_ms: 50,
+        },
+        identity,
+    }]);
+    (auth, body)
+}
+
+fn sample_export() -> Vec<u8> {
+    let payload = b"synthetic-kv".to_vec();
+    encode_export(&CommittedExport {
+        identity: synthetic_identity(1, 1),
+        source_generation: 1,
+        checksum: content_checksum(&payload),
+        payload,
+    })
+    .unwrap()
+}
+
+#[test]
+fn truncated_duplicate_and_reordered_frames_fail_closed() {
+    let (auth, body) = sample_request();
+    let frame = encode_request(&auth, &body).unwrap();
+    let (decoded_auth, decoded_body) = decode_request(&frame).unwrap();
+    assert_eq!(decoded_auth, auth);
+    assert_eq!(decoded_body, body);
+    for len in 0..frame.len() {
+        assert!(
+            decode_request(&frame[..len]).is_err(),
+            "short frame of {len} bytes must not decode"
+        );
+    }
+    let mut duplicated = frame.clone();
+    duplicated.extend_from_slice(&frame);
+    assert!(matches!(
+        decode_request(&duplicated),
+        Err(CacheError::Malformed {
+            reason: "trailing bytes"
+        })
+    ));
+    let mut reordered = frame.clone();
+    let mid = reordered.len() / 2;
+    reordered.rotate_left(mid);
+    assert!(decode_request(&reordered).is_err());
+    let export = sample_export();
+    let mut corrupt = export.clone();
+    let last = corrupt.len() - 1;
+    corrupt[last] ^= 0x01;
+    assert!(matches!(
+        decode_export(&corrupt),
+        Err(CacheError::ChecksumMismatch)
+    ));
+    assert!(decode_export(&export[..export.len() / 2]).is_err());
+    let mut huge = Vec::from(&b"GCK1"[..]);
+    huge.extend_from_slice(&WIRE_VERSION.to_le_bytes());
+    huge.extend_from_slice(&1u16.to_le_bytes());
+    huge.extend_from_slice(&(HARD_BYTES_CAP as u32).saturating_add(1).to_le_bytes());
+    assert!(decode_request(&huge).is_err());
+    assert!(decode_reply(&frame).is_err());
+    assert!(decode_hello(&frame).is_err());
+    assert!(decode_hello_ack(&frame).is_err());
+    let hello = encode_hello(&[WIRE_VERSION]).unwrap();
+    assert_eq!(decode_hello(&hello).unwrap(), vec![WIRE_VERSION]);
+}
+
+proptest! {
+    #![proptest_config(ProptestConfig::with_cases(48))]
+    #[test]
+    fn random_frames_do_not_panic_and_roundtrip_when_accepted(
+        bytes in prop::collection::vec(any::<u8>(), 0..384)
+    ) {
+        if let Ok((auth, body)) = decode_request(&bytes) {
+            let again = encode_request(&auth, &body).expect("accepted request encodes");
+            let (auth2, body2) = decode_request(&again).expect("re-encoded request decodes");
+            assert_eq!(auth, auth2);
+            assert_eq!(body, body2);
+        }
+        let _ = decode_reply(&bytes);
+        let _ = decode_export(&bytes);
+        let _ = decode_hello(&bytes);
+        let _ = decode_hello_ack(&bytes);
+    }
+}

crates/protocol/src/lib.rs

@@ -13,6 +13,9 @@ mod codec;
 mod negotiate;
 mod reply;
 
+#[cfg(test)]
+mod fuzz;
+
 pub use api::{
     identity_for_prefix_hit, BatchReply, CommitItem, CommitOutcome, GetHit, GetItem,
     InvalidateItem, InvalidateOutcome, ItemHeader, ItemResult, LookupItem, LookupOutcome,