Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 8 additions & 2 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -20,10 +20,16 @@ jobs:
run: cargo clippy --workspace --all-targets -- -D warnings
- name: Test
run: cargo test --workspace --all-targets
- name: Reject accelerator crates in V1 A
- name: Reject accelerator crates in V1
run: |
if grep -RInE 'cuda|doca|nixl|rdma|spdk' crates/*/Cargo.toml; then
echo "V1 A crates must not depend on GPU or DPU stacks"
echo "V1 crates must not depend on GPU or DPU stacks"
exit 1
fi
- name: Keep the cold tier off the hot path
run: |
if grep -n 'gpucache-cold' crates/storage/Cargo.toml crates/transport/Cargo.toml crates/protocol/Cargo.toml crates/core-types/Cargo.toml; then
echo "RustFS cold tier must not be a hot-path dependency"
exit 1
fi

Expand Down
6 changes: 4 additions & 2 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ GPUCache is the G3.5 shared KV tier. RustFS is the colder G4 backend. Read [READ

## Phase

The active phase is **V1 milestone A** unless a human explicitly starts a later milestone. Do not implement V1 B, V2 RDMA, or V3 DPU work in a V1 A change. Do not publish crates, cut releases, deploy, or edit a website unless asked.
The active phase is **V1 milestone B**. Do not implement V1 C, V2 RDMA, or V3 DPU work in a V1 B change. Do not publish crates, cut releases, deploy, or edit a website unless asked.

## Honesty rules

Expand All @@ -21,8 +21,10 @@ The active phase is **V1 milestone A** unless a human explicitly starts a later
| `gpucache-core-types` | `sha2` only |
| `gpucache-protocol` | `gpucache-core-types` |
| `gpucache-storage` | the two crates above |
| `gpucache-transport` | the three crates above. TCP only |
| `gpucache-cold` | `gpucache-core-types` and `sha2`. Not a dependency of storage or transport |

No crate may add CUDA, DOCA, NIXL, RDMA, SPDK, erasure coding, or replica placement. `unsafe` is forbidden. Registered buffers whose completion is unknown stay quarantined until drain or a newer epoch; do not free them only because a lease timer fired.
No crate may add CUDA, DOCA, NIXL, RDMA, SPDK, erasure coding, or replica placement. `unsafe` is forbidden. Registered buffers whose completion is unknown stay quarantined until drain or a newer epoch; do not free them only because a lease timer fired. The cold crate stays off the hot path.

## Checks

Expand Down
7 changes: 4 additions & 3 deletions COMPATIBILITY.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Compatibility manifest

Recorded for V1 milestone A. This file is not a performance acceptance report and not a hardware inventory of a BlueField node.
Recorded for V1 milestones A and B. This file is not a performance acceptance report and not a hardware inventory of a BlueField node.

| Component | Value | Result |
| --- | --- | --- |
Expand All @@ -15,8 +15,9 @@ Recorded for V1 milestone A. This file is not a performance acceptance report an
| LMCache | Not a dependency | `NOT_RUN_HARDWARE` (route undecided) |
| NIXL / UCX / RDMA | Not a dependency | Not in V1 |
| DOCA / BlueField BFB / firmware | Not present | `NOT_RUN_HARDWARE` |
| RustFS server / client | No endpoint authorized | `NOT_RUN_HARDWARE` |
| Direct Rust deps | `sha2` 0.10.8; dev `proptest` 1.4.0 | Apache-2.0 OR MIT; see `Cargo.lock` |
| RustFS server / client | No endpoint authorized in this environment. Contract mock covers Put, Head, RangeGet, Delete, pack, and manifest | Live probe `NOT_RUN_HARDWARE`. Mock `PASS` |
| TCP / file store | Loopback TCP and a temp directory on the build host | `PASS` for `cargo test --workspace --all-targets` on x86_64. Not an NVMe format and not a DPU run |
| Direct Rust deps | `sha2` 0.10.8; dev `proptest` 1.4.0 | Apache-2.0 OR MIT; see `Cargo.lock`. No new direct crate in V1 B |
| License of this repo | Apache-2.0 | See `LICENSE` |

CI (see `.github/workflows/ci.yml`) builds and runs the same tests on x86_64 and on aarch64 (GitHub ARM runner, plus qemu-user cross execution). An aarch64 result is a CPU result. It does not certify BlueField-3, DOCA, or NVMe-on-DPU.
Expand Down
19 changes: 18 additions & 1 deletion Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 4 additions & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@ members = [
"crates/core-types",
"crates/protocol",
"crates/storage",
"crates/cold",
"crates/transport",
]

[workspace.package]
Expand All @@ -19,6 +21,8 @@ sha2 = "=0.10.8"
gpucache-core-types = { path = "crates/core-types" }
gpucache-protocol = { path = "crates/protocol" }
gpucache-storage = { path = "crates/storage" }
gpucache-cold = { path = "crates/cold" }
gpucache-transport = { path = "crates/transport" }

[workspace.lints.rust]
unsafe_code = "forbid"
Expand Down
37 changes: 29 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,15 +8,16 @@ GPUCache does not do erasure coding and does not keep durability replicas. A los

## Status

**V1 milestone A is implemented. DPU acceleration is not.**
**V1 milestones A and B are implemented. DPU acceleration is not.**

V1 runs the cache contract on CPU. The long-term deployment target is a storage-side BlueField-3, but this repository does not link DOCA, CUDA, RDMA, or SPDK, and it does not claim that any of those paths exist. An aarch64 build is a CPU build. It is not a BlueField functional test.

| Milestone | Status |
| --- | --- |
| V1 A: identity, protocol, `MemoryStore`, error/timeout/cancel tests | Implemented in this tree |
| V1 A: identity, protocol, `MemoryStore`, error/timeout/cancel tests | Implemented |
| V1 A: real vLLM KV block export/import | `NOT_RUN_HARDWARE` (no GPU in this environment) |
| V1 B: TCP service, file/NVMe, admission, RustFS smoke | Not started |
| V1 B: TCP service, file backend, LRU admission, tenant quotas, two-client reuse | Implemented in this tree |
| V1 B: RustFS live Put/Head/RangeGet/Delete | `NOT_RUN_HARDWARE` (no authorized endpoint). Contract mock passes |
| V1 C: fuzz, fault injection, engine output comparison, paired baselines | Not started |
| V2: NIC RDMA (NIXL or Mooncake) plus a real RustFS cold path | Not started; blocked on V1 |
| V3: BlueField-3 data-path offload and production hardening | Not started; not purchased and not measured. Not an erasure-coding milestone |
Expand All @@ -34,13 +35,23 @@ Three library crates, none of which depend on a GPU or DPU:

The batch API is Lookup, Reserve/Put, Commit, Get, Release/Cancel, and Invalidate. Every item carries `request_id`, epoch, deadline, and, where it applies, generation, length, and checksum. Only a `COMMITTED` block whose identity is compatible is a hit. `ABSENT`, `RESERVED`, `WRITING`, `EVICTING`, and `QUARANTINED` blocks are not readable.

## What V1 B adds

`gpucache-transport` serves that same API over TCP. The default bind is `127.0.0.1`. The process refuses to listen on a non-loopback address unless `--allow-remote` is set, and it still requires a tenant credential fingerprint. There is no unauthenticated data path.

`FileStore` keeps committed blocks as ordinary files under a directory you pass in. It does not format a disk and it does not open a device node. When a tenant or global byte quota would fail, it evicts the oldest unpinned committed block. A pinned or quarantined block stays. `MemoryStore` still fails admission instead of evicting, so the V1 A tests keep that behavior.

Lookup can return `PrefixHit` when a committed block's hash chain is a strict prefix of the query and the layout, dtype, shard, and model identity match. The hit is the whole stored block. `Get` must use `identity_for_prefix_hit`. A shorter `valid_length` on the same chain is an incompatible miss. The service does not slice inside a block.

`gpucache-cold` packs blocks into an object plus a manifest and can Put, Head, RangeGet, and Delete through a trait. The TCP hot path does not call it. Without `GPUCACHE_RUSTFS_*` variables the live probe is `NOT_RUN_HARDWARE`. The mock still runs.

Lifecycle:

```text
ABSENT -> RESERVED -> WRITING -> COMMITTED -> EVICTING -> ABSENT
```

A failed or timed-out registered region goes to `QUARANTINED`. Lease expiry does not prove that a copy has stopped, so that buffer stays allocated until the owner attests drain or a newer store epoch fences it. Same tokens do not imply the same bytes: the logical prefix key and the physical byte key are different encodings. V1 reuses only within one tenant, one model, and one layout. A reshard, quant, or layout mismatch is a miss. There is no cross-tenant dedup and no partial-prefix hit in this milestone.
A failed or timed-out registered region goes to `QUARANTINED`. Lease expiry does not prove that a copy has stopped, so that buffer stays allocated until the owner attests drain or a newer store epoch fences it. Same tokens do not imply the same bytes: the logical prefix key and the physical byte key are different encodings. V1 reuses only within one tenant, one model, and one layout. A reshard, quant, or layout mismatch is a miss. There is no cross-tenant dedup. A longer hash chain can reuse a whole shorter committed block (`PrefixHit`); the service does not slice inside a block. See [docs/adr/0003-partial-prefix.md](docs/adr/0003-partial-prefix.md).

Resource limits cover object size, in-flight requests, memory registrations, per-tenant in-flight bytes, and retained bytes. The idempotency table is bounded.

Expand All @@ -52,17 +63,27 @@ From a clean checkout, with Rust stable (MSRV 1.85):
cargo test --workspace --all-targets
```

That command is the CPU contract repro. It includes unit tests, property tests, timeout and cancel tests, a single-process import of a synthetic block, and a cross-process import through `kv-block-xfer`. The bytes in those tests are synthetic. They are not a vLLM KV tensor.
That command is the CPU contract repro for V1 A and V1 B. It includes unit tests, property tests, timeout and cancel tests, a single-process import of a synthetic block, a cross-process import through `kv-block-xfer`, file-backend quota and restart tests, and the TCP reuse tests. The bytes are synthetic. They are not a vLLM KV tensor.

Two localhost clients sharing one file-backed service:

```bash
cargo run -p gpucache-transport --bin share-demo
```

A successful run prints `status=REUSE_OK` and `partial_prefix=WHOLE_BLOCK_PREFIX`. It does not print a sliced-block hit.

`kv-block-xfer import <credential-hex>` reads one export frame on stdin and imports it into a fresh `MemoryStore`. It does not talk to an engine.

## Layout

```text
crates/core-types identity, keys, lifecycle, errors
crates/protocol batch messages and version negotiation
crates/storage MemoryStore and kv-block-xfer
docs/adr decision records (integration route, no erasure coding)
crates/protocol batch messages, replies, version negotiation
crates/storage MemoryStore, FileStore, kv-block-xfer
crates/transport TCP server, client, share-demo
crates/cold RustFS contract, pack/manifest, live probe
docs/adr decision records
COMPATIBILITY.md environment manifest
```

Expand Down
2 changes: 1 addition & 1 deletion THIRD_PARTY_NOTICES.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Third-party notices

GPUCache V1 A is Apache-2.0. See [LICENSE](LICENSE).
GPUCache V1 A and V1 B are Apache-2.0. See [LICENSE](LICENSE).

Direct dependencies:

Expand Down
16 changes: 16 additions & 0 deletions crates/cold/Cargo.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
[package]
name = "gpucache-cold"
version.workspace = true
edition.workspace = true
license.workspace = true
rust-version.workspace = true
authors.workspace = true
description = "Optional RustFS G4 cold-tier contract. Not on the GPUCache hot path."
publish = false

[dependencies]
gpucache-core-types.workspace = true
sha2.workspace = true

[lints]
workspace = true
Loading
Loading