Skip to content

[Klaud Cold] Update GLM-5.2 FP4 GB300 Dynamo-SGLang AgentX disagg image to nightly-dev-20261024 / [Klaud Cold] 将 GLM-5.2 FP4 GB300 Dynamo-SGLang AgentX 分离式镜像更新至 nightly-dev-20261024 - #4462

Closed
Klaud-Cold wants to merge 3 commits into
mainfrom
klaud/auto-68551d95238169ff-225fe14aaa75010c
Closed

Klaud-Cold wants to merge 3 commits into
mainfrom
klaud/auto-68551d95238169ff-225fe14aaa75010c

Conversation

@Klaud-Cold

@Klaud-Cold Klaud-Cold commented Oct 24, 2026 •

Copy link
Copy Markdown
Collaborator

Goal: Update SGLang image from lmsysorg/sglang:nightly-dev-20261016-a93f16f0@sha256:0b3e39704480c423d8a00ca15ebee63154defc62944df2d4b11c594563bab457 to lmsysorg/sglang:nightly-dev-20261024-5994c559@sha256:40da9cac78a558237426971c048ab0f4f1a4b13bfb1b6e7a48d4d980e4163c8e.
Baseline: 2026-10-20 · lmsysorg/sglang:nightly-dev-20261016-a93f16f0@sha256:0b3e39704480c423d8a00ca15ebee63154defc62944df2d4b11c594563bab457
Mean latency · Sources: API 1, API 2, API 3, API 4

Point Total tok/s/GPU ↑ Output tok/s/GPU ↑ TTFT ms ↓ TPOT ms ↓
AgentX c30 PTP1x1/DTP4x5 8af31b 8,942.33 54.17 1,027.7 5.43
AgentX c40 PTP1x1/DTP4x3 cc56b9 16,924.23 109.05 1,338.55 6.2
AgentX c56 PTP1x1/DTP8x1 ad6dbf 28,406.53 202.53 1,700.22 8.81
AgentX c216 PTP1x3/DTP8x1 5e56d2 55,487.87 392.99 3,148.97 10.78
AgentX c256 PTP1x4/DTP16x1 9c93a3 44,452.67 313.82 2,417.7 8.81
AgentX c264 PTP1x3/DTP8x1 b3a4aa 63,647.21 458.15 4,336.51 11.07
Eval Score ↑ Samples
gsm8k/em_strict · c30 96.97% 1,319
gsm8k/em_strict · c40 96.51% 1,319
gsm8k/em_strict · c56 96.89% 1,319
gsm8k/em_strict · c216 96.06% 1,319
gsm8k/em_strict · c256 96.51% 1,319
gsm8k/em_strict · c264 97.27% 1,319

Stop: Add the klaud-abort label at any time to stop this candidate now; Klaud cancels its runs, closes this PR and does not retry it. Remove the label to let the planner pick it up again.

中文

**目标:**将 SGLang 镜像从 lmsysorg/sglang:nightly-dev-20261016-a93f16f0@sha256:0b3e39704480c423d8a00ca15ebee63154defc62944df2d4b11c594563bab457 更新至 lmsysorg/sglang:nightly-dev-20261024-5994c559@sha256:40da9cac78a558237426971c048ab0f4f1a4b13bfb1b6e7a48d4d980e4163c8e。
**基线:**2026-10-20 · lmsysorg/sglang:nightly-dev-20261016-a93f16f0@sha256:0b3e39704480c423d8a00ca15ebee63154defc62944df2d4b11c594563bab457
平均延迟 · 来源: API 1, API 2, API 3, API 4;数值及异常说明见上表。
**停止:**随时添加 klaud-abort 标签即可立即停止该候选:Klaud 会取消其运行、关闭此 PR 且不再重试。移除该标签后,规划器会在下次扫描时重新考虑它。

…htly-dev-20261024-5994c559

将 GLM-5.2 FP4 GB300 Dynamo-SGLang 分离式 AgentX 镜像更新至 nightly-dev-20261024-5994c559,并同步 SGLang 身份版本。

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown
Contributor

Thanks for the contribution!

  • Review: If this PR changes files owned by someone other than a repository admin or @SemiAnalysisAI/core, ask one eligible CODEOWNER to complete the latest PR_REVIEW_CHECKLIST.md before contacting a core maintainer on Slack. Follow the template exactly, including As a PR reviewer and CODEOWNER, I have reviewed this and have, so sign-off verification triggers.
  • PR verification: Sweeps run only when the PR appends an inferencex-e2e/perf-changelog.yaml entry and carries exactly one primary label: full-sweep-fail-fast (strongly recommended; canary plus per-matrix fail-fast), full-sweep-enabled (canary; matrix jobs continue after a failure), or non-canary-full-sweep-enabled (no canary or fail-fast). The modifiers all-evals, evals-only, and agentx-fast require a primary label. On fork PRs, a maintainer applies the label. See sweep labels and reuse.
  • After merging: PR authors must ensure all GitHub Actions jobs pass. Transient failures often pass on rerun; see how to rerun failed jobs.
中文

感谢你的贡献!

  • **审阅:**如果 PR 修改的文件归属于仓库管理员及 @SemiAnalysisAI/core 之外的 CODEOWNER,请先联系一位有资格的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md,再通过 Slack 联系核心维护者。必须严格遵循模板,并保留 As a PR reviewer and CODEOWNER, I have reviewed this and have,才能触发签核验证。
  • **PR 验证:**只有当 PR 在 inferencex-e2e/perf-changelog.yaml 末尾追加了条目,并且恰好带有一个主标签时,才会运行扫描:full-sweep-fail-fast(强烈推荐;canary 加逐矩阵 fail-fast)、full-sweep-enabled(有 canary;矩阵任务在失败后继续运行)或 non-canary-full-sweep-enabled(无 canary,也无 fail-fast)。修饰标签 all-evals、evals-only 和 agentx-fast 必须与主标签一起使用。fork PR 的标签由维护者添加。参见扫描标签与复用。
  • **合并后:**PR 作者必须确保所有 GitHub Actions 任务通过。临时性失败通常可以通过重新运行恢复;参见重新运行失败任务的说明。

@Klaud-Cold

Klaud-Cold commented Oct 24, 2026 •

Copy link
Copy Markdown
Collaborator Author

Initial attempt · Passed · Run 40071572624 / attempt 1 · 2026-10-24 10:55 UTC
lmsysorg/sglang:nightly-dev-20261024-5994c559@sha256:40da9cac78a558237426971c048ab0f4f1a4b13bfb1b6e7a48d4d980e4163c8e · 601e531e49a8 · Mean latency
Change: Measured SHA 15b835a: pin SGLang nightly-dev-20261024-5994c559 by digest and set identity sglang 0.0.0.dev1+g5994c5597; source diff c15edf314..5994c5597 (sgl-project/sglang@c15edf3...5994c55) keeps every recipe flag/env, bumps flashinfer 0.7.1 to 0.7.2 and sgl-deep-gemm to 0.2.3.post2; Dynamo stays on the mounted 1.6.0.dev20261016 wheel, whose compatibility with the new SGLang is unverified until smoke.

Point Output tok/s/GPU ↑ TTFT ms ↓ TPOT ms ↓
AgentX c30 PTP1x1/DTP4x5 8af31b 54 (-0.3%) 1,085.86 (+5.7%) 5.13 (-5.5%)
AgentX c40 PTP1x1/DTP4x3 cc56b9 108.84 (-0.2%) 1,402.05 (+4.7%) 6.26 (+1%)
AgentX c56 PTP1x1/DTP8x1 ad6dbf 202.77 (+0.1%) 1,759.54 (+3.5%) 8.5 (-3.5%)
AgentX c216 PTP1x3/DTP8x1 5e56d2 393.65 (+0.2%) 3,403.03 (+8.1%) 10.57 (-1.9%)
AgentX c256 PTP1x4/DTP16x1 9c93a3 313.21 (-0.2%) 2,422.15 (+0.2%) 8.72 (-1%)
AgentX c264 PTP1x3/DTP8x1 b3a4aa 446.88 (-2.5%) 5,393.62 (+24.4%) 10.9 (-1.5%)
Eval Score ↑ Samples
gsm8k/em_strict · c30 97.42% (N/A) 1,319
gsm8k/em_strict · c40 96.82% (N/A) 1,319
gsm8k/em_strict · c56 96.89% (N/A) 1,319
gsm8k/em_strict · c216 96.36% (N/A) 1,319
gsm8k/em_strict · c256 96.89% (N/A) 1,319
gsm8k/em_strict · c264 96.59% (N/A) 1,319

Note: All rows: Δ N/A: no matched eval baseline.

Next: Smoke benchmarks and gsm8k evals passed on every point; append the perf-changelog entry and run the full-sweep-fail-fast final sweep.

中文

初次尝试 · 已通过 · Run 40071572624 / attempt 1 · 2026-10-24 10:55 UTC
lmsysorg/sglang:nightly-dev-20261024-5994c559@sha256:40da9cac78a558237426971c048ab0f4f1a4b13bfb1b6e7a48d4d980e4163c8e · 601e531e49a8 · 平均延迟
**变更:**实测 SHA 15b835a:按摘要固定 SGLang nightly-dev-20261024-5994c559,并将 identity sglang 设为 0.0.0.dev1+g5994c5597;源码对比 c15edf314..5994c5597(sgl-project/sglang@c15edf3...5994c5597)保留所有配方参数与环境变量,flashinfer 从 0.7.1 升至 0.7.2,sgl-deep-gemm 升至 0.2.3.post2;Dynamo 仍使用挂载的 1.6.0.dev20261016 wheel,其与新 SGLang 的兼容性需经 smoke 验证。实测数值及异常说明见上表。
**下一步:**所有点位的 smoke 基准测试与 gsm8k 评测均已通过;追加 perf-changelog 条目并运行 full-sweep-fail-fast 最终扫描。

…e update

为 GLM-5.2 GB300 Dynamo-SGLang 分离式镜像更新追加 perf-changelog 条目。

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Klaud-Cold Klaud-Cold added the full-sweep-fail-fast Full sweep with canary gate; first failure cancels the rest of that matrix (recommended) label Oct 24, 2026
@Klaud-Cold

Klaud-Cold commented Oct 24, 2026 •

Copy link
Copy Markdown
Collaborator Author

Final full sweep · Cancelled · Run 40087863592 / attempt 1 · 2026-10-24 12:32 UTC
lmsysorg/sglang:nightly-dev-20261024-5994c559@sha256:40da9cac78a558237426971c048ab0f4f1a4b13bfb1b6e7a48d4d980e4163c8e · 3c5c2addae59 · Mean latency
Change: Append the perf-changelog entry; check-final confirmed the final matrix covers all 6 frozen baseline points and default evals.

Point Output tok/s/GPU ↑ TTFT ms ↓ TPOT ms ↓
AgentX c30 PTP1x1/DTP4x5 8af31b N/A N/A N/A
AgentX c40 PTP1x1/DTP4x3 cc56b9 N/A N/A N/A
AgentX c56 PTP1x1/DTP8x1 ad6dbf N/A N/A N/A
AgentX c216 PTP1x3/DTP8x1 5e56d2 N/A N/A N/A
AgentX c256 PTP1x4/DTP16x1 9c93a3 N/A N/A N/A
AgentX c264 PTP1x3/DTP8x1 b3a4aa N/A N/A N/A

Note: All rows: cancelled.
Note: All rows: request errors unavailable.
Note: All rows: Δ N/A: point unavailable.

Next: Main appended perf-changelog entries, so main was merged and this sweep was superseded by final sweep 40093995883 on 62a07d0.

中文

最终完整 sweep · 已取消 · Run 40087863592 / attempt 1 · 2026-10-24 12:32 UTC
lmsysorg/sglang:nightly-dev-20261024-5994c559@sha256:40da9cac78a558237426971c048ab0f4f1a4b13bfb1b6e7a48d4d980e4163c8e · 3c5c2addae59 · 平均延迟
**变更:**追加 perf-changelog 条目;check-final 已确认最终矩阵覆盖全部 6 个冻结基线点位及默认评测。实测数值及异常说明见上表。
**下一步:**main 追加了 perf-changelog 条目,因此已合并 main,本次扫描由 62a07d0 上的最终扫描 40093995883 取代。

@Klaud-Cold

Copy link
Copy Markdown
Collaborator Author

resumed · The session ended before finishing; session 2 continues it in a new run of its planner.

中文

resumed · 会话未完成即已结束;第 2 个会话将在其规划运行的新运行中继续。

…8169ff-225fe14aaa75010c

# Conflicts:
#	inferencex-e2e/perf-changelog.yaml
@Klaud-Cold

Klaud-Cold commented Oct 24, 2026 •

Copy link
Copy Markdown
Collaborator Author

Final full sweep · Running · Run 40093995883 / attempt 1 · 2026-10-24 13:32 UTC
lmsysorg/sglang:nightly-dev-20261024-5994c559@sha256:40da9cac78a558237426971c048ab0f4f1a4b13bfb1b6e7a48d4d980e4163c8e · 62a07d05c846 · Mean latency
Change: Merge main to keep the perf-changelog entry at the physical tail after main appended entries; check-final confirmed the final matrix covers all 6 frozen baseline points and default evals.

Point Output tok/s/GPU ↑ TTFT ms ↓ TPOT ms ↓
AgentX c30 PTP1x1/DTP4x5 8af31b N/A N/A N/A
AgentX c40 PTP1x1/DTP4x3 cc56b9 N/A N/A N/A
AgentX c56 PTP1x1/DTP8x1 ad6dbf N/A N/A N/A
AgentX c216 PTP1x3/DTP8x1 5e56d2 N/A N/A N/A
AgentX c256 PTP1x4/DTP16x1 9c93a3 N/A N/A N/A
AgentX c264 PTP1x3/DTP8x1 b3a4aa N/A N/A N/A

Note: All rows: pending.
Note: All rows: request errors unavailable.
Note: All rows: Δ N/A: point unavailable.

Next: Wait for final sweep 40093995883; its canary has been running the c30 benchmark for about 45 minutes.

中文

最终完整 sweep · 运行中 · Run 40093995883 / attempt 1 · 2026-10-24 13:32 UTC
lmsysorg/sglang:nightly-dev-20261024-5994c559@sha256:40da9cac78a558237426971c048ab0f4f1a4b13bfb1b6e7a48d4d980e4163c8e · 62a07d05c846 · 平均延迟
**变更:**在 main 追加条目后合并 main,使 perf-changelog 条目保持在文件末尾;check-final 已确认最终矩阵覆盖全部 6 个冻结基线点位及默认评测。实测数值及异常说明见上表。
**下一步:**等待最终扫描 40093995883;其 canary 已运行 c30 基准约 45 分钟。

@github-actions

github-actions Bot commented Oct 24, 2026 •

Copy link
Copy Markdown
Contributor

@Klaud-Cold

Klaud-Cold commented Oct 24, 2026 •

Copy link
Copy Markdown
Collaborator Author

Final full sweep · Failed · Run 40093995883 / attempt 2 · 2026-10-24 17:15 UTC
lmsysorg/sglang:nightly-dev-20261024-5994c559@sha256:40da9cac78a558237426971c048ab0f4f1a4b13bfb1b6e7a48d4d980e4163c8e · 62a07d05c846 · Mean latency
Change: Merge main to keep the perf-changelog entry at the physical tail after main appended entries; check-final confirmed the final matrix covers all 6 frozen baseline points and default evals.

Point Output tok/s/GPU ↑ TTFT ms ↓ TPOT ms ↓
AgentX c30 PTP1x1/DTP4x5 8af31b 54.01 (-0.3%) 1,033.81 (+0.6%) 5.36 (-1.3%)
AgentX c40 PTP1x1/DTP4x3 cc56b9 N/A N/A N/A
AgentX c56 PTP1x1/DTP8x1 ad6dbf N/A N/A N/A
AgentX c216 PTP1x3/DTP8x1 5e56d2 N/A N/A N/A
AgentX c256 PTP1x4/DTP16x1 9c93a3 N/A N/A N/A
AgentX c264 PTP1x3/DTP8x1 b3a4aa N/A N/A N/A

Note: AgentX c40 PTP1x1/DTP4x3 cc56b9, AgentX c56 PTP1x1/DTP8x1 ad6dbf, AgentX c216 PTP1x3/DTP8x1 5e56d2, AgentX c264 PTP1x3/DTP8x1 b3a4aa: cancelled.
Note: AgentX c40 PTP1x1/DTP4x3 cc56b9, AgentX c56 PTP1x1/DTP8x1 ad6dbf, AgentX c216 PTP1x3/DTP8x1 5e56d2, AgentX c264 PTP1x3/DTP8x1 b3a4aa: request errors unavailable.
Note: AgentX c40 PTP1x1/DTP4x3 cc56b9, AgentX c56 PTP1x1/DTP8x1 ad6dbf, AgentX c216 PTP1x3/DTP8x1 5e56d2, AgentX c256 PTP1x4/DTP16x1 9c93a3, AgentX c264 PTP1x3/DTP8x1 b3a4aa: Δ N/A: point unavailable.
Note: AgentX c256 PTP1x4/DTP16x1 9c93a3: failed.
Note: AgentX c256 PTP1x4/DTP16x1 9c93a3: 113 request errors.

Eval Score ↑ Samples
gsm8k/em_strict · c30 96.82% (N/A) 1,319
gsm8k/em_strict · c40 97.12% (N/A) 1,319
gsm8k/em_strict · c56 96.82% (N/A) 1,319
gsm8k/em_strict · c216 96.89% (N/A) 1,319
gsm8k/em_strict · c256 96.21% (N/A) 1,319
gsm8k/em_strict · c264 96.51% (N/A) 1,319

Note: All rows: Δ N/A: no matched eval baseline.

Next: c256 failed in AgentX warmup when a NIXL KV send hit NIXL_ERR_REMOTE_DISCONNECT, then prefill worker 3 hit a UCX cuStreamCreate OOM; the same point passed smoke on this image with 0 errors, so rerun the failed and cancelled jobs as infrastructure retry 1.

中文

最终完整 sweep · 失败 · Run 40093995883 / attempt 2 · 2026-10-24 17:15 UTC
lmsysorg/sglang:nightly-dev-20261024-5994c559@sha256:40da9cac78a558237426971c048ab0f4f1a4b13bfb1b6e7a48d4d980e4163c8e · 62a07d05c846 · 平均延迟
**变更:**在 main 追加条目后合并 main,使 perf-changelog 条目保持在文件末尾;check-final 已确认最终矩阵覆盖全部 6 个冻结基线点位及默认评测。实测数值及异常说明见上表。
**下一步:**c256 在 AgentX 预热阶段失败:NIXL KV 发送遇到 NIXL_ERR_REMOTE_DISCONNECT,随后 prefill worker 3 出现 UCX cuStreamCreate 显存不足;同一点位在该镜像的 smoke 中 0 错误通过,因此作为第 1 次基础设施重试重跑失败和被取消的任务。

@Klaud-Cold

Klaud-Cold commented Oct 24, 2026 •

Copy link
Copy Markdown
Collaborator Author

Infrastructure retry 1/2 · Failed · Run 40093995883 / attempt 3 · 2026-10-24 19:16 UTC
lmsysorg/sglang:nightly-dev-20261024-5994c559@sha256:40da9cac78a558237426971c048ab0f4f1a4b13bfb1b6e7a48d4d980e4163c8e · 62a07d05c846 · Mean latency
Change: No change: rerun the failed c256 job and the four fail-fast-cancelled benchmarks on the same head, reusing the passed c30 benchmark and all six evals from attempt 2.

Point Output tok/s/GPU ↑ TTFT ms ↓ TPOT ms ↓
AgentX c30 PTP1x1/DTP4x5 8af31b 54.01 (-0.3%) 1,033.81 (+0.6%) 5.36 (-1.3%)
AgentX c40 PTP1x1/DTP4x3 cc56b9 109.05 (+0%) 1,348.82 (+0.8%) 6.19 (-0.2%)
AgentX c56 PTP1x1/DTP8x1 ad6dbf 202.14 (-0.2%) 1,845.65 (+8.6%) 8.76 (-0.6%)
AgentX c216 PTP1x3/DTP8x1 5e56d2 396.51 (+0.9%) 2,976.18 (-5.5%) 10.46 (-3%)
AgentX c256 PTP1x4/DTP16x1 9c93a3 314.77 (+0.3%) 2,419.81 (+0.1%) 8.7 (-1.2%)
AgentX c264 PTP1x3/DTP8x1 b3a4aa 451.01 (-1.6%) 5,184.64 (+19.6%) 10.95 (-1.1%)
Eval Score ↑ Samples
gsm8k/em_strict · c30 96.82% (N/A) 1,319
gsm8k/em_strict · c40 97.12% (N/A) 1,319
gsm8k/em_strict · c56 96.82% (N/A) 1,319
gsm8k/em_strict · c216 96.89% (N/A) 1,319
gsm8k/em_strict · c256 96.21% (N/A) 1,319
gsm8k/em_strict · c264 96.51% (N/A) 1,319

Note: All rows: Δ N/A: no matched eval baseline.

Next: Stopped: run 40093995883 passed all 6 benchmarks and gsm8k evals, but finish cannot verify evals because the shared collect-evals row builder (infx/results/evals.py build_row) drops prefill_pp, so aggregate rows show prefill PP1 while raw rows and the matrix show PP4; that is deterministic shared tooling outside this candidate's allowed edits, so reruns cannot fix it.

中文

基础设施重试 1/2 · 失败 · Run 40093995883 / attempt 3 · 2026-10-24 19:16 UTC
lmsysorg/sglang:nightly-dev-20261024-5994c559@sha256:40da9cac78a558237426971c048ab0f4f1a4b13bfb1b6e7a48d4d980e4163c8e · 62a07d05c846 · 平均延迟
**变更:**无变更:在同一 head 上重跑失败的 c256 任务及四个被 fail-fast 取消的基准,复用第 2 次运行中已通过的 c30 基准及全部六项评测。实测数值及异常说明见上表。
**下一步:**已停止:运行 40093995883 的 6 个基准与 gsm8k 评测全部通过,但 finish 无法校验评测:共享的 collect-evals 行构建器(infx/results/evals.py build_row)丢弃了 prefill_pp,导致聚合行显示 prefill PP1,而原始行与矩阵为 PP4;这是确定性的共享工具问题,不在本候选允许修改的范围内,重跑无法修复。

@Klaud-Cold

Copy link
Copy Markdown
Collaborator Author

resumed · The session ended before finishing; session 3 continues it in a new run of its planner.

中文

resumed · 会话未完成即已结束;第 3 个会话将在其规划运行的新运行中继续。

@Klaud-Cold

Copy link
Copy Markdown
Collaborator Author

failed · Cleanup pending. Stop and confirm owned runs before closing.

中文

failed · 清理待完成。先停止并确认自有运行结束,再关闭 PR。

@Klaud-Cold Klaud-Cold removed the full-sweep-fail-fast Full sweep with canary gate; first failure cancels the rest of that matrix (recommended) label Oct 24, 2026
@Klaud-Cold Klaud-Cold closed this Oct 24, 2026
@Klaud-Cold
Klaud-Cold deleted the klaud/auto-68551d95238169ff-225fe14aaa75010c branch October 24, 2026 19:16
@Klaud-Cold

Copy link
Copy Markdown
Collaborator Author

failed · Repairs: 0 · Runs: 40071572624, 40087828484, 40087863592, 40093995883, 40124546566
Reason: Stopped: run 40093995883 passed all 6 benchmarks and gsm8k evals, but finish cannot verify evals because the shared collect-evals row builder (infx/results/evals.py build_row) drops prefill_pp, so aggregate rows show prefill PP1 while raw rows and the matrix show PP4; that is deterministic shared tooling outside this candidate's allowed edits, so reruns cannot fix it.
All owned runs ended. PR closed; branch deleted for retry.

中文

failed · 修复次数:0 · 运行:40071572624, 40087828484, 40087863592, 40093995883, 40124546566
**原因:**已停止:运行 40093995883 的 6 个基准与 gsm8k 评测全部通过,但 finish 无法校验评测:共享的 collect-evals 行构建器(infx/results/evals.py build_row)丢弃了 prefill_pp,导致聚合行显示 prefill PP1,而原始行与矩阵为 PP4;这是确定性的共享工具问题,不在本候选允许修改的范围内,重跑无法修复。
所有自有运行均已结束。PR 已关闭;分支已删除,可重新尝试。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

1 participant