Repository navigation
[Klaud Cold] minimaxm31flash-fp4-b200-sglang-agentic-dspark: day-zero MiniMax-M3.1-Flash B200 SGLang recipe - #4442
Conversation
…-Flash SGLang AgentX on B200 Serve flags follow the SGLang cookbook's MiniMax-M3.1 cells with the day-zero dev-minimax-m31-flash image. Seven-token DSpark with synthetic acceptance from the minimaxm31flash_dspark golden curve.
Oseltamivir
added
the
full-sweep-fail-fast
|
Thanks for the contribution!
中文感谢你的贡献！
|
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
| - "Serve flags follow the cookbook's B200 Balanced (TP4) and High-Throughput (TP8, attention DP2, EP8: DSpark with block verification, breakable prefill graphs, minimax-m3 reasoning and tool parsers, 8 tokenizer workers." | ||
| - "Seven-token DSpark (the bundled draft's dspark_block_size); throughput runs use synthetic acceptance from the minimaxm31flash_dspark golden curve (AL 4.04 at 7 draft tokens, #4425)." | ||
| - "Search space: Balanced TP4 at conc 1-32; TP8 + EP8 with attention DP2 at conc 64-128." | ||
| pr-link: TBD |
There was a problem hiding this comment.
🔴 The changelog check will fail this PR, and the merge helper will refuse it, because the new entry's pr-link: TBD is not an accepted placeholder. validate_added_pr_link accepts only the canonical link for this PR or one of PR_LINK_PLACEHOLDERS (XXX, https://github.com/SemiAnalysisAI/InferenceX/pull/XXX). Fix: set pr-link to https://github.com/SemiAnalysisAI/InferenceX/pull/<this PR number>, or to XXX so the merge helper fills it in.
Why this was flagged
The new block at inferencex-e2e/perf-changelog.yaml:11018 has pr-link: TBD. On a PR run, infx/workflows/validate_perf_changelog.py:147 raises new PR entry must use 'https://github.com/SemiAnalysisAI/InferenceX/pull/N' or an XXX placeholder; found 'TBD'. That check runs for every appended entry (line 193). Separately, infx/workflows/prepare_perf_changelog_merge.py:108-111 raises appended entry N has unexpected pr-link 'TBD', so merge_with_reuse cannot canonicalize the link. The changelog entry is what triggers result collection, so the new config key's sweep cannot be collected or merged until this is fixed. No existing entry in the file uses TBD. The placeholder set is XXX and https://github.com/SemiAnalysisAI/InferenceX/pull/XXX.
Verification: The appended block at the tail of inferencex-e2e/perf-changelog.yaml has pr-link: TBD. In inferencex-e2e/infx/workflows/validate_perf_changelog.py, lines 147-150 (validate_added_pr_link) raise when the link is neither a placeholder nor the expected link. The merge helper also refuses to rewrite the link. The validator is wired into CI at .github/workflows/run-sweep.yml:106.
| gpus: 8 | ||
| args: | ||
| tensor-parallel-size: 8 | ||
| attn-dp-size: 2 | ||
| enable-dp-lm-head: true | ||
| expert-parallel-size: 8 | ||
| chunked-prefill-size: 65536 | ||
| max-running-requests: 128 | ||
| cuda-graph-max-bs-decode: 64 | ||
| cuda-graph-max-bs-prefill: 16384 | ||
| mem-fraction-static: 0.75 | ||
| env: | ||
| SGLANG_OPT_DEEPGEMM_MEGA_MOE_NUM_MAX_TOKENS_PER_RANK: '32768' | ||
| benchmark: | ||
| env: | ||
| CONC: '64' | ||
| KV_OFFLOADING: none | ||
|
|
||
| override_tp8_ep8_c128: |
There was a problem hiding this comment.
🔴 The TP8 conc 64 and 128 results will not measure the DeepGEMM MegaMoE path that the recipe comment and PR description advertise. Both TP8 overrides set only SGLANG_OPT_DEEPGEMM_MEGA_MOE_NUM_MAX_TOKENS_PER_RANK, which caps per-rank tokens. Neither passes the flag that selects the MegaMoE backend, so SGLang serves EP8 with its default MoE path. Fix: add moe-a2a-backend: megamoe to both TP8 overrides (override_tp8_ep8_c64 and override_tp8_ep8_c128), and keep the per-rank token cap of at least chunked-prefill-size / attn-dp-size (32768). Otherwise, drop the MegaMoE claims.
Why this was flagged
Both TP8 matrix points ({ tp: 8, ep: 8, ... conc-list: [64, 128] } at nvidia-master.yaml:5620) bind to override_tp8_ep8_c64 / override_tp8_ep8_c128 (agentic.yaml:136-173). Those overrides set expert-parallel-size: 8 and the env SGLANG_OPT_DEEPGEMM_MEGA_MOE_NUM_MAX_TOKENS_PER_RANK: '32768'. Neither sets moe-a2a-backend or moe-runner-backend. Other SGLang recipes in this repo choose MegaMoE with --moe-a2a-backend megamoe (perf-changelog.yaml:3056, 3117, 3218, 6336, 6438). There, the NUM_MAX_TOKENS_PER_RANK variable only sizes the MegaMoE workspace, and the backend sets its companion env vars itself (perf-changelog.yaml:3221). So the c64/c128 runs fall back to SGLang's default EP MoE path. The published numbers then do not reflect the configuration described in perf-changelog.yaml:11014 and the master config comment. Because this is a new config, no base-branch result exists for these points to compare against.
Verification: In the new recipe, override_tp8_ep8_c64 and override_tp8_ep8_c128 (agentic.yaml:134-173) only set SGLANG_OPT_DEEPGEMM_MEGA_MOE_NUM_MAX_TOKENS_PER_RANK: '32768'. Neither override sets moe-a2a-backend or moe-runner-backend, and the base args don't either. Every other SGLang recipe in this repo pairs that env cap with moe-a2a-backend: megamoe. So the TP8 cells serve EP8 on SGLang's default MoE path.
| # MiniMax-M3.1-Flash AgentX on B200 with SGLang: day-zero image from the SGLang | ||
| # cookbook (https://docs.sglang.io/cookbook/autoregressive/MiniMax/MiniMax-M3.1), | ||
| # manifest digest verified 2026-10-24. Serve flags follow the cookbook's | ||
| # B200 Balanced (TP4) and High-Throughput (TP8, attention DP2, EP8; mem 0.75, prefill graphs to 16384). | ||
| # Seven-token DSpark (bundled draft); synthetic acceptance follows | ||
| # infx/golden_al_distribution/minimaxm31flash_dspark.yaml. |
There was a problem hiding this comment.
🟡 nit (optional): new header and provenance comments break the AGENTS.md comment rules, which forbid header comments and history. sweep:cookbook|digest verified|# Balanced|# High-Throughput|# TP4 Balanced Examples: the six-line header above minimaxm31flash-fp4-b200-sglang-agentic-dspark (nvidia-master.yaml:5598-5603) says where the flags came from and when the digest was verified. The recipe's "Serve flags follow the cookbook..." comment (agentic.yaml:24-25) does the same. Fix: delete header, section and provenance comments, keeping only short per-line technical why comments.
Why this was flagged
AGENTS.md "Code comments (mandatory)" bans header comments above a recipe, override or section. It also bans comments about where code came from. The diff adds a section header above the master entry (nvidia-master.yaml:5598-5603, including "manifest digest verified 2026-10-24"). It adds section headers above the override groups in agentic.yaml (lines 51-52 and 130-132). It adds a cookbook-provenance comment above args (agentic.yaml:24-25), plus "# Balanced: TP4." and "# High-Throughput: ..." labels in the search-space (nvidia-master.yaml:5617, 5619-5620). On the base branch these comments do not exist. Maintainers would inherit comments the repo rules say must be removed.
Verification: Nit. The diff breaks the repository's mandatory comment rules every time it is merged as written. AGENTS.md:60-67 says: "No header comments." It also says: "No history." nvidia-master.yaml:5598-5603 is a six-line header above minimaxm31flash-fp4-b200-sglang-agentic-dspark. Nothing breaks at runtime because these are YAML comments.
|
View unofficial run (performance): https://inferencex.semianalysis.com/inference?unofficialRun=40072566487 View unofficial run (accuracy): https://inferencex.semianalysis.com/evaluation?unofficialRun=40072566487 |
Keep main's entries and re-add the B200 SGLang changelog and master-config blocks.
Summary
Day-zero MiniMax-M3.1-Flash (
MiniMaxAI/MiniMax-M3.1-Flash, NVFP4) AgentX on B200 with SGLang, following the SGLang cookbook: https://docs.sglang.io/cookbook/autoregressive/MiniMax/MiniMax-M3.1 (config source:docs/src/snippets/configs/MiniMaxAI/minimax-m3.1.jsx).lmsysorg/sglang:dev-minimax-m31-flash, pinned by digest (verified on Docker Hub 2026-10-24).--attn-dp-size 2, EP8 MegaMoE,--mem-fraction-static 0.75,--cuda-graph-max-bs-prefill 16384) for conc 64–128. The matrix row istp: 8, ep: 8with dp-attn false, because the matrix's dp-attn means full DP attention.--speculative-algorithm DSPARK --speculative-use-block-verification,--cuda-graph-backend-prefill breakable,minimax-m3reasoning and tool parsers,--tokenizer-worker-num 8.--speculative-dspark-block-size 7, the bundled draft'sdspark_block_size). It's set explicitly so throughput runs resolve synthetic acceptance fromminimaxm31flash_dspark.yaml([Klaud Cold] Add MiniMax-M3.1-Flash DSpark golden AL curve and SPEED-Bench collector / [Klaud Cold] 新增 MiniMax-M3.1-Flash DSpark 黄金 AL 曲线与 SPEED-Bench 采集脚本 #4425): AL 4.04 at 7 tokens, thinking on (adaptive, the default).infx generatebinds every matrix point to exactly one recipe variant, withSGLANG_SIMULATE_ACC_LEN=4.04.vLLM siblings: #4419 (GB300), #4420 (MI355X), #4421 (B300), #4422 (H100), #4423 (H200), #4426/#4431 (B200), #4433 (GB200).