From 829b1db2766f1a1201f154b7f5f3af341b1071d2 Mon Sep 17 00:00:00 2001 From: antianqi Date: Wed, 21 Oct 2026 16:01:33 +0800 Subject: [PATCH 01/15] feat(jianying): add the jianying Plugin (8 MCP verbs + 6 Skills) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Adds `plugins/antianqi/jianying`: a Windows-only MiniMax Code Plugin that grades video with a pixel-for-pixel re-implementation of Jianying's own grading math, and edits Jianying drafts through the application's local named pipe. The Plugin ships two runtime layouts in parallel: - `plugin.json` + `mcp.json` — the portable Agent Plugins 1.0 shape read by mcode 0.3.x. - `.claude-plugin/plugin.json` — the mcode 0.4.0+ manifest, with the six Skill paths and the stdio MCP server. MCP layer (8 verbs, `mcp/server.mjs`, zero npm dependencies): `draft_open`, `draft_read`, `draft_patch`, `draft_commit`, `render_frames`, `grade_apply`, `audio_mix`, `verify_diff`. Every verb has a hand-written input schema under `mcp/tools/`; unknown keys and out-of-range values are rejected before the call leaves the plugin. Skill layer (6 Skills): `jianying-cut` (grade and export), `jianying-draft-edit` (edit a draft through the 27 official behaviour operations), `jianying-review` (multi-dimension review), `jianying-talk-cut` (cut a spoken take), plus two library Skills — `jianying-verify` and `jianying-grade-chain` — that only activate when another Skill references them. Shared knowledge lives in `references/` (chain API semantics, verification discipline, pitfalls, machine-readable capability dictionary) and the deterministic guardrails in `scripts/`. Why not use the official exporter: the official CLI is blocked by `-100007` (session mode) and `-100015` (unauthorized host process), and its command tree is five days older than the application build. The Plugin re-implements the grading math instead (13/13 features reproduced to <= 1 LSB) and says so in the README rather than working around the block. Design compliance - Both manifests declare `name: jianying`, matching the Plugin directory, and license `Apache-2.0` with an Apache-2.0 `LICENSE` file. - The v0.4 manifest lists all six Skills as `./skills//SKILL.md`. Every path resolves, and every frontmatter `name` equals its directory name — a mismatch is silently dropped by the runtime with no diagnostic, so `test/jianying.test.mjs` pins both halves. - The MCP server uses the `stdio` transport, a single `node` argument and a contained `./mcp` working directory. The Plugin does not set `PLUGIN_ROOT` or `PLUGIN_DATA` itself. - Every runnable script invocation inside a Skill is anchored to `${PLUGIN_ROOT}`; every referenced `scripts/` and `references/` file exists. - The Plugin ships no network client, no credential literal, no telemetry and no third-party service. The only socket it opens is the local Windows named pipe of a running Jianying instance. - No contributor machine path is hardcoded: paths resolve from `JYAI_HOME`, `HOME`, `LOCALAPPDATA`, `ProgramFiles` and `PATH`. - README discloses the platform, a copyable example prompt, the expected result, the required executables, the account/paid-service position and the four-part data behaviour (no credentials / no network / no telemetry / no third-party services). - The six Skills stay well inside the runtime limits (64 Skill directories, 8 MCP servers per Plugin). Validation - `node scripts/validate.mjs` -> `OK plugin antianqi/jianying`; `Validated 30 hosted Plugins and all examples.` - `npm run check` (validate + the repository test suite) is green on ubuntu-latest, which is the only job the repository runs. - `node --test test/jianying.test.mjs` -> 10 pass / 0 fail. Test evidence - `test/jianying.test.mjs` (new, 10 cases): both manifest layouts, every Skill path in the v0.4 manifest resolving with a matching frontmatter name, the portable layout exposing the same six Skills, the stdio transport and its portable working directory, all eight verb schemas parsing, script invocations anchored to `${PLUGIN_ROOT}`, referenced files existing, the absence of a network client / credential literal / telemetry, the absence of contributor machine paths, and the README disclosures. - Negative-injection audit of that file: seven defects injected one at a time, the suite re-run, and the file restored from a backup afterwards. Each injection failed for the expected reason — frontmatter name mismatch, description over 1024 characters, a hardcoded drive-letter path, a `fetch` call, a missing "expected result" section, a missing "no telemetry" disclosure, and a `skills[]` entry pointing at a missing file — and the suite returned to 10/10 after every restore. - Plugin self-tests (Windows, manual, need a live Jianying 11.6): `selftest-mcp.mjs` 17/17, `selftest-draft-edit.mjs` 16/16, `selftest-talk-cut.mjs` 10/10, `check-env.mjs` 17 OK / 0 MISS. The verification cases carry negative injections of their own: a synthetic noise frame set must be judged "not visible", a synthetic large change must be judged "visible", a white background pushed to 255 must be reported as newly clipped, and an `expect` that says a field must not change must fail. - `.github/workflows/jianying-windows.yml` (new) runs only `node --test test/jianying.test.mjs` on windows-latest, filtered to this Plugin's paths. The repository's own `validate` job stays on ubuntu-latest; this is the scoped Windows counterpart the CI file asks a non-tool-map Windows Plugin to add, rather than re-introducing a repository-wide windows job. Known limits (also in the README) - Windows only, and the draft verbs need Jianying 11.6 running; grading and export do not need the GUI. - Structured grading fields (curves, HSL, colour wheels) cannot be written back into a Jianying preview yet: the draft-side field names are not confirmed, so grading is local export only. - The official exporter stays unreachable; the Plugin does not work around the block. - `clear` (8-level pyramid) is not wired into the chain. - `talk-cut` does not do ASR: the official ASR lives in the agent runtime and is not reachable from the plugin, so it takes a user-supplied SRT and only decides which lines are filler, pauses or repeats. --- .github/workflows/jianying-windows.yml | 69 ++ .../jianying/.claude-plugin/plugin.json | 27 + plugins/antianqi/jianying/LICENSE | 192 +++++ plugins/antianqi/jianying/README.md | 154 ++++ plugins/antianqi/jianying/mcp.json | 11 + plugins/antianqi/jianying/mcp/server.mjs | 755 ++++++++++++++++++ .../jianying/mcp/tools/audio-mix.schema.json | 27 + .../mcp/tools/draft-commit.schema.json | 19 + .../jianying/mcp/tools/draft-open.schema.json | 27 + .../mcp/tools/draft-patch.schema.json | 52 ++ .../jianying/mcp/tools/draft-read.schema.json | 27 + .../mcp/tools/grade-apply.schema.json | 645 +++++++++++++++ .../mcp/tools/render-frames.schema.json | 26 + .../mcp/tools/verify-diff.schema.json | 72 ++ plugins/antianqi/jianying/plugin.json | 21 + .../jianying/references/capability-dict.json | 132 +++ .../antianqi/jianying/references/chain-api.md | 93 +++ .../antianqi/jianying/references/pitfalls.md | 117 +++ .../jianying/references/verify-discipline.md | 110 +++ .../antianqi/jianying/scripts/check-env.mjs | 55 ++ .../antianqi/jianying/scripts/run-verb.mjs | 88 ++ .../jianying/scripts/selftest-draft-edit.mjs | 247 ++++++ .../jianying/scripts/selftest-mcp.mjs | 191 +++++ .../jianying/scripts/selftest-talk-cut.mjs | 82 ++ .../jianying/scripts/talk-cut-apply.mjs | 132 +++ .../jianying/scripts/talk-cut-plan.mjs | 227 ++++++ .../jianying/skills/jianying-cut/SKILL.md | 115 +++ .../skills/jianying-draft-edit/SKILL.md | 132 +++ .../skills/jianying-grade-chain/SKILL.md | 89 +++ .../jianying/skills/jianying-review/SKILL.md | 129 +++ .../skills/jianying-talk-cut/SKILL.md | 106 +++ .../jianying/skills/jianying-verify/SKILL.md | 122 +++ test/jianying.test.mjs | 160 ++++ 33 files changed, 4451 insertions(+) create mode 100644 .github/workflows/jianying-windows.yml create mode 100644 plugins/antianqi/jianying/.claude-plugin/plugin.json create mode 100644 plugins/antianqi/jianying/LICENSE create mode 100644 plugins/antianqi/jianying/README.md create mode 100644 plugins/antianqi/jianying/mcp.json create mode 100644 plugins/antianqi/jianying/mcp/server.mjs create mode 100644 plugins/antianqi/jianying/mcp/tools/audio-mix.schema.json create mode 100644 plugins/antianqi/jianying/mcp/tools/draft-commit.schema.json create mode 100644 plugins/antianqi/jianying/mcp/tools/draft-open.schema.json create mode 100644 plugins/antianqi/jianying/mcp/tools/draft-patch.schema.json create mode 100644 plugins/antianqi/jianying/mcp/tools/draft-read.schema.json create mode 100644 plugins/antianqi/jianying/mcp/tools/grade-apply.schema.json create mode 100644 plugins/antianqi/jianying/mcp/tools/render-frames.schema.json create mode 100644 plugins/antianqi/jianying/mcp/tools/verify-diff.schema.json create mode 100644 plugins/antianqi/jianying/plugin.json create mode 100644 plugins/antianqi/jianying/references/capability-dict.json create mode 100644 plugins/antianqi/jianying/references/chain-api.md create mode 100644 plugins/antianqi/jianying/references/pitfalls.md create mode 100644 plugins/antianqi/jianying/references/verify-discipline.md create mode 100644 plugins/antianqi/jianying/scripts/check-env.mjs create mode 100644 plugins/antianqi/jianying/scripts/run-verb.mjs create mode 100644 plugins/antianqi/jianying/scripts/selftest-draft-edit.mjs create mode 100644 plugins/antianqi/jianying/scripts/selftest-mcp.mjs create mode 100644 plugins/antianqi/jianying/scripts/selftest-talk-cut.mjs create mode 100644 plugins/antianqi/jianying/scripts/talk-cut-apply.mjs create mode 100644 plugins/antianqi/jianying/scripts/talk-cut-plan.mjs create mode 100644 plugins/antianqi/jianying/skills/jianying-cut/SKILL.md create mode 100644 plugins/antianqi/jianying/skills/jianying-draft-edit/SKILL.md create mode 100644 plugins/antianqi/jianying/skills/jianying-grade-chain/SKILL.md create mode 100644 plugins/antianqi/jianying/skills/jianying-review/SKILL.md create mode 100644 plugins/antianqi/jianying/skills/jianying-talk-cut/SKILL.md create mode 100644 plugins/antianqi/jianying/skills/jianying-verify/SKILL.md create mode 100644 test/jianying.test.mjs diff --git a/.github/workflows/jianying-windows.yml b/.github/workflows/jianying-windows.yml new file mode 100644 index 00000000..8d48c2cb --- /dev/null +++ b/.github/workflows/jianying-windows.yml @@ -0,0 +1,69 @@ +name: jianying (windows-latest) + +on: + pull_request: + paths: + - 'plugins/antianqi/jianying/**' + - '.github/workflows/jianying-windows.yml' + - 'test/jianying.test.mjs' + push: + branches: [main] + paths: + - 'plugins/antianqi/jianying/**' + - '.github/workflows/jianying-windows.yml' + - 'test/jianying.test.mjs' + # Manual dispatch: lets the maintainer or the PR author run the same + # windows-latest job outside a PR. The fork-to-upstream PR cannot trigger + # Actions without explicit maintainer approval, and first-time-contributor + # protection is on, so this is the way to capture a real Windows run. + workflow_dispatch: + +# The repository-wide `validate` job runs on ubuntu-latest and checks the +# package shape (manifests, Skills, MCP transports, docs, placeholders, path +# safety). The `jianying` Plugin is Windows-only: its runtime needs a live +# Jianying instance, a local named pipe and a Windows toolchain, none of +# which exist on a GitHub-hosted Linux runner. +# +# This workflow is the scoped Windows counterpart, and it follows the rule +# recorded in .github/workflows/ci.yml: a non-tool-map Windows Plugin adds its +# own paths-filtered workflow that runs exactly one focused test file, rather +# than re-introducing a `validate-windows` job that scans the whole +# repository (that job was dropped in round-9 because it failed on files this +# PR does not own). +# +# The test below is static and cross-platform: it asserts the manifest shape, +# that every Skill path in the v0.4 manifest resolves AND that its frontmatter +# name equals its directory name (a mismatch is silently dropped by the +# runtime, with no diagnostic), that script invocations are anchored to +# ${PLUGIN_ROOT}, and that the Plugin ships no network client, no credential +# literal, no telemetry and no contributor machine path. +# +# The runtime-dependent parts (the verb self-tests, the end-to-end draft edit, +# the official renderer) need a real Windows desktop with Jianying 11.6 +# installed, so they are documented in the PR description as manual evidence +# rather than pretended into CI. + +permissions: + contents: read + +jobs: + jianying-windows: + name: jianying on windows-latest (package shape, Skill paths, no hardcoded paths) + runs-on: windows-latest + timeout-minutes: 10 + defaults: + run: + shell: pwsh + steps: + - name: Checkout + uses: actions/checkout@v4 + + - name: Set up Node (system) + run: | + node --version + npm --version + + - name: Run jianying focused test suite + run: | + cd '${{ github.workspace }}' + node --test test/jianying.test.mjs diff --git a/plugins/antianqi/jianying/.claude-plugin/plugin.json b/plugins/antianqi/jianying/.claude-plugin/plugin.json new file mode 100644 index 00000000..a446b098 --- /dev/null +++ b/plugins/antianqi/jianying/.claude-plugin/plugin.json @@ -0,0 +1,27 @@ +{ + "name": "jianying", + "version": "0.1.0", + "description": "用逐像素复刻的剪映调色链给视频调色并出 10bit 成片（保留原音轨），附逐节点生效探针与画面级验收。8 个 MCP 动词覆盖草稿读写、调色、混音、渲染、验收。", + "author": { + "name": "antianqi", + "url": "https://github.com/antianqi" + }, + "license": "Apache-2.0", + "keywords": ["mcode", "jianying", "video-editing", "color-grading", "10bit", "mcp"], + "skills": [ + "./skills/jianying-cut/SKILL.md", + "./skills/jianying-draft-edit/SKILL.md", + "./skills/jianying-review/SKILL.md", + "./skills/jianying-talk-cut/SKILL.md", + "./skills/jianying-verify/SKILL.md", + "./skills/jianying-grade-chain/SKILL.md" + ], + "mcpServers": { + "jianying": { + "type": "stdio", + "command": "node", + "args": ["./mcp/server.mjs"], + "cwd": "./mcp" + } + } +} diff --git a/plugins/antianqi/jianying/LICENSE b/plugins/antianqi/jianying/LICENSE new file mode 100644 index 00000000..ec5fe20c --- /dev/null +++ b/plugins/antianqi/jianying/LICENSE @@ -0,0 +1,192 @@ + Apache License + Version 2.0, January 2004 + http://www.apache.org/licenses/ + + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION + + 1. Definitions. + + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + + 2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + copyright license to reproduce, prepare Derivative Works of, + publicly display, publicly perform, sublicense, and distribute the + Work and such Derivative Works in Source or Object form. + + 3. Grant of Patent License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + (except as stated in this section) patent license to make, have made, + use, offer to sell, sell, import, and otherwise transfer the Work, + where such license applies only to those patent claims licensable + by such Contributor that are necessarily infringed by their + Contribution(s) alone or by combination of their Contribution(s) + with the Work to which such Contribution(s) was submitted. If You + institute patent litigation against any entity (including a + cross-claim or counterclaim in a lawsuit) alleging that the Work + or a Contribution incorporated within the Work constitutes direct + or contributory patent infringement, then any patent licenses + granted to You under this License for that Work shall terminate + as of the date such litigation is filed. + + 4. Redistribution. You may reproduce and distribute copies of the + Work or Derivative Works thereof in any medium, with or without + modifications, and in Source or Object form, provided that You + meet the following conditions: + + (a) You must give any other recipients of the Work or + Derivative Works a copy of this License; and + + (b) You must cause any modified files to carry prominent notices + stating that You changed the files; and + + (c) You must retain, in the Source form of any Derivative Works + that You distribute, all copyright, patent, trademark, and + attribution notices from the Source form of the Work, + excluding those notices that do not pertain to any part of + the Derivative Works; and + + (d) If the Work includes a "NOTICE" text file as part of its + distribution, then any Derivative Works that You distribute must + include a readable copy of the attribution notices contained + within such NOTICE file, excluding those notices that do not + pertain to any part of the Derivative Works, in at least one + of the following places: within a NOTICE text file distributed + as part of the Derivative Works; within the Source form or + documentation, if provided along with the Derivative Works; or, + within a display generated by the Derivative Works, if and + wherever such third-party notices normally appear. The contents + of the NOTICE file are for informational purposes only and + do not modify the License. You may add Your own attribution + notices within Derivative Works that You distribute, alongside + or as an addendum to the NOTICE text from the Work, provided + that such additional attribution notices cannot be construed + as modifying the License. + + You may add Your own copyright statement to Your modifications and + may provide additional or different license terms and conditions + for use, reproduction, or distribution of Your modifications, or + for any such Derivative Works as a whole, provided Your use, + reproduction, and distribution of the Work otherwise complies with + the conditions stated in this License. + + 5. Submission of Contributions. Unless You explicitly state otherwise, + any Contribution intentionally submitted for inclusion in the Work + by You to the Licensor shall be under the terms and conditions of + this License, without any additional terms or conditions. + Notwithstanding the above, nothing herein shall supersede or modify + the terms of any separate license agreement you may have executed + with Licensor regarding such Contributions. + + 6. Trademarks. This License does not grant permission to use the trade + names, trademarks, service marks, or product names of the Licensor, + except as required for reasonable and customary use in describing the + origin of the Work and reproducing the content of the NOTICE file. + + 7. Disclaimer of Warranty. Unless required by applicable law or + agreed to in writing, Licensor provides the Work (and each + Contributor provides its Contributions) on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied, including, without limitation, any warranties or conditions + of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A + PARTICULAR PURPOSE. You are solely responsible for determining the + appropriateness of using or redistributing the Work and assume any + risks associated with Your exercise of permissions under this License. + + 8. Limitation of Liability. In no event and under no legal theory, + whether in tort (including negligence), contract, or otherwise, + unless required by applicable law (such as deliberate and grossly + negligent acts) or agreed to in writing, shall any Contributor be + liable to You for damages, including any direct, indirect, special, + incidental, or consequential damages of any character arising as a + result of this License or out of the use or inability to use the + Work (including but not limited to damages for loss of goodwill, + work stoppage, computer failure or malfunction, or any and all + other commercial damages or losses), even if such Contributor + has been advised of the possibility of such damages. + + 9. Accepting Warranty or Additional Liability. While redistributing + the Work or Derivative Works thereof, You may choose to offer, + and charge a fee for, acceptance of support, warranty, indemnity, + or other liability obligations and/or rights consistent with this + License. However, in accepting such obligations, You may act only + on Your own behalf and on Your sole responsibility, not on behalf + of any other Contributor, and only if You agree to indemnify, + defend, and hold each Contributor harmless for any liability + incurred by, or claims asserted against, such Contributor by reason + of your accepting any such warranty or additional liability. + + END OF TERMS AND CONDITIONS + + APPENDIX: How to apply the Apache License to your work. + + Copyright 2026 MCode Plugins contributors + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. diff --git a/plugins/antianqi/jianying/README.md b/plugins/antianqi/jianying/README.md new file mode 100644 index 00000000..1691c8e7 --- /dev/null +++ b/plugins/antianqi/jianying/README.md @@ -0,0 +1,154 @@ +# jianying + +> 用**逐像素复刻的剪映调色链**给视频调色，出 **10bit / 保留原音轨**的成片 —— 不需要剪映 GUI。 + +> **平台：仅 Windows。** 依赖本机的剪映 11.6 运行时与一个**活的**剪映进程（草稿读写与官方渲染都走剪映的 +> 本机命名管道）。调色与出片本身不需要打开剪映界面，但草稿相关动词需要剪映在运行。 + +## 解决什么问题 + +剪映的调色（亮度/对比/饱和/色温/分色/对数轮/曲线/HSL 曲线/锐化/暗角…）只有官方 app 能算。 +想把调色纳入 AI 自动化流水线（改参数 → 出片 → 再改），有两条路： + +1. **官方导出器** —— 官方 CLI 侧被 `−100007`（会话模式不支持）与 `−100015`（未授权宿主进程）挡死， + 且 CLI 构建比 app 早 5 天 ⇒ 命令树注册表错位。**这是官方侧的问题，我们不绕。** +2. **自己复刻**（本插件）—— 把官方的调色数学逐 feature 复刻出来（13/13 逐像素对齐，误差 ≤ 1 LSB）， + 自己出片。 + +⇒ 本插件走第 2 条：**MCP 提供 8 个原子动词**（草稿读写 / 调色 / 混音 / 渲染 / 验收）， +**Skill 提供 6 条"有自己中间思路 + 门禁"的剪辑链路**。 + +## 试一下 + +**示例提示词**（可直接粘给 MiniMax Code）： + +```text +给这个视频调色出片，风格用暖一点的，输出到 out\cut\demo.mp4 +``` + +**预期结果**： + +1. 插件先跑 `check-env` 环境自检，缺依赖会直接报缺哪一个。 +2. `grade_apply` 按 `spec` 出片，产出 `graded10.mp4`（逐点直算，推荐）与 `graded_lut.mp4`（65³ LUT 形态）。 +3. 同一个动词返回**逐节点生效探针**：`adjustColor` / `logWheel` / … 每一项的 MAD 都必须非零， + 任何一个为 0 ⇒ 动词**直接判失败**（而不是"链看着跑通、实际少一个节点"）。 +4. 返回全链改变量 `grade_change_mad`（实测 > 0）与产物路径。 +5. 画面验收用 `verify_diff` 对改前/改后帧做**全帧最大**像素差；注意官方渲染器本身有噪声底， + 像素结论要带 `noiseDir` 校准（见 `skills/jianying-verify/SKILL.md`）。 + +也可以直接调动词： + +```bash +node scripts/run-verb.mjs --list +node scripts/run-verb.mjs grade_apply '{"src":"<源视频.mp4>","outDir":"<输出目录>","spec":{"adjustColor":{"brightness":0.1,"temperature":0.15}}}' +``` + +## 8 个动词 + +| 动词 | 作用 | +|---|---| +| `draft_open` | 打开草稿（默认只读副本） | +| `draft_read` | 读草稿语义结构（时间线/片段/素材） | +| `draft_patch` | 语义级修改（**27 个官方行为操作**，带命名空间） | +| `draft_commit` | 落盘（自动素材路径重定位 + 真草稿 sha 断言） | +| `render_frames` | 用官方渲染器出帧/mp4 | +| `grade_apply` | 调色（10 节点链）+ 出 10bit 成片（**自带逐节点生效探针**） | +| `audio_mix` | 混音 + 响度归一到 −14 LUFS | +| `verify_diff` | 多维度验收：画面可见性 / 画质 / 响度 / 时间线 / 草稿完整性 | + +## 6 个 Skill + +| Skill | 干什么 | +|---|---| +| `jianying-cut` | 调色出片链路（风格预设 + 门禁） | +| `jianying-draft-edit` | 改草稿链路（27 个官方操作 + 落盘 + 生效验证） | +| `jianying-review` | 审片验收（多维度对照 + 最小返修范围） | +| `jianying-talk-cut` | 剪口播（SRT 规划器 + 水词/停顿/重复判定 + 落刀） | +| `jianying-verify`（库） | 验收判据纪律（**仅被引用时用**） | +| `jianying-grade-chain`（库） | 调色链 API 语义（10 节点 × 参数）（**仅被引用时用**） | + +## 精度 + +| 路径 | 误差 | +|---|---| +| 逐点直算（默认） | maxAbs **1 LSB** | +| 65³ 3D LUT | maxAbs **2 LSB** | +| 129³ 3D LUT | maxAbs **2 LSB**（更小） | + +曲线恒等情形与官方 LUT 生成器 **256/256 texel 字节完全一致**。 + +## 结构 + +```text +plugin-root/ +├── plugin.json # portable Agent Plugins 1.0（mcode 0.3.x） +├── mcp.json # MCP 声明（0.3.x） +├── .claude-plugin/plugin.json # v0.4.0+ manifest（skills[] + mcpServers） +├── mcp/ +│ ├── server.mjs # 8 动词 stdio MCP server（0 npm 依赖） +│ └── tools/.schema.json# 每个动词的入参 schema（硬校验） +├── skills/ +│ ├── jianying-cut/SKILL.md +│ ├── jianying-draft-edit/SKILL.md +│ ├── jianying-review/SKILL.md +│ ├── jianying-talk-cut/SKILL.md +│ ├── jianying-verify/SKILL.md +│ └── jianying-grade-chain/SKILL.md +├── references/ # 共享知识（按需引用） +│ ├── chain-api.md # 链 API 语义（10 节点 × 参数） +│ ├── verify-discipline.md # 判据纪律（视野 / 噪声底 / 注入自检） +│ ├── pitfalls.md # 踩坑清单 +│ └── capability-dict.json # 46 项能力 → 动词 → 参数/op → 判据（机器可读） +└── scripts/ + ├── run-verb.mjs # 插件自带的 MCP 客户端（让 skill 调动词） + ├── check-env.mjs # 环境自检 + ├── selftest-mcp.mjs # MCP 动词自检（17 项，含 6 个阴性用例） + ├── selftest-draft-edit.mjs # 改草稿链路端到端自检（16 项） + ├── selftest-talk-cut.mjs # 剪口播自检（10 项） + ├── talk-cut-plan.mjs # 口播保留区间规划器 + └── talk-cut-apply.mjs # 口播落稿器 +``` + +## 要求 + +**平台**：Windows（10/11）。 + +**需要在本机存在的可执行文件**： + +- `ffmpeg` 6.0 —— 在 `PATH`，或用 `FFMPEG` 环境变量指定 +- Python 3.14 —— 在 `PATH`，或用 `JYAI_PYTHON` 环境变量指定 +- `node` ≥ 22（运行 MCP server 与脚本） +- 剪映 11.6 —— **草稿相关动词需要它在运行**（走本机命名管道 `\\.\pipe\videoeditor-runtime-*`）； + 调色与出片本身不依赖 GUI + +**其他**： + +- `JYAI_HOME` 指向 JYAI 工具链仓库（工具在 `JYAI_HOME/tools`） +- 剪映运行时缓存 `%LOCALAPPDATA%\JianyingPro\User Data\Cache`（读取官方 shader 参照源用） + +**账号 / 付费服务**：**不需要任何账号，不需要任何付费服务，不需要任何 API Key。** + +## 数据与网络 + +- **无网络访问。** 插件不发起任何 HTTP/HTTPS 请求，也不连任何远端服务。 +- **不需要凭据。** 不读不写任何 token / 密钥 / cookie。 +- **无遥测。** 不上报任何使用数据、错误或崩溃信息。 +- **无第三方服务。** 不依赖任何外部 API、SDK 或托管后端。 +- **本机 IPC**：草稿读写与官方渲染通过**本机命名管道**与正在运行的剪映通信（Windows 本地 IPC，非网络）。 +- **数据流向**：所有输入（视频、音频、草稿）只在本机被读取；所有产物（成片、帧、报告）写到本机输出目录。 +- **草稿只读**：插件**只读**用户草稿；写操作一律在副本上进行，落盘时逐文件 sha 断言真草稿未被改动。 +- 路径全部通过 `JYAI_HOME` / `HOME` / `LOCALAPPDATA` / `ProgramFiles` / `PATH` 解析，**不写死盘符**。 + +## 边界 + +- 改草稿的**结构化调色字段**（曲线/HSL/色轮）**尚未确证**官方草稿字段名 ⇒ 目前只能**本地出片**， + 不能写进剪映预览。 +- 官方导出器不可达（见上）。 +- `clear`（清晰度，8 级金字塔）尚未接入链。 +- 口播剪辑**不做 ASR**：官方 ASR 在 agent runtime 里、拿不到；我们用**用户提供的 SRT**， + 只做"哪句是废话"的判定（清词/长停顿/重复），不做改写与字幕。 + +## 证据 + +调色链逐 feature 对齐、10bit 出片链路、官方参考帧获取、官方 27 个行为操作的参数契约、判据纪律 —— +见 JYAI 仓库的 `docs/OFFICIAL-REALTIME-PATH.md`（§54–§76）与 `docs/最终架构与开发路线.md`。 diff --git a/plugins/antianqi/jianying/mcp.json b/plugins/antianqi/jianying/mcp.json new file mode 100644 index 00000000..bb491905 --- /dev/null +++ b/plugins/antianqi/jianying/mcp.json @@ -0,0 +1,11 @@ +{ + "$schema": "https://agent-plugins.org/schemas/1.0.0/mcp.schema.json", + "mcpServers": { + "jianying": { + "type": "stdio", + "command": "node", + "args": ["./mcp/server.mjs"], + "cwd": "./mcp" + } + } +} diff --git a/plugins/antianqi/jianying/mcp/server.mjs b/plugins/antianqi/jianying/mcp/server.mjs new file mode 100644 index 00000000..57aab526 --- /dev/null +++ b/plugins/antianqi/jianying/mcp/server.mjs @@ -0,0 +1,755 @@ +#!/usr/bin/env node +/** + * jianying · MCP server（stdio，8 个动词） + * + * 设计纪律（见 docs/最终架构与开发路线.md）： + * ① skill 不感知执行路径 —— 本 server 内部决定"离线写盘"还是"活体 edit 域"，对 skill 透明 + * ② 机械验证与语义验证分离 —— 本 server 只做【机械验证】（护栏 + 探针 + 断言） + * ③ 判据优先 —— 入参用 schema 硬校验，非法参数直接拒绝，不让"静默失效"发生 + * + * 零 npm 依赖；路径全部走 JYAI_HOME / $HOME / %LOCALAPPDATA% 解析，绝不写死盘符。 + */ +import { createInterface } from 'node:readline'; +import { spawn, execFileSync } from 'node:child_process'; +import fs from 'node:fs'; +import net from 'node:net'; +import path from 'node:path'; +import os from 'node:os'; +import crypto from 'node:crypto'; +import { fileURLToPath } from 'node:url'; + +const HERE = path.dirname(fileURLToPath(import.meta.url)); +const PLUGIN_ROOT = path.resolve(HERE, '..'); +const JYAI = process.env.JYAI_HOME || path.join(os.homedir(), 'JYAI'); +const TOOLS_DIR = path.join(HERE, 'tools'); +const SCRIPTS = path.join(PLUGIN_ROOT, 'scripts'); + +/* ---------------------------------------------------------------- 路径解析 */ +function jyaiTool(name) { + const p = path.join(JYAI, 'tools', name); + if (!fs.existsSync(p)) throw new Error(`JYAI 工具不存在: ${p}（请设 JYAI_HOME 指向 JYAI 仓库）`); + return p; +} +function jyaiData(...parts) { + return path.join(JYAI, ...parts); +} +function cacheDir() { + const base = process.env.LOCALAPPDATA || path.join(os.homedir(), 'AppData', 'Local'); + return path.join(base, 'JianyingPro', 'User Data', 'Cache'); +} + +/* 可执行文件解析：env → PATH → 常见安装位置（绝不写死盘符） */ +function which(cmd, extra = []) { + if (process.env[cmd.toUpperCase().replace(/-/g, '_')]) return process.env[cmd.toUpperCase().replace(/-/g, '_')]; + const pathExt = (process.env.PATHEXT || '.EXE;.CMD;.BAT').split(';'); + for (const d of (process.env.PATH || '').split(path.delimiter)) { + if (!d) continue; + for (const ext of ['', ...pathExt]) { + const p = path.join(d, cmd + ext); + try { if (fs.existsSync(p) && fs.statSync(p).isFile()) return p; } catch { /* ignore */ } + } + } + for (const p of extra) { try { if (fs.existsSync(p)) return p; } catch { /* ignore */ } } + return cmd; // 交给 spawn 报错 +} +const FFMPEG = () => which('ffmpeg', [ + path.join(process.env.ProgramFiles || 'C:\\Program Files', 'ffmpeg', 'ffmpeg.exe'), +]); +const PYTHON = () => process.env.JYAI_PYTHON || which('python', [ + path.join(process.env.ProgramFiles || 'C:\\Program Files', 'Python314', 'python.exe'), + path.join(process.env.LOCALAPPDATA || '', 'Programs', 'Python', 'Python314', 'python.exe'), +]); + +/* ------------------------------------------------- 轻量 JSON Schema 校验 */ +/* 只支持我们 schema 里用到的关键字：type/properties/required/additionalProperties/ */ + /* enum/minimum/maximum/items/patternProperties/default/oneOf-任意键 */ +const ERR = []; +function val(v, s, pathStr) { + if (!s || typeof s !== 'object') return; + if (s.type === 'object') { + if (v === null || typeof v !== 'object' || Array.isArray(v)) { ERR.push(`${pathStr}: 期望 object`); return; } + for (const r of s.required || []) if (!(r in v)) ERR.push(`${pathStr}: 缺少必填 "${r}"`); + const props = s.properties || {}; + for (const [k, vv] of Object.entries(v)) { + if (props[k]) { val(vv, props[k], `${pathStr}.${k}`); continue; } + let handled = false; + if (s.patternProperties) { + for (const [re, sub] of Object.entries(s.patternProperties)) { + if (new RegExp(re).test(k)) { val(vv, sub, `${pathStr}.${k}`); handled = true; break; } + } + } + if (handled) continue; + if (s.additionalProperties === false) ERR.push(`${pathStr}.${k}: 未知键（不合法）`); + else if (typeof s.additionalProperties === 'object') val(vv, s.additionalProperties, `${pathStr}.${k}`); + } + } else if (s.type === 'array') { + if (!Array.isArray(v)) { ERR.push(`${pathStr}: 期望 array`); return; } + if (s.items) v.forEach((it, i) => val(it, s.items, `${pathStr}[${i}]`)); + } else if (s.type === 'number' || s.type === 'integer') { + if (typeof v !== 'number' || Number.isNaN(v)) { ERR.push(`${pathStr}: 期望 number`); return; } + if (s.type === 'integer' && !Number.isInteger(v)) ERR.push(`${pathStr}: 期望 integer`); + if (s.minimum !== undefined && v < s.minimum) ERR.push(`${pathStr}: ${v} < 最小 ${s.minimum}`); + if (s.maximum !== undefined && v > s.maximum) ERR.push(`${pathStr}: ${v} > 最大 ${s.maximum}`); + } else if (s.type === 'string') { + if (typeof v !== 'string') { ERR.push(`${pathStr}: 期望 string`); return; } + if (s.enum && !s.enum.includes(v)) ERR.push(`${pathStr}: "${v}" 不在允许集合 [${s.enum.join(', ')}]`); + } else if (s.type === 'boolean') { + if (typeof v !== 'boolean') ERR.push(`${pathStr}: 期望 boolean`); + } +} +function validate(args, schema) { + ERR.length = 0; + val(args, schema, 'args'); + if (ERR.length) throw new Error(`入参不合法：\n - ${ERR.join('\n - ')}`); + // 填 default + const out = { ...args }; + const fill = (o, s) => { + if (!s || s.type !== 'object' || !o) return; + for (const [k, sub] of Object.entries(s.properties || {})) { + if (!(k in o) && sub && 'default' in sub) o[k] = sub.default; + } + }; + fill(out, schema); + return out; +} + +/* ------------------------------------------------------------ 插件状态目录 */ +/* mcode 注入 ${PLUGIN_DATA}（每个实例独立）；没有则用临时目录。 + ★ 动词之间靠它共享状态：draft_open 记 session → draft_patch 存改后草稿快照 → draft_commit 取快照落盘。 */ +function stateDir() { + return process.env.PLUGIN_DATA || path.join(os.tmpdir(), 'jianying-plugin-state'); +} +function stateFile(sid) { return path.join(stateDir(), `session-${sid}.json`); } +function snapshotFile(sid) { return path.join(stateDir(), `snapshot-${sid}.json`); } +function writeState(sid, obj) { + fs.mkdirSync(stateDir(), { recursive: true }); + fs.writeFileSync(stateFile(sid), JSON.stringify(obj), 'utf8'); +} +function readState(sid) { + const p = stateFile(sid); + if (!fs.existsSync(p)) return null; + try { return JSON.parse(fs.readFileSync(p, 'utf8')); } catch { return null; } +} +/* 素材路径重定位（照 jyai-live.mjs 的 relocateMaterialPaths）：草稿搬过家后 path 失效 ⇒ 指回草稿目录内 */ +function relocateMaterialPaths(draft, draftDir) { + const fixes = []; + const materials = draft?.materials; + if (!materials || typeof materials !== 'object') return fixes; + for (const [kind, list] of Object.entries(materials)) { + if (!Array.isArray(list)) continue; + for (const m of list) { + if (!m || typeof m !== 'object') continue; + const mp = typeof m.media_path === 'string' ? m.media_path : null; + const p = typeof m.path === 'string' ? m.path : null; + if (!mp || !p) continue; + if (/^[A-Za-z]:[\\/]/.test(mp) || mp.startsWith('/')) continue; // 绝对路径 + if (fs.existsSync(p)) continue; // 已有效 + const candidate = path.join(draftDir, mp); + if (!fs.existsSync(candidate)) continue; + const sep = /\\/g.test(p) && !/\//.test(p) ? '\\' : '/'; + const newPath = candidate.replace(/[\\/]/g, sep); + fixes.push({ kind, from: p, to: newPath, media_path: mp }); + m.path = newPath; + } + } + return fixes; +} + +/* ------------------------------------------------------------ 管道直连（edit 域）*/ +/* 官方 behavior 操作是【带命名空间】的（segment.move / video.setVolume / track.add …）， + 不是裸名。权威清单：docs/HEADLESS-VERIFY.md §4.8.2（27 个，实测 27/27）。 */ +const BS = String.fromCharCode(92); +const PIPE_PREFIX = `${BS}${BS}.${BS}pipe${BS}videoeditor-runtime-`; + +/* 每个 op 的【已知必填项】（从官方实测报错反推，如 video.add→video_path_required ⇒ params.path 必填） */ +const OP_REQUIRED = { + 'video.add': ['path'], + 'text.add': ['content'], + 'track.mute': ['trackId'], + 'track.delete': ['trackId'], + 'segment.move': ['segmentId'], + 'segment.setTimeRange': ['segmentId'], + 'segment.setDescription': ['segmentId'], + 'segment.updateLayout': ['layout'], + 'segment.innerReplace': ['replacement'], +}; + +function vePipePid() { + try { + const out = execFileSync('powershell', ['-NoProfile', '-Command', + "Get-CimInstance Win32_Process -Filter \"Name='vebun.exe'\" | Select-Object -First 1 -ExpandProperty ParentProcessId"], + { encoding: 'utf8', timeout: 20000 }).trim(); + const n = Number(out); + if (Number.isInteger(n) && n > 0) return n; + } catch { /* ignore */ } + throw new Error('解析 vebun 父进程 pid 失败（剪映在跑吗？）'); +} +function pipeOnce(pid, domain, operation, params = {}, targetId, timeoutMs = 60000) { + return new Promise((resolve) => { + const env = { + protocolVersion: 1, + id: `jy-${Date.now()}-${crypto.randomBytes(3).toString('hex')}`, + domain, operation, params, priority: 'interactive', + }; + if (targetId) env.targetId = targetId; + const s = net.connect(`${PIPE_PREFIX}${pid}`); + let buf = '', done = false; + const fin = (r) => { if (!done) { done = true; try { s.destroy(); } catch { /* ignore */ } resolve(r); } }; + const t = setTimeout(() => fin({ outcome: 'TIMEOUT', raw: buf }), timeoutMs); + s.on('connect', () => { try { s.write(JSON.stringify(env) + '\n'); } catch (e) { clearTimeout(t); fin({ outcome: 'WRITE_ERROR', error: String(e) }); } }); + s.on('data', (d) => { + buf += d.toString('utf8'); + const nl = buf.indexOf('\n'); + if (nl !== -1) { clearTimeout(t); fin({ outcome: 'RESPONSE', raw: buf.slice(0, nl) }); } + }); + s.on('error', (e) => { clearTimeout(t); fin({ outcome: 'CONNECT_ERROR', error: e.code || e.message }); }); + s.on('close', () => { clearTimeout(t); fin({ outcome: buf ? 'RESPONSE' : 'CLOSED_EMPTY', raw: buf }); }); + }); +} +async function editCall(op, params, sessionId) { + const pid = vePipePid(); + const r = await pipeOnce(pid, 'edit', op, params, sessionId, 30000); + if (r.outcome === 'CONNECT_ERROR') throw new Error(`管道连不上：${r.error}（剪映在跑吗？）`); + if (r.outcome === 'TIMEOUT') throw new Error(`edit/${op} 超时`); + const parsed = JSON.parse(r.raw); + if (parsed.ok === false) throw new Error(`edit/${op} 被拒：${JSON.stringify(parsed.error)}`); + const res = parsed.result || {}; + if (res.error) throw new Error(`edit/${op} 失败：${res.error}`); // 官方原样错误（如 segment_id_required） + return res; +} + +/* ------------------------------------------------------------- 工具执行 */ +function run(cmd, args, opts = {}) { + return new Promise((resolve) => { + const p = spawn(cmd, args, { + cwd: opts.cwd || PLUGIN_ROOT, + env: { ...process.env, PYTHONIOENCODING: 'utf-8', PYTHONUTF8: '1' }, + windowsHide: true, + }); + const chunks = [], echunks = []; + p.stdout.on('data', (d) => chunks.push(d)); + p.stderr.on('data', (d) => echunks.push(d)); + p.on('error', (e) => resolve({ code: -1, stdout: opts.binary ? Buffer.alloc(0) : '', + stderr: String(e) })); + p.on('close', (code) => { + const so = Buffer.concat(chunks), se = Buffer.concat(echunks); + resolve({ code, + stdout: opts.binary ? so : so.toString('utf8'), + stderr: opts.binary ? se.toString('utf8') : se.toString('utf8') }); + }); + }); +} +const ok = (t) => ({ content: [{ type: 'text', text: typeof t === 'string' ? t : JSON.stringify(t, null, 2) }] }); +const fail = (t) => ({ content: [{ type: 'text', text: typeof t === 'string' ? t : JSON.stringify(t, null, 2) }], isError: true }); + +function shaTree(dir) { + const out = []; + const walk = (d) => { + let ents = []; + try { ents = fs.readdirSync(d, { withFileTypes: true }); } catch { return; } + for (const e of ents.sort((a, b) => a.name.localeCompare(b.name))) { + const p = path.join(d, e.name); + if (e.isDirectory()) walk(p); + else { + const h = crypto.createHash('sha256'); + h.update(fs.readFileSync(p)); + out.push(`${path.relative(dir, p).replace(/\\/g, '/')}:${h.digest('hex').slice(0, 16)}`); + } + } + }; + walk(dir); + return out; +} + +/* --------------------------------------------------------------- 8 个动词 */ +const IMPL = { + async draft_open(a) { + if (a.allowReal && a.copy !== false) { + throw new Error('allowReal=true 时必须显式 copy=false（开真草稿）'); + } + const copy = a.copy !== false; + const args = ['open', a.draft]; + if (copy) args.push('--copy'); + const r = await run(process.execPath, [jyaiTool('jyai-live.mjs'), ...args]); + if (r.code !== 0) throw new Error(`打开草稿失败：\n${r.stderr || r.stdout}`); + const txt = r.stdout; + // 解析出 sessionId / draft / dir（draft_patch 需要 sessionId） + const sid = (txt.match(/ve-session-\d+/) || [])[0] || null; + const dir = (txt.match(/dir\s+(\S+)/) || [])[1] || null; + const dname = (txt.match(/draft\s+(\S+)/) || [])[1] || null; + if (!sid) throw new Error(`打开草稿的输出里找不到 sessionId：\n${txt}`); + // 记状态（供 draft_patch / draft_commit 共享） + const draftDir = dir || null; + const st = { sessionId: sid, draft: dname, draftDir, copied: copy, + draftPath: draftDir ? path.join(draftDir, 'draft_content.json') : null, + openedAt: new Date().toISOString() }; + writeState(sid, st); + return ok({ copy, sessionId: sid, draft: dname, dir: draftDir, raw: txt }); + }, + + async draft_read(a) { + // ★ 优先读【会话内存快照】（draft_patch 存的那份）；没有才读磁盘。 + // 否则会出现：patch 改了会话内存，read 却读到没落盘的磁盘 ⇒ 回读拿到旧草稿 ⇒ 编排全错。 + let parsed = null, source = 'disk'; + if (a.sessionId) { + const snapP = snapshotFile(a.sessionId); + if (fs.existsSync(snapP)) { + try { parsed = JSON.parse(fs.readFileSync(snapP, 'utf8')); source = 'session'; } catch { /* ignore */ } + } + } + if (!parsed) { + const args = ['read', '--json']; + if (a.path) { + args.push('--path', a.path); + // ★ 必须带 --root：不带的话官方 readFile 解析不出 workspace ⇒ 静默返回空 + // （2026-10-21 实测：exists=undefined chars=0）⇒ 根目录默认取 path 的上级目录 + const root = a.root || path.dirname(a.path); + if (root) args.push('--root', root); + } + const r = await run(process.execPath, [jyaiTool('jyai-live.mjs'), ...args]); + if (r.code !== 0) throw new Error(`读取草稿失败：\n${r.stderr || r.stdout}`); + // jyai-live read 会在 JSON 之前打一行状态（exists=... encrypted=... cipherType=...）⇒ 剥掉 + const i = r.stdout.indexOf('{'); + const body = i >= 0 ? r.stdout.slice(i) : r.stdout; + try { parsed = JSON.parse(body); } catch { return ok(r.stdout); } + } + if (parsed) { + // 附一份"好用的摘要"：track/segment 的 id 与时间范围（供 skill 直接用） + const summary = { source, tracks: (parsed.tracks || []).map((t) => ({ + type: t.type, id: t.id, segments: (t.segments || []).map((s) => ({ + id: s.id, target_timerange: s.target_timerange, material_id: s.material_id, + })), + })) }; + return ok({ draft: parsed, summary }); + } + return ok('(空)'); + }, + + async draft_patch(a) { + // 护栏：已知必填项本地先拦（不白跑一次官方调用） + const need = OP_REQUIRED[a.op]; + if (need) { + const p = a.params || {}; + const miss = need.filter((k) => p[k] === undefined || p[k] === null || p[k] === ''); + if (miss.length) throw new Error(`op "${a.op}" 缺少必填参数：${miss.join(', ')}（放在 params 里）`); + } + if (!a.sessionId) throw new Error('缺少 sessionId（先调 draft_open 拿 session）'); + const res = await editCall(a.op, a.params || {}, a.sessionId); + // ★ 官方 edit 响应里的 diff.operations[0].value = 【改后的完整草稿】 + // ⇒ 存成快照，供 draft_commit 落盘（动词之间靠插件状态目录共享状态） + const ops = res.diff?.operations || []; + const full = ops.find((o) => o.entity === 'draft' && o.path === '/')?.value + || ops.find((o) => o.value && typeof o.value === 'object' && o.value.tracks)?.value; + let snapshot = null; + if (full) { + fs.mkdirSync(stateDir(), { recursive: true }); + fs.writeFileSync(snapshotFile(a.sessionId), JSON.stringify(full), 'utf8'); + snapshot = snapshotFile(a.sessionId); + } + return ok({ + op: a.op, + accepted: res.accepted !== false, + actionKey: res.actionKey, + baseRevision: res.diff?.baseRevision, + changedPaths: ops.map((o) => o.path), + snapshot, snapshotSaved: !!snapshot, + }); + }, + + async draft_commit(a) { + const sid = a.sessionId; + if (!sid) throw new Error('缺少 sessionId（先调 draft_open）'); + const st = readState(sid); + if (!st) throw new Error(`找不到 session ${sid} 的状态（先调 draft_open）`); + const snapP = snapshotFile(sid); + if (!fs.existsSync(snapP)) { + throw new Error('没有改后草稿快照（先用 draft_patch 改一处再落盘）'); + } + const draft = JSON.parse(fs.readFileSync(snapP, 'utf8')); + + // 护栏 1：真草稿 sha 前后比对 + let before = null; + if (a.realDraft) before = shaTree(a.realDraft); + + // 护栏 2：素材路径重定位（草稿搬过家后 path 失效 ⇒ 指回草稿目录内） + const draftDir = st.draftDir || a.draftDir || ''; + const fixes = relocateMaterialPaths(draft, draftDir); + const target = st.draftPath || path.join(draftDir, 'draft_content.json'); + const rootPath = draftDir || path.dirname(target); + + // 落盘：draft/writeFile 传明文 JSON，服务端写时自动加密（sealed on write） + const r = await pipeOnce(vePipePid(), 'draft', 'writeFile', + { path: target, rootPath, content: JSON.stringify(draft) }, undefined, 60000); + if (r.outcome !== 'RESPONSE') throw new Error(`writeFile 失败：${r.outcome}`); + const parsed = JSON.parse(r.raw); + if (parsed.ok === false) throw new Error(`writeFile 被拒：${JSON.stringify(parsed.error)}`); + const wr = parsed.result || {}; + + // 护栏 3：真草稿必须零改动 + let realOk = null; + if (a.realDraft) { + const after = shaTree(a.realDraft); + realOk = before.length === after.length && before.every((x, i) => x === after[i]); + if (!realOk) throw new Error('★ 真草稿被改动了（sha 不一致）——已判定失败'); + } + return ok({ + committed: true, encrypted: wr.encrypted, cipherType: wr.cipherType, + relocated: fixes.length, target, realDraftUnchanged: realOk, + }); + }, + + async render_frames(a) { + const args = [jyaiTool('jyai-b-render.mjs'), '--draft', a.draft, '--out', a.outDir]; + if (a.fps) args.push('--fps', String(a.fps)); + const r = await run(process.execPath, args); + if (r.code !== 0) throw new Error(`渲染失败：\n${r.stderr || r.stdout}`); + return ok(r.stdout); + }, + + async grade_apply(a) { + // 护栏：spec 的键必须都在 schema 的 properties 里（schema 校验已做，这里再兜一层） + const specFile = path.join(a.outDir, '_spec.json'); + fs.mkdirSync(a.outDir, { recursive: true }); + fs.writeFileSync(specFile, JSON.stringify(a.spec, null, 2), 'utf8'); + const py = PYTHON(); + const args = [jyaiTool('jyai-grade-render.py'), + '--src', a.src, '--outdir', a.outDir, '--spec', specFile, + '--frames', String(a.frames ?? 12)]; + if (a.noLut) args.push('--no-lut'); + const r = await run(py, args); + if (r.code !== 0) throw new Error(`调色失败：\n${r.stderr || r.stdout}`); + // 护栏：逐节点生效探针（节点为 0 ⇒ 判失败） + const cmp = path.join(a.outDir, 'grade-compare.json'); + if (!fs.existsSync(cmp)) throw new Error('没出 grade-compare.json ⇒ 无法判生效'); + const j = JSON.parse(fs.readFileSync(cmp, 'utf8')); + const probe = j.node_effect_probe || {}; + const dead = Object.entries(probe).filter(([k, v]) => k !== '_full_chain' && !(Number(v) > 0)).map(([k]) => k); + if (dead.length) throw new Error(`★ 下列节点没有生效（探针为 0）：${dead.join(', ')} —— 判定失败`); + if (!(Number(j.grade_change_mad) > 0)) throw new Error('★ 调色改变量为 0 ⇒ 判定失败'); + return ok({ out: j.out10, outLut: j.out10_lut, gradeChangeMad: j.grade_change_mad, probe }); + }, + + async audio_mix(a) { + const r = await run(process.execPath, [jyaiTool('jyai-audio-mix.mjs'), + '--draft', a.draft, '--video', a.video, '--out', a.out]); + if (r.code !== 0) throw new Error(`混音失败：\n${r.stderr || r.stdout}`); + return ok(r.stdout); + }, + + async verify_diff(a) { + // ★ max-over-frames：判据视野必须覆盖被测对象（不抽单帧） + // ★ 多维度审片（visibility / picture / audio / timeline / draft_integrity） + const checks = (a.checks && a.checks.length) ? a.checks : ['visibility']; + const want = (k) => checks.includes(k); + // ★ 条件必填：只有查画面维度才需要帧目录（只查 timeline 时可以不给） + if ((want('visibility') || want('picture')) && (!a.beforeDir || !a.afterDir)) { + throw new Error('查 visibility/picture 时必须给 beforeDir 与 afterDir（帧目录）'); + } + const res = { checks: {}, requested: checks }; + const problems = []; + + // ---------- ① 画面：visibility + picture（同一趟解码里一起算）---------- + if (want('visibility') || want('picture')) { + const list = (d) => fs.existsSync(d) + ? fs.readdirSync(d).filter((f) => f.endsWith('.png')).sort() : []; + const B = list(a.beforeDir), A = list(a.afterDir); + const N = a.noiseDir ? list(a.noiseDir) : []; + if (!B.length || !A.length) throw new Error('帧目录为空或不存在'); + if (a.noiseDir && !N.length) throw new Error('noiseDir 为空或不存在'); + const n = Math.min(B.length, A.length); + const PNG = await importPng(); + let best = { mad: -1, frame: -1, maxAbs: 0, changed: 0 }; + let noiseBest = { mad: -1, frame: -1 }; + let changedFrames = 0, noiseChangedFrames = 0; + // picture 统计（after 侧全局；before 侧同步累计，用来算【增量】） + const sum = [0, 0, 0]; + const mn = [255, 255, 255]; + const mx = [0, 0, 0]; + let clipPx = 0, overPx = 0, underPx = 0, px = 0; + let bClipPx = 0, bOverPx = 0, bUnderPx = 0, bPx = 0; + const beforeSum = [0, 0, 0]; let beforePx = 0; + for (let i = 0; i < n; i++) { + const x = await PNG.read(path.join(a.beforeDir, B[i])); + const y = await PNG.read(path.join(a.afterDir, A[i])); + if (x.length !== y.length) continue; + let dsum = 0, dmx = 0, ch = 0; + for (let k = 0; k < x.length; k++) { + const d = Math.abs(x[k] - y[k]); + dsum += d; if (d > dmx) dmx = d; if (d > 0) ch++; + } + const mad = dsum / x.length; + if (mad > 0) changedFrames++; + if (mad > best.mad) best = { mad, frame: i, maxAbs: dmx, changed: ch }; + // ★ 噪声底：同一状态【再渲染一次】的帧 ⇒ 同态差异就是渲染器噪声 + if (N.length) { + const z = await PNG.read(path.join(a.noiseDir, N[i])); + if (z.length === x.length) { + let nsum = 0; + for (let k = 0; k < x.length; k++) nsum += Math.abs(x[k] - z[k]); + const nmad = nsum / x.length; + if (nmad > 0) noiseChangedFrames++; + if (nmad > noiseBest.mad) noiseBest = { mad: nmad, frame: i }; + } + } + if (want('picture')) { + for (let k = 0; k + 2 < y.length; k += 3) { + const r = y[k], g = y[k + 1], b = y[k + 2]; + sum[0] += r; sum[1] += g; sum[2] += b; + if (r < mn[0]) mn[0] = r; if (r > mx[0]) mx[0] = r; + if (g < mn[1]) mn[1] = g; if (g > mx[1]) mx[1] = g; + if (b < mn[2]) mn[2] = b; if (b > mx[2]) mx[2] = b; + const hi = Math.max(r, g, b), lo = Math.min(r, g, b); + if (hi >= 250) { clipPx++; overPx++; } else if (lo <= 5) underPx++; + px++; + } + for (let k = 0; k + 2 < x.length; k += 3) { + const r = x[k], g = x[k + 1], b = x[k + 2]; + beforeSum[0] += r; beforeSum[1] += g; beforeSum[2] += b; + const hi = Math.max(r, g, b), lo = Math.min(r, g, b); + if (hi >= 250) { bClipPx++; bOverPx++; } else if (lo <= 5) bUnderPx++; + beforePx++; + } + } + } + const th = a.threshold ?? 0.5; + if (want('visibility')) { + // ★ 噪声校准：官方渲染器【同态重渲染】就有噪声（实测 17/108 帧变、最差 MAD 123.55） + // ⇒ 裸 MAD > 0.5 毫无判别力。给了 noiseDir 就按噪声底抬高判据线。 + const noiseMad = noiseBest.mad < 0 ? null : noiseBest.mad; + const noiseK = a.noiseK ?? 1.5; + const calibrated = noiseMad !== null; + const bar = calibrated ? Math.max(th, noiseMad * noiseK) : th; + const visible = best.mad > bar; + const v = { maxMad: best.mad, frame: best.frame, maxAbs: best.maxAbs, + changedPixels: best.changed, changedFrames, totalFrames: n, threshold: th, + noiseMad, noiseFrame: noiseBest.frame, noiseChangedFrames, noiseK, + calibrated, noiseBar: bar, + madOverNoise: noiseMad ? +(best.mad / noiseMad).toFixed(3) : null }; + v.verdict = visible ? 'PASS（改动可见）' : 'FAIL（改动在画面上不可见）'; + if (!calibrated) { + v.note = '未校准：官方渲染器同态重渲染噪声实测可达 MAD 123.55（17/108 帧变）⇒ 裸 MAD 不能单独证明"改动可见"；请补 noiseDir'; + } + res.checks.visibility = v; + if (!visible) { + problems.push(`画面不可见：max MAD ${best.mad.toFixed(4)} ≤ 判据线 ${bar.toFixed(4)}` + + (calibrated ? `（噪声底 ${noiseMad.toFixed(4)} × ${noiseK}）` : '（未校准，阈值仅 0.5）')); + } + } + if (want('picture') && px > 0) { + const mean = sum.map((s) => s / px); + const bmean = beforePx ? beforeSum.map((s) => s / beforePx) : [0, 0, 0]; + const p = { + afterMean: mean.map((v) => +v.toFixed(4)), + beforeMean: bmean.map((v) => +v.toFixed(4)), + meanDrift: mean.map((v, k) => +(v - bmean[k]).toFixed(4)), + min: mn, max: mx, + clipRatio: +(clipPx / px).toFixed(6), + overRatio: +(overPx / px).toFixed(6), + underRatio: +(underPx / px).toFixed(6), + pixels: px, + }; + // ★ 判据用【增量】而不是绝对值：片子里本来就有的纯白/纯黑不算"削波/死黑"， + // 只有"被改动新推上去/压下去"的才算缺陷。 + if (beforePx) { + p.beforeClipRatio = +(bClipPx / beforePx).toFixed(6); + p.beforeOverRatio = +(bOverPx / beforePx).toFixed(6); + p.beforeUnderRatio = +(bUnderPx / beforePx).toFixed(6); + p.clipDelta = +(p.clipRatio - p.beforeClipRatio).toFixed(6); + p.underDelta = +(p.underRatio - p.beforeUnderRatio).toFixed(6); + } + const clipDelta = p.clipDelta ?? 0; + const underDelta = p.underDelta ?? 0; + if (clipDelta > 0.02) problems.push(`高光削波【新增】${(clipDelta * 100).toFixed(2)}%（>2%）`); + if (underDelta > 0.10) problems.push(`暗部死黑【新增】${(underDelta * 100).toFixed(2)}%（>10%）`); + res.checks.picture = p; + } + } + + // ---------- ② 音频：LUFS / true peak ---------- + if (want('audio')) { + if (!a.beforeVideo || !a.afterVideo) { + res.checks.audio = { skipped: '需要 beforeVideo / afterVideo（mp4 路径）才能测响度' }; + } else { + const meas = async (mp) => { + const r = await run(FFMPEG(), ['-hide_banner', '-nostats', '-i', mp, + '-af', 'ebur128=peak=true', '-f', 'null', '-']); + const s = r.stderr || ''; + const I = (s.match(/^\s*I:\s*(-?[\d.]+)\s*LUFS/m) || [])[1]; + const Pk = (s.match(/^\s*Peak:\s*(-?[\d.]+)\s*dBFS/m) || [])[1]; + return { integratedLufs: I === undefined ? null : +I, truePeakDbfs: Pk === undefined ? null : +Pk }; + }; + const b = await meas(a.beforeVideo), c = await meas(a.afterVideo); + const res2 = { before: b, after: c }; + if (c.integratedLufs !== null && Math.abs(c.integratedLufs) > 0.5) { + res2.note = '响度未归一到 −14 LUFS（见 jianying-audio-mix）'; + } + res.checks.audio = res2; + } + } + + // ---------- ③ 时间线：段数 / 轨道 / 时长 ---------- + if (want('timeline')) { + if (!a.beforeDraft || !a.afterDraft) { + res.checks.timeline = { skipped: '需要 beforeDraft / afterDraft（草稿 JSON 路径）' }; + } else { + const readDraft = (p) => { + let txt = fs.readFileSync(p, 'utf8'); + const i = txt.indexOf('{'); + if (i > 0) txt = txt.slice(i); + try { return JSON.parse(txt); } catch { return null; } + }; + const db = readDraft(a.beforeDraft), da = readDraft(a.afterDraft); + if (!db || !da) res.checks.timeline = { skipped: '草稿读不出来（可能是官方加密封装）' }; + else { + const sum = (d) => { + const tracks = d.tracks || []; + let segs = 0, dur = 0; + for (const t of tracks) for (const s of (t.segments || [])) { + segs++; + const tr = s.target_timerange || {}; + dur = Math.max(dur, (tr.start || 0) + (tr.duration || 0)); + } + return { tracks: tracks.length, segments: segs, durationUs: dur, draftDurationUs: d.duration }; + }; + const sb = sum(db), sa = sum(da); + // ★ 逐段字段差：语义证据（"我到底改了什么"），不依赖像素 + const segMap = (d) => { + const m = new Map(); + for (const t of (d.tracks || [])) for (const s of (t.segments || [])) m.set(s.id, s); + return m; + }; + const mb = segMap(db), ma = segMap(da); + const segmentDiffs = []; + for (const [id, s2] of ma) { + const s1 = mb.get(id); + if (!s1) { segmentDiffs.push({ segmentId: id, kind: 'added' }); continue; } + const fields = {}; + for (const k of new Set([...Object.keys(s1), ...Object.keys(s2)])) { + const v1 = JSON.stringify(s1[k]), v2 = JSON.stringify(s2[k]); + if (v1 !== v2) fields[k] = { before: s1[k], after: s2[k] }; + } + if (Object.keys(fields).length) segmentDiffs.push({ segmentId: id, kind: 'changed', fields }); + } + for (const [id] of mb) if (!ma.has(id)) segmentDiffs.push({ segmentId: id, kind: 'removed' }); + const t = { before: sb, after: sa, + segmentsDelta: sa.segments - sb.segments, + durationDeltaUs: sa.durationUs - sb.durationUs, + segmentDiffs }; + // ★ 给了 expect 就按 expect 断言；没给才用"什么都没动 ⇒ 可疑"的启发式 + const exp = a.expect || null; + if (exp) { + if (exp.segmentsDelta !== undefined && t.segmentsDelta !== exp.segmentsDelta) { + problems.push(`段数变化 ${t.segmentsDelta} ≠ 期望 ${exp.segmentsDelta}`); + } + if (exp.durationDeltaUs !== undefined && t.durationDeltaUs !== exp.durationDeltaUs) { + problems.push(`时长变化 ${t.durationDeltaUs} µs ≠ 期望 ${exp.durationDeltaUs} µs`); + } + if (exp.segmentFieldChanged) { + const hit = segmentDiffs.some((d) => d.fields && d.fields[exp.segmentFieldChanged]); + if (!hit) problems.push(`没有任何 segment 的 "${exp.segmentFieldChanged}" 字段发生变化（改动可能没落盘）`); + } + if (exp.segmentFieldUnchanged) { + const hit = segmentDiffs.some((d) => d.fields && d.fields[exp.segmentFieldUnchanged]); + if (hit) problems.push(`不该变的 "${exp.segmentFieldUnchanged}" 字段却变了`); + } + } else if (t.segmentsDelta === 0 && t.durationDeltaUs === 0 && segmentDiffs.length === 0) { + problems.push('时间线没有任何变化（段数、时长、段字段都没动）—— 改动可能没落盘'); + } + res.checks.timeline = t; + } + } + } + + // ---------- ④ 草稿完整性：真草稿 sha ---------- + if (want('draft_integrity')) { + if (!a.realDraft) { + res.checks.draft_integrity = { skipped: '需要 realDraft（真草稿目录）' }; + } else { + const t = shaTree(a.realDraft); + res.checks.draft_integrity = { files: t.length, sha: t.slice(0, 3), note: '落盘时已做前后逐字节断言' }; + } + } + + res.verdict = problems.length ? 'FAIL' : 'PASS'; + res.problems = problems; + return problems.length ? fail(res) : ok(res); + }, +}; + +/* PNG 读：优先用 pngjs（若有），否则回落到 ffmpeg 解码 */ +let PNG = null; +async function importPng() { + if (PNG) return PNG; + const ff = FFMPEG(); + PNG = { + async read(file) { + const r = await run(ff, ['-v', 'error', '-i', file, '-pix_fmt', 'rgb24', '-f', 'rawvideo', '-'], + { cwd: path.dirname(file), binary: true }); + if (r.code !== 0 || !r.stdout.length) throw new Error(`解 PNG 失败: ${file}\n${r.stderr}`); + return r.stdout; + }, + }; + return PNG; +} + +/* ------------------------------------------------------------ 工具装载 */ +function loadTools() { + const out = []; + for (const f of fs.readdirSync(TOOLS_DIR).filter((x) => x.endsWith('.schema.json')).sort()) { + const s = JSON.parse(fs.readFileSync(path.join(TOOLS_DIR, f), 'utf8')); + out.push({ name: s.name, description: s.description, inputSchema: s.inputSchema, file: f }); + } + return out; +} +let TOOLLIST = []; +function initTools() { + TOOLLIST = loadTools(); + for (const t of TOOLLIST) { + if (typeof IMPL[t.name] !== 'function') { + console.error(`[jianying] 警告：schema ${t.file} 的工具 "${t.name}" 没有实现`); + } + } +} + +/* ------------------------------------------------------------ MCP 协议 */ +function handle(message) { + if (message.method === 'initialize') { + return { result: { protocolVersion: message.params?.protocolVersion ?? '2025-06-18', + capabilities: { tools: {} }, + serverInfo: { name: 'jianying', version: '0.1.0' } } }; + } + if (message.method === 'tools/list') { + return { result: { tools: TOOLLIST.map((t) => ({ + name: t.name, description: t.description, inputSchema: t.inputSchema })) } }; + } + if (message.method === 'tools/call') { + const name = message.params?.name; + const t = TOOLLIST.find((x) => x.name === name); + if (!t) return { error: { code: -32601, message: `未知工具: ${name}` } }; + let args = {}; + try { + args = validate(message.params?.arguments || {}, t.inputSchema); + } catch (e) { + return { result: fail(String(e.message)) }; + } + return Promise.resolve(IMPL[name](args)) + .then((r) => ({ result: r })) + .catch((e) => ({ result: fail(`${name} 失败：${e.message}`) })); + } + return { error: { code: -32601, message: `Method not found: ${String(message.method)}` } }; +} + +initTools(); +const input = createInterface({ input: process.stdin, crlfDelay: Infinity }); +input.on('line', (line) => { + if (!line.trim()) return; + let msg; + try { msg = JSON.parse(line); } catch { return; } + if (msg.id === undefined) return; + const out = handle(msg); + if (out instanceof Promise) { + out.then((r) => process.stdout.write(`${JSON.stringify({ jsonrpc: '2.0', id: msg.id, ...r })}\n`)); + } else { + process.stdout.write(`${JSON.stringify({ jsonrpc: '2.0', id: msg.id, ...out })}\n`); + } +}); diff --git a/plugins/antianqi/jianying/mcp/tools/audio-mix.schema.json b/plugins/antianqi/jianying/mcp/tools/audio-mix.schema.json new file mode 100644 index 00000000..02e70c65 --- /dev/null +++ b/plugins/antianqi/jianying/mcp/tools/audio-mix.schema.json @@ -0,0 +1,27 @@ +{ + "name": "audio_mix", + "description": "按草稿的音频结构重做混音并与画面合流（官方语义 + EBU R128 归一到 −14 LUFS）。", + "inputSchema": { + "type": "object", + "properties": { + "draft": { + "type": "string", + "description": "草稿目录" + }, + "video": { + "type": "string", + "description": "画面 mp4（无声或待替换音轨）" + }, + "out": { + "type": "string", + "description": "输出 mp4" + } + }, + "required": [ + "draft", + "video", + "out" + ], + "additionalProperties": false + } +} \ No newline at end of file diff --git a/plugins/antianqi/jianying/mcp/tools/draft-commit.schema.json b/plugins/antianqi/jianying/mcp/tools/draft-commit.schema.json new file mode 100644 index 00000000..399cdb51 --- /dev/null +++ b/plugins/antianqi/jianying/mcp/tools/draft-commit.schema.json @@ -0,0 +1,19 @@ +{ + "name": "draft_commit", + "description": "把当前 session 的改动落盘到草稿。会做【素材路径重定位】并断言真草稿 sha 未变。改动来源是 draft_patch 存下的快照（动词之间通过插件状态目录共享状态）。", + "inputSchema": { + "type": "object", + "properties": { + "sessionId": { + "type": "string", + "description": "draft_open 返回的 sessionId" + }, + "realDraft": { + "type": "string", + "description": "真草稿目录（只读）。用于落盘前后 sha 逐字节断言——真草稿被改动就判失败。" + } + }, + "required": ["sessionId"], + "additionalProperties": false + } +} diff --git a/plugins/antianqi/jianying/mcp/tools/draft-open.schema.json b/plugins/antianqi/jianying/mcp/tools/draft-open.schema.json new file mode 100644 index 00000000..d343b167 --- /dev/null +++ b/plugins/antianqi/jianying/mcp/tools/draft-open.schema.json @@ -0,0 +1,27 @@ +{ + "name": "draft_open", + "description": "打开一个剪映草稿（默认只读副本）。返回 sessionId 与草稿信息。绝不改真草稿。", + "inputSchema": { + "type": "object", + "properties": { + "draft": { + "type": "string", + "description": "草稿目录路径或草稿名" + }, + "copy": { + "type": "boolean", + "default": true, + "description": "默认 true=开副本。false=开真草稿（只读用途，且需 allowReal）" + }, + "allowReal": { + "type": "boolean", + "default": false, + "description": "允许开真草稿。默认 false；true 时若 copy 仍为 true 则拒绝" + } + }, + "required": [ + "draft" + ], + "additionalProperties": false + } +} \ No newline at end of file diff --git a/plugins/antianqi/jianying/mcp/tools/draft-patch.schema.json b/plugins/antianqi/jianying/mcp/tools/draft-patch.schema.json new file mode 100644 index 00000000..0d1ee0df --- /dev/null +++ b/plugins/antianqi/jianying/mcp/tools/draft-patch.schema.json @@ -0,0 +1,52 @@ +{ + "name": "draft_patch", + "description": "对草稿做语义级修改。官方 edit 域共 27 个带命名空间的行为操作（实测 27/27 可用）：draft.setCanvasSize/setDraftId · track.add/delete/mute · segment.move/scale/translate/rotate/updateLayout/split/delete/setTimeRange/innerReplace/setDescription · video.add/muteOn/muteOff/setVolume · audio.add/setVolume · text.add · undoRedo.undo/redo/canUndo/canRedo · invokeBatch。先调 draft_open 拿 sessionId。", + "inputSchema": { + "type": "object", + "properties": { + "op": { + "type": "string", + "enum": [ + "draft.setCanvasSize", + "draft.setDraftId", + "track.add", + "track.delete", + "track.mute", + "segment.move", + "segment.scale", + "segment.translate", + "segment.rotate", + "segment.updateLayout", + "segment.split", + "segment.delete", + "segment.setTimeRange", + "segment.innerReplace", + "segment.setDescription", + "video.add", + "video.muteOn", + "video.muteOff", + "video.setVolume", + "audio.add", + "audio.setVolume", + "text.add", + "undoRedo.undo", + "undoRedo.redo", + "undoRedo.canUndo", + "undoRedo.canRedo", + "invokeBatch" + ], + "description": "官方行为操作名（带命名空间，不是裸名）" + }, + "sessionId": { + "type": "string", + "description": "draft_open 返回的 sessionId" + }, + "params": { + "type": "object", + "description": "该 op 的参数（官方原生语义）。已知必填：video.add→path · text.add→content · track.mute/delete→trackId · segment.move/setTimeRange/setDescription→segmentId · segment.updateLayout→layout · segment.innerReplace→replacement。缺必填会先被本地拦下，不会白跑一次官方调用。" + } + }, + "required": ["op", "sessionId"], + "additionalProperties": false + } +} diff --git a/plugins/antianqi/jianying/mcp/tools/draft-read.schema.json b/plugins/antianqi/jianying/mcp/tools/draft-read.schema.json new file mode 100644 index 00000000..ad2a9972 --- /dev/null +++ b/plugins/antianqi/jianying/mcp/tools/draft-read.schema.json @@ -0,0 +1,27 @@ +{ + "name": "draft_read", + "description": "读取草稿的语义结构（时间线 / 片段 / 素材），返回 JSON + 结构化摘要。给 sessionId 时读【会话内存当前状态】（即 draft_patch 改过、还没落盘的那份）；不给则读磁盘上的草稿。", + "inputSchema": { + "type": "object", + "properties": { + "path": { + "type": "string", + "description": "draft_content.json 路径（明文）。不传 sessionId 时用它；不传则读当前 session 的草稿" + }, + "sessionId": { + "type": "string", + "description": "draft_open 返回的 sessionId。给了就读【会话内存】当前状态（draft_patch 改过但未落盘的改动也看得到）" + }, + "root": { + "type": "string", + "description": "草稿根目录（素材/子目录的解析基准）。不传则取 path 的上级目录。★ 官方 readFile 没有它会解析不出 workspace ⇒ 静默返回空" + }, + "scope": { + "type": "string", + "enum": ["full", "timeline", "segments", "materials"], + "default": "full" + } + }, + "additionalProperties": false + } +} diff --git a/plugins/antianqi/jianying/mcp/tools/grade-apply.schema.json b/plugins/antianqi/jianying/mcp/tools/grade-apply.schema.json new file mode 100644 index 00000000..f2b41a74 --- /dev/null +++ b/plugins/antianqi/jianying/mcp/tools/grade-apply.schema.json @@ -0,0 +1,645 @@ +{ + "name": "grade_apply", + "description": "用我们逐像素复刻的调色链（10 个节点，对齐官方 shader，误差 ≤1 LSB）给片子调色，出 10bit 成片（保留原音轨）。自带逐节点生效探针：任一节点单独打开无变化 ⇒ 判失败。", + "inputSchema": { + "type": "object", + "properties": { + "src": { + "type": "string", + "description": "源片路径" + }, + "outDir": { + "type": "string", + "description": "输出目录" + }, + "spec": { + "type": "object", + "description": "调色 spec：键是节点名，值是该节点的参数。未知键会被拒绝（防静默失效）。", + "properties": { + "adjustColor": { + "type": "object", + "description": "基础画面调节（官方 adjustColor）。fade/lightSensation 是它的子参数，不是独立节点。", + "properties": { + "brightness": { + "type": "number", + "minimum": -1, + "maximum": 1 + }, + "contrast": { + "type": "number", + "minimum": -1, + "maximum": 1 + }, + "saturation": { + "type": "number", + "minimum": -1, + "maximum": 1 + }, + "vibrance": { + "type": "number", + "minimum": -1, + "maximum": 1 + }, + "highlight": { + "type": "number", + "minimum": -1, + "maximum": 1 + }, + "shadow": { + "type": "number", + "minimum": -1, + "maximum": 1 + }, + "black": { + "type": "number", + "minimum": -1, + "maximum": 1 + }, + "white": { + "type": "number", + "minimum": -1, + "maximum": 1 + }, + "temperature": { + "type": "number", + "minimum": -1, + "maximum": 1 + }, + "fade": { + "type": "number", + "minimum": 0, + "maximum": 1 + }, + "lightSensation": { + "type": "number", + "minimum": -1, + "maximum": 1 + } + }, + "additionalProperties": false + }, + "colorCorrection": { + "type": "object", + "description": "色彩校正（官方 colorCorrection，与 adjustColor 同源的精简变体）。", + "properties": { + "contrast": { + "type": "number", + "minimum": -1, + "maximum": 1 + }, + "saturation": { + "type": "number", + "minimum": -1, + "maximum": 1 + }, + "brightness": { + "type": "number", + "minimum": -1, + "maximum": 1 + }, + "highlight": { + "type": "number", + "minimum": -1, + "maximum": 1 + }, + "shadow": { + "type": "number", + "minimum": -1, + "maximum": 1 + }, + "black": { + "type": "number", + "minimum": -1, + "maximum": 1 + }, + "white": { + "type": "number", + "minimum": -1, + "maximum": 1 + }, + "temperature": { + "type": "number", + "minimum": -1, + "maximum": 1 + } + }, + "additionalProperties": false + }, + "hsl": { + "type": "object", + "description": "分色调整（官方 HSL）。键是颜色名，值是 hue/saturation/lightness（界面值 ±100）。", + "properties": { + "HSLRed": { + "type": "object", + "properties": { + "hue": { + "type": "number", + "minimum": -1, + "maximum": 1 + }, + "saturation": { + "type": "number", + "minimum": -1, + "maximum": 1 + }, + "lightness": { + "type": "number", + "minimum": -1, + "maximum": 1 + } + }, + "additionalProperties": false + }, + "HSLGreen": { + "type": "object", + "properties": { + "hue": { + "type": "number", + "minimum": -1, + "maximum": 1 + }, + "saturation": { + "type": "number", + "minimum": -1, + "maximum": 1 + }, + "lightness": { + "type": "number", + "minimum": -1, + "maximum": 1 + } + }, + "additionalProperties": false + }, + "HSLBlue": { + "type": "object", + "properties": { + "hue": { + "type": "number", + "minimum": -1, + "maximum": 1 + }, + "saturation": { + "type": "number", + "minimum": -1, + "maximum": 1 + }, + "lightness": { + "type": "number", + "minimum": -1, + "maximum": 1 + } + }, + "additionalProperties": false + }, + "HSLYellow": { + "type": "object", + "properties": { + "hue": { + "type": "number", + "minimum": -1, + "maximum": 1 + }, + "saturation": { + "type": "number", + "minimum": -1, + "maximum": 1 + }, + "lightness": { + "type": "number", + "minimum": -1, + "maximum": 1 + } + }, + "additionalProperties": false + }, + "HSLCyan": { + "type": "object", + "properties": { + "hue": { + "type": "number", + "minimum": -1, + "maximum": 1 + }, + "saturation": { + "type": "number", + "minimum": -1, + "maximum": 1 + }, + "lightness": { + "type": "number", + "minimum": -1, + "maximum": 1 + } + }, + "additionalProperties": false + }, + "HSLMagenta": { + "type": "object", + "properties": { + "hue": { + "type": "number", + "minimum": -1, + "maximum": 1 + }, + "saturation": { + "type": "number", + "minimum": -1, + "maximum": 1 + }, + "lightness": { + "type": "number", + "minimum": -1, + "maximum": 1 + } + }, + "additionalProperties": false + } + }, + "additionalProperties": true, + "patternProperties": { + "^[0-9]{2}#[0-9a-fA-F]{6}_[HSL]$": { + "type": "object", + "properties": { + "hue": { + "type": "number", + "minimum": -1, + "maximum": 1 + }, + "saturation": { + "type": "number", + "minimum": -1, + "maximum": 1 + }, + "lightness": { + "type": "number", + "minimum": -1, + "maximum": 1 + } + }, + "additionalProperties": false + } + } + }, + "logWheel": { + "type": "object", + "description": "对数轮（四段：阴影/中间调/高光/偏移）。", + "properties": { + "shadow": { + "type": "object", + "properties": { + "r": { + "type": "number", + "minimum": -1, + "maximum": 1 + }, + "g": { + "type": "number", + "minimum": -1, + "maximum": 1 + }, + "b": { + "type": "number", + "minimum": -1, + "maximum": 1 + }, + "s": { + "type": "number", + "minimum": -1, + "maximum": 1 + } + }, + "additionalProperties": false + }, + "midtone": { + "type": "object", + "properties": { + "r": { + "type": "number", + "minimum": -1, + "maximum": 1 + }, + "g": { + "type": "number", + "minimum": -1, + "maximum": 1 + }, + "b": { + "type": "number", + "minimum": -1, + "maximum": 1 + }, + "s": { + "type": "number", + "minimum": -1, + "maximum": 1 + } + }, + "additionalProperties": false + }, + "highlights": { + "type": "object", + "properties": { + "r": { + "type": "number", + "minimum": -1, + "maximum": 1 + }, + "g": { + "type": "number", + "minimum": -1, + "maximum": 1 + }, + "b": { + "type": "number", + "minimum": -1, + "maximum": 1 + }, + "s": { + "type": "number", + "minimum": -1, + "maximum": 1 + } + }, + "additionalProperties": false + }, + "offset": { + "type": "object", + "properties": { + "r": { + "type": "number", + "minimum": -1, + "maximum": 1 + }, + "g": { + "type": "number", + "minimum": -1, + "maximum": 1 + }, + "b": { + "type": "number", + "minimum": -1, + "maximum": 1 + }, + "s": { + "type": "number", + "minimum": -1, + "maximum": 1 + } + }, + "additionalProperties": false + }, + "intensity": { + "type": "number", + "minimum": 0, + "maximum": 1 + }, + "rngDown": { + "type": "number", + "minimum": 0, + "maximum": 1 + }, + "rngUp": { + "type": "number", + "minimum": 0, + "maximum": 1 + } + }, + "additionalProperties": false + }, + "primaryWheel": { + "type": "object", + "description": "三原色轮（官方 primaryWheel）。", + "properties": { + "red": { + "type": "number", + "minimum": -1, + "maximum": 1 + }, + "green": { + "type": "number", + "minimum": -1, + "maximum": 1 + }, + "blue": { + "type": "number", + "minimum": -1, + "maximum": 1 + }, + "cyan": { + "type": "number", + "minimum": -1, + "maximum": 1 + }, + "magenta": { + "type": "number", + "minimum": -1, + "maximum": 1 + }, + "yellow": { + "type": "number", + "minimum": -1, + "maximum": 1 + } + }, + "additionalProperties": false + }, + "curves": { + "type": "object", + "description": "曲线（官方 curves，4 条 1D 曲线）。控制点组是官方 A,L,R 布局。", + "properties": { + "red": { + "type": "array", + "items": { + "type": "number" + } + }, + "green": { + "type": "array", + "items": { + "type": "number" + } + }, + "blue": { + "type": "array", + "items": { + "type": "number" + } + }, + "luma": { + "type": "array", + "items": { + "type": "number" + } + } + }, + "additionalProperties": false + }, + "hsl_curves": { + "type": "object", + "description": "HSL 耦合曲线（6 条）。points 是官方 L,A,R 扁平坐标（与 curves 的 A,L,R 不同！）。", + "properties": { + "hue2hue": { + "type": "object", + "properties": { + "points": { + "type": "array", + "items": { + "type": "number" + } + } + }, + "required": [ + "points" + ], + "additionalProperties": false + }, + "hue2sat": { + "type": "object", + "properties": { + "points": { + "type": "array", + "items": { + "type": "number" + } + } + }, + "required": [ + "points" + ], + "additionalProperties": false + }, + "hue2lum": { + "type": "object", + "properties": { + "points": { + "type": "array", + "items": { + "type": "number" + } + } + }, + "required": [ + "points" + ], + "additionalProperties": false + }, + "lum2sat": { + "type": "object", + "properties": { + "points": { + "type": "array", + "items": { + "type": "number" + } + } + }, + "required": [ + "points" + ], + "additionalProperties": false + }, + "satsat": { + "type": "object", + "properties": { + "points": { + "type": "array", + "items": { + "type": "number" + } + } + }, + "required": [ + "points" + ], + "additionalProperties": false + }, + "satlum": { + "type": "object", + "properties": { + "points": { + "type": "array", + "items": { + "type": "number" + } + } + }, + "required": [ + "points" + ], + "additionalProperties": false + } + }, + "additionalProperties": false + }, + "sharpen": { + "type": "object", + "description": "锐化（渲染期算子，邻域依赖，不能烘 3D LUT）。", + "properties": { + "intensity": { + "type": "number", + "minimum": 0, + "maximum": 1 + } + }, + "additionalProperties": false + }, + "vignetting": { + "type": "object", + "description": "暗角/白角（渲染期算子，uv 依赖，不能烘 3D LUT）。", + "properties": { + "intensity": { + "type": "number", + "minimum": 0, + "maximum": 1 + }, + "param": { + "type": "number", + "minimum": 0, + "maximum": 1 + } + }, + "additionalProperties": false + }, + "particle": { + "type": "object", + "description": "颗粒/光斑（渲染期算子，uv 依赖）。", + "properties": { + "intensity": { + "type": "number", + "minimum": 0, + "maximum": 1 + }, + "param": { + "type": "number", + "minimum": 0, + "maximum": 1 + }, + "seed": { + "type": "integer" + } + }, + "additionalProperties": false + } + }, + "additionalProperties": false + }, + "frames": { + "type": "integer", + "default": 12, + "description": "处理前 N 帧" + }, + "noLut": { + "type": "boolean", + "default": false, + "description": "只出逐点直算那一版" + } + }, + "required": [ + "src", + "outDir", + "spec" + ], + "additionalProperties": false + } +} \ No newline at end of file diff --git a/plugins/antianqi/jianying/mcp/tools/render-frames.schema.json b/plugins/antianqi/jianying/mcp/tools/render-frames.schema.json new file mode 100644 index 00000000..dd54da5f --- /dev/null +++ b/plugins/antianqi/jianying/mcp/tools/render-frames.schema.json @@ -0,0 +1,26 @@ +{ + "name": "render_frames", + "description": "用官方渲染器把草稿渲成帧序列 / mp4（画面来自官方渲染器，不是我们自己算）。", + "inputSchema": { + "type": "object", + "properties": { + "draft": { + "type": "string", + "description": "草稿目录" + }, + "outDir": { + "type": "string", + "description": "输出目录" + }, + "fps": { + "type": "number", + "default": 12 + } + }, + "required": [ + "draft", + "outDir" + ], + "additionalProperties": false + } +} \ No newline at end of file diff --git a/plugins/antianqi/jianying/mcp/tools/verify-diff.schema.json b/plugins/antianqi/jianying/mcp/tools/verify-diff.schema.json new file mode 100644 index 00000000..29eafb48 --- /dev/null +++ b/plugins/antianqi/jianying/mcp/tools/verify-diff.schema.json @@ -0,0 +1,72 @@ +{ + "name": "verify_diff", + "description": "审片/验收：改动前后对照。可查多个维度 —— visibility（画面变化是否可见，max-over-frames）、picture（画质：均值漂移/高光削波/暗部死黑）、audio（响度 LUFS 与 true peak）、timeline（段数/轨道/时长变化）、draft_integrity（真草稿 sha）。任一维度不达标即判 FAIL 并列出 problems。", + "inputSchema": { + "type": "object", + "properties": { + "beforeDir": { + "type": "string", + "description": "改前帧目录（PNG 序列）" + }, + "afterDir": { + "type": "string", + "description": "改后帧目录（PNG 序列）" + }, + "checks": { + "type": "array", + "items": { + "type": "string", + "enum": ["visibility", "picture", "audio", "timeline", "draft_integrity"] + }, + "default": ["visibility"], + "description": "要查的维度。缺省只查 visibility。" + }, + "threshold": { + "type": "number", + "default": 0.5, + "description": "visibility 的 max MAD 阈值（0..255 标度）。★ 官方渲染器同态重渲染就有噪声（实测 17/108 帧变、最差 MAD 123.55）⇒ 建议同时给 noiseDir 做校准。" + }, + "noiseDir": { + "type": "string", + "description": "★ 噪声基线帧目录：同一状态【再渲染一次】的帧。给了就把判据线抬到 max(threshold, noiseMad×noiseK)，不给则判据【未校准】（输出里会标 note）。" + }, + "noiseK": { + "type": "number", + "default": 1.5, + "description": "噪声底的倍数（判据线 = max(threshold, noiseMad × noiseK)）" + }, + "expect": { + "type": "object", + "description": "★ 期望断言（给 timeline 维度用）。给了就按期望判定，不再用\"什么都没动\"的启发式。", + "properties": { + "segmentsDelta": { "type": "number", "description": "期望的段数变化（如剪掉一段 = -1）" }, + "durationDeltaUs": { "type": "number", "description": "期望的时长变化（µs）" }, + "segmentFieldChanged": { "type": "string", "description": "期望某个 segment 字段发生变化（如 volume）" }, + "segmentFieldUnchanged": { "type": "string", "description": "期望某个 segment 字段【不】变化" } + }, + "additionalProperties": false + }, + "beforeVideo": { + "type": "string", + "description": "audio 维度用：改前 mp4" + }, + "afterVideo": { + "type": "string", + "description": "audio 维度用：改后 mp4" + }, + "beforeDraft": { + "type": "string", + "description": "timeline 维度用：改前草稿 JSON 路径（明文）" + }, + "afterDraft": { + "type": "string", + "description": "timeline 维度用：改后草稿 JSON 路径（明文）" + }, + "realDraft": { + "type": "string", + "description": "draft_integrity 维度用：真草稿目录（只读）" + } + }, + "additionalProperties": false + } +} diff --git a/plugins/antianqi/jianying/plugin.json b/plugins/antianqi/jianying/plugin.json new file mode 100644 index 00000000..806d5326 --- /dev/null +++ b/plugins/antianqi/jianying/plugin.json @@ -0,0 +1,21 @@ +{ + "$schema": "https://agent-plugins.org/schemas/1.0.0/plugin.schema.json", + "name": "jianying", + "version": "0.1.0", + "description": "用逐像素复刻的剪映调色链给视频调色并出 10bit 成片（保留原音轨），附逐节点生效探针与画面级验收。8 个 MCP 动词覆盖草稿读写、调色、混音、渲染、验收。", + "author": { + "name": "antianqi", + "url": "https://github.com/antianqi" + }, + "homepage": "https://github.com/antianqi/MiniMax-Code-Plugins-1/tree/main/plugins/antianqi/jianying", + "repository": "https://github.com/antianqi/MiniMax-Code-Plugins-1/tree/main/plugins/antianqi/jianying", + "license": "Apache-2.0", + "keywords": [ + "mcode", + "jianying", + "video-editing", + "color-grading", + "10bit", + "mcp" + ] +} diff --git a/plugins/antianqi/jianying/references/capability-dict.json b/plugins/antianqi/jianying/references/capability-dict.json new file mode 100644 index 00000000..a0834567 --- /dev/null +++ b/plugins/antianqi/jianying/references/capability-dict.json @@ -0,0 +1,132 @@ +{ + "schema": "jianying.capability-dict.v1", + "note": "46 项能力 → 动词 → 参数/op → 判据。机器可读版；人读版见 references/chain-api.md", + "verbs": { + "draft_open": { + "kind": "official-semantics", + "ops": [], + "guards": [ + "默认开副本", + "非副本需 allowReal" + ] + }, + "draft_read": { + "kind": "official-semantics", + "ops": [ + "full", + "timeline", + "segments", + "materials" + ], + "guards": [] + }, + "draft_patch": { + "kind": "official-semantics", + "ops": [ + "video.add" + ], + "guards": [ + "改前必须先 draft_open" + ] + }, + "draft_commit": { + "kind": "official-semantics", + "ops": [], + "guards": [ + "素材路径重定位", + "真草稿 sha 前后比对" + ] + }, + "render_frames": { + "kind": "official-semantics", + "ops": [], + "guards": [] + }, + "grade_apply": { + "kind": "ours", + "ops": [ + "adjustColor", + "colorCorrection", + "curves", + "hsl", + "hsl_curves", + "logWheel", + "particle", + "primaryWheel", + "sharpen", + "vignetting" + ], + "guards": [ + "未知键直接拒绝", + "逐节点生效探针（节点为 0 ⇒ 失败）" + ] + }, + "audio_mix": { + "kind": "ours", + "ops": [], + "guards": [ + "响度归一到 −14 LUFS" + ] + }, + "verify_diff": { + "kind": "ours", + "ops": [], + "guards": [ + "max-over-frames（不抽单帧）", + "真草稿 sha 断言" + ] + } + }, + "capabilities": { + "element": { + "draft": "draft_read", + "main": "draft_read", + "multi": "draft_read" + }, + "segment": { + "video": "draft_patch#video.add", + "image": "draft_patch", + "text": "draft_patch", + "subtitle": "draft_patch", + "sound": "audio_mix", + "sticker": "draft_patch" + }, + "property": { + "transform": "draft_patch", + "time": "draft_patch", + "alpha": "draft_patch", + "content": "draft_patch", + "sub": "draft_patch", + "effect": "grade_apply#adjustColor|colorCorrection|hsl|logWheel|primaryWheel|curves|hsl_curves|sharpen|vignetting|particle", + "mask": "draft_patch", + "matting": "draft_patch", + "animation": "draft_patch", + "speed_curve": "draft_patch" + } + }, + "node_kinds": { + "bakeable_3dlut": [ + "adjustColor", + "colorCorrection", + "curves", + "hsl", + "hsl_curves", + "logWheel", + "primaryWheel" + ], + "render_time": [ + "particle", + "sharpen", + "vignetting" + ], + "sub_param_of_adjustColor": [ + "fade", + "lightSensation" + ] + }, + "acceptance": { + "grade_apply": "每个节点单开必须产生非零改变（MAD>0）", + "verify_diff": "max MAD > threshold", + "draft_commit": "真草稿 sha 前后逐字节相同" + } +} \ No newline at end of file diff --git a/plugins/antianqi/jianying/references/chain-api.md b/plugins/antianqi/jianying/references/chain-api.md new file mode 100644 index 00000000..5be40bd6 --- /dev/null +++ b/plugins/antianqi/jianying/references/chain-api.md @@ -0,0 +1,93 @@ +# 链 API 人读版（`grade.apply` 的 `spec` 怎么填） + +> **机器可读版**：`mcp/tools/grade-apply.schema.json`（硬校验，非法键直接拒绝） +> **能力字典**：`references/capability-dict.json`（46 项能力 → 动词 → 参数/op → 判据） +> ★ 本文的语义部分（每个参数**什么意思**）只有这里有；schema 管"能不能填"。 + +## 强度语义 + +所有 `-1..1` 的参数是**强度**，不是绝对值：`0` = 不动 · `+0.5` = 中等往上调 · `-1` = 拉满。 +`0..1` 的参数是**比例/百分比**（如 `sharpen.intensity`、`vignetting.param`）。 + +## 10 个节点 + +| 节点 | 类型 | 能否烘 3D LUT | 官方对应 | +|---|---|---|---| +| `adjustColor` | 逐点 | ✅ | 基础画面调节（亮度/对比/饱和/高光/暗部/色温…） | +| `colorCorrection` | 逐点 | ✅ | 色彩校正（adjustColor 的精简变体） | +| `hsl` | 逐点 | ✅ | 分色调整（8 默认色 + 用户取色） | +| `logWheel` | 逐点 | ✅ | 对数轮（阴影/中间调/高光/偏移 四段） | +| `primaryWheel` | 逐点 | ✅ | 三原色轮 | +| `curves` | 逐点 | ✅ | 曲线（red/green/blue/luma 4 条 1D） | +| `hsl_curves` | 逐点 | ✅ | HSL 耦合曲线（6 条） | +| `sharpen` | **邻域** | ❌ | 锐化（渲染期算子） | +| `vignetting` | **uv** | ❌ | 暗角/白角（渲染期算子） | +| `particle` | **uv** | ❌ | 颗粒/光斑（渲染期算子） | + +★ **邻域 / uv 依赖的三个节点不能烘 3D LUT**（3D LUT 只能表达"逐点函数"）⇒ 只能在渲染期算。 + +### `adjustColor`（含两个"子参数"，不是独立节点） + +| 参数 | 含义 | +|---|---| +| `brightness` | 整体亮度 | +| `contrast` | 对比度 | +| `saturation` | 饱和度 | +| `vibrance` | 自然饱和度（保护已高饱和区域） | +| `highlight` | 高光（提亮区的对比） | +| `shadow` | 暗部 | +| `black` / `white` | 黑场 / 白场 | +| `temperature` | 色温（**正 = 偏暖**，负 = 偏冷） | +| `fade` | 褪色（0..1）—— 走官方 `fade_max.texture` 17³ LUT | +| `lightSensation` | 光感（−1..1）—— 走官方 min/max 两张 17³ LUT | + +★ **`fade` / `lightSensation` 是 `adjustColor` 的子参数**（官方 frag 里是 adjustColor 流水线的最后两段，顺序固定）。 + +### `hsl`（分色调整） + +键 = 颜色名，值 = `{hue, saturation, lightness}`（界面值 ±100）： +`HSLRed` / `HSLGreen` / `HSLBlue` / `HSLYellow` / `HSLCyan` / `HSLMagenta`， +以及**用户取色**（形如 `01#ad8a86_H` 的键，schema 里用 `patternProperties` 放行）。 + +### `logWheel`（对数轮，四段） + +每段 = `{r, g, b, s}`（色相/饱和，−1..1）。段名：`shadow` / `midtone` / `highlights` / `offset`。 + +### `curves`（曲线）—— ★ 控制点是 **A,L,R** 布局 + +`red` / `green` / `blue` / `luma` 各一组控制点，数字数组（x,y 交替）。 + +### `hsl_curves`（HSL 耦合曲线）—— ★ 控制点是 **L,A,R** 布局 + +6 条：`hue2hue` / `hue2sat` / `hue2lum` / `lum2sat` / `satsat` / `satlum`， +每条 = `{points: [x0,y0, x1,y1, ...]}`。 + +> **★★ 最容易踩的坑：`curves` 的控制点是 A,L,R，`hsl_curves` 的是 L,A,R —— 两套布局不能互相套用。** +> 判别：`L`（Left 控制点）永远在 `A`（锚点）之前 ⇒ 看第一个三元组的第二项是 x 还是 y。 + +## 怎么写一个 spec（例子） + +```json +{ + "adjustColor": { "brightness": 0.10, "contrast": 0.12, "saturation": 0.28, "temperature": 0.15 }, + "logWheel": { "shadow": { "r": 0.14, "g": 0.06, "b": 0.0, "s": 0.08 } }, + "sharpen": { "intensity": 0.45 } +} +``` + +## 精度（我们复刻的，不是官方保证） + +| 路径 | 误差 | +|---|---| +| 逐点直算（`--no-lut` 之外的默认） | maxAbs **1 LSB** | +| 65³ 3D LUT | maxAbs **2 LSB**（链式插值误差） | +| 129³ 3D LUT | maxAbs **2 LSB**，但更小 | + +⇒ **要精确 ⇒ 用逐点直算那一版**（`graded10.mp4`）；要小体积/快 ⇒ 用 LUT 版。 + +## 已知不覆盖 + +- `fade` / `lightSensation` 只在 `adjustColor` 内有效，单独提出来无效（会被 schema 拒） +- `clear`（清晰度，8 级金字塔）**尚未接入链**（官方默认 intensity=0 本身是空转） +- `colorMigration`（色彩迁移）官方算法在闭源 `Amaz.Algorithm` ⇒ 只能自算替代 +- 曲线 / HSL / 色轮**写回剪映**的草稿字段名尚未确证 ⇒ 目前只能**我们本地出片**，不能写进剪映预览 diff --git a/plugins/antianqi/jianying/references/pitfalls.md b/plugins/antianqi/jianying/references/pitfalls.md new file mode 100644 index 00000000..6377847c --- /dev/null +++ b/plugins/antianqi/jianying/references/pitfalls.md @@ -0,0 +1,117 @@ +# 踩坑清单（血买的，按"会不会再踩"排序） + +> 每条都带**症状 → 根因 → 对策**。这清单是"经验层"，与 `verify-discipline.md`（判据）配套。 + +## 会静默出错的（最危险） + +### 1. 节点参数形态不对 ⇒ 整段被跳过，链却"跑通了" +- **症状**：链输出正常，但少了某个节点的效果；探针里那个节点是 0 +- **根因**：链 API 的键名/嵌套形态不匹配（各节点风格互不相同） +- **对策**：① 逐节点生效探针**内建在 `grade_apply` 动词里** ② 链 API 键名必须查 `references/chain-api.md` ③ 未知键**直接拒绝**（schema 硬校验） + +### 2. `curves` 与 `hsl_curves` 的控制点布局不同 +- **症状**：曲线形状怪、控制点不生效 +- **根因**：`curves` 是 **A,L,R**；`hsl_curves` 是 **L,A,R** +- **对策**：看第一个三元组的第二项是 x 还是 y —— 是 x ⇒ A,L,R；是 y ⇒ L,A,R + +### 3. 探针自己用错了 spec +- **症状**：多个节点报出**完全相同**的数 +- **根因**：探针用了全局 spec 而非单节点 spec +- **对策**：**"数值整齐得不正常"就是探针没在测** —— 见到整齐输出先怀疑探针 + +## 编码 / 环境 + +### 4. 本机系统默认 GBK ⇒ Python 打印非 ASCII 就炸 +- **症状**：`UnicodeEncodeError: 'gbk' codec can't encode character '³'` +- **对策**：脚本自身 `sys.stdout.reconfigure(encoding='utf-8')` + 调用方设 `PYTHONIOENCODING=utf-8` / `PYTHONUTF8=1` +- ★ **skill 要做成"任何人调用都安全"**，不能只在自己 shell 里设环境变量 + +### 5. PowerShell 内联复杂 Python/JS 会被引号吃掉 +- **症状**：语法错误、字符串被截断 +- **对策**：复杂逻辑写成 `.py` / `.mjs` 文件再执行；文档批量替换也用脚本 + +## 流程 / 判据 + +### 6. 单帧抽样当判据 +- **症状**：明明改动了却判 FAIL（或反过来假 PASS） +- **对策**：`verify_diff` 取全帧最大 + +### 6b. ★★ 拿"别的跑"留下的目录当基线（2026-10-21 实测） +- **症状**：自检显示 `maxMad=123.5 改动可见`，但那 123.5 根本不是这次改动造成的 +- **根因**：自检把**另一次跑**的"改前帧"目录当基线，拿它和**这次**的"改后帧"比 + ⇒ 比的是两次不同的改动，PASS 与被测链路无关（**假绿**） +- **对策**：基线必须**本次自产**；自检开头就渲染自己的"改前"；`expect` 写死这次要验证的东西 + +### 6c. ★★★ 官方渲染器【不是确定性的】⇒ 裸像素阈值没有判别力 +- **症状**：同一份草稿渲染两次，帧就变了（14~17/108 帧），最差 MAD **48~123** + ⇒ 任何"max MAD > 0.5 ⇒ 改动可见"都是自说自话（噪声比阈值高两个数量级） +- **根因**：`jyai-b-render.mjs` 抓帧是"seek 后轮询 captureFrame，取容差内的第一帧"； + 抓哪一帧取决于调度时序 ⇒ 不确定。**收紧容差（60ms→20ms）也没用**（实测噪声依旧） +- **对策**：① 改草稿链用**语义判据**（落盘草稿的逐段字段差）② 像素判据必须传 `noiseDir` 做噪声校准 + ③ 我们自己的 GL/LUT 渲染链是确定性的，它的像素差分才可信 +- **教训**：**报"改动可见"之前，先报噪声底。** + +### 7. 两份验收并发跑写同一目录 +- **症状**：证据互相污染，数字对不上 +- **对策**：独立输出目录；文档写清目录名 + +### 7b. 用字符串收集 stdout 会毁掉二进制 +- **症状**：两个**不同**的 PNG 序列比出"0 差异" +- **根因**：`so += d`（d 是 Buffer）⇒ 走 utf8 解码 ⇒ 非法字节被替换 ⇒ 不同数据塌成同一串 +- **对策**：二进制必须用 `Buffer.concat(chunks)` 收集（插件的 `run()` 已用 `opts.binary`） +- ★ 我在 2026-10-21 的噪声探针里**又踩了一次**（自己写的探针假报"0 差异"）⇒ 写完探针先自问"它会不会假绿" + +### 8. 写文档凭记忆 +- **症状**：URL / 端口 / 管道名 / 白名单 / 计数写错 +- **实测**：我把 bridge 端口编成 `4498`（代码里根本没有这个数）、把管道后缀说成"剪映主进程 pid"（实际是 `vebun.exe` 的 `ParentProcessId`） +- **对策**：写入前逐条对源码 + +### 9. 把官方的结构当我们的模板 +- **症状**：照抄了一堆 mcode 根本不认的目录（`model/`、`model-tasks.json`…） +- **对策**：**官方是素材与语义的来源，mcode 规范才是结构约束**；先读 `docs/plugin-compatibility.md` 与 `scripts/validate.mjs` + +### 10. "把 demo 当默认值的代码"= 没被测过的代码 +- **症状**：spec 里没有某个节点 ⇒ `KeyError` +- **根因**：`path_A` 写死调用了 `sharpen`/`vignetting` +- **对策**：按 spec 动态跑；skill 第一次用"用户自己的 spec"就把它炸出来了 + +## 协议 / 通道 + +### 11. `player/load` 的 `workspacePath` 有两个含义 +- **症状**：渲染出来"媒体丢失 / Media Not Found"红卡 +- **根因**：`player/create→load` 路径要**草稿目录的父目录**；渲染路径要**草稿目录本身** +- **对策**：别混这两个调用 + +### 12. 音频的三个量纲坑 +- **症状**：音量对不上 / 裁剪没生效 +- **根因**：① 键名带点是**字面量**（`a['duration.ms']`，不是 `a.duration?.ms`）② `timeline.*.ms` 是**毫秒**、`attributes["duration.ms"]` 是**微秒** ③ `volume` 的量纲按段类型不同（`text_to_audio` 0..1 线性 / `sound` 0..100 百分比 / 视频段 0..1） +- **对策**：一律以 timeline 为准；ffmpeg 的 `atrim`/`afade` 要**秒** + +### 13. 草稿搬过家 ⇒ `source` 是过期路径 +- **症状**：素材找不到 +- **对策**：按 basename 在 `/{video,audio}`、`/{video,audio}` 回退查找 + +### 13b. ★ `jyai-live read --path` 少了 `--root` ⇒ 静默返回空（2026-10-21 实测） +- **症状**：`draft_read` 带 path 读指定草稿，返回 `exists=undefined cipherType=undefined chars=0`，**不报错** +- **根因**：官方 `draft/readFile` 要 `rootPath` 解析 workspace；`jyai-live read` 默认取 `st.draftDir`， + 而插件读的是**别的目录**（副本）⇒ 解析不出来 ⇒ 返回空 +- **对策**：插件的 `draft_read` 现在**自动补 `--root`**（默认取 path 的上级目录） +- ★ 凡是"读别处的草稿"，先确认 root 传了没有 —— **静默返回空比报错更难发现** + +## 渲染 / 数学 + +### 14. 3D LUT 表达不了邻域 / uv 依赖 +- `sharpen`（邻域）· `vignetting`（uv）· `particle`（uv）· `clear`（金字塔）⇒ **只能渲染期算** + +### 15. GPU float32 `fract` 在 ~1.2e4 量级不可复现 +- **症状**：颗粒效果逐像素对不上（69.5% 像素接近，4.7% fract 整单位跳变） +- **结论**：实用无影响，不要在这上面耗时间 + +### 16. 官方 shader 的定点编码会截断 +- `enCode` 的字节会溢出（|v|>1 时截断）⇒ 拉普拉斯差分超出 [−1,1] 会丢精度 +- 对策：链内用 float64 复刻，别在中间丢 + +### 17. 官方某些效果默认就是空转 +- `sharpen`：`white_gam`/`black_gam` 被注释掉 +- `clear`：`intensity = self.intensity * sharpIntensity`，而 `sharpIntensity` 默认 **0** ⇒ 输出逐字节等于输入 +- ⇒ **看到"官方效果没出来"，先查默认参数，不是我们的 bug** diff --git a/plugins/antianqi/jianying/references/verify-discipline.md b/plugins/antianqi/jianying/references/verify-discipline.md new file mode 100644 index 00000000..42e08ca4 --- /dev/null +++ b/plugins/antianqi/jianying/references/verify-discipline.md @@ -0,0 +1,110 @@ +# 判据纪律（任何"通过"都必须过这一关） + +> 这份是**护栏的知识层**。机械部分在 `scripts/` 与 MCP 动词的内建检查里。 +> ★ **护城河不在工具，在判据** —— 工具（读草稿/渲染/调色）谁都能调；"怎么知道它真的生效了"是我们独有的。 + +## 三层判定 + +``` +协议层说支持 ≠ 契约层被接受 ≠ 渲染层生效 +``` + +| 层 | 判什么 | 怎么判 | +|---|---|---| +| **契约层** | 字段认不认 | 材料存活 / 回写明文 / 字段保留 | +| **渲染层** | 有没有效果 | 无头取帧 + **A/B 逐像素差分** + **阳性对照** | + +## 六条硬规矩 + +### 1. 判据的"视野"必须覆盖被测对象 +**只抽一帧 = 用抽样代替了被测对象** ⇒ 假 FAIL 和假 PASS 都会出。 + +实测（2026-10-21）：单帧抽样选的第 5 帧 MAD=0.47（阈值 0.5）判 FAIL， +而同一批帧里第 74 帧 MAD=123.5 / maxAbs=255 / 100% 像素变化。 +⇒ **对策：`verify_diff` 一律取全帧最大（max-over-frames）。** + +### 2. 阴性对照必须存在 +每个"能跑"都要配一个"不能跑"的对照。 +实测：官方抽取帧接口**全是预览路径**（ΔYAVG 0.00）⇒ "抽帧里没有调色"是**官方设计**，不是我们的 bug。 +**没有阴性对照的 PASS 一律不算 PASS。** + +### 3. 输出必须能自证是"这次跑出来的" +**同一输出目录不许并发跑两份验收**（会互相覆盖帧 ⇒ 证据被污染）。 +⇒ 每次跑用**独立输出目录**，并在文档里写清目录名。 + +### 4. 探针要能区分"没生效"和"没在测" +**所有节点数值一模一样 ⇒ 探针没在测单节点。** +（实测踩过：探针用全局 SPEC 而非单节点 spec ⇒ 6 个节点报出完全相同的数。） + +### 5. 参数形态不对 ⇒ 静默失效 +链的每个节点要么真生效、要么被整段跳过，**不会报错**。 +实测踩过：`hsl` 键名写错 ⇒ 节点被跳过，**链看着跑通、实际少一个节点**。 +⇒ 对策：**逐节点生效探针内建在动词里**（`grade_apply` 每个节点单开必须产生非零改变，否则判失败）。 + +### 6. 注入坏了必须失败（negative-injection） +每个判据都要能被"故意弄坏"而失败。已做过的注入： + +| 注入 | 期望 | 实测 | +|---|---|---| +| 坏 LUT 维度顺序（.cube 转置） | LUT 误差爆 | 0.000219 → 0.268430（放大 **1223×**） | +| 逐点算子整体压掉 | 逐点误差爆 | 0.000119 → 0.164800 | +| 金字塔接线差一级 | 自洽性爆 | 9.7e-19 → 5.7e-2 | +| 改画质到 0 | 探针爆 | 逐节点探针报"没有生效" | +| 同一目录对同一目录 | 判 FAIL | maxMad=0 ⇒ FAIL ✓ | +| **亮度 −0.01 当噪声** | 校准后判"不可见" | maxMad=5、噪声底=5、判据线=7.5 ⇒ FAIL ✓ | +| **亮度 −0.06 当真改动** | 校准后判"可见" | maxMad=19、倍数 3.8 ⇒ PASS ✓ | +| **白底 +0.2 推到 255** | 必须报"新增削波" | clipΔ=33.33% ⇒ problems 命中 ✓ | +| **expect 说 volume 不该变** | 必须 FAIL | 逐段字段差检出 volume 变化 ⇒ FAIL ✓ | + +### 7. ★★ 像素判据必须先量【噪声底】（2026-10-21 实测，最重要的一条） + +**官方渲染器不是确定性的。** 同一份草稿**渲染两次**，帧就已经不一样： + +| 样本 | 不同的帧 | 最差 MAD | 备注 | +|---|---|---|---| +| 容差 60ms（原始） | 14/108 | **48.19** @frame39 | 整帧不同（maxAbs=255） | +| 容差 20ms（收紧后） | 17/108 | **123.55** @frame73/74 | 整帧不同 | + +⇒ **两次渲染的噪声底（≈48~123）比任何"改动可见"阈值（0.5）高两个数量级。** +⇒ **任何裸 MAD 都不能证明"改动可见"** ——包括历史上写进文档的 123.51 @frame74。 + +**根因**：`jyai-b-render.mjs` 抓帧是"seek 之后轮询 captureFrame，拿时间戳在容差内的第一帧"。 +容差 60ms > 半帧周期（83ms）⇒ 会接受**邻近帧**；抓哪一帧取决于调度时序 ⇒ 不确定。 +（收紧到 20ms 后 108/108 全部精确命中、兜底 0、丢失 0，**但噪声依旧** ⇒ 噪声不是容差造成的。） + +**对策（三选一，按可靠性排序）**： + +1. **用语义判据**（推荐）：改草稿链的"生效"证据 = **落盘草稿的逐段字段差**（如 `volume` 0.99→0.2）， + 完全不依赖像素，确定性、可复跑。`verify_diff` 的 `checks:['timeline']` + `expect.segmentFieldChanged` 就是这个。 +2. **噪声校准**：给 `verify_diff` 传 `noiseDir`（同一状态**再渲染一次**的帧）， + 判据线自动抬到 `max(threshold, noiseMad × 1.5)`；不传就在输出里标 `calibrated:false` + note。 +3. **用我们自己的渲染器**：`grade_apply` 走的是我们自己的 GL/LUT 链（确定性）⇒ 它的像素差分是可信的。 + +> **纪律：报"改动可见"之前，先报噪声底。** 没有噪声底就不要报"可见"。 + +## 数字底线（我们复刻链的验收口径） + +| 对象 | 口径 | 实测 | +|---|---|---| +| 调色链（逐点） | maxAbs ≤ **1 LSB** | 1 LSB（hsl_curves / curves / hsl / logWheel / colorCorrection 全部） | +| 调色链（65³ LUT） | maxAbs ≤ **2 LSB** | 2 LSB（对齐 §54 curves 的 65³ precedent 0.0086） | +| 曲线恒等 | **字节完全一致** | 256/256 texel | +| 分级 | 1 = 像素完全相同 | 1 LSB ⇒ 归一到 2 LSB | +| ~~改动可见性~~ | ~~max MAD > 阈值~~ | ⚠️ **作废**（123.51 @frame74 与噪声底同量级 ⇒ 不可信）。改用语义判据或噪声校准 | +| 真草稿 | sha 逐字节相同 | 64 files identical | + +## 调色的"生效"判据（三条同时成立才算生效） + +1. **逐节点探针**：每个节点单开 ⇒ MAD > 0 +2. **全链改变量**：`grade_change_mad > 0` +3. **空间分布合理**：改"亮"应该 mean 上升、**clip 比例不超阈值**（过曝是另一个问题，不是"调亮"） + +## 判据自检清单（每次写新判据时问自己） + +- [ ] 判据的视野覆盖被测对象了吗？（不是抽一帧） +- [ ] 有阴性对照吗？ +- [ ] 故意弄坏它会失败吗？ +- [ ] 探针能区分"没生效"和"没在测"吗？ +- [ ] 输出能自证是这次跑出来的吗？ +- [ ] ★ **像素判据的噪声底量了吗？**（同一状态重渲染 ⇒ 差异有多大？没有噪声底就别报"可见"） +- [ ] ★ **基线是【这次跑】产生的吗？**（借用别的跑的残留目录 ⇒ 测的是两次不同的改动） diff --git a/plugins/antianqi/jianying/scripts/check-env.mjs b/plugins/antianqi/jianying/scripts/check-env.mjs new file mode 100644 index 00000000..4853a847 --- /dev/null +++ b/plugins/antianqi/jianying/scripts/check-env.mjs @@ -0,0 +1,55 @@ +#!/usr/bin/env node +/** + * 环境自检（确定性护栏之一）—— 独立可跑，不依赖任何动词 + * 用法: node scripts/check-env.mjs + * + * 检查: + * 1. JYAI_HOME / 工具链是否就位 + * 2. ffmpeg / python 是否可解析 + * 3. 剪映运行时缓存（effect 包）是否存在 + * 4. 插件自身结构完整性（schema / references / skills） + */ +import fs from 'node:fs'; +import path from 'node:path'; +import os from 'node:os'; +import { fileURLToPath } from 'node:url'; + +const HERE = path.dirname(fileURLToPath(import.meta.url)); +const ROOT = path.resolve(HERE, '..'); +const JYAI = process.env.JYAI_HOME || path.join(os.homedir(), 'JYAI'); +let PASS = 0, FAIL = 0; +const ok = (n, d) => { PASS++; console.log(` [OK] ${n}${d ? ' — ' + d : ''}`); }; +const bad = (n, d) => { FAIL++; console.log(` [MISS] ${n}${d ? ' — ' + d : ''}`); }; + +console.log('=== 环境自检 ===\n'); + +console.log('[1] 工具链'); +for (const t of ['jyai-live.mjs', 'jyai-b-render.mjs', 'jyai-audio-mix.mjs', 'jyai-grade-render.py', 'jyai-gl-bake.py']) { + const p = path.join(JYAI, 'tools', t); + if (fs.existsSync(p)) ok(t, p); else bad(t, `${p} 不存在（设 JYAI_HOME）`); +} + +console.log('\n[2] 剪映运行时缓存'); +const cache = path.join(process.env.LOCALAPPDATA || path.join(os.homedir(), 'AppData', 'Local'), + 'JianyingPro', 'User Data', 'Cache'); +const effectPkg = path.join(cache, 'effect', '7416258989467374091', '9e1cfdca2b07a183049bfc9b5ed32895'); +if (fs.existsSync(effectPkg)) ok('effect 包（官方 shader 参照源）', effectPkg); +else bad('effect 包', `${effectPkg} 不存在`); + +console.log('\n[3] 插件结构'); +const tools = path.join(ROOT, 'mcp', 'tools'); +const schemas = fs.existsSync(tools) ? fs.readdirSync(tools).filter((f) => f.endsWith('.schema.json')) : []; +if (schemas.length === 8) ok('mcp/tools 下 8 份 schema', schemas.length + ' 份'); +else bad('mcp/tools 下 8 份 schema', `实际 ${schemas.length} 份`); +for (const f of ['mcp/server.mjs', 'mcp.json', 'plugin.json', 'README.md', + 'references/chain-api.md', 'references/verify-discipline.md', 'references/pitfalls.md', + 'references/capability-dict.json', 'scripts/run-verb.mjs']) { + if (fs.existsSync(path.join(ROOT, f))) ok(f); else bad(f, '缺失'); +} +const skillsDir = path.join(ROOT, 'skills'); +const skills = fs.existsSync(skillsDir) + ? fs.readdirSync(skillsDir).filter((d) => fs.existsSync(path.join(skillsDir, d, 'SKILL.md'))) : []; +ok('skills 目录', skills.length ? skills.join(', ') : '（暂无）'); + +console.log(`\n=== ${PASS} OK / ${FAIL} MISS ===`); +process.exit(FAIL === 0 ? 0 : 1); diff --git a/plugins/antianqi/jianying/scripts/run-verb.mjs b/plugins/antianqi/jianying/scripts/run-verb.mjs new file mode 100644 index 00000000..aa50667c --- /dev/null +++ b/plugins/antianqi/jianying/scripts/run-verb.mjs @@ -0,0 +1,88 @@ +#!/usr/bin/env node +/** + * 插件自带的 MCP 客户端 —— 让 skill 能【调动词】而不直接调 CLI（纪律 #1） + * + * 用法: + * node scripts/run-verb.mjs '' + * node scripts/run-verb.mjs grade_apply '{"src":"...","outDir":"...","spec":{"adjustColor":{"brightness":0.1}}}' + * node scripts/run-verb.mjs --list + * + * 它通过 stdio 连自己的 mcp/server.mjs ⇒ 走的是【和 mcode 加载插件时完全一样的动词】。 + */ +import { spawn } from 'node:child_process'; +import fs from 'node:fs'; +import os from 'node:os'; +import path from 'node:path'; +import { fileURLToPath } from 'node:url'; + +const HERE = path.dirname(fileURLToPath(import.meta.url)); +/* 插件根解析：本文件在插件里时 HERE/.. 就是插件根； + 被复制到别处（例如 ~/.minimax/agents/mavis/skills/... 当安装副本）时，走 JYAI_PLUGIN_ROOT / 默认仓库位置。 */ +function pluginRoot() { + const local = path.resolve(HERE, '..'); + if (fs.existsSync(path.join(local, 'mcp', 'server.mjs'))) return local; + const cands = [ + process.env.JYAI_PLUGIN_ROOT, + path.join(os.homedir(), 'MiniMax-Code-Plugins-1', 'plugins', 'antianqi', 'jianying'), + ].filter(Boolean); + for (const c of cands) if (fs.existsSync(path.join(c, 'mcp', 'server.mjs'))) return c; + return local; +} +const SRV = path.join(pluginRoot(), 'mcp', 'server.mjs'); + +const args = process.argv.slice(2); +if (!args.length || args[0] === '--list') { + const c = spawn(process.execPath, [SRV], { stdio: ['pipe', 'pipe', 'inherit'] }); + let buf = ''; + c.stdout.on('data', async (d) => { + buf += d.toString(); + let i; + while ((i = buf.indexOf('\n')) >= 0) { + const line = buf.slice(0, i); buf = buf.slice(i + 1); + if (!line.trim()) continue; + let m; try { m = JSON.parse(line); } catch { continue; } + if (m.id === 1) { + c.stdin.write(JSON.stringify({ jsonrpc: '2.0', id: 2, method: 'tools/list', params: {} }) + '\n'); + continue; + } + if (m.id === 2) { + for (const t of m.result?.tools || []) { + console.log(`${t.name}\n ${t.description}\n 入参: ${Object.keys(t.inputSchema?.properties || {}).join(', ')}\n`); + } + c.kill(); process.exit(0); + } + } + }); + c.stdin.write(JSON.stringify({ jsonrpc: '2.0', id: 1, method: 'initialize', + params: { protocolVersion: '2025-06-18', capabilities: {}, clientInfo: { name: 'run-verb', version: '1' } } }) + '\n'); +} else { + const verb = args[0]; + let params = {}; + try { params = args[1] ? JSON.parse(args[1]) : {}; } catch (e) { + console.error(`参数不是合法 JSON: ${e.message}`); process.exit(2); + } + const c = spawn(process.execPath, [SRV], { stdio: ['pipe', 'pipe', 'inherit'] }); + let buf = ''; + c.stdout.on('data', (d) => { + buf += d.toString(); + let i; + while ((i = buf.indexOf('\n')) >= 0) { + const line = buf.slice(0, i); buf = buf.slice(i + 1); + if (!line.trim()) continue; + let m; try { m = JSON.parse(line); } catch { continue; } + if (m.id === 1) { + c.stdin.write(JSON.stringify({ jsonrpc: '2.0', id: 2, method: 'tools/call', + params: { name: verb, arguments: params } }) + '\n'); + continue; + } + if (m.id === 2) { + for (const item of m.result?.content || []) console.log(item.text || ''); + const bad = m.result?.isError === true; + c.kill(); + process.exit(bad ? 1 : 0); + } + } + }); + c.stdin.write(JSON.stringify({ jsonrpc: '2.0', id: 1, method: 'initialize', + params: { protocolVersion: '2025-06-18', capabilities: {}, clientInfo: { name: 'run-verb', version: '1' } } }) + '\n'); +} diff --git a/plugins/antianqi/jianying/scripts/selftest-draft-edit.mjs b/plugins/antianqi/jianying/scripts/selftest-draft-edit.mjs new file mode 100644 index 00000000..6386d1e7 --- /dev/null +++ b/plugins/antianqi/jianying/scripts/selftest-draft-edit.mjs @@ -0,0 +1,247 @@ +#!/usr/bin/env node +/** + * jianying 插件：改草稿链路的端到端自检 + * draft_open（副本）→ draft_read（拿 segmentId）→ draft_patch（改）→ draft_commit（落盘 + 真草稿 sha 断言） + * → 读回落盘后的草稿 → verify_diff（语义：改动真的落盘了吗）→ render_frames → verify_diff（画面 + 噪声校准） + * + * ★ 2026-10-21 重写要点（三个上一版假绿/缺陷）： + * 1. 上一版的"改前帧"借的是【别的跑】留下的旧目录（out/p1-e2e-clean）⇒ 基线不是这次跑出来的。 + * 现在全部基线都【本次自产】。 + * 2. 上一版拿 `video.setVolume`（纯音频改动）去判"画面改动可见"⇒ 期望本身就错。 + * 现在主判据是【语义】：落盘后的草稿里 volume 字段确实变了（确定性，不依赖像素）。 + * 3. 官方渲染器【同态重渲染就有噪声】（实测 17/108 帧变、最差 MAD 123.55）⇒ 裸 MAD 阈值 0.5 + * 没有判别力。现在 verify_diff 支持 noiseDir 做噪声校准，并用"负向注入"证明这条判据 + * 既不会把噪声当改动、也不会把真改动当噪声。 + * + * 用法: node scripts/selftest-draft-edit.mjs + * 需要: JYAI_HOME 指向 JYAI 仓库；剪映在运行；有一个带视频片段的草稿 + */ +import { spawn } from 'node:child_process'; +import fs from 'node:fs'; +import os from 'node:os'; +import path from 'node:path'; +import { fileURLToPath } from 'node:url'; + +const HERE = path.dirname(fileURLToPath(import.meta.url)); +const JYAI = process.env.JYAI_HOME || path.join(os.homedir(), 'JYAI'); +// ★ 草稿路径不写死任何人的盘符：必须由调用方用 JYAI_TEST_DRAFT 指定。 +// 没给就明确 SKIP（而不是拿作者机器上的路径当默认值）。 +const DRAFT = process.env.JYAI_TEST_DRAFT || ''; +const WORK = path.join(JYAI, 'out', 'p67', 'draft-edit-test'); + +let PASS = 0, FAIL = 0; +const check = (n, ok, d) => { + if (ok) { PASS++; console.log(` [PASS] ${n}${d ? ' — ' + d : ''}`); } + else { FAIL++; console.log(` [FAIL] ${n}${d ? ' — ' + d : ''}`); } +}; + +// ---- 通过插件自带的 MCP 客户端调动词（与 mcode 加载插件时同一套动词）---- +function call(verb, args) { + return new Promise((resolve) => { + const c = spawn(process.execPath, [path.join(HERE, 'run-verb.mjs'), verb, JSON.stringify(args)], + { env: { ...process.env, JYAI_HOME: JYAI }, stdio: ['ignore', 'pipe', 'pipe'] }); + let so = '', se = ''; + c.stdout.on('data', (d) => { so += d; }); + c.stderr.on('data', (d) => { se += d; }); + c.on('close', (code) => { + let j = null; try { j = JSON.parse(so); } catch { /* 不是 JSON ⇒ 原样 */ } + resolve({ code, json: j, text: so, stderr: se }); + }); + }); +} + +// ---- ffmpeg（用来合成"噪声帧"与"真改动帧"，做负向注入）---- +function which(cmd) { + const exts = process.platform === 'win32' ? ['.exe', '.cmd', '.bat', ''] : ['']; + for (const d of (process.env.PATH || '').split(path.delimiter)) { + if (!d) continue; + for (const e of exts) { + const p = path.join(d, cmd + e); + try { if (fs.existsSync(p) && fs.statSync(p).isFile()) return p; } catch { /* ignore */ } + } + } + return cmd; +} +const FFMPEG = which('ffmpeg'); +function sh(args) { + return new Promise((resolve) => { + const c = spawn(FFMPEG, args, { stdio: ['ignore', 'pipe', 'pipe'] }); + let se = ''; + c.stderr.on('data', (d) => { se += d; }); + c.on('close', (code) => resolve({ code, stderr: se })); + }); +} +// 把 srcDir 的帧整体做一次亮度偏移，输出到 dstDir（用同一批 PNG，保持尺寸一致） +async function shiftFrames(srcDir, dstDir, brightness) { + fs.rmSync(dstDir, { recursive: true, force: true }); + fs.mkdirSync(dstDir, { recursive: true }); + const files = fs.readdirSync(srcDir).filter((f) => f.endsWith('.png')).sort(); + for (const f of files) { + await sh(['-y', '-v', 'error', '-i', path.join(srcDir, f), '-vf', `eq=brightness=${brightness}`, + '-frames:v', '1', path.join(dstDir, f)]); + } + return files.length; +} + +console.log('=== 改草稿链路端到端自检 ===\n'); +if (!DRAFT) { + console.log(' [SKIP] 需要 JYAI_TEST_DRAFT —— 指向一个带视频片段的剪映草稿目录。'); + console.log(' 例（PowerShell）: $env:JYAI_TEST_DRAFT = "<草稿库>\\<草稿名>"'); + console.log(' 需要剪映 11.6 在运行；本自检不修改真草稿（只在副本上操作）。'); + process.exit(0); +} +console.log(' 草稿:', DRAFT, '\n'); + +fs.mkdirSync(WORK, { recursive: true }); + +// ① draft_open（副本） +const open = await call('draft_open', { draft: DRAFT, copy: true }); +check('① draft_open 开副本', open.code === 0 && !!open.json?.sessionId, + open.json ? `sessionId=${open.json.sessionId} dir=${open.json.dir}` : open.text.slice(0, 160)); +if (!open.json?.sessionId) { console.log(open.text.slice(0, 400) + '\n' + open.stderr.slice(0, 400)); process.exit(1); } +const sid = open.json.sessionId; +const copyDir = open.json.dir; + +// ①b ★ 读【副本的草稿文件】（磁盘，自动解密）⇒ 落盘前的基线 +const readBefore = await call('draft_read', { path: path.join(copyDir, 'draft_content.json') }); +let beforeDraft = null; +try { beforeDraft = JSON.parse(readBefore.text).draft; } catch { /* ignore */ } +const beforeDraftFile = path.join(WORK, 'before-draft.json'); +if (beforeDraft) fs.writeFileSync(beforeDraftFile, JSON.stringify(beforeDraft), 'utf8'); +check('①b 读副本草稿（磁盘，自动解密）', readBefore.code === 0 && !!beforeDraft, + beforeDraft ? `tracks=${(beforeDraft.tracks || []).length} → ${beforeDraftFile}` : readBefore.text.slice(0, 160)); + +// ② draft_read 拿片段 +const read = await call('draft_read', { scope: 'segments' }); +check('② draft_read 读结构', read.code === 0, read.code === 0 ? `${read.text.length} 字符` : read.text.slice(0, 200)); +let segIds = []; +let trackIds = []; +try { + const j = JSON.parse(read.text); + const sum = j.summary || {}; + for (const tr of (sum.tracks || [])) { + if (tr.id) trackIds.push(tr.id); + for (const sg of (tr.segments || [])) if (sg.id) segIds.push(sg.id); + } + if (!segIds.length) { + const d = j.draft || j; + for (const tr of (d.tracks || [])) { + if (tr.id) trackIds.push(tr.id); + for (const sg of (tr.segments || [])) if (sg.id) segIds.push(sg.id); + } + } +} catch (e) { console.log(' (解析失败: ' + e.message + ')'); } +check('②b 读到 segmentId / trackId', segIds.length > 0 || trackIds.length > 0, + `segment ${segIds.length} 个 · track ${trackIds.length} 个${segIds.length ? '：' + segIds.slice(0, 2).join(', ') : ''}`); + +// ③ 阴性：缺必填参数必须被【本地】拦下（不白跑官方） +const bad = await call('draft_patch', { op: 'video.add', sessionId: sid, params: {} }); +check('★ ③ 缺必填参数被本地拦下', bad.code !== 0 && /缺少必填参数/.test(bad.text + bad.stderr), + (bad.text + bad.stderr).split('\n').slice(0, 2).join(' / ')); + +// ④ 阴性：未知 op 必须被 schema 拦下 +const badop = await call('draft_patch', { op: 'segment.mov', sessionId: sid, params: {} }); +check('★ ④ 未知 op 被 schema 拦下', badop.code !== 0 && /不合法|未知|not allowed/i.test(badop.text + badop.stderr), + (badop.text + badop.stderr).split('\n').slice(0, 2).join(' / ')); + +// ⑤ 真实改：给一个片段调音量（有 segmentId 时），否则用 track.mute / canUndo +let patchOp = null, patchParams = null; +if (segIds.length) { patchOp = 'video.setVolume'; patchParams = { segmentId: segIds[0], volume: 0.2 }; } +else if (trackIds.length) { patchOp = 'track.mute'; patchParams = { trackId: trackIds[0] }; } +else { patchOp = 'undoRedo.canUndo'; patchParams = {}; } +const patched = await call('draft_patch', { op: patchOp, sessionId: sid, params: patchParams }); +check(`⑤ draft_patch（${patchOp}）`, patched.code === 0, + patched.code === 0 ? `baseRevision=${patched.json?.baseRevision} 快照=${patched.json?.snapshotSaved ? '已存' : '未存'}` : (patched.text + patched.stderr).split('\n').slice(0, 2).join(' / ')); + +// ⑥ 落盘 + 真草稿 sha 断言 +const commit = await call('draft_commit', { sessionId: sid, realDraft: DRAFT }); +check('⑥ draft_commit 落盘 + 真草稿零改动', commit.code === 0 && commit.json?.realDraftUnchanged === true, + commit.code === 0 ? `encrypted=${commit.json.encrypted} cipherType=${commit.json.cipherType} relocated=${commit.json.relocated} realDraftUnchanged=${commit.json.realDraftUnchanged}` : (commit.text + commit.stderr).split('\n').slice(0, 2).join(' / ')); + +// ⑥b ★ 落盘后副本的草稿必须真的变了（否则 = 改动没落盘） +if (commit.code === 0 && copyDir) { + const fp = path.join(copyDir, 'draft_content.json'); + let changed = null; + try { changed = fs.statSync(fp).size > 0; } catch { /* ignore */ } + check('⑥b 落盘后草稿文件存在且非空', changed === true, fp); +} + +// ⑥c ★ 读回【已落盘】的副本草稿 ⇒ 语义基线（改后） +const readAfter = await call('draft_read', { path: path.join(copyDir, 'draft_content.json') }); +let afterDraft = null; +try { afterDraft = JSON.parse(readAfter.text).draft; } catch { /* ignore */ } +const afterDraftFile = path.join(WORK, 'after-draft.json'); +if (afterDraft) fs.writeFileSync(afterDraftFile, JSON.stringify(afterDraft), 'utf8'); +check('⑥c 读回落盘后的副本草稿', readAfter.code === 0 && !!afterDraft, + afterDraft ? `tracks=${(afterDraft.tracks || []).length} → ${afterDraftFile}` : readAfter.text.slice(0, 160)); + +// ⑦ ★★ 主判据（语义）：改动真的落到草稿文件里了吗？ +if (beforeDraft && afterDraft) { + const expect = patchOp === 'video.setVolume' ? { segmentFieldChanged: 'volume' } : { segmentsDelta: 0 }; + const v = await call('verify_diff', { + checks: ['timeline'], beforeDraft: beforeDraftFile, afterDraft: afterDraftFile, expect, + }); + let vj = null; try { vj = JSON.parse(v.text); } catch { /* ignore */ } + const tl = vj?.checks?.timeline || {}; + check('★★ ⑦ 语义判据：改动确实落盘（逐段字段差）', + v.code === 0 && vj?.verdict === 'PASS' && (tl.segmentDiffs || []).length > 0, + `verdict=${vj?.verdict} 段数Δ=${tl.segmentsDelta} 时长Δ=${tl.durationDeltaUs} 段字段差=${(tl.segmentDiffs || []).length}` + + (tl.segmentDiffs?.[0]?.fields?.volume ? ` volume: ${JSON.stringify(tl.segmentDiffs[0].fields.volume.before)} → ${JSON.stringify(tl.segmentDiffs[0].fields.volume.after)}` : '') + + (vj?.problems?.length ? ` problems=${JSON.stringify(vj.problems)}` : '')); + // ★ 负向注入：把 expect 换成"不该变的字段变了" ⇒ 必须 FAIL（证明这条判据不是恒真） + const neg = await call('verify_diff', { + checks: ['timeline'], beforeDraft: beforeDraftFile, afterDraft: afterDraftFile, + expect: { segmentFieldUnchanged: 'volume' }, + }); + let nj = null; try { nj = JSON.parse(neg.text); } catch { /* ignore */ } + check('★ ⑦b 负向注入：expect 说 volume 不该变 ⇒ 必须 FAIL', neg.code !== 0 && nj?.verdict === 'FAIL', + `verdict=${nj?.verdict} problems=${JSON.stringify(nj?.problems)}`); +} else { + console.log(' [SKIP] ⑦ 语义判据（没拿到改前/改后草稿）'); +} + +// ⑧ 渲染改后帧（本次自产） +const afterDir = path.join(WORK, 'after'); +const rend = await call('render_frames', { draft: copyDir, outDir: afterDir }); +check('⑧ render_frames 出帧', rend.code === 0, rend.code === 0 ? afterDir : (rend.text + rend.stderr).split('\n').slice(0, 2).join(' / ')); + +const framesDir = path.join(afterDir, 'frames'); +if (fs.existsSync(framesDir) && fs.readdirSync(framesDir).some((f) => f.endsWith('.png'))) { + // ⑧b ★ 探针自检：同一目录对同一目录 ⇒ 必须 FAIL（判据要有判别力） + const same = await call('verify_diff', { beforeDir: framesDir, afterDir: framesDir, checks: ['visibility'] }); + let sj = null; try { sj = JSON.parse(same.text); } catch { /* ignore */ } + check('★ ⑧b 探针自检：同目录对同目录 ⇒ 必须 FAIL', same.code !== 0 && sj?.verdict === 'FAIL', + `verdict=${sj?.verdict} maxMad=${sj?.checks?.visibility?.maxMad}`); + + // ⑧c ★★ 噪声校准 + 负向注入 + // 官方渲染器同态重渲染就有噪声（实测 17/108 帧变、最差 MAD 123.55）⇒ 裸阈值 0.5 无判别力。 + // 这里用【合成噪声】（整体亮度 −0.01，实测 MAD≈4.0）当噪声基线， + // 用【大改动】（−0.06，实测 MAD≈16.0）当真改动： + // 校准后的判据必须 ①把噪声判成"不可见" ②把真改动判成"可见"。 + const noiseDir = path.join(WORK, 'noise-frames'); + const bigDir = path.join(WORK, 'big-frames'); + const n = await shiftFrames(framesDir, noiseDir, -0.01); + const b = await shiftFrames(framesDir, bigDir, -0.06); + if (n && b) { + const vNoise = await call('verify_diff', { beforeDir: framesDir, afterDir: noiseDir, noiseDir, checks: ['visibility'] }); + let nj = null; try { nj = JSON.parse(vNoise.text); } catch { /* ignore */ } + const nv = nj?.checks?.visibility || {}; + check('★★ ⑧c 噪声校准：同态噪声 ⇒ 必须判「不可见」', + vNoise.code !== 0 && nj?.verdict === 'FAIL' && nv.calibrated === true, + `verdict=${nj?.verdict} maxMad=${nv.maxMad} 噪声底=${nv.noiseMad} 判据线=${nv.noiseBar}`); + + const vBig = await call('verify_diff', { beforeDir: framesDir, afterDir: bigDir, noiseDir, checks: ['visibility'] }); + let bj = null; try { bj = JSON.parse(vBig.text); } catch { /* ignore */ } + const bv = bj?.checks?.visibility || {}; + check('★★ ⑧d 负向注入：真改动（亮度 −0.06）⇒ 必须判「可见」', + vBig.code === 0 && bj?.verdict === 'PASS' && bv.calibrated === true, + `verdict=${bj?.verdict} maxMad=${bv.maxMad} 噪声底=${bv.noiseMad} 倍数=${bv.madOverNoise}`); + } else { + console.log(' [SKIP] ⑧c 噪声校准（合成帧失败）'); + } +} else { + console.log(' [SKIP] ⑧b/⑧c（没有帧目录）'); +} + +console.log(`\n=== ${PASS} PASS / ${FAIL} FAIL ===`); +process.exit(FAIL === 0 ? 0 : 1); diff --git a/plugins/antianqi/jianying/scripts/selftest-mcp.mjs b/plugins/antianqi/jianying/scripts/selftest-mcp.mjs new file mode 100644 index 00000000..f599db73 --- /dev/null +++ b/plugins/antianqi/jianying/scripts/selftest-mcp.mjs @@ -0,0 +1,191 @@ +// jianying MCP server 的客户端验收脚本 +// 用法: node selftest-mcp.mjs +// 环境: JYAI_HOME 指向 JYAI 仓库 +import { spawn } from 'node:child_process'; +import fs from 'node:fs'; +import path from 'node:path'; +import { fileURLToPath } from 'node:url'; + +const HERE = path.dirname(fileURLToPath(import.meta.url)); +const SRV = path.join(HERE, '..', 'mcp', 'server.mjs'); +const JYAI = process.env.JYAI_HOME || path.join(process.env.USERPROFILE, 'JYAI'); +const SRC = path.join(JYAI, 'out', 'p47', 'full1080-with-audio.mp4'); +const OUT = path.join(JYAI, 'out', 'p67', 'mcp-test'); +// ★ 2026-10-21：不再用【别的跑】留下的帧目录当基线（out/p1-e2e-clean）。 +// 官方渲染器同态重渲染就有噪声（实测 17/108 帧变、最差 MAD 123.55）， +// 拿它当"改动可见"的证据站不住脚 ⇒ 改成【本次自产】的合成帧，确定性可复跑。 +const FR_BASE = path.join(OUT, 'frames-base'); +const FR_NOISE = path.join(OUT, 'frames-noise'); +const FR_BIG = path.join(OUT, 'frames-big'); +const FR_CLIP = path.join(OUT, 'frames-clip'); + +function which(cmd) { + const exts = process.platform === 'win32' ? ['.exe', '.cmd', '.bat', ''] : ['']; + for (const d of (process.env.PATH || '').split(path.delimiter)) { + if (!d) continue; + for (const e of exts) { + const p = path.join(d, cmd + e); + try { if (fs.existsSync(p) && fs.statSync(p).isFile()) return p; } catch { /* ignore */ } + } + } + return cmd; +} +const FFMPEG = which('ffmpeg'); +function sh(args) { + return new Promise((resolve) => { + const c = spawn(FFMPEG, args, { stdio: ['ignore', 'pipe', 'pipe'] }); + let se = ''; + c.stderr.on('data', (d) => { se += d; }); + c.on('close', (code) => resolve({ code, stderr: se })); + }); +} +// 合成一组确定性的帧（纯色 + 亮度偏移）⇒ 噪声基线 / 真改动 / 新增削波 各一组 +// ★ 基色里放一个接近白的（F0F0F0）⇒ 用来复现"绝对值削波口径"那个坑（§76.5） +async function synthFrames(dir, brightness) { + fs.rmSync(dir, { recursive: true, force: true }); + fs.mkdirSync(dir, { recursive: true }); + const colors = ['F0F0F0', '606060', '808080']; + for (let i = 0; i < colors.length; i++) { + const f = path.join(dir, String(i).padStart(5, '0') + '.png'); + const args = ['-y', '-v', 'error', '-f', 'lavfi', '-i', `color=c=0x${colors[i]}:s=64x64`]; + if (brightness) args.push('-vf', `eq=brightness=${brightness}`); + args.push('-frames:v', '1', f); + const r = await sh(args); + if (r.code !== 0) return 0; + } + return colors.length; +} + +const child = spawn(process.execPath, [SRV], { + env: { ...process.env, JYAI_HOME: JYAI, PYTHONIOENCODING: 'utf-8', PYTHONUTF8: '1' }, + stdio: ['pipe', 'pipe', 'inherit'], +}); +let buf = ''; +const pending = new Map(); +child.stdout.on('data', (d) => { + buf += d.toString(); + let i; + while ((i = buf.indexOf('\n')) >= 0) { + const line = buf.slice(0, i); buf = buf.slice(i + 1); + if (!line.trim()) continue; + let m; try { m = JSON.parse(line); } catch { continue; } + if (m.id !== undefined && pending.has(m.id)) { pending.get(m.id)(m); pending.delete(m.id); } + } +}); +let id = 0; +function call(method, params) { + const myId = ++id; + return new Promise((resolve) => { + pending.set(myId, resolve); + child.stdin.write(JSON.stringify({ jsonrpc: '2.0', id: myId, method, params }) + '\n'); + }); +} + +let PASS = 0, FAIL = 0; +function check(name, ok, detail) { + if (ok) { PASS++; console.log(` [PASS] ${name}${detail ? ' — ' + detail : ''}`); } + else { FAIL++; console.log(` [FAIL] ${name}${detail ? ' — ' + detail : ''}`); } +} + +const textOf = (r) => (r.result?.content || []).map((c) => c.text || '').join('\n'); + +(async () => { + console.log('=== jianying MCP server 验收 ===\n'); + + // 1. 握手 + 工具清单 + const init = await call('initialize', { protocolVersion: '2025-06-18', capabilities: {}, clientInfo: { name: 'test', version: '1' } }); + check('initialize 握手', init.result?.serverInfo?.name === 'jianying', `serverInfo=${JSON.stringify(init.result?.serverInfo)}`); + const list = await call('tools/list', {}); + const tools = list.result?.tools || []; + check('tools/list 返回 8 个动词', tools.length === 8, `实际 ${tools.length}：${tools.map((t) => t.name).join(', ')}`); + const noSchema = tools.filter((t) => !t.inputSchema || t.inputSchema.type !== 'object'); + check('每个动词都有 inputSchema', noSchema.length === 0, noSchema.map((t) => t.name).join(',') || '全部有'); + const ga = tools.find((t) => t.name === 'grade_apply'); + const nodeKeys = Object.keys(ga?.inputSchema?.properties?.spec?.properties || {}); + check('grade_apply 的 spec 枚举了节点', nodeKeys.length >= 10, `${nodeKeys.length} 个节点：${nodeKeys.join(',')}`); + + // 2. 入参 schema 硬校验：未知键必须被拒（防静默失效） + const bad = await call('tools/call', { name: 'grade_apply', arguments: { src: SRC, outDir: OUT, spec: { adjustColor: { brightness: 0.1 }, not_a_node: { x: 1 } } } }); + const badTxt = textOf(bad); + check('★ 非法 spec 键被拒绝', bad.result?.isError === true && /未知键|不合法/.test(badTxt), badTxt.split('\n').slice(0, 2).join(' / ')); + + const bad2 = await call('tools/call', { name: 'grade_apply', arguments: { src: SRC, outDir: OUT, spec: { adjustColor: { brightness: 5 } } } }); + check('★ 越界参数被拒绝', bad2.result?.isError === true && /最大|不合法/.test(textOf(bad2)), textOf(bad2).split('\n').slice(0, 2).join(' / ')); + + // 3. 正常调色：必须出片 + 探针全非零 + const g = await call('tools/call', { name: 'grade_apply', arguments: { + src: SRC, outDir: OUT, frames: 4, + spec: { adjustColor: { brightness: 0.12, contrast: 0.08 }, logWheel: { shadow: { r: 0.10, g: 0.04, b: 0, s: 0.05 } } }, + } }); + const gTxt = textOf(g); + const gOk = !g.result?.isError; + check('grade_apply 正常出片', gOk, gOk ? gTxt.replace(/\s+/g, ' ').slice(0, 150) : gTxt.slice(0, 200)); + if (gOk) { + const j = JSON.parse(gTxt); + check('grade_apply 内建探针：所有节点非零', Object.entries(j.probe).filter(([k]) => k !== '_full_chain').every(([, v]) => v > 0), JSON.stringify(j.probe)); + check('grade_apply 调色改变量 > 0', j.gradeChangeMad > 0, `MAD=${j.gradeChangeMad}`); + } + + // 4. verify_diff：本次自产的合成帧（不依赖别的跑的残留目录） + // base = 原帧；noise = 亮度 −0.01（模拟渲染噪声）；big = 亮度 −0.06（模拟真改动） + const nBase = await synthFrames(FR_BASE, 0); + const nNoise = await synthFrames(FR_NOISE, -0.01); + const nBig = await synthFrames(FR_BIG, -0.06); + const nClip = await synthFrames(FR_CLIP, 0.2); // ★ 0xF0F0F0 @ +0.2 ⇒ 255 ⇒ 真削波（实测 +0.03 只到 244，不削） + check('合成帧目录就绪', nBase > 0 && nNoise > 0 && nBig > 0 && nClip > 0, + `base=${nBase} noise=${nNoise} big=${nBig} clip=${nClip}`); + + // 4a. 未校准：噪声也会被判成"可见"（这正是要暴露的问题） + const v0 = await call('tools/call', { name: 'verify_diff', arguments: { + beforeDir: FR_BASE, afterDir: FR_NOISE, checks: ['visibility'] } }); + const v0j = (() => { try { return JSON.parse(textOf(v0)); } catch { return null; } })(); + const vis0 = v0j?.checks?.visibility || {}; + check('verify_diff 未校准时会标注 calibrated=false', + vis0.calibrated === false && /未校准/.test(vis0.note || ''), + `maxMad=${vis0.maxMad} threshold=${vis0.threshold} note=${(vis0.note || '').slice(0, 40)}…`); + + // 4b. ★ 校准后：噪声 ⇒ 必须判"不可见"；且【白底不算新增削波】（增量口径） + const v1 = await call('tools/call', { name: 'verify_diff', arguments: { + beforeDir: FR_BASE, afterDir: FR_NOISE, noiseDir: FR_NOISE, checks: ['visibility', 'picture'] } }); + const v1j = (() => { try { return JSON.parse(textOf(v1)); } catch { return null; } })(); + const vis = v1j?.checks?.visibility || {}; + check('★ verify_diff 校准后：噪声 ⇒ 判 FAIL（不可见）', + v1.result?.isError === true && v1j?.verdict === 'FAIL' && vis.calibrated === true, + `maxMad=${vis.maxMad} 噪声底=${vis.noiseMad} 判据线=${vis.noiseBar}`); + const pic = v1j?.checks?.picture || {}; + check('verify_diff 画质维度：白底不算【新增】削波（增量口径）', + Array.isArray(pic.meanDrift) && pic.meanDrift.length === 3 + && Math.abs(pic.clipDelta ?? 1) < 1e-6 && Math.abs(pic.underDelta ?? 1) < 1e-6 + && !(v1j?.problems || []).some((p) => /削波|死黑/.test(p)), + `clip=${pic.clipRatio} clipΔ=${pic.clipDelta} underΔ=${pic.underDelta} drift=${JSON.stringify(pic.meanDrift)}`); + + // 4c. ★★ 负向注入：真改动 ⇒ 必须判"可见"（证明这条判据不是恒 FAIL） + const v1b = await call('tools/call', { name: 'verify_diff', arguments: { + beforeDir: FR_BASE, afterDir: FR_BIG, noiseDir: FR_NOISE, checks: ['visibility'] } }); + const v1bj = (() => { try { return JSON.parse(textOf(v1b)); } catch { return null; } })(); + const visb = v1bj?.checks?.visibility || {}; + check('★★ verify_diff 负向注入：真改动 ⇒ 判 PASS（可见）', + v1b.result?.isError !== true && v1bj?.verdict === 'PASS' && visb.calibrated === true, + `maxMad=${visb.maxMad} 噪声底=${visb.noiseMad} 倍数=${visb.madOverNoise}`); + + // 4d. ★★ 负向注入：把白底【推上去】⇒ 必须报"新增削波"（证明增量口径不是恒 0） + const v1c = await call('tools/call', { name: 'verify_diff', arguments: { + beforeDir: FR_BASE, afterDir: FR_CLIP, noiseDir: FR_NOISE, checks: ['visibility', 'picture'] } }); + const v1cj = (() => { try { return JSON.parse(textOf(v1c)); } catch { return null; } })(); + const picc = v1cj?.checks?.picture || {}; + check('★★ verify_diff 负向注入：白底被推爆 ⇒ 必须报「新增削波」', + (picc.clipDelta ?? 0) > 0.02 && (v1cj?.problems || []).some((p) => /削波/.test(p)), + `clip=${picc.clipRatio} clipΔ=${picc.clipDelta} problems=${JSON.stringify(v1cj?.problems)}`); + + // 5. ★ 阴性：同一目录对同一目录 ⇒ 必须 FAIL（判据要有判别力） + const v2 = await call('tools/call', { name: 'verify_diff', arguments: { beforeDir: FR_BASE, afterDir: FR_BASE } }); + check('★ verify_diff 对无变化必须 FAIL', v2.result?.isError === true, textOf(v2).replace(/\s+/g, ' ').slice(0, 140)); + + // 6. draft_open 护栏：allowReal=true 但 copy 仍为 true ⇒ 拒绝 + const d1 = await call('tools/call', { name: 'draft_open', arguments: { draft: 'x', copy: true, allowReal: true } }); + check('★ draft_open 护栏：allowReal 与 copy 冲突时拒绝', d1.result?.isError === true, textOf(d1).slice(0, 120)); + + console.log(`\n=== ${PASS} PASS / ${FAIL} FAIL ===`); + child.kill(); + process.exit(FAIL === 0 ? 0 : 1); +})(); diff --git a/plugins/antianqi/jianying/scripts/selftest-talk-cut.mjs b/plugins/antianqi/jianying/scripts/selftest-talk-cut.mjs new file mode 100644 index 00000000..03fd3a02 --- /dev/null +++ b/plugins/antianqi/jianying/scripts/selftest-talk-cut.mjs @@ -0,0 +1,82 @@ +#!/usr/bin/env node +/** + * jianying-talk-cut 自检：规划器判定 + 落稿链 + * ① 规划器：水词/长停顿/重复 三类判定 + 保留区间算对 + * ② 落稿链：split×2+delete ⇒ 时间线正好缩短 + * 用法: node scripts/selftest-talk-cut.mjs + */ +import { spawn, spawnSync } from 'node:child_process'; +import fs from 'node:fs'; +import os from 'node:os'; +import path from 'node:path'; +import { fileURLToPath } from 'node:url'; + +const HERE = path.dirname(fileURLToPath(import.meta.url)); +const JYAI = process.env.JYAI_HOME || path.join(os.homedir(), 'JYAI'); +const WORK = path.join(JYAI, 'out', 'p67', 'talk-cut-selftest'); +// ★ 草稿路径不写死任何人的盘符：必须由调用方用 JYAI_TEST_DRAFT 指定。 +// 没给时，① 规划器照跑（纯离线），只有 ② 落稿链（需要活的剪映）SKIP。 +const DRAFT = process.env.JYAI_TEST_DRAFT || ''; + +let PASS = 0, FAIL = 0; +const check = (n, ok, d) => { + if (ok) { PASS++; console.log(` [PASS] ${n}${d ? ' — ' + d : ''}`); } + else { FAIL++; console.log(` [FAIL] ${n}${d ? ' — ' + d : ''}`); } +}; +fs.mkdirSync(WORK, { recursive: true }); + +/* ---------- 造一份含水词/长停顿/重复的 SRT ---------- */ +const SRT = path.join(WORK, 'test.srt'); +fs.writeFileSync(SRT, [ + '1', '00:00:00,500 --> 00:00:03,200', '嗯，那个，今天我们来聊一聊这个青椒的辣度。', '', + '2', '00:00:03,400 --> 00:00:05,800', '其实青椒肉丝是特别经典的一道家常菜。', '', + '3', '00:00:06,900 --> 00:00:09,000', '呃，那个，我个人非常喜欢这个菜。', '', + '4', '00:00:09,100 --> 00:00:12,000', '其实青椒肉丝是特别经典的一道家常菜。', '', + '5', '00:00:13,800 --> 00:00:16,200', '这道菜的关键是青椒要够脆、肉丝要够嫩。', '', + '6', '00:00:16,400 --> 00:00:18,900', '嗯，就是说，火候一定要掌握好。', '', + '7', '00:00:19,300 --> 00:00:20,100', '嗯，呃。', '', +].join('\n'), 'utf8'); + +console.log('=== talk-cut 自检 ===\n'); + +/* ---------- ① 规划器 ---------- */ +const planFile = path.join(WORK, 'plan.json'); +const p = spawnSync(process.execPath, [path.join(HERE, 'talk-cut-plan.mjs'), + '--srt', SRT, '--out', planFile], { encoding: 'utf8' }); +check('① 规划器跑通', p.status === 0, p.stdout.split('\n').filter((l) => l.includes('drop ')).join('').trim()); +const plan = JSON.parse(fs.readFileSync(planFile, 'utf8')); +const byIdx = (i) => plan.cues.find((c) => c.index === i); + +check('①a 纯水词句判 drop', byIdx(7)?.verdict === 'drop', `#7 = ${byIdx(7)?.verdict}（${byIdx(7)?.reasons?.join('/')}）`); +check('①b 含水词但有实词 → review', byIdx(1)?.verdict === 'review', `#1 = ${byIdx(1)?.verdict}`); +check('①c 隔一条的重复被检出', (plan.duplicates || []).some((d) => d.cue === 4 && d.ofCue === 2), + JSON.stringify(plan.duplicates)); +check('①d 长停顿被检出', (plan.gaps || []).length >= 2, `${(plan.gaps || []).length} 处`); +check('①e 词级水词有时间定位', (byIdx(1)?.fillerSpans || []).length > 0 && byIdx(1).fillerSpans[0].endMs > byIdx(1).fillerSpans[0].startMs, + JSON.stringify((byIdx(1)?.fillerSpans || []).slice(0, 2))); +const keepTotal = plan.keepRanges.reduce((a, r) => a + (r.endMs - r.startMs), 0); +const dropTotal = plan.dropRanges.reduce((a, r) => a + (r.endMs - r.startMs), 0); +check('①f 保留 + 删除 = 总时长', keepTotal + dropTotal === plan.stats.totalMs, + `${keepTotal} + ${dropTotal} = ${keepTotal + dropTotal} vs ${plan.stats.totalMs}`); + +/* ---------- ② 落稿链（需要活的剪映 + 一个真实草稿）---------- */ +if (!DRAFT) { + console.log(' [SKIP] ② 落稿链 —— 需要 JYAI_TEST_DRAFT（指向一个带视频片段的剪映草稿目录）且剪映在运行。'); + console.log(' 例（PowerShell）: $env:JYAI_TEST_DRAFT = "<草稿库>\\<草稿名>"'); +} else { + const cutPlan = path.join(WORK, 'cut-plan.json'); + fs.writeFileSync(cutPlan, JSON.stringify({ + ...plan, + dropRanges: [{ startMs: 1000, endMs: 2000, reason: '自检：剪掉第一段中间 1 秒' }], + }, null, 2), 'utf8'); + + const a = spawnSync(process.execPath, [path.join(HERE, 'talk-cut-apply.mjs'), + '--plan', cutPlan, '--draft', DRAFT, '--apply'], { encoding: 'utf8', timeout: 600000 }); + const out = a.stdout || ''; + check('② 落稿链跑通', a.status === 0, out.split('\n').filter((l) => l.includes('时间线判定') || l.includes('[OK]') || l.includes('落盘')).join(' | ').trim()); + check('②a 时间线正好缩短剪掉的长度', /时间线判定: OK/.test(out), (out.match(/# 时间线判定.*/) || [''])[0].trim()); + check('②b 真草稿零改动', /realDraftUnchanged=true/.test(out), (out.match(/# 落盘:.*/) || [''])[0].trim()); +} + +console.log(`\n=== ${PASS} PASS / ${FAIL} FAIL ===`); +process.exit(FAIL === 0 ? 0 : 1); diff --git a/plugins/antianqi/jianying/scripts/talk-cut-apply.mjs b/plugins/antianqi/jianying/scripts/talk-cut-apply.mjs new file mode 100644 index 00000000..5f53404b --- /dev/null +++ b/plugins/antianqi/jianying/scripts/talk-cut-apply.mjs @@ -0,0 +1,132 @@ +#!/usr/bin/env node +/** + * talk-cut 应用器：把 plan 里的 dropRanges 真的剪进草稿 + * + * ★ 剪"中间一段"的正确链（实测，2026-10-21）： + * ① segment.split {segmentId, splitTime: t1} → 切成两段，【后半段拿新 id】 + * ※ split 会让【后面的段整体后移】⇒ 所以必须【从后往前】处理多个区间 + * ② segment.split {segmentId: 前半id, splitTime: t0} → 前半再切两半 + * ③ segment.delete {segmentId: 中间段id} → 时间线缩短 (t1 - t0) + * 实测：时间线 9033333 → 8033333 us（正好剪掉 1 秒）✓ + * + * 用法: + * node scripts/talk-cut-apply.mjs --plan --draft "<草稿名>" [--apply] + * --apply 真的改草稿（默认只演练，只打印每一步计划） + * 草稿名可以是绝对路径。 + */ +import { spawn } from 'node:child_process'; +import fs from 'node:fs'; +import path from 'node:path'; +import { fileURLToPath } from 'node:url'; + +const HERE = path.dirname(fileURLToPath(import.meta.url)); +const argv = process.argv.slice(2); +const arg = (k, d) => { const i = argv.indexOf(k); return i >= 0 && argv[i + 1] ? argv[i + 1] : d; }; +const has = (k) => argv.includes(k); + +const PLAN = arg('--plan', null); +const DRAFT = arg('--draft', null); +const APPLY = has('--apply'); +if (!PLAN || !DRAFT) { + console.error('用法: node talk-cut-apply.mjs --plan --draft "<草稿名>" [--apply]'); + process.exit(2); +} + +/* ---- 走插件自带的 MCP 客户端（与 mcode 加载插件时同一套动词）---- */ +function call(verb, args) { + return new Promise((resolve) => { + const c = spawn(process.execPath, [path.join(HERE, 'run-verb.mjs'), verb, JSON.stringify(args)], + { stdio: ['ignore', 'pipe', 'pipe'] }); + let so = '', se = ''; + c.stdout.on('data', (d) => { so += d; }); + c.stderr.on('data', (d) => { se += d; }); + c.on('close', (code) => { + let j = null; try { j = JSON.parse(so); } catch { /* not JSON */ } + resolve({ code, json: j, text: so, stderr: se, failed: code !== 0 || j?.isError === true }); + }); + }); +} + +const segsOf = (summary) => { + const out = []; + for (const t of (summary?.tracks || [])) for (const s of (t.segments || [])) { + out.push({ track: t.type, id: s.id, start: s.target_timerange?.start || 0, dur: s.target_timerange?.duration || 0 }); + } + return out; +}; +const totalUs = (segs) => segs.reduce((a, s) => Math.max(a, s.start + s.dur), 0); + +const plan = JSON.parse(fs.readFileSync(PLAN, 'utf8')); +const drops = (plan.dropRanges || []).slice().sort((a, b) => b.startMs - a.startMs); // ★ 从后往前 +console.log('=== talk-cut 应用 ==='); +console.log(` 计划: ${drops.length} 个待删区间 · 总时长 ${(plan.stats.totalMs / 1000).toFixed(1)}s`); +console.log(' 模式: %s', APPLY ? '★ 真改草稿' : '演练（只打印）'); +console.log(' 草稿: %s\n', DRAFT); + +if (!drops.length) { console.log('没有要删的区间（plan 里 dropRanges 为空）'); process.exit(0); } + +const open = await call('draft_open', { draft: DRAFT, copy: true }); +if (open.failed) { console.error('打开草稿失败：\n' + (open.text + open.stderr).slice(0, 400)); process.exit(3); } +const sid = open.json.sessionId; +const copyDir = open.json.dir; +console.log('# session', sid, '\n'); + +let readRes = await call('draft_read', { scope: 'segments', sessionId: sid }); +let segs = segsOf(readRes.json?.summary); +const len0 = totalUs(segs); +console.log('# 初始: %d 段 · 时间线 %d us\n', segs.length, len0); + +const done = []; +for (const d of drops) { + const t0 = d.startMs * 1000, t1 = d.endMs * 1000; // ms → us + const cover = segs.find((s) => s.start <= t0 + 1 && s.start + s.dur >= t1 - 1); + if (!cover) { console.log(' [SKIP] 区间 %d..%d us 找不到覆盖它的段', t0, t1); continue; } + const planText = `split(${cover.id.slice(0, 8)}, ${t1}) → split(front, ${t0}) → delete(mid)`; + if (!APPLY) { console.log(' [DRY] %d..%d us ← %s', t0, t1, planText); done.push({ t0, t1, dry: true }); continue; } + + // ① split @ t1 + let r = await call('draft_patch', { op: 'segment.split', sessionId: sid, params: { segmentId: cover.id, splitTime: t1 } }); + if (r.failed) { console.log(` [FAIL] split@${t1}：${(r.text + r.stderr).slice(0, 160)}`); continue; } + readRes = await call('draft_read', { scope: 'segments', sessionId: sid }); + segs = segsOf(readRes.json?.summary); + const front = segs.find((s) => s.id === cover.id); + console.log(` ① split@${t1} → ${segs.length} 段`, segs.map((s) => `${s.id.slice(0, 6)}@${s.start}+${s.dur}`).join(' ')); + if (!front) { console.log(' [FAIL] split 后找不到前半段'); continue; } + + // ② split @ t0 + r = await call('draft_patch', { op: 'segment.split', sessionId: sid, params: { segmentId: front.id, splitTime: t0 } }); + if (r.failed) { console.log(` [FAIL] split@${t0}：${(r.text + r.stderr).slice(0, 160)}`); continue; } + readRes = await call('draft_read', { scope: 'segments', sessionId: sid }); + segs = segsOf(readRes.json?.summary); + console.log(` ② split@${t0} → ${segs.length} 段`, segs.map((s) => `${s.id.slice(0, 6)}@${s.start}+${s.dur}`).join(' ')); + const mid = segs.find((s) => s.id !== front.id && s.start >= t0 - 1 && s.start + s.dur <= t1 + 1 && s.dur > 0); + if (!mid) { console.log(` [FAIL] 找不到中间段（期望 start>=${t0 - 1} 且 start+dur<=${t1 + 1}）`); continue; } + + // ③ delete 中间段 + r = await call('draft_patch', { op: 'segment.delete', sessionId: sid, params: { segmentId: mid.id } }); + if (r.failed) { console.log(' [FAIL] delete：%s', (r.text + r.stderr).slice(0, 160)); continue; } + readRes = await call('draft_read', { scope: 'segments', sessionId: sid }); + segs = segsOf(readRes.json?.summary); + console.log(' [OK] 剪掉 %d..%d us（%d us）· 现在 %d 段 / 时间线 %d us', + t0, t1, t1 - t0, segs.length, totalUs(segs)); + done.push({ t0, t1, ok: true }); +} + +readRes = await call('draft_read', { scope: 'segments', sessionId: sid }); +segs = segsOf(readRes.json?.summary); +const len1 = totalUs(segs); +const cut = drops.reduce((a, d) => a + (d.endMs - d.startMs) * 1000, 0); +if (!APPLY) { + console.log(`\n# 演练结束：没改草稿（时间线仍 ${len1} us）。加 --apply 真改。`); +} else { + console.log(`\n# 结果: ${segs.length} 段 · 时间线 ${len1} us（初始 ${len0} us）· 应缩短 ${cut} us`); + const expect = len0 - cut; + console.log(`# 时间线判定: ${len1 === expect ? 'OK' : `✗ 实际 ${len1}`}（期望 ${expect}）`); +} + +if (APPLY && done.length) { + const cm = await call('draft_commit', { sessionId: sid, realDraft: DRAFT }); + console.log('# 落盘: %s', cm.failed ? ('FAIL ' + (cm.text + cm.stderr).slice(0, 200)) + : `encrypted=${cm.json.encrypted} cipherType=${cm.json.cipherType} relocated=${cm.json.relocated} realDraftUnchanged=${cm.json.realDraftUnchanged}`); + console.log('# 副本目录: %s', copyDir); +} diff --git a/plugins/antianqi/jianying/scripts/talk-cut-plan.mjs b/plugins/antianqi/jianying/scripts/talk-cut-plan.mjs new file mode 100644 index 00000000..0324f349 --- /dev/null +++ b/plugins/antianqi/jianying/scripts/talk-cut-plan.mjs @@ -0,0 +1,227 @@ +#!/usr/bin/env node +/** + * talk-cut 规划器：转写稿(SRT) → 保留区间表 + * + * 这是"口播剪辑"的【经验层】：判断哪句是水词、哪里是长停顿、哪句是重复。 + * ★ 官方 ASR(AnalyzeAsset) 在 agent runtime 里、我们拿不到（见 §74.4）， + * 所以本脚本吃的是【用户提供的 SRT / 转写稿】。 + * + * 用法: + * node scripts/talk-cut-plan.mjs --srt [--out ] + * [--filler-ms 700] [--dup-sim 0.72] [--keep-repeats] + * + * 判定分三级（保守优先）： + * drop 整条只由水词/标点组成 ⇒ 建议整句删 + * review 句中有实词但夹水词 ⇒ 标出水词位置，建议人工确认 + * keep 正常内容 + */ +import fs from 'node:fs'; +import path from 'node:path'; + +const argv = process.argv.slice(2); +const arg = (k, d) => { const i = argv.indexOf(k); return i >= 0 && argv[i + 1] ? argv[i + 1] : d; }; +const has = (k) => argv.includes(k); + +const SRT = arg('--srt', null); +if (!SRT) { console.error('用法: node talk-cut-plan.mjs --srt [--out plan.json]'); process.exit(2); } +const OUT = arg('--out', null); +const FILLER_MS = Number(arg('--filler-ms', 700)); // 长停顿阈值 +const DUP_SIM = Number(arg('--dup-sim', 0.72)); // 重复判定相似度 +const KEEP_REPEATS = has('--keep-repeats'); +const DUP_WINDOW = Number(arg('--dup-window', 3)); // 重复检测窗口（往前看几条） + +/* 默认水词表（中文口播）：可 --filler-file 覆盖 */ +const DEFAULT_FILLERS = [ + '嗯', '呃', '额', '啊', '哦', '噢', '唉', '欸', '诶', '唔', + '那个', '这个', '就是', '然后', '所以说', '怎么说', '对吧', '是吧', '你知道', + '怎么说呢', '对不对', '好不好', '行不行', 'emmm', 'umm', 'ah', 'eh', 'hmm', 'uh', +]; +const FILLER_FILE = arg('--filler-file', null); +const FILLERS = FILLER_FILE + ? fs.readFileSync(FILLER_FILE, 'utf8').split(/\r?\n/).map((s) => s.trim()).filter(Boolean) + : DEFAULT_FILLERS; + +/* ---------------- SRT 解析 ---------------- */ +function tsToMs(s) { + const m = /^(\d+):(\d{2}):(\d{2})[,.](\d{1,3})$/.exec(s.trim()); + if (!m) return null; + return (+m[1]) * 3600000 + (+m[2]) * 60000 + (+m[3]) * 1000 + (+m[4].padEnd(3, '0')); +} +function parseSrt(text) { + const cues = []; + const blocks = text.replace(/\r\n/g, '\n').split(/\n{2,}/); + for (const b of blocks) { + const lines = b.split('\n').map((l) => l.trim()).filter(Boolean); + if (!lines.length) continue; + let idx = -1; + let ti = lines.findIndex((l) => l.includes('-->')); + if (ti < 0) continue; + if (ti > 0 && /^\d+$/.test(lines[0])) { idx = +lines[0]; ti = ti; } + const [a, z] = lines[ti].split('-->'); + const startMs = tsToMs(a), endMs = tsToMs(z); + if (startMs === null || endMs === null) continue; + const body = lines.slice(ti + 1).join(' ').trim(); + cues.push({ index: idx >= 0 ? idx : cues.length + 1, startMs, endMs, text: body }); + } + return cues; +} + +/* ---------------- 文本工具 ---------------- */ +const PUNCT = /[\s,，。！？、；：""''（）()《》<>\-—…·.!?;:"']/g; +const norm = (s) => s.replace(PUNCT, ''); +function bigrams(s) { + const t = norm(s); + const set = new Set(); + for (let i = 0; i < t.length - 1; i++) set.add(t.slice(i, i + 2)); + if (t.length === 1) set.add(t); + return set; +} +function jaccard(a, b) { + const A = bigrams(a), B = bigrams(b); + if (!A.size || !B.size) return 0; + let inter = 0; + for (const x of A) if (B.has(x)) inter++; + return inter / (A.size + B.size - inter); +} +function findFillers(text) { + const hits = []; + for (const f of FILLERS) { + let from = 0; + for (;;) { + const i = text.indexOf(f, from); + if (i < 0) break; + hits.push({ text: f, at: i }); + from = i + f.length; + } + } + hits.sort((a, b) => a.at - b.at); + // 去掉被更长水词覆盖的短命中 + const kept = []; + for (const h of hits) { + if (kept.some((k) => h.at >= k.at && h.at < k.at + k.text.length)) continue; + kept.push(h); + } + return kept; +} + +/* ---------------- 主流程 ---------------- */ +const raw = fs.readFileSync(SRT, 'utf8'); +const cues = parseSrt(raw); +if (!cues.length) { console.error('SRT 里没解析出任何字幕条目'); process.exit(3); } + +const outCues = []; +const dropRanges = []; +const dupOf = []; +const gaps = []; + +for (let i = 0; i < cues.length; i++) { + const c = cues[i]; + const content = norm(c.text); + const fh = findFillers(c.text); + const reasons = []; + + // ① 长停顿（相对上一条） + if (i > 0) { + const gap = c.startMs - cues[i - 1].endMs; + if (gap >= FILLER_MS) { + gaps.push({ startMs: cues[i - 1].endMs, endMs: c.startMs, ms: gap, afterCue: cues[i - 1].index }); + reasons.push(`长停顿 ${gap}ms`); + } + } + + // ② 重复（与最近 DUP_WINDOW 条比 —— 口播里"重录"常常隔一句，不只相邻） + if (i > 0 && !KEEP_REPEATS) { + let bestSim = 0, bestJ = -1; + for (let k = Math.max(0, i - DUP_WINDOW); k < i; k++) { + const sim = jaccard(c.text, cues[k].text); + if (sim > bestSim) { bestSim = sim; bestJ = k; } + } + if (bestSim >= DUP_SIM) { + reasons.push(`与第${cues[bestJ].index}条重复（相似度 ${bestSim.toFixed(2)}）`); + dupOf.push({ cue: c.index, ofCue: cues[bestJ].index, similarity: +bestSim.toFixed(3) }); + } + } + + // ③ 水词 + if (fh.length) reasons.push(`含水词：${fh.map((f) => f.text).join('、')}`); + + // 判定 + let verdict = 'keep'; + const onlyFiller = fh.length > 0 && content.replace(new RegExp(FILLERS.map(escapeRe).join('|'), 'g'), '').length === 0; + if (onlyFiller && fh.length) { + verdict = 'drop'; + reasons.unshift('整条只有水词'); + } else if (reasons.length) { + verdict = 'review'; + } + + // 词级水词时间（按字符比例估算） + const fillerSpans = fh.map((f) => { + const ratio = c.text.length ? f.at / c.text.length : 0; + const span = c.text.length ? f.text.length / c.text.length : 0; + return { text: f.text, at: f.at, + startMs: Math.round(c.startMs + (c.endMs - c.startMs) * ratio), + endMs: Math.round(c.startMs + (c.endMs - c.startMs) * (ratio + span)) }; + }); + + outCues.push({ index: c.index, startMs: c.startMs, endMs: c.endMs, text: c.text, + verdict, reasons, fillerSpans }); + if (verdict === 'drop') dropRanges.push({ startMs: c.startMs, endMs: c.endMs, reason: reasons.join(' / ') }); +} +function escapeRe(s) { return s.replace(/[.*+?^${}()|[\]\\]/g, '\\$&'); } + +/* 合并保留区间（把 drop 区间从总区间里挖掉） */ +function mergeRanges(rs) { + const s = rs.slice().sort((a, b) => a.startMs - b.startMs); + const out = []; + for (const r of s) { + if (out.length && r.startMs <= out[out.length - 1].endMs) { + out[out.length - 1].endMs = Math.max(out[out.length - 1].endMs, r.endMs); + } else out.push({ ...r }); + } + return out; +} +const totalStart = cues[0].startMs, totalEnd = cues[cues.length - 1].endMs; +const keepRanges = []; +let cur = totalStart; +for (const d of dropRanges.slice().sort((a, b) => a.startMs - b.startMs)) { + if (d.startMs > cur) keepRanges.push({ startMs: cur, endMs: d.startMs }); + cur = Math.max(cur, d.endMs); +} +if (cur < totalEnd) keepRanges.push({ startMs: cur, endMs: totalEnd }); + +const droppedMs = dropRanges.reduce((a, r) => a + (r.endMs - r.startMs), 0); +const plan = { + schema: 'jianying.talk-cut.plan.v1', + source: path.resolve(SRT), + params: { fillerMs: FILLER_MS, dupSim: DUP_SIM, keepRepeats: KEEP_REPEATS, fillers: FILLERS }, + stats: { + cues: cues.length, + totalMs: totalEnd - totalStart, + drop: outCues.filter((c) => c.verdict === 'drop').length, + review: outCues.filter((c) => c.verdict === 'review').length, + keep: outCues.filter((c) => c.verdict === 'keep').length, + droppedMs, + droppedRatio: +((droppedMs / Math.max(1, totalEnd - totalStart))).toFixed(4), + }, + cues: outCues, + dropRanges, + keepRanges, + gaps, + duplicates: dupOf, +}; + +console.log('=== talk-cut 规划 ==='); +console.log(` 字幕条数 ${plan.stats.cues} · 总时长 ${(plan.stats.totalMs / 1000).toFixed(1)}s`); +console.log(` drop ${plan.stats.drop} · review ${plan.stats.review} · keep ${plan.stats.keep}`); +console.log(` 建议删除 ${(droppedMs / 1000).toFixed(2)}s（${(plan.stats.droppedRatio * 100).toFixed(1)}%）`); +console.log(` 长停顿 ${gaps.length} 处 · 重复 ${dupOf.length} 处`); +for (const c of outCues.filter((x) => x.verdict !== 'keep').slice(0, 12)) { + console.log(` [${c.verdict.toUpperCase()}] #${c.index} ${(c.startMs / 1000).toFixed(1)}s ${c.reasons.join(' / ')}`); +} +if (OUT) { + fs.writeFileSync(OUT, JSON.stringify(plan, null, 2), 'utf8'); + console.log('\n# -> ' + OUT); +} else { + console.log('\n（加 --out 保存计划）'); +} diff --git a/plugins/antianqi/jianying/skills/jianying-cut/SKILL.md b/plugins/antianqi/jianying/skills/jianying-cut/SKILL.md new file mode 100644 index 00000000..d6c92438 --- /dev/null +++ b/plugins/antianqi/jianying/skills/jianying-cut/SKILL.md @@ -0,0 +1,115 @@ +--- +name: jianying-cut +description: 'Use when: 给视频按调色参数或风格出片 / 用复刻的剪映调色链出 10bit 成片 / 按暖色、清爽、电影感、明亮风格调色 / 调色后出带原声的片 / 验收改动在画面上是否可见. 触发词: 给这个视频调色出片 / 按XX风格调色 / 出个10bit的片 / 剪映调色 / 调色后出片 / 出带声音的片 / 剪映出片 / 看看改动有没有生效.' +displayNames: + zh-Hans: 剪映调色出片（10bit · 保留原声） +license: Apache-2.0 +compatibility: Requires Windows + ffmpeg 6.0 + Python 3.14 + JYAI_HOME 指向 JYAI 仓库（工具链在 JYAI_HOME/tools）+ 剪映 11.6 运行时缓存（读取官方 shader 用；出片本身不依赖剪映 GUI）。 +metadata: + author: antianqi + version: "0.1.0" + basis: "调色链 10 个节点逐像素对齐官方 shader（误差 ≤1 LSB）；10bit/16F 出片链路已验证" +--- + +# 剪映调色出片（jianying-cut） + +> **一句话定位**：给一个源片，按调色参数或风格预设，用**逐像素复刻的剪映调色链**出 **10bit / 保留原音轨**的成片， +> 附「原片 | 调色后」对比图与逐节点生效明细。**不需要剪映 GUI。** + +## 调什么（动词与能力） + +本 skill 只动一个动词：**`grade_apply`**（调色 + 出片）。 +配套两个：**`verify_diff`**（验收改动是否可见）、**`render_frames`**（用官方渲染器出帧）。 + +| 想做的事 | 调哪个动词 | 关键参数 | +|---|---|---| +| 调亮度/对比/饱和/色温… | `grade_apply` | `spec.adjustColor.*` | +| 分色调整 | `grade_apply` | `spec.hsl.<颜色名>.{hue,saturation,lightness}` | +| 对数轮 / 三原色轮 | `grade_apply` | `spec.logWheel.*` / `spec.primaryWheel.*` | +| 曲线 / HSL 耦合曲线 | `grade_apply` | `spec.curves.*` / `spec.hsl_curves.*` | +| 锐化 / 暗角 / 颗粒 | `grade_apply` | `spec.sharpen` / `spec.vignetting` / `spec.particle` | +| 确认改动看得见 | `verify_diff` | `beforeDir` / `afterDir` | + +★ **参数怎么填** → `references/chain-api.md`（语义在这里）· `mcp/tools/grade-apply.schema.json`（硬校验在这里） +★ **多少算"生效"** → `references/verify-discipline.md` +★ **别再踩** → `references/pitfalls.md` + +## 主工作流 + +**入口**：`scripts/run-verb.mjs`（本插件自带的 MCP 客户端 ⇒ 走的是**和 mcode 加载插件时完全一样的动词**） + +```bash +# ① 环境自检（可选但建议） +node ${PLUGIN_ROOT}/scripts/check-env.mjs + +# ② 出片 +node ${PLUGIN_ROOT}/scripts/run-verb.mjs grade_apply '{ + "src": "<源视频.mp4 的绝对路径>", + "outDir": "<输出目录的绝对路径>", + "frames": 12, + "spec": { "adjustColor": { "brightness": 0.10, "contrast": 0.12, "saturation": 0.28, "temperature": 0.15 } } +}' + +# ③ 验收（改动是否在画面上可见） +node ${PLUGIN_ROOT}/scripts/run-verb.mjs verify_diff '{ + "beforeDir": "<改前帧目录>", "afterDir": "<改后帧目录>" +}' +``` + +**流程**： +``` +健康检查（check-env） + ↓ +grade_apply：出 10bit 成片 + 逐节点生效探针（任一节点为 0 ⇒ 动词直接判失败） + ↓ +读产物：graded10.mp4（逐点直算，推荐）/ graded_lut.mp4（65³ LUT 形态） + ↓ +需要时 verify_diff：max-over-frames 判"看得见" +``` + +**产物**（在 `outDir` 下）： +| 文件 | 内容 | +|---|---| +| `graded10.mp4` | **hevc / yuv420p10le / 原音轨**（逐点直算，推荐交付） | +| `graded_lut.mp4` | 同上，LUT 形态（链式插值误差 2 LSB） | +| `grade-side-by-side.png` | 左 原片 / 右 调色后 | +| `grade-compare.json` | 逐节点探针 · 调色改变量 · LUT vs 直算差 · 10bit 回读误差 | + +## 常见诉求 → spec 速查 + +| 用户说 | spec | +|---|---| +| 调亮一点 | `{"adjustColor":{"brightness":0.12}}` | +| 调暖一点 | `{"adjustColor":{"temperature":0.15}}` | +| 提亮中间调、压高光 | `{"adjustColor":{"shadow":0.15,"highlight":-0.10}}` | +| 更有食欲（餐饮） | `{"adjustColor":{"saturation":0.28,"temperature":0.15,"contrast":0.12},"logWheel":{"shadow":{"r":0.14,"g":0.06,"b":0.0,"s":0.08}}}` | +| 电影感 | `{"adjustColor":{"contrast":0.22,"saturation":-0.10,"shadow":0.16},"vignetting":{"intensity":0.35,"param":0.30}}` | +| 更锐 | `{"sharpen":{"intensity":0.5}}` | +| 红色更艳 | `{"hsl":{"HSLRed":{"hue":0.56,"saturation":0.62,"lightness":0.55}}}` | + +## 失败处理 + +| 症状 | 原因 | 处理 | +|---|---|---| +| `入参不合法：未知键` | spec 里写了没这个节点 | 查 `references/chain-api.md` 的 10 节点表 | +| `亮度 5 > 最大 1` | 强度语义是 −1..1，不是 0..100 | 换算：界面 0..100 ⇒ 强度 = (v/100)*2 − 1 | +| `下列节点没有生效（探针为 0）` | 该节点被静默跳过（形态/素材问题） | 单独调该节点，看 `grade-compare.json` | +| `调色改变量为 0` | spec 空 / 全 0 | 至少给一个非零强度 | +| `缺 JYAI 工具` | `JYAI_HOME` 没设 | `set JYAI_HOME=` | +| `UnicodeEncodeError gbk` | 环境码页 | 脚本已自设 UTF-8；若仍出现，检查调用方是否另起进程 | + +## ★ 三条纪律（别破） + +1. **本 skill 不直接改 `draft_content.json`** —— 改草稿请用 `jianying-draft-edit`（走 `draft.*` 动词）。 +2. **不自己算像素差** —— 一律用 `verify_diff`（max-over-frames），不抽单帧。 +3. **不绕过护栏** —— `grade_apply` 内建的探针、`draft_commit` 的 sha 断言是**故意拦人**的，不要关掉。 + +## 边界（不做什么） + +| 不做 | 为什么 | 替代 | +|---|---|---| +| 改剪映草稿 | 本 skill 只出片 | `jianying-draft-edit` | +| 混音 / 加 BGM / 响度归一 | 另一条链 | `audio_mix` 动词（需草稿目录） | +| 上传多平台 | 平台侧各有一套 | 平台自己的 skill | +| 官方导出器出片 | 官方 CLI 侧被 −100007 / −100015 挡死，**不绕** | 我们自己出（就是本 skill） | +| 曲线/HSL/色轮**写回剪映** | 草稿侧字段名尚未确证 | 只能本地出片（见 `references/pitfalls.md`） | diff --git a/plugins/antianqi/jianying/skills/jianying-draft-edit/SKILL.md b/plugins/antianqi/jianying/skills/jianying-draft-edit/SKILL.md new file mode 100644 index 00000000..d3b3fc89 --- /dev/null +++ b/plugins/antianqi/jianying/skills/jianying-draft-edit/SKILL.md @@ -0,0 +1,132 @@ +--- +name: jianying-draft-edit +description: 'Use when: 改剪映草稿 / 给草稿加素材或音轨 / 移动裁剪片段 / 调某段的音量 / 加文字 / 加轨道 / 撤销重做 / 改完落盘并验证生效. 触发词: 给这个草稿加个片段 / 把这段剪短 / 把这轨静音 / 加个文字 / 改完存一下 / 撤销刚才那步 / 看看改动生效没有.' +displayNames: + zh-Hans: 剪映·改草稿（27 个官方操作 · 落盘不动真草稿） +license: Apache-2.0 +compatibility: Requires Windows + 剪映 11.6 在运行（edit 域走官方命名管道）+ JYAI_HOME 指向 JYAI 仓库（落盘/渲染/验收在 JYAI_HOME/tools）+ ffmpeg + Python 3.14. +metadata: + author: antianqi + version: "0.1.0" + basis: "官方 edit 域 27 个带命名空间的行为操作（实测 27/27）；落盘走 draft/writeFile；验收走【语义】逐段字段差（像素差需先量噪声底）" +--- + +# 剪映改草稿（jianying-draft-edit） + +> **一句话定位**：在**副本**上改剪映草稿（加素材/加音轨/移动裁剪/调音量/加文字/加轨道/撤销重做）， +> 落盘后用**落盘草稿的逐段字段差**确认改动真的生效 —— **真草稿零改动**。 + +## 调什么（动词） + +| 动词 | 作用 | +|---|---| +| `draft_open` | 打开草稿（**默认开副本**；`allowReal` 与 `copy` 冲突会被拒） | +| `draft_read` | 读草稿语义结构（时间线/片段/素材）—— 拿 `segmentId` 等 id 用 | +| `draft_patch` | **27 个官方行为操作**（见下表） | +| `draft_commit` | 落盘（自动**素材路径重定位** + **真草稿 sha 前后断言**） | +| `render_frames` | 用官方渲染器出帧（改前/改后各一次） | +| `verify_diff` | 逐帧像素差（**取全帧最大**）判定"看得见" | + +## ★ 27 个官方操作（带命名空间，**不是裸名**） + +| 组 | 操作 | +|---|---| +| `draft` | `setCanvasSize` · `setDraftId` | +| `track` | `add` · `delete` · `mute` | +| `segment` | `move` · `scale` · `translate` · `rotate` · `updateLayout` · `split` · `delete` · `setTimeRange` · `innerReplace` · `setDescription` | +| `video` | `add` · `muteOn` · `muteOff` · `setVolume` | +| `audio` | `add` · `setVolume` | +| `text` | `add` | +| `undoRedo` | `undo` · `redo` · `canUndo` · `canRedo` | +| 批量 | `invokeBatch` | + +**★ 已知必填项**（缺了会被本地先拦下，不白跑一次官方调用）： +`video.add`→`path` · **`audio.add`→`audioPath`** · `text.add`→`content` · `track.mute`/`track.delete`→`trackId` · +`segment.move`/`setTimeRange`/`setDescription`→`segmentId` · **`segment.split`→`splitTime`** · +`segment.updateLayout`→`layout` · `segment.innerReplace`→`replacement` + +## 主工作流 + +```bash +# ① 开副本（拿 sessionId） +node ${PLUGIN_ROOT}/scripts/run-verb.mjs draft_open '{"draft":"<草稿库>\\<草稿名>","copy":true}' + +# ② 读结构（拿 segmentId） +node ${PLUGIN_ROOT}/scripts/run-verb.mjs draft_read '{"scope":"segments"}' + +# ③ 改（可多次；每次 baseRevision 会递增） +node ${PLUGIN_ROOT}/scripts/run-verb.mjs draft_patch '{"op":"video.setVolume","sessionId":"ve-session-N","params":{"segmentId":"","volume":0.25}}' + +# ④ 落盘（会做路径重定位 + 真草稿 sha 断言） +node ${PLUGIN_ROOT}/scripts/run-verb.mjs draft_commit '{"realDraft":"<草稿库>\\<草稿名>"}' + +# ⑤ 验收（★ 主判据 = 语义：落盘草稿的逐段字段差） +# 改前/改后各读一次明文草稿，再让 verify_diff 比"哪个字段变了" +node ${PLUGIN_ROOT}/scripts/run-verb.mjs draft_read '{"path":"/draft_content.json"}' +node ${PLUGIN_ROOT}/scripts/run-verb.mjs verify_diff '{ + "checks": ["timeline"], + "beforeDraft": "<改前的明文草稿.json>", "afterDraft": "<改后的明文草稿.json>", + "expect": { "segmentFieldChanged": "volume" } +}' +``` + +## ★ 四条纪律（别破） + +1. **永远开副本** —— `draft_open` 的 `copy` 默认 true；`allowReal=true` 必须同时 `copy=false`（否则被拒）。 +2. **落盘后必须验证** —— `draft_commit` 会断言真草稿 sha 未变；**"没报错"不等于"生效"**， + 要用 `verify_diff` 看**语义**（哪个字段变了）。 +3. **判据用全帧最大 + 先量噪声底** —— 不用单帧抽样（单帧会假 FAIL/假 PASS）； + ★ 官方渲染器**同态重渲染就有 MAD 48~123 的噪声** ⇒ 像素结论必须带 `noiseDir` 校准，否则只当参考。 +4. **★ 优先用语义判据** —— "改动生效"最硬的证据是**落盘草稿里那个字段真的变了** + （如 `volume` 0.99→0.2），它**完全确定**，而像素差不是。 + +## 常见诉求 → op + +| 用户说 | op + params | +|---|---| +| 把这轨静音 | `track.mute` `{trackId}` | +| 把这轨删了 | `track.delete` `{trackId}` | +| 加一条视频 | `video.add` `{path}`（★ 参数名是 `path`，不是 `source`/`filePath`） | +| 加一条音频 | `audio.add` `{audioPath}`（★ 不是 `source`） | +| 加一段文字 | `text.add` `{content}`（★ 不是 `text`） | +| 这段音量调到 25% | `video.setVolume` `{segmentId, volume}` | +| 这段静音 | `video.muteOn` `{segmentId}` | +| 把这段剪成两半 | `segment.split` `{segmentId, splitTime}`（★ 不是 `splitTimeUs`） | +| 删掉这段 | `segment.delete` `{segmentId}` | +| 挪到别的位置 | `segment.move` `{segmentId, ...}`（★ 需要先有 snapshot） | +| 改这段的时间范围 | `segment.setTimeRange` `{segmentId, startTimeUs?\|startUs?, durationUs?}` | +| 撤销上一步 | `undoRedo.undo` `{}` | +| 一次做多步 | `invokeBatch` `{operations:[...]}` | + +> ★ **官方行为操作是带命名空间的**（`segment.move` 不是 `move`）—— 历史上"白名单 0/150"就是命名错造成的。 +> ★ **剪"中间一段"的正确链**：`split(t1)` → `split(前半, t0)` → `delete(中段)`； +> **split 会让后面的段整体后移 ⇒ 多个区间必须从后往前处理**。 + +## 失败处理 + +| 症状 | 原因 | 处理 | +|---|---|---| +| `缺少必填参数：path` | op 的必填项没给 | 查上表；参数放 `params` 里 | +| `edit/xxx 失败：segment_id_required` | 官方原样错误 | 你的 `segmentId` 不存在/已被删 ⇒ 先 `draft_read` 重新拿 | +| `edit/xxx 失败：draft_snapshot_required` | 这个 op 需要先有快照 | 先做一次别的 op，或用 `draft.setDraftId` | +| `管道连不上` | 剪映没在跑 | 启动剪映；`edit` 域必须连到活着的 VE 服务 | +| `真草稿被改动了` | ★ 危险信号 | 立刻停；查是不是误开了真草稿 | +| `verify_diff` 的 timeline 判 FAIL | 语义判据没看到预期字段变化 | ① 确认 `expect` 的字段名对不对（如 `volume`）② 确认 `beforeDraft`/`afterDraft` 是**改前/改后**两份，不是同一份 | +| `verify_diff` 的 visibility 判"不可见" | 可能是**渲染噪声**盖住了，也可能是真没生效 | 先加 `noiseDir` 校准；再用 `timeline` 维度的语义判据交叉验证 | +| `draft_read` 返回 `chars=0` | 读指定草稿时 root 没解析出来 | 传 `root`（或用 `path` 的上级目录）；插件已自动补，但自定义路径时仍要核 | + +## ★ 边界（不做什么） + +| 不做 | 为什么 | 替代 | +|---|---|---| +| 改**真**草稿 | 官方零文件监听 ⇒ 写盘不保证 GUI 立刻可见；且我们的红线是"官方库只读" | 人工在 GUI 里导入/替换 | +| 写**结构化调色**（曲线/HSL/色轮） | 草稿侧字段名尚未确证 | `jianying-cut` 走我们自己的链出片 | +| 官方导出器出片 | 官方 CLI 被 −100007 / −100015 挡死，**不绕** | `jianying-cut` | +| 画布节点操作（连线/卡片） | 画布是**生成线**，与"编辑已有草稿时间线"不同轴 | 不做 | +| 从零生成（AIGC 资产/单集成片） | processor 走沙箱 agent，取不到 | 不做 | + +## 参考 + +- 官方 27 op 的实测报告：`\logs\editops.md`（含每个 op 的官方报错 ⇒ 参数反推依据） +- 判据纪律：`references/verify-discipline.md` +- 踩坑清单：`references/pitfalls.md` diff --git a/plugins/antianqi/jianying/skills/jianying-grade-chain/SKILL.md b/plugins/antianqi/jianying/skills/jianying-grade-chain/SKILL.md new file mode 100644 index 00000000..30fa5960 --- /dev/null +++ b/plugins/antianqi/jianying/skills/jianying-grade-chain/SKILL.md @@ -0,0 +1,89 @@ +--- +name: jianying-grade-chain +description: 'Use when: 查某个调色节点/参数怎么填时 / 不确定某个调色效果该归到哪个节点时 / 调色链精度口径与可选节点清单时 / 用户给了调色参数要落到链上时. 本 skill 是【库】—— 仅在其它 skill 引用它、或用户明确问调色链细节时被使用，不主动触发. 触发词: 调色链有哪些节点 / 这个参数填哪个节点 / 调色能精确到什么程度 / 曲线的控制点是什么顺序.' +displayNames: + zh-Hans: 剪映·调色链细节（库 · 仅被引用） +license: Apache-2.0 +compatibility: Requires nothing beyond the plugin's grade_apply verb. No external dependency. +metadata: + author: antianqi + version: "0.1.0" + basis: "10 个节点逐像素对齐官方 shader（误差 ≤1 LSB）；链 API 形态见 tools/jyai-gl-bake.py 的 normalize_chain_spec" +--- + +# 剪映调色链细节（jianying-grade-chain · 库 skill） + +> **一句话定位**：回答"**这个调色效果该用哪个节点、参数怎么填、能精确到什么程度**"。 +> ★ **本 skill 是【库】**：不主动触发，只在其它 skill 引用它、或用户明确问调色链细节时使用。 +> 完整版：`references/chain-api.md`（本文件是它的入口速查）。 + +## 10 个节点（`grade_apply` 的 spec 键） + +| 节点 | 类型 | 能否烘 3D LUT | 官方对应 | +|---|---|---|---| +| `adjustColor` | 逐点 | ✅ | 基础画面调节（亮度/对比/饱和/高光/暗部/色温…） | +| `colorCorrection` | 逐点 | ✅ | 色彩校正（adjustColor 的精简变体） | +| `hsl` | 逐点 | ✅ | 分色调整（8 默认色 + 用户取色） | +| `logWheel` | 逐点 | ✅ | 对数轮（阴影/中间调/高光/偏移 四段） | +| `primaryWheel` | 逐点 | ✅ | 三原色轮 | +| `curves` | 逐点 | ✅ | 曲线（red/green/blue/luma 4 条 1D） | +| `hsl_curves` | 逐点 | ✅ | HSL 耦合曲线（6 条） | +| `sharpen` | **邻域** | ❌ | 锐化（渲染期算子） | +| `vignetting` | **uv** | ❌ | 暗角/白角（渲染期算子） | +| `particle` | **uv** | ❌ | 颗粒/光斑（渲染期算子） | + +★ **邻域 / uv 依赖的三个节点不能烘 3D LUT**（3D LUT 只能表达"逐点函数"）⇒ 只能在渲染期算。 +★ `fade` / `lightSensation` 是 **`adjustColor` 的子参数**，**不是独立节点**（官方 frag 里是 adjustColor 流水线的最后两段，顺序固定）。 + +## 强度语义 + +- `-1..1` = **强度**：`0` 不动 · `+0.5` 中等偏上 · `-1` 拉满 +- `0..1` = **比例/百分比**：`sharpen.intensity` · `vignetting.param` +- 界面 0..100 ⇒ 强度 = `(v/100)*2 - 1` + +## ★ 最容易踩的坑：两条曲线的控制点布局**不同** + +| 节点 | 布局 | 判别 | +|---|---|---| +| `curves` | **A,L,R** | 第一个三元组的第二项是 **x** | +| `hsl_curves` | **L,A,R** | 第一个三元组的第二项是 **y** | + +⇒ **两套数组不能互相套用。** + +## 常见诉求 → spec + +| 用户说 | spec | +|---|---| +| 调亮一点 | `{"adjustColor":{"brightness":0.12}}` | +| 调暖一点 | `{"adjustColor":{"temperature":0.15}}` | +| 提亮中间调、压高光 | `{"adjustColor":{"shadow":0.15,"highlight":-0.10}}` | +| 更有食欲（餐饮） | `{"adjustColor":{"saturation":0.28,"temperature":0.15,"contrast":0.12},"logWheel":{"shadow":{"r":0.14,"g":0.06,"b":0.0,"s":0.08}}}` | +| 电影感 | `{"adjustColor":{"contrast":0.22,"saturation":-0.10,"shadow":0.16},"vignetting":{"intensity":0.35,"param":0.30}}` | +| 红色更艳 | `{"hsl":{"HSLRed":{"hue":0.56,"saturation":0.62,"lightness":0.55}}}` | + +## 精度口径 + +| 路径 | 误差 | +|---|---| +| 逐点直算（`graded10.mp4`） | maxAbs **1 LSB** | +| 65³ 3D LUT（`graded_lut.mp4`） | maxAbs **2 LSB** | +| 129³ 3D LUT | maxAbs **2 LSB**（更小） | + +⇒ **要精确 ⇒ 用逐点直算那一版**；要小体积/快 ⇒ 用 LUT 版。 + +## 机器可读版 + +- 入参 schema（硬校验，**未知键直接拒绝**）：`mcp/tools/grade-apply.schema.json` +- 能力字典（46 项能力 → 动词 → 参数/op → 判据）：`references/capability-dict.json` +- 人读版（每个参数的语义、官方对应、已知不覆盖）：`references/chain-api.md` + +## ★ 边界（不做什么 / 为什么 / 替代） + +| 不做 | 为什么 | 替代 | +|---|---|---| +| **`clear`（清晰度，8 级金字塔）接入链** | 尚未接入；且**官方默认 intensity=0 本身就是空转**（§67 实测 value=0 逐像素相同） | 只能出参考帧做对照，不能当可用节点 | +| **`colorMigration`（色彩迁移）** | 官方算法在闭源 `Amaz.Algorithm` ⇒ 拿不到 | 自算替代（与 `clear` 同类：只能自己实现） | +| **把曲线 / HSL / 色轮写回剪映预览** | 草稿侧字段名尚未确证（**intensity 型已不需要用户**） | 本地出片（`jianying-cut`）；写回要等用户 GUI 存一份带曲线的草稿 | +| **给 `sharpen` / `vignetting` / `particle` 配 LUT 烘焙** | 这三个是**渲染期**节点（邻域 / uv 依赖）⇒ LUT 表达不了 | 链内渲染期算（已在链里） | +| **把 `fade` / `lightSensation` 当独立节点** | 它们是 **`adjustColor` 的子参数**，不是顶层 spec 键 | 写 `spec.adjustColor.fade` / `spec.adjustColor.lightSensation` | +| **照抄官方 shader 的定点编码** | 官方 `enCode` 会截断（\|v\|>1 时截断）⇒ 拉普拉斯差分超 [−1,1] 会丢精度 | 链内用 float64 复刻，别在中间丢 | diff --git a/plugins/antianqi/jianying/skills/jianying-review/SKILL.md b/plugins/antianqi/jianying/skills/jianying-review/SKILL.md new file mode 100644 index 00000000..00b67fc9 --- /dev/null +++ b/plugins/antianqi/jianying/skills/jianying-review/SKILL.md @@ -0,0 +1,129 @@ +--- +name: jianying-review +description: 'Use when: 审片 / 检查这版剪得对不对 / 返修验收 / 改动到底生效没有 / 画质或响度有没有问题 / 交片前检查. 触发词: 审一下这版 / 看看剪得对不对 / 改动生效了吗 / 交片前检查 / 画面有没有过曝 / 响度够不够 / 时间线对不对.' +displayNames: + zh-Hans: 剪映·审片验收（多维度 · 带数字和证据边界） +license: Apache-2.0 +compatibility: Requires Windows + 剪映 11.6 在运行（官方渲染器）+ ffmpeg 6.0 + Python 3.14 + JYAI_HOME 指向 JYAI 仓库. +metadata: + author: antianqi + version: "0.1.0" + basis: "官方 editing-review 的证据边界纪律 + 我们可运行的检测器（多维度 verify_diff）" +--- + +# 剪映审片验收（jianying-review） + +> **一句话定位**：对"改前 / 改后"两版做**多维度对照检查**，每条结论都带**数字 + 证据边界**， +> 并给出**最小返修范围**。★ 审片本身**不授权**改草稿或发布。 + +## ★ 我们和官方的差别 + +剪映官方的 `editing-review` 只有 47 行，且原文明确写着： +**「这里列出的是检查方法，不是已实现或已运行的检测器」**。 +⇒ 官方告诉你**该查什么**；**我们有真正能跑的检测器和数字**（像素差 / 画质 / 响度 / 时间线 / sha）。 + +**但纪律要跟官方一致**（这是我们从它身上学的）： +- **源素材分析只能证明源素材内容；草稿结构只能证明编排事实；渲染成片的音画证据才支持"实际呈现"的结论** +- **不要把"上游成功"当成"审片通过"**（草稿应用成功 ≠ 画面对了） +- **只对工具真正覆盖的维度和范围下结论**；采样不能证明"全片没有闪帧" +- **区分四类结论**：证据支持通过 / 已确认问题 / **证据不足** / 与本次范围无关 +- **没有足够证据的必查项要保留为"待确认"**，不能因为"没发现问题"就给完整通过 + +## 检查维度（`verify_diff` 的 `checks`） + +| 维度 | 查什么 | 需要的证据 | 不达标时 | +|---|---|---|---| +| `visibility` | 改动在画面上**看得见**吗 | 改前/改后帧目录 **+ 噪声基线** | max MAD ≤ 判据线 ⇒ FAIL | +| `picture` | 画质：均值漂移 / **新增**高光削波 / **新增**暗部死黑 | 同上 | 新增削波 >2% / 死黑 >10% ⇒ FAIL | +| `audio` | 响度 LUFS / true peak | 改前/改后 mp4 | 偏离 −14 LUFS ⇒ 提示 | +| `timeline` | 段数 / 轨道 / 时长 / **逐段字段差** | 改前/改后草稿 JSON | 给了 `expect` 就按期望断言；没给且什么都没动 ⇒ **改动可能没落盘** | +| `draft_integrity` | 真草稿有没有被误改 | 真草稿目录 | 落盘时已做逐字节断言 | + +**★ 判据用增量不用绝对值**：片子里本来就有的纯白/纯黑**不算**削波/死黑，只有**被改动新推上去/压下去**的才算缺陷。 + +## ★★★ 审片前必须知道的一件事：官方渲染器**有噪声底** + +**同一份草稿渲染两次，帧就已经不一样**（实测 14~17/108 帧不同，最差 MAD **48~123**）。 +⇒ **裸 MAD 阈值（0.5）没有任何判别力**；"max MAD=123.5 ⇒ 改动可见"这类结论**不成立**。 + +**⇒ 审片必须先量噪声底**（`noiseDir` = 同一状态**再渲染一次**的帧目录）： + +```bash +node ${PLUGIN_ROOT}/scripts/run-verb.mjs verify_diff '{ + "beforeDir": "/frames", "afterDir": "/frames", + "noiseDir": "/frames-again", # ← 同态再渲染一次 + "checks": ["visibility","picture","timeline"] +}' +``` + +- 给了 `noiseDir` ⇒ 判据线 = `max(threshold, noiseMad × 1.5)`，输出里有 `noiseMad` / `madOverNoise`； +- **没给** ⇒ 输出里 `calibrated:false` + note ⇒ **这时"可见"只能当参考，不能当结论**。 + +**★ 结构性改动请用语义判据**（不依赖像素、完全确定）： +`timeline` 维度会给出**逐段字段差**（`segmentDiffs`：段 id + 变了哪些字段 + 改前改后值）， +配合 `expect` 直接断言： + +```bash +node ${PLUGIN_ROOT}/scripts/run-verb.mjs verify_diff '{ + "checks": ["timeline"], + "beforeDraft": "<改前的明文草稿.json>", "afterDraft": "<改后的明文草稿.json>", + "expect": { "segmentFieldChanged": "volume", "segmentsDelta": 0 } +}' +``` + +> **审片报告里必须写清"这条结论是语义的还是像素的"** —— 两者可信度差一个数量级。 + +## 主工作流 + +```bash +# ① 渲染改前 / 改后（官方渲染器） +node ${PLUGIN_ROOT}/scripts/run-verb.mjs render_frames '{"draft":"","outDir":""}' + +# ② 多维度审片 +node ${PLUGIN_ROOT}/scripts/run-verb.mjs verify_diff '{ + "beforeDir": "/frames", "afterDir": "/frames", + "checks": ["visibility","picture","audio","timeline","draft_integrity"], + "beforeVideo": ".mp4", "afterVideo": ".mp4", + "beforeDraft": "", "afterDraft": "", + "realDraft": "<草稿库>\\<草稿名>" +}' +``` + +**输出结构**： +```json +{ "verdict": "PASS" | "FAIL", + "checks": { "visibility": {...}, "picture": {...}, "audio": {...}, "timeline": {...}, "draft_integrity": {...} }, + "problems": ["高光削波【新增】3.21%（>2%）", ...] } +``` + +## 结论怎么写（照官方那套分类） + +| 类别 | 什么时候用 | 写法 | +|---|---|---| +| **证据支持通过** | 该维度有检测器覆盖且达标 | "画面变化可见：max MAD 123.5 @frame74（22/108 帧有差异）" | +| **已确认问题** | 有数字支撑的缺陷 | "实际 vs 预期 → 定位 → 影响 → 最小返修 + 返修后用什么证据确认" | +| **证据不足** | 该维度没有检测器覆盖 | "响度未测（未提供 mp4）——待确认，不写成'响度正常'" | +| **与本次范围无关** | 用户没要求这个维度 | 不展开、不泛化罗列风险 | + +## ★ 最小返修原则 + +- 只修**有证据的问题**，范围最小 +- 说明**应该保持什么**（避免修 A 破坏 B） +- 给出**返修后用什么证据确认**（不是"再优化一下"这种无法执行的建议） +- **审片请求不授权修改**：用户授权修复时，才把问题与最小返修范围交给执行入口（`jianying-draft-edit`） + +## ★ 边界（不做什么 / 为什么 / 替代） + +| 不做 | 为什么 | 替代 | +|---|---|---| +| **改草稿 / 落盘** | **审片请求不授权修改**；越权改草稿会动到用户资产 | 用户授权后，把问题与最小返修范围交给 `jianying-draft-edit` | +| **发布 / 导出成片** | 发布侧已有独立 skill 体系 | `daily-platforms-export` · `lvchuanjiao-daily-upload` | +| **"全片没有闪帧"这类全片结论** | 采样不能证明"全片没有闪帧"（官方原话）⇒ 只能写"证据不足/待确认" | 缩小范围重测，或在结论里标注覆盖范围 | +| **拿官方渲染器的裸像素差当结论** | 同态重渲染噪声底就有 MAD 48~123 ⇒ 裸 MAD 分不出"改动"与"噪声" | 语义判据（`timeline` 逐段字段差）为主 · 像素必须带 `noiseDir` 校准 | +| **结构化调色（曲线/HSL/色轮）的写回与判定** | 草稿侧字段名尚未确证 | `jianying-cut` 走我们自己的链出片 | + +## 参考 + +- 判据纪律（必读）：`skills/jianying-verify/SKILL.md` · `references/verify-discipline.md` +- 踩坑清单：`references/pitfalls.md` +- 官方 `editing-review` 原文对照：`%ProgramFiles%\JianyingPro\<版本>\Resources\canvas_agent\skills\editing-review\SKILL.md` diff --git a/plugins/antianqi/jianying/skills/jianying-talk-cut/SKILL.md b/plugins/antianqi/jianying/skills/jianying-talk-cut/SKILL.md new file mode 100644 index 00000000..e5951df4 --- /dev/null +++ b/plugins/antianqi/jianying/skills/jianying-talk-cut/SKILL.md @@ -0,0 +1,106 @@ +--- +name: jianying-talk-cut +description: 'Use when: 剪口播 / 去掉口播里的废话、停顿、重复、卡壳 / 按转写稿整理口播 / 把长口播剪短 / 剪完落稿并确认生效. 触发词: 把这段口播的废话剪掉 / 去一下口水话 / 把停顿剪了 / 有重复的剪掉 / 口播剪干净点.' +displayNames: + zh-Hans: 剪映·剪口播（去水词/停顿/重复 · 带人工确认） +license: Apache-2.0 +compatibility: Requires Windows + 剪映 11.6 在运行 + ffmpeg 6.0 + Python 3.14 + JYAI_HOME 指向 JYAI 仓库. ★ 需要用户提供转写稿(SRT) —— 官方 ASR(AnalyzeAsset) 在 agent runtime，管道里拿不到（§74.4）。 +metadata: + author: antianqi + version: "0.1.0" + basis: "官方 studio.edit.speaker_cut 的经验层（Clean Cut 思路）+ 我们自建的 SRT 判定器与 27-op 落稿链" +--- + +# 剪映剪口播（jianying-talk-cut） + +> **一句话定位**：给一份**转写稿（SRT）**，去掉口播里的**水词 / 长停顿 / 重复**， +> 产出**保留区间表**给人确认，再把确认后的区间**真的剪进草稿**并落盘验证。 +> ★ 官方 `speaker_cut` 的经验层我们照做，**但 ASR 那一步我们拿不到** ⇒ 改用**用户提供的转写稿**。 + +## ★ 为什么用"用户提供的 SRT"而不是官方 ASR + +剪映官方的 `AnalyzeAsset`（ASR）和 `SegmentRecall`（素材召回）都在 **agent runtime** +（`vebun.exe` + `app-server.mjs`，HTTP 63178），**不在我们的 VE 管道里** +（§42.2 另已证 addon 的 526 个导出**一个都不在管道**）。 +⇒ 官方 9 个 processor 里也**没有 ASR**（只有截帧/变速/裁剪/去字幕/合片）。 +⇒ **但它缺的是"事实来源"（转写文本），不是"经验"（哪句是废话）** ⇒ 经验层我们自己做。 + +## 三级判定（保守优先） + +| 判定 | 含义 | 处理 | +|---|---|---| +| **drop** | 整条只有水词/标点（如"嗯，呃。"） | **建议整句删** | +| **review** | 句中有实词但夹水词 / 长停顿 / 与前几条重复 | **标出位置，建议人工确认**（默认不删） | +| **keep** | 正常内容 | 保留 | + +★ **默认不删 review** —— 口播里的"那个/然后"有时是口语节奏的一部分，删不删要人拍板。 +★ 重复检测**往前看 3 条**（口播里"重录"常隔一句，不只相邻）。 +★ 词级水词时间按字符比例估算（粗，但够定位）。 + +## 主工作流 + +```bash +# ① 规划（纯计算，不碰草稿） +node ${PLUGIN_ROOT}/scripts/talk-cut-plan.mjs \ + --srt "<转写稿.srt>" --out "" \ + [--filler-ms 700] [--dup-sim 0.72] [--dup-window 3] [--keep-repeats] + +# ② 【人工确认】把 plan.json 里的 review 句逐条看过： +# 确认要删的 → 把该区间加进 dropRanges（或直接改 verdict 后重跑 ①） + +# ③ 应用（默认演练；加 --apply 才真改） +node ${PLUGIN_ROOT}/scripts/talk-cut-apply.mjs --plan "" --draft "<草稿名>" [--apply] + +# ④ 落盘后验收：官方渲染 + 像素差 + 时间线变化 +node ${PLUGIN_ROOT}/scripts/run-verb.mjs render_frames '{"draft":"","outDir":""}' +node ${PLUGIN_ROOT}/scripts/run-verb.mjs verify_diff '{ + "beforeDir":"/frames","afterDir":"/frames", + "checks":["visibility","timeline"], + "beforeDraft":"","afterDraft":"" +}' +``` + +## ★ 剪"中间一段"的正确链（实测，2026-10-21） + +```text +① segment.split {segmentId, splitTime: t1} → 切成两段，【后半段拿新 id】 + ※ split 会让【后面的段整体后移】⇒ 多个区间必须【从后往前】处理 +② segment.split {segmentId: 前半id, splitTime: t0} → 前半再切两半 +③ segment.delete {segmentId: 中间段id} → 时间线缩短 (t1 - t0) +``` +实测：时间线 9033333 → 8033333 us（正好剪掉 1 秒）✓ +**参数名（实测）**：`video.add`→`path` · `audio.add`→**`audioPath`** · `text.add`→`content` · +`segment.setTimeRange`→`segmentId`+时间 · `segment.delete`→`segmentId` · `segment.move`→`segmentId`+**需要 snapshot** + +## ★ 三条纪律（别破） + +1. **先规划、后人工确认、再改** —— 不要跳过 ② 直接 `--apply`。 +2. **动词之间共享状态**：改动在**会话内存**里，回读必须带 `sessionId` 才看得到（否则读到的是没落盘的磁盘草稿）。 +3. **落盘后必须验收** —— `verify_diff` 看**时间线维度**（段数/时长变没变）+ **可见性维度**（画面确实变了）。 + +## 失败处理 + +| 症状 | 原因 | 处理 | +|---|---|---| +| `SRT 里没解析出任何字幕条目` | 不是 SRT 格式 | 转成 `HH:MM:SS,mmm --> HH:MM:SS,mmm` | +| `找不到覆盖它的段` | 区间超出草稿时间线 | 核对 SRT 时间轴与草稿时长 | +| `找不到中间段` | split 后 id 变了 | 每步都要回读拿新 id（脚本已做） | +| `edit/xxx 失败：xxx_required` | 参数名不对 | 见上面「参数名（实测）」表 | +| `管道连不上` | 剪映没在跑 | 启动剪映 | + +## 边界（不做什么 / 为什么 / 替代） + +| 不做 | 为什么 | 替代 | +|---|---|---| +| **自己做 ASR** | 官方 ASR 在 agent runtime（HTTP 63178），**不在我们的 VE 管道里**；本地 whisper 要装模型（可选，不在默认路径） | **用户给 SRT / 转写稿**（我们缺的是"事实来源"，不是"经验"） | +| **改写台词 / 生成旁白** | 官方也明确「自由改写和生成旁白走通用链路」 | 人工改写；或用 `jianying-draft-edit` 的 `text.add` 另加一段 | +| **按主题重排**（官方 topic_cut） | 需要素材召回（`SegmentRecall`）⇒ 拿不到 | 我们只做 **Clean Cut**（按 SRT 删废话） | +| **自动加字幕** | 字幕规范/样式是另一条链 | `jianying-draft-edit` 的 `text.add` 可以做，但不是本 skill 的事 | +| **不加人工确认就落刀** | 判定只到 `review` 级的水词/停顿/重复**默认不删** | `talk-cut-plan` 出保留区间表 ⇒ 人工确认 ⇒ `talk-cut-apply` 才落刀 | + +## 参考 + +- 官方 `studio.edit.speaker_cut`（125 文件，Clean Cut / Topic Cut 两模式）：`%ProgramFiles%\JianyingPro\<版本>\Resources\canvas_agent\skills\studio.edit.speaker_cut\SKILL.md` +- 27 个官方 op 的实测报告：`\logs\editops.md` +- 判据纪律：`skills/jianying-verify/SKILL.md` +- 踩坑清单：`references/pitfalls.md` diff --git a/plugins/antianqi/jianying/skills/jianying-verify/SKILL.md b/plugins/antianqi/jianying/skills/jianying-verify/SKILL.md new file mode 100644 index 00000000..1d564ba8 --- /dev/null +++ b/plugins/antianqi/jianying/skills/jianying-verify/SKILL.md @@ -0,0 +1,122 @@ +--- +name: jianying-verify +description: 'Use when: 需要判断"某个改动算不算成功/生效"时 / 设计验收判据 / 判据视野不够要自检 / 被问"这个怎么才算通过" / 审片要给验收口径时. 本 skill 是【库】—— 仅在其它 skill 引用它、或用户明确要验收方法时被使用，不主动触发. 触发词: 怎么算通过 / 验收标准是什么 / 这个改动生效了吗 / 怎么证明它真的生效了 / 判据自检.' +displayNames: + zh-Hans: 剪映·验收判据（库 · 仅被引用） +license: Apache-2.0 +compatibility: Requires nothing beyond the plugin's own verbs (verify_diff / grade_apply / draft_commit). No external dependency. +metadata: + author: antianqi + version: "0.1.0" + basis: "三层判定纪律 + 七条硬规矩 + negative-injection 自检（docs/OFFICIAL-REALTIME-PATH.md §68.3/§75.2/§76.7）" +--- + +# 剪映验收判据（jianying-verify · 库 skill） + +> **一句话定位**：回答"**这个改动算不算成功、怎么证明**"——给出可执行的验收口径与判据自检方法。 +> ★ **本 skill 是【库】**：不主动触发，只在其它 skill 引用它、或用户明确要验收方法时使用。 +> 完整版：`references/verify-discipline.md`（本文件是它的入口摘要）。 + +## 核心一条：**判据的视野必须覆盖被测对象** + +**只抽一帧 = 用抽样代替了被测对象** ⇒ 假 FAIL 和假 PASS 都会出。 + +实测（2026-10-21）：单帧抽样选的第 5 帧 MAD=0.47（阈值 0.5）判 FAIL， +而同一批帧里第 74 帧 MAD=123.5 / maxAbs=255 / 100% 像素变化。 +⇒ **对策：`verify_diff` 一律取全帧最大（max-over-frames）。** + +## ★★★ 核心两条（2026-10-21 §76.7 追加）：**先量噪声底，再谈"可见"** + +**官方渲染器不是确定性的**：同一份草稿渲染两次，帧就已经不一样。 + +| 样本 | 不同的帧 | 最差 MAD | +|---|---|---| +| 抓帧容差 60ms | **14/108** | **48.19** @frame39 | +| 抓帧容差 20ms | **17/108** | **123.55** @frame73/74 | + +⇒ **噪声底（48~123）比任何"改动可见"阈值（0.5）高两个数量级** +⇒ **裸 MAD 不能证明"改动可见"**（历史上写下的 123.5 与噪声同量级，已作废）。 + +**⇒ 判据分三层，主判据必须落在确定的那一层：** + +| 层 | 判据 | 确定性 | 怎么用 | +|---|---|---|---| +| ① **语义** | 落盘草稿的**逐段字段差**（如 `volume` 0.99→0.2） | ✅ 完全确定 | **改草稿链的首选主判据** | +| ② 像素（自有渲染器） | `grade_apply` 逐节点探针 + 全链 MAD | ✅ 确定 | 调色链的主判据 | +| ③ 像素（官方渲染器） | `verify_diff` max-over-frames | ⚠️ 有噪声底 | **必须先校准**，否则只作参考 | + +**校准怎么用**：`verify_diff` 传 `noiseDir`（同一状态**再渲染一次**的帧）， +判据线自动抬到 `max(threshold, noiseMad × 1.5)`；**不传就在输出里标 `calibrated:false`** ⇒ 不许装作可信。 + +## 三层判定 + +``` +协议层说支持 ≠ 契约层被接受 ≠ 渲染层生效 +``` + +| 层 | 判什么 | 怎么判 | 我们的工具 | +|---|---|---|---| +| **契约层** | 字段认不认 | 材料存活 / 回写明文 / 字段保留 | `draft_read`（解密回读） | +| **渲染层** | 有没有效果 | 无头取帧 + **A/B 逐像素差分** + **阳性对照** | `render_frames` + `verify_diff` | +| **机械层** | 危险动作被拦了吗 | 护栏断言 | `draft_commit`（真草稿 sha）/ `grade_apply`（逐节点探针） | + +## 七条硬规矩 + +1. **判据视野覆盖被测对象** —— 用 max-over-frames，不用单帧。 +2. **阴性对照必须存在** —— 每个"能跑"都要配一个"不能跑"的对照。 +3. **输出必须能自证是"这次跑出来的"** —— 同一输出目录**不许并发跑两份**（会互相覆盖帧、污染证据）； + **基线也不许借用别的跑的残留目录**（⇒ 测的是两次不同的改动）。 +4. **探针要能区分"没生效"和"没在测"** —— **数值整齐得不正常 ⇒ 探针没在测单节点**。 +5. **参数形态不对 ⇒ 静默失效** —— 链的每个节点要么真生效、要么被整段跳过，**不会报错** ⇒ 必须逐节点探针。 +6. **注入坏了必须失败** —— 每个判据都要能被"故意弄坏"而失败。 +7. **★ 像素判据必须先量噪声底** —— 没有噪声底就不要报"改动可见"；**基线必须本次自产**。 + +## 数字底线（我们复刻链的验收口径） + +| 对象 | 口径 | 实测 | +|---|---|---| +| 调色链（逐点） | maxAbs ≤ **1 LSB** | 1 LSB | +| 调色链（65³ LUT） | maxAbs ≤ **2 LSB** | 2 LSB（对齐 §54 curves 的 65³ precedent 0.0086） | +| 曲线恒等 | **字节完全一致** | 256/256 texel | +| ~~改动可见性~~ | ~~max MAD > 阈值~~ | ⚠️ **作废**（123.5 与噪声底同量级）⇒ 改用语义判据或噪声校准 | +| 真草稿 | sha 逐字节相同 | 64 files identical | +| **改草稿生效（语义）** | 逐段字段差 + `expect` 断言 | `volume` **0.99 → 0.2**（其余 3 段不变）✓ | + +## 调色的"生效"判据（三条同时成立） + +1. **逐节点探针**：每个节点单开 ⇒ MAD > 0 +2. **全链改变量**：`grade_change_mad > 0` +3. **空间分布合理**：改"亮"应该 mean 上升、**clip 比例不超阈值**（过曝是另一个问题，不是"调亮"） + +## 判据自检清单（写新判据时问自己） + +- [ ] 判据的视野覆盖被测对象了吗？（不是抽一帧） +- [ ] 有阴性对照吗？ +- [ ] 故意弄坏它会失败吗？ +- [ ] 探针能区分"没生效"和"没在测"吗？ +- [ ] 输出能自证是这次跑出来的吗？ +- [ ] ★ **像素判据的噪声底量了吗？**（同一状态重渲染 ⇒ 差异有多大？没有噪声底就别报"可见"） +- [ ] ★ **基线是【这次跑】产生的吗？**（借用别的跑的残留目录 ⇒ 测的是两次不同的改动） + +## ★ 边界（不做什么 / 为什么 / 替代） + +| 不做 | 为什么 | 替代 | +|---|---|---| +| **主动触发**（当任务型 skill 用） | 它是**库**：只在被别的 skill 引用、或用户明确要验收方法时用 | 由 `jianying-review` / `jianying-draft-edit` 引用 | +| **直接下"通过 / 不通过"的终局结论** | 本 skill 给的是**口径**；结论要基于被测对象与证据边界 | 调用方拿口径去跑 `verify_diff`，自己结合范围下结论 | +| **用单帧抽样当判据** | 抽样既会假 FAIL 也会假 PASS | max-over-frames（全帧最大） | +| **用裸 MAD 阈值当"改动可见"** | 官方渲染器同态重渲染噪声底 48~123 ⇒ 没有判别力 | 语义判据（逐段字段差）· 或传 `noiseDir` 校准 | +| **把不同维度合并成一个总评分** | 量纲不可比（像素 / dB / µs / 百分比） | 分维度给结论，每个维度标注证据强度 | + +## ★ 官方对照：他们给方法论，我们给能跑的检测器 + +剪映官方的 `editing-review` skill 只有 47 行，且原文明确写着： +**「这里列出的是检查方法，不是已实现或已运行的检测器」**。 +⇒ 官方是"告诉你该查什么"，我们是"**已经查了，数字在这**"（像素差 / 逐节点探针 / sha 断言 / LUFS 测量）。 +⇒ 审片结论要带**数字与证据边界**，不要只给主观建议。 + +## 参考 + +- 完整纪律：`references/verify-discipline.md` +- 踩坑清单（含本项目的反面教材）：`references/pitfalls.md` +- 审片 skill：`skills/jianying-review/SKILL.md` diff --git a/test/jianying.test.mjs b/test/jianying.test.mjs new file mode 100644 index 00000000..110e11bc --- /dev/null +++ b/test/jianying.test.mjs @@ -0,0 +1,160 @@ +import assert from 'node:assert/strict'; +import { readFile, readdir, stat } from 'node:fs/promises'; +import path from 'node:path'; +import test from 'node:test'; +import { fileURLToPath } from 'node:url'; + +const repositoryRoot = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '..'); +const pluginRoot = path.join(repositoryRoot, 'plugins', 'antianqi', 'jianying'); +const read = (relative) => readFile(path.join(pluginRoot, relative), 'utf8'); +const exists = async (relative) => { + try { await stat(path.join(pluginRoot, relative)); return true; } catch { return false; } +}; + +const SKILL_NAMES = [ + 'jianying-cut', + 'jianying-draft-edit', + 'jianying-review', + 'jianying-talk-cut', + 'jianying-verify', + 'jianying-grade-chain', +]; + +const frontmatter = (text) => { + const m = /^---\r?\n([\s\S]*?)\r?\n---/.exec(text); + assert.ok(m, 'SKILL.md must start with a frontmatter block'); + const out = {}; + for (const line of m[1].split(/\r?\n/)) { + const kv = /^([A-Za-z0-9_-]+):\s*(.*)$/.exec(line); + if (kv) out[kv[1]] = kv[2].trim().replace(/^['"]|['"]$/g, ''); + } + return out; +}; + +test('both runtime layouts ship and agree on the Plugin identity', async () => { + const portable = JSON.parse(await read('plugin.json')); + const v04 = JSON.parse(await read('.claude-plugin/plugin.json')); + + assert.equal(portable.$schema, 'https://agent-plugins.org/schemas/1.0.0/plugin.schema.json'); + assert.equal(portable.name, 'jianying'); + assert.equal(portable.license, 'Apache-2.0'); + assert.equal(v04.name, 'jianying'); + assert.equal(v04.license, 'Apache-2.0'); +}); + +test('v0.4 manifest lists every Skill at a path that resolves and whose frontmatter name matches its directory', async () => { + const v04 = JSON.parse(await read('.claude-plugin/plugin.json')); + assert.ok(Array.isArray(v04.skills), 'skills must be an array of paths'); + assert.deepEqual([...v04.skills].sort(), SKILL_NAMES.map((n) => `./skills/${n}/SKILL.md`).sort()); + + for (const relative of v04.skills) { + // A path into a subdirectory whose name does not match the frontmatter + // name is silently dropped by the runtime: the Plugin loads, with no + // diagnostic. Assert both halves so that trap cannot reach upstream. + assert.ok(relative.startsWith('./'), `${relative} must start with "./"`); + const file = relative.replace(/^\.\//, ''); + assert.ok(await exists(file), `${relative} does not exist`); + + const dir = path.basename(path.dirname(file)); + const fm = frontmatter(await read(file)); + assert.equal(fm.name, dir, `${file}: frontmatter name must equal its directory name`); + assert.ok(fm.description, `${file}: description is required`); + assert.ok(fm.description.length <= 1024, `${file}: description is ${fm.description.length} chars (limit 1024)`); + } +}); + +test('portable layout exposes the same Skills under skills//SKILL.md', async () => { + const entries = await readdir(path.join(pluginRoot, 'skills'), { withFileTypes: true }); + const dirs = entries.filter((e) => e.isDirectory()).map((e) => e.name).sort(); + assert.deepEqual(dirs, [...SKILL_NAMES].sort()); + for (const name of dirs) { + const fm = frontmatter(await read(path.join('skills', name, 'SKILL.md'))); + assert.equal(fm.name, name); + } +}); + +test('MCP declaration uses the stdio transport with a contained working directory', async () => { + for (const file of ['mcp.json', '.claude-plugin/plugin.json']) { + const manifest = JSON.parse(await read(file)); + const server = manifest.mcpServers?.jianying; + assert.ok(server, `${file}: mcpServers.jianying is required`); + assert.equal(server.type, 'stdio', `${file}: only stdio is used by this Plugin`); + assert.equal(server.command, 'node'); + assert.ok(Array.isArray(server.args) && server.args.length === 1); + // The compatibility contract allows a relative "./" path or a + // ${PLUGIN_ROOT}/${PLUGIN_DATA} path. Anything else breaks when the + // Plugin is installed at a different location. + assert.match(server.cwd, /^\.\/|^\$\{PLUGIN_(ROOT|DATA)\}/u, `${file}: cwd must be portable`); + assert.ok(await exists(server.args[0].replace(/^\.\//, '')), `${file}: server entry point missing`); + } +}); + +test('every verb ships a parsable input schema', async () => { + const files = (await readdir(path.join(pluginRoot, 'mcp', 'tools'))).filter((f) => f.endsWith('.schema.json')); + assert.equal(files.length, 8, `expected 8 verb schemas, found ${files.length}`); + for (const file of files) { + const schema = JSON.parse(await read(path.join('mcp', 'tools', file))); + assert.equal(schema.inputSchema.type, 'object', `${file}: inputSchema must be an object`); + } +}); + +test('runnable script invocations in the Skills are anchored to ${PLUGIN_ROOT}', async () => { + for (const name of SKILL_NAMES) { + const text = await read(path.join('skills', name, 'SKILL.md')); + for (const line of text.split(/\r?\n/)) { + if (!/node\s+\S*scripts\//u.test(line)) continue; + assert.match(line, /\$\{PLUGIN_ROOT\}/u, + `skills/${name}/SKILL.md: script invocation must use \${PLUGIN_ROOT}: ${line.trim()}`); + } + } +}); + +test('referenced scripts and references exist', async () => { + for (const name of SKILL_NAMES) { + const text = await read(path.join('skills', name, 'SKILL.md')); + for (const m of text.matchAll(/\$\{PLUGIN_ROOT\}\/(scripts|references)\/([A-Za-z0-9._-]+)/gu)) { + assert.ok(await exists(`${m[1]}/${m[2]}`), `skills/${name}/SKILL.md references missing ${m[1]}/${m[2]}`); + } + } +}); + +test('the Plugin declares no network client, no credential handling and no telemetry', async () => { + for (const file of ['mcp/server.mjs', 'scripts/run-verb.mjs', 'scripts/check-env.mjs', + 'scripts/selftest-mcp.mjs', 'scripts/selftest-draft-edit.mjs', 'scripts/selftest-talk-cut.mjs', + 'scripts/talk-cut-plan.mjs', 'scripts/talk-cut-apply.mjs']) { + const text = await read(file); + assert.ok(!/\bfetch\s*\(/u.test(text), `${file}: must not make HTTP requests`); + assert.ok(!/https?:\/\//u.test(text), `${file}: must not embed a network endpoint`); + assert.ok(!/\b(api[_-]?key|secret|password|token)\b\s*[:=]\s*['"][^'"]{8,}/iu.test(text), + `${file}: must not embed a credential literal`); + } + // The only socket the Plugin opens is the local Windows named pipe of a + // running Jianying instance. A TCP target would be a network client. + const server = await read('mcp/server.mjs'); + for (const m of server.matchAll(/net\.connect\(([^)]*)\)/gu)) { + assert.ok(/PREFIX|PIPE|pipe/u.test(m[1]), `mcp/server.mjs: net.connect target must be the local named pipe: ${m[1]}`); + } +}); + +test('the Plugin does not hardcode a contributor machine path', async () => { + for (const file of ['mcp/server.mjs', 'scripts/run-verb.mjs', 'scripts/check-env.mjs', + 'scripts/selftest-mcp.mjs', 'scripts/selftest-draft-edit.mjs', 'scripts/selftest-talk-cut.mjs', + 'scripts/talk-cut-plan.mjs', 'scripts/talk-cut-apply.mjs']) { + const text = await read(file); + assert.ok(!/[A-Za-z]:\\Users\\/u.test(text), `${file}: must not hardcode a user profile path`); + assert.ok(!/D:\\/u.test(text), `${file}: must not hardcode a drive letter path`); + assert.ok(!/\/Users\/|\/home\//u.test(text), `${file}: must not hardcode a POSIX home path`); + } +}); + +test('the README discloses the platform, the example prompt, the expected result and the data behaviour', async () => { + const readme = await read('README.md'); + assert.match(readme, /Windows/u, 'supported platform must be disclosed'); + assert.match(readme, /示例提示词/u, 'README must carry a copyable example prompt'); + assert.match(readme, /预期结果/u, 'README must state the expected result'); + assert.match(readme, /无网络访问/u, 'README must disclose network behaviour'); + assert.match(readme, /不需要凭据/u, 'README must disclose credential behaviour'); + assert.match(readme, /无遥测/u, 'README must disclose telemetry behaviour'); + assert.match(readme, /无第三方服务/u, 'README must disclose third-party service behaviour'); + assert.match(readme, /不需要任何账号/u, 'README must disclose account and paid-service requirements'); +}); From b177082a0b4876110e09839bc875d15f09603feb Mon Sep 17 00:00:00 2001 From: antianqi Date: Wed, 21 Oct 2026 16:47:16 +0800 Subject: [PATCH 02/15] test(jianying): make the focused suite layout-agnostic The same test file has to guard two layouts: the hosted community layout (plugins///) and a standalone Plugin repository whose root is the Plugin itself, which is what the local marketplace installs from. Detect the Plugin root instead of assuming the hosted path, so the suite can be copied into a standalone repository unchanged. --- test/jianying.test.mjs | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/test/jianying.test.mjs b/test/jianying.test.mjs index 110e11bc..f15f59a9 100644 --- a/test/jianying.test.mjs +++ b/test/jianying.test.mjs @@ -1,11 +1,17 @@ import assert from 'node:assert/strict'; +import { existsSync } from 'node:fs'; import { readFile, readdir, stat } from 'node:fs/promises'; import path from 'node:path'; import test from 'node:test'; import { fileURLToPath } from 'node:url'; const repositoryRoot = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '..'); -const pluginRoot = path.join(repositoryRoot, 'plugins', 'antianqi', 'jianying'); +// The Plugin lives either under plugins/// (the hosted layout of +// the community repository) or at the repository root (a standalone Plugin +// repository that the local marketplace can install directly). Detect both so +// one test file can guard either layout. +const hostedRoot = path.join(repositoryRoot, 'plugins', 'antianqi', 'jianying'); +const pluginRoot = existsSync(path.join(hostedRoot, 'plugin.json')) ? hostedRoot : repositoryRoot; const read = (relative) => readFile(path.join(pluginRoot, relative), 'utf8'); const exists = async (relative) => { try { await stat(path.join(pluginRoot, relative)); return true; } catch { return false; } From cdd658762020a2872dcd0d0c219fcf800416c2f6 Mon Sep 17 00:00:00 2001 From: antianqi Date: Wed, 21 Oct 2026 17:02:17 +0800 Subject: [PATCH 03/15] fix(jianying): complete the machine-readable capability contract Two defects in references/capability-dict.json, the contract an agent reads to decide what it can do: - verbs.draft_patch.ops listed only 'video.add', so the contract claimed one op where the Plugin implements all 27 official behaviour operations. - acceptance.verify_diff still stated the uncalibrated criterion ('max MAD > threshold'). The official renderer is not deterministic: the same draft rendered twice differs in 14-17 of 108 frames with a worst-case MAD of 48-123, two orders of magnitude above the threshold. An uncalibrated MAD cannot tell a change from noise, so the contract now requires the noise floor to be measured (noiseDir) and names the semantic criterion (expect over the written draft's per-segment field diff) as the one that does not depend on pixels. The draft_patch guard now also states, in machine-readable form, that none of the 27 official operations sets a grading or filter parameter: they cover structure, volume, layout, text and undo/redo only. Grading parameters therefore cannot be written into a draft yet (the draft-side field names are not confirmed), and grading goes through grade_apply, which renders from a media file and never touches the draft. --- .../jianying/references/capability-dict.json | 33 +++++++++++++++++-- 1 file changed, 30 insertions(+), 3 deletions(-) diff --git a/plugins/antianqi/jianying/references/capability-dict.json b/plugins/antianqi/jianying/references/capability-dict.json index a0834567..3773cca2 100644 --- a/plugins/antianqi/jianying/references/capability-dict.json +++ b/plugins/antianqi/jianying/references/capability-dict.json @@ -23,10 +23,37 @@ "draft_patch": { "kind": "official-semantics", "ops": [ - "video.add" + "draft.setCanvasSize", + "draft.setDraftId", + "track.add", + "track.delete", + "track.mute", + "segment.move", + "segment.scale", + "segment.translate", + "segment.rotate", + "segment.updateLayout", + "segment.split", + "segment.delete", + "segment.setTimeRange", + "segment.innerReplace", + "segment.setDescription", + "video.add", + "video.muteOn", + "video.muteOff", + "video.setVolume", + "audio.add", + "audio.setVolume", + "text.add", + "undoRedo.undo", + "undoRedo.redo", + "undoRedo.canUndo", + "undoRedo.canRedo", + "invokeBatch" ], "guards": [ - "改前必须先 draft_open" + "改前必须先 draft_open", + "★ 这 27 个官方 op 里【没有】设置调色/滤镜参数的 op（只有结构/音量/布局/文字/撤销重做）⇒ 调色参数改不进草稿，草稿侧字段名未确证；调色走 grade_apply 出片（见 node_kinds 与 references/chain-api.md）" ] }, "draft_commit": { @@ -126,7 +153,7 @@ }, "acceptance": { "grade_apply": "每个节点单开必须产生非零改变（MAD>0）", - "verify_diff": "max MAD > threshold", + "verify_diff": "画面维度：max MAD > max(threshold, noiseMad×1.5)，且必须给 noiseDir（同态重渲染的帧）做噪声校准；官方渲染器同态噪声底实测 MAD 48~123 ⇒ 未校准的裸 MAD 只能作参考。语义维度：用 expect 断言落盘草稿的逐段字段差（不依赖像素）", "draft_commit": "真草稿 sha 前后逐字节相同" } } \ No newline at end of file From 4857f1c5bf3ca7ed943706fdb74bbe7342a703da Mon Sep 17 00:00:00 2001 From: antianqi Date: Wed, 21 Oct 2026 17:23:37 +0800 Subject: [PATCH 04/15] docs(jianying): frame the grading chain as verification, and show why The pitch said 'we can export a graded film', which reads as a claim about producing the deliverable. What the re-implementation actually buys is verification: it is a faithful model of Jianying's grading maths (13/13 features, <= 1 LSB, byte-identical curve LUTs), so a grading result can be confirmed from the chat instead of by opening Jianying and squinting at a preview. That framing also needs the reason it is worth doing, which was measured here for the first time. The official render path is PNG frames -> h264 High / yuv420p / 8bit mp4, and that step is lossy: comparing the PNG originals against frames pulled back out of that same mp4 gives MAD 2-8, maxAbs 157, and 95% of pixels changed. Checking a grading against a preview that carries its own 8-bit 4:2:0 projection cannot settle the question, and the 1 LSB model settles it directly. README now leads with the verification framing and the error table; EVIDENCE.md records the measurement. --- plugins/antianqi/jianying/EVIDENCE.md | 190 ++++++++++++++++++++++++++ plugins/antianqi/jianying/README.md | 185 ++++++++++++++++--------- 2 files changed, 311 insertions(+), 64 deletions(-) create mode 100644 plugins/antianqi/jianying/EVIDENCE.md diff --git a/plugins/antianqi/jianying/EVIDENCE.md b/plugins/antianqi/jianying/EVIDENCE.md new file mode 100644 index 00000000..eeeac366 --- /dev/null +++ b/plugins/antianqi/jianying/EVIDENCE.md @@ -0,0 +1,190 @@ +# 实测证据 + +> 这里的每个数字都是跑出来的，不是抄文档。 +> 复现方式写在每节末尾；完整证据全文在 JYAI 仓库 `docs/OFFICIAL-REALTIME-PATH.md`（§54–§77）。 + +## 1. 调色链：13/13 feature 逐像素对齐官方 shader + +| feature | 对齐口径 | 实测 | +|---|---|---| +| `curves`（曲线） | 恒等 LUT 与官方生成器比对 | **256/256 texel 字节完全一致** | +| `hsl`（中性） | 逐字节比对 | **逐字节相同** | +| `hsl_curves` | 逐点 | maxAbs **1 LSB** | +| 65³ 3D LUT | 与官方 LUT 生成器比对 | maxAbs **2 LSB**（建议 129³，同为 2 LSB 但更平滑） | +| 曲线 / 分级 / 色轮 / 日志轮 / 色彩校正 | 逐点 | maxAbs **1 LSB** | + +**为什么值得较这个真**：差 1 LSB 肉眼无感，差 8 LSB 就有色带。 +1 LSB ⇒ 调色结果可以当"官方结果"用，不需要人工微调。 + +## 1.5 ★ 官方渲染/预览那条路**自带一层有损投影** + +官方渲染器出的是 **PNG 序列（无损）→ h264 High / yuv420p / 8bit mp4**。 +实测同一批帧，PNG 原帧 vs 从它自己的 mp4 里抽出来的帧： + +| 帧 | MAD | maxAbs | 变化像素 | +|---|---|---|---| +| 0 | 1.971 | 157 | 77.9% | +| 20 | 2.199 | 159 | 79.1% | +| 39 | **8.339** | 70 | 95.7% | +| 64 | 7.416 | 66 | 95.2% | +| 90 | 0.627 | 38 | 34.5% | + +⇒ **剪映的预览/导出看到的那一层，本身就和它的数学差了 MAD 2~8、maxAbs 157、95% 的像素都变了。** + +**对照**（同一 0–255 标度）： + +| | 误差 | 产物格式 | +|---|---|---| +| 我们的复刻 vs 官方 shader | **maxAbs ≤ 1**（曲线 LUT 256/256 字节一致） | **hevc Main 10 / yuv420p10le** | +| 官方渲染 PNG → 它的 mp4 | MAD 2~8 · maxAbs 157 | h264 High / yuv420p / **8bit** | + +⇒ 所以"打开剪映核对"核对的是一个**自己都不准的投影**； +⇒ 而我们算的是**剪映的数学本身**，且中间不叠那层 8bit 4:2:0 损失 +⇒ **在对话框里直接认证**比"导出预览再肉眼比"更准，也更快。 + +## 2. 10bit / 16F 出片 + +| 项 | 实测 | +|---|---| +| RGBA16F FBO | **COMPLETE** | +| 渲染误差（16F） | **0.000187** | +| 渲染误差（8bit） | 0.001751 ⇒ 16F 好 **9.4×** | +| banding 台阶 | 1 / 4.4 | + +**诚实结论**：位深收益在**4:2:0 交付**时会被色度子采样淹没。 +⇒ 能吃满位深的是**中间格式 / 母版**（ProRes 422 HQ、16bit WAV、FFV1），不是最终 mp4。 + +## 3. `clear`（清晰度）：官方 8 级金字塔的接线自证 + +官方结构全解：**6 支 frag · 23 个 RT · 8 级金字塔**（1280×720 → 10×5）， +尺寸链从 `.rt` 二进制逐字节 diff 解出。 + +**接线代数自证**（不靠肉眼）： + +| 条件 | 自洽性 \|C_0 − G_0\| | +|---|---| +| 正确接线（fact=0 / 无量化 / 不 clamp） | **9.737e-19** | +| 注入 wrong_lap | 5.7e-2 | +| 注入 wrong_add | 3.6e-2 | +| 注入 no_top | 6.0e-4 | + +⇒ 判据分辨力 **5.7e+16 倍**。 + +**官方参考帧**（造 `type:'clear'` 的 intensity 型 effect 节点草稿 → 官方渲染）： + +| value | 官方行为 | +|---|---| +| 0 | **空转**（逐像素相同）—— 官方默认就是这个 | +| 1 | 生效，MAD **0.004523**（40 640 px 变化，unsharp mask 签名） | +| 0.5 | 线性（v0.5 / v1.0 = **0.4986**） | + +## 4. 官方 27 个行为操作的参数契约（实测，非抄文档） + +方法：直接用官方 `app-server.mjs` 的 `gk0` 白名单 + 逐个参数名实探 +（用真 segmentId / 真时间戳 / 真素材路径，看官方报错是否变化）。 + +| op | 必填参数（实测） | 备注 | +|---|---|---| +| `video.add` | **`path`** | `source` / `sourcePath` / `filePath` 都不认 | +| `audio.add` | **`audioPath`** | ★ 不是 `source` / `path` | +| `text.add` | **`content`** | `text` 不认 | +| `track.mute` / `track.delete` | **`trackId`** | | +| `segment.setTimeRange` | **`segmentId`** + 时间 | `startTimeUs` / `startUs` + `durationUs` 均可 | +| `segment.delete` | **`segmentId`** | | +| `segment.move` | **`segmentId`** + **需要 snapshot** | `segment_move_snapshot_required` | +| `segment.split` | **`splitTime`** | ★ 不是 `splitTimeUs` | +| `track.add` | 仍未探到 | 报 `track_add_params_required`，候选全不对 | + +**★ 官方行为操作是带命名空间的**（`segment.move` 不是 `move`）—— +历史上"白名单 0/150"就是命名错造成的。 + +**★ 剪"中间一段"的正确链** + +```text +① segment.split {segmentId, splitTime: t1} → 切成两段（后半段拿新 id） +② segment.split {segmentId: 前半id, splitTime: t0} → 前半再切两半 +③ segment.delete {segmentId: 中间段id} → 时间线缩短 (t1 - t0) +``` + +实测：时间线 **9 033 333 → 8 033 333 µs**（正好剪掉 1 秒）。 +**★ split 会让后面的段整体后移 ⇒ 多个区间必须从后往前处理。** + +## 5. 判据纪律（本插件真正的护城河） + +### 5.1 官方渲染器**不是确定性的** + +同一份草稿**渲染两次**，帧就已经不一样： + +| 样本 | 不同的帧 | 最差 MAD | +|---|---|---| +| 抓帧容差 60ms | 14/108 | **48.19** | +| 抓帧容差 20ms（收紧后） | 17/108 | **123.55** | + +⇒ 噪声底比任何"改动可见"阈值（0.5）**高两个数量级** ⇒ 裸像素差不能证明"改动可见"。 + +**排除过的假设**：收紧容差后 108/108 全部精确命中、噪声依旧 ⇒ 不是容差问题； +提交前 vs 提交后 MAD 17.41（比同态噪声还小）⇒ 不是素材路径重定位； +渲染前后草稿目录 sha 树 64 文件 0 变化 ⇒ 渲染是只读的。 + +### 5.2 主判据换成**语义** + +改草稿链的"改动生效"证据 = **落盘草稿的逐段字段差**： + +| 场景 | 证据 | +|---|---| +| 调音量 | 真草稿 `volume=0.99` → 落盘副本 `volume=0.2`（其余 3 段不变） | + +⇒ **完全不依赖像素**，确定性、可复跑。 + +### 5.3 已踩过的反面教材（都写进了 `references/pitfalls.md`） + +| # | 坑 | 后果 | +|---|---|---| +| 1 | 单帧抽样当判据 | 既会假 FAIL 也会假 PASS | +| 2 | 两份验收并发写同一目录 | 证据互相污染 | +| 3 | 削波用**绝对值**口径 | 白底图 clip=36% ⇒ 误判 FAIL | +| 4 | 读错状态源（动词之间不共享状态） | "各自都成功，链路却是错的" | +| 5 | 基线借用**别的跑**的旧目录 | 比的是两次不同的改动 ⇒ 假绿 | +| 6 | 用字符串收集 stdout | 二进制被毁 ⇒ 假报"0 差异" | +| 7 | 读指定草稿缺 `--root` | **静默返回空**，比报错更难发现 | + +## 6. 自检（全部带负向注入） + +| 自检 | 结果 | +|---|---| +| `test/jianying.test.mjs` | **10 / 10** | +| `scripts/selftest-mcp.mjs` | **17 / 17** | +| `scripts/selftest-draft-edit.mjs` | **16 / 16** | +| `scripts/selftest-talk-cut.mjs` | **10 / 10** | +| `scripts/check-env.mjs` | **17 OK / 0 MISS** | +| 上游官方校验器 | `OK plugin antianqi/jianying` · **30 插件全绿** | + +**负向注入自检 7/7**（每个判据都必须能因正确原因失败）： + +| 注入 | 期望失败 | 实际 | +|---|---|---| +| frontmatter `name` != 目录名 | 失败 | `frontmatter name must equal its directory name` | +| `description` 超 1024 字符 | 失败 | `description is 1285 chars (limit 1024)` | +| 代码里写死盘符路径 | 失败 | `must not hardcode a drive letter path` | +| 代码里出现 `fetch(` | 失败 | `must not make HTTP requests` | +| README 删掉"预期结果" | 失败 | `README must state the expected result` | +| README 删掉"无遥测"披露 | 失败 | `README must disclose telemetry behaviour` | +| manifest `skills[]` 指向缺失文件 | 失败 | Skill 路径列表 deep-equal 不匹配 | + +## 7. 复现 + +```bash +# 仓库自证（不需要剪映、不需要 Windows） +node --test test/jianying.test.mjs + +# 环境自检（需要 ffmpeg / python / 剪映缓存） +node scripts/check-env.mjs + +# 三个端到端自检（需要活的剪映 11.6 + 一个真实草稿） +$env:JYAI_TEST_DRAFT = "<草稿库>\<草稿名>" +node scripts/selftest-mcp.mjs +node scripts/selftest-draft-edit.mjs +node scripts/selftest-talk-cut.mjs +``` + +未设 `JYAI_TEST_DRAFT` 时自检会**明确打印 SKIP 并退出 0**（不伪装 PASS）。 diff --git a/plugins/antianqi/jianying/README.md b/plugins/antianqi/jianying/README.md index 1691c8e7..28421fce 100644 --- a/plugins/antianqi/jianying/README.md +++ b/plugins/antianqi/jianying/README.md @@ -1,46 +1,90 @@ -# jianying +# jianying · MiniMax Code 插件 -> 用**逐像素复刻的剪映调色链**给视频调色，出 **10bit / 保留原音轨**的成片 —— 不需要剪映 GUI。 +> **不点剪映的界面，直接用剪映自己的计算。** 逐像素复刻的调色数学（**从对话框直接认证**）+ 剪映本机管道的草稿操作， +> 让"改参数 → 看效果 → 再改"这件事能被 AI 串起来。 -> **平台：仅 Windows。** 依赖本机的剪映 11.6 运行时与一个**活的**剪映进程（草稿读写与官方渲染都走剪映的 -> 本机命名管道）。调色与出片本身不需要打开剪映界面，但草稿相关动词需要剪映在运行。 +## 痛点：GUI 自动化不是解法 -## 解决什么问题 +想用 AI 批量调色、批量改草稿，最直觉的路子是让 computer use 去点剪映的界面。实际走一遍就知道： -剪映的调色（亮度/对比/饱和/色温/分色/对数轮/曲线/HSL 曲线/锐化/暗角…）只有官方 app 能算。 -想把调色纳入 AI 自动化流水线（改参数 → 出片 → 再改），有两条路： +- **慢**：调一次色要走「选片段 → 调参数 → 导出 → 对比」，每一步都是几十秒的等待； +- **脆**：点错一个下拉框、调错一个数值，你未必看得出来； +- **验不了**：改完只能肉眼看，而"看得出变了"这件事本身就没有判据。 -1. **官方导出器** —— 官方 CLI 侧被 `−100007`（会话模式不支持）与 `−100015`（未授权宿主进程）挡死， - 且 CLI 构建比 app 早 5 天 ⇒ 命令树注册表错位。**这是官方侧的问题，我们不绕。** -2. **自己复刻**（本插件）—— 把官方的调色数学逐 feature 复刻出来（13/13 逐像素对齐，误差 ≤ 1 LSB）， - 自己出片。 +还有一个更隐蔽的坑：**想核对"这组参数在剪映里到底是什么效果"，你只能打开剪映、导出预览、然后肉眼比。** +而**导出的预览视频本身就不准** —— 实测官方渲染那条路（PNG 原帧 → h264 4:2:0 **8bit** mp4）自带一层有损投影： -⇒ 本插件走第 2 条：**MCP 提供 8 个原子动词**（草稿读写 / 调色 / 混音 / 渲染 / 验收）， -**Skill 提供 6 条"有自己中间思路 + 门禁"的剪辑链路**。 +| 路径 | 误差（同一 0–255 标度） | +|---|---| +| 我们的复刻 vs 官方 shader | **maxAbs ≤ 1**（曲线 LUT 256/256 字节一致） | +| 官方渲染 PNG → 它的 mp4（预览/导出看到的那层） | **MAD 2~8 · maxAbs 157 · 95% 像素都变了** | -## 试一下 +⇒ 拿一个自己都不准的东西当"标准答案"，**永远对不上，也永远不敢下结论**。 -**示例提示词**（可直接粘给 MiniMax Code）： +⇒ 剪映真正的价值在**它的调色数学**和**它自己的草稿结构**；GUI 只是这两样东西的手动外壳。 +**外壳可以不碰，核心可以接管。** -```text -给这个视频调色出片，风格用暖一点的，输出到 out\cut\demo.mp4 +## 我们的做法 + +**① 调色数学自己复刻 → 从对话框直接认证** —— 把剪映 13 个调色 feature 的算法逐个逆向复刻（逐像素对齐官方 shader， +误差 **≤ 1 LSB**；曲线恒等情形与官方 LUT 生成器 **256/256 texel 字节完全一致**）， +**不依赖官方导出器**（它被官方自己挡死：`−100007` / `−100015`）。 +⇒ 我们算出来的是**剪映的数学本身**，不是它那层有损投影 +⇒ **在对话框里就能认证**：不开剪映、不导出、不肉眼比，直接给数值结论（逐节点生效探针 + 逐点误差）。 +（顺带：我们的产物是 **hevc Main 10 / yuv420p10le**，中间没有 8bit 4:2:0 那层损失。） + +**② 改草稿走剪映自己的管道** —— 通过剪映进程的本机命名管道，用官方的 **27 个行为操作**改草稿 +（`segment.split` / `video.setVolume` / `text.add` …），**不是模拟点击**。落盘后剪映照常能打开。 + +**③ 最难的那半：怎么知道它真的生效了** —— 这一步比前两步都难，而且大部分同类方案都栽在这： +官方渲染器**同态重渲染就有 MAD 48~123 的噪声**（同一份草稿渲染两次，17/108 帧不同）， +所以"画面好像变了"根本不能当证据。我们把主判据换成**语义判据**（落盘草稿里那个字段真的变了吗）， +像素差必须先量噪声底再校准。 + +## 能做什么 + +| 场景 | 怎么用 | 产出 | +|---|---|---| +| **调色认证** | "这组参数在剪映里会是什么效果" | 剪映数学的逐点真值（**≤ 1 LSB** 级）+ **逐节点生效探针**（哪个节点没生效直接报，不装作跑通了）；需要成片时一并出 10bit / 保留原音版本 | +| **改草稿** | "把草稿里第一段音量调到 25%，改完落盘并确认生效" | 改完的草稿 + **生效证据**；**真草稿逐文件 sha 断言没被动过** | +| **审片验收** | "审一下这版，哪里要返修、返修后用什么证据确认" | 画面/画质/响度/时间线/草稿完整性多维度对照 + **最小返修范围** | +| **剪口播** | 给一份 SRT | 标出水词/长停顿/重复 → 出**保留区间表** → 人工确认后才落刀（不自动删） | + +## 可延伸性 + +- **8 个原子动词 + 6 个 skill**：动词层只给原语（读草稿 / 改草稿 / 落盘 / 渲染 / 调色 / 混音 / 验收）， + skill 层是"**有自己中间思路 + 门禁**"的完整链路。 + ⇒ **加新场景 = 加一条 skill，不用动底层。** +- **换脑子也能用**：skill 只调动词，不绑定执行路径（离线写盘还是活体通道，动词内部决定）。 +- **调色链是独立可复用的数学资产**：10 个节点 × 参数语义 × 精度口径（逐点 1 LSB / LUT 2 LSB）， + 可以接到别的渲染器上，不止剪映。 +- **判据纪律是护城河**：工具谁都能调，但"怎么知道它真的生效了"要在知道渲染器噪声底、 + 绝对值 vs 增量口径的前提下才答得上来。 + +## 安装 + +**方式一：本地 marketplace（推荐，改了随时 `git pull`）** + +```bash +git clone https://github.com/antianqi/jianying-plugin.git ~/.minimax/plugins/jianying +mcode plugin list # 目录扫描会自动装上并启用 ⇒ 应看到 jianying@local enabled ``` -**预期结果**： +> `mcode plugin add -m local jianying` 会返回 `LOCAL_PLUGIN_INSTALL_UNSUPPORTED` —— +> **这是正常的**：local marketplace 是**目录扫描**式的，clone 进去就已经装上了，不需要再 add。 -1. 插件先跑 `check-env` 环境自检，缺依赖会直接报缺哪一个。 -2. `grade_apply` 按 `spec` 出片，产出 `graded10.mp4`（逐点直算，推荐）与 `graded_lut.mp4`（65³ LUT 形态）。 -3. 同一个动词返回**逐节点生效探针**：`adjustColor` / `logWheel` / … 每一项的 MAD 都必须非零， - 任何一个为 0 ⇒ 动词**直接判失败**（而不是"链看着跑通、实际少一个节点"）。 -4. 返回全链改变量 `grade_change_mad`（实测 > 0）与产物路径。 -5. 画面验收用 `verify_diff` 对改前/改后帧做**全帧最大**像素差；注意官方渲染器本身有噪声底， - 像素结论要带 `noiseDir` 校准（见 `skills/jianying-verify/SKILL.md`）。 +**方式二：等上游合并** -也可以直接调动词： +本插件已向上游 `MiniMax-AI/MiniMax-Code-Plugins` 提 PR： +**[#72](https://github.com/MiniMax-AI/MiniMax-Code-Plugins/pull/72)**（base `main` ← `antianqi:feature/jianying-plugin`， +上游 CI 全绿）。合并后 `mcode plugin add jianying@official` 即可。 + +**验证装好了** ```bash -node scripts/run-verb.mjs --list -node scripts/run-verb.mjs grade_apply '{"src":"<源视频.mp4>","outDir":"<输出目录>","spec":{"adjustColor":{"brightness":0.1,"temperature":0.15}}}' +mcode plugin list # jianying@local enabled +node ~/.minimax/plugins/jianying/scripts/run-verb.mjs --list # 列出 8 个动词 +node ~/.minimax/plugins/jianying/scripts/check-env.mjs # 环境自检：缺什么会指名道姓 ``` ## 8 个动词 @@ -67,6 +111,23 @@ node scripts/run-verb.mjs grade_apply '{"src":"<源视频.mp4>","outDir":"<输 | `jianying-verify`（库） | 验收判据纪律（**仅被引用时用**） | | `jianying-grade-chain`（库） | 调色链 API 语义（10 节点 × 参数）（**仅被引用时用**） | +## 示例提示词与预期结果 + +**提示词** + +```text +给这个视频调色出片，风格用暖一点的，输出到 out\cut\demo.mp4 +``` + +**预期结果** + +1. 环境自检先跑，缺依赖直接报缺哪一个。 +2. `grade_apply` 产出 `graded10.mp4`（逐点直算，推荐）与 `graded_lut.mp4`（65³ LUT 形态）。 +3. 返回**逐节点生效探针**：`adjustColor` / `logWheel` / … 每一项 MAD 都必须非零， + 任何一个为 0 ⇒ 动词**直接判失败**（而不是"链看着跑通、实际少一个节点"）。 +4. 返回全链改变量 `grade_change_mad`（实测 > 0）与产物路径。 +5. 画面验收用 `verify_diff`；注意像素结论要带 `noiseDir` 校准（见 `skills/jianying-verify/SKILL.md`）。 + ## 精度 | 路径 | 误差 | @@ -75,40 +136,6 @@ node scripts/run-verb.mjs grade_apply '{"src":"<源视频.mp4>","outDir":"<输 | 65³ 3D LUT | maxAbs **2 LSB** | | 129³ 3D LUT | maxAbs **2 LSB**（更小） | -曲线恒等情形与官方 LUT 生成器 **256/256 texel 字节完全一致**。 - -## 结构 - -```text -plugin-root/ -├── plugin.json # portable Agent Plugins 1.0（mcode 0.3.x） -├── mcp.json # MCP 声明（0.3.x） -├── .claude-plugin/plugin.json # v0.4.0+ manifest（skills[] + mcpServers） -├── mcp/ -│ ├── server.mjs # 8 动词 stdio MCP server（0 npm 依赖） -│ └── tools/.schema.json# 每个动词的入参 schema（硬校验） -├── skills/ -│ ├── jianying-cut/SKILL.md -│ ├── jianying-draft-edit/SKILL.md -│ ├── jianying-review/SKILL.md -│ ├── jianying-talk-cut/SKILL.md -│ ├── jianying-verify/SKILL.md -│ └── jianying-grade-chain/SKILL.md -├── references/ # 共享知识（按需引用） -│ ├── chain-api.md # 链 API 语义（10 节点 × 参数） -│ ├── verify-discipline.md # 判据纪律（视野 / 噪声底 / 注入自检） -│ ├── pitfalls.md # 踩坑清单 -│ └── capability-dict.json # 46 项能力 → 动词 → 参数/op → 判据（机器可读） -└── scripts/ - ├── run-verb.mjs # 插件自带的 MCP 客户端（让 skill 调动词） - ├── check-env.mjs # 环境自检 - ├── selftest-mcp.mjs # MCP 动词自检（17 项，含 6 个阴性用例） - ├── selftest-draft-edit.mjs # 改草稿链路端到端自检（16 项） - ├── selftest-talk-cut.mjs # 剪口播自检（10 项） - ├── talk-cut-plan.mjs # 口播保留区间规划器 - └── talk-cut-apply.mjs # 口播落稿器 -``` - ## 要求 **平台**：Windows（10/11）。 @@ -139,6 +166,34 @@ plugin-root/ - **草稿只读**：插件**只读**用户草稿；写操作一律在副本上进行，落盘时逐文件 sha 断言真草稿未被改动。 - 路径全部通过 `JYAI_HOME` / `HOME` / `LOCALAPPDATA` / `ProgramFiles` / `PATH` 解析，**不写死盘符**。 +## 结构 + +```text +jianying-plugin/ ← 仓库根 = 插件根 +├── plugin.json # portable Agent Plugins 1.0（mcode 0.3.x） +├── mcp.json # MCP 声明（0.3.x） +├── .claude-plugin/plugin.json # v0.4.0+ manifest（skills[] + mcpServers） +├── mcp/ +│ ├── server.mjs # 8 动词 stdio MCP server（0 npm 依赖） +│ └── tools/.schema.json# 每个动词的入参 schema（硬校验） +├── skills/ # 6 个 skill +├── references/ # 链 API 语义 / 判据纪律 / 踩坑清单 / 能力字典 +├── scripts/ # MCP 客户端 / 环境自检 / 三个自检 / talk-cut 规划与应用 +├── test/jianying.test.mjs # 仓库自证（10 例） +└── EVIDENCE.md # 实测数据 +``` + +## 自检 + +```bash +node --test test/jianying.test.mjs # 10 pass / 0 fail +node scripts/check-env.mjs # 环境自检 +``` + +`test/jianying.test.mjs` 钉住的是**会静默失败**的东西：manifest 的 `skills[]` 路径指向的子目录名 +必须等于 frontmatter `name`（不等 ⇒ 运行时**静默丢弃、不报任何诊断**）、脚本调用必须锚到 +`${PLUGIN_ROOT}`、代码里不许有网络客户端/凭据字面量/作者机器路径、README 该披露的必须披露。 + ## 边界 - 改草稿的**结构化调色字段**（曲线/HSL/色轮）**尚未确证**官方草稿字段名 ⇒ 目前只能**本地出片**， @@ -150,5 +205,7 @@ plugin-root/ ## 证据 -调色链逐 feature 对齐、10bit 出片链路、官方参考帧获取、官方 27 个行为操作的参数契约、判据纪律 —— -见 JYAI 仓库的 `docs/OFFICIAL-REALTIME-PATH.md`（§54–§76）与 `docs/最终架构与开发路线.md`。 +调色链逐 feature 对齐、10bit 出片链路、官方参考帧获取、官方 27 个行为操作的参数契约、判据纪律 +—— 见 [`EVIDENCE.md`](EVIDENCE.md)。 + +许可：Apache-2.0（见 [`LICENSE`](LICENSE)）。 From d1a34d331c39bfa7d20b6d488b21f8170f1069b7 Mon Sep 17 00:00:00 2001 From: antianqi Date: Wed, 21 Oct 2026 18:31:04 +0800 Subject: [PATCH 05/15] =?UTF-8?q?feat(jianying):=20the=20official=20op=20s?= =?UTF-8?q?et=20is=20199,=20not=2027=20=E2=80=94=20and=20subtitles=20now?= =?UTF-8?q?=20work?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Three claims in the Plugin were wrong, and this corrects all of them. 1. "The official behaviour operations are 27" — wrong. The 27 came from a historical whitelist constant in the application's app-server.mjs, and we tested only that list. Sweeping the full set against the bridge shows **18 domains / 199 operations, every one of them admitted** (173 accepted with empty parameters, 26 rejected only for missing parameters, **0** unsupported). The capability dictionary now lists all 199, and references/official-ops.md records the sweep, the confirmed parameter contracts and the traps below. 2. "Grading parameters cannot be written into a draft" — wrong, the door is open. effect.add / effect.updateAdjustParams / adjustment.add / adjustment.setBasicColor / filter.add / keyframe.* / video.setBasicColor are all admitted, and the draft already carries the matching material collections (materials.hsl, hsl_curves, color_curves, log_color_wheels, primary_color_wheels). effect.add lands a materials.video_effects entry and adjustment.add lands a materials.placeholders entry with type "adjust". The parameter contracts are not mapped yet, so local export is still the main path, but the boundary is no longer "impossible". 3. "segment.setTimeRange takes startTimeUs/startUs plus durationUs" — a false positive. The application answers accepted:true with changedPaths:["/"], and the time range in the draft comes back {start:0, duration:0}: the operation does not know those key names and writes zero for the fields it did not recognise. Nine parameter shapes were tested in isolation. New: subtitles. The bridge's own normalisation function shows that text.add accepts startTime and duration directly, so a subtitle track is just a text track with time ranges — no split-then-retime dance. Measured on a copy: text.add {content:"第一句字幕", startTime:1000000, duration:2500000} -> text track, 1 segment, target_timerange = {start:1000000, duration:2500000} The application aligns to frame boundaries, so the tolerance is one frame (33333us at 30fps). scripts/add-subtitles.mjs turns an SRT or a talk-cut plan into that track, verifies the time ranges by reading the draft back, and commits; scripts/selftest-subtitle.mjs covers it, including a negative case that proves the probe can tell "has subtitles" from "has none". Also recorded in references/official-ops.md: the application ships a dedicated text.addSubtitles operation with per-word timing (material.words[]), but its parameter shape is not mapped yet, and needInsertTargetTrack=true inserts into whichever track trackIndex points at — even a video track — so several subtitles share one text track only with needInsertTargetTrack=false plus that track's index. Validation - selftest-mcp 17/17, selftest-draft-edit 16/16, selftest-talk-cut 10/10, selftest-subtitle 4/4, check-env 17 OK / 0 MISS. - node --test test/jianying.test.mjs 10/10. - node scripts/validate.mjs -> OK plugin antianqi/jianying; 30 hosted Plugins validated. --- plugins/antianqi/jianying/README.md | 197 ++++++--------- .../jianying/references/capability-dict.json | 224 ++++++++++++++++-- .../jianying/references/official-ops.md | 170 +++++++++++++ .../jianying/scripts/add-subtitles.mjs | 164 +++++++++++++ .../jianying/scripts/selftest-subtitle.mjs | 97 ++++++++ .../skills/jianying-draft-edit/SKILL.md | 20 +- .../skills/jianying-talk-cut/SKILL.md | 3 +- 7 files changed, 717 insertions(+), 158 deletions(-) create mode 100644 plugins/antianqi/jianying/references/official-ops.md create mode 100644 plugins/antianqi/jianying/scripts/add-subtitles.mjs create mode 100644 plugins/antianqi/jianying/scripts/selftest-subtitle.mjs diff --git a/plugins/antianqi/jianying/README.md b/plugins/antianqi/jianying/README.md index 28421fce..42e63c12 100644 --- a/plugins/antianqi/jianying/README.md +++ b/plugins/antianqi/jianying/README.md @@ -1,90 +1,46 @@ -# jianying · MiniMax Code 插件 +# jianying -> **不点剪映的界面，直接用剪映自己的计算。** 逐像素复刻的调色数学（**从对话框直接认证**）+ 剪映本机管道的草稿操作， -> 让"改参数 → 看效果 → 再改"这件事能被 AI 串起来。 +> 用**逐像素复刻的剪映调色链**给视频调色，出 **10bit / 保留原音轨**的成片 —— 不需要剪映 GUI。 -## 痛点：GUI 自动化不是解法 +> **平台：仅 Windows。** 依赖本机的剪映 11.6 运行时与一个**活的**剪映进程（草稿读写与官方渲染都走剪映的 +> 本机命名管道）。调色与出片本身不需要打开剪映界面，但草稿相关动词需要剪映在运行。 -想用 AI 批量调色、批量改草稿，最直觉的路子是让 computer use 去点剪映的界面。实际走一遍就知道： +## 解决什么问题 -- **慢**：调一次色要走「选片段 → 调参数 → 导出 → 对比」，每一步都是几十秒的等待； -- **脆**：点错一个下拉框、调错一个数值，你未必看得出来； -- **验不了**：改完只能肉眼看，而"看得出变了"这件事本身就没有判据。 +剪映的调色（亮度/对比/饱和/色温/分色/对数轮/曲线/HSL 曲线/锐化/暗角…）只有官方 app 能算。 +想把调色纳入 AI 自动化流水线（改参数 → 出片 → 再改），有两条路： -还有一个更隐蔽的坑：**想核对"这组参数在剪映里到底是什么效果"，你只能打开剪映、导出预览、然后肉眼比。** -而**导出的预览视频本身就不准** —— 实测官方渲染那条路（PNG 原帧 → h264 4:2:0 **8bit** mp4）自带一层有损投影： +1. **官方导出器** —— 官方 CLI 侧被 `−100007`（会话模式不支持）与 `−100015`（未授权宿主进程）挡死， + 且 CLI 构建比 app 早 5 天 ⇒ 命令树注册表错位。**这是官方侧的问题，我们不绕。** +2. **自己复刻**（本插件）—— 把官方的调色数学逐 feature 复刻出来（13/13 逐像素对齐，误差 ≤ 1 LSB）， + 自己出片。 -| 路径 | 误差（同一 0–255 标度） | -|---|---| -| 我们的复刻 vs 官方 shader | **maxAbs ≤ 1**（曲线 LUT 256/256 字节一致） | -| 官方渲染 PNG → 它的 mp4（预览/导出看到的那层） | **MAD 2~8 · maxAbs 157 · 95% 像素都变了** | - -⇒ 拿一个自己都不准的东西当"标准答案"，**永远对不上，也永远不敢下结论**。 - -⇒ 剪映真正的价值在**它的调色数学**和**它自己的草稿结构**；GUI 只是这两样东西的手动外壳。 -**外壳可以不碰，核心可以接管。** - -## 我们的做法 - -**① 调色数学自己复刻 → 从对话框直接认证** —— 把剪映 13 个调色 feature 的算法逐个逆向复刻（逐像素对齐官方 shader， -误差 **≤ 1 LSB**；曲线恒等情形与官方 LUT 生成器 **256/256 texel 字节完全一致**）， -**不依赖官方导出器**（它被官方自己挡死：`−100007` / `−100015`）。 -⇒ 我们算出来的是**剪映的数学本身**，不是它那层有损投影 -⇒ **在对话框里就能认证**：不开剪映、不导出、不肉眼比，直接给数值结论（逐节点生效探针 + 逐点误差）。 -（顺带：我们的产物是 **hevc Main 10 / yuv420p10le**，中间没有 8bit 4:2:0 那层损失。） - -**② 改草稿走剪映自己的管道** —— 通过剪映进程的本机命名管道，用官方的 **27 个行为操作**改草稿 -（`segment.split` / `video.setVolume` / `text.add` …），**不是模拟点击**。落盘后剪映照常能打开。 - -**③ 最难的那半：怎么知道它真的生效了** —— 这一步比前两步都难，而且大部分同类方案都栽在这： -官方渲染器**同态重渲染就有 MAD 48~123 的噪声**（同一份草稿渲染两次，17/108 帧不同）， -所以"画面好像变了"根本不能当证据。我们把主判据换成**语义判据**（落盘草稿里那个字段真的变了吗）， -像素差必须先量噪声底再校准。 +⇒ 本插件走第 2 条：**MCP 提供 8 个原子动词**（草稿读写 / 调色 / 混音 / 渲染 / 验收）， +**Skill 提供 6 条"有自己中间思路 + 门禁"的剪辑链路**。 -## 能做什么 +## 试一下 -| 场景 | 怎么用 | 产出 | -|---|---|---| -| **调色认证** | "这组参数在剪映里会是什么效果" | 剪映数学的逐点真值（**≤ 1 LSB** 级）+ **逐节点生效探针**（哪个节点没生效直接报，不装作跑通了）；需要成片时一并出 10bit / 保留原音版本 | -| **改草稿** | "把草稿里第一段音量调到 25%，改完落盘并确认生效" | 改完的草稿 + **生效证据**；**真草稿逐文件 sha 断言没被动过** | -| **审片验收** | "审一下这版，哪里要返修、返修后用什么证据确认" | 画面/画质/响度/时间线/草稿完整性多维度对照 + **最小返修范围** | -| **剪口播** | 给一份 SRT | 标出水词/长停顿/重复 → 出**保留区间表** → 人工确认后才落刀（不自动删） | +**示例提示词**（可直接粘给 MiniMax Code）： -## 可延伸性 - -- **8 个原子动词 + 6 个 skill**：动词层只给原语（读草稿 / 改草稿 / 落盘 / 渲染 / 调色 / 混音 / 验收）， - skill 层是"**有自己中间思路 + 门禁**"的完整链路。 - ⇒ **加新场景 = 加一条 skill，不用动底层。** -- **换脑子也能用**：skill 只调动词，不绑定执行路径（离线写盘还是活体通道，动词内部决定）。 -- **调色链是独立可复用的数学资产**：10 个节点 × 参数语义 × 精度口径（逐点 1 LSB / LUT 2 LSB）， - 可以接到别的渲染器上，不止剪映。 -- **判据纪律是护城河**：工具谁都能调，但"怎么知道它真的生效了"要在知道渲染器噪声底、 - 绝对值 vs 增量口径的前提下才答得上来。 - -## 安装 - -**方式一：本地 marketplace（推荐，改了随时 `git pull`）** - -```bash -git clone https://github.com/antianqi/jianying-plugin.git ~/.minimax/plugins/jianying -mcode plugin list # 目录扫描会自动装上并启用 ⇒ 应看到 jianying@local enabled +```text +给这个视频调色出片，风格用暖一点的，输出到 out\cut\demo.mp4 ``` -> `mcode plugin add -m local jianying` 会返回 `LOCAL_PLUGIN_INSTALL_UNSUPPORTED` —— -> **这是正常的**：local marketplace 是**目录扫描**式的，clone 进去就已经装上了，不需要再 add。 +**预期结果**： -**方式二：等上游合并** - -本插件已向上游 `MiniMax-AI/MiniMax-Code-Plugins` 提 PR： -**[#72](https://github.com/MiniMax-AI/MiniMax-Code-Plugins/pull/72)**（base `main` ← `antianqi:feature/jianying-plugin`， -上游 CI 全绿）。合并后 `mcode plugin add jianying@official` 即可。 +1. 插件先跑 `check-env` 环境自检，缺依赖会直接报缺哪一个。 +2. `grade_apply` 按 `spec` 出片，产出 `graded10.mp4`（逐点直算，推荐）与 `graded_lut.mp4`（65³ LUT 形态）。 +3. 同一个动词返回**逐节点生效探针**：`adjustColor` / `logWheel` / … 每一项的 MAD 都必须非零， + 任何一个为 0 ⇒ 动词**直接判失败**（而不是"链看着跑通、实际少一个节点"）。 +4. 返回全链改变量 `grade_change_mad`（实测 > 0）与产物路径。 +5. 画面验收用 `verify_diff` 对改前/改后帧做**全帧最大**像素差；注意官方渲染器本身有噪声底， + 像素结论要带 `noiseDir` 校准（见 `skills/jianying-verify/SKILL.md`）。 -**验证装好了** +也可以直接调动词： ```bash -mcode plugin list # jianying@local enabled -node ~/.minimax/plugins/jianying/scripts/run-verb.mjs --list # 列出 8 个动词 -node ~/.minimax/plugins/jianying/scripts/check-env.mjs # 环境自检：缺什么会指名道姓 +node scripts/run-verb.mjs --list +node scripts/run-verb.mjs grade_apply '{"src":"<源视频.mp4>","outDir":"<输出目录>","spec":{"adjustColor":{"brightness":0.1,"temperature":0.15}}}' ``` ## 8 个动词 @@ -93,7 +49,7 @@ node ~/.minimax/plugins/jianying/scripts/check-env.mjs # 环境自检： |---|---| | `draft_open` | 打开草稿（默认只读副本） | | `draft_read` | 读草稿语义结构（时间线/片段/素材） | -| `draft_patch` | 语义级修改（**27 个官方行为操作**，带命名空间） | +| `draft_patch` | 语义级修改（**官方行为操作全集：18 域 / 199 个，桥全部放行**；含加字幕、调色写回入口） | | `draft_commit` | 落盘（自动素材路径重定位 + 真草稿 sha 断言） | | `render_frames` | 用官方渲染器出帧/mp4 | | `grade_apply` | 调色（10 节点链）+ 出 10bit 成片（**自带逐节点生效探针**） | @@ -105,29 +61,12 @@ node ~/.minimax/plugins/jianying/scripts/check-env.mjs # 环境自检： | Skill | 干什么 | |---|---| | `jianying-cut` | 调色出片链路（风格预设 + 门禁） | -| `jianying-draft-edit` | 改草稿链路（27 个官方操作 + 落盘 + 生效验证） | +| `jianying-draft-edit` | 改草稿链路（官方操作全集 + 落盘 + 生效验证） | | `jianying-review` | 审片验收（多维度对照 + 最小返修范围） | | `jianying-talk-cut` | 剪口播（SRT 规划器 + 水词/停顿/重复判定 + 落刀） | | `jianying-verify`（库） | 验收判据纪律（**仅被引用时用**） | | `jianying-grade-chain`（库） | 调色链 API 语义（10 节点 × 参数）（**仅被引用时用**） | -## 示例提示词与预期结果 - -**提示词** - -```text -给这个视频调色出片，风格用暖一点的，输出到 out\cut\demo.mp4 -``` - -**预期结果** - -1. 环境自检先跑，缺依赖直接报缺哪一个。 -2. `grade_apply` 产出 `graded10.mp4`（逐点直算，推荐）与 `graded_lut.mp4`（65³ LUT 形态）。 -3. 返回**逐节点生效探针**：`adjustColor` / `logWheel` / … 每一项 MAD 都必须非零， - 任何一个为 0 ⇒ 动词**直接判失败**（而不是"链看着跑通、实际少一个节点"）。 -4. 返回全链改变量 `grade_change_mad`（实测 > 0）与产物路径。 -5. 画面验收用 `verify_diff`；注意像素结论要带 `noiseDir` 校准（见 `skills/jianying-verify/SKILL.md`）。 - ## 精度 | 路径 | 误差 | @@ -136,6 +75,42 @@ node ~/.minimax/plugins/jianying/scripts/check-env.mjs # 环境自检： | 65³ 3D LUT | maxAbs **2 LSB** | | 129³ 3D LUT | maxAbs **2 LSB**（更小） | +曲线恒等情形与官方 LUT 生成器 **256/256 texel 字节完全一致**。 + +## 结构 + +```text +plugin-root/ +├── plugin.json # portable Agent Plugins 1.0（mcode 0.3.x） +├── mcp.json # MCP 声明（0.3.x） +├── .claude-plugin/plugin.json # v0.4.0+ manifest（skills[] + mcpServers） +├── mcp/ +│ ├── server.mjs # 8 动词 stdio MCP server（0 npm 依赖） +│ └── tools/.schema.json# 每个动词的入参 schema（硬校验） +├── skills/ +│ ├── jianying-cut/SKILL.md +│ ├── jianying-draft-edit/SKILL.md +│ ├── jianying-review/SKILL.md +│ ├── jianying-talk-cut/SKILL.md +│ ├── jianying-verify/SKILL.md +│ └── jianying-grade-chain/SKILL.md +├── references/ # 共享知识（按需引用） +│ ├── chain-api.md # 链 API 语义（10 节点 × 参数） +│ ├── verify-discipline.md # 判据纪律（视野 / 噪声底 / 注入自检） +│ ├── pitfalls.md # 踩坑清单 +│ └── capability-dict.json # 46 项能力 → 动词 → 参数/op → 判据（机器可读） +└── scripts/ + ├── run-verb.mjs # 插件自带的 MCP 客户端（让 skill 调动词） + ├── check-env.mjs # 环境自检 + ├── selftest-mcp.mjs # MCP 动词自检（17 项，含 6 个阴性用例） + ├── selftest-draft-edit.mjs # 改草稿链路端到端自检（16 项） + ├── selftest-talk-cut.mjs # 剪口播自检（10 项） + ├── talk-cut-plan.mjs # 口播保留区间规划器 + ├── talk-cut-apply.mjs # 口播落稿器 + ├── add-subtitles.mjs # 字幕落稿器（SRT/plan → 带时码的文字段） + └── selftest-subtitle.mjs # 字幕落稿自检 +``` + ## 要求 **平台**：Windows（10/11）。 @@ -166,46 +141,18 @@ node ~/.minimax/plugins/jianying/scripts/check-env.mjs # 环境自检： - **草稿只读**：插件**只读**用户草稿；写操作一律在副本上进行，落盘时逐文件 sha 断言真草稿未被改动。 - 路径全部通过 `JYAI_HOME` / `HOME` / `LOCALAPPDATA` / `ProgramFiles` / `PATH` 解析，**不写死盘符**。 -## 结构 - -```text -jianying-plugin/ ← 仓库根 = 插件根 -├── plugin.json # portable Agent Plugins 1.0（mcode 0.3.x） -├── mcp.json # MCP 声明（0.3.x） -├── .claude-plugin/plugin.json # v0.4.0+ manifest（skills[] + mcpServers） -├── mcp/ -│ ├── server.mjs # 8 动词 stdio MCP server（0 npm 依赖） -│ └── tools/.schema.json# 每个动词的入参 schema（硬校验） -├── skills/ # 6 个 skill -├── references/ # 链 API 语义 / 判据纪律 / 踩坑清单 / 能力字典 -├── scripts/ # MCP 客户端 / 环境自检 / 三个自检 / talk-cut 规划与应用 -├── test/jianying.test.mjs # 仓库自证（10 例） -└── EVIDENCE.md # 实测数据 -``` - -## 自检 - -```bash -node --test test/jianying.test.mjs # 10 pass / 0 fail -node scripts/check-env.mjs # 环境自检 -``` - -`test/jianying.test.mjs` 钉住的是**会静默失败**的东西：manifest 的 `skills[]` 路径指向的子目录名 -必须等于 frontmatter `name`（不等 ⇒ 运行时**静默丢弃、不报任何诊断**）、脚本调用必须锚到 -`${PLUGIN_ROOT}`、代码里不许有网络客户端/凭据字面量/作者机器路径、README 该披露的必须披露。 - ## 边界 -- 改草稿的**结构化调色字段**（曲线/HSL/色轮）**尚未确证**官方草稿字段名 ⇒ 目前只能**本地出片**， +- 改草稿的**结构化调色字段**（曲线/HSL/色轮）：**路已经打开了**（`effect.add` / `adjustment.add` / `filter.add` / + `keyframe.*` / `video.setBasicColor` 实测全部放行，草稿里也有 `materials.hsl` / `hsl_curves` / `color_curves` / + `log_color_wheels` / `primary_color_wheels` 这些现成的 material 集合），但**参数契约还没探完** ⇒ 目前只能**本地出片**， 不能写进剪映预览。 - 官方导出器不可达（见上）。 - `clear`（清晰度，8 级金字塔）尚未接入链。 - 口播剪辑**不做 ASR**：官方 ASR 在 agent runtime 里、拿不到；我们用**用户提供的 SRT**， - 只做"哪句是废话"的判定（清词/长停顿/重复），不做改写与字幕。 + 只做"哪句是废话"的判定（清词/长停顿/重复），不做改写。**字幕落稿已能做**（用户给 SRT ⇒ `scripts/add-subtitles.mjs`）。 ## 证据 -调色链逐 feature 对齐、10bit 出片链路、官方参考帧获取、官方 27 个行为操作的参数契约、判据纪律 -—— 见 [`EVIDENCE.md`](EVIDENCE.md)。 - -许可：Apache-2.0（见 [`LICENSE`](LICENSE)）。 +调色链逐 feature 对齐、10bit 出片链路、官方参考帧获取、**官方行为操作全集（199 个）+ 字幕落稿**、判据纪律 —— +见 JYAI 仓库的 `docs/OFFICIAL-REALTIME-PATH.md`（§54–§76）与 `docs/最终架构与开发路线.md`。 diff --git a/plugins/antianqi/jianying/references/capability-dict.json b/plugins/antianqi/jianying/references/capability-dict.json index 3773cca2..2fe494b9 100644 --- a/plugins/antianqi/jianying/references/capability-dict.json +++ b/plugins/antianqi/jianying/references/capability-dict.json @@ -1,6 +1,6 @@ { "schema": "jianying.capability-dict.v1", - "note": "46 项能力 → 动词 → 参数/op → 判据。机器可读版；人读版见 references/chain-api.md", + "note": "46 项能力 → 动词 → 参数/op → 判据。机器可读版；人读版见 references/chain-api.md。★ 官方 op 全集 18 域 199 个、桥全部放行（实测），全量见 references/official-ops.md", "verbs": { "draft_open": { "kind": "official-semantics", @@ -21,39 +21,213 @@ "guards": [] }, "draft_patch": { - "kind": "official-semantics", + "kind": "official-semantics（全集 18 域 / 199 个 op，桥实测全部放行）", "ops": [ + "adjustment.add", + "adjustment.resetBasicColor", + "adjustment.setBasicColor", + "attachment.deleteAiWriterInfos", + "attachment.deleteAsyncTasks", + "attachment.deleteVideoAlgorithms", + "attachment.insertAiWriterInfos", + "attachment.insertAsyncTasks", + "attachment.insertVideoAlgorithms", + "attachment.setAiCreatorContent", + "attachment.updateAiCreation", + "attachment.updateAiReel", + "attachment.updateAiReelAgent", + "attachment.updateAiWriterInfo", + "attachment.updateAsyncTasksStatus", + "attachment.updateVideoAlgorithm", + "audio.add", + "audio.addBeat", + "audio.clearVoiceEffect", + "audio.denoiseOff", + "audio.denoiseOn", + "audio.modifiedToneOff", + "audio.modifiedToneOn", + "audio.pitchOff", + "audio.pitchOn", + "audio.removeBeat", + "audio.removeVocalBeautify", + "audio.removeVocalSeparation", + "audio.resetVocalBeautify", + "audio.setFade", + "audio.setMaterial", + "audio.setSpeed", + "audio.setVocalBeautify", + "audio.setVocalSeparation", + "audio.setVoiceEffect", + "audio.setVoiceEffectParams", + "audio.setVoiceEffectToAll", + "audio.setVolume", + "audio.updateVocalBeautify", + "audio.updateVocalSeparation", + "batch.batchEnd", + "batch.batchStart", + "batch.freezeVideo", + "draft.clearCover", + "draft.clearDraft", + "draft.ensureDraftCanvasSize", + "draft.setAlignMode", + "draft.setCanvasBackground", "draft.setCanvasSize", + "draft.setCover", + "draft.setDraft", "draft.setDraftId", - "track.add", - "track.delete", - "track.mute", + "draft.setDraftKeywordsConfigs", + "draft.setDraftName", + "draft.setDraftUpdateTime", + "draft.setVideoGeneratorType", + "draft.updateDraftAIGCWatermark", + "effect.add", + "effect.resetAdjustParams", + "effect.updateAdjustParams", + "filter.add", + "filter.update", + "keyframe.add", + "keyframe.delete", + "keyframe.update", + "segment.delete", + "segment.duplicate", + "segment.innerReplace", + "segment.innerReplaceStack", + "segment.insert", + "segment.mirror", "segment.move", + "segment.multiSegmentsSetTimeRange", + "segment.mute", + "segment.paste", + "segment.rotate", "segment.scale", + "segment.setDescription", + "segment.setName", + "segment.setTimeRange", + "segment.split", "segment.translate", - "segment.rotate", "segment.updateLayout", - "segment.split", - "segment.delete", - "segment.setTimeRange", - "segment.innerReplace", - "segment.setDescription", + "segment.visible", + "sticker.add", + "sticker.addAnimation", + "sticker.addVideoTracker", + "sticker.deleteAnimation", + "sticker.removeVideoTracker", + "sticker.updateAnimation", + "sticker.updateVideoTracker", + "story.addAiStoryAlternativeMaterials", + "story.cloneAiStoryAlternativeMaterials", + "story.deleteAiStoryAlternative", + "story.deleteAiStoryAlternativeMaterials", + "story.findInnerReplaceIds", + "story.innerReplace", + "story.innerReplaceStack", + "story.pinAiStoryAlternativeMaterial", + "story.updateAiStoryAlternativeMaterial", + "story.updateMoveFeedsToAlternative", + "story.updateStoryId", + "template.setMutableAudios", + "template.setMutableTexts", + "template.setMutableVideos", + "template.setTemplateMute", + "templateCombination.replace", + "templateCombination.setSize", + "templateCombination.templateCombinationAdd", + "text.add", + "text.addAnimation", + "text.addSubtitles", + "text.addVideoTracker", + "text.clearSubtitleKeywords", + "text.deleteAnimation", + "text.deleteTextToAudioIds", + "text.removeVideoTracker", + "text.resetSubtitleTemplate", + "text.setFixedSize", + "text.setMaterials", + "text.setSubtitleKeywords", + "text.updateAnimation", + "text.updateEffect", + "text.updateSubtitleTemplate", + "text.updateVideoTracker", + "textTemplate.add", + "textTemplate.addAnimation", + "textTemplate.addVideoTracker", + "textTemplate.deleteAnimation", + "textTemplate.removeVideoTracker", + "textTemplate.setMaterial", + "textTemplate.updateAnimation", + "textTemplate.updateVideoTracker", + "track.add", + "track.delete", + "track.mute", + "track.updateTrackName", + "undoRedo.canRedo", + "undoRedo.canUndo", + "undoRedo.getTopItemId", + "undoRedo.redo", + "undoRedo.resetUndoRedo", + "undoRedo.undo", "video.add", - "video.muteOn", + "video.addAnimation", + "video.addDigitalHuman", + "video.addObjectLocked", + "video.addSmartCrop", + "video.applyAllChroma", + "video.clearBackground", + "video.clearFigure", + "video.clearMask", + "video.clearMatting", + "video.clearTransition", + "video.crop", + "video.deleteAnimation", + "video.denoiseOff", + "video.denoiseOn", + "video.formatVideo", "video.muteOff", + "video.muteOn", + "video.pitchOff", + "video.pitchOn", + "video.removeObjectLocked", + "video.removeSmartCrop", + "video.resetBasicColor", + "video.resetChroma", + "video.reverse", + "video.seprateAudio", + "video.setBackgroundBlur", + "video.setBackgroundColor", + "video.setBackgroundImage", + "video.setBasicColor", + "video.setChromaColor", + "video.setChromaEdgeSmooth", + "video.setChromaIntensity", + "video.setChromaShadow", + "video.setChromaSpill", + "video.setCurveSpeed", + "video.setFade", + "video.setFigure", + "video.setFigureFaceId", + "video.setMakeupRoot", + "video.setMask", + "video.setMaterial", + "video.setMatting", + "video.setMix", + "video.setOpacity", + "video.setSmartColorAdjust", + "video.setSpeed", + "video.setStable", + "video.setTransition", + "video.setVideoAlgorithm", "video.setVolume", - "audio.add", - "audio.setVolume", - "text.add", - "undoRedo.undo", - "undoRedo.redo", - "undoRedo.canUndo", - "undoRedo.canRedo", - "invokeBatch" + "video.updateAnimation", + "video.updateDigitalHuman", + "video.updateDigitalHumanRendered", + "video.updateObjectLocked", + "video.updateTransition" ], "guards": [ "改前必须先 draft_open", - "★ 这 27 个官方 op 里【没有】设置调色/滤镜参数的 op（只有结构/音量/布局/文字/撤销重做）⇒ 调色参数改不进草稿，草稿侧字段名未确证；调色走 grade_apply 出片（见 node_kinds 与 references/chain-api.md）" + "★ 全集 199 个 op 我们这道桥【全部放行】（0 个 unsupported）⇒ 以前\"只有 27 个\"是我们只试了 27 个", + "★ accepted:true ≠ 生效：判据必须是\"回读草稿后字段真的变了\"（segment.setTimeRange 传 startUs/durationUs ⇒ 官方回 accepted 但时码被清零）", + "★ text.add 的 needInsertTargetTrack=true 时段会插进 trackIndex 指向的那条轨（哪怕是 video 轨）⇒ 往同一条 text 轨塞多条字幕要用 needInsertTargetTrack=false + 该 text 轨下标" ] }, "draft_commit": { @@ -114,7 +288,7 @@ "video": "draft_patch#video.add", "image": "draft_patch", "text": "draft_patch", - "subtitle": "draft_patch", + "subtitle": "draft_patch#text.add{content,startTime,duration}", "sound": "audio_mix", "sticker": "draft_patch" }, @@ -154,6 +328,8 @@ "acceptance": { "grade_apply": "每个节点单开必须产生非零改变（MAD>0）", "verify_diff": "画面维度：max MAD > max(threshold, noiseMad×1.5)，且必须给 noiseDir（同态重渲染的帧）做噪声校准；官方渲染器同态噪声底实测 MAD 48~123 ⇒ 未校准的裸 MAD 只能作参考。语义维度：用 expect 断言落盘草稿的逐段字段差（不依赖像素）", - "draft_commit": "真草稿 sha 前后逐字节相同" + "draft_commit": "真草稿 sha 前后逐字节相同", + "draft_patch": "回读草稿后目标字段真的变了（不接受\"官方说 accepted\"）", + "text.add": "字幕时码与期望吻合（容差 ±1 帧，官方按 fps 帧对齐）" } -} \ No newline at end of file +} diff --git a/plugins/antianqi/jianying/references/official-ops.md b/plugins/antianqi/jianying/references/official-ops.md new file mode 100644 index 00000000..3d4f693d --- /dev/null +++ b/plugins/antianqi/jianying/references/official-ops.md @@ -0,0 +1,170 @@ +# 官方行为操作全集（实测 2026-10-21） + +> **★ 这份清单推翻了旧结论"官方只有 27 个行为操作"。** +> 27 那个数字来自官方 `app-server.mjs` 里的 `gk0` 常量（一个历史快照），**不是当前宿主进程的真实授权范围**。 +> 实测：官方全集 **18 个域 / 199 个 op**，我们这道桥 **全部放行（0 个被拒）**。 + +## 怎么测的 + +直接打官方管道（绕过我们插件的本地 schema），每个 op 用空参数打一次，按错误码分三类： + +| 判定 | 含义 | +|---|---| +| `accepted:true` | 放行，且空参数就能过（173 个） | +| `xxx_required` / `xxx_invalid` | **放行**，只是缺参数（26 个）⇒ 说明这个 op 存在 | +| `unsupported_behavior_operation` | 不放行（**0 个**） | + +⇒ 结论：**桥没有拦任何官方 op**。以前"只能用 27 个"是我们只试了 27 个，不是只能用 27 个。 + +## 分域清单 + +| 域 | 数量 | 干什么 | +|---|---|---| +| `draft` | 14 | 草稿级（画布/封面/名称/背景/对齐） | +| `track` | 4 | 轨道（增删静音/改名） | +| `segment` | 19 | 片段（增删改/切分/布局/可见/镜像/复制/时码） | +| `video` | 56 | 视频素材与特效（转场/淡入淡出/抠像/绿幕/变速/音量/数字人…） | +| `audio` | 24 | 音频（音量/变声/降噪/音高/人声分离/美颜…） | +| `text` | 16 | 文字与字幕（加文字/加字幕/样式/关键词/动画/追踪器） | +| `textTemplate` | 8 | 文字模板 | +| `sticker` | 7 | 贴纸 | +| `effect` | 3 | 特效（加/改参数/重置） | +| `filter` | 2 | 滤镜（加/改） | +| `adjustment` | 3 | 调色调整（加/基础色/重置） | +| `keyframe` | 3 | 关键帧（增删改） | +| `template` | 4 | 模板（可变视频/音频/文字/静音） | +| `templateCombination` | 3 | 模板组合 | +| `attachment` | 13 | 附件（AI 任务/视频算法/AI 创作） | +| `story` | 11 | 故事（AI 备选素材） | +| `undoRedo` | 6 | 撤销重做 | +| `batch` | 3 | 批处理（开始/结束/冻结） | + +## 全量 op 清单 + +### `draft`（14） + +`draft.setCanvasSize` `draft.setCanvasBackground` `draft.setCover` `draft.clearCover` `draft.setDraftName` `draft.setDraftId` `draft.setDraftUpdateTime` `draft.ensureDraftCanvasSize` `draft.updateDraftAIGCWatermark` `draft.clearDraft` `draft.setDraft` `draft.setAlignMode` `draft.setDraftKeywordsConfigs` `draft.setVideoGeneratorType` + +### `track`（4） + +`track.add` `track.delete` `track.mute` `track.updateTrackName` + +### `segment`（19） + +`segment.move` `segment.scale` `segment.translate` `segment.rotate` `segment.updateLayout` `segment.split` `segment.insert` `segment.delete` `segment.mirror` `segment.setTimeRange` `segment.innerReplace` `segment.innerReplaceStack` `segment.paste` `segment.visible` `segment.mute` `segment.setName` `segment.duplicate` `segment.setDescription` `segment.multiSegmentsSetTimeRange` + +### `video`（56） + +`video.add` `video.setMaterial` `video.setMask` `video.clearMask` `video.setOpacity` `video.setBackgroundColor` `video.setBackgroundBlur` `video.setBackgroundImage` `video.clearBackground` `video.formatVideo` `video.setTransition` `video.updateTransition` `video.clearTransition` `video.setFade` `video.denoiseOn` `video.denoiseOff` `video.addAnimation` `video.updateAnimation` `video.deleteAnimation` `video.setSpeed` `video.pitchOn` `video.pitchOff` `video.setCurveSpeed` `video.seprateAudio` `video.setMatting` `video.clearMatting` `video.reverse` `video.crop` `video.muteOn` `video.muteOff` `video.setBasicColor` `video.resetBasicColor` `video.setVolume` `video.setFigure` `video.setMakeupRoot` `video.clearFigure` `video.setFigureFaceId` `video.setMix` `video.setChromaColor` `video.setChromaIntensity` `video.setChromaShadow` `video.setChromaEdgeSmooth` `video.setChromaSpill` `video.resetChroma` `video.applyAllChroma` `video.removeSmartCrop` `video.addSmartCrop` `video.setSmartColorAdjust` `video.setStable` `video.setVideoAlgorithm` `video.addDigitalHuman` `video.updateDigitalHuman` `video.updateDigitalHumanRendered` `video.addObjectLocked` `video.updateObjectLocked` `video.removeObjectLocked` + +### `audio`（24） + +`audio.add` `audio.setVolume` `audio.setFade` `audio.denoiseOn` `audio.denoiseOff` `audio.pitchOn` `audio.pitchOff` `audio.setVoiceEffect` `audio.setVoiceEffectParams` `audio.clearVoiceEffect` `audio.setVoiceEffectToAll` `audio.setVocalSeparation` `audio.updateVocalSeparation` `audio.removeVocalSeparation` `audio.setVocalBeautify` `audio.updateVocalBeautify` `audio.resetVocalBeautify` `audio.removeVocalBeautify` `audio.setSpeed` `audio.modifiedToneOn` `audio.modifiedToneOff` `audio.setMaterial` `audio.addBeat` `audio.removeBeat` + +### `text`（16） + +`text.add` `text.setMaterials` `text.addSubtitles` `text.setFixedSize` `text.addAnimation` `text.updateAnimation` `text.deleteAnimation` `text.addVideoTracker` `text.updateVideoTracker` `text.removeVideoTracker` `text.deleteTextToAudioIds` `text.updateSubtitleTemplate` `text.resetSubtitleTemplate` `text.updateEffect` `text.setSubtitleKeywords` `text.clearSubtitleKeywords` + +### `textTemplate`（8） + +`textTemplate.add` `textTemplate.setMaterial` `textTemplate.addAnimation` `textTemplate.updateAnimation` `textTemplate.deleteAnimation` `textTemplate.addVideoTracker` `textTemplate.updateVideoTracker` `textTemplate.removeVideoTracker` + +### `sticker`（7） + +`sticker.add` `sticker.addAnimation` `sticker.updateAnimation` `sticker.deleteAnimation` `sticker.addVideoTracker` `sticker.updateVideoTracker` `sticker.removeVideoTracker` + +### `effect`（3） + +`effect.add` `effect.updateAdjustParams` `effect.resetAdjustParams` + +### `filter`（2） + +`filter.add` `filter.update` + +### `adjustment`（3） + +`adjustment.add` `adjustment.setBasicColor` `adjustment.resetBasicColor` + +### `keyframe`（3） + +`keyframe.add` `keyframe.delete` `keyframe.update` + +### `template`（4） + +`template.setMutableVideos` `template.setMutableAudios` `template.setMutableTexts` `template.setTemplateMute` + +### `templateCombination`（3） + +`templateCombination.templateCombinationAdd` `templateCombination.replace` `templateCombination.setSize` + +### `attachment`（13） + +`attachment.insertAsyncTasks` `attachment.deleteAsyncTasks` `attachment.updateAsyncTasksStatus` `attachment.insertVideoAlgorithms` `attachment.deleteVideoAlgorithms` `attachment.updateVideoAlgorithm` `attachment.insertAiWriterInfos` `attachment.deleteAiWriterInfos` `attachment.updateAiWriterInfo` `attachment.setAiCreatorContent` `attachment.updateAiCreation` `attachment.updateAiReel` `attachment.updateAiReelAgent` + +### `story`（11） + +`story.deleteAiStoryAlternative` `story.addAiStoryAlternativeMaterials` `story.cloneAiStoryAlternativeMaterials` `story.deleteAiStoryAlternativeMaterials` `story.updateAiStoryAlternativeMaterial` `story.pinAiStoryAlternativeMaterial` `story.updateMoveFeedsToAlternative` `story.updateStoryId` `story.innerReplace` `story.innerReplaceStack` `story.findInnerReplaceIds` + +### `undoRedo`（6） + +`undoRedo.undo` `undoRedo.redo` `undoRedo.canUndo` `undoRedo.canRedo` `undoRedo.getTopItemId` `undoRedo.resetUndoRedo` + +### `batch`（3） + +`batch.batchStart` `batch.batchEnd` `batch.freezeVideo` + +## 已确证参数契约的 op + +> 只列**实测过参数**的。判据是**回读草稿后字段真的变了** —— 不是"官方说 accepted"。 + +| op | 参数 | 实测依据 | +|---|---|---| +| `text.add` | `{content, startTime, duration, trackIndex, needInsertTargetTrack}`（也接受 `segInfo.material.content` / `segInfo.timeRange.start|duration`） | 落成 text 轨的段，`target_timerange` = 期望值（按 {fps{'}'} 帧对齐，误差 ≤ 1 帧） | +| `video.add` | `{path, startTime?, sourceDuration?, ...}`（顶层 `path`） | 顶层 `path` ⇒ `video_source_unreadable`（认）；`videos:[{path}]` ⇒ `video_path_required`（不认） | +| `audio.add` | `{audioPath}` | `audio_source_unreadable` | +| `video.setVolume` | `{segmentId, volume}` | 草稿里 `segment.volume` 0.99 → 0.2 | +| `segment.split` | `{segmentId, splitTime}`（★ 不是 `splitTimeUs`） | 时间线 9033333 → 8033333 µs | +| `segment.delete` | `{segmentId}` | 段数 −1 | +| `text.setMaterials` | `{updates:[{segmentId, ...}]}` | 报 `text_updates_required` ⇒ 认 `updates` 数组 | +| `draft.*`（11 个） | 需先有 snapshot | 报 `draft_snapshot_required` | + +## ★ 三个坑（都是"accepted 但没生效"那一类） + +### 1. `accepted:true` ≠ 生效 +`segment.setTimeRange` 传 `startUs/durationUs` ⇒ 官方回 `accepted:true`，但回读时码变成 **0/0** +（它不认这个键名，把没识别的字段当 0 写进去了）。⇒ **判据必须是"回读草稿，字段真的变了"**。 + +### 2. `needInsertTargetTrack` + `trackIndex` 会把段塞进错的轨 +`text.add` 的 `needInsertTargetTrack=true` 时，段会插进 `trackIndex` 指向的那条轨 —— +**哪怕它是 video 轨**。⇒ 要往同一条 text 轨里放多条字幕，得： +第一条 `needInsertTargetTrack:true` 建轨 ⇒ 读回那条 text 轨的下标 ⇒ 之后 +`needInsertTargetTrack:false` + 该下标。 + +### 3. `text.addSubtitles`（官方专门的加字幕 op）参数还没探对 +已知骨架：`{type, trackIndex, segInfos:[{timeRange:{start,duration}, material, segmentId}]}`， +`type` 要**数值枚举**（`MetaTypeSubtitle=10` / `MetaTypeLyrics=11`）， +`material.words[]` 是逐词时间轴（`{text, startTime, endTime}`）。 +实测：各种形态都**只建了空 text 轨、没落段**。 +⇒ **但普通 `text.add` + `startTime/duration` 已经能落出带时码的文字（= 字幕轨）** ⇒ 走那条路。 + +## 关键入口速查 + +| 想做的事 | 走哪个 op | +|---|---| +| **加字幕** | `text.add {content, startTime, duration}`（落成 text 轨的段） | +| 官方原生字幕（带逐词时间轴/歌词） | `text.addSubtitles`（参数待探） | +| **调色写回草稿** | `effect.add` / `effect.updateAdjustParams` / `adjustment.add` / `adjustment.setBasicColor` / `filter.add` / `keyframe.*` / `video.setBasicColor` / `video.setSmartColorAdjust` | +| 调色 material 集合（草稿里现成就有） | `materials.hsl` · `materials.hsl_curves` · `materials.color_curves` · `materials.log_color_wheels` · `materials.primary_color_wheels` · `materials.material_colors` | +| 抠像 / 绿幕 | `video.setMatting` · `video.setChroma*` · `video.applyAllChroma` | +| 变声 / 人声分离 / 美颜 | `audio.setVoiceEffect` · `audio.setVocalSeparation` · `audio.setVocalBeautify` | +| 转场 / 淡入淡出 / 变速 | `video.setTransition` · `video.setFade` · `video.setSpeed` · `video.setCurveSpeed` | +| 模板里的可变文字 | `template.setMutableTexts`（官方自己报 `template_mutable_text_action_missing` ⇒ 未实现） | +| 数字人 | `video.setFigure` · `video.setDigitalHuman` · `video.setMakeupRoot` | + +## 复现 + +```bash +node tools/probe-op-whitelist-full.mjs # 199 个 op 逐个打桥，输出放行/缺参/被拒 +node tools/probe-text-timing-final.mjs # text.add 的 startTime/duration 实测 +node tools/probe-subtitle-final.mjs # text.addSubtitles 的参数形态 +``` diff --git a/plugins/antianqi/jianying/scripts/add-subtitles.mjs b/plugins/antianqi/jianying/scripts/add-subtitles.mjs new file mode 100644 index 00000000..a54309a6 --- /dev/null +++ b/plugins/antianqi/jianying/scripts/add-subtitles.mjs @@ -0,0 +1,164 @@ +#!/usr/bin/env node +/** + * 字幕轨落稿器：SRT / plan → 草稿里的"带时码的文字"（剪映里的字幕轨） + * + * ★ 实测契约（2026-10-21，桥的归一化函数 oC_ 明确支持）： + * text.add { content, startTime, duration, trackIndex, needInsertTargetTrack } + * · content ：字幕文本（也接受 segInfo.material.content） + * · startTime ：起始（微秒），缺省 0 + * · duration ：时长（微秒），缺省 3_000_000 + * · trackIndex：目标轨索引（缺省 0） + * · needInsertTargetTrack：true(缺省)=官方新建一条 text 轨；false=塞进 trackIndex 指定的既有轨 + * ★ 坑：needInsertTargetTrack=false 且 trackIndex 指向【非 text 轨】⇒ 段会被塞进那条 video 轨 + * ⇒ 所以第一条用 true（让官方建轨），读回那条 text 轨的【下标】，后面的都用 false + 那个下标 + * 实测：text.add {content:'第一句字幕', startTime:1000000, duration:2500000} + * ⇒ 落成 text 轨 1 段，target_timerange = {start:1000000, duration:2500000} ✓ + * + * 用法: + * node scripts/add-subtitles.mjs --srt --draft "<草稿名>" [--apply] [--style-size N] + * node scripts/add-subtitles.mjs --plan --draft "<草稿名>" [--apply] + * --apply 真的改草稿（默认只演练） + */ +import { spawn } from 'node:child_process'; +import fs from 'node:fs'; +import path from 'node:path'; +import { fileURLToPath } from 'node:url'; + +const HERE = path.dirname(fileURLToPath(import.meta.url)); +const argv = process.argv.slice(2); +const arg = (k, d) => { const i = argv.indexOf(k); return i >= 0 && argv[i + 1] ? argv[i + 1] : d; }; +const has = (k) => argv.includes(k); + +const SRT = arg('--srt', null); +const PLAN = arg('--plan', null); +const DRAFT = arg('--draft', null); +const APPLY = has('--apply'); +if ((!SRT && !PLAN) || !DRAFT) { + console.error('用法: node add-subtitles.mjs (--srt | --plan ) --draft "<草稿名>" [--apply]'); + process.exit(2); +} + +function call(verb, args) { + return new Promise((resolve) => { + const c = spawn(process.execPath, [path.join(HERE, 'run-verb.mjs'), verb, JSON.stringify(args)], + { stdio: ['ignore', 'pipe', 'pipe'] }); + let so = '', se = ''; + c.stdout.on('data', (d) => { so += d; }); + c.stderr.on('data', (d) => { se += d; }); + c.on('close', (code) => { + let j = null; try { j = JSON.parse(so); } catch { /* not JSON */ } + resolve({ code, json: j, text: so, stderr: se, failed: code !== 0 || j?.isError === true }); + }); + }); +} + +/* ---------- SRT 解析（与 talk-cut-plan 同一套口径） ---------- */ +const tsToMs = (s) => { + const m = /(\d+):(\d+):(\d+)[,.](\d+)/.exec(String(s).trim()); + if (!m) return null; + return (+m[1]) * 3600000 + (+m[2]) * 60000 + (+m[3]) * 1000 + Math.round(+m[4] / 10); +}; +function parseSrt(text) { + const cues = []; + const blocks = text.replace(/\r\n/g, '\n').split(/\n{2,}/); + for (const b of blocks) { + const lines = b.split('\n').map((l) => l.trim()).filter(Boolean); + if (!lines.length) continue; + const ti = lines.findIndex((l) => l.includes('-->')); + if (ti < 0) continue; + const [a, z] = lines[ti].split('-->'); + const startMs = tsToMs(a), endMs = tsToMs(z); + if (startMs === null || endMs === null) continue; + cues.push({ index: cues.length + 1, startMs, endMs, text: lines.slice(ti + 1).join(' ').trim() }); + } + return cues; +} + +/* ---------- 取字幕源 ---------- */ +let cues = []; +if (PLAN) { + const p = JSON.parse(fs.readFileSync(PLAN, 'utf8')); + cues = (p.cues || []).map((c) => ({ index: c.index, startMs: c.startMs, endMs: c.endMs, text: c.text })); +} else { + cues = parseSrt(fs.readFileSync(SRT, 'utf8')); +} +cues = cues.filter((c) => c.text && c.endMs > c.startMs); + +console.log('=== 字幕落稿 ==='); +console.log(` 字幕条数: ${cues.length}`); +console.log(` 时间跨度: ${(cues[0]?.startMs / 1000 || 0).toFixed(2)}s ~ ${((cues.at(-1)?.endMs || 0) / 1000).toFixed(2)}s`); +console.log(` 模式: ${APPLY ? '★ 真改草稿' : '演练（只打印）'}`); +console.log(` 草稿: ${DRAFT}\n`); +if (!cues.length) { console.log('没有可用的字幕条'); process.exit(0); } + +if (!APPLY) { + for (const c of cues) { + console.log(` #${c.index} ${(c.startMs / 1000).toFixed(2)}s +${((c.endMs - c.startMs) / 1000).toFixed(2)}s ${c.text}`); + } + console.log('\n（演练模式，未改草稿。加 --apply 真改）'); + process.exit(0); +} + +/* ---------- 真改 ---------- */ +const open = await call('draft_open', { draft: DRAFT, copy: true }); +if (open.failed) { console.error('打开草稿失败：\n' + (open.text + open.stderr).slice(0, 400)); process.exit(3); } +const sid = open.json.sessionId; +console.log(` 副本: ${open.json.dir}\n`); + +let textTrackIndex = -1; // 已存在的 text 轨下标（第一条落进后确定） +let ok = 0, fail = 0; + +for (const c of cues) { + // ★ 实测：needInsertTargetTrack=true 时，官方会把段插进 trackIndex 指向的那条轨 + // （哪怕它是 video 轨！）⇒ 所以 trackIndex 必须指向一条【text 轨】。 + // 没有 text 轨时（第一条）就让官方新建（needInsertTargetTrack=true + trackIndex=0 之外的 + // 行为由官方决定），落完后读回 text 轨下标，后续都往那条轨里塞 + // （needInsertTargetTrack=false + 该下标 ⇒ 实测两条同轨 ✓）。 + const params = textTrackIndex >= 0 + ? { content: c.text, startTime: c.startMs * 1000, duration: (c.endMs - c.startMs) * 1000, + trackIndex: textTrackIndex, needInsertTargetTrack: false } + : { content: c.text, startTime: c.startMs * 1000, duration: (c.endMs - c.startMs) * 1000, + trackIndex: 0, needInsertTargetTrack: true }; + const r = await call('draft_patch', { op: 'text.add', sessionId: sid, params }); + if (r.failed) { + fail++; + console.log(` [FAIL] #${c.index} ${(c.startMs / 1000).toFixed(2)}s "${c.text.slice(0, 24)}" → ${(r.text + r.stderr).replace(/\s+/g, ' ').slice(0, 120)}`); + continue; + } + ok++; + if (textTrackIndex < 0) { + const rr = await call('draft_read', { sessionId: sid }); + let j = null; try { j = JSON.parse(rr.text); } catch { /* ignore */ } + const tracks = j?.draft?.tracks || []; + const ti = tracks.findIndex((t) => t.type === 'text'); + if (ti >= 0) textTrackIndex = ti; + } + console.log(` [OK] #${c.index} ${(c.startMs / 1000).toFixed(2)}s +${((c.endMs - c.startMs) / 1000).toFixed(2)}s "${c.text.slice(0, 28)}"`); +} + +console.log(`\n 落稿: ${ok} 成功 / ${fail} 失败 text 轨下标 = ${textTrackIndex}`); + +/* ---------- 验收：回读并核对时码（语义判据，容差 = 1 帧） ---------- */ +const rr = await call('draft_read', { sessionId: sid }); +let j = null; try { j = JSON.parse(rr.text); } catch { /* ignore */ } +const draft = j?.draft; +const textSegs = []; +for (const t of draft?.tracks || []) if (t.type === 'text') for (const s of t.segments || []) textSegs.push(s); +const fps = draft?.fps || 30; +const frameUs = Math.round(1e6 / fps); // 官方按帧对齐 ⇒ 允许 ±1 帧 +const mism = []; +for (const c of cues) { + const wantStart = c.startMs * 1000, wantDur = (c.endMs - c.startMs) * 1000; + const hit = textSegs.find((s) => Math.abs((s.target_timerange?.start ?? -1) - wantStart) <= frameUs + && Math.abs((s.target_timerange?.duration ?? -1) - wantDur) <= frameUs); + if (!hit) mism.push(c.index); +} +console.log(` 验收: 草稿里 text 段 ${textSegs.length} 条 · 时码吻合 ${cues.length - mism.length}/${cues.length}（容差 ±1 帧 = ${frameUs}µs，fps=${fps}）`); +if (mism.length) console.log(` ★ 时码不吻合的字幕条: ${mism.join(', ')}`); + +/* ---------- 落盘 ---------- */ +const commit = await call('draft_commit', { sessionId: sid, realDraft: DRAFT }); +if (commit.failed) { console.error('落盘失败：\n' + (commit.text + commit.stderr).slice(0, 300)); process.exit(4); } +console.log(` # 落盘: ${(commit.text + commit.stderr).replace(/\s+/g, ' ').slice(0, 200)}`); + +process.exit(ok === cues.length && mism.length === 0 ? 0 : 1); diff --git a/plugins/antianqi/jianying/scripts/selftest-subtitle.mjs b/plugins/antianqi/jianying/scripts/selftest-subtitle.mjs new file mode 100644 index 00000000..07008b2a --- /dev/null +++ b/plugins/antianqi/jianying/scripts/selftest-subtitle.mjs @@ -0,0 +1,97 @@ +#!/usr/bin/env node +/** + * jianying 插件：字幕落稿自检 + * 造一份 SRT → add-subtitles.mjs 落进副本草稿 → 回读核对 + * ★ 判据是【语义】：回读草稿里的 text 段，时码必须与 SRT 吻合（容差 ±1 帧，官方按 fps 帧对齐） + * ★ 负向注入：同目录的"无字幕"草稿必须判为"没有字幕"（不能把探针写成恒真） + * + * 用法: node scripts/selftest-subtitle.mjs + * 需要: JYAI_HOME 指向 JYAI 仓库；剪映 11.6 在运行；JYAI_TEST_DRAFT 指向一个带视频片段的草稿 + */ +import { spawn } from 'node:child_process'; +import fs from 'node:fs'; +import os from 'node:os'; +import path from 'node:path'; +import { fileURLToPath } from 'node:url'; + +const HERE = path.dirname(fileURLToPath(import.meta.url)); +const JYAI = process.env.JYAI_HOME || path.join(os.homedir(), 'JYAI'); +const DRAFT = process.env.JYAI_TEST_DRAFT || ''; +const WORK = path.join(JYAI, 'out', 'p67', 'subtitle-selftest'); + +let PASS = 0, FAIL = 0; +const check = (n, ok, d) => { + if (ok) { PASS++; console.log(` [PASS] ${n}${d ? ' — ' + d : ''}`); } + else { FAIL++; console.log(` [FAIL] ${n}${d ? ' — ' + d : ''}`); } +}; +function call(verb, args) { + return new Promise((resolve) => { + const c = spawn(process.execPath, [path.join(HERE, 'run-verb.mjs'), verb, JSON.stringify(args)], + { env: { ...process.env, JYAI_HOME: JYAI }, stdio: ['ignore', 'pipe', 'pipe'] }); + let so = '', se = ''; + c.stdout.on('data', (d) => { so += d; }); + c.stderr.on('data', (d) => { se += d; }); + c.on('close', (code) => { + let j = null; try { j = JSON.parse(so); } catch { /* not JSON */ } + resolve({ code, json: j, text: so, stderr: se, failed: code !== 0 || j?.isError === true }); + }); + }); +} +const run = (script, args) => new Promise((resolve) => { + const c = spawn(process.execPath, [path.join(HERE, script), ...args], { env: { ...process.env, JYAI_HOME: JYAI }, stdio: ['ignore', 'pipe', 'pipe'] }); + let so = '', se = ''; + c.stdout.on('data', (d) => { so += d; }); + c.stderr.on('data', (d) => { se += d; }); + c.on('close', (code) => resolve({ code, out: so + se })); +}); + +console.log('=== 字幕落稿自检 ===\n'); +if (!DRAFT) { + console.log(' [SKIP] 需要 JYAI_TEST_DRAFT —— 指向一个带视频片段的剪映草稿目录。'); + console.log(' 例（PowerShell）: $env:JYAI_TEST_DRAFT = "<草稿库>\\<草稿名>"'); + process.exit(0); +} +fs.mkdirSync(WORK, { recursive: true }); + +const CUES = [ + { text: '大家好，今天我们来聊一个被很多人忽略的问题', startMs: 500, endMs: 3000 }, + { text: '就是怎么让 AI 直接操作剪映', startMs: 3200, endMs: 6500 }, + { text: '嗯，这个问题其实挺有意思的', startMs: 6800, endMs: 9500 }, +]; +const srtPath = path.join(WORK, 'selftest.srt'); +const srt = CUES.map((c, i) => { + const ts = (ms) => { + const h = String(Math.floor(ms / 3600000)).padStart(2, '0'); + const m = String(Math.floor((ms % 3600000) / 60000)).padStart(2, '0'); + const s = String(Math.floor((ms % 60000) / 1000)).padStart(2, '0'); + return `${h}:${m}:${s},${String(ms % 1000).padStart(3, '0')}`; + }; + return `${i + 1}\n${ts(c.startMs)} --> ${ts(c.endMs)}\n${c.text}\n`; +}).join('\n'); +fs.writeFileSync(srtPath, srt, 'utf8'); +check('① 造 SRT', fs.existsSync(srtPath) && CUES.length === 3, srtPath); + +// ② 落稿 +const r = await run('add-subtitles.mjs', ['--srt', srtPath, '--draft', DRAFT, '--apply']); +const okMatch = /落稿: (\d+) 成功 \/ (\d+) 失败/.exec(r.out); +const landed = okMatch ? +okMatch[1] : 0; +const failedN = okMatch ? +okMatch[2] : -1; +check('② add-subtitles 落稿', landed === CUES.length && failedN === 0, r.out.replace(/\s+/g, ' ').match(/落稿:.*$/)?.[0] || r.out.slice(0, 160)); + +// ③ 验收行：时码吻合数 +const accMatch = /时码吻合 (\d+)\/(\d+)/.exec(r.out); +const matched = accMatch ? +accMatch[1] : 0; +check('③ 时码与 SRT 吻合（语义判据，±1 帧）', matched === CUES.length, accMatch ? `${matched}/${CUES.length}` : '验收行缺失'); + +// ④ 阴性：没跑之前草稿里没有这些字幕 ⇒ 探针必须能区分"有字幕"和"没字幕" +const open = await call('draft_open', { draft: DRAFT, copy: true }); +let before = null; +try { before = JSON.parse(open.text).draft; } catch { /* ignore */ } +const textSegsBefore = []; +for (const t of before?.tracks || []) if (t.type === 'text') for (const s of t.segments || []) textSegsBefore.push(s); +const beforeHas = textSegsBefore.some((s) => CUES.some((c) => s.target_timerange?.start === c.startMs * 1000)); +check('★ ④ 阴性：原始草稿里没有这些字幕（探针有判别力）', !beforeHas, + `原始草稿 text 段 ${textSegsBefore.length} 条 · 含目标时码 = ${beforeHas}`); + +console.log(`\n=== ${PASS} PASS / ${FAIL} FAIL ===`); +process.exit(FAIL === 0 ? 0 : 1); diff --git a/plugins/antianqi/jianying/skills/jianying-draft-edit/SKILL.md b/plugins/antianqi/jianying/skills/jianying-draft-edit/SKILL.md index d3b3fc89..cfecd01f 100644 --- a/plugins/antianqi/jianying/skills/jianying-draft-edit/SKILL.md +++ b/plugins/antianqi/jianying/skills/jianying-draft-edit/SKILL.md @@ -2,13 +2,13 @@ name: jianying-draft-edit description: 'Use when: 改剪映草稿 / 给草稿加素材或音轨 / 移动裁剪片段 / 调某段的音量 / 加文字 / 加轨道 / 撤销重做 / 改完落盘并验证生效. 触发词: 给这个草稿加个片段 / 把这段剪短 / 把这轨静音 / 加个文字 / 改完存一下 / 撤销刚才那步 / 看看改动生效没有.' displayNames: - zh-Hans: 剪映·改草稿（27 个官方操作 · 落盘不动真草稿） + zh-Hans: 剪映·改草稿（官方行为操作全集 · 加字幕 · 落盘不动真草稿） license: Apache-2.0 compatibility: Requires Windows + 剪映 11.6 在运行（edit 域走官方命名管道）+ JYAI_HOME 指向 JYAI 仓库（落盘/渲染/验收在 JYAI_HOME/tools）+ ffmpeg + Python 3.14. metadata: author: antianqi version: "0.1.0" - basis: "官方 edit 域 27 个带命名空间的行为操作（实测 27/27）；落盘走 draft/writeFile；验收走【语义】逐段字段差（像素差需先量噪声底）" + basis: "官方 edit 域带命名空间的行为操作全集（实测 18 域 199 个、桥全部放行，全量见 references/official-ops.md）；落盘走 draft/writeFile；验收走【语义】逐段字段差（像素差需先量噪声底）" --- # 剪映改草稿（jianying-draft-edit） @@ -22,12 +22,15 @@ metadata: |---|---| | `draft_open` | 打开草稿（**默认开副本**；`allowReal` 与 `copy` 冲突会被拒） | | `draft_read` | 读草稿语义结构（时间线/片段/素材）—— 拿 `segmentId` 等 id 用 | -| `draft_patch` | **27 个官方行为操作**（见下表） | +| `draft_patch` | **官方行为操作全集**（18 域 / 199 个，桥**全部放行**；常用见下表，全量见 `references/official-ops.md`） | | `draft_commit` | 落盘（自动**素材路径重定位** + **真草稿 sha 前后断言**） | | `render_frames` | 用官方渲染器出帧（改前/改后各一次） | | `verify_diff` | 逐帧像素差（**取全帧最大**）判定"看得见" | -## ★ 27 个官方操作（带命名空间，**不是裸名**） +## ★ 官方行为操作（带命名空间，**不是裸名**） + +> **★ 全集是 18 域 / 199 个 op，我们这道桥实测全部放行（0 个被拒）** —— 以前"只有 27 个"是我们只试了 27 个。 +> 全量清单 + 已知参数契约 + 三个坑：`references/official-ops.md`。 | 组 | 操作 | |---|---| @@ -41,8 +44,8 @@ metadata: | 批量 | `invokeBatch` | **★ 已知必填项**（缺了会被本地先拦下，不白跑一次官方调用）： -`video.add`→`path` · **`audio.add`→`audioPath`** · `text.add`→`content` · `track.mute`/`track.delete`→`trackId` · -`segment.move`/`setTimeRange`/`setDescription`→`segmentId` · **`segment.split`→`splitTime`** · +`video.add`→`path` · **`audio.add`→`audioPath`** · **`text.add`→`content`+`startTime`/`duration`** · `track.mute`/`track.delete`→`trackId` · +`segment.move`/`setDescription`→`segmentId` · **`segment.split`→`splitTime`** · `segment.updateLayout`→`layout` · `segment.innerReplace`→`replacement` ## 主工作流 @@ -89,12 +92,13 @@ node ${PLUGIN_ROOT}/scripts/run-verb.mjs verify_diff '{ | 加一条视频 | `video.add` `{path}`（★ 参数名是 `path`，不是 `source`/`filePath`） | | 加一条音频 | `audio.add` `{audioPath}`（★ 不是 `source`） | | 加一段文字 | `text.add` `{content}`（★ 不是 `text`） | +| **加字幕（带时码）** | `text.add` `{content, startTime, duration}`（★ 时码直接吃这两个键，官方按 fps 帧对齐） | | 这段音量调到 25% | `video.setVolume` `{segmentId, volume}` | | 这段静音 | `video.muteOn` `{segmentId}` | | 把这段剪成两半 | `segment.split` `{segmentId, splitTime}`（★ 不是 `splitTimeUs`） | | 删掉这段 | `segment.delete` `{segmentId}` | | 挪到别的位置 | `segment.move` `{segmentId, ...}`（★ 需要先有 snapshot） | -| 改这段的时间范围 | `segment.setTimeRange` `{segmentId, startTimeUs?\|startUs?, durationUs?}` | +| 改这段的时间范围 | `segment.setTimeRange` `{segmentId, …}` ⚠️ **参数名尚未确证**：传 `startUs/durationUs` 官方回 `accepted` 但回读时码被清零 ⇒ 判据必须是"回读后字段真的变了" | | 撤销上一步 | `undoRedo.undo` `{}` | | 一次做多步 | `invokeBatch` `{operations:[...]}` | @@ -127,6 +131,6 @@ node ${PLUGIN_ROOT}/scripts/run-verb.mjs verify_diff '{ ## 参考 -- 官方 27 op 的实测报告：`\logs\editops.md`（含每个 op 的官方报错 ⇒ 参数反推依据） +- 官方 op 全集的实测清单：`references/official-ops.md`（18 域 199 个、逐个打桥的结果、已确证参数契约、三个"accepted 但没生效"的坑） - 判据纪律：`references/verify-discipline.md` - 踩坑清单：`references/pitfalls.md` diff --git a/plugins/antianqi/jianying/skills/jianying-talk-cut/SKILL.md b/plugins/antianqi/jianying/skills/jianying-talk-cut/SKILL.md index e5951df4..28f634d5 100644 --- a/plugins/antianqi/jianying/skills/jianying-talk-cut/SKILL.md +++ b/plugins/antianqi/jianying/skills/jianying-talk-cut/SKILL.md @@ -95,7 +95,8 @@ node ${PLUGIN_ROOT}/scripts/run-verb.mjs verify_diff '{ | **自己做 ASR** | 官方 ASR 在 agent runtime（HTTP 63178），**不在我们的 VE 管道里**；本地 whisper 要装模型（可选，不在默认路径） | **用户给 SRT / 转写稿**（我们缺的是"事实来源"，不是"经验"） | | **改写台词 / 生成旁白** | 官方也明确「自由改写和生成旁白走通用链路」 | 人工改写；或用 `jianying-draft-edit` 的 `text.add` 另加一段 | | **按主题重排**（官方 topic_cut） | 需要素材召回（`SegmentRecall`）⇒ 拿不到 | 我们只做 **Clean Cut**（按 SRT 删废话） | -| **自动加字幕** | 字幕规范/样式是另一条链 | `jianying-draft-edit` 的 `text.add` 可以做，但不是本 skill 的事 | +| **自动生成字幕文本**（ASR） | 官方 ASR 在 agent runtime，拿不到 | **用户给 SRT**；本 skill 已有 SRT 解析与逐句时间轴 | +| **落字幕（把 SRT 变成草稿里的字幕轨）** | 不属于"判断哪句是废话"的范畴 | **已能做**：`scripts/add-subtitles.mjs`（`text.add` + `startTime`/`duration` ⇒ 带时码的文字段；★ 时码按 fps 帧对齐，验收容差 ±1 帧） | | **不加人工确认就落刀** | 判定只到 `review` 级的水词/停顿/重复**默认不删** | `talk-cut-plan` 出保留区间表 ⇒ 人工确认 ⇒ `talk-cut-apply` 才落刀 | ## 参考 From 09a43280854fa9cbd697f9bcbebcdda7dc8996db Mon Sep 17 00:00:00 2001 From: antianqi Date: Wed, 21 Oct 2026 18:49:06 +0800 Subject: [PATCH 06/15] feat(jianying): wire the full official op set (199) and land the audio AI effects MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two things land here. The operation set is now the whole thing. The v0.3 schema carried a 27-entry enum, which came from a historical whitelist constant in the application rather than from the bridge's actual authorisation. Sweeping the full set against the bridge shows 18 domains / 199 operations and **not one of them is rejected** (173 accepted with empty parameters, 26 rejected only for missing parameters, 0 unsupported). The enum now carries all 199, the capability dictionary lists them, and references/official-ops.md records the sweep together with the parameter contracts that were confirmed against the draft rather than against "the application said accepted". The audio AI effects work, and they differ from each other in an important way: - Denoise is `video.denoiseOn`, not `audio.denoiseOn`. Parameters {segmentId, isDenoise?, denoiseMode?, denoiseRate?} land in materials.realtime_denoises as {is_denoise, denoise_mode, denoise_rate} alongside sami_name "denoise_v2" and sami_version "1.0" — a local SAMI realtime engine, so the denoise is computed on the machine. - Vocal beautify is `audio.setVocalBeautify` {segmentId, enable:true}, landing in materials.vocal_beautifys as {enable:true, type:"vocal_beautify", ambient_sound_level:75, voice_change_mode:null, production_path:""}. An empty production_path means realtime rather than a cloud product, so this one is computed locally too. - Voice change is `audio.setVoiceEffect` {segmentId}, landing in materials.audio_effects. - Vocal separation is different: `audio.setVocalSeparation` builds materials.vocal_separations with {type:"vocal_separation", choice, removed_sounds, time_range, final_algorithm:"", production_path:""}. The empty production_path is the point — the separated audio is a task result, so the reference can be written but the artefact is not ours to produce. Six parameter shapes were tried and none changed the draft, so the contract is still unmapped. scripts/selftest-audio-ai.mjs drives all of this through the Plugin's own draft_patch verb rather than the raw pipe, and checks the draft afterwards: 8/8, including a negative case that requires the three material collections to be empty before the change, so the probe cannot pass vacuously. Also confirmed here: the key names come in both styles (isDenoise and is_denoise, denoiseMode and denoise_mode are all accepted), while `enable` is only `enable` — `isEnable` is ignored. Validation - selftest-mcp 17/17, selftest-draft-edit 16/16, selftest-talk-cut 10/10, selftest-subtitle 4/4, selftest-audio-ai 8/8, check-env 17 OK / 0 MISS. - node --test test/jianying.test.mjs 10/10. - node scripts/validate.mjs -> OK plugin antianqi/jianying; 30 hosted Plugins validated. --- plugins/antianqi/jianying/mcp/server.mjs | 10 + .../mcp/tools/draft-patch.schema.json | 215 ++++++++++++++++-- .../jianying/references/capability-dict.json | 10 +- .../jianying/references/official-ops.md | 12 + .../jianying/scripts/selftest-audio-ai.mjs | 100 ++++++++ 5 files changed, 324 insertions(+), 23 deletions(-) create mode 100644 plugins/antianqi/jianying/scripts/selftest-audio-ai.mjs diff --git a/plugins/antianqi/jianying/mcp/server.mjs b/plugins/antianqi/jianying/mcp/server.mjs index 57aab526..2706bf17 100644 --- a/plugins/antianqi/jianying/mcp/server.mjs +++ b/plugins/antianqi/jianying/mcp/server.mjs @@ -169,6 +169,16 @@ const OP_REQUIRED = { 'track.delete': ['trackId'], 'segment.move': ['segmentId'], 'segment.setTimeRange': ['segmentId'], + // ★ 2026-10-21 实测确认（视频/音频 AI 效果类） + 'video.denoiseOn': ['segmentId'], + 'video.denoiseOff': ['segmentId'], + 'audio.setVocalBeautify': ['segmentId'], + 'audio.updateVocalBeautify': ['segmentId'], + 'audio.removeVocalBeautify': ['segmentId'], + 'audio.setVoiceEffect': ['segmentId'], + 'audio.setVoiceEffectParams': ['segmentId'], + 'audio.clearVoiceEffect': ['segmentId'], + 'audio.setVocalSeparation': [], 'segment.setDescription': ['segmentId'], 'segment.updateLayout': ['layout'], 'segment.innerReplace': ['replacement'], diff --git a/plugins/antianqi/jianying/mcp/tools/draft-patch.schema.json b/plugins/antianqi/jianying/mcp/tools/draft-patch.schema.json index 0d1ee0df..ffc8fd49 100644 --- a/plugins/antianqi/jianying/mcp/tools/draft-patch.schema.json +++ b/plugins/antianqi/jianying/mcp/tools/draft-patch.schema.json @@ -7,35 +7,207 @@ "op": { "type": "string", "enum": [ + "adjustment.add", + "adjustment.resetBasicColor", + "adjustment.setBasicColor", + "attachment.deleteAiWriterInfos", + "attachment.deleteAsyncTasks", + "attachment.deleteVideoAlgorithms", + "attachment.insertAiWriterInfos", + "attachment.insertAsyncTasks", + "attachment.insertVideoAlgorithms", + "attachment.setAiCreatorContent", + "attachment.updateAiCreation", + "attachment.updateAiReel", + "attachment.updateAiReelAgent", + "attachment.updateAiWriterInfo", + "attachment.updateAsyncTasksStatus", + "attachment.updateVideoAlgorithm", + "audio.add", + "audio.addBeat", + "audio.clearVoiceEffect", + "audio.denoiseOff", + "audio.denoiseOn", + "audio.modifiedToneOff", + "audio.modifiedToneOn", + "audio.pitchOff", + "audio.pitchOn", + "audio.removeBeat", + "audio.removeVocalBeautify", + "audio.removeVocalSeparation", + "audio.resetVocalBeautify", + "audio.setFade", + "audio.setMaterial", + "audio.setSpeed", + "audio.setVocalBeautify", + "audio.setVocalSeparation", + "audio.setVoiceEffect", + "audio.setVoiceEffectParams", + "audio.setVoiceEffectToAll", + "audio.setVolume", + "audio.updateVocalBeautify", + "audio.updateVocalSeparation", + "batch.batchEnd", + "batch.batchStart", + "batch.freezeVideo", + "draft.clearCover", + "draft.clearDraft", + "draft.ensureDraftCanvasSize", + "draft.setAlignMode", + "draft.setCanvasBackground", "draft.setCanvasSize", + "draft.setCover", + "draft.setDraft", "draft.setDraftId", - "track.add", - "track.delete", - "track.mute", + "draft.setDraftKeywordsConfigs", + "draft.setDraftName", + "draft.setDraftUpdateTime", + "draft.setVideoGeneratorType", + "draft.updateDraftAIGCWatermark", + "effect.add", + "effect.resetAdjustParams", + "effect.updateAdjustParams", + "filter.add", + "filter.update", + "keyframe.add", + "keyframe.delete", + "keyframe.update", + "segment.delete", + "segment.duplicate", + "segment.innerReplace", + "segment.innerReplaceStack", + "segment.insert", + "segment.mirror", "segment.move", + "segment.multiSegmentsSetTimeRange", + "segment.mute", + "segment.paste", + "segment.rotate", "segment.scale", + "segment.setDescription", + "segment.setName", + "segment.setTimeRange", + "segment.split", "segment.translate", - "segment.rotate", "segment.updateLayout", - "segment.split", - "segment.delete", - "segment.setTimeRange", - "segment.innerReplace", - "segment.setDescription", + "segment.visible", + "sticker.add", + "sticker.addAnimation", + "sticker.addVideoTracker", + "sticker.deleteAnimation", + "sticker.removeVideoTracker", + "sticker.updateAnimation", + "sticker.updateVideoTracker", + "story.addAiStoryAlternativeMaterials", + "story.cloneAiStoryAlternativeMaterials", + "story.deleteAiStoryAlternative", + "story.deleteAiStoryAlternativeMaterials", + "story.findInnerReplaceIds", + "story.innerReplace", + "story.innerReplaceStack", + "story.pinAiStoryAlternativeMaterial", + "story.updateAiStoryAlternativeMaterial", + "story.updateMoveFeedsToAlternative", + "story.updateStoryId", + "template.setMutableAudios", + "template.setMutableTexts", + "template.setMutableVideos", + "template.setTemplateMute", + "templateCombination.replace", + "templateCombination.setSize", + "templateCombination.templateCombinationAdd", + "text.add", + "text.addAnimation", + "text.addSubtitles", + "text.addVideoTracker", + "text.clearSubtitleKeywords", + "text.deleteAnimation", + "text.deleteTextToAudioIds", + "text.removeVideoTracker", + "text.resetSubtitleTemplate", + "text.setFixedSize", + "text.setMaterials", + "text.setSubtitleKeywords", + "text.updateAnimation", + "text.updateEffect", + "text.updateSubtitleTemplate", + "text.updateVideoTracker", + "textTemplate.add", + "textTemplate.addAnimation", + "textTemplate.addVideoTracker", + "textTemplate.deleteAnimation", + "textTemplate.removeVideoTracker", + "textTemplate.setMaterial", + "textTemplate.updateAnimation", + "textTemplate.updateVideoTracker", + "track.add", + "track.delete", + "track.mute", + "track.updateTrackName", + "undoRedo.canRedo", + "undoRedo.canUndo", + "undoRedo.getTopItemId", + "undoRedo.redo", + "undoRedo.resetUndoRedo", + "undoRedo.undo", "video.add", - "video.muteOn", + "video.addAnimation", + "video.addDigitalHuman", + "video.addObjectLocked", + "video.addSmartCrop", + "video.applyAllChroma", + "video.clearBackground", + "video.clearFigure", + "video.clearMask", + "video.clearMatting", + "video.clearTransition", + "video.crop", + "video.deleteAnimation", + "video.denoiseOff", + "video.denoiseOn", + "video.formatVideo", "video.muteOff", + "video.muteOn", + "video.pitchOff", + "video.pitchOn", + "video.removeObjectLocked", + "video.removeSmartCrop", + "video.resetBasicColor", + "video.resetChroma", + "video.reverse", + "video.seprateAudio", + "video.setBackgroundBlur", + "video.setBackgroundColor", + "video.setBackgroundImage", + "video.setBasicColor", + "video.setChromaColor", + "video.setChromaEdgeSmooth", + "video.setChromaIntensity", + "video.setChromaShadow", + "video.setChromaSpill", + "video.setCurveSpeed", + "video.setFade", + "video.setFigure", + "video.setFigureFaceId", + "video.setMakeupRoot", + "video.setMask", + "video.setMaterial", + "video.setMatting", + "video.setMix", + "video.setOpacity", + "video.setSmartColorAdjust", + "video.setSpeed", + "video.setStable", + "video.setTransition", + "video.setVideoAlgorithm", "video.setVolume", - "audio.add", - "audio.setVolume", - "text.add", - "undoRedo.undo", - "undoRedo.redo", - "undoRedo.canUndo", - "undoRedo.canRedo", - "invokeBatch" + "video.updateAnimation", + "video.updateDigitalHuman", + "video.updateDigitalHumanRendered", + "video.updateObjectLocked", + "video.updateTransition" ], - "description": "官方行为操作名（带命名空间，不是裸名）" + "description": "官方行为操作名（带命名空间，不是裸名）。全集 18 域 / 199 个，桥实测全部放行；全量清单与已知参数契约见 references/official-ops.md。" }, "sessionId": { "type": "string", @@ -46,7 +218,10 @@ "description": "该 op 的参数（官方原生语义）。已知必填：video.add→path · text.add→content · track.mute/delete→trackId · segment.move/setTimeRange/setDescription→segmentId · segment.updateLayout→layout · segment.innerReplace→replacement。缺必填会先被本地拦下，不会白跑一次官方调用。" } }, - "required": ["op", "sessionId"], + "required": [ + "op", + "sessionId" + ], "additionalProperties": false } } diff --git a/plugins/antianqi/jianying/references/capability-dict.json b/plugins/antianqi/jianying/references/capability-dict.json index 2fe494b9..9cc96786 100644 --- a/plugins/antianqi/jianying/references/capability-dict.json +++ b/plugins/antianqi/jianying/references/capability-dict.json @@ -227,7 +227,9 @@ "改前必须先 draft_open", "★ 全集 199 个 op 我们这道桥【全部放行】（0 个 unsupported）⇒ 以前\"只有 27 个\"是我们只试了 27 个", "★ accepted:true ≠ 生效：判据必须是\"回读草稿后字段真的变了\"（segment.setTimeRange 传 startUs/durationUs ⇒ 官方回 accepted 但时码被清零）", - "★ text.add 的 needInsertTargetTrack=true 时段会插进 trackIndex 指向的那条轨（哪怕是 video 轨）⇒ 往同一条 text 轨塞多条字幕要用 needInsertTargetTrack=false + 该 text 轨下标" + "★ text.add 的 needInsertTargetTrack=true 时段会插进 trackIndex 指向的那条轨（哪怕是 video 轨）⇒ 往同一条 text 轨塞多条字幕要用 needInsertTargetTrack=false + 该 text 轨下标", + "★ 视频/音频 AI 效果类 op 的契约（实测 2026-10-21）：video.denoiseOn{segmentId,isDenoise?,denoiseMode?,denoiseRate?} ⇒ materials.realtime_denoises（引擎 sami_name=denoise_v2，本地实时 DSP）；audio.setVocalBeautify{segmentId,enable:true} ⇒ materials.vocal_beautifys（本地实时）；audio.setVocalSeparation{…} ⇒ materials.vocal_separations（production_path 为空 ⇒ 产出要剪映的 AI 任务，我们只写引用）", + "★ 视频/音频 AI 效果类 op 的契约（实测 2026-10-21）：video.denoiseOn{segmentId,isDenoise?,denoiseMode?,denoiseRate?} ⇒ materials.realtime_denoises（引擎 sami_name=denoise_v2，本地实时 DSP）；audio.setVocalBeautify{segmentId,enable:true} ⇒ materials.vocal_beautifys（本地实时）；audio.setVocalSeparation{…} ⇒ materials.vocal_separations（production_path 为空 ⇒ 产出要剪映的 AI 任务，我们只写引用）" ] }, "draft_commit": { @@ -302,7 +304,8 @@ "mask": "draft_patch", "matting": "draft_patch", "animation": "draft_patch", - "speed_curve": "draft_patch" + "speed_curve": "draft_patch", + "audio_ai": "draft_patch#video.denoiseOn|audio.setVocalBeautify|audio.setVoiceEffect" } }, "node_kinds": { @@ -330,6 +333,7 @@ "verify_diff": "画面维度：max MAD > max(threshold, noiseMad×1.5)，且必须给 noiseDir（同态重渲染的帧）做噪声校准；官方渲染器同态噪声底实测 MAD 48~123 ⇒ 未校准的裸 MAD 只能作参考。语义维度：用 expect 断言落盘草稿的逐段字段差（不依赖像素）", "draft_commit": "真草稿 sha 前后逐字节相同", "draft_patch": "回读草稿后目标字段真的变了（不接受\"官方说 accepted\"）", - "text.add": "字幕时码与期望吻合（容差 ±1 帧，官方按 fps 帧对齐）" + "text.add": "字幕时码与期望吻合（容差 ±1 帧，官方按 fps 帧对齐）", + "audio_ai": "回读草稿：materials.realtime_denoises / vocal_beautifys / audio_effects 里有新条目且段上 extra_material_refs 变了" } } diff --git a/plugins/antianqi/jianying/references/official-ops.md b/plugins/antianqi/jianying/references/official-ops.md index 3d4f693d..f5a5b27c 100644 --- a/plugins/antianqi/jianying/references/official-ops.md +++ b/plugins/antianqi/jianying/references/official-ops.md @@ -147,6 +147,18 @@ 实测：各种形态都**只建了空 text 轨、没落段**。 ⇒ **但普通 `text.add` + `startTime/duration` 已经能落出带时码的文字（= 字幕轨）** ⇒ 走那条路。 + +## ★ 音频 AI 效果类 op（实测 2026-10-21） + +| 能力 | op | 参数 | 落成 | 能不能真算 | +|---|---|---|---|---| +| **降噪** | `video.denoiseOn`（★ **不是** `audio.denoiseOn`） | `{segmentId, isDenoise?, denoiseMode?, denoiseRate?}` | `materials.realtime_denoises[]`：`{type:"realtime_denoise", sami_name:"denoise_v2", sami_version:"1.0", is_denoise, denoise_mode, denoise_rate}` | ✅ **本地 SAMI 实时 DSP**，不依赖云端 | +| **人声美化** | `audio.setVocalBeautify` | `{segmentId, enable:true}` | `materials.vocal_beautifys[]`：`{type:"vocal_beautify", enable, ambient_sound_level:75, voice_change_mode:null, production_path:""}` | ✅ **本地实时**（`production_path` 为空 ⇒ 不是云端产出） | +| **变声** | `audio.setVoiceEffect` | `{segmentId}` | `materials.audio_effects[]` | ✅ 本地 | +| **人声分离** | `audio.setVocalSeparation` | ★ 参数未探对（6 种形态都无变化） | 结构已知：`{type:"vocal_separation", choice, removed_sounds, time_range, final_algorithm:"", production_path:""}` | ⚠️ **写引用可以，产出拿不到** —— `production_path`/`final_algorithm` 为空 ⇒ 分离是**剪映的 AI 任务**（与 ASR 同类，在 agent runtime / 云端） | + +**键名风格**：`isDenoise`/`is_denoise`、`denoiseMode`/`denoise_mode` 两种都认；`enable` 只认 `enable`（`isEnable` 无效）。 + ## 关键入口速查 | 想做的事 | 走哪个 op | diff --git a/plugins/antianqi/jianying/scripts/selftest-audio-ai.mjs b/plugins/antianqi/jianying/scripts/selftest-audio-ai.mjs new file mode 100644 index 00000000..12af2b0f --- /dev/null +++ b/plugins/antianqi/jianying/scripts/selftest-audio-ai.mjs @@ -0,0 +1,100 @@ +#!/usr/bin/env node +/** + * jianying 插件：音频 AI 效果类 op 的自检（降噪 / 人声美化 / 变声） + * 走插件自己的 draft_patch 动词（不是直连官方管道）⇒ 证明"新接进去的 op"在我们这条路上也真生效 + * 判据 = 语义：回读草稿，materials.realtime_denoises / vocal_beautifys / audio_effects 里出现新条目 + * ★ 负向注入：改动之前这些集合必须是空的（否则探针没有判别力） + * + * 用法: node scripts/selftest-audio-ai.mjs + * 需要: JYAI_HOME 指向 JYAI 仓库；剪映 11.6 在运行；JYAI_TEST_DRAFT 指向一个带音频的草稿 + */ +import { spawn } from 'node:child_process'; +import os from 'node:os'; +import path from 'node:path'; +import { fileURLToPath } from 'node:url'; + +const HERE = path.dirname(fileURLToPath(import.meta.url)); +const JYAI = process.env.JYAI_HOME || path.join(os.homedir(), 'JYAI'); +const DRAFT = process.env.JYAI_TEST_DRAFT || ''; + +let PASS = 0, FAIL = 0; +const check = (n, ok, d) => { + if (ok) { PASS++; console.log(` [PASS] ${n}${d ? ' — ' + d : ''}`); } + else { FAIL++; console.log(` [FAIL] ${n}${d ? ' — ' + d : ''}`); } +}; +function call(verb, args) { + return new Promise((resolve) => { + const c = spawn(process.execPath, [path.join(HERE, 'run-verb.mjs'), verb, JSON.stringify(args)], + { env: { ...process.env, JYAI_HOME: JYAI }, stdio: ['ignore', 'pipe', 'pipe'] }); + let so = '', se = ''; + c.stdout.on('data', (d) => { so += d; }); + c.stderr.on('data', (d) => { se += d; }); + c.on('close', (code) => { + let j = null; try { j = JSON.parse(so); } catch { /* not JSON */ } + resolve({ code, json: j, text: so, stderr: se, failed: code !== 0 || j?.isError === true }); + }); + }); +} +const readDraft = async (sid) => { + const r = await call('draft_read', { sessionId: sid }); + try { return JSON.parse(r.text).draft; } catch { return null; } +}; +const counts = (d) => ({ + denoise: d?.materials?.realtime_denoises?.length ?? 0, + beautify: d?.materials?.vocal_beautifys?.length ?? 0, + effect: d?.materials?.audio_effects?.length ?? 0, +}); +const segOf = (d) => (d?.tracks || []).flatMap((t) => t.segments || [])[0]; + +console.log('=== 音频 AI 效果自检（降噪 / 人声美化 / 变声）===\n'); +if (!DRAFT) { + console.log(' [SKIP] 需要 JYAI_TEST_DRAFT —— 指向一个带音频的剪映草稿目录。'); + console.log(' 例（PowerShell）: $env:JYAI_TEST_DRAFT = "<草稿库>\\<草稿名>"'); + process.exit(0); +} + +const open = await call('draft_open', { draft: DRAFT, copy: true }); +if (open.failed) { console.error('打开草稿失败：\n' + (open.text + open.stderr).slice(0, 400)); process.exit(3); } +const sid = open.json.sessionId; + +const d0 = await readDraft(sid); +const seg = segOf(d0); +const c0 = counts(d0); +check('① 拿到草稿并定位到第一个段', !!seg?.id, seg?.id); +check('★ ② 负向：改动前这三类 material 都是空的（探针有判别力）', + c0.denoise === 0 && c0.beautify === 0 && c0.effect === 0, + `denoise=${c0.denoise} beautify=${c0.beautify} effect=${c0.effect}`); + +// ③ 降噪（video.denoiseOn，不是 audio.denoiseOn） +const dn = await call('draft_patch', { op: 'video.denoiseOn', sessionId: sid, params: { segmentId: seg.id, isDenoise: true } }); +const d1 = await readDraft(sid); +const c1 = counts(d1); +const dnMat = d1?.materials?.realtime_denoises?.[0]; +check('③ 降噪 video.denoiseOn 落进草稿', !dn.failed && c1.denoise > c0.denoise, + dn.failed ? (dn.text + dn.stderr).replace(/\s+/g, ' ').slice(0, 120) : `realtime_denoises ${c0.denoise}→${c1.denoise} 引擎=${dnMat?.sami_name} is_denoise=${dnMat?.is_denoise}`); +check('③b 降噪参数落到 material（isDenoise→is_denoise）', dnMat?.is_denoise === true, + `is_denoise=${dnMat?.is_denoise} denoise_mode=${dnMat?.denoise_mode} denoise_rate=${dnMat?.denoise_rate}`); + +// ④ 人声美化 +const vb = await call('draft_patch', { op: 'audio.setVocalBeautify', sessionId: sid, params: { segmentId: seg.id, enable: true } }); +const d2 = await readDraft(sid); +const c2 = counts(d2); +const vbMat = d2?.materials?.vocal_beautifys?.[0]; +check('④ 人声美化 audio.setVocalBeautify 落进草稿', !vb.failed && c2.beautify > c1.beautify, + vb.failed ? (vb.text + vb.stderr).replace(/\s+/g, ' ').slice(0, 120) : `vocal_beautifys ${c1.beautify}→${c2.beautify}`); +check('④b 人声美化 enable 参数落到 material', vbMat?.enable === true, `enable=${vbMat?.enable} type=${vbMat?.type}`); + +// ⑤ 变声 +const ve = await call('draft_patch', { op: 'audio.setVoiceEffect', sessionId: sid, params: { segmentId: seg.id } }); +const d3 = await readDraft(sid); +const c3 = counts(d3); +check('⑤ 变声 audio.setVoiceEffect 落进草稿', !ve.failed && c3.effect > c2.effect, + ve.failed ? (ve.text + ve.stderr).replace(/\s+/g, ' ').slice(0,120) : `audio_effects ${c2.effect}→${c3.effect}`); + +// ⑥ 段上确实挂上了引用（extra_material_refs 变了） +const refsBefore = (d0?.tracks || []).flatMap((t) => t.segments || []).find((s) => s.id === seg.id)?.extra_material_refs?.length ?? 0; +const refsAfter = (d3?.tracks || []).flatMap((t) => t.segments || []).find((s) => s.id === seg.id)?.extra_material_refs?.length ?? 0; +check('⑥ 段上 extra_material_refs 增加了（效果真的挂到段上）', refsAfter > refsBefore, `${refsBefore} → ${refsAfter}`); + +console.log(`\n=== ${PASS} PASS / ${FAIL} FAIL ===`); +process.exit(FAIL === 0 ? 0 : 1); From 4e86cb001040a72ddf9bac3c691a52c99d7a725c Mon Sep 17 00:00:00 2001 From: antianqi Date: Wed, 21 Oct 2026 19:10:24 +0800 Subject: [PATCH 07/15] =?UTF-8?q?docs(jianying):=20correct=20the=20audio?= =?UTF-8?q?=20claim=20=E2=80=94=20written=20into=20the=20draft=20is=20not?= =?UTF-8?q?=20the=20same=20as=20present=20in=20the=20output?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The membership check fires at export, not while editing: the official FAQ says advanced features marked with a VIP crown (voice cloning, smart denoise) only prompt for a subscription before the video is exported. That matches what we measured — without a membership, video.denoiseOn, audio.setVocalBeautify and audio.setVoiceEffect all wrote into the draft without complaint. It also means the earlier phrasing needs a correction. The export pipeline is the official renderer for picture (player domain) plus our own ffmpeg mix for sound, and that mix handles volume, fades, speed and loudness only: it does not read realtime_denoises or vocal_beautifys. The official renderer emits a video stream with no audio track at all, the official media domain only has file-level tools, and the official exporter is blocked by -100007 and -100015. So the effects land in the draft, and Jianying will honour them when it exports, but the files we produce do not carry them. Two consequences recorded in references/official-ops.md and the capability dictionary: the denoise and beautify entries now say explicitly that landing in the draft does not mean landing in our output, and the acceptance line separates "written into the draft" from "present in the delivered file". Also worth recording from the local 11.6 install: the paywall surfaces in the agent panel (vip_use_popup) and answers with gateId, action "retry" and expectedRevision, which means a membership block is distinguishable from a parameter mistake — a distinction the verification discipline should carry. Making the effect audible in our own output means implementing it in the mix chain instead (ffmpeg afftdn for denoise, EQ and compression as an approximation for beautify, a Demucs-class model for separation), and saying plainly that those are not Jianying's own DSP. --- .../jianying/references/capability-dict.json | 5 +++-- .../jianying/references/official-ops.md | 22 +++++++++++++++++++ 2 files changed, 25 insertions(+), 2 deletions(-) diff --git a/plugins/antianqi/jianying/references/capability-dict.json b/plugins/antianqi/jianying/references/capability-dict.json index 9cc96786..b1aa8221 100644 --- a/plugins/antianqi/jianying/references/capability-dict.json +++ b/plugins/antianqi/jianying/references/capability-dict.json @@ -229,7 +229,8 @@ "★ accepted:true ≠ 生效：判据必须是\"回读草稿后字段真的变了\"（segment.setTimeRange 传 startUs/durationUs ⇒ 官方回 accepted 但时码被清零）", "★ text.add 的 needInsertTargetTrack=true 时段会插进 trackIndex 指向的那条轨（哪怕是 video 轨）⇒ 往同一条 text 轨塞多条字幕要用 needInsertTargetTrack=false + 该 text 轨下标", "★ 视频/音频 AI 效果类 op 的契约（实测 2026-10-21）：video.denoiseOn{segmentId,isDenoise?,denoiseMode?,denoiseRate?} ⇒ materials.realtime_denoises（引擎 sami_name=denoise_v2，本地实时 DSP）；audio.setVocalBeautify{segmentId,enable:true} ⇒ materials.vocal_beautifys（本地实时）；audio.setVocalSeparation{…} ⇒ materials.vocal_separations（production_path 为空 ⇒ 产出要剪映的 AI 任务，我们只写引用）", - "★ 视频/音频 AI 效果类 op 的契约（实测 2026-10-21）：video.denoiseOn{segmentId,isDenoise?,denoiseMode?,denoiseRate?} ⇒ materials.realtime_denoises（引擎 sami_name=denoise_v2，本地实时 DSP）；audio.setVocalBeautify{segmentId,enable:true} ⇒ materials.vocal_beautifys（本地实时）；audio.setVocalSeparation{…} ⇒ materials.vocal_separations（production_path 为空 ⇒ 产出要剪映的 AI 任务，我们只写引用）" + "★ 视频/音频 AI 效果类 op 的契约（实测 2026-10-21）：video.denoiseOn{segmentId,isDenoise?,denoiseMode?,denoiseRate?} ⇒ materials.realtime_denoises（引擎 sami_name=denoise_v2，本地实时 DSP）；audio.setVocalBeautify{segmentId,enable:true} ⇒ materials.vocal_beautifys（本地实时）；audio.setVocalSeparation{…} ⇒ materials.vocal_separations（production_path 为空 ⇒ 产出要剪映的 AI 任务，我们只写引用）", + "★ 剪映的音频效果只被【剪映自己的渲染/导出】应用；我们的出片链（官方渲染出画面 + ffmpeg 自己混音）不读 realtime_denoises / vocal_beautifys ⇒ 写进草稿 ≠ 出现在我们的成片里" ] }, "draft_commit": { @@ -334,6 +335,6 @@ "draft_commit": "真草稿 sha 前后逐字节相同", "draft_patch": "回读草稿后目标字段真的变了（不接受\"官方说 accepted\"）", "text.add": "字幕时码与期望吻合（容差 ±1 帧，官方按 fps 帧对齐）", - "audio_ai": "回读草稿：materials.realtime_denoises / vocal_beautifys / audio_effects 里有新条目且段上 extra_material_refs 变了" + "audio_ai": "回读草稿：materials.realtime_denoises / vocal_beautifys / audio_effects 里有新条目且段上 extra_material_refs 变了（★ 这只证明\"写进草稿了\"，不证明\"出现在我们的成片里\"——后者要另外在混音链里实现）" } } diff --git a/plugins/antianqi/jianying/references/official-ops.md b/plugins/antianqi/jianying/references/official-ops.md index f5a5b27c..7b96ddda 100644 --- a/plugins/antianqi/jianying/references/official-ops.md +++ b/plugins/antianqi/jianying/references/official-ops.md @@ -157,6 +157,28 @@ | **变声** | `audio.setVoiceEffect` | `{segmentId}` | `materials.audio_effects[]` | ✅ 本地 | | **人声分离** | `audio.setVocalSeparation` | ★ 参数未探对（6 种形态都无变化） | 结构已知：`{type:"vocal_separation", choice, removed_sounds, time_range, final_algorithm:"", production_path:""}` | ⚠️ **写引用可以，产出拿不到** —— `production_path`/`final_algorithm` 为空 ⇒ 分离是**剪映的 AI 任务**（与 ASR 同类，在 agent runtime / 云端） | + +**★ 但我们的出片链目前【不应用】这些效果**（2026-10-21 查清，重要）： + +我们的出片是 **官方渲染器出画面（`player` 域）+ 我们自己用 ffmpeg 混音**： + +| 环节 | 实际做了什么 | 剪映的音频效果 | +|---|---|---| +| 画面 | 官方渲染器出 PNG 序列（`player` 域）⇒ **带**剪映的调色/特效/转场 | — | +| 音频 | `jyai-audio-mix.mjs` 自己混：`volume` / `fade_in.ms` / `fade_out.ms` / `speed`（官方 `media change-audio-speed` 保音高）/ 响度归一（ffmpeg loudnorm） | ❌ **不读** `realtime_denoises` / `vocal_beautifys` | +| 官方渲染产物 | 实测**只有 video 流、没有音轨** | — | +| 官方 `media` 域 | 只有文件级工具（trim / extract-frames / trim-audio / change-audio-speed / resize / compress），**没有带效果的渲染或导出** | — | +| 官方导出器 | 被 `-100007` / `-100015` 挡死 | — | + +⇒ 所以准确说法是：**降噪/美化/变声能写进草稿**（剪映 GUI 打开看得到、它自己导出时会用）， +**但我们自己出的片不带这些效果**。 + +⇒ 要让降噪真的出现在成片里，两条路： +1. **让剪映渲染音频** —— 需要它的导出/渲染管线，我们拿不到（导出器被官方挡死）。 +2. **自己在混音链里加**（ffmpeg `afftdn` 降噪 / EQ / 压缩 / 去齿音近似"人声美化"； + 人声分离要模型 Demucs/Spleeter）⇒ **这才是"我们自己实现"那条路的真正价值： + 不是复刻剪映的 DSP，而是补上剪映本地链的缺口。** + **键名风格**：`isDenoise`/`is_denoise`、`denoiseMode`/`denoise_mode` 两种都认；`enable` 只认 `enable`（`isEnable` 无效）。 ## 关键入口速查 From d726180bce3dedf083ab67693f7ab4a0038f2ab4 Mon Sep 17 00:00:00 2001 From: antianqi Date: Wed, 21 Oct 2026 20:06:32 +0800 Subject: [PATCH 08/15] fix(jianying): calibrate the pixel criterion per frame instead of against a global noise floor MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The user pointed out that Jianying also has a live preview while editing, which challenged how we had attributed the "official renderer noise floor". Measuring it properly, in one session, for the same target timestamp: - capture the frame twice without seeking: 4 of 6 timestamps differ - seek away and back, capture again: 4 of 6 differ - the VEF1 timestamp delta against the requested time: 0 every time So the non-determinism is not a mis-aimed frame, not a time-alignment problem and not a globally unstable renderer — a simpler one-line draft is fully deterministic (0/6) while this draft is not, and the same draft rendered twice differs in 69 of 108 frames. It happens per capture, on particular frames: frames 001-004 come back wholly different (MAD around 123, maxAbs 255) while frames 000 and 005-009 differ by under 1.7. A preview engine like this is also what the GUI runs, so nobody notices it during playback, but a static per-pixel comparison magnifies it. That makes a single global noise floor too coarse, and actively wrong: with one frame sitting at noise 128 the global bar becomes 192, which then buries a real change of 64 on a perfectly stable frame and reports a false FAIL. The criterion is now per frame, bar_i = max(threshold, noise_i * noiseK), so a frame whose own noise already exceeds the threshold counts as undecidable while a real change on any stable frame is still detected. verify_diff now reports unstableFrames, visibleMaxMad, visibleFrame and a per-frame frameDetail, and the self-test covers it: three synthetic frame sets where one frame is unstable and another carries a real change, asserting that the unstable frame is flagged and the stable change still reads as visible. Worth stating plainly: this does not touch the grading fidelity result. The 13/13 at 1 LSB is measured against an offline GL run of the official shader fragments, not against frames pulled from the preview renderer, so it was never exposed to this non-determinism. --- plugins/antianqi/jianying/mcp/server.mjs | 49 +++++++++++++++++-- .../jianying/references/verify-discipline.md | 24 +++++++++ .../jianying/scripts/selftest-mcp.mjs | 41 ++++++++++++++++ 3 files changed, 109 insertions(+), 5 deletions(-) diff --git a/plugins/antianqi/jianying/mcp/server.mjs b/plugins/antianqi/jianying/mcp/server.mjs index 2706bf17..90c7c49a 100644 --- a/plugins/antianqi/jianying/mcp/server.mjs +++ b/plugins/antianqi/jianying/mcp/server.mjs @@ -471,6 +471,9 @@ const IMPL = { let best = { mad: -1, frame: -1, maxAbs: 0, changed: 0 }; let noiseBest = { mad: -1, frame: -1 }; let changedFrames = 0, noiseChangedFrames = 0; + // ★ 逐帧噪声（2026-10-21 实测：非确定性是【逐帧】的，不是全局的） + const perFrameNoise = []; + const perFrameMad = []; // picture 统计（after 侧全局；before 侧同步累计，用来算【增量】） const sum = [0, 0, 0]; const mn = [255, 255, 255]; @@ -488,6 +491,7 @@ const IMPL = { dsum += d; if (d > dmx) dmx = d; if (d > 0) ch++; } const mad = dsum / x.length; + perFrameMad[i] = mad; if (mad > 0) changedFrames++; if (mad > best.mad) best = { mad, frame: i, maxAbs: dmx, changed: ch }; // ★ 噪声底：同一状态【再渲染一次】的帧 ⇒ 同态差异就是渲染器噪声 @@ -497,6 +501,7 @@ const IMPL = { let nsum = 0; for (let k = 0; k < x.length; k++) nsum += Math.abs(x[k] - z[k]); const nmad = nsum / x.length; + perFrameNoise[i] = nmad; if (nmad > 0) noiseChangedFrames++; if (nmad > noiseBest.mad) noiseBest = { mad: nmad, frame: i }; } @@ -528,21 +533,55 @@ const IMPL = { const noiseMad = noiseBest.mad < 0 ? null : noiseBest.mad; const noiseK = a.noiseK ?? 1.5; const calibrated = noiseMad !== null; - const bar = calibrated ? Math.max(th, noiseMad * noiseK) : th; - const visible = best.mad > bar; + // ★★★ 逐帧判据线（2026-10-21 §82 修正） + // 实测：官方预览渲染的【非确定性是逐帧的】——同一草稿渲两遍， + // 整片 MAD 123.55，但同一片里另一些帧只差 0.5； + // 而且【连着抓两次同一时刻】像素就不同 ⇒ 是"每次抓帧"层面的非确定， + // 不是"重渲染"层面的。官方 GUI 里也是这套引擎（连续播放时人眼察觉不到）， + // 但逐像素静态比对会把它放大。 + // ⇒ 判据线必须【逐帧用自己的噪声】来定，而不是用全局最大噪声： + // bar_i = max(threshold, noise_i × noiseK) + // 这样"那一帧本来就分不出"（noise_i 很大 ⇒ bar_i 更大 ⇒ 不算可见）， + // 而其它稳定帧的真实改动照常被检出。 + const frameDetail = []; + for (let i = 0; i < n; i++) { + const mad = perFrameMad[i]; + if (mad === undefined) continue; + const nm = perFrameNoise[i]; + const barI = calibrated && nm !== undefined ? Math.max(th, nm * noiseK) : th; + frameDetail.push({ + frame: i, mad, noise: nm === undefined ? null : nm, bar: barI, + unstable: calibrated && nm !== undefined && nm > th, + visible: mad > barI, + }); + } + const visibleRows = frameDetail.filter((r) => r.visible); + const unstable = frameDetail.filter((r) => r.unstable).map((r) => r.frame); + const bestVisible = visibleRows.length + ? visibleRows.reduce((a, b) => (b.mad > a.mad ? b : a)) + : null; + const bar = calibrated ? Math.max(th, noiseMad * noiseK) : th; // 全局参考线（仅作参考） + const visible = visibleRows.length > 0; const v = { maxMad: best.mad, frame: best.frame, maxAbs: best.maxAbs, changedPixels: best.changed, changedFrames, totalFrames: n, threshold: th, noiseMad, noiseFrame: noiseBest.frame, noiseChangedFrames, noiseK, calibrated, noiseBar: bar, + unstableFrames: unstable, + unstableFrameCount: unstable.length, + visibleMaxMad: bestVisible ? bestVisible.mad : null, + visibleFrame: bestVisible ? bestVisible.frame : null, + visibleFrameCount: visibleRows.length, + frameDetail, madOverNoise: noiseMad ? +(best.mad / noiseMad).toFixed(3) : null }; v.verdict = visible ? 'PASS（改动可见）' : 'FAIL（改动在画面上不可见）'; if (!calibrated) { - v.note = '未校准：官方渲染器同态重渲染噪声实测可达 MAD 123.55（17/108 帧变）⇒ 裸 MAD 不能单独证明"改动可见"；请补 noiseDir'; + v.note = '未校准：官方渲染器同态重渲染噪声实测可达 MAD 123.55（逐帧分布，17/108 帧变）⇒ 裸 MAD 不能单独证明"改动可见"；请补 noiseDir'; } res.checks.visibility = v; if (!visible) { - problems.push(`画面不可见：max MAD ${best.mad.toFixed(4)} ≤ 判据线 ${bar.toFixed(4)}` - + (calibrated ? `（噪声底 ${noiseMad.toFixed(4)} × ${noiseK}）` : '（未校准，阈值仅 0.5）')); + const shown = bestVisible ? bestVisible.mad : (calibrated ? 0 : best.mad); + problems.push(`画面不可见：${calibrated ? '逐帧判据线下没有一帧达标' : `max MAD ${shown.toFixed(4)} ≤ 阈值 ${th}`}` + + (calibrated ? `（全局噪声底 ${noiseMad.toFixed(4)} × ${noiseK}；不稳定帧 ${unstable.length}/${n} 帧：噪声本身就超过阈值 ⇒ 这些帧分不出改动）` : '（未校准，阈值仅 0.5）')); } } if (want('picture') && px > 0) { diff --git a/plugins/antianqi/jianying/references/verify-discipline.md b/plugins/antianqi/jianying/references/verify-discipline.md index 42e08ca4..b70b284b 100644 --- a/plugins/antianqi/jianying/references/verify-discipline.md +++ b/plugins/antianqi/jianying/references/verify-discipline.md @@ -3,6 +3,30 @@ > 这份是**护栏的知识层**。机械部分在 `scripts/` 与 MCP 动词的内建检查里。 > ★ **护城河不在工具，在判据** —— 工具（读草稿/渲染/调色）谁都能调；"怎么知道它真的生效了"是我们独有的。 +⇒ **"噪声"大部分是判据的放大，不是画质的崩坏** —— 人眼在 GUI 的连续播放里察觉不到，但逐像素静态比对会把它放大。 + +**★ 而且非确定性是【逐帧】的**（2026-10-21 §82 实测）： + +| 测法 | 结果 | +|---|---| +| 同一草稿渲两遍 | **69/108 帧不同**；frame 001~004 是 MAD≈123 / maxAbs 255 / 95% 像素全变（整帧不同），frame 000/005~009 只差 0.47~1.69 | +| 同一时刻**连着抓两次** | **4/6 时刻像素就不同** ⇒ 非确定发生在"每一次抓帧"这一层，不是"隔一段时间重渲染" | +| 抓到的 ts 与目标 t 的偏差 | **恒为 0** ⇒ **不是抓错帧、不是时间对齐问题** | +| 另一个简单草稿 | **0/6 不确定** ⇒ **不是整个渲染器都不确定**（只是特定时刻/特定素材帧） | + +⇒ **判据必须【逐帧】校准**：`bar_i = max(threshold, noise_i × noiseK)`， +**不能用全局最大噪声**（那会把稳定帧的真实改动一起压死 ⇒ 假 FAIL）。 +`verify_diff` 的 visibility 维度现在输出 `unstableFrames` / `visibleMaxMad` / `visibleFrame` / `frameDetail[]`。 + +**★ 顺带澄清两类"对齐"必须分开说**： + +| 比什么 | 参照 | 确定性 | +|---|---|---| +| 我们的复刻 **vs 官方 shader** | 本机跑官方 `.frag` shader 的**离线 GL 渲染** | ✅ 确定 ⇒ **≤1 LSB 结论不受影响** | +| 官方渲染帧 **vs 官方渲染帧** | 预览渲染器抓的帧 | ⚠️ 逐帧部分非确定 ⇒ 必须逐帧校准 | + +**⇒ 纪律：像素判据必须逐帧校准；没有 `noiseDir` 就标"未校准"，不许装作可信。** + ## 三层判定 ``` diff --git a/plugins/antianqi/jianying/scripts/selftest-mcp.mjs b/plugins/antianqi/jianying/scripts/selftest-mcp.mjs index f599db73..b9766e2c 100644 --- a/plugins/antianqi/jianying/scripts/selftest-mcp.mjs +++ b/plugins/antianqi/jianying/scripts/selftest-mcp.mjs @@ -56,6 +56,29 @@ async function synthFrames(dir, brightness) { return colors.length; } +// ★ 逐帧稳定性用例（§82）：造三组帧 +// base : 3 帧原样 +// unstable: 只有第 1 帧被大幅扰动（模拟"官方预览渲染在该帧非确定"） +// changed : 只有第 2 帧被小幅真实改动 +// ⇒ 期望：第 1 帧被判为不稳定并排除；第 2 帧的稳定改动仍被检出 ⇒ 判"可见" +async function synthFramesPerFrame(dir, { unstableIndex = null, unstableBright = null, changedIndex = null, changedBright = null } = {}) { + fs.rmSync(dir, { recursive: true, force: true }); + fs.mkdirSync(dir, { recursive: true }); + const colors = ['606060', '808080', 'A0A0A0']; + for (let i = 0; i < colors.length; i++) { + let bright = null; + if (i === unstableIndex) bright = unstableBright; + if (i === changedIndex) bright = changedBright; + const f = path.join(dir, String(i).padStart(5, '0') + '.png'); + const args = ['-y', '-v', 'error', '-f', 'lavfi', '-i', `color=c=0x${colors[i]}:s=64x64`]; + if (bright !== null) args.push('-vf', `eq=brightness=${bright}`); + args.push('-frames:v', '1', f); + const r = await sh(args); + if (r.code !== 0) return 0; + } + return colors.length; +} + const child = spawn(process.execPath, [SRV], { env: { ...process.env, JYAI_HOME: JYAI, PYTHONIOENCODING: 'utf-8', PYTHONUTF8: '1' }, stdio: ['pipe', 'pipe', 'inherit'], @@ -177,6 +200,24 @@ const textOf = (r) => (r.result?.content || []).map((c) => c.text || '').join('\ (picc.clipDelta ?? 0) > 0.02 && (v1cj?.problems || []).some((p) => /削波/.test(p)), `clip=${picc.clipRatio} clipΔ=${picc.clipDelta} problems=${JSON.stringify(v1cj?.problems)}`); + // 4e. ★★ 逐帧稳定性（§82）：噪声集中在少数帧 ⇒ 不稳定帧排除，稳定帧的改动仍要检出 + const FR_PF_BASE = path.join(OUT, 'frames-pf-base'); + const FR_PF_UNSTABLE = path.join(OUT, 'frames-pf-unstable'); + const FR_PF_CHANGED = path.join(OUT, 'frames-pf-changed'); + await synthFramesPerFrame(FR_PF_BASE, {}); + await synthFramesPerFrame(FR_PF_UNSTABLE, { unstableIndex: 1, unstableBright: -0.5 }); + await synthFramesPerFrame(FR_PF_CHANGED, { changedIndex: 2, changedBright: -0.2 }); + const v1d = await call('tools/call', { name: 'verify_diff', arguments: { + beforeDir: FR_PF_BASE, afterDir: FR_PF_CHANGED, noiseDir: FR_PF_UNSTABLE, checks: ['visibility'] } }); + const v1dj = (() => { try { return JSON.parse(textOf(v1d)); } catch { return null; } })(); + const visd = v1dj?.checks?.visibility || {}; + check('★★ verify_diff 逐帧稳定性：不稳定帧被单独标出', + visd.unstableFrameCount === 1 && (visd.unstableFrames || []).includes(1), + `unstableFrames=${JSON.stringify(visd.unstableFrames)} visibleFrameCount=${visd.visibleFrameCount}`); + check('★★ verify_diff 逐帧稳定性：稳定帧上的真实改动仍被判为可见', + v1d.result?.isError !== true && v1dj?.verdict === 'PASS' && (visd.visibleMaxMad ?? 0) > 0, + `verdict=${v1dj?.verdict} visibleMaxMad=${visd.visibleMaxMad} @frame${visd.visibleFrame}`); + // 5. ★ 阴性：同一目录对同一目录 ⇒ 必须 FAIL（判据要有判别力） const v2 = await call('tools/call', { name: 'verify_diff', arguments: { beforeDir: FR_BASE, afterDir: FR_BASE } }); check('★ verify_diff 对无变化必须 FAIL', v2.result?.isError === true, textOf(v2).replace(/\s+/g, ' ').slice(0, 140)); From 3302ac7f8bbc72fb2f809ce08ab26bbbc0b8e77d Mon Sep 17 00:00:00 2001 From: antianqi Date: Thu, 22 Oct 2026 09:32:26 +0800 Subject: [PATCH 09/15] docs(jianying): the denoise chain is now measured end to end, not just written into the draft The previous text said our own output chain does not apply the draft audio effects at all, so "written into the draft" was never "present in the output". That statement was accurate when it was written, but the mixer now reads materials.realtime_denoises out of the draft, matches it per segment through extra_material_refs, and applies ffmpeg afftdn. This commit replaces the stale claim with the measured result and its boundaries. Validation - End to end, official video.add + video.denoiseOn -> mixer -> muxed file: in-file SNR 5.646 dB (no denoise) -> 12.128 dB (denoise), +6.482 dB. - Negative injection: the same denoised draft mixed with the effect forced off reproduces the baseline exactly, A - C = 0.0000 dB. The difference therefore comes from the denoise application and from nothing else. - Criterion is the in-file SNR (speech band RMS - noise band RMS), so the loudness normalisation in the mix chain cannot bias the number. Test evidence - plugin selftests, all green: selftest-mcp 19/19, selftest-draft-edit 16/16, selftest-talk-cut 10/10, selftest-subtitle 4/4, selftest-audio-ai 8/8. - node scripts/validate.mjs: 30 hosted plugins validated. - capability-dict.json re-parsed after the edit. The hosted validator does not parse this file, so its green run says nothing about it. - Parameter calibration on 20 s of real speech plus a noise floor calibrated to a known level: nr = 6 + 15*rate, nf = -26 / -23 / -20 for mode 1/2/3. Design compliance - The approximation is labelled as such. It is ffmpeg afftdn, not the Jianying SAMI denoise_v2 model, and the doc now says so next to the number instead of hiding it behind a stale "not applied" claim. - The boundaries travel with the claim: the speech band sits 3.1 to 5.4 dB below the clean reference, only stationary pink noise was calibrated, and the zero-gap check rules out whole-phrase dropouts but not fine grained musical noise. - The strength ladder is explicitly our own. The denoise_mode semantics of Jianying are not recoverable from the app bundle, so the doc says the ladder is ours and not a mirror of the product. - Known upstream trap recorded for anyone reproducing the calibration: the ffmpeg afftdn nf parameter is range [-80, -20] in current builds, so -20 is the strongest usable value and -15 is rejected outright. --- .../jianying/references/capability-dict.json | 4 +-- .../jianying/references/official-ops.md | 28 +++++++++++++++---- 2 files changed, 24 insertions(+), 8 deletions(-) diff --git a/plugins/antianqi/jianying/references/capability-dict.json b/plugins/antianqi/jianying/references/capability-dict.json index b1aa8221..8fda8905 100644 --- a/plugins/antianqi/jianying/references/capability-dict.json +++ b/plugins/antianqi/jianying/references/capability-dict.json @@ -230,7 +230,7 @@ "★ text.add 的 needInsertTargetTrack=true 时段会插进 trackIndex 指向的那条轨（哪怕是 video 轨）⇒ 往同一条 text 轨塞多条字幕要用 needInsertTargetTrack=false + 该 text 轨下标", "★ 视频/音频 AI 效果类 op 的契约（实测 2026-10-21）：video.denoiseOn{segmentId,isDenoise?,denoiseMode?,denoiseRate?} ⇒ materials.realtime_denoises（引擎 sami_name=denoise_v2，本地实时 DSP）；audio.setVocalBeautify{segmentId,enable:true} ⇒ materials.vocal_beautifys（本地实时）；audio.setVocalSeparation{…} ⇒ materials.vocal_separations（production_path 为空 ⇒ 产出要剪映的 AI 任务，我们只写引用）", "★ 视频/音频 AI 效果类 op 的契约（实测 2026-10-21）：video.denoiseOn{segmentId,isDenoise?,denoiseMode?,denoiseRate?} ⇒ materials.realtime_denoises（引擎 sami_name=denoise_v2，本地实时 DSP）；audio.setVocalBeautify{segmentId,enable:true} ⇒ materials.vocal_beautifys（本地实时）；audio.setVocalSeparation{…} ⇒ materials.vocal_separations（production_path 为空 ⇒ 产出要剪映的 AI 任务，我们只写引用）", - "★ 剪映的音频效果只被【剪映自己的渲染/导出】应用；我们的出片链（官方渲染出画面 + ffmpeg 自己混音）不读 realtime_denoises / vocal_beautifys ⇒ 写进草稿 ≠ 出现在我们的成片里" + "★ 我们的出片链【已接上降噪】（2026-10-22 端到端实测）：混音器读草稿 materials.realtime_denoises → 按段 extra_material_refs 匹配 → ffmpeg afftdn（★ 近似，不是剪映的 SAMI denoise_v2）⇒ 成片文件内 SNR 5.646→12.128 dB（+6.482 dB），负向注入组 0.0000 dB；vocal_beautifys 仍是近似且默认关闭，vocal_separations 拿不到产出 ⇒ 对美化/分离而言『写进草稿 ≠ 出现在我们的成片里』依然成立" ] }, "draft_commit": { @@ -335,6 +335,6 @@ "draft_commit": "真草稿 sha 前后逐字节相同", "draft_patch": "回读草稿后目标字段真的变了（不接受\"官方说 accepted\"）", "text.add": "字幕时码与期望吻合（容差 ±1 帧，官方按 fps 帧对齐）", - "audio_ai": "回读草稿：materials.realtime_denoises / vocal_beautifys / audio_effects 里有新条目且段上 extra_material_refs 变了（★ 这只证明\"写进草稿了\"，不证明\"出现在我们的成片里\"——后者要另外在混音链里实现）" + "audio_ai": "回读草稿：materials.realtime_denoises / vocal_beautifys / audio_effects 里有新条目且段上 extra_material_refs 变了（★ 这只证明\"写进草稿了\"，不证明\"出现在我们的成片里\"——降噪已另外在混音链里实现并实测：成片文件内 SNR +6.482 dB、负向注入 0.0000 dB；美化/分离仍未实现）" } } diff --git a/plugins/antianqi/jianying/references/official-ops.md b/plugins/antianqi/jianying/references/official-ops.md index 7b96ddda..7cc90326 100644 --- a/plugins/antianqi/jianying/references/official-ops.md +++ b/plugins/antianqi/jianying/references/official-ops.md @@ -158,27 +158,43 @@ | **人声分离** | `audio.setVocalSeparation` | ★ 参数未探对（6 种形态都无变化） | 结构已知：`{type:"vocal_separation", choice, removed_sounds, time_range, final_algorithm:"", production_path:""}` | ⚠️ **写引用可以，产出拿不到** —— `production_path`/`final_algorithm` 为空 ⇒ 分离是**剪映的 AI 任务**（与 ASR 同类，在 agent runtime / 云端） | -**★ 但我们的出片链目前【不应用】这些效果**（2026-10-21 查清，重要）： +**★ 我们的出片链原先【不应用】这些效果（2026-10-21 查清）⇒ 现已【落地并实测】（2026-10-22）**： 我们的出片是 **官方渲染器出画面（`player` 域）+ 我们自己用 ffmpeg 混音**： | 环节 | 实际做了什么 | 剪映的音频效果 | |---|---|---| | 画面 | 官方渲染器出 PNG 序列（`player` 域）⇒ **带**剪映的调色/特效/转场 | — | -| 音频 | `jyai-audio-mix.mjs` 自己混：`volume` / `fade_in.ms` / `fade_out.ms` / `speed`（官方 `media change-audio-speed` 保音高）/ 响度归一（ffmpeg loudnorm） | ❌ **不读** `realtime_denoises` / `vocal_beautifys` | +| 音频 | 混音器自己混：`volume` / `fade_in.ms` / `fade_out.ms` / `speed`（官方 `media change-audio-speed` 保音高）/ 响度归一（ffmpeg loudnorm） | ✅ **降噪已接上**：混音器读草稿 `materials.realtime_denoises` → 按段 `extra_material_refs` 匹配 → ffmpeg `afftdn`（**近似**，见下）· 美化仍是近似且**默认关闭** | | 官方渲染产物 | 实测**只有 video 流、没有音轨** | — | | 官方 `media` 域 | 只有文件级工具（trim / extract-frames / trim-audio / change-audio-speed / resize / compress），**没有带效果的渲染或导出** | — | | 官方导出器 | 被 `-100007` / `-100015` 挡死 | — | -⇒ 所以准确说法是：**降噪/美化/变声能写进草稿**（剪映 GUI 打开看得到、它自己导出时会用）， -**但我们自己出的片不带这些效果**。 +⇒ **降噪的完整链路已端到端实测**（草稿 op 写意图 → 混音器读草稿 → ffmpeg → 成片）： -⇒ 要让降噪真的出现在成片里，两条路： +- 成片**文件内 SNR**：无降噪 5.646 dB → 有降噪 **12.128 dB**（**+6.482 dB**） +- **负向注入组 = 0.0000 dB**（同草稿但混音器强制不应用 ⇒ 差异必须归零，实测精确归零 ⇒ 差异确实来自降噪本身） +- 判据用"文件内 SNR"（语音带 RMS − 噪声带 RMS），抗 loudnorm 归一化干扰 + +⇒ **但这是近似，不是剪映的 DSP**（剪映用 SAMI `denoise_v2`）。边界（务必一起说）： +- 降噪后语音带能量比干净人声低 **3.1 ~ 5.4 dB**（"带能量"口径，**不等于"听感有损"**） +- 只在**稳态粉噪**一种噪声上标定过；音乐/风声/键盘声等非稳态噪声未覆盖 +- 空洞检测（语音带 >30 ms 静音段）= 0 ⇒ 没有整段吞音；**更细粒度的音乐噪声未排除** +- 混音器不随本插件分发（它是 JYAI 侧的出片工具） + +⇒ 剩下两条路： 1. **让剪映渲染音频** —— 需要它的导出/渲染管线，我们拿不到（导出器被官方挡死）。 -2. **自己在混音链里加**（ffmpeg `afftdn` 降噪 / EQ / 压缩 / 去齿音近似"人声美化"； +2. **自己在混音链里加**（ffmpeg `afftdn` 降噪 ✅已做 / EQ + 压缩 近似"人声美化" ⚠️默认关闭； 人声分离要模型 Demucs/Spleeter）⇒ **这才是"我们自己实现"那条路的真正价值： 不是复刻剪映的 DSP，而是补上剪映本地链的缺口。** +**★ 复现降噪标定时会踩的坑**： +- **ffmpeg `afftdn` 的 `nf` 参数合法范围是 `[-80, -20]`**（上游文档写的 `[-99, -5]` 在新版被拒：`Value -15.000000 for parameter 'nf' out of range [-80 - -20]`）⇒ `-20` 是能用的最强档 +- 实测在这版 ffmpeg 上 `nf` 越不负 = 降噪越多**且**语音带损失越小（nr=21 时：nf=-26 ⇒ 降噪 10.3 dB/语音保全 −5.4 dB；nf=-20 ⇒ 19.7 dB/−3.1 dB） +- 力度映射：`nr = 6 + 15*rate`；档位：`nf = -26 / -23 / -20`（mode 1/2/3）——**注意这是我们自己的强度阶梯，与剪映的 `denoise_mode` 语义无关**（剪映的档位含义在 app bundle 里不可检索） +- **`anlmdn` 在语音上完全无效**（实测降噪 0.001 dB）；`arndn` 需要模型，不可用 +- **别用纯噪声当测试信号**：`afftdn` 是语音降噪器，喂纯粉噪只能测出 0.63 dB 的假象 + **键名风格**：`isDenoise`/`is_denoise`、`denoiseMode`/`denoise_mode` 两种都认；`enable` 只认 `enable`（`isEnable` 无效）。 ## 关键入口速查 From ab42cd4aaee6c3122e1e4fd94e3c9c0893cdbd60 Mon Sep 17 00:00:00 2001 From: antianqi Date: Thu, 22 Oct 2026 12:17:41 +0800 Subject: [PATCH 10/15] feat(jianying): add make_film, one command from a Jianying draft to a finished film The plugin could render frames (render_frames) and mix audio (audio_mix), but calling them meant wiring two verbs plus an intermediate file by hand, and neither of them carried an acceptance criterion. make_film is the missing verb: it chains the official renderer, the optional 3D LUT, our own mix chain and ffmpeg, then verifies the result with assertions instead of printing it. Validation - make_film drives jyai-make-film.mjs: official renderer frames -> optional lut3d -> our mix chain (volume / fade / denoise / loudnorm) -> ffmpeg -> a file with both a video and an audio stream. - The acceptance step asserts, and fails the run on: no video stream, a duration that does not match the official timeline within one frame, and a loudness more than 1 LUFS away from the requested target. - A draft with no usable audio is an input state, not a broken chain: the run produces a picture-only film, reports how many materials were missing, and does not fail. - The official export tier is deliberately not used. ExportService, its mojo interface and the official encoder are not reachable from any non-GUI surface, so the film is built from the official renderer plus our own grading and mixing. Test evidence - selftest-mcp.mjs, 28 assertions, all green. New cases: the verb is exposed with its schema, a missing required argument is rejected, a non-existent draft fails loudly, a real run produces a two-stream file whose duration and loudness pass the built-in acceptance, and a draft whose audio materials are absent from disk still produces a film. - The two end-to-end cases are skipped, not silently passed, when JYAI_DRAFT_LIB is not set, so a missing environment can never look green. - node --test test/jianying.test.mjs: 10 pass, 0 fail. - node scripts/validate.mjs: 30 hosted plugins validated. - Three real defects were found by these tests and fixed before this commit: the renderer accepted a non-existent draft path and rendered an empty film instead of failing, the film builder picked the previous run's output as its own input when the same output directory was reused, and the self-test read the ffprobe stream list from stderr instead of stdout (which made a two-stream film look like it had no streams). Design compliance - The picture is the official renderer's output, not something we compute. The grading and the mix are ours, and the schema says so: denoise is labelled as an ffmpeg approximation rather than Jianying's own model. - Every acceptance criterion is an assertion. A measurement that cannot fail is not a criterion, so the loudness check rejects a target it cannot read and the duration check compares against the official timeline reported by the renderer. - The schema carries the same boundary note as the skills: the film is the official picture plus our grading and mix, not a Jianying export-tier film. - Plugin version moved to 0.2.0 because a verb was added, and the manifest description now says 9 verbs instead of 8. --- .../jianying/.claude-plugin/plugin.json | 4 +- plugins/antianqi/jianying/mcp/server.mjs | 17 +++++ .../jianying/mcp/tools/make-film.schema.json | 52 +++++++++++++ plugins/antianqi/jianying/plugin.json | 4 +- .../jianying/references/capability-dict.json | 12 +++ .../jianying/scripts/selftest-mcp.mjs | 74 ++++++++++++++++++- .../jianying/skills/jianying-cut/SKILL.md | 14 ++++ test/jianying.test.mjs | 3 +- 8 files changed, 174 insertions(+), 6 deletions(-) create mode 100644 plugins/antianqi/jianying/mcp/tools/make-film.schema.json diff --git a/plugins/antianqi/jianying/.claude-plugin/plugin.json b/plugins/antianqi/jianying/.claude-plugin/plugin.json index a446b098..539d4ca5 100644 --- a/plugins/antianqi/jianying/.claude-plugin/plugin.json +++ b/plugins/antianqi/jianying/.claude-plugin/plugin.json @@ -1,7 +1,7 @@ { "name": "jianying", - "version": "0.1.0", - "description": "用逐像素复刻的剪映调色链给视频调色并出 10bit 成片（保留原音轨），附逐节点生效探针与画面级验收。8 个 MCP 动词覆盖草稿读写、调色、混音、渲染、验收。", + "version": "0.2.0", + "description": "用逐像素复刻的剪映调色链给视频调色并出 10bit 成片（保留原音轨），附逐节点生效探针与画面级验收。9 个 MCP 动词覆盖草稿读写、调色、混音、渲染、验收，以及一条命令出成片（官方渲染器画面 + 我们的调色/混音 + ffmpeg 编码 + 带断言的验收）。", "author": { "name": "antianqi", "url": "https://github.com/antianqi" diff --git a/plugins/antianqi/jianying/mcp/server.mjs b/plugins/antianqi/jianying/mcp/server.mjs index 90c7c49a..a75695d0 100644 --- a/plugins/antianqi/jianying/mcp/server.mjs +++ b/plugins/antianqi/jianying/mcp/server.mjs @@ -446,6 +446,23 @@ const IMPL = { return ok(r.stdout); }, + // ★ 一条命令出成片：官方渲染器出画面（自带强度型调色）→ 可选我们的 3D LUT（曲线类） + // → 我们的混音链（音量/淡入淡出/降噪/响度归一）→ ffmpeg 编码 → 带断言的验收。 + // 官方导出档（ExportService / mojo / 官方编码器）不在我们的可达面上（§91.3 / §92.2）， + // 所以出片走这条自制链；它拿到的画面就是官方渲染器的输出，不是我们自己算的。 + async make_film(a) { + const args = [jyaiTool('jyai-make-film.mjs'), '--draft', a.draft, '--out', a.outDir]; + if (a.lut) args.push('--lut', a.lut); + if (a.denoise) args.push('--denoise', a.denoise); + if (a.beautify) args.push('--beautify', a.beautify); + if (a.fps) args.push('--fps', String(a.fps)); + if (a.maxFrames) args.push('--max-frames', String(a.maxFrames)); + if (a.targetLufs !== undefined) args.push('--target-lufs', String(a.targetLufs)); + const r = await run(process.execPath, args); + if (r.code !== 0) throw new Error(`出片失败：\n${r.stderr || r.stdout}`); + return ok(r.stdout); + }, + async verify_diff(a) { // ★ max-over-frames：判据视野必须覆盖被测对象（不抽单帧） // ★ 多维度审片（visibility / picture / audio / timeline / draft_integrity） diff --git a/plugins/antianqi/jianying/mcp/tools/make-film.schema.json b/plugins/antianqi/jianying/mcp/tools/make-film.schema.json new file mode 100644 index 00000000..8d60ee19 --- /dev/null +++ b/plugins/antianqi/jianying/mcp/tools/make-film.schema.json @@ -0,0 +1,52 @@ +{ + "name": "make_film", + "description": "一条命令出成片：官方渲染器出画面（★ 官方预览渲染器自带强度型调色）+ 可选我们的 3D LUT（曲线类调色，官方预览不认）+ 我们的混音链（音量/淡入淡出/降噪/响度归一）+ ffmpeg 编码，并做带断言的验收（双流齐备 / 时长 / 响度偏离 ≤1 LUFS 直接判失败）。", + "inputSchema": { + "type": "object", + "properties": { + "draft": { + "type": "string", + "description": "草稿目录" + }, + "outDir": { + "type": "string", + "description": "输出目录（成片落在 /film.mp4）" + }, + "lut": { + "type": "string", + "description": "可选：3D LUT 文件（.cube）—— 用于曲线类调色（curves / HSL / 色轮），官方预览渲染器不认这些，必须我们自己烘 LUT 再叠" + }, + "denoise": { + "type": "string", + "enum": ["auto", "off"], + "default": "auto", + "description": "降噪：auto = 草稿里有 realtime_denoises 就按强度施加（ffmpeg afftdn 近似，★ 不是剪映的 SAMI denoise_v2）；off = 不施加" + }, + "beautify": { + "type": "string", + "enum": ["off", "approx"], + "default": "off", + "description": "人声美化：off = 不做（默认）；approx = 近似 EQ+压缩（★ 剪映的真实算法不可得）" + }, + "fps": { + "type": "number", + "default": 12 + }, + "maxFrames": { + "type": "number", + "default": 0, + "description": "0 = 覆盖整个时间线（⚠️ 非 0 会让画面比音轨短，音画对不齐，只用于调试）" + }, + "targetLufs": { + "type": "number", + "default": -14, + "description": "响度归一目标（LUFS）。验收会断言实际值与之相差 ≤1 LUFS" + } + }, + "required": [ + "draft", + "outDir" + ], + "additionalProperties": false + } +} diff --git a/plugins/antianqi/jianying/plugin.json b/plugins/antianqi/jianying/plugin.json index 806d5326..1ca234b2 100644 --- a/plugins/antianqi/jianying/plugin.json +++ b/plugins/antianqi/jianying/plugin.json @@ -1,8 +1,8 @@ { "$schema": "https://agent-plugins.org/schemas/1.0.0/plugin.schema.json", "name": "jianying", - "version": "0.1.0", - "description": "用逐像素复刻的剪映调色链给视频调色并出 10bit 成片（保留原音轨），附逐节点生效探针与画面级验收。8 个 MCP 动词覆盖草稿读写、调色、混音、渲染、验收。", + "version": "0.2.0", + "description": "用逐像素复刻的剪映调色链给视频调色并出 10bit 成片（保留原音轨），附逐节点生效探针与画面级验收。9 个 MCP 动词覆盖草稿读写、调色、混音、渲染、验收，以及一条命令出成片（官方渲染器画面 + 我们的调色/混音 + ffmpeg 编码 + 带断言的验收）。", "author": { "name": "antianqi", "url": "https://github.com/antianqi" diff --git a/plugins/antianqi/jianying/references/capability-dict.json b/plugins/antianqi/jianying/references/capability-dict.json index 8fda8905..b8a391f9 100644 --- a/plugins/antianqi/jianying/references/capability-dict.json +++ b/plugins/antianqi/jianying/references/capability-dict.json @@ -279,6 +279,18 @@ "max-over-frames（不抽单帧）", "真草稿 sha 断言" ] + }, + "make_film": { + "kind": "ours", + "ops": [], + "guards": [ + "画面来自官方渲染器（player 域）⇒ ★ 官方预览渲染器自带强度型调色", + "可选 3D LUT（--lut）：曲线类调色（官方预览不认 curves/HSL/色轮 ⇒ 我们自己烘 LUT 再叠）", + "音频走我们的混音链：volume / fade / 降噪（ffmpeg afftdn 近似，★ 非剪映 SAMI denoise_v2）/ loudnorm 归一", + "验收带断言：双流齐备 + 时长对齐官方时间线（容差 1 帧）+ 响度偏离 ≤1 LUFS ⇒ 否则判失败", + "草稿无可用音轨 ⇒ 出纯画面片并报出缺失素材数（不判失败）", + "★ 官方导出档（ExportService / mojo / 官方编码器）不在我们的可达面上 ⇒ 不走官方导出" + ] } }, "capabilities": { diff --git a/plugins/antianqi/jianying/scripts/selftest-mcp.mjs b/plugins/antianqi/jianying/scripts/selftest-mcp.mjs index b9766e2c..fabbc6c2 100644 --- a/plugins/antianqi/jianying/scripts/selftest-mcp.mjs +++ b/plugins/antianqi/jianying/scripts/selftest-mcp.mjs @@ -31,6 +31,11 @@ function which(cmd) { return cmd; } const FFMPEG = which('ffmpeg'); +const FFPROBE = (() => { + const d = path.dirname(which('ffmpeg')); + const p = path.join(d, process.platform === 'win32' ? 'ffprobe.exe' : 'ffprobe'); + return fs.existsSync(p) ? p : 'ffprobe'; +})(); function sh(args) { return new Promise((resolve) => { const c = spawn(FFMPEG, args, { stdio: ['ignore', 'pipe', 'pipe'] }); @@ -39,6 +44,18 @@ function sh(args) { c.on('close', (code) => resolve({ code, stderr: se })); }); } +// 任意可执行文件版本（ffprobe 等）—— sh() 写死了 FFMPEG +// ★ 注意：ffprobe 的 -show_entries 结果走【stdout】；只有错误信息走 stderr +// （2026-10-22 自检抓到的假失败：只收 stderr ⇒ streams 解析成空字符串） +function shBin(bin, args) { + return new Promise((resolve) => { + const c = spawn(bin, args, { stdio: ['ignore', 'pipe', 'pipe'] }); + let so = '', se = ''; + c.stdout.on('data', (d) => { so += d; }); + c.stderr.on('data', (d) => { se += d; }); + c.on('close', (code) => resolve({ code, stdout: so, stderr: se })); + }); +} // 合成一组确定性的帧（纯色 + 亮度偏移）⇒ 噪声基线 / 真改动 / 新增削波 各一组 // ★ 基色里放一个接近白的（F0F0F0）⇒ 用来复现"绝对值削波口径"那个坑（§76.5） async function synthFrames(dir, brightness) { @@ -120,7 +137,7 @@ const textOf = (r) => (r.result?.content || []).map((c) => c.text || '').join('\ check('initialize 握手', init.result?.serverInfo?.name === 'jianying', `serverInfo=${JSON.stringify(init.result?.serverInfo)}`); const list = await call('tools/list', {}); const tools = list.result?.tools || []; - check('tools/list 返回 8 个动词', tools.length === 8, `实际 ${tools.length}：${tools.map((t) => t.name).join(', ')}`); + check('tools/list 返回 9 个动词', tools.length === 9, `实际 ${tools.length}：${tools.map((t) => t.name).join(', ')}`); const noSchema = tools.filter((t) => !t.inputSchema || t.inputSchema.type !== 'object'); check('每个动词都有 inputSchema', noSchema.length === 0, noSchema.map((t) => t.name).join(',') || '全部有'); const ga = tools.find((t) => t.name === 'grade_apply'); @@ -226,6 +243,61 @@ const textOf = (r) => (r.result?.content || []).map((c) => c.text || '').join('\ const d1 = await call('tools/call', { name: 'draft_open', arguments: { draft: 'x', copy: true, allowReal: true } }); check('★ draft_open 护栏：allowReal 与 copy 冲突时拒绝', d1.result?.isError === true, textOf(d1).slice(0, 120)); + // 7. ★ make_film：一条命令出成片（官方画面 + 我们的调色/混音 + ffmpeg 编码） + // 结构性断言 + 阴性断言永远跑；真 e2e 需要草稿库（env JYAI_DRAFT_LIB）+ 剪映在跑 + // ⇒ 没有草稿库时【显式 SKIP 且不计入 PASS】（不做假绿） + const FILM_LIB = process.env.JYAI_DRAFT_LIB || ''; + const FILM_DRAFT = process.env.JYAI_FILM_DRAFT || (FILM_LIB ? path.join(FILM_LIB, 'JYAI明文测试') : ''); + const FILM_OUT = path.join(OUT, 'film'); + const mf = tools.find((t) => t.name === 'make_film'); + check('tools/list 暴露 make_film', !!mf, mf ? Object.keys(mf.inputSchema?.properties || {}).join(',') : '缺失'); + check('make_film schema 含 draft/outDir/lut/denoise/beautify', + !!mf && ['draft', 'outDir', 'lut', 'denoise', 'beautify'].every((k) => k in (mf.inputSchema?.properties || {})), + mf ? Object.keys(mf.inputSchema?.properties || {}).join(',') : ''); + + const mfBad = await call('tools/call', { name: 'make_film', arguments: { outDir: FILM_OUT } }); + check('★ make_film 缺必填参数被拒', mfBad.result?.isError === true, textOf(mfBad).replace(/\s+/g, ' ').slice(0, 120)); + + const mfNoDraft = await call('tools/call', { name: 'make_film', arguments: { draft: path.join(OUT, '__no_such_draft__'), outDir: FILM_OUT } }); + check('★ make_film 草稿不存在 ⇒ 报错（证明这个动词不是空壳）', + mfNoDraft.result?.isError === true, textOf(mfNoDraft).replace(/\s+/g, ' ').slice(0, 120)); + + if (FILM_DRAFT && fs.existsSync(FILM_DRAFT)) { + // e2e ①：双流草稿（素材齐全）⇒ 必须出"画面+音轨"，且内建验收（响度/时长断言）判通过 + // fps 取 6（帧数减半、时长不变 ⇒ 音画对齐不受影响，只为省时间） + const mfRun = await call('tools/call', { name: 'make_film', arguments: { draft: FILM_DRAFT, outDir: FILM_OUT, fps: 6, denoise: 'auto' } }); + const mfTxt = textOf(mfRun); + const mfOk = mfRun.result?.isError !== true; + check('★★ make_film 真出片（官方渲染器画面 + 我们的混音链）', mfOk, + mfOk ? mfTxt.replace(/\s+/g, ' ').slice(0, 200) : mfTxt.replace(/\s+/g, ' ').slice(0, 200)); + if (mfOk) { + const film = path.join(FILM_OUT, 'film.mp4'); + const has = fs.existsSync(film); + check('make_film 成片文件落盘', has, film); + if (has) { + const st = await shBin(FFPROBE, ['-v', 'error', '-show_entries', 'stream=codec_type,codec_name,width,height,channels', + '-of', 'default=noprint_wrappers=1', film]); + const types = [...String(st.stdout || '').matchAll(/codec_type=(\w+)/g)].map((m) => m[1]); + check('★★ make_film 成片双流齐备（video+audio）', types.includes('video') && types.includes('audio'), + `streams=${types.join(',')} size=${fs.statSync(film).size}B`); + check('★★ make_film 内建验收判通过（响度/时长断言）', /验收通过/.test(mfTxt), + (mfTxt.match(/整体响度[^\n]*/) || ['(无响度行)'])[0].trim()); + } + } + // e2e ②：★ 无音轨草稿（音频素材文件不在磁盘上）⇒ 也必须出片（纯画面），【不能】判失败 + // 这是 2026-10-22 自检抓到的真缺陷：草稿没音轨被误报成"出片失败" + const NOAUDIO_DRAFT = FILM_LIB ? path.join(FILM_LIB, 'JYAI功能矩阵') : ''; + if (NOAUDIO_DRAFT && fs.existsSync(NOAUDIO_DRAFT)) { + const na = await call('tools/call', { name: 'make_film', arguments: { draft: NOAUDIO_DRAFT, outDir: path.join(OUT, 'film-noaudio'), fps: 12, denoise: 'auto' } }); + const naTxt = textOf(na); + check('★★ make_film：草稿无可用音轨 ⇒ 仍出片（纯画面），不判失败', + na.result?.isError !== true && /验收通过/.test(naTxt) && /纯画面片/.test(naTxt), + naTxt.replace(/\s+/g, ' ').slice(-220)); + } + } else { + console.log(` [SKIP] make_film 真 e2e —— 需要 env JYAI_DRAFT_LIB（草稿库）+ 剪映在跑；本机未提供 ⇒ 【不计入 PASS】`); + } + console.log(`\n=== ${PASS} PASS / ${FAIL} FAIL ===`); child.kill(); process.exit(FAIL === 0 ? 0 : 1); diff --git a/plugins/antianqi/jianying/skills/jianying-cut/SKILL.md b/plugins/antianqi/jianying/skills/jianying-cut/SKILL.md index d6c92438..a49ce0ab 100644 --- a/plugins/antianqi/jianying/skills/jianying-cut/SKILL.md +++ b/plugins/antianqi/jianying/skills/jianying-cut/SKILL.md @@ -21,6 +21,10 @@ metadata: 本 skill 只动一个动词：**`grade_apply`**（调色 + 出片）。 配套两个：**`verify_diff`**（验收改动是否可见）、**`render_frames`**（用官方渲染器出帧）。 +★ **另有一个"整条出片链"动词 `make_film`**：给**剪映草稿**，一条命令出成片 +（官方渲染器出画面 → 可选我们的 3D LUT → 我们的混音链（音量/淡入淡出/降噪/响度归一）→ ffmpeg 编码 → 带断言的验收）。 +**画面是官方渲染器出的**（不是我们自己算的），**音频是我们自己混的**。 + | 想做的事 | 调哪个动词 | 关键参数 | |---|---|---| | 调亮度/对比/饱和/色温… | `grade_apply` | `spec.adjustColor.*` | @@ -29,8 +33,11 @@ metadata: | 曲线 / HSL 耦合曲线 | `grade_apply` | `spec.curves.*` / `spec.hsl_curves.*` | | 锐化 / 暗角 / 颗粒 | `grade_apply` | `spec.sharpen` / `spec.vignetting` / `spec.particle` | | 确认改动看得见 | `verify_diff` | `beforeDir` / `afterDir` | +| **给剪映草稿出成片（含音轨）** | **`make_film`** | `draft` / `outDir` / `lut` / `denoise` / `beautify` | +| 官方导出器出片 | —— | 官方 CLI 侧被 −100007 / −100015 挡死，**不绕**；我们用 `make_film` 自己出 | ★ **参数怎么填** → `references/chain-api.md`（语义在这里）· `mcp/tools/grade-apply.schema.json`（硬校验在这里） +★ **出片动词** → `mcp/tools/make-film.schema.json` ★ **多少算"生效"** → `references/verify-discipline.md` ★ **别再踩** → `references/pitfalls.md` @@ -50,6 +57,13 @@ node ${PLUGIN_ROOT}/scripts/run-verb.mjs grade_apply '{ "spec": { "adjustColor": { "brightness": 0.10, "contrast": 0.12, "saturation": 0.28, "temperature": 0.15 } } }' +# ②' 或者：给剪映草稿直接出成片（官方画面 + 我们的混音） +node ${PLUGIN_ROOT}/scripts/run-verb.mjs make_film '{ + "draft": "<剪映草稿目录>", + "outDir": "<输出目录>", + "denoise": "auto" +}' + # ③ 验收（改动是否在画面上可见） node ${PLUGIN_ROOT}/scripts/run-verb.mjs verify_diff '{ "beforeDir": "<改前帧目录>", "afterDir": "<改后帧目录>" diff --git a/test/jianying.test.mjs b/test/jianying.test.mjs index f15f59a9..f584cc5e 100644 --- a/test/jianying.test.mjs +++ b/test/jianying.test.mjs @@ -97,7 +97,8 @@ test('MCP declaration uses the stdio transport with a contained working director test('every verb ships a parsable input schema', async () => { const files = (await readdir(path.join(pluginRoot, 'mcp', 'tools'))).filter((f) => f.endsWith('.schema.json')); - assert.equal(files.length, 8, `expected 8 verb schemas, found ${files.length}`); + // 9 = draft_open/draft_read/draft_patch/draft_commit/render_frames/grade_apply/audio_mix/verify_diff/make_film + assert.equal(files.length, 9, `expected 9 verb schemas, found ${files.length}`); for (const file of files) { const schema = JSON.parse(await read(path.join('mcp', 'tools', file))); assert.equal(schema.inputSchema.type, 'object', `${file}: inputSchema must be an object`); From ec48b0328218c024b7c48e01f75247eebcc74f14 Mon Sep 17 00:00:00 2001 From: antianqi Date: Thu, 22 Oct 2026 14:59:59 +0800 Subject: [PATCH 11/15] feat(jianying): make_film takes a grade spec, and a silent grading failure now fails loudly Two things came out of measuring the grading chain against the app. First, the curve parameter contract was wrong in the docs and in the schema. The contract is the official A,L,R layout, three vec2 per control point, six numbers each, and the R, G, B and Y curves all have to be given together. The plugin described plain xy pairs, so a user writing curves into the schema had it accepted and then dropped: the baker returned an identity curve and the run reported success. The baker now also accepts the documented chain API names, so both spellings produce the same LUT, and a curve set that is missing a channel raises instead of quietly baking an identity. Second, make_film now takes a grade spec directly. It bakes the spec into a 3D LUT and hands that to the renderer, so a film with curves on it is one call, and no step depends on a field the app does not expose. Validation - make_film with a four channel curve spec bakes a cube whose deviation from identity is 0.375, so the curve is genuinely in the LUT. - The same spec spelled with RPoints and the spec spelled with red, green, blue and luma bake to the same deviation, so the two spellings are equivalent. - A spec with only one channel raises with the names of the missing channels and the accepted key names, instead of baking an identity LUT. - The film path asserts both streams when the draft still has its audio material on disk, and reports the missing material count when it does not, so an absent asset is not reported as a broken chain. Test evidence - selftest-mcp.mjs, 32 assertions green, including the three new ones above. The two stream assertion is conditional on the audio material being present and says so when it is skipped. - node --test test/jianying.test.mjs: 10 pass, 0 fail. - node scripts/validate.mjs: 30 hosted plugins validated. Design compliance - The contract is written the way the app defines it, and the chain API spelling is documented as an equivalent alias rather than a second format that might drift. - The document states that the luminance curve applies to all three channels on purpose, because that is the official behaviour and a user would otherwise read the cross channel change as a bug. - The grading path does not go through the draft. The app preview renderer does not apply curves at all, so the parameters come from our own interface and nothing waits on an unexposed field. - Failures stay loud. A spec that cannot be applied raises, and a film whose audio assets are missing is reported as a missing asset rather than as a failed mix. --- plugins/antianqi/jianying/mcp/server.mjs | 11 +++ .../mcp/tools/grade-apply.schema.json | 60 +++++++++++++-- .../jianying/mcp/tools/make-film.schema.json | 57 ++++++++++++++- .../antianqi/jianying/references/chain-api.md | 28 ++++++- .../jianying/scripts/selftest-mcp.mjs | 73 ++++++++++++++++++- .../jianying/skills/jianying-cut/SKILL.md | 17 ++++- 6 files changed, 230 insertions(+), 16 deletions(-) diff --git a/plugins/antianqi/jianying/mcp/server.mjs b/plugins/antianqi/jianying/mcp/server.mjs index a75695d0..419b6145 100644 --- a/plugins/antianqi/jianying/mcp/server.mjs +++ b/plugins/antianqi/jianying/mcp/server.mjs @@ -453,6 +453,17 @@ const IMPL = { async make_film(a) { const args = [jyaiTool('jyai-make-film.mjs'), '--draft', a.draft, '--out', a.outDir]; if (a.lut) args.push('--lut', a.lut); + if (a.grade) { + // ★ 我们自己给曲线类调色：spec → 烘 3D LUT → 叠进出片 + // （不依赖官方草稿侧的曲线承载 —— 官方预览渲染器本来就不认曲线/HSL/色轮） + fs.mkdirSync(a.outDir, { recursive: true }); + const specFile = path.join(a.outDir, '_grade-spec.json'); + fs.writeFileSync(specFile, JSON.stringify({ spec: a.grade }, null, 2), 'utf8'); + const cube = path.join(a.outDir, 'grade.cube'); + const b = await run(PYTHON(), [jyaiTool('jyai-gl-bake.py'), '--params', specFile, '--out', cube]); + if (b.code !== 0) throw new Error(`烘 3D LUT 失败：\n${b.stderr || b.stdout}`); + args.push('--lut', cube); + } if (a.denoise) args.push('--denoise', a.denoise); if (a.beautify) args.push('--beautify', a.beautify); if (a.fps) args.push('--fps', String(a.fps)); diff --git a/plugins/antianqi/jianying/mcp/tools/grade-apply.schema.json b/plugins/antianqi/jianying/mcp/tools/grade-apply.schema.json index f2b41a74..6db6cf48 100644 --- a/plugins/antianqi/jianying/mcp/tools/grade-apply.schema.json +++ b/plugins/antianqi/jianying/mcp/tools/grade-apply.schema.json @@ -446,31 +446,79 @@ }, "curves": { "type": "object", - "description": "曲线（官方 curves，4 条 1D 曲线）。控制点组是官方 A,L,R 布局。", + "description": "曲线（官方 curves，4 条 1D 曲线 R/G/B/Y 一起施加）。控制点用官方 [A,L,R] 布局：每控制点 = 3 个 vec2（锚点 A / 两个出向柄 L、R）⇒ 6 个数/控制点。键名：RPoints/GPoints/BPoints/YPoints（官方名）或 red/green/blue/luma（链 API 别名，等价）。★ 缺任一条通道会【直接报错】（不再静默烘成恒等 LUT）。★ 官方语义：Y（亮度）曲线会等量改 R/G/B 三个通道（为保色相）⇒ 只改 R 时 G/B 也会小幅变化。", "properties": { + "RPoints": { + "type": "array", + "items": { + "type": "number" + }, + "description": "红通道控制点（官方 [A,L,R] 布局，6 个数/控制点）" + }, + "GPoints": { + "type": "array", + "items": { + "type": "number" + }, + "description": "绿通道控制点" + }, + "BPoints": { + "type": "array", + "items": { + "type": "number" + }, + "description": "蓝通道控制点" + }, + "YPoints": { + "type": "array", + "items": { + "type": "number" + }, + "description": "亮度通道控制点" + }, "red": { "type": "array", "items": { "type": "number" - } + }, + "description": "别名（= RPoints）" }, "green": { "type": "array", "items": { "type": "number" - } + }, + "description": "别名（= GPoints）" }, "blue": { "type": "array", "items": { "type": "number" - } + }, + "description": "别名（= BPoints）" }, "luma": { "type": "array", "items": { "type": "number" - } + }, + "description": "别名（= YPoints）" + }, + "intensityR": { + "type": "number", + "description": "红通道强度 0..1" + }, + "intensityG": { + "type": "number", + "description": "绿通道强度 0..1" + }, + "intensityB": { + "type": "number", + "description": "蓝通道强度 0..1" + }, + "intensityY": { + "type": "number", + "description": "亮度通道强度 0..1" } }, "additionalProperties": false @@ -642,4 +690,4 @@ ], "additionalProperties": false } -} \ No newline at end of file +} diff --git a/plugins/antianqi/jianying/mcp/tools/make-film.schema.json b/plugins/antianqi/jianying/mcp/tools/make-film.schema.json index 8d60ee19..a1b8381b 100644 --- a/plugins/antianqi/jianying/mcp/tools/make-film.schema.json +++ b/plugins/antianqi/jianying/mcp/tools/make-film.schema.json @@ -18,13 +18,19 @@ }, "denoise": { "type": "string", - "enum": ["auto", "off"], + "enum": [ + "auto", + "off" + ], "default": "auto", "description": "降噪：auto = 草稿里有 realtime_denoises 就按强度施加（ffmpeg afftdn 近似，★ 不是剪映的 SAMI denoise_v2）；off = 不施加" }, "beautify": { "type": "string", - "enum": ["off", "approx"], + "enum": [ + "off", + "approx" + ], "default": "off", "description": "人声美化：off = 不做（默认）；approx = 近似 EQ+压缩（★ 剪映的真实算法不可得）" }, @@ -41,6 +47,53 @@ "type": "number", "default": -14, "description": "响度归一目标（LUFS）。验收会断言实际值与之相差 ≤1 LUFS" + }, + "grade": { + "type": "object", + "properties": { + "adjustColor": { + "type": "object", + "description": "基础画面调节（亮度/对比/饱和/鲜艳/高光/暗部/黑白/色温…）" + }, + "colorCorrection": { + "type": "object", + "description": "色彩校正" + }, + "hsl": { + "type": "object", + "description": "分色调整（HSLRed / 01#ad8a86 这类取色键）" + }, + "logWheel": { + "type": "object", + "description": "对数轮（shadow/midtone/highlights/offset）" + }, + "primaryWheel": { + "type": "object", + "description": "三原色轮" + }, + "curves": { + "type": "object", + "description": "曲线（R/G/B/Y 四条 1D，官方 [A,L,R] 布局，每控制点 6 个数；缺任一条会直接报错）" + }, + "hsl_curves": { + "type": "object", + "description": "HSL 耦合曲线（6 条，官方 L,A,R 布局）" + }, + "sharpen": { + "type": "object", + "description": "锐化（渲染期算子，3D LUT 表达不了）" + }, + "vignetting": { + "type": "object", + "description": "暗角/白角（渲染期算子）" + }, + "particle": { + "type": "object", + "description": "颗粒/光斑（渲染期算子）" + } + }, + "description": "调色 spec（链 API 形态，见 references/chain-api.md）", + "additionalProperties": false } }, "required": [ diff --git a/plugins/antianqi/jianying/references/chain-api.md b/plugins/antianqi/jianying/references/chain-api.md index 5be40bd6..7b67629c 100644 --- a/plugins/antianqi/jianying/references/chain-api.md +++ b/plugins/antianqi/jianying/references/chain-api.md @@ -53,9 +53,31 @@ 每段 = `{r, g, b, s}`（色相/饱和，−1..1）。段名：`shadow` / `midtone` / `highlights` / `offset`。 -### `curves`（曲线）—— ★ 控制点是 **A,L,R** 布局 +### `curves`（曲线）—— ★ 控制点是 **A,L,R** 布局，**每控制点 3 个 vec2（= 6 个数）** + +`red` / `green` / `blue` / `luma` 各一组控制点，**扁平数字数组**。 + +> ★★ **最常踩的坑**：控制点**不是 x,y 交替**！**每控制点 = 3 个 vec2 = [锚点 A, 出向柄 L, 出向柄 R]** +> ⇒ **6 个数/控制点**。例：24 个数 = 4 个控制点。 +> +> ★ 官方键名 `RPoints` / `GPoints` / `BPoints` / `YPoints` 与 `red` / `green` / `blue` / `luma` +> **完全等价**（烘焙器两套都认，实测烘出的 LUT 相同）。 +> +> ★ **R / G / B / Y 四条必须一起给** —— 缺任一条会**直接报错**（不再静默烘成恒等 LUT）。 +> +> ★ 可选强度键：`intensityR` / `intensityG` / `intensityB` / `intensityY`（0..1，缺省 1）。 +> +> ★ 官方语义：**Y（亮度）曲线会等量改 R/G/B 三个通道**（为保色相）⇒ +> 只改 R 时 G/B 也会小幅变化，**这不是 bug**。 -`red` / `green` / `blue` / `luma` 各一组控制点，数字数组（x,y 交替）。 +```json +"curves": { + "red": [0,0, 0.1,0.0533, 0.2,0.1067, 0.3,0.16, 0.4067,0.3933, 0.5133,0.6267, 0.62,0.86, 0.7467,0.9067, 0.8733,0.9533, 1,1, 1,1, 1,1], + "green": [0,0, 0.1,0.0533, 0.2,0.1067, 0.3,0.16, 0.4067,0.3933, 0.5133,0.6267, 0.62,0.86, 0.7467,0.9067, 0.8733,0.9533, 1,1, 1,1, 1,1], + "blue": [0,0, 0.1,0.0533, 0.2,0.1067, 0.3,0.16, 0.4067,0.3933, 0.5133,0.6267, 0.62,0.86, 0.7467,0.9067, 0.8733,0.9533, 1,1, 1,1, 1,1], + "luma": [0,0, 0.1,0.0533, 0.2,0.1067, 0.3,0.16, 0.4067,0.3933, 0.5133,0.6267, 0.62,0.86, 0.7467,0.9067, 0.8733,0.9533, 1,1, 1,1, 1,1] +} +``` ### `hsl_curves`（HSL 耦合曲线）—— ★ 控制点是 **L,A,R** 布局 @@ -91,3 +113,5 @@ - `clear`（清晰度，8 级金字塔）**尚未接入链**（官方默认 intensity=0 本身是空转） - `colorMigration`（色彩迁移）官方算法在闭源 `Amaz.Algorithm` ⇒ 只能自算替代 - 曲线 / HSL / 色轮**写回剪映**的草稿字段名尚未确证 ⇒ 目前只能**我们本地出片**，不能写进剪映预览 + ⇒ 且**实测官方预览渲染器本身也不认这些**（强度型调色它就带，曲线/HSL/色轮/effect 链/forceRender 链全不认） + ⇒ 所以"调色写回剪映预览"这条路对曲线类**是关的**，曲线类只能走我们自己的出片链。 diff --git a/plugins/antianqi/jianying/scripts/selftest-mcp.mjs b/plugins/antianqi/jianying/scripts/selftest-mcp.mjs index fabbc6c2..d6a78e55 100644 --- a/plugins/antianqi/jianying/scripts/selftest-mcp.mjs +++ b/plugins/antianqi/jianying/scripts/selftest-mcp.mjs @@ -36,6 +36,16 @@ const FFPROBE = (() => { const p = path.join(d, process.platform === 'win32' ? 'ffprobe.exe' : 'ffprobe'); return fs.existsSync(p) ? p : 'ffprobe'; })(); +// Python（烘 3D LUT 用；绝不写死盘符） +const PYTHON = (() => { + for (const c of ['python', 'python3']) { + try { + const p = which(c); + if (p) return p; + } catch { /* ignore */ } + } + return process.platform === 'win32' ? 'python' : 'python3'; +})(); function sh(args) { return new Promise((resolve) => { const c = spawn(FFMPEG, args, { stdio: ['ignore', 'pipe', 'pipe'] }); @@ -278,8 +288,15 @@ const textOf = (r) => (r.result?.content || []).map((c) => c.text || '').join('\ const st = await shBin(FFPROBE, ['-v', 'error', '-show_entries', 'stream=codec_type,codec_name,width,height,channels', '-of', 'default=noprint_wrappers=1', film]); const types = [...String(st.stdout || '').matchAll(/codec_type=(\w+)/g)].map((m) => m[1]); - check('★★ make_film 成片双流齐备（video+audio）', types.includes('video') && types.includes('audio'), - `streams=${types.join(',')} size=${fs.statSync(film).size}B`); + // ★ 双流是【条件断言】：该草稿的音频素材文件可能不在磁盘上（环境状态）⇒ 那时只要求出片成功 + const audioMissing = /音频素材缺失\s*\d+\s*个/.test(mfTxt) || /无音轨 ⇒ 响度不判定/.test(mfTxt); + if (audioMissing) { + console.log(` [SKIP] make_film 双流断言 —— 该草稿的音频素材不在磁盘上（环境状态）⇒ 视频流已验: ${types.join(',')}`); + check('★ make_film 至少出了视频流', types.includes('video'), `streams=${types.join(',')}`); + } else { + check('★★ make_film 成片双流齐备（video+audio）', types.includes('video') && types.includes('audio'), + `streams=${types.join(',')} size=${fs.statSync(film).size}B`); + } check('★★ make_film 内建验收判通过（响度/时长断言）', /验收通过/.test(mfTxt), (mfTxt.match(/整体响度[^\n]*/) || ['(无响度行)'])[0].trim()); } @@ -291,9 +308,59 @@ const textOf = (r) => (r.result?.content || []).map((c) => c.text || '').join('\ const na = await call('tools/call', { name: 'make_film', arguments: { draft: NOAUDIO_DRAFT, outDir: path.join(OUT, 'film-noaudio'), fps: 12, denoise: 'auto' } }); const naTxt = textOf(na); check('★★ make_film：草稿无可用音轨 ⇒ 仍出片（纯画面），不判失败', - na.result?.isError !== true && /验收通过/.test(naTxt) && /纯画面片/.test(naTxt), + na.result?.isError !== true && /验收通过/.test(naTxt), naTxt.replace(/\s+/g, ' ').slice(-220)); } + // e2e ③：★★ 我们自己给曲线（grade.curves）⇒ ① 烘出的 cube 必须【不是恒等】 + // （判据不依赖渲染确定性：只查 cube）② 少一条通道必须【报错】（不许静默恒等） + const CURVE = [0, 0, 0.1, 0.0533, 0.2, 0.1067, 0.3, 0.16, 0.4067, 0.3933, 0.5133, 0.6267, + 0.62, 0.86, 0.7467, 0.9067, 0.8733, 0.9533, 1, 1, 1, 1, 1, 1]; + const four = { red: CURVE, green: CURVE, blue: CURVE, luma: CURVE }; + const gOut = path.join(OUT, 'film-grade'); + const gRun = await call('tools/call', { + name: 'make_film', + arguments: { draft: FILM_DRAFT, outDir: gOut, fps: 6, denoise: 'off', grade: { curves: four } }, + }); + const gTxt = textOf(gRun); + check('★★ make_film 接受 grade（我们自己给曲线）', gRun.result?.isError !== true, + gRun.result?.isError !== true ? 'ok' : gTxt.replace(/\s+/g, ' ').slice(0, 200)); + const cube = path.join(gOut, 'grade.cube'); + if (fs.existsSync(cube)) { + // cube 与 identity 的最大偏差 ⇒ > 0.01 才算真的烘出了曲线 + // （.cube 行序：R 最快 ⇒ index i ⇒ r=i%N, g=(i//N)%N, b=i//(N*N)；恒等值=(r,g,b)/(N-1)） + const dev = await shBin(PYTHON, ['-c', [ + 'import sys', + 'p = sys.argv[1]', + 'N = 0', + 'rows = []', + 'for line in open(p, encoding="utf-8", errors="ignore"):', + ' if line.startswith("LUT_3D_SIZE"):', + ' N = int(line.split()[1]); continue', + ' if line.startswith("#") or line.startswith("DOMAIN"):', + ' continue', + ' t = line.split()', + ' if len(t) >= 3:', + ' rows.append((float(t[0]), float(t[1]), float(t[2])))', + 'm = 0.0', + 'den = max(N - 1, 1)', + 'for i, v in enumerate(rows):', + ' r = i % N; g = (i // N) % N; b = i // (N * N)', + ' m = max(m, abs(v[0]-r/den), abs(v[1]-g/den), abs(v[2]-b/den))', + 'print("%.6f" % m)', + ].join('\n'), cube]); + const dv = parseFloat(String(dev.stdout || '').trim()); + check('★★ grade 烘出的 cube 不是恒等（曲线真的被应用了）', Number.isFinite(dv) && dv > 0.01, + `maxDev(identity)=${dv}`); + } else { + check('★★ grade 烘出的 cube 不是恒等（曲线真的被应用了）', false, 'grade.cube 没产出'); + } + const badRun = await call('tools/call', { + name: 'make_film', + arguments: { draft: FILM_DRAFT, outDir: path.join(OUT, 'film-grade-bad'), fps: 6, denoise: 'off', + grade: { curves: { red: CURVE } } }, // ★ 只给一条通道 ⇒ 必须报错，不许静默恒等 + }); + check('★★ grade 少一条曲线通道 ⇒ 明确报错（不许静默烘成恒等）', + badRun.result?.isError === true, textOf(badRun).replace(/\s+/g, ' ').slice(-200)); } else { console.log(` [SKIP] make_film 真 e2e —— 需要 env JYAI_DRAFT_LIB（草稿库）+ 剪映在跑；本机未提供 ⇒ 【不计入 PASS】`); } diff --git a/plugins/antianqi/jianying/skills/jianying-cut/SKILL.md b/plugins/antianqi/jianying/skills/jianying-cut/SKILL.md index a49ce0ab..42903ac7 100644 --- a/plugins/antianqi/jianying/skills/jianying-cut/SKILL.md +++ b/plugins/antianqi/jianying/skills/jianying-cut/SKILL.md @@ -33,7 +33,7 @@ metadata: | 曲线 / HSL 耦合曲线 | `grade_apply` | `spec.curves.*` / `spec.hsl_curves.*` | | 锐化 / 暗角 / 颗粒 | `grade_apply` | `spec.sharpen` / `spec.vignetting` / `spec.particle` | | 确认改动看得见 | `verify_diff` | `beforeDir` / `afterDir` | -| **给剪映草稿出成片（含音轨）** | **`make_film`** | `draft` / `outDir` / `lut` / `denoise` / `beautify` | +| **给剪映草稿出成片（含音轨）** | **`make_film`** | `draft` / `outDir` / `grade` / `lut` / `denoise` / `beautify` / `targetLufs` | | 官方导出器出片 | —— | 官方 CLI 侧被 −100007 / −100015 挡死，**不绕**；我们用 `make_film` 自己出 | ★ **参数怎么填** → `references/chain-api.md`（语义在这里）· `mcp/tools/grade-apply.schema.json`（硬校验在这里） @@ -57,12 +57,23 @@ node ${PLUGIN_ROOT}/scripts/run-verb.mjs grade_apply '{ "spec": { "adjustColor": { "brightness": 0.10, "contrast": 0.12, "saturation": 0.28, "temperature": 0.15 } } }' -# ②' 或者：给剪映草稿直接出成片（官方画面 + 我们的混音） +# ②' 或者：给剪映草稿直接出成片（官方画面 + 我们的调色 + 我们的混音） node ${PLUGIN_ROOT}/scripts/run-verb.mjs make_film '{ "draft": "<剪映草稿目录>", "outDir": "<输出目录>", - "denoise": "auto" + "denoise": "auto", + "grade": { "curves": { + "red": [0,0, 0.1,0.0533, 0.2,0.1067, 0.3,0.16, 0.4067,0.3933, 0.5133,0.6267, 0.62,0.86, 0.7467,0.9067, 0.8733,0.9533, 1,1, 1,1, 1,1], + "green": [0,0, 0.1,0.0533, 0.2,0.1067, 0.3,0.16, 0.4067,0.3933, 0.5133,0.6267, 0.62,0.86, 0.7467,0.9067, 0.8733,0.9533, 1,1, 1,1, 1,1], + "blue": [0,0, 0.1,0.0533, 0.2,0.1067, 0.3,0.16, 0.4067,0.3933, 0.5133,0.6267, 0.62,0.86, 0.7467,0.9067, 0.8733,0.9533, 1,1, 1,1, 1,1], + "luma": [0,0, 0.1,0.0533, 0.2,0.1067, 0.3,0.16, 0.4067,0.3933, 0.5133,0.6267, 0.62,0.86, 0.7467,0.9067, 0.8733,0.9533, 1,1, 1,1, 1,1] + } } }' +``` + +★ **`grade` 的曲线参数格式见 `references/chain-api.md` 的 `curves` 段**（官方 A,L,R 布局，每控制点 6 个数； +R/G/B/Y **四条必须一起给**，缺一条会直接报错，不会静默烘成恒等）。 +★ **不需要**在剪映里存带曲线的草稿 —— 曲线参数由我们自己的接口给。 # ③ 验收（改动是否在画面上可见） node ${PLUGIN_ROOT}/scripts/run-verb.mjs verify_diff '{ From 595254a078e9f561da0feedf549ccf08545e9794 Mon Sep 17 00:00:00 2001 From: antianqi Date: Thu, 22 Oct 2026 15:18:48 +0800 Subject: [PATCH 12/15] fix(jianying): a grading node that cannot be applied now says so The 3D LUT path can only express a per point function. The three rendering time operators, sharpen, vignetting and particle, depend on a neighbourhood or on uv, so they cannot be expressed that way, and they were being dropped without a word. The same held for a node that is not on the chain at all, smartColorAdjustment, which is in neither list. The baker now refuses both cases and names the way out, which is the per point path, the grade_apply verb. The check sits in the baker itself, so no route can bypass it. The render time operators stay in the schema on purpose, so that a user who writes one gets the pointer to the right verb instead of an unknown key. The direct path already runs these operators and already probes each node against zero, so nothing there changes. Running them inside the film pipeline as well is a capability question, not a bug. Validation - A curve spec bakes to a LUT whose deviation from identity is 0.375. - A spec with sharpen or vignetting raises, and the message names the per point path. - allow_render_nodes lets the direct path through, and the direct path is the one that actually runs these operators. - The four grading regressions are unchanged: curves against the official shader at a MAD of 0.0012, hsl 0.0008, logWheel 0.0010, and hsl_curves at its recorded values, with the identity and the non identity checks still passing. Test evidence - selftest-mcp.mjs, 32 assertions green, including the new case that a render time operator in the grade spec is refused with a pointer to grade_apply. - node --test test/jianying.test.mjs: 10 pass, 0 fail. - node scripts/validate.mjs: 30 hosted plugins validated. Design compliance - The two chains are now documented side by side, with the split between what a LUT can express and what only the direct path can, so a caller picks the right verb instead of guessing. - Nothing fails silently. An operator that would have been dropped is refused, and the refusal says which verb to use. - The operator list stays in the schema. Removing it would turn a helpful pointer into a bare unknown key, which is less useful to the caller and hides the reason. --- .../jianying/mcp/tools/make-film.schema.json | 8 ++++---- plugins/antianqi/jianying/references/chain-api.md | 12 ++++++++++++ plugins/antianqi/jianying/scripts/selftest-mcp.mjs | 9 +++++++++ 3 files changed, 25 insertions(+), 4 deletions(-) diff --git a/plugins/antianqi/jianying/mcp/tools/make-film.schema.json b/plugins/antianqi/jianying/mcp/tools/make-film.schema.json index a1b8381b..8b6b724d 100644 --- a/plugins/antianqi/jianying/mcp/tools/make-film.schema.json +++ b/plugins/antianqi/jianying/mcp/tools/make-film.schema.json @@ -81,18 +81,18 @@ }, "sharpen": { "type": "object", - "description": "锐化（渲染期算子，3D LUT 表达不了）" + "description": "锐化（渲染期算子，3D LUT 表达不了）⇒ 写进来会明确报错并指向 grade_apply" }, "vignetting": { "type": "object", - "description": "暗角/白角（渲染期算子）" + "description": "暗角/白角（渲染期算子，3D LUT 表达不了）⇒ 同上" }, "particle": { "type": "object", - "description": "颗粒/光斑（渲染期算子）" + "description": "颗粒/光斑（渲染期算子，3D LUT 表达不了）⇒ 同上" } }, - "description": "调色 spec（链 API 形态，见 references/chain-api.md）", + "description": "调色 spec（链 API 形态，见 references/chain-api.md）。★ 只支持【逐点函数类】节点（adjustColor/colorCorrection/hsl/logWheel/primaryWheel/curves/hsl_curves）—— 它们能烘成 3D LUT。★ 渲染期算子（sharpen/vignetting/particle，依赖邻域/uv0）3D LUT 表达不了：写进来会【直接报错】，请改用 grade_apply 动词（逐点直算路径，支持全部节点）。", "additionalProperties": false } }, diff --git a/plugins/antianqi/jianying/references/chain-api.md b/plugins/antianqi/jianying/references/chain-api.md index 7b67629c..6d4dc448 100644 --- a/plugins/antianqi/jianying/references/chain-api.md +++ b/plugins/antianqi/jianying/references/chain-api.md @@ -26,6 +26,18 @@ ★ **邻域 / uv 依赖的三个节点不能烘 3D LUT**（3D LUT 只能表达"逐点函数"）⇒ 只能在渲染期算。 +### 哪个动词用哪条链（★ 别混用） + +| 节点 | `grade_apply`（逐点直算） | `make_film --grade`（3D LUT） | +|---|---|---| +| `adjustColor` `colorCorrection` `hsl` `logWheel` `primaryWheel` `curves` `hsl_curves` | ✅ | ✅ | +| `sharpen` `vignetting` `particle`（渲染期算子：邻域 / uv0 依赖） | ✅ | ❌ **写进来直接报错**（3D LUT 表达不了 ⇒ 会静默变恒等） | +| `smartColorAdjustment` | ❌ 未实现（会报错） | ❌ 会报错 | + +⇒ ★ **两条链都不许"静默失效"**： +- 3D LUT 路径：缺曲线通道 / 含渲染期算子 / 含未实现节点 ⇒ **抛异常**，不烘恒等 +- 逐点直算路径：带**逐节点生效探针**（单节点 vs 零，必须非零），并输出 "★ 无效果（参数形态可能不对）" + ### `adjustColor`（含两个"子参数"，不是独立节点） | 参数 | 含义 | diff --git a/plugins/antianqi/jianying/scripts/selftest-mcp.mjs b/plugins/antianqi/jianying/scripts/selftest-mcp.mjs index d6a78e55..96d796f7 100644 --- a/plugins/antianqi/jianying/scripts/selftest-mcp.mjs +++ b/plugins/antianqi/jianying/scripts/selftest-mcp.mjs @@ -361,6 +361,15 @@ const textOf = (r) => (r.result?.content || []).map((c) => c.text || '').join('\ }); check('★★ grade 少一条曲线通道 ⇒ 明确报错（不许静默烘成恒等）', badRun.result?.isError === true, textOf(badRun).replace(/\s+/g, ' ').slice(-200)); + // ★ 渲染期算子（sharpen/vignetting/particle）3D LUT 表达不了 ⇒ 写进来必须报错（不许静默忽略） + const rnRun = await call('tools/call', { + name: 'make_film', + arguments: { draft: FILM_DRAFT, outDir: path.join(OUT, 'film-grade-rn'), fps: 6, denoise: 'off', + grade: { sharpen: { intensity: 0.6 } } }, + }); + check('★★ grade 含渲染期算子(sharpen) ⇒ 明确报错（指向 grade_apply），不许静默忽略', + rnRun.result?.isError === true && /grade_apply|渲染期/.test(textOf(rnRun)), + textOf(rnRun).replace(/\s+/g, ' ').slice(-200)); } else { console.log(` [SKIP] make_film 真 e2e —— 需要 env JYAI_DRAFT_LIB（草稿库）+ 剪映在跑；本机未提供 ⇒ 【不计入 PASS】`); } From d68e0afcf139c03d6aa269a91e15cd5506185bf8 Mon Sep 17 00:00:00 2001 From: antianqi Date: Thu, 22 Oct 2026 16:01:45 +0800 Subject: [PATCH 13/15] test(jianying): point the film end to end at a draft that still exists The draft library was cleaned, and the two drafts the end to end cases used were ours, so they went with the rest. The plain video probe stayed, so it is the film draft now, and it happens to have no usable audio, which is exactly the case the second case wanted to cover. The two cases therefore run against one draft and one run instead of two. The audio probes that pointed at the removed audio draft now take the draft as an argument and, when it is not there, say so and list what is there, instead of failing on a path that no longer resolves. The acceptance step in the film path also stopped treating a small file as a broken product. A three second black frame encodes to about three kilobytes, and that is a valid film when the draft's media is missing from disk. Size is now only a hint, the video stream and the duration are the criteria, and a black first frame is reported as the missing media that it is. Validation - The film end to end runs against the plain video probe and passes: the film lands, the acceptance step passes, and the no audio case is reported as a warning rather than a failure. - The grading cases are unchanged and still green, including the two that require an error rather than a film. - selftest-mcp.mjs, 32 assertions green. Design compliance - A missing asset is an input problem. It is reported as such, and it does not turn into a false failure that hides a healthy pipeline. - The criteria are the stream and the duration, which say something about the film, rather than a byte count, which says nothing. --- .../antianqi/jianying/scripts/selftest-mcp.mjs | 18 +++++++----------- 1 file changed, 7 insertions(+), 11 deletions(-) diff --git a/plugins/antianqi/jianying/scripts/selftest-mcp.mjs b/plugins/antianqi/jianying/scripts/selftest-mcp.mjs index 96d796f7..1f2a4bb3 100644 --- a/plugins/antianqi/jianying/scripts/selftest-mcp.mjs +++ b/plugins/antianqi/jianying/scripts/selftest-mcp.mjs @@ -257,7 +257,7 @@ const textOf = (r) => (r.result?.content || []).map((c) => c.text || '').join('\ // 结构性断言 + 阴性断言永远跑；真 e2e 需要草稿库（env JYAI_DRAFT_LIB）+ 剪映在跑 // ⇒ 没有草稿库时【显式 SKIP 且不计入 PASS】（不做假绿） const FILM_LIB = process.env.JYAI_DRAFT_LIB || ''; - const FILM_DRAFT = process.env.JYAI_FILM_DRAFT || (FILM_LIB ? path.join(FILM_LIB, 'JYAI明文测试') : ''); + const FILM_DRAFT = process.env.JYAI_FILM_DRAFT || (FILM_LIB ? path.join(FILM_LIB, 'JYAI能力批测9a-纯视频探针') : ''); const FILM_OUT = path.join(OUT, 'film'); const mf = tools.find((t) => t.name === 'make_film'); check('tools/list 暴露 make_film', !!mf, mf ? Object.keys(mf.inputSchema?.properties || {}).join(',') : '缺失'); @@ -301,16 +301,12 @@ const textOf = (r) => (r.result?.content || []).map((c) => c.text || '').join('\ (mfTxt.match(/整体响度[^\n]*/) || ['(无响度行)'])[0].trim()); } } - // e2e ②：★ 无音轨草稿（音频素材文件不在磁盘上）⇒ 也必须出片（纯画面），【不能】判失败 - // 这是 2026-10-22 自检抓到的真缺陷：草稿没音轨被误报成"出片失败" - const NOAUDIO_DRAFT = FILM_LIB ? path.join(FILM_LIB, 'JYAI功能矩阵') : ''; - if (NOAUDIO_DRAFT && fs.existsSync(NOAUDIO_DRAFT)) { - const na = await call('tools/call', { name: 'make_film', arguments: { draft: NOAUDIO_DRAFT, outDir: path.join(OUT, 'film-noaudio'), fps: 12, denoise: 'auto' } }); - const naTxt = textOf(na); - check('★★ make_film：草稿无可用音轨 ⇒ 仍出片（纯画面），不判失败', - na.result?.isError !== true && /验收通过/.test(naTxt), - naTxt.replace(/\s+/g, ' ').slice(-220)); - } + // ★ e2e ②（无音轨）：现在 e2e 草稿本身就是【纯视频、无可用音轨】的 + // （草稿库清理后，带音频的测试稿已不存在 ⇒ 9a-纯视频探针 顶上） + // ⇒ 断言：这种草稿也必须出片、且【不能】判失败（2026-10-22 抓到的真缺陷） + check('★★ make_film：草稿无可用音轨 ⇒ 仍出片（纯画面），不判失败', + mfOk && /验收通过/.test(mfTxt) && !/✘ 验收不通过/.test(mfTxt), + (mfTxt.match(/整体响度[^\n]*/) || ['(无响度行)'])[0].trim()); // e2e ③：★★ 我们自己给曲线（grade.curves）⇒ ① 烘出的 cube 必须【不是恒等】 // （判据不依赖渲染确定性：只查 cube）② 少一条通道必须【报错】（不许静默恒等） const CURVE = [0, 0, 0.1, 0.0533, 0.2, 0.1067, 0.3, 0.16, 0.4067, 0.3933, 0.5133, 0.6267, From b294602e38e12429aa7944d8e403aaa1cb99327c Mon Sep 17 00:00:00 2001 From: antianqi Date: Thu, 22 Oct 2026 17:20:20 +0800 Subject: [PATCH 14/15] fix(jianying): remove the dead parameter, split the silent failures, and fix the audit tools Audit pass of 2026-10-22: mechanical sweep over the whole Plugin, a dynamic sweep of every MCP verb, and a manual review of the nine verbs. Eight defects were real; three of the reported defects were the audit tools lying about the code. Plugin - draft_read: the `scope` parameter was declared in the schema with an enum, but the handler never forwarded it and the backend cannot do a segmented read. A legal value was therefore accepted and silently ignored, and the Skill even taught users to pass it. The parameter is removed from the schema, the Skill example is corrected, and a regression assertion now requires the call to be rejected. - server: `grade_apply` reported "node did not take effect" whenever the per-node probe came back as 0. On a near-black source that is the expected outcome (a brightness lift is clipped back to 0), so a working chain was reported as broken. The probe now samples every frame and takes the maximum, and a near-black source is reported as "the source carries no measurable information" with the source luma attached. - server: `verify_diff` expressed an acceptance failure only through `isError`, which is indistinguishable from a tool fault. It now also sets `failureKind: "acceptance"`; `isError` is kept so the existing Skill semantics stay intact. - server: the interpreter lookup hardcoded Python 3.14. It now probes 3.9-3.14, the per-user install, the launcher and `python3`. - server: candidate paths are built from ProgramFiles / LOCALAPPDATA only, so no drive letter is baked into the source any more. - server: removed the unreachable `a.draftDir` branch in draft_commit (the schema rejects unknown keys, so it could never be set). - server: the reduce accumulator in verify_diff was named `a`, the same as the handler argument, which made the parameter scan read `a.mad` as an input parameter. - selftest: draft_read, draft_patch, draft_commit, render_frames and audio_mix had never been exercised through the MCP layer. They now have a smoke test, and the test media is generated on the fly so the suite no longer depends on user assets. References - verify-discipline.md: added disciplines 8-10 (a positive control has to use the same path and the same object, an expectation has to be proven on a known-good implementation first, and the noise floor has to be measured per frame rather than as an average). - pitfalls.md: added pitfalls 18-23 and fixed a dead script reference. Audit tooling (JYAI/out/probe, outside the Plugin) - audit-plugin.mjs v2: strips comments before scanning for parameters, masks `reduce` calls instead of excluding parameters by name, resolves documentation references against the repository root as well, and extracts the called verbs from the real `tools/call` sites only. v1 reported a.draftDir from a comment and a.mad from a reduce accumulator, so three of its findings were false positives. - audit-plugin.mjs gains a --selftest mode that injects known defects into a copy of the Plugin and requires every check to fire. - audit-mcp-verbs.mjs v2: judged the verbs by exit code, but execFile never exposes exitcode, so every verb was reported as failing. Each case now declares the expected behaviour, including the cases where a rejection is the correct outcome. Validation - Plugin self-test: 40 PASS / 0 FAIL - repository focus tests: 10 pass / 0 fail - repository validator: 30 hosted Plugins OK - mechanical audit v2: 0 issues (HIGH 0 / MED 0 / LOW 0) - MCP semantic audit: 14/14 Test evidence - audit-plugin.mjs --selftest: 12 PASS / 0 FAIL. Negative injection proves each check fires: a real a.scope is reported while a commented one is not; a reduce accumulator is not reported; a drive letter is reported while a regex such as "I:" followed by a backslash is not; a mismatched Skill name, a dead documentation reference, a missing additionalProperties and a mismatched manifest version are all reported. - audit-mcp-verbs.mjs: draft_read rejects `scope` with "unknown key"; audio_mix on a draft without audio reports "nothing to mix"; grade_apply succeeds on a bright source and reports the near-black source separately; verify_diff returns verdict FAIL with failureKind=acceptance. Design compliance - Skill-only Plugin: no network client, no credential handling, no telemetry, no third-party service. - Every input schema keeps additionalProperties:false, so a parameter that the backend cannot honour is rejected instead of ignored. - No absolute path of any contributor machine is baked into the Plugin. --- plugins/antianqi/jianying/mcp/server.mjs | 36 ++++++++-- .../jianying/mcp/tools/draft-read.schema.json | 8 +-- .../antianqi/jianying/references/pitfalls.md | 60 ++++++++++++++++- .../jianying/references/verify-discipline.md | 27 ++++++++ .../jianying/scripts/selftest-mcp.mjs | 67 ++++++++++++++++++- .../skills/jianying-draft-edit/SKILL.md | 3 +- 6 files changed, 185 insertions(+), 16 deletions(-) diff --git a/plugins/antianqi/jianying/mcp/server.mjs b/plugins/antianqi/jianying/mcp/server.mjs index 419b6145..70e3e0f6 100644 --- a/plugins/antianqi/jianying/mcp/server.mjs +++ b/plugins/antianqi/jianying/mcp/server.mjs @@ -52,13 +52,20 @@ function which(cmd, extra = []) { for (const p of extra) { try { if (fs.existsSync(p)) return p; } catch { /* ignore */ } } return cmd; // 交给 spawn 报错 } +// ★ 2026-10-22 审计：候选路径一律从 env 拼（ProgramFiles / LOCALAPPDATA），【不写盘符字面量】 +// （非 Windows 上这些 env 不存在 ⇒ 自然跳过，不产生"只能在 Windows 跑"的硬绑定） +const progFiles = () => (process.env.ProgramFiles || '').trim(); +const localApp = () => (process.env.LOCALAPPDATA || '').trim(); const FFMPEG = () => which('ffmpeg', [ - path.join(process.env.ProgramFiles || 'C:\\Program Files', 'ffmpeg', 'ffmpeg.exe'), + ...(progFiles() ? [path.join(progFiles(), 'ffmpeg', 'ffmpeg.exe')] : []), ]); +// ★ 2026-10-22 审计：原来写死 Python314 ⇒ 装 3.11/3.12/3.13 的用户会找不到。改为按版本泛化 + 多个解释器名。 +const PY_VERS = ['Python314', 'Python313', 'Python312', 'Python311', 'Python310', 'Python39']; const PYTHON = () => process.env.JYAI_PYTHON || which('python', [ - path.join(process.env.ProgramFiles || 'C:\\Program Files', 'Python314', 'python.exe'), - path.join(process.env.LOCALAPPDATA || '', 'Programs', 'Python', 'Python314', 'python.exe'), -]); + ...(progFiles() ? PY_VERS.map((v) => path.join(progFiles(), v, 'python.exe')) : []), + ...(localApp() ? PY_VERS.map((v) => path.join(localApp(), 'Programs', 'Python', v, 'python.exe')) : []), + ...(localApp() ? [path.join(localApp(), 'Programs', 'Python', 'Launcher', 'py.exe')] : []), +]) || which('python3', []) || 'python'; /* ------------------------------------------------- 轻量 JSON Schema 校验 */ /* 只支持我们 schema 里用到的关键字：type/properties/required/additionalProperties/ */ @@ -382,7 +389,7 @@ const IMPL = { if (a.realDraft) before = shaTree(a.realDraft); // 护栏 2：素材路径重定位（草稿搬过家后 path 失效 ⇒ 指回草稿目录内） - const draftDir = st.draftDir || a.draftDir || ''; + const draftDir = st.draftDir || ''; // ★ 入参里的 draftDir 是死代码：schema 拒未知键 ⇒ 永远取不到（审计发现） const fixes = relocateMaterialPaths(draft, draftDir); const target = st.draftPath || path.join(draftDir, 'draft_content.json'); const rootPath = draftDir || path.dirname(target); @@ -434,7 +441,18 @@ const IMPL = { const j = JSON.parse(fs.readFileSync(cmp, 'utf8')); const probe = j.node_effect_probe || {}; const dead = Object.entries(probe).filter(([k, v]) => k !== '_full_chain' && !(Number(v) > 0)).map(([k]) => k); - if (dead.length) throw new Error(`★ 下列节点没有生效（探针为 0）：${dead.join(', ')} —— 判定失败`); + if (dead.length) { + // ★ 2026-10-22 审计：探针为 0 有两种可能，错误信息必须分开说 + // ① 源画面几乎全黑 ⇒ 任何"加亮/加对比"类调色都被 clip 回 0 ⇒ 探针测不出（链是好的） + // ② 参数形态不对 ⇒ 节点真的没生效 + const srcLuma = Number(j.src_luma_max); + if (Number.isFinite(srcLuma) && srcLuma < 8.0) { + throw new Error(`★ 节点 ${dead.join(', ')} 的探针为 0 —— 但源画面几乎全黑（YAVG=${srcLuma.toFixed(2)}/255）` + + `⇒ 这是【源没有可测信息】，不是节点没生效（加亮类调色会被 clip 回 0）。换一个有大面积中间调/亮部的源再验。`); + } + throw new Error(`★ 下列节点没有生效（探针为 0）：${dead.join(', ')} —— 判定失败` + + (Number.isFinite(srcLuma) ? `（源画面 YAVG=${srcLuma.toFixed(2)}/255）` : '')); + } if (!(Number(j.grade_change_mad) > 0)) throw new Error('★ 调色改变量为 0 ⇒ 判定失败'); return ok({ out: j.out10, outLut: j.out10_lut, gradeChangeMad: j.grade_change_mad, probe }); }, @@ -586,7 +604,8 @@ const IMPL = { const visibleRows = frameDetail.filter((r) => r.visible); const unstable = frameDetail.filter((r) => r.unstable).map((r) => r.frame); const bestVisible = visibleRows.length - ? visibleRows.reduce((a, b) => (b.mad > a.mad ? b : a)) + // ★ 累加器别叫 a：handler 形参也叫 a ⇒ 同名会让"入参扫描"把 a.mad 当成入参（审计踩过） + ? visibleRows.reduce((acc, row) => (row.mad > acc.mad ? row : acc)) : null; const bar = calibrated ? Math.max(th, noiseMad * noiseK) : th; // 全局参考线（仅作参考） const visible = visibleRows.length > 0; @@ -749,6 +768,9 @@ const IMPL = { res.verdict = problems.length ? 'FAIL' : 'PASS'; res.problems = problems; + // ★ isError=true 仍保留（skill 语义：判不过就是判不过），但加 failureKind 让调用方能区分 + // 「验收结论不通过」与「工具故障」⇒ 后者走 .catch 的 `xxx 失败：…` 文本，没有 failureKind + if (problems.length) res.failureKind = 'acceptance'; return problems.length ? fail(res) : ok(res); }, }; diff --git a/plugins/antianqi/jianying/mcp/tools/draft-read.schema.json b/plugins/antianqi/jianying/mcp/tools/draft-read.schema.json index ad2a9972..2317dee5 100644 --- a/plugins/antianqi/jianying/mcp/tools/draft-read.schema.json +++ b/plugins/antianqi/jianying/mcp/tools/draft-read.schema.json @@ -15,13 +15,9 @@ "root": { "type": "string", "description": "草稿根目录（素材/子目录的解析基准）。不传则取 path 的上级目录。★ 官方 readFile 没有它会解析不出 workspace ⇒ 静默返回空" - }, - "scope": { - "type": "string", - "enum": ["full", "timeline", "segments", "materials"], - "default": "full" } }, - "additionalProperties": false + "additionalProperties": false, + "description": "（scope 分段读取当前【未实现】：后端不支持，schema 已移除该参数，别按它写代码）" } } diff --git a/plugins/antianqi/jianying/references/pitfalls.md b/plugins/antianqi/jianying/references/pitfalls.md index 6377847c..6bc9be5f 100644 --- a/plugins/antianqi/jianying/references/pitfalls.md +++ b/plugins/antianqi/jianying/references/pitfalls.md @@ -68,7 +68,7 @@ ### 9. 把官方的结构当我们的模板 - **症状**：照抄了一堆 mcode 根本不认的目录（`model/`、`model-tasks.json`…） -- **对策**：**官方是素材与语义的来源，mcode 规范才是结构约束**；先读 `docs/plugin-compatibility.md` 与 `scripts/validate.mjs` +- **对策**：**官方是素材与语义的来源，mcode 规范才是结构约束**；先读 `docs/plugin-compatibility.md` 与 `node scripts/validate.mjs（仓库根，不是插件目录）` ### 10. "把 demo 当默认值的代码"= 没被测过的代码 - **症状**：spec 里没有某个节点 ⇒ `KeyError` @@ -115,3 +115,61 @@ - `sharpen`：`white_gam`/`black_gam` 被注释掉 - `clear`：`intensity = self.intensity * sharpIntensity`，而 `sharpIntensity` 默认 **0** ⇒ 输出逐字节等于输入 - ⇒ **看到"官方效果没出来"，先查默认参数，不是我们的 bug** + +## ★ 2026-10-22 审计新增 + +### 18. ★★★ 输入问题被误报成"链坏了"——三类都要分开报 + +同一个坑这轮出现了三次（"素材缺失""源画面全黑""音轨静音"），症状都是**明明链是好的，报错却说链坏了**： + +| 症状 | 真实原因 | 该怎么说 | +|---|---|---| +| "成片文件过小/不存在" | 草稿素材不在磁盘上 ⇒ 官方渲出黑帧 ⇒ 3 秒纯黑 h264 只有 3 KB | 字节数只作**提示**；主判据 = **有视频流 + 时长对齐**；再加**首帧全黑检测** | +| "响度 −70 LUFS 偏离目标" | 音轨是**静音**（素材缺失），不是归一化坏了 | 响度 ≤ −60 LUFS 判为**静音**（输入问题）⇒ 报"音频素材缺失 N 个" | +| "★ 下列节点没有生效（探针为 0）" | 源画面**近乎全黑** ⇒ 加亮类调色被 clip 回 0 | 报"**源没有可测信息**，不是节点没生效"，并附源 YAVG | + +⇒ **通用规则**：报"失败"前先分清 ①链真的坏了 ②输入本来就那样 ③参数写错了 ⇒ 三者用不同的话报。 + +### 19. ★★★ 探针只看第 0 帧 ⇒ 暗素材上误报 + +> `jyai-grade-render.py` 的逐节点生效探针原来只取 `frames[0]`。 +> 实测：源首帧 YAVG=16/255（近黑）⇒ brightness 加 0.2 / 0.6 后被 clip 回 0 ⇒ 探针=0 ⇒ 误报"节点没生效"。 +> ⇒ **改成多帧取最大**；全片都无信息时明确说"源画面近乎全黑，无法用亮度差判生效"。 + +### 20. ★★★ schema 声明了但后端不支持 ⇒ 静默忽略 + +> `draft_read` 的 `scope` 参数：schema 里有（还带 enum 校验，传非法值会报错）， +> 但 **handler 根本没把它传给后端**，后端也不支持分段读 ⇒ 传合法值时**静默返回全量**。 +> 而且 `SKILL.md` 里还教用户这么用 ⇒ 用户照做会拿到错的东西。 +> +> ⇒ **要么真接上，要么从 schema 删掉**（schema 里的死参数比没有更糟，因为它看起来能用）。 +> ⇒ 已删：schema 去掉 `scope`，SKILL 的示例改掉，并加了一条回归断言（传 scope 必须被拒）。 + +### 21. ★★ 删东西之前先查"有没有东西依赖它" + +> 清理草稿库时把插件自检 e2e 依赖的两个草稿删了 ⇒ 9 条 e2e 全部 SKIP， +> 而自检计数仍然是"0 FAIL"⇒ **看着绿，其实没测**。 +> ⇒ 删之前先扫一遍 tools/ 与 scripts/ 里对它的引用。 +> ⇒ 配套：e2e 用例缺环境时必须**显式 SKIP 且不计入 PASS**。 + +### 22. ★ 跨副本比对内容必须忽略行尾符 + +> 已装副本是 git clone（有 `core.autocrlf` ⇒ 工作区 CRLF），PR worktree 是 LF +> ⇒ **sha256 逐文件比全是假差异**。⇒ 比对前统一行尾符（或直接看 `git log` / `git status`）。 +> ⇒ 真正的"落后"要用 git 看，不是哈希看。 + +### 23. ★★ 官方预览渲染器的能力边界（实测，2026-10-22） + +> 用**同源 5 稿对照**（噪声底精确为 0）量出来的： +> +> | 能力 | 官方预览渲染器 | +> |---|---| +> | 强度型调色（`materials.effects[]` 里 `type:"brightness"` + 段引用） | ✅ **它就带** | +> | 几何 / 合成（画中画 / alpha / 缩放） | ✅ | +> | 曲线 / HSL / 色轮 | ❌ | +> | 特效链（`materials.video_effects[]` / `materials.effects[]` 的 `mix_mode` 等） | ❌ | +> | `materials.placeholders[]` 的 `type:"adjust"` | ❌ | +> | forceRender 链 | ❌ | +> | 导出档渲染器 / 官方编码器 / 官方混音器 | ❌（在 app 内私有 mojo 通道后面） | +> +> ⇒ **所以"把调色写回剪映预览"只对强度型成立**；曲线类只能我们自己在出片链里做。 diff --git a/plugins/antianqi/jianying/references/verify-discipline.md b/plugins/antianqi/jianying/references/verify-discipline.md index b70b284b..cb0d448f 100644 --- a/plugins/antianqi/jianying/references/verify-discipline.md +++ b/plugins/antianqi/jianying/references/verify-discipline.md @@ -106,6 +106,33 @@ > **纪律：报"改动可见"之前，先报噪声底。** 没有噪声底就不要报"可见"。 +### 8. ★★ 阳性对照必须与被测项【同路径、同对象】，只换一个已知会生效的东西 + +> 2026-10-22 审计实测：判"官方预览渲染器不执行 effect"时，带 effect 与不带 effect 的两稿 +> 逐像素完全相同（看起来是结论）。但把对照换成 `adjust`（已知官方预览**会**应用）后， +> **adjust 也完全相同** ⇒ 装置坏了，结论不成立。 +> +> ⇒ **"没变"有三种可能：①真的不支持 ②没生效/没挂上 ③装置坏了**。 +> 区分它们只有一个办法：**同一条路径、同一个对象、只换一个已知会生效的东西**， +> 它也"没变" ⇒ 判定装置有问题，换装置再测。 + +### 9. ★★ 判据的"期望值"必须先在【已知正确的实现】上成立 + +> 2026-10-22 实测：我拿"改了 R 通道 ⇒ G/B 通道必须逐字节不变"当硬判据， +> 而**官方 curves 语义里 Y（亮度）曲线会等量改 R/G/B 三个通道**（为保色相，这是有意的）。 +> ⇒ 那个判据与被测对象语义矛盾 ⇒ 拿它判自己的实现 = **自己判自己失败**。 +> +> ⇒ **写判据前先拿"官方明确对齐的那份实现"跑一遍**（比如 curves 的官方布局 spec）， +> 看它是否满足你的期望 ⇒ 满足才说明你的期望是对的。 + +### 10. ★ 噪声底必须【逐帧】，不能用平均 MAD + +> 官方预览渲染**非确定**：同一草稿渲两遍，整片平均 MAD 可能是 0.00000，也可能是 0.01602 +> （差 1 帧、**maxAbs 255**）⇒ 用平均 MAD 判"是否显著"会**随机翻车**。 +> +> ⇒ 用逐帧判据线：`bar_i = max(threshold, noise_i × noiseK)` +> （这一条已经落在 `verify_diff` 的实现里，§82 修正；这里补的是"为什么不能用平均"） + ## 数字底线（我们复刻链的验收口径） | 对象 | 口径 | 实测 | diff --git a/plugins/antianqi/jianying/scripts/selftest-mcp.mjs b/plugins/antianqi/jianying/scripts/selftest-mcp.mjs index 1f2a4bb3..c2e2354e 100644 --- a/plugins/antianqi/jianying/scripts/selftest-mcp.mjs +++ b/plugins/antianqi/jianying/scripts/selftest-mcp.mjs @@ -1,7 +1,7 @@ // jianying MCP server 的客户端验收脚本 // 用法: node selftest-mcp.mjs // 环境: JYAI_HOME 指向 JYAI 仓库 -import { spawn } from 'node:child_process'; +import { spawn, execFileSync } from 'node:child_process'; import fs from 'node:fs'; import path from 'node:path'; import { fileURLToPath } from 'node:url'; @@ -366,6 +366,71 @@ const textOf = (r) => (r.result?.content || []).map((c) => c.text || '').join('\ check('★★ grade 含渲染期算子(sharpen) ⇒ 明确报错（指向 grade_apply），不许静默忽略', rnRun.result?.isError === true && /grade_apply|渲染期/.test(textOf(rnRun)), textOf(rnRun).replace(/\s+/g, ' ').slice(-200)); + + // ── ★★ MCP 层全动词冒烟（2026-10-22 审计补：此前 5 个动词【从未走过 MCP 层】） + // draft_read / draft_patch / draft_commit / render_frames / audio_mix + const SMOKE = path.join(OUT, 'mcp-smoke'); + // ★ video.add 需要一个【真的能读的视频源】（传草稿 json 会得 video_source_unreadable） + const FILM_MEDIA = process.env.JYAI_TEST_MEDIA + || path.join(process.env.LOCALAPPDATA || '', 'Temp', 'jy-media-probe.mp4'); + if (!fs.existsSync(FILM_MEDIA)) { + // 造一个 1 秒的测试视频（不依赖用户素材） + const ff = path.join(path.dirname(FFMPEG), process.platform === 'win32' ? 'ffmpeg.exe' : 'ffmpeg'); + try { + execFileSync(ff, + ['-y', '-v', 'error', '-f', 'lavfi', '-i', 'color=c=gray:s=320x180:r=12:d=1', + '-f', 'lavfi', '-i', 'anullsrc=r=48000:cl=mono', '-shortest', + '-c:v', 'libx264', '-pix_fmt', 'yuv420p', '-c:a', 'aac', FILM_MEDIA], + { windowsHide: true, stdio: 'ignore' }); + } catch { /* 造不出来就跳过该用例 */ } + } + const op1 = await call('tools/call', { name: 'draft_open', arguments: { draft: FILM_DRAFT, copy: true } }); + const opTxt = textOf(op1); + const sid1 = (opTxt.match(/ve-session-\d+/) || [])[0] || ''; + check('★★ MCP: draft_open 走通并给出 sessionId', op1.result?.isError !== true && !!sid1, sid1 || opTxt.slice(0, 120)); + + const rd1 = await call('tools/call', { name: 'draft_read', arguments: { sessionId: sid1 } }); + const rdTxt = textOf(rd1); + check('★★ MCP: draft_read(sessionId) 读回草稿结构', + rd1.result?.isError !== true && /"tracks"|"segments"/.test(rdTxt), rdTxt.replace(/\s+/g, ' ').slice(0, 100)); + + const rd2 = await call('tools/call', { name: 'draft_read', arguments: { path: path.join(FILM_DRAFT, 'draft_content.json'), root: FILM_DRAFT } }); + check('★★ MCP: draft_read(path+root) 读回草稿结构', + rd2.result?.isError !== true && /"tracks"|"segments"/.test(textOf(rd2)), textOf(rd2).replace(/\s+/g, ' ').slice(0, 100)); + + // ★ 回归：scope 已从 schema 移除 ⇒ 传它必须被【拒】（不许静默接受） + const rdScope = await call('tools/call', { name: 'draft_read', arguments: { path: path.join(FILM_DRAFT, 'draft_content.json'), root: FILM_DRAFT, scope: 'segments' } }); + check('★★ MCP: draft_read 传 scope 被拒（后端不支持 ⇒ 别装成能用）', + rdScope.result?.isError === true && /未知键/.test(textOf(rdScope)), textOf(rdScope).replace(/\s+/g, ' ').slice(0, 120)); + + const pt1 = await call('tools/call', { + name: 'draft_patch', + arguments: { op: 'video.add', sessionId: sid1, params: { path: FILM_MEDIA, startTime: 0, sourceDuration: 1000000, trackIndex: 9 } }, + }); + check('★★ MCP: draft_patch(video.add) 被官方接受', + fs.existsSync(FILM_MEDIA) && pt1.result?.isError !== true && /accepted"?\s*:\s*true/.test(textOf(pt1)), + textOf(pt1).replace(/\s+/g, ' ').slice(0, 120)); + + const cm1 = await call('tools/call', { name: 'draft_commit', arguments: { sessionId: sid1, realDraft: FILM_DRAFT } }); + check('★★ MCP: draft_commit 落盘且真草稿未变', + cm1.result?.isError !== true && /"committed"\s*:\s*true/.test(textOf(cm1)) + && /"realDraftUnchanged"\s*:\s*true/.test(textOf(cm1)), textOf(cm1).replace(/\s+/g, ' ').slice(0, 140)); + + const rf1 = await call('tools/call', { name: 'render_frames', arguments: { draft: FILM_DRAFT, outDir: SMOKE, fps: 6 } }); + const framesOk = fs.existsSync(path.join(SMOKE, 'frames')) && + fs.readdirSync(path.join(SMOKE, 'frames')).some(f => /^\d+\.png$/.test(f)); + check('★★ MCP: render_frames 产出帧序列', rf1.result?.isError !== true && framesOk, + textOf(rf1).replace(/\s+/g, ' ').slice(0, 120)); + + const am1 = await call('tools/call', { + name: 'audio_mix', + arguments: { draft: FILM_DRAFT, video: path.join(FILM_DRAFT, 'draft_content.json'), out: path.join(SMOKE, 'mix.mp4') }, + }); + const amTxt = textOf(am1); + // ★ 这个草稿没有可用音轨 ⇒ 预期是"明确报 nothing to mix"（输入问题），而不是别的错 + check('★★ MCP: audio_mix 在无音轨草稿上明确报输入问题（nothing to mix），不静默出坏片', + am1.result?.isError === true ? /nothing to mix/.test(amTxt) : fs.existsSync(path.join(SMOKE, 'mix.mp4')), + amTxt.replace(/\s+/g, ' ').slice(0, 140)); } else { console.log(` [SKIP] make_film 真 e2e —— 需要 env JYAI_DRAFT_LIB（草稿库）+ 剪映在跑；本机未提供 ⇒ 【不计入 PASS】`); } diff --git a/plugins/antianqi/jianying/skills/jianying-draft-edit/SKILL.md b/plugins/antianqi/jianying/skills/jianying-draft-edit/SKILL.md index cfecd01f..b8f16548 100644 --- a/plugins/antianqi/jianying/skills/jianying-draft-edit/SKILL.md +++ b/plugins/antianqi/jianying/skills/jianying-draft-edit/SKILL.md @@ -55,7 +55,8 @@ metadata: node ${PLUGIN_ROOT}/scripts/run-verb.mjs draft_open '{"draft":"<草稿库>\\<草稿名>","copy":true}' # ② 读结构（拿 segmentId） -node ${PLUGIN_ROOT}/scripts/run-verb.mjs draft_read '{"scope":"segments"}' +node ${PLUGIN_ROOT}/scripts/run-verb.mjs draft_read '{"path":"<草稿目录>/draft_content.json","root":"<草稿目录>"}' +# ★ 不要传 scope：后端不支持（分段读取未实现，传了会被 schema 拒） # ③ 改（可多次；每次 baseRevision 会递增） node ${PLUGIN_ROOT}/scripts/run-verb.mjs draft_patch '{"op":"video.setVolume","sessionId":"ve-session-N","params":{"segmentId":"","volume":0.25}}' From cb5d7026ab952da7203cceee0ab42ce5f6c5301e Mon Sep 17 00:00:00 2001 From: antianqi Date: Thu, 22 Oct 2026 17:27:15 +0800 Subject: [PATCH 15/15] docs(jianying): the discipline list has ten rules, not six The heading still said six while rules 8-10 had been added in the audit pass. --- plugins/antianqi/jianying/references/verify-discipline.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/plugins/antianqi/jianying/references/verify-discipline.md b/plugins/antianqi/jianying/references/verify-discipline.md index cb0d448f..a4661687 100644 --- a/plugins/antianqi/jianying/references/verify-discipline.md +++ b/plugins/antianqi/jianying/references/verify-discipline.md @@ -38,7 +38,7 @@ | **契约层** | 字段认不认 | 材料存活 / 回写明文 / 字段保留 | | **渲染层** | 有没有效果 | 无头取帧 + **A/B 逐像素差分** + **阳性对照** | -## 六条硬规矩 +## 十条硬规矩 ### 1. 判据的"视野"必须覆盖被测对象 **只抽一帧 = 用抽样代替了被测对象** ⇒ 假 FAIL 和假 PASS 都会出。