Skip to content

feat(dashboard): add PyPI mirror picker and speed tests for network sources - #10657

Open
XTsat wants to merge 10 commits into
AstrBotDevs:masterfrom
XTsat:feat/settings-network-proxy-dependency
Open

XTsat wants to merge 10 commits into
AstrBotDevs:masterfrom
XTsat:feat/settings-network-proxy-dependency

Conversation

@XTsat

@XTsat XTsat commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

Resolved #10577

选择镜像站时,现在能直接看到每个 GitHub 加速地址和 PyPI 源通不通、有多快,不用再一个个试

Modifications / 改动点

后端

  • astrbot/dashboard/services/stat_service.py:新增 test_pypi_index_connection(只取响应头、使用 pip 的 UA、拒绝非 http(s) 输入)和 measure_pypi_index_speed(从索引页解析出最新的 -py3-none-any.whl 并真实下载);新增 measure_ghproxy_speed,通过代理流式下载一个真实的仓库文件;test_ghproxy_connection 现在把空 proxy_url 视作"直连 GitHub",使"不使用加速"与各镜像在同等条件下被测量;统一采样预算(6 秒 / 4 MiB),避免一次点击拉取几十 MB 流量
  • astrbot/dashboard/api/stats.py:新增 POST /api/v1/stats/pypi-index/test、/pypi-index/speed、/ghproxy/speed,均为 system scope
  • astrbot/dashboard/schemas.py:新增 PypiIndexTestRequest
  • openspec/openapi-v1.yaml、dashboard/src/api/generated/openapi-v1/*、dashboard/src/api/v1.ts:契约、重新生成的客户端、statsApi 封装

前端

  • dashboard/src/components/shared/PypiMirrorSelector.vue(新增):官方、阿里云、腾讯云、中科大、清华五个预设加自定义 URL,每项带延迟与 MB/s chip,并提供两个批量测试按钮
  • dashboard/src/components/shared/ProxySelector.vue:把两级开关拍平为单个单选列表(none / 预设下标 / custom);自定义输入内容独立存放并使用独立 localStorage 键,因此切换预设不会丢失,未被选中时也仍可被测试;每条线路(含"不使用代理")都显示延迟与 MB/s chip
  • dashboard/src/views/Settings.vue:新增 PyPI 源独立卡片;pypi_index_url 从通用 network 组移除,保证该字段只渲染一次
  • zh-CN、en-US、ja-JP、ru-RU 四套 i18n

入口对照(旧 → 新)

  • 设置 → 网络 →「GitHub 加速」开关 → 「选择 GitHub 镜像源」扁平列表,其中"不使用 GitHub 镜像"成为一等选项。

  • 设置 → 网络 → 通用 pypi_index_url 配置行 → 「PyPI 软件仓库地址」卡片。

  • This is NOT a breaking change. / 这不是一个破坏性变更。

Screenshots or Test Results / 运行截图或测试结果

6c74f655726a8d58

Checklist / 检查清单

  • 😊 If there are new features added in the PR, I have discussed it with the authors through issues/emails, etc.
    / 如果 PR 中有新加入的功能,已经通过 Issue / 邮件等方式和作者讨论过。

  • 👀 My changes have been well-tested, and "Verification Steps" and "Screenshots" have been provided above.
    / 我的更改经过了良好的测试,并已在上方提供了“验证步骤”和“运行截图”。

  • 📚 I checked the affected WebUI instructions and screenshots in docs/zh and docs/en against the changed navigation, page structure, and labels, and updated them in this PR (or explained why no documentation update is needed). For renamed, moved, or merged entry points, I included an old entry → new entry mapping in the documentation and changelog.
    / 我已对照变化后的 WebUI 入口、页面结构和术语,核对并在本 PR 中更新 docs/zh 和 docs/en 的相关操作说明与截图(或说明无需更新文档的原因)。入口改名、移动或合并时,已在文档和 changelog 中补充 旧入口 → 新入口 对照。

  • 🤓 I have ensured that no new dependencies are introduced, OR if new dependencies are introduced, they have been added to the appropriate locations in requirements.txt and pyproject.toml.
    / 我确保没有引入新依赖库,或者引入了新依赖库的同时将其添加到 requirements.txt 和 pyproject.toml 文件相应位置。

  • 😮 My changes do not introduce malicious code.
    / 我的更改没有引入恶意代码。

Summary by Sourcery

Add network source pickers that let users compare PyPI mirrors and GitHub routes by availability, latency, and download speed.

New Features:

  • Add PyPI mirror selection with official, preset, and custom indexes.
  • Add connection latency and download-speed testing for PyPI indexes and GitHub routes.
  • Expose direct GitHub access as a selectable and measurable network route.

Bug Fixes:

  • Prevent PyPI index configuration from being rendered twice in Settings.
  • Preserve custom proxy and mirror URLs when switching between presets.

Enhancements:

  • Flatten GitHub proxy selection into a single radio list with status and throughput indicators.
  • Apply bounded sampling to network speed tests to limit download time and data usage.

Chores:

  • Extend the statistics API contract, generated client, schemas, and localized settings labels for network source testing.

XTsat added 7 commits October 20, 2026 23:46
- Move the connectivity test button next to the section title instead of
  the "use GitHub proxy" radio row.
- Drop the nested radio group so the proxy URLs sit at the same level as
  "no proxy", giving one flat list of choices.
- Test every option at once, including the direct (no proxy) route and a
  custom URL, and show the result per option.
- Let the ghproxy test endpoint accept an empty proxy_url to probe GitHub
  directly, which is what the "no proxy" option needs.
- Remove the now unused proxySelector.useProxy i18n key.

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've found 2 issues

Prompt for AI Agents
Please address the comments from this code review:

## Individual Comments

### Comment 1
<location path="astrbot/dashboard/services/stat_service.py" line_range="655" />
<code_context>
+                        raise StatServiceError(
+                            f"Failed. Status code: {response.status}"
+                        )
+                    page = await response.text()
+
+                wheels: list[tuple[Version, str]] = []
</code_context>
<issue_to_address>
**Large index pages exhaust memory**

When a configured or caller-supplied PyPI index returns a very large `/pip/` page, `measure_pypi_index_speed` reads the complete response into `page` with `response.text()` before applying any size limit. Large or overlapping responses consume dashboard memory and can exhaust the process.

Read the response with a byte limit and stop or reject it once that limit is reached.
</issue_to_address>

### Comment 2
<location path="astrbot/dashboard/services/stat_service.py" line_range="658" />
<code_context>
+                    page = await response.text()
+
+                wheels: list[tuple[Version, str]] = []
+                for href in re.findall(r'href="([^"#]+)', page):
+                    file_name = href.rsplit("/", 1)[-1]
+                    suffix = "-py3-none-any.whl"
</code_context>
<issue_to_address>
**Valid index markup fails speed test**

When a custom PyPI index serves its links with single-quoted `href` attributes or uppercase `HREF`, `measure_pypi_index_speed()` only extracts lowercase, double-quoted `href` attributes, so a valid simple-index page using single quotes or uppercase `HREF` yields no wheel links and the speed test reports “No pip wheel found.”

Parse the page with an HTML parser that handles valid attribute quoting and casing.
</issue_to_address>

Sourcery assessment

Needs a human reviewer. 2 findings to address first, and the new system-scoped speed probe fetches the index URL and then follows wheel links from its page, including absolute links, so a supplied index can make the server send GET requests to internal hosts. Reverting removes the probe but cannot undo any side effects those requests may already have triggered.

Blocking findings: astrbot/dashboard/services/stat_service.py:655, astrbot/dashboard/services/stat_service.py:658


Sourcery is free for open source - if you like our reviews please consider sharing them ✨

Comment thread astrbot/dashboard/services/stat_service.py Outdated
Comment thread astrbot/dashboard/services/stat_service.py Outdated
XTsat added 3 commits October 21, 2026 05:02
measure_pypi_index_speed read the whole /pip/ page with response.text()
before applying any size limit, so an oversized or endless page from a
configured or caller-supplied index could exhaust dashboard memory.

Stream the body with a hard 8 MB budget and abort with a StatServiceError
as soon as it is exceeded.
The speed test extracted wheels with a regex for lowercase, double-quoted
href attributes, so a valid simple index using single quotes, unquoted
values, or uppercase HREF reported "No pip wheel found on this index".

Collect anchor hrefs with html.parser.HTMLParser instead, which handles
attribute quoting and casing, and strip the "#sha256=..." fragment
explicitly.
`ruff format --check` flagged stat_service.py on this branch: a missing
blank line after StatServiceError and an over-long wheel filter from the
earlier speed-test work. Reformat with the pinned ruff 0.15.22 so CI is
green again.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature] 设置-网络-PyPI 软件仓库地址 提供预设的镜像站点及测速

1 participant