Skip to content

Nightly Windows test shards fail on temp-path spelling (RUNNER~1 vs canonical \\?\ paths) #1689

Open

Description

[agent] The full-scope nightly's Windows test shards failed on 2 of the last 3 nights. A different test fails each night, and every failure we could read compares one temp path in two different spellings.

2026-10-23, run 39907645034 (aa69125)

  • test (windows-latest, 1), -p socket-patch-core --lib: hosted::governing_root::tests::vlt_project_root_governs_unlisted_directories (governing_root.rs:3320). The message names the same temp dir as C:\Users\RUNNER~1\AppData\Local\Temp\.tmpnPw8bm\tools/x (8.3 short name, mixed separators) and as \\?\C:\Users\unneradmin\AppData\Local\Temp\.tmpnPw8bm (canonical, verbatim prefix).
  • Same job, -p socket-patch-cli --test user_patches: also failed. The test name isn't recoverable from the session's log access (the log is too long to tail).
  • test (windows-latest, 2), -p socket-patch-cli --test get: get_failure_paths::engine_nested_apply_preserves_conflicts_introduced_during_fetch (get_failure_paths.rs:1660), assert!(skip["reason"]...contains(patch.to_str())). This is a substring check of a raw tempdir path against a message that probably carries the canonical one.

2026-10-22, run 39730006970 (3b6a85c)

  • test (windows-latest, 1), -p socket-patch-core --lib: patch::redirect::upstream::uv::tests::global_uv_settings_reads_the_user_then_first_system_file (uv.rs:2389). left ["…\.tmp1nilbo\xdg\uv\uv.toml"], right ["…\.tmp1nilbo\xdg/uv/uv.toml", "…\.tmp1nilbo\dirs2/uv/uv.toml"].

2026-10-21 (39551481612) was green.

Likely shared cause (needs confirming): tests build expected paths from tempfile paths (RUNNER~1 8.3 form, / joins in format strings) and compare them by string or substring with paths the code canonicalized (\\?\C:\Users\<long name>). Whether a runner's %TEMP% comes back in 8.3 form seems to vary between images and runs, so it fails intermittently. A fix in the test harness would canonicalize test tempdirs once, e.g. with a dunce::canonicalize-style helper in the tempdir constructors the suites share, and build expected paths with Path::join, not format!("{}/…").

This doesn't affect PR or merge-queue CI (lean scope has no Windows), only the nightly.


Generated by Claude Code

Activity

  1. mikolalysenko commented on Oct 23, 2026

    @mikolalysenko
    CollaboratorAuthor

    [agent] Covered by existing work (CI lane, 2026-10-23 08:5x UTC):

    Leaving this open until #1686 merges and a nightly Windows test leg goes green.


    Generated by Claude Code

  2. mikolalysenko commented on Oct 24, 2026

    @mikolalysenko
    CollaboratorAuthor

    [agent] Claiming this issue (shared root cause: Windows-only test expectations that assume Unix path-join semantics). #1686 fixed the earlier failures; test (windows-latest, 2) is now green on tonight's nightly 40067511756. test (windows-latest, 1) is the run's only red leg. One new test fails, patch::redirect::yarnrc::tests::global_folder_values_read_like_yarn (added in #1696), at yarnrc_tests.rs:594: left ["\\${HOME}/x"], right ["/w/\\${HOME}/x"]. On Windows a leading \ is drive-root-relative, so cwd.join replaces the cwd's root the way Node's path.resolve does. The expectation is Unix-only. Branch: agent/fix-ci-windows-yarnrc-escaped-path. Claim-ID: 2026-10-24T05:51:12Z-c41e7a


    Generated by Claude Code

  3. mikolalysenko commented on Oct 24, 2026

    @mikolalysenko
    CollaboratorAuthor

    [agent] Draft PR: #1710 (test-only). Dispatched one full-scope CI run on its head to prove the Windows test legs are green.


    Generated by Claude Code

  4. mikolalysenko commented on Oct 24, 2026

    @mikolalysenko
    CollaboratorAuthor

    CI janitor (2026-10-24 06:5x UTC): one pattern across the last 7 nightlies on main. Main's nightly was red on 5 of the last 7 nights (10-18, 10-20, 10-22, 10-23, 10-24). Each time the cause was a regression that only the nightly-only legs run, not a flake. 3 of those 5 nights were a Windows test shard:

    Nightly Test Signature Fix
    39730006970 10-22 upstream::uv::tests::global_uv_settings_reads_the_user_then_first_system_file xdg/uv/uv.toml vs xdg\uv\uv.toml separators #1663 (merged)
    39907645034 10-23 hosted::governing_root::tests::vlt_project_root_governs_unlisted_directories, get_failure_paths::engine_nested_apply_preserves_conflicts_introduced_during_fetch RUNNER~1 vs \\?\C:\Users\runneradmin #1686 (merged)
    40067511756 10-24 patch::redirect::yarnrc::tests::global_folder_values_read_like_yarn /w/\${HOME}/x expected, Windows root-relative join gives \${HOME}/x #1710 (open, draft)

    On the other 2 nights, 10-18 poetry/PyPI (#1650) and 10-20 pipenv/uv pylock (#1639), the failure was nightly-only too.

    Every Windows failure was in a test added or changed the day before. The lean PR and merge-queue tiers only compile Windows (windows-compile-check). As a result, each new path assertion with a Unix-only expectation lands green and turns the next nightly red. A fix PR then follows about a day later. The merge queue itself is healthy: 95/96 merge_group runs in the last 24h passed, with 0 evictions.

    Possible fixes. These are cost/coverage trade-offs, so I'm leaving them to the owner and the ci-perf lane rather than doing them here:

    1. Run only cargo test -p socket-patch-core --lib on Windows in the merge queue, next to windows-compile-check. All 4 failing tests above except get_failure_paths are core lib unit tests.
    2. Or keep the cost as it is, and have the nightly's Windows failures auto-file an issue that names the PR that introduced the test.

    Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions