Vendored vendor --revert / remove of a two-package pip lock pylock.toml writes an invalid lock: the first package's [packages.wheels.hashes] table moves under the next package, pip refuses the file, and the CLI exits 0 #1623

[agent] Found by the scheduled pip / requirements.txt bug-hunt routine (ledger #309).

Summary

pip lock (pip 25.1+) writes each wheel as a [[packages.wheels]] array-of-tables entry with a [packages.wheels.hashes] sub-table. If two packages in one such pylock.toml are vendored, reverting the package that was vendored first (vendor --revert, or remove <purl>) restores its wheels array into the wrong place in the document:

• the first package (idna) gets its [[packages.wheels]] header back, but its [packages.wheels.hashes] table is written after the next [[packages]] header,
• so TOML now reads that hash as a wheels table on six, next to six's vendored archive,
• the revert of six then sees "drift" and keeps six vendored.

The CLI exits 0. The resulting file isn't a valid pylock: pip install -r pylock.toml fails with ERROR: Invalid pylock file 'pylock.toml': Missing required value in 'packages[0].wheels[0].hashes'. Re-running vendor --revert changes nothing (it reports the same drift), and the suggested remedy ("undo the drift … re-run vendor --revert") asks the user to repair damage that socket-patch did itself.

The uv-written pylock layout (inline wheels = [{…}]) reverts byte-identical, and so does any single vendored package, so the existing tests don't hit this.

Impact

A plain-pip project that uses pip lock and has two or more vendored patches can't be un-vendored. Both vendor --revert and remove of the first-vendored package leave a lock that pip won't install, with one package still pointing at the vendored wheel and the other missing its required hash. Exit code 0 means CI doesn't notice until the next install.

Repro (Linux, main 657fce2; also v5.0.1)

python3 -m venv .venv && .venv/bin/python -m pip install -q -U pip   # pip 26.2.1
printf 'six==1.16.0\nidna==3.7\n' > req.in
.venv/bin/python -m pip install -q -r req.in
.venv/bin/python -m pip lock -r req.in -o pylock.toml
git init -q && git add pylock.toml && git commit -qm init
# stage .socket/manifest.json + .socket/blobs with one patch for six.py and one for idna/__init__.py
# (any two pypi patches; I used the repo's prebuilt_common::Server::project_with_env as the vendoring service,
#  pointed to by SOCKET_VENDOR_URL, same as the e2e_vendor_pypi_build suite)
VIRTUAL_ENV=$PWD/.venv socket-patch vendor --yes          # "Vendored 2 packages."
VIRTUAL_ENV=$PWD/.venv socket-patch vendor --revert --yes  # exit 0
git diff pylock.toml
python3 -m venv /tmp/fresh && /tmp/fresh/bin/python -m pip install -U -q pip
/tmp/fresh/bin/python -m pip install -r pylock.toml

Output of the revert:

Warning: pylock.toml changed since vendoring; conflicting fields were preserved
Warning: kept .socket/vendor/pypi/4d5e6f70-…: some recorded lock entries were left alone (see the vendor_lock_entry_drifted / vendor_lock_entry_removed warnings) …
Reverted 1 vendored package.
Kept 1 drifted package: lock entries were re-resolved since vendoring, …

pylock.toml after the revert (original was the unchanged pip lock output):

[[packages]]
name = "idna"
version = "3.7"

[[packages.wheels]]
name = "idna-3.7-py3-none-any.whl"
url = "https://files.pythonhosted.org/…/idna-3.7-py3-none-any.whl"

[[packages]]
name = "six"
version = "1.16.0"
archive = { path = ".socket/vendor/pypi/4d5e6f70-…/six-1.16.0-py2.py3-none-any.whl", hashes = { sha256 = "a917…" } }

[packages.wheels.hashes]
sha256 = "82fee1fc78add43492d3a1898bfa6d8a904cc97d8427f683ed8e798d07761aa0"   # idna's wheel hash

tomllib reads that as idna.wheels = [{name, url}] (no hashes) and six = {archive, wheels: {hashes: {…}}}.

remove pkg:pypi/idna@3.7 --yes (idna was vendored first) gives the same file, exit 0. remove pkg:pypi/six@1.16.0 (the last one vendored) is clean.

Expected vs actual

• Expected: CLI_CONTRACT.md describes vendor --revert as restoring the lockfile wiring the vendoring changed, and keeping (with a vendor_lock_entry_drifted warning) only entries the user changed. A revert of an untouched pip lock file should give back the original bytes, as it already does for the uv layout and for a single vendored package. A revert should never write a lock the package manager can't read.
• Actual: an invalid pylock, one package left vendored under a bogus drift warning, exit 0, and no way to finish the revert with the CLI.

Matrix (Linux)

This reproduces with or without a sibling requirements.txt. Release v5.0.1 (npm linux-x64-gnu binary) gives the same output, so it isn't a regression since the last release. pip 25.0 and older have no pip lock. I didn't run macOS / Windows; the code path is platform-independent TOML editing.

Suspect code

crates/socket-patch-core/src/vendor/pypi_lock.rs:659, restore_table: when the live table no longer has the key (vendoring replaced wheels with archive), it puts the original item back with live.insert(&key, before.clone()). For a [[packages.wheels]] ArrayOfTables the cloned tables (and their hashes sub-table) keep their document positions from the recorded original text. toml_edit orders standard tables by position when it prints the document, so in the live document (where positions differ once another package's entry was rewritten too) the hashes sub-table is emitted after the next [[packages]] header. Re-parsing then attaches it to the wrong package. The single-package and newest-first cases never reach this, because restore_document (:844) short-circuits on live == new.

A related cosmetic effect on the same path: toml_edit's Table::insert on an existing key resets that key's decor (key_mut().fmt()), so a comment line directly above a restored key is dropped. The Poetry routine handed that over for poetry export -f pylock.toml comments (ledger #309, entry 20261016T202635Z-from-poetry).

Probe runs: none (probe branches are blocked for this routine; Linux only).