Skip to content

Authenticate index requests to private registries - #968

Open
sunsided wants to merge 12 commits into
killercup:masterfrom
sunsided:feature/auth
Open

sunsided wants to merge 12 commits into
killercup:masterfrom
sunsided:feature/auth

Conversation

@sunsided

@sunsided sunsided commented Jul 13, 2026 •

Copy link
Copy Markdown

Summary

cargo upgrade now sends the registry authentication token when querying a private sparse-registry index, so workspaces depending on an auth-required = true registry can be upgraded instead of failing with 401 Unauthorized.

Why

Reported in #931: cargo upgrade --incompatible fails with status code '401 Unauthorized': the request was not authorized against a private registry, while cargo search / build / test succeed with the same credentials, and downgrading to 0.13.1 avoids it.

The index request was always unauthenticated: RemoteIndex::krate builds it via tame-index's make_remote_request, which by design attaches no credentials, and cargo-edit never read or sent a token. Before 0.13.2 the resulting 401 was silently swallowed and the dependency skipped; #925 made index errors propagate, turning that long-standing gap into a hard failure that aborts the whole run.

Key changes

  • registry_token() in src/registry.rs resolves a registry's token from CARGO_REGISTRIES_<NAME>_TOKEN, then token under [registries.<name>] in credentials / config files, walking .cargo/ from the manifest up to $CARGO_HOME.
  • RemoteIndex carries the token and attaches it verbatim as the Authorization header on the index request, matching cargo (no Bearer prefix is injected).
  • The token is threaded through IndexCache to the single call site in cargo upgrade.
  • The crates.io index itself is left unauthenticated, so a crates.io publish token is never leaked to the CDN-fronted sparse index. When crates.io is replaced by another registry, that registry's token is used instead (see below).
  • replace-with source replacement is followed for both the index URL and the token: with [source.crates-io] replace-with = "my-registry", the request goes to my-registry's index with my-registry's token. Replacement chains of any length are resolved; a cycle (a -> b -> a) is reported as an error instead of a misleading "could not be found". Thanks to @mkatychev for this part.
  • A misconfigured registry or replacement chain is now an error from registry_token() as well, matching registry_url(), rather than silently sending no token and surfacing later as a 401.
  • registry = "crates-io" on a dependency resolves to the built-in crates.io index instead of failing with "The registry 'crates-io' could not be found".
  • IndexCache is keyed by index URL and token, so two registries sharing a URL, or one URL looked up with and without a token, no longer reuse the wrong client.
  • Regression coverage: token precedence (env over files, credentials over config, nearest directory first), env-var name mangling, single- and multi-hop source replacement, cycle and missing-source errors, plus a local HTTP server asserting the Authorization header is present with a token and absent without.

Known gap: credential providers

Only static tokens are handled. The CARGO_REGISTRIES_<NAME>_TOKEN env var and a plaintext token in credentials.toml / config.toml. Cargo credential providers (registries.<name>.credential-provider, [registry] global-credential-providers, e.g. cargo:token-from-stdout, cargo:libsecret, or any credential-process) are not invoked. Users who keep their token in a provider rather than a plaintext token will still hit the 401. Matching cargo's full credential-provider resolution is left as a follow-up.

Review instructions

Start with registry_token() in src/registry.rs, resolution order and the crates.io opt-out. resolve_registry() is shared by registry_url() and registry_token(), so both follow the same replace-with chain; the token is looked up under the last name in that chain. Then RemoteIndex::krate in src/index.rs, where the header is attached right after the tame-index request is converted to a reqwest request. The src/bin/upgrade/upgrade.rs change is just wiring the token into the existing index.krate call. Worth scrutiny: the token precedence (env over file, credentials over config, nearest dir first) and the decision to send the token verbatim.

Closes #931

`cargo upgrade` aborted with `status code '401 Unauthorized': the
request was not authorized` whenever a workspace depended on a private
registry with `auth-required = true`.

`RemoteIndex::krate` builds the sparse-index request via tame-index's
`make_remote_request`, which intentionally attaches no credentials, and
cargo-edit never read or sent a registry token. The request therefore
went out unauthenticated and the registry rejected it with 401. This was
always broken for auth-required registries; before 0.13.2 the error was
silently swallowed (the dependency was skipped), but killercup#925 made index
errors propagate, so the 401 now aborts the whole run.

Resolve the registry token (`CARGO_REGISTRIES_<NAME>_TOKEN`, then `token`
under `[registries.<name>]` in credentials/config, walking `.cargo/` up
to `$CARGO_HOME`) and attach it verbatim as the `Authorization` header,
matching cargo. The crates.io index is left unauthenticated so its
publish token is not leaked to the CDN-fronted sparse index.

Fixes killercup#931
@sunsided sunsided changed the title fix(upgrade): Authenticate index requests to private registries Authenticate index requests to private registries Jul 13, 2026
@sunsided
sunsided marked this pull request as ready for review July 13, 2026 17:32
Copilot AI review requested due to automatic review settings July 13, 2026 17:32

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates cargo upgrade to authenticate sparse index requests for private registries by resolving a registry token (matching Cargo’s precedence rules) and attaching it as the Authorization header on the index HTTP request.

Changes:

  • Added registry_token() to resolve per-registry auth tokens from env/config/credentials files while explicitly opting out for the default (crates.io) registry.
  • Threaded an optional auth value through IndexCache/RemoteIndex and attach it as the Authorization header for remote sparse index requests.
  • Wired token lookup into cargo upgrade, and added regression tests plus a changelog entry.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
src/registry.rs Adds token resolution logic (registry_token) and unit tests for precedence/behavior.
src/lib.rs Re-exports registry_token for use by binaries.
src/index.rs Threads optional auth into remote index access and attaches Authorization header; adds request-capture tests.
src/bin/upgrade/upgrade.rs Wires token resolution into the existing index lookup call in cargo upgrade.
CHANGELOG.md Documents the fix in Unreleased notes.
Comments suppressed due to low confidence (1)

src/index.rs:93

  • IndexCache is keyed only by registry Url, but the RemoteIndex stores auth as state. If the same Url is queried with different auth values in one run (e.g. default crates.io with None, then a named registry/source replacement mapping to the same Url with Some(token), or two names sharing an index Url), the cached RemoteIndex will keep the first auth value. This can either omit credentials (causing 401) or send a token on requests that were meant to be unauthenticated (credential leakage). Consider updating the cached RemoteIndex's auth on every access (or include auth in the cache key).
        if !self.index.contains_key(registry) {
            let index = AnyIndex::open(registry, self.certs_source, auth)?;
            let index = AnyIndexCache::new(index);
            self.index.insert(registry.clone(), index);
        }
        Ok(self.index.get_mut(registry).unwrap())
    }

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/registry.rs Outdated
sunsided and others added 2 commits July 14, 2026 08:54
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
@sunsided

sunsided commented Sep 9, 2026

Copy link
Copy Markdown
Author

@killercup Any interest in this?

@mkatychev

This comment was marked as outdated.

@mkatychev

This comment was marked as resolved.

@sunsided

Copy link
Copy Markdown
Author

@mkatychev Hey, thanks a lot! I didn't follow up up on merge conflicts until I got an OK from @killercup; noticed the master branch was moving and didn't want to chase it all the time. 😅 Will apply your patch though, much appreciated!

Resolve src/index.rs conflict with the ureq / tame-index EtagCache migration
from killercup#973, carrying the Authorization header over to the ureq request.
Resolution provided by @mkatychev in killercup#968.
IndexCache was keyed by URL only while RemoteIndex captured the auth token
when first opened. Two registries sharing an index URL, or the same URL
looked up with and without a token, would reuse whichever client came
first: either withholding the token (401) or sending it where none was
configured. Including the auth in the key gives each combination its own
client.
@sunsided

Copy link
Copy Markdown
Author

Re the suppressed Copilot note on IndexCache being keyed only by registry URL:

Fixed in 22e7082. IndexCache is now keyed by (Url, auth), so a token is never reused for, or withheld from, another lookup on the same index URL. Covered by auth_is_part_of_the_cache_key.

mkatychev and others added 7 commits October 23, 2026 13:09
The replacement loop compared the source name against the crates.io
index URL instead of the "crates-io" registry name, so the default index
was never filled in. Because crates-io is built in and rarely declared in
config, a chain ending at it failed with "The source 'crates-io' could
not be found". Treat it as built in, as the head of the chain already is.
Source replacement still fails with a 401 on this branch because source
replacement does not pull resolved registry credentials:
```toml
[source.crates-io]
replace-with = "<MY_REGISTRY>"
```
A dependency with `registry = "crates-io"` went through the named-registry
lookup and failed with "The registry 'crates-io' could not be found"
unless some config declared it. It now resolves to the default index like
an unnamed dependency, and no token is sent to it.

Also cover the token and source-replacement paths the auth support
relies on: the `CARGO_REGISTRIES_<NAME>_TOKEN` override (including after
replacement), multi-hop `replace-with` chains, and nearest-config-wins
for both URL and token. `registry_token` takes the environment lookup
through a private helper so these tests need no `unsafe` `set_var`.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

cargo upgrade returns 401 consistently even after changing registry token

3 participants