Skip to content

0.48.2-RC1 - #3859

Open
icanhasmath wants to merge 8 commits into
betafrom
marcg/release-0-48-2
Open

icanhasmath wants to merge 8 commits into
betafrom
marcg/release-0-48-2

Conversation

@icanhasmath

@icanhasmath icanhasmath commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

Beta release 0.48.2-RC1 (PROD-240).

Cut from beta (0.48.1) with only the changes made on master since 0.48.1 cherry-picked on top. master and beta have diverged, so merging master would have shipped about 30 unreleased master-only commits as well.

Merging this stages the release (the push to beta deploys with DEPLOY_TO_STAGING=true). After that come Windows signing, state run promote-staging-to-production, verifying with install.sh -b beta, and merging beta back into master.

Changelog

Added

  • state publish --build now reads [project].dependencies from pyproject.toml and publishes them as runtime
    dependencies of the encrypted private ingredient. Version ranges carry over, and OS and Python version markers become
    dependency conditions (unsupported markers are skipped with a notice).
  • Added state export ingredient-meta <namespace>/<name>[@version], which writes an existing ingredient's meta file for
    use with state publish --edit --meta.

Changed

  • Installing runtimes on Windows is significantly faster: link paths are resolved once per tree instead of once per
    file, and artifacts are linked concurrently.
  • A --meta file's name, version, and description now feed the built wheel (flags take precedence over the meta file,
    which takes precedence over pyproject.toml).

Fixed

  • state publish --edit no longer overwrites the meta file's dependencies and authors with the previous revision's,
    keeps inherited dependency types and conditions, and rejects author or description changes on an existing version
    up front.
  • state publish --edit --meta with an exported meta file now edits the version named in the file rather than the
    latest version.

Security

mitchell-as and others added 8 commits October 21, 2026 16:56
* Resolve link paths once per tree instead of once per file

smartlink.Link resolved both src and dest via ResolveUniquePath for every
entry it recursed into. On Windows ResolveUniquePath calls GetLongPathName,
which is a syscall, so installing a runtime with many files performed two
extra syscalls per file - a major contributor to slow Windows installs.

The tree root is already resolved by LinkContents (and now by the public
Link), and descendant paths are built by joining an already-resolved parent
with real (long) entry names, so they need no further resolution. Split the
recursion into an unexported link() worker that assumes resolved inputs and
does not re-resolve, keeping resolution only for the rare symlink-to-file
case where the pre-existing behavior links to the dereferenced target.

Also treat an "already exists" error from linkFile the same as the existing
TargetExists skip, so concurrent deploys of a shared file degrade to the
documented "first one wins" behavior rather than failing.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Deploy linked artifacts concurrently instead of serializing them

DeployViaLink held the depot-wide fsMutex for the entire link operation, so
even though the install worker pool runs several artifacts at once, the
actual file linking was serialized to one artifact at a time. On Windows,
where each link is an individual (relatively slow) syscall, this made the
install phase far slower than it needed to be.

Linking touches no shared in-memory depot state (Exists guards its own map
via mapMutex and depotPath is immutable after construction), directory
creation is idempotent, and smartlink now treats an already-existing target
as a skip, so it is safe to run without the lock. Drop it from DeployViaLink
so linking parallelizes across the worker pool.

DeployViaCopy keeps the lock: two artifacts copying the same file would race
on os.Create and could corrupt the destination. That path is only used for
artifacts needing file transforms, when hard links are unsupported, or in
portable mode, so it remains the less common case.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* Hold a read lock while linking so copies can't write through links

Linking without the depot lock let a concurrent copy deployment write
through a link created moments earlier: copyFiles checks the destination
is free, a link to another artifact's depot file appears there, and
os.Create then truncates and overwrites that depot file, corrupting the
shared cache. TestConcurrentCopyAndLinkKeepDepotIntact reproduces it in
every run without the lock.

Make fsMutex an RWMutex: DeployViaLink holds it for reading, so links
still run concurrently, while copies, Put and Undeploy hold it for
writing. Also fix the non-constant format string in DeployViaCopy, which
made go test's vet step fail the whole package.

Add smartlink tests for an existing target, concurrent links to the same
destination, and the Link refactor's handling of directories and
symlinks.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Skip per-file metadata calls when linking

link() checked each file with IsDir, IsSymlink, MkdirUnlessExists on its
parent and TargetExists before linking it, plus another IsDir inside the
Windows linkFile. Each is a file open on Windows. Instead:

- take the entry type from the directory listing (os.ReadDir already
  has it), so only symlinks need a Stat;
- rely on the parent directory existing, since the recursion creates it
  (Link creates it for its single entry);
- drop the TargetExists pre-check and treat the link call's "already
  exists" error as the skip, which already covered concurrent links;
- check for a directory only on the symlink-to-directory path, which is
  the only one that can pass one to linkFile.

A regular file now costs one syscall: the link itself.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Bump github.com/klauspost/compress (indirect, via mholt/archives) from
v1.18.0 to v1.18.7, the first patched release for GHSA-259r-337f-4rfw
(integer overflow and out-of-bounds write in s2). Re-vendored.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* Support dependencies for encrypted private ingredients

`state publish --build` can now declare and edit the dependencies of an
encrypted private ingredient:

- Read [project].dependencies from pyproject.toml and publish them as
  runtime dependencies on language/python packages, with names normalized
  by the inventory API. Version ranges carry over (~= and ==X.* are
  expanded). OS and Python version markers become conditions
  (kernel/Windows|Linux|Darwin, language/python); unsupported markers are
  skipped with a notice. The wheel's METADATA records them as
  Requires-Dist.
- Read the --meta file before building, so a meta file's name, version
  and description feed the wheel (flags > meta file > pyproject.toml).
- Add `state export ingredient-meta <namespace>/<name>[@Version]`, which
  writes an existing ingredient's meta file for use with --edit --meta.

Fix --edit along the way: keep the meta file's dependencies and authors
instead of overwriting them with the previous revision's, keep inherited
dependency types and conditions, add the private-builder dependency only
once, and reject author/description changes on an existing version up
front instead of failing in the API. InventoryRequirementsToString no
longer panics on requirements without a version.

Add a readme walking through the publish and edit workflow.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Address review: keep inherited deps, edit the meta file's version

- When every pyproject.toml dependency is skipped, keep the dependencies
  already in place (on --edit, those inherited from the previous
  revision) instead of replacing them with an empty list.
- Let a meta file's version select the version to publish or edit, like
  --version (which still takes precedence). An exported meta file passed
  back with `state publish --edit --meta` without --build was editing
  the latest version instead of the exported one.

Add tests for both, for the up-front author/description rejection on
--edit, for InventoryRequirementsToString with requirements missing a
version or comparator, and for inherited conditions that carry
requirements.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Records the version branch -> beta -> release ladder, the changelog
convention, and the post-merge staging, Windows signing, S3 promotion
and verification steps, so the next release does not have to be
reverse-engineered from CI and branch history.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
master and beta have diverged, so merging master into a version branch
ships unreleased work. Also document how the promoter's scope differs
between beta and master.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@icanhasmath icanhasmath changed the title 0.48.2-RC1: changelog and release skill 0.48.2-RC1 Oct 21, 2026
@icanhasmath
icanhasmath changed the base branch from version/0-48-2-RC1 to beta October 21, 2026 21:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants