Repository navigation
Support dependencies for encrypted private ingredients - #3857
Conversation
`state publish --build` can now declare and edit the dependencies of an encrypted private ingredient: - Read [project].dependencies from pyproject.toml and publish them as runtime dependencies on language/python packages, with names normalized by the inventory API. Version ranges carry over (~= and ==X.* are expanded). OS and Python version markers become conditions (kernel/Windows|Linux|Darwin, language/python); unsupported markers are skipped with a notice. The wheel's METADATA records them as Requires-Dist. - Read the --meta file before building, so a meta file's name, version and description feed the wheel (flags > meta file > pyproject.toml). - Add `state export ingredient-meta <namespace>/<name>[@Version]`, which writes an existing ingredient's meta file for use with --edit --meta. Fix --edit along the way: keep the meta file's dependencies and authors instead of overwriting them with the previous revision's, keep inherited dependency types and conditions, add the private-builder dependency only once, and reject author/description changes on an existing version up front instead of failing in the API. InventoryRequirementsToString no longer panics on requirements without a version. Add a readme walking through the publish and edit workflow. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
MDrakos
left a comment
There was a problem hiding this comment.
Requesting changes because --edit can now silently rewrite the wrong dependency list: one path drops the inherited deps, and the export round trip can land on a different version than the one exported.
Another note on tests: the new state export ingredient-meta command (FetchMeta), the up-front author/description rejection, and the nil-version crash fix in InventoryRequirementsToString have no tests, and the only pyproject-deps coverage is TestPublishBuildEncrypted, which per the description hasn't run yet. I think a unit test for the nil-requirement case and one for prepareEditRequest with a condition that carries requirements would be cheap and pin the crash fix.
2 inline comments below.
| if err != nil { | ||
| return errs.Wrap(err, "Could not read pyproject.toml dependencies") | ||
| } | ||
| reqVars.Dependencies = deps |
There was a problem hiding this comment.
If every pyproject.toml entry gets skipped (say the only dep has a platform_machine marker), pyprojectDependencies returns nil, and on --edit this line wipes the deps prepareEditRequest just inherited. The publish then goes out with only builder/private-builder. That contradicts the readme ("If neither the meta file nor pyproject.toml lists any, the previous revision's dependencies are kept") and it's a regression from master, where --edit --build kept them.
| reqVars.Dependencies = deps | |
| if len(deps) > 0 { | |
| reqVars.Dependencies = deps | |
| } |
There was a problem hiding this comment.
Good catch, fixed in 5dfae2a as suggested. The step now lives in applyPyprojectDependencies, which only replaces the dependencies when at least one survives translation. TestApplyPyprojectDependencies covers both no entries and all entries skipped, and fails with the guard removed.
Checked against the platform too: --edit --build on a version whose pyproject.toml only had a platform_machine dependency kept all of the previous revision's dependencies.
| return captain.NewCommand( | ||
| "ingredient-meta", | ||
| locale.Tl("export_ingredient_meta_title", "Exporting Ingredient Meta File"), | ||
| locale.Tl("export_ingredient_meta_description", "Export the meta file of a published ingredient, to modify and pass to '[ACTIONABLE]state publish --edit --meta[/RESET]'"), |
There was a problem hiding this comment.
This points users at state publish --edit --meta without --build, but on that path the meta file's version is never used: Run only looks up params.Version, so it falls through to the latest version and prepareEditRequest overwrites reqVars.Version with it. So exporting mypkg@1.0.0 while 1.1.0 is latest and publishing it back rewrites 1.1.0's dependencies with 1.0.0's list, no error. imo the fix is in Run: look up cmp.Or(params.Version, reqVars.Version) instead of params.Version, so the exported version is what gets edited (the --build path already does this through generateEncryptedArtifact).
There was a problem hiding this comment.
Agreed, fixed in 5dfae2a. Right after the meta file is read, params.Version = cmp.Or(params.Version, reqVars.Version), so the meta file's version drives the lookup, the edit and the published version, with --version still taking precedence. Doing it on params rather than only in the lookup keeps everything downstream in agreement, the same way --build already worked. As a side effect, publishing a new version from a meta file without --build no longer fails with "already exists".
Checked your scenario on the platform: with 0.0.1 and 0.0.2 published, exporting @0.0.1, changing a dependency and running state publish --edit --meta (no --build) created revision 2 of 0.0.1. 0.0.2 was unchanged.
- When every pyproject.toml dependency is skipped, keep the dependencies already in place (on --edit, those inherited from the previous revision) instead of replacing them with an empty list. - Let a meta file's version select the version to publish or edit, like --version (which still takes precedence). An exported meta file passed back with `state publish --edit --meta` without --build was editing the latest version instead of the exported one. Add tests for both, for the up-front author/description rejection on --edit, for InventoryRequirementsToString with requirements missing a version or comparator, and for inherited conditions that carry requirements. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
@MDrakos thanks, both inline issues are fixed in 5dfae2a (details in the threads). On tests:
Still not covered by unit tests:
|
* Support dependencies for encrypted private ingredients `state publish --build` can now declare and edit the dependencies of an encrypted private ingredient: - Read [project].dependencies from pyproject.toml and publish them as runtime dependencies on language/python packages, with names normalized by the inventory API. Version ranges carry over (~= and ==X.* are expanded). OS and Python version markers become conditions (kernel/Windows|Linux|Darwin, language/python); unsupported markers are skipped with a notice. The wheel's METADATA records them as Requires-Dist. - Read the --meta file before building, so a meta file's name, version and description feed the wheel (flags > meta file > pyproject.toml). - Add `state export ingredient-meta <namespace>/<name>[@Version]`, which writes an existing ingredient's meta file for use with --edit --meta. Fix --edit along the way: keep the meta file's dependencies and authors instead of overwriting them with the previous revision's, keep inherited dependency types and conditions, add the private-builder dependency only once, and reject author/description changes on an existing version up front instead of failing in the API. InventoryRequirementsToString no longer panics on requirements without a version. Add a readme walking through the publish and edit workflow. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Address review: keep inherited deps, edit the meta file's version - When every pyproject.toml dependency is skipped, keep the dependencies already in place (on --edit, those inherited from the previous revision) instead of replacing them with an empty list. - Let a meta file's version select the version to publish or edit, like --version (which still takes precedence). An exported meta file passed back with `state publish --edit --meta` without --build was editing the latest version instead of the exported one. Add tests for both, for the up-front author/description rejection on --edit, for InventoryRequirementsToString with requirements missing a version or comparator, and for inherited conditions that carry requirements. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
* Support dependencies for encrypted private ingredients `state publish --build` can now declare and edit the dependencies of an encrypted private ingredient: - Read [project].dependencies from pyproject.toml and publish them as runtime dependencies on language/python packages, with names normalized by the inventory API. Version ranges carry over (~= and ==X.* are expanded). OS and Python version markers become conditions (kernel/Windows|Linux|Darwin, language/python); unsupported markers are skipped with a notice. The wheel's METADATA records them as Requires-Dist. - Read the --meta file before building, so a meta file's name, version and description feed the wheel (flags > meta file > pyproject.toml). - Add `state export ingredient-meta <namespace>/<name>[@Version]`, which writes an existing ingredient's meta file for use with --edit --meta. Fix --edit along the way: keep the meta file's dependencies and authors instead of overwriting them with the previous revision's, keep inherited dependency types and conditions, add the private-builder dependency only once, and reject author/description changes on an existing version up front instead of failing in the API. InventoryRequirementsToString no longer panics on requirements without a version. Add a readme walking through the publish and edit workflow. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Address review: keep inherited deps, edit the meta file's version - When every pyproject.toml dependency is skipped, keep the dependencies already in place (on --edit, those inherited from the previous revision) instead of replacing them with an empty list. - Let a meta file's version select the version to publish or edit, like --version (which still takes precedence). An exported meta file passed back with `state publish --edit --meta` without --build was editing the latest version instead of the exported one. Add tests for both, for the up-front author/description rejection on --edit, for InventoryRequirementsToString with requirements missing a version or comparator, and for inherited conditions that carry requirements. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
state publish --build(encrypted private ingredients) had no practical way to set dependencies:--metawas read only after the wheel was built, and--editoverwrote whatever the meta file supplied. This adds dependency support frompyproject.toml, fixes--meta/--edit, and adds a command to export an ingredient's meta file for editing.Changes
Dependencies from
pyproject.toml(pyproject.go)[project].dependenciesbecome runtime dependencies onlanguage/pythonpackages. Names are normalized in one inventory API call (newmodel.FetchNormalizedNames).~=1.16→>=1.16,<2,==4.2.*→>=4.2,<4.3.sys_platform/platform_system/os_name == "nt"→kernel/Windows|Linux|Darwin;python_version/python_full_version→language/python.python_versionis major.minor only, so> "3.9"becomes>=3.10rather than>3.9(which would match 3.9.18).or,!=,extra,platform_machine, URL references) are skipped with a notice pointing to--depend.Requires-Dist.--depend*flags > meta file >pyproject.toml.--metawith--build: the meta file is now read before the wheel is built, so its name/version/description are used (flags > meta file >pyproject.toml). Previously a meta file without a matchingpyproject.tomlfailed with "missing metadata".--editfixes (publish.go)typeandconditions(both were dropped, so python's build/runtime/test entries collapsed into duplicates and Windows-only dependencies became unconditional).builder/private-builderis added only if not already inherited.Error received: <no value>.New command:
state export ingredient-meta <namespace>/<name>[@version]writes an existing ingredient's meta file (name, namespace, version, full dependency list with conditions) in the format--metareads, to edit and pass back withstate publish --edit --meta. Authors and description are omitted since they can't change on an existing version.Crash fix:
model.InventoryRequirementsToStringdereferenced requirements without a version (present on some public ingredients' conditions, e.g.click). It now skips them.Docs: internal/runners/publish/readme.md walks through publishing an encrypted ingredient with dependencies and editing them.
Testing
pyproject.tomlparsing.TestPublishBuildEncryptednow includes apyproject.tomldependency (six) that the package imports, so the test fails unless the dependency is published and installed. Not yet run under the integration-test account.private/ActiveStateBE/python/test-00N-greeting, with an org-key contract via env):pyproject.tomldependencies, then install and import: dependency installed, sentinel printed;python_version > "3.9"dependencies installed, Windows-only and< "3.11"not;--editwith a meta file,--editinheriting dependencies, and the export → edit →--edit --meta→ export round trip;--editpublishes; the same version without--editis refused.Not addressed
--edit, a meta file orpyproject.tomlreplaces the whole dependency list; there is no merge.Error received: <no value>; the real message is only in the log.🤖 Generated with Claude Code