[simonfaltum]

Why

CLI 1.22 resolves the skills version with an unauthenticated call to GitHub's releases API. When GitHub rate limits that call, aitools list and plugin installation fail, and the VS Code agent picker ends up empty (failing CI).

The CLI already has an embedded fallback for this, but those two commands didn't use it. The fallback was also a hand-maintained constant (0.2.30) that drifts behind the real releases.

Changes

Before: a rate-limited release lookup made aitools list and plugin installs fail, and the embedded fallback was a compiled constant.

Now: aitools list falls back to the embedded skills version, plugin installs succeed and record the plugin version as unknown, and release builds embed the skills release that is latest at build time.

• .github/scripts/pin-cli-compat.py resolves "skills": "latest" with one authenticated gh api call and writes the result to a Go build overlay under .databricks/cli-compat/. GoReleaser and the build, snapshot and snapshot-release tasks build with that overlay. The checked-in cli-compat.json stays unchanged.
• Resolved entries carry skills_resolved_from_latest: true, so the CLI still tracks latest at runtime and keeps its downgrade checks. The concrete version is only used as the fallback.
• If the build-time lookup fails, "latest" stays in the manifest and the script warns. GoReleaser hides hook output, so release builds also write the warning to the GitHub Actions job summary.
• Whenever the embedded manifest still contains "latest" (failed lookup or a plain go build), the fallback is the newest pinned range below it, as if the "latest" rows weren't there. This replaces AgentSkillsFallbackVersion.
• aitools list uses GetSkillsRefWithFallback. Plugin installs don't need a skills version (the agent's marketplace picks it), so a failed lookup records the plugin version as unknown and keeps the existing state.Release.
• Latest tracking starts at CLI 1.23.0. CLI 1.22.x stays pinned to 0.2.32 (Pin CLI 1.22 skills to v0.2.32 #7159), since those binaries don't have the fallback.
• DATABRICKS_SKILLS_API_URL overrides the GitHub API base URL so tests can stub the release lookup.

Test plan

• ./task checks, ./task lint-q, ./task fmt-q, ./task doctest
• Unit tests for libs/clicompat, libs/aitools/..., cmd/aitools and internal/build (the first three also with the build overlay applied)
• New acceptance test acceptance/cmd/aitools/list-release-rate-limited: a 403 from the release API still lists skills at 0.2.32, with the full agent list
• New unit test: plugin install succeeds when the release lookup fails and records the plugin version as unknown
• Script doctests: successful resolution, failed lookups (missing gh, non-zero exit, timeout, invalid tags) leave "latest" in place, relative overlay paths
• Build through a symlinked checkout embeds the resolved manifest
• Script with gh missing on Python 3.9: the build continues and the warning reaches stderr and the job summary

This PR was written by Codex.

This pull request and its description were written by Isaac.

[eng-dev-ecosystem-bot]

Integration test report

Commit: 065fd66

Run: 39783963165

	Env	✅​pass	🙈​skip	Time
✅​	aws linux-2core-8gb	281	16	4:17
✅​	aws-windows-latest-4core-16gb	283	14	3:15
✅​	azure linux-2core-8gb	280	16	4:41
✅​	azure-windows-latest-4core-16gb	282	14	3:08
✅​	gcp linux-2core-8gb	281	16	4:05
✅​	gcp-windows-latest-4core-16gb	283	14	3:08

Top 3 slowest tests (at least 2 minutes):

duration	env	testname
3:03	azure-windows-latest-4core-16gb	TestAccept
3:02	aws-windows-latest-4core-16gb	TestAccept
2:59	gcp-windows-latest-4core-16gb	TestAccept

[github-actions]

Commit: 065fd664b | Workflow run

Regression test report

Selected 1 changed tests (limit=20, 0 not selected)

1 tests: regression=1 unreleased=0 coverage=0 failing=0

test	branch	main (66fce77)	latest (v1.22.0)
TestAccept/cmd/aitools/list-release-rate-limited	✅	❌	❌

TestAccept/cmd/aitools/list-release-rate-limited: branch ✅ | main (66fce77) ❌ | latest (v1.22.0) ❌

main (66fce77):

acceptance_test.go:1204: Substituted DATABRICKS_SKILLS_API_URL "$DATABRICKS_HOST" -> "http://127.0.0.1:46259" ("[DATABRICKS_HOST]")
    acceptance_test.go:1204: Substituted DATABRICKS_SKILLS_BASE_URL "$DATABRICKS_HOST" -> "http://127.0.0.1:46259" ("[DATABRICKS_HOST]")
    acceptance_test.go:1307: 
        	Error Trace:	/home/runner/work/cli/cli/libs/testdiff/testdiff.go:22
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1307
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1077
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:637
        	Error:      	Not equal: 
        	            	expected: "\n=== list falls back to the embedded skills version when the release lookup is rate limited\n\n>>> [CLI] aitools list --output json\nWarn: Could not resolve latest skills release (HTTP 403 fetching latest skills release); falling back to embedded skills version 0.2.32\n{\n  \"release\": \"0.2.32\",\n  \"skills\": [\n    \"test-stable\"\n  ],\n  \"agents\": [\n    \"claude-code\",\n    \"cursor\",\n    \"codex\",\n    \"opencode\",\n    \"copilot\",\n    \"antigravity\",\n    \"pi\",\n    \"gemini\",\n    \"goose\",\n    \"kiro\"\n  ]\n}\n"
        	            	actual  : "\n=== list falls back to the embedded skills version when the release lookup is rate limited\n\n>>> [CLI] aitools list --output json\n{\n  \"release\": \"0.2.32\",\n  \"skills\": [\n    \"test-stable\"\n  ],\n  \"agents\": [\n    \"claude-code\",\n    \"cursor\",\n    \"codex\",\n    \"opencode\",\n    \"copilot\",\n    \"antigravity\",\n    \"pi\",\n    \"gemini\",\n    \"goose\",\n    \"kiro\"\n  ]\n}\n"
        	            	
        	            	Diff:
        	            	--- Expected
        	            	+++ Actual
        	            	@@ -4,3 +4,2 @@
        	            	 >>> [CLI] aitools list --output json
        	            	-Warn: Could not resolve latest skills release (HTTP 403 fetching latest skills release); falling back to embedded skills version 0.2.32
        	            	 {
        	Test:       	TestAccept/cmd/aitools/list-release-rate-limited
        	Messages:   	cmd/aitools/list-release-rate-limited/output.txt vs /tmp/TestAcceptcmdaitoolslist-release-rate-limited737567045/002/output.txt
FAIL

latest (v1.22.0):

acceptance_test.go:1204: Substituted DATABRICKS_SKILLS_API_URL "$DATABRICKS_HOST" -> "http://127.0.0.1:36733" ("[DATABRICKS_HOST]")
    acceptance_test.go:1204: Substituted DATABRICKS_SKILLS_BASE_URL "$DATABRICKS_HOST" -> "http://127.0.0.1:36733" ("[DATABRICKS_HOST]")
    acceptance_test.go:1307: 
        	Error Trace:	/home/runner/work/cli/cli/libs/testdiff/testdiff.go:22
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1307
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1077
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:637
        	Error:      	Not equal: 
        	            	expected: "\n=== list falls back to the embedded skills version when the release lookup is rate limited\n\n>>> [CLI] aitools list --output json\nWarn: Could not resolve latest skills release (HTTP 403 fetching latest skills release); falling back to embedded skills version 0.2.32\n{\n  \"release\": \"0.2.32\",\n  \"skills\": [\n    \"test-stable\"\n  ],\n  \"agents\": [\n    \"claude-code\",\n    \"cursor\",\n    \"codex\",\n    \"opencode\",\n    \"copilot\",\n    \"antigravity\",\n    \"pi\",\n    \"gemini\",\n    \"goose\",\n    \"kiro\"\n  ]\n}\n"
        	            	actual  : "\n=== list falls back to the embedded skills version when the release lookup is rate limited\n\n>>> [CLI] aitools list --output json\nError: resolve latest skills release: Get \"https://api.github.com/repos/databricks/databricks-agent-skills/releases/latest\": Bad Request\n\nExit code: 1\n"
        	            	
        	            	Diff:
        	            	--- Expected
        	            	+++ Actual
        	            	@@ -4,21 +4,5 @@
        	            	 >>> [CLI] aitools list --output json
        	            	-Warn: Could not resolve latest skills release (HTTP 403 fetching latest skills release); falling back to embedded skills version 0.2.32
        	            	-{
        	            	-  "release": "0.2.32",
        	            	-  "skills": [
        	            	-    "test-stable"
        	            	-  ],
        	            	-  "agents": [
        	            	-    "claude-code",
        	            	-    "cursor",
        	            	-    "codex",
        	            	-    "opencode",
        	            	-    "copilot",
        	            	-    "antigravity",
        	            	-    "pi",
        	            	-    "gemini",
        	            	-    "goose",
        	            	-    "kiro"
        	            	-  ]
        	            	-}
        	            	+Error: resolve latest skills release: Get "https://api.github.com/repos/databricks/databricks-agent-skills/releases/latest": Bad Request
        	            	 
        	            	+Exit code: 1
        	            	+
        	Test:       	TestAccept/cmd/aitools/list-release-rate-limited
        	Messages:   	cmd/aitools/list-release-rate-limited/output.txt vs /tmp/TestAcceptcmdaitoolslist-release-rate-limited1155822872/002/output.txt
FAIL