Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
9e7b850
chore: dedupe the merged ScorerName enum in the OpenAPI import
quinn-galileo Oct 20, 2026
535f5e0
chore: regenerate the API client from the current Client API spec
quinn-galileo Oct 20, 2026
066f32d
fix: adapt hand-written code to the regenerated client
quinn-galileo Oct 20, 2026
2221540
test: parse a project whose permissions include configure_rate_limit
quinn-galileo Oct 20, 2026
de519d0
chore: changelog for the regenerated API client
quinn-galileo Oct 20, 2026
f114686
test: cover the ProjectCreate body sent by Projects.create
quinn-galileo Oct 20, 2026
9c7a5c6
chore: share the project response fixture and mark the client changes…
quinn-galileo Oct 20, 2026
3782fdb
chore: tighten the changelog for the regenerated client
quinn-galileo Oct 20, 2026
42eb8e5
chore: soften the removed-enum note
quinn-galileo Oct 20, 2026
5fb17e7
chore: tidy comments
quinn-galileo Oct 20, 2026
99eb31b
chore: keep enum values older servers still return in the OpenAPI import
quinn-galileo Oct 20, 2026
e5dac8a
chore: regenerate the API client with the restored enum values
quinn-galileo Oct 20, 2026
4b266c0
test: parse a legacy task_type and state test intent
quinn-galileo Oct 20, 2026
2bfc0f5
chore: complete the changelog for the regenerated client
quinn-galileo Oct 20, 2026
42aa4fd
chore: check every patched enum exists before the OpenAPI import patches
quinn-galileo Oct 20, 2026
454c0dd
chore: include StepType in the enum check and correct the ground_trut…
quinn-galileo Oct 20, 2026
a698e55
chore: rewrap a changelog line
quinn-galileo Oct 20, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
37 changes: 37 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,8 +17,45 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- Scorer requests now send `X-Galileo-Scorer-Capabilities: decision`, and `ScorerTypes` and `ModelType` include
`decision`, so decision scorers appear in list and get responses and decode.

### Changed

- **Breaking:** the low-level API client (`splunk_ao.resources`) is regenerated from the current Client API
spec. Code that imports generated symbols directly may need updating:
- `ScorerConfig` is now `RuntimeScorerConfig`. It keeps the old fields, adds new ones and makes `scorer_type`
optional. The field order changed, so a positional `ScorerConfig(id, scorer_type)` call must become
`RuntimeScorerConfig(id=..., scorer_type=...)`. Keyword construction now also sends the defaults
`ground_truth=False`, `threshold=0.5` and `requires_tools_in_llm_span=False`; building one from a scorer
with `from_dict` takes `ground_truth` from the scorer and leaves the other two unset.
`Experiments.create(scorers=...)`, `update_scorer_settings()` and `create_metric_configs()` take or return
`RuntimeScorerConfig`.
- `CoreScorerName` is renamed to `ScorerName`. The old `ScorerName`, whose members were internal
underscore-prefixed values (for example `ScorerName.VALUE_1 = "_context_adherence_luna"`), is gone; use the
public names such as `ScorerName.CONTEXT_ADHERENCE`.
- Removed endpoints: experiment metrics (`/projects/{project_id}/experiments/metrics`), experiment metric settings
(`/projects/{project_id}/experiments/{experiment_id}/metric_settings`) and the Vegas gateway integration
(`/integrations/vegas_gateway`).
- Removed models: the `Customized*GPTScorer` and `*Template` scorer families, `ChunkAttributionUtilizationScorer`,
`FineTunedScorer`, `RegisteredScorer`, `ScorerType`, `ScorersConfiguration`, `MetricCritiqueColumnar`,
`MetricCritiqueContent`, `ExtendedSessionRecordWithChildren` (replaced by
`ExtendedReadSessionRecordWithChildren`), `VegasGatewayIntegration` and `VegasGatewayIntegrationCreate`.
- Removed fields:
- `BaseScorerVersionResponse.input_type` (returned by `create_custom_llm_evaluator()` and the other evaluator
create functions), and `input_type`, `output_type` and `scoreable_node_types` on `BaseScorerVersionDB`.
- `BaseScorer.sub_scorers` and `ProjectCreate.create_example_templates`.
- `scorer_type` on `MetricComputing`, `MetricError`, `MetricFailed`, `MetricNotApplicable`, `MetricNotComputed`,
`MetricPending`, `MetricRollUp` and `MetricSuccess`, and `critique` on `MetricRollUp` and `MetricSuccess`.
- `prompt_scorers_configuration`, `prompt_customized_scorers_configuration`,
`prompt_finetuned_scorers_configuration`, `prompt_registered_scorers_configuration`, `sub_scorers`,
`store_metric_ids`, `stream_metrics` and `xray` on `CreateJobRequest` and `CreateJobResponse`.
- Removed enum members: `ProjectAction.UPDATE`, `ProjectType.LLM_MONITOR` and `ProjectType.PROMPT_EVALUATION`.

### Fixed

- The API client now knows the permission and enum values that current servers return, so reading projects and
other resources no longer fails on them. For example, `Projects().get(id=...)` raised
`ValueError: 'configure_rate_limit' is not a valid ControlResourceAction` (the project permission enum
`ProjectAction` lacked `configure_rate_limit`). Values that older servers still return, `vegas_gateway` for
integrations and task types 7, 9 and 12, are kept so reads from those servers keep working.
- `SplunkAOTracingProcessor` (OpenAI Agents) now records Responses API LLM spans, the Agents SDK
default, as chat messages: tool calls and tool results keep their own roles instead of one JSON
`user` message, the output is a single assistant message, and workflow/agent spans show the
Expand Down
Loading
Loading