Repository navigation
PR #62 follow-up 16: match resolve_dataset names on token boundaries - #68
Merged
Merged
Conversation
resolve_dataset awarded the 750-point "explicitly mentioned" score whenever a dataset name occurred anywhere inside the query, so `api` matched inside "rapid" and resolved with confidence 0.95. Require the name to be a whole token, treating `-` and `_` as part of dataset names. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… fix/pr62-16-resolve-dataset-token-match # Conflicts: # docs/tool-api.md
… fix/pr62-16-resolve-dataset-token-match
…ataset-token-match
…ataset-token-match # Conflicts: # docs/tool-api.md # src/tool_executor.rs
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
resolve_dataset(resolve_dataset_candidatesinsrc/tool_executor.rs) gave a dataset 750 points ("dataset name explicitly mentioned in the question") whenever its name appeared anywhere in the query, including inside another word. With visible datasets[api, web], the queryshow rapid failuresmatchedapiinside "rapid". That madeapithe unique top candidate, so the tool returnedstatus: "resolved"with confidence 0.95 when it should have reported the query as ambiguous.Fix
The mention score now requires the dataset name to appear as a whole token. A new helper,
query_mentions_dataset_name, accepts a match only when the characters on both sides are not name characters ([A-Za-z0-9_-]). Dataset names often contain-and_, so those count as part of a name:otel-metricsis mentioned inshow otel-metrics, but not inotel-metrics-v2ormy_otel-metrics. The exact-match, reverse-substring (500) and per-term (25) scoring are unchanged.docs/tool-api.mdnow says that a name only counts as mentioned when it is a whole word.Tests
New test
shared_dataset_resolution_requires_whole_name_mentions:show rapid failuresno longer resolvesapi. This assertion fails without the fix.show api errorsanderrors in (api)?still resolveapi.show otel-metrics-v2 errorsandshow my_otel-metricsno longer resolveotel-metrics. Both fail without the fix.The existing test
shared_dataset_resolution_does_not_select_an_arbitrary_fallbackstill passes.Upstream fidelity
This changes code inherited from upstream parseablehq/parseable 32979cb (parseablehq#1831). The change is one call site plus a small free function, so it should merge cleanly with future upstream updates. Proposing the same fix upstream is a good idea; a draft issue is ready, but nothing has been filed.
Stacking
This PR is stacked on #62 (base
upstream/tool-catalog-1831). Retarget it tomainonce #62 merges.🤖 Generated with Claude Code
Need help on this PR? Tag
@codesmith-botwith what you need.