Skip to content

fix(models): keep the streamed thought signature on its own part - #7588

Open
AtulJoshi1206 wants to merge 1 commit into
google:mainfrom
AtulJoshi1206:fix/interactions-stream-thought-signature
Open

AtulJoshi1206 wants to merge 1 commit into
google:mainfrom
AtulJoshi1206:fix/interactions-stream-thought-signature

Conversation

@AtulJoshi1206

Copy link
Copy Markdown
Contributor

Please ensure you have read the contribution guide before creating a pull request.

Link to Issue or Description of Change

No existing issue; described below following the bug-report structure.

Describe the Bug:

With Gemini(use_interactions_api=True), streaming (StreamingMode.SSE) and thought summaries on, _handle_thought_signature writes each thought_signature delta onto the last thought part collected so far. That part is the summary's text part, Part(text=summary, thought=True, thought_signature=sig).

On the request that carries the tool result:

  • _get_latest_user_contents carries that part forward.
  • _convert_part_to_interaction_content checks part.text first and turns it into a model_output text step.
  • So the signature is dropped, and the private summary is replayed as model output.

Separately, when one streamed turn has more than one thought step, a later signature overwrites the earlier step's signature.

The non-streaming converter already avoids both. Commit 51f21e7 notes that "a part that also carries text is re-encoded as a text step, which drops the signature".

Impact:

Gemini 3 rejects the follow-up request unless the signature comes back verbatim. So with summaries on, every streamed tool-calling turn breaks on the request that carries the tool result, and the model sees its own private summary as prior output. ManagedAgent streams through the same converter and persists the same shape.

Observed Behavior:

After a tool call, the second interactions.create(...) (previous_interaction_id='int_1') sends:

[{"type": "model_output", "content": [{"type": "text", "text": "I should call the tool."}]},
 {"type": "function_result", "name": "get_weather", "call_id": "call_1", "result": {"weather": "sunny"}, "is_error": false}]

Expected Behavior:

The same steps the non-streaming path sends for that turn:

[{"type": "thought", "signature": "<signature>"},
 {"type": "function_result", "name": "get_weather", "call_id": "call_1", "result": {"weather": "sunny"}, "is_error": false}]

Solution:

_handle_thought_signature now always appends the signature as its own text-free Part(thought=True, thought_signature=...), the same as the non-streaming converter.

  • Summary parts keep their text and still stream as partials, but never carry a signature.
  • Each thought step keeps its own signature.
  • Partial events and the request encoder are unchanged.

I considered fixing this in the encoder instead, by emitting a thought step for text + thought + signature parts. I didn't, for two reasons: it can't recover a signature the stream handler has already overwritten, and it would change the encoding contract for parts from other producers. The producer-side fix matches the design in 51f21e7.

Testing Plan

Unit Tests:

  • I have added or updated unit tests for my change.
  • All unit tests pass locally.

All in tests/unittests/models/test_interactions_utils.py:

  • test_thought_signature_delta_after_a_summary_gets_its_own_part replaces test_thought_signature_delta_attaches_to_last_thought, which asserted the old behavior.
  • test_thought_signature_deltas_of_two_thought_steps_are_both_kept.
  • TestThoughtSignatureRoundTrip::test_signature_is_sent_back_with_the_tool_result[streamed|unary] feeds the same turn, streamed and unary, through _get_latest_user_contents and _convert_contents_to_steps. The unary case passes before and after, so it pins parity.
  • TestThoughtSignatureRoundTrip::test_runner_sends_the_streamed_signature_back runs InMemoryRunner + LlmAgent + Gemini(use_interactions_api=True) over SSE against a scripted Interactions stream, and asserts on the second interactions.create call.
# origin/main
FAILED ...::test_thought_signature_delta_after_a_summary_gets_its_own_part - assert 1 == 2
FAILED ...::test_thought_signature_deltas_of_two_thought_steps_are_both_kept - assert [b'second-sig', None, None] == [b'first-sig', None, b'second-sig', None]
FAILED ...::TestThoughtSignatureRoundTrip::test_signature_is_sent_back_with_the_tool_result[streamed] - assert ['model_output', 'function_result'] == ['thought', 'function_result']
FAILED ...::TestThoughtSignatureRoundTrip::test_runner_sends_the_streamed_signature_back - assert ['model_output', 'function_result'] == ['thought', 'function_result']

# with this change
pytest tests/unittests/models/test_interactions_utils.py   # 198 passed
pytest tests/unittests/models                              # 1565 passed

Checklist

  • I have read the CONTRIBUTING.md document.
  • I have performed a self-review of my own code.
  • I have commented my code, particularly in hard-to-understand areas.
  • I have added tests that prove my fix is effective or that my feature works.
  • New and existing unit tests pass locally with my changes.
  • I have manually tested my changes end-to-end. (The runner test uses a scripted Interactions stream, not the live API.)
  • Any dependent changes have been merged and published in downstream modules. (none)

Additional context

This is independent of #7508, which merges streamed text deltas per step. The two touch different functions; with this change, the summary parts that #7508 merges no longer carry a signature.

🤖 Generated with Claude Code

With thought summaries enabled, the Interactions streaming converter wrote
each thought signature onto the last thought part it had collected, which is
the summary's text part. The request encoder turns any part that carries text
into a text step, so the request after a tool call sent the private summary
back as model output and dropped the signature that Gemini 3 needs. When a
turn has more than one thought step, a later signature also overwrote the
earlier one.

Append every signature as a text-free thought part, which is what the
non-streaming converter already produces.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants