Skip to content

fix(channel): refuse attachments that can't fit the chat's model, and explain context overflows - #481

Merged
iamnbutler merged 3 commits into
mainfrom
attachment-context
Oct 16, 2026
Merged

iamnbutler merged 3 commits into
mainfrom
attachment-context

Conversation

@iamnbutler

Copy link
Copy Markdown
Contributor

Closes #480. Part of #26 and #5.

A message whose attached text files (or PDF text) exceed the model's context failed its run, and every later run failed the same way because the transcript keeps the message.

At send time (packages/channel, runtime-neutral): ask and say estimate the attached files' tokens and refuse a message that can't fit. The error names the files, their size, and the limit:

license.txt is about 432K tokens; GPT-4o mini reads at most about 96K tokens of attachments in one message. Attach a smaller file or an excerpt, or pick a model with a larger context.

  • ask checks against the model it resolves (the requested one, the chat's current one, or the default). The check runs before configure, so a refused ask leaves the chat's model unchanged.
  • say checks against the chat's current model, if it has one.
  • The composer already keeps the draft and attachments when a request fails and shows the error in its toast. No app change was needed.

Recovery path: if a run fails with a context overflow (detected with pi-ai's isContextOverflow), the reply's error now ends with: "This chat no longer fits the model's context, so every run fails this way until you delete the message with the large attachment or pick a model with a larger context." This hint is added when replies are projected to clients and is never stored. That means error replies already stored by 0.0.48 channels show it too. Deleting the message (#467) already replaces it in context. Nothing about the stored format changes.

Product decisions:

  • Refuse instead of silently truncating. Truncated attachments would let the agent answer about a file it only partly read.
  • The estimate uses UTF-8 bytes ÷ 3, not pi-ai's chars ÷ 4. Keynote's multilingual license PDF is about 1.0M chars / 1.29 MB and was measured at about 466K Claude tokens (≈2.8 bytes/token). chars ÷ 4 would have estimated about 250K and let it through on a 272K model. Bytes ÷ 3 slightly over-counts English text, which is the safe direction.
  • The budget for attachments in one message is ¾ of the model's context window. The remaining quarter holds the system prompt, tools, compacted history, and the answer.
  • Images are not counted. The existing limits already cap them at 4 per message and downscale them.

Smoke-tested on an isolated host from source (temp ACE_HOME, ACE_CONFIG_HOME, separate keychain service), with real OpenAI calls and Keynote's real License.pdf extracted by pdf.js the same way the app extracts it:

  • Old channel: I created a channel with main (3978bf0, which matches 0.0.48's behavior) and asked gpt-4o-mini about the license text. That run overflowed, and a plain "hello" in the same chat afterwards also overflowed: the bug reproduced.
  • Restarted the same channel's worker on this branch. Replay and new runs show the overflow error with the hint. After deleting the oversized message, the next ask succeeded ("Hello!").
  • Fresh channel on this branch:
    • With gpt-4o-mini (128K), ask and say both refused the license (432K estimated) and a 743 KB English file (248K estimated), with the errors above.
    • A refused ask left the chat's model unchanged.
    • A 31 KB file was accepted and answered.
    • gpt-4.1 (1M) accepted the license and summarized it correctly.

Built in Ace

Built with Claude Code outside Ace, in a session already running there; no Ace gap involved.

… explain context overflows

A message whose attached files exceed the model's context failed its run and every later one,
since the transcript keeps it. ask and say now estimate attached files' tokens from their UTF-8
bytes and refuse a message over three quarters of the model's context window, naming the files
and the limit. A run that still overflows, such as one over a message stored by 0.0.48, ends with
an error that says to delete the message with the large attachment or pick a larger model.

Why: packages/channel/src/room.ts#function checkContext | estimate from UTF-8 bytes, not chars; pi-ai's chars/4 undercounts non-Latin text by ~2x, which is how multilingual PDFs overflowed
Why: packages/channel/src/channel.ts#function failed | overflow errors are explained in the projection, not stored, so replies already stored by older channels get the recovery hint too
@iamnbutler
iamnbutler merged commit c26faf2 into main Oct 16, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

An attachment larger than the model's context breaks every later run in the chat

1 participant