Fixes #911: handle prompt too long errors#920
Fixes #911: handle prompt too long errors#920hemant838 wants to merge 1 commit intosourcebot-dev:mainfrom
Conversation
…ting history Prevent context window overflow by truncating large file contents (default 100K chars) and capping message history (default 50 messages). When a context window error still occurs, detect provider-specific error messages and show a user-friendly message with actionable guidance. Fixes sourcebot-dev#911 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
WalkthroughThis PR implements context window management for the chat system by adding file content truncation, message history limiting, and context window error detection. Two new environment variables configure maximum file character limits and message history size, with corresponding logic applied throughout the chat pipeline and user-facing error messaging. Changes
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Possibly related PRs
Suggested reviewers
🚥 Pre-merge checks | ✅ 3✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 3
🧹 Nitpick comments (3)
packages/web/src/app/api/(server)/chat/route.ts (1)
109-128: Context-window errors are routed to Sentry before detection
logger.errorandSentry.captureExceptionfire unconditionally, so expected context-window errors (user-triggered, not bugs) generate Sentry noise. Move the context-window check before the capture.♻️ Proposed reorder
onError: (error: unknown) => { - logger.error(error); - Sentry.captureException(error); - if (error == null) { return 'unknown error'; } const errorMessage = (() => { if (typeof error === 'string') return error; if (error instanceof Error) return error.message; return JSON.stringify(error); })(); if (isContextWindowError(errorMessage)) { return CONTEXT_WINDOW_USER_MESSAGE; } + logger.error(error); + Sentry.captureException(error); return errorMessage; }🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@packages/web/src/app/api/`(server)/chat/route.ts around lines 109 - 128, The onError handler currently calls logger.error and Sentry.captureException before checking for context-window errors, causing expected user-triggered context-window issues to be sent to Sentry; change the flow in the onError function to first derive the errorMessage (compute string from error using the existing typeof/instanceof/JSON.stringify logic), then call isContextWindowError(errorMessage) and return CONTEXT_WINDOW_USER_MESSAGE immediately if true, and only after that call logger.error(error) and Sentry.captureException(error) and return the errorMessage for other errors; reference the onError handler, the error-to-string logic, isContextWindowError, CONTEXT_WINDOW_USER_MESSAGE, logger.error and Sentry.captureException when making the reorder.packages/web/src/features/chat/tools.ts (1)
127-136: Truncation notice inherits a line number fromaddLineNumbers
truncateFileContentappends\n\n... [truncated: showing X of Y lines]to the content. WhenaddLineNumbers(content)is then applied, that notice line itself gets a line number (e.g.,52:... [truncated: showing 51 of 100 lines]), which is a bit misleading. The same pattern occurs inagent.ts(Line 209).Consider placing the truncation notice outside the numbered block by applying
addLineNumbersbefore appending the notice insidetruncateFileContent, or appending the notice after theaddLineNumberscall:♻️ Suggested approach in the callers
- const { content } = truncateFileContent(response.source, env.SOURCEBOT_CHAT_FILE_MAX_CHARACTERS); + const { content, wasTruncated, notice } = truncateFileContent(response.source, env.SOURCEBOT_CHAT_FILE_MAX_CHARACTERS); return { ... - source: addLineNumbers(content), + source: addLineNumbers(wasTruncated ? content.slice(0, content.lastIndexOf('\n\n...')) : content) + + (wasTruncated ? notice : ''), ... };Alternatively, refactor
truncateFileContentto return{ body, notice }separately, so callers can choose where to place the notice relative to line-numbered output.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@packages/web/src/features/chat/tools.ts` around lines 127 - 136, The truncation notice is being numbered because callers (map in this file and agent.ts) call addLineNumbers(content) after truncateFileContent adds the "\n\n... [truncated...]" line; fix by either (A) applying addLineNumbers to the truncated body first and appending the truncation notice afterwards in this mapper (i.e., call truncateFileContent(response.source) -> use returned body with addLineNumbers(body) then concat the notice string without numbering), or (B) refactor truncateFileContent to return an object like { body, notice } so callers such as the mapper in packages/web/src/features/chat/tools.ts and the function in agent.ts can call addLineNumbers(body) and then append notice outside the numbered block; update uses of truncateFileContent and preserve existing behaviour of FileSourceResponse mapping (path, repo, language, source, revision).packages/web/src/features/chat/utils.ts (1)
200-211:/token.?limit/iand/max_tokens/ican produce false positives
/token.?limit/imatches phrases like "You've exceeded the token limit for your subscription tier" (a billing/rate-limit error, not a context-window error)./max_tokens/imatches "Increase max_tokens to see the full response" (a configuration note). Both could cause a context-window message to be displayed for unrelated errors.♻️ Tighten the patterns
- /token.?limit/i, - /max_tokens/i, + /context.?token.?limit/i, + /max_tokens.*exceeded/i,Or simply remove them if the remaining patterns already cover the real providers you need.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@packages/web/src/features/chat/utils.ts` around lines 200 - 211, The patterns `/token.?limit/i` and `/max_tokens/i` in CONTEXT_WINDOW_ERROR_PATTERNS are too broad and cause false positives (billing/configuration messages flagged as context-window errors); fix by either removing these two entries from the CONTEXT_WINDOW_ERROR_PATTERNS array or tightening them to more specific regexes that only match true context-window messages (e.g., replace `/token.?limit/i` with a stricter pattern like `/exceeds? (the )?maximum token(?:s)? for (?:the )?context/i` and change `/max_tokens/i` to `\bmax[_-]?tokens\b` only if you need config-key matches), and ensure CONTEXT_WINDOW_ERROR_PATTERNS still covers provider-specific phrases used elsewhere in the code (refer to the CONTEXT_WINDOW_ERROR_PATTERNS constant to apply the change).
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@packages/shared/src/env.server.ts`:
- Around line 234-235: The env schema allows zero/negative values because
numberSchema (z.coerce.number()) has no minimum, causing slice(-0)/slice(0) and
full truncation edge-cases; update the SOURCEBOT_CHAT_MAX_MESSAGE_HISTORY and
SOURCEBOT_CHAT_FILE_MAX_CHARACTERS entries to enforce a sensible lower bound
(e.g., add .min(1) to numberSchema before .default or use
numberSchema.min(1).default(...)) so invalid 0/negative values are rejected, or
alternatively add an explicit guard in the chat trimming logic (in route.ts) to
treat <=0 as 1 or skip trimming; update the schema entries
SOURCEBOT_CHAT_MAX_MESSAGE_HISTORY and SOURCEBOT_CHAT_FILE_MAX_CHARACTERS
accordingly.
In `@packages/web/src/app/api/`(server)/chat/route.ts:
- Around line 208-211: The current trim uses slice(-maxMessages) which can start
with an assistant message and break providers that require the first turn to be
"user"; change the trimming logic around messageHistory, maxMessages and
trimmedMessageHistory so you compute a start index = messageHistory.length -
maxMessages, then if messageHistory[start].role === "assistant" increment start
by 1 (to ensure the first retained turn is a user) before slicing; keep other
behavior the same when messageHistory.length <= maxMessages.
In `@packages/web/src/features/chat/utils.test.ts`:
- Around line 373-374: Update the stale inline comment to match the actual test
limit: change the comment that currently reads "Limit of 20 characters" to
reflect the 15-character limit used in the test call to
truncateFileContent(source, 15), ensuring the comment accurately describes the
expected behavior for truncateFileContent in this test.
---
Nitpick comments:
In `@packages/web/src/app/api/`(server)/chat/route.ts:
- Around line 109-128: The onError handler currently calls logger.error and
Sentry.captureException before checking for context-window errors, causing
expected user-triggered context-window issues to be sent to Sentry; change the
flow in the onError function to first derive the errorMessage (compute string
from error using the existing typeof/instanceof/JSON.stringify logic), then call
isContextWindowError(errorMessage) and return CONTEXT_WINDOW_USER_MESSAGE
immediately if true, and only after that call logger.error(error) and
Sentry.captureException(error) and return the errorMessage for other errors;
reference the onError handler, the error-to-string logic, isContextWindowError,
CONTEXT_WINDOW_USER_MESSAGE, logger.error and Sentry.captureException when
making the reorder.
In `@packages/web/src/features/chat/tools.ts`:
- Around line 127-136: The truncation notice is being numbered because callers
(map in this file and agent.ts) call addLineNumbers(content) after
truncateFileContent adds the "\n\n... [truncated...]" line; fix by either (A)
applying addLineNumbers to the truncated body first and appending the truncation
notice afterwards in this mapper (i.e., call
truncateFileContent(response.source) -> use returned body with
addLineNumbers(body) then concat the notice string without numbering), or (B)
refactor truncateFileContent to return an object like { body, notice } so
callers such as the mapper in packages/web/src/features/chat/tools.ts and the
function in agent.ts can call addLineNumbers(body) and then append notice
outside the numbered block; update uses of truncateFileContent and preserve
existing behaviour of FileSourceResponse mapping (path, repo, language, source,
revision).
In `@packages/web/src/features/chat/utils.ts`:
- Around line 200-211: The patterns `/token.?limit/i` and `/max_tokens/i` in
CONTEXT_WINDOW_ERROR_PATTERNS are too broad and cause false positives
(billing/configuration messages flagged as context-window errors); fix by either
removing these two entries from the CONTEXT_WINDOW_ERROR_PATTERNS array or
tightening them to more specific regexes that only match true context-window
messages (e.g., replace `/token.?limit/i` with a stricter pattern like
`/exceeds? (the )?maximum token(?:s)? for (?:the )?context/i` and change
`/max_tokens/i` to `\bmax[_-]?tokens\b` only if you need config-key matches),
and ensure CONTEXT_WINDOW_ERROR_PATTERNS still covers provider-specific phrases
used elsewhere in the code (refer to the CONTEXT_WINDOW_ERROR_PATTERNS constant
to apply the change).
| SOURCEBOT_CHAT_FILE_MAX_CHARACTERS: numberSchema.default(100_000), | ||
| SOURCEBOT_CHAT_MAX_MESSAGE_HISTORY: numberSchema.default(50), |
There was a problem hiding this comment.
Missing lower-bound validation; SOURCEBOT_CHAT_MAX_MESSAGE_HISTORY=0 silently disables trimming
numberSchema is z.coerce.number() with no minimum. For SOURCEBOT_CHAT_MAX_MESSAGE_HISTORY, a value of 0 produces a JavaScript gotcha: Array.prototype.slice(-0) is identical to slice(0), returning the full array — so history trimming is silently skipped rather than capping to zero messages. Negative values produce similarly unexpected results.
For SOURCEBOT_CHAT_FILE_MAX_CHARACTERS, 0 would truncate every file to an empty body with just the notice.
🛡️ Proposed fix
- SOURCEBOT_CHAT_FILE_MAX_CHARACTERS: numberSchema.default(100_000),
- SOURCEBOT_CHAT_MAX_MESSAGE_HISTORY: numberSchema.default(50),
+ SOURCEBOT_CHAT_FILE_MAX_CHARACTERS: numberSchema.min(1).default(100_000),
+ SOURCEBOT_CHAT_MAX_MESSAGE_HISTORY: numberSchema.min(1).default(50),Alternatively, guard against maxMessages <= 0 in route.ts before performing the slice.
📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| SOURCEBOT_CHAT_FILE_MAX_CHARACTERS: numberSchema.default(100_000), | |
| SOURCEBOT_CHAT_MAX_MESSAGE_HISTORY: numberSchema.default(50), | |
| SOURCEBOT_CHAT_FILE_MAX_CHARACTERS: numberSchema.min(1).default(100_000), | |
| SOURCEBOT_CHAT_MAX_MESSAGE_HISTORY: numberSchema.min(1).default(50), |
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@packages/shared/src/env.server.ts` around lines 234 - 235, The env schema
allows zero/negative values because numberSchema (z.coerce.number()) has no
minimum, causing slice(-0)/slice(0) and full truncation edge-cases; update the
SOURCEBOT_CHAT_MAX_MESSAGE_HISTORY and SOURCEBOT_CHAT_FILE_MAX_CHARACTERS
entries to enforce a sensible lower bound (e.g., add .min(1) to numberSchema
before .default or use numberSchema.min(1).default(...)) so invalid 0/negative
values are rejected, or alternatively add an explicit guard in the chat trimming
logic (in route.ts) to treat <=0 as 1 or skip trimming; update the schema
entries SOURCEBOT_CHAT_MAX_MESSAGE_HISTORY and
SOURCEBOT_CHAT_FILE_MAX_CHARACTERS accordingly.
| const maxMessages = env.SOURCEBOT_CHAT_MAX_MESSAGE_HISTORY; | ||
| const trimmedMessageHistory = messageHistory.length > maxMessages | ||
| ? messageHistory.slice(-maxMessages) | ||
| : messageHistory; |
There was a problem hiding this comment.
slice(-maxMessages) can produce a history that starts with an assistant message
messageHistory follows the pattern [user₁, assistant₁, user₂, assistant₂, …, userₙ]. When messageHistory.length > maxMessages and maxMessages is even (e.g. 50), slice(-50) drops the oldest user message and the resulting history begins with an orphaned assistant turn. Providers like Anthropic's Messages API require the first turn to be "user" and will reject the request with an error when this constraint is violated.
🐛 Proposed fix — ensure trimmed history always starts with a user message
const maxMessages = env.SOURCEBOT_CHAT_MAX_MESSAGE_HISTORY;
- const trimmedMessageHistory = messageHistory.length > maxMessages
- ? messageHistory.slice(-maxMessages)
- : messageHistory;
+ let trimmedMessageHistory = messageHistory.length > maxMessages
+ ? messageHistory.slice(-maxMessages)
+ : messageHistory;
+ // Providers (e.g., Anthropic) require the first message to be from the user.
+ // If trimming produced an assistant-first sequence, drop the leading assistant turn.
+ if (trimmedMessageHistory.length > 0 && trimmedMessageHistory[0].role === 'assistant') {
+ trimmedMessageHistory = trimmedMessageHistory.slice(1);
+ }🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@packages/web/src/app/api/`(server)/chat/route.ts around lines 208 - 211, The
current trim uses slice(-maxMessages) which can start with an assistant message
and break providers that require the first turn to be "user"; change the
trimming logic around messageHistory, maxMessages and trimmedMessageHistory so
you compute a start index = messageHistory.length - maxMessages, then if
messageHistory[start].role === "assistant" increment start by 1 (to ensure the
first retained turn is a user) before slicing; keep other behavior the same when
messageHistory.length <= maxMessages.
| // Limit of 20 characters: "line 1\nline 2\nline 3" is 20 chars | ||
| const result = truncateFileContent(source, 15); |
There was a problem hiding this comment.
Stale comment — limit is 15, not 20
The inline comment says "Limit of 20 characters" but the call uses truncateFileContent(source, 15). The string 'line 1\nline 2\nline 3' is indeed 20 chars (describing the full third-line boundary), but the test limit is 15. The comment is misleading.
📝 Suggested fix
- // Limit of 20 characters: "line 1\nline 2\nline 3" is 20 chars
+ // Limit of 15 characters: last newline before index 15 is at index 13 ("line 1\nline 2")
const result = truncateFileContent(source, 15);📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| // Limit of 20 characters: "line 1\nline 2\nline 3" is 20 chars | |
| const result = truncateFileContent(source, 15); | |
| // Limit of 15 characters: last newline before index 15 is at index 13 ("line 1\nline 2") | |
| const result = truncateFileContent(source, 15); |
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@packages/web/src/features/chat/utils.test.ts` around lines 373 - 374, Update
the stale inline comment to match the actual test limit: change the comment that
currently reads "Limit of 20 characters" to reflect the 15-character limit used
in the test call to truncateFileContent(source, 15), ensuring the comment
accurately describes the expected behavior for truncateFileContent in this test.
1 participant
fix(chat): handle prompt too long errors by truncating files and limiting history
Prevent context window overflow by truncating large file contents (default 100K chars) and capping message history (default 50 messages). When a context window error still occurs, detect provider-specific error messages and show a user-friendly message with actionable guidance.
Summary by CodeRabbit
New Features
Improvements