Skip to content

fix(autocontextmemory):large message compression format adaptation#1180

Open
shiyiyue1102 wants to merge 10 commits intoagentscope-ai:mainfrom
shiyiyue1102:feature/large-message-compression-format-adaptation
Open

fix(autocontextmemory):large message compression format adaptation#1180
shiyiyue1102 wants to merge 10 commits intoagentscope-ai:mainfrom
shiyiyue1102:feature/large-message-compression-format-adaptation

Conversation

@shiyiyue1102
Copy link
Copy Markdown
Contributor

This pull request enhances the handling of large tool messages during the auto-context memory compression process to prevent model errors caused by tool message structures. It introduces logic to convert tool messages into plain text format before sending them for compression, and then reconstructs the appropriate tool message structure after summarization. Additionally, comprehensive tests are added to verify this behavior.

Improvements to tool message handling and compression:

  • In AutoContextMemory.java, large tool messages are now converted to plain text format before being sent to the model for compression, avoiding errors related to tool call/result detection during model inference. After summarization, the original tool message structure is preserved in the compressed message. [1] [2] [3] [4] [5]
  • A new helper method convertToolMessageToText is added to perform the conversion of tool messages to a text-based format suitable for model input.
  • The signature of generateLargeMessageSummary is updated to accept both the converted message and the original message, ensuring the original structure can be restored after compression.

Testing and verification:

  • A new test, testLargeToolMessageConvertedToTextDuringCompression, is added to verify that large tool messages are correctly converted to text during compression and that the model receives them in the expected format.
  • A CapturingTestModel test double is introduced to capture and inspect messages sent to the model during tests, ensuring the conversion logic works as intended.

Minor changes:

  • Import for ContentBlock added to support the new logic.
  • Minor whitespace cleanup in pom.xml.## AgentScope-Java Version

[The version of AgentScope-Java you are working on, e.g. 1.0.11, check your pom.xml dependency version or run mvn dependency:tree | grep agentscope-parent:pom(only mac/linux)]

Description

[Please describe the background, purpose, changes made, and how to test this PR]

Checklist

Please check the following items before code is ready to be reviewed.

  • Code has been formatted with mvn spotless:apply
  • All tests are passing (mvn test)
  • Javadoc comments are complete and follow project conventions
  • Related documentation has been updated (e.g. links, examples, etc.)
  • Code is ready for review

…e payload offloading

In offloadingLargePayload (Strategy 2/3), messages were offloaded as
plain TextBlock stubs regardless of role, which caused two bugs:

1. ASSISTANT messages with ToolUseBlock (tool_calls) were stripped of
   their ToolUseBlock when offloaded. The subsequent TOOL result messages
   then had no preceding tool_calls message, triggering DashScope 400:
   'messages with role tool must be a response to a preceding message
   with tool_calls'.

2. TOOL result messages were never actually compressed because
   Msg.getTextContent() only extracts top-level TextBlocks and returns
   empty string for TOOL messages (whose content is a ToolResultBlock).

Fix:
- Skip ASSISTANT+ToolUseBlock messages entirely; these pairs are handled
  exclusively by Strategy 1 (summaryToolsMessages).
- Handle TOOL result messages separately: extract output text from
  ToolResultBlock.output for size comparison, offload the original
  message, then rebuild a replacement that preserves the ToolResultBlock
  shell (id + name) while replacing only the output text with the
  offload hint. This keeps tool_call_id/name intact for API formatters.

Add three regression tests covering:
- Strategy 2/3 must not offload ASSISTANT tool-call messages as plain stubs
- Compressed TOOL result must preserve ToolResultBlock id and name
- Full conversation integrity: every TOOL result must follow a tool-call assistant

Change-Id: If62a76459d0bdb681cf04fbfd10c7df093b2ab8a
Co-developed-by: Qoder <noreply@qoder.com>
…arge tool results

When compressing a large TOOL result message, the replacement
ToolResultBlock was built without the original metadata, silently
dropping semantic flags such as agentscope_suspended and any custom
tool execution metadata written by tool implementations.

Pass originalResult.getMetadata() to ToolResultBlock.of() so all
metadata is carried through to the compressed stub.

Change-Id: I54e37a5b7e1dee6ec2de8462450016f9a1bf1124
Co-developed-by: Qoder <noreply@qoder.com>
… large message compression

- Add convertToolMessageToText() method to transform ToolUse and ToolResult
  messages into plain text ASSISTANT messages before compression
- Prevent model errors caused by tool call/tool result detection in
  compression context
- Preserve tool metadata (name, ID, arguments, output) in text format
- Add unit test to verify tool message conversion during compression
- All 180 tests pass successfully

Change-Id: I68b3a483b05f9fea14d31f93ffc442054c83645d
Co-developed-by: Qoder <noreply@qoder.com>
- Use MsgUtils.calculateMessageCharCount() instead of getTextContent()
  to correctly measure tool message size
- Convert tool messages to text format before calling generateLargeMessageSummary
- Add detailed logging to track large message detection
- Fix issue where tool messages were skipped due to empty getTextContent()
- All 180 tests pass successfully

Change-Id: I17cb10b3083fcf12357b13d6b07f1c810426ec8f
Co-developed-by: Qoder <noreply@qoder.com>
…ession

- Preserve ToolUseBlock structure (keep original tool call blocks)
- Preserve ToolResultBlock structure (keep id, name, replace only output)
- Compress only the content, maintain tool call/result format
- Add ContentBlock import
- Ensures compressed tool messages remain compatible with API formatters
- All 180 tests pass successfully

Change-Id: I162e124b9ee28d17905bea20f7340d48709b2594
Co-developed-by: Qoder <noreply@qoder.com>
…message

- Add originalMessage parameter to generateLargeMessageSummary()
- Use originalMessage (not converted message) to detect tool message type
- Fix issue where tool structure was lost after convertToolMessageToText()
- Ensure ToolUseBlock and ToolResultBlock structures are correctly preserved
- All 180 tests pass successfully

Change-Id: Ib76921c6aaee52eaae74e234ae1edc3d50ae8a11
Co-developed-by: Qoder <noreply@qoder.com>
- Remove AutoContextMemoryExample.java demo program
- Remove AUTO_CONTEXT_MEMORY_EXAMPLE.md documentation
- Remove autocontext-memory dependency from quickstart pom.xml
- Keep only core AutoContextMemory fixes in agentscope-extensions

Change-Id: I591c2d2cb453dca888c6179175181371414ff7c5
Co-developed-by: Qoder <noreply@qoder.com>
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adjusts AutoContextMemory’s large-message compression flow to avoid model errors triggered by tool-call/tool-result message structures by converting tool messages to plain text for summarization and then reconstructing tool message structures afterward.

Changes:

  • Detect large messages via character count across all content-block types (including tool blocks) and summarize them with the model.
  • Convert tool messages into plain-text messages for the compression prompt context, then rebuild tool-structured messages post-summarization.
  • Add tests with a capturing model double to inspect messages sent to the model during compression.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 7 comments.

File Description
agentscope-extensions/agentscope-extensions-autocontext-memory/src/main/java/io/agentscope/core/memory/autocontext/AutoContextMemory.java Adds tool-message-to-text conversion for compression context and reconstructs tool message structures after summarization.
agentscope-extensions/agentscope-extensions-autocontext-memory/src/test/java/io/agentscope/core/memory/autocontext/AutoContextMemoryTest.java Adds a capturing Model test double and a new test covering tool-message conversion during large-message compression.
agentscope-examples/quickstart/pom.xml Minor whitespace cleanup.

Comment on lines +656 to +661
// Preserve ToolUseBlock structure with compressed summary
List<ContentBlock> compressedBlocks = new ArrayList<>();
for (ContentBlock originalBlock : originalMessage.getContent()) {
if (originalBlock instanceof ToolUseBlock) {
ToolUseBlock originalToolUse = (ToolUseBlock) originalBlock;
// Keep original tool use block (tool calls are usually not too large)
Copy link

Copilot AI Apr 9, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the ToolUseBlock branch, the generated LLM summary (finalContent, including the offload tag) is never incorporated into the returned message. As written, a “summarized” large ToolUse message will keep the original ToolUseBlock(s) unchanged (potentially still huge) and provides no text for the model to see or for context_reload discovery. Consider embedding the summary/offload hint into the returned content (e.g., add a TextBlock alongside a minimized ToolUseBlock, or return a pure text Msg for tool-use summaries) so size is actually reduced and reload remains possible.

Suggested change
// Preserve ToolUseBlock structure with compressed summary
List<ContentBlock> compressedBlocks = new ArrayList<>();
for (ContentBlock originalBlock : originalMessage.getContent()) {
if (originalBlock instanceof ToolUseBlock) {
ToolUseBlock originalToolUse = (ToolUseBlock) originalBlock;
// Keep original tool use block (tool calls are usually not too large)
// Preserve ToolUseBlock structure and include the summary/offload hint as text
List<ContentBlock> compressedBlocks = new ArrayList<>();
if (!finalContent.isEmpty()) {
compressedBlocks.add(TextBlock.builder().text(finalContent).build());
}
for (ContentBlock originalBlock : originalMessage.getContent()) {
if (originalBlock instanceof ToolUseBlock) {
ToolUseBlock originalToolUse = (ToolUseBlock) originalBlock;
// Keep original tool use block while surfacing the compressed summary to the model

Copilot uses AI. Check for mistakes.
Comment on lines +674 to +678
List<ContentBlock> compressedBlocks = new ArrayList<>();
for (ContentBlock originalBlock : originalMessage.getContent()) {
if (originalBlock instanceof ToolResultBlock) {
ToolResultBlock originalToolResult = (ToolResultBlock) originalBlock;
// Replace output with compressed summary
Copy link

Copilot AI Apr 9, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When rebuilding a summarized ToolResult message, this loop only retains ToolResultBlock instances and drops any other ContentBlocks that may be present in the original message (e.g., TextBlocks before/after the tool result, which exist elsewhere in the codebase). This risks losing information during compression. Consider preserving non-ToolResult blocks (or at least appending them) while replacing only the ToolResultBlock.output with the compressed summary.

Copilot uses AI. Check for mistakes.
Comment on lines +678 to +683
// Replace output with compressed summary
ToolResultBlock compressedToolResult =
ToolResultBlock.builder()
.id(originalToolResult.getId())
.name(originalToolResult.getName())
.output(List.of(TextBlock.builder().text(finalContent).build()))
Copy link

Copilot AI Apr 9, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The rebuilt ToolResultBlock does not preserve originalToolResult.getMetadata() (contrast with the offload path later in this class that explicitly keeps ToolResult metadata). This can drop semantic flags like agentscope_suspended and other provider metadata. Preserve the original ToolResultBlock metadata when constructing the compressed ToolResultBlock.

Suggested change
// Replace output with compressed summary
ToolResultBlock compressedToolResult =
ToolResultBlock.builder()
.id(originalToolResult.getId())
.name(originalToolResult.getName())
.output(List.of(TextBlock.builder().text(finalContent).build()))
// Replace output with compressed summary while preserving tool result metadata
ToolResultBlock compressedToolResult =
ToolResultBlock.builder()
.id(originalToolResult.getId())
.name(originalToolResult.getName())
.output(List.of(TextBlock.builder().text(finalContent).build()))
.metadata(originalToolResult.getMetadata())

Copilot uses AI. Check for mistakes.
Comment on lines +738 to +740
if (toolUse.getContent() != null && !toolUse.getContent().isEmpty()) {
textContent.append("Arguments: ").append(toolUse.getContent()).append("\n");
}
Copy link

Copilot AI Apr 9, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For ToolUseBlock conversion, this uses toolUse.getContent() (raw streaming content) to populate “Arguments”, which is often null/empty for normal tool calls. This means the converted text sent to the model can omit the actual tool inputs (toolUse.getInput()). Consider serializing getInput() (and optionally also getContent() when present) so compression has the arguments needed for an accurate summary.

Suggested change
if (toolUse.getContent() != null && !toolUse.getContent().isEmpty()) {
textContent.append("Arguments: ").append(toolUse.getContent()).append("\n");
}
Object toolInput = toolUse.getInput();
String rawContent = toolUse.getContent();
if (toolInput != null) {
textContent.append("Arguments: ").append(String.valueOf(toolInput)).append("\n");
}
if (rawContent != null && !rawContent.isEmpty()
&& (toolInput == null || !rawContent.equals(String.valueOf(toolInput)))) {
textContent.append("Raw Content: ").append(rawContent).append("\n");
}

Copilot uses AI. Check for mistakes.
Comment on lines +756 to +760
if (toolResult.getOutput() != null) {
for (var outputBlock : toolResult.getOutput()) {
if (outputBlock instanceof TextBlock) {
String text = ((TextBlock) outputBlock).getText();
if (text != null) {
Copy link

Copilot AI Apr 9, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ToolResult conversion only extracts output from TextBlock instances. Tool outputs can contain other ContentBlock types (e.g., images), which would be silently omitted from the compression context and can degrade summarization. Consider adding a fallback representation for non-TextBlock outputs (type name, URL/metadata, or toString()), or at least a placeholder indicating omitted non-text content.

Copilot uses AI. Check for mistakes.
Comment on lines +2096 to +2100
// Verify that in compression context, TOOL role messages are converted to ASSISTANT
boolean foundConvertedToolMessage = false;
for (List<Msg> messages : capturedMessagesList) {
for (Msg msg : messages) {
// If this message contains tool result content, it should be ASSISTANT role
Copy link

Copilot AI Apr 9, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test computes foundConvertedToolMessage but never asserts it, so the test can pass even if no tool message was converted (it only asserts callCount > 0, twice). Add an assertion that foundConvertedToolMessage is true (and remove the redundant second callCount assertion) to ensure the test actually validates the conversion behavior.

Copilot uses AI. Check for mistakes.
.largePayloadThreshold(100) // low threshold to trigger compression
.lastKeep(5)
.minConsecutiveToolMessages(100) // disable Strategy 1
.minCompressionTokenThreshold(Integer.MAX_VALUE) // disable LLM compression
Copy link

Copilot AI Apr 9, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The comment // disable LLM compression is misleading here: minCompressionTokenThreshold only gates Strategy 1 (tool invocation compression). Strategy 5 (large message summarization) still calls the model regardless. Update the comment (or the config) so the test setup accurately reflects what is being disabled/enabled.

Suggested change
.minCompressionTokenThreshold(Integer.MAX_VALUE) // disable LLM compression
.minCompressionTokenThreshold(
Integer.MAX_VALUE) // disable Strategy 1 token-threshold gating only

Copilot uses AI. Check for mistakes.
@codecov
Copy link
Copy Markdown

codecov bot commented Apr 9, 2026

Codecov Report

❌ Patch coverage is 57.14286% with 36 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
...ope/core/memory/autocontext/AutoContextMemory.java 57.14% 24 Missing and 12 partials ⚠️

📢 Thoughts on this report? Let us know!


// Convert tool-related messages to text format to avoid triggering tool call/tool result
// detection in the compression context, which would cause model errors
Msg messageForCompression = convertToolMessageToText(message);
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Msg has already been processed by convertToolMessageToText in the outer method. No need for redundant processing.

* @return a text-formatted message if the original was a tool message, otherwise the original
* message unchanged
*/
private Msg convertToolMessageToText(Msg message) {
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This drops the content and only retains the tool use.

- Remove redundant convertToolMessageToText() call in summaryCurrentRoundLargeMessages()
- Let generateLargeMessageSummary() handle conversion internally (single responsibility)
- Simplify method signature: remove originalMessage parameter
- Tool message is converted once inside generateLargeMessageSummary()
- All 180 tests pass successfully

Change-Id: I0a67d7c0c0ec239caa344caee59f7fca1b5ec84e
Co-developed-by: Qoder <noreply@qoder.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants