[BOT ISSUE] Anthropic prompt caching usage metrics not captured

[braintrust-bot]

Summary

The Anthropic instrumentation extracts input_tokens and output_tokens from the Messages API response usage object, but does not capture cache_creation_input_tokens or cache_read_input_tokens. These are standard fields in the Anthropic Messages API response when prompt caching is used, and they are important for understanding cost and cache effectiveness.

What is missing

In InstrumentationSemConv.tagAnthropicResponse() (lines 204–231), the usage extraction block only handles:

if (usage.has("input_tokens")) metrics.put("prompt_tokens", usage.get("input_tokens"));
if (usage.has("output_tokens")) metrics.put("completion_tokens", usage.get("output_tokens"));

It should additionally extract:

• cache_creation_input_tokens — tokens written to the prompt cache on this request
• cache_read_input_tokens — tokens read from an existing prompt cache

These fields are present in every Anthropic Messages API response that uses prompt caching (via cache_control blocks in the request). Without them, users cannot observe cache hit rates or understand the cost breakdown of cached vs. uncached tokens in Braintrust traces.

Note: the full response JSON is stored in braintrust.output_json, so the raw data is technically present — but it is not extracted into braintrust.metrics where Braintrust's UI and cost calculations can use it.

Braintrust docs status

• Braintrust docs generically mention cached token support for cost calculations: unclear whether this is expected to work for Java SDK specifically
• No Java SDK-specific documentation on prompt caching metric extraction

Upstream sources

• Anthropic prompt caching docs: https://docs.anthropic.com/en/docs/build-with-claude/prompt-caching
• Anthropic Messages API response usage object: https://docs.anthropic.com/en/api/messages — documents cache_creation_input_tokens and cache_read_input_tokens fields
• Anthropic Java SDK: com.anthropic:anthropic-java:2.2.0 (depended on by this repo)

Local files inspected

• braintrust-sdk/src/main/java/dev/braintrust/instrumentation/InstrumentationSemConv.java — lines 204–231 (tagAnthropicResponse only extracts input_tokens and output_tokens)
• braintrust-sdk/instrumentation/anthropic_2_2_0/src/main/java/dev/braintrust/instrumentation/anthropic/v2_2_0/TracingHttpClient.java — HTTP-level instrumentation, delegates to InstrumentationSemConv
• braintrust-sdk/instrumentation/anthropic_2_2_0/src/test/java/.../BraintrustAnthropicTest.java — no test cases exercise prompt caching responses