Skip to content

fix(node): Preserve CallbackManager handlers in LangChain instrumentation#20849

Open
mdnanocom wants to merge 7 commits into
getsentry:developfrom
mdnanocom:fix/langchain-callbackmanager-preservation
Open

fix(node): Preserve CallbackManager handlers in LangChain instrumentation#20849
mdnanocom wants to merge 7 commits into
getsentry:developfrom
mdnanocom:fix/langchain-callbackmanager-preservation

Conversation

@mdnanocom
Copy link
Copy Markdown

@mdnanocom mdnanocom commented May 12, 2026

Summary

augmentCallbackHandlers in packages/node/src/integrations/tracing/langchain/instrumentation.ts previously fell into its typeof handlers === 'object' branch whenever options.callbacks was a single object and wrapped it into [handlers, sentryHandler]. When that object was a LangChain CallbackManager, downstream code treats the whole manager as one opaque handler — its inheritable children (notably LangGraph's StreamMessagesHandler installed by streamMode: ['messages'], and the LangSmith tracer) are never unpacked, so per-token streaming events and nested tracing silently disappear.

This PR detects CallbackManager via duck-typing (avoids coupling to a specific @langchain/core resolution) and registers Sentry's handler as an inheritable child on a copy via .addHandler(handler, true). The manager's existing children continue to receive handleLLMNewToken and friends.

Repro

A LangGraph compiled graph + ChatOpenAI (or any LangChain provider with bindTools(...)), driven through @ai-sdk/langchain's toUIMessageStream:

const stream = await graph.stream(
  { messages },
  { streamMode: ['values', 'messages'] },
);
return createUIMessageStreamResponse({ stream: toUIMessageStream(stream) });
  • Without this fix: the SSE output collapses to a single aggregated text-delta at end of run. Probing inside _streamResponseChunks shows the per-token chunks fire runManager.handleLLMNewToken correctly, but the LLM-level run-manager's handlers list contains only [CallbackManager, SentryCallbackHandler, langchain_tracer]StreamMessagesHandler is missing.
  • With this fix: every token is delivered as the model produces it (verified end-to-end against gpt-5.5 + useResponsesApi: true + bindTools and against gpt-4o-mini with no tools).

Tests

Added packages/node/test/integrations/tracing/langchain.test.ts with 7 unit tests covering:

  • undefined callbacks → single-handler array
  • pre-existing array → append (idempotent)
  • CallbackManager → preserves children, copies rather than mutates, deduplicates
  • opaque non-manager object → returned unchanged

Verify

  • Added unit tests for the fixed behaviour
  • yarn build succeeds
  • yarn test:unit passes (381/381 in packages/node)
  • No related issue currently open — happy to open one if preferred.

…tion

`augmentCallbackHandlers` previously wrapped a `CallbackManager` instance
into `[manager, sentryHandler]` whenever `options.callbacks` was a single
object. LangChain downstream then treats the whole manager as one opaque
handler — its inheritable children (notably LangGraph's
`StreamMessagesHandler` installed by `streamMode: ['messages']`, plus the
LangSmith tracer) are never unpacked, so per-token streaming events and
nested tracing silently disappear.

Detect `CallbackManager` via duck-typing (avoids coupling to a specific
`@langchain/core` resolution) and register Sentry's handler as an
inheritable child on a copy, so the manager's existing handlers continue
to receive `handleLLMNewToken` and friends.

Repro: LangGraph compiled graph + `ChatOpenAI` (or any provider with
`bindTools(...)`), `graph.stream(..., { streamMode: ['values','messages'] })`
piped through `@ai-sdk/langchain`'s `toUIMessageStream`. Without the fix,
the SSE output collapses to a single aggregated `text-delta`. With the
fix, every token is delivered as the model produces it.
@mdnanocom mdnanocom requested a review from a team as a code owner May 12, 2026 19:00
@mdnanocom mdnanocom force-pushed the fix/langchain-callbackmanager-preservation branch from 92409bd to 4c74bfa Compare May 12, 2026 19:03
Comment thread packages/node/src/integrations/tracing/langchain/instrumentation.ts Outdated
In practice CallbackManager keeps `inheritableHandlers ⊆ handlers`
(both `addHandler` and `setHandlers` maintain the invariant), so
checking `handlers` alone suffices. But an externally-constructed
manager subclass could in theory leak the Sentry handler onto
`inheritableHandlers` without mirroring it. Checking both lists
costs nothing and pre-empts the duplicate-span class of bug Sentry's
own reviewer flagged.
@mdnanocom
Copy link
Copy Markdown
Author

Thanks for the review — wanted to share what I found before pushing the fix.

Tracing through @langchain/core@1.1.39's CallbackManager:

  • addHandler(handler, true) pushes to both handlers and inheritableHandlers.
  • setHandlers(handlers, inherit=true) (used by getChild) clears both arrays, then calls addHandler(_, true) for each — so it also keeps both in sync.

That means in normal operation inheritableHandlers ⊆ handlers, and copied.handlers.includes(sentryHandler) already catches the nested-call case the bot describes. I don't think I can reproduce the duplicate-registration in practice with the stock CallbackManager.

That said, the suggested fix costs nothing and pre-empts the failure mode for any externally-constructed subclass that bypasses addHandler. Pushed it as commit 4c74bfa with a regression test (does not double-register when the handler lives only on inheritableHandlers) — and an explanatory comment on the dedupe so the reasoning doesn't decay.

Copy link
Copy Markdown

@cursor cursor Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit b33352b. Configure here.

Comment thread packages/node/src/integrations/tracing/langchain/instrumentation.ts Outdated
@mdnanocom
Copy link
Copy Markdown
Author

Done in 7d61cf5b (pushed). Lifted the fixed implementation to @sentry/core as mergeSentryCallback and routed both @sentry/instrumentation-langchain and @sentry/instrumentation-langgraph through it. The langgraph integration silently picks up the streaming fix as a bonus — the existing helper had the same three bugs (mutates caller manager, no inherit=true, dedupes only against handlers).

Net: 5 files, +163 / -215. Coverage is now in a single mergeSentryCallback suite (14 cases). 13/13 node tracing files (270 tests) and 83/83 core utils files (1439 tests) pass.

@mdnanocom mdnanocom force-pushed the fix/langchain-callbackmanager-preservation branch 2 times, most recently from b2b6b6c to 0c2c666 Compare May 13, 2026 08:24
…ryCallback

The langchain instrumentation's `augmentCallbackHandlers` and the
langgraph instrumentation's `mergeSentryCallback` solved the same
problem two different ways. The langgraph helper had three latent
bugs that this PR's fix already covered for langchain:

  - mutated the caller's CallbackManager (handlers accumulate across runs)
  - called `addHandler(handler)` without `inherit=true`, so the handler
    never propagated to child managers created by `getChild`
  - deduped only against `manager.handlers`, not `inheritableHandlers`

Lift the fixed implementation up to `@sentry/core` so both
instrumentations share it. The langgraph integration silently picks
up the streaming fix as a bonus. Drops the duplicate `augmentCallback`
helper and its test; behavior is covered by the expanded
`mergeSentryCallback` suite (14 cases).
@mdnanocom mdnanocom force-pushed the fix/langchain-callbackmanager-preservation branch from 0c2c666 to 9e2939d Compare May 13, 2026 08:29
Comment thread CHANGELOG.md Outdated
return false;
}
const candidate = value as { addHandler?: unknown; copy?: unknown };
return typeof candidate.addHandler === 'function' && typeof candidate.copy === 'function';
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

m: Is it 100% guaranteed that this works? addHandler and copy seem to be a little vague to check against these. Since CallbackManager is a class, maybe there is the possibility to check against its name?

We do something similar in Cloudflare:

export function isCloudflareClass(value: unknown, className: CloudflareClassName): boolean {
if (!value || typeof value !== 'function') {
return false;
}
if (value.name === className) {
return false;
}
let proto: object | null = value.prototype;
while (proto) {
const ctor = (proto as { constructor?: { name?: string } }).constructor;
const constructorName = ctor?.name;
if (constructorName === className) {
return true;
}
proto = Object.getPrototypeOf(proto);
}
return false;
}

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done, adopted the Cloudflare pattern: walk the prototype chain looking for constructor.name === 'CallbackManager', then confirm the shape.

/**
* Augments a callback handler list with Sentry's handler if not already present
*/
function augmentCallbackHandlers(handlers: unknown, sentryHandler: unknown): unknown {
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

l: I wonder if this was missed to be removed when mergeSentryCallback got added. @andreiborza any insights on this?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not dead — they cover different entry points:

wrapRunnableMethod patches chat-model prototypes (ChatOpenAI, ChatAnthropic, …) so model.invoke/stream/batch calls merge the Sentry handler. This is the bare-LangChain path (no graph involved).
The mergeSentryCallback call site in tracing/langgraph/index.ts patches StateGraph.compile() and merges at graph-entry — also tagging metadata with sentry_langgraph.
Inside a graph the two overlap (chat-model invokes also flow through wrapRunnableMethod), but for users on LangChain-only there's no graph patch to fall back on. Both call sites now share the same helper, which is what this PR consolidates.

mdnanocom added a commit to mdnanocom/sentry-javascript that referenced this pull request May 13, 2026
Address review feedback on PR getsentry#20849:

- `isCallbackManager`: in addition to duck-typing `addHandler`/`copy`,
  walk the prototype chain and require `constructor.name === 'CallbackManager'`
  (mirroring `packages/cloudflare/src/utils/isCloudflareClass.ts`). Filters out
  unrelated objects that coincidentally expose the same shape; the prototype
  walk keeps subclasses working.
- Drop the changelog entry — release process generates the changelog manually.

Two new test cases lock the behavior in: rejects lookalike duck-typed
objects, and recognizes subclasses via the prototype walk (16/16).
Address review feedback on PR getsentry#20849:

- `isCallbackManager`: in addition to duck-typing `addHandler`/`copy`,
  walk the prototype chain and require `constructor.name === 'CallbackManager'`
  (mirroring `packages/cloudflare/src/utils/isCloudflareClass.ts`). Filters out
  unrelated objects that coincidentally expose the same shape; the prototype
  walk keeps subclasses working.
- Drop the changelog entry — release process generates the changelog manually.

Two new test cases lock the behavior in: rejects lookalike duck-typed
objects, and recognizes subclasses via the prototype walk (16/16).
@mdnanocom mdnanocom force-pushed the fix/langchain-callbackmanager-preservation branch from 9f0dbc5 to 7dcdd93 Compare May 13, 2026 12:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants