fix(hermes-agent): isolate per-call state via ContextVar to fix concurrent trace corruption

[RichardoMrMu]

Summary

opentelemetry.instrumentation.hermes_agent.helpers.state(instance) currently stores its per-call telemetry state on the agent instance itself (instance._otel_hermes_state). When the same AIAgent instance is shared across concurrent calls — e.g. a TUI with multiple tabs, parallel subagents, or any code driving one AIAgent from multiple coroutines/threads — the concurrent calls share the same dict and overwrite each other's current_step_invocation, pending_step_finish_reason, token counters and last-response metadata.

This causes silent corruption of the produced traces:

• React-step spans are leaked (never closed) when a sibling call overwrites current_step_invocation before the original step is finished.
• Token usage is mixed across concurrent invocations (one invoke_agent may show input_tokens=0 because a sibling cleared the counter).
• last_response_id / last_response_model may be attributed to the wrong invocation.

This is a "half-done" concurrency story — the same package already uses contextvars.ContextVar correctly for _ACTIVE_TOOL_NAMES in wrappers.py, but state() was implemented as a plain instance attribute.

Root cause

# helpers.py (before)
def state(instance):
    current = getattr(instance, "_otel_hermes_state", None)
    if current is None:
        current = {
            "current_step_invocation": None,    # single slot, shared
            "active_llm_depth": 0,
            "input_tokens": 0,
            ...
        }
        setattr(instance, "_otel_hermes_state", current)   # shared
    return current

Because the dict lives on instance, every concurrent call hits the same single-slot fields.

Fix

Move per-call state into a ContextVar. Function signatures of state(instance) and clear_state(instance) are kept intact for backward compatibility (instance is preserved as an unused parameter so no call site needs to change).

# helpers.py (after)
_HERMES_STATE: contextvars.ContextVar["dict[str, Any] | None"] = (
    contextvars.ContextVar("opentelemetry_hermes_state", default=None)
)

def state(instance):
    current = _HERMES_STATE.get()
    if current is None:
        current = {...}
        _HERMES_STATE.set(current)
    return current

def clear_state(instance):
    _HERMES_STATE.set(None)

• asyncio.create_task automatically copies the parent context into each task → sibling coroutines (e.g. two tab_handler tasks) get independent state dicts.
• threading.Thread also gets isolated contexts → sibling threads also get independent state dicts.

Verification

Existing telemetry spec tests (regression-free)

$ pytest instrumentation-loongsuite/loongsuite-instrumentation-hermes-agent/tests/test_telemetry_spec.py
============================== 28 passed in 0.14s ==============================

All 27 pre-existing tests pass + 1 new concurrency regression test.

New regression test

Added test_state_is_isolated_across_concurrent_invocations in test_telemetry_spec.py. It runs two concurrent threads on a shared AIAgent, synchronises them with a Barrier after each thread has opened its first step span (the exact moment the buggy implementation would clobber state), and asserts:

• len(agent_spans) == 2
• len(step_spans) == 4 ← would be 2 before the fix (silent leak)
• len(llm_spans) == 4
• len(tool_spans) == 2
• Each agent span owns exactly its own complete {2 step + 2 llm + 1 tool} sub-tree (no cross-tree contamination)

Compatibility

• state(instance) and clear_state(instance) keep their existing signatures.
• All wrappers (LLMCallWrapper, ToolBatchWrapper, etc.) call these helpers through their public name, so no wrapper code changes are needed.
• The instance._otel_hermes_state attribute is no longer set, but it was never part of the public API and is not used anywhere else.

Follow-up: AgentScope likely has the same issue

opentelemetry.instrumentation.agentscope uses a similar setattr(call_self, "_react_step_state", state) pattern. A separate issue + fix is recommended, with the same ContextVar migration shape. Happy to follow up with another PR.

Files changed

File	Change
instrumentation-loongsuite/loongsuite-instrumentation-hermes-agent/src/opentelemetry/instrumentation/hermes_agent/helpers.py	Add _HERMES_STATE ContextVar; rewrite state() / clear_state() to use it. Function signatures unchanged.
instrumentation-loongsuite/loongsuite-instrumentation-hermes-agent/tests/test_telemetry_spec.py	Add test_state_is_isolated_across_concurrent_invocations regression test.

[CLAassistant]

All committers have signed the CLA.

[sipercai]

@RichardoMrMu Thanks for the contribution! Could you please complete the CLA check before we proceed?Thanks again for contributing!