Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 19 additions & 0 deletions .github/workflows/loongsuite_lint_0.yml
Original file line number Diff line number Diff line change
Expand Up @@ -241,3 +241,22 @@ jobs:
- name: Run tests
run: tox -c tox-loongsuite.ini -e lint-loongsuite-instrumentation-copaw

lint-loongsuite-instrumentation-sweagent:
name: LoongSuite loongsuite-instrumentation-sweagent
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
- name: Checkout repo @ SHA - ${{ github.sha }}
uses: actions/checkout@v4

- name: Set up Python 3.13
uses: actions/setup-python@v5
with:
python-version: "3.13"

- name: Install tox
run: pip install tox-uv

- name: Run tests
run: tox -c tox-loongsuite.ini -e lint-loongsuite-instrumentation-sweagent

114 changes: 114 additions & 0 deletions .github/workflows/loongsuite_test_0.yml
Original file line number Diff line number Diff line change
Expand Up @@ -1609,3 +1609,117 @@ jobs:
- name: Run tests
run: tox -c tox-loongsuite.ini -e py313-test-loongsuite-instrumentation-copaw -- -ra

py311-test-loongsuite-instrumentation-sweagent-oldest_ubuntu-latest:
name: LoongSuite loongsuite-instrumentation-sweagent-oldest 3.11 Ubuntu
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
- name: Checkout repo @ SHA - ${{ github.sha }}
uses: actions/checkout@v4

- name: Set up Python 3.11
uses: actions/setup-python@v5
with:
python-version: "3.11"

- name: Install tox
run: pip install tox-uv

- name: Run tests
run: tox -c tox-loongsuite.ini -e py311-test-loongsuite-instrumentation-sweagent-oldest -- -ra

py311-test-loongsuite-instrumentation-sweagent-latest_ubuntu-latest:
name: LoongSuite loongsuite-instrumentation-sweagent-latest 3.11 Ubuntu
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
- name: Checkout repo @ SHA - ${{ github.sha }}
uses: actions/checkout@v4

- name: Set up Python 3.11
uses: actions/setup-python@v5
with:
python-version: "3.11"

- name: Install tox
run: pip install tox-uv

- name: Run tests
run: tox -c tox-loongsuite.ini -e py311-test-loongsuite-instrumentation-sweagent-latest -- -ra

py312-test-loongsuite-instrumentation-sweagent-oldest_ubuntu-latest:
name: LoongSuite loongsuite-instrumentation-sweagent-oldest 3.12 Ubuntu
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
- name: Checkout repo @ SHA - ${{ github.sha }}
uses: actions/checkout@v4

- name: Set up Python 3.12
uses: actions/setup-python@v5
with:
python-version: "3.12"

- name: Install tox
run: pip install tox-uv

- name: Run tests
run: tox -c tox-loongsuite.ini -e py312-test-loongsuite-instrumentation-sweagent-oldest -- -ra

py312-test-loongsuite-instrumentation-sweagent-latest_ubuntu-latest:
name: LoongSuite loongsuite-instrumentation-sweagent-latest 3.12 Ubuntu
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
- name: Checkout repo @ SHA - ${{ github.sha }}
uses: actions/checkout@v4

- name: Set up Python 3.12
uses: actions/setup-python@v5
with:
python-version: "3.12"

- name: Install tox
run: pip install tox-uv

- name: Run tests
run: tox -c tox-loongsuite.ini -e py312-test-loongsuite-instrumentation-sweagent-latest -- -ra

py313-test-loongsuite-instrumentation-sweagent-oldest_ubuntu-latest:
name: LoongSuite loongsuite-instrumentation-sweagent-oldest 3.13 Ubuntu
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
- name: Checkout repo @ SHA - ${{ github.sha }}
uses: actions/checkout@v4

- name: Set up Python 3.13
uses: actions/setup-python@v5
with:
python-version: "3.13"

- name: Install tox
run: pip install tox-uv

- name: Run tests
run: tox -c tox-loongsuite.ini -e py313-test-loongsuite-instrumentation-sweagent-oldest -- -ra

py313-test-loongsuite-instrumentation-sweagent-latest_ubuntu-latest:
name: LoongSuite loongsuite-instrumentation-sweagent-latest 3.13 Ubuntu
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
- name: Checkout repo @ SHA - ${{ github.sha }}
uses: actions/checkout@v4

- name: Set up Python 3.13
uses: actions/setup-python@v5
with:
python-version: "3.13"

- name: Install tox
run: pip install tox-uv

- name: Run tests
run: tox -c tox-loongsuite.ini -e py313-test-loongsuite-instrumentation-sweagent-latest -- -ra

Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
# Changelog

All notable changes to this project will be documented in this file.

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## Unreleased

### Added

- Initial SWE-agent instrumentation: `SweagentInstrumentor` emits GenAI spans
(entry, `invoke_agent`, react step, `execute_tool` with LLM `tool_calls` when
available) via `ExtendedTelemetryHandler`; includes tests, examples, and
tox/CI wiring.
([#165](https://github.com/alibaba/loongsuite-python-agent/pull/165))
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
LoongSuite instrumentation for SWE-agent
========================================

This package instruments `SWE-agent <https://github.com/SWE-agent/SWE-agent>`_
using ``opentelemetry-util-genai`` ``ExtendedTelemetryHandler`` so traces align
with other LoongSuite GenAI plugins.

Spans
-----

.. list-table::
:header-rows: 1

* - SWE-agent locus
- Span name (typical)
- ``gen_ai.operation.name`` / ``gen_ai.span.kind``

* - ``CombinedRunHooks.on_instance_start`` → ``on_instance_completed``
- ``enter_ai_application_system``
- ``enter`` / ``ENTRY``

* - ``CombinedAgentHook.on_run_start`` → ``on_run_done``
- ``invoke_agent swe-agent`` (``gen_ai.agent.name``); ``input_messages`` / ``output_messages`` match the entry span (same problem-statement user text via thread-local from ``on_instance_start``, same run summary as ``AgentRunResult`` via ``info`` + ``trajectory``); ``gen_ai.conversation.id`` = problem id when set; token usage from ``info.model_stats`` when present
- ``invoke_agent`` / ``AGENT``

* - ``CombinedAgentHook.on_step_start`` → ``on_step_done``
- ``react step``
- ``react`` / ``STEP``

* - ``DefaultAgent.handle_action`` (bash / ``communicate``)
- ``execute_tool <name>`` where ``name`` is the first LLM ``tool_calls[*].function.name`` when ``StepOutput.tool_calls`` is set (function-calling mode); otherwise ``execute_tool sweagent_bash`` (thought/action parsing). Tool arguments recorded as ``function.arguments`` (JSON decoded when possible), else ``step.action``.
- ``execute_tool`` / ``TOOL``

Remote LLM calls (LiteLLM) are **not** duplicated here; enable
``loongsuite-instrumentation-litellm`` (or equivalent) for model spans.

Requirements
------------

- Python **3.11+** (matches upstream SWE-agent).

Installation
------------

From the LoongSuite repo root (after installing ``sweagent`` and ``opentelemetry-util-genai``):

::

pip install -e ./util/opentelemetry-util-genai
pip install -e ./instrumentation-loongsuite/loongsuite-instrumentation-sweagent

Usage
-----

::

from opentelemetry.instrumentation.sweagent import SweagentInstrumentor
from opentelemetry.sdk.trace import TracerProvider

provider = TracerProvider()
# add_span_processor(...) # e.g. OTLP or console

SweagentInstrumentor().instrument(tracer_provider=provider)
# ... run sweagent ...
SweagentInstrumentor().uninstrument()

Entry span input is derived from ``problem_statement.id`` and a truncated
``get_problem_statement()`` body. Output summarizes ``AgentRunResult.info`` and
trajectory length. Tool span arguments/results follow GenAI content-capture
environment variables when experimental semconv is enabled.
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
# Copyright The OpenTelemetry Authors
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
# Copyright The OpenTelemetry Authors
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

"""Minimal example: instrument run hooks and export spans to the console."""

from __future__ import annotations

from unittest.mock import MagicMock

from sweagent.run.hooks.abstract import CombinedRunHooks
from sweagent.types import AgentInfo, AgentRunResult

from opentelemetry.instrumentation.sweagent import SweagentInstrumentor
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import (
BatchSpanProcessor,
ConsoleSpanExporter,
)


def main() -> None:
exporter = ConsoleSpanExporter()
provider = TracerProvider()
provider.add_span_processor(BatchSpanProcessor(exporter))
SweagentInstrumentor().instrument(tracer_provider=provider)

hooks = CombinedRunHooks()
prob = MagicMock()
prob.id = "demo-instance"
prob.get_problem_statement.return_value = "Example task description"
hooks.on_instance_start(
index=0,
env=MagicMock(),
problem_statement=prob,
)
hooks.on_instance_completed(
result=AgentRunResult(
info=AgentInfo(exit_status="done"),
trajectory=[],
)
)
SweagentInstrumentor().uninstrument()


if __name__ == "__main__":
main()
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

[project]
name = "loongsuite-instrumentation-sweagent"
dynamic = ["version"]
description = "LoongSuite instrumentation for SWE-agent (GenAI spans via util-genai)"
readme = "README.rst"
license = "Apache-2.0"
requires-python = ">=3.11"
authors = [
{ name = "LoongSuite Python Agent Authors", email = "qp467389@alibaba-inc.com" },
]
classifiers = [
"Development Status :: 4 - Beta",
"Intended Audience :: Developers",
"License :: OSI Approved :: Apache Software License",
"Programming Language :: Python",
"Programming Language :: Python :: 3",
"Programming Language :: Python :: 3.11",
"Programming Language :: Python :: 3.12",
"Programming Language :: Python :: 3.13",
]
dependencies = [
"opentelemetry-api ~= 1.37",
"opentelemetry-instrumentation >= 0.58b0",
"opentelemetry-semantic-conventions >= 0.58b0",
"opentelemetry-util-genai >= 0.2b0",
"wrapt >= 1.17.3",
]

[project.optional-dependencies]
instruments = [
"sweagent >= 1.1.0",
]

[project.entry-points.opentelemetry_instrumentor]
sweagent = "opentelemetry.instrumentation.sweagent:SweagentInstrumentor"

[project.urls]
Homepage = "https://github.com/alibaba/loongsuite-python-agent"
Repository = "https://github.com/alibaba/loongsuite-python-agent"

[tool.hatch.version]
path = "src/opentelemetry/instrumentation/sweagent/version.py"

[tool.hatch.build.targets.sdist]
include = [
"src",
"tests",
]

[tool.hatch.build.targets.wheel]
packages = ["src/opentelemetry"]
Loading
Loading