Braintrust Python SDKs

Braintrust is a platform for evaluating and shipping AI products. Learn more at braintrust.dev and in the docs.

This repository contains Braintrust's Python SDKs and integrations, including:

• The main braintrust SDK package in ./py
• Built-in integrations under py/src/braintrust/integrations and related compatibility packages under ./integrations
• Examples, tests, and local development tooling for Python SDK development

Quickstart

Install the main SDK and scorer package:

pip install braintrust autoevals

Create tutorial_eval.py:

from autoevals import LevenshteinScorer
from braintrust import Eval

Eval(
    "Say Hi Bot",
    data=lambda: [
        {"input": "Foo", "expected": "Hi Foo"},
        {"input": "Bar", "expected": "Hello Bar"},
    ],
    task=lambda input: "Hi " + input,
    scores=[LevenshteinScorer],
)

Run it:

BRAINTRUST_API_KEY=<YOUR_API_KEY> braintrust eval tutorial_eval.py

Packages

Package	Purpose	PyPI	Docs
braintrust	Core Python SDK for logging, tracing, evals, CLI workflows, and built-in integrations.		py/README.md

Integrations included in braintrust

Integration	Auto-instrumented	Min version
OpenAI	Yes	openai>=1.71
Anthropic	Yes	anthropic>=0.48.0
LiteLLM	Yes	litellm>=1.74.0
Pydantic AI	Yes	pydantic_ai>=1.10.0
Google GenAI	Yes	latest tested in CI
Google ADK	Yes	google-adk>=1.14.1
LangChain	Yes	langchain-core>=0.3.28
OpenRouter	Yes	openrouter>=0.6.0
Agno	Yes	agno>=2.1.0
AgentScope	Yes	agentscope>=1.0.0
Claude Agent SDK	Yes	claude_agent_sdk>=0.1.10
DSPy	Yes	latest tested in CI
pytest plugin	No	pytest>=8

Documentation

• Python SDK docs: https://www.braintrust.dev/docs/reference/sdks/python
• Release notes: https://www.braintrust.dev/docs/reference/release-notes

License

Apache-2.0