Braintrust is a platform for evaluating and shipping AI products. Learn more at braintrust.dev and in the docs.
This repository contains Braintrust's Python SDKs and integrations, including:
- The main
braintrustSDK package in./py - Built-in integrations under
py/src/braintrust/integrationsand related compatibility packages under./integrations - Examples, tests, and local development tooling for Python SDK development
Install the main SDK and scorer package:
pip install braintrust autoevalsCreate tutorial_eval.py:
from autoevals import LevenshteinScorer
from braintrust import Eval
Eval(
"Say Hi Bot",
data=lambda: [
{"input": "Foo", "expected": "Hi Foo"},
{"input": "Bar", "expected": "Hello Bar"},
],
task=lambda input: "Hi " + input,
scores=[LevenshteinScorer],
)Run it:
BRAINTRUST_API_KEY=<YOUR_API_KEY> braintrust eval tutorial_eval.py| Package | Purpose | PyPI | Docs |
|---|---|---|---|
braintrust |
Core Python SDK for logging, tracing, evals, CLI workflows, and built-in integrations. |  |
py/README.md |
| Integration | Auto-instrumented | Min version |
|---|---|---|
| OpenAI | Yes | openai>=1.71 |
| Anthropic | Yes | anthropic>=0.48.0 |
| LiteLLM | Yes | litellm>=1.74.0 |
| Pydantic AI | Yes | pydantic_ai>=1.10.0 |
| Google GenAI | Yes | google-genai>=1.30.0 |
| Google ADK | Yes | google-adk>=1.14.1 |
| LangChain | Yes | langchain-core>=0.3.28 |
| OpenRouter | Yes | openrouter>=0.6.0 |
| Agno | Yes | agno>=2.1.0 |
| AgentScope | Yes | agentscope>=1.0.0 |
| Claude Agent SDK | Yes | claude_agent_sdk>=0.1.10 |
| DSPy | Yes | dspy>=2.6.0 |
| pytest plugin | No | pytest>=8 |
- Python SDK docs: https://www.braintrust.dev/docs/reference/sdks/python
- Release notes: https://www.braintrust.dev/docs/reference/release-notes
Apache-2.0