Skip to content

Add agentic AI scorers to supported models and frameworks reference#1249

Open
nrichers wants to merge 3 commits intomainfrom
nrichers/sc-15351/add-agentic-ai-documentation-to-
Open

Add agentic AI scorers to supported models and frameworks reference#1249
nrichers wants to merge 3 commits intomainfrom
nrichers/sc-15351/add-agentic-ai-documentation-to-

Conversation

@nrichers
Copy link
Copy Markdown
Collaborator

@nrichers nrichers commented Mar 25, 2026

Pull Request Description

What and why?

Adds documentation for agentic AI evaluation using DeepEval integration to the supported models and frameworks page.

Context: While reviewing a product brief from Marketing, I noticed that we were referring to recently added agentic AI tests which I had not seen before. Had a convo with the codebase to get the details.

Changes:

  • Add "Agentic AI" tab to the model type quick reference with link to detailed evaluation docs
  • Add "Agentic AI evaluation" section with:
    • Available tests: TaskCompletion, PlanQuality, PlanAdherence, ToolCorrectness, ArgumentCorrectness
    • Test requirements table showing model requirements and input columns for each test

sc-15351

How to test

Try the live preview: Supported models and frameworks

Output

Capto_ 2026-03-25_02-29-11_pm2 Capto_ 2026-03-25_02-32-49_pm

What needs special review?

N/A

Dependencies, breaking changes, and deployment notes

None — documentation only.

Release notes

N/A

Checklist

  • What and why
  • How to test
  • Labels applied
  • PR linked to Shortcut
  • Documentation updated

Loading
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

internal Not to be externalized in the release notes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants