Visualize LLM outputs against datasets, manually annotate results, and run automated evaluations to algorithmically optimize prompts.
-
Updated
Nov 22, 2025 - TypeScript
#
llm-evals
Here are 43 public repositories matching this topic...
An implementation of the Anthropic's paper and essay on "A statistical approach to model evaluations"
-
Updated
Oct 6, 2025 - Python
Create an evaluation framework for your LLM based app. Incorporate it into your test suite. Lay the monitoring foundation.
workshop llm llms llmops llm-eval llm-test llm-evaluation-framework llm-evaluation-metrics llm-monitoring llm-testing llm-evals
-
Updated
May 6, 2025 - Jupyter Notebook
A framework for evaluating large language models (LLMs) across a variety of tasks.
nlp benchmark ai evaluation-framework ai-evaluation llm llm-evaluation llm-as-a-judge g-eval llm-evals
-
Updated
Mar 18, 2026 - Python
Squeeze your model with pressure prompts to see if its behavior leaks.
reliability evaluation calibration alignment quality-assurance metamorphic-testing ai-safety trustworthiness hallucinations prompt-engineering llm-eval llm-evals
-
Updated
Mar 1, 2026 - Python
Codex-native autoresearch harness with structured worker/judge turns for optimizing anything you can measure.
-
Updated
Mar 21, 2026 - Python
Detecting Relational Boundary Erosion in AI systems. A framework for testing whether models maintain honest, calibrated, and appropriate boundaries.
python yaml calibration alignment metamorphic-testing model-evaluation ai-safety red-teaming prompt-injection hallucination-detection llm-evals evaluation-harness
-
Updated
Feb 22, 2026 - Python
Disposable Daytona sandboxes for LLM evals and isolated command execution
-
Updated
Apr 24, 2026 - TypeScript
Evaluation patterns, release gates, and anti-hallucination techniques for developer-focused AI workflows.
-
Updated
Mar 27, 2026 - Python
In this we evaluate the LLM responses and find accuracy
-
Updated
Jul 8, 2025 - Python
Local-first LLM evaluation runner with baselines, caching, markdown reports, and CI-friendly quality, latency, and cost gates.
-
Updated
Apr 13, 2026 - Python
Evaluation and reliability harness for agentic LLM systems, with task success, latency, cost, retries, fallback routing, and failure taxonomy.
benchmarking intelligence dashboard latency reliability evaluation observability ai-agents failure-analysis cost-tracking llm-evals fallback-routing
-
Updated
Apr 1, 2026 - Python
This project demonstrates a production-grade Evaluation (Evals) Framework used to benchmark multiple Large Language Models (LLMs) against a "Source of Truth" NBA dataset.
-
Updated
Apr 27, 2026 - Python
Synthetic marketplace benchmark harness with deterministic demo and Codex subagent pilot
-
Updated
Apr 25, 2026 - Python
Sovereign Adversarial Simulation & Interdiction Engine for the 0.05V Standard.
-
Updated
Mar 21, 2026 - Python
Agentic research pipeline with local retrieval, structured evaluation, conditional revision, and traceable outputs using Groq.
python cli retrieval evaluation ai-agents groq llm prompt-engineering research-agent agentic-workflow llm-evals
-
Updated
Mar 23, 2026 - TypeScript
Reproducible LLM proof grading benchmark + API for Olympiad-style math.
python benchmarking fastapi math-education llm-evaluation llm-evals proof-grading rubric-aware olympiad-math math-ed
-
Updated
Apr 24, 2026 - Python
Evaluation-first research kit for biology. Structured logs, reproducible specs, and lightweight validation for agentic workflows.
-
Updated
Dec 25, 2025 - Python
Public benchmark and reference bundle for dormant behavior audit
-
Updated
Apr 11, 2026 - Python
Improve this page
Add a description, image, and links to the llm-evals topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the llm-evals topic, visit your repo's landing page and select "manage topics."