Skip to content

Add PR #31, #32, #38 to leaderboard #39

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
shreyashankar merged 1 commit into from
Apr 22, 2026
+68 −18
Merged

Add PR #31, #32, #38 to leaderboard #39

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 13 additions & 10 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,16 +21,19 @@ Unlike prior SQL-only or single-database benchmarks, DAB stresses agents under *

## 🏆 Leaderboard

| Rank | Model | Pass@1 | Date |
| ---- | -------------- | ------------------- | ------- |
| 1 | PromptQL (Gemini-3.1-Pro) (5 trials/query) | 0.543 | 2026-03-018 |
| 2 | PromptQL (Claude-Opus-4.6) (5 trials/query) | 0.508 | 2026-03-02 |
| 3 | Claude-Opus-4.6 (5 trials/query) | 0.4376 | 2026-03-18 |
| 4 | Gemini-3-Pro | 0.38 | 2026-03-02 |
| 5 | GPT-5-mini | 0.30 | 2026-03-02 |
| 6 | GPT-5.2 | 0.25 | 2026-03-02 |
| 7 | Kimi-K2 | 0.23 | 2026-03-02 |
| 8 | Gemini-2.5-Flash | 0.09 | 2026-03-02 |
| Rank | Model | Pass@1 | Date | Submission |
| ---- | -------------- | ------------------- | ------- | ---------- |
| 1 | Pi Coding Agent (Claude-Opus-4.6) (5 trials/query) | 0.5603 | 2026-04-21 | [#31](https://github.com/ucbepic/DataAgentBench/pull/31) |
| 2 | PromptQL (Gemini-3.1-Pro) (5 trials/query) | 0.543 | 2026-03-18 | [#24](https://github.com/ucbepic/DataAgentBench/pull/24) |
| 3 | PromptQL (Claude-Opus-4.6) (5 trials/query) | 0.508 | 2026-03-02 | [#23](https://github.com/ucbepic/DataAgentBench/pull/23) |
| 4 | Oracle Forge — Tenacious Intelligence (Claude-Sonnet-4.6) (5–7 trials/query) | 0.4554 | 2026-04-21 | [#32](https://github.com/ucbepic/DataAgentBench/pull/32) |
| 5 | Claude-Opus-4.6 (5 trials/query) | 0.4376 | 2026-03-18 | [#22](https://github.com/ucbepic/DataAgentBench/pull/22) |
| 6 | Gemini-3-Pro | 0.38 | 2026-03-02 | — |
| 7 | GPT-5-mini | 0.30 | 2026-03-02 | — |
| 8 | GPT-5.2 | 0.25 | 2026-03-02 | — |
| 9 | Kimi-K2 | 0.23 | 2026-03-02 | — |
| 10 | Oracle Forge — Team Cohere (Gemini-2.0-Flash) (5 trials/query) | 0.128 | 2026-04-21 | [#38](https://github.com/ucbepic/DataAgentBench/pull/38) |
| 11 | Gemini-2.5-Flash | 0.09 | 2026-03-02 | — |


### How to Submit to the Leaderboard
Expand Down
13 changes: 13 additions & 0 deletions docs/app.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -172,13 +172,26 @@ function renderOverallLeaderboard(rows) {
const trialsTd = createElement("td", "num", String(row.trials));
const passTd = createElement("td", "num", formatPercentFromScore(row.passAt1));
const dateTd = createElement("td", "", row.date);
const prTd = document.createElement("td");
if (row.prUrl) {
const prMatch = row.prUrl.match(/\/pull\/(\d+)/);
const prLabel = prMatch ? `PR #${prMatch[1]}` : "PR";
const prLink = createElement("a", "", prLabel);
prLink.href = row.prUrl;
prLink.target = "_blank";
prLink.rel = "noopener noreferrer";
prTd.appendChild(prLink);
} else {
prTd.textContent = "—";
}

tr.appendChild(rankTd);
tr.appendChild(agentTd);
tr.appendChild(teamTd);
tr.appendChild(trialsTd);
tr.appendChild(passTd);
tr.appendChild(dateTd);
tr.appendChild(prTd);
tbody.appendChild(tr);
});
}
Expand Down
49 changes: 41 additions & 8 deletions docs/data/leaderboards.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
{
"updatedAt": "2026-03-21",
"updatedAt": "2026-04-21",
"sources": [
"README leaderboard table",
"PromptQL stratified table shared by maintainers",
Expand All @@ -8,33 +8,56 @@
"overallLeaderboard": [
{
"rank": 1,
"agent": "Pi Coding Agent + Claude Opus 4.6",
"trials": 5,
"passAt1": 0.5603,
"team": "Pi Coding Agent",
"teamUrl": "https://github.com/mariozechner/pi-coding-agent",
"prUrl": "https://github.com/ucbepic/DataAgentBench/pull/31",
"date": "2026-04-21"
},
{
"rank": 2,
"agent": "PromptQL + Gemini 3.1 Pro",
"trials": 5,
"passAt1": 0.543,
"team": "Hasura PromptQL",
"teamUrl": "https://promptql.io/",
"prUrl": "https://github.com/ucbepic/DataAgentBench/pull/24",
"date": "2026-03-18"
},
{
"rank": 2,
"rank": 3,
"agent": "PromptQL + Claude Opus 4.6",
"trials": 5,
"passAt1": 0.508,
"team": "Hasura PromptQL",
"teamUrl": "https://promptql.io/",
"prUrl": "https://github.com/ucbepic/DataAgentBench/pull/23",
"date": "2026-03-02"
},
{
"rank": 3,
"rank": 4,
"agent": "Oracle Forge (Tenacious Intelligence) + Claude Sonnet 4.6",
"trials": 5,
"passAt1": 0.4554,
"team": "Tenacious Intelligence Corp",
"teamUrl": "https://github.com/ucbepic/DataAgentBench/pull/32",
"prUrl": "https://github.com/ucbepic/DataAgentBench/pull/32",
"date": "2026-04-21"
},
{
"rank": 5,
"agent": "Claude Opus 4.6 ReAct",
"trials": 5,
"passAt1": 0.4376,
"team": "EPIC Data Lab",
"teamUrl": "https://epic.berkeley.edu/",
"prUrl": "https://github.com/ucbepic/DataAgentBench/pull/22",
"date": "2026-03-18"
},
{
"rank": 4,
"rank": 6,
"agent": "Gemini-3-Pro ReAct",
"trials": 50,
"passAt1": 0.38,
Expand All @@ -43,7 +66,7 @@
"date": "2026-03-02"
},
{
"rank": 5,
"rank": 7,
"agent": "GPT-5-mini ReAct",
"trials": 50,
"passAt1": 0.3,
Expand All @@ -52,7 +75,7 @@
"date": "2026-03-02"
},
{
"rank": 6,
"rank": 8,
"agent": "GPT-5.2 ReAct",
"trials": 50,
"passAt1": 0.25,
Expand All @@ -61,7 +84,7 @@
"date": "2026-03-02"
},
{
"rank": 7,
"rank": 9,
"agent": "Kimi-K2 ReAct",
"trials": 50,
"passAt1": 0.23,
Expand All @@ -70,7 +93,17 @@
"date": "2026-03-02"
},
{
"rank": 8,
"rank": 10,
"agent": "Oracle Forge (Team Cohere) + Gemini 2.0 Flash",
"trials": 5,
"passAt1": 0.128,
"team": "Team Cohere",
"teamUrl": "https://github.com/trp1-cohere-team/data-analytics-agent",
"prUrl": "https://github.com/ucbepic/DataAgentBench/pull/38",
"date": "2026-04-21"
},
{
"rank": 11,
"agent": "Gemini-2.5-Flash ReAct",
"trials": 50,
"passAt1": 0.09,
Expand Down
1 change: 1 addition & 0 deletions docs/index.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -128,6 +128,7 @@ <h2>Leaderboard</h2>
<th><em>n</em></th>
<th>Pass@1</th>
<th>Date</th>
<th>Submission</th>
</tr>
</thead>
<tbody></tbody>
Expand Down