[Team PaLM] — TRP1 FDE Programme, April 2026

[yosef-zewdu]

Summary

Submission for the Oracle Forge Data Agent on the UC Berkeley DataAgentBench (DAB).

• Pass@1: 0.400 (12 / 30 queries correct)
• Trials: 2 runs across 10 datasets

Architecture

Oracle Forge is a natural-language analytics agent over heterogeneous databases (PostgreSQL, MongoDB, SQLite, DuckDB), built around:

• Three-layer ContextManager — live schema introspection (Layer 1), curated KB docs in kb/domain + kb/evaluation + AGENT.md (Layer 2), and an append-only corrections log with similarity-matched proactive reuse (Layer 3).
• QueryRouter — entity extraction, multi-DB decomposition, dialect templating across 4 engines.
• ExecutionEngine + MCPToolbox — hybrid routing: Google MCP Toolbox (HTTP) for PostgreSQL/SQLite, MongoDB, and a standalone DuckDB MCP service for DuckDB.
• SelfCorrectionLoop — 5 failure categories, LLM-guided repair, known-join fixes, max 3 retries, every outcome logged back into Layer 3.

Per-Dataset Results

Dataset	Score
bookreview	3/3
googlelocal	3/4
yelp	3/7
crmarenapro	1/2
stockindex	1/3
agnews	0/3
deps_dev_v1	0/1
patents	0/1
stockmarket	0/4

[shreyashankar]

Hi @yosef-zewdu — we're missing coverage. The file has 30 entries across 9 of 12 datasets (missing GITHUB_REPOS, PANCANCER_ATLAS, music_brainz_20k) and 28 of 54 queries, with 1–2 runs each. Per the instructions in the README, we need every query across all 12 datasets with at least 5 runs per query. If you didn't attempt some queries, include those entries with "answer": "". Once it's in I'll re-run verification and post the Pass@1 here.

[yosef-zewdu]

@shreyashankar Thanks for the response. We will get back as the instruction suggest.