Add query_cve and query_usaspending datasets (10 queries each)

[shreyashankar]

Summary

• Two new DataAgentBench datasets, each with 10 queries that exercise ≥2 of DAB's 4 properties (multi-DB, ill-formatted joins, unstructured text transformation, domain knowledge).
• query_cve: NVD CVE registry + CISA KEV catalog + CPE matches + free-text descriptions across 4 DBMSes (SQLite/DuckDB/Postgres/Mongo). Bounded to 2023–2024 NVD window (~71k CVEs, ~585k CPE matches, 1583 KEV entries).
• query_usaspending: federal contract awards + SAM-derived entity registry + agency hierarchy + free-text descriptions across the same 4 DBMSes. ~9.9k FY2024-Q4 contracts sampled by start date (representative spread of award amounts so threshold filters actually filter).

Both datasets follow the existing convention — agent-visible corrupted DBs in query_dataset/, plus db_config.yaml, db_description.txt (clean schema only), db_description_withhint.txt (terse one-line corruption hints in HINTS: style matching query_yelp/query_crmarenapro), and query{1..10}/{query.json, ground_truth.csv, validate.py}. The canonical un-corrupted snapshot and corruption manifest are kept local-only (gitignored), matching what other datasets ship.

Corruption layers

Hash-deterministic transforms across both datasets:

• cross-DB ID format mixing (CVE-id / award_id / UEI)
• vendor/agency surface-form variant clustering
• vendor-alias indirection in CPE
• vulnerable_flag varied truthy/falsy tokens
• version-encoding mix in CPE versions
• English-description dropping for a deterministic subset
• duplicate rows with conflicting metadata
• packed comma-separated product lists in KEV
• referential-integrity gaps from bounded NVD window

Plus per-row LLM narrative corruption with roundtrip-classifier verification:

• query_cve.cvss_metadata.score_text: gpt-4o narrative implying CVSS magnitude band, anchored on per-row vendor/product/vulnerability summary. 100% distinct phrasings across ~225 KEV-CVE rows.
• query_cve.cve_documents.descriptions[].value: gpt-4o severity-as-prose narrative, anchored on the unique CVE description. 100% distinct across ~294 KEV-CVE rows.
• query_usaspending.contract_amounts.amount_text: gpt-4o narrative implying dollar magnitude band, anchored on per-row recipient/agency/NAICS. 100% distinct across ~9.5k rows.

Verifier pattern: each generated narrative is classified back to its canonical band by an independent LLM call before being accepted. Mismatches retry up to 3 times. Final post-audit mismatch rate is <2% on cve, <1% on usaspending.

Sonnet pass@1 (plain mode)

Dataset	pass@1
query_cve	3/10 = 30%
query_usaspending	5/10 = 50%

Both well within DAB's hardness range (Gemini-3-Pro hits 38% on the published 12-dataset benchmark). The diverse, contextually-anchored narratives defeat single-pass `CASE WHEN ... ILIKE` shortcuts on aggregation queries; agents have to do real per-row classification.

Test plan

• `python sdk_runner/seed_dbs.py` loads PG (`cve_kev`, `usaspending_contracts`) and Mongo (`cve_descriptions`, `usaspending_descriptions`) without errors
• `python sdk_runner/sweep.py --only cve --mode plain` and `--only usaspending --mode plain` both complete
• `python sdk_runner/score.py --mode plain` reports the expected per-dataset and per-query pass/fail
• Reviewer manually inspects 2-3 sample queries to confirm `query.json` / `ground_truth.csv` / `validate.py` follow benchmark conventions
• Reviewer confirms the `db_description_withhint.txt` style matches `query_yelp/db_description_withhint.txt` (terse one-liners, no enumerated examples)

🤖 Generated with Claude Code