From 5eec2f8be92c75aac94aeeafd0c2806ea55a7f49 Mon Sep 17 00:00:00 2001 From: Jonathan Jackson Date: Fri, 22 May 2026 02:31:58 -0600 Subject: [PATCH] feat(idea-to-pdd): retire idea.md / --idea operator-seed pathway The 2026-05-05 refactor reduced idea.md to an optional --idea FILE|- seed alongside the inputs/ evidence pack. The dual-path persisted but was rarely used in practice and added cognitive load: the eval rubric carried a clean-source dual-branch keyed off "idea.md contains reviewer comments," the orchestrator had a special --idea-was-passed scripted- seed flow at run start, idea-to-pdd scanned the idea.md body for Drive URLs in its permission pre-flight, and the artifact manifest carried two idea.md entries with corresponding test-side carve-outs. Operators now put any free-text seed directly into inputs/ as a regular source file. One canonical input pathway, one synthesis surface. Co-Authored-By: Claude Opus 4.7 (1M context) --- CHANGELOG.md | 25 ++++++++++++++ agents/ace-orchestrator.md | 34 +++++-------------- agents/idea-to-design.md | 13 +++---- agents/orchestrator-reference.md | 1 - commands/run.md | 34 +++++-------------- commands/step.md | 8 ++--- lib/artifact-manifest.ts | 16 --------- mcp/google-drive-server.ts | 4 +-- package-lock.json | 4 +-- skills/idea-to-pdd-eval/SKILL.md | 34 ++++++++----------- skills/idea-to-pdd/SKILL.md | 27 +++++---------- .../fixtures/CRISPR-Test-001/1-design/idea.md | 5 --- .../fixtures/CRISPR-Test-002/1-design/idea.md | 5 --- test/lib/artifact-manifest-lint.test.ts | 8 ++--- 14 files changed, 81 insertions(+), 137 deletions(-) delete mode 100644 test/fixtures/CRISPR-Test-001/1-design/idea.md delete mode 100644 test/fixtures/CRISPR-Test-002/1-design/idea.md diff --git a/CHANGELOG.md b/CHANGELOG.md index bcd5d1c3..90558a55 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,31 @@ All notable changes to the ACE plugin will be documented in this file. The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and the plugin follows [semantic versioning](https://semver.org/spec/v2.0.0.html). +## 0.13.329 — 2026-05-22 + +**Retire `idea.md` / `--idea` operator-seed pathway.** + +The 2026-05-05 refactor reduced `idea.md` to an optional `--idea FILE|-` seed alongside the `inputs/` evidence pack. The dual-path persisted but was rarely used in practice and added cognitive load: the eval rubric carried a clean-source dual-branch keyed off "idea.md contains reviewer comments," the orchestrator had a special `--idea`-was-passed scripted-seed flow at run start, `idea-to-pdd` scanned the `idea.md` body for Drive URLs in its permission pre-flight, and the artifact manifest carried two `idea.md` entries (run-root + legacy `1-design/idea.md` back-compat) with their corresponding RUN_LEVEL_EXEMPT carve-outs. + +Operators now put any free-text seed directly into `inputs/` as a regular source file alongside everything else. One canonical input pathway, one synthesis surface. + +Removed: +- `--idea FILE|-` argument from `/ace:run` +- The scripted-seed branch in `agents/ace-orchestrator.md § Starting a New Opportunity` +- The "If `--idea FILE|-` was passed" sub-step at run-folder bootstrap +- The `idea.md=` field from the orchestrator's start-of-run log line +- The optional `idea.md` table row in `skills/idea-to-pdd/SKILL.md § Inputs` +- The `idea.md`-URL extraction in `idea-to-pdd`'s pre-flight permission scan +- The "or no idea.md" arm of the missing-source-material error +- Both `idea.md` and `1-design/idea.md` (legacy back-compat) entries from `lib/artifact-manifest.ts` +- The `runs//idea.md` row in `agents/orchestrator-reference.md § Per-run` +- The two legacy `1-design/idea.md` fixture files under `test/fixtures/CRISPR-Test-00{1,2}/` + +Preserved: +- The `clean_source` detection mechanism in `idea-to-pdd-eval` (still useful — keys off reviewer-comment presence across the source pack, not specifically `idea.md`). Dual-branch dimension scoring unchanged; language updated. +- Historical changelog entries in `skills/idea-to-pdd/SKILL.md` and `skills/idea-to-pdd-eval/SKILL.md` (audit trail of evolution from single-`idea.md` → multi-doc evidence pack → no-`idea.md`). +- `scripts/migrate-drive-layout.ts` `idea.md → 1-design/idea.md` mapping (back-compat tool for migrating legacy pre-2026-05-05 runs). + ## 0.13.299 — 2026-05-21 **`/ace:doctor` catches Nova MCP stuck-in-needs-auth-cache state.** diff --git a/agents/ace-orchestrator.md b/agents/ace-orchestrator.md index 53da1544..2f7d8deb 100644 --- a/agents/ace-orchestrator.md +++ b/agents/ace-orchestrator.md @@ -266,7 +266,6 @@ message.** Issue together: - `drive_create_folder` for `/runs//` - `drive_create_file` for `runs//run_state.yaml` (initial — phases all pending) -- `drive_create_file` for `runs//idea.md` (only if `--idea FILE|-` was passed) - `drive_create_file` for `runs//inputs-manifest.yaml` (frozen file_id list from Step 3) **Step 6 — Dispatch Phase 1.** Single `Agent(idea-to-design)` call with @@ -544,7 +543,6 @@ ACE/ (= ACE_DRIVE_ROOT_FOLDER_ID) │ │ └── / (e.g. "20260502-1830") │ │ ├── run_state.yaml │ │ ├── inputs-manifest.yaml (frozen file_id list captured at run start) -│ │ ├── idea.md (optional — only present when --idea FILE|- was passed) │ │ └── 1-design/ │ │ ├── idea-to-pdd.md (the formal PDD — Phase 1 output) │ │ └── ... (other Phase 1 outputs) @@ -594,14 +592,7 @@ in `inputs/` (the manifest), not to pick one canonical PDD file. (next to `opp.yaml`) without knowing about the `inputs/` requirement. See jjackson/ace#299. - **(b) `--idea FILE|-` was passed**: scripted-seed flow. If `` - was also provided, use it; otherwise auto-generate a fresh slug - `smoke-` (today's behavior). Write the idea body - directly into `runs//idea.md` at step 5 — this path - bypasses `inputs/` entirely (scripted runs are non-interactive by - design). No `inputs/pdd.md` write. - - **(c) Zero-arg discovery** (default when neither (a) nor (b)): + **(b) Zero-arg discovery** (default when (a) does not apply): 1. Read `ACE_DRIVE_ROOT_FOLDER_ID`. If unset/empty, error: `ACE_DRIVE_ROOT_FOLDER_ID is not set in your .env (expected at @@ -652,7 +643,7 @@ in `inputs/` (the manifest), not to pick one canonical PDD file. folder ID; this is the **run folder ID** that gets passed to every downstream skill in place of the previous "opp folder ID". -5. **Capture the inputs manifest and (optionally) seed `idea.md`.** +5. **Capture the inputs manifest.** The PDD is the formal output of Phase 1, not an input. The orchestrator's job here is to record what was in `inputs/` at @@ -707,21 +698,18 @@ in `inputs/` (the manifest), not to pick one canonical PDD file. they run (see § Per-Phase Folder Lifecycle). The orchestrator does NOT pre-create `1-design/` here. - **If `--idea FILE|-` was passed**, the command has loaded the body. - Write it verbatim to `runs//idea.md` via `drive_create_file` - — this is the operator's free-text seed and stands alongside the - manifest as supplementary intent. `idea-to-pdd` reads both. - - **Otherwise**, do NOT seed an `idea.md`. The manifest alone is - sufficient — `idea-to-pdd` reads each file in the manifest as the - evidence pack and synthesizes the PDD from there. + The manifest is the sole seed for Phase 1 — `idea-to-pdd` reads + each file in the manifest as the evidence pack and synthesizes + the PDD from there. The previous single-file `pdd.md` discovery (`pdd.md` exact, `*pdd*` glob, lone-doc fallback, multi-doc error) is removed entirely. There is no longer a copy of any input file into the run folder — `inputs/` is the canonical read-only seed pack and `idea-to-pdd` reads its files directly via the manifest's - `file_id`s. + `file_id`s. (The pre-2026-05-22 `--idea FILE|-` operator-seed + flag was also retired — operators put any free-text seed directly + into `inputs/` as a regular source file.) 6. **Initialize `run_state.yaml`** at `/runs//run_state.yaml` with: - `mode`, `created` (ISO timestamp), all steps as `pending` @@ -781,7 +769,6 @@ in `inputs/` (the manifest), not to pick one canonical PDD file. inputs_folder=/inputs (read-only, files in manifest) run_folder=/runs/ manifest=/runs//inputs-manifest.yaml - idea.md= # present only when --idea FILE|- was passed ``` 9. **Begin Phase 1.** @@ -809,9 +796,6 @@ picker. > already exists), and re-run `/ace:run `. Top-level drops > directly under `ACE//` are also fine — the orchestrator > auto-migrates them into `inputs/` on next run. -> -> Or pass `--idea FILE|-` to seed a free-text idea directly without -> using `inputs/`. The legacy `PDD/` flat folder is kept readable by ace-web for back-compat viewing of legacy opps, but is no longer consulted for new runs. @@ -824,7 +808,7 @@ When invoked with an opportunity, execute these phases in order: **Dispatch:** `Agent(idea-to-design)`. -**Inputs (inline at handoff):** the inputs manifest, any `idea.md`, `run_state.yaml`. See § Pre-flight & per-phase conventions → "Pass artifacts inline at phase handoff" for the prompt template. +**Inputs (inline at handoff):** the inputs manifest, `run_state.yaml`. See § Pre-flight & per-phase conventions → "Pass artifacts inline at phase handoff" for the prompt template. **Atoms / skills used (orchestrator-visible only):** `Agent(idea-to-design)`. diff --git a/agents/idea-to-design.md b/agents/idea-to-design.md index f99bacd0..8a3ef85f 100644 --- a/agents/idea-to-design.md +++ b/agents/idea-to-design.md @@ -37,9 +37,9 @@ this subagent's steps have these read-redundancy rules: the PDD MAY be rewritten by Step 1.4's QA retry loop — if QA dispatched the producer with an `auto_fix_hint`, re-read the PDD after that loop terminates. -- **Batch the Step 1 input reads.** `inputs-manifest.yaml`, each - manifest entry, and optional `idea.md` are independent — issue them - as ONE parallel `drive_read_file` block, not sequentially. +- **Batch the Step 1 input reads.** `inputs-manifest.yaml` and each + manifest entry are independent — issue them as ONE parallel + `drive_read_file` block, not sequentially. - **Skill-level reads are governed by each `SKILL.md`.** This subagent controls only the reads it issues directly between steps; reads inside the producer/QA/eval skills are out of scope here. @@ -52,8 +52,6 @@ Invoke the `idea-to-pdd` skill. - `ACE//runs//inputs-manifest.yaml` (frozen pointer-set captured by the orchestrator from `/inputs/`) - Each file referenced in the manifest (read inline) - - Optional: `ACE//runs//idea.md` if `--idea FILE|-` - was passed (operator free-text seed; stands alongside the manifest) - Output: `ACE//runs//1-design/idea-to-pdd.md` (the formal PDD — Phase 1's primary artifact) - **Gate (review mode):** Present the PDD for approval before continuing @@ -72,9 +70,8 @@ Invoke the `idea-to-pdd-qa` skill — runs 6 static structural checks against th ### Step 1.5: Idea-to-PDD eval (independent quality re-grade) Unless `--no-evals` was passed AND QA verdict is `pass`, invoke the `idea-to-pdd-eval` skill. - Inputs: the same source material `idea-to-pdd` consumed - (`inputs-manifest.yaml` + each manifest entry, plus run-root - `idea.md` if present) + the produced PDD at - `ACE//runs//1-design/idea-to-pdd.md` + (`inputs-manifest.yaml` + each manifest entry) + the produced PDD + at `ACE//runs//1-design/idea-to-pdd.md` **(all in subagent context from Step 1 / Step 1.4 — do NOT re-read)** - Output: `ACE//runs//1-design/idea-to-pdd-eval_verdict.yaml` (machine-readable verdict in the shared shape — see `skills/README.md § QA vs Eval`) diff --git a/agents/orchestrator-reference.md b/agents/orchestrator-reference.md index 1e901298..0b2eab97 100644 --- a/agents/orchestrator-reference.md +++ b/agents/orchestrator-reference.md @@ -374,7 +374,6 @@ forking:** | `runs//run_state.yaml` | Lifecycle state — phase/step pointer, mode, `last_actor`, timestamps. New file at each new run-id. | | `runs//README.md` | Per-run index regenerated after each phase via `generateRunReadme(...)`. | | `runs//inputs-manifest.yaml` | Frozen pointer-set captured at run start (`inputs/` file_ids). Snapshots that run's view of the source pack. | -| `runs//idea.md` | Optional `--idea` seed, only when `/ace:run --idea FILE\|-` was used. | | `runs//-/.md` | Producer artifacts (PDDs, app summaries, training docs, screenshots, etc.). | | `runs//-/_verdict[-].yaml` | Producer self-evaluation (when the producing skill self-evaluates). | | `runs//-/-eval_verdict[-].yaml` | Eval-side judgment from the matching `*-eval` skill. | diff --git a/commands/run.md b/commands/run.md index f7949f20..ede8f080 100644 --- a/commands/run.md +++ b/commands/run.md @@ -1,6 +1,6 @@ --- description: Run the full CRISPR-Connect lifecycle for an opportunity -argument-hint: [[/]] [--mode default|review|auto] [--idea FILE|-] [--ace-web-url URL] [--dry-run] [--sandbox] [--no-evals] +argument-hint: [[/]] [--mode default|review|auto] [--ace-web-url URL] [--dry-run] [--sandbox] [--no-evals] allowed-tools: [Read, Write, Edit, Bash, Glob, Grep, Agent, AskUserQuestion] --- @@ -34,12 +34,6 @@ Run the full CRISPR-Connect lifecycle for a Connect opportunity. approval. Use for high-touch operations or training. - `auto` — never pause for any gate. For unattended batch runs (e.g. eval calibration). `[BLOCKER]` concerns still escalate. -- `--idea FILE|-` — pre-seed a free-text `idea.md` from a file path, or `-` - for stdin. Operator-supplied seed; stands alongside the inputs/ - evidence-pack manifest as supplementary intent. Content is uploaded - verbatim to `ACE//runs//idea.md` via `drive_create_file`. - Most runs do not need this flag — the inputs/ evidence pack alone is - sufficient seed material for `idea-to-pdd`. - `--ace-web-url URL` — after the orchestrator completes, invoke the `upload-transcript` skill to POST the run's stream-json transcript to `/api/ingest/upload`. **Smart default:** if this flag is omitted @@ -89,21 +83,18 @@ seed material for the PDD; there is no required filename. Resolution: -1. If `--idea FILE|-` was passed, scripted-seed flow: write the idea - body to `runs//idea.md` directly (operator free-text seed). - Manifest capture still runs from `inputs/` if it exists. -2. Else read `ACE_DRIVE_ROOT_FOLDER_ID`. Stop with an actionable error +1. Read `ACE_DRIVE_ROOT_FOLDER_ID`. Stop with an actionable error if unset. -3. List `ACE/`. Find subfolders containing an `inputs/` subfolder. -4. Pick the candidate with the newest `inputs/` mtime; folder name = ``. -5. If no candidate exists, stop with the new-layout setup message. -6. Generate `runId` = `YYYYMMDD-HHMM` (collision-suffixed). -7. `mkdir /runs//`; capture +2. List `ACE/`. Find subfolders containing an `inputs/` subfolder. +3. Pick the candidate with the newest `inputs/` mtime; folder name = ``. +4. If no candidate exists, stop with the new-layout setup message. +5. Generate `runId` = `YYYYMMDD-HHMM` (collision-suffixed). +6. `mkdir /runs//`; capture `runs//inputs-manifest.yaml` (frozen pointer-set of every direct child file under `inputs/`). No input file is copied — the PDD is synthesized at Phase 1 from the manifest. -8. Init `run_state.yaml`; update `opp.yaml.last_run_id`. -9. Begin Phase 1. +7. Init `run_state.yaml`; update `opp.yaml.last_run_id`. +8. Begin Phase 1. See `agents/ace-orchestrator.md` for full detail. @@ -124,12 +115,6 @@ See `agents/ace-orchestrator.md` for full detail. tell the operator "defaulting --ace-web-url to labs". - Otherwise, leave unset (skip the post-run upload hook). -1b. If `--idea` was provided, read its body: - - If the value is `-`, read stdin until EOF. - - Otherwise treat the value as a file path; read its bytes as UTF-8. - Pass the body through to the orchestrator alongside the slug so the - "Starting a New Opportunity" flow can skip its PDD-picker. - 2. **Execute the orchestration procedure inline at top-level.** Read `agents/ace-orchestrator.md` and follow it as a procedure document from this (top-level) Claude Code session. Do **not** dispatch @@ -143,7 +128,6 @@ See `agents/ace-orchestrator.md` for full detail. Inputs to thread through: - Slug - Execution mode - - Idea body (if `--idea` was provided) - Dry-run flag (if set) - Sandbox flag (if set) - Any existing state from GDrive (if resuming) diff --git a/commands/step.md b/commands/step.md index 2461ee82..eb017312 100644 --- a/commands/step.md +++ b/commands/step.md @@ -92,8 +92,8 @@ Implementation steps the agent (or a thin Bash wrapper) must perform: `inputs/`, `eval-calibration/known-issues.md`) resolve to `ACE//`. Run-scoped entries (anything under `1-design/`, `2-scenarios/`, … `8-execution/`, plus - `run_state.yaml`, `inputs-manifest.yaml`, `idea.md`, - `decisions.yaml`) resolve to `ACE//runs//`. + `run_state.yaml`, `inputs-manifest.yaml`, `decisions.yaml`) + resolve to `ACE//runs//`. - Use `drive_list_folder` (or the live Drive listing) to check that the resolved path exists. 3. If any required artifact is missing, print an error of the form: @@ -137,8 +137,8 @@ skills in dependency order) still enforces the contract. ```text /ace:step idea-to-pdd my-opp → Bare opp. Resolves runId from opp.yaml.last_run_id. - → Reads ACE/my-opp/runs//inputs/idea.md (required, - producedBy: external). OK → invoke skill. + → Required: inputs-manifest.yaml at the run root (produced by + ace-orchestrator at run start). OK → invoke skill. /ace:step app-screenshot-capture malaria-itn-app/20260517-1829 → Pinned run-id. Targets ACE/malaria-itn-app/runs/20260517-1829/ diff --git a/lib/artifact-manifest.ts b/lib/artifact-manifest.ts index c32dbd66..c363f06f 100644 --- a/lib/artifact-manifest.ts +++ b/lib/artifact-manifest.ts @@ -144,22 +144,6 @@ export const ARTIFACT_MANIFEST: readonly ArtifactEntry[] = [ required: false, description: 'Frozen pointer-set captured at run start: every direct child file under inputs/ as {file_id, name, mime_type}. idea-to-pdd reads each entry to synthesize the PDD. Lives at the run-folder root alongside run_state.yaml — both are run-level metadata, scoped beyond any single phase. Pointing at file_ids (not paths) means a human re-arranging inputs/ mid-run does not shift the source pack out from under Phase 1. NOT YET required: existing fixtures predate the 2026-05-05 evidence-pack refactor; flip to required=true once the next round of fixture updates lands.', }, - { - path: 'idea.md', - producedBy: 'external', - consumedBy: ['idea-to-pdd'], - phase: 'design', - required: false, - description: 'Optional operator-supplied free-text seed at the run-folder root. Only present when /ace:run was invoked with --idea FILE|-. Read by idea-to-pdd alongside the inputs-manifest. Most runs do not have this file — the inputs/ evidence pack is sufficient.', - }, - { - path: '1-design/idea.md', - producedBy: 'external', - consumedBy: ['idea-to-pdd'], - phase: 'design', - required: false, - description: 'Legacy path — pre-2026-05-05 the orchestrator copied inputs/pdd.md into 1-design/idea.md as the seed for idea-to-pdd. New runs do not write this file (the manifest at 1-design/inputs-manifest.yaml replaces it; --idea seeds the run-root idea.md). Kept in the manifest so older fixtures and resumed legacy runs validate cleanly.', - }, { path: '1-design/idea-to-pdd.md', producedBy: 'idea-to-pdd', diff --git a/mcp/google-drive-server.ts b/mcp/google-drive-server.ts index 08fa5283..9cf2463b 100644 --- a/mcp/google-drive-server.ts +++ b/mcp/google-drive-server.ts @@ -143,8 +143,8 @@ function error(msg: string) { * * Inflight-dedupe + 30s TTL cache (added 2026-05-10, perf lens). When N * parallel writes target the same parent — common for run-folder bootstrap - * (run_state.yaml + idea.md + inputs-manifest.yaml + gate-brief.md often go - * into the same `runs//` parent in one batched message) — the first + * (run_state.yaml + inputs-manifest.yaml often go into the same + * `runs//` parent in one batched message) — the first * caller's probe populates the inflight slot; concurrent callers await the * same promise instead of firing N redundant `files.get` round-trips. After * the probe resolves, an `ok: true` result is cached for 30s so a follow-up diff --git a/package-lock.json b/package-lock.json index 40e45491..68dfdf5a 100644 --- a/package-lock.json +++ b/package-lock.json @@ -1,12 +1,12 @@ { "name": "ace", - "version": "0.13.291", + "version": "0.13.326", "lockfileVersion": 3, "requires": true, "packages": { "": { "name": "ace", - "version": "0.13.291", + "version": "0.13.326", "license": "ISC", "dependencies": { "@anthropic-ai/sdk": "^0.94.0", diff --git a/skills/idea-to-pdd-eval/SKILL.md b/skills/idea-to-pdd-eval/SKILL.md index 2b09b6df..823a626f 100644 --- a/skills/idea-to-pdd-eval/SKILL.md +++ b/skills/idea-to-pdd-eval/SKILL.md @@ -13,8 +13,8 @@ ships a stress-test grade in the PDD itself. That self-eval has two known weaknesses: (a) the same model that wrote the PDD also grades it (over-generosity bias), and (b) the PDD checks itself against the rubric but not against the source idea — so a PDD that addressed the -intervention but missed reviewer comments from idea.md can still -self-grade 5/5 if the rubric doesn't ask "did you address every +intervention but missed reviewer comments from the source pack can +still self-grade 5/5 if the rubric doesn't ask "did you address every reviewer concern?" This skill is the independent grader. It re-runs the stress test from @@ -31,7 +31,6 @@ calibration methodology. | Source | Artifact | Used for | |---|---|---| | Phase 1 source | `inputs-manifest.yaml` + each `file_id` in it | source idea pack (the full pack is what the PDD is graded against) | -| Phase 1 source (optional) | `runs//idea.md` | operator free-text seed if present | | Phase 1 producer | `1-design/idea-to-pdd.md` | the PDD under judgment | | Phase 1 producer (optional) | `1-design/idea-to-pdd_gate-brief.md` | gate brief if present | @@ -61,33 +60,27 @@ filename rule. - `ACE//runs//inputs-manifest.yaml`, then each `file_id` it lists (the orchestrator's frozen evidence-pack pointer-set, captured at run start) - - `ACE//runs//idea.md` if present (operator - free-text seed via `--idea FILE|-`; absent on most runs) - PDD (the artifact under judgment): `ACE//runs//1-design/idea-to-pdd.md` - Optionally the gate brief if present: `ACE//runs//1-design/idea-to-pdd_gate-brief.md`. The "source idea" referenced throughout the rest of this skill is - the union of the manifest's contents + `idea.md` (if present), - treated as one synthesized seed. When grading dimensions that - reference "idea.md" below, treat them as referencing the full - source-idea pack — early-2026 versions of this rubric assumed a - single `idea.md`; the multi-doc evidence-pack model arrived in the - 2026-05-05 idea-to-pdd refactor. + the union of the manifest's contents, treated as one synthesized + seed. 2. **Extract the source idea's reviewer-comment list.** Source-idea - bodies (any file in the manifest, plus `idea.md` if present) - generally include footnoted or sectioned reviewer comments - (e.g. "[a] FLW safety risks…", "[b] vendor consent…"). Build a - structured list across all source files. + bodies (any file in the manifest) generally include footnoted or + sectioned reviewer comments (e.g. "[a] FLW safety risks…", + "[b] vendor consent…"). Build a structured list across all source + files. **Clean-source detection (added 0.10.9):** if the entire source pack contains zero reviewer comments — no `[a]/[b]` footnotes, no "Reviewer Comments" / "Comments" / "Feedback" section in any of - the manifest entries or `idea.md` — set `clean_source = true` and - skip step 3. The reviewer-comment-fidelity dimension will switch - to the deferred-decision-discipline branch (see § Dimension - below). Surfaced 0.9.11 cross-opp validation: + the manifest entries — set `clean_source = true` and skip step 3. + The reviewer-comment-fidelity dimension will switch to the + deferred-decision-discipline branch (see § Dimension below). + Surfaced 0.9.11 cross-opp validation: `turmeric-dogfood-20260427`'s source idea was clean PM-authored with no review pass; the rubric's anchors at 9.5 ("all comments addressed") were a poor fit because there were no comments to @@ -105,7 +98,7 @@ filename rule. | Dimension | Weight | Criteria | |---|---|---| - | **Reviewer-comment fidelity** | 10% | **Two branches by source type.** If `clean_source = false` (idea.md contains reviewer comments): every reviewer comment from idea.md must have a concrete disposition in the PDD (addressed via §X / scoped out / out-of-scope-for-this-opp). **Scoring anchors (tightened 0.9.4):** all comments addressed with concrete section citation = **9.5**; addressed plus one comment that's "addressed via § X" where § X is mentioned but light = **9.0**; one comment missing disposition = **7.5**; ≥2 missing = **5.0**; one false-disposition claim ("addressed via § X" but X doesn't exist) = **4.0** (3-point deduction floor); ≥2 false claims = **fail (≤3)**. (QA verifies the table EXISTS with rows when reviewer comments are referenced; this dimension grades whether each disposition is *concrete*, not whether the table is populated.) ── **Clean-source branch (added 0.10.9, when `clean_source = true`):** the dimension grades **deferred-decision discipline** instead. Anchors: every deferred decision is concrete (named question, named owner phase, named resolution mechanism) = **9.5**; section present, decisions concrete but owner phase implicit = **8.5**; section present but decisions vague ("TBD per LLO" with no question) = **7.0**; section absent AND PDD silently spec'd things that should have been deferred to LLO discovery = **5.0**; section claims to defer something that should have been Phase-1-speccable = **4.0**. | + | **Reviewer-comment fidelity** | 10% | **Two branches by source type.** If `clean_source = false` (the source pack contains reviewer comments): every reviewer comment must have a concrete disposition in the PDD (addressed via §X / scoped out / out-of-scope-for-this-opp). **Scoring anchors (tightened 0.9.4):** all comments addressed with concrete section citation = **9.5**; addressed plus one comment that's "addressed via § X" where § X is mentioned but light = **9.0**; one comment missing disposition = **7.5**; ≥2 missing = **5.0**; one false-disposition claim ("addressed via § X" but X doesn't exist) = **4.0** (3-point deduction floor); ≥2 false claims = **fail (≤3)**. (QA verifies the table EXISTS with rows when reviewer comments are referenced; this dimension grades whether each disposition is *concrete*, not whether the table is populated.) ── **Clean-source branch (added 0.10.9, when `clean_source = true`):** the dimension grades **deferred-decision discipline** instead. Anchors: every deferred decision is concrete (named question, named owner phase, named resolution mechanism) = **9.5**; section present, decisions concrete but owner phase implicit = **8.5**; section present but decisions vague ("TBD per LLO" with no question) = **7.0**; section absent AND PDD silently spec'd things that should have been deferred to LLO discovery = **5.0**; section claims to defer something that should have been Phase-1-speccable = **4.0**. | | **Archetype coherence** | 10% | The spec must follow the declared archetype's pattern *in spirit*: `atomic-visit` shouldn't introduce inter-visit stages or multi-visit case lifecycles; `focus-group` shouldn't have a single-vendor-style Deliver form; `multi-stage` should have a Stage Gate section between stages. Pattern violations are 2-point deductions per violation. (QA verifies the archetype is *declared and in the valid enum*; this dimension grades whether the structure *matches* the declared archetype.) | | **Numbers consistent** | 10% | Cross-section numerical agreement (semantic — the *meaning* of the numbers, not just regex same-value-twice). **Severity-tiered deductions:** load-bearing inconsistencies that change downstream behavior (LLO recruiting filter, FLW certification gates, payment thresholds) are **2-point deductions** per occurrence. Doc-level inconsistencies (different number presentations of the same value, ordering differences) are **0.5-point deductions**. Default tier is 1.0 (mid). | | **Feasibility of headline metrics** | 10% | Each Primary success metric must be measurable today, not aspirational. Specifically: the PDD's Layer B verification claims must reference concrete checks (file-format validations, deterministic field rules), not future capabilities (e.g. "AI-assisted photo content check" without naming the model, threshold, or expected pass rate). 1.5-point deduction per metric that depends on unspeccable Layer B. | @@ -210,3 +203,4 @@ See `skills/_eval-template.md § Dry-Run Behavior (stock)`. | 2026-05-08 | **Rubric expansion: 7 → 11 dimensions, viability axis added (40% weight).** Surfaced by canopy's holistic_adversarial probe on turmeric run 20260507-1733: rubric scored 8.65/10 on a PDD an adversarial PM-style read scored 3/10 viability (3-to-1 against on the $10K bet). 5.65-point gap = rubric was grading document quality almost exclusively. Added 4 viability dimensions: `demand_reality` (15%, named downstream consumer with pre-committed action — biggest single gap), `resource_realism` (10%, budget vs labor at recruitment-realistic rates), `mission_alignment` (5%, do Primary metrics measure the goal or a process proxy), `fallback_validates_primary` (5%, is the named fallback a real validation harness or a parallel sampling system). Reweighted: stress_test_agreement 25→10%, reviewer_comment_fidelity 20→10%, structural_completeness 15→10%, archetype_coherence 15→10%, numbers_present 10→5%; numbers_consistent + feasibility_headline_metrics held. Pairs with canopy PR #38 (lens-types/judge.md adds rubric_blind_spot signal that drove this expansion). | ACE team (0.13.81) | | 2026-05-08 | **Rubric cleanup: 11 → 10 dimensions; weight-sum bug fix; viability rebalanced to 50%.** Three fixes in one edit: (1) Removed `stress_test_agreement` (10%) — it was structurally tautological (same model applies same rubric twice; cross-model probe confirmed it doesn't discriminate, scoring 8-10 on every grade with variance from rubric ambiguity not from real artifact differences). (2) Folded `numbers_present` (5%) into `numbers_consistent` (10%) since they cover the same axis and `numbers_present` was already a soft check most PDDs trivially pass. (3) Fixed the 0.13.81 weight-sum bug: weights summed to 0.95 not 1.0. New weights cleanly total 1.00 with viability at 50%: `demand_reality` 15→20%, `resource_realism` 10→15%, `mission_alignment` 5→10%, `fallback_validates_primary` held at 5%, `feasibility_headline_metrics` 5→10%. Verification (independent re-grade on turmeric PDD with the new 11-dim rubric scored 7.55 vs old rubric's 8.65 — confirming the viability axis discriminates). | ACE team (0.13.84) | | 2026-05-08 | **QA/Eval split: removed `structural_completeness` (10%) — now lives in new `idea-to-pdd-qa` skill.** First migration of the QA/Eval split principle (PR #146). Structural completeness was a static check (regex over `## Heading` lines for the 11 required sections); moved to `skills/idea-to-pdd-qa/checks.ts` as `checkAllRequiredSectionsPresent`. The eval rubric is now quality-only: 4 doc/fidelity dimensions (40%) + 5 viability dimensions (60%). Removed weight (10%) was redistributed to viability dimensions: `demand_reality` 20→22%, `resource_realism` 15→17%, `mission_alignment` 10→12%, `fallback_validates_primary` 5→9%. QA gates eval — eval is skipped (`verdict: incomplete`) if QA fails irrecoverably. Updated dimension descriptions to clarify which structural concerns moved to QA (annotated inline). | ACE team (0.13.88) | +| 2026-05-22 | **Retire `idea.md` references.** The optional `idea.md` operator-seed input was removed from `idea-to-pdd`; this rubric loses its dual-input language. `clean_source` detection mechanism is unchanged (still keys off reviewer-comment presence across the source pack); language switched from "idea.md" to "the source pack" throughout. No scoring or weighting change. | ACE team | diff --git a/skills/idea-to-pdd/SKILL.md b/skills/idea-to-pdd/SKILL.md index 2850a087..65b0569d 100644 --- a/skills/idea-to-pdd/SKILL.md +++ b/skills/idea-to-pdd/SKILL.md @@ -16,7 +16,6 @@ Take an initial idea and iterate on it to produce a complete Program Design Doc |---|---|---| | Operator | `ACE//runs//inputs-manifest.yaml` | frozen pointer-set to source material captured at run-start | | Operator | each `file_id` in the manifest | source content (PDFs, docs, sheets, markdown) | -| Operator (optional) | `ACE//runs//idea.md` | free-text seed via `--idea FILE\|-` | ## Products @@ -63,20 +62,13 @@ orchestrator from the per-skill QA + eval verdicts on the fly. --> non-text files** — the human dropped them in for downstream skills (e.g. data spreadsheet templates for FLW reference). - Additionally, if `ACE//runs//idea.md` exists - (operator-supplied free-text seed via `--idea FILE|-`), read it - too and treat its body as the operator's primary intent — it - stands alongside the manifest's evidence pack. + If `inputs-manifest.yaml` is missing, **stop and return an + actionable error**: - If `inputs-manifest.yaml` is missing AND no `idea.md` exists at - the run root, **stop and return an actionable error**: - - "Phase 1 has no source material — `inputs-manifest.yaml` (at the run-folder root) - is missing and no operator-supplied `idea.md` was found at the - run root. The orchestrator should have written the manifest at - run-start. Re-run `/ace:run ` so the manifest is - captured from `ACE//inputs/`. If you intentionally want - a free-text-only seed, pass `--idea FILE|-`." + "Phase 1 has no source material — `inputs-manifest.yaml` (at the + run-folder root) is missing. The orchestrator should have written + the manifest at run-start. Re-run `/ace:run ` so the + manifest is captured from `ACE//inputs/`." Do not invent source material or proceed without source content. @@ -88,10 +80,7 @@ orchestrator from the per-skill QA + eval verdicts on the fly. --> 30-second share-with-the-SA fix. Track every entry from `inputs-manifest.yaml` whose - `drive_read_file` returned a permission error. Additionally, if - `idea.md` is present, scan its body for Drive URLs matching - `https://(docs|drive)\.google\.com/(document|spreadsheets|presentation|file)/d//` - and attempt to read each. + `drive_read_file` returned a permission error. If any read failed with a permission error, **stop and return an actionable error listing every inaccessible doc**: @@ -99,7 +88,6 @@ orchestrator from the per-skill QA + eval verdicts on the fly. --> "The following files are not accessible to the ACE service account: - `` (`` from inputs/) - - `` (`` referenced in idea.md) - … Share each with `ace-service-account@connect-labs.iam.gserviceaccount.com` @@ -465,3 +453,4 @@ When `--dry-run` is active: | 2026-05-15 | **Recharacterize `focus-group` archetype to attestation-form-only.** `## Archetypes § focus-group` updated: training surface is OCS chatbot + handbook gdoc + practice-session audio review (NOT a Learn app); Output Specification is the gdoc structure, not Deliver-app form fields; new "Attestation form fields" question references the canonical default in `pdd-to-deliver-app`. Decisions Log: `facilitator-training-stipend` re-pegged to practice-session-pass; new `gdoc-content-template` row; `submission-window` clarified as attestation-form submission. Prompted by `malaria-itn-fgd/20260514-2007` post-run reframe. See `docs/superpowers/specs/2026-05-15-focus-group-archetype-redefinition.md`. | ACE team | | 2026-05-15 | Pare attestation-form-fields question + Decisions Log to match the 5-field form: consent / date / venue / GPS / photo. Audio is out-of-band; gdoc_link is removed (gdoc is written after submission). Add `gps-verification-radius` and `gdoc-submission-window` decisions; recharacterize `audio-min-duration` and `audio-consent-fallback` as facilitator-protocol concerns (out-of-band, not in the form). | ACE team | | 2026-05-15 | Recharacterize `payment-rate` and `per-session-rate` Decisions Log rows: PDD captures a **range** (not a fixed number), and the actual rate is **negotiated via the solicitation response** where the LLO proposes a number with rationale. The awarded LLO's proposed rate becomes the `connect.deliver_unit` payment_unit amount at Phase 4 setup. Pairs with `solicitation-create/SKILL.md § Process`'s "per-unit payment is negotiated, not declared" design principle. | ACE team | +| 2026-05-22 | **Retire the optional `idea.md` operator-seed input.** The 2026-05-05 refactor reduced `idea.md` to an optional `--idea FILE\|-` seed alongside the `inputs/` evidence pack; the dual-path persisted but was rarely used in practice and added cognitive load (eval rubric branches, manifest-vs-idea precedence, permission-scan URL extraction). Operators now put any free-text seed directly into `inputs/` as a regular source file. Removed: optional table row, idea.md read paragraph, idea.md-URL permission scan, "or no idea.md" branch of the missing-source error. The `--idea` flag and run-root `idea.md` artifact are gone. | ACE team | diff --git a/test/fixtures/CRISPR-Test-001/1-design/idea.md b/test/fixtures/CRISPR-Test-001/1-design/idea.md deleted file mode 100644 index 82ab80c8..00000000 --- a/test/fixtures/CRISPR-Test-001/1-design/idea.md +++ /dev/null @@ -1,5 +0,0 @@ -# Initial Idea — CRISPR-Test-001 - -> Synthetic idea for the atomic-visit fixture. Used as input to `idea-to-pdd` if testing the full lifecycle from step 1. - -Pilot a Community Health Worker (CHW) training-and-deployment program in TestLand's Eastern Province, where rural clinics lack systematic CHW training for maternal and child health (MCH) services. Train CHWs in prenatal assessment, immunization tracking, and danger-sign recognition via a Connect Learn app. Then deploy a Connect Deliver app for FLWs to use during home visits — registering pregnant women and children under 5, conducting monthly prenatal assessments, tracking immunizations, and triggering emergency referrals when danger signs are observed. Goal: standardize CHW protocol and capture outcome data so we can measure whether structured FLW visits improve referral rates and immunization coverage in the pilot districts. diff --git a/test/fixtures/CRISPR-Test-002/1-design/idea.md b/test/fixtures/CRISPR-Test-002/1-design/idea.md deleted file mode 100644 index 740f4a03..00000000 --- a/test/fixtures/CRISPR-Test-002/1-design/idea.md +++ /dev/null @@ -1,5 +0,0 @@ -# Initial Idea — CRISPR-Test-002 - -> Synthetic idea for the focus-group fixture. Used as input to `idea-to-pdd` if testing the full lifecycle from step 1. - -Run focus group discussions with caregivers in TestLand's Eastern Province (Luvale + Kaonde districts) to understand the underlying drivers of childhood under-vaccination. Existing health-records data tells us *that* vaccination is below target but not *why* — we don't know which barriers are dominant (access, cost, opportunity cost, social influence, religious objections, gender dynamics, mistrust) or how they vary across community segments. Recruit 2 segments (women close to a primary health center with under-vaccinated children; men remote from PHCs with mixed vaccination status) and run 3 focus group sessions per segment, facilitated by trained community-engagement staff in Hausa, with audio recording and per-domain summary capture. Synthesize findings across sessions to produce a barriers-and-enablers report. The output should be specific enough to inform a follow-on Stage-2 household intervention (which will be a separate opportunity if findings warrant it). diff --git a/test/lib/artifact-manifest-lint.test.ts b/test/lib/artifact-manifest-lint.test.ts index af7665c2..3997aed8 100644 --- a/test/lib/artifact-manifest-lint.test.ts +++ b/test/lib/artifact-manifest-lint.test.ts @@ -8,9 +8,9 @@ * every segment resolves to a real skill (or agent) directory. * * The opp-level / run-level exempt sets cover the handful of paths that don't - * fit the per-run shape (opp.yaml lives at ACE//, idea.md is the input - * copy, etc.). Update those sets if a new path legitimately needs to skip the - * shape check — and add a comment explaining why. + * fit the per-run shape (opp.yaml lives at ACE//, run_state.yaml lives at + * the run root, etc.). Update those sets if a new path legitimately needs to + * skip the shape check — and add a comment explaining why. */ import { describe, it, expect } from 'vitest'; @@ -34,8 +34,6 @@ const OPP_LEVEL_EXEMPT = new Set([ const RUN_LEVEL_EXEMPT = new Set([ 'run_state.yaml', 'inputs-manifest.yaml', // frozen pointer-set captured at run start (orchestrator-emitted) - 'idea.md', // optional operator free-text seed via --idea FILE|- - '1-design/idea.md', // legacy pre-2026-05-05 input-copy path; kept for back-compat 'decisions.yaml', // per-run structured decisions log (rows accumulate across all phases) 'decisions.gdoc', // prose Google Doc rendering of decisions.yaml (one stable URL per run) ]);