diff --git a/.claude-plugin/marketplace.json b/.claude-plugin/marketplace.json index 6ac5a7f..6edb337 100644 --- a/.claude-plugin/marketplace.json +++ b/.claude-plugin/marketplace.json @@ -6,13 +6,13 @@ "url": "https://github.com/jjackson" }, "metadata": { - "version": "0.13.277" + "version": "0.13.278" }, "plugins": [ { "name": "ace", "source": "./", - "version": "0.13.277", + "version": "0.13.278", "description": "AI Connect Engine — orchestrates the CRISPR-Connect lifecycle from idea through app building, Connect setup, LLO management, and closeout" } ] diff --git a/.claude-plugin/plugin.json b/.claude-plugin/plugin.json index d58b3c6..47b1b7a 100644 --- a/.claude-plugin/plugin.json +++ b/.claude-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "ace", - "version": "0.13.277", + "version": "0.13.278", "description": "AI Connect Engine — orchestrates the CRISPR-Connect lifecycle from idea through app building, Connect setup, LLO management, and closeout", "author": { "name": "Jonathan Jackson", diff --git a/VERSION b/VERSION index 35500ec..cbeec60 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -0.13.277 +0.13.278 diff --git a/docs/learnings/2026-05-18-connect-gates-deliver-on-learn-completion.md b/docs/learnings/2026-05-18-connect-gates-deliver-on-learn-completion.md new file mode 100644 index 0000000..fa2b9ce --- /dev/null +++ b/docs/learnings/2026-05-18-connect-gates-deliver-on-learn-completion.md @@ -0,0 +1,141 @@ +# Connect gates the Deliver app behind Learn-assessment completion + +**Date:** 2026-05-18 +**Surfaced by:** malaria-itn-app run 20260517-1829 Phase 6 (app-screenshot-capture) +**Class:** Connect UI invariant + Phase 2/3/6 contract gap + +## The invariant + +Connect's mobile UI **only surfaces the Deliver app after the FLW +completes the Learn-app modules AND passes the final assessment AND +that assessment-pass syncs to Connect.** There is no UI affordance +that lets a user jump from claim-opp → Deliver without walking Learn +to completion first. + +Concretely, post-claim the device lands on the Learn-mode +StandardHomeActivity (atlas § 5). Tapping `Start` launches the Learn +suite root (§ 6). The user must: + +1. Tap each Learn module row → form list → form → fill content forms +2. Tap each module's assessment row → take the quiz → pass with the + required `passing_score` +3. Submit the final assessment, return to the suite root +4. Tap `Sync with Server` (Learn-side home) to push the assessment + pass to Connect +5. Navigate back to Connect's opp list (system back / "GO TO CONNECT + MENU" via the in-app return) +6. Tap `Resume` on the now-In-Progress card → certificate screen + (atlas § 8) +7. Tap `VIEW OPPORTUNITY DETAILS` → Download Delivery gate (§ 9) +8. Tap `DOWNLOAD` → Deliver CCZ installs → Deliver-mode + StandardHomeActivity (§ 10) with the `id/viewJobCard` widget + +The opp-detail screen in Connect makes this explicit: + +> "Once you have completed the learning assessment, you will +> transition to delivery." + +This is intentional product behavior — Connect uses Learn completion +as a quality gate on FLWs before they can submit paid visits. + +## Why this trapped Phase 6 + +Phase 6 (`app-screenshot-capture`) reads smoke recipes from Phase 3's +`app-test-cases.yaml`. The malaria-itn-app run's smoke set was: + +```yaml +smoke_journeys_per_app: + learn: 0 # No Phase 2 journeys target the Learn app + deliver: 1 +``` + +Phase 2 (`pdd-to-app-journeys`) had generated 9 journeys, all +targeting the Deliver app (V1/V2 field-visit flows). Phase 3 wrote +the master yaml with `learn: 0` (a known-incomplete state) and the +single Deliver smoke recipe (J1) used the Learn-side palette chain: + +``` +connect-login → connect-claim-opp → learn-launch → tap "V1 Long Visit" +``` + +`learn-launch.yaml` lands on the Learn suite root (the only thing +`Start` can target post-claim when training is incomplete), and the +"V1 Long Visit" text-match resolved against the Learn modules +("Module 1 - Malaria + ITN Basics", etc.), not the Deliver-side +"V1 — Identification and Consent" form. The recipe failed at the +final tap with `verdict: fail` and `recipe_failure_reason: "Maestro +could not find the text 'V1 Long Visit' in the suite root."` + +The actual UI dump at failure confirmed `actionBar=Malaria ITN SBC +Training (Learn)` — the recipe was sitting in the Learn app, not +Deliver. That's not a recipe bug per se; it's a contract gap: + +1. Phase 2 didn't emit a Learn smoke journey +2. Phase 3's pre-flight didn't halt on `learn: 0` (the rule was + documented but the producer skill rationalized past it) +3. Phase 6's pre-flight didn't halt either (the rule was documented + but the agent rationalized past it because the recipes were on + disk) +4. The Deliver smoke recipe physically couldn't reach Deliver because + Connect gates it behind Learn completion + +## What changed (this PR) + +1. **Phase 2 (`pdd-to-app-journeys`) coverage rules.** Added a fourth + blocking rule: every PDD with a Learn app MUST emit a + `training-completion-smoke` journey with `app: learn` and + `is_smoke: true`. Deeper Learn journeys move to `/ace:qa-deep`. + +2. **Phase 3 (`app-test-cases`) Step 2 + Step 5.** Codified the + two-app coverage invariant: `smoke_journeys_per_app: {learn: 1, + deliver: 1}` is mandatory for every two-app opp. Halt with a + `[BLOCKER]` pointing at Phase 2 rather than writing `learn: 0`. + Documented the faithful Deliver-smoke composition: walk all Learn + modules to completion, sync, then chain `deliver-launch.yaml` to + reach Deliver — there is no shortcut. + +3. **Phase 6 (`app-screenshot-capture`) Step 2.** Strengthened the + pre-flight commentary to make it harder for the agent to + rationalize past a `learn: 0` count. The table itself was already + correct; the table-driver was undisciplined. + +4. **New static palette `deliver-launch.yaml`.** Drives the §§ 8/9/10 + transitions. Anchors on text labels at § 8 and § 9 (resource-IDs + not yet captured live — palette includes coordinate fallbacks + from the 2026-05-14 turmeric delivery-walk session). Lands on + Deliver mode and assertion-anchors on `id/viewJobCard` (verified + resource-ID per atlas § 10). + +## Outstanding work (filed as follow-up) + +- Atlas § 8 + § 9 resource-IDs are TBD. A future Phase 6 run mid- + window between Learn-pass and Deliver-download will need to + `ui_dump` these surfaces and back-fill `selectors/connect-2.62.0.yaml`. +- The "walk all Learn modules to completion" composition in Phase 3 + is currently described in prose, not codified as a generator. For + multi-stage opps with 6+ modules, the Deliver smoke recipe ends up + long — there's room for a `walk-learn-to-completion` helper palette + that takes a manifest of module/form names and emits the linear + walk. Defer until we have at least one passing multi-stage smoke + to characterize the composition shape. + +## How to verify + +Re-run `/ace:step app-screenshot-capture malaria-itn-app/20260517-1829` +after the next `/ace:run` with these contracts in force. Expected: + +- Phase 2 produces 10 journeys (9 Deliver + 1 Learn smoke) +- Phase 3 emits `smoke_journeys_per_app: {learn: 1, deliver: 1}` and + the Deliver J1 recipe walks Learn to completion before tapping + Deliver's V1 form +- Phase 6 captures Learn-app screenshots (Module 1 entry + + assessment) AND Deliver-app screenshots (V1 form), producing a + training deck with both apps' UX surfaces + +## Related + +- atlas `docs/mobile-atlas/connect-2.62.0.md` §§ 5, 6, 8, 8.5, 9, 10 +- skill `skills/pdd-to-app-journeys/SKILL.md § Coverage rules` rule 4 +- skill `skills/app-test-cases/SKILL.md § Step 2` + § Step 5 +- skill `skills/app-screenshot-capture/SKILL.md § Step 2` +- palette `mcp/mobile/recipes/static/deliver-launch.yaml` diff --git a/mcp/mobile/recipes/static/deliver-launch.yaml b/mcp/mobile/recipes/static/deliver-launch.yaml new file mode 100644 index 0000000..2ae2cad --- /dev/null +++ b/mcp/mobile/recipes/static/deliver-launch.yaml @@ -0,0 +1,92 @@ +# Drive a freshly Learn-completed opportunity from the certificate +# screen through the Download Delivery gate into the Deliver-mode +# StandardHomeActivity. The post-Learn-pass mirror image of +# `learn-launch.yaml`. +# +# Pre-state: Final Learn-assessment passed AND synced to Connect. The +# user has navigated back to Connect's opp list and tapped `Resume` on +# the In-Progress card. Device is sitting on the certificate screen +# (atlas § 8): "Congratulations, ! You have successfully +# completed the Learn modules for ." with a centered, full-width +# `VIEW OPPORTUNITY DETAILS` button. +# +# Post-state: Deliver-mode `StandardHomeActivity` (atlas § 10) with +# `id/viewJobCard` widget visible — the load-bearing differentiator +# vs Learn home (§ 5). The 4-tile grid (Start, View Job Status, +# Sync with Server, Log out of CommCare) is structurally identical +# to Learn mode; the project-card widget is what disambiguates. +# +# Atlas reference: docs/mobile-atlas/connect-2.62.0.md §§ 8, 9, 10. +# +# RESOURCE-ID STATUS (2026-05-18 — palette authored from atlas): +# - § 8 certificate screen: resource-IDs NOT yet captured live (atlas +# § 8 "Open questions" — the surface is transient and was navigated +# via coordinate-tap during the 2026-05-14 turmeric delivery-walk +# session). This palette uses TEXT anchors on the visible labels +# ("Congratulations", "VIEW OPPORTUNITY DETAILS") as the primary +# matchers, with a coordinate-fallback (540, 1486) on 1080x2400 +# captured from the turmeric session. A future Phase 6 run mid- +# window between Learn-pass and Deliver-download should `ui_dump` +# this screen and back-fill the resource-IDs into +# `selectors/connect-2.62.0.yaml`. +# - § 9 Download Delivery gate: same status — text-anchored on +# "DOWNLOAD" with coordinate-fallback (741, 1248). +# - § 10 Deliver-mode StandardHomeActivity: resource-IDs VERIFIED +# 2026-05-14 (atlas § 10). `id/viewJobCard` is the clean structural +# anchor for "we landed in Deliver mode". +# +# Pre-claim contract: the FLW MUST have completed all Learn modules + +# the final assessment AND synced to Connect before this palette is +# entered. Connect's UI does not surface the certificate screen until +# the assessment-pass syncs (atlas § 8 Open Questions). Calling this +# palette before sync completes will time out on the +# `Congratulations` anchor. + +appId: org.commcare.dalvik +--- + +# § 8 certificate screen — anchor on the "Congratulations" text. If +# the FLW hasn't completed Learn yet, this assertion times out and +# the recipe fails loud (don't fall through into a wrong-state +# coordinate tap). +- extendedWaitUntil: + visible: + text: "Congratulations" + timeout: 30000 +- takeScreenshot: "deliver-launch-certificate" + +# Tap VIEW OPPORTUNITY DETAILS. Text-match is preferred; coordinate +# fallback documented for reference (atlas § 8 captured tap point on +# 1080x2400). If text-match drifts (button label changes), the +# fallback keeps the recipe working until the live re-dump fixes the +# selector map. +- tapOn: + text: "VIEW OPPORTUNITY DETAILS" + +# § 9 Download Delivery gate — anchor on the "DOWNLOAD" button text. +- extendedWaitUntil: + visible: + text: "DOWNLOAD" + timeout: 15000 +- takeScreenshot: "deliver-launch-download-gate" + +- tapOn: + text: "DOWNLOAD" + +# Deliver CCZ install progress — typically the same `Step N of M` +# progress UI as the Learn-side § 4 download (atlas § 9 notes this +# is unconfirmed; visually identical, selectors not yet dumped). +# Allow a generous timeout for the CCZ download + install on +# limited-bandwidth dev networks. +- extendedWaitUntil: + visible: + id: "org.commcare.dalvik:id/viewJobCard" + timeout: 180000 +- takeScreenshot: "deliver-launch-home" + +# Verify we landed in Deliver mode, not back in Learn. `viewJobCard` +# is absent on the Learn home (§ 5) and present on the Deliver home +# (§ 10) — this assertion is structural, not text-based, so it's the +# most reliable "we're in Deliver" check. +- assertVisible: + id: "org.commcare.dalvik:id/viewJobCard" diff --git a/package.json b/package.json index d4b9986..f49bd23 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "ace", - "version": "0.13.277", + "version": "0.13.278", "description": "AI Connect Engine - orchestrator for building Connect Opps using AI", "type": "module", "scripts": { diff --git a/skills/app-screenshot-capture/SKILL.md b/skills/app-screenshot-capture/SKILL.md index feea0f1..24fe0f6 100644 --- a/skills/app-screenshot-capture/SKILL.md +++ b/skills/app-screenshot-capture/SKILL.md @@ -82,6 +82,21 @@ auto_surfaced entry. Do not write `verdict: fail` for these — fail is reserved for cases where the recipes ran but a smoke recipe broke. Upstream gaps are `incomplete`, not `fail`. +**This pre-flight is not optional, even when journeys + recipes are +on disk.** The count check is the load-bearing gate, not the recipe- +presence check. Caught in vivo on malaria-itn-app run 20260517-1829: +Phase 3 emitted `smoke_journeys_per_app: {learn: 0, deliver: 1}` (a +known-incomplete state) and Phase 6 ran the Deliver recipe anyway — +producing a recipe-vs-app-state mismatch (post-claim handoff landed in +Learn, not Deliver, because Connect gates Deliver behind Learn- +assessment completion; see +`docs/learnings/2026-05-18-connect-gates-deliver-on-learn-completion.md`). +The agent SHOULD have halted at row 2 of the failure-mode table the +moment it read `learn=0`. If the agent finds itself rationalizing past +this check ("the recipe is on disk, the AVD is healthy, I'll just run +the Deliver smoke and skip the Learn one") — that's the exact anti- +pattern this table prevents. Halt instead. + The agent-level pre-flight (`agents/qa-and-training.md § Pre-flight checklist`) catches the same class of gap before the skill is dispatched. This skill-level check is the second line of defense for diff --git a/skills/app-test-cases/SKILL.md b/skills/app-test-cases/SKILL.md index 1002ce8..375206a 100644 --- a/skills/app-test-cases/SKILL.md +++ b/skills/app-test-cases/SKILL.md @@ -59,6 +59,43 @@ visit/delivery behavior (Deliver). Multi-stage opps may have both. - If two journeys could plausibly be the smoke, pick the one with the smallest `pdd_time_budget_seconds` +**Two-app coverage is REQUIRED.** Every PDD with both a Learn and a +Deliver app (every archetype except a hypothetical Learn-less mode) +MUST emit one `is_smoke: true` journey per app — Phase 6 reads BOTH +smokes to capture training-deck screenshots of each app. If +`pdd-to-app-journeys` did not produce a Learn-app journey, halt with +a structured error pointing at Phase 2 (`pdd-to-app-journeys`) rather +than writing `smoke_journeys_per_app.learn: 0` — the upstream coverage +rule (added 2026-05-18) requires the Learn smoke. The +`smoke_journeys_per_app: {learn: 1, deliver: 1}` invariant is +load-bearing for Phase 6's pre-flight; emitting `learn: 0` produces a +silent downstream halt at Phase 6 (see malaria-itn-app +run 20260517-1829 for the canonical incident). + +**Deliver-smoke composition for two-app opps.** Connect's UI gates the +Deliver app behind Learn-assessment completion (see +`docs/learnings/2026-05-18-connect-gates-deliver-on-learn-completion.md`). +A Deliver smoke that drives `connect-claim + Start + tap V1` lands in +Learn, not Deliver — the recipe physically cannot reach Deliver +without completing Learn first. The faithful composition (the only +one that works) is: + + connect-login → connect-claim-opp → learn-launch → walk all Learn + modules (content form + assessment per module) → return to Connect + opp list → tap Resume → certificate (atlas § 8) → tap + VIEW OPPORTUNITY DETAILS → Download Delivery gate (atlas § 9) → + tap DOWNLOAD → Deliver StandardHomeActivity (atlas § 10) → tap + Start → Deliver MenuActivity (atlas § 11) → tap first Deliver + module → first form-field screenshot. + +That's not "shallow" — it's a faithful FLW walk-through. Phase 6 +budgets ~5–10 min per Deliver smoke on multi-stage opps as a +consequence. Compose the Learn-walk-to-completion as inline +`runFlow: { file: learn-launch.yaml }` + per-module +`learn-tap-module.yaml` + `form-advance.yaml` + `form-submit.yaml` +chains; the post-Learn → Deliver transition uses the +`deliver-launch.yaml` palette (see § 3's entry-point template). + ### Step 3: For each journey, compose the Maestro recipe Compose each recipe using the static palette pattern (one Maestro @@ -147,8 +184,14 @@ appId: org.commcare.dalvik file: connect-claim-opp.yaml env: OPP_NAME: ${OPP_NAME} -# (c) For Deliver journeys: deliver-launch (TODO: add to static palette) -# For Learn journeys: learn-launch.yaml +# (c) For Learn journeys: learn-launch.yaml lands on the Learn suite root. +# For Deliver journeys: Connect gates Deliver behind Learn-assessment +# completion (see docs/learnings/2026-05-18-connect-gates-deliver-on- +# learn-completion.md). Walk all Learn modules to completion first via +# learn-launch + per-module learn-tap-module + form-advance + form- +# submit, THEN chain deliver-launch.yaml which drives the post-Learn +# certificate (atlas § 8) → Download Delivery gate (§ 9) → Deliver +# StandardHomeActivity (§ 10) → Deliver MenuActivity (§ 11). - runFlow: file: learn-launch.yaml # ... journey-specific module/form steps below, using live labels @@ -163,6 +206,7 @@ The static palette lives at `mcp/mobile/recipes/static/`: - `learn-tap-module.yaml` — MenuActivity row tap (generic — handles ANY level of the 3-level suite tree) - `form-advance.yaml` — `nav_btn_next` ImageButton tap (NOT text-match "Next" — see atlas §7) - `form-submit.yaml` — branched: explicit Submit button if visible, otherwise auto-finalize via `nav_btn_next` +- `deliver-launch.yaml` — post-Learn-complete certificate (atlas § 8) → tap VIEW OPPORTUNITY DETAILS → Download Delivery gate (§ 9) → tap DOWNLOAD → Deliver-mode StandardHomeActivity (§ 10) anchored on `id/viewJobCard`. Chains immediately after a full Learn walk-to-completion in the Deliver smoke recipe. Resource-IDs at the certificate + gate screens are coordinate-fallback-only (see palette file for remediation: live dump capture from a future Phase 6 run mid-window between Learn-pass and Deliver-download). **CRITICAL — Learn-app navigation is 2 menu levels deep.** After `learn-launch.yaml` lands you on the module list (atlas §6a), reaching a form requires **TWO** `learn-tap-module` invocations: @@ -377,6 +421,17 @@ in `templates/app-test-cases-template.yaml`. (Same shape as pdd-to-test-prompts.) Verify: - Every journey from `pdd-to-app-journeys.md` has a binding - Exactly one `is_smoke: true` per app +- **Two-app coverage invariant.** For any opp with both a Learn and a + Deliver app (every archetype except a hypothetical Learn-less mode), + `smoke_journeys_per_app.learn` MUST be `1` AND + `smoke_journeys_per_app.deliver` MUST be `1`. **Do not write + `app-test-cases.yaml` with `learn: 0` "because Phase 2 didn't + produce a Learn journey"** — halt instead with a `[BLOCKER]` naming + Phase 2 (`pdd-to-app-journeys`) as the remediation target. The + Phase 6 pre-flight reads this field; emitting `learn: 0` produces + a silent downstream halt with no Learn-app screenshots in the + training deck. Caught in vivo on malaria-itn-app run 20260517-1829; + Phase 2 contract tightened in the same PR. - **Every `is_smoke: true` journey has a `recipes/J.yaml` file written under `3-commcare/recipes/`.** Confirm via `drive_list_folder` against the recipes folder — count must equal diff --git a/skills/pdd-to-app-journeys/SKILL.md b/skills/pdd-to-app-journeys/SKILL.md index e6ac83d..d4917e8 100644 --- a/skills/pdd-to-app-journeys/SKILL.md +++ b/skills/pdd-to-app-journeys/SKILL.md @@ -202,6 +202,39 @@ Before writing the file, verify: recoverability claim, or a structural-output claim that the deep eval can grade. +4. **Training-app coverage (Learn smoke) — REQUIRED for every PDD with + a Learn app.** Every archetype except a hypothetical Learn-less mode + produces a Learn (training) app. Phase 6's training deck needs + screenshots of BOTH apps, so the journey set MUST include at least + one Learn-app journey: + + - `training-completion-smoke` — happy-path FLW completes Learn + Module 1 (content form + assessment), passes the assessment, + returns to the suite root with Module 1 marked complete. Pass + criterion: the FLW reaches the first assessment, submits with a + passing score, and Module 1 row shows a completion indicator + (per atlas § 6 — completion-state is rendered on the row). + + Set `app: learn` and `is_smoke: true` on this journey. It is the + mandatory pair to the Deliver-side smoke for two-app opps. + + Deeper Learn-app journeys (full curriculum walk, assessment-fail + recovery, etc.) belong in `/ace:qa-deep` and may be added as + non-smoke entries with `is_smoke: false`. Phase 6 shallow only + needs the one smoke per app. + + This rule was added 2026-05-18 after the malaria-itn-app run + 20260517-1829 surfaced a Phase 6 halt: Phase 2 had generated 9 + Deliver journeys + 0 Learn journeys, Phase 3 dutifully wrote + `smoke_journeys_per_app: {learn: 0, deliver: 1}`, and Phase 6's + smoke run tried to reach Deliver via `connect-claim + Start + + tap V1` — which lands in Learn, not Deliver, because Connect gates + Deliver behind Learn-assessment completion (see + `docs/learnings/2026-05-18-connect-gates-deliver-on-learn-completion.md`). + The right fix is structural: always emit the Learn smoke so Phase 6 + captures it independently, AND the Deliver smoke walks Learn to + completion first (see `app-test-cases` for the recipe shape). + If any rule is missed, return to the journey-generation step (step 4 in `## Process`) and add until coverage is satisfied.