fix: CEO reuses existing GitHub issues instead of creating duplicates#375
fix: CEO reuses existing GitHub issues instead of creating duplicates#375lukeinglis wants to merge 2 commits into
Conversation
Add pre-check in ceo.md step 2c (Improve) and R3a (Research) to parse hypothesis text for issue references and reuse existing open issues instead of always creating new ones. Fixes akashgit#372 Signed-off-by: Luke Inglis <lukeinglis21@yahoo.com>
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #375 +/- ##
==========================================
+ Coverage 87.31% 87.39% +0.08%
==========================================
Files 61 61
Lines 9339 9403 +64
==========================================
+ Hits 8154 8218 +64
Misses 1185 1185 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
xukai92
left a comment
There was a problem hiding this comment.
Dedup intent is right and the change is contained, but as-is the Builder will silently follow a stale spec whenever an issue gets reused — that's a meaningful regression risk. Two smaller robustness concerns on the matching pattern and the gh invocation are noted as well. Once the reused issue gets refreshed with the new experiment context, happy to approve.
[P2 — blocking] Reused issue body becomes stale w.r.t. current experiment
factory/agents/prompts/ceo.md around the new pre-check (lines ~1146-1156 and ~2074-2085):
When the pre-check reuses an existing open issue, $ISSUE_NUM is set but the issue body still references whatever $EXP_ID, hypothesis text, acceptance criteria, and (for operational/mixed hypotheses) ## Execution Step / ## Execution Acceptance Criteria it had at original creation. The Builder in step 2d does gh issue view $ISSUE_NUM and is instructed to "implement exactly what the issue describes" — so it will follow the old issue body, not the current CEO-approved hypothesis.
This silently re-decouples the experiment from the work being done: wrong $EXP_ID recorded on the PR, stale acceptance criteria for operational/mixed hypotheses, possibly stale surface constraints in research mode.
Suggested fix — after deciding to reuse, post an update comment with the new experiment context before invoking the Builder:
3. If the issue is open, reuse it: set `$ISSUE_NUM` to that number, post an update comment with the current experiment context, and **skip issue creation**:
```bash
gh issue comment "$ISSUE_NUM" --body "**Updated for experiment $EXP_ID**
## Current hypothesis
<hypothesis text>
## What to build (this cycle)
<specific changes>
## Acceptance criteria (this cycle)
- [ ] <outcomes>
- [ ] Tests pass
- [ ] Eval score does not regress"
```
4. Only create a new issue if no existing open reference is found[P3] #[0-9]+ is too aggressive and has no tie-break for multiple matches
The patterns list narrow forms (Addresses: #NNN, issue #NNN), but step 1 collapses to bare #[0-9]+, which matches "PR #123", "task #4", section anchors, and version tags. There's also no guidance on what to do when multiple #NNN references appear — first match wins by default, which could be the wrong one (e.g., a referenced PR rather than the tracking issue).
Suggested ordering:
- Prefer explicit
Addresses:? #NNN,**Backlog item:** #NNN, orissue #NNNmatches first. - If exactly one bare
#NNNis present and no narrow match was found, fall back to it. - If multiple bare
#NNNreferences remain ambiguous, create a new issue and link the others in the body.
[P3] gh issue view is not scoped to the project repo
The instructions don't require the CEO to cd "$PROJECT_PATH" (or pass -R owner/repo) before the lookup. If the CEO subprocess runs from a different cwd or in a worktree without an upstream, gh issue view may resolve to the wrong repo or fail. The "if open" branch is also ambiguous about what to do on gh non-zero exit — silent fall-through to "no existing issue" is fine but should be stated explicitly.
Suggested fix:
(cd "$PROJECT_PATH" && gh issue view <number> --json state --jq '.state') 2>/dev/nullIf the command fails or returns anything other than OPEN, treat as no existing issue and proceed to create a new one.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Signed-off-by: Luke Inglis <lukeinglis21@yahoo.com>
|
Addressed all three review points. Changes applied to both pre-check blocks (Improve step 2c and Research R3a): [P2] Stale issue body: Step 3 now posts a gh issue comment with the current experiment ID, hypothesis, acceptance criteria, and what to build before the Builder is invoked. This ensures the Builder always works from current context even when reusing an old issue. [P3] Pattern matching precision: Replaced the aggressive bare #[0-9]+ collapse with priority-ordered matching:
[P3] Repo-scoped gh issue view: Now runs via (cd "$PROJECT_PATH" && gh issue view ...) 2>/dev/null with explicit failure handling: non-zero exit or non-OPEN state both fall through to new issue creation. |
Summary
ceo.mdstep 2c (Improve mode) and R3a (Research mode) to parse hypothesis text for issue references (#NNN)Fixes #372