Skip to content

fix: CEO reuses existing GitHub issues instead of creating duplicates#375

Open
lukeinglis wants to merge 2 commits into
akashgit:mainfrom
lukeinglis:fix/ceo-issue-dedup
Open

fix: CEO reuses existing GitHub issues instead of creating duplicates#375
lukeinglis wants to merge 2 commits into
akashgit:mainfrom
lukeinglis:fix/ceo-issue-dedup

Conversation

@lukeinglis
Copy link
Copy Markdown
Collaborator

Summary

  • Add pre-check in ceo.md step 2c (Improve mode) and R3a (Research mode) to parse hypothesis text for issue references (#NNN)
  • If an existing open issue is found, reuse it instead of creating a duplicate
  • Prompt-only change, no code modifications

Fixes #372

Add pre-check in ceo.md step 2c (Improve) and R3a (Research) to parse
hypothesis text for issue references and reuse existing open issues
instead of always creating new ones.

Fixes akashgit#372

Signed-off-by: Luke Inglis <lukeinglis21@yahoo.com>
@codecov
Copy link
Copy Markdown

codecov Bot commented May 25, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 87.39%. Comparing base (8097461) to head (fc1a0a4).
⚠️ Report is 17 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #375      +/-   ##
==========================================
+ Coverage   87.31%   87.39%   +0.08%     
==========================================
  Files          61       61              
  Lines        9339     9403      +64     
==========================================
+ Hits         8154     8218      +64     
  Misses       1185     1185              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copy link
Copy Markdown
Collaborator

@xukai92 xukai92 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Dedup intent is right and the change is contained, but as-is the Builder will silently follow a stale spec whenever an issue gets reused — that's a meaningful regression risk. Two smaller robustness concerns on the matching pattern and the gh invocation are noted as well. Once the reused issue gets refreshed with the new experiment context, happy to approve.

[P2 — blocking] Reused issue body becomes stale w.r.t. current experiment

factory/agents/prompts/ceo.md around the new pre-check (lines ~1146-1156 and ~2074-2085):

When the pre-check reuses an existing open issue, $ISSUE_NUM is set but the issue body still references whatever $EXP_ID, hypothesis text, acceptance criteria, and (for operational/mixed hypotheses) ## Execution Step / ## Execution Acceptance Criteria it had at original creation. The Builder in step 2d does gh issue view $ISSUE_NUM and is instructed to "implement exactly what the issue describes" — so it will follow the old issue body, not the current CEO-approved hypothesis.

This silently re-decouples the experiment from the work being done: wrong $EXP_ID recorded on the PR, stale acceptance criteria for operational/mixed hypotheses, possibly stale surface constraints in research mode.

Suggested fix — after deciding to reuse, post an update comment with the new experiment context before invoking the Builder:

3. If the issue is open, reuse it: set `$ISSUE_NUM` to that number, post an update comment with the current experiment context, and **skip issue creation**:
   ```bash
   gh issue comment "$ISSUE_NUM" --body "**Updated for experiment $EXP_ID**

   ## Current hypothesis
   <hypothesis text>

   ## What to build (this cycle)
   <specific changes>

   ## Acceptance criteria (this cycle)
   - [ ] <outcomes>
   - [ ] Tests pass
   - [ ] Eval score does not regress"
   ```
4. Only create a new issue if no existing open reference is found

[P3] #[0-9]+ is too aggressive and has no tie-break for multiple matches

The patterns list narrow forms (Addresses: #NNN, issue #NNN), but step 1 collapses to bare #[0-9]+, which matches "PR #123", "task #4", section anchors, and version tags. There's also no guidance on what to do when multiple #NNN references appear — first match wins by default, which could be the wrong one (e.g., a referenced PR rather than the tracking issue).

Suggested ordering:

  1. Prefer explicit Addresses:? #NNN, **Backlog item:** #NNN, or issue #NNN matches first.
  2. If exactly one bare #NNN is present and no narrow match was found, fall back to it.
  3. If multiple bare #NNN references remain ambiguous, create a new issue and link the others in the body.

[P3] gh issue view is not scoped to the project repo

The instructions don't require the CEO to cd "$PROJECT_PATH" (or pass -R owner/repo) before the lookup. If the CEO subprocess runs from a different cwd or in a worktree without an upstream, gh issue view may resolve to the wrong repo or fail. The "if open" branch is also ambiguous about what to do on gh non-zero exit — silent fall-through to "no existing issue" is fine but should be stated explicitly.

Suggested fix:

(cd "$PROJECT_PATH" && gh issue view <number> --json state --jq '.state') 2>/dev/null

If the command fails or returns anything other than OPEN, treat as no existing issue and proceed to create a new one.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: Luke Inglis <lukeinglis21@yahoo.com>
@lukeinglis
Copy link
Copy Markdown
Collaborator Author

Addressed all three review points. Changes applied to both pre-check blocks (Improve step 2c and Research R3a):

[P2] Stale issue body: Step 3 now posts a gh issue comment with the current experiment ID, hypothesis, acceptance criteria, and what to build before the Builder is invoked. This ensures the Builder always works from current context even when reusing an old issue.

[P3] Pattern matching precision: Replaced the aggressive bare #[0-9]+ collapse with priority-ordered matching:

  1. Explicit references first: Addresses:? #NNN, Backlog item: #NNN, issue #NNN
  2. Fallback to bare #NNN only if exactly one match and no explicit match found
  3. Multiple ambiguous bare refs: skip reuse, create new issue and link the others

[P3] Repo-scoped gh issue view: Now runs via (cd "$PROJECT_PATH" && gh issue view ...) 2>/dev/null with explicit failure handling: non-zero exit or non-OPEN state both fall through to new issue creation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

CEO should reuse existing GitHub issues instead of creating duplicates

2 participants