feat: memory-lint Phase B.5 trial (provenance + contradiction detection + surfacing rule)#124
Merged
Conversation
Adds cross-file contradiction detection (Phase B.5) to the nightly memory-consolidation skill, gated by LINT_PHASE_B5_ENABLED for a 30-day trial. Phase C now persists confidence and revisit_if in frontmatter; Phase D writes a Pending Review section to MEMORY.md, appends a structured JSON line to memory/lint-stats.jsonl, and uses a parseable diary header prefix for recent-activity grepping. Auto-resolve uses evidence > confidence > recency hierarchy with anti-loop frontmatter fields (resolved_at / resolution_basis / do_not_reopen_before) to prevent re-triggering the same contradiction nightly. Time-scoped evolution is explicitly excluded from contradiction treatment. Mutation limit shared with Phase C; never silent-delete — losing claims get a (superseded: ...) annotation. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Adds platform rule requiring the agent to proactively surface pending Phase B.5 lint contradictions in conversation: topic-related inline trigger, 14-day aged escalation, one-per-session cap, no-interrupt-urgency, and after-resolution cleanup that updates both the affected memory file and the workspace MEMORY.md "Pending Review" section. References ADR-069. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Substantive corrections to the memory-consolidation SKILL.md spec and the companion memory-protocol rule, surfaced by the post-implementation review: - Mutation budget: Phase C re-zeroed mutations_applied, contradicting Phase B.5's "shared budget of 5". Initialize the counter in Phase B.5 and have Phase C continue it; explicitly skip Phase C if Phase B.5 already spent it. - Step 4c referenced evidence_date / updated_at — fields the frontmatter schema never persists. Replaced with resolved_at (the field that actually exists) and acknowledged the rule mostly defers to flag-for-review. - Step 4b: handle legacy files lacking the confidence field (default 0.7). - Per-pair anti-loop: do_not_reopen_before now scopes to a specific partner via the new do_not_reopen_partner field, rather than excluding the file wholesale from all future scans (which would silently hide unrelated third-party contradictions). - Step 5: edits now MUST use safe-edit.sh, same backup/verify/rollback flow as Phase C. Previously unprotected. - Picked one annotation style (trailing parenthetical) instead of offering "strikethrough or parenthetical" as alternatives. - Phase C frontmatter template no longer inlines commented optional fields — those got copied verbatim into new files when the LLM transcribed the template literally. Moved to prose with explicit "do not include". - Dropped the unused claim_phrases accumulator from step 1's representation. - Pending Review bullets now use a strict, machine-parseable format (detected_at=YYYY-MM-DD prefix) so pending_total and avg_age_days math has a defined parser. Added deduplication and section-removal scoping. - Sanitization rules added for free-text fields (resolution_basis, revisit_if, reason bullets): single-line, ≤200 chars, no leading #, escape ". - Auto-resolve mutation count: 1 per resolved pair (not 2 per file edit). - LLM judgment errors now treated as `unrelated` and logged. Plan validation greps still pass. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- Convert do_not_reopen_partner (scalar) to do_not_reopen_partners (list) with APPEND-not-overwrite semantics. The scalar form silently broke the anti-loop guarantee when a file participated in more than one resolved contradiction over time — the later resolution overwrote the earlier partner record, allowing the prior pair to re-trigger. - Replace literal "N" placeholders in lint-stats.jsonl JSON template with explicit <integer> / <number> angle-bracket form; add a jq validation step before append so malformed lines never corrupt the trial dataset. - Define avg_age_days formula explicitly (mean(today − detected_at); today is the same YYYY-MM-DD used in the same stats line's "date" field; round to 1 decimal). - Document SUSPICIOUS_SHRINK bypass for the Pending Review section removal case — without it, the cleanup edit silently rolls back once the section has grown to more than ~20% of MEMORY.md, leaving resolved bullets in place forever. - Update memory-protocol.md after-resolution rule to reference the new field name and document the append-not-overwrite requirement so human resolutions preserve the anti-loop chain. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- LLM judgment now returns {verdict, claim_a, claim_b} so step 5 has an exact-match anchor for the (superseded:) annotation; downgrade to unrelated if anchor doesn't appear in body
- Sanitization rules extended to do_not_reopen_before when it carries a semantic condition (date form remains unquoted YAML)
- Missing do_not_reopen_before with partner listed is now explicitly "exclusion inactive" rather than undefined
- Mutation budget consistent: Phase B.5 counts each file edit (pair=2), matching Phase C's per-file rule; pre-check verifies remaining budget fits a full pair before starting
- SUSPICIOUS_SHRINK bypass clarifies the section size MUST be measured from the backup before applying the edit, not live during the write
- Phase D preamble pins pending_added to the post-dedup value so diary line and JSONL row agree
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- Feature flag default flips to enabled during 2026-05-17 → 2026-06-17 trial window (unset env var was previously treated as skip, leaving the trial dead-on-arrival without runtime env wiring). - Phase B.5 step 1 now extracts `body_predicates` and `negation_markers` from file bodies; step 2 candidate signals reference those extracted sets instead of demanding body content the rep didn't carry. - `do_not_reopen_partners` entries normalized to bare basename at read time and deduped, closing an anti-loop bypass when prior cycles wrote heterogeneous path formats. - Step 5 paired edits now use explicit two-phase commit semantics (backup BOTH, edit BOTH, verify BOTH, then clean BOTH) so a partial failure can't leave file A annotated while file B is untouched. - Step 5 anchor re-verify added: before applying each pair's edit, re-confirm `claim_a`/`claim_b` still appears verbatim in the current file body — a prior pair in the same run may have already annotated the same line. - Mutation budget accounting timing pinned: increments fire only after both files verify and clean successfully; rolled-back pairs do not consume budget. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Six surgical spec fixes to memory-consolidation/SKILL.md and memory-protocol.md based on the 5th review pass: - title_tokens: explicit fallback when both heading and description absent - body_predicates: drop multi-token "do not" (undefined under "whole-word match"); already covered by negation_markers - pending_review canonical entry shape unified across steps 4d, 5, 6 so Phase D's parser regex matches every deferral uniformly - pending_added counter: removed in-phase increment, Phase D step 2 is the single source via post-dedup recompute - do_not_reopen_before: take MAX(existing, new) when persisting; never shorten an earlier pair's exclusion window (anti-loop preservation) - SUSPICIOUS_SHRINK bypass: reframed measurement from pre-edit MEMORY.md instead of requiring runner to know .consolidation-backup suffix - JSONL append: mandate trailing newline (printf '%s\\n') to keep file parseable as one JSON object per line memory-protocol.md updated to lockstep on the do_not_reopen_before MAX rule for the interactive resolution path. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- memory-protocol.md: add "if absent" branch and today+90 default for do_not_reopen_before; agent-driven interactive resolution previously had no guidance on what value to write when the field was missing - Phase D step 2: dedup key now (file-A, file-B, reason) so two genuinely distinct contradictions between the same pair aren't silently dropped - Phase D step 2: reason sanitization strips em-dash so it can't collide with the ` — ` field separator in the bullet format - Phase D step 1 / template: gate `### Lint (Phase B.5)` block on LINT_PHASE_B5_ENABLED so post-trial diary entries don't emit zeroed stats Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…rom ralphex run) The ralphex run for memory-lint-trial inadvertently reverted the `permissions: pull-requests: read` block in .github/workflows/pii-scan.yml. That block was added in commit b909b20 to fix private-repo gitleaks permissions (PR #123). Restore to origin/main state — out of scope for this PR.
There was a problem hiding this comment.
Pull request overview
Adds a 30-day trial specification for memory linting: cross-file contradiction detection in the memory consolidation skill, pending-review surfacing rules, and supporting trial documentation. It also restores explicit GitHub Actions permissions for the local PII scan workflow.
Changes:
- Adds Phase B.5 linting guidance, frontmatter fields, Pending Review handling, and lint stats output to
memory-consolidation. - Adds platform rules for proactively surfacing and resolving pending lint findings.
- Archives the implementation plan and restores
pull-requests: readpermissions in the PII scan wrapper workflow.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
.claude/skills/memory-consolidation/SKILL.md |
Defines the Phase B.5 contradiction-detection trial, frontmatter schema, pending-review updates, and stats/diary behavior. |
.claude/rules/platform/memory-protocol.md |
Adds conversation-time surfacing and interactive resolution protocol for pending lint findings. |
.github/workflows/pii-scan.yml |
Adds explicit token permissions for the local PII scan workflow. |
docs/plans/completed/2026-05-17-memory-lint-trial.md |
Archives the trial plan and validation checklist. |
Comments suppressed due to low confidence (3)
.claude/skills/memory-consolidation/SKILL.md:240
- Phase D now assumes the Phase B.5 accumulators exist, but the pipeline can skip directly to Phase D on any fatal failure before Phase B.5 runs. In that path
pending_resolved,pending_review, and the lint counters are undefined, so the error-reporting/cleanup phase can fail instead of recording the original failure. Initialize default lint accumulators before any phase that can fail, or have Phase D guard the lint substeps on accumulator availability as well as the feature flag.
Phase D substeps do NOT execute in literal numerical order. The diary (step 1) is written LAST among steps 1–3 so it can summarize the actual outcomes of steps 2 and 3 — including their issues — in a single write (the spec does NOT support amending an already-written diary section). Step 3 is also split: its compute-and-validate work runs early (to surface any issues), and the actual JSONL append runs after the diary. Execution order:
1. **Step 2** (MEMORY.md "Pending Review" edit) — applies `pending_resolved` removals first, then dedup-appends `pending_review` adds in a single MEMORY.md mutation, finalizes `pending_added`; may surface a `SUSPICIOUS_SHRINK` rollback issue.
2. **Step 3 compute-and-validate** — compute `pending_total` and `avg_age_days` from the post-step-2 MEMORY.md, compose the JSONL line, validate it via `jq -e`. This sub-step MAY surface issues to the Issues collector: malformed bullets whose `detected_at` fails to parse (counted with age 0; logged), or a candidate JSON line that fails `jq` validation (NOT appended; logged). Do NOT perform the actual `>>` append yet — defer it to after the diary.
3. **Step 1** (diary) — write diary using the finalized `pending_added` AND the full Issues set collected in (1) and (2).
4. **Step 3 append** — perform the actual `printf '%s\n' "$LINE" >> memory/lint-stats.jsonl`, only if (2)'s `jq` validation passed.
.claude/skills/memory-consolidation/SKILL.md:124
- Using any recent
### Memory Changesmention as “direct diary/session evidence” conflates user-grounded updates with automated lint edits from prior runs. Phase B.5 itself updatesmemory/autofiles and the diary records per-file create/update logs, so a file touched only by an auto-resolve can be treated as fresh conversation evidence for a different contradiction within 48 hours. That can make the evidence leg pick the wrong winner; the provenance check should distinguish human-session-derived Phase B updates from Phase B.5 maintenance edits.
**Provenance extraction for step 4a.** Populate `recent_diary_mention: bool` by scanning diary files in `memory/diary/` whose filename date (`YYYY-MM-DD.md`) is within the last 48 hours of today's date. In each diary file, look only at lines that fall under a `### Memory Changes` subsection (between that heading and the next `### ` or `## ` heading or EOF — Phase D step 1 writes `### Memory Changes` as the canonical heading for the per-file create/update log). Set `recent_diary_mention = true` if any such line contains either (a) the file's bare basename (e.g., `topic-slug.md`) as a substring, OR (b) the path form `memory/auto/<basename>` as a substring. This is the operational definition of "direct diary/session evidence" used by step 4a — a file mentioned in a recent diary's Memory Changes block was grounded by a real conversation event, whereas a file absent from recent diaries is older/inferred. No frontmatter field is required; provenance is derived entirely from the diary log Phase D already writes. If no diary files exist or none fall within 48 hours, `recent_diary_mention = false` for all files (step 4a then yields no winner and the resolver falls through to step 4b).
.claude/skills/memory-consolidation/SKILL.md:270
- The diary template puts explanatory text on the same line as the required heading. If the agent copies this format literally, the Markdown heading becomes
### Memory Changes # REQUIRED...rather than exactly### Memory Changes, while Phase B.5's provenance scan depends on that exact subsection name. Move the explanation outside the code block or use a real HTML/Markdown comment on a separate line so the required heading remains literal.
### Memory Changes # REQUIRED heading — do not rename; Phase B.5 step 1 parses this heading text to populate `recent_diary_mention` (renaming silently breaks the evidence leg of the auto-resolve hierarchy).
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Resolves Copilot review on PR #124: the previous wording was ambiguous — top section presented LINT_PHASE_B5_ENABLED as if it were an editable in-file value, but the Phase B.5 paragraph described it as an env var where only an explicit 'false' disables it. A rollback per the documented 'edit this file' path would have left the env-var check unaffected. Clarification: SKILL.md is the SOLE source of truth for the flag. Not an env var, not external config. To roll back: edit the line in the Feature Flags section to LINT_PHASE_B5_ENABLED=false, commit, next nightly cron run picks up the new value.
Owner
Author
|
@copilot please re-review — fixes pushed since the last review |
Re-reviewed the latest updates at |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
30-day trial of automated cross-file contradiction detection in workspace memory, with proactive in-conversation surfacing.
Phase B.5tomemory-consolidationskill: cross-file contradiction detection via cheap candidate generation → LLM judgment on candidates only (per codex-recommended algorithm). Auto-resolve hierarchy: evidence > confidence (the "recency" leg was dropped during review —resolved_atis not a valid freshness proxy for the current claim).confidence(0.0–1.0 float, matches existing Phase B scoring),revisit_if(semantic trigger, ADR-style), plus optionalresolved_at/resolution_basis/do_not_reopen(per-pair cooldown list, NOT global)..claude/rules/platform/memory-protocol.md— agent MUST proactively surface pending items in conversation (topic-relevant preferred, aged escalation >14 days, one-per-session max).LINT_PHASE_B5_ENABLED— rollback = flip flag to false, no data migration.Trial mechanism (30 days)
memory/lint-stats.jsonl(created on first run)## Pending Review (Lint findings)section in workspaceMEMORY.mdRollback criteria
If any of the following during trial: flip
LINT_PHASE_B5_ENABLED=falseand re-evaluate.Changes
.claude/skills/memory-consolidation/SKILL.md(+128 / -10): feature flag, Phase B.5, expanded Phases C and D, frontmatter format, stats file format, parseable diary prefix.claude/rules/platform/memory-protocol.md(+18): new section "Surfacing pending lint items"docs/plans/completed/2026-05-17-memory-lint-trial.md(+157): archived plan file3 files, +293/-10. Plan archived to
completed/.Test plan
grep -q 'LINT_PHASE_B5_ENABLED' .claude/skills/memory-consolidation/SKILL.mdpassesgrep -q '### Phase B.5' .claude/skills/memory-consolidation/SKILL.mdpassesgrep -q 'evidence > confidence' .claude/skills/memory-consolidation/SKILL.mdpassesgrep -q 'resolved_at' .claude/skills/memory-consolidation/SKILL.mdpassesgrep -q '## Surfacing pending lint items' .claude/rules/platform/memory-protocol.mdpassesmemory/lint-stats.jsonlcreated, no errorsReferences
@-import)Notes for reviewer
do_not_reopenevolved from a scalar to a per-pair{partner, before}list during review to prevent unrelated resolutions from biasing future contradictions.permissions: pull-requests: readblock in.github/workflows/pii-scan.yml(added in PR ci: pii-scan needs explicit pull-requests:read for private consumers #123 this morning). Restored in a separate commit on this branch.🤖 Generated with Claude Code