Skip to content

feat: safe Anthropic disablement via ANTHROPIC_DISABLED flag#4216

Open
rysweet wants to merge 3 commits intomainfrom
feat/issue-4215-anthropic-disablement
Open

feat: safe Anthropic disablement via ANTHROPIC_DISABLED flag#4216
rysweet wants to merge 3 commits intomainfrom
feat/issue-4215-anthropic-disablement

Conversation

@rysweet
Copy link
Copy Markdown
Owner

@rysweet rysweet commented Apr 3, 2026

Closes #4215

Summary

Implements a safe disablement strategy for all Anthropic-dependent components via ANTHROPIC_DISABLED=true environment variable.

Changes

  • fleet/_backends.py: AnthropicBackend raises ConfigurationError when disabled; auto_detect_backend() falls back to CopilotBackend; _SecretValue wrapper prevents API key leakage in repr/tracebacks
  • eval/gherkin_agent_evaluator.py: _has_anthropic_api_key() returns False and logs warning when disabled
  • agents/goal_seeking/hive_mind/query_expansion.py: honours flag; uses local synonym fallback when disabled
  • vendor/blarify/agents/llm_provider.py: guarded langchain-anthropic import with try/except
  • vendor/blarify/agents/rotating_provider/rotating_anthropic.py: guarded anthropic import with try/except + warning

Tests

41 tests pass (new tests in tests/test_gherkin_evaluator_guard.py, tests/hive_mind/test_query_expansion.py, src/amplihack/fleet/tests/test_backends.py)

Docs

docs/ANTHROPIC_DISABLEMENT.md — full operator guide

Co-authored-by: Copilot 223556219+Copilot@users.noreply.github.com

Copilot and others added 3 commits April 3, 2026 22:12
Add ANTHROPIC_DISABLED=true guard checks across all Anthropic-dependent
components so they fail gracefully with clear error messages rather than
silently or with obscure SDK errors.

Changes:
- fleet/_backends.py: AnthropicBackend raises ConfigurationError when
  ANTHROPIC_DISABLED=true; auto_detect_backend skips Anthropic and falls
  back to CopilotBackend; exports ConfigurationError in __all__
- eval/gherkin_agent_evaluator.py: _has_anthropic_api_key() returns False
  and logs warning when ANTHROPIC_DISABLED=true
- agents/goal_seeking/hive_mind/query_expansion.py: replace bare print()
  with logger.warning(); honour ANTHROPIC_DISABLED flag to force local
  synonym fallback even when SDK is installed
- vendor/blarify/agents/llm_provider.py: guard langchain-anthropic import
  and RotatingKeyChatAnthropic behind try/except so missing package emits
  warning rather than crashing at import time
- vendor/blarify/agents/rotating_provider/rotating_anthropic.py: guard
  anthropic import with try/except + warning

New tests:
- tests/test_gherkin_evaluator_guard.py: covers ANTHROPIC_DISABLED guard
  in gherkin evaluator
- tests/hive_mind/test_query_expansion.py: covers disabled flag and
  graceful fallback in query_expansion
- src/amplihack/fleet/tests/test_backends.py: adds ConfigurationError and
  auto_detect fallback tests (40 total, all pass)

All 40 tests pass. Every disabled path logs at minimum logger.warning
with the controlling flag name before failing or falling back.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Wrap self.api_key in a _SecretValue sentinel so the key never appears
in repr(), str(), or tracebacks. Use get_secret_value() only at the
SDK call site.

Also adds test_api_key_masked_in_repr to cover this contract.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…ingKeyChatAnthropic

- Add ANTHROPIC_DISABLED=true guard to RotatingKeyChatAnthropic.__init__
- Add comprehensive docs/ANTHROPIC_DISABLEMENT.md documentation
- Register doc in mkdocs.yml Reference section

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Apr 3, 2026

Repo Guardian - Action Required

Two files in this PR appear to be ephemeral workstream artifacts that were accidentally committed and do not belong in the repository.


run.sh (added — repo root)

Why flagged: This script was auto-generated by the workstream runner for issue #4215. It contains hardcoded session-specific values that will never be valid outside the original development environment:

cd /tmp/amplihack-workstreams/ws-4215          # temp dir that does not exist on other machines
export AMPLIHACK_TREE_ID=38a3fc70               # session-specific tree ID
export AMPLIHACK_WORKSTREAM_ISSUE=4215
export AMPLIHACK_WORKSTREAM_PROGRESS_FILE=/tmp/amplihack-workstreams/state/ws-4215.progress.json
export AMPLIHACK_WORKSTREAM_STATE_FILE=/tmp/amplihack-workstreams/state/ws-4215.json

Where it belongs: Deleted. It was a one-off execution script for a single development session.


launcher.py (added — repo root)

Why flagged: This is a workstream launcher script with baked-in issue-specific and path-specific values:

user_context = json.loads("{\"task_description\": \"...\", \"issue_number\": 4215, "
    "\"workstream_state_file\": \"/tmp/amplihack-workstreams/state/ws-4215.json\", "
    "\"workstream_progress_file\": \"/tmp/amplihack-workstreams/state/ws-4215.progress.json\"}")

The hardcoded /tmp paths and issue_number: 4215 make this script specific to one development session. It is not parameterised, not documented as a reusable tool, and will fail on any other machine or for any other issue.

Where it belongs: Deleted. Like run.sh, this was a generated execution artifact for a single workstream run.


Advisory (not a blocking violation)

.claude/context/PROJECT.md was changed to rename the project from amplihack-pm to ws-4215. The name ws-4215 follows the same workstream-ID pattern as the two files above and may have been set during the session rather than intentionally. If this rename is intentional, no action is needed — but it is worth a quick sanity check.


Files confirmed clean

File Verdict
docs/ANTHROPIC_DISABLEMENT.md ✅ Durable reference documentation
mkdocs.yml ✅ Config update
src/amplihack/**/*.py ✅ Permanent code changes
tests/**/*.py ✅ Durable test infrastructure

To override, add a PR comment containing repo-guardian:override (reason) where (reason) is a required non-empty justification for allowing run.sh and launcher.py in the repository.

Generated by Repo Guardian for issue #4216 ·

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Apr 4, 2026

PR Triage — CI Failure Detected

Atlas PR Impact Check is failing on this PR. This check asserts that atlas layer metadata is kept current when source files change.

Affected check: Atlas PR Impact Checkfailure

This PR modifies fleet/_backends.py, eval/gherkin_agent_evaluator.py, agents/goal_seeking/hive_mind/query_expansion.py, and vendor/blarify/agents/ files. If these changes affect exported symbols, class shapes, or module structure visible to the atlas layer, the atlas docs in docs/atlas/ need to be refreshed.

Recommended fix: Run bash scripts/check-atlas-staleness.sh HEAD~1 HEAD to identify stale layers, then regenerate affected atlas files.

All other checks pass (Validate Code, GitGuardian, docs checks, bot detection).

Triage summary:

  • Risk: medium
  • Tests: 41 passing
  • Priority: medium
  • Blocker: Atlas PR Impact Check CI failure must be resolved before merge

Generated by PR Triage Agent ·

This was referenced Apr 14, 2026
@rysweet
Copy link
Copy Markdown
Owner Author

rysweet commented Apr 22, 2026

Pre-Merge Cleanup Required

The PR contains ~30 __pycache__/*.pyc files that should not be committed. These may be accidental from a local build.

Before merge:

  • Add __pycache__/ to .gitignore if not already present
  • Remove all *.pyc files from this PR with: git rm -r --cached '**/__pycache__' then commit
  • Force-push the cleaned branch

The 9 substantive changes (docs/ANTHROPIC_DISABLEMENT.md, run.sh, query_expansion.py, gherkin_agent_evaluator.py, _backends.py, llm_provider.py, rotating_anthropic.py, conftest.py, test_query_expansion.py, test_gherkin_evaluator_guard.py, launcher.py) and PROJECT.md/mkdocs.yml updates look reasonable for the ANTHROPIC_DISABLED flag feature, but the pycache files block clean merge-ready review.

Co-authored-by: Copilot 223556219+Copilot@users.noreply.github.com

@rysweet
Copy link
Copy Markdown
Owner Author

rysweet commented Apr 23, 2026

Review summary

Verdict: ⚠️ Approve with conditions — code change is correct and satisfies #4215, but 3 hygiene issues block merge as-is.

Functional assessment ✅

All four ANTHROPIC_DISABLED guard sites work correctly:

  • fleet._backends.AnthropicBackend.__init__ raises ConfigurationError immediately when disabled; auto_detect_backend() falls back to CopilotBackend
  • eval.gherkin_agent_evaluator._has_anthropic_api_key() returns False, routing through _evaluate_with_cli()
  • agents.goal_seeking.hive_mind.query_expansion.HAS_ANTHROPIC set at import; expand_query() falls back to local synonyms ✓
  • vendor.blarify.RotatingKeyChatAnthropic.__init__ checks flag and raises RuntimeError before SDK use ✓

Test coverage is adequate. Documentation (docs/ANTHROPIC_DISABLEMENT.md) is excellent.

Blocking issues 🛑

  1. 22 .pyc files committed under src/amplihack/agents/goal_seeking/__pycache__/ (and sibling subdirs). All show additions:0, deletions:0 — they're binary diffs of bytecode that should never be tracked. .gitignore already excludes them; they were force-added. Fix:

    git rm --cached src/amplihack/agents/goal_seeking/**/*.pyc
    git rm --cached src/amplihack/agents/goal_seeking/**/__pycache__/*.pyc
    git commit --amend --no-edit
  2. Workstream cruft committed. launcher.py and run.sh reference /tmp/amplihack-workstreams/ws-4215-specific paths and serialized JSON contexts; they aren't referenced by any production code. .claude/context/PROJECT.md was modified to rename the project to "ws-4215" and remove Go from the languages list — looks like a local workspace mutation. Revert all three unless they're meant to be examples.

  3. Vendored blarify patches will be lost on the next vendor sync (commits 2ca1d6326, 3cdde01a show periodic syncs). Either:

    • Document the patches in src/amplihack/vendor/blarify/AMPLIHACK_PATCHES.md and add a CI guard that re-applies them, OR
    • Wrap blarify externally (check ANTHROPIC_DISABLED before any blarify import in caller code) and leave vendor pristine

Plus

  • mergeStateStatus: DIRTY — needs rebase on main after the cleanup above.
  • Minor (non-blocking): _SecretValue doesn't override __format__, so f-strings like f\"{backend._api_key}\" could leak. Add __format__ = __str__.
  • Minor (non-blocking): HAS_ANTHROPIC is resolved at import time. Setting the flag after import has no effect — the docs mention this; consider warning if mutation is detected.

Cross-ecosystem

Not directly applicable to amplihack-rs (no Anthropic Python SDK dependency). However, the pattern (explicit disablement flag → early ConfigurationError → fallback to Copilot) is worth porting to any Rust LLM-backend selection if/when added; secrecy::SecretString is the analog of _SecretValue.

Recommended path

clean .pyc + drop ws-4215 cruft → revert PROJECT.md → rebase → merge. Core change is good; just hygiene.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

implement-disablement

1 participant