Skip to content

feat(sdk): implement cloud-aware threshold sync with mode-based resolution and provenance telemetry#12

Open
miraniy98 wants to merge 1 commit intomainfrom
threshold-sync-cloud
Open

feat(sdk): implement cloud-aware threshold sync with mode-based resolution and provenance telemetry#12
miraniy98 wants to merge 1 commit intomainfrom
threshold-sync-cloud

Conversation

@miraniy98
Copy link
Copy Markdown
Contributor

Summary

Implements SDK-side threshold sync control-plane integration:

  • cloud-aware threshold resolution modes
  • provenance-rich decision model
  • remote resolve/report client with cache + debounce
  • telemetry provenance fields for observability
  • explicit opt-in config (OSS-safe defaults)

What changed

Config

  • Added ThresholdSyncConfig to ShieldConfig:
    • enabled (default false, explicit opt-in)
    • resolve_endpoint, report_endpoint
    • timeout_ms, ttl_seconds
    • report_enabled, report_debounce_seconds

Files:

  • src/pyagentshield/core/config.py

Exceptions

  • Added ThresholdUnavailableError for strict cloud_only fail-closed path.

Files:

  • src/pyagentshield/core/exceptions.py

Threshold decision model

  • Added ThresholdDecision dataclass with provenance fields.
  • Canonical source/mode token sets included for consistency.

Files:

  • src/pyagentshield/threshold/decision.py

Threshold manager

  • Added resolve_with_mode(...) supporting:
    • local_only
    • local_prefer
    • cloud_prefer
    • cloud_only
    • observe
  • Added _resolve_local_with_source(...) returning (value, source, is_auto_calibrated).
  • Added thread-local per-scan threshold override support + clear helper.

Files:

  • src/pyagentshield/threshold/manager.py

Remote cloud client

  • Added CloudThresholdClient with:
    • resolve cache keyed by (fingerprint, environment_key)
    • debounced report path keyed by (fingerprint, environment_key)
    • 4dp quantization to reduce float jitter spam
    • bounded queue + single background drain thread (no unbounded thread spawning)
  • Added CloudResolution and CloudRule.

Files:

  • src/pyagentshield/remote/client.py
  • src/pyagentshield/remote/__init__.py

Shield integration

  • Creates cloud client only when threshold_sync.enabled=true and API key present.
  • Uses resolve_with_mode(...) in scan flow.
  • Emits threshold provenance into telemetry event.
  • Reports observation payload to cloud (debounced, best-effort).

Files:

  • src/pyagentshield/core/shield.py
  • src/pyagentshield/telemetry/events.py

Compatibility

  • Default behavior unchanged for OSS users:
    • no API key OR threshold_sync.enabled=false => no cloud sync behavior
  • Existing threshold APIs remain available; this is additive.

Validation

  • Full SDK test suite run:
    • ./.venv/bin/python -m pytest -q
    • result: all tests passing in local run (254 passed)

Notes

  • This PR is SDK-side only; platform migration/API/UI rollout lands in agentshield-platform.
  • Keep enum tokens synchronized with platform report handler and DB checks:
    • sources: local_pinned, local_cache, finetuned, registry, auto_calibrated, cloud_manual, local_failopen
    • modes: local_only, local_prefer, cloud_prefer, cloud_only, observe

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: df03f31a00

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +325 to +329
fingerprint = self.threshold_manager._build_fingerprint(
model_name=self.embedding_provider.model_name,
embedding_provider=self.embedding_provider,
text_cleaner=self.text_cleaner,
) or self.embedding_provider.model_name
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Recompute fingerprint after applying cloud cleaning settings

When project_settings.cleaning_method changes (for example, heuristic to llm), the fingerprint is computed before _apply_cloud_project_settings(...) mutates self.config.cleaning.method and rebuilds cleaner/detector state. The same scan then resolves/report thresholds using the old fingerprint while detection runs with the new cleaner, so threshold override/provenance can be associated with the wrong pipeline and produce incorrect first-scan decisions after a cloud cleaning-method update.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant