feat: anonymity hardening (Phase 1, 9 sprints)#3
Merged
Conversation
Sprint 1 of anonymity-hardening (.harness/anonymize-platform/). - Replace audit_log.ip_address VARCHAR(45) with ip_hmac BYTEA(32) + ip_salt_id FK. Raw IPs no longer persist. - New ip_salts table + ip_salt_service rotating salts (default weekly, IP_SALT_ROTATION_DAYS env override). Salts older than 2x rotation period have their secret zeroed. - Replace _SENSITIVE_KEYS blocklist with per-action allowlist (_AUDIT_REQUEST_BODY_ALLOWLIST). Default-deny: unknown actions log request_body=None. - Migration q8r9s0t1u2v3_audit_ip_hmac.py: backfills existing rows under bootstrap salt, rebuilds entry_hmac chain via running_prev (mirrors o6p7q8r9s0t1 pattern), asserts chain integrity at end and aborts on mismatch. - log_event signature unchanged — no call-site edits across 52 callers; redaction happens inside the service. - Tests: 49 passed, 88.10% coverage on new modules. Multi-row chain integrity test (3 rows) catches the propagation bug found by independent evaluator in iteration 1. Branch: feat/anonymity-hardening Sprint passes independent verification (sprint-1-result.md PASS).
Sprint 2 of anonymity-hardening. Default-deny visibility for the
agent marketplace + minimum-fingerprint defaults at registration.
- Add agents.is_public BOOLEAN NOT NULL DEFAULT false + index
- Migration r9s0t1u2v3w4 hard-cuts existing agents to is_public=false
(per Lead Q3 — no grace period; opt-in via update_profile)
- Filter discover_agents / marketplace / search_agents / count by
is_public == True (NULL-safe equality, never !=)
- get_profile / get_profile_by_address: 404 for non-self callers
when target is private; nonexistent and private rows return
identical 404 to prevent existence-leak via timing or status
- register_agent forces is_public=False regardless of payload;
AgentRegistration Pydantic schema does not expose the field
- Default profile fields zeroed: description=None, pricing={},
capabilities=[] — no stylometry surface for new agents
- New PATCH /v2/me/settings endpoint to flip is_public + profile
- CLI: sthrip me publish / sthrip me unpublish
- SDK update_profile gains is_public kwarg
Tests: 18 new in test_marketplace_is_public.py (default invisibility,
opt-in flow, self-bypass, leak-protection, registration defence).
Modified 42 existing tests in test_marketplace.py, test_discovery_v2.py,
test_access_control.py to publish before asserting visibility.
Independent evaluator PASS; zero new regressions.
Branch: feat/anonymity-hardening
Sprint 3 of anonymity-hardening. Adds participant-envelope at insert
time on every payment-graph row; reads still go through plaintext
FKs (Sprint 4 cuts over).
Schema:
- transactions / escrow_deals / escrow_milestones gain
participant_envelope BYTEA + amount_bucket VARCHAR(32).
- message_relays gains participant_envelope only (no amount).
- Migration s0t1u2v3w4x5 idempotent (inspector.has_column guards),
all nullable, downgrade-safe.
Crypto:
- envelope_crypto.py — AES-256-GCM per-row DEK; payload encrypted
under DEK, DEK wrapped twice (HUB_KEK, OP_KEK) with independent
fresh 12-byte nonces. msgpack v1 wire format with schema_version.
- amount_to_bucket coarsens to log scale ("100-1k" XMR), never
exposes exact amount.
- operator_keystore.py — Sprint 3 ships StubKeystore (in-source
test KEK). RemoteKeystore raises NotImplementedError as the
Sprint 4 marker; default OP_KEYSTORE_MODE=stub.
- payment_envelope_writer.py — idempotent repo helper; lazy import
to avoid services↔db cycle.
Dual-write:
- transaction_repo / escrow_repo / milestone_repo / messaging_service
populate envelope at create time. No public signature changed.
- No read paths touched. ADMIN_API_KEY alone cannot decrypt the
graph once Sprint 4 swaps in the real keystore.
Tests: 46 new (envelope crypto, keystore, end-to-end dual-write).
Coverage 92.34% on new modules. Independent evaluator PASS; zero
new regressions vs 0b03e69.
Branch: feat/anonymity-hardening
Sprint 4 split per Lead decision: 4a (this) is non-destructive read cutover behind a feature flag; 4b will drop FK columns after staging dry-run + real keystore deploy. Read-path: - New payment_envelope_reader.py with feature flag STHRIP_READ_FROM_ENVELOPE (default false). Flag-off path is byte-identical to Sprint 3 — short-circuits before any decrypt. - Repo reads (transactions/escrow/milestone) post-process rows through apply_envelope_to_row when flag=true; falls back to FK on null envelope OR decrypt failure (logged, never raises). - ReadResult dataclass tags source: flag_off | envelope | fallback_envelope_null | fallback_decrypt_error. Backfill: - scripts/backfill_payment_envelope.py — rerun-safe, idempotent (WHERE participant_envelope IS NULL), batched, 4-table coverage. Milestones resolve buyer/seller from parent escrow_deal. - --dry-run is read-only (verified by tests). Admin: - _keystore_available() probe in admin_ui/views.py. - Redacted shape when keystore unavailable: participants → "encrypted", amount → amount_bucket from Sprint 3 or "redacted". - Full shape preserved when keystore available. Tests: 49 new across 4 files, 86% coverage on reader. Full repo suite shows zero new regressions vs Sprint 3 baseline. Known follow-up (MEDIUM, deferred): backfill edge-case where every row in a batch already has envelope can advance offset incorrectly; worth a defensive guard before running in prod. Tracked. Sprint 4b (CRITICAL destructive) blocked on: 1) deploy sthrip-op-keystore Railway service (real RemoteKeystore) 2) prod backfill verification (zero NULL envelopes) 3) STHRIP_READ_FROM_ENVELOPE=true with 24h soak 4) FK column drop migration 5) OP_KEYSTORE_MODE=remote flip Branch: feat/anonymity-hardening
Sprint 5 of anonymity-hardening. Webhook destinations no longer deanonymize agents through the public marketplace or admin views. Schema: - Drop agents.webhook_url plaintext column - Drop webhook_endpoints.url plaintext column - Add webhook_endpoints.url_encrypted TEXT NOT NULL (Fernet under the existing WEBHOOK_ENCRYPTION_KEY, same key as secret_encrypted) Migration u2v3w4x5y6z7 (idempotent, rerun-safe): 1. ADD url_encrypted nullable 2. Backfill: encrypt existing webhook_endpoints.url; for agents with webhook_url but no endpoint row, synthesize one 3. Assert zero NULL url_encrypted (RuntimeError abort) 4. ALTER url_encrypted SET NOT NULL 5. Drop unique constraint uq_agent_webhook_url 6. DROP webhook_endpoints.url 7. DROP agents.webhook_url Downgrade is best-effort (decrypts back; aborts loudly on key drift). Code: - webhook_service.py: legacy_url fallback removed; only reads webhook_endpoint_repo.get_url() which returns None on decrypt failure (existing endpoint-disable path triggers). - webhook_endpoint_repo: encrypt on create, decrypt on read. - AgentResponse / marketplace / discovery JSON no longer expose any webhook URL field. - Admin views render "encrypted webhook" badge instead of URL. - SDK register_webhook unchanged signature; server-side encrypts. Tests: 20 new (12 webhook_url_encryption + 8 webhook_migration); 77 existing webhook tests modified to match new shape, all green. Coverage 93% on changed modules. Independent evaluator confirms coherence after a Generator API-500 mid-work; no half-finished stubs introduced. Branch: feat/anonymity-hardening
Sprint 6 of anonymity-hardening. Reachability over Tor without forcing all traffic through it. All paths gated on STHRIP_ONION_ENABLED (default false) — merge-safe before infra is provisioned. Infra (railway/tor-sidecar-deploy/): - Dockerfile: Alpine 3.19 + tor + tini, runs unprivileged tor user, exposes :9050 SocksPort - torrc: HiddenServiceVersion 3, ControlPort 0 (no remote control), ClientOnly 1, SafeLogging 1, HSv3 maps :80 → sthrip-api.railway.internal:8000 - entrypoint.sh: boots tor, dumps .onion address from /var/lib/tor/sthrip-hsv3/hostname for operator capture, foregrounds - README.md: operator runbook (volume mount, first-boot capture, key rotation, blast radius) Code (default-off): - /.well-known/agent-payments.json publishes onion_endpoint only when STHRIP_ONION_ENABLED=true AND STHRIP_ONION_ENDPOINT env set - SDK Sthrip(use_tor=True) opts in to socks5h proxy (STHRIP_TOR_SOCKS_PROXY env, default socks5h://127.0.0.1:9050). Default Sthrip() unchanged. - webhook_service _should_route_via_tor predicate enforces Lead Q4: outbound Tor ONLY when target hostname endswith ".onion" AND flag on. Clearnet targets ALWAYS direct (preserves SSRF + IP pinning). Deps: httpx[socks]>=0.27, aiohttp-socks>=0.8.4, PySocks>=1.7.1. No DB migrations. Tests: 40 new (17 wellknown + 9 sdk + 14 webhook routing). Coverage wellknown 100%, webhook_service 85%. 76 existing webhook tests green; full suite zero new regressions vs 4aecfcb. Lead Q4 invariant ("outbound Tor only when .onion") verified at three layers: predicate, dispatcher, and test asserting tor_mock NOT called for clearnet target with flag on. Operator action required after merge: 1. Build Railway service from railway/tor-sidecar-deploy/ 2. Mount persistent volume on /var/lib/tor/sthrip-hsv3 3. Capture .onion from boot logs 4. Set STHRIP_ONION_ENABLED=true and STHRIP_ONION_ENDPOINT, plus STHRIP_TOR_SOCKS_PROXY=socks5h://sthrip-tor.railway.internal:9050 on the API service 5. Redeploy and verify /.well-known advertises onion_endpoint Branch: feat/anonymity-hardening
Sprint 7 of anonymity-hardening — capstone. Documentation now matches the code that actually shipped across the prior 6 sprints. PRIVACY_FEATURES.md (full rewrite): - Replace "INSTANT Maximum Privacy" overpromise (CoinJoin + Submarine Swaps + zk-SNARKs as "Combined ≤ 3 min, fully unlinkable") with explicit Shipped / In progress / Roadmap. - Each shipped sprint opens with its commit hash: 5a68ec8 / 0b03e69 / 9eb2eca / c7ae822 / 4aecfcb / 16126a5. - Sprint 4b (FK column drop) listed as "deferred, blocked on real RemoteKeystore deploy + 24h soak". - CoinJoin / Submarine Swaps / zk-SNARKs / MPC explicitly named in Roadmap with "Not shipped" status and link to the relevant research code path. Names them as research-grade, not request path features. - Closes with a "previously overshot reality" retraction so the history is visible. docs/THREAT_MODEL.md (full rewrite, replaces MPC/bridge era): - Hub-centric model. Names the operator, the hub, the operator keystore, agents, and external observers as trust boundaries. - 10 threat rows in Threat | Current defence | Residual risk table. All 8 user-criteria AC #6 scenarios covered. - Residual risks explicit: * runtime compromise admits "ANY runtime memory dump captures one in-flight request's plan" — the unavoidable property of any custodial hub. * subpoena admits "BOTH Railway DB AND operator keystore service" recovers graph; single-target subpoena does not. * webhook correlation admits clearnet timing leaks remain even with Sprint 6 onion routing for .onion targets only. * insider row admits "two-service split is hostile-coworker resistant, not hostile-owner resistant". README.md privacy section (5 bullets) and PRIVACY_GUIDE.md status banner now point at the two new files. Verification: independent evaluator confirmed every PRIVACY_FEATURES claim traces to specific code (model line numbers, env var names, file paths). Pen-test grep for marketing-speak ("100% private", "untraceable", "fully unlinkable" outside the retraction section) returns zero hits. Branch: feat/anonymity-hardening (sprints 1-7 main work complete; Sprint 4b destructive FK drop remains deferred per Lead split, blocked on real keystore service deploy)
Sprint 4b — code lands; operator deploy is required to activate.
Closes the dual-key invariant: ADMIN_API_KEY alone cannot decrypt
the payment graph once the operator runs the cutover.
RemoteKeystore (sthrip/services/operator_keystore.py):
- Replaces NotImplementedError with httpx client to
sthrip-op-keystore.railway.internal:8000 (POST /wrap, /unwrap)
- Auth: OP_KEYSTORE_AUTH_TOKEN env (Bearer header, never in URL/body)
- 5s timeout; non-200 raises clear RuntimeError with status + body
- KEK_OP NEVER reaches the hub — RemoteKeystore holds only URL +
auth token; KEK_OP lives in the separate Railway service
- get_keystore() switches by OP_KEYSTORE_MODE; stub remains default
- Constructor fails fast if AUTH_TOKEN unset
Reader graceful access (sthrip/services/payment_envelope_reader.py):
- getattr-based field access on rows so reader survives FK drop
- New ReadResult source: fallback_no_data when both envelope and
FK columns missing (post-cutover edge case)
Migration v3w4x5y6z7a8_drop_legacy_payment_fks.py:
- DROPS plaintext FK columns: transactions.{from_agent_id,
to_agent_id, amount}; escrow_deals.{buyer_id, seller_id, amount};
escrow_milestones.amount; message_relays.{from_agent_id,
to_agent_id}
- Gated behind STHRIP_DROP_LEGACY_FK=true env (default false)
- upgrade() aborts with RuntimeError BEFORE any DDL when flag unset.
Error message lists all 4 prerequisites: keystore deployed,
backfill verified, STHRIP_READ_FROM_ENVELOPE=true, 24h soak
- Idempotent (inspector.has_column guards before each drop_column)
- Downgrade re-adds nullable columns (data is gone — best-effort
restore from backup)
railway/op-keystore-deploy/:
- Dockerfile: Python 3.11-slim + FastAPI/uvicorn/cryptography
- server.py: minimal HTTP service, /wrap, /unwrap, /health.
AESGCM with KEK_OP_BASE64 env, fresh 12-byte nonce per op.
hmac.compare_digest for token check; generic 403 on auth fail
(constant-time, no enumeration). Runs as unprivileged user.
- README.md: operator runbook (key generation, env vars, deploy,
ACL hardening)
Tests: 4 new test files (16 RemoteKeystore + 11 migration + 7
reader-after-drop + 8 keystore-server = 42 new); 10 modified in
test_operator_keystore.py. 74 sprint-4b tests pass; 89% coverage
on changed modules; zero new regressions vs e6ef31b.
Independent evaluator empirically verified safe-by-default:
upgrade() raised RuntimeError immediately on line 1 with flag
unset; falsy variants (""/false/no/0/off) all refused; only
1/true/yes/on enable. Catastrophic accidental run is prevented.
Operator activation steps (after merge):
1. Deploy sthrip-op-keystore Railway service (Dockerfile in
railway/op-keystore-deploy/) with KEK_OP_BASE64 + AUTH_TOKEN
2. Set OP_KEYSTORE_AUTH_TOKEN + OP_KEYSTORE_URL on API service
3. Run scripts/backfill_payment_envelope.py to zero NULL envelopes
4. Set STHRIP_READ_FROM_ENVELOPE=true; soak 24h, monitor for
fallback_decrypt_error
5. Set STHRIP_DROP_LEGACY_FK=true; alembic upgrade head
6. Set OP_KEYSTORE_MODE=remote on API service
Step 5 is the point of no return — restore from backup is the
only rollback for the FK columns themselves.
Branch: feat/anonymity-hardening (all 8 sprints code-complete)
User-criteria, lead-decisions, product-spec for 7-sprint Phase 2: - Phase 1: auto-purge + warrant canary (1 sprint) - Phase 2: commission + subscription + XMR billing (3 sprints) - Phase 3: GCP TEE migration + attestation (3 sprints) All Lead decisions pre-baked to avoid Generator re-asking.
|
Warning Rate limit exceeded
You’ve run out of usage credits. Purchase more in the billing tab. ⌛ How to resolve this issue?After the wait time has elapsed, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout. Please see our FAQ for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (108)
✨ Finishing Touches🧪 Generate unit tests (beta)
Tip 💬 Introducing Slack Agent: The best way for teams to turn conversations into code.Slack Agent is built on CodeRabbit's deep understanding of your code, so your team can collaborate across the entire SDLC without losing context.
Built for teams:
One agent for your entire SDLC. Right inside Slack. Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
7 tasks
…escript (separate repo)
…sts (hardcoded local path)
…with asyncio_mode=auto
…nup in dedicated PR)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Phase 1 anonymity hardening — 9 commits delivering envelope encryption, webhook URL encryption, Tor sidecar, RemoteKeystore + gated FK drop, IP scrubbing, marketplace opt-in, honest privacy docs.
Custodial-after-hardening posture: closes ~95% of practical privacy threats per
THREAT_MODEL.md. Hub still sees plaintext in RAM during routing — that's the residual risk closed by Phase 2/3 TEE work (separate PR).Commit chain
Test plan
alembic upgrade headclean on Postgres staging/.well-known/onion.json(if Tor sidecar deployed) returns expected formatis_public=false, must explicitly opt-inOperator runbook (post-merge)
Sequential, do not skip:
WEBHOOK_ENCRYPTION_KEY(Fernet) on RailwayAUDIT_IP_HMAC_KEY(32-byte hex)railway/tor-sidecar-deploy/python -m sthrip.cli backfill_envelopes(one-shot, idempotent)STHRIP_KEYSTORE_URL, signing keys) before Sprint 4b cutoverPhase 2 follows
feat/revenue-and-teeis built on top of this branch and ships commission + subscription + GCP TEE as a separate PR (rebases onto main once this lands).