Skip to content

scope.md

Latest commit

 

History

History
118 lines (86 loc) · 5.33 KB

scope.md

File metadata and controls

118 lines (86 loc) · 5.33 KB

Components You Will Build

You will build a distributed secrets vault system. It includes running services and a small set of required definitions that make system behavior clear and usable by other systems.

A. Running Components (Processes)

1. Secrets API Service

A long-running HTTP service that accepts requests for creating, updating, retrieving, and deleting secrets.

It will:

  • Accept structured JSON input
  • Accept secret create requests that establish a new secret
  • Reject create requests for secrets that already exist
  • Accept secret update requests that create a new version of an existing secret
  • Reject update requests for secrets that do not exist
  • Accept secret retrieval requests for previously stored secrets
  • Accept requests to retrieve the version history of a secret
  • Repair latest-version shard placement during retrieval when the system can still reconstruct the value but available shards are close to the recovery threshold
  • Accept secret delete requests that remove enough stored shards to make reconstruction impossible
  • Reject delete requests for secrets that do not exist
  • Accept .env file content and:
    • expand secret(NAME) references by retrieving authoritative secret values
    • process enc(NAME) references by creating new secrets and returning secret(NAME)
  • Fail the entire .env request if any referenced secret already exists or cannot be resolved
  • Expose basic health and status endpoints

The API validates and forwards requests but does not determine secret existence or authoritative state.

2. Secrets Vault Service

A long-running service responsible for authoritative storage of secret state.

It will:

  • Validate and persist proposed secret creations
  • Validate and persist secret updates as new versions
  • Use the same two-phase commit locking flow for create and update operations
  • Distinguish write failures by phase: voting phase vs writing phase
  • Validate and execute secret deletes using a threshold-based deletion rule
  • Maintain a clear distinction between secret existence and secret updates
  • Persist multiple versions of its parts of a secret
  • Record validity intervals for each secret version
  • Shard secret pieces to peer vault instances
  • Serve retrieval, history, and delete requests based on authoritative state
  • Remain available under partial failure (within range of accepted Shamir's recovery threshold)
  • Perform best-effort read repair for latest-version reads when only k or k + repairTriggerBuffer shards are available
  • Recover state on restart

B. Required Artifacts (Not Processes)

These are not separate running services. They are required artifacts that define what your running system means and how other systems interact with it.

3. Secret, History, and Isolation Model

A shared behavioral model implemented consistently across all components.

It defines:

  • What it means for a secret to exist and be retrievable
  • The distinction between secret creation and secret update
  • The shared lock/commit flow used by both creation and update
  • How secret updates are versioned
  • How gateway-attached time metadata is carried into write commits
  • How historical secret values are retained
  • How secret deletion is defined and when a secret is considered non-reconstructable
  • What identifiers are used to reference secrets
  • How retries and concurrent requests are handled
  • How read repair behaves under concurrent updates and deletes
  • What duplicate and not found errors mean

The model must be documented and observable in practice.

4. Encryption and Storage Record Definition

A required definition of how secrets are stored durably.

It defines:

  • The stored record fields for each secret version, including metadata and the shard a node is privy to
  • Shamir's Secret Keeping behavior:
    • A password is separated on a single node into n (configured by user) parts
    • The data is sent out to n - 1 other nodes, with 1 piece staying local to the machine that received the request directly
  • Read repair behavior:
    • Latest-version GET requests that barely meet the reconstruction threshold may re-split the reconstructed value and restore shards for the same version
    • Repair does not create a new version and does not apply to historical version reads
  • no master key required, any node can take requests to decode
  • The rule that plaintext secret bytes are never written to durable storage or passed to other nodes

5. External API Contract

A small, explicit interface describing how clients interact with the system.

It will:

  • Define request and response formats
  • Clearly distinguish secret creation from secret update
  • Define delete request and response behavior, including threshold-based deletion success criteria
  • Specify duplicate and not found error behavior
  • Describe durability and replication guarantees
  • Describe best-effort read repair when shard availability is degraded but still reconstructable
  • Describe secret-keeping and spreading behavior and failure behavior when referenced secrets cannot be resolved
  • Describe secret history retrieval semantics, including version ordering and validity timestamps
  • Describe .env encryption and expansion semantics, including secret creation and all-or-nothing failure
  • Hide internal sharding and coordination mechanics

The contract exists to decouple system behavior from implementation details.