From 8c669ae4bca8723a9b41aa97c235c10ef29fd658 Mon Sep 17 00:00:00 2001 From: FlowmemoryAI <283694809+FlowmemoryAI@users.noreply.github.com> Date: Wed, 13 May 2026 00:49:29 -0500 Subject: [PATCH] build crypto v0 foundation --- contracts/shared/README.md | 20 ++ contracts/shared/RECEIPT_VERIFIER_BOUNDARY.md | 78 +++++ crypto/.gitignore | 3 + crypto/ATTESTATIONS.md | 198 +++++++++++++ crypto/FLOWMEMORY_CRYPTO_SPEC.md | 178 +++++++++++ crypto/MERKLE_AND_ROOTS.md | 134 +++++++++ crypto/OBSERVATION_IDENTITY.md | 152 ++++++++++ crypto/README.md | 85 ++++++ crypto/RECEIPT_HASHING.md | 143 +++++++++ crypto/TEST_VECTORS.md | 100 +++++++ crypto/fixtures/sample-flowpulse.json | 31 ++ crypto/fixtures/sample-observation.json | 47 +++ crypto/fixtures/sample-report.json | 69 +++++ crypto/fixtures/vectors.json | 276 ++++++++++++++++++ crypto/package-lock.json | 40 +++ crypto/package.json | 27 ++ crypto/src/attestations.js | 117 ++++++++ crypto/src/cli.js | 54 ++++ crypto/src/constants.js | 75 +++++ crypto/src/domains.js | 69 +++++ crypto/src/encoding.js | 127 ++++++++ crypto/src/flowpulse.js | 146 +++++++++ crypto/src/hashes.js | 36 +++ crypto/src/index.d.ts | 271 +++++++++++++++++ crypto/src/index.js | 7 + crypto/src/merkle.js | 148 ++++++++++ crypto/src/validate-vectors.js | 73 +++++ crypto/test-vectors/README.md | 11 + .../flowpulse-observation-v0.json | 107 +++++++ crypto/test/crypto.test.js | 266 +++++++++++++++++ crypto/validate_test_vectors.py | 249 ++++++++++++++++ docs/ARCHITECTURE.md | 6 + ...6-05-13-flowmemory-crypto-v0-foundation.md | 42 +++ docs/SECURITY_MODEL.md | 40 +++ research/cryptography/FUTURE_ZK_ROADMAP.md | 92 ++++++ research/cryptography/IMPLEMENTATION_PLAN.md | 168 +++++++++++ research/cryptography/THREAT_MODEL.md | 195 +++++++++++++ services/verifier/README.md | 82 ++++++ 38 files changed, 3962 insertions(+) create mode 100644 contracts/shared/README.md create mode 100644 contracts/shared/RECEIPT_VERIFIER_BOUNDARY.md create mode 100644 crypto/.gitignore create mode 100644 crypto/ATTESTATIONS.md create mode 100644 crypto/FLOWMEMORY_CRYPTO_SPEC.md create mode 100644 crypto/MERKLE_AND_ROOTS.md create mode 100644 crypto/OBSERVATION_IDENTITY.md create mode 100644 crypto/README.md create mode 100644 crypto/RECEIPT_HASHING.md create mode 100644 crypto/TEST_VECTORS.md create mode 100644 crypto/fixtures/sample-flowpulse.json create mode 100644 crypto/fixtures/sample-observation.json create mode 100644 crypto/fixtures/sample-report.json create mode 100644 crypto/fixtures/vectors.json create mode 100644 crypto/package-lock.json create mode 100644 crypto/package.json create mode 100644 crypto/src/attestations.js create mode 100644 crypto/src/cli.js create mode 100644 crypto/src/constants.js create mode 100644 crypto/src/domains.js create mode 100644 crypto/src/encoding.js create mode 100644 crypto/src/flowpulse.js create mode 100644 crypto/src/hashes.js create mode 100644 crypto/src/index.d.ts create mode 100644 crypto/src/index.js create mode 100644 crypto/src/merkle.js create mode 100644 crypto/src/validate-vectors.js create mode 100644 crypto/test-vectors/README.md create mode 100644 crypto/test-vectors/flowpulse-observation-v0.json create mode 100644 crypto/test/crypto.test.js create mode 100644 crypto/validate_test_vectors.py create mode 100644 docs/DECISIONS/2026-05-13-flowmemory-crypto-v0-foundation.md create mode 100644 research/cryptography/FUTURE_ZK_ROADMAP.md create mode 100644 research/cryptography/IMPLEMENTATION_PLAN.md create mode 100644 research/cryptography/THREAT_MODEL.md create mode 100644 services/verifier/README.md diff --git a/contracts/shared/README.md b/contracts/shared/README.md new file mode 100644 index 00000000..a7a701f2 --- /dev/null +++ b/contracts/shared/README.md @@ -0,0 +1,20 @@ +# Shared Contract Crypto Formats + +Status: placeholder for future implementation. + +This folder is reserved for shared Solidity hash helpers once the v0 crypto schemas are reviewed. + +Do not add production contract logic here until: + +- `crypto/FLOWMEMORY_CRYPTO_SPEC.md` is accepted or revised +- `crypto/TEST_VECTORS.md` has automated checks +- `docs/DECISIONS/2026-05-13-flowmemory-crypto-v0-foundation.md` is accepted or superseded + +Expected future contents: + +- type hash constants +- pure hash functions for observation ids, receipts, artifact roots, storage commitments, reports, and signatures +- Merkle proof verification for accepted root schemes +- Solidity tests against the JSON vectors + +See `RECEIPT_VERIFIER_BOUNDARY.md` for the draft boundary of a future `ReceiptVerifier` adapter. diff --git a/contracts/shared/RECEIPT_VERIFIER_BOUNDARY.md b/contracts/shared/RECEIPT_VERIFIER_BOUNDARY.md new file mode 100644 index 00000000..6bcca9b1 --- /dev/null +++ b/contracts/shared/RECEIPT_VERIFIER_BOUNDARY.md @@ -0,0 +1,78 @@ +# Future ReceiptVerifier Boundary + +Status: draft boundary for issue #28. + +This document defines what a future `ReceiptVerifier` contract may verify after FlowMemory v0 schemas are accepted. It is not a contract implementation. + +## Why Wait + +`ReceiptVerifier` should wait until these schemas are accepted or superseded: + +- FlowPulse observation identity +- receipt hashing +- verifier report id +- worker and verifier signature envelopes +- artifact root scheme +- storage receipt commitment format + +Premature implementation risks baking in the wrong observation identity or replay domain. + +## What A v0 Contract Could Verify + +A future contract can safely verify compact, already-derived inputs: + +- `observationId` hash recomputation from supplied chain/log metadata +- `eventArgsHash` recomputation from supplied FlowPulse args +- `receiptHash` recomputation from supplied observation and commitment fields +- `artifactRoot` envelope hash recomputation +- Merkle proof verification for accepted root schemes +- `reportId` recomputation from supplied report fields +- EIP-712 digest construction for worker or verifier signatures +- secp256k1 signer recovery, if key registry policy exists + +## What A v0 Contract Must Not Claim + +A future contract must not claim to verify: + +- that `txHash`, `transactionIndex`, or `logIndex` were known during FlowPulse emission +- that a supplied receipt/log is canonical without an accepted chain proof or trusted oracle path +- that off-chain artifact bytes are available forever +- that a URI is short, private, resolvable, or honest +- that a verifier attestation is a trustless proof +- that model output or worker behavior is correct +- that a storage provider will satisfy future retrieval requests + +## Required Inputs + +Any future adapter should make its trust boundary explicit by accepting already-derived fields: + +- `chainId` +- `emittingContract` +- `blockNumber` +- `blockHash` +- `txHash` +- `transactionIndex` +- `logIndex` +- `eventSignature` +- `pulseId` +- `rootfieldId` +- `eventArgsHash` +- `receiptHash` +- `reportId` + +The caller, indexer, oracle, or proof system remains responsible for establishing that those fields are canonical. + +## Implementation Prerequisites + +Before adding Solidity beyond pure helpers: + +- accept or revise `crypto/FLOWMEMORY_CRYPTO_SPEC.md` +- automate `crypto/test-vectors/flowpulse-observation-v0.json` validation +- define key registry and verifier set root governance +- decide whether on-chain verification is advisory, challenge evidence, or state-changing +- document gas limits for Merkle verification and signature recovery +- add focused tests for every accepted hash helper + +## MVP Recommendation + +Keep `ReceiptVerifier` out of production contracts for now. Add pure helper libraries in `contracts/shared/` only after schemas stabilize, then add tests against the JSON vectors. Treat any on-chain adapter as an evidence checker, not as proof of full trustlessness. diff --git a/crypto/.gitignore b/crypto/.gitignore new file mode 100644 index 00000000..2f24c57c --- /dev/null +++ b/crypto/.gitignore @@ -0,0 +1,3 @@ +node_modules/ +coverage/ +.nyc_output/ diff --git a/crypto/ATTESTATIONS.md b/crypto/ATTESTATIONS.md new file mode 100644 index 00000000..753f9bd0 --- /dev/null +++ b/crypto/ATTESTATIONS.md @@ -0,0 +1,198 @@ +# FlowMemory Attestations And Reports + +Status: draft v0. + +Attestations are signed envelopes over receipts, artifacts, or reports. They provide accountability. They do not make the system fully trustless. + +## Worker Signature Envelope + +Type string: + +```solidity +FlowMemoryWorkerSignatureV0(bytes32 receiptHash,bytes32 workerId,bytes32 workerKeyId,uint64 workerSequence,uint64 expiresAtUnixMs,bytes32 artifactRoot,bytes32 nonce) +``` + +Struct hash: + +```text +workerStructHash = keccak256(abi.encode( + WORKER_SIGNATURE_TYPEHASH, + receiptHash, + workerId, + workerKeyId, + workerSequence, + expiresAtUnixMs, + artifactRoot, + nonce +)) +``` + +Worker signatures bind an off-chain worker to a receipt and optional artifact root. They do not prove the worker's output is correct. + +## Verifier Report V0 + +Verifier reports are deterministic from their inputs. They should not include nondeterministic prose in the `reportId`. + +Type string: + +```solidity +FlowMemoryVerifierReportV0(bytes32 reportSchemaHash,bytes32 observationId,bytes32 receiptHash,bytes32 verifierId,bytes32 verifierSetRoot,uint8 status,bytes32 checksRoot,uint64 finalizedBlockNumber,bytes32 finalizedBlockHash,uint16 reportVersion) +``` + +Hash: + +```text +reportId = keccak256(abi.encode( + VERIFIER_REPORT_TYPEHASH, + reportSchemaHash, + observationId, + receiptHash, + verifierId, + verifierSetRoot, + status, + checksRoot, + finalizedBlockNumber, + finalizedBlockHash, + reportVersion +)) +``` + +`checksRoot` commits to a deterministic checks document or Merkle tree of check results. + +## Verifier Signature Envelope + +Type string: + +```solidity +FlowMemoryVerifierSignatureV0(bytes32 reportId,bytes32 verifierId,bytes32 verifierKeyId,bytes32 verifierSetRoot,uint64 issuedAtUnixMs,uint64 expiresAtUnixMs,bytes32 nonce) +``` + +Struct hash: + +```text +verifierStructHash = keccak256(abi.encode( + VERIFIER_SIGNATURE_TYPEHASH, + reportId, + verifierId, + verifierKeyId, + verifierSetRoot, + issuedAtUnixMs, + expiresAtUnixMs, + nonce +)) +``` + +The signature proves a verifier key signed the report id. It does not prove that the report is correct unless users trust the verifier or can independently replay the checks. + +## Generic Attestation Envelope + +Type string: + +```solidity +FlowMemoryAttestationEnvelopeV0(bytes32 subjectHash,uint8 subjectKind,bytes32 attesterId,bytes32 attesterKeyId,bytes32 verifierSetRoot,uint64 issuedAtUnixMs,uint64 expiresAtUnixMs,bytes32 nonce) +``` + +Hash: + +```text +attestationEnvelopeHash = keccak256(abi.encode( + ATTESTATION_ENVELOPE_TYPEHASH, + subjectHash, + subjectKind, + attesterId, + attesterKeyId, + verifierSetRoot, + issuedAtUnixMs, + expiresAtUnixMs, + nonce +)) +``` + +Suggested subject kinds: + +```text +1 = receipt +2 = verifier report +3 = artifact root +4 = storage receipt commitment +5 = challenge evidence +``` + +The package implements this as `attestationEnvelopeHash` and aliases it as `attestationDigest`. + +## Local Signature Helpers + +The package includes `publicKeyFromPrivateKey`, `signDigest`, and `verifyDigest` for deterministic secp256k1 tests using local generated test keys only. These helpers are not key management, do not load real secrets, and do not assert production verifier identity. Production services must provide their own secure signer, key registry checks, expiry policy, and verifier set validation. + +## Signature Mode Comparison + +### EIP-712 Typed Signatures + +Pros: + +- Best fit for EVM wallets, contracts, and typed domain separation. +- Binds chain id, verifying contract or registry, and deployment salt cleanly. +- Human and tooling support is better than raw hashes. + +Cons: + +- More implementation complexity. +- Needs careful domain management before registries exist. +- Less natural for non-EVM device keys. + +### Simple Domain-Separated Keccak Hashes + +Pros: + +- Simple for services, hardware, and non-EVM agents. +- Easy to reproduce in verifier tests. +- Can be wrapped by many signature systems. + +Cons: + +- Weaker wallet UX. +- More risk of accidental replay if domain fields are omitted. +- Contracts need explicit recovery logic per key type. + +MVP choice: + +- Use EIP-712 for EVM/secp256k1 worker and verifier signatures. +- Use simple typed Keccak hashes for object ids, roots, report ids, and future non-EVM key adapters. +- Do not accept ambiguous `personal_sign` messages for protocol-critical attestations. + +## Status Vocabulary + +```text +0 = reserved +1 = observed +2 = verified +3 = unresolved +4 = unsupported +5 = failed +6 = reorged +7 = superseded +``` + +Minimum evidence: + +- `observed`: receipt/log was read, but finality or evidence checks are incomplete. +- `verified`: observation id, event args, commitments, artifacts, and signatures required by policy passed. +- `unresolved`: data is missing or unavailable, but failure is not proven. +- `unsupported`: schema, pulse type, root scheme, or key type is unknown. +- `failed`: a required check failed. +- `reorged`: the block/log is no longer canonical under the finality policy. +- `superseded`: a newer report replaces this report. + +## Replay Protection + +Verifier and worker signatures must bind: + +- chain id +- deployment id or registry +- key id +- verifier set root where relevant +- sequence or nonce +- expiry +- receipt hash or report id + +Off-chain services should reject duplicate `(workerId, workerSequence)` and duplicate verifier nonces unless the policy explicitly treats exact duplicate signatures as idempotent. diff --git a/crypto/FLOWMEMORY_CRYPTO_SPEC.md b/crypto/FLOWMEMORY_CRYPTO_SPEC.md new file mode 100644 index 00000000..4ba48d15 --- /dev/null +++ b/crypto/FLOWMEMORY_CRYPTO_SPEC.md @@ -0,0 +1,178 @@ +# FlowMemory Crypto Spec v0 + +Status: draft v0. + +## Core Question + +A FlowMemory fact is a typed claim that can be identified, committed, checked, and reported without placing raw data on-chain. + +Facts can be: + +- observed chain facts, such as a FlowPulse log after its transaction receipt exists +- committed off-chain facts, such as an artifact root or storage receipt commitment +- signed claims, such as worker signatures and verifier attestations +- deterministic verification reports +- future proofs over receipts, roots, or reports + +An unverified fact is only a claim. Verification status must remain explicit. + +## Source Inputs + +FlowMemory v0 starts from the contracts foundation: + +- `FlowPulse` emits `pulseId`, `rootfieldId`, `actor`, `pulseType`, `subject`, `commitment`, `parentPulseId`, `sequence`, `occurredAt`, and `uri`. +- `pulseId` is contract-computed and domain-separated by `flowmemory.flowpulse.v0`, chain id, emitting contract, rootfield id, actor, pulse type, subject, commitment, parent pulse, and sequence. +- `txHash`, `transactionIndex`, `logIndex`, and `blockHash` are not known to the contract. Indexers derive them after reading receipts and logs. +- `uri` is advisory log data. It is not proof of storage and is not an enforced off-chain-data boundary. + +## Identifier Model + +FlowMemory uses three different identifiers: + +- `pulseId`: logical contract pulse id, available inside emitted FlowPulse args. +- `observationId`: canonical identifier for one observed log position after receipt metadata exists. +- `reportId`: deterministic identifier for a verifier report over a receipt and check set. + +Do not use `pulseId` as the sole database primary key for canonical chain observation. A reorg or replayed transaction can produce the same semantic pulse in a different block context. Use `observationId` for observed logs and link it back to `pulseId`. + +## Hash Primitive + +Protocol commitments use Keccak-256 because Base/EVM contracts can verify it natively. + +Top-level objects use: + +```text +keccak256(abi.encode(TYPEHASH, field_1, field_2, ...)) +``` + +Rules: + +- `TYPEHASH = keccak256(bytes(type_string))`. +- Field order is normative. +- Strings, JSON, byte arrays, URIs, reports, and manifests are hashed before entering typed objects. +- Use `abi.encode`, not `abi.encodePacked`, for typed object hashes. +- Merkle leaf and node hashes use the typed object hashes defined in `MERKLE_AND_ROOTS.md`. +- Unknown or not-applicable `bytes32` fields use zero bytes. + +## Canonical Encoding Strategy + +On-chain compatible typed objects: + +- Use Solidity ABI static encoding. +- Use `address` as the 20-byte EVM address left-padded in ABI words. +- Use `uint64` for block numbers, sequences, and Unix seconds or milliseconds where specified. +- Use `uint16` for schema versions when included in typed hashes. + +Off-chain documents: + +- JSON inputs use canonical JSON with recursively sorted object keys. +- Array order is preserved and is meaningful. +- Strings matching `0x[0-9a-fA-F]*` are normalized to lowercase before hashing. +- Avoid ambiguous JSON number semantics in v0. Prefer strings for large integers if a JSON field is not also encoded in a typed hash. +- URI hashes use exact UTF-8 bytes. Do not normalize, lowercase, resolve, or fetch a URI before hashing it. + +## Domain Separation + +Every accepted format must bind its domain through at least one of: + +- schema id +- type hash +- chain id +- emitting contract +- deployment id +- verifier set root +- worker or verifier key id +- root scheme id +- report schema hash + +No signature should be accepted if its domain does not match the chain, deployment, verifier set, key registry, and expiry policy being evaluated. + +Implemented domain separator names: + +```text +flowPulseObservationId +indexerCursorId +verifierReportDigest +verifierAttestationEnvelope +rootfieldNamespaceId +rootCommitment +artifactCommitment +workReceiptId +workerIdentity +verifierIdentity +merkleLeaf +merkleInternalNode +devnetBlockHash +``` + +## Versioning Strategy + +Versioning is by schema name and type string. + +- Breaking hash changes create a new type string with a new suffix, such as `FlowPulseObservationV1`. +- Non-breaking documentation changes do not change type hashes. +- Test vectors must declare the schema version that generated them. +- Contracts should not accept unknown schema ids unless a migration or adapter explicitly allows it. +- Verifiers must record which schema version produced each report. + +## MVP Split + +The current package implements: + +- FlowPulse observation identity +- receipt hashing +- artifact root recomputation +- storage receipt commitment hashing +- worker and verifier signature payloads +- local secp256k1 signing and verification helpers for deterministic tests +- deterministic verifier reports +- verifier signature envelopes +- reorg-aware status handling +- test vectors and cross-language conformance tests + +The runnable package in `crypto/src/` currently implements the v0 hash utilities and tests them against fixtures in `crypto/fixtures/`. + +MVP should remain verifier-attested for: + +- off-chain artifact availability +- URI locator policy +- storage provider claims +- model or worker behavior +- final status labels before proof systems exist + +## Future Split + +Future zk/proof-carrying work may prove: + +- artifact chunk inclusion in a committed root +- receipt consistency with event args and observation id +- verifier report consistency with deterministic checks +- recursive aggregation of many reports into a checkpoint +- selective disclosure for private artifact metadata + +Future work must still treat chain finality, data availability, key governance, and privacy policy as separate assumptions unless those systems are explicitly implemented. + +## GitHub Context Used + +This spec aligns with: + +- PR #1, bootstrap repository scaffolding. +- PR #2, FlowPulse and Rootfield contracts foundation. +- Issue #5, minimal indexer/verifier MVP loop. +- Issue #13, canonical FlowPulse observation identity. +- Issue #14, verifier result status vocabulary. +- Issue #17, v0 receipt, attestation, and commitment schema vocabulary. +- Issue #26, deferring CursorRegistry until observation identity stabilizes. + +Searches for `Claw` and `claw` in repository issues and code returned no matching GitHub artifacts during this pass. + +## Follow-Up Issues + +- Define and accept FlowPulse observation identity with indexer and verifier agents. +- Wire verifier services to validate `crypto/fixtures/vectors.json`. +- Add Solidity shared hash library under `contracts/shared/` after schema review, if on-chain adapters need it. +- Add deterministic verifier report fixtures under `services/verifier/` that consume the package outputs. +- Decide Rootfield URI policy: advisory URI fields, length caps, CID-only, or hash-only. +- Define key registry and verifier set root governance. +- Define challenge evidence and response envelopes. +- Produce a decision record before CursorRegistry or proof-carrying receipts are implemented. diff --git a/crypto/MERKLE_AND_ROOTS.md b/crypto/MERKLE_AND_ROOTS.md new file mode 100644 index 00000000..ccea39be --- /dev/null +++ b/crypto/MERKLE_AND_ROOTS.md @@ -0,0 +1,134 @@ +# Merkle Roots And Artifact Commitments + +Status: draft v0. + +FlowMemory keeps raw artifacts off-chain and commits to them through explicit root schemes. + +## Artifact Root V0 + +Type string: + +```solidity +FlowMemoryArtifactRootV0(bytes32 schemeId,bytes32 manifestHash,bytes32 contentMerkleRoot,uint64 byteLength,uint32 chunkSize,bytes32 mediaTypeHash,bytes32 metadataHash) +``` + +Hash: + +```text +artifactRoot = keccak256(abi.encode( + ARTIFACT_ROOT_TYPEHASH, + schemeId, + manifestHash, + contentMerkleRoot, + byteLength, + chunkSize, + mediaTypeHash, + metadataHash +)) +``` + +Default scheme: + +```text +FM-MERKLE-KECCAK256-BINARY-V0 +schemeId = keccak256(bytes("FM-MERKLE-KECCAK256-BINARY-V0")) +``` + +## Merkle Format + +For `FM-MERKLE-KECCAK256-BINARY-V0`: + +```text +chunkHash = keccak256(chunkBytes) +leafHash = keccak256(abi.encode(MERKLE_LEAF_TYPEHASH, index, offset, length, chunkHash)) +nodeHash = keccak256(abi.encode(MERKLE_INTERNAL_NODE_TYPEHASH, leftHash, rightHash)) +emptyRoot = keccak256(bytes("FM-MERKLE-KECCAK256-BINARY-V0:EMPTY")) +``` + +Leaf type string: + +```solidity +FlowMemoryMerkleLeafV0(uint64 index,uint64 offset,uint32 length,bytes32 chunkHash) +``` + +Internal node type string: + +```solidity +FlowMemoryMerkleInternalNodeV0(bytes32 leftHash,bytes32 rightHash) +``` + +Rules: + +- Leaves are ordered by `index`. +- `offset` is the byte offset in the reconstructed artifact. +- `length` is the exact chunk byte length. +- Adjacent pairs form node hashes. +- An odd unpaired hash is promoted unchanged to the next level. +- A one-chunk artifact uses the leaf hash as `contentMerkleRoot`. +- Empty artifacts use `emptyRoot` and `byteLength = 0`. + +## Manifest And Metadata + +`manifestHash` commits to canonical JSON describing: + +- scheme +- version +- byte length +- chunk size +- chunk indexes, offsets, lengths, and chunk hashes + +`metadataHash` commits to canonical JSON for non-sensitive metadata. Sensitive metadata should be encrypted, salted, or omitted. Hashing low-entropy private metadata without salt is not private. + +## CID And URI Policy + +FlowMemory roots should be hash-first. CIDs and URIs can be useful locators, but they are advisory unless their bytes are committed and later opened. + +Comparison: + +- CID/hash-only roots are better for contracts and deterministic verification. +- Advisory URI fields are better for operator ergonomics but leak bytes into logs and can be unavailable, mutable, or misleading. +- A URI should never be the only proof that content matches a commitment. + +MVP recommendation: + +- Use `artifactRoot` and `storageReceiptCommitment` as cryptographic commitments. +- Keep `uri` as advisory log data. +- Hash URI bytes in receipts as `uriHash` when a report needs to bind the exact emitted string. +- Prefer locator commitments for sensitive or mutable storage paths. + +## Merkle Inclusion Vs Receipt Chains + +Merkle inclusion proofs answer: "Does this chunk or leaf belong to this artifact root?" + +Receipt hash chains answer: "Did this receipt extend a prior ordered receipt stream?" + +Use Merkle proofs for artifact openings. Use receipt chains or Rootflow for ordered progression. Do not use a linear receipt chain as a substitute for efficient artifact inclusion proofs. + +## Rootflow And Rootfield + +Rootfield is the namespace and state-commitment side. It scopes roots, owners, counters, and status. + +Rootflow should be the ordered receipt/report progression side. It can chain receipt hashes or report ids into checkpoints. + +Open boundary: + +- Rootfield answers "what state/root is currently committed for this namespace?" +- Rootflow answers "what ordered work/report history led here?" + +Contracts should not depend on that split until a decision record accepts it. + +## Rootfield Namespace And Root Commitment IDs + +Rootfield namespaces can be identified without reading mutable registry state by hashing: + +```solidity +FlowMemoryRootfieldNamespaceV0(uint256 chainId,address registry,bytes32 rootfieldId,bytes32 schemaHash) +``` + +Root commitments bind a rootfield, current root, artifact commitment, parent pulse, and sequence: + +```solidity +FlowMemoryRootCommitmentV0(bytes32 rootfieldId,bytes32 root,bytes32 artifactCommitment,bytes32 parentPulseId,uint64 sequence) +``` + +These helpers are exported as `rootfieldNamespaceId` and `rootCommitment`. They are schema tools for services and future adapters; they are not a production RootfieldRegistry migration. diff --git a/crypto/OBSERVATION_IDENTITY.md b/crypto/OBSERVATION_IDENTITY.md new file mode 100644 index 00000000..60a8a395 --- /dev/null +++ b/crypto/OBSERVATION_IDENTITY.md @@ -0,0 +1,152 @@ +# FlowPulse Observation Identity + +Status: draft v0. + +This document defines how FlowMemory names FlowPulse facts after they are observed by an indexer or verifier. + +## Identifier Roles + +### `pulseId` + +`pulseId` is emitted by the contract. In the current `RootfieldRegistry` skeleton it is computed as: + +```solidity +keccak256(abi.encode( + FlowPulseTypes.SCHEMA_ID, + block.chainid, + address(this), + rootfieldId, + actor, + pulseType, + subject, + commitment, + parentPulseId, + sequence +)) +``` + +Use `pulseId` to link semantic protocol activity across docs, reports, and app views. Do not use it as the only canonical observed-log key. + +### `observationId` + +`observationId` is derived after a transaction receipt and log metadata exist. It binds a FlowPulse log to a concrete chain position. + +Type string: + +```solidity +FlowPulseObservationV0(uint256 chainId,address emittingContract,uint64 blockNumber,bytes32 blockHash,bytes32 txHash,uint32 transactionIndex,uint32 logIndex,bytes32 eventSignature,bytes32 pulseId,bytes32 rootfieldId) +``` + +Hash: + +```text +observationId = keccak256(abi.encode( + FLOWPULSE_OBSERVATION_TYPEHASH, + chainId, + emittingContract, + blockNumber, + blockHash, + txHash, + transactionIndex, + logIndex, + eventSignature, + pulseId, + rootfieldId +)) +``` + +Field sources: + +- `chainId`: chain configuration or receipt context. +- `emittingContract`: log emitter address. +- `blockNumber`: receipt or block metadata. +- `blockHash`: receipt or block metadata. +- `txHash`: transaction receipt identifier. +- `transactionIndex`: transaction position in the block. +- `logIndex`: log position from receipt/RPC metadata. +- `eventSignature`: topic 0 for `FlowPulse(bytes32,bytes32,address,uint8,bytes32,bytes32,bytes32,uint64,uint64,string)`. +- `pulseId`: topic 1 from the event. +- `rootfieldId`: topic 2 from the event. + +`txHash`, `transactionIndex`, and `logIndex` do not exist during contract execution. They must never be treated as hook-known or contract-known values. + +### `reportId` + +`reportId` identifies a deterministic verifier report. It is derived from the report body, not from the verifier signature. + +Type string: + +```solidity +FlowMemoryVerifierReportV0(bytes32 reportSchemaHash,bytes32 observationId,bytes32 receiptHash,bytes32 verifierId,bytes32 verifierSetRoot,uint8 status,bytes32 checksRoot,uint64 finalizedBlockNumber,bytes32 finalizedBlockHash,uint16 reportVersion) +``` + +Hash: + +```text +reportId = keccak256(abi.encode( + VERIFIER_REPORT_TYPEHASH, + reportSchemaHash, + observationId, + receiptHash, + verifierId, + verifierSetRoot, + status, + checksRoot, + finalizedBlockNumber, + finalizedBlockHash, + reportVersion +)) +``` + +The verifier signature envelope signs `reportId`; the signature is not part of `reportId`. + +### `cursorId` + +`cursorId` identifies an indexer checkpoint for an ordered stream. It is not a replacement for `observationId`; it points to the latest observation accepted by that stream. + +Type string: + +```solidity +FlowMemoryIndexerCursorV0(bytes32 sourceId,bytes32 streamId,uint64 sequence,bytes32 observationId,bytes32 previousCursorId) +``` + +Hash: + +```text +cursorId = keccak256(abi.encode( + INDEXER_CURSOR_TYPEHASH, + sourceId, + streamId, + sequence, + observationId, + previousCursorId +)) +``` + +## Reorg Handling + +`observationId` includes `blockHash`, so a reorg creates a different observation. Verifiers should model state transitions like this: + +```text +observed -> pending_finality -> verified +observed -> reorged +verified -> superseded +failed -> superseded +``` + +Requirements: + +- A pre-finality observation can be useful but must be labeled pending or observed. +- A reorged observation must not silently mutate into a new observation id. +- A report over a reorged observation should be marked `reorged` or `superseded`. +- Apps must display `pulseId` and `observationId` differently when both are relevant. + +## Duplicate Handling + +Duplicate logs with the same `(chainId, emittingContract, txHash, logIndex)` should produce the same `observationId`. + +Duplicate `pulseId` values in different block contexts are not necessarily duplicates. Verifiers must compare the full observation identity and the event args hash before collapsing them. + +## Rootfield Context + +`rootfieldId` is part of observation identity because FlowPulse events are Rootfield-scoped. The same `pulseId` and `rootfieldId` can be used to trace semantic continuity, while `observationId` gives receipt-level uniqueness. diff --git a/crypto/README.md b/crypto/README.md new file mode 100644 index 00000000..54ae3b69 --- /dev/null +++ b/crypto/README.md @@ -0,0 +1,85 @@ +# FlowMemory Cryptography + +Status: draft v0. + +This directory defines the cryptographic vocabulary, runnable utilities, fixtures, and tests that contracts, indexers, verifiers, workers, and future appchain research should share. The package is intentionally commitment-first and verifier-friendly. It does not claim that FlowMemory is fully trustless before proof systems, verifier enforcement, and challenge handling exist. + +## Package Commands + +Install dependencies from this directory: + +```powershell +cd E:\FlowMemory\flowmemory-crypto\crypto +npm install +``` + +Requires Node.js `>=20.19.0`. + +Run deterministic tests: + +```powershell +npm test +``` + +Print the sample hash outputs: + +```powershell +npm run vectors +``` + +Validate all package-level vector fixtures: + +```powershell +npm run validate:vectors +``` + +## Read Order + +1. `FLOWMEMORY_CRYPTO_SPEC.md` +2. `OBSERVATION_IDENTITY.md` +3. `RECEIPT_HASHING.md` +4. `MERKLE_AND_ROOTS.md` +5. `ATTESTATIONS.md` +6. `TEST_VECTORS.md` + +Runnable fixtures live in `fixtures/`. `fixtures/vectors.json` contains the current 21 package-level vectors. Supporting cross-language vectors live in `test-vectors/`. + +Validate the current vector set with: + +```powershell +python validate_test_vectors.py +``` + +The Python validator is a cross-check for the FlowPulse aggregate vector. The production-candidate package paths are `npm test` and `npm run validate:vectors`. + +## Core Vocabulary + +- `pulseId`: contract-emitted logical FlowPulse identifier. It is produced during contract execution and intentionally excludes receipt-only metadata. +- `observationId`: indexer/verifier identifier for a specific observed FlowPulse log after receipt metadata exists. +- `receiptHash`: commitment to a FlowPulse observation, event args, artifact root, storage commitment, and evidence root. +- `artifactRoot`: commitment to off-chain artifact bytes and metadata. +- `reportId`: deterministic identifier for a verifier report. +- `attestation`: signed worker or verifier envelope over a receipt, report, artifact, or root. + +## Implemented Helpers + +The package exports Keccak helpers, canonical JSON hashing, typed hash utilities, FlowPulse observation ids, cursor ids, report digests, receipt hashes, artifact/root commitments, work receipt ids, Merkle roots, worker/verifier signature payloads, verifier attestation envelope hashes, and local secp256k1 sign/verify helpers for tests. + +The implementation is ESM JavaScript with `src/index.d.ts` declarations for TypeScript consumers. + +## MVP Boundary + +MVP crypto can provide tamper-evident facts, deterministic IDs, replay-resistant signatures, artifact inclusion checks, and verifier reports. MVP crypto cannot prove data availability forever, prove model output correctness, or make verifier attestations trustless. + +## Future Boundary + +Future work may add proof-carrying receipts, zk circuits for receipt consistency, recursive report aggregation, and appchain/L1 verification tracks. Those remain research until public inputs, witnesses, circuits, and enforcement paths are specified. + +## Integration Notes + +There is no `services/shared/` package in this repository yet. Until one exists, services should either: + +- import this package directly from `crypto/` in local development, or +- mirror the exported functions from `crypto/src/index.js` with tests against `crypto/fixtures/`. + +Indexer and verifier services must not hand-roll different hash formats. If a service cannot import this package, it should copy the type strings and vectors exactly and prove compatibility by passing the same fixture hashes. diff --git a/crypto/RECEIPT_HASHING.md b/crypto/RECEIPT_HASHING.md new file mode 100644 index 00000000..cc981998 --- /dev/null +++ b/crypto/RECEIPT_HASHING.md @@ -0,0 +1,143 @@ +# FlowMemory Receipt Hashing + +Status: draft v0. + +Receipts bind an observation to event args, off-chain artifact commitments, storage commitments, and evidence roots. They do not store raw artifacts or make verifier claims trustless. + +## Event Args Hash + +Type string: + +```solidity +FlowPulseEventArgsV0(bytes32 pulseId,bytes32 rootfieldId,address actor,uint8 pulseType,bytes32 subject,bytes32 commitment,bytes32 parentPulseId,uint64 sequence,uint64 occurredAt,bytes32 uriHash) +``` + +Hash: + +```text +eventArgsHash = keccak256(abi.encode( + FLOWPULSE_EVENT_ARGS_TYPEHASH, + pulseId, + rootfieldId, + actor, + pulseType, + subject, + commitment, + parentPulseId, + sequence, + occurredAt, + uriHash +)) +``` + +`uriHash = keccak256(bytes(uri))` over exact UTF-8 bytes. URI strings remain advisory log data. + +## FlowPulse Receipt V0 + +Type string: + +```solidity +FlowPulseReceiptV0(bytes32 observationId,bytes32 eventArgsHash,bytes32 artifactRoot,bytes32 storageReceiptCommitment,bytes32 evidenceRoot,uint16 receiptVersion) +``` + +Hash: + +```text +receiptHash = keccak256(abi.encode( + FLOWPULSE_RECEIPT_TYPEHASH, + observationId, + eventArgsHash, + artifactRoot, + storageReceiptCommitment, + evidenceRoot, + receiptVersion +)) +``` + +Field notes: + +- `observationId` binds receipt metadata that only exists after execution. +- `eventArgsHash` binds the FlowPulse event args. +- `artifactRoot` is zero when no artifact is attached. +- `storageReceiptCommitment` is zero when no storage claim exists. +- `evidenceRoot` commits to verifier evidence, openings, or report inputs. It is zero if no evidence set exists. +- `receiptVersion` starts at `0`. + +## Storage Receipt Commitment + +Type string: + +```solidity +FlowMemoryStorageReceiptCommitmentV0(bytes32 artifactRoot,bytes32 providerId,bytes32 locationCommitment,bytes32 retentionPolicyHash,bytes32 encryptionCommitment,bytes32 availabilitySampleRoot,uint64 issuedAtUnixMs,uint64 expiresAtUnixMs,bytes32 nonce) +``` + +Hash: + +```text +storageReceiptCommitment = keccak256(abi.encode( + STORAGE_RECEIPT_TYPEHASH, + artifactRoot, + providerId, + locationCommitment, + retentionPolicyHash, + encryptionCommitment, + availabilitySampleRoot, + issuedAtUnixMs, + expiresAtUnixMs, + nonce +)) +``` + +This is a commitment to a storage claim. It is not a guarantee that the artifact will always be retrievable. + +## Replay Protection + +Receipt replay protection comes from: + +- `observationId`, which binds chain id, contract, block, tx, log position, event signature, pulse id, and rootfield id +- `eventArgsHash`, which binds emitted event contents +- `receiptVersion`, which prevents silent schema mutation +- signature envelopes, which bind chain id, deployment id, key id, expiry, sequence or nonce, and verifier set root + +Verifiers must reject: + +- receipts whose `observationId` cannot be recomputed from the receipt and log +- event args that do not match the log payload +- receipts from reorged observations unless the report status is explicitly historical or reorged +- unknown receipt versions +- storage receipt commitments outside their retention window for current availability claims + +## Hash Chain Use + +Receipt hash chains can model order: + +```text +nextChainHead = keccak256(abi.encode(chainHeadTypeHash, previousReceiptHash, receiptHash, sequence)) +``` + +Hash chains do not replace Merkle inclusion proofs for artifacts. Use hash chains for ordered receipt progression and Merkle roots for efficient artifact inclusion/opening. + +## Work Receipt ID + +Workers can produce replay-resistant work receipt identifiers over a receipt and worker sequence. + +Type string: + +```solidity +FlowMemoryWorkReceiptV0(bytes32 observationId,bytes32 receiptHash,bytes32 workerId,uint64 workerSequence,bytes32 nonce) +``` + +Hash: + +```text +workReceiptId = keccak256(abi.encode( + WORK_RECEIPT_TYPEHASH, + observationId, + receiptHash, + workerId, + workerSequence, + nonce +)) +``` + +`workReceiptId` is an off-chain coordination identifier. It does not prove that the worker's output is correct unless a verifier policy checks the underlying work and signature. diff --git a/crypto/TEST_VECTORS.md b/crypto/TEST_VECTORS.md new file mode 100644 index 00000000..bed19b28 --- /dev/null +++ b/crypto/TEST_VECTORS.md @@ -0,0 +1,100 @@ +# FlowMemory Crypto Test Vectors + +Status: draft v0. + +The test vectors are synthetic and contain no production secrets or signatures. + +## Vector Files + +- `fixtures/sample-flowpulse.json`: FlowPulse event args and expected `pulseId` / `eventArgsHash`. +- `fixtures/sample-observation.json`: observation metadata, artifact/storage inputs, and expected `observationId` / `receiptHash`. +- `fixtures/sample-report.json`: verifier report, worker signature payload, verifier signature payload, and attestation envelope expectations. +- `fixtures/vectors.json`: 21 package-level vectors for domains, canonical JSON, observation ids, receipts, artifacts, Merkle roots, reports, attestations, cursors, identities, root commitments, work receipts, and devnet block hashes. +- `test-vectors/flowpulse-observation-v0.json`: FlowPulse-specific observation, receipt, artifact, report, worker signature digest, and verifier signature digest. + +## FlowPulse Observation Vector Highlights + +Input: + +```text +chainId = 8453 +emittingContract = 0x1234567890abcdef1234567890abcdef12345678 +eventSignature = 0x5d07190b9ae441b4d7b16259a48424acd451492b12f5f99a29f5bfd992c13e43 +blockHash = 0x1111111111111111111111111111111111111111111111111111111111111111 +txHash = 0x2222222222222222222222222222222222222222222222222222222222222222 +transactionIndex = 7 +logIndex = 3 +``` + +Derived: + +```text +pulseId = 0x86b8325d6da0767e12097aed29aefe4820aaf4d6b7d4bb8371f1db927fda9d9d +observationId = 0xd80d0a3b317ceae266c9b7983c5a9376529f457a01469c96d8d3fd5a6c2d8a3f +receiptHash = 0xca2ebca63e004ff4b0ca9766acbb2862b45059a480d911b67dbc25e937c2e733 +artifactRoot = 0xff501ac63f870de597cdc2a28dad8aeae3b52c5f1e2a658b1ea37c440b76f644 +reportId = 0x1b1c2940d6e83ee78a7e0a8285e4ce2530da1ce7964817806e61520a2e767355 +attestationEnvelopeHash = 0x3e139c10ff22aea00c4442698f2d8650ba85f811c723cb7e4f28094d833fea80 +``` + +## Verification Requirements + +An implementation should reproduce: + +- every type hash +- `pulseId` using the current RootfieldRegistry formula +- `observationId` from receipt/log metadata +- `eventArgsHash` from decoded FlowPulse args +- `receiptHash` from observation, event args, artifact, storage, and evidence commitments +- Merkle root and artifact root +- deterministic verifier report id +- EIP-712 signing digests without requiring test private keys + +Run the package test suite: + +```powershell +cd E:\FlowMemory\flowmemory-crypto\crypto +npm test +``` + +Run the package vector validator: + +```powershell +npm run validate:vectors +``` + +Expected output: + +```text +FLOWMEMORY_CRYPTO_VECTORS_OK 21 +``` + +Print the sample vector summary: + +```powershell +npm run vectors +``` + +Run the Python cross-check validator: + +```powershell +python validate_test_vectors.py +``` + +Expected output: + +```text +FLOWPULSE_VECTOR_RECOMPUTE_OK +``` + +## Negative Cases Covered Or Remaining + +- changed `blockHash` should change `observationId` +- changed `logIndex` should change `observationId` +- changed `uri` should change `eventArgsHash` +- swapped Merkle leaves should change `contentMerkleRoot` and therefore any recomputed `artifactRoot` +- wrong verifier set root should change verifier signing digest +- expired worker signature should be rejected by verifier policy +- reorged observation should not mutate into a verified report + +The package tests cover the first five checks. Expiry and reorg-to-report policy are verifier-service responsibilities because they require policy context, not just hash recomputation. diff --git a/crypto/fixtures/sample-flowpulse.json b/crypto/fixtures/sample-flowpulse.json new file mode 100644 index 00000000..dc835723 --- /dev/null +++ b/crypto/fixtures/sample-flowpulse.json @@ -0,0 +1,31 @@ +{ + "contractInput": { + "chainId": 8453, + "emittingContract": "0x1234567890abcdef1234567890abcdef12345678", + "rootfieldId": "0x726f6f746669656c642e62657461000000000000000000000000000000000000", + "actor": "0x90f8bf6a479f320ead074411a4b0e7944ea8c9c1", + "pulseType": 1, + "subject": "0x726f6f746669656c642e62657461000000000000000000000000000000000000", + "commitment": "0xf2909ae97d47adc4ec7106ea70cbed7f582d6e6066dd058a7bb5c637e493d5f5", + "parentPulseId": "0x0000000000000000000000000000000000000000000000000000000000000000", + "sequence": 1 + }, + "input": { + "pulseId": "0x86b8325d6da0767e12097aed29aefe4820aaf4d6b7d4bb8371f1db927fda9d9d", + "rootfieldId": "0x726f6f746669656c642e62657461000000000000000000000000000000000000", + "actor": "0x90f8bf6a479f320ead074411a4b0e7944ea8c9c1", + "pulseType": 1, + "subject": "0x726f6f746669656c642e62657461000000000000000000000000000000000000", + "commitment": "0xf2909ae97d47adc4ec7106ea70cbed7f582d6e6066dd058a7bb5c637e493d5f5", + "parentPulseId": "0x0000000000000000000000000000000000000000000000000000000000000000", + "sequence": 1, + "occurredAt": 1778640000, + "uriHash": "0x573d5e91ce4505f0968237333e7faa29e5b3db0271bd53ca845f6d06315880d0" + }, + "expected": { + "schemaId": "0xece59fcceac2b4dcd9e8732bb223b2b9cd7ae3626eeb79457b11a99f9bfc6fef", + "eventSignature": "0x5d07190b9ae441b4d7b16259a48424acd451492b12f5f99a29f5bfd992c13e43", + "pulseId": "0x86b8325d6da0767e12097aed29aefe4820aaf4d6b7d4bb8371f1db927fda9d9d", + "eventArgsHash": "0xd84ede2187c11d8927a683eb4a8aea41e6266e4feb0cb10eb3aeb43b857a3ed2" + } +} diff --git a/crypto/fixtures/sample-observation.json b/crypto/fixtures/sample-observation.json new file mode 100644 index 00000000..f0a30aad --- /dev/null +++ b/crypto/fixtures/sample-observation.json @@ -0,0 +1,47 @@ +{ + "input": { + "chainId": 8453, + "emittingContract": "0x1234567890abcdef1234567890abcdef12345678", + "blockNumber": 12345678, + "blockHash": "0x1111111111111111111111111111111111111111111111111111111111111111", + "txHash": "0x2222222222222222222222222222222222222222222222222222222222222222", + "transactionIndex": 7, + "logIndex": 3, + "eventSignature": "0x5d07190b9ae441b4d7b16259a48424acd451492b12f5f99a29f5bfd992c13e43", + "pulseId": "0x86b8325d6da0767e12097aed29aefe4820aaf4d6b7d4bb8371f1db927fda9d9d", + "rootfieldId": "0x726f6f746669656c642e62657461000000000000000000000000000000000000" + }, + "artifact": { + "chunks": ["alpha", "beta", "gamma"], + "chunkSize": 5, + "mediaType": "text/plain; charset=utf-8", + "metadata": { + "name": "flowmemory-test-artifact", + "purpose": "flowpulse-observation-vector" + } + }, + "storage": { + "artifactRoot": "0xff501ac63f870de597cdc2a28dad8aeae3b52c5f1e2a658b1ea37c440b76f644", + "providerId": "0xe258c429fc59ff6f69bec97364ee5d4ad329b65231151219e7241739ed50b43c", + "locationCommitment": "0xcce30f05ced736a23714458a881d47f8c735466f8d6a9e46afb6b146ac14d938", + "retentionPolicyHash": "0x330509f9a6a21217e358dd4e862e18f0f1e02ad307685ecf6b13ac3772a1ef45", + "encryptionCommitment": "0x0000000000000000000000000000000000000000000000000000000000000000", + "availabilitySampleRoot": "0x784d414314d577f517ba5daafed500794621a7e114da383cbea7fd058fac5555", + "issuedAtUnixMs": 1767225600000, + "expiresAtUnixMs": 1893456000000, + "nonce": "0x41a13ee24f0edded7a93c874f40f87bfd6db19f4f06c3ff788ee63a902d831e3" + }, + "receipt": { + "artifactRoot": "0xff501ac63f870de597cdc2a28dad8aeae3b52c5f1e2a658b1ea37c440b76f644", + "storageReceiptCommitment": "0xb341bddb79e03b03c0a6acfcc4d01d393a84d5187f02fea2fd1761c09f345da7", + "evidenceRoot": "0xe8e37a51ab4052012bd245722091179b164d04c550d736d0b95f81da8d7b10de", + "receiptVersion": 0 + }, + "expected": { + "observationId": "0xd80d0a3b317ceae266c9b7983c5a9376529f457a01469c96d8d3fd5a6c2d8a3f", + "artifactRoot": "0xff501ac63f870de597cdc2a28dad8aeae3b52c5f1e2a658b1ea37c440b76f644", + "contentMerkleRoot": "0x71d29e65fd4f25301225dcaabeb40fddabac5cbb327d4992c21a591b05eebca2", + "storageReceiptCommitment": "0xb341bddb79e03b03c0a6acfcc4d01d393a84d5187f02fea2fd1761c09f345da7", + "receiptHash": "0xca2ebca63e004ff4b0ca9766acbb2862b45059a480d911b67dbc25e937c2e733" + } +} diff --git a/crypto/fixtures/sample-report.json b/crypto/fixtures/sample-report.json new file mode 100644 index 00000000..cb060509 --- /dev/null +++ b/crypto/fixtures/sample-report.json @@ -0,0 +1,69 @@ +{ + "input": { + "reportSchemaHash": "0x487a3a70cf72c079fa8645f8d732cbe286f7f3aa0abd654f212ac1c1911a20ae", + "observationId": "0xd80d0a3b317ceae266c9b7983c5a9376529f457a01469c96d8d3fd5a6c2d8a3f", + "receiptHash": "0xca2ebca63e004ff4b0ca9766acbb2862b45059a480d911b67dbc25e937c2e733", + "verifierId": "0xaf0bdaa3ef421cfa8494019fb436baabcfdc65b55cf858c2d605a348c8c0aa48", + "verifierSetRoot": "0x96130fe314e14b7f7d4347094f6b5ec4338b1f7730bf505e59fd3e731753ff8b", + "status": 2, + "checksRoot": "0x48468783cf12adfeba9be8e0a5e250ab04b19d5034f7e1996610cf05f4fcef83", + "finalizedBlockNumber": 12345742, + "finalizedBlockHash": "0x1111111111111111111111111111111111111111111111111111111111111111", + "reportVersion": 0 + }, + "eip712": { + "deploymentId": "0xb501a54093cdfe0afe44da8cd94f376801f8d0123c5f2aef235156e326b8d1ea", + "chainId": 8453, + "verifyingContract": "0x0000000000000000000000000000000000000000" + }, + "workerSignature": { + "input": { + "receiptHash": "0xca2ebca63e004ff4b0ca9766acbb2862b45059a480d911b67dbc25e937c2e733", + "workerId": "0x7803fe537f4b6e4ddd47f00b97a87c06aad1e42e22b83fdb522268e393319598", + "workerKeyId": "0xd2435c19a90c604a8a7e0b079be17b23846efd2313821825ebe2a9af5f3f9924", + "workerSequence": 1, + "expiresAtUnixMs": 1769817600000, + "artifactRoot": "0xff501ac63f870de597cdc2a28dad8aeae3b52c5f1e2a658b1ea37c440b76f644", + "nonce": "0x3574313751905cc45384e0a3f62258d38e9c64735e29bcee180459bb16cd4039" + }, + "expected": { + "domainSeparator": "0xb1bd381ce93a8b36ea7e8389688cd847207acef42affdaa23743079f5614947b", + "structHash": "0x726e78bb0647bd8604d9f0a84ce2cd591039515dbbd1dc1bd18977e962643bee", + "signingDigest": "0x02a43a42d3c182a55b817835e1cdbca1238a4f86cb12e92063b710d38d85fbe3" + } + }, + "verifierSignature": { + "input": { + "reportId": "0x1b1c2940d6e83ee78a7e0a8285e4ce2530da1ce7964817806e61520a2e767355", + "verifierId": "0xaf0bdaa3ef421cfa8494019fb436baabcfdc65b55cf858c2d605a348c8c0aa48", + "verifierKeyId": "0x9bc6e07b6ca6b1f21697ed511d211b82186960d193036d5817f1a8a46a6daff5", + "verifierSetRoot": "0x96130fe314e14b7f7d4347094f6b5ec4338b1f7730bf505e59fd3e731753ff8b", + "issuedAtUnixMs": 1767225660000, + "expiresAtUnixMs": 1769817600000, + "nonce": "0xb50ddd6d6d2d9d3dc8310462bad743d6383de268350d75b6fa84d59c74687b93" + }, + "expected": { + "domainSeparator": "0x199a93f8dae72a137df50c20b442f87315f4d7bf6b39cca9d62e07c48c58106a", + "structHash": "0x35b8f9982c297652622fd66fe931d9669db4847b2056cd126d92e5bd76c93315", + "signingDigest": "0x08334f56b2ac5a7c11bf33154907d37a05253f4d3324999e820ca166c0a39b1e" + } + }, + "attestationEnvelope": { + "input": { + "subjectHash": "0x1b1c2940d6e83ee78a7e0a8285e4ce2530da1ce7964817806e61520a2e767355", + "subjectKind": 2, + "attesterId": "0xaf0bdaa3ef421cfa8494019fb436baabcfdc65b55cf858c2d605a348c8c0aa48", + "attesterKeyId": "0x9bc6e07b6ca6b1f21697ed511d211b82186960d193036d5817f1a8a46a6daff5", + "verifierSetRoot": "0x96130fe314e14b7f7d4347094f6b5ec4338b1f7730bf505e59fd3e731753ff8b", + "issuedAtUnixMs": 1767225660000, + "expiresAtUnixMs": 1769817600000, + "nonce": "0xb50ddd6d6d2d9d3dc8310462bad743d6383de268350d75b6fa84d59c74687b93" + }, + "expected": { + "attestationEnvelopeHash": "0x3e139c10ff22aea00c4442698f2d8650ba85f811c723cb7e4f28094d833fea80" + } + }, + "expected": { + "reportId": "0x1b1c2940d6e83ee78a7e0a8285e4ce2530da1ce7964817806e61520a2e767355" + } +} diff --git a/crypto/fixtures/vectors.json b/crypto/fixtures/vectors.json new file mode 100644 index 00000000..961dc133 --- /dev/null +++ b/crypto/fixtures/vectors.json @@ -0,0 +1,276 @@ +{ + "schema": "flowmemory.crypto.test-vectors.v0", + "vectorCount": 21, + "vectors": [ + { + "name": "domain.flowPulseObservationId", + "function": "domainSeparator", + "input": { + "domainName": "flowPulseObservationId" + }, + "expected": "0x7a42d0c550a1e18a737aa6627ab6a76a6524750e0ee78d73a3b489f662e82474" + }, + { + "name": "canonical.json.hash.lowercase-hex-and-key-order", + "function": "canonicalJsonHash", + "input": { + "z": [ + "0xABCDEF", + { + "b": 2, + "a": "0xDEADBEEF" + } + ], + "a": { + "d": 4, + "c": 3 + } + }, + "expected": "0xa26021dad359db2e0d56b4ffecaf0d5d0772303c1f985bfbb3a9cdae67e0766b" + }, + { + "name": "flowpulse.schemaId", + "function": "flowPulseSchemaId", + "input": {}, + "expected": "0xece59fcceac2b4dcd9e8732bb223b2b9cd7ae3626eeb79457b11a99f9bfc6fef" + }, + { + "name": "flowpulse.contractPulseId", + "function": "contractPulseId", + "input": { + "chainId": 8453, + "emittingContract": "0x1234567890abcdef1234567890abcdef12345678", + "rootfieldId": "0x726f6f746669656c642e62657461000000000000000000000000000000000000", + "actor": "0x90f8bf6a479f320ead074411a4b0e7944ea8c9c1", + "pulseType": 1, + "subject": "0x726f6f746669656c642e62657461000000000000000000000000000000000000", + "commitment": "0xf2909ae97d47adc4ec7106ea70cbed7f582d6e6066dd058a7bb5c637e493d5f5", + "parentPulseId": "0x0000000000000000000000000000000000000000000000000000000000000000", + "sequence": 1 + }, + "expected": "0x86b8325d6da0767e12097aed29aefe4820aaf4d6b7d4bb8371f1db927fda9d9d" + }, + { + "name": "flowpulse.eventArgsHash", + "function": "flowPulseEventArgsHash", + "input": { + "pulseId": "0x86b8325d6da0767e12097aed29aefe4820aaf4d6b7d4bb8371f1db927fda9d9d", + "rootfieldId": "0x726f6f746669656c642e62657461000000000000000000000000000000000000", + "actor": "0x90f8bf6a479f320ead074411a4b0e7944ea8c9c1", + "pulseType": 1, + "subject": "0x726f6f746669656c642e62657461000000000000000000000000000000000000", + "commitment": "0xf2909ae97d47adc4ec7106ea70cbed7f582d6e6066dd058a7bb5c637e493d5f5", + "parentPulseId": "0x0000000000000000000000000000000000000000000000000000000000000000", + "sequence": 1, + "occurredAt": 1778640000, + "uriHash": "0x573d5e91ce4505f0968237333e7faa29e5b3db0271bd53ca845f6d06315880d0" + }, + "expected": "0xd84ede2187c11d8927a683eb4a8aea41e6266e4feb0cb10eb3aeb43b857a3ed2" + }, + { + "name": "flowpulse.observationId", + "function": "flowPulseObservationId", + "input": { + "chainId": 8453, + "emittingContract": "0x1234567890abcdef1234567890abcdef12345678", + "blockNumber": 12345678, + "blockHash": "0x1111111111111111111111111111111111111111111111111111111111111111", + "txHash": "0x2222222222222222222222222222222222222222222222222222222222222222", + "transactionIndex": 7, + "logIndex": 3, + "eventSignature": "0x5d07190b9ae441b4d7b16259a48424acd451492b12f5f99a29f5bfd992c13e43", + "pulseId": "0x86b8325d6da0767e12097aed29aefe4820aaf4d6b7d4bb8371f1db927fda9d9d", + "rootfieldId": "0x726f6f746669656c642e62657461000000000000000000000000000000000000" + }, + "expected": "0xd80d0a3b317ceae266c9b7983c5a9376529f457a01469c96d8d3fd5a6c2d8a3f" + }, + { + "name": "receipt.hash", + "function": "receiptHash", + "input": { + "artifactRoot": "0xff501ac63f870de597cdc2a28dad8aeae3b52c5f1e2a658b1ea37c440b76f644", + "storageReceiptCommitment": "0xb341bddb79e03b03c0a6acfcc4d01d393a84d5187f02fea2fd1761c09f345da7", + "evidenceRoot": "0xe8e37a51ab4052012bd245722091179b164d04c550d736d0b95f81da8d7b10de", + "receiptVersion": 0, + "observationId": "0xd80d0a3b317ceae266c9b7983c5a9376529f457a01469c96d8d3fd5a6c2d8a3f", + "eventArgsHash": "0xd84ede2187c11d8927a683eb4a8aea41e6266e4feb0cb10eb3aeb43b857a3ed2" + }, + "expected": "0xca2ebca63e004ff4b0ca9766acbb2862b45059a480d911b67dbc25e937c2e733" + }, + { + "name": "artifact.commitment", + "function": "artifactFromChunks", + "select": "artifactRoot", + "input": { + "chunks": [ + "alpha", + "beta", + "gamma" + ], + "chunkSize": 5, + "mediaType": "text/plain; charset=utf-8", + "metadata": { + "name": "flowmemory-test-artifact", + "purpose": "flowpulse-observation-vector" + } + }, + "expected": "0xff501ac63f870de597cdc2a28dad8aeae3b52c5f1e2a658b1ea37c440b76f644" + }, + { + "name": "merkle.leaf.0", + "function": "merkleLeafHash", + "input": { + "index": 0, + "offset": 0, + "length": 5, + "chunkHash": "0x6dfc21ac0c8c2db036305d8bc6f887630d35e156f37d5a7e2275bc05bc004846" + }, + "expected": "0x9a24f213573e0b107e7e45ad5cd1ecc6f0184e6933873617e58df5e1525bec4f" + }, + { + "name": "merkle.root.sampleArtifact", + "function": "merkleRoot", + "input": { + "leaves": [ + "0x9a24f213573e0b107e7e45ad5cd1ecc6f0184e6933873617e58df5e1525bec4f", + "0x980583be7de4c1cb88ba0991408490ea8a195c546982fcbdf0682b9c64987b2f", + "0x11d1a636ddfb5bc0d05f35c457bc60c7fdd8fd8555047e5e38ef94f061901e02" + ] + }, + "expected": "0x71d29e65fd4f25301225dcaabeb40fddabac5cbb327d4992c21a591b05eebca2" + }, + { + "name": "merkle.root.empty", + "function": "emptyMerkleRoot", + "input": {}, + "expected": "0xd696a744928cde1db971775966b90e254e54e2cc4a8952b099f9db5ef7bf3434" + }, + { + "name": "storage.receiptCommitment", + "function": "storageReceiptCommitmentHash", + "input": { + "artifactRoot": "0xff501ac63f870de597cdc2a28dad8aeae3b52c5f1e2a658b1ea37c440b76f644", + "providerId": "0xe258c429fc59ff6f69bec97364ee5d4ad329b65231151219e7241739ed50b43c", + "locationCommitment": "0xcce30f05ced736a23714458a881d47f8c735466f8d6a9e46afb6b146ac14d938", + "retentionPolicyHash": "0x330509f9a6a21217e358dd4e862e18f0f1e02ad307685ecf6b13ac3772a1ef45", + "encryptionCommitment": "0x0000000000000000000000000000000000000000000000000000000000000000", + "availabilitySampleRoot": "0x784d414314d577f517ba5daafed500794621a7e114da383cbea7fd058fac5555", + "issuedAtUnixMs": 1767225600000, + "expiresAtUnixMs": 1893456000000, + "nonce": "0x41a13ee24f0edded7a93c874f40f87bfd6db19f4f06c3ff788ee63a902d831e3" + }, + "expected": "0xb341bddb79e03b03c0a6acfcc4d01d393a84d5187f02fea2fd1761c09f345da7" + }, + { + "name": "verifier.reportDigest", + "function": "verifierReportHash", + "input": { + "reportSchemaHash": "0x487a3a70cf72c079fa8645f8d732cbe286f7f3aa0abd654f212ac1c1911a20ae", + "observationId": "0xd80d0a3b317ceae266c9b7983c5a9376529f457a01469c96d8d3fd5a6c2d8a3f", + "receiptHash": "0xca2ebca63e004ff4b0ca9766acbb2862b45059a480d911b67dbc25e937c2e733", + "verifierId": "0xaf0bdaa3ef421cfa8494019fb436baabcfdc65b55cf858c2d605a348c8c0aa48", + "verifierSetRoot": "0x96130fe314e14b7f7d4347094f6b5ec4338b1f7730bf505e59fd3e731753ff8b", + "status": 2, + "checksRoot": "0x48468783cf12adfeba9be8e0a5e250ab04b19d5034f7e1996610cf05f4fcef83", + "finalizedBlockNumber": 12345742, + "finalizedBlockHash": "0x1111111111111111111111111111111111111111111111111111111111111111", + "reportVersion": 0 + }, + "expected": "0x1b1c2940d6e83ee78a7e0a8285e4ce2530da1ce7964817806e61520a2e767355" + }, + { + "name": "verifier.attestationEnvelopeDigest", + "function": "attestationEnvelopeHash", + "input": { + "subjectHash": "0x1b1c2940d6e83ee78a7e0a8285e4ce2530da1ce7964817806e61520a2e767355", + "subjectKind": 2, + "attesterId": "0xaf0bdaa3ef421cfa8494019fb436baabcfdc65b55cf858c2d605a348c8c0aa48", + "attesterKeyId": "0x9bc6e07b6ca6b1f21697ed511d211b82186960d193036d5817f1a8a46a6daff5", + "verifierSetRoot": "0x96130fe314e14b7f7d4347094f6b5ec4338b1f7730bf505e59fd3e731753ff8b", + "issuedAtUnixMs": 1767225660000, + "expiresAtUnixMs": 1769817600000, + "nonce": "0xb50ddd6d6d2d9d3dc8310462bad743d6383de268350d75b6fa84d59c74687b93" + }, + "expected": "0x3e139c10ff22aea00c4442698f2d8650ba85f811c723cb7e4f28094d833fea80" + }, + { + "name": "indexer.cursorId", + "function": "indexerCursorId", + "input": { + "sourceId": "0x56795b5f5d175413c1c97d95fac513ec63c0b20a56671bfab0e6da351f8b6717", + "streamId": "0xb8fee8082266b0191ad8c431e100faf1f61cb118123f2443848dd9396d051b7a", + "sequence": 1, + "observationId": "0xd80d0a3b317ceae266c9b7983c5a9376529f457a01469c96d8d3fd5a6c2d8a3f", + "previousCursorId": "0x0000000000000000000000000000000000000000000000000000000000000000" + }, + "expected": "0x6f4c92cfc44dbc0b6cbe21a1830459196c29e04e0612003f77efae4a4cda9b15" + }, + { + "name": "rootfield.namespaceId", + "function": "rootfieldNamespaceId", + "input": { + "chainId": 8453, + "registry": "0x0000000000000000000000000000000000000000", + "rootfieldId": "0x726f6f746669656c642e62657461000000000000000000000000000000000000", + "schemaHash": "0x1707d32ac6d76da1c0257023835eccd319622ece1e73fadcf2acf462c28efe8a" + }, + "expected": "0x6da665da25815ab5c3ee446af87a6883ad577c7a218c2f3140e3bd171548a806" + }, + { + "name": "root.commitment", + "function": "rootCommitment", + "input": { + "rootfieldId": "0x726f6f746669656c642e62657461000000000000000000000000000000000000", + "root": "0x71d29e65fd4f25301225dcaabeb40fddabac5cbb327d4992c21a591b05eebca2", + "artifactCommitment": "0xff501ac63f870de597cdc2a28dad8aeae3b52c5f1e2a658b1ea37c440b76f644", + "parentPulseId": "0x0000000000000000000000000000000000000000000000000000000000000000", + "sequence": 1 + }, + "expected": "0xfa69bad84d06fa38d6928c3d8e50e926c2ef5ec0ed446858f350b49d75532e0b" + }, + { + "name": "work.receiptId", + "function": "workReceiptId", + "input": { + "observationId": "0xd80d0a3b317ceae266c9b7983c5a9376529f457a01469c96d8d3fd5a6c2d8a3f", + "receiptHash": "0xca2ebca63e004ff4b0ca9766acbb2862b45059a480d911b67dbc25e937c2e733", + "workerId": "0x7803fe537f4b6e4ddd47f00b97a87c06aad1e42e22b83fdb522268e393319598", + "workerSequence": 1, + "nonce": "0x3574313751905cc45384e0a3f62258d38e9c64735e29bcee180459bb16cd4039" + }, + "expected": "0xb7404c2b88e7f6a1991dfc5294f8f78932cbbf00ebecf302a1139950672f81f9" + }, + { + "name": "worker.identity", + "function": "workerIdentity", + "input": { + "operatorId": "0x06739c78255ec573518e97ffa9d2c5e11f49d49e0c65217c77d710a558a57f21", + "workerKeyId": "0xd2435c19a90c604a8a7e0b079be17b23846efd2313821825ebe2a9af5f3f9924", + "scopeHash": "0x904e460fff2b285b3723a0701a287fd47f11a4980dd425fba7628797f7d26527" + }, + "expected": "0x356c8e272ea376e5313a0bae08f9f154355c1658a526f557bd950bbb74c7065a" + }, + { + "name": "verifier.identity", + "function": "verifierIdentity", + "input": { + "operatorId": "0x06739c78255ec573518e97ffa9d2c5e11f49d49e0c65217c77d710a558a57f21", + "verifierKeyId": "0x9bc6e07b6ca6b1f21697ed511d211b82186960d193036d5817f1a8a46a6daff5", + "verifierSetRoot": "0x96130fe314e14b7f7d4347094f6b5ec4338b1f7730bf505e59fd3e731753ff8b" + }, + "expected": "0xaf8489608eca0cfb4b25880355fd5d36ad96df15005906191c21bd6d58f6d9c6" + }, + { + "name": "devnet.blockHash", + "function": "devnetBlockHash", + "input": { + "chainId": 8453, + "blockNumber": 12345678, + "parentHash": "0x0000000000000000000000000000000000000000000000000000000000000000", + "stateRoot": "0x8a693dbe4f75fa223c6891bebaefa82f8135bff2597a17687945949a0b6082e4", + "timestamp": 1778640000 + }, + "expected": "0x90cb545229eb785f0583ca5abafb3da199a5c68a1aa8ef140e169c024ca48e54" + } + ] +} diff --git a/crypto/package-lock.json b/crypto/package-lock.json new file mode 100644 index 00000000..9d69e1a7 --- /dev/null +++ b/crypto/package-lock.json @@ -0,0 +1,40 @@ +{ + "name": "@flowmemory/crypto", + "version": "0.0.0", + "lockfileVersion": 3, + "requires": true, + "packages": { + "": { + "name": "@flowmemory/crypto", + "version": "0.0.0", + "dependencies": { + "@noble/hashes": "2.2.0", + "@noble/secp256k1": "3.1.0" + }, + "engines": { + "node": ">=20.19.0" + } + }, + "node_modules/@noble/hashes": { + "version": "2.2.0", + "resolved": "https://registry.npmjs.org/@noble/hashes/-/hashes-2.2.0.tgz", + "integrity": "sha512-IYqDGiTXab6FniAgnSdZwgWbomxpy9FtYvLKs7wCUs2a8RkITG+DFGO1DM9cr+E3/RgADRpFjrKVaJ1z6sjtEg==", + "license": "MIT", + "engines": { + "node": ">= 20.19.0" + }, + "funding": { + "url": "https://paulmillr.com/funding/" + } + }, + "node_modules/@noble/secp256k1": { + "version": "3.1.0", + "resolved": "https://registry.npmjs.org/@noble/secp256k1/-/secp256k1-3.1.0.tgz", + "integrity": "sha512-+F7iS7tUMaNGXcc9X3PjmjvuQnXEuSjCRNzVVA2xAcKXgCaP0dHYz4SFyt4FKNHef7sOP//xihowcySSS7PK9g==", + "license": "MIT", + "funding": { + "url": "https://paulmillr.com/funding/" + } + } + } +} diff --git a/crypto/package.json b/crypto/package.json new file mode 100644 index 00000000..e1ac8fff --- /dev/null +++ b/crypto/package.json @@ -0,0 +1,27 @@ +{ + "name": "@flowmemory/crypto", + "version": "0.0.0", + "private": true, + "description": "FlowMemory crypto v0 hashes, roots, receipts, reports, and test vectors.", + "type": "module", + "main": "src/index.js", + "types": "src/index.d.ts", + "exports": { + ".": { + "types": "./src/index.d.ts", + "default": "./src/index.js" + } + }, + "scripts": { + "test": "node --test", + "vectors": "node src/cli.js", + "validate:vectors": "node src/validate-vectors.js" + }, + "dependencies": { + "@noble/hashes": "2.2.0", + "@noble/secp256k1": "3.1.0" + }, + "engines": { + "node": ">=20.19.0" + } +} diff --git a/crypto/src/attestations.js b/crypto/src/attestations.js new file mode 100644 index 00000000..09f28af2 --- /dev/null +++ b/crypto/src/attestations.js @@ -0,0 +1,117 @@ +import { TYPE_STRINGS } from "./constants.js"; +import { hexToBytes, bytesToHex } from "./encoding.js"; +import { typedHash } from "./hashes.js"; +import { eip712Digest } from "./flowpulse.js"; +import * as secp from "@noble/secp256k1"; +import { hmac } from "@noble/hashes/hmac.js"; +import { sha256 } from "@noble/hashes/sha2.js"; + +secp.hashes.sha256 = sha256; +secp.hashes.hmacSha256 = (key, ...messages) => hmac(sha256, key, secp.etc.concatBytes(...messages)); + +export function eip712DomainSeparator({ nameHash, versionHash, chainId, verifyingContract, salt }) { + return typedHash(TYPE_STRINGS.eip712Domain, [ + ["bytes32", nameHash], + ["bytes32", versionHash], + ["uint256", chainId], + ["address", verifyingContract], + ["bytes32", salt] + ]); +} + +export function workerSignatureStructHash({ + receiptHash, + workerId, + workerKeyId, + workerSequence, + expiresAtUnixMs, + artifactRoot, + nonce +}) { + return typedHash(TYPE_STRINGS.workerSignatureV0, [ + ["bytes32", receiptHash], + ["bytes32", workerId], + ["bytes32", workerKeyId], + ["uint64", workerSequence], + ["uint64", expiresAtUnixMs], + ["bytes32", artifactRoot], + ["bytes32", nonce] + ]); +} + +export function verifierSignatureStructHash({ + reportId, + verifierId, + verifierKeyId, + verifierSetRoot, + issuedAtUnixMs, + expiresAtUnixMs, + nonce +}) { + return typedHash(TYPE_STRINGS.verifierSignatureV0, [ + ["bytes32", reportId], + ["bytes32", verifierId], + ["bytes32", verifierKeyId], + ["bytes32", verifierSetRoot], + ["uint64", issuedAtUnixMs], + ["uint64", expiresAtUnixMs], + ["bytes32", nonce] + ]); +} + +export function attestationEnvelopeHash({ + subjectHash, + subjectKind, + attesterId, + attesterKeyId, + verifierSetRoot, + issuedAtUnixMs, + expiresAtUnixMs, + nonce +}) { + return typedHash(TYPE_STRINGS.attestationEnvelopeV0, [ + ["bytes32", subjectHash], + ["uint8", subjectKind], + ["bytes32", attesterId], + ["bytes32", attesterKeyId], + ["bytes32", verifierSetRoot], + ["uint64", issuedAtUnixMs], + ["uint64", expiresAtUnixMs], + ["bytes32", nonce] + ]); +} + +export const attestationDigest = attestationEnvelopeHash; + +export function workerSignaturePayload({ domainSeparator, ...payload }) { + const structHash = workerSignatureStructHash(payload); + return { + structHash, + signingDigest: eip712Digest(domainSeparator, structHash) + }; +} + +export function verifierSignaturePayload({ domainSeparator, ...payload }) { + const structHash = verifierSignatureStructHash(payload); + return { + structHash, + signingDigest: eip712Digest(domainSeparator, structHash) + }; +} + +export function publicKeyFromPrivateKey(privateKeyHex) { + return bytesToHex(secp.getPublicKey(hexToBytes(privateKeyHex, 32))); +} + +export async function signDigest({ digest, privateKey }) { + const signature = await secp.sign(hexToBytes(digest, 32), hexToBytes(privateKey, 32), { + prehash: false + }); + return bytesToHex(signature); +} + +export function verifyDigest({ digest, signature, publicKey }) { + return secp.verify(hexToBytes(signature, 64), hexToBytes(digest, 32), hexToBytes(publicKey), { + prehash: false + }); +} diff --git a/crypto/src/cli.js b/crypto/src/cli.js new file mode 100644 index 00000000..c53b7404 --- /dev/null +++ b/crypto/src/cli.js @@ -0,0 +1,54 @@ +#!/usr/bin/env node +import { readFileSync } from "node:fs"; +import { dirname, resolve } from "node:path"; +import { fileURLToPath } from "node:url"; +import { + attestationEnvelopeHash, + flowPulseEventArgsHash, + flowPulseObservationId, + receiptHash, + verifierReportHash +} from "./index.js"; + +const root = resolve(dirname(fileURLToPath(import.meta.url)), ".."); + +function readJson(path) { + return JSON.parse(readFileSync(resolve(root, path), "utf8")); +} + +const flowPulse = readJson("fixtures/sample-flowpulse.json"); +const observation = readJson("fixtures/sample-observation.json"); +const report = readJson("fixtures/sample-report.json"); + +const observationId = flowPulseObservationId(observation.input); +const eventArgsHash = flowPulseEventArgsHash(flowPulse.input); +const computedReceiptHash = receiptHash({ + ...observation.receipt, + observationId, + eventArgsHash +}); +const reportId = verifierReportHash({ + ...report.input, + observationId, + receiptHash: computedReceiptHash +}); +const computedAttestationEnvelopeHash = attestationEnvelopeHash({ + ...report.attestationEnvelope.input, + subjectHash: reportId +}); + +console.log( + JSON.stringify( + { + pulseId: flowPulse.expected.pulseId, + observationId, + eventArgsHash, + receiptHash: computedReceiptHash, + artifactRoot: observation.receipt.artifactRoot, + reportId, + attestationEnvelopeHash: computedAttestationEnvelopeHash + }, + null, + 2 + ) +); diff --git a/crypto/src/constants.js b/crypto/src/constants.js new file mode 100644 index 00000000..775d8f4d --- /dev/null +++ b/crypto/src/constants.js @@ -0,0 +1,75 @@ +export const ZERO_BYTES32 = "0x0000000000000000000000000000000000000000000000000000000000000000"; + +export const FLOWPULSE_SCHEMA_ID_PREIMAGE = "flowmemory.flowpulse.v0"; +export const FLOWPULSE_EVENT_SIGNATURE = + "FlowPulse(bytes32,bytes32,address,uint8,bytes32,bytes32,bytes32,uint64,uint64,string)"; + +export const TYPE_STRINGS = Object.freeze({ + indexerCursorV0: + "FlowMemoryIndexerCursorV0(bytes32 sourceId,bytes32 streamId,uint64 sequence,bytes32 observationId,bytes32 previousCursorId)", + flowPulseObservationV0: + "FlowPulseObservationV0(uint256 chainId,address emittingContract,uint64 blockNumber,bytes32 blockHash,bytes32 txHash,uint32 transactionIndex,uint32 logIndex,bytes32 eventSignature,bytes32 pulseId,bytes32 rootfieldId)", + flowPulseEventArgsV0: + "FlowPulseEventArgsV0(bytes32 pulseId,bytes32 rootfieldId,address actor,uint8 pulseType,bytes32 subject,bytes32 commitment,bytes32 parentPulseId,uint64 sequence,uint64 occurredAt,bytes32 uriHash)", + flowPulseReceiptV0: + "FlowPulseReceiptV0(bytes32 observationId,bytes32 eventArgsHash,bytes32 artifactRoot,bytes32 storageReceiptCommitment,bytes32 evidenceRoot,uint16 receiptVersion)", + artifactRootV0: + "FlowMemoryArtifactRootV0(bytes32 schemeId,bytes32 manifestHash,bytes32 contentMerkleRoot,uint64 byteLength,uint32 chunkSize,bytes32 mediaTypeHash,bytes32 metadataHash)", + merkleLeafV0: + "FlowMemoryMerkleLeafV0(uint64 index,uint64 offset,uint32 length,bytes32 chunkHash)", + merkleInternalNodeV0: + "FlowMemoryMerkleInternalNodeV0(bytes32 leftHash,bytes32 rightHash)", + storageReceiptCommitmentV0: + "FlowMemoryStorageReceiptCommitmentV0(bytes32 artifactRoot,bytes32 providerId,bytes32 locationCommitment,bytes32 retentionPolicyHash,bytes32 encryptionCommitment,bytes32 availabilitySampleRoot,uint64 issuedAtUnixMs,uint64 expiresAtUnixMs,bytes32 nonce)", + workerSignatureV0: + "FlowMemoryWorkerSignatureV0(bytes32 receiptHash,bytes32 workerId,bytes32 workerKeyId,uint64 workerSequence,uint64 expiresAtUnixMs,bytes32 artifactRoot,bytes32 nonce)", + verifierReportV0: + "FlowMemoryVerifierReportV0(bytes32 reportSchemaHash,bytes32 observationId,bytes32 receiptHash,bytes32 verifierId,bytes32 verifierSetRoot,uint8 status,bytes32 checksRoot,uint64 finalizedBlockNumber,bytes32 finalizedBlockHash,uint16 reportVersion)", + verifierSignatureV0: + "FlowMemoryVerifierSignatureV0(bytes32 reportId,bytes32 verifierId,bytes32 verifierKeyId,bytes32 verifierSetRoot,uint64 issuedAtUnixMs,uint64 expiresAtUnixMs,bytes32 nonce)", + attestationEnvelopeV0: + "FlowMemoryAttestationEnvelopeV0(bytes32 subjectHash,uint8 subjectKind,bytes32 attesterId,bytes32 attesterKeyId,bytes32 verifierSetRoot,uint64 issuedAtUnixMs,uint64 expiresAtUnixMs,bytes32 nonce)", + rootfieldNamespaceV0: + "FlowMemoryRootfieldNamespaceV0(uint256 chainId,address registry,bytes32 rootfieldId,bytes32 schemaHash)", + rootCommitmentV0: + "FlowMemoryRootCommitmentV0(bytes32 rootfieldId,bytes32 root,bytes32 artifactCommitment,bytes32 parentPulseId,uint64 sequence)", + workReceiptV0: + "FlowMemoryWorkReceiptV0(bytes32 observationId,bytes32 receiptHash,bytes32 workerId,uint64 workerSequence,bytes32 nonce)", + workerIdentityV0: + "FlowMemoryWorkerIdentityV0(bytes32 operatorId,bytes32 workerKeyId,bytes32 scopeHash)", + verifierIdentityV0: + "FlowMemoryVerifierIdentityV0(bytes32 operatorId,bytes32 verifierKeyId,bytes32 verifierSetRoot)", + devnetBlockHashV0: + "FlowMemoryDevnetBlockV0(uint256 chainId,uint64 blockNumber,bytes32 parentHash,bytes32 stateRoot,uint64 timestamp)", + eip712Domain: + "EIP712Domain(string name,string version,uint256 chainId,address verifyingContract,bytes32 salt)" +}); + +export const DOMAIN_STRINGS = Object.freeze({ + flowPulseObservationId: "flowmemory.v0.flowpulse.observation-id", + indexerCursorId: "flowmemory.v0.indexer.cursor-id", + verifierReportDigest: "flowmemory.v0.verifier.report-digest", + verifierAttestationEnvelope: "flowmemory.v0.verifier.attestation-envelope", + rootfieldNamespaceId: "flowmemory.v0.rootfield.namespace-id", + rootCommitment: "flowmemory.v0.root.commitment", + artifactCommitment: "flowmemory.v0.artifact.commitment", + workReceiptId: "flowmemory.v0.work.receipt-id", + workerIdentity: "flowmemory.v0.worker.identity", + verifierIdentity: "flowmemory.v0.verifier.identity", + merkleLeaf: "flowmemory.v0.merkle.leaf", + merkleInternalNode: "flowmemory.v0.merkle.internal-node", + devnetBlockHash: "flowmemory.v0.devnet.block-hash" +}); + +export const MERKLE_SCHEME_V0 = "FM-MERKLE-KECCAK256-BINARY-V0"; + +export const VERIFIER_STATUSES = Object.freeze({ + reserved: 0, + observed: 1, + verified: 2, + unresolved: 3, + unsupported: 4, + failed: 5, + reorged: 6, + superseded: 7 +}); diff --git a/crypto/src/domains.js b/crypto/src/domains.js new file mode 100644 index 00000000..68c420dc --- /dev/null +++ b/crypto/src/domains.js @@ -0,0 +1,69 @@ +import { TYPE_STRINGS } from "./constants.js"; +import { typedHash } from "./hashes.js"; + +export function indexerCursorId({ sourceId, streamId, sequence, observationId, previousCursorId }) { + return typedHash(TYPE_STRINGS.indexerCursorV0, [ + ["bytes32", sourceId], + ["bytes32", streamId], + ["uint64", sequence], + ["bytes32", observationId], + ["bytes32", previousCursorId] + ]); +} + +export const cursorId = indexerCursorId; + +export function rootfieldNamespaceId({ chainId, registry, rootfieldId, schemaHash }) { + return typedHash(TYPE_STRINGS.rootfieldNamespaceV0, [ + ["uint256", chainId], + ["address", registry], + ["bytes32", rootfieldId], + ["bytes32", schemaHash] + ]); +} + +export function rootCommitment({ rootfieldId, root, artifactCommitment, parentPulseId, sequence }) { + return typedHash(TYPE_STRINGS.rootCommitmentV0, [ + ["bytes32", rootfieldId], + ["bytes32", root], + ["bytes32", artifactCommitment], + ["bytes32", parentPulseId], + ["uint64", sequence] + ]); +} + +export function workReceiptId({ observationId, receiptHash, workerId, workerSequence, nonce }) { + return typedHash(TYPE_STRINGS.workReceiptV0, [ + ["bytes32", observationId], + ["bytes32", receiptHash], + ["bytes32", workerId], + ["uint64", workerSequence], + ["bytes32", nonce] + ]); +} + +export function workerIdentity({ operatorId, workerKeyId, scopeHash }) { + return typedHash(TYPE_STRINGS.workerIdentityV0, [ + ["bytes32", operatorId], + ["bytes32", workerKeyId], + ["bytes32", scopeHash] + ]); +} + +export function verifierIdentity({ operatorId, verifierKeyId, verifierSetRoot }) { + return typedHash(TYPE_STRINGS.verifierIdentityV0, [ + ["bytes32", operatorId], + ["bytes32", verifierKeyId], + ["bytes32", verifierSetRoot] + ]); +} + +export function devnetBlockHash({ chainId, blockNumber, parentHash, stateRoot, timestamp }) { + return typedHash(TYPE_STRINGS.devnetBlockHashV0, [ + ["uint256", chainId], + ["uint64", blockNumber], + ["bytes32", parentHash], + ["bytes32", stateRoot], + ["uint64", timestamp] + ]); +} diff --git a/crypto/src/encoding.js b/crypto/src/encoding.js new file mode 100644 index 00000000..195f2cce --- /dev/null +++ b/crypto/src/encoding.js @@ -0,0 +1,127 @@ +export function strip0x(value) { + if (typeof value !== "string") { + throw new TypeError("hex value must be a string"); + } + return value.startsWith("0x") ? value.slice(2) : value; +} + +export function bytesToHex(bytes) { + return `0x${Buffer.from(bytes).toString("hex")}`; +} + +export function hexToBytes(value, expectedLength) { + const raw = strip0x(value); + if (raw.length % 2 !== 0) { + throw new Error(`invalid hex length for ${value}`); + } + if (!/^[0-9a-fA-F]*$/.test(raw)) { + throw new Error(`invalid hex characters for ${value}`); + } + const bytes = Uint8Array.from(Buffer.from(raw, "hex")); + if (expectedLength !== undefined && bytes.length !== expectedLength) { + throw new Error(`expected ${expectedLength} bytes, got ${bytes.length}: ${value}`); + } + return bytes; +} + +export function normalizeHex(value, expectedLength) { + return bytesToHex(hexToBytes(value, expectedLength)); +} + +export function utf8Bytes(value) { + return Uint8Array.from(Buffer.from(String(value), "utf8")); +} + +export function concatBytes(...parts) { + const length = parts.reduce((total, part) => total + part.length, 0); + const out = new Uint8Array(length); + let offset = 0; + for (const part of parts) { + out.set(part, offset); + offset += part.length; + } + return out; +} + +export function uintToWord(value) { + const n = BigInt(value); + if (n < 0n) { + throw new Error(`uint cannot be negative: ${value}`); + } + const out = new Uint8Array(32); + let remaining = n; + for (let i = 31; i >= 0; i -= 1) { + out[i] = Number(remaining & 0xffn); + remaining >>= 8n; + } + if (remaining !== 0n) { + throw new Error(`uint does not fit in 256 bits: ${value}`); + } + return out; +} + +export function uintBe(value, byteLength) { + const n = BigInt(value); + if (n < 0n) { + throw new Error(`uint cannot be negative: ${value}`); + } + const out = new Uint8Array(byteLength); + let remaining = n; + for (let i = byteLength - 1; i >= 0; i -= 1) { + out[i] = Number(remaining & 0xffn); + remaining >>= 8n; + } + if (remaining !== 0n) { + throw new Error(`uint does not fit in ${byteLength} bytes: ${value}`); + } + return out; +} + +export function addressToWord(value) { + const address = hexToBytes(value, 20); + return concatBytes(new Uint8Array(12), address); +} + +export function bytes32ToWord(value) { + return hexToBytes(value, 32); +} + +export function abiEncodeStatic(fields) { + const encoded = fields.map(([type, value]) => { + if (type.startsWith("uint")) { + return uintToWord(value); + } + if (type === "address") { + return addressToWord(value); + } + if (type === "bytes32") { + return bytes32ToWord(value); + } + throw new Error(`unsupported static ABI field type: ${type}`); + }); + return concatBytes(...encoded); +} + +export function canonicalJson(value) { + return JSON.stringify(sortCanonical(value)); +} + +function sortCanonical(value) { + if (value === null || typeof value !== "object") { + if (typeof value === "number" && !Number.isFinite(value)) { + throw new Error("canonical JSON cannot encode non-finite numbers"); + } + if (typeof value === "string" && /^0x[0-9a-fA-F]*$/.test(value)) { + return value.toLowerCase(); + } + return value; + } + if (Array.isArray(value)) { + return value.map(sortCanonical); + } + return Object.fromEntries( + Object.keys(value) + .sort() + .map((key) => [key, sortCanonical(value[key])]) + ); +} diff --git a/crypto/src/flowpulse.js b/crypto/src/flowpulse.js new file mode 100644 index 00000000..58846e91 --- /dev/null +++ b/crypto/src/flowpulse.js @@ -0,0 +1,146 @@ +import { + FLOWPULSE_EVENT_SIGNATURE, + FLOWPULSE_SCHEMA_ID_PREIMAGE, + TYPE_STRINGS +} from "./constants.js"; +import { abiEncodeStatic, concatBytes, hexToBytes } from "./encoding.js"; +import { keccak256Hex, keccakUtf8, typedHash } from "./hashes.js"; + +export function flowPulseSchemaId() { + return keccakUtf8(FLOWPULSE_SCHEMA_ID_PREIMAGE); +} + +export function flowPulseEventSignature() { + return keccakUtf8(FLOWPULSE_EVENT_SIGNATURE); +} + +export function contractPulseId({ + chainId, + emittingContract, + rootfieldId, + actor, + pulseType, + subject, + commitment, + parentPulseId, + sequence +}) { + return keccak256Hex( + abiEncodeStatic([ + ["bytes32", flowPulseSchemaId()], + ["uint256", chainId], + ["address", emittingContract], + ["bytes32", rootfieldId], + ["address", actor], + ["uint8", pulseType], + ["bytes32", subject], + ["bytes32", commitment], + ["bytes32", parentPulseId], + ["uint64", sequence] + ]) + ); +} + +export function flowPulseObservationId({ + chainId, + emittingContract, + blockNumber, + blockHash, + txHash, + transactionIndex, + logIndex, + eventSignature, + pulseId, + rootfieldId +}) { + return typedHash(TYPE_STRINGS.flowPulseObservationV0, [ + ["uint256", chainId], + ["address", emittingContract], + ["uint64", blockNumber], + ["bytes32", blockHash], + ["bytes32", txHash], + ["uint32", transactionIndex], + ["uint32", logIndex], + ["bytes32", eventSignature], + ["bytes32", pulseId], + ["bytes32", rootfieldId] + ]); +} + +export function flowPulseEventArgsHash({ + pulseId, + rootfieldId, + actor, + pulseType, + subject, + commitment, + parentPulseId, + sequence, + occurredAt, + uriHash +}) { + return typedHash(TYPE_STRINGS.flowPulseEventArgsV0, [ + ["bytes32", pulseId], + ["bytes32", rootfieldId], + ["address", actor], + ["uint8", pulseType], + ["bytes32", subject], + ["bytes32", commitment], + ["bytes32", parentPulseId], + ["uint64", sequence], + ["uint64", occurredAt], + ["bytes32", uriHash] + ]); +} + +export function receiptHash({ + observationId, + eventArgsHash, + artifactRoot, + storageReceiptCommitment, + evidenceRoot, + receiptVersion +}) { + return typedHash(TYPE_STRINGS.flowPulseReceiptV0, [ + ["bytes32", observationId], + ["bytes32", eventArgsHash], + ["bytes32", artifactRoot], + ["bytes32", storageReceiptCommitment], + ["bytes32", evidenceRoot], + ["uint16", receiptVersion] + ]); +} + +export function verifierReportHash({ + reportSchemaHash, + observationId, + receiptHash, + verifierId, + verifierSetRoot, + status, + checksRoot, + finalizedBlockNumber, + finalizedBlockHash, + reportVersion +}) { + return typedHash(TYPE_STRINGS.verifierReportV0, [ + ["bytes32", reportSchemaHash], + ["bytes32", observationId], + ["bytes32", receiptHash], + ["bytes32", verifierId], + ["bytes32", verifierSetRoot], + ["uint8", status], + ["bytes32", checksRoot], + ["uint64", finalizedBlockNumber], + ["bytes32", finalizedBlockHash], + ["uint16", reportVersion] + ]); +} + +export const reportDigest = verifierReportHash; + +export function eip712Digest(domainSeparator, structHash) { + return keccak256Hex( + concatBytes(Uint8Array.of(0x19, 0x01), hexToBytes(domainSeparator, 32), hexToBytes(structHash, 32)) + ); +} diff --git a/crypto/src/hashes.js b/crypto/src/hashes.js new file mode 100644 index 00000000..e8fbd611 --- /dev/null +++ b/crypto/src/hashes.js @@ -0,0 +1,36 @@ +import { keccak_256 } from "@noble/hashes/sha3.js"; +import { DOMAIN_STRINGS } from "./constants.js"; +import { abiEncodeStatic, bytesToHex, canonicalJson, concatBytes, utf8Bytes } from "./encoding.js"; + +export function keccak256Bytes(data) { + return keccak_256(data); +} + +export function keccak256Hex(data) { + return bytesToHex(keccak256Bytes(data)); +} + +export function keccakUtf8(value) { + return keccak256Hex(utf8Bytes(value)); +} + +export function canonicalJsonHash(value) { + return keccakUtf8(canonicalJson(value)); +} + +export function typeHash(typeString) { + return keccakUtf8(typeString); +} + +export function typedHash(typeString, fields) { + return keccak256Hex(abiEncodeStatic([["bytes32", typeHash(typeString)], ...fields])); +} + +export function domainSeparatedHash(domain, payloadBytes) { + return keccak256Hex(concatBytes(keccak_256(utf8Bytes(domain)), payloadBytes)); +} + +export function domainSeparator(domainName) { + const domain = DOMAIN_STRINGS[domainName] ?? domainName; + return keccakUtf8(domain); +} diff --git a/crypto/src/index.d.ts b/crypto/src/index.d.ts new file mode 100644 index 00000000..6aa666e3 --- /dev/null +++ b/crypto/src/index.d.ts @@ -0,0 +1,271 @@ +export type Hex = `0x${string}`; +export type Address = Hex; +export type Bytes32 = Hex; + +export interface FlowPulseContractInput { + chainId: number | bigint | string; + emittingContract: Address; + rootfieldId: Bytes32; + actor: Address; + pulseType: number | bigint | string; + subject: Bytes32; + commitment: Bytes32; + parentPulseId: Bytes32; + sequence: number | bigint | string; +} + +export interface FlowPulseEventArgsInput { + pulseId: Bytes32; + rootfieldId: Bytes32; + actor: Address; + pulseType: number | bigint | string; + subject: Bytes32; + commitment: Bytes32; + parentPulseId: Bytes32; + sequence: number | bigint | string; + occurredAt: number | bigint | string; + uriHash: Bytes32; +} + +export interface FlowPulseObservationInput { + chainId: number | bigint | string; + emittingContract: Address; + blockNumber: number | bigint | string; + blockHash: Bytes32; + txHash: Bytes32; + transactionIndex: number | bigint | string; + logIndex: number | bigint | string; + eventSignature: Bytes32; + pulseId: Bytes32; + rootfieldId: Bytes32; +} + +export interface ReceiptInput { + observationId: Bytes32; + eventArgsHash: Bytes32; + artifactRoot: Bytes32; + storageReceiptCommitment: Bytes32; + evidenceRoot: Bytes32; + receiptVersion: number | bigint | string; +} + +export interface StorageReceiptCommitmentInput { + artifactRoot: Bytes32; + providerId: Bytes32; + locationCommitment: Bytes32; + retentionPolicyHash: Bytes32; + encryptionCommitment: Bytes32; + availabilitySampleRoot: Bytes32; + issuedAtUnixMs: number | bigint | string; + expiresAtUnixMs: number | bigint | string; + nonce: Bytes32; +} + +export interface VerifierReportInput { + reportSchemaHash: Bytes32; + observationId: Bytes32; + receiptHash: Bytes32; + verifierId: Bytes32; + verifierSetRoot: Bytes32; + status: number | bigint | string; + checksRoot: Bytes32; + finalizedBlockNumber: number | bigint | string; + finalizedBlockHash: Bytes32; + reportVersion: number | bigint | string; +} + +export interface SignatureDomainInput { + domainSeparator: Bytes32; +} + +export interface WorkerSignatureInput { + receiptHash: Bytes32; + workerId: Bytes32; + workerKeyId: Bytes32; + workerSequence: number | bigint | string; + expiresAtUnixMs: number | bigint | string; + artifactRoot: Bytes32; + nonce: Bytes32; +} + +export interface VerifierSignatureInput { + reportId: Bytes32; + verifierId: Bytes32; + verifierKeyId: Bytes32; + verifierSetRoot: Bytes32; + issuedAtUnixMs: number | bigint | string; + expiresAtUnixMs: number | bigint | string; + nonce: Bytes32; +} + +export interface AttestationEnvelopeInput { + subjectHash: Bytes32; + subjectKind: number | bigint | string; + attesterId: Bytes32; + attesterKeyId: Bytes32; + verifierSetRoot: Bytes32; + issuedAtUnixMs: number | bigint | string; + expiresAtUnixMs: number | bigint | string; + nonce: Bytes32; +} + +export interface MerkleLeafInput { + index: number | bigint | string; + offset: number | bigint | string; + length: number | bigint | string; + chunkHash: Bytes32; +} + +export interface ArtifactInput { + chunks: Array; + chunkSize: number | bigint | string; + mediaType: string; + metadata: unknown; +} + +export interface ArtifactOutput { + schemeId: Bytes32; + manifest: unknown; + manifestHash: Bytes32; + leafHashes: Bytes32[]; + contentMerkleRoot: Bytes32; + metadataHash: Bytes32; + mediaTypeHash: Bytes32; + artifactRoot: Bytes32; +} + +export interface IndexerCursorInput { + sourceId: Bytes32; + streamId: Bytes32; + sequence: number | bigint | string; + observationId: Bytes32; + previousCursorId: Bytes32; +} + +export interface RootfieldNamespaceInput { + chainId: number | bigint | string; + registry: Address; + rootfieldId: Bytes32; + schemaHash: Bytes32; +} + +export interface RootCommitmentInput { + rootfieldId: Bytes32; + root: Bytes32; + artifactCommitment: Bytes32; + parentPulseId: Bytes32; + sequence: number | bigint | string; +} + +export interface WorkReceiptInput { + observationId: Bytes32; + receiptHash: Bytes32; + workerId: Bytes32; + workerSequence: number | bigint | string; + nonce: Bytes32; +} + +export interface WorkerIdentityInput { + operatorId: Bytes32; + workerKeyId: Bytes32; + scopeHash: Bytes32; +} + +export interface VerifierIdentityInput { + operatorId: Bytes32; + verifierKeyId: Bytes32; + verifierSetRoot: Bytes32; +} + +export interface DevnetBlockInput { + chainId: number | bigint | string; + blockNumber: number | bigint | string; + parentHash: Bytes32; + stateRoot: Bytes32; + timestamp: number | bigint | string; +} + +export const ZERO_BYTES32: Bytes32; +export const FLOWPULSE_SCHEMA_ID_PREIMAGE: string; +export const FLOWPULSE_EVENT_SIGNATURE: string; +export const TYPE_STRINGS: Readonly>; +export const DOMAIN_STRINGS: Readonly>; +export const MERKLE_SCHEME_V0: string; +export const VERIFIER_STATUSES: Readonly>; + +export function strip0x(value: string): string; +export function bytesToHex(bytes: Uint8Array): Hex; +export function hexToBytes(value: Hex | string, expectedLength?: number): Uint8Array; +export function normalizeHex(value: Hex | string, expectedLength?: number): Hex; +export function utf8Bytes(value: unknown): Uint8Array; +export function concatBytes(...parts: Uint8Array[]): Uint8Array; +export function uintToWord(value: number | bigint | string): Uint8Array; +export function uintBe(value: number | bigint | string, byteLength: number): Uint8Array; +export function addressToWord(value: Address): Uint8Array; +export function bytes32ToWord(value: Bytes32): Uint8Array; +export function abiEncodeStatic(fields: Array<[string, unknown]>): Uint8Array; +export function canonicalJson(value: unknown): string; + +export function keccak256Bytes(data: Uint8Array): Uint8Array; +export function keccak256Hex(data: Uint8Array): Hex; +export function keccakUtf8(value: unknown): Bytes32; +export function canonicalJsonHash(value: unknown): Bytes32; +export function typeHash(typeString: string): Bytes32; +export function typedHash(typeString: string, fields: Array<[string, unknown]>): Bytes32; +export function domainSeparatedHash(domain: string, payloadBytes: Uint8Array): Bytes32; +export function domainSeparator(domainName: string): Bytes32; + +export function flowPulseSchemaId(): Bytes32; +export function flowPulseEventSignature(): Bytes32; +export function contractPulseId(input: FlowPulseContractInput): Bytes32; +export function flowPulseObservationId(input: FlowPulseObservationInput): Bytes32; +export function flowPulseEventArgsHash(input: FlowPulseEventArgsInput): Bytes32; +export function receiptHash(input: ReceiptInput): Bytes32; +export function verifierReportHash(input: VerifierReportInput): Bytes32; +export const reportDigest: typeof verifierReportHash; +export function eip712Digest(domainSeparator: Bytes32, structHash: Bytes32): Bytes32; + +export function chunkHash(chunk: string | Uint8Array): Bytes32; +export function merkleLeafHash(input: MerkleLeafInput): Bytes32; +export function merkleNodeHash(leftHash: Bytes32, rightHash: Bytes32): Bytes32; +export function emptyMerkleRoot(): Bytes32; +export function merkleRoot(leafHashes: Bytes32[]): Bytes32; +export function buildArtifactManifest(input: { chunks: Array; chunkSize: number | bigint | string }): unknown; +export function artifactCommitmentHash(input: { + schemeId: Bytes32; + manifestHash: Bytes32; + contentMerkleRoot: Bytes32; + byteLength: number | bigint | string; + chunkSize: number | bigint | string; + mediaTypeHash: Bytes32; + metadataHash: Bytes32; +}): Bytes32; +export const artifactCommitment: typeof artifactCommitmentHash; +export function artifactFromChunks(input: ArtifactInput): ArtifactOutput; +export function storageReceiptCommitmentHash(input: StorageReceiptCommitmentInput): Bytes32; + +export function eip712DomainSeparator(input: { + nameHash: Bytes32; + versionHash: Bytes32; + chainId: number | bigint | string; + verifyingContract: Address; + salt: Bytes32; +}): Bytes32; +export function workerSignatureStructHash(input: WorkerSignatureInput): Bytes32; +export function verifierSignatureStructHash(input: VerifierSignatureInput): Bytes32; +export function attestationEnvelopeHash(input: AttestationEnvelopeInput): Bytes32; +export const attestationDigest: typeof attestationEnvelopeHash; +export function workerSignaturePayload(input: SignatureDomainInput & WorkerSignatureInput): { structHash: Bytes32; signingDigest: Bytes32 }; +export function verifierSignaturePayload(input: SignatureDomainInput & VerifierSignatureInput): { structHash: Bytes32; signingDigest: Bytes32 }; +export function publicKeyFromPrivateKey(privateKeyHex: Hex): Hex; +export function signDigest(input: { digest: Bytes32; privateKey: Hex }): Promise; +export function verifyDigest(input: { digest: Bytes32; signature: Hex; publicKey: Hex }): boolean; + +export function indexerCursorId(input: IndexerCursorInput): Bytes32; +export const cursorId: typeof indexerCursorId; +export function rootfieldNamespaceId(input: RootfieldNamespaceInput): Bytes32; +export function rootCommitment(input: RootCommitmentInput): Bytes32; +export function workReceiptId(input: WorkReceiptInput): Bytes32; +export function workerIdentity(input: WorkerIdentityInput): Bytes32; +export function verifierIdentity(input: VerifierIdentityInput): Bytes32; +export function devnetBlockHash(input: DevnetBlockInput): Bytes32; diff --git a/crypto/src/index.js b/crypto/src/index.js new file mode 100644 index 00000000..ad50cab2 --- /dev/null +++ b/crypto/src/index.js @@ -0,0 +1,7 @@ +export * from "./attestations.js"; +export * from "./constants.js"; +export * from "./domains.js"; +export * from "./encoding.js"; +export * from "./flowpulse.js"; +export * from "./hashes.js"; +export * from "./merkle.js"; diff --git a/crypto/src/merkle.js b/crypto/src/merkle.js new file mode 100644 index 00000000..1ab0cbc3 --- /dev/null +++ b/crypto/src/merkle.js @@ -0,0 +1,148 @@ +import { MERKLE_SCHEME_V0, TYPE_STRINGS } from "./constants.js"; +import { abiEncodeStatic, canonicalJson, utf8Bytes } from "./encoding.js"; +import { keccak256Hex, keccakUtf8, typedHash } from "./hashes.js"; + +export function chunkHash(chunk) { + const bytes = typeof chunk === "string" ? utf8Bytes(chunk) : chunk; + return keccak256Hex(bytes); +} + +export function merkleLeafHash({ index, offset, length, chunkHash: chunkHashHex }) { + return typedHash(TYPE_STRINGS.merkleLeafV0, [ + ["uint64", index], + ["uint64", offset], + ["uint32", length], + ["bytes32", chunkHashHex] + ]); +} + +export function merkleNodeHash(leftHash, rightHash) { + return typedHash(TYPE_STRINGS.merkleInternalNodeV0, [ + ["bytes32", leftHash], + ["bytes32", rightHash] + ]); +} + +export function emptyMerkleRoot() { + return keccakUtf8(`${MERKLE_SCHEME_V0}:EMPTY`); +} + +export function merkleRoot(leafHashes) { + if (leafHashes.length === 0) { + return emptyMerkleRoot(); + } + let level = [...leafHashes]; + while (level.length > 1) { + const next = []; + for (let i = 0; i < level.length; i += 2) { + if (i + 1 < level.length) { + next.push(merkleNodeHash(level[i], level[i + 1])); + } else { + next.push(level[i]); + } + } + level = next; + } + return level[0]; +} + +export function buildArtifactManifest({ chunks, chunkSize }) { + let offset = 0; + const manifestChunks = chunks.map((chunk, index) => { + const data = typeof chunk === "string" ? utf8Bytes(chunk) : chunk; + const entry = { + index, + offset, + length: data.length, + chunkHash: keccak256Hex(data) + }; + offset += data.length; + return entry; + }); + return { + scheme: MERKLE_SCHEME_V0, + version: 0, + chunkSize, + byteLength: offset, + chunks: manifestChunks + }; +} + +export function artifactCommitmentHash({ + schemeId, + manifestHash, + contentMerkleRoot, + byteLength, + chunkSize, + mediaTypeHash, + metadataHash +}) { + return typedHash(TYPE_STRINGS.artifactRootV0, [ + ["bytes32", schemeId], + ["bytes32", manifestHash], + ["bytes32", contentMerkleRoot], + ["uint64", byteLength], + ["uint32", chunkSize], + ["bytes32", mediaTypeHash], + ["bytes32", metadataHash] + ]); +} + +export const artifactCommitment = artifactCommitmentHash; + +export function artifactFromChunks({ chunks, chunkSize, mediaType, metadata }) { + const manifest = buildArtifactManifest({ chunks, chunkSize }); + const leafHashes = manifest.chunks.map((entry) => merkleLeafHash(entry)); + const contentMerkleRoot = merkleRoot(leafHashes); + const manifestHash = keccak256Hex(utf8Bytes(canonicalJson(manifest))); + const metadataHash = keccak256Hex(utf8Bytes(canonicalJson(metadata))); + const mediaTypeHash = keccakUtf8(mediaType); + const schemeId = keccakUtf8(MERKLE_SCHEME_V0); + const artifactRoot = artifactCommitmentHash({ + schemeId, + manifestHash, + contentMerkleRoot, + byteLength: manifest.byteLength, + chunkSize, + mediaTypeHash, + metadataHash + }); + + return { + schemeId, + manifest, + manifestHash, + leafHashes, + contentMerkleRoot, + metadataHash, + mediaTypeHash, + artifactRoot + }; +} + +export function storageReceiptCommitmentHash({ + artifactRoot, + providerId, + locationCommitment, + retentionPolicyHash, + encryptionCommitment, + availabilitySampleRoot, + issuedAtUnixMs, + expiresAtUnixMs, + nonce +}) { + return keccak256Hex( + abiEncodeStatic([ + ["bytes32", keccakUtf8(TYPE_STRINGS.storageReceiptCommitmentV0)], + ["bytes32", artifactRoot], + ["bytes32", providerId], + ["bytes32", locationCommitment], + ["bytes32", retentionPolicyHash], + ["bytes32", encryptionCommitment], + ["bytes32", availabilitySampleRoot], + ["uint64", issuedAtUnixMs], + ["uint64", expiresAtUnixMs], + ["bytes32", nonce] + ]) + ); +} diff --git a/crypto/src/validate-vectors.js b/crypto/src/validate-vectors.js new file mode 100644 index 00000000..b2e65358 --- /dev/null +++ b/crypto/src/validate-vectors.js @@ -0,0 +1,73 @@ +import assert from "node:assert/strict"; +import { readFileSync } from "node:fs"; +import { resolve } from "node:path"; +import { fileURLToPath } from "node:url"; + +import { + artifactFromChunks, + attestationEnvelopeHash, + canonicalJsonHash, + contractPulseId, + devnetBlockHash, + domainSeparator, + emptyMerkleRoot, + flowPulseEventArgsHash, + flowPulseObservationId, + flowPulseSchemaId, + indexerCursorId, + merkleLeafHash, + merkleRoot, + receiptHash, + rootCommitment, + rootfieldNamespaceId, + storageReceiptCommitmentHash, + verifierIdentity, + verifierReportHash, + workReceiptId, + workerIdentity +} from "./index.js"; + +const validators = Object.freeze({ + artifactFromChunks, + attestationEnvelopeHash, + canonicalJsonHash, + contractPulseId, + devnetBlockHash, + domainSeparator: ({ domainName }) => domainSeparator(domainName), + emptyMerkleRoot, + flowPulseEventArgsHash, + flowPulseObservationId, + flowPulseSchemaId, + indexerCursorId, + merkleLeafHash, + merkleRoot: ({ leaves }) => merkleRoot(leaves), + receiptHash, + rootCommitment, + rootfieldNamespaceId, + storageReceiptCommitmentHash, + verifierIdentity, + verifierReportHash, + workReceiptId, + workerIdentity +}); + +export function validateVectors(vectorPath = resolve(import.meta.dirname, "..", "fixtures", "vectors.json")) { + const fixture = JSON.parse(readFileSync(vectorPath, "utf8")); + assert.equal(fixture.schema, "flowmemory.crypto.test-vectors.v0"); + assert.equal(fixture.vectorCount, fixture.vectors.length); + + for (const vector of fixture.vectors) { + const fn = validators[vector.function]; + assert.ok(fn, `unknown vector function: ${vector.function}`); + const result = fn(vector.input); + const actual = vector.select ? result[vector.select] : result; + assert.equal(actual, vector.expected, vector.name); + } + + return fixture.vectors.length; +} + +if (fileURLToPath(import.meta.url) === resolve(process.argv[1])) { + const count = validateVectors(process.argv[2]); + console.log(`FLOWMEMORY_CRYPTO_VECTORS_OK ${count}`); +} diff --git a/crypto/test-vectors/README.md b/crypto/test-vectors/README.md new file mode 100644 index 00000000..8dc08793 --- /dev/null +++ b/crypto/test-vectors/README.md @@ -0,0 +1,11 @@ +# FlowMemory Crypto Test Vectors + +Status: draft v0. + +`flowpulse-observation-v0.json` contains the primary FlowPulse v0 observation, receipt, artifact, verifier report, worker signature digest, and verifier signature digest vector. + +`../fixtures/vectors.json` contains the package-level vector set validated by `npm run validate:vectors`. + +The vectors are not production data and contain no production secrets. The worker signature entry intentionally includes the EIP-712 domain separator, struct hash, and signing digest, but no private key or signature. + +Use these vectors to verify independent implementations in contracts, verifier services, and off-chain tooling before any schema is treated as stable. diff --git a/crypto/test-vectors/flowpulse-observation-v0.json b/crypto/test-vectors/flowpulse-observation-v0.json new file mode 100644 index 00000000..8f5493ad --- /dev/null +++ b/crypto/test-vectors/flowpulse-observation-v0.json @@ -0,0 +1,107 @@ +{ + "schema": "flowmemory.crypto.flowpulse-observation.v0.test-vector", + "status": "synthetic draft vector; contains no production secrets or signatures", + "typeHashes": { + "flowPulseObservationV0": "0x6ca347f285c3fde2f3d15a8b0b032d89618a761700050ad8fc4f4032329eb7c9", + "flowPulseEventArgsV0": "0xf718b58c02f03b6dbc4580c0606068f89e205b3fe70738f5b04f1af15b205952", + "flowPulseReceiptV0": "0x89bf73fa3c44aec8ae3aab32a121925bad27d2d3e87b4d539f85434d769cc51c", + "artifactRootV0": "0x0c697b97c2b1158bb1cbc0b216c66e56f47677f8d2faef959160569f677e09cf", + "merkleLeafV0": "0x059deed0c546ef95cc993f54ac93c6975491e80d750868f74892c08c15d0408b", + "merkleInternalNodeV0": "0x86b0d2632b2953ba4dec85e000d0e91d9b75479f7c73b8a3f4bd22caae38e7a7", + "storageReceiptCommitmentV0": "0xb379598afafc29403a8bf80964117f2206c527fd0de2cfe1dbefd7051f08e44f", + "verifierReportV0": "0xe5d936ec5a6eaad5ab6cce4a08490a70ac27041a5334e4cf5ec1692d52219663", + "verifierSignatureV0": "0x242a58238785da3ab14aa8f209367b65f06d501737797e1db5b61983026ce6a4", + "workerSignatureV0": "0x2af0b882b42502fc02e026518c2c8c96ac730f3077184983341dbe7968d5d553", + "attestationEnvelopeV0": "0xc7b87c14150b5a44f23958ba00a9146bd9831292093dc4cdc601e00624b8d9e9", + "eip712Domain": "0xd87cd6ef79d4e2b95e15ce8abf732db51ec771f1ca2edccf22a46c729ac56472" + }, + "flowPulse": { + "schemaId": "0xece59fcceac2b4dcd9e8732bb223b2b9cd7ae3626eeb79457b11a99f9bfc6fef", + "eventSignature": "0x5d07190b9ae441b4d7b16259a48424acd451492b12f5f99a29f5bfd992c13e43", + "chainId": 8453, + "emittingContract": "0x1234567890abcdef1234567890abcdef12345678", + "rootfieldId": "0x726f6f746669656c642e62657461000000000000000000000000000000000000", + "actor": "0x90f8bf6a479f320ead074411a4b0e7944ea8c9c1", + "pulseType": 1, + "subject": "0x726f6f746669656c642e62657461000000000000000000000000000000000000", + "commitment": "0xf2909ae97d47adc4ec7106ea70cbed7f582d6e6066dd058a7bb5c637e493d5f5", + "parentPulseId": "0x0000000000000000000000000000000000000000000000000000000000000000", + "sequence": 1, + "occurredAt": 1778640000, + "uri": "ipfs://metadata", + "uriHash": "0x573d5e91ce4505f0968237333e7faa29e5b3db0271bd53ca845f6d06315880d0", + "pulseId": "0x86b8325d6da0767e12097aed29aefe4820aaf4d6b7d4bb8371f1db927fda9d9d" + }, + "observation": { + "blockNumber": 12345678, + "blockHash": "0x1111111111111111111111111111111111111111111111111111111111111111", + "txHash": "0x2222222222222222222222222222222222222222222222222222222222222222", + "transactionIndex": 7, + "logIndex": 3, + "observationId": "0xd80d0a3b317ceae266c9b7983c5a9376529f457a01469c96d8d3fd5a6c2d8a3f" + }, + "receipt": { + "eventArgsHash": "0xd84ede2187c11d8927a683eb4a8aea41e6266e4feb0cb10eb3aeb43b857a3ed2", + "artifactRoot": "0xff501ac63f870de597cdc2a28dad8aeae3b52c5f1e2a658b1ea37c440b76f644", + "storageReceiptCommitment": "0xb341bddb79e03b03c0a6acfcc4d01d393a84d5187f02fea2fd1761c09f345da7", + "evidenceRoot": "0xe8e37a51ab4052012bd245722091179b164d04c550d736d0b95f81da8d7b10de", + "receiptVersion": 0, + "receiptHash": "0xca2ebca63e004ff4b0ca9766acbb2862b45059a480d911b67dbc25e937c2e733" + }, + "artifact": { + "schemeId": "0x138660b5c871af72b8885b6fa35a598c8194d5224c9d473c67cb462ec7255491", + "contentMerkleRoot": "0x71d29e65fd4f25301225dcaabeb40fddabac5cbb327d4992c21a591b05eebca2", + "manifestCanonicalJson": "{\"byteLength\":14,\"chunkSize\":5,\"chunks\":[{\"chunkHash\":\"0x6dfc21ac0c8c2db036305d8bc6f887630d35e156f37d5a7e2275bc05bc004846\",\"index\":0,\"length\":5,\"offset\":0},{\"chunkHash\":\"0x9cfe0dc181c89029f2b16763070c20922901f0c18e094710477b8fecf8b78476\",\"index\":1,\"length\":4,\"offset\":5},{\"chunkHash\":\"0x8ec2a252e6833332e4d11a94054bb27c27c7220a2efddc664860650b61f98fc2\",\"index\":2,\"length\":5,\"offset\":9}],\"scheme\":\"FM-MERKLE-KECCAK256-BINARY-V0\",\"version\":0}", + "manifestHash": "0x8615b8096a0d4518a3fb74c4b08e231e955f58ac1bb934da88fbbe686553522f", + "metadataCanonicalJson": "{\"name\":\"flowmemory-test-artifact\",\"purpose\":\"flowpulse-observation-vector\"}", + "metadataHash": "0xa122871f2df16a7659cb1bd8f03f6bd5066d418310c8d611920de0bf835c4d7b", + "mediaTypeHash": "0x3cc4e82899630f0a4f171330a848e714845dd1eb55ac28123613ff7fcba91ccc", + "artifactRoot": "0xff501ac63f870de597cdc2a28dad8aeae3b52c5f1e2a658b1ea37c440b76f644" + }, + "verifierReport": { + "reportSchemaHash": "0x487a3a70cf72c079fa8645f8d732cbe286f7f3aa0abd654f212ac1c1911a20ae", + "verifierId": "0xaf0bdaa3ef421cfa8494019fb436baabcfdc65b55cf858c2d605a348c8c0aa48", + "verifierSetRoot": "0x96130fe314e14b7f7d4347094f6b5ec4338b1f7730bf505e59fd3e731753ff8b", + "status": 2, + "statusName": "verified", + "checksCanonicalJson": "{\"artifactRootMatchesManifest\":true,\"commitmentMatchesEvent\":true,\"cursorFromReceiptLog\":true,\"storageCommitmentWellFormed\":true,\"workerSignatureChecked\":false}", + "checksRoot": "0x48468783cf12adfeba9be8e0a5e250ab04b19d5034f7e1996610cf05f4fcef83", + "finalizedBlockNumber": 12345742, + "finalizedBlockHash": "0x1111111111111111111111111111111111111111111111111111111111111111", + "reportVersion": 0, + "reportId": "0x1b1c2940d6e83ee78a7e0a8285e4ce2530da1ce7964817806e61520a2e767355" + }, + "verifierSignature": { + "verifierKeyId": "0x9bc6e07b6ca6b1f21697ed511d211b82186960d193036d5817f1a8a46a6daff5", + "issuedAtUnixMs": 1767225660000, + "expiresAtUnixMs": 1769817600000, + "nonce": "0xb50ddd6d6d2d9d3dc8310462bad743d6383de268350d75b6fa84d59c74687b93", + "eip712DomainSeparator": "0x199a93f8dae72a137df50c20b442f87315f4d7bf6b39cca9d62e07c48c58106a", + "structHash": "0x35b8f9982c297652622fd66fe931d9669db4847b2056cd126d92e5bd76c93315", + "signingDigest": "0x08334f56b2ac5a7c11bf33154907d37a05253f4d3324999e820ca166c0a39b1e", + "signature": null + }, + "workerSignature": { + "workerId": "0x7803fe537f4b6e4ddd47f00b97a87c06aad1e42e22b83fdb522268e393319598", + "workerKeyId": "0xd2435c19a90c604a8a7e0b079be17b23846efd2313821825ebe2a9af5f3f9924", + "workerSequence": 1, + "expiresAtUnixMs": 1769817600000, + "artifactRoot": "0xff501ac63f870de597cdc2a28dad8aeae3b52c5f1e2a658b1ea37c440b76f644", + "nonce": "0x3574313751905cc45384e0a3f62258d38e9c64735e29bcee180459bb16cd4039", + "eip712DomainSeparator": "0xb1bd381ce93a8b36ea7e8389688cd847207acef42affdaa23743079f5614947b", + "structHash": "0x726e78bb0647bd8604d9f0a84ce2cd591039515dbbd1dc1bd18977e962643bee", + "signingDigest": "0x02a43a42d3c182a55b817835e1cdbca1238a4f86cb12e92063b710d38d85fbe3", + "signature": null + }, + "attestationEnvelope": { + "subjectHash": "0x1b1c2940d6e83ee78a7e0a8285e4ce2530da1ce7964817806e61520a2e767355", + "subjectKind": 2, + "attesterId": "0xaf0bdaa3ef421cfa8494019fb436baabcfdc65b55cf858c2d605a348c8c0aa48", + "attesterKeyId": "0x9bc6e07b6ca6b1f21697ed511d211b82186960d193036d5817f1a8a46a6daff5", + "verifierSetRoot": "0x96130fe314e14b7f7d4347094f6b5ec4338b1f7730bf505e59fd3e731753ff8b", + "issuedAtUnixMs": 1767225660000, + "expiresAtUnixMs": 1769817600000, + "nonce": "0xb50ddd6d6d2d9d3dc8310462bad743d6383de268350d75b6fa84d59c74687b93", + "attestationEnvelopeHash": "0x3e139c10ff22aea00c4442698f2d8650ba85f811c723cb7e4f28094d833fea80" + } +} diff --git a/crypto/test/crypto.test.js b/crypto/test/crypto.test.js new file mode 100644 index 00000000..13b9ec2f --- /dev/null +++ b/crypto/test/crypto.test.js @@ -0,0 +1,266 @@ +import assert from "node:assert/strict"; +import { readFileSync } from "node:fs"; +import { resolve } from "node:path"; +import test from "node:test"; + +import { + artifactFromChunks, + attestationEnvelopeHash, + canonicalJsonHash, + canonicalJson, + contractPulseId, + cursorId, + devnetBlockHash, + domainSeparator, + eip712DomainSeparator, + emptyMerkleRoot, + flowPulseEventArgsHash, + flowPulseEventSignature, + flowPulseObservationId, + flowPulseSchemaId, + indexerCursorId, + keccakUtf8, + merkleLeafHash, + merkleRoot, + normalizeHex, + publicKeyFromPrivateKey, + receiptHash, + rootCommitment, + rootfieldNamespaceId, + signDigest, + storageReceiptCommitmentHash, + verifierIdentity, + verifierReportHash, + verifierSignaturePayload, + verifyDigest, + workReceiptId, + workerIdentity, + workerSignaturePayload +} from "../src/index.js"; +import { validateVectors } from "../src/validate-vectors.js"; + +const root = resolve(import.meta.dirname, ".."); + +function fixture(name) { + return JSON.parse(readFileSync(resolve(root, "fixtures", name), "utf8")); +} + +const flowPulse = fixture("sample-flowpulse.json"); +const observation = fixture("sample-observation.json"); +const report = fixture("sample-report.json"); + +test("canonicalJson sorts object keys recursively", () => { + assert.equal(canonicalJson({ b: 2, a: { d: 4, c: 3 } }), '{"a":{"c":3,"d":4},"b":2}'); +}); + +test("canonicalJson normalizes hex case before hashing", () => { + const left = { z: "0xABCDEF", a: { d: 4, c: "0xDEADBEEF" } }; + const right = { a: { c: "0xdeadbeef", d: 4 }, z: "0xabcdef" }; + assert.equal(canonicalJsonHash(left), canonicalJsonHash(right)); + assert.equal(normalizeHex("0xABCDEF"), "0xabcdef"); + assert.throws(() => normalizeHex("0xzz"), /invalid hex characters/); +}); + +test("exports named domain separators and cursor/root identity helpers", () => { + const artifact = artifactFromChunks(observation.artifact); + const cursorInput = { + sourceId: keccakUtf8("indexer:base-flowpulse"), + streamId: keccakUtf8("flowpulse:rootfield.beta"), + sequence: 1, + observationId: observation.expected.observationId, + previousCursorId: "0x0000000000000000000000000000000000000000000000000000000000000000" + }; + const namespaceInput = { + chainId: observation.input.chainId, + registry: "0x0000000000000000000000000000000000000000", + rootfieldId: flowPulse.input.rootfieldId, + schemaHash: keccakUtf8("flowmemory.rootfield.beta.v0") + }; + const workReceiptInput = { + observationId: observation.expected.observationId, + receiptHash: observation.expected.receiptHash, + workerId: report.workerSignature.input.workerId, + workerSequence: report.workerSignature.input.workerSequence, + nonce: report.workerSignature.input.nonce + }; + const operatorId = keccakUtf8("operator:flowmemory-labs-devnet"); + + assert.equal( + domainSeparator("flowPulseObservationId"), + "0x7a42d0c550a1e18a737aa6627ab6a76a6524750e0ee78d73a3b489f662e82474" + ); + assert.equal(cursorId(cursorInput), indexerCursorId(cursorInput)); + assert.equal( + rootfieldNamespaceId(namespaceInput), + "0x6da665da25815ab5c3ee446af87a6883ad577c7a218c2f3140e3bd171548a806" + ); + assert.equal( + rootCommitment({ + rootfieldId: flowPulse.input.rootfieldId, + root: artifact.contentMerkleRoot, + artifactCommitment: artifact.artifactRoot, + parentPulseId: flowPulse.input.parentPulseId, + sequence: flowPulse.input.sequence + }), + "0xfa69bad84d06fa38d6928c3d8e50e926c2ef5ec0ed446858f350b49d75532e0b" + ); + assert.equal(workReceiptId(workReceiptInput), "0xb7404c2b88e7f6a1991dfc5294f8f78932cbbf00ebecf302a1139950672f81f9"); + assert.equal( + workerIdentity({ + operatorId, + workerKeyId: report.workerSignature.input.workerKeyId, + scopeHash: keccakUtf8("scope:base:rootfield.beta") + }), + "0x356c8e272ea376e5313a0bae08f9f154355c1658a526f557bd950bbb74c7065a" + ); + assert.equal( + verifierIdentity({ + operatorId, + verifierKeyId: report.verifierSignature.input.verifierKeyId, + verifierSetRoot: report.verifierSignature.input.verifierSetRoot + }), + "0xaf8489608eca0cfb4b25880355fd5d36ad96df15005906191c21bd6d58f6d9c6" + ); + assert.equal( + devnetBlockHash({ + chainId: observation.input.chainId, + blockNumber: observation.input.blockNumber, + parentHash: "0x0000000000000000000000000000000000000000000000000000000000000000", + stateRoot: keccakUtf8("state:flowmemory:devnet"), + timestamp: flowPulse.input.occurredAt + }), + "0x90cb545229eb785f0583ca5abafb3da199a5c68a1aa8ef140e169c024ca48e54" + ); +}); + +test("computes FlowPulse schema id, event signature, pulse id, and event args hash", () => { + assert.equal(flowPulseSchemaId(), flowPulse.expected.schemaId); + assert.equal(flowPulseEventSignature(), flowPulse.expected.eventSignature); + assert.equal(contractPulseId(flowPulse.contractInput), flowPulse.expected.pulseId); + assert.equal(flowPulseEventArgsHash(flowPulse.input), flowPulse.expected.eventArgsHash); +}); + +test("computes FlowPulse observation id and receipt hash", () => { + const observationId = flowPulseObservationId(observation.input); + const eventArgsHash = flowPulseEventArgsHash(flowPulse.input); + assert.equal(observationId, observation.expected.observationId); + assert.equal( + receiptHash({ ...observation.receipt, observationId, eventArgsHash }), + observation.expected.receiptHash + ); +}); + +test("computes artifact commitment, Merkle root, and storage receipt commitment", () => { + const artifact = artifactFromChunks(observation.artifact); + assert.equal(artifact.contentMerkleRoot, observation.expected.contentMerkleRoot); + assert.equal(artifact.artifactRoot, observation.expected.artifactRoot); + assert.equal(merkleLeafHash(artifact.manifest.chunks[0]), artifact.leafHashes[0]); + assert.equal(merkleRoot(artifact.leafHashes), observation.expected.contentMerkleRoot); + assert.equal(emptyMerkleRoot(), "0xd696a744928cde1db971775966b90e254e54e2cc4a8952b099f9db5ef7bf3434"); + assert.equal( + storageReceiptCommitmentHash(observation.storage), + observation.expected.storageReceiptCommitment + ); +}); + +test("computes deterministic verifier report hash", () => { + assert.equal(verifierReportHash(report.input), report.expected.reportId); +}); + +test("computes EIP-712 worker and verifier signature payloads", () => { + const versionHash = keccakUtf8("0"); + const workerDomainSeparator = eip712DomainSeparator({ + nameHash: keccakUtf8("FlowMemory Worker"), + versionHash, + chainId: report.eip712.chainId, + verifyingContract: report.eip712.verifyingContract, + salt: report.eip712.deploymentId + }); + const verifierDomainSeparator = eip712DomainSeparator({ + nameHash: keccakUtf8("FlowMemory Verifier"), + versionHash, + chainId: report.eip712.chainId, + verifyingContract: report.eip712.verifyingContract, + salt: report.eip712.deploymentId + }); + + assert.equal(workerDomainSeparator, report.workerSignature.expected.domainSeparator); + assert.equal(verifierDomainSeparator, report.verifierSignature.expected.domainSeparator); + assert.deepEqual( + workerSignaturePayload({ + domainSeparator: workerDomainSeparator, + ...report.workerSignature.input + }), + { + structHash: report.workerSignature.expected.structHash, + signingDigest: report.workerSignature.expected.signingDigest + } + ); + assert.deepEqual( + verifierSignaturePayload({ + domainSeparator: verifierDomainSeparator, + ...report.verifierSignature.input + }), + { + structHash: report.verifierSignature.expected.structHash, + signingDigest: report.verifierSignature.expected.signingDigest + } + ); +}); + +test("computes generic attestation envelope hash", () => { + assert.equal( + attestationEnvelopeHash(report.attestationEnvelope.input), + report.attestationEnvelope.expected.attestationEnvelopeHash + ); +}); + +test("observation id changes when reorg-sensitive block hash changes", () => { + const changed = flowPulseObservationId({ + ...observation.input, + blockHash: "0x3333333333333333333333333333333333333333333333333333333333333333" + }); + assert.notEqual(changed, observation.expected.observationId); +}); + +test("receipt-adjacent fields fail closed when changed", () => { + const artifact = artifactFromChunks(observation.artifact); + const changedLog = flowPulseObservationId({ ...observation.input, logIndex: 4 }); + const changedUri = flowPulseEventArgsHash({ + ...flowPulse.input, + uriHash: keccakUtf8("ipfs://different-metadata") + }); + const swappedMerkleRoot = merkleRoot([ + artifact.leafHashes[1], + artifact.leafHashes[0], + artifact.leafHashes[2] + ]); + const wrongVerifierSet = verifierSignaturePayload({ + domainSeparator: report.verifierSignature.expected.domainSeparator, + ...report.verifierSignature.input, + verifierSetRoot: keccakUtf8("wrong-verifier-set") + }); + + assert.notEqual(changedLog, observation.expected.observationId); + assert.notEqual(changedUri, flowPulse.expected.eventArgsHash); + assert.notEqual(swappedMerkleRoot, observation.expected.contentMerkleRoot); + assert.notEqual(wrongVerifierSet.signingDigest, report.verifierSignature.expected.signingDigest); +}); + +test("validates all published crypto test vectors", () => { + assert.equal(validateVectors(), 21); +}); + +test("signs and verifies verifier digests with local test keys only", async () => { + const privateKey = "0x0000000000000000000000000000000000000000000000000000000000000001"; + const wrongPrivateKey = "0x0000000000000000000000000000000000000000000000000000000000000002"; + const publicKey = publicKeyFromPrivateKey(privateKey); + const wrongPublicKey = publicKeyFromPrivateKey(wrongPrivateKey); + const digest = report.verifierSignature.expected.signingDigest; + const wrongDigest = report.workerSignature.expected.signingDigest; + + const signature = await signDigest({ digest, privateKey }); + assert.equal(verifyDigest({ digest, signature, publicKey }), true); + assert.equal(verifyDigest({ digest, signature, publicKey: wrongPublicKey }), false); + assert.equal(verifyDigest({ digest: wrongDigest, signature, publicKey }), false); +}); diff --git a/crypto/validate_test_vectors.py b/crypto/validate_test_vectors.py new file mode 100644 index 00000000..d17519de --- /dev/null +++ b/crypto/validate_test_vectors.py @@ -0,0 +1,249 @@ +"""Validate FlowMemory crypto v0 test vectors. + +This script is intentionally small and offline-only. It verifies the published +FlowPulse observation vector without reading secrets, RPC endpoints, or network +state. +""" + +from __future__ import annotations + +import json +from pathlib import Path +from typing import Iterable, Tuple + +try: + from Crypto.Hash import keccak +except ImportError as exc: # pragma: no cover - dependency guard for local use + raise SystemExit( + "Missing pycryptodome. Install a Keccak-256 provider before validating " + "FlowMemory vectors." + ) from exc + + +ROOT = Path(__file__).resolve().parent +VECTOR_PATH = ROOT / "test-vectors" / "flowpulse-observation-v0.json" + + +def keccak256(data: bytes) -> bytes: + digest = keccak.new(digest_bits=256) + digest.update(data) + return digest.digest() + + +def hex32(value: bytes) -> str: + if len(value) != 32: + raise ValueError(f"expected 32 bytes, got {len(value)}") + return "0x" + value.hex() + + +def parse_bytes32(value: str) -> bytes: + raw = value[2:] if value.startswith("0x") else value + data = bytes.fromhex(raw) + if len(data) != 32: + raise ValueError(f"expected bytes32, got {len(data)} bytes: {value}") + return data + + +def encode_address(value: str) -> bytes: + raw = value[2:] if value.startswith("0x") else value + data = bytes.fromhex(raw) + if len(data) != 20: + raise ValueError(f"expected address, got {len(data)} bytes: {value}") + return (b"\x00" * 12) + data + + +def encode_uint(value: int) -> bytes: + return int(value).to_bytes(32, "big") + + +def abi_encode_static(fields: Iterable[Tuple[str, object]]) -> bytes: + out = b"" + for field_type, value in fields: + if field_type.startswith("uint"): + out += encode_uint(int(value)) + elif field_type == "bytes32": + out += parse_bytes32(str(value)) + elif field_type == "address": + out += encode_address(str(value)) + else: + raise ValueError(f"unsupported field type: {field_type}") + return out + + +TYPE_STRINGS = { + "flowPulseObservationV0": ( + "FlowPulseObservationV0(uint256 chainId,address emittingContract," + "uint64 blockNumber,bytes32 blockHash,bytes32 txHash," + "uint32 transactionIndex,uint32 logIndex,bytes32 eventSignature," + "bytes32 pulseId,bytes32 rootfieldId)" + ), + "flowPulseEventArgsV0": ( + "FlowPulseEventArgsV0(bytes32 pulseId,bytes32 rootfieldId," + "address actor,uint8 pulseType,bytes32 subject,bytes32 commitment," + "bytes32 parentPulseId,uint64 sequence,uint64 occurredAt," + "bytes32 uriHash)" + ), + "flowPulseReceiptV0": ( + "FlowPulseReceiptV0(bytes32 observationId,bytes32 eventArgsHash," + "bytes32 artifactRoot,bytes32 storageReceiptCommitment," + "bytes32 evidenceRoot,uint16 receiptVersion)" + ), + "verifierReportV0": ( + "FlowMemoryVerifierReportV0(bytes32 reportSchemaHash," + "bytes32 observationId,bytes32 receiptHash,bytes32 verifierId," + "bytes32 verifierSetRoot,uint8 status,bytes32 checksRoot," + "uint64 finalizedBlockNumber,bytes32 finalizedBlockHash," + "uint16 reportVersion)" + ), + "attestationEnvelopeV0": ( + "FlowMemoryAttestationEnvelopeV0(bytes32 subjectHash,uint8 subjectKind," + "bytes32 attesterId,bytes32 attesterKeyId,bytes32 verifierSetRoot," + "uint64 issuedAtUnixMs,uint64 expiresAtUnixMs,bytes32 nonce)" + ), +} + + +def expect(label: str, actual: str, expected: str) -> None: + if actual != expected: + raise AssertionError(f"{label}: expected {expected}, got {actual}") + + +def validate_flowpulse_observation_vector() -> None: + vector = json.loads(VECTOR_PATH.read_text(encoding="utf-8")) + type_hashes = vector["typeHashes"] + flow_pulse = vector["flowPulse"] + observation = vector["observation"] + receipt = vector["receipt"] + report = vector["verifierReport"] + attestation = vector["attestationEnvelope"] + + for name, type_string in TYPE_STRINGS.items(): + expect(name, hex32(keccak256(type_string.encode("utf-8"))), type_hashes[name]) + + pulse_id = hex32( + keccak256( + abi_encode_static( + [ + ("bytes32", flow_pulse["schemaId"]), + ("uint256", flow_pulse["chainId"]), + ("address", flow_pulse["emittingContract"]), + ("bytes32", flow_pulse["rootfieldId"]), + ("address", flow_pulse["actor"]), + ("uint8", flow_pulse["pulseType"]), + ("bytes32", flow_pulse["subject"]), + ("bytes32", flow_pulse["commitment"]), + ("bytes32", flow_pulse["parentPulseId"]), + ("uint64", flow_pulse["sequence"]), + ] + ) + ) + ) + expect("pulseId", pulse_id, flow_pulse["pulseId"]) + + observation_id = hex32( + keccak256( + abi_encode_static( + [ + ("bytes32", type_hashes["flowPulseObservationV0"]), + ("uint256", flow_pulse["chainId"]), + ("address", flow_pulse["emittingContract"]), + ("uint64", observation["blockNumber"]), + ("bytes32", observation["blockHash"]), + ("bytes32", observation["txHash"]), + ("uint32", observation["transactionIndex"]), + ("uint32", observation["logIndex"]), + ("bytes32", flow_pulse["eventSignature"]), + ("bytes32", flow_pulse["pulseId"]), + ("bytes32", flow_pulse["rootfieldId"]), + ] + ) + ) + ) + expect("observationId", observation_id, observation["observationId"]) + + event_args_hash = hex32( + keccak256( + abi_encode_static( + [ + ("bytes32", type_hashes["flowPulseEventArgsV0"]), + ("bytes32", flow_pulse["pulseId"]), + ("bytes32", flow_pulse["rootfieldId"]), + ("address", flow_pulse["actor"]), + ("uint8", flow_pulse["pulseType"]), + ("bytes32", flow_pulse["subject"]), + ("bytes32", flow_pulse["commitment"]), + ("bytes32", flow_pulse["parentPulseId"]), + ("uint64", flow_pulse["sequence"]), + ("uint64", flow_pulse["occurredAt"]), + ("bytes32", flow_pulse["uriHash"]), + ] + ) + ) + ) + expect("eventArgsHash", event_args_hash, receipt["eventArgsHash"]) + + receipt_hash = hex32( + keccak256( + abi_encode_static( + [ + ("bytes32", type_hashes["flowPulseReceiptV0"]), + ("bytes32", observation["observationId"]), + ("bytes32", receipt["eventArgsHash"]), + ("bytes32", receipt["artifactRoot"]), + ("bytes32", receipt["storageReceiptCommitment"]), + ("bytes32", receipt["evidenceRoot"]), + ("uint16", receipt["receiptVersion"]), + ] + ) + ) + ) + expect("receiptHash", receipt_hash, receipt["receiptHash"]) + + report_id = hex32( + keccak256( + abi_encode_static( + [ + ("bytes32", type_hashes["verifierReportV0"]), + ("bytes32", report["reportSchemaHash"]), + ("bytes32", observation["observationId"]), + ("bytes32", receipt["receiptHash"]), + ("bytes32", report["verifierId"]), + ("bytes32", report["verifierSetRoot"]), + ("uint8", report["status"]), + ("bytes32", report["checksRoot"]), + ("uint64", report["finalizedBlockNumber"]), + ("bytes32", report["finalizedBlockHash"]), + ("uint16", report["reportVersion"]), + ] + ) + ) + ) + expect("reportId", report_id, report["reportId"]) + + attestation_hash = hex32( + keccak256( + abi_encode_static( + [ + ("bytes32", type_hashes["attestationEnvelopeV0"]), + ("bytes32", attestation["subjectHash"]), + ("uint8", attestation["subjectKind"]), + ("bytes32", attestation["attesterId"]), + ("bytes32", attestation["attesterKeyId"]), + ("bytes32", attestation["verifierSetRoot"]), + ("uint64", attestation["issuedAtUnixMs"]), + ("uint64", attestation["expiresAtUnixMs"]), + ("bytes32", attestation["nonce"]), + ] + ) + ) + ) + expect("attestationEnvelopeHash", attestation_hash, attestation["attestationEnvelopeHash"]) + + +def main() -> None: + validate_flowpulse_observation_vector() + print("FLOWPULSE_VECTOR_RECOMPUTE_OK") + + +if __name__ == "__main__": + main() diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index 9d804787..3eb3e1d3 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -40,6 +40,12 @@ Expected responsibilities: - Verify roots, receipts, commitments, attestations, and proofs. - Produce deterministic verification outputs. +Crypto foundation: + +- Use `crypto/FLOWMEMORY_CRYPTO_SPEC.md` as the draft v0 schema overview. +- Use `crypto/OBSERVATION_IDENTITY.md` to distinguish contract `pulseId`, indexer-derived `observationId`, and verifier `reportId`. +- Use `services/verifier/README.md` for the draft deterministic verifier report flow and status vocabulary. + ## AI Memory Layer Expected responsibilities: diff --git a/docs/DECISIONS/2026-05-13-flowmemory-crypto-v0-foundation.md b/docs/DECISIONS/2026-05-13-flowmemory-crypto-v0-foundation.md new file mode 100644 index 00000000..8e366fef --- /dev/null +++ b/docs/DECISIONS/2026-05-13-flowmemory-crypto-v0-foundation.md @@ -0,0 +1,42 @@ +# FlowMemory Crypto v0 Foundation + +Date: 2026-05-13 + +## Status + +Proposed for cross-agent review; implemented as a runnable V0 package candidate. + +## Context + +FlowMemory now has a FlowPulse contracts foundation and open issues for canonical observation identity, verifier status vocabulary, and receipt/attestation schema vocabulary. Contracts emit `pulseId`, roots, commitments, counters, and advisory URI strings. Indexers and verifiers must derive `txHash`, `transactionIndex`, `logIndex`, and block metadata after receipts and logs exist. + +## Decision + +Adopt draft v0 crypto schemas under `crypto/` as the review target for: + +- `pulseId`, `observationId`, and `reportId` separation +- Keccak-256 typed object hashes +- FlowPulse receipt hashing +- artifact roots and Merkle formats +- worker and verifier EIP-712 signature envelopes +- deterministic verifier reports +- replay protection and reorg handling + +This is not a production protocol acceptance. It is the schema foundation for cross-agent review, package tests, and deterministic test-vector validation. + +## Consequences + +- Contracts should not add CursorRegistry or proof-carrying receipt logic until observation identity is accepted. +- Verifier services can target deterministic report schemas without inventing local formats. +- Artifact and storage commitments remain off-chain and challengeable. +- Verifier attestations remain signed claims, not trustless proofs. +- Future zk work can use receipt and report hashes as stable public-input candidates. + +## Follow-Ups + +- Review and accept or revise the v0 type strings. +- Keep the reference hash implementation and tests under `crypto/` as the conformance source for services. +- Add Solidity shared hash library only after schema review. +- Use `contracts/shared/RECEIPT_VERIFIER_BOUNDARY.md` as the guardrail for issue #28 before adding any `ReceiptVerifier` contract. +- Decide URI policy separately. +- Define verifier set root and key registry governance. diff --git a/docs/SECURITY_MODEL.md b/docs/SECURITY_MODEL.md index d6ae24c9..b56098e5 100644 --- a/docs/SECURITY_MODEL.md +++ b/docs/SECURITY_MODEL.md @@ -22,6 +22,32 @@ This document captures initial security assumptions. It is not a final audit mod - Treat chain logs as observed facts only after receipts are available. - Treat hardware control channels as adversarial unless authenticated. +## Cryptographic Foundation Draft + +The draft cryptographic foundation lives in: + +- `crypto/FLOWMEMORY_CRYPTO_SPEC.md` +- `crypto/OBSERVATION_IDENTITY.md` +- `crypto/RECEIPT_HASHING.md` +- `crypto/MERKLE_AND_ROOTS.md` +- `crypto/ATTESTATIONS.md` +- `crypto/TEST_VECTORS.md` +- `services/verifier/README.md` +- `research/cryptography/THREAT_MODEL.md` +- `research/cryptography/IMPLEMENTATION_PLAN.md` +- `research/cryptography/FUTURE_ZK_ROADMAP.md` + +Current crypto assumptions: + +- FlowMemory v0 uses Keccak-256 typed hashes for Base/EVM compatibility. +- `pulseId` is a contract-emitted logical identifier; `observationId` is derived by indexers from observed receipt and log metadata, including `txHash`, `transactionIndex`, `logIndex`, and `blockHash`. +- `reportId` is a deterministic verifier report identifier; verifier signatures sign reports but do not make them trustless. +- Receipt hashes do not embed signatures. Worker signatures and verifier attestations point at receipt hashes. +- Artifact roots commit to off-chain content through explicit root schemes and Merkle formats. +- Storage receipt commitments are challengeable availability claims, not permanent availability proofs. +- Verifier attestations are signed verifier statements, not zk proofs and not full trustlessness. +- Replay protection requires chain, deployment, sequence, nonce, expiry, and verifier-set domains. + ## Threat Areas ### Protocol @@ -55,6 +81,8 @@ Follow-up: - Incorrect `txHash` or `logIndex` derivation - Non-deterministic verification output - Trusting off-chain artifacts without checking commitments +- Accepting worker signatures on the wrong chain, deployment, verifier set, or sequence +- Treating verifier attestations as proofs instead of challengeable signed statements ### AI Memory @@ -63,6 +91,7 @@ Follow-up: - Weak provenance - Confusing model output with verified state - Embedding or retrieval data that cannot be traced to a receipt +- Hashing low-entropy private metadata without salting or encryption ### Hardware @@ -79,6 +108,14 @@ Follow-up: - CI secrets exposure - Binary artifacts without provenance +### Storage And Artifacts + +- Ambiguous artifact root scheme selection +- Invalid Merkle openings +- Unavailable off-chain artifacts +- Locator leakage through public commitments +- Retention claims that cannot be challenged or sampled + ## PR Security Checklist - Does this change introduce or require secrets? @@ -88,3 +125,6 @@ Follow-up: - Does it place heavy data on-chain? - Does it assume LoRa or Meshtastic can carry high-bandwidth traffic? - Are tests or verification steps included where practical? +- Does every new receipt, root, signature, attestation, or challenge format have a domain-separated type hash? +- Does replay protection cover chain, deployment, nonce or sequence, expiry, and verifier set where relevant? +- Does the UI or documentation avoid claiming full trustlessness unless a proof and enforcement path exist? diff --git a/research/cryptography/FUTURE_ZK_ROADMAP.md b/research/cryptography/FUTURE_ZK_ROADMAP.md new file mode 100644 index 00000000..0546179f --- /dev/null +++ b/research/cryptography/FUTURE_ZK_ROADMAP.md @@ -0,0 +1,92 @@ +# Future zk And Proof-Carrying Receipt Roadmap + +Status: research draft. + +This roadmap identifies what could become proof-carrying and what should remain verifier-attested in the MVP. + +## MVP Must Remain Verifier-Attested + +The MVP should rely on deterministic verifier reports, not zk proofs, for: + +- chain finality policy and reorg handling +- RPC/indexer data source agreement +- URI and storage locator policy +- artifact availability sampling +- worker behavior and model output quality +- storage provider claims +- key registry and verifier set governance +- challenge response review + +These claims can be signed, challenged, and replayed. They are not trustless. + +## First Proof Candidates + +Good early zk candidates have small, deterministic public inputs: + +- Merkle inclusion for artifact chunks. +- Artifact root recomputation from a manifest hash and chunk openings. +- Receipt consistency from `observationId`, `eventArgsHash`, `artifactRoot`, and `storageReceiptCommitment`. +- Verifier report consistency for a fixed set of boolean checks. +- Rootflow aggregation of ordered receipt hashes. + +## Harder Proof Candidates + +These require more research: + +- proving a full transaction receipt/log was canonical without trusting an indexer +- proving off-chain data availability over time +- proving model output correctness +- proving private metadata policy compliance +- proving hardware identity without a trusted manufacturing or key enrollment process + +## Proof-Carrying Receipt Shape + +A future proof-carrying receipt should keep the v0 receipt hash as a stable public input: + +```text +public inputs: + schemaId + chainId + observationId + eventArgsHash + receiptHash + artifactRoot + storageReceiptCommitment + verifierPolicyHash + reportSchemaHash +``` + +Witnesses may include: + +- event args +- artifact manifest +- Merkle opening path +- storage receipt opening +- check result details +- worker or verifier signature preimages + +The proof should not force private artifact bytes public unless the challenge or disclosure policy requires it. + +## Recursive Aggregation Path + +1. Prove one receipt is internally consistent. +2. Prove a batch of receipts share an accepted schema and verifier policy. +3. Prove a Rootflow checkpoint aggregates a batch. +4. Attach the checkpoint to a Rootfield state commitment. +5. Consider appchain/L1 settlement only after proof cost, data availability, and governance are understood. + +## Go/No-Go Criteria + +Before implementing zk circuits, FlowMemory should have: + +- accepted observation identity +- accepted receipt and report schemas +- deterministic verifier reference implementation +- cross-language test vectors +- exact public input list +- exact witness privacy rules +- proof system choice and setup assumptions +- cost model versus ordinary verifier replay +- challenge model for failed or missing proofs + +Until those exist, zk is research, not product or protocol infrastructure. diff --git a/research/cryptography/IMPLEMENTATION_PLAN.md b/research/cryptography/IMPLEMENTATION_PLAN.md new file mode 100644 index 00000000..86faa6c3 --- /dev/null +++ b/research/cryptography/IMPLEMENTATION_PLAN.md @@ -0,0 +1,168 @@ +# FlowMemory Cryptography Implementation Plan + +Status: draft v0 for bootstrap design. + +This plan turns the draft crypto foundation into reviewable implementation steps. It stays within crypto, shared contracts, and verifier scope. It does not include tokenomics. + +## Phase 0: Schema Review + +Deliverables: + +- Review `crypto/FLOWMEMORY_CRYPTO_SPEC.md`. +- Review `crypto/OBSERVATION_IDENTITY.md`. +- Review `crypto/RECEIPT_HASHING.md`. +- Review `crypto/MERKLE_AND_ROOTS.md`. +- Review `crypto/ATTESTATIONS.md`. +- Open issues for unresolved schema choices. +- Decide whether v0 JSON canonicalization must strictly use RFC 8785 or a narrower project-specific subset. +- Decide the first accepted FlowPulse event schema with the protocol contracts agent. + +Acceptance checks: + +- Every hash has a type string. +- Every variable-length field is pre-hashed. +- Every replay domain is named. +- The docs clearly state what is not trustless yet. + +## Phase 1: Test Vector Harness + +Status: runnable package candidate exists in `crypto/` with 21 package-level vectors and a Python FlowPulse aggregate cross-check. + +Deliverables: + +- Add a small reference implementation under `crypto/` for Keccak typed hashes. +- Validate `crypto/test-vectors/flowpulse-observation-v0.json`. +- Add negative tests for swapped fields, changed type strings, wrong Merkle order, and odd-leaf handling. +- Add equivalent vectors for empty artifact and one-chunk artifact roots. + +Acceptance checks: + +- Local tests reproduce all vector hashes. +- Bad vectors fail deterministically. +- No production keys, secrets, RPC URLs, or private locators are committed. + +## Phase 2: Shared Contract Hash Library + +Scope: + +- `contracts/shared/` + +Deliverables: + +- Solidity constants for type hashes. +- Pure functions for event cursor hash, receipt hash, artifact root hash, storage commitment hash, worker struct hash, verifier attestation hash, and challenge hash. +- Merkle proof verification for `FM-MERKLE-KECCAK256-BINARY-V0`. +- Tests comparing Solidity output to the JSON vectors. + +Acceptance checks: + +- Solidity tests match off-chain vectors. +- Functions use `abi.encode`, not ambiguous packed encoding; Merkle leaf and internal node hashes are typed objects in v0. +- Contracts do not attempt to access `txHash` or `logIndex` during hook execution. + +## Phase 3: Verifier Service Reference + +Scope: + +- `services/verifier/` + +Deliverables: + +- Parser for receipt, artifact manifest, storage commitment, worker signature, and verifier attestation envelopes. +- Cursor derivation from observed receipt/log data. +- Reorg/finality status handling. +- Worker signature verification with strict domain checks. +- Artifact root recomputation from manifest and chunk openings. +- Deterministic verification report with `checksRoot`. + +Acceptance checks: + +- Verifier rejects wrong chain, wrong deployment, stale finality, expired signatures, duplicate worker sequence, and bad Merkle openings. +- Verifier labels output as observed, pending, verified, failed, challenged, or superseded. +- Verification reports are deterministic from the same inputs. + +## Phase 4: Storage Receipt And Challenge Loop + +Deliverables: + +- Define storage provider identity envelope. +- Define public versus private locator commitment policy. +- Implement availability sampling reports. +- Implement challenge evidence envelope and response envelope. +- Record failure modes without tokenomics or slashing assumptions. + +Acceptance checks: + +- Challenges can require manifest openings, chunk openings, signature proof, storage locator opening, and verifier report replay. +- Services can mark claims challenged, responded, upheld, dismissed, or expired. +- Apps and explorers can distinguish storage claim from storage proof. + +## Phase 5: Rootflow And Rootfield Binding + +Deliverables: + +- Draft Rootflow as ordered receipt progression commitment. +- Draft Rootfield as artifact/state/report commitment layer. +- Define how roots are rolled up, checkpointed, and challenged. +- Add a decision record before contracts depend on either root. + +Acceptance checks: + +- Rootflow and Rootfield have different semantics. +- Indexers can reconstruct roots from receipts and logs. +- Verifiers can recompute roots from committed inputs. + +## Phase 6: zk And Proof-Carrying Roadmap + +Research tracks: + +- Merkle inclusion and non-inclusion proof circuits. +- Receipt consistency circuits for cursor, receipt, artifact root, and storage commitment linkage. +- Verifier-report compression into proof-carrying receipts. +- Recursive aggregation of receipt proofs into Rootflow checkpoints. +- Selective disclosure for private artifact metadata. +- Hardware/device attestation research for FlowRouter identity. + +Gate before implementation: + +- Define exact public inputs. +- Define witness formats and privacy requirements. +- Decide proving system and trusted setup assumptions. +- Compare proof cost against ordinary deterministic verification. +- Record a go/no-go decision before building production circuits. + +## Recommended Issues To Create + +These are the smallest useful issues to create from this crypto pass: + +- Define the first FlowPulse event schema and payload nonce policy. +- Integrate the typed-hash reference implementation and vector tests into verifier services. +- Add Solidity shared hash library and vector tests. +- Implement verifier receipt parser and cursor derivation. +- Define worker and verifier key registry requirements. +- Define storage provider identity and locator privacy policy. +- Draft Rootflow and Rootfield semantics as a decision record. +- Research zk public inputs for proof-carrying receipts. + +Created or reused follow-up GitHub issues: + +- #28: future `ReceiptVerifier` contract boundary, with draft local boundary in `contracts/shared/RECEIPT_VERIFIER_BOUNDARY.md`. +- #38: service-side validation for crypto v0 test vectors. +- #40: verifier signature envelope validation. +- #42: zk proof-carrying receipt research milestones. +- #47: services/shared crypto package integration boundary. + +Issue #39 was created during this cycle and closed as a duplicate of #28. + +## Near-Term Pull Request Shape + +The first implementation PR should include only: + +- typed hash reference functions +- test vector validation +- no network calls +- no production contracts +- no production verifier service +- no token mechanics + +That keeps the first crypto implementation small enough to review and safe enough to change while schemas are still draft. diff --git a/research/cryptography/THREAT_MODEL.md b/research/cryptography/THREAT_MODEL.md new file mode 100644 index 00000000..2026b7b6 --- /dev/null +++ b/research/cryptography/THREAT_MODEL.md @@ -0,0 +1,195 @@ +# FlowMemory Cryptography Threat Model + +Status: draft v0 for bootstrap design. + +This threat model covers the draft cryptographic foundation in `crypto/`. It does not replace a full protocol audit, contract audit, verifier audit, hardware review, or storage provider review. + +## Assets + +- Worker signing keys and key registry state +- Verifier signing keys and verifier set roots +- Receipt hashes and cursor derivations +- Artifact manifests, Merkle openings, and artifact roots +- Storage receipt commitments and private locator metadata +- Rootflow and Rootfield roots +- Challenge evidence and responses +- Off-chain memory artifacts, embeddings, model outputs, and research data +- Chain receipts, logs, block hashes, and finality assumptions + +## Trust Boundaries + +### Chain Boundary + +Contracts can emit events and store intentional state, but they cannot know final `txHash` or `logIndex` during execution. Indexers cross this boundary when they read receipts and logs after execution. + +### Worker Boundary + +Workers can observe, process, and sign claims. A worker signature proves a key signed a specific typed message; it does not prove the worker behaved honestly or that its model output is true. + +### Verifier Boundary + +Verifiers run checks and sign attestations. A verifier attestation proves a verifier key signed a result; it is not a zero-knowledge proof and must remain challengeable. + +### Storage Boundary + +Storage providers can lose, censor, mutate, or hide artifacts. Storage receipt commitments are commitments to claims and metadata, not guarantees of perpetual data availability. + +### Hardware Boundary + +Hardware devices and radio sidecars can be spoofed, replayed, delayed, physically tampered with, or bandwidth-constrained. Hardware control messages need authentication and small payloads. + +## Threats And Mitigations + +### Ambiguous Encoding + +Threat: Two implementations hash different byte representations of the same logical receipt. + +Mitigations: + +- Use typed hashes with exact type strings. +- Hash variable-length inputs before entering typed objects. +- Use canonical JSON for JSON payloads. +- Maintain cross-language test vectors. +- Reject unknown schema versions and root schemes by default. + +Residual risk: Canonical JSON libraries can differ on edge cases such as numbers and Unicode. v0 payload schemas should avoid ambiguous numeric and text encodings until conformance tests exist. + +### Cursor Forgery Or Confusion + +Threat: A service claims a cursor for an event that did not happen, or reuses a cursor across chains, contracts, or events. + +Mitigations: + +- Derive event cursors from `chainId`, `blockHash`, `blockNumber`, `txHash`, `transactionIndex`, `logIndex`, emitter, and `topic0`. +- Require indexers to recompute cursors from observed receipts and logs. +- Treat cursors as unstable until finality policy accepts the block. +- Include deployment and chain domain data in signatures. + +Residual risk: RPC providers can return inconsistent data during outages or reorgs. Verifiers should support multiple RPC backends or independently indexed data before high-value acceptance. + +### Replay Attacks + +Threat: A valid worker signature, verifier attestation, or receipt is replayed across chains, deployments, verifier sets, or time windows. + +Mitigations: + +- Bind signatures to `chainId`, deployment salt, verifier set root, expiry, nonce, and worker sequence. +- Track consumed worker sequences per worker identity. +- Reject stale finality, expired signatures, and mismatched verifier sets. +- Bind receipt hashes to source-specific cursors and nonces. + +Residual risk: Until a registry or contract stores accepted nonces and key state, replay prevention is partially off-chain policy. + +### Artifact Substitution + +Threat: An attacker swaps artifact bytes while keeping a plausible manifest or locator. + +Mitigations: + +- Commit to chunk hashes, Merkle root, manifest hash, byte length, chunk size, media type hash, and metadata hash. +- Require Merkle openings for challenged chunks. +- Verify byte offsets and lengths, not only chunk contents. +- Reject root scheme mismatches. + +Residual risk: If the original artifact is unavailable, verifiers cannot prove it matched a root beyond available openings and prior attestations. + +### Storage Availability Failure + +Threat: A storage provider signs or implies availability but later cannot serve the artifact. + +Mitigations: + +- Treat storage receipt commitments as challengeable claims. +- Commit to retention policy, provider identity, location commitment, and availability sample roots. +- Sample availability during the retention window. +- Record failed openings and verifier failures. + +Residual risk: Availability sampling is probabilistic unless backed by stronger data availability systems or replicated retrieval guarantees. This design does not yet provide that. + +### Worker Key Compromise + +Threat: A worker signing key is stolen and used to sign false receipts. + +Mitigations: + +- Use worker key identifiers and sequence tracking. +- Add expiry to signatures. +- Rotate keys through a registry when implemented. +- Require verifier checks and challenge windows before high-confidence status. +- Support revocation roots or registry state in future contracts. + +Residual risk: Receipts signed before compromise detection may remain ambiguous unless the registry defines revocation time and acceptance policy. + +### Verifier Collusion Or Error + +Threat: Verifiers sign false pass attestations or implement checks incorrectly. + +Mitigations: + +- Make attestations explicit about result code, check root, finality depth, and verifier set root. +- Allow independent verifier recomputation. +- Keep deterministic verification reports. +- Challenge failed, inconsistent, or incomplete reports. +- Do not present verifier attestations as trustless proofs. + +Residual risk: If all accepted verifiers collude, the system can report false verified states until challenged by an independent party. + +### Chain Reorgs + +Threat: A receipt is built on a log that disappears or changes position after a reorg. + +Mitigations: + +- Include `blockHash` in event cursors. +- Require finality policy before high-confidence verification. +- Mark pre-finality receipts as pending or observed only. +- Supersede receipts whose block is reorged out. + +Residual risk: Deep reorgs or chain incidents require operational policy, not only hash schemas. + +### Privacy Leakage + +Threat: Receipt payload hashes, locator commitments, metadata hashes, or public roots leak sensitive information through small search spaces. + +Mitigations: + +- Salt low-entropy commitments. +- Use encrypted locator envelopes when locators are sensitive. +- Avoid putting raw personal, model, or hardware data in public payloads. +- Separate public receipt data from private artifact data. + +Residual risk: Hashes of low-entropy data can be brute-forced. Sensitive payload schemas must include salt or encryption requirements. + +### Challenge Abuse + +Threat: Attackers spam challenges or force expensive openings. + +Mitigations: + +- Define challenge reason codes and evidence roots. +- Require structured evidence before a challenge is accepted by services. +- Add rate limits and policy gates in verifier services. +- Keep economic bonding outside this design until tokenomics is explicitly in scope. + +Residual risk: Without on-chain challenge economics, abuse mitigation is operational and service-level. + +## Security Assumptions + +- Keccak-256 remains collision-resistant for these protocol uses. +- secp256k1 signatures are verified with strict malleability checks where used. +- Verifiers can obtain honest chain receipt/log data after finality. +- Implementations follow exact field order and encoding. +- Heavy data remains off-chain and is opened only through intended disclosure paths. + +## Required Review Gates + +Before production use, FlowMemory needs: + +- independent review of type strings and encoding +- cross-language vector tests +- contract-level hash verification tests +- verifier replay and reorg tests +- worker and verifier key management design +- storage locator privacy review +- challenge state machine review +- decision record for Rootflow and Rootfield semantics diff --git a/services/verifier/README.md b/services/verifier/README.md new file mode 100644 index 00000000..2be275e2 --- /dev/null +++ b/services/verifier/README.md @@ -0,0 +1,82 @@ +# FlowMemory Verifier MVP + +Status: specification draft. + +The verifier MVP reads FlowPulse observations, reconstructs commitments, checks evidence, and emits deterministic reports. It does not run a production service, proof network, or token system. + +## Inputs + +- decoded FlowPulse log +- receipt/log metadata used to derive `observationId` +- artifact manifest and Merkle openings, when required +- storage receipt commitment openings, when required +- worker signature envelope, when required +- verifier policy document hash + +## Import Strategy + +There is no `services/shared/` package yet. Until one exists, verifier code should import or mirror the package under `crypto/`: + +```js +import { + flowPulseObservationId, + flowPulseEventArgsHash, + receiptHash, + verifierReportHash, + verifierSignaturePayload, + verifyDigest +} from "../../crypto/src/index.js"; +``` + +If the verifier cannot import directly, it must run equivalent tests against `crypto/fixtures/`, `crypto/fixtures/vectors.json`, and `crypto/test-vectors/`. The compatibility gate is: + +```powershell +cd E:\FlowMemory\flowmemory-crypto\crypto +npm test +npm run validate:vectors +python validate_test_vectors.py +``` + +## Deterministic Report Flow + +1. Recompute `observationId`. +2. Recompute `eventArgsHash`. +3. Recompute `receiptHash`. +4. Recompute artifact root and storage commitment if openings are supplied. +5. Verify worker signatures if policy requires them. +6. Evaluate finality and reorg status. +7. Produce canonical checks JSON or a checks Merkle root. +8. Produce `reportId`. +9. Sign `reportId` with the verifier signature envelope. + +## Status Vocabulary + +```text +0 = reserved +1 = observed +2 = verified +3 = unresolved +4 = unsupported +5 = failed +6 = reorged +7 = superseded +``` + +Minimum requirements: + +- `observed`: receipt/log parsed and observation id computed. +- `verified`: all policy-required checks passed. +- `unresolved`: required evidence is missing or unavailable. +- `unsupported`: schema, pulse type, root scheme, or key type is unknown. +- `failed`: a policy-required check failed. +- `reorged`: block/log is not canonical under finality policy. +- `superseded`: a newer report intentionally replaces this report. + +## Non-Goals + +- No live RPC integration in this spec. +- No database schema. +- No API server. +- No zk proof implementation. +- No verifier economics. +- No tokenomics.