feat(internal, cmd): protocol-migrate current-state

[aristidesstaffieri]

Closes #516

What

Implements the protocol-migrate current-state CLI subcommand and extracts a shared migration engine to eliminate duplication between the history and current-state migration paths.

• protocol-migrate current-state — builds protocol current state from a user-specified --start-ledger forward to the tip, converging with live ingestion via CAS-gated cursors on protocol_{ID}_current_state_cursor. Mirrors the history migration lifecycle: validate → process batches → converge → complete.
• Shared protocolMigrateEngine with migrationStrategy struct — parameterizes the 5 substitution points between history and current-state (status field, status update method, cursor name function, persist method, start ledger source), eliminating ~1100 lines of duplication. Both history and current-state services are now thin wrappers (~100 lines each) over the shared engine.
• UpdateCurrentStateMigrationStatus added to ProtocolsModel for current-state migration lifecycle tracking.
• Shared CLI helpers (buildMigrationCommand/runMigration) extracted in cmd/protocol_migrate.go to deduplicate flag definitions and setup logic between subcommands.
• Integration test (TestCurrentStateMigrationThenLiveIngestionHandoff) verifying current-state migration hands off cleanly to live ingestion via CAS cursor convergence.

Why

Protocol current-state production requires building a complete snapshot of protocol state from each protocol's deployment ledger forward — unlike history migration which only covers the retention window. Without this command, protocols only have current state from the point live ingestion started, missing all prior state. The shared engine extraction was necessary because the history and current-state migration paths are structurally identical (validate, batch-process, CAS-advance, converge, complete) with only 5 strategy-specific substitution points.

Known limitations

No current protocols implemented yet.

Issue that this PR addresses

#516

Checklist

PR Structure

• It is not possible to break this PR down into smaller PRs.
• This PR does not mix refactoring changes with feature changes.
• This PR's title starts with name of package that is most changed in the PR, or all if the changes are broad or impact many packages.

Thoroughness

• This PR adds tests for the new functionality or fixes.
• All updated queries have been tested (refer to this check if the data set returned by the updated query is expected to be same as the original one).

Release

• This is not a breaking change.
• This is ready to be tested in development.
• The new functionality is gated with a feature flag if this is not ready for production.

[Copilot]

Pull request overview

Adds protocol state-production migration tooling and supporting infrastructure: a new protocol-migrate current-state command plus a shared migration engine, alongside protocol WASM/contract tracking, classification (“protocol-setup”), and live-ingestion handoff via CAS cursors.

Changes:

• Introduce shared protocolMigrateEngine with history/current-state strategies and add protocol-migrate current-state CLI subcommand.
• Add protocol setup/classification pipeline (WASM spec extraction + validator interface/registry) and DB models/migrations for protocols, protocol_wasms, protocol_contracts.
• Extend ingestion (live + backfill) and indexer buffer/processors to collect and persist protocol WASMs/contracts, plus new protocol-state metrics and caching.

Reviewed changes

Copilot reviewed 72 out of 73 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
internal/utils/retry.go	New generic retry helper used for ledger fetch retries.
internal/utils/retry_test.go	Unit tests for RetryWithBackoff.
internal/utils/collections.go	New BuildMap helper for slice→map with dup detection.
internal/utils/collections_test.go	Unit tests for BuildMap.
internal/utils/ingestion_utils.go	Add protocol cursor key helper functions.
internal/services/validator_registry.go	New validator registry for protocol-setup.
internal/services/validator_registry_test.go	Tests for validator registry behavior and concurrency.
internal/services/processor_registry.go	New processor registry for protocol-migrate/ingest wiring.
internal/services/processor_registry_test.go	Tests for processor registry behavior and concurrency.
internal/services/protocol_processor.go	Define ProtocolProcessor + input contract for state production.
internal/services/protocol_migrate.go	Shared migration engine with CAS cursor convergence/handoff.
internal/services/protocol_migrate_history.go	History migration wrapper wiring the shared engine.
internal/services/protocol_migrate_history_test.go	Tests for history strategy-specific behavior.
internal/services/protocol_migrate_current_state.go	Current-state migration wrapper wiring the shared engine.
internal/services/protocol_migrate_current_state_test.go	Tests for current-state strategy-specific behavior.
internal/services/protocol_validator.go	New validator/spec-extractor interfaces + wazero implementation.
internal/services/wasm_spec_extractor_test.go	Tests extracting specs from real WASMs + SEP41 validator check.
internal/services/protocol_setup.go	Protocol setup service to classify unclassified WASMs via RPC.
internal/services/protocol_setup_test.go	Unit tests for protocol setup service behavior.
internal/services/contract_validator.go	Refactor SEP-41 detection helpers + add SEP41ProtocolValidator.
internal/services/contract_validator_test.go	Update tests to use refactored helper functions.
internal/services/ingest.go	Add protocol processors + checkpoint service wiring; remove old retry helper.
internal/services/ingest_live.go	Add protocol state production pipeline, CAS gating, and contract caching.
internal/services/ingest_backfill.go	Persist protocol WASMs/contracts during catchup; use shared retry helper.
internal/services/mocks.go	Add mocks for ProtocolProcessor/Validator/WasmSpecExtractor and ChangeReader; remove obsolete token method mock.
internal/services/token_ingestion_test.go	Refactor tests for new TokenIngestionService config API and add helpers.
internal/indexer/processors/protocol_wasms.go	New processor extracting ContractCode WASM hashes.
internal/indexer/processors/protocol_wasms_test.go	Tests for protocol WASM extraction processor.
internal/indexer/processors/protocol_contracts.go	New processor mapping contract instances to WASM hashes.
internal/indexer/processors/protocol_contracts_test.go	Tests for protocol contract extraction processor.
internal/indexer/indexer.go	Wire new processors into per-operation processing and buffer.
internal/indexer/mocks.go	Add mocks for new ledger change processors.
internal/indexer/indexer_test.go	Update indexer tests to include protocol WASM/contract processors.
internal/indexer/indexer_buffer.go	Add buffer support for protocol WASMs/contracts with merge/clear semantics.
internal/indexer/indexer_buffer_test.go	Tests for new buffer protocol WASM/contract behavior.
internal/data/protocols.go	New protocols model + status update helpers.
internal/data/protocols_test.go	Tests for protocols model operations.
internal/data/protocol_wasms.go	New protocol_wasms model for batch insert + classification update.
internal/data/protocol_wasms_test.go	Tests for protocol_wasms batch insert/update.
internal/data/protocol_contracts.go	New protocol_contracts model for upsert + protocol lookup.
internal/data/protocol_contracts_test.go	Tests for protocol_contracts inserts/upserts/FK behavior.
internal/data/models.go	Add new protocol models to shared Models struct.
internal/data/mocks.go	Add mocks for new protocol models and TrustlineAssetModel.
internal/data/ingest_store.go	Add CAS method and cursor name constants.
internal/data/ingest_store_test.go	Tests for CompareAndSwap behavior.
internal/db/migrations/2026-03-09.0-protocols.sql	Create protocols table with classification + migration statuses.
internal/db/migrations/2026-02-20.0-protocol_wasms.sql	Create protocol_wasms table.
internal/db/migrations/2026-03-09.2-protocol_wasms_fk.sql	Add FK from protocol_wasms.protocol_id → protocols.id.
internal/db/migrations/2026-03-09.0-protocol_contracts.sql	Create protocol_contracts table referencing protocol_wasms.
internal/db/migrations/protocols/main.go	Add embedded “protocol registration” SQL runner.
internal/db/migrations/protocols/000_placeholder.sql	Placeholder for embedded protocol registration SQL files.
internal/db/migrate.go	Add RunProtocolMigrations entrypoint for embedded protocol SQL.
internal/metrics/metrics.go	Add protocol state production + cache metrics.
internal/metrics/metrics_test.go	Tests for newly added protocol metrics.
internal/metrics/mocks.go	Extend metrics mock with new protocol metric methods.
internal/ingest/ingest.go	Wire checkpoint service and processor registry into ingest dependency setup.
internal/loadtest/runner.go	Update loadtest wiring for new TokenIngestionService config API.
internal/integrationtests/main_test.go	Add DataMigrationTestSuite to integration test runner.
cmd/root.go	Register new protocol-setup and protocol-migrate commands.
cmd/protocol_setup.go	Add protocol-setup CLI command to classify WASMs via validators.
cmd/protocol_migrate.go	Add protocol-migrate CLI with history/current-state subcommands and shared flag setup.
docs/feature-design/data-migrations.md	Update feature design doc for protocol WASM/contract and migration flows.
internal/serve/graphql/generated/generated.go	Import ordering adjustment (generated).
internal/serve/graphql/resolvers/account.resolvers.go	Import ordering adjustment.
internal/serve/graphql/resolvers/mutations.resolvers.go	Import ordering adjustment.
internal/serve/graphql/resolvers/queries.resolvers.go	Import ordering adjustment.
internal/serve/graphql/resolvers/statechange.resolvers.go	Flatten resolver type declarations (no functional change).

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.