Ethereum State & Transaction Sharding

Block-based state and transaction sharding with 2PC coordination.

Quick Start

# Start network (8 shards + orchestrator)
docker compose up --build -d

# Test cross-shard transactions
python scripts/test_cross_shard.py

# View logs
docker compose logs -f shard-orch
docker compose logs -f shard-0

# Stop
docker compose down

Architecture

Components

Orchestrator Shard: Coordinates cross-shard transactions via 2PC
- Produces blocks with tpc_result (commit decisions) and ct_to_order (new txs)
- Broadcasts blocks to all State Shards
- Stateless: No account balances or contract storage
State Shards (0-5): Independent blockchain state
- Produce blocks with tx_ordering (executed txs) and tpc_prepare (2PC votes)
- Send blocks to Orchestrator Shard
- Maintain EVM state (balances, contracts, storage)

Transaction Flow

Users submit transactions to their local State Shard. The shard automatically detects whether it's cross-shard:

User submits POST /tx/submit to State Shard
         │
         ▼
┌─────────────────────────────────────┐
│ State Shard auto-detection:         │
│ - 'to' on different shard?          │───► Cross-shard
│ - Contract accesses other shards?   │───► Cross-shard
│ - Otherwise                         │───► Local (execute immediately)
└─────────────────────────────────────┘
         │
    Cross-shard
         │
         ▼
   Forward to Orchestrator → 2PC Protocol

Block-Based 2PC Flow (Cross-Shard Only)

Round N:
  1. Orchestrator Block N: ct_to_order = [tx1, tx2], tpc_result = {}
         ↓ broadcast to all State Shards

  2. State Shards process:
     - Source Shard: lock funds (no debit), vote
     - Dest Shard: validate ReadSet, store pending credit, vote
         ↓ produce blocks with votes

  3. State Shard Blocks: tpc_prepare = {tx1: true, tx2: false}, shard_id = N
         ↓ send to Orchestrator

  4. Orchestrator aggregates votes from ALL involved shards

Round N+1:
  5. Orchestrator Block N+1: tpc_result = {tx1: true, tx2: false}
         ↓ broadcast

  6. State Shards finalize:
     - Committed: Source debits + clears lock, Dest applies credits
     - Aborted: Source clears lock (no refund needed), Dest discards pending

Key: Users don't need to know about sharding - the system handles routing transparently.

Communication Pattern

Orchestrator Shard → State Shards (broadcast)
State Shards → Orchestrator Shard (unicast)
State Shards ↮ State Shards (NONE - isolated)

Implementation Status

✅ Implemented

Block-based 2PC coordination
State sharding (contracts on one shard, state distributed by address % 6)
Cross-shard atomic transactions
EVM execution (geth's vm + state packages)
JSON-RPC compatibility (Foundry tools)
EVM simulation for RwSet discovery (cross-shard contract calls)
Multi-shard vote aggregation (all involved shards must vote)
Multi-recipient credit handling (multiple recipients per tx)
Destination shard voting with ReadSet validation
Simulation lock lifecycle with TTL-based cleanup (2 minute TTL)
Atomic balance check and lock (prevents race conditions in 2PC prepare)
Bounded concurrency for block broadcast (max 3 concurrent HTTP requests)
Thread-safe tracking during contract simulation

⚠️ Deferred (Documented Assumptions)

Merkle Proof Validation:

ReadSetItem.Proof field exists but empty ([][]byte{})
Trust StateShardBlock.StateRoot without cryptographic verification
Rationale: Focus on 2PC mechanics before cryptographic validation
Future: Implement MPT/Verkle proof generation and validation

Light Client Protocol:

Orchestrator Shard trusts State Shard blocks without verification
Assumption: State Shards are honest
Future: Implement light client verification

Consensus:

Single validator per shard (no BFT)
Instant finality (no reorgs)
Synchronous block production (3s intervals)
Assumption: Consensus is solved, focus on cross-shard coordination
Future: Add PBFT/HotStuff per shard

See design.md for full specification (Korean).

API

State Shard Endpoints

# Unified transaction submission (auto-detects cross-shard)
POST http://shard-0:8545/tx/submit
{"from": "0x...", "to": "0x...", "value": "1000", "data": "0x...", "gas": 100000}
# Returns: {"success": true, "cross_shard": false} for local tx
# Returns: {"success": true, "cross_shard": true, "tx_id": "..."} for cross-shard tx

# Balance
GET http://shard-0:8545/balance/0x1234...

# Local transfer (legacy - prefer /tx/submit)
POST http://shard-0:8545/transfer
{"from": "0x...", "to": "0x...", "amount": "1000"}

# Faucet
POST http://shard-0:8545/faucet
{"address": "0x...", "amount": "1000000000000000000000"}

# Cross-shard transfer (legacy - prefer /tx/submit)
POST http://shard-0:8545/cross-shard/transfer
{"from": "0x...", "to": "0x...", "to_shard": 1, "amount": "100000000000000000"}

# EVM
POST http://shard-0:8545/evm/deploy
{"from": "0x...", "bytecode": "0x608060...", "gas": 3000000}

POST http://shard-0:8545/evm/call
{"from": "0x...", "to": "0x...", "data": "0xa9059cbb...", "gas": 1000000}

# Storage access
GET http://shard-0:8545/evm/storage/{address}/{slot}
# Returns: {"address": "0x...", "slot": "0x...", "value": "0x..."}

# Storage access with Merkle proof (V2.3)
GET http://shard-0:8545/evm/storage/{address}/{slot}?proof=true
# Returns: {"address": "0x...", "slot": "0x...", "value": "0x...",
#           "state_root": "0x...", "block_height": 42,
#           "account_proof": ["0x...", ...], "storage_proof": ["0x...", ...]}

# Simulation locking (used by Orchestrator)
POST http://shard-0:8545/state/lock
{"tx_id": "...", "address": "0x..."}

POST http://shard-0:8545/state/unlock
{"tx_id": "...", "address": "0x..."}

# JSON-RPC (Foundry compatible)
POST http://shard-0:8545/
{"jsonrpc": "2.0", "method": "eth_sendTransaction", "params": [...], "id": 1}

Orchestrator Shard Endpoints

# Health
GET http://shard-orch:8080/health

# Shard list
GET http://shard-orch:8080/shards

# Submit cross-shard contract call (triggers simulation)
POST http://shard-orch:8080/cross-shard/call
{"from": "0x...", "to": "0x...", "from_shard": 0, "value": "100", "data": "0xa9059cbb..."}

# Check simulation status
GET http://shard-orch:8080/cross-shard/simulation/{txid}

# Transaction status
GET http://shard-orch:8080/cross-shard/status/{txid}

Data Structures

// CrossShardTx represents a cross-shard transaction
// Destinations are derived from RwSet - each RwVariable specifies an address and shard
type CrossShardTx struct {
    ID        string         // Unique transaction ID (UUID)
    TxHash    common.Hash    // Optional: Ethereum tx hash
    FromShard int            // Source shard ID (initiator)
    From      common.Address // Sender address
    Value     *big.Int       // Transfer amount
    Data      []byte         // Optional: calldata
    RwSet     []RwVariable   // Target shards/addresses - destinations derived from this
    Status    TxStatus       // pending/prepared/committed/aborted
}

// Helper methods:
// tx.TargetShards() []int      - returns unique shard IDs from RwSet
// tx.InvolvedShards() []int    - returns FromShard + TargetShards

// Example: Simple transfer from shard 0 to shard 2
// CrossShardTx{
//     FromShard: 0,
//     From: sender,
//     Value: amount,
//     RwSet: []RwVariable{{
//         Address: recipient,
//         ReferenceBlock: Reference{ShardNum: 2},
//     }},
// }
//
// Example: Contract touching shards 1, 3, and 5
// CrossShardTx{
//     FromShard: 0,
//     From: caller,
//     RwSet: []RwVariable{
//         {Address: contract1, ReferenceBlock: Reference{ShardNum: 1}},
//         {Address: contract2, ReferenceBlock: Reference{ShardNum: 3}},
//         {Address: contract3, ReferenceBlock: Reference{ShardNum: 5}},
//     },
// }

type RwVariable struct {
    Address        common.Address
    ReferenceBlock Reference
    ReadSet        []ReadSetItem
    WriteSet       []Slot
}

type ReadSetItem struct {
    Slot  Slot
    Value []byte
    Proof [][]byte  // Empty for now (deferred)
}

// Orchestrator Shard Block
type OrchestratorShardBlock struct {
    Height    uint64
    PrevHash  BlockHash
    Timestamp uint64
    TpcResult map[string]bool  // txID → committed (from previous round)
    CtToOrder []CrossShardTx   // New cross-shard txs (for this round)
}

// State Shard Block
type StateShardBlock struct {
    ShardID    int               // Which shard produced this block
    Height     uint64
    PrevHash   BlockHash
    Timestamp  uint64
    StateRoot  common.Hash       // Merkle root
    TxOrdering []Transaction     // Executed txs
    TpcPrepare map[string]bool   // txID → can_commit (vote)
}

File Structure

internal/
├── protocol/
│   ├── types.go       # CrossShardTx, RwVariable, ReadSetItem, BigInt, HexBytes
│   └── block.go       # OrchestratorShardBlock, StateShardBlock definitions
├── shard/
│   ├── server.go      # HTTP handlers, unified /tx/submit, block producer
│   ├── server_test.go # Unit tests for /tx/submit endpoint
│   ├── chain.go       # State Shard blockchain + 2PC state (locks, pending credits, simulation locks)
│   ├── chain_test.go  # Unit tests for chain operations
│   ├── evm.go         # EVM state + SimulateCall for cross-shard detection (with mutex)
│   ├── tracking_statedb.go  # StateDB wrapper that tracks accessed addresses (thread-safe)
│   ├── security_fixes_test.go  # Tests for lock validation, atomic balance, thread safety
│   ├── receipt.go     # Transaction receipt storage
│   └── jsonrpc.go     # JSON-RPC compatibility (Foundry)
├── orchestrator/
│   ├── service.go     # HTTP handlers + block producer + vote collection
│   ├── service_test.go  # Tests for pointer aliasing, broadcast concurrency
│   ├── chain.go       # Orchestrator Shard blockchain + vote tracking
│   ├── chain_test.go  # Unit tests for orchestrator chain
│   ├── simulator.go   # EVM simulation for cross-shard transactions
│   ├── simulator_test.go  # Tests for simulation queue, EVM execution
│   ├── statedb.go     # SimulationStateDB - EVM state interface for simulation
│   ├── statedb_test.go  # Tests for refund handling, RwSet building
│   └── statefetcher.go # StateFetcher - fetches/caches state from State Shards
└── test/
    └── integration_test.go  # Integration tests for 2PC flow

cmd/
├── shard/main.go
└── orchestrator/main.go

contracts/              # Foundry project (normal Solidity)
scripts/                # Test scripts
docs/                   # Architecture and protocol documentation

Testing

# Run all Go tests
go test ./...

# Run with verbose output
go test -v ./...

# Run specific package tests
go test -v ./internal/shard/...
go test -v ./internal/orchestrator/...
go test -v ./test/...

# Cross-shard transaction (integration)
python scripts/test_cross_shard.py

# State sharding (contract on one shard)
python scripts/test_state_sharding.py

Development

# Build
go build ./...

# Test
go test ./...

# Contracts (Foundry)
cd contracts
forge test
forge build

Design Decisions

Blocks are the unit of coordination - 2PC state in blocks, not memory
State Shards are isolated - No State ↔ State communication
Minimal correctness - Focus on 2PC mechanics, assume consensus solved
Incremental complexity - Blocks → 2PC in blocks → State proofs (future)
Standalone EVM - geth's vm + state packages, not full node

Ports

Service	Internal	External
Orchestrator	8080	8080
Shard 0	8545	8545
Shard 1	8545	8546
Shard 2	8545	8547
Shard 3	8545	8548
Shard 4	8545	8549
Shard 5	8545	8550

TODOs and Open Issues

Critical (Correctness)

Vote Timeout Handling
- Currently, if a State Shard never sends its vote, the tx stays in awaitingVotes forever
- Need: Timeout mechanism to abort stale transactions after N blocks
- Location: orchestrator/chain.go:awaitingVotes
Block Height Synchronization
- Shards and orchestrator produce blocks independently every 3s
- No guarantee that TpcResult is processed before next CtToOrder
- Need: Either block height references in TpcResult, or sequence numbers
~~Duplicate Vote Prevention~~ ✅ COMPLETED
- ~~Multiple State Shard blocks could contain the same TpcPrepare vote~~
- Implementation: First vote wins - RecordVote() ignores duplicate votes from same shard
Transaction Replay Protection
- Same tx ID could be submitted multiple times
- Need: Track processed tx IDs and reject duplicates

High Priority (Reliability)

~~Graceful Error Handling in Block Processing~~ ✅ COMPLETED
- Implementation: Transaction-level error handling with fetch error tracking in SimulationStateDB
HTTP Endpoint Deprecation
- Old HTTP 2PC endpoints still exist (/cross-shard/prepare, /commit, /abort, /credit)
- These are now only used for manual testing, not block-based 2PC
- Decision: Keep for debugging or remove for clarity?
Persistent State
- All state is in-memory (maps)
- Need: Persistence layer for production (LevelDB/RocksDB)

Medium Priority (Features)

Per-Recipient Amount Distribution
- Current: Each RwSet entry gets the full tx.Value credited
- Workaround: Multi-recipient credits work (list), but value isn't split
- Next step: Add Amount field to RwVariable for explicit amounts
~~RwSet Validation~~ ✅ COMPLETED
- Implementation: ReadSet populated during simulation, validated by State Shards before voting
- Location: internal/orchestrator/statedb.go, internal/shard/server.go:validateRwVariable()
Merkle Proof Generation
- ReadSetItem.Proof is always empty ([][]byte{})
- Need: MPT/Verkle proof generation for cross-shard state reads
Block Pruning
- All blocks are kept in memory forever
- Need: Prune old blocks after finality

Low Priority (Enhancements)

Metrics and Monitoring
- No Prometheus metrics or structured logging
- Need: Add metrics for tx latency, success rate, block production time
Configuration Management
- ✅ PARTIAL: Network latency simulation via config.json (see config/config.go)
- Still hardcoded: Block time (3s), shard count (8), ports
- Need: Config file support for block time, topology parameters
~~Testing~~ ✅ COMPLETED
- Implementation: Unit tests for chain.go, integration tests for 2PC flow
- Location: internal/shard/chain_test.go, internal/orchestrator/chain_test.go, test/integration_test.go

Architectural Questions

Same-shard transfers via 2PC? Currently, same-shard transfers use direct debit/credit. Cross-shard goes through 2PC. Should same-shard also use 2PC for consistency?
~~Contract execution across shards?~~ ✅ COMPLETED Implementation: EVM simulation discovers RwSet, all involved shards vote. Location: internal/orchestrator/simulator.go
Light client verification? Orchestrator Shard trusts State Shard blocks without verification. How to add light client proofs without full consensus?

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Ethereum State & Transaction Sharding

Quick Start

Architecture

Components

Transaction Flow

Block-Based 2PC Flow (Cross-Shard Only)

Communication Pattern

Implementation Status

✅ Implemented

⚠️ Deferred (Documented Assumptions)

API

State Shard Endpoints

Orchestrator Shard Endpoints

Data Structures

File Structure

Testing

Development

Design Decisions

Ports

TODOs and Open Issues

Critical (Correctness)

High Priority (Reliability)

Medium Priority (Features)

Low Priority (Enhancements)

Architectural Questions

FilesExpand file tree

README.md

Latest commit

History

README.md

File metadata and controls

Ethereum State & Transaction Sharding

Quick Start

Architecture

Components

Transaction Flow

Block-Based 2PC Flow (Cross-Shard Only)

Communication Pattern

Implementation Status

✅ Implemented

⚠️ Deferred (Documented Assumptions)

API

State Shard Endpoints

Orchestrator Shard Endpoints

Data Structures

File Structure

Testing

Development

Design Decisions

Ports

TODOs and Open Issues

Critical (Correctness)

High Priority (Reliability)

Medium Priority (Features)

Low Priority (Enhancements)

Architectural Questions