eth/consensus : implement eccpow consensus engine#10
Open
mmingyeomm wants to merge 3253 commits intocryptoecc:worldlandfrom
Open
eth/consensus : implement eccpow consensus engine#10mmingyeomm wants to merge 3253 commits intocryptoecc:worldlandfrom
mmingyeomm wants to merge 3253 commits intocryptoecc:worldlandfrom
Conversation
It seems that the comment for CopyGas was wrongly associated to SloadGas.
In order to reduce the amount of code that is embedded into the keeper binary, I am removing all the verkle code that uses go-verkle and go-ipa. This will be followed by further PRs that are more like stubs to replace code when the keeper build is detected. I'm keeping the binary tree of course. This means that you will still see `isVerkle` variables all over the codebase, but they will be renamed when code is touched (i.e. this is not an invitation for 30+ AI slop PRs). --------- Co-authored-by: Gary Rong <garyrong0905@gmail.com>
) This is a new step in my crusade against the braindead fad of starting PR titles with a word that is completely redundant with github labels, thus wasting prime first-line real-estate for something that isn't necessary. I noticed that every single one of these PRs are low-quality AI-slop, so I think there is a strong case to be made for these PRs to be auto-closed. A message is added before closing the PR, redirecting to our contribution guidelines, so I expect quality first-time contributors to read them and reopen the PR. In the case of spam PRs, the author is unlikely to revisit a given PR, and so auto-closing might have a positive impact. That's an experiment worth trying, imo.
This pull request optimizes history indexing by splitting a single large database batch into multiple smaller chunks. Originally, the indexer will resolve a batch of state histories and commit all corresponding index entries atomically together with the indexing marker. While indexing more state histories in a single batch improves efficiency, excessively large batches can cause significant memory issues. To mitigate this, the pull request splits the mega-batch into several smaller batches and flushes them independently during indexing. However, this introduces a potential inconsistency that some index entries may be flushed while the indexing marker is not, and an unclean shutdown may leave the database in a partially updated state. This can corrupt index data. To address this, head truncation is introduced. After a restart, any excessive index entries beyond the expected indexing marker are removed, ensuring the index remains consistent after an unclean shutdown.
…33483) HeadSync kept reqFinalityEpoch entries for servers after receiving EvUnregistered, while other per-server maps were cleared. This left stale request.Server keys reachable from HeadSync, which can lead to a slow memory leak in setups that dynamically register and unregister servers. The fix adds deletion of the reqFinalityEpoch entry in the EvUnregistered handler. This aligns HeadSync with the cleanup pattern used by other sync modules and keeps the finality request bookkeeping strictly limited to currently registered servers.
This PR removes the version-check command and its associated logic as discussed in issue #31222. Removed versionCheckCommand from misccmd.go and main.go. Deleted version_check.go and its corresponding tests. Cleaned up testdata/vcheck directory (~800 lines of JSON/signatures removed). Verified build with make geth
This is used by cell-level dissemination (aka partial messages) to give the CL all blobs the EL knows about and let CL communicate efficiently about any other missing blobs. In other words, partial responses from the EL is useful now. See the related (closed) PR: ethereum/execution-apis#674 and the new PR: ethereum/execution-apis#719
…dRoot (#33486) Fix #33390 `setHeadBeyondRoot` was failing to invalidate finalized blocks because it compared against the original head instead of the rewound root. This fix updates the comparison to use the post-rewind block number, preventing the node from reporting a finalized block that no longer exists. Also added relevant test cases for it.
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com> Co-authored-by: Csaba Kiraly <csaba.kiraly@gmail.com>
Adds missing trienode freezer case to InspectFreezerTable, making it consistent with InspectFreezer which already supports it. Co-authored-by: m6xwzzz <maskk.weller@gmail.com>
Add NewPayloadEvent to track engine API newPayload block processing times and report them to ethstats. This enables monitoring of block processing performance. https://notes.ethereum.org/@savid/block-observability related: #33231 --------- Co-authored-by: MariusVanDerWijden <m.vanderwijden@live.de>
Check out https://hackmd.io/dg7rizTyTXuCf2LSa2LsyQ for more details
This logging was too intensive at debug level, it is better to have it at trace level only. Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
### Description Add a new `OnStateUpdate` hook which gets invoked after state is committed. ### Rationale For our particular use case, we need to obtain the state size metrics at every single block when fuly syncing from genesis. With the current state sizer, whenever the node is stopped, the background process must be freshly initialized. During this re-initialization, it can skip some blocks while the node continues executing blocks, causing gaps in the recorded metrics. Using this state update hook allows us to customize our own data persistence logic, and we would never skip blocks upon node restart. --------- Co-authored-by: Gary Rong <garyrong0905@gmail.com>
Adds BlobTxType and SetCodeTxType to GasPrice switch case, aligning with `MaxFeePerGas` and `MaxPriorityFeePerGas` handling. Co-authored-by: m6xwzzz <maskk.weller@gmail.com>
It's a PR based on #33303 and introduces an approach for trienode history indexing. --- In the current archive node design, resolving a historical trie node at a specific block involves the following steps: - Look up the corresponding trie node index and locate the first entry whose state ID is greater than the target state ID. - Resolve the trie node from the associated trienode history object. A naive approach would be to store mutation records for every trie node, similar to how flat state mutations are recorded. However, the total number of trie nodes is extremely large (approximately 2.4 billion), and the vast majority of them are rarely modified. Creating an index entry for each individual trie node would be very wasteful in both storage and indexing overhead. To address this, we aggregate multiple trie nodes into chunks and index mutations at the chunk level instead. --- For a storage trie, the trie is vertically partitioned into multiple sub tries, each spanning three consecutive levels. The top three levels (1 + 16 + 256 nodes) form the first chunk, and every subsequent three-level segment forms another chunk. ``` Original trie structure Level 0 [ ROOT ] 1 node Level 1 [0] [1] [2] ... [f] 16 nodes Level 2 [00] [01] ... [0f] [10] ... [ff] 256 nodes Level 3 [000] [001] ... [00f] [010] ... [fff] 4096 nodes Level 4 [0000] ... [000f] [0010] ... [001f] ... [ffff] 65536 nodes Vertical split into chunks (3 levels per chunk) Level0 [ ROOT ] 1 chunk Level3 [000] ... [fff] 4096 chunks Level6 [000000] ... [fffffff] 16777216 chunks ``` Within each chunk, there are 273 nodes in total, regardless of the chunk's depth in the trie. ``` Level 0 [ 0 ] 1 node Level 1 [ 1 ] … [ 16 ] 16 nodes Level 2 [ 17 ] … … [ 272 ] 256 nodes ``` Each chunk is uniquely identified by the path prefix of the root node of its corresponding sub-trie. Within a chunk, nodes are identified by a numeric index ranging from 0 to 272. For example, suppose that at block 100, the nodes with paths `[]`, `[0]`, `[f]`, `[00]`, and `[ff]` are modified. The mutation record for chunk 0 is then appended with the following entry: `[100 → [0, 1, 16, 17, 272]]`, `272` is the numeric ID of path `[ff]`. Furthermore, due to the structural properties of the Merkle Patricia Trie, if a child node is modified, all of its ancestors along the same path must also be updated. As a result, in the above example, recording mutations for nodes `00` and `ff` alone is sufficient, as this implicitly indicates that their ancestor nodes `[]`, `[0]` and `[f]` were also modified at block 100. --- Query processing is slightly more complicated. Since trie nodes are indexed at the chunk level, each individual trie node lookup requires an additional filtering step to ensure that a given mutation record actually corresponds to the target trie node. As mentioned earlier, mutation records store only the numeric identifiers of leaf nodes, while ancestor nodes are omitted for storage efficiency. Consequently, when querying an ancestor node, additional checks are required to determine whether the mutation record implicitly represents a modification to that ancestor. Moreover, since trie nodes are indexed at the chunk level, some trie nodes may be updated frequently, causing their mutation records to dominate the index. Queries targeting rarely modified trie nodes would then scan a large amount of irrelevant index data, significantly degrading performance. To address this issue, a bitmap is introduced for each index block and stored in the chunk's metadata. Before loading a specific index block, the bitmap is checked to determine whether the block contains mutation records relevant to the target trie node. If the bitmap indicates that the block does not contain such records, the block is skipped entirely.
Fixes #33369 This omits "topics" and "addresses" from the filter when they are unspecified. It is required for interoperability with some server implementations that cannot handle `null` for these fields.
This pull request introduces a mechanism to compress trienode history by storing only the node diffs between consecutive versions. - For full nodes, only the modified children are recorded in the history; - For short nodes, only the modified value is stored; If the node type has changed, or if the node is newly created or deleted, the entire node value is stored instead. To mitigate the overhead of reassembling nodes from diffs during history reads, checkpoints are introduced by periodically storing full node values. The current checkpoint interval is set to every 16 mutations, though this parameter may be made configurable in the future.
Allow the blobpool to accept blobs out of nonce order Previously, we were dropping blobs that arrived out-of-order. However, since fetch decisions are done on receiver side, out-of-order delivery can happen, leading to inefficiencies. This PR: - adds an in-memory blob tx storage, similar to the queue in the legacypool - a limited number of received txs can be added to this per account - txs waiting in the gapped queue are not processed further and not propagated further until they are unblocked by adding the previos nonce to the blobpool The size of the in-memory storage is currently limited per account, following a slow-start logic. An overall size limit, and a TTL is also enforced for DoS protection. --------- Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com> Co-authored-by: MariusVanDerWijden <m.vanderwijden@live.de>
…cts (#33242) - pass `rpc.BlockNumberOrHash` directly to `eth_getBlockReceipts` so `requireCanonical` and other fields survive - aligns `BlockReceipts` with other `ethclient` methods and re-enables canonical-only receipt queries
This PR fixes an issue where `evm statetest` would not verify the post-state root hash if the test case expected an exception (e.g. invalid transaction). The fix involves: 1. Modifying `tests/state_test_util.go` in the `Run` method. 2. When an expected error occurs (`err != nil`), we now check if `post.Root` is defined. 3. If defined, we recalculate the intermediate root from the current state (which is reverted to the pre-transaction snapshot upon error). 4. We use `GetChainConfig` and `IsEIP158` to ensure the correct state clearing rules are applied when calculating the root, avoiding regressions on forks that require EIP-158 state clearing. 5. If the calculated root mismatches the expected root, the test now fails. This ensures that state tests are strictly verified against their expected post-state, even for failure scenarios. Fixes issue #33527 --------- Co-authored-by: MariusVanDerWijden <m.vanderwijden@live.de>
The code was simply duplicate, so we can remove some code lines here. Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
This PR fixes an issue where the tx indexer would repeatedly try to “unindex” a block with a missing body, causing a spike in CPU usage. This change skips these blocks and advances the index tail. The fix was verified both manually on a local development chain and with a new test. resolves #33371
The coverage build path was generating go test commands with a bogus -tags flag that held the coverpkg value, so the run kept failing. I switched coverbuild to treat the optional argument as an override for -coverpkg and stopped passing coverpkg from the caller. Now the script emits a clean go test invocation that should actually succeed.
…cessors (#34056) Fix three issues in the binary trie NodeIterator: 1. Empty nodes now properly backtrack to parent and continue iteration instead of terminating the entire walk early. 2. `HashedNode` resolver handles `nil` data (all-zeros hash) gracefully by treating it as Empty rather than panicking. 3. Parent update after node resolution guards against stack underflow when resolving the root node itself. --------- Co-authored-by: tellabg <249254436+tellabg@users.noreply.github.com>
Fix incorrect key length calculation for `numHashPairings` in `InspectDatabase`, introduced in #34000. The `headerHashKey` format is `headerPrefix + num + headerHashSuffix` (10 bytes), but the check incorrectly included `common.HashLength`, expecting 42 bytes. This caused all number -- hash entries to be misclassified as unaccounted data.
…34067) Problem: The max-initcode sentinel moved from core to vm, but RPC pre-check mapping still depended on core.ErrMaxInitCodeSizeExceeded. This mismatch could surface inconsistent error mapping when oversized initcode is submitted through JSON-RPC. Solution: - Remove core.ErrMaxInitCodeSizeExceeded from the core pre-check error set. - Map max-initcode validation errors in RPC from vm.ErrMaxInitCodeSizeExceeded. - Keep the RPC error code mapping unchanged (-38025). Impact: - Restores consistent max-initcode error mapping after the sentinel move. - Preserves existing JSON-RPC client expectations for error code -38025. - No consensus, state, or protocol behavior changes.
This PR implements EIP-7708 according to the latest "rough consensus": ethereum/EIPs#9003 https://github.com/etan-status/EIPs/blob/fl-ethlogs/EIPS/eip-7708.md --------- Co-authored-by: Jared Wasinger <j-wasinger@hotmail.com> Co-authored-by: raxhvl <raxhvl@users.noreply.github.com> Co-authored-by: Gary Rong <garyrong0905@gmail.com>
In `setOpenTelemetry`, all other fields (Enabled, Endpoint, AuthUser, AuthPassword, InstanceID, Tags) are guarded by `ctx.IsSet()` checks, so they only override the config file when explicitly set via CLI flags. `SampleRatio` was the only field missing this guard, causing the flag default (`1.0`) to always overwrite whatever was loaded from the config file. - Fix OpenTelemetry `SampleRatio` being unconditionally overwritten by the CLI flag default value (`1.0`), even when the user did not pass `--rpc.telemetry.sample-ratio` - This caused config file values for `SampleRatio` to be silently ignored
conn.read() used the actual UDP packet source address for codec.Decode(), but conn.write() always used tc.remoteAddr. When the remote node is reachable via multiple Docker networks, the packet source IP differs from tc.remoteAddr, causing a session key lookup failure in the codec. Use tc.remoteAddr.String() consistently in conn.read() so the session cache key matches what was used during Encode.
Improve speed of import-history command by two orders of magnitude. Rework ImportHistory to collect up to 2500 blocks per flush instead of flushing after each block, reducing database commit overhead. --------- Co-authored-by: Sina Mahmoodi <itz.s1na@gmail.com>
Comment referenced NonceChangeTransaction which doesn't exist, should be NonceChangeAuthorization.
Leftover from d40a255 when return type changed from *txpool.Transaction to *types.Transaction.
Rename the comment so it matches the helper name.
This PR relocates the witness statistics into the witness itself, making it more self-contained.
This PR enables the block validation of keeper in the womir/openvm zkvm. It also fixes some issues related to building the executables in CI. Namely, it activates the build which was actually disabled, and also resolves some resulting build conflicts by fixing the tags. Co-authored-by: Leo <leo@powdrlabs.com>
…4059) `pool.signer.Sender(tx)` bypasses the sender cache used by types.Sender, which can force an extra signature recovery for every promotable tx (promotion runs frequently). Use `types.Sender(pool.signer, tx)` here to keep sender derivation cached and consistent.
Later on we can consider making these limits configurable if the use-case arose.
We can consider making this limit configurable if ever the need arose.
In this PR, we add support for protocol version eth/70, defined by EIP-7975. Overall changes: - Each response is buffered in the peer’s receipt buffer when the `lastBlockIncomplete` field is true. - Continued request uses the same request id of its original request(`RequestPartialReceipts`). - Partial responses are verified in `validateLastBlockReceipt`. - Even if all receipts for partial blocks of the request are collected, those partial results are not sinked to the downloader, to avoid complexity. This assumes that partial response and buffering occur only in exceptional cases. --------- Co-authored-by: Gary Rong <garyrong0905@gmail.com> Co-authored-by: Felix Lange <fjl@twurst.com>
Add persistent storage for Block Access Lists (BALs) in `core/rawdb/`. This provides read/write/delete accessors for BALs in the active key-value store. --------- Co-authored-by: Jared Wasinger <j-wasinger@hotmail.com> Co-authored-by: Gary Rong <garyrong0905@gmail.com>
This is a breaking change in the opcode (structLog) tracer. Several fields will have a slight formatting difference to conform to the newly established spec at: ethereum/execution-apis#762. The differences include: - `memory`: words will have the 0x prefix. Also last word of memory will be padded to 32-bytes. - `storage`: keys and values will have the 0x prefix. --------- Co-authored-by: Sina M <1591639+s1na@users.noreply.github.com>
This PR changes the blsync checkpoint init logic so that even if the initialization fails with a certain server and an error log message is printed, the server goes back to its initial state and is allowed to retry initialization after the failure delay period. The previous logic had an `ssDone` server state that did put the server in a permanently unusable state once the checkpoint init failed for an apparently permanent reason. This was not the correct behavior because different servers behave differently in case of overload and sometimes the response to a permanently missing item is not clearly distinguishable from an overload response. A safer logic is to never assume anything to be permanent and always give a chance to retry. The failure delay formula is also fixed; now it is properly capped at `maxFailureDelay`. The previous formula did allow the delay to grow unlimited if a retry was attempted immediately after each delay period.
Block overrides were to a great extent ignored by the gasestimator. This PR fixes that.
This PR implements the missing functionality for archive nodes by pruning stale index data. The current mechanism is relatively simple but sufficient for now: it periodically iterates over index entries and deletes outdated data on a per-block basis. The pruning process is triggered every 90,000 new blocks (approximately every 12 days), and the iteration typically takes ~30 minutes on a mainnet node. This mechanism is only applied with `gcmode=archive` enabled, having no impact on normal full node.
In this PR, the Database interface in `core/state` has been extended with one more function: ```go // Iteratee returns a state iteratee associated with the specified state root, // through which the account iterator and storage iterator can be created. Iteratee(root common.Hash) (Iteratee, error) ``` With this additional abstraction layer, the implementation details can be hidden behind the interface. For example, state traversal can now operate directly on the flat state for Verkle or binary trees, which do not natively support traversal. Moreover, state dumping will now prefer using the flat state iterator as the primary option, offering better efficiency. Edit: this PR also fixes a tiny issue in the state dump, marshalling the next field in the correct way.
Implement the snap/2 wire protocol with BAL serving --------- Co-authored-by: Gary Rong <garyrong0905@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
19 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
implements eccpow consensus engine for Worldland Network