nimbus-eth1/nimbus/stateless_runner.nim

140 lines
6.2 KiB
Nim
Raw Normal View History

2023-11-01 03:32:09 +00:00
# Nimbus
# Copyright (c) 2023 Status Research & Development GmbH
# Licensed under either of
# * Apache License, version 2.0, ([LICENSE-APACHE](LICENSE-APACHE) or
# http://www.apache.org/licenses/LICENSE-2.0)
# * MIT license ([LICENSE-MIT](LICENSE-MIT) or
# http://opensource.org/licenses/MIT)
# at your option. This file may not be copied, modified, or distributed except
# according to those terms.
import
Unified database frontend integration (#1670) * Nimbus folder environment update details: * Integrated `CoreDbRef` for the sources in the `nimbus` sub-folder. * The `nimbus` program does not compile yet as it needs the updates in the parallel `stateless` sub-folder. * Stateless environment update details: * Integrated `CoreDbRef` for the sources in the `stateless` sub-folder. * The `nimbus` program compiles now. * Premix environment update details: * Integrated `CoreDbRef` for the sources in the `premix` sub-folder. * Fluffy environment update details: * Integrated `CoreDbRef` for the sources in the `fluffy` sub-folder. * Tools environment update details: * Integrated `CoreDbRef` for the sources in the `tools` sub-folder. * Nodocker environment update details: * Integrated `CoreDbRef` for the sources in the `hive_integration/nodocker` sub-folder. * Tests environment update details: * Integrated `CoreDbRef` for the sources in the `tests` sub-folder. * The unit tests compile and run cleanly now. * Generalise `CoreDbRef` to any `select_backend` supported database why: Generalisation was just missed due to overcoming some compiler oddity which was tied to rocksdb for testing. * Suppress compiler warning for `newChainDB()` why: Warning was added to this function which must be wrapped so that any `CatchableError` is re-raised as `Defect`. * Split off persistent `CoreDbRef` constructor into separate file why: This allows to compile a memory only database version without linking the backend library. * Use memory `CoreDbRef` database by default detail: Persistent DB constructor needs to import `db/core_db/persistent why: Most tests use memory DB anyway. This avoids linking `-lrocksdb` or any other backend by default. * fix `toLegacyBackend()` availability check why: got garbled after memory/persistent split. * Clarify raw access to MPT for snap sync handler why: Logically, `kvt` is not the raw access for the hexary trie (although this holds for the legacy database)
2023-08-04 11:10:09 +00:00
std/[options, times],
chronicles,
chronos,
nimcrypto,
stew/results,
json_rpc/rpcclient,
2023-06-04 06:00:50 +00:00
eth/[common/eth_types, p2p],
Unified database frontend integration (#1670) * Nimbus folder environment update details: * Integrated `CoreDbRef` for the sources in the `nimbus` sub-folder. * The `nimbus` program does not compile yet as it needs the updates in the parallel `stateless` sub-folder. * Stateless environment update details: * Integrated `CoreDbRef` for the sources in the `stateless` sub-folder. * The `nimbus` program compiles now. * Premix environment update details: * Integrated `CoreDbRef` for the sources in the `premix` sub-folder. * Fluffy environment update details: * Integrated `CoreDbRef` for the sources in the `fluffy` sub-folder. * Tools environment update details: * Integrated `CoreDbRef` for the sources in the `tools` sub-folder. * Nodocker environment update details: * Integrated `CoreDbRef` for the sources in the `hive_integration/nodocker` sub-folder. * Tests environment update details: * Integrated `CoreDbRef` for the sources in the `tests` sub-folder. * The unit tests compile and run cleanly now. * Generalise `CoreDbRef` to any `select_backend` supported database why: Generalisation was just missed due to overcoming some compiler oddity which was tied to rocksdb for testing. * Suppress compiler warning for `newChainDB()` why: Warning was added to this function which must be wrapped so that any `CatchableError` is re-raised as `Defect`. * Split off persistent `CoreDbRef` constructor into separate file why: This allows to compile a memory only database version without linking the backend library. * Use memory `CoreDbRef` database by default detail: Persistent DB constructor needs to import `db/core_db/persistent why: Most tests use memory DB anyway. This avoids linking `-lrocksdb` or any other backend by default. * fix `toLegacyBackend()` availability check why: got garbled after memory/persistent split. * Clarify raw access to MPT for snap sync handler why: Logically, `kvt` is not the raw access for the hexary trie (although this holds for the legacy database)
2023-08-04 11:10:09 +00:00
./core/chain/chain_desc,
./core/executor/process_block,
./db/[core_db, select_backend, ledger],
Unified database frontend integration (#1670) * Nimbus folder environment update details: * Integrated `CoreDbRef` for the sources in the `nimbus` sub-folder. * The `nimbus` program does not compile yet as it needs the updates in the parallel `stateless` sub-folder. * Stateless environment update details: * Integrated `CoreDbRef` for the sources in the `stateless` sub-folder. * The `nimbus` program compiles now. * Premix environment update details: * Integrated `CoreDbRef` for the sources in the `premix` sub-folder. * Fluffy environment update details: * Integrated `CoreDbRef` for the sources in the `fluffy` sub-folder. * Tools environment update details: * Integrated `CoreDbRef` for the sources in the `tools` sub-folder. * Nodocker environment update details: * Integrated `CoreDbRef` for the sources in the `hive_integration/nodocker` sub-folder. * Tests environment update details: * Integrated `CoreDbRef` for the sources in the `tests` sub-folder. * The unit tests compile and run cleanly now. * Generalise `CoreDbRef` to any `select_backend` supported database why: Generalisation was just missed due to overcoming some compiler oddity which was tied to rocksdb for testing. * Suppress compiler warning for `newChainDB()` why: Warning was added to this function which must be wrapped so that any `CatchableError` is re-raised as `Defect`. * Split off persistent `CoreDbRef` constructor into separate file why: This allows to compile a memory only database version without linking the backend library. * Use memory `CoreDbRef` database by default detail: Persistent DB constructor needs to import `db/core_db/persistent why: Most tests use memory DB anyway. This avoids linking `-lrocksdb` or any other backend by default. * fix `toLegacyBackend()` availability check why: got garbled after memory/persistent split. * Clarify raw access to MPT for snap sync handler why: Logically, `kvt` is not the raw access for the hexary trie (although this holds for the legacy database)
2023-08-04 11:10:09 +00:00
./evm/async/[data_sources, operations, data_sources/json_rpc_data_source],
"."/[vm_state, vm_types]
from strutils import parseInt, startsWith
from common/common import initializeEmptyDb
proc coinbasesOfThisBlockAndUncles(header: BlockHeader, body: BlockBody): seq[EthAddress] =
result.add header.coinbase
for uncle in body.uncles:
result.add(uncle.coinbase)
proc createVmStateForStatelessMode*(com: CommonRef, header: BlockHeader, body: BlockBody,
parentHeader: BlockHeader, asyncFactory: AsyncOperationFactory): Result[BaseVMState, string]
{.inline.} =
let vmState = BaseVMState()
if not vmState.statelessInit(parentHeader, header, com, asyncFactory):
return err("Cannot initialise VmState for block number " & $(header.blockNumber))
waitFor(ifNecessaryGetAccounts(vmState, coinbasesOfThisBlockAndUncles(header, body)))
ok(vmState)
proc statelesslyRunBlock*(asyncDataSource: AsyncDataSource, com: CommonRef, header: BlockHeader, body: BlockBody): Result[Hash256, string] =
try:
let t0 = now()
# FIXME-Adam: this doesn't feel like the right place for this; where should it go?
Unified database frontend integration (#1670) * Nimbus folder environment update details: * Integrated `CoreDbRef` for the sources in the `nimbus` sub-folder. * The `nimbus` program does not compile yet as it needs the updates in the parallel `stateless` sub-folder. * Stateless environment update details: * Integrated `CoreDbRef` for the sources in the `stateless` sub-folder. * The `nimbus` program compiles now. * Premix environment update details: * Integrated `CoreDbRef` for the sources in the `premix` sub-folder. * Fluffy environment update details: * Integrated `CoreDbRef` for the sources in the `fluffy` sub-folder. * Tools environment update details: * Integrated `CoreDbRef` for the sources in the `tools` sub-folder. * Nodocker environment update details: * Integrated `CoreDbRef` for the sources in the `hive_integration/nodocker` sub-folder. * Tests environment update details: * Integrated `CoreDbRef` for the sources in the `tests` sub-folder. * The unit tests compile and run cleanly now. * Generalise `CoreDbRef` to any `select_backend` supported database why: Generalisation was just missed due to overcoming some compiler oddity which was tied to rocksdb for testing. * Suppress compiler warning for `newChainDB()` why: Warning was added to this function which must be wrapped so that any `CatchableError` is re-raised as `Defect`. * Split off persistent `CoreDbRef` constructor into separate file why: This allows to compile a memory only database version without linking the backend library. * Use memory `CoreDbRef` database by default detail: Persistent DB constructor needs to import `db/core_db/persistent why: Most tests use memory DB anyway. This avoids linking `-lrocksdb` or any other backend by default. * fix `toLegacyBackend()` availability check why: got garbled after memory/persistent split. * Clarify raw access to MPT for snap sync handler why: Logically, `kvt` is not the raw access for the hexary trie (although this holds for the legacy database)
2023-08-04 11:10:09 +00:00
com.db.compensateLegacySetup()
2023-06-04 06:00:50 +00:00
let blockHash: Hash256 = header.blockHash
let asyncFactory = AsyncOperationFactory(maybeDataSource: some(asyncDataSource))
let parentHeader = waitFor(asyncDataSource.fetchBlockHeaderWithHash(header.parentHash))
com.db.persistHeaderToDbWithoutSetHeadOrScore(parentHeader)
info("statelessly running block", blockNumber=header.blockNumber, blockHash=blockHash, parentHash=header.parentHash, parentStateRoot=parentHeader.stateRoot, desiredNewStateRoot=header.stateRoot)
let vmState = createVmStateForStatelessMode(com, header, body, parentHeader, asyncFactory).get
let vres = processBlock(vmState, header, body)
2023-06-04 06:00:50 +00:00
let elapsedTime = now() - t0
let headerStateRoot = header.stateRoot
let vmStateRoot = rootHash(vmState.stateDB)
info("finished statelessly running the block", vres=vres, elapsedTime=elapsedTime, durationSpentDoingFetches=durationSpentDoingFetches, fetchCounter=fetchCounter, headerStateRoot=headerStateRoot, vmStateRoot=vmStateRoot)
if headerStateRoot != vmStateRoot:
return err("State roots do not match: header says " & $(headerStateRoot) & ", vmState says " & $(vmStateRoot))
else:
if vres == ValidationResult.OK:
return ok(blockHash)
else:
return err("Error while statelessly running a block")
except:
let ex = getCurrentException()
echo getStackTrace(ex)
error "Got an exception while statelessly running a block", exMsg = ex.msg
return err("Error while statelessly running a block: " & $(ex.msg))
proc statelesslyRunBlock*(asyncDataSource: AsyncDataSource, com: CommonRef, blockHash: Hash256): Result[Hash256, string] =
let (header, body) = waitFor(asyncDataSource.fetchBlockHeaderAndBodyWithHash(blockHash))
let r = statelesslyRunBlock(asyncDataSource, com, header, body)
if r.isErr:
error("stateless execution failed", hash=blockHash, error=r.error)
else:
info("stateless execution succeeded", hash=blockHash, resultingHash=r.value)
return r
proc fetchBlockHeaderAndBodyForHashOrNumber(asyncDataSource: AsyncDataSource, hashOrNum: string): Future[(BlockHeader, BlockBody)] {.async.} =
if hashOrNum.startsWith("0x"):
return await asyncDataSource.fetchBlockHeaderAndBodyWithHash(hashOrNum.toHash)
else:
return await asyncDataSource.fetchBlockHeaderAndBodyWithNumber(u256(parseInt(hashOrNum)))
proc statelesslyRunSequentialBlocks*(asyncDataSource: AsyncDataSource, com: CommonRef, initialBlockNumber: BlockNumber): Result[Hash256, string] =
info("sequential stateless execution beginning", initialBlockNumber=initialBlockNumber)
var n = initialBlockNumber
while true:
let (header, body) = waitFor(asyncDataSource.fetchBlockHeaderAndBodyWithNumber(n))
let r = statelesslyRunBlock(asyncDataSource, com, header, body)
if r.isErr:
error("stateless execution failed", n=n, h=header.blockHash, error=r.error)
return r
else:
info("stateless execution succeeded", n=n, h=header.blockHash, resultingHash=r.value)
n = n + 1
proc statelesslyRunBlock*(asyncDataSource: AsyncDataSource, com: CommonRef, hashOrNum: string): Result[Hash256, string] =
let (header, body) = waitFor(fetchBlockHeaderAndBodyForHashOrNumber(asyncDataSource, hashOrNum))
return statelesslyRunBlock(asyncDataSource, com, header, body)
proc statelesslyRunTransaction*(asyncDataSource: AsyncDataSource, com: CommonRef, headerHash: Hash256, tx: Transaction) =
let t0 = now()
2023-06-04 06:00:50 +00:00
let (header, body) = waitFor(asyncDataSource.fetchBlockHeaderAndBodyWithHash(headerHash))
# FIXME-Adam: this doesn't feel like the right place for this; where should it go?
Unified database frontend integration (#1670) * Nimbus folder environment update details: * Integrated `CoreDbRef` for the sources in the `nimbus` sub-folder. * The `nimbus` program does not compile yet as it needs the updates in the parallel `stateless` sub-folder. * Stateless environment update details: * Integrated `CoreDbRef` for the sources in the `stateless` sub-folder. * The `nimbus` program compiles now. * Premix environment update details: * Integrated `CoreDbRef` for the sources in the `premix` sub-folder. * Fluffy environment update details: * Integrated `CoreDbRef` for the sources in the `fluffy` sub-folder. * Tools environment update details: * Integrated `CoreDbRef` for the sources in the `tools` sub-folder. * Nodocker environment update details: * Integrated `CoreDbRef` for the sources in the `hive_integration/nodocker` sub-folder. * Tests environment update details: * Integrated `CoreDbRef` for the sources in the `tests` sub-folder. * The unit tests compile and run cleanly now. * Generalise `CoreDbRef` to any `select_backend` supported database why: Generalisation was just missed due to overcoming some compiler oddity which was tied to rocksdb for testing. * Suppress compiler warning for `newChainDB()` why: Warning was added to this function which must be wrapped so that any `CatchableError` is re-raised as `Defect`. * Split off persistent `CoreDbRef` constructor into separate file why: This allows to compile a memory only database version without linking the backend library. * Use memory `CoreDbRef` database by default detail: Persistent DB constructor needs to import `db/core_db/persistent why: Most tests use memory DB anyway. This avoids linking `-lrocksdb` or any other backend by default. * fix `toLegacyBackend()` availability check why: got garbled after memory/persistent split. * Clarify raw access to MPT for snap sync handler why: Logically, `kvt` is not the raw access for the hexary trie (although this holds for the legacy database)
2023-08-04 11:10:09 +00:00
com.db.compensateLegacySetup()
2023-06-04 06:00:50 +00:00
#let blockHash: Hash256 = header.blockHash
Unified database frontend integration (#1670) * Nimbus folder environment update details: * Integrated `CoreDbRef` for the sources in the `nimbus` sub-folder. * The `nimbus` program does not compile yet as it needs the updates in the parallel `stateless` sub-folder. * Stateless environment update details: * Integrated `CoreDbRef` for the sources in the `stateless` sub-folder. * The `nimbus` program compiles now. * Premix environment update details: * Integrated `CoreDbRef` for the sources in the `premix` sub-folder. * Fluffy environment update details: * Integrated `CoreDbRef` for the sources in the `fluffy` sub-folder. * Tools environment update details: * Integrated `CoreDbRef` for the sources in the `tools` sub-folder. * Nodocker environment update details: * Integrated `CoreDbRef` for the sources in the `hive_integration/nodocker` sub-folder. * Tests environment update details: * Integrated `CoreDbRef` for the sources in the `tests` sub-folder. * The unit tests compile and run cleanly now. * Generalise `CoreDbRef` to any `select_backend` supported database why: Generalisation was just missed due to overcoming some compiler oddity which was tied to rocksdb for testing. * Suppress compiler warning for `newChainDB()` why: Warning was added to this function which must be wrapped so that any `CatchableError` is re-raised as `Defect`. * Split off persistent `CoreDbRef` constructor into separate file why: This allows to compile a memory only database version without linking the backend library. * Use memory `CoreDbRef` database by default detail: Persistent DB constructor needs to import `db/core_db/persistent why: Most tests use memory DB anyway. This avoids linking `-lrocksdb` or any other backend by default. * fix `toLegacyBackend()` availability check why: got garbled after memory/persistent split. * Clarify raw access to MPT for snap sync handler why: Logically, `kvt` is not the raw access for the hexary trie (although this holds for the legacy database)
2023-08-04 11:10:09 +00:00
let transaction = com.db.beginTransaction()
defer: transaction.rollback() # intentionally throwing away the result of this execution
let asyncFactory = AsyncOperationFactory(maybeDataSource: some(asyncDataSource))
let parentHeader = waitFor(asyncDataSource.fetchBlockHeaderWithHash(header.parentHash))
com.db.persistHeaderToDbWithoutSetHeadOrScore(parentHeader)
2023-06-04 06:00:50 +00:00
let vmState = createVmStateForStatelessMode(com, header, body, parentHeader, asyncFactory).get
let r = processTransactions(vmState, header, @[tx])
if r.isErr:
error("error statelessly running tx", tx=tx, error=r.error)
else:
let elapsedTime = now() - t0
let gasUsed = vmState.cumulativeGasUsed
info("finished statelessly running the tx", elapsedTime=elapsedTime, gasUsed=gasUsed)