nimbus-eth1/nimbus
Jacek Sieka 7bbb0f4421
Stream blocks during import (#2937)
When running the import, currently blocks are loaded in batches into a
`seq` then passed to the importer as such.

In reality, blocks are still processed one by one, so the batching does
not offer any performance advantage. It does however require that the
client wastes memory, up to several GB, on the block sequence while
they're waiting to be processed.

This PR introduces a persister that accepts these potentially large
blocks one by one and at the same time removes a number of redundant /
unnecessary copies, assignments and resets that were slowing down the
import process in general.
2024-12-18 13:21:20 +01:00
..
beacon Wire ForkedChainRef properly to TxPool (#2933) 2024-12-13 13:21:20 +07:00
common Remove `forkTx` and friends (#2951) 2024-12-18 11:56:46 +01:00
core Stream blocks during import (#2937) 2024-12-18 13:21:20 +01:00
db Stream blocks during import (#2937) 2024-12-18 13:21:20 +01:00
evm Stream blocks during import (#2937) 2024-12-18 13:21:20 +01:00
graphql Wire ForkedChainRef to graphql and rpc_utils (#2936) 2024-12-13 14:34:32 +07:00
rpc Remove `forkTx` and friends (#2951) 2024-12-18 11:56:46 +01:00
stateless replace deprecated types (#2704) 2024-10-16 08:34:12 +07:00
sync Reduce getCanonicalHead usage, and delegate to ForkedChain (#2948) 2024-12-18 11:04:23 +07:00
transaction Move EVM code initialization outside of newComputation (#2926) 2024-12-11 14:56:41 +01:00
utils Stream blocks during import (#2937) 2024-12-18 13:21:20 +01:00
common.nim implement better hardfork management 2022-12-02 13:51:42 +07:00
compile_info.nim Simplify LedgerRef: remove unnecessary abstraction (#2826) 2024-11-06 09:01:56 +07:00
config.nim Remove verifyFrom, vmState, and checkSeal from ChainRef (#2932) 2024-12-13 12:12:57 +07:00
constants.nim increase default gas limit from 30M to 36M (#2941) 2024-12-16 23:32:29 +00:00
errors.nim Remove `forkTx` and friends (#2951) 2024-12-18 11:56:46 +01:00
makefile Snap sync fix trie interpolation fringe condition (#1457) 2023-02-01 18:56:06 +00:00
nim.cfg import: read from era files (#2254) 2024-05-31 09:13:56 +02:00
nimbus_desc.nim Fix race condition on syncer termination (#2916) 2024-12-06 12:11:40 +00:00
nimbus_execution_client.nim Connect gasLimit from Config to CommonRef (#2946) 2024-12-17 10:48:31 +00:00
nimbus_import.nim Stream blocks during import (#2937) 2024-12-18 13:21:20 +01:00
rpc.nim Wire ForkedChainRef to graphql and rpc_utils (#2936) 2024-12-13 14:34:32 +07:00
tracer.nim Remove `forkTx` and friends (#2951) 2024-12-18 11:56:46 +01:00
transaction.nim Unify tx validation (#2777) 2024-10-26 09:19:48 +02:00
version.nim Feature: User configurable extraData when assemble a block (#2823) 2024-11-06 09:01:25 +07:00