nimbus-eth1/hive_integration/nodocker/rpc
Jacek Sieka 7bbb0f4421
Stream blocks during import (#2937)
When running the import, currently blocks are loaded in batches into a
`seq` then passed to the importer as such.

In reality, blocks are still processed one by one, so the batching does
not offer any performance advantage. It does however require that the
client wastes memory, up to several GB, on the block sequence while
they're waiting to be processed.

This PR introduces a persister that accepts these potentially large
blocks one by one and at the same time removes a number of redundant /
unnecessary copies, assignments and resets that were slowing down the
import process in general.
2024-12-18 13:21:20 +01:00
..
init hive: nodocker rpc simulator skeleton 2021-11-30 16:03:41 +07:00
client.nim Reduce declared but not used warnings (#2822) 2024-11-03 00:11:24 +00:00
rpc_sim.nim Stream blocks during import (#2937) 2024-12-18 13:21:20 +01:00
rpc_tests.nim replace deprecated types (#2704) 2024-10-16 08:34:12 +07:00
test_env.nim Reuse taskpool in simulator to prevent OOM (#2945) 2024-12-17 18:57:16 +07:00
vault.nim switch to Nim v2.0.12 (#2817) 2024-11-01 19:06:26 +00:00