9 Commits

Author SHA1 Message Date
Jacek Sieka
7bbb0f4421
Stream blocks during import (#2937)
When running the import, currently blocks are loaded in batches into a
`seq` then passed to the importer as such.

In reality, blocks are still processed one by one, so the batching does
not offer any performance advantage. It does however require that the
client wastes memory, up to several GB, on the block sequence while
they're waiting to be processed.

This PR introduces a persister that accepts these potentially large
blocks one by one and at the same time removes a number of redundant /
unnecessary copies, assignments and resets that were slowing down the
import process in general.
2024-12-18 13:21:20 +01:00
Chirag Parmar
2838191c4f
replace deprecated types (#2704)
* partial commit

* fixes

* remove converters too

* revert changes on nimbus_verified_proxy

* revert changes in converter

* revert changes(re-xport) in rpc_types

* update copyright year

* replace types in other binaries

* chain config bug

* fix rebase conflict imcomplete buffer

* fix more rebase buffers

* remove ditto types and converters

* fix the tests

* update copyright year
2024-10-16 08:34:12 +07:00
Jacek Sieka
a03bb56bec
update web3 types to common eth (#2674) 2024-10-02 18:22:35 +02:00
Advaita Saha
e8542f951f
External syncer (#2574)
* inital external sync structure

* add to makefile

* feat: external syncer with healing process

* fix: suggestions

* network mapping

* minor changes

* forward sync

* jwt auth

* nrpc structure

* nrpc engine-api loader

* fix: suggestions

* fix: suggestions

* remove sync_db

* fix: edge cases and forks

* fix: rebase changes

* revert nimbus config changes
2024-09-29 18:48:11 +02:00
Jacek Sieka
c210885b73
eth: bump to new types (#2660)
This is a minimal set of changes to make things work with the new types
in nim-eth - this is the minimal PR that merely resolves
incompatibilities while the full change set would include more cleanup
and migration.
2024-09-29 14:37:09 +02:00
Jacek Sieka
d72a73de8b
avoid digest when loading era block (#2572)
Computing the digest is unnecessary but takes a little bit of time -
remove computation and reduce mem usage slightly when loading era blocks
2024-08-20 15:23:14 +02:00
Jacek Sieka
bdc86b3fd4
small cleanups (#2526)
* remove some redundant EH
* avoid pessimising move (introduces a copy in this case!)
* shift less data around when reading era files (reduces stack usage)
2024-07-26 12:32:01 +07:00
Advaita Saha
08bbb0079f
faster slot finding in nimbus import (#2491)
* faster slot finding in nimbus import

* feat: blocknumber based slot finding

* fix: formatting

* added comments

* fix: added is_execution_block

* added comment
2024-07-22 21:17:07 +00:00
Advaita Saha
25af347dfd
Shift era helpers to a different file (#2475)
* shift helpers to a different file

* fix: few logic fixed for transition from era1 to era
2024-07-12 03:15:14 +00:00