nimbus-eth2/beacon_chain
Jacek Sieka c7abc97545
harden and speed up block sync (#3358)
* harden and speed up block sync

The `GetBlockBy*` server implementation currently reads SSZ bytes from
database, deserializes them into a Nim object then serializes them right
back to SSZ - here, we eliminate the deser/ser steps and send the bytes
straight to the network. Unfortunately, the snappy recoding must still
be done because of differences in framing.

Also, the quota system makes one giant request for quota right before
sending all blocks - this means that a 1024 block request will be
"paused" for a long time, then all blocks will be sent at once causing a
spike in database reads which potentially will see the reading client
time out before any block is sent.

Finally, on the reading side we make several copies of blocks as they
travel through various queues - this was not noticeable before but
becomes a problem in two cases: bellatrix blocks are up to 10mb (instead
of .. 30-40kb) and when backfilling, we process a lot more of them a lot
faster.

* fix status comparisons for nodes syncing from genesis (#3327 was a bit
too hard)
* don't hit database at all for post-altair slots in GetBlock v1
requests
2022-02-07 19:20:10 +02:00
..
consensus_object_pools fork choice proposer boosting support (#3349) 2022-02-04 12:59:40 +01:00
eth1 update to engine API alpha.6 (#3351) 2022-02-04 12:12:19 +00:00
fork_choice fork choice proposer boosting support (#3349) 2022-02-04 12:59:40 +01:00
gossip_processing deactivate doppelganger protection during genesis (#3362) 2022-02-07 07:12:36 +02:00
networking harden and speed up block sync (#3358) 2022-02-07 19:20:10 +02:00
rpc rename MERGE_FORK_EPOCH to BELLATRIX_FORK_EPOCH (#3350) 2022-02-02 14:06:55 +01:00
spec rest: fix ssz preference string (#3357) 2022-02-04 15:26:27 +02:00
sync harden and speed up block sync (#3358) 2022-02-07 19:20:10 +02:00
validator_client rename mergeData to bellatrixData and mergeFork to bellatrixFork (#3315) 2022-01-24 16:23:13 +00:00
validators deactivate doppelganger protection during genesis (#3362) 2022-02-07 07:12:36 +02:00
.editorconfig Add .editorconfig file and remove trailing newlines 2021-09-14 12:12:49 +03:00
.gitignore Rename binaries; Mimic the original repo layout in the distribution 2020-11-09 11:38:52 +02:00
beacon_chain_db.nim Store finalized block roots in database (3s startup) (#3320) 2022-01-30 18:51:04 +02:00
beacon_chain_db_immutable.nim spec URL updates (#3342) 2022-01-31 09:56:59 +00:00
beacon_clock.nim spec URL & copyright year update (#3338) 2022-01-29 01:05:39 +00:00
beacon_node.nim val_mon: register locally produced aggregates (#3352) 2022-02-04 08:33:20 +01:00
beacon_node_status.nim EH cleanup (#2455) 2021-03-26 07:52:01 +01:00
conf.nim remove --subscribe-all{att,sync}nets (#3359) 2022-02-04 12:34:03 +00:00
extras.nim use ForkedHashedBeaconState in StateData (#2634) 2021-06-11 20:51:46 +03:00
filepath.nim EH cleanup (#2455) 2021-03-26 07:52:01 +01:00
interop.nim bump `nim-stint` (#2969) 2021-10-14 14:13:51 +03:00
nim.cfg Revert writing backfill root to database (#3215) 2021-12-21 11:40:14 +01:00
nimbus_beacon_node.nim remove --subscribe-all{att,sync}nets (#3359) 2022-02-04 12:34:03 +00:00
nimbus_beacon_node.nim.cfg Revert writing backfill root to database (#3215) 2021-12-21 11:40:14 +01:00
nimbus_binary_common.nim time: spring cleaning (#3262) 2022-01-11 11:01:54 +01:00
nimbus_signing_node.nim complete switch to beacon_chain/specs/datatypes/bellatrix (#3295) 2022-01-18 13:36:52 +00:00
nimbus_signing_node.nim.cfg Revert writing backfill root to database (#3215) 2021-12-21 11:40:14 +01:00
nimbus_validator_client.nim time: spring cleaning (#3262) 2022-01-11 11:01:54 +01:00
nimbus_validator_client.nim.cfg Revert writing backfill root to database (#3215) 2021-12-21 11:40:14 +01:00
sszdump.nim complete switch to beacon_chain/specs/datatypes/bellatrix (#3295) 2022-01-18 13:36:52 +00:00
statediff.nim time: spring cleaning (#3262) 2022-01-11 11:01:54 +01:00
statusbar.nim switch result = foo to expression return; unexport rest of logtrace symbols (#2788) 2021-08-17 09:51:39 +00:00
trusted_node_sync.nim ncli_db: import states and blocks from era file (#3313) 2022-01-25 09:28:26 +01:00
version.nim Version 1.6.0 2022-01-14 13:52:06 +02:00