status-im
/
nimbus-eth2
mirror of https://github.com/status-im/nimbus-eth2.git
2
0
Fork 0
Code Issues Projects Releases Wiki Activity
nimbus-eth2/beacon_chain/gossip_processing
History
Jacek Sieka 3cb31e66b4
set upper bound on EpochRef cache (#2403) 
* set upper bound on EpochRef cache

* max 32 EpochRef instances
* less memory waste in BlockRef by removing EpochRef seq that is mostly
unused (~20mb)
* less memory waste in dag block lookup by not keeping an extra copy of
digest (~70mb)
* fix `==` and `$` for Eth2Digest
* remove `ChainDAG.tmpState` (~50mb?)

all in all, this branch cuts mainnet memory usage by ~160-180mb and puts
limits on EpochRef cache usage - where normally it hovered around 950mb
before, it's now sitting at 600-700mb on my machine.

* docs
 		2021-03-17 11:17:15 +01:00
..
README.md Reorg (5/5) (#2377) 2021-03-05 14:12:00 +01:00
consensus_manager.nim add metric for nextActionWait (#2399) 2021-03-12 09:46:26 +00:00
eth2_processor.nim set upper bound on EpochRef cache (#2403) 2021-03-17 11:17:15 +01:00
gossip_to_consensus.nim Split Eth2Processor in prep for batching (#2396) 2021-03-11 11:10:57 +01:00
gossip_validation.nim centralize p2p validation in a single file and address https://github.com/status-im/nimbus-eth2/pull/2377#issuecomment-791313118 (#2383) 2021-03-06 08:32:55 +01:00

README.md

Gossip Processing

This folders hold a collection of modules to:

  • validate raw gossip data before
    • rebroadcasting them (potentially aggregated)
    • sending it to one of the consensus object pool

Validation

Gossip Validation is different from consensus verification in particular for blocks.

There are 2 consumers of validated consensus objects:

  • a ValidationResult.Accept output triggers rebroadcasting in libp2p
    • method validate(PubSub, message) in libp2p/protocols/pubsub/pubsub.nim in the
    • which was called by rpcHandler(GossipSub, PubSubPeer, RPCMsg)
  • a xyzValidator message enqueues the validated object in one of the processing queue in eth2_processor
    • blocksQueue: AsyncQueue[BlockEntry], (shared with request_manager and sync_manager)
    • attestationsQueue: AsyncQueue[AttestationEntry]
    • aggregatesQueue: AsyncQueue[AggregateEntry]

Those queues are then regularly processed to be made available to the consensus object pools.

Security concerns

As the first line of defense in Nimbus, modules must be able to handle burst of data that may come:

  • from malicious nodes trying to DOS us
  • from long periods of non-finality, creating lots of forks, attestations, forks