9c2f43ed0e
* Speed up altair block processing >2x Like #3089, this PR drastially speeds up historical REST queries and other long state replays. * cache sync committee validator indices * use ~80mb less memory for validator pubkey mappings * batch-verify sync aggregate signature (fixes #2985) * document sync committee hack with head block vs sync message block * add batch signature verification failure tests Before: ``` ../env.sh nim c -d:release -r ncli_db --db:mainnet_0/db bench --start-slot:-1000 All time are ms Average, StdDev, Min, Max, Samples, Test Validation is turned off meaning that no BLS operations are performed 5830.675, 0.000, 5830.675, 5830.675, 1, Initialize DB 0.481, 1.878, 0.215, 59.167, 981, Load block from database 8422.566, 0.000, 8422.566, 8422.566, 1, Load state from database 6.996, 1.678, 0.042, 14.385, 969, Advance slot, non-epoch 93.217, 8.318, 84.192, 122.209, 32, Advance slot, epoch 20.513, 23.665, 11.510, 201.561, 981, Apply block, no slot processing 0.000, 0.000, 0.000, 0.000, 0, Database load 0.000, 0.000, 0.000, 0.000, 0, Database store ``` After: ``` 7081.422, 0.000, 7081.422, 7081.422, 1, Initialize DB 0.553, 2.122, 0.175, 66.692, 981, Load block from database 5439.446, 0.000, 5439.446, 5439.446, 1, Load state from database 6.829, 1.575, 0.043, 12.156, 969, Advance slot, non-epoch 94.716, 2.749, 88.395, 100.026, 32, Advance slot, epoch 11.636, 23.766, 4.889, 205.250, 981, Apply block, no slot processing 0.000, 0.000, 0.000, 0.000, 0, Database load 0.000, 0.000, 0.000, 0.000, 0, Database store ``` * add comment |
||
|---|---|---|
| .. | ||
| README.md | ||
| attestation_pool.nim | ||
| block_clearance.nim | ||
| block_pools_types.nim | ||
| block_quarantine.nim | ||
| blockchain_dag.nim | ||
| exit_pool.nim | ||
| spec_cache.nim | ||
| sync_committee_msg_pool.nim | ||
README.md
Consensus object pools
This folder holds the various consensus object pools needed for a blockchain client.
Object in those pools have passed the "gossip validation" filter according to specs:
- blocks: https://github.com/ethereum/consensus-specs/blob/v1.1.5/specs/phase0/p2p-interface.md#beacon_block
- aggregate attestations: https://github.com/ethereum/consensus-specs/blob/v1.1.5/specs/phase0/p2p-interface.md#beacon_aggregate_and_proof
- unaggregated attestation: https://github.com/ethereum/consensus-specs/blob/v1.1.5/specs/phase0/p2p-interface.md#beacon_attestation_subnet_id
- voluntary exits: https://github.com/ethereum/consensus-specs/blob/v1.1.5/specs/phase0/p2p-interface.md#voluntary_exit
- Attester slashings: https://github.com/ethereum/consensus-specs/blob/v1.1.5/specs/phase0/p2p-interface.md#attester_slashing
- Proposer slashings: https://github.com/ethereum/consensus-specs/blob/v1.1.5/specs/phase0/p2p-interface.md#proposer_slashing
After "gossip validation" the consensus objects can be rebroadcasted as they are optimistically good, however for internal processing further verification is needed. For blocks, this means verifying state transition and all contained cryptographic signatures (instead of just the proposer signature). For other consensus objects, it is possible that gossip validation is a superset of consensus verification (TODO).
The pools presenet in this folder are:
- block_pools:
- block_quarantine: for seemingly valid blocks that are on a fork unknown to us.
- block_clearance: to verify (state_transition + cryptography) candidate blocks.
- blockchain_dag: an in-memory direct-acyclic graph of fully validated and verified blockchain candidates with the tail being the last finalized epoch. A block in the DAG MUST be in the fork choice and a block in the fork choice MUST be in the DAG (except for orphans following finalization). On finalization non-empty epoch blocks are stored in the beacon_chain_db.
- attestation_pool: Handles the attestation received from gossip and collect them for fork choice.
- exit_pool: Handle voluntary exits and forced exits (attester slashings and proposer slashings)