nimbus-eth2/beacon_chain/spec
Jacek Sieka 9c2f43ed0e
Speed up altair block processing 2x (#3115)
* Speed up altair block processing >2x

Like #3089, this PR drastially speeds up historical REST queries and
other long state replays.

* cache sync committee validator indices
* use ~80mb less memory for validator pubkey mappings
* batch-verify sync aggregate signature (fixes #2985)
* document sync committee hack with head block vs sync message block
* add batch signature verification failure tests

Before:

```
../env.sh nim c -d:release -r ncli_db --db:mainnet_0/db bench --start-slot:-1000
All time are ms
     Average,       StdDev,          Min,          Max,      Samples,         Test
Validation is turned off meaning that no BLS operations are performed
    5830.675,        0.000,     5830.675,     5830.675,            1, Initialize DB
       0.481,        1.878,        0.215,       59.167,          981, Load block from database
    8422.566,        0.000,     8422.566,     8422.566,            1, Load state from database
       6.996,        1.678,        0.042,       14.385,          969, Advance slot, non-epoch
      93.217,        8.318,       84.192,      122.209,           32, Advance slot, epoch
      20.513,       23.665,       11.510,      201.561,          981, Apply block, no slot processing
       0.000,        0.000,        0.000,        0.000,            0, Database load
       0.000,        0.000,        0.000,        0.000,            0, Database store
```

After:

```
    7081.422,        0.000,     7081.422,     7081.422,            1, Initialize DB
       0.553,        2.122,        0.175,       66.692,          981, Load block from database
    5439.446,        0.000,     5439.446,     5439.446,            1, Load state from database
       6.829,        1.575,        0.043,       12.156,          969, Advance slot, non-epoch
      94.716,        2.749,       88.395,      100.026,           32, Advance slot, epoch
      11.636,       23.766,        4.889,      205.250,          981, Apply block, no slot processing
       0.000,        0.000,        0.000,        0.000,            0, Database load
       0.000,        0.000,        0.000,        0.000,            0, Database store
```

* add comment
2021-11-24 13:43:50 +01:00
..
datatypes Speed up altair block processing 2x (#3115) 2021-11-24 13:43:50 +01:00
eth2_apis Speed up altair block processing 2x (#3115) 2021-11-24 13:43:50 +01:00
presets manually-verified v1.1.5 spec URL updates (#3068) 2021-11-09 08:54:59 +00:00
beaconstate.nim Speed up altair block processing 2x (#3115) 2021-11-24 13:43:50 +01:00
crypto.nim update 18 spec URLs to v1.1.2; switch from eth2.0-specs to consensus-specs (#2990) 2021-10-14 06:30:21 +00:00
digest.nim mass/mechanical update of 1.1.4 phase0 and altair spec URLs to 1.1.5 (#3067) 2021-11-09 07:40:41 +00:00
english_word_list.txt Bump testutils 2020-06-01 16:58:28 +03:00
eth2_merkleization.nim Use nim-ssz-serialization module and rm local ssz code 2021-11-10 13:37:24 +02:00
eth2_ssz_serialization.nim Use nim-ssz-serialization module and rm local ssz code 2021-11-10 13:37:24 +02:00
forks.nim Support starting from altair (#3054) 2021-11-10 13:39:08 +02:00
helpers.nim update 22 spec URLs to v1.1.5 (#3111) 2021-11-18 08:08:00 +00:00
keystore.nim nim-libp2p auto bump (#2840) 2021-10-21 13:01:29 +02:00
light_client_sync.nim mass/mechanical update of 1.1.4 phase0 and altair spec URLs to 1.1.5 (#3067) 2021-11-09 07:40:41 +00:00
network.nim manually-verified v1.1.5 spec URL updates (#3068) 2021-11-09 08:54:59 +00:00
presets.nim mass/mechanical update of 1.1.4 phase0 and altair spec URLs to 1.1.5 (#3067) 2021-11-09 07:40:41 +00:00
signatures.nim update 22 spec URLs to v1.1.5 (#3111) 2021-11-18 08:08:00 +00:00
signatures_batch.nim Speed up altair block processing 2x (#3115) 2021-11-24 13:43:50 +01:00
ssz_codec.nim Use nim-ssz-serialization module and rm local ssz code 2021-11-10 13:37:24 +02:00
state_transition.nim ncli_db: add putState, putBlock (#3096) 2021-11-18 13:02:43 +01:00
state_transition_block.nim Speed up altair block processing 2x (#3115) 2021-11-24 13:43:50 +01:00
state_transition_epoch.nim update 22 spec URLs to v1.1.5 (#3111) 2021-11-18 08:08:00 +00:00
validator.nim ncli_db: add putState, putBlock (#3096) 2021-11-18 13:02:43 +01:00
weak_subjectivity.nim eth2.0-specs -> consensus-specs repo rename (#2801) 2021-08-20 23:37:45 +00:00