nimbus-eth2/beacon_chain
Jacek Sieka 9c2f43ed0e
Speed up altair block processing 2x (#3115)
* Speed up altair block processing >2x

Like #3089, this PR drastially speeds up historical REST queries and
other long state replays.

* cache sync committee validator indices
* use ~80mb less memory for validator pubkey mappings
* batch-verify sync aggregate signature (fixes #2985)
* document sync committee hack with head block vs sync message block
* add batch signature verification failure tests

Before:

```
../env.sh nim c -d:release -r ncli_db --db:mainnet_0/db bench --start-slot:-1000
All time are ms
     Average,       StdDev,          Min,          Max,      Samples,         Test
Validation is turned off meaning that no BLS operations are performed
    5830.675,        0.000,     5830.675,     5830.675,            1, Initialize DB
       0.481,        1.878,        0.215,       59.167,          981, Load block from database
    8422.566,        0.000,     8422.566,     8422.566,            1, Load state from database
       6.996,        1.678,        0.042,       14.385,          969, Advance slot, non-epoch
      93.217,        8.318,       84.192,      122.209,           32, Advance slot, epoch
      20.513,       23.665,       11.510,      201.561,          981, Apply block, no slot processing
       0.000,        0.000,        0.000,        0.000,            0, Database load
       0.000,        0.000,        0.000,        0.000,            0, Database store
```

After:

```
    7081.422,        0.000,     7081.422,     7081.422,            1, Initialize DB
       0.553,        2.122,        0.175,       66.692,          981, Load block from database
    5439.446,        0.000,     5439.446,     5439.446,            1, Load state from database
       6.829,        1.575,        0.043,       12.156,          969, Advance slot, non-epoch
      94.716,        2.749,       88.395,      100.026,           32, Advance slot, epoch
      11.636,       23.766,        4.889,      205.250,          981, Apply block, no slot processing
       0.000,        0.000,        0.000,        0.000,            0, Database load
       0.000,        0.000,        0.000,        0.000,            0, Database store
```

* add comment
2021-11-24 13:43:50 +01:00
..
consensus_object_pools Speed up altair block processing 2x (#3115) 2021-11-24 13:43:50 +01:00
eth1 Add support for HTTPS Web3 providers 2021-11-23 15:56:18 +02:00
fork_choice document how to run merge local testnet with Nethermind (#3110) 2021-11-17 20:45:39 +01:00
gossip_processing Speed up altair block processing 2x (#3115) 2021-11-24 13:43:50 +01:00
networking update 22 spec URLs to v1.1.5 (#3111) 2021-11-18 08:08:00 +00:00
rpc REST/JSON-RPC: speed up several requests (#3092) 2021-11-12 23:29:28 +01:00
spec Speed up altair block processing 2x (#3115) 2021-11-24 13:43:50 +01:00
sync Logging and startup improvements (#3038) 2021-11-02 18:06:36 +01:00
validator_client Better REST/RPC error messages (#3046) 2021-11-05 17:39:47 +02:00
validators Speed up altair block processing 2x (#3115) 2021-11-24 13:43:50 +01:00
.editorconfig Add .editorconfig file and remove trailing newlines 2021-09-14 12:12:49 +03:00
.gitignore Rename binaries; Mimic the original repo layout in the distribution 2020-11-09 11:38:52 +02:00
beacon_chain_db.nim ncli_db: add putState, putBlock (#3096) 2021-11-18 13:02:43 +01:00
beacon_chain_db_immutable.nim update 22 spec URLs to v1.1.5 (#3111) 2021-11-18 08:08:00 +00:00
beacon_clock.nim mass/mechanical update of 1.1.4 phase0 and altair spec URLs to 1.1.5 (#3067) 2021-11-09 07:40:41 +00:00
beacon_node.nim import cleanup (#2997) 2021-10-19 16:09:26 +02:00
beacon_node_status.nim EH cleanup (#2455) 2021-03-26 07:52:01 +01:00
conf.nim Make sync horizon configurable (#3113) 2021-11-18 20:35:26 +01:00
extras.nim use ForkedHashedBeaconState in StateData (#2634) 2021-06-11 20:51:46 +03:00
filepath.nim EH cleanup (#2455) 2021-03-26 07:52:01 +01:00
interop.nim bump `nim-stint` (#2969) 2021-10-14 14:13:51 +03:00
nimbus_beacon_node.nim Add support for HTTPS Web3 providers 2021-11-23 15:56:18 +02:00
nimbus_beacon_node.nim.cfg Logging and startup improvements (#3038) 2021-11-02 18:06:36 +01:00
nimbus_binary_common.nim Reduce the logging-related breaking changes (#3070) 2021-11-10 11:02:18 +02:00
nimbus_signing_process.nim import cleanup (#2997) 2021-10-19 16:09:26 +02:00
nimbus_validator_client.nim Logging and startup improvements (#3038) 2021-11-02 18:06:36 +01:00
nimbus_validator_client.nim.cfg Logging and startup improvements (#3038) 2021-11-02 18:06:36 +01:00
sszdump.nim ncli_db: add putState, putBlock (#3096) 2021-11-18 13:02:43 +01:00
statediff.nim update statediffs to work with Altair (#3061) 2021-11-09 16:17:57 +00:00
statusbar.nim switch result = foo to expression return; unexport rest of logtrace symbols (#2788) 2021-08-17 09:51:39 +00:00
version.nim Version 1.5.4 2021-11-10 11:55:57 +01:00