nimbus-eth2/beacon_chain/spec
Jacek Sieka b8a32419b8
async batch verification (+40% sig verification throughput) (#5176)
* async batch verification

When batch verification is done, the main thread is blocked reducing
concurrency.

With this PR, the new thread signalling primitive in chronos is used to
offload the full batch verification process to a separate thread
allowing the main threads to continue async operations while the other
threads verify signatures.

Similar to previous behavior, the number of ongoing batch verifications
is capped to prevent runaway resource usage.

In addition to the asynchronous processing, 3 addition changes help
drive throughput:

* A loop is used for batch accumulation: this prevents a stampede of
small batches in eager mode where both the eager and the scheduled batch
runner would pick batches off the queue, prematurely picking "fresh"
batches off the queue
* An additional small wait is introduced for small batches - this helps
create slightly larger batches which make better used of the increased
concurrency
* Up to 2 batches are scheduled to the threadpool during high pressure,
reducing startup latency for the threads

Together, these changes increase attestation verification throughput
under load up to 30%.

* fixup

* Update submodules

* fix blst build issues (and a PIC warning)

* bump

---------

Co-authored-by: Zahary Karadjov <zahary@gmail.com>
2023-08-03 11:36:45 +03:00
..
datatypes reduce memory allocations during state transition (#5235) 2023-08-03 01:03:40 +02:00
eth2_apis Rename "data gas" to "blob gas" (#5216) 2023-08-02 22:07:57 +00:00
mev Local validator proposals (#5137) 2023-06-30 07:39:41 +00:00
presets automated consensus spec URL updating to v1.4.0-beta.0 (#5121) 2023-06-24 15:43:30 +00:00
beacon_time.nim delay `onSlotEnd` if there are duties (#5196) 2023-07-18 20:55:36 +02:00
beaconstate.nim reduce memory allocations during state transition (#5235) 2023-08-03 01:03:40 +02:00
block_id.nim remove Nim 1.2-compatible `push raise`s and update copyright notice years (#4528) 2023-01-20 14:14:37 +00:00
crypto.nim update some consensus-specs ref URLs to v1.4.0-beta.0 (#5163) 2023-07-05 16:02:55 +00:00
deposit_snapshots.nim eip4844 block/slot sanity and transition tests (#4399) 2022-12-08 02:07:41 +00:00
digest.nim automated v1.4.0-alpha.2 to v1.4.0-alpha.3 consensus spec URL updates (#5065) 2023-06-13 14:03:49 +00:00
engine_authentication.nim update engine API URLs from v1.0.0-beta.2 to beta.3 (#4828) 2023-04-17 20:11:28 +00:00
english_word_list.txt Bump testutils 2020-06-01 16:58:28 +03:00
eth2_merkleization.nim remove Nim 1.2-compatible `push raise`s and update copyright notice years (#4528) 2023-01-20 14:14:37 +00:00
eth2_ssz_serialization.nim Remove SignedBeaconBlockAndBlobsSidecar (#4683) 2023-03-02 15:12:04 +01:00
forks.nim Local validator proposals (#5137) 2023-06-30 07:39:41 +00:00
forks_light_client.nim Rename "data gas" to "blob gas" (#5216) 2023-08-02 22:07:57 +00:00
helpers.nim reduce memory allocations during state transition (#5235) 2023-08-03 01:03:40 +02:00
keystore.nim remove unused code (#5158) 2023-07-15 16:30:52 +00:00
light_client_sync.nim automated consensus spec URL updating to v1.4.0-beta.0 (#5121) 2023-06-24 15:43:30 +00:00
network.nim update some consensus spec URLs to v1.4.0-beta.0 (#5150) 2023-06-30 16:12:54 +00:00
presets.nim add `libnimbus_lc.a` C library (#5122) 2023-07-19 09:48:39 +02:00
signatures.nim Local validator proposals (#5137) 2023-06-30 07:39:41 +00:00
signatures_batch.nim async batch verification (+40% sig verification throughput) (#5176) 2023-08-03 11:36:45 +03:00
ssz_codec.nim remove Nim 1.2-compatible `push raise`s and update copyright notice years (#4528) 2023-01-20 14:14:37 +00:00
state_transition.nim Add Deneb handling to state_transition.makeBeaconBlock() (#5118) 2023-06-26 16:05:02 +00:00
state_transition_block.nim Rename "data gas" to "blob gas" (#5216) 2023-08-02 22:07:57 +00:00
state_transition_epoch.nim reduce memory allocations during state transition (#5235) 2023-08-03 01:03:40 +02:00
validator.nim update some consensus-specs ref URLs to v1.4.0-beta.0 (#5163) 2023-07-05 16:02:55 +00:00
weak_subjectivity.nim automated consensus spec URL updating to v1.4.0-beta.0 (#5121) 2023-06-24 15:43:30 +00:00