nimbus-eth2/beacon_chain
Jacek Sieka 1ef7d237cc
Shared validator pubkey (#5883)
This PR allows sharing the pubkey data between validators by using a
thread-local cache for pubkey data, netting about a 400mb mem usage
reduction on holesky due to us keeping 3 permanent + several ephemeral
state copies in memory at all times and each state copy holding a full
validator.

The PR also introduces a hash cache for the key which gives ~14% speedup
for a full state `hash_tree_root` - the key makes up for a large part of
the `Validator` htr time.

Finally, the time it takes to copy a state goes down as well from ~80m
ms to ~60, for reasons similar to htr.

We use a `ptr` even if a `ref` could in theory have been used - there is
not much practical benefit to a `ref` (given it's mutable) while a `ptr`
is cheaper and easier to copy (when copying temporary states).

We could go further and cache a cooked pubkey but it turns out this is
quite intrusive - in all the relevant places, we're already using a
cooked key from the immutable validator data so there are no immediate
performance gains of doing so while managing the compressed -> cooked
key mapping would become more difficult - something for a future PR
perhaps.

Co-authored-by: Etan Kissling <etan@status.im>
2024-02-21 20:06:19 +01:00
..
consensus_object_pools rename new timing metrics, as `_total` suffix is implicit (#5917) 2024-02-20 20:34:41 +01:00
el add `{.push raises: [].}` to recently modified files (#5908) 2024-02-18 01:16:49 +00:00
fork_choice automated consensus spec URL updating to v1.4.0-beta.6 (#5804) 2024-01-20 11:19:47 +00:00
gossip_processing ensure reason field logging consistently uses string type (#5878) 2024-02-10 03:50:31 +01:00
libnimbus_lc automated consensus spec URL updating to v1.4.0-beta.6 (#5804) 2024-01-20 11:19:47 +00:00
networking Shared validator pubkey (#5883) 2024-02-21 20:06:19 +01:00
rpc add `{.push raises: [].}` to recently modified files (#5908) 2024-02-18 01:16:49 +00:00
spec Shared validator pubkey (#5883) 2024-02-21 20:06:19 +01:00
sync unrevert rest of https://github.com/status-im/nimbus-eth2/pull/5765 (#5867) 2024-02-09 09:35:41 +01:00
validator_client avoid `read`/`readError` in favor of `value`/`error` (#5904) 2024-02-19 10:52:35 +01:00
validators vmon: Missed block metric (#5913) 2024-02-20 06:40:18 +02:00
.editorconfig Add .editorconfig file and remove trailing newlines 2021-09-14 12:12:49 +03:00
.gitignore Rename binaries; Mimic the original repo layout in the distribution 2020-11-09 11:38:52 +02:00
beacon_chain_db.nim Shared validator pubkey (#5883) 2024-02-21 20:06:19 +01:00
beacon_chain_db_immutable.nim automated consensus spec URL updating to v1.4.0-beta.6 (#5804) 2024-01-20 11:19:47 +00:00
beacon_chain_db_light_client.nim verify `genesis_time` more strictly (fixes #1667) (#5694) 2024-01-06 15:26:56 +01:00
beacon_clock.nim add metric for database checkpoint duration (#5897) 2024-02-19 11:00:11 +01:00
beacon_node.nim implement blob_sidecar Beacon API streaming (#5728) 2024-01-13 11:52:13 +02:00
beacon_node_light_client.nim `...ExecutionPayloadHash` --> `...ExecutionBlockHash` (#5864) 2024-02-08 01:24:55 +01:00
beacon_node_status.nim verify `genesis_time` more strictly (fixes #1667) (#5694) 2024-01-06 15:26:56 +01:00
conf.nim rm some unnecessary stew/shims/net imports to reduce deprecation warnings (#5806) 2024-01-20 16:42:19 +00:00
conf_common.nim Dissect Windows specific code from beacon node. (#5612) 2024-01-13 12:53:53 +02:00
conf_light_client.nim verify `genesis_time` more strictly (fixes #1667) (#5694) 2024-01-06 15:26:56 +01:00
db_limits.nim verify `genesis_time` more strictly (fixes #1667) (#5694) 2024-01-06 15:26:56 +01:00
deposits.nim add `{.push raises: [].}` to recently modified files (#5908) 2024-02-18 01:16:49 +00:00
era_db.nim handle `Exception` during `EraFile.verify` (#5900) 2024-02-17 18:19:30 +01:00
extras.nim verify `genesis_time` more strictly (fixes #1667) (#5694) 2024-01-06 15:26:56 +01:00
filepath.nim verify `genesis_time` more strictly (fixes #1667) (#5694) 2024-01-06 15:26:56 +01:00
future_combinators.nim BN+LC+SN: Fix cancellation deprecate warnings. (#5455) 2023-09-22 11:06:27 +00:00
light_client.nim raises for gossip (#5808) 2024-01-22 17:34:54 +01:00
light_client_db.nim verify `genesis_time` more strictly (fixes #1667) (#5694) 2024-01-06 15:26:56 +01:00
nim.cfg chronos: bump (#5684) 2024-01-05 10:08:38 +01:00
nimbus_beacon_node.nim rename new timing metrics, as `_total` suffix is implicit (#5917) 2024-02-20 20:34:41 +01:00
nimbus_beacon_node.nim.cfg remove `news` leftovers (#4299) 2022-11-08 20:06:54 +00:00
nimbus_binary_common.nim Dissect Windows specific code from beacon node. (#5612) 2024-01-13 12:53:53 +02:00
nimbus_light_client.nim raises for gossip (#5808) 2024-01-22 17:34:54 +01:00
nimbus_light_client.nim.cfg remove `news` leftovers (#4299) 2022-11-08 20:06:54 +00:00
nimbus_signing_node.nim update some deprecated stew/results to results imports (#5743) 2024-01-16 22:37:14 +00:00
nimbus_signing_node.nim.cfg Revert writing backfill root to database (#3215) 2021-12-21 11:40:14 +01:00
nimbus_validator_client.nim verify `genesis_time` more strictly (fixes #1667) (#5694) 2024-01-06 15:26:56 +01:00
nimbus_validator_client.nim.cfg remove `news` leftovers (#4299) 2022-11-08 20:06:54 +00:00
sszdump.nim verify `genesis_time` more strictly (fixes #1667) (#5694) 2024-01-06 15:26:56 +01:00
statediff.nim Shared validator pubkey (#5883) 2024-02-21 20:06:19 +01:00
statusbar.nim verify `genesis_time` more strictly (fixes #1667) (#5694) 2024-01-06 15:26:56 +01:00
trusted_node_sync.nim add `{.push raises: [].}` to recently modified files (#5908) 2024-02-18 01:16:49 +00:00
version.nim Version v24.2.1 2024-02-20 05:49:56 +02:00
wallets.nim add `{.push raises: [].}` to recently modified files (#5908) 2024-02-18 01:16:49 +00:00
winservice.nim add `{.push raises: [].}` to recently modified files (#5908) 2024-02-18 01:16:49 +00:00