2
0
mirror of https://github.com/status-im/nimbus-eth2.git synced 2025-01-13 07:45:39 +00:00

1852 Commits

Author SHA1 Message Date
Jacek Sieka
1ef7d237cc
Shared validator pubkey ()
This PR allows sharing the pubkey data between validators by using a
thread-local cache for pubkey data, netting about a 400mb mem usage
reduction on holesky due to us keeping 3 permanent + several ephemeral
state copies in memory at all times and each state copy holding a full
validator.

The PR also introduces a hash cache for the key which gives ~14% speedup
for a full state `hash_tree_root` - the key makes up for a large part of
the `Validator` htr time.

Finally, the time it takes to copy a state goes down as well from ~80m
ms to ~60, for reasons similar to htr.

We use a `ptr` even if a `ref` could in theory have been used - there is
not much practical benefit to a `ref` (given it's mutable) while a `ptr`
is cheaper and easier to copy (when copying temporary states).

We could go further and cache a cooked pubkey but it turns out this is
quite intrusive - in all the relevant places, we're already using a
cooked key from the immutable validator data so there are no immediate
performance gains of doing so while managing the compressed -> cooked
key mapping would become more difficult - something for a future PR
perhaps.

Co-authored-by: Etan Kissling <etan@status.im>
2024-02-21 20:06:19 +01:00
tersec
ffbc8d1466
refactor epoch state transition to facilitate individual validator balance change calculations () 2024-02-20 05:14:52 +00:00
tersec
87ae60f780
search for validator indices backwards while processing deposits () 2024-02-20 06:34:57 +02:00
Etan Kissling
4fc1550d0f
add {.push raises: [].} to recently modified files ()
Status Nim style mandates `{.push raises: []}.` at start of modules.
Ensure that's the case so that exceptions are properly tracked.

- https://status-im.github.io/nim-style-guide/errors.exceptions.html
- https://github.com/status-im/nim-eth/pull/614#discussion_r1220906149
2024-02-18 01:16:49 +00:00
tersec
e410fe0052
https://github.com/ethereum/consensus-specs/pull/3600 () 2024-02-17 09:02:50 +00:00
tersec
ea29e0afc8
use 1.4.0-beta.7-hotfix consensus spec test vectors () 2024-02-16 04:49:18 +00:00
Jacek Sieka
9aabca6a64
Clean up debug/heads v2 types () 2024-02-07 17:51:12 +01:00
Jacek Sieka
47704bde14
raises for beacon validators & router ()
Changes here are more significant because of some good old tech debt in
block production which has grown quite hairy - the reduction in
exception handling at least provides some steps in the right direction.
2024-02-07 12:26:04 +01:00
Etan Kissling
94a65c2a9e
log extra_data instead of extra_data_len for ExecutionPayload ()
Add more details to execution payload logs, reusing the same facilities
that we already use for `GraffitiBytes`.
2024-02-07 10:09:25 +01:00
Etan Kissling
41403022bb
prevent accidentally hashing BeaconState/BeaconBlock in Deneb ()
Extend protection against accidentally calling computationally expensive
functions when a cache is available to Deneb, as done for earlier forks.
2024-02-06 19:57:53 +01:00
Eugene Kabanov
21efe7e060
VC: Use produceBlockV3 when its available. ()
* Initial commit.

* Add helper functions and publishBlock() implementations.

* Address review comments.
2024-02-02 15:24:40 +00:00
tersec
8b261dd3e0
fix blob_sidecar SSE versioned_hash field to be 0x-prefixed hex () 2024-01-31 04:50:24 +01:00
tersec
87052eba4e
implement getBlindedBlock REST API () 2024-01-31 03:18:55 +00:00
tersec
128834a8eb
use RestPlainResponse to improve builder API rerror reporting () 2024-01-24 23:27:22 +00:00
tersec
d8a2690a92
update builder API spec reference URLs to v0.4.0 () 2024-01-22 08:36:46 +01:00
tersec
4ec36e0670
Revert "use RestPlainResponse to improve builder API rerror reporting" ()
* Revert "use `RestPlainResponse` to improve builder API rerror reporting"

* Update rest_deneb_mev_calls.nim

copyright year linting

* Update rest_capella_mev_calls.nim

more copyright year linting
2024-01-21 22:39:45 +00:00
tersec
00cd032f7d
rm duplicate imports () 2024-01-21 10:21:01 +00:00
tersec
195a7525fa
revert non-PR commit 2024-01-21 07:06:09 +00:00
tersec
172374580d
rm duplicate imports 2024-01-21 07:03:42 +00:00
tersec
6c53dc1e11
automated consensus spec URL updating to v1.4.0-beta.6 () 2024-01-20 11:19:47 +00:00
tersec
545fb17649
use RestPlainResponse to improve builder API rerror reporting () 2024-01-19 03:20:47 +00:00
Etan Kissling
006fa59ec2
use consensus spec v1.4.0-beta.6 test vectors () 2024-01-18 18:14:44 +00:00
Etan Kissling
62ee92a094
bolster BlobSidecar syncing on incomplete responses ()
Avoid marking blocks invalid when corresponding `blobSidecarsByRange`
returns an incomplete / incorrect response while syncing. The block
itself may still be valid in that scenario.
2024-01-18 15:45:10 +01:00
tersec
36545e1d84
remove expensive logging from function called in prepareBeaconProposer inner loop () 2024-01-17 22:58:46 +00:00
Jacek Sieka
d5785677a8
split out eth1chain into its own module ()
reduces import junk in some places - more could be done here
2024-01-17 15:26:16 +01:00
Etan Kissling
11ebbe0cac
fix LC header production for Deneb ()
The `blob_gas_used` field was not properly populated when constructing
Deneb light client data. This is due to  not applying the change to
the entire codebase when the new field got introduced, and due to 
not catching that oversight in other modules. Also reviewed codebase and
discovered that `shortLog` for Deneb execution payloads has same bug.
2024-01-16 22:54:20 +00:00
tersec
cf1bec7670
update some deprecated stew/results to results imports () 2024-01-16 22:37:14 +00:00
Etan Kissling
7443a4ac08
load Ethereum mainnet KZG setup on Gnosis networks ()
Gnosis networks re-use the trusted setup from Ethereum mainnet.
Load it to support Deneb.
2024-01-16 01:58:07 +01:00
tersec
3541cfe020
remove extraneous length checks in KZG batch proofs ()
* remove extraneous length checks in KZG batch proofs

* re-add winservice import but only for Windows, to avoid UnusedImport warning

* also uses establishWindowsService
2024-01-15 16:53:34 +01:00
Jacek Sieka
b98f46c04d
Avoid global in p2p macro (fixes ) ()
* Avoid global in p2p macro (fixes )

* copy p2p macro to this repo and start de-crufting it
* make protocol registration dynamic, removing light client hacks et al
* split out light client protocol into its own file

* cleanups

* Option -> Opt
* remove more cruft

* further split beacon_sync

this allows the light client to respond to peer metadata messages
without exposing the block sync protocol

* better protocol init

* "constant" protocol index

* avoid casts

* copyright

* move some discovery code to discovery

* avoid extraneous data copy when sending chunks

* remove redundant forkdigest field

* document how to connect to a specific peer
2024-01-13 11:54:24 +02:00
tersec
69af8f943e
implement blob_sidecar Beacon API streaming () 2024-01-13 11:52:13 +02:00
tersec
1559a09184
add more logging of KZG commitments, proofs, and blobs () 2024-01-13 07:44:40 +01:00
Etan Kissling
16256a5230
display current fork (+ next fork if applicable) in slot start / status ()
Extend slot start message and default status bar with information about
current head fork and the next fork transition (corresponding to head).
This is useful to know whether a synced client is aware of a future fork
and can also be useful when syncing from old forks to follow progress
across the various forks.

```
 peers: 8 ❯ finalized: 741c2ce2:230474 ❯ head: b330f58b:230477:20 ❯ fork: Capella (next: Deneb:231680) ❯ time: 230599:24 (7379192) ❯ sync: 00h24m (99.63%) 2.6492slots/s (QwQUwQPQDQ:7375263)/opt
```

```
INF 2024-01-12 12:18:00.001+01:00 Slot start                                 topics="beacnde" slot=7379190 epoch=230599 fork="Capella (next: Deneb:231680)" sync="--h--m (99.62%) 0.0000slots/s (wwwwwwwwww:7375167)/opt" peers=0 head=741c2ce2:7375168 finalized=230472:723abe7e delay=1ms861us
```
2024-01-12 21:40:34 +01:00
Jacek Sieka
e4a1ae67df
ssz: bump ()
height-based merkleizer
2024-01-11 18:34:44 +01:00
tersec
07455e67a9
proc to func in engine authentication () 2024-01-08 21:12:58 +00:00
Jacek Sieka
62cbdeefc5
verify genesis_time more strictly (fixes ) ()
Bogus values lead to crashes down the line when timers overflow
2024-01-06 15:26:56 +01:00
Etan Kissling
508f3b6368
add missing std/ prefix to more imports ()
Bumping some `std` imports to explicitly include the prefix.
Also add explicit `./` prefix for local directory imports.
2024-01-06 07:18:28 +01:00
Etan Kissling
7db95f047b
track latest LightClientUpdate only once fork choice selects it ()
Instead of tracking the latest `LightClientUpdate` across all branches,
track the latest one on the current branch as selected by fork choice.
2024-01-03 23:36:05 +01:00
tersec
cafa1f3d3d
rm unused RPC signatures replaced by Plain versions () 2024-01-03 05:58:34 +00:00
Etan Kissling
583782a061
emit proposer_slashing/attester_slashing SSE on beacon-API ()
Add support for slashings on the beacon-API event stream for compat with
beacon-API specs.

- https://github.com/ethereum/beacon-APIs/pull/376
2023-12-22 18:54:55 +01:00
Etan Kissling
a2081521f6
emit bls_to_execution_change SSE on beacon-API ()
With Capella, `bls_to_execution_change` SSE should be emitted on the
event stream whenever a new `SignedBLSToExecutionChange` is received.
Add this missing functionality for compatibility with beacon-API specs.

- https://github.com/ethereum/beacon-APIs/pull/248
2023-12-22 14:52:43 +01:00
zah
29b29e1945
Use the new strict Json flavors mechanism for RestJson ()
This requires all object types to be explicitly white-listed for
default serialization. The PR makes the minimal changes, although
a number of similar mechanisms in eth2_rest_serialization can now
be removed.
2023-12-19 17:44:27 +02:00
tersec
9c6ba7d142
consensus spec v1.4.0-beta.5 URL updates () 2023-12-16 03:27:06 +01:00
andri lim
15147cccb1
Bump nim-web3 to dcabb8f29ee55afedefdf93cd3e102bb1daee354 ()
* bump nim-web3 to dcabb8f29ee55afedefdf93cd3e102bb1daee354

also bump json-rpc to a8731e91bc336d930ac66f985d3b88ed7cf2a7d7
2023-12-12 22:15:00 +07:00
tersec
df902fd00f
log execution payload (header) block_hash and parent_hash in block shortLogs () 2023-12-10 01:25:51 +01:00
Etan Kissling
d8728b7b28
re-add comment for PayloadAttributesV2 usage ()
As followup from , ensure that we still keep the comment around
referring to the correct `forkchoiceUpdated` to use being driven by
the fork schedule.
2023-12-08 03:50:38 +01:00
Etan Kissling
0a5d9ee027
use PayloadAttributesV3 in nimbus_light_client for Deneb ()
* use `PayloadAttributesV3` in `nimbus_light_client` for Deneb

From Deneb onward, `forkchoiceUpdated` requires `PayloadAttributesV3`.
In `nimbus_light_client` we still used `PayloadAttributesV2`.

Also clean up two other locations that were already correctly using
`PayloadAttributesV3`, to reduce code duplication.

* fix letter case
2023-12-07 17:10:22 +00:00
tersec
4776fecc33
consensus spec v1.4.0-beta.5 URL updates () 2023-12-06 22:16:55 +00:00
tersec
9efb2958ec
automated consensus spec URL updating to v1.4.0-beta.5 () 2023-12-05 03:34:45 +01:00
tersec
1791b0ee0a
fix XDeclaredButNotUsed warnings () 2023-12-04 21:14:35 +00:00