status-im
/
nimbus-eth2
mirror of https://github.com/status-im/nimbus-eth2.git
2
0
Fork 0
Code Issues Projects Releases Wiki Activity
4,826 Commits 113 Branches 115 Tags 213 MiB
Commit Graph

219 Commits

Author SHA1 Message Date
tersec 909c095e64
initial automated v1.2.0 -> v1.3.0-alpha.0 consensus spec URL update (#4296) 2022-11-08 02:37:28 +00:00
tersec 0a43c89cd2
run capella block sanity tests in CI (#4292) 2022-11-07 18:37:48 +00:00
tersec 5b46f0b723
add Capella support to Forked* (#4276) 
* add Capella support to Forked*

* remove cruft

* add `OnForkyBlockAdded`
 2022-11-02 16:23:30 +00:00
tersec 0cfc1b776e
add all missing epoch transition tests (#4269) 2022-10-28 08:02:33 +00:00
tersec 06ccf5b80c
capella test vector support (#4261) 2022-10-27 06:29:24 +00:00
tersec ce915c0a03
add beaconcha.in mainnet relay overview to guide (#4217) 2022-10-04 19:45:06 +03:00
tersec 57d68d0f72
re-enable randao checks (#4187) 
* re-enable randao checks

* use `asSigVerified` consistently

* fix spelling

* document why state_transition.makeBeaconBlock trusting signatures is safe
 2022-09-28 01:15:10 +00:00
tersec 3c03ba86c1
update consensus spec ref URLs to v1.2.0 (#4164) 2022-09-23 07:56:06 +00:00
Michael Sproul d6325b1ab5
api: add skip_randao_verification for produceBlockV2 (#3837) 2022-09-21 09:38:08 +02:00
tersec ad0d30093f
state/forkyState cleanup; spec URL updates; rm unused imports (#4052) 2022-08-31 13:29:34 +02:00
tersec 9ae796daed
Cache and resend, rather than recreate, builder API registrations (#4040) 2022-08-31 03:29:03 +03:00
tersec b60456fdf3
`withState`: `state` -> `forkyState` (#4038) 2022-08-26 22:47:40 +00:00
tersec c65eaca1bf
update spec ref URLs (#4005) 2022-08-20 16:03:32 +00:00
tersec c0f673dc09
spec ref URL updates: v1.2.0-rc.{1,2} for phase0/fork-choice altair/beacon-chain (#3986) 2022-08-18 07:25:33 +00:00
tersec 8274d5373b
update spec ref URLs (#3979) 2022-08-17 11:33:19 +00:00
Miran dfd4afc9f2
compatibility with Nim 1.4+ (#3888) 2022-07-29 10:53:42 +00:00
Jacek Sieka 48f01186d6
fix unnecessary HashList/HashArray cache invalidation (#3660) 
* SSZ `[]` -> `mitem`
* `[]` -> `item`

immutable access via mutable instance cannot rely on template
overloading, and `[]` cannot be a `func` because of special seq handling
in compiler.
 2022-05-30 13:30:42 +00:00
tersec bf1763fdef
consensus specs v1.2.0-rc.1 (#3661) 2022-05-25 13:49:29 +00:00
tersec b3d603f364
more CL spec URL updates to v1.2.0-rc.1 (#3657) 2022-05-24 08:26:35 +00:00
Jacek Sieka 1101c745b9
document and clean up `ValidatorIndex` usage (#3651) 
* document static vs dynamic range checking requirements
* add `vindices` iterator to iterate over valid validator indices in a
state
* clean up spec comments in general

* fixup

Co-authored-by: tersec <tersec@users.noreply.github.com>
 2022-05-23 23:39:08 +00:00
tersec c73239f60b
CL spec URL updates to v1.2.0-rc.1 (#3655) 2022-05-23 19:30:24 +00:00
zah 18968e9dfa
Bugfix: Invalid blocks were produced in the presence of invalid deposits (#3639) 
Since we were not verifying BLS signature in blocks that we produce,
we were failing to notice that some deposits need to be ignored (due
to having an invalid signature). Processing these deposits resulted
in a different ending state after the state transition which caused
our blocks to be rejected by the network.
 2022-05-17 22:56:15 +03:00
tersec 61ba308e13
stylecheck fixes (#3593) 2022-04-14 17:39:37 +02:00
Jacek Sieka bc4d4e1741
remove redundant `process_block` overloads (#3541) 2022-03-23 12:49:54 +00:00
tersec c18cd8ee0c
rename random -> prev_randao in Bellatrix for CL specs v1.1.10 (#3460) 2022-03-03 16:08:14 +00:00
tersec f0ada15dac
automated CL spec ref URL updates from v1.1.9 to v1.1.10 (#3455) 2022-03-02 10:00:21 +00:00
tersec 7de3f00f35
generic putCorruptState; {Merge=>Bellatrix}BeaconStateNoImmutableValidators (#3427) 2022-02-21 12:55:56 +01:00
tersec 79761c78a4
proc -> func, mainly in spec/state transition and adjecent modules (#3405) 2022-02-17 11:53:55 +00:00
Jacek Sieka 40fe8f5336 fix missing backfill when restarting node 
When node is restarted before backfill has started but after some blocks
have finalized with forward sync, we would not start the backfill.

* also clean up one last `SomeSome`
 2022-02-11 23:08:50 +02:00
tersec 0c814f49ee
rename sync_{committee_,}aggregate and execute_payload -> notify_new_payload (#3347) 2022-02-01 07:31:53 +00:00
EmilIvanichkovv 336403d18b Refactor `handleValidatorExitCommand` 
Make `validator exit command` work both with `JSON-RPC` and `REST` APIs
Fix problem with specifying rest-url using `localhost`
Change back exit error messages in `state_transition_block`
 2022-02-01 01:24:05 +02:00
Jacek Sieka d583e8e4ac
Store finalized block roots in database (3s startup) (#3320) 
* Store finalized block roots in database (3s startup)

When the chain has finalized a checkpoint, the history from that point
onwards becomes linear - this is exploited in `.era` files to allow
constant-time by-slot lookups.

In the database, we can do the same by storing finalized block roots in
a simple sparse table indexed by slot, bringing the two representations
closer to each other in terms of conceptual layout and performance.

Doing so has a number of interesting effects:

* mainnet startup time is improved 3-5x (3s on my laptop)
* the _first_ startup might take slightly longer as the new index is
being built - ~10s on the same laptop
* we no longer rely on the beacon block summaries to load the full dag -
this is a lot faster because we no longer have to look up each block by
parent root
* a collateral benefit is that we no longer need to load the full
summaries table into memory - we get the RSS benefits of #3164 without
the CPU hit.

Other random stuff:

* simplify forky block generics
* fix withManyWrites multiple evaluation
* fix validator key cache not being updated properly in chaindag
read-only mode
* drop pre-altair summaries from `kvstore`
* recreate missing summaries from altair+ blocks as well (in case
database has lost some to an involuntary restart)
* print database startup timings in chaindag load log
* avoid allocating superfluos state at startup
* use a recursive sql query to load the summaries of the unfinalized
blocks
 2022-01-30 18:51:04 +02:00
tersec 29e2169585
phase 0 & altair beacon chain and altair validator spec URL updates (#3339) 2022-01-29 13:53:31 +00:00
Emil efbd939108 Make `handleValidatorExitCommand` work with `REST API` 2022-01-25 14:00:29 +02:00
Zahary Karadjov 47f1f7ff1a More efficient reward data persistance; Address review comments 
The new format is based on compressed CSV files in two channels:

* Detailed per-epoch data
* Aggregated "daily" summaries

The use of append-only CSV file speeds up significantly the epoch
processing speed during data generation. The use of compression
results in smaller storage requirements overall. The use of the
aggregated files has a very minor cost in both CPU and storage,
but leads to near interactive speed for report generation.

Other changes:

- Implemented support for graceful shut downs to avoid corrupting
  the saved files.

- Fixed a memory leak caused by lacking `StateCache` clean up on each
  iteration.

- Addressed review comments

- Moved the rewards and penalties calculation code in a separate module

Required invasive changes to existing modules:

- The `data` field of the `KeyedBlockRef` type is made public to be used
  by the validator rewards monitor's Chain DAG update procedure.

- The `getForkedBlock` procedure from the `blockchain_dag.nim` module
  is made public to be used by the validator rewards monitor's Chain DAG
  update procedure.
 2022-01-18 01:56:56 +02:00
Zahary Karadjov 29aad0241b Precise per-component ETH-denominated rewards tracking 
This is an alternative take on https://github.com/status-im/nimbus-eth2/pull/3107
that aims for more minimal interventions in the spec modules at the expense of
duplicating more of the spec logic in ncli_db.
 2022-01-18 01:56:56 +02:00
tersec d878948ed2
update sync committee gossip validation comments; spec URL updates (#3280) 2022-01-13 13:46:08 +00:00
tersec 8242e57f41
initial migration from spec/datatypes/{merge => bellatrix} (#3249) 2022-01-06 12:25:35 +01:00
tersec 7594fa660d
copyright year and spec URL updates (#3243) 2022-01-05 11:07:14 +00:00
tersec da017d2ca5
update from phase0/altair v1.1.6 URLs to v1.1.8 spec URLs (#3238) 2022-01-04 03:57:15 +00:00
tersec d4680df8d2
convert between engine and consensus ExecutionPayloads (#3228) 
* convert between engine and consensus ExecutionPayloads
 2022-01-03 13:22:56 +01:00
tersec d7799ecdcc
v1.1.6 spec updates (#3206) 2021-12-17 06:56:33 +00:00
tersec 36ade1c1c6
v1.1.6 spec updates (minor, mostly URLs) (#3197) 2021-12-14 21:02:29 +00:00
Jacek Sieka 069bccd51b
batch-verify sync messages for a small perf boost (#3151) 
* batch-verify sync messages for a small perf boost

Generally reuses the same structure as attestation and aggregate
verification

* normalize `signatures` and `signature_batch` to use the same pattern
of verification
* normalize parameter names, order etc for signature stuff in general
* avoid calling `blsSign` directly - instead, go through `signatures`
consistently
 2021-12-09 14:56:54 +02:00
tersec d93a279565
engine API alpha.5 field renaming (#3174) 2021-12-09 11:18:38 +00:00
tersec cc51f3fd12
v1.1.{5 -> 6} phase 0 and altair spec URL updates (#3157) 2021-12-03 17:40:23 +00:00
Jacek Sieka cb3f1fd4fc
remove nbench (#3152) 
Used during initial development of the spec, `nbench` has fallen behind
and by and large been superceded by `block_sim`, `state_sim` and
`ncli_db bench`.
 2021-12-03 15:46:56 +01:00
Jacek Sieka 9c2f43ed0e
Speed up altair block processing 2x (#3115) 
* Speed up altair block processing >2x

Like #3089, this PR drastially speeds up historical REST queries and
other long state replays.

* cache sync committee validator indices
* use ~80mb less memory for validator pubkey mappings
* batch-verify sync aggregate signature (fixes #2985)
* document sync committee hack with head block vs sync message block
* add batch signature verification failure tests

Before:

```
../env.sh nim c -d:release -r ncli_db --db:mainnet_0/db bench --start-slot:-1000
All time are ms
     Average,       StdDev,          Min,          Max,      Samples,         Test
Validation is turned off meaning that no BLS operations are performed
    5830.675,        0.000,     5830.675,     5830.675,            1, Initialize DB
       0.481,        1.878,        0.215,       59.167,          981, Load block from database
    8422.566,        0.000,     8422.566,     8422.566,            1, Load state from database
       6.996,        1.678,        0.042,       14.385,          969, Advance slot, non-epoch
      93.217,        8.318,       84.192,      122.209,           32, Advance slot, epoch
      20.513,       23.665,       11.510,      201.561,          981, Apply block, no slot processing
       0.000,        0.000,        0.000,        0.000,            0, Database load
       0.000,        0.000,        0.000,        0.000,            0, Database store
```

After:

```
    7081.422,        0.000,     7081.422,     7081.422,            1, Initialize DB
       0.553,        2.122,        0.175,       66.692,          981, Load block from database
    5439.446,        0.000,     5439.446,     5439.446,            1, Load state from database
       6.829,        1.575,        0.043,       12.156,          969, Advance slot, non-epoch
      94.716,        2.749,       88.395,      100.026,           32, Advance slot, epoch
      11.636,       23.766,        4.889,      205.250,          981, Apply block, no slot processing
       0.000,        0.000,        0.000,        0.000,            0, Database load
       0.000,        0.000,        0.000,        0.000,            0, Database store
```

* add comment
 2021-11-24 13:43:50 +01:00
tersec 9e395011d9
update 22 spec URLs to v1.1.5 (#3111) 2021-11-18 08:08:00 +00:00
Jacek Sieka b22d86e161
REST/JSON-RPC: speed up several requests (#3092) 
REST/JSON-RPC and a few more also invalidate caches unnecessarily,
similar to https://github.com/status-im/nimbus-eth2/pull/3089

* avoid copying validator on balance request
 2021-11-12 23:29:28 +01:00