status-im/nimbus-eth2
2
0
Fork 0
mirror of https://github.com/status-im/nimbus-eth2.git synced 2025-01-09 13:56:23 +00:00
Code Issues Projects Releases Wiki Activity
5,722 Commits 108 Branches 116 Tags 242 MiB
Commit Graph

270 Commits

Author SHA1 Message Date
tersec
1d620f0123
consensus spec URL updates to v1.2.0-rc.3 (#4105) 2022-09-09 21:56:06 +00:00
tersec
e70d5e6194
update spec ref URLs in state_transition_epoch (#4016) 2022-08-23 13:06:12 +00:00
tersec
c0f673dc09
spec ref URL updates: v1.2.0-rc.{1,2} for phase0/fork-choice altair/beacon-chain (#3986) 2022-08-18 07:25:33 +00:00
tersec
8274d5373b
update spec ref URLs (#3979) 2022-08-17 11:33:19 +00:00
tersec
d62d13a23c
MEV block proposal (#3883) 
* MEV validator registration

* add nearby canary to detect new beacon chain forks

* remove special MEV graffiti

* web3signer support

* fix trace logging

* Nim 1.2 needs raises Defect

* use template rather than proc in REST JSON parsing

* use --payload-builder-enable and --payload-builder-url

* explicitly default MEV to disabled

* explicitly empty default value for payload builder URL

* revert attestation pool to unstable version
 2022-08-01 09:41:47 +03:00
Miran
dfd4afc9f2
compatibility with Nim 1.4+ (#3888) 2022-07-29 10:53:42 +00:00
tersec
ce6cbd84e2
rename verifyFinalization internal flag to strictVerification (#3866) 
* rename verifyFinalization internal flag to strictVerification

* Update beacon_chain/extras.nim

Co-authored-by: Etan Kissling <etan@status.im>

Co-authored-by: Etan Kissling <etan@status.im>
 2022-07-13 13:48:09 +00:00
Etan Kissling
2a2bcea70d
group justified and finalized Checkpoint (#3841) 
The justified and finalized `Checkpoint` are frequently passed around
together. This introduces a new `FinalityCheckpoint` data structure that
combines them into one.

Due to the large usage of this structure in fork choice, also took this
opportunity to update fork choice tests to the latest v1.2.0-rc.1 spec.
Many additional tests enabled, some need more work, e.g. EL mock blocks.
Also implemented `discard_equivocations` which was skipped in #3661,
and improved code reuse across fork choice logic while at it.
 2022-07-06 13:33:02 +03:00
zah
c24c737866
Fix #3650 (participation format in BeaconState result is out of spec) (#3776) 
* Fix #3650 (participation format in BeaconState result is out of spec)
* Make EpochParticipationFlags a distinct type
 2022-06-20 08:38:56 +03:00
tersec
65cecc50ca
cleanups: unused and duplicate imports, inconsistent naming conventions, URL updates (#3724) 2022-06-09 14:30:13 +00:00
tersec
7492f99f35
update CL spec URLs (#3696) 2022-06-03 09:01:58 +00:00
Jacek Sieka
48f01186d6
fix unnecessary HashList/HashArray cache invalidation (#3660) 
* SSZ `[]` -> `mitem`
* `[]` -> `item`

immutable access via mutable instance cannot rely on template
overloading, and `[]` cannot be a `func` because of special seq handling
in compiler.
 2022-05-30 13:30:42 +00:00
tersec
bf1763fdef
consensus specs v1.2.0-rc.1 (#3661) 2022-05-25 13:49:29 +00:00
tersec
b3d603f364
more CL spec URL updates to v1.2.0-rc.1 (#3657) 2022-05-24 08:26:35 +00:00
Jacek Sieka
1101c745b9
document and clean up ValidatorIndex usage (#3651) 
* document static vs dynamic range checking requirements
* add `vindices` iterator to iterate over valid validator indices in a
state
* clean up spec comments in general

* fixup

Co-authored-by: tersec <tersec@users.noreply.github.com>
 2022-05-23 23:39:08 +00:00
tersec
c73239f60b
CL spec URL updates to v1.2.0-rc.1 (#3655) 2022-05-23 19:30:24 +00:00
Jacek Sieka
011e0ca02f
era file verification (#3605) 
* era file verification

Implement and document era file verification

* era file states now come with block applied for easier verification
* clarify conflicting version handling
* document verification requirements
* remove count from name, use start-era, end-root to discover range

* remove obsolete todo

* abstract out block root loading
 2022-05-10 03:28:46 +03:00
tersec
c18cd8ee0c
rename random -> prev_randao in Bellatrix for CL specs v1.1.10 (#3460) 2022-03-03 16:08:14 +00:00
tersec
f0ada15dac
automated CL spec ref URL updates from v1.1.9 to v1.1.10 (#3455) 2022-03-02 10:00:21 +00:00
zah
9c1ff78f84
Fix a reward calculation bug affecting Prater epoch 64781 (#3428) 
To calculate the deltas correctly, the `process_inactivity_updates` function
must be called before the rewards and penalties processing code in order to
update the `inactivity_scores` field in the state. This would have required
duplicating more logic from the spec in the ncli modules, so I've decided to
pay the price of introducing a run-time copy of the state at each epoch which
eliminates the need to duplicate logic (both for this fix and the previous one).

Other changes:

* Fixes for the read-only mode of the `BeaconChainDb`
* Fix an uint64 underflow in the debug output procedure for printing
  balance deltas
* Allow Bellatrix states in the reward computation helpers
 2022-02-22 14:14:17 +02:00
tersec
79761c78a4
proc -> func, mainly in spec/state transition and adjecent modules (#3405) 2022-02-17 11:53:55 +00:00
tersec
c9aa1bee01
spec URL updates (#3342) 2022-01-31 09:56:59 +00:00
tersec
29e2169585
phase 0 & altair beacon chain and altair validator spec URL updates (#3339) 2022-01-29 13:53:31 +00:00
Zahary Karadjov
49b7daa39d [ncli_db] bugfix: take into account finalization delay in reward calc post Altair 
This fixes a problem affecting Prater's epoch 64444.
 2022-01-28 12:03:23 +02:00
tersec
2f635d3337
rename *_{MERGE => BELLATRIX} constant names (#3296) 2022-01-18 16:31:05 +00:00
Zahary Karadjov
47f1f7ff1a More efficient reward data persistance; Address review comments 
The new format is based on compressed CSV files in two channels:

* Detailed per-epoch data
* Aggregated "daily" summaries

The use of append-only CSV file speeds up significantly the epoch
processing speed during data generation. The use of compression
results in smaller storage requirements overall. The use of the
aggregated files has a very minor cost in both CPU and storage,
but leads to near interactive speed for report generation.

Other changes:

- Implemented support for graceful shut downs to avoid corrupting
  the saved files.

- Fixed a memory leak caused by lacking `StateCache` clean up on each
  iteration.

- Addressed review comments

- Moved the rewards and penalties calculation code in a separate module

Required invasive changes to existing modules:

- The `data` field of the `KeyedBlockRef` type is made public to be used
  by the validator rewards monitor's Chain DAG update procedure.

- The `getForkedBlock` procedure from the `blockchain_dag.nim` module
  is made public to be used by the validator rewards monitor's Chain DAG
  update procedure.
 2022-01-18 01:56:56 +02:00
Zahary Karadjov
29aad0241b Precise per-component ETH-denominated rewards tracking 
This is an alternative take on https://github.com/status-im/nimbus-eth2/pull/3107
that aims for more minimal interventions in the spec modules at the expense of
duplicating more of the spec logic in ncli_db.
 2022-01-18 01:56:56 +02:00
tersec
d878948ed2
update sync committee gossip validation comments; spec URL updates (#3280) 2022-01-13 13:46:08 +00:00
Jacek Sieka
805e85e1ff
time: spring cleaning (#3262) 
Time in the beacon chain is expressed relative to the genesis time -
this PR creates a `beacon_time` module that collects helpers and
utilities for dealing the time units - the new module does not deal with
actual wall time (that's remains in `beacon_clock`).

Collecting the time related stuff in one place makes it easier to find,
avoids some circular imports and allows more easily identifying the code
actually needs wall time to operate.

* move genesis-time-related functionality into `spec/beacon_time`
* avoid using `chronos.Duration` for time differences - it does not
support negative values (such as when something happens earlier than it
should)
* saturate conversions between `FAR_FUTURE_XXX`, so as to avoid
overflows
* fix delay reporting in validator client so it uses the expected
deadline of the slot, not "closest wall slot"
* simplify looping over the slots of an epoch
* `compute_start_slot_at_epoch` -> `start_slot`
* `compute_epoch_at_slot` -> `epoch`

A follow-up PR will (likely) introduce saturating arithmetic for the
time units - this is merely code moves, renames and fixing of small
bugs.
 2022-01-11 11:01:54 +01:00
tersec
0fd8bf7b56
spec URL updates (#3254) 2022-01-06 18:35:38 +00:00
tersec
8242e57f41
initial migration from spec/datatypes/{merge => bellatrix} (#3249) 2022-01-06 12:25:35 +01:00
Jacek Sieka
0e2b4e39fa
REST JSON support improvements (#3232) 
* support downloading blocks / states via JSON in addition to SSZ -
slow, but needed for infura support - SSZ is still used when server
supports it
* use common forked block/state reader in REST API
* fix stack overflows in REST JSON decoder
* fix invalid serialization of `justification_bits` in
`/eth/v1/debug/beacon/states` and `/eth/v2/debug/beacon/states`
* fix REST client to use `/eth/...` instead of `/api/eth/...`, update
"default" urls to expose REST api via `/eth` as well as this is what the
standard says - `/api` was added early on based on an example "base url"
in the spec that has been removed since
* expose Nimbus REST extensions via `/nimbus` in addition to
`/api/nimbus` to stay consistent with `/eth`
* fix invalid state root when reading states via REST
* fix recursive imports in `spec/ssz_codec`
* remove usages of `serialization.useCustomSerialization` - fickle
 2022-01-06 08:38:40 +01:00
tersec
da017d2ca5
update from phase0/altair v1.1.6 URLs to v1.1.8 spec URLs (#3238) 2022-01-04 03:57:15 +00:00
tersec
d7799ecdcc
v1.1.6 spec updates (#3206) 2021-12-17 06:56:33 +00:00
tersec
e6921f808f
cleanups, partly from kintsugi branch (#3161) 
* cleanups, partly from kintsugi branch

* re-export shortLog(EthBlock) and preserve exception messages in batchVerify and processBatch
 2021-12-05 17:32:41 +00:00
tersec
cc51f3fd12
v1.1.{5 -> 6} phase 0 and altair spec URL updates (#3157) 2021-12-03 17:40:23 +00:00
Jacek Sieka
cb3f1fd4fc
remove nbench (#3152) 
Used during initial development of the spec, `nbench` has fallen behind
and by and large been superceded by `block_sim`, `state_sim` and
`ncli_db bench`.
 2021-12-03 15:46:56 +01:00
Jacek Sieka
a223d62b07
Cleanups (#3123) 
Renames and cleanups split out from the validator monitoring branch, so
as to reduce conflict area vs other PR:s

* add constants for expected message timing
* name validators after the messages they validate, mostly, to make
grepping easier
* unify field naming of EpochInfo across forks to make cross-fork code
easier
 2021-11-25 13:20:36 +01:00
tersec
9e395011d9
update 22 spec URLs to v1.1.5 (#3111) 2021-11-18 08:08:00 +00:00
Jacek Sieka
00bbc8e0fe
speed up epoch processing 6x+ (#3089) 
* speed up epoch processing 6x+

This change above all helps contain long replay times on epoch change, reorg
and deep history inspection via REST/RPC

* most effective balances don't actually change due to MAX_EFFECTIVE_BALANCE
* ditto for inactivity scores
* avoid signature check for trusted sync aggregates

pre:
```
./ncli_db --db:mainnet_0/db bench --start-slot=-3200
All time are ms
     Average,       StdDev,          Min,          Max,      Samples,         Test
Validation is turned off meaning that no BLS operations are performed

    3468.621,        0.000,     3468.621,     3468.621,            1, Initialize DB
       0.357,        0.938,        0.171,       52.752,         3155, Load block from database
   15691.471,        0.000,    15691.471,    15691.471,            1, Load state from database
       6.100,        9.469,        0.033,      526.816,         3101, Advance slot, non-epoch
     579.131,        9.523,      566.936,      610.328,          100, Advance slot, epoch
      18.551,       16.317,       12.664,      136.668,         3155, Apply block, no slot processing
       0.000,        0.000,        0.000,        0.000,            0, Database load
       0.000,        0.000,        0.000,        0.000,            0, Database store
```

post:
```
     Average,       StdDev,          Min,          Max,      Samples,         Test
Validation is turned off meaning that no BLS operations are performed
    3488.541,        0.000,     3488.541,     3488.541,            1, Initialize DB
       0.369,        1.123,        0.183,       63.208,         3155, Load block from database
   13430.642,        0.000,    13430.642,    13430.642,            1, Load state from database
       6.522,        1.721,        0.034,       36.708,         3101, Advance slot, non-epoch
      89.074,        3.162,       83.573,      101.436,          100, Advance slot, epoch
      18.325,       18.346,       13.005,      145.040,         3155, Apply block, no slot processing
       0.000,        0.000,        0.000,        0.000,            0, Database load
       0.000,        0.000,        0.000,        0.000,            0, Database store
```

* Update beacon_chain/spec/state_transition_block.nim

Co-authored-by: zah <zahary@gmail.com>

* avoid copying validator data in accessor

```
    5291.227,        0.000,     5291.227,     5291.227,            1, Initialize DB
       0.436,        0.928,        0.138,       51.438,         3155, Load block from database
   11962.826,        0.000,    11962.826,    11962.826,            1, Load state from database
       6.477,        1.675,        0.037,       34.174,         3101, Advance slot, non-epoch
      76.633,        3.705,       71.106,       98.085,          100, Advance slot, epoch
      18.301,       18.593,       13.208,      149.153,         3155, Apply block, no slot processing
       0.000,        0.000,        0.000,        0.000,            0, Database load
       0.000,        0.000,        0.000,        0.000,            0, Database store
```

* work around compiler bug

Co-authored-by: zah <zahary@gmail.com>
 2021-11-11 20:24:29 +01:00
tersec
48eba59971
manually-verified v1.1.5 spec URL updates (#3068) 2021-11-09 08:54:59 +00:00
tersec
2e868dc2ba
mass/mechanical update of 1.1.4 phase0 and altair spec URLs to 1.1.5 (#3067) 2021-11-09 07:40:41 +00:00
tersec
eb3ad25859
mass/mechanical update of 1.1.3 phase/altair spec URLs to 1.1.4 (#3058) 2021-11-08 06:18:10 +00:00
Dustin Brody
023a6a9387 use v1.1.4 test vectors 2021-11-07 21:42:11 +02:00
Jacek Sieka
a086cf01ac
altair fork handling cleanups (#3050) 
* fix stack overflow crash in REST/debug/getStateV2
* introduce `ForkyXxx` for generic type matching of `Xxx` across
branches (SomeHashedBeaconState -> ForkyHashedBeaconState et al) -
`Some` is already used for other types of type classes
* consolidate function naming in BeaconChainDB, use some generics
* import `forks.nim` from other spec modules and move `Forked*` helpers
around to resolve circular imports
* remove `ForkedBeaconState`, use `ForkedHashedBeaconState` throughout
(less data shuffling between the types)
* fix several cases of states being stored on stack in tests, causing
random failures on some platforms
* remove reading json support from ncli - this should be ported to the
rest json reading instead (doesn't currently work because stack sizes)
 2021-11-05 08:34:34 +01:00
tersec
36e37bda40
v1.1.3 spec refs URLs (#3036) 2021-10-27 18:40:17 +00:00
tersec
968a5b3bec
some spec refs to v1.1.3 (#3032) 2021-10-26 18:42:48 +00:00
tersec
8307e9c601
mechanical non-merge v1.1.2 to v1.1.3 spec URL updates (#3030) 2021-10-26 16:44:23 +00:00
tersec
ecf2dbbcf3
add Altair and Merge rewards tests (#2995) 2021-10-18 16:54:17 +02:00
tersec
10981639f1
update 27 spec URLs to v1.1.2 (#2989) 2021-10-13 16:49:06 +00:00