234 Commits

Author SHA1 Message Date
tersec
073c544f0c
automated update from v1.3.0-rc.0 to v1.3.0-rc.1 consensus spec URLs (#4517) 2023-01-17 16:10:52 +00:00
henridf
727920a571
Refactor block/blobs types (#4491)
* Refactor block/blobs types

Use type system to enforce invariant that a pre-4844 block cannot have
a sidecar.

* Update beacon_chain/nimbus_beacon_node.nim

Co-authored-by: tersec <tersec@users.noreply.github.com>

* review feedback

Co-authored-by: tersec <tersec@users.noreply.github.com>
2023-01-16 16:26:48 +00:00
Etan Kissling
fda03548e3
use ForkedLightClientStore internally (#4512)
When running `nimbus_light_client`, we persist the latest header from
`LightClientStore.finalized_header` in a database across restarts.
Because the data format is derived from the latest `LightClientStore`,
this could lead to data being persisted in pre-release formats.

To enable us to test later `LightClientStore` versions on devnets,
transition to a `ForkedLightClientStore` internally that is only
migrated to newer forks on-demand (instead of starting at latest).
2023-01-16 16:53:45 +01:00
Etan Kissling
609227559f
LC data fork cleanup (#4506)
Distinguish between those code locations that need to be updated on each
light client data format change, and those others that should generally
be fine, as long as a valid light client object is processed.

The former are tagged with static assert for `LightClientDataFork.high`.

The latter are changed to `lcDataFork > LightClientDataFork.None` to
indicate that they depend only on presence of any valid object.
Also bundled a few minor cleanups and fixes.

Also add `Forky` type for `LightClientStore` and minor fixes / cleanups.
2023-01-14 22:19:50 +01:00
Etan Kissling
2324136552
add LightClientHeader wrapper (#4481)
The light client data structures were changed to accommodate additional
fields in future forks (e.g., to also hold execution data).

There is a minor change to the JSON serialization, where the `header`
properties are now nested inside a `LightClientHeader`.
The SSZ serialization remains compatible.

See https://github.com/ethereum/consensus-specs/pull/3190
and https://github.com/ethereum/beacon-APIs/pull/287
2023-01-13 16:46:35 +01:00
Etan Kissling
7e276937dc
make LC data fork aware (#4493)
In a future fork, light client data will be extended with execution info
to support more use cases. To anticipate such an upgrade, introduce
`Forky` and `Forked` types, and ready the database schema.
Because the mapping of sync committee periods to fork versions is not
necessarily unique (fork schedule not in sync with period boundaries),
an additional column is added to `period` -> `LightClientUpdate` table.
2023-01-12 18:11:38 +01:00
Jacek Sieka
6bfc766629
drop subset sync contributions in gossip (#4490)
* correctly report ignored contributions in metrics
* avoid counting subset contributions in vmon (bring in line with
attestation aggregates)
* avoid signature checks for subset attestations

A being a non-strict subset is a sufficient condition to ignore.
2023-01-12 15:08:08 +01:00
henridf
309f8690de
Wire up engine_newPayloadV3 (#4482)
* Wire up eip4844's newPayloadV3

* Add eip4844 test

* Update AllTests-mainnet.md and fix typo
2023-01-11 18:21:19 +00:00
Jacek Sieka
ba3db7aa5a
spec: Option -> Opt (#4488) 2023-01-11 12:29:21 +00:00
tersec
e28e1aeec8
a few consensus spec ref URL updates (#4483) 2023-01-10 16:14:17 +00:00
tersec
2dd3cd786f
consensus spec ref URL update v1.3.0-{alpha.2,rc.0}; copyright year update (#4477) 2023-01-09 22:44:44 +00:00
henridf
64878888bd
Blob storage (#4454)
* Blob storage

* fix indentation

* Fix build (none->Opt.none)

* putBlobs -> putBlobsSidecar

* getBlobs -> getBlobsSidecar

* Check blob correctness when storing a backfill block

* Blobs table: rename and conditionally create

* Check block<->blob match in storeBackfillBlock

* Use when .. toFork() to condition on type

* Check blob viability in block_processor.storeBlock()

* Fix build

* Review feedback
2023-01-09 18:42:10 +00:00
tersec
c5d1683f19
spec ref URL & copyright year updates (#4467) 2023-01-06 16:28:46 +00:00
Jacek Sieka
7c2ed5c609
Always-on optimistic mode (#4458)
With https://github.com/status-im/nimbus-eth2/pull/4420 implemented, the
checks that we perform are equivalent to those of a `SYNCING` EL - as
such, we can treat missing EL the same as SYNCING and proceed with an
optimistic sync.

This mode of operation significantly speeds up recovery after an offline
EL event because the CL is already synced and can immediately inform the
EL of the latest head.

It also allows using a beacon node for consensus archival queries
without an execution client.

* deprecate `--optimistic` flag
* log block details on EL error, soften log level because we can now
continue to operate
* `UnviableFork` -> `Invalid` when block hash verification fails -
failed hash verification is not a fork-related block issue
2023-01-04 15:51:14 +00:00
henridf
8251cc223d
eip4844 gossip (#4444)
* eip4844 gossip

* Check BLSFieldElement range validity in gossip validation

* lint/nits cleanup

* Use template to avoid an assignment with copy.

* More review feedback

* lint

* lint

* processSignedBeaconBlockAndBlobsSidecar: clean up error handling flow

* Undo factoring-out of beacon blocks validator installation
2023-01-04 12:34:15 +00:00
Jacek Sieka
75c7195bfd
Backfill only up to MIN_EPOCHS_FOR_BLOCK_REQUESTS blocks (#4421)
When backfilling, we only need to download blocks that are newer than
MIN_EPOCHS_FOR_BLOCK_REQUESTS - the rest cannot reliably be fetched from
the network and does not have to be provided to others.

This change affects only trusted-node-synced clients - genesis sync
continues to work as before (because it needs to construct a state by
building it from genesis).

Those wishing to complete a backfill should do so with era files
instead.
2022-12-23 08:42:55 +01:00
Etan Kissling
c91d9d61e2
validate EL block hash in EL simulation (#4420)
When simulating EL with `--optimistic` flag, perform block hash check.
2022-12-20 09:24:33 +01:00
tersec
bb4ea37baa
update EF consensus spec URLs from v1.3.0-alpha.1 to v1.3.0-alpha.2 (#4432) 2022-12-15 12:15:12 +00:00
tersec
7faef7827e
fix EIP4844 withBlck (#4411)
* fix EIP4844 withBlck

* don't raiseAssert by default
2022-12-14 18:30:56 +01:00
Jacek Sieka
6e2a02466e
unify bn/vc doppelganger detection (#4398)
* fix REST liveness endpoint responding even when gossip is not enabled
* fix VC exit code on doppelganger hit
* fix activation epoch not being updated correctly on long deposit
queues
* fix activation epoch being set incorrectly when updating validator
* move most implementation logic to `validator_pool`, add tests
* ensure consistent logging between VC and BN
* add docs
2022-12-09 17:05:55 +01:00
Etan Kissling
bbf1d6030c
add hooks for observing LC progress (#4401)
For Fluffy injection, add observer callbacks that get called whenever
new light client data is sucecssfully processed.

```
  proc onLightClientObject(
      lightClient: LightClient, obj: SomeLightClientObject) =
    info "New LC object", obj

  lightClient.bootstrapObserver =
    proc(lightClient: LightClient, obj: altair.LightClientBootstrap) =
      lightClient.onLightClientObject(obj)
  lightClient.updateObserver =
    proc(lightClient: LightClient, obj: altair.LightClientUpdate) =
      lightClient.onLightClientObject(obj)
  lightClient.finalityUpdateObserver =
    proc(lightClient: LightClient, obj: altair.LightClientFinalityUpdate) =
      lightClient.onLightClientObject(obj)
  lightClient.optimisticUpdateObserver =
    proc(lightClient: LightClient, obj: altair.LightClientOptimisticUpdate) =
      lightClient.onLightClientObject(obj)
```
2022-12-08 16:24:16 +00:00
tersec
7cf432b155
eip4844 fork and epoch transition tests; some eip4844 gossip (#4393) 2022-12-06 16:43:11 +00:00
tersec
415b11aa67
EIP4844 tweaks to pass SSZ consensus object tests (#4390) 2022-12-05 21:36:53 +00:00
tersec
474b0d8502
withUpdatedState injects updatedState rather than state template (#4375) 2022-11-30 16:37:23 +02:00
tersec
ed672113bc
support engine API execution payloads with withdrawals (#4358) 2022-11-29 05:02:16 +00:00
tersec
61c5ac32d8
automated consensus spec ref URL update to v1.3.0-alpha.1 (#4354) 2022-11-24 19:07:02 +00:00
tersec
c8083f2c32
implement more missing capella functionality (#4344) 2022-11-24 09:53:04 +02:00
Eugene Kabanov
fb4fea81b5
Fix doppelganger protection in validator duties. (#4345)
Fix missing activationEpoch setup.
2022-11-24 09:48:10 +02:00
Eugene Kabanov
eb661565ed
Per-validator doppelganger protection. (#4304)
* Initial commit.

* NextAttestationEntry type.

* Add doppelgangerCheck and actual check.

* Recover deleted check.

* Remove NextAttestainEntry changes.

* More cleanups for NextAttestationEntry.

* Address review comments.

* Remove GENESIS_EPOCH specific check branch.

* Decrease number of full epochs for doppelganger check in VC.

Co-authored-by: zah <zahary@status.im>
2022-11-20 15:55:43 +02:00
Etan Kissling
48994f67d3
rename BlockError -> VerifierError (#4310)
We currently use `BlockError` for both beacon blocks and LC objects.
In light of EIP4844, we will likely also use it for blob sidecars.
To avoid confusion, renaming it to a more generic `VerifierError`,
and update its documentation to be more generic.

To avoid long lines as a followup, also renaming the `block_processor`'s
`BlockProcessingCompleted.completed`->`ProcessingStatus.completed` and
`BlockProcessingCompleted.notCompleted`->`ProcessingStatus.notCompleted`
2022-11-10 17:40:27 +00:00
tersec
909c095e64
initial automated v1.2.0 -> v1.3.0-alpha.0 consensus spec URL update (#4296) 2022-11-08 02:37:28 +00:00
tersec
5b46f0b723
add Capella support to Forked* (#4276)
* add Capella support to Forked*

* remove cruft

* add `OnForkyBlockAdded`
2022-11-02 16:23:30 +00:00
tersec
69ed3a2fd6
fix false-positive warnings on expected VALID fcU status; adjust log levels (#4242)
* fix false-positive warnings on expected VALID fcU status; adjust log levels

* clearer info/warning message wording
2022-10-26 21:14:11 +00:00
Jacek Sieka
b08d0ff2ab
Optimistic mode (#4262)
In optimistic mode, Nimbus will sync optimistically even when the
execution client is offline / not available.

An optimistic node is less secure because it has not validated block
transactions via the execution client and can thus not be used for
validation duties.
2022-10-26 20:44:45 +00:00
tersec
fb6e6d9cf4
remove newPayload from block production flow (#4186)
* remove `newPayload` from block production flow

* refactor block_processor to run `newPayload` as part of `storeBlock`
2022-10-14 22:48:56 +03:00
tersec
ad7541567c
move LVH handling to tests/; increase maximum fork choice retries (#4205) 2022-10-03 13:10:08 +00:00
tersec
c367b14ad9
deprecate --safe-slots-to-import-optimistically (#4182) 2022-09-29 06:29:49 +00:00
tersec
1819d79e07
avoid potential database inconsistency after fcU INVALID+crash (#4192)
* avoid database race-condition inconsistency after fcU `INVALID` then crash

* ensure head doesn't fall behind finalized; add more tests for head movement/reloading DAG
2022-09-28 21:07:31 +00:00
tersec
0f6d19b4b3
implement v1.2.0 optimistic sync tests (#4174)
* implement v1.2.0 optimistic sync tests

* Update beacon_chain/consensus_object_pools/blockchain_dag.nim

Co-authored-by: Etan Kissling <etan@status.im>

* `lvh` -> `latestValidHash` and only invalidate one specific block"

* `getEarliestInvalidRoot` -> `getEarliestInvalidBlockRoot`; `defaultEarliestInvalidRoot` -> `defaultEarliestInvalidBlockRoot`

Co-authored-by: Etan Kissling <etan@status.im>
2022-09-27 15:11:47 +03:00
tersec
a0ead042ad
newPayload INVALIDATED should be unviableFork (#4180) 2022-09-26 21:24:32 +00:00
tersec
deb043796b
a few more manual v1.2.0 consensus spec ref URL updates (#4165) 2022-09-23 12:00:17 +00:00
tersec
3c03ba86c1
update consensus spec ref URLs to v1.2.0 (#4164) 2022-09-23 07:56:06 +00:00
zah
154723947b
Don't search for the TTD block after the merge (#4152) 2022-09-20 09:17:25 +03:00
tersec
56720dd808
update consensus layer spec ref URLs to v1.2.0-rc.3 (#4143) 2022-09-20 02:08:09 +02:00
tersec
ab3ac64b19
Remove optimistic sync candidate check (#4129) 2022-09-17 20:45:35 +00:00
Etan Kissling
0244671cb8
rm optimistic candidate block check from LC (#4131)
The optimistic candidate block check that only imports a new block into
the EL client if its parent block also had execution enabled is not
needed anymore, as mainnet has merged and the attack period is over.
2022-09-17 00:42:19 +00:00
tersec
8be964a152
update consensus layer spec ref URLs to v1.2.0-rc.3 (#4109) 2022-09-10 17:16:38 +00:00
tersec
19bf460a3b
more withState state -> forkyState (#4104) 2022-09-10 08:12:07 +02:00
tersec
1d620f0123
consensus spec URL updates to v1.2.0-rc.3 (#4105) 2022-09-09 21:56:06 +00:00
tersec
cd46af17e9
handle INVALIDATED forkchoiceUpdated better (#4081) 2022-09-07 22:54:37 +02:00