462 Commits

Author SHA1 Message Date
tersec
22208836b1
automated v1.4.0-alpha.1 to v1.4.0-alpha.2 consensus spec URL updates (#5056) 2023-06-10 09:56:54 +00:00
henridf
eda631cd7a
Blob accounting fixes/tweaks (#5044)
- Replace asset by warning
- change inequality to lt
2023-06-09 10:52:03 +00:00
Etan Kissling
12381d5f33
also pack attestations where LMD vote is orphaned (#5002)
* also pack attestations where LMD vote is orphaned

When `attestation.data.beacon_block_root` gets orphaned, attestations
with a good `attestation.data.target.root` may still be valuable.
The LMD GHOST vote is not relevant for attestation rewards.

Switch to use the FFG vote (`attestation.data.target.root`) instead,
gossip validation ensures it is an ancestor of `beacon_block_root`.

* lint
2023-06-04 10:39:12 +00:00
tersec
f86febc111
update consensus spec URLs to v1.4.0-alpha.1 (#5027) 2023-06-04 10:32:50 +00:00
tersec
ee71b6cc36
update consensus spec URLs to v1.4.0-alpha.0 (#5022) 2023-06-02 12:59:38 +00:00
tersec
1f535336da
automated v1.3.0 to v1.4.0-alpha.0 consensus spec URL updates (#4996)
* automated v1.3.0 to v1.4.0-alpha.0 consensus spec URL updates

* add copyright year headers
2023-05-26 00:14:28 +02:00
Etan Kissling
00728e9bb7
use executionValid bit in BlockRef (#4956)
Update `beacon_node/rpc` usage for the new `executionValid` field.
2023-05-25 15:57:24 +02:00
Jacek Sieka
8db65ef574
check flag before slower hash comparison 2023-05-23 09:09:22 +02:00
tersec
7577c625b3
implicitly backfill pre-finalized block execution validity from finalized block (#4984) 2023-05-23 08:26:31 +02:00
henridf
5ef748b19d
Clarify addOrphan error/logging (#4981)
* Clarify addOrphan error/logging

addOrphan returned a bool to indicate success. Change this to a Result
so that different errors can be distinguished.

* Update beacon_chain/consensus_object_pools/block_quarantine.nim

Co-authored-by: tersec <tersec@users.noreply.github.com>

* Update beacon_chain/gossip_processing/gossip_validation.nim

---------

Co-authored-by: tersec <tersec@users.noreply.github.com>
2023-05-21 17:47:00 +00:00
tersec
cd087b9a43
replace optimisticRoots table with field in BlockRef (#4969)
* replace optimisticRoots table with field in BlockRef

* copyright year

* mark finalized blocks as verified on load

* Update beacon_chain/consensus_object_pools/block_dag.nim

Co-authored-by: Etan Kissling <etan@status.im>

* expand non-optimistic block checking to all pre-merge blocks; refactor markBlockVerified to use BlockRef rather than block root and remove superfluous caller in newPayload path replaced by addResolvedHeadBlock BlockRef construction

* don't treat finalized block specially; VALID status is sticky

---------

Co-authored-by: Etan Kissling <etan@status.im>
2023-05-20 12:18:51 +00:00
Etan Kissling
c70fd8fe97
Merge branch 'unstable' into dev/etan/rd-shufflingacc 2023-05-17 14:06:31 +02:00
tersec
74511f61d1
Use withdrawal credentials as default fee recipient (#4968) 2023-05-17 07:56:37 +03:00
Etan Kissling
40e89937c5
segregate sync committee messages by period / fork (#4953)
`SyncCommitteeMsgPool` grouped messages by their `beacon_block_root`.
This is problematic around sync committee period boundaries and forks.
Around sync committee period boundaries, members from both the current
and next sync committee may sign the same `beacon_block_root`; mixing
the signatures from both committees together is a mistake. Likewise,
around fork transitions, the `signing_root` changes, so those messages
also need to be segregated.
2023-05-17 07:55:55 +03:00
Jacek Sieka
83393cea8d
dependent slot helpers 2023-05-16 11:04:25 +02:00
Jacek Sieka
3acbb3ea6c
fix missing shuffling dependent slot computation 2023-05-16 10:05:35 +02:00
Etan Kissling
dbba003a38
Revert "Revert "accelerate getShufflingRef (#4911)" (#4958)"
This reverts commit 748be8b67ba8598340766cabcce784cc9b0f353e.
2023-05-15 17:41:40 +02:00
Etan Kissling
748be8b67b
Revert "accelerate getShufflingRef (#4911)" (#4958)
This reverts commit ea97e93e74358bf4e40bcbdcde285420f7f5ad18.
2023-05-15 15:25:51 +00:00
henridf
573228ffa0
Rename eth1/ -> el/ and eth1_monitor.nim -> el_monitor.nim (#4944) 2023-05-15 05:05:12 +00:00
Etan Kissling
ea97e93e74
accelerate getShufflingRef (#4911)
When an uncached `ShufflingRef` is requested, we currently replay state
which can take several seconds. Acceleration is possible by:

1. Start from any state with locked-in `get_active_validator_indices`.
   Any blocks / slots applied to such a state can only affect that
   result for future epochs, so are viable for querying target epoch.
   `compute_activation_exit_epoch(state.slot.epoch) > target.epoch`

2. Determine highest common ancestor among `state` and `target.blck`.
   At the ancestor slot, same rules re `get_active_validator_indices`.
   `compute_activation_exit_epoch(ancestorSlot.epoch) > target.epoch`

3. We now have a `state` that shares history with `target.blck` up
   through a common ancestor slot. Any blocks / slots that the `state`
   contains, which are not part of the `target.blck` history, affect
   `get_active_validator_indices` at epochs _after_ `target.epoch`.

4. Select `state.randao_mixes[N]` that is closest to common ancestor.
   Either direction is fine (above / below ancestor).

5. From that RANDAO mix, mix in / out all RANDAO reveals from blocks
   in-between. This is just an XOR operation, so fully reversible.
   `mix = mix xor SHA256(blck.message.body.randao_reveal)`

6. Compute the attester dependent slot from `target.epoch`.
   `if epoch >= 2: (target.epoch - 1).start_slot - 1 else: GENESIS_SLOT`

7. Trace back from `target.blck` to the attester dependent slot.
   We now have the destination for which we want to obtain RANDAO.

8. Mix in all RANDAO reveals from blocks up through the `dependentBlck`.
   Same method, no special handling necessary for epoch transitions.

9. Combine `get_active_validator_indices` from `state` at `target.epoch`
   with the recovered RANDAO value at `dependentBlck` to obtain the
   requested shuffling, and construct the `ShufflingRef` without replay.

* more tests and simplify logic

* test with different number of deposits per branch

* Update beacon_chain/consensus_object_pools/blockchain_dag.nim

Co-authored-by: Jacek Sieka <jacek@status.im>

* `commonAncestor` tests

* lint

---------

Co-authored-by: Jacek Sieka <jacek@status.im>
2023-05-12 19:36:59 +02:00
Jacek Sieka
51418a7894
Incremental pruning (#4887)
* Incremental pruning

When turning on pruning the first time the current pruning algorithm
will prune the full database at startup. This delays restart
unnecessarily, since all of the pruned space is not needed at once.

This PR introduces incremental pruning such that we will never prune
more than 32 blocks or the sync speed, whichever is higher.

This mode is expected to become default in a follow-up release.
2023-05-12 13:37:15 +03:00
Jacek Sieka
09a69c7b07
better batch sig verification failure message 2023-05-12 08:18:13 +02:00
Etan Kissling
a09b05bc27
fix findShufflingRef slot compute around genesis (#4910)
`dependent_slot` for epoch 2 is epoch 1's start slot - 1, not 0.
2023-05-11 11:39:40 +03:00
tersec
d3929cbb45
update some beacon API spec URLs; fix some Name and DuplicateModuleImport hints (#4929) 2023-05-10 10:20:55 +00:00
henridf
23adf15e5a
Blob handling sync fixes (#4888)
* Fix groupBlobs

* Fix getShortMap

* Fix blob handling in sync

* lint

* Add some blob-related logging
2023-05-06 08:58:50 +00:00
Etan Kissling
508819633a
more consensus-specs v1.3.0 bumps (#4898)
Updates more consensus-specs references to v1.3.0.
This PR only includes updates for sections otherwise unmodified.
2023-05-05 20:38:51 +00:00
Etan Kissling
e408c7573d
bump more consensus-specs refs to v1.3.0 (with comments) (#4901)
Bump some more consensus-specs links to v1.3.0, and also align comments
with latest specs where they differ, and fix typos.
2023-05-05 20:37:56 +00:00
Etan Kissling
4a1b94bc2e
update Deneb ANSI art for readability 🐟 (#4885)
Make "L" characters wider to improve readability.

Thanks to http://beatscribe.com (beatscribe#1008 on Discord)
2023-05-03 22:25:01 +02:00
henridf
5ccb085b9b
block_quarantine.nim: remove unused import (#4877)
leftover from #4844
2023-04-28 22:41:55 +00:00
henridf
a32054cdd1
BlobQuarantine.hasBlobs: fix for loop bound (#4876) 2023-04-28 19:27:28 +00:00
henridf
ef0b95dfbc
RequestManager: add support for fetching Blobs (#4844)
* RequestManager: add support for fetching Blobs

* Review feedback

* Lint

* Change peekSortedBlobless -> peekBlobless
2023-04-28 12:57:35 +00:00
Jacek Sieka
58b93ccbe0
era: Capella+ support (fixes #4752) (#4853)
Post-Capella, historical roots are computed from historical summaries
instead of being directly stored in the beacon state.

Slightly messy to pass both lists around - this is done to avoid
computing the historical root unnecessarily.
2023-04-24 15:26:28 +02:00
tersec
b390911e93
automatic update of v1.3.0-rc.5 consensus spec URLs to v1.3.0 (#4848) 2023-04-21 18:52:43 +00:00
tersec
c0b90833db
don't crash on block production error; build lookahead proposal from current head (#4842) 2023-04-20 15:07:27 +00:00
Etan Kissling
e6e4ba9de6
clean up redundant tests and config (#4836)
The consensus-spec-tests already cover the scenarios of our custom test
runner, so the custom tests can be removed. Also cleans up unused config
flags and related unreachable logic.
2023-04-18 21:26:36 +02:00
henridf
176c80a3d5
Fix incorrect naming and use of Quarantine.peekBlobless (#4826)
The 'peek' name was incorrect as it was actually removing from the
table. It was consequently used incorrectly in block processing: the
blobless block wasn't returned to the table when it should be.
2023-04-17 18:10:13 +02:00
tersec
75be7d267d
always use fcUV2 in shapella even for non-proposer fcUs (#4817)
* always use fcUV2 in shapella even for non-proposer fcUs

* avoid template/proc naming conflict with libp2p/signed_envelope.nim having a payload proc
2023-04-17 16:17:52 +02:00
henridf
29b431e312
Simplify block quarantine blobless (#4824)
* Simplify block quarantine blobless

The quarantine blobless table was initially keyed off of (Eth2Digest,
ValidatorSig). This was modelled off the orphan table. The presence of
the signature in the key is necessary for orphans, because we can't
verify the signature for an orphan. That is not the case for a
blobless block, where the signature can be verified.

So this PR changes the blobless block table to be keyed off a
Eth2Digest only. This simplifies the retrieval and handling of
blobless blocks.

* review feedback
2023-04-16 08:37:56 +00:00
tersec
f6062a2be1
compute correct withdrawals for lookahead proposal fcUs at epoch boundaries (#4820) 2023-04-15 21:48:24 +00:00
henridf
021de18e06
Quarantine and reassembly of gossiped blobs and blocks (#4808) 2023-04-13 19:11:40 +00:00
Etan Kissling
56986c08d6
rename execution(Block|Payload)Root > executionBlockHash (#4809)
Note: `execution_payload_root` is _actually_ `htr(payload)`.
Only `executionPayloadRoot` was used as `executionBlockHash`.
2023-04-11 23:31:47 +00:00
Etan Kissling
c3d043c0e1
rename loadExecutionBlockRoot > loadExecutionBlockHash (#4807)
There are still some `executionBlockRoot` after this, separate rename.
2023-04-11 16:56:29 +00:00
henridf
a543b0b446
Quarantine.cleanupOrphans(): Delete unviable orphan (#4782) 2023-04-11 08:51:01 +02:00
Etan Kissling
566c178e59
add ANSI art for Deneb 🐟 (#4803)
Trigger ANSI art on upgrade to Deneb

Art created by http://beatscribe.com (beatscribe#1008 on Discord)
2023-04-10 12:42:19 +00:00
tersec
c31fbc3977
remove most std/options imports (#4778) 2023-03-31 20:46:47 +00:00
tersec
b4508b853a
assorted consensus spec URL updates (#4776) 2023-03-30 23:08:52 +00:00
tersec
9d2fdc2e26
assorted consensus spec URL updates to v1.3.0-rc.5 (#4768) 2023-03-27 14:41:22 +00:00
tersec
2f634c10a4
automated consensus spec URL updating from v1.3.0-rc.4 to rc.5 (#4756) 2023-03-21 00:42:22 +00:00
tersec
0c62950de6
more consensus spec v1.3.0-rc.4 URL updates (#4744) 2023-03-17 12:14:08 +00:00
tersec
ec77116414
automated consensus spec URL updating from v1.3.0-rc.3 to rc.4 (#4742) 2023-03-17 01:10:31 +00:00