Commit Graph

5996 Commits

Author SHA1 Message Date
Etan Kissling 18de7a517a
avoid polluting Jenkins CI with geth binaries (#4952)
When `scripts/geth_binaries.sh` was updated, as a side effect of dumping
downloaded Geth tarballs / zips into the repo root directory, artifacts
for Jenkins jobs started preserving those downloads with every job,
repeatedly leading to full disks. Exclude those downloads from Jenkins.
2023-05-14 23:32:45 +02:00
Etan Kissling ec049bc493
use correct root in `test_gossip_validation` (#4951)
Use block root not state root for sync committee message in test.
2023-05-14 16:18:50 +02:00
Jacek Sieka fedf04c55b
bumps (#4942)
general round of cleanup bumps

* bearssl: upstream update
* confutils: cleanups, env var support
* eth: cleanup
* faststeam: unsafeAddr fix
* json-ser: cleanups
* backtrace: harden
* metrics: cleanups
* normalize: tests
* ser: 2.0/devel fixes
* snappy: cleanup
* stew: Result updates
* stint: cleanups
* testutils: cleanups
* toml: 2.0 compat
* unicode: fixes
* ut2: cleanups
* nimcrypto: safety fixes
* sepolia: rename
2023-05-13 09:58:47 +00:00
Etan Kissling deb5818587
use compile-time year in `--version` copyright (#4946)
Instead of using current runtime year, use compile-time year in notice.

```
nimbus_beacon_node --version
```
2023-05-13 04:30:49 +00:00
Etan Kissling 0701038f76
log `bls_to_execution_changes_len` in `shortLog` (#4949)
* log `bls_to_execution_changes_len` in `shortLog`

When logging blocks to console, include num BLS to Execution changes.

* fix

* lint
2023-05-13 04:29:46 +00:00
tersec a9f080d92d
small validator pool proc/func and unused symbol cleanup (#4945) 2023-05-12 19:21:43 +00:00
Etan Kissling ea97e93e74
accelerate `getShufflingRef` (#4911)
When an uncached `ShufflingRef` is requested, we currently replay state
which can take several seconds. Acceleration is possible by:

1. Start from any state with locked-in `get_active_validator_indices`.
   Any blocks / slots applied to such a state can only affect that
   result for future epochs, so are viable for querying target epoch.
   `compute_activation_exit_epoch(state.slot.epoch) > target.epoch`

2. Determine highest common ancestor among `state` and `target.blck`.
   At the ancestor slot, same rules re `get_active_validator_indices`.
   `compute_activation_exit_epoch(ancestorSlot.epoch) > target.epoch`

3. We now have a `state` that shares history with `target.blck` up
   through a common ancestor slot. Any blocks / slots that the `state`
   contains, which are not part of the `target.blck` history, affect
   `get_active_validator_indices` at epochs _after_ `target.epoch`.

4. Select `state.randao_mixes[N]` that is closest to common ancestor.
   Either direction is fine (above / below ancestor).

5. From that RANDAO mix, mix in / out all RANDAO reveals from blocks
   in-between. This is just an XOR operation, so fully reversible.
   `mix = mix xor SHA256(blck.message.body.randao_reveal)`

6. Compute the attester dependent slot from `target.epoch`.
   `if epoch >= 2: (target.epoch - 1).start_slot - 1 else: GENESIS_SLOT`

7. Trace back from `target.blck` to the attester dependent slot.
   We now have the destination for which we want to obtain RANDAO.

8. Mix in all RANDAO reveals from blocks up through the `dependentBlck`.
   Same method, no special handling necessary for epoch transitions.

9. Combine `get_active_validator_indices` from `state` at `target.epoch`
   with the recovered RANDAO value at `dependentBlck` to obtain the
   requested shuffling, and construct the `ShufflingRef` without replay.

* more tests and simplify logic

* test with different number of deposits per branch

* Update beacon_chain/consensus_object_pools/blockchain_dag.nim

Co-authored-by: Jacek Sieka <jacek@status.im>

* `commonAncestor` tests

* lint

---------

Co-authored-by: Jacek Sieka <jacek@status.im>
2023-05-12 19:36:59 +02:00
Etan Kissling d263f7f0cb
fix SSZ response for `produceBlindedBlock` (#4943)
* fix SSZ response for `produceBlindedBlock`

In `produceBlindedBlock`, we sent the `ForkedBlindedBeaconBlock` when
requested to reply in SSZ format. However, expected result is just the
inner `ForkyBlindedBeaconBlock` together with `eth-consensus-version`.

Note: We do not use SSZ format in our VC for this endpoint at this time,
which explains why we haven't noticed earlier.

* fix Altair/Phase0
2023-05-12 15:40:45 +00:00
Jacek Sieka 51418a7894
Incremental pruning (#4887)
* Incremental pruning

When turning on pruning the first time the current pruning algorithm
will prune the full database at startup. This delays restart
unnecessarily, since all of the pruned space is not needed at once.

This PR introduces incremental pruning such that we will never prune
more than 32 blocks or the sync speed, whichever is higher.

This mode is expected to become default in a follow-up release.
2023-05-12 13:37:15 +03:00
Jacek Sieka 938d21f1ed
fix linking / compile warnings on pie/etc (#4939)
* fix linking / compile warnings on pie/etc

* oops

* too much nim of late

* quotes
2023-05-12 07:57:49 +00:00
Jacek Sieka 09a69c7b07
better batch sig verification failure message 2023-05-12 08:18:13 +02:00
tersec cc341e0ab7
remove unused definitions (#4937) 2023-05-11 14:39:36 +00:00
Jacek Sieka b3c6320d56
embed genesis states using incbin (#4905) 2023-05-11 11:11:00 +00:00
tersec 2fcc01f516
modify newPayload failure logging (#4930) 2023-05-11 13:58:25 +03:00
Etan Kissling 18bc47fe83
add Capella constants to REST config endpoint (#4925)
Capella constants were missing; if we want to support BLS changes from
VC, we need the new domain constant to check compatibility.
2023-05-11 12:53:02 +02:00
tersec 8686991890
add optimized Linux amd64 flavor (#4914)
* add optimized Linux amd64 flavor

* copyright year
2023-05-11 13:05:52 +03:00
Etan Kissling e0f024c0f3
capella preset for Gnosis (#4936)
```
MAX_WITHDRAWALS_PER_PAYLOAD* = 8
MAX_VALIDATORS_PER_WITHDRAWALS_SWEEP* = 8192
```

https://github.com/gnosischain/configs/pull/11
2023-05-11 09:56:32 +00:00
Etan Kissling 22c298c07b
final portion of trivial v1.3.0 bumps (#4926)
* final portion of trivial v1.3.0 bumps

Updates unchanged logic to latest v1.3.0 consensus-specs refs.

```
https://github.com/ethereum/consensus-specs/(blob|tree)/(?!v1\.3\.0/)
```

* lint
2023-05-11 09:54:29 +00:00
henridf 5dfd814588
Load trusted setup (#4870)
* Kzg: Load trusted setup

* scripts/launch_local_testnet.sh: set FIELD_ELEMENTS_PER_BLOB

* Use right setup file for mainnet/minimal

* Force rebuild

* Add comment explaining why build with -f
2023-05-11 11:52:44 +03:00
Etan Kissling e44b51e955
fix `attachMerkleProofs` to support multiple deposits (#4932)
`attachMerkleProofs` is used by `mockUpdateStateForNewDeposit` to create
a single deposit. The function doesn't work correctly when trying with
with multiple deposits, though. Fix this to enable more complex tests,
and also return the `deposit_root` for forming matching `Eth1Data`.
2023-05-11 11:45:55 +03:00
Etan Kissling a09b05bc27
fix `findShufflingRef` slot compute around genesis (#4910)
`dependent_slot` for epoch 2 is epoch 1's start slot - 1, not 0.
2023-05-11 11:39:40 +03:00
tersec a35c2c15b0
remove unused getBlock function (#4931) 2023-05-11 11:23:09 +03:00
henridf be3f5b1eac
More blob tweaks/fixes from running in devnet (#4933)
* BeaconNode: don't call fetchMissingblobs with empty list

* More logging

* BlockProcessor.checkBloblessSignature: Add missing return value
2023-05-11 00:36:35 +00:00
Kim De Mey 4e715c877f
Bump nim-eth for less cannot raise Defect warnings (#4934) 2023-05-11 00:35:47 +00:00
Etan Kissling 322429b191
final portion of non-trivial v1.3.0 bumps (#4927)
* final portion of non-trivial v1.3.0 bumps

Updates unchanged logic to latest v1.3.0 consensus-specs refs,
and cleans up surrounding sections / syncs comments, and so on.

```
https://github.com/ethereum/consensus-specs/(blob|tree)/(?!v1\.3\.0/)
```

* lint

* linebreak
2023-05-10 16:04:48 +02:00
tersec d3929cbb45
update some beacon API spec URLs; fix some Name and DuplicateModuleImport hints (#4929) 2023-05-10 10:20:55 +00:00
Etan Kissling 4873f8bdc4
update weak subjectivity calculations (#4923)
Weak subjectivity logic (`--weak-subjectivity-checkpoint`) was outdated.
Updated to latest specs, also taking into account total active balance.

See https://github.com/ethereum/consensus-specs/pull/2190
2023-05-10 08:23:59 +00:00
tersec e503cb4f51
make invalid execution payloads more visible (#4906) 2023-05-10 07:17:15 +00:00
Etan Kissling f4c215c5e6
cleanup `state_transition_epoch` and bump to v1.3.0 (#4922)
* cleanup `state_transition_epoch` and bump to v1.3.0

More v1.3.0 consensus-specs bumps, focused on `state_transition_epoch`.
Also fixed `current_epoch` spurious style check warning, and cleanup.

* Update beacon_chain/spec/state_transition_epoch.nim
2023-05-10 04:31:23 +02:00
Jacek Sieka 34a81f56c3
bump web3 (#4919)
and update fakeee
2023-05-09 22:51:57 +00:00
Etan Kissling 34e7181b0d
`KZG(Commitment(s)?|Proof)` > `Kzg$1` (#4917)
Fix style-check by using matching capitalization for `nim-kzg` types.
2023-05-09 19:51:36 +00:00
Etan Kissling 93899b2e2a
consensus-specs v1.3.0 bumps with cleanup (#4918)
Some more consensus-specs v1.3.0 bumps. No semantic logic changes,
but also contains cleanup, so separate PR to reduce noise.
2023-05-09 20:16:49 +02:00
Etan Kissling e9319a5947
`bls_to_execution_changes` check alternative (#4916)
Make intent clearer about when to expect `bls_to_execution_changes`,
and replace test-time errors with compile-time errors if the field
is renamed or goes away in the future.
2023-05-09 17:24:02 +00:00
Etan Kissling b04970189b
more v1.3.0 consensus-specs bumps (#4915)
More spec references for unchanged logic bumped to v1.3.0.
2023-05-09 17:20:37 +00:00
Etan Kissling 5c5fbc088f
simplify FC weight computation (#4892)
Optimizations from https://github.com/ethereum/consensus-specs/pull/3246
avoid a couple unnecessary divisions when calculating proposer boost.
2023-05-09 10:16:13 +00:00
Etan Kissling 5b3c211285
`checkedReject` > `errReject` for CI failure (#4909)
The `SignedContributionAndProof: invalid contribution signature` check
is sometimes hit around fork boundaries when running local testnet.
To avoid failing CI, revert this isntance to a plain `errReject` until
the underlying problem is addressed.
2023-05-09 09:44:24 +00:00
tersec 6458133f5c
disable conditionality of warning disabling (#4908)
* disable conditionality of warning disabling

* add copyright notice with current year

* express version constraint in nimble file

* also needs copyright year
2023-05-09 09:30:29 +00:00
zah 40253a76dd
Version 23.5.0 (#4913) 2023-05-09 11:17:18 +03:00
zah 5bf9284e62
Initial public version of the Verifying Web3Signer functionality (#4912)
* Allow the list of proved properties for web3signer to be configured
* Document the Web3Signer setups (regular, distributed and verified)
2023-05-09 11:16:43 +03:00
Etan Kissling 3a1c468991
use merge-base for copyright year check (#4907)
Instead of comparing against current base branch head, use the common
ancestor of the PR and the base branch to avoid false positives when
a year was bumped in the base branch but not yet merged into the PR.
2023-05-09 04:07:47 +02:00
tersec 8f9bb391a3
don't consider attempt to route duplicate block an error (#4904) 2023-05-08 14:59:13 +02:00
henridf 53436c2b9b
Add blob validation condition (#4902) 2023-05-06 20:09:17 +00:00
Etan Kissling a23252c297
reorder `get_initial_beacon_block` by fork (#4899)
Deneb was listed between Capella and Bellatrix. Swap with Capella.
2023-05-06 10:32:59 +00:00
henridf 23adf15e5a
Blob handling sync fixes (#4888)
* Fix groupBlobs

* Fix getShortMap

* Fix blob handling in sync

* lint

* Add some blob-related logging
2023-05-06 08:58:50 +00:00
zah ae46be7020
Sign the blinded blocks only if they provide better value than the EL block (#4894) 2023-05-06 10:32:30 +02:00
Etan Kissling 1ebcd8b473
`excess_data_gas` after `withdrawals` cleanup (#4900)
We already updated the field order in the actual `ExecutionPayload`,
but in init code and tests / logs etc we still used the old order.
Update those occurrences to also match the field order in the struct.
Furthermore, add `excess_data_gas` to last entry in `test_eth1_monitor`.
2023-05-05 23:15:47 +00:00
Etan Kissling 297881edb7
bump gossip validation refs to 1.3.0 spec (#4895)
Updates gossip validation spec references to v1.3.0 and fixes an
incorrect reference to "signed_aggregate_and_proof" in sync contribution
documentation.
2023-05-05 22:48:33 +02:00
Etan Kissling 508819633a
more consensus-specs v1.3.0 bumps (#4898)
Updates more consensus-specs references to v1.3.0.
This PR only includes updates for sections otherwise unmodified.
2023-05-05 20:38:51 +00:00
Etan Kissling e408c7573d
bump more consensus-specs refs to v1.3.0 (with comments) (#4901)
Bump some more consensus-specs links to v1.3.0, and also align comments
with latest specs where they differ, and fix typos.
2023-05-05 20:37:56 +00:00
Etan Kissling 28968c11ba
remove `SAFE_SLOTS_TO_UPDATE_JUSTIFIED` (#4897)
The `SAFE_SLOTS_TO_UPDATE_JUSTIFIED` constant is no longer used as the
bouncing attack fix was removed:
https://github.com/ethereum/consensus-specs/pull/3290

Note: Some test networks still define the constant, ignoring the config
constant for now until it is no longer used.
2023-05-05 18:11:38 +00:00