5454 Commits

Author SHA1 Message Date
zah
9b9c58c507
Don't report very brief EL connection interruptions on user-visible log levels (#4960) 2023-05-15 23:40:47 +03:00
Etan Kissling
748be8b67b
Revert "accelerate getShufflingRef (#4911)" (#4958)
This reverts commit ea97e93e74358bf4e40bcbdcde285420f7f5ad18.
2023-05-15 15:25:51 +00:00
Eugene Kabanov
a1cb31c7b4
Disable HTTP pipelining by default. (#4950) 2023-05-15 17:42:00 +03:00
Zahary Karadjov
01a5a6c14a
Remove some obsolete Makefile targets 2023-05-15 17:41:12 +03:00
henridf
573228ffa0
Rename eth1/ -> el/ and eth1_monitor.nim -> el_monitor.nim (#4944) 2023-05-15 05:05:12 +00:00
tersec
10a7281349
don't run flaky CI in Windows (#4935)
* don't run flaky CI in Windows

* reduce stack size in `test_light_client`

* regen tests

* re-enable maybe non-problematic test module on Windows

---------

Co-authored-by: Etan Kissling <etan@status.im>
2023-05-15 04:14:58 +02:00
Etan Kissling
98c30f600b
fix getSyncCommitteeDuties for ALTAIR_FORK_EPOCH.period (#4954)
When fetching historical `getSyncCommitteeDuties` for the very first
sync committee period, the case must be handled where Altair may not
have been scheduled on a sync committee period boundary.
2023-05-14 23:56:50 +00:00
tersec
6024b3e508
rm unused function and unexport some others (#4941) 2023-05-14 22:56:14 +00:00
Etan Kissling
18de7a517a
avoid polluting Jenkins CI with geth binaries (#4952)
When `scripts/geth_binaries.sh` was updated, as a side effect of dumping
downloaded Geth tarballs / zips into the repo root directory, artifacts
for Jenkins jobs started preserving those downloads with every job,
repeatedly leading to full disks. Exclude those downloads from Jenkins.
2023-05-14 23:32:45 +02:00
Etan Kissling
ec049bc493
use correct root in test_gossip_validation (#4951)
Use block root not state root for sync committee message in test.
2023-05-14 16:18:50 +02:00
Jacek Sieka
fedf04c55b
bumps (#4942)
general round of cleanup bumps

* bearssl: upstream update
* confutils: cleanups, env var support
* eth: cleanup
* faststeam: unsafeAddr fix
* json-ser: cleanups
* backtrace: harden
* metrics: cleanups
* normalize: tests
* ser: 2.0/devel fixes
* snappy: cleanup
* stew: Result updates
* stint: cleanups
* testutils: cleanups
* toml: 2.0 compat
* unicode: fixes
* ut2: cleanups
* nimcrypto: safety fixes
* sepolia: rename
2023-05-13 09:58:47 +00:00
Etan Kissling
deb5818587
use compile-time year in --version copyright (#4946)
Instead of using current runtime year, use compile-time year in notice.

```
nimbus_beacon_node --version
```
2023-05-13 04:30:49 +00:00
Etan Kissling
0701038f76
log bls_to_execution_changes_len in shortLog (#4949)
* log `bls_to_execution_changes_len` in `shortLog`

When logging blocks to console, include num BLS to Execution changes.

* fix

* lint
2023-05-13 04:29:46 +00:00
tersec
a9f080d92d
small validator pool proc/func and unused symbol cleanup (#4945) 2023-05-12 19:21:43 +00:00
Etan Kissling
ea97e93e74
accelerate getShufflingRef (#4911)
When an uncached `ShufflingRef` is requested, we currently replay state
which can take several seconds. Acceleration is possible by:

1. Start from any state with locked-in `get_active_validator_indices`.
   Any blocks / slots applied to such a state can only affect that
   result for future epochs, so are viable for querying target epoch.
   `compute_activation_exit_epoch(state.slot.epoch) > target.epoch`

2. Determine highest common ancestor among `state` and `target.blck`.
   At the ancestor slot, same rules re `get_active_validator_indices`.
   `compute_activation_exit_epoch(ancestorSlot.epoch) > target.epoch`

3. We now have a `state` that shares history with `target.blck` up
   through a common ancestor slot. Any blocks / slots that the `state`
   contains, which are not part of the `target.blck` history, affect
   `get_active_validator_indices` at epochs _after_ `target.epoch`.

4. Select `state.randao_mixes[N]` that is closest to common ancestor.
   Either direction is fine (above / below ancestor).

5. From that RANDAO mix, mix in / out all RANDAO reveals from blocks
   in-between. This is just an XOR operation, so fully reversible.
   `mix = mix xor SHA256(blck.message.body.randao_reveal)`

6. Compute the attester dependent slot from `target.epoch`.
   `if epoch >= 2: (target.epoch - 1).start_slot - 1 else: GENESIS_SLOT`

7. Trace back from `target.blck` to the attester dependent slot.
   We now have the destination for which we want to obtain RANDAO.

8. Mix in all RANDAO reveals from blocks up through the `dependentBlck`.
   Same method, no special handling necessary for epoch transitions.

9. Combine `get_active_validator_indices` from `state` at `target.epoch`
   with the recovered RANDAO value at `dependentBlck` to obtain the
   requested shuffling, and construct the `ShufflingRef` without replay.

* more tests and simplify logic

* test with different number of deposits per branch

* Update beacon_chain/consensus_object_pools/blockchain_dag.nim

Co-authored-by: Jacek Sieka <jacek@status.im>

* `commonAncestor` tests

* lint

---------

Co-authored-by: Jacek Sieka <jacek@status.im>
2023-05-12 19:36:59 +02:00
Etan Kissling
d263f7f0cb
fix SSZ response for produceBlindedBlock (#4943)
* fix SSZ response for `produceBlindedBlock`

In `produceBlindedBlock`, we sent the `ForkedBlindedBeaconBlock` when
requested to reply in SSZ format. However, expected result is just the
inner `ForkyBlindedBeaconBlock` together with `eth-consensus-version`.

Note: We do not use SSZ format in our VC for this endpoint at this time,
which explains why we haven't noticed earlier.

* fix Altair/Phase0
2023-05-12 15:40:45 +00:00
Jacek Sieka
51418a7894
Incremental pruning (#4887)
* Incremental pruning

When turning on pruning the first time the current pruning algorithm
will prune the full database at startup. This delays restart
unnecessarily, since all of the pruned space is not needed at once.

This PR introduces incremental pruning such that we will never prune
more than 32 blocks or the sync speed, whichever is higher.

This mode is expected to become default in a follow-up release.
2023-05-12 13:37:15 +03:00
Jacek Sieka
938d21f1ed
fix linking / compile warnings on pie/etc (#4939)
* fix linking / compile warnings on pie/etc

* oops

* too much nim of late

* quotes
2023-05-12 07:57:49 +00:00
Jacek Sieka
09a69c7b07
better batch sig verification failure message 2023-05-12 08:18:13 +02:00
tersec
cc341e0ab7
remove unused definitions (#4937) 2023-05-11 14:39:36 +00:00
Jacek Sieka
b3c6320d56
embed genesis states using incbin (#4905) 2023-05-11 11:11:00 +00:00
tersec
2fcc01f516
modify newPayload failure logging (#4930) 2023-05-11 13:58:25 +03:00
Etan Kissling
18bc47fe83
add Capella constants to REST config endpoint (#4925)
Capella constants were missing; if we want to support BLS changes from
VC, we need the new domain constant to check compatibility.
2023-05-11 12:53:02 +02:00
tersec
8686991890
add optimized Linux amd64 flavor (#4914)
* add optimized Linux amd64 flavor

* copyright year
2023-05-11 13:05:52 +03:00
Etan Kissling
e0f024c0f3
capella preset for Gnosis (#4936)
```
MAX_WITHDRAWALS_PER_PAYLOAD* = 8
MAX_VALIDATORS_PER_WITHDRAWALS_SWEEP* = 8192
```

https://github.com/gnosischain/configs/pull/11
2023-05-11 09:56:32 +00:00
Etan Kissling
22c298c07b
final portion of trivial v1.3.0 bumps (#4926)
* final portion of trivial v1.3.0 bumps

Updates unchanged logic to latest v1.3.0 consensus-specs refs.

```
https://github.com/ethereum/consensus-specs/(blob|tree)/(?!v1\.3\.0/)
```

* lint
2023-05-11 09:54:29 +00:00
henridf
5dfd814588
Load trusted setup (#4870)
* Kzg: Load trusted setup

* scripts/launch_local_testnet.sh: set FIELD_ELEMENTS_PER_BLOB

* Use right setup file for mainnet/minimal

* Force rebuild

* Add comment explaining why build with -f
2023-05-11 11:52:44 +03:00
Etan Kissling
e44b51e955
fix attachMerkleProofs to support multiple deposits (#4932)
`attachMerkleProofs` is used by `mockUpdateStateForNewDeposit` to create
a single deposit. The function doesn't work correctly when trying with
with multiple deposits, though. Fix this to enable more complex tests,
and also return the `deposit_root` for forming matching `Eth1Data`.
2023-05-11 11:45:55 +03:00
Etan Kissling
a09b05bc27
fix findShufflingRef slot compute around genesis (#4910)
`dependent_slot` for epoch 2 is epoch 1's start slot - 1, not 0.
2023-05-11 11:39:40 +03:00
tersec
a35c2c15b0
remove unused getBlock function (#4931) 2023-05-11 11:23:09 +03:00
henridf
be3f5b1eac
More blob tweaks/fixes from running in devnet (#4933)
* BeaconNode: don't call fetchMissingblobs with empty list

* More logging

* BlockProcessor.checkBloblessSignature: Add missing return value
2023-05-11 00:36:35 +00:00
Kim De Mey
4e715c877f
Bump nim-eth for less cannot raise Defect warnings (#4934) 2023-05-11 00:35:47 +00:00
Etan Kissling
322429b191
final portion of non-trivial v1.3.0 bumps (#4927)
* final portion of non-trivial v1.3.0 bumps

Updates unchanged logic to latest v1.3.0 consensus-specs refs,
and cleans up surrounding sections / syncs comments, and so on.

```
https://github.com/ethereum/consensus-specs/(blob|tree)/(?!v1\.3\.0/)
```

* lint

* linebreak
2023-05-10 16:04:48 +02:00
tersec
d3929cbb45
update some beacon API spec URLs; fix some Name and DuplicateModuleImport hints (#4929) 2023-05-10 10:20:55 +00:00
Etan Kissling
4873f8bdc4
update weak subjectivity calculations (#4923)
Weak subjectivity logic (`--weak-subjectivity-checkpoint`) was outdated.
Updated to latest specs, also taking into account total active balance.

See https://github.com/ethereum/consensus-specs/pull/2190
2023-05-10 08:23:59 +00:00
tersec
e503cb4f51
make invalid execution payloads more visible (#4906) 2023-05-10 07:17:15 +00:00
Etan Kissling
f4c215c5e6
cleanup state_transition_epoch and bump to v1.3.0 (#4922)
* cleanup `state_transition_epoch` and bump to v1.3.0

More v1.3.0 consensus-specs bumps, focused on `state_transition_epoch`.
Also fixed `current_epoch` spurious style check warning, and cleanup.

* Update beacon_chain/spec/state_transition_epoch.nim
2023-05-10 04:31:23 +02:00
Jacek Sieka
34a81f56c3
bump web3 (#4919)
and update fakeee
2023-05-09 22:51:57 +00:00
Etan Kissling
34e7181b0d
KZG(Commitment(s)?|Proof) > Kzg$1 (#4917)
Fix style-check by using matching capitalization for `nim-kzg` types.
2023-05-09 19:51:36 +00:00
Etan Kissling
93899b2e2a
consensus-specs v1.3.0 bumps with cleanup (#4918)
Some more consensus-specs v1.3.0 bumps. No semantic logic changes,
but also contains cleanup, so separate PR to reduce noise.
2023-05-09 20:16:49 +02:00
Etan Kissling
e9319a5947
bls_to_execution_changes check alternative (#4916)
Make intent clearer about when to expect `bls_to_execution_changes`,
and replace test-time errors with compile-time errors if the field
is renamed or goes away in the future.
2023-05-09 17:24:02 +00:00
Etan Kissling
b04970189b
more v1.3.0 consensus-specs bumps (#4915)
More spec references for unchanged logic bumped to v1.3.0.
2023-05-09 17:20:37 +00:00
Etan Kissling
5c5fbc088f
simplify FC weight computation (#4892)
Optimizations from https://github.com/ethereum/consensus-specs/pull/3246
avoid a couple unnecessary divisions when calculating proposer boost.
2023-05-09 10:16:13 +00:00
Etan Kissling
5b3c211285
checkedReject > errReject for CI failure (#4909)
The `SignedContributionAndProof: invalid contribution signature` check
is sometimes hit around fork boundaries when running local testnet.
To avoid failing CI, revert this isntance to a plain `errReject` until
the underlying problem is addressed.
2023-05-09 09:44:24 +00:00
tersec
6458133f5c
disable conditionality of warning disabling (#4908)
* disable conditionality of warning disabling

* add copyright notice with current year

* express version constraint in nimble file

* also needs copyright year
2023-05-09 09:30:29 +00:00
zah
40253a76dd
Version 23.5.0 (#4913) v23.5.0 2023-05-09 11:17:18 +03:00
zah
5bf9284e62
Initial public version of the Verifying Web3Signer functionality (#4912)
* Allow the list of proved properties for web3signer to be configured
* Document the Web3Signer setups (regular, distributed and verified)
2023-05-09 11:16:43 +03:00
Etan Kissling
3a1c468991
use merge-base for copyright year check (#4907)
Instead of comparing against current base branch head, use the common
ancestor of the PR and the base branch to avoid false positives when
a year was bumped in the base branch but not yet merged into the PR.
2023-05-09 04:07:47 +02:00
tersec
8f9bb391a3
don't consider attempt to route duplicate block an error (#4904) 2023-05-08 14:59:13 +02:00
henridf
53436c2b9b
Add blob validation condition (#4902) 2023-05-06 20:09:17 +00:00