Commit Graph

1413 Commits

Author SHA1 Message Date
Etan Kissling d9031d97dc
rename `_INDEX` generalized index constants to `_GINDEX` (#5583)
Rename constants for consistency with upcoming spec adjustment:

- https://github.com/ethereum/consensus-specs/pull/3537
2023-11-09 12:54:44 +00:00
Eugene Kabanov 6bc038e8d5
VC: Obol middleware support (#5375) 2023-11-08 14:03:51 +02:00
tersec 7e3aeaea09
automated consensus spec URL updating to v1.4.0-beta.4 (#5577) 2023-11-08 05:28:03 +00:00
Etan Kissling d8a7f0df81
update Deneb for blob sidecar inclusion proofs (#5565)
`BlobSidecar` is no longer signed, instead use Merkle proof to link
blobs with block.

- https://github.com/ethereum/consensus-specs/pull/3531

Associated beacon-API / builder-specs still TBD; minimal changes done
to compile in similar style to previous spec, but not standardized yet.

- https://github.com/ethereum/beacon-APIs/pull/369
- https://github.com/ethereum/builder-specs/pull/90
2023-11-06 07:48:43 +01:00
Etan Kissling 8d46809a5c
skip upcoming FC tests for intentional block reorgs until implemented (#5566)
v1.4.0-beta.4 adds tests for intentional block reorgs. To reflect the
implementation status, skip those tests for now and mark them as such.
2023-11-04 20:36:01 +01:00
Etan Kissling 6958f67af3
add test runner for v1.4.0-beta.4 Merkle proof tests (#5567)
Create a test runner for validating the new `Merkle proof` Deneb tests
that are added with `v1.4.0-beta.4` specs.
2023-11-04 20:35:41 +01:00
Etan Kissling 87a37a32e3
introduce `BlobId` type for tracking blob subnets (#5560)
Instead of mixing up `SubnetId` (attestation subnet type) for blobs,
introduce dedicated `BlobId` type.
2023-11-04 14:20:34 +00:00
tersec 62d59daaa7
consensus-spec URL updates to v1.4.0-beta.3 (#5541) 2023-10-30 06:44:43 +00:00
tersec 922283e2e3
clean up ncli_testnet and reduce `ValidIpAddress` usage (#5529)
* ValidIpAddress -> IpAddress

* clean up ncli_testnet and reduce ValidIpAddress usage
2023-10-26 23:16:15 +00:00
tersec 09df3f32b5
add non-SZ getBlobSidecar and BlobSidecar database tests (#5528) 2023-10-26 03:40:04 +00:00
tersec 41dfc9ed91
add `KZGProof` and `Blob` REST JSON serializations (#5526)
* add KZGProof and Blob REST JSON serializations

* error summary updates

* copyright notice

* consistent integer types for Nim 2.0
2023-10-25 23:50:59 +02:00
tersec 5aa5095e17
Fix REST JSON KzgCommitment encoding/decoding to use hex strings per spec (#5522) 2023-10-24 15:22:59 +00:00
tersec 513c9d9637
fix Deneb RestPublishedSignedBlockContents REST JSON decoding (#5520) 2023-10-24 05:50:32 +00:00
tersec fd21882e24
improve test coverage of post-Bellatrix initialize_beacon_state_from_eth1 (#5517) 2023-10-24 01:58:52 +00:00
tersec 4ddd771127
automated consensus spec URL updating to v1.4.0-beta.3 (#5514) 2023-10-19 10:26:38 +00:00
zah 35bf03a3fb
Add the --verifying-web3-signer-url configuration option (#5504) 2023-10-13 15:42:00 +03:00
tersec 40616b6243
update some consensus spec URLs to v1.4.0-beta.2 (#5497) 2023-10-12 05:07:41 +00:00
tersec 447786518f
ShufflingRef approach to next-epoch validator duty calculation/prediction (#5414)
* ShufflingRef approach to next-epoch validator duty calculation/prediction

* refactor action_tracker.updateActions to take ShufflingRef + beacon_proposers; refactor maybeUpdateActionTrackerNextEpoch to be separate and reused function; add actual fallback logic

* document one possible set of conditions

* check epoch participation flags and inactivity scores to ensure no penalties and MAX_EFFECTIVE_BALANCE to ensure rewards don't matter

* correctly (un)shuffle each proposer index

* remove debugging assertion
2023-10-10 00:02:07 +00:00
Etan Kissling dd1f362536
align `Beacon(Block(Body)?|State)Type` with other fork sugar (#5483)
The templates for `BeaconBlock`, `BeaconBlockBody` and `BeaconState`
are the only ones using a `macro` mechanism for code generation.
This prevents using the dot-syntax style `consensusFork.BeaconFoo`
in some situations, and also tends to trigger naming conflicts,
requiring the `Type` suffix. Furthermore, the `macro` only works
for types that are re-defined in every single `ConsensusFork`.

Replacing with the simpler but more verbose approach used for other
types for consistency and to avoid the downsides of the `macro`.

Furthermore, simplify `test_fixture_sanity_blocks` to use `forks` sugar.
2023-10-05 14:01:40 +02:00
Etan Kissling 297c768816
reduce redundant zero initialization for LC data objects (#5479)
Directly initialize `ForkedLightClientObj` instead of separately first
 setting the `kind` (initializing everything to zero) and then assigning
the forky data after that.
2023-10-04 18:11:45 +02:00
Etan Kissling 8aaea1a14e
simplify fork choice test (#5467)
Reduce overly verbose fork branching logic in fork choice test with
generic implementation based on `consensusFork`.
2023-09-29 13:26:34 +00:00
Etan Kissling 7c45b8f98d
convert remaining `toFork` -> `kind` for consistency (#5462)
We currently have multiple ways to obtain `ConsensusFork` or
`LcDataFork` from a forky type. Rename `toFork` to `kind`
for a consistent API naming.
2023-09-27 15:10:28 +00:00
Eugene Kabanov 4fb95d000d
REST server fixes and improvements. (#5422)
* Move from Option[T] to Opt[T] usage.

* Add `finalized` flag.

* Fix compilation issue.

* Http415 error code for some REST API calls.
Introduce more comprehensive error reporting for block calls.
Deprecate decodeEthConsensusVersion() function.

* Bump http-utils.

* Fix copyright year.

* Fix serialization issue.

* Address review comments.

* Post rebase fixes.
2023-09-27 16:45:33 +02:00
Eugene Kabanov e3fe762ec9
Fix unknown fields should be ignored when parsing liveness responses. (#5418) 2023-09-23 08:44:54 +02:00
Etan Kissling 47639ef89c
replace `Eth2NetworkMetadata` case object with `Result` (#5452)
`Eth2NetworkMetadata` has an `incompatible` case to hold an error string
in case the loaded file is not compatible with the compile-time config.
The same can be modeled with a `Result[Eth2NetworkMetadata, string]` and
avoids followup checks for the `incompatible` case.
2023-09-22 23:45:24 +02:00
tersec 2895a9a05c
automated consensus spec URL updating to v1.4.0-beta.2 (#5453) 2023-09-21 18:06:51 +00:00
Etan Kissling e7bc41e005
`blck` --> `forkyBlck` when using `withBlck` / `withStateAndBlck` (#5451)
For symmetry with `forkyState` when using `withState`, and to avoid
problems with shadowing of `blck` when using `withBlck` in `template`,
also rename the injected `blck` to `forkyBlck`.

- https://github.com/nim-lang/Nim/issues/22698
2023-09-21 12:49:14 +02:00
Etan Kissling aacb853a0b
add blob validation to `test_fixture_fork_choice` (#5441)
Preparation for processing new tests from:

- https://github.com/ethereum/consensus-specs/pull/3463
2023-09-20 01:14:49 +00:00
Etan Kissling 176ea09c2b
cleanup `OnBlockAdded` usage (#5426)
Reduce repetitiveness when using forked `OnBlockAdded` callbacks by
introducing a template to obtain appropriate cb from `ConsensusFork`.
2023-09-13 17:57:54 +00:00
Etan Kissling cd68c71c6c
add gossip tests for period boundary (#5423)
Test `validateSyncCommitteeMessage` and `validateContribution`
around sync committee period boundary to cover edge cases.
2023-09-13 08:32:11 +02:00
andri lim 5c88e74c08
Bump stint to v2.0: new array backend (#5113)
* bump stint to v2.0: new array backend

* Fix missing isEven in libnimbus_lc.nim

* bump nim-stint: compiles with arm64
2023-09-13 01:16:04 +07:00
Etan Kissling 8fa5580c50
add infrastructure to select fork choice version (#5387)
To allow testing https://github.com/ethereum/consensus-specs/issues/3466
add support for selecting fork choice version at launch. This means we
can deploy a different logic when `DENEB_FORK_EPOCH != FAR_FUTURE_EPOCH`
that won't be used on Mainnet.
2023-09-12 09:52:51 +02:00
tersec 2b4f987c80
remove pre-v1.4.0 attestation stability subnets (#5402)
* remove pre-v1.4.0 attestation stability subnets

* re-add most of should register stability subnets on attester duties test
2023-09-11 16:03:34 +00:00
Eugene Kabanov 3c3c4e8edf
Fix /eth/v1/validator/liveness/{epoch} call names and links in code (#5392)
* Fix getLiveness() call links and names.

* Add getLiveness() decoding test.

* Remove BN state updates from getLiveness() call.

* Fix AllTests.

* Reduce number of tests.
2023-09-11 12:07:34 +02:00
tersec cc13e0b7e0
restore full test coverage for process_rewards_and_penalties (#5407)
* restore full test coverage for process_rewards_and_penalties

* adjust ncli_db to use new iterator
2023-09-08 16:42:18 +00:00
zah 53589b5a7d
Add metadata for the Holesky network (#5337)
* Add metadata for the Holesky network

* Add copyright banner to the new Nim module

* Working version

* Bump Chronos to fix downloading from Github
* Add checksum check of the downloaded file
* Clean up debugging code and obsolete imports
2023-09-08 08:53:27 +03:00
zah 2b5bd74e15
Rename --validator-source to --web3-signer-url and document it (#5389)
Also allows multiple instances to be configured
2023-09-06 22:04:10 +03:00
Etan Kissling 6af70e2f80
wrap `applyExecutionPayload` in factories for Nim 2.0 (#5395)
In Nim 2.0, the `test_fixture_operations` files fail to compile with:

```
Error: 'result' is of type <Result[system.void, system.cstring]> which cannot be captured as it would violate memory safety, declared here: /nimbus-eth2/tests/consensus_spec/bellatrix/test_fixture_operations.nim(130, 5); using '-d:nimNoLentIterators' helps in some cases. Consider using a <ref Result[system.void, system.cstring]> which can be captured.
```

Wrapping the `applyExecutionPayload` in a factory that takes `path`
avoids this problem.
2023-09-06 10:35:37 +02:00
Etan Kissling 81397342d0
fix relative import paths for Nim 2.0 (#5397)
In Nim 2.0, relative `import` paths are validated more strictly;
this fixes the incorrect paths used in some tests.
2023-09-06 08:17:59 +00:00
Eugene Kabanov afb5fdae34
Dynamic validators loop (#5385)
* Add periodic dynamic validators polling.
* Add dynamic validators tests.
2023-09-04 22:14:58 +03:00
Jacek Sieka 6fab826487
bump unittest2, clean up suite names (#5384)
Suite names were not being used because `test` has to have access to it
during instantiation - this PR cleans things up a little while at the
same time upgrading unittest2.
2023-09-04 13:05:11 +00:00
tersec ff87ee9181
rm i386 test_blockchain_dag workaround (#5356) 2023-08-25 15:24:56 +00:00
Etan Kissling c211a3849e
remove `{.raises: [Defect].}` Nim 1.2 compatibility (#5352)
In Nim 1.6, `{.raises: [Defect].}` is no longer necessary. Remove it.
2023-08-25 11:29:07 +02:00
Etan Kissling 758c503a52
move mock genesis block hash to `tests` (#5353)
From old interop tests, a mock `eth1BlockHash` was defined in `base`.
To avoid accidental use by Nimbus, move to `tests` and rename it to
`mockEth1BlockHash`.
2023-08-25 11:28:42 +02:00
tersec ef184c8959
subscribe to Deneb blob subnets (#5342) 2023-08-23 19:31:41 +00:00
tersec 93d82e2892
rm more state_sim remnants (#5334) 2023-08-23 19:40:48 +03:00
tersec 6fd4983f6b
EIP-4788/parentBeaconBlockRoot fixes (#5318) 2023-08-19 08:38:17 +00:00
Etan Kissling f70dbf1f53
use different argname than `_` (#5307)
In Nim 2.0, if an argument is named `_` it can't be used in the impl.
Rename the `_` arg in `testblockutil` to something else.
2023-08-17 19:11:37 +00:00
Jacek Sieka 49729e1ef3
prevent concurrent `storeBlock` calls (fixes #5285) (#5295)
When a block is introduced to the system both via REST and gossip at the
same time, we will call `storeBlock` from two locations leading to a
dupliace check race condition as we wait for the EL.

This issue may manifest in particular when using an external block
builder that itself publishes the block onto the gossip network.

* refactor enqueue flow
* simplify calling `addBlock`
* complete request manager verifier future for blobless blocks
* re-verify parent conditions before adding block

among other things, it might have gone stale or finalized between one
call and the other
2023-08-17 15:12:37 +02:00
Eugene Kabanov 4a84457376
VC: Fix API timeouts issue. (#5297)
* Fix VC timeouts issue.
Add tests.

* Update AllTests.

* Remove assertion, which could crash VC when API call was interrupted by some higher logic.
2023-08-16 16:14:39 +00:00