1407 Commits

Author SHA1 Message Date
Etan Kissling
87a37a32e3
introduce BlobId type for tracking blob subnets (#5560)
Instead of mixing up `SubnetId` (attestation subnet type) for blobs,
introduce dedicated `BlobId` type.
2023-11-04 14:20:34 +00:00
tersec
62d59daaa7
consensus-spec URL updates to v1.4.0-beta.3 (#5541) 2023-10-30 06:44:43 +00:00
tersec
922283e2e3
clean up ncli_testnet and reduce ValidIpAddress usage (#5529)
* ValidIpAddress -> IpAddress

* clean up ncli_testnet and reduce ValidIpAddress usage
2023-10-26 23:16:15 +00:00
tersec
09df3f32b5
add non-SZ getBlobSidecar and BlobSidecar database tests (#5528) 2023-10-26 03:40:04 +00:00
tersec
41dfc9ed91
add KZGProof and Blob REST JSON serializations (#5526)
* add KZGProof and Blob REST JSON serializations

* error summary updates

* copyright notice

* consistent integer types for Nim 2.0
2023-10-25 23:50:59 +02:00
tersec
5aa5095e17
Fix REST JSON KzgCommitment encoding/decoding to use hex strings per spec (#5522) 2023-10-24 15:22:59 +00:00
tersec
513c9d9637
fix Deneb RestPublishedSignedBlockContents REST JSON decoding (#5520) 2023-10-24 05:50:32 +00:00
tersec
fd21882e24
improve test coverage of post-Bellatrix initialize_beacon_state_from_eth1 (#5517) 2023-10-24 01:58:52 +00:00
tersec
4ddd771127
automated consensus spec URL updating to v1.4.0-beta.3 (#5514) 2023-10-19 10:26:38 +00:00
zah
35bf03a3fb
Add the --verifying-web3-signer-url configuration option (#5504) 2023-10-13 15:42:00 +03:00
tersec
40616b6243
update some consensus spec URLs to v1.4.0-beta.2 (#5497) 2023-10-12 05:07:41 +00:00
tersec
447786518f
ShufflingRef approach to next-epoch validator duty calculation/prediction (#5414)
* ShufflingRef approach to next-epoch validator duty calculation/prediction

* refactor action_tracker.updateActions to take ShufflingRef + beacon_proposers; refactor maybeUpdateActionTrackerNextEpoch to be separate and reused function; add actual fallback logic

* document one possible set of conditions

* check epoch participation flags and inactivity scores to ensure no penalties and MAX_EFFECTIVE_BALANCE to ensure rewards don't matter

* correctly (un)shuffle each proposer index

* remove debugging assertion
2023-10-10 00:02:07 +00:00
Etan Kissling
dd1f362536
align Beacon(Block(Body)?|State)Type with other fork sugar (#5483)
The templates for `BeaconBlock`, `BeaconBlockBody` and `BeaconState`
are the only ones using a `macro` mechanism for code generation.
This prevents using the dot-syntax style `consensusFork.BeaconFoo`
in some situations, and also tends to trigger naming conflicts,
requiring the `Type` suffix. Furthermore, the `macro` only works
for types that are re-defined in every single `ConsensusFork`.

Replacing with the simpler but more verbose approach used for other
types for consistency and to avoid the downsides of the `macro`.

Furthermore, simplify `test_fixture_sanity_blocks` to use `forks` sugar.
2023-10-05 14:01:40 +02:00
Etan Kissling
297c768816
reduce redundant zero initialization for LC data objects (#5479)
Directly initialize `ForkedLightClientObj` instead of separately first
 setting the `kind` (initializing everything to zero) and then assigning
the forky data after that.
2023-10-04 18:11:45 +02:00
Etan Kissling
8aaea1a14e
simplify fork choice test (#5467)
Reduce overly verbose fork branching logic in fork choice test with
generic implementation based on `consensusFork`.
2023-09-29 13:26:34 +00:00
Etan Kissling
7c45b8f98d
convert remaining toFork -> kind for consistency (#5462)
We currently have multiple ways to obtain `ConsensusFork` or
`LcDataFork` from a forky type. Rename `toFork` to `kind`
for a consistent API naming.
2023-09-27 15:10:28 +00:00
Eugene Kabanov
4fb95d000d
REST server fixes and improvements. (#5422)
* Move from Option[T] to Opt[T] usage.

* Add `finalized` flag.

* Fix compilation issue.

* Http415 error code for some REST API calls.
Introduce more comprehensive error reporting for block calls.
Deprecate decodeEthConsensusVersion() function.

* Bump http-utils.

* Fix copyright year.

* Fix serialization issue.

* Address review comments.

* Post rebase fixes.
2023-09-27 16:45:33 +02:00
Eugene Kabanov
e3fe762ec9
Fix unknown fields should be ignored when parsing liveness responses. (#5418) 2023-09-23 08:44:54 +02:00
Etan Kissling
47639ef89c
replace Eth2NetworkMetadata case object with Result (#5452)
`Eth2NetworkMetadata` has an `incompatible` case to hold an error string
in case the loaded file is not compatible with the compile-time config.
The same can be modeled with a `Result[Eth2NetworkMetadata, string]` and
avoids followup checks for the `incompatible` case.
2023-09-22 23:45:24 +02:00
tersec
2895a9a05c
automated consensus spec URL updating to v1.4.0-beta.2 (#5453) 2023-09-21 18:06:51 +00:00
Etan Kissling
e7bc41e005
blck --> forkyBlck when using withBlck / withStateAndBlck (#5451)
For symmetry with `forkyState` when using `withState`, and to avoid
problems with shadowing of `blck` when using `withBlck` in `template`,
also rename the injected `blck` to `forkyBlck`.

- https://github.com/nim-lang/Nim/issues/22698
2023-09-21 12:49:14 +02:00
Etan Kissling
aacb853a0b
add blob validation to test_fixture_fork_choice (#5441)
Preparation for processing new tests from:

- https://github.com/ethereum/consensus-specs/pull/3463
2023-09-20 01:14:49 +00:00
Etan Kissling
176ea09c2b
cleanup OnBlockAdded usage (#5426)
Reduce repetitiveness when using forked `OnBlockAdded` callbacks by
introducing a template to obtain appropriate cb from `ConsensusFork`.
2023-09-13 17:57:54 +00:00
Etan Kissling
cd68c71c6c
add gossip tests for period boundary (#5423)
Test `validateSyncCommitteeMessage` and `validateContribution`
around sync committee period boundary to cover edge cases.
2023-09-13 08:32:11 +02:00
andri lim
5c88e74c08
Bump stint to v2.0: new array backend (#5113)
* bump stint to v2.0: new array backend

* Fix missing isEven in libnimbus_lc.nim

* bump nim-stint: compiles with arm64
2023-09-13 01:16:04 +07:00
Etan Kissling
8fa5580c50
add infrastructure to select fork choice version (#5387)
To allow testing https://github.com/ethereum/consensus-specs/issues/3466
add support for selecting fork choice version at launch. This means we
can deploy a different logic when `DENEB_FORK_EPOCH != FAR_FUTURE_EPOCH`
that won't be used on Mainnet.
2023-09-12 09:52:51 +02:00
tersec
2b4f987c80
remove pre-v1.4.0 attestation stability subnets (#5402)
* remove pre-v1.4.0 attestation stability subnets

* re-add most of should register stability subnets on attester duties test
2023-09-11 16:03:34 +00:00
Eugene Kabanov
3c3c4e8edf
Fix /eth/v1/validator/liveness/{epoch} call names and links in code (#5392)
* Fix getLiveness() call links and names.

* Add getLiveness() decoding test.

* Remove BN state updates from getLiveness() call.

* Fix AllTests.

* Reduce number of tests.
2023-09-11 12:07:34 +02:00
tersec
cc13e0b7e0
restore full test coverage for process_rewards_and_penalties (#5407)
* restore full test coverage for process_rewards_and_penalties

* adjust ncli_db to use new iterator
2023-09-08 16:42:18 +00:00
zah
53589b5a7d
Add metadata for the Holesky network (#5337)
* Add metadata for the Holesky network

* Add copyright banner to the new Nim module

* Working version

* Bump Chronos to fix downloading from Github
* Add checksum check of the downloaded file
* Clean up debugging code and obsolete imports
2023-09-08 08:53:27 +03:00
zah
2b5bd74e15
Rename --validator-source to --web3-signer-url and document it (#5389)
Also allows multiple instances to be configured
2023-09-06 22:04:10 +03:00
Etan Kissling
6af70e2f80
wrap applyExecutionPayload in factories for Nim 2.0 (#5395)
In Nim 2.0, the `test_fixture_operations` files fail to compile with:

```
Error: 'result' is of type <Result[system.void, system.cstring]> which cannot be captured as it would violate memory safety, declared here: /nimbus-eth2/tests/consensus_spec/bellatrix/test_fixture_operations.nim(130, 5); using '-d:nimNoLentIterators' helps in some cases. Consider using a <ref Result[system.void, system.cstring]> which can be captured.
```

Wrapping the `applyExecutionPayload` in a factory that takes `path`
avoids this problem.
2023-09-06 10:35:37 +02:00
Etan Kissling
81397342d0
fix relative import paths for Nim 2.0 (#5397)
In Nim 2.0, relative `import` paths are validated more strictly;
this fixes the incorrect paths used in some tests.
2023-09-06 08:17:59 +00:00
Eugene Kabanov
afb5fdae34
Dynamic validators loop (#5385)
* Add periodic dynamic validators polling.
* Add dynamic validators tests.
2023-09-04 22:14:58 +03:00
Jacek Sieka
6fab826487
bump unittest2, clean up suite names (#5384)
Suite names were not being used because `test` has to have access to it
during instantiation - this PR cleans things up a little while at the
same time upgrading unittest2.
2023-09-04 13:05:11 +00:00
tersec
ff87ee9181
rm i386 test_blockchain_dag workaround (#5356) 2023-08-25 15:24:56 +00:00
Etan Kissling
c211a3849e
remove {.raises: [Defect].} Nim 1.2 compatibility (#5352)
In Nim 1.6, `{.raises: [Defect].}` is no longer necessary. Remove it.
2023-08-25 11:29:07 +02:00
Etan Kissling
758c503a52
move mock genesis block hash to tests (#5353)
From old interop tests, a mock `eth1BlockHash` was defined in `base`.
To avoid accidental use by Nimbus, move to `tests` and rename it to
`mockEth1BlockHash`.
2023-08-25 11:28:42 +02:00
tersec
ef184c8959
subscribe to Deneb blob subnets (#5342) 2023-08-23 19:31:41 +00:00
tersec
93d82e2892
rm more state_sim remnants (#5334) 2023-08-23 19:40:48 +03:00
tersec
6fd4983f6b
EIP-4788/parentBeaconBlockRoot fixes (#5318) 2023-08-19 08:38:17 +00:00
Etan Kissling
f70dbf1f53
use different argname than _ (#5307)
In Nim 2.0, if an argument is named `_` it can't be used in the impl.
Rename the `_` arg in `testblockutil` to something else.
2023-08-17 19:11:37 +00:00
Jacek Sieka
49729e1ef3
prevent concurrent storeBlock calls (fixes #5285) (#5295)
When a block is introduced to the system both via REST and gossip at the
same time, we will call `storeBlock` from two locations leading to a
dupliace check race condition as we wait for the EL.

This issue may manifest in particular when using an external block
builder that itself publishes the block onto the gossip network.

* refactor enqueue flow
* simplify calling `addBlock`
* complete request manager verifier future for blobless blocks
* re-verify parent conditions before adding block

among other things, it might have gone stale or finalized between one
call and the other
2023-08-17 15:12:37 +02:00
Eugene Kabanov
4a84457376
VC: Fix API timeouts issue. (#5297)
* Fix VC timeouts issue.
Add tests.

* Update AllTests.

* Remove assertion, which could crash VC when API call was interrupted by some higher logic.
2023-08-16 16:14:39 +00:00
tersec
85e1976ac3
automated consensus spec URL updating to v1.4.0-beta.1 (#5280) 2023-08-09 03:58:47 +00:00
Etan Kissling
6a2bac5cee
need even more log lines for debugging keymanager (#5260)
Keymanager test logs are still cut off, further increase log lines.
2023-08-07 18:02:08 +02:00
tersec
2c424cac87
Merkle is someone's name (#5265) 2023-08-07 10:06:47 +00:00
Etan Kissling
5b70a686e3
further adjust test_keymanager_api logs (#5259)
It is still unclear how `test_keymanager_api` sometimes fails in CI;
further adjust logging parameters.
2023-08-05 13:12:44 +00:00
Etan Kissling
4cf1a7e791
further reduce CI keymanager test log spam (#5250)
Remove verbose `gossip_eth2` logs not relevant for `test_keymanager_api`
from the log to debug CI issue.
2023-08-04 22:16:19 +00:00
Etan Kissling
8f18491131
adjust log level to reduce ci log spam (#5232)
In keymanager test, logs are currently not analyzable due to large
number of `libp2p gossipsub` logs. Hide those.
2023-08-03 23:09:08 +00:00