Commit Graph

2653 Commits

Author SHA1 Message Date
Etan Kissling 0c00b85782
cleanup LC data helpers (#3746)
Use more general `lowSlot` in LC data helpers, and avoid using
`earliestSlot` variable name as that one has a different meaning.
2022-06-14 22:02:03 +02:00
Etan Kissling cba041ddfa
fix LC data import for Altair fork period (#3744)
The initial sync committee period follows a different finality rule than
the other ones. Instead of next sync committee finalizing as soon as the
`finalizedHead.slot >= period.start_slot` have to use Altair start slot.
2022-06-14 17:31:10 +02:00
zah 0bfe7b10bb
Support displaying the version number in the status bar; Implements #2959 (#3747) 2022-06-14 14:53:58 +03:00
Jacek Sieka 099709872e
Merge branch 'stable' into unstable 2022-06-14 13:35:38 +02:00
Etan Kissling 52ba4f7999
rename light client config parameters (#3740)
For consistency with other options, use a common prefix for light client
data configuration options.

* `--serve-light-client-data` --> `--light-client-data-serve`
* `--import-light-client-data` --> `--light-client-data-import-mode`

No deprecation of the old identifiers as they were only sparingly used
and all usage can be easily updated without interferance.
2022-06-14 12:03:39 +03:00
tersec 8d421f3d91
keep fcU consistent with actual DAG (#3748) 2022-06-14 08:28:30 +00:00
Etan Kissling e3f0d2ecbc
remove unused `getExistingForkedBlock` overload (#3742)
Removes an unused overload of a local LC data function.
2022-06-14 08:19:11 +00:00
Etan Kissling 7b04a94d43
fix #3674 (Sync progress >100% on checkpoint sync) (#3736)
Corrects an off-by-1 in the reported sync percentage computation.
New logic is based on `SyncQueue.total` and `SyncQueue.progress`
with `pivot` instead of `sq.startSlot`.
2022-06-13 20:00:36 +03:00
tersec 51885519a2
lc in sepolia (#3734) 2022-06-11 01:36:45 +00:00
tersec 604b74abd2
move assert before compress (#3733) 2022-06-11 01:16:40 +00:00
tersec aa4f105c0c
improve panda display (#3732) 2022-06-11 00:48:04 +00:00
Etan Kissling 15967c4076
keep track of latest blocks for optimistic sync (#3715)
When launched with `--light-client-enable` the latest blocks are fetched
and optimistic candidate blocks are passed to a callback (log for now).
This helps accelerate syncing in the future (optimistic sync).
2022-06-10 14:16:37 +00:00
tersec cc5f95dbbb
separate non-zero exit code for doppelganger detection (#3728) 2022-06-10 14:53:19 +03:00
tersec 65cecc50ca
cleanups: unused and duplicate imports, inconsistent naming conventions, URL updates (#3724) 2022-06-09 14:30:13 +00:00
Zahary Karadjov 9750a2a760
Change the log level for unrecognized json fields from debug to trace 2022-06-09 16:08:50 +03:00
tersec 83793c3599
fix Nim 1.6 build deprecation warnings (#3712) 2022-06-09 12:09:38 +03:00
tersec 1dec3ff8b6
remove unused stdlib imports (#3718) 2022-06-09 08:50:36 +00:00
Jacek Sieka 7ec1521c52
use unsigned literals (#3717)
in the hopes of avoiding potential for conversion bugs on i386
2022-06-08 11:09:33 +00:00
Tanguy 29297e9ce4
Bump libp2p (#3709) 2022-06-08 07:53:50 +02:00
Etan Kissling 72a46bd520
integrate light client into beacon node (#3557)
Adds a `LightClient` instance to the beacon node as preparation to
accelerate syncing in the future (optimistic sync).

- `--light-client-enable` turns on the feature
- `--light-client-trusted-block-root` configures block to start from

If no block root is configured, light client tracks DAG `finalizedHead`.
2022-06-07 19:01:11 +02:00
tersec 7078833f56
per-validator suggested fee recipients (#3652) 2022-06-06 16:58:47 +03:00
Jacek Sieka b35584632b
sync: remove `step` from sync client implementation (#3678)
* sync: remove `step` from sync client implementation

Deprecated in the spec:
https://github.com/ethereum/consensus-specs/pull/2856 - future PR:s will
deprecate server support as well.
2022-06-06 16:56:59 +03:00
tersec 1262e50fa5
document MEV/mergemock test script (#3694) 2022-06-06 16:55:02 +03:00
tersec 38737549ac
refactor fork consistency checking and gate compilation on it (#3704) 2022-06-04 19:15:15 +00:00
Dustin Brody 21200f4a64 fix false-positive in overlap between default {CAPELLA,SHARDING}_FORK_VERIONs 2022-06-04 14:52:03 +00:00
tersec faf4d4a001
initial Capella support in RuntimeConfig (#3698) 2022-06-03 14:42:40 +00:00
tersec ea113fc420
disallow non-(genesis, far-future) equal transition epochs (#3691) 2022-06-03 09:37:03 +00:00
tersec 7492f99f35
update CL spec URLs (#3696) 2022-06-03 09:01:58 +00:00
Eugene Kabanov 50f9596108
Eliminate rpc_types.nim usage. (#3692) 2022-06-02 09:39:08 +00:00
tersec ce143a1078
update CL spec URLs (#3690) 2022-06-01 15:52:45 +00:00
tersec 62bfe97bbe
fix ExecutionPayload(Header) JSON serialization (#3679) 2022-06-01 14:57:28 +02:00
Jacek Sieka 6a0433cb08
REST parameter defaults (#3689)
* add default BN for VC
* unify "default" beacon node across commands
* clean up validator exit docs
2022-06-01 10:47:52 +00:00
tersec f929980bf3
update 20 CL spec ref URLs (#3677) 2022-05-31 11:15:31 +00:00
tersec 48f9631ed1
remove JSON-RPC consensus layer API client (#3676)
* remove JSON-RPC consensus layer API client

* rm eth2_json_rpc_serialization
2022-05-31 11:05:15 +00:00
Etan Kissling 01efa93cf6
add light client (standalone) (#3653)
Introduces a new library for syncing using libp2p based light client
sync protocol, and adds a new `nimbus_light_client` executable that uses
this library for syncing. The new executable emits log messages when
new beacon block headers are received, and is integrated into testing.
2022-05-31 12:45:37 +02:00
Jacek Sieka f31f52e24a
fix missing frontfill index (fixes #3658) (#3675)
* fix key load duration log
* log broken frontfill block root
2022-05-31 10:09:01 +02:00
Dustin Brody f124f22f10
Version 22.5.2 2022-05-30 15:36:40 +00:00
Jacek Sieka 48f01186d6
fix unnecessary HashList/HashArray cache invalidation (#3660)
* SSZ `[]` -> `mitem`
* `[]` -> `item`

immutable access via mutable instance cannot rely on template
overloading, and `[]` cannot be a `func` because of special seq handling
in compiler.
2022-05-30 13:30:42 +00:00
tersec a3413963a1
update (or for one, remove) 15 CL spec ref URLs (#3671) 2022-05-30 12:24:43 +00:00
Dustin Brody 14dc3855f6
change proposer boost to 40% 2022-05-30 10:18:49 +00:00
tersec 01534b0431
🐼 (#3670)
* 🐼

* rm panda refs outside core module; preprocess text/ANSI artwork sources

* credit artwork to beatscribe
2022-05-30 08:25:27 +00:00
tersec bf1763fdef
consensus specs v1.2.0-rc.1 (#3661) 2022-05-25 13:49:29 +00:00
tersec dfd8cd22b7
bump nim-web3 and use engine API v1.0.0.alpha.9 (#3663) 2022-05-25 10:30:37 +00:00
Etan Kissling acf1ec43f8
serve light client data on `ropsten` by default (#3659)
Applies a `ropsten` testnet specific config default to serve
light client data on that network.
2022-05-24 13:04:11 +00:00
tersec b3d603f364
more CL spec URL updates to v1.2.0-rc.1 (#3657) 2022-05-24 08:26:35 +00:00
tersec d35d408fa7
remove deprecated JSON-RPC server (#3656)
* remove deprecated JSON-RPC server

* keep the command-line options around as no-ops, temporarily

* service -> server; JSON-RPC is still used elsewhere
2022-05-24 07:23:48 +00:00
Jacek Sieka 1101c745b9
document and clean up `ValidatorIndex` usage (#3651)
* document static vs dynamic range checking requirements
* add `vindices` iterator to iterate over valid validator indices in a
state
* clean up spec comments in general

* fixup

Co-authored-by: tersec <tersec@users.noreply.github.com>
2022-05-23 23:39:08 +00:00
tersec c73239f60b
CL spec URL updates to v1.2.0-rc.1 (#3655) 2022-05-23 19:30:24 +00:00
tersec 5773f71ac0
update engine API URLs to v1.0.0-alpha.9 (#3654) 2022-05-23 15:41:34 +00:00
Etan Kissling c808f17a37
update to latest light client libp2p protocol (#3623)
Incorporates the latest changes to the light client sync protocol based
on Devconnect AMS feedback. Note that this breaks compatibility with the
previous prototype, due to changes to data structures and endpoints.
See https://github.com/ethereum/consensus-specs/pull/2802
2022-05-23 14:02:54 +02:00
Zahary Karadjov f7eff8fcbe
Bump the version number 2022-05-21 13:11:11 +03:00
zah e7ce3cacd0
Add support for the Ropsten beacon chain (#3648) 2022-05-20 18:26:07 +03:00
zah 68fb3962c8
More spec-compliant handling of unknown fields in REST json (#3647)
* More spec-compliant handling of unknown fields in REST json

* Address review comments
2022-05-20 18:25:26 +03:00
tersec 50e84156cc
only send JWT headers in Bellatrix-capable networks (#3645) 2022-05-19 20:36:47 +00:00
Zahary Karadjov 592c246b26
Version 22.5.0 2022-05-17 23:09:30 +03:00
zah 18968e9dfa
Bugfix: Invalid blocks were produced in the presence of invalid deposits (#3639)
Since we were not verifying BLS signature in blocks that we produce,
we were failing to notice that some deposits need to be ignored (due
to having an invalid signature). Processing these deposits resulted
in a different ending state after the state transition which caused
our blocks to be rejected by the network.
2022-05-17 22:56:15 +03:00
zah 397033a7d6
Improve the compatibility with various Web3Signer versions and configurations (#3640)
* Some Web3Signer versions insist replying with text/plain messages
* When reading blocks, the Web3Signer uses upper-case fork identifiers
  instead of lower-case identifies like the Beacon API.
2022-05-17 19:55:03 +03:00
tersec 1177f33363
standardize on upcoming/specified engine API timeouts (#3637) 2022-05-17 13:57:33 +00:00
tersec e7ed7332b3
MEV REST serialization and call signatures (#3625) 2022-05-17 13:56:19 +00:00
zah a0a6dd2f63
Add a ncli tool for converting a regular keystore into a distributed one (#3634) 2022-05-17 16:50:49 +03:00
tersec 6e28ce425a
remove --use-jwt-debug (#3619) 2022-05-11 17:50:19 +00:00
Etan Kissling 8cfb630aa9
never request blocks before `safeSlot` in sync (#3512)
Follows up on https://github.com/status-im/nimbus-eth2/pull/3461 which
ensured that repeated `beaconBlocksByRange` requests get shrinked to
account for potential out-of-band advancements to `safeSlot`, with
similar logic for the initial request.
2022-05-10 13:46:14 +02:00
zah a2ba34f686
Implement all sync committee duties in the validator client (#3583)
Other changes:

* logtrace can now verify sync committee messages and contributions
* Many unnecessary use of pairs() have been removed for consistency
* Map 40x BN response codes to BeaconNodeStatus.Incompatible in the VC
2022-05-10 10:03:40 +00:00
zah 6d11ad6ce1
Support for distributed keystores with multiple remotes based on threshold signatures (#3616)
Other fixes:

* Fix bit rot in the `make prater-dev-deposit` target.
* Correct content-type in the responses of the Nimbus signing node
* Invalid JSON payload was being sent in the web3signer requests
2022-05-10 03:32:12 +03:00
Jacek Sieka 011e0ca02f
era file verification (#3605)
* era file verification

Implement and document era file verification

* era file states now come with block applied for easier verification
* clarify conflicting version handling
* document verification requirements
* remove count from name, use start-era, end-root to discover range

* remove obsolete todo

* abstract out block root loading
2022-05-10 03:28:46 +03:00
tersec fc75c3ce36
tracking update to MEV SSZ objects (#3622) 2022-05-09 14:55:15 +00:00
tersec 7ddba43ff9
Add signed MEV SSZ objects (#3621) 2022-05-09 11:19:20 +00:00
tersec eea33b82a9
add current MEV builder SSZ types (#3620) 2022-05-08 22:42:21 +00:00
tersec 104cc3053f
fcU on syncing newPayload syncing response (#3618) 2022-05-08 09:09:46 +02:00
Jacek Sieka c6915ed3e0
remove faststreams backend for eth2_network (#3617)
unmaintained, incompatible with altair+
2022-05-08 09:08:13 +02:00
Etan Kissling fec933432d
update presets / configs / REST config for v1.1.10 (#3614)
Updated outdated presets / configs / REST config to v1.1.10 specs.

- `TERMINAL_BLOCK_HASH_ACTIVATION_EPOCH` and `PROPOSER_SCORE_BOOST` are
  not yet used in `eth2-networks`, added configurability as TODOs.

- `MIN_ANCHOR_POW_BLOCK_DIFFICULTY` is no longer needed, put on ignore
  list as some Altair devnets still reference it.
2022-05-06 09:51:28 +02:00
Jacek Sieka 138c40161d
avoid unnecessary recompression in block protocol (#3598)
Blocks can be sent straight from compressed data sources

Co-authored-by: Etan Kissling <etan@status.im>
2022-05-05 11:00:02 +00:00
tersec 7bb40d28ae
ensure MAX_CHUNK_SIZE usage consistent in sync_protocol (#3615) 2022-05-05 09:17:14 +00:00
tersec 4a372410a4
use MAX_CHUNK_SIZE_BELLATRIX for signed Bellatrix blocks (#3613)
* use MAX_CHUNK_SIZE_BELLATRIX for signed Bellatrix blocks

* Update beacon_chain/networking/eth2_network.nim

Co-authored-by: Etan Kissling <etan@status.im>

* localPassC to localPassc

* check against maxChunkSize rather than constant

Co-authored-by: Etan Kissling <etan@status.im>
2022-05-05 05:45:35 +00:00
tersec 3088da1303
enable JWT by default (#3610) 2022-04-30 12:18:06 +00:00
tersec a1ae53fbdf
don't always broadcast sync contributions and aggregates on Altair topics (#3611) 2022-04-28 10:51:49 +00:00
zah abfef0a1b3
Fix invalid merkle proof generation when there is a large number of unprocessed deposits (#3607) 2022-04-27 12:39:41 +03:00
tersec 0d5bbbc405
mitigate Eth1Monitor being nil in merge scenario (#3600) 2022-04-26 12:25:19 +03:00
tersec 09b2f4c237
unhide --jwt-secret flag (#3604) 2022-04-19 11:34:55 +00:00
tersec 9e738a92b4
stylecheck fixes (#3595) 2022-04-15 12:46:56 +00:00
Jacek Sieka d0dbc4a8f9
Snappy revamp (#3564)
This PR makes the necessary adjustments to deal with the revamped snappy
API.

In practical terms for nimbus-eth2, there are performance increases to
gossip processing, database reading and writing as well as era file
processing. Exporting `.era` files for example, a snappy-heavy
operation, almost halves in total processing time:

Pre:

```
     Average,       StdDev,          Min,          Max,      Samples,         Test
      39.088,        8.735,       23.619,       53.301,           50, tState
     237.079,       46.692,      165.620,      355.481,           49, tBlocks
```

Post:

```
All time are ms
     Average,       StdDev,          Min,          Max,      Samples,         Test
      25.350,        5.303,       15.351,       41.856,           50, tState
     141.238,       24.164,       99.990,      199.329,           49, tBlocks
```
2022-04-15 09:44:06 +02:00
tersec ab1fac7236
post-merge Bellatrix block proposals (#3570)
* post-merge Bellatrix block proposals

* tolerate running without an Eth1Monitor better

* remove obsolete comment

* use correct empty receipts root

* handle invalid CLI parameters in parseCmdArg overloads
2022-04-14 20:15:34 +00:00
tersec 61ba308e13
stylecheck fixes (#3593) 2022-04-14 17:39:37 +02:00
Eugene Kabanov 5592c7c674
NoMonitor and removed clock check for SyncManager. (#3420)
* Add `NoMonitor` flag to stop SyncManager from monitoring sync situation.

* Remove `toleranceValue` and `PeerScoreHeadTooNew`.

Co-authored-by: Etan Kissling <etan@status.im>
2022-04-14 15:17:44 +02:00
tersec 28ba2d5544
stylecheck fixes (#3592) 2022-04-14 13:47:14 +03:00
tersec 06da9f4868
remove proposer boosting option (always enabled now) (#3586) 2022-04-13 12:14:00 +03:00
tersec 8b74847f85
only initialize JWT if useJwt is set (#3585) 2022-04-12 22:28:01 +00:00
Dustin Brody 6ab5cb1729
fix underlying issue 2022-04-12 20:17:32 +00:00
tersec ff6c581273
keep proposer boosting permanently enabled (#3565) 2022-04-12 12:06:30 +02:00
Zahary Karadjov 039bece917
version 22.4.0 2022-04-12 00:02:28 +03:00
Zahary Karadjov def69e2a06
Revert "More sparse state snapshots in the Gnosis network"
This reverts commit 557717b517.
2022-04-11 13:56:42 +03:00
Zahary Karadjov ac4e7723ea
Fix the build 2022-04-10 23:10:40 +03:00
Zahary Karadjov 557717b517
More sparse state snapshots in the Gnosis network 2022-04-09 18:07:36 +03:00
tersec 3796810bcd
rm unused imports (#3580) 2022-04-08 21:35:52 +00:00
Jacek Sieka d1a8daec7e
`gnosis-chain` -> `gnosis` (#3577)
* `gnosis-chain` -> `gnosis`

Use same name as LH/Teku throughout

* fixes #3504
* fixes large stack temporary that can cause crashes during genesis
detection
2022-04-08 23:11:37 +03:00
Zahary Karadjov d450681b15
Fix another off-by-one causing rejected sync contributions at period boundaries 2022-04-08 22:47:47 +03:00
Jacek Sieka f70ff38b53
enable `styleCheck:usages` (#3573)
Some upstream repos still need fixes, but this gets us close enough that
style hints can be enabled by default.

In general, "canonical" spellings are preferred even if they violate
nep-1 - this applies in particular to spec-related stuff like
`genesis_validators_root` which appears throughout the codebase.
2022-04-08 16:22:49 +00:00
Jacek Sieka f20c53be82
Revert "Polish, remove beta warning for validator monitor (#3531)" (#3578)
This reverts commit 540fc5a1af.
2022-04-08 15:05:17 +00:00
Jacek Sieka f9f4a49c65
stop calling `eth_protocolVersion` (fixes #3532)
no longer supported by geth
2022-04-08 13:08:35 +02:00
Jacek Sieka 7207185bc3
bump nim-ssz-serialization (#3576)
currently pointing to a branch that was merged and no longer exists
2022-04-07 20:08:20 +00:00