Commit Graph

2071 Commits

Author SHA1 Message Date
Dustin Brody 6638476b5f discard putative blocks from invalid hardforks 2021-09-16 16:18:40 +03:00
Etan Kissling 7ccaeed4a8 remove forked message accessor
The template `message` extracts fork specific data from a forked block.
However, its current implementation does not compile due to each fork
returning a different type, i.e., the generated `case` code attempts to
return different types from each branch. As the template is unused, it
is safely removed for now.
2021-09-15 13:58:35 +03:00
Zahary Karadjov a71de3feda Address review comments 2021-09-14 12:12:49 +03:00
Zahary Karadjov 3e4faaa213 Add .editorconfig file and remove trailing newlines 2021-09-14 12:12:49 +03:00
Zahary Karadjov add7daa30c Light client sync spec functions 2021-09-14 12:12:49 +03:00
tersec 97b0070774
add merge SSZ consensus object tests (#2858)
* add merge SSZ consensus object tests

* add merge attestations, attester slashing, proposer slashing, and voluntary exit test fixtures

* throw catchable exception rather than assert on invalid SingleMemberUnion selector

* hash_tree_root can assert, because it's trusted data by that point

* re-assert on writing, since also trusted data

* beta.4 update

* implement upgrade_to_merge()
2021-09-11 10:01:05 +02:00
tersec 0ad7216bc4
mass update from beta.3 spec refs to beta.4 spec refs (#2862) 2021-09-10 18:56:03 +00:00
tersec 5ea9e0baf3
use v1.1.0-beta.4 test vectors (#2861) 2021-09-10 17:35:14 +00:00
tersec ea722c6f6e
update some spec refs to beta.3 (#2853) 2021-09-08 14:49:04 +00:00
Zahary Karadjov 2121cf1f0d Clarify why 'eth1Network' is an Option type 2021-09-08 15:47:48 +03:00
tersec 9c0d9b546a
successfull -> successful (#2842) 2021-09-01 18:08:24 +02:00
tersec 9e145afda3
update 29 Altair spec ref URLs to beta.3 (#2839) 2021-08-31 12:16:27 +00:00
Zahary Karadjov 7d1efa443d Restore the sync committee pool pruning and add tests 2021-08-30 11:06:45 +03:00
zah 3689c68cbf
Carry out the sync committee gossip duties
Other changes:

* Add server getBlockV2(), and produceBlockV2().
* Add getBlockV2() to REST test suite.
* Add client getBlockV2(), and produceBlockV2().
* Fix URLs in comments.
* Add some primitives and fix some issues in forks.nim.
* Switch `validator_client` to V2 calls usage.
* Bump `chronos` with imports fixes.
* Bump `nim-json-serialization` for `requireAllFields`.
2021-08-30 03:58:30 +03:00
tersec 82eac7a522
add additional REST test suite rules (#2835) 2021-08-29 18:12:24 +00:00
tersec 2d8a796a93
altair-capable beacon block creation (#2834)
* altair-capable beacon block creation

* update block_sim to use sync committees and the new block production interface
2021-08-29 14:50:21 +00:00
tersec 3efcdb0de5
subscribe to/unsubscribe from sync committee subnets (#2832) 2021-08-29 05:58:27 +00:00
tersec 0418fbada2
introduce SyncCommitteeMsgPool to eth2_processor and nimbus_beacon_node (#2831) 2021-08-28 22:27:51 +00:00
tersec 166e22a43b
sync committee message pool and gossip validation (#2830) 2021-08-28 10:40:01 +00:00
tersec eeba2869fc
add sync committee types and some helpers (#2829) 2021-08-28 09:00:00 +00:00
Jacek Sieka 6d47d96c84
altair upgrade for prater (#2828)
and a few import fixes for free
2021-08-27 16:54:51 +00:00
Jacek Sieka 6a4bf98ea2
better error messages for keystore operations (#2812)
in particular, incluse os error string
2021-08-27 16:53:21 +00:00
Jacek Sieka 01596c45dd
cleanups and fixes (#2827)
* import cleanup
* fix json-rpc exception handlers
* avoid unnecessary presto client import
* introduce ForkedBeaconBlock, some altair logging
* url fixes
2021-08-27 11:00:06 +02:00
tersec 9725d15a3e
update spec references from eth2.0-specs to consensus-specs and to v1.1.0-beta.2 (#2822) 2021-08-26 10:21:52 +02:00
Jacek Sieka fb42a3af9b
fix mainnet eth1 for mainnet eth2 (#2819) 2021-08-25 20:10:54 +02:00
tersec 84ee4b2e9f
40% faster Altair epoch slot processing (#2814)
* construct all unslashed, participating balances in one validator scan

* remove altair benchmarking setup for block_sim

* revert a benchmarking change from proc to func

* remove more benchmarking func/proc tweaks

* re-add asSeq, which is necessary because unsafeAddr

* ... except for the block_sim benchmark part
2021-08-25 14:43:00 +00:00
tersec 98547e0c6b
double altair epoch slot processing speed (#2811)
* approximately double altair epoch processing speed

* don't redundantly clear state balances cache

* only invalidate altair state balances cache once in process_rewards_and_penalties()
2021-08-24 22:09:03 +02:00
Jacek Sieka ba06f13942
cleanups (#2809)
* cleanups

* use ForkedTrustedSignedBeaconBlock.ionit where appropriate
* move `is_aggregator` to `spec/`
* use `errReject` in a few more places
* update enr fork id when time is auspicious
* use network broadcast functions

* Return Ignore for aggregate signature validation timeouts

...consistently between aggregates and attestations.

* clean up some more reject/ignore rules
* shorten texts a bit

* errReject->checkedReject, use err helpers throughout

* get rid of quarantine in exitpool as well
2021-08-24 21:49:51 +02:00
Jacek Sieka c7d4659d32 remove `updateRoot` param from SSZ
call `readSszBytes` directly to skip root updates
2021-08-23 16:49:43 +03:00
Eugene Kabanov 66cb18d69b
Number of REST fixes for Altair. (#2790)
* Fix getForkSchedule call.
Create cache of all configuration endpoints at node startup.
Add prepareJsonResponse() call to create cached responses.
Mark all procedures with `raises`.

* Add getForkSchedule to VC.
Fix getForkSchedule return type for API.
More `raises` annotations.
Fix VC fork_service.nim.

* Use `push raises` instead of inline `raises`.

* Improvements for REST API aggregated attestations and attestations processing.

* Rename eth2_network.sendXXX procedures to eth2_network.broadcastXXX.
Add broadcastBeaconBlock() and broadcastAggregateAndProof().
Fix links to specification in REST API declarations.
Add implementation for v2 getStateV2().
Add validator_duties.sendXXX procedures which not only broadcast data, but also validate it.
Fix JSON-RPC/REST to use new validator_duties.sendXXX procedures instead of own implementations.

* Fix validator_client online nodes count incorrect value.
Fix aggregate and proof attestation could be sent too late.

* Adding timeout for block wait in attestations processing.
Fix compilation errors.

* Attempt to debug aggregate and proofs.

* Fix Beacon AIP to use `sendAttestation`.
Add link comment to produceBlockV2.

* Add debug logs before publish operation for blocks, attestations and aggregated attestations.
Fix attestations publishing issue.

* logging fixes

`indexInCommnittee` already logged in attestation

Co-authored-by: Jacek Sieka <jacek@status.im>
2021-08-23 12:41:48 +02:00
tersec 0a61d1112e
add errIgnore helper and refactor errReject helper (#2808) 2021-08-23 12:39:06 +02:00
Tanguy Cizain d9f7ba7153
Peer cycling (#2668)
* start of peer cycling

* discover new peers when low coverage

* disconnect unresponsive peers

* cleaning up a bit

* smarter cycling

* remove lowpeer event

* use sync_protocol to get metadata

* moved cycling to discovery

* fix discovery for tests

* change test

* various tweaks

* Faster discovery

* start of big refacto

* removed peer_balancer

* updated test

* added lazydial

* allow at least 10 incoming conn

* popcount -> countOnes

* allow max_conn - outgoingconn incoming connections

* add dOut check

* Removed lazyDial option and better kick

* use attnets in scoring

* only score peer with metadata

* cleanup

* use topic subscription

* rework dialing filtering, fix metadata compat issue
2021-08-23 12:29:50 +02:00
tersec 092d9350de
eth2.0-specs -> consensus-specs repo rename (#2801) 2021-08-20 23:37:45 +00:00
tersec 7d048ce783
avoid repeated `integer_squareroot`s for 20% faster Altair epoch processing (#2797)
* avoid repeated integer_squareroots for 20% faster altair epoch processing

* the square root of a Gwei isn't a Gwei
2021-08-20 16:41:32 +02:00
zah ff3766e79b
Fix the Prater ENR fork id (#2798) 2021-08-20 15:04:29 +02:00
tersec 4678a2bee7
move BeaconClock from ChainDAG to BeaconNode (#2796) 2021-08-20 08:58:15 +00:00
tersec 317b6de4e6
send attestations and exit messages on fork-appropriate topic (#2773)
* send attestations and exit messages on fork-appropriate topic

* document why use wall clock over attestation slot

* centralize some fork-topic-picking-logic in eth2_network

* pick up new test in summary

* allow specified GetTimeFn for testing purposes

* add GenesisTime and use it in eth2_network

* replace GetTimeFn and GenesisTime with GetBeaconTimeFn
2021-08-19 10:45:31 +00:00
Jacek Sieka a7a65bce42
disentangle eth2 types from the ssz library (#2785)
* reorganize ssz dependencies

This PR continues the work in
https://github.com/status-im/nimbus-eth2/pull/2646,
https://github.com/status-im/nimbus-eth2/pull/2779 as well as past
issues with serialization and type, to disentangle SSZ from eth2 and at
the same time simplify imports and exports with a structured approach.

The principal idea here is that when a library wants to introduce SSZ
support, they do so via 3 files:

* `ssz_codecs` which imports and reexports `codecs` - this covers the
basic byte conversions and ensures no overloads get lost
* `xxx_merkleization` imports and exports `merkleization` to specialize
and get access to `hash_tree_root` and friends
* `xxx_ssz_serialization` imports and exports `ssz_serialization` to
specialize ssz for a specific library

Those that need to interact with SSZ always import the `xxx_` versions
of the modules and never `ssz` itself so as to keep imports simple and
safe.

This is similar to how the REST / JSON-RPC serializers are structured in
that someone wanting to serialize spec types to REST-JSON will import
`eth2_rest_serialization` and nothing else.

* split up ssz into a core library that is independendent of eth2 types
* rename `bytes_reader` to `codec` to highlight that it contains coding
and decoding of bytes and native ssz types
* remove tricky List init overload that causes compile issues
* get rid of top-level ssz import
* reenable merkleization tests
* move some "standard" json serializers to spec
* remove `ValidatorIndex` serialization for now
* remove test_ssz_merkleization
* add tests for over/underlong byte sequences
* fix broken seq[byte] test - seq[byte] is not an SSZ type

There are a few things this PR doesn't solve:

* like #2646 this PR is weak on how to handle root and other
dontSerialize fields that "sometimes" should be computed - the same
problem appears in REST / JSON-RPC etc

* Fix a build problem on macOS

* Another way to fix the macOS builds

Co-authored-by: Zahary Karadjov <zahary@gmail.com>
2021-08-18 20:57:58 +02:00
tersec a060985abc
unexport various parts of tests/ and remove unused code (#2794) 2021-08-18 13:58:43 +00:00
Jacek Sieka 70259e4e64
treat gossip decoding errors more strictly (#2793)
* penalize peers for sending gossip messages that fail decoding
* add metrics for decoding/decompression errors
* clean up obsolete exception handlers
2021-08-18 14:30:05 +02:00
Jacek Sieka bfe5e74607
fix produceBlock response to return correct type (#2792) 2021-08-18 11:00:16 +02:00
tersec 6e46445da2
switch result = foo to expression return; unexport rest of logtrace symbols (#2788) 2021-08-17 09:51:39 +00:00
tersec a0c518cb4f
sync committee/aggregate signature signing and verification (#2784)
* sync committee/aggregate signature signing and verification

* add message signature tests
2021-08-17 08:07:17 +00:00
Jacek Sieka 63717531dc remove remaining traces of nim-prompt
upstream dead, as is feature in eth2
2021-08-16 21:56:50 +03:00
Jacek Sieka 07728e4f7c
altair devnet 3 (#2782) 2021-08-15 19:50:52 +02:00
tersec 89d9aa9240
sync committee topic names; borrow -> template/distinctbase (#2781) 2021-08-15 16:50:31 +02:00
Jacek Sieka 7a622e8505
rework spec imports (#2779)
The spec imports are a mess to work with, so this branch cleans them up
a bit to ensure that we avoid generic sandwitches and that importing
stuff generally becomes easier.

* reexport crypto/digest/presets because these are part of the public
symbol set of the rest of the spec types
* don't export `merge` types from `base` - this causes circular deps
* fix circular deps in `ssz/spec_types` - this is the first step in
disentangling ssz from spec
* be explicit about phase0 vs altair - longer term, `altair` will become
the "natural" type set, then merge and so on, so no point in giving
`phase0` special preferential treatment
2021-08-12 13:08:20 +00:00
Jacek Sieka 9697b73e71
forkedbeaconstate_helpers -> forks (#2772)
Simpler module name for stuff that covers forks

* check that runtime config matches database state
* also include some assorted altair cleanups
* use "standard" genesis fork in local testnet to work around missing
runtime config support
2021-08-10 22:46:35 +02:00
Jacek Sieka 92aba71604
Pyrmont altair fork (#2771)
Prater upgrading to altair at epoch 61650 -
https://github.com/eth2-clients/eth2-networks/pull/56

also includes altair-devnet-2 support in binary
2021-08-10 10:16:43 +02:00
tersec 2eb6246b90
altair enr switching (#2770)
* altair enr switching

* altair transitions only happen on epoch boundaries

* determine ENR period by wall time, not chain head
2021-08-10 08:19:13 +02:00