Commit Graph

3530 Commits

Author SHA1 Message Date
Tanguy Cizain d9f7ba7153
Peer cycling (#2668)
* start of peer cycling

* discover new peers when low coverage

* disconnect unresponsive peers

* cleaning up a bit

* smarter cycling

* remove lowpeer event

* use sync_protocol to get metadata

* moved cycling to discovery

* fix discovery for tests

* change test

* various tweaks

* Faster discovery

* start of big refacto

* removed peer_balancer

* updated test

* added lazydial

* allow at least 10 incoming conn

* popcount -> countOnes

* allow max_conn - outgoingconn incoming connections

* add dOut check

* Removed lazyDial option and better kick

* use attnets in scoring

* only score peer with metadata

* cleanup

* use topic subscription

* rework dialing filtering, fix metadata compat issue
2021-08-23 12:29:50 +02:00
tersec 79b2539ef0
eth2.0-spec-tests -> consensus-spec-tests repo rename (#2807) 2021-08-23 07:13:30 +00:00
Zachinquarantine 860897172a
connect-eth2.md: fix small typo (#2804) 2021-08-22 11:24:56 +02:00
Zachinquarantine 80f2d4d9ee
run-beacon-node.sh: update URL (#2803) 2021-08-21 22:39:01 +02:00
tersec 092d9350de
eth2.0-specs -> consensus-specs repo rename (#2801) 2021-08-20 23:37:45 +00:00
Jacek Sieka e6e69f5f54
module bumps (#2800) 2021-08-20 18:39:18 +00:00
tersec 7d048ce783
avoid repeated `integer_squareroot`s for 20% faster Altair epoch processing (#2797)
* avoid repeated integer_squareroots for 20% faster altair epoch processing

* the square root of a Gwei isn't a Gwei
2021-08-20 16:41:32 +02:00
Jacek Sieka a67970e9a4
subscribe to all subnets in minimal local testnet (#2799) 2021-08-20 16:41:16 +02:00
zah ff3766e79b
Fix the Prater ENR fork id (#2798) 2021-08-20 15:04:29 +02:00
Ștefan Talpalaru bc37a6eb63
Windows: fix build when '.' not in PATH (#2787) 2021-08-20 14:00:17 +02:00
tersec 4678a2bee7
move BeaconClock from ChainDAG to BeaconNode (#2796) 2021-08-20 08:58:15 +00:00
tersec 317b6de4e6
send attestations and exit messages on fork-appropriate topic (#2773)
* send attestations and exit messages on fork-appropriate topic

* document why use wall clock over attestation slot

* centralize some fork-topic-picking-logic in eth2_network

* pick up new test in summary

* allow specified GetTimeFn for testing purposes

* add GenesisTime and use it in eth2_network

* replace GetTimeFn and GenesisTime with GetBeaconTimeFn
2021-08-19 10:45:31 +00:00
Jacek Sieka a7a65bce42
disentangle eth2 types from the ssz library (#2785)
* reorganize ssz dependencies

This PR continues the work in
https://github.com/status-im/nimbus-eth2/pull/2646,
https://github.com/status-im/nimbus-eth2/pull/2779 as well as past
issues with serialization and type, to disentangle SSZ from eth2 and at
the same time simplify imports and exports with a structured approach.

The principal idea here is that when a library wants to introduce SSZ
support, they do so via 3 files:

* `ssz_codecs` which imports and reexports `codecs` - this covers the
basic byte conversions and ensures no overloads get lost
* `xxx_merkleization` imports and exports `merkleization` to specialize
and get access to `hash_tree_root` and friends
* `xxx_ssz_serialization` imports and exports `ssz_serialization` to
specialize ssz for a specific library

Those that need to interact with SSZ always import the `xxx_` versions
of the modules and never `ssz` itself so as to keep imports simple and
safe.

This is similar to how the REST / JSON-RPC serializers are structured in
that someone wanting to serialize spec types to REST-JSON will import
`eth2_rest_serialization` and nothing else.

* split up ssz into a core library that is independendent of eth2 types
* rename `bytes_reader` to `codec` to highlight that it contains coding
and decoding of bytes and native ssz types
* remove tricky List init overload that causes compile issues
* get rid of top-level ssz import
* reenable merkleization tests
* move some "standard" json serializers to spec
* remove `ValidatorIndex` serialization for now
* remove test_ssz_merkleization
* add tests for over/underlong byte sequences
* fix broken seq[byte] test - seq[byte] is not an SSZ type

There are a few things this PR doesn't solve:

* like #2646 this PR is weak on how to handle root and other
dontSerialize fields that "sometimes" should be computed - the same
problem appears in REST / JSON-RPC etc

* Fix a build problem on macOS

* Another way to fix the macOS builds

Co-authored-by: Zahary Karadjov <zahary@gmail.com>
2021-08-18 20:57:58 +02:00
tersec a060985abc
unexport various parts of tests/ and remove unused code (#2794) 2021-08-18 13:58:43 +00:00
Ștefan Talpalaru 9caf852dd4
cron CI: various fixes (#2594) 2021-08-18 14:40:03 +02:00
Jacek Sieka 70259e4e64
treat gossip decoding errors more strictly (#2793)
* penalize peers for sending gossip messages that fail decoding
* add metrics for decoding/decompression errors
* clean up obsolete exception handlers
2021-08-18 14:30:05 +02:00
Jacek Sieka bfe5e74607
fix produceBlock response to return correct type (#2792) 2021-08-18 11:00:16 +02:00
tersec 6e46445da2
switch result = foo to expression return; unexport rest of logtrace symbols (#2788) 2021-08-17 09:51:39 +00:00
tersec a0c518cb4f
sync committee/aggregate signature signing and verification (#2784)
* sync committee/aggregate signature signing and verification

* add message signature tests
2021-08-17 08:07:17 +00:00
Jacek Sieka 63717531dc remove remaining traces of nim-prompt
upstream dead, as is feature in eth2
2021-08-16 21:56:50 +03:00
Jacek Sieka c3a0e4ec16
API doc fixes
`getSpecPreset` was removed pre-1.0.0 and has never worked - debug calls
were renamed around 1.1.0.
2021-08-16 18:41:13 +02:00
Jacek Sieka 584e70e1cb
bump faststreams (#2783) 2021-08-16 07:50:38 +02:00
Jacek Sieka 07728e4f7c
altair devnet 3 (#2782) 2021-08-15 19:50:52 +02:00
tersec 89d9aa9240
sync committee topic names; borrow -> template/distinctbase (#2781) 2021-08-15 16:50:31 +02:00
tersec df48409969
enable trace-level networking logging in local simulation (#2780)
* enable trace-level networking logging in local simulation

* remove Jenkinsfile overrides
2021-08-13 14:12:40 +00:00
Jacek Sieka 7a622e8505
rework spec imports (#2779)
The spec imports are a mess to work with, so this branch cleans them up
a bit to ensure that we avoid generic sandwitches and that importing
stuff generally becomes easier.

* reexport crypto/digest/presets because these are part of the public
symbol set of the rest of the spec types
* don't export `merge` types from `base` - this causes circular deps
* fix circular deps in `ssz/spec_types` - this is the first step in
disentangling ssz from spec
* be explicit about phase0 vs altair - longer term, `altair` will become
the "natural" type set, then merge and so on, so no point in giving
`phase0` special preferential treatment
2021-08-12 13:08:20 +00:00
0xmiel 01f78fb93e
update links to joe's guides (#2775) 2021-08-11 15:29:49 +02:00
0xmiel fe96c3e993
add page on security audit (#2774)
* add page on security audit

* incorporate feedback
2021-08-11 15:16:02 +02:00
Jacek Sieka 9697b73e71
forkedbeaconstate_helpers -> forks (#2772)
Simpler module name for stuff that covers forks

* check that runtime config matches database state
* also include some assorted altair cleanups
* use "standard" genesis fork in local testnet to work around missing
runtime config support
2021-08-10 22:46:35 +02:00
Jacek Sieka 92aba71604
Pyrmont altair fork (#2771)
Prater upgrading to altair at epoch 61650 -
https://github.com/eth2-clients/eth2-networks/pull/56

also includes altair-devnet-2 support in binary
2021-08-10 10:16:43 +02:00
tersec 2eb6246b90
altair enr switching (#2770)
* altair enr switching

* altair transitions only happen on epoch boundaries

* determine ENR period by wall time, not chain head
2021-08-10 08:19:13 +02:00
Jacek Sieka a663ed65f4
Merge branch 'merge-stable' into unstable 2021-08-09 15:00:58 +02:00
tersec 2afe2802b6
altair topic switching (#2767)
* altair topic switching

* remove validate{Committee,Validator}IndexOr unused within branch
2021-08-09 12:54:45 +00:00
Jacek Sieka 7bb76a6cd1
Merge remote-tracking branch 'origin/stable' into merge-stable 2021-08-09 13:14:28 +02:00
Eugene Kabanov b69f566ed1
Implementation for v2 REST API calls. (#2731)
* Implementation for v2 rest api calls.

* fix phase0 block deserialization

Co-authored-by: Jacek Sieka <jacek@status.im>
2021-08-09 08:08:18 +02:00
tersec 61825f4979
add aggregated attestation tracing to logtrace and enable it in Jenkins (#2766)
* add aggregated attestation tracing to logtrace and enable it in Jenkins CI

* use a slightly less cryptic acronym than aasr

* mostly, nimbus and the eth2 spec use aggregate attestation, not aggregated attestation
2021-08-06 21:25:30 +00:00
Ștefan Talpalaru 780a9cfd42
release: add multi-arch Docker images (#2763) 2021-08-06 20:40:07 +02:00
Eugene Kabanov b5dd6ca11c
Bump `chronos` and `http-utils` to fix number of issues. (#2765) 2021-08-06 18:37:51 +00:00
Zahary Karadjov 1e958cfb0a version 1.4.2 2021-08-05 14:28:17 +03:00
Ștefan Talpalaru 59b1a4772c
don't try to use lsof on Windows (#2762) 2021-08-05 10:49:34 +02:00
Jacek Sieka ee79c10a7d
update validator key cache on startup (#2760)
* update validator key cache on startup

Versions prior to 1.1.0 do not write a validator key cache at all.

Versions from 1.4.0 and upwards require an immutable validator key cache
to verify blocks - normally, block verification fills the cache but that
assumes that at least one block was verified by a version that has the
key cache.

Taken together, this breaks direct upgrades from anything <1.1.0 to
1.4.0.

The fix is simply to refresh fill the cache from an existing state on
startup.

* also log serious block validation failures at info level
2021-08-05 11:26:10 +03:00
Ștefan Talpalaru c4bf4f8fff
Apple M1: disable -march=native (#2759) 2021-08-04 15:22:23 +02:00
tersec 30cd8c79d8
add FixtureAll-minimal.md (#2757) 2021-08-04 09:07:20 +00:00
Jacek Sieka 3d7bee8502
REST API client, JSON-RPC cleanups (#2756)
This refactoring puts the JSON-RPC and REST APIs on more equal footing
by renaming and moving things around, creating a separation between
client and server, and documenting what they are - the aim is to have a
simple-to-use base to start from when developing API clients, as well as
make it easier to navigate the code when looking for the legacy JSON-RPC
interface vs the new REST API.

* move REST client, serialization and supporting types to spec/eth2_apis
* REST stuff now starts with `rest_`, JSON-RPC stuff starts with `rpc_`,
more or less
* simplify imports such that there's a simple module to import for both
server and client
* map REST type and proc names to yaml spec more closely - in
particular, reuse operation and type names in `rest_types` to make
comparisons against spec more easy
* cleaner separation between client and server modules - modules common
between server and client such as `rest_types` and serialization move to
the spec folder - this allows the client to be built with less knowledge
about server internals
2021-08-03 17:17:11 +02:00
tersec d638ca0c7f
log overall sent aggregated attestation message signatures (#2754)
* log overall sent aggregated attestation message signatures

* log aggregated attestations via REST API
2021-08-03 10:32:55 +00:00
Ștefan Talpalaru ca96a98131
launch_local_testnet.sh: start the timeout earlier (#2753) 2021-08-03 12:12:21 +02:00
Mamy Ratsimbazafy b3ff184537
CI: nim-beacon-chain + update MinGW64 path (#2752)
* CI: nim-beacon-chain + update MinGW64 path

* debugging leftovers
2021-08-02 18:40:38 +02:00
tersec a2c1b96ac9
prevent resyncing from genesis with altair head block (#2750) 2021-08-01 10:20:43 +02:00
Ștefan Talpalaru e576064788
bump nimbus-build-system (#2749) 2021-07-30 10:27:56 +02:00
Kim De Mey 06b2a1642d
Remove incorrect enode import (#2748) 2021-07-29 20:53:58 +02:00