Commit Graph

609 Commits

Author SHA1 Message Date
Kim De Mey 0b1e0772de
Bump nim-eth to latest on master (#2927) 2021-09-30 09:00:47 +02:00
Zahary Karadjov 66b9f33edd Definitions related to the new execution engine JSON-RPC API 2021-09-30 00:20:21 +03:00
kdeme 933879e298 Bump nim-eth to allow for tcp/udp ports to always be configured in ENR 2021-09-29 20:15:24 +03:00
Mamy Ratsimbazafy 4aed54200d
fix multithreading memory leak (#2920)
* fix multithreading memory leak

* remove destructors for Nim 1.2 taskpool
2021-09-29 18:09:53 +02:00
Zahary Karadjov cc5d85331b Altair config for mainnet
This still doesn't work properly because it leads to a change in
the "eth" ENR field of the client. Since the client uses the value
of this field to look for other nodes on the network, the change
effectively prevents us from finding peers.
2021-09-29 14:19:20 +03:00
tersec f2df006ee9
update to v1.1.0 (#2909) 2021-09-28 04:28:33 +00:00
tersec edad6e5e83
use v1.1.0-beta.5 test vectors (#2900) 2021-09-25 03:07:07 +00:00
Etan Kissling aaaae89445
bump nimbus-build-system (#2898)
Changes:
- QUICK_AND_DIRTY_NIMBLE
- fix build output color on macOS
2021-09-24 15:34:23 +02:00
Eugene Kabanov b566d4657f
REST /eth/v1/events API call implementation. (#2878)
* Placing callbacks into strategic places.

* Initial events call implementation.

* Post rebase fixes.

* Change addSyncContribution() implementation.

* Add `attestation-sent` event.
Remove gcsafe, raises from callbacks implementations.
Move `attestation-received` fire at the end of attestation processing.

* Address review comments.
2021-09-22 14:17:15 +02:00
Jacek Sieka 855d0257ff
update lighthouse bootnodes (#2880)
https://github.com/eth2-clients/eth2-networks/pull/60
2021-09-21 13:34:25 +02:00
Mamy Ratsimbazafy 0dad1ec091
bump taskpools: remove default CPU pinning 2eb22c61b3 2021-09-21 11:09:48 +02:00
Mamy Ratsimbazafy 77f4169ffe
bump BLST v0.3.5 (#2876) 2021-09-17 10:20:21 +02:00
Mamy Ratsimbazafy d1cb5b7220
Parallel attestation verification (#2718)
* Add parallel attestation verification

* Update tests, batchVerify doesn't use the threadpool with only single core (nim-blscurve update)

* bump nim-blscurve

* Debug info for failing eth2 test vectors

* remove submodule eth2-testnets

* verbose debugging of make failure on Windows (libbacktrace?)

* Remove CI debug mode

* initialization convention

* Fix new altair tests
2021-09-17 03:13:52 +03:00
Eugene Kabanov 7ab1856c04
SSZ encoded responses for REST API calls. (#2851) 2021-09-16 16:32:32 +03:00
Zahary Karadjov a71de3feda Address review comments 2021-09-14 12:12:49 +03:00
tersec 5ea9e0baf3
use v1.1.0-beta.4 test vectors (#2861) 2021-09-10 17:35:14 +00:00
Kim De Mey ec736bc578
Bump nim-eth vendor module (#2848) 2021-09-06 21:59:20 +02:00
Jacek Sieka 6d47d96c84
altair upgrade for prater (#2828)
and a few import fixes for free
2021-08-27 16:54:51 +00:00
Jacek Sieka 01596c45dd
cleanups and fixes (#2827)
* import cleanup
* fix json-rpc exception handlers
* avoid unnecessary presto client import
* introduce ForkedBeaconBlock, some altair logging
* url fixes
2021-08-27 11:00:06 +02:00
Ștefan Talpalaru 5f148d9a26
bump vendor/news - this time the "status" branch (#2824) 2021-08-27 08:32:51 +02:00
tersec 4492775c6b
add random tests and rename "Official" to "Ethereum Foundation" (#2826)
* add random tests and rename "Official" to "Ethereum Foundation"

* checkDir = true covers dirExists(...)

* invalidate CI EF fixtures cache

* more correct cache invalidation
2021-08-27 06:30:30 +00:00
tersec 8de2342f60
use v1.1.0-beta.3 test vectors (#2825) 2021-08-26 16:01:21 +00:00
Ștefan Talpalaru b113d69f48
bump vendor/news (#2816) 2021-08-25 14:03:42 +00:00
tersec 79b2539ef0
eth2.0-spec-tests -> consensus-spec-tests repo rename (#2807) 2021-08-23 07:13:30 +00:00
Jacek Sieka e6e69f5f54
module bumps (#2800) 2021-08-20 18:39:18 +00:00
Ștefan Talpalaru bc37a6eb63
Windows: fix build when '.' not in PATH (#2787) 2021-08-20 14:00:17 +02:00
Jacek Sieka a7a65bce42
disentangle eth2 types from the ssz library (#2785)
* reorganize ssz dependencies

This PR continues the work in
https://github.com/status-im/nimbus-eth2/pull/2646,
https://github.com/status-im/nimbus-eth2/pull/2779 as well as past
issues with serialization and type, to disentangle SSZ from eth2 and at
the same time simplify imports and exports with a structured approach.

The principal idea here is that when a library wants to introduce SSZ
support, they do so via 3 files:

* `ssz_codecs` which imports and reexports `codecs` - this covers the
basic byte conversions and ensures no overloads get lost
* `xxx_merkleization` imports and exports `merkleization` to specialize
and get access to `hash_tree_root` and friends
* `xxx_ssz_serialization` imports and exports `ssz_serialization` to
specialize ssz for a specific library

Those that need to interact with SSZ always import the `xxx_` versions
of the modules and never `ssz` itself so as to keep imports simple and
safe.

This is similar to how the REST / JSON-RPC serializers are structured in
that someone wanting to serialize spec types to REST-JSON will import
`eth2_rest_serialization` and nothing else.

* split up ssz into a core library that is independendent of eth2 types
* rename `bytes_reader` to `codec` to highlight that it contains coding
and decoding of bytes and native ssz types
* remove tricky List init overload that causes compile issues
* get rid of top-level ssz import
* reenable merkleization tests
* move some "standard" json serializers to spec
* remove `ValidatorIndex` serialization for now
* remove test_ssz_merkleization
* add tests for over/underlong byte sequences
* fix broken seq[byte] test - seq[byte] is not an SSZ type

There are a few things this PR doesn't solve:

* like #2646 this PR is weak on how to handle root and other
dontSerialize fields that "sometimes" should be computed - the same
problem appears in REST / JSON-RPC etc

* Fix a build problem on macOS

* Another way to fix the macOS builds

Co-authored-by: Zahary Karadjov <zahary@gmail.com>
2021-08-18 20:57:58 +02:00
Ștefan Talpalaru 9caf852dd4
cron CI: various fixes (#2594) 2021-08-18 14:40:03 +02:00
Jacek Sieka 63717531dc remove remaining traces of nim-prompt
upstream dead, as is feature in eth2
2021-08-16 21:56:50 +03:00
Jacek Sieka 584e70e1cb
bump faststreams (#2783) 2021-08-16 07:50:38 +02:00
Jacek Sieka 07728e4f7c
altair devnet 3 (#2782) 2021-08-15 19:50:52 +02:00
Jacek Sieka 92aba71604
Pyrmont altair fork (#2771)
Prater upgrading to altair at epoch 61650 -
https://github.com/eth2-clients/eth2-networks/pull/56

also includes altair-devnet-2 support in binary
2021-08-10 10:16:43 +02:00
Eugene Kabanov b5dd6ca11c
Bump `chronos` and `http-utils` to fix number of issues. (#2765) 2021-08-06 18:37:51 +00:00
Ștefan Talpalaru e576064788
bump nimbus-build-system (#2749) 2021-07-30 10:27:56 +02:00
tersec 941cc125a3
update to v1.1.0-beta.2 tests, with exception for altair sync committees (#2736)
* update to v1.1.0-beta.2 tests, with exception for altair sync committees

* bootstrap node can't drop any attestations
2021-07-26 17:34:13 +00:00
nbc-bump-bot[bot] f68061b2d8
nim-libp2p auto bump (#2647)
* auto-bump nim-libp2p

* update peerhook for libp2p

Co-authored-by: = <Menduist@users.noreply.github.com>
Co-authored-by: Tanguy Cizain <tanguycizain@gmail.com>
2021-07-26 12:36:17 +00:00
Kim De Mey 3e3e17fec3
Remove raises Exception and bump nim-eth (#2729) 2021-07-19 14:25:11 +02:00
Jacek Sieka 2d6a661ac6
Syncv2 (#2723)
* bump libp2p

* altair sync v2

Use V2 sync requests after the altair fork has happened, according to
the wall clock

* Fix the behavior of the v1 req/resp calls after Altair

Co-authored-by: Zahary Karadjov <zahary@gmail.com>
2021-07-15 21:01:07 +02:00
tersec 11541ff61b
bump nim-eth2-scenarios to use v1.1.0-beta.1 test vectors (#2724) 2021-07-15 09:04:51 +00:00
Jacek Sieka 3f9c1fdf4e
More RuntimeConfig cleanup (#2716)
* remove from BeaconChainDB (doesn't depend on runtime config)
* eth2-testnets -> eth2-networks
* use `cfg` name throughout
2021-07-13 16:27:10 +02:00
Eugene Kabanov 3b6f4fab4a
New validator client using REST API. (#2651)
* Initial commit.

* Exporting getConfig().

* Add beacon node checking procedures.

* Post rebase fixes.

* Use runSlotLoop() from nimbus_beacon_node.
Fallback implementation.
Fixes for ETH2 REST serialization.

* Add beacon_clock.durationToNextSlot().
Move type declarations from beacon_rest_api to json_rest_serialization.
Fix seq[ValidatorIndex] serialization.
Refactor ValidatorPool and add some utility procedures.
Create separate version of validator_client.

* Post-rebase fixes.
Remove CookedPubKey from validator_pool.nim.

* Now we should be able to produce attestations and aggregate and proofs.
But its not working yet.

* Debugging attestation sending.

* Add durationToNextAttestation.
Optimize some debug logs.
Fix aggregation_bits encoding.
Bump chronos/presto.

* Its alive.

* Fixes for launch_local_testnet script.
Bump chronos.

* Switch client API to not use `/api` prefix.

* Post-rebase adjustments.

* Fix endpoint for publishBlock().

* Add CONFIG_NAME.
Add more checks to ensure that beacon_node is compatible.

* Add beacon committee subscription support to validator_client.

* Fix stacktrace should be an array of strings.
Fix committee subscriptions should not be `data` keyed.

* Log duration to next block proposal.

* Fix beacon_node_status import.

* Use jsonMsgResponse() instead of jsonError().

* Fix graffityBytes usage.
Remove unnecessary `await`.
Adjust creation of SignedBlock instance.
Remove legacy files.

* Rework durationToNextSlot() and durationToNextEpoch() to use `fromNow`.

* Fix race condition for block proposal and attestations for same slot.
Fix local_testnet script to properly kill tasks on Windows.
Bump chronos and nim-http-tools, to allow connections to infura.io (basic auth).

* Catch services errors.
Improve performance of local_testnet.sh script on Windows.
Fix race condition when attestation producing.

* Post-rebase fixes.

* Bump chronos and presto.

* Calculate block publishing delay.
Fix pkill in one more place.

* Add error handling and timeouts to firstSuccess() template.
Add onceToAll() template.
Add checkNodes() procedure.
Refactor firstSuccess() template.
Add error checking to api.nim calls.

* Deprecated usage onceToAll() for better stability.
Address comment and send attestations asap.

* Avoid unnecessary loop when calculating minimal duration.
2021-07-13 13:15:07 +02:00
tersec 667a01ef42
bump nim-eth2-scenarios to stop downloading alpha.5 test vectors (#2714) 2021-07-12 21:56:48 +00:00
Jacek Sieka 23eea197f6
Implement split preset/config support (#2710)
* Implement split preset/config support

This is the initial bulk refactor to introduce runtime config values in
a number of places, somewhat replacing the existing mechanism of loading
network metadata.

It still needs more work, this is the initial refactor that introduces
runtime configuration in some of the places that need it.

The PR changes the way presets and constants work, to match the spec. In
particular, a "preset" now refers to the compile-time configuration
while a "cfg" or "RuntimeConfig" is the dynamic part.

A single binary can support either mainnet or minimal, but not both.
Support for other presets has been removed completely (can be readded,
in case there's need).

There's a number of outstanding tasks:

* `SECONDS_PER_SLOT` still needs fixing
* loading custom runtime configs needs redoing
* checking constants against YAML file

* yeerongpilly support

`build/nimbus_beacon_node --network=yeerongpilly --discv5:no --log-level=DEBUG`

* load fork epoch from config

* fix fork digest sent in status
* nicer error string for request failures
* fix tools

* one more

* fixup

* fixup

* fixup

* use "standard" network definition folder in local testnet

Files are loaded from their standard locations, including genesis etc,
to conform to the format used in the `eth2-networks` repo.

* fix launch scripts, allow unknown config values

* fix base config of rest test

* cleanups

* bundle mainnet config using common loader
* fix spec links and names
* only include supported preset in binary

* drop yeerongpilly, add altair-devnet-0, support boot_enr.yaml
2021-07-12 15:01:38 +02:00
Ștefan Talpalaru 17da9aaa09
fix metrics on Windows (#2707) 2021-07-07 10:19:41 +02:00
Mamy Ratsimbazafy 38a5bc8de3
bump blscurve closes #2693 and #2695 (#2700) 2021-07-06 14:31:54 +02:00
tersec df381d0cf0
update spec test fixtures to v1.1.0-alpha.8 (#2687) 2021-06-29 16:47:09 +00:00
Ștefan Talpalaru 39591ac858
bump two submodules (#2657) 2021-06-17 17:07:57 +00:00
tersec 1c3314f08b
update to Altair as of v1.1.0-alpha.7 (#2649)
* update to Altair as of v1.1.0-alpha.7

* introduce Altair types into attestation pool

* avoid allocating/copying pubkeys excessively in get_next_sync_committee()
2021-06-14 17:42:46 +00:00
Jacek Sieka d859bc12f0
write uncompressed validator keys to database (#2639)
* write uncompressed validator keys to database

Loading 150k+ validator keys on startup in compressed format takes a lot
of time - better store them in uncompressed format which makes behaviour
just after startup faster / more predictable.

* refactor cached validator key access
* fix isomorphic cast to work with non-var instances
* remove cooked pubkey cache - directly use database cache in chaindag
as well (one less cache to keep in sync)
* bump blscurve, introduce loadValid for known-to-be-valid keys
2021-06-10 10:37:02 +03:00
nbc-bump-bot[bot] 1a76007858
auto-bump nim-libp2p (#2591)
Co-authored-by: = <arnetheduck@users.noreply.github.com>
2021-06-04 12:36:03 +02:00