Commit Graph

3837 Commits

Author SHA1 Message Date
Jacek Sieka 3d7bee8502
REST API client, JSON-RPC cleanups (#2756)
This refactoring puts the JSON-RPC and REST APIs on more equal footing
by renaming and moving things around, creating a separation between
client and server, and documenting what they are - the aim is to have a
simple-to-use base to start from when developing API clients, as well as
make it easier to navigate the code when looking for the legacy JSON-RPC
interface vs the new REST API.

* move REST client, serialization and supporting types to spec/eth2_apis
* REST stuff now starts with `rest_`, JSON-RPC stuff starts with `rpc_`,
more or less
* simplify imports such that there's a simple module to import for both
server and client
* map REST type and proc names to yaml spec more closely - in
particular, reuse operation and type names in `rest_types` to make
comparisons against spec more easy
* cleaner separation between client and server modules - modules common
between server and client such as `rest_types` and serialization move to
the spec folder - this allows the client to be built with less knowledge
about server internals
2021-08-03 17:17:11 +02:00
tersec d638ca0c7f
log overall sent aggregated attestation message signatures (#2754)
* log overall sent aggregated attestation message signatures

* log aggregated attestations via REST API
2021-08-03 10:32:55 +00:00
Ștefan Talpalaru ca96a98131
launch_local_testnet.sh: start the timeout earlier (#2753) 2021-08-03 12:12:21 +02:00
Mamy Ratsimbazafy b3ff184537
CI: nim-beacon-chain + update MinGW64 path (#2752)
* CI: nim-beacon-chain + update MinGW64 path

* debugging leftovers
2021-08-02 18:40:38 +02:00
tersec a2c1b96ac9
prevent resyncing from genesis with altair head block (#2750) 2021-08-01 10:20:43 +02:00
Ștefan Talpalaru e576064788
bump nimbus-build-system (#2749) 2021-07-30 10:27:56 +02:00
Kim De Mey 06b2a1642d
Remove incorrect enode import (#2748) 2021-07-29 20:53:58 +02:00
Ștefan Talpalaru 17c94c4d8e
restapi.sh: prevent port clashes (#2746) 2021-07-29 18:34:01 +02:00
tersec f0321d2eb5
remove inspector (#2737) 2021-07-29 14:00:04 +00:00
tersec 5664bf8c4d
synchronize AllTests-mainnet.md with new zero signature sanity checks (#2745) 2021-07-28 20:36:23 +00:00
Hans Löfgren 46fec09c49
Update beacon_nodes_Grafana_dashboard.json (#2744)
Changes `"datasource": null` to `"datasource: "Prometheus"` for all panels to avoid default in case Grafana has more datasources loaded and not using Prometheus as default.
2021-07-28 18:20:57 +02:00
Mamy Ratsimbazafy 3a1cab3790
Ensure that zero signatures cannot be deserialized https://github.com/status-im/nimbus-eth2/pull/2733#issuecomment-888107561 (#2743) 2021-07-28 15:20:41 +02:00
Ștefan Talpalaru 7fe4569586
launch_local_testnet.sh: check if lsof is installed (#2742) 2021-07-27 21:45:36 +02:00
Ștefan Talpalaru 6299e14ffb
launch_local_testnet.sh: kill old procs before start (#2739) 2021-07-27 17:57:16 +02:00
tersec 5f3225c6e5
fix invalid_signature_no_participants test case (#2741) 2021-07-27 14:04:41 +00:00
Eugene Kabanov 4ba69bf54d
Disable discovery5 service for REST test. (#2738)
Run REST test before Finalization test.
2021-07-26 19:55:24 +00:00
tersec 941cc125a3
update to v1.1.0-beta.2 tests, with exception for altair sync committees (#2736)
* update to v1.1.0-beta.2 tests, with exception for altair sync committees

* bootstrap node can't drop any attestations
2021-07-26 17:34:13 +00:00
nbc-bump-bot[bot] f68061b2d8
nim-libp2p auto bump (#2647)
* auto-bump nim-libp2p

* update peerhook for libp2p

Co-authored-by: = <Menduist@users.noreply.github.com>
Co-authored-by: Tanguy Cizain <tanguycizain@gmail.com>
2021-07-26 12:36:17 +00:00
tersec 38ce948647
partial altair merge (#2735)
* partial altair merge

* exclude still-in-flux sync committee data structures from partial merge

* undo the remaining sync_committee mention
2021-07-26 09:51:14 +00:00
Eugene Kabanov f9cd98702e
Fix getBlockHeaders() should return array. (#2732)
Fix tests.
2021-07-24 19:22:05 +02:00
Eugene Kabanov f0c30e31b4
VC: various fixes (#2730)
* Fix firstSuccess() template missing timeouts.

* Fix validator race condition.
Fix logs to be compatible with beacon_node logs.
Add CatchableError handlers to avoid crashes.
Move some logs from Notice to Debug level.
Fix some [unused] warnings.

* Fix block proposal issue for slots in the past and from the future.

* Change sent to published.

* Address review comments #1.
2021-07-19 14:31:02 +00:00
Kim De Mey 3e3e17fec3
Remove raises Exception and bump nim-eth (#2729) 2021-07-19 14:25:11 +02:00
tersec 9d9c37c561
some whole-file copies from altair branch (#2728)
* some whole-file copies from altair branch

* rpc/node_api and rpc/node_rest_api also need to be copied

* remove new sync committee-related functionality
2021-07-19 11:58:30 +00:00
tersec aebc606cb7
tighten local network simulation correctness checking (#2706)
* tighten local network simulation correctness checking

* rename rejectFirmly to errReject
2021-07-19 11:58:22 +00:00
zah d9f2a91374
Remove the obsolete testnet0/1 scripts (#2727)
Also fixes `make eth2_network_simulation`
2021-07-16 13:02:27 +03:00
Jacek Sieka 2d6a661ac6
Syncv2 (#2723)
* bump libp2p

* altair sync v2

Use V2 sync requests after the altair fork has happened, according to
the wall clock

* Fix the behavior of the v1 req/resp calls after Altair

Co-authored-by: Zahary Karadjov <zahary@gmail.com>
2021-07-15 21:01:07 +02:00
tersec 11541ff61b
bump nim-eth2-scenarios to use v1.1.0-beta.1 test vectors (#2724) 2021-07-15 09:04:51 +00:00
Eugene Kabanov 754aeec2b0
Fix and optimize aggregate and proofs generation algorithm. (#2722) 2021-07-15 08:17:32 +00:00
tersec e4afc36d71
use ForkedTrustedSignedBeaconBlock (#2720)
* use ForkedTrustedSignedBeaconBlock

* remove --subscribe-all-subnets

* https://ethereum.github.io/eth2.0-APIs/#/Beacon/getBlock implementation was passing through forked beaconblocks
2021-07-14 12:18:52 +00:00
0xmiel bb0a086be3
book updates (#2721)
* cp

* rename API to JSON-RPC API

* rename api to json-rpc api, add rest api page

* remove extra emacs generated file

* remove extra emacs generated file
2021-07-14 13:14:39 +02:00
Ștefan Talpalaru 840935ddc2
limit validator balance metric label values (#2719) 2021-07-14 08:22:03 +02:00
Jacek Sieka 3f9c1fdf4e
More RuntimeConfig cleanup (#2716)
* remove from BeaconChainDB (doesn't depend on runtime config)
* eth2-testnets -> eth2-networks
* use `cfg` name throughout
2021-07-13 16:27:10 +02:00
Eugene Kabanov 3b6f4fab4a
New validator client using REST API. (#2651)
* Initial commit.

* Exporting getConfig().

* Add beacon node checking procedures.

* Post rebase fixes.

* Use runSlotLoop() from nimbus_beacon_node.
Fallback implementation.
Fixes for ETH2 REST serialization.

* Add beacon_clock.durationToNextSlot().
Move type declarations from beacon_rest_api to json_rest_serialization.
Fix seq[ValidatorIndex] serialization.
Refactor ValidatorPool and add some utility procedures.
Create separate version of validator_client.

* Post-rebase fixes.
Remove CookedPubKey from validator_pool.nim.

* Now we should be able to produce attestations and aggregate and proofs.
But its not working yet.

* Debugging attestation sending.

* Add durationToNextAttestation.
Optimize some debug logs.
Fix aggregation_bits encoding.
Bump chronos/presto.

* Its alive.

* Fixes for launch_local_testnet script.
Bump chronos.

* Switch client API to not use `/api` prefix.

* Post-rebase adjustments.

* Fix endpoint for publishBlock().

* Add CONFIG_NAME.
Add more checks to ensure that beacon_node is compatible.

* Add beacon committee subscription support to validator_client.

* Fix stacktrace should be an array of strings.
Fix committee subscriptions should not be `data` keyed.

* Log duration to next block proposal.

* Fix beacon_node_status import.

* Use jsonMsgResponse() instead of jsonError().

* Fix graffityBytes usage.
Remove unnecessary `await`.
Adjust creation of SignedBlock instance.
Remove legacy files.

* Rework durationToNextSlot() and durationToNextEpoch() to use `fromNow`.

* Fix race condition for block proposal and attestations for same slot.
Fix local_testnet script to properly kill tasks on Windows.
Bump chronos and nim-http-tools, to allow connections to infura.io (basic auth).

* Catch services errors.
Improve performance of local_testnet.sh script on Windows.
Fix race condition when attestation producing.

* Post-rebase fixes.

* Bump chronos and presto.

* Calculate block publishing delay.
Fix pkill in one more place.

* Add error handling and timeouts to firstSuccess() template.
Add onceToAll() template.
Add checkNodes() procedure.
Refactor firstSuccess() template.
Add error checking to api.nim calls.

* Deprecated usage onceToAll() for better stability.
Address comment and send attestations asap.

* Avoid unnecessary loop when calculating minimal duration.
2021-07-13 13:15:07 +02:00
Kim De Mey 81d54a3c89
Avoid trying to close discv5 transport when discv5 is not enabled (#2713) 2021-07-13 09:58:45 +02:00
tersec 6b07b79427
re-add minimal tests, now that const sanity checks gone (#2715) 2021-07-13 08:38:35 +02:00
tersec 667a01ef42
bump nim-eth2-scenarios to stop downloading alpha.5 test vectors (#2714) 2021-07-12 21:56:48 +00:00
Jacek Sieka 23eea197f6
Implement split preset/config support (#2710)
* Implement split preset/config support

This is the initial bulk refactor to introduce runtime config values in
a number of places, somewhat replacing the existing mechanism of loading
network metadata.

It still needs more work, this is the initial refactor that introduces
runtime configuration in some of the places that need it.

The PR changes the way presets and constants work, to match the spec. In
particular, a "preset" now refers to the compile-time configuration
while a "cfg" or "RuntimeConfig" is the dynamic part.

A single binary can support either mainnet or minimal, but not both.
Support for other presets has been removed completely (can be readded,
in case there's need).

There's a number of outstanding tasks:

* `SECONDS_PER_SLOT` still needs fixing
* loading custom runtime configs needs redoing
* checking constants against YAML file

* yeerongpilly support

`build/nimbus_beacon_node --network=yeerongpilly --discv5:no --log-level=DEBUG`

* load fork epoch from config

* fix fork digest sent in status
* nicer error string for request failures
* fix tools

* one more

* fixup

* fixup

* fixup

* use "standard" network definition folder in local testnet

Files are loaded from their standard locations, including genesis etc,
to conform to the format used in the `eth2-networks` repo.

* fix launch scripts, allow unknown config values

* fix base config of rest test

* cleanups

* bundle mainnet config using common loader
* fix spec links and names
* only include supported preset in binary

* drop yeerongpilly, add altair-devnet-0, support boot_enr.yaml
2021-07-12 15:01:38 +02:00
Mamy André-Ratsimbazafy 07a1c5716b
[v1.4.1] Merge branch 'testing' into stable 2021-07-10 17:35:50 +02:00
Mamy André-Ratsimbazafy 6f441191e4
v1.4.1 changelog and version 2021-07-10 15:18:32 +02:00
Ștefan Talpalaru 0aef63948f
add version metric (using labels) (#2711) 2021-07-09 07:41:44 +02:00
Ștefan Talpalaru bbced08f63
fix metrics on Windows (#2707) 2021-07-07 12:12:02 +02:00
Jacek Sieka bd684d0eea
increase block attestation wait time (#2705)
We generally send out attestations 250 ms after the block arrives.
Recent efficiency improvements have led to a slightly increased
incidence of "slot 0" issues  where attestations are dropped by other
nodes because they have not yet had time to process the block due to
epoch processing taking time.

This PR mitigates the problem by increasing the window between receiving
the block and sending out attestations.
2021-07-07 12:11:51 +02:00
Jacek Sieka be756454a4
flush stdout logs (#2669)
else pipes end up not getting log output until much later
2021-07-07 12:11:12 +02:00
zah eb2dc5cbbb
Implement the new Altair req/resp protocols (#2676)
* Implement the new Altair req/resp protocols

Also fixes the altair message-id computation by providing the correct
forkdigest prefix in `isAltairTopic`.

Co-authored-by: Tanguy Cizain <tanguycizain@gmail.com>
2021-07-07 12:09:47 +03:00
Ștefan Talpalaru 17da9aaa09
fix metrics on Windows (#2707) 2021-07-07 10:19:41 +02:00
Jacek Sieka 7825d12448
increase block attestation wait time (#2705)
We generally send out attestations 250 ms after the block arrives.
Recent efficiency improvements have led to a slightly increased
incidence of "slot 0" issues  where attestations are dropped by other
nodes because they have not yet had time to process the block due to
epoch processing taking time.

This PR mitigates the problem by increasing the window between receiving
the block and sending out attestations.
2021-07-06 15:11:18 +02:00
tersec ac7f719382
use isomorphicCast between beacon block types (#2698) 2021-07-06 14:32:49 +02:00
Mamy Ratsimbazafy 38a5bc8de3
bump blscurve closes #2693 and #2695 (#2700) 2021-07-06 14:31:54 +02:00
Eugene Kabanov 41c2d5f040
Store REST test logs as CI artifacts. (#2699)
* Store resttest logs as artifacts.

* Specify data-dir and sleep-timeout.
2021-07-06 08:40:18 +00:00
tersec 5e9b932c3c
avoid recomputing base_reward_per_increment() across attestations (#2704)
* avoid recomputing base_reward_per_increment() across attestations

* add adapter template to fix nbench process_attestations build

* nfuzz also uses process_attestations()
2021-07-06 06:19:06 +00:00