Commit Graph

2012 Commits

Author SHA1 Message Date
Jacek Sieka 3d7bee8502
REST API client, JSON-RPC cleanups (#2756)
This refactoring puts the JSON-RPC and REST APIs on more equal footing
by renaming and moving things around, creating a separation between
client and server, and documenting what they are - the aim is to have a
simple-to-use base to start from when developing API clients, as well as
make it easier to navigate the code when looking for the legacy JSON-RPC
interface vs the new REST API.

* move REST client, serialization and supporting types to spec/eth2_apis
* REST stuff now starts with `rest_`, JSON-RPC stuff starts with `rpc_`,
more or less
* simplify imports such that there's a simple module to import for both
server and client
* map REST type and proc names to yaml spec more closely - in
particular, reuse operation and type names in `rest_types` to make
comparisons against spec more easy
* cleaner separation between client and server modules - modules common
between server and client such as `rest_types` and serialization move to
the spec folder - this allows the client to be built with less knowledge
about server internals
2021-08-03 17:17:11 +02:00
tersec d638ca0c7f
log overall sent aggregated attestation message signatures (#2754)
* log overall sent aggregated attestation message signatures

* log aggregated attestations via REST API
2021-08-03 10:32:55 +00:00
tersec a2c1b96ac9
prevent resyncing from genesis with altair head block (#2750) 2021-08-01 10:20:43 +02:00
Kim De Mey 06b2a1642d
Remove incorrect enode import (#2748) 2021-07-29 20:53:58 +02:00
tersec 5f3225c6e5
fix invalid_signature_no_participants test case (#2741) 2021-07-27 14:04:41 +00:00
nbc-bump-bot[bot] f68061b2d8
nim-libp2p auto bump (#2647)
* auto-bump nim-libp2p

* update peerhook for libp2p

Co-authored-by: = <Menduist@users.noreply.github.com>
Co-authored-by: Tanguy Cizain <tanguycizain@gmail.com>
2021-07-26 12:36:17 +00:00
tersec 38ce948647
partial altair merge (#2735)
* partial altair merge

* exclude still-in-flux sync committee data structures from partial merge

* undo the remaining sync_committee mention
2021-07-26 09:51:14 +00:00
Eugene Kabanov f9cd98702e
Fix getBlockHeaders() should return array. (#2732)
Fix tests.
2021-07-24 19:22:05 +02:00
Eugene Kabanov f0c30e31b4
VC: various fixes (#2730)
* Fix firstSuccess() template missing timeouts.

* Fix validator race condition.
Fix logs to be compatible with beacon_node logs.
Add CatchableError handlers to avoid crashes.
Move some logs from Notice to Debug level.
Fix some [unused] warnings.

* Fix block proposal issue for slots in the past and from the future.

* Change sent to published.

* Address review comments #1.
2021-07-19 14:31:02 +00:00
Kim De Mey 3e3e17fec3
Remove raises Exception and bump nim-eth (#2729) 2021-07-19 14:25:11 +02:00
tersec 9d9c37c561
some whole-file copies from altair branch (#2728)
* some whole-file copies from altair branch

* rpc/node_api and rpc/node_rest_api also need to be copied

* remove new sync committee-related functionality
2021-07-19 11:58:30 +00:00
tersec aebc606cb7
tighten local network simulation correctness checking (#2706)
* tighten local network simulation correctness checking

* rename rejectFirmly to errReject
2021-07-19 11:58:22 +00:00
zah d9f2a91374
Remove the obsolete testnet0/1 scripts (#2727)
Also fixes `make eth2_network_simulation`
2021-07-16 13:02:27 +03:00
Jacek Sieka 2d6a661ac6
Syncv2 (#2723)
* bump libp2p

* altair sync v2

Use V2 sync requests after the altair fork has happened, according to
the wall clock

* Fix the behavior of the v1 req/resp calls after Altair

Co-authored-by: Zahary Karadjov <zahary@gmail.com>
2021-07-15 21:01:07 +02:00
Eugene Kabanov 754aeec2b0
Fix and optimize aggregate and proofs generation algorithm. (#2722) 2021-07-15 08:17:32 +00:00
tersec e4afc36d71
use ForkedTrustedSignedBeaconBlock (#2720)
* use ForkedTrustedSignedBeaconBlock

* remove --subscribe-all-subnets

* https://ethereum.github.io/eth2.0-APIs/#/Beacon/getBlock implementation was passing through forked beaconblocks
2021-07-14 12:18:52 +00:00
Ștefan Talpalaru 840935ddc2
limit validator balance metric label values (#2719) 2021-07-14 08:22:03 +02:00
Jacek Sieka 3f9c1fdf4e
More RuntimeConfig cleanup (#2716)
* remove from BeaconChainDB (doesn't depend on runtime config)
* eth2-testnets -> eth2-networks
* use `cfg` name throughout
2021-07-13 16:27:10 +02:00
Eugene Kabanov 3b6f4fab4a
New validator client using REST API. (#2651)
* Initial commit.

* Exporting getConfig().

* Add beacon node checking procedures.

* Post rebase fixes.

* Use runSlotLoop() from nimbus_beacon_node.
Fallback implementation.
Fixes for ETH2 REST serialization.

* Add beacon_clock.durationToNextSlot().
Move type declarations from beacon_rest_api to json_rest_serialization.
Fix seq[ValidatorIndex] serialization.
Refactor ValidatorPool and add some utility procedures.
Create separate version of validator_client.

* Post-rebase fixes.
Remove CookedPubKey from validator_pool.nim.

* Now we should be able to produce attestations and aggregate and proofs.
But its not working yet.

* Debugging attestation sending.

* Add durationToNextAttestation.
Optimize some debug logs.
Fix aggregation_bits encoding.
Bump chronos/presto.

* Its alive.

* Fixes for launch_local_testnet script.
Bump chronos.

* Switch client API to not use `/api` prefix.

* Post-rebase adjustments.

* Fix endpoint for publishBlock().

* Add CONFIG_NAME.
Add more checks to ensure that beacon_node is compatible.

* Add beacon committee subscription support to validator_client.

* Fix stacktrace should be an array of strings.
Fix committee subscriptions should not be `data` keyed.

* Log duration to next block proposal.

* Fix beacon_node_status import.

* Use jsonMsgResponse() instead of jsonError().

* Fix graffityBytes usage.
Remove unnecessary `await`.
Adjust creation of SignedBlock instance.
Remove legacy files.

* Rework durationToNextSlot() and durationToNextEpoch() to use `fromNow`.

* Fix race condition for block proposal and attestations for same slot.
Fix local_testnet script to properly kill tasks on Windows.
Bump chronos and nim-http-tools, to allow connections to infura.io (basic auth).

* Catch services errors.
Improve performance of local_testnet.sh script on Windows.
Fix race condition when attestation producing.

* Post-rebase fixes.

* Bump chronos and presto.

* Calculate block publishing delay.
Fix pkill in one more place.

* Add error handling and timeouts to firstSuccess() template.
Add onceToAll() template.
Add checkNodes() procedure.
Refactor firstSuccess() template.
Add error checking to api.nim calls.

* Deprecated usage onceToAll() for better stability.
Address comment and send attestations asap.

* Avoid unnecessary loop when calculating minimal duration.
2021-07-13 13:15:07 +02:00
Kim De Mey 81d54a3c89
Avoid trying to close discv5 transport when discv5 is not enabled (#2713) 2021-07-13 09:58:45 +02:00
Jacek Sieka 23eea197f6
Implement split preset/config support (#2710)
* Implement split preset/config support

This is the initial bulk refactor to introduce runtime config values in
a number of places, somewhat replacing the existing mechanism of loading
network metadata.

It still needs more work, this is the initial refactor that introduces
runtime configuration in some of the places that need it.

The PR changes the way presets and constants work, to match the spec. In
particular, a "preset" now refers to the compile-time configuration
while a "cfg" or "RuntimeConfig" is the dynamic part.

A single binary can support either mainnet or minimal, but not both.
Support for other presets has been removed completely (can be readded,
in case there's need).

There's a number of outstanding tasks:

* `SECONDS_PER_SLOT` still needs fixing
* loading custom runtime configs needs redoing
* checking constants against YAML file

* yeerongpilly support

`build/nimbus_beacon_node --network=yeerongpilly --discv5:no --log-level=DEBUG`

* load fork epoch from config

* fix fork digest sent in status
* nicer error string for request failures
* fix tools

* one more

* fixup

* fixup

* fixup

* use "standard" network definition folder in local testnet

Files are loaded from their standard locations, including genesis etc,
to conform to the format used in the `eth2-networks` repo.

* fix launch scripts, allow unknown config values

* fix base config of rest test

* cleanups

* bundle mainnet config using common loader
* fix spec links and names
* only include supported preset in binary

* drop yeerongpilly, add altair-devnet-0, support boot_enr.yaml
2021-07-12 15:01:38 +02:00
Ștefan Talpalaru 0aef63948f
add version metric (using labels) (#2711) 2021-07-09 07:41:44 +02:00
zah eb2dc5cbbb
Implement the new Altair req/resp protocols (#2676)
* Implement the new Altair req/resp protocols

Also fixes the altair message-id computation by providing the correct
forkdigest prefix in `isAltairTopic`.

Co-authored-by: Tanguy Cizain <tanguycizain@gmail.com>
2021-07-07 12:09:47 +03:00
Jacek Sieka 7825d12448
increase block attestation wait time (#2705)
We generally send out attestations 250 ms after the block arrives.
Recent efficiency improvements have led to a slightly increased
incidence of "slot 0" issues  where attestations are dropped by other
nodes because they have not yet had time to process the block due to
epoch processing taking time.

This PR mitigates the problem by increasing the window between receiving
the block and sending out attestations.
2021-07-06 15:11:18 +02:00
tersec ac7f719382
use isomorphicCast between beacon block types (#2698) 2021-07-06 14:32:49 +02:00
tersec 5e9b932c3c
avoid recomputing base_reward_per_increment() across attestations (#2704)
* avoid recomputing base_reward_per_increment() across attestations

* add adapter template to fix nbench process_attestations build

* nfuzz also uses process_attestations()
2021-07-06 06:19:06 +00:00
tersec 62212c8493
make isomorphicCast available outside immutable beacon chain database (#2696)
* make isomorphicCast available outside immutable beacon chain database
2021-07-02 17:38:14 +00:00
tersec 68ca600d89
update 20 more spec refs to v1.1.0-alpha.8 (#2690) 2021-07-01 12:13:59 +00:00
tersec c9cec88a23
update some spec refs to v1.1.0-alpha.8 (#2689) 2021-07-01 08:55:16 +00:00
tersec 7577f8c2ef
add blockchain_dag altair database reading; add rollback tests (#2683)
* add blockchain_dag altair database reading; add rollback tests; fix some unnecessary type conversions

* remove debugging scaffolding

* proposeSignedBlock() will need to be async for merge; introduce altair types to VC
2021-06-29 15:09:29 +00:00
Jacek Sieka cc8f7c26a0
flush stdout logs (#2669)
else pipes end up not getting log output until much later
2021-06-29 16:53:36 +03:00
tersec 60f601e4e6
cache base_reward_per_increment for 5x+ Altair block_sim perf increase (#2681) 2021-06-28 06:54:06 +00:00
tersec ec54173667
optimize phase 0 -> Altair transition state processing (#2679) 2021-06-25 12:51:25 +00:00
Tanguy Cizain aeef550d0a
Altair message-id topic (#2671) 2021-06-25 03:07:46 +03:00
tersec 445def6c8b
block_clearance, ncli, and ncli_db Altair state saving (#2672)
* block_clearance, ncli, and ncli_db Altair state saving

* avoid invalidating SSZ hash caches with every assignment
2021-06-24 18:34:08 +00:00
tersec 41e0a7abc0
introduce database support for Altair (#2667)
* introduce immutable Altair BeaconState

* add database support for Altair blocks and states

* add tests for Altair get/put/contains/delete state

* enable blockchain_dag Altair state database storing

* properly return error on getting missing altair block
2021-06-24 07:11:47 +00:00
tersec ae1abf24af
add Altair support to block quarantine/clearance and block_sim (#2662)
* add Altair support to the block quarantine

* switch some spec/datatypes imports to spec/datatypes/base

* add Altair support to block_clearance

* allow runtime configuration of Altair transition slot

* enable Altair in block_sim, including in CI
2021-06-23 14:43:18 +00:00
Mamy André-Ratsimbazafy 8dc4db51c0
Merge branch 'stable' into unstable 2021-06-23 10:43:23 +02:00
Mamy André-Ratsimbazafy 4ef784fa68
v1.4.0 2021-06-21 14:02:12 +02:00
Eugene Kabanov 533cd25e1c
Add CONFIG_NAME parameter to REST API /eth/v1/config/spec response. (#2652)
* Add CONFIG_NAME parameter to /eth/v1/config/spec response.

* Add missed comma.

* Use `const_preset` instead of CONFIG_NAME.
2021-06-21 11:31:10 +02:00
tersec b1d5609171
remove false OnBlockAdded dependency on phase0 HashedBeaconState (#2661)
* remove false OnBlockAdded dependency on phase.HashedBeaconState

* introduce altair data types into block_clearance; update some alpha.6 spec refs to alpha.7; add get_active_validator_indices_len ForkedHashedBeaconState wrapper

* switch many modules from using datatypes (with phase0 states/blocks) to datatypes/base (fork-independent); update spec refs from alpha.6 to alpha.7 and remove rm'd G2_POINT_AT_INFINITY

* switch more modules from using datatypes (with phase0 states/blocks) to datatypes/base (fork-independent); update spec refs from alpha.6 to alpha.7

* remove unnecessary phase0-only wrapper of get_attesting_indices(); allow signatures_batch to process either fork; remove O(n^2) nested loop in process_inactivity_updates(); add altair support to getAttestationsforTestBlock()

* add Altair versions of asSigVerified(), asTrusted(), and makeBeaconBlock()

* fix spec URL to be Altair for Altair makeBeaconBlock()
2021-06-21 08:35:24 +00:00
Eugene Kabanov f8eb906b51
Add CONFIG_NAME parameter to REST API /eth/v1/config/spec response. (#2652)
* Add CONFIG_NAME parameter to /eth/v1/config/spec response.

* Add missed comma.

* Use `const_preset` instead of CONFIG_NAME.
2021-06-18 14:43:54 +02:00
tersec 9616220280
implement Altair attestation pool cache init (#2659)
* implement Altair attestation pool cache init

* remove code duplication around previous/current epoch updates
2021-06-17 17:13:14 +00:00
Jacek Sieka a3322acee9
Revert "fix assertion in beacon block creation rollback/restore (#2655)"
This reverts commit 9d48097de0.
2021-06-17 16:52:11 +02:00
Mamy Ratsimbazafy e14873757e
Fix #2595 - recompute wallSlot across async calls (#2650) 2021-06-17 16:43:58 +02:00
tersec 9ad8310157
doppelganger detection walltime refactor (#2656)
* strawman doppelganger detection walltime refactor

* move DoppelgangerProtection to Eth2Processor

* increase comment precision

* document difference between broadcastStartEpoch and nodeLaunchSlot, and allow for one-slot overlap to avoid false positives on intra-slot restarts
2021-06-17 16:43:58 +02:00
tersec 9d48097de0
fix assertion in beacon block creation rollback/restore (#2655) 2021-06-17 16:43:57 +02:00
Mamy Ratsimbazafy 2adf54e44a
Fix #2595 - recompute wallSlot across async calls (#2650) 2021-06-17 16:38:25 +02:00
tersec 87ed9e62a1
doppelganger detection walltime refactor (#2656)
* strawman doppelganger detection walltime refactor

* move DoppelgangerProtection to Eth2Processor

* increase comment precision

* document difference between broadcastStartEpoch and nodeLaunchSlot, and allow for one-slot overlap to avoid false positives on intra-slot restarts
2021-06-17 11:51:04 +00:00
tersec 53d05060c9
fix assertion in beacon block creation rollback/restore (#2655) 2021-06-17 09:22:39 +02:00