Commit Graph

2001 Commits

Author SHA1 Message Date
tersec aebc606cb7
tighten local network simulation correctness checking (#2706)
* tighten local network simulation correctness checking

* rename rejectFirmly to errReject
2021-07-19 11:58:22 +00:00
zah d9f2a91374
Remove the obsolete testnet0/1 scripts (#2727)
Also fixes `make eth2_network_simulation`
2021-07-16 13:02:27 +03:00
Jacek Sieka 2d6a661ac6
Syncv2 (#2723)
* bump libp2p

* altair sync v2

Use V2 sync requests after the altair fork has happened, according to
the wall clock

* Fix the behavior of the v1 req/resp calls after Altair

Co-authored-by: Zahary Karadjov <zahary@gmail.com>
2021-07-15 21:01:07 +02:00
Eugene Kabanov 754aeec2b0
Fix and optimize aggregate and proofs generation algorithm. (#2722) 2021-07-15 08:17:32 +00:00
tersec e4afc36d71
use ForkedTrustedSignedBeaconBlock (#2720)
* use ForkedTrustedSignedBeaconBlock

* remove --subscribe-all-subnets

* https://ethereum.github.io/eth2.0-APIs/#/Beacon/getBlock implementation was passing through forked beaconblocks
2021-07-14 12:18:52 +00:00
Ștefan Talpalaru 840935ddc2
limit validator balance metric label values (#2719) 2021-07-14 08:22:03 +02:00
Jacek Sieka 3f9c1fdf4e
More RuntimeConfig cleanup (#2716)
* remove from BeaconChainDB (doesn't depend on runtime config)
* eth2-testnets -> eth2-networks
* use `cfg` name throughout
2021-07-13 16:27:10 +02:00
Eugene Kabanov 3b6f4fab4a
New validator client using REST API. (#2651)
* Initial commit.

* Exporting getConfig().

* Add beacon node checking procedures.

* Post rebase fixes.

* Use runSlotLoop() from nimbus_beacon_node.
Fallback implementation.
Fixes for ETH2 REST serialization.

* Add beacon_clock.durationToNextSlot().
Move type declarations from beacon_rest_api to json_rest_serialization.
Fix seq[ValidatorIndex] serialization.
Refactor ValidatorPool and add some utility procedures.
Create separate version of validator_client.

* Post-rebase fixes.
Remove CookedPubKey from validator_pool.nim.

* Now we should be able to produce attestations and aggregate and proofs.
But its not working yet.

* Debugging attestation sending.

* Add durationToNextAttestation.
Optimize some debug logs.
Fix aggregation_bits encoding.
Bump chronos/presto.

* Its alive.

* Fixes for launch_local_testnet script.
Bump chronos.

* Switch client API to not use `/api` prefix.

* Post-rebase adjustments.

* Fix endpoint for publishBlock().

* Add CONFIG_NAME.
Add more checks to ensure that beacon_node is compatible.

* Add beacon committee subscription support to validator_client.

* Fix stacktrace should be an array of strings.
Fix committee subscriptions should not be `data` keyed.

* Log duration to next block proposal.

* Fix beacon_node_status import.

* Use jsonMsgResponse() instead of jsonError().

* Fix graffityBytes usage.
Remove unnecessary `await`.
Adjust creation of SignedBlock instance.
Remove legacy files.

* Rework durationToNextSlot() and durationToNextEpoch() to use `fromNow`.

* Fix race condition for block proposal and attestations for same slot.
Fix local_testnet script to properly kill tasks on Windows.
Bump chronos and nim-http-tools, to allow connections to infura.io (basic auth).

* Catch services errors.
Improve performance of local_testnet.sh script on Windows.
Fix race condition when attestation producing.

* Post-rebase fixes.

* Bump chronos and presto.

* Calculate block publishing delay.
Fix pkill in one more place.

* Add error handling and timeouts to firstSuccess() template.
Add onceToAll() template.
Add checkNodes() procedure.
Refactor firstSuccess() template.
Add error checking to api.nim calls.

* Deprecated usage onceToAll() for better stability.
Address comment and send attestations asap.

* Avoid unnecessary loop when calculating minimal duration.
2021-07-13 13:15:07 +02:00
Kim De Mey 81d54a3c89
Avoid trying to close discv5 transport when discv5 is not enabled (#2713) 2021-07-13 09:58:45 +02:00
Jacek Sieka 23eea197f6
Implement split preset/config support (#2710)
* Implement split preset/config support

This is the initial bulk refactor to introduce runtime config values in
a number of places, somewhat replacing the existing mechanism of loading
network metadata.

It still needs more work, this is the initial refactor that introduces
runtime configuration in some of the places that need it.

The PR changes the way presets and constants work, to match the spec. In
particular, a "preset" now refers to the compile-time configuration
while a "cfg" or "RuntimeConfig" is the dynamic part.

A single binary can support either mainnet or minimal, but not both.
Support for other presets has been removed completely (can be readded,
in case there's need).

There's a number of outstanding tasks:

* `SECONDS_PER_SLOT` still needs fixing
* loading custom runtime configs needs redoing
* checking constants against YAML file

* yeerongpilly support

`build/nimbus_beacon_node --network=yeerongpilly --discv5:no --log-level=DEBUG`

* load fork epoch from config

* fix fork digest sent in status
* nicer error string for request failures
* fix tools

* one more

* fixup

* fixup

* fixup

* use "standard" network definition folder in local testnet

Files are loaded from their standard locations, including genesis etc,
to conform to the format used in the `eth2-networks` repo.

* fix launch scripts, allow unknown config values

* fix base config of rest test

* cleanups

* bundle mainnet config using common loader
* fix spec links and names
* only include supported preset in binary

* drop yeerongpilly, add altair-devnet-0, support boot_enr.yaml
2021-07-12 15:01:38 +02:00
Ștefan Talpalaru 0aef63948f
add version metric (using labels) (#2711) 2021-07-09 07:41:44 +02:00
zah eb2dc5cbbb
Implement the new Altair req/resp protocols (#2676)
* Implement the new Altair req/resp protocols

Also fixes the altair message-id computation by providing the correct
forkdigest prefix in `isAltairTopic`.

Co-authored-by: Tanguy Cizain <tanguycizain@gmail.com>
2021-07-07 12:09:47 +03:00
Jacek Sieka 7825d12448
increase block attestation wait time (#2705)
We generally send out attestations 250 ms after the block arrives.
Recent efficiency improvements have led to a slightly increased
incidence of "slot 0" issues  where attestations are dropped by other
nodes because they have not yet had time to process the block due to
epoch processing taking time.

This PR mitigates the problem by increasing the window between receiving
the block and sending out attestations.
2021-07-06 15:11:18 +02:00
tersec ac7f719382
use isomorphicCast between beacon block types (#2698) 2021-07-06 14:32:49 +02:00
tersec 5e9b932c3c
avoid recomputing base_reward_per_increment() across attestations (#2704)
* avoid recomputing base_reward_per_increment() across attestations

* add adapter template to fix nbench process_attestations build

* nfuzz also uses process_attestations()
2021-07-06 06:19:06 +00:00
tersec 62212c8493
make isomorphicCast available outside immutable beacon chain database (#2696)
* make isomorphicCast available outside immutable beacon chain database
2021-07-02 17:38:14 +00:00
tersec 68ca600d89
update 20 more spec refs to v1.1.0-alpha.8 (#2690) 2021-07-01 12:13:59 +00:00
tersec c9cec88a23
update some spec refs to v1.1.0-alpha.8 (#2689) 2021-07-01 08:55:16 +00:00
tersec 7577f8c2ef
add blockchain_dag altair database reading; add rollback tests (#2683)
* add blockchain_dag altair database reading; add rollback tests; fix some unnecessary type conversions

* remove debugging scaffolding

* proposeSignedBlock() will need to be async for merge; introduce altair types to VC
2021-06-29 15:09:29 +00:00
Jacek Sieka cc8f7c26a0
flush stdout logs (#2669)
else pipes end up not getting log output until much later
2021-06-29 16:53:36 +03:00
tersec 60f601e4e6
cache base_reward_per_increment for 5x+ Altair block_sim perf increase (#2681) 2021-06-28 06:54:06 +00:00
tersec ec54173667
optimize phase 0 -> Altair transition state processing (#2679) 2021-06-25 12:51:25 +00:00
Tanguy Cizain aeef550d0a
Altair message-id topic (#2671) 2021-06-25 03:07:46 +03:00
tersec 445def6c8b
block_clearance, ncli, and ncli_db Altair state saving (#2672)
* block_clearance, ncli, and ncli_db Altair state saving

* avoid invalidating SSZ hash caches with every assignment
2021-06-24 18:34:08 +00:00
tersec 41e0a7abc0
introduce database support for Altair (#2667)
* introduce immutable Altair BeaconState

* add database support for Altair blocks and states

* add tests for Altair get/put/contains/delete state

* enable blockchain_dag Altair state database storing

* properly return error on getting missing altair block
2021-06-24 07:11:47 +00:00
tersec ae1abf24af
add Altair support to block quarantine/clearance and block_sim (#2662)
* add Altair support to the block quarantine

* switch some spec/datatypes imports to spec/datatypes/base

* add Altair support to block_clearance

* allow runtime configuration of Altair transition slot

* enable Altair in block_sim, including in CI
2021-06-23 14:43:18 +00:00
Mamy André-Ratsimbazafy 8dc4db51c0
Merge branch 'stable' into unstable 2021-06-23 10:43:23 +02:00
Mamy André-Ratsimbazafy 4ef784fa68
v1.4.0 2021-06-21 14:02:12 +02:00
Eugene Kabanov 533cd25e1c
Add CONFIG_NAME parameter to REST API /eth/v1/config/spec response. (#2652)
* Add CONFIG_NAME parameter to /eth/v1/config/spec response.

* Add missed comma.

* Use `const_preset` instead of CONFIG_NAME.
2021-06-21 11:31:10 +02:00
tersec b1d5609171
remove false OnBlockAdded dependency on phase0 HashedBeaconState (#2661)
* remove false OnBlockAdded dependency on phase.HashedBeaconState

* introduce altair data types into block_clearance; update some alpha.6 spec refs to alpha.7; add get_active_validator_indices_len ForkedHashedBeaconState wrapper

* switch many modules from using datatypes (with phase0 states/blocks) to datatypes/base (fork-independent); update spec refs from alpha.6 to alpha.7 and remove rm'd G2_POINT_AT_INFINITY

* switch more modules from using datatypes (with phase0 states/blocks) to datatypes/base (fork-independent); update spec refs from alpha.6 to alpha.7

* remove unnecessary phase0-only wrapper of get_attesting_indices(); allow signatures_batch to process either fork; remove O(n^2) nested loop in process_inactivity_updates(); add altair support to getAttestationsforTestBlock()

* add Altair versions of asSigVerified(), asTrusted(), and makeBeaconBlock()

* fix spec URL to be Altair for Altair makeBeaconBlock()
2021-06-21 08:35:24 +00:00
Eugene Kabanov f8eb906b51
Add CONFIG_NAME parameter to REST API /eth/v1/config/spec response. (#2652)
* Add CONFIG_NAME parameter to /eth/v1/config/spec response.

* Add missed comma.

* Use `const_preset` instead of CONFIG_NAME.
2021-06-18 14:43:54 +02:00
tersec 9616220280
implement Altair attestation pool cache init (#2659)
* implement Altair attestation pool cache init

* remove code duplication around previous/current epoch updates
2021-06-17 17:13:14 +00:00
Jacek Sieka a3322acee9
Revert "fix assertion in beacon block creation rollback/restore (#2655)"
This reverts commit 9d48097de0.
2021-06-17 16:52:11 +02:00
Mamy Ratsimbazafy e14873757e
Fix #2595 - recompute wallSlot across async calls (#2650) 2021-06-17 16:43:58 +02:00
tersec 9ad8310157
doppelganger detection walltime refactor (#2656)
* strawman doppelganger detection walltime refactor

* move DoppelgangerProtection to Eth2Processor

* increase comment precision

* document difference between broadcastStartEpoch and nodeLaunchSlot, and allow for one-slot overlap to avoid false positives on intra-slot restarts
2021-06-17 16:43:58 +02:00
tersec 9d48097de0
fix assertion in beacon block creation rollback/restore (#2655) 2021-06-17 16:43:57 +02:00
Mamy Ratsimbazafy 2adf54e44a
Fix #2595 - recompute wallSlot across async calls (#2650) 2021-06-17 16:38:25 +02:00
tersec 87ed9e62a1
doppelganger detection walltime refactor (#2656)
* strawman doppelganger detection walltime refactor

* move DoppelgangerProtection to Eth2Processor

* increase comment precision

* document difference between broadcastStartEpoch and nodeLaunchSlot, and allow for one-slot overlap to avoid false positives on intra-slot restarts
2021-06-17 11:51:04 +00:00
tersec 53d05060c9
fix assertion in beacon block creation rollback/restore (#2655) 2021-06-17 09:22:39 +02:00
Jacek Sieka a5711ecf18
remove a few obsolete raises pops (#2654) 2021-06-16 14:45:05 +02:00
tersec 1c3314f08b
update to Altair as of v1.1.0-alpha.7 (#2649)
* update to Altair as of v1.1.0-alpha.7

* introduce Altair types into attestation pool

* avoid allocating/copying pubkeys excessively in get_next_sync_committee()
2021-06-14 17:42:46 +00:00
tersec 146fa48454
use ForkedHashedBeaconState in StateData (#2634)
* use ForkedHashedBeaconState in StateData

* fix FAR_FUTURE_EPOCH -> slot overflow; almost always use assign()

* avoid stack allocation in maybeUpgradeStateToAltair()

* create and use dispatch functions for check_attester_slashing(), check_proposer_slashing(), and check_voluntary_exit()

* use getStateRoot() instead of various state.data.hbsPhase0.root

* remove withStateVars.hashedState(), which doesn't work as a design anymore

* introduce spec/datatypes/altair into beacon_chain_db

* fix inefficient codegen for getStateField(largeStateField)

* state_transition_slots() doesn't either need/use blocks or runtime presets

* combine process_slots(HBS)/state_transition_slots(HBS) which differ only in last-slot htr optimization

* getStateField(StateData, ...) was replaced by getStateField(ForkedHashedBeaconState, ...)

* fix rollback

* switch some state_transition(), process_slots, makeTestBlocks(), etc to use ForkedHashedBeaconState

* remove state_transition(phase0.HashedBeaconState)

* remove process_slots(phase0.HashedBeaconState)

* remove state_transition_block(phase0.HashedBeaconState)

* remove unused callWithBS(); separate case expression from if statement

* switch back from nested-ref-object construction to (ref Foo)(Bar())
2021-06-11 20:51:46 +03:00
cheatfate b09a3a6b4a Fix `aggregation_bits` json serialization. 2021-06-11 01:08:17 +03:00
Jacek Sieka 9193be9b7b
fix epoch logging (fixes #2283) (#2642)
Also put epoch first to disambiguate vs slot
2021-06-11 01:07:16 +03:00
Zahary Karadjov 651a806915
Fix another logging property type inconsistency 2021-06-10 12:17:17 +03:00
Zahary Karadjov 45430f2904
Fix an accidentally commented out line 2021-06-10 11:42:40 +03:00
Zahary Karadjov ff8212e58e
Fix #2287 2021-06-10 11:39:53 +03:00
Jacek Sieka dc9ced03f1 space is printable
Printing space as a string in graffiti makes a few more graffitis
visible
2021-06-10 10:40:00 +03:00
Jacek Sieka d859bc12f0
write uncompressed validator keys to database (#2639)
* write uncompressed validator keys to database

Loading 150k+ validator keys on startup in compressed format takes a lot
of time - better store them in uncompressed format which makes behaviour
just after startup faster / more predictable.

* refactor cached validator key access
* fix isomorphic cast to work with non-var instances
* remove cooked pubkey cache - directly use database cache in chaindag
as well (one less cache to keep in sync)
* bump blscurve, introduce loadValid for known-to-be-valid keys
2021-06-10 10:37:02 +03:00
Jacek Sieka 5974fb0e7d speed up initial migration
an isolated transaction when loading the database helps keep migration
time down on first start after upgrade
2021-06-09 20:04:20 +03:00