Commit Graph

335 Commits

Author SHA1 Message Date
tersec 6eb4f1f39d
initial attestation aggregation (#769)
* initial attestation aggregation

* fix usage of committee index, vs index in committee; uniformly set trailing/following distance; document how the only-broadcast-if mechanism works better and what aggregation already happens, not otherwise sufficiently clear; use correct BlockSlot across epoch boundaries

* address inconsistent notion of which slot in past to target for aggregate broadcast; follow 0.11.x aggregate broadcast p2p interface topic

* Fix get_slot_signature(...) call after get_domain(...) change required genesis_validators_root

* mark all spec references which aren't dealt with in other PRs as v0.11.1

* update two more spec refs to v0.11.1
2020-04-01 09:59:55 +00:00
tersec cd388bc9bb
extended validation (#812)
* initial extended validation setup

* flesh out all TODO items for attestation and beaconblock verification

* fix finalization and add chronicles debugging messages

* directly use blockPool.headState rather than pointlessly updating it and document this constraint

* fix logic relating to first-attestation checking; support validating blocks across multiple forks
2020-03-31 18:39:02 +00:00
tersec f5f939bd31
0.11.1 beacon chain spec update (#836)
* initial 0.11.1 spec commit; no test regressions and finalizes in eth2_network_simulation

* with BLS 0.10/0.11 available, stop skipping attester slashing, proposer slashing, and voluntary exist operations fixture tests

* switch param orders to group state.{fork, genesis_validators_root}; bump spec/datatypes spec version for network purposes

* mark attestation construction and broadcast and some minimal/mainnet constants as 0.11.1-compatible; remove phase 1 sharding constants from minimal which don't exist in that preset
2020-03-30 11:31:44 +00:00
kdeme b80a5b90df Adjustments to store bootnode enrs in discovery object + bump nim-eth 2020-03-24 22:14:40 +02:00
Zahary Karadjov 64fbad1efa Some build fixes required for pushing testnet0 2020-03-23 16:13:11 +02:00
Zahary Karadjov 46a3009c63 Restore the address file feature (and use it for synchronization in local sim) 2020-03-23 14:59:33 +02:00
Zahary Karadjov 4623aa81ec Clean up some obsolete code 2020-03-23 14:59:33 +02:00
Zahary Karadjov 0c018cb68a Mechanically remove all mentions of the daemon from the code 2020-03-23 14:59:33 +02:00
Zahary Karadjov 286d6e14da Attempt to fix the Travis ARM build 2020-03-22 19:00:09 +02:00
Zahary Karadjov 1d78a43724 Don't return Option[BlockRef] because BlockRef is a ref type 2020-03-22 19:00:09 +02:00
Zahary Karadjov d2b7ee27de The RPC service listens on localhost by default 2020-03-22 19:00:09 +02:00
Zahary Karadjov 7d2381240f Bump nim-eth and nim-json-rpc 2020-03-22 19:00:09 +02:00
Zahary Karadjov 5a2a52869e Add a RPC proc similar to /spec/eth2_config 2020-03-22 19:00:09 +02:00
Zahary Karadjov adcec61081 Initial implementation of a JSON-RPC service 2020-03-22 19:00:09 +02:00
Jacek Sieka 689bcf71c4 clean up block creation
* consistently use state at new block slot to produce block
* factor out signature funcs
* fix missing block application test
2020-03-22 11:15:07 +00:00
Dustin Brody cae1c0fd1a address beacon_node TODO regarding missing proposer index; mark initiate_validator_exit(...), is_eligible_for_activation_queue(...), Attestation, DepositMessage, BeaconBlockBody, PendingAttestation, Eth1Data, and integer_squareroot(...) as 0.11.0 2020-03-17 12:06:56 +02:00
Dustin Brody 11300fc290 fix finalization 2020-03-17 12:06:56 +02:00
Jacek Sieka 9a3db7a81f
[WIP] Fake bls at runtime (#735)
* Initial implementation of runtime bls skipping.

Add libnfuzz skipBLSValidation handling, check that it propagates.

* Rename skipBLSValidation -> skipBlsValidation, start skipStateRootValidation
* Replace skipValidation flags with more granular flags.

Also added skipBlockParentRootValidation flag
Mainly replaced with skipBlsValidation but also StateRoot or
BlockParentRootValidation flags where appropriate.

* Adjust interop test to pass when skipping merkle validation.
* Stop skipping validation for mainchain_monitor.
* Remove comment.
* Also skipMerkleValidation for test_beacon_chain_db.
2020-03-05 13:52:10 +01:00
Joao Gabriel Carvalho 91f87b55b4
SignedBeaconBlock in state_transition (#773)
* using SignedBeaconBlock in state_transition
2020-02-29 16:15:44 +01:00
Dustin Brody 425b0e13da remove potential error condition in getAttestationTopic; subscribe to all topics in parallel; add test cases for larger-than-ATTESTATION_SUBNET_COUNT committee indexes 2020-02-21 19:44:03 +00:00
Dustin Brody 6c87af9aab split attestation beacon topic per honest validator spec 2020-02-21 19:44:03 +00:00
Jacek Sieka c25f8d4adf
sqlite kvstore backend (#749) 2020-02-20 11:49:34 +01:00
Zahary Karadjov 55886ff0ac
More proper fix replacing hotfix 052a6ac3bc
nim-eth was bumped to take advantage of a fixed `==` for public eth keys
2020-02-19 11:59:38 +02:00
Zahary Karadjov 3b9e9fefa8
Make the persistent network keys compatible with the Go daemon
The daemon requires its key file to be encoded as ProtoBuf value
with a proper `key scheme` discriminator.
2020-02-19 10:58:10 +02:00
Zahary Karadjov 57811dce89 Don't log unwanted lines in beacon_node --help 2020-02-18 12:53:49 +02:00
Zahary Karadjov b2d5aba749 Advertise the TCP port of the bootstrap node properly 2020-02-18 12:53:49 +02:00
Zahary Karadjov 2c814db750 Temporary switch to supporting only ENR bootstrap addresses
Turns out the DiscV5 code relies heavily on the presence of ENR
records at the moment, so we cannot drive it with ENodes. @kdeme
is working on refactoring that will relax these requirements.
2020-02-18 12:53:49 +02:00
Zahary Karadjov 7f395e554a WIP Working local network sim with discovery
* The bootstrap_nodes.txt file in the node's data dir is now optional
* Log more data on start-up
* Use the latest ENR APIs
* Fix simulation build errors
2020-02-18 12:53:49 +02:00
Zahary Karadjov d2f59ebab9 Switch to ENR records in the bootstrap_nodes.txt file 2020-02-18 12:53:49 +02:00
Zahary Karadjov ee7c2c9dff Unify the bootstrap nodes handling code
We no longer discriminate between ENR, MultiAddress or ENode
bootstrap records (all of them are remapped to ENodes).

The discovery loop will stochastically try to reconnect to
accidentally disconnected nodes.
2020-02-18 12:53:49 +02:00
Dustin Brody bde1eefdcf improve option naming 2020-02-17 22:00:41 +00:00
Dustin Brody e13846bec6 allow beacon_node to verify finalization when appropriate, and enable by default for make eth2_network_simulation 2020-02-17 22:00:41 +00:00
Jacek Sieka 521b0ed6ba
Clean 20200205 (#729)
* beacon node code cleanup
* rudimentary error checking on mainnet monitor
* start client even when sending deposit
* work around missing block number exception
* connect to testnet with web3 url
* pretty-print digests in json
2020-02-07 08:13:38 +01:00
Dustin Brody 1ffc2df23d add a couple new deposit tests; fix the false-positive BLS verifications while keeping all but two tests working, despite mismatched 0.9/0.10 BLS standards; better-factor the skipping of BLS validation and Merkle tree validation 2020-02-04 18:39:38 +00:00
Jacek Sieka 7dc2434e0d
clean up attestation logging 2020-01-23 18:48:26 +01:00
Zahary Karadjov 0743b3507c Fix the build after the rebase 2020-01-21 01:26:57 +02:00
Zahary Karadjov 63b98f0334 Less confusing debug logging 2020-01-21 01:26:57 +02:00
Zahary Karadjov c65575d105 Improved logging related to bootstrap nodes 2020-01-21 01:26:57 +02:00
Zahary Karadjov 52878405b7 Integrate Discovery V5 and support ENR bootstrap records 2020-01-21 01:26:57 +02:00
Jacek Sieka 2a67ac3c05 trie -> kv store
* simplify data storage to key-value, tries are not relevant for NBC
* locked-down version of lmdb dependency
* easier to build / maintain on various platforms
2020-01-20 13:39:37 +00:00
Jacek Sieka 3439763a9e fix beacon node init
* update forkversion correctly
* ensure supplied state snapshot starts at genesis - we don't support
anything else
* more database sanity checks
2020-01-17 21:26:28 +00:00
Jacek Sieka 148527c716 slot fixes
* fix slot time navigation, add tests
* skip block proposal if head is more recent already - shouldn't happen
* use correct head when attesting to previous blocks
* log slot start/end processing
2019-12-23 17:56:56 +00:00
Dustin Brody 46de726e3d mark all compatible 0.9.3 functions and data structures as 0.9.4 2019-12-20 17:24:42 +00:00
Jacek Sieka 746659bdc6
bump stew, fix warnings (#655) 2019-12-20 14:25:33 +01:00
Jacek Sieka b994da78a7 Move some attestation/block logic out of beacon node
* state data cache in block pool
* keep head state around
* more attestation logic in attestation pool
* first fork choice tests (!)
* fix fork choice (it's still likely broken / out of date)
2019-12-19 16:17:45 +00:00
Dustin Brody 350a6c071a mark a couple compatible dozen 0.9.2 functions/data structures as 0.9.3 2019-12-18 16:13:39 +00:00
Dustin Brody c824416f56 initial 0.9.3 spec update 2019-12-17 00:06:03 +02:00
Jacek Sieka 4e85b50cb6 update head while syncing 2019-12-16 12:18:08 +02:00
tersec cb1bc6cbf1 move attesting up from halfway to one third of the way through slots (#643)
* per honest validator and naïve/simple aggregator attestation specs, move attesting up from halfway to one third of the way through slots

* Update beacon_chain/beacon_node.nim

Co-Authored-By: Jacek Sieka <jacek@status.im>
2019-12-11 14:02:07 +01:00
Jacek Sieka a3145f0f5c cleanups
* move stuff out of beaconnode
* fix use-after-free in query
2019-12-10 14:47:52 +00:00
Jacek Sieka eea523a89e logging fixes
* don't log out-of-sync when there's no work to do
* small logging fixes
2019-12-10 10:18:30 +00:00
Dustin Brody 5729ceb58e rename notice field 2019-12-06 12:12:59 +00:00
Dustin Brody 86f267bc5d switch debug to notice and add some contextual information 2019-12-06 12:12:59 +00:00
Dustin Brody c762c0232d render get_beacon_proposer_index(...) return value an option and remove overly aggressive assertion formerly causing crashes when missing validators (#617)
* render get_beacon_proposer_index(...) return value an option and remove overly aggressive assertion formerly causing crashes when missing validators

* follow addBlock(...) after refactoring

* [skip ci] fix typo
2019-12-04 11:49:59 +01:00
Jacek Sieka afc0686b36 save ssz dummps during processing (#471) 2019-12-03 11:32:27 +00:00
Zahary Karadjov ee2448b221 Simple validator onboarding
On your very first connection to each testnet, you'll be asked to
become a validator. Please consult our private repo for a Goerli
Eth1 private key that you can use for deposits.

Other changes:

* Added a simple wrapper ./connect-to-testnet script calling the
  nims file in the correct environment. No extension was used to
  make the command the same on Unix and Windows.

* Bumped a number of modules with fixes from this week

* `make testnet0` and `make testnet1` will no longer delete your
  existing database. This is considered a more appropriate behavior
  for testing forward sync.
2019-12-03 02:23:17 +02:00
Ștefan Talpalaru 7682a010a9 remove getCurrentExceptionMsg() 2019-12-02 22:06:42 +02:00
Jacek Sieka 90212eed2a avoid producing blocks/attestations when out of sync
this is a temporary measure until we figure something better out - as it
stands, we'll advance with empty slots and crash because all validators
are out.
2019-12-02 15:26:42 +00:00
Jacek Sieka a82869c03d add validators even if they're not in startup state 2019-11-25 18:31:22 +02:00
Jacek Sieka fd4de5de0f
start untangling beaconnode (#589)
* Move BeaconNode type to its own file (fewer imports)
* disentangle sync protocol/request manager
* fix some old nimisms
* de-fear some logs
* simplify eth1 data production
* add stack tracing to release builds
* drop release compile flag for testnet
2019-11-25 15:36:25 +01:00
Jacek Sieka 7c6d87051a
simplify validator api (#573)
* avoid passing whole state to validator api (easier when it goes
out-of-process)
* clean up some cruft
2019-11-21 10:57:59 +01:00
Dustin Brody c0f009908c
re-enable test_interop based on zcli with 0.9.1 specs and update initialize_beacon_state_from_eth1(...) to 0.9.1 (#569)
* re-enable test_interop based on zcli with 0.9.1 specs and update initialize_beacon_state_from_eth1(...) to 0.9.1

* switch many procs to funcs

* fix import os.nim instead; ospaths is deprecated [Deprecated] warnings
2019-11-21 09:15:10 +00:00
Dustin Brody d8590bbffc fix several warnings and hints 2019-11-18 17:35:58 +00:00
Jacek Sieka 14712bbbdc
random warning fixes 2019-11-15 12:05:18 +01:00
Dustin Brody 3f958d3529
Merge branch 'master' into bat 2019-11-14 19:37:47 +00:00
Dustin Brody 96dcb1cf95
Remove shard-relating code 2019-11-14 19:48:12 +01:00
Dustin Brody c08698bffe
scaffolding to remove last vestiges of get_committee_count(...), SHARD_COUNT, and rest of that 0.8.x infrastructure; this demonstrates equivalence between two, before discarding old code 2019-11-14 18:37:51 +01:00
Dustin Brody b4850fbb06 remove XDeclaredButNotUsed spam 2019-11-14 15:58:31 +00:00
Dustin Brody 5a54c823d8 update to 0.9.1: don't try to run removed tests; re-add 1.5 tests (one only in minimal); remove bls_verify_multiple(...) and AttestationDataAndCustodyBit; and update process_attester_slashing(...), get_indexed_attestation(...), and is_valid_indexed_attestation(...) 2019-11-13 13:49:38 +00:00
Zahary Karadjov f40675509a Make all testnet env file values optional [skip CI]
Also removes some usages of SHARD_COUNT
2019-11-12 23:43:38 +00:00
Zahary Karadjov 5a93e50b5e Sync-related fixes:
* off-by-one error in the returned range of blocks
* larger request time-outs to deal with non-responsive servers
* fix an unhandled exception when we fail to deliver a response chunk
2019-11-12 23:43:38 +00:00
cheatfate 6e8a518077 Fix getPersistentNetIdentity() do not use daemon anymore.
Fix change PeerInfo to MultiAddress.
2019-11-12 23:43:38 +00:00
Dustin Brody d5ce142511 fix network sim finalization; remove get_attestation_data_slot(...); remove 2 more get_crosslink_committee(...) calls 2019-11-12 14:49:26 +00:00
Zahary Karadjov 07f0bd9378
Log the executable version at start-up [skip ci] 2019-11-12 00:05:59 +00:00
Zahary Karadjov ac5aabd569 Use the latest confutils 2019-11-11 23:29:36 +00:00
Zahary Karadjov 22591deced Safer testnet restarts; Working CLI queries for inspecting the genesis states
When the connect_to_testnet script is invoked it will first verify that
the genesis file of the testnet hasn't changed. If it has changed, any
previously created database associated with the testnet will be erased.

To facilitate this, the genesis file of each network is written to the
data folder of the beacon node. The beacon node will refuse to start if
it detects a discrepancy between the data folder and any state snapshot
specified on the command-line.

Since the testnet sharing spec requires us to use SSZ snapshots, the Json
support is now phased out. To help with the transition and to preserve the
functionality of the multinet scripts, the beacon node now supports a CLI
query command that can extract any data from the genesis state. This is
based on new developments in the SSZ navigators.
2019-11-11 23:29:36 +00:00
Dustin Brody 949d735155 fix remaining test 0.9.0 test fixtures and switch back to non-transitiontest setup; remove get_committee_count(...); keep SHARD_COUNT consistent with MAX_COMMITTEES_PER_SLOT 2019-11-11 10:25:57 +00:00
Ștefan Talpalaru c1a63035b9 beacon_node: "--version" option [skip ci]
fixes #466
2019-11-09 17:43:42 +00:00
Ștefan Talpalaru 6a1d287652 beacon_node: print name, version and Git revision [skip ci] (#527) 2019-11-08 10:33:16 +00:00
Yuriy Glukhov 568e669b1c Fixed deposit/contract 2019-11-07 17:23:06 +00:00
Yuriy Glukhov 9b39c792d0 Revert f79f9deeb3 a0fbf5464a 94a9c51d8a (#523) 2019-11-06 16:56:54 +02:00
cheatfate 94a9c51d8a
Initial commit. 2019-11-06 16:41:23 +02:00
Zahary Karadjov 14374504cf
Finalization experiements
* Switched to FloodSub
* Testnet0 switched to the minimal preset. All validators deployed on the master node.
2019-11-05 03:13:16 +02:00
Dustin Brody 63e621c27d
initial 0.9.0 spec sync (#509)
* rename compute_epoch_of_slot(...) to compute_epoch_at_slot(...)

* remove some unnecessary imports; remove some crosslink-related code and tests; complete renaming of compute_epoch_of_slot(...) to compute_epoch_at_slot(...)

* rm more transfer-related code and tests; rm more unnecessary strutils imports

* rm remaining unused imports

* remove useless get_empty_per_epoch_cache(...)/compute_start_slot_of_epoch(...) calls

* rename compute_start_slot_of_epoch(...) to compute_start_slot_at_epoch(...)

* rename ACTIVATION_EXIT_DELAY to MAX_SEED_LOOKAHEAD

* update domain types to 0.9.0

* mark AttesterSlashing, IndexedAttestation, AttestationDataAndCustodyBit, DepositData, BeaconBlockHeader, Fork, integer_squareroot(...), and process_voluntary_exit(...) as 0.9.0

* mark increase_balance(...), decrease_balance(...), get_block_root(...), CheckPoint, Deposit, PendingAttestation, HistoricalBatch, is_active_validator(...), and is_slashable_attestation_data(...) as 0.9.0

* mark compute_activation_exit_epoch(...), bls_verify(...), Validator, get_active_validator_indices(...), get_current_epoch(...), get_total_active_balance(...), and get_previous_epoch(...) as 0.9.0

* mark get_block_root_at_slot(...), ProposerSlashing, get_domain(...), VoluntaryExit, mainnet preset Gwei values, minimal preset max operations, process_block_header(...), and is_slashable_validator(...) as 0.9.0

* mark makeWithdrawalCredentials(...), get_validator_churn_limit(...), get_total_balance(...), is_valid_indexed_attestation(...), bls_aggregate_pubkeys(...), initial genesis value/constants, Attestation, get_randao_mix(...), mainnet preset max operations per block constants, minimal preset Gwei values and time parameters, process_eth1_data(...), get_shuffled_seq(...), compute_committee(...), and process_slots(...) as 0.9.0; partially update get_indexed_attestation(...) to 0.9.0 by removing crosslink refs and associated tests

* mark initiate_validator_exit(...), process_registry_updates(...), BeaconBlock, Eth1Data, compute_domain(...), process_randao(...), process_attester_slashing(...), get_base_reward(...), and process_slot(...) as 0.9.0
2019-10-30 19:41:19 +00:00
Zahary Karadjov 72c42838cb
Fix a typo in start.sh; Display the full slot number in the statusbar [skip ci] 2019-10-29 21:48:32 +02:00
Zahary Karadjov 6664f1689f
Use SSZ genesis files
Multi-client testing requires more portable formats, and SSZ is
much better specified than our flavour of Json.

Tools like ncli and zcli can be now used to inspect the contents
of the SSZ files.
2019-10-29 19:16:32 +02:00
Zahary Karadjov 1ef9f458ac
Remove hard-coded variables from the manage_testnet_hosts script 2019-10-29 19:16:32 +02:00
Zahary Karadjov 831b562435
Make the use of dynamic Chronicles outputs more optional 2019-10-29 19:16:32 +02:00
Zahary Karadjov f1ea0cec32
Add support for testnets with mixed quickstart/random deposits 2019-10-29 19:16:32 +02:00
Zahary Karadjov 136a11f681
Fix the build when the Chronicles dynamic output is not used
close #504
2019-10-29 03:43:52 +02:00
Zahary Karadjov 734ab64da8
Unix-style command-line params for the 'deposit_contract' binary 2019-10-29 02:20:47 +02:00
Zahary Karadjov ba0037738b
Removed all code related to the old network metadata files
Also switched to a more "standard" naming convention for the
command-line parameters.
2019-10-29 01:04:52 +02:00
Yuriy Glukhov 9d3889cbab
WIP Goerli testnet 2019-10-28 15:51:17 +02:00
Zahary Karadjov 43b70c0330
Restore the ability to run tests/simulation/start.sh manually 2019-10-28 15:28:45 +02:00
Zahary Karadjov dfe3a6f0fb
Switch to Nim v1.0.2 2019-10-28 15:00:25 +02:00
Zahary Karadjov a6f65cd37e Add lower-level metrics for the number of gossipsub messages sent/received 2019-10-26 16:37:36 +03:00
Dustin Brody a9fcaf16a6
stop double-counting attestation-topic attestations (#501) 2019-10-25 14:00:55 +00:00
Jacek Sieka ee5d1c1467
minimize imports, specially for spec - cuts 2/3 of ncli build time (#500)
* minimize imports, specially for spec - cuts 2/3 of ncli build time
* ptr_arith->ptrops
2019-10-25 12:59:56 +02:00
Dustin Brody 700e7d921b
update to beacon chain phase 0 spec version 0.8.4; no substantive changes (#494) 2019-10-24 13:36:36 +00:00
Dustin Brody c9d2815480 implement metrics to count send/received blocks/attestations, to detect network loss 2019-10-24 11:48:56 +03:00