Commit Graph

285 Commits

Author SHA1 Message Date
Jacek Sieka eea523a89e logging fixes
* don't log out-of-sync when there's no work to do
* small logging fixes
2019-12-10 10:18:30 +00:00
Dustin Brody 5729ceb58e rename notice field 2019-12-06 12:12:59 +00:00
Dustin Brody 86f267bc5d switch debug to notice and add some contextual information 2019-12-06 12:12:59 +00:00
Dustin Brody c762c0232d render get_beacon_proposer_index(...) return value an option and remove overly aggressive assertion formerly causing crashes when missing validators (#617)
* render get_beacon_proposer_index(...) return value an option and remove overly aggressive assertion formerly causing crashes when missing validators

* follow addBlock(...) after refactoring

* [skip ci] fix typo
2019-12-04 11:49:59 +01:00
Jacek Sieka afc0686b36 save ssz dummps during processing (#471) 2019-12-03 11:32:27 +00:00
Zahary Karadjov ee2448b221 Simple validator onboarding
On your very first connection to each testnet, you'll be asked to
become a validator. Please consult our private repo for a Goerli
Eth1 private key that you can use for deposits.

Other changes:

* Added a simple wrapper ./connect-to-testnet script calling the
  nims file in the correct environment. No extension was used to
  make the command the same on Unix and Windows.

* Bumped a number of modules with fixes from this week

* `make testnet0` and `make testnet1` will no longer delete your
  existing database. This is considered a more appropriate behavior
  for testing forward sync.
2019-12-03 02:23:17 +02:00
Ștefan Talpalaru 7682a010a9 remove getCurrentExceptionMsg() 2019-12-02 22:06:42 +02:00
Jacek Sieka 90212eed2a avoid producing blocks/attestations when out of sync
this is a temporary measure until we figure something better out - as it
stands, we'll advance with empty slots and crash because all validators
are out.
2019-12-02 15:26:42 +00:00
Jacek Sieka a82869c03d add validators even if they're not in startup state 2019-11-25 18:31:22 +02:00
Jacek Sieka fd4de5de0f
start untangling beaconnode (#589)
* Move BeaconNode type to its own file (fewer imports)
* disentangle sync protocol/request manager
* fix some old nimisms
* de-fear some logs
* simplify eth1 data production
* add stack tracing to release builds
* drop release compile flag for testnet
2019-11-25 15:36:25 +01:00
Jacek Sieka 7c6d87051a
simplify validator api (#573)
* avoid passing whole state to validator api (easier when it goes
out-of-process)
* clean up some cruft
2019-11-21 10:57:59 +01:00
Dustin Brody c0f009908c
re-enable test_interop based on zcli with 0.9.1 specs and update initialize_beacon_state_from_eth1(...) to 0.9.1 (#569)
* re-enable test_interop based on zcli with 0.9.1 specs and update initialize_beacon_state_from_eth1(...) to 0.9.1

* switch many procs to funcs

* fix import os.nim instead; ospaths is deprecated [Deprecated] warnings
2019-11-21 09:15:10 +00:00
Dustin Brody d8590bbffc fix several warnings and hints 2019-11-18 17:35:58 +00:00
Jacek Sieka 14712bbbdc
random warning fixes 2019-11-15 12:05:18 +01:00
Dustin Brody 3f958d3529
Merge branch 'master' into bat 2019-11-14 19:37:47 +00:00
Dustin Brody 96dcb1cf95
Remove shard-relating code 2019-11-14 19:48:12 +01:00
Dustin Brody c08698bffe
scaffolding to remove last vestiges of get_committee_count(...), SHARD_COUNT, and rest of that 0.8.x infrastructure; this demonstrates equivalence between two, before discarding old code 2019-11-14 18:37:51 +01:00
Dustin Brody b4850fbb06 remove XDeclaredButNotUsed spam 2019-11-14 15:58:31 +00:00
Dustin Brody 5a54c823d8 update to 0.9.1: don't try to run removed tests; re-add 1.5 tests (one only in minimal); remove bls_verify_multiple(...) and AttestationDataAndCustodyBit; and update process_attester_slashing(...), get_indexed_attestation(...), and is_valid_indexed_attestation(...) 2019-11-13 13:49:38 +00:00
Zahary Karadjov f40675509a Make all testnet env file values optional [skip CI]
Also removes some usages of SHARD_COUNT
2019-11-12 23:43:38 +00:00
Zahary Karadjov 5a93e50b5e Sync-related fixes:
* off-by-one error in the returned range of blocks
* larger request time-outs to deal with non-responsive servers
* fix an unhandled exception when we fail to deliver a response chunk
2019-11-12 23:43:38 +00:00
cheatfate 6e8a518077 Fix getPersistentNetIdentity() do not use daemon anymore.
Fix change PeerInfo to MultiAddress.
2019-11-12 23:43:38 +00:00
Dustin Brody d5ce142511 fix network sim finalization; remove get_attestation_data_slot(...); remove 2 more get_crosslink_committee(...) calls 2019-11-12 14:49:26 +00:00
Zahary Karadjov 07f0bd9378
Log the executable version at start-up [skip ci] 2019-11-12 00:05:59 +00:00
Zahary Karadjov ac5aabd569 Use the latest confutils 2019-11-11 23:29:36 +00:00
Zahary Karadjov 22591deced Safer testnet restarts; Working CLI queries for inspecting the genesis states
When the connect_to_testnet script is invoked it will first verify that
the genesis file of the testnet hasn't changed. If it has changed, any
previously created database associated with the testnet will be erased.

To facilitate this, the genesis file of each network is written to the
data folder of the beacon node. The beacon node will refuse to start if
it detects a discrepancy between the data folder and any state snapshot
specified on the command-line.

Since the testnet sharing spec requires us to use SSZ snapshots, the Json
support is now phased out. To help with the transition and to preserve the
functionality of the multinet scripts, the beacon node now supports a CLI
query command that can extract any data from the genesis state. This is
based on new developments in the SSZ navigators.
2019-11-11 23:29:36 +00:00
Dustin Brody 949d735155 fix remaining test 0.9.0 test fixtures and switch back to non-transitiontest setup; remove get_committee_count(...); keep SHARD_COUNT consistent with MAX_COMMITTEES_PER_SLOT 2019-11-11 10:25:57 +00:00
Ștefan Talpalaru c1a63035b9 beacon_node: "--version" option [skip ci]
fixes #466
2019-11-09 17:43:42 +00:00
Ștefan Talpalaru 6a1d287652 beacon_node: print name, version and Git revision [skip ci] (#527) 2019-11-08 10:33:16 +00:00
Yuriy Glukhov 568e669b1c Fixed deposit/contract 2019-11-07 17:23:06 +00:00
Yuriy Glukhov 9b39c792d0 Revert f79f9deeb3 a0fbf5464a 94a9c51d8a (#523) 2019-11-06 16:56:54 +02:00
cheatfate 94a9c51d8a
Initial commit. 2019-11-06 16:41:23 +02:00
Zahary Karadjov 14374504cf
Finalization experiements
* Switched to FloodSub
* Testnet0 switched to the minimal preset. All validators deployed on the master node.
2019-11-05 03:13:16 +02:00
Dustin Brody 63e621c27d
initial 0.9.0 spec sync (#509)
* rename compute_epoch_of_slot(...) to compute_epoch_at_slot(...)

* remove some unnecessary imports; remove some crosslink-related code and tests; complete renaming of compute_epoch_of_slot(...) to compute_epoch_at_slot(...)

* rm more transfer-related code and tests; rm more unnecessary strutils imports

* rm remaining unused imports

* remove useless get_empty_per_epoch_cache(...)/compute_start_slot_of_epoch(...) calls

* rename compute_start_slot_of_epoch(...) to compute_start_slot_at_epoch(...)

* rename ACTIVATION_EXIT_DELAY to MAX_SEED_LOOKAHEAD

* update domain types to 0.9.0

* mark AttesterSlashing, IndexedAttestation, AttestationDataAndCustodyBit, DepositData, BeaconBlockHeader, Fork, integer_squareroot(...), and process_voluntary_exit(...) as 0.9.0

* mark increase_balance(...), decrease_balance(...), get_block_root(...), CheckPoint, Deposit, PendingAttestation, HistoricalBatch, is_active_validator(...), and is_slashable_attestation_data(...) as 0.9.0

* mark compute_activation_exit_epoch(...), bls_verify(...), Validator, get_active_validator_indices(...), get_current_epoch(...), get_total_active_balance(...), and get_previous_epoch(...) as 0.9.0

* mark get_block_root_at_slot(...), ProposerSlashing, get_domain(...), VoluntaryExit, mainnet preset Gwei values, minimal preset max operations, process_block_header(...), and is_slashable_validator(...) as 0.9.0

* mark makeWithdrawalCredentials(...), get_validator_churn_limit(...), get_total_balance(...), is_valid_indexed_attestation(...), bls_aggregate_pubkeys(...), initial genesis value/constants, Attestation, get_randao_mix(...), mainnet preset max operations per block constants, minimal preset Gwei values and time parameters, process_eth1_data(...), get_shuffled_seq(...), compute_committee(...), and process_slots(...) as 0.9.0; partially update get_indexed_attestation(...) to 0.9.0 by removing crosslink refs and associated tests

* mark initiate_validator_exit(...), process_registry_updates(...), BeaconBlock, Eth1Data, compute_domain(...), process_randao(...), process_attester_slashing(...), get_base_reward(...), and process_slot(...) as 0.9.0
2019-10-30 19:41:19 +00:00
Zahary Karadjov 72c42838cb
Fix a typo in start.sh; Display the full slot number in the statusbar [skip ci] 2019-10-29 21:48:32 +02:00
Zahary Karadjov 6664f1689f
Use SSZ genesis files
Multi-client testing requires more portable formats, and SSZ is
much better specified than our flavour of Json.

Tools like ncli and zcli can be now used to inspect the contents
of the SSZ files.
2019-10-29 19:16:32 +02:00
Zahary Karadjov 1ef9f458ac
Remove hard-coded variables from the manage_testnet_hosts script 2019-10-29 19:16:32 +02:00
Zahary Karadjov 831b562435
Make the use of dynamic Chronicles outputs more optional 2019-10-29 19:16:32 +02:00
Zahary Karadjov f1ea0cec32
Add support for testnets with mixed quickstart/random deposits 2019-10-29 19:16:32 +02:00
Zahary Karadjov 136a11f681
Fix the build when the Chronicles dynamic output is not used
close #504
2019-10-29 03:43:52 +02:00
Zahary Karadjov 734ab64da8
Unix-style command-line params for the 'deposit_contract' binary 2019-10-29 02:20:47 +02:00
Zahary Karadjov ba0037738b
Removed all code related to the old network metadata files
Also switched to a more "standard" naming convention for the
command-line parameters.
2019-10-29 01:04:52 +02:00
Yuriy Glukhov 9d3889cbab
WIP Goerli testnet 2019-10-28 15:51:17 +02:00
Zahary Karadjov 43b70c0330
Restore the ability to run tests/simulation/start.sh manually 2019-10-28 15:28:45 +02:00
Zahary Karadjov dfe3a6f0fb
Switch to Nim v1.0.2 2019-10-28 15:00:25 +02:00
Zahary Karadjov a6f65cd37e Add lower-level metrics for the number of gossipsub messages sent/received 2019-10-26 16:37:36 +03:00
Dustin Brody a9fcaf16a6
stop double-counting attestation-topic attestations (#501) 2019-10-25 14:00:55 +00:00
Jacek Sieka ee5d1c1467
minimize imports, specially for spec - cuts 2/3 of ncli build time (#500)
* minimize imports, specially for spec - cuts 2/3 of ncli build time
* ptr_arith->ptrops
2019-10-25 12:59:56 +02:00
Dustin Brody 700e7d921b
update to beacon chain phase 0 spec version 0.8.4; no substantive changes (#494) 2019-10-24 13:36:36 +00:00
Dustin Brody c9d2815480 implement metrics to count send/received blocks/attestations, to detect network loss 2019-10-24 11:48:56 +03:00
Jacek Sieka e8206a7489
fix datadir 2019-10-22 23:18:45 +02:00
Jacek Sieka bdb70ebe2c
log data dir at startup (fixes #229) 2019-10-22 22:53:58 +02:00
Dustin Brody 43f353e665 implement beacon_reorgs_total metric (#491) 2019-10-22 13:57:34 +02:00
Dustin Brody 3da4c02bb3 implement beacon_current_live_validators and beacon_previous_live_validators metrics 2019-10-21 20:58:08 +09:00
Jacek Sieka 360afe48a3 fix two races when producing block
there be more dragons around
2019-10-08 23:52:50 +09:00
Jacek Sieka b7f9d9e4be fix BlsValue inits 2019-10-08 23:52:50 +09:00
Zahary Karadjov c23b011c77 Cosmetic improvements to the status bar; Attempt to fix the CI 2019-10-07 23:56:34 +09:00
Zahary Karadjov f72a58595f Initial version of status bar 2019-10-07 23:56:34 +09:00
Zahary Karadjov d3f88929da Integrate nim-prompt 2019-10-07 23:56:34 +09:00
Dustin Brody 4c0b2a9a32
implement beacon_pending_deposits, beacon_processed_deposits_total, beacon_previous_validators, and beacon_current_validators metrics (#478)
* implement additional beacon_current_validators and beacon_previous_validators metrics

* tentatively implement beacon_pending_deposits additional metric

* implement beacon_processed_deposits_total additional metric
2019-10-06 04:31:50 +00:00
Dustin Brody 255980c9f6 fix issue #367 and remove too-small range type usage for ValidatorIndex 2019-10-03 16:41:25 +03:00
Dustin Brody 2122bb44c9
implement beacon_finalized_epoch, beacon_current_justified_epoch, and beacon_previous_justified_epoch (#476) 2019-10-03 10:36:31 +00:00
Eugene Kabanov 13bde5aee3 Add gossipsub messages monitoring tool. (#458)
* Add gossipsub messages monitoring tool.

* Add support of bootstrap nodes file.

* Move topic constants to spec/network.nim.

* Add ability to monitor custom topics.
2019-10-01 15:52:28 +02:00
Zahary Karadjov 2bbe2cb23c
Implement the outputBootstrapNodes option (taken from the interop branch) 2019-09-27 19:05:17 +03:00
Zahary Karadjov b56c96a4fd
Support downloading SSZ genesis files with curl 2019-09-27 18:59:28 +03:00
Zahary Karadjov 0eaa433e84
Automated scripts for resetting the Status testnets
For detailed instructions, please see

https://github.com/status-im/nimbus-private/blob/master/testnets-maintenance.md
2019-09-26 18:58:51 +03:00
Mamy Ratsimbazafy 0b68f3dc63
[Interop branch] Log improvements - part 1 (#439)
* use service/category/process for blockpool logs
Only track fork choice logs in block pool (vs beacon_node)
Reduce verbosity on usual event in block pool

* rework beacon node logs

* log for attestations in blockpool

* log - att pool improvement

* use logScope and topics cf review and discussion

* use 7 letters for beacon_node

[log] report peers at slot start + fix bracket prefix [Block pool] Attestation sent
2019-09-23 11:35:41 +02:00
Jacek Sieka 42a469603d
simplify startup, describe a few startup tricks 2019-09-11 20:29:57 -04:00
Jacek Sieka 65ff8f2886
remove networkId (unused) 2019-09-11 20:29:57 -04:00
Jacek Sieka ab2cce3951
add quickStart option that avoids network metadata 2019-09-11 20:29:57 -04:00
mratsim c619b9557a
Validator pool update to use BLS domain 2019-09-11 20:29:57 -04:00
Zahary Karadjov 2aee749ead
Save a genesis SSZ file 2019-09-11 20:29:57 -04:00
Zahary Karadjov f583135a39
The --stateSnapshot option now accepts ssz files as well 2019-09-10 10:13:49 -04:00
Jacek Sieka 86fc9a1f9e fix nil access to mainchainMonitor 2019-09-10 09:49:54 -04:00
Yuriy Glukhov d4c480ff5f
Eth1 validators (#393) 2019-09-09 18:59:02 +03:00
zah 93cdb43f1e Integrate nim-metrics and add some metrics from the ETH2 spec (#394) 2019-09-07 19:48:05 +02:00
Yuriy Glukhov e3bd4410d8
Revert "Eth1 validators" (#392)
This reverts commit bb8955bdbc.
2019-09-06 13:00:56 +03:00
Yuriy Glukhov bb8955bdbc Eth1 validators 2019-09-06 02:38:38 +02:00
Jacek Sieka 9cfbdb5e16 more interop fixes
* fix eth1 interop block hash
* update initialize_beacon_state_from_eth1 to 0.8.3
2019-09-02 22:14:18 +03:00
Jacek Sieka 7b73b40bab interop updates
* add interop launcher scripts
* stick validator_keygen into beacon_node
* fix lmd ghost slot number on missing block
* use mocked eth1data when producing blocks
* use bls public key method for withdrawal credentials
* fix deposit domain
* prefer lowercase for a bunch of toHex
* build simulation binary in data folder to avoid data types confusion
2019-09-02 22:14:18 +03:00
Dustin Brody 8665efbbd1 bump beacon chain versions to 0.8.3, which has no relevant changes in this regard (#358) 2019-08-26 11:09:47 +02:00
Zahary Karadjov 180e8ed02b
Fix a build issue after rebasing 2019-08-20 16:42:15 +03:00
Zahary Karadjov 74f630eb28
Better start-up logic when the beacon node is launched before the genesis time 2019-08-20 16:34:33 +03:00
Jacek Sieka 78f7a4a505 restore network sim finalization
* relax attestation validation when attestation is incoming but make it
more strict when adding to block
* share attestation validation logic between attestation pool and state
transition
* remove a bunch of redundant logging
* fix potential underflow in attestation delay checking
* fix committee used for attestation shard selection when attesting
* fix attestation data construction
2019-08-19 21:05:54 +03:00
Jacek Sieka 225465578c fix startup time casts, log fixes 2019-08-19 21:05:54 +03:00
Jacek Sieka e639cd95c7
humaneXNum -> shortLog 2019-08-15 18:01:55 +02:00
Zahary Karadjov 4daeebbf7e
Fix the build when json logging is enabled 2019-08-14 17:38:39 +02:00
Dustin Brody 3e587d4667
Justification/finalization amelioration using testnet1 shard count and epoch length (#331)
* working justification, 1 node, 64 validators

* closer to tn1 params: 256 validators

* more debugging output; switch to minimum test case

* working justification and finalization in local network simulation

* fix currently incorrect state transition/attestation test assumption
2019-08-14 08:56:32 +00:00
Zahary Karadjov 5ce50b3aca
Implement the latest networking spec
https://github.com/ethereum/eth2.0-specs/pull/1328
2019-08-07 05:22:28 +03:00
Zahary Karadjov 398ea55801
Implement the latest SSZ specification and integrate the official SSZ test suite 2019-08-05 03:18:48 +03:00
Yuriy Glukhov 24cbc0710f Make web3 way of getting genesis optional 2019-07-31 16:39:26 +03:00
Yuriy Glukhov 958e4cd3b4 Getting genesis through web3 2019-07-31 16:39:26 +03:00
Mamy Ratsimbazafy d1d5497233
v0.7.x-v0.8.1 state transition refactor part2 (#322)
* State sim compiles and run

* Works on libp2p testnet1

* Successfully update the state transition tests

* naive update of attestation pool (failing attstations can arrive in any order)

* Remove an official assert to allow attestation reordering
2019-07-15 23:10:40 +02:00
Ștefan Talpalaru 2768a3e89e
Ctrl+C handling and:
- added a "quitProc" for shutting down the p2pd process, after seeing it
  survive an unhandled exception (this required a global var for the
  DaemonApi instance)
- Makefile testnet0/1 targets now depend on the "p2pd" binary
- prefixed the testnet command line in the build script message with
  "../../env.sh" so it can find the superproject's "p2pd"
2019-07-11 04:36:07 +02:00
Dustin Brody dfa062db7d 0.8.0, continued (#315)
* rename get_genesis_beacon_state(...) to initialize_beacon_state_from_eth1(...); remove unused vestiges get_temporary_block_header(...) and get_empty_block(...); partly align initialize_beacon_state_from_eth1(...) with 0.8.0 version; implement CompactCommittee object type; update process_final_updates(...) to 0.8.0

* mark get_shard_delta(...) as 0.8.0

* mark get_total_active_balance(...) and epoch transition helper functions as 0.8.0

* move FORK_CHOICE_BALANCE_INCREMENT, not in spec anymore, to sole user in fork_choice
2019-07-10 14:23:02 +02:00
Jacek Sieka b9e62a1f6c
std_shims -> stew 2019-07-07 11:53:58 +02:00
Dustin Brody d7905351eb update get_crosslink_committee(...) to 0.8.0 and fix https://github.com/status-im/nim-beacon-chain/issues/307 2019-07-02 19:52:39 +03:00
Zahary Karadjov bf037eab5e
Tentative bugfix: Reset testnet was producing a genesis state without validators 2019-07-01 16:20:55 +03:00
Dustin Brody fb601bd5b8 rename BeaconState.validator_registry to BeaconState.validators 2019-07-01 14:44:57 +03:00
Dustin Brody e4321dc4ed Initial frozen phase 0/v0.8.0 spec (#305)
* rename slot_to_epoch(...) to compute_epoch_of_slot(...)

* rename aggregation_bitfield/custody_bitfield fields to aggregation_bits/custody_bits; rename convert_to_indexed(...) to get_indexed_attestation(...); mark BeaconBlockHeader as 0.8.0; update minimal preset MIN_ATTESTATION_INCLUSION_DELAY/time parameters in general to 0.8

* fix beacon node compilation; it used slightly different capitalizations of slot_to_epoch/slotToEpoch

* mark integer_squareroot(...), Deposit, VoluntaryExit, and Transfer as 0.8.0; rename get_epoch_start_slot(...) to compute_start_slot_of_epoch(...)

* add new Domain alias for uint64; rename bls_domain(...) to compute_domain(...), MAX_INDICES_PER_ATTESTATION to MAX_VALIDATORS_PER_COMMITTEE, and SignatureDomain to DomainType; update get_domain(...) to 0.8.0

* update/mark initiate_validator_exit(...) and Crosslink as 0.8.0; rename BeaconState.latest_block_roots -> BeaconState.block_roots; mark HistoricalBatch as 0.8.0
2019-07-01 09:53:42 +02:00