* Move BeaconNode type to its own file (fewer imports)
* disentangle sync protocol/request manager
* fix some old nimisms
* de-fear some logs
* simplify eth1 data production
* add stack tracing to release builds
* drop release compile flag for testnet
* re-enable test_interop based on zcli with 0.9.1 specs and update initialize_beacon_state_from_eth1(...) to 0.9.1
* switch many procs to funcs
* fix import os.nim instead; ospaths is deprecated [Deprecated] warnings
* off-by-one error in the returned range of blocks
* larger request time-outs to deal with non-responsive servers
* fix an unhandled exception when we fail to deliver a response chunk
The number of user nodes is now specified with `USER_NODES`.
To make the instructions more stable, the "numeric id" of the user
nodes will be starting from 0 (so you can always use `run_node.sh 0`
to start a user node).
If you specify a node index above the total number of nodes, you'll
launch a node without any validators attached (this is useful for
testing the sync for example).
When the connect_to_testnet script is invoked it will first verify that
the genesis file of the testnet hasn't changed. If it has changed, any
previously created database associated with the testnet will be erased.
To facilitate this, the genesis file of each network is written to the
data folder of the beacon node. The beacon node will refuse to start if
it detects a discrepancy between the data folder and any state snapshot
specified on the command-line.
Since the testnet sharing spec requires us to use SSZ snapshots, the Json
support is now phased out. To help with the transition and to preserve the
functionality of the multinet scripts, the beacon node now supports a CLI
query command that can extract any data from the genesis state. This is
based on new developments in the SSZ navigators.
* update get_seed(...) and get_beacon_proposer_index(...) to 0.9.0, implement compute_proposer_index(...), and render 3 more test fixtures working
* rm stray Crosslink reference which prevented static SSZ tests from building
* remove references to removed tests in attestations test fixture; add minimal-preset block sanity test, plus all but one of mainnet tests for block sanity to transition fixtures
* transition deposit operations fixture to 0.9.0
* mark slash_validator(...) as 0.9.0
* switch remaining non-ref objects to ref objects to maybe avoid crashes in CI
* remove unused helpers/debug_state imports
* rename compute_epoch_of_slot(...) to compute_epoch_at_slot(...)
* remove some unnecessary imports; remove some crosslink-related code and tests; complete renaming of compute_epoch_of_slot(...) to compute_epoch_at_slot(...)
* rm more transfer-related code and tests; rm more unnecessary strutils imports
* rm remaining unused imports
* remove useless get_empty_per_epoch_cache(...)/compute_start_slot_of_epoch(...) calls
* rename compute_start_slot_of_epoch(...) to compute_start_slot_at_epoch(...)
* rename ACTIVATION_EXIT_DELAY to MAX_SEED_LOOKAHEAD
* update domain types to 0.9.0
* mark AttesterSlashing, IndexedAttestation, AttestationDataAndCustodyBit, DepositData, BeaconBlockHeader, Fork, integer_squareroot(...), and process_voluntary_exit(...) as 0.9.0
* mark increase_balance(...), decrease_balance(...), get_block_root(...), CheckPoint, Deposit, PendingAttestation, HistoricalBatch, is_active_validator(...), and is_slashable_attestation_data(...) as 0.9.0
* mark compute_activation_exit_epoch(...), bls_verify(...), Validator, get_active_validator_indices(...), get_current_epoch(...), get_total_active_balance(...), and get_previous_epoch(...) as 0.9.0
* mark get_block_root_at_slot(...), ProposerSlashing, get_domain(...), VoluntaryExit, mainnet preset Gwei values, minimal preset max operations, process_block_header(...), and is_slashable_validator(...) as 0.9.0
* mark makeWithdrawalCredentials(...), get_validator_churn_limit(...), get_total_balance(...), is_valid_indexed_attestation(...), bls_aggregate_pubkeys(...), initial genesis value/constants, Attestation, get_randao_mix(...), mainnet preset max operations per block constants, minimal preset Gwei values and time parameters, process_eth1_data(...), get_shuffled_seq(...), compute_committee(...), and process_slots(...) as 0.9.0; partially update get_indexed_attestation(...) to 0.9.0 by removing crosslink refs and associated tests
* mark initiate_validator_exit(...), process_registry_updates(...), BeaconBlock, Eth1Data, compute_domain(...), process_randao(...), process_attester_slashing(...), get_base_reward(...), and process_slot(...) as 0.9.0
Multi-client testing requires more portable formats, and SSZ is
much better specified than our flavour of Json.
Tools like ncli and zcli can be now used to inspect the contents
of the SSZ files.
* use service/category/process for blockpool logs
Only track fork choice logs in block pool (vs beacon_node)
Reduce verbosity on usual event in block pool
* rework beacon node logs
* log for attestations in blockpool
* log - att pool improvement
* use logScope and topics cf review and discussion
* use 7 letters for beacon_node
[log] report peers at slot start + fix bracket prefix [Block pool] Attestation sent
* Prepare test suite for transfers
* split API process_transfer / processTransfers
* Add range checks on transfer
* Fix invalid transfer conditions
* don't test on windows 64-bit #435
Changes:
* Do not send separate network packets for response codes and msg
len prefixes
* Close streams according to the spec
* Implement more timeouts according to the spec
* Make hello requests during syncing to update our knowledge of
the head block of the other peer.
* Hello is no longer a handshake message
(all handshakes related code was deleted for clarity)
* Deal with the single-parameter inlining defined in the new spec
* add test suite for voluntary exit
* update API to process_voluntary_exit
* Add range check of validator_index for voluntary exits
* Revert to dual single + multiple voluntary exits API + enable in test suite
* no cache or mocking needed
* Add attestation unit test
* process_attestation doesn't throw exceptions
* Allow SSZ deserialization of both real and invalid signatures
* Add new process_attestation checks - pass all process_attestation tests
* Add sanity check for #361
* Fix SSZ testing after fromBytes/fromSSZBytes changes
* Fixed getBeaconBlocks() and getRecentBeaconBlocks() to use BlockPool, not db.
* Got the sync_protocol to compiling state; Removed all obsolete RPC calls
* fix network sim
* mark BeaconState, state list/vector lengths, misc values, get_base_reward(...), verifyStateRoot(...), and process_slot(...) as 0.8.3; update minimal/mainnet config initial values to 0.8.3 by removing GENESIS_FORK_VERSION
* Add sanity check for slot processing (also impacted by https://github.com/status-im/nim-beacon-chain/issues/373)
* use reportDiff also for all state tests vs EF
* initial sanity checks for blocks - workaround zero signature in block headers: https://github.com/status-im/nim-beacon-chain/issues/374
* Remove generic object variant compare commented code
* Add the one block state transition sanity checks
* generalize blocks test to multiple blocks
* simplify slots test runner
* Add official epoch transitions, sanity blocks, sanity slots to the test suite
* Fix index out-of-bounds in initiate_validator_exit - enable proposer slashings unittest
* Update BLS fixtures to 0.8.3
* Bump fixtures with shuffling with minimal preset
* Update shuffling tests
* parseTest generic over file format (Json or SSZ)
* Initial crosslink parsing commit for debugging Nim crash
* Workaround https://github.com/status-im/nim-beacon-chain/issues/369
* Crosslink test works for minimal - https://github.com/status-im/nim-beacon-chain/issues/369 is back on mainnet
* Use ref objects to workaround https://github.com/status-im/nim-beacon-chain/issues/369
* Generalize state transition epoch test to all epoch tests
* Fix slashing (potential uint64 overflow in previous spec)
* Add a state debugging macro to deeply inspect the wrong fields
* make reportDiff visible
* Improve the debug state macro for containers
* add interop launcher scripts
* stick validator_keygen into beacon_node
* fix lmd ghost slot number on missing block
* use mocked eth1data when producing blocks
* use bls public key method for withdrawal credentials
* fix deposit domain
* prefer lowercase for a bunch of toHex
* build simulation binary in data folder to avoid data types confusion
* Introduce new mocking proc to replace:
- makeFakeValidatorPrivKey
- hackPrivKey
- getNextBeaconProposerIndex
- addBlock
- makeBlock
* Add comments on datastructure unsynced with the spec
* Add merkle tree constructor and initial mocking for deposits (missing merkle proofs)
* [Mock] Implement sparse merkle tree and merkle proof builder
* [Mocking] Genesis deposits
* Add compact_committees_roots init + mock genesis state
* [Tests] Add first deposit test using the new mocking procedures
* [Tests -deposits] add at and over 32 ETH deposit tests
* [Tests - deposits] Add test for validator top-up
* [Tests -deposits] Mention the TODO to test for invalid conditions
* [Tests] Add stub to test "is_valid_genesis_state"
* [Merkle proofs] Implement round-trip checks
* Deactivate roundtrips test
* SSZ - use EF convention for hash_tree_root / hashTreeRoot
* [Tests - Attestation] Attestation mocking + initial test
* Add mocking + 3 new tests for valid attestations + mention future invalid attestation tests
* Add crosslinks test (1 failing to attestations in block being duplicated in state transition)
* Single attestation crosslink test - workaround https://github.com/status-im/nim-beacon-chain/issues/361
* Add test for failed crosslink penalty
* Rebase fixes + add refactored tests to test suite
* justif-finalization helpers first batch
* Add 234 finalization tests
* Fix justif test, Rule I 234 finalization does not happen with sufficient support.
(Also unittest check template does not fail properly in some cases)
* Add tests for all finalization rules
* Properly delete nim-byteutils following c91727e7e5 (diff-7c3613dba5171cb6027c67835dd3b9d4)
* use digest helper for deposit root
* relax attestation validation when attestation is incoming but make it
more strict when adding to block
* share attestation validation logic between attestation pool and state
transition
* remove a bunch of redundant logging
* fix potential underflow in attestation delay checking
* fix committee used for attestation shard selection when attesting
* fix attestation data construction
* working justification, 1 node, 64 validators
* closer to tn1 params: 256 validators
* more debugging output; switch to minimum test case
* working justification and finalization in local network simulation
* fix currently incorrect state transition/attestation test assumption
* State sim compiles and run
* Works on libp2p testnet1
* Successfully update the state transition tests
* naive update of attestation pool (failing attstations can arrive in any order)
* Remove an official assert to allow attestation reordering
* include both static and dynamic check for invariants to ensure that get_crosslink_committee(...) can access all committees by ensure that SHARD_COUNT >= SLOTS_PER_EPOCH
* add state_sim to CI, to ensure that it also doesn't break SHARD_COUNT/SLOTS_PER_EPOCH assumptions, along with just in general not breaking
- added a "quitProc" for shutting down the p2pd process, after seeing it
survive an unhandled exception (this required a global var for the
DaemonApi instance)
- Makefile testnet0/1 targets now depend on the "p2pd" binary
- prefixed the testnet command line in the build script message with
"../../env.sh" so it can find the superproject's "p2pd"
* rename get_genesis_beacon_state(...) to initialize_beacon_state_from_eth1(...); remove unused vestiges get_temporary_block_header(...) and get_empty_block(...); partly align initialize_beacon_state_from_eth1(...) with 0.8.0 version; implement CompactCommittee object type; update process_final_updates(...) to 0.8.0
* mark get_shard_delta(...) as 0.8.0
* mark get_total_active_balance(...) and epoch transition helper functions as 0.8.0
* move FORK_CHOICE_BALANCE_INCREMENT, not in spec anymore, to sole user in fork_choice
* remove some low-hanging perf silliness from get_winning_crosslink_and_attesting_indices(...) and switch inner/outer loop nesting order to get 10-15x function speedup for 128 and 512 validator cases by avoiding accidentally quadratic behavior, while keeping function signature unchanged and allowing easy ongoing verification of correctness of optimization
* avoid memory allocation with an openarray rather than seq
* update get_crosslink_deltas(...) to 0.8.0
* move non-spec ValidatorSetDeltaFlags from datatypes.nim to sync_protocol; fix mainnet MIN_ATTESTATION_INCLUSION_DELAY preset; update get_attestation_deltas(...) to 0.8.0; for 512 validators, get 90x speedup for get_attestation_deltas(...) from 179s to 2s
* explain purpose of when false portions, so they can be removed once accomplished
* replace BeaconState.finalized_{epoch,root} with BeaconState.finalized_checkpoint; rename get_delayed_activation_exit_epoch(...) to compute_activation_exit_epoch(...) and mark as 0.8.0; update get_churn_limit(...)/get_validator_churn_limit(...) to 0.8.0; update process_registry_updates(...) to 0.8.0
* update process_crosslinks(...) to 0.8.0; mark compute_start_slot_of_epoch(...) and get_committee_count(...) as 0.8.0
* mark Fork, is_slashable_validator(...), and get_beacon_proposer_index(...) as 0.8.0
* rename LATEST_SLASHED_EXIT_LENGTH to EPOCHS_PER_SLASHINGS_VECTOR; update process_slashings(...) to 0.8.0; remove pointless type conversion warning in get_previous_epoch(...)
* convert remaining references to finalized_epoch to finalized_checkpoint.epoch
* update slash_validator(...) to 0.8.0; mark inital value, Gwei, and time constants as 0.8.0; mark hash(...) and processBlockHeader(...) as 0.8.0
* rename WHISTLEBLOWING_REWARD_QUOTIENT to WHISTLEBLOWER_REWARD_QUOTIENT; rename LATEST_ACTIVE_INDEX_ROOTS_LENGTH to EPOCHS_PER_HISTORICAL_VECTOR (randao will also get merged into this); remove get_active_index_root(...); mark time parameter, signature domain types, and max operations per block constants as 0.8.0; update rewards and penalties constants to 0.8.0
* update is_valid_indexed_attestation(...) to 0.8.0; mark process_slot(...) as 0.8.0
* replace BeaconState.{current,previous}_justified_{epoch,root} with BeaconState.{current,previous}_justified_checkpoint
* rename slot_to_epoch(...) to compute_epoch_of_slot(...)
* rename aggregation_bitfield/custody_bitfield fields to aggregation_bits/custody_bits; rename convert_to_indexed(...) to get_indexed_attestation(...); mark BeaconBlockHeader as 0.8.0; update minimal preset MIN_ATTESTATION_INCLUSION_DELAY/time parameters in general to 0.8
* fix beacon node compilation; it used slightly different capitalizations of slot_to_epoch/slotToEpoch
* mark integer_squareroot(...), Deposit, VoluntaryExit, and Transfer as 0.8.0; rename get_epoch_start_slot(...) to compute_start_slot_of_epoch(...)
* add new Domain alias for uint64; rename bls_domain(...) to compute_domain(...), MAX_INDICES_PER_ATTESTATION to MAX_VALIDATORS_PER_COMMITTEE, and SignatureDomain to DomainType; update get_domain(...) to 0.8.0
* update/mark initiate_validator_exit(...) and Crosslink as 0.8.0; rename BeaconState.latest_block_roots -> BeaconState.block_roots; mark HistoricalBatch as 0.8.0
* migrate attestation pool tests from get_crosslink_committees_at_slot(...) to get_crosslink_committee(...)
* rm obsolete, unused get_crosslink_committees_at_slot_cached(...) and migrate tests/test_state_transition from get_crosslink_committees_at_slot(...) to get_crosslink_committee(...)
* migrate tests/testutil from get_crosslink_committees_at_slot(...) to get_crosslink_committee(...)
* use more pervasive caching infrastructure, initially of compute_committee; remove buggy (and per-index) shuffling to fix research/state_sim, which was noticing validators on multiple shard committees; rm now-unused specific get_attesting_balance_cached
* add some get_active_validator_index caching
* rm obsolete/unused get_attesting_indices_cached
* rm get_crosslink_committees_at_slot(...)
* some more 0.7 changes -- some of which can't be completed pending some data structure updates -- and shard handling changes to offset with epoch start shards
Dustin Brody
Jacek Sieka
Yuriy Glukhov
Mamy Ratsimbazafy
Zahary Karadjov
cheatfate
Ștefan Talpalaru
Yuriy Glukhov
Mamy Ratsimbazafy
Bruno Škvorc
andri lim