Commit Graph

412 Commits

Author SHA1 Message Date
Zahary Karadjov 1099548775
Restore the number of simulation nodes to 4; More block pool tracing [skip ci] 2019-11-26 19:02:56 +02:00
Zahary Karadjov 85d9df3853 Make the tests green 2019-11-26 02:45:25 +02:00
Zahary Karadjov 8b89bbe391 Fix off-by-one error in syncing 2019-11-26 02:45:25 +02:00
Zahary Karadjov 1d8bfd8c0d Allow overriding the bootstrap node in run_node.sh 2019-11-26 02:45:25 +02:00
Zahary Karadjov 8de4d38e4e More tracing 2019-11-26 02:45:25 +02:00
Dustin Brody 1494bcc262 replace links to Apache and MIT licenses with HTTPS versions (#592) 2019-11-25 16:30:02 +01:00
Dustin Brody f6de87b80e remove O(n^2) eth1 deposit processing and scaffolding comparing it with O(n) deposit processing; re-enable now-viable-to-run genesis state initialization tests with around 16,384 validators 2019-11-25 14:48:59 +00:00
Dustin Brody fa2a703a17 switch more funcs and procs from global to module scope; create scaffolding for non-O(n^2) eth1 deposit processing with assertions for equivalent functionality; fix a few more shellcheck warnings 2019-11-25 14:48:59 +00:00
Mamy André-Ratsimbazafy ea241e407d
use doAssert in static proc (https://github.com/status-im/nim-beacon-chain/pull/585#discussion_r350073704) closes #582 2019-11-25 11:09:36 +01:00
Dustin Brody a8de94ad05 add validator_leaving logging at debug chronicles level for all call paths to initiate_validator_exit(...) 2019-11-25 09:08:15 +00:00
Dustin Brody 8d9fdb0afd miscellanous cleanups and removing a few funcs/procs from unnecessary global scopes 2019-11-25 09:08:15 +00:00
Mamy Ratsimbazafy 22adccc917
support List in reportDiff and comment out BLS and shuffling tests (#585)
* support List in report diff

* Json tests were removed but walkDirRec lull-ed us into thinking we were passing them :/
2019-11-24 11:46:45 +01:00
Mamy Ratsimbazafy 1938379bcd
Automated beacon constant checks (#583)
* Fix SSZ bitlist deserialization super silly bug

* Add an automated sanity checks of the beacon chain constants

* Remove SSZ consensus skipping procs [skip ci]

* Add phase 1 domains

* Fix mainnet constants

* Add missing phase 1 constants on minimal (they are not needed somehow on mainnet)

* Rebase artifact: constants were defined twice
2019-11-22 20:56:39 +01:00
Dustin Brody edfd65fd5d stop download 0.9.1 test vectors; finish 0.9.2 update, primarily via fixing preset constants; remove a couple 0.19.6 shims 2019-11-22 16:41:43 +00:00
Dustin Brody f5b8931fad switch assert(...) to doAssert(...) and minor cleanup 2019-11-22 16:39:05 +00:00
Dustin Brody 41fa377638 mark all non-changed 0.9.1 items as 0.9.2 (#576) 2019-11-22 09:29:04 +01:00
Mamy Ratsimbazafy 69e15a48a4
bump v0.9.2 test vectors (#575) 2019-11-22 00:23:41 +01:00
Dustin Brody 85f2596604 re-enable all_tests.nim mainnet tests 2019-11-21 19:40:35 +02:00
Jacek Sieka 7c6d87051a
simplify validator api (#573)
* avoid passing whole state to validator api (easier when it goes
out-of-process)
* clean up some cruft
2019-11-21 10:57:59 +01:00
Dustin Brody c0f009908c
re-enable test_interop based on zcli with 0.9.1 specs and update initialize_beacon_state_from_eth1(...) to 0.9.1 (#569)
* re-enable test_interop based on zcli with 0.9.1 specs and update initialize_beacon_state_from_eth1(...) to 0.9.1

* switch many procs to funcs

* fix import os.nim instead; ospaths is deprecated [Deprecated] warnings
2019-11-21 09:15:10 +00:00
Mamy Ratsimbazafy bb0979f232
Ssz fixes (#566)
* SSZ signature from EF are always opaque blobs (security issue - https://github.com/status-im/nim-beacon-chain/issues/555)
Enable
- Attestation
- Beaconstate (minimal only)
- Deposit
- DepositData
- ProposerSlashing

Updates #518

* mv debug_ssz to helpers

* Small reorg of the list types

* Fix IndexedAttestation, AttesterSlashing and BeaconBlock

* Deactivate on mainnet: AttesterSlashing, BeaconBlockBody, IndexedAttestation, Attestation, BeaconBlock

* Fix Validators on minimal and mainnet
2019-11-19 12:04:51 +01:00
Dustin Brody 2749cb14a6 convert mostly-uncompiled verify_merkle_branch(...) calls to newer is_valid_merkle_branch(...) name; re-enable several lines of deposit testing code and outline roadmap for completion 2019-11-18 17:35:58 +00:00
Dustin Brody d8590bbffc fix several warnings and hints 2019-11-18 17:35:58 +00:00
Dustin Brody 6f87c8fd89 verify that state_sim is justifying and finalizing; fix 3 more warnings; rename crosslink_committee_cache to beacon_committee_cache; fix O(n^2) usage of get_base_reward(...) 2019-11-18 17:35:58 +00:00
Dustin Brody 3f958d3529
Merge branch 'master' into bat 2019-11-14 19:37:47 +00:00
Dustin Brody 7e70d16da9
remove BeaconStateNew and associated testing scaffolding/infrastructure 2019-11-14 20:03:08 +01:00
Dustin Brody 96dcb1cf95
Remove shard-relating code 2019-11-14 19:48:12 +01:00
Dustin Brody cd45b8b43f
mockAttestationData(...)/mockAttestationImpl(...) changes also match exactly 2019-11-14 19:09:18 +01:00
Dustin Brody c08698bffe
scaffolding to remove last vestiges of get_committee_count(...), SHARD_COUNT, and rest of that 0.8.x infrastructure; this demonstrates equivalence between two, before discarding old code 2019-11-14 18:37:51 +01:00
Dustin Brody b4850fbb06 remove XDeclaredButNotUsed spam 2019-11-14 15:58:31 +00:00
Mamy André-Ratsimbazafy f98dd9b1bf Bump: only keep 0.9.1 tests, remove LFS 2019-11-14 12:48:34 +00:00
Dustin Brody df1d00f13a remove UnusedImport spam (#549)
* remove UnusedImport spam

* re-add random import to beacon_node
2019-11-14 11:47:55 +01:00
Dustin Brody 24ade89e33 don't download 0.9.0 test vectors 2019-11-13 14:30:21 +00:00
Dustin Brody 5a54c823d8 update to 0.9.1: don't try to run removed tests; re-add 1.5 tests (one only in minimal); remove bls_verify_multiple(...) and AttestationDataAndCustodyBit; and update process_attester_slashing(...), get_indexed_attestation(...), and is_valid_indexed_attestation(...) 2019-11-13 13:49:38 +00:00
Zahary Karadjov f40675509a Make all testnet env file values optional [skip CI]
Also removes some usages of SHARD_COUNT
2019-11-12 23:43:38 +00:00
Zahary Karadjov 048a786d7c
Fix the local simulation on Linux [skip ci] 2019-11-12 19:24:43 +00:00
Zahary Karadjov 92fb3d8233 Fix the local network sim when multitail is used 2019-11-12 19:08:12 +00:00
Mamy André-Ratsimbazafy a44925cd24 Enable testing of a third of the SSZ consensus objects (minimal and mainnet) 2019-11-12 15:57:27 +00:00
Mamy André-Ratsimbazafy f14d4c4796 Remove the old SSZ tests [skip ci] 2019-11-12 15:57:27 +00:00
Dustin Brody 8e259368a3 flip all but 3 test fixtures to 0.9.1 test vectors 2019-11-12 14:49:26 +00:00
Dustin Brody 637054e765 stop downloading 0.8.3 test vectors 2019-11-12 14:49:26 +00:00
Dustin Brody d5ce142511 fix network sim finalization; remove get_attestation_data_slot(...); remove 2 more get_crosslink_committee(...) calls 2019-11-12 14:49:26 +00:00
Zahary Karadjov 7df6c241a7 Fix a strange build failure in the tests 2019-11-11 23:29:36 +00:00
Zahary Karadjov f5e9b9a922 Allow running local nodes without attached validators
The number of user nodes is now specified with `USER_NODES`.

To make the instructions more stable, the "numeric id" of the user
nodes will be starting from 0 (so you can always use `run_node.sh 0`
to start a user node).

If you specify a node index above the total number of nodes, you'll
launch a node without any validators attached (this is useful for
testing the sync for example).
2019-11-11 23:29:36 +00:00
Zahary Karadjov 22591deced Safer testnet restarts; Working CLI queries for inspecting the genesis states
When the connect_to_testnet script is invoked it will first verify that
the genesis file of the testnet hasn't changed. If it has changed, any
previously created database associated with the testnet will be erased.

To facilitate this, the genesis file of each network is written to the
data folder of the beacon node. The beacon node will refuse to start if
it detects a discrepancy between the data folder and any state snapshot
specified on the command-line.

Since the testnet sharing spec requires us to use SSZ snapshots, the Json
support is now phased out. To help with the transition and to preserve the
functionality of the multinet scripts, the beacon node now supports a CLI
query command that can extract any data from the genesis state. This is
based on new developments in the SSZ navigators.
2019-11-11 23:29:36 +00:00
Zahary Karadjov a644839b79 Dynamic SSZ navigator 2019-11-11 23:29:36 +00:00
Dustin Brody d457dfee81 switch all but 4 remaining callers to get_crosslink_committee(...) to get_beacon_committee(...) 2019-11-11 11:46:03 +00:00
Dustin Brody 949d735155 fix remaining test 0.9.0 test fixtures and switch back to non-transitiontest setup; remove get_committee_count(...); keep SHARD_COUNT consistent with MAX_COMMITTEES_PER_SLOT 2019-11-11 10:25:57 +00:00
Dustin Brody cb4a86756b update get_seed(...) and get_beacon_proposer_index(...) to 0.9.0, implement compute_proposer_index(...), and render 3 more test fixtures working (#532)
* update get_seed(...) and get_beacon_proposer_index(...) to 0.9.0, implement compute_proposer_index(...), and render 3 more test fixtures working

* rm stray Crosslink reference which prevented static SSZ tests from building

* remove references to removed tests in attestations test fixture; add minimal-preset block sanity test, plus all but one of mainnet tests for block sanity to transition fixtures
2019-11-10 01:03:41 +01:00
Mamy André-Ratsimbazafy d92d90eb0f add v0.9.1 fixtures 2019-11-09 10:03:11 +00:00
Dustin Brody 7e3fbbebd0 rename working_tests to more descriptive transition_0_9_0 tests 2019-11-08 13:23:08 +00:00
Dustin Brody e1cdbb5516 fix typo 2019-11-08 13:23:08 +00:00
Dustin Brody e984b2c932 mark is_valid_merkle_branch(...), process_justification_and_finalization(...), and process_rewards_and_penalties(...) as 0.9.0; update check_attestation(...) to 0.9.0; add 11 working test fixtures, having updated SSZ fixture directory to 0.9.0 2019-11-08 13:23:08 +00:00
Dustin Brody 275fb22c00 flip targeted spec version and ssz test fixtures to 0.9.0; almost completely update BeaconState to 0.9.0; update minimal and mainnet preset constants to 0.9.0; update process_proposer_slashing(...), get_attestation_deltas(...), and get_matching_head_attestations(...) to 0.9.0; mark process_final_updates(...) as 0.9.0 2019-11-08 13:23:08 +00:00
Dustin Brody 3bb95f571f remove remaining Crosslink and CompactCommittee vestiges 2019-11-08 13:23:08 +00:00
Dustin Brody 61c2cf9415 rm get_compact_committees_root(...), Transfer, and references to 0.9-removed BeaconState.{active_index_roots,compact_committees_roots; temporarily disable genesis interop test pending 0.9 BeaconState 2019-11-08 13:23:08 +00:00
Dustin Brody 68654848cb update get_attesting_indices(...) to 0.9.0; remove Crosslink from AttestationData to update AttestationData to 0.9.0; rm get_winning_crosslink_and_attesting_indices(...) and get_crosslink_deltas(...) 2019-11-08 13:23:08 +00:00
Dustin Brody c2ebe482a5 temporarily disable SSZ tests, which fail when trying to read AttestationData or structures which embed it; implemenet get_committee_count_at_slot(...) and get_beacon_committee(...); express get_crosslink_committee(...) in terms of get_beacon_committee(...) and ensure that results are identical 2019-11-08 13:23:08 +00:00
Mamy André-Ratsimbazafy 88e6bb04d2 Integrate to the test suite 2019-11-07 17:22:15 +00:00
Mamy André-Ratsimbazafy 4e5137011b Fix some declared but not used 2019-11-07 17:22:15 +00:00
Mamy André-Ratsimbazafy 05362886a9 SSZ Heterogeneous container: deserialize SingleField/Small/Fixed, Skip Var/Complex/Bits 2019-11-07 17:22:15 +00:00
Mamy André-Ratsimbazafy d17f15d020 stash support for Bitlist tests - serialization of distinct types not supported https://github.com/status-im/nim-beacon-chain/issues/518 2019-11-07 17:22:15 +00:00
Mamy André-Ratsimbazafy 1caea64ac9 add BitVector [skip ci] 2019-11-07 17:22:15 +00:00
Mamy André-Ratsimbazafy 3b8fc8fab8 Supports for parsing testing dynamically typed vectors [skip ci] 2019-11-07 17:22:15 +00:00
Mamy André-Ratsimbazafy bfbcf2a34d Skeleton of SSZ 0.9.0 for generic SSZ type (not consensus related no minimal/mainnet difference) [skip ci] 2019-11-07 17:22:15 +00:00
Mamy André-Ratsimbazafy 5087b5af30 remove unneeded imports in SSZ fixtures 2019-11-07 17:22:15 +00:00
Dustin Brody 0073555752
transition deposit operation fixtures to 0.9.0 (#515)
* transition deposit operations fixture to 0.9.0

* mark slash_validator(...) as 0.9.0

* switch remaining non-ref objects to ref objects to maybe avoid crashes in CI

* remove unused helpers/debug_state imports
2019-11-06 15:02:06 +00:00
Ștefan Talpalaru 21223d2627
fix tests/simulation/run_node.sh
when it's run by itself to start an additional sim node
2019-11-02 22:58:07 +01:00
Dustin Brody 55dac3e36f re-enable some tests on Win64 since Nim 1.0.2 fixes https://github.com/nim-lang/Nim/issues/12186 (#513) 2019-11-01 09:46:50 +01:00
Mamy Ratsimbazafy 753d5af447
Test vectors: use tarballs, allow multiple spec versions for progressive spec updates (#514)
* update test submodule [skip ci]

* rename the process_lfs script to setup_official_tests [skip ci]

* clarify that the existing LFS is for json files [skip ci]

* Update again to use download/unpack function library

* update the test downloading script [skip ci]

* Update EF fixtures with new test path

* fix symlink order

* Fix relative:absolute symlink issue

* add symlink existence check

* setup_official_tests.sh: refactoring
2019-11-01 09:44:16 +01:00
Dustin Brody 63e621c27d
initial 0.9.0 spec sync (#509)
* rename compute_epoch_of_slot(...) to compute_epoch_at_slot(...)

* remove some unnecessary imports; remove some crosslink-related code and tests; complete renaming of compute_epoch_of_slot(...) to compute_epoch_at_slot(...)

* rm more transfer-related code and tests; rm more unnecessary strutils imports

* rm remaining unused imports

* remove useless get_empty_per_epoch_cache(...)/compute_start_slot_of_epoch(...) calls

* rename compute_start_slot_of_epoch(...) to compute_start_slot_at_epoch(...)

* rename ACTIVATION_EXIT_DELAY to MAX_SEED_LOOKAHEAD

* update domain types to 0.9.0

* mark AttesterSlashing, IndexedAttestation, AttestationDataAndCustodyBit, DepositData, BeaconBlockHeader, Fork, integer_squareroot(...), and process_voluntary_exit(...) as 0.9.0

* mark increase_balance(...), decrease_balance(...), get_block_root(...), CheckPoint, Deposit, PendingAttestation, HistoricalBatch, is_active_validator(...), and is_slashable_attestation_data(...) as 0.9.0

* mark compute_activation_exit_epoch(...), bls_verify(...), Validator, get_active_validator_indices(...), get_current_epoch(...), get_total_active_balance(...), and get_previous_epoch(...) as 0.9.0

* mark get_block_root_at_slot(...), ProposerSlashing, get_domain(...), VoluntaryExit, mainnet preset Gwei values, minimal preset max operations, process_block_header(...), and is_slashable_validator(...) as 0.9.0

* mark makeWithdrawalCredentials(...), get_validator_churn_limit(...), get_total_balance(...), is_valid_indexed_attestation(...), bls_aggregate_pubkeys(...), initial genesis value/constants, Attestation, get_randao_mix(...), mainnet preset max operations per block constants, minimal preset Gwei values and time parameters, process_eth1_data(...), get_shuffled_seq(...), compute_committee(...), and process_slots(...) as 0.9.0; partially update get_indexed_attestation(...) to 0.9.0 by removing crosslink refs and associated tests

* mark initiate_validator_exit(...), process_registry_updates(...), BeaconBlock, Eth1Data, compute_domain(...), process_randao(...), process_attester_slashing(...), get_base_reward(...), and process_slot(...) as 0.9.0
2019-10-30 19:41:19 +00:00
Ștefan Talpalaru baed13dca2
sim: always enable the status bar [skip ci] 2019-10-29 21:13:24 +01:00
Zahary Karadjov 72c42838cb
Fix a typo in start.sh; Display the full slot number in the statusbar [skip ci] 2019-10-29 21:48:32 +02:00
Zahary Karadjov 6664f1689f
Use SSZ genesis files
Multi-client testing requires more portable formats, and SSZ is
much better specified than our flavour of Json.

Tools like ncli and zcli can be now used to inspect the contents
of the SSZ files.
2019-10-29 19:16:32 +02:00
Zahary Karadjov f1ea0cec32
Add support for testnets with mixed quickstart/random deposits 2019-10-29 19:16:32 +02:00
Zahary Karadjov ba0037738b
Removed all code related to the old network metadata files
Also switched to a more "standard" naming convention for the
command-line parameters.
2019-10-29 01:04:52 +02:00
Zahary Karadjov 43b70c0330
Restore the ability to run tests/simulation/start.sh manually 2019-10-28 15:28:45 +02:00
Dustin Brody bb7d892587
track attestations sent/received in grafana (#503)
* stop double-counting attestation-topic attestations

* track attestations sent/received in grafana

* address suggestion to clarify sent attestation count being across all nodes
2019-10-25 16:15:39 +00:00
Ștefan Talpalaru 3a2fc249a2
simulation: metric visualisation 2019-10-23 20:57:59 +02:00
Jacek Sieka f41fa704d5
Update readme instructions to use specific node/validator counts (fixes #483) 2019-10-23 15:22:35 +02:00
Jacek Sieka 669b81495a
simulation: make default setup finalize by running more nodes (see MISSING_NODES) 2019-10-22 15:59:42 +02:00
Zahary Karadjov 5f271fd82e
Quick and dirty script for running a Rpi tmux demos 2019-10-09 00:45:42 +09:00
Jacek Sieka b7f9d9e4be fix BlsValue inits 2019-10-08 23:52:50 +09:00
Zahary Karadjov f72a58595f Initial version of status bar 2019-10-07 23:56:34 +09:00
Dustin Brody 255980c9f6 fix issue #367 and remove too-small range type usage for ValidatorIndex 2019-10-03 16:41:25 +03:00
Yuriy Glukhov 2b7b7b1471 Deposit contract script additions (#455)
* Deposit contract script additions

* Cleanup/cosmetics
2019-10-03 09:21:28 +00:00
Jacek Sieka 2a28a95cfc update test as well 2019-10-02 12:27:51 +03:00
Mamy Ratsimbazafy 48d22d53dc
Ensure that we use the interop deposit signatures (#467)
* Ensure that we use the interop deposit signatures

* Add all interop keys to the test
2019-10-01 15:44:38 +02:00
Zahary Karadjov df7531f9bf
More testnet scripts fixes cherry-picked from the interop branch 2019-09-26 19:07:38 +03:00
Zahary Karadjov 0eaa433e84
Automated scripts for resetting the Status testnets
For detailed instructions, please see

https://github.com/status-im/nimbus-private/blob/master/testnets-maintenance.md
2019-09-26 18:58:51 +03:00
Mamy André-Ratsimbazafy 90b20fab52 Domain is now array[8, byte] instead of uint64 2019-09-26 17:53:59 +03:00
Mamy André-Ratsimbazafy 763e296731
Fix and activate the justification and finalization unit tests. 2019-09-24 16:56:12 +02:00
Mamy Ratsimbazafy 6be5805462
Enable more tests (#442)
* enable all sanity slots tests

* enable more tests in sanity blocks

* win64 strikes again #435

* workaround invalid handle in win64 CI

* empty_epoch_transition_not_finalizing only valid in minimal preset

* sanity block ETH1 data votes are also minimal only
2019-09-12 17:28:33 -04:00
Mamy Ratsimbazafy 88ec9fd4be "industrialize" skipping win64 CI (#441)
* "industrialize" skipping win64 CI
- reintroduce actually working tests
- skip BLS priv_to_pub (tested in blscurve + broken by #440 and #435)

* missing echo

* try/except OSError doesn't seem to work in unittest so plain ignore + move x64 first for fail fast

* ignore transfers completely in WIn64
2019-09-12 11:33:07 +00:00
Jacek Sieka 65ff8f2886
remove networkId (unused) 2019-09-11 20:29:57 -04:00
Mamy Ratsimbazafy 988a13b872 Official operations - deposits unit test (#408)
* [Test] Official operations - deposits unit test

* Allow ignoring deposits with invalid signature

* We need stacktraces to debug windows 64 issue https://github.com/status-im/nim-beacon-chain/pull/408#issuecomment-529236359

* fix naming of unit test proc

* Revert "We need stacktraces to debug windows 64 issue https://github.com/status-im/nim-beacon-chain/pull/408#issuecomment-529236359"

This reverts commit 04b8b05162.

* skip windows-64 in CI #435

* proposer slashing started to crash as well on win-64 #435
2019-09-11 21:10:54 +00:00
Mamy Ratsimbazafy 3dc2b87e6a Transfer - split process_transfer/processTransfers + tests + fixes (#422)
* Prepare test suite for transfers

* split API process_transfer / processTransfers

* Add range checks on transfer

* Fix invalid transfer conditions

* don't test on windows 64-bit #435
2019-09-11 20:29:00 +00:00
Dustin Brody 1061708ec2 disable proposer slashing tests only on 64-bit windows (#438)
* disable proposer slashing tests only on 64-bit windows

* add visible skipped-test output
2019-09-11 09:52:38 -04:00
Mamy Ratsimbazafy 8676bbf388 Add proposer slashing tests (#431)
* Add proposer slashing tests

* typo in import
2019-09-11 07:50:07 +00:00
Mamy Ratsimbazafy 6be2e24a8f
Attester slashing - split process_attester_slashing/processAttesterSlashings + tests (#424)
* attester slashing tests - pending #415

* split process_attester_slashing/processAttesterSlashings

* Add logs to attester_slashing

* deactivate bls tests for now (https://github.com/status-im/nim-beacon-chain/issues/429) and cherry-pick from 60f2437dd2
2019-09-10 18:03:06 -04:00
Mamy Ratsimbazafy 81b47f35d1 Voluntary exit - split single + multiple exits proc & tests (#421)
* add test suite for voluntary exit

* update API to process_voluntary_exit

* Add range check of validator_index for voluntary exits

* Revert to dual single + multiple voluntary exits API + enable in test suite

* no cache or mocking needed
2019-09-10 00:14:03 +00:00
Mamy Ratsimbazafy 061b6e0ddf
Test block headers (#418)
* Block headers tests

* Skip invalid_parent_root test due to https://github.com/status-im/nim-beacon-chain/issues/407

* Fix bls_verify with invalid sig. Activate block header tests
2019-09-09 14:40:59 -04:00
Dustin Brody a4b4c89539 remove incorrect exit condition in get_attestation_details(...) (#416)
* remove incorrect exit condition in get_attestation_details(...)

* enable historical batch unit test, which now passes
2019-09-09 09:10:03 -04:00
Mamy Ratsimbazafy 3f446e6383
Attestations EF tests + Proper Real/Opaque BlsValue deserialization (#410)
* Add attestation unit test

* process_attestation doesn't throw exceptions

* Allow SSZ deserialization of both real and invalid signatures

* Add new process_attestation checks - pass all process_attestation tests

* Add sanity check for #361

* Fix SSZ testing after fromBytes/fromSSZBytes changes
2019-09-08 23:33:24 -04:00
Mamy Ratsimbazafy 82b9e008d6
Fix genesis blockheader zero signature (supercedes #395) (#400) 2019-09-07 20:56:24 -04:00
Jacek Sieka 31db5d3a62 Deserialize serialized default bls values (#388)
Ugly workaround to quick-fix broken network/interop sim
2019-09-05 14:27:28 +00:00
Ștefan Talpalaru 33a9370c99
improve tests/simulation/start.sh and friends [skip ci] 2019-09-05 12:55:47 +02:00
zah 7ebf685fa3 Work-around for #373 (#385) 2019-09-05 10:27:26 +00:00
Mamy Ratsimbazafy c11f37e550 V0.8.3 tests part 3 - sanity checks for slots and blocks (#375)
* Add sanity check for slot processing (also impacted by https://github.com/status-im/nim-beacon-chain/issues/373)

* use reportDiff also for all state tests vs EF

* initial sanity checks for blocks - workaround zero signature in block headers: https://github.com/status-im/nim-beacon-chain/issues/374

* Remove generic object variant compare commented code

* Add the one block state transition sanity checks

* generalize blocks test to multiple blocks

* simplify slots test runner

* Add official epoch transitions, sanity blocks, sanity slots to the test suite

* Fix index out-of-bounds in initiate_validator_exit - enable proposer slashings unittest
2019-09-03 18:02:21 +00:00
Mamy Ratsimbazafy 297e9079d4 V0.8.3 tests part2 - BLS + Shuffling + State transition epoch (#372)
* Update BLS fixtures to 0.8.3

* Bump fixtures with shuffling with minimal preset

* Update shuffling tests

* parseTest generic over file format (Json or SSZ)

* Initial crosslink parsing commit for debugging Nim crash

* Workaround https://github.com/status-im/nim-beacon-chain/issues/369

* Crosslink test works for minimal - https://github.com/status-im/nim-beacon-chain/issues/369 is back on mainnet

* Use ref objects to workaround https://github.com/status-im/nim-beacon-chain/issues/369

* Generalize state transition epoch test to all epoch tests

* Fix slashing (potential uint64 overflow in previous spec)

* Add a state debugging macro to deeply inspect the wrong fields

* make reportDiff visible

* Improve the debug state macro for containers
2019-09-03 03:12:09 +00:00
Jacek Sieka 9cfbdb5e16 more interop fixes
* fix eth1 interop block hash
* update initialize_beacon_state_from_eth1 to 0.8.3
2019-09-02 22:14:18 +03:00
Jacek Sieka 7b73b40bab interop updates
* add interop launcher scripts
* stick validator_keygen into beacon_node
* fix lmd ghost slot number on missing block
* use mocked eth1data when producing blocks
* use bls public key method for withdrawal credentials
* fix deposit domain
* prefer lowercase for a bunch of toHex
* build simulation binary in data folder to avoid data types confusion
2019-09-02 22:14:18 +03:00
Mamy Ratsimbazafy 1ec4c5aef7
Prepare for v0.8.3 tests: submodules + LFS script + Appveyor updates (#368)
* Update fixtures to 0.8.3

* v0.8.1 tests use new test path

* LFS caching: handle multiple versions of json_tests

* Enable paths > 260 characters in appveyor

* Appveyor: use git clone core.longpaths + the env variable at startup

* improve process_lfs.sh and cleanup submodules
2019-08-30 17:50:46 +02:00
Mamy André-Ratsimbazafy 9cfc153620
Remove obselete mocking helper skeleton 2019-08-29 00:36:37 +02:00
Mamy André-Ratsimbazafy cfcdfff54a
Fix comments in j/f tests + add a readme for scaled image 2019-08-28 16:22:48 +02:00
Mamy Ratsimbazafy ca4f29caca v0.8.1 tests refactor (#326)
* Introduce new mocking proc to replace:
- makeFakeValidatorPrivKey
- hackPrivKey
- getNextBeaconProposerIndex
- addBlock
- makeBlock

* Add comments on datastructure unsynced with the spec

* Add merkle tree constructor and initial mocking for deposits (missing merkle proofs)

* [Mock] Implement sparse merkle tree and merkle proof builder

* [Mocking] Genesis deposits

* Add compact_committees_roots init + mock genesis state

* [Tests] Add first deposit test using the new mocking procedures

* [Tests -deposits] add at and over 32 ETH deposit tests

* [Tests - deposits] Add test for validator top-up

* [Tests -deposits] Mention the TODO to test for invalid conditions

* [Tests] Add stub to test "is_valid_genesis_state"

* [Merkle proofs] Implement round-trip checks

* Deactivate roundtrips test

* SSZ - use EF convention for hash_tree_root / hashTreeRoot

* [Tests - Attestation] Attestation mocking + initial test

* Add mocking + 3 new tests for valid attestations + mention future invalid attestation tests

* Add crosslinks test (1 failing to attestations in block being duplicated in state transition)

* Single attestation crosslink test - workaround https://github.com/status-im/nim-beacon-chain/issues/361

* Add test for failed crosslink penalty

* Rebase fixes + add refactored tests to test suite

* justif-finalization helpers first batch

* Add 234 finalization tests

* Fix justif test, Rule I 234 finalization does not happen with sufficient support.
(Also unittest check template does not fail properly in some cases)

* Add tests for all finalization rules

* Properly delete nim-byteutils following c91727e7e5 (diff-7c3613dba5171cb6027c67835dd3b9d4)

* use digest helper for deposit root
2019-08-28 12:07:00 +00:00
Zahary Karadjov f477eb6877
Add a `sszList` helper for writing tests involving SSZ lists 2019-08-20 16:34:36 +03:00
Zahary Karadjov 7d3d30c576
Restore some accidentally disabled test code 2019-08-20 16:34:35 +03:00
Jacek Sieka 78f7a4a505 restore network sim finalization
* relax attestation validation when attestation is incoming but make it
more strict when adding to block
* share attestation validation logic between attestation pool and state
transition
* remove a bunch of redundant logging
* fix potential underflow in attestation delay checking
* fix committee used for attestation shard selection when attesting
* fix attestation data construction
2019-08-19 21:05:54 +03:00
Zahary Karadjov 93336daf7f
Remove some confusing code duplication; Use multiple nodes by default in the network sim 2019-08-15 20:43:05 +02:00
Ștefan Talpalaru 7fff9a09fc handle LFS fixtures for the test suite (#339)
* handle LFS fixtures for the test suite

* test_fixture_ssz_static.nim: allow the tests to fail properly

- switch to a debug build for the failing tests

* try -d:debug
2019-08-14 13:19:17 +02:00
Dustin Brody cba25b087c
render tests/simulation/start.sh more robust to paths with spaces (#340) 2019-08-14 10:25:39 +00:00
Dustin Brody 3e587d4667
Justification/finalization amelioration using testnet1 shard count and epoch length (#331)
* working justification, 1 node, 64 validators

* closer to tn1 params: 256 validators

* more debugging output; switch to minimum test case

* working justification and finalization in local network simulation

* fix currently incorrect state transition/attestation test assumption
2019-08-14 08:56:32 +00:00
Zahary Karadjov a897ab9c70
Attempt to make the test suite green 2019-08-07 06:09:26 +03:00
Zahary Karadjov 398ea55801
Implement the latest SSZ specification and integrate the official SSZ test suite 2019-08-05 03:18:48 +03:00
Yuriy Glukhov 24cbc0710f Make web3 way of getting genesis optional 2019-07-31 16:39:26 +03:00
Yuriy Glukhov 958e4cd3b4 Getting genesis through web3 2019-07-31 16:39:26 +03:00
Jacek Sieka 356573f1ca
Use offset 1000 for fake private keys 2019-07-24 08:19:52 +02:00
Mamy André-Ratsimbazafy 3e622644e7
Remove committed binaries 2019-07-21 17:23:05 +02:00
Mamy Ratsimbazafy d1d5497233
v0.7.x-v0.8.1 state transition refactor part2 (#322)
* State sim compiles and run

* Works on libp2p testnet1

* Successfully update the state transition tests

* naive update of attestation pool (failing attstations can arrive in any order)

* Remove an official assert to allow attestation reordering
2019-07-15 23:10:40 +02:00
Dustin Brody 4fdea7a300
tentative fix for local network simulation to fix justification and finalization 2019-07-11 18:08:27 +02:00
Dustin Brody dfa062db7d 0.8.0, continued (#315)
* rename get_genesis_beacon_state(...) to initialize_beacon_state_from_eth1(...); remove unused vestiges get_temporary_block_header(...) and get_empty_block(...); partly align initialize_beacon_state_from_eth1(...) with 0.8.0 version; implement CompactCommittee object type; update process_final_updates(...) to 0.8.0

* mark get_shard_delta(...) as 0.8.0

* mark get_total_active_balance(...) and epoch transition helper functions as 0.8.0

* move FORK_CHOICE_BALANCE_INCREMENT, not in spec anymore, to sole user in fork_choice
2019-07-10 14:23:02 +02:00
Jacek Sieka b9e62a1f6c
std_shims -> stew 2019-07-07 11:53:58 +02:00
Dustin Brody 68be98cafa
More 0.8.0 updates (#311)
* replace BeaconState.finalized_{epoch,root} with BeaconState.finalized_checkpoint; rename get_delayed_activation_exit_epoch(...) to compute_activation_exit_epoch(...) and mark as 0.8.0; update get_churn_limit(...)/get_validator_churn_limit(...) to 0.8.0; update process_registry_updates(...) to 0.8.0

* update process_crosslinks(...) to 0.8.0; mark compute_start_slot_of_epoch(...) and get_committee_count(...) as 0.8.0

* mark Fork, is_slashable_validator(...), and get_beacon_proposer_index(...) as 0.8.0

* rename LATEST_SLASHED_EXIT_LENGTH to EPOCHS_PER_SLASHINGS_VECTOR; update process_slashings(...) to 0.8.0; remove pointless type conversion warning in get_previous_epoch(...)

* convert remaining references to finalized_epoch to finalized_checkpoint.epoch

* update slash_validator(...) to 0.8.0; mark inital value, Gwei, and time constants as 0.8.0; mark hash(...) and processBlockHeader(...) as 0.8.0

* rename WHISTLEBLOWING_REWARD_QUOTIENT to WHISTLEBLOWER_REWARD_QUOTIENT; rename LATEST_ACTIVE_INDEX_ROOTS_LENGTH to EPOCHS_PER_HISTORICAL_VECTOR (randao will also get merged into this); remove get_active_index_root(...); mark time parameter, signature domain types, and max operations per block constants as 0.8.0; update rewards and penalties constants to 0.8.0

* update is_valid_indexed_attestation(...) to 0.8.0; mark process_slot(...) as 0.8.0

* replace BeaconState.{current,previous}_justified_{epoch,root} with BeaconState.{current,previous}_justified_checkpoint
2019-07-05 08:30:05 +00:00
Dustin Brody d7905351eb update get_crosslink_committee(...) to 0.8.0 and fix https://github.com/status-im/nim-beacon-chain/issues/307 2019-07-02 19:52:39 +03:00
Zahary Karadjov c98426586e
Fix the use of tests/simulation/run_node.sh 2019-07-01 15:36:01 +03:00
Dustin Brody fb601bd5b8 rename BeaconState.validator_registry to BeaconState.validators 2019-07-01 14:44:57 +03:00
Dustin Brody 607a7de5eb rename get_epoch_start_shard(...) to 0.8.0 get_start_shard(...) 2019-07-01 14:44:57 +03:00
Dustin Brody e4321dc4ed Initial frozen phase 0/v0.8.0 spec (#305)
* rename slot_to_epoch(...) to compute_epoch_of_slot(...)

* rename aggregation_bitfield/custody_bitfield fields to aggregation_bits/custody_bits; rename convert_to_indexed(...) to get_indexed_attestation(...); mark BeaconBlockHeader as 0.8.0; update minimal preset MIN_ATTESTATION_INCLUSION_DELAY/time parameters in general to 0.8

* fix beacon node compilation; it used slightly different capitalizations of slot_to_epoch/slotToEpoch

* mark integer_squareroot(...), Deposit, VoluntaryExit, and Transfer as 0.8.0; rename get_epoch_start_slot(...) to compute_start_slot_of_epoch(...)

* add new Domain alias for uint64; rename bls_domain(...) to compute_domain(...), MAX_INDICES_PER_ATTESTATION to MAX_VALIDATORS_PER_COMMITTEE, and SignatureDomain to DomainType; update get_domain(...) to 0.8.0

* update/mark initiate_validator_exit(...) and Crosslink as 0.8.0; rename BeaconState.latest_block_roots -> BeaconState.block_roots; mark HistoricalBatch as 0.8.0
2019-07-01 09:53:42 +02:00
Dustin Brody 7356905f95
remove get_epoch_start_slot(...) kludge and update to work exactly as 0.7.1 specifies; remove unused get_attestation_participants_cached(...); update AttestationData to 0.7.1 (being the last data structure, I believe, remaining for such; remove potentially spurious/certainly not-in-spec assertion from compute_committee(...); fix state sim to work with new get_epoch_start_slot/AttestationData/etc setup where it can't stuff all shards' attestations from same slot in the same MIN_ATTESTATION_INCLUSION_DELAY rotating/circular buffer of seq[Attestation]s without more involved shuffling of shard/slot calculation order; fix attestation pool testing to be consistent with get_epoch_start_slot(...) (#302) 2019-06-29 09:17:24 +00:00
Zahary Karadjov 31baa77742 Restore building with chronicles_sinks=json 2019-06-24 19:38:19 +03:00
Dustin Brody c7e06374f4
Remove get_crosslink_committees_at_slot and fix research/state_sim (#291)
* migrate attestation pool tests from get_crosslink_committees_at_slot(...) to get_crosslink_committee(...)

* rm obsolete, unused get_crosslink_committees_at_slot_cached(...) and migrate tests/test_state_transition from get_crosslink_committees_at_slot(...) to get_crosslink_committee(...)

* migrate tests/testutil from get_crosslink_committees_at_slot(...) to get_crosslink_committee(...)

* use more pervasive caching infrastructure, initially of compute_committee; remove buggy (and per-index) shuffling to fix research/state_sim, which was noticing validators on multiple shard committees; rm now-unused specific get_attesting_balance_cached

* add some get_active_validator_index caching

* rm obsolete/unused get_attesting_indices_cached

* rm get_crosslink_committees_at_slot(...)

* some more 0.7 changes -- some of which can't be completed pending some data structure updates -- and shard handling changes to offset with epoch start shards
2019-06-24 09:21:56 +00:00
Zahary Karadjov 96e2a02faf
Respect the port configuration and NAT setup when using LibP2P 2019-06-24 05:34:22 +03:00
Zahary Karadjov 15fdf78d6a
[LibP2P] Persistent network key for the bootstrap node 2019-06-24 05:34:22 +03:00
Zahary Karadjov 12e9367f78
Improved error handling; Simple test case for connecting 2 peers 2019-06-24 05:34:22 +03:00
Mamy André-Ratsimbazafy 6f77351757
Update fixtures submodule to latest (remove deprecated eth2.0-tests)
https://github.com/status-im/nim-eth2-official-tests/issues/1
https://github.com/status-im/nim-beacon-chain/pull/292
2019-06-21 10:33:29 +02:00
Mamy Ratsimbazafy cacb7d6a2b
Remove v0.5 state tests (#292) 2019-06-21 10:16:14 +02:00
Mamy Ratsimbazafy 0d82be5af3
Update tests result to 0.7.1 (#293) 2019-06-21 10:14:16 +02:00
Mamy Ratsimbazafy 0926ebf8fe [Tests] SSZ unsigned integer + test suite genericity (#284)
* Generics over tests (https://github.com/status-im/nim-serialization/issues/4, https://github.com/status-im/nim-serialization/issues/5, https://github.com/nim-lang/Nim/issues/11225)

* Skeleton of SSZ uint tests

* Check all primitive uint types

* Add deserialization test. "wrong length" skipped due to https://github.com/status-im/nim-beacon-chain/issues/280

* Move test types to their specific test files

* Stint also sometimes throws an AssertionError for invalid ranges

* Add debug path for Access denied issue in Appveyor 64-bit (https://ci.appveyor.com/project/nimbus/nim-beacon-chain/builds/25278666/job/fs8q0bcluvj2gdor#L866)

* indent the Appveyor debug info
2019-06-17 09:29:23 +00:00
Dustin Brody e9180c490e
more 0.7.0 updates (#287)
* update Deposit and BeaconBlockHeader to 0.7.0

* update get_total_balance(...) and get_base_reward(...) to 0.7.0

* rename initial value constant BLS_WITHDRAWAL_PREFIX_BYTE to BLS_WITHDRAWAL_PREFIX

* mark minimal presets as entirely migrated to 0.7.0; update process_rewards_and_penalties(...) to 0.7.0

* specify that targeting 0.7.0; adopt 0.7.0 refactoring of bls_domain/get_domain to update get_domain(...) to 0.7.0; mark that mainnet constant presets and block header processing are 0.7.0-consistent; update process_justification_and_finalization to 0.7.0 to fix potential underflow errors

* address mratsim's comment to copy whole spec comment
2019-06-14 16:21:04 +00:00