Commit Graph

556 Commits

Author SHA1 Message Date
Viktor Kirilov c032366547
removed the BlockPool type and all of the proxy functions around it (#1401)
* removed the BlockPool type and all of the proxy functions around it - passing the chain DAG and the quarantine explicitly where appropriately - they don't need to be bundled in a type

* fixed the build after the rebase
2020-07-30 21:18:17 +02:00
tersec 254a510c0d
mark .borrow. procs as noSideEffect (#1402) 2020-07-30 14:47:59 +02:00
tersec 99f2d8e06c
update 14 v0.12.1 spec refs to v0.12.2 (#1400) 2020-07-30 09:47:57 +00:00
Mamy Ratsimbazafy 023f7f4518
Lazy crypto [alt design #1369] (#1371)
* Lazy loading of crypto objects

* Try to fix incorrect field access by hiding fields but no luck. SSZ/Chronicles/macro bug?

* Fix incorrect blsValue access. was "aggregate" not "chronicles"

* Fix tests that rely on the internal BLSValue representation
2020-07-29 18:13:05 +00:00
tersec d97cc35d30
switch 14 v0.12.1 spec refs to v0.12.2 spec refs (#1395) 2020-07-29 12:47:03 +00:00
tersec a07dab300c
verifyFinalization checks for justification being consistent with later finalization (#1386) 2020-07-28 17:35:32 +00:00
tersec b45de824a4
convert 22 v0.12.1 spec refs to v0.12.2 and remove 1 unnecessary spec ref (#1384) 2020-07-28 14:55:46 +00:00
Jacek Sieka 157ddd2ac4
Fork choice fixes 5 (#1381)
* limit attestations kept in attestation pool

With fork choice updated, the attestation pool only needs to keep track
of attestations that will eventually end up in blocks - we can thus
limit the horizon of attestations that we keep more aggressively.

To get here, we expose getEpochRef which gets metadata about a
particular epochref, and make sure to populate it when a block is added
- this ensures that state rewinds during block addition are minimized.

In addition, we'll use the target root/epoch when validating
attestations - this helps minimize the number of different states that
we need to rewind to, in general.

* remove CandidateChains.justifiedState

unused

* remove BlockPools.Head object

* avoid quadratic quarantine loop

* fix
2020-07-28 13:54:32 +00:00
Zahary Karadjov 40ea9e9886 Integrate the Wallet support with the Deposits creation; Produce Launchpad-compatible deposits 2020-07-28 07:36:25 +03:00
Zahary Karadjov fcd412f7a1 Finish the 'create wallet' command; Addresses #1319 2020-07-28 07:36:25 +03:00
tersec dad3dd5809
update 111 v0.12.1 beacon chain spec refs to v0.12.2 (#1380) 2020-07-27 18:49:46 +02:00
Jacek Sieka 48cebc7157
spec cleanups (#1379)
* add helper for beacon committee length (used for quickly validating
attestations)
* refactor some attestation checks to do cheap checks early
* validate attestation epoch before computing active validator set
* clean up documentation / comments
* fill state cache on demand
2020-07-27 16:04:44 +00:00
tersec 20a2525390
v0.12.2 beacon chain protocol update (#1378) 2020-07-27 12:59:57 +02:00
tersec 90708a8287
Prefer converting int` to uint64 and switch foo.len.uint64 to .len64 (#1375)
* avoid converting from uint64 to int, and where most feasible, int type conversion at all

* .len.uint64 -> .len64

* fix 32-bit compilation

* try keeping state_sim loop variable/bounds as int for 32-bit Azure

* len64 -> lenu64
2020-07-26 20:55:48 +02:00
Mamy Ratsimbazafy e5a76b0047
Stopgap don't print private key in shortlog (#1372) 2020-07-26 10:27:11 +02:00
Jacek Sieka e0a18a3105
cache beacon committee size calculation (#1363)
* cache beacon committee size calculation

this fixes a bug in get_validator_churn_limit as well

* fix

* make committee counts consistently uint64

mixing feels like the worst of the two worlds
2020-07-23 19:01:07 +02:00
Jacek Sieka fb2f742972
Fork choice fixes 2 (#1356)
* fork choice cleanup

* enable v2 pruning
* prefer `get_current_epoch`
* fix finalization check to use correct epoch

* small cleanups

* add `count_active_validators`
* remove misleading logs
* fix justified checkpoint slot calculation in rpc
2020-07-22 23:01:44 +02:00
tersec 6b77f3dda5
update compute_subnet_for_attestation() to use https://github.com/ethereum/eth2.0-specs/pull/1876 signature, which isn't in v0.12.1, which works with lookahead (#1346) 2020-07-22 08:04:21 +00:00
tersec 4a9a7be271
faster syncing (#1348)
* maybe faster syncing

* 80-character lines

* remove instrumentation debugEchos; fix target attestation epoch in attestation pool validation

* use the epoch-granularity matching in attestation.addResolved(...)
2020-07-22 09:51:45 +02:00
tersec 83abbcb917
drop get_attesting_indices()/get_unslashed_attesting_indices() from 15% to 1% of workload at block_sim at 100k validators (#1351) 2020-07-21 18:35:43 +02:00
Jacek Sieka 8b01284b0e
cache block hash (#1329)
hash_tree_root was turning up when running beacon_node, turns out to be
repeated hash_tree_root invocations - this pr brings them back down to
normal.

this PR caches the root of a block in the SignedBeaconBlock object -
this has the potential downside that even invalid blocks will be hashed
(as part of deserialization) - later, one could imagine delaying this
until checks have passed

there's also some cleanup of the `cat=` logs which were applied randomly
and haphazardly, and to a large degree are duplicated by other
information in the log statements - in particular, topics fulfill the
same role
2020-07-16 15:16:51 +02:00
Viktor Kirilov 4767eba465 - properly distributing the keys between BNs & VCs - fixing finalization - will switch to ON by default as soon as finalization becomes overall stable
- added a new altona-vc Makefile target which uses a separate VC process
2020-07-15 16:18:21 +03:00
tersec 26e893ffc2
restore EpochRef and flush statecaches on epoch transitions (#1312)
* restore EpochRef and flush statecaches on epoch transitions

* more targeted cache invalidation

* remove get_empty_per_epoch_cache(); implement simpler but still faster get_beacon_proposer_index()/compute_proposer_index() approach; add some abstraction layer for accessing the shuffled validator indices cache

* reduce integer type conversions

* remove most of rest of integer type conversion in compute_proposer_index()
2020-07-15 12:44:18 +02:00
tersec 4aefd8b021
fix over/underflow in proposerslashing processing (#1325) 2020-07-15 10:38:45 +00:00
zah 0be77f9cbc
Restore compilation with custom presets (#1309)
This also assigns precise types to the constants in the minimal
and mainnet presets in order to reduce the chance of compilation
errors when custom presets are used (previously, only the custom
presets have precisely assigned types for the constants).
2020-07-13 16:44:58 +02:00
Zahary Karadjov 74396747a3 Address review comments; Add a test 2020-07-12 21:01:31 +03:00
Zahary Karadjov 93b04bc214 Add an option for graffiti customization 2020-07-12 21:01:31 +03:00
tersec 853bd5b799
quick workaround for epochref cache issue (#1296)
* quick workaround for epochref cache issue

* disable assertion which doesn't work without epochref caches

* get local testnets and altona running again
2020-07-12 17:09:49 +02:00
Jacek Sieka d6f317950c
Compute state root instead of loading from database (#1297)
htr is fast now, so hitting the database to load the state root is no
longer motivated - the more simple code is less vulnerable to database
corruption.
2020-07-10 22:47:39 +02:00
Zahary Karadjov c06c60dcaf
Merge branch 'devel' into cli-reforms 2020-07-10 16:23:27 +03:00
tersec 5d5957fd7e
fix attestation handling bottlenecks in slot processing and quadratic-time inclusion delay rewards/palties in epoch processing; create infrastructure for dynamically (un)subscribing to/from attestation topics (#1294) 2020-07-10 09:24:04 +00:00
Zahary Karadjov 3ec6a02b12
Merge devel and resolve conflicts 2020-07-10 02:02:40 +03:00
tersec 61b0b5af17
update most remaining non-fork-choice spec refs, updating code where necessary (#1292)
* update most of the remaining non-fork-choice spec refs, updating code where necessary

* revert presumably harmless compute_signing_root() change, but this way, keep things really unchanged outside inspector
2020-07-09 11:43:27 +00:00
Mamy Ratsimbazafy 3cdae9f6be
Dual headed fork choice [Revolution] (#1238)
* Dual headed fork choice

* fix finalizedEpoch not moving

* reduce fork choice verbosity

* Add failing tests due to pruning

* Properly handle duplicate blocks in sync

* test_block_pool also add a test for duplicate blocks

* comments addressing review

* Fix fork choice v2, was missing integrating block proposed

* remove a spurious debug writeStackTrace

* update block_sim

* Use OrderedTable to ensure that we always load parents before children in fork choice

* Load the DAG data in fork choice at init if there is some (can sync witti)

* Cluster of quarantined blocks were not properly added to the fork choice

* Workaround async gcsafe warnings

* Update blockpoool tests

* Do the callback before clearing the quarantine

* Revert OrderedTable, implement topological sort of DAG, allow forkChoice to be initialized from arbitrary finalized heads

* Make it work with latest devel - Altona readyness

* Add a recovery mechanism when forkchoice desyncs with blockpool

* add the current problematic node to the stack

* Fix rebase indentation bug (but still producing invalid block)

* Fix cache at epoch boundaries and lateBlock addition
2020-07-09 11:29:32 +02:00
Dustin Brody 4140b3b9d9 update 29 spec refs to v0.12.1 2020-07-08 20:49:25 +00:00
Zahary Karadjov 318b225ccd
Merge devel and resolve the conflicts 2020-07-08 15:36:03 +03:00
Zahary Karadjov 5d056ba667
Switch to uint64 as the default type for preset values 2020-07-08 14:00:40 +03:00
Viktor Kirilov 1482b0430d - work towards more REST API endpoints being implemented
- testnets can now be launched with a separate validator client - make altona SCRIPT_PARAMS="--separateVC"
- reverted the ctrl+C signal handler code reuse - not necessary for the VC anyway (default is good enough)
- added a bit more logging in the VC
- removed unnecessary code in the VC - connect() just parses the address & port...
- fixed a couple more VC issues - when fetching the duties for an epoch fails on the BN side ==> the VC shouldn't be left in a broken state
- documented the currently supported json-rpc endpoints
- added more checks on the BN side for the API - bounds-checking the requests & also checking if the BN itself is synced
- other cleanup

currently a local sim doesn't finalize, but participation in the altona network with a separate VC is painless and works just as well as with in-process validators in a BN
2020-07-08 13:29:03 +03:00
Dustin Brody fc8502c54e halve memory usage from state caches 2020-07-08 10:21:41 +00:00
Dustin Brody 8ac613421d only calculate total_balance once per attestation delta application 2020-07-08 07:45:21 +00:00
Dustin Brody 3c030d945d allow 50k validators on block_sim by avoiding repeated get_total_active_balance() in get_proposer_reward() 2020-07-08 07:45:21 +00:00
Zahary Karadjov c4af4e2f35
Working test suite with run-time presets 2020-07-08 02:02:14 +03:00
Jacek Sieka 6fe0a623f5
Crypto rng (#1284)
* use bearssl rng throughout

* bump

* bump

* move keygen out of crypto
2020-07-07 17:51:02 +02:00
Zahary Karadjov 441ae9b58c
Introduce a RuntimePreset object for the overridable testnet preset values 2020-07-03 22:29:23 +03:00
tersec a92276d510
adopt Result[void, string] in place of some bool return signatures (#1275)
* adopt Result[void, string] in place of some bool return signatures

* string -> cstring to reduce memory allocations; ensure all err() strings are constants, with contextual information from higher-level callers

* logScope usage fixes

* homogenize err() reporting convention

* invalid signature in deposit isn't an error
2020-07-03 17:03:14 +00:00
tersec c64737e7f2
implement aggregated attestation receiving/validating (#1272)
* implement aggregated attestation receiving/validating

* document the conditions without explicit implementations in isValidAggregatedAttestation()
2020-07-02 16:15:27 +00:00
Jacek Sieka f3e92762e3
add tests for unviable blocks (#1271)
* add tests for unviable blocks

also enable finalization tests in all test configs - they're plenty fast
now
also fix newClone for non-rvo cases. sigh.

* fixes
2020-07-01 19:00:14 +02:00
tersec b4db2ad693
remove v0.11.3 support; add block_sim to CI (#1253)
* remove v0.11.3 support; add block_sim to CI

* rm stray PERSISTENT_COMMITTEE_PERIOD

* remove TopicFilter.InteropAttestations

* bump two comment-spec-refs to v0.12.1
2020-06-29 18:08:58 +00:00
Zahary Karadjov 8140b4458c Working local sim with WAIT_GENESIS=yes 2020-06-29 02:18:48 +03:00
Zahary Karadjov 31e31bb30c Switch to monitoring of Blocks; More accurate genesis detection and faster Eth1 syncing 2020-06-29 02:18:48 +03:00