734 Commits

Author SHA1 Message Date
tersec
611c5097cc
use cache in process_voluntary_exit() (#1507) 2020-08-14 12:42:59 +00:00
Jacek Sieka
a1bd44f4b0
small spec cleanups (#1501)
* clean up logging a bit
* return error on indexed attestation check
2020-08-13 13:47:06 +00:00
tersec
ab34584f23
initial dynamic subscribe/unsubscribe for attestations to/from subnets (#1462)
* initial dynamic subscribe/unsubscribe for attestations to/from subnets

* implement random stability subnet and clean up

* switch from HashSet[uint64] to set[uint8]

* refactor subnet logic out from beacon_node and actual (un)subscribing

* only try to subscribe to marginally different subnets

* add assertions

* maintain ENR subnets

* assert that beacon_node and eth2_network have consistent view of subscribed subnets

* disable actual cycling
2020-08-12 17:48:31 +00:00
tersec
af3355e0f8
create local testnet mode for eth2_network (#1494) 2020-08-12 14:16:59 +00:00
tersec
22c1ef5a8d
split subscribe into non-validating subscribe and addValidator (#1485)
* split subscribe into non-validating subscribe and addValidator

* stop exporting get_committee_assignments
2020-08-11 15:08:44 +00:00
Jacek Sieka
2b4526e743
bls: avoid exception flow on cache miss (#1479) 2020-08-10 14:51:23 +00:00
Jacek Sieka
84a501d1ff
remove one cache, add another (#1449)
* remove one cache, add another

This cache removes the need for rewinding in most attestation validation
flow since the attestations come from one of two epochs and must be
targetting a viable block.

Additionally, it also removes all state caches which are less likely to
be used over-all - more metrics are needed to track the rewinding.

On risk is that when chains don't finalize, we'll have lots of epochrefs
in memory meaning lots of validator key databases, most being exactly
the same. This can be addressed in any number of ways. Some of the
memory usage is mitigated by the fact that we previously had lots of big
state caches and now we're keeping only keys instead.

* cleanups

* doc
2020-08-06 19:48:47 +00:00
Zahary Karadjov
b902fddd19 Allow loading keystores produced by Lighthouse
The spec allows the description to be set to 'null'
2020-08-06 17:33:57 +03:00
tersec
81b3c0ea40
update spec refs to v0.12.2 (#1457)
* update spec refs to v0.12.2 and change a .len.uint64 to .lenu64

* pull back from any non-pure-comment changes, since Jenkins is being wonky
2020-08-06 13:05:13 +00:00
Zahary Karadjov
009e9a6a41 Fix 'make test' 2020-08-06 14:49:58 +03:00
protolambda
e90c5440e8 make eth1 distance runtime configurable 2020-08-06 14:49:58 +03:00
Dustin Brody
8618cbdf38 keep representations consistent for tests 2020-08-04 23:00:33 +03:00
Dustin Brody
e8c31e4ce8 just in case 2020-08-04 23:00:33 +03:00
Dustin Brody
5e3af9a458 fix validator cache keys 2020-08-04 23:00:33 +03:00
Dustin Brody
2c7d207a72 make test passes 2020-08-04 23:00:33 +03:00
Dustin Brody
9c65c50249 use case statement to avoid runtime doAssert 2020-08-04 23:00:33 +03:00
Dustin Brody
c142de4b7f be more consistent about pubkeys fed to verify_foo_signature() not being separately initialized, while pubkeys, generally, used for matching purposes, elsewhere explicitly initialized 2020-08-04 23:00:33 +03:00
Dustin Brody
db397cc5f3 lazily initialize validator public keys, which were a third or more of sync CPU usage 2020-08-04 23:00:33 +03:00
tersec
a979568d74
switch mapIt/find to in-place linear search (#1436) 2020-08-04 17:15:27 +00:00
Jacek Sieka
cf8cd8321b
Revert "Lazy crypto [alt design #1369] (#1371)" (#1435)
This reverts commit 023f7f4518fb8dd9cab2af5faba76b6975893dea.
2020-08-04 17:15:13 +00:00
Zahary Karadjov
4deed117a9
Hotfix: Allow 'make madella' to work with the latest Medalla metadata 2020-08-04 17:55:37 +03:00
tersec
df80071bcf
update attestation and block validation to v0.12.2; clean up getAncestorAt()/get_ancestor() (#1417)
* update attestation validation to v0.12.2; clean up getAncestorAt()/get_ancestor()

* update beacon block validation to v0.12.2
2020-08-03 19:47:42 +00:00
tersec
f2dd57db1e
increase allowed missing validators in verifyFinalization mode (#1414) 2020-08-03 05:31:35 +00:00
Zahary Karadjov
c882b7c2f3 Add Scrypt support in the Keystores 2020-08-02 23:00:43 +03:00
Zahary Karadjov
1aba7aed6d Updated Keystore test vectors 2020-08-02 23:00:43 +03:00
Zahary Karadjov
c293254ded Add 'deposits import' command; Switch to NJS when loading the keystores and improve the data validation 2020-08-02 23:00:43 +03:00
tersec
85febf072e
assert on missing active validators' attestations when verifying finalization (#1387)
* assert on missing > 10 supposed-to-be-active validators' attestations in finalization verification mode

* clean up state_sim imports
2020-08-01 18:24:25 +00:00
tersec
e0a6f58abe
convert 10 v0.12.1 spec refs to v0.12.2 (#1406) 2020-07-31 09:59:14 +00:00
Viktor Kirilov
c032366547
removed the BlockPool type and all of the proxy functions around it (#1401)
* removed the BlockPool type and all of the proxy functions around it - passing the chain DAG and the quarantine explicitly where appropriately - they don't need to be bundled in a type

* fixed the build after the rebase
2020-07-30 21:18:17 +02:00
tersec
254a510c0d
mark .borrow. procs as noSideEffect (#1402) 2020-07-30 14:47:59 +02:00
tersec
99f2d8e06c
update 14 v0.12.1 spec refs to v0.12.2 (#1400) 2020-07-30 09:47:57 +00:00
Mamy Ratsimbazafy
023f7f4518
Lazy crypto [alt design #1369] (#1371)
* Lazy loading of crypto objects

* Try to fix incorrect field access by hiding fields but no luck. SSZ/Chronicles/macro bug?

* Fix incorrect blsValue access. was "aggregate" not "chronicles"

* Fix tests that rely on the internal BLSValue representation
2020-07-29 18:13:05 +00:00
tersec
d97cc35d30
switch 14 v0.12.1 spec refs to v0.12.2 spec refs (#1395) 2020-07-29 12:47:03 +00:00
tersec
a07dab300c
verifyFinalization checks for justification being consistent with later finalization (#1386) 2020-07-28 17:35:32 +00:00
tersec
b45de824a4
convert 22 v0.12.1 spec refs to v0.12.2 and remove 1 unnecessary spec ref (#1384) 2020-07-28 14:55:46 +00:00
Jacek Sieka
157ddd2ac4
Fork choice fixes 5 (#1381)
* limit attestations kept in attestation pool

With fork choice updated, the attestation pool only needs to keep track
of attestations that will eventually end up in blocks - we can thus
limit the horizon of attestations that we keep more aggressively.

To get here, we expose getEpochRef which gets metadata about a
particular epochref, and make sure to populate it when a block is added
- this ensures that state rewinds during block addition are minimized.

In addition, we'll use the target root/epoch when validating
attestations - this helps minimize the number of different states that
we need to rewind to, in general.

* remove CandidateChains.justifiedState

unused

* remove BlockPools.Head object

* avoid quadratic quarantine loop

* fix
2020-07-28 13:54:32 +00:00
Zahary Karadjov
40ea9e9886 Integrate the Wallet support with the Deposits creation; Produce Launchpad-compatible deposits 2020-07-28 07:36:25 +03:00
Zahary Karadjov
fcd412f7a1 Finish the 'create wallet' command; Addresses #1319 2020-07-28 07:36:25 +03:00
tersec
dad3dd5809
update 111 v0.12.1 beacon chain spec refs to v0.12.2 (#1380) 2020-07-27 18:49:46 +02:00
Jacek Sieka
48cebc7157
spec cleanups (#1379)
* add helper for beacon committee length (used for quickly validating
attestations)
* refactor some attestation checks to do cheap checks early
* validate attestation epoch before computing active validator set
* clean up documentation / comments
* fill state cache on demand
2020-07-27 16:04:44 +00:00
tersec
20a2525390
v0.12.2 beacon chain protocol update (#1378) 2020-07-27 12:59:57 +02:00
tersec
90708a8287
Prefer converting int` to uint64 and switch foo.len.uint64 to .len64 (#1375)
* avoid converting from uint64 to int, and where most feasible, int type conversion at all

* .len.uint64 -> .len64

* fix 32-bit compilation

* try keeping state_sim loop variable/bounds as int for 32-bit Azure

* len64 -> lenu64
2020-07-26 20:55:48 +02:00
Mamy Ratsimbazafy
e5a76b0047
Stopgap don't print private key in shortlog (#1372) 2020-07-26 10:27:11 +02:00
Jacek Sieka
e0a18a3105
cache beacon committee size calculation (#1363)
* cache beacon committee size calculation

this fixes a bug in get_validator_churn_limit as well

* fix

* make committee counts consistently uint64

mixing feels like the worst of the two worlds
2020-07-23 19:01:07 +02:00
Jacek Sieka
fb2f742972
Fork choice fixes 2 (#1356)
* fork choice cleanup

* enable v2 pruning
* prefer `get_current_epoch`
* fix finalization check to use correct epoch

* small cleanups

* add `count_active_validators`
* remove misleading logs
* fix justified checkpoint slot calculation in rpc
2020-07-22 23:01:44 +02:00
tersec
6b77f3dda5
update compute_subnet_for_attestation() to use https://github.com/ethereum/eth2.0-specs/pull/1876 signature, which isn't in v0.12.1, which works with lookahead (#1346) 2020-07-22 08:04:21 +00:00
tersec
4a9a7be271
faster syncing (#1348)
* maybe faster syncing

* 80-character lines

* remove instrumentation debugEchos; fix target attestation epoch in attestation pool validation

* use the epoch-granularity matching in attestation.addResolved(...)
2020-07-22 09:51:45 +02:00
tersec
83abbcb917
drop get_attesting_indices()/get_unslashed_attesting_indices() from 15% to 1% of workload at block_sim at 100k validators (#1351) 2020-07-21 18:35:43 +02:00
Jacek Sieka
8b01284b0e
cache block hash (#1329)
hash_tree_root was turning up when running beacon_node, turns out to be
repeated hash_tree_root invocations - this pr brings them back down to
normal.

this PR caches the root of a block in the SignedBeaconBlock object -
this has the potential downside that even invalid blocks will be hashed
(as part of deserialization) - later, one could imagine delaying this
until checks have passed

there's also some cleanup of the `cat=` logs which were applied randomly
and haphazardly, and to a large degree are duplicated by other
information in the log statements - in particular, topics fulfill the
same role
2020-07-16 15:16:51 +02:00
Viktor Kirilov
4767eba465 - properly distributing the keys between BNs & VCs - fixing finalization - will switch to ON by default as soon as finalization becomes overall stable
- added a new altona-vc Makefile target which uses a separate VC process
2020-07-15 16:18:21 +03:00