Commit Graph

28 Commits

Author SHA1 Message Date
Jacek Sieka 43c64d32f8 increase attestation/aggregate queue sizes
when there are many validators, many aggregates and attestations arrive
every slot - increase the queue size a bit - also do batches on each
idle loop iteration since it's fairly quick
2021-01-26 14:29:47 +02:00
tersec 1bdbf099cc
use IntSet rather than HashSet[ValidatorIndex] (#2267)
* use IntSet rather than HashSet[ValidatorIndex]

* add bounds check before uint64 -> int conversion

* use intsets in block transitions

* remove superfluous Nim issue explanation/reference
2021-01-26 12:52:00 +01:00
Mamy Ratsimbazafy 70a03658e3
Block validation flow v2 + Batch (serial) sig verification (#2250)
* bump nim-blscurve

* Outline the block validation flow

* introduce the SigVerified types, pass the tests

* Split clearance/quarantine to prepare for batch crypto verif

* Add a batch signature collector

* Make clearance use SigVerified block and split verification between crypto and state transition

* Always use signedBeaconBlock for the onBlockAdded callback

* RANDAO signing_root is the epoch instead of the full block

* Support skipping BLS for testing

* Fix compilation of the validator client

* Try to fix strange errors MacOS and Jenkins (Clang, unknown type name br_hmac_drbg_context in stdlib_assertions.nim.c)

* address https://github.com/status-im/nimbus-eth2/pull/2250#discussion_r561819858

* address https://github.com/status-im/nimbus-eth2/pull/2250#discussion_r561828025

* onBlockAdded callback should use TrustedSignedBeaconBlock https://github.com/status-im/nimbus-eth2/pull/2250#discussion_r561837261

* address https://github.com/status-im/nimbus-eth2/pull/2250#discussion_r561828946

* Use the application RNG: https://github.com/status-im/nimbus-eth2/pull/2250#discussion_r561815336

* Improve codegen of conversion zero-cost)

* Quick fixes with loadWithCache after #2259 (TODO: graceful error since pubkey validations is now done first in signatures_batch)

* Graceful handle rogue pubkeys and signatures now that those are lazy-loaded
2021-01-25 20:45:48 +02:00
Jacek Sieka 8f6ce4f88d shorten idle timeout
this introduces the change more gently, to avoid adverse effects - the
timeout value can subsequently be tuned based on further
experimentation.
2021-01-25 19:44:05 +02:00
Dustin Brody a16f5afcd5 pre-emptive duplicate validator detection heuristic 2021-01-21 16:03:02 +02:00
Jacek Sieka c0e11cfba6 shorten idle timeout 2021-01-18 22:27:34 +02:00
Jacek Sieka 76f15302a7 better timesharing in eth2 processor
* use `idleAsync` to more evenly divide cpu attention when syncing in
particular - this gives networking better latency
* more strict exception handling in eth2_processor
2021-01-18 22:27:34 +02:00
tersec 97c4f7c5c0
fix subnet calculation in RPC and insert broadcast attestations into node's pool (#2207)
* fix subnet calculation in RPC and insert broadcast attestations into node's pool

* unify codepaths to ensure only mostly-checked-to-be-valid attestations enter the pool, even from node's own broadcasts

* update attestation pool tests for new validateAttestation param
2020-12-23 13:59:04 +01:00
tersec 8d1443f03c
detect already-aggregate-voted condition before attestation pool; add is_aggregator tests (#2170)
* detect already-aggregate-voted condition before attestation pool; add is_aggregator tests

* replace pair of attestation-per-epoch tracking lists with single list and remove Option use

* fix attestation condition

* use safer type conversions; add more is_aggregator tests
2020-12-14 20:58:32 +00:00
Jacek Sieka f10f29d829
avoid some futures and copies in processing pipeline (#2157)
`addLastNoWait` does the same thing with less task scheduling and
copying
2020-12-08 09:59:40 +01:00
Jacek Sieka e7f2735271
fix broken metrics during replay (#2090)
* move metrics out of state transition
* add validator count metric
* remove expensive beacon_current_validators, beacon_previous_validators
metrics (they should be reimplemented with cache), add cheap
beacon_active_validators to approximate
* remove unused validator count metrics
* tidy imports/defects
2020-11-27 23:16:13 +01:00
tersec 1d7fb2ed0c
remove {.inline.} pragmas (#2033)
* remove {.inline.} pragmas

* re-add inline on bitseqs functions and tweak inlining threshold

* remove macOS/LLVM inlining setting; revert non-init/module-local/tests inline pragma removal
2020-11-20 11:00:22 +01:00
tersec e2f161dbf7
fix attestation sending schedules to avoid timing attack (#1995) 2020-11-16 10:44:18 +01:00
cheatfate 87c92ea05f Address review comments.
Remove `-8.0` bucket because `-4.0` is a limit.
2020-11-12 21:52:01 +02:00
tersec 19c51e5900
use MAXIMUM_GOSSIP_CLOCK_DISPARITY for BeaconBlock verification (#1918) 2020-10-28 14:04:21 +01:00
Jacek Sieka ff3f8741de
handle fork choice selecting a head unknown to chaindag (#1907)
* move head slot metric to head selection
* avoid redundant head selection in validator duties
2020-10-28 07:55:36 +00:00
tersec a136c2e95a
bump libp2p; integrate pubsub.ValidationResult into extended validation (#1893) 2020-10-20 12:31:20 +00:00
tersec d69d89866e
fix logtrace in local network sim (#1840) 2020-10-09 10:58:54 +02:00
Mamy Ratsimbazafy 0280d6c73e
Revisiting log levels (#1788)
* Update log level - https://github.com/status-im/nim-beacon-chain/issues/1779 https://github.com/status-im/nim-beacon-chain/issues/1785

* Address review comments

* Document the logging strategy [skip ci]
2020-10-01 20:56:42 +02:00
tersec 6cf7e837ce
refactor exit pool and gossipsub-subscribe to exit pool message types (#1733)
* refactor exit pool and gossipsub-subscribe to all exit pool message types

* remove unused loop counter
2020-09-24 19:05:49 +02:00
tersec e106549efe
keep REJECT/IGNORE of messages failing validation for libp2p scoring (#1676)
* keep REJECT/IGNORE status of messages failing validation for libp2p scoring

* fix test suite
2020-09-18 13:53:09 +02:00
Jacek Sieka c76305f824
fix some todo (#1645)
* remove some superfluous gcsafes
* remove getTailState (unused)
* don't store old epochrefs in blocks
* document attestation pool a bit
* remove `pcs =` cruft from log
2020-09-14 14:50:03 +00:00
tersec 9f21bbd666
[WIP] skeleton of attester slashing pool & validators (#1639)
* skeleton of attester slashing pool & validators

* add skeleton for proposer slashings and voluntary exits; rename pool to more inclusive exit pool to stay consistent with all three; ensure is initialized by beacon_node so is safe to merge, even if it doesn't do much yet
2020-09-14 16:26:31 +02:00
tersec e4a43f7628
address issue #1580 (#1600)
* address issue #1580

* Update beacon_chain/spec/beaconstate.nim

Co-authored-by: Jacek Sieka <jacek@status.im>
2020-09-02 16:16:25 +00:00
tersec ab255662df
bound block quarantine size (#1564)
* bound block quarantine size

* add additional logging for block quarantining

* re-add quarantine.add() call

* remove pre-finalization blocks; add logging for full quarantine

* clear quarantine on chain reorganization

* update block_sim and tests

* update test_attestation_pool
2020-08-31 11:00:38 +02:00
Jacek Sieka fa1621db46
implement clock disparity for attestation validation (#1568)
This implements disparity, resolving a part of
https://github.com/status-im/nim-beacon-chain/issues/1367

* make BeaconTime a duration for fractional seconds
* factor out attestation/aggregate validation
* simplify recording of queued attestations
* simplify attestation signature check
* fix blocks_received metric
* add some trivial validation tests
* remove unresolved attestation table - attestations for unknown blocks
are dropped instead (cannot verify their signature)
2020-08-27 09:34:12 +02:00
Mamy Ratsimbazafy 81788becfc
Fork choice - almost free pruning - fix #1534 (#1535)
* initial - cheaper pruning - addresses  #1534

* Pass tests: update offset when pruning, proper handling of pruned parents

* Use options instead of nil for nilable newHead (finalization passing but rootcause not solved)

* First line of defense against stackoverflow in tests

* Fix compute_delta offset after pruning

* Rebase fix - medalla ready

* Remove Option[BlockRef]
2020-08-26 17:23:34 +02:00
Jacek Sieka 22998fdfd4 avoid double deserialization
When blocks and attestations arrive, they are SSZ-decoded twice: once
for validation and once for processing. This branch enqueues the decoded
block directly for processing, avoiding the second, slow
deserialization.

* move processing of blocks and attestations to queue
* ...and out from beacon_node
* split attestation processing into attestations and aggregates
  * also updates metrics
* clean up logging to better follow the lifetime of gossip: arrival,
validation and processing
* drop attestations and aggregates if there are too many
* try to prioritise blocks and aggregates before single-validator
attestations
2020-08-21 11:46:25 +03:00