status-im
/
nimbus-eth2
mirror of https://github.com/status-im/nimbus-eth2.git
2
0
Fork 0
Code Issues Projects Releases Wiki Activity
nimbus-eth2/tests/test_block_pool.nim

405 lines
12 KiB
Nim
Raw Normal View History

add initial block pool (#139) * implement in-memory block graph * store tail block in database * resolve unknown parents by syncing them from peers * introduce concept of resolved blocks and attestations - those that follow minimal protocol rules * update state head lazily * log more stuff * shortHash -> shortLog * start 9/10 beacon nodes by default, last can be started manually * see also #134 * fix start.sh epoch length
2019-02-28 21:21:29 +00:00
# beacon_chain
remove all but one UnusedImport warning (#881) * remove all but one UnusedImport warning * bump a few more spec version references from v0.10.1 to v0.11.1 * more v0.10.1 spec reference updates/removals * yet more v0.10.1 spec reference updates
2020-04-10 14:06:24 +00:00
# Copyright (c) 2018-2020 Status Research & Development GmbH
add initial block pool (#139) * implement in-memory block graph * store tail block in database * resolve unknown parents by syncing them from peers * introduce concept of resolved blocks and attestations - those that follow minimal protocol rules * update state head lazily * log more stuff * shortHash -> shortLog * start 9/10 beacon nodes by default, last can be started manually * see also #134 * fix start.sh epoch length
2019-02-28 21:21:29 +00:00
# Licensed and distributed under either of
replace links to Apache and MIT licenses with HTTPS versions (#592)
2019-11-25 15:30:02 +00:00
# * MIT license (license terms in the root directory or at https://opensource.org/licenses/MIT).
# * Apache v2 license (license terms in the root directory or at https://www.apache.org/licenses/LICENSE-2.0).
add initial block pool (#139) * implement in-memory block graph * store tail block in database * resolve unknown parents by syncing them from peers * introduce concept of resolved blocks and attestations - those that follow minimal protocol rules * update state head lazily * log more stuff * shortHash -> shortLog * start 9/10 beacon nodes by default, last can be started manually * see also #134 * fix start.sh epoch length
2019-02-28 21:21:29 +00:00
# at your option. This file may not be copied, modified, or distributed except according to those terms.
remove UnusedImport spam (#549) * remove UnusedImport spam * re-add random import to beacon_node
2019-11-14 10:47:55 +00:00
{.used.}
add initial block pool (#139) * implement in-memory block graph * store tail block in database * resolve unknown parents by syncing them from peers * introduce concept of resolved blocks and attestations - those that follow minimal protocol rules * update state head lazily * log more stuff * shortHash -> shortLog * start 9/10 beacon nodes by default, last can be started manually * see also #134 * fix start.sh epoch length
2019-02-28 21:21:29 +00:00
import
bump submodules (#1043)
2020-05-20 04:57:39 +00:00
options, sequtils, unittest,
re-enable all but the finalization tests for block pool in mainnet presets
2020-05-05 09:18:44 +00:00
./testutil, ./testblockutil,
restore EpochRef and flush statecaches on epoch transitions (#1312) * restore EpochRef and flush statecaches on epoch transitions * more targeted cache invalidation * remove get_empty_per_epoch_cache(); implement simpler but still faster get_beacon_proposer_index()/compute_proposer_index() approach; add some abstraction layer for accessing the shuffled validator indices cache * reduce integer type conversions * remove most of rest of integer type conversion in compute_proposer_index()
2020-07-15 10:44:18 +00:00
../beacon_chain/spec/[datatypes, digest, helpers, state_transition, presets],
state_transition implements the spec fairly directly (#1220)
2020-06-23 13:54:24 +00:00
../beacon_chain/[beacon_node_types, block_pool, ssz]
add initial block pool (#139) * implement in-memory block graph * store tail block in database * resolve unknown parents by syncing them from peers * introduce concept of resolved blocks and attestations - those that follow minimal protocol rules * update state head lazily * log more stuff * shortHash -> shortLog * start 9/10 beacon nodes by default, last can be started manually * see also #134 * fix start.sh epoch length
2019-02-28 21:21:29 +00:00
Small fixes (#1165) * random fixes * create dump dir on startup * don't crash on failure to write dump * fix a few `uint64` instances being used when indexing arrays - this should be a compile error but isn't due to compiler bugs * fix standalone test_block_pool compilation * add signed block processing in ncli * reuse cache entry instead of allocating a new one * allow for small clock disparities when validating blocks
2020-06-12 16:43:20 +00:00
when isMainModule:
import chronicles # or some random compile error happens...
Generate markdown test reports
2020-03-10 04:00:19 +00:00
suiteReport "BlockRef and helpers" & preset():
slot fixes * fix slot time navigation, add tests * skip block proposal if head is more recent already - shouldn't happen * use correct head when attesting to previous blocks * log slot start/end processing
2019-12-23 15:34:09 +00:00
timedTest "isAncestorOf sanity" & preset():
let
s0 = BlockRef(slot: Slot(0))
s1 = BlockRef(slot: Slot(1), parent: s0)
s2 = BlockRef(slot: Slot(2), parent: s1)
check:
s0.isAncestorOf(s0)
s0.isAncestorOf(s1)
s0.isAncestorOf(s2)
s1.isAncestorOf(s1)
s1.isAncestorOf(s2)
not s2.isAncestorOf(s0)
not s2.isAncestorOf(s1)
not s1.isAncestorOf(s0)
timedTest "getAncestorAt sanity" & preset():
let
s0 = BlockRef(slot: Slot(0))
s1 = BlockRef(slot: Slot(1), parent: s0)
s2 = BlockRef(slot: Slot(2), parent: s1)
s4 = BlockRef(slot: Slot(4), parent: s2)
check:
s0.getAncestorAt(Slot(0)) == s0
s0.getAncestorAt(Slot(1)) == s0
s1.getAncestorAt(Slot(0)) == s0
s1.getAncestorAt(Slot(1)) == s1
s4.getAncestorAt(Slot(0)) == s0
s4.getAncestorAt(Slot(1)) == s1
s4.getAncestorAt(Slot(2)) == s2
s4.getAncestorAt(Slot(3)) == s2
s4.getAncestorAt(Slot(4)) == s4
Generate markdown test reports
2020-03-10 04:00:19 +00:00
suiteReport "BlockSlot and helpers" & preset():
slot fixes * fix slot time navigation, add tests * skip block proposal if head is more recent already - shouldn't happen * use correct head when attesting to previous blocks * log slot start/end processing
2019-12-23 15:34:09 +00:00
timedTest "atSlot sanity" & preset():
let
s0 = BlockRef(slot: Slot(0))
s1 = BlockRef(slot: Slot(1), parent: s0)
s2 = BlockRef(slot: Slot(2), parent: s1)
s4 = BlockRef(slot: Slot(4), parent: s2)
check:
s0.atSlot(Slot(0)).blck == s0
s0.atSlot(Slot(0)) == s1.atSlot(Slot(0))
s1.atSlot(Slot(1)).blck == s1
s4.atSlot(Slot(0)).blck == s0
timedTest "parent sanity" & preset():
let
s0 = BlockRef(slot: Slot(0))
s00 = BlockSlot(blck: s0, slot: Slot(0))
s01 = BlockSlot(blck: s0, slot: Slot(1))
s2 = BlockRef(slot: Slot(2), parent: s0)
s22 = BlockSlot(blck: s2, slot: Slot(2))
s24 = BlockSlot(blck: s2, slot: Slot(4))
check:
s00.parent == BlockSlot(blck: nil, slot: Slot(0))
s01.parent == s00
s22.parent == s01
s24.parent == BlockSlot(blck: s2, slot: Slot(3))
s24.parent.parent == s22
re-enable all but the finalization tests for block pool in mainnet presets
2020-05-05 09:18:44 +00:00
suiteReport "Block pool processing" & preset():
setup:
var
db = makeTestDB(SLOTS_PER_EPOCH)
Working test suite with run-time presets
2020-07-07 23:02:14 +00:00
pool = BlockPool.init(defaultRuntimePreset, db)
re-enable all but the finalization tests for block pool in mainnet presets
2020-05-05 09:18:44 +00:00
stateData = newClone(pool.loadTailState())
restore EpochRef and flush statecaches on epoch transitions (#1312) * restore EpochRef and flush statecaches on epoch transitions * more targeted cache invalidation * remove get_empty_per_epoch_cache(); implement simpler but still faster get_beacon_proposer_index()/compute_proposer_index() approach; add some abstraction layer for accessing the shuffled validator indices cache * reduce integer type conversions * remove most of rest of integer type conversion in compute_proposer_index()
2020-07-15 10:44:18 +00:00
cache = StateCache()
12x speedup on state sim with 100k validators sans BLS by caching get_beacon_proposer_index(...)
2020-06-04 12:03:16 +00:00
b1 = addTestBlock(stateData.data, pool.tail.root, cache)
re-enable all but the finalization tests for block pool in mainnet presets
2020-05-05 09:18:44 +00:00
b1Root = hash_tree_root(b1.message)
12x speedup on state sim with 100k validators sans BLS by caching get_beacon_proposer_index(...)
2020-06-04 12:03:16 +00:00
b2 = addTestBlock(stateData.data, b1Root, cache)
re-enable all but the finalization tests for block pool in mainnet presets
2020-05-05 09:18:44 +00:00
b2Root {.used.} = hash_tree_root(b2.message)
timedTest "getRef returns nil for missing blocks":
check:
pool.getRef(default Eth2Digest) == nil
remove all but one UnusedImport warning (#881) * remove all but one UnusedImport warning * bump a few more spec version references from v0.10.1 to v0.11.1 * more v0.10.1 spec reference updates/removals * yet more v0.10.1 spec reference updates
2020-04-10 14:06:24 +00:00
re-enable all but the finalization tests for block pool in mainnet presets
2020-05-05 09:18:44 +00:00
timedTest "loadTailState gets genesis block on first load" & preset():
let
b0 = pool.get(pool.tail.root)
disable block/attestation pool tests in mainnet
2019-12-19 14:13:35 +00:00
re-enable all but the finalization tests for block pool in mainnet presets
2020-05-05 09:18:44 +00:00
check:
b0.isSome()
default preset update; rm process_slots(BeaconState); some BLS skipping disabled (#963) * re-enable BLS validation for block sanity tests * update default preset to mainnet; remove process_slots(state: var BeaconState) * bump spec ref to v0.11.1 * bump all v0.11.1 spec references except the actually changed is_valid_indexed_attestation(...) in spec/beaconstate to v0.11.2 * remove stack-risky/low-ROI doAssert
2020-05-04 21:07:18 +00:00
re-enable all but the finalization tests for block pool in mainnet presets
2020-05-05 09:18:44 +00:00
timedTest "Simple block add&get" & preset():
let
cache block hash (#1329) hash_tree_root was turning up when running beacon_node, turns out to be repeated hash_tree_root invocations - this pr brings them back down to normal. this PR caches the root of a block in the SignedBeaconBlock object - this has the potential downside that even invalid blocks will be hashed (as part of deserialization) - later, one could imagine delaying this until checks have passed there's also some cleanup of the `cat=` logs which were applied randomly and haphazardly, and to a large degree are duplicated by other information in the log statements - in particular, topics fulfill the same role
2020-07-16 13:16:51 +00:00
b1Add = pool.addRawBlock(b1) do (validBlock: BlockRef):
Dual headed fork choice [Revolution] (#1238) * Dual headed fork choice * fix finalizedEpoch not moving * reduce fork choice verbosity * Add failing tests due to pruning * Properly handle duplicate blocks in sync * test_block_pool also add a test for duplicate blocks * comments addressing review * Fix fork choice v2, was missing integrating block proposed * remove a spurious debug writeStackTrace * update block_sim * Use OrderedTable to ensure that we always load parents before children in fork choice * Load the DAG data in fork choice at init if there is some (can sync witti) * Cluster of quarantined blocks were not properly added to the fork choice * Workaround async gcsafe warnings * Update blockpoool tests * Do the callback before clearing the quarantine * Revert OrderedTable, implement topological sort of DAG, allow forkChoice to be initialized from arbitrary finalized heads * Make it work with latest devel - Altona readyness * Add a recovery mechanism when forkchoice desyncs with blockpool * add the current problematic node to the stack * Fix rebase indentation bug (but still producing invalid block) * Fix cache at epoch boundaries and lateBlock addition
2020-07-09 09:29:32 +00:00
discard
cache block hash (#1329) hash_tree_root was turning up when running beacon_node, turns out to be repeated hash_tree_root invocations - this pr brings them back down to normal. this PR caches the root of a block in the SignedBeaconBlock object - this has the potential downside that even invalid blocks will be hashed (as part of deserialization) - later, one could imagine delaying this until checks have passed there's also some cleanup of the `cat=` logs which were applied randomly and haphazardly, and to a large degree are duplicated by other information in the log statements - in particular, topics fulfill the same role
2020-07-16 13:16:51 +00:00
b1Get = pool.get(b1.root)
disable block/attestation pool tests in mainnet
2019-12-19 14:13:35 +00:00
re-enable all but the finalization tests for block pool in mainnet presets
2020-05-05 09:18:44 +00:00
check:
b1Get.isSome()
b1Get.get().refs.root == b1Root
Dual headed fork choice [Revolution] (#1238) * Dual headed fork choice * fix finalizedEpoch not moving * reduce fork choice verbosity * Add failing tests due to pruning * Properly handle duplicate blocks in sync * test_block_pool also add a test for duplicate blocks * comments addressing review * Fix fork choice v2, was missing integrating block proposed * remove a spurious debug writeStackTrace * update block_sim * Use OrderedTable to ensure that we always load parents before children in fork choice * Load the DAG data in fork choice at init if there is some (can sync witti) * Cluster of quarantined blocks were not properly added to the fork choice * Workaround async gcsafe warnings * Update blockpoool tests * Do the callback before clearing the quarantine * Revert OrderedTable, implement topological sort of DAG, allow forkChoice to be initialized from arbitrary finalized heads * Make it work with latest devel - Altona readyness * Add a recovery mechanism when forkchoice desyncs with blockpool * add the current problematic node to the stack * Fix rebase indentation bug (but still producing invalid block) * Fix cache at epoch boundaries and lateBlock addition
2020-07-09 09:29:32 +00:00
b1Add[].root == b1Get.get().refs.root
re-enable all but the finalization tests for block pool in mainnet presets
2020-05-05 09:18:44 +00:00
pool.heads.len == 1
Dual headed fork choice [Revolution] (#1238) * Dual headed fork choice * fix finalizedEpoch not moving * reduce fork choice verbosity * Add failing tests due to pruning * Properly handle duplicate blocks in sync * test_block_pool also add a test for duplicate blocks * comments addressing review * Fix fork choice v2, was missing integrating block proposed * remove a spurious debug writeStackTrace * update block_sim * Use OrderedTable to ensure that we always load parents before children in fork choice * Load the DAG data in fork choice at init if there is some (can sync witti) * Cluster of quarantined blocks were not properly added to the fork choice * Workaround async gcsafe warnings * Update blockpoool tests * Do the callback before clearing the quarantine * Revert OrderedTable, implement topological sort of DAG, allow forkChoice to be initialized from arbitrary finalized heads * Make it work with latest devel - Altona readyness * Add a recovery mechanism when forkchoice desyncs with blockpool * add the current problematic node to the stack * Fix rebase indentation bug (but still producing invalid block) * Fix cache at epoch boundaries and lateBlock addition
2020-07-09 09:29:32 +00:00
pool.heads[0].blck == b1Add[]
disable block/attestation pool tests in mainnet
2019-12-19 14:13:35 +00:00
re-enable all but the finalization tests for block pool in mainnet presets
2020-05-05 09:18:44 +00:00
let
cache block hash (#1329) hash_tree_root was turning up when running beacon_node, turns out to be repeated hash_tree_root invocations - this pr brings them back down to normal. this PR caches the root of a block in the SignedBeaconBlock object - this has the potential downside that even invalid blocks will be hashed (as part of deserialization) - later, one could imagine delaying this until checks have passed there's also some cleanup of the `cat=` logs which were applied randomly and haphazardly, and to a large degree are duplicated by other information in the log statements - in particular, topics fulfill the same role
2020-07-16 13:16:51 +00:00
b2Add = pool.addRawBlock(b2) do (validBlock: BlockRef):
Dual headed fork choice [Revolution] (#1238) * Dual headed fork choice * fix finalizedEpoch not moving * reduce fork choice verbosity * Add failing tests due to pruning * Properly handle duplicate blocks in sync * test_block_pool also add a test for duplicate blocks * comments addressing review * Fix fork choice v2, was missing integrating block proposed * remove a spurious debug writeStackTrace * update block_sim * Use OrderedTable to ensure that we always load parents before children in fork choice * Load the DAG data in fork choice at init if there is some (can sync witti) * Cluster of quarantined blocks were not properly added to the fork choice * Workaround async gcsafe warnings * Update blockpoool tests * Do the callback before clearing the quarantine * Revert OrderedTable, implement topological sort of DAG, allow forkChoice to be initialized from arbitrary finalized heads * Make it work with latest devel - Altona readyness * Add a recovery mechanism when forkchoice desyncs with blockpool * add the current problematic node to the stack * Fix rebase indentation bug (but still producing invalid block) * Fix cache at epoch boundaries and lateBlock addition
2020-07-09 09:29:32 +00:00
discard
cache block hash (#1329) hash_tree_root was turning up when running beacon_node, turns out to be repeated hash_tree_root invocations - this pr brings them back down to normal. this PR caches the root of a block in the SignedBeaconBlock object - this has the potential downside that even invalid blocks will be hashed (as part of deserialization) - later, one could imagine delaying this until checks have passed there's also some cleanup of the `cat=` logs which were applied randomly and haphazardly, and to a large degree are duplicated by other information in the log statements - in particular, topics fulfill the same role
2020-07-16 13:16:51 +00:00
b2Get = pool.get(b2.root)
disable block/attestation pool tests in mainnet
2019-12-19 14:13:35 +00:00
re-enable all but the finalization tests for block pool in mainnet presets
2020-05-05 09:18:44 +00:00
check:
b2Get.isSome()
cache block hash (#1329) hash_tree_root was turning up when running beacon_node, turns out to be repeated hash_tree_root invocations - this pr brings them back down to normal. this PR caches the root of a block in the SignedBeaconBlock object - this has the potential downside that even invalid blocks will be hashed (as part of deserialization) - later, one could imagine delaying this until checks have passed there's also some cleanup of the `cat=` logs which were applied randomly and haphazardly, and to a large degree are duplicated by other information in the log statements - in particular, topics fulfill the same role
2020-07-16 13:16:51 +00:00
b2Get.get().refs.root == b2.root
Dual headed fork choice [Revolution] (#1238) * Dual headed fork choice * fix finalizedEpoch not moving * reduce fork choice verbosity * Add failing tests due to pruning * Properly handle duplicate blocks in sync * test_block_pool also add a test for duplicate blocks * comments addressing review * Fix fork choice v2, was missing integrating block proposed * remove a spurious debug writeStackTrace * update block_sim * Use OrderedTable to ensure that we always load parents before children in fork choice * Load the DAG data in fork choice at init if there is some (can sync witti) * Cluster of quarantined blocks were not properly added to the fork choice * Workaround async gcsafe warnings * Update blockpoool tests * Do the callback before clearing the quarantine * Revert OrderedTable, implement topological sort of DAG, allow forkChoice to be initialized from arbitrary finalized heads * Make it work with latest devel - Altona readyness * Add a recovery mechanism when forkchoice desyncs with blockpool * add the current problematic node to the stack * Fix rebase indentation bug (but still producing invalid block) * Fix cache at epoch boundaries and lateBlock addition
2020-07-09 09:29:32 +00:00
b2Add[].root == b2Get.get().refs.root
re-enable all but the finalization tests for block pool in mainnet presets
2020-05-05 09:18:44 +00:00
pool.heads.len == 1
Dual headed fork choice [Revolution] (#1238) * Dual headed fork choice * fix finalizedEpoch not moving * reduce fork choice verbosity * Add failing tests due to pruning * Properly handle duplicate blocks in sync * test_block_pool also add a test for duplicate blocks * comments addressing review * Fix fork choice v2, was missing integrating block proposed * remove a spurious debug writeStackTrace * update block_sim * Use OrderedTable to ensure that we always load parents before children in fork choice * Load the DAG data in fork choice at init if there is some (can sync witti) * Cluster of quarantined blocks were not properly added to the fork choice * Workaround async gcsafe warnings * Update blockpoool tests * Do the callback before clearing the quarantine * Revert OrderedTable, implement topological sort of DAG, allow forkChoice to be initialized from arbitrary finalized heads * Make it work with latest devel - Altona readyness * Add a recovery mechanism when forkchoice desyncs with blockpool * add the current problematic node to the stack * Fix rebase indentation bug (but still producing invalid block) * Fix cache at epoch boundaries and lateBlock addition
2020-07-09 09:29:32 +00:00
pool.heads[0].blck == b2Add[]
disable block/attestation pool tests in mainnet
2019-12-19 14:13:35 +00:00
re-enable all but the finalization tests for block pool in mainnet presets
2020-05-05 09:18:44 +00:00
# Skip one slot to get a gap
check that process_slots() succeeds
2020-05-19 15:46:29 +00:00
check:
process_slots(stateData.data, stateData.data.data.slot + 1)
disable block/attestation pool tests in mainnet
2019-12-19 14:13:35 +00:00
re-enable all but the finalization tests for block pool in mainnet presets
2020-05-05 09:18:44 +00:00
let
cache block hash (#1329) hash_tree_root was turning up when running beacon_node, turns out to be repeated hash_tree_root invocations - this pr brings them back down to normal. this PR caches the root of a block in the SignedBeaconBlock object - this has the potential downside that even invalid blocks will be hashed (as part of deserialization) - later, one could imagine delaying this until checks have passed there's also some cleanup of the `cat=` logs which were applied randomly and haphazardly, and to a large degree are duplicated by other information in the log statements - in particular, topics fulfill the same role
2020-07-16 13:16:51 +00:00
b4 = addTestBlock(stateData.data, b2.root, cache)
b4Add = pool.addRawBlock(b4) do (validBlock: BlockRef):
Dual headed fork choice [Revolution] (#1238) * Dual headed fork choice * fix finalizedEpoch not moving * reduce fork choice verbosity * Add failing tests due to pruning * Properly handle duplicate blocks in sync * test_block_pool also add a test for duplicate blocks * comments addressing review * Fix fork choice v2, was missing integrating block proposed * remove a spurious debug writeStackTrace * update block_sim * Use OrderedTable to ensure that we always load parents before children in fork choice * Load the DAG data in fork choice at init if there is some (can sync witti) * Cluster of quarantined blocks were not properly added to the fork choice * Workaround async gcsafe warnings * Update blockpoool tests * Do the callback before clearing the quarantine * Revert OrderedTable, implement topological sort of DAG, allow forkChoice to be initialized from arbitrary finalized heads * Make it work with latest devel - Altona readyness * Add a recovery mechanism when forkchoice desyncs with blockpool * add the current problematic node to the stack * Fix rebase indentation bug (but still producing invalid block) * Fix cache at epoch boundaries and lateBlock addition
2020-07-09 09:29:32 +00:00
discard
disable block/attestation pool tests in mainnet
2019-12-19 14:13:35 +00:00
re-enable all but the finalization tests for block pool in mainnet presets
2020-05-05 09:18:44 +00:00
check:
Dual headed fork choice [Revolution] (#1238) * Dual headed fork choice * fix finalizedEpoch not moving * reduce fork choice verbosity * Add failing tests due to pruning * Properly handle duplicate blocks in sync * test_block_pool also add a test for duplicate blocks * comments addressing review * Fix fork choice v2, was missing integrating block proposed * remove a spurious debug writeStackTrace * update block_sim * Use OrderedTable to ensure that we always load parents before children in fork choice * Load the DAG data in fork choice at init if there is some (can sync witti) * Cluster of quarantined blocks were not properly added to the fork choice * Workaround async gcsafe warnings * Update blockpoool tests * Do the callback before clearing the quarantine * Revert OrderedTable, implement topological sort of DAG, allow forkChoice to be initialized from arbitrary finalized heads * Make it work with latest devel - Altona readyness * Add a recovery mechanism when forkchoice desyncs with blockpool * add the current problematic node to the stack * Fix rebase indentation bug (but still producing invalid block) * Fix cache at epoch boundaries and lateBlock addition
2020-07-09 09:29:32 +00:00
b4Add[].parent == b2Add[]
disable block/attestation pool tests in mainnet
2019-12-19 14:13:35 +00:00
Dual headed fork choice [Revolution] (#1238) * Dual headed fork choice * fix finalizedEpoch not moving * reduce fork choice verbosity * Add failing tests due to pruning * Properly handle duplicate blocks in sync * test_block_pool also add a test for duplicate blocks * comments addressing review * Fix fork choice v2, was missing integrating block proposed * remove a spurious debug writeStackTrace * update block_sim * Use OrderedTable to ensure that we always load parents before children in fork choice * Load the DAG data in fork choice at init if there is some (can sync witti) * Cluster of quarantined blocks were not properly added to the fork choice * Workaround async gcsafe warnings * Update blockpoool tests * Do the callback before clearing the quarantine * Revert OrderedTable, implement topological sort of DAG, allow forkChoice to be initialized from arbitrary finalized heads * Make it work with latest devel - Altona readyness * Add a recovery mechanism when forkchoice desyncs with blockpool * add the current problematic node to the stack * Fix rebase indentation bug (but still producing invalid block) * Fix cache at epoch boundaries and lateBlock addition
2020-07-09 09:29:32 +00:00
pool.updateHead(b4Add[])
more block range request updates * handle skip == 0 gracefully * avoid memory allocation at expense of more complex API * add more tests * log block request results
2020-04-21 06:43:39 +00:00
re-enable all but the finalization tests for block pool in mainnet presets
2020-05-05 09:18:44 +00:00
var blocks: array[3, BlockRef]
more block range request updates * handle skip == 0 gracefully * avoid memory allocation at expense of more complex API * add more tests * log block request results
2020-04-21 06:43:39 +00:00
re-enable all but the finalization tests for block pool in mainnet presets
2020-05-05 09:18:44 +00:00
check:
pool.getBlockRange(Slot(0), 1, blocks.toOpenArray(0, 0)) == 0
blocks[0..<1] == [pool.tail]
more block range request updates * handle skip == 0 gracefully * avoid memory allocation at expense of more complex API * add more tests * log block request results
2020-04-21 06:43:39 +00:00
re-enable all but the finalization tests for block pool in mainnet presets
2020-05-05 09:18:44 +00:00
pool.getBlockRange(Slot(0), 1, blocks.toOpenArray(0, 1)) == 0
Dual headed fork choice [Revolution] (#1238) * Dual headed fork choice * fix finalizedEpoch not moving * reduce fork choice verbosity * Add failing tests due to pruning * Properly handle duplicate blocks in sync * test_block_pool also add a test for duplicate blocks * comments addressing review * Fix fork choice v2, was missing integrating block proposed * remove a spurious debug writeStackTrace * update block_sim * Use OrderedTable to ensure that we always load parents before children in fork choice * Load the DAG data in fork choice at init if there is some (can sync witti) * Cluster of quarantined blocks were not properly added to the fork choice * Workaround async gcsafe warnings * Update blockpoool tests * Do the callback before clearing the quarantine * Revert OrderedTable, implement topological sort of DAG, allow forkChoice to be initialized from arbitrary finalized heads * Make it work with latest devel - Altona readyness * Add a recovery mechanism when forkchoice desyncs with blockpool * add the current problematic node to the stack * Fix rebase indentation bug (but still producing invalid block) * Fix cache at epoch boundaries and lateBlock addition
2020-07-09 09:29:32 +00:00
blocks[0..<2] == [pool.tail, b1Add[]]
more block range request updates * handle skip == 0 gracefully * avoid memory allocation at expense of more complex API * add more tests * log block request results
2020-04-21 06:43:39 +00:00
re-enable all but the finalization tests for block pool in mainnet presets
2020-05-05 09:18:44 +00:00
pool.getBlockRange(Slot(0), 2, blocks.toOpenArray(0, 1)) == 0
Dual headed fork choice [Revolution] (#1238) * Dual headed fork choice * fix finalizedEpoch not moving * reduce fork choice verbosity * Add failing tests due to pruning * Properly handle duplicate blocks in sync * test_block_pool also add a test for duplicate blocks * comments addressing review * Fix fork choice v2, was missing integrating block proposed * remove a spurious debug writeStackTrace * update block_sim * Use OrderedTable to ensure that we always load parents before children in fork choice * Load the DAG data in fork choice at init if there is some (can sync witti) * Cluster of quarantined blocks were not properly added to the fork choice * Workaround async gcsafe warnings * Update blockpoool tests * Do the callback before clearing the quarantine * Revert OrderedTable, implement topological sort of DAG, allow forkChoice to be initialized from arbitrary finalized heads * Make it work with latest devel - Altona readyness * Add a recovery mechanism when forkchoice desyncs with blockpool * add the current problematic node to the stack * Fix rebase indentation bug (but still producing invalid block) * Fix cache at epoch boundaries and lateBlock addition
2020-07-09 09:29:32 +00:00
blocks[0..<2] == [pool.tail, b2Add[]]
more block range request updates * handle skip == 0 gracefully * avoid memory allocation at expense of more complex API * add more tests * log block request results
2020-04-21 06:43:39 +00:00
re-enable all but the finalization tests for block pool in mainnet presets
2020-05-05 09:18:44 +00:00
pool.getBlockRange(Slot(0), 3, blocks.toOpenArray(0, 1)) == 1
blocks[0..<2] == [nil, pool.tail] # block 3 is missing!
more block range request updates * handle skip == 0 gracefully * avoid memory allocation at expense of more complex API * add more tests * log block request results
2020-04-21 06:43:39 +00:00
re-enable all but the finalization tests for block pool in mainnet presets
2020-05-05 09:18:44 +00:00
pool.getBlockRange(Slot(2), 2, blocks.toOpenArray(0, 1)) == 0
Dual headed fork choice [Revolution] (#1238) * Dual headed fork choice * fix finalizedEpoch not moving * reduce fork choice verbosity * Add failing tests due to pruning * Properly handle duplicate blocks in sync * test_block_pool also add a test for duplicate blocks * comments addressing review * Fix fork choice v2, was missing integrating block proposed * remove a spurious debug writeStackTrace * update block_sim * Use OrderedTable to ensure that we always load parents before children in fork choice * Load the DAG data in fork choice at init if there is some (can sync witti) * Cluster of quarantined blocks were not properly added to the fork choice * Workaround async gcsafe warnings * Update blockpoool tests * Do the callback before clearing the quarantine * Revert OrderedTable, implement topological sort of DAG, allow forkChoice to be initialized from arbitrary finalized heads * Make it work with latest devel - Altona readyness * Add a recovery mechanism when forkchoice desyncs with blockpool * add the current problematic node to the stack * Fix rebase indentation bug (but still producing invalid block) * Fix cache at epoch boundaries and lateBlock addition
2020-07-09 09:29:32 +00:00
blocks[0..<2] == [b2Add[], b4Add[]] # block 3 is missing!
more block range request updates * handle skip == 0 gracefully * avoid memory allocation at expense of more complex API * add more tests * log block request results
2020-04-21 06:43:39 +00:00
re-enable all but the finalization tests for block pool in mainnet presets
2020-05-05 09:18:44 +00:00
# empty length
pool.getBlockRange(Slot(2), 2, blocks.toOpenArray(0, -1)) == 0
more block range request updates * handle skip == 0 gracefully * avoid memory allocation at expense of more complex API * add more tests * log block request results
2020-04-21 06:43:39 +00:00
re-enable all but the finalization tests for block pool in mainnet presets
2020-05-05 09:18:44 +00:00
# No blocks in sight
pool.getBlockRange(Slot(5), 1, blocks.toOpenArray(0, 1)) == 2
more block range request updates * handle skip == 0 gracefully * avoid memory allocation at expense of more complex API * add more tests * log block request results
2020-04-21 06:43:39 +00:00
re-enable all but the finalization tests for block pool in mainnet presets
2020-05-05 09:18:44 +00:00
# No blocks in sight either due to gaps
pool.getBlockRange(Slot(3), 2, blocks.toOpenArray(0, 1)) == 2
blocks[0..<2] == [BlockRef nil, nil] # block 3 is missing!
more block range request updates * handle skip == 0 gracefully * avoid memory allocation at expense of more complex API * add more tests * log block request results
2020-04-21 06:43:39 +00:00
re-enable all but the finalization tests for block pool in mainnet presets
2020-05-05 09:18:44 +00:00
timedTest "Reverse order block add & get" & preset():
cache block hash (#1329) hash_tree_root was turning up when running beacon_node, turns out to be repeated hash_tree_root invocations - this pr brings them back down to normal. this PR caches the root of a block in the SignedBeaconBlock object - this has the potential downside that even invalid blocks will be hashed (as part of deserialization) - later, one could imagine delaying this until checks have passed there's also some cleanup of the `cat=` logs which were applied randomly and haphazardly, and to a large degree are duplicated by other information in the log statements - in particular, topics fulfill the same role
2020-07-16 13:16:51 +00:00
let missing = pool.addRawBlock(b2) do (validBlock: BLockRef):
Dual headed fork choice [Revolution] (#1238) * Dual headed fork choice * fix finalizedEpoch not moving * reduce fork choice verbosity * Add failing tests due to pruning * Properly handle duplicate blocks in sync * test_block_pool also add a test for duplicate blocks * comments addressing review * Fix fork choice v2, was missing integrating block proposed * remove a spurious debug writeStackTrace * update block_sim * Use OrderedTable to ensure that we always load parents before children in fork choice * Load the DAG data in fork choice at init if there is some (can sync witti) * Cluster of quarantined blocks were not properly added to the fork choice * Workaround async gcsafe warnings * Update blockpoool tests * Do the callback before clearing the quarantine * Revert OrderedTable, implement topological sort of DAG, allow forkChoice to be initialized from arbitrary finalized heads * Make it work with latest devel - Altona readyness * Add a recovery mechanism when forkchoice desyncs with blockpool * add the current problematic node to the stack * Fix rebase indentation bug (but still producing invalid block) * Fix cache at epoch boundaries and lateBlock addition
2020-07-09 09:29:32 +00:00
discard
check: missing.error == MissingParent
more block range request updates * handle skip == 0 gracefully * avoid memory allocation at expense of more complex API * add more tests * log block request results
2020-04-21 06:43:39 +00:00
re-enable all but the finalization tests for block pool in mainnet presets
2020-05-05 09:18:44 +00:00
check:
cache block hash (#1329) hash_tree_root was turning up when running beacon_node, turns out to be repeated hash_tree_root invocations - this pr brings them back down to normal. this PR caches the root of a block in the SignedBeaconBlock object - this has the potential downside that even invalid blocks will be hashed (as part of deserialization) - later, one could imagine delaying this until checks have passed there's also some cleanup of the `cat=` logs which were applied randomly and haphazardly, and to a large degree are duplicated by other information in the log statements - in particular, topics fulfill the same role
2020-07-16 13:16:51 +00:00
pool.get(b2.root).isNone() # Unresolved, shouldn't show up
FetchRecord(root: b1.root) in pool.checkMissing()
more block range request updates * handle skip == 0 gracefully * avoid memory allocation at expense of more complex API * add more tests * log block request results
2020-04-21 06:43:39 +00:00
cache block hash (#1329) hash_tree_root was turning up when running beacon_node, turns out to be repeated hash_tree_root invocations - this pr brings them back down to normal. this PR caches the root of a block in the SignedBeaconBlock object - this has the potential downside that even invalid blocks will be hashed (as part of deserialization) - later, one could imagine delaying this until checks have passed there's also some cleanup of the `cat=` logs which were applied randomly and haphazardly, and to a large degree are duplicated by other information in the log statements - in particular, topics fulfill the same role
2020-07-16 13:16:51 +00:00
let status = pool.addRawBlock(b1) do (validBlock: BlockRef):
Dual headed fork choice [Revolution] (#1238) * Dual headed fork choice * fix finalizedEpoch not moving * reduce fork choice verbosity * Add failing tests due to pruning * Properly handle duplicate blocks in sync * test_block_pool also add a test for duplicate blocks * comments addressing review * Fix fork choice v2, was missing integrating block proposed * remove a spurious debug writeStackTrace * update block_sim * Use OrderedTable to ensure that we always load parents before children in fork choice * Load the DAG data in fork choice at init if there is some (can sync witti) * Cluster of quarantined blocks were not properly added to the fork choice * Workaround async gcsafe warnings * Update blockpoool tests * Do the callback before clearing the quarantine * Revert OrderedTable, implement topological sort of DAG, allow forkChoice to be initialized from arbitrary finalized heads * Make it work with latest devel - Altona readyness * Add a recovery mechanism when forkchoice desyncs with blockpool * add the current problematic node to the stack * Fix rebase indentation bug (but still producing invalid block) * Fix cache at epoch boundaries and lateBlock addition
2020-07-09 09:29:32 +00:00
discard
check: status.isOk
more block range request updates * handle skip == 0 gracefully * avoid memory allocation at expense of more complex API * add more tests * log block request results
2020-04-21 06:43:39 +00:00
re-enable all but the finalization tests for block pool in mainnet presets
2020-05-05 09:18:44 +00:00
let
cache block hash (#1329) hash_tree_root was turning up when running beacon_node, turns out to be repeated hash_tree_root invocations - this pr brings them back down to normal. this PR caches the root of a block in the SignedBeaconBlock object - this has the potential downside that even invalid blocks will be hashed (as part of deserialization) - later, one could imagine delaying this until checks have passed there's also some cleanup of the `cat=` logs which were applied randomly and haphazardly, and to a large degree are duplicated by other information in the log statements - in particular, topics fulfill the same role
2020-07-16 13:16:51 +00:00
b1Get = pool.get(b1.root)
b2Get = pool.get(b2.root)
disable block/attestation pool tests in mainnet
2019-12-19 14:13:35 +00:00
re-enable all but the finalization tests for block pool in mainnet presets
2020-05-05 09:18:44 +00:00
check:
b1Get.isSome()
b2Get.isSome()
disable block/attestation pool tests in mainnet
2019-12-19 14:13:35 +00:00
re-enable all but the finalization tests for block pool in mainnet presets
2020-05-05 09:18:44 +00:00
b1Get.get().refs.children[0] == b2Get.get().refs
b2Get.get().refs.parent == b1Get.get().refs
disable block/attestation pool tests in mainnet
2019-12-19 14:13:35 +00:00
re-enable all but the finalization tests for block pool in mainnet presets
2020-05-05 09:18:44 +00:00
pool.updateHead(b2Get.get().refs)
disable block/attestation pool tests in mainnet
2019-12-19 14:13:35 +00:00
re-enable all but the finalization tests for block pool in mainnet presets
2020-05-05 09:18:44 +00:00
# The heads structure should have been updated to contain only the new
# b2 head
check:
pool.heads.mapIt(it.blck) == @[b2Get.get().refs]
disable block/attestation pool tests in mainnet
2019-12-19 14:13:35 +00:00
re-enable all but the finalization tests for block pool in mainnet presets
2020-05-05 09:18:44 +00:00
# check that init also reloads block graph
var
Working test suite with run-time presets
2020-07-07 23:02:14 +00:00
pool2 = BlockPool.init(defaultRuntimePreset, db)
disable block/attestation pool tests in mainnet
2019-12-19 14:13:35 +00:00
re-enable all but the finalization tests for block pool in mainnet presets
2020-05-05 09:18:44 +00:00
check:
# ensure we loaded the correct head state
pool2.head.blck.root == b2Root
hash_tree_root(pool2.headState.data.data) == b2.message.state_root
pool2.get(b1Root).isSome()
pool2.get(b2Root).isSome()
pool2.heads.len == 1
pool2.heads[0].blck.root == b2Root
Dual headed fork choice [Revolution] (#1238) * Dual headed fork choice * fix finalizedEpoch not moving * reduce fork choice verbosity * Add failing tests due to pruning * Properly handle duplicate blocks in sync * test_block_pool also add a test for duplicate blocks * comments addressing review * Fix fork choice v2, was missing integrating block proposed * remove a spurious debug writeStackTrace * update block_sim * Use OrderedTable to ensure that we always load parents before children in fork choice * Load the DAG data in fork choice at init if there is some (can sync witti) * Cluster of quarantined blocks were not properly added to the fork choice * Workaround async gcsafe warnings * Update blockpoool tests * Do the callback before clearing the quarantine * Revert OrderedTable, implement topological sort of DAG, allow forkChoice to be initialized from arbitrary finalized heads * Make it work with latest devel - Altona readyness * Add a recovery mechanism when forkchoice desyncs with blockpool * add the current problematic node to the stack * Fix rebase indentation bug (but still producing invalid block) * Fix cache at epoch boundaries and lateBlock addition
2020-07-09 09:29:32 +00:00
timedTest "Adding the same block twice returns a Duplicate error" & preset():
re-enable all but the finalization tests for block pool in mainnet presets
2020-05-05 09:18:44 +00:00
let
cache block hash (#1329) hash_tree_root was turning up when running beacon_node, turns out to be repeated hash_tree_root invocations - this pr brings them back down to normal. this PR caches the root of a block in the SignedBeaconBlock object - this has the potential downside that even invalid blocks will be hashed (as part of deserialization) - later, one could imagine delaying this until checks have passed there's also some cleanup of the `cat=` logs which were applied randomly and haphazardly, and to a large degree are duplicated by other information in the log statements - in particular, topics fulfill the same role
2020-07-16 13:16:51 +00:00
b10 = pool.addRawBlock(b1) do (validBlock: BlockRef):
Dual headed fork choice [Revolution] (#1238) * Dual headed fork choice * fix finalizedEpoch not moving * reduce fork choice verbosity * Add failing tests due to pruning * Properly handle duplicate blocks in sync * test_block_pool also add a test for duplicate blocks * comments addressing review * Fix fork choice v2, was missing integrating block proposed * remove a spurious debug writeStackTrace * update block_sim * Use OrderedTable to ensure that we always load parents before children in fork choice * Load the DAG data in fork choice at init if there is some (can sync witti) * Cluster of quarantined blocks were not properly added to the fork choice * Workaround async gcsafe warnings * Update blockpoool tests * Do the callback before clearing the quarantine * Revert OrderedTable, implement topological sort of DAG, allow forkChoice to be initialized from arbitrary finalized heads * Make it work with latest devel - Altona readyness * Add a recovery mechanism when forkchoice desyncs with blockpool * add the current problematic node to the stack * Fix rebase indentation bug (but still producing invalid block) * Fix cache at epoch boundaries and lateBlock addition
2020-07-09 09:29:32 +00:00
discard
cache block hash (#1329) hash_tree_root was turning up when running beacon_node, turns out to be repeated hash_tree_root invocations - this pr brings them back down to normal. this PR caches the root of a block in the SignedBeaconBlock object - this has the potential downside that even invalid blocks will be hashed (as part of deserialization) - later, one could imagine delaying this until checks have passed there's also some cleanup of the `cat=` logs which were applied randomly and haphazardly, and to a large degree are duplicated by other information in the log statements - in particular, topics fulfill the same role
2020-07-16 13:16:51 +00:00
b11 = pool.addRawBlock(b1) do (validBlock: BlockRef):
Dual headed fork choice [Revolution] (#1238) * Dual headed fork choice * fix finalizedEpoch not moving * reduce fork choice verbosity * Add failing tests due to pruning * Properly handle duplicate blocks in sync * test_block_pool also add a test for duplicate blocks * comments addressing review * Fix fork choice v2, was missing integrating block proposed * remove a spurious debug writeStackTrace * update block_sim * Use OrderedTable to ensure that we always load parents before children in fork choice * Load the DAG data in fork choice at init if there is some (can sync witti) * Cluster of quarantined blocks were not properly added to the fork choice * Workaround async gcsafe warnings * Update blockpoool tests * Do the callback before clearing the quarantine * Revert OrderedTable, implement topological sort of DAG, allow forkChoice to be initialized from arbitrary finalized heads * Make it work with latest devel - Altona readyness * Add a recovery mechanism when forkchoice desyncs with blockpool * add the current problematic node to the stack * Fix rebase indentation bug (but still producing invalid block) * Fix cache at epoch boundaries and lateBlock addition
2020-07-09 09:29:32 +00:00
discard
disable block/attestation pool tests in mainnet
2019-12-19 14:13:35 +00:00
re-enable all but the finalization tests for block pool in mainnet presets
2020-05-05 09:18:44 +00:00
check:
Dual headed fork choice [Revolution] (#1238) * Dual headed fork choice * fix finalizedEpoch not moving * reduce fork choice verbosity * Add failing tests due to pruning * Properly handle duplicate blocks in sync * test_block_pool also add a test for duplicate blocks * comments addressing review * Fix fork choice v2, was missing integrating block proposed * remove a spurious debug writeStackTrace * update block_sim * Use OrderedTable to ensure that we always load parents before children in fork choice * Load the DAG data in fork choice at init if there is some (can sync witti) * Cluster of quarantined blocks were not properly added to the fork choice * Workaround async gcsafe warnings * Update blockpoool tests * Do the callback before clearing the quarantine * Revert OrderedTable, implement topological sort of DAG, allow forkChoice to be initialized from arbitrary finalized heads * Make it work with latest devel - Altona readyness * Add a recovery mechanism when forkchoice desyncs with blockpool * add the current problematic node to the stack * Fix rebase indentation bug (but still producing invalid block) * Fix cache at epoch boundaries and lateBlock addition
2020-07-09 09:29:32 +00:00
b11.error == Duplicate
not b10[].isNil
disable block/attestation pool tests in mainnet
2019-12-19 14:13:35 +00:00
re-enable all but the finalization tests for block pool in mainnet presets
2020-05-05 09:18:44 +00:00
timedTest "updateHead updates head and headState" & preset():
let
cache block hash (#1329) hash_tree_root was turning up when running beacon_node, turns out to be repeated hash_tree_root invocations - this pr brings them back down to normal. this PR caches the root of a block in the SignedBeaconBlock object - this has the potential downside that even invalid blocks will be hashed (as part of deserialization) - later, one could imagine delaying this until checks have passed there's also some cleanup of the `cat=` logs which were applied randomly and haphazardly, and to a large degree are duplicated by other information in the log statements - in particular, topics fulfill the same role
2020-07-16 13:16:51 +00:00
b1Add = pool.addRawBlock(b1) do (validBlock: BlockRef):
Dual headed fork choice [Revolution] (#1238) * Dual headed fork choice * fix finalizedEpoch not moving * reduce fork choice verbosity * Add failing tests due to pruning * Properly handle duplicate blocks in sync * test_block_pool also add a test for duplicate blocks * comments addressing review * Fix fork choice v2, was missing integrating block proposed * remove a spurious debug writeStackTrace * update block_sim * Use OrderedTable to ensure that we always load parents before children in fork choice * Load the DAG data in fork choice at init if there is some (can sync witti) * Cluster of quarantined blocks were not properly added to the fork choice * Workaround async gcsafe warnings * Update blockpoool tests * Do the callback before clearing the quarantine * Revert OrderedTable, implement topological sort of DAG, allow forkChoice to be initialized from arbitrary finalized heads * Make it work with latest devel - Altona readyness * Add a recovery mechanism when forkchoice desyncs with blockpool * add the current problematic node to the stack * Fix rebase indentation bug (but still producing invalid block) * Fix cache at epoch boundaries and lateBlock addition
2020-07-09 09:29:32 +00:00
discard
re-enable all but the finalization tests for block pool in mainnet presets
2020-05-05 09:18:44 +00:00
Dual headed fork choice [Revolution] (#1238) * Dual headed fork choice * fix finalizedEpoch not moving * reduce fork choice verbosity * Add failing tests due to pruning * Properly handle duplicate blocks in sync * test_block_pool also add a test for duplicate blocks * comments addressing review * Fix fork choice v2, was missing integrating block proposed * remove a spurious debug writeStackTrace * update block_sim * Use OrderedTable to ensure that we always load parents before children in fork choice * Load the DAG data in fork choice at init if there is some (can sync witti) * Cluster of quarantined blocks were not properly added to the fork choice * Workaround async gcsafe warnings * Update blockpoool tests * Do the callback before clearing the quarantine * Revert OrderedTable, implement topological sort of DAG, allow forkChoice to be initialized from arbitrary finalized heads * Make it work with latest devel - Altona readyness * Add a recovery mechanism when forkchoice desyncs with blockpool * add the current problematic node to the stack * Fix rebase indentation bug (but still producing invalid block) * Fix cache at epoch boundaries and lateBlock addition
2020-07-09 09:29:32 +00:00
pool.updateHead(b1Add[])
re-enable all but the finalization tests for block pool in mainnet presets
2020-05-05 09:18:44 +00:00
check:
Dual headed fork choice [Revolution] (#1238) * Dual headed fork choice * fix finalizedEpoch not moving * reduce fork choice verbosity * Add failing tests due to pruning * Properly handle duplicate blocks in sync * test_block_pool also add a test for duplicate blocks * comments addressing review * Fix fork choice v2, was missing integrating block proposed * remove a spurious debug writeStackTrace * update block_sim * Use OrderedTable to ensure that we always load parents before children in fork choice * Load the DAG data in fork choice at init if there is some (can sync witti) * Cluster of quarantined blocks were not properly added to the fork choice * Workaround async gcsafe warnings * Update blockpoool tests * Do the callback before clearing the quarantine * Revert OrderedTable, implement topological sort of DAG, allow forkChoice to be initialized from arbitrary finalized heads * Make it work with latest devel - Altona readyness * Add a recovery mechanism when forkchoice desyncs with blockpool * add the current problematic node to the stack * Fix rebase indentation bug (but still producing invalid block) * Fix cache at epoch boundaries and lateBlock addition
2020-07-09 09:29:32 +00:00
pool.head.blck == b1Add[]
pool.headState.data.data.slot == b1Add[].slot
re-enable all but the finalization tests for block pool in mainnet presets
2020-05-05 09:18:44 +00:00
timedTest "updateStateData sanity" & preset():
let
cache block hash (#1329) hash_tree_root was turning up when running beacon_node, turns out to be repeated hash_tree_root invocations - this pr brings them back down to normal. this PR caches the root of a block in the SignedBeaconBlock object - this has the potential downside that even invalid blocks will be hashed (as part of deserialization) - later, one could imagine delaying this until checks have passed there's also some cleanup of the `cat=` logs which were applied randomly and haphazardly, and to a large degree are duplicated by other information in the log statements - in particular, topics fulfill the same role
2020-07-16 13:16:51 +00:00
b1Add = pool.addRawBlock(b1) do (validBlock: BlockRef):
Dual headed fork choice [Revolution] (#1238) * Dual headed fork choice * fix finalizedEpoch not moving * reduce fork choice verbosity * Add failing tests due to pruning * Properly handle duplicate blocks in sync * test_block_pool also add a test for duplicate blocks * comments addressing review * Fix fork choice v2, was missing integrating block proposed * remove a spurious debug writeStackTrace * update block_sim * Use OrderedTable to ensure that we always load parents before children in fork choice * Load the DAG data in fork choice at init if there is some (can sync witti) * Cluster of quarantined blocks were not properly added to the fork choice * Workaround async gcsafe warnings * Update blockpoool tests * Do the callback before clearing the quarantine * Revert OrderedTable, implement topological sort of DAG, allow forkChoice to be initialized from arbitrary finalized heads * Make it work with latest devel - Altona readyness * Add a recovery mechanism when forkchoice desyncs with blockpool * add the current problematic node to the stack * Fix rebase indentation bug (but still producing invalid block) * Fix cache at epoch boundaries and lateBlock addition
2020-07-09 09:29:32 +00:00
discard
cache block hash (#1329) hash_tree_root was turning up when running beacon_node, turns out to be repeated hash_tree_root invocations - this pr brings them back down to normal. this PR caches the root of a block in the SignedBeaconBlock object - this has the potential downside that even invalid blocks will be hashed (as part of deserialization) - later, one could imagine delaying this until checks have passed there's also some cleanup of the `cat=` logs which were applied randomly and haphazardly, and to a large degree are duplicated by other information in the log statements - in particular, topics fulfill the same role
2020-07-16 13:16:51 +00:00
b2Add = pool.addRawBlock(b2) do (validBlock: BlockRef):
Dual headed fork choice [Revolution] (#1238) * Dual headed fork choice * fix finalizedEpoch not moving * reduce fork choice verbosity * Add failing tests due to pruning * Properly handle duplicate blocks in sync * test_block_pool also add a test for duplicate blocks * comments addressing review * Fix fork choice v2, was missing integrating block proposed * remove a spurious debug writeStackTrace * update block_sim * Use OrderedTable to ensure that we always load parents before children in fork choice * Load the DAG data in fork choice at init if there is some (can sync witti) * Cluster of quarantined blocks were not properly added to the fork choice * Workaround async gcsafe warnings * Update blockpoool tests * Do the callback before clearing the quarantine * Revert OrderedTable, implement topological sort of DAG, allow forkChoice to be initialized from arbitrary finalized heads * Make it work with latest devel - Altona readyness * Add a recovery mechanism when forkchoice desyncs with blockpool * add the current problematic node to the stack * Fix rebase indentation bug (but still producing invalid block) * Fix cache at epoch boundaries and lateBlock addition
2020-07-09 09:29:32 +00:00
discard
bs1 = BlockSlot(blck: b1Add[], slot: b1.message.slot)
bs1_3 = b1Add[].atSlot(3.Slot)
bs2_3 = b2Add[].atSlot(3.Slot)
re-enable all but the finalization tests for block pool in mainnet presets
2020-05-05 09:18:44 +00:00
add tests for unviable blocks (#1271) * add tests for unviable blocks also enable finalization tests in all test configs - they're plenty fast now also fix newClone for non-rvo cases. sigh. * fixes
2020-07-01 17:00:14 +00:00
var tmpState = assignClone(pool.headState)
re-enable all but the finalization tests for block pool in mainnet presets
2020-05-05 09:18:44 +00:00
# move to specific block
pool.updateStateData(tmpState[], bs1)
check:
Dual headed fork choice [Revolution] (#1238) * Dual headed fork choice * fix finalizedEpoch not moving * reduce fork choice verbosity * Add failing tests due to pruning * Properly handle duplicate blocks in sync * test_block_pool also add a test for duplicate blocks * comments addressing review * Fix fork choice v2, was missing integrating block proposed * remove a spurious debug writeStackTrace * update block_sim * Use OrderedTable to ensure that we always load parents before children in fork choice * Load the DAG data in fork choice at init if there is some (can sync witti) * Cluster of quarantined blocks were not properly added to the fork choice * Workaround async gcsafe warnings * Update blockpoool tests * Do the callback before clearing the quarantine * Revert OrderedTable, implement topological sort of DAG, allow forkChoice to be initialized from arbitrary finalized heads * Make it work with latest devel - Altona readyness * Add a recovery mechanism when forkchoice desyncs with blockpool * add the current problematic node to the stack * Fix rebase indentation bug (but still producing invalid block) * Fix cache at epoch boundaries and lateBlock addition
2020-07-09 09:29:32 +00:00
tmpState.blck == b1Add[]
re-enable all but the finalization tests for block pool in mainnet presets
2020-05-05 09:18:44 +00:00
tmpState.data.data.slot == bs1.slot
# Skip slots
pool.updateStateData(tmpState[], bs1_3) # skip slots
check:
Dual headed fork choice [Revolution] (#1238) * Dual headed fork choice * fix finalizedEpoch not moving * reduce fork choice verbosity * Add failing tests due to pruning * Properly handle duplicate blocks in sync * test_block_pool also add a test for duplicate blocks * comments addressing review * Fix fork choice v2, was missing integrating block proposed * remove a spurious debug writeStackTrace * update block_sim * Use OrderedTable to ensure that we always load parents before children in fork choice * Load the DAG data in fork choice at init if there is some (can sync witti) * Cluster of quarantined blocks were not properly added to the fork choice * Workaround async gcsafe warnings * Update blockpoool tests * Do the callback before clearing the quarantine * Revert OrderedTable, implement topological sort of DAG, allow forkChoice to be initialized from arbitrary finalized heads * Make it work with latest devel - Altona readyness * Add a recovery mechanism when forkchoice desyncs with blockpool * add the current problematic node to the stack * Fix rebase indentation bug (but still producing invalid block) * Fix cache at epoch boundaries and lateBlock addition
2020-07-09 09:29:32 +00:00
tmpState.blck == b1Add[]
re-enable all but the finalization tests for block pool in mainnet presets
2020-05-05 09:18:44 +00:00
tmpState.data.data.slot == bs1_3.slot
# Move back slots, but not blocks
pool.updateStateData(tmpState[], bs1_3.parent())
check:
Dual headed fork choice [Revolution] (#1238) * Dual headed fork choice * fix finalizedEpoch not moving * reduce fork choice verbosity * Add failing tests due to pruning * Properly handle duplicate blocks in sync * test_block_pool also add a test for duplicate blocks * comments addressing review * Fix fork choice v2, was missing integrating block proposed * remove a spurious debug writeStackTrace * update block_sim * Use OrderedTable to ensure that we always load parents before children in fork choice * Load the DAG data in fork choice at init if there is some (can sync witti) * Cluster of quarantined blocks were not properly added to the fork choice * Workaround async gcsafe warnings * Update blockpoool tests * Do the callback before clearing the quarantine * Revert OrderedTable, implement topological sort of DAG, allow forkChoice to be initialized from arbitrary finalized heads * Make it work with latest devel - Altona readyness * Add a recovery mechanism when forkchoice desyncs with blockpool * add the current problematic node to the stack * Fix rebase indentation bug (but still producing invalid block) * Fix cache at epoch boundaries and lateBlock addition
2020-07-09 09:29:32 +00:00
tmpState.blck == b1Add[]
re-enable all but the finalization tests for block pool in mainnet presets
2020-05-05 09:18:44 +00:00
tmpState.data.data.slot == bs1_3.parent().slot
# Move to different block and slot
pool.updateStateData(tmpState[], bs2_3)
check:
Dual headed fork choice [Revolution] (#1238) * Dual headed fork choice * fix finalizedEpoch not moving * reduce fork choice verbosity * Add failing tests due to pruning * Properly handle duplicate blocks in sync * test_block_pool also add a test for duplicate blocks * comments addressing review * Fix fork choice v2, was missing integrating block proposed * remove a spurious debug writeStackTrace * update block_sim * Use OrderedTable to ensure that we always load parents before children in fork choice * Load the DAG data in fork choice at init if there is some (can sync witti) * Cluster of quarantined blocks were not properly added to the fork choice * Workaround async gcsafe warnings * Update blockpoool tests * Do the callback before clearing the quarantine * Revert OrderedTable, implement topological sort of DAG, allow forkChoice to be initialized from arbitrary finalized heads * Make it work with latest devel - Altona readyness * Add a recovery mechanism when forkchoice desyncs with blockpool * add the current problematic node to the stack * Fix rebase indentation bug (but still producing invalid block) * Fix cache at epoch boundaries and lateBlock addition
2020-07-09 09:29:32 +00:00
tmpState.blck == b2Add[]
re-enable all but the finalization tests for block pool in mainnet presets
2020-05-05 09:18:44 +00:00
tmpState.data.data.slot == bs2_3.slot
# Move back slot and block
pool.updateStateData(tmpState[], bs1)
check:
Dual headed fork choice [Revolution] (#1238) * Dual headed fork choice * fix finalizedEpoch not moving * reduce fork choice verbosity * Add failing tests due to pruning * Properly handle duplicate blocks in sync * test_block_pool also add a test for duplicate blocks * comments addressing review * Fix fork choice v2, was missing integrating block proposed * remove a spurious debug writeStackTrace * update block_sim * Use OrderedTable to ensure that we always load parents before children in fork choice * Load the DAG data in fork choice at init if there is some (can sync witti) * Cluster of quarantined blocks were not properly added to the fork choice * Workaround async gcsafe warnings * Update blockpoool tests * Do the callback before clearing the quarantine * Revert OrderedTable, implement topological sort of DAG, allow forkChoice to be initialized from arbitrary finalized heads * Make it work with latest devel - Altona readyness * Add a recovery mechanism when forkchoice desyncs with blockpool * add the current problematic node to the stack * Fix rebase indentation bug (but still producing invalid block) * Fix cache at epoch boundaries and lateBlock addition
2020-07-09 09:29:32 +00:00
tmpState.blck == b1Add[]
re-enable all but the finalization tests for block pool in mainnet presets
2020-05-05 09:18:44 +00:00
tmpState.data.data.slot == bs1.slot
# Move back to genesis
pool.updateStateData(tmpState[], bs1.parent())
check:
Dual headed fork choice [Revolution] (#1238) * Dual headed fork choice * fix finalizedEpoch not moving * reduce fork choice verbosity * Add failing tests due to pruning * Properly handle duplicate blocks in sync * test_block_pool also add a test for duplicate blocks * comments addressing review * Fix fork choice v2, was missing integrating block proposed * remove a spurious debug writeStackTrace * update block_sim * Use OrderedTable to ensure that we always load parents before children in fork choice * Load the DAG data in fork choice at init if there is some (can sync witti) * Cluster of quarantined blocks were not properly added to the fork choice * Workaround async gcsafe warnings * Update blockpoool tests * Do the callback before clearing the quarantine * Revert OrderedTable, implement topological sort of DAG, allow forkChoice to be initialized from arbitrary finalized heads * Make it work with latest devel - Altona readyness * Add a recovery mechanism when forkchoice desyncs with blockpool * add the current problematic node to the stack * Fix rebase indentation bug (but still producing invalid block) * Fix cache at epoch boundaries and lateBlock addition
2020-07-09 09:29:32 +00:00
tmpState.blck == b1Add[].parent
re-enable all but the finalization tests for block pool in mainnet presets
2020-05-05 09:18:44 +00:00
tmpState.data.data.slot == bs1.parent.slot
add tests for unviable blocks (#1271) * add tests for unviable blocks also enable finalization tests in all test configs - they're plenty fast now also fix newClone for non-rvo cases. sigh. * fixes
2020-07-01 17:00:14 +00:00
suiteReport "BlockPool finalization tests" & preset():
setup:
var
db = makeTestDB(SLOTS_PER_EPOCH)
Working test suite with run-time presets
2020-07-07 23:02:14 +00:00
pool = BlockPool.init(defaultRuntimePreset, db)
restore EpochRef and flush statecaches on epoch transitions (#1312) * restore EpochRef and flush statecaches on epoch transitions * more targeted cache invalidation * remove get_empty_per_epoch_cache(); implement simpler but still faster get_beacon_proposer_index()/compute_proposer_index() approach; add some abstraction layer for accessing the shuffled validator indices cache * reduce integer type conversions * remove most of rest of integer type conversion in compute_proposer_index()
2020-07-15 10:44:18 +00:00
cache = StateCache()
fix state rewind * rewind fast path comparison was not taking skipped slots into account properly * less messy blockref creation
2020-02-05 11:41:46 +00:00
add tests for unviable blocks (#1271) * add tests for unviable blocks also enable finalization tests in all test configs - they're plenty fast now also fix newClone for non-rvo cases. sigh. * fixes
2020-07-01 17:00:14 +00:00
timedTest "prune heads on finalization" & preset():
# Create a fork that will not be taken
var
blck = makeTestBlock(pool.headState.data, pool.head.blck.root, cache)
tmpState = assignClone(pool.headState.data)
check:
process_slots(
tmpState[], tmpState.data.slot + (5 * SLOTS_PER_EPOCH).uint64)
let lateBlock = makeTestBlock(tmpState[], pool.head.blck.root, cache)
Dual headed fork choice [Revolution] (#1238) * Dual headed fork choice * fix finalizedEpoch not moving * reduce fork choice verbosity * Add failing tests due to pruning * Properly handle duplicate blocks in sync * test_block_pool also add a test for duplicate blocks * comments addressing review * Fix fork choice v2, was missing integrating block proposed * remove a spurious debug writeStackTrace * update block_sim * Use OrderedTable to ensure that we always load parents before children in fork choice * Load the DAG data in fork choice at init if there is some (can sync witti) * Cluster of quarantined blocks were not properly added to the fork choice * Workaround async gcsafe warnings * Update blockpoool tests * Do the callback before clearing the quarantine * Revert OrderedTable, implement topological sort of DAG, allow forkChoice to be initialized from arbitrary finalized heads * Make it work with latest devel - Altona readyness * Add a recovery mechanism when forkchoice desyncs with blockpool * add the current problematic node to the stack * Fix rebase indentation bug (but still producing invalid block) * Fix cache at epoch boundaries and lateBlock addition
2020-07-09 09:29:32 +00:00
block:
cache block hash (#1329) hash_tree_root was turning up when running beacon_node, turns out to be repeated hash_tree_root invocations - this pr brings them back down to normal. this PR caches the root of a block in the SignedBeaconBlock object - this has the potential downside that even invalid blocks will be hashed (as part of deserialization) - later, one could imagine delaying this until checks have passed there's also some cleanup of the `cat=` logs which were applied randomly and haphazardly, and to a large degree are duplicated by other information in the log statements - in particular, topics fulfill the same role
2020-07-16 13:16:51 +00:00
let status = pool.addRawBlock(blck) do (validBlock: BlockRef):
Dual headed fork choice [Revolution] (#1238) * Dual headed fork choice * fix finalizedEpoch not moving * reduce fork choice verbosity * Add failing tests due to pruning * Properly handle duplicate blocks in sync * test_block_pool also add a test for duplicate blocks * comments addressing review * Fix fork choice v2, was missing integrating block proposed * remove a spurious debug writeStackTrace * update block_sim * Use OrderedTable to ensure that we always load parents before children in fork choice * Load the DAG data in fork choice at init if there is some (can sync witti) * Cluster of quarantined blocks were not properly added to the fork choice * Workaround async gcsafe warnings * Update blockpoool tests * Do the callback before clearing the quarantine * Revert OrderedTable, implement topological sort of DAG, allow forkChoice to be initialized from arbitrary finalized heads * Make it work with latest devel - Altona readyness * Add a recovery mechanism when forkchoice desyncs with blockpool * add the current problematic node to the stack * Fix rebase indentation bug (but still producing invalid block) * Fix cache at epoch boundaries and lateBlock addition
2020-07-09 09:29:32 +00:00
discard
check: status.isOk()
add tests for unviable blocks (#1271) * add tests for unviable blocks also enable finalization tests in all test configs - they're plenty fast now also fix newClone for non-rvo cases. sigh. * fixes
2020-07-01 17:00:14 +00:00
for i in 0 ..< (SLOTS_PER_EPOCH * 6):
if i == 1:
# There are 2 heads now because of the fork at slot 1
check:
pool.tail.children.len == 2
pool.heads.len == 2
Dual headed fork choice [Revolution] (#1238) * Dual headed fork choice * fix finalizedEpoch not moving * reduce fork choice verbosity * Add failing tests due to pruning * Properly handle duplicate blocks in sync * test_block_pool also add a test for duplicate blocks * comments addressing review * Fix fork choice v2, was missing integrating block proposed * remove a spurious debug writeStackTrace * update block_sim * Use OrderedTable to ensure that we always load parents before children in fork choice * Load the DAG data in fork choice at init if there is some (can sync witti) * Cluster of quarantined blocks were not properly added to the fork choice * Workaround async gcsafe warnings * Update blockpoool tests * Do the callback before clearing the quarantine * Revert OrderedTable, implement topological sort of DAG, allow forkChoice to be initialized from arbitrary finalized heads * Make it work with latest devel - Altona readyness * Add a recovery mechanism when forkchoice desyncs with blockpool * add the current problematic node to the stack * Fix rebase indentation bug (but still producing invalid block) * Fix cache at epoch boundaries and lateBlock addition
2020-07-09 09:29:32 +00:00
blck = makeTestBlock(
pool.headState.data, pool.head.blck.root, cache,
attestations = makeFullAttestations(
pool.headState.data.data, pool.head.blck.root,
pool.headState.data.data.slot, cache, {}))
cache block hash (#1329) hash_tree_root was turning up when running beacon_node, turns out to be repeated hash_tree_root invocations - this pr brings them back down to normal. this PR caches the root of a block in the SignedBeaconBlock object - this has the potential downside that even invalid blocks will be hashed (as part of deserialization) - later, one could imagine delaying this until checks have passed there's also some cleanup of the `cat=` logs which were applied randomly and haphazardly, and to a large degree are duplicated by other information in the log statements - in particular, topics fulfill the same role
2020-07-16 13:16:51 +00:00
let added = pool.addRawBlock(blck) do (validBlock: BlockRef):
Dual headed fork choice [Revolution] (#1238) * Dual headed fork choice * fix finalizedEpoch not moving * reduce fork choice verbosity * Add failing tests due to pruning * Properly handle duplicate blocks in sync * test_block_pool also add a test for duplicate blocks * comments addressing review * Fix fork choice v2, was missing integrating block proposed * remove a spurious debug writeStackTrace * update block_sim * Use OrderedTable to ensure that we always load parents before children in fork choice * Load the DAG data in fork choice at init if there is some (can sync witti) * Cluster of quarantined blocks were not properly added to the fork choice * Workaround async gcsafe warnings * Update blockpoool tests * Do the callback before clearing the quarantine * Revert OrderedTable, implement topological sort of DAG, allow forkChoice to be initialized from arbitrary finalized heads * Make it work with latest devel - Altona readyness * Add a recovery mechanism when forkchoice desyncs with blockpool * add the current problematic node to the stack * Fix rebase indentation bug (but still producing invalid block) * Fix cache at epoch boundaries and lateBlock addition
2020-07-09 09:29:32 +00:00
discard
check: added.isOk()
pool.updateHead(added[])
add tests for unviable blocks (#1271) * add tests for unviable blocks also enable finalization tests in all test configs - they're plenty fast now also fix newClone for non-rvo cases. sigh. * fixes
2020-07-01 17:00:14 +00:00
check:
pool.heads.len() == 1
pool.head.justified.slot.compute_epoch_at_slot() == 5
pool.tail.children.len == 1
Dual headed fork choice [Revolution] (#1238) * Dual headed fork choice * fix finalizedEpoch not moving * reduce fork choice verbosity * Add failing tests due to pruning * Properly handle duplicate blocks in sync * test_block_pool also add a test for duplicate blocks * comments addressing review * Fix fork choice v2, was missing integrating block proposed * remove a spurious debug writeStackTrace * update block_sim * Use OrderedTable to ensure that we always load parents before children in fork choice * Load the DAG data in fork choice at init if there is some (can sync witti) * Cluster of quarantined blocks were not properly added to the fork choice * Workaround async gcsafe warnings * Update blockpoool tests * Do the callback before clearing the quarantine * Revert OrderedTable, implement topological sort of DAG, allow forkChoice to be initialized from arbitrary finalized heads * Make it work with latest devel - Altona readyness * Add a recovery mechanism when forkchoice desyncs with blockpool * add the current problematic node to the stack * Fix rebase indentation bug (but still producing invalid block) * Fix cache at epoch boundaries and lateBlock addition
2020-07-09 09:29:32 +00:00
block:
add tests for unviable blocks (#1271) * add tests for unviable blocks also enable finalization tests in all test configs - they're plenty fast now also fix newClone for non-rvo cases. sigh. * fixes
2020-07-01 17:00:14 +00:00
# The late block is a block whose parent was finalized long ago and thus
# is no longer a viable head candidate
cache block hash (#1329) hash_tree_root was turning up when running beacon_node, turns out to be repeated hash_tree_root invocations - this pr brings them back down to normal. this PR caches the root of a block in the SignedBeaconBlock object - this has the potential downside that even invalid blocks will be hashed (as part of deserialization) - later, one could imagine delaying this until checks have passed there's also some cleanup of the `cat=` logs which were applied randomly and haphazardly, and to a large degree are duplicated by other information in the log statements - in particular, topics fulfill the same role
2020-07-16 13:16:51 +00:00
let status = pool.addRawBlock(lateBlock) do (validBlock: BlockRef):
Dual headed fork choice [Revolution] (#1238) * Dual headed fork choice * fix finalizedEpoch not moving * reduce fork choice verbosity * Add failing tests due to pruning * Properly handle duplicate blocks in sync * test_block_pool also add a test for duplicate blocks * comments addressing review * Fix fork choice v2, was missing integrating block proposed * remove a spurious debug writeStackTrace * update block_sim * Use OrderedTable to ensure that we always load parents before children in fork choice * Load the DAG data in fork choice at init if there is some (can sync witti) * Cluster of quarantined blocks were not properly added to the fork choice * Workaround async gcsafe warnings * Update blockpoool tests * Do the callback before clearing the quarantine * Revert OrderedTable, implement topological sort of DAG, allow forkChoice to be initialized from arbitrary finalized heads * Make it work with latest devel - Altona readyness * Add a recovery mechanism when forkchoice desyncs with blockpool * add the current problematic node to the stack * Fix rebase indentation bug (but still producing invalid block) * Fix cache at epoch boundaries and lateBlock addition
2020-07-09 09:29:32 +00:00
discard
check: status.error == Unviable
add tests for unviable blocks (#1271) * add tests for unviable blocks also enable finalization tests in all test configs - they're plenty fast now also fix newClone for non-rvo cases. sigh. * fixes
2020-07-01 17:00:14 +00:00
let
Working test suite with run-time presets
2020-07-07 23:02:14 +00:00
pool2 = BlockPool.init(defaultRuntimePreset, db)
add tests for unviable blocks (#1271) * add tests for unviable blocks also enable finalization tests in all test configs - they're plenty fast now also fix newClone for non-rvo cases. sigh. * fixes
2020-07-01 17:00:14 +00:00
# check that the state reloaded from database resembles what we had before
check:
pool2.tail.root == pool.tail.root
pool2.head.blck.root == pool.head.blck.root
pool2.finalizedHead.blck.root == pool.finalizedHead.blck.root
pool2.finalizedHead.slot == pool.finalizedHead.slot
hash_tree_root(pool2.headState.data.data) ==
hash_tree_root(pool.headState.data.data)
hash_tree_root(pool2.justifiedState.data.data) ==
hash_tree_root(pool.justifiedState.data.data)
Revert "Dual headed fork choice [Reloaded] (#1223)" (#1234) This reverts commit 6836d41ebda0ac2181adfd82489f67f968a44aa3.
2020-06-25 09:36:03 +00:00
Dual headed fork choice [Revolution] (#1238) * Dual headed fork choice * fix finalizedEpoch not moving * reduce fork choice verbosity * Add failing tests due to pruning * Properly handle duplicate blocks in sync * test_block_pool also add a test for duplicate blocks * comments addressing review * Fix fork choice v2, was missing integrating block proposed * remove a spurious debug writeStackTrace * update block_sim * Use OrderedTable to ensure that we always load parents before children in fork choice * Load the DAG data in fork choice at init if there is some (can sync witti) * Cluster of quarantined blocks were not properly added to the fork choice * Workaround async gcsafe warnings * Update blockpoool tests * Do the callback before clearing the quarantine * Revert OrderedTable, implement topological sort of DAG, allow forkChoice to be initialized from arbitrary finalized heads * Make it work with latest devel - Altona readyness * Add a recovery mechanism when forkchoice desyncs with blockpool * add the current problematic node to the stack * Fix rebase indentation bug (but still producing invalid block) * Fix cache at epoch boundaries and lateBlock addition
2020-07-09 09:29:32 +00:00
# timedTest "init with gaps" & preset():
restore EpochRef and flush statecaches on epoch transitions (#1312) * restore EpochRef and flush statecaches on epoch transitions * more targeted cache invalidation * remove get_empty_per_epoch_cache(); implement simpler but still faster get_beacon_proposer_index()/compute_proposer_index() approach; add some abstraction layer for accessing the shuffled validator indices cache * reduce integer type conversions * remove most of rest of integer type conversion in compute_proposer_index()
2020-07-15 10:44:18 +00:00
# var cache = StateCache()
Dual headed fork choice [Revolution] (#1238) * Dual headed fork choice * fix finalizedEpoch not moving * reduce fork choice verbosity * Add failing tests due to pruning * Properly handle duplicate blocks in sync * test_block_pool also add a test for duplicate blocks * comments addressing review * Fix fork choice v2, was missing integrating block proposed * remove a spurious debug writeStackTrace * update block_sim * Use OrderedTable to ensure that we always load parents before children in fork choice * Load the DAG data in fork choice at init if there is some (can sync witti) * Cluster of quarantined blocks were not properly added to the fork choice * Workaround async gcsafe warnings * Update blockpoool tests * Do the callback before clearing the quarantine * Revert OrderedTable, implement topological sort of DAG, allow forkChoice to be initialized from arbitrary finalized heads * Make it work with latest devel - Altona readyness * Add a recovery mechanism when forkchoice desyncs with blockpool * add the current problematic node to the stack * Fix rebase indentation bug (but still producing invalid block) * Fix cache at epoch boundaries and lateBlock addition
2020-07-09 09:29:32 +00:00
# for i in 0 ..< (SLOTS_PER_EPOCH * 6 - 2):
# var
# blck = makeTestBlock(
# pool.headState.data, pool.head.blck.root, cache,
# attestations = makeFullAttestations(
# pool.headState.data.data, pool.head.blck.root,
# pool.headState.data.data.slot, cache, {}))
# let added = pool.addRawBlock(hash_tree_root(blck.message), blck) do (validBlock: BlockRef):
# discard
# check: added.isOk()
# pool.updateHead(added[])
# # Advance past epoch so that the epoch transition is gapped
# check:
# process_slots(
# pool.headState.data, Slot(SLOTS_PER_EPOCH * 6 + 2) )
# var blck = makeTestBlock(
# pool.headState.data, pool.head.blck.root, cache,
# attestations = makeFullAttestations(
# pool.headState.data.data, pool.head.blck.root,
# pool.headState.data.data.slot, cache, {}))
# let added = pool.addRawBlock(hash_tree_root(blck.message), blck) do (validBlock: BlockRef):
# discard
# check: added.isOk()
# pool.updateHead(added[])
# let
# pool2 = BlockPool.init(db)
# # check that the state reloaded from database resembles what we had before
# check:
# pool2.tail.root == pool.tail.root
# pool2.head.blck.root == pool.head.blck.root
# pool2.finalizedHead.blck.root == pool.finalizedHead.blck.root
# pool2.finalizedHead.slot == pool.finalizedHead.slot
# hash_tree_root(pool2.headState.data.data) ==
# hash_tree_root(pool.headState.data.data)
# hash_tree_root(pool2.justifiedState.data.data) ==
# hash_tree_root(pool.justifiedState.data.data)