status-im
/
nimbus-eth2
mirror of https://github.com/status-im/nimbus-eth2.git
2
0
Fork 0
Code Issues Projects Releases Wiki Activity
nimbus-eth2/tests/test_block_quarantine.nim

79 lines
2.5 KiB
Nim
Raw Permalink Normal View History

Harden handling of unviable forks (#3312) * Harden handling of unviable forks In our current handling of unviable forks, we allow peers to send us blocks that come from a different fork - this is not necessarily an error as it can happen naturally, but it does open up the client to a case where the same unviable fork keeps getting requested - rather than allowing this to happen, we'll now give these peers a small negative score - if it keeps happening, we'll disconnect them. * keep track of unviable forks in quarantine, to avoid filling it with known junk * collect peer scores in single module * descore peers when they send unviable blocks during sync * don't give score for duplicate blocks * increase quarantine size to a level that allows finality to happen under optimal conditions - this helps avoid downloading the same blocks over and over in case of an unviable fork * increase initial score for new peers to make room for one more failure before disconnection * log and score invalid/unviable blocks in requestmanager too * avoid ChainDAG dependency in quarantine * reject gossip blocks with unviable parent * continue processing unviable sync blocks in order to build unviable dag * docs * Update beacon_chain/consensus_object_pools/block_pools_types.nim * add unviable queue test
2022-01-26 12:20:08 +00:00
# beacon_chain
finish eip4844 to deneb module rename (#4705)
2023-03-09 00:34:17 +00:00
# Copyright (c) 2022-2023 Status Research & Development GmbH
Harden handling of unviable forks (#3312) * Harden handling of unviable forks In our current handling of unviable forks, we allow peers to send us blocks that come from a different fork - this is not necessarily an error as it can happen naturally, but it does open up the client to a case where the same unviable fork keeps getting requested - rather than allowing this to happen, we'll now give these peers a small negative score - if it keeps happening, we'll disconnect them. * keep track of unviable forks in quarantine, to avoid filling it with known junk * collect peer scores in single module * descore peers when they send unviable blocks during sync * don't give score for duplicate blocks * increase quarantine size to a level that allows finality to happen under optimal conditions - this helps avoid downloading the same blocks over and over in case of an unviable fork * increase initial score for new peers to make room for one more failure before disconnection * log and score invalid/unviable blocks in requestmanager too * avoid ChainDAG dependency in quarantine * reject gossip blocks with unviable parent * continue processing unviable sync blocks in order to build unviable dag * docs * Update beacon_chain/consensus_object_pools/block_pools_types.nim * add unviable queue test
2022-01-26 12:20:08 +00:00
# Licensed and distributed under either of
# * MIT license (license terms in the root directory or at https://opensource.org/licenses/MIT).
# * Apache v2 license (license terms in the root directory or at https://www.apache.org/licenses/LICENSE-2.0).
# at your option. This file may not be copied, modified, or distributed except according to those terms.
{.used.}
import
unittest2,
../beacon_chain/spec/forks,
finish eip4844 to deneb module rename (#4705)
2023-03-09 00:34:17 +00:00
../beacon_chain/spec/datatypes/[phase0, deneb],
Harden handling of unviable forks (#3312) * Harden handling of unviable forks In our current handling of unviable forks, we allow peers to send us blocks that come from a different fork - this is not necessarily an error as it can happen naturally, but it does open up the client to a case where the same unviable fork keeps getting requested - rather than allowing this to happen, we'll now give these peers a small negative score - if it keeps happening, we'll disconnect them. * keep track of unviable forks in quarantine, to avoid filling it with known junk * collect peer scores in single module * descore peers when they send unviable blocks during sync * don't give score for duplicate blocks * increase quarantine size to a level that allows finality to happen under optimal conditions - this helps avoid downloading the same blocks over and over in case of an unviable fork * increase initial score for new peers to make room for one more failure before disconnection * log and score invalid/unviable blocks in requestmanager too * avoid ChainDAG dependency in quarantine * reject gossip blocks with unviable parent * continue processing unviable sync blocks in order to build unviable dag * docs * Update beacon_chain/consensus_object_pools/block_pools_types.nim * add unviable queue test
2022-01-26 12:20:08 +00:00
../beacon_chain/consensus_object_pools/block_quarantine
proc -> func, mainly in spec/state transition and adjecent modules (#3405)
2022-02-17 11:53:55 +00:00
func makeBlock(slot: Slot, parent: Eth2Digest): ForkedSignedBeaconBlock =
Harden handling of unviable forks (#3312) * Harden handling of unviable forks In our current handling of unviable forks, we allow peers to send us blocks that come from a different fork - this is not necessarily an error as it can happen naturally, but it does open up the client to a case where the same unviable fork keeps getting requested - rather than allowing this to happen, we'll now give these peers a small negative score - if it keeps happening, we'll disconnect them. * keep track of unviable forks in quarantine, to avoid filling it with known junk * collect peer scores in single module * descore peers when they send unviable blocks during sync * don't give score for duplicate blocks * increase quarantine size to a level that allows finality to happen under optimal conditions - this helps avoid downloading the same blocks over and over in case of an unviable fork * increase initial score for new peers to make room for one more failure before disconnection * log and score invalid/unviable blocks in requestmanager too * avoid ChainDAG dependency in quarantine * reject gossip blocks with unviable parent * continue processing unviable sync blocks in order to build unviable dag * docs * Update beacon_chain/consensus_object_pools/block_pools_types.nim * add unviable queue test
2022-01-26 12:20:08 +00:00
var
b = phase0.SignedBeaconBlock(
message: phase0.BeaconBlock(slot: slot, parent_root: parent))
b.root = hash_tree_root(b.message)
ForkedSignedBeaconBlock.init(b)
Quarantine and reassembly of gossiped blobs and blocks (#4808)
2023-04-13 19:11:40 +00:00
func makeBlobbyBlock(slot: Slot, parent: Eth2Digest): deneb.SignedBeaconBlock =
var
b = deneb.SignedBeaconBlock(
message: deneb.BeaconBlock(slot: slot, parent_root: parent))
b.root = hash_tree_root(b.message)
b
Harden handling of unviable forks (#3312) * Harden handling of unviable forks In our current handling of unviable forks, we allow peers to send us blocks that come from a different fork - this is not necessarily an error as it can happen naturally, but it does open up the client to a case where the same unviable fork keeps getting requested - rather than allowing this to happen, we'll now give these peers a small negative score - if it keeps happening, we'll disconnect them. * keep track of unviable forks in quarantine, to avoid filling it with known junk * collect peer scores in single module * descore peers when they send unviable blocks during sync * don't give score for duplicate blocks * increase quarantine size to a level that allows finality to happen under optimal conditions - this helps avoid downloading the same blocks over and over in case of an unviable fork * increase initial score for new peers to make room for one more failure before disconnection * log and score invalid/unviable blocks in requestmanager too * avoid ChainDAG dependency in quarantine * reject gossip blocks with unviable parent * continue processing unviable sync blocks in order to build unviable dag * docs * Update beacon_chain/consensus_object_pools/block_pools_types.nim * add unviable queue test
2022-01-26 12:20:08 +00:00
suite "Block quarantine":
test "Unviable smoke test":
let
use ZERO_HASH for default(Eth2Digest)/Eth2Digest() in func calls (#3770)
2022-06-18 04:57:37 +00:00
b0 = makeBlock(Slot 0, ZERO_HASH)
Harden handling of unviable forks (#3312) * Harden handling of unviable forks In our current handling of unviable forks, we allow peers to send us blocks that come from a different fork - this is not necessarily an error as it can happen naturally, but it does open up the client to a case where the same unviable fork keeps getting requested - rather than allowing this to happen, we'll now give these peers a small negative score - if it keeps happening, we'll disconnect them. * keep track of unviable forks in quarantine, to avoid filling it with known junk * collect peer scores in single module * descore peers when they send unviable blocks during sync * don't give score for duplicate blocks * increase quarantine size to a level that allows finality to happen under optimal conditions - this helps avoid downloading the same blocks over and over in case of an unviable fork * increase initial score for new peers to make room for one more failure before disconnection * log and score invalid/unviable blocks in requestmanager too * avoid ChainDAG dependency in quarantine * reject gossip blocks with unviable parent * continue processing unviable sync blocks in order to build unviable dag * docs * Update beacon_chain/consensus_object_pools/block_pools_types.nim * add unviable queue test
2022-01-26 12:20:08 +00:00
b1 = makeBlock(Slot 1, b0.root)
b2 = makeBlock(Slot 2, b1.root)
b3 = makeBlock(Slot 3, b2.root)
b4 = makeBlock(Slot 4, b2.root)
Quarantine and reassembly of gossiped blobs and blocks (#4808)
2023-04-13 19:11:40 +00:00
b5 = makeBlobbyBlock(Slot 4, b3.root)
b6 = makeBlobbyBlock(Slot 4, b4.root)
Harden handling of unviable forks (#3312) * Harden handling of unviable forks In our current handling of unviable forks, we allow peers to send us blocks that come from a different fork - this is not necessarily an error as it can happen naturally, but it does open up the client to a case where the same unviable fork keeps getting requested - rather than allowing this to happen, we'll now give these peers a small negative score - if it keeps happening, we'll disconnect them. * keep track of unviable forks in quarantine, to avoid filling it with known junk * collect peer scores in single module * descore peers when they send unviable blocks during sync * don't give score for duplicate blocks * increase quarantine size to a level that allows finality to happen under optimal conditions - this helps avoid downloading the same blocks over and over in case of an unviable fork * increase initial score for new peers to make room for one more failure before disconnection * log and score invalid/unviable blocks in requestmanager too * avoid ChainDAG dependency in quarantine * reject gossip blocks with unviable parent * continue processing unviable sync blocks in order to build unviable dag * docs * Update beacon_chain/consensus_object_pools/block_pools_types.nim * add unviable queue test
2022-01-26 12:20:08 +00:00
var quarantine: Quarantine
quarantine.addMissing(b1.root)
check:
Fetch by-root request directly from quarantine (#5167) When the requestmanager is busy fetching blocks, the queue might get filled with multiple entries of the same root - since there is no deduplication, requests containing the same root multiple times will be sent out. Also, because the items sit in the queue for a long time potentially, the request might be stale by the time that the manager is ready with the previous request. This PR removes the queue and directly fetches the blocks to download from the quarantine which solves both problems (the quarantine already de-duplicates and is clean of stale information). Removing the queue for blobs is left for a future PR. Co-authored-by: tersec <tersec@users.noreply.github.com>
2023-07-11 16:22:02 +00:00
FetchRecord(root: b1.root) in quarantine.checkMissing(32)
Harden handling of unviable forks (#3312) * Harden handling of unviable forks In our current handling of unviable forks, we allow peers to send us blocks that come from a different fork - this is not necessarily an error as it can happen naturally, but it does open up the client to a case where the same unviable fork keeps getting requested - rather than allowing this to happen, we'll now give these peers a small negative score - if it keeps happening, we'll disconnect them. * keep track of unviable forks in quarantine, to avoid filling it with known junk * collect peer scores in single module * descore peers when they send unviable blocks during sync * don't give score for duplicate blocks * increase quarantine size to a level that allows finality to happen under optimal conditions - this helps avoid downloading the same blocks over and over in case of an unviable fork * increase initial score for new peers to make room for one more failure before disconnection * log and score invalid/unviable blocks in requestmanager too * avoid ChainDAG dependency in quarantine * reject gossip blocks with unviable parent * continue processing unviable sync blocks in order to build unviable dag * docs * Update beacon_chain/consensus_object_pools/block_pools_types.nim * add unviable queue test
2022-01-26 12:20:08 +00:00
Clarify addOrphan error/logging (#4981) * Clarify addOrphan error/logging addOrphan returned a bool to indicate success. Change this to a Result so that different errors can be distinguished. * Update beacon_chain/consensus_object_pools/block_quarantine.nim Co-authored-by: tersec <tersec@users.noreply.github.com> * Update beacon_chain/gossip_processing/gossip_validation.nim --------- Co-authored-by: tersec <tersec@users.noreply.github.com>
2023-05-21 17:47:00 +00:00
quarantine.addOrphan(Slot 0, b1).isOk
Harden handling of unviable forks (#3312) * Harden handling of unviable forks In our current handling of unviable forks, we allow peers to send us blocks that come from a different fork - this is not necessarily an error as it can happen naturally, but it does open up the client to a case where the same unviable fork keeps getting requested - rather than allowing this to happen, we'll now give these peers a small negative score - if it keeps happening, we'll disconnect them. * keep track of unviable forks in quarantine, to avoid filling it with known junk * collect peer scores in single module * descore peers when they send unviable blocks during sync * don't give score for duplicate blocks * increase quarantine size to a level that allows finality to happen under optimal conditions - this helps avoid downloading the same blocks over and over in case of an unviable fork * increase initial score for new peers to make room for one more failure before disconnection * log and score invalid/unviable blocks in requestmanager too * avoid ChainDAG dependency in quarantine * reject gossip blocks with unviable parent * continue processing unviable sync blocks in order to build unviable dag * docs * Update beacon_chain/consensus_object_pools/block_pools_types.nim * add unviable queue test
2022-01-26 12:20:08 +00:00
Fetch by-root request directly from quarantine (#5167) When the requestmanager is busy fetching blocks, the queue might get filled with multiple entries of the same root - since there is no deduplication, requests containing the same root multiple times will be sent out. Also, because the items sit in the queue for a long time potentially, the request might be stale by the time that the manager is ready with the previous request. This PR removes the queue and directly fetches the blocks to download from the quarantine which solves both problems (the quarantine already de-duplicates and is clean of stale information). Removing the queue for blobs is left for a future PR. Co-authored-by: tersec <tersec@users.noreply.github.com>
2023-07-11 16:22:02 +00:00
FetchRecord(root: b1.root) notin quarantine.checkMissing(32)
Harden handling of unviable forks (#3312) * Harden handling of unviable forks In our current handling of unviable forks, we allow peers to send us blocks that come from a different fork - this is not necessarily an error as it can happen naturally, but it does open up the client to a case where the same unviable fork keeps getting requested - rather than allowing this to happen, we'll now give these peers a small negative score - if it keeps happening, we'll disconnect them. * keep track of unviable forks in quarantine, to avoid filling it with known junk * collect peer scores in single module * descore peers when they send unviable blocks during sync * don't give score for duplicate blocks * increase quarantine size to a level that allows finality to happen under optimal conditions - this helps avoid downloading the same blocks over and over in case of an unviable fork * increase initial score for new peers to make room for one more failure before disconnection * log and score invalid/unviable blocks in requestmanager too * avoid ChainDAG dependency in quarantine * reject gossip blocks with unviable parent * continue processing unviable sync blocks in order to build unviable dag * docs * Update beacon_chain/consensus_object_pools/block_pools_types.nim * add unviable queue test
2022-01-26 12:20:08 +00:00
Clarify addOrphan error/logging (#4981) * Clarify addOrphan error/logging addOrphan returned a bool to indicate success. Change this to a Result so that different errors can be distinguished. * Update beacon_chain/consensus_object_pools/block_quarantine.nim Co-authored-by: tersec <tersec@users.noreply.github.com> * Update beacon_chain/gossip_processing/gossip_validation.nim --------- Co-authored-by: tersec <tersec@users.noreply.github.com>
2023-05-21 17:47:00 +00:00
quarantine.addOrphan(Slot 0, b2).isOk
quarantine.addOrphan(Slot 0, b3).isOk
quarantine.addOrphan(Slot 0, b4).isOk
Harden handling of unviable forks (#3312) * Harden handling of unviable forks In our current handling of unviable forks, we allow peers to send us blocks that come from a different fork - this is not necessarily an error as it can happen naturally, but it does open up the client to a case where the same unviable fork keeps getting requested - rather than allowing this to happen, we'll now give these peers a small negative score - if it keeps happening, we'll disconnect them. * keep track of unviable forks in quarantine, to avoid filling it with known junk * collect peer scores in single module * descore peers when they send unviable blocks during sync * don't give score for duplicate blocks * increase quarantine size to a level that allows finality to happen under optimal conditions - this helps avoid downloading the same blocks over and over in case of an unviable fork * increase initial score for new peers to make room for one more failure before disconnection * log and score invalid/unviable blocks in requestmanager too * avoid ChainDAG dependency in quarantine * reject gossip blocks with unviable parent * continue processing unviable sync blocks in order to build unviable dag * docs * Update beacon_chain/consensus_object_pools/block_pools_types.nim * add unviable queue test
2022-01-26 12:20:08 +00:00
Quarantine and reassembly of gossiped blobs and blocks (#4808)
2023-04-13 19:11:40 +00:00
quarantine.addBlobless(Slot 0, b5)
quarantine.addBlobless(Slot 0, b6)
Harden handling of unviable forks (#3312) * Harden handling of unviable forks In our current handling of unviable forks, we allow peers to send us blocks that come from a different fork - this is not necessarily an error as it can happen naturally, but it does open up the client to a case where the same unviable fork keeps getting requested - rather than allowing this to happen, we'll now give these peers a small negative score - if it keeps happening, we'll disconnect them. * keep track of unviable forks in quarantine, to avoid filling it with known junk * collect peer scores in single module * descore peers when they send unviable blocks during sync * don't give score for duplicate blocks * increase quarantine size to a level that allows finality to happen under optimal conditions - this helps avoid downloading the same blocks over and over in case of an unviable fork * increase initial score for new peers to make room for one more failure before disconnection * log and score invalid/unviable blocks in requestmanager too * avoid ChainDAG dependency in quarantine * reject gossip blocks with unviable parent * continue processing unviable sync blocks in order to build unviable dag * docs * Update beacon_chain/consensus_object_pools/block_pools_types.nim * add unviable queue test
2022-01-26 12:20:08 +00:00
(b4.root, ValidatorSig()) in quarantine.orphans
Simplify block quarantine blobless (#4824) * Simplify block quarantine blobless The quarantine blobless table was initially keyed off of (Eth2Digest, ValidatorSig). This was modelled off the orphan table. The presence of the signature in the key is necessary for orphans, because we can't verify the signature for an orphan. That is not the case for a blobless block, where the signature can be verified. So this PR changes the blobless block table to be keyed off a Eth2Digest only. This simplifies the retrieval and handling of blobless blocks. * review feedback
2023-04-16 08:37:56 +00:00
b5.root in quarantine.blobless
b6.root in quarantine.blobless
Harden handling of unviable forks (#3312) * Harden handling of unviable forks In our current handling of unviable forks, we allow peers to send us blocks that come from a different fork - this is not necessarily an error as it can happen naturally, but it does open up the client to a case where the same unviable fork keeps getting requested - rather than allowing this to happen, we'll now give these peers a small negative score - if it keeps happening, we'll disconnect them. * keep track of unviable forks in quarantine, to avoid filling it with known junk * collect peer scores in single module * descore peers when they send unviable blocks during sync * don't give score for duplicate blocks * increase quarantine size to a level that allows finality to happen under optimal conditions - this helps avoid downloading the same blocks over and over in case of an unviable fork * increase initial score for new peers to make room for one more failure before disconnection * log and score invalid/unviable blocks in requestmanager too * avoid ChainDAG dependency in quarantine * reject gossip blocks with unviable parent * continue processing unviable sync blocks in order to build unviable dag * docs * Update beacon_chain/consensus_object_pools/block_pools_types.nim * add unviable queue test
2022-01-26 12:20:08 +00:00
quarantine.addUnviable(b4.root)
check:
(b4.root, ValidatorSig()) notin quarantine.orphans
Simplify block quarantine blobless (#4824) * Simplify block quarantine blobless The quarantine blobless table was initially keyed off of (Eth2Digest, ValidatorSig). This was modelled off the orphan table. The presence of the signature in the key is necessary for orphans, because we can't verify the signature for an orphan. That is not the case for a blobless block, where the signature can be verified. So this PR changes the blobless block table to be keyed off a Eth2Digest only. This simplifies the retrieval and handling of blobless blocks. * review feedback
2023-04-16 08:37:56 +00:00
b5.root in quarantine.blobless
b6.root notin quarantine.blobless
Quarantine and reassembly of gossiped blobs and blocks (#4808)
2023-04-13 19:11:40 +00:00
Harden handling of unviable forks (#3312) * Harden handling of unviable forks In our current handling of unviable forks, we allow peers to send us blocks that come from a different fork - this is not necessarily an error as it can happen naturally, but it does open up the client to a case where the same unviable fork keeps getting requested - rather than allowing this to happen, we'll now give these peers a small negative score - if it keeps happening, we'll disconnect them. * keep track of unviable forks in quarantine, to avoid filling it with known junk * collect peer scores in single module * descore peers when they send unviable blocks during sync * don't give score for duplicate blocks * increase quarantine size to a level that allows finality to happen under optimal conditions - this helps avoid downloading the same blocks over and over in case of an unviable fork * increase initial score for new peers to make room for one more failure before disconnection * log and score invalid/unviable blocks in requestmanager too * avoid ChainDAG dependency in quarantine * reject gossip blocks with unviable parent * continue processing unviable sync blocks in order to build unviable dag * docs * Update beacon_chain/consensus_object_pools/block_pools_types.nim * add unviable queue test
2022-01-26 12:20:08 +00:00
quarantine.addUnviable(b1.root)
check:
(b1.root, ValidatorSig()) notin quarantine.orphans
(b2.root, ValidatorSig()) notin quarantine.orphans
(b3.root, ValidatorSig()) notin quarantine.orphans
Quarantine and reassembly of gossiped blobs and blocks (#4808)
2023-04-13 19:11:40 +00:00
Simplify block quarantine blobless (#4824) * Simplify block quarantine blobless The quarantine blobless table was initially keyed off of (Eth2Digest, ValidatorSig). This was modelled off the orphan table. The presence of the signature in the key is necessary for orphans, because we can't verify the signature for an orphan. That is not the case for a blobless block, where the signature can be verified. So this PR changes the blobless block table to be keyed off a Eth2Digest only. This simplifies the retrieval and handling of blobless blocks. * review feedback
2023-04-16 08:37:56 +00:00
b5.root notin quarantine.blobless
b6.root notin quarantine.blobless