nimbus-eth1/nimbus/sync/protocol/eth66.nim

315 lines
11 KiB
Nim
Raw Normal View History

# Nimbus - Ethereum Wire Protocol
Sync: Support for `eth/65` protocol This patch adds the `eth/65` protocol, documented at https://github.com/ethereum/devp2p/blob/master/caps/eth.md. This is an intentionally simple patch, designed to not break, change or depend on other functionality much, so that the "_old_ sync" methods can be run usefully again and observed. This patch isn't "new sync" (a different set of sync algorithms), but it is one of the foundations. For a while now Nimbus Eth1 only supported protocol `eth/63`. But that's obsolete, and very few nodes still support it. This meant Nimbus Eth1 would make slow progress trying to sync, as most up to date clients rejected it. The current specification is `eth/66`, and the functionality we really need is in `eth/64`. So why `eth/65`? - `eth/64` is essential because of the `forkId` feature. But `eth/64` is on its way out as well. Many clients, especially the most up to date Geth running the current hard-forks (Berlin/London) don't talk `eth/64` any more. - `eth/66` is the current specification, but some clients don't talk `eth/66` yet. We'd like to have the best peer connectivity during tests, and currently everything that talks `eth/66` also talks `eth/65`. - Nimbus Eth1 RLPx only talks one version at a time. (Without changes to the RLPx module. When those go in we'll add `eth/64..eth/66` for greater peer reach and testing the `eth/66` behaviour. For simplicity and decoupling, this patch contains just one version, the most useful.) What are `eth/64` and `eth/65`? - `eth/64` (EIP-2364) added `forkId` which allows nodes to distinguish between Ethereum (ETH) and Ethereum Classic (ETC) blockchains, which share the same genesis block. `forkId` also protects the system when a new hard fork is going to be rolled out, by blocking interaction with out of date nodes. The feature is required nowadays. We send the right details to allow connection (this has been tested a lot), but don't apply the full validation rules of EIP-2124/EIP-2364 in this patch. It's to keep this patch simple (in its effects) and because those rules have consequences best tested separately. In practice the other node will reject us when we would reject it, so this is ok for testing, as long as it doesn't get seriously deployed. - `eth/65` added more efficient transaction pool methods. - Then a later version of `eth/65` (without a new number) added typed transactions, described in [EIP-2976](https://eips.ethereum.org/EIPS/eip-2976). Why it's moved to `nimbus-eth1`: - Mainly because `eth/64` onwards depend on the current state of block synchronisation, as well as the blockchain's sequence of hard-fork block numbers, both of which are part of `nimbus-eth1` run-time state. These aren't available to pure `nim-eth` code. Although it would be possible to add an API to let `nimbus-eth1` set these numbers, there isn't any point because the protocol would still only be useful to `nimbus-eth1`. Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-07-21 20:00:03 +00:00
#
# Copyright (c) 2018-2024 Status Research & Development GmbH
Sync: Support for `eth/65` protocol This patch adds the `eth/65` protocol, documented at https://github.com/ethereum/devp2p/blob/master/caps/eth.md. This is an intentionally simple patch, designed to not break, change or depend on other functionality much, so that the "_old_ sync" methods can be run usefully again and observed. This patch isn't "new sync" (a different set of sync algorithms), but it is one of the foundations. For a while now Nimbus Eth1 only supported protocol `eth/63`. But that's obsolete, and very few nodes still support it. This meant Nimbus Eth1 would make slow progress trying to sync, as most up to date clients rejected it. The current specification is `eth/66`, and the functionality we really need is in `eth/64`. So why `eth/65`? - `eth/64` is essential because of the `forkId` feature. But `eth/64` is on its way out as well. Many clients, especially the most up to date Geth running the current hard-forks (Berlin/London) don't talk `eth/64` any more. - `eth/66` is the current specification, but some clients don't talk `eth/66` yet. We'd like to have the best peer connectivity during tests, and currently everything that talks `eth/66` also talks `eth/65`. - Nimbus Eth1 RLPx only talks one version at a time. (Without changes to the RLPx module. When those go in we'll add `eth/64..eth/66` for greater peer reach and testing the `eth/66` behaviour. For simplicity and decoupling, this patch contains just one version, the most useful.) What are `eth/64` and `eth/65`? - `eth/64` (EIP-2364) added `forkId` which allows nodes to distinguish between Ethereum (ETH) and Ethereum Classic (ETC) blockchains, which share the same genesis block. `forkId` also protects the system when a new hard fork is going to be rolled out, by blocking interaction with out of date nodes. The feature is required nowadays. We send the right details to allow connection (this has been tested a lot), but don't apply the full validation rules of EIP-2124/EIP-2364 in this patch. It's to keep this patch simple (in its effects) and because those rules have consequences best tested separately. In practice the other node will reject us when we would reject it, so this is ok for testing, as long as it doesn't get seriously deployed. - `eth/65` added more efficient transaction pool methods. - Then a later version of `eth/65` (without a new number) added typed transactions, described in [EIP-2976](https://eips.ethereum.org/EIPS/eip-2976). Why it's moved to `nimbus-eth1`: - Mainly because `eth/64` onwards depend on the current state of block synchronisation, as well as the blockchain's sequence of hard-fork block numbers, both of which are part of `nimbus-eth1` run-time state. These aren't available to pure `nim-eth` code. Although it would be possible to add an API to let `nimbus-eth1` set these numbers, there isn't any point because the protocol would still only be useful to `nimbus-eth1`. Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-07-21 20:00:03 +00:00
# Licensed under either of
# * Apache License, version 2.0, ([LICENSE-APACHE](LICENSE-APACHE) or
# http://www.apache.org/licenses/LICENSE-2.0)
# * MIT license ([LICENSE-MIT](LICENSE-MIT) or
# http://opensource.org/licenses/MIT)
# at your option. This file may not be copied, modified, or distributed
# except according to those terms.
## This module implements Ethereum Wire Protocol version 66, `eth/66`.
## Specification:
## `eth/66 <https://github.com/ethereum/devp2p/blob/master/caps/eth.md>`_
Sync: Support for `eth/65` protocol This patch adds the `eth/65` protocol, documented at https://github.com/ethereum/devp2p/blob/master/caps/eth.md. This is an intentionally simple patch, designed to not break, change or depend on other functionality much, so that the "_old_ sync" methods can be run usefully again and observed. This patch isn't "new sync" (a different set of sync algorithms), but it is one of the foundations. For a while now Nimbus Eth1 only supported protocol `eth/63`. But that's obsolete, and very few nodes still support it. This meant Nimbus Eth1 would make slow progress trying to sync, as most up to date clients rejected it. The current specification is `eth/66`, and the functionality we really need is in `eth/64`. So why `eth/65`? - `eth/64` is essential because of the `forkId` feature. But `eth/64` is on its way out as well. Many clients, especially the most up to date Geth running the current hard-forks (Berlin/London) don't talk `eth/64` any more. - `eth/66` is the current specification, but some clients don't talk `eth/66` yet. We'd like to have the best peer connectivity during tests, and currently everything that talks `eth/66` also talks `eth/65`. - Nimbus Eth1 RLPx only talks one version at a time. (Without changes to the RLPx module. When those go in we'll add `eth/64..eth/66` for greater peer reach and testing the `eth/66` behaviour. For simplicity and decoupling, this patch contains just one version, the most useful.) What are `eth/64` and `eth/65`? - `eth/64` (EIP-2364) added `forkId` which allows nodes to distinguish between Ethereum (ETH) and Ethereum Classic (ETC) blockchains, which share the same genesis block. `forkId` also protects the system when a new hard fork is going to be rolled out, by blocking interaction with out of date nodes. The feature is required nowadays. We send the right details to allow connection (this has been tested a lot), but don't apply the full validation rules of EIP-2124/EIP-2364 in this patch. It's to keep this patch simple (in its effects) and because those rules have consequences best tested separately. In practice the other node will reject us when we would reject it, so this is ok for testing, as long as it doesn't get seriously deployed. - `eth/65` added more efficient transaction pool methods. - Then a later version of `eth/65` (without a new number) added typed transactions, described in [EIP-2976](https://eips.ethereum.org/EIPS/eip-2976). Why it's moved to `nimbus-eth1`: - Mainly because `eth/64` onwards depend on the current state of block synchronisation, as well as the blockchain's sequence of hard-fork block numbers, both of which are part of `nimbus-eth1` run-time state. These aren't available to pure `nim-eth` code. Although it would be possible to add an API to let `nimbus-eth1` set these numbers, there isn't any point because the protocol would still only be useful to `nimbus-eth1`. Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-07-21 20:00:03 +00:00
import
stint,
chronicles,
chronos,
eth/[common, p2p, p2p/private/p2p_types],
stew/byteutils,
./trace_config,
./eth/eth_types,
../types,
2022-12-02 04:39:12 +00:00
../../utils/utils
export
eth_types
Sync: Support for `eth/65` protocol This patch adds the `eth/65` protocol, documented at https://github.com/ethereum/devp2p/blob/master/caps/eth.md. This is an intentionally simple patch, designed to not break, change or depend on other functionality much, so that the "_old_ sync" methods can be run usefully again and observed. This patch isn't "new sync" (a different set of sync algorithms), but it is one of the foundations. For a while now Nimbus Eth1 only supported protocol `eth/63`. But that's obsolete, and very few nodes still support it. This meant Nimbus Eth1 would make slow progress trying to sync, as most up to date clients rejected it. The current specification is `eth/66`, and the functionality we really need is in `eth/64`. So why `eth/65`? - `eth/64` is essential because of the `forkId` feature. But `eth/64` is on its way out as well. Many clients, especially the most up to date Geth running the current hard-forks (Berlin/London) don't talk `eth/64` any more. - `eth/66` is the current specification, but some clients don't talk `eth/66` yet. We'd like to have the best peer connectivity during tests, and currently everything that talks `eth/66` also talks `eth/65`. - Nimbus Eth1 RLPx only talks one version at a time. (Without changes to the RLPx module. When those go in we'll add `eth/64..eth/66` for greater peer reach and testing the `eth/66` behaviour. For simplicity and decoupling, this patch contains just one version, the most useful.) What are `eth/64` and `eth/65`? - `eth/64` (EIP-2364) added `forkId` which allows nodes to distinguish between Ethereum (ETH) and Ethereum Classic (ETC) blockchains, which share the same genesis block. `forkId` also protects the system when a new hard fork is going to be rolled out, by blocking interaction with out of date nodes. The feature is required nowadays. We send the right details to allow connection (this has been tested a lot), but don't apply the full validation rules of EIP-2124/EIP-2364 in this patch. It's to keep this patch simple (in its effects) and because those rules have consequences best tested separately. In practice the other node will reject us when we would reject it, so this is ok for testing, as long as it doesn't get seriously deployed. - `eth/65` added more efficient transaction pool methods. - Then a later version of `eth/65` (without a new number) added typed transactions, described in [EIP-2976](https://eips.ethereum.org/EIPS/eip-2976). Why it's moved to `nimbus-eth1`: - Mainly because `eth/64` onwards depend on the current state of block synchronisation, as well as the blockchain's sequence of hard-fork block numbers, both of which are part of `nimbus-eth1` run-time state. These aren't available to pure `nim-eth` code. Although it would be possible to add an API to let `nimbus-eth1` set these numbers, there isn't any point because the protocol would still only be useful to `nimbus-eth1`. Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-07-21 20:00:03 +00:00
logScope:
topics = "eth66"
static:
const stopCompilerGossip {.used.} = Hash256().toHex
Sync: Support for `eth/65` protocol This patch adds the `eth/65` protocol, documented at https://github.com/ethereum/devp2p/blob/master/caps/eth.md. This is an intentionally simple patch, designed to not break, change or depend on other functionality much, so that the "_old_ sync" methods can be run usefully again and observed. This patch isn't "new sync" (a different set of sync algorithms), but it is one of the foundations. For a while now Nimbus Eth1 only supported protocol `eth/63`. But that's obsolete, and very few nodes still support it. This meant Nimbus Eth1 would make slow progress trying to sync, as most up to date clients rejected it. The current specification is `eth/66`, and the functionality we really need is in `eth/64`. So why `eth/65`? - `eth/64` is essential because of the `forkId` feature. But `eth/64` is on its way out as well. Many clients, especially the most up to date Geth running the current hard-forks (Berlin/London) don't talk `eth/64` any more. - `eth/66` is the current specification, but some clients don't talk `eth/66` yet. We'd like to have the best peer connectivity during tests, and currently everything that talks `eth/66` also talks `eth/65`. - Nimbus Eth1 RLPx only talks one version at a time. (Without changes to the RLPx module. When those go in we'll add `eth/64..eth/66` for greater peer reach and testing the `eth/66` behaviour. For simplicity and decoupling, this patch contains just one version, the most useful.) What are `eth/64` and `eth/65`? - `eth/64` (EIP-2364) added `forkId` which allows nodes to distinguish between Ethereum (ETH) and Ethereum Classic (ETC) blockchains, which share the same genesis block. `forkId` also protects the system when a new hard fork is going to be rolled out, by blocking interaction with out of date nodes. The feature is required nowadays. We send the right details to allow connection (this has been tested a lot), but don't apply the full validation rules of EIP-2124/EIP-2364 in this patch. It's to keep this patch simple (in its effects) and because those rules have consequences best tested separately. In practice the other node will reject us when we would reject it, so this is ok for testing, as long as it doesn't get seriously deployed. - `eth/65` added more efficient transaction pool methods. - Then a later version of `eth/65` (without a new number) added typed transactions, described in [EIP-2976](https://eips.ethereum.org/EIPS/eip-2976). Why it's moved to `nimbus-eth1`: - Mainly because `eth/64` onwards depend on the current state of block synchronisation, as well as the blockchain's sequence of hard-fork block numbers, both of which are part of `nimbus-eth1` run-time state. These aren't available to pure `nim-eth` code. Although it would be possible to add an API to let `nimbus-eth1` set these numbers, there isn't any point because the protocol would still only be useful to `nimbus-eth1`. Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-07-21 20:00:03 +00:00
const
ethVersion* = 66
prettyEthProtoName* = "[eth/" & $ethVersion & "]"
Sync: Support for `eth/65` protocol This patch adds the `eth/65` protocol, documented at https://github.com/ethereum/devp2p/blob/master/caps/eth.md. This is an intentionally simple patch, designed to not break, change or depend on other functionality much, so that the "_old_ sync" methods can be run usefully again and observed. This patch isn't "new sync" (a different set of sync algorithms), but it is one of the foundations. For a while now Nimbus Eth1 only supported protocol `eth/63`. But that's obsolete, and very few nodes still support it. This meant Nimbus Eth1 would make slow progress trying to sync, as most up to date clients rejected it. The current specification is `eth/66`, and the functionality we really need is in `eth/64`. So why `eth/65`? - `eth/64` is essential because of the `forkId` feature. But `eth/64` is on its way out as well. Many clients, especially the most up to date Geth running the current hard-forks (Berlin/London) don't talk `eth/64` any more. - `eth/66` is the current specification, but some clients don't talk `eth/66` yet. We'd like to have the best peer connectivity during tests, and currently everything that talks `eth/66` also talks `eth/65`. - Nimbus Eth1 RLPx only talks one version at a time. (Without changes to the RLPx module. When those go in we'll add `eth/64..eth/66` for greater peer reach and testing the `eth/66` behaviour. For simplicity and decoupling, this patch contains just one version, the most useful.) What are `eth/64` and `eth/65`? - `eth/64` (EIP-2364) added `forkId` which allows nodes to distinguish between Ethereum (ETH) and Ethereum Classic (ETC) blockchains, which share the same genesis block. `forkId` also protects the system when a new hard fork is going to be rolled out, by blocking interaction with out of date nodes. The feature is required nowadays. We send the right details to allow connection (this has been tested a lot), but don't apply the full validation rules of EIP-2124/EIP-2364 in this patch. It's to keep this patch simple (in its effects) and because those rules have consequences best tested separately. In practice the other node will reject us when we would reject it, so this is ok for testing, as long as it doesn't get seriously deployed. - `eth/65` added more efficient transaction pool methods. - Then a later version of `eth/65` (without a new number) added typed transactions, described in [EIP-2976](https://eips.ethereum.org/EIPS/eip-2976). Why it's moved to `nimbus-eth1`: - Mainly because `eth/64` onwards depend on the current state of block synchronisation, as well as the blockchain's sequence of hard-fork block numbers, both of which are part of `nimbus-eth1` run-time state. These aren't available to pure `nim-eth` code. Although it would be possible to add an API to let `nimbus-eth1` set these numbers, there isn't any point because the protocol would still only be useful to `nimbus-eth1`. Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-07-21 20:00:03 +00:00
Snap sync simplify object inheritance (#1098) * Reorg SnapPeerBase descriptor, notably start/stop flags details: Instead of using three boolean flags startedFetch, stopped, and stopThisState a single enum type is used with values SyncRunningOk, SyncStopRequest, and SyncStopped. * Restricting snap to eth66 and later why: Id-tracked request/response wire protocol can handle overlapped responses when requests are sent in row. * Align function names with source code file names why: Easier to reconcile when following the implemented logic. * Update trace logging (want file locations) why: The macros previously used hid the relevant file location (when `chroniclesLineNumbers` turned on.) It rather printed the file location of the template that was wrapping `trace`. * Use KeyedQueue table instead of sequence why: Quick access, easy configuration as LRU or FIFO with max entries (currently LRU.) * Dissolve `SnapPeerEx` object extension into `SnapPeer` why; It is logically cleaner and more obvious not to inherit from `SnapPeerBase` but to specify opaque field object references of the merged `SnapPeer` object. These can then be locally inherited. * Dissolve `SnapSyncEx` object extension into `SnapSync` why; It is logically cleaner and more obvious not to inherit from `SnapSyncEx` but to specify opaque field object references of the `SnapPeer` object. These can then be locally inherited. Also, in the re-factored code here the interface descriptor `SnapSyncCtx` inherited `SnapSyncEx` which was sub-optimal (OO inheritance makes it easier to work with call back functions.)
2022-05-23 16:53:19 +00:00
# Pickeled tracer texts
trEthRecvReceived* =
"<< " & prettyEthProtoName & " Received "
trEthRecvReceivedBlockHeaders* =
trEthRecvReceived & "BlockHeaders (0x04)"
trEthRecvReceivedBlockBodies* =
trEthRecvReceived & "BlockBodies (0x06)"
Snap sync simplify object inheritance (#1098) * Reorg SnapPeerBase descriptor, notably start/stop flags details: Instead of using three boolean flags startedFetch, stopped, and stopThisState a single enum type is used with values SyncRunningOk, SyncStopRequest, and SyncStopped. * Restricting snap to eth66 and later why: Id-tracked request/response wire protocol can handle overlapped responses when requests are sent in row. * Align function names with source code file names why: Easier to reconcile when following the implemented logic. * Update trace logging (want file locations) why: The macros previously used hid the relevant file location (when `chroniclesLineNumbers` turned on.) It rather printed the file location of the template that was wrapping `trace`. * Use KeyedQueue table instead of sequence why: Quick access, easy configuration as LRU or FIFO with max entries (currently LRU.) * Dissolve `SnapPeerEx` object extension into `SnapPeer` why; It is logically cleaner and more obvious not to inherit from `SnapPeerBase` but to specify opaque field object references of the merged `SnapPeer` object. These can then be locally inherited. * Dissolve `SnapSyncEx` object extension into `SnapSync` why; It is logically cleaner and more obvious not to inherit from `SnapSyncEx` but to specify opaque field object references of the `SnapPeer` object. These can then be locally inherited. Also, in the re-factored code here the interface descriptor `SnapSyncCtx` inherited `SnapSyncEx` which was sub-optimal (OO inheritance makes it easier to work with call back functions.)
2022-05-23 16:53:19 +00:00
trEthRecvProtocolViolation* =
"<< " & prettyEthProtoName & " Protocol violation, "
trEthRecvError* =
"<< " & prettyEthProtoName & " Error "
trEthRecvTimeoutWaiting* =
"<< " & prettyEthProtoName & " Timeout waiting "
trEthRecvDiscarding* =
"<< " & prettyEthProtoName & " Discarding "
Snap sync simplify object inheritance (#1098) * Reorg SnapPeerBase descriptor, notably start/stop flags details: Instead of using three boolean flags startedFetch, stopped, and stopThisState a single enum type is used with values SyncRunningOk, SyncStopRequest, and SyncStopped. * Restricting snap to eth66 and later why: Id-tracked request/response wire protocol can handle overlapped responses when requests are sent in row. * Align function names with source code file names why: Easier to reconcile when following the implemented logic. * Update trace logging (want file locations) why: The macros previously used hid the relevant file location (when `chroniclesLineNumbers` turned on.) It rather printed the file location of the template that was wrapping `trace`. * Use KeyedQueue table instead of sequence why: Quick access, easy configuration as LRU or FIFO with max entries (currently LRU.) * Dissolve `SnapPeerEx` object extension into `SnapPeer` why; It is logically cleaner and more obvious not to inherit from `SnapPeerBase` but to specify opaque field object references of the merged `SnapPeer` object. These can then be locally inherited. * Dissolve `SnapSyncEx` object extension into `SnapSync` why; It is logically cleaner and more obvious not to inherit from `SnapSyncEx` but to specify opaque field object references of the `SnapPeer` object. These can then be locally inherited. Also, in the re-factored code here the interface descriptor `SnapSyncCtx` inherited `SnapSyncEx` which was sub-optimal (OO inheritance makes it easier to work with call back functions.)
2022-05-23 16:53:19 +00:00
trEthSendSending* =
">> " & prettyEthProtoName & " Sending "
trEthSendSendingGetBlockHeaders* =
trEthSendSending & "GetBlockHeaders (0x03)"
trEthSendSendingGetBlockBodies* =
trEthSendSending & "GetBlockBodies (0x05)"
Snap sync simplify object inheritance (#1098) * Reorg SnapPeerBase descriptor, notably start/stop flags details: Instead of using three boolean flags startedFetch, stopped, and stopThisState a single enum type is used with values SyncRunningOk, SyncStopRequest, and SyncStopped. * Restricting snap to eth66 and later why: Id-tracked request/response wire protocol can handle overlapped responses when requests are sent in row. * Align function names with source code file names why: Easier to reconcile when following the implemented logic. * Update trace logging (want file locations) why: The macros previously used hid the relevant file location (when `chroniclesLineNumbers` turned on.) It rather printed the file location of the template that was wrapping `trace`. * Use KeyedQueue table instead of sequence why: Quick access, easy configuration as LRU or FIFO with max entries (currently LRU.) * Dissolve `SnapPeerEx` object extension into `SnapPeer` why; It is logically cleaner and more obvious not to inherit from `SnapPeerBase` but to specify opaque field object references of the merged `SnapPeer` object. These can then be locally inherited. * Dissolve `SnapSyncEx` object extension into `SnapSync` why; It is logically cleaner and more obvious not to inherit from `SnapSyncEx` but to specify opaque field object references of the `SnapPeer` object. These can then be locally inherited. Also, in the re-factored code here the interface descriptor `SnapSyncCtx` inherited `SnapSyncEx` which was sub-optimal (OO inheritance makes it easier to work with call back functions.)
2022-05-23 16:53:19 +00:00
trEthSendReplying* =
">> " & prettyEthProtoName & " Replying "
Snap sync simplify object inheritance (#1098) * Reorg SnapPeerBase descriptor, notably start/stop flags details: Instead of using three boolean flags startedFetch, stopped, and stopThisState a single enum type is used with values SyncRunningOk, SyncStopRequest, and SyncStopped. * Restricting snap to eth66 and later why: Id-tracked request/response wire protocol can handle overlapped responses when requests are sent in row. * Align function names with source code file names why: Easier to reconcile when following the implemented logic. * Update trace logging (want file locations) why: The macros previously used hid the relevant file location (when `chroniclesLineNumbers` turned on.) It rather printed the file location of the template that was wrapping `trace`. * Use KeyedQueue table instead of sequence why: Quick access, easy configuration as LRU or FIFO with max entries (currently LRU.) * Dissolve `SnapPeerEx` object extension into `SnapPeer` why; It is logically cleaner and more obvious not to inherit from `SnapPeerBase` but to specify opaque field object references of the merged `SnapPeer` object. These can then be locally inherited. * Dissolve `SnapSyncEx` object extension into `SnapSync` why; It is logically cleaner and more obvious not to inherit from `SnapSyncEx` but to specify opaque field object references of the `SnapPeer` object. These can then be locally inherited. Also, in the re-factored code here the interface descriptor `SnapSyncCtx` inherited `SnapSyncEx` which was sub-optimal (OO inheritance makes it easier to work with call back functions.)
2022-05-23 16:53:19 +00:00
trEthSendDelaying* =
">> " & prettyEthProtoName & " Delaying "
trEthRecvNewBlock* =
"<< " & prettyEthProtoName & " Received NewBlock"
trEthRecvNewBlockHashes* =
"<< " & prettyEthProtoName & " Received NewBlockHashes"
trEthSendNewBlock* =
">> " & prettyEthProtoName & " Sending NewBlock"
trEthSendNewBlockHashes* =
">> " & prettyEthProtoName & " Sending NewBlockHashes"
template handleHandlerError(x: untyped) =
if x.isErr:
raise newException(EthP2PError, x.error)
p2pProtocol eth66(version = ethVersion,
rlpxName = "eth",
peerState = EthPeerState,
networkState = EthWireBase,
useRequestIds = true):
Sync: Support for `eth/65` protocol This patch adds the `eth/65` protocol, documented at https://github.com/ethereum/devp2p/blob/master/caps/eth.md. This is an intentionally simple patch, designed to not break, change or depend on other functionality much, so that the "_old_ sync" methods can be run usefully again and observed. This patch isn't "new sync" (a different set of sync algorithms), but it is one of the foundations. For a while now Nimbus Eth1 only supported protocol `eth/63`. But that's obsolete, and very few nodes still support it. This meant Nimbus Eth1 would make slow progress trying to sync, as most up to date clients rejected it. The current specification is `eth/66`, and the functionality we really need is in `eth/64`. So why `eth/65`? - `eth/64` is essential because of the `forkId` feature. But `eth/64` is on its way out as well. Many clients, especially the most up to date Geth running the current hard-forks (Berlin/London) don't talk `eth/64` any more. - `eth/66` is the current specification, but some clients don't talk `eth/66` yet. We'd like to have the best peer connectivity during tests, and currently everything that talks `eth/66` also talks `eth/65`. - Nimbus Eth1 RLPx only talks one version at a time. (Without changes to the RLPx module. When those go in we'll add `eth/64..eth/66` for greater peer reach and testing the `eth/66` behaviour. For simplicity and decoupling, this patch contains just one version, the most useful.) What are `eth/64` and `eth/65`? - `eth/64` (EIP-2364) added `forkId` which allows nodes to distinguish between Ethereum (ETH) and Ethereum Classic (ETC) blockchains, which share the same genesis block. `forkId` also protects the system when a new hard fork is going to be rolled out, by blocking interaction with out of date nodes. The feature is required nowadays. We send the right details to allow connection (this has been tested a lot), but don't apply the full validation rules of EIP-2124/EIP-2364 in this patch. It's to keep this patch simple (in its effects) and because those rules have consequences best tested separately. In practice the other node will reject us when we would reject it, so this is ok for testing, as long as it doesn't get seriously deployed. - `eth/65` added more efficient transaction pool methods. - Then a later version of `eth/65` (without a new number) added typed transactions, described in [EIP-2976](https://eips.ethereum.org/EIPS/eip-2976). Why it's moved to `nimbus-eth1`: - Mainly because `eth/64` onwards depend on the current state of block synchronisation, as well as the blockchain's sequence of hard-fork block numbers, both of which are part of `nimbus-eth1` run-time state. These aren't available to pure `nim-eth` code. Although it would be possible to add an API to let `nimbus-eth1` set these numbers, there isn't any point because the protocol would still only be useful to `nimbus-eth1`. Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-07-21 20:00:03 +00:00
onPeerConnected do (peer: Peer):
let
network = peer.network
ctx = peer.networkState
statusRes = ctx.getStatus()
handleHandlerError(statusRes)
let status = statusRes.get
Sync: Support for `eth/65` protocol This patch adds the `eth/65` protocol, documented at https://github.com/ethereum/devp2p/blob/master/caps/eth.md. This is an intentionally simple patch, designed to not break, change or depend on other functionality much, so that the "_old_ sync" methods can be run usefully again and observed. This patch isn't "new sync" (a different set of sync algorithms), but it is one of the foundations. For a while now Nimbus Eth1 only supported protocol `eth/63`. But that's obsolete, and very few nodes still support it. This meant Nimbus Eth1 would make slow progress trying to sync, as most up to date clients rejected it. The current specification is `eth/66`, and the functionality we really need is in `eth/64`. So why `eth/65`? - `eth/64` is essential because of the `forkId` feature. But `eth/64` is on its way out as well. Many clients, especially the most up to date Geth running the current hard-forks (Berlin/London) don't talk `eth/64` any more. - `eth/66` is the current specification, but some clients don't talk `eth/66` yet. We'd like to have the best peer connectivity during tests, and currently everything that talks `eth/66` also talks `eth/65`. - Nimbus Eth1 RLPx only talks one version at a time. (Without changes to the RLPx module. When those go in we'll add `eth/64..eth/66` for greater peer reach and testing the `eth/66` behaviour. For simplicity and decoupling, this patch contains just one version, the most useful.) What are `eth/64` and `eth/65`? - `eth/64` (EIP-2364) added `forkId` which allows nodes to distinguish between Ethereum (ETH) and Ethereum Classic (ETC) blockchains, which share the same genesis block. `forkId` also protects the system when a new hard fork is going to be rolled out, by blocking interaction with out of date nodes. The feature is required nowadays. We send the right details to allow connection (this has been tested a lot), but don't apply the full validation rules of EIP-2124/EIP-2364 in this patch. It's to keep this patch simple (in its effects) and because those rules have consequences best tested separately. In practice the other node will reject us when we would reject it, so this is ok for testing, as long as it doesn't get seriously deployed. - `eth/65` added more efficient transaction pool methods. - Then a later version of `eth/65` (without a new number) added typed transactions, described in [EIP-2976](https://eips.ethereum.org/EIPS/eip-2976). Why it's moved to `nimbus-eth1`: - Mainly because `eth/64` onwards depend on the current state of block synchronisation, as well as the blockchain's sequence of hard-fork block numbers, both of which are part of `nimbus-eth1` run-time state. These aren't available to pure `nim-eth` code. Although it would be possible to add an API to let `nimbus-eth1` set these numbers, there isn't any point because the protocol would still only be useful to `nimbus-eth1`. Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-07-21 20:00:03 +00:00
Snap sync simplify object inheritance (#1098) * Reorg SnapPeerBase descriptor, notably start/stop flags details: Instead of using three boolean flags startedFetch, stopped, and stopThisState a single enum type is used with values SyncRunningOk, SyncStopRequest, and SyncStopped. * Restricting snap to eth66 and later why: Id-tracked request/response wire protocol can handle overlapped responses when requests are sent in row. * Align function names with source code file names why: Easier to reconcile when following the implemented logic. * Update trace logging (want file locations) why: The macros previously used hid the relevant file location (when `chroniclesLineNumbers` turned on.) It rather printed the file location of the template that was wrapping `trace`. * Use KeyedQueue table instead of sequence why: Quick access, easy configuration as LRU or FIFO with max entries (currently LRU.) * Dissolve `SnapPeerEx` object extension into `SnapPeer` why; It is logically cleaner and more obvious not to inherit from `SnapPeerBase` but to specify opaque field object references of the merged `SnapPeer` object. These can then be locally inherited. * Dissolve `SnapSyncEx` object extension into `SnapSync` why; It is logically cleaner and more obvious not to inherit from `SnapSyncEx` but to specify opaque field object references of the `SnapPeer` object. These can then be locally inherited. Also, in the re-factored code here the interface descriptor `SnapSyncCtx` inherited `SnapSyncEx` which was sub-optimal (OO inheritance makes it easier to work with call back functions.)
2022-05-23 16:53:19 +00:00
trace trEthSendSending & "Status (0x00)", peer,
td = status.totalDifficulty,
bestHash = short(status.bestBlockHash),
networkId = network.networkId,
genesis = short(status.genesisHash),
forkHash = status.forkId.forkHash.toHex,
forkNext = status.forkId.forkNext
let m = await peer.status(ethVersion,
Sync: Support for `eth/65` protocol This patch adds the `eth/65` protocol, documented at https://github.com/ethereum/devp2p/blob/master/caps/eth.md. This is an intentionally simple patch, designed to not break, change or depend on other functionality much, so that the "_old_ sync" methods can be run usefully again and observed. This patch isn't "new sync" (a different set of sync algorithms), but it is one of the foundations. For a while now Nimbus Eth1 only supported protocol `eth/63`. But that's obsolete, and very few nodes still support it. This meant Nimbus Eth1 would make slow progress trying to sync, as most up to date clients rejected it. The current specification is `eth/66`, and the functionality we really need is in `eth/64`. So why `eth/65`? - `eth/64` is essential because of the `forkId` feature. But `eth/64` is on its way out as well. Many clients, especially the most up to date Geth running the current hard-forks (Berlin/London) don't talk `eth/64` any more. - `eth/66` is the current specification, but some clients don't talk `eth/66` yet. We'd like to have the best peer connectivity during tests, and currently everything that talks `eth/66` also talks `eth/65`. - Nimbus Eth1 RLPx only talks one version at a time. (Without changes to the RLPx module. When those go in we'll add `eth/64..eth/66` for greater peer reach and testing the `eth/66` behaviour. For simplicity and decoupling, this patch contains just one version, the most useful.) What are `eth/64` and `eth/65`? - `eth/64` (EIP-2364) added `forkId` which allows nodes to distinguish between Ethereum (ETH) and Ethereum Classic (ETC) blockchains, which share the same genesis block. `forkId` also protects the system when a new hard fork is going to be rolled out, by blocking interaction with out of date nodes. The feature is required nowadays. We send the right details to allow connection (this has been tested a lot), but don't apply the full validation rules of EIP-2124/EIP-2364 in this patch. It's to keep this patch simple (in its effects) and because those rules have consequences best tested separately. In practice the other node will reject us when we would reject it, so this is ok for testing, as long as it doesn't get seriously deployed. - `eth/65` added more efficient transaction pool methods. - Then a later version of `eth/65` (without a new number) added typed transactions, described in [EIP-2976](https://eips.ethereum.org/EIPS/eip-2976). Why it's moved to `nimbus-eth1`: - Mainly because `eth/64` onwards depend on the current state of block synchronisation, as well as the blockchain's sequence of hard-fork block numbers, both of which are part of `nimbus-eth1` run-time state. These aren't available to pure `nim-eth` code. Although it would be possible to add an API to let `nimbus-eth1` set these numbers, there isn't any point because the protocol would still only be useful to `nimbus-eth1`. Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-07-21 20:00:03 +00:00
network.networkId,
status.totalDifficulty,
status.bestBlockHash,
status.genesisHash,
status.forkId,
Sync: Support for `eth/65` protocol This patch adds the `eth/65` protocol, documented at https://github.com/ethereum/devp2p/blob/master/caps/eth.md. This is an intentionally simple patch, designed to not break, change or depend on other functionality much, so that the "_old_ sync" methods can be run usefully again and observed. This patch isn't "new sync" (a different set of sync algorithms), but it is one of the foundations. For a while now Nimbus Eth1 only supported protocol `eth/63`. But that's obsolete, and very few nodes still support it. This meant Nimbus Eth1 would make slow progress trying to sync, as most up to date clients rejected it. The current specification is `eth/66`, and the functionality we really need is in `eth/64`. So why `eth/65`? - `eth/64` is essential because of the `forkId` feature. But `eth/64` is on its way out as well. Many clients, especially the most up to date Geth running the current hard-forks (Berlin/London) don't talk `eth/64` any more. - `eth/66` is the current specification, but some clients don't talk `eth/66` yet. We'd like to have the best peer connectivity during tests, and currently everything that talks `eth/66` also talks `eth/65`. - Nimbus Eth1 RLPx only talks one version at a time. (Without changes to the RLPx module. When those go in we'll add `eth/64..eth/66` for greater peer reach and testing the `eth/66` behaviour. For simplicity and decoupling, this patch contains just one version, the most useful.) What are `eth/64` and `eth/65`? - `eth/64` (EIP-2364) added `forkId` which allows nodes to distinguish between Ethereum (ETH) and Ethereum Classic (ETC) blockchains, which share the same genesis block. `forkId` also protects the system when a new hard fork is going to be rolled out, by blocking interaction with out of date nodes. The feature is required nowadays. We send the right details to allow connection (this has been tested a lot), but don't apply the full validation rules of EIP-2124/EIP-2364 in this patch. It's to keep this patch simple (in its effects) and because those rules have consequences best tested separately. In practice the other node will reject us when we would reject it, so this is ok for testing, as long as it doesn't get seriously deployed. - `eth/65` added more efficient transaction pool methods. - Then a later version of `eth/65` (without a new number) added typed transactions, described in [EIP-2976](https://eips.ethereum.org/EIPS/eip-2976). Why it's moved to `nimbus-eth1`: - Mainly because `eth/64` onwards depend on the current state of block synchronisation, as well as the blockchain's sequence of hard-fork block numbers, both of which are part of `nimbus-eth1` run-time state. These aren't available to pure `nim-eth` code. Although it would be possible to add an API to let `nimbus-eth1` set these numbers, there isn't any point because the protocol would still only be useful to `nimbus-eth1`. Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-07-21 20:00:03 +00:00
timeout = chronos.seconds(10))
Snap sync simplify object inheritance (#1098) * Reorg SnapPeerBase descriptor, notably start/stop flags details: Instead of using three boolean flags startedFetch, stopped, and stopThisState a single enum type is used with values SyncRunningOk, SyncStopRequest, and SyncStopped. * Restricting snap to eth66 and later why: Id-tracked request/response wire protocol can handle overlapped responses when requests are sent in row. * Align function names with source code file names why: Easier to reconcile when following the implemented logic. * Update trace logging (want file locations) why: The macros previously used hid the relevant file location (when `chroniclesLineNumbers` turned on.) It rather printed the file location of the template that was wrapping `trace`. * Use KeyedQueue table instead of sequence why: Quick access, easy configuration as LRU or FIFO with max entries (currently LRU.) * Dissolve `SnapPeerEx` object extension into `SnapPeer` why; It is logically cleaner and more obvious not to inherit from `SnapPeerBase` but to specify opaque field object references of the merged `SnapPeer` object. These can then be locally inherited. * Dissolve `SnapSyncEx` object extension into `SnapSync` why; It is logically cleaner and more obvious not to inherit from `SnapSyncEx` but to specify opaque field object references of the `SnapPeer` object. These can then be locally inherited. Also, in the re-factored code here the interface descriptor `SnapSyncCtx` inherited `SnapSyncEx` which was sub-optimal (OO inheritance makes it easier to work with call back functions.)
2022-05-23 16:53:19 +00:00
when trEthTraceHandshakesOk:
trace "Handshake: Local and remote networkId",
local=network.networkId, remote=m.networkId
trace "Handshake: Local and remote genesisHash",
local=status.genesisHash, remote=m.genesisHash
trace "Handshake: Local and remote forkId",
local=(status.forkId.forkHash.toHex & "/" & $status.forkId.forkNext),
remote=(m.forkId.forkHash.toHex & "/" & $m.forkId.forkNext)
Sync: Support for `eth/65` protocol This patch adds the `eth/65` protocol, documented at https://github.com/ethereum/devp2p/blob/master/caps/eth.md. This is an intentionally simple patch, designed to not break, change or depend on other functionality much, so that the "_old_ sync" methods can be run usefully again and observed. This patch isn't "new sync" (a different set of sync algorithms), but it is one of the foundations. For a while now Nimbus Eth1 only supported protocol `eth/63`. But that's obsolete, and very few nodes still support it. This meant Nimbus Eth1 would make slow progress trying to sync, as most up to date clients rejected it. The current specification is `eth/66`, and the functionality we really need is in `eth/64`. So why `eth/65`? - `eth/64` is essential because of the `forkId` feature. But `eth/64` is on its way out as well. Many clients, especially the most up to date Geth running the current hard-forks (Berlin/London) don't talk `eth/64` any more. - `eth/66` is the current specification, but some clients don't talk `eth/66` yet. We'd like to have the best peer connectivity during tests, and currently everything that talks `eth/66` also talks `eth/65`. - Nimbus Eth1 RLPx only talks one version at a time. (Without changes to the RLPx module. When those go in we'll add `eth/64..eth/66` for greater peer reach and testing the `eth/66` behaviour. For simplicity and decoupling, this patch contains just one version, the most useful.) What are `eth/64` and `eth/65`? - `eth/64` (EIP-2364) added `forkId` which allows nodes to distinguish between Ethereum (ETH) and Ethereum Classic (ETC) blockchains, which share the same genesis block. `forkId` also protects the system when a new hard fork is going to be rolled out, by blocking interaction with out of date nodes. The feature is required nowadays. We send the right details to allow connection (this has been tested a lot), but don't apply the full validation rules of EIP-2124/EIP-2364 in this patch. It's to keep this patch simple (in its effects) and because those rules have consequences best tested separately. In practice the other node will reject us when we would reject it, so this is ok for testing, as long as it doesn't get seriously deployed. - `eth/65` added more efficient transaction pool methods. - Then a later version of `eth/65` (without a new number) added typed transactions, described in [EIP-2976](https://eips.ethereum.org/EIPS/eip-2976). Why it's moved to `nimbus-eth1`: - Mainly because `eth/64` onwards depend on the current state of block synchronisation, as well as the blockchain's sequence of hard-fork block numbers, both of which are part of `nimbus-eth1` run-time state. These aren't available to pure `nim-eth` code. Although it would be possible to add an API to let `nimbus-eth1` set these numbers, there isn't any point because the protocol would still only be useful to `nimbus-eth1`. Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-07-21 20:00:03 +00:00
if m.networkId != network.networkId:
trace "Peer for a different network (networkId)", peer,
expectNetworkId=network.networkId, gotNetworkId=m.networkId
Snap sync simplify object inheritance (#1098) * Reorg SnapPeerBase descriptor, notably start/stop flags details: Instead of using three boolean flags startedFetch, stopped, and stopThisState a single enum type is used with values SyncRunningOk, SyncStopRequest, and SyncStopped. * Restricting snap to eth66 and later why: Id-tracked request/response wire protocol can handle overlapped responses when requests are sent in row. * Align function names with source code file names why: Easier to reconcile when following the implemented logic. * Update trace logging (want file locations) why: The macros previously used hid the relevant file location (when `chroniclesLineNumbers` turned on.) It rather printed the file location of the template that was wrapping `trace`. * Use KeyedQueue table instead of sequence why: Quick access, easy configuration as LRU or FIFO with max entries (currently LRU.) * Dissolve `SnapPeerEx` object extension into `SnapPeer` why; It is logically cleaner and more obvious not to inherit from `SnapPeerBase` but to specify opaque field object references of the merged `SnapPeer` object. These can then be locally inherited. * Dissolve `SnapSyncEx` object extension into `SnapSync` why; It is logically cleaner and more obvious not to inherit from `SnapSyncEx` but to specify opaque field object references of the `SnapPeer` object. These can then be locally inherited. Also, in the re-factored code here the interface descriptor `SnapSyncCtx` inherited `SnapSyncEx` which was sub-optimal (OO inheritance makes it easier to work with call back functions.)
2022-05-23 16:53:19 +00:00
raise newException(
UselessPeerError, "Eth handshake for different network")
Sync: Support for `eth/65` protocol This patch adds the `eth/65` protocol, documented at https://github.com/ethereum/devp2p/blob/master/caps/eth.md. This is an intentionally simple patch, designed to not break, change or depend on other functionality much, so that the "_old_ sync" methods can be run usefully again and observed. This patch isn't "new sync" (a different set of sync algorithms), but it is one of the foundations. For a while now Nimbus Eth1 only supported protocol `eth/63`. But that's obsolete, and very few nodes still support it. This meant Nimbus Eth1 would make slow progress trying to sync, as most up to date clients rejected it. The current specification is `eth/66`, and the functionality we really need is in `eth/64`. So why `eth/65`? - `eth/64` is essential because of the `forkId` feature. But `eth/64` is on its way out as well. Many clients, especially the most up to date Geth running the current hard-forks (Berlin/London) don't talk `eth/64` any more. - `eth/66` is the current specification, but some clients don't talk `eth/66` yet. We'd like to have the best peer connectivity during tests, and currently everything that talks `eth/66` also talks `eth/65`. - Nimbus Eth1 RLPx only talks one version at a time. (Without changes to the RLPx module. When those go in we'll add `eth/64..eth/66` for greater peer reach and testing the `eth/66` behaviour. For simplicity and decoupling, this patch contains just one version, the most useful.) What are `eth/64` and `eth/65`? - `eth/64` (EIP-2364) added `forkId` which allows nodes to distinguish between Ethereum (ETH) and Ethereum Classic (ETC) blockchains, which share the same genesis block. `forkId` also protects the system when a new hard fork is going to be rolled out, by blocking interaction with out of date nodes. The feature is required nowadays. We send the right details to allow connection (this has been tested a lot), but don't apply the full validation rules of EIP-2124/EIP-2364 in this patch. It's to keep this patch simple (in its effects) and because those rules have consequences best tested separately. In practice the other node will reject us when we would reject it, so this is ok for testing, as long as it doesn't get seriously deployed. - `eth/65` added more efficient transaction pool methods. - Then a later version of `eth/65` (without a new number) added typed transactions, described in [EIP-2976](https://eips.ethereum.org/EIPS/eip-2976). Why it's moved to `nimbus-eth1`: - Mainly because `eth/64` onwards depend on the current state of block synchronisation, as well as the blockchain's sequence of hard-fork block numbers, both of which are part of `nimbus-eth1` run-time state. These aren't available to pure `nim-eth` code. Although it would be possible to add an API to let `nimbus-eth1` set these numbers, there isn't any point because the protocol would still only be useful to `nimbus-eth1`. Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-07-21 20:00:03 +00:00
if m.genesisHash != status.genesisHash:
Sync: Support for `eth/65` protocol This patch adds the `eth/65` protocol, documented at https://github.com/ethereum/devp2p/blob/master/caps/eth.md. This is an intentionally simple patch, designed to not break, change or depend on other functionality much, so that the "_old_ sync" methods can be run usefully again and observed. This patch isn't "new sync" (a different set of sync algorithms), but it is one of the foundations. For a while now Nimbus Eth1 only supported protocol `eth/63`. But that's obsolete, and very few nodes still support it. This meant Nimbus Eth1 would make slow progress trying to sync, as most up to date clients rejected it. The current specification is `eth/66`, and the functionality we really need is in `eth/64`. So why `eth/65`? - `eth/64` is essential because of the `forkId` feature. But `eth/64` is on its way out as well. Many clients, especially the most up to date Geth running the current hard-forks (Berlin/London) don't talk `eth/64` any more. - `eth/66` is the current specification, but some clients don't talk `eth/66` yet. We'd like to have the best peer connectivity during tests, and currently everything that talks `eth/66` also talks `eth/65`. - Nimbus Eth1 RLPx only talks one version at a time. (Without changes to the RLPx module. When those go in we'll add `eth/64..eth/66` for greater peer reach and testing the `eth/66` behaviour. For simplicity and decoupling, this patch contains just one version, the most useful.) What are `eth/64` and `eth/65`? - `eth/64` (EIP-2364) added `forkId` which allows nodes to distinguish between Ethereum (ETH) and Ethereum Classic (ETC) blockchains, which share the same genesis block. `forkId` also protects the system when a new hard fork is going to be rolled out, by blocking interaction with out of date nodes. The feature is required nowadays. We send the right details to allow connection (this has been tested a lot), but don't apply the full validation rules of EIP-2124/EIP-2364 in this patch. It's to keep this patch simple (in its effects) and because those rules have consequences best tested separately. In practice the other node will reject us when we would reject it, so this is ok for testing, as long as it doesn't get seriously deployed. - `eth/65` added more efficient transaction pool methods. - Then a later version of `eth/65` (without a new number) added typed transactions, described in [EIP-2976](https://eips.ethereum.org/EIPS/eip-2976). Why it's moved to `nimbus-eth1`: - Mainly because `eth/64` onwards depend on the current state of block synchronisation, as well as the blockchain's sequence of hard-fork block numbers, both of which are part of `nimbus-eth1` run-time state. These aren't available to pure `nim-eth` code. Although it would be possible to add an API to let `nimbus-eth1` set these numbers, there isn't any point because the protocol would still only be useful to `nimbus-eth1`. Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-07-21 20:00:03 +00:00
trace "Peer for a different network (genesisHash)", peer,
expectGenesis=short(status.genesisHash), gotGenesis=short(m.genesisHash)
Snap sync simplify object inheritance (#1098) * Reorg SnapPeerBase descriptor, notably start/stop flags details: Instead of using three boolean flags startedFetch, stopped, and stopThisState a single enum type is used with values SyncRunningOk, SyncStopRequest, and SyncStopped. * Restricting snap to eth66 and later why: Id-tracked request/response wire protocol can handle overlapped responses when requests are sent in row. * Align function names with source code file names why: Easier to reconcile when following the implemented logic. * Update trace logging (want file locations) why: The macros previously used hid the relevant file location (when `chroniclesLineNumbers` turned on.) It rather printed the file location of the template that was wrapping `trace`. * Use KeyedQueue table instead of sequence why: Quick access, easy configuration as LRU or FIFO with max entries (currently LRU.) * Dissolve `SnapPeerEx` object extension into `SnapPeer` why; It is logically cleaner and more obvious not to inherit from `SnapPeerBase` but to specify opaque field object references of the merged `SnapPeer` object. These can then be locally inherited. * Dissolve `SnapSyncEx` object extension into `SnapSync` why; It is logically cleaner and more obvious not to inherit from `SnapSyncEx` but to specify opaque field object references of the `SnapPeer` object. These can then be locally inherited. Also, in the re-factored code here the interface descriptor `SnapSyncCtx` inherited `SnapSyncEx` which was sub-optimal (OO inheritance makes it easier to work with call back functions.)
2022-05-23 16:53:19 +00:00
raise newException(
UselessPeerError, "Eth handshake for different network")
Sync: Support for `eth/65` protocol This patch adds the `eth/65` protocol, documented at https://github.com/ethereum/devp2p/blob/master/caps/eth.md. This is an intentionally simple patch, designed to not break, change or depend on other functionality much, so that the "_old_ sync" methods can be run usefully again and observed. This patch isn't "new sync" (a different set of sync algorithms), but it is one of the foundations. For a while now Nimbus Eth1 only supported protocol `eth/63`. But that's obsolete, and very few nodes still support it. This meant Nimbus Eth1 would make slow progress trying to sync, as most up to date clients rejected it. The current specification is `eth/66`, and the functionality we really need is in `eth/64`. So why `eth/65`? - `eth/64` is essential because of the `forkId` feature. But `eth/64` is on its way out as well. Many clients, especially the most up to date Geth running the current hard-forks (Berlin/London) don't talk `eth/64` any more. - `eth/66` is the current specification, but some clients don't talk `eth/66` yet. We'd like to have the best peer connectivity during tests, and currently everything that talks `eth/66` also talks `eth/65`. - Nimbus Eth1 RLPx only talks one version at a time. (Without changes to the RLPx module. When those go in we'll add `eth/64..eth/66` for greater peer reach and testing the `eth/66` behaviour. For simplicity and decoupling, this patch contains just one version, the most useful.) What are `eth/64` and `eth/65`? - `eth/64` (EIP-2364) added `forkId` which allows nodes to distinguish between Ethereum (ETH) and Ethereum Classic (ETC) blockchains, which share the same genesis block. `forkId` also protects the system when a new hard fork is going to be rolled out, by blocking interaction with out of date nodes. The feature is required nowadays. We send the right details to allow connection (this has been tested a lot), but don't apply the full validation rules of EIP-2124/EIP-2364 in this patch. It's to keep this patch simple (in its effects) and because those rules have consequences best tested separately. In practice the other node will reject us when we would reject it, so this is ok for testing, as long as it doesn't get seriously deployed. - `eth/65` added more efficient transaction pool methods. - Then a later version of `eth/65` (without a new number) added typed transactions, described in [EIP-2976](https://eips.ethereum.org/EIPS/eip-2976). Why it's moved to `nimbus-eth1`: - Mainly because `eth/64` onwards depend on the current state of block synchronisation, as well as the blockchain's sequence of hard-fork block numbers, both of which are part of `nimbus-eth1` run-time state. These aren't available to pure `nim-eth` code. Although it would be possible to add an API to let `nimbus-eth1` set these numbers, there isn't any point because the protocol would still only be useful to `nimbus-eth1`. Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-07-21 20:00:03 +00:00
trace "Peer matches our network", peer
peer.state.initialized = true
peer.state.bestDifficulty = m.totalDifficulty
peer.state.bestBlockHash = m.bestHash
handshake:
# User message 0x00: Status.
proc status(peer: Peer,
ethVersionArg: uint,
Sync: Support for `eth/65` protocol This patch adds the `eth/65` protocol, documented at https://github.com/ethereum/devp2p/blob/master/caps/eth.md. This is an intentionally simple patch, designed to not break, change or depend on other functionality much, so that the "_old_ sync" methods can be run usefully again and observed. This patch isn't "new sync" (a different set of sync algorithms), but it is one of the foundations. For a while now Nimbus Eth1 only supported protocol `eth/63`. But that's obsolete, and very few nodes still support it. This meant Nimbus Eth1 would make slow progress trying to sync, as most up to date clients rejected it. The current specification is `eth/66`, and the functionality we really need is in `eth/64`. So why `eth/65`? - `eth/64` is essential because of the `forkId` feature. But `eth/64` is on its way out as well. Many clients, especially the most up to date Geth running the current hard-forks (Berlin/London) don't talk `eth/64` any more. - `eth/66` is the current specification, but some clients don't talk `eth/66` yet. We'd like to have the best peer connectivity during tests, and currently everything that talks `eth/66` also talks `eth/65`. - Nimbus Eth1 RLPx only talks one version at a time. (Without changes to the RLPx module. When those go in we'll add `eth/64..eth/66` for greater peer reach and testing the `eth/66` behaviour. For simplicity and decoupling, this patch contains just one version, the most useful.) What are `eth/64` and `eth/65`? - `eth/64` (EIP-2364) added `forkId` which allows nodes to distinguish between Ethereum (ETH) and Ethereum Classic (ETC) blockchains, which share the same genesis block. `forkId` also protects the system when a new hard fork is going to be rolled out, by blocking interaction with out of date nodes. The feature is required nowadays. We send the right details to allow connection (this has been tested a lot), but don't apply the full validation rules of EIP-2124/EIP-2364 in this patch. It's to keep this patch simple (in its effects) and because those rules have consequences best tested separately. In practice the other node will reject us when we would reject it, so this is ok for testing, as long as it doesn't get seriously deployed. - `eth/65` added more efficient transaction pool methods. - Then a later version of `eth/65` (without a new number) added typed transactions, described in [EIP-2976](https://eips.ethereum.org/EIPS/eip-2976). Why it's moved to `nimbus-eth1`: - Mainly because `eth/64` onwards depend on the current state of block synchronisation, as well as the blockchain's sequence of hard-fork block numbers, both of which are part of `nimbus-eth1` run-time state. These aren't available to pure `nim-eth` code. Although it would be possible to add an API to let `nimbus-eth1` set these numbers, there isn't any point because the protocol would still only be useful to `nimbus-eth1`. Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-07-21 20:00:03 +00:00
networkId: NetworkId,
totalDifficulty: DifficultyInt,
bestHash: Hash256,
genesisHash: Hash256,
forkId: ChainForkId) =
Snap sync simplify object inheritance (#1098) * Reorg SnapPeerBase descriptor, notably start/stop flags details: Instead of using three boolean flags startedFetch, stopped, and stopThisState a single enum type is used with values SyncRunningOk, SyncStopRequest, and SyncStopped. * Restricting snap to eth66 and later why: Id-tracked request/response wire protocol can handle overlapped responses when requests are sent in row. * Align function names with source code file names why: Easier to reconcile when following the implemented logic. * Update trace logging (want file locations) why: The macros previously used hid the relevant file location (when `chroniclesLineNumbers` turned on.) It rather printed the file location of the template that was wrapping `trace`. * Use KeyedQueue table instead of sequence why: Quick access, easy configuration as LRU or FIFO with max entries (currently LRU.) * Dissolve `SnapPeerEx` object extension into `SnapPeer` why; It is logically cleaner and more obvious not to inherit from `SnapPeerBase` but to specify opaque field object references of the merged `SnapPeer` object. These can then be locally inherited. * Dissolve `SnapSyncEx` object extension into `SnapSync` why; It is logically cleaner and more obvious not to inherit from `SnapSyncEx` but to specify opaque field object references of the `SnapPeer` object. These can then be locally inherited. Also, in the re-factored code here the interface descriptor `SnapSyncCtx` inherited `SnapSyncEx` which was sub-optimal (OO inheritance makes it easier to work with call back functions.)
2022-05-23 16:53:19 +00:00
trace trEthRecvReceived & "Status (0x00)", peer,
networkId, totalDifficulty, bestHash=short(bestHash), genesisHash=short(genesisHash),
forkHash=forkId.forkHash.toHex, forkNext=forkId.forkNext
Sync: Support for `eth/65` protocol This patch adds the `eth/65` protocol, documented at https://github.com/ethereum/devp2p/blob/master/caps/eth.md. This is an intentionally simple patch, designed to not break, change or depend on other functionality much, so that the "_old_ sync" methods can be run usefully again and observed. This patch isn't "new sync" (a different set of sync algorithms), but it is one of the foundations. For a while now Nimbus Eth1 only supported protocol `eth/63`. But that's obsolete, and very few nodes still support it. This meant Nimbus Eth1 would make slow progress trying to sync, as most up to date clients rejected it. The current specification is `eth/66`, and the functionality we really need is in `eth/64`. So why `eth/65`? - `eth/64` is essential because of the `forkId` feature. But `eth/64` is on its way out as well. Many clients, especially the most up to date Geth running the current hard-forks (Berlin/London) don't talk `eth/64` any more. - `eth/66` is the current specification, but some clients don't talk `eth/66` yet. We'd like to have the best peer connectivity during tests, and currently everything that talks `eth/66` also talks `eth/65`. - Nimbus Eth1 RLPx only talks one version at a time. (Without changes to the RLPx module. When those go in we'll add `eth/64..eth/66` for greater peer reach and testing the `eth/66` behaviour. For simplicity and decoupling, this patch contains just one version, the most useful.) What are `eth/64` and `eth/65`? - `eth/64` (EIP-2364) added `forkId` which allows nodes to distinguish between Ethereum (ETH) and Ethereum Classic (ETC) blockchains, which share the same genesis block. `forkId` also protects the system when a new hard fork is going to be rolled out, by blocking interaction with out of date nodes. The feature is required nowadays. We send the right details to allow connection (this has been tested a lot), but don't apply the full validation rules of EIP-2124/EIP-2364 in this patch. It's to keep this patch simple (in its effects) and because those rules have consequences best tested separately. In practice the other node will reject us when we would reject it, so this is ok for testing, as long as it doesn't get seriously deployed. - `eth/65` added more efficient transaction pool methods. - Then a later version of `eth/65` (without a new number) added typed transactions, described in [EIP-2976](https://eips.ethereum.org/EIPS/eip-2976). Why it's moved to `nimbus-eth1`: - Mainly because `eth/64` onwards depend on the current state of block synchronisation, as well as the blockchain's sequence of hard-fork block numbers, both of which are part of `nimbus-eth1` run-time state. These aren't available to pure `nim-eth` code. Although it would be possible to add an API to let `nimbus-eth1` set these numbers, there isn't any point because the protocol would still only be useful to `nimbus-eth1`. Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-07-21 20:00:03 +00:00
# User message 0x01: NewBlockHashes.
proc newBlockHashes(peer: Peer, hashes: openArray[NewBlockHashesAnnounce]) =
Snap sync simplify object inheritance (#1098) * Reorg SnapPeerBase descriptor, notably start/stop flags details: Instead of using three boolean flags startedFetch, stopped, and stopThisState a single enum type is used with values SyncRunningOk, SyncStopRequest, and SyncStopped. * Restricting snap to eth66 and later why: Id-tracked request/response wire protocol can handle overlapped responses when requests are sent in row. * Align function names with source code file names why: Easier to reconcile when following the implemented logic. * Update trace logging (want file locations) why: The macros previously used hid the relevant file location (when `chroniclesLineNumbers` turned on.) It rather printed the file location of the template that was wrapping `trace`. * Use KeyedQueue table instead of sequence why: Quick access, easy configuration as LRU or FIFO with max entries (currently LRU.) * Dissolve `SnapPeerEx` object extension into `SnapPeer` why; It is logically cleaner and more obvious not to inherit from `SnapPeerBase` but to specify opaque field object references of the merged `SnapPeer` object. These can then be locally inherited. * Dissolve `SnapSyncEx` object extension into `SnapSync` why; It is logically cleaner and more obvious not to inherit from `SnapSyncEx` but to specify opaque field object references of the `SnapPeer` object. These can then be locally inherited. Also, in the re-factored code here the interface descriptor `SnapSyncCtx` inherited `SnapSyncEx` which was sub-optimal (OO inheritance makes it easier to work with call back functions.)
2022-05-23 16:53:19 +00:00
when trEthTraceGossipOk:
trace trEthRecvReceived & "NewBlockHashes (0x01)", peer,
Snap sync simplify object inheritance (#1098) * Reorg SnapPeerBase descriptor, notably start/stop flags details: Instead of using three boolean flags startedFetch, stopped, and stopThisState a single enum type is used with values SyncRunningOk, SyncStopRequest, and SyncStopped. * Restricting snap to eth66 and later why: Id-tracked request/response wire protocol can handle overlapped responses when requests are sent in row. * Align function names with source code file names why: Easier to reconcile when following the implemented logic. * Update trace logging (want file locations) why: The macros previously used hid the relevant file location (when `chroniclesLineNumbers` turned on.) It rather printed the file location of the template that was wrapping `trace`. * Use KeyedQueue table instead of sequence why: Quick access, easy configuration as LRU or FIFO with max entries (currently LRU.) * Dissolve `SnapPeerEx` object extension into `SnapPeer` why; It is logically cleaner and more obvious not to inherit from `SnapPeerBase` but to specify opaque field object references of the merged `SnapPeer` object. These can then be locally inherited. * Dissolve `SnapSyncEx` object extension into `SnapSync` why; It is logically cleaner and more obvious not to inherit from `SnapSyncEx` but to specify opaque field object references of the `SnapPeer` object. These can then be locally inherited. Also, in the re-factored code here the interface descriptor `SnapSyncCtx` inherited `SnapSyncEx` which was sub-optimal (OO inheritance makes it easier to work with call back functions.)
2022-05-23 16:53:19 +00:00
hashes=hashes.len
let ctx = peer.networkState()
let res = ctx.handleNewBlockHashes(peer, hashes)
handleHandlerError(res)
Sync: Support for `eth/65` protocol This patch adds the `eth/65` protocol, documented at https://github.com/ethereum/devp2p/blob/master/caps/eth.md. This is an intentionally simple patch, designed to not break, change or depend on other functionality much, so that the "_old_ sync" methods can be run usefully again and observed. This patch isn't "new sync" (a different set of sync algorithms), but it is one of the foundations. For a while now Nimbus Eth1 only supported protocol `eth/63`. But that's obsolete, and very few nodes still support it. This meant Nimbus Eth1 would make slow progress trying to sync, as most up to date clients rejected it. The current specification is `eth/66`, and the functionality we really need is in `eth/64`. So why `eth/65`? - `eth/64` is essential because of the `forkId` feature. But `eth/64` is on its way out as well. Many clients, especially the most up to date Geth running the current hard-forks (Berlin/London) don't talk `eth/64` any more. - `eth/66` is the current specification, but some clients don't talk `eth/66` yet. We'd like to have the best peer connectivity during tests, and currently everything that talks `eth/66` also talks `eth/65`. - Nimbus Eth1 RLPx only talks one version at a time. (Without changes to the RLPx module. When those go in we'll add `eth/64..eth/66` for greater peer reach and testing the `eth/66` behaviour. For simplicity and decoupling, this patch contains just one version, the most useful.) What are `eth/64` and `eth/65`? - `eth/64` (EIP-2364) added `forkId` which allows nodes to distinguish between Ethereum (ETH) and Ethereum Classic (ETC) blockchains, which share the same genesis block. `forkId` also protects the system when a new hard fork is going to be rolled out, by blocking interaction with out of date nodes. The feature is required nowadays. We send the right details to allow connection (this has been tested a lot), but don't apply the full validation rules of EIP-2124/EIP-2364 in this patch. It's to keep this patch simple (in its effects) and because those rules have consequences best tested separately. In practice the other node will reject us when we would reject it, so this is ok for testing, as long as it doesn't get seriously deployed. - `eth/65` added more efficient transaction pool methods. - Then a later version of `eth/65` (without a new number) added typed transactions, described in [EIP-2976](https://eips.ethereum.org/EIPS/eip-2976). Why it's moved to `nimbus-eth1`: - Mainly because `eth/64` onwards depend on the current state of block synchronisation, as well as the blockchain's sequence of hard-fork block numbers, both of which are part of `nimbus-eth1` run-time state. These aren't available to pure `nim-eth` code. Although it would be possible to add an API to let `nimbus-eth1` set these numbers, there isn't any point because the protocol would still only be useful to `nimbus-eth1`. Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-07-21 20:00:03 +00:00
# User message 0x02: Transactions.
proc transactions(peer: Peer, transactions: openArray[Transaction]) =
Snap sync simplify object inheritance (#1098) * Reorg SnapPeerBase descriptor, notably start/stop flags details: Instead of using three boolean flags startedFetch, stopped, and stopThisState a single enum type is used with values SyncRunningOk, SyncStopRequest, and SyncStopped. * Restricting snap to eth66 and later why: Id-tracked request/response wire protocol can handle overlapped responses when requests are sent in row. * Align function names with source code file names why: Easier to reconcile when following the implemented logic. * Update trace logging (want file locations) why: The macros previously used hid the relevant file location (when `chroniclesLineNumbers` turned on.) It rather printed the file location of the template that was wrapping `trace`. * Use KeyedQueue table instead of sequence why: Quick access, easy configuration as LRU or FIFO with max entries (currently LRU.) * Dissolve `SnapPeerEx` object extension into `SnapPeer` why; It is logically cleaner and more obvious not to inherit from `SnapPeerBase` but to specify opaque field object references of the merged `SnapPeer` object. These can then be locally inherited. * Dissolve `SnapSyncEx` object extension into `SnapSync` why; It is logically cleaner and more obvious not to inherit from `SnapSyncEx` but to specify opaque field object references of the `SnapPeer` object. These can then be locally inherited. Also, in the re-factored code here the interface descriptor `SnapSyncCtx` inherited `SnapSyncEx` which was sub-optimal (OO inheritance makes it easier to work with call back functions.)
2022-05-23 16:53:19 +00:00
when trEthTraceGossipOk:
trace trEthRecvReceived & "Transactions (0x02)", peer,
Snap sync simplify object inheritance (#1098) * Reorg SnapPeerBase descriptor, notably start/stop flags details: Instead of using three boolean flags startedFetch, stopped, and stopThisState a single enum type is used with values SyncRunningOk, SyncStopRequest, and SyncStopped. * Restricting snap to eth66 and later why: Id-tracked request/response wire protocol can handle overlapped responses when requests are sent in row. * Align function names with source code file names why: Easier to reconcile when following the implemented logic. * Update trace logging (want file locations) why: The macros previously used hid the relevant file location (when `chroniclesLineNumbers` turned on.) It rather printed the file location of the template that was wrapping `trace`. * Use KeyedQueue table instead of sequence why: Quick access, easy configuration as LRU or FIFO with max entries (currently LRU.) * Dissolve `SnapPeerEx` object extension into `SnapPeer` why; It is logically cleaner and more obvious not to inherit from `SnapPeerBase` but to specify opaque field object references of the merged `SnapPeer` object. These can then be locally inherited. * Dissolve `SnapSyncEx` object extension into `SnapSync` why; It is logically cleaner and more obvious not to inherit from `SnapSyncEx` but to specify opaque field object references of the `SnapPeer` object. These can then be locally inherited. Also, in the re-factored code here the interface descriptor `SnapSyncCtx` inherited `SnapSyncEx` which was sub-optimal (OO inheritance makes it easier to work with call back functions.)
2022-05-23 16:53:19 +00:00
transactions=transactions.len
let ctx = peer.networkState()
let res = ctx.handleAnnouncedTxs(peer, transactions)
handleHandlerError(res)
Sync: Support for `eth/65` protocol This patch adds the `eth/65` protocol, documented at https://github.com/ethereum/devp2p/blob/master/caps/eth.md. This is an intentionally simple patch, designed to not break, change or depend on other functionality much, so that the "_old_ sync" methods can be run usefully again and observed. This patch isn't "new sync" (a different set of sync algorithms), but it is one of the foundations. For a while now Nimbus Eth1 only supported protocol `eth/63`. But that's obsolete, and very few nodes still support it. This meant Nimbus Eth1 would make slow progress trying to sync, as most up to date clients rejected it. The current specification is `eth/66`, and the functionality we really need is in `eth/64`. So why `eth/65`? - `eth/64` is essential because of the `forkId` feature. But `eth/64` is on its way out as well. Many clients, especially the most up to date Geth running the current hard-forks (Berlin/London) don't talk `eth/64` any more. - `eth/66` is the current specification, but some clients don't talk `eth/66` yet. We'd like to have the best peer connectivity during tests, and currently everything that talks `eth/66` also talks `eth/65`. - Nimbus Eth1 RLPx only talks one version at a time. (Without changes to the RLPx module. When those go in we'll add `eth/64..eth/66` for greater peer reach and testing the `eth/66` behaviour. For simplicity and decoupling, this patch contains just one version, the most useful.) What are `eth/64` and `eth/65`? - `eth/64` (EIP-2364) added `forkId` which allows nodes to distinguish between Ethereum (ETH) and Ethereum Classic (ETC) blockchains, which share the same genesis block. `forkId` also protects the system when a new hard fork is going to be rolled out, by blocking interaction with out of date nodes. The feature is required nowadays. We send the right details to allow connection (this has been tested a lot), but don't apply the full validation rules of EIP-2124/EIP-2364 in this patch. It's to keep this patch simple (in its effects) and because those rules have consequences best tested separately. In practice the other node will reject us when we would reject it, so this is ok for testing, as long as it doesn't get seriously deployed. - `eth/65` added more efficient transaction pool methods. - Then a later version of `eth/65` (without a new number) added typed transactions, described in [EIP-2976](https://eips.ethereum.org/EIPS/eip-2976). Why it's moved to `nimbus-eth1`: - Mainly because `eth/64` onwards depend on the current state of block synchronisation, as well as the blockchain's sequence of hard-fork block numbers, both of which are part of `nimbus-eth1` run-time state. These aren't available to pure `nim-eth` code. Although it would be possible to add an API to let `nimbus-eth1` set these numbers, there isn't any point because the protocol would still only be useful to `nimbus-eth1`. Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-07-21 20:00:03 +00:00
requestResponse:
# User message 0x03: GetBlockHeaders.
proc getBlockHeaders(peer: Peer, request: BlocksRequest) =
Snap sync simplify object inheritance (#1098) * Reorg SnapPeerBase descriptor, notably start/stop flags details: Instead of using three boolean flags startedFetch, stopped, and stopThisState a single enum type is used with values SyncRunningOk, SyncStopRequest, and SyncStopped. * Restricting snap to eth66 and later why: Id-tracked request/response wire protocol can handle overlapped responses when requests are sent in row. * Align function names with source code file names why: Easier to reconcile when following the implemented logic. * Update trace logging (want file locations) why: The macros previously used hid the relevant file location (when `chroniclesLineNumbers` turned on.) It rather printed the file location of the template that was wrapping `trace`. * Use KeyedQueue table instead of sequence why: Quick access, easy configuration as LRU or FIFO with max entries (currently LRU.) * Dissolve `SnapPeerEx` object extension into `SnapPeer` why; It is logically cleaner and more obvious not to inherit from `SnapPeerBase` but to specify opaque field object references of the merged `SnapPeer` object. These can then be locally inherited. * Dissolve `SnapSyncEx` object extension into `SnapSync` why; It is logically cleaner and more obvious not to inherit from `SnapSyncEx` but to specify opaque field object references of the `SnapPeer` object. These can then be locally inherited. Also, in the re-factored code here the interface descriptor `SnapSyncCtx` inherited `SnapSyncEx` which was sub-optimal (OO inheritance makes it easier to work with call back functions.)
2022-05-23 16:53:19 +00:00
when trEthTracePacketsOk:
trace trEthRecvReceived & "GetBlockHeaders (0x03)", peer,
count=request.maxResults
Sync: Support for `eth/65` protocol This patch adds the `eth/65` protocol, documented at https://github.com/ethereum/devp2p/blob/master/caps/eth.md. This is an intentionally simple patch, designed to not break, change or depend on other functionality much, so that the "_old_ sync" methods can be run usefully again and observed. This patch isn't "new sync" (a different set of sync algorithms), but it is one of the foundations. For a while now Nimbus Eth1 only supported protocol `eth/63`. But that's obsolete, and very few nodes still support it. This meant Nimbus Eth1 would make slow progress trying to sync, as most up to date clients rejected it. The current specification is `eth/66`, and the functionality we really need is in `eth/64`. So why `eth/65`? - `eth/64` is essential because of the `forkId` feature. But `eth/64` is on its way out as well. Many clients, especially the most up to date Geth running the current hard-forks (Berlin/London) don't talk `eth/64` any more. - `eth/66` is the current specification, but some clients don't talk `eth/66` yet. We'd like to have the best peer connectivity during tests, and currently everything that talks `eth/66` also talks `eth/65`. - Nimbus Eth1 RLPx only talks one version at a time. (Without changes to the RLPx module. When those go in we'll add `eth/64..eth/66` for greater peer reach and testing the `eth/66` behaviour. For simplicity and decoupling, this patch contains just one version, the most useful.) What are `eth/64` and `eth/65`? - `eth/64` (EIP-2364) added `forkId` which allows nodes to distinguish between Ethereum (ETH) and Ethereum Classic (ETC) blockchains, which share the same genesis block. `forkId` also protects the system when a new hard fork is going to be rolled out, by blocking interaction with out of date nodes. The feature is required nowadays. We send the right details to allow connection (this has been tested a lot), but don't apply the full validation rules of EIP-2124/EIP-2364 in this patch. It's to keep this patch simple (in its effects) and because those rules have consequences best tested separately. In practice the other node will reject us when we would reject it, so this is ok for testing, as long as it doesn't get seriously deployed. - `eth/65` added more efficient transaction pool methods. - Then a later version of `eth/65` (without a new number) added typed transactions, described in [EIP-2976](https://eips.ethereum.org/EIPS/eip-2976). Why it's moved to `nimbus-eth1`: - Mainly because `eth/64` onwards depend on the current state of block synchronisation, as well as the blockchain's sequence of hard-fork block numbers, both of which are part of `nimbus-eth1` run-time state. These aren't available to pure `nim-eth` code. Although it would be possible to add an API to let `nimbus-eth1` set these numbers, there isn't any point because the protocol would still only be useful to `nimbus-eth1`. Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-07-21 20:00:03 +00:00
if request.maxResults > uint64(maxHeadersFetch):
debug "GetBlockHeaders (0x03) requested too many headers",
peer, requested=request.maxResults, max=maxHeadersFetch
Sync: Support for `eth/65` protocol This patch adds the `eth/65` protocol, documented at https://github.com/ethereum/devp2p/blob/master/caps/eth.md. This is an intentionally simple patch, designed to not break, change or depend on other functionality much, so that the "_old_ sync" methods can be run usefully again and observed. This patch isn't "new sync" (a different set of sync algorithms), but it is one of the foundations. For a while now Nimbus Eth1 only supported protocol `eth/63`. But that's obsolete, and very few nodes still support it. This meant Nimbus Eth1 would make slow progress trying to sync, as most up to date clients rejected it. The current specification is `eth/66`, and the functionality we really need is in `eth/64`. So why `eth/65`? - `eth/64` is essential because of the `forkId` feature. But `eth/64` is on its way out as well. Many clients, especially the most up to date Geth running the current hard-forks (Berlin/London) don't talk `eth/64` any more. - `eth/66` is the current specification, but some clients don't talk `eth/66` yet. We'd like to have the best peer connectivity during tests, and currently everything that talks `eth/66` also talks `eth/65`. - Nimbus Eth1 RLPx only talks one version at a time. (Without changes to the RLPx module. When those go in we'll add `eth/64..eth/66` for greater peer reach and testing the `eth/66` behaviour. For simplicity and decoupling, this patch contains just one version, the most useful.) What are `eth/64` and `eth/65`? - `eth/64` (EIP-2364) added `forkId` which allows nodes to distinguish between Ethereum (ETH) and Ethereum Classic (ETC) blockchains, which share the same genesis block. `forkId` also protects the system when a new hard fork is going to be rolled out, by blocking interaction with out of date nodes. The feature is required nowadays. We send the right details to allow connection (this has been tested a lot), but don't apply the full validation rules of EIP-2124/EIP-2364 in this patch. It's to keep this patch simple (in its effects) and because those rules have consequences best tested separately. In practice the other node will reject us when we would reject it, so this is ok for testing, as long as it doesn't get seriously deployed. - `eth/65` added more efficient transaction pool methods. - Then a later version of `eth/65` (without a new number) added typed transactions, described in [EIP-2976](https://eips.ethereum.org/EIPS/eip-2976). Why it's moved to `nimbus-eth1`: - Mainly because `eth/64` onwards depend on the current state of block synchronisation, as well as the blockchain's sequence of hard-fork block numbers, both of which are part of `nimbus-eth1` run-time state. These aren't available to pure `nim-eth` code. Although it would be possible to add an API to let `nimbus-eth1` set these numbers, there isn't any point because the protocol would still only be useful to `nimbus-eth1`. Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-07-21 20:00:03 +00:00
await peer.disconnect(BreachOfProtocol)
return
let ctx = peer.networkState()
let headers = ctx.getBlockHeaders(request)
handleHandlerError(headers)
if headers.get.len > 0:
Snap sync simplify object inheritance (#1098) * Reorg SnapPeerBase descriptor, notably start/stop flags details: Instead of using three boolean flags startedFetch, stopped, and stopThisState a single enum type is used with values SyncRunningOk, SyncStopRequest, and SyncStopped. * Restricting snap to eth66 and later why: Id-tracked request/response wire protocol can handle overlapped responses when requests are sent in row. * Align function names with source code file names why: Easier to reconcile when following the implemented logic. * Update trace logging (want file locations) why: The macros previously used hid the relevant file location (when `chroniclesLineNumbers` turned on.) It rather printed the file location of the template that was wrapping `trace`. * Use KeyedQueue table instead of sequence why: Quick access, easy configuration as LRU or FIFO with max entries (currently LRU.) * Dissolve `SnapPeerEx` object extension into `SnapPeer` why; It is logically cleaner and more obvious not to inherit from `SnapPeerBase` but to specify opaque field object references of the merged `SnapPeer` object. These can then be locally inherited. * Dissolve `SnapSyncEx` object extension into `SnapSync` why; It is logically cleaner and more obvious not to inherit from `SnapSyncEx` but to specify opaque field object references of the `SnapPeer` object. These can then be locally inherited. Also, in the re-factored code here the interface descriptor `SnapSyncCtx` inherited `SnapSyncEx` which was sub-optimal (OO inheritance makes it easier to work with call back functions.)
2022-05-23 16:53:19 +00:00
trace trEthSendReplying & "with BlockHeaders (0x04)", peer,
sent=headers.get.len, requested=request.maxResults
else:
Snap sync simplify object inheritance (#1098) * Reorg SnapPeerBase descriptor, notably start/stop flags details: Instead of using three boolean flags startedFetch, stopped, and stopThisState a single enum type is used with values SyncRunningOk, SyncStopRequest, and SyncStopped. * Restricting snap to eth66 and later why: Id-tracked request/response wire protocol can handle overlapped responses when requests are sent in row. * Align function names with source code file names why: Easier to reconcile when following the implemented logic. * Update trace logging (want file locations) why: The macros previously used hid the relevant file location (when `chroniclesLineNumbers` turned on.) It rather printed the file location of the template that was wrapping `trace`. * Use KeyedQueue table instead of sequence why: Quick access, easy configuration as LRU or FIFO with max entries (currently LRU.) * Dissolve `SnapPeerEx` object extension into `SnapPeer` why; It is logically cleaner and more obvious not to inherit from `SnapPeerBase` but to specify opaque field object references of the merged `SnapPeer` object. These can then be locally inherited. * Dissolve `SnapSyncEx` object extension into `SnapSync` why; It is logically cleaner and more obvious not to inherit from `SnapSyncEx` but to specify opaque field object references of the `SnapPeer` object. These can then be locally inherited. Also, in the re-factored code here the interface descriptor `SnapSyncCtx` inherited `SnapSyncEx` which was sub-optimal (OO inheritance makes it easier to work with call back functions.)
2022-05-23 16:53:19 +00:00
trace trEthSendReplying & "EMPTY BlockHeaders (0x04)", peer,
sent=0, requested=request.maxResults
await response.send(headers.get)
Sync: Support for `eth/65` protocol This patch adds the `eth/65` protocol, documented at https://github.com/ethereum/devp2p/blob/master/caps/eth.md. This is an intentionally simple patch, designed to not break, change or depend on other functionality much, so that the "_old_ sync" methods can be run usefully again and observed. This patch isn't "new sync" (a different set of sync algorithms), but it is one of the foundations. For a while now Nimbus Eth1 only supported protocol `eth/63`. But that's obsolete, and very few nodes still support it. This meant Nimbus Eth1 would make slow progress trying to sync, as most up to date clients rejected it. The current specification is `eth/66`, and the functionality we really need is in `eth/64`. So why `eth/65`? - `eth/64` is essential because of the `forkId` feature. But `eth/64` is on its way out as well. Many clients, especially the most up to date Geth running the current hard-forks (Berlin/London) don't talk `eth/64` any more. - `eth/66` is the current specification, but some clients don't talk `eth/66` yet. We'd like to have the best peer connectivity during tests, and currently everything that talks `eth/66` also talks `eth/65`. - Nimbus Eth1 RLPx only talks one version at a time. (Without changes to the RLPx module. When those go in we'll add `eth/64..eth/66` for greater peer reach and testing the `eth/66` behaviour. For simplicity and decoupling, this patch contains just one version, the most useful.) What are `eth/64` and `eth/65`? - `eth/64` (EIP-2364) added `forkId` which allows nodes to distinguish between Ethereum (ETH) and Ethereum Classic (ETC) blockchains, which share the same genesis block. `forkId` also protects the system when a new hard fork is going to be rolled out, by blocking interaction with out of date nodes. The feature is required nowadays. We send the right details to allow connection (this has been tested a lot), but don't apply the full validation rules of EIP-2124/EIP-2364 in this patch. It's to keep this patch simple (in its effects) and because those rules have consequences best tested separately. In practice the other node will reject us when we would reject it, so this is ok for testing, as long as it doesn't get seriously deployed. - `eth/65` added more efficient transaction pool methods. - Then a later version of `eth/65` (without a new number) added typed transactions, described in [EIP-2976](https://eips.ethereum.org/EIPS/eip-2976). Why it's moved to `nimbus-eth1`: - Mainly because `eth/64` onwards depend on the current state of block synchronisation, as well as the blockchain's sequence of hard-fork block numbers, both of which are part of `nimbus-eth1` run-time state. These aren't available to pure `nim-eth` code. Although it would be possible to add an API to let `nimbus-eth1` set these numbers, there isn't any point because the protocol would still only be useful to `nimbus-eth1`. Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-07-21 20:00:03 +00:00
# User message 0x04: BlockHeaders.
proc blockHeaders(p: Peer, headers: openArray[BlockHeader])
requestResponse:
# User message 0x05: GetBlockBodies.
proc getBlockBodies(peer: Peer, hashes: openArray[Hash256]) =
Snap sync simplify object inheritance (#1098) * Reorg SnapPeerBase descriptor, notably start/stop flags details: Instead of using three boolean flags startedFetch, stopped, and stopThisState a single enum type is used with values SyncRunningOk, SyncStopRequest, and SyncStopped. * Restricting snap to eth66 and later why: Id-tracked request/response wire protocol can handle overlapped responses when requests are sent in row. * Align function names with source code file names why: Easier to reconcile when following the implemented logic. * Update trace logging (want file locations) why: The macros previously used hid the relevant file location (when `chroniclesLineNumbers` turned on.) It rather printed the file location of the template that was wrapping `trace`. * Use KeyedQueue table instead of sequence why: Quick access, easy configuration as LRU or FIFO with max entries (currently LRU.) * Dissolve `SnapPeerEx` object extension into `SnapPeer` why; It is logically cleaner and more obvious not to inherit from `SnapPeerBase` but to specify opaque field object references of the merged `SnapPeer` object. These can then be locally inherited. * Dissolve `SnapSyncEx` object extension into `SnapSync` why; It is logically cleaner and more obvious not to inherit from `SnapSyncEx` but to specify opaque field object references of the `SnapPeer` object. These can then be locally inherited. Also, in the re-factored code here the interface descriptor `SnapSyncCtx` inherited `SnapSyncEx` which was sub-optimal (OO inheritance makes it easier to work with call back functions.)
2022-05-23 16:53:19 +00:00
trace trEthRecvReceived & "GetBlockBodies (0x05)", peer,
hashes=hashes.len
Sync: Support for `eth/65` protocol This patch adds the `eth/65` protocol, documented at https://github.com/ethereum/devp2p/blob/master/caps/eth.md. This is an intentionally simple patch, designed to not break, change or depend on other functionality much, so that the "_old_ sync" methods can be run usefully again and observed. This patch isn't "new sync" (a different set of sync algorithms), but it is one of the foundations. For a while now Nimbus Eth1 only supported protocol `eth/63`. But that's obsolete, and very few nodes still support it. This meant Nimbus Eth1 would make slow progress trying to sync, as most up to date clients rejected it. The current specification is `eth/66`, and the functionality we really need is in `eth/64`. So why `eth/65`? - `eth/64` is essential because of the `forkId` feature. But `eth/64` is on its way out as well. Many clients, especially the most up to date Geth running the current hard-forks (Berlin/London) don't talk `eth/64` any more. - `eth/66` is the current specification, but some clients don't talk `eth/66` yet. We'd like to have the best peer connectivity during tests, and currently everything that talks `eth/66` also talks `eth/65`. - Nimbus Eth1 RLPx only talks one version at a time. (Without changes to the RLPx module. When those go in we'll add `eth/64..eth/66` for greater peer reach and testing the `eth/66` behaviour. For simplicity and decoupling, this patch contains just one version, the most useful.) What are `eth/64` and `eth/65`? - `eth/64` (EIP-2364) added `forkId` which allows nodes to distinguish between Ethereum (ETH) and Ethereum Classic (ETC) blockchains, which share the same genesis block. `forkId` also protects the system when a new hard fork is going to be rolled out, by blocking interaction with out of date nodes. The feature is required nowadays. We send the right details to allow connection (this has been tested a lot), but don't apply the full validation rules of EIP-2124/EIP-2364 in this patch. It's to keep this patch simple (in its effects) and because those rules have consequences best tested separately. In practice the other node will reject us when we would reject it, so this is ok for testing, as long as it doesn't get seriously deployed. - `eth/65` added more efficient transaction pool methods. - Then a later version of `eth/65` (without a new number) added typed transactions, described in [EIP-2976](https://eips.ethereum.org/EIPS/eip-2976). Why it's moved to `nimbus-eth1`: - Mainly because `eth/64` onwards depend on the current state of block synchronisation, as well as the blockchain's sequence of hard-fork block numbers, both of which are part of `nimbus-eth1` run-time state. These aren't available to pure `nim-eth` code. Although it would be possible to add an API to let `nimbus-eth1` set these numbers, there isn't any point because the protocol would still only be useful to `nimbus-eth1`. Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-07-21 20:00:03 +00:00
if hashes.len > maxBodiesFetch:
debug "GetBlockBodies (0x05) requested too many bodies",
peer, requested=hashes.len, max=maxBodiesFetch
Sync: Support for `eth/65` protocol This patch adds the `eth/65` protocol, documented at https://github.com/ethereum/devp2p/blob/master/caps/eth.md. This is an intentionally simple patch, designed to not break, change or depend on other functionality much, so that the "_old_ sync" methods can be run usefully again and observed. This patch isn't "new sync" (a different set of sync algorithms), but it is one of the foundations. For a while now Nimbus Eth1 only supported protocol `eth/63`. But that's obsolete, and very few nodes still support it. This meant Nimbus Eth1 would make slow progress trying to sync, as most up to date clients rejected it. The current specification is `eth/66`, and the functionality we really need is in `eth/64`. So why `eth/65`? - `eth/64` is essential because of the `forkId` feature. But `eth/64` is on its way out as well. Many clients, especially the most up to date Geth running the current hard-forks (Berlin/London) don't talk `eth/64` any more. - `eth/66` is the current specification, but some clients don't talk `eth/66` yet. We'd like to have the best peer connectivity during tests, and currently everything that talks `eth/66` also talks `eth/65`. - Nimbus Eth1 RLPx only talks one version at a time. (Without changes to the RLPx module. When those go in we'll add `eth/64..eth/66` for greater peer reach and testing the `eth/66` behaviour. For simplicity and decoupling, this patch contains just one version, the most useful.) What are `eth/64` and `eth/65`? - `eth/64` (EIP-2364) added `forkId` which allows nodes to distinguish between Ethereum (ETH) and Ethereum Classic (ETC) blockchains, which share the same genesis block. `forkId` also protects the system when a new hard fork is going to be rolled out, by blocking interaction with out of date nodes. The feature is required nowadays. We send the right details to allow connection (this has been tested a lot), but don't apply the full validation rules of EIP-2124/EIP-2364 in this patch. It's to keep this patch simple (in its effects) and because those rules have consequences best tested separately. In practice the other node will reject us when we would reject it, so this is ok for testing, as long as it doesn't get seriously deployed. - `eth/65` added more efficient transaction pool methods. - Then a later version of `eth/65` (without a new number) added typed transactions, described in [EIP-2976](https://eips.ethereum.org/EIPS/eip-2976). Why it's moved to `nimbus-eth1`: - Mainly because `eth/64` onwards depend on the current state of block synchronisation, as well as the blockchain's sequence of hard-fork block numbers, both of which are part of `nimbus-eth1` run-time state. These aren't available to pure `nim-eth` code. Although it would be possible to add an API to let `nimbus-eth1` set these numbers, there isn't any point because the protocol would still only be useful to `nimbus-eth1`. Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-07-21 20:00:03 +00:00
await peer.disconnect(BreachOfProtocol)
return
let ctx = peer.networkState()
let bodies = ctx.getBlockBodies(hashes)
handleHandlerError(bodies)
if bodies.get.len > 0:
Snap sync simplify object inheritance (#1098) * Reorg SnapPeerBase descriptor, notably start/stop flags details: Instead of using three boolean flags startedFetch, stopped, and stopThisState a single enum type is used with values SyncRunningOk, SyncStopRequest, and SyncStopped. * Restricting snap to eth66 and later why: Id-tracked request/response wire protocol can handle overlapped responses when requests are sent in row. * Align function names with source code file names why: Easier to reconcile when following the implemented logic. * Update trace logging (want file locations) why: The macros previously used hid the relevant file location (when `chroniclesLineNumbers` turned on.) It rather printed the file location of the template that was wrapping `trace`. * Use KeyedQueue table instead of sequence why: Quick access, easy configuration as LRU or FIFO with max entries (currently LRU.) * Dissolve `SnapPeerEx` object extension into `SnapPeer` why; It is logically cleaner and more obvious not to inherit from `SnapPeerBase` but to specify opaque field object references of the merged `SnapPeer` object. These can then be locally inherited. * Dissolve `SnapSyncEx` object extension into `SnapSync` why; It is logically cleaner and more obvious not to inherit from `SnapSyncEx` but to specify opaque field object references of the `SnapPeer` object. These can then be locally inherited. Also, in the re-factored code here the interface descriptor `SnapSyncCtx` inherited `SnapSyncEx` which was sub-optimal (OO inheritance makes it easier to work with call back functions.)
2022-05-23 16:53:19 +00:00
trace trEthSendReplying & "with BlockBodies (0x06)", peer,
sent=bodies.get.len, requested=hashes.len
else:
Snap sync simplify object inheritance (#1098) * Reorg SnapPeerBase descriptor, notably start/stop flags details: Instead of using three boolean flags startedFetch, stopped, and stopThisState a single enum type is used with values SyncRunningOk, SyncStopRequest, and SyncStopped. * Restricting snap to eth66 and later why: Id-tracked request/response wire protocol can handle overlapped responses when requests are sent in row. * Align function names with source code file names why: Easier to reconcile when following the implemented logic. * Update trace logging (want file locations) why: The macros previously used hid the relevant file location (when `chroniclesLineNumbers` turned on.) It rather printed the file location of the template that was wrapping `trace`. * Use KeyedQueue table instead of sequence why: Quick access, easy configuration as LRU or FIFO with max entries (currently LRU.) * Dissolve `SnapPeerEx` object extension into `SnapPeer` why; It is logically cleaner and more obvious not to inherit from `SnapPeerBase` but to specify opaque field object references of the merged `SnapPeer` object. These can then be locally inherited. * Dissolve `SnapSyncEx` object extension into `SnapSync` why; It is logically cleaner and more obvious not to inherit from `SnapSyncEx` but to specify opaque field object references of the `SnapPeer` object. These can then be locally inherited. Also, in the re-factored code here the interface descriptor `SnapSyncCtx` inherited `SnapSyncEx` which was sub-optimal (OO inheritance makes it easier to work with call back functions.)
2022-05-23 16:53:19 +00:00
trace trEthSendReplying & "EMPTY BlockBodies (0x06)", peer,
sent=0, requested=hashes.len
await response.send(bodies.get)
Sync: Support for `eth/65` protocol This patch adds the `eth/65` protocol, documented at https://github.com/ethereum/devp2p/blob/master/caps/eth.md. This is an intentionally simple patch, designed to not break, change or depend on other functionality much, so that the "_old_ sync" methods can be run usefully again and observed. This patch isn't "new sync" (a different set of sync algorithms), but it is one of the foundations. For a while now Nimbus Eth1 only supported protocol `eth/63`. But that's obsolete, and very few nodes still support it. This meant Nimbus Eth1 would make slow progress trying to sync, as most up to date clients rejected it. The current specification is `eth/66`, and the functionality we really need is in `eth/64`. So why `eth/65`? - `eth/64` is essential because of the `forkId` feature. But `eth/64` is on its way out as well. Many clients, especially the most up to date Geth running the current hard-forks (Berlin/London) don't talk `eth/64` any more. - `eth/66` is the current specification, but some clients don't talk `eth/66` yet. We'd like to have the best peer connectivity during tests, and currently everything that talks `eth/66` also talks `eth/65`. - Nimbus Eth1 RLPx only talks one version at a time. (Without changes to the RLPx module. When those go in we'll add `eth/64..eth/66` for greater peer reach and testing the `eth/66` behaviour. For simplicity and decoupling, this patch contains just one version, the most useful.) What are `eth/64` and `eth/65`? - `eth/64` (EIP-2364) added `forkId` which allows nodes to distinguish between Ethereum (ETH) and Ethereum Classic (ETC) blockchains, which share the same genesis block. `forkId` also protects the system when a new hard fork is going to be rolled out, by blocking interaction with out of date nodes. The feature is required nowadays. We send the right details to allow connection (this has been tested a lot), but don't apply the full validation rules of EIP-2124/EIP-2364 in this patch. It's to keep this patch simple (in its effects) and because those rules have consequences best tested separately. In practice the other node will reject us when we would reject it, so this is ok for testing, as long as it doesn't get seriously deployed. - `eth/65` added more efficient transaction pool methods. - Then a later version of `eth/65` (without a new number) added typed transactions, described in [EIP-2976](https://eips.ethereum.org/EIPS/eip-2976). Why it's moved to `nimbus-eth1`: - Mainly because `eth/64` onwards depend on the current state of block synchronisation, as well as the blockchain's sequence of hard-fork block numbers, both of which are part of `nimbus-eth1` run-time state. These aren't available to pure `nim-eth` code. Although it would be possible to add an API to let `nimbus-eth1` set these numbers, there isn't any point because the protocol would still only be useful to `nimbus-eth1`. Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-07-21 20:00:03 +00:00
# User message 0x06: BlockBodies.
proc blockBodies(peer: Peer, blocks: openArray[BlockBody])
# User message 0x07: NewBlock.
proc newBlock(peer: Peer, blk: EthBlock, totalDifficulty: DifficultyInt) =
# (Note, needs to use `EthBlock` instead of its alias `NewBlockAnnounce`
# because either `p2pProtocol` or RLPx doesn't work with an alias.)
Snap sync simplify object inheritance (#1098) * Reorg SnapPeerBase descriptor, notably start/stop flags details: Instead of using three boolean flags startedFetch, stopped, and stopThisState a single enum type is used with values SyncRunningOk, SyncStopRequest, and SyncStopped. * Restricting snap to eth66 and later why: Id-tracked request/response wire protocol can handle overlapped responses when requests are sent in row. * Align function names with source code file names why: Easier to reconcile when following the implemented logic. * Update trace logging (want file locations) why: The macros previously used hid the relevant file location (when `chroniclesLineNumbers` turned on.) It rather printed the file location of the template that was wrapping `trace`. * Use KeyedQueue table instead of sequence why: Quick access, easy configuration as LRU or FIFO with max entries (currently LRU.) * Dissolve `SnapPeerEx` object extension into `SnapPeer` why; It is logically cleaner and more obvious not to inherit from `SnapPeerBase` but to specify opaque field object references of the merged `SnapPeer` object. These can then be locally inherited. * Dissolve `SnapSyncEx` object extension into `SnapSync` why; It is logically cleaner and more obvious not to inherit from `SnapSyncEx` but to specify opaque field object references of the `SnapPeer` object. These can then be locally inherited. Also, in the re-factored code here the interface descriptor `SnapSyncCtx` inherited `SnapSyncEx` which was sub-optimal (OO inheritance makes it easier to work with call back functions.)
2022-05-23 16:53:19 +00:00
when trEthTraceGossipOk:
trace trEthRecvReceived & "NewBlock (0x07)", peer,
Snap sync simplify object inheritance (#1098) * Reorg SnapPeerBase descriptor, notably start/stop flags details: Instead of using three boolean flags startedFetch, stopped, and stopThisState a single enum type is used with values SyncRunningOk, SyncStopRequest, and SyncStopped. * Restricting snap to eth66 and later why: Id-tracked request/response wire protocol can handle overlapped responses when requests are sent in row. * Align function names with source code file names why: Easier to reconcile when following the implemented logic. * Update trace logging (want file locations) why: The macros previously used hid the relevant file location (when `chroniclesLineNumbers` turned on.) It rather printed the file location of the template that was wrapping `trace`. * Use KeyedQueue table instead of sequence why: Quick access, easy configuration as LRU or FIFO with max entries (currently LRU.) * Dissolve `SnapPeerEx` object extension into `SnapPeer` why; It is logically cleaner and more obvious not to inherit from `SnapPeerBase` but to specify opaque field object references of the merged `SnapPeer` object. These can then be locally inherited. * Dissolve `SnapSyncEx` object extension into `SnapSync` why; It is logically cleaner and more obvious not to inherit from `SnapSyncEx` but to specify opaque field object references of the `SnapPeer` object. These can then be locally inherited. Also, in the re-factored code here the interface descriptor `SnapSyncCtx` inherited `SnapSyncEx` which was sub-optimal (OO inheritance makes it easier to work with call back functions.)
2022-05-23 16:53:19 +00:00
totalDifficulty,
blockNumber = blk.header.blockNumber,
blockDifficulty = blk.header.difficulty
let ctx = peer.networkState()
let res = ctx.handleNewBlock(peer, blk, totalDifficulty)
handleHandlerError(res)
Sync: Support for `eth/65` protocol This patch adds the `eth/65` protocol, documented at https://github.com/ethereum/devp2p/blob/master/caps/eth.md. This is an intentionally simple patch, designed to not break, change or depend on other functionality much, so that the "_old_ sync" methods can be run usefully again and observed. This patch isn't "new sync" (a different set of sync algorithms), but it is one of the foundations. For a while now Nimbus Eth1 only supported protocol `eth/63`. But that's obsolete, and very few nodes still support it. This meant Nimbus Eth1 would make slow progress trying to sync, as most up to date clients rejected it. The current specification is `eth/66`, and the functionality we really need is in `eth/64`. So why `eth/65`? - `eth/64` is essential because of the `forkId` feature. But `eth/64` is on its way out as well. Many clients, especially the most up to date Geth running the current hard-forks (Berlin/London) don't talk `eth/64` any more. - `eth/66` is the current specification, but some clients don't talk `eth/66` yet. We'd like to have the best peer connectivity during tests, and currently everything that talks `eth/66` also talks `eth/65`. - Nimbus Eth1 RLPx only talks one version at a time. (Without changes to the RLPx module. When those go in we'll add `eth/64..eth/66` for greater peer reach and testing the `eth/66` behaviour. For simplicity and decoupling, this patch contains just one version, the most useful.) What are `eth/64` and `eth/65`? - `eth/64` (EIP-2364) added `forkId` which allows nodes to distinguish between Ethereum (ETH) and Ethereum Classic (ETC) blockchains, which share the same genesis block. `forkId` also protects the system when a new hard fork is going to be rolled out, by blocking interaction with out of date nodes. The feature is required nowadays. We send the right details to allow connection (this has been tested a lot), but don't apply the full validation rules of EIP-2124/EIP-2364 in this patch. It's to keep this patch simple (in its effects) and because those rules have consequences best tested separately. In practice the other node will reject us when we would reject it, so this is ok for testing, as long as it doesn't get seriously deployed. - `eth/65` added more efficient transaction pool methods. - Then a later version of `eth/65` (without a new number) added typed transactions, described in [EIP-2976](https://eips.ethereum.org/EIPS/eip-2976). Why it's moved to `nimbus-eth1`: - Mainly because `eth/64` onwards depend on the current state of block synchronisation, as well as the blockchain's sequence of hard-fork block numbers, both of which are part of `nimbus-eth1` run-time state. These aren't available to pure `nim-eth` code. Although it would be possible to add an API to let `nimbus-eth1` set these numbers, there isn't any point because the protocol would still only be useful to `nimbus-eth1`. Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-07-21 20:00:03 +00:00
# User message 0x08: NewPooledTransactionHashes.
proc newPooledTransactionHashes(peer: Peer, txHashes: openArray[Hash256]) =
Snap sync simplify object inheritance (#1098) * Reorg SnapPeerBase descriptor, notably start/stop flags details: Instead of using three boolean flags startedFetch, stopped, and stopThisState a single enum type is used with values SyncRunningOk, SyncStopRequest, and SyncStopped. * Restricting snap to eth66 and later why: Id-tracked request/response wire protocol can handle overlapped responses when requests are sent in row. * Align function names with source code file names why: Easier to reconcile when following the implemented logic. * Update trace logging (want file locations) why: The macros previously used hid the relevant file location (when `chroniclesLineNumbers` turned on.) It rather printed the file location of the template that was wrapping `trace`. * Use KeyedQueue table instead of sequence why: Quick access, easy configuration as LRU or FIFO with max entries (currently LRU.) * Dissolve `SnapPeerEx` object extension into `SnapPeer` why; It is logically cleaner and more obvious not to inherit from `SnapPeerBase` but to specify opaque field object references of the merged `SnapPeer` object. These can then be locally inherited. * Dissolve `SnapSyncEx` object extension into `SnapSync` why; It is logically cleaner and more obvious not to inherit from `SnapSyncEx` but to specify opaque field object references of the `SnapPeer` object. These can then be locally inherited. Also, in the re-factored code here the interface descriptor `SnapSyncCtx` inherited `SnapSyncEx` which was sub-optimal (OO inheritance makes it easier to work with call back functions.)
2022-05-23 16:53:19 +00:00
when trEthTraceGossipOk:
trace trEthRecvReceived & "NewPooledTransactionHashes (0x08)", peer,
Snap sync simplify object inheritance (#1098) * Reorg SnapPeerBase descriptor, notably start/stop flags details: Instead of using three boolean flags startedFetch, stopped, and stopThisState a single enum type is used with values SyncRunningOk, SyncStopRequest, and SyncStopped. * Restricting snap to eth66 and later why: Id-tracked request/response wire protocol can handle overlapped responses when requests are sent in row. * Align function names with source code file names why: Easier to reconcile when following the implemented logic. * Update trace logging (want file locations) why: The macros previously used hid the relevant file location (when `chroniclesLineNumbers` turned on.) It rather printed the file location of the template that was wrapping `trace`. * Use KeyedQueue table instead of sequence why: Quick access, easy configuration as LRU or FIFO with max entries (currently LRU.) * Dissolve `SnapPeerEx` object extension into `SnapPeer` why; It is logically cleaner and more obvious not to inherit from `SnapPeerBase` but to specify opaque field object references of the merged `SnapPeer` object. These can then be locally inherited. * Dissolve `SnapSyncEx` object extension into `SnapSync` why; It is logically cleaner and more obvious not to inherit from `SnapSyncEx` but to specify opaque field object references of the `SnapPeer` object. These can then be locally inherited. Also, in the re-factored code here the interface descriptor `SnapSyncCtx` inherited `SnapSyncEx` which was sub-optimal (OO inheritance makes it easier to work with call back functions.)
2022-05-23 16:53:19 +00:00
hashes=txHashes.len
let ctx = peer.networkState()
let res = ctx.handleAnnouncedTxsHashes(peer, txHashes)
handleHandlerError(res)
Sync: Support for `eth/65` protocol This patch adds the `eth/65` protocol, documented at https://github.com/ethereum/devp2p/blob/master/caps/eth.md. This is an intentionally simple patch, designed to not break, change or depend on other functionality much, so that the "_old_ sync" methods can be run usefully again and observed. This patch isn't "new sync" (a different set of sync algorithms), but it is one of the foundations. For a while now Nimbus Eth1 only supported protocol `eth/63`. But that's obsolete, and very few nodes still support it. This meant Nimbus Eth1 would make slow progress trying to sync, as most up to date clients rejected it. The current specification is `eth/66`, and the functionality we really need is in `eth/64`. So why `eth/65`? - `eth/64` is essential because of the `forkId` feature. But `eth/64` is on its way out as well. Many clients, especially the most up to date Geth running the current hard-forks (Berlin/London) don't talk `eth/64` any more. - `eth/66` is the current specification, but some clients don't talk `eth/66` yet. We'd like to have the best peer connectivity during tests, and currently everything that talks `eth/66` also talks `eth/65`. - Nimbus Eth1 RLPx only talks one version at a time. (Without changes to the RLPx module. When those go in we'll add `eth/64..eth/66` for greater peer reach and testing the `eth/66` behaviour. For simplicity and decoupling, this patch contains just one version, the most useful.) What are `eth/64` and `eth/65`? - `eth/64` (EIP-2364) added `forkId` which allows nodes to distinguish between Ethereum (ETH) and Ethereum Classic (ETC) blockchains, which share the same genesis block. `forkId` also protects the system when a new hard fork is going to be rolled out, by blocking interaction with out of date nodes. The feature is required nowadays. We send the right details to allow connection (this has been tested a lot), but don't apply the full validation rules of EIP-2124/EIP-2364 in this patch. It's to keep this patch simple (in its effects) and because those rules have consequences best tested separately. In practice the other node will reject us when we would reject it, so this is ok for testing, as long as it doesn't get seriously deployed. - `eth/65` added more efficient transaction pool methods. - Then a later version of `eth/65` (without a new number) added typed transactions, described in [EIP-2976](https://eips.ethereum.org/EIPS/eip-2976). Why it's moved to `nimbus-eth1`: - Mainly because `eth/64` onwards depend on the current state of block synchronisation, as well as the blockchain's sequence of hard-fork block numbers, both of which are part of `nimbus-eth1` run-time state. These aren't available to pure `nim-eth` code. Although it would be possible to add an API to let `nimbus-eth1` set these numbers, there isn't any point because the protocol would still only be useful to `nimbus-eth1`. Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-07-21 20:00:03 +00:00
requestResponse:
# User message 0x09: GetPooledTransactions.
proc getPooledTransactions(peer: Peer, txHashes: openArray[Hash256]) =
Snap sync simplify object inheritance (#1098) * Reorg SnapPeerBase descriptor, notably start/stop flags details: Instead of using three boolean flags startedFetch, stopped, and stopThisState a single enum type is used with values SyncRunningOk, SyncStopRequest, and SyncStopped. * Restricting snap to eth66 and later why: Id-tracked request/response wire protocol can handle overlapped responses when requests are sent in row. * Align function names with source code file names why: Easier to reconcile when following the implemented logic. * Update trace logging (want file locations) why: The macros previously used hid the relevant file location (when `chroniclesLineNumbers` turned on.) It rather printed the file location of the template that was wrapping `trace`. * Use KeyedQueue table instead of sequence why: Quick access, easy configuration as LRU or FIFO with max entries (currently LRU.) * Dissolve `SnapPeerEx` object extension into `SnapPeer` why; It is logically cleaner and more obvious not to inherit from `SnapPeerBase` but to specify opaque field object references of the merged `SnapPeer` object. These can then be locally inherited. * Dissolve `SnapSyncEx` object extension into `SnapSync` why; It is logically cleaner and more obvious not to inherit from `SnapSyncEx` but to specify opaque field object references of the `SnapPeer` object. These can then be locally inherited. Also, in the re-factored code here the interface descriptor `SnapSyncCtx` inherited `SnapSyncEx` which was sub-optimal (OO inheritance makes it easier to work with call back functions.)
2022-05-23 16:53:19 +00:00
trace trEthRecvReceived & "GetPooledTransactions (0x09)", peer,
hashes=txHashes.len
let ctx = peer.networkState()
let txs = ctx.getPooledTxs(txHashes)
handleHandlerError(txs)
if txs.get.len > 0:
trace trEthSendReplying & "with PooledTransactions (0x0a)", peer,
sent=txs.get.len, requested=txHashes.len
else:
trace trEthSendReplying & "EMPTY PooledTransactions (0x0a)", peer,
sent=0, requested=txHashes.len
await response.send(txs.get)
Sync: Support for `eth/65` protocol This patch adds the `eth/65` protocol, documented at https://github.com/ethereum/devp2p/blob/master/caps/eth.md. This is an intentionally simple patch, designed to not break, change or depend on other functionality much, so that the "_old_ sync" methods can be run usefully again and observed. This patch isn't "new sync" (a different set of sync algorithms), but it is one of the foundations. For a while now Nimbus Eth1 only supported protocol `eth/63`. But that's obsolete, and very few nodes still support it. This meant Nimbus Eth1 would make slow progress trying to sync, as most up to date clients rejected it. The current specification is `eth/66`, and the functionality we really need is in `eth/64`. So why `eth/65`? - `eth/64` is essential because of the `forkId` feature. But `eth/64` is on its way out as well. Many clients, especially the most up to date Geth running the current hard-forks (Berlin/London) don't talk `eth/64` any more. - `eth/66` is the current specification, but some clients don't talk `eth/66` yet. We'd like to have the best peer connectivity during tests, and currently everything that talks `eth/66` also talks `eth/65`. - Nimbus Eth1 RLPx only talks one version at a time. (Without changes to the RLPx module. When those go in we'll add `eth/64..eth/66` for greater peer reach and testing the `eth/66` behaviour. For simplicity and decoupling, this patch contains just one version, the most useful.) What are `eth/64` and `eth/65`? - `eth/64` (EIP-2364) added `forkId` which allows nodes to distinguish between Ethereum (ETH) and Ethereum Classic (ETC) blockchains, which share the same genesis block. `forkId` also protects the system when a new hard fork is going to be rolled out, by blocking interaction with out of date nodes. The feature is required nowadays. We send the right details to allow connection (this has been tested a lot), but don't apply the full validation rules of EIP-2124/EIP-2364 in this patch. It's to keep this patch simple (in its effects) and because those rules have consequences best tested separately. In practice the other node will reject us when we would reject it, so this is ok for testing, as long as it doesn't get seriously deployed. - `eth/65` added more efficient transaction pool methods. - Then a later version of `eth/65` (without a new number) added typed transactions, described in [EIP-2976](https://eips.ethereum.org/EIPS/eip-2976). Why it's moved to `nimbus-eth1`: - Mainly because `eth/64` onwards depend on the current state of block synchronisation, as well as the blockchain's sequence of hard-fork block numbers, both of which are part of `nimbus-eth1` run-time state. These aren't available to pure `nim-eth` code. Although it would be possible to add an API to let `nimbus-eth1` set these numbers, there isn't any point because the protocol would still only be useful to `nimbus-eth1`. Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-07-21 20:00:03 +00:00
# User message 0x0a: PooledTransactions.
proc pooledTransactions(peer: Peer, transactions: openArray[Transaction])
nextId 0x0d
# User message 0x0d: GetNodeData.
proc getNodeData(peer: Peer, nodeHashes: openArray[Hash256]) =
trace trEthRecvReceived & "GetNodeData (0x0d)", peer,
hashes=nodeHashes.len
let ctx = peer.networkState()
let data = ctx.getStorageNodes(nodeHashes)
handleHandlerError(data)
trace trEthSendReplying & "NodeData (0x0e)", peer,
sent=data.get.len, requested=nodeHashes.len
await peer.nodeData(data.get)
# User message 0x0e: NodeData.
proc nodeData(peer: Peer, data: openArray[Blob]) =
trace trEthRecvReceived & "NodeData (0x0e)", peer,
bytes=data.len
let ctx = peer.networkState()
let res = ctx.handleNodeData(peer, data)
handleHandlerError(res)
Sync: Support for `eth/65` protocol This patch adds the `eth/65` protocol, documented at https://github.com/ethereum/devp2p/blob/master/caps/eth.md. This is an intentionally simple patch, designed to not break, change or depend on other functionality much, so that the "_old_ sync" methods can be run usefully again and observed. This patch isn't "new sync" (a different set of sync algorithms), but it is one of the foundations. For a while now Nimbus Eth1 only supported protocol `eth/63`. But that's obsolete, and very few nodes still support it. This meant Nimbus Eth1 would make slow progress trying to sync, as most up to date clients rejected it. The current specification is `eth/66`, and the functionality we really need is in `eth/64`. So why `eth/65`? - `eth/64` is essential because of the `forkId` feature. But `eth/64` is on its way out as well. Many clients, especially the most up to date Geth running the current hard-forks (Berlin/London) don't talk `eth/64` any more. - `eth/66` is the current specification, but some clients don't talk `eth/66` yet. We'd like to have the best peer connectivity during tests, and currently everything that talks `eth/66` also talks `eth/65`. - Nimbus Eth1 RLPx only talks one version at a time. (Without changes to the RLPx module. When those go in we'll add `eth/64..eth/66` for greater peer reach and testing the `eth/66` behaviour. For simplicity and decoupling, this patch contains just one version, the most useful.) What are `eth/64` and `eth/65`? - `eth/64` (EIP-2364) added `forkId` which allows nodes to distinguish between Ethereum (ETH) and Ethereum Classic (ETC) blockchains, which share the same genesis block. `forkId` also protects the system when a new hard fork is going to be rolled out, by blocking interaction with out of date nodes. The feature is required nowadays. We send the right details to allow connection (this has been tested a lot), but don't apply the full validation rules of EIP-2124/EIP-2364 in this patch. It's to keep this patch simple (in its effects) and because those rules have consequences best tested separately. In practice the other node will reject us when we would reject it, so this is ok for testing, as long as it doesn't get seriously deployed. - `eth/65` added more efficient transaction pool methods. - Then a later version of `eth/65` (without a new number) added typed transactions, described in [EIP-2976](https://eips.ethereum.org/EIPS/eip-2976). Why it's moved to `nimbus-eth1`: - Mainly because `eth/64` onwards depend on the current state of block synchronisation, as well as the blockchain's sequence of hard-fork block numbers, both of which are part of `nimbus-eth1` run-time state. These aren't available to pure `nim-eth` code. Although it would be possible to add an API to let `nimbus-eth1` set these numbers, there isn't any point because the protocol would still only be useful to `nimbus-eth1`. Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-07-21 20:00:03 +00:00
requestResponse:
# User message 0x0f: GetReceipts.
proc getReceipts(peer: Peer, hashes: openArray[Hash256]) =
Snap sync simplify object inheritance (#1098) * Reorg SnapPeerBase descriptor, notably start/stop flags details: Instead of using three boolean flags startedFetch, stopped, and stopThisState a single enum type is used with values SyncRunningOk, SyncStopRequest, and SyncStopped. * Restricting snap to eth66 and later why: Id-tracked request/response wire protocol can handle overlapped responses when requests are sent in row. * Align function names with source code file names why: Easier to reconcile when following the implemented logic. * Update trace logging (want file locations) why: The macros previously used hid the relevant file location (when `chroniclesLineNumbers` turned on.) It rather printed the file location of the template that was wrapping `trace`. * Use KeyedQueue table instead of sequence why: Quick access, easy configuration as LRU or FIFO with max entries (currently LRU.) * Dissolve `SnapPeerEx` object extension into `SnapPeer` why; It is logically cleaner and more obvious not to inherit from `SnapPeerBase` but to specify opaque field object references of the merged `SnapPeer` object. These can then be locally inherited. * Dissolve `SnapSyncEx` object extension into `SnapSync` why; It is logically cleaner and more obvious not to inherit from `SnapSyncEx` but to specify opaque field object references of the `SnapPeer` object. These can then be locally inherited. Also, in the re-factored code here the interface descriptor `SnapSyncCtx` inherited `SnapSyncEx` which was sub-optimal (OO inheritance makes it easier to work with call back functions.)
2022-05-23 16:53:19 +00:00
trace trEthRecvReceived & "GetReceipts (0x0f)", peer,
hashes=hashes.len
let ctx = peer.networkState()
let rec = ctx.getReceipts(hashes)
handleHandlerError(rec)
if rec.get.len > 0:
trace trEthSendReplying & "with Receipts (0x10)", peer,
sent=rec.get.len, requested=hashes.len
else:
trace trEthSendReplying & "EMPTY Receipts (0x10)", peer,
sent=0, requested=hashes.len
await response.send(rec.get)
Sync: Support for `eth/65` protocol This patch adds the `eth/65` protocol, documented at https://github.com/ethereum/devp2p/blob/master/caps/eth.md. This is an intentionally simple patch, designed to not break, change or depend on other functionality much, so that the "_old_ sync" methods can be run usefully again and observed. This patch isn't "new sync" (a different set of sync algorithms), but it is one of the foundations. For a while now Nimbus Eth1 only supported protocol `eth/63`. But that's obsolete, and very few nodes still support it. This meant Nimbus Eth1 would make slow progress trying to sync, as most up to date clients rejected it. The current specification is `eth/66`, and the functionality we really need is in `eth/64`. So why `eth/65`? - `eth/64` is essential because of the `forkId` feature. But `eth/64` is on its way out as well. Many clients, especially the most up to date Geth running the current hard-forks (Berlin/London) don't talk `eth/64` any more. - `eth/66` is the current specification, but some clients don't talk `eth/66` yet. We'd like to have the best peer connectivity during tests, and currently everything that talks `eth/66` also talks `eth/65`. - Nimbus Eth1 RLPx only talks one version at a time. (Without changes to the RLPx module. When those go in we'll add `eth/64..eth/66` for greater peer reach and testing the `eth/66` behaviour. For simplicity and decoupling, this patch contains just one version, the most useful.) What are `eth/64` and `eth/65`? - `eth/64` (EIP-2364) added `forkId` which allows nodes to distinguish between Ethereum (ETH) and Ethereum Classic (ETC) blockchains, which share the same genesis block. `forkId` also protects the system when a new hard fork is going to be rolled out, by blocking interaction with out of date nodes. The feature is required nowadays. We send the right details to allow connection (this has been tested a lot), but don't apply the full validation rules of EIP-2124/EIP-2364 in this patch. It's to keep this patch simple (in its effects) and because those rules have consequences best tested separately. In practice the other node will reject us when we would reject it, so this is ok for testing, as long as it doesn't get seriously deployed. - `eth/65` added more efficient transaction pool methods. - Then a later version of `eth/65` (without a new number) added typed transactions, described in [EIP-2976](https://eips.ethereum.org/EIPS/eip-2976). Why it's moved to `nimbus-eth1`: - Mainly because `eth/64` onwards depend on the current state of block synchronisation, as well as the blockchain's sequence of hard-fork block numbers, both of which are part of `nimbus-eth1` run-time state. These aren't available to pure `nim-eth` code. Although it would be possible to add an API to let `nimbus-eth1` set these numbers, there isn't any point because the protocol would still only be useful to `nimbus-eth1`. Signed-off-by: Jamie Lokier <jamie@shareable.org>
2021-07-21 20:00:03 +00:00
# User message 0x10: Receipts.
proc receipts(peer: Peer, receipts: openArray[seq[Receipt]])