Instead of patching BlockHeader or BlockBody codec
each time it get additional optional fields,
this PR make the rlp codec automatically handle
optional fields. Thus rlp codec overloading of
EthBlock, BlockHeader, and BlockBody can be removed.
Using exception in rlp.append turn out creates more problem than it solve.
`blockHash` operation is literally everywhere in nimbus-eth1, nimbus-eth2, and fluffy-code.
The compiler begins to complain about unlisted exception.
Rlp bytes coming from network and test vectors already verified by the decoder.
So, any invalid optional fields of blockheader that come into rlp.append means
programming error. Assertion will be more apropiate in this situation and less problematic.
including:
- fix rlp.append of BlockHeader
- add tests for rlp.readRecordType of BlockBody
- add tests for rlp.append of BlockHeader
- add tests for EthBlock roundtrip
* In the incomplete-db node-existence check, don't use contains.
(Using contains led to a problem with CaptureDB.)
* In the incomplete-db check, just checking len > 0 isn't right.
* Oh, I needed the AssertionDefect thing too.
* Need this when compiling under older versions of Nim.
* Sometimes we want missing nodes to be errors, sometimes not.
* Add metrics related to devp2p peer connections
* Avoid reconnecting to peers that just failed connection
- Add SeenTable to avoid reconnecting to peers immediately after
a failed connect. Depending on the failure, the amount of time is
different. This is similar to what is done in nimbus-eth2.
- Attempt to rework rlpxConnect at the same time, in order to
make sure that errors are properly handled. The current structure
is far from ideal, but it is hopefully a small step in the right
direction. To many oddities in there right now to really rework
rlpxConnect properply.
* Fix rlpx thunk fuzzer
The old code, which did it in sequence, made it take a very long
time to accumulate peers in the peer pool, particularly because
occasionally a connection-attempt would hang for a couple of minutes
before timing out.
Adds `Withdrawal` type according to EIP-4895, and extends `BlockHeader`
accordingly. Also adds RLP encoding support for `Withdrawal` to enable
building `BlockHeader` (used by Nimbus-CL in empty block prod fallback).
* persist ENRs (parsed) from the discovery5 protocol
* filter fork_digest from eth2 data
* Update eth/p2p/discoveryv5/dcli.nim
Co-authored-by: Kim De Mey <kim.demey@gmail.com>
* apply reviewer's suggestions
Co-authored-by: Kim De Mey <kim.demey@gmail.com>
- Fix two possible RangeErrors, due to negative seq allocation
in decodeAuthMessageEIP8 and in decodeAckMessageEIP8
- Set the minimum auth message size to AuthMessageEIP8Length,
in case there are clients that no longer add padding
- Add tests for invalid length cases
RLP Enum deserialization would currently not check if "hole values"
were attempted to be converted to the enum type. Now use
checkedEnumAssign and fail with RlpTypeMismatch on invalid values.
There is at least one occurance of an enum with holes in rlpx p2p:
DisconnectionReason. For this enum the issue could occur.
Also:
- Added enum RLP tests and rlpx p2p disconnect message tests to
test the DisconnectionReason with enum hole value.
- Fixed worse custom DisconnectionReason decoding occurance
in rlpx in waitSingleMsg proc where this issue could occur.
- Unify the debug log and info log into one. Removing the
redundant information
- Log the custom ENR fields more pretty
- Make the JSON format logging more pretty for several types
Nim devel brach(1.7.1) introduce gc=orc as default mode.
Because the p2p protocol using unsafe pointer operations
for it's ProtocolInfo and using global variables scattered
around, the orc mistakenly(or maybe correctly) crash the protocol.
Seems there are two occasions possible where we try to ping the
own local node which causes an assert
- One via the bounding
- One via a received ping, which is strange in the first place but
notice in a stack trace
* Move NetworkId type to common eth_types
NetworkId is after all a common type and this way it avoid an
application that requires it to also import all devp2p related
network types.
RLP related calls are moved to eth_types_rlp, this means that p2p
protocols that send NetworkId over the wire need to import this
too. Or just common in general.
* Remove # in front of multiline comment end bracket
These seem to be interpreted wrongly by the GitHub code browser.
This is specifically useful for when the failure is caused by an
"Operation not permitted", as then we can see for which specific
port(s) a firewall might be blocking outgoing traffic.
* tighter nimcrypto imports
* install openssl on macos for Nim devel
* add getTotalDifficulty base method to AbtractChainDB
Co-authored-by: Jacek Sieka <jacek@status.im>
`eth_types` is being imported from many projects and ends up causing
long build times due to its extensive import lists - this PR starts
cleaning some of that up by moving the chain DB and RLP to their own
modules.
this PR also moves `keccakHash` to its own module and uses it in many
places.
* Fix stability issues
why:
Handling malformed messages typically raises `RangeError` exceptions
when de-serialising RLP, or decoding message data. This is an
(incomplete) attempt to weed out some out it driven by real live
tests.
remark:
Employing the new `snap` protocol there might be different views on what
the messages really contain (currently specs are more a hint.)
* Update RLP exception handling
* Undo effect-less patch
why:
problem occurred somewhere above the try/catch handler
* Using `checkedEnumAssign()` for RLP enum
* Provide dedicated `DisconnectionReason` enum type RLP reader
why:
Without this reader, the program communicating via RLPX will crash when
receiving out of bound reason codes disconnect message.
Out of bound value assignments to an enum causes a `RangeError`defect
and consequently the program to terminate. This `RangeError` is avoided
here and a `MalformedRlpError` catchable error raised.
* Using default exception type in bespoke `read(DisconnectionReason)`
why:
This should not differ from the default enum parser. The particular
message is different and more targeted, here.
Note: The default RLP parser was not used because `
`array[1,DisconnectionReason]` is currently not properly handled and
should give a siliar error message as a `DisconnectionReason` error.
* De-clutter, custom read() was not needed
Co-authored-by: jordan <jordan@curd.mjh-it.com>
* Handle the decodeAuthMessage error case separatly and log to trace
Garbage data on the TCP port (e.g. from port scanners) would
cause lots of error log messages, so log this to trace and get rid
of a (little) bit of exception usage in the process.
* Remove usage of result var in rlpxAccept and rlpxConnect
* Discv4: Add ENRRequest & ENRResponse msgs to avoid fails on these
Fix#499
These messages are not implemented yet however, but just ignored.
* disc: updateExternalIp()
New public proc that can be used to inform the discovery subsystem about
a changed external IP (as reported by UPnP/NAT-PMP in some other module).
why:
Rlp errors throw exceptions which cause the dispatcher loop to
terminate the current session immediately.
details:
The DisconnectionReasonList message requires a single entry list.
Observed and now accepted deviations are:
Geth: single byte number
bor(a Geth fork): blobbed single entry list containing a number
why:
This is a legacy feature and its usage should peter out over time.
details:
Use -d:chunked_rlpx_enabled for enabling chunked RLPx message handling.
why:
For some reason, Nethermind insists on sending chunked messages to
the syncing peer. Unfortunately, for the test networks the Nethermind
modes are the importent ones as they speak eth/65 as well while others
like Geth only support eth/66 which is not implemented here, yet.
- add eventLoop to control all incoming events
- change semantic of write to asynchronously block only when send buffer is full, and not when bytes do not fit into send window
- change handling of receive buffer, to start dropping packets if the reorder buffer and receive buffer are full. Old behaviour was to async block unless there is space which could lead to resource exhaustion attacks
The ENR code used to be solely exception based, and these
exceptions where a left-over of that. They are useless as later
calls use Result anyhow.
Additionally, they cause quite the performance loss because they
are used in the "common path" for the toTypedRecord call, e.g.
when reading the fields of ip6, tcp6 and udp6.
Currently only setting `--styleCheck:hint` as there are some
dependency fixes required and the compiler seems to trip over the
findnode MessageKind, findnode Message field and the findNode
proc. Also over protocol.Protocol usage.
Kim De Mey
tersec
Jacek Sieka
andri lim
Ivan Folgueira Bande
Adam Spitz
Jordan Hrycaj
zah
Etan Kissling
Lorenzo Delgado
KonradStaniec
Mikel Cortes
Justin Traglia
oskarth
Ștefan Talpalaru
Jordan Hrycaj
Zahary Karadjov