1.0.0 was never an intentional release and the nim-eth repo is not yet
stable in terms of API, thus we start at 0.5.0 and go from there for the
first tagged release.
Most of the eth code in this repo now is aligned with various specs
meaning that most API can be considered "mostly" stable, but there are
still aspects being worked on as well as a potential future
reorganisation of the code turning nim-eth into a more "pure"
spec-driven core ethereum infrastructure repo, removing in the process
application-level stuff that happens to be common between nimbus-eth1
and eth2 and ended up in here for convenience.
The assert would occur when the rlp size of the a node in the
nodes rlp list is incorrectly set too high and then the next
`listElem` call for the next node will start from the
incorrect data. When that data is not a list the assert in
`listElem` will be triggered.
Fixed by adding a `listLen` call which checks if it is a list.
Added also more strictness by:
- Checking if that list is of len 4, which it must be
- raising immediatly on invalid IP length
- raising immediatly on invalid public key / node id
+ test cases
Without these added transanction/header import/exports decoding
blocks or block bodies would fail if one only had imported
blocks_rlp.
Also removes unused import + add copyright comments in rlp code.
* drop support for v4 (obsolete, doesn't work with all clients since
they use chunking and other obsolete v4 features that we're missing or
don't support it at all)
* rework asyncraises
* always store generated p2p macro code (similar to eth2)
* preparation for chronos cancellation support (more to be done here)
* when peer is disconnected, ensure pending handshakes and requests are
notified (instead of waiting for timeout)
* disallow raising from `onPeerDisconnected` - this simplifies
disconnection coordination among async tasks
* introduce several warning logs for protocol breaches - these should be
removed eventually, pending q/a on the rlpx layer in general
* fix snappy compression - the payload without msgId should be
compressed
* remove strict checks on unused fields in RLPx message header (this
matches geth behavior and the spirit of EIP-8)
* add snappy dep
* cleanup macros
* add test cases and fix counting function
* add check statements
* remove deprecated support for Option
* replace some logic
* remove debug print
* disconnect peers that send non-hello messages during initial hello
step
* fix devp2p protocol version - 4 because we don't implement snappy
(yet) - this is cosmetic since this particular version field is not
actually being used
* fix ack message length checking
* move RLPx transport code to separate module, annotate with asyncraises
* increase max RLPx message size to 16mb, per EIP-706
* make sure both accept/connect timeout after 10s
* aim to log every connection attempt once at debug level
* make capability-id/context-id check more accurate
* disallow random messages before hello
Support for previous versions was
[removed](7194c847b6)
from geth in 2021 after other clients had migrated - should be safe to
remove here also.
* fix snappy detection - it should use the hello version, not the RLPx
handshake
* simplify generation of auth messages
* pad 100-300 bytes like spec suggests
TODO: error handling is all over the place - will be addressed in a
follow-up PR
The warning gives too many false positives - until things work properly,
locally one can just remove it to find instances where it gets used..
https://github.com/nim-lang/Nim/issues/24241
For EIP-7702 SetCode transaction, it is necessary to identify the
authority issuing an authorization. Its account's code is set to
proxy to the authorization address if successful.
Root encoding is on the hot path for block verification both in the
consensus (when syncing) and execution clients and oddly consititutes a
significant part of resource usage even though it is not that much work.
While the trie code is capable of producing a transaction root and
similar feats, it turns out that it is quite inefficient - even for
small work loads.
This PR brings in a helper for the specific use case of building tries
of lists of values whose key is the RLP-encoded index of the item.
As it happens, such keys follow a particular structure where items end
up "almost" sorted, with the exception for the item at index 0 which
gets encoded as `[0x80]`, ie the empty list, thus moving it to a new
location.
Armed with this knowledge and the understanding that inserting ordered
items into a trie easily can be done with a simple recursion, this PR
brings a ~100x improvement in CPU usage (360ms vs 33s) and a ~50x
reduction in memory usage (70mb vs >3gb!) for the simple test of
encoding 1000000 keys.
In part, the memory usage reduction is due to a trick where the hash of
the item is computed as the item is being added instead of storing it in
the value.
There are further reductions possible such as maintaining a hasher per
level instead of storing hash values as well as using a direct-to-hash
rlp encoder.
Transaction signing is something that happens in a lot of places - this
PR introduces primitives for transaction signing in `transaction_utils`
such that we can use the same logic across web3/eth1/etc for this simple
operation.
`transaction_utils` also contains a few more "spec-derived" helpers for
working with transactions, such as the computation of a contract address
etc that cannot easily be introduced in `transactions` itself without
bringing in dependencies like secp and rlp, so they end up in a separate
module.
Finally, since these modules collect "versions" of these transaction
types across different eips, some tests are moved to follow the same
structure.
As it happens, the two share the exact same interface (even the test
suite removed in this PR passes) - `minilru` has an edge on efficiency
however, avoiding the doubly linked list node allocations etc
serialize fewer non-eth types and more eth types - this module is
fraught with issues however since there's no one good "canonical"
encoding to choose - this goes back to the json ser framework lacking
good isolation between projects.
Since these types were written, we've gained an executable spec:
https://github.com/ethereum/execution-specs
This PR aligns some of the types we use with this spec to simplify
comparisons and cross-referencing.
Using a `distinct` type is a tradeoff between nim ergonomics, type
safety and the ability to work around nim quirks and stdlib weaknesses.
In particular, it allows us to overload common functions such as `hash`
with correct and performant versions as well as maintain control over
string conversions etc at the cost of a little bit of ceremony when
instantiating them.
Apart from distinct byte types, `Hash32`, is introduced in lieu of the
existing `Hash256`, again aligning this commonly used type with the spec
which picks bytes rather than bits in the name.
* Using unsigned types for message type and requst IDs
why:
Negative values are neither defined for RLP nor in the protocol specs
which refer to the RLPs (see yellow paper app B clause (199).
* Fix `int` argument (must be `uint`) in fuzzing tests
why:
Not part of all tests so it slipped through.
* Restricting exception catcher
why:
`CatchableError` is not needed here
* Check data length before converting to `openArray[]`
why:
Getting the first entry of an `openArray[]` crashes with `IndexDefect`.
This is particularly annoying when decoding messages in rlpx.
* Added unit test using rlpx message that causes this problem to detect
Other implementations of MPT delete entries when attempting to put empty
value, because empty value cannot exist in RLP. We should match the
behaviour.
- https://github.com/ethereum/py-trie/pull/109
Also cross-checked with Geth and Ethereumjs implementations.
- Rework to have exception raise only at rlp decoding and use
result types from then onwards
- Adjust the current API to have result versions and deprecated
the ones which had var Record + bool
- Add PublickKey to the Record object, as this allows us to skip
fromRaw calls whenever access is needed to the public key
- Add a TypedRecord.fromRecord which cannot fail and deprecate
the old one
- Some other minor clean-up & re-ordering
- Rework adding and updating of fields by having an insert call
that gets used everywhere. Avoiding also duplicate keys. One
side-effect of this is that ENR sequence number will always get
updated on an update call, even if nothing changes.
- Deprecate initRecord as it is only used in tests and is flawed
- Assert when predefined keys go into the extra custom pairs.
Any of the predefined keys are only to be passed now via specific
parameters to make sure that the correct types are stored in ENR.
- Clearify the Opt.none behaviour for Record.update
- When setting ipv6, allow for tcp/udp port fields to be used
default
- General clean-up
- Rework/clean-up completely the ENR tests.
Jacek Sieka
Kim De Mey
Chirag Parmar
Csaba Kiraly
andri lim
Etan Kissling
tersec
Miran
Kim De Mey
Jordan Hrycaj
Etan Kissling