Commit Graph

376 Commits

Author SHA1 Message Date
Kim De Mey 8088fe72d7
Update discv5 to use non deprecated ENR calls and simplify code (#710)
And due to avoiding an extra PublicKey.fromRaw call we get a
little performance boost also.
2024-06-27 16:18:21 +02:00
Kim De Mey d7577f59d7
Rework of ENR decoding code (#709)
- Rework to have exception raise only at rlp decoding and use
result types from then onwards
- Adjust the current API to have result versions and deprecated
the ones which had var Record + bool
- Add PublickKey to the Record object, as this allows us to skip
fromRaw calls whenever access is needed to the public key
- Add a TypedRecord.fromRecord which cannot fail and deprecate
the old one
- Some other minor clean-up & re-ordering
2024-06-27 15:15:23 +02:00
Kim De Mey 7f20d79945
Overhaul of ENR implementation - part I (#707)
- Rework adding and updating of fields by having an insert call
that gets used everywhere. Avoiding also duplicate keys. One
side-effect of this is that ENR sequence number will always get
updated on an update call, even if nothing changes.
- Deprecate initRecord as it is only used in tests and is flawed
- Assert when predefined keys go into the extra custom pairs.
Any of the predefined keys are only to be passed now via specific
parameters to make sure that the correct types are stored in ENR.
- Clearify the Opt.none behaviour for Record.update
- When setting ipv6, allow for tcp/udp port fields to be used
default
- General clean-up
- Rework/clean-up completely the ENR tests.
2024-06-24 14:57:39 +02:00
Kim De Mey 26212c881b
Use results.Opt instead of option in discv5, utp and nat (#705)
+ some other minor cleanups
2024-06-18 18:09:27 +02:00
Kim De Mey 207244c2db
Add more missing async raises in uTP (#702) 2024-06-12 18:01:23 +02:00
andri lim 9b6497ed8a
GasInt, BlockNumber, fields name closer to the spec (#696) 2024-06-12 00:13:11 +07:00
Kim De Mey c3f9160fd2
Add async raises annotations for uTP code (#692)
* Add async raises annotations for uTP code

* Avoid compiler error + further clean-up
2024-06-11 13:56:37 +02:00
Jacek Sieka 5599435901
ethblock: txs -> transactions (#694)
Using the same name as BlockBody to make them easily interchangeable -
* add a few inits
2024-06-08 19:25:25 +02:00
Jacek Sieka bb5cb6a4d0
rlp: refresh code (#683)
A first step in cleaning up RLP, which has lots of interesting issues -
the next step would be to clean up the exception handling as well
(Resultify?)

* remove `RlpNode` (unused)
* single-pass parsing for most functionality via RlpItem
* stricter conformance to spec
  * remove float support
  * warn about signed integers
  * raise on invalid RLP earlier
* avoid several pointless allocations, in particular in `listLen`,
`listElem` etc
* include spec docs
2024-05-26 09:58:24 +02:00
Jacek Sieka 0addffcc6c
binary tries: remove (#684)
not used anywhere
2024-05-26 09:57:59 +02:00
Etan Kissling c482b4c5b6
Introduce wrapper type for EIP-4844 transactions (#682)
EIP-4844 blob sidecars are a concept that only exists in the mempool.
After inclusion of a transaction into an execution block, only the
versioned hash within the transaction remains. To improve type safety,
replace the `Transaction.networkPayload` member with a wrapper type
`PooledTransaction` that is used in contexts where blob sidecars exist.
2024-05-13 11:19:00 +03:00
andri lim d8209f623f
Fix improper yield usage in rlpx and refine exception handling (#679)
* Fix improper yield usage in rlpx and refine exception handling

* Handle post hello step error
2024-02-19 14:16:33 +07:00
Kim De Mey 8ab060b19f
Move Ethereum specific RLP encoding tests under tests/common (#673)
This is more according to the structure of the code itself.
And also, currently some of these tests live under tests/rlp but
some under tests/common. Could use some further re-organisation
within the different tests, but this move is a first step.
2024-02-09 17:30:24 +01:00
Kim De Mey 2ad07a6010
Fix two bugs in Receipts RLP encoding/decoding (#672)
1. Fix Assertion error when receipt is not a List nor has a single
byte value. Receiving such garbage data would cause a crash.
2. Fix decoding of Receipt list by adding the missing Blob
encapsulation

Also added tests for these scenarios.
2024-02-05 23:22:15 +01:00
Kim De Mey 21f3014f54
Remove usage of stew/shim/net (#670)
* Remove usage of stew/shim/net and replace with std/net where needed

* Remove chronosStrictException compile flag
2024-01-24 09:09:11 +01:00
Kim De Mey 19965bab95
Fix chronos related deprecated warnings in uTP code part II (#669)
This also makes the uTP SendCallBack not returning a Future any
more as it is not used in sendData anyhow. And in case of uTP
over discv5, discv5 send call is already not async.

This gives quite a noticable throughput benchmark improvement over
with uTP over UDP, and a slightly noticable with uTP over discv5
2024-01-23 18:41:38 +01:00
Kim De Mey 496bcdab74
Fix chronos related deprecated warnings in discv5 and uTP (#667)
* Fix chronos related deprecated warnings in discv5

* Fix chronos related deprecated warnings in uTP code

* Improve discv5 closeWait order of events and add noCancel
2024-01-22 17:08:08 +01:00
andri lim b75ee36576
Refactor p2pProtocol internals (#657)
* Refactor p2pProtocol internals

* Attempt to fix rlp crash with Nim v2 (#658)

* Attempt to fix rlp crash with Nim v2
* Fix test_ecies for nim v2
* Reduce compiler warnings
* Resolve ambiquity in testutils

* Disable nim devel continue-on-error
2024-01-22 16:47:46 +07:00
Kim De Mey 974a995b21
Add uTP over discv5 test and small uTP performance improvements (#663)
- Add a multiple sockets use test for uTP over discv5
- Use assign2 for the biggest consumer of genericAssignAux in uTP
- Avoid calling exists on the growable buffer when there is no
place in the socket window.
2024-01-18 12:07:03 +01:00
Kim De Mey d4927593a1
Add data over multiple sockets uTP test (#662) 2024-01-17 22:45:20 +01:00
Etan Kissling a6f79a6000
Adjust test names and comments for `blobGasUsed` field (#661)
`blobGasUsed` was not mentioned in `test_rlp_codec` test names, despite
being used. Further, update the idea of `genOptionalFieldsValidation`
to also check for `blobGasUsed`, and fix the check to catch unsupported
combinations of optional fields.
2024-01-16 22:02:06 +01:00
Kim De Mey e52f5fac0a
Clean-up, correct and clarify utp_protocol tests (#660) 2024-01-16 18:10:26 +01:00
Kim De Mey cbcd1fd307
Fix for uTP issues with latest chronos (#655)
* Fix for uTP issues with latest chronos

* Fix for discv5 issue with latest chronos

* Increase amount of data send in uTP and uTP over discv5 tests
2024-01-12 17:35:44 +01:00
Kim De Mey 11049fb87d
Fix a bunch of compiler hints and warnings in uTP and discv5 (#652)
* Fix a bunch of compiler hints and warnings in uTP and discv5

* Leave in the cancel() for now
2024-01-12 09:49:06 +01:00
web3-developer 5aaeb67ebe
Added additional tests to cover HexaryTrie MPT proofs. (#650)
* Added additional tests to cover HexaryTrie MPT proofs.

* Minor updates based on PR comments.
2023-12-22 23:34:12 +08:00
Kim De Mey e5c2b1784e
Replace registerCustomScalarFunction with createCustomFunction (#649)
* Replace registerCustomScalarFunction with createCustomFunction

The nice thing about `registerCustomScalarFunction` and
`registerCustomScalarFunction` was that it kept the sqlite calls
of type sqlite3_value_xxx inside kvstore_sqlite.

The big downside however is that the "custom" call is awfully
specific due to the specific function signature it demands.
It is likely that for each different use case, a new version
needs to be added.

In this PR the code gets altered to `createCustomFunction` which
allows to just create a custom function with directly xFunc
signature of sqlite. The downside of this is that you still need
to import sqlite_abi in your local code to have access to each of
the sqlite3_value_xxx and sqlite3_result_xxx calls. These could of
course also be wrapped.

* Fix custom sql func test
2023-11-21 14:39:24 +01:00
tersec ca4898e24a
switch NAT module from ValidIpAddress to IpAddress (#647)
* switch NAT module from ValidIpAddress to IpAddress

* fix dcli build

* switch discv5 from ValidIpAddress to IpAddress

* fix fuzzers build

* mostly finish nim-eth to IpAddress transition
2023-11-10 09:28:21 +03:00
tersec 957d8491af
fix XDeclaredButNotUsed warnings in tests/ (#645) 2023-10-29 04:06:48 +00:00
andri lim 4b00b230d0
Remove unused bootnodes and add Holesky bootnodes (#644)
* Remove unused bootnodes and add Holesky bootnodes

* Fix enode test
2023-10-25 11:07:55 +07:00
andri lim fe88d9e6b7
Change maxFeePerBlobGas from GasInt to UInt256 (#640)
* Change maxFeePerBlobGas from GasInt to UInt256

per latest spec:
https://eips.ethereum.org/EIPS/eip-4844#blob-transaction

* fix eip-4844 test
2023-09-24 14:33:24 +07:00
Jacek Sieka 02c8a1276b
avoid use of shallow (#636)
this probably has some tiny overhead but it shouldn't practically matter
2023-09-20 09:24:24 +02:00
Kim De Mey 0d7e7448c4
Allow passing along the handshake ENR through talkresp handler (#634)
This allows for protocols build on top of discv5 to use the ENR
provided in the handshake directly, instead of having to rely
on requesting it from the discv5 routing table.
2023-08-30 17:44:05 +02:00
jangko 074edff1b4
EIP-4788: Add parentBeaconBlockRoot field to BlockHeader 2023-08-18 17:52:55 +07:00
jangko 521db96a65
rename data gas to blob gas 2023-07-28 10:19:00 +07:00
jangko 6b8a7b009e
implement EIP-4844: Shard Blobs Transactions 2023-06-23 13:48:51 +07:00
Kim De Mey c5dd5e4115
Enable tests with also json sink and fix failures (#624)
And a few minor cleanups
2023-06-22 16:13:16 +00:00
Kim De Mey d74dc40bee
Style fixes and comment improvements on uTP code (#623) 2023-06-22 11:31:30 +00:00
tersec d766fc435e
remove stddefects import (#618) 2023-06-11 12:14:10 +00:00
tersec c608426d03
add/change Deneb fields per https://github.com/ethereum/execution-apis/pull/417 (#616) 2023-06-03 18:47:55 +00:00
andri lim 91b2b9d2ed
rlp codec support optional fields (#613)
Instead of patching BlockHeader or BlockBody codec
each time it get additional optional fields,
this PR make the rlp codec automatically handle
optional fields. Thus rlp codec overloading of
EthBlock, BlockHeader, and BlockBody can be removed.
2023-05-30 16:02:02 +03:00
jangko 67bbd88616
Replace exception with assertion in rlp.append of BlockHeader
Using exception in rlp.append turn out creates more problem than it solve.
`blockHash` operation is literally everywhere in nimbus-eth1, nimbus-eth2, and fluffy-code.
The compiler begins to complain about unlisted exception.
Rlp bytes coming from network and test vectors already verified by the decoder.
So, any invalid optional fields of blockheader that come into rlp.append means
programming error. Assertion will be more apropiate in this situation and less problematic.
2023-05-25 15:27:37 +07:00
jangko 7ff6de2367
fix rlp.readRecordType of BlockBody and friends
including:
- fix rlp.append of BlockHeader
- add tests for rlp.readRecordType of BlockBody
- add tests for rlp.append of BlockHeader
- add tests for EthBlock roundtrip
2023-05-23 14:37:47 +07:00
Kim De Mey d57a1094b1
Remove all Defect raises to avoid unnecessary warnings (#607)
As this module dropped support for Nim versions < 1.6, all Defect
raises can be removed to avoid unnecessary warnings
2023-05-10 15:50:04 +02:00
tersec 7564b21ade
support only Nim 1.6+ (#606) 2023-05-09 20:19:44 +00:00
jangko 917888356e
no more wild constants and rename ecdhSecret to ecdhSharedSecret 2023-04-21 16:08:57 +07:00
jangko 9bf2e1786b
remove ecdhRaw usage and replace it with ecdh+custom hash function 2023-04-20 15:45:23 +07:00
Jordan Hrycaj 9a1d35f803
Fix block body encoding (#598)
why:
  List wrapper was missing in `append()` mixin.
2023-04-19 13:26:41 +01:00
Kim De Mey d2ba753792
Add rlpx metrics and avoid immediate peer reconnections (#585)
* Add metrics related to devp2p peer connections

* Avoid reconnecting to peers that just failed connection

- Add SeenTable to avoid reconnecting to peers immediately after
a failed connect. Depending on the failure, the amount of time is
different. This is similar to what is done in nimbus-eth2.
- Attempt to rework rlpxConnect at the same time, in order to
make sure that errors are properly handled. The current structure
is far from ideal, but it is hopefully a small step in the right
direction. To many oddities in there right now to really rework
rlpxConnect properply.

* Fix rlpx thunk fuzzer
2023-03-16 16:45:12 +01:00
Kim De Mey 29b14749fa
Improve uTP decoded packet logs + style and comment clean-up (#593)
* Improve uTP decoded packet logs + style and comment clean-up

* Don't test for the exact error strings in uTP decode + clean-up
2023-03-14 16:17:39 +00:00
Jacek Sieka ade6d19691
Defect fixes in kvstore (#589)
* Defect fixes in kvstore

* compile fixes
2023-02-21 18:54:30 +01:00