Commit Graph

791 Commits

Author SHA1 Message Date
andri lim 224048a1d5
Revert perPeerMsgIdVar type until it properly fixed (#701)
* Revert perPeerMsgIdVar type until it properly fixed

* Add reminder
2024-06-12 13:29:57 +07:00
andri lim 4cccf25af7
Reduce compiler warnings related to rlp encode int (#699) 2024-06-12 09:29:18 +07:00
andri lim 9b6497ed8a
GasInt, BlockNumber, fields name closer to the spec (#696) 2024-06-12 00:13:11 +07:00
Kim De Mey c3f9160fd2
Add async raises annotations for uTP code (#692)
* Add async raises annotations for uTP code

* Avoid compiler error + further clean-up
2024-06-11 13:56:37 +02:00
Kim De Mey 3d66c5b899
Fix ImplicitDefaultValue warnings + replace some obsolete inits (#697) 2024-06-10 21:19:25 +02:00
Kim De Mey 50e71b2daa
Add async raises annotations for discv5 (#690) 2024-06-10 13:47:35 +02:00
Jacek Sieka c9b545b6c4
No sink (#695)
https://github.com/nim-lang/Nim/issues/23354
2024-06-10 10:12:32 +02:00
Jacek Sieka 5599435901
ethblock: txs -> transactions (#694)
Using the same name as BlockBody to make them easily interchangeable -
* add a few inits
2024-06-08 19:25:25 +02:00
Jacek Sieka b9b522f7ac
keccak doesn't need `init` (#693) 2024-06-07 22:30:32 +02:00
Jacek Sieka c02e050db8
avoid `burnMem` for `keccakHash` (#691)
We don't need to clear keccak context after hashing ethereum data since
it is already public - similar to
https://github.com/status-im/nimbus-eth2/pull/222.

Because we overwrite the result fully, we also don't need to zero first.
2024-06-07 16:27:38 +02:00
Jacek Sieka d935c0de47
rlp: avoid or make faster seq copies (#689) 2024-06-06 20:27:43 +02:00
Jacek Sieka 68bd675b75
faster `hash` for `Address` (#688) 2024-06-05 22:46:23 +02:00
Jacek Sieka 0e83cfda97
results: use standalone repo (#687) 2024-05-30 14:46:57 +02:00
Jacek Sieka 50b3d0193c
Revert missing hexary trie node work (#686)
* Revert "In the incomplete-db node-existence check, don't use contains. (#603)"

This reverts commit 4b818e8307.

* Revert "Some changes to make hexary.nim better able to handle incomplete DBs (#602)"

This reverts commit f5dd26eac0.

* Revert "Added maybeGet, for working with incomplete DBs. (#595)"

This reverts commit 4754543605.
2024-05-28 10:05:57 +02:00
Jacek Sieka 258798873c
eth_types: remove unused ref types (#685)
remove unused ref types
2024-05-28 08:42:13 +02:00
Jacek Sieka bb5cb6a4d0
rlp: refresh code (#683)
A first step in cleaning up RLP, which has lots of interesting issues -
the next step would be to clean up the exception handling as well
(Resultify?)

* remove `RlpNode` (unused)
* single-pass parsing for most functionality via RlpItem
* stricter conformance to spec
  * remove float support
  * warn about signed integers
  * raise on invalid RLP earlier
* avoid several pointless allocations, in particular in `listLen`,
`listElem` etc
* include spec docs
2024-05-26 09:58:24 +02:00
Jacek Sieka 0addffcc6c
binary tries: remove (#684)
not used anywhere
2024-05-26 09:57:59 +02:00
Etan Kissling c482b4c5b6
Introduce wrapper type for EIP-4844 transactions (#682)
EIP-4844 blob sidecars are a concept that only exists in the mempool.
After inclusion of a transaction into an execution block, only the
versioned hash within the transaction remains. To improve type safety,
replace the `Transaction.networkPayload` member with a wrapper type
`PooledTransaction` that is used in contexts where blob sidecars exist.
2024-05-13 11:19:00 +03:00
Eugene Kabanov 4cd9e27c09
Enable chronos dualstack with new constructor. (#681)
* Add constructor which will use latest chronos feature.

* Add ability to send to IPv4 addresses via dualstack socket.

* Fix decoding IPv4 mapped addresses.

* Use `*` instead of IPv6 address `::`.

* Removing debugging log statements.

* Address review comments.

* Remove debugging statements.

* Eliminate unnecessary variables.
2024-04-24 14:38:52 +03:00
Etan Kissling d66a29db7c
avoid `XDeclaredButNotUsed` in `kvstore_sqlite3` with empty tuple (#680)
Suppresses an annoying hint that gets triggered when `params is tuple`
but it's an empty tuple without fields.

```
Hint: 'i' is declared but not used [XDeclaredButNotUsed]
```
2024-02-28 22:03:04 +00:00
andri lim d8209f623f
Fix improper yield usage in rlpx and refine exception handling (#679)
* Fix improper yield usage in rlpx and refine exception handling

* Handle post hello step error
2024-02-19 14:16:33 +07:00
andri lim efe610e27f
Don't auto write p2pProtocol macro expansion to file (#678) 2024-02-16 14:13:03 +07:00
andri lim b9c40e1380
Use asyncraises in p2p (#675)
* Use asyncraises in p2p

* Fix capture error in async proc

* Remove gcsafe from async procs
2024-02-14 15:59:13 +07:00
andri lim bb1ab544fc
Reduce compiler warnings in rlp (#674) 2024-02-13 19:34:27 +07:00
Kim De Mey 2ad07a6010
Fix two bugs in Receipts RLP encoding/decoding (#672)
1. Fix Assertion error when receipt is not a List nor has a single
byte value. Receiving such garbage data would cause a crash.
2. Fix decoding of Receipt list by adding the missing Blob
encapsulation

Also added tests for these scenarios.
2024-02-05 23:22:15 +01:00
Kim De Mey 21f3014f54
Remove usage of stew/shim/net (#670)
* Remove usage of stew/shim/net and replace with std/net where needed

* Remove chronosStrictException compile flag
2024-01-24 09:09:11 +01:00
Kim De Mey 19965bab95
Fix chronos related deprecated warnings in uTP code part II (#669)
This also makes the uTP SendCallBack not returning a Future any
more as it is not used in sendData anyhow. And in case of uTP
over discv5, discv5 send call is already not async.

This gives quite a noticable throughput benchmark improvement over
with uTP over UDP, and a slightly noticable with uTP over discv5
2024-01-23 18:41:38 +01:00
Kim De Mey 496bcdab74
Fix chronos related deprecated warnings in discv5 and uTP (#667)
* Fix chronos related deprecated warnings in discv5

* Fix chronos related deprecated warnings in uTP code

* Improve discv5 closeWait order of events and add noCancel
2024-01-22 17:08:08 +01:00
andri lim b75ee36576
Refactor p2pProtocol internals (#657)
* Refactor p2pProtocol internals

* Attempt to fix rlp crash with Nim v2 (#658)

* Attempt to fix rlp crash with Nim v2
* Fix test_ecies for nim v2
* Reduce compiler warnings
* Resolve ambiquity in testutils

* Disable nim devel continue-on-error
2024-01-22 16:47:46 +07:00
Kim De Mey 2c6049b126
Fix a future .callback usage in rlpx (#666) 2024-01-19 12:29:11 +01:00
andri lim c57cebfd81
Fix discv4.send: use addCallback instead of .callback (#665) 2024-01-19 11:31:17 +01:00
Kim De Mey 159ae697c5
Remove last leftovers of ValidIpAddress (#664) 2024-01-19 11:15:57 +01:00
Kim De Mey 974a995b21
Add uTP over discv5 test and small uTP performance improvements (#663)
- Add a multiple sockets use test for uTP over discv5
- Use assign2 for the biggest consumer of genericAssignAux in uTP
- Avoid calling exists on the growable buffer when there is no
place in the socket window.
2024-01-18 12:07:03 +01:00
Etan Kissling a6f79a6000
Adjust test names and comments for `blobGasUsed` field (#661)
`blobGasUsed` was not mentioned in `test_rlp_codec` test names, despite
being used. Further, update the idea of `genOptionalFieldsValidation`
to also check for `blobGasUsed`, and fix the check to catch unsupported
combinations of optional fields.
2024-01-16 22:02:06 +01:00
Jacek Sieka e5e695c396
better async timeout wait (#659)
* don't call timeout code if future finishes before getting cancelled
* avoid extra raises effect resulting from `read` (vs `await`)
2024-01-16 18:12:47 +01:00
Kim De Mey cbcd1fd307
Fix for uTP issues with latest chronos (#655)
* Fix for uTP issues with latest chronos

* Fix for discv5 issue with latest chronos

* Increase amount of data send in uTP and uTP over discv5 tests
2024-01-12 17:35:44 +01:00
Kim De Mey a6942e3a30
Fix missing std/times import for the metrics 0.0.1 case (#656) 2024-01-12 17:35:14 +01:00
Kim De Mey 11049fb87d
Fix a bunch of compiler hints and warnings in uTP and discv5 (#652)
* Fix a bunch of compiler hints and warnings in uTP and discv5

* Leave in the cancel() for now
2024-01-12 09:49:06 +01:00
Jacek Sieka 0fc5dc03c5
support nim-metrics 0.1.0+ (#651) 2024-01-05 15:17:27 +01:00
Kim De Mey e5c2b1784e
Replace registerCustomScalarFunction with createCustomFunction (#649)
* Replace registerCustomScalarFunction with createCustomFunction

The nice thing about `registerCustomScalarFunction` and
`registerCustomScalarFunction` was that it kept the sqlite calls
of type sqlite3_value_xxx inside kvstore_sqlite.

The big downside however is that the "custom" call is awfully
specific due to the specific function signature it demands.
It is likely that for each different use case, a new version
needs to be added.

In this PR the code gets altered to `createCustomFunction` which
allows to just create a custom function with directly xFunc
signature of sqlite. The downside of this is that you still need
to import sqlite_abi in your local code to have access to each of
the sqlite3_value_xxx and sqlite3_result_xxx calls. These could of
course also be wrapped.

* Fix custom sql func test
2023-11-21 14:39:24 +01:00
tersec ca4898e24a
switch NAT module from ValidIpAddress to IpAddress (#647)
* switch NAT module from ValidIpAddress to IpAddress

* fix dcli build

* switch discv5 from ValidIpAddress to IpAddress

* fix fuzzers build

* mostly finish nim-eth to IpAddress transition
2023-11-10 09:28:21 +03:00
Kim De Mey 700360fde2
Rename routing_table_nodes metric for discv4 routing table (#646) 2023-10-31 18:28:21 +01:00
andri lim 4b00b230d0
Remove unused bootnodes and add Holesky bootnodes (#644)
* Remove unused bootnodes and add Holesky bootnodes

* Fix enode test
2023-10-25 11:07:55 +07:00
andri lim 1bae1d8426
Fix rlpHash of Transaction: remove EIP-4844 blobs before hashing (#643) 2023-10-22 14:37:49 +07:00
jangko e34a9c4e5d
Fix EthTime.now from func to proc due to sideeffects 2023-10-17 17:20:34 +07:00
jangko 2be8c183d1
Change EthTime from Time to distinct uint64
Cancun test vector demand a uint64 of block timestamp.
Therefore using std.Time will not able to pass test.
2023-10-17 16:00:57 +07:00
Jacek Sieka 34ab6bd986
rlp: avoid aliasing casts (#637)
* also get rid of unused concept
2023-09-25 11:30:47 +02:00
andri lim fe88d9e6b7
Change maxFeePerBlobGas from GasInt to UInt256 (#640)
* Change maxFeePerBlobGas from GasInt to UInt256

per latest spec:
https://eips.ethereum.org/EIPS/eip-4844#blob-transaction

* fix eip-4844 test
2023-09-24 14:33:24 +07:00
Jacek Sieka 02c8a1276b
avoid use of shallow (#636)
this probably has some tiny overhead but it shouldn't practically matter
2023-09-20 09:24:24 +02:00
jangko ac680ed79b
Add closeWait to EthereumNode 2023-09-08 21:45:41 +07:00
Kim De Mey 0d7e7448c4
Allow passing along the handshake ENR through talkresp handler (#634)
This allows for protocols build on top of discv5 to use the ENR
provided in the handshake directly, instead of having to rely
on requesting it from the discv5 routing table.
2023-08-30 17:44:05 +02:00
Jacek Sieka 22df101fcf
keyfile: reduce imports (#633) 2023-08-21 11:11:52 +02:00
Etan Kissling 894ec07d9f
fix `{.raises.}` annotation to `writeValue` (#631)
`writeValue` doesn't raise `SerializationError`, so the `{.push.}`
is not optimal. Move `{.raises.}` to each `proc`, same as other modules.
2023-08-19 19:03:10 +02:00
jangko 074edff1b4
EIP-4788: Add parentBeaconBlockRoot field to BlockHeader 2023-08-18 17:52:55 +07:00
Etan Kissling 946ffe0a4a
arc/orc lacks `shallowCopy` --> use `move` (#630)
`shallowCopy` is not available in `--mm:arc/orc`, but our usage can be
replaced with `move`.
2023-08-15 14:48:41 +02:00
andri lim 2ed8e991b5
add mixin env to sqlite3 generic procs (#627) 2023-08-03 22:00:47 +07:00
tersec d217d309eb
revert some func back to proc; unused by nim-eth tests (#629) 2023-08-02 23:00:49 +00:00
tersec 01ae82fd70
specify exceptions of callback called by exception-free function (#628) 2023-08-01 06:05:22 +00:00
jangko 521db96a65
rename data gas to blob gas 2023-07-28 10:19:00 +07:00
Jacek Sieka 15a09fab73
sqlite3: better error message (#622)
* sqlite3: better error message

* fix dispose
2023-07-06 06:53:29 +00:00
Kim De Mey 26ae539598
Use results Opt for discv5 getNode and resolve (#625) 2023-06-26 11:30:33 +02:00
jangko 6b8a7b009e
implement EIP-4844: Shard Blobs Transactions 2023-06-23 13:48:51 +07:00
Kim De Mey c5dd5e4115
Enable tests with also json sink and fix failures (#624)
And a few minor cleanups
2023-06-22 16:13:16 +00:00
Kim De Mey d74dc40bee
Style fixes and comment improvements on uTP code (#623) 2023-06-22 11:31:30 +00:00
jangko 6dacb2ca5c
fixes related to stint breaking changes 2023-06-21 17:55:00 +07:00
Kim De Mey c3860244b1
Fix potentential uTP socket leak for socket in SynSent state (#620)
Socket leak could because ST_DATA (and ST_FIN) packets for sockets
not in connected stated were only dropped after appling acks.
2023-06-21 12:44:52 +02:00
Kim De Mey cea23299be
Avoid double socket destroy on timed-out uTP SYN-ACK (#619) 2023-06-15 12:26:22 +02:00
tersec c608426d03
add/change Deneb fields per https://github.com/ethereum/execution-apis/pull/417 (#616) 2023-06-03 18:47:55 +00:00
Jacek Sieka 55b9da0bea
fix cancellation syntax (#615)
* fix cancellation syntax

* simplify
2023-06-02 14:33:06 +02:00
andri lim 91b2b9d2ed
rlp codec support optional fields (#613)
Instead of patching BlockHeader or BlockBody codec
each time it get additional optional fields,
this PR make the rlp codec automatically handle
optional fields. Thus rlp codec overloading of
EthBlock, BlockHeader, and BlockBody can be removed.
2023-05-30 16:02:02 +03:00
jangko 67bbd88616
Replace exception with assertion in rlp.append of BlockHeader
Using exception in rlp.append turn out creates more problem than it solve.
`blockHash` operation is literally everywhere in nimbus-eth1, nimbus-eth2, and fluffy-code.
The compiler begins to complain about unlisted exception.
Rlp bytes coming from network and test vectors already verified by the decoder.
So, any invalid optional fields of blockheader that come into rlp.append means
programming error. Assertion will be more apropiate in this situation and less problematic.
2023-05-25 15:27:37 +07:00
jangko 7ff6de2367
fix rlp.readRecordType of BlockBody and friends
including:
- fix rlp.append of BlockHeader
- add tests for rlp.readRecordType of BlockBody
- add tests for rlp.append of BlockHeader
- add tests for EthBlock roundtrip
2023-05-23 14:37:47 +07:00
jangko e639dc1e14
fix missing withdrawals field in EthBlock 2023-05-22 16:11:30 +07:00
Ivan Folgueira Bande 285da12bf3
nat.nim: breaking dependency with 'confutils' (#609)
* nat.nim: breaking dependency with 'confutils'

The main purpose of this change is to break the dependency with
`confutils`.
2023-05-16 13:43:31 +02:00
Jacek Sieka 6e5ee490c2
avoid ptr_arith (#608) 2023-05-11 07:39:21 +00:00
Kim De Mey d57a1094b1
Remove all Defect raises to avoid unnecessary warnings (#607)
As this module dropped support for Nim versions < 1.6, all Defect
raises can be removed to avoid unnecessary warnings
2023-05-10 15:50:04 +02:00
tersec 7564b21ade
support only Nim 1.6+ (#606) 2023-05-09 20:19:44 +00:00
jangko 607539d821
upgrade github actions to v3 2023-05-05 17:25:15 +07:00
Adam Spitz 4b818e8307
In the incomplete-db node-existence check, don't use contains. (#603)
* In the incomplete-db node-existence check, don't use contains.

(Using contains led to a problem with CaptureDB.)

* In the incomplete-db check, just checking len > 0 isn't right.

* Oh, I needed the AssertionDefect thing too.

* Need this when compiling under older versions of Nim.

* Sometimes we want missing nodes to be errors, sometimes not.
2023-04-28 12:04:33 -04:00
Adam Spitz f5dd26eac0
Some changes to make hexary.nim better able to handle incomplete DBs (#602)
* Added maybeGet, for working with incomplete DBs.

* Made trie.del throw an exception if it encounters a missing node.
2023-04-21 07:59:15 -04:00
jangko 917888356e
no more wild constants and rename ecdhSecret to ecdhSharedSecret 2023-04-21 16:08:57 +07:00
jangko 9bf2e1786b
remove ecdhRaw usage and replace it with ecdh+custom hash function 2023-04-20 15:45:23 +07:00
Jordan Hrycaj 9a1d35f803
Fix block body encoding (#598)
why:
  List wrapper was missing in `append()` mixin.
2023-04-19 13:26:41 +01:00
Kim De Mey 25b0da0280
Fix JSON writer for logging in devp2p code (#597) 2023-04-18 13:51:02 +02:00
Kim De Mey ea3c164a00
uTP clarification on SYN-ACK seqnr and some cleanup (#596)
- Clarify why on SYN-ACK seqNr gets -1
- Cleanup comments
- Style fixes
- Reuse WriteResult
2023-04-04 17:11:36 +02:00
Adam Spitz 4754543605
Added maybeGet, for working with incomplete DBs. (#595) 2023-04-03 13:01:25 -04:00
Adam Spitz 9e89f0dccc
Made rlp.readRecordType(BlockBody, false) include withdrawals. (#594) 2023-03-17 10:41:38 -04:00
Kim De Mey d2ba753792
Add rlpx metrics and avoid immediate peer reconnections (#585)
* Add metrics related to devp2p peer connections

* Avoid reconnecting to peers that just failed connection

- Add SeenTable to avoid reconnecting to peers immediately after
a failed connect. Depending on the failure, the amount of time is
different. This is similar to what is done in nimbus-eth2.
- Attempt to rework rlpxConnect at the same time, in order to
make sure that errors are properly handled. The current structure
is far from ideal, but it is hopefully a small step in the right
direction. To many oddities in there right now to really rework
rlpxConnect properply.

* Fix rlpx thunk fuzzer
2023-03-16 16:45:12 +01:00
Kim De Mey 29b14749fa
Improve uTP decoded packet logs + style and comment clean-up (#593)
* Improve uTP decoded packet logs + style and comment clean-up

* Don't test for the exact error strings in uTP decode + clean-up
2023-03-14 16:17:39 +00:00
Jacek Sieka 72c9858927
kvstore: remove unnecessary EH (#592) 2023-02-22 09:03:13 +00:00
Jacek Sieka ca1c205441
fix missing gcsafe (#591) 2023-02-22 08:11:32 +00:00
Jacek Sieka ade6d19691
Defect fixes in kvstore (#589)
* Defect fixes in kvstore

* compile fixes
2023-02-21 18:54:30 +01:00
Kim De Mey d2a3727c4c
Use chronos isGlobal to verify public IPs for net/nat and discv5 (#588) 2023-02-21 09:34:26 +01:00
Kim De Mey 4d55bfc9e8
Move dcli tool to a tools folder (#584) 2023-01-27 18:40:40 +01:00
Adam Spitz 20d2d318f0
Fixed a bug that I just introduced in the BlockBody RLP code. (#583)
The nimbus-eth1 tests seem to think that this is the right thing to do.
2023-01-21 14:41:30 +01:00
Adam Spitz 5b189ce5e2
EIP-4895: added withdrawals to BlockBody. (#581) 2023-01-20 12:18:39 +01:00
Adam Spitz 71b148a5b0
Propagating the del-returns-a-bool change through the trie DB code. (#578)
See here:
8f0ae55353
2023-01-17 16:39:28 -05:00
tersec 68f4c53828
withdrawals amount in uint64 gwei: https://github.com/ethereum/execution-apis/pull/354 (#577) 2023-01-14 14:39:23 +00:00
zah 0807378e7f
Migrate some functions from nimbus-eth1 in order to make them available in nimbus-eth2 (#576) 2023-01-10 18:56:27 +02:00
Adam Spitz 58284ffeda
Attempt to connect to many peers concurrently, not in sequence. (#575)
The old code, which did it in sequence, made it take a very long
time to accumulate peers in the peer pool, particularly because
occasionally a connection-attempt would hang for a couple of minutes
before timing out.
2023-01-04 05:56:23 -05:00