597 Commits

Author SHA1 Message Date
kdeme
242b9e033d
Remove SSZ Union usage in BlockHeaderWithProof type
Remove SSZ Union in BlockHeaderWithProof type by making the proof
an SSZ encoded ByteList. The right type for the proof can be
selected at the decoding step by first looking into the header
for the timestamp and selecting the right type based on the
hardfork the block is in.
2025-02-07 12:18:41 +01:00
Jacek Sieka
8690a03af7
Fix poor eth_getLogs performance (fixes #3033) (#3040)
* Fix poor eth_getLogs performance (fixes #3033)

* don't recompute txhash in inner log loop (!)
* filter logs before computing hashes

* copyright
2025-01-30 19:38:24 +00:00
kdeme
cdd71cac2e
Portal: Remove pruneDeprecatedAccumulatorRecords (#3031)
This was added to cleanup database from a content type that was
removed.

After ping extensions upgrade all active nodes must have been
updated to remain compatible and thus must have ran this pruning
code.
2025-01-29 10:20:29 +01:00
bhartnett
f78f82e8c9
Fluffy: Randomize gossip in neighborhoodGossip (#3029) 2025-01-29 08:49:37 +08:00
kdeme
28f9b792d9
Minimal implementation of Portal ping payload extensions spec (#3010)
* Minimal implementation of Portal ping payload extensions spec

* Remove serialization layer for CustomPayload (spec change)
2025-01-27 19:26:23 +01:00
tersec
49523c5d8f
rm unused pre-1559 gas limit update calculation (#3011)
* rm unused pre-1559 gas limit update calculation

* wiki links 404
2025-01-25 17:36:43 +07:00
kdeme
b917be54ab
Fix wrong Result error usage in eth_data_exporter (#3018) 2025-01-24 08:36:28 +01:00
kdeme
9a5fd34dac
Portal history: Add ephemeral content type (#3017)
This does not yet enable the offer/accept and storage of the
ephemeral headers as it requires database changes.
Follow-up PR for that functionality.
2025-01-23 21:45:51 +01:00
kdeme
4b893eb88e
Discard received data on uTP content stream timeout (#3014)
Discard the received data on uTP content stream read timeout.
Before the data was still added to the queue and being processed
and should normally fail in validation. However as we know not all
data got read it should not even move to the validation step.

Added however a FIN send after the timeout instead of the delayed
socket clean-up which does not make much sense in that scenario
either. Basically either be nice and still send a FIN or just
destroy the socket immediatly.
2025-01-21 17:44:02 +01:00
kdeme
234487a8cb
Portal JSON-RPC: add eth_blockNumber and fix eth_getBlockByNumber (#3013)
- add eth_blockNumber
- fix eth_getBlockByNumber: latest vs safe fix.
2025-01-21 09:44:13 +01:00
bhartnett
285b6de4bc
Fluffy: Cleanup unused code from PortalProtocol (#3006) 2025-01-20 20:57:18 +08:00
kdeme
fa147f90ed
Portal JSON-RPC: minor clean-up (#3000)
Minor cleanup/refactor + add a basic TraceObject when returning
locally found content.
2025-01-17 11:57:15 +01:00
bhartnett
19ea82bf50
Fluffy: State bridge clean shutdown (#2996)
* Implement clean shutdown for state bridge.

* Handle shutdown in history and beacon bridges.
2025-01-16 08:48:10 +08:00
bhartnett
6e83a48969
Fluffy: Refactor and organize state bridge code (#2992) 2025-01-13 16:49:35 +08:00
bhartnett
58d447564a
Fluffy: Improve testnet script parameters (#2989) 2025-01-10 08:48:49 +08:00
bhartnett
ec41a8d6fe
Fluffy: Add additional cli parameters for configuring the WebSocket server (#2988)
* Add additional rpc parameters for configuring the WebSocket server.
2025-01-10 08:37:44 +08:00
bhartnett
a5194846af
Fluffy: Support gossiping content to multiple portal client endpoints in the state bridge (#2987) 2025-01-09 11:42:16 +08:00
bhartnett
13a58abf5c
Fluffy: Improve state bridge gossip retry mechanism and logging (#2986)
- Minor refactor and cleanup of gossip retry and logging.

- Wait time before verifying the gossip for a block is now proportional to the number of offers per block.

- Don't retry gossipping content after finding it in the network. When retrying gossip of a block, only the offers not yet found in the network will be re-sent.
2025-01-08 15:19:37 +08:00
bhartnett
e33ab7d468
Fluffy: Minor refactor, cleanup and improve variable names in JSON-RPC handlers (#2981) 2025-01-03 15:24:12 +08:00
bhartnett
7c347e1a2a
Fluffy: Update portal_*PutContent JSON-RPC endpoints to return metadata (#2980)
* Update portal_*PutContent JSON-RPC endpoints to return metadata as defined in the portal specs.
2025-01-02 16:28:12 +08:00
bhartnett
c714fa8735
Fluffy: Rename portal_*Gossip JSON-RPC endpoints to be in line with updated portal spec (#2970)
* Revert previous change in PortalStream. Allow zero as a valid connectionId if randomly generated.

* Rename portal_*Gossip JSON-RPC endpoints to portal_*PutContent to be in line with updated portal spec.
2024-12-23 15:09:48 +08:00
Kim De Mey
7112a19d6c
Adjust some portal wire and history network (error) logging (#2965) 2024-12-21 20:22:16 +07:00
Jacek Sieka
7bbb0f4421
Stream blocks during import (#2937)
When running the import, currently blocks are loaded in batches into a
`seq` then passed to the importer as such.

In reality, blocks are still processed one by one, so the batching does
not offer any performance advantage. It does however require that the
client wastes memory, up to several GB, on the block sequence while
they're waiting to be processed.

This PR introduces a persister that accepts these potentially large
blocks one by one and at the same time removes a number of redundant /
unnecessary copies, assignments and resets that were slowing down the
import process in general.
2024-12-18 13:21:20 +01:00
andri lim
b8932d9519
Remove MergeForkBlock alias and use MergeNetSplitBlock only (#2947) 2024-12-17 11:42:13 +00:00
bhartnett
c32726671f
Fluffy State Bridge: Support running without an EL for block ranges where we have the state diffs in the database (#2920) 2024-12-09 21:57:41 +08:00
bhartnett
17da64628a
Fluffy: Fix broken portal hive tests (#2917)
Fix bug in portal stream where connection id was not correctly generated when handling requests from peers.
2024-12-06 23:53:15 +08:00
Kim De Mey
56caa5f62f
Add radius sort in trace lookup and logging improvements (#2905)
The radius sort performance improvement in content lookups was
not implemented in the trace version.

Also cleanup some part of the logging related to uTP connection
setup.
2024-12-04 13:59:08 +01:00
bhartnett
359eb6d974
Fluffy: Portal stream improvements and pending transfers prune fix (#2900)
* Fix defect in for loop when deleting element.

* Prune offers and requests before accepting.

* Store content requests and offers by connectionId.
2024-12-04 08:34:13 +08:00
Kim De Mey
3bf0920a16
Remove Portal beacon-lc-bridge (#2897)
The idea of the beacon-lc-bridge was to allow to bridge data into
the Portal network while only using p2p protocols to get access
to the data.

It is however incomplete as for history content the receipts are
missing. These could be added by also adding devp2p access.
But for the beacon content, there would be no way for getting the
historical summaries over p2p.

And then we did not even look yet on how to do this for state.

Considering it is incomplete it was also not being used by anyone
and thus we remove it.
2024-12-02 17:30:17 +01:00
Kim De Mey
0f18de61dc
Revert commit 6142183 and partial of b446d2a (#2898)
There is an assertion hitting due to the additon of an iterator
that deletes items from the sequence while iteratting over it.
Before the keepIf helper was used that has different code for
doing this similar work.
2024-12-02 14:09:58 +01:00
bhartnett
b446d2a2e8
Fluffy: Cleanup state bridge preimages backend and add logging for transfer limit reached (#2896) 2024-12-02 16:09:43 +08:00
bhartnett
6142183d2c
Fluffy: Limit concurrent offers that can be received from each peer (#2885)
* Limit offer transfers per peer.

* Remove pending transfers in prune.

* Limit content lookups.

* Improve performance of canAddPendingTransfer and addPendingTransfer.
2024-11-29 11:31:46 +08:00
Kim De Mey
c0199e8944
Fix startedAtMs in traceContentLookup to use system clock (#2891)
startedAtMs is the time passed since UNIX epoch in milliseconds.
Cannot use a monotime clock for that. As we want to keep mono
for other Moment.now() usage, imported epochTime from std/times
instead.
2024-11-28 19:05:15 +01:00
bhartnett
e74d5c3f22
Fluffy: Reduce info logs when looking up state. (#2890) 2024-11-28 22:00:03 +08:00
bhartnett
23a43d1d15
Fluffy: Implement poke in state network (#2750)
* Implement poke in state network.
2024-11-28 20:33:57 +08:00
andri lim
5e90522e70
Bump nim-web3 to c8f36f59cb354196cfe117b6866e81d450c8cfd7 (#2878)
* Bump nim-web3 to c8f36f59cb354196cfe117b6866e81d450c8cfd7

* Fix portal bridge

* Comply to nph format style
2024-11-27 20:16:31 +07:00
bhartnett
6e7e63adf5
Fluffy: Make the Kademlia concurrency factor configurable via a debug parameter (#2868) 2024-11-25 20:16:27 +08:00
andri lim
fbfc1611d7
Implement EIP-7702: Set EOA account code (#2631)
* Implement EIP-7702 part 1: Behavior

* Implement EIP-7702 part 2: Tx validation

* Implement EIP-7702 part 3: Delegation Designation and Gas Costs
2024-11-25 11:28:03 +01:00
bhartnett
78c5770b2f
Fluffy State Bridge: Support skipping gossip when content is found in the network (#2867) 2024-11-25 11:40:09 +08:00
Kim De Mey
453cb2f33e
portal_bridge: Add concurrency to the history content gossip (#2855) 2024-11-21 15:30:42 +01:00
Kim De Mey
107db3ae16
fluffy: revert iplimit/bitperhop debug flags (#2860)
Revert for now to their previous names to avoid fluffy fleet
breakage.
2024-11-21 13:36:18 +01:00
bhartnett
4b63cdeaac
Remove direct local content lookup from PortalRpcClient. (#2857) 2024-11-21 20:20:09 +08:00
bhartnett
508ce79cdd
Fluffy: Log nodeId when receiving offers (#2856)
* Log nodeId when receiving offers.
2024-11-21 20:17:38 +08:00
Kim De Mey
b671499fdc
fluffy: Make concurrent offers configurable at cli (#2854) 2024-11-21 04:05:33 +07:00
Kim De Mey
3ea5c531d1
portal_bridge history: fully support gossip of block by number (#2853)
+ some minor cleanup & reuse of code
2024-11-20 14:07:36 +01:00
Kim De Mey
d496793fa6
fluffy/portal: Add missing imports (#2851)
Would/could fail compilation when using portal_node as module,
depending on import order.
2024-11-19 15:38:10 +01:00
Kim De Mey
32b5b8a173
Fluffy guide: fix formatting for development status docs (#2850) 2024-11-18 10:30:23 +01:00
Kim De Mey
d643556d4b
fluffy.guide: Update development status (#2846) 2024-11-07 17:17:40 +01:00
bhartnett
9dceb58ad0
Fluffy: Prioritize nodes that have radius in range of target content in lookup (#2841)
* Prioritize nodes that have radius in range of target content in content lookup.
2024-11-07 16:59:59 +08:00
Kim De Mey
eaf98dccb1
Change some spammy info/error logs to debug (#2840)
These logs are not really usful to an end user, and are rather
for developers, so we move them to debug.
2024-11-06 15:44:20 +01:00