Commit Graph

99 Commits

Author SHA1 Message Date
bhartnett 6e7e63adf5
Fluffy: Make the Kademlia concurrency factor configurable via a debug parameter (#2868) 2024-11-25 20:16:27 +08:00
Kim De Mey b671499fdc
fluffy: Make concurrent offers configurable at cli (#2854) 2024-11-21 04:05:33 +07:00
Kim De Mey bef4c05561
Add custom json-rpc method to initialize the trusted block root (#2805) 2024-10-30 18:32:36 +01:00
bhartnett ca24dd915f
Fluffy: Make content request retries configurable (#2770)
* Make content request retries configurable and add to state network and history network.

* Set retries to 1. Use uint to prevent negative values. Make contentRequestRetries a debug parameter.
2024-10-25 15:15:01 +08:00
Jacek Sieka 5fc4c13ab1
Increase JSON-RPC limits, bump json-rpc (#2759)
* blocks can be bigger than the default 1mb when json-rpc-encoded - this
happens on sepolia for example
* json-rpc bump improves debug logging and fixes a number of bugs
* json-serialization bump fixes a crash on invalid arrays in json data

At some point, it would probably be better to compute the maximum block
size from actual block constraints, though this is somewhat tricky and
depends on gas limits etc. Until then, 16mb should be plenty.

With this, sepolia can be synced :)
2024-10-23 10:26:56 +02:00
bhartnett 54528fb24b
Fluffy: Enable content cache for state network (#2739)
* Enable content cache for state network.

* Update state json-rpc endpoints to return local content which uses cache if enabled.

* Add content cache metrics.

* Make content cache configurable.

* Add content cache tests.
2024-10-16 21:05:39 +08:00
bhartnett 344a236c31
Fluffy: Create separate portal rpc handlers for each sub-network (#2705)
* Create separate portal rpc handlers for each sub-network.

* Remove random gossip from history and state networks.

* Move shared portal rpc endpoints into common handler.

* Use enum instead of network string.
2024-10-07 16:33:02 +02:00
Kim De Mey dbe3393f5c
Fix eth/common & web3 related deprecation warnings for fluffy (#2698)
* Fix eth/common & web3 related deprecation warnings for fluffy

This commit uses the new types in the new eth/common/ structure
to remove deprecation warnings.

It is however more than just a mass replace as also all places
where eth/common or eth/common/eth_types or eth/common/eth_types_rlp
got imported have been revised and adjusted to a better per submodule
based import.

There are still a bunch of toMDigest deprecation warnings but that
convertor is not needed for fluffy code anymore so in theory it
should not be used (bug?). It seems to still get imported via export
leaks ffrom imported nimbus code I think.

* Address review comments

* Remove two more unused eth/common imports
2024-10-04 23:21:26 +02:00
Bhaskar Metiya b0a4a9ef84
Fix multiple instances running from same dataDir (#2647)
* Fix multiple instances running from same dataDir

* Add exclusive lock on lock file

* Unlock lock file on process exit

* Fix minor issues in lock file implementation
2024-09-26 10:09:34 +02:00
web3-developer 69d58e8215
Fluffy: Configure RPC APIs via CLI parameter (#2657)
* Support RPC API namespaces as cli parameter.

* Fluffy now uses rpcFlags on startup.

* Update testnet script to enable all RPC APIs.

* Update Fluffy book and move web3 call into eth calls.
2024-09-25 22:44:46 +08:00
web3-developer cb69723ff3
Fluffy: Make number of nodes to gossip content to configurable (#2653) 2024-09-25 15:15:20 +08:00
web3-developer b1dc1578f0
Move web3 rpc handler into eth handler. Create debug rpc handler and move existing debug endpoints. (#2655) 2024-09-25 11:28:45 +08:00
web3-developer 07193968ca
Implement graceful shutdown in Fluffy (#2645)
* Make stop functions wait for completion before return.

* Implement graceful shutdown.

* Shutdown rpc and metric servers if enabled.

* Move metrics and rpc servers out of PortalNode.
2024-09-20 20:54:36 +08:00
web3-developer ea74e035ab
Add WebSocket support to Fluffy. (#2636)
* Add WebSocket support to Fluffy.

* Support websocket compression.

* Create setupRpcServer closure to remove code duplication.
2024-09-18 21:27:24 +08:00
web3-developer 0ee8e61a3a
Remove RpcProxy from Fluffy and use RpcHttpServer instead. (#2634)
* Remove RpcProxy from Fluffy and use RpcHttpServer instead.

* Cleanup test.
2024-09-18 15:46:50 +08:00
Kim De Mey 0a80a3bb25
Reverse calculate the radius at node restart (#2593)
This avoid restarting the node always with a full radius, which
causes the node the be bombarded with offers which it later has
to delete anyhow.

In order to implement this functionality, several changes were
made as the radius needed to move from the Portal wire protocol
current location to the contentDB and beaconDB, which is
conceptually more correct anyhow.

So radius is now part of the database objects and a handler is
used in the portal wire protocol to access its value.
2024-09-05 18:31:55 +02:00
Kim De Mey c2d9c0bc45
Remove deprecated cli options for fluffy (#2517) 2024-07-23 15:40:28 +02:00
Kim De Mey 6cf57d9912
Remove duplicate logging code from fluffy (#2488)
Code was copied back when nimbus-eth2 was not a dependency.
Now that is an established dependency for many parts, lets reuse
the original code.
2024-07-15 11:41:17 +02:00
Kim De Mey 54e3fd1a94
Move Portal wire and networks setup to new portal_node module (#2464) 2024-07-09 19:22:25 +02:00
Kim De Mey ae094692fb
Bump nim-eth for ENR changes and related clean-up (#2422) 2024-06-27 17:59:08 +02:00
Kim De Mey be02242ccd
Fix usage of deprecated startMetricsHttpServer call (#2032) 2024-06-25 12:06:39 +02:00
web3-developer bd2ca07da6
Update Fluffy data directory location based on selected network. (#2400) 2024-06-20 10:48:45 +08:00
Kim De Mey 4fd2ecddec
Bump nim-eth/web3/kzg4844/nimbus-eth2 and related fixes (#2392)
Bump nim-eth, which requires nimbus-eth2 bump, which requires
bumps of web3 and kzg4844 + related fixes to all those bumps.
2024-06-19 08:57:45 +07:00
web3-developer e3d14bd921
Fluffy portal testnet support (#2383)
* Bump portal-mainnet repo.

* Update command line arguments and parsing on startup.

* Read in angelfood bootstrap nodes and update Fluffy guide.

* Configure subnetwork protocol ids.
2024-06-18 15:32:57 +08:00
web3-developer 1377f93d50
Implement Fluffy JSON-RPC endpoints. (#2364)
* Implement eth_getBalance, eth_getTransactionCount, eth_getStorageAt and eth_getCode JSON-RPCs.

* Fixes.
2024-06-15 01:03:18 +08:00
web3-developer 9c26fa3298
Updates to Fluffy to support hive tests. (#2333)
* Updates to Fluffy book for hive tests.

* Add support to disable state root checks for state content from the command line.

* Update portal_stateStore endpoint to support decoding offer and storing retrieval value.
2024-06-11 21:01:35 +08:00
Jacek Sieka 919242c98e
results: use canonical import (#2248) 2024-05-30 14:54:03 +02:00
Kim De Mey ba2dd76f30
Rename the Portal testnet0 to mainnet (#2204)
Also adds the new testnet as option, but this needs to be still
implemented by adding testnet bootstrap nodes and protocol ids.
2024-05-22 17:10:31 +02:00
web3-developer 2891b9aa7d
Fluffy state offer validation (#2170)
* Implemented offer validation interface.

* Implement high level offer validation steps.

* Completed state validation tests.

* Update validation to use result type.

* Update state proof verification tests to test offer verification.

* Query history network to get state root by block hash.

* Fix state network test and remove usage of CoreDb.

* Fix state network gossip test and PR comment updates.

* Add trieproof state validation tests and fix for short nodes.
2024-05-16 16:16:16 +08:00
Kim De Mey 6c2c99a9ae
Add reclaim to the fcli_db prune cmd (#2122) 2024-04-02 20:50:05 +02:00
Kim De Mey 49a199dbc3
Add beacon state historical_roots to fluffy (#2085)
- Add historical_roots in the binary for proof verification
pre-capella
- Add command to export historical_roots to file in
eth_data_exporter tool
2024-03-19 16:45:32 +01:00
Kim De Mey 461c7e7d85
Add Era1 backfill to portal_bridge history mode (#2077) 2024-03-15 10:41:41 +01:00
Kim De Mey 0debf1a122
Introduce --networks and --portal-network cli options (#2061)
The --networks option is to select the networks alias the Portal
sub-protocols.
The --network option is deprecated and replaced with the
--portal-network option to avoid confusion with --networks option.

This commit also remove the --state flag which was a temporary
quick-fix.
2024-03-05 13:45:48 +01:00
Kim De Mey 2d76b8c010
Formatting fluffy with nph v0.5.1 + CI check (#2020)
* Add nph check to fluffy CI lint

* Add a section on nph usage in the fluffy.guide

* Update copyright years for altered files

* Avoid chained methods formatting style in db code

* Update nph in CI to v0.5

* Remove leftover commented import

* Move comment to avoid nph turning complex list into simple list (nph bug)

* Update nph in CI to v0.5.1

* Formatting fluffy with nph v0.5.1
2024-02-28 18:31:45 +01:00
Daniel Sobol 2766e20179
Add flag to explicitly enable state network (#2031) 2024-02-17 01:38:58 +03:00
Kim De Mey 3199857ed5
Further fluffy code clean-up from warnings/hints (#1987) 2024-01-25 11:04:09 +01:00
Kim De Mey aad4adde07
Move all Fluffy database code to database folder (#1918) 2023-12-01 17:20:52 +01:00
Kim De Mey c46706ac75
Add force pruning for the storage-capacity option (#1901) 2023-11-23 11:44:57 +01:00
Kim De Mey a7bb52e5b5
Add force prune for statically set radius (#1896) 2023-11-21 16:16:15 +01:00
Kim De Mey c41d531c51
Adjust storage capacity option (#1891)
- Rename storage-size to storage capacity
- Change type to uint64 to avoid potential wrap around
- Change to MB instead of KB and set default to 2GB
2023-11-10 17:16:15 +01:00
Kim De Mey 436ef0e40e
Portal BeaconLightClientNetwork -> BeaconNetwork renaming (#1832) 2023-10-20 12:06:25 +02:00
Kim De Mey 0472b75e23
Rework fluffy beacon lc db to allow storing processed content (#1821) 2023-10-18 16:59:44 +02:00
Kim De Mey effe9c7ad8
Implement safe and finalized tag for eth_getBlockByNumber (#1814)
Only available when running the beacon LC and the portal beacon
network.
2023-10-09 17:49:15 +02:00
Kim De Mey f433297631
Improvements to portal beacon network (#1806)
- Adjust beacon lc db to avoid requirement of pruning optimistic
and finality updates
- Enable Portal config for beacon chain network to apply IP limits
configs on routing table
2023-10-05 08:57:45 +02:00
Kim De Mey 2f2c5127ea
Further implementation to working prototype for Portal beacon LC (#1781)
- Link network gossip validation to LC processor validation
- QuickFix put/get for optimistic and finality updates
- Minor fixes and clean-up
- Improve bridge gossip for new LC Updates
- Adjust local testnet script to be able to locally test this
- Adjust test and skip broken test
2023-09-28 18:16:41 +02:00
Kim De Mey dd074d4df8
Load and use the stored ENR to allow incr of seqNum on updates (#1763) 2023-09-25 21:08:10 +02:00
Kim De Mey 53df21f000
Add clientInfo to ENR for easier testnet debugging (#1513)
* Add clientInfo to ENR for easier testnet debugging

* Remove git rev from ENR client info

This allows for a shorter ENR which allows more ENRs to be packed
in the Nodes reply. And for glados, only the client name is used
anyhow.
2023-09-20 12:19:40 +02:00
andri lim 56215ed83f
Bump stint to v2.0: new array backend (#1747)
* Bump stint to v2.0: new array backend
2023-09-13 09:32:38 +07:00
Kim De Mey c04f2477ac
Add version to fluffy cli flags and metrics (#1736)
Also update Grafana dashboard to incorperate the metrics and do
a confutils module bump.
2023-09-07 16:27:13 +02:00
Daniel Sobol 85134eb24b
add hidden --disable-poke option into fluffy [#1640] (#1711) 2023-08-30 10:01:00 +02:00