Commit Graph

4273 Commits

Author SHA1 Message Date
Kim De Mey 9c9a41a16d
More verbose error handling for Portal stream content reading (#2737) 2024-10-14 11:53:28 +02:00
bhartnett bdbea9853e
Rename Fluffy rpc methods to match updated spec (#2735) 2024-10-14 16:46:35 +08:00
Kim De Mey 3e36f52fd7
Rename the debug docker files + add a simpler/faster one for Linux (#2731)
Also update related documentation
2024-10-13 18:07:17 +02:00
tersec eec41bc0e8
test with aarch64 macOS (#2729) 2024-10-13 08:20:05 +07:00
Advaita Saha 1858d55748
shift kurtosis + assertor to stable release (#2733) 2024-10-13 08:19:33 +07:00
bhartnett 9db31ac17b
Fluffy: Fix state validation issue causing portal hive test failure (#2732) 2024-10-13 00:51:50 +08:00
Kim De Mey 6e114dc950
Fix JSON encoding for NodeId, no leading zeroes dropped (#2730)
Portal JSON-RPC API specifications dictate that the NodeId must be
the 32 byte identifier. We had leading zeroes being dropped in
our implementation.
2024-10-11 16:43:06 +02:00
Kim De Mey 0ebab78136
Update Grafana dashboard for Portal (#2727) 2024-10-10 20:11:04 +02:00
Kim De Mey a5f0b12bd1
Remove old portal_debug API for seeding data into the network (#2726)
This was the first API created for this, it has been superseded
by the API that makes use of era1 files and/or the portal_bridge.

Only usage was still in test_portal_testnet which has now been
altered to make use if API calls from Portal specification.
2024-10-10 16:42:57 +02:00
andri lim 59dde39d95
Run t8n test in CI (#2724)
* Enable t8n test to run in CI

* Disable trace test for evmc
2024-10-10 20:25:27 +07:00
tersec 266b72698d
rm obsolete MergeTracker (#2725) 2024-10-10 20:25:12 +07:00
bhartnett 4ae87e6d19
Fluffy: Add validation and local storage of content in remaining state portal rpc methods (#2723)
* Add validation functions to be used in state portal rpc.

* Add validation to remaining state portal rpc methods.

* Lookup local content in recursiveFindContent rpc methods.

* portal_stateFindContent and portal_stateOffer no longer store in db.
2024-10-10 21:24:39 +08:00
tersec 69f646f417
bump nim-eth1 for updated Sepolia and Holesky bootnodes (#2722) 2024-10-09 22:45:33 +00:00
Jordan Hrycaj f937f57838
Beacon sync targets cons head rather than finalised block (#2721)
* Fix fringe condition clarifying how to handle an empty range

why:
  The `interval_set` module would treat an undefined interval construct
  `[2,1]` as`[2,2]` (the right bound being `max(2,1)`.)

* Use the `consensus head` rather than the `finalised` block as sync target

why:
  The former is ahead of the `finalised` block.

* In ctx descriptor rename `final` field to `target`

* Update docu, rename `F` -> `T`
2024-10-09 18:00:00 +00:00
bhartnett b13f06fcfb
Fluffy state portal rpc validation (#2719)
* Validate content key in portal_stateLocalContent.

* Add additional validation to stateStore rpc method.
2024-10-09 20:23:46 +08:00
Kim De Mey 5edb0b320f
Allow Portal beacon LC sync to start from a stored LC bootstrap (#2715)
Portal beacon LC sync can be started now from a provided trusted
block root or, in case it has been running before, from a
previously verified and stored LC bootstrap.

This required altering the the beacon db on how the bootstraps
are stored.
2024-10-09 10:21:00 +02:00
Jacek Sieka 11646ad3c4
Ordered trie (#2712)
Speed up trie computations and remove redundant ways of performing this
operation.

Co-authored-by: jangko <jangko128@gmail.com>
2024-10-09 09:44:15 +02:00
andri lim e59d2825f4
Bump nim-stew (#2718) 2024-10-09 11:47:27 +07:00
andri lim 52e3e20643
Bump nim-unittest2 (#2717) 2024-10-09 11:47:15 +07:00
andri lim fe4527617f
Bump nim-eth (#2716) 2024-10-09 11:47:02 +07:00
bhartnett 72ee610826
Rename Fluffy debug rpc methods (#2710)
* Rename Fluffy debug rpc methods.

* Organize portal json-rpc errors.
2024-10-08 15:21:27 +08:00
andri lim 6f9fc3eced
Fix eth/common related deprecation warnings for t8n and evmstate tools (#2711)
* Fix eth/common related deprecation warnings for t8n

* Fix eth/common related deprecation warnings for evmstate
2024-10-08 12:52:32 +07:00
andri lim 22134fcf4c
Move premix's downloader and parser to fluffy/tools/eth_data_exporter (#2709)
* Move premix's downloader and parser to fluffy/tools/eth_data_exporter

* Run nph formatter
2024-10-08 03:07:24 +00:00
andri lim 76c2a75a53
Proof-of-stakiness based on block header (#2682)
* Proof-of-stakiness based on block header

* Remove unnecessary PoS check from test_txpool2

* Fix engine api simulator

* Fix indentation

* Fix vmstate debug util

* Fix MainNet ForkId calculation issue
2024-10-08 09:37:36 +07:00
Kim De Mey a3bb5d4428
Small clean-up of the differen Portal json-rpc APIs (#2708)
Clean-up removing:
- no longer relevant comments
- unused types
- unused imports
2024-10-08 09:15:29 +08:00
Kim De Mey 833719a866
Remove usage of aliases for Hash32 such as BlockHash (#2707)
These create only confusion as if they are actual different types
and it is within their usage already clear what they are about
because of the name of the variable or the function.

They are also nowhere aliased like this in any of the Portal
specification.
2024-10-07 22:39:07 +02:00
Kim De Mey e1c942ccef
Remove unneeded w3Hash and similar calls (#2706) 2024-10-07 20:54:48 +02:00
bhartnett 344a236c31
Fluffy: Create separate portal rpc handlers for each sub-network (#2705)
* Create separate portal rpc handlers for each sub-network.

* Remove random gossip from history and state networks.

* Move shared portal rpc endpoints into common handler.

* Use enum instead of network string.
2024-10-07 16:33:02 +02:00
Kim De Mey 778c1d589f
Adapt portal_*Offer method to the latest specification (#2689) 2024-10-07 10:49:04 +02:00
tersec 1ac6ee1aef
use macos-13 builders; macos-12 EOL (#2703) 2024-10-06 17:17:57 +00:00
tersec 845f3276e3
bump nimbus-build-system to use Nim v2.0.10 (#2684)
* bump nimbus-build-system to use Nim v2.0.10

* 2.0.10 fixes

* fluffy linting

* make trivial change which should trigger whole-nimbus+fluffy rebuild/ci

* Nim v2.0.10 chronicles.error/macros.error ambiguity workaround

* another contentType enum specifier

* fluffy linting
2024-10-06 12:15:54 +00:00
Jordan Hrycaj 3822c57ddc
Remove `hunter` (#2697)
* Remove `hunter`

why:
  Neither functional anymore, nor used

* Remove obsolete premix

* Remove obsolete launcher

---------

Co-authored-by: jangko <jangko128@gmail.com>
2024-10-06 10:11:44 +00:00
Advaita Saha 6565544d35
add rpc `eth_estimateGas` - dependency for blob spammer (#2701)
* add eth_estimateGas, dependency for blob spammer

* remove restriction

* fix indentation
2024-10-06 09:39:47 +07:00
Kim De Mey 91e18caf84
Fix Portal beacon statusLogLoop that was not running (#2702) 2024-10-05 13:39:42 +02:00
Kim De Mey dbe3393f5c
Fix eth/common & web3 related deprecation warnings for fluffy (#2698)
* Fix eth/common & web3 related deprecation warnings for fluffy

This commit uses the new types in the new eth/common/ structure
to remove deprecation warnings.

It is however more than just a mass replace as also all places
where eth/common or eth/common/eth_types or eth/common/eth_types_rlp
got imported have been revised and adjusted to a better per submodule
based import.

There are still a bunch of toMDigest deprecation warnings but that
convertor is not needed for fluffy code anymore so in theory it
should not be used (bug?). It seems to still get imported via export
leaks ffrom imported nimbus code I think.

* Address review comments

* Remove two more unused eth/common imports
2024-10-04 23:21:26 +02:00
Jordan Hrycaj 9b666860db
Beacon sync: Fix race condition with peer fifo overflow (#2699)
why:
  Must not call `runStop()` twice rather rely on the worker to honour
  to `zombie` flag (no harm if it does not.)
2024-10-04 20:23:30 +00:00
Jacek Sieka 08ffb3161c
Use eth/common transaction signature utilities (#2696)
* Use eth/common transaction signature utilities

* bump

* bump

* bump

* bump

* bump

* bump
2024-10-04 16:34:31 +02:00
Jacek Sieka c42ae8a037
t8n: use common transaction reader (#2688)
* t8n: use common transaction reader

* nested tx item
2024-10-04 11:27:22 +02:00
Advaita Saha e7782fd669
rpc fixes and enable kurtosis (#2681)
* fix: rpc can't serve blocks in db

* shift db access to forkedchainref

* cleanup

* kurtosis test fix, should fail + eth_getTransactionReceipt

* remove kurtosis not + cleanup

* alter CI check to pass

* optimize impl

* cleanup

* fix loop case
2024-10-04 07:59:38 +00:00
bhartnett 56724536a4
Fluffy State Bridge: Improvements to websocket rpc connections (#2694)
Handle reconnect for websocket rpc client and create rpc client instance per worker in order to improve websocket throughput. (#2694)
2024-10-04 14:17:18 +08:00
tersec d7a9d946a3
bump nim-json-serialization for Nim 2.0.10 compatibility (#2693) 2024-10-04 05:16:00 +00:00
Advaita Saha 3519b4a38c
fix: show correct blockNumber and log in end of era files (#2691) 2024-10-04 01:37:50 +00:00
Advaita Saha e3b0edd397
Fix nrpc edge cases (#2692)
* fix: engineAPI unable to connect

* fix: pre-merge situation
2024-10-04 01:37:24 +00:00
Jordan Hrycaj d6eb8c36f5
Beacon sync align internal names and docu update (#2690)
* Rename `base` -> `coupler`, `B` -> `C`

why:
  Glossary: The jargon `base` is used for the `base state` block number
  which can be smaller than what is now the `coupler`.

* Rename `global state` -> `base`, `T` -> `B`

why:
  See glossary

* Rename `final` -> `end`, `F` -> `E`

why:
  See glossary. Previously, `final` denoted some finalised block but not
  `the finalised` block from the glossary (which is maximal.)

* Properly name finalised block as such, rename `Z` -> `F`

why:
  See glossary

* Rename `least` -> `dangling`, `L` -> `D`

* Metrics update (variables not covered yet)

* Docu update and corrections

* Logger updates

* Remove obsolete `skeleton*Key` kvt columns from `storage_types` module
2024-10-03 20:19:11 +00:00
Kim De Mey 0b81a8c783
Adjust error code on portal_*TraceRecursiveFindContent (#2687) 2024-10-03 14:25:49 +02:00
Kim De Mey 764c950135
Remove PREFER_BLST_SHA256 disable, allowing for BLST SHA256 in SSZ (#2686) 2024-10-03 14:25:30 +02:00
Jacek Sieka ce331b4de8
post-merge nrpc fix (#2685)
* post-merge nrpc fix

* bump

* bump

* bump

* bump

* bump

* bump

* bump

* bump

* bump

* bump

* bump
2024-10-03 11:42:24 +00:00
bhartnett 9192aa13ed
Fluffy state bridge updates (#2683)
* Use Json decode when handling portal rpc client error.

* Remove preimages cache.

* Support disabling state gossip in order to just build state.
2024-10-03 10:57:19 +08:00
Jacek Sieka a03bb56bec
update web3 types to common eth (#2674) 2024-10-02 18:22:35 +02:00
Jordan Hrycaj 05483d89bd
Rename flare as beacon (#2680)
* Remove `--sync-mode` option from nimbus config

why:
  Currently there is only one sync mode available.

* Rename `flare` -> `beacon`, but not base module folder and nim source

why:
  The name `flare` was used do designate an alternative `beacon` mode that.

  Leaving the base folder and source as-is for a moment, makes it easier
  to read change diffs.

* Rename `flare` base module folder and nim source: `flare` -> `beacon`
2024-10-02 11:31:33 +00:00