Commit Graph

4160 Commits

Author SHA1 Message Date
Jordan Hrycaj f937f57838
Beacon sync targets cons head rather than finalised block (#2721)
* Fix fringe condition clarifying how to handle an empty range

why:
  The `interval_set` module would treat an undefined interval construct
  `[2,1]` as`[2,2]` (the right bound being `max(2,1)`.)

* Use the `consensus head` rather than the `finalised` block as sync target

why:
  The former is ahead of the `finalised` block.

* In ctx descriptor rename `final` field to `target`

* Update docu, rename `F` -> `T`
2024-10-09 18:00:00 +00:00
bhartnett b13f06fcfb
Fluffy state portal rpc validation (#2719)
* Validate content key in portal_stateLocalContent.

* Add additional validation to stateStore rpc method.
2024-10-09 20:23:46 +08:00
Kim De Mey 5edb0b320f
Allow Portal beacon LC sync to start from a stored LC bootstrap (#2715)
Portal beacon LC sync can be started now from a provided trusted
block root or, in case it has been running before, from a
previously verified and stored LC bootstrap.

This required altering the the beacon db on how the bootstraps
are stored.
2024-10-09 10:21:00 +02:00
Jacek Sieka 11646ad3c4
Ordered trie (#2712)
Speed up trie computations and remove redundant ways of performing this
operation.

Co-authored-by: jangko <jangko128@gmail.com>
2024-10-09 09:44:15 +02:00
andri lim e59d2825f4
Bump nim-stew (#2718) 2024-10-09 11:47:27 +07:00
andri lim 52e3e20643
Bump nim-unittest2 (#2717) 2024-10-09 11:47:15 +07:00
andri lim fe4527617f
Bump nim-eth (#2716) 2024-10-09 11:47:02 +07:00
bhartnett 72ee610826
Rename Fluffy debug rpc methods (#2710)
* Rename Fluffy debug rpc methods.

* Organize portal json-rpc errors.
2024-10-08 15:21:27 +08:00
andri lim 6f9fc3eced
Fix eth/common related deprecation warnings for t8n and evmstate tools (#2711)
* Fix eth/common related deprecation warnings for t8n

* Fix eth/common related deprecation warnings for evmstate
2024-10-08 12:52:32 +07:00
andri lim 22134fcf4c
Move premix's downloader and parser to fluffy/tools/eth_data_exporter (#2709)
* Move premix's downloader and parser to fluffy/tools/eth_data_exporter

* Run nph formatter
2024-10-08 03:07:24 +00:00
andri lim 76c2a75a53
Proof-of-stakiness based on block header (#2682)
* Proof-of-stakiness based on block header

* Remove unnecessary PoS check from test_txpool2

* Fix engine api simulator

* Fix indentation

* Fix vmstate debug util

* Fix MainNet ForkId calculation issue
2024-10-08 09:37:36 +07:00
Kim De Mey a3bb5d4428
Small clean-up of the differen Portal json-rpc APIs (#2708)
Clean-up removing:
- no longer relevant comments
- unused types
- unused imports
2024-10-08 09:15:29 +08:00
Kim De Mey 833719a866
Remove usage of aliases for Hash32 such as BlockHash (#2707)
These create only confusion as if they are actual different types
and it is within their usage already clear what they are about
because of the name of the variable or the function.

They are also nowhere aliased like this in any of the Portal
specification.
2024-10-07 22:39:07 +02:00
Kim De Mey e1c942ccef
Remove unneeded w3Hash and similar calls (#2706) 2024-10-07 20:54:48 +02:00
bhartnett 344a236c31
Fluffy: Create separate portal rpc handlers for each sub-network (#2705)
* Create separate portal rpc handlers for each sub-network.

* Remove random gossip from history and state networks.

* Move shared portal rpc endpoints into common handler.

* Use enum instead of network string.
2024-10-07 16:33:02 +02:00
Kim De Mey 778c1d589f
Adapt portal_*Offer method to the latest specification (#2689) 2024-10-07 10:49:04 +02:00
tersec 1ac6ee1aef
use macos-13 builders; macos-12 EOL (#2703) 2024-10-06 17:17:57 +00:00
tersec 845f3276e3
bump nimbus-build-system to use Nim v2.0.10 (#2684)
* bump nimbus-build-system to use Nim v2.0.10

* 2.0.10 fixes

* fluffy linting

* make trivial change which should trigger whole-nimbus+fluffy rebuild/ci

* Nim v2.0.10 chronicles.error/macros.error ambiguity workaround

* another contentType enum specifier

* fluffy linting
2024-10-06 12:15:54 +00:00
Jordan Hrycaj 3822c57ddc
Remove `hunter` (#2697)
* Remove `hunter`

why:
  Neither functional anymore, nor used

* Remove obsolete premix

* Remove obsolete launcher

---------

Co-authored-by: jangko <jangko128@gmail.com>
2024-10-06 10:11:44 +00:00
Advaita Saha 6565544d35
add rpc `eth_estimateGas` - dependency for blob spammer (#2701)
* add eth_estimateGas, dependency for blob spammer

* remove restriction

* fix indentation
2024-10-06 09:39:47 +07:00
Kim De Mey 91e18caf84
Fix Portal beacon statusLogLoop that was not running (#2702) 2024-10-05 13:39:42 +02:00
Kim De Mey dbe3393f5c
Fix eth/common & web3 related deprecation warnings for fluffy (#2698)
* Fix eth/common & web3 related deprecation warnings for fluffy

This commit uses the new types in the new eth/common/ structure
to remove deprecation warnings.

It is however more than just a mass replace as also all places
where eth/common or eth/common/eth_types or eth/common/eth_types_rlp
got imported have been revised and adjusted to a better per submodule
based import.

There are still a bunch of toMDigest deprecation warnings but that
convertor is not needed for fluffy code anymore so in theory it
should not be used (bug?). It seems to still get imported via export
leaks ffrom imported nimbus code I think.

* Address review comments

* Remove two more unused eth/common imports
2024-10-04 23:21:26 +02:00
Jordan Hrycaj 9b666860db
Beacon sync: Fix race condition with peer fifo overflow (#2699)
why:
  Must not call `runStop()` twice rather rely on the worker to honour
  to `zombie` flag (no harm if it does not.)
2024-10-04 20:23:30 +00:00
Jacek Sieka 08ffb3161c
Use eth/common transaction signature utilities (#2696)
* Use eth/common transaction signature utilities

* bump

* bump

* bump

* bump

* bump

* bump
2024-10-04 16:34:31 +02:00
Jacek Sieka c42ae8a037
t8n: use common transaction reader (#2688)
* t8n: use common transaction reader

* nested tx item
2024-10-04 11:27:22 +02:00
Advaita Saha e7782fd669
rpc fixes and enable kurtosis (#2681)
* fix: rpc can't serve blocks in db

* shift db access to forkedchainref

* cleanup

* kurtosis test fix, should fail + eth_getTransactionReceipt

* remove kurtosis not + cleanup

* alter CI check to pass

* optimize impl

* cleanup

* fix loop case
2024-10-04 07:59:38 +00:00
bhartnett 56724536a4
Fluffy State Bridge: Improvements to websocket rpc connections (#2694)
Handle reconnect for websocket rpc client and create rpc client instance per worker in order to improve websocket throughput. (#2694)
2024-10-04 14:17:18 +08:00
tersec d7a9d946a3
bump nim-json-serialization for Nim 2.0.10 compatibility (#2693) 2024-10-04 05:16:00 +00:00
Advaita Saha 3519b4a38c
fix: show correct blockNumber and log in end of era files (#2691) 2024-10-04 01:37:50 +00:00
Advaita Saha e3b0edd397
Fix nrpc edge cases (#2692)
* fix: engineAPI unable to connect

* fix: pre-merge situation
2024-10-04 01:37:24 +00:00
Jordan Hrycaj d6eb8c36f5
Beacon sync align internal names and docu update (#2690)
* Rename `base` -> `coupler`, `B` -> `C`

why:
  Glossary: The jargon `base` is used for the `base state` block number
  which can be smaller than what is now the `coupler`.

* Rename `global state` -> `base`, `T` -> `B`

why:
  See glossary

* Rename `final` -> `end`, `F` -> `E`

why:
  See glossary. Previously, `final` denoted some finalised block but not
  `the finalised` block from the glossary (which is maximal.)

* Properly name finalised block as such, rename `Z` -> `F`

why:
  See glossary

* Rename `least` -> `dangling`, `L` -> `D`

* Metrics update (variables not covered yet)

* Docu update and corrections

* Logger updates

* Remove obsolete `skeleton*Key` kvt columns from `storage_types` module
2024-10-03 20:19:11 +00:00
Kim De Mey 0b81a8c783
Adjust error code on portal_*TraceRecursiveFindContent (#2687) 2024-10-03 14:25:49 +02:00
Kim De Mey 764c950135
Remove PREFER_BLST_SHA256 disable, allowing for BLST SHA256 in SSZ (#2686) 2024-10-03 14:25:30 +02:00
Jacek Sieka ce331b4de8
post-merge nrpc fix (#2685)
* post-merge nrpc fix

* bump

* bump

* bump

* bump

* bump

* bump

* bump

* bump

* bump

* bump

* bump
2024-10-03 11:42:24 +00:00
bhartnett 9192aa13ed
Fluffy state bridge updates (#2683)
* Use Json decode when handling portal rpc client error.

* Remove preimages cache.

* Support disabling state gossip in order to just build state.
2024-10-03 10:57:19 +08:00
Jacek Sieka a03bb56bec
update web3 types to common eth (#2674) 2024-10-02 18:22:35 +02:00
Jordan Hrycaj 05483d89bd
Rename flare as beacon (#2680)
* Remove `--sync-mode` option from nimbus config

why:
  Currently there is only one sync mode available.

* Rename `flare` -> `beacon`, but not base module folder and nim source

why:
  The name `flare` was used do designate an alternative `beacon` mode that.

  Leaving the base folder and source as-is for a moment, makes it easier
  to read change diffs.

* Rename `flare` base module folder and nim source: `flare` -> `beacon`
2024-10-02 11:31:33 +00:00
Jacek Sieka e19790614b
avoid threadvar in nim-serialization (#2675) 2024-10-02 09:01:45 +02:00
bhartnett 44fea2348b
Fluffy Portal RPC Client (#2649)
* Portal rpc client implementation.
2024-10-02 12:23:13 +08:00
tersec 216604d0d6
add eth_sendRawTransaction to server API (#2678) 2024-10-02 03:56:39 +00:00
Jordan Hrycaj 5b6ccddaa0
Db folder sources and related remove compiler warnings (#2673)
* Aristo: Rename `Hash256` -> `Hash32`

* CoreDb: Rename `Hash256` -> `Hash32`

* Ledger: Rename `Hash256` -> `Hash32`

* StorageTypes: Rename `Hash256` -> `Hash32`

* Aristo: Rename `Blob` -> `seq[byte]`, `keccakHash` -> `keccak256`

* Kvt: Rename `Blob` -> `seq[byte]`

* CoreDb: Rename `Blob` -> `seq[byte]`, `keccakHash` -> `keccak256`

* Ledger: Rename `Blob` -> `seq[byte]`, `keccakHash` -> `keccak256`

* CoreDb: Rename `BlockHeader` -> `Header`, `BlockNonce` -> `Bytes8`

* Misc: Rename `StorageKey` -> `Bytes32`

* Tracer: `Hash256` -> `Hash32`, `BlockHeader` -> `Header`, etc.

* Fix copyright header
2024-10-01 21:03:10 +00:00
Advaita Saha 900b108a41
fix CI by using a older assertor branch (#2676) 2024-10-02 00:59:46 +05:30
Jacek Sieka 219b22b1f5
Versioned hash32 (#2672) 2024-10-01 19:40:37 +02:00
Jordan Hrycaj c022b29d14
Clean up modules in sync folder (#2670)
* Dissolve legacy `sync/types.nim` into `*/eth/eth_types.nim`

* Flare sync: Simplify scheduler and remove `runSingle()` method

why:
  `runSingle()` is not used anymore (main purpose was for negotiating
  best headers in legacy full sync.)

  Also, `runMulti()` was renamed `runPeer()`

* Flare sync: Move `chain` field from `sync_desc` -> `worker_desc`

* Flare sync: Remove handler descriptor lateral reference

why:
  Not used anymore. It enabled to turn on/off eth handler activity with
  regards to the sync state, i.e.from with in the sync worker.

* Flare sync: Update `Hash256` and other deprecated `std/eth` symbols

* Protocols: Update `Hash256` and other deprecated `std/eth` symbols

* Eth handler: Update `Hash256` and other deprecated `std/eth` symbols

* Update flare TODO

* Remove redundant `sync/type` import

why:
  The import module `type` has been removed

* Remove duplicate implementation
2024-10-01 09:19:29 +00:00
bhartnett 0ce9658b9c
Bump nim-rocksdb - Updates RocksDb to v9.6.1 (#2671) 2024-10-01 13:33:59 +08:00
tersec 036a15c8f1
use non-deprecated engine API method for Hive JWT tests (#2668)
* use non-deprecated engine API method for Hive JWT tests

* also remove reference in nrpc
2024-10-01 03:28:14 +00:00
Advaita Saha 8d4b789039
fix: post bump migration in nrpc (#2667) 2024-09-29 22:52:51 +02:00
Advaita Saha e8542f951f
External syncer (#2574)
* inital external sync structure

* add to makefile

* feat: external syncer with healing process

* fix: suggestions

* network mapping

* minor changes

* forward sync

* jwt auth

* nrpc structure

* nrpc engine-api loader

* fix: suggestions

* fix: suggestions

* remove sync_db

* fix: edge cases and forks

* fix: rebase changes

* revert nimbus config changes
2024-09-29 18:48:11 +02:00
Jacek Sieka c210885b73
eth: bump to new types (#2660)
This is a minimal set of changes to make things work with the new types
in nim-eth - this is the minimal PR that merely resolves
incompatibilities while the full change set would include more cleanup
and migration.
2024-09-29 14:37:09 +02:00
andri lim 5f1b945ebe
Remove beacon sync (#2666) 2024-09-29 02:13:50 +00:00