Commit Graph

3629 Commits

Author SHA1 Message Date
Jordan Hrycaj 0d4ef023ed
Update aristo journal functionality (#2155)
* Aristo: Code cosmetics, e.g. update some CamelCase names

* CoreDb+Aristo: Provide oldest known state root implied

details:
  The Aristo journal allows to recover earlier but not all state roots.

* Aristo: Fix journal backward index operator, e.g. `[^1]`

* Aristo: Fix journal updater

why:
  The `fifosStore()` store function slightly misinterpreted the update
  instructions when translation is to database `put()` functions. The
  effect was that the journal was ever growing due to stale entries which
  were never deleted.

* CoreDb+Aristo: Provide utils for purging stale data from the KVT

details:
  See earlier patch, not all state roots are available. This patch
  provides a mapping from some state root to a block number and allows to
  remove all KVT data related to a particular block number

* Aristo+Kvt: Implement a clean up schedule for expired data in KVT

why:
  For a single state ledger like `Aristo`, there is only a limited
  backlog of states. So KVT data (i.e. headers etc.) are cleaned up
  regularly

* Fix copyright year
2024-04-26 13:43:52 +00:00
Kim De Mey 1512f95067
Remove unneeded nimbus imports (#2154)
Also specifically select from the still required
nimbus imports.
2024-04-25 15:21:09 +02:00
web3-developer d12300ecc9
Remove usages of CoreDb in Fluffy. (#2152)
* Remove references to CoreDb in Fluffy code.

* Add missing import.

* Remove coredb imports and use dynamic server port in tests.

* Fix formatting using nph.
2024-04-25 12:22:21 +02:00
andri lim 70be982259
Fix CI badge URL (#2150) 2024-04-24 08:17:13 +07:00
andri lim 47605dbdf3
Fix txPool: excessBlobGas should be calculated before entering EVM, not after (#2149) 2024-04-24 07:50:50 +07:00
Jordan Hrycaj b9187e0493
Aristo selective read cashing for rocksdb backend (#2145)
* Aristo+Kvt: Better RocksDB profiling

why:
  Providing more detailed information, mainly for `Aristo`

* Aristo: Renamed journal `stats()` to `capacity()`

why:
  `Stats()` was a misnomer

* Aristo: Provide backend read caches for key and vertex IDs

why:
  Dedicated LRU caching for particular types gives a throughput advantage.
  The sizes of the LRU queues used for caching are currently constant
  but might be adjusted at a later time.

* Fix copyright year
2024-04-22 19:02:22 +00:00
Kim De Mey 1d90b924ee
Adjust color scheme for fluffy.guide (#2144)
Use the same scheme as for the nimbus.guide.
Two differences however:
- Set the dark scheme as default, considering the new nimbus.team
website also has dark as default.
- Remove the orange/amber for the light scheme, considering the
light version of nimbus.team has no more orange.
2024-04-22 15:33:51 +02:00
andri lim 4078cb14c5
Engine api simulator: fix payload customizer (#2143)
* Engine api simulator: fix payload customizer

* Move versionedHashes validation before blockHash validation in newPayload

* More descriptive error message

* Fix Web3Hash to eth.Hash256 conversion
2024-04-21 21:44:05 +07:00
andri lim 6694e240d7
Repositioning blob hash validation in newPayload of engine API (#2141) 2024-04-20 02:43:13 +07:00
Jordan Hrycaj 7d9e1d8607
Misc updates for full sync (#2140)
* Code cosmetics

* Aristo+Kvt: Fix api wrappers

why:
  Api setup killed the backend descriptor when backend mapping was
  disabled.

* Aristo: Implement masked profiling entries

why:
  Database backend should be listed but not counted in tally

* CoreDb: Simplify backend() methods

why:
  DBMS backend access Was provided very early and over engineered. Now
  there are only two backend machines, one for `Kvt` and the other one
  for an `Mpt` available only via new API.

* CoreDb: Code cleanup regarding descriptor types

* CoreDb: Refactor/redefine `persistent()` methods

why:
  There were `persistent()` methods for any type of caching storage
  facilities `Kvt`, `Mpt`, `Phk`, and `Acc`. Now there is only a single
  `persistent()` method storing all facilities in tandem (similar to
  how transactions work.)

  For non shared `Kvt` tables, there is now an extra storage method
  `saveOffSite()`.

* CoreDb lingo update: `trie` becomes `column`

why:
  Notion of a `trie` is pretty much hidden by the new `CoreDb` api.
  Revealed are sort of database columns for accounts an storage data,
  any of which have an internal state represented by a Keccack hash.
  So a `trie` or `MPT` becomes a `column` and a `rootHash` becomes a
  column state.

* Aristo: rename backend filed `filters` => `journal`

* Update full sync logging

details:
  + Disable eth handler noise while syncing
  + Log journal depth (if available)

* Fix copyright year

* Fix cruft and unwanted imports
2024-04-19 18:37:27 +00:00
jangko e2c873b16f
Fix engine API and engine API simulator according to latest spec. 2024-04-18 14:55:49 +07:00
Jordan Hrycaj e8eb3268f5
Generalise prune mode option 4 different db models (#2139)
* Update README

* Nimbus-main: replaced `PruneMode` options by `ChainDbMode` options

details:
  For the legacy database, this changes the phrase
  - `conf.pruneMode == PruneMode.Full` to the expression
  + `conf.chainDbMode == ChainDbMode.Prune`.

* Fix issues moaned about by NIM compiler

* Fix copyright year
2024-04-17 18:09:55 +00:00
Kim De Mey c610053a09
Prepare Fluffy for usage of historical_roots (#2138) 2024-04-17 17:58:47 +02:00
Kim De Mey d0ac3a1986
Add docs on pruning in Fluffy (#2137) 2024-04-17 14:06:53 +02:00
jangko 6c1afe1127
Engine API: forkChoiceUpdated V2 And V3 should return invalidPayloadAttributes error instead of invalidParams error upon receiving erroneous payload attributes 2024-04-17 15:45:39 +07:00
Jordan Hrycaj d6a4205324
Aristo update rocksdb backend drivers (#2135)
* Aristo+RocksDB: Update backend drivers

why:
  RocksDB update allows use some of the newly provided methods which
  were previously implemented by using the very C backend (for the lack
  of NIM methods.)

* Aristo+RocksDB: Simplify drivers wrapper

* Kvt: Update backend drivers and wrappers similar to `Aristo`

* Aristo+Kvm: Use column families for RocksDB

* Aristo+MemoryDB: Code cosmetics

* Aristo: Provide guest column family for export

why:
  So `Kvt` can piggyback on `Aristo` so there avoiding to run a second
  DBMS system in parallel.

* Kvt: Provide import mechanism for RoksDB guest column family

why:
  So `Kvt` can piggyback on `Aristo` so there avoiding to run a second
   DBMS system in parallel.

* CoreDb+Aristo: Run persistent `Kvt` DB piggybacked on `Aristo`

why:
  Avoiding to run two DBMS systems in parallel.

* Fix copyright year

* Ditto
2024-04-16 20:39:11 +00:00
jangko 160e033663
Remove intermediate RpcCallData 2024-04-16 19:52:07 +07:00
jangko 7a941c8ca0
Rename hasCodeOrNonce to contractCollision 2024-04-16 09:31:10 +07:00
andri lim e713f3c287
Implement eth_feeHistory (#2130)
* Implement eth_feeHistory

* Fix copyright year
2024-04-16 08:02:42 +07:00
Jordan Hrycaj dac7a2cbe1
Providing eth68 stubs while setting eth67 as default (#2129)
* Suspend `snap` sync tests

why:
  Snap needs to be updated. While this is not done, tests are obsolete
  and eat up memory resources only

* Suspend rocks DB timing tests

why:
  Currently of no use for general test suite.

* Mothballed unused evm code for opportunistic DB handling

why:
  Needs to be refactored anyway. Uses hard coded protocol dependencies.

* Update `eth` protocol configuration

why:
 + new upcoming version `eth68`
 + prepare for eth-multi protocol stack support

* Strip the `legacy_` prefix from eth66 compiler flag variable

why:
  Being it is an oddity having `eth67_enabled`, `eth68_enabled`, and
  `legacy_eth_66_enabled`.

* Providing eth68 stubs

* Fix copyright year

* Fix eth68 stub method name
2024-04-12 11:21:17 +00:00
andri lim b6f8a000b9
Bump nim-web3 to a31bc63448d8259255a00c130af68d6e558c60f5 (#2126) 2024-04-04 15:23:25 +07:00
andri lim 516dd95566
Implement eth_blobBaseFee (#2124) 2024-04-04 07:25:56 +07:00
Jordan Hrycaj 1502014e36
Core db+aristo re org tracer (#2123)
* Kvt: Update API hooks

* Aristo: Generalised merging snap proofs, now for multiple state roots

why:
  This accommodates pre-loading partial tries for unit tests

* Aristo: Update some unit tests

* CoreDb+Aristo: Re-factor tracer

why:
  Was bonkers anyway. The main change is that the trace journal is now
  kept in a way similar to a transaction layer so that it can predictably
  interact with DB transactions.

* Ledger: Debugging helper

* Update tracer unit test applicable for `Aristo`

* Fix copyright year

* Disable `dump()` function as compile time default

why:
  This needs to pull in the `rocks_db` library at compile time.
2024-04-03 15:48:35 +00:00
Kim De Mey 6c2c99a9ae
Add reclaim to the fcli_db prune cmd (#2122) 2024-04-02 20:50:05 +02:00
Kim De Mey 2cbc16be8b
Use test vector files for state content key tests + cleanup (#2119)
- Use yaml test vectors instead of in code data
- Add also test for content id
- Some clean-up + add content key init helper calls
2024-03-29 17:41:19 +01:00
web3-developer c17966d585
Update nim-rocksdb version. (#2120)
Update nim-rocksdb version to latest.
2024-03-30 00:35:56 +08:00
Kim De Mey 095bcae8b7
Use yaml versions of the portal beacon lc test vectors (#2117) 2024-03-29 17:12:23 +01:00
andri lim ea6701a68f
Bump ssz-serialization to 248f2bdca2d65ff920920c72b764d0622d522596 (#2118) 2024-03-29 11:21:02 +07:00
web3-developer 0c9f25a845
Try fixing nightly build. (#2115)
* Add -lrocksdb to fix windows and macos builds.

* Link in windows required libraries.
2024-03-28 23:20:43 +08:00
andri lim 0d4db21dad
Add engine_getPayloadV4 and engine_newPayloadV4 (#2116) 2024-03-28 18:59:23 +07:00
andri lim fd9bb28b20
Bump nim-web3 to 9e370474fb16039860d074535dd1649f1c2307b2 (#2112) 2024-03-28 14:16:40 +07:00
andri lim be74ea83f9
Bump nim-evmc to 86d22a026b0aa07c07b3afd7d91ca475e0eae12a (#2113) 2024-03-28 13:47:02 +07:00
andri lim 64fe19a590
Bump nim-kzg4844 to 4fbcfbe4c452313bd440936318a87ed708987d8b (#2111) 2024-03-28 09:16:02 +07:00
web3-developer 0c0141817c
Static link rocksdb with Nimbus. (#2084)
* Static link rocksdb into Nimbus.

* Link in additional compression libraries.

* Support static linking for MacOS.

* Add flag to support disabling rocksdb static linking.

* Static linking is disabled by default. Build nimbus with static linking using: make nimbus_rocksdb_static

* Update nim-rocksdb to latest.
2024-03-28 09:55:12 +08:00
Kim De Mey 058f412bbc
Small adjustment to portal_bridge + docs update (#2110) 2024-03-27 19:16:24 +01:00
Kim De Mey 8e4368195a
Add an audit mode to portal_bridge history backfill (#2109) 2024-03-26 22:27:31 +01:00
web3-developer 547f198ace
Build hunter tool as a separate binary. (#2108) 2024-03-27 00:18:03 +08:00
andri lim ef378c88a5
Fix payload validation in engine_newPayload (#2107)
* Fix validatePayload in engine_newPayload

* Fix copyright year
2024-03-26 22:13:30 +07:00
andri lim 33976e8875
Replace invalidParams error with invalidForkchoiceState error in forkchoiceUpdated handler (#2106) 2024-03-26 09:31:21 +07:00
andri lim 32f631534d
Bump nim-web3 to 4d80530dd54af2c3285496317eddb2b7a82b10ad (#2105) 2024-03-26 09:31:10 +07:00
BiKangNing dce00079c4
Remove redundant words (#2100)
* Remove redundant words

Signed-off-by: depthlending <bikangning@outlook.com>

* Fix copyright year

---------

Signed-off-by: depthlending <bikangning@outlook.com>
Co-authored-by: jangko <jangko128@gmail.com>
2024-03-26 07:10:46 +07:00
Kim De Mey f3de770c14
Bump nimbus-eth2 module (#2104) 2024-03-25 13:40:57 +01:00
Snoppy 4848fd19d5
chore: fix typos (#2103) 2024-03-25 09:14:47 +01:00
Kim De Mey 1c98733d4d
Move portal bridge beacon code and add skeleton for state bridge (#2101) 2024-03-22 19:01:22 +01:00
Jordan Hrycaj 889a1165b0
Core db+aristo update tracer for non trivial operations (#2102)
* Remove cruft

* Docu/code cosmetics

* Aristo: Update `forkBase()`

why:
  Was not up to the job

* Update/correct tracer for running against `Aristo`

details:
  This patch makes sure that before creating a new `BaseVMState` the
  `CoreDb` context is adjusted to accommodate for the state root that
  is passed to the `BaseVMState` constructor.

* CpreDb+legacy: Always return current context with `ctxFromTx()`

why:
  There was an experimental setting trying to find the node with the
  proper setting in the KVT (not the hexary tie layer) which currently
  does not work reliable, probably due to `Ledger` caching effects.
2024-03-22 17:31:56 +00:00
andri lim eb67e78fde
Implement RPC methods: debug_getRawBlock, debug_getRawHeader, debug_g… (#2098)
* Implement RPC methods: debug_getRawBlock, debug_getRawHeader, debug_getRawReceipts, debug_getRawTransaction

* Fix engine api sim when calling setupDebugRpc
2024-03-22 23:40:16 +07:00
Kim De Mey 1379e05db1
Add bellatrix block proof test using new test vectors (#2099) 2024-03-22 13:05:06 +01:00
Kim De Mey d3a706c229
Replace status-im/portal-spec-tests with ethereum fork version (#2097)
- The fluffy test vector repo got forked (well, copied rather) to
become the official one under ethereum github org, so we change
to that repo now and archive ours.
- Our repo also stored accumulator / historical_roots, replace
that with a new repo which is only for network configs.
- Several changes needed to be made due to test vectors that got
updated + some of them got changed to / are yaml format instead of
json.
2024-03-22 11:28:44 +01:00
andri lim 9cd80800df
Load Kzg trusted setup at the start of consensus sim (#2096) 2024-03-22 12:49:26 +07:00
Kim De Mey 5c1cf0410c
Add exportBlockProofBellatrix command to eth_data_exporter (#2093)
Add functionality to  export bellatrix block proofs from era files
to the eth_data_exporter tool.
2024-03-21 13:25:32 +01:00