Commit Graph

3816 Commits

Author SHA1 Message Date
Daniel Sobol 35131b6d55
nibble encoding update (#2021)
* nibble encoding update

* portal-spec-tests update
2024-02-12 15:39:01 +03:00
Daniil Sobol 009caee2dc
portal-spec-tests update 2024-02-12 12:07:57 +03:00
jangko d8ec962b6a
Bump confutils: no more exception leak 2024-02-12 13:01:05 +07:00
Kim De Mey cc4d13835f
Add copyright year linter for fluffy and verified proxy (#2018) 2024-02-10 14:49:34 +01:00
Peter Straus 3486e08492
fix: outdated link (#2017) 2024-02-09 20:00:47 +01:00
Kim De Mey 3f86986658
Fix current fail on excessBlobGas field in block header (#2016) 2024-02-09 15:50:37 +01:00
Jordan Hrycaj 9c37e73ff7
Core db tests supersedes part of retired custom network tests (#2015)
* Remove custom block chain unit tests

why:
  The custom block chain unit test functionality is superseded by
  `test_coredb`.

  All the custom block chains used here are hopelessly out of date and
  the configs were never updated regarding fork and ttd settings (while
  the production code has moved on.)

* CoreDb: Update unit tests suite

details:
  Can accommodate non-built in network dumps. Inherited some functionality
  from the now retired `test_custom_network` test.
2024-02-09 13:30:07 +00:00
Kim De Mey 2fb90cfb43
Initial implementation of era1 + export + verify (#1998)
* Initial implementation of era1 + export + verify

Initial implementation of era1 + export and verify command in the
existing eth_data_exporter.

* Quick verify version to test with geth created era1 files

* Add Era1File object and getter for specific block tuple

* Add getAccumulatorRoot and update AccumulatorRoot type name

+ clean-up of some comments.

* Implement BlockIndex based verify + additional helper calls

+ fix bug for reading last (incomplete) Era1 file

* Fix init BlockIndex offsets for last era + extra check on reading

* Remove duplicated e2store code
2024-02-09 11:13:12 +01:00
web3-developer 93fb4c8f59
More witness fixes (#2009)
* Update experimental rpc test to do further validation on proof responses.

* Enable zero value storage slots in the witness cache data so that proofs will be returned when a storage slot is updated to zero. Refactor and simplify implementation of getProofs endpoint.

* Improve test validation.

* Minor fixes and added test to track the changes introduced in every block using a local state.

* Refactor and cleanup test.

* Comments added to test and account cache fixes applied to account ledger.

* Return updated storage slots even when storage is empty and add test to build.

* Fix copyright and remove incorrect depth check during witness building in writeShortRlp assertion.
2024-02-09 12:09:02 +08:00
Jordan Hrycaj 2c35390bdf
Core db and aristo maintenance update (#2014)
* Aristo: Update error return code

why:
  Failing of `Aristo` function `delete()` might fail because there is
  no such data item on the db. This must return a single error code
  as is done with `fetch()`.

* Ledger: Better error handling

why:
  The `expect()` clauses have been replaced by raising asserts indicating
  the error from the database backend.

   Also, `delete()` failures are legitimate if the item to delete does not
   exist.

* Aristo: Delete function must always leave a label on DB for `hashify()`

why:
  The `hashify()` uses the labels left bu `merge()` and `delete()` to
  compile (and optimise) a scheduler for subsequent hashing.

  Originally, the labels were not used for deleted entries and `delete()`
  still had some edge case where the deletion label was not properly
  handled.

* Aristo: Update `hashify()` scheduler, remove buggy optimisation

why:
  Was left over from version without virtual state roots which did not
  know about account payload leaf vertices referring to storage roots.

* Aristo: Label storage trie account in `delete()` similar to `merge()`

details;
  The `delete()` function applied to a non-static state root (assumed
  to be a storage root) will check the payload of an accounts leaf
  and mark its Merkle keys to be re-checked when runninh `hashify()`

* Aristo: Clean up and re-org recycled vertex IDs in `hashify()`

why:
  Re-organising the recycled vertex IDs list intends to reduce the size of the
  list.

  This list is organised as a LIFO (or stack.) By reorganising it in a way
  so that the least vertex ID numbers are on top, the list will be kept
  smaller as observed on some examples (less than 30%.)

* CoreDb: Accept storage trie deletion requests in non-initialised state

why:
  Due to lazy initialisation, the root vertex ID might not yet exist. So
  the `Aristo` database handlers would reject this call with an error and
  this condition needs to be handled by the API (which realises the lazy
  feature.)

* Cosmetics & code massage, prettify logging

* fix missing import
2024-02-08 16:32:16 +00:00
Kim De Mey fa7f5de162
Bump nimbus-eth2 to latest unstable + related bumps (#2013) 2024-02-08 12:27:13 +01:00
Kim De Mey 7b2dc8f8e9
Fix ReceiptType in premix + Apply validate flag for receipts (#2012) 2024-02-08 11:50:35 +01:00
Agnish Ghosh beb13b2c40
typo fix on line 14 aristo_delete.nim (#2011)
Came across while reading through the codebase.
2024-02-07 15:29:57 +01:00
Kim De Mey 04d9ebcf33
Bump nim-eth for RLP receipts serdes fixes (#2010) 2024-02-06 07:49:50 +01:00
andri lim 8a40521cbe
Fix 'trustedSetupFile' is not accessible for type NimbusConf.cmd == import (#2008) 2024-02-04 22:45:54 +07:00
andri lim a441ec3cb1
Remove obsolete select_backend and fix simulators CI (#2007)
* Remove obsolete select_backend

* Fix copyright year
2024-02-04 21:28:20 +07:00
Daniel Sobol 9c53c73173
state network test vectors update (#2003)
* state network test vectors update
* nibbles codec
* moving test vectors into portal-spec-tests
* fix copyright year in all_fluffy_tests
2024-02-03 00:12:36 +03:00
Jordan Hrycaj f1e9ca8526
Core db+ledger aristo backend update (#2006)
* CoreDb: Improve API and API tracking

why:
  Now logs state roots where appropriate

* CoreDb: re-implement `CoreDbVidRef` => `CoreDbTrieRef`

why:
  Instead of a root vertex ID wrapper, the purpose of this object type
  has been upgrades to a sub-trie prototype.

caveat:
  `Aristo` backend not fully functional, yet.

* CoreDb: Update `Aristo` backend

why:
  Supports virtual sub-tries

* CoreDb: Account address tracking for `StorageTrie` virtual tries

details:
  Supported with API tracking/logging

* CoreDb: Keep account address in payload object

why:
  No need to provide extra address argument for `merge()`, also
  provides tracking possibility for account debugging.

* Ledger: Update new API for `Aristo` specific storage trie handling

* CoreDb+Ledger: Update unit tests

* Fix copyright headers
2024-02-02 20:23:04 +00:00
Jordan Hrycaj ac8ea4dcc2
CoreDb: Re-implemented closure iterators as inline iterators (#2005)
why:
  Compiling problems, generic closure iterator implementation seemed
  quite brittle
2024-02-02 10:58:35 +00:00
jangko 66e0c6a56c
Bump json-rpc: Support more optional types 2024-02-02 14:40:15 +07:00
Jordan Hrycaj 3b306a9689
Aristo: Update unit test suite (#2002)
* Aristo: Update unit test suite

* Aristo/Kvt: Fix iterators

why:
  Generic iterators were not properly updated after backend change

* Aristo: Add sub-trie deletion functionality

why:
  For storage tries linked to an account payload vertex ID, a the
  whole storage trie needs to be deleted with the account.

* Aristo: Reserve vertex ID numbers for static custom state roots

why:
  Static custom state roots may be controlled by an application,
  e.g. for a receipt or a transaction root. The `Aristo` functions
  are agnostic of what the static state roots are when different
  from the internal tree vertex ID 1.

details;
  The `merge()` function applied to a non-static state root (assumed
  to be a storage root) will check the payload of an accounts leaf
  and mark its Merkle keys to be re-checked.

* Aristo: Correct error code symbol

* Aristo: Update error code symbols

* Aristo: Code cosmetics/comments

* Aristo: Fix hashify schedule calculator

why:
  Had a tendency to stop early leaving an incomplete job
2024-02-01 21:27:48 +00:00
web3-developer a5ac5d3078
Premix persist tool improvements (#2000)
* Add premix persist tool to Makefile to build as separate binary.

* Fix rootBytes.len > 0 assertion defect in HexaryTrie by adding call to com.db.compensateLegacySetup().

* Run hardForkTransition before executing transactions in getBlockWitness.

* Improve logging and add help message.

* Retry requests for block data.
2024-02-01 22:11:41 +08:00
andri lim 24f7a94d5b
Bump json-rpc to v0.4.0: Support RPC batch call (#1997) 2024-01-31 18:56:03 +07:00
andri lim 09f95962ef
Allow dbBackend == none to compile (#1996)
* Allow dbBackend == none to compile

* Fix copyright year
2024-01-31 18:55:30 +07:00
andri lim c635e160d9
Implement combo http server for rpc, engine_api, and graphql services (#1992)
* Combo HTTP server implementation

* Use json flavor for jwt_auth decoder
2024-01-29 20:20:04 +07:00
jangko 61abdac2d7
Fix EthTime json encode in chain_config.nim 2024-01-29 11:26:20 +07:00
andri lim edba847d42
Utilize Json flavor for Chain config parser (#1993) 2024-01-28 08:52:20 +07:00
andri lim 94eda1f3e6
GitHub actions v4 (#1991)
* Upgrade github actions to v4

* Fix copyright year
2024-01-27 20:34:11 +07:00
Kim De Mey a39b51e3e1
Add historical summaries to Portal beacon network (#1990) 2024-01-26 23:38:12 +01:00
Vit∀ly Vlasov 048fc380a9
Nimbus light client integration with status-go rebased version (#1971)
* Nimbus light client integration with status-go

* Add cleanup code, address review comments

* Disable metrics for libverifproxy only

* Update confutils

* missing import

* build proxy in tests

* more build stuff

* namespace make vars

* export NimMain for windows

* reduce dependency on Nim compiler in header file

* copyright

---------

Co-authored-by: Vitaliy Vlasov <siphiuel@protonmail.com>
Co-authored-by: Jacek Sieka <jacek@status.im>
2024-01-26 23:04:08 +01:00
jangko 5b7857e181
Bump graphql, json-rpc and websock in preparation for combo HTTP server 2024-01-25 20:27:32 +07:00
Kim De Mey 3ad00686f4
Fix missing ssz encoding overhead in truncateEnrs (#1988) 2024-01-25 12:30:37 +01:00
Kim De Mey 3199857ed5
Further fluffy code clean-up from warnings/hints (#1987) 2024-01-25 11:04:09 +01:00
web3-developer 54644fa3cb
Witness Generation Bug Fixes (#1981)
* Return slots when verifying witness regardless of code length.

* Prevent AccountCache WitnessData codeTouched from being reset.

* Default to using {wfNoFlag} for witness flags in tests to allow running with data from before EIP170.

* Add additional json files to experimental JSON RPC test.

* Use HTTP RPC server in tests.

* Add test to check that block witness contains bytecode.
2024-01-25 01:18:45 +08:00
Kim De Mey dbc1ae86e2
Vendor bumps + related fixes + warning fixes (#1985)
- Vendor bump of stew, nim-eth, chronos, nimbus-eth2 and libp2p
- Bump related fixes + fixes of deprecation warnings
- Several other warnings fixed.
2024-01-24 16:28:03 +01:00
andri lim 31c288d5e5
Implement JSON-RPC method: eth_getBlockReceipts (#1984)
* Implement JSON-RPC method: eth_getBlockReceipts

* Increment index in eth_getBlockReceipts
2024-01-24 18:04:59 +07:00
Guido Vranken b6599b73f0
Add missing import to aristo_blobify.nim (#1983) 2024-01-24 12:09:05 +07:00
andri lim cba5d166f4
Bump websock (#1982)
* Bump nim-websock to 475ad35c89b0c6e63d8fda556c108a02b5f5e447

* Fix obsolete Rng cast

* Fix copyright year

* Remove obsolete comments
2024-01-23 22:29:12 +07:00
andri lim 107d81dbc9
Bump unittest2 to 0fe665e509340fa5ed88632e2e32692b867b08fc (#1980)
* Bump unittest2 to 0fe665e509340fa5ed88632e2e32692b867b08fc

* Bump nim-unittest2
2024-01-22 19:56:10 +07:00
web3-developer 48630ccb03
Experimental RPC endpoints for generating block witnesses (#1977)
* Completed draft implementation of witness JSON-RPC endpoints for portal network bridge.

* Updated Nimbus RPC configuration to support enabling experimental endpoints.

* Moved witness verification tests.

* Added json test for getProof.

* Added main procs to new tests to fix test suite.

* Added getBlockWitness test to blockchain json test suite.

* Added tests for experimental RPC endpoints and improved the API to support returning state proofs from before or after block execution.

* Correctly rollback transaction in getBlockWitness proc.
2024-01-22 17:11:37 +08:00
andri lim b098484e4d
Bump nat-traversal to 3398a649f4e895cae86fbe85b54769ebc0967878 (#1979) 2024-01-20 07:58:54 +07:00
Daniel Sobol 7ecd38347f
New state network types update (#1976) 2024-01-19 20:18:57 +03:00
kdeme 9875eb11d6
Bumps for chronos, nim-eth and unittest2 and future callback fixes 2024-01-19 13:37:14 +01:00
jangko 36dd297331
Bump nim-json-rpc to 8d79d528412d7cdd022ff91fb6bacd4f5bb01d58
Fix improper yield usage in json-rpc-websocketserver
Improve both client and server resilience against fields and elements with null value (#195)
2024-01-17 14:30:59 +07:00
jangko 1d50d03456
Bump nim-web3 to ce47cde31c9b08e4b702766d4d788cc83fa55cf0
Synchronize order of blobGasUsed / excessBlobGas in test (#126)
2024-01-17 13:56:08 +07:00
jangko 4d44f7b576
Bump nim-json-serialization to d9394dc7286064902d825bbc1203d03d7218633a
Add flavor feature: reader will skip null fields
add lexer test cases (#76)
Deconvolute optional fields writer (#77)
Resilience against null fields (#78)
2024-01-17 13:53:10 +07:00
Kim De Mey 2df9e2b7b1
Cleanup utp_test and make the code more flexible to change amounts (#1972) 2024-01-15 17:29:29 +01:00
jangko fb2aa1b6f9
Fix engine_sim regression 2024-01-15 13:03:29 +07:00
jangko 4fb94cacad
Bump nim-web3 to c595e0d6c8ec1ea7c28b267c8141fa70437cccaa 2024-01-14 15:25:57 +07:00
jangko 3e21281d12
Bump nim-web3 and others
Bump nim-json-rpc and nimbus-eth2 too.
Reason: both nim-json-rpc and nim-web3 migrate from
stdlib/json to nim-json-serialization
2024-01-14 10:41:23 +07:00