Commit Graph

3502 Commits

Author SHA1 Message Date
andri lim 8a40521cbe
Fix 'trustedSetupFile' is not accessible for type NimbusConf.cmd == import (#2008) 2024-02-04 22:45:54 +07:00
andri lim a441ec3cb1
Remove obsolete select_backend and fix simulators CI (#2007)
* Remove obsolete select_backend

* Fix copyright year
2024-02-04 21:28:20 +07:00
Daniel Sobol 9c53c73173
state network test vectors update (#2003)
* state network test vectors update
* nibbles codec
* moving test vectors into portal-spec-tests
* fix copyright year in all_fluffy_tests
2024-02-03 00:12:36 +03:00
Jordan Hrycaj f1e9ca8526
Core db+ledger aristo backend update (#2006)
* CoreDb: Improve API and API tracking

why:
  Now logs state roots where appropriate

* CoreDb: re-implement `CoreDbVidRef` => `CoreDbTrieRef`

why:
  Instead of a root vertex ID wrapper, the purpose of this object type
  has been upgrades to a sub-trie prototype.

caveat:
  `Aristo` backend not fully functional, yet.

* CoreDb: Update `Aristo` backend

why:
  Supports virtual sub-tries

* CoreDb: Account address tracking for `StorageTrie` virtual tries

details:
  Supported with API tracking/logging

* CoreDb: Keep account address in payload object

why:
  No need to provide extra address argument for `merge()`, also
  provides tracking possibility for account debugging.

* Ledger: Update new API for `Aristo` specific storage trie handling

* CoreDb+Ledger: Update unit tests

* Fix copyright headers
2024-02-02 20:23:04 +00:00
Jordan Hrycaj ac8ea4dcc2
CoreDb: Re-implemented closure iterators as inline iterators (#2005)
why:
  Compiling problems, generic closure iterator implementation seemed
  quite brittle
2024-02-02 10:58:35 +00:00
jangko 66e0c6a56c
Bump json-rpc: Support more optional types 2024-02-02 14:40:15 +07:00
Jordan Hrycaj 3b306a9689
Aristo: Update unit test suite (#2002)
* Aristo: Update unit test suite

* Aristo/Kvt: Fix iterators

why:
  Generic iterators were not properly updated after backend change

* Aristo: Add sub-trie deletion functionality

why:
  For storage tries linked to an account payload vertex ID, a the
  whole storage trie needs to be deleted with the account.

* Aristo: Reserve vertex ID numbers for static custom state roots

why:
  Static custom state roots may be controlled by an application,
  e.g. for a receipt or a transaction root. The `Aristo` functions
  are agnostic of what the static state roots are when different
  from the internal tree vertex ID 1.

details;
  The `merge()` function applied to a non-static state root (assumed
  to be a storage root) will check the payload of an accounts leaf
  and mark its Merkle keys to be re-checked.

* Aristo: Correct error code symbol

* Aristo: Update error code symbols

* Aristo: Code cosmetics/comments

* Aristo: Fix hashify schedule calculator

why:
  Had a tendency to stop early leaving an incomplete job
2024-02-01 21:27:48 +00:00
web3-developer a5ac5d3078
Premix persist tool improvements (#2000)
* Add premix persist tool to Makefile to build as separate binary.

* Fix rootBytes.len > 0 assertion defect in HexaryTrie by adding call to com.db.compensateLegacySetup().

* Run hardForkTransition before executing transactions in getBlockWitness.

* Improve logging and add help message.

* Retry requests for block data.
2024-02-01 22:11:41 +08:00
andri lim 24f7a94d5b
Bump json-rpc to v0.4.0: Support RPC batch call (#1997) 2024-01-31 18:56:03 +07:00
andri lim 09f95962ef
Allow dbBackend == none to compile (#1996)
* Allow dbBackend == none to compile

* Fix copyright year
2024-01-31 18:55:30 +07:00
andri lim c635e160d9
Implement combo http server for rpc, engine_api, and graphql services (#1992)
* Combo HTTP server implementation

* Use json flavor for jwt_auth decoder
2024-01-29 20:20:04 +07:00
jangko 61abdac2d7
Fix EthTime json encode in chain_config.nim 2024-01-29 11:26:20 +07:00
andri lim edba847d42
Utilize Json flavor for Chain config parser (#1993) 2024-01-28 08:52:20 +07:00
andri lim 94eda1f3e6
GitHub actions v4 (#1991)
* Upgrade github actions to v4

* Fix copyright year
2024-01-27 20:34:11 +07:00
Kim De Mey a39b51e3e1
Add historical summaries to Portal beacon network (#1990) 2024-01-26 23:38:12 +01:00
Vit∀ly Vlasov 048fc380a9
Nimbus light client integration with status-go rebased version (#1971)
* Nimbus light client integration with status-go

* Add cleanup code, address review comments

* Disable metrics for libverifproxy only

* Update confutils

* missing import

* build proxy in tests

* more build stuff

* namespace make vars

* export NimMain for windows

* reduce dependency on Nim compiler in header file

* copyright

---------

Co-authored-by: Vitaliy Vlasov <siphiuel@protonmail.com>
Co-authored-by: Jacek Sieka <jacek@status.im>
2024-01-26 23:04:08 +01:00
jangko 5b7857e181
Bump graphql, json-rpc and websock in preparation for combo HTTP server 2024-01-25 20:27:32 +07:00
Kim De Mey 3ad00686f4
Fix missing ssz encoding overhead in truncateEnrs (#1988) 2024-01-25 12:30:37 +01:00
Kim De Mey 3199857ed5
Further fluffy code clean-up from warnings/hints (#1987) 2024-01-25 11:04:09 +01:00
web3-developer 54644fa3cb
Witness Generation Bug Fixes (#1981)
* Return slots when verifying witness regardless of code length.

* Prevent AccountCache WitnessData codeTouched from being reset.

* Default to using {wfNoFlag} for witness flags in tests to allow running with data from before EIP170.

* Add additional json files to experimental JSON RPC test.

* Use HTTP RPC server in tests.

* Add test to check that block witness contains bytecode.
2024-01-25 01:18:45 +08:00
Kim De Mey dbc1ae86e2
Vendor bumps + related fixes + warning fixes (#1985)
- Vendor bump of stew, nim-eth, chronos, nimbus-eth2 and libp2p
- Bump related fixes + fixes of deprecation warnings
- Several other warnings fixed.
2024-01-24 16:28:03 +01:00
andri lim 31c288d5e5
Implement JSON-RPC method: eth_getBlockReceipts (#1984)
* Implement JSON-RPC method: eth_getBlockReceipts

* Increment index in eth_getBlockReceipts
2024-01-24 18:04:59 +07:00
Guido Vranken b6599b73f0
Add missing import to aristo_blobify.nim (#1983) 2024-01-24 12:09:05 +07:00
andri lim cba5d166f4
Bump websock (#1982)
* Bump nim-websock to 475ad35c89b0c6e63d8fda556c108a02b5f5e447

* Fix obsolete Rng cast

* Fix copyright year

* Remove obsolete comments
2024-01-23 22:29:12 +07:00
andri lim 107d81dbc9
Bump unittest2 to 0fe665e509340fa5ed88632e2e32692b867b08fc (#1980)
* Bump unittest2 to 0fe665e509340fa5ed88632e2e32692b867b08fc

* Bump nim-unittest2
2024-01-22 19:56:10 +07:00
web3-developer 48630ccb03
Experimental RPC endpoints for generating block witnesses (#1977)
* Completed draft implementation of witness JSON-RPC endpoints for portal network bridge.

* Updated Nimbus RPC configuration to support enabling experimental endpoints.

* Moved witness verification tests.

* Added json test for getProof.

* Added main procs to new tests to fix test suite.

* Added getBlockWitness test to blockchain json test suite.

* Added tests for experimental RPC endpoints and improved the API to support returning state proofs from before or after block execution.

* Correctly rollback transaction in getBlockWitness proc.
2024-01-22 17:11:37 +08:00
andri lim b098484e4d
Bump nat-traversal to 3398a649f4e895cae86fbe85b54769ebc0967878 (#1979) 2024-01-20 07:58:54 +07:00
Daniel Sobol 7ecd38347f
New state network types update (#1976) 2024-01-19 20:18:57 +03:00
kdeme 9875eb11d6
Bumps for chronos, nim-eth and unittest2 and future callback fixes 2024-01-19 13:37:14 +01:00
jangko 36dd297331
Bump nim-json-rpc to 8d79d528412d7cdd022ff91fb6bacd4f5bb01d58
Fix improper yield usage in json-rpc-websocketserver
Improve both client and server resilience against fields and elements with null value (#195)
2024-01-17 14:30:59 +07:00
jangko 1d50d03456
Bump nim-web3 to ce47cde31c9b08e4b702766d4d788cc83fa55cf0
Synchronize order of blobGasUsed / excessBlobGas in test (#126)
2024-01-17 13:56:08 +07:00
jangko 4d44f7b576
Bump nim-json-serialization to d9394dc7286064902d825bbc1203d03d7218633a
Add flavor feature: reader will skip null fields
add lexer test cases (#76)
Deconvolute optional fields writer (#77)
Resilience against null fields (#78)
2024-01-17 13:53:10 +07:00
Kim De Mey 2df9e2b7b1
Cleanup utp_test and make the code more flexible to change amounts (#1972) 2024-01-15 17:29:29 +01:00
jangko fb2aa1b6f9
Fix engine_sim regression 2024-01-15 13:03:29 +07:00
jangko 4fb94cacad
Bump nim-web3 to c595e0d6c8ec1ea7c28b267c8141fa70437cccaa 2024-01-14 15:25:57 +07:00
jangko 3e21281d12
Bump nim-web3 and others
Bump nim-json-rpc and nimbus-eth2 too.
Reason: both nim-json-rpc and nim-web3 migrate from
stdlib/json to nim-json-serialization
2024-01-14 10:41:23 +07:00
Kim De Mey 964b355dc8
Bump Nim version, nimbus-eth2 and ssz-serialization (#1967) 2024-01-13 22:46:06 +01:00
Kim De Mey cbf5f59681
Bump chronos and other sub modules + related fixes (#1966)
* Bump chronos and additional modules + fixes

* Additional chronos bump + nim-eth bump for utp and discv5 fixes

* Fix the raw Exception errors in async evm

---------

Co-authored-by: jangko <jangko128@gmail.com>
2024-01-12 22:06:19 +01:00
andri lim 3f4d3fef25
Bump serialization family (#1965)
* Bump serialization family
2024-01-12 21:18:15 +07:00
andri lim f1ede724ff
Add Cancun time to network preset(Goerli, Sepolia, and Holesky) (#1963)
* Add Cancun time to network preset

* Fix copyright year
2024-01-12 09:59:27 +07:00
Jordan Hrycaj a1161b537b
Core db update storage root management for sub tries (#1964)
* Aristo: Re-phrase `LayerDelta` and `LayerFinal` as object references

why:
  Avoids copying in some cases

* Fix copyright header

* Aristo: Verify `leafTie.root` function argument for `merge()` proc

why:
  Zero root will lead to inconsistent DB entry

* Aristo: Update failure condition for hash labels compiler `hashify()`

why:
  Node need not be rejected as long as links are on the schedule. In
  that case, `redo[]` is to become `wff.base[]` at a later stage.

  This amends an earlier fix, part of #1952 by also testing against
  the target nodes of the `wff.base[]` sets.

* Aristo: Add storage root glue record to `hashify()` schedule

why:
  An account leaf node might refer to a non-resolvable storage root ID.
  Storage root node chains will end up at the storage root. So the link
  `storage-root->account-leaf` needs an extra item in the schedule.

* Aristo: fix error code returned by `fetchPayload()`

details:
  Final error code is implied by the error code form the `hikeUp()`
  function.

* CoreDb: Discard `createOk` argument in API `getRoot()` function

why:
  Not needed for the legacy DB. For the `Arsto` DB, a lazy approach is
  implemented where a stprage root node is created on-the-fly.

* CoreDb: Prevent `$$` logging in some cases

why:
  Logging the function `$$` is not useful when it is used for internal
  use, i.e. retrieving an an error text for logging.

* CoreDb: Add `tryHashFn()` to API for pretty printing

why:
  Pretty printing must not change the hashification status for the
  `Aristo` DB. So there is an independent API wrapper for getting the
  node hash which never updated the hashes.

* CoreDb: Discard `update` argument in API `hash()` function

why:
  When calling the API function `hash()`, the latest state is always
  wanted. For a version that uses the current state as-is without checking,
  the function `tryHash()` was added to the backend.

* CoreDb: Update opaque vertex ID objects for the `Aristo` backend

why:
  For `Aristo`, vID objects encapsulate a numeric `VertexID`
  referencing a vertex (rather than a node hash as used on the
  legacy backend.) For storage sub-tries, there might be no initial
  vertex known when the descriptor is created. So opaque vertex ID
  objects are supported without a valid `VertexID` which will be
  initalised on-the-fly when the first item is merged.

* CoreDb: Add pretty printer for opaque vertex ID objects

* Cosmetics, printing profiling data

* CoreDb: Fix segfault in `Aristo` backend when creating MPT descriptor

why:
  Missing initialisation  error

* CoreDb: Allow MPT to inherit shared context on `Aristo` backend

why:
  Creates descriptors with different storage roots for the same
  shared `Aristo` DB descriptor.

* Cosmetics, update diagnostic message items for `Aristo` backend

* Fix Copyright year
2024-01-11 19:11:38 +00:00
web3-developer a69b1ea90d
Fix clashing import of AccountProof type. (#1962)
* Fix clashing import of AccountProof type.
2024-01-10 16:17:58 +08:00
web3-developer e60e5801da
Fix vmflags for witness generation 1934. (#1961)
* Fix issue causing vmflags to be reset during call to processBlocks and enable witness generation in test_blockchain_json test.

* Fix copyright notice on updated files.
2024-01-09 23:15:19 +08:00
web3-developer 84e7c680dd
Initial implementation of block witness verification. (#1958)
* Remove support for validation of missing leafs.

* Move proof types into separate file and borrow required procs.

* Added state proof types.

* Completed initial implementation of witness verification.

* Reject empty proofs.

* Added tests for verification of block witnesses.
2024-01-09 23:09:02 +08:00
Daniel Sobol 70013422cd
Initial impl of state network bridge 1902 (#1948) 2024-01-09 12:32:29 +03:00
web3-developer 79c6bdc214
Implementation of Nimbus eth_getProof RPC Endpoint. (#1960)
* Initial implementation of eth_getProof endpoint.

* Implemented generation of account and storage proofs.

* Minor fixes and additional tests.

* Refactor getBranch code into a separate file.

* Improve usage of test data.

* Fix copyright year.

* Return zero hash for codeHash and storageHash if account doesn't exist.

* Update copyright notice and moved trie key hashing to inside getBranch proc.
2024-01-09 10:05:52 +08:00
jangko c0d52ba179
Get rid of HexQuantityStr and HexDataStr usage
Both types are not safe and require validation/conversion
from rpc implementer.
This PR change it to safer types and delegate the conversion
and validation to the rpc library.
2023-12-27 07:48:53 +07:00
jangko 5e95df6bde
Bump json-serialization to v0.2.0: Improve lexer flexibility 2023-12-26 17:27:06 +07:00
andri lim 540db59357
bump toml-serialization to v0.2.10: improve strictness (#1954) 2023-12-26 14:00:38 +07:00
web3-developer 32e1f94d78
Portal State Proof Verification with Tests (#1951)
* Initial implementation of account and storage proof verification for the portal state network.

* Completed initial state proof verification loop test.

* Completed proof verification tests.

* Minor updates based on PR feedback and comments.

* Add state proof verification test to test runner.
2023-12-21 17:05:53 +08:00