Commit Graph

250 Commits

Author SHA1 Message Date
Kim De Mey c31fc37c62
Add async raises annotations for Portal wire and Portal networks (#2361) 2024-06-14 14:21:30 +02:00
Jacek Sieka 8a0772ac10
cleanup a few more rlpHash calls (#2359) 2024-06-14 17:42:31 +07:00
andri lim 5a18537450
Bump nim-eth, nim-web3, nimbus-eth2 (#2344)
* Bump nim-eth, nim-web3, nimbus-eth2

- Replace std.Option with results.Opt
- Fields name changes

* More fixes

* Fix Portal stream async raises and portal testnet Opt usage

* Bump eth + nimbus-eth2 + more fixes related to eth_types changes

* Fix in utp test app and nimbus-eth2 bump

* Fix test_blockchain_json rebase conflict

* Fix EVMC block_timestamp conversion plus commentary

---------

Co-authored-by: kdeme <kim.demey@gmail.com>
2024-06-14 14:31:08 +07:00
web3-developer 7fd777cfa9
Add push raises to Fluffy state network code. (#2352)
* Add push raises to Fluffy state network code.

* Refactor handling of rlp errors.
2024-06-14 13:38:24 +08:00
web3-developer f326ae2ee1
Cleanup state network recursive gossip which was removed from the portal specs. (#2345) 2024-06-13 13:53:49 +08:00
web3-developer 9c26fa3298
Updates to Fluffy to support hive tests. (#2333)
* Updates to Fluffy book for hive tests.

* Add support to disable state root checks for state content from the command line.

* Update portal_stateStore endpoint to support decoding offer and storing retrieval value.
2024-06-11 21:01:35 +08:00
web3-developer c72d6aa5d6
Implementation of procs to support Fluffy state JSON-RPC endpoints. (#2318)
* Started implementation of state endpoints.

* Add rpc calls and server stubs.

* Initial implementation of getAccountProof and getStorageProof.

* Refactor validation to use toAccount utils functions.

* Add state endpoints tests.
2024-06-10 18:47:09 +08:00
Jacek Sieka f6be4bd0ec
avoid initTable (#2328)
`initTable` is obsolete since nim 0.19 and can introduce significant
memory overhead while providing no benefit (since the table will be
grown to the default initial size on first use anyway).

In particular, aristo layers will not necessarily use all tables they
initialize, for exampe when many empty accounts are being created.
2024-06-10 11:05:30 +02:00
Kim De Mey d814d84b9b
Use --styleCheck:error for Fluffy + fixes (#2247)
* Use --styleCheck:error for Fluffy + fixes

There seems to be a clash between the names of an object field and
a proc here. As workaround names of the procs are changed.

* Fix case style for rpc client calls in test_portal_testnet

* Fix style case for utp test
2024-05-30 18:12:28 +02:00
Jacek Sieka 919242c98e
results: use canonical import (#2248) 2024-05-30 14:54:03 +02:00
web3-developer 1eb170e686
Fluffy state network tests (#2245)
* Add state network get content test.

* Completed state network get content tests.

* Completed state gossip offer content test.

* Improve state network offer content test.

* Completed state gossip offer test.

* Rename tests.
2024-05-30 17:00:57 +08:00
web3-developer 9354cb8411
Fluffy recursive gossip improvements, fixes and tests. (#2231)
* Refactor get parent gossip code and add Nibbles helper function.

* Add logging to state gossip.

* Unit test recursive gossip using state gossip getParent functions.

* Add recursive gossip genesis json test and fix bug in state gossip getParent.

* Add Nibbles len function.
2024-05-28 08:45:30 +08:00
Kim De Mey 72912626a2
Several spec fixes for HistoricalSummaries + add test-vector test (#2217) 2024-05-24 23:15:04 +02:00
web3-developer 771d6fd3e8
Implementation of Fluffy state network gossip. (#2210)
* Add toContentKey helper functions.

* Implement typed lookup functions for getting account and storage trie nodes and bytecode.

* Re-implementation of recursive gossip.

* Offer validation improvements. Don't allow key end in Nibbles for leafs as defined in the spec.

* Minor refactor to validation and add addition bytecode check.
2024-05-24 15:49:51 +08:00
web3-developer e566e89d03
Fluffy state network refactor (#2200)
* Add additional getTrieProof test.

* Refactor state network: Removed variant objects for values and split state_content into multiple files.
2024-05-22 09:22:07 +08:00
web3-developer 047f8ec4b2
Bump portal spec tests and update state validation tests to use state roots in test vectors. (#2193)
* Bump portal spec tests to version a549242

* Update state validation tests to use state roots in portal spec test vectors.
2024-05-17 23:20:57 +08:00
web3-developer 2891b9aa7d
Fluffy state offer validation (#2170)
* Implemented offer validation interface.

* Implement high level offer validation steps.

* Completed state validation tests.

* Update validation to use result type.

* Update state proof verification tests to test offer verification.

* Query history network to get state root by block hash.

* Fix state network test and remove usage of CoreDb.

* Fix state network gossip test and PR comment updates.

* Add trieproof state validation tests and fix for short nodes.
2024-05-16 16:16:16 +08:00
Kim De Mey 2e01878afe
Add the epoch to the HistoricalSummariesKey (#2179) 2024-05-13 18:49:21 +02:00
web3-developer 5c4358a969
Add retrieval content validation to state network. (#2166)
* Add retrieval content validation to state network.

* Complete state validation tests for retrieval content.

* Improve naming functions and parameters.
2024-05-06 23:13:35 +08:00
web3-developer 7da36bf459
Minor refactor to packNibbles and unpackNibbles and add additional tests. (#2162) 2024-05-03 22:55:51 +08:00
Kim De Mey 9c64b44c89
Add post Bellatrix beacon block proof exports (#2158)
- Add post Bellatrix Beacon block proof exporting to
eth_data_exporter
- Rework the Bellatrix Beacon block proof exports to us yaml
format
- Move some common code of the two different beacon block proofs
to beacon_chain_block_proof_common module
- Move + add to yaml utils to be usable not just for the tests but
also for eth data exporter
2024-05-03 11:50:18 +02:00
Kim De Mey 1512f95067
Remove unneeded nimbus imports (#2154)
Also specifically select from the still required
nimbus imports.
2024-04-25 15:21:09 +02:00
web3-developer d12300ecc9
Remove usages of CoreDb in Fluffy. (#2152)
* Remove references to CoreDb in Fluffy code.

* Add missing import.

* Remove coredb imports and use dynamic server port in tests.

* Fix formatting using nph.
2024-04-25 12:22:21 +02:00
Kim De Mey c610053a09
Prepare Fluffy for usage of historical_roots (#2138) 2024-04-17 17:58:47 +02:00
Kim De Mey 2cbc16be8b
Use test vector files for state content key tests + cleanup (#2119)
- Use yaml test vectors instead of in code data
- Add also test for content id
- Some clean-up + add content key init helper calls
2024-03-29 17:41:19 +01:00
Kim De Mey 8e4368195a
Add an audit mode to portal_bridge history backfill (#2109) 2024-03-26 22:27:31 +01:00
BiKangNing dce00079c4
Remove redundant words (#2100)
* Remove redundant words

Signed-off-by: depthlending <bikangning@outlook.com>

* Fix copyright year

---------

Signed-off-by: depthlending <bikangning@outlook.com>
Co-authored-by: jangko <jangko128@gmail.com>
2024-03-26 07:10:46 +07:00
Kim De Mey 5c1cf0410c
Add exportBlockProofBellatrix command to eth_data_exporter (#2093)
Add functionality to  export bellatrix block proofs from era files
to the eth_data_exporter tool.
2024-03-21 13:25:32 +01:00
Kim De Mey 1159b0114e
Add portal_bridge history mode (#2067)
Portal bridge mode for following latest and injecting the latest
block data into the Portal network.
2024-03-11 18:20:29 +01:00
Kim De Mey 2d76b8c010
Formatting fluffy with nph v0.5.1 + CI check (#2020)
* Add nph check to fluffy CI lint

* Add a section on nph usage in the fluffy.guide

* Update copyright years for altered files

* Avoid chained methods formatting style in db code

* Update nph in CI to v0.5

* Remove leftover commented import

* Move comment to avoid nph turning complex list into simple list (nph bug)

* Update nph in CI to v0.5.1

* Formatting fluffy with nph v0.5.1
2024-02-28 18:31:45 +01:00
Kim De Mey b656875778
Quickfix for possible IndexError in state network recursive gossip (#2052)
This would currently manifest sometimes in state gossip test in
CI or locally. But could occur on fluffy binary in case state
network  is enable and data is gossiped.
2024-02-27 17:39:23 +01:00
Daniel Sobol 95a9145f9e
add validators for data received from state network via contentLookup (#2039) 2024-02-20 16:15:24 +03:00
Daniel Sobol 47b254d8b5
State network content retrieval (#2033)
* initial rework of contentLoop for state network

* prepare content for storage and retrieval

* modify test to account for retrieval data
2024-02-19 17:35:53 +03:00
Daniel Sobol 3305c02856
Naive implementation of recursive gossip (#2024) 2024-02-15 18:37:05 +03:00
Daniel Sobol 35131b6d55
nibble encoding update (#2021)
* nibble encoding update

* portal-spec-tests update
2024-02-12 15:39:01 +03:00
Kim De Mey 3f86986658
Fix current fail on excessBlobGas field in block header (#2016) 2024-02-09 15:50:37 +01:00
Kim De Mey 2fb90cfb43
Initial implementation of era1 + export + verify (#1998)
* Initial implementation of era1 + export + verify

Initial implementation of era1 + export and verify command in the
existing eth_data_exporter.

* Quick verify version to test with geth created era1 files

* Add Era1File object and getter for specific block tuple

* Add getAccumulatorRoot and update AccumulatorRoot type name

+ clean-up of some comments.

* Implement BlockIndex based verify + additional helper calls

+ fix bug for reading last (incomplete) Era1 file

* Fix init BlockIndex offsets for last era + extra check on reading

* Remove duplicated e2store code
2024-02-09 11:13:12 +01:00
Daniel Sobol 9c53c73173
state network test vectors update (#2003)
* state network test vectors update
* nibbles codec
* moving test vectors into portal-spec-tests
* fix copyright year in all_fluffy_tests
2024-02-03 00:12:36 +03:00
Kim De Mey a39b51e3e1
Add historical summaries to Portal beacon network (#1990) 2024-01-26 23:38:12 +01:00
Kim De Mey 3ad00686f4
Fix missing ssz encoding overhead in truncateEnrs (#1988) 2024-01-25 12:30:37 +01:00
Kim De Mey 3199857ed5
Further fluffy code clean-up from warnings/hints (#1987) 2024-01-25 11:04:09 +01:00
web3-developer 54644fa3cb
Witness Generation Bug Fixes (#1981)
* Return slots when verifying witness regardless of code length.

* Prevent AccountCache WitnessData codeTouched from being reset.

* Default to using {wfNoFlag} for witness flags in tests to allow running with data from before EIP170.

* Add additional json files to experimental JSON RPC test.

* Use HTTP RPC server in tests.

* Add test to check that block witness contains bytecode.
2024-01-25 01:18:45 +08:00
Kim De Mey dbc1ae86e2
Vendor bumps + related fixes + warning fixes (#1985)
- Vendor bump of stew, nim-eth, chronos, nimbus-eth2 and libp2p
- Bump related fixes + fixes of deprecation warnings
- Several other warnings fixed.
2024-01-24 16:28:03 +01:00
web3-developer 48630ccb03
Experimental RPC endpoints for generating block witnesses (#1977)
* Completed draft implementation of witness JSON-RPC endpoints for portal network bridge.

* Updated Nimbus RPC configuration to support enabling experimental endpoints.

* Moved witness verification tests.

* Added json test for getProof.

* Added main procs to new tests to fix test suite.

* Added getBlockWitness test to blockchain json test suite.

* Added tests for experimental RPC endpoints and improved the API to support returning state proofs from before or after block execution.

* Correctly rollback transaction in getBlockWitness proc.
2024-01-22 17:11:37 +08:00
Daniel Sobol 7ecd38347f
New state network types update (#1976) 2024-01-19 20:18:57 +03:00
Kim De Mey cbf5f59681
Bump chronos and other sub modules + related fixes (#1966)
* Bump chronos and additional modules + fixes

* Additional chronos bump + nim-eth bump for utp and discv5 fixes

* Fix the raw Exception errors in async evm

---------

Co-authored-by: jangko <jangko128@gmail.com>
2024-01-12 22:06:19 +01:00
web3-developer a69b1ea90d
Fix clashing import of AccountProof type. (#1962)
* Fix clashing import of AccountProof type.
2024-01-10 16:17:58 +08:00
web3-developer 84e7c680dd
Initial implementation of block witness verification. (#1958)
* Remove support for validation of missing leafs.

* Move proof types into separate file and borrow required procs.

* Added state proof types.

* Completed initial implementation of witness verification.

* Reject empty proofs.

* Added tests for verification of block witnesses.
2024-01-09 23:09:02 +08:00
Daniel Sobol 70013422cd
Initial impl of state network bridge 1902 (#1948) 2024-01-09 12:32:29 +03:00
web3-developer 32e1f94d78
Portal State Proof Verification with Tests (#1951)
* Initial implementation of account and storage proof verification for the portal state network.

* Completed initial state proof verification loop test.

* Completed proof verification tests.

* Minor updates based on PR feedback and comments.

* Add state proof verification test to test runner.
2023-12-21 17:05:53 +08:00