Commit Graph

372 Commits

Author SHA1 Message Date
Kim De Mey d3a706c229
Replace status-im/portal-spec-tests with ethereum fork version (#2097)
- The fluffy test vector repo got forked (well, copied rather) to
become the official one under ethereum github org, so we change
to that repo now and archive ours.
- Our repo also stored accumulator / historical_roots, replace
that with a new repo which is only for network configs.
- Several changes needed to be made due to test vectors that got
updated + some of them got changed to / are yaml format instead of
json.
2024-03-22 11:28:44 +01:00
Kim De Mey 5c1cf0410c
Add exportBlockProofBellatrix command to eth_data_exporter (#2093)
Add functionality to  export bellatrix block proofs from era files
to the eth_data_exporter tool.
2024-03-21 13:25:32 +01:00
andri lim 30277be1f3
Bump nim-web3 to 285d97c2b05bbe2a13dab4b52ea878157fb1a1a1 (#2088)
* Bump nim-web3 to 285d97c2b05bbe2a13dab4b52ea878157fb1a1a1

Unify EthCall/EthSend into TransactionArgs (#138)

* bump ssz-serialization

* Fix BlockNumber conversion

* Bump ssz-serialization: Restrict toSszType usage to non SszType in readSszBytes (#81)
2024-03-21 08:05:22 +07:00
Kim De Mey 49a199dbc3
Add beacon state historical_roots to fluffy (#2085)
- Add historical_roots in the binary for proof verification
pre-capella
- Add command to export historical_roots to file in
eth_data_exporter tool
2024-03-19 16:45:32 +01:00
Kim De Mey bfa756c36b
Bump json-rpc and web3 and remove unneeded errors import/export (#2078) 2024-03-15 11:44:18 +01:00
Kim De Mey 461c7e7d85
Add Era1 backfill to portal_bridge history mode (#2077) 2024-03-15 10:41:41 +01:00
Kim De Mey 0fb7632aaa
Fix BlockNumber conversion for getBlockReceipts in portal_bridge (#2074) 2024-03-14 17:21:10 +01:00
Kim De Mey 7287efc7b4
Clean-up of some Portal json-rpc code (#2073)
- Use the new createRpcSigsFromNim for client json-rpc API
- Avoid importing any nimbus/rpc specifics, use only web3 and
fluffy local rpc code
- Adjust tools making use of the client side API
2024-03-13 16:58:50 +01:00
Kim De Mey 849607cfd1
Remove option of using beacon_lc_bridge together with a full node (#2070)
Keep the beacon_lc_bridge a fully standalone bridge, and instead
use the new portal_bridge which works by means of full nodes or
web3 providers.
2024-03-12 14:06:22 +01:00
Kim De Mey 1159b0114e
Add portal_bridge history mode (#2067)
Portal bridge mode for following latest and injecting the latest
block data into the Portal network.
2024-03-11 18:20:29 +01:00
Kim De Mey f8fca07d37
Update protocol-interop-testing.md with correct dcli build dir (#2064) 2024-03-08 10:32:53 +01:00
Kim De Mey 88a93beb26
Provide a specific error in JSON-RPC API when content is not found (#2059) 2024-03-06 08:57:09 +01:00
Kim De Mey 0debf1a122
Introduce --networks and --portal-network cli options (#2061)
The --networks option is to select the networks alias the Portal
sub-protocols.
The --network option is deprecated and replaced with the
--portal-network option to avoid confusion with --networks option.

This commit also remove the --state flag which was a temporary
quick-fix.
2024-03-05 13:45:48 +01:00
Kim De Mey 2d76b8c010
Formatting fluffy with nph v0.5.1 + CI check (#2020)
* Add nph check to fluffy CI lint

* Add a section on nph usage in the fluffy.guide

* Update copyright years for altered files

* Avoid chained methods formatting style in db code

* Update nph in CI to v0.5

* Remove leftover commented import

* Move comment to avoid nph turning complex list into simple list (nph bug)

* Update nph in CI to v0.5.1

* Formatting fluffy with nph v0.5.1
2024-02-28 18:31:45 +01:00
Kim De Mey b656875778
Quickfix for possible IndexError in state network recursive gossip (#2052)
This would currently manifest sometimes in state gossip test in
CI or locally. But could occur on fluffy binary in case state
network  is enable and data is gossiped.
2024-02-27 17:39:23 +01:00
Daniel Sobol 95a9145f9e
add validators for data received from state network via contentLookup (#2039) 2024-02-20 16:15:24 +03:00
Daniel Sobol 47b254d8b5
State network content retrieval (#2033)
* initial rework of contentLoop for state network

* prepare content for storage and retrieval

* modify test to account for retrieval data
2024-02-19 17:35:53 +03:00
Daniel Sobol 2766e20179
Add flag to explicitly enable state network (#2031) 2024-02-17 01:38:58 +03:00
Kim De Mey 9378774b9f
Add Era1 based gossip calls + Era1 helpers (#2029)
- Add Era1 helpers to be able to iterate fast over block tuples
and individual total difficulties
- Add buildAccumulator from Era1 file
- Add Era1 based BlockHeader with proof + bodies/receipts gossip
calls
- Add new JSON-RPC debug methods to be able to test the above
with a standalone fluffy node
2024-02-15 16:49:22 +01:00
Daniel Sobol 3305c02856
Naive implementation of recursive gossip (#2024) 2024-02-15 18:37:05 +03:00
Daniel Sobol 35131b6d55
nibble encoding update (#2021)
* nibble encoding update

* portal-spec-tests update
2024-02-12 15:39:01 +03:00
Kim De Mey 3f86986658
Fix current fail on excessBlobGas field in block header (#2016) 2024-02-09 15:50:37 +01:00
Kim De Mey 2fb90cfb43
Initial implementation of era1 + export + verify (#1998)
* Initial implementation of era1 + export + verify

Initial implementation of era1 + export and verify command in the
existing eth_data_exporter.

* Quick verify version to test with geth created era1 files

* Add Era1File object and getter for specific block tuple

* Add getAccumulatorRoot and update AccumulatorRoot type name

+ clean-up of some comments.

* Implement BlockIndex based verify + additional helper calls

+ fix bug for reading last (incomplete) Era1 file

* Fix init BlockIndex offsets for last era + extra check on reading

* Remove duplicated e2store code
2024-02-09 11:13:12 +01:00
Daniel Sobol 9c53c73173
state network test vectors update (#2003)
* state network test vectors update
* nibbles codec
* moving test vectors into portal-spec-tests
* fix copyright year in all_fluffy_tests
2024-02-03 00:12:36 +03:00
Kim De Mey a39b51e3e1
Add historical summaries to Portal beacon network (#1990) 2024-01-26 23:38:12 +01:00
Kim De Mey 3ad00686f4
Fix missing ssz encoding overhead in truncateEnrs (#1988) 2024-01-25 12:30:37 +01:00
Kim De Mey 3199857ed5
Further fluffy code clean-up from warnings/hints (#1987) 2024-01-25 11:04:09 +01:00
web3-developer 54644fa3cb
Witness Generation Bug Fixes (#1981)
* Return slots when verifying witness regardless of code length.

* Prevent AccountCache WitnessData codeTouched from being reset.

* Default to using {wfNoFlag} for witness flags in tests to allow running with data from before EIP170.

* Add additional json files to experimental JSON RPC test.

* Use HTTP RPC server in tests.

* Add test to check that block witness contains bytecode.
2024-01-25 01:18:45 +08:00
Kim De Mey dbc1ae86e2
Vendor bumps + related fixes + warning fixes (#1985)
- Vendor bump of stew, nim-eth, chronos, nimbus-eth2 and libp2p
- Bump related fixes + fixes of deprecation warnings
- Several other warnings fixed.
2024-01-24 16:28:03 +01:00
web3-developer 48630ccb03
Experimental RPC endpoints for generating block witnesses (#1977)
* Completed draft implementation of witness JSON-RPC endpoints for portal network bridge.

* Updated Nimbus RPC configuration to support enabling experimental endpoints.

* Moved witness verification tests.

* Added json test for getProof.

* Added main procs to new tests to fix test suite.

* Added getBlockWitness test to blockchain json test suite.

* Added tests for experimental RPC endpoints and improved the API to support returning state proofs from before or after block execution.

* Correctly rollback transaction in getBlockWitness proc.
2024-01-22 17:11:37 +08:00
Daniel Sobol 7ecd38347f
New state network types update (#1976) 2024-01-19 20:18:57 +03:00
Kim De Mey 2df9e2b7b1
Cleanup utp_test and make the code more flexible to change amounts (#1972) 2024-01-15 17:29:29 +01:00
jangko 3e21281d12
Bump nim-web3 and others
Bump nim-json-rpc and nimbus-eth2 too.
Reason: both nim-json-rpc and nim-web3 migrate from
stdlib/json to nim-json-serialization
2024-01-14 10:41:23 +07:00
Kim De Mey cbf5f59681
Bump chronos and other sub modules + related fixes (#1966)
* Bump chronos and additional modules + fixes

* Additional chronos bump + nim-eth bump for utp and discv5 fixes

* Fix the raw Exception errors in async evm

---------

Co-authored-by: jangko <jangko128@gmail.com>
2024-01-12 22:06:19 +01:00
web3-developer a69b1ea90d
Fix clashing import of AccountProof type. (#1962)
* Fix clashing import of AccountProof type.
2024-01-10 16:17:58 +08:00
web3-developer 84e7c680dd
Initial implementation of block witness verification. (#1958)
* Remove support for validation of missing leafs.

* Move proof types into separate file and borrow required procs.

* Added state proof types.

* Completed initial implementation of witness verification.

* Reject empty proofs.

* Added tests for verification of block witnesses.
2024-01-09 23:09:02 +08:00
Daniel Sobol 70013422cd
Initial impl of state network bridge 1902 (#1948) 2024-01-09 12:32:29 +03:00
jangko c0d52ba179
Get rid of HexQuantityStr and HexDataStr usage
Both types are not safe and require validation/conversion
from rpc implementer.
This PR change it to safer types and delegate the conversion
and validation to the rpc library.
2023-12-27 07:48:53 +07:00
web3-developer 32e1f94d78
Portal State Proof Verification with Tests (#1951)
* Initial implementation of account and storage proof verification for the portal state network.

* Completed initial state proof verification loop test.

* Completed proof verification tests.

* Minor updates based on PR feedback and comments.

* Add state proof verification test to test runner.
2023-12-21 17:05:53 +08:00
Kim De Mey dded8643d9
Update the beacon and state network content key prefixes (#1950) 2023-12-19 19:59:38 +01:00
Kim De Mey 3675cd6736
Update protocol-interop-testing.md with fixed dcli build location (#1946) 2023-12-15 09:22:03 +01:00
Daniel Sobol 5b448e87aa
trin's bootstrap ENR update. Fix #1926 (#1927) 2023-12-15 09:04:26 +01:00
Kim De Mey 64bfe4d63a
Fix missing parameter passing for fullTransactions in JSON-RPC API (#1942) 2023-12-13 14:49:10 +01:00
jangko 5fd54961e3
Bump nim-web3: remove rpc types duplicates 2023-12-13 07:58:12 +07:00
Kim De Mey c99c0687f8
Rename beacon bridge to portal_bridge and prep for 1 bridge binary (#1936)
Renaming the beacon_chain_bridge to portal_bridge and preparing
for stuffing all bridge functionality for all networks under this
2023-12-12 21:08:58 +01:00
Kim De Mey 2b71e3e600
Remove contentSize and contentCount from history network log (#1933) 2023-12-11 22:00:56 +01:00
Daniel Sobol dc55ee0c87
fix local dev container for portal-hive (#1931)
* fix local dev container for portal-hive
2023-12-11 14:56:00 +01:00
Kim De Mey 78ec80eea0
Add trace logs when ENR gets added to portal network routing table (#1932) 2023-12-11 11:53:32 +01:00
Kim De Mey b27c2ff449
Add fcli_db tool for offline interactions with a fluffy ContentDb (#1929) 2023-12-07 13:08:49 +01:00
Kim De Mey 443e7c49a9
Fix state network RPC methods in interop docs (#1922) 2023-12-04 16:27:04 +01:00