Commit Graph

155 Commits

Author SHA1 Message Date
Kim De Mey 1379e05db1
Add bellatrix block proof test using new test vectors (#2099) 2024-03-22 13:05:06 +01:00
Kim De Mey d3a706c229
Replace status-im/portal-spec-tests with ethereum fork version (#2097)
- The fluffy test vector repo got forked (well, copied rather) to
become the official one under ethereum github org, so we change
to that repo now and archive ours.
- Our repo also stored accumulator / historical_roots, replace
that with a new repo which is only for network configs.
- Several changes needed to be made due to test vectors that got
updated + some of them got changed to / are yaml format instead of
json.
2024-03-22 11:28:44 +01:00
Kim De Mey 49a199dbc3
Add beacon state historical_roots to fluffy (#2085)
- Add historical_roots in the binary for proof verification
pre-capella
- Add command to export historical_roots to file in
eth_data_exporter tool
2024-03-19 16:45:32 +01:00
Kim De Mey 2d76b8c010
Formatting fluffy with nph v0.5.1 + CI check (#2020)
* Add nph check to fluffy CI lint

* Add a section on nph usage in the fluffy.guide

* Update copyright years for altered files

* Avoid chained methods formatting style in db code

* Update nph in CI to v0.5

* Remove leftover commented import

* Move comment to avoid nph turning complex list into simple list (nph bug)

* Update nph in CI to v0.5.1

* Formatting fluffy with nph v0.5.1
2024-02-28 18:31:45 +01:00
Daniel Sobol 47b254d8b5
State network content retrieval (#2033)
* initial rework of contentLoop for state network

* prepare content for storage and retrieval

* modify test to account for retrieval data
2024-02-19 17:35:53 +03:00
Daniel Sobol 3305c02856
Naive implementation of recursive gossip (#2024) 2024-02-15 18:37:05 +03:00
Daniel Sobol 35131b6d55
nibble encoding update (#2021)
* nibble encoding update

* portal-spec-tests update
2024-02-12 15:39:01 +03:00
Daniel Sobol 9c53c73173
state network test vectors update (#2003)
* state network test vectors update
* nibbles codec
* moving test vectors into portal-spec-tests
* fix copyright year in all_fluffy_tests
2024-02-03 00:12:36 +03:00
Kim De Mey a39b51e3e1
Add historical summaries to Portal beacon network (#1990) 2024-01-26 23:38:12 +01:00
Kim De Mey dbc1ae86e2
Vendor bumps + related fixes + warning fixes (#1985)
- Vendor bump of stew, nim-eth, chronos, nimbus-eth2 and libp2p
- Bump related fixes + fixes of deprecation warnings
- Several other warnings fixed.
2024-01-24 16:28:03 +01:00
web3-developer 48630ccb03
Experimental RPC endpoints for generating block witnesses (#1977)
* Completed draft implementation of witness JSON-RPC endpoints for portal network bridge.

* Updated Nimbus RPC configuration to support enabling experimental endpoints.

* Moved witness verification tests.

* Added json test for getProof.

* Added main procs to new tests to fix test suite.

* Added getBlockWitness test to blockchain json test suite.

* Added tests for experimental RPC endpoints and improved the API to support returning state proofs from before or after block execution.

* Correctly rollback transaction in getBlockWitness proc.
2024-01-22 17:11:37 +08:00
Daniel Sobol 7ecd38347f
New state network types update (#1976) 2024-01-19 20:18:57 +03:00
jangko 3e21281d12
Bump nim-web3 and others
Bump nim-json-rpc and nimbus-eth2 too.
Reason: both nim-json-rpc and nim-web3 migrate from
stdlib/json to nim-json-serialization
2024-01-14 10:41:23 +07:00
Kim De Mey cbf5f59681
Bump chronos and other sub modules + related fixes (#1966)
* Bump chronos and additional modules + fixes

* Additional chronos bump + nim-eth bump for utp and discv5 fixes

* Fix the raw Exception errors in async evm

---------

Co-authored-by: jangko <jangko128@gmail.com>
2024-01-12 22:06:19 +01:00
web3-developer a69b1ea90d
Fix clashing import of AccountProof type. (#1962)
* Fix clashing import of AccountProof type.
2024-01-10 16:17:58 +08:00
web3-developer 84e7c680dd
Initial implementation of block witness verification. (#1958)
* Remove support for validation of missing leafs.

* Move proof types into separate file and borrow required procs.

* Added state proof types.

* Completed initial implementation of witness verification.

* Reject empty proofs.

* Added tests for verification of block witnesses.
2024-01-09 23:09:02 +08:00
Daniel Sobol 70013422cd
Initial impl of state network bridge 1902 (#1948) 2024-01-09 12:32:29 +03:00
web3-developer 32e1f94d78
Portal State Proof Verification with Tests (#1951)
* Initial implementation of account and storage proof verification for the portal state network.

* Completed initial state proof verification loop test.

* Completed proof verification tests.

* Minor updates based on PR feedback and comments.

* Add state proof verification test to test runner.
2023-12-21 17:05:53 +08:00
Kim De Mey dded8643d9
Update the beacon and state network content key prefixes (#1950) 2023-12-19 19:59:38 +01:00
Kim De Mey aad4adde07
Move all Fluffy database code to database folder (#1918) 2023-12-01 17:20:52 +01:00
Kim De Mey 49d6c5728b
Bump nim-ssz-ser for fix for Empty SSZ list in Union + add test (#1909) 2023-12-01 13:04:34 +01:00
Kim De Mey c46706ac75
Add force pruning for the storage-capacity option (#1901) 2023-11-23 11:44:57 +01:00
Kim De Mey 6338969dd5
Lower the content deletion fraction and other contentdb clean-up (#1893)
- The current setting of 0.25 was very big. Set to 0.5, which is
potentially still large. This change did expose some issues with
the current implementation and especially testing.
- General clean-up, renaming for consistency, and
re-ordering/restructuring/deletion of some code.
- Fixed several typos
- ...
2023-11-16 15:27:30 +01:00
Kim De Mey 86f93fdba5
ValidIpAddress -> IpAddress related vendor bumps and changes (#1892) 2023-11-10 19:38:11 +01:00
Kim De Mey 03a739ff1b
Add contentdb contentCount and contentdb renames/cleanup (#1886) 2023-11-07 19:46:26 +01:00
Kim De Mey 436ef0e40e
Portal BeaconLightClientNetwork -> BeaconNetwork renaming (#1832) 2023-10-20 12:06:25 +02:00
Kim De Mey 0472b75e23
Rework fluffy beacon lc db to allow storing processed content (#1821) 2023-10-18 16:59:44 +02:00
Kim De Mey ca61a7009d
Bump submodules and related fixes (#1729)
* Bump submodules and related fixes

* Fix some warnings due to chronos and Nim bumps

* Seems macOS and Windows have different behavior
2023-10-17 14:19:50 +02:00
Kim De Mey 3444ffaf30
Update test to latest beacon LC test vectors (#1815) 2023-10-11 15:44:23 +02:00
Kim De Mey 7b313ec1ac
Use signature slot instead of attested header slot (#1811)
Use signature slot instead of attested header slot in the
OptimisticUpdate content key.
2023-10-06 17:42:34 +02:00
Kim De Mey f72f02c88b
Remove optimistic slot from FinalityUpdate content key (#1810) 2023-10-06 15:46:53 +02:00
Kim De Mey 492543a007
Fix and enable optimistic/finality update portal network test (#1794) 2023-10-02 11:41:11 +02:00
Kim De Mey 2f2c5127ea
Further implementation to working prototype for Portal beacon LC (#1781)
- Link network gossip validation to LC processor validation
- QuickFix put/get for optimistic and finality updates
- Minor fixes and clean-up
- Improve bridge gossip for new LC Updates
- Adjust local testnet script to be able to locally test this
- Adjust test and skip broken test
2023-09-28 18:16:41 +02:00
Kim De Mey c405f79a84
Update the portal beacon lc manager (#1770)
Similar changes sd done on the libp2p version. To be able to have
same improvements and reuse more of the lc sync helpers code.
2023-09-22 18:42:02 +02:00
andri lim 56215ed83f
Bump stint to v2.0: new array backend (#1747)
* Bump stint to v2.0: new array backend
2023-09-13 09:32:38 +07:00
Daniel Sobol 69a1a988b0
don't gossip data back to the node we rcvd it from [#1227] (#1719) 2023-09-04 12:21:01 +02:00
Kim De Mey 7ae333ac8b
Add discv5 provided ENR directly to Portal protocol routing table (#1714)
* Add discv5 provided ENR directly to Portal protocol routing table

Previous version of getting the ENR from the discv5 routing table
would not work due to the order of first calling the talk protocol
handler and only after that the addEnr to the disc5 routing table.

Instead of changing this order, pass along the ENR directly to
avoid this additional getNode call.

* Still request ENR from discv5 if it wasn't passed via handshake
2023-08-30 20:38:15 +02:00
Kim De Mey 7aead6151e
Add utpTransfer bool to recursiveFindContent JSON-RPC result (#1710)
Rename portal protocol ContentInfo to ContentKV in the
process, due to duplicate types and simply better name
for this object.
2023-08-24 18:19:29 +02:00
Jordan Hrycaj 221e6c9e2f
Unified database frontend integration (#1670)
* Nimbus folder environment update

details:
* Integrated `CoreDbRef` for the sources in the `nimbus` sub-folder.
* The `nimbus` program does not compile yet as it needs the updates
  in the parallel `stateless` sub-folder.

* Stateless environment update

details:
* Integrated `CoreDbRef` for the sources in the `stateless` sub-folder.
* The `nimbus` program compiles now.

* Premix environment update

details:
* Integrated `CoreDbRef` for the sources in the `premix` sub-folder.

* Fluffy environment update

details:
* Integrated `CoreDbRef` for the sources in the `fluffy` sub-folder.

* Tools environment update

details:
* Integrated `CoreDbRef` for the sources in the `tools` sub-folder.

* Nodocker environment update

details:
* Integrated `CoreDbRef` for the sources in the
  `hive_integration/nodocker` sub-folder.

* Tests environment update

details:
* Integrated `CoreDbRef` for the sources in the `tests` sub-folder.
* The unit tests compile and run cleanly now.

* Generalise `CoreDbRef` to any `select_backend` supported database

why:
  Generalisation was just missed due to overcoming some compiler oddity
  which was tied to rocksdb for testing.

* Suppress compiler warning for `newChainDB()`

why:
  Warning was added to this function which must be wrapped so that
  any `CatchableError` is re-raised as `Defect`.

* Split off persistent `CoreDbRef` constructor into separate file

why:
  This allows to compile a memory only database version without linking
  the backend library.

* Use memory `CoreDbRef` database by default

detail:
 Persistent DB constructor needs to import `db/core_db/persistent

why:
 Most tests use memory DB anyway. This avoids linking `-lrocksdb` or
 any other backend by default.

* fix `toLegacyBackend()` availability check

why:
  got garbled after memory/persistent split.

* Clarify raw access to MPT for snap sync handler

why:
  Logically, `kvt` is not the raw access for the hexary trie (although
  this holds for the legacy database)
2023-08-04 12:10:09 +01:00
Kim De Mey 89b3e679e2
Add new portal beacon chain lc content encoding tests (#1636) 2023-07-08 17:01:33 +02:00
Kim De Mey c7302b4ece
Compile Fluffy tests and fluffy beacon LC tests together (#1633)
This wins some time.

Also, remove and move some of the compile time defines to the
cfg files.
2023-07-06 18:06:44 +02:00
jangko 67aaf92c1d
bump submodules 2023-06-07 18:12:02 +07:00
Kim De Mey bf21484f70
Add EL block proofs verifable from historical_summaries (#1588) 2023-05-30 07:02:39 +02:00
Kim De Mey c9f3f82877
Add new Portal BlockBody type for Shanghai fork (#1589) 2023-05-30 06:56:54 +02:00
Kim De Mey 33fd8b1fae
Add historical_roots with proof + test (#1579) 2023-05-11 19:08:34 +02:00
Kim De Mey 3eafb15d7c
Make beacon LC adjustements to support capella in light clients (#1545)
- Bump nimbus-eth2 repo
- Beacon LC adjustements in fluffy
- Beacon LC adjustements in nimbus_verified_proxy
2023-04-13 18:40:01 +02:00
Kim De Mey 2d51196645
Add usage of header 14764013 for historya content test vectors (#1533) 2023-04-06 19:57:01 +02:00
Kim De Mey 0b960f323c
Add test vectors for history content block bodies and receipts (#1525) 2023-03-30 14:21:09 +02:00
Kim De Mey e6964a029b
Remove unused, underdeveloped BridgeClient from Fluffy (#1520)
It is only (mostly) a skeleton, not further developed and not
used. It is unlikely the way forward either when further
developing the Portal state network.
2023-03-23 13:41:40 +01:00
Kim De Mey 8989141446
Renames on Portal beacon LC code (#1496) 2023-03-13 21:30:57 +01:00