142 Commits

Author SHA1 Message Date
Kim De Mey
cbf5f59681
Bump chronos and other sub modules + related fixes (#1966)
* Bump chronos and additional modules + fixes

* Additional chronos bump + nim-eth bump for utp and discv5 fixes

* Fix the raw Exception errors in async evm

---------

Co-authored-by: jangko <jangko128@gmail.com>
2024-01-12 22:06:19 +01:00
web3-developer
a69b1ea90d
Fix clashing import of AccountProof type. (#1962)
* Fix clashing import of AccountProof type.
2024-01-10 16:17:58 +08:00
web3-developer
84e7c680dd
Initial implementation of block witness verification. (#1958)
* Remove support for validation of missing leafs.

* Move proof types into separate file and borrow required procs.

* Added state proof types.

* Completed initial implementation of witness verification.

* Reject empty proofs.

* Added tests for verification of block witnesses.
2024-01-09 23:09:02 +08:00
Daniel Sobol
70013422cd
Initial impl of state network bridge 1902 (#1948) 2024-01-09 12:32:29 +03:00
web3-developer
32e1f94d78
Portal State Proof Verification with Tests (#1951)
* Initial implementation of account and storage proof verification for the portal state network.

* Completed initial state proof verification loop test.

* Completed proof verification tests.

* Minor updates based on PR feedback and comments.

* Add state proof verification test to test runner.
2023-12-21 17:05:53 +08:00
Kim De Mey
dded8643d9
Update the beacon and state network content key prefixes (#1950) 2023-12-19 19:59:38 +01:00
Kim De Mey
aad4adde07
Move all Fluffy database code to database folder (#1918) 2023-12-01 17:20:52 +01:00
Kim De Mey
49d6c5728b
Bump nim-ssz-ser for fix for Empty SSZ list in Union + add test (#1909) 2023-12-01 13:04:34 +01:00
Kim De Mey
c46706ac75
Add force pruning for the storage-capacity option (#1901) 2023-11-23 11:44:57 +01:00
Kim De Mey
6338969dd5
Lower the content deletion fraction and other contentdb clean-up (#1893)
- The current setting of 0.25 was very big. Set to 0.5, which is
potentially still large. This change did expose some issues with
the current implementation and especially testing.
- General clean-up, renaming for consistency, and
re-ordering/restructuring/deletion of some code.
- Fixed several typos
- ...
2023-11-16 15:27:30 +01:00
Kim De Mey
86f93fdba5
ValidIpAddress -> IpAddress related vendor bumps and changes (#1892) 2023-11-10 19:38:11 +01:00
Kim De Mey
03a739ff1b
Add contentdb contentCount and contentdb renames/cleanup (#1886) 2023-11-07 19:46:26 +01:00
Kim De Mey
436ef0e40e
Portal BeaconLightClientNetwork -> BeaconNetwork renaming (#1832) 2023-10-20 12:06:25 +02:00
Kim De Mey
0472b75e23
Rework fluffy beacon lc db to allow storing processed content (#1821) 2023-10-18 16:59:44 +02:00
Kim De Mey
ca61a7009d
Bump submodules and related fixes (#1729)
* Bump submodules and related fixes

* Fix some warnings due to chronos and Nim bumps

* Seems macOS and Windows have different behavior
2023-10-17 14:19:50 +02:00
Kim De Mey
3444ffaf30
Update test to latest beacon LC test vectors (#1815) 2023-10-11 15:44:23 +02:00
Kim De Mey
7b313ec1ac
Use signature slot instead of attested header slot (#1811)
Use signature slot instead of attested header slot in the
OptimisticUpdate content key.
2023-10-06 17:42:34 +02:00
Kim De Mey
f72f02c88b
Remove optimistic slot from FinalityUpdate content key (#1810) 2023-10-06 15:46:53 +02:00
Kim De Mey
492543a007
Fix and enable optimistic/finality update portal network test (#1794) 2023-10-02 11:41:11 +02:00
Kim De Mey
2f2c5127ea
Further implementation to working prototype for Portal beacon LC (#1781)
- Link network gossip validation to LC processor validation
- QuickFix put/get for optimistic and finality updates
- Minor fixes and clean-up
- Improve bridge gossip for new LC Updates
- Adjust local testnet script to be able to locally test this
- Adjust test and skip broken test
2023-09-28 18:16:41 +02:00
Kim De Mey
c405f79a84
Update the portal beacon lc manager (#1770)
Similar changes sd done on the libp2p version. To be able to have
same improvements and reuse more of the lc sync helpers code.
2023-09-22 18:42:02 +02:00
andri lim
56215ed83f
Bump stint to v2.0: new array backend (#1747)
* Bump stint to v2.0: new array backend
2023-09-13 09:32:38 +07:00
Daniel Sobol
69a1a988b0
don't gossip data back to the node we rcvd it from [#1227] (#1719) 2023-09-04 12:21:01 +02:00
Kim De Mey
7ae333ac8b
Add discv5 provided ENR directly to Portal protocol routing table (#1714)
* Add discv5 provided ENR directly to Portal protocol routing table

Previous version of getting the ENR from the discv5 routing table
would not work due to the order of first calling the talk protocol
handler and only after that the addEnr to the disc5 routing table.

Instead of changing this order, pass along the ENR directly to
avoid this additional getNode call.

* Still request ENR from discv5 if it wasn't passed via handshake
2023-08-30 20:38:15 +02:00
Kim De Mey
7aead6151e
Add utpTransfer bool to recursiveFindContent JSON-RPC result (#1710)
Rename portal protocol ContentInfo to ContentKV in the
process, due to duplicate types and simply better name
for this object.
2023-08-24 18:19:29 +02:00
Jordan Hrycaj
221e6c9e2f
Unified database frontend integration (#1670)
* Nimbus folder environment update

details:
* Integrated `CoreDbRef` for the sources in the `nimbus` sub-folder.
* The `nimbus` program does not compile yet as it needs the updates
  in the parallel `stateless` sub-folder.

* Stateless environment update

details:
* Integrated `CoreDbRef` for the sources in the `stateless` sub-folder.
* The `nimbus` program compiles now.

* Premix environment update

details:
* Integrated `CoreDbRef` for the sources in the `premix` sub-folder.

* Fluffy environment update

details:
* Integrated `CoreDbRef` for the sources in the `fluffy` sub-folder.

* Tools environment update

details:
* Integrated `CoreDbRef` for the sources in the `tools` sub-folder.

* Nodocker environment update

details:
* Integrated `CoreDbRef` for the sources in the
  `hive_integration/nodocker` sub-folder.

* Tests environment update

details:
* Integrated `CoreDbRef` for the sources in the `tests` sub-folder.
* The unit tests compile and run cleanly now.

* Generalise `CoreDbRef` to any `select_backend` supported database

why:
  Generalisation was just missed due to overcoming some compiler oddity
  which was tied to rocksdb for testing.

* Suppress compiler warning for `newChainDB()`

why:
  Warning was added to this function which must be wrapped so that
  any `CatchableError` is re-raised as `Defect`.

* Split off persistent `CoreDbRef` constructor into separate file

why:
  This allows to compile a memory only database version without linking
  the backend library.

* Use memory `CoreDbRef` database by default

detail:
 Persistent DB constructor needs to import `db/core_db/persistent

why:
 Most tests use memory DB anyway. This avoids linking `-lrocksdb` or
 any other backend by default.

* fix `toLegacyBackend()` availability check

why:
  got garbled after memory/persistent split.

* Clarify raw access to MPT for snap sync handler

why:
  Logically, `kvt` is not the raw access for the hexary trie (although
  this holds for the legacy database)
2023-08-04 12:10:09 +01:00
Kim De Mey
89b3e679e2
Add new portal beacon chain lc content encoding tests (#1636) 2023-07-08 17:01:33 +02:00
Kim De Mey
c7302b4ece
Compile Fluffy tests and fluffy beacon LC tests together (#1633)
This wins some time.

Also, remove and move some of the compile time defines to the
cfg files.
2023-07-06 18:06:44 +02:00
jangko
67aaf92c1d
bump submodules 2023-06-07 18:12:02 +07:00
Kim De Mey
bf21484f70
Add EL block proofs verifable from historical_summaries (#1588) 2023-05-30 07:02:39 +02:00
Kim De Mey
c9f3f82877
Add new Portal BlockBody type for Shanghai fork (#1589) 2023-05-30 06:56:54 +02:00
Kim De Mey
33fd8b1fae
Add historical_roots with proof + test (#1579) 2023-05-11 19:08:34 +02:00
Kim De Mey
3eafb15d7c
Make beacon LC adjustements to support capella in light clients (#1545)
- Bump nimbus-eth2 repo
- Beacon LC adjustements in fluffy
- Beacon LC adjustements in nimbus_verified_proxy
2023-04-13 18:40:01 +02:00
Kim De Mey
2d51196645
Add usage of header 14764013 for historya content test vectors (#1533) 2023-04-06 19:57:01 +02:00
Kim De Mey
0b960f323c
Add test vectors for history content block bodies and receipts (#1525) 2023-03-30 14:21:09 +02:00
Kim De Mey
e6964a029b
Remove unused, underdeveloped BridgeClient from Fluffy (#1520)
It is only (mostly) a skeleton, not further developed and not
used. It is unlikely the way forward either when further
developing the Portal state network.
2023-03-23 13:41:40 +01:00
Kim De Mey
8989141446
Renames on Portal beacon LC code (#1496) 2023-03-13 21:30:57 +01:00
Kim De Mey
06512409ea
Fix, rework and re-enable Portal beacon light client (#1481) 2023-02-26 19:18:03 +01:00
Adam Spitz
fad3ed64cf
Time based forking (#1465)
* Refactoring in preparation for time-based forking.

* Timestamp-based hard-fork-transition.

* Workaround SideEffect issue / compiler bug for both failing locations in Portal history code

---------

Co-authored-by: kdeme <kim.demey@gmail.com>
2023-02-16 12:40:07 +01:00
Kim De Mey
73e93f1f11
Reduce Nim 1.6 compiler warnings & hints
* Reduce Nim 1.6 compiler warnings/hints for Fluffy and Nimbus proxy

Mostly raises Defect removals, TaintedString removal and some
unnecessary imports.

Also updating the copyright years alongside.

* Further reduce Nim 1.6 compiler warnings/hints for Nimbus
2023-01-31 13:38:08 +01:00
Kim De Mey
12f66ae598
Move BlockHeaderWithProof content to content key selector 0 (#1379)
* Move BlockHeaderWithProof content to content key selector 0

- Remove as content type with content key selector 4
- Replace regular block header with BlockHeaderWithProof at
content key selector 0

* Apply blockHeader content key also to bridge

* Add tests for header with proof generation and verification
2023-01-20 23:04:58 +01:00
Kim De Mey
9efb40336b
Add possible historical_roots with proof structure (#1439)
This is an example of how the beacon state historical_roots could
be added with a proof to be able to provide on the network and
verify by means of proof against the state root.
2023-01-20 13:19:49 +01:00
Kim De Mey
90f91e5ebd
Reorganize eth data files parsing code (#1407) 2023-01-05 15:26:58 +01:00
Kim De Mey
624c87d8b3
Add experimental beacon chain block proofs for Portal network (#1376)
* Add experimental beacon chain block proofs for Portal network

* Add Caveat on proving recent blocks
2023-01-02 16:46:48 +01:00
Kim De Mey
c962bafd5a
Fix Portal Hive fails by correcting Portal history JSON RPC API (#1365)
* Fix Portal Hive fails by correcting Portal history JSON RPC API

- Field naming in discv5_nodeInfo
- Call naming of portal_historyStore
- Other: some proc to func adjustements
2022-12-13 19:22:36 +01:00
KonradStaniec
1cc6fc5cff
bump nimbus-eth2 and web3 (#1360)
- nimbus-eth2 exposes new callbacks in LightClient
- web3 exposes createAccessList call
2022-12-09 18:00:45 +01:00
Kim De Mey
c5ecba83cd
Refactor and clean-up of history network and eth rpc (#1357)
- Switch from using Option to Opt which allows for smoother
usage with already existing Result types
- With all moved to Opt, make more use of valueOr to avoid
too many if else clause indentation and unstead have a more
clear error path at each step
- Remove dead code, char limits, style guide, etc.
- Replace getEncodedKeyForContent with ContentKey.init
and use ContentKey.init for each type
2022-12-09 17:59:36 +01:00
jangko
94a94c5b65 implement better hardfork management 2022-12-02 13:51:42 +07:00
KonradStaniec
5374e7d37f
Initial implementation of getting range of lc updates (#1301)
* Initial implementation of getting lc updates
2022-11-18 10:00:06 +01:00
KonradStaniec
d75afd9f8a
Make portal network generic in db store and db get (#1290)
* Make portal network generic in db store and db get
2022-11-08 18:31:45 +01:00