Commit Graph

167 Commits

Author SHA1 Message Date
Kim De Mey c04f2477ac
Add version to fluffy cli flags and metrics (#1736)
Also update Grafana dashboard to incorperate the metrics and do
a confutils module bump.
2023-09-07 16:27:13 +02:00
Daniel Sobol 69a1a988b0
don't gossip data back to the node we rcvd it from [#1227] (#1719) 2023-09-04 12:21:01 +02:00
Kim De Mey 7ae333ac8b
Add discv5 provided ENR directly to Portal protocol routing table (#1714)
* Add discv5 provided ENR directly to Portal protocol routing table

Previous version of getting the ENR from the discv5 routing table
would not work due to the order of first calling the talk protocol
handler and only after that the addEnr to the disc5 routing table.

Instead of changing this order, pass along the ENR directly to
avoid this additional getNode call.

* Still request ENR from discv5 if it wasn't passed via handshake
2023-08-30 20:38:15 +02:00
Daniel Sobol 85134eb24b
add hidden --disable-poke option into fluffy [#1640] (#1711) 2023-08-30 10:01:00 +02:00
Kim De Mey 7aead6151e
Add utpTransfer bool to recursiveFindContent JSON-RPC result (#1710)
Rename portal protocol ContentInfo to ContentKV in the
process, due to duplicate types and simply better name
for this object.
2023-08-24 18:19:29 +02:00
Kim De Mey 6535b39e47
Add SszError to message decoding result (#1660)
This allows for improved error messages. Required changing the PortalResult from cstring to string for the error.
2023-08-23 18:03:09 +02:00
andri lim bdaeedb09f
rename data gas to blob gas (#1659)
* rename data gas to blob gas

* bump more submodules
* extend evmc tx_context with EIP-4844 blob_hashes
2023-08-04 19:43:30 +07:00
Jordan Hrycaj 221e6c9e2f
Unified database frontend integration (#1670)
* Nimbus folder environment update

details:
* Integrated `CoreDbRef` for the sources in the `nimbus` sub-folder.
* The `nimbus` program does not compile yet as it needs the updates
  in the parallel `stateless` sub-folder.

* Stateless environment update

details:
* Integrated `CoreDbRef` for the sources in the `stateless` sub-folder.
* The `nimbus` program compiles now.

* Premix environment update

details:
* Integrated `CoreDbRef` for the sources in the `premix` sub-folder.

* Fluffy environment update

details:
* Integrated `CoreDbRef` for the sources in the `fluffy` sub-folder.

* Tools environment update

details:
* Integrated `CoreDbRef` for the sources in the `tools` sub-folder.

* Nodocker environment update

details:
* Integrated `CoreDbRef` for the sources in the
  `hive_integration/nodocker` sub-folder.

* Tests environment update

details:
* Integrated `CoreDbRef` for the sources in the `tests` sub-folder.
* The unit tests compile and run cleanly now.

* Generalise `CoreDbRef` to any `select_backend` supported database

why:
  Generalisation was just missed due to overcoming some compiler oddity
  which was tied to rocksdb for testing.

* Suppress compiler warning for `newChainDB()`

why:
  Warning was added to this function which must be wrapped so that
  any `CatchableError` is re-raised as `Defect`.

* Split off persistent `CoreDbRef` constructor into separate file

why:
  This allows to compile a memory only database version without linking
  the backend library.

* Use memory `CoreDbRef` database by default

detail:
 Persistent DB constructor needs to import `db/core_db/persistent

why:
 Most tests use memory DB anyway. This avoids linking `-lrocksdb` or
 any other backend by default.

* fix `toLegacyBackend()` availability check

why:
  got garbled after memory/persistent split.

* Clarify raw access to MPT for snap sync handler

why:
  Logically, `kvt` is not the raw access for the hexary trie (although
  this holds for the legacy database)
2023-08-04 12:10:09 +01:00
Kim De Mey 89b3e679e2
Add new portal beacon chain lc content encoding tests (#1636) 2023-07-08 17:01:33 +02:00
Kim De Mey 8ebac4c878
Some more Options to Opt and similar changes (#1611)
Includes nim-eth bump.
2023-06-27 19:43:32 +02:00
Kim De Mey 2de9c95770
Remove a socket.close after destroy + bump nim-eth for uTP fixes (#1604) 2023-06-21 18:56:57 +02:00
Kim De Mey dd4da7491d
Remove yield usage from portal stream and protocol (#1602) 2023-06-15 14:32:33 +02:00
jangko 67aaf92c1d
bump submodules 2023-06-07 18:12:02 +07:00
Kim De Mey bf21484f70
Add EL block proofs verifable from historical_summaries (#1588) 2023-05-30 07:02:39 +02:00
Kim De Mey c9f3f82877
Add new Portal BlockBody type for Shanghai fork (#1589) 2023-05-30 06:56:54 +02:00
Kim De Mey 33fd8b1fae
Add historical_roots with proof + test (#1579) 2023-05-11 19:08:34 +02:00
Kim De Mey 408394a2bd
Bump nim-eth and remove unneeded Defect raises (#1575) 2023-05-10 18:04:35 +02:00
Kim De Mey 2ba974a7cd
Adjust portal FindContent JSON-RPC according to latest specs (#1566)
Also remove the FindContentFull unspecced JSON-RPC as the new
FindContent specification replaces this functionality basically.
2023-04-28 09:04:21 +02:00
Kim De Mey 3eafb15d7c
Make beacon LC adjustements to support capella in light clients (#1545)
- Bump nimbus-eth2 repo
- Beacon LC adjustements in fluffy
- Beacon LC adjustements in nimbus_verified_proxy
2023-04-13 18:40:01 +02:00
Kim De Mey 52d8308821
Some quick fixes for Shanghai support in Fluffy (#1541)
- Remove failing on withdrawalsRoot to allow Shanghai BlockHeader.
Still need to add real checks in block header / body validation.
- Add withdrawals array to Block object for JSON-RPC API
2023-04-12 14:20:52 +02:00
Kim De Mey 034d0e6305
Add exporting of headersWithProof and configurable web3 url (#1539) 2023-04-11 14:06:45 +02:00
Kim De Mey fdad9cfd00
Move content offered log after socket close wait (#1530)
To be clear that the socket also got closed before this.
Also explain that on socket close, the actual socket still waits
for an ACK.
2023-04-04 17:36:57 +02:00
Kim De Mey 9453d5bb3c
Add loop to history network to log status message on interval (#1526) 2023-03-31 18:52:21 +02:00
Kim De Mey 0b960f323c
Add test vectors for history content block bodies and receipts (#1525) 2023-03-30 14:21:09 +02:00
Kim De Mey 4c19a0e48e
Add support for injecting receipts into the Portal Network (#1523)
Allow for the Fluffy Portal bridge to inject receipts. This
requires a web3 endpoint to be provided, and currently only
Alchemy is supported due to the used JSON-RPC endpoint.
2023-03-28 15:50:23 +02:00
Kim De Mey 84d9b9889f
Add beacon chain lc content bridging to the Fluffy bridge (#1517)
Also delete the old, obsolete bridge that only stores content into
the db.
2023-03-23 08:49:16 +01:00
Kim De Mey 11fc2de060
Prepare Fluffy for beacon light client bridge (#1506) 2023-03-17 10:19:17 +01:00
Kim De Mey 8989141446
Renames on Portal beacon LC code (#1496) 2023-03-13 21:30:57 +01:00
Kim De Mey 06512409ea
Fix, rework and re-enable Portal beacon light client (#1481) 2023-02-26 19:18:03 +01:00
Adam Spitz fad3ed64cf
Time based forking (#1465)
* Refactoring in preparation for time-based forking.

* Timestamp-based hard-fork-transition.

* Workaround SideEffect issue / compiler bug for both failing locations in Portal history code

---------

Co-authored-by: kdeme <kim.demey@gmail.com>
2023-02-16 12:40:07 +01:00
jangko 0f065e91b5
bump submodules to further reduce compiler warnings 2023-02-02 09:29:02 +07:00
Kim De Mey 73e93f1f11
Reduce Nim 1.6 compiler warnings & hints
* Reduce Nim 1.6 compiler warnings/hints for Fluffy and Nimbus proxy

Mostly raises Defect removals, TaintedString removal and some
unnecessary imports.

Also updating the copyright years alongside.

* Further reduce Nim 1.6 compiler warnings/hints for Nimbus
2023-01-31 13:38:08 +01:00
Kim De Mey 12f66ae598
Move BlockHeaderWithProof content to content key selector 0 (#1379)
* Move BlockHeaderWithProof content to content key selector 0

- Remove as content type with content key selector 4
- Replace regular block header with BlockHeaderWithProof at
content key selector 0

* Apply blockHeader content key also to bridge

* Add tests for header with proof generation and verification
2023-01-20 23:04:58 +01:00
Kim De Mey 9efb40336b
Add possible historical_roots with proof structure (#1439)
This is an example of how the beacon state historical_roots could
be added with a proof to be able to provide on the network and
verify by means of proof against the state root.
2023-01-20 13:19:49 +01:00
Kim De Mey 3a079bf30e
Add first iteration of a fluffy beacon bridge for block data (#1437) 2023-01-19 09:33:11 +01:00
Kim De Mey a1163b4ade
Do not allow pre-merge headers without an AccumulatorProof (#1426) 2023-01-17 14:47:22 +01:00
Kim De Mey 52a2257922
Adjust Portal JSON-RPC Offer call and add Gossip call (#1418) 2023-01-17 14:47:10 +01:00
Kim De Mey 624c87d8b3
Add experimental beacon chain block proofs for Portal network (#1376)
* Add experimental beacon chain block proofs for Portal network

* Add Caveat on proving recent blocks
2023-01-02 16:46:48 +01:00
KonradStaniec 2bce04afc7
Add light client mode to fluffy (#1396) 2022-12-27 15:57:10 +01:00
KonradStaniec 270ce41d5c
Add light client bridge (#1386)
* Add light client bridge binary
2022-12-27 15:25:20 +01:00
Kim De Mey 733d7e5ceb
Add new portal JSON-RPC OfferReal as Offer is doing gossip now (#1387)
The portal_*Offer call was changed in the specs to actually do
gossip. Make it no longer possible to test purely an offer with
one node. Add OfferReal call for now until spec potentially gets
adjusted.

Also Add some Node information logging for FindContent and Offer
to be able to better debug failures and interoperability.
2022-12-16 17:47:52 +01:00
Kim De Mey 727b2445b7
Fix Portal history/state JSON-RPC API and add missing calls (#1380) 2022-12-16 08:49:18 +01:00
Kim De Mey 706effea2a
Disable canonical verification for blockheaders post-merge (#1377) 2022-12-15 14:13:26 +01:00
Etan Kissling 22338b7870 bump `nim-eth` for `eip4844` support
The `BlockHeader` structure in `nim-eth` was updated with support for
EIP-4844 (danksharding). To enable the `nim-eth` bump, the ingress of
`BlockHeader` structures has been hardened to reject headers that have
the new `excessDataGas` field until proper EIP4844 support exists.
https://github.com/status-im/nim-eth/pull/570
2022-12-14 11:04:13 +02:00
Kim De Mey c5ecba83cd
Refactor and clean-up of history network and eth rpc (#1357)
- Switch from using Option to Opt which allows for smoother
usage with already existing Result types
- With all moved to Opt, make more use of valueOr to avoid
too many if else clause indentation and unstead have a more
clear error path at each step
- Remove dead code, char limits, style guide, etc.
- Replace getEncodedKeyForContent with ContentKey.init
and use ContentKey.init for each type
2022-12-09 17:59:36 +01:00
KonradStaniec f4cacdfc6a
bump nimbus-eth2 (#1345)
* bump nimbus-eth2
2022-12-02 16:09:31 +01:00
Etan Kissling bc3f164b97
bump `nim-eth` for `withdrawalsRoot` support (#1326)
The `BlockHeader` structure in `nim-eth` was updated with support for
EIP-4895 (withdrawals). To enable the `nim-eth` bump, the ingress of
`BlockHeader` structures has been hardened to reject headers that have
the new `withdrawalsRoot` field until proper withdrawals support exists.
https://github.com/status-im/nim-eth/pull/562
2022-11-26 15:59:19 +01:00
KonradStaniec 5374e7d37f
Initial implementation of getting range of lc updates (#1301)
* Initial implementation of getting lc updates
2022-11-18 10:00:06 +01:00
KonradStaniec d75afd9f8a
Make portal network generic in db store and db get (#1290)
* Make portal network generic in db store and db get
2022-11-08 18:31:45 +01:00
Kim De Mey 36d430aaa2
Add headers with proof content type and use it for verification (#1281)
* Add headers with proof content type and use it for verification

- Add BlockHeaderWithProof content type & content
- Use BlockHeaderWithProof content to verify if chain data is
part of the canonical chain
- Adjust parser & seeder code to be able to seed these headers
with proof
- Adjust eth_data_exporter to be able to export custom header
ranges for which to build proofs (mostly for testing)

There is currently quite some ugliness & clean-up needed for which
a big part is due tos upporting both BlockHeader and
BlockHeaderWithProof on the network.

* Change accumulator proof to array / SSZ vector type

- Change accumulator proof to SSZ vector instead of SSZ list.
- Add and use general buildProof and buildHeaderWithProof func.

* Make the BlockHeaderWithProof an SSZ Union with None option

* Update portal-spec-tests to master commit
2022-11-04 09:27:01 +01:00