Commit Graph

304 Commits

Author SHA1 Message Date
Kim De Mey 5048c87679
First go at the Fluffy book using mkdocs as for nimbus.guide (#1842)
* First go at the Fluffy book using mkdocs as for nimbus.guide

* reorganize a little the index page

* Add small paragraph on the what in index page
2023-10-23 15:29:54 +02:00
Kim De Mey 9563beea56
Don't send the processed bitlist if contentQueue is full (#1835)
This would lead to sending an AcceptMessage with a zeroed out
connection id but still potentially bits set in the bitlist.
Requesting nodes could try to open up an uTP connection that
would never be accepted on the responder side.
2023-10-20 18:03:20 +02:00
Kim De Mey b7d0b06e79
Change some Fluffy related cli defaults (#1833) 2023-10-20 14:30:21 +02:00
Kim De Mey 436ef0e40e
Portal BeaconLightClientNetwork -> BeaconNetwork renaming (#1832) 2023-10-20 12:06:25 +02:00
Kim De Mey 14ee5f6820
Clean-up/refactor in beacon lc db (#1830) 2023-10-19 15:47:29 +02:00
Kim De Mey 6199ddc1c9
Better validation logs for Portal beacon network (#1823) 2023-10-18 23:29:20 +02:00
Kim De Mey 0472b75e23
Rework fluffy beacon lc db to allow storing processed content (#1821) 2023-10-18 16:59:44 +02:00
andri lim 04c7ed8ec9
Change EthTime from std.Time to distinct uint64 (#1820)
* Change EthTime from std.Time to distinct uint64

* Bump nimbus-eth2
2023-10-18 09:16:11 +07:00
Kim De Mey ca61a7009d
Bump submodules and related fixes (#1729)
* Bump submodules and related fixes

* Fix some warnings due to chronos and Nim bumps

* Seems macOS and Windows have different behavior
2023-10-17 14:19:50 +02:00
Kim De Mey 3444ffaf30
Update test to latest beacon LC test vectors (#1815) 2023-10-11 15:44:23 +02:00
Kim De Mey effe9c7ad8
Implement safe and finalized tag for eth_getBlockByNumber (#1814)
Only available when running the beacon LC and the portal beacon
network.
2023-10-09 17:49:15 +02:00
Kim De Mey 7b313ec1ac
Use signature slot instead of attested header slot (#1811)
Use signature slot instead of attested header slot in the
OptimisticUpdate content key.
2023-10-06 17:42:34 +02:00
Kim De Mey f72f02c88b
Remove optimistic slot from FinalityUpdate content key (#1810) 2023-10-06 15:46:53 +02:00
Kim De Mey 43b6014ffe
Further Portal beacon network improvements (#1808) 2023-10-05 19:29:39 +02:00
Kim De Mey f433297631
Improvements to portal beacon network (#1806)
- Adjust beacon lc db to avoid requirement of pruning optimistic
and finality updates
- Enable Portal config for beacon chain network to apply IP limits
configs on routing table
2023-10-05 08:57:45 +02:00
Daniel Sobol 20885f4ca5
don't ping nodes during revalidation if they were pinged recently (#1779) 2023-10-03 19:48:00 +03:00
Kim De Mey 492543a007
Fix and enable optimistic/finality update portal network test (#1794) 2023-10-02 11:41:11 +02:00
Kim De Mey 6253d2a456
Update some outdated information in Fluffy README.md (#1793) 2023-10-02 10:24:03 +02:00
Kim De Mey 2f2c5127ea
Further implementation to working prototype for Portal beacon LC (#1781)
- Link network gossip validation to LC processor validation
- QuickFix put/get for optimistic and finality updates
- Minor fixes and clean-up
- Improve bridge gossip for new LC Updates
- Adjust local testnet script to be able to locally test this
- Adjust test and skip broken test
2023-09-28 18:16:41 +02:00
Kim De Mey dd074d4df8
Load and use the stored ENR to allow incr of seqNum on updates (#1763) 2023-09-25 21:08:10 +02:00
Kim De Mey c405f79a84
Update the portal beacon lc manager (#1770)
Similar changes sd done on the libp2p version. To be able to have
same improvements and reuse more of the lc sync helpers code.
2023-09-22 18:42:02 +02:00
Daniel Sobol bd1a647987
Dockerfile for portal-hive development (#1759)
improves build time for a local dev container for a portal-hive test suite
2023-09-20 19:07:09 +03:00
Kim De Mey 53df21f000
Add clientInfo to ENR for easier testnet debugging (#1513)
* Add clientInfo to ENR for easier testnet debugging

* Remove git rev from ENR client info

This allows for a shorter ENR which allows more ENRs to be packed
in the Nodes reply. And for glados, only the client name is used
anyhow.
2023-09-20 12:19:40 +02:00
andri lim 7af9e3dc53
Refactor beacon skeleton (#1761)
* Fix unlisted exception due to recent modification in CoreDB interface

* Refactor beacon skeleton

* More unlisted exception fix
2023-09-19 11:52:28 +07:00
Kim De Mey 18b3ff3390
Correct the shortcut for ommershash verification post Shanghai (#1756)
* Correct the shortcut for ommershash verification post Shanghai

There was a faulty length check being done in the assumption that
the uncles field was already the list. However it is still the
RLP encoded data at that point, hence we check directly for the
RLP encoded value for empty list.

* Fix copy-paste error in withdrawal root verification

And print out the hashes in txs root and withdrawal root
verification
2023-09-18 14:50:07 +02:00
Kim De Mey a65b1c9062
Do not accept new offers if our contentQueue is full (#1753) 2023-09-16 16:06:11 +02:00
Kim De Mey cb4626b488
Initial implementation of Portal fluffy beacon chain bridge (#1751)
* Initial implementation of Portal fluffy beacon chain bridge

* Run windows ci for fluffy with -j1 to avoid OOM
2023-09-15 16:21:00 +02:00
andri lim 56215ed83f
Bump stint to v2.0: new array backend (#1747)
* Bump stint to v2.0: new array backend
2023-09-13 09:32:38 +07:00
Kim De Mey b0442dc41a
Add instance/container select for version panel in grafana dashboard (#1737) 2023-09-08 09:05:00 +02:00
Kim De Mey c04f2477ac
Add version to fluffy cli flags and metrics (#1736)
Also update Grafana dashboard to incorperate the metrics and do
a confutils module bump.
2023-09-07 16:27:13 +02:00
Daniel Sobol 69a1a988b0
don't gossip data back to the node we rcvd it from [#1227] (#1719) 2023-09-04 12:21:01 +02:00
Kim De Mey 7ae333ac8b
Add discv5 provided ENR directly to Portal protocol routing table (#1714)
* Add discv5 provided ENR directly to Portal protocol routing table

Previous version of getting the ENR from the discv5 routing table
would not work due to the order of first calling the talk protocol
handler and only after that the addEnr to the disc5 routing table.

Instead of changing this order, pass along the ENR directly to
avoid this additional getNode call.

* Still request ENR from discv5 if it wasn't passed via handshake
2023-08-30 20:38:15 +02:00
Daniel Sobol 85134eb24b
add hidden --disable-poke option into fluffy [#1640] (#1711) 2023-08-30 10:01:00 +02:00
Kim De Mey 7aead6151e
Add utpTransfer bool to recursiveFindContent JSON-RPC result (#1710)
Rename portal protocol ContentInfo to ContentKV in the
process, due to duplicate types and simply better name
for this object.
2023-08-24 18:19:29 +02:00
Kim De Mey 745ca026fe
Return false instead of exception on failed deleteEnr JSON-RPC (#1706) 2023-08-23 21:04:17 +02:00
Kim De Mey 0105e3e8cf
Let getEnr JSON-RPC also return local ENR (#1705) 2023-08-23 18:18:03 +02:00
Kim De Mey 6535b39e47
Add SszError to message decoding result (#1660)
This allows for improved error messages. Required changing the PortalResult from cstring to string for the error.
2023-08-23 18:03:09 +02:00
jangko c6e7ac1647
fix fluffy rpc_eth_api tx object initialization 2023-08-17 14:02:48 +07:00
andri lim bdaeedb09f
rename data gas to blob gas (#1659)
* rename data gas to blob gas

* bump more submodules
* extend evmc tx_context with EIP-4844 blob_hashes
2023-08-04 19:43:30 +07:00
Jordan Hrycaj 221e6c9e2f
Unified database frontend integration (#1670)
* Nimbus folder environment update

details:
* Integrated `CoreDbRef` for the sources in the `nimbus` sub-folder.
* The `nimbus` program does not compile yet as it needs the updates
  in the parallel `stateless` sub-folder.

* Stateless environment update

details:
* Integrated `CoreDbRef` for the sources in the `stateless` sub-folder.
* The `nimbus` program compiles now.

* Premix environment update

details:
* Integrated `CoreDbRef` for the sources in the `premix` sub-folder.

* Fluffy environment update

details:
* Integrated `CoreDbRef` for the sources in the `fluffy` sub-folder.

* Tools environment update

details:
* Integrated `CoreDbRef` for the sources in the `tools` sub-folder.

* Nodocker environment update

details:
* Integrated `CoreDbRef` for the sources in the
  `hive_integration/nodocker` sub-folder.

* Tests environment update

details:
* Integrated `CoreDbRef` for the sources in the `tests` sub-folder.
* The unit tests compile and run cleanly now.

* Generalise `CoreDbRef` to any `select_backend` supported database

why:
  Generalisation was just missed due to overcoming some compiler oddity
  which was tied to rocksdb for testing.

* Suppress compiler warning for `newChainDB()`

why:
  Warning was added to this function which must be wrapped so that
  any `CatchableError` is re-raised as `Defect`.

* Split off persistent `CoreDbRef` constructor into separate file

why:
  This allows to compile a memory only database version without linking
  the backend library.

* Use memory `CoreDbRef` database by default

detail:
 Persistent DB constructor needs to import `db/core_db/persistent

why:
 Most tests use memory DB anyway. This avoids linking `-lrocksdb` or
 any other backend by default.

* fix `toLegacyBackend()` availability check

why:
  got garbled after memory/persistent split.

* Clarify raw access to MPT for snap sync handler

why:
  Logically, `kvt` is not the raw access for the hexary trie (although
  this holds for the legacy database)
2023-08-04 12:10:09 +01:00
Kim De Mey 31cff910b1
Rename beacon_chain_bridge to beacon_lc_bridge (#1655) 2023-07-25 20:53:31 +02:00
Kim De Mey c20892f805
Allow starting beacon network in local testnet and add lc bridge (#1654)
Add the option to start the fluffy nodes with the beacon network
activated and allow for running a beacon lc bridge in the local
testnet script.
2023-07-25 14:52:44 +02:00
Kim De Mey edc87b702b
Launch Fluffy builds directly from make to avoid compile issue (#1646)
* Launch Fluffy builds directly from make to avoid compile issue

Without this change, builds on latest macos fails when ulimit is
not set to 1024. But it will still cause libbacktrace error to occur
when launching the binaries so it would be still advised to
set it to 1024.

* Fix fluffy local testnet for some macOS systems

And some additional improvements to the script + run the fluffy
nodes at INFO log-level to speed-up the testing time.

* Split up fluffy tests in separate targets

This way the two test binaries can be build and ran
concurrently.
2023-07-21 16:30:22 +02:00
Kim De Mey f950e0aac8
Remove unused variable in local testnet script (#1641) 2023-07-13 13:43:50 +02:00
Kim De Mey f6674acda2
Add some minor docs to indicate the existance of eth_data_exporter (#1637) 2023-07-10 19:35:28 +02:00
Kim De Mey 89b3e679e2
Add new portal beacon chain lc content encoding tests (#1636) 2023-07-08 17:01:33 +02:00
Kim De Mey c7302b4ece
Compile Fluffy tests and fluffy beacon LC tests together (#1633)
This wins some time.

Also, remove and move some of the compile time defines to the
cfg files.
2023-07-06 18:06:44 +02:00
Kim De Mey ab13e43db8
Add export of beacon LC data to eth_data_exporter (#1629)
* Add export of beacon LC data to eth_data_exporter

* Update content seeding docs with new eth_data_exporter cli
2023-07-05 20:17:03 +02:00
Kim De Mey b3d06c14b6
Build Fluffy tools individually through Makefile (#1626)
* Build Fluffy tools individually through Makefile

Before the builds were launched sequentually from a nimble task,
this would not allow for concurrent builds of the individual tools

* Some Makefile / nimble clean-up
2023-07-05 16:44:18 +02:00
Kim De Mey 389ef89632
Let Fluffy default join testnet via the testnet bootstrap nodes (#1620) 2023-06-28 23:53:27 +02:00