Commit Graph

275 Commits

Author SHA1 Message Date
Kim De Mey c04f2477ac
Add version to fluffy cli flags and metrics (#1736)
Also update Grafana dashboard to incorperate the metrics and do
a confutils module bump.
2023-09-07 16:27:13 +02:00
Daniel Sobol 69a1a988b0
don't gossip data back to the node we rcvd it from [#1227] (#1719) 2023-09-04 12:21:01 +02:00
Kim De Mey 7ae333ac8b
Add discv5 provided ENR directly to Portal protocol routing table (#1714)
* Add discv5 provided ENR directly to Portal protocol routing table

Previous version of getting the ENR from the discv5 routing table
would not work due to the order of first calling the talk protocol
handler and only after that the addEnr to the disc5 routing table.

Instead of changing this order, pass along the ENR directly to
avoid this additional getNode call.

* Still request ENR from discv5 if it wasn't passed via handshake
2023-08-30 20:38:15 +02:00
Daniel Sobol 85134eb24b
add hidden --disable-poke option into fluffy [#1640] (#1711) 2023-08-30 10:01:00 +02:00
Kim De Mey 7aead6151e
Add utpTransfer bool to recursiveFindContent JSON-RPC result (#1710)
Rename portal protocol ContentInfo to ContentKV in the
process, due to duplicate types and simply better name
for this object.
2023-08-24 18:19:29 +02:00
Kim De Mey 745ca026fe
Return false instead of exception on failed deleteEnr JSON-RPC (#1706) 2023-08-23 21:04:17 +02:00
Kim De Mey 0105e3e8cf
Let getEnr JSON-RPC also return local ENR (#1705) 2023-08-23 18:18:03 +02:00
Kim De Mey 6535b39e47
Add SszError to message decoding result (#1660)
This allows for improved error messages. Required changing the PortalResult from cstring to string for the error.
2023-08-23 18:03:09 +02:00
jangko c6e7ac1647
fix fluffy rpc_eth_api tx object initialization 2023-08-17 14:02:48 +07:00
andri lim bdaeedb09f
rename data gas to blob gas (#1659)
* rename data gas to blob gas

* bump more submodules
* extend evmc tx_context with EIP-4844 blob_hashes
2023-08-04 19:43:30 +07:00
Jordan Hrycaj 221e6c9e2f
Unified database frontend integration (#1670)
* Nimbus folder environment update

details:
* Integrated `CoreDbRef` for the sources in the `nimbus` sub-folder.
* The `nimbus` program does not compile yet as it needs the updates
  in the parallel `stateless` sub-folder.

* Stateless environment update

details:
* Integrated `CoreDbRef` for the sources in the `stateless` sub-folder.
* The `nimbus` program compiles now.

* Premix environment update

details:
* Integrated `CoreDbRef` for the sources in the `premix` sub-folder.

* Fluffy environment update

details:
* Integrated `CoreDbRef` for the sources in the `fluffy` sub-folder.

* Tools environment update

details:
* Integrated `CoreDbRef` for the sources in the `tools` sub-folder.

* Nodocker environment update

details:
* Integrated `CoreDbRef` for the sources in the
  `hive_integration/nodocker` sub-folder.

* Tests environment update

details:
* Integrated `CoreDbRef` for the sources in the `tests` sub-folder.
* The unit tests compile and run cleanly now.

* Generalise `CoreDbRef` to any `select_backend` supported database

why:
  Generalisation was just missed due to overcoming some compiler oddity
  which was tied to rocksdb for testing.

* Suppress compiler warning for `newChainDB()`

why:
  Warning was added to this function which must be wrapped so that
  any `CatchableError` is re-raised as `Defect`.

* Split off persistent `CoreDbRef` constructor into separate file

why:
  This allows to compile a memory only database version without linking
  the backend library.

* Use memory `CoreDbRef` database by default

detail:
 Persistent DB constructor needs to import `db/core_db/persistent

why:
 Most tests use memory DB anyway. This avoids linking `-lrocksdb` or
 any other backend by default.

* fix `toLegacyBackend()` availability check

why:
  got garbled after memory/persistent split.

* Clarify raw access to MPT for snap sync handler

why:
  Logically, `kvt` is not the raw access for the hexary trie (although
  this holds for the legacy database)
2023-08-04 12:10:09 +01:00
Kim De Mey 31cff910b1
Rename beacon_chain_bridge to beacon_lc_bridge (#1655) 2023-07-25 20:53:31 +02:00
Kim De Mey c20892f805
Allow starting beacon network in local testnet and add lc bridge (#1654)
Add the option to start the fluffy nodes with the beacon network
activated and allow for running a beacon lc bridge in the local
testnet script.
2023-07-25 14:52:44 +02:00
Kim De Mey edc87b702b
Launch Fluffy builds directly from make to avoid compile issue (#1646)
* Launch Fluffy builds directly from make to avoid compile issue

Without this change, builds on latest macos fails when ulimit is
not set to 1024. But it will still cause libbacktrace error to occur
when launching the binaries so it would be still advised to
set it to 1024.

* Fix fluffy local testnet for some macOS systems

And some additional improvements to the script + run the fluffy
nodes at INFO log-level to speed-up the testing time.

* Split up fluffy tests in separate targets

This way the two test binaries can be build and ran
concurrently.
2023-07-21 16:30:22 +02:00
Kim De Mey f950e0aac8
Remove unused variable in local testnet script (#1641) 2023-07-13 13:43:50 +02:00
Kim De Mey f6674acda2
Add some minor docs to indicate the existance of eth_data_exporter (#1637) 2023-07-10 19:35:28 +02:00
Kim De Mey 89b3e679e2
Add new portal beacon chain lc content encoding tests (#1636) 2023-07-08 17:01:33 +02:00
Kim De Mey c7302b4ece
Compile Fluffy tests and fluffy beacon LC tests together (#1633)
This wins some time.

Also, remove and move some of the compile time defines to the
cfg files.
2023-07-06 18:06:44 +02:00
Kim De Mey ab13e43db8
Add export of beacon LC data to eth_data_exporter (#1629)
* Add export of beacon LC data to eth_data_exporter

* Update content seeding docs with new eth_data_exporter cli
2023-07-05 20:17:03 +02:00
Kim De Mey b3d06c14b6
Build Fluffy tools individually through Makefile (#1626)
* Build Fluffy tools individually through Makefile

Before the builds were launched sequentually from a nimble task,
this would not allow for concurrent builds of the individual tools

* Some Makefile / nimble clean-up
2023-07-05 16:44:18 +02:00
Kim De Mey 389ef89632
Let Fluffy default join testnet via the testnet bootstrap nodes (#1620) 2023-06-28 23:53:27 +02:00
Kim De Mey d4bf1ffbcf
Update Fluffy bootstrap ENRs to nodes on new provider (#1619) 2023-06-28 21:29:48 +02:00
Kim De Mey 8ebac4c878
Some more Options to Opt and similar changes (#1611)
Includes nim-eth bump.
2023-06-27 19:43:32 +02:00
Kim De Mey 2de9c95770
Remove a socket.close after destroy + bump nim-eth for uTP fixes (#1604) 2023-06-21 18:56:57 +02:00
Kim De Mey dd4da7491d
Remove yield usage from portal stream and protocol (#1602) 2023-06-15 14:32:33 +02:00
jangko 67aaf92c1d
bump submodules 2023-06-07 18:12:02 +07:00
Kim De Mey bf21484f70
Add EL block proofs verifable from historical_summaries (#1588) 2023-05-30 07:02:39 +02:00
Kim De Mey c9f3f82877
Add new Portal BlockBody type for Shanghai fork (#1589) 2023-05-30 06:56:54 +02:00
Kim De Mey 33fd8b1fae
Add historical_roots with proof + test (#1579) 2023-05-11 19:08:34 +02:00
Kim De Mey 408394a2bd
Bump nim-eth and remove unneeded Defect raises (#1575) 2023-05-10 18:04:35 +02:00
Kim De Mey 9097548a10
Adjust discv5_updateNodeInfo JSON-RPC to specify key and value (#1569) 2023-05-09 14:34:01 +02:00
Kim De Mey 2ba974a7cd
Adjust portal FindContent JSON-RPC according to latest specs (#1566)
Also remove the FindContentFull unspecced JSON-RPC as the new
FindContent specification replaces this functionality basically.
2023-04-28 09:04:21 +02:00
Kim De Mey acc2d53b52
Further clean-up of Portal and discv5 JSON-RPC (#1565)
- Removal of TODOs fixed in specification instead
- Fix some other TODOs
- General clean-up such as to0xHex usage
2023-04-27 14:07:57 +02:00
Kim De Mey 21ef655b9c
Fix dataRadius in result of Portal ping JSON-RPC (#1553) 2023-04-21 17:00:55 +02:00
Kim De Mey 04137f1c98
Fix name of fluffy config file and disable log colors for Windows (#1551) 2023-04-20 12:58:56 +02:00
Kim De Mey ff90f4fd22
Improve logging and logging options in Fluffy (#1548)
* Improve logging and logging options in Fluffy

- Allow selection of log format, including:
  - JSON
  - automatic selection based on tty
- Allow log levels per topic configured on cli
2023-04-19 17:01:01 +02:00
Kim De Mey 3eafb15d7c
Make beacon LC adjustements to support capella in light clients (#1545)
- Bump nimbus-eth2 repo
- Beacon LC adjustements in fluffy
- Beacon LC adjustements in nimbus_verified_proxy
2023-04-13 18:40:01 +02:00
Kim De Mey b195f0893b
Minor naming adjustement to a discv5 JSON-RPC call (#1540) 2023-04-13 18:39:46 +02:00
Kim De Mey 9fde5c96e8
Prepare Portal bridge for Capella by adding withdrawalsRoot (#1543) 2023-04-13 00:02:38 +02:00
Kim De Mey 52d8308821
Some quick fixes for Shanghai support in Fluffy (#1541)
- Remove failing on withdrawalsRoot to allow Shanghai BlockHeader.
Still need to add real checks in block header / body validation.
- Add withdrawals array to Block object for JSON-RPC API
2023-04-12 14:20:52 +02:00
Kim De Mey 034d0e6305
Add exporting of headersWithProof and configurable web3 url (#1539) 2023-04-11 14:06:45 +02:00
Kim De Mey 2d51196645
Add usage of header 14764013 for historya content test vectors (#1533) 2023-04-06 19:57:01 +02:00
Kim De Mey fb81937708
Add a version to fluffy and add web3_clientVersion JSON-RPC (#1531) 2023-04-05 16:14:42 +02:00
Kim De Mey fdad9cfd00
Move content offered log after socket close wait (#1530)
To be clear that the socket also got closed before this.
Also explain that on socket close, the actual socket still waits
for an ACK.
2023-04-04 17:36:57 +02:00
Kim De Mey 9453d5bb3c
Add loop to history network to log status message on interval (#1526) 2023-03-31 18:52:21 +02:00
Kim De Mey 0b960f323c
Add test vectors for history content block bodies and receipts (#1525) 2023-03-30 14:21:09 +02:00
Kim De Mey 4c19a0e48e
Add support for injecting receipts into the Portal Network (#1523)
Allow for the Fluffy Portal bridge to inject receipts. This
requires a web3 endpoint to be provided, and currently only
Alchemy is supported due to the used JSON-RPC endpoint.
2023-03-28 15:50:23 +02:00
Kim De Mey e6964a029b
Remove unused, underdeveloped BridgeClient from Fluffy (#1520)
It is only (mostly) a skeleton, not further developed and not
used. It is unlikely the way forward either when further
developing the Portal state network.
2023-03-23 13:41:40 +01:00
Kim De Mey 84d9b9889f
Add beacon chain lc content bridging to the Fluffy bridge (#1517)
Also delete the old, obsolete bridge that only stores content into
the db.
2023-03-23 08:49:16 +01:00
Kim De Mey 249cabd64e
Portal JSON-RPC API fixes (#1511) 2023-03-20 17:25:34 +01:00