Commit Graph

283 Commits

Author SHA1 Message Date
Daniel Sobol bd1a647987
Dockerfile for portal-hive development (#1759)
improves build time for a local dev container for a portal-hive test suite
2023-09-20 19:07:09 +03:00
Kim De Mey 53df21f000
Add clientInfo to ENR for easier testnet debugging (#1513)
* Add clientInfo to ENR for easier testnet debugging

* Remove git rev from ENR client info

This allows for a shorter ENR which allows more ENRs to be packed
in the Nodes reply. And for glados, only the client name is used
anyhow.
2023-09-20 12:19:40 +02:00
andri lim 7af9e3dc53
Refactor beacon skeleton (#1761)
* Fix unlisted exception due to recent modification in CoreDB interface

* Refactor beacon skeleton

* More unlisted exception fix
2023-09-19 11:52:28 +07:00
Kim De Mey 18b3ff3390
Correct the shortcut for ommershash verification post Shanghai (#1756)
* Correct the shortcut for ommershash verification post Shanghai

There was a faulty length check being done in the assumption that
the uncles field was already the list. However it is still the
RLP encoded data at that point, hence we check directly for the
RLP encoded value for empty list.

* Fix copy-paste error in withdrawal root verification

And print out the hashes in txs root and withdrawal root
verification
2023-09-18 14:50:07 +02:00
Kim De Mey a65b1c9062
Do not accept new offers if our contentQueue is full (#1753) 2023-09-16 16:06:11 +02:00
Kim De Mey cb4626b488
Initial implementation of Portal fluffy beacon chain bridge (#1751)
* Initial implementation of Portal fluffy beacon chain bridge

* Run windows ci for fluffy with -j1 to avoid OOM
2023-09-15 16:21:00 +02:00
andri lim 56215ed83f
Bump stint to v2.0: new array backend (#1747)
* Bump stint to v2.0: new array backend
2023-09-13 09:32:38 +07:00
Kim De Mey b0442dc41a
Add instance/container select for version panel in grafana dashboard (#1737) 2023-09-08 09:05:00 +02:00
Kim De Mey c04f2477ac
Add version to fluffy cli flags and metrics (#1736)
Also update Grafana dashboard to incorperate the metrics and do
a confutils module bump.
2023-09-07 16:27:13 +02:00
Daniel Sobol 69a1a988b0
don't gossip data back to the node we rcvd it from [#1227] (#1719) 2023-09-04 12:21:01 +02:00
Kim De Mey 7ae333ac8b
Add discv5 provided ENR directly to Portal protocol routing table (#1714)
* Add discv5 provided ENR directly to Portal protocol routing table

Previous version of getting the ENR from the discv5 routing table
would not work due to the order of first calling the talk protocol
handler and only after that the addEnr to the disc5 routing table.

Instead of changing this order, pass along the ENR directly to
avoid this additional getNode call.

* Still request ENR from discv5 if it wasn't passed via handshake
2023-08-30 20:38:15 +02:00
Daniel Sobol 85134eb24b
add hidden --disable-poke option into fluffy [#1640] (#1711) 2023-08-30 10:01:00 +02:00
Kim De Mey 7aead6151e
Add utpTransfer bool to recursiveFindContent JSON-RPC result (#1710)
Rename portal protocol ContentInfo to ContentKV in the
process, due to duplicate types and simply better name
for this object.
2023-08-24 18:19:29 +02:00
Kim De Mey 745ca026fe
Return false instead of exception on failed deleteEnr JSON-RPC (#1706) 2023-08-23 21:04:17 +02:00
Kim De Mey 0105e3e8cf
Let getEnr JSON-RPC also return local ENR (#1705) 2023-08-23 18:18:03 +02:00
Kim De Mey 6535b39e47
Add SszError to message decoding result (#1660)
This allows for improved error messages. Required changing the PortalResult from cstring to string for the error.
2023-08-23 18:03:09 +02:00
jangko c6e7ac1647
fix fluffy rpc_eth_api tx object initialization 2023-08-17 14:02:48 +07:00
andri lim bdaeedb09f
rename data gas to blob gas (#1659)
* rename data gas to blob gas

* bump more submodules
* extend evmc tx_context with EIP-4844 blob_hashes
2023-08-04 19:43:30 +07:00
Jordan Hrycaj 221e6c9e2f
Unified database frontend integration (#1670)
* Nimbus folder environment update

details:
* Integrated `CoreDbRef` for the sources in the `nimbus` sub-folder.
* The `nimbus` program does not compile yet as it needs the updates
  in the parallel `stateless` sub-folder.

* Stateless environment update

details:
* Integrated `CoreDbRef` for the sources in the `stateless` sub-folder.
* The `nimbus` program compiles now.

* Premix environment update

details:
* Integrated `CoreDbRef` for the sources in the `premix` sub-folder.

* Fluffy environment update

details:
* Integrated `CoreDbRef` for the sources in the `fluffy` sub-folder.

* Tools environment update

details:
* Integrated `CoreDbRef` for the sources in the `tools` sub-folder.

* Nodocker environment update

details:
* Integrated `CoreDbRef` for the sources in the
  `hive_integration/nodocker` sub-folder.

* Tests environment update

details:
* Integrated `CoreDbRef` for the sources in the `tests` sub-folder.
* The unit tests compile and run cleanly now.

* Generalise `CoreDbRef` to any `select_backend` supported database

why:
  Generalisation was just missed due to overcoming some compiler oddity
  which was tied to rocksdb for testing.

* Suppress compiler warning for `newChainDB()`

why:
  Warning was added to this function which must be wrapped so that
  any `CatchableError` is re-raised as `Defect`.

* Split off persistent `CoreDbRef` constructor into separate file

why:
  This allows to compile a memory only database version without linking
  the backend library.

* Use memory `CoreDbRef` database by default

detail:
 Persistent DB constructor needs to import `db/core_db/persistent

why:
 Most tests use memory DB anyway. This avoids linking `-lrocksdb` or
 any other backend by default.

* fix `toLegacyBackend()` availability check

why:
  got garbled after memory/persistent split.

* Clarify raw access to MPT for snap sync handler

why:
  Logically, `kvt` is not the raw access for the hexary trie (although
  this holds for the legacy database)
2023-08-04 12:10:09 +01:00
Kim De Mey 31cff910b1
Rename beacon_chain_bridge to beacon_lc_bridge (#1655) 2023-07-25 20:53:31 +02:00
Kim De Mey c20892f805
Allow starting beacon network in local testnet and add lc bridge (#1654)
Add the option to start the fluffy nodes with the beacon network
activated and allow for running a beacon lc bridge in the local
testnet script.
2023-07-25 14:52:44 +02:00
Kim De Mey edc87b702b
Launch Fluffy builds directly from make to avoid compile issue (#1646)
* Launch Fluffy builds directly from make to avoid compile issue

Without this change, builds on latest macos fails when ulimit is
not set to 1024. But it will still cause libbacktrace error to occur
when launching the binaries so it would be still advised to
set it to 1024.

* Fix fluffy local testnet for some macOS systems

And some additional improvements to the script + run the fluffy
nodes at INFO log-level to speed-up the testing time.

* Split up fluffy tests in separate targets

This way the two test binaries can be build and ran
concurrently.
2023-07-21 16:30:22 +02:00
Kim De Mey f950e0aac8
Remove unused variable in local testnet script (#1641) 2023-07-13 13:43:50 +02:00
Kim De Mey f6674acda2
Add some minor docs to indicate the existance of eth_data_exporter (#1637) 2023-07-10 19:35:28 +02:00
Kim De Mey 89b3e679e2
Add new portal beacon chain lc content encoding tests (#1636) 2023-07-08 17:01:33 +02:00
Kim De Mey c7302b4ece
Compile Fluffy tests and fluffy beacon LC tests together (#1633)
This wins some time.

Also, remove and move some of the compile time defines to the
cfg files.
2023-07-06 18:06:44 +02:00
Kim De Mey ab13e43db8
Add export of beacon LC data to eth_data_exporter (#1629)
* Add export of beacon LC data to eth_data_exporter

* Update content seeding docs with new eth_data_exporter cli
2023-07-05 20:17:03 +02:00
Kim De Mey b3d06c14b6
Build Fluffy tools individually through Makefile (#1626)
* Build Fluffy tools individually through Makefile

Before the builds were launched sequentually from a nimble task,
this would not allow for concurrent builds of the individual tools

* Some Makefile / nimble clean-up
2023-07-05 16:44:18 +02:00
Kim De Mey 389ef89632
Let Fluffy default join testnet via the testnet bootstrap nodes (#1620) 2023-06-28 23:53:27 +02:00
Kim De Mey d4bf1ffbcf
Update Fluffy bootstrap ENRs to nodes on new provider (#1619) 2023-06-28 21:29:48 +02:00
Kim De Mey 8ebac4c878
Some more Options to Opt and similar changes (#1611)
Includes nim-eth bump.
2023-06-27 19:43:32 +02:00
Kim De Mey 2de9c95770
Remove a socket.close after destroy + bump nim-eth for uTP fixes (#1604) 2023-06-21 18:56:57 +02:00
Kim De Mey dd4da7491d
Remove yield usage from portal stream and protocol (#1602) 2023-06-15 14:32:33 +02:00
jangko 67aaf92c1d
bump submodules 2023-06-07 18:12:02 +07:00
Kim De Mey bf21484f70
Add EL block proofs verifable from historical_summaries (#1588) 2023-05-30 07:02:39 +02:00
Kim De Mey c9f3f82877
Add new Portal BlockBody type for Shanghai fork (#1589) 2023-05-30 06:56:54 +02:00
Kim De Mey 33fd8b1fae
Add historical_roots with proof + test (#1579) 2023-05-11 19:08:34 +02:00
Kim De Mey 408394a2bd
Bump nim-eth and remove unneeded Defect raises (#1575) 2023-05-10 18:04:35 +02:00
Kim De Mey 9097548a10
Adjust discv5_updateNodeInfo JSON-RPC to specify key and value (#1569) 2023-05-09 14:34:01 +02:00
Kim De Mey 2ba974a7cd
Adjust portal FindContent JSON-RPC according to latest specs (#1566)
Also remove the FindContentFull unspecced JSON-RPC as the new
FindContent specification replaces this functionality basically.
2023-04-28 09:04:21 +02:00
Kim De Mey acc2d53b52
Further clean-up of Portal and discv5 JSON-RPC (#1565)
- Removal of TODOs fixed in specification instead
- Fix some other TODOs
- General clean-up such as to0xHex usage
2023-04-27 14:07:57 +02:00
Kim De Mey 21ef655b9c
Fix dataRadius in result of Portal ping JSON-RPC (#1553) 2023-04-21 17:00:55 +02:00
Kim De Mey 04137f1c98
Fix name of fluffy config file and disable log colors for Windows (#1551) 2023-04-20 12:58:56 +02:00
Kim De Mey ff90f4fd22
Improve logging and logging options in Fluffy (#1548)
* Improve logging and logging options in Fluffy

- Allow selection of log format, including:
  - JSON
  - automatic selection based on tty
- Allow log levels per topic configured on cli
2023-04-19 17:01:01 +02:00
Kim De Mey 3eafb15d7c
Make beacon LC adjustements to support capella in light clients (#1545)
- Bump nimbus-eth2 repo
- Beacon LC adjustements in fluffy
- Beacon LC adjustements in nimbus_verified_proxy
2023-04-13 18:40:01 +02:00
Kim De Mey b195f0893b
Minor naming adjustement to a discv5 JSON-RPC call (#1540) 2023-04-13 18:39:46 +02:00
Kim De Mey 9fde5c96e8
Prepare Portal bridge for Capella by adding withdrawalsRoot (#1543) 2023-04-13 00:02:38 +02:00
Kim De Mey 52d8308821
Some quick fixes for Shanghai support in Fluffy (#1541)
- Remove failing on withdrawalsRoot to allow Shanghai BlockHeader.
Still need to add real checks in block header / body validation.
- Add withdrawals array to Block object for JSON-RPC API
2023-04-12 14:20:52 +02:00
Kim De Mey 034d0e6305
Add exporting of headersWithProof and configurable web3 url (#1539) 2023-04-11 14:06:45 +02:00
Kim De Mey 2d51196645
Add usage of header 14764013 for historya content test vectors (#1533) 2023-04-06 19:57:01 +02:00