Commit Graph

225 Commits

Author SHA1 Message Date
Kim De Mey 11fc2de060
Prepare Fluffy for beacon light client bridge (#1506) 2023-03-17 10:19:17 +01:00
Kim De Mey 8989141446
Renames on Portal beacon LC code (#1496) 2023-03-13 21:30:57 +01:00
Adam Spitz d8a1adacaa
More work on withdrawals (#1482)
* Part of EIP-4895: add withdrawals processing to block processing.

* Refactoring: extracted the engine API handler bodies into procs.

Intending to implement the V2 versions next. (I need the bodies to be
in separate procs so that multiple versions can use them.)

* Working on Engine API changes for Shanghai.

* Updated nim-web3, resolved ambiguity in Hash256 type.

* Updated nim-eth3 to point to master, now that I've merged that.

* I'm confused about what's going on with engine_client.

But let's try resolving this Hash256 ambiguity.

* Still trying to fix this conflict with the Hash256 types.

* Does this work now that nimbus-eth2 has been updated?

* Corrected blockValue in getPayload responses back to UInt256.

c834f67a37

* Working on getting the withdrawals-related tests to pass.

* Fixing more of those Hash256 ambiguities.

(I'm not sure why the nim-web3 library introduced a conflicting type
named Hash256, but right now I just want to get this code to compile again.)

* Bumped a couple of libraries to fix some error messages.

* Needed to get "make fluffy-tools" to pass, too.

* Getting "make nimbus_verified_proxy" to build.
2023-03-09 18:40:55 -05:00
Kim De Mey 8b674bb4ae
Update Trin and Ultralight bootstrap nodes (#1489)
* Update to the latest Trin bootstrap nodes

* Update to the latest Ultralight bootstrap nodes
2023-03-08 16:36:12 +01:00
Kim De Mey 06512409ea
Fix, rework and re-enable Portal beacon light client (#1481) 2023-02-26 19:18:03 +01:00
Kim De Mey b64135b830
Bump nimbus-eth2 and fix beacon LC related changes (#1473)
* Bump nimbus-eth2 and fix beacon LC related changes

* Disable local_testnet test in CI
Latest version is no longer compatible with the old nightlies that are being downloaded for the test
2023-02-26 16:44:05 +01:00
Adam Spitz fad3ed64cf
Time based forking (#1465)
* Refactoring in preparation for time-based forking.

* Timestamp-based hard-fork-transition.

* Workaround SideEffect issue / compiler bug for both failing locations in Portal history code

---------

Co-authored-by: kdeme <kim.demey@gmail.com>
2023-02-16 12:40:07 +01:00
Kim De Mey 77135e7001
Avoid raw Exception raises by moving confutils config load (#1461) 2023-02-07 17:21:12 +01:00
jangko 0f065e91b5
bump submodules to further reduce compiler warnings 2023-02-02 09:29:02 +07:00
Kim De Mey 73e93f1f11
Reduce Nim 1.6 compiler warnings & hints
* Reduce Nim 1.6 compiler warnings/hints for Fluffy and Nimbus proxy

Mostly raises Defect removals, TaintedString removal and some
unnecessary imports.

Also updating the copyright years alongside.

* Further reduce Nim 1.6 compiler warnings/hints for Nimbus
2023-01-31 13:38:08 +01:00
Kim De Mey 8523d8b40e
Bump submodules and fix related issues (#1448)
* Updated to the latest nim-eth, nim-rocksdb, nim-web3

* Bump nimbus-eth2 module and fix related issues

Temporarily disabling Portal beacon light client network as it is
a lot of copy pasted code that did not yet take into account
forks. This will require a bigger rework and was not yet tested
in an actual network anyhow.

* More nimbus fixes after module bumps

---------

Co-authored-by: Adam Spitz <adamspitz@status.im>
Co-authored-by: jangko <jangko128@gmail.com>
2023-01-27 15:57:48 +01:00
Kim De Mey a669b51ec5
Bump Nim to 1.6 and resolve the related issues (#1445)
Two unresolved items currently:
- Three tests that are temporarily disabled as they fail in the
macro_assembler code, which seems to be due to an ambigious
identifier Stop (Ops and chronos ServerCommand enum).
- i386 CI disabled as it fails at Nim compilation already. Failed
tests where already ignored for this target.
2023-01-26 13:37:19 +01:00
Kim De Mey 12f66ae598
Move BlockHeaderWithProof content to content key selector 0 (#1379)
* Move BlockHeaderWithProof content to content key selector 0

- Remove as content type with content key selector 4
- Replace regular block header with BlockHeaderWithProof at
content key selector 0

* Apply blockHeader content key also to bridge

* Add tests for header with proof generation and verification
2023-01-20 23:04:58 +01:00
Kim De Mey 9efb40336b
Add possible historical_roots with proof structure (#1439)
This is an example of how the beacon state historical_roots could
be added with a proof to be able to provide on the network and
verify by means of proof against the state root.
2023-01-20 13:19:49 +01:00
Kim De Mey 3a079bf30e
Add first iteration of a fluffy beacon bridge for block data (#1437) 2023-01-19 09:33:11 +01:00
Kim De Mey a1163b4ade
Do not allow pre-merge headers without an AccumulatorProof (#1426) 2023-01-17 14:47:22 +01:00
Kim De Mey 52a2257922
Adjust Portal JSON-RPC Offer call and add Gossip call (#1418) 2023-01-17 14:47:10 +01:00
Kim De Mey 96776deed6
Update Fluffy docs on how to seed data (#1408) 2023-01-09 17:57:55 +01:00
Kim De Mey 90f91e5ebd
Reorganize eth data files parsing code (#1407) 2023-01-05 15:26:58 +01:00
Kim De Mey 624c87d8b3
Add experimental beacon chain block proofs for Portal network (#1376)
* Add experimental beacon chain block proofs for Portal network

* Add Caveat on proving recent blocks
2023-01-02 16:46:48 +01:00
Kim De Mey 1bfbf4f02f
Clarify Fluffy's docs and config (#1402) 2023-01-02 16:37:22 +01:00
KonradStaniec 2bce04afc7
Add light client mode to fluffy (#1396) 2022-12-27 15:57:10 +01:00
KonradStaniec 270ce41d5c
Add light client bridge (#1386)
* Add light client bridge binary
2022-12-27 15:25:20 +01:00
Kim De Mey 05ac755e84
Set disableMarchNative for Fluffy Dockerfile (#1391) 2022-12-19 13:19:50 +01:00
Kim De Mey 69204fffe9
Update two of the Fluffy bootstrap nodes for testnet (#1389) 2022-12-16 21:11:34 +01:00
Kim De Mey 591a1e7062
Add content verifier tool check data availability on the network (#1388) 2022-12-16 21:11:09 +01:00
Kim De Mey 733d7e5ceb
Add new portal JSON-RPC OfferReal as Offer is doing gossip now (#1387)
The portal_*Offer call was changed in the specs to actually do
gossip. Make it no longer possible to test purely an offer with
one node. Add OfferReal call for now until spec potentially gets
adjusted.

Also Add some Node information logging for FindContent and Offer
to be able to better debug failures and interoperability.
2022-12-16 17:47:52 +01:00
Kim De Mey e234a73b7e
Remove libpcre3-dev and librocksdb-dev deps from Fluffy Dockerfile (#1384) 2022-12-16 14:53:31 +01:00
Kim De Mey 3f74f084c3
Add LocalContent call to Portal JSON-RPC API (#1383) 2022-12-16 11:00:10 +01:00
Kim De Mey 727b2445b7
Fix Portal history/state JSON-RPC API and add missing calls (#1380) 2022-12-16 08:49:18 +01:00
Kim De Mey 095854107c
Fix Portal discv5 JSON-RPC API where it deviates from spec (#1378) 2022-12-15 16:24:23 +01:00
Kim De Mey 706effea2a
Disable canonical verification for blockheaders post-merge (#1377) 2022-12-15 14:13:26 +01:00
Etan Kissling 22338b7870 bump `nim-eth` for `eip4844` support
The `BlockHeader` structure in `nim-eth` was updated with support for
EIP-4844 (danksharding). To enable the `nim-eth` bump, the ingress of
`BlockHeader` structures has been hardened to reject headers that have
the new `excessDataGas` field until proper EIP4844 support exists.
https://github.com/status-im/nim-eth/pull/570
2022-12-14 11:04:13 +02:00
Kim De Mey c962bafd5a
Fix Portal Hive fails by correcting Portal history JSON RPC API (#1365)
* Fix Portal Hive fails by correcting Portal history JSON RPC API

- Field naming in discv5_nodeInfo
- Call naming of portal_historyStore
- Other: some proc to func adjustements
2022-12-13 19:22:36 +01:00
KonradStaniec 1cc6fc5cff
bump nimbus-eth2 and web3 (#1360)
- nimbus-eth2 exposes new callbacks in LightClient
- web3 exposes createAccessList call
2022-12-09 18:00:45 +01:00
Kim De Mey c5ecba83cd
Refactor and clean-up of history network and eth rpc (#1357)
- Switch from using Option to Opt which allows for smoother
usage with already existing Result types
- With all moved to Opt, make more use of valueOr to avoid
too many if else clause indentation and unstead have a more
clear error path at each step
- Remove dead code, char limits, style guide, etc.
- Replace getEncodedKeyForContent with ContentKey.init
and use ContentKey.init for each type
2022-12-09 17:59:36 +01:00
KonradStaniec f4cacdfc6a
bump nimbus-eth2 (#1345)
* bump nimbus-eth2
2022-12-02 16:09:31 +01:00
jangko 94a94c5b65 implement better hardfork management 2022-12-02 13:51:42 +07:00
Etan Kissling bc3f164b97
bump `nim-eth` for `withdrawalsRoot` support (#1326)
The `BlockHeader` structure in `nim-eth` was updated with support for
EIP-4895 (withdrawals). To enable the `nim-eth` bump, the ingress of
`BlockHeader` structures has been hardened to reject headers that have
the new `withdrawalsRoot` field until proper withdrawals support exists.
https://github.com/status-im/nim-eth/pull/562
2022-11-26 15:59:19 +01:00
KonradStaniec 5374e7d37f
Initial implementation of getting range of lc updates (#1301)
* Initial implementation of getting lc updates
2022-11-18 10:00:06 +01:00
KonradStaniec d75afd9f8a
Make portal network generic in db store and db get (#1290)
* Make portal network generic in db store and db get
2022-11-08 18:31:45 +01:00
Kim De Mey 36d430aaa2
Add headers with proof content type and use it for verification (#1281)
* Add headers with proof content type and use it for verification

- Add BlockHeaderWithProof content type & content
- Use BlockHeaderWithProof content to verify if chain data is
part of the canonical chain
- Adjust parser & seeder code to be able to seed these headers
with proof
- Adjust eth_data_exporter to be able to export custom header
ranges for which to build proofs (mostly for testing)

There is currently quite some ugliness & clean-up needed for which
a big part is due tos upporting both BlockHeader and
BlockHeaderWithProof on the network.

* Change accumulator proof to array / SSZ vector type

- Change accumulator proof to SSZ vector instead of SSZ list.
- Add and use general buildProof and buildHeaderWithProof func.

* Make the BlockHeaderWithProof an SSZ Union with None option

* Update portal-spec-tests to master commit
2022-11-04 09:27:01 +01:00
KonradStaniec 36a478afa7
Extend light client network (#1287) 2022-11-03 09:12:32 +01:00
KonradStaniec 1c1c6f56cc
Add initial implementation of portal powered light client (#1280)
* Add initial implementation of portal powered light client
2022-10-28 09:49:18 +02:00
KonradStaniec 306143f3d1
Add initial scaffold for portal lc network (#1277)
* Add initial scaffold for portal lc network
2022-10-24 14:16:40 +02:00
Kim De Mey 74a83c1229
Bump nim-eth and remove all now unneeded p2p related imports (#1273) 2022-10-20 10:34:59 +02:00
kdeme e6d8bb4f2f Optionally write all epoch accumulators when building accumulator
- Can write epoch accumulators to files now with eth_data_exporter
- RPC requests to gossip epoch accumulators now uses these files
instead of building on the fly
- Other build accumulator calls are adjusted and only used for
tests and thus moved to testing folder
2022-10-18 16:23:10 +03:00
Kim De Mey 09766ef283
Remove accumulator_db and minor clean-up in eth_data_exporter (#1268)
For accumulator building we now use intermediary header epoch
files which renders the use of a temporary db for this no longer
needed. Code was already no longer in use.
2022-10-18 10:53:26 +02:00
Kim De Mey 9430619272
Remove Accumulator as network content type and add it to binary (#1267)
Portal master accumulator was removed from the network specs as a
content type shared on the network, as since the merge this is
a finite accumulator (pre-merge only).
So in this PR the accumulator gets removed as network type and
gets instead baked into the library. Building it is done by
seperate tooling (eth_data_exporter).
Because of this a lot of extra code can be removed that was
located in history_network, content_db, portal_protocol, etc.

Also removed to option to build the accumulator at start-up
of fluffy as this takes several minutes making it not viable.
It can still be loaded from a provided file however.

The ssz accumulator file is for now stored in the recently
created portal-spec-tests repository.
2022-10-17 20:38:51 +02:00
Kim De Mey d667346834
Adjustments in eth_data_exporter related to accumulator generation (#1261)
- Add exportHeaders to write headers in e2s file per epoch
- Add verifyHeaders to verify decoding the above files
- Adjust exportAccumulatorData to use generated header epoch files
and write accumulator in SSZ (no hex)
- Add printAccumulatorData to print all root hashes
2022-10-14 08:57:17 +02:00