Commit Graph

57 Commits

Author SHA1 Message Date
Kim De Mey 63b0945b52
Use json rpc client to run tests on portal testnet (#899)
Tests have also been added & adjusted. Original ones were actually
failing but unnoticed due to mistake in port passed in curl command.
2021-12-03 09:51:25 +01:00
Kim De Mey ed04aad203
Fix fluffy discovery rpc tests after nim-json-rpc bump (#897) 2021-11-30 16:04:24 +01:00
Kim De Mey 0072089909
Implement part of the discv5 json-rpc calls (#894)
* Implement part of the discv5 json-rpc calls

* Use json marshalling for NodeId and Record everywhere
2021-11-29 10:39:37 +01:00
Kim De Mey 5f4ea7822c
Have one generic installPortalApiHandlers to be used for different Portal networks (#893) 2021-11-24 14:53:01 +01:00
Kim De Mey 77cc9b6215
Add support for passing bootstrap file and write local enr file (#892)
Also use this now in the local testnet script instead of hardcoded
values.
2021-11-24 12:12:25 +01:00
Kim De Mey 903350bdde
Add local testnet script and required json-rpc calls (#891)
- Add basic discv5 and portal json-rpc calls and activate them in
fluffy
- Renames in the rpc folder
- Add local testnet script and run this script in CI
- bump nim-eth
2021-11-24 08:45:55 +01:00
Kim De Mey 78f585399f
Bump submodules (#887)
* Bump submodules

* Fix missing import for state and history networks
2021-11-18 12:06:53 +01:00
Kim De Mey bdbd8e4c87
Add SSZ Unions through case objects (#882)
* Add SSZ Unions through case objects

* Add connection id content response test and improve other test vectors

* Implement content keys and ids for state network as per spec

Content keys case object is used so that it can be serialized and
deserialized as an SSZ Union.

* Let message Union in Portal wire protocol start at 0 as per new spec
2021-11-17 17:11:17 +01:00
Kim De Mey 414fdafab9
Improve the populating of the routing table (#886)
- Search for the node on an incoming portal message and try to add
it to the routing table
- Don't rely on discv5 nodes for bootstrapping portal networks for
now.
- Attempt to repopulate the routing table when it is at 0 or drops
to 0.
2021-11-16 17:50:08 +01:00
Kim De Mey 0ba3c682e4
Use SSZ code from nim-ssz-serialization module (#875) 2021-10-23 14:28:12 +02:00
Kim De Mey 02e8b2eb12
Clean up old unused code and unused imports (#877) 2021-10-22 21:36:53 +02:00
Kim De Mey 92dfb02ad7
Use ByteList.init instead of List 2048 (#876) 2021-10-22 17:49:23 +02:00
Kim De Mey 4b498356fc
Improve portal wire logging on sending/receiving messages (#874) 2021-10-22 11:03:00 +02:00
Kim De Mey 4b85c73f43
Update portal wire protocol ids to latest spec (#873) 2021-10-20 22:31:05 +02:00
Kim De Mey b77034c870
Update portal wire protocol according to latest spec changes (#860)
This is not complete, in order to be according to specification
two SSZ Union types need to be used.
2021-10-13 21:35:54 +02:00
Kim De Mey 447deb29ff
Add HistoryNetwork that uses Portal wire protocol (#856)
* Add HistoryNetwork that uses Portal wire protocol

* Further implement getContent and adjust naming of distance calc calls
2021-10-09 13:22:03 +02:00
KonradStaniec c62905db25
Re enable using custom distance function (#854) 2021-10-05 21:16:33 +02:00
Kim De Mey b58920f29f
Set config default descriptions (#852)
* Update fluffy config to set defaults descriptions

* Update portalcli config to set defaults descriptions
2021-09-29 13:58:55 +02:00
Kim De Mey 51626c5831
Add a basic ContentDB for portal networks (#848)
* Add a basic ContentDB for Portal networks

* Use ContentDB in StateNetwork

* Avoid probably some form of sandwich problem by re-exporting kvstore_sqlite3 from content_db
2021-09-28 19:58:41 +02:00
Kim De Mey 8e26acc4eb
Move tools and add common types (#846)
* Move some duplicated types to common_types

* Move portalcli to tools/ and update the READMEs
2021-09-24 17:18:54 +02:00
Kim De Mey 16a36453b5
Rework ContentId, ContentKey, GetContent to be more clear (#845)
* Small clarification in naming and note where GetContent is actually needed

* Rework contentId and contentKey
2021-09-24 11:22:07 +02:00
Kim De Mey 785a3b47b0
Allow for passing Portal specific bootstrap nodes (#844)
* Allow for passing Portal specific bootstrap nodes

* Fix to also replaceNode when decodeMessage fails

* Add portal bootstrap node tests and reorder test cases
2021-09-23 14:26:41 +02:00
Kim De Mey 6f6345a022
Move Portal wire protocol out of state specific code (#843)
* Allow for custom protocol id in portal wire protocol

* Move Portal wire protocol out of state specific code
2021-09-22 17:07:14 +02:00
Kim De Mey 11c120e400
Update wire protocol of state network spec (#829)
* Update wire protocol of state network spec

* BytesN should be SSZ Vector not List

* Add specific lists and bitlists for content keys
2021-09-22 11:28:04 +02:00
Kim De Mey 0135189e4c
Oops, isMainModule will not run of course for all_tests (#841) 2021-09-21 18:38:08 +02:00
Kim De Mey 1afdfdf2ff
Disable mixed usage of cliBuilder to fix incorrect test failure reporting (#836) 2021-09-21 14:35:41 +02:00
Kim De Mey 7a42d037f2
Disable the custom distance functions for Portal Protocol (#832)
The Portal Network tests fail with the custom functions,
on one part due to the fact that the portal tests still
uses the xor functions, but also because the
`neighboursAtDistances` call appears to filter them out.
The reverse calculation might be off.
2021-09-20 12:55:03 +02:00
jangko a3badea928
config: fix new config based on input from jamie and zahary 2021-09-18 17:34:51 +07:00
KonradStaniec 6192cd7dc1
Use custom distance function in state network (#831)
* Use custom distance calculator in state network
2021-09-16 16:13:36 +02:00
KonradStaniec 8f683bd318
bump nim-eth module (#828)
* bump-nim-eth

* fix breaking changes

* Increase result limit
2021-09-15 11:24:03 +02:00
KonradStaniec 58b11e683d
Implement custom distance function (#827)
* Implement custom distance function

* More docs about function equivalence
2021-09-14 13:36:55 +02:00
Kim De Mey d34681d48f
Little nitpick clean-up (#826) 2021-09-13 15:56:44 +02:00
jangko bf87c6e83c
tests: fixes fluffy test due to recent changes
- CustomGenesis -> CustomNetwork
- loadCustomGenesis -> loadCustomNetwork
- BaseChainDB constructor changes
2021-09-09 08:09:35 +07:00
Kim De Mey 07b2116f6b
Another nim-eth bump and API changes fix (#821) 2021-09-07 15:56:51 +02:00
KonradStaniec 2c620c007d
Fix bugs in lookup and add test case (#820) 2021-09-07 14:22:25 +02:00
Kim De Mey 832a50df6e
Bump nim-eth module and adjust for `distanceTo` API change (#819) 2021-09-06 17:04:21 +02:00
KonradStaniec a083fb30fc
Generalize network layer for portal (#814)
* Generalize netork layer for portal

* Make messages free from any content references

* Use portal network in main fluffy module

* Fix cli

* Use lookup in portal network

* Avoid using result
2021-09-03 10:57:19 +02:00
KonradStaniec df3e7bb368
Add content lookup function (#807)
* Add content lookup function

* Pr comments

Refactor contentLookup function
Add additional logginf for protocol breaches
in find content responses
2021-09-02 14:35:25 +02:00
KonradStaniec f0637cfbfc
Add rpc for local node info (#810)
* Add rpc for local node info

* Group let statement
2021-08-27 18:04:55 +02:00
KonradStaniec cff5c5afc4
Update just seen on message replies (#805) 2021-08-20 16:02:29 +02:00
KonradStaniec f4d9421836
Fix format of content message (#803) 2021-08-18 09:23:57 +02:00
KonradStaniec edd642c048
Move state network to separate folder (#800) 2021-08-17 10:58:40 +02:00
KonradStaniec 87b54b3321
Add history messages from spec (#792) 2021-08-10 14:13:52 +02:00
Kim De Mey a05f366ca2
Add monthly development updates link (#776)
* Update readme with adjusted intro and dev updates link

* Remove pcre from prerequisites as it no longer is

* Change to hackmd doc as status notes seem not accessible
2021-08-09 16:10:58 +02:00
KonradStaniec 63fa6d0865
Remove duplicated code (#783) 2021-08-05 16:04:29 +02:00
KonradStaniec 4b1fa050e9
Use websockets in proxy (#779) 2021-08-05 08:14:25 +02:00
Kim De Mey e2b50d2339
Add routing table and required lookup code to Portal protocol (#773)
* Add routing table and required lookup code to Portal protocol

* Enable Portal lookup and validate loops + minor adjustments
2021-07-30 21:19:03 +02:00
Kim De Mey 66532e8ad5
Use ssz vectors for the hashes and add enrs in the tests (#763) 2021-07-20 14:04:57 +02:00
Kim De Mey e37bafd47e
Add a first, simple, content network test (#760)
* Add a first, simple, content network test

* Use sqlite as nimbus_db_backend for fluffy tests

* Fix backend selection for sqlite
2021-07-15 15:12:33 +02:00
KonradStaniec 0fb0fc4680
[Minor] use proper constructor for proxy in fluffy (#759) 2021-07-13 15:49:53 +02:00