a8fdf8ec36
Improve utp test app ( #947 )
...
* Improve utp test app
2022-01-31 18:40:00 +01:00
990846135a
Bump nim-eth ( #941 )
...
* Bump nim-eth
2022-01-25 10:19:16 +01:00
d2f56463ce
Remove the use of ByteList in PortalStream and for FoundContent ( #937 )
2022-01-20 21:21:20 +01:00
3909675f29
Use socket destroy instead of close to avoid extra FIN send ( #935 )
...
Also add missing push raises Defect
2022-01-20 15:04:23 +01:00
cccd1e4640
Add tooling necessary to improve utp-testing ( #931 )
...
* Add tooling necessary to improve utp-testing
2022-01-19 16:06:23 +01:00
69b0cc07b6
Add data radius cache per portal protocol ( #930 )
2022-01-19 15:56:14 +01:00
14dd763900
Add populate history db option to fluffy ( #929 )
2022-01-18 15:08:02 +01:00
81ebfd2b2a
Add Portal protocol config and add config options to fluffy cli ( #928 )
2022-01-18 09:01:22 +01:00
f19a64a2fe
Add uTP over discv5 in portal wire protocol ( #922 )
...
* Add uTP over discv5 in portal wire protocol
2022-01-14 16:07:14 +01:00
79647e5580
Allow access to contentDB from portal wire protocol ( #920 )
...
- Allow access to contentDB from portal wire protocol
- Use this to do the db.get in `handleFindContent` directly
- Use this to check the `contentKeys` list in `handleOffer`
2022-01-06 09:06:05 +01:00
a8640fe57c
History content keys ( #919 )
...
* Change History content key to us SSZ Union and adjust tests
* Change slot to byteBE instead of LE
This is currently not specified in the Portal network
specifications, but we are using already BE for the actual content
key, so change it also here to remain consistent.
2022-01-05 09:49:49 +01:00
f4ca598c03
Some version of the findContent JSON-RPC proc ( #918 )
...
Also includes fixing instantiation of the history network json-rpc
API handles.
2021-12-20 11:57:55 +01:00
796787e0e5
Add state and history content key tests ( #916 )
2021-12-20 09:53:00 +01:00
092d946c72
Add state ContentKey test and adjust history ContentKey test ( #914 )
2021-12-15 08:48:56 +01:00
f3d0e97997
Add Portal protocol testing docs and more ( #912 )
...
- Add portal network ping and findNodes JSON-RPC endpoints
- clean-up of some cli arguments
- Add protocol_interop.md document and adjust/fix other
documentation
2021-12-13 14:12:51 +01:00
70625bc02c
Use one bootstrap nodes argument for discv5 and Portal ( #911 )
...
Currently bootstrap nodes for discv5 and for the Portal nodes
were provided through separate cli arguments. This is however
confusing and cumbersome as typically when (currently) testing
a node will have both discv5 and the Portal networks enabled.
We merge them into one argument.
If a node happens not to support a Portal network, it will be
removed after message request failure.
Commit also contains additional clean-up and nim-eth bump.
2021-12-13 09:06:29 +01:00
10a3946194
Bit of cleanup ( #903 )
...
* Rename FindNode to FindNodes as per spec
* Use consistently lower case starting camelCase for consts
Style guide nep1 allows for both CamelCase and camelCase for
consts, but we seem to use more often camelCase. Using this now
consistently.
* Change some procs to func
2021-12-08 11:54:22 +01:00
2ea35ef82a
Add resolve call for Portal networks ( #902 )
...
* Add resolve call for Portal networks
And:
- Refactor some code by adding a findNodeVerified call
- add the portal network lookup json-rpc call that uses resolve
- add usage of this lookup in the portal testnet tests
- Additional comments
* Let recordsFromBytes fail on invalid ENRs
This behaviour is more similar as how it is done in discovery v5
base layer.
2021-12-08 09:26:31 +01:00
63b0945b52
Use json rpc client to run tests on portal testnet ( #899 )
...
Tests have also been added & adjusted. Original ones were actually
failing but unnoticed due to mistake in port passed in curl command.
2021-12-03 09:51:25 +01:00
ed04aad203
Fix fluffy discovery rpc tests after nim-json-rpc bump ( #897 )
2021-11-30 16:04:24 +01:00
0072089909
Implement part of the discv5 json-rpc calls ( #894 )
...
* Implement part of the discv5 json-rpc calls
* Use json marshalling for NodeId and Record everywhere
2021-11-29 10:39:37 +01:00
5f4ea7822c
Have one generic installPortalApiHandlers to be used for different Portal networks ( #893 )
2021-11-24 14:53:01 +01:00
77cc9b6215
Add support for passing bootstrap file and write local enr file ( #892 )
...
Also use this now in the local testnet script instead of hardcoded
values.
2021-11-24 12:12:25 +01:00
903350bdde
Add local testnet script and required json-rpc calls ( #891 )
...
- Add basic discv5 and portal json-rpc calls and activate them in
fluffy
- Renames in the rpc folder
- Add local testnet script and run this script in CI
- bump nim-eth
2021-11-24 08:45:55 +01:00
78f585399f
Bump submodules ( #887 )
...
* Bump submodules
* Fix missing import for state and history networks
2021-11-18 12:06:53 +01:00
bdbd8e4c87
Add SSZ Unions through case objects ( #882 )
...
* Add SSZ Unions through case objects
* Add connection id content response test and improve other test vectors
* Implement content keys and ids for state network as per spec
Content keys case object is used so that it can be serialized and
deserialized as an SSZ Union.
* Let message Union in Portal wire protocol start at 0 as per new spec
2021-11-17 17:11:17 +01:00
414fdafab9
Improve the populating of the routing table ( #886 )
...
- Search for the node on an incoming portal message and try to add
it to the routing table
- Don't rely on discv5 nodes for bootstrapping portal networks for
now.
- Attempt to repopulate the routing table when it is at 0 or drops
to 0.
2021-11-16 17:50:08 +01:00
0ba3c682e4
Use SSZ code from nim-ssz-serialization module ( #875 )
2021-10-23 14:28:12 +02:00
02e8b2eb12
Clean up old unused code and unused imports ( #877 )
2021-10-22 21:36:53 +02:00
92dfb02ad7
Use ByteList.init instead of List 2048 ( #876 )
2021-10-22 17:49:23 +02:00
4b498356fc
Improve portal wire logging on sending/receiving messages ( #874 )
2021-10-22 11:03:00 +02:00
4b85c73f43
Update portal wire protocol ids to latest spec ( #873 )
2021-10-20 22:31:05 +02:00
b77034c870
Update portal wire protocol according to latest spec changes ( #860 )
...
This is not complete, in order to be according to specification
two SSZ Union types need to be used.
2021-10-13 21:35:54 +02:00
447deb29ff
Add HistoryNetwork that uses Portal wire protocol ( #856 )
...
* Add HistoryNetwork that uses Portal wire protocol
* Further implement getContent and adjust naming of distance calc calls
2021-10-09 13:22:03 +02:00
c62905db25
Re enable using custom distance function ( #854 )
2021-10-05 21:16:33 +02:00
b58920f29f
Set config default descriptions ( #852 )
...
* Update fluffy config to set defaults descriptions
* Update portalcli config to set defaults descriptions
2021-09-29 13:58:55 +02:00
51626c5831
Add a basic ContentDB for portal networks ( #848 )
...
* Add a basic ContentDB for Portal networks
* Use ContentDB in StateNetwork
* Avoid probably some form of sandwich problem by re-exporting kvstore_sqlite3 from content_db
2021-09-28 19:58:41 +02:00
8e26acc4eb
Move tools and add common types ( #846 )
...
* Move some duplicated types to common_types
* Move portalcli to tools/ and update the READMEs
2021-09-24 17:18:54 +02:00
16a36453b5
Rework ContentId, ContentKey, GetContent to be more clear ( #845 )
...
* Small clarification in naming and note where GetContent is actually needed
* Rework contentId and contentKey
2021-09-24 11:22:07 +02:00
785a3b47b0
Allow for passing Portal specific bootstrap nodes ( #844 )
...
* Allow for passing Portal specific bootstrap nodes
* Fix to also replaceNode when decodeMessage fails
* Add portal bootstrap node tests and reorder test cases
2021-09-23 14:26:41 +02:00
6f6345a022
Move Portal wire protocol out of state specific code ( #843 )
...
* Allow for custom protocol id in portal wire protocol
* Move Portal wire protocol out of state specific code
2021-09-22 17:07:14 +02:00
11c120e400
Update wire protocol of state network spec ( #829 )
...
* Update wire protocol of state network spec
* BytesN should be SSZ Vector not List
* Add specific lists and bitlists for content keys
2021-09-22 11:28:04 +02:00
0135189e4c
Oops, isMainModule will not run of course for all_tests ( #841 )
2021-09-21 18:38:08 +02:00
1afdfdf2ff
Disable mixed usage of cliBuilder to fix incorrect test failure reporting ( #836 )
2021-09-21 14:35:41 +02:00
7a42d037f2
Disable the custom distance functions for Portal Protocol ( #832 )
...
The Portal Network tests fail with the custom functions,
on one part due to the fact that the portal tests still
uses the xor functions, but also because the
`neighboursAtDistances` call appears to filter them out.
The reverse calculation might be off.
2021-09-20 12:55:03 +02:00
a3badea928
config: fix new config based on input from jamie and zahary
2021-09-18 17:34:51 +07:00
6192cd7dc1
Use custom distance function in state network ( #831 )
...
* Use custom distance calculator in state network
2021-09-16 16:13:36 +02:00
8f683bd318
bump nim-eth module ( #828 )
...
* bump-nim-eth
* fix breaking changes
* Increase result limit
2021-09-15 11:24:03 +02:00
58b11e683d
Implement custom distance function ( #827 )
...
* Implement custom distance function
* More docs about function equivalence
2021-09-14 13:36:55 +02:00
d34681d48f
Little nitpick clean-up ( #826 )
2021-09-13 15:56:44 +02:00
bf87c6e83c
tests: fixes fluffy test due to recent changes
...
- CustomGenesis -> CustomNetwork
- loadCustomGenesis -> loadCustomNetwork
- BaseChainDB constructor changes
2021-09-09 08:09:35 +07:00
07b2116f6b
Another nim-eth bump and API changes fix ( #821 )
2021-09-07 15:56:51 +02:00
2c620c007d
Fix bugs in lookup and add test case ( #820 )
2021-09-07 14:22:25 +02:00
832a50df6e
Bump nim-eth module and adjust for `distanceTo` API change ( #819 )
2021-09-06 17:04:21 +02:00
a083fb30fc
Generalize network layer for portal ( #814 )
...
* Generalize netork layer for portal
* Make messages free from any content references
* Use portal network in main fluffy module
* Fix cli
* Use lookup in portal network
* Avoid using result
2021-09-03 10:57:19 +02:00
df3e7bb368
Add content lookup function ( #807 )
...
* Add content lookup function
* Pr comments
Refactor contentLookup function
Add additional logginf for protocol breaches
in find content responses
2021-09-02 14:35:25 +02:00
f0637cfbfc
Add rpc for local node info ( #810 )
...
* Add rpc for local node info
* Group let statement
2021-08-27 18:04:55 +02:00
cff5c5afc4
Update just seen on message replies ( #805 )
2021-08-20 16:02:29 +02:00
f4d9421836
Fix format of content message ( #803 )
2021-08-18 09:23:57 +02:00
edd642c048
Move state network to separate folder ( #800 )
2021-08-17 10:58:40 +02:00
87b54b3321
Add history messages from spec ( #792 )
2021-08-10 14:13:52 +02:00
a05f366ca2
Add monthly development updates link ( #776 )
...
* Update readme with adjusted intro and dev updates link
* Remove pcre from prerequisites as it no longer is
* Change to hackmd doc as status notes seem not accessible
2021-08-09 16:10:58 +02:00
63fa6d0865
Remove duplicated code ( #783 )
2021-08-05 16:04:29 +02:00
4b1fa050e9
Use websockets in proxy ( #779 )
2021-08-05 08:14:25 +02:00
e2b50d2339
Add routing table and required lookup code to Portal protocol ( #773 )
...
* Add routing table and required lookup code to Portal protocol
* Enable Portal lookup and validate loops + minor adjustments
2021-07-30 21:19:03 +02:00
66532e8ad5
Use ssz vectors for the hashes and add enrs in the tests ( #763 )
2021-07-20 14:04:57 +02:00
e37bafd47e
Add a first, simple, content network test ( #760 )
...
* Add a first, simple, content network test
* Use sqlite as nimbus_db_backend for fluffy tests
* Fix backend selection for sqlite
2021-07-15 15:12:33 +02:00
0fb0fc4680
[Minor] use proper constructor for proxy in fluffy ( #759 )
2021-07-13 15:49:53 +02:00
25522b2c24
Further implement Portal Nodes and FoundContent messages ( #757 )
2021-07-13 15:15:33 +02:00
c7569c2247
Add/update ssz calls for fluffy ( #750 )
2021-07-09 16:15:10 +02:00
424fd405e9
Move Portal wire protocol from nim-eth to fluffy ( #749 )
...
* Move Portal wire protocol from nim-eth to fluffy
* Rename fluffy make targets
* Move all_fluffy_tests a folder down
2021-07-09 13:34:16 +02:00
1c5c2cba8b
[FEATURE] Use rpc proxy ( #747 )
2021-07-07 14:13:27 +02:00
a59a2c61b6
[FEATURE] Add support for handling experimental api call ( #746 )
...
* [FEATURE] Add support for handling experimental api call
2021-07-07 11:04:18 +02:00
65ea8a12a9
[FEATURE] Stub for bridge client ( #739 )
...
* [FEATURE] Stub for bridge client
* [FEATURE] Add client to fluffy runner
* [FEATURE] Fix params in logs
* [FEATURE] Rename config option
2021-07-02 07:30:48 +02:00
f8b3922eb5
Change name from nlpn to fluffy ( #734 )
2021-06-28 17:53:13 +02:00
KonradStaniec
Kim De Mey
jangko