* Improve the tests of the local testnet
The local testnet test was rather flaky and would occasionally
fail. It has been made more robust by adding the ENRs directly
to the routing table instead of doing some random lookups.
Additionally, the amount of nodes were increased (=64), ip limits
configuration was added, and the bits-per-hop value was set to 1
in order to make the lookups more likely to hit the network
instead of only the local routing table.
Failure is obviously still possible to happen when sufficient
packets get lost. If this turns out to be the case with the current
amount of nodes, we might have to revise the testing strategy here.
* Disable lookup test for State network
Disable lookup test for State network due to issue with custom
distance function causing the lookup to not always converging
towards the target.
- Allow access to contentDB from portal wire protocol
- Use this to do the db.get in `handleFindContent` directly
- Use this to check the `contentKeys` list in `handleOffer`
* Change History content key to us SSZ Union and adjust tests
* Change slot to byteBE instead of LE
This is currently not specified in the Portal network
specifications, but we are using already BE for the actual content
key, so change it also here to remain consistent.
- Add portal network ping and findNodes JSON-RPC endpoints
- clean-up of some cli arguments
- Add protocol_interop.md document and adjust/fix other
documentation
Currently bootstrap nodes for discv5 and for the Portal nodes
were provided through separate cli arguments. This is however
confusing and cumbersome as typically when (currently) testing
a node will have both discv5 and the Portal networks enabled.
We merge them into one argument.
If a node happens not to support a Portal network, it will be
removed after message request failure.
Commit also contains additional clean-up and nim-eth bump.
* Rename FindNode to FindNodes as per spec
* Use consistently lower case starting camelCase for consts
Style guide nep1 allows for both CamelCase and camelCase for
consts, but we seem to use more often camelCase. Using this now
consistently.
* Change some procs to func
* Add resolve call for Portal networks
And:
- Refactor some code by adding a findNodeVerified call
- add the portal network lookup json-rpc call that uses resolve
- add usage of this lookup in the portal testnet tests
- Additional comments
* Let recordsFromBytes fail on invalid ENRs
This behaviour is more similar as how it is done in discovery v5
base layer.
- Add basic discv5 and portal json-rpc calls and activate them in
fluffy
- Renames in the rpc folder
- Add local testnet script and run this script in CI
- bump nim-eth
* Add SSZ Unions through case objects
* Add connection id content response test and improve other test vectors
* Implement content keys and ids for state network as per spec
Content keys case object is used so that it can be serialized and
deserialized as an SSZ Union.
* Let message Union in Portal wire protocol start at 0 as per new spec
- Search for the node on an incoming portal message and try to add
it to the routing table
- Don't rely on discv5 nodes for bootstrapping portal networks for
now.
- Attempt to repopulate the routing table when it is at 0 or drops
to 0.
* Add a basic ContentDB for Portal networks
* Use ContentDB in StateNetwork
* Avoid probably some form of sandwich problem by re-exporting kvstore_sqlite3 from content_db
* Allow for passing Portal specific bootstrap nodes
* Fix to also replaceNode when decodeMessage fails
* Add portal bootstrap node tests and reorder test cases
The Portal Network tests fail with the custom functions,
on one part due to the fact that the portal tests still
uses the xor functions, but also because the
`neighboursAtDistances` call appears to filter them out.
The reverse calculation might be off.
* Generalize netork layer for portal
* Make messages free from any content references
* Use portal network in main fluffy module
* Fix cli
* Use lookup in portal network
* Avoid using result
* Update readme with adjusted intro and dev updates link
* Remove pcre from prerequisites as it no longer is
* Change to hackmd doc as status notes seem not accessible
Kim De Mey
KonradStaniec
jangko