47 Commits

Author SHA1 Message Date
Csaba Kiraly
02bc12e639
change node seen flag to an exponential moving average
keep defaults as before

Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>

# Conflicts:
#	codexdht/private/eth/p2p/discoveryv5/node.nim
#	codexdht/private/eth/p2p/discoveryv5/routing_table.nim
2024-10-14 15:33:29 +02:00
Ben Bierens
5f38fd9570
GCC-14 (#98)
* bumps bearssl

* updates version of bearssl in lockfiles

* fixes that checksum

* attempt to bump various dependencies

* updates asynctest version tag

* asynctest sha

* bumps to working version of nim-datastore

* adjusts asynctest imports for chronos

* chronos checksum

* checksum for datastore

* libp2p version tag

* libp2p checksum

* moves libp2p from codex-branch to latest master

* libp2p checksum

* splits the test dependencies from the dev dependencies (example nim-ethers)

* sets path

* pathing in tests

* oops wrong version

* adds build.nims to installfiles for test module

* attempt to fix import paths

* bumps nim-datastore

* datastore checksum

* greatly simplify CI

* fixes asynctest import

* builds parallel tests before running

* bumps datastore

* turns nim-stable back off

* pins nim-datastore version

* bumps checkout to v4

* Review comment by Mark

Co-authored-by: markspanbroek <mark@spanbroek.net>

* Review comment by Mark

Co-authored-by: markspanbroek <mark@spanbroek.net>

---------

Co-authored-by: markspanbroek <mark@spanbroek.net>
2024-08-20 11:04:48 +02:00
Giuliano Mega
63822e8356
Update nim-codex-dht to Chronos V4 (#90)
Update nim-codex-dht to Chronos v4
2024-05-23 17:49:44 -03:00
Dmitriy Ryajov
dd4985435a
Fix timeout and delete (#86)
* use unix time for ttl

* don't remove all entries on peer removal

* cleanup questionable tuple destructure

* ignore vscode

* fix endians decoding

* allow removing by peerId

* invalidate cache by peerId on remove

* update test
2023-11-17 14:01:16 -08:00
Csaba Kiraly
66116b9bf6
Fix: queue messages when there is no encryption key (#83)
* encodeMessagePacket: expose haskey

encodeMessagePacket checks for session and behaves differently
based on that. Exposing this difference in behavior.

Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>

* improve tracing of message exchange

run e.g. as
```
nim c -r -d:debug -d:chronicles_enabled=on -d:chronicles_log_level=TRACE -d:chronicles_sinks=textlines[nocolors,stdout] tests/dht/test_providers.nim >err
```

Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>

* add debug on Handshake timeour

Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>

* queue messages during handshake and send later

If a handshake was already in progress, messages were dropped.
Instead of this, it is better to queue these and send as soon
as the handshake is finished and thus the encryption key is known.

Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>

* rename handshakeInProgress to keyexchangeInProgress

Handshake is also a name of a message, which makes previous
name less clear.

Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>

* keyexchangeInProgress: do not remove on handshake received

This is the wrong direction, not needed

Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>

---------

Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2023-11-17 11:18:48 -08:00
Jaremy Creechley
60dc4e764c
Fix changes from stint (#81)
* some formatting tweaks to make errors easier to grok

* stint removed overloads for regular ints - use stew versions instead

* various name style fixes

* ignore vscode stuff

* revert style changes

* revert unneeded var rename changes

---------

Co-authored-by: Dmitriy Ryajov <dryajov@gmail.com>
2023-11-17 11:08:45 -08:00
Jaremy Creechley
b585290397
Bump deps (#77)
Updates all Status IM and Codex Storage deps to the latest except for nim-stint which is held back due to some compiler issue. 

* fix nimble name
* don't override nimble
* update all deps
* import nimble.lock and fix urls
* don't forget nim
* bump to chronos with async notifications
2023-07-21 15:29:38 -07:00
Dmitriy Ryajov
9ae0bfb1c3
Fix nimble install (#78)
* make encryption scheme explicit

* supress compiler noise

* make `nimble install` without `-d` work

* move `libp2p_pki_schemes=secp256k1` to config.nims

* fix include
2023-07-21 15:51:42 -06:00
Jaremy Creechley
d4331f8062
Remove nimbus build (#73)
* Remove Nimbus
* adds docs for Nimble 0.14
* cleanup
2023-07-17 12:43:14 -07:00
Jaremy Creechley
7464c8e9a6
Split windows tests (#70)
Changes the CI to properly cache Nimble deps *including* Nim binaries. 

* parallelize tests
* speed up tests
* cache whole nimble
* use nimble install without -d
* bump version
* new nimble cache
* fix github_env
* compare speed
* readd msys2
* don't need make for nimble
* ugh renames
2023-07-13 20:42:43 -07:00
Jaremy Creechley
d7cc0ae783
renaming imports 2023-07-12 14:58:29 -07:00
Dmitriy Ryajov
e4e7a3e11f
track nim-libp2p's unstable branch and nim-bearssl's master branch (#51)
* track nim-libp2p's unstable branch and nim-bearssl's master branch

refactor accordingly: mainly switching from `import bearssl` to
`import bearssl/rand`, `BrHmacDrbgContext` to `HmacDrbgContext`, and related
changes

* fix ambiguous identifier

* nim 1.4 is deprecated

Co-authored-by: Michael Bradley, Jr <michaelsbradleyjr@gmail.com>
2022-11-02 10:21:05 -06:00
Dmitriy Ryajov
d6d255b4b5
Cleanups (#55)
* limit query to batchSize

* allow initializing node from ip and port

* misc cleanups
2022-10-31 22:41:33 -06:00
Dmitriy Ryajov
4b9fa0356e
Adding maintenance routines to cleanup expired and orphaned records (#52)
* add maintenance routines and split out cache

* start provider manager

* export provider manager

* add maintenance tests

* start is now async
2022-10-01 10:05:03 -06:00
Dmitriy Ryajov
69ae7c2012
Persist providers (#49)
* initial providers manager implementation

* misc license update

* spelling

* rename provider tests

* add provider tests

* reexport provider manager

* only update provider record if it changed

* small bug in getProvByKey

* pass providers to both constructors

* fix records retrieval

* disable cache during tests

* avoid redefining data

* adding back tests with cache

* use `.errorOption`

* proper err msg unpacking
2022-09-29 17:49:55 -06:00
Dmitriy Ryajov
9dfa86767d Revert "track nim-libp2p's unstable branch and nim-bearssl's master branch"
This reverts commit c480fb573242ab13fab57446aeece90eaabbe104.
2022-09-23 15:59:31 -06:00
Michael Bradley, Jr
c480fb5732 track nim-libp2p's unstable branch and nim-bearssl's master branch
refactor accordingly: mainly switching from `import bearssl` to
`import bearssl/rand`, `BrHmacDrbgContext` to `HmacDrbgContext`, and related
changes
2022-09-19 20:05:34 -05:00
Dmitriy Ryajov
fca255b2f7
use pure enums 2022-09-12 18:06:48 -06:00
Dmitriy Ryajov
3be2c8445c
suppress logging noise 2022-09-12 17:13:34 -06:00
Dmitriy Ryajov
1cbe8c12a6
suppress logging noise 2022-09-12 17:13:34 -06:00
Michael Bradley, Jr
faf5cad9d4 protobuf encoding for all message types 2022-04-07 14:47:35 -05:00
Eric Mastro
4c65f4bd94 chore: remove nim-eth/keys in favour of over to libp2p/crypto
Closes: #2.

Libp2p supports multiple cryptographic curves, however we have currently only implented support for secp256k1.

This needs to be run with the compiler flag `libp2p_pki_schemes` set to `secp256k1`. If running the tests, this can be run like so: `nimble test —libp2p_pki_schemes=secp256k1` to put secp as the first supported crypto scheme.
2022-04-07 14:11:48 -05:00
Csaba Kiraly
667a8b91a4 test lookup with traditional findNode as well 2022-03-22 15:46:19 +01:00
Csaba Kiraly
50df73da1b fixup: tests: expose startup delay parameter 2022-03-22 11:53:55 +01:00
Csaba Kiraly
d14222e8d1 tests: add back test after bootnode dies
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2022-03-22 11:53:55 +01:00
Csaba Kiraly
0431587765 test_providers: add optional delay during bootstrap
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2022-03-22 11:53:55 +01:00
Csaba Kiraly
4d7e773c42 fix tests: make bootstrap nodes async 2022-03-22 11:53:55 +01:00
Csaba Kiraly
9a01791e11 feat: Swap ENR to libp2p SignedPeerRecords
Swap all instances of Record with SignedPeerRecord.

Allow for `SignedPeerRecord`s to be updated by updating the first multiaddress in the `PeerRecord`. This also increments the `seqNo` in the `PeerRecord` only if the address was actually updated.
2022-03-22 11:00:26 +11:00
Eric Mastro
b843c8823c feat: merge add/get providers with discv5
Merge the add/get providers messages with the discovery v5 messages inside of the discovery v5 code. Discovery v5 uses RLP encoding, while the add/get providers uses protobufs, as per the libp2p spec. The merge of these two uses RLP on the outer wrapper of the message, and a protobuf-encoded blob on the “inside” for the add/get providers messages only. Eventually this needs to change so that we use protobufs only.

There is still some code in the libp2pdht/dht directory, which is being read from the discovery v5 directory. Eventually these two should merge as well.
2022-03-17 14:56:54 +11:00
Eric Mastro
cbcaf926a1 feat: Swap PeerRecord for SignedPeerReocrd (PR comments)
- Add example addresses in tests
- Change bootnode private key in tests
- bump libp2p to get access to SignedPeerRecord.encode
2022-03-10 17:00:33 +11:00
Eric Mastro
35026762b4 feat: Swap PeerRecords with SignedPeerRecords
Providers now add/get SignedPeerRecords to/from the DHT.

Changed the PeerId calculation to use the public key of the discovery node.
2022-03-09 15:03:16 +11:00
Csaba Kiraly
b65e0aa7ef comment out test instead of using skip()
after calling skip(), the test was still
executed and failed. Better to disable the
whole test.
2022-03-07 15:11:31 +11:00
Eric Mastro
0d0da71b65
build: Add CI Actions
Add CI workflow to run tests.

Add code coverage workflow that publishes results to CodeCov.

Add CI and coverage badges to README.

Bump asynctest to allow for `setupAll` and `teardownAll`.

Add local coverage nimble task.

Combine tests in to a testAll file.
2022-03-04 18:04:50 +11:00
Eric Mastro
55a43b2d89 chore: bump asynctest, use all/each test harness nouns
Using `setupAll`/`setupEach` and `teardownAll`/`teardownEach` for test harnesses.
2022-02-25 14:06:43 +11:00
Eric Mastro
acd03ff831 feat: reorg project to add nimble build system
Project has been updated to use nimble as a build system. All required dependencies have been added.

All nim-eth mirrored files were added to a private folder in the libp2pdht module. A libp2pdht/discv5 module was added to alias the nim-eth modules (which will change over time).

Test have been updated to use status-im/asynctest. This PR uses a branch of asynctest that supports async suite before/after. This seemed like the only the tests would work without throwing gcsafe errors.

All tests working.
2022-02-25 09:10:43 +11:00
Csaba Kiraly
e6cea68326 tests: give more time to bootstrap 2022-02-23 22:12:32 +01:00
Csaba Kiraly
0b301fdeae moving test under dht folder 2022-02-23 22:12:32 +01:00
Csaba Kiraly
bef2b711e5 just a comment 2022-02-23 22:12:32 +01:00
Csaba Kiraly
b118a34043 tests: make all tests work 2022-02-23 22:12:32 +01:00
Csaba Kiraly
77d93d71bd tests: generate PeerRecords from NodeId
this is just for the testing
2022-02-23 22:12:32 +01:00
Csaba Kiraly
54fde4e09e test fix: init nodes correctly 2022-02-23 22:12:32 +01:00
Csaba Kiraly
be5d911cda partial tests (AddProvider) 2022-02-23 22:12:32 +01:00
Csaba Kiraly
86328cd17e adding base for test_discovery
based on discv4 version
2022-02-23 22:12:32 +01:00
Csaba Kiraly
ca3f634239 import eth/[rlp, keys, async_utils, net/utils] directly from eth 2022-02-23 22:11:27 +01:00
Csaba Kiraly
b03d5eb3c7 fixup: fix imports in test_discoveryv5_encoding
Note that the test does not compile, but it was
also not compiling before

Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2022-02-23 11:02:06 +01:00
Csaba Kiraly
fd350b4fe5 splitting protocol in lower and upper half
Protocol was actually made of two sub-protocols.
  * a lower-half handling authentication, encryption,
key exchange, and request/response. This is now called
Transport.
  * an upper-half handling DHT messages. This is still
called Protocol.

Separation of these two reduces dependencies and simplifies
modifications to the protocol.
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2022-02-23 11:02:06 +01:00
Csaba Kiraly
cfa97f4995 initial commit based on nim-eth@779d767b024175a51cf74c79ec7513301ebe2f46 2022-02-14 01:51:28 +01:00