179 Commits

Author SHA1 Message Date
Csaba Kiraly
4ccaaee721
rename metrics to dht_ from discovery_
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>

# Conflicts:
#	codexdht/private/eth/p2p/discoveryv5/transport.nim
2024-10-10 11:44:26 +02:00
Csaba Kiraly
80cc069c5e
metrics: add transport byte counters
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>

# Conflicts:
#	codexdht/private/eth/p2p/discoveryv5/transport.nim
2024-10-10 11:43:23 +02:00
Csaba Kiraly
ffeeeeb3fb
transport: add metrics
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>

# Conflicts:
#	codexdht/private/eth/p2p/discoveryv5/transport.nim
2024-10-10 11:42:11 +02:00
Csaba Kiraly
4d2250477e
metrics: add discovery_routing_table_buckets
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2024-10-10 11:40:45 +02:00
Csaba Kiraly
b7b04ed9e4
metrics: rename routing_table_nodes to discovery_routing_table_nodes
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2024-10-10 11:40:44 +02:00
Csaba Kiraly
6e180af4aa
Merge pull request #101 from codex-storage/logging
Logging updates
2024-10-10 11:22:23 +02:00
Csaba Kiraly
f6971cc947
logging: better logging of SPR update
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2024-10-08 11:15:50 +02:00
Csaba Kiraly
4d9e39d86c
transport: improve logging
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>

# Conflicts:
#	codexdht/private/eth/p2p/discoveryv5/transport.nim
2024-10-08 11:15:20 +02:00
Csaba Kiraly
b8bcb2d08d
Merge pull request #95 from codex-storage/factorize
Factorize code
2024-10-07 14:06:59 +02:00
Csaba Kiraly
f121d080e7
Merge pull request #96 from codex-storage/reduce-timeouts
Reduce timeouts
2024-10-03 10:54:44 +02:00
Csaba Kiraly
fef297c622
Merge pull request #94 from codex-storage/feature-FindNodeFastResultLimit
Add separate limit for results returned in FindNodeFast
2024-10-01 15:04:26 +02:00
Csaba Kiraly
936a5ec6fa
Merge pull request #93 from codex-storage/fix-FindNodeResultLimit
fix returning too many nodes when FindNodeResultLimit!=BUCKET_SIZE
2024-10-01 14:51:33 +02:00
Ben Bierens
9acdca795b
routing table logging update (#97)
* Clear logs for adding and removing of nodes. routingtable log topic for filtering.

* Makes node ID shortening consistent with other short-id formats

* redundant else block

* fixes dependencies
2024-09-23 15:49:08 +02:00
Ben Bierens
5f38fd9570
GCC-14 (#98)
* bumps bearssl

* updates version of bearssl in lockfiles

* fixes that checksum

* attempt to bump various dependencies

* updates asynctest version tag

* asynctest sha

* bumps to working version of nim-datastore

* adjusts asynctest imports for chronos

* chronos checksum

* checksum for datastore

* libp2p version tag

* libp2p checksum

* moves libp2p from codex-branch to latest master

* libp2p checksum

* splits the test dependencies from the dev dependencies (example nim-ethers)

* sets path

* pathing in tests

* oops wrong version

* adds build.nims to installfiles for test module

* attempt to fix import paths

* bumps nim-datastore

* datastore checksum

* greatly simplify CI

* fixes asynctest import

* builds parallel tests before running

* bumps datastore

* turns nim-stable back off

* pins nim-datastore version

* bumps checkout to v4

* Review comment by Mark

Co-authored-by: markspanbroek <mark@spanbroek.net>

* Review comment by Mark

Co-authored-by: markspanbroek <mark@spanbroek.net>

---------

Co-authored-by: markspanbroek <mark@spanbroek.net>
2024-08-20 11:04:48 +02:00
Csaba Kiraly
5624700855
reduce default timeouts
We really don't need these to be 2 and 4 seconds.
Later we should tune it better based on measurements
or estimates. We should also check the relation between
these three values.

Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2024-07-01 04:34:10 +02:00
Csaba Kiraly
76da855725
use handshakeTimeout if handshake starting in sendMessage
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2024-07-01 04:20:20 +02:00
Csaba Kiraly
4c9c92232b
remove unused sendRequest call
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2024-07-01 04:14:17 +02:00
Csaba Kiraly
148b10908d
trace log: do not log binary encoding
Even at trace level this feels too much.

Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2024-07-01 04:14:13 +02:00
Csaba Kiraly
f299c23e2e
remove lookupWorkerFast duplicate code
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2024-07-01 04:14:03 +02:00
Csaba Kiraly
bdf57381e3
introduce FindNodeFastResultLimit
We do not need that many responses with FindNodeFast, since the
reposes can be ordered by distance

Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2024-07-01 04:06:43 +02:00
Csaba Kiraly
4b82bdc2f9
fix returning too many nodes when FindNodeResultLimit!=BUCKET_SIZE
Code assumed these two values to be the same, resulting in
reception errors.

Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2024-07-01 03:55:03 +02:00
Csaba Kiraly
d8160ff0f7
add logging helper for Protocol
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2024-06-28 17:39:13 +02:00
Csaba Kiraly
f766cb39b1
encoding: introducing type cipher=aes128
Introducing the cipher type to ease changing cipher.
No functional change

Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2024-06-28 17:37:26 +02:00
Csaba Kiraly
316464fc71
dht: waitMessage: expose timeout as parameter, keeping default
defults to ResponseTimeout as before

Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2024-06-28 17:35:29 +02:00
Csaba Kiraly
6e61e02091
fixup: move sendRequest forward
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2024-06-28 17:34:49 +02:00
Csaba Kiraly
dfff39091b
introduce waitResponse wrapper
initialize wait for response before sending request.
This is needed in cases where the response arrives before
moving to the next instruction, such as a directly connected
test.

Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2024-06-28 17:33:56 +02:00
Giuliano Mega
63822e8356
Update nim-codex-dht to Chronos V4 (#90)
Update nim-codex-dht to Chronos v4
v0.5.0
2024-05-23 17:49:44 -03:00
Giuliano Mega
2299317116
Merge pull request #91 from codex-storage/chore/update-libp2p-repo
Update repo for libp2p
v0.4.1
2024-03-18 19:40:01 -03:00
gmega
717cd0a50c
Merge branch 'master' into chore/update-libp2p-repo 2024-03-18 19:11:57 -03:00
Giuliano Mega
223ce9240b
Merge pull request #92 from codex-storage/fix/nimble-archive-osx
fix name for nimble archive for OSX
2024-03-18 19:11:28 -03:00
gmega
709a873862
fix name for nimble archive for OSX 2024-03-18 18:48:04 -03:00
gmega
b3d01245e9
update repo for libp2p 2024-03-18 18:42:48 -03:00
Dmitriy Ryajov
beefafcc6f
Update CleanupInterval to 24 hours (#88) 2023-11-21 17:14:15 -08:00
Dmitriy Ryajov
a7f14bc9b7
Fix logging format (#87)
* add shortLog for Address

* compile with trace logging to catch errors
2023-11-20 09:34:40 -08:00
Dmitriy Ryajov
dd4985435a
Fix timeout and delete (#86)
* use unix time for ttl

* don't remove all entries on peer removal

* cleanup questionable tuple destructure

* ignore vscode

* fix endians decoding

* allow removing by peerId

* invalidate cache by peerId on remove

* update test
2023-11-17 14:01:16 -08:00
Csaba Kiraly
91b2eaec89
Fix: arrive to working keys in case of simultaneous cross connect (#84)
* improve tracing of message exchange

run e.g. as
```
nim c -r -d:debug -d:chronicles_enabled=on -d:chronicles_log_level=TRACE -d:chronicles_sinks=textlines[nocolors,stdout] tests/dht/test_providers.nim >err
```

Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>

* add debug on Handshake timeour

Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>

* queue messages during handshake and send later

If a handshake was already in progress, messages were dropped.
Instead of this, it is better to queue these and send as soon
as the handshake is finished and thus the encryption key is known.

Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>

* rename handshakeInProgress to keyexchangeInProgress

Handshake is also a name of a message, which makes previous
name less clear.

Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>

* keyexchangeInProgress: do not remove on handshake received

This is the wrong direction, not needed

Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>

* fix cross-connect key exchange

Since key exchange can be started both ways simultaneously, and
these might not get finalised with UDP transport, we can't be
sure what encryption key will be used by the other side:
- the one derived in the key-exchange started by us,
- the one derived in the key-exchange started by the other node.
To alleviate this issue, we store two decryption keys in each session.

Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>

---------

Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2023-11-17 11:50:28 -08:00
Csaba Kiraly
66116b9bf6
Fix: queue messages when there is no encryption key (#83)
* encodeMessagePacket: expose haskey

encodeMessagePacket checks for session and behaves differently
based on that. Exposing this difference in behavior.

Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>

* improve tracing of message exchange

run e.g. as
```
nim c -r -d:debug -d:chronicles_enabled=on -d:chronicles_log_level=TRACE -d:chronicles_sinks=textlines[nocolors,stdout] tests/dht/test_providers.nim >err
```

Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>

* add debug on Handshake timeour

Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>

* queue messages during handshake and send later

If a handshake was already in progress, messages were dropped.
Instead of this, it is better to queue these and send as soon
as the handshake is finished and thus the encryption key is known.

Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>

* rename handshakeInProgress to keyexchangeInProgress

Handshake is also a name of a message, which makes previous
name less clear.

Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>

* keyexchangeInProgress: do not remove on handshake received

This is the wrong direction, not needed

Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>

---------

Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2023-11-17 11:18:48 -08:00
Jaremy Creechley
60dc4e764c
Fix changes from stint (#81)
* some formatting tweaks to make errors easier to grok

* stint removed overloads for regular ints - use stew versions instead

* various name style fixes

* ignore vscode stuff

* revert style changes

* revert unneeded var rename changes

---------

Co-authored-by: Dmitriy Ryajov <dryajov@gmail.com>
2023-11-17 11:08:45 -08:00
Dmitriy Ryajov
ee5d8acb05
cleanup and avoid lockups (#85) 2023-09-20 09:20:26 -07:00
Slava
ed7caa119d
Update CI workflow trigger branch (#82)
* Update CI workflow trigger branch

* Update Codecov workflow trigger branch and badges
2023-08-26 09:31:31 +03:00
Jaremy Creechley
fdd02450aa
bump release (#79) 2023-07-25 19:56:25 -07:00
Jaremy Creechley
b585290397
Bump deps (#77)
Updates all Status IM and Codex Storage deps to the latest except for nim-stint which is held back due to some compiler issue. 

* fix nimble name
* don't override nimble
* update all deps
* import nimble.lock and fix urls
* don't forget nim
* bump to chronos with async notifications
2023-07-21 15:29:38 -07:00
Dmitriy Ryajov
9ae0bfb1c3
Fix nimble install (#78)
* make encryption scheme explicit

* supress compiler noise

* make `nimble install` without `-d` work

* move `libp2p_pki_schemes=secp256k1` to config.nims

* fix include
2023-07-21 15:51:42 -06:00
Jaremy Creechley
1f27eb4aff
Upgrade secp256k1 deps (#76)
* import full secret hash from upstream
* use full secret hash including prefix byte 0x02 / 0x03
* import nimble lock
* fix atlas lock
* update stint
* cleanup urls
* bump lock files
* match lockfiles
2023-07-19 17:04:28 -07:00
Jaremy Creechley
fc7d7ef80c
version bump - stable ci version v0.3.2 2023-07-17 15:34:59 -07:00
Ivan Yonchovski
ae844ec4c5
Restore coverage (#74)
* Restore coverage
* use nimble setup for now
* Update codecov.yml
* fix lcov
* re-add nim binary to lockfile

---------

Co-authored-by: Jaremy Creechley <creechley@gmail.com>
2023-07-17 15:32:55 -07:00
Jaremy Creechley
d4331f8062
Remove nimbus build (#73)
* Remove Nimbus
* adds docs for Nimble 0.14
* cleanup
2023-07-17 12:43:14 -07:00
Jaremy Creechley
7464c8e9a6
Split windows tests (#70)
Changes the CI to properly cache Nimble deps *including* Nim binaries. 

* parallelize tests
* speed up tests
* cache whole nimble
* use nimble install without -d
* bump version
* new nimble cache
* fix github_env
* compare speed
* readd msys2
* don't need make for nimble
* ugh renames
v0.3.1
2023-07-13 20:42:43 -07:00
Jaremy Creechley
18dbe4cc04
Merge pull request #69 from codex-storage/rename-library
Finish renaming library
2023-07-12 16:13:13 -07:00
Jaremy Creechley
127d7be4e5
bump version v0.3.0 2023-07-12 15:47:16 -07:00