Commit Graph

51 Commits

Author SHA1 Message Date
Csaba Kiraly 57f4b6f7cb
Merge pull request #103 from codex-storage/fix-randomNodes
fix potential infinite loop in randomNodes
2024-10-18 20:45:49 +02:00
Csaba Kiraly a6cfe1a084
fix potential infinite loop in randomNodes
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2024-10-17 12:38:54 +02:00
Csaba Kiraly 1a344f1fd7
log reliability based on loss statistics
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2024-10-15 18:17:49 +02:00
Csaba Kiraly fee5a9ced2
set NoreplyRemoveThreshold to 0.5
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2024-10-14 15:35:15 +02:00
Csaba Kiraly 6310c50ce0
introduce NoreplyRemoveThreshold
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>

# Conflicts:
#	codexdht/private/eth/p2p/discoveryv5/protocol.nim
2024-10-14 15:35:10 +02:00
Csaba Kiraly 7507e99c96
register "not seen" when missing replies
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2024-10-14 15:33:34 +02:00
Csaba Kiraly 02bc12e639
change node seen flag to an exponential moving average
keep defaults as before

Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>

# Conflicts:
#	codexdht/private/eth/p2p/discoveryv5/node.nim
#	codexdht/private/eth/p2p/discoveryv5/routing_table.nim
2024-10-14 15:33:29 +02:00
Csaba Kiraly e1c1089e4f
fix aggressive node removal from on first packet loss
UDP packets get lost easily. We can't just remove
nodes from the routing table at first loss, as it can
create issues in small networks and in cases of temporary
connection failures.

Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2024-10-14 15:28:09 +02:00
Csaba Kiraly c1d2ea410d
Merge pull request #102 from codex-storage/measure-rtt-bw
Measure rtt, estimate bw, and log every 5 minutes
2024-10-14 14:19:35 +02:00
Csaba Kiraly 8b1660464d
don't log bandwidth estimates
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2024-10-14 13:57:52 +02:00
Csaba Kiraly 7057663f81
fixup: remove excessive debug
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2024-10-14 11:19:36 +02:00
Csaba Kiraly 4ccaaee721
rename metrics to dht_ from discovery_
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>

# Conflicts:
#	codexdht/private/eth/p2p/discoveryv5/transport.nim
2024-10-10 11:44:26 +02:00
Csaba Kiraly 80cc069c5e
metrics: add transport byte counters
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>

# Conflicts:
#	codexdht/private/eth/p2p/discoveryv5/transport.nim
2024-10-10 11:43:23 +02:00
Csaba Kiraly ffeeeeb3fb
transport: add metrics
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>

# Conflicts:
#	codexdht/private/eth/p2p/discoveryv5/transport.nim
2024-10-10 11:42:11 +02:00
Csaba Kiraly 4d2250477e
metrics: add discovery_routing_table_buckets
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2024-10-10 11:40:45 +02:00
Csaba Kiraly b7b04ed9e4
metrics: rename routing_table_nodes to discovery_routing_table_nodes
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2024-10-10 11:40:44 +02:00
Csaba Kiraly 706cb50041
add debugPrintLoop to print neighborhood info
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2024-10-08 11:31:06 +02:00
Csaba Kiraly 0825d887ea
add bandwidth estimate
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2024-10-08 11:31:06 +02:00
Csaba Kiraly ec4f0d4a84
add transport level RTT measurement
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2024-10-08 11:18:02 +02:00
Csaba Kiraly 0b69de242f
add rtt measurement
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2024-10-08 11:17:58 +02:00
Csaba Kiraly f3eec2a202
node: add RTT and bandwidth measurement holders
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2024-10-08 11:17:29 +02:00
Csaba Kiraly f6971cc947
logging: better logging of SPR update
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2024-10-08 11:15:50 +02:00
Csaba Kiraly 4d9e39d86c
transport: improve logging
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>

# Conflicts:
#	codexdht/private/eth/p2p/discoveryv5/transport.nim
2024-10-08 11:15:20 +02:00
Csaba Kiraly b8bcb2d08d
Merge pull request #95 from codex-storage/factorize
Factorize code
2024-10-07 14:06:59 +02:00
Csaba Kiraly f121d080e7
Merge pull request #96 from codex-storage/reduce-timeouts
Reduce timeouts
2024-10-03 10:54:44 +02:00
Csaba Kiraly fef297c622
Merge pull request #94 from codex-storage/feature-FindNodeFastResultLimit
Add separate limit for results returned in FindNodeFast
2024-10-01 15:04:26 +02:00
Csaba Kiraly 936a5ec6fa
Merge pull request #93 from codex-storage/fix-FindNodeResultLimit
fix returning too many nodes when FindNodeResultLimit!=BUCKET_SIZE
2024-10-01 14:51:33 +02:00
Ben Bierens 9acdca795b
routing table logging update (#97)
* Clear logs for adding and removing of nodes. routingtable log topic for filtering.

* Makes node ID shortening consistent with other short-id formats

* redundant else block

* fixes dependencies
2024-09-23 15:49:08 +02:00
Csaba Kiraly 5624700855
reduce default timeouts
We really don't need these to be 2 and 4 seconds.
Later we should tune it better based on measurements
or estimates. We should also check the relation between
these three values.

Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2024-07-01 04:34:10 +02:00
Csaba Kiraly 76da855725
use handshakeTimeout if handshake starting in sendMessage
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2024-07-01 04:20:20 +02:00
Csaba Kiraly 4c9c92232b
remove unused sendRequest call
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2024-07-01 04:14:17 +02:00
Csaba Kiraly 148b10908d
trace log: do not log binary encoding
Even at trace level this feels too much.

Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2024-07-01 04:14:13 +02:00
Csaba Kiraly f299c23e2e
remove lookupWorkerFast duplicate code
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2024-07-01 04:14:03 +02:00
Csaba Kiraly bdf57381e3
introduce FindNodeFastResultLimit
We do not need that many responses with FindNodeFast, since the
reposes can be ordered by distance

Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2024-07-01 04:06:43 +02:00
Csaba Kiraly 4b82bdc2f9
fix returning too many nodes when FindNodeResultLimit!=BUCKET_SIZE
Code assumed these two values to be the same, resulting in
reception errors.

Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2024-07-01 03:55:03 +02:00
Csaba Kiraly d8160ff0f7
add logging helper for Protocol
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2024-06-28 17:39:13 +02:00
Csaba Kiraly f766cb39b1
encoding: introducing type cipher=aes128
Introducing the cipher type to ease changing cipher.
No functional change

Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2024-06-28 17:37:26 +02:00
Csaba Kiraly 316464fc71
dht: waitMessage: expose timeout as parameter, keeping default
defults to ResponseTimeout as before

Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2024-06-28 17:35:29 +02:00
Csaba Kiraly 6e61e02091
fixup: move sendRequest forward
Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2024-06-28 17:34:49 +02:00
Csaba Kiraly dfff39091b
introduce waitResponse wrapper
initialize wait for response before sending request.
This is needed in cases where the response arrives before
moving to the next instruction, such as a directly connected
test.

Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2024-06-28 17:33:56 +02:00
Giuliano Mega 63822e8356
Update nim-codex-dht to Chronos V4 (#90)
Update nim-codex-dht to Chronos v4
2024-05-23 17:49:44 -03:00
Dmitriy Ryajov beefafcc6f
Update CleanupInterval to 24 hours (#88) 2023-11-21 17:14:15 -08:00
Dmitriy Ryajov a7f14bc9b7
Fix logging format (#87)
* add shortLog for Address

* compile with trace logging to catch errors
2023-11-20 09:34:40 -08:00
Dmitriy Ryajov dd4985435a
Fix timeout and delete (#86)
* use unix time for ttl

* don't remove all entries on peer removal

* cleanup questionable tuple destructure

* ignore vscode

* fix endians decoding

* allow removing by peerId

* invalidate cache by peerId on remove

* update test
2023-11-17 14:01:16 -08:00
Csaba Kiraly 91b2eaec89
Fix: arrive to working keys in case of simultaneous cross connect (#84)
* improve tracing of message exchange

run e.g. as
```
nim c -r -d:debug -d:chronicles_enabled=on -d:chronicles_log_level=TRACE -d:chronicles_sinks=textlines[nocolors,stdout] tests/dht/test_providers.nim >err
```

Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>

* add debug on Handshake timeour

Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>

* queue messages during handshake and send later

If a handshake was already in progress, messages were dropped.
Instead of this, it is better to queue these and send as soon
as the handshake is finished and thus the encryption key is known.

Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>

* rename handshakeInProgress to keyexchangeInProgress

Handshake is also a name of a message, which makes previous
name less clear.

Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>

* keyexchangeInProgress: do not remove on handshake received

This is the wrong direction, not needed

Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>

* fix cross-connect key exchange

Since key exchange can be started both ways simultaneously, and
these might not get finalised with UDP transport, we can't be
sure what encryption key will be used by the other side:
- the one derived in the key-exchange started by us,
- the one derived in the key-exchange started by the other node.
To alleviate this issue, we store two decryption keys in each session.

Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>

---------

Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2023-11-17 11:50:28 -08:00
Csaba Kiraly 66116b9bf6
Fix: queue messages when there is no encryption key (#83)
* encodeMessagePacket: expose haskey

encodeMessagePacket checks for session and behaves differently
based on that. Exposing this difference in behavior.

Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>

* improve tracing of message exchange

run e.g. as
```
nim c -r -d:debug -d:chronicles_enabled=on -d:chronicles_log_level=TRACE -d:chronicles_sinks=textlines[nocolors,stdout] tests/dht/test_providers.nim >err
```

Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>

* add debug on Handshake timeour

Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>

* queue messages during handshake and send later

If a handshake was already in progress, messages were dropped.
Instead of this, it is better to queue these and send as soon
as the handshake is finished and thus the encryption key is known.

Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>

* rename handshakeInProgress to keyexchangeInProgress

Handshake is also a name of a message, which makes previous
name less clear.

Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>

* keyexchangeInProgress: do not remove on handshake received

This is the wrong direction, not needed

Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>

---------

Signed-off-by: Csaba Kiraly <csaba.kiraly@gmail.com>
2023-11-17 11:18:48 -08:00
Jaremy Creechley 60dc4e764c
Fix changes from stint (#81)
* some formatting tweaks to make errors easier to grok

* stint removed overloads for regular ints - use stew versions instead

* various name style fixes

* ignore vscode stuff

* revert style changes

* revert unneeded var rename changes

---------

Co-authored-by: Dmitriy Ryajov <dryajov@gmail.com>
2023-11-17 11:08:45 -08:00
Dmitriy Ryajov ee5d8acb05
cleanup and avoid lockups (#85) 2023-09-20 09:20:26 -07:00
Dmitriy Ryajov 9ae0bfb1c3
Fix nimble install (#78)
* make encryption scheme explicit

* supress compiler noise

* make `nimble install` without `-d` work

* move `libp2p_pki_schemes=secp256k1` to config.nims

* fix include
2023-07-21 15:51:42 -06:00
Jaremy Creechley 1f27eb4aff
Upgrade secp256k1 deps (#76)
* import full secret hash from upstream
* use full secret hash including prefix byte 0x02 / 0x03
* import nimble lock
* fix atlas lock
* update stint
* cleanup urls
* bump lock files
* match lockfiles
2023-07-19 17:04:28 -07:00