311 Commits

Author SHA1 Message Date
Giovanni Petrantoni
e225f3cf5a Merge branch 'master' into gossip-one-one 2020-08-24 17:41:45 +09:00
Dmitriy Ryajov
d3182c4dba
No raise send (#339)
* dont raise in send

* check that the lock is acquire on release
2020-08-20 20:50:33 -06:00
Jacek Sieka
eb13845f65 work around send that may raise
`send` can raise exceptions that together with asyncCheck will
crash NBC
2020-08-19 14:25:30 +03:00
Giovanni Petrantoni
3b8e85c792 Merge branch 'master' into gossip-one-one 2020-08-19 18:41:38 +09:00
Zahary Karadjov
af0955c58b
Add comments explaning a possible deadlock 2020-08-18 13:51:41 +03:00
Zahary Karadjov
60122a044c
Restore interop with Lighthouse by preventing concurrent meshsub dials 2020-08-17 22:40:58 +03:00
Jacek Sieka
53877e97bd
trace logs 2020-08-17 12:39:25 +02:00
Jacek Sieka
f46bf0faa4
remove send lock (#334)
* remove send lock

When mplex receives data it will block until a reader has processed the
data. Thus, when a large message is received, such as a gossipsub
subscription table, all of mplex will be blocked until all reading is
finished.

However, if at the same time a `dial` to establish a gossipsub send
connection is ongoing, that `dial` will be blocked because mplex is no
longer reading data - specifically, it might indeed be the connection
that's processing the previous data that is waiting for a send
connection.

There are other problems with the current code:
* If an exception is raised, it is not necessarily raised for the same
connection as `p.sendConn`, so resetting `p.sendConn` in the exception
handling is wrong
* `p.isConnected` is checked before taking the lock - thus, if it
returns false, a new dial will be started. If a new task enters `send`
before dial is finished, it will also determine `p.isConnected` is
false, then get stuck on the lock - when the previous task finishes and
releases the lock, the new task will _also_ dial and thus reset
`p.sendConn` causing a leak.

* prefer existing connection

simplifies flow
2020-08-17 12:38:27 +02:00
Jacek Sieka
b12145dff7
avoid crash when subscribe is received (#333)
...by making subscribeTopic synchronous, avoiding a peer table lookup
completely.

rebalanceMesh will be called a second later - it's fine
2020-08-17 12:10:22 +02:00
Giovanni Petrantoni
eebaeb779f Merge branch 'master' into gossip-one-one 2020-08-17 11:44:23 +09:00
Giovanni Petrantoni
afaa7f2a84 finishup with score and ihave budget 2020-08-17 01:20:50 +09:00
Jacek Sieka
ab864fc747
logging cleanups and small fixes (#331) 2020-08-15 21:50:31 +02:00
Giovanni Petrantoni
1661344e17 oversub prune score based, finish outbound quota 2020-08-15 16:25:29 +09:00
Jacek Sieka
9c7e055310
set activity flag on noise / secio (#330) 2020-08-15 07:36:15 +02:00
Giovanni Petrantoni
69ff05a4f4 outbound mesh quota, internal tests fixing 2020-08-15 12:51:51 +09:00
Giovanni Petrantoni
309a4998c8 Adaptive gossip dissemination 2020-08-15 11:57:22 +09:00
Giovanni Petrantoni
3e8349b918 IWANT cap/budget 2020-08-14 17:07:26 +09:00
Giovanni Petrantoni
c6fc8dee54 backoff time management 2020-08-14 11:59:19 +09:00
Giovanni Petrantoni
ce61d84db4 add an hard cap to PX 2020-08-13 21:21:17 +09:00
Giovanni Petrantoni
585cfc5996 implement peer exchange graft message 2020-08-13 20:41:54 +09:00
Giovanni Petrantoni
05463873a2 restore explicit peering 2020-08-12 18:25:57 +09:00
Giovanni Petrantoni
296ca14133 rebalance improvements and opportunistic grafting 2020-08-12 16:30:03 +09:00
Giovanni Petrantoni
f7fee2ca97 further merge fixes 2020-08-12 12:04:54 +09:00
Giovanni Petrantoni
c8954619ab post merge - builds gossip tests 2020-08-12 11:17:36 +09:00
Giovanni Petrantoni
90a33c0a29 Merge branch 'master' into gossip-one-one 2020-08-12 10:57:32 +09:00
Dmitriy Ryajov
b76b3e0e9b
Rework pubsub (#322)
* move pubsub of off switch, pass switch into pubsub

* use join on lpstreams

* properly cleanup up failed peers

* fix tests

* fix peertable hasPeerId

* fix tests

* rework sending, remove helpers from pubsubpeer, unify in broadcast

* further split broadcast into send

* use send where appropriate

* use formatIt

* improve trace

Co-authored-by: Giovanni Petrantoni <giovanni@fragcolor.xyz>
2020-08-11 18:05:49 -06:00
Dmitriy Ryajov
2325692f55
Fix half closed (#324)
* don't call `close` in `remoteClose`

* make sure timeout are properly propagted

* fix tests

* adding remote close write test
2020-08-10 16:17:11 -06:00
Giovanni Petrantoni
66c9cd2c8c score fixes 2020-08-08 18:27:57 +09:00
Giovanni Petrantoni
4dcc3b6dda Merge remote-tracking branch 'origin/master' into gossip-one-one 2020-08-08 16:53:14 +09:00
Jacek Sieka
f303954989
peer hooks -> events (#320)
* peer hooks -> events

* peerinfo -> peerid
* include connection direction in event
* check connection status after event
* lock connmanager lookup also when dialling peer
* clean up un-upgraded connection when upgrade fails
* await peer eventing

* remove join/lifetime future from peerinfo

Peerinfo instances are not unique per peer so the lifetime future is
misleading - it fires when a random connection is closed, not the "last"
one

* document switch values

* naming

* peerevent->conneevent
2020-08-08 08:52:20 +02:00
zah
fbb59c3638
msg is a reserved property name in Chronicles (#321)
Every Chronicles log record has an existing `msg` property matching
the static string supplied in the log statement. Thus, it's currently
not possible to use `msg` as the name of a user property:

https://github.com/status-im/nim-chronicles/issues/86
2020-08-07 16:46:00 -06:00
Jacek Sieka
7c2ab38da1
cleanups (#319) 2020-08-06 20:14:40 +02:00
Jacek Sieka
c6c0c152c0
Dial peerid (#308)
* prefer PeerID in switch api

This avoids ref issues like ref identity and nil

* use existing peerinfo instance if possible

* remove secureCodec

there may be multiple connections per peerinfo with different codecs

* avoid some extra async::
2020-08-06 09:29:27 +02:00
Giovanni Petrantoni
9bbe5e4841
Fix subclass calls to handleDisconnect (#314)
* Fix subclass calls to handleDisconnect

* add peer ID to nil peer debug message
2020-08-06 11:12:52 +09:00
Giovanni Petrantoni
5c986cf657
Fix build, add some raises (#315)
* Fix build, add some raises

* wip

* wip more raises

* missing exc object in mplex

* proper lifetime for subscribePeer

Co-authored-by: Dmitriy Ryajov <dryajov@gmail.com>
2020-08-05 19:30:57 -06:00
Ștefan Talpalaru
bd5d43874a
more expensive metrics (#312) 2020-08-05 14:02:26 +02:00
Ștefan Talpalaru
843d32f8db
put expensive metrics under a Nim define (#310) 2020-08-04 17:27:59 -06:00
Giovanni Petrantoni
5f0637c49a
Audit curve fixes part2 (#298)
* refactor and fix mulgen (curve25519)

* crypto tests fixing

* fix some confusion in curve25519 mul

* removing ForbiddenCurveValues table and checks

* fix remaining merge issues
2020-08-04 18:19:26 +09:00
Giovanni Petrantoni
5cd93540fa
add a timeout during noise handshake (#294)
* add a timeout during noise handshake

* noise hs timeout into const
2020-08-04 17:04:16 +09:00
Giovanni Petrantoni
504e0444d3
refactor and fix mulgen (curve25519) (#293)
* refactor and fix mulgen (curve25519)

* crypto tests fixing
2020-08-04 14:07:53 +09:00
Dmitriy Ryajov
b6877b8aac
increase send timeout for prune and graft msgs (#306)
* increase send timeout for prune and graft msgs

* use trace logs for subscribe monitor
2020-08-03 17:55:42 -06:00
Dmitriy Ryajov
980764774e
pubsub timeouts tuning (#295)
* add finegrained timeouts to pubsub

* use 10 millis timeout in tests

* finalization

* revert timeouts

* use `atEof` for reads

* adjust timeouts and use atEof for reads

* use atEof for reads

* set isEof flag

* no backoff for pubsub streams

* temp timer increase, make macos finalize

* don't call `subscribePeer` in libp2p anymore

* more traces

* leak tests

* lower timeouts

* handle exceptions in control message

* don't use `cancelAndWait`

* handle exceptions in helpers

* wip

* don't send empty messages

* check for leaks properly

* don't use cancelAndWait

* don't await subscribption sends

* remove subscrivePeer calls from switch

* trying without the hooks again
2020-08-02 23:20:11 -06:00
Jacek Sieka
e655a510cd
misc cleanups (#303) 2020-08-02 12:22:49 +02:00
Giovanni Petrantoni
7946743ae5 wip 2020-07-29 09:41:34 +09:00
Giovanni Petrantoni
a34eee7ed4 pruning improvements 2020-07-28 14:09:37 +09:00
Giovanni Petrantoni
880acd9dc1 Merge branch 'master' into gossip-one-one 2020-07-28 12:06:16 +09:00
Dmitriy Ryajov
f7fdf31365
Pubsub lifetime (#284)
* lifecycle hooks

* tests

* move trace after closed check

* restore 1 second heartbeat

* await close event

* fix tests

* print direction string

* more trace logging

* add pubsub monitor

* add log scope

* adjust idle timeout

* add exc.msg to trace
2020-07-27 13:33:51 -06:00
Giovanni Petrantoni
a7b1417fd7 Merge branch 'master' into gossip-one-one 2020-07-23 09:14:06 +09:00
Giovanni Petrantoni
f73cf36afb add a handleConnect in subscribeTopic 2020-07-22 11:00:02 +09:00
Giovanni Petrantoni
f122ec9dbf fix grafted defect 2020-07-22 10:19:53 +09:00