739 Commits

Author SHA1 Message Date
Giovanni Petrantoni
84cc85b536 Merge branch 'master' into gossip-one-one 2020-09-08 18:03:13 +09:00
Jacek Sieka
82c179db9e
mplex fixes (#356)
* close the right connection when channel send fails
* don't crash on channel id that is not unique
2020-09-08 08:24:28 +02:00
Jacek Sieka
2b72d485a3
a few more log fixes (#355) 2020-09-07 14:15:11 +02:00
Jacek Sieka
c1856fda53
simplify and unify logging (#353)
* use short format for logging peerid
* log peerid:oid for connections
2020-09-06 10:31:47 +02:00
Jacek Sieka
9b815efe8f
gossipsub: don't subscribe to floodsub also (#352) 2020-09-04 22:53:03 +02:00
Jacek Sieka
16a008db75
fix connection event order when connection dies early (#351)
if the connection is already closed (because the remote closes during
identfiy for example), an exception would be raised which would leave
the connection in limbo, beacuse it would not go through the rest of
internalConnect.

Also, if the connection is already closed, the disconnect event would be
scheduled before the connect event :/
2020-09-04 20:30:26 +02:00
Jacek Sieka
6d91d61844
small cleanups & docs (#347)
* simplify gossipsub heartbeat start / stop
* avoid alloc in peerid check
* stop iterating over seq after unsubscribing item (could crash)
* don't crash on missing private key with enabled sigs (shouldn't happen
but...)
2020-09-04 18:31:43 +02:00
Eugene Kabanov
0b85192119
Remove asyncCheck from codebase. (#345)
* Remove asyncCheck from codebase.

* Replace all `discard` statements with new `asyncSpawn`.

* Bump `nim-chronos` requirement.
2020-09-04 18:30:45 +02:00
Jacek Sieka
5819c6a9a7
gossipsub / floodsub fixes (#348)
* mcache fixes

* remove timed cache - the window shifting already removes old messages
* ref -> object
* avoid unnecessary allocations with `[]` operator

* simplify init

* fix several gossipsub/floodsub issues

* floodsub, gossipsub: don't rebroadcast messages that fail validation
(!)
* floodsub, gossipsub: don't crash when unsubscribing from unknown
topics (!)
* gossipsub: don't send message to peers that are not interested in the
topic, when messages don't share topic list
* floodsub: don't repeat all messages for each message when
rebroadcasting
* floodsub: allow sending empty data
* floodsub: fix inefficient unsubscribe
* sync floodsub/gossipsub logging
* gossipsub: include incoming messages in mcache (!)
* gossipsub: don't rebroadcast already-seen messages (!)
* pubsubpeer: remove incoming/outgoing seen caches - these are already
handled in gossipsub, floodsub and will cause trouble when peers try to
resubscribe / regraft topics (because control messages will have same
digest)
* timedcache: reimplement without timers (fixes timer leaks and extreme
inefficiency due to per-message closures, futures etc)
* timedcache: ref -> obj
2020-09-04 08:10:32 +02:00
Jacek Sieka
cd1c68dbc5
avoid send deadlock by not allowing send to block (#342)
* avoid send deadlock by not allowing send to block

* handle message issues more consistently
2020-09-01 09:33:03 +02:00
Giovanni Petrantoni
e225f3cf5a Merge branch 'master' into gossip-one-one 2020-08-24 17:41:45 +09:00
Dmitriy Ryajov
d3182c4dba
No raise send (#339)
* dont raise in send

* check that the lock is acquire on release
2020-08-20 20:50:33 -06:00
Giovanni Petrantoni
840a76915e warn -> debug log levels in errors.nim 2020-08-20 16:53:28 +09:00
Jacek Sieka
eb13845f65 work around send that may raise
`send` can raise exceptions that together with asyncCheck will
crash NBC
2020-08-19 14:25:30 +03:00
Giovanni Petrantoni
3b8e85c792 Merge branch 'master' into gossip-one-one 2020-08-19 18:41:38 +09:00
Zahary Karadjov
af0955c58b
Add comments explaning a possible deadlock 2020-08-18 13:51:41 +03:00
Zahary Karadjov
60122a044c
Restore interop with Lighthouse by preventing concurrent meshsub dials 2020-08-17 22:40:58 +03:00
Jacek Sieka
833a5b8e57
add muxer nil check 2020-08-17 13:32:02 +02:00
Jacek Sieka
cfcda3c3ef
work around race conditions between identify and other protocols
when identify is run on incoming connections, the connmanager tables are
updated too late for incoming connections to properly be handled

this is a quickfix that will eventually need cleaning up
2020-08-17 13:29:45 +02:00
Jacek Sieka
790b67c923
work around bufferstream deadlock (#332)
mplex backpressure handling deadlocks with something
2020-08-17 12:45:54 +02:00
Jacek Sieka
53877e97bd
trace logs 2020-08-17 12:39:25 +02:00
Jacek Sieka
f46bf0faa4
remove send lock (#334)
* remove send lock

When mplex receives data it will block until a reader has processed the
data. Thus, when a large message is received, such as a gossipsub
subscription table, all of mplex will be blocked until all reading is
finished.

However, if at the same time a `dial` to establish a gossipsub send
connection is ongoing, that `dial` will be blocked because mplex is no
longer reading data - specifically, it might indeed be the connection
that's processing the previous data that is waiting for a send
connection.

There are other problems with the current code:
* If an exception is raised, it is not necessarily raised for the same
connection as `p.sendConn`, so resetting `p.sendConn` in the exception
handling is wrong
* `p.isConnected` is checked before taking the lock - thus, if it
returns false, a new dial will be started. If a new task enters `send`
before dial is finished, it will also determine `p.isConnected` is
false, then get stuck on the lock - when the previous task finishes and
releases the lock, the new task will _also_ dial and thus reset
`p.sendConn` causing a leak.

* prefer existing connection

simplifies flow
2020-08-17 12:38:27 +02:00
Jacek Sieka
b12145dff7
avoid crash when subscribe is received (#333)
...by making subscribeTopic synchronous, avoiding a peer table lookup
completely.

rebalanceMesh will be called a second later - it's fine
2020-08-17 12:10:22 +02:00
Giovanni Petrantoni
eebaeb779f Merge branch 'master' into gossip-one-one 2020-08-17 11:44:23 +09:00
Giovanni Petrantoni
afaa7f2a84 finishup with score and ihave budget 2020-08-17 01:20:50 +09:00
Jacek Sieka
ab864fc747
logging cleanups and small fixes (#331) 2020-08-15 21:50:31 +02:00
Giovanni Petrantoni
1661344e17 oversub prune score based, finish outbound quota 2020-08-15 16:25:29 +09:00
Jacek Sieka
397f9edfd4
simplify mplex (#327)
* less async
* less copying of data
* less redundant cleanup
2020-08-15 07:58:30 +02:00
Jacek Sieka
9c7e055310
set activity flag on noise / secio (#330) 2020-08-15 07:36:15 +02:00
Giovanni Petrantoni
69ff05a4f4 outbound mesh quota, internal tests fixing 2020-08-15 12:51:51 +09:00
Giovanni Petrantoni
309a4998c8 Adaptive gossip dissemination 2020-08-15 11:57:22 +09:00
Giovanni Petrantoni
3e8349b918 IWANT cap/budget 2020-08-14 17:07:26 +09:00
Giovanni Petrantoni
c6fc8dee54 backoff time management 2020-08-14 11:59:19 +09:00
Giovanni Petrantoni
ce61d84db4 add an hard cap to PX 2020-08-13 21:21:17 +09:00
Giovanni Petrantoni
585cfc5996 implement peer exchange graft message 2020-08-13 20:41:54 +09:00
Giovanni Petrantoni
05463873a2 restore explicit peering 2020-08-12 18:25:57 +09:00
Giovanni Petrantoni
296ca14133 rebalance improvements and opportunistic grafting 2020-08-12 16:30:03 +09:00
Dmitriy Ryajov
d1f1e1b31e
add missing mplex half closed test (#326) 2020-08-12 07:23:49 +02:00
Giovanni Petrantoni
f7fee2ca97 further merge fixes 2020-08-12 12:04:54 +09:00
Giovanni Petrantoni
c8954619ab post merge - builds gossip tests 2020-08-12 11:17:36 +09:00
Giovanni Petrantoni
90a33c0a29 Merge branch 'master' into gossip-one-one 2020-08-12 10:57:32 +09:00
Dmitriy Ryajov
b76b3e0e9b
Rework pubsub (#322)
* move pubsub of off switch, pass switch into pubsub

* use join on lpstreams

* properly cleanup up failed peers

* fix tests

* fix peertable hasPeerId

* fix tests

* rework sending, remove helpers from pubsubpeer, unify in broadcast

* further split broadcast into send

* use send where appropriate

* use formatIt

* improve trace

Co-authored-by: Giovanni Petrantoni <giovanni@fragcolor.xyz>
2020-08-11 18:05:49 -06:00
Eugene Kabanov
59b290fcc7
Refactor minasn1 and fix security issues. (#323)
* Refactor minasn1 and fix security issues.

* Fix for RSA test vectors.
2020-08-11 16:58:51 -06:00
Eugene Kabanov
d47b2d805f
Use constant-time hex encoding/decoding procedures explicitly. (#305)
* Use constant-time hex encoding/decoding procedures explicitly.

* Add comments.
2020-08-11 08:48:21 -06:00
Dmitriy Ryajov
2325692f55
Fix half closed (#324)
* don't call `close` in `remoteClose`

* make sure timeout are properly propagted

* fix tests

* adding remote close write test
2020-08-10 16:17:11 -06:00
Giovanni Petrantoni
66c9cd2c8c score fixes 2020-08-08 18:27:57 +09:00
Giovanni Petrantoni
4dcc3b6dda Merge remote-tracking branch 'origin/master' into gossip-one-one 2020-08-08 16:53:14 +09:00
Giovanni Petrantoni
6ffd5be059 fix crash at TRACE log level 2020-08-08 16:31:37 +09:00
Eugene Kabanov
7c1aac5dc1
Use constant-time comparison for keys and signatures. (#299) 2020-08-08 08:53:33 +02:00
Jacek Sieka
f303954989
peer hooks -> events (#320)
* peer hooks -> events

* peerinfo -> peerid
* include connection direction in event
* check connection status after event
* lock connmanager lookup also when dialling peer
* clean up un-upgraded connection when upgrade fails
* await peer eventing

* remove join/lifetime future from peerinfo

Peerinfo instances are not unique per peer so the lifetime future is
misleading - it fires when a random connection is closed, not the "last"
one

* document switch values

* naming

* peerevent->conneevent
2020-08-08 08:52:20 +02:00