112 Commits

Author SHA1 Message Date
Giovanni Petrantoni
0776cc77e7 Merge branch 'master' into gossip-one-one 2020-07-21 11:02:02 +09:00
Giovanni Petrantoni
3b088f8980
Fix some unsubscribe issues and add unsubscribeAll helper (#282)
* Fix some unsub issues and add unsuball helper

* batch sendprune in unsubscribe methods

* add unsubscribeAll for floodsub
2020-07-20 10:16:13 -06:00
Giovanni Petrantoni
88ace2c5dd Merge branch 'master' into gossip-one-one 2020-07-18 12:08:25 +09:00
Dmitriy Ryajov
ba071cafa6
Channel timeout (#278)
* add support for channel timeouts

* tests for channel timeout

* add timeouts to standard switch

* fix mplex init

* cleanup timer on stream close

* add comment for `isConnected`

* move cleanup event
2020-07-17 12:44:41 -06:00
Dmitriy Ryajov
0348773ec9
Connection manager (#277)
* splitting out connection management

* wip

* wip conn mngr tests

* set peerinfo in contructor

* comments and documentation

* tests

* wip

* add `None` to detect untagged connections

* use `PeerID` to index connections

* fix tests

* remove useless equality
2020-07-17 09:36:48 -06:00
Giovanni Petrantoni
ef410509e6 tests fixing 2020-07-15 15:53:19 +09:00
Giovanni Petrantoni
8078fec0f0 Merge branch 'master' into gossip-one-one 2020-07-15 08:47:43 +09:00
Ștefan Talpalaru
b8b0a2b4bc
CI: build binaries with TRACE & JSON logs (#268)
Also: remove unused imports.
2020-07-14 02:02:16 +02:00
Dmitriy Ryajov
4c815d75e7
More gossip cleanup (#257)
* more cleanup

* correct pubsub peer count

* close the stream first

* handle cancelation

* fix tests

* fix fanout ttl

* merging master

* remove `withLock` as it conflicts with stdlib

* fix trace build

Co-authored-by: Giovanni Petrantoni <giovanni@fragcolor.xyz>
2020-07-09 14:21:47 -06:00
Giovanni Petrantoni
a7d90214ea Merge branch 'gossip-one-one' of github.com:status-im/nim-libp2p into gossip-one-one 2020-07-08 11:52:54 +09:00
Giovanni Petrantoni
0fbc0ec4b2 fix multistream 1.1 multi proto 2020-07-08 11:52:44 +09:00
Giovanni Petrantoni
38422ff4cb Merge branch 'master' into gossip-one-one 2020-07-08 11:39:33 +09:00
Dmitriy Ryajov
a52763cc6d
fix publishing (#250)
* use var semantics to optimize table access

* wip... lvalues don't work properly sadly...

* big publish refactor, replenish and balance

* fix internal tests

* use g.peers for fanout (todo: don't include flood peers)

* exclude non gossip from fanout

* internal test fixes

* fix flood tests

* fix test's trypublish

* test interop fixes

* make sure to not remove peers from gossip table

* restore old replenishFanout

* cleanups

* Cleanup resources (#246)

* consolidate reading in lpstream

* remove debug echo

* tune log level

* add channel cleanup and cancelation handling

* cancelation handling

* cancelation handling

* cancelation handling

* cancelation handling

* cleanup and cancelation handling

* cancelation handling

* cancelation

* tests

* rename isConnected to connected

* remove testing trace

* comment out debug stacktraces

* explicit raises

* restore trace vs debug in gossip

* improve fanout replenish behavior further

* cleanup stale peers more eaguerly

* synchronize connection cleanup and small refactor

* close client first and call parent second

* disconnect failed peers on publish

* check for publish result

* fix tests

* fix tests

* always call close

Co-authored-by: Giovanni Petrantoni <giovanni@fragcolor.xyz>
2020-07-07 18:33:05 -06:00
Giovanni Petrantoni
57e6a95b9c Merge branch 'publish-backporting' into gossip-one-one 2020-07-07 21:54:07 +09:00
Giovanni Petrantoni
2d0e285810 Merge branch 'master' into publish-backporting 2020-07-07 20:58:58 +09:00
Jacek Sieka
d522537b19
reuse single RNG instance for all crypto key generation (#249)
* reuse single RNG instance for all crypto key generation

* use foolproof rng

* initRng -> newRng (because it's ref)

* fix test

* imports/exports, chat fix

* fix rsa

* imports and exports

* work around threadvar issue

* fixup

* mac workaround test
2020-07-07 13:14:11 +02:00
Giovanni Petrantoni
be0375f840 fix up merge 2020-07-04 11:51:09 +09:00
Giovanni Petrantoni
afb5cb88f9 Merge remote-tracking branch 'origin/publish-backporting' into gossip-one-one 2020-07-04 11:42:16 +09:00
Giovanni Petrantoni
5e4934bcce Merge branch 'master' into publish-backporting 2020-07-02 17:44:23 +09:00
Giovanni Petrantoni
ec00c7fc50
Peer resultification and defect only (#245)
* Peer resultification and defect only

* Fixing some tests

* test fixes

* Rename peer into peerid

* better result error message in identify

* further merge fixes
2020-07-01 08:25:09 +02:00
Dmitriy Ryajov
c788a6a3c0
Cleanup resources (#246)
* consolidate reading in lpstream

* remove debug echo

* tune log level

* add channel cleanup and cancelation handling

* cancelation handling

* cancelation handling

* cancelation handling

* cancelation handling

* cleanup and cancelation handling

* cancelation handling

* cancelation

* tests

* rename isConnected to connected

* remove testing trace

* comment out debug stacktraces

* explicit raises
2020-06-29 09:15:31 -06:00
Giovanni Petrantoni
3a657135dc Merge branch 'master' into publish-backporting 2020-06-29 22:02:58 +09:00
Dmitriy Ryajov
902880ef1f
consolidate reading in lpstream (#241)
* consolidate reading in lpstream

* remove debug echo

* throw if not enough bytes where read

* tune log level

* set eof flag

* test readExactly to fail on not enough bytes
2020-06-27 11:33:34 -06:00
Giovanni Petrantoni
dc3c568dc0 wip... lvalues don't work properly sadly... 2020-06-27 14:59:48 +09:00
Dmitriy Ryajov
7a95f1844b
Concurrent dials (#238)
* count published messages

* don't call `switch.dial` in `subscribeToPeer`

* add secureconn constructor

* close in the correct order

* concurent dial lock and track in/out conns better

* make tests pass

* add todo comment

* disconect peers that open too many connections

* wip

* do connection and muxer tracking in one place

* prevent nil pointer in observers

* drop connections when peers is over max

* prevent channel leaks

* don't use closure to handle channel
2020-06-24 09:08:44 -06:00
Giovanni Petrantoni
f8292f7086 explicit peering part 2 2020-06-23 15:56:07 +09:00
Giovanni Petrantoni
b3aebb18e9 explicit peering part 1 2020-06-23 14:23:00 +09:00
Giovanni Petrantoni
ee6e545878
multistream select make sure to not report NA (#235)
* multistream select make sure to not report NA but rather empty string if all fails

Also re-enable tests

* avoid using bad constructs, make multistream.select flow crystal clear
2020-06-22 15:38:48 -06:00
Jacek Sieka
b99fd88deb
logging fixes 2020-06-21 11:14:19 +02:00
Giovanni Petrantoni
7852c6dd0f
Noise and eth2/nbc fixes (#226)
* Remove noise padding payload (spec removed it)

* add log scope in secure

* avoid defect array out of range in switch secure when "na"

* improve identify traces

* wip noise fixes

* noise protobuf adjustments (trying)

* add more debugging messages/traces, improve their actual contents

* re-enable ID check in noise

* bump go daemon tag version

* bump go daemon tag version

* enable noise in daemonapi

* interop testing, (both secio and noise will be tested)

* azure cache bump (p2pd)

* CI changes

- Travis: use Go 1.14
- azure-pipelines.yml: big cleanup
- Azure: bump cache keys
- build 64-bit p2pd on 32-bit Windows
- install both Mingw-w64 architectures

* noise logging fixes

* alternate testing between noise and secio

* increase timeout to avoid VM errors in CI (multistream tests)

* refactor heartbeat management in gossipsub

* remove locking within heartbeat

* refactor heartbeat management in gossipsub

* remove locking within heartbeat

Co-authored-by: Ștefan Talpalaru <stefantalpalaru@yahoo.com>
2020-06-20 19:56:55 +09:00
Dmitriy Ryajov
5b28e8c488
Cleanup lpstream, Connection and BufferStream (#228)
* count published messages

* don't call `switch.dial` in `subscribeToPeer`

* don't use delegation in connection

* move connection out to own file

* don't breakout on reset

* make sure to call close on secured conn

* add lpstream tracing

* don't breackdown by conn id

* fix import

* remove unused lable

* reset  connection on exception

* add additional metrics for skipped messages

* check for nil in secure.close
2020-06-19 11:29:43 -06:00
Dmitriy Ryajov
719744f46a
Small fixes (#230)
* count published messages

* don't call `switch.dial` in `subscribeToPeer`

* make sure sending doesn't fail

* add `contains`

* review comment from prev pr
2020-06-19 11:29:25 -06:00
Dmitriy Ryajov
5f75e5adc6
don't use switch.dial in subscribeToPeer (#227)
* count published messages

* don't call `switch.dial` in `subscribeToPeer`
2020-06-17 20:12:32 -06:00
Dmitriy Ryajov
fe828d87d8
count published messages (#224) 2020-06-16 22:14:02 -06:00
Jacek Sieka
523eed65ce
metrics: use eth2 name for peers metric (#223) 2020-06-16 14:43:45 -06:00
Dmitriy Ryajov
6b196ad7b4
remove pubsub peer on disconnect (#212)
* remove pubsub peer on disconnect

* make sure lock is aquired

* add $

* count upgrades/dials/disconnects
2020-06-11 08:45:59 -06:00
Viktor Kirilov
1afec627c2
proper name for topics so that we can filter dynamically using chronicles (#210)
* proper name for topics so that we can filter dynamically using chronicles

* lowercase
2020-06-10 10:48:01 +02:00
Jacek Sieka
8d9e231a74
grab agentversion/protoversion (#211) 2020-06-09 12:42:52 -06:00
Dmitriy Ryajov
35ff99829e
close streams (#208)
* close streams

* don't warn on missing proto
2020-06-08 17:42:27 -06:00
Dmitriy Ryajov
130c64f33a
don't return nil in dial (#205)
* dont return nil in dial

* don't crash on pubsub send
2020-06-05 18:17:05 -06:00
Dmitriy Ryajov
bb8bff2195
add sparse message propagation tests to gossipsub (#202)
* add sparce tests to gossipsub

* add send hooks

* remove `all`
2020-06-02 17:53:38 -06:00
Dmitriy Ryajov
285884c20c
Close peers (#201)
* wip

* exceptions and resource cleanup

* correct peerlifetime on disconnect

* emulate defered

* remove comment
2020-06-02 11:32:42 -06:00
Dmitriy Ryajov
abf659a01a more consistent dialing proto selecting logic 2020-06-02 09:10:27 -06:00
Dmitriy Ryajov
815282a5da remove all() 2020-06-02 09:10:27 -06:00
Dmitriy Ryajov
abc12d0fb5 move stram close to a better location 2020-06-02 09:10:27 -06:00
Dmitriy Ryajov
640c3bdc45 better exception handling and resource cleanup 2020-06-02 09:10:27 -06:00
Giovanni Petrantoni
37b98ad45c
Secure managers are now sorted, giving priority to noise (#191)
* Secure managers are now sorted, giving priority to noise

* fix nimble test command

* Fix native tests

* fix directchat sample

* Could not write to connection - reduce verbosity

* fix interop testing

* Remove more tables

* test interop fixes

* directchat fix

* fix interop/remove some deprecation
2020-06-01 08:41:32 +02:00
Dmitriy Ryajov
7b6e1c0688
Gossipsub interop (#189)
* interop fixes

* add custom messageid provider and fix seqno

* use ECDSA for speed

* adding messageid tests

* breakout from publish loop

* addressing review comments

* remove unneded var

* dont stop broadcasting on failed peers
2020-05-27 12:33:49 -06:00
Dmitriy Ryajov
ba53c08b3c
Track incoming connections (#181)
* call write until all is written out

* wip: rework with proper half-closed

* add eof and closed handling

* wip

* close connection on chronos close

* don't use read

* make noise work again

* don't reraise just yet

* fixes after backporting

* remove on transport close cleanup

* revert back allread

* rust interop fixes

* read from stream

* inc count before closing

* rebasing master

* store incomming connections

* fix merge

* remove unneeded changes

* use internal close flag to indicate disposal
2020-05-21 11:33:48 -06:00
Dmitriy Ryajov
5583168965
Track all connections (#175)
* adding to do for future refactor

* no mix conn and transport stopping or it will race

* don't use intermediary vars
2020-05-18 09:08:10 -06:00