76 Commits

Author SHA1 Message Date
Giovanni Petrantoni
255d9d0324 wip parameters 2020-07-15 17:44:23 +09:00
Giovanni Petrantoni
543ebdbdf5 support interop testing 2020-07-15 13:03:10 +09:00
Giovanni Petrantoni
8078fec0f0 Merge branch 'master' into gossip-one-one 2020-07-15 08:47:43 +09:00
Giovanni Petrantoni
92623eea53 gossipsub11 -> gossipsub 2020-07-15 08:27:58 +09:00
Jacek Sieka
c6c2d99907
one more log fix 2020-07-13 20:19:20 +02:00
Jacek Sieka
76853f064a
json logging again 2020-07-13 19:59:49 +02:00
Jacek Sieka
6620b7a00b
more comment fixes 2020-07-13 19:30:18 +02:00
Jacek Sieka
0d4c74b33a
comment log that can't be json-serialized 2020-07-13 18:36:49 +02:00
Jacek Sieka
061c54d3c6
logging fixes 2020-07-13 17:26:05 +02:00
Giovanni Petrantoni
fcda0f6ce1
PubSubPeer tables refactor (#263)
* refactor peer tables

* tests fixing

* override PubSubPeer equality

* fix pubsubpeer comparison
2020-07-13 15:32:38 +02:00
Dmitriy Ryajov
bec9a0658f
Cleanup rpc handler (#261)
* more cleanup

* fix tests

* merging master

* remove `withLock` as it conflicts with stdlib

* wip

* more fanout ttl

Co-authored-by: Giovanni Petrantoni <giovanni@fragcolor.xyz>
2020-07-09 17:54:16 -06:00
Dmitriy Ryajov
4c815d75e7
More gossip cleanup (#257)
* more cleanup

* correct pubsub peer count

* close the stream first

* handle cancelation

* fix tests

* fix fanout ttl

* merging master

* remove `withLock` as it conflicts with stdlib

* fix trace build

Co-authored-by: Giovanni Petrantoni <giovanni@fragcolor.xyz>
2020-07-09 14:21:47 -06:00
Jacek Sieka
c720e042fc
clean up mesh handling logic (#260)
* gossipsub is a function of subscription messages only
* graft/prune work with mesh, get filled up from gossipsub
* fix race conditions with await
* fix exception unsafety when grafting/pruning
* fix allowing up to DHi peers in mesh on incoming graft
* fix metrics in several places
2020-07-09 11:16:46 -06:00
Giovanni Petrantoni
4e12d0d97a nil check peer before disconnect 2020-07-09 17:20:45 +09:00
Giovanni Petrantoni
4bcb567d47 fix gossip tests 2020-07-09 12:34:36 +09:00
Giovanni Petrantoni
fec507e755
Add peers back to gossipsub table, slow down heartbeat (#256)
* Add peers back to gossipsub table, slow down heartbeat

* exclude on unsub from mesh and fanout
2020-07-08 11:06:26 -06:00
Giovanni Petrantoni
853250918c wip 2020-07-08 15:45:11 +09:00
Giovanni Petrantoni
be98a478b4 Fix merge oops 2020-07-08 12:58:39 +09:00
Giovanni Petrantoni
38422ff4cb Merge branch 'master' into gossip-one-one 2020-07-08 11:39:33 +09:00
Dmitriy Ryajov
a52763cc6d
fix publishing (#250)
* use var semantics to optimize table access

* wip... lvalues don't work properly sadly...

* big publish refactor, replenish and balance

* fix internal tests

* use g.peers for fanout (todo: don't include flood peers)

* exclude non gossip from fanout

* internal test fixes

* fix flood tests

* fix test's trypublish

* test interop fixes

* make sure to not remove peers from gossip table

* restore old replenishFanout

* cleanups

* Cleanup resources (#246)

* consolidate reading in lpstream

* remove debug echo

* tune log level

* add channel cleanup and cancelation handling

* cancelation handling

* cancelation handling

* cancelation handling

* cancelation handling

* cleanup and cancelation handling

* cancelation handling

* cancelation

* tests

* rename isConnected to connected

* remove testing trace

* comment out debug stacktraces

* explicit raises

* restore trace vs debug in gossip

* improve fanout replenish behavior further

* cleanup stale peers more eaguerly

* synchronize connection cleanup and small refactor

* close client first and call parent second

* disconnect failed peers on publish

* check for publish result

* fix tests

* fix tests

* always call close

Co-authored-by: Giovanni Petrantoni <giovanni@fragcolor.xyz>
2020-07-07 18:33:05 -06:00
Giovanni Petrantoni
73cefce798 remove assertion in publish 2020-07-07 19:42:36 +09:00
Giovanni Petrantoni
479c83c1df with sugar 2020-07-04 19:36:15 +09:00
Giovanni Petrantoni
5e0870d4a4 minor fixes 2020-07-04 14:22:58 +09:00
Giovanni Petrantoni
aed8dcadaf actually lock the heartbeat 2020-07-04 14:09:42 +09:00
Giovanni Petrantoni
ad7db1ca26 add more heartbeat locking to prevent races 2020-07-04 14:04:44 +09:00
Giovanni Petrantoni
0d0309a601 better diagnostics 2020-07-03 09:28:20 +09:00
Giovanni Petrantoni
27d22d534d simplify rebalanceMesh (following spec) and make it finally reach D_high 2020-07-02 23:07:17 +09:00
Giovanni Petrantoni
5e4934bcce Merge branch 'master' into publish-backporting 2020-07-02 17:44:23 +09:00
Giovanni Petrantoni
d9e0ca6091 getGossipPeers fixes 2020-07-01 17:26:42 +09:00
Giovanni Petrantoni
a976c22dae triage publish nil peers (issue is on master too but just hidden behind a if/in) 2020-07-01 16:54:28 +09:00
Giovanni Petrantoni
ec00c7fc50
Peer resultification and defect only (#245)
* Peer resultification and defect only

* Fixing some tests

* test fixes

* Rename peer into peerid

* better result error message in identify

* further merge fixes
2020-07-01 08:25:09 +02:00
Giovanni Petrantoni
f49e59cb4e improve fanout replenish behavior further 2020-06-30 11:36:47 +09:00
Giovanni Petrantoni
509d84f03d restore trace vs debug in gossip 2020-06-30 11:28:45 +09:00
Giovanni Petrantoni
9d33390239 restore utility module import 2020-06-30 00:17:58 +09:00
Dmitriy Ryajov
c788a6a3c0
Cleanup resources (#246)
* consolidate reading in lpstream

* remove debug echo

* tune log level

* add channel cleanup and cancelation handling

* cancelation handling

* cancelation handling

* cancelation handling

* cancelation handling

* cleanup and cancelation handling

* cancelation handling

* cancelation

* tests

* rename isConnected to connected

* remove testing trace

* comment out debug stacktraces

* explicit raises
2020-06-29 09:15:31 -06:00
Giovanni Petrantoni
680dcfd9cb cleanups 2020-06-30 00:10:11 +09:00
Giovanni Petrantoni
3a657135dc Merge branch 'master' into publish-backporting 2020-06-29 22:02:58 +09:00
Giovanni Petrantoni
3dd28adc9f restore old replenishFanout 2020-06-29 10:46:59 +09:00
Giovanni Petrantoni
2356f6ab68 make sure to not remove peers from gossip table 2020-06-29 10:45:06 +09:00
Jacek Sieka
aa6756dfe0
allow message id provider to be specified (#243)
* don't send public key in message when not signing (information leak)
* don't run rebalance if there are peers in gossip (see #242)
* don't crash randomly on bad peer id from remote
2020-06-28 09:56:38 -06:00
Giovanni Petrantoni
b7382eaf25 exclude non gossip from fanout 2020-06-28 10:55:44 +09:00
Giovanni Petrantoni
8e2c9301a0 use g.peers for fanout (todo: don't include flood peers) 2020-06-28 10:53:43 +09:00
Giovanni Petrantoni
ccb323ad0f big publish refactor, replenish and balance 2020-06-27 20:24:29 +09:00
Giovanni Petrantoni
dc3c568dc0 wip... lvalues don't work properly sadly... 2020-06-27 14:59:48 +09:00
Giovanni Petrantoni
1a2f336eb5 use var semantics to optimize table access 2020-06-27 13:35:07 +09:00
Giovanni Petrantoni
7852c6dd0f
Noise and eth2/nbc fixes (#226)
* Remove noise padding payload (spec removed it)

* add log scope in secure

* avoid defect array out of range in switch secure when "na"

* improve identify traces

* wip noise fixes

* noise protobuf adjustments (trying)

* add more debugging messages/traces, improve their actual contents

* re-enable ID check in noise

* bump go daemon tag version

* bump go daemon tag version

* enable noise in daemonapi

* interop testing, (both secio and noise will be tested)

* azure cache bump (p2pd)

* CI changes

- Travis: use Go 1.14
- azure-pipelines.yml: big cleanup
- Azure: bump cache keys
- build 64-bit p2pd on 32-bit Windows
- install both Mingw-w64 architectures

* noise logging fixes

* alternate testing between noise and secio

* increase timeout to avoid VM errors in CI (multistream tests)

* refactor heartbeat management in gossipsub

* remove locking within heartbeat

* refactor heartbeat management in gossipsub

* remove locking within heartbeat

Co-authored-by: Ștefan Talpalaru <stefantalpalaru@yahoo.com>
2020-06-20 19:56:55 +09:00
Dmitriy Ryajov
7a1c1c2ea6
fixing some key not found exceptions (#231) 2020-06-19 15:19:07 -06:00
Dmitriy Ryajov
5b28e8c488
Cleanup lpstream, Connection and BufferStream (#228)
* count published messages

* don't call `switch.dial` in `subscribeToPeer`

* don't use delegation in connection

* move connection out to own file

* don't breakout on reset

* make sure to call close on secured conn

* add lpstream tracing

* don't breackdown by conn id

* fix import

* remove unused lable

* reset  connection on exception

* add additional metrics for skipped messages

* check for nil in secure.close
2020-06-19 11:29:43 -06:00
Dmitriy Ryajov
719744f46a
Small fixes (#230)
* count published messages

* don't call `switch.dial` in `subscribeToPeer`

* make sure sending doesn't fail

* add `contains`

* review comment from prev pr
2020-06-19 11:29:25 -06:00
Dmitriy Ryajov
fe828d87d8
count published messages (#224) 2020-06-16 22:14:02 -06:00