a9948b0b05
clarify validation messages ( #431 )
...
* clarify validation messages
* add codecov threshold
2020-11-12 01:42:12 +09:00
e85569800c
Codecov extras ( #430 )
...
* try fix codecov html artifacts path
* wip
* wip
* wip
* wip
* fix options matrix
2020-11-11 23:22:39 +09:00
96ed9fbe85
start by building with codecov on (linux) ( #424 )
...
* start by building with codecov on (linux)
* travis coverage builds
* wip
* wip
* wip
* wip
* codecov.io integration
* merge multiple coverage
* improve nimble codecov
* fileExists
* wip codecov CI refactor
* add nimble install
* wip
* fix path
* wip
* add lcov command
* try bump ubuntu...
* wip
* add genhtml to debug
* wip
* wip
* fix lcov extract
* wip
* wip
* try again exporting coverage htmls
* make artifacts unique
* another attempt
* fix env var
* restore regular travis and nimble file
2020-11-10 17:16:39 +09:00
90921bff09
move some importance trace logs to debug ( #428 )
2020-11-09 22:14:46 -06:00
331961ef14
dont use double allFinished in mplex tests ( #427 )
2020-11-06 12:20:59 -06:00
4fb3f50d2c
Reset channels on close ( #425 )
...
* reset when failed to read/write muxed conn
* add more comprehensive resource cleanup tests
* style
* cleanup tests
2020-11-06 09:24:24 -06:00
3956f3fd69
make sure all streams are tracked ( #422 )
...
* make sure all streams are tracked
* revert unnecesary change
2020-11-04 21:52:54 -06:00
6040cb4ef1
fix debugutils ( #423 )
2020-11-04 19:56:28 -06:00
7cc42ce219
start adding more tests + minor fixes ( #419 )
...
* start adding more tests + minor fixes
* add wrong secure negotiation test
* add noise failed handshake test
2020-11-04 23:24:41 +09:00
e496802943
Least expensive metrics ( #421 )
...
* add more general and useful metrics
* fix gossipsub peers metrics in heartbeat
2020-11-04 15:18:00 +01:00
7b5259dbc7
Move triggers ( #416 )
...
* move event triggers to connmanager
* use base error type
* avoid deadlocks
* handle eof and closed when identifying incoming
* use `closeWait`
2020-11-02 14:35:26 -06:00
43a77e60a1
split stream counts by direction ( #418 )
2020-11-01 16:23:26 -06:00
03639f1446
Revert "Channel leaks ( #413 )" ( #417 )
...
This reverts commit 1de1d49223
2020-11-01 14:49:25 -06:00
9c1633bf87
fix ValidIpAddress multiaddress init return type
2020-10-31 13:20:29 +09:00
04c95cb7b0
Fix write should be writeArray.
2020-10-31 03:23:34 +02:00
ff48d0b1a2
Proper fix for init(ValidIpAddress).
2020-10-30 17:52:38 +02:00
3d9948a65e
ensure all multiaddress routines use Result
2020-10-30 23:50:04 +09:00
75b023c9e5
gossipsub audit fixes ( #412 )
...
* [SEC] gossipsub - rebalanceMesh grafts peers giving preference to low scores #405
* comment score choices
* compiler warning fixes/bug fixes (unsubscribe)
* rebalanceMesh does not enforce D_out quota
* fix outbound grafting
* fight the nim compiler
* fix closure capture bs...
* another closure fix
* #403 rebalance prune fixes
* more test fixing
* #403 fixes
* #402 avoid removing scores on unsub
* #401 handleGraft improvements
* [SEC] handleIHAVE/handleIWANT recommendations
* add a note about peer exchange handling
2020-10-30 21:49:54 +09:00
1de1d49223
Channel leaks ( #413 )
...
* break stream tracking by type
* use closeWithEOF to await wrapped stream
* fix cancelation leaks
* fix channel leaks
* logging
* use close monitor and always call closeUnderlying
* don't use closeWithEOF
* removing close monitor
* logging
2020-10-27 11:21:03 -06:00
eeaa62feec
add more debug details to multiaddress assertions
2020-10-21 18:29:59 +09:00
462da1f7a8
gossip MessageID as seq[byte] ( #391 )
...
* gossip MessageID as seq[byte]
* combina hashes in defaultMsgIdProvider
* wip
* fix defaultMsgIdProvider
2020-10-21 12:26:04 +09:00
27b9bf436e
fix validation according to specification ( #410 )
2020-10-21 12:25:42 +09:00
5c19668b2d
avoid verbose EOF messages in readOnce(secure) ( #411 )
...
* avoid verbose EOF messages in readOnce(secure)
* shorten azure tests further
2020-10-21 10:08:24 +09:00
9c58356823
change repository name for nimbus auto-bump
2020-10-19 14:39:05 +09:00
32623b930e
handle secure errors in readOnce (secure) ( #397 )
...
* handle secure errors in readOnce(secure)
* small synthax fix
* fix mistake in readOnce's isNil
2020-10-19 14:13:14 +09:00
bd70515087
fix first interop test ( #398 )
2020-10-15 10:37:20 +09:00
556213abf4
Extended validators ( #395 )
...
* gossip extended validation
* fix flood tests
* fix gossip 1.0 tests
* synthax consistency
2020-10-12 16:56:00 +09:00
c81b665b0d
replace nim-beacon-chain URLs with nimbus-eth2 URLs ( #396 )
2020-10-09 18:02:03 +02:00
e3bdb9eb13
decode properly ControlPrune ( #392 )
2020-10-09 09:12:38 +09:00
98d82fce5c
fix opportunistic graft in internal 11 testing ( #390 )
2020-10-05 11:35:03 +09:00
0f2435f551
better opportunistic grafting score (when score is disabled) ( #389 )
2020-10-03 09:26:45 +09:00
853238a215
feature/expose-matcher ( #387 )
...
* exposes matcher
* might work
* fix
* fix
2020-10-02 08:59:15 -06:00
4a98a8af5a
gossip pruning fixes related to #371 ( #385 )
...
* gossip pruning fixes related to #371
* better trace for grafted/pruned
* shorted azure testing again
2020-10-02 13:09:31 +09:00
03f5bbba6d
saner logging ( #381 )
2020-09-29 09:40:06 -06:00
98d0cc3a16
defaultMsgIdProvider alternative/test anonymize ( #379 )
...
* defaultMsgIdProvider alternative/test anonymize
* avoid freeze during flood tests
* avoid `empty message, skipping` situation
* test observers
* avoid double initPubSub
* fix gossip testing (specially when anonymize is on)
* make azure tests shorter
2020-09-28 09:11:18 +02:00
8ecef46738
reencode gossipsub messages with anonymization ( #378 )
...
This helps protect against clients sending more data than they should
and thus getting penalized on topics that require anonymity
2020-09-25 18:39:34 +02:00
17e00e642a
limit write queue length ( #376 )
...
To break a potential read/write deadlock, gossipsub uses an unbounded
queue for writes - when peers are too slow to process this queue, it may
end up growing without bounds causing high memory usage.
Here, we introduce a maximum write queue length after which the peer is
disconnected - the queue is generous enough that any "normal" usage
should be fine - writes that are `await`:ed are not affected, only
writes that are launched in an `asyncSpawn` task or similar.
* avoid unnecessary copy of message when there are no send observers
* release message memory earlier in gossipsub
* simplify pubsubpeer logging
2020-09-24 18:43:20 +02:00
58f26d4164
temporarily skip broken test on windows ( #375 )
2020-09-24 14:50:04 +02:00
25bd0a18f4
small fixes ( #374 )
...
* add helper to read EOF marker after closing stream (else stream stay
alive until timeout/reset)
* don't assert on empty channel message
* don't loop when writing to chronos (no need)
2020-09-24 07:30:19 +02:00
ec322124ac
allow to omit peerId and seqno ( #372 )
...
* allow to omit peerId and seqno
* small refactor
* wip
* fix message encoding
* improve rpc signature logic
* remove peerid from verify
* trace fixes
* fix message test
* fix gossip 1.0 tests
2020-09-23 17:56:33 +02:00
abd234601b
move events to conn manager ( #373 )
2020-09-23 08:07:16 -06:00
471e5906f6
fix gossipsub memory leak on disconnected peer ( #371 )
...
When messages can't be sent to peer, we try to establish a send
connection - this causes messages to stack up as more and more unsent
messages are blocked on the dial lock.
* remove dial lock
* run reconnection loop in background task
2020-09-22 09:05:53 +02:00
49a12e619d
channel close race and deadlock fixes ( #368 )
...
* channel close race and deadlock fixes
* remove send lock, write chunks in one go
* push some of half-closed implementation to BufferStream
* fix some hangs where LPChannel readers and writers would not always
wake up
* simplify lazy channels
* fix close happening more than once in some orderings
* reenable connection tracking tests
* close channels first on mplex close such that consumers can read bytes
A notable difference is that BufferedStream is no longer considered EOF
until someone has actually read the EOF marker.
* docs, simplification
2020-09-21 19:48:19 +02:00
b99d2039a8
Gossip one one ( #240 )
...
* allow multiple codecs per protocol (without breaking things)
* add 1.1 protocol to gossip
* explicit peering part 1
* explicit peering part 2
* explicit peering part 3
* PeerInfo and ControlPrune protocols
* fix encodePrune
* validated always, even explicit peers
* prune by score (score is stub still)
* add a way to pass parameters to gossip
* standard setup fixes
* take into account explicit direct peers in publish
* add floodPublish logic
* small fixes, publish still half broken
* make sure to waitsub in sparse test
* use var semantics to optimize table access
* wip... lvalues don't work properly sadly...
* big publish refactor, replenish and balance
* fix internal tests
* use g.peers for fanout (todo: don't include flood peers)
* exclude non gossip from fanout
* internal test fixes
* fix flood tests
* fix test's trypublish
* test interop fixes
* make sure to not remove peers from gossip table
* restore old replenishFanout
* cleanups
* restore utility module import
* restore trace vs debug in gossip
* improve fanout replenish behavior further
* triage publish nil peers (issue is on master too but just hidden behind a if/in)
* getGossipPeers fixes
* remove topics from pubsubpeer (was unused)
* simplify rebalanceMesh (following spec) and make it finally reach D_high
* better diagnostics
* merge new pubsubpeer, copy 1.1 to new module
* fix up merge
* conditional enable gossip11 module
* add back topics in peers, re-enable flood publish
* add more heartbeat locking to prevent races
* actually lock the heartbeat
* minor fixes
* with sugar
* merge 1.0
* remove assertion in publish
* fix multistream 1.1 multi proto
* Fix merge oops
* wip
* fix gossip 11 upstream
* gossipsub11 -> gossipsub
* support interop testing
* tests fixing
* fix directchat build
* control prune updates (pb)
* wip parameters
* gossip internal tests fixes
* parameters wip
* finishup with params
* cleanups/wip
* small sugar
* grafted and pruned procs
* wip updateScores
* wip
* fix logging issue
* pubsubpeer, chronicles explicit override
* fix internal gossip tests
* wip
* tables troubleshooting
* score wip
* score wip
* fixes
* fix test utils generateNodes
* don't delete while iterating in score update
* fix grafted defect
* add a handleConnect in subscribeTopic
* pruning improvements
* wip
* score fixes
* post merge - builds gossip tests
* further merge fixes
* rebalance improvements and opportunistic grafting
* fix test for now
* restore explicit peering
* implement peer exchange graft message
* add an hard cap to PX
* backoff time management
* IWANT cap/budget
* Adaptive gossip dissemination
* outbound mesh quota, internal tests fixing
* oversub prune score based, finish outbound quota
* finishup with score and ihave budget
* use go daemon 0.3.0
* import fixes
* byScore cleanup score sorting
* remove pointless scaling in `/` Duration operator
* revert using libp2p org for daemon
* interop fixes
* fixes and cleanup
* remove heartbeat assertion, minor debug fixes
* logging improvements and cleaning up
* (to revert) add some traces
* add explicit topic to gossip rpcs
* pubsub merge fixes and type fix in switch
* Revert "(to revert) add some traces"
This reverts commit 4663eaab6cc336c81cee50bc54025cf0b7bcbd99.
* cleanup some now irrelevant todo
* shuffle peers anyway as score might be disabled
* add missing shuffle
* old merge fix
* more merge fixes
* debug improvements
* re-enable gossip internal tests
* add gossip10 fallback (dormant but tested)
* split gossipsub internal tests into 1.0 and 1.1
Co-authored-by: Dmitriy Ryajov <dryajov@gmail.com>
2020-09-21 11:16:29 +02:00
b7e5d1122c
cleanups ( #366 )
...
* reuse connection timeout for noise handshake (avoid extra timer)
* enforce nbytes > 0 for readOnce
* avoid some unnecessary memory zeroing
* simplify noise
* fix dumping when noise splits message
2020-09-16 11:55:25 +02:00
b0d86b95dd
add peer lifecycle events ( #357 )
...
* add peer lifecycle events
* rework peer events to not use connection events
* don't use result in pubsub and switch init
* wip
* use ordered hashes and remove logscope
* logging
* add missing test
* small fixes
2020-09-15 14:19:22 -06:00
a6007be428
avoid sending empty seqno and/or fromPeer (gossip rpc) ( #364 )
2020-09-15 12:33:18 +02:00
e2d46c6b53
Add `libp2p_dump` and `libp2p_dump_dir` compiler time options. ( #359 )
...
* Add `libp2p_dump` and `libp2p_dump_dir` compiler time options.
* Add tools/pbcap_parser.
* Add mplex control messages decoding.
2020-09-15 11:16:43 +02:00
5e66f6fbd8
Add logScope to connmanager and pubsubprotobuf ( #363 )
2020-09-15 08:03:53 +02:00
0db45462cd
mplex fixes ( #362 )
...
* remove almost-empty types module
* lock when writing message (that's the only place the lock matters, and
only when the message is > max msg size)
* logging updates (log in consistent order, makes reading logs easier)
* raise EOF from readExactly only if no bytes have been read (to signal
that _no_ bytes were lost)
2020-09-14 10:19:54 +02:00
Giovanni Petrantoni
Dmitriy Ryajov
Jacek Sieka
Giovanni Petrantoni
cheatfate
Giovanni Petrantoni
tersec
Dean Eigenmann
Mamy Ratsimbazafy
Oskar Thorén