Commit Graph

983 Commits

Author SHA1 Message Date
Giovanni Petrantoni 6c2e743ff3
Race condition in pubsub #469 (#471)
* Race condition in pubsub #469

* use allFinished

* improve cancellation handling
2020-12-19 00:56:46 +09:00
Jacek Sieka a1a5f9abac
nice msgid log formatting (#491) 2020-12-18 16:14:07 +01:00
Giovanni Petrantoni 52628d1a2e avoid using debug logging in gossip, just because 2020-12-17 17:21:09 +09:00
Jacek Sieka 0c331d5200
simplify noise frame construction (#478) 2020-12-16 13:10:06 +01:00
Giovanni Petrantoni 051681678b better codecov.yml comments 2020-12-16 13:07:22 +09:00
Giovanni Petrantoni a50b1c186c
Improve PR codecov output (#482)
* wip

* set 48 as n builds

* fixes
2020-12-16 13:04:57 +09:00
Jacek Sieka 9e5ba64c48
avoid creating prune message unless we're pruning (#487) 2020-12-15 22:46:03 +01:00
Dmitriy Ryajov ea6988d380
track stew head (#488) 2020-12-15 14:12:22 -06:00
Giovanni Petrantoni 18d69a5c41
Workaround the gossipsub table race condition (#486) 2020-12-15 12:32:11 -06:00
Jacek Sieka b52dab9fd7
use stew/leb128 (#481)
* avoids multiple reallocations in readLp
* simplifies varint implementation
* remove vbuffer.length (unused)
2020-12-15 12:15:22 -06:00
Giovanni Petrantoni 5543f6681f
first pass, use results for Cid module (#480)
* first pass, use results for Cid module

* improvements to decode
2020-12-15 14:19:18 +01:00
Dmitriy Ryajov a990fe95a0
Fixing range error introduces in v1.2.8 (#485) 2020-12-15 06:58:38 +01:00
Giovanni Petrantoni f8f0bc1bd8
Gossip improvements (#476)
* add more traces, remove async from rebalance

* more traces

* avoid computng scores when weight is 0.0

* debug colocation, fix an indent in unsubpeer (minor)

* add full ValidationResult coverage

* store in cache only after validation

* gossip 1.0 fixes

* fix typo

* gossip 10 internal test fixes

* test fixing

* refactor peerstats usages

* populate tables if missing when scoring
2020-12-15 10:25:22 +09:00
Dmitriy Ryajov 4224f12503
fix memory safety issue in tests (#484) 2020-12-14 15:22:53 -06:00
Mamy Ratsimbazafy 42d264d8b0
Rm bearssl + Deactivate Travis completely (#477)
* Rm bearssl added in #2167

* Travis ARM doesn't work
2020-12-10 14:19:27 +01:00
Mamy André-Ratsimbazafy 8805e5f061 Use Travis only for ARM64 - https://github.com/status-im/nimbus-eth2/pull/2167 2020-12-09 16:05:41 -06:00
Jacek Sieka 6f1ecc8df7
streamline socket read/write hot path (#473)
* streamline socket read/write hot path

This avoids some unnecessary memory copying on the hot path of noise /
mplex, as well as getting rid of a few futures - profiling shows that
this is one of the main culprits of small memory allocations, which
makes sense - this is where gossip fan-out happens.

* fewer futures (and corresponding closures) when sending lpchannel
messages
* avoid data copies when encrypting and framing noise messages
* avoid copying tuple when reading noise data (poor c codegen)
* fix setting eof flag in secure read

* write noise frames in one go

...and closing secure socket once is enough
2020-12-09 08:56:40 -06:00
Jacek Sieka 1befeb8c2e
clean up peerid (#470)
* fix dangling cstring on error return
* remove some useless inlines
* less mallocs in shortlog
* proc -> func
* rename test
2020-12-03 13:53:16 -06:00
Dmitriy Ryajov e9d4679059
Race in connection setup (#464)
* check that connection is not closed or eof

* don't release connection lock prematurely

* test that only valid connections can be added

* correct exception type on closed connection

* add clarifying comment

* use closeWithEOF for more stable test

* misc comments

* log stream id in buffestream asserts

* use closeWithEOF to prevent races in tests

* give some time to the remote handler to trigger

* adding more tests to make codecov happy
2020-12-02 19:24:48 -06:00
Dmitriy Ryajov d1c689e5ab
adding libp2p tag to logScope (#465) 2020-12-01 11:34:27 -06:00
Giovanni Petrantoni e1648d4404 fix mcache logic check in gossipsub 2020-12-01 23:55:51 +09:00
Giovanni Petrantoni b4738d723c
Some gossip fixes (#467)
* fix some missing rpc in rebalanceMesh

* clarify some variable names and lifetime

* further improvements
2020-12-01 11:44:09 +01:00
Dmitriy Ryajov 94e672ead0
allow concurrent closeWithEOF (#466)
* allow concurrent closeWithEOF

* add dedicated closedWithEOF flag
2020-12-01 09:44:21 +01:00
Jacek Sieka 5c2a54bdd9
fix timeoutmonitor loop (#463)
* fix timeoutmonitor loop

* Clarify that cancellation can happen while in timeoutMonitor
2020-11-29 13:34:19 +01:00
Dmitriy Ryajov 18443dafc1
rework peer event to take an initiator flag (#456)
* rework peer event to take an initiator flag

* use correct direction for initiator
2020-11-28 10:59:47 -06:00
Dmitriy Ryajov 3d44fcb8b3
use cancelAndAwait to mitigate further hangs (#459) 2020-11-28 09:48:06 -06:00
Dmitriy Ryajov a8f5f7a8bb
move dialing logic to it's own proc to avoid try/finally bugs (#461)
* move dialing logic to it's own proc to avoid try/finally bugs

* re-export transport

* lint

* add cancelation test

* test remote conn close on dial
2020-11-28 09:05:12 +01:00
Giovanni Petrantoni 02b20440f2
Limit ihave emission (#462)
* add some limits to ihave emission, go has them, rust does not actually

* restore shuffling of IDs

* add some context
2020-11-28 16:27:39 +09:00
Giovanni Petrantoni 12db9a4cf2 TopicParams validation tuning 2020-11-28 00:23:09 +09:00
Giovanni Petrantoni 809df8d04d add some extra gossip metrics 2020-11-26 16:20:34 +09:00
Giovanni Petrantoni 6c7f2766fe
expose seenTTL parameters (#457) 2020-11-26 14:45:10 +09:00
Dmitriy Ryajov ca9c5c85e4
dont break chronicles logging streamline connsetup (#455) 2020-11-25 13:34:48 -06:00
Dmitriy Ryajov 7b1e652224
Allow custom identify agent string (#451)
* allow custom agent version string

* rework tests and add test for custom agent version
2020-11-25 07:42:02 -06:00
Dmitriy Ryajov 164892776b
get rid of hangs cleanup (#453) 2020-11-25 07:35:25 -06:00
Dmitriy Ryajov 21110636cb
fixing log level to avoid sacring users (#452) 2020-11-24 12:07:27 -06:00
Giovanni Petrantoni a3b21f7cbb update create-pull-request 2020-11-24 17:23:28 +09:00
Dmitriy Ryajov 351489bfa9
getMuxedStream to more appropriate getStream (#448) 2020-11-24 00:37:45 -06:00
Dmitriy Ryajov 69ae24dc8d
less leak prone cleanup (#447)
* less leak prone cleanup

* fix double allFinished
2020-11-23 18:22:15 -06:00
Dmitriy Ryajov 6cc3f4283a
update conn peerinfo instead of replacing (#445)
* update conn peerinfo instead of replacing

* remove unnecesary peerid var
2020-11-23 15:15:55 -06:00
Dmitriy Ryajov 034a1e8b1b
small cleanups from tcp-limits2 (#446) 2020-11-23 15:02:23 -06:00
Dmitriy Ryajov 1d16d22f5f
Don't allow concurrent pushdata (#444)
* handle resets properly with/without pushes/reads

* add clarifying comments

* pushEof should also not be concurrent

* move channel reset to bufferstream

this is where the action happens - lpchannel merely redefines how close
is done

Co-authored-by: Jacek Sieka <jacek@status.im>
2020-11-23 09:07:11 -06:00
Dmitriy Ryajov c42009d56e
don't quit accept prematurelly (#443) 2020-11-19 09:10:25 -06:00
Giovanni Petrantoni 93b6c4dc52
Gossip runtime params (#437)
* move gossip parameters to runtime

* internal test fixes

* add missing params

* restore const parameters are soldi base and use them in init

* more constants tuning
2020-11-19 16:48:17 +09:00
Dmitriy Ryajov 92fa4110c1
Rework transport to use chronos accept (#420)
* rework transport to use the new accept api

* use the new chronos primits

* fixup tests to use the new transport api

* handle all exceptions in upgradeIncoming

* master merge

* add multiaddress exception type

* raise appropriate exception on invalida address

* allow retrying on TransportTooManyError

* adding TODO

* wip

* merge master

* add sleep if nil is returned

* accept loop handles all exceptions

* avoid issues with tray/except/finally

* make consistent with master

* cleanup accept loop

* logging

* Update libp2p/transports/tcptransport.nim

Co-authored-by: Jacek Sieka <jacek@status.im>

* use Direction enum instead of initiator flag

* use consistent import style

* remove experimental `closeWithEOF()`

Co-authored-by: Jacek Sieka <jacek@status.im>
2020-11-18 20:06:42 -06:00
Dmitriy Ryajov 8c8d73380f
Re-add connection manager tests (#441)
* use table.getOrDefault()

* re-add missing connection manager tests
2020-11-17 18:48:26 -06:00
Jacek Sieka 74acd0a33a
fix channels not being reset (#439)
* fix channels not being reset

silly for loop..

* allow only one concurrent read
* fix mplex test race condition
* add some bufferstream eof tests

* deadlock, lost data and hung channel fixes

* prevent concurrent `reset` calls
* reset LPChannel when read is cancelled (since data is lost)
* ensure there's one, and one only, 0-byte readOnce on EOF
* ensure that all data is returned before EOF is returned
* keep running activity monitor for half-closed channels (or they never
get closed)
2020-11-17 08:59:25 -06:00
Jacek Sieka 51a0aec058
read failure (#436)
* read failure

Test showing read failure on cancel

* adding one more test case from jacek

Co-authored-by: Dmitriy Ryajov <dryajov@gmail.com>
2020-11-13 12:30:14 -06:00
Jacek Sieka 5e6ec6f422
Revert "failing test"
This reverts commit 5dbed426c2.
2020-11-13 12:26:08 +01:00
Jacek Sieka 5dbed426c2
failing test 2020-11-13 12:24:23 +01:00
Dmitriy Ryajov 55b763264e
Cleanup tests (#435)
* add async testing methods

* refactor with async testing methods

* use iffy in async tests
2020-11-12 21:44:02 -06:00