40e21e3375
Cleanup testinterop imports ( #589 )
...
* cleanup exports/imports, specially for testinterop
* export pubsub
2021-06-11 16:33:47 -06:00
3da656687b
use LPError more consistently ( #582 )
...
* use LPError more consistently
* don't use Exceptino
* annotate with raises
* don't panic on concatenation
* further rework error handling
2021-06-02 15:39:10 +02:00
1b94c3feda
fix #581 ( #583 )
2021-06-01 18:06:08 -06:00
34787728a3
don't use MaResult as default in newStandardSwitch ( #578 )
2021-05-24 16:48:18 -06:00
24132d7129
More raises cleanup ( #575 )
...
* use toException to map errors
* don't initialize address twice
* better error messages
* allow LPError to escape
* allow LPError to escape
2021-05-22 12:27:30 -06:00
ac4e060e1a
adding raises defect across the codebase ( #572 )
...
* adding raises defect across the codebase
* use unittest2
* add windows deps caching
* update mingw link
* die on failed peerinfo initialization
* use result.expect instead of get
* use expect more consistently and rework inits
* use expect more consistently
* throw on missing public key
* remove unused closure annotation
* merge master
2021-05-21 10:27:01 -06:00
9674a6a6f6
simplify connmanager ( #573 )
...
* no need to init orderedset
* array more simple than table
2021-05-19 08:49:55 +02:00
83a20a992a
gossipsub: unsubscribe fixes ( #569 )
...
* gossipsub: unsubscribe fixes
* fix KeyError when updating metric of unsubscribed topic
* fix unsubscribe message not being sent to all peers causing them to
keep thinking we're still subscribed
* release memory earlier in a few places
* floodsub fix
2021-05-07 00:43:45 +02:00
9f301964ed
fix control messages ( #566 )
...
* remove unused control graft check in handleControl
* avoid sending empty Iwant messages
2021-04-28 10:03:03 +09:00
f81a085d0b
More gossip coverage ( #553 )
...
* add floodPublish test
* test delivery via control Iwant/have mechanics
* fix issues in control, and add testing
* fix possible backoff issue with pruned routine overriding it
2021-04-22 18:51:22 +09:00
e285d8bbf4
mem usage cleanups for pubsub ( #564 )
...
In `async` functions, a closure environment is created for variables
that cross an await boundary - this closure environment is kept in
memory for the lifetime of the associated future - this means that
although _some_ variables are no longer used, they still take up memory
for a long time.
In Nimbus, message validation is processed in batches meaning the future
of an incoming gossip message stays around for quite a while - this
leads to memory consumption peaks of 100-200 mb when there are many
attestations in the pipeline.
To avoid excessive memory usage, it's generally better to move non-async
code into proc's such that the variables therein can be released earlier
- this includes the many hidden variables introduced by macro and
template expansion (ie chronicles that does expensive exception
handling)
* move seen table salt to floodsub, use there as well
* shorten seen table salt to size of hash
* avoid unnecessary memory allocations and copies in a few places
* factor out message scoring
* avoid reencoding outgoing message for every peer
* keep checking validators until reject (in case there's both reject and
ignore)
* `readOnce` avoids `readExactly` overhead for single-byte read
* genericAssign -> assign2
2021-04-18 10:08:33 +02:00
6b930ae7e6
More builders ( #560 )
...
* address some issues pointed out in review
* re-add to prevent breaking other projects
2021-04-06 14:16:23 -06:00
290866dd62
builders ( #559 )
2021-04-05 16:06:45 -06:00
795a651839
use a builder pattern to build the switch ( #551 )
...
* use a builder pattern to build the switch
* with with
* more refs
2021-04-02 10:20:51 +09:00
df497660bc
fix nimbus ref for auto-bump stable's PR
2021-03-23 16:46:24 +09:00
54031c9e9b
Fix minor exception issues ( #550 )
...
Makes code compatible with
https://github.com/status-im/nim-chronos/pull/166 without requiring it.
2021-03-23 07:45:25 +01:00
38333e45ae
put conditional locks on nbc bump ( #549 )
2021-03-22 22:13:00 +09:00
3bf6acef23
merge nbc auto bump with CI in order to bump only on CI success
2021-03-16 09:31:06 +09:00
2d3595708c
fix auto-bump CI
2021-03-16 08:45:37 +09:00
9175a9476e
add stable/unstable auto bumps
2021-03-16 08:31:39 +09:00
f7a9d83545
add mounted protos to identify message ( #546 )
2021-03-15 15:29:05 -06:00
03bbdd2261
Revisit Floodsub ( #543 )
...
Fixes #525
add coverage to unsubscribeAll and testing
2021-03-15 13:16:03 -06:00
a54e1cc699
add coverage to disabled metrics ( #545 )
2021-03-15 21:36:57 +09:00
aeb18c4e41
now wild `except:`
2021-03-15 16:48:11 +09:00
c5b9fd3353
Try fix ci ( #544 )
...
* use apt-get
* try bump ubuntu ver
2021-03-15 14:50:36 +09:00
ba0a57ee34
fix reports number
2021-03-15 12:16:01 +09:00
996dc5be9c
add testfloodsub coverage for different build options
2021-03-15 12:15:29 +09:00
4760df1e31
fix build with libp2p_agents_metrics switch
2021-03-15 01:42:47 +00:00
70deac9e0d
fix peer score accumulation ( #541 )
...
* fix accumulating peer score
* fix missing exception handling
* remove unnecessary initHashSet/initTable calls
* simplify peer stats management
* clean up tests a little
* fix some missing raises annotations
2021-03-09 13:22:52 +01:00
269d3df351
Consolidate metrics collection for mesh ( #540 )
...
* Consolidate metrics collection for mesh
* more fixes
* wrapping up
* Update libp2p/protocols/pubsub/gossipsub/behavior.nim
Co-authored-by: Jacek Sieka <jacek@status.im>
* Update libp2p/protocols/pubsub/gossipsub/behavior.nim
Co-authored-by: Jacek Sieka <jacek@status.im>
* Update libp2p/protocols/pubsub/gossipsub/behavior.nim
Co-authored-by: Jacek Sieka <jacek@status.im>
Co-authored-by: Jacek Sieka <jacek@status.im>
2021-03-03 22:11:21 +01:00
34c2fbeb16
small gossipsub metrics change
2021-03-03 08:41:21 +00:00
02ad017107
Gossipsub fixes and Initiator flagging fixes ( #539 )
...
* properly propagate initiator information for gossipsub
* Fix pubsubpeer lifetime management
* restore old behavior
* tests fixing
* clamp backoff time value received
* fix member name collisions
* internal test fixes
* better names and explaining of the importance of transport direction
* fixes
2021-03-03 08:23:40 +09:00
c1334c6d89
pubsubpeer better address management
2021-02-28 04:53:17 +00:00
7b2727d930
avoid leaking in peersInIP, don't depend on sendConn
2021-02-27 23:49:56 +09:00
67d0926e89
use in any case PeerID for peersInIP to avoid keeping references
2021-02-27 21:31:59 +09:00
fae38e0146
fix PubSubPeer hashing issues
2021-02-26 19:19:15 +09:00
45300c28a9
[SEC] gossipsub - handleIHAVE/handleIWANT recommendations & notes ( #535 )
...
Fixes #400
2021-02-26 14:27:42 +09:00
d7469b2286
[SEC] gossipsub - a peer score is not retained up until expiry if abusive peer unsubscribes ( #534 )
...
* [SEC] gossipsub - a peer score is not retained up until expiry if abusive peer unsubscribes
Fixes #402
* remove debug logging
2021-02-26 14:15:58 +09:00
c1d8317e3c
fix badly merged code in gossipsub.colocationFactor
2021-02-26 12:39:57 +09:00
eac6cd3dbf
Debt: cleanup warnings #426 ( #536 )
...
* testswitch cleanups
* Debt: cleanup warnings
Fixes #426
2021-02-25 09:24:49 -06:00
2015a878ad
fix missing coverage ( #533 )
...
* investigate missing coverage
* test different pattern
* wip pattern testing
* try globastar
2021-02-23 12:50:43 +09:00
8236319a91
[SEC] gossipsub - handleGraft/handlePrune recommendations & notes ( #530 )
...
Fixes #401
2021-02-22 12:04:20 +09:00
1368bf7ecb
test rebalanceMesh with low score peers ( #529 )
2021-02-22 10:05:25 +09:00
922cd92f94
don't check if peers have `sendConn` when disconnecting for bad scoring
2021-02-22 10:04:02 +09:00
51d8cd4ade
[SEC] gossipsub - rebalanceMesh may prune up to D_lo on oversubscription ( #531 )
...
Fixes #403
2021-02-13 13:39:32 +09:00
e124e342b0
n subscription limits ( #528 )
...
* subscription high water, cleanups
* subscription limits test
* newline
2021-02-12 12:27:26 +09:00
12adefb4de
add multi types to exports ( #527 )
...
* add multitypes to exports
* export standard setup
2021-02-10 11:42:46 -06:00
f4145ebbfa
More exports cleanup ( #522 )
...
* annotate `SecureProtocol.Secio` as deprecated
* dont export varint
* add `errors` to exports - convenient error utils
2021-02-09 15:41:49 -06:00
fff54fa23c
add more diagnostics when gossip publish fails
2021-02-09 18:42:59 +09:00
d9563d65ae
support compilation with Nim-1.4 HEAD ( #521 )
2021-02-08 15:21:43 -06:00
Dmitriy Ryajov
Jacek Sieka
Giovanni Petrantoni
Ștefan Talpalaru