* Chore(filter v2) test updates (#811)
* test: Test incorrect protocol identifiers
* fix: return errors in FilterSubscribeOption
* test: Test incorrect push identifier added
- test incorrect subscribe identifier separated
* test: Test Ping failure after unsubscription
* test: Test PubSub with single content topic
* test: Simplify test PubSub with single content topic
* test: Test with single pubsub and multiple content topics
* test: Test with multiple PubSub and multiple contentTopic
* test: Test with multiple overlaping contentTopics
- test contentTopics limit
* test: refactor tests to fix concurrent run errors
* test: Test subscription refresh
* test: Test error handling for subscribe
* test: Test subscription to multiple full nodes
* update test to fix#804
* Update waku/v2/protocol/filter/filter_test.go
Combine log messages
Co-authored-by: richΛrd <info@richardramos.me>
* Update waku/v2/protocol/filter/filter_test.go
Delete commented - temporary code
Co-authored-by: richΛrd <info@richardramos.me>
* Update waku/v2/protocol/filter/filter_test.go
fmt.Sprintf instead of "+" suffix => more performance and beauty
Co-authored-by: richΛrd <info@richardramos.me>
* Update waku/v2/protocol/filter/filter_test.go
Adjust comment with code
Co-authored-by: richΛrd <info@richardramos.me>
* Update waku/v2/protocol/filter/filter_test.go
Combine multiple related log entries into one.
Co-authored-by: richΛrd <info@richardramos.me>
* Update waku/v2/protocol/filter/filter_test.go
Align comment with the code
Co-authored-by: richΛrd <info@richardramos.me>
* Update waku/v2/protocol/filter/filter_test.go
Use fmt.Sprintf() instead of "+" for more beauty and speed
Co-authored-by: richΛrd <info@richardramos.me>
* test: refactor tests with prepareData()
* test: Test incorrect protocol identifiers
* chore: rebase onto latest master
* test: Test incorrect push identifier added
- test incorrect subscribe identifier separated
* test: Test Ping failure after unsubscription
* test: Test PubSub with single content topic
* test: Simplify test PubSub with single content topic
* test: Test with single pubsub and multiple content topics
* test: Test with multiple PubSub and multiple contentTopic
* test: Test with multiple overlaping contentTopics
- test contentTopics limit
* test: refactor tests to fix concurrent run errors
* test: Test subscription refresh
* test: Test error handling for subscribe
* test: Test subscription to multiple full nodes
* update test to fix#804
* Update waku/v2/protocol/filter/filter_test.go
Combine log messages
Co-authored-by: richΛrd <info@richardramos.me>
* Update waku/v2/protocol/filter/filter_test.go
Delete commented - temporary code
Co-authored-by: richΛrd <info@richardramos.me>
* Update waku/v2/protocol/filter/filter_test.go
fmt.Sprintf instead of "+" suffix => more performance and beauty
Co-authored-by: richΛrd <info@richardramos.me>
* Update waku/v2/protocol/filter/filter_test.go
Adjust comment with code
Co-authored-by: richΛrd <info@richardramos.me>
* Update waku/v2/protocol/filter/filter_test.go
Combine multiple related log entries into one.
Co-authored-by: richΛrd <info@richardramos.me>
* Update waku/v2/protocol/filter/filter_test.go
Align comment with the code
Co-authored-by: richΛrd <info@richardramos.me>
* Update waku/v2/protocol/filter/filter_test.go
Use fmt.Sprintf() instead of "+" for more beauty and speed
Co-authored-by: richΛrd <info@richardramos.me>
* test: refactor tests with prepareData()
* test: Test incorrect protocol identifiers
* fix: return errors in FilterSubscribeOption
* test: Test incorrect push identifier added
- test incorrect subscribe identifier separated
* test: Test Ping failure after unsubscription
* test: Test PubSub with single content topic
* test: Simplify test PubSub with single content topic
* test: Test with single pubsub and multiple content topics
* test: Test with multiple PubSub and multiple contentTopic
* test: Test with multiple overlaping contentTopics
- test contentTopics limit
* test: refactor tests to fix concurrent run errors
* test: Test subscription refresh
* test: Test error handling for subscribe
* test: Test subscription to multiple full nodes
* update test to fix#804
* Update waku/v2/protocol/filter/filter_test.go
Combine log messages
Co-authored-by: richΛrd <info@richardramos.me>
* Update waku/v2/protocol/filter/filter_test.go
Delete commented - temporary code
Co-authored-by: richΛrd <info@richardramos.me>
* Update waku/v2/protocol/filter/filter_test.go
fmt.Sprintf instead of "+" suffix => more performance and beauty
Co-authored-by: richΛrd <info@richardramos.me>
* Update waku/v2/protocol/filter/filter_test.go
Adjust comment with code
Co-authored-by: richΛrd <info@richardramos.me>
* Update waku/v2/protocol/filter/filter_test.go
Combine multiple related log entries into one.
Co-authored-by: richΛrd <info@richardramos.me>
* Update waku/v2/protocol/filter/filter_test.go
Align comment with the code
Co-authored-by: richΛrd <info@richardramos.me>
* Update waku/v2/protocol/filter/filter_test.go
Use fmt.Sprintf() instead of "+" for more beauty and speed
Co-authored-by: richΛrd <info@richardramos.me>
* test: refactor tests with prepareData()
* Fix error during rebase
* Sync filter tests with latest master
* Refactor context initialization for test
* test: Incorrect Subscribe Identifier refactored with custom subscribe
* test: refactor into multiple files
* test: Subscribe with multiple light nodes to one full node
* test: shared mode for full node creation
- test preview Subscribe fullNode to fullNode
* test: test Subscribe fullNode to fullNode
---------
Co-authored-by: Richard Ramos <info@richardramos.me>
Co-authored-by: Prem Chaitanya Prathi <chaitanyaprem@gmail.com>
* test: unsubscribe all without content topics
* test: unsubscribe all without any filter specification
* test: move unsubscribe all tests to unsubscribe file
---------
Co-authored-by: Richard Ramos <info@richardramos.me>
Co-authored-by: Prem Chaitanya Prathi <chaitanyaprem@gmail.com>
* refactor discovery and common service to separate package to remove package inter-dependencies
* relay on-demand discovery ,use proto to enr field mapping
* chore: no need to dial discovered peers as peermanager already does that
* on demand discovery for service peers during peer selection
* identify supported protocols for discovered peers and add to service slots
* fix: tests to use proper static sharding topics
* fix: random selection with default pubsubTopic
---------
Co-authored-by: richΛrd <info@richardramos.me>
* update relay REST API's to remove duplicate message cache, fix relay tests and admin test
* chore: enable REST and RPC unit tests
* update lightpush rest api to match yaml
* fix: filter rest unit test failures
* skipping legacy filter tests
* chore: add unit tests for autosharding relay REST API, fix success response (#868)
* Chore(filter v2) test updates (#811)
* test: Test incorrect protocol identifiers
* fix: return errors in FilterSubscribeOption
* test: Test incorrect push identifier added
- test incorrect subscribe identifier separated
* test: Test Ping failure after unsubscription
* test: Test PubSub with single content topic
* test: Simplify test PubSub with single content topic
* test: Test with single pubsub and multiple content topics
* test: Test with multiple PubSub and multiple contentTopic
* test: Test with multiple overlaping contentTopics
- test contentTopics limit
* test: refactor tests to fix concurrent run errors
* test: Test subscription refresh
* test: Test error handling for subscribe
* test: Test subscription to multiple full nodes
* update test to fix#804
* Update waku/v2/protocol/filter/filter_test.go
Combine log messages
Co-authored-by: richΛrd <info@richardramos.me>
* Update waku/v2/protocol/filter/filter_test.go
Delete commented - temporary code
Co-authored-by: richΛrd <info@richardramos.me>
* Update waku/v2/protocol/filter/filter_test.go
fmt.Sprintf instead of "+" suffix => more performance and beauty
Co-authored-by: richΛrd <info@richardramos.me>
* Update waku/v2/protocol/filter/filter_test.go
Adjust comment with code
Co-authored-by: richΛrd <info@richardramos.me>
* Update waku/v2/protocol/filter/filter_test.go
Combine multiple related log entries into one.
Co-authored-by: richΛrd <info@richardramos.me>
* Update waku/v2/protocol/filter/filter_test.go
Align comment with the code
Co-authored-by: richΛrd <info@richardramos.me>
* Update waku/v2/protocol/filter/filter_test.go
Use fmt.Sprintf() instead of "+" for more beauty and speed
Co-authored-by: richΛrd <info@richardramos.me>
* test: refactor tests with prepareData()
* test: Test incorrect protocol identifiers
* chore: rebase onto latest master
* test: Test incorrect push identifier added
- test incorrect subscribe identifier separated
* test: Test Ping failure after unsubscription
* test: Test PubSub with single content topic
* test: Simplify test PubSub with single content topic
* test: Test with single pubsub and multiple content topics
* test: Test with multiple PubSub and multiple contentTopic
* test: Test with multiple overlaping contentTopics
- test contentTopics limit
* test: refactor tests to fix concurrent run errors
* test: Test subscription refresh
* test: Test error handling for subscribe
* test: Test subscription to multiple full nodes
* update test to fix#804
* Update waku/v2/protocol/filter/filter_test.go
Combine log messages
Co-authored-by: richΛrd <info@richardramos.me>
* Update waku/v2/protocol/filter/filter_test.go
Delete commented - temporary code
Co-authored-by: richΛrd <info@richardramos.me>
* Update waku/v2/protocol/filter/filter_test.go
fmt.Sprintf instead of "+" suffix => more performance and beauty
Co-authored-by: richΛrd <info@richardramos.me>
* Update waku/v2/protocol/filter/filter_test.go
Adjust comment with code
Co-authored-by: richΛrd <info@richardramos.me>
* Update waku/v2/protocol/filter/filter_test.go
Combine multiple related log entries into one.
Co-authored-by: richΛrd <info@richardramos.me>
* Update waku/v2/protocol/filter/filter_test.go
Align comment with the code
Co-authored-by: richΛrd <info@richardramos.me>
* Update waku/v2/protocol/filter/filter_test.go
Use fmt.Sprintf() instead of "+" for more beauty and speed
Co-authored-by: richΛrd <info@richardramos.me>
* test: refactor tests with prepareData()
* test: Test incorrect protocol identifiers
* fix: return errors in FilterSubscribeOption
* test: Test incorrect push identifier added
- test incorrect subscribe identifier separated
* test: Test Ping failure after unsubscription
* test: Test PubSub with single content topic
* test: Simplify test PubSub with single content topic
* test: Test with single pubsub and multiple content topics
* test: Test with multiple PubSub and multiple contentTopic
* test: Test with multiple overlaping contentTopics
- test contentTopics limit
* test: refactor tests to fix concurrent run errors
* test: Test subscription refresh
* test: Test error handling for subscribe
* test: Test subscription to multiple full nodes
* update test to fix#804
* Update waku/v2/protocol/filter/filter_test.go
Combine log messages
Co-authored-by: richΛrd <info@richardramos.me>
* Update waku/v2/protocol/filter/filter_test.go
Delete commented - temporary code
Co-authored-by: richΛrd <info@richardramos.me>
* Update waku/v2/protocol/filter/filter_test.go
fmt.Sprintf instead of "+" suffix => more performance and beauty
Co-authored-by: richΛrd <info@richardramos.me>
* Update waku/v2/protocol/filter/filter_test.go
Adjust comment with code
Co-authored-by: richΛrd <info@richardramos.me>
* Update waku/v2/protocol/filter/filter_test.go
Combine multiple related log entries into one.
Co-authored-by: richΛrd <info@richardramos.me>
* Update waku/v2/protocol/filter/filter_test.go
Align comment with the code
Co-authored-by: richΛrd <info@richardramos.me>
* Update waku/v2/protocol/filter/filter_test.go
Use fmt.Sprintf() instead of "+" for more beauty and speed
Co-authored-by: richΛrd <info@richardramos.me>
* test: refactor tests with prepareData()
* Fix error during rebase
* Sync filter tests with latest master
* Refactor context initialization for test
* test: Incorrect Subscribe Identifier refactored with custom subscribe
* test: refactor into multiple files
* test: Subscribe with multiple light nodes to one full node
* test: shared mode for full node creation
- test preview Subscribe fullNode to fullNode
* test: test Subscribe fullNode to fullNode
---------
Co-authored-by: Richard Ramos <info@richardramos.me>
Co-authored-by: Prem Chaitanya Prathi <chaitanyaprem@gmail.com>
* test: unsubscribe with single contentTopic
* test: extend test - unsubscribe with single contentTopic
* test: unsubscribe with multiple contentTopic
* test: unsubscribe with multiple pubSub/contentTopic
* test: refactor back to use waitForTimeout()
* test: unsubscribe error handling
---------
Co-authored-by: Richard Ramos <info@richardramos.me>
Co-authored-by: Prem Chaitanya Prathi <chaitanyaprem@gmail.com>
* fix: modify store test to not depend on order of msgs
Co-authored-by: richΛrd <info@richardramos.me>
---------
Co-authored-by: richΛrd <info@richardramos.me>
* fix: using relay without bcaster should consume and drop messages
* update relay api usage
* move subscription to broadcaster
* move filter logic under subscription
* Support more than 1 relay subscription for a pubSubTopic
* modify relay Publish API to derive pubSubTopic based on autosharding
* implement relay RPC methods for autosharding
* remove relay msgChannel and relay on pubsub buffersize for subscription
Co-authored-by: richΛrd <info@richardramos.me>
* handle relay subscribe with noConsumer and address issue reported in code review
* chore: reorg relay code
---------
Co-authored-by: richΛrd <info@richardramos.me>
* feat: update lightpush API to make pubSubTopic optional as per autosharding
* Extract contentFilter and subscriptions out of filter to reuse in relay (#779)
* chore: extract contentFilter outside filter package
* chore: move subscription outside of filter so that it can be modified and reused for relay
* Feat: filter select peer for sharding (#783)
* update selectPeer to support pubsubTopic based selection
* feat: update filter client to support autosharding
* chore: add filter tests for autoshard
* chore:update filter API docs for autosharding
* chore: docs changes to indicate sharding impact on pubSubTopic
* fix: handle partial errors during subscribe and return failed content-topic details
* fix: and optimising fetching membership events
* fix: start from lastProcessedBlock+1
* test: fetching membership logic
* refactor: usage of rlnInstance,rootTracker,groupManager
rlnInstance, rootTrack were previously created while creating rlnRelay
but were assigned to groupManager on Start of rlnRelay. This created
unncessary dependency of passing them to static and dynamic group
manager.
Web3Config uses interface EthClientI for client, so that we can pass
mock client for testing MembershipFetcher.
* fix: failing test
* fix: lint error
* fix: account for PR suggestions
* fix: failing race test
* fix: dont' increase fromBlock on error
* nit: fix naming and add comments
* fix: slice error for index out of bounds
this commit fixes slicing `notConnectedPeers` array with 0 to a negative value of `numPeersToConnect` by first checking if `numPeersToConnect` are greater than 0
* chore: add shard choice simulation test
* feat: add new flags for pubsub and contentTopics and deprecate topic flag
* chore: remove store-resume-peer config and comment out functionality until redesign of store is done
* chore: fix code to use contentTopics value
* fix: use default waku topic only if no other topics are provided in the config
* feat: add serviceSlot ds
* test: service slot
* nit: codeClimate related changs
* nit: based on comments
* nit: dont' run getPeers for WakuRelayIDv200
* refactor: peerConnector
* fix: code climate and dont waitOn subscriptions PeerData
* fix: check in peerConnector is on outRelay connections
* fix: introduced bug in peerConnector
* feat:Register for reachability change and log status
Co-authored-by: richΛrd <info@richardramos.me>
---------
Co-authored-by: richΛrd <info@richardramos.me>
- use atomic operations API
- resume onchain sync from persisted tree db
- close eth client and db connection appropriately
- pass in the path to the tree db
- fix nwaku compatibility issues
* feat: support peermanager serviceslots and update store protocol to use serviceslots
* fix: lint errors in test code
* fix: error in nix build due to vendor sha change
* fix: set host in peermanager even if relay is disabled
* chore: fix codeclimate issues
* chore: using common filterPeer function to avoid duplication
* feat:use service slots in other service protocols
* chore: fix codeclimate issues
* chore: move AddPeer to peermanager
* Apply suggestions from code review
Co-authored-by: richΛrd <info@richardramos.me>
* chore:address review comments
* feat: implement RemovePeer #638
* chore: fix test failure
* Support for multiple slots for service peers
Adding discovered peers also moved to peer manager
---------
Co-authored-by: richΛrd <info@richardramos.me>
* chore: refactor existing code into peer maanger package
* feat: move peer connection related code into peer manager
* feat: in relay peer connection pruning
* feat: add max-connections CLI flag and limit outRelayPeers based on max-connections #621
* tested both in and out relay connection limits
Co-authored-by: richΛrd <info@richardramos.me>
* Review comment, use context to pause connectivity loop during node shutdown.
Co-authored-by: richΛrd <info@richardramos.me>
* address review comments
---------
Co-authored-by: richΛrd <info@richardramos.me>
richΛrd
Prem Chaitanya Prathi
Roman Zajic
harsh jain
Anton Iakimov
Vitaliy Vlasov
Siddarth Kumar
Andrea Maria Piana