Commit Graph

388 Commits

Author SHA1 Message Date
richΛrd 5aa4d578aa
fix: panic if it is not possible to obtain the merkle root (#873) 2023-11-09 16:17:41 -04:00
richΛrd ff94b1faf0
chore: remove --store-message-db-vacuum (#883) 2023-11-09 16:10:40 -04:00
richΛrd 3d217ed5ff
fix: WakuMessage json encoding (#880) 2023-11-08 12:09:10 -04:00
richΛrd 70cb6de576
fix: noisy peer exchange log (#881) 2023-11-08 12:03:08 -04:00
Prem Chaitanya Prathi fab51beadf
fix : issues with get messages API (#878)
* fix issues with get messages API

---------

Co-authored-by: richΛrd <info@richardramos.me>
2023-11-08 18:46:24 +05:30
Prem Chaitanya Prathi 28c0cd5d8e
fix: content topic validation as per rfc 51 (#874)
* fix: content topic validation as per rfc 51

* chore: update library API's and examples
2023-11-08 18:24:24 +05:30
Roman Zajic 43412c9da5
Chore: filter v2 tests unsubscribe all (#875)
* Chore(filter v2) test updates (#811)

* test: Test incorrect protocol identifiers

* fix: return errors in FilterSubscribeOption

* test: Test incorrect push identifier added
- test incorrect subscribe identifier separated

* test: Test Ping failure after unsubscription

* test: Test PubSub with single content topic

* test: Simplify test PubSub with single content topic

* test: Test with single pubsub and multiple content topics

* test: Test with multiple PubSub and multiple contentTopic

* test: Test with multiple overlaping contentTopics
- test contentTopics limit

* test: refactor tests to fix concurrent run errors

* test: Test subscription refresh

* test: Test error handling for subscribe

* test: Test subscription to multiple full nodes

* update test to fix #804

* Update waku/v2/protocol/filter/filter_test.go

Combine log messages

Co-authored-by: richΛrd <info@richardramos.me>

* Update waku/v2/protocol/filter/filter_test.go

Delete commented - temporary code

Co-authored-by: richΛrd <info@richardramos.me>

* Update waku/v2/protocol/filter/filter_test.go

fmt.Sprintf instead of "+" suffix => more performance and beauty

Co-authored-by: richΛrd <info@richardramos.me>

* Update waku/v2/protocol/filter/filter_test.go

Adjust comment with code

Co-authored-by: richΛrd <info@richardramos.me>

* Update waku/v2/protocol/filter/filter_test.go

Combine multiple related log entries into one.

Co-authored-by: richΛrd <info@richardramos.me>

* Update waku/v2/protocol/filter/filter_test.go

Align comment with the code

Co-authored-by: richΛrd <info@richardramos.me>

* Update waku/v2/protocol/filter/filter_test.go

Use fmt.Sprintf() instead of "+" for more beauty and speed

Co-authored-by: richΛrd <info@richardramos.me>

* test: refactor tests with prepareData()

* test: Test incorrect protocol identifiers

* chore: rebase onto latest master

* test: Test incorrect push identifier added
- test incorrect subscribe identifier separated

* test: Test Ping failure after unsubscription

* test: Test PubSub with single content topic

* test: Simplify test PubSub with single content topic

* test: Test with single pubsub and multiple content topics

* test: Test with multiple PubSub and multiple contentTopic

* test: Test with multiple overlaping contentTopics
- test contentTopics limit

* test: refactor tests to fix concurrent run errors

* test: Test subscription refresh

* test: Test error handling for subscribe

* test: Test subscription to multiple full nodes

* update test to fix #804

* Update waku/v2/protocol/filter/filter_test.go

Combine log messages

Co-authored-by: richΛrd <info@richardramos.me>

* Update waku/v2/protocol/filter/filter_test.go

Delete commented - temporary code

Co-authored-by: richΛrd <info@richardramos.me>

* Update waku/v2/protocol/filter/filter_test.go

fmt.Sprintf instead of "+" suffix => more performance and beauty

Co-authored-by: richΛrd <info@richardramos.me>

* Update waku/v2/protocol/filter/filter_test.go

Adjust comment with code

Co-authored-by: richΛrd <info@richardramos.me>

* Update waku/v2/protocol/filter/filter_test.go

Combine multiple related log entries into one.

Co-authored-by: richΛrd <info@richardramos.me>

* Update waku/v2/protocol/filter/filter_test.go

Align comment with the code

Co-authored-by: richΛrd <info@richardramos.me>

* Update waku/v2/protocol/filter/filter_test.go

Use fmt.Sprintf() instead of "+" for more beauty and speed

Co-authored-by: richΛrd <info@richardramos.me>

* test: refactor tests with prepareData()

* test: Test incorrect protocol identifiers

* fix: return errors in FilterSubscribeOption

* test: Test incorrect push identifier added
- test incorrect subscribe identifier separated

* test: Test Ping failure after unsubscription

* test: Test PubSub with single content topic

* test: Simplify test PubSub with single content topic

* test: Test with single pubsub and multiple content topics

* test: Test with multiple PubSub and multiple contentTopic

* test: Test with multiple overlaping contentTopics
- test contentTopics limit

* test: refactor tests to fix concurrent run errors

* test: Test subscription refresh

* test: Test error handling for subscribe

* test: Test subscription to multiple full nodes

* update test to fix #804

* Update waku/v2/protocol/filter/filter_test.go

Combine log messages

Co-authored-by: richΛrd <info@richardramos.me>

* Update waku/v2/protocol/filter/filter_test.go

Delete commented - temporary code

Co-authored-by: richΛrd <info@richardramos.me>

* Update waku/v2/protocol/filter/filter_test.go

fmt.Sprintf instead of "+" suffix => more performance and beauty

Co-authored-by: richΛrd <info@richardramos.me>

* Update waku/v2/protocol/filter/filter_test.go

Adjust comment with code

Co-authored-by: richΛrd <info@richardramos.me>

* Update waku/v2/protocol/filter/filter_test.go

Combine multiple related log entries into one.

Co-authored-by: richΛrd <info@richardramos.me>

* Update waku/v2/protocol/filter/filter_test.go

Align comment with the code

Co-authored-by: richΛrd <info@richardramos.me>

* Update waku/v2/protocol/filter/filter_test.go

Use fmt.Sprintf() instead of "+" for more beauty and speed

Co-authored-by: richΛrd <info@richardramos.me>

* test: refactor tests with prepareData()

* Fix error during rebase

* Sync filter tests with latest master

* Refactor context initialization for test

* test: Incorrect Subscribe Identifier refactored with custom subscribe

* test: refactor into multiple files

* test: Subscribe with multiple light nodes to one full node

* test: shared mode for full node creation
- test preview Subscribe fullNode to fullNode

* test: test Subscribe fullNode to fullNode

---------

Co-authored-by: Richard Ramos <info@richardramos.me>
Co-authored-by: Prem Chaitanya Prathi <chaitanyaprem@gmail.com>

* test: unsubscribe all without content topics

* test: unsubscribe all without any filter specification

* test: move unsubscribe all tests to unsubscribe file

---------

Co-authored-by: Richard Ramos <info@richardramos.me>
Co-authored-by: Prem Chaitanya Prathi <chaitanyaprem@gmail.com>
2023-11-08 20:19:07 +08:00
richΛrd 150ade6f33
chore: use waku-org/waku-proto repository for protobuffer definitions (#828) 2023-11-07 15:48:43 -04:00
Prem Chaitanya Prathi 3226def4cf
feat: On Demand Peer Discovery based on shard and service (#834)
* refactor discovery and common service to separate package to remove package inter-dependencies

* relay on-demand discovery ,use proto to enr field mapping

* chore: no need to dial discovered peers as peermanager already does that

* on demand discovery for service peers during peer selection

* identify supported protocols for discovered peers and add to service slots

* fix: tests to use proper static sharding topics

* fix: random selection with default pubsubTopic

---------

Co-authored-by: richΛrd <info@richardramos.me>
2023-11-07 22:43:19 +05:30
Prem Chaitanya Prathi 2616d43c9d
chore: update relay REST and RPC API's and fix unit tests (#866)
* update relay REST API's to remove duplicate message cache, fix relay tests and admin test

* chore: enable REST and RPC unit tests

* update lightpush rest api to match yaml

* fix: filter rest unit test failures

* skipping legacy filter tests

* chore: add unit tests for autosharding relay REST API, fix success response (#868)
2023-11-07 20:26:48 +05:30
harsh jain 532a04013f
feat(rest-filterv2): get message (#856)
* feat: add getMessage endpoint

* test: getMessage filter v2
2023-11-04 14:24:20 +07:00
harsh jain a0bc53c679
fix(subscription-map): uniform operations and encapsulation (#853)
* fix(subscription-map): uniform operations and encapsulation

* nit: fixes based on comments
2023-11-04 14:16:24 +07:00
Roman Zajic 02f2800b04
chore: filter v2 tests unsubscribe (#855)
* Chore(filter v2) test updates (#811)

* test: Test incorrect protocol identifiers

* fix: return errors in FilterSubscribeOption

* test: Test incorrect push identifier added
- test incorrect subscribe identifier separated

* test: Test Ping failure after unsubscription

* test: Test PubSub with single content topic

* test: Simplify test PubSub with single content topic

* test: Test with single pubsub and multiple content topics

* test: Test with multiple PubSub and multiple contentTopic

* test: Test with multiple overlaping contentTopics
- test contentTopics limit

* test: refactor tests to fix concurrent run errors

* test: Test subscription refresh

* test: Test error handling for subscribe

* test: Test subscription to multiple full nodes

* update test to fix #804

* Update waku/v2/protocol/filter/filter_test.go

Combine log messages

Co-authored-by: richΛrd <info@richardramos.me>

* Update waku/v2/protocol/filter/filter_test.go

Delete commented - temporary code

Co-authored-by: richΛrd <info@richardramos.me>

* Update waku/v2/protocol/filter/filter_test.go

fmt.Sprintf instead of "+" suffix => more performance and beauty

Co-authored-by: richΛrd <info@richardramos.me>

* Update waku/v2/protocol/filter/filter_test.go

Adjust comment with code

Co-authored-by: richΛrd <info@richardramos.me>

* Update waku/v2/protocol/filter/filter_test.go

Combine multiple related log entries into one.

Co-authored-by: richΛrd <info@richardramos.me>

* Update waku/v2/protocol/filter/filter_test.go

Align comment with the code

Co-authored-by: richΛrd <info@richardramos.me>

* Update waku/v2/protocol/filter/filter_test.go

Use fmt.Sprintf() instead of "+" for more beauty and speed

Co-authored-by: richΛrd <info@richardramos.me>

* test: refactor tests with prepareData()

* test: Test incorrect protocol identifiers

* chore: rebase onto latest master

* test: Test incorrect push identifier added
- test incorrect subscribe identifier separated

* test: Test Ping failure after unsubscription

* test: Test PubSub with single content topic

* test: Simplify test PubSub with single content topic

* test: Test with single pubsub and multiple content topics

* test: Test with multiple PubSub and multiple contentTopic

* test: Test with multiple overlaping contentTopics
- test contentTopics limit

* test: refactor tests to fix concurrent run errors

* test: Test subscription refresh

* test: Test error handling for subscribe

* test: Test subscription to multiple full nodes

* update test to fix #804

* Update waku/v2/protocol/filter/filter_test.go

Combine log messages

Co-authored-by: richΛrd <info@richardramos.me>

* Update waku/v2/protocol/filter/filter_test.go

Delete commented - temporary code

Co-authored-by: richΛrd <info@richardramos.me>

* Update waku/v2/protocol/filter/filter_test.go

fmt.Sprintf instead of "+" suffix => more performance and beauty

Co-authored-by: richΛrd <info@richardramos.me>

* Update waku/v2/protocol/filter/filter_test.go

Adjust comment with code

Co-authored-by: richΛrd <info@richardramos.me>

* Update waku/v2/protocol/filter/filter_test.go

Combine multiple related log entries into one.

Co-authored-by: richΛrd <info@richardramos.me>

* Update waku/v2/protocol/filter/filter_test.go

Align comment with the code

Co-authored-by: richΛrd <info@richardramos.me>

* Update waku/v2/protocol/filter/filter_test.go

Use fmt.Sprintf() instead of "+" for more beauty and speed

Co-authored-by: richΛrd <info@richardramos.me>

* test: refactor tests with prepareData()

* test: Test incorrect protocol identifiers

* fix: return errors in FilterSubscribeOption

* test: Test incorrect push identifier added
- test incorrect subscribe identifier separated

* test: Test Ping failure after unsubscription

* test: Test PubSub with single content topic

* test: Simplify test PubSub with single content topic

* test: Test with single pubsub and multiple content topics

* test: Test with multiple PubSub and multiple contentTopic

* test: Test with multiple overlaping contentTopics
- test contentTopics limit

* test: refactor tests to fix concurrent run errors

* test: Test subscription refresh

* test: Test error handling for subscribe

* test: Test subscription to multiple full nodes

* update test to fix #804

* Update waku/v2/protocol/filter/filter_test.go

Combine log messages

Co-authored-by: richΛrd <info@richardramos.me>

* Update waku/v2/protocol/filter/filter_test.go

Delete commented - temporary code

Co-authored-by: richΛrd <info@richardramos.me>

* Update waku/v2/protocol/filter/filter_test.go

fmt.Sprintf instead of "+" suffix => more performance and beauty

Co-authored-by: richΛrd <info@richardramos.me>

* Update waku/v2/protocol/filter/filter_test.go

Adjust comment with code

Co-authored-by: richΛrd <info@richardramos.me>

* Update waku/v2/protocol/filter/filter_test.go

Combine multiple related log entries into one.

Co-authored-by: richΛrd <info@richardramos.me>

* Update waku/v2/protocol/filter/filter_test.go

Align comment with the code

Co-authored-by: richΛrd <info@richardramos.me>

* Update waku/v2/protocol/filter/filter_test.go

Use fmt.Sprintf() instead of "+" for more beauty and speed

Co-authored-by: richΛrd <info@richardramos.me>

* test: refactor tests with prepareData()

* Fix error during rebase

* Sync filter tests with latest master

* Refactor context initialization for test

* test: Incorrect Subscribe Identifier refactored with custom subscribe

* test: refactor into multiple files

* test: Subscribe with multiple light nodes to one full node

* test: shared mode for full node creation
- test preview Subscribe fullNode to fullNode

* test: test Subscribe fullNode to fullNode

---------

Co-authored-by: Richard Ramos <info@richardramos.me>
Co-authored-by: Prem Chaitanya Prathi <chaitanyaprem@gmail.com>

* test: unsubscribe with single contentTopic

* test: extend test - unsubscribe with single contentTopic

* test: unsubscribe with multiple contentTopic

* test: unsubscribe with multiple pubSub/contentTopic

* test: refactor back to use waitForTimeout()

* test: unsubscribe error handling

---------

Co-authored-by: Richard Ramos <info@richardramos.me>
Co-authored-by: Prem Chaitanya Prathi <chaitanyaprem@gmail.com>
2023-11-02 12:39:44 +08:00
richΛrd 5dfbd98c74
chore: lower dhigh to limit amplification factor (#850) 2023-10-31 08:41:19 -04:00
richΛrd 36beb9de75
refactor: fix nomenclature for shards (#849) 2023-10-31 06:50:13 -04:00
Richard Ramos cf82f66d12
revert(#820): msg digest matches msg hash 2023-10-30 16:25:21 -04:00
richΛrd 4584bb4324
refactor: validate protobuffer for store (#841) 2023-10-30 12:55:36 -04:00
richΛrd 38202e7a2e
refactor: publish API for relay and lightpush (#845) 2023-10-30 12:30:25 -04:00
harsh jain ddf188bbf8
feat: remove named topic (#844)
* feat: remove named topic

* fix: update examples

* Update library/mobile/api.go
2023-10-30 21:56:26 +07:00
richΛrd db222a24ef
fix(c-bindings): userdata (#785) 2023-10-28 19:37:53 -04:00
Roman Zajic fc3b2f76d5
chore(filter v2): test updates (#811)
---------

Co-authored-by: Richard Ramos <info@richardramos.me>
Co-authored-by: Prem Chaitanya Prathi <chaitanyaprem@gmail.com>
2023-10-28 19:27:01 -04:00
harsh jain 0868f5d4dd
feat: add filter v2 rpc (#798)
* feat: add filter v2 rpc

ping, subscribe/unsubscribe and unsubscribeAll.



* test(filterRest): pingFailure, subscribe-ping, unsubscribe and unsubscribeAll
2023-10-27 06:21:50 +07:00
harsh jain 0ba8b2caeb
fix(filter-subscribe): params.selectedPeer not set (#836)
* fix(filter-subscribe): params.selectedPeer not set
2023-10-26 05:06:44 +07:00
richΛrd 2701a38b2a
refactor: validate protobuffer for filter (#833) 2023-10-25 08:49:25 -04:00
Prem Chaitanya Prathi 18c16de94e
fix: modify store test to not depend on order of msgs (#832)
* fix: modify store test to not depend on order of msgs

Co-authored-by: richΛrd <info@richardramos.me>

---------

Co-authored-by: richΛrd <info@richardramos.me>
2023-10-25 03:41:00 +05:30
richΛrd 94f18c537c refactor: validate protobuffers for lightpush and relay (#824) 2023-10-24 17:48:22 -04:00
Richard Ramos fa51d10b4b fix: stream close/reset (#823) 2023-10-24 17:48:22 -04:00
Prem Chaitanya Prathi a3c3aab44e
fix: propagate relay subscribe opt properly (#829) 2023-10-25 02:11:42 +05:30
richΛrd 077df2fbb6
fix: do not disconnect peers in `TestWakuMetadataRequest` (#826) 2023-10-24 16:01:11 -04:00
Richard Ramos 9161c4f7fe fix: stream closing/reset 2023-10-20 20:30:23 -04:00
Richard Ramos 519fa2977a refactor: rename `connOpt` to `stream` 2023-10-20 20:30:23 -04:00
Richard Ramos 19ba25ffcb feat: metadata protocol 2023-10-20 20:30:23 -04:00
richΛrd 4181655b7a
fix: msg digest matches msg hash (#820) 2023-10-20 16:25:54 -04:00
Prem Chaitanya Prathi b5be83a02e
feat : autoshard relay api (#807)
* fix: using relay without bcaster should consume and drop messages

* update relay api usage

* move subscription to broadcaster

* move filter logic under subscription

* Support more than 1 relay subscription for a pubSubTopic

* modify relay Publish API to derive pubSubTopic based on autosharding

* implement relay RPC methods for autosharding

* remove relay msgChannel and relay on pubsub buffersize for subscription

Co-authored-by: richΛrd <info@richardramos.me>

* handle relay subscribe with noConsumer and address issue reported in code review

* chore: reorg relay code

---------

Co-authored-by: richΛrd <info@richardramos.me>
2023-10-21 01:26:18 +05:30
Prem Chaitanya Prathi ac9d826b03
fix: handle empty content topics in filter subcribe and unsubscribe (#812)
* fix: handle empty content topics in filter subcribe and unsubscribe
2023-10-18 01:23:40 +05:30
Richard Ramos d4abe15634 fix: addr update 2023-10-17 09:00:30 -04:00
Richard Ramos ee94581d0a feat: prefer circuit relay addr over ws, and update cache with enrs with newer seq number 2023-10-17 08:29:24 -04:00
Prem Chaitanya Prathi 6955d01498
Update peer selection options for light protocols (#787)
* Update peer selection options for lightPush

* Update peer selection options for filter

* migrate peer selection functionality from peer manager

Co-authored-by: richΛrd <info@richardramos.me>

---------

Co-authored-by: richΛrd <info@richardramos.me>
2023-10-16 22:12:01 +05:30
Richard Ramos 7826e31f14 fix(store): query time comparison and max rows per page 2023-10-12 10:58:55 -04:00
richΛrd 3aa477cbc6
fix: return errors in FilterSubscribeOption (#794) 2023-10-07 16:19:53 -04:00
richΛrd dcc828749f
fix: use https for cloning submodules instead of ssh (#780)
* fix: use https for cloning submodules instead of ssh
* chore: move to libs/

This simplifies the checkout process for github actions
2023-09-30 13:57:49 -04:00
Prem Chaitanya Prathi 47c961dcbb
feat: update lightpush API for autosharding (#774)
* feat: update lightpush API to make pubSubTopic optional as per autosharding

* Extract contentFilter and subscriptions out of filter to reuse in relay (#779)

* chore: extract contentFilter outside filter package

* chore: move subscription outside of filter so that it can be modified and reused for relay

* Feat: filter select peer for sharding (#783)

* update selectPeer to support pubsubTopic based selection
2023-09-29 10:43:25 +05:30
Prem Chaitanya Prathi 388f56b43f
feat: Sharded peer management - Relay (#764)
* feat: connect/disconnect to peers based on node topic sub/unsub

* feat: maintain healty relay connections per pubSubTopic

Co-authored-by: richΛrd <info@richardramos.me>

* chore: add config to limit peerstore capacity (#770)
2023-09-27 12:16:37 +05:30
richΛrd d324234c81
fix(filter2): add requestID to pings and remove unneeded log (#776) 2023-09-26 12:27:29 -04:00
Vitaliy Vlasov 16ec22596e feat: change UnsubscribeWithSubscription so that it's single sub-specific
Also merge FilterSubscribe and FilterUnsubscribe options/params
2023-09-22 17:53:33 +03:00
Prem Chaitanya Prathi 054bdae1de
Feat/autoshard filter (#723)
* feat: update filter client to support autosharding

* chore: add filter tests for autoshard

* chore:update filter API docs for autosharding

* chore: docs changes to indicate sharding impact on pubSubTopic

* fix: handle partial errors during subscribe and return failed content-topic details
2023-09-20 11:26:55 +05:30
Vitaliy Vlasov 81638fe111
Use PubsubTopic naming; enforce unique ContentTopics (#750) 2023-09-19 18:22:11 +05:30
Prem Chaitanya Prathi 9b05d48318
Feat : handle dynamic peer topic sub unsub (#751)
* feat: handle dynamic peer join and leave a pubSub topic

Co-authored-by: richΛrd <info@richardramos.me>


---------

Co-authored-by: richΛrd <info@richardramos.me>
2023-09-19 11:35:29 +05:30
Prem Chaitanya Prathi dd5dc7a9c8
fix: change mutex to rwLock and fix relay unsubscribe (#749)
* fix: change mutex to rwLock and fix relay unsubscribe

* chore: modify relay test to cover unsubscribe and few more relay functions
2023-09-18 18:48:16 +05:30
Richard Ramos 2060c8c837 feat: allow running rln with no credentials 2023-09-15 10:18:44 -04:00