Commit Graph

248 Commits

Author SHA1 Message Date
Igor Sirotin d977914f6b
chore_: skip waku tests as flaky (#5487)
* chore_: skip `TestBasicWakuV2` as flaky

* chore_: skip TestWakuV2Filter
2024-07-05 11:00:33 +01:00
Prem Chaitanya Prathi 437f830b51
feat_: aggregate filter subscriptions to do bulk subs with peer (#5440)
* feat_: aggregate filter subscriptions to do bulk subs with peer

* chore_: take possible deadlock fix in go-waku

* fix_: don't resubscribe filters unless there is a change in shard for community (#5467)
2024-07-04 10:34:53 +05:30
richΛrd 01895bab30
fix_: missing messages delay should be substracted (#5453) 2024-07-01 11:38:29 -04:00
Prem Chaitanya Prathi 9ddc6e6df1
fix_: wakuv2 failing tests #5439 (#5444) 2024-06-28 23:42:12 +05:30
richΛrd e47e867b9b
refactor_: use go-waku onlinechecker to manage connection state (#5340) 2024-06-28 09:54:48 -04:00
Arseniy Klempner 5934233266
feat_: call telemetry upon error pushing envelope (#5430)
* feat_: call telemetry upon error pushing envelope

* feat_: call telemetry upon error pushing envelope

---------

Co-authored-by: Václav Pavlín <vaclav.pavlin@gmail.com>
2024-06-28 13:24:04 +03:00
Prem Chaitanya Prathi bd8aa0dac7
fix: enable pxclient in relay and lightClient and some fixes (#5350)
* fix_: use cluster and shard to filter peers received via peerExchange

* chore_: enable pxClient in relay and increase relay peer connections (#5411)

* chore_: update go-waku with fixes
2024-06-28 13:56:56 +05:30
Igor Sirotin ad96e557bf
fix_: bring back waku tests (#5437) 2024-06-28 10:02:41 +02:00
kaichao d8a49c538b
Reset MVDS epoch after peer is online (#5349)
* feat_: reset epoch for online peer

* chore_: fix

* chore_: refactor

* chore_: update mvds

* chore_: fix lint

* chore_: update mvds

* chore_: make vendor

* chore_: fix tst

* chore_: tuning store query hash parameter

* chore_: non-blocking mvds status change channel
2024-06-27 09:54:31 +08:00
richΛrd 7731f56b17
feat(waku2)_: check for missing messages (#5281) 2024-06-26 14:25:44 -04:00
Igor Sirotin d2f4cae18f
fix: merge coverage reports from each test rerun (#5360)
* fix_: fix test coverage when rerunning tests

* fix_: make lint-fix

* chore_: change test coverage report artifact name

* chore_: added codeclimate.json to artifacts

* chore_: bring in gocovmerge util

---------

Co-authored-by: Siddarth Kumar <siddarthkay@gmail.com>
2024-06-19 11:40:52 +01:00
richΛrd a300e12853
feat_: wakuext_enr (#5367) 2024-06-18 15:48:49 -04:00
richΛrd b18baea5cd
fix_: peer counter (#5348) 2024-06-14 12:35:05 -04:00
richΛrd eb0f907137
fix_: use node context instead of context with timeout for discv5 (#5347) 2024-06-14 12:05:17 -04:00
Vit∀ly Vlasov 68acef62d4
feat: Simplify FilterManager and move subscription management to waku (#4665)
* feat_: simplify filter management

fix_: try using shard as a default topic

fix_: filter test to work with shards.staging fleet

* fix_: handle connection status change and manage filter subscriptions better

* chore_: bump go-waku to latest and with some fixes

Co-authored-by: richΛrd <info@richardramos.me>

* chore_: disabling pxClient so that only fleet nodes are used for now

---------

Co-authored-by: Prem Chaitanya Prathi <chaitanyaprem@gmail.com>
Co-authored-by: richΛrd <info@richardramos.me>
2024-06-14 18:11:45 +05:30
Arseniy Klempner 1bbb2537b4
feat_: batch all telemetry data and send request every 10 seconds (#5251)
* fix_: add status telemetry client to cli

* feat_: call telemetry when pushing an envelope

* feat_: log status version in all telemetry calls

* feat_: batch all telemetry data and send request every 10 seconds
2024-06-13 15:31:09 -07:00
kaichao 47899fd045
feat_: hash based query for outgoing messages. (#5217)
* feat_: hash based query for outgoing messages.

* chore_: more logs

* chore_: fix comments

* chore_: do not lock when send queries

* chore_: use constant for magic number

* chore_: remove message ids from query queue after ack

* chore_: fix ack clean process

* chore_: fix message resend test

* chore_: add test for waku confirm message sent.

* chore_: fix tests.

* chore_: fix more

* chore_: set store peer id when mailserver updates

* fix_: tests

* chore_: increase max hash query length

* chore_: remove debug log of ack message

* chore_: remove automatic peer selection

* chore_: mark raw message to sent after ack

* chore_: fix test

* chore_: fix test
2024-06-11 15:45:01 +08:00
richΛrd b1433e6bfa
chore_: improve store logging (#5298)
- Logs the requestID in the response
- Log cursor details
- Refactor_: reduce number of parameters for query and fix tests
- Fix_: rename `DefaultShard` to `DefaultNonProtectedShard`
2024-06-06 09:52:51 -04:00
kaichao 1c116589cd
chore_: bump go-waku (#5269) 2024-06-03 21:19:18 +08:00
Andrea Maria Piana 0061c563f2
bug_: fix resend type for private group messages (#5258)
* chore_: bump go-waku

* fix_: fix resend type for private group messages

---------

Co-authored-by: Richard Ramos <info@richardramos.me>
2024-05-30 14:18:54 +02:00
richΛrd 69514629f0
refactor(wakuv2)_: handle discv5 restarts in a single place (#5213) 2024-05-28 13:38:06 -04:00
richΛrd 0a9cff25f4
refactor_: do not use identify protocol, and actually use the nodes from the config for peer exchange (#5212) 2024-05-27 09:43:47 -04:00
richΛrd 94e5431ea3
chore: add fleet ENRs to the list of discv5 nodes (#5184)
* chore_: add fleet ENRs to the list of discv5 nodes
* fix_: handle scenario in which there are dnsDiscoveryURLs and enrs in the DiscV5Bootnode list
2024-05-26 19:16:26 -04:00
Vaclav Pavlin d31200e55b
fix(waku)_: only use requestId in store query if defined (#5211) 2024-05-23 08:26:21 -04:00
richΛrd 0937850268
chore: use custom logger for message timestamps and log query duration (#5199)
* chore_: use custom logger for message timestamps
* chore_: log store query duration
2024-05-21 13:29:29 -04:00
richΛrd 82ae41f6d5
feat(waku2)_: allow using an env variable to set the node's key (#5152) 2024-05-15 19:15:22 -04:00
richΛrd 9e0fb30f8d
chore_: bump go-waku (#5150) 2024-05-15 19:15:00 -04:00
frank 22bea87bb2 fix_: not enough peers to publish 2024-05-02 05:40:49 +08:00
frank 8c5a735438 feat_: retry sending specific messages 2024-05-02 05:40:49 +08:00
richΛrd 3de2756660
refactor_: expire envelope cache (#5061) 2024-04-17 08:19:03 -04:00
Igor Sirotin 5a54d703c7
fix: flaky test request profile info (#4895) 2024-03-08 15:50:46 +00:00
frank f4013665b1 use config instead of setting in filter_manager.go 2024-03-07 17:20:53 +08:00
frank 7eeeb07cab fix: no peers discovered 2024-03-07 17:20:53 +08:00
frank 881da9e654 address review feedback 2024-03-07 17:20:53 +08:00
frank 3de945feaf chore: WakuV2 use config instead of settings 2024-03-07 17:20:53 +08:00
Prem Chaitanya Prathi 92685e5a7b chore: use available peers rather than connected peers, since in case of lightMode we don't connect to all boot nodes 2024-03-07 17:20:53 +08:00
Prem Chaitanya Prathi 500755110a fix: peer exchange client logic to identify and connect to peers 2024-03-07 17:20:53 +08:00
Vitaly Vlasov 81be9ea190 fix crashes on mobile 2024-03-07 17:20:53 +08:00
Prem Chaitanya Prathi 16b3e99205 chore: fix lint errors 2024-03-07 17:20:53 +08:00
Prem Chaitanya Prathi cbec0dc726 fix: disable discv5 in lightmode and enable peer-exchange in lightmode 2024-03-07 17:20:53 +08:00
Vitaly Vlasov 637ce1d5eb Only use DiscV5 for full nodes 2024-03-07 17:20:53 +08:00
Igor Sirotin 515dbdf2b3
fix: waku connection status subscription lock (#4762)
* fix: lock waku connStatus subscription for sending
* update skipped tests
2024-02-19 23:44:38 +00:00
Igor Sirotin 44c39d345e
chore: testWakuV2Config (#4704)
* chore: testWakuV2Config

* renamed to `NewTestWakuV2`
2024-02-12 12:53:15 +00:00
Igor Sirotin 8fa31df498
chore: better lib-p2p logging (#4706)
* fix: libp2p debug level follows main logger level
* chore: implement pubsub tracer
2024-02-12 12:10:29 +00:00
Andrea Maria Piana 605fe40e32 Fix encryption metadata issues #4613
This commit fixes a few issues with communities encryption:

Key distribution was disconnected from the community description, this created a case where the key would arrive after the community description and that would result in the client thinking that it was kicked.
To overcome this, we added a message that signals the user that is kicked. Also, we distribute the key with the community description so that there's no more issues with timing.
This is a bit expensive for large communities, and it will require some further optimizations.

Key distribution is now also connected to the request to join response, so there are no timing issues.

Fixes an issue with key distribution (race condition) where the community would be modified before being compared, resulting in a comparison of two identical communities, which would result in no key being distributed. This commit only partially address the issue.
2024-02-07 10:25:41 +00:00
richΛrd f31808d938
fix: full nodes should run filter and lightpush (#4655) 2024-01-31 14:45:24 -04:00
Igor Sirotin 0c2a935578
fix: fetch history when back online (#4651) 2024-01-30 18:13:18 +00:00
richΛrd 98c1ebec05
fix: handle community shard unassignment and update (#4627) 2024-01-30 13:56:59 -04:00
richΛrd e2341248b3
feat: add peer count to logs (#4648) 2024-01-30 12:42:26 -04:00
Igor Sirotin 1f0fc2935c
fix: clear waku envelopes cache when deleting a chat (#4621)
* chore: extract `ErrPermissionToJoinNotSatisfied`
* chore: disable resending messages in communities tests
* chore: move newTestMessenger extraOptions to testMessengerConfig
* chore: `WithTestStoreNode` messenger option
* feat: waku `ClearEnvelopesCache` method
* fix: call `ClearEnvelopesCache` when deleting chat
* chore: `TestBecomeMemberPermissions` checks messages after rejoin
2024-01-30 13:43:34 +00:00
frank 69948a7024
fix: endless logout (#4563) 2024-01-30 19:45:08 +08:00
frank 9050ed7aaf
fix: waku udp port conflict (#4610) 2024-01-24 09:09:43 +08:00
Igor Sirotin 2a91fba1fa
chore: added and fixed some logs (#4577)
* chore: logs for successful storenode request
2024-01-23 18:20:01 +00:00
Andrea Maria Piana e65760ca85 Add basic peersyncing
This commit adds basic syncing capabilities with peers if they are both
online.

It updates the work done on MVDS, but I decided to create the code in
status-go instead, since it's very tight to the application (similarly
the code that was the inspiration for mvds, bramble, is all tight
together at the database level).

I reused parts of the protobufs.

The flow is:

1) An OFFER message is sent periodically with a bunch of message-ids and
   group-ids.
2) Anyone can REQUEST some of those messages if not present in their
   database.

3) The peer will then send over those messages.

It's disabled by default, but I am planning to add a way to set up the
flags.
2024-01-23 12:46:17 +00:00
Siddarth Kumar 926f6a3c72 Revert "test: bump go-libp2p"
This reverts commit d0ca4447c6.
2024-01-18 20:29:33 +00:00
Richard Ramos d0ca4447c6
test: bump go-libp2p 2024-01-18 14:28:06 +00:00
Igor Sirotin a5acffc001
fix: TestFetchRealCommunity with shards fleet (#4553)
* fix: TestFetchRealCommunity with shards fleet
* fix: DefaultShardPubsubTopic
* chore: print waku query request id
2024-01-16 13:38:41 +03:00
kaichao 0c474bb42c
feat: use automatic peer selection for filter. (#4531)
* feat: use automatic peer selection for filter.

* fix: remove sucess peers too.

* chore: remove filter manager state of peer candidates
2024-01-12 15:09:35 +08:00
richΛrd 1ac99c2dcb
chore: bump go-waku (#4505) 2024-01-05 16:43:05 -04:00
Igor Sirotin 195982c950
fix_(StoreNodeRequestManager): various fixes and improvements (#4509) 2023-12-27 13:53:19 +00:00
richΛrd b6f2aced08
chore: add dns discovery URL to the list of default nodes and add extra logging for store queries (#4490)
* chore: add dns discovery url to list of default discv5 nodes

* chore: add logs for store queries
2023-12-20 10:04:00 -04:00
Igor Sirotin 92f0479a0c
chore: fetch contact with `StoreNodeRequestManager` (#4484) 2023-12-20 12:49:12 +00:00
richΛrd e5ce11f067
fix: panic in broadcast when logout (#4475) 2023-12-19 11:42:50 -04:00
Igor Sirotin e3ef8c649a
chore: store node requests manager (#4446) 2023-12-15 19:50:12 +00:00
richΛrd 93aeefcb89
refactor(waku2): publish messages on a goroutine (#4466) 2023-12-15 08:02:04 -04:00
kaichao 9be202be50
chore: fix peer exchange and unit testing (#4381)
* chore: peer exchange local test

* chore: use local enr resolver

* remove light client close peer.

* fix: peer exchange not start when enabled

* chore: remove ws config

* uncomment light client

* use resolver option

* chore: use option param for dns resolver

* chore: fix vendor changes.

* chore: lint
2023-12-07 08:28:08 +08:00
kaichao e28eca1c54
Bump waku version (#4407)
* chore: make vendor

* chore: fix data types

* Update wakuv2/common/message.go

Co-authored-by: richΛrd <info@richardramos.me>

* Update wakuv2/common/message.go

Co-authored-by: richΛrd <info@richardramos.me>

* Update wakuv2/persistence/dbstore.go

Co-authored-by: richΛrd <info@richardramos.me>

* chore: use safe method to get timestamp.

* chore: use proto.Uint64 to convert reference

* chore: manual fix lint issue when import dependency

---------

Co-authored-by: richΛrd <info@richardramos.me>
2023-12-05 12:29:27 +08:00
Vitaly Vlasov 384543d3a6 Re-send messages with ResendAutomatically=true 2023-11-30 10:46:38 +02:00
Igor Sirotin e32c5546e1
test: request community from storenode (#4364)
* feat: request community info from storenode test

* shutdownWaitGroup

* fix requestCommunityInfoFromMailserver timestamp roundin
2023-11-25 23:24:20 +00:00
Pablo Lopez e9c9bbb7b1
Adding store wakuv2 test (#4340)
* chore: wip tests

* chore: wakuv2 store test

* chore: cleanup

* chore: cleanup

* chore: avoid using sleep in test

* cleanup
2023-11-24 10:13:26 +00:00
Prem Chaitanya Prathi f622265679 chore: fix linter error 2023-11-23 11:16:50 -04:00
Prem Chaitanya Prathi 9510ad0f5d fix: specify clusterid as 16 when using sharding fleet so that metadata proto negotiation doesn't fail 2023-11-23 11:16:50 -04:00
Richard Ramos 40c9ba82a5 fix(wakuv2): store query pubsubTopic 2023-11-22 14:02:50 -04:00
Vitaly Vlasov 1794b93c16 Always set PubsubTopic in filters 2023-11-18 02:26:34 +02:00
Richard Ramos d2578bf0cb fix: reset context on stop 2023-11-17 13:37:41 -04:00
Richard Ramos 573a97791e fix: start idService and add wg.Done to fnApply 2023-11-17 13:37:41 -04:00
Richard Ramos 89385cfcaf fix(waku2): use a cancelable context for initial bootnode discovery 2023-11-17 13:37:41 -04:00
Pablo Lopez f1d31d6339
add message hash to login (#4315) 2023-11-15 16:33:23 +02:00
richΛrd de12ca885c
fix(wakuv2): don't wait for connections to login (#4293)
- Identify will not block logout
- Use `peer.AddrInfo` instead of multiaddresses
- Modifies some logs to reduce noise
2023-11-10 11:31:59 -04:00
richΛrd 2c954d42cf
feat: replace DefaultPubsubTopic by Shard 32 (#4161) 2023-11-09 20:29:15 -04:00
Michal Iskierko 02e4cc6e1f feat: Send envelopes to telemetry service
Issue #12430
2023-11-08 11:34:10 +01:00
Igor Sirotin 9f69c32593
fix(wakuv2): post envelopes previously cached but not processed (#4246) 2023-11-02 14:16:59 +00:00
Andrea Maria Piana d3558d8e09
Disable waku test 2023-11-01 19:27:53 +00:00
Andrea Maria Piana a89a386c82
Increase timeout on waku tests 2023-11-01 15:47:41 +00:00
Patryk Osmaczko 2a5d3e5490 fix: mitigate flakiness of TestBasicWakuV2
fixes: #4105
2023-10-20 20:41:23 +02:00
Vitaliy Vlasov abac55c778 fix waku2 FilterManager.pingPeers() range variable issue; modify logs 2023-10-17 17:24:33 +03:00
richΛrd ba5ed725ce
waku2: static shards (#3944)
- use protected topics for communities
- associate chats to pubsub topics and populate these depending if the chat belongs to a community or not
- mailserver functions should be aware of pubsub topics
- generate private key for pubsub topic protection when creating a community
- add shard cluster and index to communities
- setup shards for existing communities
- distribute pubsubtopic password
- fix: do not send the requests to join and cancel in the protected topic
- fix: undefined shard values for backward compatibility
- refactor: use shard message in protobuffers
2023-10-12 15:21:49 -04:00
Andrea Maria Piana 9034f0a984 Update waku nodes 2023-10-05 18:20:44 +01:00
Vitaliy Vlasov 356b2f5ca3 Refactor filter health checks; add debugging logs 2023-10-04 00:39:33 +03:00
Richard Ramos 7d03ae8272 feat: enable filter full node via node config 2023-09-28 21:31:20 -04:00
Andrea Maria Piana 6f4f57b7a8
Handle nil error on request to join 2023-08-29 13:04:00 +01:00
Prem Chaitanya Prathi b3a93fff12
chore: bumped go-waku to recent commit which contains changes for improved peer management 2023-08-29 13:02:19 +01:00
Richard Ramos d62a5736a5 fix: return default pubsub topic, and add more logs 2023-08-25 13:34:34 -04:00
Richard Ramos d900974519 fix: code review 2023-08-23 13:56:00 -04:00
Richard Ramos f9ec588c4e feat: use protected topics for communities
refactor: associate chats to pubsub topics and populate these depending if the chat belongs to a community or not
refactor: add pubsub topic to mailserver batches
chore: ensure default relay messages continue working as they should
refactor: mailserver functions should be aware of pubsub topics
fix: use []byte for communityIDs
2023-08-23 13:56:00 -04:00
Richard Ramos b9b86712e7 refactor: use context instead of quit channel in wakuv2/waku.go 2023-08-15 10:15:45 -04:00
Andrea Maria Piana df2a5d8e21 Update go waku & Shorten shutdown 2023-07-17 14:44:32 +01:00
Andrea Maria Piana c3ae0e98fe Enable lightclient 2023-07-07 14:49:40 +01:00
Andrea Maria Piana e08aad7d73 Send account/settings on login 2023-07-06 17:35:32 +01:00
Richard Ramos 28229faec0 chore: bump libp2p related dependencies 2023-06-30 14:23:08 -04:00