Commit Graph

55 Commits

Author SHA1 Message Date
kaichao d8a49c538b
Reset MVDS epoch after peer is online (#5349)
* feat_: reset epoch for online peer

* chore_: fix

* chore_: refactor

* chore_: update mvds

* chore_: fix lint

* chore_: update mvds

* chore_: make vendor

* chore_: fix tst

* chore_: tuning store query hash parameter

* chore_: non-blocking mvds status change channel
2024-06-27 09:54:31 +08:00
richΛrd 7731f56b17
feat(waku2)_: check for missing messages (#5281) 2024-06-26 14:25:44 -04:00
richΛrd 4c090d5699
chore(waku2)_: increase store query pagesize (#5341) 2024-06-14 12:35:16 -04:00
Andrea Maria Piana 1844ab7c83
feat(spiff-workflow)_: disable store nodes for spiffy workfluffy 2024-06-14 10:07:16 +02:00
Pablo Lopez 3f19972c8e
enable custom community storenodes (#4532)
* enable custom community store nodes

* fix

* fix

* fix

* fix

* cleanup

* fix

* migration

* fix

* cleanup

* fix

* cleanup

* fix

* fix

* cleanup

* message to update the community storenodes

* rename

* fix test

* wait for availability only if global storenode

* fix test

* fix typo

* sync community storenodes

* remove unused

* add tests

* fix imports

* fix todo

* unused

* pr comments

* pr feedback

* revert merge deleted

* fix lint

* fix db and perform ms request

* typo

* fix log

* fix go imports

* refactor handle message

* cleanup public message

* add tests

* fix test

* cleanup test

* fix test

* avoid making one file to big to keep codeclimate from complaining

* fix lint

* revert

* Update protocol/storenodes/database.go

Co-authored-by: richΛrd <info@richardramos.me>

* Update protocol/messenger_mailserver_cycle.go

Co-authored-by: richΛrd <info@richardramos.me>

* PR comment

* fix tx

* proto files

* pr comment

---------

Co-authored-by: richΛrd <info@richardramos.me>
2024-02-20 17:49:39 +02:00
Igor Sirotin 8d4f4904c6
fix: better calculation of from/to mailserver batch parameters (#4702) 2024-02-12 12:20:56 +00:00
richΛrd e9b10c4beb
chore: sync history regardless of connection type (#4656) 2024-02-02 13:03:55 -04:00
Igor Sirotin 0c2a935578
fix: fetch history when back online (#4651) 2024-01-30 18:13:18 +00:00
Igor Sirotin 2a91fba1fa
chore: added and fixed some logs (#4577)
* chore: logs for successful storenode request
2024-01-23 18:20:01 +00:00
Andrea Maria Piana e65760ca85 Add basic peersyncing
This commit adds basic syncing capabilities with peers if they are both
online.

It updates the work done on MVDS, but I decided to create the code in
status-go instead, since it's very tight to the application (similarly
the code that was the inspiration for mvds, bramble, is all tight
together at the database level).

I reused parts of the protobufs.

The flow is:

1) An OFFER message is sent periodically with a bunch of message-ids and
   group-ids.
2) Anyone can REQUEST some of those messages if not present in their
   database.

3) The peer will then send over those messages.

It's disabled by default, but I am planning to add a way to set up the
flags.
2024-01-23 12:46:17 +00:00
Michal Iskierko 53ac61bb8b fix: a few adjustments for matterbridge
- exposig Online info
- adding ability to disable backup loop and auto message loop

Issue #12710
2024-01-08 14:21:26 +01:00
Igor Sirotin 195982c950
fix_(StoreNodeRequestManager): various fixes and improvements (#4509) 2023-12-27 13:53:19 +00:00
Igor Sirotin e3ef8c649a
chore: store node requests manager (#4446) 2023-12-15 19:50:12 +00:00
richΛrd 42cf9fa740
fix: mailserver batches should be grouped by pubsub topics (#4458) 2023-12-13 17:14:06 -04:00
Andrea Maria Piana 3ab1afaae8 feat_: Allow to set custom nodes & log level 2023-11-29 09:31:02 +00:00
Andrea Maria Piana e1f61515ef feat_: Add sync chat endpoint 2023-11-29 09:31:02 +00:00
Patryk Osmaczko 4a9220bf96 refactor: eliminate datasync initialization logic duplication 2023-11-18 15:37:54 +01:00
Vitaly Vlasov 1794b93c16 Always set PubsubTopic in filters 2023-11-18 02:26:34 +02:00
richΛrd 2c954d42cf
feat: replace DefaultPubsubTopic by Shard 32 (#4161) 2023-11-09 20:29:15 -04:00
Igor Sirotin c6ff315dfc
fix: set pubsubTopic in request community info from mailserver (#4194) 2023-10-25 11:13:35 +01:00
Jonathan Rainville c8910d69fe
feat(pairing): sync 1-1 chats and groups on pairing devices (#4102)
Fixes #3547
2023-10-17 09:24:15 -04:00
richΛrd ba5ed725ce
waku2: static shards (#3944)
- use protected topics for communities
- associate chats to pubsub topics and populate these depending if the chat belongs to a community or not
- mailserver functions should be aware of pubsub topics
- generate private key for pubsub topic protection when creating a community
- add shard cluster and index to communities
- setup shards for existing communities
- distribute pubsubtopic password
- fix: do not send the requests to join and cancel in the protected topic
- fix: undefined shard values for backward compatibility
- refactor: use shard message in protobuffers
2023-10-12 15:21:49 -04:00
Richard Ramos f9ec588c4e feat: use protected topics for communities
refactor: associate chats to pubsub topics and populate these depending if the chat belongs to a community or not
refactor: add pubsub topic to mailserver batches
chore: ensure default relay messages continue working as they should
refactor: mailserver functions should be aware of pubsub topics
fix: use []byte for communityIDs
2023-08-23 13:56:00 -04:00
Andrea Maria Piana 8dd1b66d69 Always use protobufs by reference & generate handlers 2023-08-22 12:08:54 +01:00
Sale Djenic 56cad423f0 chore: `forceFetchingBackup` param added to `RequestAllHistoricMessages` and `RequestAllHistoricMessagesWithRetries` functions 2023-06-09 17:42:11 +02:00
Richard Ramos 975883407f refactor: improve mailserver batch handling 2023-05-02 07:27:57 -04:00
Richard Ramos 2ec97ee861 feat: use 24h batches for retrieving message history 2023-05-02 07:25:59 -04:00
Richard Ramos 323a6ce6dd test: processMailserverBatch 2023-04-30 10:04:04 -04:00
Richard Ramos ddfe16f56f fix: add cursors to store queries 2023-01-27 17:03:59 -04:00
Richard Ramos eb36eb3ee5 fix: cancel mailserver requests when stopping messenger 2023-01-25 11:15:12 -04:00
Richard Ramos b1960a6ca9 chore: increase mailserver timeout to 30s 2023-01-23 16:29:23 -04:00
Andrea Maria Piana 8593866862 Restart discovery when it fails
When discovery fails to be seeded with bootstrap/fallback nodes, it
never recovers.

This commit changes the behavior so that status-go retries fetching
bootnodes, and restarts discovery when that happens.
2022-12-14 20:54:48 +00:00
Richard Ramos eaced1c1e9 fix(store): generate signals per storenode request with custom requestIDs 2022-11-28 08:40:58 -04:00
Richard Ramos 4f5a25befa fix: limit store queries to 10 content topics 2022-11-09 21:38:05 +03:00
Pascal Precht 7032fc9dcc Introduce community history archive routine
This introduces logic needed to:

- Create WakuMessageArchives and and indices from store waku messages
- History archive torrent data to disk and create .torrent file from
  that
- Seed and unseed history archive torrents as necessary
- Starting/stopping the torrent client
- Enabling/disabling community history support for individual components
  and starting/stopping the routine intervals accordingly

This does not yet handle magnet links (#2568)

Closes #2567
2022-04-06 13:00:33 +02:00
Andrea Maria Piana d60a6713fe Enable mailserver cycle by default
This commit enables mailserver cycle logic by default and make a few
changes:

1) Nodes are graylisted instead of being blacklisted for a set amount of
   time. The reason is that if we blacklist, any cut in connectivity
   might result in long delays before reconnecting, especially on spotty
   connections.

2) Fixes an issue on the devp2p server, whereby the node would not
   connect to one of the static nodes since all the connection slots
   where filled. The fix is a bit inelegant, it always connects to
   static nodes, ignoring maxpeers, but it's tricky to get it to work
   since the code is clearly not written to select a specific node.

3) Adds support to pinned mailservers

4) Add retries to mailservers requests. It uses a closure for now, I
   think we should eventually have a channel etc, but I'd leave that for
   later.
2022-03-19 08:56:22 +00:00
Roman Volosovskyi 62a0439234
[#12783] fix index out of range when there are no filters 2022-02-02 16:52:50 +02:00
Roman Volosovskyi c0c90947ab
[#12783] Improve order of mailserver requests 2022-02-02 09:38:30 +02:00
Richard Ramos 6dcc059031
chore: increase mailserver request timeout (#2497) 2022-01-26 11:20:27 -04:00
Richard Ramos ebd4511d24
fix: remove mailserver topic when deactivating a chat (#2460)
This fixes an issue where re-joining a chat does not load the messages again from the mailserver
2021-12-09 17:55:00 -04:00
Andrea Maria Piana f60cf5cd47 Wait for p2p messages to be processed 2021-12-01 12:58:46 +00:00
Anthony Laibe 25e1c64ef5
fix: iterate through message with batch topics (#2444)
When iterating, reuse the list of topics rather
than all topics
2021-11-26 12:59:53 +01:00
Andrea Maria Piana 6724cf4c75 Block contact endpoint 2021-11-15 18:53:35 +00:00
Andrea Maria Piana ca6246d5f2 Set last backup 2021-11-15 18:53:35 +00:00
Roman Volosovskyi 53bbfb3f08
Allow sending messages offline 2021-11-11 19:12:31 +02:00
Richard Ramos 7440c184c0
feat: add batch information and request ID in history requests (#2397) 2021-11-09 16:16:19 -05:00
Andrea Maria Piana 2d13fa1e25 Add logging of message-id 2021-11-01 11:53:08 +00:00
Richard Ramos 3b05d379bd
feat: add signals that are emitted before/after history messages are requested (#2378)
* fix: sqlite variable limit

* feat: add signal for history messages
2021-09-28 14:41:35 -04:00
Andrea Maria Piana c35ceff06e Set timeout on mailserver requests 2021-08-03 11:16:37 +02:00
RichΛrd bfdc000bbc
go-waku: RequestAllHistoricMessages (#2258)
* feat: update wakuv2 store

* check online using waku version for now

* fix peerId and from/to multiplier

* fix: lint

* fix: handle waku2 cursors

* fix: code review

* fix: code review 2

* update go-waku version

* update vendor folder
2021-07-21 15:02:50 -04:00