Commit Graph

69 Commits

Author SHA1 Message Date
Richard Ramos 62d4ce9b7e
fix_: simplify `performStorenodeTask` 2024-11-07 16:05:29 +07:00
Richard Ramos 441fee76d2
fix!: code review 2024-11-07 16:05:27 +07:00
Richard Ramos 95b6a17719
feat!: extract storenode cycle to go-waku api 2024-11-07 16:05:25 +07:00
Patryk Osmaczko 291b87f82d chore(mailserver)_: reduce debug logs size in messenger_mailserver
Logs from `messenger_mailserver` occupy 40 out of 100MB of data in
my `geth.log`.

Instead of extending `logger` with many parameters, such as `chatIDs`,
`topic`, etc., the hash of those parameters is calucalted and kept as a
context.

iterates: #5889
2024-09-30 18:45:29 +02:00
frank 38308d48f2
feat_: log on panic (#5849)
* feat_: log error and stacktrace when panic in goroutine

* test_: add test TestSafeGo

* chore_: rename logAndCall to call

* chore_: rename SafeGo to Go

* chore_: make lint-fix

* chore_: use t.Cleanup

* chore_: Revert "chore_: use t.Cleanup"

This reverts commit 4eb420d179cc0e208e84c13cb941e6b3d1ed9819.

* chore_: Revert "chore_: make lint-fix"

This reverts commit fcc995f157e671a4229b47419c3a0e4004b5fdab.

* chore_: Revert "chore_: rename SafeGo to Go"

This reverts commit a6d73d6df583f313032d79aac62f66328039cb55.

* chore_: Revert "chore_: rename logAndCall to call"

This reverts commit 8fbe993bedb9fbba67349a44f151e2dd5e3bc4cc.

* chore_: Revert "test_: add test TestSafeGo"

This reverts commit a1fa91839f3960398980c6bf456e6462ec944819.

* chore_: Revert "feat_: log error and stacktrace when panic in goroutine"

This reverts commit f612dd828fa2ce410d0e806fe773ecbe3e86a68a.

* feat_: log error and stacktrace when panic in goroutine

* chore_: make lint-fix

* chore_: rename logAndCall to call

* chore_: renaming LogOnPanic

* chore_: update rest goroutine function calls

* chore_: make lint-fix
2024-09-27 06:37:32 +08:00
richΛrd c24eba8af2
fix_: make mailserver availability subscriptions concurrency safe (#5804) 2024-09-05 15:18:23 -04:00
richΛrd 771a0c3562
refactor: remove wakuv1 code from mailserver cycle and add ENR to list of storenodes (#5753)
* test_: remove eth.prod mailservers from cycle
* test_: disable store request for wakuv1
* chore_: add ENR and remove V1 code
* refactor_: storenode cycle and allow ENRs and multiaddresses
2024-09-05 10:25:26 -04:00
Pablo Lopez 0c0888927c
fix_: store query lastrequest (#5767) 2024-08-27 08:58:52 +03:00
richΛrd 4aa51f9882
fix_: wait for mailserver available signal to change missing messages criteria (#5686) 2024-08-16 14:26:10 -04:00
richΛrd 48ebe24f8e
fix_: tmp remove subscription to mailserver available signal to avoid leak (#5684) 2024-08-09 10:04:45 -04:00
richΛrd 20d6d4eb9a
refactor: extract missing messages logic to go-waku (#5638) 2024-08-06 14:49:26 -04:00
richΛrd 5d309e2c64
feat_: use storev3 instead of v2 for history queries (#5123) 2024-07-30 14:28:27 -04:00
Andrea Maria Piana 3cc4767638
Check for mobile data connection if setting is on (#5511)
* feat_: Check for mobile data connection if setting is on

* fix_: check code control flag inside  asyncRequestAllHistoricMessages

---------

Co-authored-by: Igor Sirotin <sirotin@status.im>
2024-07-26 22:34:57 +01:00
Andrey Bocharnikov 5d113071db fix(NodeConfig)_: Do not add backoff if user action triggered store node disconnection
fixes status-im/status-desktop#14929
2024-07-26 22:24:55 +07:00
kaichao d8a49c538b
Reset MVDS epoch after peer is online (#5349)
* feat_: reset epoch for online peer

* chore_: fix

* chore_: refactor

* chore_: update mvds

* chore_: fix lint

* chore_: update mvds

* chore_: make vendor

* chore_: fix tst

* chore_: tuning store query hash parameter

* chore_: non-blocking mvds status change channel
2024-06-27 09:54:31 +08:00
richΛrd 7731f56b17
feat(waku2)_: check for missing messages (#5281) 2024-06-26 14:25:44 -04:00
richΛrd 4c090d5699
chore(waku2)_: increase store query pagesize (#5341) 2024-06-14 12:35:16 -04:00
Andrea Maria Piana 1844ab7c83
feat(spiff-workflow)_: disable store nodes for spiffy workfluffy 2024-06-14 10:07:16 +02:00
Pablo Lopez 3f19972c8e
enable custom community storenodes (#4532)
* enable custom community store nodes

* fix

* fix

* fix

* fix

* cleanup

* fix

* migration

* fix

* cleanup

* fix

* cleanup

* fix

* fix

* cleanup

* message to update the community storenodes

* rename

* fix test

* wait for availability only if global storenode

* fix test

* fix typo

* sync community storenodes

* remove unused

* add tests

* fix imports

* fix todo

* unused

* pr comments

* pr feedback

* revert merge deleted

* fix lint

* fix db and perform ms request

* typo

* fix log

* fix go imports

* refactor handle message

* cleanup public message

* add tests

* fix test

* cleanup test

* fix test

* avoid making one file to big to keep codeclimate from complaining

* fix lint

* revert

* Update protocol/storenodes/database.go

Co-authored-by: richΛrd <info@richardramos.me>

* Update protocol/messenger_mailserver_cycle.go

Co-authored-by: richΛrd <info@richardramos.me>

* PR comment

* fix tx

* proto files

* pr comment

---------

Co-authored-by: richΛrd <info@richardramos.me>
2024-02-20 17:49:39 +02:00
Igor Sirotin 8d4f4904c6
fix: better calculation of from/to mailserver batch parameters (#4702) 2024-02-12 12:20:56 +00:00
richΛrd e9b10c4beb
chore: sync history regardless of connection type (#4656) 2024-02-02 13:03:55 -04:00
Igor Sirotin 0c2a935578
fix: fetch history when back online (#4651) 2024-01-30 18:13:18 +00:00
Igor Sirotin 2a91fba1fa
chore: added and fixed some logs (#4577)
* chore: logs for successful storenode request
2024-01-23 18:20:01 +00:00
Andrea Maria Piana e65760ca85 Add basic peersyncing
This commit adds basic syncing capabilities with peers if they are both
online.

It updates the work done on MVDS, but I decided to create the code in
status-go instead, since it's very tight to the application (similarly
the code that was the inspiration for mvds, bramble, is all tight
together at the database level).

I reused parts of the protobufs.

The flow is:

1) An OFFER message is sent periodically with a bunch of message-ids and
   group-ids.
2) Anyone can REQUEST some of those messages if not present in their
   database.

3) The peer will then send over those messages.

It's disabled by default, but I am planning to add a way to set up the
flags.
2024-01-23 12:46:17 +00:00
Michal Iskierko 53ac61bb8b fix: a few adjustments for matterbridge
- exposig Online info
- adding ability to disable backup loop and auto message loop

Issue #12710
2024-01-08 14:21:26 +01:00
Igor Sirotin 195982c950
fix_(StoreNodeRequestManager): various fixes and improvements (#4509) 2023-12-27 13:53:19 +00:00
Igor Sirotin e3ef8c649a
chore: store node requests manager (#4446) 2023-12-15 19:50:12 +00:00
richΛrd 42cf9fa740
fix: mailserver batches should be grouped by pubsub topics (#4458) 2023-12-13 17:14:06 -04:00
Andrea Maria Piana 3ab1afaae8 feat_: Allow to set custom nodes & log level 2023-11-29 09:31:02 +00:00
Andrea Maria Piana e1f61515ef feat_: Add sync chat endpoint 2023-11-29 09:31:02 +00:00
Patryk Osmaczko 4a9220bf96 refactor: eliminate datasync initialization logic duplication 2023-11-18 15:37:54 +01:00
Vitaly Vlasov 1794b93c16 Always set PubsubTopic in filters 2023-11-18 02:26:34 +02:00
richΛrd 2c954d42cf
feat: replace DefaultPubsubTopic by Shard 32 (#4161) 2023-11-09 20:29:15 -04:00
Igor Sirotin c6ff315dfc
fix: set pubsubTopic in request community info from mailserver (#4194) 2023-10-25 11:13:35 +01:00
Jonathan Rainville c8910d69fe
feat(pairing): sync 1-1 chats and groups on pairing devices (#4102)
Fixes #3547
2023-10-17 09:24:15 -04:00
richΛrd ba5ed725ce
waku2: static shards (#3944)
- use protected topics for communities
- associate chats to pubsub topics and populate these depending if the chat belongs to a community or not
- mailserver functions should be aware of pubsub topics
- generate private key for pubsub topic protection when creating a community
- add shard cluster and index to communities
- setup shards for existing communities
- distribute pubsubtopic password
- fix: do not send the requests to join and cancel in the protected topic
- fix: undefined shard values for backward compatibility
- refactor: use shard message in protobuffers
2023-10-12 15:21:49 -04:00
Richard Ramos f9ec588c4e feat: use protected topics for communities
refactor: associate chats to pubsub topics and populate these depending if the chat belongs to a community or not
refactor: add pubsub topic to mailserver batches
chore: ensure default relay messages continue working as they should
refactor: mailserver functions should be aware of pubsub topics
fix: use []byte for communityIDs
2023-08-23 13:56:00 -04:00
Andrea Maria Piana 8dd1b66d69 Always use protobufs by reference & generate handlers 2023-08-22 12:08:54 +01:00
Sale Djenic 56cad423f0 chore: `forceFetchingBackup` param added to `RequestAllHistoricMessages` and `RequestAllHistoricMessagesWithRetries` functions 2023-06-09 17:42:11 +02:00
Richard Ramos 975883407f refactor: improve mailserver batch handling 2023-05-02 07:27:57 -04:00
Richard Ramos 2ec97ee861 feat: use 24h batches for retrieving message history 2023-05-02 07:25:59 -04:00
Richard Ramos 323a6ce6dd test: processMailserverBatch 2023-04-30 10:04:04 -04:00
Richard Ramos ddfe16f56f fix: add cursors to store queries 2023-01-27 17:03:59 -04:00
Richard Ramos eb36eb3ee5 fix: cancel mailserver requests when stopping messenger 2023-01-25 11:15:12 -04:00
Richard Ramos b1960a6ca9 chore: increase mailserver timeout to 30s 2023-01-23 16:29:23 -04:00
Andrea Maria Piana 8593866862 Restart discovery when it fails
When discovery fails to be seeded with bootstrap/fallback nodes, it
never recovers.

This commit changes the behavior so that status-go retries fetching
bootnodes, and restarts discovery when that happens.
2022-12-14 20:54:48 +00:00
Richard Ramos eaced1c1e9 fix(store): generate signals per storenode request with custom requestIDs 2022-11-28 08:40:58 -04:00
Richard Ramos 4f5a25befa fix: limit store queries to 10 content topics 2022-11-09 21:38:05 +03:00
Pascal Precht 7032fc9dcc Introduce community history archive routine
This introduces logic needed to:

- Create WakuMessageArchives and and indices from store waku messages
- History archive torrent data to disk and create .torrent file from
  that
- Seed and unseed history archive torrents as necessary
- Starting/stopping the torrent client
- Enabling/disabling community history support for individual components
  and starting/stopping the routine intervals accordingly

This does not yet handle magnet links (#2568)

Closes #2567
2022-04-06 13:00:33 +02:00
Andrea Maria Piana d60a6713fe Enable mailserver cycle by default
This commit enables mailserver cycle logic by default and make a few
changes:

1) Nodes are graylisted instead of being blacklisted for a set amount of
   time. The reason is that if we blacklist, any cut in connectivity
   might result in long delays before reconnecting, especially on spotty
   connections.

2) Fixes an issue on the devp2p server, whereby the node would not
   connect to one of the static nodes since all the connection slots
   where filled. The fix is a bit inelegant, it always connects to
   static nodes, ignoring maxpeers, but it's tricky to get it to work
   since the code is clearly not written to select a specific node.

3) Adds support to pinned mailservers

4) Add retries to mailservers requests. It uses a closure for now, I
   think we should eventually have a channel etc, but I'd leave that for
   later.
2022-03-19 08:56:22 +00:00