Commit Graph

180 Commits

Author SHA1 Message Date
Mykhailo Prakhov 30fee0cfd3 fix(communities)_: validate if RawMessage from DB is valid before sending it 2024-07-23 19:54:39 +02:00
Prem Chaitanya Prathi 9a703162c4
refactor: only use shards (#5474)
* refactor_: use shards by default

* fix_: metadata lightclient check

* chore_: update go-waku
2024-07-15 20:55:12 +05:30
Andrea Maria Piana d69b3e5cc9 chore_: limit max number of ephemeral keys to 3
This commit limits the number of ephemeral keys used by the push
notification client to 3, as it was noticed that it would greatly
increase the number of filters installed and was unbound.
2024-07-04 15:23:25 +01:00
kaichao d8a49c538b
Reset MVDS epoch after peer is online (#5349)
* feat_: reset epoch for online peer

* chore_: fix

* chore_: refactor

* chore_: update mvds

* chore_: fix lint

* chore_: update mvds

* chore_: make vendor

* chore_: fix tst

* chore_: tuning store query hash parameter

* chore_: non-blocking mvds status change channel
2024-06-27 09:54:31 +08:00
Andrea Maria Piana 5cfaa00195 chore_: Add script for extracting logs 2024-06-24 11:56:57 +01:00
Sale Djenic 9c3b49b866 chore_: celer bridge disabled, due to making correct routes using a single (hop) bridge 2024-06-14 15:44:38 +02:00
kaichao 47899fd045
feat_: hash based query for outgoing messages. (#5217)
* feat_: hash based query for outgoing messages.

* chore_: more logs

* chore_: fix comments

* chore_: do not lock when send queries

* chore_: use constant for magic number

* chore_: remove message ids from query queue after ack

* chore_: fix ack clean process

* chore_: fix message resend test

* chore_: add test for waku confirm message sent.

* chore_: fix tests.

* chore_: fix more

* chore_: set store peer id when mailserver updates

* fix_: tests

* chore_: increase max hash query length

* chore_: remove debug log of ack message

* chore_: remove automatic peer selection

* chore_: mark raw message to sent after ack

* chore_: fix test

* chore_: fix test
2024-06-11 15:45:01 +08:00
richΛrd b1433e6bfa
chore_: improve store logging (#5298)
- Logs the requestID in the response
- Log cursor details
- Refactor_: reduce number of parameters for query and fix tests
- Fix_: rename `DefaultShard` to `DefaultNonProtectedShard`
2024-06-06 09:52:51 -04:00
Igor Sirotin 19875ed9b5
fix: archive decryption fail (#5076)
* fix_: delay archive import if decryption failed

* chore_: minor cleanup

* chore_: test archive import of encrypted channel

* test(TestImportDecryptedArchiveMessages)_: first iteration

* feat_: GetHashRatchetMessagesCountForGroup

* chore_: log community description and shard info message ids

* test(TestImportDecryptedArchiveMessages)_: cleanup

* fix_: rebase issues

* chore_: remove temporal test

* test(TestImportDecryptedArchiveMessages)_: cleanup

* chore_: lint fix

* fix(TestImportDecryptedArchiveMessages)_: ForceMembersReevaluation
2024-05-27 13:49:09 +01:00
Parvesh Monu 644f64b260
feat_: Return parsed bridge messages (#5192) 2024-05-27 13:32:30 +05:30
Patryk Osmaczko 00ee631a22 chore_: cleanup `hash_ratchet_encrypted_messages` periodically
closes: #5036
2024-05-16 16:15:30 +02:00
frank 8c5a735438 feat_: retry sending specific messages 2024-05-02 05:40:49 +08:00
Patryk Osmaczko 9e5462eb9e feat_: introduce forward error correction in segmentation layer
closes: #4330
2024-04-17 22:05:53 +02:00
Michal Iskierko 9db149d4f6 feat: handling replies for bridged messages
BridgeMessage struct has MessageID and ParentMessageID.
MessageID keeps the original Discord message ID.
ParentMessageID keeps the original Discord parent message ID (response_to).
When the new bridge message is received, corresponding status message response_to field is updated.

Issue #13258
2024-03-13 12:50:15 +01:00
Ibrahem Khalil 8c0e24dc26
Add favicons to external link previews (#4788) 2024-03-12 22:47:51 +02:00
Igor Sirotin c8044bf400
chore: disable curated communities loop in tests (#4894) 2024-03-09 09:42:06 +00:00
Patryk Osmaczko bafb0a71d0 fix: prevent `panic: send on closed channel` 2024-02-27 11:00:29 +01:00
Igor Sirotin 432bfeea2f
fix: flaky `MessengerStoreNodeRequestSuite` (#4721) 2024-02-12 22:30:19 +00:00
Mikhail Rogachev 3ea2002904
Feat: Modify grants for profile showcase use (#4694)
* chore: regenerate proto binaries

* feat: remove unused grant code for community's canPost
2024-02-11 15:11:04 +03:00
Andrea Maria Piana 605fe40e32 Fix encryption metadata issues #4613
This commit fixes a few issues with communities encryption:

Key distribution was disconnected from the community description, this created a case where the key would arrive after the community description and that would result in the client thinking that it was kicked.
To overcome this, we added a message that signals the user that is kicked. Also, we distribute the key with the community description so that there's no more issues with timing.
This is a bit expensive for large communities, and it will require some further optimizations.

Key distribution is now also connected to the request to join response, so there are no timing issues.

Fixes an issue with key distribution (race condition) where the community would be modified before being compared, resulting in a comparison of two identical communities, which would result in no key being distributed. This commit only partially address the issue.
2024-02-07 10:25:41 +00:00
Michal Iskierko 73a5189398 feat(BridgeMessage): Add new type of chat message content: BridgeMessage
BridgeMessage is a type of chat message content which will be sent from  Matterbridge.
It contains fields:
- bridge name - depends on the used bridge, eg. "discord", "slack", etc...
- user name - username the message was received from
- content - message content
- user avatar
- message id
- parent message id - used in case of replies

Message is saved to a separated table: bridge_messages, similarly to discord messages.
The user_messages table is untouched.
bridge_messages table contains user_messages_id in order to join with user_messages table.

Issue #13098
2024-01-30 12:14:54 +01:00
Godfrain Jacques 5f6f7e502d
(fix/status-go) fix profile picture update/removal (#4570)
This PR fixes [9947](status-im/status-desktop#9947) and contains :

    - Commit to fix the changing of custom picture and having the change
      reflected on contact's side
    - Commit to fix the deleting of picture and having the change reflected
      on contact's side
    - Rename confusing `ImageType` to `ImageFormat`
2024-01-24 12:09:28 -08:00
Andrea Maria Piana e65760ca85 Add basic peersyncing
This commit adds basic syncing capabilities with peers if they are both
online.

It updates the work done on MVDS, but I decided to create the code in
status-go instead, since it's very tight to the application (similarly
the code that was the inspiration for mvds, bramble, is all tight
together at the database level).

I reused parts of the protobufs.

The flow is:

1) An OFFER message is sent periodically with a bunch of message-ids and
   group-ids.
2) Anyone can REQUEST some of those messages if not present in their
   database.

3) The peer will then send over those messages.

It's disabled by default, but I am planning to add a way to set up the
flags.
2024-01-23 12:46:17 +00:00
Michal Iskierko 3d7ab79d91 fix: execute BeforeDispatch only once
Issue #4557
2024-01-15 17:03:33 +01:00
Godfrain Jacques 9d8b55e3d9
This PR (status-go) fixes #12821 (#4516)
- Add album_images column to the user_messages table
- Migrate the database
2024-01-08 10:18:57 -08:00
Michal Iskierko 53ac61bb8b fix: a few adjustments for matterbridge
- exposig Online info
- adding ability to disable backup loop and auto message loop

Issue #12710
2024-01-08 14:21:26 +01:00
Mykhailo Prakhov 134137f9c5
feat: implemented a mechanism to retrieve shard information for a given community ID (#4499)
* feat: a mechanism to retrieve shard information for a given community ID
2023-12-22 13:37:37 +01:00
Andrea Maria Piana 88fd1ce942 Surface acks, offers, requests to messenger 2023-12-18 10:33:05 +00:00
Igor Sirotin e32c5546e1
test: request community from storenode (#4364)
* feat: request community info from storenode test

* shutdownWaitGroup

* fix requestCommunityInfoFromMailserver timestamp roundin
2023-11-25 23:24:20 +00:00
Patryk Osmaczko d73d1e2488 chore_: adapt tracking for segmented messages
closes: #4310
2023-11-24 17:41:52 +01:00
Patryk Osmaczko bddc48f265 fix: allocate space for hashes instead of creation
Otherwise hashes would be twice in size and messages tracking
wouldn't work.
2023-11-20 21:26:19 +01:00
Patryk Osmaczko 4a9220bf96 refactor: eliminate datasync initialization logic duplication 2023-11-18 15:37:54 +01:00
Patryk Osmaczko 7b3013a010 chore: remove message chunking from datasync
It was redundant to recently introduced messages segmentation layer.
2023-11-18 15:37:54 +01:00
Vitaly Vlasov 1794b93c16 Always set PubsubTopic in filters 2023-11-18 02:26:34 +02:00
Patryk Osmaczko d04f99d56d chore: cleanup message segments
closes: #4297
2023-11-15 16:52:35 +01:00
Patryk Osmaczko de29ec71ae feat: enable messages segmentation
closes: status-im/status-desktop#12188
2023-11-13 17:32:00 +01:00
Patryk Osmaczko 6bb806caad feat: introduce messages segmentation 2023-11-13 17:32:00 +01:00
Patryk Osmaczko fa44e03ac2 refactor: eliminate logic duplication in MessageSender.HandleMessages 2023-11-13 17:32:00 +01:00
richΛrd 2c954d42cf
feat: replace DefaultPubsubTopic by Shard 32 (#4161) 2023-11-09 20:29:15 -04:00
Mikhail Rogachev 03c32f620f
feat: Profile showcase data sharing (#4209)
* feat: Add profile showcase messaging part with ecrypted data

* feat: Separate profile showcase categories to provide ablity to store custom data

* fix: review fixes

* feat: move profile showcase out of contact data

* fix: create index on contact id for profile tables

* chore: remove logger from link preview
2023-11-09 22:59:01 +04:00
Igor Sirotin f8236fb555
fix: link previews public keys (#4272) 2023-11-09 16:24:35 +00:00
Patryk Osmaczko f7042e4b9e refactor: extract layers in StatusMessage
Extracted:
- TransportLayer
- EncryptionLayer
- ApplicationLayer
2023-11-08 21:46:22 +01:00
Andrea Maria Piana 85f8c92cde Delete processed messages & add tests
This commit adds a test for out of order messages, which were only
implicitly tested.
It also deletes them after being processed, otherwise they would be
reproceessed each time a message was sent
2023-10-30 14:11:29 +00:00
Andrea Maria Piana 360c2d50d4 Fix issues with out of order encryption and batched messages
This commit fixes 3 issues:

1) In some cases, the hash ratchet was not correctly found
2) Out of order messages were not processed correctly as the wrong error
   was returned
3) Batched non datasync messages were not processed correctly

Fixes: #4170
2023-10-30 10:11:32 +00:00
Andrea Maria Piana 23f71c1125 Fix encryption id && rekey with a single message
This commit changes the format of the encryption id to be based off 3
things:

1) The group id
2) The timestamp
3) The actual key

Previously this was solely based on the timestamp and the group id, but
this might lead to conflicts. Moreover the format of the key was an
uint32 and so it would wrap periodically.

The migration is a bit tricky, so first we cleared the cache of keys,
that's easier than migrating, and second we set the new field hash_id to
the concatenation of group_id / key_id.
This might lead on some duplication in case keys are re-received, but it
should not have an impact on the correctness of the code.

I have added 2 tests covering compatibility between old/new clients, as
this should not be a breaking change.

It also adds a new message to rekey in a single go, instead of having to
send multiple messages
2023-10-24 20:48:54 +01:00
frank 3326362b90
optimised finding server cert (#4148)
* optimised finding server cert

* make sure `close(done)` invoked only once

* remove sleep

* resolve IDE warning

* refactor for findServerCert
2023-10-18 14:17:49 +08:00
Igor Sirotin aded258ccb
feature: Unfurl status links (#4033) 2023-10-13 13:25:34 +01:00
Richard Ramos de5c7b8e5b feat: shard fleet
Adds shard.test fleet and changes the non protected shard index to 64
2023-10-12 17:07:57 -04:00
Richard Ramos ace5b26137 fix: do not send the requests to join and cancel in the protected topic 2023-10-12 17:07:57 -04:00
richΛrd ba5ed725ce
waku2: static shards (#3944)
- use protected topics for communities
- associate chats to pubsub topics and populate these depending if the chat belongs to a community or not
- mailserver functions should be aware of pubsub topics
- generate private key for pubsub topic protection when creating a community
- add shard cluster and index to communities
- setup shards for existing communities
- distribute pubsubtopic password
- fix: do not send the requests to join and cancel in the protected topic
- fix: undefined shard values for backward compatibility
- refactor: use shard message in protobuffers
2023-10-12 15:21:49 -04:00