Commit Graph

850 Commits

Author SHA1 Message Date
Jonathan Rainville 880a201037 fix(reply-test): fix reply test because msgs could come out of order
Fixes #3070
2023-01-11 16:24:42 -05:00
Patryk Osmaczko 186454528b chore: make `createMessagesRequestV2` respect `mailserverRequestTimeout`
closes: #3053
2023-01-11 20:26:10 +01:00
Jonathan Rainville e98e6b61af fix: use the uncompressed community pubkey as topic to receive requests 2023-01-11 13:41:29 -04:00
Jonathan Rainville 0da5e621ac test(persistence): improve reply test to test deleted reply 2023-01-10 15:34:48 -05:00
Jonathan Rainville 574bf90618 fix(message_persistence): don't return reply data of deleted messages 2023-01-10 15:34:48 -05:00
Jonathan Rainville c4bf60b815 feat: make replies act as mentions
and add a reply test to the messenger
2023-01-10 13:39:57 -05:00
frank ec7c0e9c7d
Sync all devices after initial pairing (#3047) 2023-01-06 20:21:14 +08:00
frank bea710c8be
fix inconsistent contact request state (#3056) 2023-01-06 18:24:39 +08:00
Richard Ramos ad67eb52e5 chore: log errors generated while registering filters 2023-01-05 13:30:46 -05:00
Parvesh Monu d40290a649
Persist switcher cards (#3049) 2023-01-05 21:41:57 +05:30
frank fe2270540e
fix TestGroupChatMembersRemoval sometimes failed (#3037) 2023-01-05 23:37:50 +08:00
Jakub Sokołowski cceba787fa
replace deprecated ioutil.TempDir with os.MkdirTemp
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-01-03 15:40:11 +01:00
frank 9137257638
synchronize display name (#2989) 2022-12-29 14:16:19 +08:00
Mikhail Rogachev 5005b7b3cf
feat(ActivityCenter): Add missing AC notifications for verification requests (#3029)
* feat(ActivityCenter): Add missing AC notifications for verification requests
* fix(Contacts): Fix updates for trusted and untrustworthy statuses
* feat(ActivityCenter): Add test for trusted verification request and AC notifications
* feat(Contacts): Trusted and untrustworthy statuses should be unknown for verificated side
2022-12-26 08:35:46 +04:00
Sale Djenic d98d83d308 chore: code review comments applied
- sync clock set for all settings prop participating in syncing
- if other sql error happens we return immediately
2022-12-23 15:07:14 +01:00
Sale Djenic 1340a55c1d feat: backup profile data to waku and sync them from waku
Changes applied here introduce backing up profile data (display name and identity
images) to waku and fetch them from waku. Information about those data is sent
as a separate signal to a client via `sync.from.waku.profile` signal.

New signal `sync.from.waku.progress` is introduced which will be used to notify a client
about the progress of fetching data from waku.
2022-12-23 15:07:14 +01:00
Pascal Precht b4bdfd3df6 Keep track of last seen magnetlink
This is to prevent processing magnetlinks if they haven't changed
2022-12-20 15:16:08 +01:00
Pascal Precht e7d827fff1 feat: recgonize `0x00001` as system mention
This "id" or "tag" will be interpeted/interpolated as `@everyone` by
clients.

For this change to work

https://github.com/status-im/markdown/pull/5

is needed.
2022-12-20 15:13:22 +01:00
Pascal Precht b1ff13eb7d fix: ensure queries coalesce `album_id` value when scanning
Otherwise databases with existing messages that don't have an `album_id`
will run into and `converting NULL to string is unsupported` error.

See: https://github.com/status-im/status-go/pull/3021#issuecomment-1351623814
2022-12-16 11:10:44 +01:00
Jonathan Rainville 4a0eb56574 fix: fix special messages not being received because not encrypted 2022-12-15 10:48:41 -05:00
Andrea Maria Piana 8593866862 Restart discovery when it fails
When discovery fails to be seeded with bootstrap/fallback nodes, it
never recovers.

This commit changes the behavior so that status-go retries fetching
bootnodes, and restarts discovery when that happens.
2022-12-14 20:54:48 +00:00
Pascal Precht dd8be1d8b0 feat(communities): add magnetlink to RequestToJoinResponse
This is done so that when member join a community by being accepted by
the community owner, they also receive the most up-to-date magnetlink
along with it.
2022-12-14 13:43:43 +01:00
Omar Basem d03691c2f2
Images Album (#3021)
* feat: add `AlbumID` field to messages
2022-12-14 16:25:45 +04:00
Patryk Osmaczko 3025cdcba4 feat: add `FirstUnseenMessageID` API 2022-12-14 09:49:54 +01:00
Mikhail Rogachev cd9fb48579
feat(ActivityCenter): Add message for canceling contact requests (#2992)
* feat(ActivityCenter): Add message for canceling contact requests

* feat(Contacts): GetVerificationRequestSentTo returns last verification request

* Fix error message in protocol/messenger_contact_verification.go

Co-authored-by: Jonathan Rainville <rainville.jonathan@gmail.com>

Co-authored-by: Jonathan Rainville <rainville.jonathan@gmail.com>
2022-12-14 12:27:02 +04:00
Pascal Precht f5dfa58602 refactor(communities): better archive download handling
This commit makes a few changes to the community history archive
download routine to make it more robust:

1. Prior to this commit, even when there were no archives to be
   downloaded, we were still trying to extract messages from archive
   data.
2. Logs have been improved as they were sometimes showing confusing
   information
3. We now handle interruption of ongoing download + data import much
   better in case of multiple magnetlinks being processed in roughly the
   same time.
4. We now keep track of which archive has been successfully imported
   into the database. Without this, Status would consider any downloaded
   archives as "done" even though they haven't actually been imported
   into the database yet. This way Status should be able to pick up its
   work were it left of the last time, in case a user closes the app, or
   another magnetlink interrupts the ongoing process.
2022-12-13 16:13:15 +01:00
Pascal Precht 4bddfbd466 chore: better community archive protocol logs 2022-12-13 16:13:15 +01:00
Richard Ramos c7ce9adb5e debug: add more logs to determine why community requests are missing 2022-12-13 10:25:24 -04:00
frank 7f109710e2
fix: TestDeleteMessageForEveryone fail (#3020) 2022-12-13 16:06:01 +08:00
Omar Basem d1a4b53d5c
Remove group members (#3011)
* remove list of members from group
2022-12-10 08:26:51 +04:00
Jonathan Rainville cee5313e0e hotfix: comment out failing test 2022-12-09 14:43:56 -05:00
Jonathan Rainville d87ed0e357 fix(community): don't delete chat and messages when leaving community
This creates a smoother experience for users when they leave a community since they can see the exact same messages they had before without having to rely on the mailserver.
2022-12-09 14:43:56 -05:00
yqrashawn 49b3c573f6
fix: do not filter out deleted messages in MessageByChatID (#3008) 2022-12-08 18:13:56 +08:00
Pascal Precht 8ac154ee0c fix: don't keep Messenger from starting when torrent client fails to
start

There are various cases that could make the torrent client fail to
start. We don't want this to keep the entire messenger from starting.
2022-12-06 18:19:36 +01:00
Michal Iskierko 8fc7d921a7 fix(ImportCommunity): Setting admin roles during import community.
Issue #7414
2022-12-06 10:25:21 +01:00
Pascal Precht 22b5690cda feat: add signal indicating that history messages are being imported
In order to give clients more insights about archive messages being
processed, we're adding this additional signal that informs clients when
the import of downloaded history archive messages has started.
2022-12-05 10:51:53 +01:00
frank 684e9654de
Allow owner/admin to delete messages of a community (#2958) 2022-12-02 19:34:02 +08:00
frank 8865a0e4b0
fix map iteration order (#2991) 2022-12-02 18:21:47 +08:00
Pascal Precht 194f26f3b4 feat: add signal for starting history archive downloads 2022-12-01 16:12:00 +01:00
Michal Iskierko 6e96586bdb fix(RequestCommunityInfoFromMailserver): Add useDatabase flag to read/not read community info before requesting mailserver..
Fix #2878
2022-11-30 11:10:13 +01:00
Samuel Hawksby-Robinson eadb698603 ColourHash is a distinct type that has a discrete pattern
This change codifies that a ColourHash is a slice of [2]int, as code depends on the inner slice to have exactly 2 indexes
2022-11-28 16:26:07 +00:00
Richard Ramos eaced1c1e9 fix(store): generate signals per storenode request with custom requestIDs 2022-11-28 08:40:58 -04:00
Richard Ramos ee6bda5bcc feat: add wakuext_listenAddresses 2022-11-28 08:10:51 -04:00
Samuel Hawksby-Robinson f87a38074a Addressing feedback 2022-11-25 11:33:58 +00:00
Samuel Hawksby-Robinson 3e8eed5faf Added device name set on Messenger.Start() 2022-11-25 11:33:58 +00:00
Richard Ramos a6c7067f3b feat: add multiaddresses to waku2 peers 2022-11-24 11:47:06 -04:00
Pascal Precht 019d520e28 fix(communities): properly delete remove chats when handling community
updates

When channels where removed from communities by the community owner,
community members would not remove the chats in their own databases.

See https://github.com/status-im/status-desktop/issues/8000 for more
info.

This commit ensures that nodes delete removed chats from their local
chats table and also deregister the corresponding transport filters.
2022-11-24 11:50:25 +01:00
Patryk Osmaczko 22fc10c248 chore: add group messages delivery regression test
part of: status-im/status-desktop#8263
2022-11-18 14:31:25 +01:00
Andrea Maria Piana 1036243276 Disable abridged sending 2022-11-18 11:43:23 +00:00
Andrea Maria Piana 9b670ff453 Correctly abridge events after changes in joined users 2022-11-18 11:43:23 +00:00
Pascal Precht 4d93ea83be fix: typo in message handler which I found quite annoying 2022-11-17 19:15:14 +01:00
yqrashawn 68d2d6bdfb
feat: delete message for new design (#2922) 2022-11-17 18:11:58 +08:00
Roman Volosovskyi 9227c631c3
Fix overwriting of content type to unknown on message editing 2022-11-17 07:11:12 +01:00
Andrea Maria Piana 2572321063 Add notification for trustworthy/untrustworthy 2022-11-16 16:17:13 +00:00
Andrea Maria Piana 98f7324ae1 Add reply message 2022-11-16 16:17:13 +00:00
Andrea Maria Piana c85df62d3c Sync community encryption keys 2022-11-15 10:11:01 +00:00
Roman Volosovskyi 7662cd613a
Allow to pass activity center status as a parameter 2022-11-14 10:17:43 +01:00
Patryk Osmaczko e48083e31d fix: make blocking contact local only
fixes: status-im/status-desktop#7127
2022-11-11 15:44:22 +01:00
Patryk Osmaczko b5887c9c60 chore: extend `TestRetrieveBlockedContact`
Extended test with unblock scenario.
2022-11-11 15:44:22 +01:00
Richard Ramos 4f5a25befa fix: limit store queries to 10 content topics 2022-11-09 21:38:05 +03:00
Richard Ramos bdbb9e2748 chore: info log the message id and envelope hash 2022-11-09 17:39:17 +03:00
Richard Ramos 0f7c9f52d8 feat: mark automatic status updates as ephemeral 2022-11-09 15:49:26 +03:00
Pascal Precht d216840db8 feat(communities): add community images to import progress signal
This is needed so clients can render the selected community image
while the import is running.
2022-11-07 17:16:00 +01:00
Richard Ramos cd34538937 fix: set waku version to mailservers and set min peers for relay to 1 2022-11-07 18:03:52 +03:00
Pascal Precht c528660aa3 fix(communities): improve error message about duplicated chat names
We don't allow multiple channels with the same name in communities.
Discord allows for multiple channels with the same name (living in
different categories), so this is an error case in our import tool.

This commit improves the user facing error message of this scenario.
2022-11-04 13:24:20 +01:00
Pascal Precht 8ad9fb7713 fix(discord): always emit import error if it's of type error
We've been limiting the amount of errors being emitted to clients
to reduce payload pressure and also due to the fact that we won't be
rendering more than 3 error items in the UI.

We want to let actual errors through (as opposed to warnings), even if
the limit was reached.
2022-11-04 10:06:04 +01:00
Ibrahem Khalil bfab85f73b
Fix private group members details not being sent on preview (#2936) 2022-11-03 11:57:51 +03:00
Pascal Precht caa20e616e feat(communities): handle markdown in imported discord messages 2022-10-28 20:03:39 +02:00
Andrea Maria Piana 2162b76546 Reintroduce api endpoint 2022-10-28 15:46:58 +01:00
Boris Melnik 63d46481fd
feat(communities): Cancel request to join community (#2915) 2022-10-28 11:41:20 +03:00
Pascal Precht 0bdb596d3b feat: introduce discord import tool 2022-10-28 09:52:26 +02:00
Richard Ramos bb4237f616 fix: wakuv2 mailserver cycle ticker 2022-10-27 09:52:41 -04:00
Andrea Maria Piana de61ed1213 Handle identity verifications 2022-10-26 17:19:44 +01:00
Andrea Maria Piana a89f4b2d71 Add read/unread messenger functions for activity center 2022-10-26 17:19:44 +01:00
Mikhail Rogachev 60cb15739d
feat(ActivityCenter): Add community membership AC notifications (#2886)
* feat(ActivityCenter): Add community request AC notification

* feat(ActivityCenter): Add CommunityID to AC notification

* feat(ActivityCenter): Add membership status for community membership AC notifications

* feat(ActivityCenter): Add tests for community notifications and fix naming

* Add notification for kicked from community action

* feat(ActivityCenter): Fix for missing notification objects for tests
2022-10-26 02:06:20 +04:00
Pascal Precht 0ccdec2985 feat(CommunitiesManager): introduce `CreateHistoryArchiveTorrentFromMessages` API
Prior to this commit we had a `CreateHistoryArchiveTorrent()` API which
takes a `startDate`, an `endDate` and a `partition` to create a bunch of
message archives, given a certain time range.

The function expects the messages to live in the database, which means,
all messages that need to be archived have to be saved there at some
point.

This turns out to be an issue when importing communities from third
party services, where, sometimes, there are several thousands of messages
including attachment payloads, that have to be save to the database
first.

There are only two options to get the messages into the database:

1. Make one write operation with all messages - this slow, takes a long
   time and blocks the database until done
2. Create message chunks and perform multiple write operations - this is
   also slow, takes long but makes the database a bit more responsive as
   it's many smaller operations instead of one big one

Option 2) turned out to not be super feasible either as sometimes,
inserting even a single such message can take up to 10 seconds
(depending on payload)

Which brings me to the third option.

**A third option** is to not store those imported messages as waku
message into the database, just to later query them again to create the
archives, but instead create the archives right away from all the
messages that have been loaded into memory.

This is significantly faster and doesn't block the database.

To make this possible, this commit introduces
a `CreateHistoryArchiveTorrentFromMessages()` API, and
a `CreateHistoryArchiveTorrentFromDB()` API which can be used for
different use cases.
2022-10-24 13:28:55 +02:00
Pascal Precht 7eb66d09e7 feat(communities): add encryption suppord to archive protocol 2022-10-24 12:35:38 +02:00
Pascal Precht b2dce92f3f fix(Communities): correct query params order when updating community
settings

Turns out `UpdateCommunitySettings()` has never worked. Two parameters
where in the wrong order, cause the SQL statement to never find the row
it has to update.
2022-10-24 12:02:52 +02:00
Andrea Maria Piana bef211b5d6 Send all encryption keys 2022-10-20 12:19:44 +01:00
Pascal Precht 69e84b5673 feat(communities): retry downloading archive data
When fetching torrent info after receiving a magnet link,
it can happen that the request times out.

We want to retry downloading the data again at least once more
before giving up
2022-10-17 11:33:02 +02:00
Pascal Precht dd49c4c954 feat(communities): add archiveLogger for stdout logs
The default logger writes to `geth.log`, which makes debugging
the archive protocol pretty hard.

This adds an additional logger that logs to stdout, while keeping
the default logger intact for production.
2022-10-14 16:31:21 +02:00
Pascal Precht 616a58f5c9 fix(MessengerResponse): ensure `savedAddresses` response as `omitempty`
Otherwise the json response will be `null` instead of an empty array
(which broke desktop): https://github.com/status-im/status-desktop/pull/7888
2022-10-13 14:53:09 +02:00
Anthony Laibe d9b44777c9
feat: update opensea testnet to use goerli (#2894) 2022-10-12 09:59:38 +02:00
Richard Ramos d4d35710be fix: community encryption was not being marshalled 2022-10-10 13:11:10 +02:00
Stefan b9d6e6d8c8 feat: sync saved addresses
Main changes:
- Extend saved addresses DB with sync info: sync timestamp, update timestamp
and soft removed flag
- Create custom protobuf message payload to sync saved addresses
- Cleanup saved addresses on each start of messenger, by deleting
soft removed older entries
- Sync all saved addresses on Messenger.SyncDevices calls
- Sync particular changes to saved addresses
- Add SavedAddressManager instance to messenger
  - Note, can't find a clean way to pass the SavedAddressManager to the
  messenger, so we create another one
- Add tests for sync and new DB API

Closes: #7229
2022-10-10 11:26:35 +02:00
r4bbit.eth 6c148389c1
fix(message_persistence): ensure query cursor value is supplied where (#2898)
needed

Unfortunately, this one slipped through when introducing helper
functions to retrieve messages.

Sometimes, queries need an additional empty `cursor` value.
2022-10-07 13:38:28 -04:00
Stefan daee116878 fix: compiling chat_messsage.proto due to unscoped ContentType 2022-10-07 12:49:04 +02:00
Patryk Osmaczko 73a45dd58e feat: add `SpectateCommunity` api
- added `SpectateCommunity` endpoint, it is supposed to be used in
  scenarios where we want to "Go to public Community" and see its
  content without joining
- added `spectated` field to `Community`, it means we are observing the
  community and its chats but we are not members

Use case:
https://github.com/status-im/status-desktop/issues/7072#issuecomment-1246560885
2022-10-06 21:21:37 +02:00
Omar Basem 938e7987ca
feat: edit/delete/reply for emoji/sticker/audio (#2771)
* feat: edit/delete/reply for emoji/sticker/audio
2022-10-05 21:54:47 +04:00
Pascal Precht 9bb1e75939 fix: ensure discord attachment existence is checked with message ID 2022-10-05 13:17:57 +02:00
Pascal Precht f7faebe775 chore: save discord message should use REPLACE INTO 2022-10-05 13:13:38 +02:00
Pascal Precht ffc9aea24f fix: ensure torrent download does not finish too early 2022-10-05 13:13:16 +02:00
Michał 057c7915f4
chore: use provided pictures visibility options when creating settings db entry (#2867)
When initially creating settings, properties 'initialize profile_pictures_show_to'
and 'profile_pictures_visibility' are set according to the provided Setting object.
2022-10-05 12:53:42 +02:00
Pascal Precht 1eb8a5cc2c feat(MediaServer): add support for discord author and attachment URLs 2022-09-29 16:33:15 +02:00
Pascal Precht b01a861e8e feat: add `DiscordMessageAttachment` types and APIs
This adds a new `DiscordMessageAttachment` type which is part of
`DiscordMessage`. Along with that type, there's also a new database
table for `discord_message_attachments` and corresponding persistence
APIs.

This commit also changes how chat messages are retrieved.
Here's why:

`DiscordMessage` can have multiple `DiscordMessageAttachment`.
A chat message can have a `DiscordMessage`.

Because we're `LEFT JOIN`'ing the discord message attachments into the
chat messages, there's a possibility of multiple rows per message.

Hence, this commit ensures we collect queried discord message
attachments on chat messages.
2022-09-29 11:38:29 +02:00
Pascal Precht c01ad05525 feat(common): add `ThirdPartyID` to `Message` and `protobuf.WakuMessage`
Usually, message IDs are generated by their payload and signature and
in receiving nodes calculated in based on the same data as well.

There's no ID attached to messages in-flight.

This turns out to be a bit of a problem for messages that are being
imported from third party systems like discord, as the conversion
and saving of such messages and handling of their possible assets and
attachments are done in separate steps, which changes the message
payloads after their IDs have been generated.

Hence, we're introducing a `ThirdPartyID` property to `common.Message`
and `protobuf.WakuMessage` so receiving nodes of such messages (via the
archive protocol primarily) can easily detect third party/imported
messages and give them special treatment.
2022-09-29 11:15:47 +02:00
yqrashawn f47cb8572d
feat: delete for me (#2866) 2022-09-28 19:42:17 +08:00
Pascal Precht 81e279a055 chore(community): community itself has admin rights
This might look like a weird requirement at a fist glance.
The reason this is needed, is because some message signals require
admin rights to take effect (e.g. PinMessage).

When messages are imported from third-party services,
translated to status messages, signed by the community, and eventually distributed
via the archive protocol, we need to ensure that messages signed
by the community itself are considered as admin privileges as well,
so they can be correctly replayed into the database.
2022-09-28 13:03:56 +02:00
Richard Ramos b8fd999b54
fix: lint (#2845)
Co-authored-by: Samuel Hawksby-Robinson <samuel@samyoul.com>
2022-09-27 18:59:02 -04:00
Richard Ramos bf9abfc350
feat: make kdf iterations configurable when creating a multiaccount (#2796) 2022-09-27 16:27:20 -04:00