status-go/protocol/messenger_response.go

917 lines
31 KiB
Go
Raw Normal View History

package protocol
import (
2021-01-11 10:32:51 +00:00
"encoding/json"
"golang.org/x/exp/maps"
ensservice "github.com/status-im/status-go/services/ens"
2022-01-17 03:42:11 +00:00
"github.com/status-im/status-go/services/browsers"
"github.com/status-im/status-go/services/wallet"
2022-01-17 03:42:11 +00:00
Anon Metrics Broadcast (#2198) * Protobufs and adapters * Added basic anon metric service and config init * Added fibonacci interval incrementer * Added basic Client.Start func and integrated interval incrementer * Added new processed field to app metrics table * Added id column to app metrics table * Added migration clean up * Added appmetrics GetUnprocessed and SetToProcessedByIDs and tests There was a wierd bug where metrics in the db that did not explicitly insert a value would be NULL, so could not be found by . In addition I've added a new primary id field to the app_metrics table so that updates could be done against very specific metric rows. * Updated adaptors and db to handle proto_id I need a way to distinguish individual metric items from each other so that I can ignore the ones that have been seen before. * Moved incrementer into dedicated file * Resolve incrementer test fail * Finalised the main loop functionality * Implemented delete loop framework * Updated adaptors file name * Added delete loop delay and quit, and tweak on RawMessage gen * Completed delete loop logic * Added DBLock to prevent deletion during mainLoop * Added postgres DB connection, integrated into anonmetrics.Server * Removed proto_id from SQL migration and model * Integrated postgres with Server and updated adaptors * Function name update * Added sample config files for client and server * Fixes and testing for low level e2e * make generate * Fix lint * Fix for receiving an anonMetricBatch not in server mode * Postgres test fixes * Tidy up, make vendor and make generate * delinting * Fixing database tests * Attempted fix of does: cannot open `does' (No such file or directory) not: cannot open `not' (No such file or directory) exist: cannot open `exist' (No such file or directory) error on sql resource loas * Moved all anon metric postgres migration logic and sources into a the protocol/anonmetrics package or sub packages. I don't know if this will fix the does: cannot open `does' (No such file or directory) not: cannot open `not' (No such file or directory) exist: cannot open `exist' (No such file or directory) error that happens in Jenkins but this could work * Lint for the lint god * Why doesn't the linter list all its problems at once? * test tweaks * Fix for wakuV2 change * DB reset change * Fix for postgres db migrations fails * More robust implementation of postgres test setup and teardown * Added block for anon metrics functionality * Version Bump to 0.84.0 * Added test to check anon metrics broadcast is deactivated * Protobufs and adapters * Added basic anon metric service and config init * Added new processed field to app metrics table * Added id column to app metrics table * Added migration clean up * Added appmetrics GetUnprocessed and SetToProcessedByIDs and tests There was a wierd bug where metrics in the db that did not explicitly insert a value would be NULL, so could not be found by . In addition I've added a new primary id field to the app_metrics table so that updates could be done against very specific metric rows. * Updated adaptors and db to handle proto_id I need a way to distinguish individual metric items from each other so that I can ignore the ones that have been seen before. * Added postgres DB connection, integrated into anonmetrics.Server * Removed proto_id from SQL migration and model * Integrated postgres with Server and updated adaptors * Added sample config files for client and server * Fix lint * Fix for receiving an anonMetricBatch not in server mode * Postgres test fixes * Tidy up, make vendor and make generate * Moved all anon metric postgres migration logic and sources into a the protocol/anonmetrics package or sub packages. I don't know if this will fix the does: cannot open `does' (No such file or directory) not: cannot open `not' (No such file or directory) exist: cannot open `exist' (No such file or directory) error that happens in Jenkins but this could work
2021-09-01 12:02:18 +00:00
"github.com/status-im/status-go/appmetrics"
2022-03-24 09:35:56 +00:00
"github.com/status-im/status-go/images"
2022-05-18 10:42:51 +00:00
"github.com/status-im/status-go/multiaccounts/accounts"
Sync Settings (#2478) * Sync Settings * Added valueHandlers and Database singleton Some issues remain, need a way to comparing incoming sql.DB to check if the connection is to a different file or not. Maybe make singleton instance per filename * Added functionality to check the sqlite filename * Refactor of Database.SaveSyncSettings to be used as a handler * Implemented inteface for setting sync protobuf factories * Refactored and completed adhoc send setting sync * Tidying up * Immutability refactor * Refactor settings into dedicated package * Breakout structs * Tidy up * Refactor of bulk settings sync * Bug fixes * Addressing feedback * Fix code dropped during rebase * Fix for db closed * Fix for node config related crashes * Provisional fix for type assertion - issue 2 * Adding robust type assertion checks * Partial fix for null literal db storage and json encoding * Fix for passively handling nil sql.DB, and checking if elem has len and if len is 0 * Added test for preferred name behaviour * Adding saved sync settings to MessengerResponse * Completed granular initial sync and clock from network on save * add Settings to isEmpty * Refactor of protobufs, partially done * Added syncSetting receiver handling, some bug fixes * Fix for sticker packs * Implement inactive flag on sync protobuf factory * Refactor of types and structs * Added SettingField.CanSync functionality * Addressing rebase artifact * Refactor of Setting SELECT queries * Refactor of string return queries * VERSION bump and migration index bump * Deactiveate Sync Settings * Deactiveated preferred_name and send_status_updates Co-authored-by: Andrea Maria Piana <andrea.maria.piana@gmail.com>
2022-03-23 18:47:00 +00:00
"github.com/status-im/status-go/multiaccounts/settings"
2023-12-04 10:18:05 +00:00
walletsettings "github.com/status-im/status-go/multiaccounts/settings_wallet"
"github.com/status-im/status-go/protocol/common"
"github.com/status-im/status-go/protocol/communities"
feat: introduce messenger APIs to extract discord channels As part of the new Discord <-> Status Community Import functionality, we're adding an API that extracts all discord categories and channels from a previously exported discord export file. These APIs can be used in clients to show the user what categories and channels will be imported later on. There are two APIs: 1. `Messenger.ExtractDiscordCategoriesAndChannels(filesToimport []string) (*MessengerResponse, map[string]*discord.ImportError)` This takes a list of exported discord export (JSON) files (typically one per channel), reads them, and extracts the categories and channels into dedicated data structures (`[]DiscordChannel` and `[]DiscordCategory`) It also returns the oldest message timestamp found in all extracted channels. The API is synchronous and returns the extracted data as a `*MessengerResponse`. This allows to make the API available status-go's RPC interface. The error case is a `map[string]*discord.ImportError` where each key is a file path of a JSON file that we tried to extract data from, and the value a `discord.ImportError` which holds an error message and an error code, allowing for distinguishing between "critical" errors and "non-critical" errors. 2. `Messenger.RequestExtractDiscordCategoriesAndChannels(filesToImport []string)` This is the asynchronous counterpart to `ExtractDiscordCategoriesAndChannels`. The reason this API has been added is because discord servers can have a lot of message and channel data, which causes `ExtractDiscordCategoriesAndChannels` to block the thread for too long, making apps potentially feel like they are stuck. This API runs inside a go routine, eventually calls `ExtractDiscordCategoriesAndChannels`, and then emits a newly introduced `DiscordCategoriesAndChannelsExtractedSignal` that clients can react to. Failure of extraction has to be determined by the `discord.ImportErrors` emitted by the signal. **A note about exported discord history files** We expect users to export their discord histories via the [DiscordChatExporter](https://github.com/Tyrrrz/DiscordChatExporter/wiki/GUI%2C-CLI-and-Formats-explained#exportguild) tool. The tool allows to export the data in different formats, such as JSON, HTML and CSV. We expect users to have their data exported as JSON. Closes: https://github.com/status-im/status-desktop/issues/6690
2022-07-13 09:33:53 +00:00
"github.com/status-im/status-go/protocol/discord"
"github.com/status-im/status-go/protocol/encryption/multidevice"
2022-09-29 11:50:23 +00:00
"github.com/status-im/status-go/protocol/protobuf"
"github.com/status-im/status-go/protocol/storenodes"
"github.com/status-im/status-go/protocol/verification"
localnotifications "github.com/status-im/status-go/services/local-notifications"
"github.com/status-im/status-go/services/mailservers"
)
2021-08-20 14:26:13 +00:00
type RemovedMessage struct {
ChatID string `json:"chatId"`
MessageID string `json:"messageId"`
DeletedBy string `json:"deletedBy,omitempty"`
2021-08-20 14:26:13 +00:00
}
2022-02-10 10:00:59 +00:00
type ClearedHistory struct {
ChatID string `json:"chatId"`
ClearedAt uint64 `json:"clearedAt"`
}
type SeenUnseenMessages struct {
ChatID string `json:"chatId"`
Count uint64 `json:"count"`
CountWithMentions uint64 `json:"countWithMentions"`
Seen bool `json:"seen"`
}
type MessengerResponse struct {
feat: introduce messenger APIs to extract discord channels As part of the new Discord <-> Status Community Import functionality, we're adding an API that extracts all discord categories and channels from a previously exported discord export file. These APIs can be used in clients to show the user what categories and channels will be imported later on. There are two APIs: 1. `Messenger.ExtractDiscordCategoriesAndChannels(filesToimport []string) (*MessengerResponse, map[string]*discord.ImportError)` This takes a list of exported discord export (JSON) files (typically one per channel), reads them, and extracts the categories and channels into dedicated data structures (`[]DiscordChannel` and `[]DiscordCategory`) It also returns the oldest message timestamp found in all extracted channels. The API is synchronous and returns the extracted data as a `*MessengerResponse`. This allows to make the API available status-go's RPC interface. The error case is a `map[string]*discord.ImportError` where each key is a file path of a JSON file that we tried to extract data from, and the value a `discord.ImportError` which holds an error message and an error code, allowing for distinguishing between "critical" errors and "non-critical" errors. 2. `Messenger.RequestExtractDiscordCategoriesAndChannels(filesToImport []string)` This is the asynchronous counterpart to `ExtractDiscordCategoriesAndChannels`. The reason this API has been added is because discord servers can have a lot of message and channel data, which causes `ExtractDiscordCategoriesAndChannels` to block the thread for too long, making apps potentially feel like they are stuck. This API runs inside a go routine, eventually calls `ExtractDiscordCategoriesAndChannels`, and then emits a newly introduced `DiscordCategoriesAndChannelsExtractedSignal` that clients can react to. Failure of extraction has to be determined by the `discord.ImportErrors` emitted by the signal. **A note about exported discord history files** We expect users to export their discord histories via the [DiscordChatExporter](https://github.com/Tyrrrz/DiscordChatExporter/wiki/GUI%2C-CLI-and-Formats-explained#exportguild) tool. The tool allows to export the data in different formats, such as JSON, HTML and CSV. We expect users to have their data exported as JSON. Closes: https://github.com/status-im/status-desktop/issues/6690
2022-07-13 09:33:53 +00:00
Contacts []*Contact
Invitations []*GroupChatInvitation
CommunityChanges []*communities.CommunityChanges
AnonymousMetrics []*appmetrics.AppMetric
Mailservers []mailservers.Mailserver
CommunityStorenodes []storenodes.Storenode
feat: introduce messenger APIs to extract discord channels As part of the new Discord <-> Status Community Import functionality, we're adding an API that extracts all discord categories and channels from a previously exported discord export file. These APIs can be used in clients to show the user what categories and channels will be imported later on. There are two APIs: 1. `Messenger.ExtractDiscordCategoriesAndChannels(filesToimport []string) (*MessengerResponse, map[string]*discord.ImportError)` This takes a list of exported discord export (JSON) files (typically one per channel), reads them, and extracts the categories and channels into dedicated data structures (`[]DiscordChannel` and `[]DiscordCategory`) It also returns the oldest message timestamp found in all extracted channels. The API is synchronous and returns the extracted data as a `*MessengerResponse`. This allows to make the API available status-go's RPC interface. The error case is a `map[string]*discord.ImportError` where each key is a file path of a JSON file that we tried to extract data from, and the value a `discord.ImportError` which holds an error message and an error code, allowing for distinguishing between "critical" errors and "non-critical" errors. 2. `Messenger.RequestExtractDiscordCategoriesAndChannels(filesToImport []string)` This is the asynchronous counterpart to `ExtractDiscordCategoriesAndChannels`. The reason this API has been added is because discord servers can have a lot of message and channel data, which causes `ExtractDiscordCategoriesAndChannels` to block the thread for too long, making apps potentially feel like they are stuck. This API runs inside a go routine, eventually calls `ExtractDiscordCategoriesAndChannels`, and then emits a newly introduced `DiscordCategoriesAndChannelsExtractedSignal` that clients can react to. Failure of extraction has to be determined by the `discord.ImportErrors` emitted by the signal. **A note about exported discord history files** We expect users to export their discord histories via the [DiscordChatExporter](https://github.com/Tyrrrz/DiscordChatExporter/wiki/GUI%2C-CLI-and-Formats-explained#exportguild) tool. The tool allows to export the data in different formats, such as JSON, HTML and CSV. We expect users to have their data exported as JSON. Closes: https://github.com/status-im/status-desktop/issues/6690
2022-07-13 09:33:53 +00:00
Bookmarks []*browsers.Bookmark
Settings []*settings.SyncSettingField
IdentityImages []images.IdentityImage
CustomizationColor string
WatchOnlyAccounts []*accounts.Account
Keypairs []*accounts.Keypair
AccountsPositions []*accounts.Account
2023-12-04 10:18:05 +00:00
TokenPreferences []walletsettings.TokenPreferences
CollectiblePreferences []walletsettings.CollectiblePreferences
feat: introduce messenger APIs to extract discord channels As part of the new Discord <-> Status Community Import functionality, we're adding an API that extracts all discord categories and channels from a previously exported discord export file. These APIs can be used in clients to show the user what categories and channels will be imported later on. There are two APIs: 1. `Messenger.ExtractDiscordCategoriesAndChannels(filesToimport []string) (*MessengerResponse, map[string]*discord.ImportError)` This takes a list of exported discord export (JSON) files (typically one per channel), reads them, and extracts the categories and channels into dedicated data structures (`[]DiscordChannel` and `[]DiscordCategory`) It also returns the oldest message timestamp found in all extracted channels. The API is synchronous and returns the extracted data as a `*MessengerResponse`. This allows to make the API available status-go's RPC interface. The error case is a `map[string]*discord.ImportError` where each key is a file path of a JSON file that we tried to extract data from, and the value a `discord.ImportError` which holds an error message and an error code, allowing for distinguishing between "critical" errors and "non-critical" errors. 2. `Messenger.RequestExtractDiscordCategoriesAndChannels(filesToImport []string)` This is the asynchronous counterpart to `ExtractDiscordCategoriesAndChannels`. The reason this API has been added is because discord servers can have a lot of message and channel data, which causes `ExtractDiscordCategoriesAndChannels` to block the thread for too long, making apps potentially feel like they are stuck. This API runs inside a go routine, eventually calls `ExtractDiscordCategoriesAndChannels`, and then emits a newly introduced `DiscordCategoriesAndChannelsExtractedSignal` that clients can react to. Failure of extraction has to be determined by the `discord.ImportErrors` emitted by the signal. **A note about exported discord history files** We expect users to export their discord histories via the [DiscordChatExporter](https://github.com/Tyrrrz/DiscordChatExporter/wiki/GUI%2C-CLI-and-Formats-explained#exportguild) tool. The tool allows to export the data in different formats, such as JSON, HTML and CSV. We expect users to have their data exported as JSON. Closes: https://github.com/status-im/status-desktop/issues/6690
2022-07-13 09:33:53 +00:00
DiscordCategories []*discord.Category
DiscordChannels []*discord.Channel
DiscordOldestMessageTimestamp int
BackupHandled bool
2021-01-11 10:32:51 +00:00
// notifications a list of notifications derived from messenger events
// that are useful to notify the user about
feat: fallback pairing seed (#5614) * feat(pairing)!: Add extra parameters and remove v2 compatibility This commit includes the following changes: I have added a flag to maintain 2.29 compatibility. Breaking change in connection string The local pairing code that was parsing the connection string had a few non-upgradable features: It was strictly checking the number of parameters, throwing an error if the number was different. This made it impossible to add parameters to it without breaking. It was strictly checking the version number. This made increasing the version number impossible as older client would just refuse to connect. The code has been changed so that: Two parameters have been added, installation-id and key-uid. Those are needed for the fallback flow. I have also removed version from the payload, since it wasn't used. This means that we don't support v1 anymore. V2 parsing is supported . Going forward there's a clear strategy on how to update the protocol (append parameters, don't change existing one). https://www.youtube.com/watch?v=oyLBGkS5ICk Is a must watch video for understanding the strategy Changed MessengerResponse to use internally a map of installations rather than an array (minor) Just moving towards maps as arrays tend to lead to subtle bugs. Moved pairing methods to messenger_pairing.go Just moved some methods Added 2 new methods for the fallback flow FinishPairingThroughSeedPhraseProcess https://github.com/status-im/status-go/pull/5567/files#diff-1ad620b07fa3bd5fbc96c9f459d88829938a162bf1aaf41c61dea6e38b488d54R29 EnableAndSyncInstallation https://github.com/status-im/status-go/pull/5567/files#diff-1ad620b07fa3bd5fbc96c9f459d88829938a162bf1aaf41c61dea6e38b488d54R18 Flow for clients Client A1 is logged in Client A2 is logged out Client A1 shows a QR code Client A2 scans a QR code If connection fails on A2, the user will be prompted to enter a seed phrase. If the generated account matches the key-uid from the QR code, A2 should call FinishPairingThroughSeedPhraseProcess with the installation id passed in the QR code. This will send installation information over waku. The user should be shown its own installation id and prompted to check the other device. Client A1 will receive new installation data through waku, if they are still on the qr code page, they should show a popup to the user showing the received installation id, and a way to Enable and Sync, which should call the EnableAndSyncInstallation endpoint. This should finish the fallback syncing flow. Current issues Currently I haven't tested that all the data is synced after finishing the flow. I see that the two devices are paired correctly, but for example the DisplayName is not changed on the receiving device. I haven't had time to look into it further. * test_: add more test for connection string parser * fix_: fix panic when parse old connection string * test_: add comments for TestMessengerPairAfterSeedPhrase * fix_: correct error description * feat_:rename FinishPairingThroughSeedPhraseProcess to EnableInstallationAndPair * fix_: delete leftover * fix_: add UniqueKey method * fix_: unify the response for InputConnectionStringForBootstrapping * fix_: remove fields installationID and keyUID in GethStatusBackend * fix_: rename messenger_pairing to messenger_pairing_and_syncing --------- Co-authored-by: Andrea Maria Piana <andrea.maria.piana@gmail.com>
2024-07-30 09:14:05 +00:00
installations map[string]*multidevice.Installation
notifications map[string]*localnotifications.Notification
requestsToJoinCommunity map[string]*communities.RequestToJoin
chats map[string]*Chat
removedChats map[string]bool
removedMessages map[string]*RemovedMessage
deletedMessages map[string]string
communities map[string]*communities.Community
communitiesSettings map[string]*communities.CommunitySettings
activityCenterNotifications map[string]*ActivityCenterNotification
activityCenterState *ActivityCenterState
messages map[string]*common.Message
pinMessages map[string]*common.PinMessage
discordMessages map[string]*protobuf.DiscordMessage
discordMessageAttachments map[string]*protobuf.DiscordMessageAttachment
discordMessageAuthors map[string]*protobuf.DiscordMessageAuthor
currentStatus *UserStatus
statusUpdates map[string]UserStatus
clearedHistories map[string]*ClearedHistory
verificationRequests map[string]*verification.Request
trustStatus map[string]verification.TrustStatus
emojiReactions map[string]*EmojiReaction
savedAddresses map[string]*wallet.SavedAddress
ensUsernameDetails []*ensservice.UsernameDetail
updatedProfileShowcaseContactIDs map[string]bool
seenAndUnseenMessages map[string]*SeenUnseenMessages
2021-01-11 10:32:51 +00:00
}
func (r *MessengerResponse) MarshalJSON() ([]byte, error) {
responseItem := struct {
Chats []*Chat `json:"chats,omitempty"`
RemovedChats []string `json:"removedChats,omitempty"`
RemovedMessages []*RemovedMessage `json:"removedMessages,omitempty"`
DeletedMessages map[string][]string `json:"deletedMessages,omitempty"`
Messages []*common.Message `json:"messages,omitempty"`
Contacts []*Contact `json:"contacts,omitempty"`
Installations []*multidevice.Installation `json:"installations,omitempty"`
PinMessages []*common.PinMessage `json:"pinMessages,omitempty"`
EmojiReactions []*EmojiReaction `json:"emojiReactions,omitempty"`
Invitations []*GroupChatInvitation `json:"invitations,omitempty"`
CommunityChanges []*communities.CommunityChanges `json:"communityChanges,omitempty"`
RequestsToJoinCommunity []*communities.RequestToJoin `json:"requestsToJoinCommunity,omitempty"`
Mailservers []mailservers.Mailserver `json:"mailservers,omitempty"`
CommunityStorenodes []storenodes.Storenode `json:"communityStorenodes,omitempty"`
Bookmarks []*browsers.Bookmark `json:"bookmarks,omitempty"`
ClearedHistories []*ClearedHistory `json:"clearedHistories,omitempty"`
VerificationRequests []*verification.Request `json:"verificationRequests,omitempty"`
TrustStatus map[string]verification.TrustStatus `json:"trustStatus,omitempty"`
// Notifications a list of notifications derived from messenger events
// that are useful to notify the user about
Notifications []*localnotifications.Notification `json:"notifications"`
Communities []*communities.Community `json:"communities,omitempty"`
CommunitiesSettings []*communities.CommunitySettings `json:"communitiesSettings,omitempty"`
ActivityCenterNotifications []*ActivityCenterNotification `json:"activityCenterNotifications,omitempty"`
ActivityCenterState *ActivityCenterState `json:"activityCenterState,omitempty"`
CurrentStatus *UserStatus `json:"currentStatus,omitempty"`
StatusUpdates []UserStatus `json:"statusUpdates,omitempty"`
Settings []*settings.SyncSettingField `json:"settings,omitempty"`
IdentityImages []images.IdentityImage `json:"identityImages,omitempty"`
CustomizationColor string `json:"customizationColor,omitempty"`
WatchOnlyAccounts []*accounts.Account `json:"watchOnlyAccounts,omitempty"`
Keypairs []*accounts.Keypair `json:"keypairs,omitempty"`
AccountsPositions []*accounts.Account `json:"accountsPositions,omitempty"`
TokenPreferences []walletsettings.TokenPreferences `json:"tokenPreferences,omitempty"`
CollectiblePreferences []walletsettings.CollectiblePreferences `json:"collectiblePreferences,omitempty"`
DiscordCategories []*discord.Category `json:"discordCategories,omitempty"`
DiscordChannels []*discord.Channel `json:"discordChannels,omitempty"`
DiscordOldestMessageTimestamp int `json:"discordOldestMessageTimestamp"`
DiscordMessages []*protobuf.DiscordMessage `json:"discordMessages,omitempty"`
DiscordMessageAttachments []*protobuf.DiscordMessageAttachment `json:"discordMessageAtachments,omitempty"`
SavedAddresses []*wallet.SavedAddress `json:"savedAddresses,omitempty"`
EnsUsernameDetails []*ensservice.UsernameDetail `json:"ensUsernameDetails,omitempty"`
UpdatedProfileShowcaseContactIDs []string `json:"updatedProfileShowcaseContactIDs,omitempty"`
SeenAndUnseenMessages []*SeenUnseenMessages `json:"seenAndUnseenMessages,omitempty"`
2021-01-11 10:32:51 +00:00
}{
Contacts: r.Contacts,
feat: fallback pairing seed (#5614) * feat(pairing)!: Add extra parameters and remove v2 compatibility This commit includes the following changes: I have added a flag to maintain 2.29 compatibility. Breaking change in connection string The local pairing code that was parsing the connection string had a few non-upgradable features: It was strictly checking the number of parameters, throwing an error if the number was different. This made it impossible to add parameters to it without breaking. It was strictly checking the version number. This made increasing the version number impossible as older client would just refuse to connect. The code has been changed so that: Two parameters have been added, installation-id and key-uid. Those are needed for the fallback flow. I have also removed version from the payload, since it wasn't used. This means that we don't support v1 anymore. V2 parsing is supported . Going forward there's a clear strategy on how to update the protocol (append parameters, don't change existing one). https://www.youtube.com/watch?v=oyLBGkS5ICk Is a must watch video for understanding the strategy Changed MessengerResponse to use internally a map of installations rather than an array (minor) Just moving towards maps as arrays tend to lead to subtle bugs. Moved pairing methods to messenger_pairing.go Just moved some methods Added 2 new methods for the fallback flow FinishPairingThroughSeedPhraseProcess https://github.com/status-im/status-go/pull/5567/files#diff-1ad620b07fa3bd5fbc96c9f459d88829938a162bf1aaf41c61dea6e38b488d54R29 EnableAndSyncInstallation https://github.com/status-im/status-go/pull/5567/files#diff-1ad620b07fa3bd5fbc96c9f459d88829938a162bf1aaf41c61dea6e38b488d54R18 Flow for clients Client A1 is logged in Client A2 is logged out Client A1 shows a QR code Client A2 scans a QR code If connection fails on A2, the user will be prompted to enter a seed phrase. If the generated account matches the key-uid from the QR code, A2 should call FinishPairingThroughSeedPhraseProcess with the installation id passed in the QR code. This will send installation information over waku. The user should be shown its own installation id and prompted to check the other device. Client A1 will receive new installation data through waku, if they are still on the qr code page, they should show a popup to the user showing the received installation id, and a way to Enable and Sync, which should call the EnableAndSyncInstallation endpoint. This should finish the fallback syncing flow. Current issues Currently I haven't tested that all the data is synced after finishing the flow. I see that the two devices are paired correctly, but for example the DisplayName is not changed on the receiving device. I haven't had time to look into it further. * test_: add more test for connection string parser * fix_: fix panic when parse old connection string * test_: add comments for TestMessengerPairAfterSeedPhrase * fix_: correct error description * feat_:rename FinishPairingThroughSeedPhraseProcess to EnableInstallationAndPair * fix_: delete leftover * fix_: add UniqueKey method * fix_: unify the response for InputConnectionStringForBootstrapping * fix_: remove fields installationID and keyUID in GethStatusBackend * fix_: rename messenger_pairing to messenger_pairing_and_syncing --------- Co-authored-by: Andrea Maria Piana <andrea.maria.piana@gmail.com>
2024-07-30 09:14:05 +00:00
Installations: r.Installations(),
Invitations: r.Invitations,
CommunityChanges: r.CommunityChanges,
RequestsToJoinCommunity: r.RequestsToJoinCommunity(),
Mailservers: r.Mailservers,
CommunityStorenodes: r.CommunityStorenodes,
Bookmarks: r.Bookmarks,
CurrentStatus: r.currentStatus,
Settings: r.Settings,
IdentityImages: r.IdentityImages,
CustomizationColor: r.CustomizationColor,
WatchOnlyAccounts: r.WatchOnlyAccounts,
Keypairs: r.Keypairs,
AccountsPositions: r.AccountsPositions,
2023-12-04 10:18:05 +00:00
TokenPreferences: r.TokenPreferences,
CollectiblePreferences: r.CollectiblePreferences,
Messages: r.Messages(),
VerificationRequests: r.VerificationRequests(),
SavedAddresses: r.SavedAddresses(),
Notifications: r.Notifications(),
Chats: r.Chats(),
Communities: r.Communities(),
CommunitiesSettings: r.CommunitiesSettings(),
RemovedChats: r.RemovedChats(),
RemovedMessages: r.RemovedMessages(),
DeletedMessages: r.DeletedMessagesInChats(),
ClearedHistories: r.ClearedHistories(),
ActivityCenterNotifications: r.ActivityCenterNotifications(),
ActivityCenterState: r.ActivityCenterState(),
PinMessages: r.PinMessages(),
EmojiReactions: r.EmojiReactions(),
StatusUpdates: r.StatusUpdates(),
DiscordCategories: r.DiscordCategories,
DiscordChannels: r.DiscordChannels,
DiscordOldestMessageTimestamp: r.DiscordOldestMessageTimestamp,
EnsUsernameDetails: r.EnsUsernameDetails(),
UpdatedProfileShowcaseContactIDs: r.GetUpdatedProfileShowcaseContactIDs(),
SeenAndUnseenMessages: r.GetSeenAndUnseenMessages(),
2021-01-11 10:32:51 +00:00
}
responseItem.TrustStatus = r.TrustStatus()
2021-01-11 10:32:51 +00:00
return json.Marshal(responseItem)
}
2021-01-11 10:32:51 +00:00
func (r *MessengerResponse) Chats() []*Chat {
var chats []*Chat
for _, chat := range r.chats {
chats = append(chats, chat)
}
return chats
}
feat: fallback pairing seed (#5614) * feat(pairing)!: Add extra parameters and remove v2 compatibility This commit includes the following changes: I have added a flag to maintain 2.29 compatibility. Breaking change in connection string The local pairing code that was parsing the connection string had a few non-upgradable features: It was strictly checking the number of parameters, throwing an error if the number was different. This made it impossible to add parameters to it without breaking. It was strictly checking the version number. This made increasing the version number impossible as older client would just refuse to connect. The code has been changed so that: Two parameters have been added, installation-id and key-uid. Those are needed for the fallback flow. I have also removed version from the payload, since it wasn't used. This means that we don't support v1 anymore. V2 parsing is supported . Going forward there's a clear strategy on how to update the protocol (append parameters, don't change existing one). https://www.youtube.com/watch?v=oyLBGkS5ICk Is a must watch video for understanding the strategy Changed MessengerResponse to use internally a map of installations rather than an array (minor) Just moving towards maps as arrays tend to lead to subtle bugs. Moved pairing methods to messenger_pairing.go Just moved some methods Added 2 new methods for the fallback flow FinishPairingThroughSeedPhraseProcess https://github.com/status-im/status-go/pull/5567/files#diff-1ad620b07fa3bd5fbc96c9f459d88829938a162bf1aaf41c61dea6e38b488d54R29 EnableAndSyncInstallation https://github.com/status-im/status-go/pull/5567/files#diff-1ad620b07fa3bd5fbc96c9f459d88829938a162bf1aaf41c61dea6e38b488d54R18 Flow for clients Client A1 is logged in Client A2 is logged out Client A1 shows a QR code Client A2 scans a QR code If connection fails on A2, the user will be prompted to enter a seed phrase. If the generated account matches the key-uid from the QR code, A2 should call FinishPairingThroughSeedPhraseProcess with the installation id passed in the QR code. This will send installation information over waku. The user should be shown its own installation id and prompted to check the other device. Client A1 will receive new installation data through waku, if they are still on the qr code page, they should show a popup to the user showing the received installation id, and a way to Enable and Sync, which should call the EnableAndSyncInstallation endpoint. This should finish the fallback syncing flow. Current issues Currently I haven't tested that all the data is synced after finishing the flow. I see that the two devices are paired correctly, but for example the DisplayName is not changed on the receiving device. I haven't had time to look into it further. * test_: add more test for connection string parser * fix_: fix panic when parse old connection string * test_: add comments for TestMessengerPairAfterSeedPhrase * fix_: correct error description * feat_:rename FinishPairingThroughSeedPhraseProcess to EnableInstallationAndPair * fix_: delete leftover * fix_: add UniqueKey method * fix_: unify the response for InputConnectionStringForBootstrapping * fix_: remove fields installationID and keyUID in GethStatusBackend * fix_: rename messenger_pairing to messenger_pairing_and_syncing --------- Co-authored-by: Andrea Maria Piana <andrea.maria.piana@gmail.com>
2024-07-30 09:14:05 +00:00
func (r *MessengerResponse) Installations() []*multidevice.Installation {
var is []*multidevice.Installation
for _, i := range r.installations {
is = append(is, i)
}
return is
}
2021-01-11 10:32:51 +00:00
func (r *MessengerResponse) RemovedChats() []string {
var chats []string
for chatID := range r.removedChats {
chats = append(chats, chatID)
}
return chats
}
2021-08-20 14:26:13 +00:00
func (r *MessengerResponse) RemovedMessages() []*RemovedMessage {
var messages []*RemovedMessage
for messageID := range r.removedMessages {
2021-08-20 14:26:13 +00:00
messages = append(messages, r.removedMessages[messageID])
}
return messages
}
func (r *MessengerResponse) DeletedMessages() map[string]string {
return r.deletedMessages
}
func (r *MessengerResponse) DeletedMessagesInChats() map[string][]string {
deletedMessagesInChats := make(map[string][]string)
for messageID, chatID := range r.deletedMessages {
deletedMessagesInChats[chatID] = append(deletedMessagesInChats[chatID], messageID)
}
return deletedMessagesInChats
}
2022-02-10 10:00:59 +00:00
func (r *MessengerResponse) ClearedHistories() []*ClearedHistory {
var clearedHistories []*ClearedHistory
for chatID := range r.clearedHistories {
clearedHistories = append(clearedHistories, r.clearedHistories[chatID])
}
return clearedHistories
}
2021-01-11 10:32:51 +00:00
func (r *MessengerResponse) Communities() []*communities.Community {
var communities []*communities.Community
for _, c := range r.communities {
communities = append(communities, c)
}
return communities
}
func (r *MessengerResponse) CommunitiesSettings() []*communities.CommunitySettings {
var settings []*communities.CommunitySettings
for _, s := range r.communitiesSettings {
settings = append(settings, s)
}
return settings
}
func (r *MessengerResponse) Notifications() []*localnotifications.Notification {
var notifications []*localnotifications.Notification
for _, n := range r.notifications {
notifications = append(notifications, n)
}
return notifications
}
func (r *MessengerResponse) VerificationRequests() []*verification.Request {
var verifications []*verification.Request
for _, v := range r.verificationRequests {
verifications = append(verifications, v)
}
return verifications
}
func (r *MessengerResponse) PinMessages() []*common.PinMessage {
var pinMessages []*common.PinMessage
for _, pm := range r.pinMessages {
pinMessages = append(pinMessages, pm)
}
return pinMessages
}
func (r *MessengerResponse) TrustStatus() map[string]verification.TrustStatus {
if len(r.trustStatus) == 0 {
return nil
}
result := make(map[string]verification.TrustStatus)
for contactID, trustStatus := range r.trustStatus {
result[contactID] = trustStatus
}
return result
}
func (r *MessengerResponse) StatusUpdates() []UserStatus {
var userStatus []UserStatus
for pk, s := range r.statusUpdates {
s.PublicKey = pk
userStatus = append(userStatus, s)
}
return userStatus
}
2021-01-11 10:32:51 +00:00
func (r *MessengerResponse) IsEmpty() bool {
return len(r.chats)+
2021-06-03 13:11:55 +00:00
len(r.messages)+
len(r.pinMessages)+
2021-01-11 10:32:51 +00:00
len(r.Contacts)+
2022-01-17 03:42:11 +00:00
len(r.Bookmarks)+
2022-02-10 10:00:59 +00:00
len(r.clearedHistories)+
Sync Settings (#2478) * Sync Settings * Added valueHandlers and Database singleton Some issues remain, need a way to comparing incoming sql.DB to check if the connection is to a different file or not. Maybe make singleton instance per filename * Added functionality to check the sqlite filename * Refactor of Database.SaveSyncSettings to be used as a handler * Implemented inteface for setting sync protobuf factories * Refactored and completed adhoc send setting sync * Tidying up * Immutability refactor * Refactor settings into dedicated package * Breakout structs * Tidy up * Refactor of bulk settings sync * Bug fixes * Addressing feedback * Fix code dropped during rebase * Fix for db closed * Fix for node config related crashes * Provisional fix for type assertion - issue 2 * Adding robust type assertion checks * Partial fix for null literal db storage and json encoding * Fix for passively handling nil sql.DB, and checking if elem has len and if len is 0 * Added test for preferred name behaviour * Adding saved sync settings to MessengerResponse * Completed granular initial sync and clock from network on save * add Settings to isEmpty * Refactor of protobufs, partially done * Added syncSetting receiver handling, some bug fixes * Fix for sticker packs * Implement inactive flag on sync protobuf factory * Refactor of types and structs * Added SettingField.CanSync functionality * Addressing rebase artifact * Refactor of Setting SELECT queries * Refactor of string return queries * VERSION bump and migration index bump * Deactiveate Sync Settings * Deactiveated preferred_name and send_status_updates Co-authored-by: Andrea Maria Piana <andrea.maria.piana@gmail.com>
2022-03-23 18:47:00 +00:00
len(r.Settings)+
feat: fallback pairing seed (#5614) * feat(pairing)!: Add extra parameters and remove v2 compatibility This commit includes the following changes: I have added a flag to maintain 2.29 compatibility. Breaking change in connection string The local pairing code that was parsing the connection string had a few non-upgradable features: It was strictly checking the number of parameters, throwing an error if the number was different. This made it impossible to add parameters to it without breaking. It was strictly checking the version number. This made increasing the version number impossible as older client would just refuse to connect. The code has been changed so that: Two parameters have been added, installation-id and key-uid. Those are needed for the fallback flow. I have also removed version from the payload, since it wasn't used. This means that we don't support v1 anymore. V2 parsing is supported . Going forward there's a clear strategy on how to update the protocol (append parameters, don't change existing one). https://www.youtube.com/watch?v=oyLBGkS5ICk Is a must watch video for understanding the strategy Changed MessengerResponse to use internally a map of installations rather than an array (minor) Just moving towards maps as arrays tend to lead to subtle bugs. Moved pairing methods to messenger_pairing.go Just moved some methods Added 2 new methods for the fallback flow FinishPairingThroughSeedPhraseProcess https://github.com/status-im/status-go/pull/5567/files#diff-1ad620b07fa3bd5fbc96c9f459d88829938a162bf1aaf41c61dea6e38b488d54R29 EnableAndSyncInstallation https://github.com/status-im/status-go/pull/5567/files#diff-1ad620b07fa3bd5fbc96c9f459d88829938a162bf1aaf41c61dea6e38b488d54R18 Flow for clients Client A1 is logged in Client A2 is logged out Client A1 shows a QR code Client A2 scans a QR code If connection fails on A2, the user will be prompted to enter a seed phrase. If the generated account matches the key-uid from the QR code, A2 should call FinishPairingThroughSeedPhraseProcess with the installation id passed in the QR code. This will send installation information over waku. The user should be shown its own installation id and prompted to check the other device. Client A1 will receive new installation data through waku, if they are still on the qr code page, they should show a popup to the user showing the received installation id, and a way to Enable and Sync, which should call the EnableAndSyncInstallation endpoint. This should finish the fallback syncing flow. Current issues Currently I haven't tested that all the data is synced after finishing the flow. I see that the two devices are paired correctly, but for example the DisplayName is not changed on the receiving device. I haven't had time to look into it further. * test_: add more test for connection string parser * fix_: fix panic when parse old connection string * test_: add comments for TestMessengerPairAfterSeedPhrase * fix_: correct error description * feat_:rename FinishPairingThroughSeedPhraseProcess to EnableInstallationAndPair * fix_: delete leftover * fix_: add UniqueKey method * fix_: unify the response for InputConnectionStringForBootstrapping * fix_: remove fields installationID and keyUID in GethStatusBackend * fix_: rename messenger_pairing to messenger_pairing_and_syncing --------- Co-authored-by: Andrea Maria Piana <andrea.maria.piana@gmail.com>
2024-07-30 09:14:05 +00:00
len(r.installations)+
2021-01-11 10:32:51 +00:00
len(r.Invitations)+
len(r.emojiReactions)+
2021-01-11 10:32:51 +00:00
len(r.communities)+
len(r.CommunityChanges)+
len(r.removedChats)+
len(r.removedMessages)+
len(r.deletedMessages)+
2021-01-11 10:32:51 +00:00
len(r.Mailservers)+
len(r.CommunityStorenodes)+
2022-03-24 09:35:56 +00:00
len(r.IdentityImages)+
len(r.WatchOnlyAccounts)+
len(r.Keypairs)+
len(r.AccountsPositions)+
2023-12-04 10:18:05 +00:00
len(r.TokenPreferences)+
len(r.CollectiblePreferences)+
len(r.notifications)+
len(r.statusUpdates)+
len(r.activityCenterNotifications)+
len(r.trustStatus)+
len(r.verificationRequests)+
len(r.requestsToJoinCommunity)+
len(r.savedAddresses)+
len(r.updatedProfileShowcaseContactIDs)+
len(r.seenAndUnseenMessages)+
len(r.ensUsernameDetails) == 0 &&
r.currentStatus == nil &&
2023-06-05 11:10:26 +00:00
r.activityCenterState == nil &&
r.CustomizationColor == ""
}
2020-12-03 17:04:38 +00:00
// Merge takes another response and appends the new Chats & new Messages and replaces
// the existing Messages & Chats if they have the same ID
2021-01-11 10:32:51 +00:00
func (r *MessengerResponse) Merge(response *MessengerResponse) error {
if len(response.Invitations)+
2021-01-11 10:32:51 +00:00
len(response.Mailservers)+
2022-02-10 10:00:59 +00:00
len(response.clearedHistories)+
feat: introduce messenger APIs to extract discord channels As part of the new Discord <-> Status Community Import functionality, we're adding an API that extracts all discord categories and channels from a previously exported discord export file. These APIs can be used in clients to show the user what categories and channels will be imported later on. There are two APIs: 1. `Messenger.ExtractDiscordCategoriesAndChannels(filesToimport []string) (*MessengerResponse, map[string]*discord.ImportError)` This takes a list of exported discord export (JSON) files (typically one per channel), reads them, and extracts the categories and channels into dedicated data structures (`[]DiscordChannel` and `[]DiscordCategory`) It also returns the oldest message timestamp found in all extracted channels. The API is synchronous and returns the extracted data as a `*MessengerResponse`. This allows to make the API available status-go's RPC interface. The error case is a `map[string]*discord.ImportError` where each key is a file path of a JSON file that we tried to extract data from, and the value a `discord.ImportError` which holds an error message and an error code, allowing for distinguishing between "critical" errors and "non-critical" errors. 2. `Messenger.RequestExtractDiscordCategoriesAndChannels(filesToImport []string)` This is the asynchronous counterpart to `ExtractDiscordCategoriesAndChannels`. The reason this API has been added is because discord servers can have a lot of message and channel data, which causes `ExtractDiscordCategoriesAndChannels` to block the thread for too long, making apps potentially feel like they are stuck. This API runs inside a go routine, eventually calls `ExtractDiscordCategoriesAndChannels`, and then emits a newly introduced `DiscordCategoriesAndChannelsExtractedSignal` that clients can react to. Failure of extraction has to be determined by the `discord.ImportErrors` emitted by the signal. **A note about exported discord history files** We expect users to export their discord histories via the [DiscordChatExporter](https://github.com/Tyrrrz/DiscordChatExporter/wiki/GUI%2C-CLI-and-Formats-explained#exportguild) tool. The tool allows to export the data in different formats, such as JSON, HTML and CSV. We expect users to have their data exported as JSON. Closes: https://github.com/status-im/status-desktop/issues/6690
2022-07-13 09:33:53 +00:00
len(response.DiscordChannels)+
len(response.DiscordCategories) != 0 {
return ErrNotImplemented
}
2021-01-11 10:32:51 +00:00
r.AddChats(response.Chats())
r.AddRemovedChats(response.RemovedChats())
r.AddRemovedMessages(response.RemovedMessages())
r.MergeDeletedMessages(response.DeletedMessages())
r.AddNotifications(response.Notifications())
2021-06-03 13:11:55 +00:00
r.AddMessages(response.Messages())
r.AddContacts(response.Contacts)
2021-01-11 10:32:51 +00:00
r.AddCommunities(response.Communities())
r.UpdateCommunitySettings(response.CommunitiesSettings())
r.AddPinMessages(response.PinMessages())
r.AddVerificationRequests(response.VerificationRequests())
r.AddTrustStatuses(response.trustStatus)
r.AddActivityCenterNotifications(response.ActivityCenterNotifications())
r.SetActivityCenterState(response.ActivityCenterState())
r.AddEmojiReactions(response.EmojiReactions())
feat: fallback pairing seed (#5614) * feat(pairing)!: Add extra parameters and remove v2 compatibility This commit includes the following changes: I have added a flag to maintain 2.29 compatibility. Breaking change in connection string The local pairing code that was parsing the connection string had a few non-upgradable features: It was strictly checking the number of parameters, throwing an error if the number was different. This made it impossible to add parameters to it without breaking. It was strictly checking the version number. This made increasing the version number impossible as older client would just refuse to connect. The code has been changed so that: Two parameters have been added, installation-id and key-uid. Those are needed for the fallback flow. I have also removed version from the payload, since it wasn't used. This means that we don't support v1 anymore. V2 parsing is supported . Going forward there's a clear strategy on how to update the protocol (append parameters, don't change existing one). https://www.youtube.com/watch?v=oyLBGkS5ICk Is a must watch video for understanding the strategy Changed MessengerResponse to use internally a map of installations rather than an array (minor) Just moving towards maps as arrays tend to lead to subtle bugs. Moved pairing methods to messenger_pairing.go Just moved some methods Added 2 new methods for the fallback flow FinishPairingThroughSeedPhraseProcess https://github.com/status-im/status-go/pull/5567/files#diff-1ad620b07fa3bd5fbc96c9f459d88829938a162bf1aaf41c61dea6e38b488d54R29 EnableAndSyncInstallation https://github.com/status-im/status-go/pull/5567/files#diff-1ad620b07fa3bd5fbc96c9f459d88829938a162bf1aaf41c61dea6e38b488d54R18 Flow for clients Client A1 is logged in Client A2 is logged out Client A1 shows a QR code Client A2 scans a QR code If connection fails on A2, the user will be prompted to enter a seed phrase. If the generated account matches the key-uid from the QR code, A2 should call FinishPairingThroughSeedPhraseProcess with the installation id passed in the QR code. This will send installation information over waku. The user should be shown its own installation id and prompted to check the other device. Client A1 will receive new installation data through waku, if they are still on the qr code page, they should show a popup to the user showing the received installation id, and a way to Enable and Sync, which should call the EnableAndSyncInstallation endpoint. This should finish the fallback syncing flow. Current issues Currently I haven't tested that all the data is synced after finishing the flow. I see that the two devices are paired correctly, but for example the DisplayName is not changed on the receiving device. I haven't had time to look into it further. * test_: add more test for connection string parser * fix_: fix panic when parse old connection string * test_: add comments for TestMessengerPairAfterSeedPhrase * fix_: correct error description * feat_:rename FinishPairingThroughSeedPhraseProcess to EnableInstallationAndPair * fix_: delete leftover * fix_: add UniqueKey method * fix_: unify the response for InputConnectionStringForBootstrapping * fix_: remove fields installationID and keyUID in GethStatusBackend * fix_: rename messenger_pairing to messenger_pairing_and_syncing --------- Co-authored-by: Andrea Maria Piana <andrea.maria.piana@gmail.com>
2024-07-30 09:14:05 +00:00
r.AddInstallations(response.Installations())
r.AddSavedAddresses(response.SavedAddresses())
r.AddEnsUsernameDetails(response.EnsUsernameDetails())
r.AddRequestsToJoinCommunity(response.RequestsToJoinCommunity())
r.AddBookmarks(response.GetBookmarks())
r.AddSeveralUpdatedProfileShowcaseContactIDs(response.GetUpdatedProfileShowcaseContactIDs())
r.AddSeveralSeenAndUnseenMessages(response.GetSeenAndUnseenMessages())
r.CommunityChanges = append(r.CommunityChanges, response.CommunityChanges...)
r.BackupHandled = response.BackupHandled
r.CustomizationColor = response.CustomizationColor
r.WatchOnlyAccounts = append(r.WatchOnlyAccounts, response.WatchOnlyAccounts...)
r.Keypairs = append(r.Keypairs, response.Keypairs...)
2023-12-04 10:18:05 +00:00
r.AccountsPositions = append(r.AccountsPositions, response.AccountsPositions...)
r.TokenPreferences = append(r.TokenPreferences, response.TokenPreferences...)
r.CollectiblePreferences = append(r.CollectiblePreferences, response.CollectiblePreferences...)
2020-12-03 17:04:38 +00:00
return nil
}
2021-01-11 10:32:51 +00:00
func (r *MessengerResponse) AddCommunities(communities []*communities.Community) {
for _, overrideCommunity := range communities {
r.AddCommunity(overrideCommunity)
}
}
func (r *MessengerResponse) AddCommunity(c *communities.Community) {
if r.communities == nil {
r.communities = make(map[string]*communities.Community)
}
r.communities[c.IDString()] = c
}
func (r *MessengerResponse) AddCommunitySettings(c *communities.CommunitySettings) {
if r.communitiesSettings == nil {
r.communitiesSettings = make(map[string]*communities.CommunitySettings)
}
r.communitiesSettings[c.CommunityID] = c
}
func (r *MessengerResponse) UpdateCommunitySettings(communitySettings []*communities.CommunitySettings) {
for _, communitySetting := range communitySettings {
r.AddCommunitySettings(communitySetting)
}
}
func (r *MessengerResponse) AddRequestsToJoinCommunity(requestsToJoin []*communities.RequestToJoin) {
for _, rq := range requestsToJoin {
r.AddRequestToJoinCommunity(rq)
}
}
func (r *MessengerResponse) AddRequestToJoinCommunity(requestToJoin *communities.RequestToJoin) {
if r.requestsToJoinCommunity == nil {
r.requestsToJoinCommunity = make(map[string]*communities.RequestToJoin)
}
r.requestsToJoinCommunity[requestToJoin.ID.String()] = requestToJoin
}
func (r *MessengerResponse) RequestsToJoinCommunity() []*communities.RequestToJoin {
var rs []*communities.RequestToJoin
for _, r := range r.requestsToJoinCommunity {
rs = append(rs, r)
}
return rs
}
2022-12-29 06:16:19 +00:00
func (r *MessengerResponse) AddSetting(s *settings.SyncSettingField) {
r.Settings = append(r.Settings, s)
}
2022-01-17 03:42:11 +00:00
func (r *MessengerResponse) AddBookmark(bookmark *browsers.Bookmark) {
r.Bookmarks = append(r.Bookmarks, bookmark)
}
func (r *MessengerResponse) AddBookmarks(bookmarks []*browsers.Bookmark) {
for _, b := range bookmarks {
r.AddBookmark(b)
}
}
func (r *MessengerResponse) GetBookmarks() []*browsers.Bookmark {
return r.Bookmarks
}
func (r *MessengerResponse) AddVerificationRequest(vr *verification.Request) {
if r.verificationRequests == nil {
r.verificationRequests = make(map[string]*verification.Request)
}
r.verificationRequests[vr.ID] = vr
}
func (r *MessengerResponse) AddVerificationRequests(vrs []*verification.Request) {
for _, vr := range vrs {
r.AddVerificationRequest(vr)
}
}
func (r *MessengerResponse) AddTrustStatus(contactID string, trustStatus verification.TrustStatus) {
if r.trustStatus == nil {
r.trustStatus = make(map[string]verification.TrustStatus)
}
r.trustStatus[contactID] = trustStatus
}
func (r *MessengerResponse) AddTrustStatuses(ts map[string]verification.TrustStatus) {
if r.trustStatus == nil {
r.trustStatus = make(map[string]verification.TrustStatus)
}
for k, v := range ts {
r.trustStatus[k] = v
}
}
2021-01-11 10:32:51 +00:00
func (r *MessengerResponse) AddChat(c *Chat) {
if r.chats == nil {
r.chats = make(map[string]*Chat)
}
r.chats[c.ID] = c
}
func (r *MessengerResponse) AddChats(chats []*Chat) {
for _, c := range chats {
r.AddChat(c)
}
}
func (r *MessengerResponse) AddEmojiReactions(ers []*EmojiReaction) {
for _, e := range ers {
r.AddEmojiReaction(e)
}
}
func (r *MessengerResponse) AddEmojiReaction(er *EmojiReaction) {
if r.emojiReactions == nil {
r.emojiReactions = make(map[string]*EmojiReaction)
}
r.emojiReactions[er.ID()] = er
}
func (r *MessengerResponse) EmojiReactions() []*EmojiReaction {
var ers []*EmojiReaction
for _, er := range r.emojiReactions {
ers = append(ers, er)
}
return ers
}
func (r *MessengerResponse) AddSavedAddresses(ers []*wallet.SavedAddress) {
for _, e := range ers {
r.AddSavedAddress(e)
}
}
func (r *MessengerResponse) AddSavedAddress(er *wallet.SavedAddress) {
if r.savedAddresses == nil {
r.savedAddresses = make(map[string]*wallet.SavedAddress)
}
r.savedAddresses[er.ID()] = er
}
func (r *MessengerResponse) SavedAddresses() []*wallet.SavedAddress {
return maps.Values(r.savedAddresses)
}
func (r *MessengerResponse) AddEnsUsernameDetail(detail *ensservice.UsernameDetail) {
r.ensUsernameDetails = append(r.ensUsernameDetails, detail)
}
func (r *MessengerResponse) AddEnsUsernameDetails(details []*ensservice.UsernameDetail) {
r.ensUsernameDetails = append(r.ensUsernameDetails, details...)
}
func (r *MessengerResponse) EnsUsernameDetails() []*ensservice.UsernameDetail {
return r.ensUsernameDetails
}
func (r *MessengerResponse) AddNotification(n *localnotifications.Notification) {
if r.notifications == nil {
r.notifications = make(map[string]*localnotifications.Notification)
}
r.notifications[n.ID.String()] = n
}
func (r *MessengerResponse) ClearNotifications() {
r.notifications = nil
}
func (r *MessengerResponse) AddNotifications(notifications []*localnotifications.Notification) {
for _, c := range notifications {
r.AddNotification(c)
}
}
2021-01-11 10:32:51 +00:00
func (r *MessengerResponse) AddRemovedChats(chats []string) {
for _, c := range chats {
r.AddRemovedChat(c)
}
}
func (r *MessengerResponse) AddRemovedChat(chatID string) {
if r.removedChats == nil {
r.removedChats = make(map[string]bool)
}
r.removedChats[chatID] = true
}
2021-08-20 14:26:13 +00:00
func (r *MessengerResponse) AddRemovedMessages(messages []*RemovedMessage) {
for _, m := range messages {
r.AddRemovedMessage(m)
}
}
2021-08-20 14:26:13 +00:00
func (r *MessengerResponse) AddRemovedMessage(rm *RemovedMessage) {
if r.removedMessages == nil {
2021-08-20 14:26:13 +00:00
r.removedMessages = make(map[string]*RemovedMessage)
}
2021-08-20 14:26:13 +00:00
r.removedMessages[rm.MessageID] = rm
2021-09-08 09:55:06 +00:00
// Remove original message from the map
if len(r.messages) != 0 && r.messages[rm.MessageID] != nil {
delete(r.messages, rm.MessageID)
}
}
func (r *MessengerResponse) AddDeletedMessages(messagesToAdd []*protobuf.DeleteCommunityMemberMessage) {
if r.deletedMessages == nil {
r.deletedMessages = make(map[string]string)
}
for _, message := range messagesToAdd {
r.deletedMessages[message.Id] = message.ChatId
}
}
func (r *MessengerResponse) MergeDeletedMessages(messagesToAdd map[string]string) {
if r.deletedMessages == nil {
r.deletedMessages = make(map[string]string)
}
for messageID, chatID := range messagesToAdd {
r.deletedMessages[messageID] = chatID
}
}
2022-02-10 10:00:59 +00:00
func (r *MessengerResponse) AddClearedHistory(ch *ClearedHistory) {
if r.clearedHistories == nil {
r.clearedHistories = make(map[string]*ClearedHistory)
}
existingClearedHistory, ok := r.clearedHistories[ch.ChatID]
if !ok || existingClearedHistory.ClearedAt < ch.ClearedAt {
r.clearedHistories[ch.ChatID] = ch
}
}
func (r *MessengerResponse) AddActivityCenterNotifications(ns []*ActivityCenterNotification) {
for _, n := range ns {
r.AddActivityCenterNotification(n)
}
}
func (r *MessengerResponse) AddActivityCenterNotification(n *ActivityCenterNotification) {
if r.activityCenterNotifications == nil {
r.activityCenterNotifications = make(map[string]*ActivityCenterNotification)
}
r.activityCenterNotifications[n.ID.String()] = n
}
func (r *MessengerResponse) RemoveActivityCenterNotification(id string) bool {
if r.activityCenterNotifications == nil {
return false
}
if _, ok := r.activityCenterNotifications[id]; ok {
delete(r.activityCenterNotifications, id)
return true
}
return false
}
func (r *MessengerResponse) ActivityCenterNotifications() []*ActivityCenterNotification {
var ns []*ActivityCenterNotification
for _, n := range r.activityCenterNotifications {
ns = append(ns, n)
}
return ns
}
func (r *MessengerResponse) SetActivityCenterState(activityCenterState *ActivityCenterState) {
r.activityCenterState = activityCenterState
}
func (r *MessengerResponse) ActivityCenterState() *ActivityCenterState {
return r.activityCenterState
}
func (r *MessengerResponse) AddPinMessage(pm *common.PinMessage) {
if r.pinMessages == nil {
r.pinMessages = make(map[string]*common.PinMessage)
}
r.pinMessages[pm.ID] = pm
}
func (r *MessengerResponse) AddPinMessages(pms []*common.PinMessage) {
for _, pm := range pms {
r.AddPinMessage(pm)
}
}
2021-03-25 15:15:22 +00:00
func (r *MessengerResponse) SetCurrentStatus(status UserStatus) {
r.currentStatus = &status
}
func (r *MessengerResponse) AddStatusUpdate(upd UserStatus) {
if r.statusUpdates == nil {
r.statusUpdates = make(map[string]UserStatus)
}
r.statusUpdates[upd.PublicKey] = upd
}
2022-09-29 11:50:23 +00:00
func (r *MessengerResponse) DiscordMessages() []*protobuf.DiscordMessage {
var ms []*protobuf.DiscordMessage
for _, m := range r.discordMessages {
ms = append(ms, m)
}
return ms
}
func (r *MessengerResponse) DiscordMessageAuthors() []*protobuf.DiscordMessageAuthor {
var authors []*protobuf.DiscordMessageAuthor
for _, author := range r.discordMessageAuthors {
authors = append(authors, author)
}
return authors
}
func (r *MessengerResponse) DiscordMessageAttachments() []*protobuf.DiscordMessageAttachment {
var attachments []*protobuf.DiscordMessageAttachment
for _, a := range r.discordMessageAttachments {
attachments = append(attachments, a)
}
return attachments
}
// Messages extracts the messages from the response and returns them as a slice.
// Since 'r.messages' is a map, the order of messages in the resulting slice is not
// guaranteed and can vary with each call to this method. This is inherent to Go's map
// iteration behavior, which does not define a sequence for the order of map elements.
// Consumers should not depend on the ordering of messages in the slice for any logic
// that requires consistent ordering, as map iteration order can change when keys are
// added or deleted. Consider sorting the slice after retrieval if a specific order is needed.
2021-06-03 13:11:55 +00:00
func (r *MessengerResponse) Messages() []*common.Message {
var ms []*common.Message
for _, m := range r.messages {
ms = append(ms, m)
}
return ms
}
2022-09-29 11:50:23 +00:00
func (r *MessengerResponse) AddDiscordMessageAuthor(author *protobuf.DiscordMessageAuthor) {
if r.discordMessageAuthors == nil {
r.discordMessageAuthors = make(map[string]*protobuf.DiscordMessageAuthor)
}
r.discordMessageAuthors[author.Id] = author
}
func (r *MessengerResponse) AddDiscordMessageAttachment(attachment *protobuf.DiscordMessageAttachment) {
if r.discordMessageAttachments == nil {
r.discordMessageAttachments = make(map[string]*protobuf.DiscordMessageAttachment)
}
r.discordMessageAttachments[attachment.Id] = attachment
}
func (r *MessengerResponse) AddDiscordMessageAttachments(attachments []*protobuf.DiscordMessageAttachment) {
for _, attachment := range attachments {
r.AddDiscordMessageAttachment(attachment)
}
}
func (r *MessengerResponse) AddDiscordMessage(message *protobuf.DiscordMessage) {
if r.discordMessages == nil {
r.discordMessages = make(map[string]*protobuf.DiscordMessage)
}
r.discordMessages[message.Id] = message
}
2021-06-03 13:11:55 +00:00
func (r *MessengerResponse) AddMessages(ms []*common.Message) {
for _, m := range ms {
r.AddMessage(m)
}
}
2021-03-25 15:15:22 +00:00
func (r *MessengerResponse) AddMessage(message *common.Message) {
2021-06-03 13:11:55 +00:00
if r.messages == nil {
r.messages = make(map[string]*common.Message)
}
if message.Deleted {
return
}
2021-06-03 13:11:55 +00:00
r.messages[message.ID] = message
}
func (r *MessengerResponse) AddContact(c *Contact) {
for idx, c1 := range r.Contacts {
if c1.ID == c.ID {
r.Contacts[idx] = c
return
}
}
r.Contacts = append(r.Contacts, c)
}
2023-02-10 12:20:06 +00:00
func (r *MessengerResponse) AddContacts(contacts []*Contact) {
for idx := range contacts {
r.AddContact(contacts[idx])
}
}
func (r *MessengerResponse) AddInstallation(i *multidevice.Installation) {
feat: fallback pairing seed (#5614) * feat(pairing)!: Add extra parameters and remove v2 compatibility This commit includes the following changes: I have added a flag to maintain 2.29 compatibility. Breaking change in connection string The local pairing code that was parsing the connection string had a few non-upgradable features: It was strictly checking the number of parameters, throwing an error if the number was different. This made it impossible to add parameters to it without breaking. It was strictly checking the version number. This made increasing the version number impossible as older client would just refuse to connect. The code has been changed so that: Two parameters have been added, installation-id and key-uid. Those are needed for the fallback flow. I have also removed version from the payload, since it wasn't used. This means that we don't support v1 anymore. V2 parsing is supported . Going forward there's a clear strategy on how to update the protocol (append parameters, don't change existing one). https://www.youtube.com/watch?v=oyLBGkS5ICk Is a must watch video for understanding the strategy Changed MessengerResponse to use internally a map of installations rather than an array (minor) Just moving towards maps as arrays tend to lead to subtle bugs. Moved pairing methods to messenger_pairing.go Just moved some methods Added 2 new methods for the fallback flow FinishPairingThroughSeedPhraseProcess https://github.com/status-im/status-go/pull/5567/files#diff-1ad620b07fa3bd5fbc96c9f459d88829938a162bf1aaf41c61dea6e38b488d54R29 EnableAndSyncInstallation https://github.com/status-im/status-go/pull/5567/files#diff-1ad620b07fa3bd5fbc96c9f459d88829938a162bf1aaf41c61dea6e38b488d54R18 Flow for clients Client A1 is logged in Client A2 is logged out Client A1 shows a QR code Client A2 scans a QR code If connection fails on A2, the user will be prompted to enter a seed phrase. If the generated account matches the key-uid from the QR code, A2 should call FinishPairingThroughSeedPhraseProcess with the installation id passed in the QR code. This will send installation information over waku. The user should be shown its own installation id and prompted to check the other device. Client A1 will receive new installation data through waku, if they are still on the qr code page, they should show a popup to the user showing the received installation id, and a way to Enable and Sync, which should call the EnableAndSyncInstallation endpoint. This should finish the fallback syncing flow. Current issues Currently I haven't tested that all the data is synced after finishing the flow. I see that the two devices are paired correctly, but for example the DisplayName is not changed on the receiving device. I haven't had time to look into it further. * test_: add more test for connection string parser * fix_: fix panic when parse old connection string * test_: add comments for TestMessengerPairAfterSeedPhrase * fix_: correct error description * feat_:rename FinishPairingThroughSeedPhraseProcess to EnableInstallationAndPair * fix_: delete leftover * fix_: add UniqueKey method * fix_: unify the response for InputConnectionStringForBootstrapping * fix_: remove fields installationID and keyUID in GethStatusBackend * fix_: rename messenger_pairing to messenger_pairing_and_syncing --------- Co-authored-by: Andrea Maria Piana <andrea.maria.piana@gmail.com>
2024-07-30 09:14:05 +00:00
if len(r.installations) == 0 {
r.installations = make(map[string]*multidevice.Installation)
}
feat: fallback pairing seed (#5614) * feat(pairing)!: Add extra parameters and remove v2 compatibility This commit includes the following changes: I have added a flag to maintain 2.29 compatibility. Breaking change in connection string The local pairing code that was parsing the connection string had a few non-upgradable features: It was strictly checking the number of parameters, throwing an error if the number was different. This made it impossible to add parameters to it without breaking. It was strictly checking the version number. This made increasing the version number impossible as older client would just refuse to connect. The code has been changed so that: Two parameters have been added, installation-id and key-uid. Those are needed for the fallback flow. I have also removed version from the payload, since it wasn't used. This means that we don't support v1 anymore. V2 parsing is supported . Going forward there's a clear strategy on how to update the protocol (append parameters, don't change existing one). https://www.youtube.com/watch?v=oyLBGkS5ICk Is a must watch video for understanding the strategy Changed MessengerResponse to use internally a map of installations rather than an array (minor) Just moving towards maps as arrays tend to lead to subtle bugs. Moved pairing methods to messenger_pairing.go Just moved some methods Added 2 new methods for the fallback flow FinishPairingThroughSeedPhraseProcess https://github.com/status-im/status-go/pull/5567/files#diff-1ad620b07fa3bd5fbc96c9f459d88829938a162bf1aaf41c61dea6e38b488d54R29 EnableAndSyncInstallation https://github.com/status-im/status-go/pull/5567/files#diff-1ad620b07fa3bd5fbc96c9f459d88829938a162bf1aaf41c61dea6e38b488d54R18 Flow for clients Client A1 is logged in Client A2 is logged out Client A1 shows a QR code Client A2 scans a QR code If connection fails on A2, the user will be prompted to enter a seed phrase. If the generated account matches the key-uid from the QR code, A2 should call FinishPairingThroughSeedPhraseProcess with the installation id passed in the QR code. This will send installation information over waku. The user should be shown its own installation id and prompted to check the other device. Client A1 will receive new installation data through waku, if they are still on the qr code page, they should show a popup to the user showing the received installation id, and a way to Enable and Sync, which should call the EnableAndSyncInstallation endpoint. This should finish the fallback syncing flow. Current issues Currently I haven't tested that all the data is synced after finishing the flow. I see that the two devices are paired correctly, but for example the DisplayName is not changed on the receiving device. I haven't had time to look into it further. * test_: add more test for connection string parser * fix_: fix panic when parse old connection string * test_: add comments for TestMessengerPairAfterSeedPhrase * fix_: correct error description * feat_:rename FinishPairingThroughSeedPhraseProcess to EnableInstallationAndPair * fix_: delete leftover * fix_: add UniqueKey method * fix_: unify the response for InputConnectionStringForBootstrapping * fix_: remove fields installationID and keyUID in GethStatusBackend * fix_: rename messenger_pairing to messenger_pairing_and_syncing --------- Co-authored-by: Andrea Maria Piana <andrea.maria.piana@gmail.com>
2024-07-30 09:14:05 +00:00
r.installations[i.UniqueKey()] = i
}
func (r *MessengerResponse) AddInstallations(installations []*multidevice.Installation) {
for idx := range installations {
r.AddInstallation(installations[idx])
}
}
2021-06-03 13:11:55 +00:00
func (r *MessengerResponse) SetMessages(messages []*common.Message) {
r.messages = make(map[string]*common.Message)
r.AddMessages(messages)
2021-03-25 15:15:22 +00:00
}
2021-06-03 14:09:30 +00:00
func (r *MessengerResponse) GetMessage(messageID string) *common.Message {
if r.messages == nil {
return nil
}
return r.messages[messageID]
}
feat: introduce messenger APIs to extract discord channels As part of the new Discord <-> Status Community Import functionality, we're adding an API that extracts all discord categories and channels from a previously exported discord export file. These APIs can be used in clients to show the user what categories and channels will be imported later on. There are two APIs: 1. `Messenger.ExtractDiscordCategoriesAndChannels(filesToimport []string) (*MessengerResponse, map[string]*discord.ImportError)` This takes a list of exported discord export (JSON) files (typically one per channel), reads them, and extracts the categories and channels into dedicated data structures (`[]DiscordChannel` and `[]DiscordCategory`) It also returns the oldest message timestamp found in all extracted channels. The API is synchronous and returns the extracted data as a `*MessengerResponse`. This allows to make the API available status-go's RPC interface. The error case is a `map[string]*discord.ImportError` where each key is a file path of a JSON file that we tried to extract data from, and the value a `discord.ImportError` which holds an error message and an error code, allowing for distinguishing between "critical" errors and "non-critical" errors. 2. `Messenger.RequestExtractDiscordCategoriesAndChannels(filesToImport []string)` This is the asynchronous counterpart to `ExtractDiscordCategoriesAndChannels`. The reason this API has been added is because discord servers can have a lot of message and channel data, which causes `ExtractDiscordCategoriesAndChannels` to block the thread for too long, making apps potentially feel like they are stuck. This API runs inside a go routine, eventually calls `ExtractDiscordCategoriesAndChannels`, and then emits a newly introduced `DiscordCategoriesAndChannelsExtractedSignal` that clients can react to. Failure of extraction has to be determined by the `discord.ImportErrors` emitted by the signal. **A note about exported discord history files** We expect users to export their discord histories via the [DiscordChatExporter](https://github.com/Tyrrrz/DiscordChatExporter/wiki/GUI%2C-CLI-and-Formats-explained#exportguild) tool. The tool allows to export the data in different formats, such as JSON, HTML and CSV. We expect users to have their data exported as JSON. Closes: https://github.com/status-im/status-desktop/issues/6690
2022-07-13 09:33:53 +00:00
func (r *MessengerResponse) AddDiscordCategory(dc *discord.Category) {
for idx, c := range r.DiscordCategories {
if dc.ID == c.ID {
r.DiscordCategories[idx] = dc
return
}
}
r.DiscordCategories = append(r.DiscordCategories, dc)
}
func (r *MessengerResponse) AddDiscordChannel(dc *discord.Channel) {
for idx, c := range r.DiscordChannels {
if dc.ID == c.ID {
r.DiscordChannels[idx] = dc
return
}
}
r.DiscordChannels = append(r.DiscordChannels, dc)
}
func (r *MessengerResponse) HasDiscordCategory(id string) bool {
for _, c := range r.DiscordCategories {
if id == c.ID {
return true
}
}
return false
}
func (r *MessengerResponse) HasDiscordChannel(id string) bool {
for _, c := range r.DiscordChannels {
if id == c.ID {
return true
}
}
return false
}
func (r *MessengerResponse) AddSeveralUpdatedProfileShowcaseContactIDs(contactIDs []string) {
for _, contactID := range contactIDs {
r.AddUpdatedProfileShowcaseContactID(contactID)
}
}
func (r *MessengerResponse) AddUpdatedProfileShowcaseContactID(contactID string) {
if r.updatedProfileShowcaseContactIDs == nil {
r.updatedProfileShowcaseContactIDs = make(map[string]bool)
}
if _, exists := r.updatedProfileShowcaseContactIDs[contactID]; exists {
return
}
r.updatedProfileShowcaseContactIDs[contactID] = true
}
func (r *MessengerResponse) GetUpdatedProfileShowcaseContactIDs() []string {
var contactIDs []string
for contactID := range r.updatedProfileShowcaseContactIDs {
contactIDs = append(contactIDs, contactID)
}
return contactIDs
}
func (r *MessengerResponse) AddSeveralSeenAndUnseenMessages(messages []*SeenUnseenMessages) {
for _, message := range messages {
r.AddSeenAndUnseenMessages(message)
}
}
func (r *MessengerResponse) AddSeenAndUnseenMessages(message *SeenUnseenMessages) {
if r.seenAndUnseenMessages == nil {
r.seenAndUnseenMessages = make(map[string]*SeenUnseenMessages)
}
r.seenAndUnseenMessages[message.ChatID] = message
}
func (r *MessengerResponse) GetSeenAndUnseenMessages() []*SeenUnseenMessages {
var messages []*SeenUnseenMessages
for _, message := range r.seenAndUnseenMessages {
messages = append(messages, message)
}
return messages
}