status-go/protocol/messenger_response.go

708 lines
22 KiB
Go
Raw Normal View History

package protocol
import (
2021-01-11 10:32:51 +00:00
"encoding/json"
2022-01-17 03:42:11 +00:00
"github.com/status-im/status-go/services/browsers"
"github.com/status-im/status-go/services/wallet"
2022-01-17 03:42:11 +00:00
Anon Metrics Broadcast (#2198) * Protobufs and adapters * Added basic anon metric service and config init * Added fibonacci interval incrementer * Added basic Client.Start func and integrated interval incrementer * Added new processed field to app metrics table * Added id column to app metrics table * Added migration clean up * Added appmetrics GetUnprocessed and SetToProcessedByIDs and tests There was a wierd bug where metrics in the db that did not explicitly insert a value would be NULL, so could not be found by . In addition I've added a new primary id field to the app_metrics table so that updates could be done against very specific metric rows. * Updated adaptors and db to handle proto_id I need a way to distinguish individual metric items from each other so that I can ignore the ones that have been seen before. * Moved incrementer into dedicated file * Resolve incrementer test fail * Finalised the main loop functionality * Implemented delete loop framework * Updated adaptors file name * Added delete loop delay and quit, and tweak on RawMessage gen * Completed delete loop logic * Added DBLock to prevent deletion during mainLoop * Added postgres DB connection, integrated into anonmetrics.Server * Removed proto_id from SQL migration and model * Integrated postgres with Server and updated adaptors * Function name update * Added sample config files for client and server * Fixes and testing for low level e2e * make generate * Fix lint * Fix for receiving an anonMetricBatch not in server mode * Postgres test fixes * Tidy up, make vendor and make generate * delinting * Fixing database tests * Attempted fix of does: cannot open `does' (No such file or directory) not: cannot open `not' (No such file or directory) exist: cannot open `exist' (No such file or directory) error on sql resource loas * Moved all anon metric postgres migration logic and sources into a the protocol/anonmetrics package or sub packages. I don't know if this will fix the does: cannot open `does' (No such file or directory) not: cannot open `not' (No such file or directory) exist: cannot open `exist' (No such file or directory) error that happens in Jenkins but this could work * Lint for the lint god * Why doesn't the linter list all its problems at once? * test tweaks * Fix for wakuV2 change * DB reset change * Fix for postgres db migrations fails * More robust implementation of postgres test setup and teardown * Added block for anon metrics functionality * Version Bump to 0.84.0 * Added test to check anon metrics broadcast is deactivated * Protobufs and adapters * Added basic anon metric service and config init * Added new processed field to app metrics table * Added id column to app metrics table * Added migration clean up * Added appmetrics GetUnprocessed and SetToProcessedByIDs and tests There was a wierd bug where metrics in the db that did not explicitly insert a value would be NULL, so could not be found by . In addition I've added a new primary id field to the app_metrics table so that updates could be done against very specific metric rows. * Updated adaptors and db to handle proto_id I need a way to distinguish individual metric items from each other so that I can ignore the ones that have been seen before. * Added postgres DB connection, integrated into anonmetrics.Server * Removed proto_id from SQL migration and model * Integrated postgres with Server and updated adaptors * Added sample config files for client and server * Fix lint * Fix for receiving an anonMetricBatch not in server mode * Postgres test fixes * Tidy up, make vendor and make generate * Moved all anon metric postgres migration logic and sources into a the protocol/anonmetrics package or sub packages. I don't know if this will fix the does: cannot open `does' (No such file or directory) not: cannot open `not' (No such file or directory) exist: cannot open `exist' (No such file or directory) error that happens in Jenkins but this could work
2021-09-01 12:02:18 +00:00
"github.com/status-im/status-go/appmetrics"
2022-03-24 09:35:56 +00:00
"github.com/status-im/status-go/images"
2022-05-18 10:42:51 +00:00
"github.com/status-im/status-go/multiaccounts/accounts"
Sync Settings (#2478) * Sync Settings * Added valueHandlers and Database singleton Some issues remain, need a way to comparing incoming sql.DB to check if the connection is to a different file or not. Maybe make singleton instance per filename * Added functionality to check the sqlite filename * Refactor of Database.SaveSyncSettings to be used as a handler * Implemented inteface for setting sync protobuf factories * Refactored and completed adhoc send setting sync * Tidying up * Immutability refactor * Refactor settings into dedicated package * Breakout structs * Tidy up * Refactor of bulk settings sync * Bug fixes * Addressing feedback * Fix code dropped during rebase * Fix for db closed * Fix for node config related crashes * Provisional fix for type assertion - issue 2 * Adding robust type assertion checks * Partial fix for null literal db storage and json encoding * Fix for passively handling nil sql.DB, and checking if elem has len and if len is 0 * Added test for preferred name behaviour * Adding saved sync settings to MessengerResponse * Completed granular initial sync and clock from network on save * add Settings to isEmpty * Refactor of protobufs, partially done * Added syncSetting receiver handling, some bug fixes * Fix for sticker packs * Implement inactive flag on sync protobuf factory * Refactor of types and structs * Added SettingField.CanSync functionality * Addressing rebase artifact * Refactor of Setting SELECT queries * Refactor of string return queries * VERSION bump and migration index bump * Deactiveate Sync Settings * Deactiveated preferred_name and send_status_updates Co-authored-by: Andrea Maria Piana <andrea.maria.piana@gmail.com>
2022-03-23 18:47:00 +00:00
"github.com/status-im/status-go/multiaccounts/settings"
"github.com/status-im/status-go/protocol/common"
"github.com/status-im/status-go/protocol/communities"
feat: introduce messenger APIs to extract discord channels As part of the new Discord <-> Status Community Import functionality, we're adding an API that extracts all discord categories and channels from a previously exported discord export file. These APIs can be used in clients to show the user what categories and channels will be imported later on. There are two APIs: 1. `Messenger.ExtractDiscordCategoriesAndChannels(filesToimport []string) (*MessengerResponse, map[string]*discord.ImportError)` This takes a list of exported discord export (JSON) files (typically one per channel), reads them, and extracts the categories and channels into dedicated data structures (`[]DiscordChannel` and `[]DiscordCategory`) It also returns the oldest message timestamp found in all extracted channels. The API is synchronous and returns the extracted data as a `*MessengerResponse`. This allows to make the API available status-go's RPC interface. The error case is a `map[string]*discord.ImportError` where each key is a file path of a JSON file that we tried to extract data from, and the value a `discord.ImportError` which holds an error message and an error code, allowing for distinguishing between "critical" errors and "non-critical" errors. 2. `Messenger.RequestExtractDiscordCategoriesAndChannels(filesToImport []string)` This is the asynchronous counterpart to `ExtractDiscordCategoriesAndChannels`. The reason this API has been added is because discord servers can have a lot of message and channel data, which causes `ExtractDiscordCategoriesAndChannels` to block the thread for too long, making apps potentially feel like they are stuck. This API runs inside a go routine, eventually calls `ExtractDiscordCategoriesAndChannels`, and then emits a newly introduced `DiscordCategoriesAndChannelsExtractedSignal` that clients can react to. Failure of extraction has to be determined by the `discord.ImportErrors` emitted by the signal. **A note about exported discord history files** We expect users to export their discord histories via the [DiscordChatExporter](https://github.com/Tyrrrz/DiscordChatExporter/wiki/GUI%2C-CLI-and-Formats-explained#exportguild) tool. The tool allows to export the data in different formats, such as JSON, HTML and CSV. We expect users to have their data exported as JSON. Closes: https://github.com/status-im/status-desktop/issues/6690
2022-07-13 09:33:53 +00:00
"github.com/status-im/status-go/protocol/discord"
"github.com/status-im/status-go/protocol/encryption/multidevice"
2022-09-29 11:50:23 +00:00
"github.com/status-im/status-go/protocol/protobuf"
"github.com/status-im/status-go/protocol/verification"
localnotifications "github.com/status-im/status-go/services/local-notifications"
"github.com/status-im/status-go/services/mailservers"
)
2021-08-20 14:26:13 +00:00
type RemovedMessage struct {
ChatID string `json:"chatId"`
MessageID string `json:"messageId"`
DeletedBy string `json:"deletedBy,omitempty"`
2021-08-20 14:26:13 +00:00
}
2022-02-10 10:00:59 +00:00
type ClearedHistory struct {
ChatID string `json:"chatId"`
ClearedAt uint64 `json:"clearedAt"`
}
type MessengerResponse struct {
feat: introduce messenger APIs to extract discord channels As part of the new Discord <-> Status Community Import functionality, we're adding an API that extracts all discord categories and channels from a previously exported discord export file. These APIs can be used in clients to show the user what categories and channels will be imported later on. There are two APIs: 1. `Messenger.ExtractDiscordCategoriesAndChannels(filesToimport []string) (*MessengerResponse, map[string]*discord.ImportError)` This takes a list of exported discord export (JSON) files (typically one per channel), reads them, and extracts the categories and channels into dedicated data structures (`[]DiscordChannel` and `[]DiscordCategory`) It also returns the oldest message timestamp found in all extracted channels. The API is synchronous and returns the extracted data as a `*MessengerResponse`. This allows to make the API available status-go's RPC interface. The error case is a `map[string]*discord.ImportError` where each key is a file path of a JSON file that we tried to extract data from, and the value a `discord.ImportError` which holds an error message and an error code, allowing for distinguishing between "critical" errors and "non-critical" errors. 2. `Messenger.RequestExtractDiscordCategoriesAndChannels(filesToImport []string)` This is the asynchronous counterpart to `ExtractDiscordCategoriesAndChannels`. The reason this API has been added is because discord servers can have a lot of message and channel data, which causes `ExtractDiscordCategoriesAndChannels` to block the thread for too long, making apps potentially feel like they are stuck. This API runs inside a go routine, eventually calls `ExtractDiscordCategoriesAndChannels`, and then emits a newly introduced `DiscordCategoriesAndChannelsExtractedSignal` that clients can react to. Failure of extraction has to be determined by the `discord.ImportErrors` emitted by the signal. **A note about exported discord history files** We expect users to export their discord histories via the [DiscordChatExporter](https://github.com/Tyrrrz/DiscordChatExporter/wiki/GUI%2C-CLI-and-Formats-explained#exportguild) tool. The tool allows to export the data in different formats, such as JSON, HTML and CSV. We expect users to have their data exported as JSON. Closes: https://github.com/status-im/status-desktop/issues/6690
2022-07-13 09:33:53 +00:00
Contacts []*Contact
Installations []*multidevice.Installation
Invitations []*GroupChatInvitation
CommunityChanges []*communities.CommunityChanges
RequestsToJoinCommunity []*communities.RequestToJoin
AnonymousMetrics []*appmetrics.AppMetric
Mailservers []mailservers.Mailserver
Bookmarks []*browsers.Bookmark
Settings []*settings.SyncSettingField
IdentityImages []images.IdentityImage
feat: introduce messenger APIs to extract discord channels As part of the new Discord <-> Status Community Import functionality, we're adding an API that extracts all discord categories and channels from a previously exported discord export file. These APIs can be used in clients to show the user what categories and channels will be imported later on. There are two APIs: 1. `Messenger.ExtractDiscordCategoriesAndChannels(filesToimport []string) (*MessengerResponse, map[string]*discord.ImportError)` This takes a list of exported discord export (JSON) files (typically one per channel), reads them, and extracts the categories and channels into dedicated data structures (`[]DiscordChannel` and `[]DiscordCategory`) It also returns the oldest message timestamp found in all extracted channels. The API is synchronous and returns the extracted data as a `*MessengerResponse`. This allows to make the API available status-go's RPC interface. The error case is a `map[string]*discord.ImportError` where each key is a file path of a JSON file that we tried to extract data from, and the value a `discord.ImportError` which holds an error message and an error code, allowing for distinguishing between "critical" errors and "non-critical" errors. 2. `Messenger.RequestExtractDiscordCategoriesAndChannels(filesToImport []string)` This is the asynchronous counterpart to `ExtractDiscordCategoriesAndChannels`. The reason this API has been added is because discord servers can have a lot of message and channel data, which causes `ExtractDiscordCategoriesAndChannels` to block the thread for too long, making apps potentially feel like they are stuck. This API runs inside a go routine, eventually calls `ExtractDiscordCategoriesAndChannels`, and then emits a newly introduced `DiscordCategoriesAndChannelsExtractedSignal` that clients can react to. Failure of extraction has to be determined by the `discord.ImportErrors` emitted by the signal. **A note about exported discord history files** We expect users to export their discord histories via the [DiscordChatExporter](https://github.com/Tyrrrz/DiscordChatExporter/wiki/GUI%2C-CLI-and-Formats-explained#exportguild) tool. The tool allows to export the data in different formats, such as JSON, HTML and CSV. We expect users to have their data exported as JSON. Closes: https://github.com/status-im/status-desktop/issues/6690
2022-07-13 09:33:53 +00:00
Accounts []*accounts.Account
DiscordCategories []*discord.Category
DiscordChannels []*discord.Channel
DiscordOldestMessageTimestamp int
2021-01-11 10:32:51 +00:00
// notifications a list of notifications derived from messenger events
// that are useful to notify the user about
notifications map[string]*localnotifications.Notification
chats map[string]*Chat
removedChats map[string]bool
2021-08-20 14:26:13 +00:00
removedMessages map[string]*RemovedMessage
communities map[string]*communities.Community
communitiesSettings map[string]*communities.CommunitySettings
activityCenterNotifications map[string]*ActivityCenterNotification
2021-06-03 13:11:55 +00:00
messages map[string]*common.Message
pinMessages map[string]*common.PinMessage
2022-09-29 11:50:23 +00:00
discordMessages map[string]*protobuf.DiscordMessage
discordMessageAttachments map[string]*protobuf.DiscordMessageAttachment
discordMessageAuthors map[string]*protobuf.DiscordMessageAuthor
currentStatus *UserStatus
statusUpdates map[string]UserStatus
2022-02-10 10:00:59 +00:00
clearedHistories map[string]*ClearedHistory
verificationRequests map[string]*verification.Request
trustStatus map[string]verification.TrustStatus
emojiReactions map[string]*EmojiReaction
savedAddresses map[string]*wallet.SavedAddress
2021-01-11 10:32:51 +00:00
}
func (r *MessengerResponse) MarshalJSON() ([]byte, error) {
responseItem := struct {
Chats []*Chat `json:"chats,omitempty"`
RemovedChats []string `json:"removedChats,omitempty"`
RemovedMessages []*RemovedMessage `json:"removedMessages,omitempty"`
Messages []*common.Message `json:"messages,omitempty"`
Contacts []*Contact `json:"contacts,omitempty"`
Installations []*multidevice.Installation `json:"installations,omitempty"`
PinMessages []*common.PinMessage `json:"pinMessages,omitempty"`
EmojiReactions []*EmojiReaction `json:"emojiReactions,omitempty"`
Invitations []*GroupChatInvitation `json:"invitations,omitempty"`
CommunityChanges []*communities.CommunityChanges `json:"communityChanges,omitempty"`
RequestsToJoinCommunity []*communities.RequestToJoin `json:"requestsToJoinCommunity,omitempty"`
Mailservers []mailservers.Mailserver `json:"mailservers,omitempty"`
Bookmarks []*browsers.Bookmark `json:"bookmarks,omitempty"`
ClearedHistories []*ClearedHistory `json:"clearedHistories,omitempty"`
VerificationRequests []*verification.Request `json:"verificationRequests,omitempty"`
TrustStatus map[string]verification.TrustStatus `json:"trustStatus,omitempty"`
// Notifications a list of notifications derived from messenger events
// that are useful to notify the user about
2022-09-29 11:50:23 +00:00
Notifications []*localnotifications.Notification `json:"notifications"`
Communities []*communities.Community `json:"communities,omitempty"`
CommunitiesSettings []*communities.CommunitySettings `json:"communitiesSettings,omitempty"`
ActivityCenterNotifications []*ActivityCenterNotification `json:"activityCenterNotifications,omitempty"`
CurrentStatus *UserStatus `json:"currentStatus,omitempty"`
StatusUpdates []UserStatus `json:"statusUpdates,omitempty"`
Settings []*settings.SyncSettingField `json:"settings,omitempty"`
IdentityImages []images.IdentityImage `json:"identityImages,omitempty"`
Accounts []*accounts.Account `json:"accounts,omitempty"`
DiscordCategories []*discord.Category `json:"discordCategories,omitempty"`
DiscordChannels []*discord.Channel `json:"discordChannels,omitempty"`
DiscordOldestMessageTimestamp int `json:"discordOldestMessageTimestamp"`
DiscordMessages []*protobuf.DiscordMessage `json:"discordMessages,omitempty"`
DiscordMessageAttachments []*protobuf.DiscordMessageAttachment `json:"discordMessageAtachments,omitempty"`
SavedAddresses []*wallet.SavedAddress `json:"savedAddresses,omitempty"`
2021-01-11 10:32:51 +00:00
}{
Contacts: r.Contacts,
Installations: r.Installations,
Invitations: r.Invitations,
CommunityChanges: r.CommunityChanges,
RequestsToJoinCommunity: r.RequestsToJoinCommunity,
Mailservers: r.Mailservers,
Bookmarks: r.Bookmarks,
CurrentStatus: r.currentStatus,
Settings: r.Settings,
IdentityImages: r.IdentityImages,
Accounts: r.Accounts,
feat: introduce messenger APIs to extract discord channels As part of the new Discord <-> Status Community Import functionality, we're adding an API that extracts all discord categories and channels from a previously exported discord export file. These APIs can be used in clients to show the user what categories and channels will be imported later on. There are two APIs: 1. `Messenger.ExtractDiscordCategoriesAndChannels(filesToimport []string) (*MessengerResponse, map[string]*discord.ImportError)` This takes a list of exported discord export (JSON) files (typically one per channel), reads them, and extracts the categories and channels into dedicated data structures (`[]DiscordChannel` and `[]DiscordCategory`) It also returns the oldest message timestamp found in all extracted channels. The API is synchronous and returns the extracted data as a `*MessengerResponse`. This allows to make the API available status-go's RPC interface. The error case is a `map[string]*discord.ImportError` where each key is a file path of a JSON file that we tried to extract data from, and the value a `discord.ImportError` which holds an error message and an error code, allowing for distinguishing between "critical" errors and "non-critical" errors. 2. `Messenger.RequestExtractDiscordCategoriesAndChannels(filesToImport []string)` This is the asynchronous counterpart to `ExtractDiscordCategoriesAndChannels`. The reason this API has been added is because discord servers can have a lot of message and channel data, which causes `ExtractDiscordCategoriesAndChannels` to block the thread for too long, making apps potentially feel like they are stuck. This API runs inside a go routine, eventually calls `ExtractDiscordCategoriesAndChannels`, and then emits a newly introduced `DiscordCategoriesAndChannelsExtractedSignal` that clients can react to. Failure of extraction has to be determined by the `discord.ImportErrors` emitted by the signal. **A note about exported discord history files** We expect users to export their discord histories via the [DiscordChatExporter](https://github.com/Tyrrrz/DiscordChatExporter/wiki/GUI%2C-CLI-and-Formats-explained#exportguild) tool. The tool allows to export the data in different formats, such as JSON, HTML and CSV. We expect users to have their data exported as JSON. Closes: https://github.com/status-im/status-desktop/issues/6690
2022-07-13 09:33:53 +00:00
Messages: r.Messages(),
VerificationRequests: r.VerificationRequests(),
SavedAddresses: r.SavedAddresses(),
feat: introduce messenger APIs to extract discord channels As part of the new Discord <-> Status Community Import functionality, we're adding an API that extracts all discord categories and channels from a previously exported discord export file. These APIs can be used in clients to show the user what categories and channels will be imported later on. There are two APIs: 1. `Messenger.ExtractDiscordCategoriesAndChannels(filesToimport []string) (*MessengerResponse, map[string]*discord.ImportError)` This takes a list of exported discord export (JSON) files (typically one per channel), reads them, and extracts the categories and channels into dedicated data structures (`[]DiscordChannel` and `[]DiscordCategory`) It also returns the oldest message timestamp found in all extracted channels. The API is synchronous and returns the extracted data as a `*MessengerResponse`. This allows to make the API available status-go's RPC interface. The error case is a `map[string]*discord.ImportError` where each key is a file path of a JSON file that we tried to extract data from, and the value a `discord.ImportError` which holds an error message and an error code, allowing for distinguishing between "critical" errors and "non-critical" errors. 2. `Messenger.RequestExtractDiscordCategoriesAndChannels(filesToImport []string)` This is the asynchronous counterpart to `ExtractDiscordCategoriesAndChannels`. The reason this API has been added is because discord servers can have a lot of message and channel data, which causes `ExtractDiscordCategoriesAndChannels` to block the thread for too long, making apps potentially feel like they are stuck. This API runs inside a go routine, eventually calls `ExtractDiscordCategoriesAndChannels`, and then emits a newly introduced `DiscordCategoriesAndChannelsExtractedSignal` that clients can react to. Failure of extraction has to be determined by the `discord.ImportErrors` emitted by the signal. **A note about exported discord history files** We expect users to export their discord histories via the [DiscordChatExporter](https://github.com/Tyrrrz/DiscordChatExporter/wiki/GUI%2C-CLI-and-Formats-explained#exportguild) tool. The tool allows to export the data in different formats, such as JSON, HTML and CSV. We expect users to have their data exported as JSON. Closes: https://github.com/status-im/status-desktop/issues/6690
2022-07-13 09:33:53 +00:00
Notifications: r.Notifications(),
Chats: r.Chats(),
Communities: r.Communities(),
CommunitiesSettings: r.CommunitiesSettings(),
RemovedChats: r.RemovedChats(),
RemovedMessages: r.RemovedMessages(),
ClearedHistories: r.ClearedHistories(),
ActivityCenterNotifications: r.ActivityCenterNotifications(),
PinMessages: r.PinMessages(),
EmojiReactions: r.EmojiReactions(),
feat: introduce messenger APIs to extract discord channels As part of the new Discord <-> Status Community Import functionality, we're adding an API that extracts all discord categories and channels from a previously exported discord export file. These APIs can be used in clients to show the user what categories and channels will be imported later on. There are two APIs: 1. `Messenger.ExtractDiscordCategoriesAndChannels(filesToimport []string) (*MessengerResponse, map[string]*discord.ImportError)` This takes a list of exported discord export (JSON) files (typically one per channel), reads them, and extracts the categories and channels into dedicated data structures (`[]DiscordChannel` and `[]DiscordCategory`) It also returns the oldest message timestamp found in all extracted channels. The API is synchronous and returns the extracted data as a `*MessengerResponse`. This allows to make the API available status-go's RPC interface. The error case is a `map[string]*discord.ImportError` where each key is a file path of a JSON file that we tried to extract data from, and the value a `discord.ImportError` which holds an error message and an error code, allowing for distinguishing between "critical" errors and "non-critical" errors. 2. `Messenger.RequestExtractDiscordCategoriesAndChannels(filesToImport []string)` This is the asynchronous counterpart to `ExtractDiscordCategoriesAndChannels`. The reason this API has been added is because discord servers can have a lot of message and channel data, which causes `ExtractDiscordCategoriesAndChannels` to block the thread for too long, making apps potentially feel like they are stuck. This API runs inside a go routine, eventually calls `ExtractDiscordCategoriesAndChannels`, and then emits a newly introduced `DiscordCategoriesAndChannelsExtractedSignal` that clients can react to. Failure of extraction has to be determined by the `discord.ImportErrors` emitted by the signal. **A note about exported discord history files** We expect users to export their discord histories via the [DiscordChatExporter](https://github.com/Tyrrrz/DiscordChatExporter/wiki/GUI%2C-CLI-and-Formats-explained#exportguild) tool. The tool allows to export the data in different formats, such as JSON, HTML and CSV. We expect users to have their data exported as JSON. Closes: https://github.com/status-im/status-desktop/issues/6690
2022-07-13 09:33:53 +00:00
StatusUpdates: r.StatusUpdates(),
DiscordCategories: r.DiscordCategories,
DiscordChannels: r.DiscordChannels,
DiscordOldestMessageTimestamp: r.DiscordOldestMessageTimestamp,
2021-01-11 10:32:51 +00:00
}
responseItem.TrustStatus = r.TrustStatus()
2021-01-11 10:32:51 +00:00
return json.Marshal(responseItem)
}
2021-01-11 10:32:51 +00:00
func (r *MessengerResponse) Chats() []*Chat {
var chats []*Chat
for _, chat := range r.chats {
chats = append(chats, chat)
}
return chats
}
func (r *MessengerResponse) RemovedChats() []string {
var chats []string
for chatID := range r.removedChats {
chats = append(chats, chatID)
}
return chats
}
2021-08-20 14:26:13 +00:00
func (r *MessengerResponse) RemovedMessages() []*RemovedMessage {
var messages []*RemovedMessage
for messageID := range r.removedMessages {
2021-08-20 14:26:13 +00:00
messages = append(messages, r.removedMessages[messageID])
}
return messages
}
2022-02-10 10:00:59 +00:00
func (r *MessengerResponse) ClearedHistories() []*ClearedHistory {
var clearedHistories []*ClearedHistory
for chatID := range r.clearedHistories {
clearedHistories = append(clearedHistories, r.clearedHistories[chatID])
}
return clearedHistories
}
2021-01-11 10:32:51 +00:00
func (r *MessengerResponse) Communities() []*communities.Community {
var communities []*communities.Community
for _, c := range r.communities {
communities = append(communities, c)
}
return communities
}
func (r *MessengerResponse) CommunitiesSettings() []*communities.CommunitySettings {
var settings []*communities.CommunitySettings
for _, s := range r.communitiesSettings {
settings = append(settings, s)
}
return settings
}
func (r *MessengerResponse) Notifications() []*localnotifications.Notification {
var notifications []*localnotifications.Notification
for _, n := range r.notifications {
notifications = append(notifications, n)
}
return notifications
}
func (r *MessengerResponse) VerificationRequests() []*verification.Request {
var verifications []*verification.Request
for _, v := range r.verificationRequests {
verifications = append(verifications, v)
}
return verifications
}
func (r *MessengerResponse) PinMessages() []*common.PinMessage {
var pinMessages []*common.PinMessage
for _, pm := range r.pinMessages {
pinMessages = append(pinMessages, pm)
}
return pinMessages
}
func (r *MessengerResponse) TrustStatus() map[string]verification.TrustStatus {
if len(r.trustStatus) == 0 {
return nil
}
result := make(map[string]verification.TrustStatus)
for contactID, trustStatus := range r.trustStatus {
result[contactID] = trustStatus
}
return result
}
func (r *MessengerResponse) StatusUpdates() []UserStatus {
var userStatus []UserStatus
for pk, s := range r.statusUpdates {
s.PublicKey = pk
userStatus = append(userStatus, s)
}
return userStatus
}
2021-01-11 10:32:51 +00:00
func (r *MessengerResponse) IsEmpty() bool {
return len(r.chats)+
2021-06-03 13:11:55 +00:00
len(r.messages)+
len(r.pinMessages)+
2021-01-11 10:32:51 +00:00
len(r.Contacts)+
2022-01-17 03:42:11 +00:00
len(r.Bookmarks)+
2022-02-10 10:00:59 +00:00
len(r.clearedHistories)+
Sync Settings (#2478) * Sync Settings * Added valueHandlers and Database singleton Some issues remain, need a way to comparing incoming sql.DB to check if the connection is to a different file or not. Maybe make singleton instance per filename * Added functionality to check the sqlite filename * Refactor of Database.SaveSyncSettings to be used as a handler * Implemented inteface for setting sync protobuf factories * Refactored and completed adhoc send setting sync * Tidying up * Immutability refactor * Refactor settings into dedicated package * Breakout structs * Tidy up * Refactor of bulk settings sync * Bug fixes * Addressing feedback * Fix code dropped during rebase * Fix for db closed * Fix for node config related crashes * Provisional fix for type assertion - issue 2 * Adding robust type assertion checks * Partial fix for null literal db storage and json encoding * Fix for passively handling nil sql.DB, and checking if elem has len and if len is 0 * Added test for preferred name behaviour * Adding saved sync settings to MessengerResponse * Completed granular initial sync and clock from network on save * add Settings to isEmpty * Refactor of protobufs, partially done * Added syncSetting receiver handling, some bug fixes * Fix for sticker packs * Implement inactive flag on sync protobuf factory * Refactor of types and structs * Added SettingField.CanSync functionality * Addressing rebase artifact * Refactor of Setting SELECT queries * Refactor of string return queries * VERSION bump and migration index bump * Deactiveate Sync Settings * Deactiveated preferred_name and send_status_updates Co-authored-by: Andrea Maria Piana <andrea.maria.piana@gmail.com>
2022-03-23 18:47:00 +00:00
len(r.Settings)+
2021-01-11 10:32:51 +00:00
len(r.Installations)+
len(r.Invitations)+
len(r.emojiReactions)+
2021-01-11 10:32:51 +00:00
len(r.communities)+
len(r.CommunityChanges)+
len(r.removedChats)+
len(r.removedMessages)+
2021-01-11 10:32:51 +00:00
len(r.Mailservers)+
2022-03-24 09:35:56 +00:00
len(r.IdentityImages)+
2022-05-18 10:42:51 +00:00
len(r.Accounts)+
len(r.notifications)+
len(r.statusUpdates)+
len(r.activityCenterNotifications)+
len(r.trustStatus)+
len(r.verificationRequests)+
len(r.RequestsToJoinCommunity) == 0 &&
len(r.savedAddresses) == 0 &&
r.currentStatus == nil
}
2020-12-03 17:04:38 +00:00
// Merge takes another response and appends the new Chats & new Messages and replaces
// the existing Messages & Chats if they have the same ID
2021-01-11 10:32:51 +00:00
func (r *MessengerResponse) Merge(response *MessengerResponse) error {
if len(response.Invitations)+
2021-01-11 10:32:51 +00:00
len(response.RequestsToJoinCommunity)+
len(response.Mailservers)+
2022-01-17 03:42:11 +00:00
len(response.Bookmarks)+
2022-02-10 10:00:59 +00:00
len(response.clearedHistories)+
feat: introduce messenger APIs to extract discord channels As part of the new Discord <-> Status Community Import functionality, we're adding an API that extracts all discord categories and channels from a previously exported discord export file. These APIs can be used in clients to show the user what categories and channels will be imported later on. There are two APIs: 1. `Messenger.ExtractDiscordCategoriesAndChannels(filesToimport []string) (*MessengerResponse, map[string]*discord.ImportError)` This takes a list of exported discord export (JSON) files (typically one per channel), reads them, and extracts the categories and channels into dedicated data structures (`[]DiscordChannel` and `[]DiscordCategory`) It also returns the oldest message timestamp found in all extracted channels. The API is synchronous and returns the extracted data as a `*MessengerResponse`. This allows to make the API available status-go's RPC interface. The error case is a `map[string]*discord.ImportError` where each key is a file path of a JSON file that we tried to extract data from, and the value a `discord.ImportError` which holds an error message and an error code, allowing for distinguishing between "critical" errors and "non-critical" errors. 2. `Messenger.RequestExtractDiscordCategoriesAndChannels(filesToImport []string)` This is the asynchronous counterpart to `ExtractDiscordCategoriesAndChannels`. The reason this API has been added is because discord servers can have a lot of message and channel data, which causes `ExtractDiscordCategoriesAndChannels` to block the thread for too long, making apps potentially feel like they are stuck. This API runs inside a go routine, eventually calls `ExtractDiscordCategoriesAndChannels`, and then emits a newly introduced `DiscordCategoriesAndChannelsExtractedSignal` that clients can react to. Failure of extraction has to be determined by the `discord.ImportErrors` emitted by the signal. **A note about exported discord history files** We expect users to export their discord histories via the [DiscordChatExporter](https://github.com/Tyrrrz/DiscordChatExporter/wiki/GUI%2C-CLI-and-Formats-explained#exportguild) tool. The tool allows to export the data in different formats, such as JSON, HTML and CSV. We expect users to have their data exported as JSON. Closes: https://github.com/status-im/status-desktop/issues/6690
2022-07-13 09:33:53 +00:00
len(response.DiscordChannels)+
len(response.DiscordCategories) != 0 {
return ErrNotImplemented
}
2021-01-11 10:32:51 +00:00
r.AddChats(response.Chats())
r.AddRemovedChats(response.RemovedChats())
r.AddRemovedMessages(response.RemovedMessages())
r.AddNotifications(response.Notifications())
2021-06-03 13:11:55 +00:00
r.AddMessages(response.Messages())
r.AddContacts(response.Contacts)
2021-01-11 10:32:51 +00:00
r.AddCommunities(response.Communities())
r.AddPinMessages(response.PinMessages())
r.AddVerificationRequests(response.VerificationRequests())
r.AddTrustStatuses(response.trustStatus)
r.AddActivityCenterNotifications(response.ActivityCenterNotifications())
r.AddEmojiReactions(response.EmojiReactions())
r.AddInstallations(response.Installations)
r.AddSavedAddresses(response.SavedAddresses())
r.CommunityChanges = append(r.CommunityChanges, response.CommunityChanges...)
2020-12-03 17:04:38 +00:00
return nil
}
2021-01-11 10:32:51 +00:00
func (r *MessengerResponse) AddCommunities(communities []*communities.Community) {
for _, overrideCommunity := range communities {
r.AddCommunity(overrideCommunity)
}
}
func (r *MessengerResponse) AddCommunity(c *communities.Community) {
if r.communities == nil {
r.communities = make(map[string]*communities.Community)
}
r.communities[c.IDString()] = c
}
func (r *MessengerResponse) AddCommunitySettings(c *communities.CommunitySettings) {
if r.communitiesSettings == nil {
r.communitiesSettings = make(map[string]*communities.CommunitySettings)
}
r.communitiesSettings[c.CommunityID] = c
}
2022-12-29 06:16:19 +00:00
func (r *MessengerResponse) AddSetting(s *settings.SyncSettingField) {
r.Settings = append(r.Settings, s)
}
2022-01-17 03:42:11 +00:00
func (r *MessengerResponse) AddBookmark(bookmark *browsers.Bookmark) {
r.Bookmarks = append(r.Bookmarks, bookmark)
}
func (r *MessengerResponse) AddBookmarks(bookmarks []*browsers.Bookmark) {
for _, b := range bookmarks {
r.AddBookmark(b)
}
}
func (r *MessengerResponse) AddVerificationRequest(vr *verification.Request) {
if r.verificationRequests == nil {
r.verificationRequests = make(map[string]*verification.Request)
}
r.verificationRequests[vr.ID] = vr
}
func (r *MessengerResponse) AddVerificationRequests(vrs []*verification.Request) {
for _, vr := range vrs {
r.AddVerificationRequest(vr)
}
}
func (r *MessengerResponse) AddTrustStatus(contactID string, trustStatus verification.TrustStatus) {
if r.trustStatus == nil {
r.trustStatus = make(map[string]verification.TrustStatus)
}
r.trustStatus[contactID] = trustStatus
}
func (r *MessengerResponse) AddTrustStatuses(ts map[string]verification.TrustStatus) {
if r.trustStatus == nil {
r.trustStatus = make(map[string]verification.TrustStatus)
}
for k, v := range ts {
r.trustStatus[k] = v
}
}
2021-01-11 10:32:51 +00:00
func (r *MessengerResponse) AddChat(c *Chat) {
if r.chats == nil {
r.chats = make(map[string]*Chat)
}
r.chats[c.ID] = c
}
func (r *MessengerResponse) AddChats(chats []*Chat) {
for _, c := range chats {
r.AddChat(c)
}
}
func (r *MessengerResponse) AddEmojiReactions(ers []*EmojiReaction) {
for _, e := range ers {
r.AddEmojiReaction(e)
}
}
func (r *MessengerResponse) AddEmojiReaction(er *EmojiReaction) {
if r.emojiReactions == nil {
r.emojiReactions = make(map[string]*EmojiReaction)
}
r.emojiReactions[er.ID()] = er
}
func (r *MessengerResponse) EmojiReactions() []*EmojiReaction {
var ers []*EmojiReaction
for _, er := range r.emojiReactions {
ers = append(ers, er)
}
return ers
}
func (r *MessengerResponse) AddSavedAddresses(ers []*wallet.SavedAddress) {
for _, e := range ers {
r.AddSavedAddress(e)
}
}
func (r *MessengerResponse) AddSavedAddress(er *wallet.SavedAddress) {
if r.savedAddresses == nil {
r.savedAddresses = make(map[string]*wallet.SavedAddress)
}
r.savedAddresses[er.ID()] = er
}
func (r *MessengerResponse) SavedAddresses() []*wallet.SavedAddress {
var ers []*wallet.SavedAddress
for _, er := range r.savedAddresses {
ers = append(ers, er)
}
return ers
}
func (r *MessengerResponse) AddNotification(n *localnotifications.Notification) {
if r.notifications == nil {
r.notifications = make(map[string]*localnotifications.Notification)
}
r.notifications[n.ID.String()] = n
}
func (r *MessengerResponse) ClearNotifications() {
r.notifications = nil
}
func (r *MessengerResponse) AddNotifications(notifications []*localnotifications.Notification) {
for _, c := range notifications {
r.AddNotification(c)
}
}
2021-01-11 10:32:51 +00:00
func (r *MessengerResponse) AddRemovedChats(chats []string) {
for _, c := range chats {
r.AddRemovedChat(c)
}
}
func (r *MessengerResponse) AddRemovedChat(chatID string) {
if r.removedChats == nil {
r.removedChats = make(map[string]bool)
}
r.removedChats[chatID] = true
}
2021-08-20 14:26:13 +00:00
func (r *MessengerResponse) AddRemovedMessages(messages []*RemovedMessage) {
for _, m := range messages {
r.AddRemovedMessage(m)
}
}
2021-08-20 14:26:13 +00:00
func (r *MessengerResponse) AddRemovedMessage(rm *RemovedMessage) {
if r.removedMessages == nil {
2021-08-20 14:26:13 +00:00
r.removedMessages = make(map[string]*RemovedMessage)
}
2021-08-20 14:26:13 +00:00
r.removedMessages[rm.MessageID] = rm
2021-09-08 09:55:06 +00:00
// Remove original message from the map
if len(r.messages) != 0 && r.messages[rm.MessageID] != nil {
delete(r.messages, rm.MessageID)
}
}
2022-02-10 10:00:59 +00:00
func (r *MessengerResponse) AddClearedHistory(ch *ClearedHistory) {
if r.clearedHistories == nil {
r.clearedHistories = make(map[string]*ClearedHistory)
}
existingClearedHistory, ok := r.clearedHistories[ch.ChatID]
if !ok || existingClearedHistory.ClearedAt < ch.ClearedAt {
r.clearedHistories[ch.ChatID] = ch
}
}
func (r *MessengerResponse) AddActivityCenterNotifications(ns []*ActivityCenterNotification) {
for _, n := range ns {
r.AddActivityCenterNotification(n)
}
}
func (r *MessengerResponse) AddActivityCenterNotification(n *ActivityCenterNotification) {
if r.activityCenterNotifications == nil {
r.activityCenterNotifications = make(map[string]*ActivityCenterNotification)
}
r.activityCenterNotifications[n.ID.String()] = n
}
func (r *MessengerResponse) RemoveActivityCenterNotification(id string) bool {
if r.activityCenterNotifications == nil {
return false
}
if _, ok := r.activityCenterNotifications[id]; ok {
delete(r.activityCenterNotifications, id)
return true
}
return false
}
func (r *MessengerResponse) ActivityCenterNotifications() []*ActivityCenterNotification {
var ns []*ActivityCenterNotification
for _, n := range r.activityCenterNotifications {
ns = append(ns, n)
}
return ns
}
func (r *MessengerResponse) AddPinMessage(pm *common.PinMessage) {
if r.pinMessages == nil {
r.pinMessages = make(map[string]*common.PinMessage)
}
r.pinMessages[pm.ID] = pm
}
func (r *MessengerResponse) AddPinMessages(pms []*common.PinMessage) {
for _, pm := range pms {
r.AddPinMessage(pm)
}
}
2021-03-25 15:15:22 +00:00
func (r *MessengerResponse) SetCurrentStatus(status UserStatus) {
r.currentStatus = &status
}
func (r *MessengerResponse) AddStatusUpdate(upd UserStatus) {
if r.statusUpdates == nil {
r.statusUpdates = make(map[string]UserStatus)
}
r.statusUpdates[upd.PublicKey] = upd
}
2022-09-29 11:50:23 +00:00
func (r *MessengerResponse) DiscordMessages() []*protobuf.DiscordMessage {
var ms []*protobuf.DiscordMessage
for _, m := range r.discordMessages {
ms = append(ms, m)
}
return ms
}
func (r *MessengerResponse) DiscordMessageAuthors() []*protobuf.DiscordMessageAuthor {
var authors []*protobuf.DiscordMessageAuthor
for _, author := range r.discordMessageAuthors {
authors = append(authors, author)
}
return authors
}
func (r *MessengerResponse) DiscordMessageAttachments() []*protobuf.DiscordMessageAttachment {
var attachments []*protobuf.DiscordMessageAttachment
for _, a := range r.discordMessageAttachments {
attachments = append(attachments, a)
}
return attachments
}
2021-06-03 13:11:55 +00:00
func (r *MessengerResponse) Messages() []*common.Message {
var ms []*common.Message
for _, m := range r.messages {
ms = append(ms, m)
}
return ms
}
2022-09-29 11:50:23 +00:00
func (r *MessengerResponse) AddDiscordMessageAuthor(author *protobuf.DiscordMessageAuthor) {
if r.discordMessageAuthors == nil {
r.discordMessageAuthors = make(map[string]*protobuf.DiscordMessageAuthor)
}
r.discordMessageAuthors[author.Id] = author
}
func (r *MessengerResponse) AddDiscordMessageAttachment(attachment *protobuf.DiscordMessageAttachment) {
if r.discordMessageAttachments == nil {
r.discordMessageAttachments = make(map[string]*protobuf.DiscordMessageAttachment)
}
r.discordMessageAttachments[attachment.Id] = attachment
}
func (r *MessengerResponse) AddDiscordMessageAttachments(attachments []*protobuf.DiscordMessageAttachment) {
for _, attachment := range attachments {
r.AddDiscordMessageAttachment(attachment)
}
}
func (r *MessengerResponse) AddDiscordMessage(message *protobuf.DiscordMessage) {
if r.discordMessages == nil {
r.discordMessages = make(map[string]*protobuf.DiscordMessage)
}
r.discordMessages[message.Id] = message
}
2021-06-03 13:11:55 +00:00
func (r *MessengerResponse) AddMessages(ms []*common.Message) {
for _, m := range ms {
r.AddMessage(m)
}
}
2021-03-25 15:15:22 +00:00
func (r *MessengerResponse) AddMessage(message *common.Message) {
2021-06-03 13:11:55 +00:00
if r.messages == nil {
r.messages = make(map[string]*common.Message)
}
if message.Deleted {
return
}
2021-06-03 13:11:55 +00:00
r.messages[message.ID] = message
}
func (r *MessengerResponse) AddContact(c *Contact) {
for idx, c1 := range r.Contacts {
if c1.ID == c.ID {
r.Contacts[idx] = c
return
}
}
r.Contacts = append(r.Contacts, c)
}
2023-02-10 12:20:06 +00:00
func (r *MessengerResponse) AddContacts(contacts []*Contact) {
for idx := range contacts {
r.AddContact(contacts[idx])
}
}
func (r *MessengerResponse) AddInstallation(i *multidevice.Installation) {
for idx, i1 := range r.Installations {
if i1.ID == i.ID {
r.Installations[idx] = i
return
}
}
r.Installations = append(r.Installations, i)
}
func (r *MessengerResponse) AddInstallations(installations []*multidevice.Installation) {
for idx := range installations {
r.AddInstallation(installations[idx])
}
}
2021-06-03 13:11:55 +00:00
func (r *MessengerResponse) SetMessages(messages []*common.Message) {
r.messages = make(map[string]*common.Message)
r.AddMessages(messages)
2021-03-25 15:15:22 +00:00
}
2021-06-03 14:09:30 +00:00
func (r *MessengerResponse) GetMessage(messageID string) *common.Message {
if r.messages == nil {
return nil
}
return r.messages[messageID]
}
feat: introduce messenger APIs to extract discord channels As part of the new Discord <-> Status Community Import functionality, we're adding an API that extracts all discord categories and channels from a previously exported discord export file. These APIs can be used in clients to show the user what categories and channels will be imported later on. There are two APIs: 1. `Messenger.ExtractDiscordCategoriesAndChannels(filesToimport []string) (*MessengerResponse, map[string]*discord.ImportError)` This takes a list of exported discord export (JSON) files (typically one per channel), reads them, and extracts the categories and channels into dedicated data structures (`[]DiscordChannel` and `[]DiscordCategory`) It also returns the oldest message timestamp found in all extracted channels. The API is synchronous and returns the extracted data as a `*MessengerResponse`. This allows to make the API available status-go's RPC interface. The error case is a `map[string]*discord.ImportError` where each key is a file path of a JSON file that we tried to extract data from, and the value a `discord.ImportError` which holds an error message and an error code, allowing for distinguishing between "critical" errors and "non-critical" errors. 2. `Messenger.RequestExtractDiscordCategoriesAndChannels(filesToImport []string)` This is the asynchronous counterpart to `ExtractDiscordCategoriesAndChannels`. The reason this API has been added is because discord servers can have a lot of message and channel data, which causes `ExtractDiscordCategoriesAndChannels` to block the thread for too long, making apps potentially feel like they are stuck. This API runs inside a go routine, eventually calls `ExtractDiscordCategoriesAndChannels`, and then emits a newly introduced `DiscordCategoriesAndChannelsExtractedSignal` that clients can react to. Failure of extraction has to be determined by the `discord.ImportErrors` emitted by the signal. **A note about exported discord history files** We expect users to export their discord histories via the [DiscordChatExporter](https://github.com/Tyrrrz/DiscordChatExporter/wiki/GUI%2C-CLI-and-Formats-explained#exportguild) tool. The tool allows to export the data in different formats, such as JSON, HTML and CSV. We expect users to have their data exported as JSON. Closes: https://github.com/status-im/status-desktop/issues/6690
2022-07-13 09:33:53 +00:00
func (r *MessengerResponse) AddDiscordCategory(dc *discord.Category) {
for idx, c := range r.DiscordCategories {
if dc.ID == c.ID {
r.DiscordCategories[idx] = dc
return
}
}
r.DiscordCategories = append(r.DiscordCategories, dc)
}
func (r *MessengerResponse) AddDiscordChannel(dc *discord.Channel) {
for idx, c := range r.DiscordChannels {
if dc.ID == c.ID {
r.DiscordChannels[idx] = dc
return
}
}
r.DiscordChannels = append(r.DiscordChannels, dc)
}
func (r *MessengerResponse) HasDiscordCategory(id string) bool {
for _, c := range r.DiscordCategories {
if id == c.ID {
return true
}
}
return false
}
func (r *MessengerResponse) HasDiscordChannel(id string) bool {
for _, c := range r.DiscordChannels {
if id == c.ID {
return true
}
}
return false
}