0ccdec2985
Prior to this commit we had a `CreateHistoryArchiveTorrent()` API which takes a `startDate`, an `endDate` and a `partition` to create a bunch of message archives, given a certain time range. The function expects the messages to live in the database, which means, all messages that need to be archived have to be saved there at some point. This turns out to be an issue when importing communities from third party services, where, sometimes, there are several thousands of messages including attachment payloads, that have to be save to the database first. There are only two options to get the messages into the database: 1. Make one write operation with all messages - this slow, takes a long time and blocks the database until done 2. Create message chunks and perform multiple write operations - this is also slow, takes long but makes the database a bit more responsive as it's many smaller operations instead of one big one Option 2) turned out to not be super feasible either as sometimes, inserting even a single such message can take up to 10 seconds (depending on payload) Which brings me to the third option. **A third option** is to not store those imported messages as waku message into the database, just to later query them again to create the archives, but instead create the archives right away from all the messages that have been loaded into memory. This is significantly faster and doesn't block the database. To make this possible, this commit introduces a `CreateHistoryArchiveTorrentFromMessages()` API, and a `CreateHistoryArchiveTorrentFromDB()` API which can be used for different use cases. |
||
|---|---|---|
| .. | ||
| anonmetrics | ||
| audio | ||
| common | ||
| communities | ||
| datasync | ||
| discord | ||
| encryption | ||
| ens | ||
| identity | ||
| images | ||
| internal/sqlite | ||
| migrations | ||
| protobuf | ||
| pushnotificationclient | ||
| pushnotificationserver | ||
| requests | ||
| sqlite | ||
| subscription | ||
| transport | ||
| tt | ||
| urls | ||
| v1 | ||
| verification | ||
| zaputil | ||
| LICENSE | ||
| README.md | ||
| activity_center.go | ||
| activity_center_persistence.go | ||
| anon_metrics_test.go | ||
| chat.go | ||
| chat_group_proxy.go | ||
| chat_test.go | ||
| communities_messenger_test.go | ||
| contact.go | ||
| delete_for_me_message.go | ||
| delete_message.go | ||
| edit_message.go | ||
| emoji_reaction.go | ||
| errors.go | ||
| group_chat_invitation.go | ||
| group_chat_system_messages.go | ||
| identity_images.go | ||
| local_notifications.go | ||
| message_builder.go | ||
| message_persistence.go | ||
| message_validator.go | ||
| message_validator_test.go | ||
| messenger.go | ||
| messenger_activity_center.go | ||
| messenger_activity_center_test.go | ||
| messenger_auto_message.go | ||
| messenger_backup.go | ||
| messenger_backup_test.go | ||
| messenger_bookmarks.go | ||
| messenger_browsers.go | ||
| messenger_browsers_test.go | ||
| messenger_chats.go | ||
| messenger_communities.go | ||
| messenger_config.go | ||
| messenger_contact_requests_test.go | ||
| messenger_contact_update_test.go | ||
| messenger_contact_verification.go | ||
| messenger_contacts.go | ||
| messenger_delete_message_for_me_test.go | ||
| messenger_delete_message_test.go | ||
| messenger_discv5.go | ||
| messenger_edit_message_test.go | ||
| messenger_emoji_test.go | ||
| messenger_ens.go | ||
| messenger_group_chat.go | ||
| messenger_group_chat_test.go | ||
| messenger_handler.go | ||
| messenger_handler_test.go | ||
| messenger_identity.go | ||
| messenger_identity_image_test.go | ||
| messenger_images.go | ||
| messenger_installations_test.go | ||
| messenger_mailserver.go | ||
| messenger_mailserver_cycle.go | ||
| messenger_maps.go | ||
| messenger_messages.go | ||
| messenger_mute_test.go | ||
| messenger_peers.go | ||
| messenger_pin_message_test.go | ||
| messenger_pin_messages.go | ||
| messenger_response.go | ||
| messenger_response_test.go | ||
| messenger_saved_address.go | ||
| messenger_share_image_test.go | ||
| messenger_status_updates.go | ||
| messenger_status_updates_test.go | ||
| messenger_sync_bookmark_test.go | ||
| messenger_sync_chat_test.go | ||
| messenger_sync_clear_history_test.go | ||
| messenger_sync_profile_picture_test.go | ||
| messenger_sync_saved_addresses_test.go | ||
| messenger_sync_settings.go | ||
| messenger_sync_settings_test.go | ||
| messenger_sync_verification_test.go | ||
| messenger_sync_wallets_test.go | ||
| messenger_test.go | ||
| messenger_walletconnect.go | ||
| messenger_walletconnect_test.go | ||
| persistence.go | ||
| persistence_test.go | ||
| push_notification_test.go | ||
| status_update.go | ||
| transaction_validator.go | ||
| transaction_validator_test.go | ||
README.md
status-go/protocol
This is an implementation of the secure transport and payloads which are a part of the Status Client specification.
This implementation uses SQLite and SQLCipher for persistent storage.
The payloads are encoded using protocol-buffers.
Content
messenger.gois the main file which exportsMessengerstruct. This is a public API to interact with this implementation of the Status Chat Protocol.protobuf/contains protobuf files implementing payloads described in the Payloads spec.encryption/implements the Secure Transport spec.transport/connects the Status Chat Protocol with a wire-protocol which in our case is either Whisper or Waku.datasync/is an adapter for MVDS.applicationmetadata/is an outer layer wrapping a payload with an app-specific metadata like a signature.identity/implements details related to creating a three-word name and identicon.migrations/contains implementation specific migrations for the sqlite database which is used byMessengeras a persistent data store.
History
Originally this package was a dedicated repo called status-protocol-go and was migrated into status-go. The new status-go/protocol package maintained its own dependencies until sub modules were removed and the root go.mod file managed all dependencies for the entire status-go repo.