This introduces the ability for status notes to handle community
history archive magnetlinks. To make this work, a few things are needed:
1. A new database table has been introduced to store message archive
hashes. This is necessary so status nodes can determine whether or
not they need to download a certain archive
2. The messenger's `handleRetrievedMessages()` has been exteded to take
magnetlink messages into account
3. New APIs were added to download torrent data given a magnetlink and
also to extract messages from downloaded archives, which are then
later fed to `handleRetrievedMessages`
Closes#2568
Previously, we've turned archive support off by default, this means,
community member nodes won't try to download torrent data when they
receive a magnetlink.
This commit ensures that communities joined in the future will have
this setting on, and also all existing communities that the node
isn't admin of will have this feature enabled.
* feat: add colorId utility
it returns color id for given pubkey
* feat: populate Account with colorHash and colorId
accounts displayed to users on login page should display colorHash and
avatar fallback color (aka colorId)
This introduces logic needed to:
- Create WakuMessageArchives and and indices from store waku messages
- History archive torrent data to disk and create .torrent file from
that
- Seed and unseed history archive torrents as necessary
- Starting/stopping the torrent client
- Enabling/disabling community history support for individual components
and starting/stopping the routine intervals accordingly
This does not yet handle magnet links (#2568)
Closes#2567
This is needed so that when they are bundled into archives, receiving
nodes can still verify the messages payload using its signature.
This commit introduces a new `waku_messages` table and APIs to store
such messages. Waku message payload is store for any message that has
a topic that matches any of the admin communities chats.
Closes#2566
* Sync Settings
* Added valueHandlers and Database singleton
Some issues remain, need a way to comparing incoming sql.DB to check if the connection is to a different file or not. Maybe make singleton instance per filename
* Added functionality to check the sqlite filename
* Refactor of Database.SaveSyncSettings to be used as a handler
* Implemented inteface for setting sync protobuf factories
* Refactored and completed adhoc send setting sync
* Tidying up
* Immutability refactor
* Refactor settings into dedicated package
* Breakout structs
* Tidy up
* Refactor of bulk settings sync
* Bug fixes
* Addressing feedback
* Fix code dropped during rebase
* Fix for db closed
* Fix for node config related crashes
* Provisional fix for type assertion - issue 2
* Adding robust type assertion checks
* Partial fix for null literal db storage and json encoding
* Fix for passively handling nil sql.DB, and checking if elem has len and if len is 0
* Added test for preferred name behaviour
* Adding saved sync settings to MessengerResponse
* Completed granular initial sync and clock from network on save
* add Settings to isEmpty
* Refactor of protobufs, partially done
* Added syncSetting receiver handling, some bug fixes
* Fix for sticker packs
* Implement inactive flag on sync protobuf factory
* Refactor of types and structs
* Added SettingField.CanSync functionality
* Addressing rebase artifact
* Refactor of Setting SELECT queries
* Refactor of string return queries
* VERSION bump and migration index bump
* Deactiveate Sync Settings
* Deactiveated preferred_name and send_status_updates
Co-authored-by: Andrea Maria Piana <andrea.maria.piana@gmail.com>
These are used to store settings for individual communities a
user is part of, either as member or as owner.
This included whether or not the community history archive protocol
is enabled.
This adds a new `CommunitySettings` type and adds
a migration script that introduces a new `communities_settings`
table.
It also extends the `MessengerResponse` type to include
`CommunitySettings` which are honored when communities are being
added, edited, joined or left.
Lastly, this adds a new RPC API to retreive the settings.
Closes#2564
This commit introduces a new `TorrentConfig` as per #2563 with dedicated
default values and new options/flags for the status-go CLI.
Since it's part of `NodeConfig`, which is stored per user in the
database, this commit also adds a migration script that adds a new
`torrent_config` table and database APIs to insert and retreive the
torrent config.
Closes#2563
This commit adds the special magnetlink channel that is used to
distribute community archive magnetlinks to community members.
The channel shoudn't be visible in the UI, so instead creating an
actual channel instance, it just sets up the filters.
Closes#2565
This commit enables mailserver cycle logic by default and make a few
changes:
1) Nodes are graylisted instead of being blacklisted for a set amount of
time. The reason is that if we blacklist, any cut in connectivity
might result in long delays before reconnecting, especially on spotty
connections.
2) Fixes an issue on the devp2p server, whereby the node would not
connect to one of the static nodes since all the connection slots
where filled. The fix is a bit inelegant, it always connects to
static nodes, ignoring maxpeers, but it's tricky to get it to work
since the code is clearly not written to select a specific node.
3) Adds support to pinned mailservers
4) Add retries to mailservers requests. It uses a closure for now, I
think we should eventually have a channel etc, but I'd leave that for
later.
Prior to this commit, when a community owner leaves her community,
status-go would just set the `joined` flag to `false` without updating the
members list. This results in wrong data as the number of members doesn't
match the reality.
This commit removes the owner of the community from the members list
when she's leaving the community.
We periodically check whether we should backup contacts through waku.
Currently it's not tied to any event, it will only schedule a backup on
a tick, or through the provided API endpoint.
Sometimes the message scheduled & message sent notifications are
received out of order.
Now we use a single channel for both so order is maintained.
This was causing the tests to be flaky and it might have happened in
production as well.
We need to discard installations with no metadata, as some might be sent
before, so we need to check in the loop that the one we care about is
received.
This commit fixes one source of flakyness in the tests, which was an
actual bug.
If 1 device is registering with a push notification server, if there's
another device with the same public key, both would mark themselves as
registered, while maybe only one has been actually registered.
To fix this, we keep track of the request ids we send (in memory for
now), and only mark it as registered if the request was originating on
this device.
* Protobufs and adapters
* Added basic anon metric service and config init
* Added fibonacci interval incrementer
* Added basic Client.Start func and integrated interval incrementer
* Added new processed field to app metrics table
* Added id column to app metrics table
* Added migration clean up
* Added appmetrics GetUnprocessed and SetToProcessedByIDs and tests
There was a wierd bug where metrics in the db that did not explicitly insert a value would be NULL, so could not be found by . In addition I've added a new primary id field to the app_metrics table so that updates could be done against very specific metric rows.
* Updated adaptors and db to handle proto_id
I need a way to distinguish individual metric items from each other so that I can ignore the ones that have been seen before.
* Moved incrementer into dedicated file
* Resolve incrementer test fail
* Finalised the main loop functionality
* Implemented delete loop framework
* Updated adaptors file name
* Added delete loop delay and quit, and tweak on RawMessage gen
* Completed delete loop logic
* Added DBLock to prevent deletion during mainLoop
* Added postgres DB connection, integrated into anonmetrics.Server
* Removed proto_id from SQL migration and model
* Integrated postgres with Server and updated adaptors
* Function name update
* Added sample config files for client and server
* Fixes and testing for low level e2e
* make generate
* Fix lint
* Fix for receiving an anonMetricBatch not in server mode
* Postgres test fixes
* Tidy up, make vendor and make generate
* delinting
* Fixing database tests
* Attempted fix of does: cannot open `does' (No such file or directory)
not: cannot open `not' (No such file or directory)
exist: cannot open `exist' (No such file or directory) error on sql resource loas
* Moved all anon metric postgres migration logic and sources into a the protocol/anonmetrics package or sub packages. I don't know if this will fix the does: cannot open `does' (No such file or directory)
not: cannot open `not' (No such file or directory)
exist: cannot open `exist' (No such file or directory) error that happens in Jenkins but this could work
* Lint for the lint god
* Why doesn't the linter list all its problems at once?
* test tweaks
* Fix for wakuV2 change
* DB reset change
* Fix for postgres db migrations fails
* More robust implementation of postgres test setup and teardown
* Added block for anon metrics functionality
* Version Bump to 0.84.0
* Added test to check anon metrics broadcast is deactivated
* Protobufs and adapters
* Added basic anon metric service and config init
* Added new processed field to app metrics table
* Added id column to app metrics table
* Added migration clean up
* Added appmetrics GetUnprocessed and SetToProcessedByIDs and tests
There was a wierd bug where metrics in the db that did not explicitly insert a value would be NULL, so could not be found by . In addition I've added a new primary id field to the app_metrics table so that updates could be done against very specific metric rows.
* Updated adaptors and db to handle proto_id
I need a way to distinguish individual metric items from each other so that I can ignore the ones that have been seen before.
* Added postgres DB connection, integrated into anonmetrics.Server
* Removed proto_id from SQL migration and model
* Integrated postgres with Server and updated adaptors
* Added sample config files for client and server
* Fix lint
* Fix for receiving an anonMetricBatch not in server mode
* Postgres test fixes
* Tidy up, make vendor and make generate
* Moved all anon metric postgres migration logic and sources into a the protocol/anonmetrics package or sub packages. I don't know if this will fix the does: cannot open `does' (No such file or directory)
not: cannot open `not' (No such file or directory)
exist: cannot open `exist' (No such file or directory) error that happens in Jenkins but this could work
AllMessagesFromChatsAndCommunitiesWhichMatchTerm method added which returns all messages which
match the search term, if they belong to either any chat from the chatIds array or any channel of
any community from communityIds array.
This api point is necessary for desktop issue 2934.
Fixes: #2934
* Added community sync protobuf
* Updated community sync send logic
* Integrated syncCommunity handling
* Added synced_at field and tidied up some other logic
* persistence testing
* Added testing and join functionality
* Fixed issue with empty scan params
* Finshed persistence tests for new db funcs
* Midway debug of description not persisting after sync
* Resolved final issues and tidied up
* Polish
* delint
* Fix error not handled on SetPrivateKey
* fix infinite loop, again
* Added muted option and test fix
* Added Muted to syncing functions, not just in persistence
* Fix bug introduced with Muted property
* Added a couple of notes for future devs
* Added most of the sync RequestToJoin functionality
Tests need to be completed and tests are giving some errors
* Finished tests for getJoinedAndPending
* Added note
* Resolving lint
* Fix of protobuf gen bug
* Fixes to community sync tests
* Fixes to test
* Continued fix of e2e
* Final fix to e2e testing
* Updated migration position
* resolve missing import
* Apparently the linter spellchecks
* Fix bug from #2276 merge
* Bug fix for leaving quirkiness
* Addressed superfluous MessengerResponse field
* Addressed feedback
* VERSION bump
* feature(@status-go/chat): implement search on sqlcipher (status-go side)
Searching messages by some term for a specific channel is added on the side of status-go.
Fixes: #2912
* Linting
Co-authored-by: Andrea Maria Piana <andrea.maria.piana@gmail.com>
Changes the community invitation text notifications from
“Upgrade to see a community invitation”
to
**InviteUsersToCommunity**
```golang
fmt.Sprintf("You have been invited to the community %s", community.Name())
```
**ShareCommunity**
```golang
fmt.Sprintf("Community %s has been shared with you", community.Name())
```
* Adding wakunode module
* Adding wakuv2 fleet files
* Add waku fleets to update-fleet-config script
* Adding config items for waku v2
* Conditionally start waku v2 node depending on config
* Adapting common code to use go-waku
* Setting log level to info
* update dependencies
* update fleet config to use WakuNodes instead of BootNodes
* send and receive messages
* use hash returned when publishing a message
* add waku store protocol
* trigger signal after receiving store messages
* exclude linting rule SA1019 to check deprecated packages
This commit adds a setting for mailserver, that allows the client to
specify the syncing period.
Also fixes a minor issue with deletion of public chats.
* feat: Add edit communities
Allow Communities to be edited, including display name, description, color, membership, and permissions.
* Added EditCommunity request type
* Fix lint errors
* Allow editing community without changing image
Previously, retaining an existing community image was not possible because the existing community image path had to be provided in the `editCommunity` RPC call to retain the image. However, once the image is processed by status-go, it is encoded as a base64 string and therefore it is not possible to get the original file path back from this string.
This commit allows for the original to be retained by passing an empty string for the image field in the RPC call.
* Don't change permissions. Fixed clock updating
Co-authored-by: Volodymyr Kozieiev <vkjr.sp@gmail.com>
This property is useful for clients to know when a channel or chat
was joined so they can use that to calculate the order of channels
and chats shown in applications.
Changes include a new joined property on the Chat struct,
as well as adjustments in the persistence layer to retreive and
update chat data in the database.
In addition there's a migration script that alters the existing
chat table to introduce a new column for the joined field.
It also updates all existing rows in the database to set `joined`
to `0`.
* add PinMessage and PinnedMessage
* fix gruop pin messages
* add SkipGroupMessageWrap to pin messages
* update pinMessage ID generation to be symmetric
* Added anon metrics send opt in setting
* resolved rebase conflict, renamed migration to use unixtimestamp
Theres always conflicts with migrations using sequential numbers, less so with unix timestamp
* Add extra event to capture other type of navigations, allow empty screen name, rename cofx to get rid of clj ns, update tests
* Some view ids are greater than 16 characters. Made it 32 to be safe.
* Tab navigation events occur outside nav, add a new validator for them
* Remove navigate to cofx event, capture screens on will focus, get rid of enum and make valid screens a string less than 32 characters
* Run make generate
* Fix test
* Bump version to 0.75.1
This commit introduces the following changes:
- `local-notifications` require as body an interface complying with
`json.Marshaler`
- removed unmarshaling of `Notifications` as not used (we only Marshal
notifications)
- `protocol/messenger.go` creates directly a `Notification` instead of
having an intermediate format
- add community notifications on request to join
- move parsing of text in status-go for notifications
* Migrations in place, how to run them?
* Remove down migrations and touch database.go
* Database and Database Test package in place, added functions to get and store app metrics
* make generate output
* Minor bug fix on app metrics insert and select
* Add a validation layer to restrict what can be saved in the database
* Make validation more terse, throw error if schema doesn't exist, expose appmetrics service
* service updates
* Compute all errors before sending them out
* Trying to bring a closjure to appmetrics go
* Expose appmetrics via an api, skip fancy
* Address value as Jason Dawt Rawmasage to ease parsing
* Introduce a buffered chan with magic cap of 8 to minimize writes to DB. Tests for service and API. Also expose GetAppMetrics function.
* Lint issues
* Remove autoincrement, undo waku.json changes, fix error being shadowed, return nil where nil ought to be returned, get rid of buffered channel
* Bump migration number
* Fix API factory usage
* Add comment re:json.RawMessage instead of strings
* Get rid of test vars, throw save error inside the loop
* Update version
Co-authored-by: Samuel Hawksby-Robinson <samuel@samyoul.com>
This commit expands the confirmation mechanism to allow private group
chat messages to be confirmed:
Changes:
- Added a separate table for message confirmations as group chat
messages have same messageID but multiple datasyncID
- Removed DataSyncID from raw message (I haven't removed the column name
as it can't be done in sqlite without copying over the table)
Anthony Laibe
Pascal Precht
Richard Ramos
Roman Volosovskyi
Vitaliy Vlasov
Sale Djenic
osmaczko
Parvesh Monu
Andrea Maria Piana
Samuel Hawksby-Robinson
andrey
Patryk Osmaczko
John M. Ngei
frank
Jonathan Rainville
flexsurfer
Anthony Laibe
Michele Balistreri
Pascal Precht
Volodymyr Kozieiev
saledjenic
Brian Sztamfater
Jakub Sokołowski
Ismael
Eric Mastro
Andrea Franz
Brian Sztamfater
Shivek Khurana