There is a tiny chance that without this fix some users which had message
duplicates because of issues with `message-id` calculation **BEFORE** `0.9.31`
still have duplicates in their DB and migrations will not pass without
this change. It only checks if the message with a new `message-id` has been
added and removes duplicate. The same way it removes duplicates from
`user-status` entity.
Currently, we calculate `message-id` as `sha3(from + raw_payload)`,
but we do not store `raw_payload` and it might be problematic
to restore it from DB because:
1) `content` field might be changed and so `Message` record
will differ from the original one
2) it is even more problematic for `GroupMembershipUpdate`
message because we don't save it in DB
In order to handle this, we can store `sha3(raw_payload)` as `raw-payload-hash`
prop of `message` entity and use it in case of emergency :)
`message-id` will be calculated as `sha3(from + sha3(raw_payload))`
Implementation:
1. `transport.utils/message-id` function is called only in three places now
and accepts `from` and `raw_payload` as parameters.
ID is calculated as `sha3(from + raw_payload)`.
2. This means that for wrapped private group chat message
the raw payload of `GroupMembershipUpdate` is used.
Issue was caused by https://github.com/status-im/status-react/pull/6722
Implementation:
1. `old-message-id` field (indexed) was introduced in `message` entity
and is calculated as `message-id` was calculated in `0.9.31`
```clojure
(defn old-message-id
[message]
(sha3 (pr-str message)))
```
2. When a reply message is sent from the PR version of app both `response-to`
and `response-to-v2` fields are sent as a part of `message`'s `content`
field, so that it can be recognized by `0.9.31`.
3. When PR version of app receives reply from `0.9.31` we check whether
message's `content` contains `response-to` but doesn't contain
`response-to-v2`, and if so we check whether DB contains message with
`old-message-id=response-to`. If such message has been found we assoc
`response-to-v2` to content.
4. If message from DB contains only `response-to` but not `response-to-v2`
attempt to fetch the message by `old-message-id` is done.
Migration failures are handled separately from other errors which might appear
during opening account's realm DB. In case if user chooses to erase
the account's database, only this database will be removed and other accounts
will not be touched.
Before we fetched ALL user-statuses with `status=received` (which means that
a message hasn't been seen), iterated them, grouped by chat and then stored
`message-ids` of these `user-statuses` in chat's `:unviewed-messages` key.
This commit introduces :unviewed-messages-count field in chat entity.
That means that there is no need to iterate `user-statuses` in order to count
a total number of unviewed messages, it is always stored along with chat.
In the rest of it, the difference is only that chat's db record should be
updated each time when unviewed messages are seen.
Everytime a contact request is sent/confirmed a sync message is also
sent to other devices so the contact is kept in sync.
Signed-off-by: Andrea Maria Piana <andrea.maria.piana@gmail.com>
- do not logout and remove previous mailserver
from peers when changing mailserver
- rename wnode mailserver
- move transport.inbox to mailserver.core
- fix all subs and db keys
Signed-off-by: yenda <eric@status.im>
Adds a `chat-id` field in `content` map.
The reason it has been added to the map instead of augmenting transit is
that it would simplify the calculation of `message-id`, which in this
case is consistent for both old & new clients.
`chat-id` also represents the `chat-id` with respect of the sender, as
in 1-to-1 chats that is asymmetric.
Signed-off-by: Andrea Maria Piana <andrea.maria.piana@gmail.com>
- fetch 7 days of history when joining a chat
- make 7 24h requests to request 7 days because mailservers
ignores requests for a timespan > 24h
- make requests sequentially to avoid timeouts
- change mailserver after 3 timeouts on a request
Signed-off-by: yenda <eric@status.im>
the purpose of this change is to avoid the kind of mistakes that
happened in the past were copy/pasting imports led to wrong
migrations that went unnoticed
it will also make it easier to do migrations in the future as the
different versions of the same schemas are grouped together in the
same file and past migrations are grouped in the same file as well
so previous helper functions can be reused
Signed-off-by: yenda <eric@status.im>
