Chat messages are now sent in order using a different endpoint
`sendChatMessages`.
Text should always be displayed after images.
This is not implementing a Caption field, that would require either a
protocol change or leverage the `text` in the message.
It applies for both normal chats and timelines.
Move also all inputs under `chat/inputs` so we avoid re-renders as
`chats` has changed.
Signed-off-by: Andrea Maria Piana <andrea.maria.piana@gmail.com>
This commit does a few things:
Move collections top level
Move `messages`,`message-lists`,`pagination-info` from nested in
`chats` to top level at the db.
The reason for this change is that if any of the `messages` fields
change, any `sub` that relies on `chat` will be recomputed, which is
unnecessary.
Move chat-name to events
`chat-name` was computed dynamically, while it is now only calculated
when loading chat the first time around.
Remove `enrich-chats`
Enrich chats was doing a lot of work, and many subscriptions were
relying on it.
Not all the computations were necessary, for example it would always
calculate the name of who invited the user to a group chat, regardless
of whether it was actually used in the view.
This commit changes that behavior so that we use smaller subscriptions
to calculate such fields.
In general we should move computations to events, if that's not
desirable (there are some cases where we might not want to do that), we
should have "bottom/leaf heavy" subscriptions as opposed to "top heavy",
especially if they are to be shared, so only when (and if) we load that
particular view, the subscription is triggered, while others can be
re-used.
I have compared performance with current release, and there's a
noticeable difference. Opening a chat is faster (messages are loaded
faster), and clicking on the home view on a chat is more responsing
(the animation on-press is much quicker).
In some instances when receiving messages from a mailsever in the chat
you are in, the flag `all-loaded?` would not be reset, meaning that
messages not in the current view would be added to the db, but would not be seen until actually
reloading the chat (go back home, open again).
Signed-off-by: Andrea Maria Piana <andrea.maria.piana@gmail.com>
Get rid of navigation wrapper
Use new API to declare navigation
Update tabbar component
Update to use new navigation events
Add ios presentation modal
Navigation cleanups
Android specific updates
Use letsubs for stack subscriptions
Keycard did load event backward compatibility
Fix tabbar and wallet on-focus bad rebase
Do not keep welcome screen into the stack
Comment outdated test
Fix rebase on home PR
Cancel back button on screens which can't be popped
Signed-off-by: Gheorghe Pinzaru <feross95@gmail.com>
While on tabs we want a slightly different behavior:
Unread counter should increase, message should be loaded in the chat
On moving to the chat tab from one of these tabs, it should mark the
messages as seen.
Signed-off-by: Andrea Maria Piana <andrea.maria.piana@gmail.com>
This commit allows setting waku-mode and waku-bloom-filter-mode
dynamically.
It requires a relogin for the changes to take effect.
Signed-off-by: Andrea Maria Piana <andrea.maria.piana@gmail.com>
This commit does a few things:
1) Messages are offloaded from any chat once we go back from the home.
This allows us to ignore any message that is coming in from a chat we
are not currently focused.
2) After 5 seconds of not-scrolling activity, any received message that
is not currently visible will be offloaded to the database.
3) Similarly received messages that are not visible will be offloaded to
the database directly
Signed-off-by: Andrea Maria Piana <andrea.maria.piana@gmail.com>
This commit does a few things:
==== Ordering of messages ====
Change the ordering of messages from a mixture of timestamp/clock-value to use
only clock-value.
Datemarks are now not used for sorting anymore, which means that the
order of messages is always causally related (not the case before, as we
were breaking this property by sorting by datemark), but datemark
calculation is unreliable (a reply to a message might have a timestamp <
then the message that is replied to).
So for timestamp calculation we
naively group them ignoring "out-of-order timestamp" messages, although
there's much to improve.
It fixes an issue whereby the user would change their time and the
message will be displayed in the past, although it is still possible to
craft a message with a lower clock value and order it in the past
(there's no way we can prevent this to some extent, but there are ways
to mitigate, but outside the scope of this PR).
==== Performance of receiving messages ====
The app would freeze on pulling messages from a mailserver (100 or so).
This is due to the JS Thread being hogged by CPU calculation, coupled
with the fact that we always tried to process messages all in one go.
This strategy can't scale, and given x is big enough (200,300,1000) the
UI will freeze.
Instead, each message is now processed separately, and we leave a gap
between processing each message for the UI to respond to user input
(otherwise the app freezes again).
Pulling messages will be longer overall, but the app will be usuable
while this happen (albeit it might slow down).
Other strategies are possible (calculate off-db and do a big swap,
avoiding many re-renders etc), but this is the reccommended strategy by
re-frame author (Solving the CPU Hog problem), so sounds like a safe
base point.
The underlying data structure for holding messages was also changed, we
used an immutable Red and Black Tree, same as a sorted map for clojure, but we use
a js library as is twice as performing then clojure sorted map.
We also don't sort messages again each time we receive them O(nlogn), but we
insert them in order O(logn).
Other data structures considered but discarded:
1) Plain vector, but performance prepending/insertion in the middle
(both O(n)) were not great, as not really suited for these operations.
2) Linked list, appealing as append/prepend is O(1), while insertion is
O(n). This is probably acceptable as messages tend to come in order
(from the db, so adding N messages is O(n)), or the network (most of
them prepends, or close to the head), while mailserver would not follow this path.
An implementation of a linked list was built, which performed roughtly the
same as a clojure sorted-map (although faster append/prepend), but not
worth the complexity of having our own implementation.
3) Clojure sorted-map, probably the most versatile, performance were
acceptable, but nowhere near the javascript implementation we decided on
4) Priority map, much slower than a sorted map (twice as slow)
5) Mutable sorted map, js implementation, (bintrees), not explored this very much, but from
just a quick benchmark, performance were much worse that clojure
immutable sorted map
Given that each message is now processed separately, saving the chat /
messages is also debounced to avoid spamming status-go with network
requests. This is a temporary measure for now until that's done directly
in status-go, without having to ping-pong with status-react.
Next steps performance wise is to move stuff to status-go, parsing of
transit, validation, which is heavy, at which point we can re-consider
performance and how to handle messages.
Fixes also an issue with the last message in the chat, we were using the
last message in the chat list, which might not necessarely be the last
message the chat has seen, in case messages were not loaded and a more
recent message is the database (say you fetch historical messages for
1-to-1 A, you don't have any messages in 1-to-1 chat B loaded, you receive an
historical message for chat B, it sets it as last message).
Also use clj beans instead of js->clj for type conversion
Signed-off-by: Andrea Maria Piana <andrea.maria.piana@gmail.com>
This commit does a few things:
1) Move messages to status-go
2) Use message-id computed from status-go
3) Remove old replies
Old message id was used for compatibility of replies with older clients.
Given that v1 is breaking, this is not needed anymore and simplifies
moving messages to status-go. No protocol/data-store change is made, to minimize
changes.
Signed-off-by: Andrea Maria Piana <andrea.maria.piana@gmail.com>
This commit moves chats to status-go.
I have changed the logic to load all chats in one go for simplicity and
while that might have a performance impact, I think it's premature to
optimize this flow as there will be more changes to the login flow.
Also currently this is likely to be slower as we need to wait for the
status-service to be initialized, as well as realm.
No migration is provided as we are past the point of no return, so by
installing this version you will lose your chats.
Signed-off-by: Andrea Maria Piana <andrea.maria.piana@gmail.com>
All the code has been implemented in statusgo: status-im/status-go#1466
Basically all the whisper filter management is done at that level.
Technical description
On startup we load all chats and send a list of them to status go:
For a public chat: {:chatId "status"}, we create a single filter, based on the name of the chat.
For each contact added by us, each user in a group chat and each one to one chat open, we send:
{:chatId "0x", :oneToOne true}. This will create a chats, to listen to their contact code.
Any previously negotiated topic is also returned.
Once loaded, we create our filters, and upsert the mailserver topics, both of which are solely based on the filters loaded.
In order to remove a chat, we delete/stopwatching first the the filter in status-react and then ask status-go to remove the filter. For a public chat we always remove, for a one-to-one we remove only if the user is not in our contacts, or in a group chat or we have a chat open. Negotiated topics are never removed, as otherwise the other user won't be able to contact us anymore.
On stopping whisper we don't have to ask status-go to remove filters as they are removed automatically.
Some more logic can be pushed in status-go, but that will be in subsequent PRs.
Signed-off-by: Andrea Maria Piana <andrea.maria.piana@gmail.com>
- upgrade to realm 2.28 to benefit from perf improvements
- remove user-statuses and replace by seen and outgoing-status fields
to get rid of a lot of bloat queries and computations
- remove unused seen message, bottom-infos
- remove unused fields in transport schema
- use objectForPrimaryKey whenever possible instead of get by field
Signed-off-by: yenda <eric@status.im>
Connect to stubs of status-go protocol API, behind the flag. Since status-go isn't updated yet, setting this flag will break the app.
What needs to be tested is no regressions in a normal mode.
Signed-off-by: Igor Mandrigin <i@mandrigin.ru>
- add block/unblock action to user profile
- blocking deletes all messages from user and ignores future messages
- unblocking stops ignoring new messages from user but doesn't recover past ones
[feature] add contact list
[tests] added scroll to BackupRecoveryPhraseButton
[tests] added scroll to public key
Signed-off-by: yenda <eric@status.im>
10 last chats are loaded to `app-db` before showing `:home` screen, in
result a user will not see two consequent activity indicators. In this
case opening of `:home` screen is a bit slower but looks better from
UI/UX pov. As it is limited to 10 chats on initialization, the time
necessary for opening `:home` screen will not depend on a total number
of chats in `app-db` if an account contains 10+ chats.
Signed-off-by: Igor Mandrigin <i@mandrigin.ru>
Paginating using the count of loaded messages might result in some
messages being skipped and not being loaded in the database, in case of
out-of-order messages received.
This commit changes the behavior to sort by `clock-value` and
`message-id`, which gives a consistent sorting.
The initial idea was to use a cursor `clock-value-message-id` and
iterate on that, but realm does not support filtering on string (</>),
so instead we keep track of messages with identical clock-value and
exclude those in the next page query.
The change might result in pages that have duplicates (so messages needs
to be deduped), but won't result in skipped messages.
The last messages of the chats are necessary to properly show the chat
list, which is shown right after signing in. Before this commit, the
last message was retrieved as one of 20 last messages fetched for each
chat.
Implementation:
- `:last-message-content` and `:last-message-type` fields were added to
`chat` entity
- both fields are updated when messages are received/sent
- loading of the last 20 messages for each chat was removed as
initialization step
Issue was caused by https://github.com/status-im/status-react/pull/6722
Implementation:
1. `old-message-id` field (indexed) was introduced in `message` entity
and is calculated as `message-id` was calculated in `0.9.31`
```clojure
(defn old-message-id
[message]
(sha3 (pr-str message)))
```
2. When a reply message is sent from the PR version of app both `response-to`
and `response-to-v2` fields are sent as a part of `message`'s `content`
field, so that it can be recognized by `0.9.31`.
3. When PR version of app receives reply from `0.9.31` we check whether
message's `content` contains `response-to` but doesn't contain
`response-to-v2`, and if so we check whether DB contains message with
`old-message-id=response-to`. If such message has been found we assoc
`response-to-v2` to content.
4. If message from DB contains only `response-to` but not `response-to-v2`
attempt to fetch the message by `old-message-id` is done.
Before we fetched ALL user-statuses with `status=received` (which means that
a message hasn't been seen), iterated them, grouped by chat and then stored
`message-ids` of these `user-statuses` in chat's `:unviewed-messages` key.
This commit introduces :unviewed-messages-count field in chat entity.
That means that there is no need to iterate `user-statuses` in order to count
a total number of unviewed messages, it is always stored along with chat.
In the rest of it, the difference is only that chat's db record should be
updated each time when unviewed messages are seen.
- in future PRs we want to reduce the expensive sort operations
on list of messages while keeping the possibility to get a message
by its ID
- priority map allow to keep a map sorted by it's value which
is what we want to do here. we want to keep the messages ordered
by clock-value
Signed-off-by: yenda <eric@status.im>
Andrea Maria Piana
andrey
Roman Volosovskyi
Gheorghe Pinzaru
yenda
Igor Mandrigin
janherich