status-im
/
status-mobile
mirror of https://github.com/status-im/status-mobile.git
2
0
Fork 0
Code Issues Projects Releases Wiki Activity

Fix message ordering and improve performance rec. messages 

This commit does a few things:

==== Ordering of messages ====

Change the ordering of messages from a mixture of timestamp/clock-value to use
only clock-value.

Datemarks are now not used for sorting anymore, which means that the
order of messages is always causally related (not the case before, as we
were breaking this property by sorting by datemark), but datemark
calculation is unreliable (a reply to a message might have a timestamp <
then the message that is replied to).
So for timestamp calculation we
naively group them ignoring "out-of-order timestamp" messages, although
there's much to improve.
It fixes an issue whereby the user would change their time and the
message will be displayed in the past, although it is still possible to
craft a message with a lower clock value and order it in the past
(there's no way we can prevent this to some extent, but there are ways
to mitigate, but outside the scope of this PR).

==== Performance of receiving messages ====

The app would freeze on pulling messages from a mailserver (100 or so).
This is due to the JS Thread being hogged by CPU calculation, coupled
with the fact that we always tried to process messages all in one go.

This strategy can't scale, and given x is big enough (200,300,1000) the
UI will freeze.

Instead, each message is now processed separately, and we leave a gap
between processing each message for the UI to respond to user input
(otherwise the app freezes again).
Pulling messages will be longer overall, but the app will be usuable
while this happen (albeit it might slow down).
Other strategies are possible (calculate off-db and do a big swap,
avoiding many re-renders etc), but this is the reccommended strategy by
re-frame author (Solving the CPU Hog problem), so sounds like a safe
base point.

The underlying data structure for holding messages was also changed, we
used an immutable Red and Black Tree, same as a sorted map for clojure, but we use
a js library as is twice as performing then clojure sorted map.

We also don't sort messages again each time we receive them O(nlogn), but we
insert them in order O(logn).

Other data structures considered but discarded:
1) Plain vector, but performance prepending/insertion in the middle
(both O(n)) were not great, as not really suited for these operations.

2) Linked list, appealing as append/prepend is O(1), while insertion is
O(n). This is probably acceptable as messages tend to come in order
(from the db, so adding N messages is O(n)), or the network (most of
them prepends, or close to the head), while mailserver would not follow this path.
An implementation of a linked list was built, which performed roughtly the
same as a clojure sorted-map (although faster append/prepend), but not
worth the complexity of having our own implementation.

3) Clojure sorted-map, probably the most versatile, performance were
acceptable, but nowhere near the javascript implementation we decided on

4) Priority map, much slower than a sorted map (twice as slow)

5) Mutable sorted map, js implementation, (bintrees), not explored this very much, but from
just a quick benchmark, performance were much worse that clojure
immutable sorted map

Given that each message is now processed separately, saving the chat /
messages is also debounced to avoid spamming status-go with network
requests. This is a temporary measure for now until that's done directly
in status-go, without having to ping-pong with status-react.

Next steps performance wise is to move stuff to status-go, parsing of
transit, validation, which is heavy, at which point we can re-consider
performance and how to handle messages.

Fixes also an issue with the last message in the chat, we were using the
last message in the chat list, which might not necessarely be the last
message the chat has seen, in case messages were not loaded and a more
recent message is the database (say you fetch historical messages for
1-to-1 A, you don't have any messages in 1-to-1 chat B loaded, you receive an
historical message for chat B, it sets it as last message).

Also use clj beans instead of js->clj for type conversion

Signed-off-by: Andrea Maria Piana <andrea.maria.piana@gmail.com>
This commit is contained in:
Andrea Maria Piana 2019-10-24 16:23:20 +02:00
parent e579412334
commit c69863cda2
No known key found for this signature in database
GPG Key ID: AA6CCA6DE0E06424
34 changed files with 897 additions and 866 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
clj-rn.conf.edn
View File

@ -38,6 +38,7 @@
"react-navigation"
"react-native-navigation-twopane"
"hi-base32"
"functional-red-black-tree"
"react-native-mail"
"react-native-shake"
"@react-native-community/netinfo"]
@ -74,6 +75,7 @@
"react-native-desktop-gesture-handler"
"web3-utils"
"react-navigation"
"functional-red-black-tree"
"react-native-navigation-twopane"
"hi-base32"]

3
desktop/js_files/package.json
View File

@ -15,6 +15,7 @@
"emojilib": "^2.2.9",
"eth-phishing-detect": "^1.1.13",
"events": "^1.1.1",
"functional-red-black-tree": "^1.0.1",
"google-breakpad": "git+https://github.com/status-im/google-breakpad.git#v0.9.0",
"hi-base32": "^0.5.0",
"i18n-js": "^3.1.0",
@ -52,8 +53,8 @@
"@babel/preset-env": "7.1.0",
"@babel/register": "7.6.2",
"babel-preset-react-native": "^5.0.2",
"metro-react-native-babel-preset": "^0.45.6",
"coveralls": "^3.0.4",
"metro-react-native-babel-preset": "^0.45.6",
"nyc": "^14.1.1",
"patch-package": "^5.1.1",
"rn-snoopy": "git+https://github.com/status-im/rn-snoopy.git#v2.0.2-status"

5
desktop/js_files/yarn.lock
View File

@ -3210,6 +3210,11 @@ function-bind@^1.0.2, function-bind@^1.1.1:
resolved "https://registry.yarnpkg.com/function-bind/-/function-bind-1.1.1.tgz#a56899d3ea3c9bab874bb9773b7c5ede92f4895d"
integrity sha512-yIovAzMX49sF8Yl58fSCWJ5svSLuaibPxXQJFLmBObTuCr0Mf1KiPopGM9NiFjiYBCbfaa2Fh6breQ6ANVTI0A==
functional-red-black-tree@^1.0.1:
version "1.0.1"
resolved "https://registry.yarnpkg.com/functional-red-black-tree/-/functional-red-black-tree-1.0.1.tgz#1b0ab3bd553b2a0d6399d29c0e3ea0b252078327"
integrity sha1-GwqzvVU7Kg1jmdKcDj6gslIHgyc=
gauge@~1.2.5:
version "1.2.7"
resolved "https://registry.yarnpkg.com/gauge/-/gauge-1.2.7.tgz#e9cec5483d3d4ee0ef44b60a7d99e4935e136d93"

10
externs.js
View File

@ -575,6 +575,16 @@ var TopLevel = {
"FIRSTWEEKCUTOFFDAY": function () {},
"decimalPlaces": function () {},
"_android": function () {},
"next": function() {},
"prev": function() {},
"hasNext": function() {},
"hasPrev": function() {},
"key": function() {},
"keys": function() {},
"values": function() {},
"find": function() {},
"insert": function() {},
"update": function() {},
"isSupported" : function () {},
"authenticate" : function () {},
"createAppContainer" : function () {},

1
mobile/js_files/package.json
View File

@ -13,6 +13,7 @@
"create-react-class": "^15.6.2",
"emojilib": "^2.4.0",
"eth-phishing-detect": "^1.1.13",
"functional-red-black-tree": "^1.0.1",
"hermes-engine": "0.2.1",
"hi-base32": "^0.5.0",
"i18n-js": "^3.3.0",

5
mobile/js_files/yarn.lock
View File

@ -2577,6 +2577,11 @@ function-bind@^1.1.1:
resolved "https://registry.yarnpkg.com/function-bind/-/function-bind-1.1.1.tgz#a56899d3ea3c9bab874bb9773b7c5ede92f4895d"
integrity sha512-yIovAzMX49sF8Yl58fSCWJ5svSLuaibPxXQJFLmBObTuCr0Mf1KiPopGM9NiFjiYBCbfaa2Fh6breQ6ANVTI0A==
functional-red-black-tree@^1.0.1:
version "1.0.1"
resolved "https://registry.yarnpkg.com/functional-red-black-tree/-/functional-red-black-tree-1.0.1.tgz#1b0ab3bd553b2a0d6399d29c0e3ea0b252078327"
integrity sha1-GwqzvVU7Kg1jmdKcDj6gslIHgyc=
gauge@~2.7.3:
version "2.7.4"
resolved "https://registry.yarnpkg.com/gauge/-/gauge-2.7.4.tgz#2c03405c7538c39d7eb37b317022e325fb018bf7"

3
project.clj
View File

@ -78,7 +78,8 @@
[day8.re-frame/tracing "0.5.0"]
[hawk "0.2.11"]]
:source-paths ["src" "env/dev" "react-native/src/cljsjs" "components/src" "dev"]}]
:test {:dependencies [[day8.re-frame/test "0.1.5"]]
:test {:dependencies [[com.taoensso/tufte "2.1.0"]
[day8.re-frame/test "0.1.5"]]
:plugins [[lein-doo "0.1.9"]]
:cljsbuild {:builds
[{:id "test"

177
src/status_im/chat/db.cljs
View File

@ -53,63 +53,59 @@
{}
chats))
(defn sort-message-groups
"Sorts message groups according to timestamp of first message in group"
[message-groups messages]
(sort-by
(comp :timestamp (partial get messages) :message-id first second)
message-groups))
(defn quoted-message-data
"Selects certain data from quoted message which must be available in the view"
[message-id messages]
(when-let [{:keys [from content]} (get messages message-id)]
{:from from
:text (:text content)}))
(defn add-datemark
[[datemark
message-references]]
(let [{:keys [whisper-timestamp timestamp]} (first message-references)]
(conj message-references
{:value datemark
:type :datemark
:whisper-timestamp whisper-timestamp
:timestamp timestamp})))
(defn datemark? [{:keys [type]}]
(= type :datemark))
(defn intersperse-datemark
"Reduce step which expects the input list of messages to be sorted by clock value.
It makes best effort to group them by day.
We cannot sort them by :timestamp, as that represents the clock of the sender
and we have no guarantees on the order.
We naively and arbitrarly group them assuming that out-of-order timestamps
fall in the previous bucket.
A sends M1 to B with timestamp 2000-01-01T00:00:00
B replies M2 with timestamp 1999-12-31-23:59:59
M1 needs to be displayed before M2
so we bucket both in 1999-12-31"
[{:keys [acc last-timestamp last-datemark]} {:keys [whisper-timestamp datemark] :as msg}]
(cond (empty? acc) ; initial element
{:last-timestamp whisper-timestamp
:last-datemark datemark
:acc (conj acc msg)}
(and (not= last-datemark datemark) ; not the same day
(< whisper-timestamp last-timestamp)) ; not out-of-order
{:last-timestamp whisper-timestamp
:last-datemark datemark
:acc (conj acc {:value last-datemark ; intersperse datemark message
:type :datemark}
msg)}
:else
{:last-timestamp (min whisper-timestamp last-timestamp) ; use last datemark
:last-datemark last-datemark
:acc (conj acc (assoc msg :datemark last-datemark))}))
(defn add-datemarks
"Add a datemark in between an ordered seq of messages when two datemarks are not
the same. Ignore messages with out-of-order timestamps"
[messages]
(when (seq messages)
(let [messages-with-datemarks (:acc (reduce intersperse-datemark {:acc []} messages))]
; Append last datemark
(conj messages-with-datemarks {:value (:datemark (peek messages-with-datemarks))
:type :datemark}))))
(defn gap? [{:keys [type]}]
(= type :gap))
(defn transform-message
[messages]
(fn [{:keys [message-id timestamp-str] :as reference}]
(if (or (datemark? reference)
(gap? reference))
reference
(let [{:keys [content quoted-message] :as message} (get messages message-id)
{:keys [response-to response-to-v2]} content
quote (if quoted-message
quoted-message
(some-> (or response-to-v2 response-to)
(quoted-message-data messages)))]
(cond-> (-> message
(update :content dissoc :response-to :response-to-v2)
(assoc :timestamp-str timestamp-str))
;; quoted message reference
quote
(assoc-in [:content :response-to] quote))))))
(defn check-gap
[gaps previous-message next-message]
(let [previous-timestamp (:whisper-timestamp previous-message)
next-whisper-timestamp (:whisper-timestamp next-message)
next-timestamp (quot (:timestamp next-message) 1000)
next-timestamp (:timestamp next-message)
ignore-next-message? (> (js/Math.abs
(- next-whisper-timestamp next-timestamp))
120)]
120000)]
(reduce
(fn [acc {:keys [from to id]}]
(if (and next-message
@ -137,15 +133,13 @@
:value (clojure.string/join (:ids gaps))
:gaps gaps}))
(defn messages-with-datemarks
(defn add-gaps
"Converts message groups into sequence of messages interspersed with datemarks,
with correct user statuses associated into message"
[message-groups messages messages-gaps
[message-list messages-gaps
{:keys [highest-request-to lowest-request-from]} all-loaded? public?]
(transduce
(comp
(mapcat add-datemark)
(map (transform-message messages)))
(map identity)
(fn
([]
(let [acc {:messages (list)
@ -162,9 +156,8 @@
:value (str :first-gap)
:first-gap? true})
acc)))
([{:keys [messages datemark-reference previous-message gaps]} message]
(let [new-datemark? (datemark? message)
{:keys [gaps-number gap]}
([{:keys [messages previous-message gaps]} message]
(let [{:keys [gaps-number gap]}
(check-gap gaps previous-message message)
add-gap? (pos? gaps-number)]
{:messages (cond-> messages
@ -173,16 +166,8 @@
(add-gap gap)
:always
(conj
(cond-> message
(not new-datemark?)
(assoc
:datemark
(:value datemark-reference)))))
(conj message))
:previous-message message
:datemark-reference (if new-datemark?
message
datemark-reference)
:gaps (if add-gap?
(drop gaps-number gaps)
gaps)}))
@ -190,73 +175,7 @@
(cond-> messages
(seq gaps)
(add-gap {:ids (map :id gaps)}))))
message-groups))
(defn- set-previous-message-info [stream]
(let [{:keys [display-photo? message-type] :as previous-message} (peek stream)]
(conj (pop stream) (assoc previous-message
:display-username? (and display-photo?
(not= :system-message message-type))
:first-in-group? true))))
(defn display-photo? [{:keys [outgoing message-type]}]
(or (= :system-message message-type)
(and (not outgoing)
(not (= :user-message message-type)))))
;; any message that comes after this amount of ms will be grouped separately
(def ^:private group-ms 60000)
(defn add-positional-metadata
"Reduce step which adds positional metadata to a message and conditionally
update the previous message with :first-in-group?."
[{:keys [stream last-outgoing-seen]}
{:keys [type message-type from datemark outgoing timestamp] :as message}]
(let [previous-message (peek stream)
;; Was the previous message from a different author or this message
;; comes after x ms
last-in-group? (or (= :system-message message-type)
(not= from (:from previous-message))
(> (- (:timestamp previous-message) timestamp) group-ms))
;; Have we seen an outgoing message already?
last-outgoing? (and (not last-outgoing-seen)
outgoing)
datemark? (= :datemark (:type message))
;; If this is a datemark or this is the last-message of a group,
;; then the previous message was the first
previous-first-in-group? (or datemark?
last-in-group?)
new-message (assoc message
:display-photo? (display-photo? message)
:last-in-group? last-in-group?
:last-outgoing? last-outgoing?)]
{:stream (cond-> stream
previous-first-in-group?
;; update previous message if necessary
set-previous-message-info
:always
(conj new-message))
;; mark the last message sent by the user
:last-outgoing-seen (or last-outgoing-seen last-outgoing?)}))
(defn messages-stream
"Enhances the messages in message sequence interspersed with datemarks
with derived stream context information, like:
`:first-in-group?`, `last-in-group?`, `:last?` and `:last-outgoing?` flags."
[ordered-messages]
(when (seq ordered-messages)
(let [initial-message (first ordered-messages)
message-with-metadata (assoc initial-message
:last-in-group? true
:last? true
:display-photo? (display-photo? initial-message)
:last-outgoing? (:outgoing initial-message))]
(->> (rest ordered-messages)
(reduce add-positional-metadata
{:stream [message-with-metadata]
:last-outgoing-seen (:last-outgoing? message-with-metadata)})
:stream))))
(reverse message-list)))
(def map->sorted-seq
(comp (partial map second) (partial sort-by first)))

32
src/status_im/chat/models.cljs
View File

@ -19,7 +19,6 @@
[status-im.utils.fx :as fx]
[status-im.utils.gfycat.core :as gfycat]
[status-im.utils.platform :as platform]
[status-im.utils.priority-map :refer [empty-message-map]]
[status-im.utils.utils :as utils]
[taoensso.timbre :as log]))
@ -92,18 +91,35 @@
:timestamp now
:contacts #{chat-id}
:last-clock-value 0
:messages empty-message-map}))
:messages {}}))
(fx/defn upsert-chat
"Upsert chat when not deleted"
(fx/defn ensure-chat
"Add chat to db and update"
[{:keys [db] :as cofx} {:keys [chat-id] :as chat-props}]
(let [chat (merge
(or (get (:chats db) chat-id)
(create-new-chat chat-id cofx))
chat-props)]
(fx/merge cofx
{:db (update-in db [:chats chat-id] merge chat)}
(chats-store/save-chat chat))))
{:db (update-in db [:chats chat-id] merge chat)}))
(fx/defn upsert-chat
"Upsert chat when not deleted"
[{:keys [db] :as cofx} {:keys [chat-id] :as chat-props}]
(fx/merge cofx
(ensure-chat chat-props)
#(chats-store/save-chat % (get-in % [:db :chats chat-id]))))
(fx/defn handle-save-chat
{:events [::save-chat]}
[{:keys [db] :as cofx} chat-id]
(chats-store/save-chat cofx (get-in db [:chats chat-id])))
(fx/defn save-chat-delayed
"Debounce saving the chat"
[_ chat-id]
{:dispatch-debounce [{:key :save-chat
:event [::save-chat chat-id]
:delay 500}]})
(fx/defn add-public-chat
"Adds new public group chat to db"
@ -134,7 +150,7 @@
(fx/merge
cofx
{:db (update-in db [:chats chat-id] merge
{:messages empty-message-map
{:messages {}
:message-groups {}
:last-message-content nil
:last-message-content-type nil

78
src/status_im/chat/models/loading.cljs
View File

@ -12,51 +12,9 @@
[status-im.utils.datetime :as time]
[status-im.utils.fx :as fx]
[status-im.utils.priority-map :refer [empty-message-map]]
[status-im.chat.models.message-list :as message-list]
[taoensso.timbre :as log]))
(def index-messages (partial into empty-message-map
(map (juxt :message-id identity))))
(defn- sort-references
"Sorts message-references sequence primary by clock value,
breaking ties by `:message-id`"
[messages message-references]
(sort-by (juxt (comp :clock-value (partial get messages) :message-id)
:message-id)
message-references))
(fx/defn group-chat-messages
"Takes chat-id, new messages + cofx and properly groups them
into the `:message-groups`index in db"
[{:keys [db]} chat-id messages]
{:db (reduce (fn [db [datemark grouped-messages]]
(update-in db [:chats chat-id :message-groups datemark]
(fn [message-references]
(->> grouped-messages
(map (fn [{:keys [message-id timestamp whisper-timestamp]}]
{:message-id message-id
:timestamp-str (time/timestamp->time timestamp)
:timestamp timestamp
:whisper-timestamp whisper-timestamp}))
(into (or message-references '()))
(sort-references (get-in db [:chats chat-id :messages]))))))
db
(group-by (comp time/day-relative :timestamp) messages))})
(defn- get-referenced-ids
"Takes map of `message-id->messages` and returns set of maps of
`{:response-to-v2 message-id}`,
excluding any `message-id` which is already in the original map"
[message-id->messages]
(into #{}
(comp (keep (fn [{:keys [content]}]
(let [response-to-id
(select-keys content [:response-to-v2])]
(when (some (complement nil?) (vals response-to-id))
response-to-id))))
(remove #(some message-id->messages (vals %))))
(vals message-id->messages)))
(fx/defn update-chats-in-app-db
{:events [:chats-list/load-success]}
[{:keys [db] :as cofx} new-chats]
@ -91,28 +49,34 @@
(when-not (or (nil? current-chat-id)
(not= chat-id current-chat-id))
(let [already-loaded-messages (get-in db [:chats current-chat-id :messages])
loaded-unviewed-messages-ids (get-in db [:chats current-chat-id :loaded-unviewed-messages-ids] #{})
;; We remove those messages that are already loaded, as we might get some duplicates
new-messages (remove (comp already-loaded-messages :message-id)
messages)
unviewed-message-ids (reduce
(fn [acc {:keys [seen message-id] :as message}]
(if (not seen)
(conj acc message-id)
acc))
#{}
new-messages)
{:keys [all-messages
new-messages
unviewed-message-ids]} (reduce (fn [{:keys [all-messages] :as acc}
{:keys [seen message-id] :as message}]
(cond-> acc
(not seen)
(update :unviewed-message-ids conj message-id)
indexed-messages (index-messages new-messages)
new-message-ids (keys indexed-messages)]
(nil? (get all-messages message-id))
(update :new-messages conj message)
:always
(update :all-messages assoc message-id message)))
{:all-messages already-loaded-messages
:unviewed-message-ids loaded-unviewed-messages-ids
:new-messages []}
messages)]
(fx/merge cofx
{:db (-> db
(update-in [:chats current-chat-id :loaded-unviewed-messages-ids] clojure.set/union unviewed-message-ids)
(assoc-in [:chats current-chat-id :loaded-unviewed-messages-ids] unviewed-message-ids)
(assoc-in [:chats current-chat-id :messages-initialized?] true)
(update-in [:chats current-chat-id :messages] merge indexed-messages)
(assoc-in [:chats current-chat-id :messages] all-messages)
(update-in [:chats current-chat-id :message-list] message-list/add-many new-messages)
(assoc-in [:chats current-chat-id :cursor] cursor)
(assoc-in [:chats current-chat-id :all-loaded?]
(empty? cursor)))}
(group-chat-messages current-chat-id new-messages)
(chat-model/mark-messages-seen current-chat-id)))))
(fx/defn load-more-messages

221
src/status_im/chat/models/message.cljs
View File

@ -5,6 +5,7 @@
[status-im.chat.db :as chat.db]
[status-im.chat.models :as chat-model]
[status-im.chat.models.loading :as chat-loading]
[status-im.chat.models.message-list :as message-list]
[status-im.chat.models.message-content :as message-content]
[status-im.constants :as constants]
[status-im.contact.db :as contact.db]
@ -51,27 +52,6 @@
(defn system-message? [{:keys [message-type]}]
(= :system-message message-type))
(fx/defn re-index-message-groups
"Relative datemarks of message groups can get obsolete with passing time,
this function re-indexes them for given chat"
[{:keys [db]} chat-id]
(let [chat-messages (get-in db [:chats chat-id :messages])]
{:db (update-in db
[:chats chat-id :message-groups]
(partial reduce-kv (fn [groups datemark message-refs]
(let [new-datemark (->> message-refs
first
:message-id
(get chat-messages)
:timestamp
time/day-relative)]
(if (= datemark new-datemark)
;; nothing to re-index
(assoc groups datemark message-refs)
;; relative datemark shifted, reindex
(assoc groups new-datemark message-refs))))
{}))}))
(defn add-outgoing-status
[{:keys [from outgoing-status] :as message} current-public-key]
(if (and (= from current-public-key)
@ -105,11 +85,15 @@
(fx/defn add-message
[{:keys [db] :as cofx}
{{:keys [chat-id message-id clock-value timestamp from] :as message} :message
:keys [current-chat? batch? metadata raw-message]}]
:keys [current-chat? batch?]}]
(let [current-public-key (multiaccounts.model/current-public-key cofx)
prepared-message (-> message
(prepare-message chat-id current-chat?)
(add-outgoing-status current-public-key))]
(add-outgoing-status current-public-key))
chat-initialized?
(or
current-chat?
(get-in db [:chats chat-id :messages-initialized?]))]
(when (and platform/desktop?
(not= from current-public-key)
(get-in db [:multiaccount :desktop-notifications?])
@ -119,9 +103,12 @@
(fx/merge cofx
{:db (cond->
(-> db
(update-in [:chats chat-id :last-clock-value] (partial utils.clocks/receive clock-value))
;; We should not be always adding to the list, as it does not make sense
;; if the chat has not been initialized, but run into
;; some troubles disabling it, so next time
(update-in [:chats chat-id :messages] assoc message-id prepared-message)
(update-in [:chats chat-id :last-clock-value] (partial utils.clocks/receive clock-value)))
(update-in [:chats chat-id :message-list] message-list/add prepared-message))
(and (not current-chat?)
(not= from current-public-key))
(update-in [:chats chat-id :loaded-unviewed-messages-ids]
@ -130,44 +117,26 @@
(when (and platform/desktop?
(not batch?)
(not (system-message? prepared-message)))
(chat-model/update-dock-badge-label))
(when-not batch?
(re-index-message-groups chat-id))
(when-not batch?
(chat-loading/group-chat-messages chat-id [message])))))
(defn ensure-clock-value [{:keys [clock-value] :as message} {:keys [last-clock-value]}]
(if clock-value
message
(assoc message :clock-value (utils.clocks/send last-clock-value))))
(chat-model/update-dock-badge-label)))))
(fx/defn add-received-message
[{:keys [db] :as cofx}
{:keys [from message-id chat-id js-obj content metadata] :as raw-message}]
{:keys [from message-id chat-id content metadata] :as raw-message}]
(let [{:keys [current-chat-id view-id]} db
current-public-key (multiaccounts.model/current-public-key cofx)
current-chat? (and (or (= :chat view-id)
(= :chat-modal view-id))
(= current-chat-id chat-id))
{:keys [group-chat] :as chat} (get-in db [:chats chat-id])
{:keys [outgoing] :as message} (-> raw-message
(commands-receiving/enhance-receive-parameters cofx)
(ensure-clock-value chat)
;; TODO (cammellos): Refactor so it's not computed twice
(add-outgoing-status current-public-key))]
message (-> raw-message
(commands-receiving/enhance-receive-parameters cofx))]
(fx/merge cofx
(add-message {:batch? true
:message message
:metadata metadata
:current-chat current-chat?
:raw-message js-obj})
:current-chat? current-chat?})
(commands-receiving/receive message))))
(fx/defn update-group-messages [cofx chat->message chat-id]
(fx/merge cofx
(re-index-message-groups chat-id)
(chat-loading/group-chat-messages chat-id (get chat->message chat-id))))
(defn- add-to-chat?
[{:keys [db]} {:keys [chat-id clock-value message-id from]}]
(let [{:keys [deleted-at-clock-value messages]}
@ -175,22 +144,6 @@
(not (or (get messages message-id)
(>= deleted-at-clock-value clock-value)))))
(defn- filter-messages [cofx messages]
(:accumulated
(reduce (fn [{:keys [seen-ids] :as acc}
{:keys [message-id] :as message}]
(if (and (add-to-chat? cofx message)
(not (seen-ids message-id)))
(-> acc
(update :seen-ids conj message-id)
(update :accumulated
(fn [acc]
(update acc :messages conj message))))
acc))
{:seen-ids #{}
:accumulated {:messages []}}
messages)))
(defn extract-chat-id [cofx {:keys [chat-id from message-type]}]
"Validate and return a valid chat-id"
(cond
@ -203,99 +156,64 @@
(= (multiaccounts.model/current-public-key cofx) from)) chat-id
(= :user-message message-type) from))
(defn calculate-unviewed-messages-count
[{:keys [db] :as cofx} chat-id messages]
(defn calculate-unviewed-message-count
[{:keys [db] :as cofx} {:keys [chat-id from]}]
(let [{:keys [current-chat-id view-id]} db
chat-view? (or (= :chat view-id)
(= :chat-modal view-id))
current-public-key (multiaccounts.model/current-public-key cofx)]
(+ (get-in db [:chats chat-id :unviewed-messages-count])
(if (and chat-view? (= current-chat-id chat-id))
0
(count (remove
(fn [{:keys [from]}]
(= from current-public-key))
messages))))))
current-count (get-in db [:chats chat-id :unviewed-messages-count])]
(if (or (and chat-view? (= current-chat-id chat-id))
(= from (multiaccounts.model/current-public-key cofx)))
current-count
(inc current-count))))
(defn- update-last-message [all-chats chat-id]
(let [{:keys [messages message-groups]}
(get all-chats chat-id)
{:keys [content content-type clock-value timestamp]}
(->> (chat.db/sort-message-groups message-groups messages)
last
second
last
:message-id
(get messages))]
(chat-model/upsert-chat
{:chat-id chat-id
:last-message-content content
:last-message-timestamp timestamp
:last-message-content-type content-type})))
(fx/defn update-unviewed-count [{:keys [now db] :as cofx} {:keys [chat-id] :as message}]
{:db (update-in db [:chats chat-id]
assoc
:is-active true
:timestamp now
:unviewed-messages-count (calculate-unviewed-message-count cofx message))})
(fx/defn update-last-messages
[{:keys [db] :as cofx} chat-ids]
(apply fx/merge cofx
(map (partial update-last-message (:chats db)) chat-ids)))
(fx/defn update-last-message [{:keys [db]} {:keys [clock-value chat-id content timestamp content-type]}]
(let [last-chat-clock-value (get-in db [:chats chat-id :last-message-clock-value])]
;; We should also compare message-id in case of clashes, but not sure it's worth
(when (> clock-value last-chat-clock-value)
{:db (update-in db [:chats chat-id]
assoc
:last-message-clock-value clock-value
:last-message-content content
:last-message-timestamp timestamp
:last-message-content-type content-type)})))
(fx/defn declare-syncd-public-chats!
[cofx chat-ids]
(apply fx/merge cofx
(map (partial chat-model/join-time-messages-checked cofx) chat-ids)))
(defn- chat-ids->never-synced-public-chat-ids [chats chat-ids]
(let [never-synced-public-chat-ids (mailserver/chats->never-synced-public-chats chats)]
(when (seq never-synced-public-chat-ids)
(-> never-synced-public-chat-ids
(select-keys (vec chat-ids))
keys))))
(fx/defn receive-many
[{:keys [now] :as cofx} messages]
(let [valid-messages (keep #(when-let [chat-id (extract-chat-id cofx %)]
(assoc % :chat-id chat-id)) messages)
filtered-messages (filter-messages cofx valid-messages)
deduped-messages (:messages filtered-messages)
chat->message (group-by :chat-id deduped-messages)
chat-ids (keys chat->message)
never-synced-public-chat-ids (chat-ids->never-synced-public-chat-ids
(get-in cofx [:db :chats]) chat-ids)
chats-fx-fns (map (fn [chat-id]
(let [unviewed-messages-count
(calculate-unviewed-messages-count
cofx
chat-id
(get chat->message chat-id))]
(chat-model/upsert-chat
{:chat-id chat-id
:is-active true
:timestamp now
:unviewed-messages-count unviewed-messages-count})))
chat-ids)
messages-fx-fns (map add-received-message deduped-messages)
groups-fx-fns (map #(update-group-messages chat->message %) chat-ids)]
(apply fx/merge cofx (concat chats-fx-fns
messages-fx-fns
groups-fx-fns
(when platform/desktop?
[(chat-model/update-dock-badge-label)])
[(update-last-messages chat-ids)]
(when (seq never-synced-public-chat-ids)
[(declare-syncd-public-chats! never-synced-public-chat-ids)])))))
(fx/defn receive-one
[{:keys [now] :as cofx} message]
(when-let [chat-id (extract-chat-id cofx message)]
(let [message-with-chat-id (assoc message :chat-id chat-id)]
(when (add-to-chat? cofx message-with-chat-id)
(fx/merge cofx
(chat-model/ensure-chat {:chat-id chat-id})
(add-received-message message-with-chat-id)
(update-unviewed-count message-with-chat-id)
(chat-model/join-time-messages-checked chat-id)
(update-last-message message-with-chat-id)
(when platform/desktop?
(chat-model/update-dock-badge-label))
;; And save chat
(chat-model/save-chat-delayed chat-id))))))
(defn system-message [{:keys [now] :as cofx} {:keys [clock-value chat-id content from]}]
(let [{:keys [last-clock-value]} (get-in cofx [:db :chats chat-id])
message {:chat-id chat-id
:from from
:timestamp now
:clock-value (or clock-value
(utils.clocks/send last-clock-value))
:content content
:message-type :system-message
:content-type constants/content-type-status}]
message {:chat-id chat-id
:from from
:timestamp now
:whisper-timestamp now
:clock-value (or clock-value
(utils.clocks/send last-clock-value))
:content content
:message-type :system-message
:content-type constants/content-type-status}]
(assoc message
:message-id (transport.utils/system-message-id message)
:raw-payload-hash "system")))
:message-id (transport.utils/system-message-id message))))
(defn group-message? [{:keys [message-type]}]
(#{:group-user-message :public-group-user-message} message-type))
@ -411,7 +329,7 @@
message-data (-> message
(assoc :from (multiaccounts.model/current-public-key cofx)
:timestamp now
:whisper-timestamp (quot now 1000)
:whisper-timestamp now
:clock-value (utils.clocks/send
last-clock-value))
(tribute-to-talk/add-transaction-hash db)
@ -425,10 +343,3 @@
(fx/defn confirm-message-processed
[_ raw-message]
{:transport/confirm-messages-processed [raw-message]})
;; effects
(re-frame.core/reg-fx
:chat-received-message/add-fx
(fn [messages]
(re-frame/dispatch [:message/add messages])))

156
src/status_im/chat/models/message_list.cljs Normal file
View File

@ -0,0 +1,156 @@
(ns status-im.chat.models.message-list
(:require
[status-im.js-dependencies :as dependencies]
[taoensso.timbre :as log]
[status-im.utils.fx :as fx]
[status-im.chat.db :as chat.db]
[status-im.utils.datetime :as time]))
(defn- add-datemark [{:keys [whisper-timestamp] :as msg}]
(assoc msg :datemark (time/day-relative whisper-timestamp)))
(defn- add-timestamp [{:keys [whisper-timestamp] :as msg}]
(assoc msg :timestamp-str (time/timestamp->time whisper-timestamp)))
(defn prepare-message [{:keys [message-id
clock-value
message-type
outgoing
whisper-timestamp]}]
(-> {:whisper-timestamp whisper-timestamp
:one-to-one? (= :user-message message-type)
:system-message? (= :system-message message-type)
:clock-value clock-value
:type :message
:message-id message-id
:outgoing outgoing}
add-datemark
add-timestamp))
;; any message that comes after this amount of ms will be grouped separately
(def ^:private group-ms 60000)
(defn same-group?
"Whether a message is in the same group as the one after it.
We check the time, and the author"
[a b]
(and
(= (:from a) (:from b))
(<= (js/Math.abs (- (:whisper-timestamp a) (:whisper-timestamp b))) group-ms)))
(defn display-photo?
"We display photos for other users, and not in 1-to-1 chats"
[{:keys [system-message? one-to-one?
outgoing message-type]}]
(or system-message?
(and
(not outgoing)
(not one-to-one?))))
(defn compare-fn
"Compare two messages, first compare by clock-value, and break ties by message-id,
which gives us total ordering across all clients"
[a b]
(let [initial-comparison (compare (:clock-value b) (:clock-value a))]
(if (= initial-comparison 0)
(compare (:message-id a) (:message-id b))
initial-comparison)))
(defn add-group-info
"Add positional data to a message, based on the next and previous message.
We divide messages in groups. Messages are sorted descending so :first? is
the most recent message, similarly :first-in-group? is the most recent message
in a group."
[{:keys [one-to-one? outgoing] :as current-message}
{:keys [outgoing-seen?] :as previous-message}
next-message]
(let [last-in-group? (or (nil? next-message)
(not (same-group? current-message next-message)))]
(assoc current-message
:first? (nil? previous-message)
:first-outgoing? (and outgoing
(not outgoing-seen?))
:outgoing-seen? (or outgoing-seen?
outgoing)
:first-in-group? (or (nil? previous-message)
(not (same-group? current-message previous-message)))
:last-in-group? (or (nil? next-message)
(not (same-group? current-message next-message)))
:display-username? (and last-in-group?
(not outgoing)
(not one-to-one?))
:display-photo? (display-photo? current-message))))
(defn update-next-message
"Update next message in the list, we set :first? to false, and check if it
:first-outgoing? state has changed because of the insertion"
[current-message next-message]
(assoc
next-message
:first? false
:first-outgoing? (and
(not (:first-outgoing? current-message))
(:first-outgoing? next-message))
:outgoing-seen? (:outgoing-seen? current-message)
:first-in-group?
(not (same-group? current-message next-message))))
(defn update-previous-message
"If this is a new group, we mark the previous as the last one in the group"
[current-message {:keys [one-to-one? outgoing] :as previous-message}]
(let [last-in-group? (not (same-group? current-message previous-message))]
(assoc previous-message
:display-username? (and last-in-group?
(not outgoing)
(not one-to-one?))
:last-in-group? last-in-group?)))
(defn get-prev-element
"Get previous item in the iterator, and wind it back to the initial state"
[iter]
(.prev iter)
(let [e (.-value iter)]
(.next iter)
e))
(defn get-next-element
"Get next item in the iterator, and wind it back to the initial state"
[iter]
(.next iter)
(let [e (.-value iter)]
(.prev iter)
e))
(defn insert-message
"Insert a message in the list, pull it's left and right messages, calculate
its positional metadata, and update the left & right messages if necessary,
this operation is O(logN) for insertion, and O(logN) for the updates, as
we need to re-find (there's probably a better way)"
[old-message-list {:keys [key] :as prepared-message}]
(let [tree (.insert old-message-list prepared-message prepared-message)
iter (.find tree prepared-message)
previous-message (when (.-hasPrev iter)
(get-prev-element iter))
next-message (when (.-hasNext iter)
(get-next-element iter))
message-with-pos-data (add-group-info prepared-message previous-message next-message)]
(cond->
(.update iter message-with-pos-data)
next-message
(-> (.find next-message)
(.update (update-next-message message-with-pos-data next-message)))
(and previous-message
(not= :datemark (:type previous-message)))
(-> (.find previous-message)
(.update (update-previous-message message-with-pos-data previous-message))))))
(defn add [message-list message]
(insert-message (or message-list (dependencies/rb-tree compare-fn))
(prepare-message message)))
(defn add-many [message-list messages]
(reduce add
message-list
messages))

13
src/status_im/contact/block.cljs
View File

@ -2,6 +2,8 @@
(:require [re-frame.core :as re-frame]
[status-im.chat.models :as chat.models]
[status-im.chat.models.loading :as chat.models.loading]
[status-im.chat.models.message-list :as message-list]
[status-im.chat.models.message :as chat.models.message]
[status-im.contact.db :as contact.db]
[status-im.data-store.chats :as chats-store]
@ -35,19 +37,14 @@
#(apply dissoc % removed-messages-ids))
;; remove message groups
(update-in [:chats chat-id]
dissoc :message-groups)
dissoc :message-list)
(update-in [:chats chat-id]
assoc
:unviewed-messages-count unviewed-messages-count
:last-message-content last-message-content
:last-message-timestamp last-message-timestamp
:last-message-content-type last-message-content-type))]
(fx/merge cofx
{:db db}
;; recompute message group
(chat.models.loading/group-chat-messages
chat-id
(vals (get-in db [:chats chat-id :messages]))))))
{:db (update-in db [:chats chat-id :message-list] message-list/add-many (vals (get-in db [:chats chat-id :messages])))}))
(fx/defn contact-blocked
{:events [::contact-blocked]}
@ -86,4 +83,4 @@
{:db (-> db
(update :contacts/blocked disj public-key)
(assoc-in [:contacts/contacts public-key] contact))}
(contacts-store/save-contact contact))))
(contacts-store/save-contact contact))))

22
src/status_im/data_store/messages.cljs
View File

@ -17,7 +17,7 @@
(defn ->rpc [message]
(-> message
(dissoc :js-obj :dedup-id)
(dissoc :dedup-id)
(update :message-type name)
(update :outgoing-status #(if % (name %) ""))
(utils/update-if-present :content prepare-content)
@ -109,8 +109,24 @@
(fn [messages]
(save-messages-rpc messages)))
(fx/defn save-message [cofx message]
{::save-message [message]})
(fx/defn save-messages [{:keys [db]}]
(when-let [messages (vals (:messages/stored db))]
;; Pull message from database to pick up most recent changes, default to
;; stored one in case it has been offloaded
(let [hydrated-messages (map #(get-in db [:chats (-> % :content :chat-id) :messages (:message-id %)] %) messages)]
{:db (dissoc db :messages/stored)
::save-message hydrated-messages})))
(fx/defn handle-save-messages
{:events [::save-messages]}
[cofx]
(save-messages cofx))
(fx/defn save-message [{:keys [db]} {:keys [message-id] :as message}]
{:db (assoc-in db [:messages/stored message-id] message)
:dispatch-debounce [{:key :save-messages
:event [::save-messages]
:delay 500}]})
(fx/defn delete-message [cofx id]
(delete-message-rpc id))

18
src/status_im/events.cljs
View File

@ -2,6 +2,8 @@
(:require [clojure.string :as string]
[re-frame.core :as re-frame]
[status-im.multiaccounts.core :as multiaccounts]
[status-im.data-store.messages :as data-store.messages]
[status-im.multiaccounts.create.core :as multiaccounts.create]
[status-im.multiaccounts.login.core :as multiaccounts.login]
[status-im.multiaccounts.logout.core :as multiaccounts.logout]
@ -163,7 +165,10 @@
(handlers/register-handler-fx
:multiaccounts.logout.ui/logout-confirmed
(fn [cofx _]
(multiaccounts.logout/logout cofx)))
(fx/merge
cofx
(data-store.messages/save-messages)
(multiaccounts.logout/logout))))
;; multiaccounts update module
@ -631,11 +636,6 @@
(fn [cofx _]
(chat/disable-chat-cooldown cofx)))
(handlers/register-handler-fx
:message/add
(fn [cofx [_ messages]]
(chat.message/receive-many cofx messages)))
(handlers/register-handler-fx
:message/update-message-status
(fn [cofx [_ chat-id message-id status]]
@ -1238,12 +1238,6 @@
;; transport module
(handlers/register-handler-fx
:transport/messages-received
[handlers/logged-in (re-frame/inject-cofx :random-id-generator)]
(fn [cofx [_ js-error js-messages chat-id]]
(transport.message/receive-whisper-messages cofx js-error js-messages chat-id)))
(handlers/register-handler-fx
:transport/send-status-message-error
(fn [{:keys [db] :as cofx} [_ err]]

11
src/status_im/group_chats/core.cljs
View File

@ -470,7 +470,7 @@
[cofx {:keys [chat-id
message
membership-updates] :as membership-update}
{:keys [raw-payload metadata]}
{:keys [whisper-timestamp metadata]}
sender-signature]
(let [dev-mode? (get-in cofx [:db :multiaccount :dev-mode?])]
(when (valid-chat-id? chat-id (extract-creator membership-update))
@ -498,10 +498,11 @@
;; don't allow anything but group messages
(instance? protocol/Message message)
(= :group-user-message (:message-type message)))
(protocol/receive message chat-id sender-signature nil
(assoc %
:metadata metadata
:js-obj #js {:payload raw-payload}))))))))
(protocol/receive message
chat-id
sender-signature
whisper-timestamp
(assoc % :metadata metadata))))))))
(defn handle-sign-success
"Upsert chat and send signed payload to group members"

1
src/status_im/js_dependencies.cljs
View File

@ -6,3 +6,4 @@
(def BigNumber (js/require "bignumber.js"))
(def web3-utils (js/require "web3-utils"))
(def hi-base32 (js/require "hi-base32"))
(def rb-tree (js/require "functional-red-black-tree"))

62
src/status_im/subs.cljs
View File

@ -730,19 +730,42 @@
(fn [chat]
(:public? chat)))
(re-frame/reg-sub
:chats/message-list
:<- [:chats/current-chat]
(fn [chat]
(:message-list chat)))
(re-frame/reg-sub
:chats/messages
:<- [:chats/current-chat]
(fn [chat]
(:messages chat)))
(defn hydrate-messages
"Pull data from messages and add it to the sorted list"
[message-list messages]
(keep #(if (= :message (% :type))
(when-let [message (messages (% :message-id))]
(merge message %))
%)
message-list))
(re-frame/reg-sub
:chats/current-chat-messages-stream
:<- [:chats/current-chat-messages]
:<- [:chats/current-chat-message-groups]
:<- [:chats/message-list]
:<- [:chats/messages]
:<- [:chats/messages-gaps]
:<- [:chats/range]
:<- [:chats/all-loaded?]
:<- [:chats/public?]
(fn [[messages message-groups messages-gaps range all-loaded? public?]]
(-> (chat.db/sort-message-groups message-groups messages)
(chat.db/messages-with-datemarks
messages messages-gaps range all-loaded? public?)
chat.db/messages-stream)))
(fn [[message-list messages messages-gaps range all-loaded? public?]]
(-> (if message-list
(array-seq (.-values message-list))
[])
(chat.db/add-datemarks)
(hydrate-messages messages)
(chat.db/add-gaps messages-gaps range all-loaded? public?))))
(re-frame/reg-sub
:chats/current-chat-intro-status
@ -1585,7 +1608,7 @@
(fn [[contacts current-multiaccount] [_ identity]]
(let [me? (= (:public-key current-multiaccount) identity)]
(if me?
{:ens-name (:name current-multiaccount)
{:ens-name (:preferred-name current-multiaccount)
:alias (gfycat/generate-gfy identity)}
(let [contact (or (contacts identity)
(contact.db/public-key->new-contact identity))]
@ -1594,6 +1617,29 @@
:alias (or (:alias contact)
(gfycat/generate-gfy identity))})))))
(re-frame/reg-sub
:messages/quote-info
:<- [:chats/messages]
:<- [:contacts/contacts]
:<- [:multiaccount]
(fn [[messages contacts current-multiaccount] [_ message-id]]
(when-let [message (get messages message-id)]
(let [identity (:from message)
me? (= (:public-key current-multiaccount) identity)]
(if me?
{:quote {:from identity
:text (get-in message [:content :text])}
:ens-name (:preferred-name current-multiaccount)
:alias (gfycat/generate-gfy identity)}
(let [contact (or (contacts identity)
(contact.db/public-key->new-contact identity))]
{:quote {:from identity
:text (get-in message [:content :text])}
:ens-name (when (:ens-verified contact)
(:name contact))
:alias (or (:alias contact)
(gfycat/generate-gfy identity))}))))))
(re-frame/reg-sub
:contacts/all-contacts-not-in-current-chat
:<- [::query-current-chat-contacts remove]

24
src/status_im/transport/impl/receive.cljs
View File

@ -2,6 +2,7 @@
(:require [status-im.group-chats.core :as group-chats]
[status-im.contact.core :as contact]
[status-im.utils.fx :as fx]
[status-im.chat.models.message :as chat.message]
[status-im.ens.core :as ens]
[status-im.pairing.core :as pairing]
[status-im.transport.message.contact :as transport.contact]
@ -13,9 +14,9 @@
(extend-type transport.group-chat/GroupMembershipUpdate
protocol/StatusMessage
(receive [this _ signature _ {:keys [metadata js-obj] :as cofx}]
(group-chats/handle-membership-update-received cofx this signature {:metadata metadata
:raw-payload (.-payload js-obj)})))
(receive [this _ signature timestamp {:keys [metadata js-obj] :as cofx}]
(group-chats/handle-membership-update-received cofx this signature {:whisper-timestamp timestamp
:metadata metadata})))
(extend-type transport.contact/ContactRequest
protocol/StatusMessage
@ -53,7 +54,16 @@
(extend-type protocol/Message
protocol/StatusMessage
(receive [this chat-id signature timestamp cofx]
(fx/merge cofx
(transport.message/receive-transit-message this chat-id signature timestamp)
(ens/verify-names-from-message this signature))))
(receive [this chat-id signature timestamp {:keys [db] :as cofx}]
(let [message (assoc (into {} this)
:message-id
(get-in cofx [:metadata :messageId])
:chat-id chat-id
:whisper-timestamp (* 1000 timestamp)
:alias (get-in cofx [:metadata :author :alias])
:identicon (get-in cofx [:metadata :author :identicon])
:from signature
:metadata (:metadata cofx))]
(chat.message/receive-one cofx message))))
; disable verification until enabled in status-go
; (ens/verify-names-from-message this signature))))

112
src/status_im/transport/message/core.cljs
View File

@ -3,6 +3,7 @@
(:require [goog.object :as o]
[re-frame.core :as re-frame]
[status-im.chat.models.message :as models.message]
[status-im.utils.handlers :as handlers]
[status-im.ethereum.json-rpc :as json-rpc]
[status-im.ethereum.core :as ethereum]
[status-im.transport.message.contact :as contact]
@ -15,18 +16,13 @@
[taoensso.timbre :as log]
[status-im.ethereum.json-rpc :as json-rpc]))
(defn add-raw-payload
"Add raw payload for id calculation"
[{:keys [message] :as m}]
(assoc m :raw-payload (clj->js message)))
(fx/defn receive-message
"Receive message handles a new status-message.
dedup-id is passed by status-go and is used to deduplicate messages at that layer.
Once a message has been successfuly processed, that id needs to be sent back
in order to stop receiving that message"
[cofx now-in-s filter-chat-id message-js]
(let [blocked-contacts (get-in cofx [:db :contacts/blocked] #{})
[{:keys [db]} now-in-s filter-chat-id message-js]
(let [blocked-contacts (get db :contacts/blocked #{})
payload (.-payload message-js)
timestamp (.-timestamp (.-message message-js))
metadata-js (.-metadata message-js)
@ -45,16 +41,17 @@
(not (blocked-contacts sig)))
(try
(when-let [valid-message (protocol/validate status-message)]
(fx/merge (assoc cofx :js-obj raw-payload :metadata metadata)
#(protocol/receive (assoc valid-message
:metadata metadata)
(or
filter-chat-id
(get-in valid-message [:content :chat-id])
sig)
sig
timestamp
%)))
(protocol/receive
(assoc valid-message
:metadata metadata)
(or
filter-chat-id
(get-in valid-message [:content :chat-id])
sig)
sig
timestamp
{:db db
:metadata metadata}))
(catch :default e nil))))) ; ignore unknown message types
(defn- js-obj->seq [obj]
@ -64,36 +61,42 @@
(aget obj i))
[obj]))
(fx/defn receive-whisper-messages
[{:keys [now] :as cofx} error messages chat-id]
(if (and (not error)
messages)
(let [now-in-s (quot now 1000)
receive-message-fxs (map (fn [message]
(receive-message now-in-s chat-id message))
messages)]
(apply fx/merge cofx receive-message-fxs))
(log/error "Something went wrong" error messages)))
(handlers/register-handler-fx
::process
(fn [cofx [_ messages now-in-s]]
(let [[chat-id message] (first messages)
remaining-messages (rest messages)]
(if (seq remaining-messages)
(assoc
(receive-message cofx now-in-s chat-id message)
;; We dispatch later to let the UI thread handle events, without this
;; it will keep processing events ignoring user input.
:dispatch-later [{:ms 20 :dispatch [::process remaining-messages now-in-s]}])
(receive-message cofx now-in-s chat-id message)))))
(fx/defn receive-messages [cofx event-js]
(let [fxs (keep
(fn [message-specs]
(let [chat (.-chat message-specs)
messages (.-messages message-specs)
error (.-error message-specs)]
(when (seq messages)
(receive-whisper-messages
error
messages
;; For discovery and negotiated filters we don't
;; set a chatID, and we use the signature of the message
;; to indicate which chat it is for
(if (or (.-discovery chat)
(.-negotiated chat))
nil
(.-chatId chat))))))
(.-messages event-js))]
(apply fx/merge cofx fxs)))
(fx/defn receive-messages
"Initialize the ::process event, which will process messages one by one
dispatching later to itself"
[{:keys [now] :as cofx} event-js]
(let [now-in-s (quot now 1000)
events (reduce
(fn [acc message-specs]
(let [chat (.-chat message-specs)
messages (.-messages message-specs)
error (.-error message-specs)
chat-id (if (or (.-discovery chat)
(.-negotiated chat))
nil
(.-chatId chat))]
(if (seq messages)
(reduce (fn [acc m]
(conj acc [chat-id m]))
acc
messages)
acc)))
[]
(.-messages event-js))]
{:dispatch [::process events now-in-s]}))
(fx/defn remove-hash
[{:keys [db] :as cofx} envelope-hash]
@ -173,20 +176,3 @@
:on-success #(log/debug "successfully confirmed messages")
:on-failure #(log/error "failed to confirm messages" %)}))))
(fx/defn receive-transit-message [cofx message chat-id signature timestamp]
(let [received-message-fx {:chat-received-message/add-fx
[(assoc (into {} message)
:message-id
(get-in cofx [:metadata :messageId])
:chat-id chat-id
:whisper-timestamp timestamp
:alias (get-in cofx [:metadata :author :alias])
:identicon (get-in cofx [:metadata :author :identicon])
:from signature
:metadata (:metadata cofx)
:js-obj (:js-obj cofx))]}]
(whitelist/filter-message cofx
received-message-fx
(:message-type message)
(get-in message [:content :tribute-transaction])
signature)))

60
src/status_im/ui/screens/chat/message/message.cljs
View File

@ -41,16 +41,20 @@
(get content :command-ref))
content content-type]])
(defview quoted-message [{:keys [from text]} outgoing current-public-key]
(letsubs [{:keys [ens-name alias]} [:contacts/contact-name-by-identity from]]
[react/view {:style (style/quoted-message-container outgoing)}
[react/view {:style style/quoted-message-author-container}
[vector-icons/tiny-icon :tiny-icons/tiny-reply {:color (if outgoing colors/white-transparent colors/gray)}]
(chat.utils/format-reply-author from alias ens-name current-public-key (partial style/quoted-message-author outgoing))]
(defview quoted-message [message-id {:keys [from text]} outgoing current-public-key]
(letsubs [{:keys [quote
ens-name
alias]}
[:messages/quote-info message-id]]
(when (or quote text)
[react/view {:style (style/quoted-message-container outgoing)}
[react/view {:style style/quoted-message-author-container}
[vector-icons/tiny-icon :tiny-icons/tiny-reply {:color (if outgoing colors/white-transparent colors/gray)}]
(chat.utils/format-reply-author (or from (:from quote)) alias ens-name current-public-key (partial style/quoted-message-author outgoing))]
[react/text {:style (style/quoted-message-text outgoing)
:number-of-lines 5}
text]]))
[react/text {:style (style/quoted-message-text outgoing)
:number-of-lines 5}
(or text (:text quote))]])))
(defview message-content-status [{:keys [content]}]
[react/view style/status-container
@ -63,12 +67,16 @@
(i18n/label (if expanded? :show-less :show-more))])
(defn text-message
[{:keys [chat-id message-id content timestamp-str group-chat outgoing current-public-key expanded?] :as message}]
[{:keys [chat-id message-id content
timestamp-str group-chat outgoing current-public-key expanded?] :as message}]
[message-view message
(let [collapsible? (and (:should-collapse? content) group-chat)]
(let [response-to (or (:response-to content)
(:response-to-v2 content))
collapsible? (and (:should-collapse? content) group-chat)]
[react/view
(when (:response-to content)
[quoted-message (:response-to content) outgoing current-public-key])
(when response-to
[quoted-message response-to (:quoted-message message) outgoing current-public-key])
(apply react/nested-text
(cond-> {:style (style/text-message collapsible? outgoing)
:text-break-strategy :balanced
@ -90,12 +98,14 @@
(defn emoji-message
[{:keys [content current-public-key alias] :as message}]
[message-view message
[react/view {:style (style/style-message-text false)}
(when (:response-to content)
[quoted-message (:response-to content) alias false current-public-key])
[react/text {:style (style/emoji-message message)}
(:text content)]]])
(let [response-to (or (:response-to content)
(:response-to-v2 content))]
[message-view message
[react/view {:style (style/style-message-text false)}
(when response-to
[quoted-message response-to (:quoted-message message) alias false current-public-key])
[react/text {:style (style/emoji-message message)}
(:text content)]]]))
(defmulti message-content (fn [_ message _] (message :content-type)))
@ -169,12 +179,13 @@
(defn message-delivery-status
[{:keys [chat-id message-id outgoing-status
content last-outgoing? message-type] :as message}]
first-outgoing?
content message-type] :as message}]
(when (not= :system-message message-type)
(case outgoing-status
:sending [message-activity-indicator]
:not-sent [message-not-sent-text chat-id message-id]
:sent (when last-outgoing?
:sent (when first-outgoing?
[react/view style/delivery-view
[react/text {:style style/delivery-text}
(i18n/label :t/status-sent)]])
@ -187,9 +198,10 @@
(chat.utils/format-author alias style/message-author-name ens-name)))
(defn message-body
[{:keys [last-in-group?
[{:keys [alias
last-in-group?
first-in-group?
display-photo?
alias
display-username?
from
outgoing
@ -199,7 +211,7 @@
[react/view (style/message-body message)
(when display-photo?
[react/view (style/message-author outgoing)
(when last-in-group?
(when first-in-group?
[react/touchable-highlight {:on-press #(when-not modal? (re-frame/dispatch [:chat.ui/show-profile from]))}
[react/view
[photos/member-photo from]]])])

12
src/status_im/ui/screens/chat/styles/message/message.cljs
View File

@ -10,15 +10,15 @@
{:color (if outgoing colors/white colors/text)})
(defn message-padding-top
[{:keys [first-in-group? display-username?]}]
[{:keys [last-in-group? display-username?]}]
(if (and display-username?
first-in-group?)
last-in-group?)
6
2))
(defn last-message-padding
[{:keys [last? typing]}]
(when (and last? (not typing))
[{:keys [first? typing]}]
(when (and first? (not typing))
{:padding-bottom 16}))
(defn message-body
@ -139,11 +139,11 @@
:margin-top (if incoming-group 4 0)})
(defn message-view
[{:keys [content-type outgoing group-chat first-in-group?]}]
[{:keys [content-type outgoing group-chat last-in-group?]}]
(merge {:padding-vertical 6
:padding-horizontal 12
:border-radius 8
:margin-top (if (and first-in-group?
:margin-top (if (and last-in-group?
(or outgoing
(not group-chat)))
16

11
src/status_im/ui/screens/chat/toolbar_content.cljs
View File

@ -7,17 +7,6 @@
[status-im.utils.datetime :as time])
(:require-macros [status-im.utils.views :refer [defview letsubs]]))
(defn- online-text [contact chat-id]
(if contact
(let [last-online (get contact :last-online)
last-online-date (time/to-date last-online)
now-date (t/now)]
(if (and (pos? last-online)
(<= last-online-date now-date))
(time/time-ago last-online-date)
(i18n/label :t/active-unknown)))
(i18n/label :t/active-unknown)))
(defn- in-progress-text [{:keys [highestBlock currentBlock startBlock]}]
(let [total (- highestBlock startBlock)
ready (- currentBlock startBlock)

4
src/status_im/ui/screens/chat/utils.cljs
View File

@ -46,8 +46,8 @@
:on-press #(re-frame/dispatch [:chat.ui/start-public-chat (subs text 1) {:navigation-reset? true}])})})
(defn- lookup-props [text-chunk message kind]
(let [prop (get styling->prop kind)
prop-fn (get action->prop-fn kind)]
(let [prop (get styling->prop (keyword kind))
prop-fn (get action->prop-fn (keyword kind))]
(if prop-fn (prop-fn text-chunk message) prop)))
(defn render-chunks [render-recipe message]

2
src/status_im/utils/fx.cljs
View File

@ -12,7 +12,7 @@
cofx))
(def ^:private mergeable-keys
#{:chat-received-message/add-fx
#{:dispatch-debounce
:filters/load-filters
:pairing/set-installation-metadata
:status-im.data-store.messages/save-message

6
src/status_im/utils/types.cljs
View File

@ -1,5 +1,7 @@
(ns status-im.utils.types
(:require [cognitect.transit :as transit]))
(:require
[cljs-bean.core :as clj-bean]
[cognitect.transit :as transit]))
(defn to-string [s]
(if (keyword? s)
@ -7,7 +9,7 @@
s))
(defn clj->json [data]
(.stringify js/JSON (clj->js data)))
(.stringify js/JSON (clj-bean/->js data)))
(defn json->clj [json]
(when-not (= json "undefined")

25
src/status_im/utils/utils.cljs
View File

@ -85,6 +85,31 @@
(when address
(get-shortened-address (eip55/address->checksum (ethereum/normalized-address address)))))
;; debounce, taken from https://github.com/johnswanson/re-frame-debounce-fx
; {:dispatch-debounce {:key :search
; :event [:search value]
; :delay 250}}))
(def registered-keys (atom nil))
(defn dispatch-if-not-superceded [{:keys [key delay event time-received]}]
(when (= time-received (get @registered-keys key))
;; no new events on this key!
(re-frame/dispatch event)))
(defn dispatch-debounced [{:keys [delay] :as debounce}]
(js/setTimeout
(fn [] (dispatch-if-not-superceded debounce))
delay))
(re-frame/reg-fx
:dispatch-debounce
(fn dispatch-debounce [debounces]
(doseq [debounce debounces]
(let [ts (.getTime (js/Date.))]
(swap! registered-keys assoc (:key debounce) ts)
(dispatch-debounced (assoc debounce :time-received ts))))))
;; background-timer
(defn set-timeout [cb ms]

380
test/cljs/status_im/test/chat/db.cljs
View File

@ -15,97 +15,31 @@
:chat-id "1"
:name "unchanged"})))))
(deftest message-stream-tests
(testing "messages with no interspersed datemarks"
(let [m1 {:from "1"
:datemark "a"
:outgoing false}
m2 {:from "2"
:datemark "a"
:outgoing true}
m3 {:from "2"
:datemark "a"
:outgoing true}
dm1 {:type :datemark
:value "a"}
messages [m1 m2 m3 dm1]
[actual-m1
actual-m2
actual-m3] (db/messages-stream messages)]
(testing "it marks only the first message as :last?"
(is (:last? actual-m1))
(is (not (:last? actual-m2)))
(is (not (:last? actual-m3))))
(testing "it marks the first outgoing message as :last-outgoing?"
(is (not (:last-outgoing? actual-m1)))
(is (:last-outgoing? actual-m2))
(is (not (:last-outgoing? actual-m3))))
(testing "it marks messages from the same author next to another with :first-in-group?"
(is (:first-in-group? actual-m1))
(is (not (:first-in-group? actual-m2)))
(is (:first-in-group? actual-m3)))
(testing "it marks messages with display-photo? when they are not outgoing and we are in a group chat"
(is (:display-photo? actual-m1))
(is (not (:display-photo? actual-m2)))
(is (not (:display-photo? actual-m3))))
(testing "it marks messages with display-username? when we display the photo and are the first in a group"
(is (:display-username? actual-m1))
(is (not (:display-username? actual-m2)))
(is (not (:display-username? actual-m3))))
(testing "it marks the last message from the same author with :last-in-group?"
(is (:last-in-group? actual-m1))
(is (:last-in-group? actual-m2))
(is (not (:last-in-group? actual-m3))))))
(testing "messages with interspersed datemarks"
(let [m1 {:from "2" ; first & last in group
:timestamp 63000
:outgoing true}
dm1 {:type :datemark
:value "a"}
m2 {:from "2" ; first & last in group as more than 1 minute after previous message
:timestamp 62000
:outgoing false}
m3 {:from "2" ; last in group
:timestamp 1
:outgoing false}
m4 {:from "2" ; first in group
:timestamp 0
:outgoing false}
dm2 {:type :datemark
:value "b"}
messages [m1 dm1 m2 m3 m4 dm2]
[actual-m1
_
actual-m2
actual-m3
actual-m4
_] (db/messages-stream messages)]
(testing "it marks the first outgoing message as :last-outgoing?"
(is (:last-outgoing? actual-m1))
(is (not (:last-outgoing? actual-m2)))
(is (not (:last-outgoing? actual-m3)))
(is (not (:last-outgoing? actual-m4))))
(testing "it sets :first-in-group? after a datemark"
(is (:first-in-group? actual-m1))
(is (:first-in-group? actual-m4)))
(testing "it sets :first-in-group? if more than 60s have passed since last message"
(is (:first-in-group? actual-m2)))
(testing "it sets :last-in-group? after a datemark"
(is (:last-in-group? actual-m1))
(is (:last-in-group? actual-m2))
(is (:last-in-group? actual-m3))
(is (not (:last-in-group? actual-m4))))))
(testing "system-messages"
(let [m1 {:from "system"
:message-type :system-message
:datemark "a"
:outgoing false}
messages [m1]
[actual-m1] (db/messages-stream messages)]
(testing "it does display the photo"
(is (:display-photo? actual-m1))
(testing "it does not display the username"
(is (not (:display-username? actual-m1))))))))
(deftest intersperse-datemarks
(testing "it mantains the order even when timestamps are across days"
(let [message-1 {:datemark "Dec 31, 1999"
:whisper-timestamp 946641600000} ; 1999}
message-2 {:datemark "Jan 1, 2000"
:whisper-timestamp 946728000000} ; 2000 this will displayed in 1999
message-3 {:datemark "Dec 31, 1999"
:whisper-timestamp 946641600000} ; 1999
message-4 {:datemark "Jan 1, 2000"
:whisper-timestamp 946728000000} ; 2000
ordered-messages [message-4
message-3
message-2
message-1]
[m1 d1 m2 m3 m4 d2 :as ms] (db/add-datemarks ordered-messages)]
(is (= "Jan 1, 2000"
(:datemark m1)))
(is (= {:type :datemark
:value "Jan 1, 2000"} d1))
(is (= "Dec 31, 1999"
(:datemark m2)
(:datemark m3)
(:datemark m4)))
(is (= {:type :datemark
:value "Dec 31, 1999"} d2)))))
(deftest active-chats-test
(with-redefs [gfycat/generate-gfy (constantly "generated")
@ -119,227 +53,123 @@
(is (= #{"1" "2"}
(set (keys (db/active-chats {} chats {})))))))))
#_(deftest messages-with-datemarks
(testing "empty state"
(is (empty?
(db/messages-with-datemarks
nil
nil
nil
nil
nil
false
false))))
(testing "empty state pub-chat"
(is (=
[{:type :gap
:value ":first-gap"
:first-gap? true}]
(db/messages-with-datemarks
nil
nil
nil
nil
{:lowest-request-from 10
:highest-request-to 30}
true
true))))
(testing "simple case"
(is (=
'({:whisper-timestamp 40
:timestamp 40
:content nil
:timestamp-str "14:00"
:datemark "today"}
{:whisper-timestamp 30
:timestamp 30
:content nil
:timestamp-str "13:00"
:datemark "today"}
{:value "today"
:type :datemark
:whisper-timestamp 30
:timestamp 30}
{:whisper-timestamp 20
:timestamp 20
:content nil
:timestamp-str "12:00"
:datemark "yesterday"}
{:whisper-timestamp 10
:timestamp 10
:content nil
:timestamp-str "11:00"
:datemark "yesterday"}
{:value "yesterday"
:type :datemark
:whisper-timestamp 10
:timestamp 10})
(db/messages-with-datemarks
{"yesterday"
(list
{:message-id :m1
:timestamp-str "11:00"
:whisper-timestamp 10
:timestamp 10}
{:message-id :m2
:timestamp-str "12:00"
:whisper-timestamp 20
:timestamp 20})
"today"
(list
{:message-id :m3
:timestamp-str "13:00"
:whisper-timestamp 30
:timestamp 30}
{:message-id :m4
:timestamp-str "14:00"
:whisper-timestamp 40
:timestamp 40})}
{:m1 {:whisper-timestamp 10
:timestamp 10}
:m2 {:whisper-timestamp 20
:timestamp 20}
:m3 {:whisper-timestamp 30
:timestamp 30}
:m4 {:whisper-timestamp 40
:timestamp 40}}
nil
nil
nil
nil
nil))))
(testing "simple case with gap"
(is (=
'({:whisper-timestamp 40
:timestamp 40
:content nil
:timestamp-str "14:00"
:datemark "today"}
(deftest add-gaps
(testing "empty state"
(is (empty?
(db/add-gaps
nil
nil
nil
false
false))))
(testing "empty state pub-chat"
(is (=
[{:type :gap
:value ":first-gap"
:first-gap? true}]
(db/add-gaps
nil
nil
{:lowest-request-from 10
:highest-request-to 30}
true
true))))
(testing "simple case with gap"
(is (= '({:whisper-timestamp 40
:message-id :m4
:timestamp 40}
{:type :gap
:value ":gapid1"
:gaps {:ids [:gapid1]}}
{:whisper-timestamp 30
:timestamp 30
:content nil
:timestamp-str "13:00"
:datemark "today"}
:message-id :m3}
{:value "today"
:type :datemark
:whisper-timestamp 30
:timestamp 30}
{:whisper-timestamp 20
:timestamp 20
:content nil
:timestamp-str "12:00"
:datemark "yesterday"}
:message-id :m2}
{:whisper-timestamp 10
:timestamp 10
:content nil
:timestamp-str "11:00"
:datemark "yesterday"}
:message-id :m1}
{:value "yesterday"
:type :datemark
:whisper-timestamp 10
:timestamp 10})
(db/messages-with-datemarks
{"yesterday"
(list
{:message-id :m1
:timestamp-str "11:00"
:whisper-timestamp 10
:timestamp 10}
{:message-id :m2
:timestamp-str "12:00"
:whisper-timestamp 20
:timestamp 20})
"today"
(list
{:message-id :m3
:timestamp-str "13:00"
:whisper-timestamp 30
:timestamp 30}
{:message-id :m4
:timestamp-str "14:00"
:whisper-timestamp 40
:timestamp 40})}
{:m1 {:whisper-timestamp 10
:timestamp 10}
:m2 {:whisper-timestamp 20
:timestamp 20}
:m3 {:whisper-timestamp 30
:timestamp 30}
:m4 {:whisper-timestamp 40
:timestamp 40}}
nil
(db/add-gaps
[{:message-id :m4
:whisper-timestamp 40
:timestamp 40}
{:message-id :m3
:whisper-timestamp 30
:timestamp 30}
{:type :datemark
:value "today"
:whisper-timestamp 30
:timestamp 30}
{:message-id :m2
:whisper-timestamp 20
:timestamp 20}
{:message-id :m1
:whisper-timestamp 10
:timestamp 10}
{:type :datemark
:value "yesterday"
:whisper-timestamp 10
:timestamp 10}]
[{:from 25
:to 30
:id :gapid1}]
nil
nil
nil))))
(testing "simple case with gap after all messages"
(is (=
'({:type :gap
(testing "simple case with gap after all messages"
(is (= '({:type :gap
:value ":gapid1"
:gaps {:ids (:gapid1)}}
{:whisper-timestamp 40
:timestamp 40
:content nil
:timestamp-str "14:00"
:datemark "today"}
:message-id :m4
:timestamp 40}
{:whisper-timestamp 30
:timestamp 30
:content nil
:timestamp-str "13:00"
:datemark "today"}
:message-id :m3
:timestamp 30}
{:value "today"
:type :datemark
:whisper-timestamp 30
:timestamp 30}
{:whisper-timestamp 20
:timestamp 20
:content nil
:timestamp-str "12:00"
:datemark "yesterday"}
:message-id :m2
:timestamp 20}
{:whisper-timestamp 10
:timestamp 10
:content nil
:timestamp-str "11:00"
:datemark "yesterday"}
:message-id :m1
:timestamp 10}
{:value "yesterday"
:type :datemark
:whisper-timestamp 10
:timestamp 10})
(db/messages-with-datemarks
{"yesterday"
(list
{:message-id :m1
:timestamp-str "11:00"
:whisper-timestamp 10
:timestamp 10}
{:message-id :m2
:timestamp-str "12:00"
:whisper-timestamp 20
:timestamp 20})
"today"
(list
{:message-id :m3
:timestamp-str "13:00"
:whisper-timestamp 30
:timestamp 30}
{:message-id :m4
:timestamp-str "14:00"
:whisper-timestamp 40
:timestamp 40})}
{:m1 {:whisper-timestamp 10
:timestamp 10}
:m2 {:whisper-timestamp 20
:timestamp 20}
:m3 {:whisper-timestamp 30
:timestamp 30}
:m4 {:whisper-timestamp 40
:timestamp 40}}
nil
(db/add-gaps
[{:message-id :m4
:whisper-timestamp 40
:timestamp 40}
{:message-id :m3
:whisper-timestamp 30
:timestamp 30}
{:type :datemark
:value "today"
:whisper-timestamp 30
:timestamp 30}
{:message-id :m2
:whisper-timestamp 20
:timestamp 20}
{:message-id :m1
:whisper-timestamp 10
:timestamp 10}
{:type :datemark
:value "yesterday"
:whisper-timestamp 10
:timestamp 10}]
[{:from 100
:to 110
:id :gapid1}]

40
test/cljs/status_im/test/chat/models/loading.cljs
View File

@ -1,40 +0,0 @@
(ns status-im.test.chat.models.loading
(:require [cljs.test :refer-macros [deftest is testing]]
[status-im.chat.models.loading :as loading]))
(deftest group-chat-messages
(let [cofx {:db {:chats {"chat-id" {:messages {0 {:message-id 0
:content "a"
:clock-value 0
:timestamp 0}
1 {:message-id 1
:content "b"
:clock-value 1
:timestamp 1}
2 {:message-id 2
:content "c"
:clock-value 2
:timestamp 2}
3 {:message-id 3
:content "d"
:clock-value 3
:timestamp 3}}}}}}
new-messages [{:message-id 1
:content "b"
:clock-value 1
:timestamp 1}
{:message-id 2
:content "c"
:clock-value 2
:timestamp 2}
{:message-id 3
:content "d"
:clock-value 3
:timestamp 3}]]
(testing "New messages are grouped/sorted correctly"
(is (= '(1 2 3)
(map :message-id
(-> (get-in (loading/group-chat-messages cofx "chat-id" new-messages)
[:db :chats "chat-id" :message-groups])
first
second)))))))

84
test/cljs/status_im/test/chat/models/message.cljs
View File

@ -44,47 +44,50 @@
:current-chat-id "chat-id"
:chats {"chat-id" {:messages {}}}}]
(testing "a message coming from you!"
(let [actual (message/receive-many {:db db}
[{:from "me"
:message-type :user-message
:timestamp 0
:message-id "id"
:chat-id "chat-id"
:content "b"
:clock-value 1}])
(let [actual (message/receive-one {:db db}
{:from "me"
:message-type :user-message
:timestamp 0
:whisper-timestamp 0
:message-id "id"
:chat-id "chat-id"
:content "b"
:clock-value 1})
message (get-in actual [:db :chats "chat-id" :messages "id"])]
(testing "it adds the message"
(is message))
(testing "it marks the message as outgoing"
(is (= true (:outgoing message))))))))
(deftest receive-many-clock-value
(deftest receive-one-clock-value
(let [db {:multiaccount {:public-key "me"}
:view-id :chat
:current-chat-id "chat-id"
:chats {"chat-id" {:last-clock-value 10
:messages {}}}}]
(testing "a message with a higher clock value"
(let [actual (message/receive-many {:db db}
[{:from "chat-id"
:message-type :user-message
:timestamp 0
:message-id "id"
:chat-id "chat-id"
:content "b"
:clock-value 12}])
(let [actual (message/receive-one {:db db}
{:from "chat-id"
:message-type :user-message
:timestamp 0
:whisper-timestamp 0
:message-id "id"
:chat-id "chat-id"
:content "b"
:clock-value 12})
chat-clock-value (get-in actual [:db :chats "chat-id" :last-clock-value])]
(testing "it sets last-clock-value"
(is (= 12 chat-clock-value)))))
(testing "a message with a lower clock value"
(let [actual (message/receive-many {:db db}
[{:from "chat-id"
:message-type :user-message
:timestamp 0
:message-id "id"
:chat-id "chat-id"
:content "b"
:clock-value 2}])
(let [actual (message/receive-one {:db db}
{:from "chat-id"
:message-type :user-message
:timestamp 0
:whisper-timestamp 0
:message-id "id"
:chat-id "chat-id"
:content "b"
:clock-value 2})
chat-clock-value (get-in actual [:db :chats "chat-id" :last-clock-value])]
(testing "it sets last-clock-value"
(is (= 10 chat-clock-value)))))))
@ -101,27 +104,30 @@
:message-type :group-user-message
:message-id "1"
:clock-value 1
:whisper-timestamp 0
:timestamp 0}
bad-chat-id-message {:chat-id "bad-chat-id"
:from "present"
:message-type :group-user-message
:message-id "1"
:clock-value 1
:whisper-timestamp 0
:timestamp 0}
bad-from-message {:chat-id "chat-id"
:from "not-present"
:message-type :group-user-message
:message-id "1"
:clock-value 1
:whisper-timestamp 0
:timestamp 0}]
(testing "a valid message"
(is (get-in (message/receive-many cofx [valid-message]) [:db :chats "chat-id" :messages "1"])))
(is (get-in (message/receive-one cofx valid-message) [:db :chats "chat-id" :messages "1"])))
(testing "a message from someone not in the list of participants"
(is (= cofx (message/receive-many cofx [bad-from-message]))))
(is (not (message/receive-one cofx bad-from-message))))
(testing "a message with non existing chat-id"
(is (= cofx (message/receive-many cofx [bad-chat-id-message]))))
(is (not (message/receive-one cofx bad-chat-id-message))))
(testing "a message from a delete chat"
(is (= cofx-without-member (message/receive-many cofx-without-member [valid-message]))))))
(is (not (message/receive-one cofx-without-member valid-message))))))
(deftest receive-public-chats
(let [cofx {:db {:chats {"chat-id" {:public? true}}
@ -133,17 +139,19 @@
:message-type :public-group-user-message
:message-id "1"
:clock-value 1
:whisper-timestamp 0
:timestamp 0}
bad-chat-id-message {:chat-id "bad-chat-id"
:from "present"
:message-type :public-group-user-message
:message-id "1"
:clock-value 1
:whisper-timestamp 0
:timestamp 0}]
(testing "a valid message"
(is (get-in (message/receive-many cofx [valid-message]) [:db :chats "chat-id" :messages "1"])))
(is (get-in (message/receive-one cofx valid-message) [:db :chats "chat-id" :messages "1"])))
(testing "a message with non existing chat-id"
(is (= cofx (message/receive-many cofx [bad-chat-id-message]))))))
(is (not (message/receive-one cofx bad-chat-id-message))))))
(deftest receive-one-to-one
(with-redefs [gfycat/generate-gfy (constantly "generated")
@ -151,19 +159,21 @@
(let [cofx {:db {:chats {"matching" {}}
:multiaccount {:public-key "me"}
:current-chat-id "chat-id"
:current-chat-id "matching"
:view-id :chat}}
valid-message {:chat-id "matching"
:from "matching"
:message-type :user-message
:message-id "1"
:clock-value 1
:whisper-timestamp 0
:timestamp 0}
own-message {:chat-id "matching"
:from "me"
:message-type :user-message
:message-id "1"
:clock-value 1
:whisper-timestamp 0
:timestamp 0}
bad-chat-id-message {:chat-id "bad-chat-id"
@ -171,29 +181,33 @@
:message-type :user-message
:message-id "1"
:clock-value 1
:whisper-timestamp 0
:timestamp 0}]
(testing "a valid message"
(is (get-in (message/receive-many cofx [valid-message]) [:db :chats "matching" :messages "1"])))
(is (get-in (message/receive-one cofx valid-message) [:db :chats "matching" :messages "1"])))
(testing "our own message"
(is (get-in (message/receive-many cofx [own-message]) [:db :chats "matching" :messages "1"])))
(is (get-in (message/receive-one cofx own-message) [:db :chats "matching" :messages "1"])))
(testing "a message with non matching chat-id"
(is (get-in (message/receive-many cofx [bad-chat-id-message]) [:db :chats "not-matching" :messages "1"]))))))
(is (get-in (message/receive-one cofx bad-chat-id-message) [:db :chats "not-matching" :messages "1"]))))))
(deftest delete-message
(let [timestamp (time/now)
cofx1 {:db {:chats {"chat-id" {:messages {0 {:message-id 0
:content "a"
:clock-value 0
:whisper-timestamp (- timestamp 1)
:timestamp (- timestamp 1)}
1 {:message-id 1
:content "b"
:clock-value 1
:whisper-timestamp timestamp
:timestamp timestamp}}
:message-groups {"datetime-today" '({:message-id 1}
{:message-id 0})}}}}}
cofx2 {:db {:chats {"chat-id" {:messages {0 {:message-id 0
:content "a"
:clock-value 0
:whisper-timestamp timestamp
:timestamp timestamp}}
:message-groups {"datetime-today" '({:message-id 0})}}}}}
fx1 (message/delete-message cofx1 "chat-id" 1)

170
test/cljs/status_im/test/chat/models/message_list.cljs Normal file
View File

@ -0,0 +1,170 @@
(ns status-im.test.chat.models.message-list
(:require [cljs.test :refer-macros [deftest is testing]]
[status-im.constants :as const]
[taoensso.tufte :as tufte :refer-macros (defnp p profiled profile)]
[status-im.chat.models.loading :as l]
[status-im.chat.models.message-list :as s]))
(deftest message-stream-tests
(testing "building the list"
(let [m1 {:from "1"
:clock-value 1
:message-id "message-1"
:whisper-timestamp 1
:outgoing false}
m2 {:from "2"
:clock-value 2
:message-id "message-2"
:whisper-timestamp 2
:outgoing true}
m3 {:from "2"
:clock-value 3
:message-id "message-3"
:whisper-timestamp 3
:outgoing true}
messages (shuffle [m1 m2 m3])
[actual-m1
actual-m2
actual-m3] (vals (s/build messages))]
(testing "it sorts them correclty"
(is (= "message-3" (:message-id actual-m1)))
(is (= "message-2" (:message-id actual-m2)))
(is (= "message-1" (:message-id actual-m3))))
(testing "it marks only the first message as :first?"
(is (:first? actual-m1))
(is (not (:first? actual-m2)))
(is (not (:first? actual-m3))))
(testing "it marks the first outgoing message as :first-outgoing?"
(is (:first-outgoing? actual-m1))
(is (not (:first-outgoing? actual-m2)))
(is (not (:first-outgoing? actual-m3))))
(testing "it marks messages from the same author next to another with :first-in-group?"
(is (:first-in-group? actual-m1))
(is (not (:first-in-group? actual-m2)))
(is (:first-in-group? actual-m3)))
(testing "it marks messages with display-photo? when they are not outgoing and are first-in-group?"
(is (not (:display-photo? actual-m1)))
(is (not (:display-photo? actual-m2)))
(is (:display-photo? actual-m3)))
(testing "it marks the last message from the same author with :last-in-group?"
(is (not (:last-in-group? actual-m1)))
(is (:last-in-group? actual-m2))
(is (:last-in-group? actual-m3))))))
(def ascending-range (mapv
#(let [i (+ 100000 %)]
{:clock-value i
:whisper-timestamp i
:timestamp i
:message-id (str i)})
(range 2000)))
(def descending-range (reverse ascending-range))
(def random-range (shuffle ascending-range))
(defnp build-message-list [messages]
(s/build messages))
(defnp append-to-message-list [l message]
(s/add l message))
(defnp prepend-to-message-list [l message]
(s/add l message))
(defnp insert-close-to-head-message-list [l message]
(s/add l message))
(defnp insert-middle-message-list [l message]
(s/add l message))
(tufte/add-basic-println-handler! {:format-pstats-opts {:columns [:n-calls :mean :min :max :clock :total]
:format-id-fn name}})
(deftest ^:benchmark benchmark-list
(let [messages (sort-by :timestamp (mapv (fn [i] (let [i (+ 100000 i 1)] {:timestamp i :clock-value i :message-id (str i) :whisper-timestamp i})) (range 100)))
built-list (s/build messages)]
(testing "prepending to list"
(profile {} (dotimes [_ 10] (prepend-to-message-list
built-list
{:clock-value 200000
:message-id "200000"
:whisper-timestamp 21
:timestamp 21}))))
(testing "append to list"
(profile {} (dotimes [_ 10] (append-to-message-list
built-list
{:clock-value 100000
:message-id "100000"
:whisper-timestamp 100000
:timestamp 100000}))))
(testing "insert close to head"
(profile {} (dotimes [_ 10] (insert-close-to-head-message-list
built-list
{:clock-value 109970
:message-id "109970"
:whisper-timestamp 1000
:timestamp 1000}))))
(testing "insert into the middle list"
(profile {} (dotimes [_ 10] (insert-middle-message-list
built-list
{:clock-value 105000
:message-id "10500"
:whisper-timestamp 1000
:timestamp 1000}))))))
(deftest message-list
(let [current-messages [{:clock-value 109
:message-id "109"
:timestamp 9
:whisper-timestamp 9}
{:clock-value 106
:message-id "106"
:timestamp 6
:whisper-timestamp 6}
{:clock-value 103
:message-id "103"
:timestamp 3
:whisper-timestamp 3}]
current-list (s/build current-messages)]
(testing "inserting a newer message"
(let [new-message {:timestamp 12
:clock-value 112
:message-id "112"
:whisper-timestamp 12}]
(is (= 112
(-> (s/add current-list new-message)
vals
first
:clock-value)))))
(testing "inserting an older message"
(let [new-message {:timestamp 0
:clock-value 100
:message-id "100"
:whisper-timestamp 0}]
(is (= 100
(-> (s/add current-list new-message)
vals
last
:clock-value)))))
(testing "inserting in the middle of the list"
(let [new-message {:timestamp 7
:clock-value 107
:message-id "107"
:whisper-timestamp 7}]
(is (= 107
(-> (s/add current-list new-message)
vals
(nth 1)
:clock-value)))))
(testing "inserting in the middle of the list, clock-value clash"
(let [new-message {:timestamp 6
:clock-value 106
:message-id "106a"
:whisper-timestamp 6}]
(is (= "106a"
(-> (s/add current-list new-message)
vals
(nth 1)
:message-id)))))))

1
test/cljs/status_im/test/data_store/messages.cljs
View File

@ -13,7 +13,6 @@
:response-to-v2 "id-2"
:text "hta"}
:whisper-timestamp 1
:js-obj {}
:dedup-id "ATIwMTkwODE0YTdkNWZhZGY1N2E0ZDU3MzUxZmJkNDZkZGM1ZTU4ZjRlYzUyYWYyMDA5NTc2NWYyYmIxOTQ2OTM3NGUwNjdiMvEpTIGEjHOTAyqsrN39wST4npnSAv1AR8jJWeubanjkoGIyJooD5RVRnx6ZMt+/JzBOD2hoZzlHQWA0bC6XbdU="
:outgoing-status :sending
:message-type :public-group-user-message

2
test/cljs/status_im/test/runner.cljs
View File

@ -7,7 +7,6 @@
[status-im.test.chat.commands.input]
[status-im.test.chat.db]
[status-im.test.chat.models.input]
[status-im.test.chat.models.loading]
[status-im.test.chat.models.message-content]
[status-im.test.chat.models.message]
[status-im.test.chat.models]
@ -85,7 +84,6 @@
'status-im.test.chat.db
'status-im.test.chat.models
'status-im.test.chat.models.input
'status-im.test.chat.models.loading
'status-im.test.chat.models.message
'status-im.test.chat.models.message-content
'status-im.test.chat.views.photos

10
test/cljs/status_im/test/transport/core.cljs
View File

@ -58,16 +58,6 @@
:pow 0.002631578947368421
:hash "0x220ef9994a4fae64c112b27ed07ef910918159cbe6fcf8ac515ee2bf9a6711a0"}})])
(deftest receive-whisper-messages-test
(testing "an error is reported"
(is (nil? (:chat-received-message/add-fx (message/receive-whisper-messages {:db {}} "error" #js [] nil)))))
(testing "messages is undefined"
(is (nil? (:chat-received-message/add-fx (message/receive-whisper-messages {:db {}} nil js/undefined nil)))))
(testing "happy path"
(let [actual (message/receive-whisper-messages {:db {}} nil messages sig)]
(testing "it add an fx for the message"
(is (:chat-received-message/add-fx actual))))))
(deftest message-envelopes
(let [chat-id "chat-id"
from "from"