status-im
/
status-mobile
mirror of https://github.com/status-im/status-mobile.git
2
0
Fork 0
Code Issues Projects Releases Wiki Activity

Fix message ordering and improve performance rec. messages 

This commit does a few things:

==== Ordering of messages ====

Change the ordering of messages from a mixture of timestamp/clock-value to use
only clock-value.

Datemarks are now not used for sorting anymore, which means that the
order of messages is always causally related (not the case before, as we
were breaking this property by sorting by datemark), but datemark
calculation is unreliable (a reply to a message might have a timestamp <
then the message that is replied to).
So for timestamp calculation we
naively group them ignoring "out-of-order timestamp" messages, although
there's much to improve.
It fixes an issue whereby the user would change their time and the
message will be displayed in the past, although it is still possible to
craft a message with a lower clock value and order it in the past
(there's no way we can prevent this to some extent, but there are ways
to mitigate, but outside the scope of this PR).

==== Performance of receiving messages ====

The app would freeze on pulling messages from a mailserver (100 or so).
This is due to the JS Thread being hogged by CPU calculation, coupled
with the fact that we always tried to process messages all in one go.

This strategy can't scale, and given x is big enough (200,300,1000) the
UI will freeze.

Instead, each message is now processed separately, and we leave a gap
between processing each message for the UI to respond to user input
(otherwise the app freezes again).
Pulling messages will be longer overall, but the app will be usuable
while this happen (albeit it might slow down).
Other strategies are possible (calculate off-db and do a big swap,
avoiding many re-renders etc), but this is the reccommended strategy by
re-frame author (Solving the CPU Hog problem), so sounds like a safe
base point.

The underlying data structure for holding messages was also changed, we
used an immutable Red and Black Tree, same as a sorted map for clojure, but we use
a js library as is twice as performing then clojure sorted map.

We also don't sort messages again each time we receive them O(nlogn), but we
insert them in order O(logn).

Other data structures considered but discarded:
1) Plain vector, but performance prepending/insertion in the middle
(both O(n)) were not great, as not really suited for these operations.

2) Linked list, appealing as append/prepend is O(1), while insertion is
O(n). This is probably acceptable as messages tend to come in order
(from the db, so adding N messages is O(n)), or the network (most of
them prepends, or close to the head), while mailserver would not follow this path.
An implementation of a linked list was built, which performed roughtly the
same as a clojure sorted-map (although faster append/prepend), but not
worth the complexity of having our own implementation.

3) Clojure sorted-map, probably the most versatile, performance were
acceptable, but nowhere near the javascript implementation we decided on

4) Priority map, much slower than a sorted map (twice as slow)

5) Mutable sorted map, js implementation, (bintrees), not explored this very much, but from
just a quick benchmark, performance were much worse that clojure
immutable sorted map

Given that each message is now processed separately, saving the chat /
messages is also debounced to avoid spamming status-go with network
requests. This is a temporary measure for now until that's done directly
in status-go, without having to ping-pong with status-react.

Next steps performance wise is to move stuff to status-go, parsing of
transit, validation, which is heavy, at which point we can re-consider
performance and how to handle messages.

Fixes also an issue with the last message in the chat, we were using the
last message in the chat list, which might not necessarely be the last
message the chat has seen, in case messages were not loaded and a more
recent message is the database (say you fetch historical messages for
1-to-1 A, you don't have any messages in 1-to-1 chat B loaded, you receive an
historical message for chat B, it sets it as last message).

Also use clj beans instead of js->clj for type conversion

Signed-off-by: Andrea Maria Piana <andrea.maria.piana@gmail.com>
This commit is contained in:
Andrea Maria Piana 2019-10-24 16:23:20 +02:00
parent e579412334
commit c69863cda2
No known key found for this signature in database
GPG Key ID: AA6CCA6DE0E06424
34 changed files with 897 additions and 866 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
clj-rn.conf.edn
View File

@ -38,6 +38,7 @@
"react-navigation" "react-navigation"
"react-native-navigation-twopane" "react-native-navigation-twopane"
"hi-base32" "hi-base32"
"functional-red-black-tree"
"react-native-mail" "react-native-mail"
"react-native-shake" "react-native-shake"
"@react-native-community/netinfo"] "@react-native-community/netinfo"]
@ -74,6 +75,7 @@
"react-native-desktop-gesture-handler" "react-native-desktop-gesture-handler"
"web3-utils" "web3-utils"
"react-navigation" "react-navigation"
"functional-red-black-tree"
"react-native-navigation-twopane" "react-native-navigation-twopane"
"hi-base32"] "hi-base32"]

3
desktop/js_files/package.json
View File

@ -15,6 +15,7 @@
"emojilib": "^2.2.9", "emojilib": "^2.2.9",
"eth-phishing-detect": "^1.1.13", "eth-phishing-detect": "^1.1.13",
"events": "^1.1.1", "events": "^1.1.1",
"functional-red-black-tree": "^1.0.1",
"google-breakpad": "git+https://github.com/status-im/google-breakpad.git#v0.9.0", "google-breakpad": "git+https://github.com/status-im/google-breakpad.git#v0.9.0",
"hi-base32": "^0.5.0", "hi-base32": "^0.5.0",
"i18n-js": "^3.1.0", "i18n-js": "^3.1.0",
@ -52,8 +53,8 @@
"@babel/preset-env": "7.1.0", "@babel/preset-env": "7.1.0",
"@babel/register": "7.6.2", "@babel/register": "7.6.2",
"babel-preset-react-native": "^5.0.2", "babel-preset-react-native": "^5.0.2",
"metro-react-native-babel-preset": "^0.45.6",
"coveralls": "^3.0.4", "coveralls": "^3.0.4",
"metro-react-native-babel-preset": "^0.45.6",
"nyc": "^14.1.1", "nyc": "^14.1.1",
"patch-package": "^5.1.1", "patch-package": "^5.1.1",
"rn-snoopy": "git+https://github.com/status-im/rn-snoopy.git#v2.0.2-status" "rn-snoopy": "git+https://github.com/status-im/rn-snoopy.git#v2.0.2-status"

5
desktop/js_files/yarn.lock
View File

@ -3210,6 +3210,11 @@ function-bind@^1.0.2, function-bind@^1.1.1:
resolved "https://registry.yarnpkg.com/function-bind/-/function-bind-1.1.1.tgz#a56899d3ea3c9bab874bb9773b7c5ede92f4895d" resolved "https://registry.yarnpkg.com/function-bind/-/function-bind-1.1.1.tgz#a56899d3ea3c9bab874bb9773b7c5ede92f4895d"
integrity sha512-yIovAzMX49sF8Yl58fSCWJ5svSLuaibPxXQJFLmBObTuCr0Mf1KiPopGM9NiFjiYBCbfaa2Fh6breQ6ANVTI0A== integrity sha512-yIovAzMX49sF8Yl58fSCWJ5svSLuaibPxXQJFLmBObTuCr0Mf1KiPopGM9NiFjiYBCbfaa2Fh6breQ6ANVTI0A==
functional-red-black-tree@^1.0.1:
version "1.0.1"
resolved "https://registry.yarnpkg.com/functional-red-black-tree/-/functional-red-black-tree-1.0.1.tgz#1b0ab3bd553b2a0d6399d29c0e3ea0b252078327"
integrity sha1-GwqzvVU7Kg1jmdKcDj6gslIHgyc=
gauge@~1.2.5: gauge@~1.2.5:
version "1.2.7" version "1.2.7"
resolved "https://registry.yarnpkg.com/gauge/-/gauge-1.2.7.tgz#e9cec5483d3d4ee0ef44b60a7d99e4935e136d93" resolved "https://registry.yarnpkg.com/gauge/-/gauge-1.2.7.tgz#e9cec5483d3d4ee0ef44b60a7d99e4935e136d93"

10
externs.js
View File

@ -575,6 +575,16 @@ var TopLevel = {
"FIRSTWEEKCUTOFFDAY": function () {}, "FIRSTWEEKCUTOFFDAY": function () {},
"decimalPlaces": function () {}, "decimalPlaces": function () {},
"_android": function () {}, "_android": function () {},
"next": function() {},
"prev": function() {},
"hasNext": function() {},
"hasPrev": function() {},
"key": function() {},
"keys": function() {},
"values": function() {},
"find": function() {},
"insert": function() {},
"update": function() {},
"isSupported" : function () {}, "isSupported" : function () {},
"authenticate" : function () {}, "authenticate" : function () {},
"createAppContainer" : function () {}, "createAppContainer" : function () {},

1
mobile/js_files/package.json
View File

@ -13,6 +13,7 @@
"create-react-class": "^15.6.2", "create-react-class": "^15.6.2",
"emojilib": "^2.4.0", "emojilib": "^2.4.0",
"eth-phishing-detect": "^1.1.13", "eth-phishing-detect": "^1.1.13",
"functional-red-black-tree": "^1.0.1",
"hermes-engine": "0.2.1", "hermes-engine": "0.2.1",
"hi-base32": "^0.5.0", "hi-base32": "^0.5.0",
"i18n-js": "^3.3.0", "i18n-js": "^3.3.0",

5
mobile/js_files/yarn.lock
View File

@ -2577,6 +2577,11 @@ function-bind@^1.1.1:
resolved "https://registry.yarnpkg.com/function-bind/-/function-bind-1.1.1.tgz#a56899d3ea3c9bab874bb9773b7c5ede92f4895d" resolved "https://registry.yarnpkg.com/function-bind/-/function-bind-1.1.1.tgz#a56899d3ea3c9bab874bb9773b7c5ede92f4895d"
integrity sha512-yIovAzMX49sF8Yl58fSCWJ5svSLuaibPxXQJFLmBObTuCr0Mf1KiPopGM9NiFjiYBCbfaa2Fh6breQ6ANVTI0A== integrity sha512-yIovAzMX49sF8Yl58fSCWJ5svSLuaibPxXQJFLmBObTuCr0Mf1KiPopGM9NiFjiYBCbfaa2Fh6breQ6ANVTI0A==
functional-red-black-tree@^1.0.1:
version "1.0.1"
resolved "https://registry.yarnpkg.com/functional-red-black-tree/-/functional-red-black-tree-1.0.1.tgz#1b0ab3bd553b2a0d6399d29c0e3ea0b252078327"
integrity sha1-GwqzvVU7Kg1jmdKcDj6gslIHgyc=
gauge@~2.7.3: gauge@~2.7.3:
version "2.7.4" version "2.7.4"
resolved "https://registry.yarnpkg.com/gauge/-/gauge-2.7.4.tgz#2c03405c7538c39d7eb37b317022e325fb018bf7" resolved "https://registry.yarnpkg.com/gauge/-/gauge-2.7.4.tgz#2c03405c7538c39d7eb37b317022e325fb018bf7"

3
project.clj
View File

@ -78,7 +78,8 @@
[day8.re-frame/tracing "0.5.0"] [day8.re-frame/tracing "0.5.0"]
[hawk "0.2.11"]] [hawk "0.2.11"]]
:source-paths ["src" "env/dev" "react-native/src/cljsjs" "components/src" "dev"]}] :source-paths ["src" "env/dev" "react-native/src/cljsjs" "components/src" "dev"]}]
:test {:dependencies [[day8.re-frame/test "0.1.5"]] :test {:dependencies [[com.taoensso/tufte "2.1.0"]
[day8.re-frame/test "0.1.5"]]
:plugins [[lein-doo "0.1.9"]] :plugins [[lein-doo "0.1.9"]]
:cljsbuild {:builds :cljsbuild {:builds
[{:id "test" [{:id "test"

177
src/status_im/chat/db.cljs
View File

@ -53,63 +53,59 @@
{} {}
chats)) chats))
(defn sort-message-groups
"Sorts message groups according to timestamp of first message in group"
[message-groups messages]
(sort-by
(comp :timestamp (partial get messages) :message-id first second)
message-groups))
(defn quoted-message-data
"Selects certain data from quoted message which must be available in the view"
[message-id messages]
(when-let [{:keys [from content]} (get messages message-id)]
{:from from
:text (:text content)}))
(defn add-datemark
[[datemark
message-references]]
(let [{:keys [whisper-timestamp timestamp]} (first message-references)]
(conj message-references
{:value datemark
:type :datemark
:whisper-timestamp whisper-timestamp
:timestamp timestamp})))
(defn datemark? [{:keys [type]}] (defn datemark? [{:keys [type]}]
(= type :datemark)) (= type :datemark))
(defn intersperse-datemark
"Reduce step which expects the input list of messages to be sorted by clock value.
It makes best effort to group them by day.
We cannot sort them by :timestamp, as that represents the clock of the sender
and we have no guarantees on the order.
We naively and arbitrarly group them assuming that out-of-order timestamps
fall in the previous bucket.
A sends M1 to B with timestamp 2000-01-01T00:00:00
B replies M2 with timestamp 1999-12-31-23:59:59
M1 needs to be displayed before M2
so we bucket both in 1999-12-31"
[{:keys [acc last-timestamp last-datemark]} {:keys [whisper-timestamp datemark] :as msg}]
(cond (empty? acc) ; initial element
{:last-timestamp whisper-timestamp
:last-datemark datemark
:acc (conj acc msg)}
(and (not= last-datemark datemark) ; not the same day
(< whisper-timestamp last-timestamp)) ; not out-of-order
{:last-timestamp whisper-timestamp
:last-datemark datemark
:acc (conj acc {:value last-datemark ; intersperse datemark message
:type :datemark}
msg)}
:else
{:last-timestamp (min whisper-timestamp last-timestamp) ; use last datemark
:last-datemark last-datemark
:acc (conj acc (assoc msg :datemark last-datemark))}))
(defn add-datemarks
"Add a datemark in between an ordered seq of messages when two datemarks are not
the same. Ignore messages with out-of-order timestamps"
[messages]
(when (seq messages)
(let [messages-with-datemarks (:acc (reduce intersperse-datemark {:acc []} messages))]
; Append last datemark
(conj messages-with-datemarks {:value (:datemark (peek messages-with-datemarks))
:type :datemark}))))
(defn gap? [{:keys [type]}] (defn gap? [{:keys [type]}]
(= type :gap)) (= type :gap))
(defn transform-message
[messages]
(fn [{:keys [message-id timestamp-str] :as reference}]
(if (or (datemark? reference)
(gap? reference))
reference
(let [{:keys [content quoted-message] :as message} (get messages message-id)
{:keys [response-to response-to-v2]} content
quote (if quoted-message
quoted-message
(some-> (or response-to-v2 response-to)
(quoted-message-data messages)))]
(cond-> (-> message
(update :content dissoc :response-to :response-to-v2)
(assoc :timestamp-str timestamp-str))
;; quoted message reference
quote
(assoc-in [:content :response-to] quote))))))
(defn check-gap (defn check-gap
[gaps previous-message next-message] [gaps previous-message next-message]
(let [previous-timestamp (:whisper-timestamp previous-message) (let [previous-timestamp (:whisper-timestamp previous-message)
next-whisper-timestamp (:whisper-timestamp next-message) next-whisper-timestamp (:whisper-timestamp next-message)
next-timestamp (quot (:timestamp next-message) 1000) next-timestamp (:timestamp next-message)
ignore-next-message? (> (js/Math.abs ignore-next-message? (> (js/Math.abs
(- next-whisper-timestamp next-timestamp)) (- next-whisper-timestamp next-timestamp))
120)] 120000)]
(reduce (reduce
(fn [acc {:keys [from to id]}] (fn [acc {:keys [from to id]}]
(if (and next-message (if (and next-message
@ -137,15 +133,13 @@
:value (clojure.string/join (:ids gaps)) :value (clojure.string/join (:ids gaps))
:gaps gaps})) :gaps gaps}))
(defn messages-with-datemarks (defn add-gaps
"Converts message groups into sequence of messages interspersed with datemarks, "Converts message groups into sequence of messages interspersed with datemarks,
with correct user statuses associated into message" with correct user statuses associated into message"
[message-groups messages messages-gaps [message-list messages-gaps
{:keys [highest-request-to lowest-request-from]} all-loaded? public?] {:keys [highest-request-to lowest-request-from]} all-loaded? public?]
(transduce (transduce
(comp (map identity)
(mapcat add-datemark)
(map (transform-message messages)))
(fn (fn
([] ([]
(let [acc {:messages (list) (let [acc {:messages (list)
@ -162,9 +156,8 @@
:value (str :first-gap) :value (str :first-gap)
:first-gap? true}) :first-gap? true})
acc))) acc)))
([{:keys [messages datemark-reference previous-message gaps]} message] ([{:keys [messages previous-message gaps]} message]
(let [new-datemark? (datemark? message) (let [{:keys [gaps-number gap]}
{:keys [gaps-number gap]}
(check-gap gaps previous-message message) (check-gap gaps previous-message message)
add-gap? (pos? gaps-number)] add-gap? (pos? gaps-number)]
{:messages (cond-> messages {:messages (cond-> messages
@ -173,16 +166,8 @@
(add-gap gap) (add-gap gap)
:always :always
(conj (conj message))
(cond-> message
(not new-datemark?)
(assoc
:datemark
(:value datemark-reference)))))
:previous-message message :previous-message message
:datemark-reference (if new-datemark?
message
datemark-reference)
:gaps (if add-gap? :gaps (if add-gap?
(drop gaps-number gaps) (drop gaps-number gaps)
gaps)})) gaps)}))
@ -190,73 +175,7 @@
(cond-> messages (cond-> messages
(seq gaps) (seq gaps)
(add-gap {:ids (map :id gaps)})))) (add-gap {:ids (map :id gaps)}))))
message-groups)) (reverse message-list)))
(defn- set-previous-message-info [stream]
(let [{:keys [display-photo? message-type] :as previous-message} (peek stream)]
(conj (pop stream) (assoc previous-message
:display-username? (and display-photo?
(not= :system-message message-type))
:first-in-group? true))))
(defn display-photo? [{:keys [outgoing message-type]}]
(or (= :system-message message-type)
(and (not outgoing)
(not (= :user-message message-type)))))
;; any message that comes after this amount of ms will be grouped separately
(def ^:private group-ms 60000)
(defn add-positional-metadata
"Reduce step which adds positional metadata to a message and conditionally
update the previous message with :first-in-group?."
[{:keys [stream last-outgoing-seen]}
{:keys [type message-type from datemark outgoing timestamp] :as message}]
(let [previous-message (peek stream)
;; Was the previous message from a different author or this message
;; comes after x ms
last-in-group? (or (= :system-message message-type)
(not= from (:from previous-message))
(> (- (:timestamp previous-message) timestamp) group-ms))
;; Have we seen an outgoing message already?
last-outgoing? (and (not last-outgoing-seen)
outgoing)
datemark? (= :datemark (:type message))
;; If this is a datemark or this is the last-message of a group,
;; then the previous message was the first
previous-first-in-group? (or datemark?
last-in-group?)
new-message (assoc message
:display-photo? (display-photo? message)
:last-in-group? last-in-group?
:last-outgoing? last-outgoing?)]
{:stream (cond-> stream
previous-first-in-group?
;; update previous message if necessary
set-previous-message-info
:always
(conj new-message))
;; mark the last message sent by the user
:last-outgoing-seen (or last-outgoing-seen last-outgoing?)}))
(defn messages-stream
"Enhances the messages in message sequence interspersed with datemarks
with derived stream context information, like:
`:first-in-group?`, `last-in-group?`, `:last?` and `:last-outgoing?` flags."
[ordered-messages]
(when (seq ordered-messages)
(let [initial-message (first ordered-messages)
message-with-metadata (assoc initial-message
:last-in-group? true
:last? true
:display-photo? (display-photo? initial-message)
:last-outgoing? (:outgoing initial-message))]
(->> (rest ordered-messages)
(reduce add-positional-metadata
{:stream [message-with-metadata]
:last-outgoing-seen (:last-outgoing? message-with-metadata)})
:stream))))
(def map->sorted-seq (def map->sorted-seq
(comp (partial map second) (partial sort-by first))) (comp (partial map second) (partial sort-by first)))

32
src/status_im/chat/models.cljs
View File

@ -19,7 +19,6 @@
[status-im.utils.fx :as fx] [status-im.utils.fx :as fx]
[status-im.utils.gfycat.core :as gfycat] [status-im.utils.gfycat.core :as gfycat]
[status-im.utils.platform :as platform] [status-im.utils.platform :as platform]
[status-im.utils.priority-map :refer [empty-message-map]]
[status-im.utils.utils :as utils] [status-im.utils.utils :as utils]
[taoensso.timbre :as log])) [taoensso.timbre :as log]))
@ -92,18 +91,35 @@
:timestamp now :timestamp now
:contacts #{chat-id} :contacts #{chat-id}
:last-clock-value 0 :last-clock-value 0
:messages empty-message-map})) :messages {}}))
(fx/defn upsert-chat (fx/defn ensure-chat
"Upsert chat when not deleted" "Add chat to db and update"
[{:keys [db] :as cofx} {:keys [chat-id] :as chat-props}] [{:keys [db] :as cofx} {:keys [chat-id] :as chat-props}]
(let [chat (merge (let [chat (merge
(or (get (:chats db) chat-id) (or (get (:chats db) chat-id)
(create-new-chat chat-id cofx)) (create-new-chat chat-id cofx))
chat-props)] chat-props)]
(fx/merge cofx {:db (update-in db [:chats chat-id] merge chat)}))
{:db (update-in db [:chats chat-id] merge chat)}
(chats-store/save-chat chat)))) (fx/defn upsert-chat
"Upsert chat when not deleted"
[{:keys [db] :as cofx} {:keys [chat-id] :as chat-props}]
(fx/merge cofx
(ensure-chat chat-props)
#(chats-store/save-chat % (get-in % [:db :chats chat-id]))))
(fx/defn handle-save-chat
{:events [::save-chat]}
[{:keys [db] :as cofx} chat-id]
(chats-store/save-chat cofx (get-in db [:chats chat-id])))
(fx/defn save-chat-delayed
"Debounce saving the chat"
[_ chat-id]
{:dispatch-debounce [{:key :save-chat
:event [::save-chat chat-id]
:delay 500}]})
(fx/defn add-public-chat (fx/defn add-public-chat
"Adds new public group chat to db" "Adds new public group chat to db"
@ -134,7 +150,7 @@
(fx/merge (fx/merge
cofx cofx
{:db (update-in db [:chats chat-id] merge {:db (update-in db [:chats chat-id] merge
{:messages empty-message-map {:messages {}
:message-groups {} :message-groups {}
:last-message-content nil :last-message-content nil
:last-message-content-type nil :last-message-content-type nil

78
src/status_im/chat/models/loading.cljs
View File

@ -12,51 +12,9 @@
[status-im.utils.datetime :as time] [status-im.utils.datetime :as time]
[status-im.utils.fx :as fx] [status-im.utils.fx :as fx]
[status-im.utils.priority-map :refer [empty-message-map]] [status-im.utils.priority-map :refer [empty-message-map]]
[status-im.chat.models.message-list :as message-list]
[taoensso.timbre :as log])) [taoensso.timbre :as log]))
(def index-messages (partial into empty-message-map
(map (juxt :message-id identity))))
(defn- sort-references
"Sorts message-references sequence primary by clock value,
breaking ties by `:message-id`"
[messages message-references]
(sort-by (juxt (comp :clock-value (partial get messages) :message-id)
:message-id)
message-references))
(fx/defn group-chat-messages
"Takes chat-id, new messages + cofx and properly groups them
into the `:message-groups`index in db"
[{:keys [db]} chat-id messages]
{:db (reduce (fn [db [datemark grouped-messages]]
(update-in db [:chats chat-id :message-groups datemark]
(fn [message-references]
(->> grouped-messages
(map (fn [{:keys [message-id timestamp whisper-timestamp]}]
{:message-id message-id
:timestamp-str (time/timestamp->time timestamp)
:timestamp timestamp
:whisper-timestamp whisper-timestamp}))
(into (or message-references '()))
(sort-references (get-in db [:chats chat-id :messages]))))))
db
(group-by (comp time/day-relative :timestamp) messages))})
(defn- get-referenced-ids
"Takes map of `message-id->messages` and returns set of maps of
`{:response-to-v2 message-id}`,
excluding any `message-id` which is already in the original map"
[message-id->messages]
(into #{}
(comp (keep (fn [{:keys [content]}]
(let [response-to-id
(select-keys content [:response-to-v2])]
(when (some (complement nil?) (vals response-to-id))
response-to-id))))
(remove #(some message-id->messages (vals %))))
(vals message-id->messages)))
(fx/defn update-chats-in-app-db (fx/defn update-chats-in-app-db
{:events [:chats-list/load-success]} {:events [:chats-list/load-success]}
[{:keys [db] :as cofx} new-chats] [{:keys [db] :as cofx} new-chats]
@ -91,28 +49,34 @@
(when-not (or (nil? current-chat-id) (when-not (or (nil? current-chat-id)
(not= chat-id current-chat-id)) (not= chat-id current-chat-id))
(let [already-loaded-messages (get-in db [:chats current-chat-id :messages]) (let [already-loaded-messages (get-in db [:chats current-chat-id :messages])
loaded-unviewed-messages-ids (get-in db [:chats current-chat-id :loaded-unviewed-messages-ids] #{})
;; We remove those messages that are already loaded, as we might get some duplicates ;; We remove those messages that are already loaded, as we might get some duplicates
new-messages (remove (comp already-loaded-messages :message-id) {:keys [all-messages
messages) new-messages
unviewed-message-ids (reduce unviewed-message-ids]} (reduce (fn [{:keys [all-messages] :as acc}
(fn [acc {:keys [seen message-id] :as message}] {:keys [seen message-id] :as message}]
(if (not seen) (cond-> acc
(conj acc message-id) (not seen)
acc)) (update :unviewed-message-ids conj message-id)
#{}
new-messages)
indexed-messages (index-messages new-messages) (nil? (get all-messages message-id))
new-message-ids (keys indexed-messages)] (update :new-messages conj message)
:always
(update :all-messages assoc message-id message)))
{:all-messages already-loaded-messages
:unviewed-message-ids loaded-unviewed-messages-ids
:new-messages []}
messages)]
(fx/merge cofx (fx/merge cofx
{:db (-> db {:db (-> db
(update-in [:chats current-chat-id :loaded-unviewed-messages-ids] clojure.set/union unviewed-message-ids) (assoc-in [:chats current-chat-id :loaded-unviewed-messages-ids] unviewed-message-ids)
(assoc-in [:chats current-chat-id :messages-initialized?] true) (assoc-in [:chats current-chat-id :messages-initialized?] true)
(update-in [:chats current-chat-id :messages] merge indexed-messages) (assoc-in [:chats current-chat-id :messages] all-messages)
(update-in [:chats current-chat-id :message-list] message-list/add-many new-messages)
(assoc-in [:chats current-chat-id :cursor] cursor) (assoc-in [:chats current-chat-id :cursor] cursor)
(assoc-in [:chats current-chat-id :all-loaded?] (assoc-in [:chats current-chat-id :all-loaded?]
(empty? cursor)))} (empty? cursor)))}
(group-chat-messages current-chat-id new-messages)
(chat-model/mark-messages-seen current-chat-id))))) (chat-model/mark-messages-seen current-chat-id)))))
(fx/defn load-more-messages (fx/defn load-more-messages

221
src/status_im/chat/models/message.cljs
View File

@ -5,6 +5,7 @@
[status-im.chat.db :as chat.db] [status-im.chat.db :as chat.db]
[status-im.chat.models :as chat-model] [status-im.chat.models :as chat-model]
[status-im.chat.models.loading :as chat-loading] [status-im.chat.models.loading :as chat-loading]
[status-im.chat.models.message-list :as message-list]
[status-im.chat.models.message-content :as message-content] [status-im.chat.models.message-content :as message-content]
[status-im.constants :as constants] [status-im.constants :as constants]
[status-im.contact.db :as contact.db] [status-im.contact.db :as contact.db]
@ -51,27 +52,6 @@
(defn system-message? [{:keys [message-type]}] (defn system-message? [{:keys [message-type]}]
(= :system-message message-type)) (= :system-message message-type))
(fx/defn re-index-message-groups
"Relative datemarks of message groups can get obsolete with passing time,
this function re-indexes them for given chat"
[{:keys [db]} chat-id]
(let [chat-messages (get-in db [:chats chat-id :messages])]
{:db (update-in db
[:chats chat-id :message-groups]
(partial reduce-kv (fn [groups datemark message-refs]
(let [new-datemark (->> message-refs
first
:message-id
(get chat-messages)
:timestamp
time/day-relative)]
(if (= datemark new-datemark)
;; nothing to re-index
(assoc groups datemark message-refs)
;; relative datemark shifted, reindex
(assoc groups new-datemark message-refs))))
{}))}))
(defn add-outgoing-status (defn add-outgoing-status
[{:keys [from outgoing-status] :as message} current-public-key] [{:keys [from outgoing-status] :as message} current-public-key]
(if (and (= from current-public-key) (if (and (= from current-public-key)
@ -105,11 +85,15 @@
(fx/defn add-message (fx/defn add-message
[{:keys [db] :as cofx} [{:keys [db] :as cofx}
{{:keys [chat-id message-id clock-value timestamp from] :as message} :message {{:keys [chat-id message-id clock-value timestamp from] :as message} :message
:keys [current-chat? batch? metadata raw-message]}] :keys [current-chat? batch?]}]
(let [current-public-key (multiaccounts.model/current-public-key cofx) (let [current-public-key (multiaccounts.model/current-public-key cofx)
prepared-message (-> message prepared-message (-> message
(prepare-message chat-id current-chat?) (prepare-message chat-id current-chat?)
(add-outgoing-status current-public-key))] (add-outgoing-status current-public-key))
chat-initialized?
(or
current-chat?
(get-in db [:chats chat-id :messages-initialized?]))]
(when (and platform/desktop? (when (and platform/desktop?
(not= from current-public-key) (not= from current-public-key)
(get-in db [:multiaccount :desktop-notifications?]) (get-in db [:multiaccount :desktop-notifications?])
@ -119,9 +103,12 @@
(fx/merge cofx (fx/merge cofx
{:db (cond-> {:db (cond->
(-> db (-> db
(update-in [:chats chat-id :last-clock-value] (partial utils.clocks/receive clock-value))
;; We should not be always adding to the list, as it does not make sense
;; if the chat has not been initialized, but run into
;; some troubles disabling it, so next time
(update-in [:chats chat-id :messages] assoc message-id prepared-message) (update-in [:chats chat-id :messages] assoc message-id prepared-message)
(update-in [:chats chat-id :last-clock-value] (partial utils.clocks/receive clock-value))) (update-in [:chats chat-id :message-list] message-list/add prepared-message))
(and (not current-chat?) (and (not current-chat?)
(not= from current-public-key)) (not= from current-public-key))
(update-in [:chats chat-id :loaded-unviewed-messages-ids] (update-in [:chats chat-id :loaded-unviewed-messages-ids]
@ -130,44 +117,26 @@
(when (and platform/desktop? (when (and platform/desktop?
(not batch?) (not batch?)
(not (system-message? prepared-message))) (not (system-message? prepared-message)))
(chat-model/update-dock-badge-label))
(when-not batch?
(re-index-message-groups chat-id))
(when-not batch?
(chat-loading/group-chat-messages chat-id [message])))))
(defn ensure-clock-value [{:keys [clock-value] :as message} {:keys [last-clock-value]}] (chat-model/update-dock-badge-label)))))
(if clock-value
message
(assoc message :clock-value (utils.clocks/send last-clock-value))))
(fx/defn add-received-message (fx/defn add-received-message
[{:keys [db] :as cofx} [{:keys [db] :as cofx}
{:keys [from message-id chat-id js-obj content metadata] :as raw-message}] {:keys [from message-id chat-id content metadata] :as raw-message}]
(let [{:keys [current-chat-id view-id]} db (let [{:keys [current-chat-id view-id]} db
current-public-key (multiaccounts.model/current-public-key cofx) current-public-key (multiaccounts.model/current-public-key cofx)
current-chat? (and (or (= :chat view-id) current-chat? (and (or (= :chat view-id)
(= :chat-modal view-id)) (= :chat-modal view-id))
(= current-chat-id chat-id)) (= current-chat-id chat-id))
{:keys [group-chat] :as chat} (get-in db [:chats chat-id]) message (-> raw-message
{:keys [outgoing] :as message} (-> raw-message (commands-receiving/enhance-receive-parameters cofx))]
(commands-receiving/enhance-receive-parameters cofx)
(ensure-clock-value chat)
;; TODO (cammellos): Refactor so it's not computed twice
(add-outgoing-status current-public-key))]
(fx/merge cofx (fx/merge cofx
(add-message {:batch? true (add-message {:batch? true
:message message :message message
:metadata metadata :metadata metadata
:current-chat current-chat? :current-chat? current-chat?})
:raw-message js-obj})
(commands-receiving/receive message)))) (commands-receiving/receive message))))
(fx/defn update-group-messages [cofx chat->message chat-id]
(fx/merge cofx
(re-index-message-groups chat-id)
(chat-loading/group-chat-messages chat-id (get chat->message chat-id))))
(defn- add-to-chat? (defn- add-to-chat?
[{:keys [db]} {:keys [chat-id clock-value message-id from]}] [{:keys [db]} {:keys [chat-id clock-value message-id from]}]
(let [{:keys [deleted-at-clock-value messages]} (let [{:keys [deleted-at-clock-value messages]}
@ -175,22 +144,6 @@
(not (or (get messages message-id) (not (or (get messages message-id)
(>= deleted-at-clock-value clock-value))))) (>= deleted-at-clock-value clock-value)))))
(defn- filter-messages [cofx messages]
(:accumulated
(reduce (fn [{:keys [seen-ids] :as acc}
{:keys [message-id] :as message}]
(if (and (add-to-chat? cofx message)
(not (seen-ids message-id)))
(-> acc
(update :seen-ids conj message-id)
(update :accumulated
(fn [acc]
(update acc :messages conj message))))
acc))
{:seen-ids #{}
:accumulated {:messages []}}
messages)))
(defn extract-chat-id [cofx {:keys [chat-id from message-type]}] (defn extract-chat-id [cofx {:keys [chat-id from message-type]}]
"Validate and return a valid chat-id" "Validate and return a valid chat-id"
(cond (cond
@ -203,99 +156,64 @@
(= (multiaccounts.model/current-public-key cofx) from)) chat-id (= (multiaccounts.model/current-public-key cofx) from)) chat-id
(= :user-message message-type) from)) (= :user-message message-type) from))
(defn calculate-unviewed-messages-count (defn calculate-unviewed-message-count
[{:keys [db] :as cofx} chat-id messages] [{:keys [db] :as cofx} {:keys [chat-id from]}]
(let [{:keys [current-chat-id view-id]} db (let [{:keys [current-chat-id view-id]} db
chat-view? (or (= :chat view-id) chat-view? (or (= :chat view-id)
(= :chat-modal view-id)) (= :chat-modal view-id))
current-public-key (multiaccounts.model/current-public-key cofx)] current-count (get-in db [:chats chat-id :unviewed-messages-count])]
(+ (get-in db [:chats chat-id :unviewed-messages-count]) (if (or (and chat-view? (= current-chat-id chat-id))
(if (and chat-view? (= current-chat-id chat-id)) (= from (multiaccounts.model/current-public-key cofx)))
0 current-count
(count (remove (inc current-count))))
(fn [{:keys [from]}]
(= from current-public-key))
messages))))))
(defn- update-last-message [all-chats chat-id] (fx/defn update-unviewed-count [{:keys [now db] :as cofx} {:keys [chat-id] :as message}]
(let [{:keys [messages message-groups]} {:db (update-in db [:chats chat-id]
(get all-chats chat-id) assoc
{:keys [content content-type clock-value timestamp]} :is-active true
(->> (chat.db/sort-message-groups message-groups messages) :timestamp now
last :unviewed-messages-count (calculate-unviewed-message-count cofx message))})
second
last
:message-id
(get messages))]
(chat-model/upsert-chat
{:chat-id chat-id
:last-message-content content
:last-message-timestamp timestamp
:last-message-content-type content-type})))
(fx/defn update-last-messages (fx/defn update-last-message [{:keys [db]} {:keys [clock-value chat-id content timestamp content-type]}]
[{:keys [db] :as cofx} chat-ids] (let [last-chat-clock-value (get-in db [:chats chat-id :last-message-clock-value])]
(apply fx/merge cofx ;; We should also compare message-id in case of clashes, but not sure it's worth
(map (partial update-last-message (:chats db)) chat-ids))) (when (> clock-value last-chat-clock-value)
{:db (update-in db [:chats chat-id]
assoc
:last-message-clock-value clock-value
:last-message-content content
:last-message-timestamp timestamp
:last-message-content-type content-type)})))
(fx/defn declare-syncd-public-chats! (fx/defn receive-one
[cofx chat-ids] [{:keys [now] :as cofx} message]
(apply fx/merge cofx (when-let [chat-id (extract-chat-id cofx message)]
(map (partial chat-model/join-time-messages-checked cofx) chat-ids))) (let [message-with-chat-id (assoc message :chat-id chat-id)]
(when (add-to-chat? cofx message-with-chat-id)
(defn- chat-ids->never-synced-public-chat-ids [chats chat-ids] (fx/merge cofx
(let [never-synced-public-chat-ids (mailserver/chats->never-synced-public-chats chats)] (chat-model/ensure-chat {:chat-id chat-id})
(when (seq never-synced-public-chat-ids) (add-received-message message-with-chat-id)
(-> never-synced-public-chat-ids (update-unviewed-count message-with-chat-id)
(select-keys (vec chat-ids)) (chat-model/join-time-messages-checked chat-id)
keys)))) (update-last-message message-with-chat-id)
(when platform/desktop?
(fx/defn receive-many (chat-model/update-dock-badge-label))
[{:keys [now] :as cofx} messages] ;; And save chat
(let [valid-messages (keep #(when-let [chat-id (extract-chat-id cofx %)] (chat-model/save-chat-delayed chat-id))))))
(assoc % :chat-id chat-id)) messages)
filtered-messages (filter-messages cofx valid-messages)
deduped-messages (:messages filtered-messages)
chat->message (group-by :chat-id deduped-messages)
chat-ids (keys chat->message)
never-synced-public-chat-ids (chat-ids->never-synced-public-chat-ids
(get-in cofx [:db :chats]) chat-ids)
chats-fx-fns (map (fn [chat-id]
(let [unviewed-messages-count
(calculate-unviewed-messages-count
cofx
chat-id
(get chat->message chat-id))]
(chat-model/upsert-chat
{:chat-id chat-id
:is-active true
:timestamp now
:unviewed-messages-count unviewed-messages-count})))
chat-ids)
messages-fx-fns (map add-received-message deduped-messages)
groups-fx-fns (map #(update-group-messages chat->message %) chat-ids)]
(apply fx/merge cofx (concat chats-fx-fns
messages-fx-fns
groups-fx-fns
(when platform/desktop?
[(chat-model/update-dock-badge-label)])
[(update-last-messages chat-ids)]
(when (seq never-synced-public-chat-ids)
[(declare-syncd-public-chats! never-synced-public-chat-ids)])))))
(defn system-message [{:keys [now] :as cofx} {:keys [clock-value chat-id content from]}] (defn system-message [{:keys [now] :as cofx} {:keys [clock-value chat-id content from]}]
(let [{:keys [last-clock-value]} (get-in cofx [:db :chats chat-id]) (let [{:keys [last-clock-value]} (get-in cofx [:db :chats chat-id])
message {:chat-id chat-id message {:chat-id chat-id
:from from :from from
:timestamp now :timestamp now
:clock-value (or clock-value :whisper-timestamp now
(utils.clocks/send last-clock-value)) :clock-value (or clock-value
:content content (utils.clocks/send last-clock-value))
:message-type :system-message :content content
:content-type constants/content-type-status}] :message-type :system-message
:content-type constants/content-type-status}]
(assoc message (assoc message
:message-id (transport.utils/system-message-id message) :message-id (transport.utils/system-message-id message))))
:raw-payload-hash "system")))
(defn group-message? [{:keys [message-type]}] (defn group-message? [{:keys [message-type]}]
(#{:group-user-message :public-group-user-message} message-type)) (#{:group-user-message :public-group-user-message} message-type))
@ -411,7 +329,7 @@
message-data (-> message message-data (-> message
(assoc :from (multiaccounts.model/current-public-key cofx) (assoc :from (multiaccounts.model/current-public-key cofx)
:timestamp now :timestamp now
:whisper-timestamp (quot now 1000) :whisper-timestamp now
:clock-value (utils.clocks/send :clock-value (utils.clocks/send
last-clock-value)) last-clock-value))
(tribute-to-talk/add-transaction-hash db) (tribute-to-talk/add-transaction-hash db)
@ -425,10 +343,3 @@
(fx/defn confirm-message-processed (fx/defn confirm-message-processed
[_ raw-message] [_ raw-message]
{:transport/confirm-messages-processed [raw-message]}) {:transport/confirm-messages-processed [raw-message]})
;; effects
(re-frame.core/reg-fx
:chat-received-message/add-fx
(fn [messages]
(re-frame/dispatch [:message/add messages])))

156
src/status_im/chat/models/message_list.cljs Normal file
View File

@ -0,0 +1,156 @@
(ns status-im.chat.models.message-list
(:require
[status-im.js-dependencies :as dependencies]
[taoensso.timbre :as log]
[status-im.utils.fx :as fx]
[status-im.chat.db :as chat.db]
[status-im.utils.datetime :as time]))
(defn- add-datemark [{:keys [whisper-timestamp] :as msg}]
(assoc msg :datemark (time/day-relative whisper-timestamp)))
(defn- add-timestamp [{:keys [whisper-timestamp] :as msg}]
(assoc msg :timestamp-str (time/timestamp->time whisper-timestamp)))
(defn prepare-message [{:keys [message-id
clock-value
message-type
outgoing
whisper-timestamp]}]
(-> {:whisper-timestamp whisper-timestamp
:one-to-one? (= :user-message message-type)
:system-message? (= :system-message message-type)
:clock-value clock-value
:type :message
:message-id message-id
:outgoing outgoing}
add-datemark
add-timestamp))
;; any message that comes after this amount of ms will be grouped separately
(def ^:private group-ms 60000)
(defn same-group?
"Whether a message is in the same group as the one after it.
We check the time, and the author"
[a b]
(and
(= (:from a) (:from b))
(<= (js/Math.abs (- (:whisper-timestamp a) (:whisper-timestamp b))) group-ms)))
(defn display-photo?
"We display photos for other users, and not in 1-to-1 chats"
[{:keys [system-message? one-to-one?
outgoing message-type]}]
(or system-message?
(and
(not outgoing)
(not one-to-one?))))
(defn compare-fn
"Compare two messages, first compare by clock-value, and break ties by message-id,
which gives us total ordering across all clients"
[a b]
(let [initial-comparison (compare (:clock-value b) (:clock-value a))]
(if (= initial-comparison 0)
(compare (:message-id a) (:message-id b))
initial-comparison)))
(defn add-group-info
"Add positional data to a message, based on the next and previous message.
We divide messages in groups. Messages are sorted descending so :first? is
the most recent message, similarly :first-in-group? is the most recent message
in a group."
[{:keys [one-to-one? outgoing] :as current-message}
{:keys [outgoing-seen?] :as previous-message}
next-message]
(let [last-in-group? (or (nil? next-message)
(not (same-group? current-message next-message)))]
(assoc current-message
:first? (nil? previous-message)
:first-outgoing? (and outgoing
(not outgoing-seen?))
:outgoing-seen? (or outgoing-seen?
outgoing)
:first-in-group? (or (nil? previous-message)
(not (same-group? current-message previous-message)))
:last-in-group? (or (nil? next-message)
(not (same-group? current-message next-message)))
:display-username? (and last-in-group?
(not outgoing)
(not one-to-one?))
:display-photo? (display-photo? current-message))))
(defn update-next-message
"Update next message in the list, we set :first? to false, and check if it
:first-outgoing? state has changed because of the insertion"
[current-message next-message]
(assoc
next-message
:first? false
:first-outgoing? (and
(not (:first-outgoing? current-message))
(:first-outgoing? next-message))
:outgoing-seen? (:outgoing-seen? current-message)
:first-in-group?
(not (same-group? current-message next-message))))
(defn update-previous-message
"If this is a new group, we mark the previous as the last one in the group"
[current-message {:keys [one-to-one? outgoing] :as previous-message}]
(let [last-in-group? (not (same-group? current-message previous-message))]
(assoc previous-message
:display-username? (and last-in-group?
(not outgoing)
(not one-to-one?))
:last-in-group? last-in-group?)))
(defn get-prev-element
"Get previous item in the iterator, and wind it back to the initial state"
[iter]
(.prev iter)
(let [e (.-value iter)]
(.next iter)
e))
(defn get-next-element
"Get next item in the iterator, and wind it back to the initial state"
[iter]
(.next iter)
(let [e (.-value iter)]
(.prev iter)
e))
(defn insert-message
"Insert a message in the list, pull it's left and right messages, calculate
its positional metadata, and update the left & right messages if necessary,
this operation is O(logN) for insertion, and O(logN) for the updates, as
we need to re-find (there's probably a better way)"
[old-message-list {:keys [key] :as prepared-message}]
(let [tree (.insert old-message-list prepared-message prepared-message)
iter (.find tree prepared-message)
previous-message (when (.-hasPrev iter)
(get-prev-element iter))
next-message (when (.-hasNext iter)
(get-next-element iter))
message-with-pos-data (add-group-info prepared-message previous-message next-message)]
(cond->
(.update iter message-with-pos-data)
next-message
(-> (.find next-message)
(.update (update-next-message message-with-pos-data next-message)))
(and previous-message
(not= :datemark (:type previous-message)))
(-> (.find previous-message)
(.update (update-previous-message message-with-pos-data previous-message))))))
(defn add [message-list message]
(insert-message (or message-list (dependencies/rb-tree compare-fn))
(prepare-message message)))
(defn add-many [message-list messages]
(reduce add
message-list
messages))

11
src/status_im/contact/block.cljs
View File

@ -2,6 +2,8 @@
(:require [re-frame.core :as re-frame] (:require [re-frame.core :as re-frame]
[status-im.chat.models :as chat.models] [status-im.chat.models :as chat.models]
[status-im.chat.models.loading :as chat.models.loading] [status-im.chat.models.loading :as chat.models.loading]
[status-im.chat.models.message-list :as message-list]
[status-im.chat.models.message :as chat.models.message] [status-im.chat.models.message :as chat.models.message]
[status-im.contact.db :as contact.db] [status-im.contact.db :as contact.db]
[status-im.data-store.chats :as chats-store] [status-im.data-store.chats :as chats-store]
@ -35,19 +37,14 @@
#(apply dissoc % removed-messages-ids)) #(apply dissoc % removed-messages-ids))
;; remove message groups ;; remove message groups
(update-in [:chats chat-id] (update-in [:chats chat-id]
dissoc :message-groups) dissoc :message-list)
(update-in [:chats chat-id] (update-in [:chats chat-id]
assoc assoc
:unviewed-messages-count unviewed-messages-count :unviewed-messages-count unviewed-messages-count
:last-message-content last-message-content :last-message-content last-message-content
:last-message-timestamp last-message-timestamp :last-message-timestamp last-message-timestamp
:last-message-content-type last-message-content-type))] :last-message-content-type last-message-content-type))]
(fx/merge cofx {:db (update-in db [:chats chat-id :message-list] message-list/add-many (vals (get-in db [:chats chat-id :messages])))}))
{:db db}
;; recompute message group
(chat.models.loading/group-chat-messages
chat-id
(vals (get-in db [:chats chat-id :messages]))))))
(fx/defn contact-blocked (fx/defn contact-blocked
{:events [::contact-blocked]} {:events [::contact-blocked]}

22
src/status_im/data_store/messages.cljs
View File

@ -17,7 +17,7 @@
(defn ->rpc [message] (defn ->rpc [message]
(-> message (-> message
(dissoc :js-obj :dedup-id) (dissoc :dedup-id)
(update :message-type name) (update :message-type name)
(update :outgoing-status #(if % (name %) "")) (update :outgoing-status #(if % (name %) ""))
(utils/update-if-present :content prepare-content) (utils/update-if-present :content prepare-content)
@ -109,8 +109,24 @@
(fn [messages] (fn [messages]
(save-messages-rpc messages))) (save-messages-rpc messages)))
(fx/defn save-message [cofx message] (fx/defn save-messages [{:keys [db]}]
{::save-message [message]}) (when-let [messages (vals (:messages/stored db))]
;; Pull message from database to pick up most recent changes, default to
;; stored one in case it has been offloaded
(let [hydrated-messages (map #(get-in db [:chats (-> % :content :chat-id) :messages (:message-id %)] %) messages)]
{:db (dissoc db :messages/stored)
::save-message hydrated-messages})))
(fx/defn handle-save-messages
{:events [::save-messages]}
[cofx]
(save-messages cofx))
(fx/defn save-message [{:keys [db]} {:keys [message-id] :as message}]
{:db (assoc-in db [:messages/stored message-id] message)
:dispatch-debounce [{:key :save-messages
:event [::save-messages]
:delay 500}]})
(fx/defn delete-message [cofx id] (fx/defn delete-message [cofx id]
(delete-message-rpc id)) (delete-message-rpc id))

18
src/status_im/events.cljs
View File

@ -2,6 +2,8 @@
(:require [clojure.string :as string] (:require [clojure.string :as string]
[re-frame.core :as re-frame] [re-frame.core :as re-frame]
[status-im.multiaccounts.core :as multiaccounts] [status-im.multiaccounts.core :as multiaccounts]
[status-im.data-store.messages :as data-store.messages]
[status-im.multiaccounts.create.core :as multiaccounts.create] [status-im.multiaccounts.create.core :as multiaccounts.create]
[status-im.multiaccounts.login.core :as multiaccounts.login] [status-im.multiaccounts.login.core :as multiaccounts.login]
[status-im.multiaccounts.logout.core :as multiaccounts.logout] [status-im.multiaccounts.logout.core :as multiaccounts.logout]
@ -163,7 +165,10 @@
(handlers/register-handler-fx (handlers/register-handler-fx
:multiaccounts.logout.ui/logout-confirmed :multiaccounts.logout.ui/logout-confirmed
(fn [cofx _] (fn [cofx _]
(multiaccounts.logout/logout cofx))) (fx/merge
cofx
(data-store.messages/save-messages)
(multiaccounts.logout/logout))))
;; multiaccounts update module ;; multiaccounts update module
@ -631,11 +636,6 @@
(fn [cofx _] (fn [cofx _]
(chat/disable-chat-cooldown cofx))) (chat/disable-chat-cooldown cofx)))
(handlers/register-handler-fx
:message/add
(fn [cofx [_ messages]]
(chat.message/receive-many cofx messages)))
(handlers/register-handler-fx (handlers/register-handler-fx
:message/update-message-status :message/update-message-status
(fn [cofx [_ chat-id message-id status]] (fn [cofx [_ chat-id message-id status]]
@ -1238,12 +1238,6 @@
;; transport module ;; transport module
(handlers/register-handler-fx
:transport/messages-received
[handlers/logged-in (re-frame/inject-cofx :random-id-generator)]
(fn [cofx [_ js-error js-messages chat-id]]
(transport.message/receive-whisper-messages cofx js-error js-messages chat-id)))
(handlers/register-handler-fx (handlers/register-handler-fx
:transport/send-status-message-error :transport/send-status-message-error
(fn [{:keys [db] :as cofx} [_ err]] (fn [{:keys [db] :as cofx} [_ err]]

11
src/status_im/group_chats/core.cljs
View File

@ -470,7 +470,7 @@
[cofx {:keys [chat-id [cofx {:keys [chat-id
message message
membership-updates] :as membership-update} membership-updates] :as membership-update}
{:keys [raw-payload metadata]} {:keys [whisper-timestamp metadata]}
sender-signature] sender-signature]
(let [dev-mode? (get-in cofx [:db :multiaccount :dev-mode?])] (let [dev-mode? (get-in cofx [:db :multiaccount :dev-mode?])]
(when (valid-chat-id? chat-id (extract-creator membership-update)) (when (valid-chat-id? chat-id (extract-creator membership-update))
@ -498,10 +498,11 @@
;; don't allow anything but group messages ;; don't allow anything but group messages
(instance? protocol/Message message) (instance? protocol/Message message)
(= :group-user-message (:message-type message))) (= :group-user-message (:message-type message)))
(protocol/receive message chat-id sender-signature nil (protocol/receive message
(assoc % chat-id
:metadata metadata sender-signature
:js-obj #js {:payload raw-payload})))))))) whisper-timestamp
(assoc % :metadata metadata))))))))
(defn handle-sign-success (defn handle-sign-success
"Upsert chat and send signed payload to group members" "Upsert chat and send signed payload to group members"

1
src/status_im/js_dependencies.cljs
View File

@ -6,3 +6,4 @@
(def BigNumber (js/require "bignumber.js")) (def BigNumber (js/require "bignumber.js"))
(def web3-utils (js/require "web3-utils")) (def web3-utils (js/require "web3-utils"))
(def hi-base32 (js/require "hi-base32")) (def hi-base32 (js/require "hi-base32"))
(def rb-tree (js/require "functional-red-black-tree"))

62
src/status_im/subs.cljs
View File

@ -730,19 +730,42 @@
(fn [chat] (fn [chat]
(:public? chat))) (:public? chat)))
(re-frame/reg-sub
:chats/message-list
:<- [:chats/current-chat]
(fn [chat]
(:message-list chat)))
(re-frame/reg-sub
:chats/messages
:<- [:chats/current-chat]
(fn [chat]
(:messages chat)))
(defn hydrate-messages
"Pull data from messages and add it to the sorted list"
[message-list messages]
(keep #(if (= :message (% :type))
(when-let [message (messages (% :message-id))]
(merge message %))
%)
message-list))
(re-frame/reg-sub (re-frame/reg-sub
:chats/current-chat-messages-stream :chats/current-chat-messages-stream
:<- [:chats/current-chat-messages] :<- [:chats/message-list]
:<- [:chats/current-chat-message-groups] :<- [:chats/messages]
:<- [:chats/messages-gaps] :<- [:chats/messages-gaps]
:<- [:chats/range] :<- [:chats/range]
:<- [:chats/all-loaded?] :<- [:chats/all-loaded?]
:<- [:chats/public?] :<- [:chats/public?]
(fn [[messages message-groups messages-gaps range all-loaded? public?]] (fn [[message-list messages messages-gaps range all-loaded? public?]]
(-> (chat.db/sort-message-groups message-groups messages) (-> (if message-list
(chat.db/messages-with-datemarks (array-seq (.-values message-list))
messages messages-gaps range all-loaded? public?) [])
chat.db/messages-stream))) (chat.db/add-datemarks)
(hydrate-messages messages)
(chat.db/add-gaps messages-gaps range all-loaded? public?))))
(re-frame/reg-sub (re-frame/reg-sub
:chats/current-chat-intro-status :chats/current-chat-intro-status
@ -1585,7 +1608,7 @@
(fn [[contacts current-multiaccount] [_ identity]] (fn [[contacts current-multiaccount] [_ identity]]
(let [me? (= (:public-key current-multiaccount) identity)] (let [me? (= (:public-key current-multiaccount) identity)]
(if me? (if me?
{:ens-name (:name current-multiaccount) {:ens-name (:preferred-name current-multiaccount)
:alias (gfycat/generate-gfy identity)} :alias (gfycat/generate-gfy identity)}
(let [contact (or (contacts identity) (let [contact (or (contacts identity)
(contact.db/public-key->new-contact identity))] (contact.db/public-key->new-contact identity))]
@ -1594,6 +1617,29 @@
:alias (or (:alias contact) :alias (or (:alias contact)
(gfycat/generate-gfy identity))}))))) (gfycat/generate-gfy identity))})))))
(re-frame/reg-sub
:messages/quote-info
:<- [:chats/messages]
:<- [:contacts/contacts]
:<- [:multiaccount]
(fn [[messages contacts current-multiaccount] [_ message-id]]
(when-let [message (get messages message-id)]
(let [identity (:from message)
me? (= (:public-key current-multiaccount) identity)]
(if me?
{:quote {:from identity
:text (get-in message [:content :text])}
:ens-name (:preferred-name current-multiaccount)
:alias (gfycat/generate-gfy identity)}
(let [contact (or (contacts identity)
(contact.db/public-key->new-contact identity))]
{:quote {:from identity
:text (get-in message [:content :text])}
:ens-name (when (:ens-verified contact)
(:name contact))
:alias (or (:alias contact)
(gfycat/generate-gfy identity))}))))))
(re-frame/reg-sub (re-frame/reg-sub
:contacts/all-contacts-not-in-current-chat :contacts/all-contacts-not-in-current-chat
:<- [::query-current-chat-contacts remove] :<- [::query-current-chat-contacts remove]

24
src/status_im/transport/impl/receive.cljs
View File

@ -2,6 +2,7 @@
(:require [status-im.group-chats.core :as group-chats] (:require [status-im.group-chats.core :as group-chats]
[status-im.contact.core :as contact] [status-im.contact.core :as contact]
[status-im.utils.fx :as fx] [status-im.utils.fx :as fx]
[status-im.chat.models.message :as chat.message]
[status-im.ens.core :as ens] [status-im.ens.core :as ens]
[status-im.pairing.core :as pairing] [status-im.pairing.core :as pairing]
[status-im.transport.message.contact :as transport.contact] [status-im.transport.message.contact :as transport.contact]
@ -13,9 +14,9 @@
(extend-type transport.group-chat/GroupMembershipUpdate (extend-type transport.group-chat/GroupMembershipUpdate
protocol/StatusMessage protocol/StatusMessage
(receive [this _ signature _ {:keys [metadata js-obj] :as cofx}] (receive [this _ signature timestamp {:keys [metadata js-obj] :as cofx}]
(group-chats/handle-membership-update-received cofx this signature {:metadata metadata (group-chats/handle-membership-update-received cofx this signature {:whisper-timestamp timestamp
:raw-payload (.-payload js-obj)}))) :metadata metadata})))
(extend-type transport.contact/ContactRequest (extend-type transport.contact/ContactRequest
protocol/StatusMessage protocol/StatusMessage
@ -53,7 +54,16 @@
(extend-type protocol/Message (extend-type protocol/Message
protocol/StatusMessage protocol/StatusMessage
(receive [this chat-id signature timestamp cofx] (receive [this chat-id signature timestamp {:keys [db] :as cofx}]
(fx/merge cofx (let [message (assoc (into {} this)
(transport.message/receive-transit-message this chat-id signature timestamp) :message-id
(ens/verify-names-from-message this signature)))) (get-in cofx [:metadata :messageId])
:chat-id chat-id
:whisper-timestamp (* 1000 timestamp)
:alias (get-in cofx [:metadata :author :alias])
:identicon (get-in cofx [:metadata :author :identicon])
:from signature
:metadata (:metadata cofx))]
(chat.message/receive-one cofx message))))
; disable verification until enabled in status-go
; (ens/verify-names-from-message this signature))))

112
src/status_im/transport/message/core.cljs
View File

@ -3,6 +3,7 @@
(:require [goog.object :as o] (:require [goog.object :as o]
[re-frame.core :as re-frame] [re-frame.core :as re-frame]
[status-im.chat.models.message :as models.message] [status-im.chat.models.message :as models.message]
[status-im.utils.handlers :as handlers]
[status-im.ethereum.json-rpc :as json-rpc] [status-im.ethereum.json-rpc :as json-rpc]
[status-im.ethereum.core :as ethereum] [status-im.ethereum.core :as ethereum]
[status-im.transport.message.contact :as contact] [status-im.transport.message.contact :as contact]
@ -15,18 +16,13 @@
[taoensso.timbre :as log] [taoensso.timbre :as log]
[status-im.ethereum.json-rpc :as json-rpc])) [status-im.ethereum.json-rpc :as json-rpc]))
(defn add-raw-payload
"Add raw payload for id calculation"
[{:keys [message] :as m}]
(assoc m :raw-payload (clj->js message)))
(fx/defn receive-message (fx/defn receive-message
"Receive message handles a new status-message. "Receive message handles a new status-message.
dedup-id is passed by status-go and is used to deduplicate messages at that layer. dedup-id is passed by status-go and is used to deduplicate messages at that layer.
Once a message has been successfuly processed, that id needs to be sent back Once a message has been successfuly processed, that id needs to be sent back
in order to stop receiving that message" in order to stop receiving that message"
[cofx now-in-s filter-chat-id message-js] [{:keys [db]} now-in-s filter-chat-id message-js]
(let [blocked-contacts (get-in cofx [:db :contacts/blocked] #{}) (let [blocked-contacts (get db :contacts/blocked #{})
payload (.-payload message-js) payload (.-payload message-js)
timestamp (.-timestamp (.-message message-js)) timestamp (.-timestamp (.-message message-js))
metadata-js (.-metadata message-js) metadata-js (.-metadata message-js)
@ -45,16 +41,17 @@
(not (blocked-contacts sig))) (not (blocked-contacts sig)))
(try (try
(when-let [valid-message (protocol/validate status-message)] (when-let [valid-message (protocol/validate status-message)]
(fx/merge (assoc cofx :js-obj raw-payload :metadata metadata) (protocol/receive
#(protocol/receive (assoc valid-message (assoc valid-message
:metadata metadata) :metadata metadata)
(or (or
filter-chat-id filter-chat-id
(get-in valid-message [:content :chat-id]) (get-in valid-message [:content :chat-id])
sig) sig)
sig sig
timestamp timestamp
%))) {:db db
:metadata metadata}))
(catch :default e nil))))) ; ignore unknown message types (catch :default e nil))))) ; ignore unknown message types
(defn- js-obj->seq [obj] (defn- js-obj->seq [obj]
@ -64,36 +61,42 @@
(aget obj i)) (aget obj i))
[obj])) [obj]))
(fx/defn receive-whisper-messages (handlers/register-handler-fx
[{:keys [now] :as cofx} error messages chat-id] ::process
(if (and (not error) (fn [cofx [_ messages now-in-s]]
messages) (let [[chat-id message] (first messages)
(let [now-in-s (quot now 1000) remaining-messages (rest messages)]
receive-message-fxs (map (fn [message] (if (seq remaining-messages)
(receive-message now-in-s chat-id message)) (assoc
messages)] (receive-message cofx now-in-s chat-id message)
(apply fx/merge cofx receive-message-fxs)) ;; We dispatch later to let the UI thread handle events, without this
(log/error "Something went wrong" error messages))) ;; it will keep processing events ignoring user input.
:dispatch-later [{:ms 20 :dispatch [::process remaining-messages now-in-s]}])
(receive-message cofx now-in-s chat-id message)))))
(fx/defn receive-messages [cofx event-js] (fx/defn receive-messages
(let [fxs (keep "Initialize the ::process event, which will process messages one by one
(fn [message-specs] dispatching later to itself"
(let [chat (.-chat message-specs) [{:keys [now] :as cofx} event-js]
messages (.-messages message-specs) (let [now-in-s (quot now 1000)
error (.-error message-specs)] events (reduce
(when (seq messages) (fn [acc message-specs]
(receive-whisper-messages (let [chat (.-chat message-specs)
error messages (.-messages message-specs)
messages error (.-error message-specs)
;; For discovery and negotiated filters we don't chat-id (if (or (.-discovery chat)
;; set a chatID, and we use the signature of the message (.-negotiated chat))
;; to indicate which chat it is for nil
(if (or (.-discovery chat) (.-chatId chat))]
(.-negotiated chat)) (if (seq messages)
nil (reduce (fn [acc m]
(.-chatId chat)))))) (conj acc [chat-id m]))
(.-messages event-js))] acc
(apply fx/merge cofx fxs))) messages)
acc)))
[]
(.-messages event-js))]
{:dispatch [::process events now-in-s]}))
(fx/defn remove-hash (fx/defn remove-hash
[{:keys [db] :as cofx} envelope-hash] [{:keys [db] :as cofx} envelope-hash]
@ -173,20 +176,3 @@
:on-success #(log/debug "successfully confirmed messages") :on-success #(log/debug "successfully confirmed messages")
:on-failure #(log/error "failed to confirm messages" %)})))) :on-failure #(log/error "failed to confirm messages" %)}))))
(fx/defn receive-transit-message [cofx message chat-id signature timestamp]
(let [received-message-fx {:chat-received-message/add-fx
[(assoc (into {} message)
:message-id
(get-in cofx [:metadata :messageId])
:chat-id chat-id
:whisper-timestamp timestamp
:alias (get-in cofx [:metadata :author :alias])
:identicon (get-in cofx [:metadata :author :identicon])
:from signature
:metadata (:metadata cofx)
:js-obj (:js-obj cofx))]}]
(whitelist/filter-message cofx
received-message-fx
(:message-type message)
(get-in message [:content :tribute-transaction])
signature)))

60
src/status_im/ui/screens/chat/message/message.cljs
View File

@ -41,16 +41,20 @@
(get content :command-ref)) (get content :command-ref))
content content-type]]) content content-type]])
(defview quoted-message [{:keys [from text]} outgoing current-public-key] (defview quoted-message [message-id {:keys [from text]} outgoing current-public-key]
(letsubs [{:keys [ens-name alias]} [:contacts/contact-name-by-identity from]] (letsubs [{:keys [quote
[react/view {:style (style/quoted-message-container outgoing)} ens-name
[react/view {:style style/quoted-message-author-container} alias]}
[vector-icons/tiny-icon :tiny-icons/tiny-reply {:color (if outgoing colors/white-transparent colors/gray)}] [:messages/quote-info message-id]]
(chat.utils/format-reply-author from alias ens-name current-public-key (partial style/quoted-message-author outgoing))] (when (or quote text)
[react/view {:style (style/quoted-message-container outgoing)}
[react/view {:style style/quoted-message-author-container}
[vector-icons/tiny-icon :tiny-icons/tiny-reply {:color (if outgoing colors/white-transparent colors/gray)}]
(chat.utils/format-reply-author (or from (:from quote)) alias ens-name current-public-key (partial style/quoted-message-author outgoing))]
[react/text {:style (style/quoted-message-text outgoing) [react/text {:style (style/quoted-message-text outgoing)
:number-of-lines 5} :number-of-lines 5}
text]])) (or text (:text quote))]])))
(defview message-content-status [{:keys [content]}] (defview message-content-status [{:keys [content]}]
[react/view style/status-container [react/view style/status-container
@ -63,12 +67,16 @@
(i18n/label (if expanded? :show-less :show-more))]) (i18n/label (if expanded? :show-less :show-more))])
(defn text-message (defn text-message
[{:keys [chat-id message-id content timestamp-str group-chat outgoing current-public-key expanded?] :as message}] [{:keys [chat-id message-id content
timestamp-str group-chat outgoing current-public-key expanded?] :as message}]
[message-view message [message-view message
(let [collapsible? (and (:should-collapse? content) group-chat)] (let [response-to (or (:response-to content)
(:response-to-v2 content))
collapsible? (and (:should-collapse? content) group-chat)]
[react/view [react/view
(when (:response-to content) (when response-to
[quoted-message (:response-to content) outgoing current-public-key]) [quoted-message response-to (:quoted-message message) outgoing current-public-key])
(apply react/nested-text (apply react/nested-text
(cond-> {:style (style/text-message collapsible? outgoing) (cond-> {:style (style/text-message collapsible? outgoing)
:text-break-strategy :balanced :text-break-strategy :balanced
@ -90,12 +98,14 @@
(defn emoji-message (defn emoji-message
[{:keys [content current-public-key alias] :as message}] [{:keys [content current-public-key alias] :as message}]
[message-view message (let [response-to (or (:response-to content)
[react/view {:style (style/style-message-text false)} (:response-to-v2 content))]
(when (:response-to content) [message-view message
[quoted-message (:response-to content) alias false current-public-key]) [react/view {:style (style/style-message-text false)}
[react/text {:style (style/emoji-message message)} (when response-to
(:text content)]]]) [quoted-message response-to (:quoted-message message) alias false current-public-key])
[react/text {:style (style/emoji-message message)}
(:text content)]]]))
(defmulti message-content (fn [_ message _] (message :content-type))) (defmulti message-content (fn [_ message _] (message :content-type)))
@ -169,12 +179,13 @@
(defn message-delivery-status (defn message-delivery-status
[{:keys [chat-id message-id outgoing-status [{:keys [chat-id message-id outgoing-status
content last-outgoing? message-type] :as message}] first-outgoing?
content message-type] :as message}]
(when (not= :system-message message-type) (when (not= :system-message message-type)
(case outgoing-status (case outgoing-status
:sending [message-activity-indicator] :sending [message-activity-indicator]
:not-sent [message-not-sent-text chat-id message-id] :not-sent [message-not-sent-text chat-id message-id]
:sent (when last-outgoing? :sent (when first-outgoing?
[react/view style/delivery-view [react/view style/delivery-view
[react/text {:style style/delivery-text} [react/text {:style style/delivery-text}
(i18n/label :t/status-sent)]]) (i18n/label :t/status-sent)]])
@ -187,9 +198,10 @@
(chat.utils/format-author alias style/message-author-name ens-name))) (chat.utils/format-author alias style/message-author-name ens-name)))
(defn message-body (defn message-body
[{:keys [last-in-group? [{:keys [alias
last-in-group?
first-in-group?
display-photo? display-photo?
alias
display-username? display-username?
from from
outgoing outgoing
@ -199,7 +211,7 @@
[react/view (style/message-body message) [react/view (style/message-body message)
(when display-photo? (when display-photo?
[react/view (style/message-author outgoing) [react/view (style/message-author outgoing)
(when last-in-group? (when first-in-group?
[react/touchable-highlight {:on-press #(when-not modal? (re-frame/dispatch [:chat.ui/show-profile from]))} [react/touchable-highlight {:on-press #(when-not modal? (re-frame/dispatch [:chat.ui/show-profile from]))}
[react/view [react/view
[photos/member-photo from]]])]) [photos/member-photo from]]])])

12
src/status_im/ui/screens/chat/styles/message/message.cljs
View File

@ -10,15 +10,15 @@
{:color (if outgoing colors/white colors/text)}) {:color (if outgoing colors/white colors/text)})
(defn message-padding-top (defn message-padding-top
[{:keys [first-in-group? display-username?]}] [{:keys [last-in-group? display-username?]}]
(if (and display-username? (if (and display-username?
first-in-group?) last-in-group?)
6 6
2)) 2))
(defn last-message-padding (defn last-message-padding
[{:keys [last? typing]}] [{:keys [first? typing]}]
(when (and last? (not typing)) (when (and first? (not typing))
{:padding-bottom 16})) {:padding-bottom 16}))
(defn message-body (defn message-body
@ -139,11 +139,11 @@
:margin-top (if incoming-group 4 0)}) :margin-top (if incoming-group 4 0)})
(defn message-view (defn message-view
[{:keys [content-type outgoing group-chat first-in-group?]}] [{:keys [content-type outgoing group-chat last-in-group?]}]
(merge {:padding-vertical 6 (merge {:padding-vertical 6
:padding-horizontal 12 :padding-horizontal 12
:border-radius 8 :border-radius 8
:margin-top (if (and first-in-group? :margin-top (if (and last-in-group?
(or outgoing (or outgoing
(not group-chat))) (not group-chat)))
16 16

11
src/status_im/ui/screens/chat/toolbar_content.cljs
View File

@ -7,17 +7,6 @@
[status-im.utils.datetime :as time]) [status-im.utils.datetime :as time])
(:require-macros [status-im.utils.views :refer [defview letsubs]])) (:require-macros [status-im.utils.views :refer [defview letsubs]]))
(defn- online-text [contact chat-id]
(if contact
(let [last-online (get contact :last-online)
last-online-date (time/to-date last-online)
now-date (t/now)]
(if (and (pos? last-online)
(<= last-online-date now-date))
(time/time-ago last-online-date)
(i18n/label :t/active-unknown)))
(i18n/label :t/active-unknown)))
(defn- in-progress-text [{:keys [highestBlock currentBlock startBlock]}] (defn- in-progress-text [{:keys [highestBlock currentBlock startBlock]}]
(let [total (- highestBlock startBlock) (let [total (- highestBlock startBlock)
ready (- currentBlock startBlock) ready (- currentBlock startBlock)

4
src/status_im/ui/screens/chat/utils.cljs
View File

@ -46,8 +46,8 @@
:on-press #(re-frame/dispatch [:chat.ui/start-public-chat (subs text 1) {:navigation-reset? true}])})}) :on-press #(re-frame/dispatch [:chat.ui/start-public-chat (subs text 1) {:navigation-reset? true}])})})
(defn- lookup-props [text-chunk message kind] (defn- lookup-props [text-chunk message kind]
(let [prop (get styling->prop kind) (let [prop (get styling->prop (keyword kind))
prop-fn (get action->prop-fn kind)] prop-fn (get action->prop-fn (keyword kind))]
(if prop-fn (prop-fn text-chunk message) prop))) (if prop-fn (prop-fn text-chunk message) prop)))
(defn render-chunks [render-recipe message] (defn render-chunks [render-recipe message]

2
src/status_im/utils/fx.cljs
View File

@ -12,7 +12,7 @@
cofx)) cofx))
(def ^:private mergeable-keys (def ^:private mergeable-keys
#{:chat-received-message/add-fx #{:dispatch-debounce
:filters/load-filters :filters/load-filters
:pairing/set-installation-metadata :pairing/set-installation-metadata
:status-im.data-store.messages/save-message :status-im.data-store.messages/save-message

6
src/status_im/utils/types.cljs
View File

@ -1,5 +1,7 @@
(ns status-im.utils.types (ns status-im.utils.types
(:require [cognitect.transit :as transit])) (:require
[cljs-bean.core :as clj-bean]
[cognitect.transit :as transit]))
(defn to-string [s] (defn to-string [s]
(if (keyword? s) (if (keyword? s)
@ -7,7 +9,7 @@
s)) s))
(defn clj->json [data] (defn clj->json [data]
(.stringify js/JSON (clj->js data))) (.stringify js/JSON (clj-bean/->js data)))
(defn json->clj [json] (defn json->clj [json]
(when-not (= json "undefined") (when-not (= json "undefined")

25
src/status_im/utils/utils.cljs
View File

@ -85,6 +85,31 @@
(when address (when address
(get-shortened-address (eip55/address->checksum (ethereum/normalized-address address))))) (get-shortened-address (eip55/address->checksum (ethereum/normalized-address address)))))
;; debounce, taken from https://github.com/johnswanson/re-frame-debounce-fx
; {:dispatch-debounce {:key :search
; :event [:search value]
; :delay 250}}))
(def registered-keys (atom nil))
(defn dispatch-if-not-superceded [{:keys [key delay event time-received]}]
(when (= time-received (get @registered-keys key))
;; no new events on this key!
(re-frame/dispatch event)))
(defn dispatch-debounced [{:keys [delay] :as debounce}]
(js/setTimeout
(fn [] (dispatch-if-not-superceded debounce))
delay))
(re-frame/reg-fx
:dispatch-debounce
(fn dispatch-debounce [debounces]
(doseq [debounce debounces]
(let [ts (.getTime (js/Date.))]
(swap! registered-keys assoc (:key debounce) ts)
(dispatch-debounced (assoc debounce :time-received ts))))))
;; background-timer ;; background-timer
(defn set-timeout [cb ms] (defn set-timeout [cb ms]

380
test/cljs/status_im/test/chat/db.cljs
View File

@ -15,97 +15,31 @@
:chat-id "1" :chat-id "1"
:name "unchanged"}))))) :name "unchanged"})))))
(deftest message-stream-tests (deftest intersperse-datemarks
(testing "messages with no interspersed datemarks" (testing "it mantains the order even when timestamps are across days"
(let [m1 {:from "1" (let [message-1 {:datemark "Dec 31, 1999"
:datemark "a" :whisper-timestamp 946641600000} ; 1999}
:outgoing false} message-2 {:datemark "Jan 1, 2000"
m2 {:from "2" :whisper-timestamp 946728000000} ; 2000 this will displayed in 1999
:datemark "a" message-3 {:datemark "Dec 31, 1999"
:outgoing true} :whisper-timestamp 946641600000} ; 1999
m3 {:from "2" message-4 {:datemark "Jan 1, 2000"
:datemark "a" :whisper-timestamp 946728000000} ; 2000
:outgoing true} ordered-messages [message-4
dm1 {:type :datemark message-3
:value "a"} message-2
messages [m1 m2 m3 dm1] message-1]
[actual-m1 [m1 d1 m2 m3 m4 d2 :as ms] (db/add-datemarks ordered-messages)]
actual-m2 (is (= "Jan 1, 2000"
actual-m3] (db/messages-stream messages)] (:datemark m1)))
(testing "it marks only the first message as :last?" (is (= {:type :datemark
(is (:last? actual-m1)) :value "Jan 1, 2000"} d1))
(is (not (:last? actual-m2))) (is (= "Dec 31, 1999"
(is (not (:last? actual-m3)))) (:datemark m2)
(testing "it marks the first outgoing message as :last-outgoing?" (:datemark m3)
(is (not (:last-outgoing? actual-m1))) (:datemark m4)))
(is (:last-outgoing? actual-m2)) (is (= {:type :datemark
(is (not (:last-outgoing? actual-m3)))) :value "Dec 31, 1999"} d2)))))
(testing "it marks messages from the same author next to another with :first-in-group?"
(is (:first-in-group? actual-m1))
(is (not (:first-in-group? actual-m2)))
(is (:first-in-group? actual-m3)))
(testing "it marks messages with display-photo? when they are not outgoing and we are in a group chat"
(is (:display-photo? actual-m1))
(is (not (:display-photo? actual-m2)))
(is (not (:display-photo? actual-m3))))
(testing "it marks messages with display-username? when we display the photo and are the first in a group"
(is (:display-username? actual-m1))
(is (not (:display-username? actual-m2)))
(is (not (:display-username? actual-m3))))
(testing "it marks the last message from the same author with :last-in-group?"
(is (:last-in-group? actual-m1))
(is (:last-in-group? actual-m2))
(is (not (:last-in-group? actual-m3))))))
(testing "messages with interspersed datemarks"
(let [m1 {:from "2" ; first & last in group
:timestamp 63000
:outgoing true}
dm1 {:type :datemark
:value "a"}
m2 {:from "2" ; first & last in group as more than 1 minute after previous message
:timestamp 62000
:outgoing false}
m3 {:from "2" ; last in group
:timestamp 1
:outgoing false}
m4 {:from "2" ; first in group
:timestamp 0
:outgoing false}
dm2 {:type :datemark
:value "b"}
messages [m1 dm1 m2 m3 m4 dm2]
[actual-m1
_
actual-m2
actual-m3
actual-m4
_] (db/messages-stream messages)]
(testing "it marks the first outgoing message as :last-outgoing?"
(is (:last-outgoing? actual-m1))
(is (not (:last-outgoing? actual-m2)))
(is (not (:last-outgoing? actual-m3)))
(is (not (:last-outgoing? actual-m4))))
(testing "it sets :first-in-group? after a datemark"
(is (:first-in-group? actual-m1))
(is (:first-in-group? actual-m4)))
(testing "it sets :first-in-group? if more than 60s have passed since last message"
(is (:first-in-group? actual-m2)))
(testing "it sets :last-in-group? after a datemark"
(is (:last-in-group? actual-m1))
(is (:last-in-group? actual-m2))
(is (:last-in-group? actual-m3))
(is (not (:last-in-group? actual-m4))))))
(testing "system-messages"
(let [m1 {:from "system"
:message-type :system-message
:datemark "a"
:outgoing false}
messages [m1]
[actual-m1] (db/messages-stream messages)]
(testing "it does display the photo"
(is (:display-photo? actual-m1))
(testing "it does not display the username"
(is (not (:display-username? actual-m1))))))))
(deftest active-chats-test (deftest active-chats-test
(with-redefs [gfycat/generate-gfy (constantly "generated") (with-redefs [gfycat/generate-gfy (constantly "generated")
@ -119,227 +53,123 @@
(is (= #{"1" "2"} (is (= #{"1" "2"}
(set (keys (db/active-chats {} chats {}))))))))) (set (keys (db/active-chats {} chats {})))))))))
#_(deftest messages-with-datemarks (deftest add-gaps
(testing "empty state" (testing "empty state"
(is (empty? (is (empty?
(db/messages-with-datemarks (db/add-gaps
nil nil
nil nil
nil nil
nil false
nil false))))
false (testing "empty state pub-chat"
false)))) (is (=
(testing "empty state pub-chat" [{:type :gap
(is (= :value ":first-gap"
[{:type :gap :first-gap? true}]
:value ":first-gap" (db/add-gaps
:first-gap? true}] nil
(db/messages-with-datemarks nil
nil {:lowest-request-from 10
nil :highest-request-to 30}
nil true
nil true))))
{:lowest-request-from 10 (testing "simple case with gap"
:highest-request-to 30} (is (= '({:whisper-timestamp 40
true :message-id :m4
true)))) :timestamp 40}
(testing "simple case"
(is (=
'({:whisper-timestamp 40
:timestamp 40
:content nil
:timestamp-str "14:00"
:datemark "today"}
{:whisper-timestamp 30
:timestamp 30
:content nil
:timestamp-str "13:00"
:datemark "today"}
{:value "today"
:type :datemark
:whisper-timestamp 30
:timestamp 30}
{:whisper-timestamp 20
:timestamp 20
:content nil
:timestamp-str "12:00"
:datemark "yesterday"}
{:whisper-timestamp 10
:timestamp 10
:content nil
:timestamp-str "11:00"
:datemark "yesterday"}
{:value "yesterday"
:type :datemark
:whisper-timestamp 10
:timestamp 10})
(db/messages-with-datemarks
{"yesterday"
(list
{:message-id :m1
:timestamp-str "11:00"
:whisper-timestamp 10
:timestamp 10}
{:message-id :m2
:timestamp-str "12:00"
:whisper-timestamp 20
:timestamp 20})
"today"
(list
{:message-id :m3
:timestamp-str "13:00"
:whisper-timestamp 30
:timestamp 30}
{:message-id :m4
:timestamp-str "14:00"
:whisper-timestamp 40
:timestamp 40})}
{:m1 {:whisper-timestamp 10
:timestamp 10}
:m2 {:whisper-timestamp 20
:timestamp 20}
:m3 {:whisper-timestamp 30
:timestamp 30}
:m4 {:whisper-timestamp 40
:timestamp 40}}
nil
nil
nil
nil
nil))))
(testing "simple case with gap"
(is (=
'({:whisper-timestamp 40
:timestamp 40
:content nil
:timestamp-str "14:00"
:datemark "today"}
{:type :gap {:type :gap
:value ":gapid1" :value ":gapid1"
:gaps {:ids [:gapid1]}} :gaps {:ids [:gapid1]}}
{:whisper-timestamp 30 {:whisper-timestamp 30
:timestamp 30 :timestamp 30
:content nil :message-id :m3}
:timestamp-str "13:00"
:datemark "today"}
{:value "today" {:value "today"
:type :datemark :type :datemark
:whisper-timestamp 30 :whisper-timestamp 30
:timestamp 30} :timestamp 30}
{:whisper-timestamp 20 {:whisper-timestamp 20
:timestamp 20 :timestamp 20
:content nil :message-id :m2}
:timestamp-str "12:00"
:datemark "yesterday"}
{:whisper-timestamp 10 {:whisper-timestamp 10
:timestamp 10 :timestamp 10
:content nil :message-id :m1}
:timestamp-str "11:00"
:datemark "yesterday"}
{:value "yesterday" {:value "yesterday"
:type :datemark :type :datemark
:whisper-timestamp 10 :whisper-timestamp 10
:timestamp 10}) :timestamp 10})
(db/messages-with-datemarks (db/add-gaps
{"yesterday" [{:message-id :m4
(list :whisper-timestamp 40
{:message-id :m1 :timestamp 40}
:timestamp-str "11:00" {:message-id :m3
:whisper-timestamp 10 :whisper-timestamp 30
:timestamp 10} :timestamp 30}
{:message-id :m2 {:type :datemark
:timestamp-str "12:00" :value "today"
:whisper-timestamp 20 :whisper-timestamp 30
:timestamp 20}) :timestamp 30}
"today" {:message-id :m2
(list :whisper-timestamp 20
{:message-id :m3 :timestamp 20}
:timestamp-str "13:00" {:message-id :m1
:whisper-timestamp 30 :whisper-timestamp 10
:timestamp 30} :timestamp 10}
{:message-id :m4 {:type :datemark
:timestamp-str "14:00" :value "yesterday"
:whisper-timestamp 40 :whisper-timestamp 10
:timestamp 40})} :timestamp 10}]
{:m1 {:whisper-timestamp 10
:timestamp 10}
:m2 {:whisper-timestamp 20
:timestamp 20}
:m3 {:whisper-timestamp 30
:timestamp 30}
:m4 {:whisper-timestamp 40
:timestamp 40}}
nil
[{:from 25 [{:from 25
:to 30 :to 30
:id :gapid1}] :id :gapid1}]
nil nil
nil nil
nil)))) nil))))
(testing "simple case with gap after all messages" (testing "simple case with gap after all messages"
(is (= (is (= '({:type :gap
'({:type :gap
:value ":gapid1" :value ":gapid1"
:gaps {:ids (:gapid1)}} :gaps {:ids (:gapid1)}}
{:whisper-timestamp 40 {:whisper-timestamp 40
:timestamp 40 :message-id :m4
:content nil :timestamp 40}
:timestamp-str "14:00"
:datemark "today"}
{:whisper-timestamp 30 {:whisper-timestamp 30
:timestamp 30 :message-id :m3
:content nil :timestamp 30}
:timestamp-str "13:00"
:datemark "today"}
{:value "today" {:value "today"
:type :datemark :type :datemark
:whisper-timestamp 30 :whisper-timestamp 30
:timestamp 30} :timestamp 30}
{:whisper-timestamp 20 {:whisper-timestamp 20
:timestamp 20 :message-id :m2
:content nil :timestamp 20}
:timestamp-str "12:00"
:datemark "yesterday"}
{:whisper-timestamp 10 {:whisper-timestamp 10
:timestamp 10 :message-id :m1
:content nil :timestamp 10}
:timestamp-str "11:00"
:datemark "yesterday"}
{:value "yesterday" {:value "yesterday"
:type :datemark :type :datemark
:whisper-timestamp 10 :whisper-timestamp 10
:timestamp 10}) :timestamp 10})
(db/messages-with-datemarks (db/add-gaps
{"yesterday" [{:message-id :m4
(list :whisper-timestamp 40
{:message-id :m1 :timestamp 40}
:timestamp-str "11:00" {:message-id :m3
:whisper-timestamp 10 :whisper-timestamp 30
:timestamp 10} :timestamp 30}
{:message-id :m2 {:type :datemark
:timestamp-str "12:00" :value "today"
:whisper-timestamp 20 :whisper-timestamp 30
:timestamp 20}) :timestamp 30}
"today" {:message-id :m2
(list :whisper-timestamp 20
{:message-id :m3 :timestamp 20}
:timestamp-str "13:00" {:message-id :m1
:whisper-timestamp 30 :whisper-timestamp 10
:timestamp 30} :timestamp 10}
{:message-id :m4 {:type :datemark
:timestamp-str "14:00" :value "yesterday"
:whisper-timestamp 40 :whisper-timestamp 10
:timestamp 40})} :timestamp 10}]
{:m1 {:whisper-timestamp 10
:timestamp 10}
:m2 {:whisper-timestamp 20
:timestamp 20}
:m3 {:whisper-timestamp 30
:timestamp 30}
:m4 {:whisper-timestamp 40
:timestamp 40}}
nil
[{:from 100 [{:from 100
:to 110 :to 110
:id :gapid1}] :id :gapid1}]

40
test/cljs/status_im/test/chat/models/loading.cljs
View File

@ -1,40 +0,0 @@
(ns status-im.test.chat.models.loading
(:require [cljs.test :refer-macros [deftest is testing]]
[status-im.chat.models.loading :as loading]))
(deftest group-chat-messages
(let [cofx {:db {:chats {"chat-id" {:messages {0 {:message-id 0
:content "a"
:clock-value 0
:timestamp 0}
1 {:message-id 1
:content "b"
:clock-value 1
:timestamp 1}
2 {:message-id 2
:content "c"
:clock-value 2
:timestamp 2}
3 {:message-id 3
:content "d"
:clock-value 3
:timestamp 3}}}}}}
new-messages [{:message-id 1
:content "b"
:clock-value 1
:timestamp 1}
{:message-id 2
:content "c"
:clock-value 2
:timestamp 2}
{:message-id 3
:content "d"
:clock-value 3
:timestamp 3}]]
(testing "New messages are grouped/sorted correctly"
(is (= '(1 2 3)
(map :message-id
(-> (get-in (loading/group-chat-messages cofx "chat-id" new-messages)
[:db :chats "chat-id" :message-groups])
first
second)))))))

84
test/cljs/status_im/test/chat/models/message.cljs
View File

@ -44,47 +44,50 @@
:current-chat-id "chat-id" :current-chat-id "chat-id"
:chats {"chat-id" {:messages {}}}}] :chats {"chat-id" {:messages {}}}}]
(testing "a message coming from you!" (testing "a message coming from you!"
(let [actual (message/receive-many {:db db} (let [actual (message/receive-one {:db db}
[{:from "me" {:from "me"
:message-type :user-message :message-type :user-message
:timestamp 0 :timestamp 0
:message-id "id" :whisper-timestamp 0
:chat-id "chat-id" :message-id "id"
:content "b" :chat-id "chat-id"
:clock-value 1}]) :content "b"
:clock-value 1})
message (get-in actual [:db :chats "chat-id" :messages "id"])] message (get-in actual [:db :chats "chat-id" :messages "id"])]
(testing "it adds the message" (testing "it adds the message"
(is message)) (is message))
(testing "it marks the message as outgoing" (testing "it marks the message as outgoing"
(is (= true (:outgoing message)))))))) (is (= true (:outgoing message))))))))
(deftest receive-many-clock-value (deftest receive-one-clock-value
(let [db {:multiaccount {:public-key "me"} (let [db {:multiaccount {:public-key "me"}
:view-id :chat :view-id :chat
:current-chat-id "chat-id" :current-chat-id "chat-id"
:chats {"chat-id" {:last-clock-value 10 :chats {"chat-id" {:last-clock-value 10
:messages {}}}}] :messages {}}}}]
(testing "a message with a higher clock value" (testing "a message with a higher clock value"
(let [actual (message/receive-many {:db db} (let [actual (message/receive-one {:db db}
[{:from "chat-id" {:from "chat-id"
:message-type :user-message :message-type :user-message
:timestamp 0 :timestamp 0
:message-id "id" :whisper-timestamp 0
:chat-id "chat-id" :message-id "id"
:content "b" :chat-id "chat-id"
:clock-value 12}]) :content "b"
:clock-value 12})
chat-clock-value (get-in actual [:db :chats "chat-id" :last-clock-value])] chat-clock-value (get-in actual [:db :chats "chat-id" :last-clock-value])]
(testing "it sets last-clock-value" (testing "it sets last-clock-value"
(is (= 12 chat-clock-value))))) (is (= 12 chat-clock-value)))))
(testing "a message with a lower clock value" (testing "a message with a lower clock value"
(let [actual (message/receive-many {:db db} (let [actual (message/receive-one {:db db}
[{:from "chat-id" {:from "chat-id"
:message-type :user-message :message-type :user-message
:timestamp 0 :timestamp 0
:message-id "id" :whisper-timestamp 0
:chat-id "chat-id" :message-id "id"
:content "b" :chat-id "chat-id"
:clock-value 2}]) :content "b"
:clock-value 2})
chat-clock-value (get-in actual [:db :chats "chat-id" :last-clock-value])] chat-clock-value (get-in actual [:db :chats "chat-id" :last-clock-value])]
(testing "it sets last-clock-value" (testing "it sets last-clock-value"
(is (= 10 chat-clock-value))))))) (is (= 10 chat-clock-value)))))))
@ -101,27 +104,30 @@
:message-type :group-user-message :message-type :group-user-message
:message-id "1" :message-id "1"
:clock-value 1 :clock-value 1
:whisper-timestamp 0
:timestamp 0} :timestamp 0}
bad-chat-id-message {:chat-id "bad-chat-id" bad-chat-id-message {:chat-id "bad-chat-id"
:from "present" :from "present"
:message-type :group-user-message :message-type :group-user-message
:message-id "1" :message-id "1"
:clock-value 1 :clock-value 1
:whisper-timestamp 0
:timestamp 0} :timestamp 0}
bad-from-message {:chat-id "chat-id" bad-from-message {:chat-id "chat-id"
:from "not-present" :from "not-present"
:message-type :group-user-message :message-type :group-user-message
:message-id "1" :message-id "1"
:clock-value 1 :clock-value 1
:whisper-timestamp 0
:timestamp 0}] :timestamp 0}]
(testing "a valid message" (testing "a valid message"
(is (get-in (message/receive-many cofx [valid-message]) [:db :chats "chat-id" :messages "1"]))) (is (get-in (message/receive-one cofx valid-message) [:db :chats "chat-id" :messages "1"])))
(testing "a message from someone not in the list of participants" (testing "a message from someone not in the list of participants"
(is (= cofx (message/receive-many cofx [bad-from-message])))) (is (not (message/receive-one cofx bad-from-message))))
(testing "a message with non existing chat-id" (testing "a message with non existing chat-id"
(is (= cofx (message/receive-many cofx [bad-chat-id-message])))) (is (not (message/receive-one cofx bad-chat-id-message))))
(testing "a message from a delete chat" (testing "a message from a delete chat"
(is (= cofx-without-member (message/receive-many cofx-without-member [valid-message])))))) (is (not (message/receive-one cofx-without-member valid-message))))))
(deftest receive-public-chats (deftest receive-public-chats
(let [cofx {:db {:chats {"chat-id" {:public? true}} (let [cofx {:db {:chats {"chat-id" {:public? true}}
@ -133,17 +139,19 @@
:message-type :public-group-user-message :message-type :public-group-user-message
:message-id "1" :message-id "1"
:clock-value 1 :clock-value 1
:whisper-timestamp 0
:timestamp 0} :timestamp 0}
bad-chat-id-message {:chat-id "bad-chat-id" bad-chat-id-message {:chat-id "bad-chat-id"
:from "present" :from "present"
:message-type :public-group-user-message :message-type :public-group-user-message
:message-id "1" :message-id "1"
:clock-value 1 :clock-value 1
:whisper-timestamp 0
:timestamp 0}] :timestamp 0}]
(testing "a valid message" (testing "a valid message"
(is (get-in (message/receive-many cofx [valid-message]) [:db :chats "chat-id" :messages "1"]))) (is (get-in (message/receive-one cofx valid-message) [:db :chats "chat-id" :messages "1"])))
(testing "a message with non existing chat-id" (testing "a message with non existing chat-id"
(is (= cofx (message/receive-many cofx [bad-chat-id-message])))))) (is (not (message/receive-one cofx bad-chat-id-message))))))
(deftest receive-one-to-one (deftest receive-one-to-one
(with-redefs [gfycat/generate-gfy (constantly "generated") (with-redefs [gfycat/generate-gfy (constantly "generated")
@ -151,19 +159,21 @@
(let [cofx {:db {:chats {"matching" {}} (let [cofx {:db {:chats {"matching" {}}
:multiaccount {:public-key "me"} :multiaccount {:public-key "me"}
:current-chat-id "chat-id" :current-chat-id "matching"
:view-id :chat}} :view-id :chat}}
valid-message {:chat-id "matching" valid-message {:chat-id "matching"
:from "matching" :from "matching"
:message-type :user-message :message-type :user-message
:message-id "1" :message-id "1"
:clock-value 1 :clock-value 1
:whisper-timestamp 0
:timestamp 0} :timestamp 0}
own-message {:chat-id "matching" own-message {:chat-id "matching"
:from "me" :from "me"
:message-type :user-message :message-type :user-message
:message-id "1" :message-id "1"
:clock-value 1 :clock-value 1
:whisper-timestamp 0
:timestamp 0} :timestamp 0}
bad-chat-id-message {:chat-id "bad-chat-id" bad-chat-id-message {:chat-id "bad-chat-id"
@ -171,29 +181,33 @@
:message-type :user-message :message-type :user-message
:message-id "1" :message-id "1"
:clock-value 1 :clock-value 1
:whisper-timestamp 0
:timestamp 0}] :timestamp 0}]
(testing "a valid message" (testing "a valid message"
(is (get-in (message/receive-many cofx [valid-message]) [:db :chats "matching" :messages "1"]))) (is (get-in (message/receive-one cofx valid-message) [:db :chats "matching" :messages "1"])))
(testing "our own message" (testing "our own message"
(is (get-in (message/receive-many cofx [own-message]) [:db :chats "matching" :messages "1"]))) (is (get-in (message/receive-one cofx own-message) [:db :chats "matching" :messages "1"])))
(testing "a message with non matching chat-id" (testing "a message with non matching chat-id"
(is (get-in (message/receive-many cofx [bad-chat-id-message]) [:db :chats "not-matching" :messages "1"])))))) (is (get-in (message/receive-one cofx bad-chat-id-message) [:db :chats "not-matching" :messages "1"]))))))
(deftest delete-message (deftest delete-message
(let [timestamp (time/now) (let [timestamp (time/now)
cofx1 {:db {:chats {"chat-id" {:messages {0 {:message-id 0 cofx1 {:db {:chats {"chat-id" {:messages {0 {:message-id 0
:content "a" :content "a"
:clock-value 0 :clock-value 0
:whisper-timestamp (- timestamp 1)
:timestamp (- timestamp 1)} :timestamp (- timestamp 1)}
1 {:message-id 1 1 {:message-id 1
:content "b" :content "b"
:clock-value 1 :clock-value 1
:whisper-timestamp timestamp
:timestamp timestamp}} :timestamp timestamp}}
:message-groups {"datetime-today" '({:message-id 1} :message-groups {"datetime-today" '({:message-id 1}
{:message-id 0})}}}}} {:message-id 0})}}}}}
cofx2 {:db {:chats {"chat-id" {:messages {0 {:message-id 0 cofx2 {:db {:chats {"chat-id" {:messages {0 {:message-id 0
:content "a" :content "a"
:clock-value 0 :clock-value 0
:whisper-timestamp timestamp
:timestamp timestamp}} :timestamp timestamp}}
:message-groups {"datetime-today" '({:message-id 0})}}}}} :message-groups {"datetime-today" '({:message-id 0})}}}}}
fx1 (message/delete-message cofx1 "chat-id" 1) fx1 (message/delete-message cofx1 "chat-id" 1)

170
test/cljs/status_im/test/chat/models/message_list.cljs Normal file
View File

@ -0,0 +1,170 @@
(ns status-im.test.chat.models.message-list
(:require [cljs.test :refer-macros [deftest is testing]]
[status-im.constants :as const]
[taoensso.tufte :as tufte :refer-macros (defnp p profiled profile)]
[status-im.chat.models.loading :as l]
[status-im.chat.models.message-list :as s]))
(deftest message-stream-tests
(testing "building the list"
(let [m1 {:from "1"
:clock-value 1
:message-id "message-1"
:whisper-timestamp 1
:outgoing false}
m2 {:from "2"
:clock-value 2
:message-id "message-2"
:whisper-timestamp 2
:outgoing true}
m3 {:from "2"
:clock-value 3
:message-id "message-3"
:whisper-timestamp 3
:outgoing true}
messages (shuffle [m1 m2 m3])
[actual-m1
actual-m2
actual-m3] (vals (s/build messages))]
(testing "it sorts them correclty"
(is (= "message-3" (:message-id actual-m1)))
(is (= "message-2" (:message-id actual-m2)))
(is (= "message-1" (:message-id actual-m3))))
(testing "it marks only the first message as :first?"
(is (:first? actual-m1))
(is (not (:first? actual-m2)))
(is (not (:first? actual-m3))))
(testing "it marks the first outgoing message as :first-outgoing?"
(is (:first-outgoing? actual-m1))
(is (not (:first-outgoing? actual-m2)))
(is (not (:first-outgoing? actual-m3))))
(testing "it marks messages from the same author next to another with :first-in-group?"
(is (:first-in-group? actual-m1))
(is (not (:first-in-group? actual-m2)))
(is (:first-in-group? actual-m3)))
(testing "it marks messages with display-photo? when they are not outgoing and are first-in-group?"
(is (not (:display-photo? actual-m1)))
(is (not (:display-photo? actual-m2)))
(is (:display-photo? actual-m3)))
(testing "it marks the last message from the same author with :last-in-group?"
(is (not (:last-in-group? actual-m1)))
(is (:last-in-group? actual-m2))
(is (:last-in-group? actual-m3))))))
(def ascending-range (mapv
#(let [i (+ 100000 %)]
{:clock-value i
:whisper-timestamp i
:timestamp i
:message-id (str i)})
(range 2000)))
(def descending-range (reverse ascending-range))
(def random-range (shuffle ascending-range))
(defnp build-message-list [messages]
(s/build messages))
(defnp append-to-message-list [l message]
(s/add l message))
(defnp prepend-to-message-list [l message]
(s/add l message))
(defnp insert-close-to-head-message-list [l message]
(s/add l message))
(defnp insert-middle-message-list [l message]
(s/add l message))
(tufte/add-basic-println-handler! {:format-pstats-opts {:columns [:n-calls :mean :min :max :clock :total]
:format-id-fn name}})
(deftest ^:benchmark benchmark-list
(let [messages (sort-by :timestamp (mapv (fn [i] (let [i (+ 100000 i 1)] {:timestamp i :clock-value i :message-id (str i) :whisper-timestamp i})) (range 100)))
built-list (s/build messages)]
(testing "prepending to list"
(profile {} (dotimes [_ 10] (prepend-to-message-list
built-list
{:clock-value 200000
:message-id "200000"
:whisper-timestamp 21
:timestamp 21}))))
(testing "append to list"
(profile {} (dotimes [_ 10] (append-to-message-list
built-list
{:clock-value 100000
:message-id "100000"
:whisper-timestamp 100000
:timestamp 100000}))))
(testing "insert close to head"
(profile {} (dotimes [_ 10] (insert-close-to-head-message-list
built-list
{:clock-value 109970
:message-id "109970"
:whisper-timestamp 1000
:timestamp 1000}))))
(testing "insert into the middle list"
(profile {} (dotimes [_ 10] (insert-middle-message-list
built-list
{:clock-value 105000
:message-id "10500"
:whisper-timestamp 1000
:timestamp 1000}))))))
(deftest message-list
(let [current-messages [{:clock-value 109
:message-id "109"
:timestamp 9
:whisper-timestamp 9}
{:clock-value 106
:message-id "106"
:timestamp 6
:whisper-timestamp 6}
{:clock-value 103
:message-id "103"
:timestamp 3
:whisper-timestamp 3}]
current-list (s/build current-messages)]
(testing "inserting a newer message"
(let [new-message {:timestamp 12
:clock-value 112
:message-id "112"
:whisper-timestamp 12}]
(is (= 112
(-> (s/add current-list new-message)
vals
first
:clock-value)))))
(testing "inserting an older message"
(let [new-message {:timestamp 0
:clock-value 100
:message-id "100"
:whisper-timestamp 0}]
(is (= 100
(-> (s/add current-list new-message)
vals
last
:clock-value)))))
(testing "inserting in the middle of the list"
(let [new-message {:timestamp 7
:clock-value 107
:message-id "107"
:whisper-timestamp 7}]
(is (= 107
(-> (s/add current-list new-message)
vals
(nth 1)
:clock-value)))))
(testing "inserting in the middle of the list, clock-value clash"
(let [new-message {:timestamp 6
:clock-value 106
:message-id "106a"
:whisper-timestamp 6}]
(is (= "106a"
(-> (s/add current-list new-message)
vals
(nth 1)
:message-id)))))))

1
test/cljs/status_im/test/data_store/messages.cljs
View File

@ -13,7 +13,6 @@
:response-to-v2 "id-2" :response-to-v2 "id-2"
:text "hta"} :text "hta"}
:whisper-timestamp 1 :whisper-timestamp 1
:js-obj {}
:dedup-id "ATIwMTkwODE0YTdkNWZhZGY1N2E0ZDU3MzUxZmJkNDZkZGM1ZTU4ZjRlYzUyYWYyMDA5NTc2NWYyYmIxOTQ2OTM3NGUwNjdiMvEpTIGEjHOTAyqsrN39wST4npnSAv1AR8jJWeubanjkoGIyJooD5RVRnx6ZMt+/JzBOD2hoZzlHQWA0bC6XbdU=" :dedup-id "ATIwMTkwODE0YTdkNWZhZGY1N2E0ZDU3MzUxZmJkNDZkZGM1ZTU4ZjRlYzUyYWYyMDA5NTc2NWYyYmIxOTQ2OTM3NGUwNjdiMvEpTIGEjHOTAyqsrN39wST4npnSAv1AR8jJWeubanjkoGIyJooD5RVRnx6ZMt+/JzBOD2hoZzlHQWA0bC6XbdU="
:outgoing-status :sending :outgoing-status :sending
:message-type :public-group-user-message :message-type :public-group-user-message

2
test/cljs/status_im/test/runner.cljs
View File

@ -7,7 +7,6 @@
[status-im.test.chat.commands.input] [status-im.test.chat.commands.input]
[status-im.test.chat.db] [status-im.test.chat.db]
[status-im.test.chat.models.input] [status-im.test.chat.models.input]
[status-im.test.chat.models.loading]
[status-im.test.chat.models.message-content] [status-im.test.chat.models.message-content]
[status-im.test.chat.models.message] [status-im.test.chat.models.message]
[status-im.test.chat.models] [status-im.test.chat.models]
@ -85,7 +84,6 @@
'status-im.test.chat.db 'status-im.test.chat.db
'status-im.test.chat.models 'status-im.test.chat.models
'status-im.test.chat.models.input 'status-im.test.chat.models.input
'status-im.test.chat.models.loading
'status-im.test.chat.models.message 'status-im.test.chat.models.message
'status-im.test.chat.models.message-content 'status-im.test.chat.models.message-content
'status-im.test.chat.views.photos 'status-im.test.chat.views.photos

10
test/cljs/status_im/test/transport/core.cljs
View File

@ -58,16 +58,6 @@
:pow 0.002631578947368421 :pow 0.002631578947368421
:hash "0x220ef9994a4fae64c112b27ed07ef910918159cbe6fcf8ac515ee2bf9a6711a0"}})]) :hash "0x220ef9994a4fae64c112b27ed07ef910918159cbe6fcf8ac515ee2bf9a6711a0"}})])
(deftest receive-whisper-messages-test
(testing "an error is reported"
(is (nil? (:chat-received-message/add-fx (message/receive-whisper-messages {:db {}} "error" #js [] nil)))))
(testing "messages is undefined"
(is (nil? (:chat-received-message/add-fx (message/receive-whisper-messages {:db {}} nil js/undefined nil)))))
(testing "happy path"
(let [actual (message/receive-whisper-messages {:db {}} nil messages sig)]
(testing "it add an fx for the message"
(is (:chat-received-message/add-fx actual))))))
(deftest message-envelopes (deftest message-envelopes
(let [chat-id "chat-id" (let [chat-id "chat-id"
from "from" from "from"