All the code has been implemented in statusgo: status-im/status-go#1466
Basically all the whisper filter management is done at that level.
Technical description
On startup we load all chats and send a list of them to status go:
For a public chat: {:chatId "status"}, we create a single filter, based on the name of the chat.
For each contact added by us, each user in a group chat and each one to one chat open, we send:
{:chatId "0x", :oneToOne true}. This will create a chats, to listen to their contact code.
Any previously negotiated topic is also returned.
Once loaded, we create our filters, and upsert the mailserver topics, both of which are solely based on the filters loaded.
In order to remove a chat, we delete/stopwatching first the the filter in status-react and then ask status-go to remove the filter. For a public chat we always remove, for a one-to-one we remove only if the user is not in our contacts, or in a group chat or we have a chat open. Negotiated topics are never removed, as otherwise the other user won't be able to contact us anymore.
On stopping whisper we don't have to ask status-go to remove filters as they are removed automatically.
Some more logic can be pushed in status-go, but that will be in subsequent PRs.
Signed-off-by: Andrea Maria Piana <andrea.maria.piana@gmail.com>
- replace web3-prototype wherever possible
- currently only the money namespace is left
for future refactoring, the ideal solution
would be to use strings for big numbers all
the time and only convert for arithmetic operations
- use json-rpc call to replace trivial web3 calls
Signed-off-by: yenda <eric@status.im>
- upgrade to realm 2.28 to benefit from perf improvements
- remove user-statuses and replace by seen and outgoing-status fields
to get rid of a lot of bloat queries and computations
- remove unused seen message, bottom-infos
- remove unused fields in transport schema
- use objectForPrimaryKey whenever possible instead of get by field
Signed-off-by: yenda <eric@status.im>
Connect to stubs of status-go protocol API, behind the flag. Since status-go isn't updated yet, setting this flag will break the app.
What needs to be tested is no regressions in a normal mode.
Signed-off-by: Igor Mandrigin <i@mandrigin.ru>
the defn macro knows register the function as an events under the
keywords provided in the :events key of the attributes map. It also
adds the interceptors provided in the :interceptors map
exemple:
```clojure
(fx/defn hello4
{:doc "this function is useless as well"
:events [:test/valid1 :test/valid2]}
[{:keys [db]} b]
{:db (assoc db :a b) :b (:a db)})
```
Signed-off-by: yenda <eric@status.im>
move utils.ethereum.tokens to ethereum.tokens
move utils.ethereum.abi-spec to ethereum.abi-spec
move utils.ethereum.core to ethereum.core
move utils.ethereum.eip165 to ethereum.eip165
move utils.ethereum.eip55 to ethereum.eip55
move utils.ethereum.eip681 to ethereum.eip681
move utils.ethereum.ens to ethereum.ens
move utils.ethereum.erc721 to ethereum.erc721
move utils.ethereum.mnemonics to ethereum.mnemonics
move utils.ethereum.resolver to ethereum.resolver
move utils.ethereum.macros to ethereum.macros
Signed-off-by: yenda <eric@status.im>
- introduce json-rpc namespace, which provides `call` and `eth-call`,
a generic way of calling a json-rpc method taking care of conversions
and error handling
- remove web3 usage from wallet
- clean up effects, reducing the amount of computations when login in
`wallet-autoconfig-token` is a very expensive call on mainnet
because it checks the balance of every known token.
it is called:
- when wallet is refreshed by pulling
- when user goes on any wallet screen
this PR changes that by:
- calling it only when the wallet is initialized and there is no
visible-token configuration
it only calls update-wallet when a new transaction arrives
- remove the transaction fetching loop entirely to rely only on subscription
for live transactions and token transfer updates
- fetch token transfers history via etherscan API to lift the 100000 blocks
limit on token transfers history
- inbound token transfers are catched via a filter on ethlogs
- outbound token transfers and other transactions are catched by filtering
transaction in current block that have the wallet address as to or from field
- removes fetching of last 100000 blocks of token transfers from
the wallet pull loop
- fetches the last 100000 blocks of token transfers at startup
- replaces pulling by subscriptions to ethlogs for token transfers
This PR is part of network incentivisation. It adds a way for a client
to pull nodes from a contract.
This is done by selecting the `eth.contract` fleet. If that is selected
on login it will fetch nodes from a contract and pass them to status-go.
If these can't be fetched, it will default to `eth.beta`.
Currently contract information are hard-coded, but eventually the user
will be able to add their own (probably).
Toggled off in release.
Signed-off-by: Andrea Maria Piana <andrea.maria.piana@gmail.com>
[#7454] fix add basic copy to public chat empty screen state + chat messages-views intro screens for all chats
Signed-off-by: Igor Mandrigin <i@mandrigin.ru>
- support for ipfs only
- provides fns to encode and decode contenthashes as defined in EIP1577
- provides cat fx to retrieve contenthash
Signed-off-by: yenda <eric@status.im>
Sometimes it happens that the expired signal is received while the
there's a new request in flight.
This happens in cases such as:
1) We send a request (A)
2) We get disconnected from the mailserver
3) We connect to a new mailserver
4) We send a request (B)
5) We receive an expired signal for A
In such cases the request should not be retried or counted as a failure.
Signed-off-by: Igor Mandrigin <i@mandrigin.ru>
Currently the separate topic was not used, as it's a bit tricky to
coordinate when multiple devices from different versions are present,
with the partitioned topic, probably this optimisation is not necessary
anymore, so removing this for now.
Signed-off-by: Andrea Maria Piana <andrea.maria.piana@gmail.com>
When we are offline, we don't try to change mailserver, and we don't
show a pop up to the user, as it is not that the mailserver is not
working, we are just offline.
Signed-off-by: Andrea Maria Piana <andrea.maria.piana@gmail.com>
The denormalized-clock-value was erroneously set to the one of the last
message received. This meant that on chats were the clock-value raced
ahead of the timestamp (#status), a message from the mailserver or a
message from someone with an old clock-value would basically make those
messages be sorted in the past.
The correct behavior is that last-clock-value for a given chat should be
the maximum last clock value ever seen for that chat.
Signed-off-by: Andrea Maria Piana <andrea.maria.piana@gmail.com>
We keep tokens synchronized across devices, so that the user can notify
us on any paired device.
Currently we record the installation id associated to the fcm-token even
though is not necessary, but it will be once we send device-to-device
messages, in which case we want to notify only those devices.
Signed-off-by: Andrea Maria Piana <andrea.maria.piana@gmail.com>
Currently it's very easy for contact details to get out of sync, the
simplest example is:
A & B are contacts.
A changes name.
B receives the updated name.
B re-install the app.
Until A changes name again, B will not see their name, picture and won't
be able to send push notifications.
This PR changes the behavior to publish account informations to contacts
every 24 hrs, to add some redundancy in this cases.
It also publishes a contact code every 12hrs.
Signed-off-by: Andrea Maria Piana <andrea.maria.piana@gmail.com>
Currently we use a single topic for discovery.
This provides the best obscurity at the cost of bandwidth, as a message
sent on the discovery topic will be received by any peer.
This PR changes this behavior and start listening on a partitioned
topic.
Each pk will be hashed to a limited number of topics.
Everytime someone is in a conversation with someone from another topic
they will have to listen as well to avoid loosing obscurity, because we
only forward messages that we also advertise in the bloom filter.
The choice for the number of partitions depends on 2 factors:
1) The expected number of users using the network
2) The average number of contacts each user
Any change to the discovery topic will need to be split across 3
releases, to avoid breaking compatibility:
1) Listen to the new and old topic, publish to the old topic
2) Listen to the new and old topic, publish to the new topic
3) Listen to the new topic, publish to the new topic
This is step 1.
This PR enables pairing outside of dev-mode and contact-recovery, which
is useful in the case a new device is added or re-installed.
Signed-off-by: Andrea Maria Piana <andrea.maria.piana@gmail.com>
As per announcement, we need to switch our Infura project IDs.
> As previously announced, accessing Infura will begin requiring a Project ID generated from the new Infura Dashboard. If you are using Infura and have not yet migrated your project, please take the time to do so now. The first milestone in this transition will be activated next week on January 23, 2019 at 20:00 UTC.
https://blog.infura.io/infura-dashboard-transition-update-c670945a922a
The new project is created with ID `f315575765b14720b32382a61a89341a`
and the API keys are updated.
Signed-off-by: Igor Mandrigin <i@mandrigin.ru>
When someone is sending a pfs message to us but did not include our own
device, a pop up is shown propmting the user to connect with the user.
The reason for receiving messages that are not targeting our devices are
various:
1) The account was just recovered (which means it is a new installation
id)
2) More than 3 devices are in use (we only keep max 3 devices in sync)
3) The sender has used an old bundle which does not include the current
device
Eventually we will reduce the likelihood of this scenario happening, but
we can't dismiss it completely.
It's only enabled when PFS is enabled.
Signed-off-by: Andrea Maria Piana <andrea.maria.piana@gmail.com>
This commit moves group chats to their own topic, based on the randomly
generated chat-id. It falls back on the discovery topic for those peers
who we can't fingerprint the version, for backward compatibility.
Members will now have to explicitly join a group chat to start receiving
messages from it.
Messages are still sent to users who have not joined for backward
compatibility.
Group updates are unaffected.
The bug was introduced in #7055.
`message-type` was stored and used instead of `content-type` which
caused incorrect displaying of the last message preview if the one was a
command.
This commit changes the way how keys are restored:
1. batch of `ssh.addSymKey` requests for all sym keys is sent at once
2. `:status-im.transport.core/sym-keys-added` event is dispatched with
results of all successful `ssh.addSymKey` calls
3. filter is created via `ssh.newMessageFilter`
4. `:shh.callback/filters-added` event is dispatched with all added
filters as a parameter
5. profit
In ideal case only 2 `re-frame` events are dispatched.
In order to get `:last-clock-value` one extra query was executed for
each chat during initialization.
Implementation:
- `:last-clock-value` field was added to `chat` entity
- this field is updated when the message is sent/received
- extra query was removed
The last messages of the chats are necessary to properly show the chat
list, which is shown right after signing in. Before this commit, the
last message was retrieved as one of 20 last messages fetched for each
chat.
Implementation:
- `:last-message-content` and `:last-message-type` fields were added to
`chat` entity
- both fields are updated when messages are received/sent
- loading of the last 20 messages for each chat was removed as
initialization step
This commit changes the way how/when the node is started/stopped:
1. `node` is not started on the app startup
2. When the user presses "Sign in" button the node is started
with user specific configs (`InstallationID`, custom bootnodes, etc),
and only after that `Login` call is performed.
3. When the user creates a new account, at first the node is started
with default params (the same as would be used when user signs into
the app after account creation whithout changing any setting), then
`CreateAccount` call happens, then `Login`.
4. When the user restores their account, the flow is the same as `3`
but with `RecoverAccount` instead of `CreateAccount`
5. When the user logs out the node is stopped. That's it.
We now check that we are only connected to some `peers` instead of using `NetInfo` from `react-native`.
This is because it has been reported to be quite flaky at times, not reporting online status after sleeping, and for privacy concerns (on ios it pings `apple.com`, on desktop `google.com`).
Adds a new banner `Wallet Offline` and change `Connecting to peers` to `Chat offline`.
A message will be marked as `Sent` only if it made it to the mailserver you are connected to, which will increase the guarantees that we can make about a message (if you see it as sent, it has reached at least a mailserver), this has the consequence that:
- If you are not connected to any mailserver or the mailserver is non responsive/down, and you send a message, it will be marked as `Not sent`, although it might have been actually made it in the network.
Probably this is something that we would like to communicate to the user through UX (i.e. tick if made it to at least a peer, double tick if it made to a mailserver )
Currently I have only enabled this feature in nightlies & devs, I would give it a run and see how we feel about it.
unread-messages-count fn, based off Maciej's comment on #6749
adding test, some difficulty running the test suite locally via clj
actually use the new functionality for displaying on desktop
first pass at updated styling for unread count chat icons
styling pass 2: finer attention to detail for the purple/blue icons
a bit more padding bloat to accomodate for the number 1, which occurs
frequently :)
remove counter from "home" icon on desktop
Signed-off-by: Vitaliy Vlasov <siphiuel@gmail.com>
Implementation:
1. `transport.utils/message-id` function is called only in three places now
and accepts `from` and `raw_payload` as parameters.
ID is calculated as `sha3(from + raw_payload)`.
2. This means that for wrapped private group chat message
the raw payload of `GroupMembershipUpdate` is used.
We add syncing of account fields in pairing messages (only photo-path &
name for now). Also a sync message is sent each time we send a
contact-update, to keep other devices in sync. The change is compatible
with previous clients as it's just an accretion of transit.
Signed-off-by: Andrea Maria Piana <andrea.maria.piana@gmail.com>
Before we fetched ALL user-statuses with `status=received` (which means that
a message hasn't been seen), iterated them, grouped by chat and then stored
`message-ids` of these `user-statuses` in chat's `:unviewed-messages` key.
This commit introduces :unviewed-messages-count field in chat entity.
That means that there is no need to iterate `user-statuses` in order to count
a total number of unviewed messages, it is always stored along with chat.
In the rest of it, the difference is only that chat's db record should be
updated each time when unviewed messages are seen.
Go blocks parse try catch blocks and turn them into event
dispatches. This captures the original intent of the code to catch
errors and terminate the current async worker execution.
Signed-off-by: yenda <eric@status.im>
functions defined in subs need to be reusable and testable independently
from re-frame framework. they are moved into db namespace for that purpose
Signed-off-by: yenda <eric@status.im>
Everytime a contact request is sent/confirmed a sync message is also
sent to other devices so the contact is kept in sync.
Signed-off-by: Andrea Maria Piana <andrea.maria.piana@gmail.com>
- do not logout and remove previous mailserver
from peers when changing mailserver
- rename wnode mailserver
- move transport.inbox to mailserver.core
- fix all subs and db keys
Signed-off-by: yenda <eric@status.im>
Adds a `chat-id` field in `content` map.
The reason it has been added to the map instead of augmenting transit is
that it would simplify the calculation of `message-id`, which in this
case is consistent for both old & new clients.
`chat-id` also represents the `chat-id` with respect of the sender, as
in 1-to-1 chats that is asymmetric.
Signed-off-by: Andrea Maria Piana <andrea.maria.piana@gmail.com>
- chat name is not updated with contact name
- when a contact adds the user and is added back by user, the name
of the chat is still the random name of the contact
- in active-chats subscriptions, replace the name of the chat by the current
name of the contact
- 1-1 chats don't have a topic anymore because they only use the
discovery topic so topic is nil for these chat
- this was causing an error when initializing whisper because the app
was trying to start a filter for each of the chat including the 1-1 with
no topic
- we now filter the transport/chats to only recover sym-key and start filter
for those with a topic
- fetch 7 days of history when joining a chat
- make 7 24h requests to request 7 days because mailservers
ignores requests for a timespan > 24h
- make requests sequentially to avoid timeouts
- change mailserver after 3 timeouts on a request
Signed-off-by: yenda <eric@status.im>
Andrea Maria Piana
Vitaliy Vlasov
Julien Eluard
Roman Volosovskyi
Andrey Shovkoplyas
yenda
Igor Mandrigin
tbenr
Dmitry Novotochinov
bitsikka
mestorlx
kn
Andrea Franz
Pedro Pombeiro
Rob Culliton
Bruce Hauman
Goran Jovic
janherich
Aleksandr Pantiukhov