Commit Graph

106 Commits

Author SHA1 Message Date
Adam Babik 1ab2e88bf5
Sync messages in a loop until error or no cursor (#1502) 2019-06-26 18:17:41 +02:00
Andrea Maria Piana 5335a2b4fd
Move installations to status-go (#1499)
* Move installations to status-go

This commit moves installations management/storage to status-go.
We remove the native binding and provide RPC endpoints to set the
metadata and return a list of our own installations.
2019-06-26 11:32:59 +02:00
Andrea Maria Piana 0ade9a6cbb
Cache keys (#1497)
* Cache keys

Generating a symkey can take up to a second on slow devices, this commit
makes so that keys are saved once generated and stored in the database.
2019-06-24 09:26:25 +02:00
Andrea Maria Piana f6fba1d3d6 Publish contact code periodically 2019-06-19 17:38:45 +02:00
Andrea Maria Piana 1aa3e2812a Add support for partitioned topic 2019-06-19 17:38:45 +02:00
Andrea Maria Piana 78ed35d2fe Add protocol version to bundle 2019-06-19 17:38:45 +02:00
Andrea Maria Piana cef7f367ab Add topic negotiation
This commit add topic negotiation to the protocol.

On receiving a message from a client with version >= 1, we will generate
a shared key using Diffie-Hellman. We will record also which
installationID has sent us a message.

This key will be passed back to the above layer, which will then use to
start listening to a whisper topic (the `chat` namespace has no
knowledge of whisper).

When sending a message to a set of InstallationIDs, we check whether we
have agreed on a topic with all of them, and if so, we will send on this
separate topic, otherwise we fallback on discovery.

This change is backward compatible, as long as there is no downgrade of
the app on the other side.

A few changes:

* Factored out the DB in a separate namespace as now it is
being used by multiple services (TopicService and EncryptionService).

* Factored out multidevice management in a separate namespace

* Moved all the test to test the whole protoocl rather than just the encryption service

* Moved all the filter management in status-go
2019-06-19 17:38:45 +02:00
Dmitry Shulyak 047c9b5263
Download transfers starting from latest block header (#1467) 2019-06-14 13:16:30 +03:00
Adam Babik 1c33220834 update linter 2019-06-12 13:12:00 +02:00
Adam Babik c9e99c432d migrate to go 1.12 and go modules 2019-06-12 13:12:00 +02:00
Dmitry Shulyak 4cafe63ab7
Unmarshall wrapped into string integers as string (#1470) 2019-05-24 13:22:13 +03:00
Dmitry Shulyak d32cd18c09
Fix channel subscriber that was blocking sending new requests (#1465)
In RequestMessagesSync subscriber is listening to a feed where all whisper
events are posted. After we received event with a request hash - subscriber will
stop actively consuming messages from a feed, as a subscription channel will
get overflow and whole feed will get blocked.

Some events are posted to a feed before request is sent, so blocked feed results
in blocked sending.

Now we will unsubscribe after relevant event was received, and terminate subscriber
explicitly by timeout.
2019-05-20 11:10:26 +03:00
Andrea Maria Piana 4ab08629f6 Add postgres
This commits adds support for postgres database.
Currently two fields are stored: the bloom filter and the topic.
Only the bloom filter is actually used to query, but potentially we will
use also the topic in the future, so easier to separate it now in order
to avoid a migration.
2019-05-15 11:01:34 +02:00
Igor Mandrigin 10fc860a5f
fix parameters parsing for eth_* filters subscriptions (#1461) 2019-05-10 12:56:58 +02:00
Andrea Maria Piana 061f10e58d
Publish rlp.RawValue instead of envelope (#1459)
As part of a performance profiling of mailserver we noticed that most of
the resources on a query are spend decoding the whisper envelope.
This PR changes the way we store envelopes encoding the Topic into the
database key, so we can check that and we are able to publish the
envelope rawValue if it matches.
The change is backward compatible as only newly added envelopes will
have the new key, while old ones will have to be unmarshaled.
2019-05-09 12:58:02 +02:00
Igor Mandrigin 4c1f96d255
Implement subscription for eth and shh filters using signals. (#1455) 2019-05-07 09:05:38 +02:00
Dmitry Shulyak 14c513bd5a
Execute writes atomically only after request was processed without errors (#1454)
* Replace request ID when same request is restarted

* Remove unnecessary changes

* Execute all writes atomically only if request was processed succesfully

* Fix linter

* Fix shadowed errors

* Fix spelling

* Do not append same reference to a byte slice
2019-05-06 09:33:19 +03:00
Dmitry Shulyak cba00153e2 Replace request ID when same request is restarted (#1453) 2019-05-01 12:33:18 +02:00
Dmitry Shulyak 218a35e609
Notify users that envelope was discarded and retry sending it (#1446)
API for querying mail servers that skips already received data on subsequent requests
2019-04-30 09:46:12 +03:00
Andrea Maria Piana 354e6981ba
Add network incentivisation service (#1452) 2019-04-29 14:05:49 +02:00
Dmitry Shulyak a904d9325e
Notify users that envelope was discarded and retry sending it (#1424)
* Notify users that envelope was discarded and retry sending it

* Update Gopkg files with released whisper version

* Forgot to remove signal after refactoring
2019-04-02 13:40:45 +03:00
Andrea Franz 8fe14e8f23
hash typed data (#1426)
* add HashTypedData to backend

* add HashTypedData to backand, lib, and mobile

* rename typeddata.Hash to typeddata.ValidateAndHash
2019-03-28 15:56:21 +01:00
Adam Babik 4b790adf34
add shhext_requestMessagesNew (#1412) 2019-03-15 14:27:08 +01:00
Dmitry Shulyak aa28f652e3 Re-send user payload if previously sent envelope wasn't acknowledged (#1386)
* Split shhext.tracker into envelopes and mail monitors

* Send envelopes on every new attempt to deliver a message

* Re-send user payloads if previous envelopes weren't acknowledged

* Remove debug api across the codebase
2019-03-01 14:36:21 +01:00
Andrea Maria Piana 38e5335e18
Allow messages to be decrypted from previous bundles (#1400)
Currently we only decrypt messages if received on the current bundle.
This changes the behavior so that messages can be decrypted if sent to
previous bundles as well, as otherwise is a bit restrictive
2019-02-28 13:09:43 +01:00
Dmitry ccce8b9a80 Clear all request internal cache when node is stopped 2019-02-27 07:44:38 +02:00
Andrea Maria Piana e88e590028
Fix wrong datadir in desktop (#1389)
Desktop was wrongly passing a network dependent data-dir (mainnet_rpc).
The code migrates the db if present, moving it to the new location.
2019-02-25 08:51:25 +01:00
Dmitry f2a400dc44 Split shhext.tracker into envelopes and mail monitors 2019-02-22 09:33:46 +02:00
Dmitry Shulyak 224bc7857c Mark envelope as confirmed when it was received from a mail server (#1378) 2019-02-20 18:10:59 +01:00
Dmitry 381a1941a0 Unregister request whisper failed to send it 2019-02-20 16:09:45 +02:00
Andrea Maria Piana 81d8ca82a2
Add confirm messages processed by ID (#1375)
Currently PFS messages are decrypted and therefore modified before being
passed to the client. This make IDs computation difficult, as we pass
the whole object to the client and expect the object be passed back once
confirmed.
This changes the behavior allowing confirmation by ID, which is passed
to the client instead of the raw object.
This is a breaking change, but status-react is already forward
compatible.
2019-02-19 13:58:42 +01:00
Andrea Maria Piana 2a4382369a
Add GetContactCode call, add DH flag (#1367)
This PR does a few things:

1) Add a call GetContactCode to check whether we have a bundle for a
given user.

2) Add a DH flag to the API (non-breaking change), for those messages
that we want to target all devices (contact-requests for example).

3) Fixes a few small issues with installations, namely if for example a
messages is sent without a bundle (currently not done by any client),
we still infer installation info, so that we can communicate securely
and making it truly optional.
2019-02-12 12:07:13 +01:00
Ivan Daniluk 4f3f5ee574 Gomobile support (#1164) 2019-02-01 18:02:52 +01:00
Andrea Franz 0e0c3cd859
add shhext.InitProtocolWithEncryptionKey (#1362)
* add shhext.InitProtocolWithEncryptionKey

* update test to avoid lint error
2019-01-25 11:31:51 +01:00
Dmitry c8a616688c Add request with retries api call 2019-01-24 14:31:56 +02:00
Andrea Maria Piana 2df5422337
Add backward compatibility for single bundle (#1355)
We change the protocol to accomodate publishing multiple bundles, in
order to propagate bundles for group chats and have a way to extend it
further.
This commit re-introduces backward compatibility for direct messages,
to be removed once that is not an issue anymore.
2019-01-21 12:33:18 +01:00
Andrea Maria Piana ad51c019e8
Fix broken migration (#1354)
The migration from unencrypted->encrypted did not work properly, it was
deleting and re-creating the database. This correctly migrates the db.
2019-01-19 08:30:05 +01:00
Dmitry aa09d854ca Prevent frequent requests with same topics 2019-01-18 14:52:33 +02:00
Andrea Franz 4939268edf
Unlock wallet and chat keys (#1346)
* select account decrypting wallet and chat keys

* adapt account tests to use chat and wallet account/keys

* fix tests using chat address

* changes after review

* fix status service e2e tests

* add account.Info struct returned when creating and recovering an account

* use s.EqualValues to compare recovered accounts

* return Info instead of *Info

* add both address and walletAddress to responses

* Update lib/types.go

Co-Authored-By: gravityblast <andrea@gravityblast.com>

* Update lib/types.go

Co-Authored-By: gravityblast <andrea@gravityblast.com>

* update comment to fix lint
2019-01-18 10:01:14 +01:00
Dmitry Shulyak 863d07f798 Accept status service config directly from json input (#1337) 2019-01-17 13:56:22 +01:00
Adam Babik fd48ad8b29
geth: upgrade geth to 1.8.21 (#1350) 2019-01-16 15:42:00 +01:00
Andrea Maria Piana 59426d628c
Migrate from nokey on ios to re-keyed (#1345) 2019-01-11 11:55:18 +01:00
Roman Volosovskyi :: Darkviolet Lightgreen Halcyon d6216f1aa4 [slow sign in] Reduce number of kdf iterations for PDF database key (#1343)
`kdf_iter` parameter is reduced to 3200. This change is done because of
performance reasons, currently key derivation is too slow on some mobile
devices. The number of iterations before this commit is 64000, default
value in `sqlcipher` from version `3.0.0`.
fda4c68bb4/CHANGELOG.md (300---2013-11-05)

Implementation:
`sqlcipher_export` is used for migration, check out the link below
for details
https://www.zetetic.net/sqlcipher/sqlcipher-api/#sqlcipher_export
2019-01-11 11:12:23 +01:00
Adam Babik fa97e6927d
Add CreateAccount RPC function (#1330) (#1334) 2018-12-28 12:37:22 +01:00
Andrea Maria Piana 30dbacdcca
Initialize public key (#1332) 2018-12-24 08:18:27 +01:00
Andrea Maria Piana 3292538b41
Allow multiple bundles, change n devices & bundle refresh (#1331)
Change to support sending multiple bundles, as needed for group chats,
limit number of devices to 3 as already done in the UI and refresh
bundle daily.
2018-12-21 11:07:25 +01:00
Andrea Maria Piana 79bf92bea5
Allow specify topic in rpc endpoint (#1328) 2018-12-18 16:27:12 +01:00
Adam Babik 74cb16c456
add SyncMessages method to shhext api (#1309) 2018-12-14 12:21:34 +01:00
Dmitry Shulyak f2c6fef64c
Persist selected mail server using separate monitor (#1303)
This change allows to connect to the mail server that we were using before the app was restarted. Separate loop is listening for whisper events, and when we receive event that request was completed we will update time on a peer record.

Records are stored in leveldb. Body of the record is marshaled using json. At this point the only field is a timestamp when record was used.

This loop doesn't control connections, it only tracks what mail server we ended up using. It works asynchronously to connection management loop. Which tracks events that are related to connection state and expiry of the requests.

When app starts we look into the database and select the most recently used record. This record is added to connection management loop first. So if this server is available we will stick to using it. If we weren't able to connect to the same server in configured timeout (5s) we will try to connect to any other server from list of active servers.

closes: #1285
2018-12-12 11:39:00 +02:00
Adam Babik e2682486fd
Fix cursor encoding (#1308)
It fixes encoding and decoding cursor sent in historic messages requests. `hex` package from the standard library is used.
2018-12-11 11:23:47 +01:00