Commit Graph

1019 Commits

Author SHA1 Message Date
Andrea Franz fdcd20e14d
add missing patch file (#1319) 2018-12-12 16:09:39 +01:00
Adam Babik 5dbde3a94b
fix flaky TestSyncBetweenTwoMailServers (#1316) 2018-12-12 11:21:52 +01:00
Andrea Franz e1c258d041
update go-ethereum with patch 0044-import-extended-key-by-purpose (#1318) 2018-12-12 11:13:10 +01:00
Dmitry Shulyak f2c6fef64c
Persist selected mail server using separate monitor (#1303)
This change allows to connect to the mail server that we were using before the app was restarted. Separate loop is listening for whisper events, and when we receive event that request was completed we will update time on a peer record.

Records are stored in leveldb. Body of the record is marshaled using json. At this point the only field is a timestamp when record was used.

This loop doesn't control connections, it only tracks what mail server we ended up using. It works asynchronously to connection management loop. Which tracks events that are related to connection state and expiry of the requests.

When app starts we look into the database and select the most recently used record. This record is added to connection management loop first. So if this server is available we will stick to using it. If we weren't able to connect to the same server in configured timeout (5s) we will try to connect to any other server from list of active servers.

closes: #1285
2018-12-12 11:39:00 +02:00
Andrea Franz 023d2f4e7a
add ExtendedKey.ChildForPurpose method to derive wallet or chat keys (#1312)
* add ExtendedKey.ChildForPurpose method to derive wallet or chat keys

* Update extkeys/hdkey.go

Co-Authored-By: gravityblast <andrea@gravityblast.com>

* Update extkeys/hdkey_test.go

Co-Authored-By: gravityblast <andrea@gravityblast.com>

* Update extkeys/hdkey_test.go

Co-Authored-By: gravityblast <andrea@gravityblast.com>

* Update extkeys/hdkey_test.go

Co-Authored-By: gravityblast <andrea@gravityblast.com>

* Update extkeys/hdkey_test.go

Co-Authored-By: gravityblast <andrea@gravityblast.com>

* refactoring and comments

* set EIP1581KeyTypeChat type to uint32

* set 0x00 instead of zero to avoid lint errors
2018-12-11 17:01:30 +01:00
Adam Babik e2682486fd
Fix cursor encoding (#1308)
It fixes encoding and decoding cursor sent in historic messages requests. `hex` package from the standard library is used.
2018-12-11 11:23:47 +01:00
Jakub Sokołowski 7d651afaae add desktop build step
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2018-12-11 09:30:35 +01:00
Andrea Maria Piana b676de9dac
Hash password before using it (#1306) 2018-12-10 14:11:19 +01:00
Adam Babik db691fd763
fix golang/mock (#1310) 2018-12-10 13:53:27 +01:00
Rob Culliton 3657b8b057
change echo message for linux compilation (#1307) 2018-12-07 08:35:05 -05:00
Adam Babik 913dbfca9b
Support Mail Server data synchronization (#1302)
These changes add a support for syncing data between two Mail Servers. It does not give an easy way to start syncing. This will be solved in the next PR.
2018-12-06 10:48:28 +01:00
Dmitry Shulyak d51761f83e Ensure sent event is received from a mailserver (#1304)
In previous change i used same filtering for a handler that processes confirmations.
But if we are connected with peers that do not support confirmations on whisper level,
whisper may send "Sent" event when envelope is written to a socket. This is our old behaviour.

So, this PR allows to use confirmations from a mail servers and ensure that transition between
multiple version of peers will be smooth.
2018-12-06 10:30:57 +01:00
Dmitry Shulyak a609b468fe
Mail peer store and connection manager (#1295)
This change implements connection manager that monitors 3 types of events:
1. update of the selected mail servers
2. disconnect from a mail server
3. errors for requesting mail history

When selected mail servers provided we will try to connect with as many as possible, and later disconnect the surplus. For example if we want to connect with one mail server and 3 were selected, we try to connect with all (3), and later disconnect with 2. It will to establish connection with live mail server faster.

If mail server disconnects we will choose any other mail server from the list of selected. Unless we have only one mail server. In such case we don't have any other choice and we will leave things as is.

If request for history was expired we will disconnect such peer and try to find another one. We will follow same rules as described above.

We will have two components that will rely on this logic:
1. requesting history

If target peer is provided we will use that peer, otherwise we will request history from any selected mail server that is connected at the time of request.

2. confirmation from selected mail server

Confirmation from any selected mail server will bee used to send a feedback that envelope was sent.

I will add several extensions, but probably in separate PRs:
1. prioritize connection with mail server that was used before reboot
2. disconnect from mail servers if history request wasn't expired but failed.
3. wait some time in RequestsMessage RPC to establish connection with any mail server

Currently this feature is hidden, as certain changes will be necessary in status-react. 

partially implements: https://github.com/status-im/status-go/issues/1285
2018-12-05 15:57:05 +02:00
Andrea Maria Piana afc3017e07
Notify user when the device is missing (#1298)
* Notify user when the device is missing

* Update services/shhext/chat/encryption.go

Co-Authored-By: cammellos <andrea.maria.piana@gmail.com>
2018-12-05 09:22:49 +01:00
Jakub Sokołowski 49cfcd3e24 stop adding 'v' before the contents of VERSION file
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2018-12-03 14:55:45 +01:00
Pedro Pombeiro b72d7d1950 Update version to 0.18.0
- due to API breaking change on `NotifyUsers`
2018-12-03 11:30:13 +01:00
Pedro Pombeiro cddf2e1c6c Change `NotifyUsers` to allow only data JSON element. Part of status-im/status-react#6772 2018-11-30 13:26:27 +01:00
Adam Babik e447750f4d
Bump 0.17.2-beta 2018-11-29 15:06:48 +01:00
Adam Babik 3ab928169e
Release 0.17.1 2018-11-29 15:05:30 +01:00
Pedro Pombeiro 055486f0ca Log release and commit hash 2018-11-28 19:18:20 +01:00
Andrea Maria Piana 38bb4d8ef3
Add versioning & tests, migrate db files (#1293)
We are preparing for the release of this to general public, so a few
things have been added:

1) Add versioning for bundles, and make refresh interval configurable
2) Move files to installationID so no metadata is leaked
3) Re-key using user password db
2018-11-28 12:34:39 +01:00
Dmitry e60dbe3c1b Update selected mail servers that are used in envelope tracker 2018-11-28 11:16:37 +02:00
Jakub Sokołowski d61e838235 use VERSION file for suffix when releasing
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2018-11-27 19:26:09 +01:00
Andrea Maria Piana 6112ca0289
Add tests for multi-device and refactor encryption service config (#1277) 2018-11-27 09:54:20 +01:00
Dmitry Shulyak aac706fe4c
Use confirmation that bundle is delivered for sending feedback to user (#1284)
* Pull whisper confirmations changes

* Use batch confirmations as a signal that envelope was sent into the network
2018-11-27 08:30:15 +02:00
Jakub Sokołowski 6b09eea749 install xgo in setup target
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2018-11-26 14:07:48 +01:00
Jakub Sokołowski 07eb60df7d
pyt timestmap before commit for better ordering
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2018-11-26 13:33:07 +01:00
Jakub Sokołowski def3af61a7
remove unused call
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2018-11-26 09:05:33 +01:00
Jakub Sokołowski d1847a3adc change Jenkinsfile to use parallel declarative format
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2018-11-22 10:12:20 +01:00
Pedro Pombeiro 134cce5b13 Reduce number of days required to consider a PR stale to 14 2018-11-20 13:14:33 +01:00
Adam Babik e658366d1e
Improve Makefile (#1275)
* change docker image tag to RELEASE_TAG
* use changelog in release
2018-11-20 11:13:42 +01:00
Pedro Pombeiro d3c8079075 Update mailserver canary to be compatible with geth 1.8.17 upgrade 2018-11-20 08:59:36 +01:00
Pedro Pombeiro 92748f1334 Add `canary-test` make target to test beta fleet mailservers. Closes #1274 2018-11-19 19:02:32 +01:00
Adam Babik 860da591be
add mailserver batch requests counter and network processing time (#1280) 2018-11-19 10:14:03 +01:00
Adam Babik 8efd1fe84e
Bump version to 0.17.1-beta 2018-11-14 08:18:59 +01:00
Adam Babik 52a1bdfed6
Upgrade geth 1.8.17 plus add metrics during compilation time (#1273)
This commit updates geth to 1.8.17 and adds a possibility to enable metrics during compilation time.

The cascade of issues forced us to upgrade geth to 1.8.17 in order to allow enabling metrics during compilation time. 1.8.17 introduced `NodeID` refactoring and `enode` package which affected our peers pool and integration with Discovery V5.
2018-11-14 08:03:58 +01:00
Adam Babik 5b2d7dc212
bump version to 0.16.8-beta 2018-11-14 01:31:47 +01:00
Adam Babik 941790f357
bump version to 0.16.7 2018-11-14 01:08:09 +01:00
Dmitry Shulyak cb15ca6e74
Mark peers that were added by peer pool (#1271)
* Mark peers that were added by peer pool

* Implement thorough test to verify that connection was indeed preserved

* Add more debug logs and remove caching of the peers
2018-11-13 14:58:26 +01:00
Adam Babik 04ba4d44e8
fix collecting p2p/Peers metric (#1269) 2018-11-12 10:27:09 +01:00
Pedro Pombeiro 5d0ed0303e Add timeout parameter to node-canary 2018-11-09 17:02:55 +01:00
Pedro Pombeiro 87f1af0743 Ensure that `NewNodeConfigWithDefaults` builds a valid configuration 2018-11-09 17:02:55 +01:00
Andrea Maria Piana 1f6cccd0fc
Add enabling/disabling of installations (#1264)
This commit adds a list new table, installations, which is used to keep
track of which installation are active for a given identity key.

In general, we limit the number of installation that we keep
synchronized to 5, to avoid excessive usage of resources.

Any installation coming from our own identity, will have to be manually
enabled, otherwise we trust the other peer has correctly paired their
devices.

We use a timestamp to decide which installations to keep synchronized as
a logical clock would have make the creation of the bundle more
complicated, but this can always be converted to a logical clock at
later stages without breaking compatibility.
2018-11-06 09:05:32 +01:00
Dmitry Shulyak 3fe5b25ff2
Implement eth_getFilterLogs (#1265)
Queries logs from remote server using original filter criteria.
2018-11-06 07:41:36 +01:00
Dmitry Shulyak db786ef1d2
Sign typed data implementation (#1250)
* Implement EIP 712

* Cover majority of cases with tests

* Add solidity and signer tests and cover integer edges case

* Add thin api to sign type data using selected status account

* All integers are extended to int256 and marshalled into big.Int

* Document how deps works

* Fix linter

* Fix errors test

* Add validation tests

* Unmarshal every atomic type in separate functions
2018-11-06 07:26:12 +01:00
Andrea Maria Piana ee3c05c79b
Change handling of skipped/deleted keys & add version (#1261)
- Skipped keys

The purpose of limiting the number of skipped keys generated is to avoid a dos
attack whereby an attacker would send a large N, forcing the device to
compute all the keys between currentN..N .

Previously the logic for handling skipped keys was:

- If in the current receiving chain there are more than maxSkip keys,
throw an error

This is problematic as in long-lived session dropped/unreceived messages starts
piling up, eventually reaching the threshold (1000 dropped/unreceived
messages).

This logic has been changed to be more inline with signals spec, and now
it is:

- If N is > currentN + maxSkip, throw an error

The purpose of limiting the number of skipped keys stored is to avoid a dos
attack whereby an attacker would force us to store a large number of
keys, filling up our storage.

Previously the logic for handling old keys was:

- Once you have maxKeep ratchet steps, delete any key from
currentRatchet - maxKeep.

This, in combination with the maxSkip implementation, capped the number of stored keys to
maxSkip * maxKeep.

The logic has been changed to:

- Keep a maximum of MaxMessageKeysPerSession

and additionally we delete any key that has a sequence number <
currentSeqNum - maxKeep

- Version

We check now the version of the bundle so that when we get a bundle from
the same installationID with a higher version, we mark the previous
bundle as expired and use the new bundle the next time a message is sent
2018-11-05 20:00:04 +01:00
Andrea Maria Piana 58bd36e79e
Automate github releases (#1263) 2018-11-05 15:02:16 +01:00
Adam Babik 0961e1039e
fix peerID logging in mail server (#1260) 2018-10-28 23:33:58 +01:00
Dmitry Shulyak 539fa01d48
Implement cache for recording removed logs due to reorg (#1241)
Adjust criteria if replaced logs were received and add more tests

Improve validation

Changes after review
2018-10-28 17:12:16 +01:00
Andrea Maria Piana 22eb53ea8c shhext: increase max skip keys (#1257) 2018-10-26 10:44:10 +02:00