50 Commits

Author SHA1 Message Date
Adam Babik
a636f33109
Set Whisper version submodule to v6 (#1725) 2019-12-09 11:36:14 +01:00
Adam Babik
4ac4a61e20
Move whisper to status-go monorepo (#1720) 2019-12-09 11:06:04 +01:00
Pedro Pombeiro
26b6d7c36a Create a home submodule for Eth node bridges- Rename StatusBackend to GethStatusBackend 2019-11-27 17:02:09 +01:00
Adam Babik
26880b83d7
Upgrade geth to 1.9.5 and Whisper (#1617) 2019-10-04 17:21:24 +02:00
Dmitry Shulyak
be9c55bc16
Accounts data management (#1530)
* WIP accounts implementation

* Accounts datasore and changes to status mobile API

* Add library changes and method to update config

* Handle error after account selection

* Add two methods to start account to backend

* Use encrypted database for settings and add a service for them

* Resolve linter warning

* Bring back StartNode StopNode for tests

* Add sub accounts and get/save api

* Changes to accounts structure

* Login use root address and fetch necessary info from database

* Cover accounts store with tests

* Refactor in progress

* Initialize status keystore instance before starting ethereum node

* Rework library tests

* Resolve failures in private api test and send transaction test

* Pass pointer to initialized config to unmarshal

* Use multiaccounts/accounts naming consistently

Multiaccount is used as a login identifier
Account references an address and a key, if account is not watch-only.

* Add login timestamp stored in the database to accounts.Account object

* Add photo-path field for multiaccount struct

* Add multiaccoutns rpc with updateAccount method

Update to any other account that wasn't used for login will return an error

* Fix linter in services/accounts

* Select account before starting a node

* Save list of accounts on first login

* Pass account manager to accounts service to avoid selecting account before starting a node

* Add logs to login with save and regualr login
2019-08-20 18:38:40 +03:00
Andrea Maria Piana
cb769ccca9 Revert "Initialize status keystore instance before starting ethereum node"
This reverts commit 476240fbd107fe193096350f5181c32c0539eff0.
2019-08-05 16:03:18 +02:00
Dmitry
476240fbd1 Initialize status keystore instance before starting ethereum node 2019-08-01 11:01:02 +03:00
Adam Babik
8583eb2f37
fix TestSyncBetweenTwoMailServers flakiness 2019-02-22 10:24:06 +01:00
Adam Babik
fd48ad8b29
geth: upgrade geth to 1.8.21 (#1350) 2019-01-16 15:42:00 +01:00
Adam Babik
74cb16c456
add SyncMessages method to shhext api (#1309) 2018-12-14 12:21:34 +01:00
Adam Babik
5dbde3a94b
fix flaky TestSyncBetweenTwoMailServers (#1316) 2018-12-12 11:21:52 +01:00
Adam Babik
913dbfca9b
Support Mail Server data synchronization (#1302)
These changes add a support for syncing data between two Mail Servers. It does not give an easy way to start syncing. This will be solved in the next PR.
2018-12-06 10:48:28 +01:00
Adam Babik
52a1bdfed6
Upgrade geth 1.8.17 plus add metrics during compilation time (#1273)
This commit updates geth to 1.8.17 and adds a possibility to enable metrics during compilation time.

The cascade of issues forced us to upgrade geth to 1.8.17 in order to allow enabling metrics during compilation time. 1.8.17 introduced `NodeID` refactoring and `enode` package which affected our peers pool and integration with Discovery V5.
2018-11-14 08:03:58 +01:00
Dmitry
5b551c67fa Use status-im whisper/whisperv6 module everywhere 2018-10-03 09:27:51 +03:00
Pedro Pombeiro
3d00af7fa3
Streamline configuration in status-go. Part of #1180 (#1183)
- Replace command line flags with `-c` config flag. Part of #1180
- Convert node config private keys to hex-encoded string versions.
- Remove `GenerateConfig` from library.
- Remove unused `FirebaseConfig` from library.
- Fix loading of `config/status-chain-genesis.json` in non-dev machines.
2018-09-13 18:31:29 +02:00
Pedro Pombeiro
fb1e30c5f4 Disable NTP sync on the remaining e2e tests 2018-08-07 18:02:42 +02:00
Pedro Pombeiro
840b5b64d2 Implement mailserver canary service. Closes #1086 2018-08-07 17:17:59 +02:00
Adam Babik
71b8e0e73d
Remove shh and shhext from APIModules (#1131)
Addresses #912. By removing these two modules from APIModules, we disallow DApps from using dangerous methods like shh.SetMinPoW
2018-08-02 08:48:44 +02:00
Adam Babik
384567213c
fix flaky TestRequestMessageFromMailboxAsync test (#1105) 2018-07-20 08:34:38 +02:00
Andrea Franz
8ce522f430
remove old methods addPeerSync from tests (#1080) 2018-07-04 17:53:02 +02:00
Andrea Franz
6d1bcff446
use envelopeFeed to wait for envelopes to be available in tests (#1079)
* remove sleeps in WhisperMailboxSuite

* use envelopeFeed to wait for envelopes to be available in group chat test

* remove unsed envelopeTracer in whisper e2e tests

* unsubscribe from envelopeFeed

* use s.addPeerSync instead of WaitForPeerAsync
2018-07-04 13:07:40 +02:00
Adam Babik
38a60135b2 Add asymmetric key support for MailServer requests (#1075)
* add Asymmetric Key support for MailServer requests

* remove deprecated notice

* fix linter

* refactoring Whisper config related to MailServer

* fix race condition
2018-07-04 11:30:57 +02:00
Andrea Franz
809db97e54
mailserver pagination (#1039)
* mailserver sends envelopes in descending order

* add limit value in mailserver request payload

* mailserver sends messages up to the limit specified in the request

* update Archive method to return key and error

* processRequest returns the next page cursor

* add cursor to mailserver request

* add limit and cursor to request payload

* fix request limit encoding

* wait for request completed event in TrackerSuite/TestRequestCompleted

* add cursor to mailserver response

* fix cursor position in payload

* add e2e test for mail server pagination

* validate mail server response size

* remove old limitReached var

* fix lint warnings

* add whisper patch

* fix tests after rebase

* check all return values to avoid lint warnings

* check that all messages have been retrieved after 2 paginated requests

* fix lint warnings

* rename geth patch

* merge mailserver patches into one

* add last envelope hash to mailserver response and EventEnvelopeAvailable event

* update whisper patch

* add docs to MailServerResponse

* update whisper patch

* fix tests and lint warnings

* send mailserver response data on EventMailServerRequestCompleted signal

* update tracker tests

* optimise pagination test waiting for mailserver to archive only before requesting

* rollback mailserver interface changes

* refactoring and docs changes

* fix payload size check to determine if a limit is specified

* add more docs to the processRequest method

* add constants for request payload field lengths

* add const noLimits to specify that limit=0 means no limits
2018-07-02 09:38:10 +02:00
Pedro Pombeiro
ab7b9b914c Recover from panics which can get sent from goleveldb due to bad arguments. Closes #1059 2018-06-26 16:49:13 +02:00
Pedro Pombeiro
2b290d1a76 Address PR review comments 2018-06-18 15:24:27 +02:00
Pedro Pombeiro
fac5576988 Use EnvelopeTracer as an event source to avoid relying on timeouts in tests. Fixes #1021
Fix bug in mailserver implementation which was using upper bound `to` parameter as an exclusive, rather than inclusive parameter, as stated in the documentation: https://github.com/status-im/status-go/wiki/Additional-JSON-RPC-API
2018-06-18 15:24:27 +02:00
Andrea Franz
fa390a52ae
RequestHistoricMessages response (#1009)
* refactor TestRequestMessageFromMailboxAsync to use s.requestHistoricMessages helper

* send p2pRequestResponseCode from mailserver

* send p2p message response to after sending all historic messages

* mailserver sends `whisper.NewSentMessage` as response

* add mailserver Client and p2pRequestAckCode watchers

* send event with envelopeFeed when p2pRequestAckCode is received

* test request completed event in tracker

* rename mailserver response events and code to RequestCompleteCode

* wait for mailserver response in e2e test

* use SendHistoricMessageResponse method name for mailserver response

* fix lint warnings

* add mailserver request expiration

* send mailserver response without envelope

* add `ttl` to Request struct in shhext_requestMessages

* test that tracker calls handler.MailServerRequestExpired

* add geth patch

* rename TTL to Timeout

* split tracker.handleEvent in multiple methods
2018-06-15 17:12:31 +02:00
Pedro Pombeiro
354e23aaf5 Fix issues reported by lint. Part of #1017 2018-06-14 13:52:51 +02:00
Adrià Cidre
d5be8c525d
[#856] move geth subpackages to root level (#1007) 2018-06-08 13:29:50 +02:00
Pedro Pombeiro
feed9158bd Temporarily revert "Remove shh and shhext from APIModules and update tests (#993)"
This reverts commit 7581766470ca7ff7a86685268b2d932edea3fcb5.
2018-06-07 18:18:40 +02:00
Sebastian Delgado
7581766470
Remove shh and shhext from APIModules and update tests (#993) 2018-06-04 13:13:02 -04:00
Adrià Cidre
cf8ad7e002
[#ISSUE-947/ISSUE-946] Rate limit and batch limit on any mail server query (#949) 2018-05-17 13:21:04 +02:00
Dmitry Shulyak
707221954f
Make whisper tolerant to local time skews (#864)
This change adds adds an ability to use different source of time for whisper:

when envelope is created it is used to set expiry
to track when envelope needs to be expired
This time is then used to check validity of the envelope when it is received. Currently If we receive an envelope that is sent from future - peer will get disconnected. If envelope that was received has an expiry less then now it will be simply dropped, if expiry is less than now + 10*2 seconds peer will get dropped.

So, it is clear that whisper depends on time. And any time we get a skew with peers that is > 20s reliability will be grealy reduced.

In this change another source of time for whisper will be used. This time source will use ntp servers from pool.ntp.org to compute offset. When whisper queries time - this offset will be added/substracted from current time.

Query is executed every 2 mins, queries 5 different servers, cut offs min and max and the computes mean value. pool.ntp.org is resolved to different servers and according to documentation you will rarely hit the same.

Closes: #687
2018-05-04 11:23:38 +03:00
Dmitry Shulyak
357786eeca Fix bloom filter expectations in group chat test
Every peer must be subscribed to the topic that is used to send messages.
In the test Alice was communicating with Bob and Charlie over custom topic, but
that topic wasn't added to a bloom filter, thus a certain flake was possible.

Normally it wasn't causing problems because syncAllowance in whisper, which is 10s:
- we set bloom filter to all zeros
- but we still will accept all envelopes for 10s
- in case we send first envelope into such channel after sync allowance - we will get an error such
that envelope doesn't match the bloom filter
2018-05-04 10:09:27 +03:00
Frank Mueller
b543d32a31
Move ssh_requestMessages to sshext_requestMessages (#862)
* Add RequestMessage to sshext

* E2E tests now use shhext_requestMessages

* Typo in comment

* Enhanced maintainability

* Drop former mailservice

* Code reorg after review

* Fix missed changes after update to 1.8.5
2018-04-26 07:56:19 +02:00
Dmitry Shulyak
b37fda7731
Update go-ethereum to 1.8.5 (#854)
* Rebase on 1.8.5

* Remove outdated patches and apply all others

* Use shh_post that returns hash

* Use bloom filter for request to mailserver

* Remove tests for sending messages without subbing first

* Fix deadlock in ethdb

* Expect null if receipt is not yet created

* Subscribe to messages before sending them in whisper test
2018-04-24 18:50:26 +03:00
Adam Babik
364f88efd9
Allow to start ephemeral StatusNode (#829)
This change will greatly simplify writing unit tests when a node is required but data persistence is irrelevant.

I also Introduced some refactoring and unit tests for `StatusNode`.
2018-04-16 14:36:09 +02:00
Adrià Cidre
359b3621e9 [#797] Simplify node.Manager public api and rename it to StatusNode
- [x] [#797] : Remove unused methods PopulateStaticPeers, ReconnectStaticPeers, removeStaticPeers, removePeer
- [x] [#797] : Rename node.Manager to node. StatusNode and simplify its public api
- [x] [#797] : Rename all references to nodeManager to statusNode
2018-04-05 16:47:27 +02:00
Evgeny Danienko
9ccd9b20e8
Merge branch 'develop' into issue/allow-messages-without-subscription-#766 2018-03-27 18:35:58 +03:00
Pedro Pombeiro
c4d7a6167e
Add race-check target to Makefile. Closes #710 (#769)
* Add `race-check` and `test-unit-race` target to Makefile. Closes #710
2018-03-27 14:38:41 +02:00
Evgeny Danienko
b9598a1a31
led comments to the standard 2018-03-26 20:05:24 +03:00
Evgeny Danienko
a90be3c0de
fixes after code review 2018-03-26 19:54:27 +03:00
Evgeny Danienko
aaf2b41e2e
test sending message without subscription 2018-03-26 19:02:02 +03:00
Pedro Pombeiro
6cdea4ef97
Update codebase to leverage Whisper v6 (#703)
* Update project to use Whisper v6. Part of #638

* Revert "Add patch to downgrade usage of Whisper v6 to v5 in some geth 1.8.1 vendor files. Part of #665" - this reverts commit 6aefb4c8fd02dbcfffac6b69e8bb22b13ef86b6b.

* Enable light mode on Whisper v6 for non-mail servers. Part of #638

* Fix race condition in whisperv6/peer.go. Part of #665 (PR already accepted upstream for 1.8.2)

* Update bootnode addresses in staticnodes.json. Part of #638

* Add `shh.lightclient` flag and tests for bloom filter setting logic. Part of #638

* Move MakeTestNodeConfig to utils. Part of #638

* Reduce PoW in `whisper_jail_test.go` to fix flaky test. Part of #638
2018-03-02 10:25:30 +01:00
Pedro Pombeiro
e4cbce12c4
Update to go-ethereum 1.8.1 (#702)
* Update `github.com/ethereum/go-ethereum` package to 1.8.1 branch. Part of #638
* Fix code due to some signature changes. Part of #638
* use upstream for whisper backend
* Add patch to downgrade usage of Whisper v6 to v5 in some geth 1.8.1 vendor files. Part of #638
* Take into account the DNS rebinding protection introduced in 1.8.0 by adding exception for localhost. Part of #638
* Add patches required for cross-compiled builds starting with geth 1.8.0. Only applied during build. Part of #638
* Update expected JSON result in `TestRegressionGetTransactionReceipt()` and `TestCallRawResultGetTransactionReceipt()`. Part of #665
* Fix some failing e2e tests. Part of #638
* Address comments in PR #702. Part of #638
2018-02-27 11:39:30 +01:00
Igor Mandrigin
c06d58addd Remove notifications package from go-ethereum. 2018-02-22 11:10:21 +01:00
Pedro Pombeiro
2993dcc3db
Fix a couple race conditions in tests (#689)
* Fix race condition on `LoopSuite`
* Fix race condition in the access to a Otto.Value in tests (quick fix, need to improve approach in the future)
2018-02-21 13:18:32 +01:00
Alex Kohler
365bc662a2 Enable gometalinter on tests and fix static analysis issues #631 (#644)
* Enable gometalinter on tests and fix static analysis issues

* Remove unneeded change

* Fix additional lint errors

* Add nolint directives and error checks

* Add error assertions instead of nolint directives

* Go back to using lint directive for loop.Run goroutine

* Add error check to loop.Run
2018-02-12 13:16:06 +02:00
Dmitry Shulyak
2d964bfe9f Remove async operations from node manager (#584)
The main goal of this change is to remove async operations from node manager.
Additionally all of the signals from node manager are moved to status backend.

All of the async operation now will have the following behaviour:
- If node in the correct state exit immediatly without error
- If node not in the correct state exit immediatly with error
- In all other cases spawn a goroutine with wanted operation
- All the progress regarding that operation will be reported
  by using signals
- Signals should be handled in once place, which is StatusBackend

There are 2 potentially breaking changes:
- Empty event field will be ommited when Envelope is sent to a client
- All errors will be delivered to a client as an Envelope, previously
  some errors (NodeExists, NoRunningNode) were delivered synchronously

Signed-off-by: Dmitry Shulyak <yashulyak@gmail.com>
2018-02-09 14:37:56 +01:00
Sebastian Klier
70912ab8f6 Group tests in a dedicated testing package (#629) 2018-02-08 13:52:47 +01:00