Commit Graph

310 Commits

Author SHA1 Message Date
Igor Mandrigin 31cf2297d2 Implement block filters API for the RPC mode.
Some operations (like deploying contracts) require filter APIs to work.
Since these operations aren't supported on Infura anymore, and we don't
run LES, a separate implemenation of filters is required.

Signed-off-by: Igor Mandrigin <i@mandrigin.ru>
2018-05-08 19:46:25 +02:00
Dmitry Shulyak 707221954f
Make whisper tolerant to local time skews ()
This change adds adds an ability to use different source of time for whisper:

when envelope is created it is used to set expiry
to track when envelope needs to be expired
This time is then used to check validity of the envelope when it is received. Currently If we receive an envelope that is sent from future - peer will get disconnected. If envelope that was received has an expiry less then now it will be simply dropped, if expiry is less than now + 10*2 seconds peer will get dropped.

So, it is clear that whisper depends on time. And any time we get a skew with peers that is > 20s reliability will be grealy reduced.

In this change another source of time for whisper will be used. This time source will use ntp servers from pool.ntp.org to compute offset. When whisper queries time - this offset will be added/substracted from current time.

Query is executed every 2 mins, queries 5 different servers, cut offs min and max and the computes mean value. pool.ntp.org is resolved to different servers and according to documentation you will rarely hit the same.

Closes: 
2018-05-04 11:23:38 +03:00
Dmitry Shulyak 357786eeca Fix bloom filter expectations in group chat test
Every peer must be subscribed to the topic that is used to send messages.
In the test Alice was communicating with Bob and Charlie over custom topic, but
that topic wasn't added to a bloom filter, thus a certain flake was possible.

Normally it wasn't causing problems because syncAllowance in whisper, which is 10s:
- we set bloom filter to all zeros
- but we still will accept all envelopes for 10s
- in case we send first envelope into such channel after sync allowance - we will get an error such
that envelope doesn't match the bloom filter
2018-05-04 10:09:27 +03:00
Adrià Cidre 8c9db81bec
Expose status specific methods (login/signup/joinpublicchannel) through the RPC api () 2018-05-03 12:36:56 +02:00
Ivan Danyliuk 953c26e8cf
Move signal logic into signal package 2018-05-03 09:35:58 +02:00
Frank Mueller c85e0802eb
Remove mention of DevMode ()
* Remove mention of DevMode

* Remove prodMode
2018-04-26 19:59:57 +02:00
Andrea Franz 5f45f6498d
Add LogEnabled attribute to NodeConfig to, used in the call from status-react ()
* add LogEnabled attribute to NodeConfig, used in the call from status-react

* fix use of OverrideRootLogin in tests

* enable logger in tests based on LogLevel

* move LogEnabled before LogFile
2018-04-26 18:28:42 +02:00
Frank Mueller b543d32a31
Move ssh_requestMessages to sshext_requestMessages ()
* Add RequestMessage to sshext

* E2E tests now use shhext_requestMessages

* Typo in comment

* Enhanced maintainability

* Drop former mailservice

* Code reorg after review

* Fix missed changes after update to 1.8.5
2018-04-26 07:56:19 +02:00
Dmitry Shulyak f3e2631c1d
Add signals for notifying react about discovery activity ()
* Add signal for notifying react on discovery activity

* Whitelist expected signals in e2e tests
2018-04-25 10:13:59 +03:00
Dmitry Shulyak b37fda7731
Update go-ethereum to 1.8.5 ()
* Rebase on 1.8.5

* Remove outdated patches and apply all others

* Use shh_post that returns hash

* Use bloom filter for request to mailserver

* Remove tests for sending messages without subbing first

* Fix deadlock in ethdb

* Expect null if receipt is not yet created

* Subscribe to messages before sending them in whisper test
2018-04-24 18:50:26 +03:00
Adam Babik b85e50cbc9
Remove any logic from api.go [breaking-change] ()
Deprecated `Notify` binding got removed.
2018-04-23 15:34:49 +02:00
Adam Babik 5f075eeaba
Enable logs in tests ()
Pass `-log` option to `go test` in order to change the log level. By default, it's `INFO`.
2018-04-22 18:50:34 +02:00
Adam Babik 07b2c3d750
Move concurrent backend test from e2e to unit () 2018-04-20 17:39:53 +02:00
Dmitry Shulyak 2f2dfe16c0
Add default peer limits configuration ()
* Add default peer limits configuration

If discovery is enabled for a given cluster - we will set a default
expected number of peers for each enabled service. For example:

- if cluster is rinkeby has a discovery enabled we will
  check which services are enabled
- if whisper is enabled we will set min and max limits by default
- if les is enabled and infura is not used we will set limits too

When statusd is used - configuration must be provided using configuration
supported by statusd.


* Fix deadlock in les peer set
2018-04-20 15:23:18 +03:00
Igor Mandrigin f0f55d408f Make sure that only `personal_sign` and `personal_ecRecover` are exported. 2018-04-18 18:22:21 +02:00
Frank Mueller 6598371dc0
Prepare tests for mainnet ()
* Start enabling to test Mainnet

* Minor corrections found during code scan

* Set mainnet blocker in E2E again

* Introduced securing of mainnet transaction tests

* Fix typing error

* Typo led to follow-up errors

* Linter problem

* Change to individual test skips after review

* More flexible skip control for mainnet and status chain

* Fix double space

* Change of skipping method and fixing wrong skipped networks
2018-04-18 17:13:27 +02:00
Igor Mandrigin 19556554a6 Fix dataraces in e2e/transactions tests 2018-04-18 15:02:59 +02:00
Igor Mandrigin 4cc6028d59 Implement `personal_sign`.
This commit implements `personal_sign` RPC or web3 personal.sign
methods.

NB! Contains breaking API changes.
2018-04-17 11:18:58 +02:00
Adam Babik 364f88efd9
Allow to start ephemeral StatusNode ()
This change will greatly simplify writing unit tests when a node is required but data persistence is irrelevant.

I also Introduced some refactoring and unit tests for `StatusNode`.
2018-04-16 14:36:09 +02:00
Dmitry Shulyak 0b123ed407 Send an expiration signal when envelope wasn't delivered to any peer 2018-04-15 10:44:49 +03:00
Adam Babik 0d652c3851
Expose only public APIs in inproc RPC client [breaking-change] ()
`CallRPC` binding, which is used as a provider for web3.js, exposes only public or whitelisted APIs.
2018-04-12 18:17:10 +02:00
Dmitry Shulyak ba9a25e284
Implement shh api extension that allows to confirm that message is sent ()
* Implement shh api extension that allows to confirm that message is sent

* Add a patch

* Fix linter

* Add readme

* Add tests for tracker

* Address review
2018-04-11 18:41:51 +03:00
Igor Mandrigin a9eb5a7d2b Generalize signing requests.
We need to be able to sign more than just transactions to make DApps
work properly. This change separates signing requests from the
transactions and make it more general to prepare to intoduce different
types of signing requests.

This change is designed to preserve status APIs, so it is
backward-comparible with the current API bindings.
2018-04-09 20:48:00 +02:00
Adam Babik 32d71da046
fix linter in transactions_test.go () 2018-04-09 14:14:02 +02:00
Dmitry Shulyak 4d00fa80b0 Ensure that To field is nil when contract is created 2018-04-09 14:41:50 +03:00
Adrià Cidre df35ad6dbe
[] Unify EnsureSync on t/utils/utils.go and StatusManager () 2018-04-09 09:16:43 +02:00
Adrià Cidre 359b3621e9 [] Simplify node.Manager public api and rename it to StatusNode
- [x] [] : Remove unused methods PopulateStaticPeers, ReconnectStaticPeers, removeStaticPeers, removePeer
- [x] [] : Rename node.Manager to node. StatusNode and simplify its public api
- [x] [] : Rename all references to nodeManager to statusNode
2018-04-05 16:47:27 +02:00
Adrià Cidre aa0f2ede6d
[] : Remove geth/common package () 2018-04-04 19:39:38 +02:00
Adrià Cidre 518148cab9
Merge pull request from status-im/improvement/issue-779/common-NodeManager-removal
[] common.NodeManager removal
2018-03-29 09:57:18 +02:00
Igor Mandrigin 3b32ab8426 Support both "input" and "data" fields when creating a transaction. 2018-03-28 20:28:35 +02:00
Adrià Cidre 1b724a81a5
[] common.NodeManager removal 2018-03-28 17:39:09 +02:00
Adrià Cidre e646001578
Merge pull request from status-im/transactions-and-accountmanager
[] txQueueManager to not depend on AccountManager and nodeManager
2018-03-28 11:06:33 +02:00
Adrià Cidre c76b2a0f62 [] : Split accounts and whisper tests () 2018-03-28 10:24:55 +02:00
Adrià Cidre 13cb2daaa2
[] txQueueManager to not depend on AccountManager 2018-03-28 09:58:05 +02:00
Evgeny Danienko 9ccd9b20e8
Merge branch 'develop' into 2018-03-27 18:35:58 +03:00
Pedro Pombeiro d9b28a06d9
Take into consideration that path may contain multiple `status-go` directories, take the last ()
- Happens in Jenkins
2018-03-27 17:06:36 +02:00
Pedro Pombeiro c4d7a6167e
Add `race-check` target to Makefile. Closes ()
* Add `race-check` and `test-unit-race` target to Makefile. Closes 
2018-03-27 14:38:41 +02:00
Evgeny Danienko b9598a1a31
led comments to the standard 2018-03-26 20:05:24 +03:00
Evgeny Danienko a90be3c0de
fixes after code review 2018-03-26 19:54:27 +03:00
Evgeny Danienko aaf2b41e2e
test sending message without subscription 2018-03-26 19:02:02 +03:00
Adrià Cidre 18fbebc942
[#issue-403] Account package cleanup 2018-03-23 08:21:40 +01:00
Alex Kohler 4a01a395b4 Move status-go over to go-ethereum/log () 2018-03-20 19:35:28 +01:00
Frank Mueller 8e42014671
Static peers are now also optionally configurable ()
* Rename bootnode to static peers

Also add some groupings between types.

* Remove not needed genesisHash

* More cleanup of bootnodes with static peers

* Add option of cluster configuration file

* New generated bindata.go

* Changes after npm install

* Add argument for cluster configuration file

* Add test for dynamic cluster loading

Not yet sure with name "cluster config".

* Solved conflicts

* Renaming of static peers

* Remove static peers population

* Missing argument for config

* Renaming of static peers to boot nodes for consistency

* Fix of name change

* Cluster config is now cluster data

* Load static nodes from configuration

* Final renaming of var for file content
2018-03-19 17:22:09 +01:00
Frank Mueller 81c8669518
Code review for Mainnet ()
* Add debug log to test package t

* Add comments for test restrictions

* Add explicit errors for mainnet

* Panic instead of error in case of mainnet during tests

* Now panic with illegal network

* Validate the compiler flag for mainnet

* Made test for unset mainnet explicit

* Forgot to use my flag

* Linter has troubles with unused field

* Add default network id

In e2e tests there's no default anymore in go code. So ensure in Makefile.

* Fix dumb syntax error

* CI needs build with Mainnet

* Explicit rebuild only for tests

* No more separation of mainnet and testnet builds

* Build flags now set in Makefile
2018-03-15 12:36:28 +01:00
Dmitry Shulyak a5cec358a9 Post whisper message asynchronously in tests
I am not 100% percent sure what is happening but it seems that
newMessageFilter is async operation, result of this operation is assumed
to be used in callbacks. All other tests are doing at least 1 io operation
in between creating a filter and posting a message, and it must be enough
for newMessageFilter to complete.

Setting higher GOMAXPROCS allows Otto vm to execute received io requests
immediatly. While lower number of processes may result in events re-ordering.

https://github.com/ethereum/web3.js/blob/develop/lib/web3/methods/shh.js#L39-L41

I tested this change with GOMAXPROCS=1 and it passes consistently.
2018-03-02 19:43:08 +01:00
Pedro Pombeiro 6cdea4ef97
Update codebase to leverage Whisper v6 ()
* Update project to use Whisper v6. Part of 

* Revert "Add patch to downgrade usage of Whisper v6 to v5 in some geth 1.8.1 vendor files. Part of #665" - this reverts commit 6aefb4c8fd02dbcfffac6b69e8bb22b13ef86b6b.

* Enable light mode on Whisper v6 for non-mail servers. Part of 

* Fix race condition in whisperv6/peer.go. Part of  (PR already accepted upstream for 1.8.2)

* Update bootnode addresses in staticnodes.json. Part of 

* Add `shh.lightclient` flag and tests for bloom filter setting logic. Part of 

* Move MakeTestNodeConfig to utils. Part of 

* Reduce PoW in `whisper_jail_test.go` to fix flaky test. Part of 
2018-03-02 10:25:30 +01:00
Alex Kohler 5fbacb49e1 Remove GetObjectValue from JailCell interface () 2018-03-01 17:48:30 +01:00
Pedro Pombeiro e4cbce12c4
Update to go-ethereum 1.8.1 ()
* Update `github.com/ethereum/go-ethereum` package to 1.8.1 branch. Part of 
* Fix code due to some signature changes. Part of 
* use upstream for whisper backend
* Add patch to downgrade usage of Whisper v6 to v5 in some geth 1.8.1 vendor files. Part of 
* Take into account the DNS rebinding protection introduced in 1.8.0 by adding exception for localhost. Part of 
* Add patches required for cross-compiled builds starting with geth 1.8.0. Only applied during build. Part of 
* Update expected JSON result in `TestRegressionGetTransactionReceipt()` and `TestCallRawResultGetTransactionReceipt()`. Part of 
* Fix some failing e2e tests. Part of 
* Address comments in PR . Part of 
2018-02-27 11:39:30 +01:00
Pedro Pombeiro 0332465952
Update `make generate` target to account for new `_assets` folder 2018-02-26 16:04:40 +01:00
Dmitry Shulyak 741422af73
Add a test with network connection problem while syncing a chain ()
Test waits till synchronization is started, then interrupts network connection, wait for a failure event and restore network connection, confirming that after the connection is restored synchronization will proceed.
2018-02-23 09:03:55 +02:00
Igor Mandrigin c06d58addd Remove `notifications` package from go-ethereum. 2018-02-22 11:10:21 +01:00
Pedro Pombeiro 2993dcc3db
Fix a couple race conditions in tests ()
* Fix race condition on `LoopSuite`
* Fix race condition in the access to a Otto.Value in tests (quick fix, need to improve approach in the future)
2018-02-21 13:18:32 +01:00
Dmitry Shulyak b2580c79d7
Add destructive tests ()
Network disconnect is introduced by removing default gateway, easily reversible condition.

On my local machine it takes 30 seconds for peers to reconnect after connectivity is restored. As you guess this is not an accident, and there is 30 seconds timeout for dial expiration. This dial expiration is used in p2p.Server to guarantee that peers are not dialed too often.

Additionally I added small script to Makefile to run such tests in docker environment, usage example:

```
make docker-test ARGS="./t/destructive/ -v -network=4"
```
2018-02-19 17:32:58 +02:00
Pedro Pombeiro 7410532879 Patcher improvement ()
Improve patcher tool:
* fix broken `-p` flag,
* rename it to `-b`,
* add a new `-p` flag that allows specifying the source of the patches, so that it can be used for more than just go-ethereum patches,
* make it work correctly independently of the order that options are specified in the command line.
2018-02-18 09:14:27 -07:00
Dmitry Shulyak 4a49e840e3 Remove status backend from les internal api () 2018-02-16 15:40:37 +01:00
Dmitry Shulyak 0b979c507d Fix async status api () 2018-02-14 17:32:36 +01:00
Adam Babik adbc7b5ae6 use s.Require() in Whisper tests 2018-02-14 10:45:29 +01:00
Alex Kohler 365bc662a2 Enable gometalinter on tests and fix static analysis issues ()
* Enable gometalinter on tests and fix static analysis issues

* Remove unneeded change

* Fix additional lint errors

* Add nolint directives and error checks

* Add error assertions instead of nolint directives

* Go back to using lint directive for loop.Run goroutine

* Add error check to loop.Run
2018-02-12 13:16:06 +02:00
Dmitry Shulyak 2d964bfe9f Remove async operations from node manager ()
The main goal of this change is to remove async operations from node manager.
Additionally all of the signals from node manager are moved to status backend.

All of the async operation now will have the following behaviour:
- If node in the correct state exit immediatly without error
- If node not in the correct state exit immediatly with error
- In all other cases spawn a goroutine with wanted operation
- All the progress regarding that operation will be reported
  by using signals
- Signals should be handled in once place, which is StatusBackend

There are 2 potentially breaking changes:
- Empty event field will be ommited when Envelope is sent to a client
- All errors will be delivered to a client as an Envelope, previously
  some errors (NodeExists, NoRunningNode) were delivered synchronously

Signed-off-by: Dmitry Shulyak <yashulyak@gmail.com>
2018-02-09 14:37:56 +01:00
Sebastian Klier 70912ab8f6 Group tests in a dedicated testing package () 2018-02-08 13:52:47 +01:00