Commit Graph

791 Commits

Author SHA1 Message Date
Adrià Cidre 888ad10b26
[#969] Decrease NTP sync period after first success (#977) 2018-05-22 17:38:38 +02:00
Dmitry Shulyak 345b152a8b
Heap queue stores only peers that were not added to p2p server (#979)
* Heap queue stores only peers that were not added to p2p server

The primary goal of this change is to keep whitelist of peers
that are managed by topic pool while also preventing same peer
from being selected from heap queue multiple times.
2018-05-22 16:11:21 +03:00
Adam Babik 4dfbef3867
Fix p2p/Peers metric (#975)
Do not use server.PeerCount() when getting peers right after receiving add/remove peer event as it does not contain updated info.
2018-05-22 12:26:03 +02:00
Adrià Cidre ace175ba69 Fix flaky test TestPeerPoolSimulationSuite/TestSingleTopicDiscoveryWithFailover 2018-05-22 12:34:32 +03:00
Adrià Cidre 06e64cdde2
Refactor mailserver tests (#970) 2018-05-21 13:30:37 +02:00
Adrià Cidre e9da21cf87 Rollback mailserver DataDir to previous leveldb data directory 2018-05-20 08:34:47 +03:00
Frank Mueller 7325b38ef9
Quick fix of non-responding nodes by adding 51.15.68.93 to Rinkeby static nodes (#974)
* Add 51.15.68.93 to Rinkeby static nodes

* Add 51.15.75.244 for Rinkeby too
2018-05-18 17:35:36 +02:00
Dmitry Shulyak ed1cdf0418
Custom status bootnodes (#968)
This change makes invalidation mechanism more aggressive. With a primary goal to invalidate short living nodes faster. In current setup any node that became known in terms of discovery will stay in this state until it will fail to respond to 5 queries. Removing them earlier from a table allows to reduce latency for finding required nodes.

The second change, one adds a version for discovery, separates status dht from ethereum dht.
After we rolled out discovery it became obvious that our boot nodes became spammed with irrelevant nodes. And this made discovery process very long, for example with separate dht discovery takes ~2s, with mutual dht - it can take 1m-10m and there is still no guarantee to find a max amount of peers, cause status nodes is a very small part of whole ethereum infra.

In my understanding, we don't need to be a part of ethereum dht, and lower latency is way more important for us.

Closes: #941
Partially closes: #960 (960 requires futher investigations on devices)
2018-05-18 16:43:07 +03:00
Andrea Franz 4317b8f687
add cmd/statusd-prune (#957)
add mailserver cleaner

use memstorage for leveldb in tests

avoid write if batch size is 0

add comments

add cmd/statusd-prune

rmeove batch size var in prune method

validate range values

pass only flag name to missingFlag

refactor Cleaner.prune method

update batch not to be a pointer

removed extra batch counter increment

don't increment counter if batch returns errors

add README
2018-05-17 19:24:00 +02:00
Ivan Daniluk 4b17ea1ced
Exit when extra/errorneous flags are set (#965) 2018-05-17 16:03:23 +03:00
Adrià Cidre cf8ad7e002
[#ISSUE-947/ISSUE-946] Rate limit and batch limit on any mail server query (#949) 2018-05-17 13:21:04 +02:00
Adam Babik 6da469140f
Refactor in-app metrics (#963)
Remove Prometheus and other metric types and use metrics package from go-ethereum.
2018-05-16 17:36:59 +02:00
Dmitry Shulyak ca962ddaa3 Whisper broadcast loop spams way too hard with debug level 2018-05-16 17:54:40 +03:00
Dmitry Shulyak 5aae87aba8
Prevent sending messages to flaky peers (#917) 2018-05-15 20:08:31 +03:00
Dmitry Shulyak ee60b7e727 Fix mid computation in timesource
Additionally improved test suite
2018-05-15 19:34:37 +03:00
Dmitry Shulyak f5a8be08de
Use cache in topic pool (#955)
It must have slipped when i moved significant chunk of logic
from peer pool to topic pool and because of the != nil protection
this bug was missed
2018-05-15 18:05:57 +03:00
Ivan Daniluk bd68fa15c9 Use different pool hostnames for NTP timesource (#939) (#952)
* Use different pool hostnames for NTP timesource

* Make govet happy

* Add check for empty servers list
2018-05-15 17:10:51 +03:00
Dmitry Shulyak 9e65b5a6ae Dump peer infos as discovery summary 2018-05-15 13:11:28 +03:00
Adam Babik 9268f55d7d
Prevent Discovery from timeout when registering topics (#944)
When the node registers topics, Discovery V5 should never be stopped as otherwise, no peer will be able to find the node.
2018-05-15 11:16:25 +02:00
Andrea Franz 52cdcf8f0f
wallet compatibility (#858)
* update master key generations using "Bitcoin seed" as hmac key following BIP32

* use `"mnemonic" + passphrase` as salt for pbkdf2 following BIP39

* test generated addresses and compatibility with BIP44

* check generated public keys

* test children private keys

* update MasterKey to be a constant

* don't export salt and masterKey constants

* use `crypto.FromECDSA` to get the hex representation of the priv key

* use empty pwd for the generation of the BIP39 seed, keeping pwd to encrypt keys

* add comment before the seed generation with empty passphrase
2018-05-14 19:13:56 +02:00
Adam Babik 80be64a1ec
Fix flaky TestStatusNodeReconnectStaticPeers test (#945) 2018-05-14 15:16:45 +02:00
Dmitry Shulyak 938113882a
Add cluster for ropsten/mainnet (#929) 2018-05-14 08:24:49 +03:00
Adrià Cidre 08b4d515c6
[#ISSUE-937] : Move mail server implementation to status-go repo (#942)
* [#ISSUE-937] : Move mail server implementation to status-go repo

* Applying dep-ensure
2018-05-11 21:43:07 +02:00
Igor Mandrigin d7653beedd Fix deduplicator being created with nil db. 2018-05-11 09:58:21 +02:00
Andrea Franz 6c469299b6
fix default range value for shhext_requestMessages (#934)
* fix default range value for shhext_requestMessages

* use t.Run to run scenarions

* fix case when To - 24 hours is negative
2018-05-10 16:47:54 +02:00
Igor Mandrigin a933885806 Deduplicator: add API to confirm received messages. 2018-05-10 15:28:48 +02:00
Adrià Cidre 37a58a513d [#930] : False positive e2e test fix 2018-05-10 15:18:37 +02:00
Dmitry Shulyak 375d5ec8c3
Explicitly disable discovery (#886)
* Make it possible to explicitly disable discovery

Discovery will be disabled in following cases:
- if there are not bootnodes - v5 server will be disabled
because there is no point in running it
- if user defined in config NoDiscovery=true this value will be preserved
even if we have bootnodes

So, basically discovery will be always enabled by default on mobile, unless
it is explicitly specified otherwise.

When statusd is used current behavior is that discovery is disabled by default.
I kept it in this change, but it would be better to change it.

* Fix leftovers

* Add wait group to peer pool to protect from races with p2p.Server

* Change fields only when all goroutines finished

* Turn off discovery after topic searches are stopped

* Don't set period to nil to avoid race with SearchTopic

* Close period chan only when all writers are finished
2018-05-10 14:45:51 +03:00
Andrea Franz f7dac0bcc4
add ipcfile flag to specify IPC file path (#896) 2018-05-10 01:59:05 +02:00
Igor Mandrigin 0241f95d6a Support chain diverges in block filters. 2018-05-09 18:07:26 +02:00
Ivan Danyliuk 6fec9fa653 Bump up Go version to 1.10 2018-05-09 16:13:34 +02:00
Igor Mandrigin 397bf83924 Add testcases for contract mining to the jail RPC tests. 2018-05-09 14:34:10 +02:00
Dmitry Shulyak 6c2a8ef1f9
Define shorter timeout for ntp calls (2s) and tolerate failures (#911)
* Define shorter timeout for ntp calls (2s) and tolerate failures

* Use allowed failures instead of tolerated errorss
2018-05-09 08:10:48 +03:00
Andrea Franz c673148bf4
add `-status` flag to enable the Status service (#902)
* add `-status` flag to enable the Status service

* remove status from default APIModules and add it only from statusd if specified

* remove AddAPIModule method

* allow -status flag values to be http or ipc
2018-05-08 23:57:29 +02:00
Igor Mandrigin 31cf2297d2 Implement block filters API for the RPC mode.
Some operations (like deploying contracts) require filter APIs to work.
Since these operations aren't supported on Infura anymore, and we don't
run LES, a separate implemenation of filters is required.

Signed-off-by: Igor Mandrigin <i@mandrigin.ru>
2018-05-08 19:46:25 +02:00
Adrià Cidre 6e1fc99365
Notifier package to interact with gorush notifications (#910) 2018-05-08 16:30:03 +02:00
Dmitry Shulyak 7aa508765e Set timesource from config (#915) 2018-05-08 13:38:54 +03:00
Adam Babik b6851d8186 Run tests with race and coverage flags (#909) 2018-05-07 18:08:00 +03:00
Adrià Cidre 246c5321ac
Fix services mock generation (#901) 2018-05-07 10:22:38 +02:00
Pedro Pombeiro 1afb26c3f4 Use `make setup` on Travis CI 2018-05-06 22:59:36 +02:00
Igor Mandrigin 153d4160e1 Fix `make ci` if `dep` wasn't pre-installed. 2018-05-04 14:17:17 +02:00
Pedro Pombeiro 5d802abe9e Increase timeouts in order to accommodate `-race` flag
- Fixes https://jenkins.status.im/job/status-go/job/race-check/56/console
2018-05-04 14:02:13 +02:00
Dmitry Shulyak 84cb5ca917 Ensure that wg.Add is called before wg.Wait
Now if Add is to be called it will be called before Wait, this is achieved
by doing following:
- if cancel() gets lock first and closes channelCh before spawnSync is called
we will exit right away
- if not than we will ensure that we hold a lock until syncers are spawned
so that cancel() will be blocked for this time and it will prevent whole Terminate() from
progressing
2018-05-04 10:28:15 +02:00
Dmitry Shulyak 707221954f
Make whisper tolerant to local time skews (#864)
This change adds adds an ability to use different source of time for whisper:

when envelope is created it is used to set expiry
to track when envelope needs to be expired
This time is then used to check validity of the envelope when it is received. Currently If we receive an envelope that is sent from future - peer will get disconnected. If envelope that was received has an expiry less then now it will be simply dropped, if expiry is less than now + 10*2 seconds peer will get dropped.

So, it is clear that whisper depends on time. And any time we get a skew with peers that is > 20s reliability will be grealy reduced.

In this change another source of time for whisper will be used. This time source will use ntp servers from pool.ntp.org to compute offset. When whisper queries time - this offset will be added/substracted from current time.

Query is executed every 2 mins, queries 5 different servers, cut offs min and max and the computes mean value. pool.ntp.org is resolved to different servers and according to documentation you will rarely hit the same.

Closes: #687
2018-05-04 11:23:38 +03:00
Dmitry Shulyak 357786eeca Fix bloom filter expectations in group chat test
Every peer must be subscribed to the topic that is used to send messages.
In the test Alice was communicating with Bob and Charlie over custom topic, but
that topic wasn't added to a bloom filter, thus a certain flake was possible.

Normally it wasn't causing problems because syncAllowance in whisper, which is 10s:
- we set bloom filter to all zeros
- but we still will accept all envelopes for 10s
- in case we send first envelope into such channel after sync allowance - we will get an error such
that envelope doesn't match the bloom filter
2018-05-04 10:09:27 +03:00
Pedro Pombeiro acbf251a3c Fix race condition in `LDBDatabase.Close()`
- https://jenkins.status.im/job/status-go/job/race-check/54/consoleFull
2018-05-03 20:39:42 +03:00
Pedro Pombeiro 04fa765daf Increase timeout for Rinkeby sync on Travis CI.
- Fixes https://travis-ci.org/status-im/status-go/jobs/374296755
2018-05-03 13:16:56 +02:00
Adrià Cidre 8c9db81bec
Expose status specific methods (login/signup/joinpublicchannel) through the RPC api (#877) 2018-05-03 12:36:56 +02:00
Ivan Danyliuk 953c26e8cf
Move signal logic into signal package 2018-05-03 09:35:58 +02:00
Dmitry Shulyak c8a553f9c1 Fix 0002 xgo patch which changed because metrics got forked 2018-05-02 22:59:54 +02:00