Commit Graph

175 Commits

Author SHA1 Message Date
Andrea Maria Piana 401d18e558
Enabled push notifications by default and add fleet 2020-08-27 18:54:06 +02:00
Andrea Maria Piana c5577418b7 Implement rate limiting by bytes
Why make this change?

Currently we only rate limit by number of messages.
This works well if we want to limit a large amount of small messages but
breaks down if sending a smaller amount of large messages.

This commit extends the current code to limit by size as well, setting
the default to 1MB per second, which should be enough.

What has changed?

- Rate limiter for waku only
2020-06-15 19:46:23 +02:00
Andrea Maria Piana 2d12ac4fbb Rename rate limiting to packet rate limiting 2020-06-15 19:46:23 +02:00
Andrea Maria Piana d5086d6e89 Remove incentivisation service
Incentivisation was an experiment in running an incentivised fleet that
rewarded nodes based on their well behavior. It was heavily influenced
by https://docs.loki.network/ . It is currently not used anymore, so
removing.
2020-06-01 11:30:04 +02:00
Andrea Maria Piana aa7f591587 Move networking code for waku under `v0` namespace
Why make the change?

As discussed previously, the way we will move across versions is to maintain completely separate
codebases and eventually remove those that are not supported anymore.

This has the drawback of some code duplication, but the advantage is that is more
explicit what each version requires, and changes in one version will not
impact the other, so we won't pile up backward compatible code.
This is the same strategy used by `whisper` in go ethereum and is influenced by
https://www.youtube.com/watch?v=oyLBGkS5ICk .

All the code that is used for the networking protocol is now under `v0/`.
Some of the common parts might still be refactored out.
The main namespace `waku` deals with `host`->`waku` interactions (through RPC),
while `v0` deals with `waku`->`remote-waku` interactions.

In order to support `v1`, the namespace `v0` will be copied over, and changed to
support `v1`. Once `v0` will be not used anymore, the whole namespace will be removed.

This PR does not actually implement `v1`, I'd rather get things looked over to
make sure the structure is what we would like before implementing the changes.

What has changed?

- Moved all code for the common parts under `waku/common/` namespace
- Moved code used for bloomfilters in `waku/common/bloomfilter.go`
- Removed all version specific code from `waku/common/const` (`ProtocolVersion`, status-codes etc)
- Added interfaces for `WakuHost` and `Peer` under `waku/common/protocol.go`

Things still to do

Some tests in `waku/` are still testing by stubbing components of a particular version (`v0`).
I started moving those tests to instead of stubbing using the actual component, which increases
the testing surface. Some other tests that can't be easily ported should be likely moved under
`v0` instead. Ideally no version specif code should be exported from a version namespace (for
example the various codes, as those might change across versions). But this will be a work-in-progress.

Some code that will be common in `v0`/`v1` could still be extract to avoid duplication, and duplicated only
when implementations diverge across versions.
2020-04-27 14:58:02 +02:00
Adam Babik b33ad8147d
Add example to demonstrate Whisper-Waku bridge (#1875) 2020-02-26 20:36:25 +01:00
Adam Babik f335d1cf7e
Prevent starting node without APIModules (#1876) 2020-02-26 20:35:47 +01:00
Andrea Maria Piana 849492fda9
Add statusUpdate code (#1861)
* Add status-option code

This commits changes the behavior of waku introducing a new status-code,
`2`, that replaces the current single options codes.

* linting
2020-02-21 15:48:53 +01:00
Adam Babik e4b0a97cc5
add bridge metrics to Whisper and Waku (#1864) 2020-02-19 15:36:16 +01:00
Adam Babik 76b5dc29dc
Integrate Whisper-Waku bridge in status-go (#1854) 2020-02-18 12:21:01 +01:00
Andrea Maria Piana 141f917e7c
Initialize protocol when waku active (#1832) 2020-02-17 15:38:59 +01:00
Volodymyr Kozieiev 19fc448c85
pending-stickers field added to settings, unit-tests fixed (#1830) 2020-02-13 13:48:04 +02:00
Roman Volosovskyi 9cf640842b
[status-im/status-react#9942] Upgradable paths in configs
Storing absolute path for different configs breaks compatibility on iOS
as app's dir is changed after upgrade. The solution is to store relative
paths and to concatenate it with `backend.rootDataDir`. The only
exception is `LogFile` as it is stored outside `backend.rootDataDir` on
Android. `LogDir` config was added to allow adding of custom dir for log
file.
Configs concerned:
`DataDir`
`LogDir`
`LogFile`
`KeystoreDir`
`BackupDisabledDataDir`
2020-02-03 20:02:01 +02:00
Adam Babik 79b8112f89
Split shhext into shhext and wakuext (#1803) 2020-01-20 21:56:06 +01:00
Pedro Pombeiro 6537cae606 Nimbus node support 2020-01-20 13:15:17 +01:00
Adam Babik 44aa313981
Make shhext and protocol work with Waku (#1777)
This change makes shhext and protocol submodule work with Waku and Whisper.
2020-01-13 20:17:30 +01:00
Adam Babik 37a508a97b
Integrate Waku service (#1759) 2020-01-08 12:12:23 +01:00
Pedro Pombeiro c8a911ebd1 Use goimports instead of gofmt 2020-01-06 10:17:23 +01:00
Adam Babik 4f45bceb9a
Drop peers when rate limit is exceeded (#1732) 2019-12-16 20:38:50 +01:00
Adam Babik a636f33109
Set Whisper version submodule to v6 (#1725) 2019-12-09 11:36:14 +01:00
Adam Babik 4ac4a61e20
Move whisper to status-go monorepo (#1720) 2019-12-09 11:06:04 +01:00
Pedro Pombeiro 26b6d7c36a Create a home submodule for Eth node bridges- Rename StatusBackend to GethStatusBackend 2019-11-27 17:02:09 +01:00
Pedro Pombeiro ed5a5c154d
Move to monorepo structure (#1684)
Move to a monorepo structure with submodules

- Rename status-protocol-go to status-go/protocol
2019-11-21 17:19:22 +01:00
Pedro Pombeiro 2dd74da23d Integrate Nimbus status-protocol-go 2019-11-18 14:30:03 +01:00
Adam Babik c199c8f342
Update Whisper and introduce rate limiting (#1673) 2019-11-18 11:22:23 +01:00
Pedro Pombeiro c874960215
Use status-protocol-go/bridge/geth (#1638)
* Use status-protocol-go/bridge/geth
2019-10-14 09:53:38 +02:00
Pedro Pombeiro a2f106e4c5
Update for status-protocol-go API changes (#1627) 2019-10-09 16:22:53 +02:00
Adam Babik 26880b83d7
Upgrade geth to 1.9.5 and Whisper (#1617) 2019-10-04 17:21:24 +02:00
Dmitry 494cb5bb33 Create single database appdatase 2019-08-21 10:44:50 +03:00
Dmitry Shulyak be9c55bc16
Accounts data management (#1530)
* WIP accounts implementation

* Accounts datasore and changes to status mobile API

* Add library changes and method to update config

* Handle error after account selection

* Add two methods to start account to backend

* Use encrypted database for settings and add a service for them

* Resolve linter warning

* Bring back StartNode StopNode for tests

* Add sub accounts and get/save api

* Changes to accounts structure

* Login use root address and fetch necessary info from database

* Cover accounts store with tests

* Refactor in progress

* Initialize status keystore instance before starting ethereum node

* Rework library tests

* Resolve failures in private api test and send transaction test

* Pass pointer to initialized config to unmarshal

* Use multiaccounts/accounts naming consistently

Multiaccount is used as a login identifier
Account references an address and a key, if account is not watch-only.

* Add login timestamp stored in the database to accounts.Account object

* Add photo-path field for multiaccount struct

* Add multiaccoutns rpc with updateAccount method

Update to any other account that wasn't used for login will return an error

* Fix linter in services/accounts

* Select account before starting a node

* Save list of accounts on first login

* Pass account manager to accounts service to avoid selecting account before starting a node

* Add logs to login with save and regualr login
2019-08-20 18:38:40 +03:00
Andrea Maria Piana cb769ccca9 Revert "Initialize status keystore instance before starting ethereum node"
This reverts commit 476240fbd1.
2019-08-05 16:03:18 +02:00
Dmitry 476240fbd1 Initialize status keystore instance before starting ethereum node 2019-08-01 11:01:02 +03:00
Andrea Franz dcb0fa5262
add multi-account support (#1500)
* add multiaccount support

add multi account ImportPrivateKey and StoreAccount

test derivation from normal keys

* add multiaccount to mobile pkg

* use multiaccount params structs from the mobile pkg

* move multiaccount tests together with the other lib tests

* fix codeclimate warning and temporarily increase methods threshold

* split library_test_utils.go to avoid linter warnings
2019-07-24 20:59:15 +02:00
Dmitry Shulyak 9723b64827
Permissions api (#1524)
* Add permissions api

* Integrate permissions service

* Reduce cyclomatic complexity of the MakeNode function
2019-07-21 08:41:30 +03:00
Dmitry Shulyak 9dbf5a0c86
Add browsers API (#1523)
Add browsers API
2019-07-17 08:28:37 +03:00
Andrea Maria Piana 0ade9a6cbb
Cache keys (#1497)
* Cache keys

Generating a symkey can take up to a second on slow devices, this commit
makes so that keys are saved once generated and stored in the database.
2019-06-24 09:26:25 +02:00
Dmitry Shulyak 047c9b5263
Download transfers starting from latest block header (#1467) 2019-06-14 13:16:30 +03:00
Andrea Maria Piana 354e6981ba
Add network incentivisation service (#1452) 2019-04-29 14:05:49 +02:00
Adam Babik 4c1b6c12e4 [Chaos Unicorn] Expose ChaosModeUpdate (#1422) 2019-03-29 12:55:54 +01:00
Dmitry eeaf669006 Add goerli testnet 2019-03-19 10:56:09 +02:00
Ivan Daniluk 4f3f5ee574 Gomobile support (#1164) 2019-02-01 18:02:52 +01:00
Adam Babik 838acce91f
fix deadlock on starting discovery with custom AdvertiseAddr (#1363) 2019-01-25 19:17:08 +01:00
Dmitry Shulyak 863d07f798 Accept status service config directly from json input (#1337) 2019-01-17 13:56:22 +01:00
Dmitry Shulyak 0b72aea56a Ensure that all discovery compoenents are set to nil on stop (#1351)
Previously we always tried to stop discovery if it was enabled according to
configuration. Now we check if it was started, because it can be started separataly from
a node.
2019-01-17 12:02:45 +01:00
Roman Volosovskyi :: Darkviolet Lightgreen Halcyon 66fb99d5d6 Start discovery after sending "node.ready" signal (#1333)
We would like to optimize sending "node.ready" signal and do it as soon as possible. The peers discovery protocol can be started after the signal is sent.
2019-01-02 19:57:36 +01:00
Dmitry Shulyak d51761f83e Ensure sent event is received from a mailserver (#1304)
In previous change i used same filtering for a handler that processes confirmations.
But if we are connected with peers that do not support confirmations on whisper level,
whisper may send "Sent" event when envelope is written to a socket. This is our old behaviour.

So, this PR allows to use confirmations from a mail servers and ensure that transition between
multiple version of peers will be smooth.
2018-12-06 10:30:57 +01:00
Adam Babik 52a1bdfed6
Upgrade geth 1.8.17 plus add metrics during compilation time (#1273)
This commit updates geth to 1.8.17 and adds a possibility to enable metrics during compilation time.

The cascade of issues forced us to upgrade geth to 1.8.17 in order to allow enabling metrics during compilation time. 1.8.17 introduced `NodeID` refactoring and `enode` package which affected our peers pool and integration with Discovery V5.
2018-11-14 08:03:58 +01:00
b00ris bdbceba6eb
added ulc config params (#1255)
* added ulc config params

* fmt

* fix lint
2018-10-24 19:31:41 +03:00
Adam Babik a75f9c34cf
Request historical messages in batches (#1245) 2018-10-19 11:09:13 +02:00
Adam Babik 20f8f1f2cc
allow configure HTTPVirtualHosts and HTTPCors settings (#1236) 2018-10-12 14:58:32 +02:00
Ivan Daniluk 4d5f808085
Add IPC path command line flag (#1229) 2018-10-11 12:29:59 +02:00
Dmitry 5b551c67fa Use status-im whisper/whisperv6 module everywhere 2018-10-03 09:27:51 +03:00
Andrea Maria Piana dcaf8caed0
Add x3dh key exchange (#1127)
* Add x3dh key exchange

* Encrypt using the double ratchet

* Multi device with auto-pairing

* Add pfs enabled flag
2018-09-24 20:07:34 +02:00
Adam Babik 8aef7c4f15
tune CLI for better docs experience (#1207) 2018-09-21 16:09:31 +02:00
Pedro Pombeiro 452b7e2198
Disable listening on port 30303 for the canary (#1198) 2018-09-19 14:45:43 +02:00
Pedro Pombeiro 1ce6f44b5b
Use in-memory DB for test. (#1191) 2018-09-14 11:09:39 +02:00
Pedro Pombeiro 3d00af7fa3
Streamline configuration in status-go. Part of #1180 (#1183)
- Replace command line flags with `-c` config flag. Part of #1180
- Convert node config private keys to hex-encoded string versions.
- Remove `GenerateConfig` from library.
- Remove unused `FirebaseConfig` from library.
- Fix loading of `config/status-chain-genesis.json` in non-dev machines.
2018-09-13 18:31:29 +02:00
Adam Babik bf1d1ee0ea
Remove EnvelopeTranced and add Whisper metrics as a patch (#1177) 2018-08-31 10:09:38 +02:00
Adam Babik 776248a42f
Prevent RendezvousNodes to be overwritten by empty cli flag (#1174) 2018-08-27 16:10:58 +02:00
Andrea Franz 874a3e8151
Feature/mailserver registry smart contract (#1135)
* add verifier and test using simulated backend

* add ContractCaller

* commit simulated backend after deploy and after smart contract writes

* use bind.NewKeyedTransactor for all transactions in tests

* rename RegistryVerifier to Verifier

* initialize contract verifier if MailServerRegistryAddress config is set

* use contractAddress.Hash()

* refactoring

* use fmt.Sprintf to format contract address in logs

* fix test and lint warnings

* update Gopkg.lock

* update Gopkg.lock once more
2018-08-20 15:55:43 +02:00
Adam Babik cf21f981f7
Fix rendezvous loop when discovery protocol is stopped; add advertiseaddr flag (#1151) 2018-08-17 08:25:55 +02:00
Pedro Pombeiro fb1e30c5f4 Disable NTP sync on the remaining e2e tests 2018-08-07 18:02:42 +02:00
Pedro Pombeiro 840b5b64d2 Implement mailserver canary service. Closes #1086 2018-08-07 17:17:59 +02:00
Dmitry a76bf4ec4d Implement discovery multiplexer 2018-08-01 08:28:49 +03:00
Adam Babik 3b8c6c8260
Add interface to verify trusted MailServers (#1112)
The goal of this PR is to add an interface to verify MailServers. In this PR, MailServers are hardcoded in status-go. The next iteration will use a smart contract.
2018-07-25 16:48:02 +02:00
Dmitry eeca435064 Add rendezvous implementation for discovery interface
Update vendor

Integrate rendezvous into status node

Add a test with failover using rendezvous

Use multiple servers in client

Use discovery V5 by default and test that node can be started with rendezvous discovet

Fix linter

Update rendezvous client to one with instrumented stream

Address feedback

Fix test with updated topic limits

Apply several suggestions

Change log to debug for request errors because we continue execution

Remove web3js after rebase

Update rendezvous package
2018-07-25 15:10:57 +03:00
Adrià Cidre bfbb02019f
Discover mail servers on demand. (#1082)
* [#1076] Discover mail servers on start up.

* [#1076] On-demand mail-server discovery
2018-07-16 09:40:40 +02:00
Dmitry 40fa97ef33 Do not pass discovery option to p2p server as we are managing it now 2018-07-06 11:30:16 +03:00
Dmitry 737f966dbe Decouple discovery from p2p server 2018-07-05 16:45:28 +03:00
Adam Babik 38a60135b2 Add asymmetric key support for MailServer requests (#1075)
* add Asymmetric Key support for MailServer requests

* remove deprecated notice

* fix linter

* refactoring Whisper config related to MailServer

* fix race condition
2018-07-04 11:30:57 +02:00
Adrià Cidre 3ca120c2aa
[#1054] : Introducing t/helpers and WaitForPeer (#1064)
* [#1054] : Introducing t/helpers and WaitForPeer

* waitForPeer to receive a p2p.Server
2018-06-27 09:55:25 +02:00
Adrià Cidre 5bbfabde94
New debug_postSync endpoint (#1046)
* [#ISSUE-1041] New debug_postconfirm endpoint

* move debug service api inside shhext service
2018-06-25 15:27:17 +02:00
Pedro Pombeiro 354e23aaf5 Fix issues reported by lint. Part of #1017 2018-06-14 13:52:51 +02:00
Adrià Cidre 6173d3ee06
[#1022] Make ntp synchronization configuratble (#1026) 2018-06-13 13:24:04 +02:00
Adrià Cidre d5be8c525d
[#856] move geth subpackages to root level (#1007) 2018-06-08 13:29:50 +02:00