Commit Graph

214 Commits

Author SHA1 Message Date
Adam Babik e93d994460 integrate status-protocol-go 2019-07-24 18:46:43 +02:00
Adam Babik e28d4ef1a3
Create messaging package and move some packages into it (#1505) 2019-06-26 20:17:41 +02:00
Andrea Maria Piana 1aa3e2812a Add support for partitioned topic 2019-06-19 17:38:45 +02:00
Andrea Maria Piana cef7f367ab Add topic negotiation
This commit add topic negotiation to the protocol.

On receiving a message from a client with version >= 1, we will generate
a shared key using Diffie-Hellman. We will record also which
installationID has sent us a message.

This key will be passed back to the above layer, which will then use to
start listening to a whisper topic (the `chat` namespace has no
knowledge of whisper).

When sending a message to a set of InstallationIDs, we check whether we
have agreed on a topic with all of them, and if so, we will send on this
separate topic, otherwise we fallback on discovery.

This change is backward compatible, as long as there is no downgrade of
the app on the other side.

A few changes:

* Factored out the DB in a separate namespace as now it is
being used by multiple services (TopicService and EncryptionService).

* Factored out multidevice management in a separate namespace

* Moved all the test to test the whole protoocl rather than just the encryption service

* Moved all the filter management in status-go
2019-06-19 17:38:45 +02:00
Dmitry Shulyak 804a109b26
Set BINARY variable for golangci-lint.sh (#1493)
* Set BINARY variable for golangci-lint.sh

* Set higher deadline for linter
2019-06-13 13:24:28 +03:00
Adam Babik 1c33220834 update linter 2019-06-12 13:12:00 +02:00
Adam Babik c9e99c432d migrate to go 1.12 and go modules 2019-06-12 13:12:00 +02:00
Andrea Maria Piana 4ab08629f6 Add postgres
This commits adds support for postgres database.
Currently two fields are stored: the bloom filter and the topic.
Only the bloom filter is actually used to query, but potentially we will
use also the topic in the future, so easier to separate it now in order
to avoid a migration.
2019-05-15 11:01:34 +02:00
Igor Mandrigin 4c1f96d255
Implement subscription for eth and shh filters using signals. (#1455) 2019-05-07 09:05:38 +02:00
Dmitry Shulyak 442a12e996
Removing discovery-proxy from sources (#1445)
It was going to be used as proxy from discv5 to rendezvous for les nodes.
We are not using it at the moment, and not clear when we will be using it
and if we are going to use it all.

Additionally it has some memory leaks that need to be fixed, so it is better
to remove it for now and restore once/if we will need it again.
2019-04-18 10:39:55 +03:00
Jakub 2c0c0fff24 fix release builds (#1440)
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2019-04-15 11:57:06 +02:00
Jakub Sokołowski 2af27dc6b3 build regular linux binary with xgo for uploading to GitHub
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2019-04-14 18:14:02 +02:00
Jakub Sokołowski d38a07f679 add Jenkinsfile.docker for pushing new images
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2019-03-26 09:14:56 +01:00
Jakub Sokołowski a4453a2471 forgot to update main README with link to instructions
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2019-03-25 09:47:16 +01:00
Adam Babik c3b0644bbc
prefix git tag with 'v' (#1418) 2019-03-18 09:14:21 +01:00
Dmitry 9441f798bb Remove mock step from ci pipeline 2019-03-15 08:14:53 +02:00
Igor Mandrigin 0282f481bf
Fix `admin_nodeInfo` to provide the correct version again (#1394) 2019-02-25 17:23:30 +01:00
Jakub 0238d84924 declare all parameters, fix names of files on release (#1392)
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2019-02-25 15:59:12 +01:00
Jakub Sokołowski 51ff6395b8 fix env variables pointing at android ndk for gomobile
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2019-02-25 12:45:06 +01:00
Jakub Sokołowski c26cfb0468 add a docker-compose.yml and Makefile for running a mailserver
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2019-02-13 18:05:39 +01:00
Jakub Sokołowski de269e5830 add build status notifications like in status-react
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2019-02-11 17:16:21 +01:00
Ivan Daniluk 4f3f5ee574 Gomobile support (#1164) 2019-02-01 18:02:52 +01:00
Adam Babik 74cb16c456
add SyncMessages method to shhext api (#1309) 2018-12-14 12:21:34 +01:00
Adam Babik 2f65dd58f9
Enable metrics by default and pass build flags to statusgo-library (#1320) 2018-12-14 12:16:45 +01:00
Jakub Sokołowski 7d651afaae add desktop build step
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2018-12-11 09:30:35 +01:00
Adam Babik db691fd763
fix golang/mock (#1310) 2018-12-10 13:53:27 +01:00
Rob Culliton 3657b8b057
change echo message for linux compilation (#1307) 2018-12-07 08:35:05 -05:00
Adam Babik e658366d1e
Improve Makefile (#1275)
* change docker image tag to RELEASE_TAG
* use changelog in release
2018-11-20 11:13:42 +01:00
Pedro Pombeiro 92748f1334 Add `canary-test` make target to test beta fleet mailservers. Closes #1274 2018-11-19 19:02:32 +01:00
Adam Babik 52a1bdfed6
Upgrade geth 1.8.17 plus add metrics during compilation time (#1273)
This commit updates geth to 1.8.17 and adds a possibility to enable metrics during compilation time.

The cascade of issues forced us to upgrade geth to 1.8.17 in order to allow enabling metrics during compilation time. 1.8.17 introduced `NodeID` refactoring and `enode` package which affected our peers pool and integration with Discovery V5.
2018-11-14 08:03:58 +01:00
Andrea Maria Piana 58bd36e79e
Automate github releases (#1263) 2018-11-05 15:02:16 +01:00
Adam Babik b71058d73f
update golangci to 1.10.2 (#1246) 2018-10-17 12:17:58 +02:00
Adam Babik ac8da3c0f0
Use patched go-ethereum from our fork instead of patching it on-the-fly (#1184)
* update to 1.8.16
2018-09-27 21:16:15 +02:00
Pedro Pombeiro 6a1cd7f2b0
Fix xgo creating files as `root` (#1210) 2018-09-24 22:21:13 +02:00
Andrea Maria Piana dcaf8caed0
Add x3dh key exchange (#1127)
* Add x3dh key exchange

* Encrypt using the double ratchet

* Multi device with auto-pairing

* Add pfs enabled flag
2018-09-24 20:07:34 +02:00
Pedro Pombeiro 518234803a
Add support for testing static nodes to node-canary. Closes #1202 (#1203) 2018-09-20 13:25:32 +02:00
Pedro Pombeiro 4e7ae20eae
Add script to update fleets static JSON. Closes #1188 (#1189) 2018-09-14 10:46:34 +02:00
Pedro Pombeiro 3d00af7fa3
Streamline configuration in status-go. Part of #1180 (#1183)
- Replace command line flags with `-c` config flag. Part of #1180
- Convert node config private keys to hex-encoded string versions.
- Remove `GenerateConfig` from library.
- Remove unused `FirebaseConfig` from library.
- Fix loading of `config/status-chain-genesis.json` in non-dev machines.
2018-09-13 18:31:29 +02:00
Adam Babik e8c6841f30
fix getting git commit in Makefile (#1181) 2018-09-04 17:19:16 +02:00
Adam Babik 5664a24b10
remove 'v' from version from git tag in Makefile (#1178) 2018-08-31 07:28:12 +02:00
Adam Babik 16c7b94eb5
Add labels to docker images and add Jenkinsfile to build docker images (#1160)
* fixed generating `params.Version`
* added labels to all Docker images including git_commit and the author of the build (using `$USER` env variable)
2018-08-24 12:25:07 +02:00
Dmitry Shulyak 6858662d59
Proxy records found in ethereum discovery v5 (#1165) 2018-08-24 06:17:32 +03:00
Kishan Sagathiya 0f5ad2545d Issue #765 Read configuration without go-bindata (#1158)
In the current codebase if you want to use different keys (or any
other configuration) for tests it is required to place them at a
certain path in static folder, bundle them with go-bindata and only
then run tests. This is simply unnecessary.

This commit instead directs loads configurations from file without
relying on go-bindata
2018-08-21 14:46:10 +02:00
Pedro Pombeiro ebc77374b2 Remove unused `test.sol` file
- File is no longer used after jail code was removed
- Ensure that `go-bindata` is set up on `make setup`
2018-08-15 14:52:40 +02:00
Adam Babik f26e849fb2
Revert "Add build tags to signal package (#1119)" (#1148)
This reverts commit 27e432a5b1.
2018-08-10 13:23:28 +02:00
Pedro Pombeiro 840b5b64d2 Implement mailserver canary service. Closes #1086 2018-08-07 17:17:59 +02:00
Ivan Daniluk 27e432a5b1
Add build tags to signal package (#1119) 2018-08-07 11:10:20 +03:00
Ivan Daniluk 4b8c669154
Speedup Makefile (#1123) 2018-08-02 19:16:39 +02:00
Ivan Daniluk 801053a17d
Remove jail package (#1106) 2018-07-24 12:05:21 +02:00
Adam Babik 03bf6e37a3
silence error when calculating GIT_REMOTE without develop branch (#1107) 2018-07-20 13:32:26 +02:00
Adrià Cidre bfbb02019f
Discover mail servers on demand. (#1082)
* [#1076] Discover mail servers on start up.

* [#1076] On-demand mail-server discovery
2018-07-16 09:40:40 +02:00
Adam Babik f33c803614
make calculating GIT_REMOTE in Makefile more explicit (#1092) 2018-07-12 11:45:31 +02:00
Adrià Cidre f19e3eae8e
Move TestSentEnvelope to status-scale repo (#1071) 2018-07-02 22:29:03 +02:00
Pedro Pombeiro 7c51f9a382 Update to golangci-lint 1.9.1. Get rid of standalone misspell linter. 2018-07-02 18:41:10 +02:00
Jakub 4f9a7a908e Make sure all new docker images are also tagged with commit (#1062)
This change also defines new makefile targets which prevent from pushing `latest` image tags by accident.
2018-07-02 18:27:59 +02:00
Adrià Cidre 5bbfabde94
New debug_postSync endpoint (#1046)
* [#ISSUE-1041] New debug_postconfirm endpoint

* move debug service api inside shhext service
2018-06-25 15:27:17 +02:00
Pedro Pombeiro 3b052098fe Update to geth 1.8.11. Closes #1033
- Replace deprecated common.Hex with hexutil.Encode.
- Remove upstreamed 0010-geth-17-fix-npe-in-filter-system.patch.
- Remove upstreamed 0020-discv5-metrics.patch.
- Remove upstreamed 0026-ethdb-error-deadlock.patch.
- Update goleveldb to same version used by geth 1.8.11.
- Update PublicTransactionPoolAPI.GasPrice return type to match type in internal geth interface.
2018-06-22 18:04:13 +02:00
Pedro Pombeiro 3abfe3ffda Replace gometalinter with golangci-lint (4.5x faster). Closes #1017 2018-06-14 13:52:51 +02:00
Ivan Daniluk 5f0768472f
Add GOPATH check to Makefile (#995) 2018-06-14 13:28:09 +02:00
Adam Babik bbe7fba0c9
Benchmark tests (#986)
A set of tests used to benchmark Whisper and MailServer nodes.
2018-06-11 12:48:42 +02:00
Adrià Cidre d5be8c525d
[#856] move geth subpackages to root level (#1007) 2018-06-08 13:29:50 +02:00
Adam Babik 8ce6efc749
Make statusd distinct from StatusIM client (#1008) 2018-06-08 11:20:30 +02:00
Andrea Franz 5d0120ee28
add Dockerfile and Makefile tasks to build statusd-prune docker image (#1005)
* add Dockerfile and Makefile tasks to build statusd-prune docker image

* add image ENTRYPOINT

* set default CMD to statusd-prune
2018-06-07 12:30:18 +02:00
Dmitry Shulyak ed1cdf0418
Custom status bootnodes (#968)
This change makes invalidation mechanism more aggressive. With a primary goal to invalidate short living nodes faster. In current setup any node that became known in terms of discovery will stay in this state until it will fail to respond to 5 queries. Removing them earlier from a table allows to reduce latency for finding required nodes.

The second change, one adds a version for discovery, separates status dht from ethereum dht.
After we rolled out discovery it became obvious that our boot nodes became spammed with irrelevant nodes. And this made discovery process very long, for example with separate dht discovery takes ~2s, with mutual dht - it can take 1m-10m and there is still no guarantee to find a max amount of peers, cause status nodes is a very small part of whole ethereum infra.

In my understanding, we don't need to be a part of ethereum dht, and lower latency is way more important for us.

Closes: #941
Partially closes: #960 (960 requires futher investigations on devices)
2018-05-18 16:43:07 +03:00
Ivan Danyliuk 6fec9fa653 Bump up Go version to 1.10 2018-05-09 16:13:34 +02:00
Igor Mandrigin 153d4160e1 Fix `make ci` if `dep` wasn't pre-installed. 2018-05-04 14:17:17 +02:00
Adrià Cidre 8c9db81bec
Expose status specific methods (login/signup/joinpublicchannel) through the RPC api (#877) 2018-05-03 12:36:56 +02:00
Pedro Pombeiro 4438d116b1 Ensure that Travis CI fails the build if the patches are incorrect 2018-05-02 13:28:39 +02:00
Igor Mandrigin f4cd8d27b5 Add `shhext_getNewFilterMessages` function to RPC APIs.
This function returns only the new messages from the filter, never
returns the same message for the same user twice.
2018-04-27 15:24:37 +02:00
Igor Mandrigin 4cc6028d59 Implement `personal_sign`.
This commit implements `personal_sign` RPC or web3 personal.sign
methods.

NB! Contains breaking API changes.
2018-04-17 11:18:58 +02:00
Adrià Cidre e277ae8b2d
Mockgen import cycle not allowed && recover missing mockgens (#824) 2018-04-13 15:10:37 +02:00
Adam Babik 0d652c3851
Expose only public APIs in inproc RPC client [breaking-change] (#815)
`CallRPC` binding, which is used as a provider for web3.js, exposes only public or whitelisted APIs.
2018-04-12 18:17:10 +02:00
Dmitry Shulyak fc3decf661 Add tests for topic flags 2018-04-10 16:18:02 +03:00
Daniel Regeci e34af8d6eb Create status-go API to return a current revision (#804)
Change `BUILD_FLAGS` in Makefile and update `github.com/status-im/status-go/geth/params.VersionMeta` value with the current Git SHA.
2018-04-10 12:36:11 +02:00
Dmitry Shulyak b120587c9b Update bindata 2018-04-09 15:42:36 +02:00
Adam Babik e9c2292776
Switch mockgen to reflect mode (#800)
Switch mockgen to use reflect mode as source mode does not support vendored packages.
2018-04-05 15:14:47 +02:00
Igor Mandrigin d2df4bb323 Unbreak `make mock` command and compilation of our develop branch. 2018-03-29 17:51:00 +02:00
Pedro Pombeiro f329ad2256
Add `setup` makefile target and rename `race-check` to `test-e2e-race` for consistency
- `make setup` is used to simplify CI script
2018-03-27 17:07:06 +02:00
Pedro Pombeiro c4d7a6167e
Add `race-check` target to Makefile. Closes #710 (#769)
* Add `race-check` and `test-unit-race` target to Makefile. Closes #710
2018-03-27 14:38:41 +02:00
Adam Babik 8b264199c9
Enable metrics by default when building docker image (#774)
Prometheus metrics are enabled by default when running `make docker-image`.
2018-03-27 14:36:40 +02:00
Adrià Cidre e22fa36111
[#755] Restructure notifier interfaces 2018-03-23 09:55:05 +01:00
Pedro Pombeiro d84510de92
Remove unused JavaScript tests for RPC API (#737) 2018-03-16 18:49:42 +01:00
Frank Mueller 81c8669518
Code review for Mainnet (#706)
* Add debug log to test package t

* Add comments for test restrictions

* Add explicit errors for mainnet

* Panic instead of error in case of mainnet during tests

* Now panic with illegal network

* Validate the compiler flag for mainnet

* Made test for unset mainnet explicit

* Forgot to use my flag

* Linter has troubles with unused field

* Add default network id

In e2e tests there's no default anymore in go code. So ensure in Makefile.

* Fix dumb syntax error

* CI needs build with Mainnet

* Explicit rebuild only for tests

* No more separation of mainnet and testnet builds

* Build flags now set in Makefile
2018-03-15 12:36:28 +01:00
Adrià Cidre dea43dc4ed Makefile GOBIN to use absolute path 2018-03-01 17:18:12 +01:00
Pedro Pombeiro e4cbce12c4
Update to go-ethereum 1.8.1 (#702)
* Update `github.com/ethereum/go-ethereum` package to 1.8.1 branch. Part of #638
* Fix code due to some signature changes. Part of #638
* use upstream for whisper backend
* Add patch to downgrade usage of Whisper v6 to v5 in some geth 1.8.1 vendor files. Part of #638
* Take into account the DNS rebinding protection introduced in 1.8.0 by adding exception for localhost. Part of #638
* Add patches required for cross-compiled builds starting with geth 1.8.0. Only applied during build. Part of #638
* Update expected JSON result in `TestRegressionGetTransactionReceipt()` and `TestCallRawResultGetTransactionReceipt()`. Part of #665
* Fix some failing e2e tests. Part of #638
* Address comments in PR #702. Part of #638
2018-02-27 11:39:30 +01:00
Adam Babik 0a15c9ff9a
Configure mainnet nodes (#701)
Add mainnet static peers to static/config/staticpeers.json.
2018-02-26 19:36:04 +01:00
Pedro Pombeiro 0332465952
Update `make generate` target to account for new `_assets` folder 2018-02-26 16:04:40 +01:00
Pedro Pombeiro 134d01fcc1
Fix incorrect path in Makefile and run `make generate`. Part of #692 2018-02-22 12:19:31 +01:00
Alex Kohler c5b9f00467 Adding update-geth target (#673) 2018-02-20 19:04:25 +01:00
Dmitry Shulyak b2580c79d7
Add destructive tests (#646)
Network disconnect is introduced by removing default gateway, easily reversible condition.

On my local machine it takes 30 seconds for peers to reconnect after connectivity is restored. As you guess this is not an accident, and there is 30 seconds timeout for dial expiration. This dial expiration is used in p2p.Server to guarantee that peers are not dialed too often.

Additionally I added small script to Makefile to run such tests in docker environment, usage example:

```
make docker-test ARGS="./t/destructive/ -v -network=4"
```
2018-02-19 17:32:58 +02:00
Caner Çıdam 15c72f3c6b Apply geth patches on the fly #653 (#660) 2018-02-14 18:44:27 +01:00
Adam Babik 9ed2663b6e add docker-image-tag target to Makefile 2018-02-13 10:30:47 +01:00
Caner Çıdam a4c36fe96a Move non-go related files to the `_assets` folder. #621 (#630) 2018-02-09 14:38:18 +01:00
Sebastian Klier 70912ab8f6 Group tests in a dedicated testing package (#629) 2018-02-08 13:52:47 +01:00
Athos Ribeiro d58a06c323 Fix string printed after compilation is done (#505)
* Fix string printed after compilation is done

On GNU coreutils 8.27, echo prints the \n occurrences as literals. This
commit changes the calls to echo containing such character to calls to
printf, and appends another \n in the end of the line.

* Fix indent
2018-02-02 16:54:14 +01:00
Azer Koçulu 8b56060e21 Add vendor-check script, Makefile target and CI step (#581) 2018-02-01 18:14:51 +01:00
Adam Babik 89d89681a3
Add support for metrics (#558)
It adds support for metrics (expvar and Prometheus) along with docker-compose files to run a Whisper test cluster.
2018-01-30 12:51:48 +01:00
Dmitry Shulyak 680d0513b7 Refactoring of TxQueue and Manager (#530) 2018-01-25 21:59:21 -08:00
Daniel Nephin 755a646bd4 Use gometalinter config instead of makefile (#560) 2018-01-24 00:25:28 -08:00
Dmitry Shulyak 0771e7d1b7 Use single codepath for sending transactions to a local and remote nodes (#527)
- new EthereumTransactor that provides higher level API for working with ethereum network, and it is fully conformant with ethclient
- new test rpc service that improves flexibility and coverage of txqueue manager tests
- run complete transaction sequantially for each address
- go-ethereum: Get pending nonce from transaction pool
- add a patch with getting nonce from transaction pool
2018-01-18 17:55:17 +01:00