Commit Graph

206 Commits

Author SHA1 Message Date
Andrea Maria Piana e58ba1e9c8
Add audio messages
Why make this change?

We are adding support of audio recorded files, similarly to how we did
with images

What has changed?

- Added protobuf definition, only AAC supported
- Added migrations to store files
- Fixed an issue with nil pointer when transaction would fail to be
created, causing the application to crash
2020-07-27 17:14:50 +02:00
Andrea Maria Piana cbaf523b0e Address feedback 2020-06-03 11:07:04 +02:00
Andrea Maria Piana f5ab58b87f Add support for images
This commit adds support for images in protobuf messages.

The client can specify a path which will be used to load the image
and set the corresponding fields.

This makes the assumption that the RCP server runs on the same machine
as the client and they have access to the same files. This holds
currently for both status-react and status-console-client, we could
revisit and adds an upload if that changes in the future.
2020-06-01 11:30:04 +02:00
Jakub Sokołowski 42baf2251d expose the mailserver metrics port by default
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2020-04-08 17:08:49 +02:00
Jakub Sokołowski bc70895615 enable waku for mailservers by default
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2020-04-08 17:08:49 +02:00
Adam Babik 4f9f7cd258
Fix setup-dev target (#1926) 2020-04-06 09:54:06 +02:00
Jakub Sokołowski 0c8efb4da5 fix update-fleet-config.sh to include rendezvous
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2020-03-17 15:47:07 +01:00
Adam Babik 6fdeb9b115
Fix listen flag (#1897) 2020-03-10 13:40:35 +01:00
Jakub Sokołowski 86f54c3f5f remove eth.beta fleet as it has been decomissioned
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2020-02-21 14:48:47 +01:00
Jakub Sokołowski e1dee26ba9 docs: clarify purpose and describe ports
Also make bootnode use 30301 by default to avoid clash/confusion.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2020-02-06 18:44:23 +01:00
Jakub Sokołowski 14501520be add option to show enode as QR code if qrencode is available
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2020-02-05 16:50:49 +01:00
Jakub Sokołowski b476d5c6f3 include mail password when getting enode with get_enode.sh
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2020-02-05 16:50:49 +01:00
Jakub Sokołowski d29c6aad50 use more descriptive names for systemd services
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2020-01-08 14:51:26 +01:00
Jakub Sokołowski fc841f884a add config for new eth.prod fleet
also move _assets/ci/update-fleet-config.sh to scripts

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2020-01-08 14:51:26 +01:00
Jakub Sokołowski 1d4b3b9241 allow running systemd service as root
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2020-01-02 10:36:13 +01:00
Pedro Pombeiro 0ed66b98ae Fail build if `make vendor` hasn't been run 2019-12-30 13:25:12 +01:00
Jakub Sokołowski ccf3cd5098 add missing jq checks, make formatting vars the same
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2019-12-21 09:16:54 +01:00
Jakub Sokołowski 0f06b4ca6a fix entrypoint for bootnode Docker image
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2019-12-17 22:04:34 +01:00
Jakub Sokołowski 1eb0ad91bd add extra checks to docker compose Makefiles
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2019-12-17 22:04:34 +01:00
Jakub Sokołowski e249f35a8d fix missing genkey task in systemd bootnode makefile
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2019-12-10 13:46:32 +01:00
Jakub Sokołowski 429d2b4291 create a Makefile for running a bootnode under systemd
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2019-12-04 13:44:05 +01:00
Jakub Sokołowski e139af9a16 rewrite mailserver systemd setup to a Makefile
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2019-12-04 13:44:05 +01:00
Jakub Sokołowski 92d6b1c439 add systemd service leanup, fix config generation
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2019-11-15 11:30:23 +01:00
Jakub Sokołowski c1f17cb4a3 use prometheus/client_golang for metrics
also re-implement mailserver metrics

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2019-11-04 13:18:50 +01:00
Adam Babik a244b01a26
Upgrade migrate (#1643) 2019-10-14 16:10:48 +02:00
Jakub Sokołowski 294399916e add script for easier setup of mailserver
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2019-10-10 11:27:32 +02:00
Adam Babik e164cbe421
Upgrade Go to 1.13 in our Docker images (#1629) 2019-10-05 16:27:48 +02:00
Adam Babik 94228bce2e
Remove xgo builds (#1628) 2019-10-05 11:13:53 +02:00
Jakub Sokołowski 4f1a3283e6
update one missing location where old pkg dir is used
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2019-08-20 09:28:48 -04:00
Jakub Sokołowski 652d8a67b6 use out for output, use CGO_ENABLED=1 and GO111MODULE=off for Android and iOS (#1574) 2019-08-20 09:05:15 +02:00
Jakub Sokołowski 5f763439cb add a check for already existing release tag
also run cleanup in post-always Jenkinsfile block

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2019-07-30 09:26:34 -04:00
Jalil F 19c26b129d Update mailserver docker readme. (#1528)
Include enode details, update make config doc, and add a F.A.Q.

Co-Authored-By: Adam Babik <a.babik@designfortress.com>
2019-07-22 08:57:29 -04:00
Adam Babik 0e13406151 remove GOCACHE=off from Jenkinfile.xgo 2019-06-12 13:12:00 +02:00
Jakub Sokołowski 012a1b54bc add /usr/local/go/bin as proper Go path
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2019-06-12 13:12:00 +02:00
Jakub Sokołowski b9829e0fca use newer image for building go docker image
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2019-06-12 13:12:00 +02:00
Adam Babik c9e99c432d migrate to go 1.12 and go modules 2019-06-12 13:12:00 +02:00
Dmitry Shulyak 442a12e996
Removing discovery-proxy from sources (#1445)
It was going to be used as proxy from discv5 to rendezvous for les nodes.
We are not using it at the moment, and not clear when we will be using it
and if we are going to use it all.

Additionally it has some memory leaks that need to be fixed, so it is better
to remove it for now and restore once/if we will need it again.
2019-04-18 10:39:55 +03:00
Jakub 2c0c0fff24 fix release builds (#1440)
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2019-04-15 11:57:06 +02:00
Jakub Sokołowski 2af27dc6b3 build regular linux binary with xgo for uploading to GitHub
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2019-04-14 18:14:02 +02:00
Jakub Sokołowski 96aba9f3af explicitly enable whisper for mailserver
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2019-04-01 14:38:41 +02:00
Jakub Sokołowski 3c6d10b597 bootnodes listen on UDP
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2019-04-01 12:14:18 +02:00
Jakub bf209cff85 don't overwrite linux build variable with docker build (#1431)
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2019-03-29 18:49:18 +01:00
Jakub Sokołowski b80a9f5aaa add explicit TMPDIR, otherwise go get fails
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2019-03-28 09:55:34 +01:00
Jakub Sokołowski d38a07f679 add Jenkinsfile.docker for pushing new images
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2019-03-26 09:14:56 +01:00
Jakub Sokołowski 2766372eca add compose setup for a bootnode
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2019-03-21 16:42:44 +01:00
Dmitry 9441f798bb Remove mock step from ci pipeline 2019-03-15 08:14:53 +02:00
Jakub Sokołowski cbe50f557c move running tests to a separate jenkins job
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2019-03-05 08:11:32 +01:00
Jakub 0238d84924 declare all parameters, fix names of files on release (#1392)
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2019-02-25 15:59:12 +01:00
Jakub Sokołowski 51ff6395b8 fix env variables pointing at android ndk for gomobile
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2019-02-25 12:45:06 +01:00
Jakub Sokołowski 0995f7c878 fix Makefile respects env variables exported by user
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2019-02-25 09:45:58 +01:00
Jakub Sokołowski c26cfb0468 add a docker-compose.yml and Makefile for running a mailserver
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2019-02-13 18:05:39 +01:00
Andrea Franz a249151d05 add LoginWithKeycard to mobile pkg (#1372) 2019-02-11 21:20:59 +01:00
Jakub Sokołowski de269e5830 add build status notifications like in status-react
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2019-02-11 17:16:21 +01:00
Ivan Daniluk 4f3f5ee574 Gomobile support (#1164) 2019-02-01 18:02:52 +01:00
Jakub Sokołowski 7d651afaae add desktop build step
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2018-12-11 09:30:35 +01:00
Andrea Maria Piana afc3017e07
Notify user when the device is missing (#1298)
* Notify user when the device is missing

* Update services/shhext/chat/encryption.go

Co-Authored-By: cammellos <andrea.maria.piana@gmail.com>
2018-12-05 09:22:49 +01:00
Jakub Sokołowski 49cfcd3e24 stop adding 'v' before the contents of VERSION file
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2018-12-03 14:55:45 +01:00
Jakub Sokołowski d61e838235 use VERSION file for suffix when releasing
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2018-11-27 19:26:09 +01:00
Jakub Sokołowski 6b09eea749 install xgo in setup target
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2018-11-26 14:07:48 +01:00
Jakub Sokołowski 07eb60df7d
pyt timestmap before commit for better ordering
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2018-11-26 13:33:07 +01:00
Jakub Sokołowski def3af61a7
remove unused call
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2018-11-26 09:05:33 +01:00
Jakub Sokołowski d1847a3adc change Jenkinsfile to use parallel declarative format
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2018-11-22 10:12:20 +01:00
Pedro Pombeiro 92748f1334 Add `canary-test` make target to test beta fleet mailservers. Closes #1274 2018-11-19 19:02:32 +01:00
Andrea Maria Piana 58bd36e79e
Automate github releases (#1263) 2018-11-05 15:02:16 +01:00
Adam Babik ac8da3c0f0
Use patched go-ethereum from our fork instead of patching it on-the-fly (#1184)
* update to 1.8.16
2018-09-27 21:16:15 +02:00
Dmitry Shulyak 529e18af33
Update geth to v1.8.15 (#1213)
* Update geth to v1.8.15

* Apply patches
2018-09-25 11:55:54 +03:00
Pedro Pombeiro 6a1cd7f2b0
Fix xgo creating files as `root` (#1210) 2018-09-24 22:21:13 +02:00
Andrea Maria Piana dcaf8caed0
Add x3dh key exchange (#1127)
* Add x3dh key exchange

* Encrypt using the double ratchet

* Multi device with auto-pairing

* Add pfs enabled flag
2018-09-24 20:07:34 +02:00
Pedro Pombeiro 99d7bc207d
Only keep two random static nodes from fleets.status.im (#1196) 2018-09-19 17:08:54 +02:00
Pedro Pombeiro 4e7ae20eae
Add script to update fleets static JSON. Closes #1188 (#1189) 2018-09-14 10:46:34 +02:00
Pedro Pombeiro 3d00af7fa3
Streamline configuration in status-go. Part of #1180 (#1183)
- Replace command line flags with `-c` config flag. Part of #1180
- Convert node config private keys to hex-encoded string versions.
- Remove `GenerateConfig` from library.
- Remove unused `FirebaseConfig` from library.
- Fix loading of `config/status-chain-genesis.json` in non-dev machines.
2018-09-13 18:31:29 +02:00
Adam Babik bf1d1ee0ea
Remove EnvelopeTranced and add Whisper metrics as a patch (#1177) 2018-08-31 10:09:38 +02:00
Adam Babik 4e386f4097
Update version format of artifacts in Jenkins (#1173) 2018-08-29 12:52:19 +02:00
Adam Babik a571eeb7a4
add patch 0036-fix-whisper-expire-loop (#1175) 2018-08-28 10:49:08 +02:00
Dmitry Shulyak f150d678de
Update geth to 1.8.14 (#1171)
* Update to geth v1.8.14

* Remove patches that were merged upstream

* Apply patches before 0016

* Fix 0016 and apply it

* Apply everything else

* Pass gas limit as a second argument to simulated backend
2018-08-27 11:22:21 +03:00
Adam Babik 16c7b94eb5
Add labels to docker images and add Jenkinsfile to build docker images (#1160)
* fixed generating `params.Version`
* added labels to all Docker images including git_commit and the author of the build (using `$USER` env variable)
2018-08-24 12:25:07 +02:00
Dmitry Shulyak 2cfcde0283
Remove patches to discovery v5 (#1144) 2018-08-24 11:24:22 +03:00
Dmitry Shulyak 6858662d59
Proxy records found in ethereum discovery v5 (#1165) 2018-08-24 06:17:32 +03:00
Dmitry Shulyak d8b5ba7fbf
Add generated cht for ropsten and mainnet (#1166) 2018-08-24 06:05:06 +03:00
Dmitry bfdf823628 Add delay to refresh cycle with patch 2018-08-15 04:25:18 +03:00
Dmitry 6c6fb4d0b3 Remove patch 0031 2018-08-15 04:25:18 +03:00
Adam Babik 7c70ce02bc
allow building status-go from tags (#1122) 2018-08-10 10:28:04 +02:00
Adam Babik 4219d56909
Fix xgo patch 0002 (#1141) 2018-08-08 09:39:51 +02:00
Adam Babik ecc39735f5
Upgrade geth to 1.8.13 (#1140) 2018-08-07 15:31:06 +02:00
Jakub Sokołowski ef7da275d4
fix ENTRYPOINT syntax and add CMD for easier use by new people; 2018-07-11 13:45:27 +02:00
Jakub 6d4f657106 Add ENTRYPOINT for status-go image (#1084) 2018-07-10 11:29:40 +02:00
Ivan Daniluk 93210061ad
Add goroutines count to geth metrics (#1077) 2018-07-03 20:15:16 +02:00
Andrea Franz 809db97e54
mailserver pagination (#1039)
* mailserver sends envelopes in descending order

* add limit value in mailserver request payload

* mailserver sends messages up to the limit specified in the request

* update Archive method to return key and error

* processRequest returns the next page cursor

* add cursor to mailserver request

* add limit and cursor to request payload

* fix request limit encoding

* wait for request completed event in TrackerSuite/TestRequestCompleted

* add cursor to mailserver response

* fix cursor position in payload

* add e2e test for mail server pagination

* validate mail server response size

* remove old limitReached var

* fix lint warnings

* add whisper patch

* fix tests after rebase

* check all return values to avoid lint warnings

* check that all messages have been retrieved after 2 paginated requests

* fix lint warnings

* rename geth patch

* merge mailserver patches into one

* add last envelope hash to mailserver response and EventEnvelopeAvailable event

* update whisper patch

* add docs to MailServerResponse

* update whisper patch

* fix tests and lint warnings

* send mailserver response data on EventMailServerRequestCompleted signal

* update tracker tests

* optimise pagination test waiting for mailserver to archive only before requesting

* rollback mailserver interface changes

* refactoring and docs changes

* fix payload size check to determine if a limit is specified

* add more docs to the processRequest method

* add constants for request payload field lengths

* add const noLimits to specify that limit=0 means no limits
2018-07-02 09:38:10 +02:00
Andrea Franz b913c98573
remove formatting from metrics.ResettingTimer when requested in raw format (#1052) 2018-06-27 11:46:50 +02:00
Pedro Pombeiro 3b052098fe Update to geth 1.8.11. Closes #1033
- Replace deprecated common.Hex with hexutil.Encode.
- Remove upstreamed 0010-geth-17-fix-npe-in-filter-system.patch.
- Remove upstreamed 0020-discv5-metrics.patch.
- Remove upstreamed 0026-ethdb-error-deadlock.patch.
- Update goleveldb to same version used by geth 1.8.11.
- Update PublicTransactionPoolAPI.GasPrice return type to match type in internal geth interface.
2018-06-22 18:04:13 +02:00
Adrià Cidre 51ab9053ab
[#1040] Patch go-ethereum reducing from Warn to Info "message loop" logs (#1048) 2018-06-22 11:57:52 +02:00
Andrea Franz fa390a52ae
RequestHistoricMessages response (#1009)
* refactor TestRequestMessageFromMailboxAsync to use s.requestHistoricMessages helper

* send p2pRequestResponseCode from mailserver

* send p2p message response to after sending all historic messages

* mailserver sends `whisper.NewSentMessage` as response

* add mailserver Client and p2pRequestAckCode watchers

* send event with envelopeFeed when p2pRequestAckCode is received

* test request completed event in tracker

* rename mailserver response events and code to RequestCompleteCode

* wait for mailserver response in e2e test

* use SendHistoricMessageResponse method name for mailserver response

* fix lint warnings

* add mailserver request expiration

* send mailserver response without envelope

* add `ttl` to Request struct in shhext_requestMessages

* test that tracker calls handler.MailServerRequestExpired

* add geth patch

* rename TTL to Timeout

* split tracker.handleEvent in multiple methods
2018-06-15 17:12:31 +02:00
Pedro Pombeiro 354e23aaf5 Fix issues reported by lint. Part of #1017 2018-06-14 13:52:51 +02:00
Adam Babik 8ce6efc749
Make statusd distinct from StatusIM client (#1008) 2018-06-08 11:20:30 +02:00
Andrea Franz 5d0120ee28
add Dockerfile and Makefile tasks to build statusd-prune docker image (#1005)
* add Dockerfile and Makefile tasks to build statusd-prune docker image

* add image ENTRYPOINT

* set default CMD to statusd-prune
2018-06-07 12:30:18 +02:00
Adam Babik b5f05b1ab5
Fix flaky TestSingleTopicDiscoveryWithFailover and others (#992)
Other changes:
* needed to patch that loop implementation in Discover V5 implementation in go-ethereum,
* fixed TestStatusNodeReconnectStaticPeers,
* fixed TestBackendAccountsConcurrently.
2018-05-26 09:37:13 +02:00
Dmitry Shulyak ed1cdf0418
Custom status bootnodes (#968)
This change makes invalidation mechanism more aggressive. With a primary goal to invalidate short living nodes faster. In current setup any node that became known in terms of discovery will stay in this state until it will fail to respond to 5 queries. Removing them earlier from a table allows to reduce latency for finding required nodes.

The second change, one adds a version for discovery, separates status dht from ethereum dht.
After we rolled out discovery it became obvious that our boot nodes became spammed with irrelevant nodes. And this made discovery process very long, for example with separate dht discovery takes ~2s, with mutual dht - it can take 1m-10m and there is still no guarantee to find a max amount of peers, cause status nodes is a very small part of whole ethereum infra.

In my understanding, we don't need to be a part of ethereum dht, and lower latency is way more important for us.

Closes: #941
Partially closes: #960 (960 requires futher investigations on devices)
2018-05-18 16:43:07 +03:00
Adam Babik 6da469140f
Refactor in-app metrics (#963)
Remove Prometheus and other metric types and use metrics package from go-ethereum.
2018-05-16 17:36:59 +02:00
Dmitry Shulyak ca962ddaa3 Whisper broadcast loop spams way too hard with debug level 2018-05-16 17:54:40 +03:00
Dmitry Shulyak 5aae87aba8
Prevent sending messages to flaky peers (#917) 2018-05-15 20:08:31 +03:00