Commit Graph

190 Commits

Author SHA1 Message Date
Adam Babik 0c76505b95
expose more logs and metrics in DeliverMail (#1398) 2019-02-27 15:30:08 +01:00
Adam Babik 38aafe01db
Fix deadlock in mailserver DeliverMail method (#1388) 2019-02-22 09:55:37 +01:00
Adam Babik e38833b17a
add README for mailserver syncing (#1377) 2019-02-19 19:09:55 +01:00
Andrea Maria Piana da3964ba31
Change publishing envelopes behaviour (#1371) 2019-02-08 16:39:24 +01:00
Adam Babik d20b5dc3b3
mailserver: build pruning into WMailServer (#1342)
MailServer pruning was implemented as a separate command but that required stopping a mail server and executing the command manually. This change builds pruning into MailServer and can be set using MailServerDataRetention in WhisperConfig.
2019-01-23 14:32:45 +01:00
Adam Babik 8f2e347e4f
mailserver: refactor mailserver's rate limiter (#1341) 2019-01-10 17:07:16 +01:00
Adam Babik 74cb16c456
add SyncMessages method to shhext api (#1309) 2018-12-14 12:21:34 +01:00
Adam Babik e2682486fd
Fix cursor encoding (#1308)
It fixes encoding and decoding cursor sent in historic messages requests. `hex` package from the standard library is used.
2018-12-11 11:23:47 +01:00
Adam Babik 913dbfca9b
Support Mail Server data synchronization (#1302)
These changes add a support for syncing data between two Mail Servers. It does not give an easy way to start syncing. This will be solved in the next PR.
2018-12-06 10:48:28 +01:00
Adam Babik 860da591be
add mailserver batch requests counter and network processing time (#1280) 2018-11-19 10:14:03 +01:00
Adam Babik 52a1bdfed6
Upgrade geth 1.8.17 plus add metrics during compilation time (#1273)
This commit updates geth to 1.8.17 and adds a possibility to enable metrics during compilation time.

The cascade of issues forced us to upgrade geth to 1.8.17 in order to allow enabling metrics during compilation time. 1.8.17 introduced `NodeID` refactoring and `enode` package which affected our peers pool and integration with Discovery V5.
2018-11-14 08:03:58 +01:00
Adam Babik 0961e1039e
fix peerID logging in mail server (#1260) 2018-10-28 23:33:58 +01:00
Adam Babik a75f9c34cf
Request historical messages in batches (#1245) 2018-10-19 11:09:13 +02:00
Igor Mandrigin ba504e99c4
Mailserver: return error response. (#1244) 2018-10-18 12:25:00 +02:00
Andrea Franz b9baa48d87 Add metrics for mailserver errors (#1242) 2018-10-16 17:24:37 +02:00
Dmitry 5b551c67fa Use status-im whisper/whisperv6 module everywhere 2018-10-03 09:27:51 +03:00
Pedro Pombeiro 3d00af7fa3
Streamline configuration in status-go. Part of #1180 (#1183)
- Replace command line flags with `-c` config flag. Part of #1180
- Convert node config private keys to hex-encoded string versions.
- Remove `GenerateConfig` from library.
- Remove unused `FirebaseConfig` from library.
- Fix loading of `config/status-chain-genesis.json` in non-dev machines.
2018-09-13 18:31:29 +02:00
Dmitry Shulyak f150d678de
Update geth to 1.8.14 (#1171)
* Update to geth v1.8.14

* Remove patches that were merged upstream

* Apply patches before 0016

* Fix 0016 and apply it

* Apply everything else

* Pass gas limit as a second argument to simulated backend
2018-08-27 11:22:21 +03:00
Andrea Franz 874a3e8151
Feature/mailserver registry smart contract (#1135)
* add verifier and test using simulated backend

* add ContractCaller

* commit simulated backend after deploy and after smart contract writes

* use bind.NewKeyedTransactor for all transactions in tests

* rename RegistryVerifier to Verifier

* initialize contract verifier if MailServerRegistryAddress config is set

* use contractAddress.Hash()

* refactoring

* use fmt.Sprintf to format contract address in logs

* fix test and lint warnings

* update Gopkg.lock

* update Gopkg.lock once more
2018-08-20 15:55:43 +02:00
Adrià Cidre 9ce96baf5d
[#1093] Mailserver compatible with sync AND async keys (#1096) 2018-07-16 11:07:17 +02:00
Adam Babik 38a60135b2 Add asymmetric key support for MailServer requests (#1075)
* add Asymmetric Key support for MailServer requests

* remove deprecated notice

* fix linter

* refactoring Whisper config related to MailServer

* fix race condition
2018-07-04 11:30:57 +02:00
Adrià Cidre 237aeea7b2
Centralize leveldb.OpenFile allowing Recover corrupted db everywhere (#1072) 2018-07-02 10:42:16 +02:00
Andrea Franz 8aa046e893
update mailserver tests to write to a temporary directory instead of using /tmp (#1066) 2018-07-02 09:44:02 +02:00
Andrea Franz 809db97e54
mailserver pagination (#1039)
* mailserver sends envelopes in descending order

* add limit value in mailserver request payload

* mailserver sends messages up to the limit specified in the request

* update Archive method to return key and error

* processRequest returns the next page cursor

* add cursor to mailserver request

* add limit and cursor to request payload

* fix request limit encoding

* wait for request completed event in TrackerSuite/TestRequestCompleted

* add cursor to mailserver response

* fix cursor position in payload

* add e2e test for mail server pagination

* validate mail server response size

* remove old limitReached var

* fix lint warnings

* add whisper patch

* fix tests after rebase

* check all return values to avoid lint warnings

* check that all messages have been retrieved after 2 paginated requests

* fix lint warnings

* rename geth patch

* merge mailserver patches into one

* add last envelope hash to mailserver response and EventEnvelopeAvailable event

* update whisper patch

* add docs to MailServerResponse

* update whisper patch

* fix tests and lint warnings

* send mailserver response data on EventMailServerRequestCompleted signal

* update tracker tests

* optimise pagination test waiting for mailserver to archive only before requesting

* rollback mailserver interface changes

* refactoring and docs changes

* fix payload size check to determine if a limit is specified

* add more docs to the processRequest method

* add constants for request payload field lengths

* add const noLimits to specify that limit=0 means no limits
2018-07-02 09:38:10 +02:00
Andrea Franz 29d90b651d
test leveldb panics on mailserver (#1065)
* test leveldb panics on mailserver

* refactoring

* rename DB interface to dbImpl and add documentation
2018-06-27 14:22:09 +02:00
Pedro Pombeiro ab7b9b914c Recover from panics which can get sent from goleveldb due to bad arguments. Closes #1059 2018-06-26 16:49:13 +02:00
Ivan Daniluk b003400b4b Validate lower/upper limit in mailserver request (#1058) 2018-06-26 09:33:05 +02:00
Pedro Pombeiro fac5576988 Use `EnvelopeTracer` as an event source to avoid relying on timeouts in tests. Fixes #1021
Fix bug in mailserver implementation which was using upper bound `to` parameter as an exclusive, rather than inclusive parameter, as stated in the documentation: https://github.com/status-im/status-go/wiki/Additional-JSON-RPC-API
2018-06-18 15:24:27 +02:00
Andrea Franz fa390a52ae
RequestHistoricMessages response (#1009)
* refactor TestRequestMessageFromMailboxAsync to use s.requestHistoricMessages helper

* send p2pRequestResponseCode from mailserver

* send p2p message response to after sending all historic messages

* mailserver sends `whisper.NewSentMessage` as response

* add mailserver Client and p2pRequestAckCode watchers

* send event with envelopeFeed when p2pRequestAckCode is received

* test request completed event in tracker

* rename mailserver response events and code to RequestCompleteCode

* wait for mailserver response in e2e test

* use SendHistoricMessageResponse method name for mailserver response

* fix lint warnings

* add mailserver request expiration

* send mailserver response without envelope

* add `ttl` to Request struct in shhext_requestMessages

* test that tracker calls handler.MailServerRequestExpired

* add geth patch

* rename TTL to Timeout

* split tracker.handleEvent in multiple methods
2018-06-15 17:12:31 +02:00
Adam Babik b3a8991eb1
Update mailserver metrics for archived envelopes (#1035) 2018-06-15 11:19:17 +02:00
Pedro Pombeiro 354e23aaf5 Fix issues reported by lint. Part of #1017 2018-06-14 13:52:51 +02:00
Adam Babik bbe7fba0c9
Benchmark tests (#986)
A set of tests used to benchmark Whisper and MailServer nodes.
2018-06-11 12:48:42 +02:00
Adrià Cidre d5be8c525d
[#856] move geth subpackages to root level (#1007) 2018-06-08 13:29:50 +02:00
Andrea Franz 4b938be856
add metrics to mailserver (#999)
* add metrics to mailserver

* sentEnvelopesSize existing sentEnvelopesSize
2018-05-30 14:31:47 +02:00
Adrià Cidre 4e33dfe7bc
Fix for mail server not limiting certain messages (#981) 2018-05-22 17:51:05 +02:00
Adrià Cidre 06e64cdde2
Refactor mailserver tests (#970) 2018-05-21 13:30:37 +02:00
Adrià Cidre e9da21cf87 Rollback mailserver DataDir to previous leveldb data directory 2018-05-20 08:34:47 +03:00
Andrea Franz 4317b8f687
add cmd/statusd-prune (#957)
add mailserver cleaner

use memstorage for leveldb in tests

avoid write if batch size is 0

add comments

add cmd/statusd-prune

rmeove batch size var in prune method

validate range values

pass only flag name to missingFlag

refactor Cleaner.prune method

update batch not to be a pointer

removed extra batch counter increment

don't increment counter if batch returns errors

add README
2018-05-17 19:24:00 +02:00
Adrià Cidre cf8ad7e002
[#ISSUE-947/ISSUE-946] Rate limit and batch limit on any mail server query (#949) 2018-05-17 13:21:04 +02:00
Adrià Cidre 08b4d515c6
[#ISSUE-937] : Move mail server implementation to status-go repo (#942)
* [#ISSUE-937] : Move mail server implementation to status-go repo

* Applying dep-ensure
2018-05-11 21:43:07 +02:00