Commit Graph

142 Commits

Author SHA1 Message Date
Andrea Maria Piana 8baae97e2e Add GetNodesFromContract endpoint
We added an endpoint that given an RPC url and a contract will fetch a
list of nodes and return them to the client.
2019-03-29 12:55:54 +01:00
Andrea Maria Piana b574e3e53f
Default limit & upper bound (#1411)
We allow the user not to specify an upper bound, in which case it will
default to Time.Now() + 60 seconds, to accomodate for messages with
higher timestamp.

Limit is also defaulted to 2000 if not set.

We removed as well the 24 hours limitation as now all the requests will
come with a limit.
2019-03-12 12:14:35 +01:00
Adam Babik 0c76505b95
expose more logs and metrics in DeliverMail (#1398) 2019-02-27 15:30:08 +01:00
Adam Babik 38aafe01db
Fix deadlock in mailserver DeliverMail method (#1388) 2019-02-22 09:55:37 +01:00
Adam Babik e38833b17a
add README for mailserver syncing (#1377) 2019-02-19 19:09:55 +01:00
Andrea Maria Piana da3964ba31
Change publishing envelopes behaviour (#1371) 2019-02-08 16:39:24 +01:00
Adam Babik d20b5dc3b3
mailserver: build pruning into WMailServer (#1342)
MailServer pruning was implemented as a separate command but that required stopping a mail server and executing the command manually. This change builds pruning into MailServer and can be set using MailServerDataRetention in WhisperConfig.
2019-01-23 14:32:45 +01:00
Adam Babik 8f2e347e4f
mailserver: refactor mailserver's rate limiter (#1341) 2019-01-10 17:07:16 +01:00
Adam Babik 74cb16c456
add SyncMessages method to shhext api (#1309) 2018-12-14 12:21:34 +01:00
Adam Babik e2682486fd
Fix cursor encoding (#1308)
It fixes encoding and decoding cursor sent in historic messages requests. `hex` package from the standard library is used.
2018-12-11 11:23:47 +01:00
Adam Babik 913dbfca9b
Support Mail Server data synchronization (#1302)
These changes add a support for syncing data between two Mail Servers. It does not give an easy way to start syncing. This will be solved in the next PR.
2018-12-06 10:48:28 +01:00
Adam Babik 860da591be
add mailserver batch requests counter and network processing time (#1280) 2018-11-19 10:14:03 +01:00
Adam Babik 52a1bdfed6
Upgrade geth 1.8.17 plus add metrics during compilation time (#1273)
This commit updates geth to 1.8.17 and adds a possibility to enable metrics during compilation time.

The cascade of issues forced us to upgrade geth to 1.8.17 in order to allow enabling metrics during compilation time. 1.8.17 introduced `NodeID` refactoring and `enode` package which affected our peers pool and integration with Discovery V5.
2018-11-14 08:03:58 +01:00
Adam Babik 0961e1039e
fix peerID logging in mail server (#1260) 2018-10-28 23:33:58 +01:00
Adam Babik a75f9c34cf
Request historical messages in batches (#1245) 2018-10-19 11:09:13 +02:00
Igor Mandrigin ba504e99c4
Mailserver: return error response. (#1244) 2018-10-18 12:25:00 +02:00
Andrea Franz b9baa48d87 Add metrics for mailserver errors (#1242) 2018-10-16 17:24:37 +02:00
Dmitry 5b551c67fa Use status-im whisper/whisperv6 module everywhere 2018-10-03 09:27:51 +03:00
Pedro Pombeiro 3d00af7fa3
Streamline configuration in status-go. Part of #1180 (#1183)
- Replace command line flags with `-c` config flag. Part of #1180
- Convert node config private keys to hex-encoded string versions.
- Remove `GenerateConfig` from library.
- Remove unused `FirebaseConfig` from library.
- Fix loading of `config/status-chain-genesis.json` in non-dev machines.
2018-09-13 18:31:29 +02:00
Dmitry Shulyak f150d678de
Update geth to 1.8.14 (#1171)
* Update to geth v1.8.14

* Remove patches that were merged upstream

* Apply patches before 0016

* Fix 0016 and apply it

* Apply everything else

* Pass gas limit as a second argument to simulated backend
2018-08-27 11:22:21 +03:00
Andrea Franz 874a3e8151
Feature/mailserver registry smart contract (#1135)
* add verifier and test using simulated backend

* add ContractCaller

* commit simulated backend after deploy and after smart contract writes

* use bind.NewKeyedTransactor for all transactions in tests

* rename RegistryVerifier to Verifier

* initialize contract verifier if MailServerRegistryAddress config is set

* use contractAddress.Hash()

* refactoring

* use fmt.Sprintf to format contract address in logs

* fix test and lint warnings

* update Gopkg.lock

* update Gopkg.lock once more
2018-08-20 15:55:43 +02:00
Adrià Cidre 9ce96baf5d
[#1093] Mailserver compatible with sync AND async keys (#1096) 2018-07-16 11:07:17 +02:00
Adam Babik 38a60135b2 Add asymmetric key support for MailServer requests (#1075)
* add Asymmetric Key support for MailServer requests

* remove deprecated notice

* fix linter

* refactoring Whisper config related to MailServer

* fix race condition
2018-07-04 11:30:57 +02:00
Adrià Cidre 237aeea7b2
Centralize leveldb.OpenFile allowing Recover corrupted db everywhere (#1072) 2018-07-02 10:42:16 +02:00
Andrea Franz 8aa046e893
update mailserver tests to write to a temporary directory instead of using /tmp (#1066) 2018-07-02 09:44:02 +02:00
Andrea Franz 809db97e54
mailserver pagination (#1039)
* mailserver sends envelopes in descending order

* add limit value in mailserver request payload

* mailserver sends messages up to the limit specified in the request

* update Archive method to return key and error

* processRequest returns the next page cursor

* add cursor to mailserver request

* add limit and cursor to request payload

* fix request limit encoding

* wait for request completed event in TrackerSuite/TestRequestCompleted

* add cursor to mailserver response

* fix cursor position in payload

* add e2e test for mail server pagination

* validate mail server response size

* remove old limitReached var

* fix lint warnings

* add whisper patch

* fix tests after rebase

* check all return values to avoid lint warnings

* check that all messages have been retrieved after 2 paginated requests

* fix lint warnings

* rename geth patch

* merge mailserver patches into one

* add last envelope hash to mailserver response and EventEnvelopeAvailable event

* update whisper patch

* add docs to MailServerResponse

* update whisper patch

* fix tests and lint warnings

* send mailserver response data on EventMailServerRequestCompleted signal

* update tracker tests

* optimise pagination test waiting for mailserver to archive only before requesting

* rollback mailserver interface changes

* refactoring and docs changes

* fix payload size check to determine if a limit is specified

* add more docs to the processRequest method

* add constants for request payload field lengths

* add const noLimits to specify that limit=0 means no limits
2018-07-02 09:38:10 +02:00
Andrea Franz 29d90b651d
test leveldb panics on mailserver (#1065)
* test leveldb panics on mailserver

* refactoring

* rename DB interface to dbImpl and add documentation
2018-06-27 14:22:09 +02:00
Pedro Pombeiro ab7b9b914c Recover from panics which can get sent from goleveldb due to bad arguments. Closes #1059 2018-06-26 16:49:13 +02:00
Ivan Daniluk b003400b4b Validate lower/upper limit in mailserver request (#1058) 2018-06-26 09:33:05 +02:00
Pedro Pombeiro fac5576988 Use `EnvelopeTracer` as an event source to avoid relying on timeouts in tests. Fixes #1021
Fix bug in mailserver implementation which was using upper bound `to` parameter as an exclusive, rather than inclusive parameter, as stated in the documentation: https://github.com/status-im/status-go/wiki/Additional-JSON-RPC-API
2018-06-18 15:24:27 +02:00
Andrea Franz fa390a52ae
RequestHistoricMessages response (#1009)
* refactor TestRequestMessageFromMailboxAsync to use s.requestHistoricMessages helper

* send p2pRequestResponseCode from mailserver

* send p2p message response to after sending all historic messages

* mailserver sends `whisper.NewSentMessage` as response

* add mailserver Client and p2pRequestAckCode watchers

* send event with envelopeFeed when p2pRequestAckCode is received

* test request completed event in tracker

* rename mailserver response events and code to RequestCompleteCode

* wait for mailserver response in e2e test

* use SendHistoricMessageResponse method name for mailserver response

* fix lint warnings

* add mailserver request expiration

* send mailserver response without envelope

* add `ttl` to Request struct in shhext_requestMessages

* test that tracker calls handler.MailServerRequestExpired

* add geth patch

* rename TTL to Timeout

* split tracker.handleEvent in multiple methods
2018-06-15 17:12:31 +02:00
Adam Babik b3a8991eb1
Update mailserver metrics for archived envelopes (#1035) 2018-06-15 11:19:17 +02:00
Pedro Pombeiro 354e23aaf5 Fix issues reported by lint. Part of #1017 2018-06-14 13:52:51 +02:00
Adam Babik bbe7fba0c9
Benchmark tests (#986)
A set of tests used to benchmark Whisper and MailServer nodes.
2018-06-11 12:48:42 +02:00
Adrià Cidre d5be8c525d
[#856] move geth subpackages to root level (#1007) 2018-06-08 13:29:50 +02:00
Andrea Franz 4b938be856
add metrics to mailserver (#999)
* add metrics to mailserver

* sentEnvelopesSize existing sentEnvelopesSize
2018-05-30 14:31:47 +02:00
Adrià Cidre 4e33dfe7bc
Fix for mail server not limiting certain messages (#981) 2018-05-22 17:51:05 +02:00
Adrià Cidre 06e64cdde2
Refactor mailserver tests (#970) 2018-05-21 13:30:37 +02:00
Adrià Cidre e9da21cf87 Rollback mailserver DataDir to previous leveldb data directory 2018-05-20 08:34:47 +03:00
Andrea Franz 4317b8f687
add cmd/statusd-prune (#957)
add mailserver cleaner

use memstorage for leveldb in tests

avoid write if batch size is 0

add comments

add cmd/statusd-prune

rmeove batch size var in prune method

validate range values

pass only flag name to missingFlag

refactor Cleaner.prune method

update batch not to be a pointer

removed extra batch counter increment

don't increment counter if batch returns errors

add README
2018-05-17 19:24:00 +02:00
Adrià Cidre cf8ad7e002
[#ISSUE-947/ISSUE-946] Rate limit and batch limit on any mail server query (#949) 2018-05-17 13:21:04 +02:00
Adrià Cidre 08b4d515c6
[#ISSUE-937] : Move mail server implementation to status-go repo (#942)
* [#ISSUE-937] : Move mail server implementation to status-go repo

* Applying dep-ensure
2018-05-11 21:43:07 +02:00