status-go/mailserver
Andrea Maria Piana aa7f591587 Move networking code for waku under `v0` namespace
Why make the change?

As discussed previously, the way we will move across versions is to maintain completely separate
codebases and eventually remove those that are not supported anymore.

This has the drawback of some code duplication, but the advantage is that is more
explicit what each version requires, and changes in one version will not
impact the other, so we won't pile up backward compatible code.
This is the same strategy used by `whisper` in go ethereum and is influenced by
https://www.youtube.com/watch?v=oyLBGkS5ICk .

All the code that is used for the networking protocol is now under `v0/`.
Some of the common parts might still be refactored out.
The main namespace `waku` deals with `host`->`waku` interactions (through RPC),
while `v0` deals with `waku`->`remote-waku` interactions.

In order to support `v1`, the namespace `v0` will be copied over, and changed to
support `v1`. Once `v0` will be not used anymore, the whole namespace will be removed.

This PR does not actually implement `v1`, I'd rather get things looked over to
make sure the structure is what we would like before implementing the changes.

What has changed?

- Moved all code for the common parts under `waku/common/` namespace
- Moved code used for bloomfilters in `waku/common/bloomfilter.go`
- Removed all version specific code from `waku/common/const` (`ProtocolVersion`, status-codes etc)
- Added interfaces for `WakuHost` and `Peer` under `waku/common/protocol.go`

Things still to do

Some tests in `waku/` are still testing by stubbing components of a particular version (`v0`).
I started moving those tests to instead of stubbing using the actual component, which increases
the testing surface. Some other tests that can't be easily ported should be likely moved under
`v0` instead. Ideally no version specif code should be exported from a version namespace (for
example the various codes, as those might change across versions). But this will be a work-in-progress.

Some code that will be common in `v0`/`v1` could still be extract to avoid duplication, and duplicated only
when implementations diverge across versions.
2020-04-27 14:58:02 +02:00
..
migrations Add statusUpdate code (#1861) 2020-02-21 15:48:53 +01:00
registry Use goimports instead of gofmt 2020-01-06 10:17:23 +01:00
README.md Sync messages in a loop until error or no cursor (#1502) 2019-06-26 18:17:41 +02:00
cleaner.go Add postgres 2019-05-15 11:01:34 +02:00
cleaner_test.go set archived_envelopes_total on start based on DB count 2020-04-09 13:08:14 +02:00
db_key.go Create a home submodule for Eth node bridges- Rename StatusBackend to GethStatusBackend 2019-11-27 17:02:09 +01:00
db_key_test.go Use goimports instead of gofmt 2020-01-06 10:17:23 +01:00
limiter.go mailserver: refactor mailserver's rate limiter (#1341) 2019-01-10 17:07:16 +01:00
limiter_test.go mailserver: refactor mailserver's rate limiter (#1341) 2019-01-10 17:07:16 +01:00
mailserver.go Move networking code for waku under `v0` namespace 2020-04-27 14:58:02 +02:00
mailserver_db.go set archived_envelopes_total on start based on DB count 2020-04-09 13:08:14 +02:00
mailserver_db_leveldb.go set archived_envelopes_total on start based on DB count 2020-04-09 13:08:14 +02:00
mailserver_db_postgres.go set archived_envelopes_total on start based on DB count 2020-04-09 13:08:14 +02:00
mailserver_db_postgres_test.go rename gomarkdown/markdown to status-im/markdown 2020-02-11 16:06:43 +01:00
mailserver_test.go Add support for request messages by topics (#1805) 2020-01-21 08:11:24 +01:00
metrics.go set archived_envelopes_total on start based on DB count 2020-04-09 13:08:14 +02:00
request.go Add support for request messages by topics (#1805) 2020-01-21 08:11:24 +01:00

README.md

MailServer

This document is meant to collect various information about our MailServer implementation.

Syncing between mail servers

It might happen that one mail server is behind other due to various reasons like a machine being down for a few minutes etc.

There is an option to fix such a mail server:

  1. SSH to a machine where this broken mail server runs,
  2. Add a mail server from which you want to sync:
# sudo might be not needed in your setup
$ echo '{"jsonrpc":"2.0","method":"admin_addPeer", "params": ["enode://c42f368a23fa98ee546fd247220759062323249ef657d26d357a777443aec04db1b29a3a22ef3e7c548e18493ddaf51a31b0aed6079bd6ebe5ae838fcfaf3a49@206.189.243.162:30504"], "id":1}' | \
    sudo socat -d -d - UNIX-CONNECT:/docker/statusd-mail/data/geth.ipc
  1. Mark it as a trusted peer:
# sudo might be not needed in your setup
$ echo '{"jsonrpc":"2.0","method":"shh_markTrustedPeer", "params": ["enode://c42f368a23fa98ee546fd247220759062323249ef657d26d357a777443aec04db1b29a3a22ef3e7c548e18493ddaf51a31b0aed6079bd6ebe5ae838fcfaf3a49@206.189.243.162:30504"], "id":1}' | \
    sudo socat -d -d - UNIX-CONNECT:/docker/statusd-mail/data/geth.ipc
  1. Finally, trigger the sync command:
# sudo might be not needed in your setup
$ echo '{"jsonrpc":"2.0","method":"shhext_syncMessages","params":[{"mailServerPeer":"enode://c42f368a23fa98ee546fd247220759062323249ef657d26d357a777443aec04db1b29a3a22ef3e7c548e18493ddaf51a31b0aed6079bd6ebe5ae838fcfaf3a49@206.189.243.162:30504", "to": 1550479953, "from": 1550393583, "limit": 1000}],"id":1}' | \
    sudo socat -d -d - UNIX-CONNECT:/docker/statusd-mail/data/geth.ipc

You can add "followCursor": true if you want it to automatically download messages until the cursor is empty meaning all data was synced.

Debugging

To verify that your mail server received any responses, watch logs and seek for logs like this:

INFO [02-18|09:08:54.257] received sync response count=217 final=false err= cursor=[]

And it should finish with:

INFO [02-18|09:08:54.431] received sync response count=0 final=true err= cursor=[]