status-go/mailserver
Andrea Maria Piana e65760ca85 Add basic peersyncing
This commit adds basic syncing capabilities with peers if they are both
online.

It updates the work done on MVDS, but I decided to create the code in
status-go instead, since it's very tight to the application (similarly
the code that was the inspiration for mvds, bramble, is all tight
together at the database level).

I reused parts of the protobufs.

The flow is:

1) An OFFER message is sent periodically with a bunch of message-ids and
   group-ids.
2) Anyone can REQUEST some of those messages if not present in their
   database.

3) The peer will then send over those messages.

It's disabled by default, but I am planning to add a way to set up the
flags.
2024-01-23 12:46:17 +00:00
..
migrations Add basic peersyncing 2024-01-23 12:46:17 +00:00
README.md Sync messages in a loop until error or no cursor (#1502) 2019-06-26 18:17:41 +02:00
cleaner.go Add postgres 2019-05-15 11:01:34 +02:00
cleaner_test.go [Fixes: #2328] Fixes iterator on leveldb 2021-08-24 15:40:40 +02:00
db_key.go Create a home submodule for Eth node bridges- Rename StatusBackend to GethStatusBackend 2019-11-27 17:02:09 +01:00
db_key_test.go Use goimports instead of gofmt 2020-01-06 10:17:23 +01:00
limiter.go mailserver: refactor mailserver's rate limiter (#1341) 2019-01-10 17:07:16 +01:00
limiter_test.go mailserver: refactor mailserver's rate limiter (#1341) 2019-01-10 17:07:16 +01:00
mailserver.go [Fixes: #2328] Fixes iterator on leveldb 2021-08-24 15:40:40 +02:00
mailserver_db.go [Fixes: #2328] Fixes iterator on leveldb 2021-08-24 15:40:40 +02:00
mailserver_db_leveldb.go mailserver: add DB query metrics 2022-07-26 17:23:54 +02:00
mailserver_db_leveldb_test.go test: use `T.TempDir` to create temporary test directory (#2746) 2023-04-26 21:39:51 +01:00
mailserver_db_postgres.go mailserver: add DB query metrics 2022-07-26 17:23:54 +02:00
mailserver_db_postgres_test.go Turning the tests back on (#2519) 2022-02-03 20:37:41 +00:00
mailserver_test.go Revert "test: bump go-libp2p" 2024-01-18 20:29:33 +00:00
metrics.go mailserver: add DB query metrics 2022-07-26 17:23:54 +02:00
request.go Remove whisper 2021-05-21 07:22:58 +02:00

README.md

MailServer

This document is meant to collect various information about our MailServer implementation.

Syncing between mail servers

It might happen that one mail server is behind other due to various reasons like a machine being down for a few minutes etc.

There is an option to fix such a mail server:

  1. SSH to a machine where this broken mail server runs,
  2. Add a mail server from which you want to sync:
# sudo might be not needed in your setup
$ echo '{"jsonrpc":"2.0","method":"admin_addPeer", "params": ["enode://c42f368a23fa98ee546fd247220759062323249ef657d26d357a777443aec04db1b29a3a22ef3e7c548e18493ddaf51a31b0aed6079bd6ebe5ae838fcfaf3a49@206.189.243.162:30504"], "id":1}' | \
    sudo socat -d -d - UNIX-CONNECT:/docker/statusd-mail/data/geth.ipc
  1. Mark it as a trusted peer:
# sudo might be not needed in your setup
$ echo '{"jsonrpc":"2.0","method":"shh_markTrustedPeer", "params": ["enode://c42f368a23fa98ee546fd247220759062323249ef657d26d357a777443aec04db1b29a3a22ef3e7c548e18493ddaf51a31b0aed6079bd6ebe5ae838fcfaf3a49@206.189.243.162:30504"], "id":1}' | \
    sudo socat -d -d - UNIX-CONNECT:/docker/statusd-mail/data/geth.ipc
  1. Finally, trigger the sync command:
# sudo might be not needed in your setup
$ echo '{"jsonrpc":"2.0","method":"shhext_syncMessages","params":[{"mailServerPeer":"enode://c42f368a23fa98ee546fd247220759062323249ef657d26d357a777443aec04db1b29a3a22ef3e7c548e18493ddaf51a31b0aed6079bd6ebe5ae838fcfaf3a49@206.189.243.162:30504", "to": 1550479953, "from": 1550393583, "limit": 1000}],"id":1}' | \
    sudo socat -d -d - UNIX-CONNECT:/docker/statusd-mail/data/geth.ipc

You can add "followCursor": true if you want it to automatically download messages until the cursor is empty meaning all data was synced.

Debugging

To verify that your mail server received any responses, watch logs and seek for logs like this:

INFO [02-18|09:08:54.257] received sync response count=217 final=false err= cursor=[]

And it should finish with:

INFO [02-18|09:08:54.431] received sync response count=0 final=true err= cursor=[]