Commit Graph

305 Commits

Author SHA1 Message Date
Andrea Maria Piana 9ddd963591
vendor 2020-07-30 20:20:56 +02:00
Samuel Hawksby-Robinson c8f9dad554
Feature/key compression (#1990)
## What has changed?

I've introduced to the public binding functionality that will compress and decompress public keys of a variety of encoding and key types. This functionality supports all major byte encoding formats and the following EC public key types:

- `secp256k1` pks
- `bls12-381 g1` pks
- `bls12-381 g2` pks

## Why make the change?

We want shorter public (chat) keys and we want to be future proof and encoding agnostic. See the issue here https://github.com/status-im/status-go/issues/1937

---

* Added basic signature for compresspk and uncompresspk

* Added basic encoding information

* make vendor

* formatted imports for the linter

* Reformatted imports hoping linter likes it

* This linter is capricious

* Added check that the secp256k1 key is valid

* Added test for valid key

* Added multiformat/go-varint dep

* Added public key type handling

* Added key decompression with key type handling

* Added handling for '0x' type indentifying

* Added more robust testing

* Less lint for the linting gods

* make vendor for bls12_381

* Added bls12-381 compression tests

* Added decompress key expected results

* Refactor of typed and untyped keys in tests

* Lint god appeasment

* Refactor of sample public keys

* Implemented bls12-381 decompression

* gofmt

* Renamed decode/encode funcs to be more descriptive

* Added binary bindings for key de/compression

* Refactor of func parameters

gomobile is a bit tempermental using raw bytes as a parameter, so I've decided to use string only inputs and outputs

* gofmt

* Added function documentation

* Moved multiformat de/compression into api/multiformat ns

* Moved multiformat de/compression into api/multiformat ns

* Changed compress to serialize on API
2020-06-23 11:47:17 +01:00
Adam Babik d7bb02540a
Update go-ethereum to v1.9.5-status.9 (#1898) 2020-03-12 13:05:12 +01:00
Adam Babik 831ab85561
Upgrade deps (#1892) 2020-03-09 08:55:58 +01:00
Andrea Maria Piana a82de9f5f9
Upgrade extkeys (#1878) 2020-03-04 09:30:31 +01:00
Andrea Maria Piana ddf7c2d1c2 upgrade whisper 2020-02-28 13:27:00 +01:00
Andrea Maria Piana 2ae04f2a2d acknowledge spelling 2020-02-28 13:27:00 +01:00
Andrea Maria Piana 7a64fbf5f7
make apis private (#1866)
* make apis private

* Bump whisper
2020-02-25 08:03:01 +01:00
Adam Babik 91e658aeb6
bump Whisper to 6.2.1 (#1869) 2020-02-21 16:22:25 +01:00
Andrea Maria Piana 849492fda9
Add statusUpdate code (#1861)
* Add status-option code

This commits changes the behavior of waku introducing a new status-code,
`2`, that replaces the current single options codes.

* linting
2020-02-21 15:48:53 +01:00
Adam Babik 76b5dc29dc
Integrate Whisper-Waku bridge in status-go (#1854) 2020-02-18 12:21:01 +01:00
Jakub Sokołowski cca47ef6e1 rename gomarkdown/markdown to status-im/markdown
This resolves a dependency conflict we have with MatterBridge
which was using a newer version of the same package.
This resulted in a JSON marshalling bug that would crash the bridge.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2020-02-11 16:06:43 +01:00
Adam Babik 8b61d92d58
Remove protocol and eth-node submodules (#1835) 2020-02-10 12:22:37 +01:00
Andrea Maria Piana 0b6ad662b5
Bug/validate pubkey (#1838)
* validate chat before persisting

* add comment to public key generation
2020-02-07 12:56:30 +01:00
Adam Babik d27a507e0d
make Messenger.Timesource public (#1837) 2020-02-07 12:30:26 +01:00
Andrea Maria Piana f2eebd1e49
Verify ENS in the background (#1824)
Currently ENS are verified explicitly by status-react, this is not ideal
as if that fails it will have to be explicilty retried in status-react.
This commits changes that behavior so that ENS are verified in a loop
and updated if new messages are received.
2020-02-05 11:09:33 +01:00
Andrea Maria Piana 8931b14c4e
Explicitly init messenger (#1821)
Messenger needs to wait until all the topics/mailservers are loaded in
the client, so we explicitly add a way to call Init.
2020-01-29 20:40:06 +01:00
Pedro Pombeiro 741f43cb0c Fix wrong field being logged in `EnvelopesMonitor` 2020-01-23 17:19:10 +01:00
Andrea Maria Piana a4f88d0017
Fix ens names and contact (#1810)
Currently the wrong pk format was used to query the in-memory contacts,
this commit corrects the behavior.
2020-01-23 09:24:28 +01:00
Adam Babik bc2d018483
Add support for request messages by topics (#1805) 2020-01-21 08:11:24 +01:00
Adam Babik 79b8112f89
Split shhext into shhext and wakuext (#1803) 2020-01-20 21:56:06 +01:00
Andrea Maria Piana 456bcfa022
Peg clock value to whisper timestamp (#1804)
This commit pegs the clock value to maximum + 120 seconds from the whisper
timestamp.
In this way the we avoid the scenario where a client makes the timestamp
increase arbitrarely.
2020-01-20 17:44:32 +01:00
Pedro Pombeiro d4710faae2 In progress: Use Nimbus keystore 2020-01-20 13:15:17 +01:00
Pedro Pombeiro 6537cae606 Nimbus node support 2020-01-20 13:15:17 +01:00
Andrea Maria Piana 25d46c6d82
Fix waku tests & contact ens (#1802) 2020-01-17 13:39:09 +01:00
Pedro Pombeiro 7f45c43196 make lint-fix 2020-01-16 13:18:36 +01:00
Andrea Maria Piana c569d8a4ed
Sync installation messages & contact requests (#1791) 2020-01-15 08:25:09 +01:00
Adam Babik 44aa313981
Make shhext and protocol work with Waku (#1777)
This change makes shhext and protocol submodule work with Waku and Whisper.
2020-01-13 20:17:30 +01:00
Jakub Sokołowski 330d177de6 specify gomarkdown/markdown version to fix conflicts
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2020-01-11 19:41:24 +01:00
Andrea Maria Piana 88a1d0111e
Add Commands (#1731)
This commit adds handling of Transaction commands.
2020-01-10 19:59:01 +01:00
yenda d3de88f790 fix stickers persistence (#1779) 2020-01-10 16:12:09 +01:00
Adam Babik 37a508a97b
Integrate Waku service (#1759) 2020-01-08 12:12:23 +01:00
Pedro Pombeiro c8a911ebd1 Use goimports instead of gofmt 2020-01-06 10:17:23 +01:00
Pedro Pombeiro 3eaacaa2f4 Move keystore_geth.go to eth-node/bridge/geth 2020-01-06 10:17:23 +01:00
Pedro Pombeiro 287e5cdf79 Abstract `accounts.Key` and geth `keystore` 2020-01-06 10:17:23 +01:00
Pedro Pombeiro 41cba814c8 Run `make vendor` 2019-12-30 13:25:12 +01:00
Pedro Pombeiro f494576ca2 Run `gofmt -s -w` 2019-12-20 11:24:26 +01:00
Adam Babik ea0f12b920
Waku/0 service (#1715) 2019-12-20 10:40:50 +01:00
Pedro Pombeiro 8d8880dc62 Get rid of `types.Whisper.Poll` method 2019-12-19 14:11:48 +01:00
Pedro Pombeiro ef87c330ce Improve parameter naming in `Whisper.DeleteKeyPair` 2019-12-19 14:11:48 +01:00
Pedro Pombeiro 0cf24d2c3e Remove unnecessary call to `GetFilter` 2019-12-19 14:11:48 +01:00
Pedro Pombeiro a5cf17cf4b Improve logging format in `messageProcessor.handleMessages` 2019-12-19 14:11:48 +01:00
Pedro Pombeiro 7f11030896 Implement AddPeer/RemovePeer on eth-node 2019-12-19 14:11:48 +01:00
Pedro Pombeiro 41a6502340 Remove `SelectKeyPair` from `eth-nodes/types` 2019-12-19 14:11:48 +01:00
Adam Babik 4f45bceb9a
Drop peers when rate limit is exceeded (#1732) 2019-12-16 20:38:50 +01:00
acolytec3 2d2f2fbc51 Add ValidateMnemonic method (#1713) 2019-12-16 14:55:02 +01:00
Andrea Maria Piana baa0767c26
Handle membership update message
This commit does a few things:

1) Handle membership updates using protobuf and adds the relevant
endpoints.
2) Store in memory a map of chats + contacts for faster lookups, which
are then flushed to disk on each update
3) Validate incoming messages

Sorry for the large pr, but you know, v1 :)
2019-12-10 15:20:28 +01:00
Adam Babik 0c6a4e0e65
fix Whisper submodule version; bump go-ethereum version 2019-12-09 16:26:00 +01:00
Adam Babik 6284d45b69
fix docker image build 2019-12-09 16:04:32 +01:00
Adam Babik a636f33109
Set Whisper version submodule to v6 (#1725) 2019-12-09 11:36:14 +01:00
Adam Babik 4ac4a61e20
Move whisper to status-go monorepo (#1720) 2019-12-09 11:06:04 +01:00
Andrea Maria Piana fd49b0140e
Move to protobuf for Message type (#1706)
* Use a single Message type `v1/message.go` and `message.go` are the same now, and they embed `protobuf.ChatMessage`

* Use `SendChatMessage` for sending chat messages, this is basically the old `Send` but a bit more flexible so we can send different message types (stickers,commands), and not just text.

* Remove dedup from services/shhext. Because now we process in status-protocol, dedup makes less sense, as those messages are going to be processed anyway, so removing for now, we can re-evaluate if bringing it to status-go or not.

* Change the various retrieveX method to a single one:
`RetrieveAll` will be processing those messages that it can process (Currently only `Message`), and return the rest in `RawMessages` (still transit). The format for the response is:
`Chats`: -> The chats updated by receiving the message
`Messages`: -> The messages retrieved (already matched to a chat)
`Contacts`: -> The contacts updated by the messages
`RawMessages` -> Anything else that can't be parsed, eventually as we move everything to status-protocol-go this will go away.
2019-12-05 17:25:34 +01:00
Jakub Sokołowski f4d4c66d5b upgrade whisper: 1.6.1 > 1.6.2
This adds new Prometheus metrics

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2019-12-02 13:17:52 +01:00
Pedro Pombeiro 26b6d7c36a Create a home submodule for Eth node bridges- Rename StatusBackend to GethStatusBackend 2019-11-27 17:02:09 +01:00
Adam Babik 47ccf32cdb
Upgrade zap to v1.13.0 (#1694) 2019-11-25 21:16:00 +01:00
Adam Babik 10635d555f
Make extkeys as a submodule (#1691) 2019-11-22 13:58:03 +01:00
Adam Babik e1a8ba1ba5
Fix modvendor (#1690) 2019-11-22 13:24:20 +01:00
Pedro Pombeiro ed5a5c154d
Move to monorepo structure (#1684)
Move to a monorepo structure with submodules

- Rename status-protocol-go to status-go/protocol
2019-11-21 17:19:22 +01:00
Pedro Pombeiro 2dd74da23d Integrate Nimbus status-protocol-go 2019-11-18 14:30:03 +01:00
Adam Babik c199c8f342
Update Whisper and introduce rate limiting (#1673) 2019-11-18 11:22:23 +01:00
Pedro Pombeiro 6ce437255e Remove FCM implementation 2019-11-17 17:54:55 +01:00
Jakub Sokołowski 4c313c7032 add tcp-pinger for measuring rtt of mailservers
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2019-11-15 13:18:08 +01:00
Andrea Maria Piana 9d7c570593
Add PrepareContent and upgrade status-go (#1674) 2019-11-15 09:52:28 +01:00
Andrea Maria Piana 89659f85b4
Upgrade status-protocol-go (#1664) 2019-11-08 08:36:12 +01:00
Andrea Maria Piana a659685e3f
Add parsed message (#1660)
This commits add a field (parsedMessage) to the json payload sent to
status-react.
This field is the parsed version of the transit message.
The code is all in dedup, I will re-organize it once we made all the
necesseary changes.
2019-11-06 17:23:11 +01:00
Adam Babik 183d626256
Implement mailserver Deliver() with MessagesRequest (#1653)
The main difference is that it takes `whisper.MessagesRequest` as an argument instead of `whisper.Envelope`.
2019-11-04 21:15:21 +01:00
Jakub Sokołowski c1f17cb4a3 use prometheus/client_golang for metrics
also re-implement mailserver metrics

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2019-11-04 13:18:50 +01:00
Andrea Maria Piana 9f2a4b378f
Verify ens endpoint, only against mainnet (#1657)
* String payload

* Verify ENS Names

* Update status-protocol-go
2019-11-04 11:08:22 +01:00
Adam Babik a244b01a26
Upgrade migrate (#1643) 2019-10-14 16:10:48 +02:00
Pedro Pombeiro c874960215
Use status-protocol-go/bridge/geth (#1638)
* Use status-protocol-go/bridge/geth
2019-10-14 09:53:38 +02:00
Pedro Pombeiro d87caf57e3
Add dependabot config file (#1636)
* Add dependabot config file

* Fix status-protocol-go dependency
2019-10-11 07:00:46 +02:00
Jakub Sokołowski 294399916e add script for easier setup of mailserver
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2019-10-10 11:27:32 +02:00
Pedro Pombeiro a2f106e4c5
Update for status-protocol-go API changes (#1627) 2019-10-09 16:22:53 +02:00
Adam Babik 26880b83d7
Upgrade geth to 1.9.5 and Whisper (#1617) 2019-10-04 17:21:24 +02:00
Andrea Maria Piana 40e66e6186
upgrade status-protocol-go (#1619) 2019-09-26 11:26:33 +02:00
Andrea Maria Piana 9a3ed980c9
Generate gfycat and identicon in status-protocol-go (#1611) 2019-09-26 09:01:17 +02:00
Andrea Maria Piana 1a47893e75
Upgrade status-protocol-go (#1593) 2019-09-02 11:29:06 +02:00
Andrea Maria Piana b8ea79a3f0
Upgrade status-protocol-go (#1586) 2019-08-29 08:33:46 +02:00
Andrea Maria Piana e118e89107
Add persistent datasync, use single database and handle negotiated filters (#1580) 2019-08-27 14:04:15 +02:00
Andrea Maria Piana 06dc227071
Add Metadata to messages, expose new messenger methods (#1571)
* Add Metadata to messages, expose new messenger methods

This commits modifies deduplicator so that it takes a `StatusMessage`
instead of `WhisperMessage` and also returns a `Metadata` field which is
then passed back by the client when confirming messages, which fixes the
issue we had with not confirming pfs messages.
2019-08-20 13:20:25 +02:00
Adam Babik 8383feea04 add API to shhext to manage messages 2019-08-12 19:44:31 +02:00
Andrea Maria Piana 9ae7d2b6d5 Move envelopes tracking to status-protocol-go
This commit moves envelopes tracking to status-go.
Post endpoint is not going to track envelopes anymore, as that's taken
care on status-protocol-go side, so this is a breaking change, and
version is updated accordingly.
2019-08-12 19:44:31 +02:00
Andrea Maria Piana a3a413b5d9 Add SaveChat, Chats, DeleteChat, SaveConctat & Contacts endpoint 2019-08-05 16:03:18 +02:00
Adam Babik ce7b0fc376 conslidate protocol databases 2019-08-01 10:40:54 +02:00
Andrea Maria Piana 9de77b21b2 Add datasync, v1messages & disable discovery topic options
Adds support for datasync, V1Messages and disabling the discovery topic.
This is a backward compatible change as long as they are not toggled on
(they are not by default).
2019-07-29 19:39:44 +02:00
Andrea Franz dcb0fa5262
add multi-account support (#1500)
* add multiaccount support

add multi account ImportPrivateKey and StoreAccount

test derivation from normal keys

* add multiaccount to mobile pkg

* use multiaccount params structs from the mobile pkg

* move multiaccount tests together with the other lib tests

* fix codeclimate warning and temporarily increase methods threshold

* split library_test_utils.go to avoid linter warnings
2019-07-24 20:59:15 +02:00
Adam Babik e93d994460 integrate status-protocol-go 2019-07-24 18:46:43 +02:00
Adam Babik 664488faf9
add an adapter for zap and geth loggers (#1534) 2019-07-24 09:58:01 +02:00
Adam Babik 1ab2e88bf5
Sync messages in a loop until error or no cursor (#1502) 2019-06-26 18:17:41 +02:00
Andrea Maria Piana f6fba1d3d6 Publish contact code periodically 2019-06-19 17:38:45 +02:00
Andrea Maria Piana 1aa3e2812a Add support for partitioned topic 2019-06-19 17:38:45 +02:00
Adam Babik c9e99c432d migrate to go 1.12 and go modules 2019-06-12 13:12:00 +02:00
Andrea Maria Piana 4ab08629f6 Add postgres
This commits adds support for postgres database.
Currently two fields are stored: the bloom filter and the topic.
Only the bloom filter is actually used to query, but potentially we will
use also the topic in the future, so easier to separate it now in order
to avoid a migration.
2019-05-15 11:01:34 +02:00
Andrea Maria Piana 061f10e58d
Publish rlp.RawValue instead of envelope (#1459)
As part of a performance profiling of mailserver we noticed that most of
the resources on a query are spend decoding the whisper envelope.
This PR changes the way we store envelopes encoding the Topic into the
database key, so we can check that and we are able to publish the
envelope rawValue if it matches.
The change is backward compatible as only newly added envelopes will
have the new key, while old ones will have to be unmarshaled.
2019-05-09 12:58:02 +02:00
Andrea Maria Piana 354e6981ba
Add network incentivisation service (#1452) 2019-04-29 14:05:49 +02:00
Dmitry Shulyak 866342d5e3
Update whisper to v1.4.12 (#1447) 2019-04-23 20:16:48 +03:00
Dmitry Shulyak a904d9325e
Notify users that envelope was discarded and retry sending it (#1424)
* Notify users that envelope was discarded and retry sending it

* Update Gopkg files with released whisper version

* Forgot to remove signal after refactoring
2019-04-02 13:40:45 +03:00
Dmitry af403280a6 Upgrade geth to 1.8.23 (includes necessary whisper update) 2019-03-14 14:39:31 +02:00
Dmitry b2f02d0dbe Use go-ethereum with fixed logger 2019-03-07 16:26:20 +02:00
Dmitry Shulyak cf8f20b23c Use lumberjack library to rotate logs and add config that enables it (#1399) 2019-03-01 14:37:13 +01:00