Commit Graph

217 Commits

Author SHA1 Message Date
Roman Volosovskyi a36144983a
Update markdown module 2020-10-19 13:48:03 +03:00
Andrea Maria Piana 9ddd963591
vendor 2020-07-30 20:20:56 +02:00
Samuel Hawksby-Robinson de79f2ced0
Fix bug, logger var name collided with package name 2020-07-30 20:20:35 +02:00
Samuel Hawksby-Robinson c8f9dad554
Feature/key compression (#1990)
## What has changed?

I've introduced to the public binding functionality that will compress and decompress public keys of a variety of encoding and key types. This functionality supports all major byte encoding formats and the following EC public key types:

- `secp256k1` pks
- `bls12-381 g1` pks
- `bls12-381 g2` pks

## Why make the change?

We want shorter public (chat) keys and we want to be future proof and encoding agnostic. See the issue here https://github.com/status-im/status-go/issues/1937

---

* Added basic signature for compresspk and uncompresspk

* Added basic encoding information

* make vendor

* formatted imports for the linter

* Reformatted imports hoping linter likes it

* This linter is capricious

* Added check that the secp256k1 key is valid

* Added test for valid key

* Added multiformat/go-varint dep

* Added public key type handling

* Added key decompression with key type handling

* Added handling for '0x' type indentifying

* Added more robust testing

* Less lint for the linting gods

* make vendor for bls12_381

* Added bls12-381 compression tests

* Added decompress key expected results

* Refactor of typed and untyped keys in tests

* Lint god appeasment

* Refactor of sample public keys

* Implemented bls12-381 decompression

* gofmt

* Renamed decode/encode funcs to be more descriptive

* Added binary bindings for key de/compression

* Refactor of func parameters

gomobile is a bit tempermental using raw bytes as a parameter, so I've decided to use string only inputs and outputs

* gofmt

* Added function documentation

* Moved multiformat de/compression into api/multiformat ns

* Moved multiformat de/compression into api/multiformat ns

* Changed compress to serialize on API
2020-06-23 11:47:17 +01:00
Andrea Maria Piana aa7f591587 Move networking code for waku under `v0` namespace
Why make the change?

As discussed previously, the way we will move across versions is to maintain completely separate
codebases and eventually remove those that are not supported anymore.

This has the drawback of some code duplication, but the advantage is that is more
explicit what each version requires, and changes in one version will not
impact the other, so we won't pile up backward compatible code.
This is the same strategy used by `whisper` in go ethereum and is influenced by
https://www.youtube.com/watch?v=oyLBGkS5ICk .

All the code that is used for the networking protocol is now under `v0/`.
Some of the common parts might still be refactored out.
The main namespace `waku` deals with `host`->`waku` interactions (through RPC),
while `v0` deals with `waku`->`remote-waku` interactions.

In order to support `v1`, the namespace `v0` will be copied over, and changed to
support `v1`. Once `v0` will be not used anymore, the whole namespace will be removed.

This PR does not actually implement `v1`, I'd rather get things looked over to
make sure the structure is what we would like before implementing the changes.

What has changed?

- Moved all code for the common parts under `waku/common/` namespace
- Moved code used for bloomfilters in `waku/common/bloomfilter.go`
- Removed all version specific code from `waku/common/const` (`ProtocolVersion`, status-codes etc)
- Added interfaces for `WakuHost` and `Peer` under `waku/common/protocol.go`

Things still to do

Some tests in `waku/` are still testing by stubbing components of a particular version (`v0`).
I started moving those tests to instead of stubbing using the actual component, which increases
the testing surface. Some other tests that can't be easily ported should be likely moved under
`v0` instead. Ideally no version specif code should be exported from a version namespace (for
example the various codes, as those might change across versions). But this will be a work-in-progress.

Some code that will be common in `v0`/`v1` could still be extract to avoid duplication, and duplicated only
when implementations diverge across versions.
2020-04-27 14:58:02 +02:00
Adam Babik d7bb02540a
Update go-ethereum to v1.9.5-status.9 (#1898) 2020-03-12 13:05:12 +01:00
Adam Babik 831ab85561
Upgrade deps (#1892) 2020-03-09 08:55:58 +01:00
Andrea Maria Piana a82de9f5f9
Upgrade extkeys (#1878) 2020-03-04 09:30:31 +01:00
Andrea Maria Piana ddf7c2d1c2 upgrade whisper 2020-02-28 13:27:00 +01:00
Andrea Maria Piana 2ae04f2a2d acknowledge spelling 2020-02-28 13:27:00 +01:00
Andrea Maria Piana 7a64fbf5f7
make apis private (#1866)
* make apis private

* Bump whisper
2020-02-25 08:03:01 +01:00
Adam Babik 91e658aeb6
bump Whisper to 6.2.1 (#1869) 2020-02-21 16:22:25 +01:00
Andrea Maria Piana 849492fda9
Add statusUpdate code (#1861)
* Add status-option code

This commits changes the behavior of waku introducing a new status-code,
`2`, that replaces the current single options codes.

* linting
2020-02-21 15:48:53 +01:00
Adam Babik 76b5dc29dc
Integrate Whisper-Waku bridge in status-go (#1854) 2020-02-18 12:21:01 +01:00
Jakub Sokołowski cca47ef6e1 rename gomarkdown/markdown to status-im/markdown
This resolves a dependency conflict we have with MatterBridge
which was using a newer version of the same package.
This resulted in a JSON marshalling bug that would crash the bridge.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2020-02-11 16:06:43 +01:00
Adam Babik 8b61d92d58
Remove protocol and eth-node submodules (#1835) 2020-02-10 12:22:37 +01:00
Pedro Pombeiro 6537cae606 Nimbus node support 2020-01-20 13:15:17 +01:00
Andrea Maria Piana c569d8a4ed
Sync installation messages & contact requests (#1791) 2020-01-15 08:25:09 +01:00
Andrea Maria Piana 88a1d0111e
Add Commands (#1731)
This commit adds handling of Transaction commands.
2020-01-10 19:59:01 +01:00
yenda d3de88f790 fix stickers persistence (#1779) 2020-01-10 16:12:09 +01:00
Adam Babik 26a64e57da
bump extkeys and waku submodules 2020-01-10 14:03:17 +01:00
Adam Babik 37a508a97b
Integrate Waku service (#1759) 2020-01-08 12:12:23 +01:00
Pedro Pombeiro 8d8880dc62 Get rid of `types.Whisper.Poll` method 2019-12-19 14:11:48 +01:00
Adam Babik 4f45bceb9a
Drop peers when rate limit is exceeded (#1732) 2019-12-16 20:38:50 +01:00
acolytec3 2d2f2fbc51 Add ValidateMnemonic method (#1713) 2019-12-16 14:55:02 +01:00
Andrea Maria Piana baa0767c26
Handle membership update message
This commit does a few things:

1) Handle membership updates using protobuf and adds the relevant
endpoints.
2) Store in memory a map of chats + contacts for faster lookups, which
are then flushed to disk on each update
3) Validate incoming messages

Sorry for the large pr, but you know, v1 :)
2019-12-10 15:20:28 +01:00
Adam Babik 0c6a4e0e65
fix Whisper submodule version; bump go-ethereum version 2019-12-09 16:26:00 +01:00
Adam Babik 6284d45b69
fix docker image build 2019-12-09 16:04:32 +01:00
Adam Babik 4ac4a61e20
Move whisper to status-go monorepo (#1720) 2019-12-09 11:06:04 +01:00
Andrea Maria Piana fd49b0140e
Move to protobuf for Message type (#1706)
* Use a single Message type `v1/message.go` and `message.go` are the same now, and they embed `protobuf.ChatMessage`

* Use `SendChatMessage` for sending chat messages, this is basically the old `Send` but a bit more flexible so we can send different message types (stickers,commands), and not just text.

* Remove dedup from services/shhext. Because now we process in status-protocol, dedup makes less sense, as those messages are going to be processed anyway, so removing for now, we can re-evaluate if bringing it to status-go or not.

* Change the various retrieveX method to a single one:
`RetrieveAll` will be processing those messages that it can process (Currently only `Message`), and return the rest in `RawMessages` (still transit). The format for the response is:
`Chats`: -> The chats updated by receiving the message
`Messages`: -> The messages retrieved (already matched to a chat)
`Contacts`: -> The contacts updated by the messages
`RawMessages` -> Anything else that can't be parsed, eventually as we move everything to status-protocol-go this will go away.
2019-12-05 17:25:34 +01:00
Jakub Sokołowski f4d4c66d5b upgrade whisper: 1.6.1 > 1.6.2
This adds new Prometheus metrics

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2019-12-02 13:17:52 +01:00
Pedro Pombeiro 26b6d7c36a Create a home submodule for Eth node bridges- Rename StatusBackend to GethStatusBackend 2019-11-27 17:02:09 +01:00
Adam Babik 47ccf32cdb
Upgrade zap to v1.13.0 (#1694) 2019-11-25 21:16:00 +01:00
Adam Babik 10635d555f
Make extkeys as a submodule (#1691) 2019-11-22 13:58:03 +01:00
Pedro Pombeiro ed5a5c154d
Move to monorepo structure (#1684)
Move to a monorepo structure with submodules

- Rename status-protocol-go to status-go/protocol
2019-11-21 17:19:22 +01:00
Pedro Pombeiro 2dd74da23d Integrate Nimbus status-protocol-go 2019-11-18 14:30:03 +01:00
Adam Babik c199c8f342
Update Whisper and introduce rate limiting (#1673) 2019-11-18 11:22:23 +01:00
Pedro Pombeiro 6ce437255e Remove FCM implementation 2019-11-17 17:54:55 +01:00
Jakub Sokołowski 4c313c7032 add tcp-pinger for measuring rtt of mailservers
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2019-11-15 13:18:08 +01:00
Andrea Maria Piana 9d7c570593
Add PrepareContent and upgrade status-go (#1674) 2019-11-15 09:52:28 +01:00
Andrea Maria Piana 89659f85b4
Upgrade status-protocol-go (#1664) 2019-11-08 08:36:12 +01:00
Andrea Maria Piana a659685e3f
Add parsed message (#1660)
This commits add a field (parsedMessage) to the json payload sent to
status-react.
This field is the parsed version of the transit message.
The code is all in dedup, I will re-organize it once we made all the
necesseary changes.
2019-11-06 17:23:11 +01:00
Adam Babik 183d626256
Implement mailserver Deliver() with MessagesRequest (#1653)
The main difference is that it takes `whisper.MessagesRequest` as an argument instead of `whisper.Envelope`.
2019-11-04 21:15:21 +01:00
Jakub Sokołowski c1f17cb4a3 use prometheus/client_golang for metrics
also re-implement mailserver metrics

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2019-11-04 13:18:50 +01:00
Andrea Maria Piana 9f2a4b378f
Verify ens endpoint, only against mainnet (#1657)
* String payload

* Verify ENS Names

* Update status-protocol-go
2019-11-04 11:08:22 +01:00
Adam Babik a244b01a26
Upgrade migrate (#1643) 2019-10-14 16:10:48 +02:00
Pedro Pombeiro c874960215
Use status-protocol-go/bridge/geth (#1638)
* Use status-protocol-go/bridge/geth
2019-10-14 09:53:38 +02:00
Pedro Pombeiro d87caf57e3
Add dependabot config file (#1636)
* Add dependabot config file

* Fix status-protocol-go dependency
2019-10-11 07:00:46 +02:00
Jakub Sokołowski 294399916e add script for easier setup of mailserver
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2019-10-10 11:27:32 +02:00
Pedro Pombeiro a2f106e4c5
Update for status-protocol-go API changes (#1627) 2019-10-09 16:22:53 +02:00
Adam Babik 26880b83d7
Upgrade geth to 1.9.5 and Whisper (#1617) 2019-10-04 17:21:24 +02:00
Andrea Maria Piana 40e66e6186
upgrade status-protocol-go (#1619) 2019-09-26 11:26:33 +02:00
Andrea Maria Piana 9a3ed980c9
Generate gfycat and identicon in status-protocol-go (#1611) 2019-09-26 09:01:17 +02:00
Andrea Maria Piana 1a47893e75
Upgrade status-protocol-go (#1593) 2019-09-02 11:29:06 +02:00
Andrea Maria Piana b8ea79a3f0
Upgrade status-protocol-go (#1586) 2019-08-29 08:33:46 +02:00
Andrea Maria Piana e118e89107
Add persistent datasync, use single database and handle negotiated filters (#1580) 2019-08-27 14:04:15 +02:00
Andrea Maria Piana 06dc227071
Add Metadata to messages, expose new messenger methods (#1571)
* Add Metadata to messages, expose new messenger methods

This commits modifies deduplicator so that it takes a `StatusMessage`
instead of `WhisperMessage` and also returns a `Metadata` field which is
then passed back by the client when confirming messages, which fixes the
issue we had with not confirming pfs messages.
2019-08-20 13:20:25 +02:00
Adam Babik 8383feea04 add API to shhext to manage messages 2019-08-12 19:44:31 +02:00
Andrea Maria Piana 9ae7d2b6d5 Move envelopes tracking to status-protocol-go
This commit moves envelopes tracking to status-go.
Post endpoint is not going to track envelopes anymore, as that's taken
care on status-protocol-go side, so this is a breaking change, and
version is updated accordingly.
2019-08-12 19:44:31 +02:00
Andrea Maria Piana a3a413b5d9 Add SaveChat, Chats, DeleteChat, SaveConctat & Contacts endpoint 2019-08-05 16:03:18 +02:00
Adam Babik ce7b0fc376 conslidate protocol databases 2019-08-01 10:40:54 +02:00
Andrea Maria Piana 9de77b21b2 Add datasync, v1messages & disable discovery topic options
Adds support for datasync, V1Messages and disabling the discovery topic.
This is a backward compatible change as long as they are not toggled on
(they are not by default).
2019-07-29 19:39:44 +02:00
Andrea Franz dcb0fa5262
add multi-account support (#1500)
* add multiaccount support

add multi account ImportPrivateKey and StoreAccount

test derivation from normal keys

* add multiaccount to mobile pkg

* use multiaccount params structs from the mobile pkg

* move multiaccount tests together with the other lib tests

* fix codeclimate warning and temporarily increase methods threshold

* split library_test_utils.go to avoid linter warnings
2019-07-24 20:59:15 +02:00
Adam Babik e93d994460 integrate status-protocol-go 2019-07-24 18:46:43 +02:00
Adam Babik 664488faf9
add an adapter for zap and geth loggers (#1534) 2019-07-24 09:58:01 +02:00
Adam Babik 1ab2e88bf5
Sync messages in a loop until error or no cursor (#1502) 2019-06-26 18:17:41 +02:00
Adam Babik c9e99c432d migrate to go 1.12 and go modules 2019-06-12 13:12:00 +02:00