Commit Graph

1522 Commits

Author SHA1 Message Date
Andrea Maria Piana 3afde67022
Move message processor to common and allow subscribing to sent messages 2020-07-27 08:51:00 +02:00
Andrea Maria Piana 9da64ef251
add basic config 2020-07-27 08:50:59 +02:00
Andrea Maria Piana 15a3b710a0
Add parsing of messages 2020-07-27 08:50:58 +02:00
Andrea Maria Piana 927f762589
Send gorush notifications 2020-07-27 08:50:57 +02:00
Andrea Maria Piana 6ed7f705b9
validate token type 2020-07-27 08:50:56 +02:00
Andrea Maria Piana 52da9de0c6
Handle query from client 2020-07-27 08:50:56 +02:00
Andrea Maria Piana 2eee956dc6
Store hashes pk 2020-07-27 08:50:55 +02:00
Andrea Maria Piana a15919527b
implement handle push notification registration 2020-07-27 08:50:54 +02:00
Andrea Maria Piana 91c7869d98
implement migrations & persistence for pns 2020-07-27 08:50:53 +02:00
Andrea Maria Piana 24df11bcf9
Pull push notification options from persistence 2020-07-27 08:50:52 +02:00
Andrea Maria Piana eed0df3420
Return decrypted options on validation 2020-07-27 08:50:51 +02:00
Andrea Maria Piana c5077609ee
remove push notification preferences 2020-07-27 08:50:50 +02:00
Andrea Maria Piana f984a041df
decrypt push notification preferences 2020-07-27 08:50:49 +02:00
Andrea Maria Piana e19c799be2
Remove PushNotificationRegister as we can use ApplicationMessage as a wrapper 2020-07-27 08:50:48 +02:00
Andrea Maria Piana 4ded7bf74c
Add skeleton for server and separate namespace for client 2020-07-27 08:50:48 +02:00
Andrea Maria Piana 92b699b59d
Add push notification register message 2020-07-27 08:50:47 +02:00
Andrea Maria Piana c4fa9825a9
Mute chat 2020-07-27 08:50:46 +02:00
Andrea Maria Piana 58fcf809ea
Bug/fix pending peers segfault (#2004)
* Replace mclock with time in peer pools

we used mclock as golang before 1.9 did not support monotonic clocks,
https://github.com/gavv/monotime, it does now https://golang.org/pkg/time/
so we can fallback on the system implementation, which will return
nanoseconds with a resolution that is system dependent.

* Handle case where peer have same discovered time

In some system the resolution of the clock is not high enough so
multiple peers are added on the same nanosecond.

This result in the peer just added being immediately removed.

This code adds a check making sure we don't assume that a peer is added.

Another approach would be to make sure to include the peer in the list,
so prevent the peer just being added to be evicted, but it's slightly
more complicated and the resolution is generally accurate enough for our
purpose so that peers will be fresh enough if they have the same
discovered time.

It also adds a regression test, I had to use an interface to stub the
clock.

Fixes: https://github.com/status-im/nim-status-client/issues/522

* bump version to 0.55.3
2020-07-24 11:14:05 -04:00
Oskar Thorén 26978ca03d
Update README (#2003)
Geth decided to link to this place here https://github.com/ethereum/whisper, and it isn't a very welcoming README. Hopefully this points people in the right direction :)
2020-07-24 11:16:15 +08:00
Roman Volosovskyi 0b3cdf7362
DeleteMultiaccount method 2020-07-13 16:20:02 +03:00
Roman Volosovskyi 6d5a93287b
Separate keystore directory per multiacc
This commit provides `MigrateKeyStoreDir` method which moves multiacc's
keys from a common keystore directory to multiacc specific directory.
2020-07-09 12:41:47 +03:00
Samuel Hawksby-Robinson 157f20a7c8
Added link to specs for key compression API (#1995) 2020-06-24 14:39:42 +01:00
Samuel Hawksby-Robinson c8f9dad554
Feature/key compression (#1990)
## What has changed?

I've introduced to the public binding functionality that will compress and decompress public keys of a variety of encoding and key types. This functionality supports all major byte encoding formats and the following EC public key types:

- `secp256k1` pks
- `bls12-381 g1` pks
- `bls12-381 g2` pks

## Why make the change?

We want shorter public (chat) keys and we want to be future proof and encoding agnostic. See the issue here https://github.com/status-im/status-go/issues/1937

---

* Added basic signature for compresspk and uncompresspk

* Added basic encoding information

* make vendor

* formatted imports for the linter

* Reformatted imports hoping linter likes it

* This linter is capricious

* Added check that the secp256k1 key is valid

* Added test for valid key

* Added multiformat/go-varint dep

* Added public key type handling

* Added key decompression with key type handling

* Added handling for '0x' type indentifying

* Added more robust testing

* Less lint for the linting gods

* make vendor for bls12_381

* Added bls12-381 compression tests

* Added decompress key expected results

* Refactor of typed and untyped keys in tests

* Lint god appeasment

* Refactor of sample public keys

* Implemented bls12-381 decompression

* gofmt

* Renamed decode/encode funcs to be more descriptive

* Added binary bindings for key de/compression

* Refactor of func parameters

gomobile is a bit tempermental using raw bytes as a parameter, so I've decided to use string only inputs and outputs

* gofmt

* Added function documentation

* Moved multiformat de/compression into api/multiformat ns

* Moved multiformat de/compression into api/multiformat ns

* Changed compress to serialize on API
2020-06-23 11:47:17 +01:00
Andrea Maria Piana 4720224ba2 expose getContactByID method 2020-06-17 12:24:58 +02:00
Andrea Maria Piana 82deccef60 Use a single throttler per type 2020-06-15 19:46:23 +02:00
Andrea Maria Piana 35e95d1568 Move bytes rate limits to a different rlp key 2020-06-15 19:46:23 +02:00
Andrea Maria Piana c5577418b7 Implement rate limiting by bytes
Why make this change?

Currently we only rate limit by number of messages.
This works well if we want to limit a large amount of small messages but
breaks down if sending a smaller amount of large messages.

This commit extends the current code to limit by size as well, setting
the default to 1MB per second, which should be enough.

What has changed?

- Rate limiter for waku only
2020-06-15 19:46:23 +02:00
Andrea Maria Piana 2d12ac4fbb Rename rate limiting to packet rate limiting 2020-06-15 19:46:23 +02:00
Andrea Maria Piana 20f45a7c1c Create different index for filtering
The index for message was fairly inefficient as it was only using the
cursor, as it was referring to the old `chat_id` field.

This meant that newer messages would be fetched much faster then older
messages.

The index has been changed so that now it includes `local_chat_id`
(which is currently used for filtering), and not using `hide`.

The reason being is that `hide` is a low cardinality index, so there's
no performance benefit to have it in, also it's mostly ignored by the
query planner.

This commit also adds the missing migrations, we generated the file, but
the source was missing, probably I forgot to add them in a rebase. They
have been generated from the migration file, using `RestoreAsset`.
2020-06-08 10:02:31 +02:00
Samuel Hawksby-Robinson 88f4f75ae1
Added basic history of the protocol package (#1984)
'm trying to add more documentation to parts of the repo that I go into.

I feel this basic history will make it easier to understand why the protocol package is as big as it is compared to the other packages in the repo, and help in understanding its existence.
2020-06-04 11:23:09 +01:00
Jakub Sokołowski 1e3801d105 use types.BytesToHash to print []byte variables
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2020-06-04 09:45:47 +02:00
Andrea Maria Piana cbaf523b0e Address feedback 2020-06-03 11:07:04 +02:00
Andrea Maria Piana de698e64a3 Fix export & handle unsafe.pointer types 2020-06-03 11:07:04 +02:00
Andrea Maria Piana b8d64cbb1e Generate bindings dynamically
Before we had two directories `lib/` and `mobile/` that generated
respectively the bindings for desktop and ios/android.

This needed to be kept in sync and there was a fair amount of code
duplication, plus some missing methods on one or the other side.

I have made a change so the whole `lib/` namespace is generated by
parsing the `AST` of `mobile`, and bindings are generated before
compiling.
2020-06-03 11:07:04 +02:00
Andrea Maria Piana 0bffeab908 Address feedback from code review and add quoted image 2020-06-01 11:30:04 +02:00
Andrea Maria Piana d5086d6e89 Remove incentivisation service
Incentivisation was an experiment in running an incentivised fleet that
rewarded nodes based on their well behavior. It was heavily influenced
by https://docs.loki.network/ . It is currently not used anymore, so
removing.
2020-06-01 11:30:04 +02:00
Andrea Maria Piana 2d17c40631 Refactor *Raw methods in message_processor.go
We used *Raw method in message_processor as before we had non-Raw method
during the transition from status-react to status-go. This naming is not
meaningful anymore, so I have changed it.
2020-06-01 11:30:04 +02:00
Andrea Maria Piana 8ba6625df0 Lower PoW for messages
Images are too large to be sent over waku/whisper with the
current PoW (0.002).

This commit lowers the PoW for sending messages to a lower value,
depending on the size.

This means that older clients will not be able to receive messages that
are greater than 50KB (none of the messages we currently send is anywhere close to
that, most of them are less than 1KB).

I would have preferred to set the PoW explicitly for images to be lower,
but it's not trivial as we use `datasync` to send messages and its
interface takes a payload that is to be dispatched and a function to
dispatch, at initialization stage.

This make it difficult (impossible?) to set a different PoW for a
particular message, without changing function signature in datasync
(which is agnostic to the transport used).

So a less cumbersome approach is to just fingerprint on size.
2020-06-01 11:30:04 +02:00
Andrea Maria Piana f5ab58b87f Add support for images
This commit adds support for images in protobuf messages.

The client can specify a path which will be used to load the image
and set the corresponding fields.

This makes the assumption that the RCP server runs on the same machine
as the client and they have access to the same files. This holds
currently for both status-react and status-console-client, we could
revisit and adds an upload if that changes in the future.
2020-06-01 11:30:04 +02:00
Andrea Maria Piana 3e312cb167 Lower default MinimumPoW
This commit lowers the default minimum pow for both Waku & Whisper.
We need to override the value as mobile clients have this already stored
in the database.

This is done in order to support sending of images.
2020-06-01 11:30:04 +02:00
Andrea Maria Piana 12c7253b83 Add protocol to generate make target 2020-06-01 11:30:04 +02:00
Andrea Maria Piana ee0a83fdc4 Load identicon & alias in chat.
Why make the changes?

Mainly performance, those fields are almost always present in the
database but they are re-calculated on load by the client as it does not
have necessarily access to it.

What has changed?
- Remove `_legacy` persistence namespaces as it's a vestige of the
initial move frmo status-react to status-go
- Pulling chats is now a join with contacts to add contact & alias
2020-05-25 14:00:04 +02:00
Samuel Hawksby-Robinson 2d13ccf5ec
Patch/dependabot protocol (#1980)
* Removed protocol dependabot management, now redundant

resolves GH-1894
2020-05-19 00:14:02 +01:00
Samuel Hawksby-Robinson cc9c352337
Minor dependabot config yml format fix (#1979)
resolved GH-1977
2020-05-15 19:46:29 +01:00
Samuel Hawksby-Robinson 9e2e94d851
Updated dependabot config to security only (#1975)
see for details : https://dependabot.com/docs/config-file/#allowed_updates

## Why make the change?

@dependabot makes PRs for upgrading dependencies, but it doesn't automatically call `make vendor`, which is a problem because that means any merge for a @dependabot PR will require manual intervention.

I've looked through the @dependabot PR history and there has never been a successfully merged PR. [See here](https://github.com/status-im/status-go/pulls?q=is%3Apr+author%3Aapp%2Fdependabot-preview+is%3Aclosed)

If we are going to use this service we should only be warned of security updates, until we can implement a `make-vendor-bot` to automatically update the vendored files.

## What has changed?

Changed @dependabot settings to only make a PR for security updates.
2020-05-15 16:01:41 +01:00
André Medeiros be683556ff
fix: close resultsets so we don't leak them (#1809)
* fix: close resultsets so we don't leak them

* Refactor browsers/database

To implement PR suggestions and improve code quality.

* Refactor services/permissions/database

To implement PR suggestions and improve code quality.

Co-authored-by: Samuel Hawksby-Robinson <samuel@samyoul.com>
2020-05-14 11:51:32 +01:00
Samuel Hawksby-Robinson 77d9d0f4c9
Waku package README.md documentation (#1949)
* Added waku README.md, starting text and structure

Resolves GH-1944

* Added api, config and doc details

* Created Docs for waku.const, envelope and events

* Added notes on Filter

* Rewrote the filter section

* Restructured docs to match new versioning structure

* Moved package files out of the main ToC

* Added waku/common/README.md

* Added details to mailserver.go and notes to api.go

* Update to bloomfilter and const common

* Added common/ helpers.go, errors.go and message.go details

* Added common/metrics.go and protocol details

* Added rate_limiter and topic.go details

* fix rate limiter table

* Added Waku Versioning details

* Added Versioned sub-package file details

* Added better detail for Waku Filters

Addressing https://github.com/status-im/status-go/pull/1949#discussion_r419232235

* Added notes about light nodes not being actually new

* Added more detail for Waku node configs and defaults

* Added details to the version message.go section

* Added details to waku/common/rate_limiter

* Minor typo fix in docs
2020-05-14 08:49:06 +01:00
Samuel Hawksby-Robinson d8309afa07
Updated dependabot config to assign Samyoul (#1972) 2020-05-14 08:47:38 +01:00
Andrea Maria Piana a605442de5 Re-introduce deprecated method of requesting messages
During the move to waku/1 we removed handing of a deprecated format.
aa7f591587 (diff-ea5e44cf3db4a12b3a2a246c7fa39602R290)
Turns out that no client is using the new format and the only live
client is using the deprecated format.
This commit re-introduces the functionality.
2020-05-13 14:10:43 +02:00
Samuel Hawksby-Robinson a21f8a39bc
Minor Waku version subpackage message.go tidy (#1964)
Moved sendBundle() from v*/message_response.go to v*/peer.go
Moved TestSendBundle() from v*/message_response_test.go to v*/peer_test.go
Renamed message.go to v*/message_response.go
Renamed message_test.go to v*/message_response_test.go
2020-05-07 15:45:25 +01:00