Commit Graph

1733 Commits

Author SHA1 Message Date
Andrea Maria Piana 3e312cb167 Lower default MinimumPoW
This commit lowers the default minimum pow for both Waku & Whisper.
We need to override the value as mobile clients have this already stored
in the database.

This is done in order to support sending of images.
2020-06-01 11:30:04 +02:00
Andrea Maria Piana 12c7253b83 Add protocol to generate make target 2020-06-01 11:30:04 +02:00
Andrea Maria Piana ee0a83fdc4 Load identicon & alias in chat.
Why make the changes?

Mainly performance, those fields are almost always present in the
database but they are re-calculated on load by the client as it does not
have necessarily access to it.

What has changed?
- Remove `_legacy` persistence namespaces as it's a vestige of the
initial move frmo status-react to status-go
- Pulling chats is now a join with contacts to add contact & alias
2020-05-25 14:00:04 +02:00
Samuel Hawksby-Robinson 2d13ccf5ec
Patch/dependabot protocol (#1980)
* Removed protocol dependabot management, now redundant

resolves GH-1894
2020-05-19 00:14:02 +01:00
Samuel Hawksby-Robinson cc9c352337
Minor dependabot config yml format fix (#1979)
resolved GH-1977
2020-05-15 19:46:29 +01:00
Samuel Hawksby-Robinson 9e2e94d851
Updated dependabot config to security only (#1975)
see for details : https://dependabot.com/docs/config-file/#allowed_updates

## Why make the change?

@dependabot makes PRs for upgrading dependencies, but it doesn't automatically call `make vendor`, which is a problem because that means any merge for a @dependabot PR will require manual intervention.

I've looked through the @dependabot PR history and there has never been a successfully merged PR. [See here](https://github.com/status-im/status-go/pulls?q=is%3Apr+author%3Aapp%2Fdependabot-preview+is%3Aclosed)

If we are going to use this service we should only be warned of security updates, until we can implement a `make-vendor-bot` to automatically update the vendored files.

## What has changed?

Changed @dependabot settings to only make a PR for security updates.
2020-05-15 16:01:41 +01:00
André Medeiros be683556ff
fix: close resultsets so we don't leak them (#1809)
* fix: close resultsets so we don't leak them

* Refactor browsers/database

To implement PR suggestions and improve code quality.

* Refactor services/permissions/database

To implement PR suggestions and improve code quality.

Co-authored-by: Samuel Hawksby-Robinson <samuel@samyoul.com>
2020-05-14 11:51:32 +01:00
Samuel Hawksby-Robinson 77d9d0f4c9
Waku package README.md documentation (#1949)
* Added waku README.md, starting text and structure

Resolves GH-1944

* Added api, config and doc details

* Created Docs for waku.const, envelope and events

* Added notes on Filter

* Rewrote the filter section

* Restructured docs to match new versioning structure

* Moved package files out of the main ToC

* Added waku/common/README.md

* Added details to mailserver.go and notes to api.go

* Update to bloomfilter and const common

* Added common/ helpers.go, errors.go and message.go details

* Added common/metrics.go and protocol details

* Added rate_limiter and topic.go details

* fix rate limiter table

* Added Waku Versioning details

* Added Versioned sub-package file details

* Added better detail for Waku Filters

Addressing https://github.com/status-im/status-go/pull/1949#discussion_r419232235

* Added notes about light nodes not being actually new

* Added more detail for Waku node configs and defaults

* Added details to the version message.go section

* Added details to waku/common/rate_limiter

* Minor typo fix in docs
2020-05-14 08:49:06 +01:00
Samuel Hawksby-Robinson d8309afa07
Updated dependabot config to assign Samyoul (#1972) 2020-05-14 08:47:38 +01:00
Andrea Maria Piana a605442de5 Re-introduce deprecated method of requesting messages
During the move to waku/1 we removed handing of a deprecated format.
aa7f591587 (diff-ea5e44cf3db4a12b3a2a246c7fa39602R290)
Turns out that no client is using the new format and the only live
client is using the deprecated format.
This commit re-introduces the functionality.
2020-05-13 14:10:43 +02:00
Samuel Hawksby-Robinson a21f8a39bc
Minor Waku version subpackage message.go tidy (#1964)
Moved sendBundle() from v*/message_response.go to v*/peer.go
Moved TestSendBundle() from v*/message_response_test.go to v*/peer_test.go
Renamed message.go to v*/message_response.go
Renamed message_test.go to v*/message_response_test.go
2020-05-07 15:45:25 +01:00
Samuel Hawksby-Robinson 4d00656c41
Refactor/waku tidy 1 (#1958)
* Refactor tidy of waku package

* Added deprecation warning on whisper README.md

* Appeasing the lint gods and testing is good

* Place Whisper deprecation warning in the correct package README

:facepalm

* Implementing changes after team feedback

* More offerings to the lint gods

* Remove apparently redundant context params

* Correctly handle concurrent HandlePeer err

* Revert "Remove apparently redundant context params"

This reverts commit 557dbd0d64.

* Added note to waku/api.go about context

* renamed statusoptions and removed unused global

* Removed OnNewP2PEnvelopes() from WakuHost interface

* Matched v1 Peer with new interface sig

Also changed common/helper.go to common/helpers.go

* Formatting of waku tests and some additional error handling

* Changed version to 0.53.0

* Removed redundant type declaration

* Moved TopicToBloom function into a Topic{} method

* Moved GenerateSecureRandomData() into helpers.go
2020-05-01 19:14:01 +01:00
Andrea Maria Piana 0e27e464d3 Add tests for sendBundle and fix flaky tests 2020-05-01 10:14:04 +02:00
Andrea Maria Piana 299b3fc093 Implement waku/1
Why make the change?

This implements waku/1 which is a breaking change as waku/0 diverged
from the specs.

What has changed?

- Added v1 namespace
- Changed the testing code to test from the outside rather than stubbing
p2p specific messages, so that any version specific code is under `vx`
namespaces
- Split waku vs waku/x test code
- Added a test suite that can be used for different versions

Things still to do

I kept the tests we have for the versioned aspect of waku, probably we
want to add some or change other to provide the optimal test coverage.
2020-05-01 10:14:04 +02:00
Andrea Maria Piana 8aa42e4148 Create waku/1 namespace
This commit creates a waku/1 namespace and adds the code to it.
No changes in the protocol have been made (i.e waku/1 is the same as
waku/1 in this commit).
2020-05-01 10:14:04 +02:00
Samuel Hawksby-Robinson b025db235f
Refactor/waku tidy (#1957)
* Refactor tidy of waku package

* Added deprecation warning on whisper README.md

* Appeasing the lint gods and testing is good

* Place Whisper deprecation warning in the correct package README

:facepalm

* Implementing changes after team feedback

* More offerings to the lint gods

* Remove apparently redundant context params

* Correctly handle concurrent HandlePeer err

* Revert "Remove apparently redundant context params"

This reverts commit 557dbd0d64.

* Added note to waku/api.go about context
2020-04-30 13:52:48 +01:00
Andrea Maria Piana aa7f591587 Move networking code for waku under `v0` namespace
Why make the change?

As discussed previously, the way we will move across versions is to maintain completely separate
codebases and eventually remove those that are not supported anymore.

This has the drawback of some code duplication, but the advantage is that is more
explicit what each version requires, and changes in one version will not
impact the other, so we won't pile up backward compatible code.
This is the same strategy used by `whisper` in go ethereum and is influenced by
https://www.youtube.com/watch?v=oyLBGkS5ICk .

All the code that is used for the networking protocol is now under `v0/`.
Some of the common parts might still be refactored out.
The main namespace `waku` deals with `host`->`waku` interactions (through RPC),
while `v0` deals with `waku`->`remote-waku` interactions.

In order to support `v1`, the namespace `v0` will be copied over, and changed to
support `v1`. Once `v0` will be not used anymore, the whole namespace will be removed.

This PR does not actually implement `v1`, I'd rather get things looked over to
make sure the structure is what we would like before implementing the changes.

What has changed?

- Moved all code for the common parts under `waku/common/` namespace
- Moved code used for bloomfilters in `waku/common/bloomfilter.go`
- Removed all version specific code from `waku/common/const` (`ProtocolVersion`, status-codes etc)
- Added interfaces for `WakuHost` and `Peer` under `waku/common/protocol.go`

Things still to do

Some tests in `waku/` are still testing by stubbing components of a particular version (`v0`).
I started moving those tests to instead of stubbing using the actual component, which increases
the testing surface. Some other tests that can't be easily ported should be likely moved under
`v0` instead. Ideally no version specif code should be exported from a version namespace (for
example the various codes, as those might change across versions). But this will be a work-in-progress.

Some code that will be common in `v0`/`v1` could still be extract to avoid duplication, and duplicated only
when implementations diverge across versions.
2020-04-27 14:58:02 +02:00
Samuel Hawksby-Robinson a98e31c20f
Refactor moved message functionality from filter to message (#1951)
When documenting the `filter.go` file I found that there was a lot of functionality that belongs in the `message.go` file, so I've moved it.
2020-04-24 15:34:45 +01:00
Samuel Hawksby-Robinson cc8d57752f
Refactor/waku.doc (#1950)
* Refactor of waku.doc.go

Moved out all code from docs.go, placed much of the seemingly misc code into much more closely related files, created a dedicated const file for housing all the package consts.

* Moved doc_test to message_test

* Refactored type collision and check on nil err

* message_test.go fmt

* Implemented whisper/doc.go refactor

* import order change to attempt to please the linter.

* Bug fix on whisper_test.go

I also spotted a load of redundant type casting and fixed some of them
2020-04-23 20:54:24 +01:00
Andrea Maria Piana 7bbe9561de Set chat active on being re-invited
If the user deletes/leaves a group chat, the chat is set as not active.
This means that if we are re-invited to the chat it won't be shown to
the user.

This commit changes this behavior so that if we are re-invited to the
chat it is set as active again.
2020-04-23 11:04:50 +02:00
Samuel Hawksby-Robinson 7d3655976c Reverting most tests related to handshake change 2020-04-21 10:40:39 +02:00
Samuel Hawksby-Robinson 59530b9d1e Reverting most of the handshake functionality
See GH-1938
2020-04-21 10:40:39 +02:00
Samuel Hawksby-Robinson 900bad769a Added better prediction of waku RLP key type and value 2020-04-21 10:40:39 +02:00
Samuel Hawksby-Robinson a50e35b72d Added error handling for waku handshake DecodeRLP 2020-04-21 10:40:39 +02:00
Samuel Hawksby-Robinson 042ae30f9f Refactor to waku handshake keyType identification
Presuming that all RLP keys will be of the same type. Still failing on forward and backward compatibility, so I will fidn a solution for this next
2020-04-21 10:40:39 +02:00
Samuel Hawksby-Robinson c16eba55f6 Implemented RLP key field initialisation
This replaces the statically declared values, which previously replaced function varialbes see https://github.com/status-im/status-go/pull/1931#discussion_r406123786
2020-04-21 10:40:39 +02:00
Samuel Hawksby-Robinson 91e1bdc8ce Apply suggestions from code review
Co-Authored-By: Adam Babik <a.babik@designfortress.com>
2020-04-21 10:40:39 +02:00
Samuel Hawksby-Robinson 960284056b Added handling for mixed RLP key data types
This the initial implementation, a number of known test fails have been introduced with this commit and will be resolved as a matter of priority.
2020-04-21 10:40:39 +02:00
Samuel Hawksby-Robinson 7d95118b35 Added new test and used static var declarations for keyFieldIdx 2020-04-21 10:40:39 +02:00
Samuel Hawksby-Robinson a44fb3c372 Added handling of parseStatuOptionKeys() error 2020-04-21 10:40:39 +02:00
Samuel Hawksby-Robinson 1858ad3ae2 Preliminary fix for GH-1929 2020-04-21 10:40:39 +02:00
Samuel Hawksby-Robinson 4b0d0f56dd Added IDE file to gitignore 2020-04-21 10:40:39 +02:00
Samuel Hawksby-Robinson 3455b57b4c
Update team collaboration forum (#1946)
As per discussion on Discord https://discordapp.com/channels/624205794384281629/626738033701683200/701827035382808666 , updating the Slack channel to the Discord server.
2020-04-21 08:12:01 +01:00
Andrea Maria Piana 2d1f6c1cfa Don't pass contacts without custom fields to the client
When receiving a message with save a contact in the database in order to
avoid re-calculating image/profile.
This contact is then passed to the client, which can negatively impact
performance.

This commit changes the behavior so that only those contacts that have
some custom fields (have been explicitly added by the user, have been
blocked by the user, have sent a contact request or have a verified ens
name) are passed to the client.
2020-04-21 09:09:27 +02:00
Andrea Maria Piana 3a84afd0f1 Add replies to messages
Currently replies to messages are handled in status-react.
This causes some issues with the fact that sometimes replies might come
out of order, they might be offloaded to the database etc.

This commit changes the behavior so that status-go always returns the
replies, and in case a reply comes out of order (first the reply, later
the message being replied to), it will include in the messages the
updated message.

It also adds some fields (RTL,Replace,LineCount) to the database which
were not previously saved, resulting in some potential bugs.

The method that we use to pull replies is currently a bit naive, we just
pull all the message again from the database, but has the advantage of
being simple. It will go through performance testing to make sure
performnace are acceptable, if so I think it's reasonable to avoid some
complexity.
2020-04-16 15:51:28 +02:00
Andrea Maria Piana 4d2fb67add Allow to leave chat without removing it 2020-04-14 14:08:54 +02:00
Andrea Maria Piana f3fc6812cd Add change group chat name endpoint 2020-04-14 14:08:54 +02:00
Andrea Maria Piana 79be6eaf3a Increment retry count when ENS verification fails
We did not increment the counter when verification failed, which meant
that we would not backoff and retry more often that we should have.
2020-04-14 11:16:43 +02:00
Adam Babik 2948544b7a
update protocol package README (#1934) 2020-04-09 23:28:40 +02:00
Adam Babik 13da3a09c5
Explain how to run Waku node in README (#1921) 2020-04-09 18:33:53 +02:00
Jakub Sokołowski c8d20f84f8 set archived_envelopes_total on start based on DB count
Before the count was reset every time process was restarted.
Also add 'db' label to distinguish between waku and whisper DBs.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2020-04-09 13:08:14 +02:00
Jakub Sokołowski 42baf2251d expose the mailserver metrics port by default
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2020-04-08 17:08:49 +02:00
Jakub Sokołowski bc70895615 enable waku for mailservers by default
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2020-04-08 17:08:49 +02:00
Andrea Maria Piana 3c5354280c Return count of updated messages in MarkMessagesSeen
The frontend needs to know if the messaged marked as seen was actually
seen or not, as we might not have it loaded in the frontend.
2020-04-07 13:23:21 +02:00
Adam Babik 4f9f7cd258
Fix setup-dev target (#1926) 2020-04-06 09:54:06 +02:00
Adam Babik f86c8cf91e
Update eth-node README (#1916) 2020-04-01 20:13:54 +02:00
Andrea Maria Piana 44272944ab Make sure WakuEnabled is respected on upgrade 2020-03-30 15:28:16 +02:00
Andrea Maria Piana 070a5adbee Enable waku node default 2020-03-30 10:52:30 +02:00
Adam Babik f9ba05d3ae
Reduce bridge logging (#1918) 2020-03-26 21:33:47 +01:00
Andrea Maria Piana a5874fdb21 Add content type for group chats 2020-03-25 09:29:08 +01:00