Commit Graph

268 Commits

Author SHA1 Message Date
Andrea Maria Piana ddc93981a7 Add V4 endpoints 2021-04-19 14:49:28 +02:00
Shivek Khurana 1097b14a7f
🎭 📊 Anonymous Metrics V0 (#2170)
* Migrations in place, how to run them?

* Remove down migrations and touch database.go

* Database and Database Test package in place, added functions to get and store app metrics

* make generate output

* Minor bug fix on app metrics insert and select

* Add a validation layer to restrict what can be saved in the database

* Make validation more terse, throw error if schema doesn't exist, expose appmetrics service

* service updates

* Compute all errors before sending them out

* Trying to bring a closjure to appmetrics go

* Expose appmetrics via an api, skip fancy

* Address value as Jason Dawt Rawmasage to ease parsing

* Introduce a buffered chan with magic cap of 8 to minimize writes to DB. Tests for service and API. Also expose GetAppMetrics function.

* Lint issues

* Remove autoincrement, undo waku.json changes, fix error being shadowed, return nil where nil ought to be returned, get rid of buffered channel

* Bump migration number

* Fix API factory usage

* Add comment re:json.RawMessage instead of strings

* Get rid of test vars, throw save error inside the loop

* Update version

Co-authored-by: Samuel Hawksby-Robinson <samuel@samyoul.com>
2021-03-17 18:09:28 +05:30
Jakub Sokołowski fe8dab7391 go-ethereum: Drop throttling of inbound connections
This fix puts an end to a saga that essentially start during the
Status Prague Meetup at the end of October 2018. At the time we were
experiencing massive issues with `Connecting...` spinners in the app in the
venue we rented. We were pulling our hairs out what to do and we could not
find the cause of the issue at the time.

Three months later I deployed the following change:
https://github.com/status-im/infra-eth-cluster/commit/63a13eed

Which used `iptables` to map the `443` port onto our `30504` Status node port
using `PREROUTING` chain and `REDIRECT` jump in order to fix issues people
have been complaining about when using WiFi networks in various venues:
https://github.com/status-im/status-react/issues/6351

Our thinking when trying to resolve the reported issue assumed that some
networks might block outgoing connections on non-standard ports other than
the usual `80`(HTTP)/`443`(HTTPS) which would disrupt Status connectivity.
While this fix could have indeed helped a few edge cases, what it really
did was cause the Status node to stop seeing actual public IPs of the clients.

But __pure accident__ this change caused the code we inherited from
`go-ethereum` implementation of DevP2P protocol to stop throttling new
incoming connections, because the IP as which they appeared was a
`172.16.0.0/12` network address of the Docker bridge.

The `go-ethereum` code used the `!netutil.IsLAN(remoteIP)` check to
avoid throttling connections from local addresses, which included the
local Docker bridge address:
https://github.com/status-im/status-go/blob/82680830/vendor/github.com/ethereum/go-ethereum/p2p/netutil/net.go#L36

The fix intended to target a small number of networks with fortified
firewall configuration accidentally resolved our issues with
`Connecting...` prompts that our application showed us en masse during
our Prauge Meetup. Part of the reason for that is that venues like that
normally give out local IP addresses and use NAT to translate them onto
the only public IP address they possess.

Since out application is supposed to be usable from within networks
behind NAT like airport WiFi networks for example, it makes no sense to
keep the inbound connection throttle time implemented in `go-ethereum`.

I'm leaving `inboundThrottleTime` in because it's used to calculate
value for `dialHistoryExpiration` in:
`vendor/github.com/ethereum/go-ethereum/p2p/dial.go`

I believe reducing that value one we deploy this change should also
increase the speed with which the Status application is able to reconnect
to a node that was temporarily unavailable, instead waiting the 5*30 seconds.

Research issue: https://github.com/status-im/infra-eth-cluster/issues/35

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2021-02-16 20:38:47 +01:00
Andrea Maria Piana e5115d60f1 Upgrade pq library
There seems to be an issue with version 1.3, querying for topics on
postgres returns
and error:

```
panic: pq: invalid byte sequence for encoding "UTF8"
```

Upgrading pq fixes the issue

¯\_(ツ)_/¯
2021-01-27 09:23:11 +01:00
Andrea Maria Piana a1b3e3a772
Clean topics that we don't listen to
There was a bug on status-react where it would save filters that were
not listened to.
This commit adds a task to clean up those filters as they might result
in long syncing times.

This commit also returns topics/ranges/mailserves from messenger in
order to make the initialization of the app simpler and start moving
logic to status-go.

It also removes whisper from vendor.
2021-01-26 09:39:57 +01:00
Frank Tang 922e785512 Add GitHub to whitelist for URL unfurling #11426 2021-01-11 11:15:09 +01:00
Andrea Maria Piana 2695b70159 use own fork of resize 2020-12-17 14:10:00 +01:00
Andrea Maria Piana e8ffee3892 avoid division by 0 2020-12-17 14:10:00 +01:00
Samuel Hawksby-Robinson a0ea922017 make vendor 2020-12-17 14:10:00 +01:00
Andrea Maria Piana 6ad047d56f Fix datasync retry
This commit fixes a bug on the mvds library where the nextEpoch would be
incorrectly summed to the retry time, resulting in messages not being
retried, or retried much less frequently the longer the app was running.
It also updates the retry timing to backoff exponentially at multiple of
30 seconds.
2020-11-26 16:25:52 +01:00
Jonathan Rainville b2d8f7f416
feat: add bookmarks table for the browser bookmarks
plus get bookmark icon using a lib
2020-11-16 13:31:34 -05:00
Roman Volosovskyi 7467ca7b10
[markdown] Ignore status tags with upper case letters
related to https://github.com/status-im/status-react/issues/11317
2020-10-22 13:37:03 +03:00
Roman Volosovskyi a36144983a
Update markdown module 2020-10-19 13:48:03 +03:00
Andrea Maria Piana 9ddd963591
vendor 2020-07-30 20:20:56 +02:00
Samuel Hawksby-Robinson c8f9dad554
Feature/key compression (#1990)
## What has changed?

I've introduced to the public binding functionality that will compress and decompress public keys of a variety of encoding and key types. This functionality supports all major byte encoding formats and the following EC public key types:

- `secp256k1` pks
- `bls12-381 g1` pks
- `bls12-381 g2` pks

## Why make the change?

We want shorter public (chat) keys and we want to be future proof and encoding agnostic. See the issue here https://github.com/status-im/status-go/issues/1937

---

* Added basic signature for compresspk and uncompresspk

* Added basic encoding information

* make vendor

* formatted imports for the linter

* Reformatted imports hoping linter likes it

* This linter is capricious

* Added check that the secp256k1 key is valid

* Added test for valid key

* Added multiformat/go-varint dep

* Added public key type handling

* Added key decompression with key type handling

* Added handling for '0x' type indentifying

* Added more robust testing

* Less lint for the linting gods

* make vendor for bls12_381

* Added bls12-381 compression tests

* Added decompress key expected results

* Refactor of typed and untyped keys in tests

* Lint god appeasment

* Refactor of sample public keys

* Implemented bls12-381 decompression

* gofmt

* Renamed decode/encode funcs to be more descriptive

* Added binary bindings for key de/compression

* Refactor of func parameters

gomobile is a bit tempermental using raw bytes as a parameter, so I've decided to use string only inputs and outputs

* gofmt

* Added function documentation

* Moved multiformat de/compression into api/multiformat ns

* Moved multiformat de/compression into api/multiformat ns

* Changed compress to serialize on API
2020-06-23 11:47:17 +01:00
Adam Babik d7bb02540a
Update go-ethereum to v1.9.5-status.9 (#1898) 2020-03-12 13:05:12 +01:00
Adam Babik 831ab85561
Upgrade deps (#1892) 2020-03-09 08:55:58 +01:00
Andrea Maria Piana a82de9f5f9
Upgrade extkeys (#1878) 2020-03-04 09:30:31 +01:00
Andrea Maria Piana ddf7c2d1c2 upgrade whisper 2020-02-28 13:27:00 +01:00
Andrea Maria Piana 2ae04f2a2d acknowledge spelling 2020-02-28 13:27:00 +01:00
Andrea Maria Piana 7a64fbf5f7
make apis private (#1866)
* make apis private

* Bump whisper
2020-02-25 08:03:01 +01:00
Adam Babik 91e658aeb6
bump Whisper to 6.2.1 (#1869) 2020-02-21 16:22:25 +01:00
Andrea Maria Piana 849492fda9
Add statusUpdate code (#1861)
* Add status-option code

This commits changes the behavior of waku introducing a new status-code,
`2`, that replaces the current single options codes.

* linting
2020-02-21 15:48:53 +01:00
Adam Babik 76b5dc29dc
Integrate Whisper-Waku bridge in status-go (#1854) 2020-02-18 12:21:01 +01:00
Jakub Sokołowski cca47ef6e1 rename gomarkdown/markdown to status-im/markdown
This resolves a dependency conflict we have with MatterBridge
which was using a newer version of the same package.
This resulted in a JSON marshalling bug that would crash the bridge.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2020-02-11 16:06:43 +01:00
Adam Babik 8b61d92d58
Remove protocol and eth-node submodules (#1835) 2020-02-10 12:22:37 +01:00
Andrea Maria Piana 0b6ad662b5
Bug/validate pubkey (#1838)
* validate chat before persisting

* add comment to public key generation
2020-02-07 12:56:30 +01:00
Adam Babik d27a507e0d
make Messenger.Timesource public (#1837) 2020-02-07 12:30:26 +01:00
Andrea Maria Piana f2eebd1e49
Verify ENS in the background (#1824)
Currently ENS are verified explicitly by status-react, this is not ideal
as if that fails it will have to be explicilty retried in status-react.
This commits changes that behavior so that ENS are verified in a loop
and updated if new messages are received.
2020-02-05 11:09:33 +01:00
Andrea Maria Piana 8931b14c4e
Explicitly init messenger (#1821)
Messenger needs to wait until all the topics/mailservers are loaded in
the client, so we explicitly add a way to call Init.
2020-01-29 20:40:06 +01:00
Pedro Pombeiro 741f43cb0c Fix wrong field being logged in `EnvelopesMonitor` 2020-01-23 17:19:10 +01:00
Andrea Maria Piana a4f88d0017
Fix ens names and contact (#1810)
Currently the wrong pk format was used to query the in-memory contacts,
this commit corrects the behavior.
2020-01-23 09:24:28 +01:00
Adam Babik bc2d018483
Add support for request messages by topics (#1805) 2020-01-21 08:11:24 +01:00
Adam Babik 79b8112f89
Split shhext into shhext and wakuext (#1803) 2020-01-20 21:56:06 +01:00
Andrea Maria Piana 456bcfa022
Peg clock value to whisper timestamp (#1804)
This commit pegs the clock value to maximum + 120 seconds from the whisper
timestamp.
In this way the we avoid the scenario where a client makes the timestamp
increase arbitrarely.
2020-01-20 17:44:32 +01:00
Pedro Pombeiro d4710faae2 In progress: Use Nimbus keystore 2020-01-20 13:15:17 +01:00
Pedro Pombeiro 6537cae606 Nimbus node support 2020-01-20 13:15:17 +01:00
Andrea Maria Piana 25d46c6d82
Fix waku tests & contact ens (#1802) 2020-01-17 13:39:09 +01:00
Pedro Pombeiro 7f45c43196 make lint-fix 2020-01-16 13:18:36 +01:00
Andrea Maria Piana c569d8a4ed
Sync installation messages & contact requests (#1791) 2020-01-15 08:25:09 +01:00
Adam Babik 44aa313981
Make shhext and protocol work with Waku (#1777)
This change makes shhext and protocol submodule work with Waku and Whisper.
2020-01-13 20:17:30 +01:00
Jakub Sokołowski 330d177de6 specify gomarkdown/markdown version to fix conflicts
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2020-01-11 19:41:24 +01:00
Andrea Maria Piana 88a1d0111e
Add Commands (#1731)
This commit adds handling of Transaction commands.
2020-01-10 19:59:01 +01:00
yenda d3de88f790 fix stickers persistence (#1779) 2020-01-10 16:12:09 +01:00
Adam Babik 37a508a97b
Integrate Waku service (#1759) 2020-01-08 12:12:23 +01:00
Pedro Pombeiro c8a911ebd1 Use goimports instead of gofmt 2020-01-06 10:17:23 +01:00
Pedro Pombeiro 3eaacaa2f4 Move keystore_geth.go to eth-node/bridge/geth 2020-01-06 10:17:23 +01:00
Pedro Pombeiro 287e5cdf79 Abstract `accounts.Key` and geth `keystore` 2020-01-06 10:17:23 +01:00
Pedro Pombeiro 41cba814c8 Run `make vendor` 2019-12-30 13:25:12 +01:00
Pedro Pombeiro f494576ca2 Run `gofmt -s -w` 2019-12-20 11:24:26 +01:00