Commit Graph

210 Commits

Author SHA1 Message Date
r4bbit.eth 6c148389c1
fix(message_persistence): ensure query cursor value is supplied where (#2898)
needed

Unfortunately, this one slipped through when introducing helper
functions to retrieve messages.

Sometimes, queries need an additional empty `cursor` value.
2022-10-07 13:38:28 -04:00
Richard Ramos bf04ff3502 fix: waku2 store queries 2022-10-05 18:43:10 -04:00
Richard Ramos 7272bf0d91 fix: waku2 storenode cursor 2022-10-03 18:49:41 -04:00
Pascal Precht 9ee2967e78 feat(message_persistence): add discord message author image payload fields
This also adds some persistence APIs to operate on the new field (later
needed for the discord imoprt tool)
2022-09-19 13:47:16 +02:00
Richard Ramos 824bb17ba9
fix: determine if a enr contains a UDP port (#2864) 2022-09-16 09:09:13 -04:00
frank 6e4ac1c495 Use image server for all kind of images and integrate identity rings 2022-09-16 09:52:08 +01:00
frank 325efd01d0 Implement identity ring 2022-09-16 09:52:08 +01:00
Anthony Laibe 48f678052c
feat: batch balance (#2833) 2022-09-09 08:58:36 +02:00
Pascal Precht 3f987cc565 fix(messenger): don't remove inactive chats when syncing
This commit ensures we're relying on `chat.DeletedAtClockValue` instead
of `chat.Active` to know whether or not we need to remove the chat from
paired devices.

Because we were relying on `Active != true`, we ended up with a serious
but that would result in deactivating all chats on paired devices.

The reason the chats would disappear on paired devices is because, when
setting up a new device by importing a seedphrase, chances are this
device will receive `HandleBackUp` signals (which original from other
devices with the same account that backed up contacts etc).

When backups are handled, we create chats for every contact that's part
of the backup signal. Those chats are set to `Active = false` because
the signal handling shouldn't cause those chats to show up in the UI.

However, because those are set to `Active = false`, the next time the
user tries to sync from this devices, all those chats are considered as
"removed", hence sending "chat removed" signals when syncing (which then
causes those chats to disappear on all paired devices.

We need to rely on `DeletedAtClockValue` to know whether a chat was
indeed removed and only then emit such a signal.
2022-08-25 11:51:01 +02:00
Samuel Hawksby-Robinson 1e55797ec8 Added challenge middleware 2022-08-25 00:02:32 +01:00
Richard Ramos ad326fa290
feat: wakuv2 store (#2780)
Allows runnning a store node depending on node config settings.
2022-08-19 12:34:07 -04:00
Richard Ramos 05073a9640
chore: bump markdown (#2726) 2022-06-29 08:15:21 -04:00
Pascal Precht c855272340 feat: add garbage collection for soft deleted bookmarks
This introduces a simple garbage collection which checks for all soft
deleted bookmarks (`removed = 1`) which have been marked for garbage
collection more than 30 days ago from the time of bootstapping the
messenger.

Closes #2705
2022-06-17 14:21:21 +02:00
Pascal Precht 9d5bcc3c60 feat: sync `CommunitySettings` across paired devices
This commit introduces a new `clock` field in the
`communities_settings` table so that it can be leveraged for syncing
community settings across devices.

It alsoe exends existing `syncCommunity` APIs to generate
`SyncCommunitySettings` as well, avoiding sending additional sync messages
for community settings.

When editing communities however, we still sync community settings
explicitly are we aren't syncing the community itself in that case.
2022-06-16 12:45:05 +02:00
Samuel Hawksby-Robinson 05b8ddf57a
LAN connection string (#2653)
* Version bump

* Implemented lan connection string functionality

Also added more robust testing

* Added ConnectionParams struct and related funcs

* Add server mode to ConnectionParams
2022-06-15 16:36:17 +01:00
Richard Ramos 30f8bd6dfb
feat: add service for checking if there are new versions available (#2552) 2022-06-08 08:38:26 -04:00
Sale Djenic 78cba969cc notifications settings end points added and `notification_settings` table 2022-05-26 11:49:08 +02:00
Andrea Maria Piana 4d15ae8a85 Upgrade to go 1.18 2022-05-11 12:39:54 +01:00
Richard Ramos ca468e21e5 chore: bump go-waku 2022-04-06 10:36:06 -04:00
Pascal Precht 364838532f Add torrent library dependency 2022-04-06 11:48:16 +02:00
Richard Ramos faf8f34fef fix: adding missing config settings 2022-04-01 17:39:10 -04:00
Andrea Maria Piana 8a5f77dc37 Address linter issues 2022-03-28 13:14:12 +01:00
Andrea Maria Piana f0d3e0419d Backup communities 2022-03-23 14:45:52 +00:00
Richard Ramos cbe2560d92 chore: bump go-waku 2022-03-23 09:34:59 -04:00
Pascal Precht 86926258ee Intrododuce `NodeConfig.TorrentConfig` and CLI flags
This commit introduces a new `TorrentConfig` as per #2563 with dedicated
default values and new options/flags for the status-go CLI.

Since it's part of `NodeConfig`, which is stored per user in the
database, this commit also adds a migration script that adds a new
`torrent_config` table and database APIs to insert and retreive the
torrent config.

Closes #2563
2022-03-21 12:59:09 +01:00
Andrea Maria Piana d60a6713fe Enable mailserver cycle by default
This commit enables mailserver cycle logic by default and make a few
changes:

1) Nodes are graylisted instead of being blacklisted for a set amount of
   time. The reason is that if we blacklist, any cut in connectivity
   might result in long delays before reconnecting, especially on spotty
   connections.

2) Fixes an issue on the devp2p server, whereby the node would not
   connect to one of the static nodes since all the connection slots
   where filled. The fix is a bit inelegant, it always connects to
   static nodes, ignoring maxpeers, but it's tricky to get it to work
   since the code is clearly not written to select a specific node.

3) Adds support to pinned mailservers

4) Add retries to mailservers requests. It uses a closure for now, I
   think we should eventually have a channel etc, but I'd leave that for
   later.
2022-03-19 08:56:22 +00:00
Noelia 7ef8bc68c8
feat: Added new endpoint to get password strength info (#2589)
By integrating `zxcvbn` module, it has been added a new endpoint to get password strength quality information like Entropy, CrackTime, CrackTimeDisplay, Score, MatchSequence and CalcTime.

Added related dependences.

Closes #4980
2022-03-18 13:20:13 +01:00
Richard Ramos a15c27c588 feat: unified chat API pt. 3 (#2535) 2022-03-03 15:37:07 -04:00
Richard Ramos a0210ac281 chore: bump go-waku
Includes support for dns4 multiaddresses
2022-03-03 13:40:26 -04:00
Richard Ramos 559e6dddfa fix: limit number of concurrent requests 2022-03-02 17:46:16 -04:00
Richard Ramos 0a758b0f7d feat: stickerpacks service 2022-03-02 17:46:16 -04:00
Richard Ramos 37b06cd3b1 chore: bump go-waku 2022-03-02 12:56:50 -04:00
Richard Ramos 381c5f6116 chore: bump go-waku 2022-02-24 10:07:42 -04:00
Richard Ramos 18854c4cfa chore: bump go-waku 2022-02-18 16:12:42 -04:00
Richard Ramos 1a439baa5c
chore: bump go-waku (#2494) 2022-01-18 17:19:43 -04:00
Anthony Laibe 12c727df25
feat: Add ens service (#2467) 2021-12-21 16:05:09 +01:00
Richard Ramos b244188702
feat: LoginWithConfig (#2469)
Adds a new funtion to login with an account passing a node configuration that will override any specified attribute from the stored account nodeconfig
2021-12-21 10:27:18 -04:00
Richard Ramos 87a033c249
bump go-waku (#2442) 2021-11-25 10:37:06 -04:00
Richard Ramos d32f0467ae
feat: Add DiscV5 to Waku2 (#2434)
* feat: Add DiscV5 to Waku2
* fix: null verification for discV5
2021-11-22 09:40:14 -04:00
Anthony Laibe f24cc2e9fc
chore: bum go-waku (#2430) 2021-11-15 09:44:03 +01:00
Richard Ramos d35e0a339d
feat(waku2): use relay or lightpush depending on the number of peers (#2425) 2021-11-09 15:16:05 -04:00
Richard Ramos f47229a466
feat: add dns discovery to wakuv2 (#2422) 2021-11-09 08:22:34 -04:00
Richard Ramos 4602982c77
waku2: enable message confirmations (#2416) 2021-11-02 14:27:37 -04:00
Richard Ramos bade0a24ff
chore: bump go-waku (#2404) 2021-10-19 09:43:41 -04:00
Richard Ramos cf11713d9c
feat: wakuv2 peer persistence (#2287) 2021-10-12 08:39:28 -04:00
Richard Ramos a8d661ee71
bump go-waku (#2400) 2021-10-08 10:48:57 -04:00
Richard Ramos 4dbef97367
feat: waku2 + rendezvous (#2393) 2021-10-06 12:08:54 -04:00
Richard Ramos 065defd35b
bump markdown version (#2238) 2021-10-06 12:08:28 -04:00
Richard Ramos 6bb04e0858 fix: dial peer async 2021-10-01 08:29:23 -04:00
Andrea Maria Piana 902b97be06 Upgrade status-go ethereum 2021-09-28 08:22:17 +01:00
RichΛrd da63894a5a feat: wakuV2 light client using lightpush / filter (#2351) 2021-09-27 09:04:28 -04:00
Vitaliy Vlasov 049afe5765 Pass connection status chan to go-waku
Fix conn status chan gorouting never quitting

Fix import cycle

Add keepAlive interval to wakuv2 node

Add CustomNodes

Do not resume wakuv2 store
2021-09-23 19:00:40 +03:00
Roman Volosovskyi 839d6b3194
[status-im/status-react#12517] Ensure that all necessary contact fields are synced 2021-09-23 12:22:49 +03:00
Jakub Sokołowski 116f76e189 Makefile: pin tool versions, use go install
Using latest version every time is just asking for weird random issues.

This commit in `gomobile` has made Go `1.16` required:
80cb72e7af

Since we upgarded Go to 1.16 on CI hosts we can now use the recommended
method of installing tools, which is using `go install`.

In addition to that I've pinned other tools that we install to specific
versions, and dropped use of `modvendor` fork thanks to merging of:
https://github.com/goware/modvendor/pull/13

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2021-09-17 14:39:30 +02:00
RichΛrd 12ddb0739e
feat: obtain external address for rendezvous (#2333)
* feat: obtain external address for rendezvous
If the ext ip returned by geth is 127.0.0.1, it will attempt to obtain the external IP address via rendezvous and use that to register the ens record later
* fix: failing test
* fix: code review, and adding external ip address to logs
2021-09-06 09:46:35 -04:00
RichΛrd 719a303b05
feat: wakuV2 bandwidth stats (#2325) 2021-08-30 17:35:37 -04:00
RichΛrd 0aab3e238b
chore: bump go-waku (#2292) 2021-08-03 16:40:22 -04:00
Michele Balistreri c61cf4e1b2
re-apply signTypedData fix (#2297) 2021-08-03 19:10:16 +02:00
RichΛrd bfdc000bbc
go-waku: RequestAllHistoricMessages (#2258)
* feat: update wakuv2 store

* check online using waku version for now

* fix peerId and from/to multiplier

* fix: lint

* fix: handle waku2 cursors

* fix: code review

* fix: code review 2

* update go-waku version

* update vendor folder
2021-07-21 15:02:50 -04:00
Andrea Maria Piana d84da4f515 Fix some tests and restore private/public RPC client 2021-07-20 10:57:38 +02:00
Andrea Maria Piana 1e8688ac56 Upgrade to london signer and cleanup 2021-07-20 10:57:38 +02:00
Andrea Maria Piana 36430257fd Upgrade geth
Point to the new fork of geth.
2021-07-20 10:57:38 +02:00
Andrea Maria Piana c1db89e657 Fix iconcache
Iconcache library needed to be upgrade to be compatible with the
libraries required in the latest geth.
2021-07-20 10:57:38 +02:00
Andrea Maria Piana 2f539d3bd2 Upgrade go-ens
Go ens needs to be updated to be compatible with the lastest geth
version
2021-07-20 10:57:38 +02:00
Andrea Maria Piana b4f8ae7602 waku go version 2021-06-29 13:15:15 +02:00
Andrea Maria Piana eeb691df73 more shooting in the dark 2021-06-29 13:15:15 +02:00
Andrea Maria Piana d6a90e49c2 upgrade x/sys 2021-06-29 13:15:15 +02:00
Andrea Maria Piana 8ac498a5fe Fix go-waku version so it builds on ios 2021-06-24 12:32:31 +02:00
RichΛrd 40359f9c1b
go-waku integration (#2247)
* Adding wakunode module
* Adding wakuv2 fleet files
* Add waku fleets to update-fleet-config script
* Adding config items for waku v2
* Conditionally start waku v2 node depending on config
* Adapting common code to use go-waku
* Setting log level to info
* update dependencies
* update fleet config to use WakuNodes instead of BootNodes
* send and receive messages
* use hash returned when publishing a message
* add waku store protocol
* trigger signal after receiving store messages
* exclude linting rule SA1019 to check deprecated packages
2021-06-16 16:19:45 -04:00
RichΛrd 92032c7158
Community categories (#2228)
* create and edit community categories
* edit categories order
* adding category to chat
* Adding categories to json
2021-05-23 09:34:17 -04:00
Andrea Maria Piana d50fee6bb2 Handle connection state 2021-05-21 07:22:58 +02:00
Bitgamma 6a930ed0c6
accept a JSON string as chainID in signTypedData_v3 (#2226)
accept either a JSON number or string as chainID in signTypedData_v3/signTypedData_v4
2021-05-12 12:25:42 +03:00
Volodymyr Kozieiev 9241903edb
Refactorings: whisper transport removed (#2200) 2021-04-22 16:36:18 +03:00
Andrea Maria Piana ddc93981a7 Add V4 endpoints 2021-04-19 14:49:28 +02:00
Volodymyr Kozieiev 8a370c1c41
Request info about community from mailserver (#2177) 2021-04-19 15:09:46 +03:00
Shivek Khurana 203574c580
👤 Add random sessionId when API starts (#2187)
* Add random sessionId when API starts

* Fix lint

* Expose sessionId and created at
2021-04-12 19:16:11 +05:30
Shivek Khurana 1097b14a7f
🎭 📊 Anonymous Metrics V0 (#2170)
* Migrations in place, how to run them?

* Remove down migrations and touch database.go

* Database and Database Test package in place, added functions to get and store app metrics

* make generate output

* Minor bug fix on app metrics insert and select

* Add a validation layer to restrict what can be saved in the database

* Make validation more terse, throw error if schema doesn't exist, expose appmetrics service

* service updates

* Compute all errors before sending them out

* Trying to bring a closjure to appmetrics go

* Expose appmetrics via an api, skip fancy

* Address value as Jason Dawt Rawmasage to ease parsing

* Introduce a buffered chan with magic cap of 8 to minimize writes to DB. Tests for service and API. Also expose GetAppMetrics function.

* Lint issues

* Remove autoincrement, undo waku.json changes, fix error being shadowed, return nil where nil ought to be returned, get rid of buffered channel

* Bump migration number

* Fix API factory usage

* Add comment re:json.RawMessage instead of strings

* Get rid of test vars, throw save error inside the loop

* Update version

Co-authored-by: Samuel Hawksby-Robinson <samuel@samyoul.com>
2021-03-17 18:09:28 +05:30
Jakub Sokołowski fe8dab7391 go-ethereum: Drop throttling of inbound connections
This fix puts an end to a saga that essentially start during the
Status Prague Meetup at the end of October 2018. At the time we were
experiencing massive issues with `Connecting...` spinners in the app in the
venue we rented. We were pulling our hairs out what to do and we could not
find the cause of the issue at the time.

Three months later I deployed the following change:
https://github.com/status-im/infra-eth-cluster/commit/63a13eed

Which used `iptables` to map the `443` port onto our `30504` Status node port
using `PREROUTING` chain and `REDIRECT` jump in order to fix issues people
have been complaining about when using WiFi networks in various venues:
https://github.com/status-im/status-react/issues/6351

Our thinking when trying to resolve the reported issue assumed that some
networks might block outgoing connections on non-standard ports other than
the usual `80`(HTTP)/`443`(HTTPS) which would disrupt Status connectivity.
While this fix could have indeed helped a few edge cases, what it really
did was cause the Status node to stop seeing actual public IPs of the clients.

But __pure accident__ this change caused the code we inherited from
`go-ethereum` implementation of DevP2P protocol to stop throttling new
incoming connections, because the IP as which they appeared was a
`172.16.0.0/12` network address of the Docker bridge.

The `go-ethereum` code used the `!netutil.IsLAN(remoteIP)` check to
avoid throttling connections from local addresses, which included the
local Docker bridge address:
https://github.com/status-im/status-go/blob/82680830/vendor/github.com/ethereum/go-ethereum/p2p/netutil/net.go#L36

The fix intended to target a small number of networks with fortified
firewall configuration accidentally resolved our issues with
`Connecting...` prompts that our application showed us en masse during
our Prauge Meetup. Part of the reason for that is that venues like that
normally give out local IP addresses and use NAT to translate them onto
the only public IP address they possess.

Since out application is supposed to be usable from within networks
behind NAT like airport WiFi networks for example, it makes no sense to
keep the inbound connection throttle time implemented in `go-ethereum`.

I'm leaving `inboundThrottleTime` in because it's used to calculate
value for `dialHistoryExpiration` in:
`vendor/github.com/ethereum/go-ethereum/p2p/dial.go`

I believe reducing that value one we deploy this change should also
increase the speed with which the Status application is able to reconnect
to a node that was temporarily unavailable, instead waiting the 5*30 seconds.

Research issue: https://github.com/status-im/infra-eth-cluster/issues/35

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2021-02-16 20:38:47 +01:00
Andrea Maria Piana e5115d60f1 Upgrade pq library
There seems to be an issue with version 1.3, querying for topics on
postgres returns
and error:

```
panic: pq: invalid byte sequence for encoding "UTF8"
```

Upgrading pq fixes the issue

¯\_(ツ)_/¯
2021-01-27 09:23:11 +01:00
Andrea Maria Piana a1b3e3a772
Clean topics that we don't listen to
There was a bug on status-react where it would save filters that were
not listened to.
This commit adds a task to clean up those filters as they might result
in long syncing times.

This commit also returns topics/ranges/mailserves from messenger in
order to make the initialization of the app simpler and start moving
logic to status-go.

It also removes whisper from vendor.
2021-01-26 09:39:57 +01:00
Frank Tang 922e785512 Add GitHub to whitelist for URL unfurling #11426 2021-01-11 11:15:09 +01:00
Andrea Maria Piana 2695b70159 use own fork of resize 2020-12-17 14:10:00 +01:00
Samuel Hawksby-Robinson 0a5ce9d489 make generate for changes to identity_images sql 2020-12-17 14:10:00 +01:00
Samuel Hawksby-Robinson 6eb4484eef bindata for profile_image migrations 2020-12-17 14:10:00 +01:00
Samuel Hawksby-Robinson a0ea922017 make vendor 2020-12-17 14:10:00 +01:00
Samuel Hawksby-Robinson f93d09c2b6 make vendor 2020-12-17 14:10:00 +01:00
Samuel Hawksby-Robinson d77f85b7df Added file decoding
reused protocol file type detection
2020-12-17 14:10:00 +01:00
Samuel Hawksby-Robinson 17f0bac6ae Added basic code from https://github.com/status-im/image-resizer
Made some modifications to suit a general approach, more changes need to be applied
2020-12-17 14:10:00 +01:00
Volodymyr Kozieiev d4518d7f16
resend emoji reaction (#2088) 2020-12-15 16:43:41 +02:00
Andrea Maria Piana 6ad047d56f Fix datasync retry
This commit fixes a bug on the mvds library where the nextEpoch would be
incorrectly summed to the retry time, resulting in messages not being
retried, or retried much less frequently the longer the app was running.
It also updates the retry timing to backoff exponentially at multiple of
30 seconds.
2020-11-26 16:25:52 +01:00
Jonathan Rainville b2d8f7f416
feat: add bookmarks table for the browser bookmarks
plus get bookmark icon using a lib
2020-11-16 13:31:34 -05:00
Roman Volosovskyi 7467ca7b10
[markdown] Ignore status tags with upper case letters
related to https://github.com/status-im/status-react/issues/11317
2020-10-22 13:37:03 +03:00
Roman Volosovskyi a36144983a
Update markdown module 2020-10-19 13:48:03 +03:00
Andrea Maria Piana 9ddd963591
vendor 2020-07-30 20:20:56 +02:00
Samuel Hawksby-Robinson de79f2ced0
Fix bug, logger var name collided with package name 2020-07-30 20:20:35 +02:00
Samuel Hawksby-Robinson c8f9dad554
Feature/key compression (#1990)
## What has changed?

I've introduced to the public binding functionality that will compress and decompress public keys of a variety of encoding and key types. This functionality supports all major byte encoding formats and the following EC public key types:

- `secp256k1` pks
- `bls12-381 g1` pks
- `bls12-381 g2` pks

## Why make the change?

We want shorter public (chat) keys and we want to be future proof and encoding agnostic. See the issue here https://github.com/status-im/status-go/issues/1937

---

* Added basic signature for compresspk and uncompresspk

* Added basic encoding information

* make vendor

* formatted imports for the linter

* Reformatted imports hoping linter likes it

* This linter is capricious

* Added check that the secp256k1 key is valid

* Added test for valid key

* Added multiformat/go-varint dep

* Added public key type handling

* Added key decompression with key type handling

* Added handling for '0x' type indentifying

* Added more robust testing

* Less lint for the linting gods

* make vendor for bls12_381

* Added bls12-381 compression tests

* Added decompress key expected results

* Refactor of typed and untyped keys in tests

* Lint god appeasment

* Refactor of sample public keys

* Implemented bls12-381 decompression

* gofmt

* Renamed decode/encode funcs to be more descriptive

* Added binary bindings for key de/compression

* Refactor of func parameters

gomobile is a bit tempermental using raw bytes as a parameter, so I've decided to use string only inputs and outputs

* gofmt

* Added function documentation

* Moved multiformat de/compression into api/multiformat ns

* Moved multiformat de/compression into api/multiformat ns

* Changed compress to serialize on API
2020-06-23 11:47:17 +01:00
Adam Babik d7bb02540a
Update go-ethereum to v1.9.5-status.9 (#1898) 2020-03-12 13:05:12 +01:00
Adam Babik 831ab85561
Upgrade deps (#1892) 2020-03-09 08:55:58 +01:00
Andrea Maria Piana a82de9f5f9
Upgrade extkeys (#1878) 2020-03-04 09:30:31 +01:00
Andrea Maria Piana ddf7c2d1c2 upgrade whisper 2020-02-28 13:27:00 +01:00