Commit Graph

286 Commits

Author SHA1 Message Date
Roman Volosovskyi d21cd6aba1
[wallet] Set initial blocks range for a new empty account 2021-03-03 10:05:52 +02:00
Andrea Maria Piana 0a686bba4c Fix syncing of public chats
Timeline chats are created as public chats, in any place where we sync
them we need to check that they are excluded.
2021-03-03 08:57:48 +01:00
Volodymyr Kozieiev 09942bf200 Confirmation settings now correctly processed on mailserver 2021-03-01 11:23:17 +01:00
Andrea Maria Piana f115b8d289 Request/Decline access to communities 2021-02-26 15:35:43 +01:00
Roman Volosovskyi 99a304686f
Filter message PNs on status go side 2021-02-25 10:19:10 +02:00
Frank Tang 42ab47a9ee fix "Pending" status for messages in private group chat 2021-02-24 08:17:24 +01:00
Volodymyr Kozieiev c38439e664
Listen for delivered messages (#2150) 2021-02-23 17:47:45 +02:00
Andrea Maria Piana 2c0383ec22 Bump version to 0.71.4 2021-02-17 10:21:49 +01:00
Jakub Sokołowski 2ae1931e6c bump version to 0.71.3
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2021-02-16 20:38:47 +01:00
Andrea Maria Piana 31b9a924ce Fix image messages being dropped in profile chats
It looks like profile chats are created as public chats, and the code
will drop messages for it.

This commit fixes the issue by checking for the "@" prefix in profile
chat names, until we fix the issue by migrating those chats.
2021-02-15 20:07:41 +01:00
Andrea Maria Piana 6ac1c43926 Add status.im and medium.com to link unfurling 2021-02-15 11:51:14 +01:00
Andrea Maria Piana add5040b48 Drop image messages in public chats
Currently image messages are actually displayed, we only prevent
sending.
This commit changes the behavior so that we drop them.
2021-02-15 08:47:41 +01:00
Roman Volosovskyi 8268083008
[wallet] Merge blocks ranges when possible
- old existing ranges are merged when wallet service is started
- a new range is merged with an existing one if possible

This will decrease the number of entries in blocks_range table as
currently it can grow indefinitely (@flexsurfer reported 23307 entries).

This change is also needed for further optimisations of RPC usage.
2021-02-09 16:32:20 +02:00
Andrea Maria Piana d862b042ae
Bump version to 0.70.0 2021-02-05 16:48:47 +02:00
Roman Volosovskyi e18050b87f
Make sure tokens balances can be fetched 2021-02-03 11:54:24 +02:00
Andrea Maria Piana ee8f333dbf Avoid concurrent access to contacts
We were not locking before accessing the contacts map and it would panic
in some cases.

I have changed the code to pull contacts from db so we move away from
having locks.
2021-01-27 10:54:52 +01:00
Andrea Maria Piana e5115d60f1 Upgrade pq library
There seems to be an issue with version 1.3, querying for topics on
postgres returns
and error:

```
panic: pq: invalid byte sequence for encoding "UTF8"
```

Upgrading pq fixes the issue

¯\_(ツ)_/¯
2021-01-27 09:23:11 +01:00
Andrea Maria Piana a1b3e3a772
Clean topics that we don't listen to
There was a bug on status-react where it would save filters that were
not listened to.
This commit adds a task to clean up those filters as they might result
in long syncing times.

This commit also returns topics/ranges/mailserves from messenger in
order to make the initialization of the app simpler and start moving
logic to status-go.

It also removes whisper from vendor.
2021-01-26 09:39:57 +01:00
Andrea Maria Piana fb6411af24
Use personal topic for push notification registration
One of the issues we noticed is that the partitioned topic
in push notification is heavy in traffic, as any user using a particular
mailserver will use that partitioned topic to register for PNs.

This commit moves from the partitioned topic to the personal topic of
the PN server, so it does not clash with other users that might happen
to have the same partitioned topic as the mailserver, resulting in long
sync times.

Another issue that will need to be addressed separately is that once you
send a message to a topic, because of the way how waku/whisper works,
you will have to register to that topic, meaning that you will receive
that data. Currently waku does not support unsubscribing from a topic
without logging in and out, so that needs also to be addressed.
2021-01-26 09:39:53 +01:00
Roman Volosovskyi d8bccaff18
[wallet] Detect non archival RPC node 2021-01-25 14:52:14 +02:00
Andrea Maria Piana 2427b3c0a5 Pass topics to mailserver cursor
The topics field was not passed to the mailserver, which meant that
queries were still using the old bloom filter.
Hopefully this is the last place where we need to pass this.
2021-01-25 11:52:37 +01:00
Andrea Maria Piana e3969a7752 Cache waku messages
This commit re-introduces a feature that we lost during the migration to
status-go.
Messages are cached for a couple of days if processed correctly by
status-go, to avoid performance issues.
2021-01-18 09:38:27 +01:00
Andrea Maria Piana b5b1e19c24 Add ProfilePicturesVisibility 2021-01-15 13:39:13 +01:00
Roman Volosovskyi 79716227db
[wallet] Show PNs right after enabling 2021-01-13 12:33:09 +02:00
Samuel Hawksby-Robinson 46157dc4dc
Expand Local Notifications to support multiple Notification types (#2100)
* Initial work on expanding Local Notifications

Adding functionality to support multiple notification types in Notification.Body. Currently have a bug that I think is caused by a the jsonMarshal func not working as intented, need to resolve this next before proceeding

* Fixed json.Marshaller issue and implemented json.Unmarshaller

* Tweak errors, go convention is errors don't begin with capital letters

* Added notificationMessageBody with un/marshalling

Also removed the Body interface

* Added check for bodyType mismatch

* Implement building and sending new message notifications

* Refactor to remove cycle imports

* Resolved linting issue ... Hopefully

* Resolving an implicit memory aliasing in a for loop

* version bump

* Added Notification.Category consts
2021-01-12 14:28:27 +00:00
Andrea Maria Piana b331b61807 Add ClearHistory & DeactivateChat methods 2021-01-11 13:36:05 +01:00
Frank Tang 922e785512 Add GitHub to whitelist for URL unfurling #11426 2021-01-11 11:15:09 +01:00
Andrea Maria Piana e1465ca890 Bump version to 0.67.2 2021-01-11 09:01:26 +01:00
Andrea Maria Piana 6f088bb5c5 Fix communities migration
In some instances the communities migration would be skipped but not
marked as `dirty`.

This commit addresses the issue by:
- Making sure that if dirty is set the migration is not skipped but
replayed
- If the version is on the communities migration and dirty is false, we
check for the presence of the communities table. If not present we
replay the communities migration.
- Make community_id field in user_messages nullable

It also removes all the `down` migration, as we can't use them
effectively, as explained in the README.md added.
2021-01-08 08:43:16 +01:00
Andrea Maria Piana a0d5f66c8f Leave import community data 2020-12-23 17:20:55 +01:00
Andrea Maria Piana fee08aafbe bump version to 0.66.2 2020-12-23 09:32:28 +01:00
Roman Volosovskyi bf703254ba
[wallet] Dispatch event on tx history fetching failure 2020-12-21 15:45:10 +02:00
Eric Mastro 149877a939
feat: allow getPreviewLinkData for all whitelisted links (#2094)
* feat: allow getPreviewLinkData for all links

Only YouTube links were supported in `getPreviewLinkData` previously.

Now, any whitelisted links can have their data retreived using this function, returning the true content type (by examining the header bytes of the content).

feat: add tenor.com and giphy.com to whitelisted urls list

* Fix json format

* bump VERSION

* Lint urls.go

Co-authored-by: Andrea Maria Piana <andrea.maria.piana@gmail.com>
2020-12-18 19:33:24 +11:00
Andrea Maria Piana 523d4a7156 bump version to 0.65.0 2020-12-17 14:10:00 +01:00
Andrea Maria Piana 50b17308bd Bump version to 0.64.8 2020-12-15 16:35:47 +01:00
Volodymyr Kozieiev d4518d7f16
resend emoji reaction (#2088) 2020-12-15 16:43:41 +02:00
Roman Volosovskyi d39ca7fea4
[wallet] Fix multiple notifications on a single erc20 transfer 2020-12-11 16:47:38 +02:00
Andrea Maria Piana d65946e9c0 Use IsOwnAccount instead of Wallet when watching for transactions
There was an issue in using the `Wallet` flag when checking accounts to
watch for transactions.
`Wallet` indicates that it's the default wallet, not whether is a wallet
account.
That can only be checked by looking at the type (and the `Wallet` flag).
If the type is `generated`, `key` or `seed` it should be watched for
transactions.
2020-12-07 16:12:50 +01:00
Andrea Maria Piana 0b2bd2863b Log mailserver error
while debugging I noticed we don't log `err` if the query fails.
2020-12-07 15:03:06 +01:00
Andrea Maria Piana 156c0de832 Add SendChatMessages endpoints
This commit adds an endpoint to batch the sending of messages.
This is useful to simplify client logic when sending a batch of messages
and ensuring the correct order in the message stream.

It currently implements only what's needed, and naively return an error
if any of the messages fail.
2020-12-03 18:25:53 +01:00
Andrea Maria Piana 0304b3fa46 [Fixes: #2068] Pass topics for mailserver
We were not actually passing the topics in the request, therefore we
were using bloom filter for query, which resulted in long syncing times
for some users.
2020-11-30 12:03:27 +01:00
Andrea Maria Piana d8307a60cf Bump version to 0.64.1 2020-11-26 16:25:52 +01:00
Roman Volosovskyi 002f9a5597
[wallet] Reduce number of RPC requests
- Wallet service is not started on foreground event on status-go side
  anymore, it leaves a client side opportunity to decide whether new
  blocks should be watched.
- `watchNewBlocks` parameter is added to `StartWallet`.
- Some requests are removed/moved to the place where they are necessary.
2020-11-25 12:20:23 +02:00
andrey a4195e5b5c custom nonce 2020-11-24 12:20:57 +01:00
Andrea Maria Piana 9814a9a5e7 Dont crash if to or from are nil 2020-11-19 08:14:51 +01:00
Andrea Maria Piana 70414c90b9 Fix nil pointer local notifications 2020-11-17 17:07:43 +01:00
Jonathan Rainville b2d8f7f416
feat: add bookmarks table for the browser bookmarks
plus get bookmark icon using a lib
2020-11-16 13:31:34 -05:00
flexsurfer e8dbc66227
timeline reactions (#2079) 2020-11-16 12:54:39 +01:00
Roman Volosovskyi 4026841dc1
Add `New` flag to a received message
This change allows to count unviewed messages properly on the client
side when replied message is attached to reply before being passed.
2020-11-10 18:02:38 +02:00
Roman Volosovskyi d20cf2583f
Revert "Allow receiving old messages after rejoining a pubchat"
This reverts commit fa136ebaa7.
2020-11-06 17:09:49 +02:00
andrey 40e2fc7526 fix timeline 2020-11-06 12:31:14 +01:00
andrey 6f207def2b timeline 2020-11-06 06:35:35 +01:00
Roman Volosovskyi fa136ebaa7
Allow receiving old messages after rejoining a pubchat 2020-11-05 15:13:38 +02:00
Andrea Maria Piana 75e0809f50 Split datasync messages in batches
When sending messages in quick succession, it might be that multiple
messages are batched together in datasync, resulting in a single large
payload.
This commit changes the behavior so that we can pass a max-message-size
and we split the message in batches before sending.

A more elegant way would be to split at the transport layer (i.e
waku/whisper), but that would be incompatible with older client.

We can still do that eventually to support larger messages.
2020-11-05 12:45:29 +01:00
Andrea Maria Piana 86e0ec8e10 Log disconnect/connect event
Marketing was relying on mailserver entries for checking the time a
give peer spent on the app.

This was not accurate as the assumption was that a peer would "ping" a
mailserver every 15s, which is not the case, it only hit the mailserver
as it comes from an "offline" state.

This commit changes the log level of two entries so that we have
connect/disconnect time for a given peer, which should be enough to
calculate roughly the time a peer has been online if connected to our
fleet.
2020-11-03 15:58:26 +01:00
Andrea Maria Piana 40343b3140 Remove unused onboarding code
Fixes: #1539
Fixes: #1538

Unused code from onboarding
2020-11-03 11:18:05 +01:00
Andrea Maria Piana 3f63b0c23c Check that server is nil before calling peers count
On logout happens sometimes that `PeersCount` is called when the server
has been removed.
This commit adds a guard to make sure that the server is not nil when
calling `PeersCount`.
2020-11-03 07:21:52 +01:00
Volodymyr Kozieiev 9dffff042e Don't send new members event when there are no new members 2020-11-02 17:05:56 +01:00
Gheorghe Pinzaru d04e54e54e
Local notifications service (#2026)
Also adds implementation for eth transactions notifications
2020-10-28 10:56:14 +03:00
Volodymyr Kozieiev 3e446eed8c
Link previews support (#2059) 2020-10-27 19:35:28 +02:00
andrey b388002771 profile status updates 2020-10-27 14:56:35 +01:00
Roman Volosovskyi 7467ca7b10
[markdown] Ignore status tags with upper case letters
related to https://github.com/status-im/status-react/issues/11317
2020-10-22 13:37:03 +03:00
Roman Volosovskyi a36144983a
Update markdown module 2020-10-19 13:48:03 +03:00
Andrea Maria Piana 4ecca1169d Add use-mailservers flag 2020-10-14 10:23:57 +02:00
Andrea Maria Piana 1a7a63a8e5 Log on db failure 2020-10-14 08:07:05 +02:00
Andrea Maria Piana 34c41d4bc8 bump version 2020-10-08 13:23:00 +02:00
Andrea Maria Piana efbf093bc4 Don't fail if one request fails in getTokensBalances
If one request failed, the whole batch would fail.
This caused issue as one of the contract is constantly returning an
error now, and essentially there was not way to fetch balance.
Also extend the timeout to 20s as we throw 165 request to Infura in one
go and it takes its time to reply to those, although it seems like we
should batch them on our side instead of sending them all cuncurrently.
2020-10-05 12:53:51 +02:00
Andrea Maria Piana 896f5c24bf Force migration if dirty 2020-10-01 13:47:59 +02:00
Andrea Maria Piana 2aea867cee Migrate journals as well 2020-09-30 12:13:22 +02:00
Andrea Maria Piana 200273aa0e Count runes and not characters 2020-09-25 14:02:09 +02:00
Andrea Maria Piana 0ba4694f21 [Fix: #2027] Migrate db to shorter path name 2020-09-25 14:02:09 +02:00
andrey 397ee840c6 [#2042] Wrong ERC20 decimal rounding in "Set max" 2020-09-22 16:00:47 +02:00
andrey 682722b973 add wallet favourites 2020-09-21 10:48:00 +02:00
Andrea Maria Piana 36e742cb70 Bump version to 0.60.1 2020-09-17 12:10:26 +02:00
Gheorghe Pinzaru 9e148eab89
Dont allow disabling waku (#2040) 2020-09-16 10:31:01 +03:00
Volodymyr Kozieiev 8f83c5f462
Add `allow permission request` field to settings (#2035)
Add settings field webview-allow-permission-requests?
2020-09-10 13:22:30 +03:00
Andrea Maria Piana 61b345ff33 Add index on seen & hide long messages
This commit does two things:
1) Add an index on seen & update only the not-seen messages in the query
2) Hide long messages in the database, as that's likely spam
2020-09-10 10:26:59 +02:00
Andrea Maria Piana 3c6b741f71
Add tests for send notification & review feedback 2020-09-09 21:22:34 +02:00
Roman Volosovskyi 57728224d4
[mentions] ParsedText in quoted message
ParsedText is needed to properly display mention in quoted message.
2020-09-09 16:07:31 +03:00
Andrea Maria Piana 32b0af8ecc Add index to emoji reactions 2020-09-09 15:02:54 +02:00
andrey 3b748a2e46 group chat invitation 2020-09-07 12:15:58 +02:00
Volodymyr Kozieiev 88a3022ea8
Drop messages with text longer than 4096 characters (#2029) 2020-09-01 17:38:36 +03:00
Andrea Maria Piana 401d18e558
Enabled push notifications by default and add fleet 2020-08-27 18:54:06 +02:00
Andrea Maria Piana 4e9928ab5f Don't log chat as it makes status-desktop blow up
For some reason when calling saveChat from desktop with `lastMessage`
set to null, a sigsev is received.
The issue seems to be in logFormat
05280a7ae3/log/format.go (L356)
which for some reason blows up if passed a nil pointer (`lastMessage`).
Can't replicate on any other platform or running it locally, but hey,
this fixes the issue.
2020-08-27 17:27:01 +02:00
Andrea Maria Piana 56c0142f16 Honor FullNode config in waku
previously FullNode would only result in setting a bloom filter to full.

This behavior caused issues as you effectively cannot install a filter
on a FullNode, as it would advertise the new topic/bloom filter and stop
receiving all the messages.

This caused an issue when running push notifications servers together
with mailservers, also the behavior is a bit counter-intuitive as I
would expect the FullNode config to be honored no matter of what filters
are installed.
2020-08-26 14:11:13 +02:00
Andrea Maria Piana 2b2616f82c
Bump version to 0.57.0 2020-08-18 16:29:55 +02:00
Andrea Maria Piana 435eacecb5 Handle wallet initialization
StartWallet was called before service initialization.
After the recent changes this call was moved after initialization, but
the geth system automatically start services.
This meant that `IsStarted()` returned true, although the reactor was
not started, and only after calling `StopWallet()` and `StartWallet()`
again the system would reach the right state.

This commit changes the behavior so that we only check whether the
reactor has been started when calling `IsStarted()` and we allow
multiple calls to `Start()` on the signal service, which won't return an
error (it's a noop if callled multiple times).
2020-08-18 14:13:08 +02:00
Andrea Maria Piana 2e231e690e Fix mark messages seen
If a chat had a last message the persistence call `Chat()` would not
decode `LastMessage` correctly.
This commit fixes the issue.
2020-08-17 13:40:10 +02:00
Andrea Maria Piana 7319384baf Bump version to 0.57.0 2020-08-17 12:53:37 +02:00
Andrea Maria Piana 2d0818d873 Add SignMessageWithChatKey 2020-07-31 14:51:24 +02:00
Andrea Maria Piana ab01a05cd6 Don't use bytes anymore for LastMessage
LastMessage in chat was encoded in bytes so that we don't have to
encoded/decode everytime we save to db or pass the client.

An issue with emoji surfaced a problem with this approach.
Chat.LastClockValue represent the last clock value of any type of
message exchanged in a chat (emoji,group membership updates, contact
updates).
So when receving a new message, we should update LastMessage if the
clock of the LastMessage is lower than the received message, and we
should not only check LastClockValue, otherwise the message might be
discarded although it is the most recent.

This commit fixes the issue by keeping LastMessage as an object and
comparing LastMessage.Clock instead of LastClockValue
2020-07-31 13:40:11 +02:00
Andrea Maria Piana b346d08d91
bump version 2020-07-30 20:25:06 +02:00
Andrea Maria Piana eadf68325e Fix audio null value
If a message was inserted before the migration the field
audio_duration_ms would be set to NULL, and would not be serialized into
go correctly, as uint is non-nullable.
this commit fixes the issue by calling COALESCE on the value.
2020-07-30 15:00:39 +02:00
Andrea Maria Piana ec39bef088 Bump version to 0.56.2 2020-07-29 12:51:48 +02:00
Andrea Maria Piana 45815e57d7
Add feature flag for push notifications and bump version 2020-07-27 17:20:16 +02:00
Andrea Maria Piana c61bf0cd9c
bump version 2020-07-27 10:39:27 +02:00
Andrea Maria Piana 58fcf809ea
Bug/fix pending peers segfault (#2004)
* Replace mclock with time in peer pools

we used mclock as golang before 1.9 did not support monotonic clocks,
https://github.com/gavv/monotime, it does now https://golang.org/pkg/time/
so we can fallback on the system implementation, which will return
nanoseconds with a resolution that is system dependent.

* Handle case where peer have same discovered time

In some system the resolution of the clock is not high enough so
multiple peers are added on the same nanosecond.

This result in the peer just added being immediately removed.

This code adds a check making sure we don't assume that a peer is added.

Another approach would be to make sure to include the peer in the list,
so prevent the peer just being added to be evicted, but it's slightly
more complicated and the resolution is generally accurate enough for our
purpose so that peers will be fresh enough if they have the same
discovered time.

It also adds a regression test, I had to use an interface to stub the
clock.

Fixes: https://github.com/status-im/nim-status-client/issues/522

* bump version to 0.55.3
2020-07-24 11:14:05 -04:00
Roman Volosovskyi 0b3cdf7362
DeleteMultiaccount method 2020-07-13 16:20:02 +03:00
Roman Volosovskyi 6d5a93287b
Separate keystore directory per multiacc
This commit provides `MigrateKeyStoreDir` method which moves multiacc's
keys from a common keystore directory to multiacc specific directory.
2020-07-09 12:41:47 +03:00
Andrea Maria Piana 2d12ac4fbb Rename rate limiting to packet rate limiting 2020-06-15 19:46:23 +02:00