Commit Graph

416 Commits

Author SHA1 Message Date
Andrea Maria Piana fb6411af24
Use personal topic for push notification registration
One of the issues we noticed is that the partitioned topic
in push notification is heavy in traffic, as any user using a particular
mailserver will use that partitioned topic to register for PNs.

This commit moves from the partitioned topic to the personal topic of
the PN server, so it does not clash with other users that might happen
to have the same partitioned topic as the mailserver, resulting in long
sync times.

Another issue that will need to be addressed separately is that once you
send a message to a topic, because of the way how waku/whisper works,
you will have to register to that topic, meaning that you will receive
that data. Currently waku does not support unsubscribing from a topic
without logging in and out, so that needs also to be addressed.
2021-01-26 09:39:53 +01:00
Andrea Maria Piana 9d2117e227
Send abridged version of history with messages in group chats
When sending a message in a private group chat we send the whole history
for redundancy and allow out-of-order processing.
This can be very expensive in some chats, resulting in long delay when
sending a message and calculating the POW.
This commit improves the performance  by only forwarding the events
necessary for the user to be able to construct a group chat and process
correctly the message.
2021-01-26 09:39:52 +01:00
Andrea Maria Piana d7772d3fc6
Skip wrapping emojis in private group chats
This commit fixes a couple of issues:
1) Emojis were sent to any member of the group chat, regardless of
whether they joined
2) We don't want to wrap emojis, as there's no need to do so, only
messages are to be wrapped
2021-01-26 09:39:47 +01:00
Volodymyr Kozieiev 31f2196cb2 Fix segmentation failure on mailserver 2021-01-26 08:28:24 +01:00
Andrea Maria Piana e3969a7752 Cache waku messages
This commit re-introduces a feature that we lost during the migration to
status-go.
Messages are cached for a couple of days if processed correctly by
status-go, to avoid performance issues.
2021-01-18 09:38:27 +01:00
Roman Volosovskyi fbec17af18 fix protocol.MessageNotificationBody marshalling 2021-01-15 14:00:51 +01:00
Andrea Maria Piana b5b1e19c24 Add ProfilePicturesVisibility 2021-01-15 13:39:13 +01:00
Andrea Maria Piana 665bc3743c Fix push notification flaky test 2021-01-15 13:02:10 +01:00
Samuel Hawksby-Robinson 46157dc4dc
Expand Local Notifications to support multiple Notification types (#2100)
* Initial work on expanding Local Notifications

Adding functionality to support multiple notification types in Notification.Body. Currently have a bug that I think is caused by a the jsonMarshal func not working as intented, need to resolve this next before proceeding

* Fixed json.Marshaller issue and implemented json.Unmarshaller

* Tweak errors, go convention is errors don't begin with capital letters

* Added notificationMessageBody with un/marshalling

Also removed the Body interface

* Added check for bodyType mismatch

* Implement building and sending new message notifications

* Refactor to remove cycle imports

* Resolved linting issue ... Hopefully

* Resolving an implicit memory aliasing in a for loop

* version bump

* Added Notification.Category consts
2021-01-12 14:28:27 +00:00
Andrea Maria Piana b331b61807 Add ClearHistory & DeactivateChat methods 2021-01-11 13:36:05 +01:00
Frank Tang 922e785512 Add GitHub to whitelist for URL unfurling #11426 2021-01-11 11:15:09 +01:00
Andrea Maria Piana 4084c9d82c Don't resend automatically group chat messages
Since we fixed re-sending of messages, it has been noticed a performance
degradation in private group chats.
This is due to too aggressive re-sending of messages.
This commit disables resending of private group chat messages for now
(same as 1.9, so no changes), but keeps it enabled for 1-to-1s.
2021-01-11 09:01:26 +01:00
Andrea Maria Piana cd21f9b0e2 Fix retry count in push notifications
In some instances the retry mechanism would get into a busy loop.
That's due to the fact that we would fetch some non-retriable
notifications but not act on them.

This commit fixes the issue by filtering them from the database query
and making sure that we at least wait 1 second.
2021-01-11 09:01:26 +01:00
Andrea Maria Piana 6f088bb5c5 Fix communities migration
In some instances the communities migration would be skipped but not
marked as `dirty`.

This commit addresses the issue by:
- Making sure that if dirty is set the migration is not skipped but
replayed
- If the version is on the communities migration and dirty is false, we
check for the presence of the communities table. If not present we
replay the communities migration.
- Make community_id field in user_messages nullable

It also removes all the `down` migration, as we can't use them
effectively, as explained in the README.md added.
2021-01-08 08:43:16 +01:00
Andrea Maria Piana 7387049d4b Upgrade linter and address issues 2020-12-28 16:55:14 +01:00
Andrea Maria Piana f5482ec187 Remove filters 2020-12-23 17:20:55 +01:00
Andrea Maria Piana 06dcf0cea9 Allow emoji reactions 2020-12-23 17:20:55 +01:00
Andrea Maria Piana f3afa962c6 Delete messages from chat when leaving org 2020-12-23 17:20:55 +01:00
Andrea Maria Piana b4905fdbd4 address feedback 2020-12-23 17:20:55 +01:00
Andrea Maria Piana 3be6d86326 Remove user from community 2020-12-23 17:20:55 +01:00
Andrea Maria Piana a0d5f66c8f Leave import community data 2020-12-23 17:20:55 +01:00
Andrea Maria Piana 5d929c3584 Handle organisation membership requests 2020-12-23 17:20:55 +01:00
Andrea Maria Piana 84e91d85d2 Consolidate test methods 2020-12-22 15:43:48 +01:00
Andrea Maria Piana 4185420897 Send image on update
This commit changes the behavior so that when the image is updated it
will be published on the contact code topic.
If that does not happen because we are offline, it will be scheduled for
the next time we are online.
2020-12-22 15:43:48 +01:00
Eric Mastro 149877a939
feat: allow getPreviewLinkData for all whitelisted links (#2094)
* feat: allow getPreviewLinkData for all links

Only YouTube links were supported in `getPreviewLinkData` previously.

Now, any whitelisted links can have their data retreived using this function, returning the true content type (by examining the header bytes of the content).

feat: add tenor.com and giphy.com to whitelisted urls list

* Fix json format

* bump VERSION

* Lint urls.go

Co-authored-by: Andrea Maria Piana <andrea.maria.piana@gmail.com>
2020-12-18 19:33:24 +11:00
Andrea Maria Piana cdca42b90f Dont lock on envelope sent 2020-12-17 14:10:00 +01:00
Samuel Hawksby-Robinson b3187abce1 Fix for int64 pointer bug 2020-12-17 14:10:00 +01:00
Samuel Hawksby-Robinson ba559c5291 Handled errors from defer rows.Close() and tx.Commit() 2020-12-17 14:10:00 +01:00
Samuel Hawksby-Robinson 0db7a23d5f Added chatContext type and set chatContext consts 2020-12-17 14:10:00 +01:00
Andrea Maria Piana 0a93513531 Add images field to HasCustomFields 2020-12-17 14:10:00 +01:00
Andrea Maria Piana ac37e4f412 Add multiaccount to push notification tests 2020-12-17 14:10:00 +01:00
Samuel Hawksby-Robinson e0eb96a992 Fix failing tests due to account not present in test env 2020-12-17 14:10:00 +01:00
Samuel Hawksby-Robinson ca80140c1a Resolved broken tests 2020-12-17 14:10:00 +01:00
Andrea Maria Piana 31a885ced8 remove photo path in favor of images in contact 2020-12-17 14:10:00 +01:00
Andrea Maria Piana 1adfa38611 Check hash in shouldPublish 2020-12-17 14:10:00 +01:00
Andrea Maria Piana b7a6bbf1e7 dont send if empty images 2020-12-17 14:10:00 +01:00
Andrea Maria Piana 69de77c36a return error on nil image 2020-12-17 14:10:00 +01:00
Samuel Hawksby-Robinson 16fecf5520 Passing Master info to Messenger 2020-12-17 14:10:00 +01:00
Samuel Hawksby-Robinson 0b6061b444 Added ChatIdentity to HandleApplication() 2020-12-17 14:10:00 +01:00
Samuel Hawksby-Robinson 279c395f23 Added heavy logging to the chat identity builder 2020-12-17 14:10:00 +01:00
Samuel Hawksby-Robinson fe9e04ca13 Updating debug lvl to info 2020-12-17 14:10:00 +01:00
Samuel Hawksby-Robinson 481c9c1287 Added debug to HandleChatIdentity 2020-12-17 14:10:00 +01:00
Samuel Hawksby-Robinson 0df825164c Placating the linting old ones 2020-12-17 14:10:00 +01:00
Samuel Hawksby-Robinson 5d0906b9d8 Resolving fix for receiving chat identities with empty images map 2020-12-17 14:10:00 +01:00
Samuel Hawksby-Robinson 43b8e94b2b Resolved an additional fail test related to multiaccounts in the Messenger 2020-12-17 14:10:00 +01:00
Samuel Hawksby-Robinson 0c1ebaa547 Resolving goimports lint ... :/ 2020-12-17 14:10:00 +01:00
Samuel Hawksby-Robinson 5aa8492315 Fix failing tests 2020-12-17 14:10:00 +01:00
Andrea Maria Piana 32907b59b2 Fix linting 2020-12-17 14:10:00 +01:00
Samuel Hawksby-Robinson 1b527906ee Resolved hopefully the last of the lint 2020-12-17 14:10:00 +01:00
Samuel Hawksby-Robinson 7b2d96de05 Resolved yet more lint 2020-12-17 14:10:00 +01:00
Samuel Hawksby-Robinson b7a7035894 Resolved further linting issues 2020-12-17 14:10:00 +01:00
Samuel Hawksby-Robinson a77a40fdf4 Resolved linting issues 2020-12-17 14:10:00 +01:00
Samuel Hawksby-Robinson 190b1a70f9 Removed usages of multiaccounts.Account.PhotoPath 2020-12-17 14:10:00 +01:00
Samuel Hawksby-Robinson 193ab30ada Refactor injecting multiaccounts into messenger and service 2020-12-17 14:10:00 +01:00
Samuel Hawksby-Robinson 7b230d5c9d Updated message handling from public and private chat 2020-12-17 14:10:00 +01:00
Samuel Hawksby-Robinson 056649fd73 Changed identity image migration 2020-12-17 14:10:00 +01:00
Samuel Hawksby-Robinson a564066c4f Added public chat ChatIdentity publish 2020-12-17 14:10:00 +01:00
Samuel Hawksby-Robinson 48a34072bb Added ChatIdentity attachment to ContactCodeAdvert 2020-12-17 14:10:00 +01:00
Samuel Hawksby-Robinson 1f1e784d80 Added additional registry testing 2020-12-17 14:10:00 +01:00
Samuel Hawksby-Robinson 7b77ae9eab Fix CI register persistence bug 2020-12-17 14:10:00 +01:00
Samuel Hawksby-Robinson 5426fac0a3 Added CHAT_IDENTITY application meta type
also make generate
2020-12-17 14:10:00 +01:00
Samuel Hawksby-Robinson fe4b7c6d80 Added migrations for chat_identity_last_published register 2020-12-17 14:10:00 +01:00
Samuel Hawksby-Robinson af7117070f Added protocol register of ChatIdentity last published times per chat
This register is only used for publishing ChatIdentity to public chat
2020-12-17 14:10:00 +01:00
Samuel Hawksby-Robinson 1b59d088d8 Integrated GetMimeType() 2020-12-17 14:10:00 +01:00
Samuel Hawksby-Robinson 5a4e23a0ea Fix protobuf ImageType change 2020-12-17 14:10:00 +01:00
Samuel Hawksby-Robinson e8d883edb6 bindata for profile_image migrations 2020-12-17 14:10:00 +01:00
Samuel Hawksby-Robinson 6bab64cc9b Fix getImageMessageMIME() bug 2020-12-17 14:10:00 +01:00
Samuel Hawksby-Robinson 7f8d3674d4 Removing superfluous image types
Additionally updating the protobufs that use ImageType
2020-12-17 14:10:00 +01:00
Samuel Hawksby-Robinson c6accd8ce3 make generate 2020-12-17 14:10:00 +01:00
Samuel Hawksby-Robinson d77f85b7df Added file decoding
reused protocol file type detection
2020-12-17 14:10:00 +01:00
Samuel Hawksby-Robinson d7eac44c77 Updated comment on IdentityImage mapping 2020-12-17 14:10:00 +01:00
Samuel Hawksby-Robinson 5043c1d1af Updated ImageType to handle SVG and AVATAR image types 2020-12-17 14:10:00 +01:00
Samuel Hawksby-Robinson 6c4eea2a6b Moved ImageType enum to enums.proto 2020-12-17 14:10:00 +01:00
Samuel Hawksby-Robinson f1f409c209 Added basic chat identity protobuf 2020-12-17 14:10:00 +01:00
Andrea Maria Piana f3f6cdcd93 Mark message as seen when hiding
When hiding a message, we want to mark it as seen in order to preserve
the right count
2020-12-15 15:47:02 +01:00
Volodymyr Kozieiev d4518d7f16
resend emoji reaction (#2088) 2020-12-15 16:43:41 +02:00
Andrea Maria Piana 14f4c40404 Split Merge into functions 2020-12-03 18:25:53 +01:00
Andrea Maria Piana 156c0de832 Add SendChatMessages endpoints
This commit adds an endpoint to batch the sending of messages.
This is useful to simplify client logic when sending a batch of messages
and ensuring the correct order in the message stream.

It currently implements only what's needed, and naively return an error
if any of the messages fail.
2020-12-03 18:25:53 +01:00
Andrea Maria Piana 938e0d77dd Don't return an error on datasync Send
if an error is returned on the Send function, datasync will keep
  retrying a message at each epoch. If the message cannot be sent (for
  example is too large), then no messages will be sent until logout.
2020-11-26 16:25:52 +01:00
Andrea Maria Piana 0c79e50ca8 Remove unnecessary logs & fix flaky test 2020-11-26 16:25:52 +01:00
Andrea Maria Piana 6ad047d56f Fix datasync retry
This commit fixes a bug on the mvds library where the nextEpoch would be
incorrectly summed to the retry time, resulting in messages not being
retried, or retried much less frequently the longer the app was running.
It also updates the retry timing to backoff exponentially at multiple of
30 seconds.
2020-11-26 16:25:52 +01:00
Jonathan Rainville b2d8f7f416
feat: add bookmarks table for the browser bookmarks
plus get bookmark icon using a lib
2020-11-16 13:31:34 -05:00
flexsurfer e8dbc66227
timeline reactions (#2079) 2020-11-16 12:54:39 +01:00
Roman Volosovskyi 4026841dc1
Add `New` flag to a received message
This change allows to count unviewed messages properly on the client
side when replied message is attached to reply before being passed.
2020-11-10 18:02:38 +02:00
Andrea Maria Piana f10e70ce96 Incerase verified count when ENS name does not verify
In some cases no error is returned but the name is not verified.
Before this ENS names would get verified again every 30s.
This commit changes the logic so that if they fail to verify without
error they will still exponentially backoff.
2020-11-10 08:11:39 +01:00
Roman Volosovskyi d20cf2583f
Revert "Allow receiving old messages after rejoining a pubchat"
This reverts commit fa136ebaa7.
2020-11-06 17:09:49 +02:00
andrey 40e2fc7526 fix timeline 2020-11-06 12:31:14 +01:00
Andrea Maria Piana 8966d81199 linting 2020-11-06 06:35:35 +01:00
andrey 6f207def2b timeline 2020-11-06 06:35:35 +01:00
Roman Volosovskyi fa136ebaa7
Allow receiving old messages after rejoining a pubchat 2020-11-05 15:13:38 +02:00
Andrea Maria Piana 75e0809f50 Split datasync messages in batches
When sending messages in quick succession, it might be that multiple
messages are batched together in datasync, resulting in a single large
payload.
This commit changes the behavior so that we can pass a max-message-size
and we split the message in batches before sending.

A more elegant way would be to split at the transport layer (i.e
waku/whisper), but that would be incompatible with older client.

We can still do that eventually to support larger messages.
2020-11-05 12:45:29 +01:00
Volodymyr Kozieiev 9dffff042e Don't send new members event when there are no new members 2020-11-02 17:05:56 +01:00
Volodymyr Kozieiev 3e446eed8c
Link previews support (#2059) 2020-10-27 19:35:28 +02:00
andrey b388002771 profile status updates 2020-10-27 14:56:35 +01:00
Andrea Maria Piana 5cea355541 Use in memory database for messenger tests 2020-10-14 08:37:23 +02:00
Andrea Maria Piana b11399ffc6 Use ErrRecordNotFound instead of sql.ErrNoRows
We were checking for the wrong error kind when pulling messages from the
database, which resulted in the code not retrying to pull the message,
giving flaky tests / race condition (that's present in production as
well)
2020-10-08 13:23:00 +02:00
Andrea Maria Piana 896f5c24bf Force migration if dirty 2020-10-01 13:47:59 +02:00
Andrea Maria Piana 200273aa0e Count runes and not characters 2020-09-25 14:02:09 +02:00
Andrea Maria Piana a759d9dd67 Fix push notification & mentions
The code had an issue where it would not register a chat if just joined
as re-register push notifications was called before the chat had been
added.
This commit fixes the behavior by making sure that the chat just joined
is included.
2020-09-17 07:35:18 +02:00
Andrea Maria Piana 61b345ff33 Add index on seen & hide long messages
This commit does two things:
1) Add an index on seen & update only the not-seen messages in the query
2) Hide long messages in the database, as that's likely spam
2020-09-10 10:26:59 +02:00
Andrea Maria Piana 3c6b741f71
Add tests for send notification & review feedback 2020-09-09 21:22:34 +02:00
Andrea Maria Piana 28e06daf6d
Dont sent messages to user who have not joined 2020-09-09 21:22:18 +02:00
Andrea Maria Piana 2d525f9503
Set block mentions 2020-09-09 21:22:17 +02:00
Andrea Maria Piana 4e18d21129
Re-register on adding or removing a chat 2020-09-09 21:22:16 +02:00
Andrea Maria Piana 0d998e1858
Show correct text in case of mentions 2020-09-09 21:22:15 +02:00
Andrea Maria Piana 18877cae6f
Handle mentions on server 2020-09-09 21:22:14 +02:00
Andrea Maria Piana 8ee3c75510
Support for basic push notification for mentions 2020-09-09 21:22:13 +02:00
Andrea Maria Piana 00c5b60d7f
Move message to common namespace 2020-09-09 21:22:12 +02:00
Andrea Maria Piana 7d5d413222
Add parsing and storing of mentions 2020-09-09 21:22:07 +02:00
Roman Volosovskyi 57728224d4
[mentions] ParsedText in quoted message
ParsedText is needed to properly display mention in quoted message.
2020-09-09 16:07:31 +03:00
Andrea Maria Piana 32b0af8ecc Add index to emoji reactions 2020-09-09 15:02:54 +02:00
andrey 3b748a2e46 group chat invitation 2020-09-07 12:15:58 +02:00
andrey 2eaa1fcad7 [#11046] Add local contact names 2020-09-07 11:34:06 +02:00
Volodymyr Kozieiev 88a3022ea8
Drop messages with text longer than 4096 characters (#2029) 2020-09-01 17:38:36 +03:00
Andrea Maria Piana 4a87764cca
seed mrand 2020-09-01 10:31:12 +02:00
Andrea Maria Piana 2ef1aa89f4
Handle connection change
Currently messenger has no notion of being online.
This might cause a problem as we retry to register with a push
notification server even if not connected to any peer, which will
inevitably fail.
This commit adds a method `handleConnectionChange` that will be called
every time the connection change state.
2020-08-27 18:54:31 +02:00
Andrea Maria Piana 0ff2542939
Don't add contact on update
When sending a contact update we automatically added the contact,
but that resulted in the contact not being synced correctly as
`saveContact` will not trigger the side effects.

For now I have removed this behavior. Ideally we should have a single
call that handles the side effects, but for that ENS names should be
stored in messenger, so we can propagate it.
2020-08-27 18:54:30 +02:00
Andrea Maria Piana 640f5533ca
handle notifications at-most-once 2020-08-27 18:54:29 +02:00
Andrea Maria Piana d7222c89e0
shuffle servers and limit & filter by chat id 2020-08-27 18:54:28 +02:00
Andrea Maria Piana 401d18e558
Enabled push notifications by default and add fleet 2020-08-27 18:54:06 +02:00
Andrea Maria Piana d09eaeacb8 Allow node to start a push notification server
This commit allows a node to start a push notification server.
If the config is set it will start a messenger with a corresponding pn
server.
2020-08-20 10:54:33 +02:00
Andrea Maria Piana ac6aff2486
Re-issue bundle on changed parameters 2020-08-19 11:57:39 +02:00
Andrea Maria Piana 4cf491ae38
add push notification type 2020-08-18 16:29:58 +02:00
Andrea Maria Piana 874d1d0fa6
Update protocol/pushnotificationclient/client.go
Co-authored-by: Samuel Hawksby-Robinson <samuel@samyoul.com>
2020-08-18 16:29:57 +02:00
Andrea Maria Piana 2b2616f82c
Bump version to 0.57.0 2020-08-18 16:29:55 +02:00
Andrea Maria Piana 5544e710ab
send push notification info with contact code 2020-08-18 16:29:38 +02:00
Andrea Maria Piana 541756c777
move contact code to subscription 2020-08-18 16:29:37 +02:00
Andrea Maria Piana b557a64612
move shared secrets to subscription 2020-08-18 16:29:36 +02:00
Andrea Maria Piana dd3cf55556
move installations to subscription 2020-08-18 16:29:35 +02:00
Andrea Maria Piana be8e64fe8d
fix version check 2020-08-18 16:29:35 +02:00
Andrea Maria Piana 46aef6f3df
nil registration when unregistering 2020-08-18 16:29:34 +02:00
Andrea Maria Piana aa78b89b40
handle unregistered clients 2020-08-18 16:29:33 +02:00
Andrea Maria Piana 4b9d3df5ad
Set RemotePushNotifications 2020-08-18 16:29:32 +02:00
Andrea Maria Piana 91074ac95e
Add apn-topic & token-type set by the client 2020-08-18 16:29:31 +02:00
Andrea Maria Piana 60e61caad8
invalidate token if allow from contacts only has been enabled 2020-08-18 16:29:30 +02:00
Andrea Maria Piana e8daee3712
Add topic 2020-08-18 16:29:29 +02:00
Andrea Maria Piana 2e231e690e Fix mark messages seen
If a chat had a last message the persistence call `Chat()` would not
decode `LastMessage` correctly.
This commit fixes the issue.
2020-08-17 13:40:10 +02:00
Samuel Hawksby-Robinson 19487da894
Fixes a minor rebase quirk (#2016)
See commit fc7b311838
2020-08-13 18:57:17 +01:00
Andrea Maria Piana 2d0818d873 Add SignMessageWithChatKey 2020-07-31 14:51:24 +02:00
Andrea Maria Piana ab01a05cd6 Don't use bytes anymore for LastMessage
LastMessage in chat was encoded in bytes so that we don't have to
encoded/decode everytime we save to db or pass the client.

An issue with emoji surfaced a problem with this approach.
Chat.LastClockValue represent the last clock value of any type of
message exchanged in a chat (emoji,group membership updates, contact
updates).
So when receving a new message, we should update LastMessage if the
clock of the LastMessage is lower than the received message, and we
should not only check LastClockValue, otherwise the message might be
discarded although it is the most recent.

This commit fixes the issue by keeping LastMessage as an object and
comparing LastMessage.Clock instead of LastClockValue
2020-07-31 13:40:11 +02:00
Andrea Maria Piana 5e4d58a807
Fix lastMessage when receive an emoji out of order 2020-07-30 21:14:47 +02:00
Andrea Maria Piana 2cc3a9bab1
increase count & add comment 2020-07-30 20:21:10 +02:00
Andrea Maria Piana b17d51e83d
describe chat entity comment 2020-07-30 20:21:08 +02:00
Andrea Maria Piana 33c87a670a
Uncommet test 2020-07-30 20:21:07 +02:00
Andrea Maria Piana 5910de3683
Handle emoji reaction in group chat 2020-07-30 20:21:06 +02:00
Andrea Maria Piana b3c6afe99b
dont use a pointer 2020-07-30 20:21:05 +02:00
Andrea Maria Piana eb087a00a4
add retracted 2020-07-30 20:21:04 +02:00
Andrea Maria Piana fcb901daf2
remove leftover emojialias 2020-07-30 20:21:03 +02:00
Andrea Maria Piana 4a83c86c3e
Add EmojiReaction to membershipupdatemessage 2020-07-30 20:21:02 +02:00
Andrea Maria Piana 7e7d95a907
camel case emojiReactions 2020-07-30 20:21:01 +02:00