Commit Graph

716 Commits

Author SHA1 Message Date
Icaro Motta 6fa8c11382
URL unfurling (initial implementation) (#3471)
This is the initial implementation for the new URL unfurling requirements. The
most important one is that only the message sender will pay the privacy cost for
unfurling and extracting metadata from websites. Once the message is sent, the
unfurled data will be stored at the protocol level and receivers will just
profit and happily decode the metadata to render it.

Further development of this URL unfurling capability will be mostly guided by
issues created on clients. For the moment in status-mobile:
https://github.com/status-im/status-mobile/labels/url-preview

- https://github.com/status-im/status-mobile/issues/15918
- https://github.com/status-im/status-mobile/issues/15917
- https://github.com/status-im/status-mobile/issues/15910
- https://github.com/status-im/status-mobile/issues/15909
- https://github.com/status-im/status-mobile/issues/15908
- https://github.com/status-im/status-mobile/issues/15906
- https://github.com/status-im/status-mobile/issues/15905

### Terminology

In the code, I've tried to stick to the word "unfurl URL" to really mean the
process of extracting metadata from a website, sort of lower level. I use "link
preview" to mean a higher level structure which is enriched by unfurled data.
"link preview" is also how designers refer to it.

### User flows

1. Carol needs to see link previews while typing in the chat input field. Notice
   from the diagram nothing is persisted and that status-go endpoints are
   essentially stateless.

```
#+begin_src plantuml :results verbatim
  Client->>Server: Call wakuext_getTextURLs
  Server-->>Client: Normalized URLs
  Client->>Client: Render cached unfurled URLs
  Client->>Server: Unfurl non-cached URLs.\nCall wakuext_unfurlURLs
  Server->>Website: Fetch metadata
  Website-->>Server: Metadata (thumbnail URL, title, etc)
  Server->>Website: Fetch thumbnail
  Server->>Website: Fetch favicon
  Website-->>Server: Favicon bytes
  Website-->>Server: Thumbnail bytes
  Server->>Server: Decode & process images
  Server-->>Client: Unfurled data (thumbnail data URI, etc)
#+end_src
```

```
     ,------.                                 ,------.                             ,-------.
     |Client|                                 |Server|                             |Website|
     `--+---'                                 `--+---'                             `---+---'
        |        Call wakuext_getTextURLs        |                                     |
        | --------------------------------------->                                     |
        |                                        |                                     |
        |             Normalized URLs            |                                     |
        | <- - - - - - - - - - - - - - - - - - - -                                     |
        |                                        |                                     |
        |----.                                   |                                     |
        |    | Render cached unfurled URLs       |                                     |
        |<---'                                   |                                     |
        |                                        |                                     |
        |         Unfurl non-cached URLs.        |                                     |
        |         Call wakuext_unfurlURLs        |                                     |
        | --------------------------------------->                                     |
        |                                        |                                     |
        |                                        |            Fetch metadata           |
        |                                        | ------------------------------------>
        |                                        |                                     |
        |                                        | Metadata (thumbnail URL, title, etc)|
        |                                        | <- - - - - - - - - - - - - - - - - -
        |                                        |                                     |
        |                                        |           Fetch thumbnail           |
        |                                        | ------------------------------------>
        |                                        |                                     |
        |                                        |            Fetch favicon            |
        |                                        | ------------------------------------>
        |                                        |                                     |
        |                                        |            Favicon bytes            |
        |                                        | <- - - - - - - - - - - - - - - - - -
        |                                        |                                     |
        |                                        |           Thumbnail bytes           |
        |                                        | <- - - - - - - - - - - - - - - - - -
        |                                        |                                     |
        |                                        |----.                                |
        |                                        |    | Decode & process images        |
        |                                        |<---'                                |
        |                                        |                                     |
        | Unfurled data (thumbnail data URI, etc)|                                     |
        | <- - - - - - - - - - - - - - - - - - - -                                     |
     ,--+---.                                 ,--+---.                             ,---+---.
     |Client|                                 |Server|                             |Website|
     `------'                                 `------'                             `-------'
```

2. Carol sends the text message with link previews in the RPC request
   wakuext_sendChatMessages. status-go assumes the link previews are good
   because it can't and shouldn't attempt to re-unfurl them.

```
#+begin_src plantuml :results verbatim
  Client->>Server: Call wakuext_sendChatMessages
  Server->>Server: Transform link previews to\nbe proto-marshalled
  Server->DB: Write link previews serialized as JSON
  Server-->>Client: Updated message response
#+end_src
```

```
     ,------.                       ,------.                                  ,--.
     |Client|                       |Server|                                  |DB|
     `--+---'                       `--+---'                                  `+-'
        | Call wakuext_sendChatMessages|                                       |
        | ----------------------------->                                       |
        |                              |                                       |
        |                              |----.                                  |
        |                              |    | Transform link previews to       |
        |                              |<---' be proto-marshalled              |
        |                              |                                       |
        |                              |                                       |
        |                              | Write link previews serialized as JSON|
        |                              | -------------------------------------->
        |                              |                                       |
        |   Updated message response   |                                       |
        | <- - - - - - - - - - - - - - -                                       |
     ,--+---.                       ,--+---.                                  ,+-.
     |Client|                       |Server|                                  |DB|
     `------'                       `------'                                  `--'
```

3. The message was sent over waku and persisted locally in Carol's device. She
   should now see the link previews in the chat history. There can be many link
   previews shared by other chat members, therefore it is important to serve the
   assets via the media server to avoid overloading the ReactNative bridge with
   lots of big JSON payloads containing base64 encoded data URIs (maybe this
   concern is meaningless for desktop). When a client is rendering messages with
   link previews, they will have the field linkPreviews, and the thumbnail URL
   will point to the local media server.

```
 #+begin_src plantuml :results verbatim
   Client->>Server: GET /link-preview/thumbnail (media server)
   Server->>DB: Read from user_messages.unfurled_links
   Server->Server: Unmarshal JSON
   Server-->>Client: HTTP Content-Type: image/jpeg/etc
 #+end_src
```

```
     ,------.                                    ,------.                                  ,--.
     |Client|                                    |Server|                                  |DB|
     `--+---'                                    `--+---'                                  `+-'
        | GET /link-preview/thumbnail (media server)|                                       |
        | ------------------------------------------>                                       |
        |                                           |                                       |
        |                                           | Read from user_messages.unfurled_links|
        |                                           | -------------------------------------->
        |                                           |                                       |
        |                                           |----.                                  |
        |                                           |    | Unmarshal JSON                   |
        |                                           |<---'                                  |
        |                                           |                                       |
        |     HTTP Content-Type: image/jpeg/etc     |                                       |
        | <- - - - - - - - - - - - - - - - - - - - -                                        |
     ,--+---.                                    ,--+---.                                  ,+-.
     |Client|                                    |Server|                                  |DB|
     `------'                                    `------'                                  `--'
```

### Some limitations of the current implementation

The following points will become separate issues in status-go that I'll work on
over the next couple weeks. In no order of importance:

- Improve how multiple links are fetched; retries on failure and testing how
  unfurling behaves around the timeout limits (deterministically, not by making
  real HTTP calls as I did). https://github.com/status-im/status-go/issues/3498
- Unfurl favicons and store them in the protobuf too.
- For this PR, I added unfurling support only for websites with OpenGraph
  https://ogp.me/ meta tags. Other unfurlers will be implemented on demand. The
  next one will probably be for oEmbed https://oembed.com/, the protocol
  supported by YouTube, for example.
- Resize and/or compress thumbnails (and favicons). Often times, thumbnails are
  huge for the purposes of link previews. There is already support for
  compressing JPEGs in status-go, but I prefer to work with compression in a
  separate PR because I'd like to also solve the problem for PNGs (probably
  convert them to JPEGs, plus compress them). This would be a safe choice for
  thumbnails, favicons not so much because transparency is desirable.
- Editing messages is not yet supported.
- I haven't coded any artificial limit on the number of previews or on the size
  of the thumbnail payload. This will be done in a separate issue. I have heard
  the ideal solution may be to split messages into smaller chunks of ~125 KiB
  because of libp2p, but that might be too complicated at this stage of the
  product (?).
- Link preview deletion.
- For the moment, OpenGraph metadata is extracted by requesting data for the
  English language (and fallback to whatever is available). In the future, we'll
  want to unfurl by respecting the user's local device language. Some websites,
  like GoDaddy, are already localized based on the device's IP, but many aren't.
- The website's description text should be limited by a certain number of
  characters, especially because it's outside our control. Exactly how much has
  not been decided yet, so it'll be done separately.
- URL normalization can be tricky, so I implemented only the basics to help with
  caching. For example, the url https://status.im and HTTPS://status.im are
  considered identical. Also, a URL is considered valid for unfurling if its TLD
  exists according to publicsuffix.EffectiveTLDPlusOne. This was essential,
  otherwise the default Go url.Parse approach would consider many invalid URLs
  valid, and thus the server would waste resources trying to unfurl the
  unfurleable.

### Other requirements

- If the message is edited, the link previews should reflect the edited text,
  not the original one. This has been aligned with the design team as well.
- If the website's thumbnail or the favicon can't be fetched, just ignore them.
  The only mandatory piece of metadata is the website's title and URL.
- Link previews in clients should be generated in near real-time, that is, as
  the user types, previews are updated. In mobile this performs very well, and
  it's what other clients like WhatsApp, Telegram, and Facebook do.

### Decisions

- While the user typing in the input field, the client is constantly (debounced)
  asking status-go to parse the text and extract normalized URLs and then the
  client checks if they're already in its in-memory cache. If they are, no RPC
  call is made. I chose this approach to achieve the best possible performance
  in mobile and avoid the whole RPC overhead, since the chat experience is
  already not smooth enough. The mobile client uses URLs as cache keys in a
  hashmap, i.e. if the key is present, it means the preview is readily available
  (naive, but good enough for now). This decision also gave me more flexibility
  to find the best UX at this stage of the feature.
- Due to the requirement that users should be able to see independent loading
  indicators for each link preview, when status-go can't unfurl a URL, it
  doesn't return it in the response.
- As an initial implementation, I added the BLOB column unfurled_links to the
  user_messages table. The preview data is then serialized as JSON before being
  stored in this column. I felt that creating a separate table and the related
  code for this initial PR would be inconvenient. Is that reasonable to you?
  Once things stabilize I can create a proper table if we want to avoid this
  kind of solution with serialized columns.
2023-05-18 15:43:06 -03:00
frank 39f486a47a
optimize mention (#3479)
* optimize mention

* bump version
2023-05-15 15:21:41 +08:00
Stefan c020222f1b feat(Wallet): add activity filter api prototype
Add the possibility of retrieving the metadata of wallet activity based
on the given filter criteria.

Current implementation relies that after fetching the metadata, user
will follow up with more requests for details. However, after some
experimenting I'm considering extracting all required information
for the summary viewing while filtering. This way there will be no
need for another batch requests for transfers, multi-transactions and
pending transactions to show the summary. Only when user wants to see
the details for one will specifically request it.

For this first prototype, the filter criteria is limited to:
- time
- type
- addresses

Major changes:
- Add the filter definition to be used in propagating the filter
  information
- Add GetActivityEntries API to return the list of activity entries
  for the given addresses/chainIDs by a view in the complete list
- GetTransfersForIdentities to batch retrieve further details of the
  transfers
- GetPendingTransactionsForIdentities to batch retrieve further details
  of the pending transactions
- Added a new package testutils for tests.
- Added tests

Updates status-desktop #10366
Closes status-desktop #10633
2023-05-14 23:05:28 +02:00
Sale Djenic 189c7a6919 chore: `keypairs` package which actually refers to keycards is renamed to `keycards` package
This commit renames few api endpoints:
- old `AddMigratedKeyPairOrAddAccountsIfKeyPairIsAdded` renamed to `AddKeycardOrAddAccountsIfKeycardIsAdded`
- old `GetAllMigratedKeyPairs` renamed to `GetAllKnownKeycardsGroupedByKeyUID`
- old `GetMigratedKeyPairByKeyUID` renamed to `GetKeycardByKeyUID`
- old `DeleteKeypair` renamed to `DeleteAllKeycardsWithKeyUID`
2023-05-10 09:27:51 +02:00
Sale Djenic 31144ed5a3 chore: `password` removed from the `DeleteAccount` function
Asking for a password when removing a key is not necessary.
2023-05-09 10:48:44 +02:00
Stefan c5a8b40d9c fix(wallet) don't delete pending transactions for multi-transaction
This is done in Nim after the watched pending transaction changes state.

Note: tried to delete all transactions, not only multi-transaction,
as soon as the state is downloaded but it didn't work: noticed crashes
with minted transactions after restart.

Updates status-desktop #10474
2023-05-08 11:24:37 +02:00
IvanBelyakoff cb9f2dd7d2
feat(wallet): refactor transfers to allow multiple transfer fetching (#3432)
strategies.

On mobile devices we might not want to load all transfers history
to avoid high internet traffic consumption or to save some space.
On desktop we will load all transfers in background.

- Moved current fetching transfers implementation to
OnDemandFetchStrategy
- Implemented QueuedAtomicGroup to cache outgoing commands above a
threshold
- Refactored some type names, methods for better clarity:
  - LastKnownBlock name was misused
  - Block renamed to BlockDAO to clarify what it does and avoid
confusion with geth Block type
- Cleanup - removed unused parameters and types' members
- Changed tests to use in memory DB for speedup
- Added more logging and comments

Updates #10246
2023-05-08 08:02:00 +02:00
Michal Iskierko 4935848287 fix(GetAllCommunityTokens): Add GetAllCommunityTokens function
Issue #10254
2023-05-05 18:14:55 +02:00
Anthony Laibe 722390c784 fix: reponse may contains less results than expected 2023-05-03 08:38:29 +02:00
Jonathan Rainville e7fbc191f4
refactor(chat/api): re-add chats in the getChannelGroups api (#3439)
* refactor(chat/api): re-add chats in the getChannelGroups api

* feat(chat/api): add GetChannelGroupById api function
2023-05-02 13:54:44 -04:00
Stefan a6d553c937 fix(wallet) ClientWithFallback handling state errors required by BH
The "not found" and "no contract" expected error is caught and wrapped
by ClientWithFallback. The fetching of balance history of next blocks
is aborted which is not desired.

Fix by not treating the error as a connection error in
ClientWithFallback.
2023-05-02 12:07:06 +02:00
frank 8608aecdb4
fix mobile mention issue #15616 (#3388)
* fix mobile mention issue #15616

* add state != nil

* clear previous text when clear mentions

* fix: after selected mention user, and type @ not working

* bump version
2023-04-27 11:06:40 +08:00
Eng Zer Jun 83ad76637a
test: use `T.TempDir` to create temporary test directory (#2746)
This commit replaces `os.MkdirTemp` with `t.TempDir` in tests. The
directory created by `t.TempDir` is automatically removed when the test
and all its subtests complete.

Prior to this commit, temporary directory created using `os.MkdirTemp`
needs to be removed manually by calling `os.RemoveAll`, which is omitted
in some tests. The error handling boilerplate e.g.
	defer func() {
		if err := os.RemoveAll(dir); err != nil {
			t.Fatal(err)
		}
	}
is also tedious, but `t.TempDir` handles this for us nicely.

Reference: https://pkg.go.dev/testing#T.TempDir

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2023-04-26 21:39:51 +01:00
frank a2ed1b78dd
feat:support backup/sync ens name (#3415) 2023-04-26 23:37:18 +08:00
Sale Djenic 392d808af0 feat: `GetAccountsByKeyUID` endpoint added 2023-04-25 14:03:32 +02:00
Sale Djenic c8ef2c1a7f fix: deleting derived from address keystore file applied to all keypairs being migrated instead to a profile keypair only 2023-04-25 14:03:32 +02:00
Anthony Laibe dd7c59423f feat: add saved address created at 2023-04-22 10:22:43 +02:00
Sale Djenic 34badf2405 fix: `SaveAccounts` endpoint changed to `SaveAccount` which requires a single account to be provided instead of array of accounts 2023-04-21 16:35:24 +02:00
Sale Djenic 5d7da45f07 feat: `VerifyKeystoreFileForAccount` endpoint added 2023-04-21 16:35:24 +02:00
Sale Djenic f1e3ae5b46 feat: sync and backup wallet accounts
Changes applied here introduce:
- improvements to sync wallet accounts among devices (including all account types)
- backing up wallet accounts to and fetch them from waku (an information about received
wallet accounts is sent via `waku.backedup.wallet-account` signal to a client)
2023-04-21 16:35:24 +02:00
Mohamed Javid 0eff61c57a
Community request to join changes for pending and declined states (#3387)
* Community request to join changes

* Fix read state for request to join notification

* Bring back deleted notification when updated with response

* Update Request timeout to 7 days

* Update VERSION
2023-04-21 14:48:47 +05:30
Dario Gabriel Lipicar 4ef5eba6b5 chore: move chainIDs to constants file 2023-04-19 08:30:10 -03:00
Dario Gabriel Lipicar 633c8d1ed5 chore: clean up passing api keys to the wallet service 2023-04-19 08:30:10 -03:00
Dario Gabriel Lipicar c8f0ceccc8 feat: add api to get list of owners from a given nft contract
Fixes #10290
2023-04-19 08:30:10 -03:00
Ibrahem Khalil 719af90fcc
Add muted_till param for chats (#3258) 2023-04-16 17:06:00 +02:00
Jonathan Rainville e5093124f2 fix(chat-api): fix unread count on channelGroup condition 2023-04-13 10:04:18 -04:00
Michal Iskierko 4e3ba2941c fix(AddTokenOwners): Move AddTokenOwners from MintTo
Issue #10161
2023-04-13 11:43:48 +02:00
Sale Djenic a37b586915 fix: delete account endpoint updated
Local keystore file deletion is done only if an account being deleted is migrated to a keycard.
2023-04-11 12:14:35 +02:00
frank fd6073ab2e
rewrite mention in go (#3309) 2023-04-07 16:47:38 +08:00
Sale Djenic 1754c23763 chore: improvements to `GetWalletToken` function 2023-04-06 19:09:13 +02:00
Dario Gabriel Lipicar ff755a1176 fix: add supported chains to OpenSea 2023-04-06 13:42:38 -03:00
Igor Sirotin fb36298e37
feature(messenger): RequestContactInfoFromMailserver (#3376) 2023-04-05 23:25:44 +03:00
Michal Iskierko db6c05a8e8 fix(DeployCollectiblesEstimate): Add DeployCollectiblesEstimate function
Issue #10035
2023-04-05 12:23:19 +02:00
Dario Gabriel Lipicar 5bad40c8be feat: add collectible animation info to data fetched from Opensea 2023-04-03 12:32:32 -03:00
Anthony Laibe c84d6fcc35 feat: remove consecutive 2023-03-31 19:22:53 +02:00
Anthony Laibe 9c33049b4f chore: update connected on global failure 2023-03-31 19:22:53 +02:00
Anthony Laibe 0c8ce79941 feat: returns tokens even when error 2023-03-31 19:22:53 +02:00
Anthony Laibe 540001e137 feat: chunk correctly coinmarketcap calls 2023-03-31 19:22:53 +02:00
Anthony Laibe e92049b9ca fix: wallet token return on no network 2023-03-31 19:22:53 +02:00
Anthony Laibe d86446ad53 chore: adding debug 2023-03-31 19:22:53 +02:00
Anthony Laibe 2baa8e8ead feat: filter token by valid chain ID 2023-03-31 19:22:53 +02:00
Anthony Laibe c9a8ab13c2 fix: set is connected for chain without fallback 2023-03-31 19:22:53 +02:00
Anthony Laibe ec5449c2bd fix: collectible status event 2023-03-31 19:22:53 +02:00
Anthony Laibe 62220cedee feat: update chunk limit for token 2023-03-31 19:22:53 +02:00
Anthony Laibe fae86fcfb7 feat: group all blockchain event 2023-03-31 19:22:53 +02:00
Anthony Laibe ba75bda39e feat: event on error 2023-03-31 19:22:53 +02:00
Dario Gabriel Lipicar 4e222cc404 feat: collectibles cache 2023-03-31 12:24:33 -03:00
Dario Gabriel Lipicar f097d3430a feat: reduce usage of OpenSea api key 2023-03-31 12:23:50 -03:00
Sale Djenic c791163136 feat: backup display name improvements
- Display name is now backed up only as a part of `protobuf.BackedUpProfile` message,
it is not backed up via `protobuf.SyncSetting` any more (this refers only to backing up to
and fetching data from waku, regular syncing (among devices) remains unchanged)

- When saving the display name fetched from waku, before a clock was set to the current
time when that operation is made, which was incorrect, now we're using the clock from
backed up message (`SaveSyncDisplayName` function)
2023-03-31 16:20:50 +02:00
Patryk Osmaczko d077c7e4bd fix: make `GetChatsByChannelGroupID` return chats with members
fixes: status-im/status-desktop#10066
2023-03-30 16:02:10 +02:00