Commit Graph

198 Commits

Author SHA1 Message Date
Roman Volosovskyi 4de912baba [pairing] Sync chat removing 2021-10-06 12:38:57 +01:00
Parvesh Monu c3ced09839
Storing emoji values for Custom Emoji Thumbnails for Community Channels (#2366) 2021-10-04 18:32:25 +05:30
Andrea Maria Piana 5e83d8e95e Add HasAddedUs field 2021-10-04 12:19:15 +02:00
Andrea Maria Piana 438c2e8883 contacts performance 2021-10-04 12:19:15 +02:00
Samuel Hawksby-Robinson aaf9aeca8a Added encryption keys to the IdentityImage protobuf 2021-09-30 13:02:41 +01:00
Roman Volosovskyi 839d6b3194
[status-im/status-react#12517] Ensure that all necessary contact fields are synced 2021-09-23 12:22:49 +03:00
Samuel Hawksby-Robinson 89251e8416
Sync Communities (#2253)
* Added community sync protobuf

* Updated community sync send logic

* Integrated syncCommunity handling

* Added synced_at field and tidied up some other logic

* persistence testing

* Added testing and join functionality

* Fixed issue with empty scan params

* Finshed persistence tests for new db funcs

* Midway debug of description not persisting after sync

* Resolved final issues and tidied up

* Polish

* delint

* Fix error not handled on SetPrivateKey

* fix infinite loop, again

* Added muted option and test fix

* Added Muted to syncing functions, not just in persistence

* Fix bug introduced with Muted property

* Added a couple of notes for future devs

* Added most of the sync RequestToJoin functionality

Tests need to be completed and tests are giving some errors

* Finished tests for getJoinedAndPending

* Added note

* Resolving lint

* Fix of protobuf gen bug

* Fixes to community sync tests

* Fixes to test

* Continued fix of e2e

* Final fix to e2e testing

* Updated migration position

* resolve missing import

* Apparently the linter spellchecks

* Fix bug from  #2276 merge

* Bug fix for leaving quirkiness

* Addressed superfluous MessengerResponse field

* Addressed feedback

* VERSION bump
2021-08-06 16:40:23 +01:00
Jonathan Rainville cdc7c55030
Delete messages (#2279)
* feat: add api and handling to delete messages

* fix(delete): add whitelist to message types that can be deleted
2021-07-26 17:06:32 -04:00
Andrea Maria Piana f31dc5df92 Use marshal binary 2021-07-20 10:57:38 +02:00
Andrea Maria Piana a4701f41ee linting and remove unused code 2021-07-20 10:57:38 +02:00
Andrea Maria Piana d84da4f515 Fix some tests and restore private/public RPC client 2021-07-20 10:57:38 +02:00
Andrea Maria Piana 799bd93451 Memove mailserver registry and update ierc20 contract bindings
This commit removes the mailserver contract registry, as not used
anymore, and upgrade the ERC20 contract bindings.
2021-07-20 10:57:38 +02:00
Brian Sztamfater 92404a5ab7
Add reply message to activity center notification of type reply (#2272) 2021-07-15 17:21:44 -03:00
Jonathan Rainville 11c46edd8b
feat(community): add muted to community and function to set it (#2271) 2021-06-30 09:29:43 -04:00
Richard Ramos 217bace570 feat: edit messages 2021-06-29 13:15:15 +02:00
Jonathan Rainville 14cfcebb5a
feat: add author to notification in activity center (#2259) 2021-06-17 11:08:28 -04:00
Andrea Maria Piana 3a61c3bae9 Allow setting mailserver sync value
This commit adds a setting for mailserver, that allows the client to
specify the syncing period.
Also fixes a minor issue with deletion of public chats.
2021-06-01 12:59:52 +02:00
Brian Sztamfater a72c32009a
Add mention notification to activity center (#2239)
Signed-off-by: Brian Sztamfater <brian@status.im>
2021-05-29 14:05:25 -03:00
Andrea Maria Piana d50fee6bb2 Handle connection state 2021-05-21 07:22:58 +02:00
Andrea Maria Piana cf6ef3171a Add mailserver logic 2021-05-21 07:22:58 +02:00
Pascal Precht dd49e604d4 feat: introduce new `joined` property in `Chat` struct
This property is useful for clients to know when a channel or chat
was joined so they can use that to calculate the order of channels
and chats shown in applications.

Changes include a new joined property on the Chat struct,
as well as adjustments in the persistence layer to retreive and
update chat data in the database.

In addition there's a migration script that alters the existing
chat table to introduce a new column for the joined field.
It also updates all existing rows in the database to set `joined`
to `0`.
2021-05-18 11:29:03 +02:00
Roman Volosovskyi 6037570901
[wallet] Pending transactions API 2021-04-23 18:20:38 +03:00
Andrea Maria Piana c739f73f49 Add activity center & messages from contacts only 2021-04-16 20:42:40 +02:00
Samuel Hawksby-Robinson 6d7028a8d9
Added Anonymous Metrics Send Opt-In Setting (#2195)
* Added anon metrics send opt in setting

* resolved rebase conflict, renamed migration to use unixtimestamp

Theres always conflicts with migrations using sequential numbers, less so with unix timestamp
2021-04-13 14:00:18 +01:00
Shivek Khurana 79bf90e990
👮🏽‍♂️ Update validators and add a new event (#2189)
* Add extra event to capture other type of navigations, allow empty screen name, rename cofx to get rid of clj ns, update tests

* Some view ids are greater than 16 characters. Made it 32 to be safe.

* Tab navigation events occur outside nav, add a new validator for them

* Remove navigate to cofx event, capture screens on will focus, get rid of enum and make valid screens a string less than 32 characters

* Run make generate

* Fix test

* Bump version to 0.75.1
2021-04-12 17:55:53 +05:30
Andrea Maria Piana d22182ea82 Ban user from community
This commit introduces a new endpoint to ban a user from a community.

Messages from this user will be ignored.
2021-03-24 09:29:21 +01:00
Roman Volosovskyi 6bea21c197
[wallet] Skip transaction on chain id mismatch
Sometimes eth_getBlockByNumber returns txs with chainId which is not
equal to chanin's id. That caused an error and tx fetching was
interrupted. From now on such txs will be skipped.
2021-03-19 15:54:39 +02:00
Shivek Khurana 1097b14a7f
🎭 📊 Anonymous Metrics V0 (#2170)
* Migrations in place, how to run them?

* Remove down migrations and touch database.go

* Database and Database Test package in place, added functions to get and store app metrics

* make generate output

* Minor bug fix on app metrics insert and select

* Add a validation layer to restrict what can be saved in the database

* Make validation more terse, throw error if schema doesn't exist, expose appmetrics service

* service updates

* Compute all errors before sending them out

* Trying to bring a closjure to appmetrics go

* Expose appmetrics via an api, skip fancy

* Address value as Jason Dawt Rawmasage to ease parsing

* Introduce a buffered chan with magic cap of 8 to minimize writes to DB. Tests for service and API. Also expose GetAppMetrics function.

* Lint issues

* Remove autoincrement, undo waku.json changes, fix error being shadowed, return nil where nil ought to be returned, get rid of buffered channel

* Bump migration number

* Fix API factory usage

* Add comment re:json.RawMessage instead of strings

* Get rid of test vars, throw save error inside the loop

* Update version

Co-authored-by: Samuel Hawksby-Robinson <samuel@samyoul.com>
2021-03-17 18:09:28 +05:30
Andrea Maria Piana f75f7bb738 Expand confirmations to private group chat messages
This commit expands the confirmation mechanism to allow private group
chat messages to be confirmed:

Changes:

- Added a separate table for message confirmations as group chat
messages have same messageID but multiple datasyncID
- Removed DataSyncID from raw message (I haven't removed the column name
as it can't be done in sqlite without copying over the table)
2021-03-16 13:41:14 +01:00
Roman Volosovskyi a7f2e94999
[wallet] Store account's last known balance and nonce 2021-03-15 16:18:12 +02:00
Andrea Maria Piana a1b3e3a772
Clean topics that we don't listen to
There was a bug on status-react where it would save filters that were
not listened to.
This commit adds a task to clean up those filters as they might result
in long syncing times.

This commit also returns topics/ranges/mailserves from messenger in
order to make the initialization of the app simpler and start moving
logic to status-go.

It also removes whisper from vendor.
2021-01-26 09:39:57 +01:00
Andrea Maria Piana 7387049d4b Upgrade linter and address issues 2020-12-28 16:55:14 +01:00
Andrea Maria Piana c78e084eb9 Fix tests and lint 2020-12-17 14:10:00 +01:00
Samuel Hawksby-Robinson 16fecf5520 Passing Master info to Messenger 2020-12-17 14:10:00 +01:00
Samuel Hawksby-Robinson e8d883edb6 bindata for profile_image migrations 2020-12-17 14:10:00 +01:00
Jonathan Rainville b2d8f7f416
feat: add bookmarks table for the browser bookmarks
plus get bookmark icon using a lib
2020-11-16 13:31:34 -05:00
Andrea Maria Piana ba126e9fe0 remove broken tests 2020-08-17 12:53:37 +02:00
Andrea Maria Piana 849492fda9
Add statusUpdate code (#1861)
* Add status-option code

This commits changes the behavior of waku introducing a new status-code,
`2`, that replaces the current single options codes.

* linting
2020-02-21 15:48:53 +01:00
Andrea Maria Piana 141f917e7c
Initialize protocol when waku active (#1832) 2020-02-17 15:38:59 +01:00
Volodymyr Kozieiev 19fc448c85
pending-stickers field added to settings, unit-tests fixed (#1830) 2020-02-13 13:48:04 +02:00
Roman Volosovskyi 9cf640842b
[status-im/status-react#9942] Upgradable paths in configs
Storing absolute path for different configs breaks compatibility on iOS
as app's dir is changed after upgrade. The solution is to store relative
paths and to concatenate it with `backend.rootDataDir`. The only
exception is `LogFile` as it is stored outside `backend.rootDataDir` on
Android. `LogDir` config was added to allow adding of custom dir for log
file.
Configs concerned:
`DataDir`
`LogDir`
`LogFile`
`KeystoreDir`
`BackupDisabledDataDir`
2020-02-03 20:02:01 +02:00
Roman Volosovskyi a92a95cf83
status-im/status-react#9203 Faster tx fetching with less request
*** How it worked before this PR on multiaccount creation:
- On multiacc creation we scanned chain for eth and erc20 transfers. For
  each address of a new empty multiaccount this scan required
  1. two `eth_getBalance` requests to find out that there is no any
     balance change between zero and the last block, for eth transfers
  2. and `chain-size/100000` (currently ~100) `eth_getLogs` requests,
     for erc20 transfers
- For some reason we scanned an address of the chat account as well, and
  also accounts were not deduplicated. So even for an empty multiacc we
  scanned chain twice for each chat and main wallet addresses, in result
  app had to execute about 400 requests.
- As mentioned above, `eth_getBalance` requests were used to check if
  there were any eth transfers, and that caused empty history in case
  if user already used all available eth (so that both zero and latest
  blocks show 0 eth for an address). There might have been transactions
  but we wouldn't fetch/show them.
- There was no upper limit for the number of rpc requests during the
  scan, so it could require indefinite number of requests; the scanning
  algorithm was written so that we persisted the whole history of
  transactions or tried to scan form the beginning again in case of
  failure, giving up only after 10 minutes of failures. In result
  addresses with sufficient number of transactions would never be fully
  scanned and during these 10 minutes app could use gigabytes of
  internet data.
- Failures were caused by `eth_getBlockByNumber`/`eth_getBlockByHash`
  requests. These requests return significantly bigger responses than
  `eth_getBalance`/`eth_transactionsCount` and it is likely that
  execution of thousands of them in parallel caused failures for
  accounts with hundreds of transactions. Even for an account with 12k
  we could successfully determine blocks with transaction in a few
  minutes using `eth_getBalance` requests, but `eth_getBlock...`
  couldn't be processed for this acc.
- There was no caching for for `eth_getBalance` requests, and this
  caused in average 3-4 times more such requests than is needed.

*** How it works now on multiaccount creation:
- On multiacc creation we scan chain for last ~30 eth transactions and
  then check erc20 in the range where these eth transactions were found.
  For an empty address in multiacc this means:
  1. two `eth_getBalance` transactions to determine that there was no
     balance change between zero and the last block; two
     `eth_transactionsCount` requests to determine there are no outgoing
     transactions for this address; total 4 requests for eth transfers
  2. 20 `eth_getLogs` for erc20 transfers. This number can be lowered,
     but that's not a big deal
- Deduplication of addresses is added and also we don't scan chat
  account, so a new multiacc requires ~25 (we also request latest block
  number and probably execute a few other calls) request to determine
  that multiacc is empty (comparing to ~400 before)
- In case if address contains transactions we:
  1. determine the range which contains 20-25 outgoing eth/erc20
     transactions. This usually requires up to 10 `eth_transactionCount`
     requests
  2. then we scan chain for eth transfers using `eth_getBalance` and
     `eth_transactionCount` (for double checking zero balances)
  3. we make sure that we do not scan db for more than 30 blocks with
     transfers. That's important for accounts with mostly incoming
     transactions, because the range found on the first step might
     contain any number of incoming transfers, but only 20-25 outgoing
     transactions
  4. when we found ~30 blocks in a given range, we update initial
     range `from` block using the oldest found block
  5. and now we scan db for erc20transfers using `eth_getLogs`
     `oldest-found-eth-block`-`latest-block`, we make not more than 20 calls
  6. when all blocks which contain incoming/outgoing transfers for a
     given address are found, we save these blocks to db and mark that
     transfers from these blocks are still to be fetched
  7. Then we select latest ~30 (the number can be adjusted) blocks from
     these which were found and fetch transfers, this requires 3-4
     requests per transfer.
  8. we persist scanned range so that we know were to start next time
  9. we dispatch an event which tells client that transactions are found
  10. client fetches latest 20 transfers
- when user presses "fetch more" button we check if app's db contains next
  20 transfers, if not we scan chain again and return transfers after

small fixes
2020-01-23 10:36:11 +02:00
Adam Babik bc2d018483
Add support for request messages by topics (#1805) 2020-01-21 08:11:24 +01:00
Adam Babik 79b8112f89
Split shhext into shhext and wakuext (#1803) 2020-01-20 21:56:06 +01:00
Adam Babik 44aa313981
Make shhext and protocol work with Waku (#1777)
This change makes shhext and protocol submodule work with Waku and Whisper.
2020-01-13 20:17:30 +01:00
Andrea Maria Piana 88a1d0111e
Add Commands (#1731)
This commit adds handling of Transaction commands.
2020-01-10 19:59:01 +01:00
Adam Babik 37a508a97b
Integrate Waku service (#1759) 2020-01-08 12:12:23 +01:00
Pedro Pombeiro c8a911ebd1 Use goimports instead of gofmt 2020-01-06 10:17:23 +01:00
Pedro Pombeiro 287e5cdf79 Abstract `accounts.Key` and geth `keystore` 2020-01-06 10:17:23 +01:00
Pedro Pombeiro 18af9175ac Prevent crash in TestWhisperTestSuite when run isolated 2019-12-30 12:56:05 +01:00