Commit Graph

748 Commits

Author SHA1 Message Date
Dario Gabriel Lipicar c17829bf8d fix: properly fill collectible community data 2023-11-02 11:14:20 -03:00
Cuteivist f57c6a26b5
fix: Correct skip check statement (#4248) 2023-11-02 09:11:48 +01:00
Cuteivist 3c5b0529c7
fix: Add network id for multi tx (#4199) 2023-10-30 10:03:29 +01:00
Cuteivist fa5765cf83
fix: Update timestamp when upserting multi tx (#4224) 2023-10-30 07:53:22 +01:00
Dario Gabriel Lipicar a38b34ae49 feat: cache community metadata in wallet
Fixes #12521
2023-10-26 17:36:45 -03:00
Dario Gabriel Lipicar dca38d1d32 fix: small openseaV2 client fix 2023-10-26 17:36:45 -03:00
Cuteivist e5fbe40b9a
fix: Show community token mint as Mint tx type (#4214) 2023-10-26 20:39:31 +02:00
Cuteivist debf3b6e4d
feat: Added ERC20 community id (#4189) 2023-10-25 18:49:18 +02:00
Roman Volosovskyi 9d59d889f6
[#3833] Increase intreval of new blocks detection command 2023-10-25 14:45:06 +02:00
Roman Volosovskyi 42527723f2
[#4087] Omit history scanning on generated multiacc creation 2023-10-19 15:07:34 +02:00
Sale Djenic b348cca15c feat: added functionality to ask a client to sign a transaction and an endpoint to continue sending using provided signature
This functionality is needed in case the user wants to send a transaction and
signs it using the signature provided by the keycard (or any other compatible way).
2023-10-17 19:42:06 +02:00
Anthony Laibe 21ddaa4b9f feat: arbitrum sepolia 2023-10-17 09:28:13 +02:00
Jonathan Rainville de34ea37f2
fix(keycard_pairings): don't write the pairing file if content is empty (#4113)
Fixes https://github.com/status-im/status-desktop/issues/12342
2023-10-12 16:44:41 -04:00
Dario Gabriel Lipicar 2a5327d8d2 feat: emit event on received community collectibles 2023-10-12 15:42:45 -03:00
Roman Volosovskyi 778753bb57
Fix checking of ERC20 tail on new blocks scanning 2023-10-12 16:21:09 +02:00
Dario Gabriel Lipicar 28cfeb4bc8 feat: refresh collectibles when testnet mode setting changes 2023-10-12 10:18:25 -03:00
Dario Gabriel Lipicar c2ff377585 chore: moved accounts event watcher to accountsevent package 2023-10-12 10:18:25 -03:00
Cuteivist 5674a010cd
feat: Handle Status Mint transaction (#4137) 2023-10-12 12:21:03 +02:00
Dario Gabriel Lipicar 094228871e feat: trigger collectibles refresh on transfer 2023-10-11 15:22:46 -03:00
Dario Gabriel Lipicar addf6e4aaf feat: implemented openseaV2 api changes 2023-10-11 14:41:18 -03:00
Dario Gabriel Lipicar 544e2ea76f feat: removed soon-to-be-deprecated openseaV1 client 2023-10-11 14:41:18 -03:00
Dario Gabriel Lipicar bc33f6d54c feat: removed soon-to-be-deprecated Infura client 2023-10-11 14:41:18 -03:00
Dario Gabriel Lipicar d1f63a8d71 feat: exponential backoff for Alchemy 2023-10-11 14:41:18 -03:00
Dario Gabriel Lipicar 94ae683bc2 fix: ensure full collectible list is fetched from a single provider 2023-10-11 14:41:18 -03:00
Cuteivist 6798d1ac5c
feat: Handled token mint activity (#4126) 2023-10-11 07:10:08 +02:00
Anthony Laibe 39676c8c01 feat: add sepolia toggle 2023-10-09 12:55:29 +02:00
Ivan Belyakov 01526c344a fix(wallet): new blocks were not found 2023-10-06 15:51:57 +02:00
Anthony Laibe 305b520994 fix: token balance 2023-10-06 11:47:23 +02:00
Roman Volosovskyi d82c50b50a
[#3930] Prevent repeated eth_getLogs calls (ERC20 history tail) 2023-10-05 19:45:40 +02:00
Dario Gabriel Lipicar 80f25d5ff7 fix: misc collectibles fixes 2023-10-05 09:40:18 -03:00
Roman Volosovskyi 6a110ca3df
[#3930] Make sure eth_getLogs for outgoing transfers are not executed for erc20 tail 2023-10-05 11:44:09 +02:00
IvanBelyakoff 9d6577049f
Implemented balance history based on transfers (#4022)
* feat(wallet): implement balance history based on fetched transfers
* Added vendor 'ttlcache'
2023-10-04 15:00:12 +03:00
Patryk Osmaczko 53423e58ba fix: use proper migrations for protocol's test database
- use `appdatabse.DbInitializer{}` in tests to ensure consistent migrations

- remove protocol's open database functions due to improper
  initialization caused by missing node config migration

- introduce `PushNotificationServerConfig` to resolve cyclic dependency
  issues
2023-10-03 15:11:58 +02:00
Cuteivist ecc8b4cb55
feat: Wallet activity collectibles model (#4074) 2023-10-03 12:49:04 +02:00
Cuteivist cff96f99e0
Bug: Refresh recipients model (#4076) 2023-10-02 13:46:05 +02:00
Roman Volosovskyi 382fcde74e
Scanning of ERC20 tail of transfers history 2023-10-01 10:09:11 +02:00
Dario Gabriel Lipicar bd6f9b098b feat: implement collectible connection status 2023-09-27 16:44:37 -03:00
Dario Gabriel Lipicar ba1f8ba923 feat: add api to refetch owned collectibles 2023-09-27 16:44:37 -03:00
Dario Gabriel Lipicar ba5cd9c1a4 feat(wallet): add community info to collectibles 2023-09-22 17:55:30 -03:00
Stefan 5c7748dbf7 feat(wallet) add API to cancel current activity filter
Closes status-desktop #11036
2023-09-21 13:56:44 +02:00
Stefan 579f7e4a52 chore(wallet) optimize the filer query
Main changes:

- Use tr_type instead of IN clause
- Use binary (X'...' syntax) directly into the query instead of
  converting DB values to HEX
  - Found to be slightly faster than query parameters in the dedicated benchmark
  - Didn't see much improvement in filter benchmarks
- Tried various combinations of optimizations but without impressive performance results

Benchmark results:

| Name                   | Original   | tr_type   | join     | hex      | no-db     | db_only   |     last |    net_j |
|:-----------------------|:-----------|:----------|:---------|:---------|:----------|:----------|---------:|---------:|
| RAM_NoFilter-10        | 49580229   | 51253242  | 51112462 | 50915133 | 121217817 | 141691008 | 50908642 | 50239712 |
| SSD_NoFilter-10        | 49963604   | 51393588  | 51213038 | 50881483 | 120785679 | 141063467 | 50462767 | 49676867 |
| SSD_MovingWindow-10    | 53695712   | 54155292  | 54161733 | 54061325 | 126966633 | 146866017 | 53479929 | 53350475 |
| SSD_AllAddr_AllTos-10  | 41382804   | 41195225  | 51684175 | 52107262 | 64348100  | 97608833  | 50523529 | 49968321 |
| SSD_OneAddress-10      | 34945275   | 35103850  | 31066429 | 31328762 | 50927300  | 54322971  | 30098529 | 30252546 |
| FilterSend_AllAddr-10  | 39546808   | 37566604  | 38389725 | 38260738 | 114820458 | 125588408 | 37127625 | 36864575 |
| FilterSend_6Addr-10    | 41221458   | 41111225  | 40848288 | 40135492 | 118629700 | 128200467 | 38942521 | 39012100 |
| FilterThreeNetworks-10 | -          | -         | -        | -        | -         | -         | 50058929 | 49854450 |

Update status-desktop: #11036
2023-09-21 13:56:44 +02:00
Cuteivist bc4093299e
feat: Filter by collectibles (#4028) 2023-09-21 08:58:36 +02:00
Michał 4cab5b6106
feat: expose raw token's balance (#4030) 2023-09-20 12:48:08 +02:00
Cuteivist d29c6c5b6f
feat: Calculate finalize status and filter by it (#3969) 2023-09-20 10:30:31 +02:00
Roman Volosovskyi 1dca3adb89
Basic test for findBlocksCommand 2023-09-19 13:17:36 +02:00
Stefan ad971278d9 chore(wallet) remove activity filter dependency on accounts
Require that the activity filter is passed at least one account address
to extract type sent/received

Updates status-desktop #11980
2023-09-19 09:06:57 +02:00
Dario Gabriel Lipicar e337ab4f13 feat: implement collectible ownership status 2023-09-15 18:55:09 -03:00
Dario Gabriel Lipicar e502ba82ce feat: implement single account fetch on add and partial progress report 2023-09-15 18:55:09 -03:00
Dario Gabriel Lipicar b4d5c22050 feat: implement collectibles ownership update timestamp db 2023-09-15 18:55:09 -03:00
Sale Djenic 6894295ac3 feat: register and maintain keycard local pairing file by `status-go`
Closes: #4003
2023-09-13 17:15:32 +02:00
Dario Gabriel Lipicar eb8f3173e4 fix: proper openseaV2 detailed nft unmarshalling 2023-09-13 08:43:09 -03:00
Dario Gabriel Lipicar 8d1992d2e1 fix: protect token list with a mutex 2023-09-13 08:43:09 -03:00
Anthony Laibe 5d1c766382 perf: avoid call to market manager
Optimise call to market manager for symbol that are going to be displayed only
2023-09-13 08:55:35 +02:00
Anthony Laibe 1b07ecaaf7 fix: send nft in other network than mainnet 2023-09-12 16:28:12 +02:00
Anthony Laibe 01babe3632 feat: route-send-collectible 2023-09-12 14:02:42 +02:00
Stefan 195214765b fix(wallet) fix filter by activity type
The activity type filtering was not stable in relation to addresses
filter which was generating unexpected Send/Receive type in the
corner-case when both sender and receiver was in the address list.

Updates status-desktop #11960
2023-09-12 11:58:24 +02:00
Stefan 70341f85a5 fix(wallet) send/receive for duplicate transactions
Brings consistency in case when sender and receiver are both in the
filter address list. This fixes the case of sender and receiver in
addresses and filters out duplicate entries.

Also

- refactor tests to provide support for owners
- adapt TestGetActivityEntriesWithSameTransactionForSenderAndReceiverInDB
  to the use of owner instead of from
2023-09-12 11:58:24 +02:00
Cuteivist e77fc59f5e
feat: Add activity filtering by contract deploy and minting (#4009) 2023-09-11 11:54:37 +02:00
Anthony Laibe 8ba9f38ce7 chore: mark api deprecated 2023-09-11 11:20:51 +02:00
Ivan Belyakov 81b94b7a4e fix(wallet): added ChainID to balance cache, as now it is shared between
services and contains balances for all addresses and chains.
Made rpc chain client return ChainID property on `NetworkID()` method
2023-09-07 15:00:19 +02:00
Cuteivist 83d1354845
chore: Use proper Gwei decimals (#3990) 2023-09-07 13:02:52 +02:00
Anthony Laibe 22fc83de59 feat: create unknown erc20 2023-09-07 10:04:58 +02:00
Stefan 555aae4d0f fix(wallet) filter by sub-transaction chain ID
Closes status-desktop #12077
2023-09-06 21:08:39 +02:00
Stefan c3d2ff0723 chore(wallet) move filter activity sql query in a separate file
This should make the complex query more readable and maintainable
Embed the file content in the complied binary

Updates status-desktop #12077
2023-09-05 22:15:04 +02:00
Ivan Belyakov 20c585a621 fix(wallet): new transfers were not loaded in real time 2023-09-05 14:19:36 +02:00
Ivan Belyakov 24bf9aada5 feat(wallet): move balance cache to a common place and make it a
parameter to share between transfers and balance history in an upcoming
commit.
Had to refactor its interface for that reason.
2023-09-05 12:20:49 +02:00
Cuteivist a0eb205ca9
feat: Estimate block number (#3973) 2023-09-04 12:18:46 +02:00
Stefan c0f32748b4 feat(wallet) lazy load activity info optimization
Trigger async fetching of extra information on each activity filtering
request. Only emit the update event for incomplete entries.

Other changes:

- Make DataEntry light as event payload by making all the fields
  optional
- Add new required fields to the activity DataEntry
- Add collectibles.ManagerInterface to aid testing

Note: this PR keeps compatibility with current master by always
providing non-optional multi-transaction ID. The TODO will be executed
before merging the status-desktop PR.

Experienced a hang on FetchAssetsByCollectibleUniqueID call with:
[{{5 0x21263a042aFE4bAE34F08Bb318056C181bD96D3b} 1209},
{{5 0x9A95631794a42d30C47f214fBe02A72585df35e1} 237},
{{5 0x9A95631794a42d30C47f214fBe02A72585df35e1} 236},
{{5 0x9A95631794a42d30C47f214fBe02A72585df35e1} 832},
{{5 0x9A95631794a42d30C47f214fBe02A72585df35e1} 830},
{{5 0x9A95631794a42d30C47f214fBe02A72585df35e1} 853}]

Updates status-desktop #11597
2023-09-01 19:11:14 +02:00
Dario Gabriel Lipicar d61677403a feat: add extra collection data to collectible header struct 2023-09-01 11:14:29 -03:00
Stefan 71800a19f1 fix(wallet) fix pending transactions notification
Also, add regression test.
2023-09-01 00:18:26 +02:00
Stefan 9cb5e1d3e6 chore(wallet) move computing totalFees to status-go
Updates status-desktop #11597
2023-08-30 13:59:52 +02:00
Ivan Belyakov c24c3966e2 fix(wallet): balance history used a wrong db for accessing accounts.
Balance history was not checked for all chains if no history on
some chain.
Removed `SetInitialRange` from wallet API as internal implementation.
This method was called on adding a brand new Status account to initialize
blocks_range table to avoid transfers history checks.
2023-08-30 10:17:57 +02:00
Stefan 423991fb30 chore(wallet) lookup symbols for wallet activity filter
Updates status-desktop #11597
2023-08-29 17:25:48 +02:00
Andrea Maria Piana 4fba5647d1
Use NewContractMaker for messenger 2023-08-29 14:17:49 +01:00
Cuteivist dd3e408a4e
feat(wallet/activity): Added API for tx and multiTx details (#3939) 2023-08-24 14:23:40 +02:00
Dario Gabriel Lipicar 8d8bd4fc92 fix: handle opensea v2 api error for accounts holding no collectibles 2023-08-24 08:28:36 +02:00
Dario Gabriel Lipicar c494904f18 fix: use all available providers when fetching collectibles by ID 2023-08-24 08:28:36 +02:00
Stefan 524c21834b fix(wallet) propagate multi-transactions IDs to transfers
Mainly refactor API to have control on pending_transactions operations.
Use the new API to migrate the multi-transaction ID from to transfers
in one SQL transaction.
The refactoring was done to better mirror the purpose of pending_transactions

Also:
- Externalize TransactionManager from WalletService to be used by
  other services
- Extract walletEvent as a dependency for all services that need to
  propagate events
- Batch chain requests
- Remove unused APIs
- Add auto delete option for clients that fire and forget transactions

Updates status-desktop #11754
2023-08-22 18:39:42 +02:00
Stefan afe0fcdcf4 fix(wallet) fix test on checking order of async task
Updates status-desktop: #11404
2023-08-22 18:39:42 +02:00
Anthony Laibe 9d782edb4d feat: wallet custom token with communities 2023-08-22 12:41:35 +02:00
Dario Gabriel Lipicar ae17ad57c3 fix: change cursor of opensea v2 ownership endpoint 2023-08-18 11:55:39 -03:00
Dario Gabriel Lipicar ad0e2c2450 feat: implement opensea v2 endpoints 2023-08-18 09:59:38 -03:00
Dario Gabriel Lipicar 3734f03645 chore: split collection data provider 2023-08-18 09:59:38 -03:00
Ivan Belyakov ea38b788b3 tests: Added unit test for copying 'keypairs_accounts' table to the wallet db.
Added a unit test for changing app and wallet DBs passwords.
Refactored geth_backend to simplify and allow wallet db password changing.
Fixed opening database with wrong password.
2023-08-18 09:00:56 +02:00
Ivan Belyakov 2df9df10ab fix(tests): moved test db setup to a common place 't/helpers', created
interface for initializing db, which is implemented for appdatabase and
walletdatabase. TBD for multiaccounts DB.
Unified DB initializion for all tests using helpers and new interface.
Reduced sqlcipher kdf iterations for all tests to 1.
2023-08-18 09:00:56 +02:00
Ivan Belyakov d106b449b6 feat(wallet): move wallet-related tables to a dedicated db.
The only place where appDB is used in wallet is activity,
which refers to `keycards_accounts` table. So a temporary
table `keycards_accounts` is created in wallet db and updated
before each activity query.
2023-08-18 09:00:56 +02:00
Dario Gabriel Lipicar 25ff1dd758 feat: make collectibles api support multiple clients 2023-08-15 10:17:10 -03:00
Dario Gabriel Lipicar d6aae82566 chore: move multi-client scheduler implementation to separate file 2023-08-15 10:17:10 -03:00
Michal Iskierko 9d0acc2265 feat(OwnerToken): Owner token and master token deployment
Adding new smart contracts and generated go files.
Deploy token owner function and master token address getter.
Adding deployer and privilegesLevel columns to community_tokens table.
Passing addressFrom to API calls.

Issue #11250
2023-08-14 20:57:00 +02:00
Dario Gabriel Lipicar 4b394e088e fix: minor collectible provider fixes 2023-08-09 11:21:59 -03:00
Dario Gabriel Lipicar cd83a5fe00 chore: make use of new collectibles data DB 2023-08-09 11:21:59 -03:00
Dario Gabriel Lipicar 33c116f7b1 chore: implement collectibles data db cache 2023-08-09 11:21:59 -03:00
Dario Gabriel Lipicar e30cba4303 fix: parse collectible ownership response from v3 alchemy endpoint properly
Fixes #11832
2023-08-08 13:20:36 -03:00
Dario Gabriel Lipicar d5974dd52e feat: fetch collection metadata when missing 2023-08-04 09:54:18 -03:00
Cuteivist 1f510eae70
fix: Query token id for transactions (#3851) 2023-08-04 13:40:36 +02:00
Cuteivist 06c4eb5df9
feat: Handle collectible mint (#3852) 2023-08-04 12:47:22 +02:00
Dario Gabriel Lipicar 078f71a235 feat: use new collectible account ownership providers 2023-08-03 12:53:00 -03:00
Dario Gabriel Lipicar c92a10b846 feat: implement alchemy and infura collectible account ownership endpoints 2023-08-03 12:53:00 -03:00
Dario Gabriel Lipicar 51d676bb08 chore: reorganized collectibles clients code 2023-08-03 12:53:00 -03:00
Dario Gabriel Lipicar c2ac108556 chore: rework data structures to accomodate new providers 2023-08-03 12:53:00 -03:00
Anthony Laibe 0cd140c9b2 feat: change network selection strategy
1) Only fetch history for networks that match test mode enabled
* Trade Off: it will be only refetch in 12 hours so changing test mode won't trigger a refetch until app is restarted or 12 hours.
I think it is ok as change test mode is not a common use case
2) Do not consider networks that are enabled or not as this can be change more often than every 12 hours
2023-08-02 12:45:37 +02:00
Anthony Laibe 21d9d17cdc chore: reduce batch size for goerli arbitrum/optimism 2023-08-01 13:37:45 +02:00
Stefan eb8d74e1ae feat(wallet) add request id to activity events
Add and use the optional chainID in the wallet event structure.

Updates status-desktop #11380
2023-07-31 18:22:13 +02:00
Anthony Laibe a139871383 fix: celer check on tokens 2023-07-31 13:16:33 +02:00
Dario Gabriel Lipicar 10a42e639d feat: parallel collectibles fetching per account and chain 2023-07-27 12:12:03 -03:00
Dario Gabriel Lipicar 57424e076c chore: moved collectibles ownership cache to db 2023-07-27 12:12:03 -03:00
Dario Gabriel Lipicar 85d8e83394 feat: trigger collectibles fetch on new account 2023-07-27 12:12:03 -03:00
Dario Gabriel Lipicar 5ba5611a8d feat: implement collectibles service 2023-07-27 12:12:03 -03:00
Dario Gabriel Lipicar b1cf54974e chore: make opensea client return common types 2023-07-27 12:12:03 -03:00
Dario Gabriel Lipicar 849be69299 chore: use single opensea client instance 2023-07-27 12:12:03 -03:00
Dario Gabriel Lipicar f138964a9c chore: move scheduler to async dir 2023-07-27 12:12:03 -03:00
Dario Gabriel Lipicar b5224b3cc5 chore: rename and reorder types 2023-07-27 12:12:03 -03:00
Stefan d956a3e854 feat(wallet) chain id multi-transaction filtering
Updates status-desktop #11631
2023-07-26 13:54:55 +02:00
Anthony Laibe e5ce2c7c03 feat: make activity check optional 2023-07-26 09:07:55 +02:00
Sale Djenic 6a5cdbaa8b fix: review comments applied
`GetKeypairs(includeRemoved bool)` removed
`GetActiveKeypairs()` added
`GetAllKeypairs()` added

`GetAccounts(includeRemoved bool)` removed
`GetActiveAccounts()` added
`GetAllAccounts()` added
2023-07-25 18:11:05 +02:00
Sale Djenic d5bba5d80f chore: wallet accounts improvements - soft delete 2023-07-25 18:11:05 +02:00
Cuteivist 84bfdf4aab
feat: Add contract deployment tx type (#3770) 2023-07-20 16:04:30 +02:00
Stefan 8926990c2d chore(wallet) retrieve all required activity header data as metadata
Add missing fields to activity metadata.

Updates status-desktop #11173
2023-07-20 11:34:41 +02:00
Dario Gabriel Lipicar 1f379aec1f feat: implemented multi chain collectible ownership provider 2023-07-19 07:58:05 -03:00
Anthony Laibe 0f8347dc59 feat: get chain id for url api 2023-07-18 13:50:13 +02:00
Stefan 64f7706567 chore(wallet) index activity filter columns to speed up queries
Implemented required benchmarks

Also

- refactor tests as needed
- don't filter temporary or incomplete transfers (transfers.loaded != 1)

Updates status-desktop #11351
2023-07-14 16:11:14 +02:00
Stefan e6705f264c chore(wallet) refactor activity tests 2023-07-14 16:11:14 +02:00
Dario Gabriel Lipicar 40eed0fd01 fix: ensure correct initialization order for wallet and mesenger services 2023-07-13 16:16:31 -03:00
Anthony Laibe 104d9c8ff6
feat: add related chain id to networks (#3697) 2023-07-13 16:03:49 +02:00
Stefan 8bfcb80ed6 fix(wallet) fix scan to/from hash when for multi_transactions 2023-07-13 00:24:36 +02:00
Dario Gabriel Lipicar 9ee523be99 fix: remove unsupported chainIDs from opensea client 2023-07-12 15:49:18 -03:00
Stefan 69412cf3a3 fix(wallet) fix scan of default NULL multi_transactions.*_network_id 2023-07-12 10:34:32 +02:00
Anthony Laibe 23492467b9 feat: clear balance cache after usage 2023-07-12 09:23:56 +02:00
Stefan 3ce851b84b feat(wallet) report new multi-transactions updates
Updates status-desktop #11233
2023-07-11 22:34:43 +02:00
Ivan Belyakov 09dff82db5 feat(wallet): Move pending transactions to `transactions` module.
Handle creation and deletion of pending transactions automatically
on status-go side.
2023-07-11 16:07:42 +02:00
Dario Gabriel Lipicar 66a21aa7e4 feat(wallet): detect ETH L2 to L1/L2 bridge transactions 2023-07-10 09:59:05 -03:00
Dario Gabriel Lipicar e6711c8cc8 feat(wallet): detect ETH L1 to L2 bridge transactions 2023-07-10 09:59:05 -03:00
Dario Gabriel Lipicar fb6e2a41f7 chore(wallet): reorganize multitx processing code 2023-07-10 09:59:05 -03:00
Anthony Laibe bc92df79d5 fix: fourbyte integration no arg 2023-07-10 12:26:42 +02:00
Stefan 9a30674590 feat(wallet) deliver pending changed events to clients
Add wallet events feed to TransactionManager and send pending changed
events on add and delete

Also

- Remove hardcoded values in the filter query
- Small improvement to query

Updates status-desktop #11233
2023-07-07 15:06:14 +02:00
Anthony Laibe dffcb708ea feat: add fourbyte github integration 2023-07-06 11:31:14 +02:00
Anthony Laibe b5d5fa828c fix: allow l2 bridges 2023-07-05 13:14:19 +02:00
Stefan 0b9555bf31 fix(wallet) returning wrong address identity in filter
This fixes returning address identity after that was changed by
the real to/from fix. Previously the address was wrongly used as from

Updates status-desktop #11233
2023-07-05 10:36:45 +02:00
Anthony Laibe 1cb1c63e82 fix: bonder fees must be lower than amount 2023-07-04 22:32:51 +02:00
Stefan 3b0f4ee2c1 fix(wallet) fix GetMultiTransactions for to_amount nil
Updates status-desktop #11233
2023-07-04 16:49:08 +02:00
Stefan 293fa9d248 fix(wallet) fix filtering failing with NULL values in to_address
Added test to validate the fix

Also change JOIN to LEFT JOIN between `multi_transactions` and `transactions` table so that we return all entries from `multi_transactions` table even if there is no matching entry in `transactions` table, which might be the case for multi transactions that that could not be detected. This will postpone the error case until we get into details of the multi transaction.

Updates status-desktop #11233
2023-07-04 15:51:33 +02:00
Stefan e5e5229e6a feat(tests) disable 4byte remote server dependent tests 2023-07-03 16:50:48 +02:00
Stefan c61a4000d8 feat(wallet) make remaining filter apis async
Implement activity.Scheduler to serialize and limit the number of
calls on the activity service. This way we protect form inefficient
parallel queries and easy support async and rate limiting based on the
API requirements.

Refactor the activity APIs async and use the Scheduler for managing
the activity service calls configured with one of the two rules: cancel
ignore.

Updates status-desktop #11170
2023-07-03 16:50:48 +02:00
Ivan Belyakov 1efecbf73b Fixed concurrent write to blockchain status map 2023-06-30 14:59:07 +02:00
Ivan Belyakov 2f515447d2 fix(wallet): Fixed missing loop for transfers loading.
Made channel for transfers loading buffered - non-blocking
2023-06-28 16:18:43 +02:00
Dario Gabriel Lipicar ce45a7b1e8 fix: properly initialize NFTMetadataProvider in wallet service 2023-06-28 09:22:40 -03:00
Stefan f07a79cd18 fix(wallet) fix crash if to_address is NULL in transfers
Add nill tests for TestGetRecipients, GetOldestTimestamp
Also fix returning duplicate addresses in TestGetRecipients

Updates status-desktop #11170
2023-06-28 09:55:43 +02:00
Dario Gabriel Lipicar afb9621f01 fix: protect OpenSea instances with a mutex 2023-06-27 11:09:02 -03:00
Stefan ea7a389075 feat(wallet) api to retrieve last activity timestamp
Also fix the lookup source for `GetRecipients` API

Updates status-desktop #11169
2023-06-22 14:06:55 +02:00
Dario Gabriel Lipicar 0f305df3b5 fix: identify ETH asset properly in swap transactions 2023-06-21 12:20:22 -03:00
Dario Gabriel Lipicar bf54a57780 fix: correct transfer type detection 2023-06-21 12:10:25 -03:00
Anthony Laibe 8ef3c52f4d fix: parse input data with bytes 2023-06-21 11:42:40 +01:00
Stefan 8e63f44735 feat(wallet) filter activity by erc20
Main changes:

- Refactor activity API to propagate token identities.
- Extend service to convert token identities to symbols for filtering
  multi-transaction
- Filter transfers, pending_transactions and multi-transactions based
  on the provided token identities
- Return involved token identities in activity API
- Test token filtering

Also:

- Fixed calling cancel on a filer activity completed task to release
  resources

Notes:

- Found limitations with the token identity which complicates things
  by not allowing to filter by token groups (like token-code does)

Updates status-desktop #11025
2023-06-20 14:06:49 +02:00
Dario Gabriel Lipicar 60b160997c fix(wallet): properly parse multitransaction amounts 2023-06-16 10:05:49 -03:00
Emil Sawicki ad9ad4026b feat: add signature id 2023-06-16 12:10:22 +01:00
Anthony Laibe 9d5b123c95 feat: add api to parse tx input 2023-06-16 12:10:22 +01:00
Anthony Laibe 2375b10b03 feat: load token list in code 2023-06-16 09:00:06 +01:00
Dario Gabriel Lipicar e26c2a7095 feat: add transaction amounts to activity entry 2023-06-15 22:04:19 -03:00
Dario Gabriel Lipicar 86350379b1 chore: rework activity test utils 2023-06-15 22:04:19 -03:00
Stefan 6f2c338f72 feat(wallet) add filter api to retrieve recipients of a wallet
The new API returns all known recipients of a wallet, by
sourcing transfers, pending_transactions and multi_transactions tables
The API is synchronous. Future work will be to make it async.
In some corner cases, when watching a famous wallet, it can
be that there are too many recipients to be returned in one go. Offset
and limit can be used to paginate through the results.

Updates status-desktop #10025
2023-06-15 20:40:44 -03:00
Dario Gabriel Lipicar a2b1640ad7 fix(wallet): respect limits for collectibles fetching 2023-06-15 16:44:02 -03:00
Michal Iskierko ccde92377d feat(BurnCollectibles): Add burn collectibles functionality
Change smart contract with new API.
Update gas amount for deployment.
Add Burn() and EstimateBurn() functions.
Add RemainingSupply() functions.

Issue #10816
2023-06-15 15:38:19 +02:00
Ivan Belyakov 30af25198e feat(wallet): Improved performance of fetching tranfers for Sequential
fetch strategy:
Before:
 - block fetching commands for different accounts were in the same wait
group, making them dependent on each iteration.
 - transfers loading command was checking database for new unloaded
blocks on timeout and was in the same wait group with block fetching, so
it was often blocked until all block fetching commands finish for
iteration.
Now:
 - block fetching commands run independently for each account
 - transfers fetching command is run once on startup for unloaded blocks
from DB
 - fetch history blocks commands are launched once on startup for
accounts with no full history loaded
 - transfers are loaded on each iteration of block range check
without waiting for all ranges to be checked
2023-06-14 15:12:12 +02:00
Ivan Belyakov 2b02968819 fix(wallet): Fixed nil pointer dereferencing for multiTransaction.ToAmount
when sending tx on Optimism
2023-06-14 07:24:18 -03:00
Dario Gabriel Lipicar ffc959a9e3 chore(wallet): split multitransaction command from full struct
Part of #10791
2023-06-13 07:58:40 -03:00
Stefan 4c8fd538ba feat(wallet) return offset in the activity result
Updates status-desktop #10994
2023-06-12 23:04:12 +02:00
Stefan d8eb038d7d feat(wallet): make filer API async
Refactor the filter interface to be an async call which returns
the result using a wallet event
A call to the filter API will cancel the ongoing filter and receive
an error result event

Closes status-desktop #10994
2023-06-12 15:42:26 +02:00
Stefan 87ce6cfbcd feat(wallet) filter activity by status
Main changes:

- extend DB query include status for transactions and pending_transactions
- extend DB query to aggregate data from complex activity entries
- extend tests to include the new status values

Other changes:

- Improve tests with mocked addresses in DB to have a true to/from state

Update status-desktop #10746
2023-06-10 11:43:00 +02:00
Dario Gabriel Lipicar 4eb3d7485a fix(wallet): handle error when calling ERC721 TokenURI function
Fixes #10959
2023-06-08 10:37:20 -03:00
Stefan 5b6f7226bb feat(wallet) extract json blobs and add custom migration support
Extended the migration process with a generic way of applying custom
migration code on top of the SQL files. The implementation provides
a safer way to run GO code along with the SQL migrations and possibility
of rolling back the changes in case of failure to keep the database
consistent.
This custom GO migration is needed to extract the status from
the JSON blob receipt and store it in transfers table.

Other changes:
- Add NULL DB value tracking to JSONBlob helper
- Index status column on transfers table
- Remove unnecessary panic calls
- Move log_parser to wallet's common package and use to extract token
  identity from the logs

Notes:
- there is already an index on transfers table, sqlite creates one for
  each unique constraint therefore add only status to a new index
- the planned refactoring and improvements to the database have been
  postponed due to time constraints. Got the time to migrate the data
  though, extracting it can be done later for a more efficient
  implementation

Update status-desktop #10746
2023-06-08 13:54:40 +02:00
Dario Gabriel Lipicar d73c886d3b fix(wallet) limit nft description length
Fixes #10923
2023-06-07 15:11:29 -03:00
Ivan Belyakov d74d930b70 feat(wallet): enable sequential transfers fetching by default.
"EventNewTransfers' event is sent on each block processed, otherwise
it could take minutes or longer before first 'EventNewTransfers'
is sent.
2023-06-07 08:45:45 +02:00
Dario Gabriel Lipicar 449314a4dc feat(Wallet) detect uniswapV3 swap events
Part of #10251
2023-06-05 12:08:18 -03:00
Dario Gabriel Lipicar 2fc79fb9b5 feat(Wallet) detect uniswapV2 swap events
Part of #10251
2023-06-05 12:08:18 -03:00
Dario Gabriel Lipicar 7557f0c799 feat(Wallet): add function to get erc20 token by contract address
Part of #10251
2023-06-05 12:08:18 -03:00
Dario Gabriel Lipicar da8e8716b2 fix(wallet): Fix block hash for detected blocks with ETH transfers
Part of #10251
2023-06-05 12:08:18 -03:00
Ivan Belyakov ad1b8b6d43 fix(wallet): added `remove` event for accounts. Stop loading
transfers for removed account

Updates #10246
2023-06-05 08:22:08 +02:00
IvanBelyakoff 7adfbb5467
feat(wallet): SequentialFetchStrategy improvements: (#3551)
- reverted a change that stopped looking for ERC20 transfers if no nonce
  and balance change found within a block range for ETH
- implemented sending EventRecentHistoryReady event at a proper time
- moved EventFetchingRecentHistory event to Strategy type as it does not make
sense to send this event in loop
- moved iterating through blocks logic to inside of `loadTransfers` command, which
now accepts a block range.
- reuse `uniqueHeaders` function in commands.go
- clean up

Updates #10246
2023-06-01 16:09:50 +03:00
Anthony Laibe 645cd5d03b feat: return only enabled networks from the reader 2023-05-29 08:37:08 +02:00
IvanBelyakoff 2dc6700e38
chore(wallet): fix annoying error on missing argument while logging (#3532)
error on token fetch from store
2023-05-26 11:41:33 +03:00
IvanBelyakoff dc84afb751
Add new transfers fetching for sequential strategy (#3520)
* chore(wallet): refactor sequential transfers commands

* feat(desktop/wallet): add fetching new blocks and transfers for
transfers SequentialFetchStrategy

Updates #10246
2023-05-26 11:27:48 +03:00
Sale Djenic 03d9af0b95 feat: `AddressExists` endpoint added 2023-05-26 07:12:40 +02:00
Roman Volosovskyi ee4003279a
Changes necessary to restore wallet on mobile app 2023-05-25 21:08:34 +02:00
Sale Djenic 0eca7f5088 fix: issues after rebase fixed 2023-05-25 19:46:47 +02:00
Sale Djenic eeaaf0ce3f feat: accounts improvements applied
- old `accounts` table is moved/mapped to `keypairs` and `keypairs_accounts`
- `keycards` table has foreign key which refers to `keypairs.key_uid`
- `Keypair` introduced as a new type
- api endpoints updated according to this change
2023-05-25 19:46:47 +02:00
Michal Iskierko 1c17fbeacc feat(RemoteBurn): Add remote burn collectibles functionality
* Updating smart contract (new burn function)
* remoteBurn() and estimateRemoteBurn() functions
2023-05-25 13:34:53 +02:00
Stefan e78a73bd9f feat(Wallet) complete the filter API
It uses the current data only and doesn't extend with new types or
include new features in activity sources DBs.

Major changes:
- Partially filter by chain IDs
- Partially filter by Status if it is the case
- Partially filter by token types
- Filter by counterparty addresses
- Use wallet accounts for TO/FROM instead of filters

Closes: #10634
2023-05-23 14:48:29 +02:00
Anthony Laibe 8b05ba6d8b fix: check balance for address details 2023-05-22 09:13:09 +02:00
Andrea Maria Piana 6acb34bab7 Allow setting up wallet with new config
This commit does a few things:

1) Extend create/import account endpoint to get wallet config, some of
   which has been moved to the backend
2) Set up a loop for retrieving balances every 10 minutes, caching the
   balances
3) Return information about which checks are not passing when trying to
   join a token gated community
4) Add tests to the token gated communities
5) Fixes an issue with addresses not matching when checking for
   permissions

The move to the wallet as a background task is not yet complete, I need
to publish a signal, and most likely I will disable it before merging
for now, as it's currently not used by desktop/mobile, but the PR was
  getting to big
2023-05-19 13:23:21 +01:00
IvanBelyakoff 17aaaf1dca
fix(desktop/wallet): fix bug in balance_cache - balances and nonces (#3509)
were stored in cache by pointers, which caused falsy cache hits in loop
because pointers with same address were created for different block
numbers. Now cache uses block numbers of uint64 as key, which can
overflow but it is not a problem since we use this cache for values
comparison, not as user data.
Fix crash on nil pointer in log.
Remove some unused code.
Protect nonceRanges with mutex while reading.

Updates #10246
2023-05-19 14:46:54 +03:00
IvanBelyakoff 94c7cd32af
feat(wallet): Implemented SequentialFetchStrategy for transfers. (#3480)
By default OnDemandFetchStategy is still used.

Updates #10246
2023-05-19 11:19:48 +03:00
Stefan c020222f1b feat(Wallet): add activity filter api prototype
Add the possibility of retrieving the metadata of wallet activity based
on the given filter criteria.

Current implementation relies that after fetching the metadata, user
will follow up with more requests for details. However, after some
experimenting I'm considering extracting all required information
for the summary viewing while filtering. This way there will be no
need for another batch requests for transfers, multi-transactions and
pending transactions to show the summary. Only when user wants to see
the details for one will specifically request it.

For this first prototype, the filter criteria is limited to:
- time
- type
- addresses

Major changes:
- Add the filter definition to be used in propagating the filter
  information
- Add GetActivityEntries API to return the list of activity entries
  for the given addresses/chainIDs by a view in the complete list
- GetTransfersForIdentities to batch retrieve further details of the
  transfers
- GetPendingTransactionsForIdentities to batch retrieve further details
  of the pending transactions
- Added a new package testutils for tests.
- Added tests

Updates status-desktop #10366
Closes status-desktop #10633
2023-05-14 23:05:28 +02:00
Stefan c5a8b40d9c fix(wallet) don't delete pending transactions for multi-transaction
This is done in Nim after the watched pending transaction changes state.

Note: tried to delete all transactions, not only multi-transaction,
as soon as the state is downloaded but it didn't work: noticed crashes
with minted transactions after restart.

Updates status-desktop #10474
2023-05-08 11:24:37 +02:00
IvanBelyakoff cb9f2dd7d2
feat(wallet): refactor transfers to allow multiple transfer fetching (#3432)
strategies.

On mobile devices we might not want to load all transfers history
to avoid high internet traffic consumption or to save some space.
On desktop we will load all transfers in background.

- Moved current fetching transfers implementation to
OnDemandFetchStrategy
- Implemented QueuedAtomicGroup to cache outgoing commands above a
threshold
- Refactored some type names, methods for better clarity:
  - LastKnownBlock name was misused
  - Block renamed to BlockDAO to clarify what it does and avoid
confusion with geth Block type
- Cleanup - removed unused parameters and types' members
- Changed tests to use in memory DB for speedup
- Added more logging and comments

Updates #10246
2023-05-08 08:02:00 +02:00
Anthony Laibe 722390c784 fix: reponse may contains less results than expected 2023-05-03 08:38:29 +02:00
Stefan a6d553c937 fix(wallet) ClientWithFallback handling state errors required by BH
The "not found" and "no contract" expected error is caught and wrapped
by ClientWithFallback. The fetching of balance history of next blocks
is aborted which is not desired.

Fix by not treating the error as a connection error in
ClientWithFallback.
2023-05-02 12:07:06 +02:00
Anthony Laibe dd7c59423f feat: add saved address created at 2023-04-22 10:22:43 +02:00
Dario Gabriel Lipicar 4ef5eba6b5 chore: move chainIDs to constants file 2023-04-19 08:30:10 -03:00
Dario Gabriel Lipicar 633c8d1ed5 chore: clean up passing api keys to the wallet service 2023-04-19 08:30:10 -03:00
Dario Gabriel Lipicar c8f0ceccc8 feat: add api to get list of owners from a given nft contract
Fixes #10290
2023-04-19 08:30:10 -03:00
Michal Iskierko 4e3ba2941c fix(AddTokenOwners): Move AddTokenOwners from MintTo
Issue #10161
2023-04-13 11:43:48 +02:00
Sale Djenic 1754c23763 chore: improvements to `GetWalletToken` function 2023-04-06 19:09:13 +02:00
Dario Gabriel Lipicar ff755a1176 fix: add supported chains to OpenSea 2023-04-06 13:42:38 -03:00
Dario Gabriel Lipicar 5bad40c8be feat: add collectible animation info to data fetched from Opensea 2023-04-03 12:32:32 -03:00
Anthony Laibe c84d6fcc35 feat: remove consecutive 2023-03-31 19:22:53 +02:00
Anthony Laibe 9c33049b4f chore: update connected on global failure 2023-03-31 19:22:53 +02:00
Anthony Laibe 0c8ce79941 feat: returns tokens even when error 2023-03-31 19:22:53 +02:00
Anthony Laibe 540001e137 feat: chunk correctly coinmarketcap calls 2023-03-31 19:22:53 +02:00
Anthony Laibe e92049b9ca fix: wallet token return on no network 2023-03-31 19:22:53 +02:00
Anthony Laibe d86446ad53 chore: adding debug 2023-03-31 19:22:53 +02:00
Anthony Laibe 2baa8e8ead feat: filter token by valid chain ID 2023-03-31 19:22:53 +02:00
Anthony Laibe c9a8ab13c2 fix: set is connected for chain without fallback 2023-03-31 19:22:53 +02:00
Anthony Laibe ec5449c2bd fix: collectible status event 2023-03-31 19:22:53 +02:00
Anthony Laibe 62220cedee feat: update chunk limit for token 2023-03-31 19:22:53 +02:00
Anthony Laibe fae86fcfb7 feat: group all blockchain event 2023-03-31 19:22:53 +02:00
Anthony Laibe ba75bda39e feat: event on error 2023-03-31 19:22:53 +02:00
Dario Gabriel Lipicar 4e222cc404 feat: collectibles cache 2023-03-31 12:24:33 -03:00
Dario Gabriel Lipicar f097d3430a feat: reduce usage of OpenSea api key 2023-03-31 12:23:50 -03:00
Dario Gabriel Lipicar d8ee8786ff fix: simplified tokenURI retrieval. Extended NFTMetadata struct. 2023-03-30 09:17:53 -03:00
Dario Gabriel Lipicar 48e16317a7 fix: added global lock for Opensea requests. Increased number of retries. 2023-03-29 18:20:09 +02:00
Sale Djenic d30c88b80e neat: accounts and wallet api sorted out
Unused endpoints removed, new ones with more meaningful naming are added and
their purposes were revised.
2023-03-28 16:19:27 +02:00
Anthony Laibe 60dbc5d44d feat: config optimisation 2023-03-28 15:11:35 +02:00
Dario Gabriel Lipicar a1e7eed141 feat: fetch NFT metadata from Communities 2023-03-24 15:17:41 -03:00
Khushboo-dev-cpp a3f1a84d29
chore: Improve error logging for OpenSea errors / generic json unmarshalling errors (#3298)
fixes #8798
2023-03-20 16:53:39 +01:00
Dario Gabriel Lipicar bd82250cf7 feat(Wallet): add API to fetch collectibles by owner and contract address 2023-03-20 10:52:16 -03:00
Khushboo-dev-cpp bbec93b501
fix: Return token items even when market is down (#3291) 2023-03-20 14:44:23 +01:00
IvanBelyakoff fe4e079bea
fix(wallet): fix crash on market price cache concurrent write (#3307) 2023-03-17 09:33:20 +03:00
IvanBelyakoff 612efb9b89
fix(Wallet): Fix uninitialized token map accessed (#3289) 2023-03-15 18:10:36 +03:00
IvanBelyakoff 2c9714f0f8
fix(wallet/coingecko): Fix coingecko crash on race writing to tokens map (#3274) 2023-03-14 21:59:10 +03:00
IvanBelyakoff 804b5b43b4
feat(Wallet): implement token list from Uniswap (#3273)
Fixes #8807
2023-03-14 20:33:05 +03:00
Dario Gabriel Lipicar bac7eb08ca feat(Wallet): add API to fetch collectibles in a paginated way 2023-03-10 11:27:03 -03:00
Dario Gabriel Lipicar 7b6fc87f89 fix(wallet): use OpenSea api key only for mainnet endpoint 2023-03-10 08:55:58 -03:00
Stefan f607164122 feat(wallet) add GetMultiTransactions wallet API
Updates status-desktop #7663
2023-03-07 19:38:31 +04:00
Anthony Laibe bbe4e57f0f fix: use correct chain client 2023-03-07 16:24:32 +01:00
Dario Gabriel Lipicar 3f3e8f8894 feat(Wallet): add opensea asset fetching by contract address and token id 2023-03-07 10:33:50 -03:00
Dario Gabriel Lipicar 67bcf10466 feat(Wallet): handle erc721 transfers 2023-03-07 10:33:50 -03:00
IvanBelyakoff 48078393f9
feat: Update saved addresses DB to store ENS, isTest flag and chains (#3183)
prefixes. Changed primary keys and API methods.
Fixed tests and added new ones.
Fixed saved addresses and transaction tests to use ':memory:' sqlite
DB instead of a tmp file to speed up testing by hundred of times.

Fixes #8599
2023-03-06 16:26:46 +03:00
Stefan 7c7c3a1f13 feat(wallet): keep multi-transactions relation after transaction is complete
Updates status-desktop #7663
2023-03-01 23:34:00 +04:00
Stefan 226fa7d696 chore(wallet) increase the all-time fetching sample count
Also:
- extend maximum expected DB entries to fetch
- fix test TestBalanceHistoryGetWithoutOverlappingFetch passing because of all-time stride side effect

Closes status-desktop #9624
2023-03-01 19:55:07 +04:00
Anthony Laibe 42ee98e295 feat: Add circuit breaker for market provider 2023-03-01 08:58:00 +01:00
Stefan 411607d43d chore(wallet): hardcode specific average block duration for L2
The L2 networks have their own block number and average block duration,
so we need to hardcode to keep the logic simplified.

Updates status-desktop: #9582
2023-02-28 12:03:43 +04:00
Anthony Laibe 60e1978bb5 feat: fallback rpc endpoint 2023-02-28 08:43:57 +01:00
Anthony Laibe e543fda4b5 feat: add coingecko fallback 2023-02-27 09:55:04 +01:00
Anthony Laibe f81758ec6e feat: add time to check 2023-02-23 18:23:29 +01:00
Khushboo-dev-cpp dd3f4058c2
fix: Return statement after sendToL2 was missing (#3217)
fixes #9483
2023-02-22 08:39:26 +01:00
dlipicar 1d1a95091d
feat: implement currency formatting calculation (#3191) 2023-02-17 11:11:07 -03:00
Stefan f4f6b25302 feat: convert history balance tokens to fiat
Add history.exchange sub-package with following responsibilities

- fetch and caches daily exchange rates for tokens
    - Partial update if missing form yesterday back
- Implement all time fetching special case
- Fetches the price of the token after merging entries for the selected
chains

history.service package changes

- Return ValuePoint instead of DataPoint
    - Value point contains the value in fiat as float64 instead
- Convert merged values from tokens balance (wei) to fiat

Other changes

- Move start/stop balance history to startWallet/stopWallet
- Move cryptocompare to thirdparty package to avoid recursive import
- Rename DataPoint.Value to DataPoint.Balance
- Don't merge entries for single chain
2023-02-15 18:17:21 +04:00
Andrea Maria Piana b69042e7d7 Fix duplicated notifications on receiving contact update
This commit fixes an issue where when accepting a contact request
the other end would display an extra notification.

It also changes WaitOnResponse to collect results. This should make
tests less flaky, since sometimes messages are processed in different
batches.
Now we need to be though exact on what we expect from the response (i.e
use == instead of >, otherwise the same behavior applies)
This uncovered a couple of issues with messenger.Merge, so I have moved
the struct to use a map based collection instead of an array.
2023-02-14 09:22:37 +00:00
Michal Iskierko 8acc46f758 feat(collectibles): Mint collectibles (ERC-721):
Add testing smart contract and go api.
Add collectibles service.

Issue #3051
2023-02-13 12:59:55 +01:00
Khushboo-dev-cpp 9b32c6067a
fix: Fix issue where best routes with not enough balance is returned (#3164) 2023-02-08 16:22:50 +05:30
Stefan 8934426cbd fix: request token balance for overridden tokens
Fix balance for tokens overrides are not requested in case of fall back
when etherscan bulk fetch not working.

Updates status-desktop: #9091
2023-02-07 19:38:53 +04:00
Anthony Laibe b6a33089c8 fix: ensure bonder fees are lower than amount 2023-02-06 17:18:30 +01:00
Sale Djenic d44fa42cbf feat: `GetDerivedAddressForPath` endpoint added 2023-02-01 09:03:59 +01:00
Stefan d024d36718 fix: fix wallet.history.TestServiceAtEnd
First check was relying on sorting result of two items that had the same
weight and is not guaranteed to be deterministic.
2023-01-30 20:11:01 +04:00
Stefan 90b39eeb41 fix: merge balance history using block time
This change improves on the previous implementation which used
the block number that doesn't work with incompatible blockchains e.g. L1 vs. L2

Closes: #9205
2023-01-27 16:28:59 +04:00
Stefan a2ff03c79e feat: retrieve balance history for tokens and cache it to DB
Extends wallet module with the history package with the following
components:

BalanceDB (balance_db.go)

- Keeps track of balance information (token count, block, block timestamp)
for a token identity (chain, address, currency)
- The cached data is stored in `balance_history` table.
- Uniqueness constrained is enforced by the `balance_history_identify_entry`
UNIQUE index.
- Optimal DB fetching is ensured by the `balance_history_filter_entries`
index

Balance (balance.go)

- Provides two stages:
    - Fetch of balance history using RPC calls (Balance.update function)
    - Retrieving of cached balance data from the DB it exists (Balance.get
    function)
- Fetching and retrieving of data is done for specific time intervals
    defined by TimeInterval "enumeration"
- Update process is done for a token identity by the Balance.Update function
- The granularity of data points returned is defined by the constant
increment step define in `timeIntervalToStride` for each time interval.
- The `blocksStride` values have a common divisor to have cache hit
between time intervals.

Service (service.go)

- Main APIs
    - StartBalanceHistory: Regularly updates balance history for all
    enabled networks, available accounts and provided tokens.
    - GetBalanceHistory: retrieves cached token count for a token identity
    (chain, address, currency) for multiple chains
    - UpdateVisibleTokens: will set the list of tokens to have historical
    balance fetched. This is a simplification to limit tokens to a small
    list that make sense

Fetch balance history for ECR20 tokens

- Add token.Manager.GetTokenBalanceAt to fetch balance of a specific
block number of ECR20.
- Add tokenChainClientSource concrete implementation of DataSource
to fetch balance of ECR20 tokens.
- Chose the correct DataSource implementation based on the token
"is native" property.

Tests

Tests are implemented using a mock of `DataSource` interface used
to intercept the RPC calls.

Notes:

- the timestamp used for retrieving block balance is constant

Closes status-desktop: #8175, #8226, #8862
2023-01-25 22:25:50 +04:00
Anthony Laibe e8cd779e35 feat: api to return service up or down 2023-01-21 13:11:03 +01:00
dlipicar c2a8dd8a0c
feat(wallet): add price cache (#3107) 2023-01-19 11:49:48 -03:00
Andrea Maria Piana ed9ca8392c Upgrade linter to 1.50.0 2023-01-13 17:52:03 +00:00
dlipicar 77a56e2b93
Feature/8640 unified currency rounding (#3079)
* feat(@desktop/wallet): added iso4217 library for fiat currency display decimals

* feat(@desktop/wallet): added token peg info and use numbers for token market values

* feat(@desktop/wallet): added
2023-01-12 14:54:14 -03:00
Anthony Laibe 883f9677c5 feat: nonce management in multi tx 2023-01-12 12:56:57 +01:00
Anthony Laibe 04de916cba fix: wallet downloader for l2 chain 2023-01-11 09:56:08 +01:00
Anthony Laibe 78795ce3e0 feat: reduce infura call
check if token exists before making the call
2023-01-10 09:42:08 +01:00
dlipicar 7d04be3613
Revert "feat/currency rounding (#3041)" (#3046)
This reverts commit 3d22f4a86d.
2023-01-04 09:40:26 -03:00
Anthony Laibe 99ff0a18b0 feat: cbridge integration 2023-01-04 08:23:18 +01:00
dlipicar 3d22f4a86d
feat/currency rounding (#3041)
* feat(@desktop/wallet): added iso4217 library for fiat currency display decimals

* feat(@desktop/wallet): added token peg info and use numbers for token market values

* feat(@desktop/wallet): extend wallet api to fetch prices in multiple currencies

* chore(@desktop/wallet): rename token peg field for clarity
2023-01-03 10:24:55 -03:00
frank 9137257638
synchronize display name (#2989) 2022-12-29 14:16:19 +08:00
Anthony Laibe 6abbe98cd2 feat: allow to lock amount in router 2022-12-21 16:58:33 +01:00
Anthony Laibe d7bf19fdbb feat: allow to get wallet token for specific addresses 2022-12-08 09:16:17 +01:00
Anthony Laibe c735e2a6bb feat: add wallet ticker 2022-12-02 10:53:21 +01:00
Anthony Laibe 0d1837f858 feat: add ticker in backend rather than in clients 2022-12-02 10:53:21 +01:00
Anthony Laibe eff02a79a9 feat: add wallet token endpoint 2022-12-02 09:32:07 +01:00
Anthony Laibe f3b1448efa fix: router preferred chains 2022-12-02 09:27:44 +01:00
Anthony Laibe 4c29c97591 feat: bridge tx 2022-11-30 09:18:24 +01:00
dlipicar ba053c8a5d
fix: keep original case for collectible trait types (#2984) 2022-11-29 09:28:37 -03:00
Khushboo-dev-cpp d41fcaf8a9
feat: delete message for new design (#2922) (#2965)
Co-authored-by: yqrashawn <namy.19@gmail.com>
2022-11-22 11:12:16 +01:00
Anthony Laibe 7ea9a3f230 fix: router fees for EIP1559 2022-11-14 13:32:29 +01:00
Stefan 601484af3e feat: Retrieve balance history for native token
Add functionality to sample and retrieve balance history and cache
it in memory for the current transfer controller.

The end of the balance history is snapped at twice per day to
avoid having to query the blockchain again for each fetching within
12 hours interval

The functionality will be extended with DB caching, API call batching,
"smarter" cache hitting and syncing between devices

Updates: #7662
2022-11-07 22:39:21 +01:00
Sale Djenic b0b23076a0 feat: `GetDerivedAddressDetails` added to accounts api 2022-10-28 13:27:55 +02:00
Anthony Laibe 32ffb30bf0 feat: stt market data handler 2022-10-28 10:21:27 +02:00
Anthony Laibe 63eb3d71ba feat: run go mod tiny/vendor 2022-10-26 09:39:30 +02:00
Andrea Maria Piana fc836e4c76 upgrade ethereum 2022-10-26 09:39:30 +02:00
Anthony Laibe b11643e66d feat: hop integration 2022-10-26 09:39:30 +02:00
Anthony Laibe 37c9eb1a9d
fix: token override when scanning (#2910) 2022-10-24 08:45:14 +02:00
Roman Volosovskyi 5c3435c12f
Changes necessary to make wallet work pokt networks 2022-10-19 10:40:13 +02:00
Anthony Laibe d9b44777c9
feat: update opensea testnet to use goerli (#2894) 2022-10-12 09:59:38 +02:00
Khushboo-dev-cpp 3364c159f4
feat: Added API for getting historical data for a token (#2882) 2022-10-10 21:02:04 +02:00
Stefan b9d6e6d8c8 feat: sync saved addresses
Main changes:
- Extend saved addresses DB with sync info: sync timestamp, update timestamp
and soft removed flag
- Create custom protobuf message payload to sync saved addresses
- Cleanup saved addresses on each start of messenger, by deleting
soft removed older entries
- Sync all saved addresses on Messenger.SyncDevices calls
- Sync particular changes to saved addresses
- Add SavedAddressManager instance to messenger
  - Note, can't find a clean way to pass the SavedAddressManager to the
  messenger, so we create another one
- Add tests for sync and new DB API

Closes: #7229
2022-10-10 11:26:35 +02:00
Richard Ramos bf9abfc350
feat: make kdf iterations configurable when creating a multiaccount (#2796) 2022-09-27 16:27:20 -04:00
Roman Volosovskyi 506921509e
Add effective gas price to tx view 2022-09-27 17:15:56 +02:00
Anthony Laibe cdca3d86e2
feat(token): enable changing token address from the node config (#2848)
Co-authored-by: Jonathan Rainville <rainville.jonathan@gmail.com>
2022-09-13 11:30:52 +02:00
Stefan aa4d95917c feat: merge favourites with saved address
Remove Favourites APIs and update the saved address APIs

Added up migration scripts that move the favourites from the old table
to the saved_addresses table with true flag and then drop the favourites table.

Required by #6546
2022-09-12 13:46:04 +02:00
Khushboo-dev-cpp 1485b3b4c8
feat: Added new BaseFee param inside transactions (#2847) 2022-09-09 20:22:59 +02:00
Samuel Hawksby-Robinson 41cb9d7bea Added httptest.NewServer to DecodeFromURL and CryptoOnRamp tests 2022-09-09 15:07:48 +01:00
Anthony Laibe 48f678052c
feat: batch balance (#2833) 2022-09-09 08:58:36 +02:00
Khushboo-dev-cpp 6376ad6e2d
feat: Add param for change24Hour to get 24 hours change value for a token from cryptocompare (#2817) 2022-08-30 10:21:01 +02:00
Khushboo-dev-cpp 25a80cf20b
feat(CryptoCompareApi): Get token details from crypto compare (#2793) 2022-08-23 10:46:15 +02:00
Richard Ramos 2edcf0e360
fix: ens release username hash (#2756)
* fix: ens release username hash
* fix: IN parameters when obtaining multiple pending trxs
2022-08-08 14:25:53 -04:00
Khushboo-dev-cpp 6967732a06
feat: Update the asset traist to have the correct uppercase formats for trait values (#2781) 2022-08-03 09:42:56 +02:00
Anthony Laibe 19807ce338
feat: add arbitrum tokens (#2778) 2022-08-02 11:11:27 +02:00
Anthony Laibe 9e0c9a5875
fix: remove duplicate token (#2777) 2022-07-29 09:40:07 +02:00
Anthony Laibe 4e2b02bedd feat: add goerli contracts 2022-07-29 08:57:38 +02:00
Churikova Tetiana 3cd92fda78
goerli: contracts, bump version 2022-07-20 14:15:15 +02:00
Jakub Sokołowski 530f3c7a3a
rename status-react to status-mobile
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2022-07-17 15:37:14 +02:00
Anthony Laibe ce0caa0f7f
feat: multi transaction (#2751) 2022-07-15 10:53:56 +02:00
Khushboo-dev-cpp b2ce92fd41
feat: Add logic to re caculate route for sending a transaction based on networks disabled by the user (#2742) 2022-07-14 16:22:42 +02:00
Anthony Laibe 35c4001e57
feat: estimate time for a transaction (#2744) 2022-07-12 14:25:32 +02:00
Vitaliy Vlasov 3dee94e505 Wallet sync for generated accounts 2022-07-06 19:24:43 +03:00
Anthony Laibe 4a3c4ad0ca
feat: allows to pass array of chain ids (#2733) 2022-07-04 09:48:30 +02:00
Churikova Tetiana 63d6119794
goerli: add assets 2022-07-01 12:42:06 +02:00
Anthony Laibe 8c0f230644
feat: Simple transaction router (#2686) 2022-06-09 15:09:56 +02:00
Anthony Laibe 97a99d6254
fix: get balance with multiple chain (#2689) 2022-05-26 06:44:29 -04:00
Vitaliy Vlasov 011238b1d1 Wallet sync 2022-05-18 15:25:20 +03:00
Khushboo-dev-cpp 15e5584ed2
feat: Add hasActivity param to derived addresses (#2663)
Added functionality to find target address when 6th param in path is added
for ex: "m'/44'/60'/0'/0/500" reperents the Address at the 500th index

Added a api to get the Address derived from a private key
2022-05-18 13:31:45 +02:00
Anthony Laibe e199ddbe9d
feat: get wallet api (#2619) 2022-05-10 09:48:05 +02:00
Richard Ramos 0048aaebcc
feat: add IPFS rate limiter for downloading stickers and use http server for retrieving stickers (#2611) 2022-05-09 09:07:57 -04:00
Anthony Laibe 907ba8ee5c
feat: visible token api (#2624) 2022-04-13 09:55:38 +02:00
Anthony Laibe 4d39547b81
Feat/expose registrar address (#2600)
* feat: expose registrar address

* fix code gen
2022-04-04 12:54:44 -04:00
Anthony Laibe 16311512cb
feat: add suggested fees api (#2604) 2022-03-29 17:12:05 -04:00
Samuel Hawksby-Robinson e67592d556
Sync Settings (#2478)
* Sync Settings

* Added valueHandlers and Database singleton

Some issues remain, need a way to comparing incoming sql.DB to check if the connection is to a different file or not. Maybe make singleton instance per filename

* Added functionality to check the sqlite filename

* Refactor of Database.SaveSyncSettings to be used as a handler

* Implemented inteface for setting sync protobuf factories

* Refactored and completed adhoc send setting sync

* Tidying up

* Immutability refactor

* Refactor settings into dedicated package

* Breakout structs

* Tidy up

* Refactor of bulk settings sync

* Bug fixes

* Addressing feedback

* Fix code dropped during rebase

* Fix for db closed

* Fix for node config related crashes

* Provisional fix for type assertion - issue 2

* Adding robust type assertion checks

* Partial fix for null literal db storage and json encoding

* Fix for passively handling nil sql.DB, and checking if elem has len and if len is 0

* Added test for preferred name behaviour

* Adding saved sync settings to MessengerResponse

* Completed granular initial sync and clock from network on save

* add Settings to isEmpty

* Refactor of protobufs, partially done

* Added syncSetting receiver handling, some bug fixes

* Fix for sticker packs

* Implement inactive flag on sync protobuf factory

* Refactor of types and structs

* Added SettingField.CanSync functionality

* Addressing rebase artifact

* Refactor of Setting SELECT queries

* Refactor of string return queries

* VERSION bump and migration index bump

* Deactiveate Sync Settings

* Deactiveated preferred_name and send_status_updates

Co-authored-by: Andrea Maria Piana <andrea.maria.piana@gmail.com>
2022-03-23 18:47:00 +00:00
andrey 3f9d689f90 [#13197] Update moonpay onramp URL 2022-03-23 10:29:49 +01:00
Anthony Laibe b629d6fa74
feat: move fetch prices to status-go (#2595) 2022-03-23 09:35:58 +01:00
Richard Ramos 0a758b0f7d feat: stickerpacks service 2022-03-02 17:46:16 -04:00
Anthony Laibe 66d511e33f
feat: add opensea api key (#2528) 2022-02-16 10:22:19 +01:00
andrey 2e8007c12d use token list from status-react 2022-02-01 12:44:34 +01:00
Anthony Laibe 76436f7768 fix: Add user agent for opensea query 2022-01-18 16:30:38 +00:00
Anthony Laibe 9d8ec4432e
feat: Add token store API (#2482) 2022-01-14 10:21:00 +01:00
Richard Ramos ee41e30881
feat: extract node config from settings to individual tables (#2470) 2022-01-12 16:04:43 -04:00
Roman Volosovskyi 7ef2efaabd
Support for binance chains 2021-11-27 10:47:57 +02:00
Richard Ramos 38cc695753
waku2: log envelope hash and ids for all messages (#2427) 2021-11-11 12:13:55 -04:00
Samuel Hawksby-Robinson 20b6f8d96e
Added currency filter to ramp (#2390) 2021-10-01 14:25:54 +01:00
Anthony Laibe 26bfeddad4
feat: Rpc client manage multiple eth client (#2359) 2021-09-22 13:49:20 -04:00
Anthony Laibe 5d1fe07544 fix: get token balance 2021-09-22 16:44:22 +02:00
Anthony Laibe 95dcbef5e5
feat: enable rinkeby for opensea (#2357) 2021-09-20 12:24:07 -04:00
Anthony Laibe fb3ef387ac
feat: Add more attributes from opensea (#2354)
* feat: Add more attributes from opensea

Add sale / orders information as well as collection traits

* feat: add background color to opensea response

* feat: Add max value and handle float to opensea response
2021-09-17 10:01:35 +02:00
Eric Mastro 06d4163de8
feat: Add Saved Addresses CRUD (#2356) 2021-09-10 14:08:22 -04:00
Anthony Laibe 07651d4d06
feat: enable wallet without network binding (#2349)
* feat: enable wallet without network binding

* feat: make transfer network aware

* feat: allow to pass initial networks via config

* fix: nil check and feed

* feat: Add documentation with better function name

* fix: do not init the manager more than once

* fix: PR feedbacks

* Bump version

* Update Jenkinsfile.tests

* Convert int to string

Co-authored-by: RichΛrd <info@richardramos.me>
2021-09-09 16:28:54 +02:00
Anthony Laibe 048511cc99 feat: Add trait to collectible
Build a custom unmarshaller for field that can be int or string

see: https://docs.opensea.io/reference/asset-object
2021-09-01 09:56:26 +02:00
Anthony Laibe 8f9c644dae
feat: fetch assets from opensea (#2320) 2021-08-20 15:53:24 -04:00
Roman Volosovskyi c78eac8d94
Add MaxFeePerGas and MaxPriorityFeePerGas to TransferView 2021-08-12 19:42:13 +03:00
Andrea Maria Piana b3a5455c46 Fix some tests & upgrade contracts 2021-07-20 10:57:38 +02:00
Andrea Maria Piana d84da4f515 Fix some tests and restore private/public RPC client 2021-07-20 10:57:38 +02:00
Andrea Maria Piana 1e8688ac56 Upgrade to london signer and cleanup 2021-07-20 10:57:38 +02:00
Andrea Maria Piana 4b0daeb47b Move services to status-node
Move all the services to status-node, as some of them were there, some
of them were in the geth backend and scattered around.
2021-07-20 10:57:38 +02:00
Andrea Maria Piana d092a2bb49 Fix wallet big int conversion 2021-07-20 10:57:38 +02:00
Andrea Maria Piana 799bd93451 Memove mailserver registry and update ierc20 contract bindings
This commit removes the mailserver contract registry, as not used
anymore, and upgrade the ERC20 contract bindings.
2021-07-20 10:57:38 +02:00
Roman Volosovskyi d1dc5f1e27
[wallet] Add wallet_getChachedBalances method 2021-05-25 10:03:05 +03:00
Samuel Hawksby-Robinson bb58d82661
Updated Ramp siteUrl to use URL in docs and use the referral code (#2218)
* Updated Ramp siteUrl to use URL in docs and use the referral code

see https://docs.ramp.network/configuration

* Moved Ramp to the top

* Removed local cryptos and transak
2021-05-17 15:51:33 +01:00
Roman Volosovskyi 6037570901
[wallet] Pending transactions API 2021-04-23 18:20:38 +03:00
Roman Volosovskyi fd49876a47
[wallet] fix UpsertRange 2021-04-20 10:37:26 +03:00
Samuel Hawksby-Robinson 07a713cc71
Removed Wyre from on ramp options (#2207)
* Removed Wyre from on ramp options

* VERSION bump
2021-04-19 12:17:34 +01:00
Roman Volosovskyi ddc48c7cf4
[wallet] Download transactions with unspecified chainId 2021-04-15 16:39:08 +03:00
Roman Volosovskyi f3ee49c110
[wallet] cleanup 2021-04-08 23:30:27 +03:00
Roman Volosovskyi 5a76e93063
[wallet] WatchTransaction method 2021-03-30 21:18:48 +03:00
Samuel Hawksby-Robinson bd45811ba6
Removed region field from on ramp struct (#2178)
* Removed region field from on ramp struct

* Added basic placeholder for latamex

* Added latamex on ramp option

* fee rate change to Ramp

* Updated VERSION

* Bump to major version
2021-03-25 13:00:55 +00:00
Roman Volosovskyi 04fb7fdb3d
[wallet] GetTransfersByAddress for preloaded transfers 2021-03-24 17:35:34 +02:00
Roman Volosovskyi 6bea21c197
[wallet] Skip transaction on chain id mismatch
Sometimes eth_getBlockByNumber returns txs with chainId which is not
equal to chanin's id. That caused an error and tx fetching was
interrupted. From now on such txs will be skipped.
2021-03-19 15:54:39 +02:00
Roman Volosovskyi 66fbfc1daf
[wallet] Nonce range cache 2021-03-16 16:20:11 +02:00
Roman Volosovskyi a7f2e94999
[wallet] Store account's last known balance and nonce 2021-03-15 16:18:12 +02:00
Roman Volosovskyi 3a408135d8
[wallet] RPC usage stats 2021-03-10 13:32:46 +02:00
Roman Volosovskyi d21cd6aba1
[wallet] Set initial blocks range for a new empty account 2021-03-03 10:05:52 +02:00
Roman Volosovskyi 363ab0a2ab
[wallet] Avoid some unnecessary RPC requests
- avoid making RPC request for `zero - zero` range
- avoid checking of nonce for a lower block in the range if it is zero
  in a higher block
- on `wallet_getTransfersByAddress` scanning of history is skipped if
  zero block is already reached
- no need to fetch block num before fetching token balances
2021-02-17 13:55:27 +02:00
Samuel Hawksby-Robinson db962e0135 Added API for getting on ramp data 2021-02-17 09:53:45 +01:00
Roman Volosovskyi 8268083008
[wallet] Merge blocks ranges when possible
- old existing ranges are merged when wallet service is started
- a new range is merged with an existing one if possible

This will decrease the number of entries in blocks_range table as
currently it can grow indefinitely (@flexsurfer reported 23307 entries).

This change is also needed for further optimisations of RPC usage.
2021-02-09 16:32:20 +02:00
Roman Volosovskyi e18050b87f
Make sure tokens balances can be fetched 2021-02-03 11:54:24 +02:00
Roman Volosovskyi d8bccaff18
[wallet] Detect non archival RPC node 2021-01-25 14:52:14 +02:00
Andrea Maria Piana 7387049d4b Upgrade linter and address issues 2020-12-28 16:55:14 +01:00
Roman Volosovskyi bf703254ba
[wallet] Dispatch event on tx history fetching failure 2020-12-21 15:45:10 +02:00
Roman Volosovskyi d39ca7fea4
[wallet] Fix multiple notifications on a single erc20 transfer 2020-12-11 16:47:38 +02:00
Roman Volosovskyi 002f9a5597
[wallet] Reduce number of RPC requests
- Wallet service is not started on foreground event on status-go side
  anymore, it leaves a client side opportunity to decide whether new
  blocks should be watched.
- `watchNewBlocks` parameter is added to `StartWallet`.
- Some requests are removed/moved to the place where they are necessary.
2020-11-25 12:20:23 +02:00
Andrea Maria Piana 9814a9a5e7 Dont crash if to or from are nil 2020-11-19 08:14:51 +01:00
Gheorghe Pinzaru d04e54e54e
Local notifications service (#2026)
Also adds implementation for eth transactions notifications
2020-10-28 10:56:14 +03:00
Andrea Maria Piana efbf093bc4 Don't fail if one request fails in getTokensBalances
If one request failed, the whole batch would fail.
This caused issue as one of the contract is constantly returning an
error now, and essentially there was not way to fetch balance.
Also extend the timeout to 20s as we throw 165 request to Infura in one
go and it takes its time to reply to those, although it seems like we
should batch them on our side instead of sending them all cuncurrently.
2020-10-05 12:53:51 +02:00
andrey 397ee840c6 [#2042] Wrong ERC20 decimal rounding in "Set max" 2020-09-22 16:00:47 +02:00
andrey 682722b973 add wallet favourites 2020-09-21 10:48:00 +02:00
RichΛrd 62c9d56fe8
Add API functions to store pending transaction data (#2037)
* Add API functions to store pending transaction data
* fix: code review
2020-09-14 09:39:24 -04:00
Andrea Maria Piana 435eacecb5 Handle wallet initialization
StartWallet was called before service initialization.
After the recent changes this call was moved after initialization, but
the geth system automatically start services.
This meant that `IsStarted()` returned true, although the reactor was
not started, and only after calling `StopWallet()` and `StartWallet()`
again the system would reach the right state.

This commit changes the behavior so that we only check whether the
reactor has been started when calling `IsStarted()` and we allow
multiple calls to `Start()` on the signal service, which won't return an
error (it's a noop if callled multiple times).
2020-08-18 14:13:08 +02:00
Roman Volosovskyi aa1b898fe4 Stop/start wallet methods 2020-08-17 12:53:37 +02:00
Roman Volosovskyi 9d5684162a
Stop fetching new blocks while app is not active
Saves some traffic (blocks with transactions are heavy) and infura
requests.
2020-03-11 08:39:41 +02:00
Adam Babik 8b61d92d58
Remove protocol and eth-node submodules (#1835) 2020-02-10 12:22:37 +01:00
Roman Volosovskyi dc80cb0e06
Cleanup after tx fetching changes
- unused API methods are removed
- some unusued code is removed too
- API docs are updated

That's just a portion of clean up that should be done,
but the rest of it will probably happen in different PR
with changes to the way how we watch to chain updates.
2020-02-09 12:05:36 +02:00
Roman Volosovskyi c2f22f1fbc
[status-im/status-react#9927] Fast blocks sync after delay
- In order to avoid handling of the reorganized blocks we use an offset
from the latest known block when start listening to new blocks. Before
this commit the offset was 15 blocks for all networks. This offset is
too big for mainnet and causes noticeable delay of marking a transfer as
confirmed in Status (comparing to etherscan). So it was changed to be 5
blocks on mainnet and is still 15 blocks on other networks.
- Also before this commit all new blocks were handled one by one with
network specific interval (10s for mainnet), which means that in case of
lost internet connection or application suspension (happens on iOS)
receiving of new blocks would be paused and then resumed with the same
"speed" - 1 blocks per 10s. In case if that pause is big enough the
application would never catch up with the latest block in the network,
and this also causes the state of transfers to be delayed in the
application. In this commit in case if there was more than 40s delay
after receiving of the previous block the whole history in range between
the previous received block and ("latest"-reorgeSafetyDepth) block is
checked at once and app catches up with a recent state of the chain.
2020-01-30 17:25:56 +02:00
Roman Volosovskyi a92a95cf83
status-im/status-react#9203 Faster tx fetching with less request
*** How it worked before this PR on multiaccount creation:
- On multiacc creation we scanned chain for eth and erc20 transfers. For
  each address of a new empty multiaccount this scan required
  1. two `eth_getBalance` requests to find out that there is no any
     balance change between zero and the last block, for eth transfers
  2. and `chain-size/100000` (currently ~100) `eth_getLogs` requests,
     for erc20 transfers
- For some reason we scanned an address of the chat account as well, and
  also accounts were not deduplicated. So even for an empty multiacc we
  scanned chain twice for each chat and main wallet addresses, in result
  app had to execute about 400 requests.
- As mentioned above, `eth_getBalance` requests were used to check if
  there were any eth transfers, and that caused empty history in case
  if user already used all available eth (so that both zero and latest
  blocks show 0 eth for an address). There might have been transactions
  but we wouldn't fetch/show them.
- There was no upper limit for the number of rpc requests during the
  scan, so it could require indefinite number of requests; the scanning
  algorithm was written so that we persisted the whole history of
  transactions or tried to scan form the beginning again in case of
  failure, giving up only after 10 minutes of failures. In result
  addresses with sufficient number of transactions would never be fully
  scanned and during these 10 minutes app could use gigabytes of
  internet data.
- Failures were caused by `eth_getBlockByNumber`/`eth_getBlockByHash`
  requests. These requests return significantly bigger responses than
  `eth_getBalance`/`eth_transactionsCount` and it is likely that
  execution of thousands of them in parallel caused failures for
  accounts with hundreds of transactions. Even for an account with 12k
  we could successfully determine blocks with transaction in a few
  minutes using `eth_getBalance` requests, but `eth_getBlock...`
  couldn't be processed for this acc.
- There was no caching for for `eth_getBalance` requests, and this
  caused in average 3-4 times more such requests than is needed.

*** How it works now on multiaccount creation:
- On multiacc creation we scan chain for last ~30 eth transactions and
  then check erc20 in the range where these eth transactions were found.
  For an empty address in multiacc this means:
  1. two `eth_getBalance` transactions to determine that there was no
     balance change between zero and the last block; two
     `eth_transactionsCount` requests to determine there are no outgoing
     transactions for this address; total 4 requests for eth transfers
  2. 20 `eth_getLogs` for erc20 transfers. This number can be lowered,
     but that's not a big deal
- Deduplication of addresses is added and also we don't scan chat
  account, so a new multiacc requires ~25 (we also request latest block
  number and probably execute a few other calls) request to determine
  that multiacc is empty (comparing to ~400 before)
- In case if address contains transactions we:
  1. determine the range which contains 20-25 outgoing eth/erc20
     transactions. This usually requires up to 10 `eth_transactionCount`
     requests
  2. then we scan chain for eth transfers using `eth_getBalance` and
     `eth_transactionCount` (for double checking zero balances)
  3. we make sure that we do not scan db for more than 30 blocks with
     transfers. That's important for accounts with mostly incoming
     transactions, because the range found on the first step might
     contain any number of incoming transfers, but only 20-25 outgoing
     transactions
  4. when we found ~30 blocks in a given range, we update initial
     range `from` block using the oldest found block
  5. and now we scan db for erc20transfers using `eth_getLogs`
     `oldest-found-eth-block`-`latest-block`, we make not more than 20 calls
  6. when all blocks which contain incoming/outgoing transfers for a
     given address are found, we save these blocks to db and mark that
     transfers from these blocks are still to be fetched
  7. Then we select latest ~30 (the number can be adjusted) blocks from
     these which were found and fetch transfers, this requires 3-4
     requests per transfer.
  8. we persist scanned range so that we know were to start next time
  9. we dispatch an event which tells client that transactions are found
  10. client fetches latest 20 transfers
- when user presses "fetch more" button we check if app's db contains next
  20 transfers, if not we scan chain again and return transfers after

small fixes
2020-01-23 10:36:11 +02:00
Andrea Maria Piana 25d46c6d82
Fix waku tests & contact ens (#1802) 2020-01-17 13:39:09 +01:00
Pedro Pombeiro c8a911ebd1 Use goimports instead of gofmt 2020-01-06 10:17:23 +01:00
Pedro Pombeiro dd894ece15 Start abstracting geth Keystore 2019-12-19 14:11:48 +01:00
André Medeiros 0571f561f0
APIs to Get and Create custom Tokens (#1717) 2019-12-10 12:31:08 -05:00
Pedro Pombeiro 2dd74da23d Integrate Nimbus status-protocol-go 2019-11-18 14:30:03 +01:00
Adam Babik 26880b83d7
Upgrade geth to 1.9.5 and Whisper (#1617) 2019-10-04 17:21:24 +02:00
dmitry 7454586889 Match transfer to every account that is being watched 2019-08-28 17:38:58 +03:00
Dmitry Shulyak 0165b028c9
Watch new accounts aftter they were saved to accounts table (#1569)
* Watch new accounts once they are saved in accounts table

* Add test that reactor can be restarted and watch new accounts
2019-08-28 10:49:03 +03:00
Dmitry a67184adbd Wallet database isolated by the network id
Wallet database refactored so that every query ensures isolation by the network id.
Network id provided when database object is created, thus it is transparent to other parts
of the wallet module.

Additionally every uniqueness index is changed to ensure that it doesn't prevent adding
object with same id but from a different network.
2019-08-21 10:44:50 +03:00
Dmitry 494cb5bb33 Create single database appdatase 2019-08-21 10:44:50 +03:00
Dmitry Shulyak 9dbf5a0c86
Add browsers API (#1523)
Add browsers API
2019-07-17 08:28:37 +03:00
Dmitry Shulyak 40b6b3da13
Simplify transfer object (#1521)
* Store tx and receipt in db and cast it to TransferView on read

* Store Log instead of log index

* Use contract from log and bring back address field

* Add tx status and id fields
2019-07-15 14:16:07 +03:00
Dmitry Shulyak e20648ecc7
[services/wallet] Several changes in API after feedback (#1517)
[services/wallet] Several changes in API after feedback

- Timestamp from block header stored in blocks table and added to each transfers
- From field is computed from signature to offload this computation from client side
- `history` event is added back, so that client can ignore historical blocks when watching
only for new blocks
-  block number and timestamp are marshalled in hex. consistent with ethereum data structures
2019-07-10 12:08:43 +03:00
Dmitry a2295baaf9 Add api method to query token balances 2019-07-05 08:38:29 +03:00
Dmitry Shulyak 047c9b5263
Download transfers starting from latest block header (#1467) 2019-06-14 13:16:30 +03:00