Commit Graph

199 Commits

Author SHA1 Message Date
andrey bc63fa606e delete imported key 2021-08-18 12:56:53 +02:00
Bitgamma 6c2e9652d0
add account conversion (#2277) 2021-07-20 14:48:10 +03:00
Andrea Maria Piana a4701f41ee linting and remove unused code 2021-07-20 10:57:38 +02:00
Andrea Maria Piana bdf3775b0b Remove subs test
The test was relying on a method `UnlockAccount` which is not exported
anymore.
This method seems not something we should generally expose, and in any
case it does not seem to be used by the frontend.
2021-07-20 10:57:38 +02:00
Andrea Maria Piana b3a5455c46 Fix some tests & upgrade contracts 2021-07-20 10:57:38 +02:00
Andrea Maria Piana d84da4f515 Fix some tests and restore private/public RPC client 2021-07-20 10:57:38 +02:00
Andrea Maria Piana 1e8688ac56 Upgrade to london signer and cleanup 2021-07-20 10:57:38 +02:00
Andrea Maria Piana 4b0daeb47b Move services to status-node
Move all the services to status-node, as some of them were there, some
of them were in the geth backend and scattered around.
2021-07-20 10:57:38 +02:00
Andrea Maria Piana 799bd93451 Memove mailserver registry and update ierc20 contract bindings
This commit removes the mailserver contract registry, as not used
anymore, and upgrade the ERC20 contract bindings.
2021-07-20 10:57:38 +02:00
Shivek Khurana 81171ad9e6
Expose password reset function (#2223)
Handled edge-cases
2021-06-23 14:51:21 +05:30
RichΛrd 40359f9c1b
go-waku integration (#2247)
* Adding wakunode module
* Adding wakuv2 fleet files
* Add waku fleets to update-fleet-config script
* Adding config items for waku v2
* Conditionally start waku v2 node depending on config
* Adapting common code to use go-waku
* Setting log level to info
* update dependencies
* update fleet config to use WakuNodes instead of BootNodes
* send and receive messages
* use hash returned when publishing a message
* add waku store protocol
* trigger signal after receiving store messages
* exclude linting rule SA1019 to check deprecated packages
2021-06-16 16:19:45 -04:00
Andrea Maria Piana d50fee6bb2 Handle connection state 2021-05-21 07:22:58 +02:00
Andrea Maria Piana 566e9a3ade remove nimbus 2021-05-21 07:22:58 +02:00
Andrea Maria Piana 57b1bc193f Remove whisper 2021-05-21 07:22:58 +02:00
Andrea Maria Piana be01875d1d remove mailserver logic 2021-05-21 07:22:58 +02:00
Andrea Maria Piana cf6ef3171a Add mailserver logic 2021-05-21 07:22:58 +02:00
RichΛrd 244686e3a8
add GetNodeConfig() function to backend API (#2216) 2021-04-29 15:22:10 -04:00
Andrea Maria Piana ddc93981a7 Add V4 endpoints 2021-04-19 14:49:28 +02:00
Roman Volosovskyi f3ee49c110
[wallet] cleanup 2021-04-08 23:30:27 +03:00
Shivek Khurana 1097b14a7f
🎭 📊 Anonymous Metrics V0 (#2170)
* Migrations in place, how to run them?

* Remove down migrations and touch database.go

* Database and Database Test package in place, added functions to get and store app metrics

* make generate output

* Minor bug fix on app metrics insert and select

* Add a validation layer to restrict what can be saved in the database

* Make validation more terse, throw error if schema doesn't exist, expose appmetrics service

* service updates

* Compute all errors before sending them out

* Trying to bring a closjure to appmetrics go

* Expose appmetrics via an api, skip fancy

* Address value as Jason Dawt Rawmasage to ease parsing

* Introduce a buffered chan with magic cap of 8 to minimize writes to DB. Tests for service and API. Also expose GetAppMetrics function.

* Lint issues

* Remove autoincrement, undo waku.json changes, fix error being shadowed, return nil where nil ought to be returned, get rid of buffered channel

* Bump migration number

* Fix API factory usage

* Add comment re:json.RawMessage instead of strings

* Get rid of test vars, throw save error inside the loop

* Update version

Co-authored-by: Samuel Hawksby-Robinson <samuel@samyoul.com>
2021-03-17 18:09:28 +05:30
Roman Volosovskyi 3a408135d8
[wallet] RPC usage stats 2021-03-10 13:32:46 +02:00
Roman Volosovskyi 8268083008
[wallet] Merge blocks ranges when possible
- old existing ranges are merged when wallet service is started
- a new range is merged with an existing one if possible

This will decrease the number of entries in blocks_range table as
currently it can grow indefinitely (@flexsurfer reported 23307 entries).

This change is also needed for further optimisations of RPC usage.
2021-02-09 16:32:20 +02:00
Roman Volosovskyi e18050b87f
Make sure tokens balances can be fetched 2021-02-03 11:54:24 +02:00
Andrea Maria Piana b1a1a223c2
Add export/import methods 2021-01-26 09:39:54 +01:00
Roman Volosovskyi 79716227db
[wallet] Show PNs right after enabling 2021-01-13 12:33:09 +02:00
Andrea Maria Piana 7387049d4b Upgrade linter and address issues 2020-12-28 16:55:14 +01:00
Andrea Maria Piana c78e084eb9 Fix tests and lint 2020-12-17 14:10:00 +01:00
Samuel Hawksby-Robinson 829108c96f Changes to tests to include account where required for account injection 2020-12-17 14:10:00 +01:00
Samuel Hawksby-Robinson 8663d636d3 Reversed commented out error 2020-12-17 14:10:00 +01:00
Samuel Hawksby-Robinson 30c2b64cf5 Test condition MUST REVERT 2020-12-17 14:10:00 +01:00
Samuel Hawksby-Robinson 16fecf5520 Passing Master info to Messenger 2020-12-17 14:10:00 +01:00
Samuel Hawksby-Robinson 190b1a70f9 Removed usages of multiaccounts.Account.PhotoPath 2020-12-17 14:10:00 +01:00
Samuel Hawksby-Robinson 193ab30ada Refactor injecting multiaccounts into messenger and service 2020-12-17 14:10:00 +01:00
Samuel Hawksby-Robinson 4ddec82e49 Removed superfluous debug logging 2020-12-17 14:10:00 +01:00
Samuel Hawksby-Robinson d6c3153fe8 tweak to debug output for StoreII 2020-12-17 14:10:00 +01:00
Samuel Hawksby-Robinson 9545f8e306 Added log to debug an image store nil pointer panic 2020-12-17 14:10:00 +01:00
Samuel Hawksby-Robinson fcbd9285f6 Implemented API for full CRD of IdentityImages 2020-12-17 14:10:00 +01:00
Samuel Hawksby-Robinson 2960fc49d4 Implemented GetImages and StoreImages in API 2020-12-17 14:10:00 +01:00
Samuel Hawksby-Robinson 1b59d088d8 Integrated GetMimeType() 2020-12-17 14:10:00 +01:00
Samuel Hawksby-Robinson c8450586fd Moved profile image handling into geth_backend 2020-12-17 14:10:00 +01:00
Roman Volosovskyi 002f9a5597
[wallet] Reduce number of RPC requests
- Wallet service is not started on foreground event on status-go side
  anymore, it leaves a client side opportunity to decide whether new
  blocks should be watched.
- `watchNewBlocks` parameter is added to `StartWallet`.
- Some requests are removed/moved to the place where they are necessary.
2020-11-25 12:20:23 +02:00
Andrea Maria Piana 70414c90b9 Fix nil pointer local notifications 2020-11-17 17:07:43 +01:00
Gheorghe Pinzaru d04e54e54e
Local notifications service (#2026)
Also adds implementation for eth transactions notifications
2020-10-28 10:56:14 +03:00
Andrea Maria Piana 1a7a63a8e5 Log on db failure 2020-10-14 08:07:05 +02:00
Andrea Maria Piana 2aea867cee Migrate journals as well 2020-09-30 12:13:22 +02:00
Andrea Maria Piana 0ba4694f21 [Fix: #2027] Migrate db to shorter path name 2020-09-25 14:02:09 +02:00
Gheorghe Pinzaru 9e148eab89
Dont allow disabling waku (#2040) 2020-09-16 10:31:01 +03:00
Andrea Maria Piana 401d18e558
Enabled push notifications by default and add fleet 2020-08-27 18:54:06 +02:00
Roman Volosovskyi aa1b898fe4 Stop/start wallet methods 2020-08-17 12:53:37 +02:00
Roman Volosovskyi 0b3cdf7362
DeleteMultiaccount method 2020-07-13 16:20:02 +03:00
Roman Volosovskyi 6d5a93287b
Separate keystore directory per multiacc
This commit provides `MigrateKeyStoreDir` method which moves multiacc's
keys from a common keystore directory to multiacc specific directory.
2020-07-09 12:41:47 +03:00
Samuel Hawksby-Robinson 157f20a7c8
Added link to specs for key compression API (#1995) 2020-06-24 14:39:42 +01:00
Samuel Hawksby-Robinson c8f9dad554
Feature/key compression (#1990)
## What has changed?

I've introduced to the public binding functionality that will compress and decompress public keys of a variety of encoding and key types. This functionality supports all major byte encoding formats and the following EC public key types:

- `secp256k1` pks
- `bls12-381 g1` pks
- `bls12-381 g2` pks

## Why make the change?

We want shorter public (chat) keys and we want to be future proof and encoding agnostic. See the issue here https://github.com/status-im/status-go/issues/1937

---

* Added basic signature for compresspk and uncompresspk

* Added basic encoding information

* make vendor

* formatted imports for the linter

* Reformatted imports hoping linter likes it

* This linter is capricious

* Added check that the secp256k1 key is valid

* Added test for valid key

* Added multiformat/go-varint dep

* Added public key type handling

* Added key decompression with key type handling

* Added handling for '0x' type indentifying

* Added more robust testing

* Less lint for the linting gods

* make vendor for bls12_381

* Added bls12-381 compression tests

* Added decompress key expected results

* Refactor of typed and untyped keys in tests

* Lint god appeasment

* Refactor of sample public keys

* Implemented bls12-381 decompression

* gofmt

* Renamed decode/encode funcs to be more descriptive

* Added binary bindings for key de/compression

* Refactor of func parameters

gomobile is a bit tempermental using raw bytes as a parameter, so I've decided to use string only inputs and outputs

* gofmt

* Added function documentation

* Moved multiformat de/compression into api/multiformat ns

* Moved multiformat de/compression into api/multiformat ns

* Changed compress to serialize on API
2020-06-23 11:47:17 +01:00
Andrea Maria Piana 44272944ab Make sure WakuEnabled is respected on upgrade 2020-03-30 15:28:16 +02:00
Roman Volosovskyi 9d5684162a
Stop fetching new blocks while app is not active
Saves some traffic (blocks with transactions are heavy) and infura
requests.
2020-03-11 08:39:41 +02:00
Adam Babik f335d1cf7e
Prevent starting node without APIModules (#1876) 2020-02-26 20:35:47 +01:00
Roman Volosovskyi 6cc1365536
Start wallet services when log into keycard account 2020-02-26 18:23:04 +02:00
Adam Babik e4b0a97cc5
add bridge metrics to Whisper and Waku (#1864) 2020-02-19 15:36:16 +01:00
Adam Babik 76b5dc29dc
Integrate Whisper-Waku bridge in status-go (#1854) 2020-02-18 12:21:01 +01:00
Andrea Maria Piana 9c2c08d44c
Init protocol when shh is active (#1862) 2020-02-17 18:18:30 +01:00
Andrea Maria Piana 141f917e7c
Initialize protocol when waku active (#1832) 2020-02-17 15:38:59 +01:00
Jakub Sokołowski cca47ef6e1 rename gomarkdown/markdown to status-im/markdown
This resolves a dependency conflict we have with MatterBridge
which was using a newer version of the same package.
This resulted in a JSON marshalling bug that would crash the bridge.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2020-02-11 16:06:43 +01:00
Roman Volosovskyi 9cf640842b
[status-im/status-react#9942] Upgradable paths in configs
Storing absolute path for different configs breaks compatibility on iOS
as app's dir is changed after upgrade. The solution is to store relative
paths and to concatenate it with `backend.rootDataDir`. The only
exception is `LogFile` as it is stored outside `backend.rootDataDir` on
Android. `LogDir` config was added to allow adding of custom dir for log
file.
Configs concerned:
`DataDir`
`LogDir`
`LogFile`
`KeystoreDir`
`BackupDisabledDataDir`
2020-02-03 20:02:01 +02:00
Roman Volosovskyi a92a95cf83
status-im/status-react#9203 Faster tx fetching with less request
*** How it worked before this PR on multiaccount creation:
- On multiacc creation we scanned chain for eth and erc20 transfers. For
  each address of a new empty multiaccount this scan required
  1. two `eth_getBalance` requests to find out that there is no any
     balance change between zero and the last block, for eth transfers
  2. and `chain-size/100000` (currently ~100) `eth_getLogs` requests,
     for erc20 transfers
- For some reason we scanned an address of the chat account as well, and
  also accounts were not deduplicated. So even for an empty multiacc we
  scanned chain twice for each chat and main wallet addresses, in result
  app had to execute about 400 requests.
- As mentioned above, `eth_getBalance` requests were used to check if
  there were any eth transfers, and that caused empty history in case
  if user already used all available eth (so that both zero and latest
  blocks show 0 eth for an address). There might have been transactions
  but we wouldn't fetch/show them.
- There was no upper limit for the number of rpc requests during the
  scan, so it could require indefinite number of requests; the scanning
  algorithm was written so that we persisted the whole history of
  transactions or tried to scan form the beginning again in case of
  failure, giving up only after 10 minutes of failures. In result
  addresses with sufficient number of transactions would never be fully
  scanned and during these 10 minutes app could use gigabytes of
  internet data.
- Failures were caused by `eth_getBlockByNumber`/`eth_getBlockByHash`
  requests. These requests return significantly bigger responses than
  `eth_getBalance`/`eth_transactionsCount` and it is likely that
  execution of thousands of them in parallel caused failures for
  accounts with hundreds of transactions. Even for an account with 12k
  we could successfully determine blocks with transaction in a few
  minutes using `eth_getBalance` requests, but `eth_getBlock...`
  couldn't be processed for this acc.
- There was no caching for for `eth_getBalance` requests, and this
  caused in average 3-4 times more such requests than is needed.

*** How it works now on multiaccount creation:
- On multiacc creation we scan chain for last ~30 eth transactions and
  then check erc20 in the range where these eth transactions were found.
  For an empty address in multiacc this means:
  1. two `eth_getBalance` transactions to determine that there was no
     balance change between zero and the last block; two
     `eth_transactionsCount` requests to determine there are no outgoing
     transactions for this address; total 4 requests for eth transfers
  2. 20 `eth_getLogs` for erc20 transfers. This number can be lowered,
     but that's not a big deal
- Deduplication of addresses is added and also we don't scan chat
  account, so a new multiacc requires ~25 (we also request latest block
  number and probably execute a few other calls) request to determine
  that multiacc is empty (comparing to ~400 before)
- In case if address contains transactions we:
  1. determine the range which contains 20-25 outgoing eth/erc20
     transactions. This usually requires up to 10 `eth_transactionCount`
     requests
  2. then we scan chain for eth transfers using `eth_getBalance` and
     `eth_transactionCount` (for double checking zero balances)
  3. we make sure that we do not scan db for more than 30 blocks with
     transfers. That's important for accounts with mostly incoming
     transactions, because the range found on the first step might
     contain any number of incoming transfers, but only 20-25 outgoing
     transactions
  4. when we found ~30 blocks in a given range, we update initial
     range `from` block using the oldest found block
  5. and now we scan db for erc20transfers using `eth_getLogs`
     `oldest-found-eth-block`-`latest-block`, we make not more than 20 calls
  6. when all blocks which contain incoming/outgoing transfers for a
     given address are found, we save these blocks to db and mark that
     transfers from these blocks are still to be fetched
  7. Then we select latest ~30 (the number can be adjusted) blocks from
     these which were found and fetch transfers, this requires 3-4
     requests per transfer.
  8. we persist scanned range so that we know were to start next time
  9. we dispatch an event which tells client that transactions are found
  10. client fetches latest 20 transfers
- when user presses "fetch more" button we check if app's db contains next
  20 transfers, if not we scan chain again and return transfers after

small fixes
2020-01-23 10:36:11 +02:00
Pedro Pombeiro db01f0b3e4 Avoid passing node to subscriptions service 2020-01-20 13:15:17 +01:00
Pedro Pombeiro 6537cae606 Nimbus node support 2020-01-20 13:15:17 +01:00
Pedro Pombeiro c8a911ebd1 Use goimports instead of gofmt 2020-01-06 10:17:23 +01:00
Pedro Pombeiro 527cd38e1f Address code review comments 2020-01-06 10:17:23 +01:00
Pedro Pombeiro 287e5cdf79 Abstract `accounts.Key` and geth `keystore` 2020-01-06 10:17:23 +01:00
Adam Babik 1ac515f19e remove reSelectAccount from backend, SelectKeyPair and SelectedKeyPairID methods from Whisper 2019-12-30 12:56:05 +01:00
yenda 024f30f0b9 Put settings in a table with schema (#1746)
Move settings table schema from a key-value store to a one row table with many columns.

We now save the first row with initial data in saveAccountAndLogin and follow up saveSetting calls are only saving one setting at a time.

Co-authored-by: Adam Babik <a.babik@designfortress.com>
2019-12-27 10:58:25 +01:00
Pedro Pombeiro b99af530e1 Migrate more files to eth-node/crypto 2019-12-20 11:24:26 +01:00
Pedro Pombeiro dd894ece15 Start abstracting geth Keystore 2019-12-19 14:11:48 +01:00
yenda 655031616c
fix 1741 (#1742)
* fix 1741

save accounts with SaveAccountAndStartNodeWithKey

* initialize keycard accountManager with the proper addresses
2019-12-18 16:09:04 +01:00
Roman Volosovskyi 4c0d8dedea
Replace address with keyUid in accounts db
Account's address was used as a primary key in accounts db and as a
deterministic id of an account in some API calls. Also it was used as a
part of the name of the account specific database. This revealed some
extra information about the account and wasn't necessary.
At first the hash of the address was planned to be used as a
deterministic id, but we already have a keyUid which is calculated as
sha256 hash of account's public key and has similar properties:
- it is deterministic
- doesn't reveal accounts public key or address in plain
2019-12-09 11:20:12 +02:00
Pedro Pombeiro 26b6d7c36a Create a home submodule for Eth node bridges- Rename StatusBackend to GethStatusBackend 2019-11-27 17:02:09 +01:00
Pedro Pombeiro ed5a5c154d
Move to monorepo structure (#1684)
Move to a monorepo structure with submodules

- Rename status-protocol-go to status-go/protocol
2019-11-21 17:19:22 +01:00
Pedro Pombeiro 2dd74da23d Integrate Nimbus status-protocol-go 2019-11-18 14:30:03 +01:00
Pedro Pombeiro 6ce437255e Remove FCM implementation 2019-11-17 17:54:55 +01:00
Jakub Sokołowski 98c861cfe2 make sure Version is always set in loadNodeConfig()
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2019-10-29 12:53:27 +01:00
Adam Babik 26880b83d7
Upgrade geth to 1.9.5 and Whisper (#1617) 2019-10-04 17:21:24 +02:00
yenda 963ea94e08 Update account timestamp when login with keycard (#1614)
Closes #1613
2019-09-24 08:32:32 +02:00
Dmitry Novotochinov d263be10c4
[#1599] send 'node.login' signal when login with keycard (#1600) 2019-09-06 17:23:33 +03:00
Adam Babik d7f454fe4c
Add mailservers service (#1595) 2019-09-04 08:25:33 +02:00
Andrea Maria Piana 42199e682f
Add error to login signal (#1594)
We add an error to login signal so that the caller can tell whether
login was successful or not
2019-09-03 15:00:14 +02:00
Dmitry Shulyak 63dcdd4e94 Statefull login with keycard (#1587) 2019-09-02 21:03:15 +02:00
dmitry bf95a71f6f Verify valid sub-accounts using database 2019-08-30 14:11:00 +03:00
Dmitry Shulyak 0165b028c9
Watch new accounts aftter they were saved to accounts table (#1569)
* Watch new accounts once they are saved in accounts table

* Add test that reactor can be restarted and watch new accounts
2019-08-28 10:49:03 +03:00
Andrea Maria Piana e118e89107
Add persistent datasync, use single database and handle negotiated filters (#1580) 2019-08-27 14:04:15 +02:00
Dmitry a67184adbd Wallet database isolated by the network id
Wallet database refactored so that every query ensures isolation by the network id.
Network id provided when database object is created, thus it is transparent to other parts
of the wallet module.

Additionally every uniqueness index is changed to ensure that it doesn't prevent adding
object with same id but from a different network.
2019-08-21 10:44:50 +03:00
Dmitry 494cb5bb33 Create single database appdatase 2019-08-21 10:44:50 +03:00
Dmitry Shulyak be9c55bc16
Accounts data management (#1530)
* WIP accounts implementation

* Accounts datasore and changes to status mobile API

* Add library changes and method to update config

* Handle error after account selection

* Add two methods to start account to backend

* Use encrypted database for settings and add a service for them

* Resolve linter warning

* Bring back StartNode StopNode for tests

* Add sub accounts and get/save api

* Changes to accounts structure

* Login use root address and fetch necessary info from database

* Cover accounts store with tests

* Refactor in progress

* Initialize status keystore instance before starting ethereum node

* Rework library tests

* Resolve failures in private api test and send transaction test

* Pass pointer to initialized config to unmarshal

* Use multiaccounts/accounts naming consistently

Multiaccount is used as a login identifier
Account references an address and a key, if account is not watch-only.

* Add login timestamp stored in the database to accounts.Account object

* Add photo-path field for multiaccount struct

* Add multiaccoutns rpc with updateAccount method

Update to any other account that wasn't used for login will return an error

* Fix linter in services/accounts

* Select account before starting a node

* Save list of accounts on first login

* Pass account manager to accounts service to avoid selecting account before starting a node

* Add logs to login with save and regualr login
2019-08-20 18:38:40 +03:00
Andrea Maria Piana cb769ccca9 Revert "Initialize status keystore instance before starting ethereum node"
This reverts commit 476240fbd1.
2019-08-05 16:03:18 +02:00
Dmitry 476240fbd1 Initialize status keystore instance before starting ethereum node 2019-08-01 11:01:02 +03:00
Andrea Franz acfe9a721c
Multi account login (#1525)
* multi-account login and signing

put methods count threshold back to 20

* validate login params

* refactoring

* use common.Address

* remove unused var in test
2019-07-26 16:45:10 +02:00
Andrea Franz dcb0fa5262
add multi-account support (#1500)
* add multiaccount support

add multi account ImportPrivateKey and StoreAccount

test derivation from normal keys

* add multiaccount to mobile pkg

* use multiaccount params structs from the mobile pkg

* move multiaccount tests together with the other lib tests

* fix codeclimate warning and temporarily increase methods threshold

* split library_test_utils.go to avoid linter warnings
2019-07-24 20:59:15 +02:00
Adam Babik e93d994460 integrate status-protocol-go 2019-07-24 18:46:43 +02:00
Dmitry Shulyak 9723b64827
Permissions api (#1524)
* Add permissions api

* Integrate permissions service

* Reduce cyclomatic complexity of the MakeNode function
2019-07-21 08:41:30 +03:00
Dmitry Shulyak 9dbf5a0c86
Add browsers API (#1523)
Add browsers API
2019-07-17 08:28:37 +03:00
Adam Babik dbaf622e12
Refactor messaging (#1510)
This change flattens messaging/chat package. It also removes dependency between multidevice and chat/protobuf packages.

The Publisher interface was also changed a bit to support more native types.

Version got bumped to 0.29.0-beta.3.
2019-07-03 21:13:11 +02:00