Commit Graph

101 Commits

Author SHA1 Message Date
Roman Volosovskyi aa1b898fe4 Stop/start wallet methods 2020-08-17 12:53:37 +02:00
Roman Volosovskyi 0b3cdf7362
DeleteMultiaccount method 2020-07-13 16:20:02 +03:00
Roman Volosovskyi 6d5a93287b
Separate keystore directory per multiacc
This commit provides `MigrateKeyStoreDir` method which moves multiacc's
keys from a common keystore directory to multiacc specific directory.
2020-07-09 12:41:47 +03:00
Samuel Hawksby-Robinson 157f20a7c8
Added link to specs for key compression API (#1995) 2020-06-24 14:39:42 +01:00
Samuel Hawksby-Robinson c8f9dad554
Feature/key compression (#1990)
## What has changed?

I've introduced to the public binding functionality that will compress and decompress public keys of a variety of encoding and key types. This functionality supports all major byte encoding formats and the following EC public key types:

- `secp256k1` pks
- `bls12-381 g1` pks
- `bls12-381 g2` pks

## Why make the change?

We want shorter public (chat) keys and we want to be future proof and encoding agnostic. See the issue here https://github.com/status-im/status-go/issues/1937

---

* Added basic signature for compresspk and uncompresspk

* Added basic encoding information

* make vendor

* formatted imports for the linter

* Reformatted imports hoping linter likes it

* This linter is capricious

* Added check that the secp256k1 key is valid

* Added test for valid key

* Added multiformat/go-varint dep

* Added public key type handling

* Added key decompression with key type handling

* Added handling for '0x' type indentifying

* Added more robust testing

* Less lint for the linting gods

* make vendor for bls12_381

* Added bls12-381 compression tests

* Added decompress key expected results

* Refactor of typed and untyped keys in tests

* Lint god appeasment

* Refactor of sample public keys

* Implemented bls12-381 decompression

* gofmt

* Renamed decode/encode funcs to be more descriptive

* Added binary bindings for key de/compression

* Refactor of func parameters

gomobile is a bit tempermental using raw bytes as a parameter, so I've decided to use string only inputs and outputs

* gofmt

* Added function documentation

* Moved multiformat de/compression into api/multiformat ns

* Moved multiformat de/compression into api/multiformat ns

* Changed compress to serialize on API
2020-06-23 11:47:17 +01:00
Andrea Maria Piana 44272944ab Make sure WakuEnabled is respected on upgrade 2020-03-30 15:28:16 +02:00
Roman Volosovskyi 9d5684162a
Stop fetching new blocks while app is not active
Saves some traffic (blocks with transactions are heavy) and infura
requests.
2020-03-11 08:39:41 +02:00
Adam Babik f335d1cf7e
Prevent starting node without APIModules (#1876) 2020-02-26 20:35:47 +01:00
Roman Volosovskyi 6cc1365536
Start wallet services when log into keycard account 2020-02-26 18:23:04 +02:00
Adam Babik e4b0a97cc5
add bridge metrics to Whisper and Waku (#1864) 2020-02-19 15:36:16 +01:00
Adam Babik 76b5dc29dc
Integrate Whisper-Waku bridge in status-go (#1854) 2020-02-18 12:21:01 +01:00
Andrea Maria Piana 9c2c08d44c
Init protocol when shh is active (#1862) 2020-02-17 18:18:30 +01:00
Andrea Maria Piana 141f917e7c
Initialize protocol when waku active (#1832) 2020-02-17 15:38:59 +01:00
Jakub Sokołowski cca47ef6e1 rename gomarkdown/markdown to status-im/markdown
This resolves a dependency conflict we have with MatterBridge
which was using a newer version of the same package.
This resulted in a JSON marshalling bug that would crash the bridge.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2020-02-11 16:06:43 +01:00
Roman Volosovskyi 9cf640842b
[status-im/status-react#9942] Upgradable paths in configs
Storing absolute path for different configs breaks compatibility on iOS
as app's dir is changed after upgrade. The solution is to store relative
paths and to concatenate it with `backend.rootDataDir`. The only
exception is `LogFile` as it is stored outside `backend.rootDataDir` on
Android. `LogDir` config was added to allow adding of custom dir for log
file.
Configs concerned:
`DataDir`
`LogDir`
`LogFile`
`KeystoreDir`
`BackupDisabledDataDir`
2020-02-03 20:02:01 +02:00
Roman Volosovskyi a92a95cf83
status-im/status-react#9203 Faster tx fetching with less request
*** How it worked before this PR on multiaccount creation:
- On multiacc creation we scanned chain for eth and erc20 transfers. For
  each address of a new empty multiaccount this scan required
  1. two `eth_getBalance` requests to find out that there is no any
     balance change between zero and the last block, for eth transfers
  2. and `chain-size/100000` (currently ~100) `eth_getLogs` requests,
     for erc20 transfers
- For some reason we scanned an address of the chat account as well, and
  also accounts were not deduplicated. So even for an empty multiacc we
  scanned chain twice for each chat and main wallet addresses, in result
  app had to execute about 400 requests.
- As mentioned above, `eth_getBalance` requests were used to check if
  there were any eth transfers, and that caused empty history in case
  if user already used all available eth (so that both zero and latest
  blocks show 0 eth for an address). There might have been transactions
  but we wouldn't fetch/show them.
- There was no upper limit for the number of rpc requests during the
  scan, so it could require indefinite number of requests; the scanning
  algorithm was written so that we persisted the whole history of
  transactions or tried to scan form the beginning again in case of
  failure, giving up only after 10 minutes of failures. In result
  addresses with sufficient number of transactions would never be fully
  scanned and during these 10 minutes app could use gigabytes of
  internet data.
- Failures were caused by `eth_getBlockByNumber`/`eth_getBlockByHash`
  requests. These requests return significantly bigger responses than
  `eth_getBalance`/`eth_transactionsCount` and it is likely that
  execution of thousands of them in parallel caused failures for
  accounts with hundreds of transactions. Even for an account with 12k
  we could successfully determine blocks with transaction in a few
  minutes using `eth_getBalance` requests, but `eth_getBlock...`
  couldn't be processed for this acc.
- There was no caching for for `eth_getBalance` requests, and this
  caused in average 3-4 times more such requests than is needed.

*** How it works now on multiaccount creation:
- On multiacc creation we scan chain for last ~30 eth transactions and
  then check erc20 in the range where these eth transactions were found.
  For an empty address in multiacc this means:
  1. two `eth_getBalance` transactions to determine that there was no
     balance change between zero and the last block; two
     `eth_transactionsCount` requests to determine there are no outgoing
     transactions for this address; total 4 requests for eth transfers
  2. 20 `eth_getLogs` for erc20 transfers. This number can be lowered,
     but that's not a big deal
- Deduplication of addresses is added and also we don't scan chat
  account, so a new multiacc requires ~25 (we also request latest block
  number and probably execute a few other calls) request to determine
  that multiacc is empty (comparing to ~400 before)
- In case if address contains transactions we:
  1. determine the range which contains 20-25 outgoing eth/erc20
     transactions. This usually requires up to 10 `eth_transactionCount`
     requests
  2. then we scan chain for eth transfers using `eth_getBalance` and
     `eth_transactionCount` (for double checking zero balances)
  3. we make sure that we do not scan db for more than 30 blocks with
     transfers. That's important for accounts with mostly incoming
     transactions, because the range found on the first step might
     contain any number of incoming transfers, but only 20-25 outgoing
     transactions
  4. when we found ~30 blocks in a given range, we update initial
     range `from` block using the oldest found block
  5. and now we scan db for erc20transfers using `eth_getLogs`
     `oldest-found-eth-block`-`latest-block`, we make not more than 20 calls
  6. when all blocks which contain incoming/outgoing transfers for a
     given address are found, we save these blocks to db and mark that
     transfers from these blocks are still to be fetched
  7. Then we select latest ~30 (the number can be adjusted) blocks from
     these which were found and fetch transfers, this requires 3-4
     requests per transfer.
  8. we persist scanned range so that we know were to start next time
  9. we dispatch an event which tells client that transactions are found
  10. client fetches latest 20 transfers
- when user presses "fetch more" button we check if app's db contains next
  20 transfers, if not we scan chain again and return transfers after

small fixes
2020-01-23 10:36:11 +02:00
Pedro Pombeiro db01f0b3e4 Avoid passing node to subscriptions service 2020-01-20 13:15:17 +01:00
Pedro Pombeiro 6537cae606 Nimbus node support 2020-01-20 13:15:17 +01:00
Pedro Pombeiro c8a911ebd1 Use goimports instead of gofmt 2020-01-06 10:17:23 +01:00
Pedro Pombeiro 527cd38e1f Address code review comments 2020-01-06 10:17:23 +01:00
Pedro Pombeiro 287e5cdf79 Abstract `accounts.Key` and geth `keystore` 2020-01-06 10:17:23 +01:00
Adam Babik 1ac515f19e remove reSelectAccount from backend, SelectKeyPair and SelectedKeyPairID methods from Whisper 2019-12-30 12:56:05 +01:00
yenda 024f30f0b9 Put settings in a table with schema (#1746)
Move settings table schema from a key-value store to a one row table with many columns.

We now save the first row with initial data in saveAccountAndLogin and follow up saveSetting calls are only saving one setting at a time.

Co-authored-by: Adam Babik <a.babik@designfortress.com>
2019-12-27 10:58:25 +01:00
Pedro Pombeiro b99af530e1 Migrate more files to eth-node/crypto 2019-12-20 11:24:26 +01:00
Pedro Pombeiro dd894ece15 Start abstracting geth Keystore 2019-12-19 14:11:48 +01:00
yenda 655031616c
fix 1741 (#1742)
* fix 1741

save accounts with SaveAccountAndStartNodeWithKey

* initialize keycard accountManager with the proper addresses
2019-12-18 16:09:04 +01:00
Roman Volosovskyi 4c0d8dedea
Replace address with keyUid in accounts db
Account's address was used as a primary key in accounts db and as a
deterministic id of an account in some API calls. Also it was used as a
part of the name of the account specific database. This revealed some
extra information about the account and wasn't necessary.
At first the hash of the address was planned to be used as a
deterministic id, but we already have a keyUid which is calculated as
sha256 hash of account's public key and has similar properties:
- it is deterministic
- doesn't reveal accounts public key or address in plain
2019-12-09 11:20:12 +02:00
Pedro Pombeiro 26b6d7c36a Create a home submodule for Eth node bridges- Rename StatusBackend to GethStatusBackend 2019-11-27 17:02:09 +01:00
Pedro Pombeiro ed5a5c154d
Move to monorepo structure (#1684)
Move to a monorepo structure with submodules

- Rename status-protocol-go to status-go/protocol
2019-11-21 17:19:22 +01:00
Pedro Pombeiro 2dd74da23d Integrate Nimbus status-protocol-go 2019-11-18 14:30:03 +01:00
Pedro Pombeiro 6ce437255e Remove FCM implementation 2019-11-17 17:54:55 +01:00
Jakub Sokołowski 98c861cfe2 make sure Version is always set in loadNodeConfig()
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2019-10-29 12:53:27 +01:00
Adam Babik 26880b83d7
Upgrade geth to 1.9.5 and Whisper (#1617) 2019-10-04 17:21:24 +02:00
yenda 963ea94e08 Update account timestamp when login with keycard (#1614)
Closes #1613
2019-09-24 08:32:32 +02:00
Dmitry Novotochinov d263be10c4
[#1599] send 'node.login' signal when login with keycard (#1600) 2019-09-06 17:23:33 +03:00
Adam Babik d7f454fe4c
Add mailservers service (#1595) 2019-09-04 08:25:33 +02:00
Andrea Maria Piana 42199e682f
Add error to login signal (#1594)
We add an error to login signal so that the caller can tell whether
login was successful or not
2019-09-03 15:00:14 +02:00
Dmitry Shulyak 63dcdd4e94 Statefull login with keycard (#1587) 2019-09-02 21:03:15 +02:00
dmitry bf95a71f6f Verify valid sub-accounts using database 2019-08-30 14:11:00 +03:00
Dmitry Shulyak 0165b028c9
Watch new accounts aftter they were saved to accounts table (#1569)
* Watch new accounts once they are saved in accounts table

* Add test that reactor can be restarted and watch new accounts
2019-08-28 10:49:03 +03:00
Andrea Maria Piana e118e89107
Add persistent datasync, use single database and handle negotiated filters (#1580) 2019-08-27 14:04:15 +02:00
Dmitry a67184adbd Wallet database isolated by the network id
Wallet database refactored so that every query ensures isolation by the network id.
Network id provided when database object is created, thus it is transparent to other parts
of the wallet module.

Additionally every uniqueness index is changed to ensure that it doesn't prevent adding
object with same id but from a different network.
2019-08-21 10:44:50 +03:00
Dmitry 494cb5bb33 Create single database appdatase 2019-08-21 10:44:50 +03:00
Dmitry Shulyak be9c55bc16
Accounts data management (#1530)
* WIP accounts implementation

* Accounts datasore and changes to status mobile API

* Add library changes and method to update config

* Handle error after account selection

* Add two methods to start account to backend

* Use encrypted database for settings and add a service for them

* Resolve linter warning

* Bring back StartNode StopNode for tests

* Add sub accounts and get/save api

* Changes to accounts structure

* Login use root address and fetch necessary info from database

* Cover accounts store with tests

* Refactor in progress

* Initialize status keystore instance before starting ethereum node

* Rework library tests

* Resolve failures in private api test and send transaction test

* Pass pointer to initialized config to unmarshal

* Use multiaccounts/accounts naming consistently

Multiaccount is used as a login identifier
Account references an address and a key, if account is not watch-only.

* Add login timestamp stored in the database to accounts.Account object

* Add photo-path field for multiaccount struct

* Add multiaccoutns rpc with updateAccount method

Update to any other account that wasn't used for login will return an error

* Fix linter in services/accounts

* Select account before starting a node

* Save list of accounts on first login

* Pass account manager to accounts service to avoid selecting account before starting a node

* Add logs to login with save and regualr login
2019-08-20 18:38:40 +03:00
Andrea Maria Piana cb769ccca9 Revert "Initialize status keystore instance before starting ethereum node"
This reverts commit 476240fbd1.
2019-08-05 16:03:18 +02:00
Dmitry 476240fbd1 Initialize status keystore instance before starting ethereum node 2019-08-01 11:01:02 +03:00
Andrea Franz acfe9a721c
Multi account login (#1525)
* multi-account login and signing

put methods count threshold back to 20

* validate login params

* refactoring

* use common.Address

* remove unused var in test
2019-07-26 16:45:10 +02:00
Andrea Franz dcb0fa5262
add multi-account support (#1500)
* add multiaccount support

add multi account ImportPrivateKey and StoreAccount

test derivation from normal keys

* add multiaccount to mobile pkg

* use multiaccount params structs from the mobile pkg

* move multiaccount tests together with the other lib tests

* fix codeclimate warning and temporarily increase methods threshold

* split library_test_utils.go to avoid linter warnings
2019-07-24 20:59:15 +02:00
Adam Babik e93d994460 integrate status-protocol-go 2019-07-24 18:46:43 +02:00
Dmitry Shulyak 9723b64827
Permissions api (#1524)
* Add permissions api

* Integrate permissions service

* Reduce cyclomatic complexity of the MakeNode function
2019-07-21 08:41:30 +03:00