status-go/services/wallet
Stefan a2ff03c79e feat: retrieve balance history for tokens and cache it to DB
Extends wallet module with the history package with the following
components:

BalanceDB (balance_db.go)

- Keeps track of balance information (token count, block, block timestamp)
for a token identity (chain, address, currency)
- The cached data is stored in `balance_history` table.
- Uniqueness constrained is enforced by the `balance_history_identify_entry`
UNIQUE index.
- Optimal DB fetching is ensured by the `balance_history_filter_entries`
index

Balance (balance.go)

- Provides two stages:
    - Fetch of balance history using RPC calls (Balance.update function)
    - Retrieving of cached balance data from the DB it exists (Balance.get
    function)
- Fetching and retrieving of data is done for specific time intervals
    defined by TimeInterval "enumeration"
- Update process is done for a token identity by the Balance.Update function
- The granularity of data points returned is defined by the constant
increment step define in `timeIntervalToStride` for each time interval.
- The `blocksStride` values have a common divisor to have cache hit
between time intervals.

Service (service.go)

- Main APIs
    - StartBalanceHistory: Regularly updates balance history for all
    enabled networks, available accounts and provided tokens.
    - GetBalanceHistory: retrieves cached token count for a token identity
    (chain, address, currency) for multiple chains
    - UpdateVisibleTokens: will set the list of tokens to have historical
    balance fetched. This is a simplification to limit tokens to a small
    list that make sense

Fetch balance history for ECR20 tokens

- Add token.Manager.GetTokenBalanceAt to fetch balance of a specific
block number of ECR20.
- Add tokenChainClientSource concrete implementation of DataSource
to fetch balance of ECR20 tokens.
- Chose the correct DataSource implementation based on the token
"is native" property.

Tests

Tests are implemented using a mock of `DataSource` interface used
to intercept the RPC calls.

Notes:

- the timestamp used for retrieving block balance is constant

Closes status-desktop: #8175, #8226, #8862
2023-01-25 22:25:50 +04:00
..
async feat: enable wallet without network binding (#2349) 2021-09-09 16:28:54 +02:00
bigint feat: enable wallet without network binding (#2349) 2021-09-09 16:28:54 +02:00
bridge feat: nonce management in multi tx 2023-01-12 12:56:57 +01:00
chain feat: api to return service up or down 2023-01-21 13:11:03 +01:00
history feat: retrieve balance history for tokens and cache it to DB 2023-01-25 22:25:50 +04:00
token feat: retrieve balance history for tokens and cache it to DB 2023-01-25 22:25:50 +04:00
transfer feat: retrieve balance history for tokens and cache it to DB 2023-01-25 22:25:50 +04:00
walletevent feat: add wallet ticker 2022-12-02 10:53:21 +01:00
README.md rename status-react to status-mobile 2022-07-17 15:37:14 +02:00
api.go feat: retrieve balance history for tokens and cache it to DB 2023-01-25 22:25:50 +04:00
cryptocompare.go feat: api to return service up or down 2023-01-21 13:11:03 +01:00
fees.go fix: router fees for EIP1559 2022-11-14 13:32:29 +01:00
on_ramp.go Added httptest.NewServer to DecodeFromURL and CryptoOnRamp tests 2022-09-09 15:07:48 +01:00
on_ramp_test.go Added httptest.NewServer to DecodeFromURL and CryptoOnRamp tests 2022-09-09 15:07:48 +01:00
opensea.go feat: api to return service up or down 2023-01-21 13:11:03 +01:00
opensea_test.go feat: fetch assets from opensea (#2320) 2021-08-20 15:53:24 -04:00
price.go feat: api to return service up or down 2023-01-21 13:11:03 +01:00
price_test.go feat: api to return service up or down 2023-01-21 13:11:03 +01:00
reader.go feat: api to return service up or down 2023-01-21 13:11:03 +01:00
router.go feat(wallet): add price cache (#3107) 2023-01-19 11:49:48 -03:00
saved_addresses.go feat: sync saved addresses 2022-10-10 11:26:35 +02:00
saved_addresses_test.go feat: sync saved addresses 2022-10-10 11:26:35 +02:00
service.go feat: retrieve balance history for tokens and cache it to DB 2023-01-25 22:25:50 +04:00
transaction.go feat: delete message for new design (#2922) (#2965) 2022-11-22 11:12:16 +01:00
transaction_test.go Upgrade linter to 1.50.0 2023-01-13 17:52:03 +00:00

README.md

Wallet service API

Wallet service provides RPC API for checking transfers history and other methods related to wallet functionality. To enable service two values need to be changed in the config:

  1. Set Enable to true in WalletConfig
{
  "WalletConfig": {
    "Enabled": true,
  }
}
  1. And expose wallet API with APIModules
{
  APIModules: "eth,net,web3,peer,wallet",
}

API

wallet_getTransfersByAddress

Returns avaiable transfers in a given range.

Parameters

  • address: HEX - ethereum address encoded in hex
  • toBlock: BIGINT - end of the range. if nil query will return last transfers.
  • limit: BIGINT - limit of returned transfers.
  • fetchMore: BOOLEAN - if true, there are less than limit fetched transfers in the database, and zero block is not reached yet, history will be scanned for more transfers. If false only transfers which are already fetched to the app's database will be returned.

Examples

{
  "jsonrpc":"2.0",
  "id":7,
  "method":"wallet_getTransfersByAddress",
  "params":[
    "0xb81a6845649fa8c042dfaceb3f7a684873406993",
    "0x0",
    "0x5",
    true
  ]
}

Returns

[
  {
    "id":"0xb1a8adeaa0e6727bf01d6d8431b6238bdefa915e19ae7e8ceb16886c9f5e",
    "type":"eth",
    "address":"0xd65f3cb52605a54a833ae118fb13",
    "blockNumber":"0xb7190",
    "blockhash":"0x8d98aa2297fe322d0093b24372e2ead98414959093b479baf670",
    "timestamp":"0x6048ec6",
    "gasPrice":"0x346308a00",
    "gasLimit":"0x508",
    "gasUsed":"0x520",
    "nonce":"0x13",
    "txStatus":"0x1",
    "input":"0x",
    "txHash":"0x1adeaa0e672d7e67bf01d8431b6238bdef15e19ae7e8ceb16886c",
    "value":"0x1",
    "from":"0x2f865fb5dfdf0dfdf54a833ae118fb1363aaasd",
    "to":"0xaaaaaaf3cb52605a54a833ae118fb1363a123123",
    "contract":"0x0000000000000000000000000000000000000000",
    "NetworkID":1
  },...
]

GetTransfersByAddressAndChainID

Returns avaiable transfers in a given range.

Parameters

  • chainID: INT - ethereum chain ID
  • address: HEX - ethereum address encoded in hex
  • toBlock: BIGINT - end of the range. if nil query will return last transfers.
  • limit: BIGINT - limit of returned transfers.
  • fetchMore: BOOLEAN - if true, there are less than limit fetched transfers in the database, and zero block is not reached yet, history will be scanned for more transfers. If false only transfers which are already fetched to the app's database will be returned.

Examples

{
  "jsonrpc":"2.0",
  "id":7,
  "method":"wallet_getTransfersByAddressAndChainID",
  "params":[
    1,
    "0xb81a6845649fa8c042dfaceb3f7a684873406993",
    "0x0",
    "0x5",
    true
  ]
}

Returns

[
  {
    "id":"0xb1a8adeaa0e6727bf01d6d8431b6238bdefa915e19ae7e8ceb16886c9f5e",
    "type":"eth",
    "address":"0xd65f3cb52605a54a833ae118fb13",
    "blockNumber":"0xb7190",
    "blockhash":"0x8d98aa2297fe322d0093b24372e2ead98414959093b479baf670",
    "timestamp":"0x6048ec6",
    "gasPrice":"0x346308a00",
    "gasLimit":"0x508",
    "gasUsed":"0x520",
    "nonce":"0x13",
    "txStatus":"0x1",
    "input":"0x",
    "txHash":"0x1adeaa0e672d7e67bf01d8431b6238bdef15e19ae7e8ceb16886c",
    "value":"0x1",
    "from":"0x2f865fb5dfdf0dfdf54a833ae118fb1363aaasd",
    "to":"0xaaaaaaf3cb52605a54a833ae118fb1363a123123",
    "contract":"0x0000000000000000000000000000000000000000",
    "NetworkID":1
  },...
]

wallet_setInitialBlocksRange

Sets zero block - latest block range as scanned for an account. It is used when a new multiaccount is generated to avoid scanning transfers history.

Example

{"jsonrpc":"2.0","id":7,"method":"wallet_setInitialBlocksRange","params":[]}

setInitialBlocksRangeForChainIDs

Sets zero block - latest block range as scanned for an account. It is used when a new multiaccount is generated to avoid scanning transfers history.

Parameters

  • chainIDs: []INT - array of ethereum chain ID to be initialized

Example

{"jsonrpc":"2.0","id":7,"method":"wallet_setInitialBlocksRangeForChainIDs","params":[[1, 2]]}

wallet_watchTransaction

Starts watching for transaction confirmation/rejection. If transaction was not confirmed/rejected in 10 minutes the call is timed out with error.

Parameters

  • tx-id: HEX - transaction hash

Example

{
  "jsonrpc":"2.0",
  "id":7,
  "method":"wallet_watchTransaction",
  "params":[
    "0xaaaaaaaa11111112222233333333"
  ]
}

wallet_watchTransactionByChainID

Starts watching for transaction confirmation/rejection. If transaction was not confirmed/rejected in 10 minutes the call is timed out with error.

Parameters

  • chainID: HEX - ethereum chain id
  • tx-id: HEX - transaction hash

Example

{
  "jsonrpc":"2.0",
  "id":7,
  "method":"wallet_watchTransactionByChainID",
  "params":[
    1,
    "0xaaaaaaaa11111112222233333333"
  ]
}

wallet_checkRecentHistory

Parameters

  • addresses: []HEX - array of addresses to be checked

Example

{
  "jsonrpc":"2.0",
  "id":1,
  "method":"wallet_checkRecentHistory",
  "params":[
    [
      "0x23458d65f3cB52605a54AaA833ae118fb1111aaa",
      "0x24568B4166D11aaa1194097C60Cdc714F7e11111"
    ]
  ]
}

wallet_checkRecentHistoryForChainIDs

Parameters

  • chainIDs: []INT - array of ethereum chain ID to be checked
  • addresses: []HEX - array of addresses to be checked

Example

{
  "jsonrpc":"2.0",
  "id":1,
  "method":"wallet_checkRecentHistoryForChainIDs",
  "params":[
    [1, 2],
    [
      "0x23458d65f3cB52605a54AaA833ae118fb1111aaa",
      "0x24568B4166D11aaa1194097C60Cdc714F7e11111"
    ]
  ]
}

wallet_getTokensBalancesForChainIDs

Returns tokens balances mapping for every account. See section below for the response example.

Parameters

  • chainIDs: []INT - array of ethereum chain ID
  • accounts HEX - list of ethereum addresses encoded in hex
  • tokens HEX - list of ethereum addresses encoded in hex

Request

{"jsonrpc":"2.0","id":11,"method":"wallet_getTokensBalancesForChainIDs","params":[
  [1, 2]
  ["0x066ed5c2ed45d70ad72f40de0b4dd97bd67d84de", "0x0ed535be4c0aa276942a1a782669790547ad8768"], 
  ["0x5e4bbdc178684478a615354d83c748a4393b20f0", "0x5e4bbdc178684478a615354d83c748a4393b20f0"]]
}

Returns

First level keys accounts, second level keys are tokens.

{
  "0x066ed5c2ed45d70ad72f40de0b4dd97bd67d84de": {
    "0x1dfb2099f936b3e98bfc9b7059a8fb04edcce5b3": 12,
    "0x5e4bbdc178684478a615354d83c748a4393b20f0": 12
  },
  "0x0ed535be4c0aa276942a1a782669790547ad8768": {
    "0x1dfb2099f936b3e98bfc9b7059a8fb04edcce5b3": 14,
    "0x5e4bbdc178684478a615354d83c748a4393b20f0": 14
  }
}

wallet_getTokensBalances

Returns tokens balances mapping for every account. See section below for the response example.

Parameters

  • accounts HEX - list of ethereum addresses encoded in hex
  • tokens HEX - list of ethereum addresses encoded in hex

Request

{"jsonrpc":"2.0","id":11,"method":"wallet_getTokensBalances","params":[["0x066ed5c2ed45d70ad72f40de0b4dd97bd67d84de", "0x0ed535be4c0aa276942a1a782669790547ad8768"], ["0x5e4bbdc178684478a615354d83c748a4393b20f0", "0x5e4bbdc178684478a615354d83c748a4393b20f0"]]}

Returns

First level keys accounts, second level keys are tokens.

{
  "0x066ed5c2ed45d70ad72f40de0b4dd97bd67d84de": {
    "0x1dfb2099f936b3e98bfc9b7059a8fb04edcce5b3": 12,
    "0x5e4bbdc178684478a615354d83c748a4393b20f0": 12
  },
  "0x0ed535be4c0aa276942a1a782669790547ad8768": {
    "0x1dfb2099f936b3e98bfc9b7059a8fb04edcce5b3": 14,
    "0x5e4bbdc178684478a615354d83c748a4393b20f0": 14
  }
}

wallet_storePendingTransaction

Stores pending transation in the database.

Parameters

  • transaction OBJECT - list of ethereum addresses encoded in hex
Transaction
  • hash HEX
  • timestamp ``INT`
  • from HEX
  • to HEX
  • symbol VARCHAR - "ETH" for ethereum, otherwise ERC20 tokaen name, null for contract call
  • gasPrice BIGINT
  • gasLimit BIGINT
  • value BIGINT
  • data TEXT - transaction's data field
  • type VARCHAR
  • additionalData TEXT - arbitrary additional data
  • network_id INT - an optional network id

Request example

{
  "jsonrpc":"2.0",
  "id":1,
  "method":"wallet_storePendingTransaction",
  "params":[
    {
      "hash":"0x3bce2c2d0fffbd2862ef3ec61a62872e54954551585fa0072d8e5c2f6be3523e",
      "symbol":"ETH",
      "gasPrice":"2000000000",
      "value":"1000000000000000",
      "from":"0xaaaad65f3cB52605433ae118fb1363aaaaad2",
      "timestamp":1618584138787,
      "gasLimit":"21000",
      "to":"0x237f8B4166D64a2b94097C60Cdc714F7eC3aa079",
      "data":null
    }
  ]
}

wallet_getPendingTransactions

Returns all stored pending transactions.

Request

{"jsonrpc":"2.0","id":1,"method":"wallet_getPendingTransactions","params":[]}

Returns

First level keys accounts, second level keys are tokens.

{
  "jsonrpc":"2.0",
  "id":1,
  "result":[
    {
      "hash":"0x3bce2c2d0fffbd2862ef3ec61a62872e54954551585fa0072d8e5c2f6be3523e",
      "timestamp":1618584138787,
      "value":"1000000000000000",
      "from":"0xaaaaaaaa605a54a833ae118fb1aaaaaaaaaaa",
      "to":"0x237f8b4166d64a2b94097c60cdc714f7ec3aa079",
      "data":"",
      "symbol":"ETH",
      "gasPrice":"2000000000",
      "gasLimit":"21000",
      "type":"",
      "additionalData":""
    },
    ...
  ]
}

wallet_getPendingTransactionsByChainID

Returns all stored pending transactions.

Parameters

  • chainID INT - ethereum chain ID

Request

{"jsonrpc":"2.0","id":1,"method":"wallet_getPendingTransactions","params":[1]}

Returns

First level keys accounts, second level keys are tokens.

{
  "jsonrpc":"2.0",
  "id":1,
  "result":[
    {
      "hash":"0x3bce2c2d0fffbd2862ef3ec61a62872e54954551585fa0072d8e5c2f6be3523e",
      "timestamp":1618584138787,
      "value":"1000000000000000",
      "from":"0xaaaaaaaa605a54a833ae118fb1aaaaaaaaaaa",
      "to":"0x237f8b4166d64a2b94097c60cdc714f7ec3aa079",
      "data":"",
      "symbol":"ETH",
      "gasPrice":"2000000000",
      "gasLimit":"21000",
      "type":"",
      "additionalData":"",
      "network_id": 1
    },
    ...
  ]
}

wallet_getPendingOutboundTransactionsByAddress

Returns all stored pending transaction sent from address.

Parameters

  • address HEX

Request

{
  "jsonrpc":"2.0",
  "id":1,
  "method":"wallet_getPendingOutboundTransactionsByAddress",
  "params":[
    "0xaaaaaaaa605a54a833ae118fb1aaaaaaaaaaa"
  ]
}

Returns

First level keys accounts, second level keys are tokens.

{
  "jsonrpc":"2.0",
  "id":1,
  "result":[
    {
      "hash":"0x3bce2c2d0fffbd2862ef3ec61a62872e54954551585fa0072d8e5c2f6be3523e",
      "timestamp":1618584138787,
      "value":"1000000000000000",
      "from":"0xaaaaaaaa605a54a833ae118fb1aaaaaaaaaaa",
      "to":"0x237f8b4166d64a2b94097c60cdc714f7ec3aa079",
      "data":"",
      "symbol":"ETH",
      "gasPrice":"2000000000",
      "gasLimit":"21000",
      "type":"",
      "additionalData":""
    },
    ...
  ]
}

wallet_getPendingOutboundTransactionsByAddressAndChainID

Returns all stored pending transaction sent from address.

Parameters

  • chainID INT
  • address HEX

Request

{
  "jsonrpc":"2.0",
  "id":1,
  "method":"wallet_getPendingOutboundTransactionsByAddress",
  "params":[
    1,
    "0xaaaaaaaa605a54a833ae118fb1aaaaaaaaaaa"
  ]
}

Returns

First level keys accounts, second level keys are tokens.

{
  "jsonrpc":"2.0",
  "id":1,
  "result":[
    {
      "hash":"0x3bce2c2d0fffbd2862ef3ec61a62872e54954551585fa0072d8e5c2f6be3523e",
      "timestamp":1618584138787,
      "value":"1000000000000000",
      "from":"0xaaaaaaaa605a54a833ae118fb1aaaaaaaaaaa",
      "to":"0x237f8b4166d64a2b94097c60cdc714f7ec3aa079",
      "data":"",
      "symbol":"ETH",
      "gasPrice":"2000000000",
      "gasLimit":"21000",
      "type":"",
      "additionalData":"",
      "network_id": 1
    },
    ...
  ]
}

wallet_deletePendingTransaction

Deletes pending transaction from the database by hash.

Parameters

  • hash HEX

Request

{
  "jsonrpc":"2.0",
  "id":1,
  "method":"wallet_deletePendingTransaction",
  "params":[
    "0x3bce2c2d0fffbd2862ef3ec61a62872e54954551585fa0072d8e5c2f6be3523e"
  ]
}

wallet_deletePendingTransactionByChainID

Deletes pending transaction from the database by hash.

Parameters

  • chainID INT
  • hash HEX

Request

{
  "jsonrpc":"2.0",
  "id":1,
  "method":"wallet_deletePendingTransaction",
  "params":[
    1,
    "0x3bce2c2d0fffbd2862ef3ec61a62872e54954551585fa0072d8e5c2f6be3523e"
  ]
}

Signals


All events are of the same format:

{
  "type": "wallet",
  "event": {
    "type": "event-type",
    "blockNumber": 0,
    "accounts": [
      "0x42c8f505b4006d417dd4e0ba0e880692986adbd8",
      "0x3129mdasmeo132128391fml1130410k312312mll"
    ],
    "message": "something might be here"
  }
}
  1. new-transfers

Emitted when transfers are detected. In this case block number is a block number of the latest found transfer. Client expected to request transfers starting from received block.

  1. recent-history-fetching

Emitted when history scanning is started.

  1. recent-history-ready

Emitted when history scanning is ended.

  1. fetching-history-error

Emitted when when history can't be fetched because some error. Error's decritption can be found in message field.

  1. non-archival-node-detected

Emitted when the application is connected to a non-archival node.

Flows

Account creation

When a new multiaccount is created corresponding address will not contain any transaction. Thus no point in checking history, it will be empty.

  1. Call wallet_setInitialRange
  2. Call wallet_checkRecentHistory
  3. On recent-history-ready request transactions via wallet_getTransfersByAddress
  4. Repeat wallet_checkRecentHistory in N minutes (currently 20 minutes in status-mobile for upstream RPC node. If a custom node is used interval can be arbitrary)

Logging into application

  1. Call wallet_checkRecentHistory
  2. On recent-history-ready request transactions via wallet_getTransfersByAddress
  3. Repeat wallet_checkRecentHistory in N minutes (currently 20 minutes in status-mobile for upstream RPC node. If a custom node is used interval can be arbitrary)

Watching transaction

  1. Call wallet_watchTransaction
  2. On success call wallet_checkRecentHistory
  3. On recent-history-ready request transactions via wallet_getTransfersByAddress