status-im
/
status-go
mirror of https://github.com/status-im/status-go.git
2
0
Fork 0
Code Issues Projects Releases Wiki Activity
status-go/services/wallet/history/service_test.go

285 lines
10 KiB
Go
Raw Normal View History

feat: retrieve balance history for tokens and cache it to DB Extends wallet module with the history package with the following components: BalanceDB (balance_db.go) - Keeps track of balance information (token count, block, block timestamp) for a token identity (chain, address, currency) - The cached data is stored in `balance_history` table. - Uniqueness constrained is enforced by the `balance_history_identify_entry` UNIQUE index. - Optimal DB fetching is ensured by the `balance_history_filter_entries` index Balance (balance.go) - Provides two stages: - Fetch of balance history using RPC calls (Balance.update function) - Retrieving of cached balance data from the DB it exists (Balance.get function) - Fetching and retrieving of data is done for specific time intervals defined by TimeInterval "enumeration" - Update process is done for a token identity by the Balance.Update function - The granularity of data points returned is defined by the constant increment step define in `timeIntervalToStride` for each time interval. - The `blocksStride` values have a common divisor to have cache hit between time intervals. Service (service.go) - Main APIs - StartBalanceHistory: Regularly updates balance history for all enabled networks, available accounts and provided tokens. - GetBalanceHistory: retrieves cached token count for a token identity (chain, address, currency) for multiple chains - UpdateVisibleTokens: will set the list of tokens to have historical balance fetched. This is a simplification to limit tokens to a small list that make sense Fetch balance history for ECR20 tokens - Add token.Manager.GetTokenBalanceAt to fetch balance of a specific block number of ECR20. - Add tokenChainClientSource concrete implementation of DataSource to fetch balance of ECR20 tokens. - Chose the correct DataSource implementation based on the token "is native" property. Tests Tests are implemented using a mock of `DataSource` interface used to intercept the RPC calls. Notes: - the timestamp used for retrieving block balance is constant Closes status-desktop: #8175, #8226, #8862
2022-11-15 12:14:41 +00:00
package history
import (
feat: convert history balance tokens to fiat Add history.exchange sub-package with following responsibilities - fetch and caches daily exchange rates for tokens - Partial update if missing form yesterday back - Implement all time fetching special case - Fetches the price of the token after merging entries for the selected chains history.service package changes - Return ValuePoint instead of DataPoint - Value point contains the value in fiat as float64 instead - Convert merged values from tokens balance (wei) to fiat Other changes - Move start/stop balance history to startWallet/stopWallet - Move cryptocompare to thirdparty package to avoid recursive import - Rename DataPoint.Value to DataPoint.Balance - Don't merge entries for single chain
2023-01-25 18:28:51 +00:00
"context"
"math"
feat: retrieve balance history for tokens and cache it to DB Extends wallet module with the history package with the following components: BalanceDB (balance_db.go) - Keeps track of balance information (token count, block, block timestamp) for a token identity (chain, address, currency) - The cached data is stored in `balance_history` table. - Uniqueness constrained is enforced by the `balance_history_identify_entry` UNIQUE index. - Optimal DB fetching is ensured by the `balance_history_filter_entries` index Balance (balance.go) - Provides two stages: - Fetch of balance history using RPC calls (Balance.update function) - Retrieving of cached balance data from the DB it exists (Balance.get function) - Fetching and retrieving of data is done for specific time intervals defined by TimeInterval "enumeration" - Update process is done for a token identity by the Balance.Update function - The granularity of data points returned is defined by the constant increment step define in `timeIntervalToStride` for each time interval. - The `blocksStride` values have a common divisor to have cache hit between time intervals. Service (service.go) - Main APIs - StartBalanceHistory: Regularly updates balance history for all enabled networks, available accounts and provided tokens. - GetBalanceHistory: retrieves cached token count for a token identity (chain, address, currency) for multiple chains - UpdateVisibleTokens: will set the list of tokens to have historical balance fetched. This is a simplification to limit tokens to a small list that make sense Fetch balance history for ECR20 tokens - Add token.Manager.GetTokenBalanceAt to fetch balance of a specific block number of ECR20. - Add tokenChainClientSource concrete implementation of DataSource to fetch balance of ECR20 tokens. - Chose the correct DataSource implementation based on the token "is native" property. Tests Tests are implemented using a mock of `DataSource` interface used to intercept the RPC calls. Notes: - the timestamp used for retrieving block balance is constant Closes status-desktop: #8175, #8226, #8862
2022-11-15 12:14:41 +00:00
"math/big"
"testing"
fix: merge balance history using block time This change improves on the previous implementation which used the block number that doesn't work with incompatible blockchains e.g. L1 vs. L2 Closes: #9205
2023-01-20 16:06:50 +00:00
"time"
feat: retrieve balance history for tokens and cache it to DB Extends wallet module with the history package with the following components: BalanceDB (balance_db.go) - Keeps track of balance information (token count, block, block timestamp) for a token identity (chain, address, currency) - The cached data is stored in `balance_history` table. - Uniqueness constrained is enforced by the `balance_history_identify_entry` UNIQUE index. - Optimal DB fetching is ensured by the `balance_history_filter_entries` index Balance (balance.go) - Provides two stages: - Fetch of balance history using RPC calls (Balance.update function) - Retrieving of cached balance data from the DB it exists (Balance.get function) - Fetching and retrieving of data is done for specific time intervals defined by TimeInterval "enumeration" - Update process is done for a token identity by the Balance.Update function - The granularity of data points returned is defined by the constant increment step define in `timeIntervalToStride` for each time interval. - The `blocksStride` values have a common divisor to have cache hit between time intervals. Service (service.go) - Main APIs - StartBalanceHistory: Regularly updates balance history for all enabled networks, available accounts and provided tokens. - GetBalanceHistory: retrieves cached token count for a token identity (chain, address, currency) for multiple chains - UpdateVisibleTokens: will set the list of tokens to have historical balance fetched. This is a simplification to limit tokens to a small list that make sense Fetch balance history for ECR20 tokens - Add token.Manager.GetTokenBalanceAt to fetch balance of a specific block number of ECR20. - Add tokenChainClientSource concrete implementation of DataSource to fetch balance of ECR20 tokens. - Chose the correct DataSource implementation based on the token "is native" property. Tests Tests are implemented using a mock of `DataSource` interface used to intercept the RPC calls. Notes: - the timestamp used for retrieving block balance is constant Closes status-desktop: #8175, #8226, #8862
2022-11-15 12:14:41 +00:00
feat: convert history balance tokens to fiat Add history.exchange sub-package with following responsibilities - fetch and caches daily exchange rates for tokens - Partial update if missing form yesterday back - Implement all time fetching special case - Fetches the price of the token after merging entries for the selected chains history.service package changes - Return ValuePoint instead of DataPoint - Value point contains the value in fiat as float64 instead - Convert merged values from tokens balance (wei) to fiat Other changes - Move start/stop balance history to startWallet/stopWallet - Move cryptocompare to thirdparty package to avoid recursive import - Rename DataPoint.Value to DataPoint.Balance - Don't merge entries for single chain
2023-01-25 18:28:51 +00:00
"github.com/ethereum/go-ethereum/common"
feat: retrieve balance history for tokens and cache it to DB Extends wallet module with the history package with the following components: BalanceDB (balance_db.go) - Keeps track of balance information (token count, block, block timestamp) for a token identity (chain, address, currency) - The cached data is stored in `balance_history` table. - Uniqueness constrained is enforced by the `balance_history_identify_entry` UNIQUE index. - Optimal DB fetching is ensured by the `balance_history_filter_entries` index Balance (balance.go) - Provides two stages: - Fetch of balance history using RPC calls (Balance.update function) - Retrieving of cached balance data from the DB it exists (Balance.get function) - Fetching and retrieving of data is done for specific time intervals defined by TimeInterval "enumeration" - Update process is done for a token identity by the Balance.Update function - The granularity of data points returned is defined by the constant increment step define in `timeIntervalToStride` for each time interval. - The `blocksStride` values have a common divisor to have cache hit between time intervals. Service (service.go) - Main APIs - StartBalanceHistory: Regularly updates balance history for all enabled networks, available accounts and provided tokens. - GetBalanceHistory: retrieves cached token count for a token identity (chain, address, currency) for multiple chains - UpdateVisibleTokens: will set the list of tokens to have historical balance fetched. This is a simplification to limit tokens to a small list that make sense Fetch balance history for ECR20 tokens - Add token.Manager.GetTokenBalanceAt to fetch balance of a specific block number of ECR20. - Add tokenChainClientSource concrete implementation of DataSource to fetch balance of ECR20 tokens. - Chose the correct DataSource implementation based on the token "is native" property. Tests Tests are implemented using a mock of `DataSource` interface used to intercept the RPC calls. Notes: - the timestamp used for retrieving block balance is constant Closes status-desktop: #8175, #8226, #8862
2022-11-15 12:14:41 +00:00
"github.com/ethereum/go-ethereum/common/hexutil"
feat: remove consecutive
2023-03-31 09:20:02 +00:00
"github.com/ethereum/go-ethereum/event"
feat: convert history balance tokens to fiat Add history.exchange sub-package with following responsibilities - fetch and caches daily exchange rates for tokens - Partial update if missing form yesterday back - Implement all time fetching special case - Fetches the price of the token after merging entries for the selected chains history.service package changes - Return ValuePoint instead of DataPoint - Value point contains the value in fiat as float64 instead - Convert merged values from tokens balance (wei) to fiat Other changes - Move start/stop balance history to startWallet/stopWallet - Move cryptocompare to thirdparty package to avoid recursive import - Rename DataPoint.Value to DataPoint.Balance - Don't merge entries for single chain
2023-01-25 18:28:51 +00:00
gethrpc "github.com/ethereum/go-ethereum/rpc"
"github.com/golang/mock/gomock"
"github.com/status-im/status-go/params"
statusRPC "github.com/status-im/status-go/rpc"
feat: add coingecko fallback
2023-02-21 09:05:16 +00:00
"github.com/status-im/status-go/services/wallet/market"
"github.com/status-im/status-go/services/wallet/thirdparty/cryptocompare"
fix(tests): moved test db setup to a common place 't/helpers', created interface for initializing db, which is implemented for appdatabase and walletdatabase. TBD for multiaccounts DB. Unified DB initializion for all tests using helpers and new interface. Reduced sqlcipher kdf iterations for all tests to 1.
2023-08-11 11:28:45 +00:00
"github.com/status-im/status-go/t/helpers"
feat: convert history balance tokens to fiat Add history.exchange sub-package with following responsibilities - fetch and caches daily exchange rates for tokens - Partial update if missing form yesterday back - Implement all time fetching special case - Fetches the price of the token after merging entries for the selected chains history.service package changes - Return ValuePoint instead of DataPoint - Value point contains the value in fiat as float64 instead - Convert merged values from tokens balance (wei) to fiat Other changes - Move start/stop balance history to startWallet/stopWallet - Move cryptocompare to thirdparty package to avoid recursive import - Rename DataPoint.Value to DataPoint.Balance - Don't merge entries for single chain
2023-01-25 18:28:51 +00:00
"github.com/status-im/status-go/transactions/fake"
fix(tests): moved test db setup to a common place 't/helpers', created interface for initializing db, which is implemented for appdatabase and walletdatabase. TBD for multiaccounts DB. Unified DB initializion for all tests using helpers and new interface. Reduced sqlcipher kdf iterations for all tests to 1.
2023-08-11 11:28:45 +00:00
"github.com/status-im/status-go/walletdatabase"
feat: retrieve balance history for tokens and cache it to DB Extends wallet module with the history package with the following components: BalanceDB (balance_db.go) - Keeps track of balance information (token count, block, block timestamp) for a token identity (chain, address, currency) - The cached data is stored in `balance_history` table. - Uniqueness constrained is enforced by the `balance_history_identify_entry` UNIQUE index. - Optimal DB fetching is ensured by the `balance_history_filter_entries` index Balance (balance.go) - Provides two stages: - Fetch of balance history using RPC calls (Balance.update function) - Retrieving of cached balance data from the DB it exists (Balance.get function) - Fetching and retrieving of data is done for specific time intervals defined by TimeInterval "enumeration" - Update process is done for a token identity by the Balance.Update function - The granularity of data points returned is defined by the constant increment step define in `timeIntervalToStride` for each time interval. - The `blocksStride` values have a common divisor to have cache hit between time intervals. Service (service.go) - Main APIs - StartBalanceHistory: Regularly updates balance history for all enabled networks, available accounts and provided tokens. - GetBalanceHistory: retrieves cached token count for a token identity (chain, address, currency) for multiple chains - UpdateVisibleTokens: will set the list of tokens to have historical balance fetched. This is a simplification to limit tokens to a small list that make sense Fetch balance history for ECR20 tokens - Add token.Manager.GetTokenBalanceAt to fetch balance of a specific block number of ECR20. - Add tokenChainClientSource concrete implementation of DataSource to fetch balance of ECR20 tokens. - Chose the correct DataSource implementation based on the token "is native" property. Tests Tests are implemented using a mock of `DataSource` interface used to intercept the RPC calls. Notes: - the timestamp used for retrieving block balance is constant Closes status-desktop: #8175, #8226, #8862
2022-11-15 12:14:41 +00:00
"github.com/stretchr/testify/require"
)
feat: convert history balance tokens to fiat Add history.exchange sub-package with following responsibilities - fetch and caches daily exchange rates for tokens - Partial update if missing form yesterday back - Implement all time fetching special case - Fetches the price of the token after merging entries for the selected chains history.service package changes - Return ValuePoint instead of DataPoint - Value point contains the value in fiat as float64 instead - Convert merged values from tokens balance (wei) to fiat Other changes - Move start/stop balance history to startWallet/stopWallet - Move cryptocompare to thirdparty package to avoid recursive import - Rename DataPoint.Value to DataPoint.Balance - Don't merge entries for single chain
2023-01-25 18:28:51 +00:00
func setupDummyServiceNoDependencies(t *testing.T) (service *Service, closeFn func()) {
fix(tests): moved test db setup to a common place 't/helpers', created interface for initializing db, which is implemented for appdatabase and walletdatabase. TBD for multiaccounts DB. Unified DB initializion for all tests using helpers and new interface. Reduced sqlcipher kdf iterations for all tests to 1.
2023-08-11 11:28:45 +00:00
db, err := helpers.SetupTestMemorySQLDB(walletdatabase.DbInitializer{})
feat: convert history balance tokens to fiat Add history.exchange sub-package with following responsibilities - fetch and caches daily exchange rates for tokens - Partial update if missing form yesterday back - Implement all time fetching special case - Fetches the price of the token after merging entries for the selected chains history.service package changes - Return ValuePoint instead of DataPoint - Value point contains the value in fiat as float64 instead - Convert merged values from tokens balance (wei) to fiat Other changes - Move start/stop balance history to startWallet/stopWallet - Move cryptocompare to thirdparty package to avoid recursive import - Rename DataPoint.Value to DataPoint.Balance - Don't merge entries for single chain
2023-01-25 18:28:51 +00:00
require.NoError(t, err)
feat: add coingecko fallback
2023-02-21 09:05:16 +00:00
cryptoCompare := cryptocompare.NewClient()
feat: convert history balance tokens to fiat Add history.exchange sub-package with following responsibilities - fetch and caches daily exchange rates for tokens - Partial update if missing form yesterday back - Implement all time fetching special case - Fetches the price of the token after merging entries for the selected chains history.service package changes - Return ValuePoint instead of DataPoint - Value point contains the value in fiat as float64 instead - Convert merged values from tokens balance (wei) to fiat Other changes - Move start/stop balance history to startWallet/stopWallet - Move cryptocompare to thirdparty package to avoid recursive import - Rename DataPoint.Value to DataPoint.Balance - Don't merge entries for single chain
2023-01-25 18:28:51 +00:00
// Creating a dummy status node to simulate what it's done in get_status_node.go
upstreamConfig := params.UpstreamRPCConfig{
URL: "https://mainnet.infura.io/v3/800c641949d64d768a5070a1b0511938",
Enabled: true,
}
txServiceMockCtrl := gomock.NewController(t)
server, _ := fake.NewTestServer(txServiceMockCtrl)
client := gethrpc.DialInProc(server)
rpcClient, err := statusRPC.NewClient(client, 1, upstreamConfig, nil, db)
require.NoError(t, err)
feat: remove consecutive
2023-03-31 09:20:02 +00:00
return NewService(db, nil, rpcClient, nil, market.NewManager(cryptoCompare, cryptoCompare, &event.Feed{})), func() {
feat: convert history balance tokens to fiat Add history.exchange sub-package with following responsibilities - fetch and caches daily exchange rates for tokens - Partial update if missing form yesterday back - Implement all time fetching special case - Fetches the price of the token after merging entries for the selected chains history.service package changes - Return ValuePoint instead of DataPoint - Value point contains the value in fiat as float64 instead - Convert merged values from tokens balance (wei) to fiat Other changes - Move start/stop balance history to startWallet/stopWallet - Move cryptocompare to thirdparty package to avoid recursive import - Rename DataPoint.Value to DataPoint.Balance - Don't merge entries for single chain
2023-01-25 18:28:51 +00:00
require.NoError(t, db.Close())
}
}
feat: retrieve balance history for tokens and cache it to DB Extends wallet module with the history package with the following components: BalanceDB (balance_db.go) - Keeps track of balance information (token count, block, block timestamp) for a token identity (chain, address, currency) - The cached data is stored in `balance_history` table. - Uniqueness constrained is enforced by the `balance_history_identify_entry` UNIQUE index. - Optimal DB fetching is ensured by the `balance_history_filter_entries` index Balance (balance.go) - Provides two stages: - Fetch of balance history using RPC calls (Balance.update function) - Retrieving of cached balance data from the DB it exists (Balance.get function) - Fetching and retrieving of data is done for specific time intervals defined by TimeInterval "enumeration" - Update process is done for a token identity by the Balance.Update function - The granularity of data points returned is defined by the constant increment step define in `timeIntervalToStride` for each time interval. - The `blocksStride` values have a common divisor to have cache hit between time intervals. Service (service.go) - Main APIs - StartBalanceHistory: Regularly updates balance history for all enabled networks, available accounts and provided tokens. - GetBalanceHistory: retrieves cached token count for a token identity (chain, address, currency) for multiple chains - UpdateVisibleTokens: will set the list of tokens to have historical balance fetched. This is a simplification to limit tokens to a small list that make sense Fetch balance history for ECR20 tokens - Add token.Manager.GetTokenBalanceAt to fetch balance of a specific block number of ECR20. - Add tokenChainClientSource concrete implementation of DataSource to fetch balance of ECR20 tokens. - Chose the correct DataSource implementation based on the token "is native" property. Tests Tests are implemented using a mock of `DataSource` interface used to intercept the RPC calls. Notes: - the timestamp used for retrieving block balance is constant Closes status-desktop: #8175, #8226, #8862
2022-11-15 12:14:41 +00:00
type TestDataPoint struct {
value int64
timestamp uint64
blockNumber int64
fix: merge balance history using block time This change improves on the previous implementation which used the block number that doesn't work with incompatible blockchains e.g. L1 vs. L2 Closes: #9205
2023-01-20 16:06:50 +00:00
chainID chainIdentity
feat: retrieve balance history for tokens and cache it to DB Extends wallet module with the history package with the following components: BalanceDB (balance_db.go) - Keeps track of balance information (token count, block, block timestamp) for a token identity (chain, address, currency) - The cached data is stored in `balance_history` table. - Uniqueness constrained is enforced by the `balance_history_identify_entry` UNIQUE index. - Optimal DB fetching is ensured by the `balance_history_filter_entries` index Balance (balance.go) - Provides two stages: - Fetch of balance history using RPC calls (Balance.update function) - Retrieving of cached balance data from the DB it exists (Balance.get function) - Fetching and retrieving of data is done for specific time intervals defined by TimeInterval "enumeration" - Update process is done for a token identity by the Balance.Update function - The granularity of data points returned is defined by the constant increment step define in `timeIntervalToStride` for each time interval. - The `blocksStride` values have a common divisor to have cache hit between time intervals. Service (service.go) - Main APIs - StartBalanceHistory: Regularly updates balance history for all enabled networks, available accounts and provided tokens. - GetBalanceHistory: retrieves cached token count for a token identity (chain, address, currency) for multiple chains - UpdateVisibleTokens: will set the list of tokens to have historical balance fetched. This is a simplification to limit tokens to a small list that make sense Fetch balance history for ECR20 tokens - Add token.Manager.GetTokenBalanceAt to fetch balance of a specific block number of ECR20. - Add tokenChainClientSource concrete implementation of DataSource to fetch balance of ECR20 tokens. - Chose the correct DataSource implementation based on the token "is native" property. Tests Tests are implemented using a mock of `DataSource` interface used to intercept the RPC calls. Notes: - the timestamp used for retrieving block balance is constant Closes status-desktop: #8175, #8226, #8862
2022-11-15 12:14:41 +00:00
}
// generateTestDataForElementCount generates dummy consecutive blocks of data for the same chain_id, address and currency
fix: merge balance history using block time This change improves on the previous implementation which used the block number that doesn't work with incompatible blockchains e.g. L1 vs. L2 Closes: #9205
2023-01-20 16:06:50 +00:00
func prepareTestData(data []TestDataPoint) map[chainIdentity][]*DataPoint {
res := make(map[chainIdentity][]*DataPoint)
feat: retrieve balance history for tokens and cache it to DB Extends wallet module with the history package with the following components: BalanceDB (balance_db.go) - Keeps track of balance information (token count, block, block timestamp) for a token identity (chain, address, currency) - The cached data is stored in `balance_history` table. - Uniqueness constrained is enforced by the `balance_history_identify_entry` UNIQUE index. - Optimal DB fetching is ensured by the `balance_history_filter_entries` index Balance (balance.go) - Provides two stages: - Fetch of balance history using RPC calls (Balance.update function) - Retrieving of cached balance data from the DB it exists (Balance.get function) - Fetching and retrieving of data is done for specific time intervals defined by TimeInterval "enumeration" - Update process is done for a token identity by the Balance.Update function - The granularity of data points returned is defined by the constant increment step define in `timeIntervalToStride` for each time interval. - The `blocksStride` values have a common divisor to have cache hit between time intervals. Service (service.go) - Main APIs - StartBalanceHistory: Regularly updates balance history for all enabled networks, available accounts and provided tokens. - GetBalanceHistory: retrieves cached token count for a token identity (chain, address, currency) for multiple chains - UpdateVisibleTokens: will set the list of tokens to have historical balance fetched. This is a simplification to limit tokens to a small list that make sense Fetch balance history for ECR20 tokens - Add token.Manager.GetTokenBalanceAt to fetch balance of a specific block number of ECR20. - Add tokenChainClientSource concrete implementation of DataSource to fetch balance of ECR20 tokens. - Chose the correct DataSource implementation based on the token "is native" property. Tests Tests are implemented using a mock of `DataSource` interface used to intercept the RPC calls. Notes: - the timestamp used for retrieving block balance is constant Closes status-desktop: #8175, #8226, #8862
2022-11-15 12:14:41 +00:00
for i := 0; i < len(data); i++ {
entry := data[i]
_, found := res[entry.chainID]
if !found {
res[entry.chainID] = make([]*DataPoint, 0)
}
res[entry.chainID] = append(res[entry.chainID], &DataPoint{
BlockNumber: (*hexutil.Big)(big.NewInt(data[i].blockNumber)),
Timestamp: data[i].timestamp,
feat: convert history balance tokens to fiat Add history.exchange sub-package with following responsibilities - fetch and caches daily exchange rates for tokens - Partial update if missing form yesterday back - Implement all time fetching special case - Fetches the price of the token after merging entries for the selected chains history.service package changes - Return ValuePoint instead of DataPoint - Value point contains the value in fiat as float64 instead - Convert merged values from tokens balance (wei) to fiat Other changes - Move start/stop balance history to startWallet/stopWallet - Move cryptocompare to thirdparty package to avoid recursive import - Rename DataPoint.Value to DataPoint.Balance - Don't merge entries for single chain
2023-01-25 18:28:51 +00:00
Balance: (*hexutil.Big)(big.NewInt(data[i].value)),
feat: retrieve balance history for tokens and cache it to DB Extends wallet module with the history package with the following components: BalanceDB (balance_db.go) - Keeps track of balance information (token count, block, block timestamp) for a token identity (chain, address, currency) - The cached data is stored in `balance_history` table. - Uniqueness constrained is enforced by the `balance_history_identify_entry` UNIQUE index. - Optimal DB fetching is ensured by the `balance_history_filter_entries` index Balance (balance.go) - Provides two stages: - Fetch of balance history using RPC calls (Balance.update function) - Retrieving of cached balance data from the DB it exists (Balance.get function) - Fetching and retrieving of data is done for specific time intervals defined by TimeInterval "enumeration" - Update process is done for a token identity by the Balance.Update function - The granularity of data points returned is defined by the constant increment step define in `timeIntervalToStride` for each time interval. - The `blocksStride` values have a common divisor to have cache hit between time intervals. Service (service.go) - Main APIs - StartBalanceHistory: Regularly updates balance history for all enabled networks, available accounts and provided tokens. - GetBalanceHistory: retrieves cached token count for a token identity (chain, address, currency) for multiple chains - UpdateVisibleTokens: will set the list of tokens to have historical balance fetched. This is a simplification to limit tokens to a small list that make sense Fetch balance history for ECR20 tokens - Add token.Manager.GetTokenBalanceAt to fetch balance of a specific block number of ECR20. - Add tokenChainClientSource concrete implementation of DataSource to fetch balance of ECR20 tokens. - Chose the correct DataSource implementation based on the token "is native" property. Tests Tests are implemented using a mock of `DataSource` interface used to intercept the RPC calls. Notes: - the timestamp used for retrieving block balance is constant Closes status-desktop: #8175, #8226, #8862
2022-11-15 12:14:41 +00:00
})
}
return res
}
fix: merge balance history using block time This change improves on the previous implementation which used the block number that doesn't work with incompatible blockchains e.g. L1 vs. L2 Closes: #9205
2023-01-20 16:06:50 +00:00
// getBlockNumbers returns -1 if block number is nil
feat: retrieve balance history for tokens and cache it to DB Extends wallet module with the history package with the following components: BalanceDB (balance_db.go) - Keeps track of balance information (token count, block, block timestamp) for a token identity (chain, address, currency) - The cached data is stored in `balance_history` table. - Uniqueness constrained is enforced by the `balance_history_identify_entry` UNIQUE index. - Optimal DB fetching is ensured by the `balance_history_filter_entries` index Balance (balance.go) - Provides two stages: - Fetch of balance history using RPC calls (Balance.update function) - Retrieving of cached balance data from the DB it exists (Balance.get function) - Fetching and retrieving of data is done for specific time intervals defined by TimeInterval "enumeration" - Update process is done for a token identity by the Balance.Update function - The granularity of data points returned is defined by the constant increment step define in `timeIntervalToStride` for each time interval. - The `blocksStride` values have a common divisor to have cache hit between time intervals. Service (service.go) - Main APIs - StartBalanceHistory: Regularly updates balance history for all enabled networks, available accounts and provided tokens. - GetBalanceHistory: retrieves cached token count for a token identity (chain, address, currency) for multiple chains - UpdateVisibleTokens: will set the list of tokens to have historical balance fetched. This is a simplification to limit tokens to a small list that make sense Fetch balance history for ECR20 tokens - Add token.Manager.GetTokenBalanceAt to fetch balance of a specific block number of ECR20. - Add tokenChainClientSource concrete implementation of DataSource to fetch balance of ECR20 tokens. - Chose the correct DataSource implementation based on the token "is native" property. Tests Tests are implemented using a mock of `DataSource` interface used to intercept the RPC calls. Notes: - the timestamp used for retrieving block balance is constant Closes status-desktop: #8175, #8226, #8862
2022-11-15 12:14:41 +00:00
func getBlockNumbers(data []*DataPoint) []int64 {
res := make([]int64, 0)
for _, entry := range data {
fix: merge balance history using block time This change improves on the previous implementation which used the block number that doesn't work with incompatible blockchains e.g. L1 vs. L2 Closes: #9205
2023-01-20 16:06:50 +00:00
if entry.BlockNumber == nil {
res = append(res, -1)
} else {
res = append(res, entry.BlockNumber.ToInt().Int64())
}
feat: retrieve balance history for tokens and cache it to DB Extends wallet module with the history package with the following components: BalanceDB (balance_db.go) - Keeps track of balance information (token count, block, block timestamp) for a token identity (chain, address, currency) - The cached data is stored in `balance_history` table. - Uniqueness constrained is enforced by the `balance_history_identify_entry` UNIQUE index. - Optimal DB fetching is ensured by the `balance_history_filter_entries` index Balance (balance.go) - Provides two stages: - Fetch of balance history using RPC calls (Balance.update function) - Retrieving of cached balance data from the DB it exists (Balance.get function) - Fetching and retrieving of data is done for specific time intervals defined by TimeInterval "enumeration" - Update process is done for a token identity by the Balance.Update function - The granularity of data points returned is defined by the constant increment step define in `timeIntervalToStride` for each time interval. - The `blocksStride` values have a common divisor to have cache hit between time intervals. Service (service.go) - Main APIs - StartBalanceHistory: Regularly updates balance history for all enabled networks, available accounts and provided tokens. - GetBalanceHistory: retrieves cached token count for a token identity (chain, address, currency) for multiple chains - UpdateVisibleTokens: will set the list of tokens to have historical balance fetched. This is a simplification to limit tokens to a small list that make sense Fetch balance history for ECR20 tokens - Add token.Manager.GetTokenBalanceAt to fetch balance of a specific block number of ECR20. - Add tokenChainClientSource concrete implementation of DataSource to fetch balance of ECR20 tokens. - Chose the correct DataSource implementation based on the token "is native" property. Tests Tests are implemented using a mock of `DataSource` interface used to intercept the RPC calls. Notes: - the timestamp used for retrieving block balance is constant Closes status-desktop: #8175, #8226, #8862
2022-11-15 12:14:41 +00:00
}
return res
}
func getValues(data []*DataPoint) []int64 {
res := make([]int64, 0)
for _, entry := range data {
feat: convert history balance tokens to fiat Add history.exchange sub-package with following responsibilities - fetch and caches daily exchange rates for tokens - Partial update if missing form yesterday back - Implement all time fetching special case - Fetches the price of the token after merging entries for the selected chains history.service package changes - Return ValuePoint instead of DataPoint - Value point contains the value in fiat as float64 instead - Convert merged values from tokens balance (wei) to fiat Other changes - Move start/stop balance history to startWallet/stopWallet - Move cryptocompare to thirdparty package to avoid recursive import - Rename DataPoint.Value to DataPoint.Balance - Don't merge entries for single chain
2023-01-25 18:28:51 +00:00
res = append(res, entry.Balance.ToInt().Int64())
feat: retrieve balance history for tokens and cache it to DB Extends wallet module with the history package with the following components: BalanceDB (balance_db.go) - Keeps track of balance information (token count, block, block timestamp) for a token identity (chain, address, currency) - The cached data is stored in `balance_history` table. - Uniqueness constrained is enforced by the `balance_history_identify_entry` UNIQUE index. - Optimal DB fetching is ensured by the `balance_history_filter_entries` index Balance (balance.go) - Provides two stages: - Fetch of balance history using RPC calls (Balance.update function) - Retrieving of cached balance data from the DB it exists (Balance.get function) - Fetching and retrieving of data is done for specific time intervals defined by TimeInterval "enumeration" - Update process is done for a token identity by the Balance.Update function - The granularity of data points returned is defined by the constant increment step define in `timeIntervalToStride` for each time interval. - The `blocksStride` values have a common divisor to have cache hit between time intervals. Service (service.go) - Main APIs - StartBalanceHistory: Regularly updates balance history for all enabled networks, available accounts and provided tokens. - GetBalanceHistory: retrieves cached token count for a token identity (chain, address, currency) for multiple chains - UpdateVisibleTokens: will set the list of tokens to have historical balance fetched. This is a simplification to limit tokens to a small list that make sense Fetch balance history for ECR20 tokens - Add token.Manager.GetTokenBalanceAt to fetch balance of a specific block number of ECR20. - Add tokenChainClientSource concrete implementation of DataSource to fetch balance of ECR20 tokens. - Chose the correct DataSource implementation based on the token "is native" property. Tests Tests are implemented using a mock of `DataSource` interface used to intercept the RPC calls. Notes: - the timestamp used for retrieving block balance is constant Closes status-desktop: #8175, #8226, #8862
2022-11-15 12:14:41 +00:00
}
return res
}
func getTimestamps(data []*DataPoint) []int64 {
res := make([]int64, 0)
for _, entry := range data {
res = append(res, int64(entry.Timestamp))
}
return res
}
fix: merge balance history using block time This change improves on the previous implementation which used the block number that doesn't work with incompatible blockchains e.g. L1 vs. L2 Closes: #9205
2023-01-20 16:06:50 +00:00
func TestServiceMergeDataPoints(t *testing.T) {
strideDuration := 5 * time.Second
feat: retrieve balance history for tokens and cache it to DB Extends wallet module with the history package with the following components: BalanceDB (balance_db.go) - Keeps track of balance information (token count, block, block timestamp) for a token identity (chain, address, currency) - The cached data is stored in `balance_history` table. - Uniqueness constrained is enforced by the `balance_history_identify_entry` UNIQUE index. - Optimal DB fetching is ensured by the `balance_history_filter_entries` index Balance (balance.go) - Provides two stages: - Fetch of balance history using RPC calls (Balance.update function) - Retrieving of cached balance data from the DB it exists (Balance.get function) - Fetching and retrieving of data is done for specific time intervals defined by TimeInterval "enumeration" - Update process is done for a token identity by the Balance.Update function - The granularity of data points returned is defined by the constant increment step define in `timeIntervalToStride` for each time interval. - The `blocksStride` values have a common divisor to have cache hit between time intervals. Service (service.go) - Main APIs - StartBalanceHistory: Regularly updates balance history for all enabled networks, available accounts and provided tokens. - GetBalanceHistory: retrieves cached token count for a token identity (chain, address, currency) for multiple chains - UpdateVisibleTokens: will set the list of tokens to have historical balance fetched. This is a simplification to limit tokens to a small list that make sense Fetch balance history for ECR20 tokens - Add token.Manager.GetTokenBalanceAt to fetch balance of a specific block number of ECR20. - Add tokenChainClientSource concrete implementation of DataSource to fetch balance of ECR20 tokens. - Chose the correct DataSource implementation based on the token "is native" property. Tests Tests are implemented using a mock of `DataSource` interface used to intercept the RPC calls. Notes: - the timestamp used for retrieving block balance is constant Closes status-desktop: #8175, #8226, #8862
2022-11-15 12:14:41 +00:00
testData := prepareTestData([]TestDataPoint{
// Drop 100
{value: 1, timestamp: 100, blockNumber: 100, chainID: 1},
{value: 1, timestamp: 100, blockNumber: 100, chainID: 2},
// Keep 105
{value: 1, timestamp: 105, blockNumber: 105, chainID: 1},
{value: 1, timestamp: 105, blockNumber: 105, chainID: 2},
{value: 1, timestamp: 105, blockNumber: 105, chainID: 3},
// Drop 110
{value: 1, timestamp: 105, blockNumber: 105, chainID: 2},
{value: 1, timestamp: 105, blockNumber: 105, chainID: 3},
// Keep 115
{value: 2, timestamp: 115, blockNumber: 115, chainID: 1},
{value: 2, timestamp: 115, blockNumber: 115, chainID: 2},
{value: 2, timestamp: 115, blockNumber: 115, chainID: 3},
// Drop 120
{value: 1, timestamp: 120, blockNumber: 120, chainID: 3},
// Keep 125
{value: 3, timestamp: 125, blockNumber: 125, chainID: 1},
{value: 3, timestamp: 125, blockNumber: 125, chainID: 2},
{value: 3, timestamp: 125, blockNumber: 125, chainID: 3},
// Keep 130
{value: 4, timestamp: 130, blockNumber: 130, chainID: 1},
{value: 4, timestamp: 130, blockNumber: 130, chainID: 2},
{value: 4, timestamp: 130, blockNumber: 130, chainID: 3},
// Drop 135
{value: 1, timestamp: 135, blockNumber: 135, chainID: 1},
})
fix: merge balance history using block time This change improves on the previous implementation which used the block number that doesn't work with incompatible blockchains e.g. L1 vs. L2 Closes: #9205
2023-01-20 16:06:50 +00:00
res, err := mergeDataPoints(testData, strideDuration)
feat: retrieve balance history for tokens and cache it to DB Extends wallet module with the history package with the following components: BalanceDB (balance_db.go) - Keeps track of balance information (token count, block, block timestamp) for a token identity (chain, address, currency) - The cached data is stored in `balance_history` table. - Uniqueness constrained is enforced by the `balance_history_identify_entry` UNIQUE index. - Optimal DB fetching is ensured by the `balance_history_filter_entries` index Balance (balance.go) - Provides two stages: - Fetch of balance history using RPC calls (Balance.update function) - Retrieving of cached balance data from the DB it exists (Balance.get function) - Fetching and retrieving of data is done for specific time intervals defined by TimeInterval "enumeration" - Update process is done for a token identity by the Balance.Update function - The granularity of data points returned is defined by the constant increment step define in `timeIntervalToStride` for each time interval. - The `blocksStride` values have a common divisor to have cache hit between time intervals. Service (service.go) - Main APIs - StartBalanceHistory: Regularly updates balance history for all enabled networks, available accounts and provided tokens. - GetBalanceHistory: retrieves cached token count for a token identity (chain, address, currency) for multiple chains - UpdateVisibleTokens: will set the list of tokens to have historical balance fetched. This is a simplification to limit tokens to a small list that make sense Fetch balance history for ECR20 tokens - Add token.Manager.GetTokenBalanceAt to fetch balance of a specific block number of ECR20. - Add tokenChainClientSource concrete implementation of DataSource to fetch balance of ECR20 tokens. - Chose the correct DataSource implementation based on the token "is native" property. Tests Tests are implemented using a mock of `DataSource` interface used to intercept the RPC calls. Notes: - the timestamp used for retrieving block balance is constant Closes status-desktop: #8175, #8226, #8862
2022-11-15 12:14:41 +00:00
require.NoError(t, err)
require.Equal(t, 4, len(res))
require.Equal(t, []int64{105, 115, 125, 130}, getBlockNumbers(res))
require.Equal(t, []int64{3, 3 * 2, 3 * 3, 3 * 4}, getValues(res))
fix: merge balance history using block time This change improves on the previous implementation which used the block number that doesn't work with incompatible blockchains e.g. L1 vs. L2 Closes: #9205
2023-01-20 16:06:50 +00:00
require.Equal(t, []int64{110, 120, 130, 135}, getTimestamps(res))
feat: retrieve balance history for tokens and cache it to DB Extends wallet module with the history package with the following components: BalanceDB (balance_db.go) - Keeps track of balance information (token count, block, block timestamp) for a token identity (chain, address, currency) - The cached data is stored in `balance_history` table. - Uniqueness constrained is enforced by the `balance_history_identify_entry` UNIQUE index. - Optimal DB fetching is ensured by the `balance_history_filter_entries` index Balance (balance.go) - Provides two stages: - Fetch of balance history using RPC calls (Balance.update function) - Retrieving of cached balance data from the DB it exists (Balance.get function) - Fetching and retrieving of data is done for specific time intervals defined by TimeInterval "enumeration" - Update process is done for a token identity by the Balance.Update function - The granularity of data points returned is defined by the constant increment step define in `timeIntervalToStride` for each time interval. - The `blocksStride` values have a common divisor to have cache hit between time intervals. Service (service.go) - Main APIs - StartBalanceHistory: Regularly updates balance history for all enabled networks, available accounts and provided tokens. - GetBalanceHistory: retrieves cached token count for a token identity (chain, address, currency) for multiple chains - UpdateVisibleTokens: will set the list of tokens to have historical balance fetched. This is a simplification to limit tokens to a small list that make sense Fetch balance history for ECR20 tokens - Add token.Manager.GetTokenBalanceAt to fetch balance of a specific block number of ECR20. - Add tokenChainClientSource concrete implementation of DataSource to fetch balance of ECR20 tokens. - Chose the correct DataSource implementation based on the token "is native" property. Tests Tests are implemented using a mock of `DataSource` interface used to intercept the RPC calls. Notes: - the timestamp used for retrieving block balance is constant Closes status-desktop: #8175, #8226, #8862
2022-11-15 12:14:41 +00:00
}
fix: merge balance history using block time This change improves on the previous implementation which used the block number that doesn't work with incompatible blockchains e.g. L1 vs. L2 Closes: #9205
2023-01-20 16:06:50 +00:00
func TestServiceMergeDataPointsAllMatch(t *testing.T) {
strideDuration := 10 * time.Second
feat: retrieve balance history for tokens and cache it to DB Extends wallet module with the history package with the following components: BalanceDB (balance_db.go) - Keeps track of balance information (token count, block, block timestamp) for a token identity (chain, address, currency) - The cached data is stored in `balance_history` table. - Uniqueness constrained is enforced by the `balance_history_identify_entry` UNIQUE index. - Optimal DB fetching is ensured by the `balance_history_filter_entries` index Balance (balance.go) - Provides two stages: - Fetch of balance history using RPC calls (Balance.update function) - Retrieving of cached balance data from the DB it exists (Balance.get function) - Fetching and retrieving of data is done for specific time intervals defined by TimeInterval "enumeration" - Update process is done for a token identity by the Balance.Update function - The granularity of data points returned is defined by the constant increment step define in `timeIntervalToStride` for each time interval. - The `blocksStride` values have a common divisor to have cache hit between time intervals. Service (service.go) - Main APIs - StartBalanceHistory: Regularly updates balance history for all enabled networks, available accounts and provided tokens. - GetBalanceHistory: retrieves cached token count for a token identity (chain, address, currency) for multiple chains - UpdateVisibleTokens: will set the list of tokens to have historical balance fetched. This is a simplification to limit tokens to a small list that make sense Fetch balance history for ECR20 tokens - Add token.Manager.GetTokenBalanceAt to fetch balance of a specific block number of ECR20. - Add tokenChainClientSource concrete implementation of DataSource to fetch balance of ECR20 tokens. - Chose the correct DataSource implementation based on the token "is native" property. Tests Tests are implemented using a mock of `DataSource` interface used to intercept the RPC calls. Notes: - the timestamp used for retrieving block balance is constant Closes status-desktop: #8175, #8226, #8862
2022-11-15 12:14:41 +00:00
testData := prepareTestData([]TestDataPoint{
// Keep 105
{value: 1, timestamp: 105, blockNumber: 105, chainID: 1},
{value: 1, timestamp: 105, blockNumber: 105, chainID: 2},
{value: 1, timestamp: 105, blockNumber: 105, chainID: 3},
// Keep 115
{value: 2, timestamp: 115, blockNumber: 115, chainID: 1},
{value: 2, timestamp: 115, blockNumber: 115, chainID: 2},
{value: 2, timestamp: 115, blockNumber: 115, chainID: 3},
// Keep 125
{value: 3, timestamp: 125, blockNumber: 125, chainID: 1},
{value: 3, timestamp: 125, blockNumber: 125, chainID: 2},
{value: 3, timestamp: 125, blockNumber: 125, chainID: 3},
// Keep 135
{value: 4, timestamp: 135, blockNumber: 135, chainID: 1},
{value: 4, timestamp: 135, blockNumber: 135, chainID: 2},
{value: 4, timestamp: 135, blockNumber: 135, chainID: 3},
})
fix: merge balance history using block time This change improves on the previous implementation which used the block number that doesn't work with incompatible blockchains e.g. L1 vs. L2 Closes: #9205
2023-01-20 16:06:50 +00:00
res, err := mergeDataPoints(testData, strideDuration)
feat: retrieve balance history for tokens and cache it to DB Extends wallet module with the history package with the following components: BalanceDB (balance_db.go) - Keeps track of balance information (token count, block, block timestamp) for a token identity (chain, address, currency) - The cached data is stored in `balance_history` table. - Uniqueness constrained is enforced by the `balance_history_identify_entry` UNIQUE index. - Optimal DB fetching is ensured by the `balance_history_filter_entries` index Balance (balance.go) - Provides two stages: - Fetch of balance history using RPC calls (Balance.update function) - Retrieving of cached balance data from the DB it exists (Balance.get function) - Fetching and retrieving of data is done for specific time intervals defined by TimeInterval "enumeration" - Update process is done for a token identity by the Balance.Update function - The granularity of data points returned is defined by the constant increment step define in `timeIntervalToStride` for each time interval. - The `blocksStride` values have a common divisor to have cache hit between time intervals. Service (service.go) - Main APIs - StartBalanceHistory: Regularly updates balance history for all enabled networks, available accounts and provided tokens. - GetBalanceHistory: retrieves cached token count for a token identity (chain, address, currency) for multiple chains - UpdateVisibleTokens: will set the list of tokens to have historical balance fetched. This is a simplification to limit tokens to a small list that make sense Fetch balance history for ECR20 tokens - Add token.Manager.GetTokenBalanceAt to fetch balance of a specific block number of ECR20. - Add tokenChainClientSource concrete implementation of DataSource to fetch balance of ECR20 tokens. - Chose the correct DataSource implementation based on the token "is native" property. Tests Tests are implemented using a mock of `DataSource` interface used to intercept the RPC calls. Notes: - the timestamp used for retrieving block balance is constant Closes status-desktop: #8175, #8226, #8862
2022-11-15 12:14:41 +00:00
require.NoError(t, err)
require.Equal(t, 4, len(res))
require.Equal(t, []int64{105, 115, 125, 135}, getBlockNumbers(res))
require.Equal(t, []int64{3, 3 * 2, 3 * 3, 3 * 4}, getValues(res))
fix: merge balance history using block time This change improves on the previous implementation which used the block number that doesn't work with incompatible blockchains e.g. L1 vs. L2 Closes: #9205
2023-01-20 16:06:50 +00:00
require.Equal(t, []int64{115, 125, 135, 145}, getTimestamps(res))
feat: retrieve balance history for tokens and cache it to DB Extends wallet module with the history package with the following components: BalanceDB (balance_db.go) - Keeps track of balance information (token count, block, block timestamp) for a token identity (chain, address, currency) - The cached data is stored in `balance_history` table. - Uniqueness constrained is enforced by the `balance_history_identify_entry` UNIQUE index. - Optimal DB fetching is ensured by the `balance_history_filter_entries` index Balance (balance.go) - Provides two stages: - Fetch of balance history using RPC calls (Balance.update function) - Retrieving of cached balance data from the DB it exists (Balance.get function) - Fetching and retrieving of data is done for specific time intervals defined by TimeInterval "enumeration" - Update process is done for a token identity by the Balance.Update function - The granularity of data points returned is defined by the constant increment step define in `timeIntervalToStride` for each time interval. - The `blocksStride` values have a common divisor to have cache hit between time intervals. Service (service.go) - Main APIs - StartBalanceHistory: Regularly updates balance history for all enabled networks, available accounts and provided tokens. - GetBalanceHistory: retrieves cached token count for a token identity (chain, address, currency) for multiple chains - UpdateVisibleTokens: will set the list of tokens to have historical balance fetched. This is a simplification to limit tokens to a small list that make sense Fetch balance history for ECR20 tokens - Add token.Manager.GetTokenBalanceAt to fetch balance of a specific block number of ECR20. - Add tokenChainClientSource concrete implementation of DataSource to fetch balance of ECR20 tokens. - Chose the correct DataSource implementation based on the token "is native" property. Tests Tests are implemented using a mock of `DataSource` interface used to intercept the RPC calls. Notes: - the timestamp used for retrieving block balance is constant Closes status-desktop: #8175, #8226, #8862
2022-11-15 12:14:41 +00:00
}
fix: merge balance history using block time This change improves on the previous implementation which used the block number that doesn't work with incompatible blockchains e.g. L1 vs. L2 Closes: #9205
2023-01-20 16:06:50 +00:00
func TestServiceMergeDataPointsOneChain(t *testing.T) {
strideDuration := 10 * time.Second
feat: retrieve balance history for tokens and cache it to DB Extends wallet module with the history package with the following components: BalanceDB (balance_db.go) - Keeps track of balance information (token count, block, block timestamp) for a token identity (chain, address, currency) - The cached data is stored in `balance_history` table. - Uniqueness constrained is enforced by the `balance_history_identify_entry` UNIQUE index. - Optimal DB fetching is ensured by the `balance_history_filter_entries` index Balance (balance.go) - Provides two stages: - Fetch of balance history using RPC calls (Balance.update function) - Retrieving of cached balance data from the DB it exists (Balance.get function) - Fetching and retrieving of data is done for specific time intervals defined by TimeInterval "enumeration" - Update process is done for a token identity by the Balance.Update function - The granularity of data points returned is defined by the constant increment step define in `timeIntervalToStride` for each time interval. - The `blocksStride` values have a common divisor to have cache hit between time intervals. Service (service.go) - Main APIs - StartBalanceHistory: Regularly updates balance history for all enabled networks, available accounts and provided tokens. - GetBalanceHistory: retrieves cached token count for a token identity (chain, address, currency) for multiple chains - UpdateVisibleTokens: will set the list of tokens to have historical balance fetched. This is a simplification to limit tokens to a small list that make sense Fetch balance history for ECR20 tokens - Add token.Manager.GetTokenBalanceAt to fetch balance of a specific block number of ECR20. - Add tokenChainClientSource concrete implementation of DataSource to fetch balance of ECR20 tokens. - Chose the correct DataSource implementation based on the token "is native" property. Tests Tests are implemented using a mock of `DataSource` interface used to intercept the RPC calls. Notes: - the timestamp used for retrieving block balance is constant Closes status-desktop: #8175, #8226, #8862
2022-11-15 12:14:41 +00:00
testData := prepareTestData([]TestDataPoint{
// Keep 105
{value: 1, timestamp: 105, blockNumber: 105, chainID: 1},
// Keep 115
{value: 2, timestamp: 115, blockNumber: 115, chainID: 1},
// Keep 125
{value: 3, timestamp: 125, blockNumber: 125, chainID: 1},
})
fix: merge balance history using block time This change improves on the previous implementation which used the block number that doesn't work with incompatible blockchains e.g. L1 vs. L2 Closes: #9205
2023-01-20 16:06:50 +00:00
res, err := mergeDataPoints(testData, strideDuration)
feat: retrieve balance history for tokens and cache it to DB Extends wallet module with the history package with the following components: BalanceDB (balance_db.go) - Keeps track of balance information (token count, block, block timestamp) for a token identity (chain, address, currency) - The cached data is stored in `balance_history` table. - Uniqueness constrained is enforced by the `balance_history_identify_entry` UNIQUE index. - Optimal DB fetching is ensured by the `balance_history_filter_entries` index Balance (balance.go) - Provides two stages: - Fetch of balance history using RPC calls (Balance.update function) - Retrieving of cached balance data from the DB it exists (Balance.get function) - Fetching and retrieving of data is done for specific time intervals defined by TimeInterval "enumeration" - Update process is done for a token identity by the Balance.Update function - The granularity of data points returned is defined by the constant increment step define in `timeIntervalToStride` for each time interval. - The `blocksStride` values have a common divisor to have cache hit between time intervals. Service (service.go) - Main APIs - StartBalanceHistory: Regularly updates balance history for all enabled networks, available accounts and provided tokens. - GetBalanceHistory: retrieves cached token count for a token identity (chain, address, currency) for multiple chains - UpdateVisibleTokens: will set the list of tokens to have historical balance fetched. This is a simplification to limit tokens to a small list that make sense Fetch balance history for ECR20 tokens - Add token.Manager.GetTokenBalanceAt to fetch balance of a specific block number of ECR20. - Add tokenChainClientSource concrete implementation of DataSource to fetch balance of ECR20 tokens. - Chose the correct DataSource implementation based on the token "is native" property. Tests Tests are implemented using a mock of `DataSource` interface used to intercept the RPC calls. Notes: - the timestamp used for retrieving block balance is constant Closes status-desktop: #8175, #8226, #8862
2022-11-15 12:14:41 +00:00
require.NoError(t, err)
require.Equal(t, 3, len(res))
require.Equal(t, []int64{105, 115, 125}, getBlockNumbers(res))
require.Equal(t, []int64{1, 2, 3}, getValues(res))
feat: convert history balance tokens to fiat Add history.exchange sub-package with following responsibilities - fetch and caches daily exchange rates for tokens - Partial update if missing form yesterday back - Implement all time fetching special case - Fetches the price of the token after merging entries for the selected chains history.service package changes - Return ValuePoint instead of DataPoint - Value point contains the value in fiat as float64 instead - Convert merged values from tokens balance (wei) to fiat Other changes - Move start/stop balance history to startWallet/stopWallet - Move cryptocompare to thirdparty package to avoid recursive import - Rename DataPoint.Value to DataPoint.Balance - Don't merge entries for single chain
2023-01-25 18:28:51 +00:00
require.Equal(t, []int64{105, 115, 125}, getTimestamps(res), "Expect no merging for one chain")
feat: retrieve balance history for tokens and cache it to DB Extends wallet module with the history package with the following components: BalanceDB (balance_db.go) - Keeps track of balance information (token count, block, block timestamp) for a token identity (chain, address, currency) - The cached data is stored in `balance_history` table. - Uniqueness constrained is enforced by the `balance_history_identify_entry` UNIQUE index. - Optimal DB fetching is ensured by the `balance_history_filter_entries` index Balance (balance.go) - Provides two stages: - Fetch of balance history using RPC calls (Balance.update function) - Retrieving of cached balance data from the DB it exists (Balance.get function) - Fetching and retrieving of data is done for specific time intervals defined by TimeInterval "enumeration" - Update process is done for a token identity by the Balance.Update function - The granularity of data points returned is defined by the constant increment step define in `timeIntervalToStride` for each time interval. - The `blocksStride` values have a common divisor to have cache hit between time intervals. Service (service.go) - Main APIs - StartBalanceHistory: Regularly updates balance history for all enabled networks, available accounts and provided tokens. - GetBalanceHistory: retrieves cached token count for a token identity (chain, address, currency) for multiple chains - UpdateVisibleTokens: will set the list of tokens to have historical balance fetched. This is a simplification to limit tokens to a small list that make sense Fetch balance history for ECR20 tokens - Add token.Manager.GetTokenBalanceAt to fetch balance of a specific block number of ECR20. - Add tokenChainClientSource concrete implementation of DataSource to fetch balance of ECR20 tokens. - Chose the correct DataSource implementation based on the token "is native" property. Tests Tests are implemented using a mock of `DataSource` interface used to intercept the RPC calls. Notes: - the timestamp used for retrieving block balance is constant Closes status-desktop: #8175, #8226, #8862
2022-11-15 12:14:41 +00:00
}
fix: merge balance history using block time This change improves on the previous implementation which used the block number that doesn't work with incompatible blockchains e.g. L1 vs. L2 Closes: #9205
2023-01-20 16:06:50 +00:00
func TestServiceMergeDataPointsDropAll(t *testing.T) {
strideDuration := 10 * time.Second
feat: retrieve balance history for tokens and cache it to DB Extends wallet module with the history package with the following components: BalanceDB (balance_db.go) - Keeps track of balance information (token count, block, block timestamp) for a token identity (chain, address, currency) - The cached data is stored in `balance_history` table. - Uniqueness constrained is enforced by the `balance_history_identify_entry` UNIQUE index. - Optimal DB fetching is ensured by the `balance_history_filter_entries` index Balance (balance.go) - Provides two stages: - Fetch of balance history using RPC calls (Balance.update function) - Retrieving of cached balance data from the DB it exists (Balance.get function) - Fetching and retrieving of data is done for specific time intervals defined by TimeInterval "enumeration" - Update process is done for a token identity by the Balance.Update function - The granularity of data points returned is defined by the constant increment step define in `timeIntervalToStride` for each time interval. - The `blocksStride` values have a common divisor to have cache hit between time intervals. Service (service.go) - Main APIs - StartBalanceHistory: Regularly updates balance history for all enabled networks, available accounts and provided tokens. - GetBalanceHistory: retrieves cached token count for a token identity (chain, address, currency) for multiple chains - UpdateVisibleTokens: will set the list of tokens to have historical balance fetched. This is a simplification to limit tokens to a small list that make sense Fetch balance history for ECR20 tokens - Add token.Manager.GetTokenBalanceAt to fetch balance of a specific block number of ECR20. - Add tokenChainClientSource concrete implementation of DataSource to fetch balance of ECR20 tokens. - Chose the correct DataSource implementation based on the token "is native" property. Tests Tests are implemented using a mock of `DataSource` interface used to intercept the RPC calls. Notes: - the timestamp used for retrieving block balance is constant Closes status-desktop: #8175, #8226, #8862
2022-11-15 12:14:41 +00:00
testData := prepareTestData([]TestDataPoint{
{value: 1, timestamp: 100, blockNumber: 100, chainID: 1},
fix: merge balance history using block time This change improves on the previous implementation which used the block number that doesn't work with incompatible blockchains e.g. L1 vs. L2 Closes: #9205
2023-01-20 16:06:50 +00:00
{value: 1, timestamp: 110, blockNumber: 110, chainID: 2},
{value: 1, timestamp: 120, blockNumber: 120, chainID: 3},
{value: 1, timestamp: 130, blockNumber: 130, chainID: 4},
feat: retrieve balance history for tokens and cache it to DB Extends wallet module with the history package with the following components: BalanceDB (balance_db.go) - Keeps track of balance information (token count, block, block timestamp) for a token identity (chain, address, currency) - The cached data is stored in `balance_history` table. - Uniqueness constrained is enforced by the `balance_history_identify_entry` UNIQUE index. - Optimal DB fetching is ensured by the `balance_history_filter_entries` index Balance (balance.go) - Provides two stages: - Fetch of balance history using RPC calls (Balance.update function) - Retrieving of cached balance data from the DB it exists (Balance.get function) - Fetching and retrieving of data is done for specific time intervals defined by TimeInterval "enumeration" - Update process is done for a token identity by the Balance.Update function - The granularity of data points returned is defined by the constant increment step define in `timeIntervalToStride` for each time interval. - The `blocksStride` values have a common divisor to have cache hit between time intervals. Service (service.go) - Main APIs - StartBalanceHistory: Regularly updates balance history for all enabled networks, available accounts and provided tokens. - GetBalanceHistory: retrieves cached token count for a token identity (chain, address, currency) for multiple chains - UpdateVisibleTokens: will set the list of tokens to have historical balance fetched. This is a simplification to limit tokens to a small list that make sense Fetch balance history for ECR20 tokens - Add token.Manager.GetTokenBalanceAt to fetch balance of a specific block number of ECR20. - Add tokenChainClientSource concrete implementation of DataSource to fetch balance of ECR20 tokens. - Chose the correct DataSource implementation based on the token "is native" property. Tests Tests are implemented using a mock of `DataSource` interface used to intercept the RPC calls. Notes: - the timestamp used for retrieving block balance is constant Closes status-desktop: #8175, #8226, #8862
2022-11-15 12:14:41 +00:00
})
fix: merge balance history using block time This change improves on the previous implementation which used the block number that doesn't work with incompatible blockchains e.g. L1 vs. L2 Closes: #9205
2023-01-20 16:06:50 +00:00
res, err := mergeDataPoints(testData, strideDuration)
feat: retrieve balance history for tokens and cache it to DB Extends wallet module with the history package with the following components: BalanceDB (balance_db.go) - Keeps track of balance information (token count, block, block timestamp) for a token identity (chain, address, currency) - The cached data is stored in `balance_history` table. - Uniqueness constrained is enforced by the `balance_history_identify_entry` UNIQUE index. - Optimal DB fetching is ensured by the `balance_history_filter_entries` index Balance (balance.go) - Provides two stages: - Fetch of balance history using RPC calls (Balance.update function) - Retrieving of cached balance data from the DB it exists (Balance.get function) - Fetching and retrieving of data is done for specific time intervals defined by TimeInterval "enumeration" - Update process is done for a token identity by the Balance.Update function - The granularity of data points returned is defined by the constant increment step define in `timeIntervalToStride` for each time interval. - The `blocksStride` values have a common divisor to have cache hit between time intervals. Service (service.go) - Main APIs - StartBalanceHistory: Regularly updates balance history for all enabled networks, available accounts and provided tokens. - GetBalanceHistory: retrieves cached token count for a token identity (chain, address, currency) for multiple chains - UpdateVisibleTokens: will set the list of tokens to have historical balance fetched. This is a simplification to limit tokens to a small list that make sense Fetch balance history for ECR20 tokens - Add token.Manager.GetTokenBalanceAt to fetch balance of a specific block number of ECR20. - Add tokenChainClientSource concrete implementation of DataSource to fetch balance of ECR20 tokens. - Chose the correct DataSource implementation based on the token "is native" property. Tests Tests are implemented using a mock of `DataSource` interface used to intercept the RPC calls. Notes: - the timestamp used for retrieving block balance is constant Closes status-desktop: #8175, #8226, #8862
2022-11-15 12:14:41 +00:00
require.NoError(t, err)
require.Equal(t, 0, len(res))
}
fix: merge balance history using block time This change improves on the previous implementation which used the block number that doesn't work with incompatible blockchains e.g. L1 vs. L2 Closes: #9205
2023-01-20 16:06:50 +00:00
func TestServiceMergeDataPointsEmptyDB(t *testing.T) {
feat: retrieve balance history for tokens and cache it to DB Extends wallet module with the history package with the following components: BalanceDB (balance_db.go) - Keeps track of balance information (token count, block, block timestamp) for a token identity (chain, address, currency) - The cached data is stored in `balance_history` table. - Uniqueness constrained is enforced by the `balance_history_identify_entry` UNIQUE index. - Optimal DB fetching is ensured by the `balance_history_filter_entries` index Balance (balance.go) - Provides two stages: - Fetch of balance history using RPC calls (Balance.update function) - Retrieving of cached balance data from the DB it exists (Balance.get function) - Fetching and retrieving of data is done for specific time intervals defined by TimeInterval "enumeration" - Update process is done for a token identity by the Balance.Update function - The granularity of data points returned is defined by the constant increment step define in `timeIntervalToStride` for each time interval. - The `blocksStride` values have a common divisor to have cache hit between time intervals. Service (service.go) - Main APIs - StartBalanceHistory: Regularly updates balance history for all enabled networks, available accounts and provided tokens. - GetBalanceHistory: retrieves cached token count for a token identity (chain, address, currency) for multiple chains - UpdateVisibleTokens: will set the list of tokens to have historical balance fetched. This is a simplification to limit tokens to a small list that make sense Fetch balance history for ECR20 tokens - Add token.Manager.GetTokenBalanceAt to fetch balance of a specific block number of ECR20. - Add tokenChainClientSource concrete implementation of DataSource to fetch balance of ECR20 tokens. - Chose the correct DataSource implementation based on the token "is native" property. Tests Tests are implemented using a mock of `DataSource` interface used to intercept the RPC calls. Notes: - the timestamp used for retrieving block balance is constant Closes status-desktop: #8175, #8226, #8862
2022-11-15 12:14:41 +00:00
testData := prepareTestData([]TestDataPoint{})
fix: merge balance history using block time This change improves on the previous implementation which used the block number that doesn't work with incompatible blockchains e.g. L1 vs. L2 Closes: #9205
2023-01-20 16:06:50 +00:00
strideDuration := 10 * time.Second
res, err := mergeDataPoints(testData, strideDuration)
feat: retrieve balance history for tokens and cache it to DB Extends wallet module with the history package with the following components: BalanceDB (balance_db.go) - Keeps track of balance information (token count, block, block timestamp) for a token identity (chain, address, currency) - The cached data is stored in `balance_history` table. - Uniqueness constrained is enforced by the `balance_history_identify_entry` UNIQUE index. - Optimal DB fetching is ensured by the `balance_history_filter_entries` index Balance (balance.go) - Provides two stages: - Fetch of balance history using RPC calls (Balance.update function) - Retrieving of cached balance data from the DB it exists (Balance.get function) - Fetching and retrieving of data is done for specific time intervals defined by TimeInterval "enumeration" - Update process is done for a token identity by the Balance.Update function - The granularity of data points returned is defined by the constant increment step define in `timeIntervalToStride` for each time interval. - The `blocksStride` values have a common divisor to have cache hit between time intervals. Service (service.go) - Main APIs - StartBalanceHistory: Regularly updates balance history for all enabled networks, available accounts and provided tokens. - GetBalanceHistory: retrieves cached token count for a token identity (chain, address, currency) for multiple chains - UpdateVisibleTokens: will set the list of tokens to have historical balance fetched. This is a simplification to limit tokens to a small list that make sense Fetch balance history for ECR20 tokens - Add token.Manager.GetTokenBalanceAt to fetch balance of a specific block number of ECR20. - Add tokenChainClientSource concrete implementation of DataSource to fetch balance of ECR20 tokens. - Chose the correct DataSource implementation based on the token "is native" property. Tests Tests are implemented using a mock of `DataSource` interface used to intercept the RPC calls. Notes: - the timestamp used for retrieving block balance is constant Closes status-desktop: #8175, #8226, #8862
2022-11-15 12:14:41 +00:00
require.NoError(t, err)
require.Equal(t, 0, len(res))
}
fix: merge balance history using block time This change improves on the previous implementation which used the block number that doesn't work with incompatible blockchains e.g. L1 vs. L2 Closes: #9205
2023-01-20 16:06:50 +00:00
func TestServiceFindFirstStrideWindowFirstForAllChainInOneStride(t *testing.T) {
strideDuration := 10 * time.Second
testData := prepareTestData([]TestDataPoint{
{value: 1, timestamp: 103, blockNumber: 101, chainID: 2},
{value: 1, timestamp: 106, blockNumber: 102, chainID: 3},
{value: 1, timestamp: 100, blockNumber: 100, chainID: 1},
{value: 1, timestamp: 110, blockNumber: 103, chainID: 1},
{value: 1, timestamp: 110, blockNumber: 103, chainID: 2},
})
feat: convert history balance tokens to fiat Add history.exchange sub-package with following responsibilities - fetch and caches daily exchange rates for tokens - Partial update if missing form yesterday back - Implement all time fetching special case - Fetches the price of the token after merging entries for the selected chains history.service package changes - Return ValuePoint instead of DataPoint - Value point contains the value in fiat as float64 instead - Convert merged values from tokens balance (wei) to fiat Other changes - Move start/stop balance history to startWallet/stopWallet - Move cryptocompare to thirdparty package to avoid recursive import - Rename DataPoint.Value to DataPoint.Balance - Don't merge entries for single chain
2023-01-25 18:28:51 +00:00
startTimestamp, pos := findFirstStrideWindow(testData, strideDuration)
fix: merge balance history using block time This change improves on the previous implementation which used the block number that doesn't work with incompatible blockchains e.g. L1 vs. L2 Closes: #9205
2023-01-20 16:06:50 +00:00
require.Equal(t, testData[1][0].Timestamp, uint64(startTimestamp))
feat: convert history balance tokens to fiat Add history.exchange sub-package with following responsibilities - fetch and caches daily exchange rates for tokens - Partial update if missing form yesterday back - Implement all time fetching special case - Fetches the price of the token after merging entries for the selected chains history.service package changes - Return ValuePoint instead of DataPoint - Value point contains the value in fiat as float64 instead - Convert merged values from tokens balance (wei) to fiat Other changes - Move start/stop balance history to startWallet/stopWallet - Move cryptocompare to thirdparty package to avoid recursive import - Rename DataPoint.Value to DataPoint.Balance - Don't merge entries for single chain
2023-01-25 18:28:51 +00:00
require.Equal(t, map[chainIdentity]int{1: 0, 2: 0, 3: 0}, pos)
fix: merge balance history using block time This change improves on the previous implementation which used the block number that doesn't work with incompatible blockchains e.g. L1 vs. L2 Closes: #9205
2023-01-20 16:06:50 +00:00
}
func TestServiceSortTimeAsc(t *testing.T) {
testData := prepareTestData([]TestDataPoint{
{value: 3, timestamp: 103, blockNumber: 103, chainID: 3},
{value: 4, timestamp: 104, blockNumber: 104, chainID: 4},
{value: 2, timestamp: 102, blockNumber: 102, chainID: 2},
{value: 1, timestamp: 101, blockNumber: 101, chainID: 1},
})
sorted := sortTimeAsc(testData, map[chainIdentity]int{4: 0, 3: 0, 2: 0, 1: 0})
require.Equal(t, []timeIdentity{{1, 0}, {2, 0}, {3, 0}, {4, 0}}, sorted)
}
func TestServiceAtEnd(t *testing.T) {
testData := prepareTestData([]TestDataPoint{
{value: 1, timestamp: 101, blockNumber: 101, chainID: 1},
fix: fix wallet.history.TestServiceAtEnd First check was relying on sorting result of two items that had the same weight and is not guaranteed to be deterministic.
2023-01-30 15:40:44 +00:00
{value: 1, timestamp: 103, blockNumber: 103, chainID: 2},
{value: 1, timestamp: 105, blockNumber: 105, chainID: 1},
fix: merge balance history using block time This change improves on the previous implementation which used the block number that doesn't work with incompatible blockchains e.g. L1 vs. L2 Closes: #9205
2023-01-20 16:06:50 +00:00
})
sorted := sortTimeAsc(testData, map[chainIdentity]int{1: 0, 2: 0})
require.False(t, sorted[0].atEnd(testData))
require.True(t, sorted[1].atEnd(testData))
sorted = sortTimeAsc(testData, map[chainIdentity]int{1: 1, 2: 0})
require.True(t, sorted[1].atEnd(testData))
}
feat: convert history balance tokens to fiat Add history.exchange sub-package with following responsibilities - fetch and caches daily exchange rates for tokens - Partial update if missing form yesterday back - Implement all time fetching special case - Fetches the price of the token after merging entries for the selected chains history.service package changes - Return ValuePoint instead of DataPoint - Value point contains the value in fiat as float64 instead - Convert merged values from tokens balance (wei) to fiat Other changes - Move start/stop balance history to startWallet/stopWallet - Move cryptocompare to thirdparty package to avoid recursive import - Rename DataPoint.Value to DataPoint.Balance - Don't merge entries for single chain
2023-01-25 18:28:51 +00:00
func TestServiceTokenToValue(t *testing.T) {
weisInOneMain := big.NewFloat(math.Pow(10, 18.0))
res := tokenToValue(big.NewInt(12345), 1000, weisInOneMain)
require.Equal(t, 0.000000000012345, res)
in, ok := new(big.Int).SetString("1234567890000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000", 10)
require.True(t, ok)
res = tokenToValue(in, 10000, weisInOneMain)
require.Equal(t, 1.23456789e+112, res)
res = tokenToValue(big.NewInt(1000000000000000000), 1.0, weisInOneMain)
require.Equal(t, 1.0, res)
res = tokenToValue(big.NewInt(1), 1.23456789, weisInOneMain)
require.InEpsilonf(t, 1.23456789e-18, res, 1.0e-8, "Expects error for handling such low values")
res = tokenToValue(new(big.Int).Exp(big.NewInt(10), big.NewInt(254), nil), 100000, weisInOneMain)
require.Equal(t, 1e+241, res, "Expect exponent 254-18+5")
}
func TestServiceGetBalanceHistoryNoData(t *testing.T) {
service, closeFn := setupDummyServiceNoDependencies(t)
defer closeFn()
res, err := service.GetBalanceHistory(context.Background(), []uint64{777}, common.HexToAddress(`0x1`), "ETH", "EUR", time.Now().Unix(), BalanceHistory1Year)
require.NoError(t, err)
require.Equal(t, 0, len(res))
}