status-im
/
status-go
mirror of https://github.com/status-im/status-go.git
2
0
Fork 0
Code Issues Projects Releases Wiki Activity
status-go/services/wallet/history/service_test.go

282 lines
9.8 KiB
Go
Raw Normal View History

feat: retrieve balance history for tokens and cache it to DB Extends wallet module with the history package with the following components: BalanceDB (balance_db.go) - Keeps track of balance information (token count, block, block timestamp) for a token identity (chain, address, currency) - The cached data is stored in `balance_history` table. - Uniqueness constrained is enforced by the `balance_history_identify_entry` UNIQUE index. - Optimal DB fetching is ensured by the `balance_history_filter_entries` index Balance (balance.go) - Provides two stages: - Fetch of balance history using RPC calls (Balance.update function) - Retrieving of cached balance data from the DB it exists (Balance.get function) - Fetching and retrieving of data is done for specific time intervals defined by TimeInterval "enumeration" - Update process is done for a token identity by the Balance.Update function - The granularity of data points returned is defined by the constant increment step define in `timeIntervalToStride` for each time interval. - The `blocksStride` values have a common divisor to have cache hit between time intervals. Service (service.go) - Main APIs - StartBalanceHistory: Regularly updates balance history for all enabled networks, available accounts and provided tokens. - GetBalanceHistory: retrieves cached token count for a token identity (chain, address, currency) for multiple chains - UpdateVisibleTokens: will set the list of tokens to have historical balance fetched. This is a simplification to limit tokens to a small list that make sense Fetch balance history for ECR20 tokens - Add token.Manager.GetTokenBalanceAt to fetch balance of a specific block number of ECR20. - Add tokenChainClientSource concrete implementation of DataSource to fetch balance of ECR20 tokens. - Chose the correct DataSource implementation based on the token "is native" property. Tests Tests are implemented using a mock of `DataSource` interface used to intercept the RPC calls. Notes: - the timestamp used for retrieving block balance is constant Closes status-desktop: #8175, #8226, #8862
2022-11-15 12:14:41 +00:00
package history
import (
feat: convert history balance tokens to fiat Add history.exchange sub-package with following responsibilities - fetch and caches daily exchange rates for tokens - Partial update if missing form yesterday back - Implement all time fetching special case - Fetches the price of the token after merging entries for the selected chains history.service package changes - Return ValuePoint instead of DataPoint - Value point contains the value in fiat as float64 instead - Convert merged values from tokens balance (wei) to fiat Other changes - Move start/stop balance history to startWallet/stopWallet - Move cryptocompare to thirdparty package to avoid recursive import - Rename DataPoint.Value to DataPoint.Balance - Don't merge entries for single chain
2023-01-25 18:28:51 +00:00
"context"
"math"
feat: retrieve balance history for tokens and cache it to DB Extends wallet module with the history package with the following components: BalanceDB (balance_db.go) - Keeps track of balance information (token count, block, block timestamp) for a token identity (chain, address, currency) - The cached data is stored in `balance_history` table. - Uniqueness constrained is enforced by the `balance_history_identify_entry` UNIQUE index. - Optimal DB fetching is ensured by the `balance_history_filter_entries` index Balance (balance.go) - Provides two stages: - Fetch of balance history using RPC calls (Balance.update function) - Retrieving of cached balance data from the DB it exists (Balance.get function) - Fetching and retrieving of data is done for specific time intervals defined by TimeInterval "enumeration" - Update process is done for a token identity by the Balance.Update function - The granularity of data points returned is defined by the constant increment step define in `timeIntervalToStride` for each time interval. - The `blocksStride` values have a common divisor to have cache hit between time intervals. Service (service.go) - Main APIs - StartBalanceHistory: Regularly updates balance history for all enabled networks, available accounts and provided tokens. - GetBalanceHistory: retrieves cached token count for a token identity (chain, address, currency) for multiple chains - UpdateVisibleTokens: will set the list of tokens to have historical balance fetched. This is a simplification to limit tokens to a small list that make sense Fetch balance history for ECR20 tokens - Add token.Manager.GetTokenBalanceAt to fetch balance of a specific block number of ECR20. - Add tokenChainClientSource concrete implementation of DataSource to fetch balance of ECR20 tokens. - Chose the correct DataSource implementation based on the token "is native" property. Tests Tests are implemented using a mock of `DataSource` interface used to intercept the RPC calls. Notes: - the timestamp used for retrieving block balance is constant Closes status-desktop: #8175, #8226, #8862
2022-11-15 12:14:41 +00:00
"math/big"
"testing"
fix: merge balance history using block time This change improves on the previous implementation which used the block number that doesn't work with incompatible blockchains e.g. L1 vs. L2 Closes: #9205
2023-01-20 16:06:50 +00:00
"time"
feat: retrieve balance history for tokens and cache it to DB Extends wallet module with the history package with the following components: BalanceDB (balance_db.go) - Keeps track of balance information (token count, block, block timestamp) for a token identity (chain, address, currency) - The cached data is stored in `balance_history` table. - Uniqueness constrained is enforced by the `balance_history_identify_entry` UNIQUE index. - Optimal DB fetching is ensured by the `balance_history_filter_entries` index Balance (balance.go) - Provides two stages: - Fetch of balance history using RPC calls (Balance.update function) - Retrieving of cached balance data from the DB it exists (Balance.get function) - Fetching and retrieving of data is done for specific time intervals defined by TimeInterval "enumeration" - Update process is done for a token identity by the Balance.Update function - The granularity of data points returned is defined by the constant increment step define in `timeIntervalToStride` for each time interval. - The `blocksStride` values have a common divisor to have cache hit between time intervals. Service (service.go) - Main APIs - StartBalanceHistory: Regularly updates balance history for all enabled networks, available accounts and provided tokens. - GetBalanceHistory: retrieves cached token count for a token identity (chain, address, currency) for multiple chains - UpdateVisibleTokens: will set the list of tokens to have historical balance fetched. This is a simplification to limit tokens to a small list that make sense Fetch balance history for ECR20 tokens - Add token.Manager.GetTokenBalanceAt to fetch balance of a specific block number of ECR20. - Add tokenChainClientSource concrete implementation of DataSource to fetch balance of ECR20 tokens. - Chose the correct DataSource implementation based on the token "is native" property. Tests Tests are implemented using a mock of `DataSource` interface used to intercept the RPC calls. Notes: - the timestamp used for retrieving block balance is constant Closes status-desktop: #8175, #8226, #8862
2022-11-15 12:14:41 +00:00
feat: convert history balance tokens to fiat Add history.exchange sub-package with following responsibilities - fetch and caches daily exchange rates for tokens - Partial update if missing form yesterday back - Implement all time fetching special case - Fetches the price of the token after merging entries for the selected chains history.service package changes - Return ValuePoint instead of DataPoint - Value point contains the value in fiat as float64 instead - Convert merged values from tokens balance (wei) to fiat Other changes - Move start/stop balance history to startWallet/stopWallet - Move cryptocompare to thirdparty package to avoid recursive import - Rename DataPoint.Value to DataPoint.Balance - Don't merge entries for single chain
2023-01-25 18:28:51 +00:00
"github.com/ethereum/go-ethereum/common"
feat: retrieve balance history for tokens and cache it to DB Extends wallet module with the history package with the following components: BalanceDB (balance_db.go) - Keeps track of balance information (token count, block, block timestamp) for a token identity (chain, address, currency) - The cached data is stored in `balance_history` table. - Uniqueness constrained is enforced by the `balance_history_identify_entry` UNIQUE index. - Optimal DB fetching is ensured by the `balance_history_filter_entries` index Balance (balance.go) - Provides two stages: - Fetch of balance history using RPC calls (Balance.update function) - Retrieving of cached balance data from the DB it exists (Balance.get function) - Fetching and retrieving of data is done for specific time intervals defined by TimeInterval "enumeration" - Update process is done for a token identity by the Balance.Update function - The granularity of data points returned is defined by the constant increment step define in `timeIntervalToStride` for each time interval. - The `blocksStride` values have a common divisor to have cache hit between time intervals. Service (service.go) - Main APIs - StartBalanceHistory: Regularly updates balance history for all enabled networks, available accounts and provided tokens. - GetBalanceHistory: retrieves cached token count for a token identity (chain, address, currency) for multiple chains - UpdateVisibleTokens: will set the list of tokens to have historical balance fetched. This is a simplification to limit tokens to a small list that make sense Fetch balance history for ECR20 tokens - Add token.Manager.GetTokenBalanceAt to fetch balance of a specific block number of ECR20. - Add tokenChainClientSource concrete implementation of DataSource to fetch balance of ECR20 tokens. - Chose the correct DataSource implementation based on the token "is native" property. Tests Tests are implemented using a mock of `DataSource` interface used to intercept the RPC calls. Notes: - the timestamp used for retrieving block balance is constant Closes status-desktop: #8175, #8226, #8862
2022-11-15 12:14:41 +00:00
"github.com/ethereum/go-ethereum/common/hexutil"
feat: convert history balance tokens to fiat Add history.exchange sub-package with following responsibilities - fetch and caches daily exchange rates for tokens - Partial update if missing form yesterday back - Implement all time fetching special case - Fetches the price of the token after merging entries for the selected chains history.service package changes - Return ValuePoint instead of DataPoint - Value point contains the value in fiat as float64 instead - Convert merged values from tokens balance (wei) to fiat Other changes - Move start/stop balance history to startWallet/stopWallet - Move cryptocompare to thirdparty package to avoid recursive import - Rename DataPoint.Value to DataPoint.Balance - Don't merge entries for single chain
2023-01-25 18:28:51 +00:00
gethrpc "github.com/ethereum/go-ethereum/rpc"
"github.com/golang/mock/gomock"
"github.com/status-im/status-go/appdatabase"
"github.com/status-im/status-go/params"
statusRPC "github.com/status-im/status-go/rpc"
"github.com/status-im/status-go/services/wallet/thirdparty"
"github.com/status-im/status-go/transactions/fake"
feat: retrieve balance history for tokens and cache it to DB Extends wallet module with the history package with the following components: BalanceDB (balance_db.go) - Keeps track of balance information (token count, block, block timestamp) for a token identity (chain, address, currency) - The cached data is stored in `balance_history` table. - Uniqueness constrained is enforced by the `balance_history_identify_entry` UNIQUE index. - Optimal DB fetching is ensured by the `balance_history_filter_entries` index Balance (balance.go) - Provides two stages: - Fetch of balance history using RPC calls (Balance.update function) - Retrieving of cached balance data from the DB it exists (Balance.get function) - Fetching and retrieving of data is done for specific time intervals defined by TimeInterval "enumeration" - Update process is done for a token identity by the Balance.Update function - The granularity of data points returned is defined by the constant increment step define in `timeIntervalToStride` for each time interval. - The `blocksStride` values have a common divisor to have cache hit between time intervals. Service (service.go) - Main APIs - StartBalanceHistory: Regularly updates balance history for all enabled networks, available accounts and provided tokens. - GetBalanceHistory: retrieves cached token count for a token identity (chain, address, currency) for multiple chains - UpdateVisibleTokens: will set the list of tokens to have historical balance fetched. This is a simplification to limit tokens to a small list that make sense Fetch balance history for ECR20 tokens - Add token.Manager.GetTokenBalanceAt to fetch balance of a specific block number of ECR20. - Add tokenChainClientSource concrete implementation of DataSource to fetch balance of ECR20 tokens. - Chose the correct DataSource implementation based on the token "is native" property. Tests Tests are implemented using a mock of `DataSource` interface used to intercept the RPC calls. Notes: - the timestamp used for retrieving block balance is constant Closes status-desktop: #8175, #8226, #8862
2022-11-15 12:14:41 +00:00
"github.com/stretchr/testify/require"
)
feat: convert history balance tokens to fiat Add history.exchange sub-package with following responsibilities - fetch and caches daily exchange rates for tokens - Partial update if missing form yesterday back - Implement all time fetching special case - Fetches the price of the token after merging entries for the selected chains history.service package changes - Return ValuePoint instead of DataPoint - Value point contains the value in fiat as float64 instead - Convert merged values from tokens balance (wei) to fiat Other changes - Move start/stop balance history to startWallet/stopWallet - Move cryptocompare to thirdparty package to avoid recursive import - Rename DataPoint.Value to DataPoint.Balance - Don't merge entries for single chain
2023-01-25 18:28:51 +00:00
func setupDummyServiceNoDependencies(t *testing.T) (service *Service, closeFn func()) {
db, err := appdatabase.InitializeDB(":memory:", "wallet-history-service-tests", 1)
require.NoError(t, err)
cryptoCompare := thirdparty.NewCryptoCompare()
// Creating a dummy status node to simulate what it's done in get_status_node.go
upstreamConfig := params.UpstreamRPCConfig{
URL: "https://mainnet.infura.io/v3/800c641949d64d768a5070a1b0511938",
Enabled: true,
}
txServiceMockCtrl := gomock.NewController(t)
server, _ := fake.NewTestServer(txServiceMockCtrl)
client := gethrpc.DialInProc(server)
rpcClient, err := statusRPC.NewClient(client, 1, upstreamConfig, nil, db)
require.NoError(t, err)
return NewService(db, nil, rpcClient, nil, cryptoCompare), func() {
require.NoError(t, db.Close())
}
}
feat: retrieve balance history for tokens and cache it to DB Extends wallet module with the history package with the following components: BalanceDB (balance_db.go) - Keeps track of balance information (token count, block, block timestamp) for a token identity (chain, address, currency) - The cached data is stored in `balance_history` table. - Uniqueness constrained is enforced by the `balance_history_identify_entry` UNIQUE index. - Optimal DB fetching is ensured by the `balance_history_filter_entries` index Balance (balance.go) - Provides two stages: - Fetch of balance history using RPC calls (Balance.update function) - Retrieving of cached balance data from the DB it exists (Balance.get function) - Fetching and retrieving of data is done for specific time intervals defined by TimeInterval "enumeration" - Update process is done for a token identity by the Balance.Update function - The granularity of data points returned is defined by the constant increment step define in `timeIntervalToStride` for each time interval. - The `blocksStride` values have a common divisor to have cache hit between time intervals. Service (service.go) - Main APIs - StartBalanceHistory: Regularly updates balance history for all enabled networks, available accounts and provided tokens. - GetBalanceHistory: retrieves cached token count for a token identity (chain, address, currency) for multiple chains - UpdateVisibleTokens: will set the list of tokens to have historical balance fetched. This is a simplification to limit tokens to a small list that make sense Fetch balance history for ECR20 tokens - Add token.Manager.GetTokenBalanceAt to fetch balance of a specific block number of ECR20. - Add tokenChainClientSource concrete implementation of DataSource to fetch balance of ECR20 tokens. - Chose the correct DataSource implementation based on the token "is native" property. Tests Tests are implemented using a mock of `DataSource` interface used to intercept the RPC calls. Notes: - the timestamp used for retrieving block balance is constant Closes status-desktop: #8175, #8226, #8862
2022-11-15 12:14:41 +00:00
type TestDataPoint struct {
value int64
timestamp uint64
blockNumber int64
fix: merge balance history using block time This change improves on the previous implementation which used the block number that doesn't work with incompatible blockchains e.g. L1 vs. L2 Closes: #9205
2023-01-20 16:06:50 +00:00
chainID chainIdentity
feat: retrieve balance history for tokens and cache it to DB Extends wallet module with the history package with the following components: BalanceDB (balance_db.go) - Keeps track of balance information (token count, block, block timestamp) for a token identity (chain, address, currency) - The cached data is stored in `balance_history` table. - Uniqueness constrained is enforced by the `balance_history_identify_entry` UNIQUE index. - Optimal DB fetching is ensured by the `balance_history_filter_entries` index Balance (balance.go) - Provides two stages: - Fetch of balance history using RPC calls (Balance.update function) - Retrieving of cached balance data from the DB it exists (Balance.get function) - Fetching and retrieving of data is done for specific time intervals defined by TimeInterval "enumeration" - Update process is done for a token identity by the Balance.Update function - The granularity of data points returned is defined by the constant increment step define in `timeIntervalToStride` for each time interval. - The `blocksStride` values have a common divisor to have cache hit between time intervals. Service (service.go) - Main APIs - StartBalanceHistory: Regularly updates balance history for all enabled networks, available accounts and provided tokens. - GetBalanceHistory: retrieves cached token count for a token identity (chain, address, currency) for multiple chains - UpdateVisibleTokens: will set the list of tokens to have historical balance fetched. This is a simplification to limit tokens to a small list that make sense Fetch balance history for ECR20 tokens - Add token.Manager.GetTokenBalanceAt to fetch balance of a specific block number of ECR20. - Add tokenChainClientSource concrete implementation of DataSource to fetch balance of ECR20 tokens. - Chose the correct DataSource implementation based on the token "is native" property. Tests Tests are implemented using a mock of `DataSource` interface used to intercept the RPC calls. Notes: - the timestamp used for retrieving block balance is constant Closes status-desktop: #8175, #8226, #8862
2022-11-15 12:14:41 +00:00
}
// generateTestDataForElementCount generates dummy consecutive blocks of data for the same chain_id, address and currency
fix: merge balance history using block time This change improves on the previous implementation which used the block number that doesn't work with incompatible blockchains e.g. L1 vs. L2 Closes: #9205
2023-01-20 16:06:50 +00:00
func prepareTestData(data []TestDataPoint) map[chainIdentity][]*DataPoint {
res := make(map[chainIdentity][]*DataPoint)
feat: retrieve balance history for tokens and cache it to DB Extends wallet module with the history package with the following components: BalanceDB (balance_db.go) - Keeps track of balance information (token count, block, block timestamp) for a token identity (chain, address, currency) - The cached data is stored in `balance_history` table. - Uniqueness constrained is enforced by the `balance_history_identify_entry` UNIQUE index. - Optimal DB fetching is ensured by the `balance_history_filter_entries` index Balance (balance.go) - Provides two stages: - Fetch of balance history using RPC calls (Balance.update function) - Retrieving of cached balance data from the DB it exists (Balance.get function) - Fetching and retrieving of data is done for specific time intervals defined by TimeInterval "enumeration" - Update process is done for a token identity by the Balance.Update function - The granularity of data points returned is defined by the constant increment step define in `timeIntervalToStride` for each time interval. - The `blocksStride` values have a common divisor to have cache hit between time intervals. Service (service.go) - Main APIs - StartBalanceHistory: Regularly updates balance history for all enabled networks, available accounts and provided tokens. - GetBalanceHistory: retrieves cached token count for a token identity (chain, address, currency) for multiple chains - UpdateVisibleTokens: will set the list of tokens to have historical balance fetched. This is a simplification to limit tokens to a small list that make sense Fetch balance history for ECR20 tokens - Add token.Manager.GetTokenBalanceAt to fetch balance of a specific block number of ECR20. - Add tokenChainClientSource concrete implementation of DataSource to fetch balance of ECR20 tokens. - Chose the correct DataSource implementation based on the token "is native" property. Tests Tests are implemented using a mock of `DataSource` interface used to intercept the RPC calls. Notes: - the timestamp used for retrieving block balance is constant Closes status-desktop: #8175, #8226, #8862
2022-11-15 12:14:41 +00:00
for i := 0; i < len(data); i++ {
entry := data[i]
_, found := res[entry.chainID]
if !found {
res[entry.chainID] = make([]*DataPoint, 0)
}
res[entry.chainID] = append(res[entry.chainID], &DataPoint{
BlockNumber: (*hexutil.Big)(big.NewInt(data[i].blockNumber)),
Timestamp: data[i].timestamp,
feat: convert history balance tokens to fiat Add history.exchange sub-package with following responsibilities - fetch and caches daily exchange rates for tokens - Partial update if missing form yesterday back - Implement all time fetching special case - Fetches the price of the token after merging entries for the selected chains history.service package changes - Return ValuePoint instead of DataPoint - Value point contains the value in fiat as float64 instead - Convert merged values from tokens balance (wei) to fiat Other changes - Move start/stop balance history to startWallet/stopWallet - Move cryptocompare to thirdparty package to avoid recursive import - Rename DataPoint.Value to DataPoint.Balance - Don't merge entries for single chain
2023-01-25 18:28:51 +00:00
Balance: (*hexutil.Big)(big.NewInt(data[i].value)),
feat: retrieve balance history for tokens and cache it to DB Extends wallet module with the history package with the following components: BalanceDB (balance_db.go) - Keeps track of balance information (token count, block, block timestamp) for a token identity (chain, address, currency) - The cached data is stored in `balance_history` table. - Uniqueness constrained is enforced by the `balance_history_identify_entry` UNIQUE index. - Optimal DB fetching is ensured by the `balance_history_filter_entries` index Balance (balance.go) - Provides two stages: - Fetch of balance history using RPC calls (Balance.update function) - Retrieving of cached balance data from the DB it exists (Balance.get function) - Fetching and retrieving of data is done for specific time intervals defined by TimeInterval "enumeration" - Update process is done for a token identity by the Balance.Update function - The granularity of data points returned is defined by the constant increment step define in `timeIntervalToStride` for each time interval. - The `blocksStride` values have a common divisor to have cache hit between time intervals. Service (service.go) - Main APIs - StartBalanceHistory: Regularly updates balance history for all enabled networks, available accounts and provided tokens. - GetBalanceHistory: retrieves cached token count for a token identity (chain, address, currency) for multiple chains - UpdateVisibleTokens: will set the list of tokens to have historical balance fetched. This is a simplification to limit tokens to a small list that make sense Fetch balance history for ECR20 tokens - Add token.Manager.GetTokenBalanceAt to fetch balance of a specific block number of ECR20. - Add tokenChainClientSource concrete implementation of DataSource to fetch balance of ECR20 tokens. - Chose the correct DataSource implementation based on the token "is native" property. Tests Tests are implemented using a mock of `DataSource` interface used to intercept the RPC calls. Notes: - the timestamp used for retrieving block balance is constant Closes status-desktop: #8175, #8226, #8862
2022-11-15 12:14:41 +00:00
})
}
return res
}
fix: merge balance history using block time This change improves on the previous implementation which used the block number that doesn't work with incompatible blockchains e.g. L1 vs. L2 Closes: #9205
2023-01-20 16:06:50 +00:00
// getBlockNumbers returns -1 if block number is nil
feat: retrieve balance history for tokens and cache it to DB Extends wallet module with the history package with the following components: BalanceDB (balance_db.go) - Keeps track of balance information (token count, block, block timestamp) for a token identity (chain, address, currency) - The cached data is stored in `balance_history` table. - Uniqueness constrained is enforced by the `balance_history_identify_entry` UNIQUE index. - Optimal DB fetching is ensured by the `balance_history_filter_entries` index Balance (balance.go) - Provides two stages: - Fetch of balance history using RPC calls (Balance.update function) - Retrieving of cached balance data from the DB it exists (Balance.get function) - Fetching and retrieving of data is done for specific time intervals defined by TimeInterval "enumeration" - Update process is done for a token identity by the Balance.Update function - The granularity of data points returned is defined by the constant increment step define in `timeIntervalToStride` for each time interval. - The `blocksStride` values have a common divisor to have cache hit between time intervals. Service (service.go) - Main APIs - StartBalanceHistory: Regularly updates balance history for all enabled networks, available accounts and provided tokens. - GetBalanceHistory: retrieves cached token count for a token identity (chain, address, currency) for multiple chains - UpdateVisibleTokens: will set the list of tokens to have historical balance fetched. This is a simplification to limit tokens to a small list that make sense Fetch balance history for ECR20 tokens - Add token.Manager.GetTokenBalanceAt to fetch balance of a specific block number of ECR20. - Add tokenChainClientSource concrete implementation of DataSource to fetch balance of ECR20 tokens. - Chose the correct DataSource implementation based on the token "is native" property. Tests Tests are implemented using a mock of `DataSource` interface used to intercept the RPC calls. Notes: - the timestamp used for retrieving block balance is constant Closes status-desktop: #8175, #8226, #8862
2022-11-15 12:14:41 +00:00
func getBlockNumbers(data []*DataPoint) []int64 {
res := make([]int64, 0)
for _, entry := range data {
fix: merge balance history using block time This change improves on the previous implementation which used the block number that doesn't work with incompatible blockchains e.g. L1 vs. L2 Closes: #9205
2023-01-20 16:06:50 +00:00
if entry.BlockNumber == nil {
res = append(res, -1)
} else {
res = append(res, entry.BlockNumber.ToInt().Int64())
}
feat: retrieve balance history for tokens and cache it to DB Extends wallet module with the history package with the following components: BalanceDB (balance_db.go) - Keeps track of balance information (token count, block, block timestamp) for a token identity (chain, address, currency) - The cached data is stored in `balance_history` table. - Uniqueness constrained is enforced by the `balance_history_identify_entry` UNIQUE index. - Optimal DB fetching is ensured by the `balance_history_filter_entries` index Balance (balance.go) - Provides two stages: - Fetch of balance history using RPC calls (Balance.update function) - Retrieving of cached balance data from the DB it exists (Balance.get function) - Fetching and retrieving of data is done for specific time intervals defined by TimeInterval "enumeration" - Update process is done for a token identity by the Balance.Update function - The granularity of data points returned is defined by the constant increment step define in `timeIntervalToStride` for each time interval. - The `blocksStride` values have a common divisor to have cache hit between time intervals. Service (service.go) - Main APIs - StartBalanceHistory: Regularly updates balance history for all enabled networks, available accounts and provided tokens. - GetBalanceHistory: retrieves cached token count for a token identity (chain, address, currency) for multiple chains - UpdateVisibleTokens: will set the list of tokens to have historical balance fetched. This is a simplification to limit tokens to a small list that make sense Fetch balance history for ECR20 tokens - Add token.Manager.GetTokenBalanceAt to fetch balance of a specific block number of ECR20. - Add tokenChainClientSource concrete implementation of DataSource to fetch balance of ECR20 tokens. - Chose the correct DataSource implementation based on the token "is native" property. Tests Tests are implemented using a mock of `DataSource` interface used to intercept the RPC calls. Notes: - the timestamp used for retrieving block balance is constant Closes status-desktop: #8175, #8226, #8862
2022-11-15 12:14:41 +00:00
}
return res
}
func getValues(data []*DataPoint) []int64 {
res := make([]int64, 0)
for _, entry := range data {
feat: convert history balance tokens to fiat Add history.exchange sub-package with following responsibilities - fetch and caches daily exchange rates for tokens - Partial update if missing form yesterday back - Implement all time fetching special case - Fetches the price of the token after merging entries for the selected chains history.service package changes - Return ValuePoint instead of DataPoint - Value point contains the value in fiat as float64 instead - Convert merged values from tokens balance (wei) to fiat Other changes - Move start/stop balance history to startWallet/stopWallet - Move cryptocompare to thirdparty package to avoid recursive import - Rename DataPoint.Value to DataPoint.Balance - Don't merge entries for single chain
2023-01-25 18:28:51 +00:00
res = append(res, entry.Balance.ToInt().Int64())
feat: retrieve balance history for tokens and cache it to DB Extends wallet module with the history package with the following components: BalanceDB (balance_db.go) - Keeps track of balance information (token count, block, block timestamp) for a token identity (chain, address, currency) - The cached data is stored in `balance_history` table. - Uniqueness constrained is enforced by the `balance_history_identify_entry` UNIQUE index. - Optimal DB fetching is ensured by the `balance_history_filter_entries` index Balance (balance.go) - Provides two stages: - Fetch of balance history using RPC calls (Balance.update function) - Retrieving of cached balance data from the DB it exists (Balance.get function) - Fetching and retrieving of data is done for specific time intervals defined by TimeInterval "enumeration" - Update process is done for a token identity by the Balance.Update function - The granularity of data points returned is defined by the constant increment step define in `timeIntervalToStride` for each time interval. - The `blocksStride` values have a common divisor to have cache hit between time intervals. Service (service.go) - Main APIs - StartBalanceHistory: Regularly updates balance history for all enabled networks, available accounts and provided tokens. - GetBalanceHistory: retrieves cached token count for a token identity (chain, address, currency) for multiple chains - UpdateVisibleTokens: will set the list of tokens to have historical balance fetched. This is a simplification to limit tokens to a small list that make sense Fetch balance history for ECR20 tokens - Add token.Manager.GetTokenBalanceAt to fetch balance of a specific block number of ECR20. - Add tokenChainClientSource concrete implementation of DataSource to fetch balance of ECR20 tokens. - Chose the correct DataSource implementation based on the token "is native" property. Tests Tests are implemented using a mock of `DataSource` interface used to intercept the RPC calls. Notes: - the timestamp used for retrieving block balance is constant Closes status-desktop: #8175, #8226, #8862
2022-11-15 12:14:41 +00:00
}
return res
}
func getTimestamps(data []*DataPoint) []int64 {
res := make([]int64, 0)
for _, entry := range data {
res = append(res, int64(entry.Timestamp))
}
return res
}
fix: merge balance history using block time This change improves on the previous implementation which used the block number that doesn't work with incompatible blockchains e.g. L1 vs. L2 Closes: #9205
2023-01-20 16:06:50 +00:00
func TestServiceMergeDataPoints(t *testing.T) {
strideDuration := 5 * time.Second
feat: retrieve balance history for tokens and cache it to DB Extends wallet module with the history package with the following components: BalanceDB (balance_db.go) - Keeps track of balance information (token count, block, block timestamp) for a token identity (chain, address, currency) - The cached data is stored in `balance_history` table. - Uniqueness constrained is enforced by the `balance_history_identify_entry` UNIQUE index. - Optimal DB fetching is ensured by the `balance_history_filter_entries` index Balance (balance.go) - Provides two stages: - Fetch of balance history using RPC calls (Balance.update function) - Retrieving of cached balance data from the DB it exists (Balance.get function) - Fetching and retrieving of data is done for specific time intervals defined by TimeInterval "enumeration" - Update process is done for a token identity by the Balance.Update function - The granularity of data points returned is defined by the constant increment step define in `timeIntervalToStride` for each time interval. - The `blocksStride` values have a common divisor to have cache hit between time intervals. Service (service.go) - Main APIs - StartBalanceHistory: Regularly updates balance history for all enabled networks, available accounts and provided tokens. - GetBalanceHistory: retrieves cached token count for a token identity (chain, address, currency) for multiple chains - UpdateVisibleTokens: will set the list of tokens to have historical balance fetched. This is a simplification to limit tokens to a small list that make sense Fetch balance history for ECR20 tokens - Add token.Manager.GetTokenBalanceAt to fetch balance of a specific block number of ECR20. - Add tokenChainClientSource concrete implementation of DataSource to fetch balance of ECR20 tokens. - Chose the correct DataSource implementation based on the token "is native" property. Tests Tests are implemented using a mock of `DataSource` interface used to intercept the RPC calls. Notes: - the timestamp used for retrieving block balance is constant Closes status-desktop: #8175, #8226, #8862
2022-11-15 12:14:41 +00:00
testData := prepareTestData([]TestDataPoint{
// Drop 100
{value: 1, timestamp: 100, blockNumber: 100, chainID: 1},
{value: 1, timestamp: 100, blockNumber: 100, chainID: 2},
// Keep 105
{value: 1, timestamp: 105, blockNumber: 105, chainID: 1},
{value: 1, timestamp: 105, blockNumber: 105, chainID: 2},
{value: 1, timestamp: 105, blockNumber: 105, chainID: 3},
// Drop 110
{value: 1, timestamp: 105, blockNumber: 105, chainID: 2},
{value: 1, timestamp: 105, blockNumber: 105, chainID: 3},
// Keep 115
{value: 2, timestamp: 115, blockNumber: 115, chainID: 1},
{value: 2, timestamp: 115, blockNumber: 115, chainID: 2},
{value: 2, timestamp: 115, blockNumber: 115, chainID: 3},
// Drop 120
{value: 1, timestamp: 120, blockNumber: 120, chainID: 3},
// Keep 125
{value: 3, timestamp: 125, blockNumber: 125, chainID: 1},
{value: 3, timestamp: 125, blockNumber: 125, chainID: 2},
{value: 3, timestamp: 125, blockNumber: 125, chainID: 3},
// Keep 130
{value: 4, timestamp: 130, blockNumber: 130, chainID: 1},
{value: 4, timestamp: 130, blockNumber: 130, chainID: 2},
{value: 4, timestamp: 130, blockNumber: 130, chainID: 3},
// Drop 135
{value: 1, timestamp: 135, blockNumber: 135, chainID: 1},
})
fix: merge balance history using block time This change improves on the previous implementation which used the block number that doesn't work with incompatible blockchains e.g. L1 vs. L2 Closes: #9205
2023-01-20 16:06:50 +00:00
res, err := mergeDataPoints(testData, strideDuration)
feat: retrieve balance history for tokens and cache it to DB Extends wallet module with the history package with the following components: BalanceDB (balance_db.go) - Keeps track of balance information (token count, block, block timestamp) for a token identity (chain, address, currency) - The cached data is stored in `balance_history` table. - Uniqueness constrained is enforced by the `balance_history_identify_entry` UNIQUE index. - Optimal DB fetching is ensured by the `balance_history_filter_entries` index Balance (balance.go) - Provides two stages: - Fetch of balance history using RPC calls (Balance.update function) - Retrieving of cached balance data from the DB it exists (Balance.get function) - Fetching and retrieving of data is done for specific time intervals defined by TimeInterval "enumeration" - Update process is done for a token identity by the Balance.Update function - The granularity of data points returned is defined by the constant increment step define in `timeIntervalToStride` for each time interval. - The `blocksStride` values have a common divisor to have cache hit between time intervals. Service (service.go) - Main APIs - StartBalanceHistory: Regularly updates balance history for all enabled networks, available accounts and provided tokens. - GetBalanceHistory: retrieves cached token count for a token identity (chain, address, currency) for multiple chains - UpdateVisibleTokens: will set the list of tokens to have historical balance fetched. This is a simplification to limit tokens to a small list that make sense Fetch balance history for ECR20 tokens - Add token.Manager.GetTokenBalanceAt to fetch balance of a specific block number of ECR20. - Add tokenChainClientSource concrete implementation of DataSource to fetch balance of ECR20 tokens. - Chose the correct DataSource implementation based on the token "is native" property. Tests Tests are implemented using a mock of `DataSource` interface used to intercept the RPC calls. Notes: - the timestamp used for retrieving block balance is constant Closes status-desktop: #8175, #8226, #8862
2022-11-15 12:14:41 +00:00
require.NoError(t, err)
require.Equal(t, 4, len(res))
require.Equal(t, []int64{105, 115, 125, 130}, getBlockNumbers(res))
require.Equal(t, []int64{3, 3 * 2, 3 * 3, 3 * 4}, getValues(res))
fix: merge balance history using block time This change improves on the previous implementation which used the block number that doesn't work with incompatible blockchains e.g. L1 vs. L2 Closes: #9205
2023-01-20 16:06:50 +00:00
require.Equal(t, []int64{110, 120, 130, 135}, getTimestamps(res))
feat: retrieve balance history for tokens and cache it to DB Extends wallet module with the history package with the following components: BalanceDB (balance_db.go) - Keeps track of balance information (token count, block, block timestamp) for a token identity (chain, address, currency) - The cached data is stored in `balance_history` table. - Uniqueness constrained is enforced by the `balance_history_identify_entry` UNIQUE index. - Optimal DB fetching is ensured by the `balance_history_filter_entries` index Balance (balance.go) - Provides two stages: - Fetch of balance history using RPC calls (Balance.update function) - Retrieving of cached balance data from the DB it exists (Balance.get function) - Fetching and retrieving of data is done for specific time intervals defined by TimeInterval "enumeration" - Update process is done for a token identity by the Balance.Update function - The granularity of data points returned is defined by the constant increment step define in `timeIntervalToStride` for each time interval. - The `blocksStride` values have a common divisor to have cache hit between time intervals. Service (service.go) - Main APIs - StartBalanceHistory: Regularly updates balance history for all enabled networks, available accounts and provided tokens. - GetBalanceHistory: retrieves cached token count for a token identity (chain, address, currency) for multiple chains - UpdateVisibleTokens: will set the list of tokens to have historical balance fetched. This is a simplification to limit tokens to a small list that make sense Fetch balance history for ECR20 tokens - Add token.Manager.GetTokenBalanceAt to fetch balance of a specific block number of ECR20. - Add tokenChainClientSource concrete implementation of DataSource to fetch balance of ECR20 tokens. - Chose the correct DataSource implementation based on the token "is native" property. Tests Tests are implemented using a mock of `DataSource` interface used to intercept the RPC calls. Notes: - the timestamp used for retrieving block balance is constant Closes status-desktop: #8175, #8226, #8862
2022-11-15 12:14:41 +00:00
}
fix: merge balance history using block time This change improves on the previous implementation which used the block number that doesn't work with incompatible blockchains e.g. L1 vs. L2 Closes: #9205
2023-01-20 16:06:50 +00:00
func TestServiceMergeDataPointsAllMatch(t *testing.T) {
strideDuration := 10 * time.Second
feat: retrieve balance history for tokens and cache it to DB Extends wallet module with the history package with the following components: BalanceDB (balance_db.go) - Keeps track of balance information (token count, block, block timestamp) for a token identity (chain, address, currency) - The cached data is stored in `balance_history` table. - Uniqueness constrained is enforced by the `balance_history_identify_entry` UNIQUE index. - Optimal DB fetching is ensured by the `balance_history_filter_entries` index Balance (balance.go) - Provides two stages: - Fetch of balance history using RPC calls (Balance.update function) - Retrieving of cached balance data from the DB it exists (Balance.get function) - Fetching and retrieving of data is done for specific time intervals defined by TimeInterval "enumeration" - Update process is done for a token identity by the Balance.Update function - The granularity of data points returned is defined by the constant increment step define in `timeIntervalToStride` for each time interval. - The `blocksStride` values have a common divisor to have cache hit between time intervals. Service (service.go) - Main APIs - StartBalanceHistory: Regularly updates balance history for all enabled networks, available accounts and provided tokens. - GetBalanceHistory: retrieves cached token count for a token identity (chain, address, currency) for multiple chains - UpdateVisibleTokens: will set the list of tokens to have historical balance fetched. This is a simplification to limit tokens to a small list that make sense Fetch balance history for ECR20 tokens - Add token.Manager.GetTokenBalanceAt to fetch balance of a specific block number of ECR20. - Add tokenChainClientSource concrete implementation of DataSource to fetch balance of ECR20 tokens. - Chose the correct DataSource implementation based on the token "is native" property. Tests Tests are implemented using a mock of `DataSource` interface used to intercept the RPC calls. Notes: - the timestamp used for retrieving block balance is constant Closes status-desktop: #8175, #8226, #8862
2022-11-15 12:14:41 +00:00
testData := prepareTestData([]TestDataPoint{
// Keep 105
{value: 1, timestamp: 105, blockNumber: 105, chainID: 1},
{value: 1, timestamp: 105, blockNumber: 105, chainID: 2},
{value: 1, timestamp: 105, blockNumber: 105, chainID: 3},
// Keep 115
{value: 2, timestamp: 115, blockNumber: 115, chainID: 1},
{value: 2, timestamp: 115, blockNumber: 115, chainID: 2},
{value: 2, timestamp: 115, blockNumber: 115, chainID: 3},
// Keep 125
{value: 3, timestamp: 125, blockNumber: 125, chainID: 1},
{value: 3, timestamp: 125, blockNumber: 125, chainID: 2},
{value: 3, timestamp: 125, blockNumber: 125, chainID: 3},
// Keep 135
{value: 4, timestamp: 135, blockNumber: 135, chainID: 1},
{value: 4, timestamp: 135, blockNumber: 135, chainID: 2},
{value: 4, timestamp: 135, blockNumber: 135, chainID: 3},
})
fix: merge balance history using block time This change improves on the previous implementation which used the block number that doesn't work with incompatible blockchains e.g. L1 vs. L2 Closes: #9205
2023-01-20 16:06:50 +00:00
res, err := mergeDataPoints(testData, strideDuration)
feat: retrieve balance history for tokens and cache it to DB Extends wallet module with the history package with the following components: BalanceDB (balance_db.go) - Keeps track of balance information (token count, block, block timestamp) for a token identity (chain, address, currency) - The cached data is stored in `balance_history` table. - Uniqueness constrained is enforced by the `balance_history_identify_entry` UNIQUE index. - Optimal DB fetching is ensured by the `balance_history_filter_entries` index Balance (balance.go) - Provides two stages: - Fetch of balance history using RPC calls (Balance.update function) - Retrieving of cached balance data from the DB it exists (Balance.get function) - Fetching and retrieving of data is done for specific time intervals defined by TimeInterval "enumeration" - Update process is done for a token identity by the Balance.Update function - The granularity of data points returned is defined by the constant increment step define in `timeIntervalToStride` for each time interval. - The `blocksStride` values have a common divisor to have cache hit between time intervals. Service (service.go) - Main APIs - StartBalanceHistory: Regularly updates balance history for all enabled networks, available accounts and provided tokens. - GetBalanceHistory: retrieves cached token count for a token identity (chain, address, currency) for multiple chains - UpdateVisibleTokens: will set the list of tokens to have historical balance fetched. This is a simplification to limit tokens to a small list that make sense Fetch balance history for ECR20 tokens - Add token.Manager.GetTokenBalanceAt to fetch balance of a specific block number of ECR20. - Add tokenChainClientSource concrete implementation of DataSource to fetch balance of ECR20 tokens. - Chose the correct DataSource implementation based on the token "is native" property. Tests Tests are implemented using a mock of `DataSource` interface used to intercept the RPC calls. Notes: - the timestamp used for retrieving block balance is constant Closes status-desktop: #8175, #8226, #8862
2022-11-15 12:14:41 +00:00
require.NoError(t, err)
require.Equal(t, 4, len(res))
require.Equal(t, []int64{105, 115, 125, 135}, getBlockNumbers(res))
require.Equal(t, []int64{3, 3 * 2, 3 * 3, 3 * 4}, getValues(res))
fix: merge balance history using block time This change improves on the previous implementation which used the block number that doesn't work with incompatible blockchains e.g. L1 vs. L2 Closes: #9205
2023-01-20 16:06:50 +00:00
require.Equal(t, []int64{115, 125, 135, 145}, getTimestamps(res))
feat: retrieve balance history for tokens and cache it to DB Extends wallet module with the history package with the following components: BalanceDB (balance_db.go) - Keeps track of balance information (token count, block, block timestamp) for a token identity (chain, address, currency) - The cached data is stored in `balance_history` table. - Uniqueness constrained is enforced by the `balance_history_identify_entry` UNIQUE index. - Optimal DB fetching is ensured by the `balance_history_filter_entries` index Balance (balance.go) - Provides two stages: - Fetch of balance history using RPC calls (Balance.update function) - Retrieving of cached balance data from the DB it exists (Balance.get function) - Fetching and retrieving of data is done for specific time intervals defined by TimeInterval "enumeration" - Update process is done for a token identity by the Balance.Update function - The granularity of data points returned is defined by the constant increment step define in `timeIntervalToStride` for each time interval. - The `blocksStride` values have a common divisor to have cache hit between time intervals. Service (service.go) - Main APIs - StartBalanceHistory: Regularly updates balance history for all enabled networks, available accounts and provided tokens. - GetBalanceHistory: retrieves cached token count for a token identity (chain, address, currency) for multiple chains - UpdateVisibleTokens: will set the list of tokens to have historical balance fetched. This is a simplification to limit tokens to a small list that make sense Fetch balance history for ECR20 tokens - Add token.Manager.GetTokenBalanceAt to fetch balance of a specific block number of ECR20. - Add tokenChainClientSource concrete implementation of DataSource to fetch balance of ECR20 tokens. - Chose the correct DataSource implementation based on the token "is native" property. Tests Tests are implemented using a mock of `DataSource` interface used to intercept the RPC calls. Notes: - the timestamp used for retrieving block balance is constant Closes status-desktop: #8175, #8226, #8862
2022-11-15 12:14:41 +00:00
}
fix: merge balance history using block time This change improves on the previous implementation which used the block number that doesn't work with incompatible blockchains e.g. L1 vs. L2 Closes: #9205
2023-01-20 16:06:50 +00:00
func TestServiceMergeDataPointsOneChain(t *testing.T) {
strideDuration := 10 * time.Second
feat: retrieve balance history for tokens and cache it to DB Extends wallet module with the history package with the following components: BalanceDB (balance_db.go) - Keeps track of balance information (token count, block, block timestamp) for a token identity (chain, address, currency) - The cached data is stored in `balance_history` table. - Uniqueness constrained is enforced by the `balance_history_identify_entry` UNIQUE index. - Optimal DB fetching is ensured by the `balance_history_filter_entries` index Balance (balance.go) - Provides two stages: - Fetch of balance history using RPC calls (Balance.update function) - Retrieving of cached balance data from the DB it exists (Balance.get function) - Fetching and retrieving of data is done for specific time intervals defined by TimeInterval "enumeration" - Update process is done for a token identity by the Balance.Update function - The granularity of data points returned is defined by the constant increment step define in `timeIntervalToStride` for each time interval. - The `blocksStride` values have a common divisor to have cache hit between time intervals. Service (service.go) - Main APIs - StartBalanceHistory: Regularly updates balance history for all enabled networks, available accounts and provided tokens. - GetBalanceHistory: retrieves cached token count for a token identity (chain, address, currency) for multiple chains - UpdateVisibleTokens: will set the list of tokens to have historical balance fetched. This is a simplification to limit tokens to a small list that make sense Fetch balance history for ECR20 tokens - Add token.Manager.GetTokenBalanceAt to fetch balance of a specific block number of ECR20. - Add tokenChainClientSource concrete implementation of DataSource to fetch balance of ECR20 tokens. - Chose the correct DataSource implementation based on the token "is native" property. Tests Tests are implemented using a mock of `DataSource` interface used to intercept the RPC calls. Notes: - the timestamp used for retrieving block balance is constant Closes status-desktop: #8175, #8226, #8862
2022-11-15 12:14:41 +00:00
testData := prepareTestData([]TestDataPoint{
// Keep 105
{value: 1, timestamp: 105, blockNumber: 105, chainID: 1},
// Keep 115
{value: 2, timestamp: 115, blockNumber: 115, chainID: 1},
// Keep 125
{value: 3, timestamp: 125, blockNumber: 125, chainID: 1},
})
fix: merge balance history using block time This change improves on the previous implementation which used the block number that doesn't work with incompatible blockchains e.g. L1 vs. L2 Closes: #9205
2023-01-20 16:06:50 +00:00
res, err := mergeDataPoints(testData, strideDuration)
feat: retrieve balance history for tokens and cache it to DB Extends wallet module with the history package with the following components: BalanceDB (balance_db.go) - Keeps track of balance information (token count, block, block timestamp) for a token identity (chain, address, currency) - The cached data is stored in `balance_history` table. - Uniqueness constrained is enforced by the `balance_history_identify_entry` UNIQUE index. - Optimal DB fetching is ensured by the `balance_history_filter_entries` index Balance (balance.go) - Provides two stages: - Fetch of balance history using RPC calls (Balance.update function) - Retrieving of cached balance data from the DB it exists (Balance.get function) - Fetching and retrieving of data is done for specific time intervals defined by TimeInterval "enumeration" - Update process is done for a token identity by the Balance.Update function - The granularity of data points returned is defined by the constant increment step define in `timeIntervalToStride` for each time interval. - The `blocksStride` values have a common divisor to have cache hit between time intervals. Service (service.go) - Main APIs - StartBalanceHistory: Regularly updates balance history for all enabled networks, available accounts and provided tokens. - GetBalanceHistory: retrieves cached token count for a token identity (chain, address, currency) for multiple chains - UpdateVisibleTokens: will set the list of tokens to have historical balance fetched. This is a simplification to limit tokens to a small list that make sense Fetch balance history for ECR20 tokens - Add token.Manager.GetTokenBalanceAt to fetch balance of a specific block number of ECR20. - Add tokenChainClientSource concrete implementation of DataSource to fetch balance of ECR20 tokens. - Chose the correct DataSource implementation based on the token "is native" property. Tests Tests are implemented using a mock of `DataSource` interface used to intercept the RPC calls. Notes: - the timestamp used for retrieving block balance is constant Closes status-desktop: #8175, #8226, #8862
2022-11-15 12:14:41 +00:00
require.NoError(t, err)
require.Equal(t, 3, len(res))
require.Equal(t, []int64{105, 115, 125}, getBlockNumbers(res))
require.Equal(t, []int64{1, 2, 3}, getValues(res))
feat: convert history balance tokens to fiat Add history.exchange sub-package with following responsibilities - fetch and caches daily exchange rates for tokens - Partial update if missing form yesterday back - Implement all time fetching special case - Fetches the price of the token after merging entries for the selected chains history.service package changes - Return ValuePoint instead of DataPoint - Value point contains the value in fiat as float64 instead - Convert merged values from tokens balance (wei) to fiat Other changes - Move start/stop balance history to startWallet/stopWallet - Move cryptocompare to thirdparty package to avoid recursive import - Rename DataPoint.Value to DataPoint.Balance - Don't merge entries for single chain
2023-01-25 18:28:51 +00:00
require.Equal(t, []int64{105, 115, 125}, getTimestamps(res), "Expect no merging for one chain")
feat: retrieve balance history for tokens and cache it to DB Extends wallet module with the history package with the following components: BalanceDB (balance_db.go) - Keeps track of balance information (token count, block, block timestamp) for a token identity (chain, address, currency) - The cached data is stored in `balance_history` table. - Uniqueness constrained is enforced by the `balance_history_identify_entry` UNIQUE index. - Optimal DB fetching is ensured by the `balance_history_filter_entries` index Balance (balance.go) - Provides two stages: - Fetch of balance history using RPC calls (Balance.update function) - Retrieving of cached balance data from the DB it exists (Balance.get function) - Fetching and retrieving of data is done for specific time intervals defined by TimeInterval "enumeration" - Update process is done for a token identity by the Balance.Update function - The granularity of data points returned is defined by the constant increment step define in `timeIntervalToStride` for each time interval. - The `blocksStride` values have a common divisor to have cache hit between time intervals. Service (service.go) - Main APIs - StartBalanceHistory: Regularly updates balance history for all enabled networks, available accounts and provided tokens. - GetBalanceHistory: retrieves cached token count for a token identity (chain, address, currency) for multiple chains - UpdateVisibleTokens: will set the list of tokens to have historical balance fetched. This is a simplification to limit tokens to a small list that make sense Fetch balance history for ECR20 tokens - Add token.Manager.GetTokenBalanceAt to fetch balance of a specific block number of ECR20. - Add tokenChainClientSource concrete implementation of DataSource to fetch balance of ECR20 tokens. - Chose the correct DataSource implementation based on the token "is native" property. Tests Tests are implemented using a mock of `DataSource` interface used to intercept the RPC calls. Notes: - the timestamp used for retrieving block balance is constant Closes status-desktop: #8175, #8226, #8862
2022-11-15 12:14:41 +00:00
}
fix: merge balance history using block time This change improves on the previous implementation which used the block number that doesn't work with incompatible blockchains e.g. L1 vs. L2 Closes: #9205
2023-01-20 16:06:50 +00:00
func TestServiceMergeDataPointsDropAll(t *testing.T) {
strideDuration := 10 * time.Second
feat: retrieve balance history for tokens and cache it to DB Extends wallet module with the history package with the following components: BalanceDB (balance_db.go) - Keeps track of balance information (token count, block, block timestamp) for a token identity (chain, address, currency) - The cached data is stored in `balance_history` table. - Uniqueness constrained is enforced by the `balance_history_identify_entry` UNIQUE index. - Optimal DB fetching is ensured by the `balance_history_filter_entries` index Balance (balance.go) - Provides two stages: - Fetch of balance history using RPC calls (Balance.update function) - Retrieving of cached balance data from the DB it exists (Balance.get function) - Fetching and retrieving of data is done for specific time intervals defined by TimeInterval "enumeration" - Update process is done for a token identity by the Balance.Update function - The granularity of data points returned is defined by the constant increment step define in `timeIntervalToStride` for each time interval. - The `blocksStride` values have a common divisor to have cache hit between time intervals. Service (service.go) - Main APIs - StartBalanceHistory: Regularly updates balance history for all enabled networks, available accounts and provided tokens. - GetBalanceHistory: retrieves cached token count for a token identity (chain, address, currency) for multiple chains - UpdateVisibleTokens: will set the list of tokens to have historical balance fetched. This is a simplification to limit tokens to a small list that make sense Fetch balance history for ECR20 tokens - Add token.Manager.GetTokenBalanceAt to fetch balance of a specific block number of ECR20. - Add tokenChainClientSource concrete implementation of DataSource to fetch balance of ECR20 tokens. - Chose the correct DataSource implementation based on the token "is native" property. Tests Tests are implemented using a mock of `DataSource` interface used to intercept the RPC calls. Notes: - the timestamp used for retrieving block balance is constant Closes status-desktop: #8175, #8226, #8862
2022-11-15 12:14:41 +00:00
testData := prepareTestData([]TestDataPoint{
{value: 1, timestamp: 100, blockNumber: 100, chainID: 1},
fix: merge balance history using block time This change improves on the previous implementation which used the block number that doesn't work with incompatible blockchains e.g. L1 vs. L2 Closes: #9205
2023-01-20 16:06:50 +00:00
{value: 1, timestamp: 110, blockNumber: 110, chainID: 2},
{value: 1, timestamp: 120, blockNumber: 120, chainID: 3},
{value: 1, timestamp: 130, blockNumber: 130, chainID: 4},
feat: retrieve balance history for tokens and cache it to DB Extends wallet module with the history package with the following components: BalanceDB (balance_db.go) - Keeps track of balance information (token count, block, block timestamp) for a token identity (chain, address, currency) - The cached data is stored in `balance_history` table. - Uniqueness constrained is enforced by the `balance_history_identify_entry` UNIQUE index. - Optimal DB fetching is ensured by the `balance_history_filter_entries` index Balance (balance.go) - Provides two stages: - Fetch of balance history using RPC calls (Balance.update function) - Retrieving of cached balance data from the DB it exists (Balance.get function) - Fetching and retrieving of data is done for specific time intervals defined by TimeInterval "enumeration" - Update process is done for a token identity by the Balance.Update function - The granularity of data points returned is defined by the constant increment step define in `timeIntervalToStride` for each time interval. - The `blocksStride` values have a common divisor to have cache hit between time intervals. Service (service.go) - Main APIs - StartBalanceHistory: Regularly updates balance history for all enabled networks, available accounts and provided tokens. - GetBalanceHistory: retrieves cached token count for a token identity (chain, address, currency) for multiple chains - UpdateVisibleTokens: will set the list of tokens to have historical balance fetched. This is a simplification to limit tokens to a small list that make sense Fetch balance history for ECR20 tokens - Add token.Manager.GetTokenBalanceAt to fetch balance of a specific block number of ECR20. - Add tokenChainClientSource concrete implementation of DataSource to fetch balance of ECR20 tokens. - Chose the correct DataSource implementation based on the token "is native" property. Tests Tests are implemented using a mock of `DataSource` interface used to intercept the RPC calls. Notes: - the timestamp used for retrieving block balance is constant Closes status-desktop: #8175, #8226, #8862
2022-11-15 12:14:41 +00:00
})
fix: merge balance history using block time This change improves on the previous implementation which used the block number that doesn't work with incompatible blockchains e.g. L1 vs. L2 Closes: #9205
2023-01-20 16:06:50 +00:00
res, err := mergeDataPoints(testData, strideDuration)
feat: retrieve balance history for tokens and cache it to DB Extends wallet module with the history package with the following components: BalanceDB (balance_db.go) - Keeps track of balance information (token count, block, block timestamp) for a token identity (chain, address, currency) - The cached data is stored in `balance_history` table. - Uniqueness constrained is enforced by the `balance_history_identify_entry` UNIQUE index. - Optimal DB fetching is ensured by the `balance_history_filter_entries` index Balance (balance.go) - Provides two stages: - Fetch of balance history using RPC calls (Balance.update function) - Retrieving of cached balance data from the DB it exists (Balance.get function) - Fetching and retrieving of data is done for specific time intervals defined by TimeInterval "enumeration" - Update process is done for a token identity by the Balance.Update function - The granularity of data points returned is defined by the constant increment step define in `timeIntervalToStride` for each time interval. - The `blocksStride` values have a common divisor to have cache hit between time intervals. Service (service.go) - Main APIs - StartBalanceHistory: Regularly updates balance history for all enabled networks, available accounts and provided tokens. - GetBalanceHistory: retrieves cached token count for a token identity (chain, address, currency) for multiple chains - UpdateVisibleTokens: will set the list of tokens to have historical balance fetched. This is a simplification to limit tokens to a small list that make sense Fetch balance history for ECR20 tokens - Add token.Manager.GetTokenBalanceAt to fetch balance of a specific block number of ECR20. - Add tokenChainClientSource concrete implementation of DataSource to fetch balance of ECR20 tokens. - Chose the correct DataSource implementation based on the token "is native" property. Tests Tests are implemented using a mock of `DataSource` interface used to intercept the RPC calls. Notes: - the timestamp used for retrieving block balance is constant Closes status-desktop: #8175, #8226, #8862
2022-11-15 12:14:41 +00:00
require.NoError(t, err)
require.Equal(t, 0, len(res))
}
fix: merge balance history using block time This change improves on the previous implementation which used the block number that doesn't work with incompatible blockchains e.g. L1 vs. L2 Closes: #9205
2023-01-20 16:06:50 +00:00
func TestServiceMergeDataPointsEmptyDB(t *testing.T) {
feat: retrieve balance history for tokens and cache it to DB Extends wallet module with the history package with the following components: BalanceDB (balance_db.go) - Keeps track of balance information (token count, block, block timestamp) for a token identity (chain, address, currency) - The cached data is stored in `balance_history` table. - Uniqueness constrained is enforced by the `balance_history_identify_entry` UNIQUE index. - Optimal DB fetching is ensured by the `balance_history_filter_entries` index Balance (balance.go) - Provides two stages: - Fetch of balance history using RPC calls (Balance.update function) - Retrieving of cached balance data from the DB it exists (Balance.get function) - Fetching and retrieving of data is done for specific time intervals defined by TimeInterval "enumeration" - Update process is done for a token identity by the Balance.Update function - The granularity of data points returned is defined by the constant increment step define in `timeIntervalToStride` for each time interval. - The `blocksStride` values have a common divisor to have cache hit between time intervals. Service (service.go) - Main APIs - StartBalanceHistory: Regularly updates balance history for all enabled networks, available accounts and provided tokens. - GetBalanceHistory: retrieves cached token count for a token identity (chain, address, currency) for multiple chains - UpdateVisibleTokens: will set the list of tokens to have historical balance fetched. This is a simplification to limit tokens to a small list that make sense Fetch balance history for ECR20 tokens - Add token.Manager.GetTokenBalanceAt to fetch balance of a specific block number of ECR20. - Add tokenChainClientSource concrete implementation of DataSource to fetch balance of ECR20 tokens. - Chose the correct DataSource implementation based on the token "is native" property. Tests Tests are implemented using a mock of `DataSource` interface used to intercept the RPC calls. Notes: - the timestamp used for retrieving block balance is constant Closes status-desktop: #8175, #8226, #8862
2022-11-15 12:14:41 +00:00
testData := prepareTestData([]TestDataPoint{})
fix: merge balance history using block time This change improves on the previous implementation which used the block number that doesn't work with incompatible blockchains e.g. L1 vs. L2 Closes: #9205
2023-01-20 16:06:50 +00:00
strideDuration := 10 * time.Second
res, err := mergeDataPoints(testData, strideDuration)
feat: retrieve balance history for tokens and cache it to DB Extends wallet module with the history package with the following components: BalanceDB (balance_db.go) - Keeps track of balance information (token count, block, block timestamp) for a token identity (chain, address, currency) - The cached data is stored in `balance_history` table. - Uniqueness constrained is enforced by the `balance_history_identify_entry` UNIQUE index. - Optimal DB fetching is ensured by the `balance_history_filter_entries` index Balance (balance.go) - Provides two stages: - Fetch of balance history using RPC calls (Balance.update function) - Retrieving of cached balance data from the DB it exists (Balance.get function) - Fetching and retrieving of data is done for specific time intervals defined by TimeInterval "enumeration" - Update process is done for a token identity by the Balance.Update function - The granularity of data points returned is defined by the constant increment step define in `timeIntervalToStride` for each time interval. - The `blocksStride` values have a common divisor to have cache hit between time intervals. Service (service.go) - Main APIs - StartBalanceHistory: Regularly updates balance history for all enabled networks, available accounts and provided tokens. - GetBalanceHistory: retrieves cached token count for a token identity (chain, address, currency) for multiple chains - UpdateVisibleTokens: will set the list of tokens to have historical balance fetched. This is a simplification to limit tokens to a small list that make sense Fetch balance history for ECR20 tokens - Add token.Manager.GetTokenBalanceAt to fetch balance of a specific block number of ECR20. - Add tokenChainClientSource concrete implementation of DataSource to fetch balance of ECR20 tokens. - Chose the correct DataSource implementation based on the token "is native" property. Tests Tests are implemented using a mock of `DataSource` interface used to intercept the RPC calls. Notes: - the timestamp used for retrieving block balance is constant Closes status-desktop: #8175, #8226, #8862
2022-11-15 12:14:41 +00:00
require.NoError(t, err)
require.Equal(t, 0, len(res))
}
fix: merge balance history using block time This change improves on the previous implementation which used the block number that doesn't work with incompatible blockchains e.g. L1 vs. L2 Closes: #9205
2023-01-20 16:06:50 +00:00
func TestServiceFindFirstStrideWindowFirstForAllChainInOneStride(t *testing.T) {
strideDuration := 10 * time.Second
testData := prepareTestData([]TestDataPoint{
{value: 1, timestamp: 103, blockNumber: 101, chainID: 2},
{value: 1, timestamp: 106, blockNumber: 102, chainID: 3},
{value: 1, timestamp: 100, blockNumber: 100, chainID: 1},
{value: 1, timestamp: 110, blockNumber: 103, chainID: 1},
{value: 1, timestamp: 110, blockNumber: 103, chainID: 2},
})
feat: convert history balance tokens to fiat Add history.exchange sub-package with following responsibilities - fetch and caches daily exchange rates for tokens - Partial update if missing form yesterday back - Implement all time fetching special case - Fetches the price of the token after merging entries for the selected chains history.service package changes - Return ValuePoint instead of DataPoint - Value point contains the value in fiat as float64 instead - Convert merged values from tokens balance (wei) to fiat Other changes - Move start/stop balance history to startWallet/stopWallet - Move cryptocompare to thirdparty package to avoid recursive import - Rename DataPoint.Value to DataPoint.Balance - Don't merge entries for single chain
2023-01-25 18:28:51 +00:00
startTimestamp, pos := findFirstStrideWindow(testData, strideDuration)
fix: merge balance history using block time This change improves on the previous implementation which used the block number that doesn't work with incompatible blockchains e.g. L1 vs. L2 Closes: #9205
2023-01-20 16:06:50 +00:00
require.Equal(t, testData[1][0].Timestamp, uint64(startTimestamp))
feat: convert history balance tokens to fiat Add history.exchange sub-package with following responsibilities - fetch and caches daily exchange rates for tokens - Partial update if missing form yesterday back - Implement all time fetching special case - Fetches the price of the token after merging entries for the selected chains history.service package changes - Return ValuePoint instead of DataPoint - Value point contains the value in fiat as float64 instead - Convert merged values from tokens balance (wei) to fiat Other changes - Move start/stop balance history to startWallet/stopWallet - Move cryptocompare to thirdparty package to avoid recursive import - Rename DataPoint.Value to DataPoint.Balance - Don't merge entries for single chain
2023-01-25 18:28:51 +00:00
require.Equal(t, map[chainIdentity]int{1: 0, 2: 0, 3: 0}, pos)
fix: merge balance history using block time This change improves on the previous implementation which used the block number that doesn't work with incompatible blockchains e.g. L1 vs. L2 Closes: #9205
2023-01-20 16:06:50 +00:00
}
func TestServiceSortTimeAsc(t *testing.T) {
testData := prepareTestData([]TestDataPoint{
{value: 3, timestamp: 103, blockNumber: 103, chainID: 3},
{value: 4, timestamp: 104, blockNumber: 104, chainID: 4},
{value: 2, timestamp: 102, blockNumber: 102, chainID: 2},
{value: 1, timestamp: 101, blockNumber: 101, chainID: 1},
})
sorted := sortTimeAsc(testData, map[chainIdentity]int{4: 0, 3: 0, 2: 0, 1: 0})
require.Equal(t, []timeIdentity{{1, 0}, {2, 0}, {3, 0}, {4, 0}}, sorted)
}
func TestServiceAtEnd(t *testing.T) {
testData := prepareTestData([]TestDataPoint{
{value: 1, timestamp: 101, blockNumber: 101, chainID: 1},
fix: fix wallet.history.TestServiceAtEnd First check was relying on sorting result of two items that had the same weight and is not guaranteed to be deterministic.
2023-01-30 15:40:44 +00:00
{value: 1, timestamp: 103, blockNumber: 103, chainID: 2},
{value: 1, timestamp: 105, blockNumber: 105, chainID: 1},
fix: merge balance history using block time This change improves on the previous implementation which used the block number that doesn't work with incompatible blockchains e.g. L1 vs. L2 Closes: #9205
2023-01-20 16:06:50 +00:00
})
sorted := sortTimeAsc(testData, map[chainIdentity]int{1: 0, 2: 0})
require.False(t, sorted[0].atEnd(testData))
require.True(t, sorted[1].atEnd(testData))
sorted = sortTimeAsc(testData, map[chainIdentity]int{1: 1, 2: 0})
require.True(t, sorted[1].atEnd(testData))
}
feat: convert history balance tokens to fiat Add history.exchange sub-package with following responsibilities - fetch and caches daily exchange rates for tokens - Partial update if missing form yesterday back - Implement all time fetching special case - Fetches the price of the token after merging entries for the selected chains history.service package changes - Return ValuePoint instead of DataPoint - Value point contains the value in fiat as float64 instead - Convert merged values from tokens balance (wei) to fiat Other changes - Move start/stop balance history to startWallet/stopWallet - Move cryptocompare to thirdparty package to avoid recursive import - Rename DataPoint.Value to DataPoint.Balance - Don't merge entries for single chain
2023-01-25 18:28:51 +00:00
func TestServiceTokenToValue(t *testing.T) {
weisInOneMain := big.NewFloat(math.Pow(10, 18.0))
res := tokenToValue(big.NewInt(12345), 1000, weisInOneMain)
require.Equal(t, 0.000000000012345, res)
in, ok := new(big.Int).SetString("1234567890000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000", 10)
require.True(t, ok)
res = tokenToValue(in, 10000, weisInOneMain)
require.Equal(t, 1.23456789e+112, res)
res = tokenToValue(big.NewInt(1000000000000000000), 1.0, weisInOneMain)
require.Equal(t, 1.0, res)
res = tokenToValue(big.NewInt(1), 1.23456789, weisInOneMain)
require.InEpsilonf(t, 1.23456789e-18, res, 1.0e-8, "Expects error for handling such low values")
res = tokenToValue(new(big.Int).Exp(big.NewInt(10), big.NewInt(254), nil), 100000, weisInOneMain)
require.Equal(t, 1e+241, res, "Expect exponent 254-18+5")
}
func TestServiceGetBalanceHistoryNoData(t *testing.T) {
service, closeFn := setupDummyServiceNoDependencies(t)
defer closeFn()
res, err := service.GetBalanceHistory(context.Background(), []uint64{777}, common.HexToAddress(`0x1`), "ETH", "EUR", time.Now().Unix(), BalanceHistory1Year)
require.NoError(t, err)
require.Equal(t, 0, len(res))
}