status-im
/
status-go
mirror of https://github.com/status-im/status-go.git
2
0
Fork 0
Code Issues Projects Releases Wiki Activity

feat(wallet) chain id multi-transaction filtering 

Updates status-desktop #11631
This commit is contained in:
Stefan 2023-07-25 17:03:33 +01:00 committed by Stefan Dunca
parent e5ce2c7c03
commit d956a3e854
3 changed files with 343 additions and 395 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

161
services/wallet/activity/TODO.md
View File

@ -1,161 +0,0 @@
# DB extension and refactoring for activity
## Work in progress
| | Buy | Swap | Bridge | Send/Receive |
| ------------- | -------------- | -------------- | ------------- | ------------- |
| Activity data | ~~API~~ ~~DB~~ | ~~API~~ DB | _API_ DB | _API_ DB |
| Raw data | ~~API~~ ~~DB~~ | ~~API~~ DB | _API_ DB | _API_ DB |
| Pending data | ~~API~~ ~~DB~~ | ~~API~~ DB | _API_ DB | _API_ DB |
Legend:
- ~~API~~ - not much or at all provided
- _API_ - partially provided
- API - complete
### Summary
Improve on the identified limitations
- [x] Missing filtering data
- [x] Missing cached (not extracted as a column)
- Extracting the data from the raw data is expensive but might be negligible given that usually we should not expect more than 20 entries per second in the worst case scenario.
- [x] Table extensions
- ~~Activity specific info in activity data store (multi_transaction table)~~
- Activity specific info in in the transactions data store (transfers table)
### Missing data
Filter requirements
- [x] Activity operation status
- [x] `pending`: have to aggregate for `Buy`, `Swap`, `Bridge`
- already there for `Send`, `Receive`
- [x] `complete`: only extract and check for `status` in the `receipt` for `Send`, `Receive`
- For complex operations aggregate the `complete` status `Buy`, `Swap`, `Bridge`
- [x] `finalized`: similar to `complete` for `Send`, `Receive`
- all sub-transactions are `complete` for `Buy`, `Swap`, `Bridge`
- [x] `failed`: extract from `status` for all sub-transactions
- [x] `chainID`: aggregate data for activity entries `Bridge`, `Buy`, `Swap`
- [x] `tokenCode` for activity entries `Send`, `Receive`
- For `Bridge` its already there and `Buy`, `Swap` is coming soon
- [ ] `collectibles`: require adding collectible attributes to activity data (`token_address` and `tokenId`)
UX requirements
- [x] `status`: for status icon and label
- [ ] `chainIDs`: for chain icons
- Missing for `Bridge`, `Buy`, `Swap`
- [x] `amount`s: add to the activity.Entry
- already in DB
- [x] `tokenCode`s: add to the activity.Entry
- already in DB
- [ ] `to`/`from`/`owner`: add to the activity.Entry
- already in DB, coming soon
- [ ] `tokenIdentity`: collectible is missing (`chainId`, `address`, `tokenId`)
- `tokenCode` should be covering fungible operations
- [x] `identity`: for all the sources
- [x] `type`: for the main icon and label
- [x] `time`: timestamp
### Refactoring
Extend `entry.nim:ActivityEntry` and `activity.go:Entry` with presentation layer data
- [x] `activityType`: instead of the current `MultiTransactionType`
- [x] `status`: for status icon and label
## Current state
### Transfers Table
The `transfers` transactions raw data
- Transaction identity: `network_id`, `hash`, `address`
- Implementation by `sqlite_autoindex_transfers_1` unique index
- `multi_transaction_id`: `multi_transaction` entries to `transfers` entries mapping (one to many)
- Raw data:
- `tx` transaction
- `receipt`: transfer receipt
### Multi-Transaction Table
Represented by `multi_transaction`
Responsibilities
- UX metadata for transactions originating from our APP.
- `from_address`, `to_address`
- `from_asset`, `to_asset`
- `from_amount`, `to_amount` (token codes)
- `type` identifies the type (initially only Send and Bridge)
- `timestamp` the timestamp of the execution
- Multi-transaction to sub-transaction mapping
- The `multi_transaction_id` in the `transfers` and `pending_transaction` table corresponds to the `ROWID` in the `multi_transactions`.
### Pending Transactions Table
The `pending_transactions` table represents transactions initiated from the app
- Transaction identity
- `network_id`, `hash`
- implemented by the `sqlite_autoindex_pending_transactions_1` index
- Note how this is different from the `transfers` table, where the `address` is also part of the identity.
- `timestamp`: The timestamp of the pending transaction.
- `multi_transaction_id`: `multi_transaction` entries to `pending_transactions` entries mapping (one to many)
### Schema
Relationships between the tables
```mermaid
erDiagram
multi_transaction ||--o{ transfers : has
multi_transaction ||--o{ pending_transactions : has
transfers {
network_id BIGINT
hash VARCHAR
timestamp BIGINT
multi_transaction_id INT
tx BLOB
receipt BLOB
log BLOB
}
multi_transaction {
ROWID INT
type VARCHAR
}
pending_transactions {
network_id BIGINT
hash VARCHAR
timestamp INT
multi_transaction_id INT
}
```
### Dropped tasks
Dropped the DB refactoring and improvements after further discussion and concerns
- [x] Terminology proposal
- [x] using `transactions` instead of `transfers` for the raw data
- [x] using `activity` instead of `multi-transaction` to better match the new requirements
- [x] Convert JSON blobs into structured data
Dropped benchmark performance and move on using theoretical knowledge by adding indexes for what we know only
Will leave the performance concerns for the next milestone
- [ ] Joining DBs
- [ ] One activity DB for all require metadata
- Pros:
- Faster to query (don't know the numbers)
- Simpler query will decrease maintenance
- Cons:
- have to migrate all data, extract and fill the activity on every download of updates for all activities
- [ ] Keep only filter specific metadata in the Activity DB
- Pros:
- Less changes to migrate existing data. Still have to maintain activity filtering specific data
- Cons:
- Slower to query (don't know how much yet)
- Complex query increases maintenance

475
services/wallet/activity/activity.go
View File

@ -246,7 +246,6 @@ const (
fromTrType = byte(1) fromTrType = byte(1)
toTrType = byte(2) toTrType = byte(2)
// TODO: Multi-transaction network information is missing in filtering
// TODO optimization: consider implementing nullable []byte instead of using strings for addresses // TODO optimization: consider implementing nullable []byte instead of using strings for addresses
// or insert binary (X'...' syntax) directly into the query // or insert binary (X'...' syntax) directly into the query
// //
@ -270,268 +269,302 @@ const (
// 2. Filtering by token identity (chain and address for transfers table) where the symbol is ignored and all the // 2. Filtering by token identity (chain and address for transfers table) where the symbol is ignored and all the
// token identities must be provided // token identities must be provided
queryFormatString = ` queryFormatString = `
WITH filter_conditions AS ( WITH filter_conditions AS (
SELECT SELECT
? AS startFilterDisabled, ? AS startFilterDisabled,
? AS startTimestamp, ? AS startTimestamp,
? AS endFilterDisabled, ? AS endFilterDisabled,
? AS endTimestamp, ? AS endTimestamp,
? AS filterActivityTypeAll, ? AS filterActivityTypeAll,
? AS filterActivityTypeSend, ? AS filterActivityTypeSend,
? AS filterActivityTypeReceive, ? AS filterActivityTypeReceive,
? AS fromTrType, ? AS fromTrType,
? AS toTrType, ? AS toTrType,
? AS filterAllAddresses, ? AS filterAllAddresses,
? AS filterAllToAddresses, ? AS filterAllToAddresses,
? AS filterAllActivityStatus, ? AS filterAllActivityStatus,
? AS filterStatusCompleted, ? AS filterStatusCompleted,
? AS filterStatusFailed, ? AS filterStatusFailed,
? AS filterStatusFinalized, ? AS filterStatusFinalized,
? AS filterStatusPending, ? AS filterStatusPending,
? AS statusFailed, ? AS statusFailed,
? AS statusSuccess, ? AS statusSuccess,
? AS statusPending, ? AS statusPending,
? AS includeAllTokenTypeAssets, ? AS includeAllTokenTypeAssets,
? AS includeAllNetworks ? AS includeAllNetworks
), ),
filter_addresses(address) AS ( filter_addresses(address) AS (
SELECT HEX(address) FROM keypairs_accounts WHERE (SELECT filterAllAddresses FROM filter_conditions) != 0 SELECT HEX(address) FROM keypairs_accounts WHERE (SELECT filterAllAddresses FROM filter_conditions) != 0
UNION ALL UNION ALL
SELECT * FROM (VALUES %s) WHERE (SELECT filterAllAddresses FROM filter_conditions) = 0 SELECT * FROM (VALUES %s) WHERE (SELECT filterAllAddresses FROM filter_conditions) = 0
), ),
filter_to_addresses(address) AS ( filter_to_addresses(address) AS (
VALUES %s VALUES %s
), ),
assets_token_codes(token_code) AS ( assets_token_codes(token_code) AS (
VALUES %s VALUES %s
), ),
assets_erc20(chain_id, token_address) AS ( assets_erc20(chain_id, token_address) AS (
VALUES %s VALUES %s
), ),
filter_networks(network_id) AS ( filter_networks(network_id) AS (
VALUES %s VALUES %s
), ),
tr_status AS ( tr_status AS (
SELECT SELECT
multi_transaction_id, multi_transaction_id,
MIN(status) AS min_status, MIN(status) AS min_status,
COUNT(*) AS count COUNT(*) AS count,
FROM network_id
transfers FROM
WHERE transfers.loaded == 1 transfers
AND transfers.multi_transaction_id != 0 WHERE transfers.loaded == 1
AND transfers.multi_transaction_id != 0
GROUP BY transfers.multi_transaction_id GROUP BY transfers.multi_transaction_id
), ),
pending_status AS ( tr_network_ids AS (
SELECT SELECT
multi_transaction_id, multi_transaction_id
COUNT(*) AS count FROM
FROM transfers
pending_transactions WHERE transfers.loaded == 1
WHERE pending_transactions.multi_transaction_id != 0 AND transfers.multi_transaction_id != 0
GROUP BY pending_transactions.multi_transaction_id AND network_id IN filter_networks
) GROUP BY transfers.multi_transaction_id
SELECT ),
transfers.hash AS transfer_hash, pending_status AS (
NULL AS pending_hash, SELECT
transfers.network_id AS network_id, multi_transaction_id,
0 AS multi_tx_id, COUNT(*) AS count,
transfers.timestamp AS timestamp, network_id
NULL AS mt_type, FROM
pending_transactions
WHERE pending_transactions.multi_transaction_id != 0
GROUP BY pending_transactions.multi_transaction_id
),
pending_network_ids AS (
SELECT
multi_transaction_id
FROM
pending_transactions
WHERE pending_transactions.multi_transaction_id != 0
AND pending_transactions.network_id IN filter_networks
GROUP BY pending_transactions.multi_transaction_id
)
SELECT
transfers.hash AS transfer_hash,
NULL AS pending_hash,
transfers.network_id AS network_id,
0 AS multi_tx_id,
transfers.timestamp AS timestamp,
NULL AS mt_type,
CASE CASE
WHEN from_join.address IS NOT NULL AND to_join.address IS NULL THEN fromTrType WHEN from_join.address IS NOT NULL AND to_join.address IS NULL THEN fromTrType
WHEN to_join.address IS NOT NULL AND from_join.address IS NULL THEN toTrType WHEN to_join.address IS NOT NULL AND from_join.address IS NULL THEN toTrType
WHEN from_join.address IS NOT NULL AND to_join.address IS NOT NULL THEN WHEN from_join.address IS NOT NULL AND to_join.address IS NOT NULL THEN
CASE CASE
WHEN from_join.address < to_join.address THEN fromTrType WHEN from_join.address < to_join.address THEN fromTrType
ELSE toTrType ELSE toTrType
END END
ELSE NULL ELSE NULL
END as tr_type, END as tr_type,
transfers.tx_from_address AS from_address, transfers.tx_from_address AS from_address,
transfers.tx_to_address AS to_address, transfers.tx_to_address AS to_address,
transfers.address AS owner_address, transfers.address AS owner_address,
transfers.amount_padded128hex AS tr_amount, transfers.amount_padded128hex AS tr_amount,
NULL AS mt_from_amount, NULL AS mt_from_amount,
NULL AS mt_to_amount, NULL AS mt_to_amount,
CASE CASE
WHEN transfers.status IS 1 THEN statusSuccess WHEN transfers.status IS 1 THEN statusSuccess
ELSE statusFailed ELSE statusFailed
END AS agg_status, END AS agg_status,
1 AS agg_count, 1 AS agg_count,
transfers.token_address AS token_address, transfers.token_address AS token_address,
NULL AS token_code, NULL AS token_code,
NULL AS from_token_code, NULL AS from_token_code,
NULL AS to_token_code, NULL AS to_token_code,
NULL AS out_network_id, NULL AS out_network_id,
NULL AS in_network_id, NULL AS in_network_id,
transfers.type AS type, transfers.type AS type,
transfers.contract_address AS contract_address transfers.contract_address AS contract_address
FROM transfers, filter_conditions FROM transfers, filter_conditions
LEFT JOIN LEFT JOIN
filter_addresses from_join ON HEX(transfers.tx_from_address) = from_join.address filter_addresses from_join ON HEX(transfers.tx_from_address) = from_join.address
LEFT JOIN LEFT JOIN
filter_addresses to_join ON HEX(transfers.tx_to_address) = to_join.address filter_addresses to_join ON HEX(transfers.tx_to_address) = to_join.address
WHERE WHERE
transfers.loaded == 1 transfers.loaded == 1
AND transfers.multi_transaction_id = 0 AND transfers.multi_transaction_id = 0
AND ((startFilterDisabled OR transfers.timestamp >= startTimestamp) AND ((startFilterDisabled OR transfers.timestamp >= startTimestamp)
AND (endFilterDisabled OR transfers.timestamp <= endTimestamp) AND (endFilterDisabled OR transfers.timestamp <= endTimestamp)
) )
AND (filterActivityTypeAll AND (filterActivityTypeAll
OR (filterActivityTypeSend OR (filterActivityTypeSend
AND (filterAllAddresses AND (filterAllAddresses
OR (HEX(transfers.tx_from_address) IN filter_addresses) OR (HEX(transfers.tx_from_address) IN filter_addresses)
) )
) )
OR (filterActivityTypeReceive OR (filterActivityTypeReceive
AND (filterAllAddresses AND (filterAllAddresses
OR (HEX(transfers.tx_to_address) IN filter_addresses)) OR (HEX(transfers.tx_to_address) IN filter_addresses))
) )
) )
AND (filterAllAddresses AND (filterAllAddresses
OR (HEX(transfers.tx_from_address) IN filter_addresses) OR (HEX(transfers.tx_from_address) IN filter_addresses)
OR (HEX(transfers.tx_to_address) IN filter_addresses) OR (HEX(transfers.tx_to_address) IN filter_addresses)
) )
AND (filterAllToAddresses AND (filterAllToAddresses
OR (HEX(transfers.tx_to_address) IN filter_to_addresses) OR (HEX(transfers.tx_to_address) IN filter_to_addresses)
) )
AND (includeAllTokenTypeAssets OR (transfers.type = "eth" AND ("ETH" IN assets_token_codes)) AND (includeAllTokenTypeAssets OR (transfers.type = "eth" AND ("ETH" IN assets_token_codes))
OR (transfers.type = "erc20" AND ((transfers.network_id, HEX(transfers.token_address)) IN assets_erc20))) OR (transfers.type = "erc20" AND ((transfers.network_id, HEX(transfers.token_address)) IN assets_erc20))
AND (includeAllNetworks OR (transfers.network_id IN filter_networks)) )
AND (filterAllActivityStatus OR ((filterStatusCompleted OR filterStatusFinalized) AND transfers.status = 1) AND (includeAllNetworks OR (transfers.network_id IN filter_networks))
OR (filterStatusFailed AND transfers.status = 0) AND (filterAllActivityStatus OR ((filterStatusCompleted OR filterStatusFinalized) AND transfers.status = 1)
) OR (filterStatusFailed AND transfers.status = 0)
)
UNION ALL UNION ALL
SELECT SELECT
NULL AS transfer_hash, NULL AS transfer_hash,
pending_transactions.hash AS pending_hash, pending_transactions.hash AS pending_hash,
pending_transactions.network_id AS network_id, pending_transactions.network_id AS network_id,
0 AS multi_tx_id, 0 AS multi_tx_id,
pending_transactions.timestamp AS timestamp, pending_transactions.timestamp AS timestamp,
NULL AS mt_type, NULL AS mt_type,
CASE CASE
WHEN from_join.address IS NOT NULL AND to_join.address IS NULL THEN fromTrType WHEN from_join.address IS NOT NULL AND to_join.address IS NULL THEN fromTrType
WHEN to_join.address IS NOT NULL AND from_join.address IS NULL THEN toTrType WHEN to_join.address IS NOT NULL AND from_join.address IS NULL THEN toTrType
WHEN from_join.address IS NOT NULL AND to_join.address IS NOT NULL THEN WHEN from_join.address IS NOT NULL AND to_join.address IS NOT NULL THEN
CASE CASE
WHEN from_join.address < to_join.address THEN fromTrType WHEN from_join.address < to_join.address THEN fromTrType
ELSE toTrType ELSE toTrType
END END
ELSE NULL ELSE NULL
END as tr_type, END as tr_type,
pending_transactions.from_address AS from_address, pending_transactions.from_address AS from_address,
pending_transactions.to_address AS to_address, pending_transactions.to_address AS to_address,
NULL AS owner_address, NULL AS owner_address,
pending_transactions.value AS tr_amount, pending_transactions.value AS tr_amount,
NULL AS mt_from_amount, NULL AS mt_from_amount,
NULL AS mt_to_amount, NULL AS mt_to_amount,
statusPending AS agg_status, statusPending AS agg_status,
1 AS agg_count, 1 AS agg_count,
NULL AS token_address, NULL AS token_address,
pending_transactions.symbol AS token_code, pending_transactions.symbol AS token_code,
NULL AS from_token_code, NULL AS from_token_code,
NULL AS to_token_code, NULL AS to_token_code,
NULL AS out_network_id, NULL AS out_network_id,
NULL AS in_network_id, NULL AS in_network_id,
pending_transactions.type AS type, pending_transactions.type AS type,
NULL as contract_address NULL as contract_address
FROM pending_transactions, filter_conditions FROM pending_transactions, filter_conditions
LEFT JOIN LEFT JOIN
filter_addresses from_join ON HEX(pending_transactions.from_address) = from_join.address filter_addresses from_join ON HEX(pending_transactions.from_address) = from_join.address
LEFT JOIN LEFT JOIN
filter_addresses to_join ON HEX(pending_transactions.to_address) = to_join.address filter_addresses to_join ON HEX(pending_transactions.to_address) = to_join.address
WHERE pending_transactions.multi_transaction_id = 0 WHERE pending_transactions.multi_transaction_id = 0
AND (filterAllActivityStatus OR filterStatusPending) AND (filterAllActivityStatus OR filterStatusPending)
AND ((startFilterDisabled OR timestamp >= startTimestamp) AND ((startFilterDisabled OR timestamp >= startTimestamp)
AND (endFilterDisabled OR timestamp <= endTimestamp) AND (endFilterDisabled OR timestamp <= endTimestamp)
) )
AND (filterActivityTypeAll OR filterActivityTypeSend) AND (filterActivityTypeAll OR filterActivityTypeSend)
AND (filterAllAddresses AND (filterAllAddresses
OR (HEX(pending_transactions.from_address) IN filter_addresses) OR (HEX(pending_transactions.from_address) IN filter_addresses)
OR (HEX(pending_transactions.to_address) IN filter_addresses) OR (HEX(pending_transactions.to_address) IN filter_addresses)
) )
AND (filterAllToAddresses AND (filterAllToAddresses
OR (HEX(pending_transactions.to_address) IN filter_to_addresses) OR (HEX(pending_transactions.to_address) IN filter_to_addresses)
) )
AND (includeAllTokenTypeAssets OR (UPPER(pending_transactions.symbol) IN assets_token_codes)) AND (includeAllTokenTypeAssets OR (UPPER(pending_transactions.symbol) IN assets_token_codes))
AND (includeAllNetworks OR (pending_transactions.network_id IN filter_networks)) AND (includeAllNetworks OR (pending_transactions.network_id IN filter_networks))
UNION ALL UNION ALL
SELECT SELECT
NULL AS transfer_hash, NULL AS transfer_hash,
NULL AS pending_hash, NULL AS pending_hash,
NULL AS network_id, NULL AS network_id,
multi_transactions.ROWID AS multi_tx_id, multi_transactions.ROWID AS multi_tx_id,
multi_transactions.timestamp AS timestamp, multi_transactions.timestamp AS timestamp,
multi_transactions.type AS mt_type, multi_transactions.type AS mt_type,
NULL as tr_type, NULL as tr_type,
multi_transactions.from_address AS from_address, multi_transactions.from_address AS from_address,
multi_transactions.to_address AS to_address, multi_transactions.to_address AS to_address,
NULL AS owner_address, NULL AS owner_address,
NULL AS tr_amount, NULL AS tr_amount,
multi_transactions.from_amount AS mt_from_amount, multi_transactions.from_amount AS mt_from_amount,
multi_transactions.to_amount AS mt_to_amount, multi_transactions.to_amount AS mt_to_amount,
CASE CASE
WHEN tr_status.min_status = 1 AND COALESCE(pending_status.count, 0) = 0 THEN statusSuccess WHEN tr_status.min_status = 1 AND COALESCE(pending_status.count, 0) = 0 THEN statusSuccess
WHEN tr_status.min_status = 0 THEN statusFailed WHEN tr_status.min_status = 0 THEN statusFailed
ELSE statusPending ELSE statusPending
END AS agg_status, END AS agg_status,
COALESCE(tr_status.count, 0) + COALESCE(pending_status.count, 0) AS agg_count, COALESCE(tr_status.count, 0) + COALESCE(pending_status.count, 0) AS agg_count,
NULL AS token_address, NULL AS token_address,
NULL AS token_code, NULL AS token_code,
multi_transactions.from_asset AS from_token_code, multi_transactions.from_asset AS from_token_code,
multi_transactions.to_asset AS to_token_code, multi_transactions.to_asset AS to_token_code,
multi_transactions.from_network_id AS out_network_id, multi_transactions.from_network_id AS out_network_id,
multi_transactions.to_network_id AS in_network_id, multi_transactions.to_network_id AS in_network_id,
NULL AS type, NULL AS type,
NULL as contract_address NULL as contract_address
FROM multi_transactions, filter_conditions FROM multi_transactions, filter_conditions
LEFT JOIN tr_status ON multi_transactions.ROWID = tr_status.multi_transaction_id LEFT JOIN tr_status ON multi_transactions.ROWID = tr_status.multi_transaction_id
LEFT JOIN pending_status ON multi_transactions.ROWID = pending_status.multi_transaction_id LEFT JOIN pending_status ON multi_transactions.ROWID = pending_status.multi_transaction_id
WHERE WHERE
((startFilterDisabled OR multi_transactions.timestamp >= startTimestamp) ((startFilterDisabled OR multi_transactions.timestamp >= startTimestamp)
AND (endFilterDisabled OR multi_transactions.timestamp <= endTimestamp) AND (endFilterDisabled OR multi_transactions.timestamp <= endTimestamp)
) )
AND (filterActivityTypeAll OR (multi_transactions.type IN (%s))) AND (filterActivityTypeAll OR (multi_transactions.type IN (%s)))
AND (filterAllAddresses AND (filterAllAddresses
OR (HEX(multi_transactions.from_address) IN filter_addresses) OR (HEX(multi_transactions.from_address) IN filter_addresses)
OR (HEX(multi_transactions.to_address) IN filter_addresses) OR (HEX(multi_transactions.to_address) IN filter_addresses)
) )
AND (filterAllToAddresses AND (filterAllToAddresses
OR (HEX(multi_transactions.to_address) IN filter_to_addresses) OR (HEX(multi_transactions.to_address) IN filter_to_addresses)
) )
AND (includeAllTokenTypeAssets AND (includeAllTokenTypeAssets
OR (multi_transactions.from_asset != '' AND (UPPER(multi_transactions.from_asset) IN assets_token_codes)) OR (multi_transactions.from_asset != '' AND (UPPER(multi_transactions.from_asset) IN assets_token_codes))
OR (multi_transactions.to_asset != '' AND (UPPER(multi_transactions.to_asset) IN assets_token_codes)) OR (multi_transactions.to_asset != '' AND (UPPER(multi_transactions.to_asset) IN assets_token_codes))
) )
AND (filterAllActivityStatus OR ((filterStatusCompleted OR filterStatusFinalized) AND agg_status = statusSuccess) AND (filterAllActivityStatus
OR (filterStatusFailed AND agg_status = statusFailed) OR (filterStatusPending AND agg_status = statusPending)) OR ((filterStatusCompleted OR filterStatusFinalized) AND agg_status = statusSuccess)
OR (filterStatusFailed AND agg_status = statusFailed) OR (filterStatusPending AND agg_status = statusPending)
)
AND (includeAllNetworks
OR (multi_transactions.from_network_id IN filter_networks)
OR (multi_transactions.to_network_id IN filter_networks)
OR (multi_transactions.from_network_id IS NULL
AND multi_transactions.to_network_id IS NULL
AND (EXISTS (SELECT 1 FROM tr_network_ids WHERE multi_transactions.ROWID = tr_network_ids.multi_transaction_id)
OR EXISTS (SELECT 1 FROM pending_network_ids WHERE multi_transactions.ROWID = pending_network_ids.multi_transaction_id)
)
)
)
ORDER BY timestamp DESC ORDER BY timestamp DESC
LIMIT ? OFFSET ?` LIMIT ? OFFSET ?`
noEntriesInTmpTableSQLValues = "(NULL)" noEntriesInTmpTableSQLValues = "(NULL)"
noEntriesInTwoColumnsTmpTableSQLValues = "(NULL, NULL)" noEntriesInTwoColumnsTmpTableSQLValues = "(NULL, NULL)"

102
services/wallet/activity/activity_test.go
View File

@ -127,9 +127,6 @@ func fillTestData(t *testing.T, db *sql.DB) (td testData, fromAddresses, toAddre
td.multiTx1Tr1 = trs[2] td.multiTx1Tr1 = trs[2]
td.multiTx1Tr2 = trs[4] td.multiTx1Tr2 = trs[4]
// TODO: This got automatically set by GenerateTestTransfers to USDC/Mainnet
//td.multiTx1Tr1.Token = &transfer.SntMainnet
td.multiTx1 = transfer.GenerateTestSendMultiTransaction(td.multiTx1Tr1) td.multiTx1 = transfer.GenerateTestSendMultiTransaction(td.multiTx1Tr1)
td.multiTx1.ToToken = testutils.DaiSymbol td.multiTx1.ToToken = testutils.DaiSymbol
td.multiTx1ID = transfer.InsertTestMultiTransaction(t, db, &td.multiTx1) td.multiTx1ID = transfer.InsertTestMultiTransaction(t, db, &td.multiTx1)
@ -302,8 +299,7 @@ func TestGetActivityEntriesWithSameTransactionForSenderAndReceiverInDB(t *testin
require.Equal(t, SendAT, entries[0].activityType) require.Equal(t, SendAT, entries[0].activityType)
require.NotEqual(t, eth.Address{}, entries[0].transaction.Address) require.NotEqual(t, eth.Address{}, entries[0].transaction.Address)
// TODO: extract and use to/from Address to compare instead of identity require.Equal(t, td.tr1.To, *entries[0].recipient)
require.Equal(t, td.tr1.To, entries[0].transaction.Address)
entries, err = getActivityEntries(context.Background(), deps, []eth.Address{}, []common.ChainID{}, filter, 0, 10) entries, err = getActivityEntries(context.Background(), deps, []eth.Address{}, []common.ChainID{}, filter, 0, 10)
require.NoError(t, err) require.NoError(t, err)
@ -909,15 +905,45 @@ func TestGetActivityEntriesFilterByToAddresses(t *testing.T) {
require.NoError(t, err) require.NoError(t, err)
require.Equal(t, 2, len(entries)) require.Equal(t, 2, len(entries))
} }
func TestGetActivityEntriesFilterByNetworks(t *testing.T) { func TestGetActivityEntriesFilterByNetworks(t *testing.T) {
deps, close := setupTestActivityDB(t) deps, close := setupTestActivityDB(t)
defer close() defer close()
// Adds 4 extractable transactions // Adds 4 extractable transactions
td, fromTds, toTds := fillTestData(t, deps.db) td, fromTds, toTds := fillTestData(t, deps.db)
chainToEntryCount := make(map[common.ChainID]map[int]int)
recordPresence := func(chainID common.ChainID, entry int) {
if _, ok := chainToEntryCount[chainID]; !ok {
chainToEntryCount[chainID] = make(map[int]int)
chainToEntryCount[chainID][entry] = 1
} else {
if _, ok := chainToEntryCount[chainID][entry]; !ok {
chainToEntryCount[chainID][entry] = 1
} else {
chainToEntryCount[chainID][entry]++
}
}
}
recordPresence(td.tr1.ChainID, 0)
recordPresence(td.pendingTr.ChainID, 1)
recordPresence(td.multiTx1Tr1.ChainID, 2)
if td.multiTx1Tr2.ChainID != td.multiTx1Tr1.ChainID {
recordPresence(td.multiTx1Tr2.ChainID, 2)
}
recordPresence(td.multiTx2Tr1.ChainID, 3)
if td.multiTx2Tr2.ChainID != td.multiTx2Tr1.ChainID {
recordPresence(td.multiTx2Tr2.ChainID, 3)
}
if td.multiTx2PendingTr.ChainID != td.multiTx2Tr1.ChainID && td.multiTx2PendingTr.ChainID != td.multiTx2Tr2.ChainID {
recordPresence(td.multiTx2PendingTr.ChainID, 3)
}
// Add 6 extractable transactions // Add 6 extractable transactions
trs, fromTrs, toTrs := transfer.GenerateTestTransfers(t, deps.db, td.nextIndex, 6) trs, fromTrs, toTrs := transfer.GenerateTestTransfers(t, deps.db, td.nextIndex, 6)
for i := range trs { for i := range trs {
recordPresence(trs[i].ChainID, 4+i)
transfer.InsertTestTransfer(t, deps.db, trs[i].To, &trs[i]) transfer.InsertTestTransfer(t, deps.db, trs[i].To, &trs[i])
} }
mockTestAccountsWithAddresses(t, deps.db, append(append(append(fromTds, toTds...), fromTrs...), toTrs...)) mockTestAccountsWithAddresses(t, deps.db, append(append(append(fromTds, toTds...), fromTrs...), toTrs...))
@ -931,14 +957,67 @@ func TestGetActivityEntriesFilterByNetworks(t *testing.T) {
chainIDs = []common.ChainID{5674839210} chainIDs = []common.ChainID{5674839210}
entries, err = getActivityEntries(context.Background(), deps, []eth.Address{}, chainIDs, filter, 0, 15) entries, err = getActivityEntries(context.Background(), deps, []eth.Address{}, chainIDs, filter, 0, 15)
require.NoError(t, err) require.NoError(t, err)
// TODO: update after multi-transactions are filterable by ChainID require.Equal(t, 0, len(entries))
require.Equal(t, 2 /*0*/, len(entries))
chainIDs = []common.ChainID{td.pendingTr.ChainID, td.multiTx2Tr1.ChainID, trs[3].ChainID} chainIDs = []common.ChainID{td.pendingTr.ChainID, td.multiTx2Tr1.ChainID, trs[3].ChainID}
entries, err = getActivityEntries(context.Background(), deps, []eth.Address{}, chainIDs, filter, 0, 15) entries, err = getActivityEntries(context.Background(), deps, []eth.Address{}, chainIDs, filter, 0, 15)
require.NoError(t, err) require.NoError(t, err)
// TODO: update after multi-transactions are filterable by ChainID expectedResults := make(map[int]int)
require.Equal(t, 8 /*6*/, len(entries)) for _, chainID := range chainIDs {
for entry := range chainToEntryCount[chainID] {
if _, ok := expectedResults[entry]; !ok {
expectedResults[entry]++
}
}
}
require.Equal(t, len(expectedResults), len(entries))
}
func TestGetActivityEntriesFilterByNetworksOfSubTransactions(t *testing.T) {
deps, close := setupTestActivityDB(t)
defer close()
// Add 6 extractable transactions
trs, _, toTrs := transfer.GenerateTestTransfers(t, deps.db, 0, 5)
trs[0].ChainID = 1231
trs[1].ChainID = 1232
trs[2].ChainID = 1233
mt1 := transfer.GenerateTestBridgeMultiTransaction(trs[0], trs[1])
trs[0].MultiTransactionID = transfer.InsertTestMultiTransaction(t, deps.db, &mt1)
trs[1].MultiTransactionID = mt1.MultiTransactionID
trs[2].MultiTransactionID = mt1.MultiTransactionID
trs[3].ChainID = 1234
mt2 := transfer.GenerateTestSwapMultiTransaction(trs[3], testutils.SntSymbol, 100)
trs[3].MultiTransactionID = transfer.InsertTestMultiTransaction(t, deps.db, &mt2)
for i := range trs {
if i == 2 {
transfer.InsertTestPendingTransaction(t, deps.db, &trs[i])
} else {
transfer.InsertTestTransfer(t, deps.db, trs[i].To, &trs[i])
}
}
var filter Filter
chainIDs := allNetworksFilter()
entries, err := getActivityEntries(context.Background(), deps, toTrs, chainIDs, filter, 0, 15)
require.NoError(t, err)
require.Equal(t, 3, len(entries))
// Extract sub-transactions by pending
chainIDs = []common.ChainID{trs[0].ChainID, trs[1].ChainID}
entries, err = getActivityEntries(context.Background(), deps, toTrs, chainIDs, filter, 0, 15)
require.NoError(t, err)
require.Equal(t, 1, len(entries))
require.Equal(t, entries[0].id, mt1.MultiTransactionID)
// Extract sub-transactions by
chainIDs = []common.ChainID{trs[2].ChainID}
entries, err = getActivityEntries(context.Background(), deps, toTrs, chainIDs, filter, 0, 15)
require.NoError(t, err)
require.Equal(t, 1, len(entries))
require.Equal(t, entries[0].id, mt1.MultiTransactionID)
} }
func TestGetActivityEntriesCheckToAndFrom(t *testing.T) { func TestGetActivityEntriesCheckToAndFrom(t *testing.T) {
@ -963,8 +1042,7 @@ func TestGetActivityEntriesCheckToAndFrom(t *testing.T) {
require.Equal(t, SendAT, entries[5].activityType) // td.tr1 require.Equal(t, SendAT, entries[5].activityType) // td.tr1
require.NotEqual(t, eth.Address{}, entries[5].transaction.Address) // td.tr1 require.NotEqual(t, eth.Address{}, entries[5].transaction.Address) // td.tr1
// TODO: extract to/from Address and use it for comparison instead of identity address require.Equal(t, td.tr1.To, *entries[5].recipient) // td.tr1
require.Equal(t, td.tr1.To, entries[5].transaction.Address) // td.tr1
require.Equal(t, SendAT, entries[4].activityType) // td.pendingTr require.Equal(t, SendAT, entries[4].activityType) // td.pendingTr
@ -982,8 +1060,6 @@ func TestGetActivityEntriesCheckToAndFrom(t *testing.T) {
// TODO: Test with all addresses // TODO: Test with all addresses
} }
// TODO test sub-transaction count for multi-transactions
func TestGetActivityEntriesCheckContextCancellation(t *testing.T) { func TestGetActivityEntriesCheckContextCancellation(t *testing.T) {
deps, close := setupTestActivityDB(t) deps, close := setupTestActivityDB(t)
defer close() defer close()