Switch from the prototype of duplicating the SQL filter as a runtime
and keeping them in sync on each event that might invalidate the current
filtered entries to a simpler approach of requesting the filter again
and doing the diff to detect the new changes.
Also add a new reset API to model the new entries design requirements.
The new approach shows less corner-case to handle and follows one source
of truth concept making debugging and future maintenance easier.
Other changes
- Fix pending mocking to work with multiple calls
- Refactor tests to account for the new changes
Updates status-desktop #12120
This commit introduces the first steps towards implementing a session-based activity API to support dynamic updates of the current visualized filter in the wallet activity service. This change is necessary to move away from static paginated filtering, which was previously done in SQL, to a more dynamic approach that can handle updates in real-time.
The main changes include:
- Add basic `EventActivitySessionUpdated` support for pending transactions.
- Added a `TODO.md` file outlining the plan and requirements for dynamic activity updates.
- New session-related API to the `activity.Service`
- `session.go` contains the logic for session management and event processing related to activity updates.
- Add test case for incremental filter updates.
The commit also includes:
- various other minor changes and refactoring to support the new session-based approach.
- Deprecation notices added to the `api.go` file for methods that are no longer used by the status-desktop application.
- Clarification comments added to the `scheduler.go` file regarding replacement policies.
Updates: #12120
ghstack-source-id: a61ef74184
Pull Request resolved: https://github.com/status-im/status-go/pull/4480
The reading of the amount for pending transactions was done in the same
way as for transfers table. However, the transfers table has a string
hex representation of the amount, while the pending transactions table
has a binary representation of the amount (*big.Int). This was
triggering the not int warning and value was missing.
Updates status-desktop #12120
Brings consistency in case when sender and receiver are both in the
filter address list. This fixes the case of sender and receiver in
addresses and filters out duplicate entries.
Also
- refactor tests to provide support for owners
- adapt TestGetActivityEntriesWithSameTransactionForSenderAndReceiverInDB
to the use of owner instead of from
Trigger async fetching of extra information on each activity filtering
request. Only emit the update event for incomplete entries.
Other changes:
- Make DataEntry light as event payload by making all the fields
optional
- Add new required fields to the activity DataEntry
- Add collectibles.ManagerInterface to aid testing
Note: this PR keeps compatibility with current master by always
providing non-optional multi-transaction ID. The TODO will be executed
before merging the status-desktop PR.
Experienced a hang on FetchAssetsByCollectibleUniqueID call with:
[{{5 0x21263a042aFE4bAE34F08Bb318056C181bD96D3b} 1209},
{{5 0x9A95631794a42d30C47f214fBe02A72585df35e1} 237},
{{5 0x9A95631794a42d30C47f214fBe02A72585df35e1} 236},
{{5 0x9A95631794a42d30C47f214fBe02A72585df35e1} 832},
{{5 0x9A95631794a42d30C47f214fBe02A72585df35e1} 830},
{{5 0x9A95631794a42d30C47f214fBe02A72585df35e1} 853}]
Updates status-desktop #11597
Mainly refactor API to have control on pending_transactions operations.
Use the new API to migrate the multi-transaction ID from to transfers
in one SQL transaction.
The refactoring was done to better mirror the purpose of pending_transactions
Also:
- Externalize TransactionManager from WalletService to be used by
other services
- Extract walletEvent as a dependency for all services that need to
propagate events
- Batch chain requests
- Remove unused APIs
- Add auto delete option for clients that fire and forget transactions
Updates status-desktop #11754
Added a unit test for changing app and wallet DBs passwords.
Refactored geth_backend to simplify and allow wallet db password changing.
Fixed opening database with wrong password.
The only place where appDB is used in wallet is activity,
which refers to `keycards_accounts` table. So a temporary
table `keycards_accounts` is created in wallet db and updated
before each activity query.
Add wallet events feed to TransactionManager and send pending changed
events on add and delete
Also
- Remove hardcoded values in the filter query
- Small improvement to query
Updates status-desktop #11233
This fixes returning address identity after that was changed by
the real to/from fix. Previously the address was wrongly used as from
Updates status-desktop #11233
Added test to validate the fix
Also change JOIN to LEFT JOIN between `multi_transactions` and `transactions` table so that we return all entries from `multi_transactions` table even if there is no matching entry in `transactions` table, which might be the case for multi transactions that that could not be detected. This will postpone the error case until we get into details of the multi transaction.
Updates status-desktop #11233
Implement activity.Scheduler to serialize and limit the number of
calls on the activity service. This way we protect form inefficient
parallel queries and easy support async and rate limiting based on the
API requirements.
Refactor the activity APIs async and use the Scheduler for managing
the activity service calls configured with one of the two rules: cancel
ignore.
Updates status-desktop #11170
Main changes:
- Refactor activity API to propagate token identities.
- Extend service to convert token identities to symbols for filtering
multi-transaction
- Filter transfers, pending_transactions and multi-transactions based
on the provided token identities
- Return involved token identities in activity API
- Test token filtering
Also:
- Fixed calling cancel on a filer activity completed task to release
resources
Notes:
- Found limitations with the token identity which complicates things
by not allowing to filter by token groups (like token-code does)
Updates status-desktop #11025
Refactor the filter interface to be an async call which returns
the result using a wallet event
A call to the filter API will cancel the ongoing filter and receive
an error result event
Closes status-desktop #10994
Main changes:
- extend DB query include status for transactions and pending_transactions
- extend DB query to aggregate data from complex activity entries
- extend tests to include the new status values
Other changes:
- Improve tests with mocked addresses in DB to have a true to/from state
Update status-desktop #10746
Extended the migration process with a generic way of applying custom
migration code on top of the SQL files. The implementation provides
a safer way to run GO code along with the SQL migrations and possibility
of rolling back the changes in case of failure to keep the database
consistent.
This custom GO migration is needed to extract the status from
the JSON blob receipt and store it in transfers table.
Other changes:
- Add NULL DB value tracking to JSONBlob helper
- Index status column on transfers table
- Remove unnecessary panic calls
- Move log_parser to wallet's common package and use to extract token
identity from the logs
Notes:
- there is already an index on transfers table, sqlite creates one for
each unique constraint therefore add only status to a new index
- the planned refactoring and improvements to the database have been
postponed due to time constraints. Got the time to migrate the data
though, extracting it can be done later for a more efficient
implementation
Update status-desktop #10746
It uses the current data only and doesn't extend with new types or
include new features in activity sources DBs.
Major changes:
- Partially filter by chain IDs
- Partially filter by Status if it is the case
- Partially filter by token types
- Filter by counterparty addresses
- Use wallet accounts for TO/FROM instead of filters
Closes: #10634
Add the possibility of retrieving the metadata of wallet activity based
on the given filter criteria.
Current implementation relies that after fetching the metadata, user
will follow up with more requests for details. However, after some
experimenting I'm considering extracting all required information
for the summary viewing while filtering. This way there will be no
need for another batch requests for transfers, multi-transactions and
pending transactions to show the summary. Only when user wants to see
the details for one will specifically request it.
For this first prototype, the filter criteria is limited to:
- time
- type
- addresses
Major changes:
- Add the filter definition to be used in propagating the filter
information
- Add GetActivityEntries API to return the list of activity entries
for the given addresses/chainIDs by a view in the complete list
- GetTransfersForIdentities to batch retrieve further details of the
transfers
- GetPendingTransactionsForIdentities to batch retrieve further details
of the pending transactions
- Added a new package testutils for tests.
- Added tests
Updates status-desktop #10366
Closes status-desktop #10633
Sale Djenic
Cuteivist
Stefan
Ivan Belyakov
Dario Gabriel Lipicar