author shashankshampi <shashank.sanket1995@gmail.com> 1729780155 +0530
committer shashankshampi <shashank.sanket1995@gmail.com> 1730274350 +0530
test: Code Migration from status-cli-tests
fix_: functional tests (#5979)
* fix_: generate on test-functional
* chore(test)_: fix functional test assertion
---------
Co-authored-by: Siddarth Kumar <siddarthkay@gmail.com>
feat(accounts)_: cherry-pick Persist acceptance of Terms of Use & Privacy policy (#5766) (#5977)
* feat(accounts)_: Persist acceptance of Terms of Use & Privacy policy (#5766)
The original GH issue https://github.com/status-im/status-mobile/issues/21113
came from a request from the Legal team. We must show to Status v1 users the new
terms (Terms of Use & Privacy Policy) right after they upgrade to Status v2
from the stores.
The solution we use is to create a flag in the accounts table, named
hasAcceptedTerms. The flag will be set to true on the first account ever
created in v2 and we provide a native call in mobile/status.go#AcceptTerms,
which allows the client to persist the user's choice in case they are upgrading
(from v1 -> v2, or from a v2 older than this PR).
This solution is not the best because we should store the setting in a separate
table, not in the accounts table.
Related Mobile PR https://github.com/status-im/status-mobile/pull/21124
* fix(test)_: Compare addresses using uppercased strings
---------
Co-authored-by: Icaro Motta <icaro.ldm@gmail.com>
test_: restore account (#5960)
feat_: `LogOnPanic` linter (#5969)
* feat_: LogOnPanic linter
* fix_: add missing defer LogOnPanic
* chore_: make vendor
* fix_: tests, address pr comments
* fix_: address pr comments
fix(ci)_: remove workspace and tmp dir
This ensures we do not encounter weird errors like:
```
+ ln -s /home/jenkins/workspace/go_prs_linux_x86_64_main_PR-5907 /home/jenkins/workspace/go_prs_linux_x86_64_main_PR-5907@tmp/go/src/github.com/status-im/status-go
ln: failed to create symbolic link '/home/jenkins/workspace/go_prs_linux_x86_64_main_PR-5907@tmp/go/src/github.com/status-im/status-go': File exists
script returned exit code 1
```
Signed-off-by: Jakub Sokołowski <jakub@status.im>
chore_: enable windows and macos CI build (#5840)
- Added support for Windows and macOS in CI pipelines
- Added missing dependencies for Windows and x86-64-darwin
- Resolved macOS SDK version compatibility for darwin-x86_64
The `mkShell` override was necessary to ensure compatibility with the newer
macOS SDK (version 11.0) for x86_64. The default SDK (10.12) was causing build failures
because of the missing libs and frameworks. OverrideSDK creates a mapping from
the default SDK in all package categories to the requested SDK (11.0).
fix(contacts)_: fix trust status not being saved to cache when changed (#5965)
Fixes https://github.com/status-im/status-desktop/issues/16392
cleanup
added logger and cleanup
review comments changes
fix_: functional tests (#5979)
* fix_: generate on test-functional
* chore(test)_: fix functional test assertion
---------
Co-authored-by: Siddarth Kumar <siddarthkay@gmail.com>
feat(accounts)_: cherry-pick Persist acceptance of Terms of Use & Privacy policy (#5766) (#5977)
* feat(accounts)_: Persist acceptance of Terms of Use & Privacy policy (#5766)
The original GH issue https://github.com/status-im/status-mobile/issues/21113
came from a request from the Legal team. We must show to Status v1 users the new
terms (Terms of Use & Privacy Policy) right after they upgrade to Status v2
from the stores.
The solution we use is to create a flag in the accounts table, named
hasAcceptedTerms. The flag will be set to true on the first account ever
created in v2 and we provide a native call in mobile/status.go#AcceptTerms,
which allows the client to persist the user's choice in case they are upgrading
(from v1 -> v2, or from a v2 older than this PR).
This solution is not the best because we should store the setting in a separate
table, not in the accounts table.
Related Mobile PR https://github.com/status-im/status-mobile/pull/21124
* fix(test)_: Compare addresses using uppercased strings
---------
Co-authored-by: Icaro Motta <icaro.ldm@gmail.com>
test_: restore account (#5960)
feat_: `LogOnPanic` linter (#5969)
* feat_: LogOnPanic linter
* fix_: add missing defer LogOnPanic
* chore_: make vendor
* fix_: tests, address pr comments
* fix_: address pr comments
chore_: enable windows and macos CI build (#5840)
- Added support for Windows and macOS in CI pipelines
- Added missing dependencies for Windows and x86-64-darwin
- Resolved macOS SDK version compatibility for darwin-x86_64
The `mkShell` override was necessary to ensure compatibility with the newer
macOS SDK (version 11.0) for x86_64. The default SDK (10.12) was causing build failures
because of the missing libs and frameworks. OverrideSDK creates a mapping from
the default SDK in all package categories to the requested SDK (11.0).
fix(contacts)_: fix trust status not being saved to cache when changed (#5965)
Fixes https://github.com/status-im/status-desktop/issues/16392
test_: remove port bind
chore(wallet)_: move route execution code to separate module
chore_: replace geth logger with zap logger (#5962)
closes: #6002
feat(telemetry)_: add metrics for message reliability (#5899)
* feat(telemetry)_: track message reliability
Add metrics for dial errors, missed messages,
missed relevant messages, and confirmed delivery.
* fix_: handle error from json marshal
chore_: use zap logger as request logger
iterates: status-im/status-desktop#16536
test_: unique project per run
test_: use docker compose v2, more concrete project name
fix(codecov)_: ignore folders without tests
Otherwise Codecov reports incorrect numbers when making changes.
https://docs.codecov.com/docs/ignoring-paths
Signed-off-by: Jakub Sokołowski <jakub@status.im>
test_: verify schema of signals during init; fix schema verification warnings (#5947)
fix_: update defaultGorushURL (#6011)
fix(tests)_: use non-standard port to avoid conflicts
We have observed `nimbus-eth2` build failures reporting this port:
```json
{
"lvl": "NTC",
"ts": "2024-10-28 13:51:32.308+00:00",
"msg": "REST HTTP server could not be started",
"topics": "beacnde",
"address": "127.0.0.1:5432",
"reason": "(98) Address already in use"
}
```
https://ci.status.im/job/nimbus-eth2/job/platforms/job/linux/job/x86_64/job/main/job/PR-6683/3/
Signed-off-by: Jakub Sokołowski <jakub@status.im>
fix_: create request logger ad-hoc in tests
Fixes `TestCall` failing when run concurrently.
chore_: configure codecov (#6005)
* chore_: configure codecov
* fix_: after_n_builds
This commit simplifies the sending process of the best route suggested by the router.
It also makes the sending process the same for accounts (key pairs) migrated to a keycard
and those stored locally in local keystore files.
Deprecated endpoints:
- `CreateMultiTransaction`
- `ProceedWithTransactionsSignatures`
Deprecated signal:
- `wallet.sign.transactions`
New endpoints:
- `BuildTransactionsFromRoute`
- `SendRouterTransactionsWithSignatures`
The flow for sending the best router suggested by the router:
- call `BuildTransactionsFromRoute`
- wait for the `wallet.router.sign-transactions` signal
- sign received hashes using `SignMessage` call or sign on keycard
- call `SendRouterTransactionsWithSignatures` with the signatures of signed hashes from the previous step
- `wallet.router.transactions-sent` signal will be sent after transactions are sent or if an error occurs
New signals:
- `wallet.router.sending-transactions-started` // notifies client that the sending transactions process started
- `wallet.router.sign-transactions` // notifies client about the list of transactions that need to be signed
- `wallet.router.transactions-sent` // notifies client about transactions that are sent
- `wallet.transaction.status-changed` // notifies about status of sent transactions
* feat(connector)_: impl `eth_requestAccounts` for browser plugin
* feat(connector)_: add impl for `wallet_switchEthereumChain` and `eth_chainId`
* feat(connector)_: add impl for `eth_sendTransaction`
* feat(connector)_: add a signal and an endpoint for wallet ui side
* chore_: refactor connector tests
* feat(connector)_: impl `eth_requestAccounts` with signal
* chore(connector)_: Add test, covering full transaction flow
And polish impl & test for connector endpoints
* fix(connector)_: temporary allow all origins for ws connection
* chore_: review fixes
* fix(connector)_: make user select chain id for dApp
* fix(connector)_: add requestID and fine tune endpoints
* chore(connector)_: naming fixes and tests improvments
A signal type `wallet` which had an event `type` set to `sing-transactions` is now a new signal
type `wallt.sing.transactions` with event which represents list of transactions that need to be signed.
Send information to owners and token masters about operations: burn, airdrop, remote destruct.
Add CommunityTokenActionSignal to signalize client side.
Fix#13371
Use EventWatcher to catch wallet events.
Handling all community tokens wallet events in communitytokens service (database and messenger operations).
Adding new signal to nim: CommunityTokenTransactionSignal, which is emitted everytime when the event is received.
Issue #4351
This functionality is needed in case the user wants to send a transaction and
signs it using the signature provided by the keycard (or any other compatible way).
* sync preferred name;
remove settings.usernames
* update account name when handle settings.preferred_name from backup message
* fix Error:Field validation for 'KeycardPairingDataFile' failed on the 'required' tag
* bump version
* rebase
This is the first step of improvements over keypairs/keycards/accounts.
- `SyncKeypairFull` protobuf removed
- `SyncKeypair` protobuf is used for syncing all but the watch only accounts
- `SyncAccount` is used only for syncing watch only accounts
- related keycards are synced together with a keypair
- on any keypair change (either it's just a keypair name or any change made over an
account which belongs to that keypair) entire keypair is synced including related keycards
- on any watch only account related change, that account is synced with all its details
* perf(sqlCipher): Increase cipher page size to 8192
Increasing the cipher page size to 8192 requires DB re-encryption. The process is as follows:
//Login to v3 DB
PRAGMA key = 'key';
PRAGMA cipher_page_size = 1024"; // old Page size
PRAGMA cipher_hmac_algorithm = HMAC_SHA1";
PRAGMA cipher_kdf_algorithm = PBKDF2_HMAC_SHA1";
PRAGMA kdf_iter = kdfIterationsNumber";
//Create V4 DB with increased page size
ATTACH DATABASE 'newdb.db' AS newdb KEY 'key';
PRAGMA newdb.cipher_page_size = 8192; // new Page size
PRAGMA newdb.cipher_hmac_algorithm = HMAC_SHA1"; // same as in v3
PRAGMA newdb.cipher_kdf_algorithm = PBKDF2_HMAC_SHA1"; // same as in v3
PRAGMA newdb.kdf_iter = kdfIterationsNumber"; // same as in v3
SELECT sqlcipher_export('newdb');
DETACH DATABASE newdb;
//Login to V4 DB
...
Worth noting:
The DB migration will happen on the first successful login.
The new DB version will have a different name to be able to distinguish between different DB versions.Versions naming mirrors sqlcipher major version (naming conventions used by sqlcipher), meaning that we're migrating from V3 to V4 DB (even if we're not fully aligned with V4 standards). The DB is not migrated to the v4 standard `SHA512` due to performance reasons. Our custom `SHA1` implementation is fully optimised for perfomance.
* perf(sqlCipher): Fixing failing tests
Update the new DB file format in Delete account, Change password and Decrypt database flows
* perf(SQLCipher): Increase page size - send events to notify when the DB re-encryption starts/ends
- old `accounts` table is moved/mapped to `keypairs` and `keypairs_accounts`
- `keycards` table has foreign key which refers to `keypairs.key_uid`
- `Keypair` introduced as a new type
- api endpoints updated according to this change
Changes applied here introduce:
- improvements to sync wallet accounts among devices (including all account types)
- backing up wallet accounts to and fetch them from waku (an information about received
wallet accounts is sent via `waku.backedup.wallet-account` signal to a client)
Changes applied here introduce backing up keycards data to and fetch them from waku.
Information about received keycards data is sent via `sync.from.waku.keycards` signal.
Changes applied here introduce backing up profile data (display name and identity
images) to waku and fetch them from waku. Information about those data is sent
as a separate signal to a client via `sync.from.waku.profile` signal.
New signal `sync.from.waku.progress` is introduced which will be used to notify a client
about the progress of fetching data from waku.
In order to give clients more insights about archive messages being
processed, we're adding this additional signal that informs clients when
the import of downloaded history archive messages has started.
This adds a new `DownloadingHistoryArchivesFinished` signal to the
family of community archive signals. It's emitted when all to be
downloaded archives have been downloaded and handled.
As part of the new Discord <-> Status Community Import functionality,
we're adding an API that extracts all discord categories and channels
from a previously exported discord export file.
These APIs can be used in clients to show the user what categories and
channels will be imported later on.
There are two APIs:
1. `Messenger.ExtractDiscordCategoriesAndChannels(filesToimport
[]string) (*MessengerResponse, map[string]*discord.ImportError)`
This takes a list of exported discord export (JSON) files (typically one per
channel), reads them, and extracts the categories and channels into
dedicated data structures (`[]DiscordChannel` and `[]DiscordCategory`)
It also returns the oldest message timestamp found in all extracted
channels.
The API is synchronous and returns the extracted data as
a `*MessengerResponse`. This allows to make the API available
status-go's RPC interface.
The error case is a `map[string]*discord.ImportError` where each key
is a file path of a JSON file that we tried to extract data from, and
the value a `discord.ImportError` which holds an error message and an
error code, allowing for distinguishing between "critical" errors and
"non-critical" errors.
2. `Messenger.RequestExtractDiscordCategoriesAndChannels(filesToImport
[]string)`
This is the asynchronous counterpart to
`ExtractDiscordCategoriesAndChannels`. The reason this API has been
added is because discord servers can have a lot of message and
channel data, which causes `ExtractDiscordCategoriesAndChannels` to
block the thread for too long, making apps potentially feel like they
are stuck.
This API runs inside a go routine, eventually calls
`ExtractDiscordCategoriesAndChannels`, and then emits a newly
introduced `DiscordCategoriesAndChannelsExtractedSignal` that clients
can react to.
Failure of extraction has to be determined by the
`discord.ImportErrors` emitted by the signal.
**A note about exported discord history files**
We expect users to export their discord histories via the
[DiscordChatExporter](https://github.com/Tyrrrz/DiscordChatExporter/wiki/GUI%2C-CLI-and-Formats-explained#exportguild)
tool. The tool allows to export the data in different formats, such as
JSON, HTML and CSV.
We expect users to have their data exported as JSON.
Closes: https://github.com/status-im/status-desktop/issues/6690
This introduces logic needed to:
- Create WakuMessageArchives and and indices from store waku messages
- History archive torrent data to disk and create .torrent file from
that
- Seed and unseed history archive torrents as necessary
- Starting/stopping the torrent client
- Enabling/disabling community history support for individual components
and starting/stopping the routine intervals accordingly
This does not yet handle magnet links (#2568)
Closes#2567
This commit introduces a new `TorrentConfig` as per #2563 with dedicated
default values and new options/flags for the status-go CLI.
Since it's part of `NodeConfig`, which is stored per user in the
database, this commit also adds a migration script that adds a new
`torrent_config` table and database APIs to insert and retreive the
torrent config.
Closes#2563
This commit enables mailserver cycle logic by default and make a few
changes:
1) Nodes are graylisted instead of being blacklisted for a set amount of
time. The reason is that if we blacklist, any cut in connectivity
might result in long delays before reconnecting, especially on spotty
connections.
2) Fixes an issue on the devp2p server, whereby the node would not
connect to one of the static nodes since all the connection slots
where filled. The fix is a bit inelegant, it always connects to
static nodes, ignoring maxpeers, but it's tricky to get it to work
since the code is clearly not written to select a specific node.
3) Adds support to pinned mailservers
4) Add retries to mailservers requests. It uses a closure for now, I
think we should eventually have a channel etc, but I'd leave that for
later.
We periodically check whether we should backup contacts through waku.
Currently it's not tied to any event, it will only schedule a backup on
a tick, or through the provided API endpoint.
shashankshampi
Sale Djenic
richΛrd
Igor Sirotin
Mikhail Rogachev
Michal Iskierko
Jonathan Rainville
Boris Melnik
frank
Volodymyr Kozieiev
Andrea Maria Piana
Alex Jbanca
Pascal Precht
Samuel Hawksby-Robinson
Parvesh Monu
Jakub Sokołowski
Vitaliy Vlasov