triggerE2E passed only the package job's fullProjectName, so tests-e2e
resolved the artifact with lastWithArtifacts() and copied whichever
build of that job had archived most recently. The job is shared by
every status-go PR check and by status-app's own nightly and release
builds, so a PR's e2e could run another run's binary.
Passes the package build's number alongside the path so the downstream
job can select it with specific(). It is stringified because
jenkins.mapToParams only maps String and Boolean and silently drops
anything else.
Requires status-im/status-app#22110, which adds the SOURCE_BUILD_NUMBER
parameter. Unknown parameters are ignored, so merge order is free.
protobuf v36 added `--<lang>_prefix=COMMAND`, which runs a protoc plugin
through a wrapper command instead of looking it up on PATH. That lets the
Go generator be declared in go.mod like every other tool dependency, so
its version is tracked in one place instead of three.
go-generate-fast rejects protoc flags it doesn't know, so the fork pin
moves to a commit that teaches it about --go_prefix.
Part of #7099.
Bumps the compiler from 29.x to 36.0 and the Go runtime module from
v1.36.4 to v1.36.12.
The pinned nixpkgs only packages protobuf up to 33, so protobuf_36 is
built in the overlay from the same nixpkgs generic builder.
* build: switch go-generate-fast to upstream v0.5.0
`feat: support go tool` landed upstream (oNaiPs/go-generate-fast#26), so
the status-im fork and its replace directive are no longer needed.
v0.5.0 carries upstream's dependency refresh, which raises minimum
versions across the module graph.
* test: stop expecting bare IP URLs to be unfurled
`parseValidURL` rejects hosts without a registrable domain. Up to
golang.org/x/net v0.46.0, `EffectiveTLDPlusOne("192.168.0.100")` returned
"0.100" with no error, so IP hosts passed that check by accident. v0.48.0
reports an error for them, which is correct.
Nothing unfurls a link preview for a bare IP any more, and nothing should:
the request would be issued by the recipient's client, and there is no
private-address guard in the unfurlers.
Allow clients to use the profile's data-encryption key as the biometric login credential instead
of the raw password:
- LoginAccount: new `dek` request field (32 hex bytes, validated and normalized to lowercase,
mutually exclusive with password/mnemonic/keycard keys)
- resolveProfileSecret: accepts the client-hashed DEK ("0x" + keccak256), so a stored DEK
credential works through the existing hashed-password paths on the client side
- New ExportProfileDEK endpoint: returns the DEK for a valid credential
Two proxy-backed paths asked for a currency the proxy does not answer in:
- /coins/markets normalizes vs_currency to usd for cache consistency, so a
request for any other currency came back in USD and was then labelled with
the user's currency. convert_currency is the parameter that has an effect
there.
- /simple/price only serves the currencies the proxy holds, so anything else
came back without the requested key and every price silently read 0.
Both now ask for the currency the caller wants through convert_currency and
let the proxy decide where the values come from. The converted currency is
never also listed in vs_currencies, which keeps the request valid whether
the proxy serves provider values for it or converts them.
Only the client pointed at the proxy sends convert_currency; the direct
api.coingecko.com fallback keeps using vs_currency/vs_currencies, which is
what it understands.
The leaderboard endpoints were always fetched without a currency, so the
Market tab showed USD values labelled with whatever currency the user had
selected (status-app#21273). Send convert_currency so the proxy serves the
values in that currency.
The currency comes from the settings DB, seeded at start and followed
through the accounts publisher; the currency a client passes to
FetchMarketTokenPageAsync keeps working as before. Cached values, ETags and
the persisted snapshot all belong to the currency they were fetched in, so a
change drops them and triggers the refresh loops, which push the replacement
data to the client over the same path as any other refresh. With no client
listening the cache is simply left invalidated for the next page request.
Because fetching and storing are not one atomic step, a response carries the
currency it was requested in and is dropped if that no longer matches - a
request in flight across a currency change would otherwise label one
currency's values as another's and refresh the timestamp that decides
whether anything needs fetching.
If the proxy rejects the currency with a 400 the request is retried once
without the conversion, so the tab falls back to USD values rather than
staying empty, and that currency is not asked for again.
Rows persisted before the migration are USD, which is what the proxy served
when no conversion was requested.
doGetRequest built its non-2xx error with fmt.Errorf, leaving a caller that
has to react to one specific status no option but to match on the message.
Return an *HTTPStatusError instead, carrying the code and the body, with an
unchanged Error() string.
The seven network methods on the wallet API were pass-throughs to the
network manager. They now live on the networks service, under the
networks_ namespace.
The three deprecated ones are carried over rather than dropped: the
functional tests still use addEthereumChain to attach the Anvil chain
with a user provider, and getEthereumChains to read it back.
The python client gains a NetworksService and the two call sites move
to it.
params/networkhelper only ever served the network manager and the
default network table, both of which now live in pkg/services/networks.
Eight of its ten functions are unexported now that their callers are in
the same package. OverrideBasicAuth and GetEmbeddedProviders stay
exported: tests in other packages build networks with them.
The two test files become internal tests, since what they cover is no
longer exported.
params/networkdefaults holds the embedded network table and builds the
default network list. It belongs with the service that owns networks,
not in params.
Dissolved into the networks package rather than kept as a subpackage:
BuildDefaultNetworks reads better than defaults.BuildDefaultNetworks,
and the next commit unexports the helpers both halves share.
The network manager lived in internal/rpc and was constructed, started
and stopped by rpc.Client, which reached back into it to route calls by
chain. It is now a service of its own at pkg/services/networks.
StatusNode owns the manager and hands it to rpc.Client through
ClientConfig, so the client depends on ManagerInterface rather than the
concrete type. The service owns the manager lifecycle.
The four live network RPC methods are registered under the networks_
namespace. The wallet_ ones are left in place so nothing breaks before
the app migrates; they are removed at the end of the stack.
BuildDefaultNetworks took requests.WalletSecretsConfig, the raw wire
struct, reaching past the translation that already exists in
buildWalletConfig. It now takes params.WalletConfig, which already
carried six of the seven fields it needs.
PoktAPIKey is added to params.WalletConfig alongside the other provider
keys; it was the only field missing.
This drops networkdefaults' dependency on internal/protocol/requests,
and with it the whole protocol package graph.
Skip the automatic Status support bot contact request for existing and
upgraded accounts, while marking the request state complete to prevent
retries on later startups.
New accounts retain the automatic request and welcome message.
Needed for https://github.com/status-im/status-app/issues/21861
Automatically send a contact request to the Status support bot when a profile starts:
- Use a new-user or upgraded-user message based on account creation flow
- Persist request state to prevent repeat sends
- Suppress duplicates on paired devices and when the bot contact already exists
- Add settings migration, pairing handling, startup wiring, and focused tests
Removes the test-only fields (TestsMode, TestEstimationMap, TestBonderFeeMap,
TestApprovalGasEstimation, TestApprovalL1Fee) from ProcessorInputParams and
TestsMode/TestParams (with RouterTestParams and Estimation types) from
requests.RouteInputParams.
Updates the production code accordinglly.
* fix: map envelope.ErrInvalidKEK from the DB-open to incorrect password error
* fix: wrong integrator used
* feat: Status' fee fraction for the LiFi swap added
migrateProfileToDEK now verifies oldKEK before anything is written, by opening the app DB with it
via a new small verifyDBKey helper. A wrong password now returns an error.
Given a profile's wrapped-DEK file and the profile password, prints the DEK.
The secret the profile's databases and keystore files are encrypted with and
the sqlcipher pragmas needed to open the databases manually.
The wire format stays password-encrypted keystore files, so pairing works across app
versions (not migrated and DEK migrated profiles) in both directions (sender/receiver):
- sender: migrated profiles re-encrypt keystore files in memory from the DEK to the transfer
password before marshalling
- receiver (account transfer): a brand-new profile adopts a fresh device-local DEK
- receiver (keystore-files transfer): received files are re-encrypted per file to the logged-in
profile's keystore secret
- every failure after the profile keystore directory is created cleans up the profile state
New profiles use the DEK scheme from day one (kdf_iter 3200).
Password change now auto-detects the profile's encryption scheme:
- profile on the DEK scheme, rekey=false → fast path: only the wrapped-DEK file is re-wrapped (no new DEK is generated)
- profile on the DEK scheme, rekey=true → deep rekey: fresh DEK, databases and keystore re-encrypted (new DEK is generated)
- legacy profile → one-time migration to the DEK scheme (full re-encryption, new DEK is generated)
Api changes:
- GetProfileEncryptionInfo endpoint added
- a rekey flag on ChangeDatabasePasswordV2
Part of the Go project layout migration, item 3a: the signing phrase is
no longer a feature, so nothing should generate, store or read it.
pkg/backend/defaults.go buildSigningPhrase and its call
pkg/backend/seed_phrase_dictionary.go the 626-word list it drew from,
which had no other consumer
settings.Settings.SigningPhrase the field and its json tag
settings/database.go the column in INSERT and SELECT
migrations ALTER TABLE settings DROP COLUMN
The column is NOT NULL with no default, so dropping it from the writes
and dropping it from the table have to land together. The migration
follows the pattern of 1779877216_drop_keycard_settings_columns.
refs #7067
Part of the Go project layout migration, item 4.
internal/db/walletdatabase -> internal/db/walletdb, and its one 45-line
file splits along the two jobs it was doing:
open.go DbInitializer, InitializeDB, OpenDB
migrate.go walletCustomSteps, doMigration, MigrateDB
scripts/migration_check.sh listed this migration directory as
"walletdatabase/migrations/sql" and appdatabase's as
"appdatabase/migrations/sql". Neither path has existed since those
packages moved under internal/db/, so the check has been silently
skipping both. Both are corrected here.
refs #7067
The package sat at the repo root but is imported from exactly one place,
internal/protocol/pinnedcommunities/loader.go. It moves in beside it as
.../pinnedcommunities/assets, and the package is renamed to match its
directory so the import no longer needs an alias.
The embedded payload travels with it; the //go:embed pattern is relative
to the package directory, so it is unchanged.
refs #7067
Reproducing a reporter's performance problem currently means guessing the
shape of their account: how many messages they hold, how those spread over
chats, how far behind their sync state is. Guessing that cost days on
status-im/status-app#21605.
Add a storagestats service that answers those questions with numbers a
reporter can paste into a public ticket. `storagestats_collect` starts a
background walk and returns immediately; progress arrives as
storage-stats.progress ("N of M", with M known upfront because the table
list is enumerated first) and the finished profile as storage-stats.result.
Nothing is collected unless a caller asks.
The walk is serial on purpose: COUNT(*) on a sqlcipher table is a full
decrypting scan, so running several at once would only starve the process it
is meant to describe. It must never be driven from a client's UI thread.
Privacy is enforced by construction: time appears only as relative day
counts, no per-entity row is ever read (the per-chat histogram query selects
counts and nothing else), and the only strings in the artifact are table
names from our own schema.
Section one carries curated metrics whose names mirror the seeder harness
knobs so a profile maps onto a harness run; section two is a schema-agnostic
table -> {rows, bytes} dump of both databases, insurance against the next
surprise living in a table nobody predicted. A step that fails names itself
in `incomplete` rather than leaving a zero that reads as "this account has
none".
Part of the Go project layout migration, item 31.
Pure move plus import-path rewrite across 687 files. No API or behaviour
change.
The services keep their grouping under pkg/services/<name> rather than
being promoted to pkg/<name>: 27 top-level directories in pkg/ would read
worse than what we have, and the grouping is what makes "an RPC service"
identifiable at a glance.
Paths that follow the move: the logosstorage test target and generate
step, the two wallet token-list tools, the migration-order check (and the
pre-rebase hook symlinked to it), and the storage env helper.
refs #7067
Part of the Go project layout migration, item 27.
Pure move plus import-path rewrite across 502 files. No API or behaviour
change. `internal/` keeps the messaging application logic unimportable
from outside the module, which is what the issue asks for -- status-go is
consumed through the C-bindings in mobile/, not as a Go library.
Things that had to follow the move, beyond the Go imports:
- tools/generate-handlers/template.txt. messenger_handlers.go is
generated, and the template hard-codes the imports it emits, so the
generated file kept importing protocol/common and failed typecheck.
- .gitignore. The ignore rule for that generated file was pinned to the
old path; without moving it, a 1486-line generated file starts being
tracked.
- Makefile: the logosstorage and torrent test targets (both the archive
packages and ./protocol itself), the archive README, migration-protocol.
- scripts/run_unit_tests.sh, which names the protocol package explicitly
to shard its tests.
- scripts/cleanup_generated_files.sh and .golangci.yml.
scripts/migration_check.sh also needed a fix that is not specific to this
move: it validated every file the branch touched under a migration dir
against the timestamp naming rule, and a directory rename makes every
migration in it look newly added. It now excludes renames, so moving a
migration is not mistaken for adding one.
refs #7067
tests-functional/ -> test/functional/
tests-unit-network/ -> test/unit-network/
No Go code changes beyond the two package paths. Every reference follows:
Makefile targets, the pytest-lint and reliability workflows, the three
Jenkinsfiles, the root Dockerfile, pyrightconfig.json, .gitignore, the
benchmark and functional-test scripts, and the docs.
All paths inside the moved directories are self-contained (docker compose
uses `context: .`, the Python helpers resolve from __file__), so the extra
level of nesting does not reach outside.
refs #7067
`server` was one Go package doing three unrelated jobs. Each moves to
where it belongs:
server/pairing/ -> services/pairing/
server_media*.go,
handlers*.go, testdata/ -> services/media/
server.go, certs.go,
ips.go, timeout.go,
device.go, listen_*.go,
servertest/ -> internal/httpserver/
MediaServer is renamed to media.Server (and NewMediaServer to
media.NewServer) now that it has a package to be named against.
Deliberately untouched: StatusNode.MediaServer(), which is an accessor
method rather than the type, and the unrelated identifiers that only
share the prefix (MediaServerImageID, MediaServerContactIcon,
MediaServerEnableTLS, ...).
Two identifiers had to be reassigned to make the boundary clean:
- certs.go held both the generic X509/TLS helpers and the media server's
process-global certificate. Split: the generic half stays in
internal/httpserver, generateMediaTLSCert and PublicMediaTLSCert move
to services/media.
- HandlerPatternMap was declared in handlers.go but is a plain HTTP type
that server.go depends on; it moves to internal/httpserver.
The media server URL tests moved with the type, and reached three
unexported Server fields they could touch while everything shared a
package. internal/httpserver now exposes ListeningAddr() and CachedPort()
(both reasonable API) plus a clearly-marked SetURLStateForTest.
Nothing else crossed the boundary, and the split is one-directional:
services/media and services/pairing import internal/httpserver, never the
reverse.
refs #7067
`common` was a grab-bag with no domain: the issue's own preamble names
it as the kind of package that must not exist. Every symbol moves to the
package that owns it, and the directory is deleted.
common/dbsetup -> internal/db/dbsetup
common/devices.go -> internal/platform
common/pausable*.go -> internal/pausable
LogOnPanic -> internal/panics
TruncateWithDot(N) -> internal/logutils
RecoverKey, ValidateDisplayName, display-name errors -> protocol/common
IpfsGatewayURL -> internal/ipfs.GatewayURL
Archives/TorrentTorrentsRelativePath, MainnetEthereumNetworkURL -> params
StatusService -> pkg/backend/node
ErrBigIntSetFromString -> services/wallet
IsNil, Ptr -> inlined at their call sites
IsENSName -> deleted, it had no callers
Notes:
- LogOnPanic gets its own package rather than living in logutils. It
reports to Sentry, and logutils is imported by nearly everything: put
the guard in logutils and the Sentry SDK lands in every dependency
graph in the tree (213 -> 250 packages). internal/panics imports
logutils and sentry, which is the direction root `common` had.
- TruncateWithDot is log redaction, not string formatting: every one of
its 121 call sites is inside a log or error message, so it belongs
next to the logger.
- Moving RecoverKey and ValidateDisplayName into protocol/common removes
the common -> protocol layering inversion; all their callers were
already inside protocol/.
- Makefile lint-panics target follows LogOnPanic to its new path.
refs #7067
Part of the Go project layout migration. Pure move plus import-path
rewrite; the public API is unchanged.
The "emit signals without global functions" redesign is a behavioural
change that touches the C-binding callback and is deliberately left out
of this PR.
refs #7067
* test: extend ENS functional tests with visibility and verification flows
Adds TestEnsVisibility covering ENS name propagation to a contact via
sendContactUpdates and verification via the wakuext_ensVerified RPC, and
switches the router release test to release through the wallet router
(ENSRelease send type) instead of an on-chain registrar call.
Also fixes zero-padding of storage slots in sync_ens_registry.sh.
* test(functional): mark bug-blocked ENS flows as strict xfail
Automatic ENS verification of a contact never triggers (#7712) and the
wallet router sends the ENS release tx to the SNT token contract instead
of the registrar (#7714). Mark both flows xfail(strict=True) so they
start failing the suite once the bugs are fixed.
---------
Co-authored-by: Egor Rachkovskii <egorrachkovskii@status.im>
The test wrote a file and assumed its mtime therefore landed after
processStartTime. Measured, that assumption has almost no margin:
gap between processStartTime and the test running:
full package run ~0.5ms
single test via -test.run ~0.1ms (what gotestsum --rerun-fails does)
filesystem mtime lag behind time.Now(), same machine:
-0.2ms to -1.3ms, several writes sharing one mtime
So the file routinely stats as older than the process, rotateLogFileForNewSession
archives it as a previous session's log, and the Stat assertion fails. Because
test-unit runs with UNIT_TEST_RERUN_FAILS=true, a first failure is re-run in
isolation -- the configuration with the *smallest* margin -- so the rerun
confirms rather than clears it.
Stamping the timestamp mirrors what the sibling test already does for its
"previous session" file, and makes this one assert the session logic instead of
the clock. 0/15 failures on a full coverpkg run and 0/10 on the isolated rerun,
against consistent failures before.
The production path is unaffected: rotateLogFileForNewSession runs once at
startup, when the file it inspects is either absent or genuinely from an earlier
session with a much older mtime.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QmB4TDHDayGGK4Y6jyaV5R
Fixes#7620
Return ErrNoStorenodesReachable when a history query arrives before
StoreClient initialization, instead of dereferencing a nil client.
Publish the fully configured StoreClient under a mutex and synchronize
query and active-storenode reads to prevent a startup race.
Add a regression test covering StoreQuery before Waku Start.
Part of https://github.com/status-im/status-app/issues/21544
Run the missing community encryption-key recovery check after a short startup settle delay, then continue on the regular interval. Extract the iteration into
a reusable method.
Allow retry request timestamps to update after backward clock changes, preventing
persisted retry state from freezing key recovery. Add focused recovery and
persistence regressions.
Part of https://github.com/status-im/status-app/issues/21544
Use a non-blocking, quit-aware send for new hash ratchet key notifications.
Drop and warn when the subscriber queue is full, and skip notifications for
empty key payloads.
Add regressions for full notification queues and empty key payloads.
Part of https://github.com/status-im/status-app/issues/21544
Treat key requests received by non-control nodes as expected broadcasts and
ignore them with debug logging.
Rate-limit control-node key distribution to one request per community member per minute, cleaning expired entries in memory.
Add regressions for non-control-node handling and duplicate-request suppression.
Part of https://github.com/status-im/status-app/issues/21544
Skip encrypted channel key distribution when the requester is no longer a
channel member, logging the dropped request for diagnosis. Use isolated member
maps for community and channel key actions to prevent one action from mutating
another.
Add handler regressions for stale channel membership and action-map isolation.
Part of https://github.com/status-im/status-app/issues/21544
Detect when an encrypted community description cannot be decrypted because its
community-level key is missing, even when no channels are visible.
Track these requests with the existing retry persistence and send them with an
empty channel list so the control node returns the community key. Add coverage
for detection, backoff, recovery, and request payload construction.
Igor Sirotin