After a LOT of work from @jrainville bisecting the release branch he
identified the issue to be this commit:
https://github.com/status-im/status-desktop/commit/dad8e453
ci: bump client build retries to 20
Which actually was poorly squashed by @iurimatias and contains the
commit that also adds the retries in the first place:
https://github.com/status-im/status-desktop/commit/a5ba2537
ci: retry nim_status_client build 3 times
The reason why that's an issue is because normally it was caled by `make
tgz-linux` target which included this setting:
```
override RESOURCES_LAYOUT := $(PRODUCTION_PARAMETERS)
```
Which caused `NIM_PARAMS` to use `-d:development` instead of `-d:production`.
The result was the app trying to write config file changes to the read-only
filesystem of the AppImage.
Resolves: https://github.com/status-im/status-desktop/issues/11295
Signed-off-by: Jakub Sokołowski <jakub@status.im>
For some reason the an infra change has triggered the behavior of `make`
where the `if` clauses that compares `QT5_PKGCONFIG_INSTALL_PREFIX` with
`QT5_INSTALL_PREFIX` would be true, which would cause `QT5_LIBDIR` to
not be exported, and some `NIM_PARAMS` to not be set at all.
Resolves: https://github.com/status-im/status-desktop/issues/10876
Signed-off-by: Jakub Sokołowski <jakub@status.im>
Avoid using QTDIR env and hardcoded subdirectories.
Use 'qmake' to detect paths to lib, qml, etc.
It fixes build on hosts with multiple QT installations, where
one of them is installed system-wide.
It needs to be separate script, because once the submodules are broken
and the `variables.mk` file is not available from `nimbus-build-system`
then we cannot even call `make clean-git` successfully because it will
be caught by the `if` clause that checks for `variables.mk`.
Possible fix for issues with submodules not being updated in some
windows release builds.
Signed-off-by: Jakub Sokołowski <jakub@status.im>
This avoids a situation in which `variables.mk` from
`nimbus-build-system` does't exist, but at the same time `git submodule
update` doesn't work due to sumodules stuck in a bad state.
Signed-off-by: Jakub Sokołowski <jakub@status.im>
This way we avoid errors about missing QT when calling targets that do
not require it, like `make clean` or `make deps`.
Signed-off-by: Jakub Sokołowski <jakub@status.im>
This is a continuation of attempts to fix provlems with Windows CI hosts
getting into a broken state due to `.git` directory becoming a file for
random submodules for unknown reason.
Instead of relying on Jenkins functionality of `cleanWs()` function
provided by the [Workspace Cleanup Plugin](https://plugins.jenkins.io/ws-cleanup)
we use Git which should be more reliable, and possibly also speed up the
initial checkout of the repo and submodules.
A nice side-effect is that the repo checkout on all builds after the first
one takes ~1 minute instead of ~5 minutes.
Previous issues:
- https://github.com/status-im/status-desktop/pull/7968
- https://github.com/status-im/status-desktop/pull/8046
Signed-off-by: Jakub Sokołowski <jakub@status.im>
Keycard implementation affected onboarding/login flows.
- new user - first run - new keys into keycard
- new user - first run - import seed phrase into keycard
- old user - first run - login importing from keycard
- login the app using keycard
Fixes: #5972
Otherwise `make pkg` would deploy already built status client,
regardless of its initial configuration.
`make clean` is not required before `make pkg` anymore.
fixes: #6241
Logs are named app_yyyyMMdd_HHmmss.log
Version, commit and start date information are displayed in logs at the begining.
Injecting commit version from Makefile.
Moving DESKTOP_VERSION const to constants.nim
Fix#3611
Qt provides support for various input methods through plugins.
Since the fcitx plugin is not delivered with Qt, it needs to be
built and deployed by us.
Fixes: #4436
These changes are a follow-up to #3582.
Also support `REBUILD_NIM=true` and `REBUILD_UI=true` variables so that it's
possible to ensure a rebuild for the next invocation of `make` or `make run`.
Dev docs have been updated: https://hackmd.io/OtFjDnH6QtOl3K65fKKEGg
Also adjust run targets so that e.g. `make -j16 run` works correctly.
Closes#3639.
With these changes it will be easier to maintain, i.e. to add/remove bottles
just modify the `BOTTLES :=` list.
`brew update` is removed from `scripts/fetch-brew-bottle.sh` and instead done
in an [order-only
prerequisite](https://www.gnu.org/software/make/manual/html_node/Prerequisite-Types.html). This
allows multiple bottles to be fetched in parallel (e.g. `make -j16`) without
overlapping invocations of `brew update` (which causes script failure).
This feature works for MacOs only, for now.
On login, whether new or already created user may select between options:
"Store" - store password to the Keychain
"Not now" - don't store it now, but ask next time again
"Never" - don't store them ever and don't ask again
Selected preference may be changed later in:
`ProfileSettings > Privacy and security > Store pass to Keychain`
On the next app run, if `Store` was selected, a user will be asked to confirm
his identity using Touch Id in order to log in the app. If any error happens
he will be able to login using password.
Fixes: #2675
Adds `scripts/sign-linux-file.sh` which expectes the following variables set:
* `LINUX_GPG_PRIVATE_KEY_FILE` - Path to the GPG export of private key.
* `LINUX_GPG_PRIVATE_KEY_PASS` - Password necessary to use the private key.
Given a file it creates a file with a `.asc` suffix containing the signature:
```
> wget -q https://status-im-prs.ams3.digitaloceanspaces.com/StatusIm-210809-104514-156806-pr.tar.gz
> tar xvf StatusIm-210809-104514-156806-pr.tar.gz
StatusIm-210809-104514-156806-pr.AppImage
StatusIm-210809-104514-156806-pr.AppImage.asc
> gpg --verify StatusIm-210809-104514-156806-pr.AppImage.asc
gpg: assuming signed data in 'StatusIm-210809-104514-156806-pr.AppImage'
gpg: Signature made Mon 09 Aug 2021 12:54:49 PM CEST using RSA key ID E20B4DFD
gpg: Good signature from "Status.im Devel Signing (GPG key for signing Status.im development builds.) <devel@status.im>" [ultimate]
Primary key fingerprint: BBF0 5F92 536B ED19 30A9 FD44 009F B3BF E20B 4DFD
```
Issue: https://github.com/status-im/infra-ci/issues/25
Requires: https://github.com/status-im/status-jenkins-lib/pull/32
Signed-off-by: Jakub Sokołowski <jakub@status.im>
We create a separate make target to avoid tarballing every time.
This is really useful only for releases we upload for users.
Signed-off-by: Jakub Sokołowski <jakub@status.im>
This introduces an automated MacOS notarization process for Jenkins CI.
The process involves:
* Uploading the signed DMG file to the notary service
* Checking periodically if the scanning process has completed
* Stapling the successful scan ticket to the DMG file
This is done by the `scripts/notarize-macos-pkg.sh` via the `make notarize-macos` target.
The whole process is described in more details in `docs/macos_notarization.md`.
Depends on: https://github.com/status-im/status-jenkins-lib/pull/27
Resolves: https://github.com/status-im/status-desktop/issues/2169
Signed-off-by: Jakub Sokołowski <jakub@status.im>
Fix a typo in a recently added header file in DOtherSide that causes problems
on case-sensitive filesystems, e.g. on Linux.
Also unify the build command for DOtherSide across platforms.
This introduces the `scripts/sign-windows-bin.sh` script which is used
by the `Makefile` to sign application libraries and executables. It also
implements the logic necessary to distinguish between different types of
builds: release and non-release builds.
Some other changes:
* Refactore the `Makefile` target that creates the Windows ZIP to make less verbose.
* Added `Microsoft.VisualStudio.Component.Windows10SDK.10240` to VisualStudio component
* Added `BUILD_TYPE` parameter to `Jenkinsfile`s for different platform builds
Signed-off-by: Jakub Sokołowski <jakub@status.im>
Make the docker-related logic for doing a local packaged build on Linux match
with the Jenkins CI build for Linux.
Also fix the problems with packaged builds for Linux (local or CI) where
gstreamer errors were displayed in the terminal shortly after app start/login
and there was no sound output.
In the repo:
```
$ bin/nim_status_client --help
```
In the packaged app (macOS example):
```
$ cd /Applications/Status.app/Contents/MacOS
$ ./nim_status_client --help
```
Output:
```
Usage:
nim_status_client [OPTIONS]...
The following options are available:
-d, --dataDir Status Desktop data directory.
```
**Using the option**
```
$ cd ~/status-ci-builds/master/Status.app/Contents/MacOS
$ ./nim_status_client --dataDir:"${HOME}/status-dirs/master"
```
In another terminal:
```
$ cd ~/status-ci-builds/PR-4242/Status.app/Contents/MacOS
$ ./nim_status_client --dataDir:"${HOME}/status-dirs/PR-4242"
```
The path supplied can be relative or absolute, and can be specified with
`--dataDir:[path]`, `--dataDir=[path]`, `-d:[path]`, or `-d=[path]`.
Either `:` or `=` must be used, i.e. this *will not* work: `--dataDir [path]`
or `-d [path]`.
The name of the option follows Nim's partial case-insensitivity rules, so
`--dataDir`, `--datadir`, and `--data_dir` are all equivalent. See
[Identifier equality][ieq] in the Nim Manual.
It is possible to run the same build in multiple terminals by supplying
different `--dataDir`, i.e. this works:
```
$ cd /Applications/Status.app/Contents/MacOS
$ ./nim_status_client --dataDir="${HOME}/temp/some1"
```
In another terminal:
```
$ cd /Applications/Status.app/Contents/MacOS
$ ./nim_status_client --dataDir="${HOME}/temp/some2"
```
**Windows**
It is recommended to use a Git Bash or MSYS2 terminal when invoking
`bin/nim_status_client.exe` (development build) or `bin/Status.exe` (production
build) on the command-line. The reason is that if the exe is invoked in a
session of `cmd.exe` it will return to the prompt immediately; the app will run
but there will be no output in the terminal. In any case, the `--dataDir`
option will take effect whether the exe is invoked in `cmd.exe` or a
recommended terminal.
For development builds, when invoking `bin/nim_status_client.exe` directly
instead of via `make run`, because e.g. you wish to use the `--dataDir` option,
it is required to first setup the `PATH` environment variable correctly. See
the `run-windows` target in this repo's Makefile for more information.
**Linux**
The `--dataDir` option may be passed to command-line invocation of a
production (AppImage) build in the same way as passing it to a development
build:
```
$ Status.AppImage --dataDir:/path/to/wherever
```
For development builds, when invoking `bin/nim_status_client` directly instead
of via `make run`, because e.g. you wish to use the `--dataDir` option, it is
required to setup the `LD_LIBRARY_PATH` environment variable correctly. See the
`run-linux` target in this repo's Makefile for more information.
---
BREAKING CHANGE: The `qt` subdir of the app's data directory is now a sibling
of the status-go directory rather than a subdir of the status-go directory:
```
Status (app data directory)
├── data (status-go directory)
├── qt
└── tmp
```
Because app settings are stored in the `qt` directory that means that existing
installations will lose their customized settings.
At app startup, it would be possible to detect `Status/data/qt` and if
`Status/qt` doesn't exist yet then copy `Status/data/qt` to
`Status/qt`. However, there was some concern that behavior could lead to
problems later on if we forget the workaround is in place. So for now that
settings preservation strategy has not been implemented, but it might be before
this commit is merged pending full team awareness/consensus.
---
Command-line option support is provided by
[nim-confutils](https://github.com/status-im/nim-confutils).
The environment variable `NIM_STATUS_CLIENT_DEV` has been removed in favor of
passing a "define" option to the Nim compiler: `-d:development` for development
builds (e.g. `make V=1`) and `-d:production` for packaged builds (e.g. `make
V=1 pkg`). Passing the correct option is handled automatically by the Makefile.
A make variable named `RELEASE` has been introduced, which defaults to
`false`. Presently the `RELEASE` variable should not be set on the command-line
nor in CI as more work needs to be done to toggle the proper compiler flags. In
the case of Status Desktop, "release vs. debug" is a concern orthogonal to
"production vs. development". At present, production builds and development
builds are all debug builds, but that will likely change in the future: we can
have non-release CI production builds and local development builds be debug
builds, while release builds in CI would be production builds with
`RELEASE=true` (the compiled executable will be fully optimized).
Prior to the changes in this PR, symmetry is somewhat lacking between
development and production (packaged) builds with respect to the concept of the
"data directory". In development builds the root of the repo effectively serves
as the `Status` directory used by production builds, e.g. on macOS
`~/Library/Application Support/Status`. Also, there's a bit of confusion as to
whether "data directory" refers to a directory for the desktop app's overall
data (including status-go data) or to the specific directory used by status-go.
This PR attempts to provide symmetry and reduce confusion:
* The term "data directory" means the directory used by the desktop app to
store multiple kinds of data and is not a reference to the subdirectory used by
status-go.
* For development builds the "data directory" defaults to `./Status/` relative
to the root of the repo.
* For production builds the "data directory" default is the same as before,
e.g. on macOS it's ` ~/Library/Application Support/Status/`.
The directory used by status-go is `Status/data/`. To be clear, that should be
referred to as the "status-go directory" and not the app's "data directory". It
would nice if we could rename it from `Status/data/` to `Status/status-go/`. We
can do that, I already checked that it works correctly; however, for existing
installations it would require that at app launch we check for the presence of
`Status/data/` and rename it to `Status/status-go`. While simple enough to do,
I was concerned that there might be edge cases where the directory rename could
cause a problem (e.g. if another copy of the app is running) so chose for now
to stick with the status-go directory being `Status/data/`.
---
**NOTES**
More work needs to be done to ensure that all data written by the app is
contained in the default or cli-specified data directory. Currently, both
development and production (packaged) builds are writing to common directories
outside of the data directory, e.g. located within `~/Library/` on
macOS. Changing that behavior seems like it will mainly involve changing
defaults related to Qt components such as the web engine. See:
https://github.com/status-im/status-desktop/issues/1141.
In general, additional refactoring could be done in the future. For
example, implementing `StatusDesktopConfig` in
`src/status/libstatus/accounts/constants.nim` (as done in this PR) works fine
for now, but better code organization is desirable.
---
Closes#2268
[ieq]: https://nim-lang.org/docs/manual.html#lexical-analysis-identifier-equality
It's not supported to run `brew update` multiple times simultaenously. A better
fix for this can be implemented, but for now in the Makefile have target
`bottles/pcre` depend on target `bottles/openssl` so that they run serially
when `-jN` is passed to `make`.
I was originally going to change `brew update >/dev/null` to
`brew update >/dev/null || true` but decided the Makefile change provides a
better guarantee that simultaenous `brew` commands won't interfere with each
other. In the process, I revised some of the script's `echo` output just a bit.
The way it was done previously didn't always produce the intended result (for
unknown reasons), i.e. the dev app would sometimes launch with a transparent
blank square appearing in place of the expected icon.
By default the desktop app uses port 30305, unless a value is specifically set
in environment variable `STATUS_PORT`. For convenience of those developing the
app and running development builds, while simultaneously running production
builds (a.k.a. packaged builds, whether built locally or in CI), have `make
run` invoke `bin/nim_status_client` with `STATUS_PORT=30306`. That value can
still be overriden by manually invoking `make run` with a different value,
e.g. `make STATUS_PORT=30307 run` and `STATUS_PORT=30307 make run` are both
valid and achieve the same effect. NOTE: the port "sticks" in the database
after the first clean run, so when changing ports developers will need to
delete the data dir within their local repository, else the existing database
for dev builds will be stuck using whatever port was used previously. In the
future, we can figure out a means to always update the port setting in the
database just after the login event (but see #1505).
Also, for development builds use an icon (for the running app) that is
orangered (`#ff4500`) instead of the official blue color. This makes it much
easier to select between a running production instance and development instance
when Cmd-Tab'ing (on macOS, or equiv on Linux and Windows) through open
applications. Not all icons displayed at runtime have been changed in this
manner for development builds, just the main application icon, and that seems
to be sufficient to achieve the desired effect; though in the future we could
do similar for notification icon, menu bar icon, etc.
In the context of Windows packaged builds, this is consistent with other
resources needed by the application; organizing them in this way keeps the
top-level directory of Windows packaged builds clear of files that most users
should never have to see or think about.
Allow environmental override at runtime. Also, in the Makefile set a free-tier
default token so that setting up an Infura account isn't strictly necessary for
community contributors to build the app, even though in our docs it should be
recommended they do so.
Core contributors should setup their own free-tier Infura account, create a
key, and set it in the environment variable INFURA_TOKEN in their environment
used to build the desktop app locally.
There is one aspect of this work that is incomplete. Ideally, in the handler
for the `login` event the relevant settings in the database should always be
updated with the resolved Infura key. However, when calling
`getSetting[string](Setting.Networks_Networks)` in the handler it causes a
segfault every time. Neither the reason for the crash nor a workaround have
been worked out at this time.
Jakub Sokołowski
Jonathan Rainville
Patryk Osmaczko
Alex Jbanca
Igor Sirotin
Anthony Laibe
Michał Cieślak
Ivan Belyakov
Boris Melnik
Richard Ramos
Stefan
Sale Djenic
Michal Iskierko
Michele Balistreri
Michael Bradley, Jr
Andrei Smirnov
B.Melnik
Jacek Sieka