Commit Graph

48 Commits

Author SHA1 Message Date
Jakub Sokołowski 4b2f2c32ad
ci: separate buckets for mobile and desktop builds
Possible fix for slow upload speeds and failures caused by most probably
hitting per-bucket rate limits of DigitalOcean:

>- 500 total operations per second to any individual bucket.
>- 300 combined PUT, POST, COPY, DELETE, and LIST operations per second
>  to any individual Space. We may further limit LIST operations if
>  necessary under periods of high load.

https://docs.digitalocean.com/products/spaces/details/limits/#rate-limits

Depends on: https://github.com/status-im/status-jenkins-lib/pull/52

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2022-11-22 17:32:53 +01:00
Jakub Sokołowski ff9c678a1e
ci: extract git cleanup to separate script
It needs to be separate script, because once the submodules are broken
and the `variables.mk` file is not available from `nimbus-build-system`
then we cannot even call `make clean-git` successfully because it will
be caught by the `if` clause that checks for `variables.mk`.

Possible fix for issues with submodules not being updated in some
windows release builds.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2022-11-22 14:28:14 +01:00
Jakub Sokołowski 6fa84fb221 ci: add make update to for other platforms
This was added to fix some build issues to Linux:
https://github.com/status-im/status-desktop/pull/8233

But other 2 platforms also have had the cleanup stage change which could
have cause these submodules update issues.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2022-11-14 16:03:18 -05:00
Anthony Laibe 1ecb9f64d4 fix: ci build on master linux 2022-11-14 13:41:16 -05:00
Jakub Sokołowski 38f0973dc3
ci: use git clean instead of cleanWs() function
This is a continuation of attempts to fix provlems with Windows CI hosts
getting into a broken state due to `.git` directory becoming a file for
random submodules for unknown reason.

Instead of relying on Jenkins functionality of `cleanWs()` function
provided by the [Workspace Cleanup Plugin](https://plugins.jenkins.io/ws-cleanup)
we use Git which should be more reliable, and possibly also speed up the
initial checkout of the repo and submodules.

A nice side-effect is that the repo checkout on all builds after the first
one takes ~1 minute instead of ~5 minutes.

Previous issues:
- https://github.com/status-im/status-desktop/pull/7968
- https://github.com/status-im/status-desktop/pull/8046

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2022-11-06 20:20:54 +01:00
Jakub Sokołowski 4f60c31282
ci: use cleanup instead of always step in post steps
Simply appropriate.
https://www.jenkins.io/doc/book/pipeline/syntax/#post

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2022-10-26 13:45:48 +02:00
Jakub Sokołowski 8c457a2495
ci: abort old PR builds to make queue shorter
In most cases developers only care about the most recent version.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2022-09-21 12:18:15 +02:00
Jakub Sokołowski 83c066f517
ci: fix upload speeds on Windows
By using `s3cmd` through PowerShell instead of Git Bash we improve
upload speeds on Windows slightly, as described here:
https://github.com/status-im/infra-ci/issues/40

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2022-09-15 11:41:12 +02:00
Jonathan Rainville 7544bd6fb5
fix: change gateway and remove old code to decode the hash
Fixes:
- https://github.com/status-im/status-desktop/issues/6964
Depends on:
- https://github.com/status-im/status-go/pull/2795
- https://github.com/status-im/status-jenkins-lib/pull/44

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2022-08-26 19:47:02 +02:00
Richard Ramos ea44816071 chore: update go version in dockerfile 2022-08-03 09:40:47 -04:00
Jakub Sokołowski ce1436ac15 ci: remove caching of dependencies
Upgrade of Jenkins to `2.343` has introduced a security fix that breaks
caching plugin when it's configured to store cache on Master host:
https://issues.jenkins.io/browse/JENKINS-67173

Sine the [Caching plugin](https://plugins.jenkins.io/jobcacher/) hasn't
been upgraded in 5 years the only good temporary workaround is just drop
caching of dependencies like Nim compiler entirely.

In the future we can try some other caching methods.

Related: https://github.com/status-im/nimbus-eth2/pull/3594

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2022-04-15 12:45:27 +02:00
Richard Ramos 3dfc8d3a79 fix: code review 2022-03-10 14:12:11 -04:00
Anthony Laibe 03b4bb0511 feat(@wallet): Add opensea api key 2022-02-16 10:29:00 +01:00
Jakub Sokołowski 2689a08ebb ci: add VERBOSE parameter to change build verbosity
This depends on the `V` paramerter in `nimbus-build-system`:
https://github.com/status-im/nimbus-build-system/blob/25a4c270/README.md#L54-L61

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2022-02-09 13:05:32 +01:00
Jakub Sokołowski 0d04deb2ad ci: drop unused BUILD_TYPE, add useful RELEASE param
It appears this was some leftover from ancient times and wasn't being used.

On the other hand the `RELEASE` environment variable controls if Nim builds
of the client binary include debug symbols:
ba7a6d5d34/Makefile (L177-L184)

The shorthand `?:` symbol means that if this is changed for a given job it stays changed.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2022-02-09 13:05:32 +01:00
Patryk Osmaczko ee462ea178 fix(@desktop/input): deploy Qt plugin to support fcitx
Qt provides support for various input methods through plugins.
Since the fcitx plugin is not delivered with Qt, it needs to be
built and deployed by us.

Fixes: #4436
2022-01-26 18:11:39 +01:00
Jakub Sokołowski 977293c1ce ci: manually fix expired LetsEncrypt X3 root cert
https://www.openssl.org/blog/blog/2021/09/13/LetsEncryptRootCertExpire/
https://blog.dnsimple.com/2021/09/letsencrypt-intermediate-expiration/
https://scotthelme.co.uk/lets-encrypt-old-root-expiration/

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2021-11-22 14:26:22 +01:00
Jakub Sokołowski 8b19ac0a64 ci: bump Linux and MacOS timeouts to 20 minutes
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2021-11-22 12:22:46 +01:00
Jakub Sokołowski c4c0484e61 ci: upgrade ca-certificates in Linux Docker image
Otherwise cloning of BearSSL fails with:
```
Cloning into '/home/jenkins/workspace/status-desktop/platforms/linux/vendor/status-lib/vendor/nim-task-runner/vendor/nim-bearssl/bearssl/csources'...
fatal: unable to access 'https://www.bearssl.org/git/BearSSL/': server certificate verification failed. CAfile: /etc/ssl/certs/ca-certificates.crt CRLfile: none
fatal: clone of 'https://www.bearssl.org/git/BearSSL' into submodule path '/home/jenkins/workspace/status-desktop/platforms/linux/vendor/status-lib/vendor/nim-task-runner/vendor/nim-bearssl/bearssl/csources' failed
Failed to clone 'bearssl/csources'. Retry scheduled
```
https://blog.dnsimple.com/2021/09/letsencrypt-intermediate-expiration/

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2021-11-22 12:22:46 +01:00
Richard Ramos fe6c5a455a fix: jenkinsfile docker image version 2021-10-19 23:05:44 +02:00
Michele Balistreri a22d936df7 update Jenkinsfile 2021-10-06 08:32:32 -04:00
Jakub Sokołowski 145652d2d0 ci: upgrade library and use of pkgFilename()
Depends on: https://github.com/status-im/status-jenkins-lib/pull/36

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2021-09-29 10:19:15 +02:00
Jakub Sokołowski 98f2a68209
ci: fix Jenkinsfile library enties to use tags instead of a branch
Again we merged a PR before the `status-jenkins-lib` PR was merged:
https://github.com/status-im/status-desktop/pull/3185

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2021-08-19 15:16:54 +02:00
Jakub Sokołowski 2df6def7f9 ci: add scripts/sign-linux-tarball.sh for GPG signing
Adds `scripts/sign-linux-file.sh` which expectes the following variables set:

* `LINUX_GPG_PRIVATE_KEY_FILE` - Path to the GPG export of private key.
* `LINUX_GPG_PRIVATE_KEY_PASS` - Password necessary to use the private key.

Given a file it creates a file with a `.asc` suffix containing the signature:
```
 > wget -q https://status-im-prs.ams3.digitaloceanspaces.com/StatusIm-210809-104514-156806-pr.tar.gz

 > tar xvf StatusIm-210809-104514-156806-pr.tar.gz
StatusIm-210809-104514-156806-pr.AppImage
StatusIm-210809-104514-156806-pr.AppImage.asc

 > gpg --verify StatusIm-210809-104514-156806-pr.AppImage.asc
gpg: assuming signed data in 'StatusIm-210809-104514-156806-pr.AppImage'
gpg: Signature made Mon 09 Aug 2021 12:54:49 PM CEST using RSA key ID E20B4DFD
gpg: Good signature from "Status.im Devel Signing (GPG key for signing Status.im development builds.) <devel@status.im>" [ultimate]
Primary key fingerprint: BBF0 5F92 536B ED19 30A9  FD44 009F B3BF E20B 4DFD
```

Issue: https://github.com/status-im/infra-ci/issues/25
Requires: https://github.com/status-im/status-jenkins-lib/pull/32

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2021-08-16 11:21:36 -04:00
Jakub Sokołowski d6266384b7 ci: fix JSON parsing in release process
Fixes:
```
Caused: java.io.NotSerializableException: groovy.json.internal.LazyMap
```
https://ci.status.im/job/status-desktop/job/release/job/release%252F0.1.0-beta.11/2/

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2021-07-28 10:15:08 +02:00
B.Melnik 0def782841 fix: wrong tag to jenkins lib and make parameter 2021-07-20 10:22:21 -04:00
Michael Bradley, Jr a564b48f61 build: on Linux use tar+gzip to preserve +x attribute of .AppImage
We create a separate make target to avoid tarballing every time.
This is really useful only for releases we upload for users.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2021-06-07 14:26:28 -04:00
Jakub Sokołowski 6a5b6e3593 ci: do no delete non-draft releases
We identified this issue in status-desktop release from branch
`release/0.1.0-beta.10` which had old version in VERSION file.
https://ci.status.im/job/status-desktop/job/release/job/release%252F0.1.0-beta.10/

Depends on: https://github.com/status-im/status-jenkins-lib/pull/28

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2021-06-07 10:18:01 -04:00
Jakub Sokołowski 089ee7ee80 ci: scripts/sign-windows-bin.sh for signing Windows binaries
This introduces the `scripts/sign-windows-bin.sh` script which is used
by the `Makefile` to sign application libraries and executables. It also
implements the logic necessary to distinguish between different types of
builds: release and non-release builds.

Some other changes:

* Refactore the `Makefile` target that creates the Windows ZIP to make less verbose.
* Added `Microsoft.VisualStudio.Component.Windows10SDK.10240` to VisualStudio component
* Added `BUILD_TYPE` parameter to `Jenkinsfile`s for different platform builds

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2021-04-28 18:19:17 +02:00
Michael Bradley, Jr 37c51cc25f build: use same container as Jenkins CI for local packaged builds on Linux
Make the docker-related logic for doing a local packaged build on Linux match
with the Jenkins CI build for Linux.

Also fix the problems with packaged builds for Linux (local or CI) where
gstreamer errors were displayed in the terminal shortly after app start/login
and there was no sound output.
2021-04-27 13:49:46 -05:00
Jakub Sokołowski a308851f10 ci: bump jenkins lib to make release filenames nicer
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2021-04-22 12:01:42 -04:00
Jakub Sokołowski 655097e2a2 ci: add ability to create draft GitHub releases
This adds a `Publish` stage to the combined `Jenkinsfile` which when the
`PUBLISH` parameters is true pushes a __draft__ release to GitHub.
The job remembers the last value of `PUBLISH` parameter selected.

The release uses the contents of the `VERSION` file at the root of the
repo as the name for the release, and leaves the contents to be filled
in by whoever will approve the release.

The automation overwrites - or to be exact, deletes and recreates - the
release, so releated builds for the same release will simply re-create
it. All the built artifacts are included in the release.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2021-04-20 12:53:43 -04:00
Jakub Sokołowski 0d136d8879 ci: add a combined Jenkinsfile for releases
This is part of a general restructuring in layout of Jenkins job folders
before we can properly introduce Release signing and notarization.
We need this to distinguish between pr/dev builds and release ones to
avoid signing the dev builds with a release certificate.

The meta job managed with `ci/Jenkinsfile.combined` runs a job for all 3
platforms and currently is quite basic, but in the future can be
extended to include - like the mobile one - updating the nightlies page,
or publishing draft GitHub releases.

The addition of `make check-pkg-target-*` steps to other `Jenkinsfile`s
is necessary because the sub-jobs under `platforms` have no option for
checking out Git submodules at the beginning, so I'm making use of how
the `Makefile` works and triggering that with a target that doesn't do much.

Example job:
https://ci.status.im/job/status-desktop/job/release/job/ci-meta-release-job/

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2021-04-14 15:36:00 -04:00
Jakub Sokołowski 06f24e90a2 ci: upgrade status-react-jenkins to 1.2.11
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2021-04-14 15:36:00 -04:00
Jakub Sokołowski cc6bf79d8c ci: lower number of artifacts kept to 3
Desktop builds are taking up a ridiculous amount of space.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2021-03-26 09:49:10 +01:00
Michael Bradley, Jr 10d82124fe refactor: embed Infura key at compile time
Allow environmental override at runtime. Also, in the Makefile set a free-tier
default token so that setting up an Infura account isn't strictly necessary for
community contributors to build the app, even though in our docs it should be
recommended they do so.

Core contributors should setup their own free-tier Infura account, create a
key, and set it in the environment variable INFURA_TOKEN in their environment
used to build the desktop app locally.

There is one aspect of this work that is incomplete. Ideally, in the handler
for the `login` event the relevant settings in the database should always be
updated with the resolved Infura key. However, when calling
`getSetting[string](Setting.Networks_Networks)` in the handler it causes a
segfault every time. Neither the reason for the crash nor a workaround have
been worked out at this time.
2020-12-09 12:10:12 -06:00
Jakub Sokołowski 6b3f9a31c2 lower number of artifacts to keep in CI to 10
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2020-11-24 21:00:52 +01:00
Richard Ramos cd8f49921f Update docker linux version 2020-11-06 16:22:07 -05:00
Jakub Sokołowski 4f81032f5e ci: provide INFURA_TOKEN to builds
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2020-11-05 11:19:13 -05:00
Jakub Sokołowski a88aea4b11 ci: update Docker image to include QtWebEngine
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2020-10-07 16:11:58 +02:00
Jakub Sokołowski ef4c84afe1 ci: drop running make update, CI already fetches submodules
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2020-09-29 15:11:27 +02:00
Jakub Sokołowski 8717a86ff8 ci: upgrade jenkins library to v1.2.4
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2020-09-29 15:11:27 +02:00
Jakub Sokołowski b169783847 ci: drop ineffective V env variable
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2020-09-29 15:11:27 +02:00
Jakub Sokołowski 339afac117 ci: add result URL to build description
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2020-09-29 15:11:27 +02:00
Jakub Sokołowski 7d16216c3b add support for posting PR builds in GitHub comments
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2020-07-13 12:15:02 +02:00
Jakub Sokołowski 3a6655f8f3
ci: bump timeouts from 10 to 15 minutes
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2020-07-02 18:35:24 +02:00
Jakub Sokołowski 6683313c83 fix QTDIR paths for CI
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2020-07-02 10:48:32 +02:00
Jakub Sokołowski 85a3557f1e add Jenkinsfiles and Dockerfile for CI
Changes:
- Adds `ci/Dockerfile` for creating `statusteam/nim-status-client-build:latest` used in builds
- Adds `ci/Jenkinsfile.linux` and `ci/Jenkinsfile.macos` for respective platforms
- Simplifies MacOS signing by adding `scripts/sign-macos-pkg.sh` script
- Makes `Makefile` use `scripts/sign-macos-pkg.sh` to make the DMG
- Makes `APPIMAGE` and `DMG` in `Makefile` modifiable by environment
- Adds `--passL:"-headerpad_max_install_names"` to `NIM_PARAMS` to fix MacOS signing issues

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2020-07-01 22:06:37 +02:00