Otherwise we can end up with Gradle failing to find the dependencies
because we've patched away all entries referencing external repos.
Also made the regex in AWS parser a but more strict.
Signed-off-by: Jakub Sokołowski <jakub@status.im>
Cocoapod installations can fail with errors like:
```
tclsh /Users/jenkins/Library/Caches/CocoaPods/Pods/Release/SQLCipher/3.4.2-f9fcf/tool/addopcodes.tcl parse.h.temp >parse.h
./configure: line 11729: tclsh: command not found
./configure: line 12262: tclsh: command not found
./configure: line 12276: tclsh: command not found
```
If a pure shell is used. Also `pgrep` would be missing from a pure shell.
Signed-off-by: Jakub Sokołowski <jakub@status.im>
Before we were patching only `build.gradle`, which is not the only type
of Gradle config file. If we do not cover them all we can encounter
errors about missing package because they will continue using remote
repositories instead of `mavenLocal()`, to which pass Nix store path.
We also need to cover `gradlePluginPortal()` to provide plugins.
This is also necessary for the React Native upgrade:
https://github.com/status-im/status-mobile/pull/15203
Signed-off-by: Jakub Sokołowski <jakub@status.im>
Refactoring the derivation that fetches all the POMs, JARs,
and AARs in order to make it more generic and easier to extend.
The main change is adding `files` key in `deps.json` which contains
a dict of all the files reletad to given package.
This way we can more easily include other files that might be available
for download, like AARs with ASC suffix, or `nodeps` JARs.
This is also necessary for the React Native upgrade:
https://github.com/status-im/status-mobile/pull/15203
Signed-off-by: Jakub Sokołowski <jakub@status.im>
As discovered in:
https://github.com/status-im/status-mobile/pull/15225
The attempt to fix this in:
https://github.com/status-im/status-mobile/pull/15180
But it doesn't appear to work, so instead I'm allowing an override of
`NODE_BINARY` variable and spetting it when defining the Nix shell.
The key things here are:
* Xcode injects its own paths into `PATH` which breaks Nix env.
* Combining Nix shells with `inputsFrom` does not inherit all vars.
It's important to set these variables in `shellHook` and not elsewhere.
Signed-off-by: Jakub Sokołowski <jakub@status.im>
* nix: update gradle dependencies
Signed-off-by: Jakub Sokołowski <jakub@status.im>
* nix: include nodeps JARs in Gradle deps
Signed-off-by: Jakub Sokołowski <jakub@status.im>
* nix: include nodeps JAR for semver4j 0.16.4
Can cause failures like in some cases:
```
A problem occurred configuring project ':react-native-hole-view'.
> Could not resolve all files for configuration ':react-native-hole-view:classpath'.
> Could not find semver4j-0.16.4-nodeps.jar (com.github.gundy:semver4j:0.16.4).
Searched in the following locations:
file:/nix/store/3n2pxsqa2izlx8c23s6jgqai0bqaklm1-status-mobile-maven-deps/com/github/gundy/semver4j/0.16.4/semver4j-0.16.4-nodeps.jar
```
Signed-off-by: Jakub Sokołowski <jakub@status.im>
---------
Signed-off-by: Jakub Sokołowski <jakub@status.im>
Fixes issue with following failure:
```
Failed to construct transformer: Error: error:0308010C:digital envelope routines::unsupported
at new Hash (node:internal/crypto/hash:71:19)
at Object.createHash (node:crypto:130:10)
at stableHash (/Users/jenkins/workspace/status-mobile/platforms/ios/node_modules/metro-cache/src/stableHash.js:19:8)
at JsTransformer.getCacheKey (/Users/jenkins/workspace/status-mobile/platforms/ios/node_modules/metro/src/JSTransformer/worker.js:471:7)
at getTransformCacheKey (/Users/jenkins/workspace/status-mobile/platforms/ios/node_modules/metro/src/DeltaBundler/Transformer/getTransformCacheKey.js:39:29)
at new Transformer (/Users/jenkins/workspace/status-mobile/platforms/ios/node_modules/metro/src/DeltaBundler/Transformer.js:147:28)
at /Users/jenkins/workspace/status-mobile/platforms/ios/node_modules/metro/src/Bundler.js:54:29
at runMicrotasks (<anonymous>)
at processTicksAndRejections (node:internal/process/task_queues:96:5) {
opensslErrorStack: [ 'error:03000086:digital envelope routines::initialization error' ],
library: 'digital envelope routines',
reason: 'unsupported',
code: 'ERR_OSSL_EVP_UNSUPPORTED'
```
https://roytuts.com/how-to-fix-err_ossl_evp_unsupported-in-react-js-application/
Signed-off-by: Jakub Sokołowski <jakub@status.im>
* nix: upgrade zprint from 1.2.4 to 1.2.5
To address issue described in:
https://github.com/kkinnear/zprint/issues/273
Signed-off-by: Jakub Sokołowski <jakub@status.im>
* chore: use zprint :multi-lhs-hang
* refactor: re-format clojure using zprint 1.2.5
---------
Signed-off-by: Jakub Sokołowski <jakub@status.im>
Co-authored-by: yqrashawn <namy.19@gmail.com>
On M1 calling `shadow-cljs` fails with:
```
Execution error (UnsatisfiedLinkError) at java.lang.ClassLoader$NativeLibrary/load (ClassLoader.java:-2).
/private/var/folders/__/x311ykg17rqgq2wyl4kn1pdr0001yh/T/jna8753030888504535661.tmp:
dlopen(/private/var/folders/__/x311ykg17rqgq2wyl4kn1pdr0001yh/T/jna8753030888504535661.tmp, 0x0001):
tried: '/private/var/folders/__/x311ykg17rqgq2wyl4kn1pdr0001yh/T/jna8753030888504535661.tmp'
(fat file, but missing compatible architecture (have (unknown,i386,x86_64), need (arm64e)))
```
This is due to an outdeted dependency on JNA 3.2.2, which is pulled in
by `hawk` package which up until release `2.11.16` was a `shadow-clj`
dependency which was removed because it was:
>Only used to be used on macOS since it was slightly faster than the default
>JVM implementation. However in Big Sur it seems to cause issues and break
>completely or just be a lot slower.
https://github.com/thheller/shadow-cljs/commit/f3b89b5a
Dropped the explicit dependency on `org.clojure/core.async` to avoid:
```
WARNING: The org.clojure/core.async dependency in shadow-cljs.edn was ignored.
Default version is used and override is not allowed to ensure compatibility.
```
Resolves: https://github.com/status-im/status-mobile/issues/14196
Signed-off-by: Jakub Sokołowski <jakub@status.im>
Notable upgrades:
* Go `1.17.11` to `1.18.6`
* NodeJS `16.15.0` to `16.17.1`
* Clojure `1.11.1.1139` to `1.11.1.1165`
* Ruby Gem `3.2.26` to `3.3.20`
* Bundler `2.3.9` to `2.3.22`
* Git `2.36.1` to `2.37.3`
* Curl `7.83.1` to `7.85.0`
* OpenSSL `1.1.1o` to `3.0.5`
* PatchELF `0.14.5` to `0.15.0`
* Android SDK Platform Tools `33.0.1` to `33.0.2`
Signed-off-by: Jakub Sokołowski <jakub@status.im>
This resolves issues with weird `status-go` build errors like:
```
cgo: C compiler "2022-09-07" not found: exec: "2022-09-07": executable file not found in $PATH
```
Such errors are caused by `xcrun` spewing warnings to `stdout`. The PR
to fix this issue permanently has been created but it might take a while
before it's merged.
Fix PR: https://github.com/golang/mobile/pull/84
Resolves: https://github.com/status-im/status-mobile/issues/13949
Signed-off-by: Jakub Sokołowski <jakub@status.im>
Otherwise we get weird failures like these:
```
clang-11: error: cannot use 'cpp-output' output with multiple -arch options
clang-11: error: invalid argument '-mmacos-version-min=10.12' not allowed with '-miphoneos-version-min=8.0'
clang-11: error: invalid argument '-mmacos-version-min=10.12' not allowed with '-miphoneos-version-min=8.0'
```
Depends on: https://github.com/status-im/status-jenkins-lib/pull/47
Signed-off-by: Jakub Sokołowski <jakub@status.im>
Possible fix for errors like:
```
bundler: failed to load command: fastlane (/Users/jenkins/.bundle/ruby/2.7.0/bin/fastlane)
/Users/jenkins/.bundle/ruby/2.7.0/gems/fastlane-2.205.2/fastlane_core/lib/fastlane_core/ui/interface.rb:153:in `shell_error!': [!] Shell command exited with exit status 51 instead of 0. (FastlaneCore::Interface::FastlaneShellError)
```
Signed-off-by: Jakub Sokołowski <jakub@status.im>
This passing of Watchman socket was implemented in order to avoid this:
```
Error: EMFILE: too many open files, watch
at FSEvent.FSWatcher._handle.onchange (node:internal/fs/watchers:204:21)
Emitted 'error' event on NodeWatcher instance at:
at NodeWatcher.checkedEmitError (/private/tmp/nix-build-status-mobile-build-nightly-android.drv-0/node_modules/sane/src/node_watcher.js:143:12)
at FSWatcher.emit (node:events:527:28)
at FSEvent.FSWatcher._handle.onchange (node:internal/fs/watchers:210:12) {
errno: -24,
syscall: 'watch',
code: 'EMFILE',
filename: null
}
```
Which is caused by `jest-haste-map` used by `metro` starting to watch
the filesystem for file changes, which is pointless when doing a
one-off build using Nix.
But by setting `CI=true` we can make `metro` not start this waching of
files in the first place, removing the need for use of Watchman entirely.
By entirely dropping use of Watchman we also fix the following issue:
```
[cli] unable to talk to your watchman on /tmp/tmp-status-mobile-ABC/jenkins-state/sock! (Permission denied)
```
Which happens on multi-user Nix installations becuase the user that the
Nix build is executed as is not the same as the user that starts
Watchman and creates the socket file.
Issue: https://github.com/status-im/status-mobile/issues/13783
Signed-off-by: Jakub Sokołowski <jakub@status.im>
This way the name of the repo makes at least some sense and
matches the `status-desktop` repo naming.
Also updated `status-jenkins-lib` since it also contained
references to `status-react` repo and job names.
Signed-off-by: Jakub Sokołowski <jakub@status.im>
Possible fix fix failing `status-go` builds:
https://github.com/status-im/status-react/issues/13346
Other notable upgrades:
* NodeJS `16.15.1` to `16.15.0`
* Yarn `1.22.18` to `1.22.19`
* Clojure `1.11.1.1113` to `1.11.1.1139`
Signed-off-by: Jakub Sokołowski <jakub@status.im>
This handles the usual case where a missing Gradle version causes the
call to `make nix-update-gradle` to fail since call to Gradle also fails.
This is simpler than getting a dev to run commands manually.
Signed-off-by: Jakub Sokołowski <jakub@status.im>
While working on Nix builds for `go-waku` I figured this derivation
could use some cleanup, to make it shorter and more readable.
Signed-off-by: Jakub Sokołowski <jakub@status.im>
Notable version changes:
- Coreutils `9.0` to `9.1`
- OpenSSL `1.1.1n` to `1.1.1o`
- NodeJS `16.14.2` to `16.15.1`
- Clojure `1.11.1.1107` to `1.11.1.1113`
- Ruby `2.7.5p203` to `2.7.6p219`
- Cocoapods `1.11.0` to `1.11.3`
- Git `2.35.1` to `2.36.1`
- Curl `7.82.0` to `7.83.1`
- Android SDK Platform Tools `31.0.3` to `33.0.1`
Most important is the Coreutils upgrade to 9.1 which includes a fix for
iOS builds on new M1 ARM64 processors:
https://github.com/status-im/status-react/issues/12799
Also fixes broken Android SDK builds on Linux due to `auto-patchelf-hook` change:
https://github.com/NixOS/nixpkgs/pull/163924
I've fixed this in `nixpkgs` PR:
https://github.com/NixOS/nixpkgs/pull/173376
Signed-off-by: Jakub Sokołowski <jakub@status.im>
If we keep using specific `buildGo117Package` we can easily forge to
upgrade when we bump the Go compiler itself. By locking those explicitly
in `overlay.nix` we make sure they all get bumped together.
Signed-off-by: Jakub Sokołowski <jakub@status.im>
This has several benefits:
* Less abuse of `extra-sandbox-paths` Nix option
* Less inputs to the Android release build derivation
* Easier for users to sign the build themselves
* Simplification of `scripts/release-android.sh`
* Preparation for building using Nix Flakes
The only two remaining credentials passed via `extra-sandbox-paths` is
the Infura and OpenSea API keys, and there is no way around that other
than passing them via Nix arguments, but that would cause them to end up
in `/nix/store` as part of `.drv` files.
I'm also renaming `release-fdroid` to `build-fdroid` to be consistent.
Depends on: https://github.com/status-im/status-jenkins-lib/pull/42
Signed-off-by: Jakub Sokołowski <jakub@status.im>
Otherwise on some devices with with good connecitons rate limiting might
cause failures to fetch POMs or JARs and in result failing the whole update.
Signed-off-by: Jakub Sokołowski <jakub@status.im>
This fixes two issues with the `nix-update-gradle` target:
* It now fails when a JAR is missing which used to be ignored.
* It ignores dependencies that have no JARs, like Eclipse plugins.
This makes the process more robust, since we can see something is
missing right away, and a developer may re-run the process to take
account of possible temporary networking failures or rate limiting.
It also slims down the size of the `deps.json` by removing dependencies
which contribute no actual JARs or AARs to the build process.
Signed-off-by: Jakub Sokołowski <jakub@status.im>
Also use `ssh-ng://` protocol implementation instead of `ssh://`.
Might fix the following error:
```
error (ignored): error: reached end of FramedSource
error: sized: unexpected end-of-file
error: unexpected end-of-file
```
For more details see: https://github.com/NixOS/nix/issues/6495
Signed-off-by: Jakub Sokołowski <jakub@status.im>
Minor upgrade done to match the version deployed in CI due to a bug
with `nix-copy-closure` which broken Nix cache jobs:
https://github.com/status-im/infra-ci/issues/49
This will not force upgrade on developes, nor require any `make nix-purge`.
If however and upgrade is wanted it can be simply done with `nix upgrade-nix`.
Signed-off-by: Jakub Sokołowski <jakub@status.im>
Notable software upgrades:
- Go `1.16.8` to `1.17.3`
- NodeJS `12.22.7` to `16.14.2`
- OpenJDK `8u272-b10` to `8u322-ga`
- Clojure `1.10.3.1029` to `1.11.1.1107`
- Git `2.33.1` to `2.35.1`
- CMake `3.18.1` to `3.22.3`
- Curl `7.79.1` to `7.82.0`
- GNU Awk `5.1.0` to `5.1.1`
The Go upgrade is done in advance of upgrade to `1.18.1`.
The Node.js upgrade is done since `12.x` security support [is over](https://endoflife.date/nodejs).
Signed-off-by: Jakub Sokołowski <jakub@status.im>
Incorrect reference to `nimbus.src-override` causes Gradle builds to fail with:
```
Caused by: org.gradle.internal.resolve.ModuleVersionResolveException: Could not resolve status-im:status-go:e940434.
```
Due to this `if` clause not triggering:
a573ae70e5/modules/react-native-status/android/build.gradle (L4-L8)
Signed-off-by: Jakub Sokołowski <jakub@status.im>
Nesting of Nix shells does not work, since we have no way to safely do
that without starting an infinite loop of nested Nix shells.
Signed-off-by: Jakub Sokołowski <jakub@status.im>
This will make identifying installed software easier.
It can also be used to generate F-Droid RPs more easily.
Depends on:
https://github.com/status-im/status-jenkins-lib/pull/35
Signed-off-by: Jakub Sokołowski <jakub@status.im>
With upgrade to Go above `1.15` and Gomobile we now drop support for darwin/arm32 builds.
This means all devices like iPhone 5 or older.
For more details see:
https://tip.golang.org/doc/go1.15#darwin
Signed-off-by: Jakub Sokołowski <jakub@status.im>
This simplifies the package, and also drops the need for `deps.nix` file.
It also slims down the #11986 Go upgrade PR for easier debugging.
Signed-off-by: Jakub Sokołowski <jakub@status.im>
Notable software upgrades:
- Go `1.14.13` to `1.14.15`
- Clojure `1.10.1.763` to `1.10.3.855`
- NodeJS `12.20.1` to `12.22.1`
- Git `2.29.2` to `2.31.1`
- CMake `3.10.2` to `3.18.1`
- Curl `7.74.0` to `7.76.1`
- Android NDK `21.3.6528147` to `22.1.7171670`
- Android Platform Tools `30.0.5` to `31.0.2`
Signed-off-by: Jakub Sokołowski <jakub@status.im>
This should also fix issues with MacOS sandboxing:
```
sandbox-exec: execvp() of '/nix/store/...-bash-4.4-p23/bin/bash' failed: Operation not permitted
```
https://github.com/cachix/install-nix-action/issues/15
Signed-off-by: Jakub Sokołowski <jakub@status.im>
This fixes an issue first detected in:
https://github.com/status-im/status-react/pull/12141
Which resulted in dependencies being ignored for `react-native-nagivation`.
Signed-off-by: Jakub Sokołowski <jakub@status.im>
It seems like this worked before purely because the `cljfmt` library was
already in the `~/.m2` cache folder. This issue was noticed when I
cleaned up the `~/.m2` folder on one Jenkins slave host and the Lint stage
started randomly failing.
Signed-off-by: Jakub Sokołowski <jakub@status.im>
Signed-off-by: andrey <motor4ik@gmail.com>
Alexander
Jakub Sokołowski
Brian Sztamfater
Michele Balistreri
Parvesh Monu
yqrashawn
Jamie Caprani
Richard Ramos
Roman Volosovskyi
Icaro Motta
Audrius Molis
Andrea Maria Piana
andrey
Jakub