Otherwise Nix produces warnings like this:
```
warning: ignoring untrusted substituter 'https://nix-cache.status.im/', you are not a trusted user.
```
Since adding users to `trusted-users` essentially gives them `root`:
>Adding a user to trusted-users is essentially equivalent to giving that user root access to the system.
> — https://nixos.org/manual/nix/stable/command-ref/conf-file.html#conf-trusted-users
A more kosher approach might be to just add the cache config itself globally.
Signed-off-by: Jakub Sokołowski <jakub@status.im>
Should resolve errors like this:
```
error: file 'nixpkgs' was not found in the Nix search path (add it using $NIX_PATH or -I)
```
Signed-off-by: Jakub Sokołowski <jakub@status.im>
We use `builtins.fetchTarball` to avoid having to first import a
`nixpkgs` from existing channels, which breaks if there are no channels.
Otherwise new Nix 2.14 version can result in:
```
error: file 'nixpkgs' was not found in the Nix search path (add it using $NIX_PATH or -I)
```
Which happens due to no default `nixpkgs` channel existing:
```
> nix-channel --list
```
Resolves: https://github.com/status-im/status-mobile/issues/16342
Signed-off-by: Jakub Sokołowski <jakub@status.im>
According to this line from the docs:
>The system-wide configuration file sysconfdir/nix/nix.conf (i.e. /etc/nix/nix.conf),
>or $NIX_CONF_DIR/nix.conf if NIX_CONF_DIR is set. Values loaded in this file are not
>forwarded to the Nix daemon. The client assumes that the daemon has already loaded them.
https://nixos.org/manual/nix/stable/command-ref/conf-file.html#description
Our usage of `NIX_CONF_DIR` has been wrong for a while now.
The correct way of applying this config is using `NIX_USER_CONF_FILES`.
In addition the `extra-substituters` no longer exists in the docs.
Use of `trusted-substituters` is necessary according to:
>At least one of the following conditions must be met for Nix to use a substituter:
>
>- the substituter is in the trusted-substituters list
>- the user calling Nix is in the trusted-users list
https://nixos.org/manual/nix/stable/command-ref/conf-file.html#conf-substituters
Signed-off-by: Jakub Sokołowski <jakub@status.im>
Notable upgrades:
* Bash `5.1` to `5.2`
* Git `2.37.3` to `2.40.1`
* Curl `7.85.0` to `8.0.1`
* OpenSSL `3.0.5` to `3.0.8`
* Go `1.18.6` to `1.18.9`
* NodeJS `18.9.1` to `18.16.0`
* Java `1.8.0_322` to `11.0.11`
* Ruby `3.1.2` to `3.1.4`
* Python `2.7.18` to `3.10.11`
* Clojure `1.11.1.1165` to `1.11.1.1273`
* Clj-kondo `v2022.10.05` to `v2023.04.14`
* Zprint `1.2.5` to `1.2.6`
* Bundler `2.3.22` to `2.4.13`
* Gradle `6.9.2` to `6.9.4`
* Android Platform Tools `33.0.2` to `33.0.3`
* Android SDK Tools to Android SDK Command-Line Tools
Removals:
* Zprint since the version in `nixpkgs` was newer than in overlay.
* Xcode wrapper definition was removed since my fixes were merged:
- https://github.com/NixOS/nixpkgs/pull/204278
- https://github.com/NixOS/nixpkgs/pull/228696
Signed-off-by: Jakub Sokołowski <jakub@status.im>
This commit upgrades re-frame to v1.3.0 (latest stable release), released ~9
months ago, in 2022-08-27. This is a solid upgrade, with no breaking changes as
far as I tested status-mobile. It's a great testament of re-frame's stability
and commitment to backwards compatibility, as are many Clojure libs.
Fixes https://github.com/status-im/status-mobile/issues/15963
The big, and truly relevant addition is the introduction of the :fx built-in
effect that was added ~3 years ago in Aug/2020 in v1.1.0.
Relevant changelog:
- Global interceptors are now supported (added in v1.0.0).
- reg-event-fx will just warn (not generate an error) if the effect map returned
contains an unknown effect key.
- re-frame will now warn us when we are calling subscribe outside of a reactive
context.
- "re-frame now guarantees that a :db effect, if present, will be actioned
before any other sibling effects. re-frame continues to provide NO guarantees
about the order in which other effects will be actioned."
(https://day8.github.io/re-frame/releases/2020/#110-2020-08-24)
- There's syntactic sugar for trivial reg-sub declarations (added in v1.3.0).
See the documentation for reg-sub for more details
https://day8.github.io/re-frame/api-re-frame.core/#reg-sub
- "The built-in effect :dispatch-later can now take a single map value.
Supplying a sequence of maps is now deprecated in favor of using multiple
:dispatch-later effects within the new :fx effect."
https://day8.github.io/re-frame/releases/2020/#111-2020-08-26
For some unknown to me reason we are using a different Yarn call to
Shadow-cljs to generate the JSBundle for iOS builds, while the one
created by the Android derivation shoudl be exactly the same.
I'm changing the target to just be `make jsbundle` while keeping aliases
referencing old naming, and moving things around in `nix` folder to
reflect the fact that the derivation is no longer Android-specific.
Also, crucially, I've changed the `import` in `index.js` to use the
`./result/index.js` path, since that's what Nix creates. I'm not sure if
this clashes with any developer workflow that takes place locally, so
I'd appreciate some testing from developers.
Depends on: https://github.com/status-im/status-jenkins-lib/pull/67
Signed-off-by: Jakub Sokołowski <jakub@status.im>
This way we will use the same format as Nix logs and errors when a hash
changes, which avoids confusion.
Signed-off-by: Jakub Sokołowski <jakub@status.im>
Otherwise we can get a failure like this:
```
Regenerating Nix files...
Found 40 sub-projects...
Found 609 direct dependencies...
Found 889 dependency URLs...
! Failed to fetch: https://repo.maven.apache.org/maven2/org/ow2/asm/asm-bom/9.5/asm-bom-9.5.jar
```
When using `make nix-update-gradle`.
Signed-off-by: Jakub Sokołowski <jakub@status.im>
Otherwise we see weird errors like this:
```
/nix/store/g0jijpgcb4q54zbvz5p8yvxcnb6lshnk-stdenv-darwin/setup: line 1391: /nix/store/sja8sqq4y5s9ijkb97i3pi2jrhsy40cz-xcode-wrapper-14.3/bin/xcodebuild: Operation not permitted
We require xcodebuild version: 14.3
error: builder for '/nix/store/d1dji0ywl851wgj9vv58ibpm32gq3wsm-xcode-wrapper-14.3.drv' failed with exit code 1;
last 2 log lines:
> /nix/store/g0jijpgcb4q54zbvz5p8yvxcnb6lshnk-stdenv-darwin/setup: line 1391: /nix/store/sja8sqq4y5s9ijkb97i3pi2jrhsy40cz-xcode-wrapper-14.3/bin/xcodebuild: Operation not permitted
> We require xcodebuild version: 14.3
```
Related to: https://github.com/NixOS/nixpkgs/pull/228696
Signed-off-by: Jakub Sokołowski <jakub@status.im>
Also adds an assertion that gives a better error when trying to use an
`ios` Nix shell on an unsupported operating system.
Resolves:
https://github.com/status-im/status-mobile/issues/15695
Signed-off-by: Jakub Sokołowski <jakub@status.im>
Some users have reported local `make run-ios` failures with errors like:
```
Command PhaseScriptExecution failed with a nonzero exit code
```
Where the actual error that can be found in `ios/logs/react-native-xcode.log` is:
```
node: bad option: --openssl-legacy-provider
```
Caused by a flag we apply to our pinned Node.js `18.9.1` to fix:
```
opensslErrorStack: [ 'error:03000086:digital envelope routines::initialization error' ],
library: 'digital envelope routines',
reason: 'unsupported',
code: 'ERR_OSSL_EVP_UNSUPPORTED'
```
From: https://github.com/status-im/status-mobile/pull/15167
And the need for that should go away once we upgrade Node.js further.
But for now a decent fix is to not apply that flag directly in Xcode
config so as to avoid the `bad option` error when Xcode us called
outside of Nix context.
Fixes: https://github.com/status-im/status-mobile/issues/15631
Signed-off-by: Jakub Sokołowski <jakub@status.im>
* [IOS Only] react-native 0.63 to 0.67
* [Android Only] react-native 0.63 to 0.67
* bring back all the jenkinsfiles
* make auto-complete prop for text-input compatible
* [IOS Only] react-native 0.63 to 0.67
* [Android Only] react-native 0.63 to 0.67
* bring back all the jenkinsfiles
* nix: drop unnecessary set -x from status-go build
Signed-off-by: Jakub Sokołowski <jakub@status.im>
* add explicity implementation line for soloader
And add deleteDebugFilesForVariant fix for libhermes.
Signed-off-by: Jakub Sokołowski <jakub@status.im>
* use fast-image for link previews
* fix extra line in message composer on android
This elevation prop is un-necessary and causes an extra line to appear which looks like a border but its actually a shadow.
* don't use `fast-image` for url preview favicon
* fix audio record button interfering cancel android
This fix was not needed in react-native 0.63.4 but is needed after we upgraded to 0.67.5
* get rid of unused platform import
---------
Co-authored-by: Jakub Sokołowski <jakub@status.im>
Without this fix:
https://github.com/NixOS/nixpkgs/commit/d0c06fa3
The `apksigner` utility is unavailable on macOS:
```
error: Package ‘apksigner-33.0.1’ in .../pkgs/development/tools/apksigner/default.nix:86
is not supported on ‘x86_64-darwin’, refusing to evaluate.
```
Signed-off-by: Jakub Sokołowski <jakub@status.im>
Otherwise we can end up with Gradle failing to find the dependencies
because we've patched away all entries referencing external repos.
Also made the regex in AWS parser a but more strict.
Signed-off-by: Jakub Sokołowski <jakub@status.im>
Cocoapod installations can fail with errors like:
```
tclsh /Users/jenkins/Library/Caches/CocoaPods/Pods/Release/SQLCipher/3.4.2-f9fcf/tool/addopcodes.tcl parse.h.temp >parse.h
./configure: line 11729: tclsh: command not found
./configure: line 12262: tclsh: command not found
./configure: line 12276: tclsh: command not found
```
If a pure shell is used. Also `pgrep` would be missing from a pure shell.
Signed-off-by: Jakub Sokołowski <jakub@status.im>
Before we were patching only `build.gradle`, which is not the only type
of Gradle config file. If we do not cover them all we can encounter
errors about missing package because they will continue using remote
repositories instead of `mavenLocal()`, to which pass Nix store path.
We also need to cover `gradlePluginPortal()` to provide plugins.
This is also necessary for the React Native upgrade:
https://github.com/status-im/status-mobile/pull/15203
Signed-off-by: Jakub Sokołowski <jakub@status.im>
Refactoring the derivation that fetches all the POMs, JARs,
and AARs in order to make it more generic and easier to extend.
The main change is adding `files` key in `deps.json` which contains
a dict of all the files reletad to given package.
This way we can more easily include other files that might be available
for download, like AARs with ASC suffix, or `nodeps` JARs.
This is also necessary for the React Native upgrade:
https://github.com/status-im/status-mobile/pull/15203
Signed-off-by: Jakub Sokołowski <jakub@status.im>
As discovered in:
https://github.com/status-im/status-mobile/pull/15225
The attempt to fix this in:
https://github.com/status-im/status-mobile/pull/15180
But it doesn't appear to work, so instead I'm allowing an override of
`NODE_BINARY` variable and spetting it when defining the Nix shell.
The key things here are:
* Xcode injects its own paths into `PATH` which breaks Nix env.
* Combining Nix shells with `inputsFrom` does not inherit all vars.
It's important to set these variables in `shellHook` and not elsewhere.
Signed-off-by: Jakub Sokołowski <jakub@status.im>
* nix: update gradle dependencies
Signed-off-by: Jakub Sokołowski <jakub@status.im>
* nix: include nodeps JARs in Gradle deps
Signed-off-by: Jakub Sokołowski <jakub@status.im>
* nix: include nodeps JAR for semver4j 0.16.4
Can cause failures like in some cases:
```
A problem occurred configuring project ':react-native-hole-view'.
> Could not resolve all files for configuration ':react-native-hole-view:classpath'.
> Could not find semver4j-0.16.4-nodeps.jar (com.github.gundy:semver4j:0.16.4).
Searched in the following locations:
file:/nix/store/3n2pxsqa2izlx8c23s6jgqai0bqaklm1-status-mobile-maven-deps/com/github/gundy/semver4j/0.16.4/semver4j-0.16.4-nodeps.jar
```
Signed-off-by: Jakub Sokołowski <jakub@status.im>
---------
Signed-off-by: Jakub Sokołowski <jakub@status.im>
Fixes issue with following failure:
```
Failed to construct transformer: Error: error:0308010C:digital envelope routines::unsupported
at new Hash (node:internal/crypto/hash:71:19)
at Object.createHash (node:crypto:130:10)
at stableHash (/Users/jenkins/workspace/status-mobile/platforms/ios/node_modules/metro-cache/src/stableHash.js:19:8)
at JsTransformer.getCacheKey (/Users/jenkins/workspace/status-mobile/platforms/ios/node_modules/metro/src/JSTransformer/worker.js:471:7)
at getTransformCacheKey (/Users/jenkins/workspace/status-mobile/platforms/ios/node_modules/metro/src/DeltaBundler/Transformer/getTransformCacheKey.js:39:29)
at new Transformer (/Users/jenkins/workspace/status-mobile/platforms/ios/node_modules/metro/src/DeltaBundler/Transformer.js:147:28)
at /Users/jenkins/workspace/status-mobile/platforms/ios/node_modules/metro/src/Bundler.js:54:29
at runMicrotasks (<anonymous>)
at processTicksAndRejections (node:internal/process/task_queues:96:5) {
opensslErrorStack: [ 'error:03000086:digital envelope routines::initialization error' ],
library: 'digital envelope routines',
reason: 'unsupported',
code: 'ERR_OSSL_EVP_UNSUPPORTED'
```
https://roytuts.com/how-to-fix-err_ossl_evp_unsupported-in-react-js-application/
Signed-off-by: Jakub Sokołowski <jakub@status.im>
* nix: upgrade zprint from 1.2.4 to 1.2.5
To address issue described in:
https://github.com/kkinnear/zprint/issues/273
Signed-off-by: Jakub Sokołowski <jakub@status.im>
* chore: use zprint :multi-lhs-hang
* refactor: re-format clojure using zprint 1.2.5
---------
Signed-off-by: Jakub Sokołowski <jakub@status.im>
Co-authored-by: yqrashawn <namy.19@gmail.com>
On M1 calling `shadow-cljs` fails with:
```
Execution error (UnsatisfiedLinkError) at java.lang.ClassLoader$NativeLibrary/load (ClassLoader.java:-2).
/private/var/folders/__/x311ykg17rqgq2wyl4kn1pdr0001yh/T/jna8753030888504535661.tmp:
dlopen(/private/var/folders/__/x311ykg17rqgq2wyl4kn1pdr0001yh/T/jna8753030888504535661.tmp, 0x0001):
tried: '/private/var/folders/__/x311ykg17rqgq2wyl4kn1pdr0001yh/T/jna8753030888504535661.tmp'
(fat file, but missing compatible architecture (have (unknown,i386,x86_64), need (arm64e)))
```
This is due to an outdeted dependency on JNA 3.2.2, which is pulled in
by `hawk` package which up until release `2.11.16` was a `shadow-clj`
dependency which was removed because it was:
>Only used to be used on macOS since it was slightly faster than the default
>JVM implementation. However in Big Sur it seems to cause issues and break
>completely or just be a lot slower.
https://github.com/thheller/shadow-cljs/commit/f3b89b5a
Dropped the explicit dependency on `org.clojure/core.async` to avoid:
```
WARNING: The org.clojure/core.async dependency in shadow-cljs.edn was ignored.
Default version is used and override is not allowed to ensure compatibility.
```
Resolves: https://github.com/status-im/status-mobile/issues/14196
Signed-off-by: Jakub Sokołowski <jakub@status.im>
Notable upgrades:
* Go `1.17.11` to `1.18.6`
* NodeJS `16.15.0` to `16.17.1`
* Clojure `1.11.1.1139` to `1.11.1.1165`
* Ruby Gem `3.2.26` to `3.3.20`
* Bundler `2.3.9` to `2.3.22`
* Git `2.36.1` to `2.37.3`
* Curl `7.83.1` to `7.85.0`
* OpenSSL `1.1.1o` to `3.0.5`
* PatchELF `0.14.5` to `0.15.0`
* Android SDK Platform Tools `33.0.1` to `33.0.2`
Signed-off-by: Jakub Sokołowski <jakub@status.im>
This resolves issues with weird `status-go` build errors like:
```
cgo: C compiler "2022-09-07" not found: exec: "2022-09-07": executable file not found in $PATH
```
Such errors are caused by `xcrun` spewing warnings to `stdout`. The PR
to fix this issue permanently has been created but it might take a while
before it's merged.
Fix PR: https://github.com/golang/mobile/pull/84
Resolves: https://github.com/status-im/status-mobile/issues/13949
Signed-off-by: Jakub Sokołowski <jakub@status.im>
Otherwise we get weird failures like these:
```
clang-11: error: cannot use 'cpp-output' output with multiple -arch options
clang-11: error: invalid argument '-mmacos-version-min=10.12' not allowed with '-miphoneos-version-min=8.0'
clang-11: error: invalid argument '-mmacos-version-min=10.12' not allowed with '-miphoneos-version-min=8.0'
```
Depends on: https://github.com/status-im/status-jenkins-lib/pull/47
Signed-off-by: Jakub Sokołowski <jakub@status.im>
Possible fix for errors like:
```
bundler: failed to load command: fastlane (/Users/jenkins/.bundle/ruby/2.7.0/bin/fastlane)
/Users/jenkins/.bundle/ruby/2.7.0/gems/fastlane-2.205.2/fastlane_core/lib/fastlane_core/ui/interface.rb:153:in `shell_error!': [!] Shell command exited with exit status 51 instead of 0. (FastlaneCore::Interface::FastlaneShellError)
```
Signed-off-by: Jakub Sokołowski <jakub@status.im>
This passing of Watchman socket was implemented in order to avoid this:
```
Error: EMFILE: too many open files, watch
at FSEvent.FSWatcher._handle.onchange (node:internal/fs/watchers:204:21)
Emitted 'error' event on NodeWatcher instance at:
at NodeWatcher.checkedEmitError (/private/tmp/nix-build-status-mobile-build-nightly-android.drv-0/node_modules/sane/src/node_watcher.js:143:12)
at FSWatcher.emit (node:events:527:28)
at FSEvent.FSWatcher._handle.onchange (node:internal/fs/watchers:210:12) {
errno: -24,
syscall: 'watch',
code: 'EMFILE',
filename: null
}
```
Which is caused by `jest-haste-map` used by `metro` starting to watch
the filesystem for file changes, which is pointless when doing a
one-off build using Nix.
But by setting `CI=true` we can make `metro` not start this waching of
files in the first place, removing the need for use of Watchman entirely.
By entirely dropping use of Watchman we also fix the following issue:
```
[cli] unable to talk to your watchman on /tmp/tmp-status-mobile-ABC/jenkins-state/sock! (Permission denied)
```
Which happens on multi-user Nix installations becuase the user that the
Nix build is executed as is not the same as the user that starts
Watchman and creates the socket file.
Issue: https://github.com/status-im/status-mobile/issues/13783
Signed-off-by: Jakub Sokołowski <jakub@status.im>
This way the name of the repo makes at least some sense and
matches the `status-desktop` repo naming.
Also updated `status-jenkins-lib` since it also contained
references to `status-react` repo and job names.
Signed-off-by: Jakub Sokołowski <jakub@status.im>
Possible fix fix failing `status-go` builds:
https://github.com/status-im/status-react/issues/13346
Other notable upgrades:
* NodeJS `16.15.1` to `16.15.0`
* Yarn `1.22.18` to `1.22.19`
* Clojure `1.11.1.1113` to `1.11.1.1139`
Signed-off-by: Jakub Sokołowski <jakub@status.im>
This handles the usual case where a missing Gradle version causes the
call to `make nix-update-gradle` to fail since call to Gradle also fails.
This is simpler than getting a dev to run commands manually.
Signed-off-by: Jakub Sokołowski <jakub@status.im>
While working on Nix builds for `go-waku` I figured this derivation
could use some cleanup, to make it shorter and more readable.
Signed-off-by: Jakub Sokołowski <jakub@status.im>
Notable version changes:
- Coreutils `9.0` to `9.1`
- OpenSSL `1.1.1n` to `1.1.1o`
- NodeJS `16.14.2` to `16.15.1`
- Clojure `1.11.1.1107` to `1.11.1.1113`
- Ruby `2.7.5p203` to `2.7.6p219`
- Cocoapods `1.11.0` to `1.11.3`
- Git `2.35.1` to `2.36.1`
- Curl `7.82.0` to `7.83.1`
- Android SDK Platform Tools `31.0.3` to `33.0.1`
Most important is the Coreutils upgrade to 9.1 which includes a fix for
iOS builds on new M1 ARM64 processors:
https://github.com/status-im/status-react/issues/12799
Also fixes broken Android SDK builds on Linux due to `auto-patchelf-hook` change:
https://github.com/NixOS/nixpkgs/pull/163924
I've fixed this in `nixpkgs` PR:
https://github.com/NixOS/nixpkgs/pull/173376
Signed-off-by: Jakub Sokołowski <jakub@status.im>
If we keep using specific `buildGo117Package` we can easily forge to
upgrade when we bump the Go compiler itself. By locking those explicitly
in `overlay.nix` we make sure they all get bumped together.
Signed-off-by: Jakub Sokołowski <jakub@status.im>
This has several benefits:
* Less abuse of `extra-sandbox-paths` Nix option
* Less inputs to the Android release build derivation
* Easier for users to sign the build themselves
* Simplification of `scripts/release-android.sh`
* Preparation for building using Nix Flakes
The only two remaining credentials passed via `extra-sandbox-paths` is
the Infura and OpenSea API keys, and there is no way around that other
than passing them via Nix arguments, but that would cause them to end up
in `/nix/store` as part of `.drv` files.
I'm also renaming `release-fdroid` to `build-fdroid` to be consistent.
Depends on: https://github.com/status-im/status-jenkins-lib/pull/42
Signed-off-by: Jakub Sokołowski <jakub@status.im>
Otherwise on some devices with with good connecitons rate limiting might
cause failures to fetch POMs or JARs and in result failing the whole update.
Signed-off-by: Jakub Sokołowski <jakub@status.im>
This fixes two issues with the `nix-update-gradle` target:
* It now fails when a JAR is missing which used to be ignored.
* It ignores dependencies that have no JARs, like Eclipse plugins.
This makes the process more robust, since we can see something is
missing right away, and a developer may re-run the process to take
account of possible temporary networking failures or rate limiting.
It also slims down the size of the `deps.json` by removing dependencies
which contribute no actual JARs or AARs to the build process.
Signed-off-by: Jakub Sokołowski <jakub@status.im>
Also use `ssh-ng://` protocol implementation instead of `ssh://`.
Might fix the following error:
```
error (ignored): error: reached end of FramedSource
error: sized: unexpected end-of-file
error: unexpected end-of-file
```
For more details see: https://github.com/NixOS/nix/issues/6495
Signed-off-by: Jakub Sokołowski <jakub@status.im>
Minor upgrade done to match the version deployed in CI due to a bug
with `nix-copy-closure` which broken Nix cache jobs:
https://github.com/status-im/infra-ci/issues/49
This will not force upgrade on developes, nor require any `make nix-purge`.
If however and upgrade is wanted it can be simply done with `nix upgrade-nix`.
Signed-off-by: Jakub Sokołowski <jakub@status.im>
Notable software upgrades:
- Go `1.16.8` to `1.17.3`
- NodeJS `12.22.7` to `16.14.2`
- OpenJDK `8u272-b10` to `8u322-ga`
- Clojure `1.10.3.1029` to `1.11.1.1107`
- Git `2.33.1` to `2.35.1`
- CMake `3.18.1` to `3.22.3`
- Curl `7.79.1` to `7.82.0`
- GNU Awk `5.1.0` to `5.1.1`
The Go upgrade is done in advance of upgrade to `1.18.1`.
The Node.js upgrade is done since `12.x` security support [is over](https://endoflife.date/nodejs).
Signed-off-by: Jakub Sokołowski <jakub@status.im>
Jakub Sokołowski
Mohamed Javid
Jamie Caprani
Siddarth Kumar
flexsurfer
Icaro Motta
Ajay Sivan
Ibrahem Khalil
Alexander
Brian Sztamfater
Michele Balistreri
Parvesh Monu
yqrashawn
Richard Ramos
Roman Volosovskyi
Audrius Molis
Andrea Maria Piana
andrey