watchman was upgraded significantly during the last #14944 (4.9.0 (Aug 16, 2017) to 2023.01.30.00 - 6 years between):
status-im/nixpkgs@4e9c02b
Probably causing developers to have "too many files open" issue #16341
This PR is an attempt to fix the issue by downgrading the watchman
Signed-off-by: Jakub Sokołowski <jakub@status.im>
Otherwise if in Nix context we can accidentally use GNU stat which
results in a different output due to different command line flags:
```
[nix-shell:~/status-mobile]$ stat -Lf "%Sg" "/nix/store"
stat: cannot read file system information for '%Sg': No such file or directory
File: "/nix/store"
ID: 10000110000001a Namelen: ? Type: apfs
Block size: 4096 Fundamental block size: 4096
Blocks: Total: 242837545 Free: 199242283 Available: 199242283
Inodes: Total: 7971454780 Free: 7969691320
```
And it should be just owner group name or ID. Which in turn results in:
```
Unknown Nix installtion type!
```
Signed-off-by: Jakub Sokołowski <jakub@status.im>
Android SDK in nixpkg doesn't support `aarch64-darwin`, for details see
https://github.com/status-im/status-mobile/issues/12794
The workaround was to run terminal application with Rosetta or make with `aarch` tool.
This PR overrides system architecture to always be `x86_64` for Android shell and build targets ran by make.
CI is not affected as it runs `nix` directly.
Signed-off-by: Jakub Sokołowski <jakub@status.im>
On macOS `git-lfs` checks fail sometimes and they also take a lot of time to execute.
```log
FAIL github.com/git-lfs/git-lfs/v3/lfshttp 0.820s
FAIL
error: builder for '/nix/store/abd6spw876qvcmlpcwsljsnbpbnvx1fg-git-lfs-3.3.0.drv' failed with exit code 1;
last 10 log lines:
> Error Trace: certs_test.go:210
> Error: Expected value not to be nil.
> Test: TestCertFromSSLCAPathEnv
> certs_test.go:210:
> Error Trace: certs_test.go:210
> Error: Expected value not to be nil.
> Test: TestCertFromSSLCAPathEnv
> FAIL
> FAIL github.com/git-lfs/git-lfs/v3/lfshttp 0.820s
> FAIL
For full logs, run 'nix-store -l /nix/store/abd6spw876qvcmlpcwsljsnbpbnvx1fg-git-lfs-3.3.0.drv'.
```
Failures initially observed by me and @smohamedjavid and then today again reported by @yqrashawn
This commit adds a `nix` overlay that skips those checks
This is a workaround for a well known issue of Nix failing to download
bigger packages from Nix cache due to errors like this:
```
HTTP error 200 (curl error: Transferred a partial file)
```
or
```
HTTP error 200 (curl error: Stream error in the HTTP/2 framing layer)
```
From docs: https://nixos.org/manual/nix/stable/command-ref/conf-file.html#conf-fallback
>If set to true, Nix will fall back to building from source if a binary
>substitute fails. This is equivalent to the --fallback flag. The default is false.
Realted: https://github.com/status-im/infra-ci/issues/17
Signed-off-by: Jakub Sokołowski <jakub@status.im>
Fixes following error:
```
ld: framework not found IOKit
clang-11: error: linker command failed with exit code 1 (use -v to see invocation)
make[1]: *** [status_nodejs_addon.target.mk:175: Release/status_nodejs_addon.node] Error 1
```
The `make test` target started to fail on macOS after #14944.
This PR adds 2 Apple SDK frameworks to default shell: `IOKit` and `CoreServices`
The other 2 mentioned in `binding.gyp` are not added, but build is ok.
Resolves: https://github.com/status-im/status-mobile/issues/16356
Signed-off-by: Jakub Sokołowski <jakub@status.im>
Otherwise Nix produces warnings like this:
```
warning: ignoring untrusted substituter 'https://nix-cache.status.im/', you are not a trusted user.
```
Since adding users to `trusted-users` essentially gives them `root`:
>Adding a user to trusted-users is essentially equivalent to giving that user root access to the system.
> — https://nixos.org/manual/nix/stable/command-ref/conf-file.html#conf-trusted-users
A more kosher approach might be to just add the cache config itself globally.
Signed-off-by: Jakub Sokołowski <jakub@status.im>
Should resolve errors like this:
```
error: file 'nixpkgs' was not found in the Nix search path (add it using $NIX_PATH or -I)
```
Signed-off-by: Jakub Sokołowski <jakub@status.im>
We use `builtins.fetchTarball` to avoid having to first import a
`nixpkgs` from existing channels, which breaks if there are no channels.
Otherwise new Nix 2.14 version can result in:
```
error: file 'nixpkgs' was not found in the Nix search path (add it using $NIX_PATH or -I)
```
Which happens due to no default `nixpkgs` channel existing:
```
> nix-channel --list
```
Resolves: https://github.com/status-im/status-mobile/issues/16342
Signed-off-by: Jakub Sokołowski <jakub@status.im>
According to this line from the docs:
>The system-wide configuration file sysconfdir/nix/nix.conf (i.e. /etc/nix/nix.conf),
>or $NIX_CONF_DIR/nix.conf if NIX_CONF_DIR is set. Values loaded in this file are not
>forwarded to the Nix daemon. The client assumes that the daemon has already loaded them.
https://nixos.org/manual/nix/stable/command-ref/conf-file.html#description
Our usage of `NIX_CONF_DIR` has been wrong for a while now.
The correct way of applying this config is using `NIX_USER_CONF_FILES`.
In addition the `extra-substituters` no longer exists in the docs.
Use of `trusted-substituters` is necessary according to:
>At least one of the following conditions must be met for Nix to use a substituter:
>
>- the substituter is in the trusted-substituters list
>- the user calling Nix is in the trusted-users list
https://nixos.org/manual/nix/stable/command-ref/conf-file.html#conf-substituters
Signed-off-by: Jakub Sokołowski <jakub@status.im>
Notable upgrades:
* Bash `5.1` to `5.2`
* Git `2.37.3` to `2.40.1`
* Curl `7.85.0` to `8.0.1`
* OpenSSL `3.0.5` to `3.0.8`
* Go `1.18.6` to `1.18.9`
* NodeJS `18.9.1` to `18.16.0`
* Java `1.8.0_322` to `11.0.11`
* Ruby `3.1.2` to `3.1.4`
* Python `2.7.18` to `3.10.11`
* Clojure `1.11.1.1165` to `1.11.1.1273`
* Clj-kondo `v2022.10.05` to `v2023.04.14`
* Zprint `1.2.5` to `1.2.6`
* Bundler `2.3.22` to `2.4.13`
* Gradle `6.9.2` to `6.9.4`
* Android Platform Tools `33.0.2` to `33.0.3`
* Android SDK Tools to Android SDK Command-Line Tools
Removals:
* Zprint since the version in `nixpkgs` was newer than in overlay.
* Xcode wrapper definition was removed since my fixes were merged:
- https://github.com/NixOS/nixpkgs/pull/204278
- https://github.com/NixOS/nixpkgs/pull/228696
Signed-off-by: Jakub Sokołowski <jakub@status.im>
This commit upgrades re-frame to v1.3.0 (latest stable release), released ~9
months ago, in 2022-08-27. This is a solid upgrade, with no breaking changes as
far as I tested status-mobile. It's a great testament of re-frame's stability
and commitment to backwards compatibility, as are many Clojure libs.
Fixes https://github.com/status-im/status-mobile/issues/15963
The big, and truly relevant addition is the introduction of the :fx built-in
effect that was added ~3 years ago in Aug/2020 in v1.1.0.
Relevant changelog:
- Global interceptors are now supported (added in v1.0.0).
- reg-event-fx will just warn (not generate an error) if the effect map returned
contains an unknown effect key.
- re-frame will now warn us when we are calling subscribe outside of a reactive
context.
- "re-frame now guarantees that a :db effect, if present, will be actioned
before any other sibling effects. re-frame continues to provide NO guarantees
about the order in which other effects will be actioned."
(https://day8.github.io/re-frame/releases/2020/#110-2020-08-24)
- There's syntactic sugar for trivial reg-sub declarations (added in v1.3.0).
See the documentation for reg-sub for more details
https://day8.github.io/re-frame/api-re-frame.core/#reg-sub
- "The built-in effect :dispatch-later can now take a single map value.
Supplying a sequence of maps is now deprecated in favor of using multiple
:dispatch-later effects within the new :fx effect."
https://day8.github.io/re-frame/releases/2020/#111-2020-08-26
For some unknown to me reason we are using a different Yarn call to
Shadow-cljs to generate the JSBundle for iOS builds, while the one
created by the Android derivation shoudl be exactly the same.
I'm changing the target to just be `make jsbundle` while keeping aliases
referencing old naming, and moving things around in `nix` folder to
reflect the fact that the derivation is no longer Android-specific.
Also, crucially, I've changed the `import` in `index.js` to use the
`./result/index.js` path, since that's what Nix creates. I'm not sure if
this clashes with any developer workflow that takes place locally, so
I'd appreciate some testing from developers.
Depends on: https://github.com/status-im/status-jenkins-lib/pull/67
Signed-off-by: Jakub Sokołowski <jakub@status.im>
This way we will use the same format as Nix logs and errors when a hash
changes, which avoids confusion.
Signed-off-by: Jakub Sokołowski <jakub@status.im>
Otherwise we can get a failure like this:
```
Regenerating Nix files...
Found 40 sub-projects...
Found 609 direct dependencies...
Found 889 dependency URLs...
! Failed to fetch: https://repo.maven.apache.org/maven2/org/ow2/asm/asm-bom/9.5/asm-bom-9.5.jar
```
When using `make nix-update-gradle`.
Signed-off-by: Jakub Sokołowski <jakub@status.im>
Otherwise we see weird errors like this:
```
/nix/store/g0jijpgcb4q54zbvz5p8yvxcnb6lshnk-stdenv-darwin/setup: line 1391: /nix/store/sja8sqq4y5s9ijkb97i3pi2jrhsy40cz-xcode-wrapper-14.3/bin/xcodebuild: Operation not permitted
We require xcodebuild version: 14.3
error: builder for '/nix/store/d1dji0ywl851wgj9vv58ibpm32gq3wsm-xcode-wrapper-14.3.drv' failed with exit code 1;
last 2 log lines:
> /nix/store/g0jijpgcb4q54zbvz5p8yvxcnb6lshnk-stdenv-darwin/setup: line 1391: /nix/store/sja8sqq4y5s9ijkb97i3pi2jrhsy40cz-xcode-wrapper-14.3/bin/xcodebuild: Operation not permitted
> We require xcodebuild version: 14.3
```
Related to: https://github.com/NixOS/nixpkgs/pull/228696
Signed-off-by: Jakub Sokołowski <jakub@status.im>
Also adds an assertion that gives a better error when trying to use an
`ios` Nix shell on an unsupported operating system.
Resolves:
https://github.com/status-im/status-mobile/issues/15695
Signed-off-by: Jakub Sokołowski <jakub@status.im>
Some users have reported local `make run-ios` failures with errors like:
```
Command PhaseScriptExecution failed with a nonzero exit code
```
Where the actual error that can be found in `ios/logs/react-native-xcode.log` is:
```
node: bad option: --openssl-legacy-provider
```
Caused by a flag we apply to our pinned Node.js `18.9.1` to fix:
```
opensslErrorStack: [ 'error:03000086:digital envelope routines::initialization error' ],
library: 'digital envelope routines',
reason: 'unsupported',
code: 'ERR_OSSL_EVP_UNSUPPORTED'
```
From: https://github.com/status-im/status-mobile/pull/15167
And the need for that should go away once we upgrade Node.js further.
But for now a decent fix is to not apply that flag directly in Xcode
config so as to avoid the `bad option` error when Xcode us called
outside of Nix context.
Fixes: https://github.com/status-im/status-mobile/issues/15631
Signed-off-by: Jakub Sokołowski <jakub@status.im>
* [IOS Only] react-native 0.63 to 0.67
* [Android Only] react-native 0.63 to 0.67
* bring back all the jenkinsfiles
* make auto-complete prop for text-input compatible
* [IOS Only] react-native 0.63 to 0.67
* [Android Only] react-native 0.63 to 0.67
* bring back all the jenkinsfiles
* nix: drop unnecessary set -x from status-go build
Signed-off-by: Jakub Sokołowski <jakub@status.im>
* add explicity implementation line for soloader
And add deleteDebugFilesForVariant fix for libhermes.
Signed-off-by: Jakub Sokołowski <jakub@status.im>
* use fast-image for link previews
* fix extra line in message composer on android
This elevation prop is un-necessary and causes an extra line to appear which looks like a border but its actually a shadow.
* don't use `fast-image` for url preview favicon
* fix audio record button interfering cancel android
This fix was not needed in react-native 0.63.4 but is needed after we upgraded to 0.67.5
* get rid of unused platform import
---------
Co-authored-by: Jakub Sokołowski <jakub@status.im>
Without this fix:
https://github.com/NixOS/nixpkgs/commit/d0c06fa3
The `apksigner` utility is unavailable on macOS:
```
error: Package ‘apksigner-33.0.1’ in .../pkgs/development/tools/apksigner/default.nix:86
is not supported on ‘x86_64-darwin’, refusing to evaluate.
```
Signed-off-by: Jakub Sokołowski <jakub@status.im>
Otherwise we can end up with Gradle failing to find the dependencies
because we've patched away all entries referencing external repos.
Also made the regex in AWS parser a but more strict.
Signed-off-by: Jakub Sokołowski <jakub@status.im>
Cocoapod installations can fail with errors like:
```
tclsh /Users/jenkins/Library/Caches/CocoaPods/Pods/Release/SQLCipher/3.4.2-f9fcf/tool/addopcodes.tcl parse.h.temp >parse.h
./configure: line 11729: tclsh: command not found
./configure: line 12262: tclsh: command not found
./configure: line 12276: tclsh: command not found
```
If a pure shell is used. Also `pgrep` would be missing from a pure shell.
Signed-off-by: Jakub Sokołowski <jakub@status.im>
Before we were patching only `build.gradle`, which is not the only type
of Gradle config file. If we do not cover them all we can encounter
errors about missing package because they will continue using remote
repositories instead of `mavenLocal()`, to which pass Nix store path.
We also need to cover `gradlePluginPortal()` to provide plugins.
This is also necessary for the React Native upgrade:
https://github.com/status-im/status-mobile/pull/15203
Signed-off-by: Jakub Sokołowski <jakub@status.im>
Refactoring the derivation that fetches all the POMs, JARs,
and AARs in order to make it more generic and easier to extend.
The main change is adding `files` key in `deps.json` which contains
a dict of all the files reletad to given package.
This way we can more easily include other files that might be available
for download, like AARs with ASC suffix, or `nodeps` JARs.
This is also necessary for the React Native upgrade:
https://github.com/status-im/status-mobile/pull/15203
Signed-off-by: Jakub Sokołowski <jakub@status.im>
As discovered in:
https://github.com/status-im/status-mobile/pull/15225
The attempt to fix this in:
https://github.com/status-im/status-mobile/pull/15180
But it doesn't appear to work, so instead I'm allowing an override of
`NODE_BINARY` variable and spetting it when defining the Nix shell.
The key things here are:
* Xcode injects its own paths into `PATH` which breaks Nix env.
* Combining Nix shells with `inputsFrom` does not inherit all vars.
It's important to set these variables in `shellHook` and not elsewhere.
Signed-off-by: Jakub Sokołowski <jakub@status.im>
* nix: update gradle dependencies
Signed-off-by: Jakub Sokołowski <jakub@status.im>
* nix: include nodeps JARs in Gradle deps
Signed-off-by: Jakub Sokołowski <jakub@status.im>
* nix: include nodeps JAR for semver4j 0.16.4
Can cause failures like in some cases:
```
A problem occurred configuring project ':react-native-hole-view'.
> Could not resolve all files for configuration ':react-native-hole-view:classpath'.
> Could not find semver4j-0.16.4-nodeps.jar (com.github.gundy:semver4j:0.16.4).
Searched in the following locations:
file:/nix/store/3n2pxsqa2izlx8c23s6jgqai0bqaklm1-status-mobile-maven-deps/com/github/gundy/semver4j/0.16.4/semver4j-0.16.4-nodeps.jar
```
Signed-off-by: Jakub Sokołowski <jakub@status.im>
---------
Signed-off-by: Jakub Sokołowski <jakub@status.im>
Fixes issue with following failure:
```
Failed to construct transformer: Error: error:0308010C:digital envelope routines::unsupported
at new Hash (node:internal/crypto/hash:71:19)
at Object.createHash (node:crypto:130:10)
at stableHash (/Users/jenkins/workspace/status-mobile/platforms/ios/node_modules/metro-cache/src/stableHash.js:19:8)
at JsTransformer.getCacheKey (/Users/jenkins/workspace/status-mobile/platforms/ios/node_modules/metro/src/JSTransformer/worker.js:471:7)
at getTransformCacheKey (/Users/jenkins/workspace/status-mobile/platforms/ios/node_modules/metro/src/DeltaBundler/Transformer/getTransformCacheKey.js:39:29)
at new Transformer (/Users/jenkins/workspace/status-mobile/platforms/ios/node_modules/metro/src/DeltaBundler/Transformer.js:147:28)
at /Users/jenkins/workspace/status-mobile/platforms/ios/node_modules/metro/src/Bundler.js:54:29
at runMicrotasks (<anonymous>)
at processTicksAndRejections (node:internal/process/task_queues:96:5) {
opensslErrorStack: [ 'error:03000086:digital envelope routines::initialization error' ],
library: 'digital envelope routines',
reason: 'unsupported',
code: 'ERR_OSSL_EVP_UNSUPPORTED'
```
https://roytuts.com/how-to-fix-err_ossl_evp_unsupported-in-react-js-application/
Signed-off-by: Jakub Sokołowski <jakub@status.im>
* nix: upgrade zprint from 1.2.4 to 1.2.5
To address issue described in:
https://github.com/kkinnear/zprint/issues/273
Signed-off-by: Jakub Sokołowski <jakub@status.im>
* chore: use zprint :multi-lhs-hang
* refactor: re-format clojure using zprint 1.2.5
---------
Signed-off-by: Jakub Sokołowski <jakub@status.im>
Co-authored-by: yqrashawn <namy.19@gmail.com>
Anton Iakimov
Jakub Sokołowski
Siddarth Kumar
Ibrahem
Mohamed Javid
Jamie Caprani
flexsurfer
Icaro Motta
Ajay Sivan
Alexander
Brian Sztamfater
Michele Balistreri
Parvesh Monu
yqrashawn