We are moving location of symlinks for build derivations for `gcroots`
from `/nix/var/nix/gcroots/per-user` to `.nix-gcroots` in the repo to
avoid errors like this caused by profile migration in `2.14` release:
```
error: creating directory '/nix/var/nix/gcroots/per-user/joe': Permission denied
```
For more details see: https://github.com/NixOS/nix/issues/8564
To upgrade without using `make nix-purge` use `make nix-upgrade`.
Related infra change:
https://github.com/status-im/infra-ci/commit/37c6ce47
Signed-off-by: Jakub Sokołowski <jakub@status.im>
The existing purge script for Darwin would fail and stop at many stages.
This commit makes the script more robust to certain failures that were observed and documented in https://github.com/status-im/status-mobile/issues/16404
`react-native-reanimated` generates write protected files during build time which are problematic because we copy over `node_modules` from `nix` store and when replacing them with new node modules we are manually asked to confirm this overwrite.
This used to happen before with `.cxx` files but now it also happens with `.so` files.
This commit fixes those permissions.
partially fixes#18211
This commit does many things :
- Upgrade `react-native ` to `0.72.5`
- Upgrade `react-native-reanimated` to `3.5.4`
- Upgrade `react-native-navigation` to `7.37.0`
- `ndkVersion` has been bumped to `25.2.9519653`
- `cmakeVersion` has been bumped to `3.22.1`
- `kotlinVersion` has been bumped to `1.7.22`
- `AGP` has been bumped to `7.4.2`
- `Gradle` has been upgraded to `8.0.1`
- Android `CompileSDK` and `TargetSDK` have been bumped to 33
- `@react-native-async-storage/async-storage` has been upgraded to `1.19.3`
- `@walletconnect/client` has been nuked
- some of the old `react-native-reanimated` code has been nuked
- `react-native-keychain` fork has been replaced with `8.1.2`
- On Android we are currently relying on `Hermes` Engine.
- On iOS we are currently relying on `JSC`
- We are not enabling new architecture for now (I have plans for that in the future) ref: https://github.com/status-im/status-mobile/issues/18138
IOS only PR : https://github.com/status-im/status-mobile/pull/16721
Android only PR : https://github.com/status-im/status-mobile/pull/17062
- `make run-metro` now has a target of `android` which was `clojure` earlier, this will increase the time it takes to start metro terminal but this is needed otherwise you will get a nasty error while developing for android locally.
Otherwise if in Nix context we can accidentally use GNU stat which
results in a different output due to different command line flags:
```
[nix-shell:~/status-mobile]$ stat -Lf "%Sg" "/nix/store"
stat: cannot read file system information for '%Sg': No such file or directory
File: "/nix/store"
ID: 10000110000001a Namelen: ? Type: apfs
Block size: 4096 Fundamental block size: 4096
Blocks: Total: 242837545 Free: 199242283 Available: 199242283
Inodes: Total: 7971454780 Free: 7969691320
```
And it should be just owner group name or ID. Which in turn results in:
```
Unknown Nix installtion type!
```
Signed-off-by: Jakub Sokołowski <jakub@status.im>
Android SDK in nixpkg doesn't support `aarch64-darwin`, for details see
https://github.com/status-im/status-mobile/issues/12794
The workaround was to run terminal application with Rosetta or make with `aarch` tool.
This PR overrides system architecture to always be `x86_64` for Android shell and build targets ran by make.
CI is not affected as it runs `nix` directly.
Signed-off-by: Jakub Sokołowski <jakub@status.im>
Otherwise Nix produces warnings like this:
```
warning: ignoring untrusted substituter 'https://nix-cache.status.im/', you are not a trusted user.
```
Since adding users to `trusted-users` essentially gives them `root`:
>Adding a user to trusted-users is essentially equivalent to giving that user root access to the system.
> — https://nixos.org/manual/nix/stable/command-ref/conf-file.html#conf-trusted-users
A more kosher approach might be to just add the cache config itself globally.
Signed-off-by: Jakub Sokołowski <jakub@status.im>
Should resolve errors like this:
```
error: file 'nixpkgs' was not found in the Nix search path (add it using $NIX_PATH or -I)
```
Signed-off-by: Jakub Sokołowski <jakub@status.im>
This way the name of the repo makes at least some sense and
matches the `status-desktop` repo naming.
Also updated `status-jenkins-lib` since it also contained
references to `status-react` repo and job names.
Signed-off-by: Jakub Sokołowski <jakub@status.im>
While working on Nix builds for `go-waku` I figured this derivation
could use some cleanup, to make it shorter and more readable.
Signed-off-by: Jakub Sokołowski <jakub@status.im>
This has several benefits:
* Less abuse of `extra-sandbox-paths` Nix option
* Less inputs to the Android release build derivation
* Easier for users to sign the build themselves
* Simplification of `scripts/release-android.sh`
* Preparation for building using Nix Flakes
The only two remaining credentials passed via `extra-sandbox-paths` is
the Infura and OpenSea API keys, and there is no way around that other
than passing them via Nix arguments, but that would cause them to end up
in `/nix/store` as part of `.drv` files.
I'm also renaming `release-fdroid` to `build-fdroid` to be consistent.
Depends on: https://github.com/status-im/status-jenkins-lib/pull/42
Signed-off-by: Jakub Sokołowski <jakub@status.im>
Minor upgrade done to match the version deployed in CI due to a bug
with `nix-copy-closure` which broken Nix cache jobs:
https://github.com/status-im/infra-ci/issues/49
This will not force upgrade on developes, nor require any `make nix-purge`.
If however and upgrade is wanted it can be simply done with `nix upgrade-nix`.
Signed-off-by: Jakub Sokołowski <jakub@status.im>
Nesting of Nix shells does not work, since we have no way to safely do
that without starting an infinite loop of nested Nix shells.
Signed-off-by: Jakub Sokołowski <jakub@status.im>
This should also fix issues with MacOS sandboxing:
```
sandbox-exec: execvp() of '/nix/store/...-bash-4.4-p23/bin/bash' failed: Operation not permitted
```
https://github.com/cachix/install-nix-action/issues/15
Signed-off-by: Jakub Sokołowski <jakub@status.im>
This should fix the following error showing up on MacOS:
```
error: assertion ((__lessThan 0) ((builtins).stringLength watchmanSockPath))
failed at /Users/jenkins/repos/status-react/nix/mobile/android/watchman.nix:8:9
```
Which was caused by flags like `--argstr` not being passed.
Signed-off-by: Jakub Sokołowski <jakub@status.im>
In order to prevent `nix-store --gc` from removing too much I've:
- Added the `keep-outputs = true` setting to `nix/nix.conf`
- Fixed `nix/scripts/gcroots.sh` to make symlinks in `/nix/var/nix/gcroots`
- Made `nix/scripts/build.sh` and `nix/scripts/shell.sh` use it
This way when running `make nix-gc` most recently used shells and built
derivations won't be removed along with their dependencies.
Signed-off-by: Jakub Sokołowski <jakub@status.im>
Changes:
- MOAR functions so it's easier to follow
- Don't use /tmp, put the copied node_modules in repo root first
- Ignore more node_modules sub folders when checking for modifications
- */unpacked_bin/clj-kondo
- */scripts/.packager.env
Signed-off-by: Jakub Sokołowski <jakub@status.im>
Changes:
- Drop `nix/mobile/android/maven-and-npm-deps/default.nix`
- Replace it with much simpler `nix/tools/patchNodeModules`
- Move Gradle patching tool to `nix/pkgs/patch-maven-srcs`
- Simplify it by using `gradle.deps` and patched node modules separately
- Change `TARGET` for `release-android` to `default`
- Move `mobile/reset-node_modules.sh` to `scripts/node_modules.sh`
- Move `nix/mobile/android/targets/release-android.nix` to `nix/mobile/android/release.nix`
Signed-off-by: Jakub Sokołowski <jakub@status.im>
Changes:
- Fixed `nix/status-go/desktop` builds
- Dropped nimbus wrapper for `status-go` for now
- Split `status-go` builds into subfolders: `mobile`, `desktop`
- Fixed shells for desktop builds: `linux`,`macos`,`windows`
- Added `make status-go-*` targets for building them
- Moved source management to `nix/status-go/source.nix`
- Moved `nix/status-go/build.nix` into `nix/status-go/mobile`
- Moved `nix/desktop/cmake/qtkeychain` to `nix/pkgs/qtkeychain-src`
- Moved `nix/desktop/linux/linuxdeployqt` to `nix/pkgs`
- Moved `nix/desktop/linux/appimagekit` to `nix/pkgs`
- Dropped `nix/tools/mkShell.nix` since it did almost nothing
- Dropped `nix/desktop/cmake/snorenotify` since it's broken
- Moved setup from `nix/tools/mkShell.nix` to `nix/shells.nix`
- Simplified `nix/mobile/ios/status-go-shell.nix`
- Simplified `nix/status-go/default.nix`
- Updated the `nix/DETAILS.md` and `nix/README.md`
- Moved known issues to `nix/KNOWN_ISSUES.md`
- Improved output of `nix/scripts/build.sh`
Signed-off-by: Jakub Sokołowski <jakub@status.im>
Changes:
- Add `nix/DETAILS.md` for more in-depth info
- Rename some of `config.status-im` variables
- Drop `env` argument for Android build
- Use `overlays` instead of `packageOverrides`
- Move the `pkgs` overlay to `nix/overlay.nix`
- Move `nix/status-go/utils.nix` to `nix/tools`
- Make `shell.nix` use the `shells.default` only
- Use `default.nix` as target for `nix/scripts/shell.sh`
- Make `nix/scripts/shell.sh` use `--attr` instead of `target`
- Drop the `target` argument in favour of using `--attr`
- Drop unnecessary `src` from `nix/mobile/ios/default.nix`
- Move `mkShell` and `mergeSh` under `lib`
- Move `patched-go` package to `nix/pkgs` directory
- Move `gomobile` package to `nix/pkgs` directory
- Move `ANDROID_ABI_SPLIT` to `config.status-im.android.abi-split`
- Move `ANDROID_ABI_INCLUDE to `config.status-im.android.abi-include`
Signed-off-by: Jakub Sokołowski <jakub@status.im>
Changes:
* Drop --keep from nix/scripts/build.sh as it does nothing
* Allow scripts/release-android.sh take passwords from env
* Use env file in extra-sandbox-paths to pass keystore passwords
* Quote ever segment of Nix options to avoid parsing errors
Signed-off-by: Jakub Sokołowski <jakub@status.im>
changes:
- Stops scripts/generate-keystore.sh runnig for every new TARGET=android shell
- Moves running scripts/generate-keystore.sh to keystore Makefile target
- Makes generate-keystore.sh accept STATUS_RELEASE_STORE_FILE as env variable
- Fixes error about missing lsof from start-react-native.sh
- Fixes error about unknown -keystore command in scripts/generate-keystore.sh
- Fixes getting OS_NAME without calling Nix shell
- Fixes getting TMP_BUILD_NUMBER without calling Nix shell
- Adds lsof used in start-react-native.sh to Android release Nix derivation
- Simplified how nix/scripts/build.sh appends the --keep flag
- Made use of colors.sh in more scripts
Signed-off-by: Jakub Sokołowski <jakub@status.im>
Changes:
- Add missing lazy assignment for OS_NAME, fixes double Nix shells
- Moved `scripts/add-nix-gcroots.sh` to `nix/scripts/gcroots.sh`
- Moved Nix package manager setup to `nix/scripts/setup.sh`
- Created `nix/scripts/source.sh` to reuse in all Nix scripts
- Created `STARTING_GUIDE.md` with instructions for contributors
- Created `scripts/colors.sh` for definition of shell colors
- Removed `scripts/setup` in favor of `nix/scripts/setup.sh`
- Removed all of `scripts/lib` since it was useless
Signed-off-by: Jakub Sokołowski <jakub@status.im>
Changes:
- Adds a new `nix-gc` Makefile target for removing old packages
- Moves all `nix/*.sh` files to `nix/scripts/*.sh` to make things more tidy
- Renames `TARGET_OS` into `TARGET` and makes it effective only with `nix/scripts/shell.sh`
- Renames `target-os` Nix argument to just `target` and makes it effective only with `shell.nix`
- Drops `IN_CI_ENVIRONMENT` env variable which was useless
- Drops use of `target-os` argument outside of `shell.nix` (with few exceptions, but just in naming)
- `nix/platform.nix` has been made obsolete and removed
- Moves the definition of all major targets to `nix/targets.nix`
- Moves the definition of all major shells to `nix/shells.nix`
- Makes `default.nix` and `shell.nix` just thin wrappers around `nix/default.nix`
- `nix/nixpkgs-bootstrap.nix` has been moved to `nix/pkgs.nix`
- All package and tool overrides have been moved to `nix/pkgs.nix`
- Explicit passing of contents of `pkgs` has been removed in favor of `callPackage` doing it for us
- `nix/bootstrapped-shell.nix` has been moved to `nix/tools/mkShell.nix`
- A new `mergeSh` tool has been added to `pkgs` from `nix/tools/mergeSh.nix`
- This tool is used to merge shells created using `mkShell`
- `mobile/targets/jsbundle.nix` has been moved to `mobile/android/jsbundle/default.nix`
- Moves `status-go` version sanitization to `nix/status-go/utils.nix`
- Renames version to rawVersion and versionName to cleanVersion in status-go derivation
- Ports nix/mobile/ios/install-pods-and-status-go.sh to Nix sub-shells
- Moves adjustment of `inotify/max_user_watches` out into `scripts/inotify_fix.sh`
- Makes iOS builds use the Nix version of Fastlane
Signed-off-by: Jakub Sokołowski <jakub@status.im>
Jakub Sokołowski
Siddarth Kumar
Anton Iakimov
Icaro Motta
Jakub
Pedro Pombeiro
Pedro Pombeiro