## Summary
This commit also points to status-go branch where we have upgraded go to 1.20
Related status-go PR -> https://github.com/status-im/status-go/pull/5027
### Testing notes
Please test everything, specially the store node stuff.
#### Platforms
- Android
- iOS
status: ready
fixes#18311
This PR upgrades `nixpkgs` to latest release version of 23-11
ref -> https://github.com/NixOS/nixpkgs/releases/tag/23.11
- `Gradle` from `8.0.1` -> `8.4`
- `Git` from `2.40.1` -> `2.42.0`
- `Curl` from `8.0.1` -> `8.4.0`
- `OpenSSL` from `3.0.8` -> `3.0.13`
- `NodeJS` from `18.16.0` -> `18.19.1`
- `Python` from `3.10.11` -> `3.10.13`
- `Clojure` from `1.11.1.1273` -> `1.11.1.1413`
- and some more stuff
- `xcbeautify` , this was added in 23-11 ->https://github.com/NixOS/nixpkgs/pull/289446
- `idb-companion`, this was also added in 23-11 -> https://github.com/NixOS/nixpkgs/pull/296440
fixes#18291
## Summary
Changes worth mentioning are :
- More hacks/patches
- Force app to use `Java 17` everywhere to compile `kotlin,java`
- `gems` were upgraded after a long time
- `aapt2` was bumped to `8.1.1`
- `metro` is now at `0.80.4`
- `xcbeautify` was bumped to `1.4.0`
- `@react-native-community/clipboard` lib was replaced with `@react-native-clipboard/clipboard`
- `react-native-dialogs` lib was upgraded to `1.1.2`
- `react-native-gesture-handler` lib was upgraded to `2.14.1`
- `react-native-navigation` was upgraded to `7.37.2`
## Platforms
- Android
- iOS
fixes#16310
We used to reply on `react-native cli` and would pass a `--device` flag to deploy the debug variant of `iOS` app on connected `iPhone`.
`react-native cli` under the hood uses `ios-deploy` library to achieve this functionality.
This showed many weird issues, specifically in locating connected devices and failures at build step with ambiguous error messages.
This commit fixes it by using our custom script `run-ios-devices.sh` which does not rely on `ios-deploy`.
We use `libimobiledevice` to identify `UDID` of a connected `iPhone`.
We use `xcrun devicectl device install app` and `xcrun devicectl device process launch` to install and launch the app.
This works well with `Xcode 15` and `iOS 17.x`.
We can now remove `ios-deploy` from `iOS` shell and `nix` overlay.
We also set up a logs folder and add a Readme.
## Review notes
- connect your iPhone to your Laptop via a cable
- `make run-clojure`
- `make run-ios-device`
(note: no need to pass device name now)
## Platforms
- iOS
The purpose of this commit is to allow running `maestro` automations to enhance productivity by **automating** mundane tasks.
Other flows like
- send contact request
- sync devices
- join a community
etc will be a part of follow ups.
In this commit we provide a `make` command to run a very common task like creating account & login :
`make auto-login`
We also allow another `make` command to run any other custom flow :
`make auto-custom FLOW="maestro/create-account-or-login.yaml"`
A `maestro` folder has been added with 3 `yaml` files which are very easy to understand
ref : https://maestro.mobile.dev/api-reference/commands
I can't remember the number of times I have had to ask developers to run `make run-ios | xcbeautify` when debugging iOS build failures and they do not have `xcbeautify` installed on their environment.
`xcbeautify` helps make `xcodebuild` output more readable and to identify problems quickly.
This commit adds `xcbeautify` to iOS shell and to `make run-ios` so that we get readable output by default.
This commit does many things :
- Upgrade `react-native ` to `0.72.5`
- Upgrade `react-native-reanimated` to `3.5.4`
- Upgrade `react-native-navigation` to `7.37.0`
- `ndkVersion` has been bumped to `25.2.9519653`
- `cmakeVersion` has been bumped to `3.22.1`
- `kotlinVersion` has been bumped to `1.7.22`
- `AGP` has been bumped to `7.4.2`
- `Gradle` has been upgraded to `8.0.1`
- Android `CompileSDK` and `TargetSDK` have been bumped to 33
- `@react-native-async-storage/async-storage` has been upgraded to `1.19.3`
- `@walletconnect/client` has been nuked
- some of the old `react-native-reanimated` code has been nuked
- `react-native-keychain` fork has been replaced with `8.1.2`
- On Android we are currently relying on `Hermes` Engine.
- On iOS we are currently relying on `JSC`
- We are not enabling new architecture for now (I have plans for that in the future) ref: https://github.com/status-im/status-mobile/issues/18138
IOS only PR : https://github.com/status-im/status-mobile/pull/16721
Android only PR : https://github.com/status-im/status-mobile/pull/17062
- `make run-metro` now has a target of `android` which was `clojure` earlier, this will increase the time it takes to start metro terminal but this is needed otherwise you will get a nasty error while developing for android locally.
After upgrading MacOS to Sonoma and Xcode to 15, pod install stage would fail with clang errors.
Turns out that react-native's configure glog script sets 2 additional flags, CC and CXX which conflicts with flags already set in nix iOS shell.
In this commit we :
- bump clang from 10.0.0 to 15.0.7
- patch glog configure script to avoid setting those flags
This fixes the iOS builds.
Since the `default` shell doesn't have Ruby, the `nix-update-gems`
target would incorrectly use the system Ruby instead of the one from Nix.
Signed-off-by: Jakub Sokołowski <jakub@status.im>
watchman was upgraded significantly during the last #14944 (4.9.0 (Aug 16, 2017) to 2023.01.30.00 - 6 years between):
status-im/nixpkgs@4e9c02b
Probably causing developers to have "too many files open" issue #16341
This PR is an attempt to fix the issue by downgrading the watchman
Signed-off-by: Jakub Sokołowski <jakub@status.im>
On macOS `git-lfs` checks fail sometimes and they also take a lot of time to execute.
```log
FAIL github.com/git-lfs/git-lfs/v3/lfshttp 0.820s
FAIL
error: builder for '/nix/store/abd6spw876qvcmlpcwsljsnbpbnvx1fg-git-lfs-3.3.0.drv' failed with exit code 1;
last 10 log lines:
> Error Trace: certs_test.go:210
> Error: Expected value not to be nil.
> Test: TestCertFromSSLCAPathEnv
> certs_test.go:210:
> Error Trace: certs_test.go:210
> Error: Expected value not to be nil.
> Test: TestCertFromSSLCAPathEnv
> FAIL
> FAIL github.com/git-lfs/git-lfs/v3/lfshttp 0.820s
> FAIL
For full logs, run 'nix-store -l /nix/store/abd6spw876qvcmlpcwsljsnbpbnvx1fg-git-lfs-3.3.0.drv'.
```
Failures initially observed by me and @smohamedjavid and then today again reported by @yqrashawn
This commit adds a `nix` overlay that skips those checks
Notable upgrades:
* Bash `5.1` to `5.2`
* Git `2.37.3` to `2.40.1`
* Curl `7.85.0` to `8.0.1`
* OpenSSL `3.0.5` to `3.0.8`
* Go `1.18.6` to `1.18.9`
* NodeJS `18.9.1` to `18.16.0`
* Java `1.8.0_322` to `11.0.11`
* Ruby `3.1.2` to `3.1.4`
* Python `2.7.18` to `3.10.11`
* Clojure `1.11.1.1165` to `1.11.1.1273`
* Clj-kondo `v2022.10.05` to `v2023.04.14`
* Zprint `1.2.5` to `1.2.6`
* Bundler `2.3.22` to `2.4.13`
* Gradle `6.9.2` to `6.9.4`
* Android Platform Tools `33.0.2` to `33.0.3`
* Android SDK Tools to Android SDK Command-Line Tools
Removals:
* Zprint since the version in `nixpkgs` was newer than in overlay.
* Xcode wrapper definition was removed since my fixes were merged:
- https://github.com/NixOS/nixpkgs/pull/204278
- https://github.com/NixOS/nixpkgs/pull/228696
Signed-off-by: Jakub Sokołowski <jakub@status.im>
Also adds an assertion that gives a better error when trying to use an
`ios` Nix shell on an unsupported operating system.
Resolves:
https://github.com/status-im/status-mobile/issues/15695
Signed-off-by: Jakub Sokołowski <jakub@status.im>
* [IOS Only] react-native 0.63 to 0.67
* [Android Only] react-native 0.63 to 0.67
* bring back all the jenkinsfiles
* make auto-complete prop for text-input compatible
* [IOS Only] react-native 0.63 to 0.67
* [Android Only] react-native 0.63 to 0.67
* bring back all the jenkinsfiles
* nix: drop unnecessary set -x from status-go build
Signed-off-by: Jakub Sokołowski <jakub@status.im>
* add explicity implementation line for soloader
And add deleteDebugFilesForVariant fix for libhermes.
Signed-off-by: Jakub Sokołowski <jakub@status.im>
* use fast-image for link previews
* fix extra line in message composer on android
This elevation prop is un-necessary and causes an extra line to appear which looks like a border but its actually a shadow.
* don't use `fast-image` for url preview favicon
* fix audio record button interfering cancel android
This fix was not needed in react-native 0.63.4 but is needed after we upgraded to 0.67.5
* get rid of unused platform import
---------
Co-authored-by: Jakub Sokołowski <jakub@status.im>
* nix: upgrade zprint from 1.2.4 to 1.2.5
To address issue described in:
https://github.com/kkinnear/zprint/issues/273
Signed-off-by: Jakub Sokołowski <jakub@status.im>
* chore: use zprint :multi-lhs-hang
* refactor: re-format clojure using zprint 1.2.5
---------
Signed-off-by: Jakub Sokołowski <jakub@status.im>
Co-authored-by: yqrashawn <namy.19@gmail.com>
Notable upgrades:
* Go `1.17.11` to `1.18.6`
* NodeJS `16.15.0` to `16.17.1`
* Clojure `1.11.1.1139` to `1.11.1.1165`
* Ruby Gem `3.2.26` to `3.3.20`
* Bundler `2.3.9` to `2.3.22`
* Git `2.36.1` to `2.37.3`
* Curl `7.83.1` to `7.85.0`
* OpenSSL `1.1.1o` to `3.0.5`
* PatchELF `0.14.5` to `0.15.0`
* Android SDK Platform Tools `33.0.1` to `33.0.2`
Signed-off-by: Jakub Sokołowski <jakub@status.im>
This resolves issues with weird `status-go` build errors like:
```
cgo: C compiler "2022-09-07" not found: exec: "2022-09-07": executable file not found in $PATH
```
Such errors are caused by `xcrun` spewing warnings to `stdout`. The PR
to fix this issue permanently has been created but it might take a while
before it's merged.
Fix PR: https://github.com/golang/mobile/pull/84
Resolves: https://github.com/status-im/status-mobile/issues/13949
Signed-off-by: Jakub Sokołowski <jakub@status.im>
If we keep using specific `buildGo117Package` we can easily forge to
upgrade when we bump the Go compiler itself. By locking those explicitly
in `overlay.nix` we make sure they all get bumped together.
Signed-off-by: Jakub Sokołowski <jakub@status.im>
Notable software upgrades:
- Go `1.16.8` to `1.17.3`
- NodeJS `12.22.7` to `16.14.2`
- OpenJDK `8u272-b10` to `8u322-ga`
- Clojure `1.10.3.1029` to `1.11.1.1107`
- Git `2.33.1` to `2.35.1`
- CMake `3.18.1` to `3.22.3`
- Curl `7.79.1` to `7.82.0`
- GNU Awk `5.1.0` to `5.1.1`
The Go upgrade is done in advance of upgrade to `1.18.1`.
The Node.js upgrade is done since `12.x` security support [is over](https://endoflife.date/nodejs).
Signed-off-by: Jakub Sokołowski <jakub@status.im>
With upgrade to Go above `1.15` and Gomobile we now drop support for darwin/arm32 builds.
This means all devices like iPhone 5 or older.
For more details see:
https://tip.golang.org/doc/go1.15#darwin
Signed-off-by: Jakub Sokołowski <jakub@status.im>
This makes use of the refactoring of Android SDK done in:
https://github.com/NixOS/nixpkgs/pull/89775
Which allows us to drop the use of our own fork of `nixpkgs`.
Android Upgrades:
* Build Tools - `29.0.2` to `30.0.3`
* Platform Tools - `29.0.6` to `30.0.5`
* NDK Bundle - `21.0.6113669` to `21.3.6528147`
Other Upgrades:
* Git - `2.28.0` to `2.29.2`
* Go - `1.14.7` to `1.14.13`
* Clojure - `1.10.1.645` to `1.10.1.763`
* NodeJS - `12.18.3` to `12.20.1`
* Yarn - `1.22.4` to `1.22.10`
* OpenJDK - `8u265-ga` to `8u272-b10`
* PatchElf - `0.11` to `0.12`
* CoreUtils - `8.31` to `8.32`
Signed-off-by: Jakub Sokołowski <jakub@status.im>
Changes:
* Create `nix/config.nix` with `config` defaults
* Add `nix/tools/gradlePropParser.nix` for reading `gradle.properties`
* Add `nix/mobile/android/keystore.nix` for generating a keystore
* Load keystore generation in `nix/mobile/android/default.nix`
* Use generated keystore if it's not provided via `config`
* Add `-deststoretype pkcs12` in `scripts/generate-keystore.sh`
* Add `nix/lib/assertEnvVarSet.nix` for checking if env var is set
Signed-off-by: Jakub Sokołowski <jakub@status.im>
This Go package was implemented by me to avoid having to use Mavan or
Gradle to resolve all the dependencies for Nix:
https://github.com/status-im/go-maven-resolver
The Go tool is essentially a slightly smarter and more efficient version
of the `get_urls.sh` script, which was mostly slow due to the `mvn
dependency:list` calls and slow HTTP requests.
This takes down the `make nix-update-gradle` target to under 2 minutes
on my machine.
Signed-off-by: Jakub Sokołowski <jakub@status.im>
Becuase `clojure` shell used unpatched `node_modules` while the
`android` shell used the patched version when starting these shells
the `node_modules` would keep being switched between them wasting time.
Changes:
- Move `nix/tools/patchNodeModules.nix` to `nix/deps/nodejs-patched`
- Make `nix/deps/nodejs-patched` a normal derivation without extra arguments
- Use `deps.nodejs-patched` in `nix/mobile/android` shell
- Use `deps.nodejs-patched` in `nix/shells.nix` node shell
- Use `with pkgs` to reduce arguments in `nix/mobile/android/release.nix`
Signed-off-by: Jakub Sokołowski <jakub@status.im>
Changes:
- Drop `nix/mobile/android/maven-and-npm-deps/default.nix`
- Replace it with much simpler `nix/tools/patchNodeModules`
- Move Gradle patching tool to `nix/pkgs/patch-maven-srcs`
- Simplify it by using `gradle.deps` and patched node modules separately
- Change `TARGET` for `release-android` to `default`
- Move `mobile/reset-node_modules.sh` to `scripts/node_modules.sh`
- Move `nix/mobile/android/targets/release-android.nix` to `nix/mobile/android/release.nix`
Signed-off-by: Jakub Sokołowski <jakub@status.im>
changes:
- Moved Gradle deps setup to `nix/deps/gradle`
- Dropped `nix/mobile/android/maven-and-npm-deps/maven`
- Used GNU Parallel to optimize `nix/deps/gradle/generate.sh`
- Move Maven+Node shell setup from `release-android.nix` to Android shell
- Moved AAPT2 patching to `nix/pkgs/aapt2`
- Drop `patchPhase` and `gradlew` use from `release-android.nix`
- Simplify symlinking `{mobile,desktop}/js_files/*`
- Add new `nodejs` shell and merge it with `clojure` and `gradle`
- Re-created Gradle deps files with new scripts:
- `nix/deps/gradle/proj.list`
- `nix/deps/gradle/deps.list`
- `nix/deps/gradle/deps.urls`
- `nix/deps/gradle/deps.nix`
Signed-off-by: Jakub Sokołowski <jakub@status.im>
Changes:
- Fixed `nix/status-go/desktop` builds
- Dropped nimbus wrapper for `status-go` for now
- Split `status-go` builds into subfolders: `mobile`, `desktop`
- Fixed shells for desktop builds: `linux`,`macos`,`windows`
- Added `make status-go-*` targets for building them
- Moved source management to `nix/status-go/source.nix`
- Moved `nix/status-go/build.nix` into `nix/status-go/mobile`
- Moved `nix/desktop/cmake/qtkeychain` to `nix/pkgs/qtkeychain-src`
- Moved `nix/desktop/linux/linuxdeployqt` to `nix/pkgs`
- Moved `nix/desktop/linux/appimagekit` to `nix/pkgs`
- Dropped `nix/tools/mkShell.nix` since it did almost nothing
- Dropped `nix/desktop/cmake/snorenotify` since it's broken
- Moved setup from `nix/tools/mkShell.nix` to `nix/shells.nix`
- Simplified `nix/mobile/ios/status-go-shell.nix`
- Simplified `nix/status-go/default.nix`
- Updated the `nix/DETAILS.md` and `nix/README.md`
- Moved known issues to `nix/KNOWN_ISSUES.md`
- Improved output of `nix/scripts/build.sh`
Signed-off-by: Jakub Sokołowski <jakub@status.im>
Changes:
- Add `nix/DETAILS.md` for more in-depth info
- Rename some of `config.status-im` variables
- Drop `env` argument for Android build
- Use `overlays` instead of `packageOverrides`
- Move the `pkgs` overlay to `nix/overlay.nix`
- Move `nix/status-go/utils.nix` to `nix/tools`
- Make `shell.nix` use the `shells.default` only
- Use `default.nix` as target for `nix/scripts/shell.sh`
- Make `nix/scripts/shell.sh` use `--attr` instead of `target`
- Drop the `target` argument in favour of using `--attr`
- Drop unnecessary `src` from `nix/mobile/ios/default.nix`
- Move `mkShell` and `mergeSh` under `lib`
- Move `patched-go` package to `nix/pkgs` directory
- Move `gomobile` package to `nix/pkgs` directory
- Move `ANDROID_ABI_SPLIT` to `config.status-im.android.abi-split`
- Move `ANDROID_ABI_INCLUDE to `config.status-im.android.abi-include`
Signed-off-by: Jakub Sokołowski <jakub@status.im>
Siddarth Kumar
Jakub Sokołowski
Icaro Motta
Anton Iakimov
Ibrahem Khalil