Commit Graph

34 Commits

Author SHA1 Message Date
Jakub Sokołowski 83f622d349
nix: make fetching Gradle dependencies more generic
Before what we did was essentially guess what files might exist for any
given package. This approach mostly works, but not entirely.
This is especially problematic when dealing with weird edge case
packages like `react-native`, which you can read about here:

https://github.com/react-native-community/discussions-and-proposals/pull/508
https://github.com/react-native-community/discussions-and-proposals/blob/4a06fc64/proposals/0508-out-of-npm-artifacts.md#the-react-native-android-archive

In order to avoid as much the guessing aspect of fetching Gradle
dependencies we are using both HTML listsings of files and
`artifact-metadata.json` files that exist for more recent packages.

This way we can avoid having to add special edge cases that have been
found out when working on React Native 72 upgrade in:
https://github.com/status-im/status-mobile/pull/17062

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-09-04 09:57:53 +02:00
Anton Iakimov 34f7a48503 nix: map android arch to status-go builds
Fixes partially #15595
In order to build less targets, when not needed we introduce this
mapping logic.
If only specific ABI is required - status-go will have the same
arhitecuture.
2023-08-23 12:15:05 +03:00
Jakub Sokołowski c97559793c
nix: upgrade nixpkgs to latest nixos-22.11
Notable upgrades:

* Bash `5.1` to `5.2`
* Git `2.37.3` to `2.40.1`
* Curl `7.85.0` to `8.0.1`
* OpenSSL `3.0.5` to `3.0.8`
* Go `1.18.6` to `1.18.9`
* NodeJS `18.9.1` to `18.16.0`
* Java `1.8.0_322` to `11.0.11`
* Ruby `3.1.2` to `3.1.4`
* Python `2.7.18` to `3.10.11`
* Clojure `1.11.1.1165` to `1.11.1.1273`
* Clj-kondo `v2022.10.05` to `v2023.04.14`
* Zprint `1.2.5` to `1.2.6`
* Bundler `2.3.22` to `2.4.13`
* Gradle `6.9.2` to `6.9.4`
* Android Platform Tools `33.0.2` to `33.0.3`
* Android SDK Tools to Android SDK Command-Line Tools

Removals:

* Zprint since the version in `nixpkgs` was newer than in overlay.
* Xcode wrapper definition was removed since my fixes were merged:
  - https://github.com/NixOS/nixpkgs/pull/204278
  - https://github.com/NixOS/nixpkgs/pull/228696

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-06-20 12:53:06 +02:00
Siddarth Kumar 350303256b
upgrade `react-native` from 0.67.5 to 0.69.10 (#16016) 2023-06-14 07:17:41 +05:30
Jakub Sokołowski 3e8647d626
nix: add ios-deploy package to ios shell
Also adds an assertion that gives a better error when trying to use an
`ios` Nix shell on an unsupported operating system.

Resolves:
https://github.com/status-im/status-mobile/issues/15695

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-04-20 20:17:38 +02:00
Jakub Sokołowski d71cfd12c1
nix: unpatched Node modules for Gradle deps update
Otherwise we can end up with Gradle failing to find the dependencies
because we've patched away all entries referencing external repos.

Also made the regex in AWS parser a but more strict.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-03-21 15:21:58 +01:00
yqrashawn 39e29cfb5a
feat: replace clj-fmt with zprint (#14520) 2022-12-20 21:57:49 +08:00
Icaro Motta 5d775d93fa
Use native binary for the clj-kondo linter (#14179) 2022-10-19 09:54:01 -03:00
Jakub Sokołowski bcf919fcbb
nix: use android-sdk shell for fdroid-pr target
Also renamed `android-env` shell to `android-sdk` to be more descriptive.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2022-09-29 12:54:47 +02:00
Jakub Sokołowski ca8b721135
nix: fix order of iOS shell components
Possible fix for errors like:
```
bundler: failed to load command: fastlane (/Users/jenkins/.bundle/ruby/2.7.0/bin/fastlane)
/Users/jenkins/.bundle/ruby/2.7.0/gems/fastlane-2.205.2/fastlane_core/lib/fastlane_core/ui/interface.rb:153:in `shell_error!': [!] Shell command exited with exit status 51 instead of 0. (FastlaneCore::Interface::FastlaneShellError)
```

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2022-08-30 09:27:49 +02:00
Jakub Sokołowski c27a74e378
nix: quote path uses to avoid issues with spaces
This is most important on MacOS, but in general is a good idea.

Resolves:
https://github.com/status-im/status-mobile/issues/13715

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2022-07-27 16:11:28 +02:00
Jakub Sokołowski 1f7fd17ff1
rename status-react to status-mobile
This way the name of the repo makes at least some sense and
matches the `status-desktop` repo naming.

Also updated `status-jenkins-lib` since it also contained
references to `status-react` repo and job names.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2022-07-17 14:46:16 +02:00
Jakub Sokołowski 5d9a29b3c8
nix: cleanup of unused variables
Mostly achieved by running via `nix-linter`.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2022-05-31 12:03:20 +02:00
Jakub Sokołowski acfa73ab43
nix: build unsigned Android APK, sign separately
This has several benefits:

* Less abuse of `extra-sandbox-paths` Nix option
* Less inputs to the Android release build derivation
* Easier for users to sign the build themselves
* Simplification of `scripts/release-android.sh`
* Preparation for building using Nix Flakes

The only two remaining credentials passed via `extra-sandbox-paths` is
the Infura and OpenSea API keys, and there is no way around that other
than passing them via Nix arguments, but that would cause them to end up
in `/nix/store` as part of `.drv` files.

I'm also renaming `release-fdroid` to `build-fdroid` to be consistent.

Depends on: https://github.com/status-im/status-jenkins-lib/pull/42

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2022-05-21 11:10:11 +02:00
Jakub Sokołowski 200eb189ea
nix: fix use of androidPkgs in gradle shell
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2021-10-26 15:39:54 +02:00
Jakub Sokołowski 293fd5fae1
nix: add missing cljfmt dependencies to nix/deps/clojure
It seems like this worked before purely because the `cljfmt` library was
already in the `~/.m2` cache folder. This issue was noticed when I
cleaned up the `~/.m2` folder on one Jenkins slave host and the Lint stage
started randomly failing.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
Signed-off-by: andrey <motor4ik@gmail.com>
2021-03-01 16:42:38 +01:00
Jakub Sokołowski 112132aa94
drop all desktop related files and references
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2020-07-08 10:26:51 +02:00
Jakub Sokołowski dbe3e0a3f9
nix: minor cleanup and documentation updates
Changes:
- Simplified `default.nix` and `shell.nix`
- Moved the default shell to `nix/shell.nix`
- Dropped unnecessary merge from  `nix/shells.nix`
- Fixed `nix/lib/getConfig.nix` to return default on `null`
- Expanded `nix/DETAILS.md` with more info
- Added links to presentations in `nix/README.md`
- Fixed a few typos

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2020-06-05 14:49:34 +02:00
Jakub Sokołowski 12e67ad6d2
nix: improve nix/scripts/node_modules.sh
Changes:
- MOAR functions so it's easier to follow
- Don't use /tmp, put the copied node_modules in repo root first
- Ignore more node_modules sub folders when checking for modifications
  - */unpacked_bin/clj-kondo
  - */scripts/.packager.env

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2020-06-02 16:26:19 +02:00
Jakub Sokołowski d0216674da
nix: use go-maven-resolver to fix nix-update-gradle
This Go package was implemented by me to avoid having to use Mavan or
Gradle to resolve all the dependencies for Nix:
https://github.com/status-im/go-maven-resolver

The Go tool is essentially a slightly smarter and more efficient version
of the `get_urls.sh` script, which was mostly slow due to the `mvn
dependency:list` calls and slow HTTP requests.

This takes down the `make nix-update-gradle` target to under 2 minutes
on my machine.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2020-05-24 18:08:05 +02:00
Jakub Sokołowski 883af7b57a
nix: add missing Gradle repo path in gradle shell
Also rename STATUSREACT_NIX_MAVEN_REPO to STATUS_NIX_MAVEN_REPO.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2020-05-20 13:59:49 +02:00
Jakub Sokołowski 604362958b
nix: always use patched node_modules in shells
Becuase `clojure` shell used unpatched `node_modules` while the
`android` shell used the patched version when starting these shells
the `node_modules` would keep being switched between them wasting time.

Changes:
- Move `nix/tools/patchNodeModules.nix` to `nix/deps/nodejs-patched`
- Make `nix/deps/nodejs-patched` a normal derivation without extra arguments
- Use `deps.nodejs-patched` in `nix/mobile/android` shell
- Use `deps.nodejs-patched` in `nix/shells.nix` node shell
- Use `with pkgs` to reduce arguments in `nix/mobile/android/release.nix`

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2020-05-18 13:54:26 +02:00
Jakub Sokołowski 2f9593a7d2
unify multiple Makefile targets
Changes:
- Drop a bunch of `watch-{android,ios}-*` tagets
- Replace them with one `run-clojure`
- Drop a bunch of `react-native-*` targets
- Replace them with one `run-metro`
- Replace `run-{android,ios}` with `run-{android,ios}`
- Drop `startdev-{android,ios,desktop}*` targets
- Drop `prod-build-{android,ios}` as deprecated
- Drop `src/status_im/android/core.cljs`
- Drop `src/status_im/ios/core.cljs`
- Move `lsof` tool to `default` shell
- Replace them with one `init` `src/status_im/core.cljs`
- Use `init` in one `shadow-cljs.edn` target `mobile`
- Use `mobile` target in `nix/mobile/android/jsbundle`
- Update instructions in `STARTING_GUIDE.md`
- Use `gradle` shell for `android-clean` target

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2020-05-15 18:29:31 +02:00
Jakub Sokołowski 707a9b7095
nix: add status-go shell
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2020-05-15 16:19:34 +02:00
Jakub Sokołowski 73a8992db7
nix: refactor loading of node.js modules
Changes:
- Drop `nix/mobile/android/maven-and-npm-deps/default.nix`
- Replace it with much simpler `nix/tools/patchNodeModules`
- Move Gradle patching tool to `nix/pkgs/patch-maven-srcs`
- Simplify it by using `gradle.deps` and patched node modules separately
- Change `TARGET` for `release-android` to `default`
- Move `mobile/reset-node_modules.sh` to `scripts/node_modules.sh`
- Move `nix/mobile/android/targets/release-android.nix` to `nix/mobile/android/release.nix`

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2020-05-15 16:19:27 +02:00
Jakub Sokołowski ed4d0a1ed9
nix: refactor updating Gradle dependencies
changes:
- Moved Gradle deps setup to `nix/deps/gradle`
- Dropped `nix/mobile/android/maven-and-npm-deps/maven`
- Used GNU Parallel to optimize `nix/deps/gradle/generate.sh`
- Move Maven+Node shell setup from `release-android.nix` to Android shell
- Moved AAPT2 patching to `nix/pkgs/aapt2`
- Drop `patchPhase` and `gradlew` use from `release-android.nix`
- Simplify symlinking `{mobile,desktop}/js_files/*`
- Add new `nodejs` shell and merge it with `clojure` and `gradle`
- Re-created Gradle deps files with new scripts:
  - `nix/deps/gradle/proj.list`
  - `nix/deps/gradle/deps.list`
  - `nix/deps/gradle/deps.urls`
  - `nix/deps/gradle/deps.nix`

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2020-05-14 09:46:30 +02:00
Jakub Sokołowski 42fb40476c
nix: reafactoring of status-go builds
Changes:
- Fixed `nix/status-go/desktop` builds
- Dropped nimbus wrapper for `status-go` for now
- Split `status-go` builds into subfolders: `mobile`, `desktop`
- Fixed shells for desktop builds: `linux`,`macos`,`windows`
- Added `make status-go-*` targets for building them
- Moved source management to `nix/status-go/source.nix`
- Moved `nix/status-go/build.nix` into `nix/status-go/mobile`
- Moved `nix/desktop/cmake/qtkeychain` to `nix/pkgs/qtkeychain-src`
- Moved `nix/desktop/linux/linuxdeployqt` to `nix/pkgs`
- Moved `nix/desktop/linux/appimagekit` to `nix/pkgs`
- Dropped `nix/tools/mkShell.nix` since it did almost nothing
- Dropped `nix/desktop/cmake/snorenotify` since it's broken
- Moved setup from `nix/tools/mkShell.nix` to `nix/shells.nix`
- Simplified `nix/mobile/ios/status-go-shell.nix`
- Simplified `nix/status-go/default.nix`
- Updated the `nix/DETAILS.md` and `nix/README.md`
- Moved known issues to `nix/KNOWN_ISSUES.md`
- Improved output of `nix/scripts/build.sh`

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2020-05-04 20:55:07 +02:00
Jakub Sokołowski a40d599e55
fix broken nix-update-clojure by always using GNU grep
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2020-05-03 23:37:04 +02:00
yenda d5ef218584
use shadow-cljs
Signed-off-by: Jakub Sokołowski <jakub@status.im>
Signed-off-by: yenda <eric@status.im>
2020-04-30 15:43:46 +02:00
Jakub Sokołowski 7b6b620ceb
nix: use overlays instead of packageOverrides
Changes:
- Add `nix/DETAILS.md` for more in-depth info
- Rename some of `config.status-im` variables
- Drop `env` argument for Android build
- Use `overlays` instead of `packageOverrides`
- Move the `pkgs` overlay to `nix/overlay.nix`
- Move `nix/status-go/utils.nix` to `nix/tools`
- Make `shell.nix` use the `shells.default` only
- Use `default.nix` as target for `nix/scripts/shell.sh`
- Make `nix/scripts/shell.sh` use `--attr` instead of `target`
- Drop the `target` argument in favour of using `--attr`
- Drop unnecessary `src` from `nix/mobile/ios/default.nix`
- Move `mkShell` and `mergeSh` under `lib`
- Move `patched-go` package to `nix/pkgs` directory
- Move `gomobile` package to `nix/pkgs` directory
- Move `ANDROID_ABI_SPLIT` to `config.status-im.android.abi-split`
- Move `ANDROID_ABI_INCLUDE to `config.status-im.android.abi-include`

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2020-04-29 11:53:33 +02:00
Jakub Sokołowski 2f412d95ee
nix: rename adb target to android-env
Makes more sense since it contains entire Android SDK and NDK.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2020-03-16 12:11:08 +01:00
Jakub Sokołowski de53581ef4
nix: don't generate keystore inside of Nix build
changes:
- Stops scripts/generate-keystore.sh runnig for every new TARGET=android shell
- Moves running scripts/generate-keystore.sh to keystore Makefile target
- Makes generate-keystore.sh accept STATUS_RELEASE_STORE_FILE as env variable
- Fixes error about missing lsof from start-react-native.sh
- Fixes error about unknown -keystore command in scripts/generate-keystore.sh
- Fixes getting OS_NAME without calling Nix shell
- Fixes getting TMP_BUILD_NUMBER without calling Nix shell
- Adds lsof used in start-react-native.sh to Android release Nix derivation
- Simplified how nix/scripts/build.sh appends the --keep flag
- Made use of colors.sh in more scripts

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2020-03-05 10:56:54 +01:00
Pedro Pombeiro e220ecbb0a
Cleanup
Signed-off-by: Pedro Pombeiro <pombeirp@users.noreply.github.com>
2020-01-10 19:59:54 +01:00
Jakub Sokołowski aca703a011
major nix refactor
Changes:
- Adds a new `nix-gc` Makefile target for removing old packages
- Moves all `nix/*.sh` files to `nix/scripts/*.sh` to make things more tidy
- Renames `TARGET_OS` into `TARGET` and makes it effective only with `nix/scripts/shell.sh`
- Renames `target-os` Nix argument to just `target` and makes it effective only with `shell.nix`
- Drops `IN_CI_ENVIRONMENT` env variable which was useless
- Drops use of `target-os` argument outside of `shell.nix` (with few exceptions, but just in naming)
- `nix/platform.nix` has been made obsolete and removed
- Moves the definition of all major targets to `nix/targets.nix`
- Moves the definition of all major shells to `nix/shells.nix`
- Makes `default.nix` and `shell.nix` just thin wrappers around `nix/default.nix`
- `nix/nixpkgs-bootstrap.nix` has been moved to `nix/pkgs.nix`
- All package and tool overrides have been moved to `nix/pkgs.nix`
- Explicit passing of contents of `pkgs` has been removed in favor of `callPackage` doing it for us
- `nix/bootstrapped-shell.nix` has been moved to `nix/tools/mkShell.nix`
- A new `mergeSh` tool has been added to `pkgs` from `nix/tools/mergeSh.nix`
  - This tool is used to merge shells created using `mkShell`
- `mobile/targets/jsbundle.nix` has been moved to `mobile/android/jsbundle/default.nix`
- Moves `status-go` version sanitization to `nix/status-go/utils.nix`
- Renames version to rawVersion and versionName to cleanVersion in status-go derivation
- Ports nix/mobile/ios/install-pods-and-status-go.sh to Nix sub-shells
- Moves adjustment of `inotify/max_user_watches` out into `scripts/inotify_fix.sh`
- Makes iOS builds use the Nix version of Fastlane

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2019-12-12 18:51:21 +01:00