This passing of Watchman socket was implemented in order to avoid this:
```
Error: EMFILE: too many open files, watch
at FSEvent.FSWatcher._handle.onchange (node:internal/fs/watchers:204:21)
Emitted 'error' event on NodeWatcher instance at:
at NodeWatcher.checkedEmitError (/private/tmp/nix-build-status-mobile-build-nightly-android.drv-0/node_modules/sane/src/node_watcher.js:143:12)
at FSWatcher.emit (node:events:527:28)
at FSEvent.FSWatcher._handle.onchange (node:internal/fs/watchers:210:12) {
errno: -24,
syscall: 'watch',
code: 'EMFILE',
filename: null
}
```
Which is caused by `jest-haste-map` used by `metro` starting to watch
the filesystem for file changes, which is pointless when doing a
one-off build using Nix.
But by setting `CI=true` we can make `metro` not start this waching of
files in the first place, removing the need for use of Watchman entirely.
By entirely dropping use of Watchman we also fix the following issue:
```
[cli] unable to talk to your watchman on /tmp/tmp-status-mobile-ABC/jenkins-state/sock! (Permission denied)
```
Which happens on multi-user Nix installations becuase the user that the
Nix build is executed as is not the same as the user that starts
Watchman and creates the socket file.
Issue: https://github.com/status-im/status-mobile/issues/13783
Signed-off-by: Jakub Sokołowski <jakub@status.im>
This way the name of the repo makes at least some sense and
matches the `status-desktop` repo naming.
Also updated `status-jenkins-lib` since it also contained
references to `status-react` repo and job names.
Signed-off-by: Jakub Sokołowski <jakub@status.im>
This has several benefits:
* Less abuse of `extra-sandbox-paths` Nix option
* Less inputs to the Android release build derivation
* Easier for users to sign the build themselves
* Simplification of `scripts/release-android.sh`
* Preparation for building using Nix Flakes
The only two remaining credentials passed via `extra-sandbox-paths` is
the Infura and OpenSea API keys, and there is no way around that other
than passing them via Nix arguments, but that would cause them to end up
in `/nix/store` as part of `.drv` files.
I'm also renaming `release-fdroid` to `build-fdroid` to be consistent.
Depends on: https://github.com/status-im/status-jenkins-lib/pull/42
Signed-off-by: Jakub Sokołowski <jakub@status.im>
This script fetches a specified APK and analyzes it for values like
version code or commit and then based on that creates a branchy and a
commit in `fdroiddata` repository that can be used to create a release PR.
Signed-off-by: Jakub Sokołowski <jakub@status.im>
People on HackerOne keep reporting this file as containing a secret token.
The reality is that this token is no longer usable since it was
published to a public repo, which would have caused it to be revoked
automatically though GitHub automation:
https://docs.github.com/en/code-security/secret-scanning/about-secret-scanning
Other than that I'm pretty sure nobody is using this script.
Signed-off-by: Jakub Sokołowski <jakub@status.im>
This will make identifying installed software easier.
It can also be used to generate F-Droid RPs more easily.
Depends on:
https://github.com/status-im/status-jenkins-lib/pull/35
Signed-off-by: Jakub Sokołowski <jakub@status.im>
The dependency on `make shell` is confusing and unnecessary since
`update-status-go.sh` script only requires availability of the
`nix-prefetch-url` utility to check SHA256 of the `status-go` verison.
Use of `make shell` with this script can also cause issues when running
`make run-android` if already in the shell, which prevents spawning a
new one and effectively stops changes to `status-go-verison.json` file
taking effect.
Signed-off-by: Jakub Sokołowski <jakub@status.im>
With these targets added we can simplify the format of the F-Droid YAML
metadata file which defines how to run `status-react` builds.
Signed-off-by: Jakub Sokołowski <jakub@status.im>
Before it was possible to break the format of `status-go-version.json`:
```
> git ls-remote https://github.com/status-im/status-go v0.62.3.hotfix.3
59e6602405bfbcf8446d26aca9b8087e84529f8e refs/heads/release/v0.62.3.hotfix.3
59e6602405bfbcf8446d26aca9b8087e84529f8e refs/tags/v0.62.3.hotfix.3
```
Which would result in `commit-sha1` key being set to two commits insted of one.
Signed-off-by: Jakub Sokołowski <jakub@status.im>
Changes:
* Create `nix/config.nix` with `config` defaults
* Add `nix/tools/gradlePropParser.nix` for reading `gradle.properties`
* Add `nix/mobile/android/keystore.nix` for generating a keystore
* Load keystore generation in `nix/mobile/android/default.nix`
* Use generated keystore if it's not provided via `config`
* Add `-deststoretype pkcs12` in `scripts/generate-keystore.sh`
* Add `nix/lib/assertEnvVarSet.nix` for checking if env var is set
Signed-off-by: Jakub Sokołowski <jakub@status.im>
Changes:
- MOAR functions so it's easier to follow
- Don't use /tmp, put the copied node_modules in repo root first
- Ignore more node_modules sub folders when checking for modifications
- */unpacked_bin/clj-kondo
- */scripts/.packager.env
Signed-off-by: Jakub Sokołowski <jakub@status.im>
The mess with regexes is hard to read and think about which is why it
had bugs with handling some Gradle formats.
It also lowers further the number of dependencies pulled from 785 to 744.
Changes:
- Added `gradle_parser.awk` script for getting dependencies from Gradle
- Changed the `deps.urls` file to contain full URLs to POMs
- Dropped the `deps.urls.old` part since `get_urls.sh` no longer exists
- Added `CLR` for learing line to `scripts/colors.sh`
- Wrote a new `nix/deps/gradle/README.md`
- Re-generated `nix/pkgs/go-maven-resolver/deps.nix`
Signed-off-by: Jakub Sokołowski <jakub@status.im>
Changes:
- Fixed `nix/status-go/desktop` builds
- Dropped nimbus wrapper for `status-go` for now
- Split `status-go` builds into subfolders: `mobile`, `desktop`
- Fixed shells for desktop builds: `linux`,`macos`,`windows`
- Added `make status-go-*` targets for building them
- Moved source management to `nix/status-go/source.nix`
- Moved `nix/status-go/build.nix` into `nix/status-go/mobile`
- Moved `nix/desktop/cmake/qtkeychain` to `nix/pkgs/qtkeychain-src`
- Moved `nix/desktop/linux/linuxdeployqt` to `nix/pkgs`
- Moved `nix/desktop/linux/appimagekit` to `nix/pkgs`
- Dropped `nix/tools/mkShell.nix` since it did almost nothing
- Dropped `nix/desktop/cmake/snorenotify` since it's broken
- Moved setup from `nix/tools/mkShell.nix` to `nix/shells.nix`
- Simplified `nix/mobile/ios/status-go-shell.nix`
- Simplified `nix/status-go/default.nix`
- Updated the `nix/DETAILS.md` and `nix/README.md`
- Moved known issues to `nix/KNOWN_ISSUES.md`
- Improved output of `nix/scripts/build.sh`
Signed-off-by: Jakub Sokołowski <jakub@status.im>
Changes:
- Add `nix/DETAILS.md` for more in-depth info
- Rename some of `config.status-im` variables
- Drop `env` argument for Android build
- Use `overlays` instead of `packageOverrides`
- Move the `pkgs` overlay to `nix/overlay.nix`
- Move `nix/status-go/utils.nix` to `nix/tools`
- Make `shell.nix` use the `shells.default` only
- Use `default.nix` as target for `nix/scripts/shell.sh`
- Make `nix/scripts/shell.sh` use `--attr` instead of `target`
- Drop the `target` argument in favour of using `--attr`
- Drop unnecessary `src` from `nix/mobile/ios/default.nix`
- Move `mkShell` and `mergeSh` under `lib`
- Move `patched-go` package to `nix/pkgs` directory
- Move `gomobile` package to `nix/pkgs` directory
- Move `ANDROID_ABI_SPLIT` to `config.status-im.android.abi-split`
- Move `ANDROID_ABI_INCLUDE to `config.status-im.android.abi-include`
Signed-off-by: Jakub Sokołowski <jakub@status.im>
changes:
- Drop use of /dev/stderr, doesn't work on MacOS
- Use uname -s instead of $OS which was removed
- Remove leading whitespace that fucks with Nix
- Check if `/etc/os-release` exists before reading it
Signed-off-by: Jakub Sokołowski <jakub@status.im>
Changes:
* Drop --keep from nix/scripts/build.sh as it does nothing
* Allow scripts/release-android.sh take passwords from env
* Use env file in extra-sandbox-paths to pass keystore passwords
* Quote ever segment of Nix options to avoid parsing errors
Signed-off-by: Jakub Sokołowski <jakub@status.im>
This PR extracts all the ci/*.groovy scripts into a separate private repo located at:
https://github.com/status-im/status-react-jenkins
The main reasons for a separate repo are:
* Hiding the internal details of our CI setup
* Hiding names of Jenkins credentials available in CI jobs
* Lowering attack surface for malicious external contributors
* Increasing focus on PRs related to CI setup
You can read more about how Jenkins pipeline shared libraries work here:
https://jenkins.io/doc/book/pipeline/shared-libraries/
In simple terms I've added the repo to the main Jenkins configuration in "Global Pipeline Libraries" section and load it using:
library 'status-react-jenkins@master'
Which makes globally available all of the libraries defined in the `vars` directory of that repo.
This also eliminates the need for statements like `android = load 'ci/android.groovy'`.
Signed-off-by: Jakub Sokołowski <jakub@status.im>
changes:
- Stops scripts/generate-keystore.sh runnig for every new TARGET=android shell
- Moves running scripts/generate-keystore.sh to keystore Makefile target
- Makes generate-keystore.sh accept STATUS_RELEASE_STORE_FILE as env variable
- Fixes error about missing lsof from start-react-native.sh
- Fixes error about unknown -keystore command in scripts/generate-keystore.sh
- Fixes getting OS_NAME without calling Nix shell
- Fixes getting TMP_BUILD_NUMBER without calling Nix shell
- Adds lsof used in start-react-native.sh to Android release Nix derivation
- Simplified how nix/scripts/build.sh appends the --keep flag
- Made use of colors.sh in more scripts
Signed-off-by: Jakub Sokołowski <jakub@status.im>
Changes:
- Add missing lazy assignment for OS_NAME, fixes double Nix shells
- Moved `scripts/add-nix-gcroots.sh` to `nix/scripts/gcroots.sh`
- Moved Nix package manager setup to `nix/scripts/setup.sh`
- Created `nix/scripts/source.sh` to reuse in all Nix scripts
- Created `STARTING_GUIDE.md` with instructions for contributors
- Created `scripts/colors.sh` for definition of shell colors
- Removed `scripts/setup` in favor of `nix/scripts/setup.sh`
- Removed all of `scripts/lib` since it was useless
Signed-off-by: Jakub Sokołowski <jakub@status.im>
There was some confusion when given ref/tag did not exist.
Script would just use the value as commit SHA1 even if it was not one.
Signed-off-by: Jakub Sokołowski <jakub@status.im>
And we have to trim the one used for Android.
For more details see:
>Warning: The greatest value Google Play allows for versionCode is 2100000000.
https://developer.android.com/studio/publish/versioning
Signed-off-by: Jakub Sokołowski <jakub@status.im>
Changes:
- Adds a new `nix-gc` Makefile target for removing old packages
- Moves all `nix/*.sh` files to `nix/scripts/*.sh` to make things more tidy
- Renames `TARGET_OS` into `TARGET` and makes it effective only with `nix/scripts/shell.sh`
- Renames `target-os` Nix argument to just `target` and makes it effective only with `shell.nix`
- Drops `IN_CI_ENVIRONMENT` env variable which was useless
- Drops use of `target-os` argument outside of `shell.nix` (with few exceptions, but just in naming)
- `nix/platform.nix` has been made obsolete and removed
- Moves the definition of all major targets to `nix/targets.nix`
- Moves the definition of all major shells to `nix/shells.nix`
- Makes `default.nix` and `shell.nix` just thin wrappers around `nix/default.nix`
- `nix/nixpkgs-bootstrap.nix` has been moved to `nix/pkgs.nix`
- All package and tool overrides have been moved to `nix/pkgs.nix`
- Explicit passing of contents of `pkgs` has been removed in favor of `callPackage` doing it for us
- `nix/bootstrapped-shell.nix` has been moved to `nix/tools/mkShell.nix`
- A new `mergeSh` tool has been added to `pkgs` from `nix/tools/mergeSh.nix`
- This tool is used to merge shells created using `mkShell`
- `mobile/targets/jsbundle.nix` has been moved to `mobile/android/jsbundle/default.nix`
- Moves `status-go` version sanitization to `nix/status-go/utils.nix`
- Renames version to rawVersion and versionName to cleanVersion in status-go derivation
- Ports nix/mobile/ios/install-pods-and-status-go.sh to Nix sub-shells
- Moves adjustment of `inotify/max_user_watches` out into `scripts/inotify_fix.sh`
- Makes iOS builds use the Nix version of Fastlane
Signed-off-by: Jakub Sokołowski <jakub@status.im>
- Move `mobile_files` into `mobile/js_files`
- Move `desktop_files` into `desktop/js_files`
Signed-off-by: Pedro Pombeiro <pombeirp@users.noreply.github.com>
Fixes#8635 by adding VERSION and BUILD_NUMBER files in the correct nix
template and updating bash script in order to prevent it from failing,
due to the git repository being not initialized in the nix environment.
Move scripts/build_no.sh and scripts/gen_build_no.sh to
scripts/version/build_no.sh to prevent Nix from rebuilding when
unrelated scripts are touched.
Signed-off-by: Andrey Shovkoplyas <motor4ik@gmail.com>
Jakub Sokołowski
andrey
Roman Volosovskyi
Audrius Molis
Parvesh Monu
Andrea Maria Piana
yenda
Pedro Pombeiro
Pedro Pombeiro
Volodymyr Kozieiev
MajorTomSec
tbenr
Igor Mandrigin