Commit Graph

11 Commits

Author SHA1 Message Date
Jakub Sokołowski 83f622d349
nix: make fetching Gradle dependencies more generic
Before what we did was essentially guess what files might exist for any
given package. This approach mostly works, but not entirely.
This is especially problematic when dealing with weird edge case
packages like `react-native`, which you can read about here:

https://github.com/react-native-community/discussions-and-proposals/pull/508
https://github.com/react-native-community/discussions-and-proposals/blob/4a06fc64/proposals/0508-out-of-npm-artifacts.md#the-react-native-android-archive

In order to avoid as much the guessing aspect of fetching Gradle
dependencies we are using both HTML listsings of files and
`artifact-metadata.json` files that exist for more recent packages.

This way we can avoid having to add special edge cases that have been
found out when working on React Native 72 upgrade in:
https://github.com/status-im/status-mobile/pull/17062

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-09-04 09:57:53 +02:00
Jakub Sokołowski ac3eb3e660
nix: use SRI hashes for Gradle dependencies
This way we will use the same format as Nix logs and errors when a hash
changes, which avoids confusion.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-05-10 15:45:32 +02:00
Jakub Sokołowski f03e11c877
nix: fix handling POMs without JARs
Otherwise we can get a failure like this:
```
Regenerating Nix files...
Found 40 sub-projects...
Found 609 direct dependencies...
Found 889 dependency URLs...
 ! Failed to fetch: https://repo.maven.apache.org/maven2/org/ow2/asm/asm-bom/9.5/asm-bom-9.5.jar
```
When using `make nix-update-gradle`.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-05-08 15:48:19 +02:00
Jakub Sokołowski 20a0cc01f6
nix: refactor Gradle deps to be more generic
Refactoring the derivation that fetches all the POMs, JARs,
and AARs in order to make it more generic and easier to extend.
The main change is adding `files` key in `deps.json` which contains
a dict of all the files reletad to given package.

This way we can more easily include other files that might be available
for download, like AARs with ASC suffix, or `nodeps` JARs.

This is also necessary for the React Native upgrade:
https://github.com/status-im/status-mobile/pull/15203

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-03-04 12:41:29 +01:00
Jakub Sokołowski 51276237f2
nix: include nodeps JARs in Gradle deps (#15212)
* nix: update gradle dependencies

Signed-off-by: Jakub Sokołowski <jakub@status.im>

* nix: include nodeps JARs in Gradle deps

Signed-off-by: Jakub Sokołowski <jakub@status.im>

* nix: include nodeps JAR for semver4j 0.16.4

Can cause failures like in some cases:
```
A problem occurred configuring project ':react-native-hole-view'.
> Could not resolve all files for configuration ':react-native-hole-view:classpath'.
   > Could not find semver4j-0.16.4-nodeps.jar (com.github.gundy:semver4j:0.16.4).
     Searched in the following locations:
         file:/nix/store/3n2pxsqa2izlx8c23s6jgqai0bqaklm1-status-mobile-maven-deps/com/github/gundy/semver4j/0.16.4/semver4j-0.16.4-nodeps.jar
```

Signed-off-by: Jakub Sokołowski <jakub@status.im>

---------

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-02-28 17:13:44 +05:30
Jakub Sokołowski 372942ba8a
nix: fortify the nix-update-gradle process
This fixes two issues with the `nix-update-gradle` target:

* It now fails when a JAR is missing which used to be ignored.
* It ignores dependencies that have no JARs, like Eclipse plugins.

This makes the process more robust, since we can see something is
missing right away, and a developer may re-run the process to take
account of possible temporary networking failures or rate limiting.

It also slims down the size of the `deps.json` by removing dependencies
which contribute no actual JARs or AARs to the build process.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2022-05-17 20:04:04 +02:00
Jakub Sokołowski 43da5a60fe
nix: make url2json.sh parse POM content for packaging
Otherwise we can download a JAR when an AAR is necessary.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2021-11-16 11:12:06 +01:00
Jakub Sokołowski 0d403739f9
nix: upgrade go-maven-resolver to 1.1.1, add more repos
I've noticed issues with the https://repository.sonatype.org/ Maven repository we were using.
They appear to have purged their storage of a lot of packages, including our own for example:
https://repository.sonatype.org/content/groups/sonatype-public-grid/com/github/status-im/

For this reason I've release a new version of `go-maven-resolver` that includes two more Maven repos:
- https://plugins.gradle.org/m2
- https://jitpack.io

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2021-06-30 18:49:55 +02:00
Jakub Sokołowski 2f65cedd2d
nix: replace grep and sed with AWK for parsing Gradle deps
The mess with regexes is hard to read and think about which is why it
had bugs with handling some Gradle formats.

It also lowers further the number of dependencies pulled from 785 to 744.

Changes:
- Added `gradle_parser.awk` script for getting dependencies from Gradle
- Changed the `deps.urls` file to contain full URLs to POMs
- Dropped the `deps.urls.old` part since `get_urls.sh` no longer exists
- Added `CLR` for learing line to `scripts/colors.sh`
- Wrote a new `nix/deps/gradle/README.md`
- Re-generated `nix/pkgs/go-maven-resolver/deps.nix`

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2020-05-25 19:34:56 +02:00
Jakub Sokołowski d0216674da
nix: use go-maven-resolver to fix nix-update-gradle
This Go package was implemented by me to avoid having to use Mavan or
Gradle to resolve all the dependencies for Nix:
https://github.com/status-im/go-maven-resolver

The Go tool is essentially a slightly smarter and more efficient version
of the `get_urls.sh` script, which was mostly slow due to the `mvn
dependency:list` calls and slow HTTP requests.

This takes down the `make nix-update-gradle` target to under 2 minutes
on my machine.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
2020-05-24 18:08:05 +02:00
Jakub Sokołowski 5607540ac1
nix: data in nix/deps/gradle should be JSON
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2020-05-14 17:24:59 +02:00