nix: add a way to deal with new gradle version to docs

Also adjusted the `generate.sh` script and split stuff into separate function
to make it easier to run specific stages of the generation process.

Signed-off-by: Jakub Sokołowski <jakub@status.im>
This commit is contained in:
Jakub Sokołowski 2020-11-27 12:35:21 +01:00
parent f47c816ce0
commit f0d9f2ac54
No known key found for this signature in database
GPG Key ID: 4EF064D0E6D63020
2 changed files with 81 additions and 30 deletions

View File

@ -42,3 +42,34 @@ dr-xr-xr-x 3 root root 3 Jan 1 1970 commons-cli
# Dependencies # Dependencies
One dependency these scripts require is the [go-maven-resolver](https://github.com/status-im/go-maven-resolver) which turns a list of packages into list of all URLs of POMs for them and their dependencies. One dependency these scripts require is the [go-maven-resolver](https://github.com/status-im/go-maven-resolver) which turns a list of packages into list of all URLs of POMs for them and their dependencies.
# Known Issues
One edge case that is currently not handled by this setup is an addition of a new dependency that requires a version of Gradle we don't have. In that case running `make nix-update-gradle` would fail with:
```
AILURE: Build failed with an exception.
* What went wrong:
A problem occurred configuring project ':react-native-example-project'.
> Could not resolve all artifacts for configuration ':react-native-example-project:classpath'.
> Could not find com.android.tools.build:gradle:3.4.0.
Searched in the following locations:
- file:/nix/store/34a4qd5qhg2a9kq6a0q9lp7hgmi48q4x-status-react-maven-deps/com/android/tools/build/gradle/3.4.0/gradle-3.4.0.pom
- file:/nix/store/34a4qd5qhg2a9kq6a0q9lp7hgmi48q4x-status-react-maven-deps/com/android/tools/build/gradle/3.4.0/gradle-3.4.0.jar
Required by:
project :react-native-example-project
```
This happens because `get_projects.sh` and `get_deps.sh` depend on running Gradle to generate their output, and that will not work because Gradle cannot find the matching JAR for the new dependency.
A manual solution would involve adding that specific Gradle version into `deps.urls`:
```
echo com.android.tools.build:gradle:3.4.0 | go-maven-resolver >> nix/deps/gradle/deps.urls
```
Then you'll need to remove duplicates and sort `deps.urls`:
```
sort -o nix/deps/gradle/deps.urls nix/deps/gradle/deps.urls
```
And finally generate the `deps.json` for Nix using:
```
nix/deps/gradle/generate.sh gen_deps_json
```

View File

@ -26,6 +26,45 @@ DEPS_JSON="${CUR_DIR}/deps.json"
# Raise limit of file descriptors # Raise limit of file descriptors
ulimit -n 16384 ulimit -n 16384
# Generate list of Gradle sub-projects.
function gen_proj_list() {
${CUR_DIR}/get_projects.sh | sort -u -o ${PROJ_LIST}
echo -e "Found ${GRN}$(wc -l < ${PROJ_LIST})${RST} sub-projects..."
}
# Check each sub-project in parallel, the ":" is for local deps.
function gen_deps_list() {
PROJECTS=$(cat ${PROJ_LIST})
${CUR_DIR}/get_deps.sh ":" ${PROJECTS[@]} | sort -uV -o ${DEPS_LIST}
echo -e "${CLR}Found ${GRN}$(wc -l < ${DEPS_LIST})${RST} direct dependencies..."
}
# Find download URLs for each dependency.
function gen_deps_urls() {
cat ${DEPS_LIST} | go-maven-resolver | sort -uV -o ${DEPS_URLS}
echo -e "${CLR}Found ${GRN}$(wc -l < ${DEPS_URLS})${RST} dependency URLs..."
}
# Generate the JSON that Nix will consume.
function gen_deps_json() {
# Open the Nix attribute set.
echo -n "[" > ${DEPS_JSON}
# Format URLs into a Nix consumable file.
URLS=$(cat ${DEPS_URLS})
parallel --will-cite --keep-order \
"${CUR_DIR}/url2json.sh" \
::: ${URLS} \
>> ${DEPS_JSON}
# Drop tailing comma on last object, stupid JSON
sed -i '$ s/},/}/' ${DEPS_JSON}
# Close the Nix attribute set
echo "]" >> ${DEPS_JSON}
}
# ------------------------------------------------------------------------------
echo "Regenerating Nix files..." echo "Regenerating Nix files..."
# Gradle needs to be run in 'android' subfolder # Gradle needs to be run in 'android' subfolder
@ -34,37 +73,18 @@ cd $GIT_ROOT/android
# Stop gradle daemons to avoid locking # Stop gradle daemons to avoid locking
./gradlew --stop >/dev/null ./gradlew --stop >/dev/null
# Generate list of Gradle sub-projects ---------------------------------------- # A way to run a specific stage of generation
${CUR_DIR}/get_projects.sh | sort -u -o ${PROJ_LIST} if [[ -n "${1}" ]] && type ${1} > /dev/null; then
${1}; exit 0
elif [[ -n "${1}" ]]; then
echo "No such function: ${1}"; exit 1
fi
echo -e "Found ${GRN}$(wc -l < ${PROJ_LIST})${RST} sub-projects..." # Run each stage in order
gen_proj_list
# Check each sub-project in parallel, the ":" is for local deps --------------- gen_deps_list
PROJECTS=$(cat ${PROJ_LIST}) gen_deps_urls
${CUR_DIR}/get_deps.sh ":" ${PROJECTS[@]} | sort -uV -o ${DEPS_LIST} gen_deps_json
echo -e "${CLR}Found ${GRN}$(wc -l < ${DEPS_LIST})${RST} direct dependencies..."
# Find download URLs for each dependency --------------------------------------
cat ${DEPS_LIST} | go-maven-resolver | sort -uV -o ${DEPS_URLS}
echo -e "${CLR}Found ${GRN}$(wc -l < ${DEPS_URLS})${RST} dependency URLs..."
# Open the Nix attribute set --------------------------------------------------
echo -n "[" > ${DEPS_JSON}
# Format URLs into a Nix consumable file.
URLS=$(cat ${DEPS_URLS})
parallel --will-cite --keep-order \
"${CUR_DIR}/url2json.sh" \
::: ${URLS} \
>> ${DEPS_JSON}
# Drop tailing comma on last object, stupid JSON
sed -i '$ s/},/}/' ${DEPS_JSON}
# Close the Nix attribute set
echo "]" >> ${DEPS_JSON}
REL_DEPS_JSON=$(realpath --relative-to=${PWD} ${DEPS_JSON}) REL_DEPS_JSON=$(realpath --relative-to=${PWD} ${DEPS_JSON})
echo -e "${CLR}Generated Nix deps file: ${REL_DEPS_JSON#../}" echo -e "${CLR}Generated Nix deps file: ${REL_DEPS_JSON#../}"