remove `Nim-csources-v1` and `nimble` checkouts (#75)

When `build_nim.sh` detects that `skipIntegrityCheck` is supported,
it runs the upstream toolchain for building Nim, ignoring the locally
checked out `Nim-csources-v1` and `nimble` sub-repositories. So, having
those repos checked out is not useful in that scenario.

When `skipIntegrityCheck` is unsupported, `build_nim.sh` uses the local
sub-repositories if they are available. But, it also clones them if they
are unavailable using commits as specified by environment variables.
Because the clone will happen as part of the script, having the repos
linked as sub-repositories is also not useful. Even worse, if the script
has different commit hashes than the subrepository, the behaviour is
different depending on whether `build_nim.sh` is run before or after
the submodules are checked out.

Therefore, we can remove the `Nim-csources-v1` and `nimble` sub-repos,
using the upstream build system if available, or the hardcoded commits
inside `build_nim.sh` to have a manual build, as before.

Also moves the various variables that control the used versions into
the manual section to emphasize that these only take effect in manual
mode, but not when using upstream build system (`skipIntegrityCheck`).

If one wants to use a custom `Nimble` version, fork `Nim` and adjust the
config in that fork. That also works with the upstream build system.
This commit is contained in:
Etan Kissling 2024-01-15 16:21:06 +01:00 committed by GitHub
parent c01eb708bc
commit 3866a8ab98
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
5 changed files with 20 additions and 26 deletions

10
.gitmodules vendored
View File

@ -3,13 +3,3 @@
url = https://github.com/status-im/Nim.git url = https://github.com/status-im/Nim.git
ignore = dirty ignore = dirty
branch = nimbus branch = nimbus
[submodule "vendor/nimble"]
path = vendor/nimble
url = https://github.com/nim-lang/nimble.git
ignore = dirty
branch = master
[submodule "vendor/Nim-csources-v1"]
path = vendor/Nim-csources-v1
url = https://github.com/nim-lang/csources_v1.git
ignore = untracked
branch = master

View File

@ -85,7 +85,7 @@ build-nim: | sanity-checks
MAKE="$(MAKE)" \ MAKE="$(MAKE)" \
ARCH_OVERRIDE=$(ARCH_OVERRIDE) \ ARCH_OVERRIDE=$(ARCH_OVERRIDE) \
QUICK_AND_DIRTY_COMPILER=$(QUICK_AND_DIRTY_COMPILER) \ QUICK_AND_DIRTY_COMPILER=$(QUICK_AND_DIRTY_COMPILER) \
"$(CURDIR)/$(BUILD_SYSTEM_DIR)/scripts/build_nim.sh" "$(NIM_DIR)" ../Nim-csources-v1 ../nimble "$(CI_CACHE)" "$(CURDIR)/$(BUILD_SYSTEM_DIR)/scripts/build_nim.sh" "$(NIM_DIR)" ../Nim-csources ../nimble "$(CI_CACHE)"
# Check if the update might cause loss of work. Abort, if so, while allowing an override mechanism. # Check if the update might cause loss of work. Abort, if so, while allowing an override mechanism.
update-test: update-test:

View File

@ -10,23 +10,14 @@
set -e set -e
# Git commits
: ${CSOURCES_V1_COMMIT:=561b417c65791cd8356b5f73620914ceff845d10}
: ${CSOURCES_V2_COMMIT:=86742fb02c6606ab01a532a0085784effb2e753e}
: ${CSOURCES_V1_REPO:=https://github.com/nim-lang/csources_v1.git}
: ${CSOURCES_V2_REPO:=https://github.com/nim-lang/csources_v2.git}
# After this Nim commit, use csources v2
: ${CSOURCES_V2_START_COMMIT:=f7c203fb6c89b5cef83c4f326aeb23ef8c4a2c40}
: ${NIMBLE_COMMIT:=d13f3b8ce288b4dc8c34c219a4e050aaeaf43fc9} # 0.13.1
# NIM_COMMIT could be a (partial) commit hash, a tag, a branch name, etc. Empty by default. # NIM_COMMIT could be a (partial) commit hash, a tag, a branch name, etc. Empty by default.
NIM_COMMIT_HASH="" # full hash for NIM_COMMIT, retrieved in "nim_needs_rebuilding()" NIM_COMMIT_HASH="" # full hash for NIM_COMMIT, retrieved in "nim_needs_rebuilding()"
# script arguments # script arguments
[[ $# -ne 4 ]] && { echo "Usage: $0 nim_dir csources_dir nimble_dir ci_cache_dir"; exit 1; } [[ $# -ne 4 ]] && { echo "Usage: $0 nim_dir csources_dir nimble_dir ci_cache_dir"; exit 1; }
NIM_DIR="$1" NIM_DIR="$1"
CSOURCES_DIR="$2" # can be relative to NIM_DIR CSOURCES_DIR="$2" # can be relative to NIM_DIR; only used when `skipIntegrityCheck` unsupported
NIMBLE_DIR="$3" # can be relative to NIM_DIR NIMBLE_DIR="$3" # can be relative to NIM_DIR; only used when `skipIntegrityCheck` unsupported
CI_CACHE="$4" CI_CACHE="$4"
## env vars ## env vars
@ -128,7 +119,11 @@ build_nim() {
pushd "$NIM_DIR" pushd "$NIM_DIR"
if grep -q skipIntegrityCheck koch.nim; then if grep -q skipIntegrityCheck koch.nim; then
# Run Nim buildchain # Run Nim buildchain, with matching dependency versions
# - CSOURCES_REPO from Nim/config/build_config.txt (nim_csourcesUrl)
# - CSOURCES_COMMIT from Nim/config/build_config.txt (nim_csourcesHash)
# - NIMBLE_REPO from Nim/koch.nim (bundleNimbleExe)
# - NIMBLE_COMMIT from Nim/koch.nim (NimbleStableCommit)
. ci/funs.sh . ci/funs.sh
NIMCORES=1 nimBuildCsourcesIfNeeded $UCPU NIMCORES=1 nimBuildCsourcesIfNeeded $UCPU
bin/nim c --noNimblePath --skipUserCfg --skipParentCfg --warnings:off --hints:off koch bin/nim c --noNimblePath --skipUserCfg --skipParentCfg --warnings:off --hints:off koch
@ -141,6 +136,17 @@ build_nim() {
./koch nimble -d:release --skipUserCfg --skipParentCfg --warnings:off --hints:off ./koch nimble -d:release --skipUserCfg --skipParentCfg --warnings:off --hints:off
fi fi
else else
# Git commits
: ${CSOURCES_V1_COMMIT:=561b417c65791cd8356b5f73620914ceff845d10}
: ${CSOURCES_V2_COMMIT:=86742fb02c6606ab01a532a0085784effb2e753e}
: ${CSOURCES_V1_REPO:=https://github.com/nim-lang/csources_v1.git}
: ${CSOURCES_V2_REPO:=https://github.com/nim-lang/csources_v2.git}
# After this Nim commit, use csources v2
: ${CSOURCES_V2_START_COMMIT:=f7c203fb6c89b5cef83c4f326aeb23ef8c4a2c40}
: ${NIMBLE_REPO:=https://github.com/nim-lang/nimble.git}
: ${NIMBLE_COMMIT:=d13f3b8ce288b4dc8c34c219a4e050aaeaf43fc9} # 0.13.1
# Custom buildchain for older versions # Custom buildchain for older versions
# TODO Remove this once the default NIM_COMMIT supports `--skipIntegrityCheck` # TODO Remove this once the default NIM_COMMIT supports `--skipIntegrityCheck`
# We will still be able to compile older versions by removing the flag, # We will still be able to compile older versions by removing the flag,
@ -170,7 +176,7 @@ build_nim() {
if [[ ! -d "$NIMBLE_DIR" ]]; then if [[ ! -d "$NIMBLE_DIR" ]]; then
mkdir -p "$NIMBLE_DIR" mkdir -p "$NIMBLE_DIR"
pushd "$NIMBLE_DIR" pushd "$NIMBLE_DIR"
git clone https://github.com/nim-lang/nimble.git . git clone $NIMBLE_REPO .
git checkout $NIMBLE_COMMIT git checkout $NIMBLE_COMMIT
# we have to delete .git or koch.nim will checkout a branch tip, overriding our target commit # we have to delete .git or koch.nim will checkout a branch tip, overriding our target commit
rm -rf .git rm -rf .git

@ -1 +0,0 @@
Subproject commit 561b417c65791cd8356b5f73620914ceff845d10

1
vendor/nimble vendored

@ -1 +0,0 @@
Subproject commit d13f3b8ce288b4dc8c34c219a4e050aaeaf43fc9