constantine/azure-pipelines.yml

248 lines
7.6 KiB
YAML
Raw Normal View History

strategy:
maxParallel: 10
matrix:
# TODO: Broken by OpenSSL not found for nimble
# https://github.com/nim-lang/Nim/blob/bf320ed172f74f60fd274338e82bdc9ce3520dd9/lib/wrappers/openssl.nim#L61
# could not load: (libcrypto-1_1|libeay32).dll
# (bad format; library may be wrong architecture)
# Windows_devel_32bit:
# VM: 'windows-latest'
# UCPU: i686
# CHANNEL: devel
# TEST_LANG: c
Windows_devel_64bit:
VM: 'windows-latest'
UCPU: amd64
CHANNEL: devel
TEST_LANG: c
Windows_cpp_devel_64bit:
VM: 'windows-latest'
UCPU: amd64
CHANNEL: devel
TEST_LANG: cpp
Linux_devel_64bit:
Internals refactor + renewed focus on perf (#17) * Lay out the refactoring objectives and tradeoffs * Refactor the 32 and 64-bit primitives [skip ci] * BigInts and Modular BigInts compile * Make the bigints test compile * Fix modular reduction * Fix reduction tests vs GMP * Implement montegomery mul, pow, inverse, WIP finite field compilation * Make FiniteField compile * Fix exponentiation compilation * Fix Montgomery magic constant computation for 2^64 words * Fix typo in non-optimized CIOS - passing finite fields IO tests * Add limbs comparisons [skip ci] * Fix on precomputation of the Montgomery magic constant * Passing all tests including 𝔽p2 * modular addition, the test for mersenne prime was wrong * update benches * Fix "nimble test" + typo on out-of-place field addition * bigint division, normalization is needed: https://travis-ci.com/github/mratsim/constantine/jobs/298359743 * missing conversion in subborrow non-x86 fallback - https://travis-ci.com/github/mratsim/constantine/jobs/298359744 * Fix little-endian serialization * Constantine32 flag to run 32-bit constantine on 64-bit machines * IO Field test, ensure that BaseType is used instead of uint64 when the prime can field in uint32 * Implement proper addcarry and subborrow fallback for the compile-time VM * Fix export issue when the logical wordbitwidth == physical wordbitwidth - passes all tests (32-bit and 64-bit) * Fix uint128 on ARM * Fix C++ conditional copy and ARM addcarry/subborrow * Add investigation for SIGFPE in Travis * Fix debug display for unsafeDiv2n1n * multiplexer typo * moveMem bug in glibc of Ubuntu 16.04? * Was probably missing an early clobbered register annotation on conditional mov * Note on Montgomery-friendly moduli * Strongly suspect a GCC before GCC 7 codegen bug (https://gcc.gnu.org/bugzilla/show_bug.cgi?id=87139) * hex conversion was (for debugging) not taking requested order into account + inlining comment * Use 32-bit limbs on ARM64, uint128 builtin __udivti4 bug? * Revert "Use 32-bit limbs on ARM64, uint128 builtin __udivti4 bug?" This reverts commit 087f9aa7fb40bbd058d05cbd8eec7fc082911f49. * Fix subborrow fallback for non-x86 (need to maks the borrow)
2020-03-16 15:33:51 +00:00
VM: 'ubuntu-18.04'
UCPU: amd64
CHANNEL: devel
TEST_LANG: c
Linux_cpp_devel_64bit:
Internals refactor + renewed focus on perf (#17) * Lay out the refactoring objectives and tradeoffs * Refactor the 32 and 64-bit primitives [skip ci] * BigInts and Modular BigInts compile * Make the bigints test compile * Fix modular reduction * Fix reduction tests vs GMP * Implement montegomery mul, pow, inverse, WIP finite field compilation * Make FiniteField compile * Fix exponentiation compilation * Fix Montgomery magic constant computation for 2^64 words * Fix typo in non-optimized CIOS - passing finite fields IO tests * Add limbs comparisons [skip ci] * Fix on precomputation of the Montgomery magic constant * Passing all tests including 𝔽p2 * modular addition, the test for mersenne prime was wrong * update benches * Fix "nimble test" + typo on out-of-place field addition * bigint division, normalization is needed: https://travis-ci.com/github/mratsim/constantine/jobs/298359743 * missing conversion in subborrow non-x86 fallback - https://travis-ci.com/github/mratsim/constantine/jobs/298359744 * Fix little-endian serialization * Constantine32 flag to run 32-bit constantine on 64-bit machines * IO Field test, ensure that BaseType is used instead of uint64 when the prime can field in uint32 * Implement proper addcarry and subborrow fallback for the compile-time VM * Fix export issue when the logical wordbitwidth == physical wordbitwidth - passes all tests (32-bit and 64-bit) * Fix uint128 on ARM * Fix C++ conditional copy and ARM addcarry/subborrow * Add investigation for SIGFPE in Travis * Fix debug display for unsafeDiv2n1n * multiplexer typo * moveMem bug in glibc of Ubuntu 16.04? * Was probably missing an early clobbered register annotation on conditional mov * Note on Montgomery-friendly moduli * Strongly suspect a GCC before GCC 7 codegen bug (https://gcc.gnu.org/bugzilla/show_bug.cgi?id=87139) * hex conversion was (for debugging) not taking requested order into account + inlining comment * Use 32-bit limbs on ARM64, uint128 builtin __udivti4 bug? * Revert "Use 32-bit limbs on ARM64, uint128 builtin __udivti4 bug?" This reverts commit 087f9aa7fb40bbd058d05cbd8eec7fc082911f49. * Fix subborrow fallback for non-x86 (need to maks the borrow)
2020-03-16 15:33:51 +00:00
VM: 'ubuntu-18.04'
UCPU: amd64
CHANNEL: devel
TEST_LANG: cpp
# Deactivated for now, this is cross-compilation and tricky to get right
# Linux_devel_32bit:
# VM: 'ubuntu-16.04'
# UCPU: i686
# CHANNEL: devel
# TEST_LANG: c
MacOS_devel_64bit:
VM: 'macOS-10.15'
UCPU: amd64
CHANNEL: devel
TEST_LANG: c
pool:
vmImage: $(VM)
steps:
- task: CacheBeta@1
displayName: 'cache Nim binaries'
inputs:
key: NimBinaries | $(Agent.OS) | $(CHANNEL) | $(UCPU)
path: NimBinaries
- task: CacheBeta@1
displayName: 'cache MinGW-w64'
inputs:
key: mingwCache | 8_1_0 | $(UCPU)
path: mingwCache
condition: eq(variables['Agent.OS'], 'Windows_NT')
- powershell: |
Set-ItemProperty -Path 'HKLM:\SYSTEM\CurrentControlSet\Control\FileSystem' -Name 'LongPathsEnabled' -Value 1
displayName: 'long path support'
condition: eq(variables['Agent.OS'], 'Windows_NT')
- bash: |
echo "PATH=${PATH}"
set -e
echo "Installing MinGW-w64"
if [[ $UCPU == "i686" ]]; then
MINGW_FILE="i686-8.1.0-release-posix-dwarf-rt_v6-rev0.7z"
MINGW_URL="https://sourceforge.net/projects/mingw-w64/files/Toolchains%20targetting%20Win32/Personal%20Builds/mingw-builds/8.1.0/threads-posix/dwarf/${MINGW_FILE}"
MINGW_DIR="mingw32"
else
MINGW_FILE="x86_64-8.1.0-release-posix-seh-rt_v6-rev0.7z"
MINGW_URL="https://sourceforge.net/projects/mingw-w64/files/Toolchains%20targetting%20Win64/Personal%20Builds/mingw-builds/8.1.0/threads-posix/seh/${MINGW_FILE}"
MINGW_DIR="mingw64"
fi
mkdir -p mingwCache
pushd mingwCache
if [[ ! -e "$MINGW_FILE" ]]; then
rm -f *.7z
curl -OLsS "$MINGW_URL"
fi
7z x -y -bd "$MINGW_FILE" >/dev/null
mkdir -p /c/custom
mv "$MINGW_DIR" /c/custom/
popd
# Workaround https://developercommunity.visualstudio.com/content/problem/891929/windows-2019-cygheap-base-mismatch-detected-git-ba.html
echo "##vso[task.prependpath]/usr/bin"
echo "##vso[task.prependpath]/mingw64/bin"
echo "##vso[task.setvariable variable=MINGW_DIR;]$MINGW_DIR"
displayName: 'Install dependencies (Windows)'
condition: eq(variables['Agent.OS'], 'Windows_NT')
- powershell: |
# export custom mingw PATH to other tasks
echo "##vso[task.prependpath]c:\custom\$(MINGW_DIR)\bin"
displayName: 'Mingw PATH (Windows)'
condition: eq(variables['Agent.OS'], 'Windows_NT')
- bash: |
echo "PATH=${PATH}"
export ncpu=
case '$(Agent.OS)' in
'Linux')
ncpu=$(nproc)
;;
'Darwin')
ncpu=$(sysctl -n hw.ncpu)
;;
'Windows_NT')
ncpu=$NUMBER_OF_PROCESSORS
;;
esac
[[ -z "$ncpu" || $ncpu -le 0 ]] && ncpu=1
echo "Found ${ncpu} cores"
echo "##vso[task.setvariable variable=ncpu;]$ncpu"
displayName: 'Detecting number of cores'
2020-08-31 21:04:41 +00:00
- bash: |
sudo dpkg --add-architecture i386
sudo apt-fast update -qq
# `:i386` (e.g. in `libffi-dev:i386`) is needed otherwise you may get:
# `could not load: libffi.so` during dynamic loading.
DEBIAN_FRONTEND='noninteractive' \
sudo apt-fast install --no-install-recommends --allow-downgrades -yq \
g++-multilib gcc-multilib libcurl4-openssl-dev:i386 libgc-dev:i386 \
libffi-dev:i386
2020-09-01 10:33:41 +00:00
mkdir -p bin
cat << EOF > bin/gcc
2020-08-31 21:04:41 +00:00
#!/bin/bash
exec $(which gcc) -m32 "\$@"
EOF
2020-09-01 10:33:41 +00:00
cat << EOF > bin/g++
2020-08-31 21:04:41 +00:00
#!/bin/bash
exec $(which g++) -m32 "\$@"
EOF
2020-09-01 10:33:41 +00:00
2020-08-31 21:04:41 +00:00
chmod 755 bin/gcc
chmod 755 bin/g++
2020-09-01 08:29:02 +00:00
# Add to path
echo '##vso[task.prependpath]$(System.DefaultWorkingDirectory)/bin'
2020-08-31 21:04:41 +00:00
displayName: 'Install dependencies (i386 Linux)'
2020-09-01 09:53:58 +00:00
condition: and(eq(variables['Agent.OS'], 'Linux'), eq(variables['UCPU'], 'i686'))
2020-08-31 21:04:41 +00:00
- bash: |
echo "PATH=${PATH}"
gcc -v
export ucpu=${UCPU}
if [ "${CHANNEL}" = stable ]; then
BRANCH="v$(curl https://nim-lang.org/channels/stable)"
else
BRANCH="${CHANNEL}"
fi
mkdir -p NimBinaries
pushd NimBinaries
if [ ! -x "nim-${CHANNEL}/bin/nim" ]; then
git clone -b "${BRANCH}" https://github.com/nim-lang/nim "nim-${CHANNEL}/"
pushd "nim-${CHANNEL}"
git clone --depth 1 https://github.com/nim-lang/csources csources/
pushd csources
make -j $ncpu ucpu=${UCPU} CC=gcc
popd
rm -rf csources
bin/nim c koch
./koch boot -d:release
./koch tools
else
pushd "nim-${CHANNEL}"
git fetch origin "${BRANCH}"
if [[ $(git merge FETCH_HEAD | grep -c "Already up to date.") -ne 1 ]]; then
bin/nim c koch
./koch boot -d:release
./koch tools
fi
fi
popd # exit nim-${CHANNEL}
popd # exit NimBinaries
displayName: 'Building Nim'
- powershell: |
echo "##vso[task.prependpath]$pwd\NimBinaries\nim-$(CHANNEL)\bin"
displayName: 'Set env variable (Windows)'
condition: eq(variables['Agent.OS'], 'Windows_NT')
- bash: |
echo "##vso[task.prependpath]$PWD/NimBinaries/nim-${CHANNEL}/bin"
displayName: 'Set env variable (Posix)'
condition: ne(variables['Agent.OS'], 'Windows_NT')
# GMP is preinstalled on Azure
# - bash: |
# echo "PATH=${PATH}"
# sudo apt-get install libgmp-dev
# displayName: 'Downloading GMP (Linux)'
# condition: eq(variables['Agent.OS'], 'Linux')
#
# - bash: |
# echo "PATH=${PATH}"
# brew install gmp
# displayName: 'Downloading GMP (MacOS)'
# condition: eq(variables['Agent.OS'], 'Darwin')
2020-08-31 21:04:41 +00:00
- bash: |
echo "PATH=${PATH}"
sudo apt-get install libgmp-dev
displayName: 'Downloading GMP (Linux 32-bit)'
2020-08-31 21:18:16 +00:00
condition: and(eq(variables['Agent.OS'], 'Linux'), eq(variables['UCPU'], 'i686'))
2020-08-31 21:04:41 +00:00
# Workaround #113 and https://github.com/status-im/nim-serialization/issues/33
# and nimble flaky pinning / dependency resolution,
# json_serialization install would override nim-serialization pinning
- bash: |
echo "PATH=${PATH}"
nimble refresh
nimble install -y gmp stew jsony
displayName: 'Installing package and testing dependencies'
- bash: |
echo "PATH=${PATH}"
export ucpu=${UCPU}
nimble test_parallel
displayName: 'Testing Constantine with Assembler and with GMP'
condition: ne(variables['Agent.OS'], 'Windows_NT')
- bash: |
echo "PATH=${PATH}"
export ucpu=${UCPU}
nimble test_parallel_no_assembler
displayName: 'Testing Constantine without Assembler and with GMP'
condition: ne(variables['Agent.OS'], 'Windows_NT')
- bash: |
echo "PATH=${PATH}"
export ucpu=${UCPU}
nimble test_no_gmp
displayName: 'Testing the package (without GMP and ASM on Windows)'
condition: eq(variables['Agent.OS'], 'Windows_NT')