ae9e648526 Define SECP256K1_BUILD in secp256k1.c directly. (Gregory Maxwell)
Pull request description:
This avoids building without it and makes it safer to use a custom
building environment. Test harnesses need to #include secp256k1.c
first now.
Fixes#927
ACKs for top commit:
sipa:
utACK ae9e648526
real-or-random:
ACK ae9e648526
Tree-SHA512: 65ccc15c18f111ba926db1bb25f06c2beb2997c6f42c6d3ebc371ca84f4b5918379efd25c30556cedfd2e4275758bd79d733e80a11159c6ec013dd4707a683ad
This makes a difference with mingw builds on Wine, where the subsequent
fread() may abort early in the default text mode.
The Microsoft C docs say:
"In text mode, CTRL+Z is interpreted as an EOF character on input."
be0609fd54 Add unit tests for edge cases with delta=1/2 variant of divsteps (Pieter Wuille)
cd393ce228 Optimization: only do 59 hddivsteps per iteration instead of 62 (Pieter Wuille)
277b224b6a Use modified divsteps with initial delta=1/2 for constant-time (Pieter Wuille)
376ca366db Fix typo in explanation (Pieter Wuille)
Pull request description:
This updates the divsteps-based modular inverse code to use the modified version which starts with delta=1/2. For variable time, the delta=1 variant is still used as it appears to be faster.
See https://github.com/sipa/safegcd-bounds/tree/master/coq and https://medium.com/blockstream/a-formal-proof-of-safegcd-bounds-695e1735a348 for a proof of correctness of this variant.
TODO:
* [x] Update unit tests to include edge cases specific to this variant
I'm still running the Coq proof verification for the 590 bound in non-native mode. It's unclear how long this will take.
ACKs for top commit:
gmaxwell:
ACK be0609fd54
sanket1729:
crACK be0609fd54
real-or-random:
ACK be0609fd54 careful code review and some testing
Tree-SHA512: 2f8f400ba3ac8dbd08622d564c3b3e5ff30768bd0eb559f2c4279c6c813e17cdde71b1c16f05742c5657b5238b4d592b48306f9f47d7dbdb57907e58dd99b47a
Before this commit, gen_context.c both included libsecp256k1-config.h
and basic-config.h: The former only to obtain ECMULT_GEN_PREC_BITS
and the latter to obtain a basic working configuration to be able to
use the library.
This was inelegant and confusing: It meant that basic-config.h needs
to #undef all the macros defined in libsecp256k1-config.h. Moreover,
it meant that basic-config.h cannot define ECMULT_GEN_PREC_BITS,
essentially making this file specific for use in gen_context.c.
After this commit, gen_context.c include only libsecp256k1-config.h.
basic-config.h is not necessary anymore for the modules used in
gen_context.c because 79f1f7a made the preprocessor detect all the
relevant config options.
On the way, we remove an unused #define in basic-config.h.
Instead of using eta=-delta, use zeta=-(delta+1/2) to represent
delta. This variant only needs at most 590 iterations for 256-bit
inputs rather than 724 (by convex hull bounds analysis).
4504472269 changed import to use brackets <> for openssl as they are not local to the project (William Bright)
Pull request description:
ACKs for top commit:
real-or-random:
ACK 4504472269
jonasnick:
ACK 4504472269
Tree-SHA512: e35c202835a82dab5fe9f2f75e7752e70b15d5d2ee7485790749f145b35e8e995c4978b4015c726387c24248a7efb636d28791fe882581a144a0ddfb27e14075
24ad04fc06 Make scalar_inverse{,_var} benchmark scale with SECP256K1_BENCH_ITERS (Pieter Wuille)
ebc1af700f Optimization: track f,g limb count and pass to new variable-time update_fg_var (Peter Dettman)
b306935ac1 Optimization: use formulas instead of lookup tables for cancelling g bits (Peter Dettman)
9164a1b658 Optimization: special-case zero modulus limbs in modinv64 (Pieter Wuille)
1f233b3fa0 Remove num/gmp support (Pieter Wuille)
20448b8d09 Remove unused Jacobi symbol support (Pieter Wuille)
5437e7bdfb Remove unused scalar_sqr (Pieter Wuille)
aa9cc52180 Improve field/scalar inverse tests (Pieter Wuille)
1e0e885c8a Make field/scalar code use the new modinv modules for inverses (Pieter Wuille)
436281afdc Move secp256k1_fe_inverse{_var} to per-impl files (Pieter Wuille)
aa404d53be Move secp256k1_scalar_{inverse{_var},is_even} to per-impl files (Pieter Wuille)
08d54964e5 Improve bounds checks in modinv modules (Pieter Wuille)
151aac00d3 Add tests for modinv modules (Pieter Wuille)
d8a92fcc4c Add extensive comments on the safegcd algorithm and implementation (Pieter Wuille)
8e415acba2 Add safegcd based modular inverse modules (Peter Dettman)
de0a643c3d Add secp256k1_ctz{32,64}_var functions (Pieter Wuille)
Pull request description:
This is a rebased and squashed version of #767, adding safegcd-based implementations of constant-time and variable-time modular inverses for scalars and field elements, by Peter Dettman. The PR is organized as follows:
* **Add secp256k1_ctz{32,64}_var functions** Introduction of ctz functions to util.h (which use `__builtin_ctz` on recent GCC and Clang, but fall back to using a software emulation using de Bruijn on other platforms). This isn't used anywhere in this commit, but does include tests.
* **Add safegcd based modular inverse modules** Add Peter Dettman's safegcd code from #767 (without some of his optimizations, which are moved to later commits), turned into separate modules by me.
* **Add extensive comments on the safegcd algorithm and implementation** Add a long description of the algorithm and optimizations to `doc/safegcd_implementation.md`, as well as additional comments to the code itself. It is probably best to review this together with the previous commit (they're separated to keep authorship).
* **Add tests for modinv modules** Adds tests on the modinv interface directly, for arbitrary moduli.
* **Improve bounds checks in modinv modules** Adds a lot of sanity checking to the modinv modules.
* **Move secp256k1_scalar_{inverse{_var},is_even} to per-impl files** A pure refactor to prepare for switching the field and scalar code to modinv.
* **Make field/scalar code use the new modinv modules for inverses** Actually switch over.
* **Add extra modular inverse tests** This adds modular inverse tests through the field/scalar interface, now that those use modinv.
* **Remove unused Jacobi symbol support** No longer needed.
* **Remove num/gmp support** Bye-bye.
* 3 commits with further optimizations.
ACKs for top commit:
gmaxwell:
ACK 24ad04fc06
sanket1729:
ACK 24ad04fc06
real-or-random:
ACK 24ad04fc06 careful code review, some testing
Tree-SHA512: 732fe29315965e43ec9a10ee8c71eceeb983c43fe443da9dc5380a5a11b5e40b06e98d6abf67b773b1de74571fd2014973c6376f3a0caeac85e0cf163ba2144b
The magnitude of the f and g variables generally goes down as the algorithm
progresses. Make use of this by keeping tracking how many limbs are used, and
when the number becomes small enough, make use of this to reduce the complexity
of arithmetic on them.
Refactored by: Pieter Wuille <pieter@wuille.net>
Both the field and scalar modulus can be written in signed{30,62} notation
with one or more zero limbs. Make use of this in the update_de function to
avoid a few wide multiplications when that is the case.
This doesn't appear to be a win in the 32-bit implementation, so only
do it for the 64-bit one.
Add a new run_inverse_tests that replaces all existing field/scalar inverse tests,
and tests a few identities for fixed inputs, small numbers (-999...999), random
inputs (structured and unstructured), as well as comparing with the output of
secp256k1_fe_inv_all_var.
This commit adds functions to verify and compare numbers in signed{30,62} notation,
and uses that to do more extensive bounds checking on various variables in the modinv
code.
This adds tests for the modinv{32,64}_impl.h directly (before the functions are used
inside the field/scalar code). It uses a naive implementation of modular multiplication
and gcds in order to verify the modular inverses themselves.
This adds a long comment explaining the algorithm and implementation choices by building
it up step by step in Python.
Comments in the code are also reworked/added, with references to the long explanation.
9361f360bb ci: Select number of parallel make jobs depending on CI environment (Tim Ruffing)
28eccdf806 ci: Split output of logs into multiple sections (Tim Ruffing)
c7f754fe4d ci: Run PRs on merge result instead of on the source branch (Tim Ruffing)
b994a8be3c ci: Print information about binaries using "file" (Tim Ruffing)
f24e122d13 ci: Switch all Linux builds to Debian (Tim Ruffing)
Pull request description:
Best reviewed commit by commit
ACKs for top commit:
jonasnick:
ACK 9361f360bb
sipa:
utACK 9361f360bb
Tree-SHA512: fc754e8b57dc58058cebbf63a60ca76e08dbaefea1508ea27b7f962bce697c10033da3f57a35f731bc7cf3e210eb00e3b8985ae8b729d7bd83faee085b878b9c
The experiment of using Nix Shell was not really successful. Most
notably, Nix uses a bunch of wrapper scripts around compilers, which
make the build much less "pure". This may be useful but it's exactly
not what we want for CI. In particular, this resulted in gcc being used
for the "clang" builds because a wrapper script redefined the CC env
variable.
This now builds a single docker image (Debian) for all architectures
that we test in CI on Linux.
7d3497cdc4 ctime_test: move context randomization test to the end (Jonas Nick)
Pull request description:
ACKs for top commit:
real-or-random:
ACK 7d3497cdc4 diff looks good
Tree-SHA512: aef006c43df4cab254ee7de79cdd34c4e2f7a463f29d1da6d285006b32bb4e18d0b914a305f371b8b5f5a20594c37ee464eb1e59d1978db9b06bf6b642e651d8
This compiler flag is available for clang but not gcc.
Test plan:
```
autogen.sh
./configure
make check
CC=clang ./configure
make check
```
If a variable is used uninitialized, the warning should look something
like:
```
CC src/tests-tests.o
src/tests.c:4336:15: warning: variable 'recid' may be uninitialized when used here [-Wconditional-uninitialized]
CHECK(recid >= 0 && recid < 4);
^~~~~
./src/util.h:54:18: note: expanded from macro 'CHECK'
if (EXPECT(!(cond), 0)) { \
^~~~
./src/util.h:41:39: note: expanded from macro 'EXPECT'
^
src/tests.c:4327:14: note: initialize the variable 'recid' to silence this warning
int recid;
^
= 0
1 warning generated.
```
This was detected while running the tests with the `-Wconditional-uninitialized` flag
```
./autogen.sh
CC=clang CFLAGS="-Wconditional-uninitialized" ./configure
make check
```
The resulting warning is a false positive, but setting the value to -1
ensures that the CHECK below will fail if recid is never written to.
e491d06b98 Use bit ops instead of int mult for constant-time logic in gej_add_ge (Tim Ruffing)
Pull request description:
ACKs for top commit:
sipa:
utACK e491d06b98. Seems obviously better.
elichai:
ACK e491d06b98
jonasnick:
ACK e491d06b98
Tree-SHA512: 65977d3405e3b6c184c736d46898b615689b56f7562165114429dea49c0f9feb81d021cbe196c8a813b6239254b394cc24ac8d278dab37e521843a1bb0f70c47
Gregory Maxwell
Jonas Nick
Tim Ruffing
Pieter Wuille
Aaron Voisine
William Bright
Peter Dettman
PiRK