c-kzg - work in progress

The very beginnings of a simple implementation of KZG commitments in C, using the Blst library from Supranational for field and curve operations.

Initially, at least, this largely follows the go-kzg implementation.

Done so far:

• FFT and inverse FFT over the finite field.
• FFTs over the G1 group
• Polynomial single commitment and verification
• Polynomial multi commitment and verification
• FK20 single proof method (normal, and optimised for data availability)
• FK20 multi proof method (normal, and optimised for data availability)
• Polynomial extension for data availability sampling
• Calculation of zero polynomials
• Data recovery from samples

That's basically all the necessary stuff for Eth2 use cases. Things remaining (aside from tidying up, which is never ending):

• Document the underlying mathematics to help with checking correctness of the implementation and writing better tests
• End-to-end tests
• Performance tuning
• Robustness checking (don't crash on errors; return the correct error codes; no buffer overflows)
• Java interface: see jc-kzg
• (Optional) Use alternative back-end libraries to Blst (e.g. Herumi mcl)
• Nice build process
• Make it portable

Install

Build the Blst library following the instructions there. Then,

1. Copy the resulting libblst.a file into the lib/ directory here.
2. From Blst's bindings/ directory copy blst.h and blst_aux.h to inc/

That is,

cp ../blst/libblast.a lib/
cp ../blst/bindings/*.h inc/

This version of c-kzg is tested with Blst's master branch, commit d4b40c3. Blst release 0.3.4 is not sufficient since we make use of the more recently implemented Pippenger multiscalar multiplication for the polynomial commitments.

Build

Build the libckzg.a library:

cd src
make lib

Build a debug version that aborts on error conditions and attempts to print some helpful info (file, line number, condition that failed):

cd src
make debuglib

Run tests

cd src
make test

Unit tests for an individual file can be built and run with make fft_fr_test for example. Once a test runner such as fft_fr_test has been built, individual unit tests can be run with ./fft_fr_test <test-name>.

Thanks to Acutest for the unit test harness, which is used here under the MIT licence.

Run benchmarks

This will run all available benchmarks, for the default one second per test size:

cd src
make bench

You can run individual benchmarks, and optionally specify how long (in seconds) to run each test size:

make fft_fr_bench
./fft_fr_bench 5

Doing make clean should resolve any weird build issues.

Make debug builds of the tests

The default build is designed not to crash on errors, and will (should) return fairly coarse error codes for any issue. This is good for a utility library, but unhelpful for debugging. The -DDEBUG compiler flag builds a version such that any assertion failure aborts the run and outputs file and line info. This is much more useful for tracking down deeply buried errors.

Each test suite can be compiled into its debug version. For example, as follows:

make fk20_proofs_test_debug
./fk20_proofs_test_debug fk_single_strided

This magic is implemented via the CHECK and TRY macros in c_kzg.h.

Make documentation

doxygen style comments are in place throughout, although some places need more work. Build the docs in the top directory as follows:

doxygen Doxyfile

This will produce a doc/html directory. Visit the doc/html/files.html file in a browser to view the documentation.

Prerequisites

• Blst library (see above)
• clang compiler. I'm using Clang 10.0.0. I'll likely add gcc options in future.
• The Makefile is GNU make compatible.
• I'm developing on Ubuntu 20.04. Will check portability later.
• Doxygen for building the documentation. I'm using v1.8.17 right now.