c-kzg-4844/README.md

117 lines
4.0 KiB
Markdown
Raw Normal View History

# c-kzg - work in progress
2021-02-01 20:15:45 +00:00
The very beginnings of a simple implementation of [KZG commitments](https://dankradfeist.de/ethereum/2020/06/16/kate-polynomial-commitments.html) in C, using the [Blst library](https://github.com/supranational/blst) from Supranational for field and curve operations.
Initially, at least, this largely follows the [go-kzg](https://github.com/protolambda/go-kzg) implementation.
Done so far:
2021-02-03 17:00:23 +00:00
- FFT and inverse FFT over the finite field.
- FFTs over the G1 group
2021-02-05 20:48:56 +00:00
- Polynomial single commitment and verification
- Polynomial multi commitment and verification
2021-02-15 20:48:16 +00:00
- [FK20](https://github.com/khovratovich/Kate/blob/master/Kate_amortized.pdf) single proof method (normal, and optimised for data availability)
- FK20 multi proof method (normal, and optimised for data availability)
2021-02-17 12:25:03 +00:00
- Polynomial extension for data availability sampling
2021-02-27 15:21:53 +00:00
- Calculation of zero polynomials
- Data recovery from samples
2021-02-01 20:15:45 +00:00
2021-03-01 11:39:18 +00:00
That's basically all the necessary stuff for Eth2 use cases. Things remaining (aside from tidying up, which is never ending):
- [ ] Document the underlying mathematics to help with checking correctness of the implementation and writing better tests
- [ ] End-to-end tests
- [ ] Performance tuning
- [ ] Robustness checking (don't crash on errors; return the correct error codes; no buffer overflows)
- [ ] Java interface
- [ ] (Optional) Use alternative back-end libraries to Blst (e.g. [Herumi mcl](https://github.com/herumi/mcl))
- [ ] Nice build process
- [ ] Make it portable
2021-02-11 08:21:13 +00:00
## Install
2021-02-01 20:15:45 +00:00
Build the [Blst library](https://github.com/supranational/blst) following the instructions there. Then,
1. Copy the resulting `libblst.a` file into the `lib/` directory here.
2. From Blst's `bindings/` directory copy `blst.h` and `blst_aux.h` to `inc/`
That is,
```
cp ../blst/libblast.a lib/
cp ../blst/bindings/*.h inc/
```
2021-02-03 17:00:23 +00:00
## Build
Build the `libckzg.a` library:
```
cd src
make lib
```
Build a debug version that aborts on error conditions and attempts to print some helpful info (file, line number, condition that failed):
```
cd src
make debuglib
```
2021-02-01 20:15:45 +00:00
## Run tests
```
cd src
2021-02-01 20:15:45 +00:00
make test
```
2021-02-03 17:00:23 +00:00
Unit tests for an individual file can be built and run with `make fft_fr_test` for example. Once a test runner such as *fft_fr_test* has been built, individual unit tests can be run with `./fft_fr_test <test-name>`.
2021-02-01 20:15:45 +00:00
Thanks to [Acutest](https://github.com/mity/acutest) for the unit test harness, which is used here under the MIT licence.
2021-02-08 08:39:24 +00:00
## Run benchmarks
This will run all available benchmarks, for the default one second per test size:
```
cd src
make bench
```
2021-03-01 11:39:18 +00:00
You can run individual benchmarks, and optionally specify how long (in seconds) to run each test size:
2021-02-08 08:39:24 +00:00
```
make fft_fr_bench
./fft_fr_bench 5
```
Doing `make clean` should resolve any weird build issues.
2021-02-10 11:55:38 +00:00
## Make debug builds of the tests
2021-02-28 11:43:54 +00:00
The default build is designed not to crash on errors, and will (should) return fairly coarse error codes for any issue. This is good for a utility library, but unhelpful for debugging. The `-DDEBUG` compiler flag builds a version such that any assertion failure aborts the run and outputs file and line info. This is much more useful for tracking down deeply buried errors.
2021-02-10 11:55:38 +00:00
Each test suite can be compiled into its debug version. For example, as follows:
```
make fk20_proofs_test_debug
./fk20_proofs_test_debug fk_single_strided
```
2021-02-17 15:18:46 +00:00
This magic is implemented via the `CHECK` and `TRY` macros in _c_kzg.h_.
2021-02-10 11:57:33 +00:00
2021-02-11 08:21:13 +00:00
## Make documentation
2021-02-12 16:56:32 +00:00
`doxygen` style comments are in place throughout, although some places need more work. Build the docs in the top directory as follows:
2021-02-11 08:21:13 +00:00
```
doxygen Doxyfile
```
2021-02-12 16:56:32 +00:00
This will produce a _doc/html_ directory. Visit the _doc/html/files.html_ file in a browser to view the documentation.
2021-02-11 08:21:13 +00:00
## Prerequisites
- Blst library (see above)
- `clang` compiler. I'm using Clang 10.0.0. I'll likely add `gcc` options in future.
2021-02-10 11:55:38 +00:00
- The Makefile is GNU make compatible.
- I'm developing on Ubuntu 20.04. Will check portability later.
2021-02-11 08:21:13 +00:00
- [Doxygen](https://www.doxygen.nl/index.html) for building the documentation. I'm using v1.8.17 right now.