mirror of
https://github.com/logos-storage/zk-benchmarks.git
synced 2026-01-05 23:33:07 +00:00
47 lines
1.8 KiB
Markdown
47 lines
1.8 KiB
Markdown
|
|
||
|
|
Benchmarking framework
|
||
|
|
----------------------
|
||
|
|
|
||
|
|
The role of this program is to build, setup, and run benchmarks with various
|
||
|
|
parameter settings, collecting the timing results together.
|
||
|
|
|
||
|
|
A benchmark is defined by the following shell scripts:
|
||
|
|
|
||
|
|
- `build.sh` - build the code
|
||
|
|
- `setup.sh` - run some additional setup, for example Groth16 circuit-specific setup
|
||
|
|
- `witness.sh` - run witness generation for SNARKs (separate from `setup` because we may want to measure it)
|
||
|
|
- `run.sh` - run the benchmark itself
|
||
|
|
|
||
|
|
These are run in this order, and results are cached unless explicitely requested.
|
||
|
|
All except `run.sh` are optional.
|
||
|
|
|
||
|
|
Recommended organization is to put all build artifacts into a `build` subdirectory.
|
||
|
|
|
||
|
|
Benchmarks can be parameterized using environment variables. By convention, we
|
||
|
|
start the names of these environment variables with the `ZKBENCH_` prefix.
|
||
|
|
|
||
|
|
An additional file `benchmark.cfg` specifies the configuration and parameter ranges.
|
||
|
|
Example file:
|
||
|
|
|
||
|
|
name: "Poseidon2 Groth16 benchmarks"
|
||
|
|
timeout: 300
|
||
|
|
rerunFrom: build
|
||
|
|
params:
|
||
|
|
[ PROVER: [ snarkjs, rapidsnark ]
|
||
|
|
, INPUT_SIZE: [ 256, 512, 1024, 2048 ]
|
||
|
|
, WHICH: [ hash_sponge, hash_sponge_rate2, hash_merkle ]
|
||
|
|
]
|
||
|
|
|
||
|
|
Note: in case of an arithmetic circuit, every step of the build process must be
|
||
|
|
rerun if the circuit changes, and the circuit depends on the input size...
|
||
|
|
The `rerunFrom` parameter allows to set this. Normally you want it te be `run`
|
||
|
|
(only rerun the `run.sh` script), but in case of Groth16 you want that to be `build`.
|
||
|
|
|
||
|
|
`timeout` (in seconds) sets the maximum target time we should spend on this specific
|
||
|
|
benchmark. If the initial run is fast enough, we will rerun it up to 10 times
|
||
|
|
and everage them to get a less noisy result.
|
||
|
|
|
||
|
|
`params` corresponds to the possible values of the corresponding environment
|
||
|
|
variables (in this example, `ZKBENCH_PROVER`, etc)
|
||
|
|
|