diff --git a/ceremony/.gitignore b/ceremony/.gitignore new file mode 100644 index 0000000..e7e1420 --- /dev/null +++ b/ceremony/.gitignore @@ -0,0 +1,2 @@ +.DS_Store +*.ptau diff --git a/ceremony/README.md b/ceremony/README.md new file mode 100644 index 0000000..4d253bd --- /dev/null +++ b/ceremony/README.md @@ -0,0 +1,7 @@ + +Place the "powers of tau" ceremony file(s) here. + +In the RLN case, normally a size `2^13 = 8192` ceremony should be enough. + +You can find a set of pre-made public ceremony files in the README of the +[`snarkjs` repo](https://github.com/iden3/snarkjs). diff --git a/workflow/.gitignore b/workflow/.gitignore new file mode 100644 index 0000000..aaf010f --- /dev/null +++ b/workflow/.gitignore @@ -0,0 +1,2 @@ +.DS_Store +build \ No newline at end of file diff --git a/workflow/PROOFS.md b/workflow/PROOFS.md new file mode 100644 index 0000000..4ac5b5f --- /dev/null +++ b/workflow/PROOFS.md @@ -0,0 +1,164 @@ + +The cogs of a ZK proof +====================== + +The goal of this document is to describe all the different moving parts +(circuit, witness, public input, etc) of a ZK proof through a simple example, +hopefully giving some intuition about how these cogs fit together. + + +The example: Fibonacci sequence +------------------------------- + +ZK proofs (or more precisely: "succint arguments") are all about convicing +another party (the "verifier"), about the truth of some statement, without +telling them _everything_. + +In our toy example, the statement will be the 1000th element of a Fibonacci +sequence. + +A Fibonacci sequence is generated deterministically from its first two elements, +let's call them `U` and `V`. These are numbers. The sequence is then defined as + + a[0] = U + a[1] = V + a[n] = a[n-1] + a[n-2] + +In the standard Fibonacci sequence, we have `U=0` and `V=1` (or sometimes `U=V=1`, +which results in the same sequence but shifted by one). But we can consider +this generalized setting with `U` and `V` being arbitrary numbers. + +The statement I want to convince you, is that for a given `U` (which I tell you, +say `U=53`), I know a value for `V` (which I don't tell you), such that `a[1000]` +is some given value (of course, I tell you this value too, otherwise there is +nothing to be convinced about). + +Pretty simple so far. + + +The statement +------------- + +A proof always start with a statement. Usually, the statement is descibed by +a computer program (in our case, the program is computing the Fibonacci sequence). +A program normally has some inputs and some ouputs. Some of the inputs are +public (I tell you what they are), some others can be secret (I don't tell you what +they are). And the statement is _usually_ that if you run this program with the given +inputs, then the program runs normally (it doesn't throw an error) and you get the +given output. + +Note: you can freely move things from outputs to inputs. In our case, the output +would be `a[1000]`, but equivalently, I can make another program where there is +a third input `W`, and the program just throws an error at the end if `a[1000] != W`. +Because of this, we often just say IO for input/output, and "public IO" for the +part of it I tell you, and "private IO" or "secret IO" for the part I don't tell +you. + +So far we have: + +- the statement (a computer program) +- the public IO +- and the private IO + +The computer program is often called a "circuit", because for technical reasons +it often needs to be described as an "arithmetic circuit", which is something +similar to digital electronics circuits (but in math). This gets a bit more +complicated with zkVM-s, where the circuit itself is something like a CPU, and your +statement is an actual program running on that (virtual) CPU. + + +The witness +----------- + +How essentially all proof system works is to translate the statement you want +the prove into some math equations, and then prove that I know a solution for +those equations. Because of this, the prover needs to know not only the input +and the output, but all the intermediate results too - essentially a very detailed +log of all individual operations happening in the computer program describing +the statement. This huge logfile is called "the witness", and in practice it's +usually just a large array of numbers (more precisely, finite field elements). + +Unfortunately, the word "witness" is a bit overloaded: Sometimes it refers only +to the private IO, and sometimes to the whole log. While these are kind of the same +from a holistic point of view, it still makes sense to distinguish them, and +I will call the whole execution log the witness. + +Creating a proof always start by running the program which descibes statement, +and creating this very detailed log, the witness (sometimes also called a +"transcript"). This step is usually called the "witness generation". + + +Trusted setup +------------- + +Some, but not all, proof systems requires something called a "trusted setup". +This is some structured piece of information produced by an external party, +which has some secret encoded in it. It's important that the secret is +deleted (hence the word "trusted") after generating this setup, because anybody +in possesion of the secret can cheat and create fake proofs. + +For example in the popular scheme called "KZG commitments", the secret is +a number (a field element) usually called `tau`, and the result of the +trusted setup is a sequence + + [ g , tau*g , tau^2*g , ... , tau^N*g ] + +where `g` is a fixed group element. Because of this, the procedure generating +this sequence is called a "powers-of-tau ceremony". Assuming the discrete +logarithm problem is hard in the group, it's not feasible to compute `tau` +itself given that sequence, but it's easy to compute `f(tau)*g` for a polynomial +`f` with degree at most `N`. + +In practice such a trusted setup can be accomplished using multiparty computation +(MPC), so that if there is at least _one_ honest participant, then the resulting +sequence is safe to use. These events are called "ceremonies" (in early days +people actually had to do this in person, and at the end they ceremonially destroyed +the computer which contained the random secret `tau`). + +There are two kinds of trusted setups: universal and circuit-specific. An universal +one can be used to prove many different statements, while a circuit-specific one +can be used to prove a single fixed statement, or circuit (but with different inputs). + +For example Groth16 needs a circuit-specific trusted setup, Plonk+KZG only needs +a universal trusted setup, while Plonk+FRI does not need any such setup at all. +However usually systems with trusted setups are more efficient (for example Groth16 has +the smallest proofs among all known proof system). In case of Groth16, the circuit-specific +setup is generated from an already existing universal setup and the circuit. + + +The files +--------- + +Using the popular [`circom`](https://docs.circom.io/) + [`snarkjs`](https://github.com/iden3/snarkjs) +ecosystem, all the above parts are in different files, which are produced by +different commands. + +These are: + +- `.circom` files contain the source code of your circuit (both the equations and + the actual program, interleaved together) +- `input.json` contains the circuit inputs (both public and private inputs) +- `.r1cs` file contains the circuit, that is, the statement you want to prove + (R1CS is short for Rank-1 Constraint System, a specific form of a "circuit"). + This is one output of the `circom` compiler, which reads the above source code and produces `.r1cs` files +- `.wtns` files contain the witness. This is generated by the so-called "witness generator", which is + another output of the `circom` compiler. Essentially the compiler separates the equations (goes into `.r1cs`) + and running the program (goes into the witness generator, which in this case can be either a WASM or a C++ program). + When you run the witness generator, it takes the `input.json` and produces the `.wtns` file +- `.ptau` files are containing universal trusted setups +- `.zkey` files are called a "prover key" (it contains everything the prover needs), and + in case of Groth16 it also corresponds to a circuit-specific trusted setup. +- the prover takes the `.zkey` and the `.wtns` files, and produces two outputs: a `proof.json` + containing the proof itself, and a `public.json` containing only the public input (so this + is copied from `input.json`, but the latter also contains the private inputs) +- from the `.zkey` file, a "verification key" can be extracted (this is again `.json` file), + which contains everything the verifier needs (in particular, it contains something like a + hash of the circuit. At least once this has to be checked against a source code, trusted setup + and resulting `.zkey`, so that you actually know which statement you verify! + However you don't want to do this agin and again, because the statement - the circuit - can be very big). + In practice the verifier key is often hardcoded in the on-chain verifier contract. +- then finally the verifier takes this verification key, the proof file, and the public input file + (all `.json` files here), and outputs either "valid" or "invalid" + +The command line workflow of doing all this is described in the file +[README.md](README.md). diff --git a/workflow/README.md b/workflow/README.md new file mode 100644 index 0000000..273e944 --- /dev/null +++ b/workflow/README.md @@ -0,0 +1,134 @@ + +Guide though the whole proof workflow +------------------------------------- + +The workflow described below is implemented with shell scripts in this directory. +So the below is more like an explanation. + +To run the full workflow: + +- set the parameters by editing `params.sh` +- run `setup.sh` to do the circuit-specific setup +- run `prove.sh` to generate input, compute witness and create (and verify) the proof + +NOTE: the examples below assume `bash`. In particular, it won't work with `zsh` +(which is the dafault on newer macOS)! Because, you know, reasons... + +To have an overview of what all the different steps and files are, see [PROOFS.md](PROOFS.md). + +### Preliminaries + +- install `circom`, `snarkjs`, `rapidsnark`: +- furthermore install `circom-witnesscalc`: (note: we need the legacy `build-circuit` version!) +- install Nim: + +TODO: fix this: +Build the Nim cli proof input generator: + + $ cd ../reference/nim/proof_input/ + $ nimble build -d:release cli + $ cd ../../../workflow + +### Powers of tau setup + +Either download a ready-to-use "powers of tau" setup file (section 7), or generate one +youself using `snarkjs` (sections 1..7), see the README here: + +Size `2^13` (file size about 10MB) should be big enough: + + $ cd ../ceremony + $ wget https://storage.googleapis.com/zkevm/ptau/powersOfTau28_hez_final_13.ptau + $ cd ../workflow + +Note: generating this yourself will probably take quite some time (though this size is relatively small, so maybe not that bad). + +### Set the parameters + +There are quite a few parameters (run `cli --help` too see them), it's probably +best to collect them into a parameter file. Check out `params.sh` and `cli_args.sh` +to see one way to do that. + +You can edit `params.sh` to your taste before running the workflow scripts. + +### Compile the circuit + +Create a build directory so we don't pollute the repo: + + $ mkdir -p build + $ cd build + +After that, the first real step is to create the main component: + + $ source ../cli_args.sh && ../../reference/nim/proof_input/cli $CLI_ARGS -v --circom="rln_main.circom" + +Then compile the circuit: + + $ export CIRCUIT_LIBDIRS="-l../../circuit/lib -l../../circuit/poseidon2 -l../../circuit/codex" + $ circom --r1cs --wasm --O2 ${CIRCUIT_LIBDIRS} rln_main.circom + +### Extract the witness computation graph + + $ build-circuit rln_main.circom rln_main.graph ${CIRCUIT_LIBDIRS} + +### Do the circuit-specific setup + +See the [`snarkjs` README](https://github.com/iden3/snarkjs) for an overview of +the whole process. + + $ snarkjs groth16 setup rln_main.r1cs ../../ceremony/powersOfTau28_hez_final_21.ptau rln_main_0000.zkey + $ snarkjs zkey contribute rln_main_0000.zkey rln_main_0001.zkey --name="1st Contributor Name" + +NOTE: with large circuits, javascript can run out of heap. You can increase the +heap limit with (but as this is a small circuit, this is not necessary): + + $ NODE_OPTIONS="--max-old-space-size=8192" snarkjs groth16 setup <...> + +You can add more contributors here if you want. + +Finally rename the last contributions result and export the verification key: + + $ rm rln_main_0000.zkey + $ mv rln_main_0001.zkey rln_main.zkey + + $ snarkjs zkey export verificationkey rln_main.zkey rln_main_verification_key.json + +NOTE: You have redo all the above if you change any of the five parameters the circuit +depends on (these are: maxdepth, maxslots, cellsize, blocksize, nsamples). + +### Generate an input to the circuit + + $ source ../cli_args.sh && ../../reference/nim/proof_input/cli $CLI_ARGS -v --output=input.json + +### Generate the witness + + $ cd rln_main_js + $ time node generate_witness.js rln_main.wasm ../input.json ../witness.wtns + $ cd .. + +### Create the proof + +Using `snarkjs` (very slow, but more portable): + + $ snarkjs groth16 prove rln_main.zkey witness.wtns proof.json public.json + +Or using `rapidsnark` (fast, but not very portable): + + $ rapidsnark rln_main.zkey witness.wtns proof.json public.json + +Or using `nim-groth16` (experimental): + + $ nim-groth16 -p -z=rln_main.zkey -w=witness.wtns -o=proof.json -i=public.json + +The output of this step will consist of: + +- `proof.json` containing the proof itself +- `public.json` containing the public inputs + +### Verify the proof (on CPU) + + $ snarkjs groth16 verify rln_main_verification_key.json public.json proof.json + +### Generate solidity verifier contract + + $ snarkjs zkey export solidityverifier rln_main.zkey verifier.sol + diff --git a/workflow/cli_args.sh b/workflow/cli_args.sh new file mode 100644 index 0000000..810c99b --- /dev/null +++ b/workflow/cli_args.sh @@ -0,0 +1,15 @@ +#!/bin/bash + +MY_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd ) + +source ${MY_DIR}/params.sh + +CLI_ARGS="--merkle_depth=${MERKLE_DEPTH} \ + --limit_bits=${LIMIT_BITS} \ + --seed=$SEED" + +if [[ "$1" == "--export" ]] +then + echo "exporting CLI_ARGS" + export CLI_ARGS +fi diff --git a/workflow/params.sh b/workflow/params.sh new file mode 100644 index 0000000..edf2539 --- /dev/null +++ b/workflow/params.sh @@ -0,0 +1,6 @@ +#!/bin/bash + +MERKLE_DEPTH=20 # depth of the Merkle tree +LIMIT_BITS=16 # log2 of the maximal possible rate limit per epoch + +SEED=1234567 # seed for creating fake data diff --git a/workflow/paths.sh b/workflow/paths.sh new file mode 100644 index 0000000..a813cb6 --- /dev/null +++ b/workflow/paths.sh @@ -0,0 +1,14 @@ +#!/bin/bash + +ORIG=`pwd` + +NIMCLI_DIR="${ORIG}/../test_data/proof_input/" +PTAU_DIR="${ORIG}/../ceremony" + +CIRCUIT_ROOT="${ORIG}/../circuit/" +CIRCUIT_POS_DIR="${CIRCUIT_ROOT}/poseidon2/" + +PTAU_FILE="powersOfTau28_hez_final_13.ptau" +PTAU_PATH="${PTAU_DIR}/${PTAU_FILE}" + +CIRCUIT_MAIN="rln_main" diff --git a/workflow/prove.sh b/workflow/prove.sh new file mode 100755 index 0000000..f0898de --- /dev/null +++ b/workflow/prove.sh @@ -0,0 +1,91 @@ +#!/bin/bash + +source ./paths.sh +source ./cli_args.sh + +# --- setup build directory --- + +mkdir -p build +cd build + +# --- export the witness computation graph --- + +if command -v build-circuit +then + CIRCUIT_INCLUDES="-l${CIRCUIT_LIB_DIR} -l${CIRCUIT_POS_DIR} -l${CIRCUIT_PRF_DIR}" + build-circuit ${CIRCUIT_MAIN}.circom ${CIRCUIT_MAIN}_graph.bin ${CIRCUIT_INCLUDES} +else + echo " " + echo "\`circom-witnesscalc\` not found; skipping graph extraction" +fi + +# --- generate input for the circuit --- + +echo "" +echo "generating the input for the proof circuit..." +${NIMCLI_DIR}/cli $CLI_ARGS -v --output=input.json + +# --- generate the witness --- + +start=`date +%s` +echo "" +echo "generating the witness..." +cd ${CIRCUIT_MAIN}_js +time node generate_witness.js ${CIRCUIT_MAIN}.wasm ../input.json ../witness.wtns +cd ${ORIG}/build +end=`date +%s` +echo "Generating the witness took `expr $end - $start` seconds." + +# --- create the proof --- + +PROVER="snarkjs" + +RS=`which rapidsnark` +if [[ ! -z "$RS" ]] +then + PROVER="rapidsnark" +fi + +# PROVER="zikkurat" +# PROVER="nim" + +echo "" +echo "creating the proof... using prover: \`$PROVER\`" + +start=`date +%s` +case $PROVER in + snarkjs) + time snarkjs groth16 prove ${CIRCUIT_MAIN}.zkey witness.wtns proof.json public.json + ;; + rapidsnark) + time rapidsnark ${CIRCUIT_MAIN}.zkey witness.wtns proof.json public.json + ;; + nim) + time nim-groth16 -tpv --zkey=${CIRCUIT_MAIN}.zkey --wtns=witness.wtns -o=proof.json -i=public.json + ;; + zikkurat) + time zikkurat-groth16 -tpv --zkey=${CIRCUIT_MAIN}.zkey --wtns=witness.wtns # -o=proof.json -i=public.json + ;; + *) + echo "unknown prover \`$PROVER\`" + exit 99 + ;; +esac +end=`date +%s` +echo "Creating the proof took `expr $end - $start` seconds." + +# --- verify the proof --- + +echo "" +echo "verifying the proof:" +snarkjs groth16 verify ${CIRCUIT_MAIN}_verification_key.json public.json proof.json + +# --- create solidity verifier contract --- + +echo "" +echo "creating solidity verifier contract:" +snarkjs zkey export solidityverifier ${CIRCUIT_MAIN}.zkey verifier.sol + +# --- finish --- + +cd $ORIG diff --git a/workflow/setup.sh b/workflow/setup.sh new file mode 100755 index 0000000..d9b0737 --- /dev/null +++ b/workflow/setup.sh @@ -0,0 +1,48 @@ +#!/bin/bash + +source ./paths.sh +source ./cli_args.sh + +# --- setup build directory --- + +mkdir -p build +cd build + +# --- generate the main component --- + +### ${NIMCLI_DIR}/cli $CLI_ARGS -v --circom=${CIRCUIT_MAIN}.circom + +cp ${CIRCUIT_ROOT}/example_main.circom ./${CIRCUIT_MAIN}.circom + +# --- compile the circuit --- + +echo "" +start=`date +%s` +CIRCUIT_INCLUDES="-l${CIRCUIT_ROOT} -l${CIRCUIT_POS_DIR}" +circom --r1cs --wasm --O2 ${CIRCUIT_INCLUDES} ${CIRCUIT_MAIN}.circom +end=`date +%s` +echo "Compiling the circuit took `expr $end - $start` seconds." +echo "" + +# --- extract the witness computation graph --- + +build-circuit --O2 ${CIRCUIT_MAIN}.circom ${CIRCUIT_MAIN}.graph ${CIRCUIT_INCLUDES} + +# --- circuit specific setup --- + +start=`date +%s` + +NODE_OPTIONS="--max-old-space-size=8192" snarkjs groth16 setup ${CIRCUIT_MAIN}.r1cs $PTAU_PATH ${CIRCUIT_MAIN}_0000.zkey +echo "some_entropy_75289v3b7rcawcsyiur" | \ +NODE_OPTIONS="--max-old-space-size=8192" snarkjs zkey contribute ${CIRCUIT_MAIN}_0000.zkey ${CIRCUIT_MAIN}_0001.zkey --name="1st Contributor Name" + +rm ${CIRCUIT_MAIN}_0000.zkey +mv ${CIRCUIT_MAIN}_0001.zkey ${CIRCUIT_MAIN}.zkey +snarkjs zkey export verificationkey ${CIRCUIT_MAIN}.zkey ${CIRCUIT_MAIN}_verification_key.json + +end=`date +%s` +echo "The circuit specific setup took `expr $end - $start` seconds." + +# --- finish the setup --- + +cd $ORIG