plonky2-verifier/commentary/Challenges.md

Fiat-Shamir Challenges

The verifier challenges are genered via Fiat-Shamir heuristics.

This uses the hash permutation in a duplex construction, alternatively absorbing the transcript and squeezing challenge elements. This is implemented in iop/challenger.rs.

All the challenges in the proof are summarized in the following data structure

struct ProofChallenges<F: RichField + Extendable<D>, const D: usize> {
  plonk_betas:    Vec<F>,         // Random values used in Plonk's permutation argument.
  plonk_gammas:   Vec<F>,         // Random values used in Plonk's permutation argument.
  plonk_alphas:   Vec<F>,         // Random values used to combine PLONK constraints.
  plonk_deltas:   Vec<F>,         // Lookup challenges (4 x num_challenges many). 
  plonk_zeta:     F::Extension,   // Point at which the PLONK polynomials are opened.
  fri_challenges: FriChallenges<F, D>,
}

And the FRI-specific challenges are:

struct FriChallenges<F: RichField + Extendable<D>, const D: usize> {
  fri_alpha: F::Extension,         // Scaling factor to combine polynomials.
  fri_betas: Vec<F::Extension>,    // Betas used in the FRI commit phase reductions.
  fri_pow_response:  F,            // proof-of-work challenge response
  fri_query_indices: Vec<usize>,   // Indices at which the oracle is queried in FRI.
}

Duplex construction

The duplex construction is similar the sponge construction, but it interleaves absorbing and squeezing. This is a very natural fit for Fiat-Shamir, as these are exactlty the operations we do:

• absorb the prover's message (transcript so far);
• then generate some challenges;
• then the prover replies with more messsages;
• based on which we generate more challenges;
• and so on...

We have to be careful with the details though.

To have a nice API, we want an API which can absorb and generate various data types (Goldilocks elements, field extension elements, hash digests, etc).

The duplex can be modelled as a state machine:

• We are either in absorbing or squeezing mode.
• In absorbing mode, we just collect inputs in a buffer
• In squeezing mode, we have a buffer initialized by rate elements of the state, from which we can produce outputs, until it becomes empty; then we do a permutation which refills the buffer
• When switching from absorbing mode to squeezing mode, we absorb the collected buffer exactly like a sponge.

Duplex algorithm pseudo code

type State = [F]

permute :: State -> State
permute = ...

rate = 8

data DuplexState 
  = Absorbing State [F]
  | Squeezing State [F]

initialDuplexState = Absorbing zeroState []

-- Absorb an element
absorb :: F -> DuplexState -> DuplexState
absorb what duplex = case duplex of
  Absorbing state list -> Absorbing state (list ++ [what])
  Squeezing state _    -> Absorbing state [what]

-- Squeeze an element
squeeze :: DuplexState -> (F, DuplexState)
squeeze duplex = case duplex of
  Absorbing state list -> squeezing (sponge list state)
  Squeezing state buf  -> case buf of
    []     -> squeezing (permute state)
    (y:ys) -> (y, Squeezing state ys)

-- Helper function
squeezing :: State -> (F, DuplexState)
squeezing state = squeeze $ Squeezing state (extract state)

-- Classic sponge in overwrite mode
sponge :: [F] -> State -> State
sponge = go where
  go []   state = state
  go list state = case splitAt rate list of
    (this,rest) -> go rest (permute $ overwrite this state)

-- Plonky2 uses "overwrite mode"
overwrite :: [F] -> State -> State
overwrite input state 
  | k > rate   = error "overwrite: expecting at most `rate` elements"
  | otherwise  = input ++ drop k state
  where
    k = length input

-- The reverse is because Plonky2 uses `pop` (from the end of) the buffer 
extract :: State -> [F]
extract = reverse . take rate

Transcript

Usually the communication (in an IOP) between the prover and the verifier is called "the transcript", and the Fiat-Shamir challenger should absorb all messages of the prover.

The duplex state is initialized by absorbing the "circuit digest".

This is the hash of the following data:

• the Merkle cap of the constant columns (including the selectors and permutation sigmas)
• the hash of the optional domain separator data (which is by default an empty vector)
• the size (number of rows) of the circuit

Thus the challenge generation starts by absorbing:

• the circuit digest
• the hash of the public inputs
• the Merkle cap of the witness matrix commitment

Then the \beta\in\mathbb{F}^r and \gamma\in\mathbb{F}^r challenges are generated, where r = num_challenges.

If lookups are present, next the lookup challenges are generated. This is a bit ugly. We need 4\times r such challenges, but as an optimization, the \beta,\gamma are reused. So 2\times r more \delta challenges are generated, then these are concatenated into (\beta\|\gamma\|\delta)\in\mathbb{F}^{4r}, and finally this vector is chunked into r pieces of 4-vectors...

Next, the Merkle cap of the partial product columns is absorbed; and after that, the \alpha\in\mathbb{F}^r combining challenges are generated.

Then, the Merkle cap of the quotient polynomials is absorbed, and the \zeta\in\widetilde{\mathbb{F}} evaluation point is generated.

Finally, the FRI challenges are generated.

FRI challenges

First, we absorb all the opening (a full row, involving all the 4 committed matrix; and some parts of the "next row").

Then the \alpha\in\widetilde{\mathbb{F}} combining challenge is generated (NOTE: this is different from the above $\alpha$-s!)

Next, the commit_phase_merkle_caps are absorbed, and after each one, a \beta_i\in\widetilde{\mathbb{F}} is generated (again, different $\beta$-s from above!).

Then we absorb the coefficients of the final (low-degree) folded FRI polynomial. This is at most 2^5=32 coefficients in the default configuration.

Next, the proof-of-work "grinding" is handled. This is done a bit strange way: first we absorb the candidate prover witness, then we generate the response, and check the leading zeros of that. I guess you can get away with 1 less hashing in the verifier this way...

Finally, we generate the FRI query indices. These are indices of rows in the LDE matrix, that is, 0 \le q_i < 2^{n+\mathtt{rate\_bits}}.

For this, we generate num_query_rounds field elements, and take them modulo this size.