eth2.0-specs/specs/_features/eip7594/polynomial-commitments-samp...

709 lines
26 KiB
Markdown
Raw Normal View History

# EIP-7594 -- Polynomial Commitments Sampling
## Table of contents
<!-- TOC -->
<!-- START doctoc generated TOC please keep comment here to allow auto update -->
<!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE -->
- [Introduction](#introduction)
- [Public Methods](#public-methods)
- [Custom types](#custom-types)
- [Constants](#constants)
- [Preset](#preset)
2023-12-23 14:52:39 +00:00
- [Cells](#cells)
- [Helper functions](#helper-functions)
2024-01-15 16:27:55 +00:00
- [BLS12-381 helpers](#bls12-381-helpers)
- [`cell_to_coset_evals`](#cell_to_coset_evals)
- [`coset_evals_to_cell`](#coset_evals_to_cell)
- [Linear combinations](#linear-combinations)
- [`g2_lincomb`](#g2_lincomb)
- [FFTs](#ffts)
- [`_fft_field`](#_fft_field)
- [`fft_field`](#fft_field)
chore: Refactor cell recovery code (#3781) * multi: - Remove shift_polynomial_coeff - Remove recover_shifted_data - Remove recover_original_data - Move `zero_poly_eval_brp ` under sanity check comment as its only used for sanity checking * chore: remove sanity check -- this was doing a wasteful `compute_root_of_unity` operation * chore: add previous sanity check as a unit test * chore: copy values python was taking a reference, so it passes in our regular codepaths but not in isolated test * chore: add coset_fft test * Update specs/_features/eip7594/polynomial-commitments-sampling.md Co-authored-by: Justin Traglia <95511699+jtraglia@users.noreply.github.com> * Update specs/_features/eip7594/polynomial-commitments-sampling.md Co-authored-by: Justin Traglia <95511699+jtraglia@users.noreply.github.com> * chore: linter * chore: asn (switch to bls_modular_inverse) * chore: (ben) rename func to test_construct_vanishing_polynomial * chore: (ben) rename `extended_evaluations_coeffs` to `extended_evaluation_times_zero_coeffs` * chore: compute `roots_of_unity_extended` in recover_data method * chore: add more comments explaining whats happening in recover_data * chore: compute_zero_poly_coeff in recover_data * chore: make lint * chore: add doc comment to coset_fft_field * chore: (ben) add code to generate the vanishing polynomial when all cells are missing * chore: remove handling of edge case when constructing a vanishing polynomial * chore: rename H(x) to Q_3(x) * chore: remove trailing whitespace * chore: add whitespace between comments * chore: (asn) add assert that num missing cells is not 0 * chore: (justin) address comments * chore: merge resolution * chore: fixup remaining IDs -> indices * chore: use indice nomenclature in tests --------- Co-authored-by: Justin Traglia <95511699+jtraglia@users.noreply.github.com>
2024-06-11 17:26:19 +00:00
- [`coset_fft_field`](#coset_fft_field)
- [Polynomials in coefficient form](#polynomials-in-coefficient-form)
- [`polynomial_eval_to_coeff`](#polynomial_eval_to_coeff)
- [`add_polynomialcoeff`](#add_polynomialcoeff)
- [`neg_polynomialcoeff`](#neg_polynomialcoeff)
- [`multiply_polynomialcoeff`](#multiply_polynomialcoeff)
- [`divide_polynomialcoeff`](#divide_polynomialcoeff)
- [`interpolate_polynomialcoeff`](#interpolate_polynomialcoeff)
- [`vanishing_polynomialcoeff`](#vanishing_polynomialcoeff)
- [`evaluate_polynomialcoeff`](#evaluate_polynomialcoeff)
- [KZG multiproofs](#kzg-multiproofs)
- [`compute_kzg_proof_multi_impl`](#compute_kzg_proof_multi_impl)
- [`verify_kzg_proof_multi_impl`](#verify_kzg_proof_multi_impl)
2023-12-23 14:52:39 +00:00
- [Cell cosets](#cell-cosets)
- [`coset_for_cell`](#coset_for_cell)
- [Cells](#cells-1)
- [Cell computation](#cell-computation)
- [`compute_cells_and_kzg_proofs`](#compute_cells_and_kzg_proofs)
2023-12-23 14:52:39 +00:00
- [Cell verification](#cell-verification)
- [`verify_cell_kzg_proof`](#verify_cell_kzg_proof)
- [`verify_cell_kzg_proof_batch`](#verify_cell_kzg_proof_batch)
- [Reconstruction](#reconstruction)
- [`construct_vanishing_polynomial`](#construct_vanishing_polynomial)
chore: Refactor cell recovery code (#3781) * multi: - Remove shift_polynomial_coeff - Remove recover_shifted_data - Remove recover_original_data - Move `zero_poly_eval_brp ` under sanity check comment as its only used for sanity checking * chore: remove sanity check -- this was doing a wasteful `compute_root_of_unity` operation * chore: add previous sanity check as a unit test * chore: copy values python was taking a reference, so it passes in our regular codepaths but not in isolated test * chore: add coset_fft test * Update specs/_features/eip7594/polynomial-commitments-sampling.md Co-authored-by: Justin Traglia <95511699+jtraglia@users.noreply.github.com> * Update specs/_features/eip7594/polynomial-commitments-sampling.md Co-authored-by: Justin Traglia <95511699+jtraglia@users.noreply.github.com> * chore: linter * chore: asn (switch to bls_modular_inverse) * chore: (ben) rename func to test_construct_vanishing_polynomial * chore: (ben) rename `extended_evaluations_coeffs` to `extended_evaluation_times_zero_coeffs` * chore: compute `roots_of_unity_extended` in recover_data method * chore: add more comments explaining whats happening in recover_data * chore: compute_zero_poly_coeff in recover_data * chore: make lint * chore: add doc comment to coset_fft_field * chore: (ben) add code to generate the vanishing polynomial when all cells are missing * chore: remove handling of edge case when constructing a vanishing polynomial * chore: rename H(x) to Q_3(x) * chore: remove trailing whitespace * chore: add whitespace between comments * chore: (asn) add assert that num missing cells is not 0 * chore: (justin) address comments * chore: merge resolution * chore: fixup remaining IDs -> indices * chore: use indice nomenclature in tests --------- Co-authored-by: Justin Traglia <95511699+jtraglia@users.noreply.github.com>
2024-06-11 17:26:19 +00:00
- [`recover_data`](#recover_data)
- [`recover_cells_and_kzg_proofs`](#recover_cells_and_kzg_proofs)
<!-- END doctoc generated TOC please keep comment here to allow auto update -->
<!-- /TOC -->
## Introduction
This document extends [polynomial-commitments.md](https://github.com/ethereum/consensus-specs/blob/dev/specs/deneb/polynomial-commitments.md) with the functions required for data availability sampling (DAS). It is not part of the core Deneb spec but an extension that can be optionally implemented to allow nodes to reduce their load using DAS.
## Public Methods
For any KZG library extended to support DAS, functions flagged as "Public method" MUST be provided by the underlying KZG library as public functions. All other functions are private functions used internally by the KZG library.
Public functions MUST accept raw bytes as input and perform the required cryptographic normalization before invoking any internal functions.
The following is a list of the public methods:
chore: Refactor cell recovery code (#3781) * multi: - Remove shift_polynomial_coeff - Remove recover_shifted_data - Remove recover_original_data - Move `zero_poly_eval_brp ` under sanity check comment as its only used for sanity checking * chore: remove sanity check -- this was doing a wasteful `compute_root_of_unity` operation * chore: add previous sanity check as a unit test * chore: copy values python was taking a reference, so it passes in our regular codepaths but not in isolated test * chore: add coset_fft test * Update specs/_features/eip7594/polynomial-commitments-sampling.md Co-authored-by: Justin Traglia <95511699+jtraglia@users.noreply.github.com> * Update specs/_features/eip7594/polynomial-commitments-sampling.md Co-authored-by: Justin Traglia <95511699+jtraglia@users.noreply.github.com> * chore: linter * chore: asn (switch to bls_modular_inverse) * chore: (ben) rename func to test_construct_vanishing_polynomial * chore: (ben) rename `extended_evaluations_coeffs` to `extended_evaluation_times_zero_coeffs` * chore: compute `roots_of_unity_extended` in recover_data method * chore: add more comments explaining whats happening in recover_data * chore: compute_zero_poly_coeff in recover_data * chore: make lint * chore: add doc comment to coset_fft_field * chore: (ben) add code to generate the vanishing polynomial when all cells are missing * chore: remove handling of edge case when constructing a vanishing polynomial * chore: rename H(x) to Q_3(x) * chore: remove trailing whitespace * chore: add whitespace between comments * chore: (asn) add assert that num missing cells is not 0 * chore: (justin) address comments * chore: merge resolution * chore: fixup remaining IDs -> indices * chore: use indice nomenclature in tests --------- Co-authored-by: Justin Traglia <95511699+jtraglia@users.noreply.github.com>
2024-06-11 17:26:19 +00:00
- [`compute_cells_and_kzg_proofs`](#compute_cells_and_kzg_proofs)
- [`verify_cell_kzg_proof`](#verify_cell_kzg_proof)
- [`verify_cell_kzg_proof_batch`](#verify_cell_kzg_proof_batch)
- [`recover_cells_and_kzg_proofs`](#recover_cells_and_kzg_proofs)
## Custom types
| Name | SSZ equivalent | Description |
| - | - | - |
| `PolynomialCoeff` | `List[BLSFieldElement, FIELD_ELEMENTS_PER_EXT_BLOB]` | A polynomial in coefficient form |
| `Coset` | `Vector[BLSFieldElement, FIELD_ELEMENTS_PER_CELL]` | The evaluation domain of a cell |
| `CosetEvals` | `Vector[BLSFieldElement, FIELD_ELEMENTS_PER_CELL]` | The internal representation of a cell (the evaluations over its Coset) |
| `Cell` | `ByteVector[BYTES_PER_FIELD_ELEMENT * FIELD_ELEMENTS_PER_CELL]` | The unit of blob data that can come with its own KZG proof |
2024-06-11 12:41:52 +00:00
| `CellIndex` | `uint64` | Validation: `x < CELLS_PER_EXT_BLOB` |
## Constants
| Name | Value | Notes |
| - | - | - |
## Preset
2023-12-23 14:52:39 +00:00
### Cells
Cells are the smallest unit of blob data that can come with their own KZG proofs. Samples can be constructed from one or several cells (e.g. an individual cell or line).
| Name | Value | Description |
| - | - | - |
| `FIELD_ELEMENTS_PER_EXT_BLOB` | `2 * FIELD_ELEMENTS_PER_BLOB` | Number of field elements in a Reed-Solomon extended blob |
2023-12-23 14:44:58 +00:00
| `FIELD_ELEMENTS_PER_CELL` | `uint64(64)` | Number of field elements in a cell |
| `BYTES_PER_CELL` | `FIELD_ELEMENTS_PER_CELL * BYTES_PER_FIELD_ELEMENT` | The number of bytes in a cell |
2024-04-18 16:38:04 +00:00
| `CELLS_PER_EXT_BLOB` | `FIELD_ELEMENTS_PER_EXT_BLOB // FIELD_ELEMENTS_PER_CELL` | The number of cells in an extended blob |
| `RANDOM_CHALLENGE_KZG_CELL_BATCH_DOMAIN` | `b'RCKZGCBATCH__V1_'` |
## Helper functions
2024-01-15 16:27:55 +00:00
### BLS12-381 helpers
#### `cell_to_coset_evals`
2024-01-15 16:27:55 +00:00
```python
def cell_to_coset_evals(cell: Cell) -> CosetEvals:
2024-01-15 16:27:55 +00:00
"""
Convert an untrusted ``Cell`` into a trusted ``CosetEvals``.
2024-01-15 16:27:55 +00:00
"""
evals = []
for i in range(FIELD_ELEMENTS_PER_CELL):
start = i * BYTES_PER_FIELD_ELEMENT
end = (i + 1) * BYTES_PER_FIELD_ELEMENT
value = bytes_to_bls_field(cell[start:end])
evals.append(value)
return CosetEvals(evals)
```
#### `coset_evals_to_cell`
```python
def coset_evals_to_cell(coset_evals: CosetEvals) -> Cell:
"""
Convert a trusted ``CosetEval`` into an untrusted ``Cell``.
"""
cell = []
for i in range(FIELD_ELEMENTS_PER_CELL):
cell += bls_field_to_bytes(coset_evals[i])
return Cell(cell)
2024-01-15 16:27:55 +00:00
```
### Linear combinations
#### `g2_lincomb`
```python
def g2_lincomb(points: Sequence[G2Point], scalars: Sequence[BLSFieldElement]) -> Bytes96:
"""
2024-04-23 13:21:54 +00:00
BLS multiscalar multiplication in G2. This can be naively implemented using double-and-add.
"""
assert len(points) == len(scalars)
2024-04-23 13:55:37 +00:00
if len(points) == 0:
return bls.G2_to_bytes96(bls.Z2())
2024-04-23 13:55:37 +00:00
2024-04-23 12:38:27 +00:00
points_g2 = []
for point in points:
points_g2.append(bls.bytes96_to_G2(point))
2024-04-23 13:55:37 +00:00
2024-04-23 13:21:54 +00:00
result = bls.multi_exp(points_g2, scalars)
return Bytes96(bls.G2_to_bytes96(result))
```
### FFTs
#### `_fft_field`
```python
2024-01-05 12:31:09 +00:00
def _fft_field(vals: Sequence[BLSFieldElement],
roots_of_unity: Sequence[BLSFieldElement]) -> Sequence[BLSFieldElement]:
if len(vals) == 1:
2023-12-23 14:44:58 +00:00
return vals
L = _fft_field(vals[::2], roots_of_unity[::2])
R = _fft_field(vals[1::2], roots_of_unity[::2])
2024-01-05 12:31:09 +00:00
o = [BLSFieldElement(0) for _ in vals]
for i, (x, y) in enumerate(zip(L, R)):
2024-01-05 12:31:09 +00:00
y_times_root = (int(y) * int(roots_of_unity[i])) % BLS_MODULUS
o[i] = BLSFieldElement((int(x) + y_times_root) % BLS_MODULUS)
o[i + len(L)] = BLSFieldElement((int(x) - y_times_root + BLS_MODULUS) % BLS_MODULUS)
return o
```
#### `fft_field`
```python
2024-01-05 12:31:09 +00:00
def fft_field(vals: Sequence[BLSFieldElement],
roots_of_unity: Sequence[BLSFieldElement],
inv: bool=False) -> Sequence[BLSFieldElement]:
if inv:
# Inverse FFT
invlen = pow(len(vals), BLS_MODULUS - 2, BLS_MODULUS)
2024-01-05 12:31:09 +00:00
return [BLSFieldElement((int(x) * invlen) % BLS_MODULUS)
for x in _fft_field(vals, list(roots_of_unity[0:1]) + list(roots_of_unity[:0:-1]))]
else:
# Regular FFT
return _fft_field(vals, roots_of_unity)
```
chore: Refactor cell recovery code (#3781) * multi: - Remove shift_polynomial_coeff - Remove recover_shifted_data - Remove recover_original_data - Move `zero_poly_eval_brp ` under sanity check comment as its only used for sanity checking * chore: remove sanity check -- this was doing a wasteful `compute_root_of_unity` operation * chore: add previous sanity check as a unit test * chore: copy values python was taking a reference, so it passes in our regular codepaths but not in isolated test * chore: add coset_fft test * Update specs/_features/eip7594/polynomial-commitments-sampling.md Co-authored-by: Justin Traglia <95511699+jtraglia@users.noreply.github.com> * Update specs/_features/eip7594/polynomial-commitments-sampling.md Co-authored-by: Justin Traglia <95511699+jtraglia@users.noreply.github.com> * chore: linter * chore: asn (switch to bls_modular_inverse) * chore: (ben) rename func to test_construct_vanishing_polynomial * chore: (ben) rename `extended_evaluations_coeffs` to `extended_evaluation_times_zero_coeffs` * chore: compute `roots_of_unity_extended` in recover_data method * chore: add more comments explaining whats happening in recover_data * chore: compute_zero_poly_coeff in recover_data * chore: make lint * chore: add doc comment to coset_fft_field * chore: (ben) add code to generate the vanishing polynomial when all cells are missing * chore: remove handling of edge case when constructing a vanishing polynomial * chore: rename H(x) to Q_3(x) * chore: remove trailing whitespace * chore: add whitespace between comments * chore: (asn) add assert that num missing cells is not 0 * chore: (justin) address comments * chore: merge resolution * chore: fixup remaining IDs -> indices * chore: use indice nomenclature in tests --------- Co-authored-by: Justin Traglia <95511699+jtraglia@users.noreply.github.com>
2024-06-11 17:26:19 +00:00
#### `coset_fft_field`
```python
def coset_fft_field(vals: Sequence[BLSFieldElement],
roots_of_unity: Sequence[BLSFieldElement],
inv: bool=False) -> Sequence[BLSFieldElement]:
"""
Computes an FFT/IFFT over a coset of the roots of unity.
This is useful for when one wants to divide by a polynomial which
vanishes on one or more elements in the domain.
"""
vals = vals.copy()
def shift_vals(vals: Sequence[BLSFieldElement], factor: BLSFieldElement) -> Sequence[BLSFieldElement]:
"""
Multiply each entry in `vals` by succeeding powers of `factor`
i.e., [vals[0] * factor^0, vals[1] * factor^1, ..., vals[n] * factor^n]
"""
shift = 1
for i in range(len(vals)):
vals[i] = BLSFieldElement((int(vals[i]) * shift) % BLS_MODULUS)
shift = (shift * int(factor)) % BLS_MODULUS
return vals
# This is the coset generator; it is used to compute a FFT/IFFT over a coset of
# the roots of unity.
shift_factor = BLSFieldElement(PRIMITIVE_ROOT_OF_UNITY)
if inv:
vals = fft_field(vals, roots_of_unity, inv)
shift_inv = bls_modular_inverse(shift_factor)
return shift_vals(vals, shift_inv)
else:
vals = shift_vals(vals, shift_factor)
return fft_field(vals, roots_of_unity, inv)
```
### Polynomials in coefficient form
#### `polynomial_eval_to_coeff`
```python
def polynomial_eval_to_coeff(polynomial: Polynomial) -> PolynomialCoeff:
"""
Interpolates a polynomial (given in evaluation form) to a polynomial in coefficient form.
"""
roots_of_unity = compute_roots_of_unity(FIELD_ELEMENTS_PER_BLOB)
polynomial_coeff = fft_field(bit_reversal_permutation(list(polynomial)), roots_of_unity, inv=True)
return polynomial_coeff
```
#### `add_polynomialcoeff`
```python
def add_polynomialcoeff(a: PolynomialCoeff, b: PolynomialCoeff) -> PolynomialCoeff:
"""
Sum the coefficient form polynomials ``a`` and ``b``.
"""
a, b = (a, b) if len(a) >= len(b) else (b, a)
2024-01-16 08:51:05 +00:00
length_a = len(a)
length_b = len(b)
return [(a[i] + (b[i] if i < length_b else 0)) % BLS_MODULUS for i in range(length_a)]
```
#### `neg_polynomialcoeff`
```python
def neg_polynomialcoeff(a: PolynomialCoeff) -> PolynomialCoeff:
"""
Negative of coefficient form polynomial ``a``
"""
return [(BLS_MODULUS - x) % BLS_MODULUS for x in a]
```
#### `multiply_polynomialcoeff`
```python
def multiply_polynomialcoeff(a: PolynomialCoeff, b: PolynomialCoeff) -> PolynomialCoeff:
"""
Multiplies the coefficient form polynomials ``a`` and ``b``
"""
assert len(a) + len(b) <= FIELD_ELEMENTS_PER_EXT_BLOB
r = [0]
for power, coef in enumerate(a):
summand = [0] * power + [int(coef) * int(x) % BLS_MODULUS for x in b]
r = add_polynomialcoeff(r, summand)
return r
```
#### `divide_polynomialcoeff`
```python
def divide_polynomialcoeff(a: PolynomialCoeff, b: PolynomialCoeff) -> PolynomialCoeff:
"""
Long polynomial division for two coefficient form polynomials ``a`` and ``b``
"""
a = a.copy() # Make a copy since `a` is passed by reference
o: List[BLSFieldElement] = []
apos = len(a) - 1
bpos = len(b) - 1
diff = apos - bpos
while diff >= 0:
quot = div(a[apos], b[bpos])
o.insert(0, quot)
for i in range(bpos, -1, -1):
a[diff + i] = (int(a[diff + i]) - int(b[i] + BLS_MODULUS) * int(quot)) % BLS_MODULUS
apos -= 1
diff -= 1
return [x % BLS_MODULUS for x in o]
```
#### `interpolate_polynomialcoeff`
```python
def interpolate_polynomialcoeff(xs: Sequence[BLSFieldElement], ys: Sequence[BLSFieldElement]) -> PolynomialCoeff:
"""
Lagrange interpolation: Finds the lowest degree polynomial that takes the value ``ys[i]`` at ``x[i]``
for all i.
Outputs a coefficient form polynomial. Leading coefficients may be zero.
"""
assert len(xs) == len(ys)
r = [0]
for i in range(len(xs)):
summand = [ys[i]]
for j in range(len(ys)):
if j != i:
weight_adjustment = bls_modular_inverse(int(xs[i]) - int(xs[j]))
summand = multiply_polynomialcoeff(
summand, [((BLS_MODULUS - int(weight_adjustment)) * int(xs[j])) % BLS_MODULUS, weight_adjustment]
)
r = add_polynomialcoeff(r, summand)
2024-01-15 15:34:28 +00:00
return r
```
#### `vanishing_polynomialcoeff`
```python
def vanishing_polynomialcoeff(xs: Sequence[BLSFieldElement]) -> PolynomialCoeff:
"""
Compute the vanishing polynomial on ``xs`` (in coefficient form)
"""
p = [1]
for x in xs:
p = multiply_polynomialcoeff(p, [-int(x) + BLS_MODULUS, 1])
return p
```
#### `evaluate_polynomialcoeff`
```python
def evaluate_polynomialcoeff(polynomial_coeff: PolynomialCoeff, z: BLSFieldElement) -> BLSFieldElement:
"""
Evaluate a coefficient form polynomial at ``z`` using Horner's schema
"""
y = 0
for coef in polynomial_coeff[::-1]:
2024-01-05 12:31:09 +00:00
y = (int(y) * int(z) + int(coef)) % BLS_MODULUS
return BLSFieldElement(y % BLS_MODULUS)
```
### KZG multiproofs
Extended KZG functions for multiproofs
#### `compute_kzg_proof_multi_impl`
```python
2023-12-12 16:25:30 +00:00
def compute_kzg_proof_multi_impl(
polynomial_coeff: PolynomialCoeff,
zs: Coset) -> Tuple[KZGProof, CosetEvals]:
"""
2024-04-18 16:52:33 +00:00
Compute a KZG multi-evaluation proof for a set of `k` points.
This is done by committing to the following quotient polynomial:
2024-04-22 08:57:58 +00:00
Q(X) = f(X) - I(X) / Z(X)
2024-04-18 16:52:33 +00:00
Where:
2024-04-22 08:57:58 +00:00
- I(X) is the degree `k-1` polynomial that agrees with f(x) at all `k` points
2024-04-18 16:52:33 +00:00
- Z(X) is the degree `k` polynomial that evaluates to zero on all `k` points
2024-04-22 08:57:58 +00:00
We further note that since the degree of I(X) is less than the degree of Z(X),
the computation can be simplified in monomial form to Q(X) = f(X) / Z(X)
"""
# For all points, compute the evaluation of those points
ys = [evaluate_polynomialcoeff(polynomial_coeff, z) for z in zs]
2024-04-18 16:52:33 +00:00
# Compute Z(X)
denominator_poly = vanishing_polynomialcoeff(zs)
2024-04-18 16:52:33 +00:00
# Compute the quotient polynomial directly in monomial form
quotient_polynomial = divide_polynomialcoeff(polynomial_coeff, denominator_poly)
return KZGProof(g1_lincomb(KZG_SETUP_G1_MONOMIAL[:len(quotient_polynomial)], quotient_polynomial)), ys
```
#### `verify_kzg_proof_multi_impl`
```python
def verify_kzg_proof_multi_impl(commitment: KZGCommitment,
zs: Coset,
ys: CosetEvals,
proof: KZGProof) -> bool:
"""
2024-04-19 18:19:17 +00:00
Verify a KZG multi-evaluation proof for a set of `k` points.
This is done by checking if the following equation holds:
2024-04-22 08:57:58 +00:00
Q(x) Z(x) = f(X) - I(X)
2024-04-19 18:19:17 +00:00
Where:
2024-04-22 08:57:58 +00:00
f(X) is the polynomial that we want to verify opens at `k` points to `k` values
2024-04-19 18:19:17 +00:00
Q(X) is the quotient polynomial computed by the prover
2024-04-22 08:57:58 +00:00
I(X) is the degree k-1 polynomial that evaluates to `ys` at all `zs`` points
2024-04-19 18:19:17 +00:00
Z(X) is the polynomial that evaluates to zero on all `k` points
The verifier receives the commitments to Q(X) and f(X), so they check the equation
holds by using the following pairing equation:
2024-04-22 08:57:58 +00:00
e([Q(X)]_1, [Z(X)]_2) == e([f(X)]_1 - [I(X)]_1, [1]_2)
"""
assert len(zs) == len(ys)
2024-04-19 18:19:17 +00:00
# Compute [Z(X)]_2
zero_poly = g2_lincomb(KZG_SETUP_G2_MONOMIAL[:len(zs) + 1], vanishing_polynomialcoeff(zs))
2024-04-22 08:57:58 +00:00
# Compute [I(X)]_1
interpolated_poly = g1_lincomb(KZG_SETUP_G1_MONOMIAL[:len(zs)], interpolate_polynomialcoeff(zs, ys))
2023-12-12 16:25:30 +00:00
return (bls.pairing_check([
[bls.bytes48_to_G1(proof), bls.bytes96_to_G2(zero_poly)],
[
bls.add(bls.bytes48_to_G1(commitment), bls.neg(bls.bytes48_to_G1(interpolated_poly))),
bls.neg(bls.bytes96_to_G2(KZG_SETUP_G2_MONOMIAL[0])),
2023-12-12 16:25:30 +00:00
],
]))
```
2023-12-23 14:52:39 +00:00
### Cell cosets
2023-12-23 14:52:39 +00:00
#### `coset_for_cell`
```python
2024-06-11 12:41:52 +00:00
def coset_for_cell(cell_index: CellIndex) -> Coset:
"""
2024-06-11 12:41:52 +00:00
Get the coset for a given ``cell_index``.
"""
2024-06-11 12:41:52 +00:00
assert cell_index < CELLS_PER_EXT_BLOB
roots_of_unity_brp = bit_reversal_permutation(
compute_roots_of_unity(FIELD_ELEMENTS_PER_EXT_BLOB)
)
2024-06-11 12:41:52 +00:00
return Coset(roots_of_unity_brp[FIELD_ELEMENTS_PER_CELL * cell_index:FIELD_ELEMENTS_PER_CELL * (cell_index + 1)])
```
2023-12-23 14:52:39 +00:00
## Cells
2023-12-23 14:52:39 +00:00
### Cell computation
#### `compute_cells_and_kzg_proofs`
```python
def compute_cells_and_kzg_proofs(blob: Blob) -> Tuple[
2024-04-18 16:38:04 +00:00
Vector[Cell, CELLS_PER_EXT_BLOB],
Vector[KZGProof, CELLS_PER_EXT_BLOB]]:
"""
2024-04-18 17:11:41 +00:00
Compute all the cell proofs for an extended blob. This is an inefficient O(n^2) algorithm,
for performant implementation the FK20 algorithm that runs in O(n log n) should be
used instead.
2023-12-04 01:26:07 +00:00
Public method.
"""
assert len(blob) == BYTES_PER_BLOB
polynomial = blob_to_polynomial(blob)
polynomial_coeff = polynomial_eval_to_coeff(polynomial)
2023-12-23 14:52:39 +00:00
cells = []
proofs = []
2024-04-18 16:38:04 +00:00
for i in range(CELLS_PER_EXT_BLOB):
2024-06-11 12:41:52 +00:00
coset = coset_for_cell(CellIndex(i))
proof, ys = compute_kzg_proof_multi_impl(polynomial_coeff, coset)
cells.append(coset_evals_to_cell(ys))
proofs.append(proof)
2023-12-23 14:52:39 +00:00
return cells, proofs
```
2023-12-23 14:52:39 +00:00
### Cell verification
#### `verify_cell_kzg_proof`
```python
def verify_cell_kzg_proof(commitment_bytes: Bytes48,
2024-06-11 12:41:52 +00:00
cell_index: CellIndex,
2024-04-25 22:34:10 +00:00
cell: Cell,
proof_bytes: Bytes48) -> bool:
"""
2023-12-23 14:52:39 +00:00
Check a cell proof
2023-12-04 01:26:07 +00:00
2023-12-23 14:44:58 +00:00
Public method.
"""
assert len(commitment_bytes) == BYTES_PER_COMMITMENT
2024-06-11 12:41:52 +00:00
assert cell_index < CELLS_PER_EXT_BLOB
assert len(cell) == BYTES_PER_CELL
assert len(proof_bytes) == BYTES_PER_PROOF
2024-06-11 12:41:52 +00:00
coset = coset_for_cell(cell_index)
2024-01-15 16:27:55 +00:00
return verify_kzg_proof_multi_impl(
bytes_to_kzg_commitment(commitment_bytes),
coset,
cell_to_coset_evals(cell),
2024-01-17 12:18:03 +00:00
bytes_to_kzg_proof(proof_bytes))
```
#### `verify_cell_kzg_proof_batch`
```python
def verify_cell_kzg_proof_batch(row_commitments_bytes: Sequence[Bytes48],
2024-04-25 22:34:10 +00:00
row_indices: Sequence[RowIndex],
column_indices: Sequence[ColumnIndex],
cells: Sequence[Cell],
proofs_bytes: Sequence[Bytes48]) -> bool:
"""
chore: Refactor cell recovery code (#3781) * multi: - Remove shift_polynomial_coeff - Remove recover_shifted_data - Remove recover_original_data - Move `zero_poly_eval_brp ` under sanity check comment as its only used for sanity checking * chore: remove sanity check -- this was doing a wasteful `compute_root_of_unity` operation * chore: add previous sanity check as a unit test * chore: copy values python was taking a reference, so it passes in our regular codepaths but not in isolated test * chore: add coset_fft test * Update specs/_features/eip7594/polynomial-commitments-sampling.md Co-authored-by: Justin Traglia <95511699+jtraglia@users.noreply.github.com> * Update specs/_features/eip7594/polynomial-commitments-sampling.md Co-authored-by: Justin Traglia <95511699+jtraglia@users.noreply.github.com> * chore: linter * chore: asn (switch to bls_modular_inverse) * chore: (ben) rename func to test_construct_vanishing_polynomial * chore: (ben) rename `extended_evaluations_coeffs` to `extended_evaluation_times_zero_coeffs` * chore: compute `roots_of_unity_extended` in recover_data method * chore: add more comments explaining whats happening in recover_data * chore: compute_zero_poly_coeff in recover_data * chore: make lint * chore: add doc comment to coset_fft_field * chore: (ben) add code to generate the vanishing polynomial when all cells are missing * chore: remove handling of edge case when constructing a vanishing polynomial * chore: rename H(x) to Q_3(x) * chore: remove trailing whitespace * chore: add whitespace between comments * chore: (asn) add assert that num missing cells is not 0 * chore: (justin) address comments * chore: merge resolution * chore: fixup remaining IDs -> indices * chore: use indice nomenclature in tests --------- Co-authored-by: Justin Traglia <95511699+jtraglia@users.noreply.github.com>
2024-06-11 17:26:19 +00:00
Verify a set of cells, given their corresponding proofs and their coordinates (row_index, column_index) in the blob
2024-01-17 15:20:36 +00:00
matrix. The list of all commitments is also provided in row_commitments_bytes.
This function implements the naive algorithm of checking every cell
individually; an efficient algorithm can be found here:
https://ethresear.ch/t/a-universal-verification-equation-for-data-availability-sampling/13240
2023-12-04 01:26:07 +00:00
This implementation does not require randomness, but for the algorithm that
requires it, `RANDOM_CHALLENGE_KZG_CELL_BATCH_DOMAIN` should be used to compute
the challenge value.
2023-12-04 01:26:07 +00:00
Public method.
"""
assert len(cells) == len(proofs_bytes) == len(row_indices) == len(column_indices)
for commitment_bytes in row_commitments_bytes:
assert len(commitment_bytes) == BYTES_PER_COMMITMENT
for row_index in row_indices:
assert row_index < len(row_commitments_bytes)
for column_index in column_indices:
assert column_index < CELLS_PER_EXT_BLOB
for cell in cells:
assert len(cell) == BYTES_PER_CELL
for proof_bytes in proofs_bytes:
assert len(proof_bytes) == BYTES_PER_PROOF
chore: Refactor cell recovery code (#3781) * multi: - Remove shift_polynomial_coeff - Remove recover_shifted_data - Remove recover_original_data - Move `zero_poly_eval_brp ` under sanity check comment as its only used for sanity checking * chore: remove sanity check -- this was doing a wasteful `compute_root_of_unity` operation * chore: add previous sanity check as a unit test * chore: copy values python was taking a reference, so it passes in our regular codepaths but not in isolated test * chore: add coset_fft test * Update specs/_features/eip7594/polynomial-commitments-sampling.md Co-authored-by: Justin Traglia <95511699+jtraglia@users.noreply.github.com> * Update specs/_features/eip7594/polynomial-commitments-sampling.md Co-authored-by: Justin Traglia <95511699+jtraglia@users.noreply.github.com> * chore: linter * chore: asn (switch to bls_modular_inverse) * chore: (ben) rename func to test_construct_vanishing_polynomial * chore: (ben) rename `extended_evaluations_coeffs` to `extended_evaluation_times_zero_coeffs` * chore: compute `roots_of_unity_extended` in recover_data method * chore: add more comments explaining whats happening in recover_data * chore: compute_zero_poly_coeff in recover_data * chore: make lint * chore: add doc comment to coset_fft_field * chore: (ben) add code to generate the vanishing polynomial when all cells are missing * chore: remove handling of edge case when constructing a vanishing polynomial * chore: rename H(x) to Q_3(x) * chore: remove trailing whitespace * chore: add whitespace between comments * chore: (asn) add assert that num missing cells is not 0 * chore: (justin) address comments * chore: merge resolution * chore: fixup remaining IDs -> indices * chore: use indice nomenclature in tests --------- Co-authored-by: Justin Traglia <95511699+jtraglia@users.noreply.github.com>
2024-06-11 17:26:19 +00:00
# Get commitments via row indices
2024-01-29 09:29:20 +00:00
commitments_bytes = [row_commitments_bytes[row_index] for row_index in row_indices]
2024-01-15 16:27:55 +00:00
# Get objects from bytes
commitments = [bytes_to_kzg_commitment(commitment_bytes) for commitment_bytes in commitments_bytes]
cosets_evals = [cell_to_coset_evals(cell) for cell in cells]
2024-01-15 16:27:55 +00:00
proofs = [bytes_to_kzg_proof(proof_bytes) for proof_bytes in proofs_bytes]
return all(
verify_kzg_proof_multi_impl(commitment, coset_for_cell(column_index), coset_evals, proof)
for commitment, column_index, coset_evals, proof in zip(commitments, column_indices, cosets_evals, proofs)
)
```
## Reconstruction
### `construct_vanishing_polynomial`
```python
chore: Refactor cell recovery code (#3781) * multi: - Remove shift_polynomial_coeff - Remove recover_shifted_data - Remove recover_original_data - Move `zero_poly_eval_brp ` under sanity check comment as its only used for sanity checking * chore: remove sanity check -- this was doing a wasteful `compute_root_of_unity` operation * chore: add previous sanity check as a unit test * chore: copy values python was taking a reference, so it passes in our regular codepaths but not in isolated test * chore: add coset_fft test * Update specs/_features/eip7594/polynomial-commitments-sampling.md Co-authored-by: Justin Traglia <95511699+jtraglia@users.noreply.github.com> * Update specs/_features/eip7594/polynomial-commitments-sampling.md Co-authored-by: Justin Traglia <95511699+jtraglia@users.noreply.github.com> * chore: linter * chore: asn (switch to bls_modular_inverse) * chore: (ben) rename func to test_construct_vanishing_polynomial * chore: (ben) rename `extended_evaluations_coeffs` to `extended_evaluation_times_zero_coeffs` * chore: compute `roots_of_unity_extended` in recover_data method * chore: add more comments explaining whats happening in recover_data * chore: compute_zero_poly_coeff in recover_data * chore: make lint * chore: add doc comment to coset_fft_field * chore: (ben) add code to generate the vanishing polynomial when all cells are missing * chore: remove handling of edge case when constructing a vanishing polynomial * chore: rename H(x) to Q_3(x) * chore: remove trailing whitespace * chore: add whitespace between comments * chore: (asn) add assert that num missing cells is not 0 * chore: (justin) address comments * chore: merge resolution * chore: fixup remaining IDs -> indices * chore: use indice nomenclature in tests --------- Co-authored-by: Justin Traglia <95511699+jtraglia@users.noreply.github.com>
2024-06-11 17:26:19 +00:00
def construct_vanishing_polynomial(missing_cell_indices: Sequence[CellIndex]) -> Sequence[BLSFieldElement]:
"""
chore: Refactor cell recovery code (#3781) * multi: - Remove shift_polynomial_coeff - Remove recover_shifted_data - Remove recover_original_data - Move `zero_poly_eval_brp ` under sanity check comment as its only used for sanity checking * chore: remove sanity check -- this was doing a wasteful `compute_root_of_unity` operation * chore: add previous sanity check as a unit test * chore: copy values python was taking a reference, so it passes in our regular codepaths but not in isolated test * chore: add coset_fft test * Update specs/_features/eip7594/polynomial-commitments-sampling.md Co-authored-by: Justin Traglia <95511699+jtraglia@users.noreply.github.com> * Update specs/_features/eip7594/polynomial-commitments-sampling.md Co-authored-by: Justin Traglia <95511699+jtraglia@users.noreply.github.com> * chore: linter * chore: asn (switch to bls_modular_inverse) * chore: (ben) rename func to test_construct_vanishing_polynomial * chore: (ben) rename `extended_evaluations_coeffs` to `extended_evaluation_times_zero_coeffs` * chore: compute `roots_of_unity_extended` in recover_data method * chore: add more comments explaining whats happening in recover_data * chore: compute_zero_poly_coeff in recover_data * chore: make lint * chore: add doc comment to coset_fft_field * chore: (ben) add code to generate the vanishing polynomial when all cells are missing * chore: remove handling of edge case when constructing a vanishing polynomial * chore: rename H(x) to Q_3(x) * chore: remove trailing whitespace * chore: add whitespace between comments * chore: (asn) add assert that num missing cells is not 0 * chore: (justin) address comments * chore: merge resolution * chore: fixup remaining IDs -> indices * chore: use indice nomenclature in tests --------- Co-authored-by: Justin Traglia <95511699+jtraglia@users.noreply.github.com>
2024-06-11 17:26:19 +00:00
Given the cells indices that are missing from the data, compute the polynomial that vanishes at every point that
corresponds to a missing field element.
chore: Refactor cell recovery code (#3781) * multi: - Remove shift_polynomial_coeff - Remove recover_shifted_data - Remove recover_original_data - Move `zero_poly_eval_brp ` under sanity check comment as its only used for sanity checking * chore: remove sanity check -- this was doing a wasteful `compute_root_of_unity` operation * chore: add previous sanity check as a unit test * chore: copy values python was taking a reference, so it passes in our regular codepaths but not in isolated test * chore: add coset_fft test * Update specs/_features/eip7594/polynomial-commitments-sampling.md Co-authored-by: Justin Traglia <95511699+jtraglia@users.noreply.github.com> * Update specs/_features/eip7594/polynomial-commitments-sampling.md Co-authored-by: Justin Traglia <95511699+jtraglia@users.noreply.github.com> * chore: linter * chore: asn (switch to bls_modular_inverse) * chore: (ben) rename func to test_construct_vanishing_polynomial * chore: (ben) rename `extended_evaluations_coeffs` to `extended_evaluation_times_zero_coeffs` * chore: compute `roots_of_unity_extended` in recover_data method * chore: add more comments explaining whats happening in recover_data * chore: compute_zero_poly_coeff in recover_data * chore: make lint * chore: add doc comment to coset_fft_field * chore: (ben) add code to generate the vanishing polynomial when all cells are missing * chore: remove handling of edge case when constructing a vanishing polynomial * chore: rename H(x) to Q_3(x) * chore: remove trailing whitespace * chore: add whitespace between comments * chore: (asn) add assert that num missing cells is not 0 * chore: (justin) address comments * chore: merge resolution * chore: fixup remaining IDs -> indices * chore: use indice nomenclature in tests --------- Co-authored-by: Justin Traglia <95511699+jtraglia@users.noreply.github.com>
2024-06-11 17:26:19 +00:00
This method assumes that all of the cells cannot be missing. In this case the vanishing polynomial
could be computed as Z(x) = x^n - 1, where `n` is FIELD_ELEMENTS_PER_EXT_BLOB.
We never encounter this case however because this method is used solely for recovery and recovery only
works if at least half of the cells are available.
"""
chore: Refactor cell recovery code (#3781) * multi: - Remove shift_polynomial_coeff - Remove recover_shifted_data - Remove recover_original_data - Move `zero_poly_eval_brp ` under sanity check comment as its only used for sanity checking * chore: remove sanity check -- this was doing a wasteful `compute_root_of_unity` operation * chore: add previous sanity check as a unit test * chore: copy values python was taking a reference, so it passes in our regular codepaths but not in isolated test * chore: add coset_fft test * Update specs/_features/eip7594/polynomial-commitments-sampling.md Co-authored-by: Justin Traglia <95511699+jtraglia@users.noreply.github.com> * Update specs/_features/eip7594/polynomial-commitments-sampling.md Co-authored-by: Justin Traglia <95511699+jtraglia@users.noreply.github.com> * chore: linter * chore: asn (switch to bls_modular_inverse) * chore: (ben) rename func to test_construct_vanishing_polynomial * chore: (ben) rename `extended_evaluations_coeffs` to `extended_evaluation_times_zero_coeffs` * chore: compute `roots_of_unity_extended` in recover_data method * chore: add more comments explaining whats happening in recover_data * chore: compute_zero_poly_coeff in recover_data * chore: make lint * chore: add doc comment to coset_fft_field * chore: (ben) add code to generate the vanishing polynomial when all cells are missing * chore: remove handling of edge case when constructing a vanishing polynomial * chore: rename H(x) to Q_3(x) * chore: remove trailing whitespace * chore: add whitespace between comments * chore: (asn) add assert that num missing cells is not 0 * chore: (justin) address comments * chore: merge resolution * chore: fixup remaining IDs -> indices * chore: use indice nomenclature in tests --------- Co-authored-by: Justin Traglia <95511699+jtraglia@users.noreply.github.com>
2024-06-11 17:26:19 +00:00
assert len(missing_cell_indices) != 0
# Get the small domain
2024-04-18 16:38:04 +00:00
roots_of_unity_reduced = compute_roots_of_unity(CELLS_PER_EXT_BLOB)
# Compute polynomial that vanishes at all the missing cells (over the small domain)
2024-01-05 10:33:01 +00:00
short_zero_poly = vanishing_polynomialcoeff([
2024-06-11 12:41:52 +00:00
roots_of_unity_reduced[reverse_bits(missing_cell_index, CELLS_PER_EXT_BLOB)]
for missing_cell_index in missing_cell_indices
2024-01-05 10:33:01 +00:00
])
# Extend vanishing polynomial to full domain using the closed form of the vanishing polynomial over a coset
zero_poly_coeff = [BLSFieldElement(0)] * FIELD_ELEMENTS_PER_EXT_BLOB
for i, coeff in enumerate(short_zero_poly):
zero_poly_coeff[i * FIELD_ELEMENTS_PER_CELL] = coeff
chore: Refactor cell recovery code (#3781) * multi: - Remove shift_polynomial_coeff - Remove recover_shifted_data - Remove recover_original_data - Move `zero_poly_eval_brp ` under sanity check comment as its only used for sanity checking * chore: remove sanity check -- this was doing a wasteful `compute_root_of_unity` operation * chore: add previous sanity check as a unit test * chore: copy values python was taking a reference, so it passes in our regular codepaths but not in isolated test * chore: add coset_fft test * Update specs/_features/eip7594/polynomial-commitments-sampling.md Co-authored-by: Justin Traglia <95511699+jtraglia@users.noreply.github.com> * Update specs/_features/eip7594/polynomial-commitments-sampling.md Co-authored-by: Justin Traglia <95511699+jtraglia@users.noreply.github.com> * chore: linter * chore: asn (switch to bls_modular_inverse) * chore: (ben) rename func to test_construct_vanishing_polynomial * chore: (ben) rename `extended_evaluations_coeffs` to `extended_evaluation_times_zero_coeffs` * chore: compute `roots_of_unity_extended` in recover_data method * chore: add more comments explaining whats happening in recover_data * chore: compute_zero_poly_coeff in recover_data * chore: make lint * chore: add doc comment to coset_fft_field * chore: (ben) add code to generate the vanishing polynomial when all cells are missing * chore: remove handling of edge case when constructing a vanishing polynomial * chore: rename H(x) to Q_3(x) * chore: remove trailing whitespace * chore: add whitespace between comments * chore: (asn) add assert that num missing cells is not 0 * chore: (justin) address comments * chore: merge resolution * chore: fixup remaining IDs -> indices * chore: use indice nomenclature in tests --------- Co-authored-by: Justin Traglia <95511699+jtraglia@users.noreply.github.com>
2024-06-11 17:26:19 +00:00
return zero_poly_coeff
```
chore: Refactor cell recovery code (#3781) * multi: - Remove shift_polynomial_coeff - Remove recover_shifted_data - Remove recover_original_data - Move `zero_poly_eval_brp ` under sanity check comment as its only used for sanity checking * chore: remove sanity check -- this was doing a wasteful `compute_root_of_unity` operation * chore: add previous sanity check as a unit test * chore: copy values python was taking a reference, so it passes in our regular codepaths but not in isolated test * chore: add coset_fft test * Update specs/_features/eip7594/polynomial-commitments-sampling.md Co-authored-by: Justin Traglia <95511699+jtraglia@users.noreply.github.com> * Update specs/_features/eip7594/polynomial-commitments-sampling.md Co-authored-by: Justin Traglia <95511699+jtraglia@users.noreply.github.com> * chore: linter * chore: asn (switch to bls_modular_inverse) * chore: (ben) rename func to test_construct_vanishing_polynomial * chore: (ben) rename `extended_evaluations_coeffs` to `extended_evaluation_times_zero_coeffs` * chore: compute `roots_of_unity_extended` in recover_data method * chore: add more comments explaining whats happening in recover_data * chore: compute_zero_poly_coeff in recover_data * chore: make lint * chore: add doc comment to coset_fft_field * chore: (ben) add code to generate the vanishing polynomial when all cells are missing * chore: remove handling of edge case when constructing a vanishing polynomial * chore: rename H(x) to Q_3(x) * chore: remove trailing whitespace * chore: add whitespace between comments * chore: (asn) add assert that num missing cells is not 0 * chore: (justin) address comments * chore: merge resolution * chore: fixup remaining IDs -> indices * chore: use indice nomenclature in tests --------- Co-authored-by: Justin Traglia <95511699+jtraglia@users.noreply.github.com>
2024-06-11 17:26:19 +00:00
### `recover_data`
```python
chore: Refactor cell recovery code (#3781) * multi: - Remove shift_polynomial_coeff - Remove recover_shifted_data - Remove recover_original_data - Move `zero_poly_eval_brp ` under sanity check comment as its only used for sanity checking * chore: remove sanity check -- this was doing a wasteful `compute_root_of_unity` operation * chore: add previous sanity check as a unit test * chore: copy values python was taking a reference, so it passes in our regular codepaths but not in isolated test * chore: add coset_fft test * Update specs/_features/eip7594/polynomial-commitments-sampling.md Co-authored-by: Justin Traglia <95511699+jtraglia@users.noreply.github.com> * Update specs/_features/eip7594/polynomial-commitments-sampling.md Co-authored-by: Justin Traglia <95511699+jtraglia@users.noreply.github.com> * chore: linter * chore: asn (switch to bls_modular_inverse) * chore: (ben) rename func to test_construct_vanishing_polynomial * chore: (ben) rename `extended_evaluations_coeffs` to `extended_evaluation_times_zero_coeffs` * chore: compute `roots_of_unity_extended` in recover_data method * chore: add more comments explaining whats happening in recover_data * chore: compute_zero_poly_coeff in recover_data * chore: make lint * chore: add doc comment to coset_fft_field * chore: (ben) add code to generate the vanishing polynomial when all cells are missing * chore: remove handling of edge case when constructing a vanishing polynomial * chore: rename H(x) to Q_3(x) * chore: remove trailing whitespace * chore: add whitespace between comments * chore: (asn) add assert that num missing cells is not 0 * chore: (justin) address comments * chore: merge resolution * chore: fixup remaining IDs -> indices * chore: use indice nomenclature in tests --------- Co-authored-by: Justin Traglia <95511699+jtraglia@users.noreply.github.com>
2024-06-11 17:26:19 +00:00
def recover_data(cell_indices: Sequence[CellIndex],
cells: Sequence[Cell],
) -> Sequence[BLSFieldElement]:
"""
chore: Refactor cell recovery code (#3781) * multi: - Remove shift_polynomial_coeff - Remove recover_shifted_data - Remove recover_original_data - Move `zero_poly_eval_brp ` under sanity check comment as its only used for sanity checking * chore: remove sanity check -- this was doing a wasteful `compute_root_of_unity` operation * chore: add previous sanity check as a unit test * chore: copy values python was taking a reference, so it passes in our regular codepaths but not in isolated test * chore: add coset_fft test * Update specs/_features/eip7594/polynomial-commitments-sampling.md Co-authored-by: Justin Traglia <95511699+jtraglia@users.noreply.github.com> * Update specs/_features/eip7594/polynomial-commitments-sampling.md Co-authored-by: Justin Traglia <95511699+jtraglia@users.noreply.github.com> * chore: linter * chore: asn (switch to bls_modular_inverse) * chore: (ben) rename func to test_construct_vanishing_polynomial * chore: (ben) rename `extended_evaluations_coeffs` to `extended_evaluation_times_zero_coeffs` * chore: compute `roots_of_unity_extended` in recover_data method * chore: add more comments explaining whats happening in recover_data * chore: compute_zero_poly_coeff in recover_data * chore: make lint * chore: add doc comment to coset_fft_field * chore: (ben) add code to generate the vanishing polynomial when all cells are missing * chore: remove handling of edge case when constructing a vanishing polynomial * chore: rename H(x) to Q_3(x) * chore: remove trailing whitespace * chore: add whitespace between comments * chore: (asn) add assert that num missing cells is not 0 * chore: (justin) address comments * chore: merge resolution * chore: fixup remaining IDs -> indices * chore: use indice nomenclature in tests --------- Co-authored-by: Justin Traglia <95511699+jtraglia@users.noreply.github.com>
2024-06-11 17:26:19 +00:00
Recover the missing evaluations for the extended blob, given at least half of the evaluations.
"""
chore: Refactor cell recovery code (#3781) * multi: - Remove shift_polynomial_coeff - Remove recover_shifted_data - Remove recover_original_data - Move `zero_poly_eval_brp ` under sanity check comment as its only used for sanity checking * chore: remove sanity check -- this was doing a wasteful `compute_root_of_unity` operation * chore: add previous sanity check as a unit test * chore: copy values python was taking a reference, so it passes in our regular codepaths but not in isolated test * chore: add coset_fft test * Update specs/_features/eip7594/polynomial-commitments-sampling.md Co-authored-by: Justin Traglia <95511699+jtraglia@users.noreply.github.com> * Update specs/_features/eip7594/polynomial-commitments-sampling.md Co-authored-by: Justin Traglia <95511699+jtraglia@users.noreply.github.com> * chore: linter * chore: asn (switch to bls_modular_inverse) * chore: (ben) rename func to test_construct_vanishing_polynomial * chore: (ben) rename `extended_evaluations_coeffs` to `extended_evaluation_times_zero_coeffs` * chore: compute `roots_of_unity_extended` in recover_data method * chore: add more comments explaining whats happening in recover_data * chore: compute_zero_poly_coeff in recover_data * chore: make lint * chore: add doc comment to coset_fft_field * chore: (ben) add code to generate the vanishing polynomial when all cells are missing * chore: remove handling of edge case when constructing a vanishing polynomial * chore: rename H(x) to Q_3(x) * chore: remove trailing whitespace * chore: add whitespace between comments * chore: (asn) add assert that num missing cells is not 0 * chore: (justin) address comments * chore: merge resolution * chore: fixup remaining IDs -> indices * chore: use indice nomenclature in tests --------- Co-authored-by: Justin Traglia <95511699+jtraglia@users.noreply.github.com>
2024-06-11 17:26:19 +00:00
# Get the extended domain. This will be referred to as the FFT domain.
roots_of_unity_extended = compute_roots_of_unity(FIELD_ELEMENTS_PER_EXT_BLOB)
# Flatten the cells into evaluations.
# If a cell is missing, then its evaluation is zero.
extended_evaluation_rbo = [0] * FIELD_ELEMENTS_PER_EXT_BLOB
2024-06-11 12:41:52 +00:00
for cell_index, cell in zip(cell_indices, cells):
start = cell_index * FIELD_ELEMENTS_PER_CELL
end = (cell_index + 1) * FIELD_ELEMENTS_PER_CELL
extended_evaluation_rbo[start:end] = cell
extended_evaluation = bit_reversal_permutation(extended_evaluation_rbo)
chore: Refactor cell recovery code (#3781) * multi: - Remove shift_polynomial_coeff - Remove recover_shifted_data - Remove recover_original_data - Move `zero_poly_eval_brp ` under sanity check comment as its only used for sanity checking * chore: remove sanity check -- this was doing a wasteful `compute_root_of_unity` operation * chore: add previous sanity check as a unit test * chore: copy values python was taking a reference, so it passes in our regular codepaths but not in isolated test * chore: add coset_fft test * Update specs/_features/eip7594/polynomial-commitments-sampling.md Co-authored-by: Justin Traglia <95511699+jtraglia@users.noreply.github.com> * Update specs/_features/eip7594/polynomial-commitments-sampling.md Co-authored-by: Justin Traglia <95511699+jtraglia@users.noreply.github.com> * chore: linter * chore: asn (switch to bls_modular_inverse) * chore: (ben) rename func to test_construct_vanishing_polynomial * chore: (ben) rename `extended_evaluations_coeffs` to `extended_evaluation_times_zero_coeffs` * chore: compute `roots_of_unity_extended` in recover_data method * chore: add more comments explaining whats happening in recover_data * chore: compute_zero_poly_coeff in recover_data * chore: make lint * chore: add doc comment to coset_fft_field * chore: (ben) add code to generate the vanishing polynomial when all cells are missing * chore: remove handling of edge case when constructing a vanishing polynomial * chore: rename H(x) to Q_3(x) * chore: remove trailing whitespace * chore: add whitespace between comments * chore: (asn) add assert that num missing cells is not 0 * chore: (justin) address comments * chore: merge resolution * chore: fixup remaining IDs -> indices * chore: use indice nomenclature in tests --------- Co-authored-by: Justin Traglia <95511699+jtraglia@users.noreply.github.com>
2024-06-11 17:26:19 +00:00
# Compute Z(x) in monomial form
# Z(x) is the polynomial which vanishes on all of the evaluations which are missing
missing_cell_indices = [CellIndex(cell_index) for cell_index in range(CELLS_PER_EXT_BLOB)
if cell_index not in cell_indices]
zero_poly_coeff = construct_vanishing_polynomial(missing_cell_indices)
chore: Refactor cell recovery code (#3781) * multi: - Remove shift_polynomial_coeff - Remove recover_shifted_data - Remove recover_original_data - Move `zero_poly_eval_brp ` under sanity check comment as its only used for sanity checking * chore: remove sanity check -- this was doing a wasteful `compute_root_of_unity` operation * chore: add previous sanity check as a unit test * chore: copy values python was taking a reference, so it passes in our regular codepaths but not in isolated test * chore: add coset_fft test * Update specs/_features/eip7594/polynomial-commitments-sampling.md Co-authored-by: Justin Traglia <95511699+jtraglia@users.noreply.github.com> * Update specs/_features/eip7594/polynomial-commitments-sampling.md Co-authored-by: Justin Traglia <95511699+jtraglia@users.noreply.github.com> * chore: linter * chore: asn (switch to bls_modular_inverse) * chore: (ben) rename func to test_construct_vanishing_polynomial * chore: (ben) rename `extended_evaluations_coeffs` to `extended_evaluation_times_zero_coeffs` * chore: compute `roots_of_unity_extended` in recover_data method * chore: add more comments explaining whats happening in recover_data * chore: compute_zero_poly_coeff in recover_data * chore: make lint * chore: add doc comment to coset_fft_field * chore: (ben) add code to generate the vanishing polynomial when all cells are missing * chore: remove handling of edge case when constructing a vanishing polynomial * chore: rename H(x) to Q_3(x) * chore: remove trailing whitespace * chore: add whitespace between comments * chore: (asn) add assert that num missing cells is not 0 * chore: (justin) address comments * chore: merge resolution * chore: fixup remaining IDs -> indices * chore: use indice nomenclature in tests --------- Co-authored-by: Justin Traglia <95511699+jtraglia@users.noreply.github.com>
2024-06-11 17:26:19 +00:00
# Convert Z(x) to evaluation form over the FFT domain
zero_poly_eval = fft_field(zero_poly_coeff, roots_of_unity_extended)
chore: Refactor cell recovery code (#3781) * multi: - Remove shift_polynomial_coeff - Remove recover_shifted_data - Remove recover_original_data - Move `zero_poly_eval_brp ` under sanity check comment as its only used for sanity checking * chore: remove sanity check -- this was doing a wasteful `compute_root_of_unity` operation * chore: add previous sanity check as a unit test * chore: copy values python was taking a reference, so it passes in our regular codepaths but not in isolated test * chore: add coset_fft test * Update specs/_features/eip7594/polynomial-commitments-sampling.md Co-authored-by: Justin Traglia <95511699+jtraglia@users.noreply.github.com> * Update specs/_features/eip7594/polynomial-commitments-sampling.md Co-authored-by: Justin Traglia <95511699+jtraglia@users.noreply.github.com> * chore: linter * chore: asn (switch to bls_modular_inverse) * chore: (ben) rename func to test_construct_vanishing_polynomial * chore: (ben) rename `extended_evaluations_coeffs` to `extended_evaluation_times_zero_coeffs` * chore: compute `roots_of_unity_extended` in recover_data method * chore: add more comments explaining whats happening in recover_data * chore: compute_zero_poly_coeff in recover_data * chore: make lint * chore: add doc comment to coset_fft_field * chore: (ben) add code to generate the vanishing polynomial when all cells are missing * chore: remove handling of edge case when constructing a vanishing polynomial * chore: rename H(x) to Q_3(x) * chore: remove trailing whitespace * chore: add whitespace between comments * chore: (asn) add assert that num missing cells is not 0 * chore: (justin) address comments * chore: merge resolution * chore: fixup remaining IDs -> indices * chore: use indice nomenclature in tests --------- Co-authored-by: Justin Traglia <95511699+jtraglia@users.noreply.github.com>
2024-06-11 17:26:19 +00:00
# Compute (E*Z)(x) = E(x) * Z(x) in evaluation form over the FFT domain
extended_evaluation_times_zero = [BLSFieldElement(int(a) * int(b) % BLS_MODULUS)
for a, b in zip(zero_poly_eval, extended_evaluation)]
2023-12-12 16:25:30 +00:00
chore: Refactor cell recovery code (#3781) * multi: - Remove shift_polynomial_coeff - Remove recover_shifted_data - Remove recover_original_data - Move `zero_poly_eval_brp ` under sanity check comment as its only used for sanity checking * chore: remove sanity check -- this was doing a wasteful `compute_root_of_unity` operation * chore: add previous sanity check as a unit test * chore: copy values python was taking a reference, so it passes in our regular codepaths but not in isolated test * chore: add coset_fft test * Update specs/_features/eip7594/polynomial-commitments-sampling.md Co-authored-by: Justin Traglia <95511699+jtraglia@users.noreply.github.com> * Update specs/_features/eip7594/polynomial-commitments-sampling.md Co-authored-by: Justin Traglia <95511699+jtraglia@users.noreply.github.com> * chore: linter * chore: asn (switch to bls_modular_inverse) * chore: (ben) rename func to test_construct_vanishing_polynomial * chore: (ben) rename `extended_evaluations_coeffs` to `extended_evaluation_times_zero_coeffs` * chore: compute `roots_of_unity_extended` in recover_data method * chore: add more comments explaining whats happening in recover_data * chore: compute_zero_poly_coeff in recover_data * chore: make lint * chore: add doc comment to coset_fft_field * chore: (ben) add code to generate the vanishing polynomial when all cells are missing * chore: remove handling of edge case when constructing a vanishing polynomial * chore: rename H(x) to Q_3(x) * chore: remove trailing whitespace * chore: add whitespace between comments * chore: (asn) add assert that num missing cells is not 0 * chore: (justin) address comments * chore: merge resolution * chore: fixup remaining IDs -> indices * chore: use indice nomenclature in tests --------- Co-authored-by: Justin Traglia <95511699+jtraglia@users.noreply.github.com>
2024-06-11 17:26:19 +00:00
# Convert (E*Z)(x) to monomial form
extended_evaluation_times_zero_coeffs = fft_field(extended_evaluation_times_zero, roots_of_unity_extended, inv=True)
2024-01-15 15:34:28 +00:00
chore: Refactor cell recovery code (#3781) * multi: - Remove shift_polynomial_coeff - Remove recover_shifted_data - Remove recover_original_data - Move `zero_poly_eval_brp ` under sanity check comment as its only used for sanity checking * chore: remove sanity check -- this was doing a wasteful `compute_root_of_unity` operation * chore: add previous sanity check as a unit test * chore: copy values python was taking a reference, so it passes in our regular codepaths but not in isolated test * chore: add coset_fft test * Update specs/_features/eip7594/polynomial-commitments-sampling.md Co-authored-by: Justin Traglia <95511699+jtraglia@users.noreply.github.com> * Update specs/_features/eip7594/polynomial-commitments-sampling.md Co-authored-by: Justin Traglia <95511699+jtraglia@users.noreply.github.com> * chore: linter * chore: asn (switch to bls_modular_inverse) * chore: (ben) rename func to test_construct_vanishing_polynomial * chore: (ben) rename `extended_evaluations_coeffs` to `extended_evaluation_times_zero_coeffs` * chore: compute `roots_of_unity_extended` in recover_data method * chore: add more comments explaining whats happening in recover_data * chore: compute_zero_poly_coeff in recover_data * chore: make lint * chore: add doc comment to coset_fft_field * chore: (ben) add code to generate the vanishing polynomial when all cells are missing * chore: remove handling of edge case when constructing a vanishing polynomial * chore: rename H(x) to Q_3(x) * chore: remove trailing whitespace * chore: add whitespace between comments * chore: (asn) add assert that num missing cells is not 0 * chore: (justin) address comments * chore: merge resolution * chore: fixup remaining IDs -> indices * chore: use indice nomenclature in tests --------- Co-authored-by: Justin Traglia <95511699+jtraglia@users.noreply.github.com>
2024-06-11 17:26:19 +00:00
# Convert (E*Z)(x) to evaluation form over a coset of the FFT domain
extended_evaluations_over_coset = coset_fft_field(extended_evaluation_times_zero_coeffs, roots_of_unity_extended)
chore: Refactor cell recovery code (#3781) * multi: - Remove shift_polynomial_coeff - Remove recover_shifted_data - Remove recover_original_data - Move `zero_poly_eval_brp ` under sanity check comment as its only used for sanity checking * chore: remove sanity check -- this was doing a wasteful `compute_root_of_unity` operation * chore: add previous sanity check as a unit test * chore: copy values python was taking a reference, so it passes in our regular codepaths but not in isolated test * chore: add coset_fft test * Update specs/_features/eip7594/polynomial-commitments-sampling.md Co-authored-by: Justin Traglia <95511699+jtraglia@users.noreply.github.com> * Update specs/_features/eip7594/polynomial-commitments-sampling.md Co-authored-by: Justin Traglia <95511699+jtraglia@users.noreply.github.com> * chore: linter * chore: asn (switch to bls_modular_inverse) * chore: (ben) rename func to test_construct_vanishing_polynomial * chore: (ben) rename `extended_evaluations_coeffs` to `extended_evaluation_times_zero_coeffs` * chore: compute `roots_of_unity_extended` in recover_data method * chore: add more comments explaining whats happening in recover_data * chore: compute_zero_poly_coeff in recover_data * chore: make lint * chore: add doc comment to coset_fft_field * chore: (ben) add code to generate the vanishing polynomial when all cells are missing * chore: remove handling of edge case when constructing a vanishing polynomial * chore: rename H(x) to Q_3(x) * chore: remove trailing whitespace * chore: add whitespace between comments * chore: (asn) add assert that num missing cells is not 0 * chore: (justin) address comments * chore: merge resolution * chore: fixup remaining IDs -> indices * chore: use indice nomenclature in tests --------- Co-authored-by: Justin Traglia <95511699+jtraglia@users.noreply.github.com>
2024-06-11 17:26:19 +00:00
# Convert Z(x) to evaluation form over a coset of the FFT domain
zero_poly_over_coset = coset_fft_field(zero_poly_coeff, roots_of_unity_extended)
chore: Refactor cell recovery code (#3781) * multi: - Remove shift_polynomial_coeff - Remove recover_shifted_data - Remove recover_original_data - Move `zero_poly_eval_brp ` under sanity check comment as its only used for sanity checking * chore: remove sanity check -- this was doing a wasteful `compute_root_of_unity` operation * chore: add previous sanity check as a unit test * chore: copy values python was taking a reference, so it passes in our regular codepaths but not in isolated test * chore: add coset_fft test * Update specs/_features/eip7594/polynomial-commitments-sampling.md Co-authored-by: Justin Traglia <95511699+jtraglia@users.noreply.github.com> * Update specs/_features/eip7594/polynomial-commitments-sampling.md Co-authored-by: Justin Traglia <95511699+jtraglia@users.noreply.github.com> * chore: linter * chore: asn (switch to bls_modular_inverse) * chore: (ben) rename func to test_construct_vanishing_polynomial * chore: (ben) rename `extended_evaluations_coeffs` to `extended_evaluation_times_zero_coeffs` * chore: compute `roots_of_unity_extended` in recover_data method * chore: add more comments explaining whats happening in recover_data * chore: compute_zero_poly_coeff in recover_data * chore: make lint * chore: add doc comment to coset_fft_field * chore: (ben) add code to generate the vanishing polynomial when all cells are missing * chore: remove handling of edge case when constructing a vanishing polynomial * chore: rename H(x) to Q_3(x) * chore: remove trailing whitespace * chore: add whitespace between comments * chore: (asn) add assert that num missing cells is not 0 * chore: (justin) address comments * chore: merge resolution * chore: fixup remaining IDs -> indices * chore: use indice nomenclature in tests --------- Co-authored-by: Justin Traglia <95511699+jtraglia@users.noreply.github.com>
2024-06-11 17:26:19 +00:00
# Compute Q_3(x) = (E*Z)(x) / Z(x) in evaluation form over a coset of the FFT domain
reconstructed_poly_over_coset = [
2023-12-12 16:25:30 +00:00
div(a, b)
chore: Refactor cell recovery code (#3781) * multi: - Remove shift_polynomial_coeff - Remove recover_shifted_data - Remove recover_original_data - Move `zero_poly_eval_brp ` under sanity check comment as its only used for sanity checking * chore: remove sanity check -- this was doing a wasteful `compute_root_of_unity` operation * chore: add previous sanity check as a unit test * chore: copy values python was taking a reference, so it passes in our regular codepaths but not in isolated test * chore: add coset_fft test * Update specs/_features/eip7594/polynomial-commitments-sampling.md Co-authored-by: Justin Traglia <95511699+jtraglia@users.noreply.github.com> * Update specs/_features/eip7594/polynomial-commitments-sampling.md Co-authored-by: Justin Traglia <95511699+jtraglia@users.noreply.github.com> * chore: linter * chore: asn (switch to bls_modular_inverse) * chore: (ben) rename func to test_construct_vanishing_polynomial * chore: (ben) rename `extended_evaluations_coeffs` to `extended_evaluation_times_zero_coeffs` * chore: compute `roots_of_unity_extended` in recover_data method * chore: add more comments explaining whats happening in recover_data * chore: compute_zero_poly_coeff in recover_data * chore: make lint * chore: add doc comment to coset_fft_field * chore: (ben) add code to generate the vanishing polynomial when all cells are missing * chore: remove handling of edge case when constructing a vanishing polynomial * chore: rename H(x) to Q_3(x) * chore: remove trailing whitespace * chore: add whitespace between comments * chore: (asn) add assert that num missing cells is not 0 * chore: (justin) address comments * chore: merge resolution * chore: fixup remaining IDs -> indices * chore: use indice nomenclature in tests --------- Co-authored-by: Justin Traglia <95511699+jtraglia@users.noreply.github.com>
2024-06-11 17:26:19 +00:00
for a, b in zip(extended_evaluations_over_coset, zero_poly_over_coset)
2023-12-12 16:25:30 +00:00
]
chore: Refactor cell recovery code (#3781) * multi: - Remove shift_polynomial_coeff - Remove recover_shifted_data - Remove recover_original_data - Move `zero_poly_eval_brp ` under sanity check comment as its only used for sanity checking * chore: remove sanity check -- this was doing a wasteful `compute_root_of_unity` operation * chore: add previous sanity check as a unit test * chore: copy values python was taking a reference, so it passes in our regular codepaths but not in isolated test * chore: add coset_fft test * Update specs/_features/eip7594/polynomial-commitments-sampling.md Co-authored-by: Justin Traglia <95511699+jtraglia@users.noreply.github.com> * Update specs/_features/eip7594/polynomial-commitments-sampling.md Co-authored-by: Justin Traglia <95511699+jtraglia@users.noreply.github.com> * chore: linter * chore: asn (switch to bls_modular_inverse) * chore: (ben) rename func to test_construct_vanishing_polynomial * chore: (ben) rename `extended_evaluations_coeffs` to `extended_evaluation_times_zero_coeffs` * chore: compute `roots_of_unity_extended` in recover_data method * chore: add more comments explaining whats happening in recover_data * chore: compute_zero_poly_coeff in recover_data * chore: make lint * chore: add doc comment to coset_fft_field * chore: (ben) add code to generate the vanishing polynomial when all cells are missing * chore: remove handling of edge case when constructing a vanishing polynomial * chore: rename H(x) to Q_3(x) * chore: remove trailing whitespace * chore: add whitespace between comments * chore: (asn) add assert that num missing cells is not 0 * chore: (justin) address comments * chore: merge resolution * chore: fixup remaining IDs -> indices * chore: use indice nomenclature in tests --------- Co-authored-by: Justin Traglia <95511699+jtraglia@users.noreply.github.com>
2024-06-11 17:26:19 +00:00
# Convert Q_3(x) to monomial form
reconstructed_poly_coeff = coset_fft_field(reconstructed_poly_over_coset, roots_of_unity_extended, inv=True)
chore: Refactor cell recovery code (#3781) * multi: - Remove shift_polynomial_coeff - Remove recover_shifted_data - Remove recover_original_data - Move `zero_poly_eval_brp ` under sanity check comment as its only used for sanity checking * chore: remove sanity check -- this was doing a wasteful `compute_root_of_unity` operation * chore: add previous sanity check as a unit test * chore: copy values python was taking a reference, so it passes in our regular codepaths but not in isolated test * chore: add coset_fft test * Update specs/_features/eip7594/polynomial-commitments-sampling.md Co-authored-by: Justin Traglia <95511699+jtraglia@users.noreply.github.com> * Update specs/_features/eip7594/polynomial-commitments-sampling.md Co-authored-by: Justin Traglia <95511699+jtraglia@users.noreply.github.com> * chore: linter * chore: asn (switch to bls_modular_inverse) * chore: (ben) rename func to test_construct_vanishing_polynomial * chore: (ben) rename `extended_evaluations_coeffs` to `extended_evaluation_times_zero_coeffs` * chore: compute `roots_of_unity_extended` in recover_data method * chore: add more comments explaining whats happening in recover_data * chore: compute_zero_poly_coeff in recover_data * chore: make lint * chore: add doc comment to coset_fft_field * chore: (ben) add code to generate the vanishing polynomial when all cells are missing * chore: remove handling of edge case when constructing a vanishing polynomial * chore: rename H(x) to Q_3(x) * chore: remove trailing whitespace * chore: add whitespace between comments * chore: (asn) add assert that num missing cells is not 0 * chore: (justin) address comments * chore: merge resolution * chore: fixup remaining IDs -> indices * chore: use indice nomenclature in tests --------- Co-authored-by: Justin Traglia <95511699+jtraglia@users.noreply.github.com>
2024-06-11 17:26:19 +00:00
# Convert Q_3(x) to evaluation form over the FFT domain and bit reverse the result
reconstructed_data = bit_reversal_permutation(fft_field(reconstructed_poly_coeff, roots_of_unity_extended))
return reconstructed_data
```
### `recover_cells_and_kzg_proofs`
```python
2024-06-11 12:41:52 +00:00
def recover_cells_and_kzg_proofs(cell_indices: Sequence[CellIndex],
cells: Sequence[Cell],
proofs_bytes: Sequence[Bytes48]) -> Tuple[
Vector[Cell, CELLS_PER_EXT_BLOB],
Vector[KZGProof, CELLS_PER_EXT_BLOB]]:
"""
Given at least 50% of cells/proofs for a blob, recover all the cells/proofs.
This algorithm uses FFTs to recover cells faster than using Lagrange
implementation, as can be seen here:
https://ethresear.ch/t/reed-solomon-erasure-code-recovery-in-n-log-2-n-time-with-ffts/3039
A faster version thanks to Qi Zhou can be found here:
https://github.com/ethereum/research/blob/51b530a53bd4147d123ab3e390a9d08605c2cdb8/polynomial_reconstruction/polynomial_reconstruction_danksharding.py
Public method.
"""
2024-06-11 12:41:52 +00:00
assert len(cell_indices) == len(cells) == len(proofs_bytes)
# Check we have enough cells to be able to perform the reconstruction
2024-06-11 12:41:52 +00:00
assert CELLS_PER_EXT_BLOB / 2 <= len(cell_indices) <= CELLS_PER_EXT_BLOB
# Check for duplicates
2024-06-11 12:41:52 +00:00
assert len(cell_indices) == len(set(cell_indices))
# Check that the cell indices are within bounds
for cell_index in cell_indices:
assert cell_index < CELLS_PER_EXT_BLOB
# Check that each cell is the correct length
for cell in cells:
assert len(cell) == BYTES_PER_CELL
# Check that each proof is the correct length
for proof_bytes in proofs_bytes:
assert len(proof_bytes) == BYTES_PER_PROOF
# Convert cells to coset evals
cosets_evals = [cell_to_coset_evals(cell) for cell in cells]
chore: Refactor cell recovery code (#3781) * multi: - Remove shift_polynomial_coeff - Remove recover_shifted_data - Remove recover_original_data - Move `zero_poly_eval_brp ` under sanity check comment as its only used for sanity checking * chore: remove sanity check -- this was doing a wasteful `compute_root_of_unity` operation * chore: add previous sanity check as a unit test * chore: copy values python was taking a reference, so it passes in our regular codepaths but not in isolated test * chore: add coset_fft test * Update specs/_features/eip7594/polynomial-commitments-sampling.md Co-authored-by: Justin Traglia <95511699+jtraglia@users.noreply.github.com> * Update specs/_features/eip7594/polynomial-commitments-sampling.md Co-authored-by: Justin Traglia <95511699+jtraglia@users.noreply.github.com> * chore: linter * chore: asn (switch to bls_modular_inverse) * chore: (ben) rename func to test_construct_vanishing_polynomial * chore: (ben) rename `extended_evaluations_coeffs` to `extended_evaluation_times_zero_coeffs` * chore: compute `roots_of_unity_extended` in recover_data method * chore: add more comments explaining whats happening in recover_data * chore: compute_zero_poly_coeff in recover_data * chore: make lint * chore: add doc comment to coset_fft_field * chore: (ben) add code to generate the vanishing polynomial when all cells are missing * chore: remove handling of edge case when constructing a vanishing polynomial * chore: rename H(x) to Q_3(x) * chore: remove trailing whitespace * chore: add whitespace between comments * chore: (asn) add assert that num missing cells is not 0 * chore: (justin) address comments * chore: merge resolution * chore: fixup remaining IDs -> indices * chore: use indice nomenclature in tests --------- Co-authored-by: Justin Traglia <95511699+jtraglia@users.noreply.github.com>
2024-06-11 17:26:19 +00:00
reconstructed_data = recover_data(cell_indices, cosets_evals)
2024-06-11 12:41:52 +00:00
for cell_index, coset_evals in zip(cell_indices, cosets_evals):
start = cell_index * FIELD_ELEMENTS_PER_CELL
end = (cell_index + 1) * FIELD_ELEMENTS_PER_CELL
assert reconstructed_data[start:end] == coset_evals
recovered_cells = [
coset_evals_to_cell(reconstructed_data[i * FIELD_ELEMENTS_PER_CELL:(i + 1) * FIELD_ELEMENTS_PER_CELL])
2024-04-19 11:50:22 +00:00
for i in range(CELLS_PER_EXT_BLOB)]
polynomial_eval = reconstructed_data[:FIELD_ELEMENTS_PER_BLOB]
polynomial_coeff = polynomial_eval_to_coeff(polynomial_eval)
recovered_proofs = [None] * CELLS_PER_EXT_BLOB
2024-06-11 12:41:52 +00:00
for i, cell_index in enumerate(cell_indices):
recovered_proofs[cell_index] = bytes_to_kzg_proof(proofs_bytes[i])
for i in range(CELLS_PER_EXT_BLOB):
if recovered_proofs[i] is None:
2024-06-11 12:41:52 +00:00
coset = coset_for_cell(CellIndex(i))
proof, ys = compute_kzg_proof_multi_impl(polynomial_coeff, coset)
assert coset_evals_to_cell(ys) == recovered_cells[i]
recovered_proofs[i] = proof
return recovered_cells, recovered_proofs
2023-12-12 16:25:30 +00:00
```