dynamic-data-experiments/README.md

Dynamic Data Experiments

This is a prototype implementation of the proposed Codex storage proofs for dynamic data.

Erasure Coding & Commitment

• Organize data as byte Matrix with k rows and m columns
• Convert the byte Matrix to Field Matrix with k rows and m columns
• Erasure code the columns -> end up with n*m Matrix
• Commit to each row independently with KZG

Note: in the above I switched the directions of the encoding and commitment (opposite of the proposal) just because it was easier to implement but basically it is same thing.

Sampling

• Select a set of rows randomly
• Generate a KZG evaluation proof at random point for each selected row

Updating the Data

• Select a column (or multiple)
• Query the original column
• Update the cells in that column
• Erasure code the updated column

Updating the Commitments

• Query the old column and receive the new column
• Iterate over all row commitments
• Commit to each old cell c_i and new cell c'_i in each row i:
• Compute delta_i = c'_i - c_i
• Compute the new row commitment row_comm_i' = row_comm_i + delta_i

Prove Data & Commitment Update

• TODO...

Additional functionalities

• BLS encoder: erasure coding over Bls12_381

TODO:

• implement matrix with "fat" cell and let encoding and commitment work over such matrix.
• fix conversion between byte to field matrix.
• Aggregate the KZG proofs.
• Build a Merkle tree with the KZG commitments.
• Simulate interactions between Client (Data Owner) and SP (Storage Provider).
• Clean up and optimize.
• Add details and write-up & experimentation/benchmark results.

WARNING: This repository contains work-in-progress prototypes, and has not received careful code review. It is NOT ready for production use.