Currently we use subnets as a unit of custody in the PeerDAS core
protocol because it doesn't make sense to partially custody only some
columns in the subnets and waste the bandwidth to download the columns
the node doesn't custody.
Since subnets correspond to GossipSub topics which are in a layer lower
than the core protocol, using subnets as a unit of custody makes the
core layer and the network layer too coupled to each other and leave no
room for the network layer flexibility.
This commit introduces "custody groups" which are used a unit of custody
instead of subnets.
The immediate benefit of the decoupling is that we can immediately
increase the number of subnets without affecting the expected number of
peers to cover all columns and affecting the network stability and
without touching the core protocol.
The reason we want to increase the number of subnets to match the number
of columns is that the columns will be propagated through the network
faster when they have their own subnets. Just like EIP-4844, each
blob has its own subnet because, if all the blobs are in a single subnet,
the blobs will be propagated more slowly.
Since we keep the number of custody groups the same as the previous
number of subnets (32), the expected number of peers you need to cover
all the columns is not changed. In fact, you need only NUMBER_OF_COLUMNS
and NUMBER_OF_CUSTODY_GROUPS to analyze the expected number, which
makes the core protocol completely decoupled from the network layer.
This also removes references to the "extended matrix" in favor of just "matrix"
which I think is better. It's not an extended matrix, it's a matrix of extended
blobs. Technically it's just a matrix of cells/proofs.
wip
Add `TARGET_NUMBER_OF_PEERS`
Add networking spec draft
fix
simplification
Rename `DoYouHave` to `GetCustodyStatus`
Add DataLineSidecar design
Apply suggestions from code review
Co-authored-by: dankrad <mail@dankradfeist.de>
Co-authored-by: danny <dannyjryan@gmail.com>
Revamp after reviews and discussion
Remove `CustodyStatus`
minor fix
Change`DataColumn` to `List[DataCell, MAX_BLOBS_PER_BLOCK]`
Move folder
Replace `DataColumnByRootAndIndex` with `DataColumnSidecarByRoot` message. Add extended data description
Remove `DataRow`
Apply suggestions from @jacobkaufmann code review
Co-authored-by: Jacob Kaufmann <jacobkaufmann18@gmail.com>
Represent matrix in `BLSFieldElement` form
Add `assert time >= store.time` to `on_tick`
Revert the spec. Only handle it in tests
Remove extra tick
cleanup leftover
Add randomized block cases
Specify RPC byRoot blocks-sidecars elegibility
fix typo
Update specs/phase0/p2p-interface.md
Co-authored-by: Mikhail Kalinin <noblesse.knight@gmail.com>
Update specs/deneb/p2p-interface.md
Co-authored-by: Mikhail Kalinin <noblesse.knight@gmail.com>
add failed on_block condition
rephrase
Update specs/phase0/p2p-interface.md
Co-authored-by: Mikhail Kalinin <noblesse.knight@gmail.com>
apply suggestion
Update specs/deneb/p2p-interface.md
Co-authored-by: danny <dannyjryan@gmail.com>
Update specs/deneb/p2p-interface.md
Co-authored-by: danny <dannyjryan@gmail.com>
remove the last consider
from on_block to state_transition
simplify and add a new rule
Update specs/phase0/p2p-interface.md
Co-authored-by: Mikhail Kalinin <noblesse.knight@gmail.com>
Update specs/deneb/p2p-interface.md
Co-authored-by: Mikhail Kalinin <noblesse.knight@gmail.com>
Update specs/deneb/p2p-interface.md
Co-authored-by: danny <dannyjryan@gmail.com>
remove gossip failure rules
Apply suggestions from code review
bump version to v1.4.0-beta.5
Move `blob_sidecar_{subnet_id}` to `Blob subnets` section
Misc minor fix
Add linter support
Add column subnet validation. Split `verify_column_sidecar` into two functions
Fix `get_data_column_sidecars` by using `compute_samples_and_proofs`
Apply suggestions from code review
Co-authored-by: danny <dannyjryan@gmail.com>
Do not assign row custody
Apply suggestions from code review
Co-authored-by: danny <dannyjryan@gmail.com>
Revamp reconstruction section
Use depth as the primary preset for inclusion proof. Fix `get_data_column_sidecars` and add tests for merkle proof
Change `SAMPLES_PER_SLOT` to 8 and add tests (requirement TBD)
Apply PR feedback from @ppopth and @jtraglia
Fix `get_data_column_sidecars`
Co-authored-by: Pop Chunhapanya <haxx.pop@gmail.com>
Apply suggestions from code review
Co-authored-by: Pop Chunhapanya <haxx.pop@gmail.com>
Apply suggestions from code review
Co-authored-by: fradamt <104826920+fradamt@users.noreply.github.com>
Co-authored-by: Jacob Kaufmann <jacobkaufmann18@gmail.com>
Fix `get_data_column_sidecars` and `get_custody_lines`
Apply suggestions from code review
Co-authored-by: Jacob Kaufmann <jacobkaufmann18@gmail.com>
Enhance tests
fix typo
Co-authored-by: fradamt <104826920+fradamt@users.noreply.github.com>
Remove `epoch` from `get_custody_lines`
fix
fix
* Moved configuration into network preset instead of constants.
Now that `MAX_CHUNK_SIZE` and `GOSSIP_MAX_SIZE` are in configuration, we no longer need separate constants to represent them in the spec when they change in Bellatrix.
I've changed the usage, and put the values into the presets, but I'm not sure if I've updated the descriptions in the best way...
This is following on from the work in #3375 where a number of constants got moved into configuration, so we no longer need these constants to be separately represented, they can simply be updated in presets.
* Update presets/minimal/bellatrix.yaml
Co-authored-by: Hsiao-Wei Wang <hsiaowei.eth@gmail.com>
* Update presets/mainnet/bellatrix.yaml
Co-authored-by: Hsiao-Wei Wang <hsiaowei.eth@gmail.com>
* Moved preset items into the correct section and updated TOC.
It looked like the items listed in configuration about the max size and chunk size were no longer needed since we're updating preset values now and the preset changes seem to only be listed in the changes at the top.
* review feedback
* hopefully correct this time! Moved the 2 fields from configs into presets completely as suggested.
* WIP - changing back to being in config and updating the phase 0 value... I think this should be close but want to see what's outstanding.
* fix intellij's formatting of table.
* more fixes
---------
Co-authored-by: Hsiao-Wei Wang <hsiaowei.eth@gmail.com>
