* Base cleaning of da to new traits/structure
Added new da protocols and types
* Implement base structure for verifier service
* Added comments and todo!
* Cleanup imports
* Size of VidCert in full replication
* Nomos Da Verifier service crate
* Extension replaced with metadata
* Storage adapter trait in verifier
* Manage lifecycle and messages in verifier
* Common nomos da storage crate
* Use updated nomos da storage in indexer
* Verifier storage adapter
* Libp2p adaper for verifier
* Kzgrs backend in verifier service
* Fix fmt
* Data availability tests module
* Return attestation in service msg response
* Common definitions for da tests
* Serde for kzgrs proofs and commitments
* Da verifier integration test
* WIP nomos-core in kzgrs backend
* Kzgrs blob to common module
* Add client zone to verifier test and check if attestations are created
* Cleanup and comments
* Use libp2p only for verifier and indexer service tests
* Lint in da tests
* Simplify da blob serialization
* Remove attester from nomos-core attestation
* Verifier backend error
---------
Co-authored-by: danielsanchezq <sanchez.quiros.daniel@gmail.com>
* Base cleaning of da to new traits/structure
Added new da protocols and types
* Implement base structure for verifier service
* Added comments and todo!
* Cleanup imports
* Size of VidCert in full replication
* Nomos Da Verifier service crate
* Extension replaced with metadata
* Fix DaIndexer service name
* Storage adapter trait in verifier
* Manage lifecycle and messages in verifier
* Blob trait in core
* Common nomos da storage crate
* Use updated nomos da storage in indexer
* Verifier storage adapter
* Libp2p adaper for verifier
* Kzgrs backend in verifier service
* Fix fmt
* Clippy happy
---------
Co-authored-by: Gusto <bacvinka@gmail.com>
* Rocksb adapter in da indexer
* Handle da service messages
* Remove indexer trait, use storage directly in the indexer service
* Return unordered indexes range
* Load blob by vid from file
* Use VID in consensus
* Change full replication index type to byte array
* Change payload to cert and item to vid where required
* Service integration tests for indexer
* Feature gate rocksdb backend
* Propagate range response send error
* FRIndex to Index
* VID to VidCertificate
* Pass blobs file dir via settings
* Base cleaning of da to new traits/structure
Added new da protocols and types
* Remove da availability crate completely
* Scaffold for da storage service
* Indexer service responsible for storage and blockchain subscription
* Handle index related ops only
* Acquire storage and consensus relays
* Indexer trait
* wip: storage adapter
* Use storage adapter instead of storage
* Add consensus adapter trait for block subscriptions
* Consensus block subscriber adapter
* Use metadata from da core in indexer
* Update nomos-services/data-availability/indexer/src/lib.rs
Co-authored-by: Daniel Sanchez <sanchez.quiros.daniel@gmail.com>
* Update nomos-services/data-availability/indexer/src/lib.rs
Co-authored-by: Daniel Sanchez <sanchez.quiros.daniel@gmail.com>
* Use std::ops::Range for da queries
* Return certificate metadata
* Da storage adapter methods
---------
Co-authored-by: danielsanchezq <sanchez.quiros.daniel@gmail.com>
* Base cleaning of da to new traits/structure
Added new da protocols and types
* DA: KZG+RS core (#632)
* Removed old kzg rs modules
* Added new kzgrs core module
* Implemented bytes_to_polynomial and tests
* Use coefficient form
* Refactor evaluations into method
* Use domain elements instead of roots of unity in tests
* Fix encoding and test
* Clippy happy
* Add comments
* Implement polynomial commitment
* Implement proof generation
* Sketch fn signature for verification
* implement proof verification
* Implemented verification and tests
* Return evaluations from bytes_to_polynomial as well
* Use modular le bytes
* Implement rs encode/decode
* Implement decoding tests
* Implement decode using lagrange
* Cleanup imports
* Reduce abstraction for certificate and vid metadata
* Allow payload to mempool as long as it converts into item
* Da Certificate verifier
* Add mock certificate for core tests
* Mempool payload verification
* Integrate mock verifiers for tx and certs
* Detach verification from cert and tx
* Seperate payload and item in mempools
* Mempools in integration tests
* Remove old cert verifier
* Network payload to item constraints in da mempool
* Update nomos-da/full-replication/src/lib.rs
Co-authored-by: Daniel Sanchez <sanchez.quiros.daniel@gmail.com>
* Sort attestations for cert signature
* Update nomos-da/full-replication/src/lib.rs
Co-authored-by: Daniel Sanchez <sanchez.quiros.daniel@gmail.com>
---------
Co-authored-by: danielsanchezq <sanchez.quiros.daniel@gmail.com>
* Removed old kzg rs modules
* Added new kzgrs core module
* Implemented bytes_to_polynomial and tests
* Use coefficient form
* Refactor evaluations into method
* Use domain elements instead of roots of unity in tests
* Fix encoding and test
* Clippy happy
* Add comments
* Implement polynomial commitment
* Implement proof generation
* Sketch fn signature for verification
* implement proof verification
* Implemented verification and tests
* Return evaluations from bytes_to_polynomial as well
* Use modular le bytes
* Implement rs encode/decode
* Implement decoding tests
* Implement decode using lagrange
* Cleanup imports
* Refactor Block/Header definition
Refactor block/header definition so that it's now responsibility
of the nomos-core crate. This removes definitions in ledger/consensus
crates since there's no need at that level to have an understanding
of the block format.
The new header format supports both carnot and cryptarchia.
Block.id is a necessary piece of information in the context
of the consensus engine since there it's not possible to recover
the id of the block since the contents are not available.
Instead, we should only skip that field when serializing/deserializing
a full block.
* Use selection for blob certificates
* Fix bin imports
* Fix rebase
* Missing blobs -> certificates refactor
* Fix attestation and certificate as_bytes
* More naming refactors
* Make the data availability service work with multiple protocols
* Add a generic way to instantiate DaProtocol
Add settings type and a new `new(Self::Settings)` method to
build a new DaProtocol instance
* Add data availability service to node
* fix tests
* fix imports
* Add `mixnode` and `mixnet-client` crate (#302)
* Add `mixnode` binary (#317)
* Integrate mixnet with libp2p network backend (#318)
* Fix#312: proper delays (#321)
* proper delays
* add missing duration param
* tiny fix: compilation error caused by `rand` 0.8 -> 0.7
* use `get_available_port()` for mixnet integration tests (#333)
* add missing comments
* Overwatch mixnet node (#339)
* Add mixnet service and overwatch app
* remove #[tokio::main]
---------
Co-authored-by: Youngjoon Lee <taxihighway@gmail.com>
* fix tests for the overwatch mixnode (#342)
* fix panic when corner case happen in RandomDelayIter (#335)
* Use `log` service for `mixnode` bin (#341)
* Use `wire` for MixnetMessage in libp2p (#347)
* Prevent tmixnet tests from running forever (#363)
* Use random delay when sending msgs to mixnet (#362)
* fix a minor compilation error caused by the latest master
* Fix run output fd (#343)
* add a connection pool
* Exp backoff (#332)
* move mixnet listening into separate task
* add exponential retry for insufficient peers in libp2p
* fix logging
* Fix MutexGuard across await (#373)
* Fix MutexGuard across await
Holding a MutexGuard across an await point is not a good idea.
Removing that solves the issues we had with the mixnet test
* Make mixnode handle bodies coming from the same source concurrently (#372)
---------
Co-authored-by: Youngjoon Lee <taxihighway@gmail.com>
* Move wait at network startup (#338)
We now wait after the call to 'subscribe' to give the network
the time to register peers in the mesh before starting to
publish messages
* Remove unused functions from mixnet connpool (#374)
* Mixnet benchmark (#375)
* merge fixes
* add `connection_pool_size` field to `config.yaml`
* Simplify mixnet topology (#393)
* Simplify bytes and duration range ser/de (#394)
* optimize bytes serde and duration serde
---------
Co-authored-by: Al Liu <scygliu1@gmail.com>
Co-authored-by: Daniel Sanchez <sanchez.quiros.daniel@gmail.com>
Co-authored-by: Giacomo Pasini <Zeegomo@users.noreply.github.com>
* Adopt a sans-IO style for DaProtocol
Remove async streams from the DaProtocol trait to avoid creating
dependencies on actual I/O. The protocol it's now transformed into
a state machine with an input and output buffer.
In addition, a method to create an attestation was added to the
trait.
* add forgotten &mut to certify_dispersal
* add forgotten &self param to validate certificate
* Add basic da module and traits
* Pipe new blobs and internal message handling
* Add and pipe send attestation method
* Add blob trait
* Make da backend async
* Implement mocka backend
* Bound blob in da backend to blob trait
* Added remove blob
* Rename reply to attestation
* move overlay to consensus engine
* Integrate random beacon in overlay
This commit integrates the random beacon as specified in the nomos
spec into the consensus engine library as part of the overlay.
In addition, it separates the overlay part responsible for leader
selection as an independent trait LeaderSelection, so as to share
the overall overlay structure among most constructions.
Furthermore, a leader proof has been added to a block to verify
that the proposer had valid rights to do so.
The current implementation hardcodes the leader selection update
in the consensus service, but we probably want to abstract it away
through something like the adapter pattern we use of other services
in the node.
* Move leader selection update to separate function
* Add generic support for leader selection in consensus service (#170)
* Add generic support for leader selection in consensus service
* fix
* use settings struct instead of tuple
* fix tests
* Added Qc type to Tally trait
* Add generic Qc to block header
* Use blockid instead of unnecessary header
* Expand Qc over generics
* Build up block with proper qc header
* Add basic encode/decode test
* Use Stream for trait instead of Iterator
* Removed unnecessary pin
* Add custom fountain error
* Add failing path to tests
* Added docs
* Normalized chunks to bytes
* Added settings initialization for fountain codes trait
* Pipe fountain code through consensus
* Implement broadcast block and block reconstruction for Member Overlay
* Fix failing raptor test
* Use seed for raptorq tests
* Use const for topics instead of static
* Clippy happy
* Refactor mempool naming and added settings to backend trait
* Implement mempool networking traits and waku backend
* Transaction networking plumbing in mempool service
* Make TransactionMsg generic
Use bincode to deserialize tx messages
* Make wakuadapter generic over tx and tx-id
* Fix wrong backend type bound
* Adapt to waku beta2
* Thread tx and id together for adapter and pool
* Panic on subscribing error
* Prune unnecessary Id type bounds
* Remove transaction placeholder
* Remove Id bound from waku adapter
* Remove empty transactions module