* Metrics layer configuration in nomos-tracing
* Add metrics layer to tracing service
* Use http/proto for prometheus
* Add metrics layer to integration tests
* Use metrics layer in cfgsync
* Plug metric in node and tests
* Remove old metrics
* Use otlp metrics in mempool
* tmp grafana stack
* Testnet configure otlp tracing
* tmp: Configurable loki and tempo in cfgsync
* Rename nomos-log to nomos-tracing-service
* nomos-tracing crate for tracing layer creation
* Use multiple layers in nomos-tracing-service
* Tracing in tests and testnet
* Tempo volume instead of local host dir
* Use executor in tests instead of nomos node
* Executor config from node config
* Bring generics to the testing game
* Fill in missing gaps in test
* Implement testnode wrapper
* Use sleep on dispersal service instead
* Fix cfgsync
* Clippy happy
* Clippy happy tests
* Mixnet config in tests for validator
* Tests: General config and multiple nodes (#832)
* Use executor in tests instead of nomos node
* Bring generics to the testing game
* Fill in missing gaps in test
* Clippy happy
* Mixnet config in tests for validator
* Derive different types of configs from general in tests
* Validator and executor in cfgsync
---------
Co-authored-by: danielSanchezQ <3danimanimal@gmail.com>
* Tests executor node mix config (#834)
* Merge branch 'master' into tests-executor-node-mix-config
* add mix configs
---------
Co-authored-by: danielSanchezQ <3danimanimal@gmail.com>
Co-authored-by: Youngjoon Lee <5462944+youngjoon-lee@users.noreply.github.com>
* Mix: remove all of the previous mixnet stuff
* Mix: Add mix service
* Integrate mix service to the node
* Add mock mix service for DA integration tests
* clippy happy
* use explicit function name and add TODOs
* remove the redundant use of tokio::sync::broadcast channel in mix adapter
* add mix service to nomos-executor binary
* removed mock mix backend and related adapters
Use the real mix backend for DA integration tests
* fix compile error generated when merging master
* rename all `MixNetwork*` to `Mix*` for further changes
* use explicit name for adapters
* Mix: Perform broadcasting within the mix service (#829)
* rename relay variables properly
* Implement adapter for executor
Refactored into common macro
* Executor and node indexer types
* Sampling service generic over network adapter
* Verifier changes tmp
---------
Co-authored-by: Gusto <bacvinka@gmail.com>
* Add macro to share adapter implementation for both validator and executor
* Fix imports in binaries
* Adapt cryptarchia types in binaries
* Fix tests build
* Added basic dispersal structure
* Expand impl
* Pull events from network service
* Implement network adapter
Tie types together in service
* Fill up service processing
* Tie up types in kzgrs backend implementation
* Implement disperse
* Pipe encode and dispersal
* Create mempool adapter
* Add mempool adapter
* Tiny comment
* Refactor common things in backends
* Further extract to common
* Extract dial_peers
* Pipe out executor events
* Add wrapper settings
* Dial up dispersal peers
* Clippy happy
* dial_peers -> dial_validator_subnetwork_peers
* Add members list to membership trait
* Implement peers selection and dialing up from specification
* Fix tests
* Fix tiny doc
* Handle replication messages
* Multiple indexes for kzgrs verifier
* Add rocksdb storage to node config
* Configurable delay before sending blob info to mempool
* Dial peers in same subnet for replication
* Update da integration tests
* Update nomos node tests
* Load kzgrs config from file
* SLOW_ENV set true for macos CI
* Assert retrieved blobs in dissemination test
* Kzgrs params deserialize unchecked
* Use common test node in da tests
* Pick ranged subnetids when sampling
* Da network settings for integration tests
* Predefined node keys and deterministic rng
* Disperse kzgrs encoded blob
* Cli swarm fixes
* Increase indexer integration test timeout
* Check dispersal responses in cli da adapter
* DA membership configuration in node tests
* Nomos Cli act as a node in tests
* Increase timeout for dispersal tests
* Different node configurations for da tests
* Collect unique ids for sampling
* Add PoL crate
* promote cl crate to nomos-node repo
* Integrate PoL into consensus
* remove metal feature
* install risc0 on ci
* more risc0 install
* fix da tests
* pass gh token
* fix tests and run in release mode
* clippy allow
* do not run in release mode
* drop windows in CI
* execute heavy tests nightly or accelerated in macos
* fmt
* configurable slot time
* use xlarge workers
* fix workflows
* remove unstable test
* use xlarge runners
* separate integrations tests
* fix features
* add token to ci
* downgrade runners
* fix integration tests
* revert testing changes and use risc0 dev mode
* fix coverage risc0 install
* increase slot time
* use cargo binstall
* clean before coverage
* first iteration, does not compile due to incompatible types
* Trigger sampling in node (#725)
* Sampling service in node
* Sampling in da integration tests
---------
Co-authored-by: gusto <bacv@users.noreply.github.com>
* Add sampling relay to consensus and massage all generics
* Pipe in sampling filtering of blob info
* Add mark in block
* Pipe validate block
* Refactor mark_in_block -> mark_complete
* Fix generics on tests
* Fix generics on tests
* Fix rebase
* Cargo fmt after rebase
* Sampling service configuration
* Sampling service config in indexer integration tests
---------
Co-authored-by: Gusto <bacvinka@gmail.com>
* initial
* first iteration, lots of lifetime and trait issues, does not compile
* Daniel/sampling service (#706)
* Implement sampling service
* Implement libp2p adapter listen_to_sampling_messages
* temporary empty backend mod; implement start_sampling
* addressed PR comments; moved random sampling subnet list generation to backend (todo impl)
* addressed PR nitpicks
* remove empty mod which was failing CI due to cargo fmt
* removed unused code
---------
Co-authored-by: holisticode <holistic.computing@gmail.com>
Co-authored-by: Daniel Sanchez <sanchez.quiros.daniel@gmail.com>
* Nomos cli executor swarm
* Executor swarm and backend in nomos-cli
* Libp2p executor adapter in nomos cli
* Update nomos-cli/src/da/network/backend.rs
Co-authored-by: Daniel Sanchez <sanchez.quiros.daniel@gmail.com>
* Use FillFromNodeList with executor backend
* Integrate Executor backend into the cli
* Use subnetwork id where appropriate
* Common clone_deserialize_error
* Use thiserror
* Minimal dialing in cli swarm
* Send dispersal messages async
* Remove unused connection id
---------
Co-authored-by: Daniel Sanchez <sanchez.quiros.daniel@gmail.com>
* ColumnIndex type in Blob trait
* Store blob id and column after verification
* Use blob_id+column_idx storage scheme in indexer
* Store and load multiple columns for the same blob
* Add comments
* WIP: cli dispersal reenable
* WIP: Nomos da network mock in cli app
* Mock adapter with KzgEncoder in cli
* Send BlobInfo to the mempool
* KzgrsSettings in dispersal cli app
* Dispersal backend mock for executor
* build_attestation_message to build_blob_id refactor
* Pass blob metadata via the cli args
* DispersedBlobData in nomos-core
* Implement BlobInfo in kzgrs-backend
* Fullreplication BlobInfo
* Use block info in da mempool
* Blob info in consensus
* Blob info in da-verifier
* BlobInfo in da indexer
* BlobInfo in nomos-api
* Verifier and Indexer integration tests with BlobInfo
* Blob info in nomos node
* Import DaEncoder trait
* test: add metadata getter test for VidCertificate
* fix: uncomment assertion for SeqCst
* fix: formatting
* test: get Metadata for Certificate
- generate signature for Certificate
* test: generate hash for Attestation
* fix: add assertion for generate hash
* fix: wrap in test module
- add helper functions
* fix: undo wrap in test module
* test: Hash implementation for Certificate with default Hasher
* Use fft for rs
* Specify evaluation domains
* Fix domains in verifier, decoder and dispersal
* Fix domains in verifier, decoder and dispersal
* Fix size in verifier to fit tests
* Base cleaning of da to new traits/structure
Added new da protocols and types
* DA: KZG+RS core (#632)
* Removed old kzg rs modules
* Added new kzgrs core module
* Implemented bytes_to_polynomial and tests
* Use coefficient form
* Refactor evaluations into method
* Use domain elements instead of roots of unity in tests
* Fix encoding and test
* Clippy happy
* Add comments
* Implement polynomial commitment
* Implement proof generation
* Sketch fn signature for verification
* implement proof verification
* Implemented verification and tests
* Return evaluations from bytes_to_polynomial as well
* Use modular le bytes
* Implement rs encode/decode
* Implement decoding tests
* Implement decode using lagrange
* Cleanup imports
* Da: v1 encoder (#633)
* Added new kzgrs core module
* Implemented bytes_to_polynomial and tests
* Use coefficient form
* Refactor evaluations into method
* Use domain elements instead of roots of unity in tests
* Fix encoding and test
* Clippy happy
* Add comments
* Implement polynomial commitment
* Implement proof generation
* Sketch fn signature for verification
* implement proof verification
* Implemented verification and tests
* Return evaluations from bytes_to_polynomial as well
* Use modular le bytes
* Implement rs encode/decode
* Implement decoding tests
* Implement decode using lagrange
* Added chunksmatrix
* Implement encoder with chunkify
* Added missing files
* Implement commit row commitments
* Implement compute elements (row) proofs
* Fix lib and types exposures
* Implement encoder main methods
* Implement encode method
* Implement chunkify test
Fix related bugs
* Implement compute row kzg commitments
Fix related bugs
* Implement rs encode rows test
Fix related bugs
Refactored API
* Implement row proofs tests
Fix fieldelement encoding/decoding bug
* Implement aggregated commitment test
Implement aggregated column proofs test
* Cleanup
* Fix deps
* Fix tests
* Fix chunk too big test
* Da: v1 verifier (#635)
* Fix encoding and test
* Implement commit row commitments
* Implemented dablob
* Implement verifier new
Implement verifier check column
* Clippy cleanup
* Implement verifier
* Implemented verify column test
* Implemented full verify test
* DA API Payload to Item in mempool (#634)
* Base cleaning of da to new traits/structure
Added new da protocols and types
* DA: KZG+RS core (#632)
* Removed old kzg rs modules
* Added new kzgrs core module
* Implemented bytes_to_polynomial and tests
* Use coefficient form
* Refactor evaluations into method
* Use domain elements instead of roots of unity in tests
* Fix encoding and test
* Clippy happy
* Add comments
* Implement polynomial commitment
* Implement proof generation
* Sketch fn signature for verification
* implement proof verification
* Implemented verification and tests
* Return evaluations from bytes_to_polynomial as well
* Use modular le bytes
* Implement rs encode/decode
* Implement decoding tests
* Implement decode using lagrange
* Cleanup imports
* Reduce abstraction for certificate and vid metadata
* Allow payload to mempool as long as it converts into item
* Da Certificate verifier
* Add mock certificate for core tests
* Mempool payload verification
* Integrate mock verifiers for tx and certs
* Detach verification from cert and tx
* Seperate payload and item in mempools
* Mempools in integration tests
* Remove old cert verifier
* Network payload to item constraints in da mempool
* Update nomos-da/full-replication/src/lib.rs
Co-authored-by: Daniel Sanchez <sanchez.quiros.daniel@gmail.com>
* Sort attestations for cert signature
* Update nomos-da/full-replication/src/lib.rs
Co-authored-by: Daniel Sanchez <sanchez.quiros.daniel@gmail.com>
---------
Co-authored-by: danielsanchezq <sanchez.quiros.daniel@gmail.com>
* DA API Certificate verification (#641)
* Redo certificate verification in mempool
* FullReplication verifier params provider
* Integrate da params provider into the node
* DA API Indexer service (#643)
* Base cleaning of da to new traits/structure
Added new da protocols and types
* Remove da availability crate completely
* Scaffold for da storage service
* Indexer service responsible for storage and blockchain subscription
* Handle index related ops only
* Acquire storage and consensus relays
* Indexer trait
* wip: storage adapter
* Use storage adapter instead of storage
* Add consensus adapter trait for block subscriptions
* Consensus block subscriber adapter
* Use metadata from da core in indexer
* Update nomos-services/data-availability/indexer/src/lib.rs
Co-authored-by: Daniel Sanchez <sanchez.quiros.daniel@gmail.com>
* Update nomos-services/data-availability/indexer/src/lib.rs
Co-authored-by: Daniel Sanchez <sanchez.quiros.daniel@gmail.com>
* Use std::ops::Range for da queries
* Return certificate metadata
* Da storage adapter methods
---------
Co-authored-by: danielsanchezq <sanchez.quiros.daniel@gmail.com>
* Reuse evaluations when computing proofs (#647)
* Reuse precomputed evaluations instead of evaluation polynomial for each proof
* kzgrs benchmarks
* Clippy happy
* DA API indexer implementation (#644)
* Rocksb adapter in da indexer
* Handle da service messages
* Remove indexer trait, use storage directly in the indexer service
* Return unordered indexes range
* Load blob by vid from file
* Use VID in consensus
* Change full replication index type to byte array
* Change payload to cert and item to vid where required
* Service integration tests for indexer
* Feature gate rocksdb backend
* Propagate range response send error
* FRIndex to Index
* VID to VidCertificate
* Pass blobs file dir via settings
* Da v1 multiple proofs bench (#648)
* Parallel proof generation bench
* Added bench comment
* Modify domain to fit exact sizes
* Fix domain in benches
* Force parallelization features in lib
* DA: Implement base structure for verifier service (#627)
* Base cleaning of da to new traits/structure
Added new da protocols and types
* Implement base structure for verifier service
* Added comments and todo!
* Cleanup imports
* Size of VidCert in full replication
* Nomos Da Verifier service crate
* Extension replaced with metadata
* Fix DaIndexer service name
* Storage adapter trait in verifier
* Manage lifecycle and messages in verifier
* Blob trait in core
* Common nomos da storage crate
* Use updated nomos da storage in indexer
* Verifier storage adapter
* Libp2p adaper for verifier
* Kzgrs backend in verifier service
* Fix fmt
* Clippy happy
---------
Co-authored-by: Gusto <bacvinka@gmail.com>
* DA Verifier service integration tests (#650)
* Base cleaning of da to new traits/structure
Added new da protocols and types
* Implement base structure for verifier service
* Added comments and todo!
* Cleanup imports
* Size of VidCert in full replication
* Nomos Da Verifier service crate
* Extension replaced with metadata
* Storage adapter trait in verifier
* Manage lifecycle and messages in verifier
* Common nomos da storage crate
* Use updated nomos da storage in indexer
* Verifier storage adapter
* Libp2p adaper for verifier
* Kzgrs backend in verifier service
* Fix fmt
* Data availability tests module
* Return attestation in service msg response
* Common definitions for da tests
* Serde for kzgrs proofs and commitments
* Da verifier integration test
* WIP nomos-core in kzgrs backend
* Kzgrs blob to common module
* Add client zone to verifier test and check if attestations are created
* Cleanup and comments
* Use libp2p only for verifier and indexer service tests
* Lint in da tests
* Simplify da blob serialization
* Remove attester from nomos-core attestation
* Verifier backend error
---------
Co-authored-by: danielsanchezq <sanchez.quiros.daniel@gmail.com>
* DA Kzgrs Backend Certificate implementation (#651)
* Kzgrs backend certificate definition
* Encoded data to certificate test
* Nomos da domain specific tag
* Handle errors in da certificate creation
* Add nomos core traits to da cert
* Derive ordering traits for da index
* Add failure test cases to kzgrs certificate
* Da v1 benches expand (#658)
* Update benches with more cases
* Expand benches
* Added parallel feature
* Fix test comment
Co-authored-by: Youngjoon Lee <5462944+youngjoon-lee@users.noreply.github.com>
* Remove outdated comment
---------
Co-authored-by: Gusto <bacvinka@gmail.com>
Co-authored-by: gusto <bacv@users.noreply.github.com>
Co-authored-by: Youngjoon Lee <5462944+youngjoon-lee@users.noreply.github.com>
Youngjoon Lee
gusto
Daniel Sanchez
Giacomo Pasini
holisticode
Roman Zajic