* Use more descriptive names for generic parameters
We're starting to have tens on generic parameters, if we don't use
descriptive names it'll be pretty hard to understand what they do.
This commit changes the names of the mempool parameters in
preparation for the insertion of the da certificates mempool to
distinguish it from cl transactions.
* Add mempool for da certificates
* Add separate certificates mempool to binary
* ignore clippy lints
* Add `send` method to mempool network adapter
Centralize responsabilities for mempool-network interface in the
adapter trait.
* Update nomos-services/mempool/src/lib.rs
Co-authored-by: Youngjoon Lee <taxihighway@gmail.com>
---------
Co-authored-by: Youngjoon Lee <taxihighway@gmail.com>
Overwatch requires all services to have a different service id.
Unfortunately, such service id can't depend on generic parameters,
which means that we can't have two instances of the mempool
service even if they are instantiated with different types.
This commit circuments this limitation by adding another
type parameter.
* Make mempool item generic
Make the mempool generic with respect to the item and remove
mentions of specific transaction formats/traits. This will allow
us to reuse the same code for both coordination layer transactions
and certificates, or in general, whatever items need to be included
in a block.
* Add mempool network adapter settings
Allow for greater customization of the mempool network adapter by
adding a settings field.
* update node after mempool changes
* fix waku mempool adapter
* fmt
* fix tests
* fmt
* Use selection for blob certificates
* Fix bin imports
* Fix rebase
* Missing blobs -> certificates refactor
* Fix attestation and certificate as_bytes
* More naming refactors
* Make nomos-cli config public
Split nomos-cli into lib and bin and make config public
so that it can be reused in tests
* Add tests for nomos-cli disseminate
* fmt
* feature gate whole file instead of function
* Exit unsuccessfully when failing dissemination
* Add nomos-cli command to disseminate data across the network
Add nomos-cli to hold various utilities for the node.
To start, this commit adds a command to disseminate some
data through the network and build a certificate of correct
dispersal for inclusion in a block
* fmt
* reorder loop instructions
* Send blobs concurrently
* Make the data availability service work with multiple protocols
* Add a generic way to instantiate DaProtocol
Add settings type and a new `new(Self::Settings)` method to
build a new DaProtocol instance
* Add data availability service to node
* fix tests
* fix imports
* Update rust build image version
* Docker compose file for testnet
* Wrap tcp into dns transport in order to resolve hostnames (#346)
* Docker compose for small libp2p node network
* Install etcdctl to node containers
* Register libp2p nodes on etcd
* Register IP address in KV store
* Interconnect libp2p nodes
* Use delimiter in cli and env variables
* Use docker compose initial env config
* Leave main Dockerfile as is, use new ones in testnet dir
* Remove etcd installation script
* run_mixnet.sh placeholder
* Use .env file for docker compose config
* Ignore local .env file
* Wrap sh envvars used in strings
* Remove mixnode placeholders
* Use default values for envconfig
* Update labels in Dockerfiles
* Sanitize scripts via shellcheck
* Export env for nomos node
* Updated to latest libp2p config
* Pass config to bootstrap node
* Add `mixnode` and `mixnet-client` crate (#302)
* Add `mixnode` binary (#317)
* Integrate mixnet with libp2p network backend (#318)
* Fix#312: proper delays (#321)
* proper delays
* add missing duration param
* tiny fix: compilation error caused by `rand` 0.8 -> 0.7
* use `get_available_port()` for mixnet integration tests (#333)
* add missing comments
* Overwatch mixnet node (#339)
* Add mixnet service and overwatch app
* remove #[tokio::main]
---------
Co-authored-by: Youngjoon Lee <taxihighway@gmail.com>
* fix tests for the overwatch mixnode (#342)
* fix panic when corner case happen in RandomDelayIter (#335)
* Use `log` service for `mixnode` bin (#341)
* Use `wire` for MixnetMessage in libp2p (#347)
* Prevent tmixnet tests from running forever (#363)
* Use random delay when sending msgs to mixnet (#362)
* fix a minor compilation error caused by the latest master
* Fix run output fd (#343)
* add a connection pool
* Exp backoff (#332)
* move mixnet listening into separate task
* add exponential retry for insufficient peers in libp2p
* fix logging
* Fix MutexGuard across await (#373)
* Fix MutexGuard across await
Holding a MutexGuard across an await point is not a good idea.
Removing that solves the issues we had with the mixnet test
* Make mixnode handle bodies coming from the same source concurrently (#372)
---------
Co-authored-by: Youngjoon Lee <taxihighway@gmail.com>
* Move wait at network startup (#338)
We now wait after the call to 'subscribe' to give the network
the time to register peers in the mesh before starting to
publish messages
* Remove unused functions from mixnet connpool (#374)
* Mixnet benchmark (#375)
* merge fixes
* add `connection_pool_size` field to `config.yaml`
* Simplify mixnet topology (#393)
* Simplify bytes and duration range ser/de (#394)
* optimize bytes serde and duration serde
---------
Co-authored-by: Al Liu <scygliu1@gmail.com>
Co-authored-by: Daniel Sanchez <sanchez.quiros.daniel@gmail.com>
Co-authored-by: Giacomo Pasini <Zeegomo@users.noreply.github.com>
Firstly, a failure in deserialization for a network message is not
an error especially since we're using a public channel.
Secondly, that same channel is shared by different kind of messages
so trying to interpret one as the other will surely lead to a
unsuccessfull attempt.
Add an initial simple but functional implementation for a data
availability protocol.
Full replication simply encodes bytes in a single blob which is
replicated in all nodes.
* Adopt a sans-IO style for DaProtocol
Remove async streams from the DaProtocol trait to avoid creating
dependencies on actual I/O. The protocol it's now transformed into
a state machine with an input and output buffer.
In addition, a method to create an attestation was added to the
trait.
* add forgotten &mut to certify_dispersal
* add forgotten &self param to validate certificate
* Move config to a new location
* Log config override with env variables
* Network config override with env variables
* Http env vars
* Consensus env vars
* Remove explicit target
* Use default profile for windows build
* Use stable-gnu for tests
* Make polars optional feature in simulations
* Remove unnecessary ci debug steps
* Add basic da module and traits
* Pipe new blobs and internal message handling
* Add and pipe send attestation method
* Add blob trait
* Make da backend async
* Implement mocka backend
* Bound blob in da backend to blob trait
* Added remove blob
* Rename reply to attestation
Giacomo Pasini
Al Liu
Giacomo Pasini
Youngjoon Lee
Daniel Sanchez
gusto