Commit Graph

32 Commits

Author SHA1 Message Date
gusto 97c653efe3
Docker compose for small libp2p node network (#364)
* Update rust build image version

* Docker compose file for testnet

* Wrap tcp into dns transport in order to resolve hostnames (#346)

* Docker compose for small libp2p node network

* Install etcdctl to node containers

* Register libp2p nodes on etcd

* Register IP address in KV store

* Interconnect libp2p nodes

* Use delimiter in cli and env variables

* Use docker compose initial env config

* Leave main Dockerfile as is, use new ones in testnet dir

* Remove etcd installation script

* run_mixnet.sh placeholder

* Use .env file for docker compose config

* Ignore local .env file

* Wrap sh envvars used in strings

* Remove mixnode placeholders

* Use default values for envconfig

* Update labels in Dockerfiles

* Sanitize scripts via shellcheck

* Export env for nomos node

* Updated to latest libp2p config

* Pass config to bootstrap node
2023-09-15 17:27:38 +03:00
Youngjoon Lee 8c4d1577df
Add mixnet usage docs (#408) 2023-09-15 17:36:28 +09:00
Giacomo Pasini 3121a41b60
Change default network backend to libp2p (#403)
We've been using it for a while and we don't plan to switch back
to Waku anytime soon.
2023-09-15 09:58:26 +02:00
Daniel Sanchez 04d07038dc
Remove unused fountain codes (#407)
* Remove unused fountain codes

* Refactor ProposalChunk to Proposal
2023-09-15 09:39:53 +02:00
Youngjoon Lee 8449c81d0f
Mixnet PoC base branch (#316)
* Add `mixnode` and `mixnet-client` crate (#302)

* Add `mixnode` binary (#317)

* Integrate mixnet with libp2p network backend (#318)

* Fix #312: proper delays (#321)

* proper delays

* add missing duration param

* tiny fix: compilation error caused by `rand` 0.8 -> 0.7

* use `get_available_port()` for mixnet integration tests (#333)

* add missing comments

* Overwatch mixnet node (#339)

* Add mixnet service and overwatch app

* remove #[tokio::main]

---------

Co-authored-by: Youngjoon Lee <taxihighway@gmail.com>

* fix tests for the overwatch mixnode (#342)

* fix panic when corner case happen in RandomDelayIter (#335)

* Use `log` service for `mixnode` bin (#341)

* Use `wire` for MixnetMessage in libp2p (#347)

* Prevent tmixnet tests from running forever (#363)

* Use random delay when sending msgs to mixnet (#362)

* fix a minor compilation error caused by the latest master

* Fix run output fd (#343)

* add a connection pool

* Exp backoff (#332)

* move mixnet listening into separate task

* add exponential retry for insufficient peers in libp2p

* fix logging

* Fix MutexGuard across await (#373)

* Fix MutexGuard across await

Holding a MutexGuard across an await point is not a good idea.
Removing that solves the issues we had with the mixnet test

* Make mixnode handle bodies coming from the same source concurrently (#372)

---------

Co-authored-by: Youngjoon Lee <taxihighway@gmail.com>

* Move wait at network startup (#338)

We now wait after the call to 'subscribe' to give the network
the time to register peers in the mesh before starting to
publish messages

* Remove unused functions from mixnet connpool (#374)

* Mixnet benchmark (#375)

* merge fixes

* add `connection_pool_size` field to `config.yaml`

* Simplify mixnet topology (#393)

* Simplify bytes and duration range ser/de (#394)

* optimize bytes serde and duration serde

---------

Co-authored-by: Al Liu <scygliu1@gmail.com>
Co-authored-by: Daniel Sanchez <sanchez.quiros.daniel@gmail.com>
Co-authored-by: Giacomo Pasini <Zeegomo@users.noreply.github.com>
2023-09-14 17:38:47 +09:00
gusto 8e0937beb7
Node envconfig (#382)
* Move config to a new location

* Log config override with env variables

* Network config override with env variables

* Http env vars

* Consensus env vars
2023-09-11 14:56:35 +03:00
Daniel Sanchez 57746f5e76
Block refactor (#368)
* Add sidecars to block

* Use cl and bl prefixes
2023-09-05 13:43:36 +02:00
gusto e9b752aa4d
Set node_key from cli args (#361) 2023-08-31 14:48:49 +03:00
Daniel Sanchez 26a3910566
Pipe addtx endpoint for libp2p (#345)
* Pipe add tx for both backends

* Cleanup imports
2023-08-31 09:13:35 +02:00
Giacomo Pasini 78c6566d8a
Add libp2p backend to nomos-node (#285)
* Add support for libp2p backend in integration tests

* Add support for libp2p in nomos-node

* change default to waku

* add mutually exclusive features warning

* disable default features to avoid unification

* disable default features

* remove leftover cargo build

* Make sure we are subscribed to libp2p topic at startup

* unify imports

* typo in ci config

* Sequential build and test steps for features

* Add RandomBeaconState to libp2p carnot variant

---------

Co-authored-by: Gusto <bacvinka@gmail.com>
2023-08-09 07:42:08 +02:00
Daniel Sanchez 4bdc3ed15a
Add update committees structure for overlay (#295)
* Move roundrobin to leadership module

* Use references in leader selection

* Add membership traits to overlay and create membership module

* Implement committee membership for random beacon state

* Update flat overlay

* Create updateable membership traits and impls

* Update tree overlay

* Update overlay on consensus service

* Update overlay on simulations nomos node

* Update types on tests and modules

* Use chacha for shuffling

* Change to mut slice instead of inner cloning

* Use fisher yates shuffle from scratch

* Stylish and clippy happy
2023-08-08 10:34:02 +02:00
Youngjoon Lee 2b9769b5b7
feat: add `nomos-libp2p` crate (for nomos-network backend) (#237)
* feat: add libp2p network backend skeleton

* use tokio runtime managed by Overwatch

* feat: add nomos-libp2p crate

* remove gossipsub_message_id_fn

* clippy

* use next() instead of select_next_some()

* rename send_command to execute_command

* const timeout

* disable authn / msg signing to start from a clean slate

* rename CommandSender to CommandResultSender

* add comments

* move node machinery to networkbackend

* fmt

* logs more network events

---------

Co-authored-by: Giacomo Pasini <g.pasini98@gmail.com>
2023-07-11 10:33:57 +02:00
Giacomo Pasini 09370dcef8
Integrations tests (#221)
* make config struct fields public

* add basic integration tests

* Add libssl-dev dependency

* fix comments

---------

Co-authored-by: Gusto <bacvinka@gmail.com>
2023-06-27 13:05:09 +02:00
Giacomo Pasini 8fad13b0cc
Separate nomos node into bin and lib sections (#209) 2023-06-22 16:58:47 +02:00
Alberto Soutullo 53a89c33a8
Fixed ident in initial_peers and added relayTopics that was missing (#213) 2023-06-21 14:16:45 +02:00
Giacomo Pasini 2fc10f94f8
Rename mockpool-node to nomos-node (#203) 2023-06-20 17:04:52 +02:00
Youngjoon Lee f02234f9d1
deps: bump waku-bindings to handle `ephemeral` field (#160) 2023-06-20 20:58:22 +09:00
Giacomo Pasini 2c23bda702
Carnot info api (#177)
* add missing accessors

* add api to return info from consensus service

* move processing into its own function

* feat: add HTTP GET `/carnot/info` API (#178)

---------

Co-authored-by: Youngjoon Lee <taxihighway@gmail.com>
2023-06-15 11:29:50 +02:00
Giacomo Pasini 9c81b72711
Random beacon (#167)
* move overlay to consensus engine

* Integrate random beacon in overlay

This commit integrates the random beacon as specified in the nomos
spec into the consensus engine library as part of the overlay.
In addition, it separates the overlay part responsible for leader
selection as an independent trait LeaderSelection, so as to share
the overall overlay structure among most constructions.

Furthermore, a leader proof has been added to a block to verify
that the proposer had valid rights to do so.

The current implementation hardcodes the leader selection update
in the consensus service, but we probably want to abstract it away
through something like the adapter pattern we use of other services
in the node.

* Move leader selection update to separate function

* Add generic support for leader selection in consensus service (#170)

* Add generic support for leader selection in consensus service

* fix

* use settings struct instead of tuple

* fix tests
2023-06-12 15:14:49 +02:00
Youngjoon Lee 2d60ce9921
fix: remove duplicate `tracing_subscriber` init for `nomos-node` bin (#157) 2023-05-29 22:40:22 +09:00
Daniel Sanchez bc453b686f
Consensus orchestrator (#128)
* Add happy-path consensus engine

* tmp

* Fit types from spec (#124)

* Match types to spec

* Remove Output import

* Consensus engine rework (#126)

* rework

* fix test

* clippy happy

---------

Co-authored-by: Giacomo Pasini <Zeegomo@users.noreply.github.com>

* Adapt carnot network adapter interfaces and implementations

* Fix errors

* Update network with engine types

* Fit types yet again

* Remove leadership and old overlay
Create carnot event builder
Added some adjustments

* Add view to vote

* Fix serde derive in consensus-engine

* Add serde feature for engine in core

* Use view in tally

* Move carnot tally to consensus service

* Add new view msg

* Fit engine types in adapter

* Missing serde feature in consensus service

* Implement carnot event builder

* Implement even builder run main tasks

* Fill up view resolver

* Fix errors on network adapter implementations

* Clippy happy

* Extract event handling to independent methods in View

* Fix test

* Refactor carnot event builder (#135)

* refactor

* format

* Discriminate proposal messages (#136)

* Derive block id from wire format (#139)

* Derive block id from wire format

* Derive id on block creation

* Use compile time hash size

* Add leader role (#138)

* add leadership stub

* fix gather_new_views

* fmt

* actually build qc

* remove redundant fields

* add flat overlay (#143)

* add flat overlay

* fix

* sort imports

* fix tests

* Unhappy tally (#137)

* Refactor tally module

* Implement tally for new view messages

* Assess pr comments

* Fix rebase

* simplify tally

---------

Co-authored-by: Giacomo Pasini <g.pasini98@gmail.com>

* Working node (#149)

* fix gather_new_views

* working node

* fix unhappy path

* remove leftover

* fix comments

* update waku (#146)

* update waku

* Fix waku update

---------

Co-authored-by: danielsanchezq <sanchez.quiros.daniel@gmail.com>

* little fixes

* Consensus tasks cancellation (#147)

* fix

* Create view cancel and cancel cache

* Attach cancellation to consensus tasks

* Fix view binds

---------

Co-authored-by: Giacomo Pasini <g.pasini98@gmail.com>

---------

Co-authored-by: danielsanchezq <sanchez.quiros.daniel@gmail.com>

* Remove clones on consts

---------

Co-authored-by: Al Liu <scygliu1@gmail.com>
Co-authored-by: Giacomo Pasini <g.pasini98@gmail.com>
Co-authored-by: Giacomo Pasini <Zeegomo@users.noreply.github.com>
2023-05-22 14:56:56 +02:00
Giacomo Pasini fdc22111d3
use nanosecond precision in waku timestamp (#152) 2023-05-18 15:19:36 +02:00
Giacomo Pasini bbd313f70a
update waku (#146)
* update waku

* Fix waku update

---------

Co-authored-by: danielsanchezq <sanchez.quiros.daniel@gmail.com>
2023-05-15 16:21:49 +02:00
Daniel Sanchez 91ce4e6fa1
Make a transaction trait (#98)
* Impl Transaction trait

* Impl Transaction for MockTransaction

* <ake carnot transaction a module

* Refactor consensus to use Transaction

* Fix tests

* Constrain Transaction::Hash

* Refactor redundant Carnot in CarnotTx
2023-03-16 22:42:56 -07:00
gusto 4981c724af
Http status codes (#88)
* Include status codes in http error responses

* Mockpool bridges error handling

* Last TX in milliseconds in mempool metrics

* u64 for last tx metrics
2023-03-06 15:19:27 +02:00
gusto 42ea8f9be3
Update waku-bindings to 0.1.0-rc.2 (#86) 2023-03-06 15:18:27 +02:00
Daniel Sanchez 55aece674b
Update to waku rc 1 (#80) 2023-02-22 15:55:59 +01:00
Daniel Sanchez 320755d19d
Detach tx broadcast from mempool (#69)
* Remove send tx method from mempool network adapter

* Add error reporting to add_tx operation in mempool
Delegate broadcasting to external caller
2023-02-08 11:07:09 +01:00
gusto 5f21a2734a
Add endpoint to make a new peer conn for mockpool node (#68)
* Add enpoint to make a new peer conn for mockpool node

* Pass multiple addresses in connection request

* Join async reqs to network service, cleanup

* Collect and join reqs
2023-02-07 17:50:38 +02:00
Daniel Sanchez 9a8400f4ce
Use blake 2b instead of 2s for mockpool node TxId (#62)
* Use blake 2b instead of 2s

* Clippy happy
2023-02-02 00:20:16 -08:00
Daniel Sanchez 567188c248
Fix blake2b usage on mockpool node tx (#61) 2023-02-01 04:10:57 -08:00
Daniel Sanchez dbe36bba3c
Mockpool node (#53)
* Create nodes folder
Kickstart mockpool node

* Create nodes folder
Added bridges file

* Added metrics to mempool

* Remove metrics from node

* Added mempool metrics bridge

* Pipe in mempool_metric bridge

* Add wakuinfo to waku network service

* Add waku network info bridge

* Added waku info bridge to node

* Use mock Tx wrapper over a string

* Create add tx http bridge

* Add tx bridge to http config

* Use hash for Tx

* Remove tracing subscriber from binary

* Fix bridges routes

* Added mimimal configuration example

* Remove subscribing to default waku pubsub topic

* Use addtx payload for tx

* Remove pub serde mod from core transaction

* Clippy happy

* Id from &Tx instead of owned value

* Removed mempool metrics feature
2023-01-25 07:24:33 -08:00