Commit Graph

93 Commits

Author SHA1 Message Date
Daniel Sanchez bc453b686f
Consensus orchestrator (#128)
* Add happy-path consensus engine

* tmp

* Fit types from spec (#124)

* Match types to spec

* Remove Output import

* Consensus engine rework (#126)

* rework

* fix test

* clippy happy

---------

Co-authored-by: Giacomo Pasini <Zeegomo@users.noreply.github.com>

* Adapt carnot network adapter interfaces and implementations

* Fix errors

* Update network with engine types

* Fit types yet again

* Remove leadership and old overlay
Create carnot event builder
Added some adjustments

* Add view to vote

* Fix serde derive in consensus-engine

* Add serde feature for engine in core

* Use view in tally

* Move carnot tally to consensus service

* Add new view msg

* Fit engine types in adapter

* Missing serde feature in consensus service

* Implement carnot event builder

* Implement even builder run main tasks

* Fill up view resolver

* Fix errors on network adapter implementations

* Clippy happy

* Extract event handling to independent methods in View

* Fix test

* Refactor carnot event builder (#135)

* refactor

* format

* Discriminate proposal messages (#136)

* Derive block id from wire format (#139)

* Derive block id from wire format

* Derive id on block creation

* Use compile time hash size

* Add leader role (#138)

* add leadership stub

* fix gather_new_views

* fmt

* actually build qc

* remove redundant fields

* add flat overlay (#143)

* add flat overlay

* fix

* sort imports

* fix tests

* Unhappy tally (#137)

* Refactor tally module

* Implement tally for new view messages

* Assess pr comments

* Fix rebase

* simplify tally

---------

Co-authored-by: Giacomo Pasini <g.pasini98@gmail.com>

* Working node (#149)

* fix gather_new_views

* working node

* fix unhappy path

* remove leftover

* fix comments

* update waku (#146)

* update waku

* Fix waku update

---------

Co-authored-by: danielsanchezq <sanchez.quiros.daniel@gmail.com>

* little fixes

* Consensus tasks cancellation (#147)

* fix

* Create view cancel and cancel cache

* Attach cancellation to consensus tasks

* Fix view binds

---------

Co-authored-by: Giacomo Pasini <g.pasini98@gmail.com>

---------

Co-authored-by: danielsanchezq <sanchez.quiros.daniel@gmail.com>

* Remove clones on consts

---------

Co-authored-by: Al Liu <scygliu1@gmail.com>
Co-authored-by: Giacomo Pasini <g.pasini98@gmail.com>
Co-authored-by: Giacomo Pasini <Zeegomo@users.noreply.github.com>
2023-05-22 14:56:56 +02:00
Giacomo Pasini fdc22111d3
use nanosecond precision in waku timestamp (#152) 2023-05-18 15:19:36 +02:00
Giacomo Pasini bbd313f70a
update waku (#146)
* update waku

* Fix waku update

---------

Co-authored-by: danielsanchezq <sanchez.quiros.daniel@gmail.com>
2023-05-15 16:21:49 +02:00
Al Liu 4b880778ab
finish subscriber manager (#140)
* finish subscriber manager
2023-05-11 16:09:53 +08:00
Giacomo Pasini eb6f36782d
fix link flgs (#144) 2023-05-08 17:28:10 +02:00
gusto 0bf6e6d272
Simulation initialization (#122)
* Use enums for different settings types

* Enum for overlay settings

* Configurable simulation overlay

* Use duration type for network behaviour delays

* Configurable simulation nodes

* Runner for different node types

* Seedable rng

* Convert settings to required objects

* Implement IOStreamSettings deserialization

* Use common run method for different node types

* Configuration for simapp

* Testcase for region distribution

* Use unix time if seed is not provided
2023-05-08 13:06:39 +03:00
Daniel Sanchez 26d10856ff
Adapt carnot network adapter interfaces and implementations to latest spec (#121)
* Add happy-path consensus engine

* tmp

* Fit types from spec (#124)

* Match types to spec

* Remove Output import

* Consensus engine rework (#126)

* rework

* fix test

* clippy happy

---------

Co-authored-by: Giacomo Pasini <Zeegomo@users.noreply.github.com>

* Adapt carnot network adapter interfaces and implementations

* Fix errors

* Update network with engine types

* Fit types yet again

---------

Co-authored-by: Al Liu <scygliu1@gmail.com>
Co-authored-by: Giacomo Pasini <g.pasini98@gmail.com>
Co-authored-by: Giacomo Pasini <Zeegomo@users.noreply.github.com>
2023-05-01 17:47:56 +02:00
Giacomo Pasini f8617d7331
Consensus engine rework (#127)
---------

Co-authored-by: Giacomo Pasini <Zeegomo@users.noreply.github.com>

---------

Co-authored-by: Al Liu <scygliu1@gmail.com>
Co-authored-by: Daniel Sanchez <sanchez.quiros.daniel@gmail.com>
2023-04-27 19:18:24 +02:00
Al Liu ea7896f06c
Simulation streaming and gracefully shutdown (#119)
* add stream supports

* add test case

* add polars stream, and force to use stream for the runner

* using arcswap instead RefCell for producers

* finish gracefully shutdown

* - add IOProducer and IOSubscriber
- fix deadlock in sync runner
- fix testcases
2023-04-25 19:14:30 +08:00
Álvaro Castro-Castilla b1381d727f
Update README.md (#120) 2023-04-24 11:05:00 +02:00
Al Liu c2d1ee1291
add rustfmt.toml (#118) 2023-04-19 15:59:51 +08:00
gusto 2206d7a291
Use par_iter in network module (#116)
* Use par_iter in network module

* Pass rng from filter function

* Remove locking for network messages
2023-04-13 17:24:27 +03:00
gusto a87418c908
Test voting steps with various node roles (#114)
* Single step send receive runner tests

* Rename network to overlay state

* Store every overlay that might happen

* Get related nodes for from node id

* Add leader role

* WIP dummy handle events

* Leader and root roles

* Add intent as helper to dummy vote

* Check if has enough votes

* Proposal to vote to new view tests

* Update sync runner tests

* Smallrng in send receive tests

* Send initial vote next view leaders in tests

* Local view struct for node

* Large node number tests

* Fix internal to internal case
2023-04-10 13:18:20 +03:00
Al Liu a19278480d
Pipe output data (#111) 2023-04-04 20:26:57 +08:00
Al Liu d07da7ba5a
add missing ward (#112) 2023-04-03 17:17:14 +08:00
gusto 901ebf4152
Message sending between simulation nodes (#110)
* Dummy node for simulations

* Shared network state for nodes

* Runner one step test

* Beginning of network interface

* Connect dummy node to network

* Network step tests

* Pop messages that are being sent

* Regions send receive tests

* Setup network in sync runner tests

* Dispatch and collect node messages during sim step

* Improve network interface receiver
2023-03-31 12:48:06 +03:00
Al Liu c882a58286
Add StalledViewWard (#109) 2023-03-30 16:35:29 +08:00
Al Liu 61356f1116
Add minmax warding (#108)
* add minmax warding
2023-03-28 00:20:01 +08:00
Al Liu f4b94c8267
Simple warding fix (#107)
* fix negated analyze
2023-03-27 19:16:48 +08:00
Al Liu ae16e8583b
impl ward condition for max view ward (#106)
* impl ward condition for max view ward
2023-03-27 18:45:50 +08:00
Daniel Sanchez 92ef9e5a77
Simulation app runners (#105)
* Integrate new runners, fit types...make it compile

* Added missing runners

* Fix rebased changes

* Make tests pass

* Clippy happy

* More clippy happy
2023-03-24 13:21:10 -07:00
Al Liu 4a90ba6926
chore: types wrapper (#104)
* chore: types wrapper
2023-03-24 19:50:30 +08:00
Al Liu ea589d018a
Simulation App: Make step cost configurable (#102)
* WIP: make step cost configurable

* fix fmt
2023-03-23 16:57:18 +08:00
gusto 2d8c9f1099
Tree overlay for simulation app (#101)
* Add simulations crate

* Firs working runner

* Cleanupt

* Extract overlay from runner

* Fix off id calls

* Fix testing values

* WIP: Tree overlay for simulation app

* Build tree overlay from depth and size

* Tree overlay generation tests

* WIP: Committee role in layout

* Use commtittee role when constructing node

* Pairprogramming role calculation

* WIP: Extract step runner to node

* Aggregate times from different overlay layers

* Runner for overlay with layers

* Use StepRng in runner tests

* Decouple role from node

* Define leader steps and behaviour

* Get possible node ids from the overlay

* Handle unknown state in flat overlay

* Use solver type for deserialization

* Remove refcell from nodes hashmap

* Create layout from the provided node ids

* Break runner loops into seperate functions

* Use for loop to collect node step times

* Revert leader times collection

* Add interegion latency tests

---------

Co-authored-by: danielsanchezq <sanchez.quiros.daniel@gmail.com>
2023-03-23 01:32:54 -07:00
Al Liu 8a1af8c234
Add configuration for simulation app (#94)
* make simulation app compile to wasm

* add configuration

* improve args parsing

* add steps in configuration

---------

Co-authored-by: Daniel Sanchez Quiros <sanchez.quiros.daniel@gmail.com>
2023-03-20 17:13:55 +08:00
Daniel Sanchez 1ec4231a7a
Add Qc to block (#99)
* Added Qc type to Tally trait

* Add generic Qc to block header

* Use blockid instead of unnecessary header

* Expand Qc over generics

* Build up block with proper qc header
2023-03-17 06:23:50 -07:00
Daniel Sanchez 91ce4e6fa1
Make a transaction trait (#98)
* Impl Transaction trait

* Impl Transaction for MockTransaction

* <ake carnot transaction a module

* Refactor consensus to use Transaction

* Fix tests

* Constrain Transaction::Hash

* Refactor redundant Carnot in CarnotTx
2023-03-16 22:42:56 -07:00
danielsanchezq 780276497f Fix endless loop on mocktally 2023-03-15 10:02:29 +01:00
Al Liu 818d7f29cd
Add generic block (#93)
* add generic block

* fix PR comments

* Block uses tx hashes

* Refactor bounds and generics to accept block type

* remove Tx generics

* add generic for block

* Remove unnecessary bounds on leadership

* Impl from with ownership for mock tx and txid

* feature gate

---------

Co-authored-by: danielsanchezq <sanchez.quiros.daniel@gmail.com>
2023-03-14 09:55:08 -07:00
Daniel Sanchez 6ce9fdf553
[WIP] Simulation app barebones (#90)
* Add simulations crate

* Kickstart modules

* Add overlay

* Use node steps

* Add initial carnot node

* Implement Carnot node costs simulation

* Firs working runner

* Cleanup

* Extract overlay from runner

* Fix off id calls

* Fix testing values

* Clippy happy

* make simulation app compile to wasm

---------

Co-authored-by: al8n <scygliu1@gmail.com>
2023-03-14 07:15:45 -07:00
Daniel Sanchez 274e8d55fd
Voting core primitive (#82)
* Add tally trait
Implement mock tally

* Add tally to consensus

* Use tally in implemented overlays

* Clippy happy

* Scratch carnot voting as per specification

* Add missing derives

* Clippy happy

* Fix tests

* Add checks on valid votes

* Refactor ApprovalMsg to VoteMsg

* Remove no vote in MockVote and MockQC

* Remove no vote in MockVote and MockQC

* Remove timeout todo

* Fix tests
2023-03-14 03:32:36 -07:00
gusto ae5c5b9d4c
Update rust docker image for CI (#91)
* Update rust docker image for CI

* Use macos aarch64 until x86 issues are solved
2023-03-07 21:19:38 +02:00
gusto 4981c724af
Http status codes (#88)
* Include status codes in http error responses

* Mockpool bridges error handling

* Last TX in milliseconds in mempool metrics

* u64 for last tx metrics
2023-03-06 15:19:27 +02:00
gusto 42ea8f9be3
Update waku-bindings to 0.1.0-rc.2 (#86) 2023-03-06 15:18:27 +02:00
Daniel Sanchez c6bc35a931
Make carnot overlay generic (#87)
* Make carnot overlay generic

* support generic

* add back 'view for View fns

* add assertion on view number

* remove unused comments

* fix fmt

---------

Co-authored-by: al8n <scygliu1@gmail.com>
2023-03-03 00:23:51 +08:00
Álvaro Castro-Castilla dfe17696e6
Fix Nix derivation so it works in a Darwin Nix environment (explicitly requiring clang and setting env variable) (#84) 2023-02-24 11:08:46 +02:00
Daniel Sanchez 55aece674b
Update to waku rc 1 (#80) 2023-02-22 15:55:59 +01:00
Al Liu 7f609db62a
Improve Mock network (#78)
* add MockTxId and send back transaction response messages
2023-02-21 16:10:26 +08:00
gusto 7f7a0db88a
Update rust and go versions for macos builds (#77)
* Add additional libs for macos nix env

* Add CoreFoundation and Security for macos

* Update rust 1.67.0 and go 1.19.5
2023-02-16 17:42:31 +02:00
Daniel Sanchez 8cc37385b3
Waku cached streams consensus adapter (#70)
* Added waku archive message to waku network backend

* Use cached streams in consensus waku adapter

* Fix mock test

* Add missing import

* Join requests tasks

* Use waku-bindings beta4

* Get stream from archive query method

* Set store protocol active for waku backend

* Implement local query stream response

* Add missing linking flags for new waku-bindings version

* Cleanup unbounded sender fuse/unwrap

* Clippy happy
2023-02-15 16:49:49 +01:00
Daniel Sanchez 1d195959d8
Add services and project information to project README (#72)
* Add services and project information to project README

* Better phrased nomos services
2023-02-10 17:07:04 +01:00
Al Liu 6b45bf408e
add Debug and constructor for CarnotSettings (#71) 2023-02-10 19:04:56 +08:00
Daniel Sanchez 320755d19d
Detach tx broadcast from mempool (#69)
* Remove send tx method from mempool network adapter

* Add error reporting to add_tx operation in mempool
Delegate broadcasting to external caller
2023-02-08 11:07:09 +01:00
Giacomo Pasini bbb783e1da
Complete consensus (#65)
* complete consensus

* review comments
2023-02-08 10:23:55 +01:00
gusto 5f21a2734a
Add endpoint to make a new peer conn for mockpool node (#68)
* Add enpoint to make a new peer conn for mockpool node

* Pass multiple addresses in connection request

* Join async reqs to network service, cleanup

* Collect and join reqs
2023-02-07 17:50:38 +02:00
Al Liu 146001c9fe
Mock mempool integration test (#66)
* finish mock mempool integration test

* use Log service for test

* remove unused example

* use millis and merge log PR
2023-02-07 17:13:22 +08:00
Daniel Sanchez c5ac1db44c
Network transaction broadcasting (#63)
* Broadcast transaction when validated in mempool

* Clippy happy

* Use standard bincode config for tx message decoding

* add send_transaction for mock (#64)

* add send_transaction for mock

* Use wire instead of direct bincode

* Use wire instead of direct bincode on deserialization

---------

Co-authored-by: Al Liu <scygliu1@gmail.com>
2023-02-06 12:49:24 +01:00
Al Liu 4705cc213b
Add constructor for LogSettings (#67)
* add constructor for LogSettings
2023-02-06 19:23:25 +08:00
Daniel Sanchez 9a8400f4ce
Use blake 2b instead of 2s for mockpool node TxId (#62)
* Use blake 2b instead of 2s

* Clippy happy
2023-02-02 00:20:16 -08:00
Al Liu 3d3d2760ec
Implement #31: Mock network backend (#41)
* relax trait bounds

* mock mempool

* remove unused generic

* add mock network test case

* fix some PR comments

* simplify match branch

* finish mempool adapter example

* clippy happy

* mock consensus
2023-02-01 21:58:23 +08:00