Youngjoon Lee
98aa138b87
feat: enforce TimeoutQc to be constructed only by new() ( #229 )
2023-06-28 23:37:27 +09:00
Giacomo Pasini
b884e1ceca
Fix waku backend ( #231 )
...
* Fix order of network streams
When fetching a message from the network, we need to first listen
for incoming messages and then look at the storage. If we do this
in the opposite order, there's a brief moment where we've freezed
our view of stored messages and are not yet listening for incoming
ones, thus risking loosing messages.
* Wait for waku db spurious delays
Sometimes waku takes some time (e.g. a few seconds) before making
a received message available through the archive query. Let's
account for this by making repeated calls if the first one is not
successful.
* Add initial network wait
We've observed in testing that even if waku reports that some
peers are connected it can't really deliver a message. To overcome
this limitation, we add a wait at the network service startup.
We know this is not ideal, but Waku will eventually be replaced and
we're looking for a quick fix.
* fmt
2023-06-27 16:29:01 +02:00
Giacomo Pasini
09370dcef8
Integrations tests ( #221 )
...
* make config struct fields public
* add basic integration tests
* Add libssl-dev dependency
* fix comments
---------
Co-authored-by: Gusto <bacvinka@gmail.com>
2023-06-27 13:05:09 +02:00
Al Liu
c74b53be2e
Log message sending failure ( #206 )
...
* fix a todo in waku adapter
2023-06-21 16:30:47 +08:00
Youngjoon Lee
f02234f9d1
deps: bump waku-bindings to handle ephemeral field ( #160 )
2023-06-20 20:58:22 +09:00
Youngjoon Lee
6061ba3f3d
fix: prevent skipping one view when proposing a block in unhappy path ( #199 )
2023-06-20 00:09:33 +09:00
Giacomo Pasini
2c23bda702
Carnot info api ( #177 )
...
* add missing accessors
* add api to return info from consensus service
* move processing into its own function
* feat: add HTTP GET `/carnot/info` API (#178 )
---------
Co-authored-by: Youngjoon Lee <taxihighway@gmail.com>
2023-06-15 11:29:50 +02:00
Al Liu
27d9f72035
Simulation happy path ( #161 )
...
* finish subscriber manager
* optimize subscribe on SimulationRunnerHandle
* fix comment
* replace std locks to parking_lot locks
* move producer initialization out simulate fn
* WIP
* optimize run fn
* update condition
* fix CI
* collect run times
* Add happy-path consensus engine
* tmp
* Fit types from spec (#124 )
* Match types to spec
* Remove Output import
* Consensus engine rework (#126 )
* rework
* fix test
* clippy happy
---------
Co-authored-by: Giacomo Pasini <Zeegomo@users.noreply.github.com>
* Adapt carnot network adapter interfaces and implementations
* Fix errors
* Update network with engine types
* Fit types yet again
* Remove leadership and old overlay
Create carnot event builder
Added some adjustments
* Add view to vote
* Fix serde derive in consensus-engine
* Add serde feature for engine in core
* Use view in tally
* Move carnot tally to consensus service
* Add new view msg
* Fit engine types in adapter
* Missing serde feature in consensus service
* Implement carnot event builder
* Implement even builder run main tasks
* Fill up view resolver
* Fix errors on network adapter implementations
* Clippy happy
* Extract event handling to independent methods in View
* Fix test
* Refactor carnot event builder (#135 )
* refactor
* format
* Discriminate proposal messages (#136 )
* Derive block id from wire format (#139 )
* Derive block id from wire format
* Derive id on block creation
* Use compile time hash size
* Add leader role (#138 )
* add leadership stub
* fix gather_new_views
* fmt
* actually build qc
* remove redundant fields
* add flat overlay (#143 )
* add flat overlay
* fix
* sort imports
* fix tests
* Fix waku update
* rewrite data collection add different kind of subscribers
* fix fmt
* Unhappy tally (#137 )
* Refactor tally module
* Implement tally for new view messages
* Assess pr comments
* Fix rebase
* simplify tally
---------
Co-authored-by: Giacomo Pasini <g.pasini98@gmail.com>
* fix gather_new_views
* working node
* fix unhappy path
* remove leftover
* Kickstart event building in sim app
* finish event builder
* fix comment
* add Tally
* gather enough new views then construct ProposalBlock event
* Revert "gather enough new views then construct ProposalBlock event"
This reverts commit 87da2bdd0c5d7ba19c110e128749ee356934ccbd.
* WIP: CarnotNode
* WIP
* finish event handle
* dump state
* WIP
* finish message sending
* fix some compile errors
* make project compile
* update
* fix fmt and clippy
* optimize json ser/deser and add a config
* update Cargo.toml
* Implement leader proposing (#154 )
* Implement leader proposing
* fix fmt
---------
Co-authored-by: al8n <scygliu1@gmail.com>
* fix ser/deser bugs
* fix subscriber bugs
* Fix proposing genesis
* Fix genesis retrieval in consensus-engine
* Bring back general block proposal event
* Fix leaf voting
* fix init node bugs
* add more tracing
* fix empty qc
* fix data race
* fix all panics
* cleanup
* propose new blocks
* fix comment
* do not approve for the same block
* no panics
* fix some comments
* use serde_with
* Bring back genesis on 0
* Fix genesis retrieval
Replace output enum
Vote for genesis proposal
* Genesis methods
* fix StardardQc::genesis()
* fix genesis block bug
* fix PR comment
* fix PR comment
* fix PR comment
* fix PR comment
* fix PR comment
* Fix tally
Fix proposing
* Remove public block building
Added raw method
* Missing fmt
* clippy happy
* fix io stream downcast
* optional stream-type arg, by default we do not run any subscriber
* fmt
* cleanup
* Remove from header block constructor
* Fix duplicated approve (#180 )
* fix duplicated approve
* Success tally just on threshold
* Integrate random beacon on happy path
* Fix missing updating beacon
* Replicate consensus output
* Prune older non relevant messages from cache
* Remove view info just again
* Refactor Block deps
* Reverse wrong parent committee call in Consensus engine
* Remove useless event builder settings
* Remove blocks store from event builder
* Remove unnecessary carnot seed
* Remove duplicated proposals check
---------
Co-authored-by: Giacomo Pasini <g.pasini98@gmail.com>
Co-authored-by: Daniel Sanchez <sanchez.quiros.daniel@gmail.com>
Co-authored-by: Giacomo Pasini <Zeegomo@users.noreply.github.com>
2023-06-14 16:52:37 +02:00
Giacomo Pasini
75025e8cf0
Consensus service refactor ( #176 )
...
The consensus service was becoming a bit messy and difficult to maintain.
This PR moves handling invididual events to their own function and refactors the cancelabe task system in a separate struct.
2023-06-13 12:11:55 +02:00
Giacomo Pasini
9c81b72711
Random beacon ( #167 )
...
* move overlay to consensus engine
* Integrate random beacon in overlay
This commit integrates the random beacon as specified in the nomos
spec into the consensus engine library as part of the overlay.
In addition, it separates the overlay part responsible for leader
selection as an independent trait LeaderSelection, so as to share
the overall overlay structure among most constructions.
Furthermore, a leader proof has been added to a block to verify
that the proposer had valid rights to do so.
The current implementation hardcodes the leader selection update
in the consensus service, but we probably want to abstract it away
through something like the adapter pattern we use of other services
in the node.
* Move leader selection update to separate function
* Add generic support for leader selection in consensus service (#170 )
* Add generic support for leader selection in consensus service
* fix
* use settings struct instead of tuple
* fix tests
2023-06-12 15:14:49 +02:00
Youngjoon Lee
a055c2524a
fix: prevent skipping one view when proposing a block in unhappy path ( #166 )
2023-06-07 09:56:21 +09:00
Youngjoon Lee
6229fc98bc
fix gather_timeout being pending ( #156 )
2023-05-27 09:32:39 +09:00
Giacomo Pasini
6c64720e39
Add root timeout handling ( #153 )
...
* Add handling of root timeout
* Use StandardQc instead of Qc for high_qc
Since high_qc is guaranteed to be a standard qc, let's just use
StandardQc as a type in all messages that use it.
2023-05-22 15:23:30 +02:00
Daniel Sanchez
bc453b686f
Consensus orchestrator ( #128 )
...
* Add happy-path consensus engine
* tmp
* Fit types from spec (#124 )
* Match types to spec
* Remove Output import
* Consensus engine rework (#126 )
* rework
* fix test
* clippy happy
---------
Co-authored-by: Giacomo Pasini <Zeegomo@users.noreply.github.com>
* Adapt carnot network adapter interfaces and implementations
* Fix errors
* Update network with engine types
* Fit types yet again
* Remove leadership and old overlay
Create carnot event builder
Added some adjustments
* Add view to vote
* Fix serde derive in consensus-engine
* Add serde feature for engine in core
* Use view in tally
* Move carnot tally to consensus service
* Add new view msg
* Fit engine types in adapter
* Missing serde feature in consensus service
* Implement carnot event builder
* Implement even builder run main tasks
* Fill up view resolver
* Fix errors on network adapter implementations
* Clippy happy
* Extract event handling to independent methods in View
* Fix test
* Refactor carnot event builder (#135 )
* refactor
* format
* Discriminate proposal messages (#136 )
* Derive block id from wire format (#139 )
* Derive block id from wire format
* Derive id on block creation
* Use compile time hash size
* Add leader role (#138 )
* add leadership stub
* fix gather_new_views
* fmt
* actually build qc
* remove redundant fields
* add flat overlay (#143 )
* add flat overlay
* fix
* sort imports
* fix tests
* Unhappy tally (#137 )
* Refactor tally module
* Implement tally for new view messages
* Assess pr comments
* Fix rebase
* simplify tally
---------
Co-authored-by: Giacomo Pasini <g.pasini98@gmail.com>
* Working node (#149 )
* fix gather_new_views
* working node
* fix unhappy path
* remove leftover
* fix comments
* update waku (#146 )
* update waku
* Fix waku update
---------
Co-authored-by: danielsanchezq <sanchez.quiros.daniel@gmail.com>
* little fixes
* Consensus tasks cancellation (#147 )
* fix
* Create view cancel and cancel cache
* Attach cancellation to consensus tasks
* Fix view binds
---------
Co-authored-by: Giacomo Pasini <g.pasini98@gmail.com>
---------
Co-authored-by: danielsanchezq <sanchez.quiros.daniel@gmail.com>
* Remove clones on consts
---------
Co-authored-by: Al Liu <scygliu1@gmail.com>
Co-authored-by: Giacomo Pasini <g.pasini98@gmail.com>
Co-authored-by: Giacomo Pasini <Zeegomo@users.noreply.github.com>
2023-05-22 14:56:56 +02:00
Giacomo Pasini
fdc22111d3
use nanosecond precision in waku timestamp ( #152 )
2023-05-18 15:19:36 +02:00
Giacomo Pasini
bbd313f70a
update waku ( #146 )
...
* update waku
* Fix waku update
---------
Co-authored-by: danielsanchezq <sanchez.quiros.daniel@gmail.com>
2023-05-15 16:21:49 +02:00
Daniel Sanchez
26d10856ff
Adapt carnot network adapter interfaces and implementations to latest spec ( #121 )
...
* Add happy-path consensus engine
* tmp
* Fit types from spec (#124 )
* Match types to spec
* Remove Output import
* Consensus engine rework (#126 )
* rework
* fix test
* clippy happy
---------
Co-authored-by: Giacomo Pasini <Zeegomo@users.noreply.github.com>
* Adapt carnot network adapter interfaces and implementations
* Fix errors
* Update network with engine types
* Fit types yet again
---------
Co-authored-by: Al Liu <scygliu1@gmail.com>
Co-authored-by: Giacomo Pasini <g.pasini98@gmail.com>
Co-authored-by: Giacomo Pasini <Zeegomo@users.noreply.github.com>
2023-05-01 17:47:56 +02:00
Daniel Sanchez
1ec4231a7a
Add Qc to block ( #99 )
...
* Added Qc type to Tally trait
* Add generic Qc to block header
* Use blockid instead of unnecessary header
* Expand Qc over generics
* Build up block with proper qc header
2023-03-17 06:23:50 -07:00
Daniel Sanchez
91ce4e6fa1
Make a transaction trait ( #98 )
...
* Impl Transaction trait
* Impl Transaction for MockTransaction
* <ake carnot transaction a module
* Refactor consensus to use Transaction
* Fix tests
* Constrain Transaction::Hash
* Refactor redundant Carnot in CarnotTx
2023-03-16 22:42:56 -07:00
Al Liu
818d7f29cd
Add generic block ( #93 )
...
* add generic block
* fix PR comments
* Block uses tx hashes
* Refactor bounds and generics to accept block type
* remove Tx generics
* add generic for block
* Remove unnecessary bounds on leadership
* Impl from with ownership for mock tx and txid
* feature gate
---------
Co-authored-by: danielsanchezq <sanchez.quiros.daniel@gmail.com>
2023-03-14 09:55:08 -07:00
Daniel Sanchez
274e8d55fd
Voting core primitive ( #82 )
...
* Add tally trait
Implement mock tally
* Add tally to consensus
* Use tally in implemented overlays
* Clippy happy
* Scratch carnot voting as per specification
* Add missing derives
* Clippy happy
* Fix tests
* Add checks on valid votes
* Refactor ApprovalMsg to VoteMsg
* Remove no vote in MockVote and MockQC
* Remove no vote in MockVote and MockQC
* Remove timeout todo
* Fix tests
2023-03-14 03:32:36 -07:00
gusto
42ea8f9be3
Update waku-bindings to 0.1.0-rc.2 ( #86 )
2023-03-06 15:18:27 +02:00
Daniel Sanchez
c6bc35a931
Make carnot overlay generic ( #87 )
...
* Make carnot overlay generic
* support generic
* add back 'view for View fns
* add assertion on view number
* remove unused comments
* fix fmt
---------
Co-authored-by: al8n <scygliu1@gmail.com>
2023-03-03 00:23:51 +08:00
Daniel Sanchez
55aece674b
Update to waku rc 1 ( #80 )
2023-02-22 15:55:59 +01:00
Daniel Sanchez
8cc37385b3
Waku cached streams consensus adapter ( #70 )
...
* Added waku archive message to waku network backend
* Use cached streams in consensus waku adapter
* Fix mock test
* Add missing import
* Join requests tasks
* Use waku-bindings beta4
* Get stream from archive query method
* Set store protocol active for waku backend
* Implement local query stream response
* Add missing linking flags for new waku-bindings version
* Cleanup unbounded sender fuse/unwrap
* Clippy happy
2023-02-15 16:49:49 +01:00
Al Liu
6b45bf408e
add Debug and constructor for CarnotSettings ( #71 )
2023-02-10 19:04:56 +08:00
Daniel Sanchez
320755d19d
Detach tx broadcast from mempool ( #69 )
...
* Remove send tx method from mempool network adapter
* Add error reporting to add_tx operation in mempool
Delegate broadcasting to external caller
2023-02-08 11:07:09 +01:00
Giacomo Pasini
bbb783e1da
Complete consensus ( #65 )
...
* complete consensus
* review comments
2023-02-08 10:23:55 +01:00
Al Liu
3d3d2760ec
Implement #31 : Mock network backend ( #41 )
...
* relax trait bounds
* mock mempool
* remove unused generic
* add mock network test case
* fix some PR comments
* simplify match branch
* finish mempool adapter example
* clippy happy
* mock consensus
2023-02-01 21:58:23 +08:00
Daniel Sanchez
1e20c3b6cc
Use commitee and view information for consensus network adapter ( #57 )
...
* Add committee and view information to network adapter
* Use committee and view on waku adapter
* Add committee and view info to flat view implementation.
* Clippy happy
* Rename flat -> root
* Split broadcast block iterator into l/r childs
* Extract topics to builder functions
2023-01-31 03:03:45 -08:00
Giacomo Pasini
f5175c74c0
Add flat overlay ( #55 )
2023-01-27 10:37:04 +01:00
Giacomo Pasini
f5a1dd5513
Relax type bounds ( #49 )
2023-01-18 11:30:37 +01:00
Daniel Sanchez
7ff63d4824
Add fountain codes to consensus ( #38 )
...
* Add basic encode/decode test
* Use Stream for trait instead of Iterator
* Removed unnecessary pin
* Add custom fountain error
* Add failing path to tests
* Added docs
* Normalized chunks to bytes
* Added settings initialization for fountain codes trait
* Pipe fountain code through consensus
* Implement broadcast block and block reconstruction for Member Overlay
* Fix failing raptor test
* Use seed for raptorq tests
* Use const for topics instead of static
* Clippy happy
2023-01-10 12:58:51 +01:00
Giacomo Pasini
15f97dcace
fix consensus merge ( #42 )
2023-01-10 12:26:36 +01:00
Giacomo Pasini
0229337414
Add leadership stub ( #22 )
...
* add leadership stub
* move types to core
* clippy happy
2023-01-10 11:33:08 +01:00
Daniel Sanchez
9721e6f5fa
Mempool network adapter ( #30 )
...
* Refactor mempool naming and added settings to backend trait
* Implement mempool networking traits and waku backend
* Transaction networking plumbing in mempool service
* Make TransactionMsg generic
Use bincode to deserialize tx messages
* Make wakuadapter generic over tx and tx-id
* Fix wrong backend type bound
* Adapt to waku beta2
* Thread tx and id together for adapter and pool
* Panic on subscribing error
* Prune unnecessary Id type bounds
* Remove transaction placeholder
* Remove Id bound from waku adapter
* Remove empty transactions module
2023-01-09 17:22:46 +01:00
Daniel Sanchez
664cb3ccd8
Update waku-bindings to beta2 ( #40 )
2023-01-09 12:20:14 +01:00
Daniel Sanchez
fb3fd6f3b1
Nomos core ( #28 )
...
* Extract block to core crate
* Added linking flag for waku
* Cleanup imports
* Add missing core files
* Create more base mods
2022-12-13 15:35:11 +01:00
Giacomo Pasini
7cc9181574
Consensus backbone ( #16 )
...
* tmp
* add peers implem
* update return values
* Backport network consensus adapter (#26 )
* Pipe network adapter with proper types over original implementation
* Hold relay on CarnotConsensus
* Scratch Network adapter methods
* Fix tests blocking CI
* Fix waku feature on network crate
* Fix waku_bindings refs
* Restructure consensus network
* Stream block chunk
* Pipe adapter creation with subscription
* Add placeholder proposal chunk and approval messages
* Implement waku backend
* Clippy happy
* Use full path for tokio oneshot and error types in message_subscriber_channel method
* Clean imports
* small fixes
Co-authored-by: Daniel Sanchez Quiros <sanchez.quiros.daniel@gmail.com>
Co-authored-by: Daniel Sanchez Quiros <sanchez.quiros.daniel@gmail.com>
2022-12-13 11:15:54 +01:00
