ba90ed1b55
Nomos http API ( #476 )
...
* Info api
* Da blob api (#487 )
* Add storage api for new http backend (#488 )
* Mempool add APIs (#489 )
2023-11-01 17:14:58 +08:00
ff8fc577a0
Fix typos ( #481 )
...
Co-authored-by: Daniel Sanchez <sanchez.quiros.daniel@gmail.com>
2023-10-31 14:38:42 +01:00
9b3c675b3a
Metrics api ( #466 )
...
* Add metrics API
2023-10-31 17:20:04 +08:00
1553f29bd9
Humanize array ser/deser ( #468 )
...
* humanize array ser/deser
* split fns
* use `const-hex`
* fix fmt
* create `nomos-utils` crate
* Human serde committeeid (#478 )
* Human readable serde for CommitteeId
* Deserialize bytes to string if human readable
* Don't allocate if possible in human serde bytes
---------
Co-authored-by: gusto <bacv@users.noreply.github.com>
2023-10-26 23:16:10 +08:00
dbac7d7597
fix clippy and check warnings ( #452 )
2023-10-06 15:08:52 +08:00
99d77f7e1c
Nomos node tree overlay ( #415 )
...
* Use tree overlay in nomos node
* Use tree overlay in tests module
* Handle unexpected consensus vote stream end in tally
* Spawn the next leader first (#425 )
* Report unhappy blocks in happy path test (#430 )
* report unhappy blocks in the happy path test
* Make threshold configurable for TreeOverlay (#426 )
* Modified test, so that all nodes don’t all connect to the first node only (#442 )
* merge fix
---------
Co-authored-by: Youngjoon Lee <taxihighway@gmail.com>
Co-authored-by: Al Liu <scygliu1@gmail.com>
2023-10-05 11:56:24 +03:00
70dc4f3ae6
Simulation branch with one committee fix ( #402 )
...
* Tree and branch overlay comparison tests
* Update leader super majority fn in branch overlay
* Cleanup tests
2023-09-15 09:10:46 +03:00
5391382928
Using CSV format by default for simulations ( #304 )
2023-08-21 12:02:10 +08:00
2612c77306
Implement research overlay ( #291 )
...
* implement research overlay
* Add branch overlay
* Branch overlay config in simulation
* Remove research related extensions
* Use branch_depth and chunks
---------
Co-authored-by: Gusto <bacvinka@gmail.com>
2023-08-10 14:31:15 +03:00
44367a15a5
Sim update overlay ( #298 )
...
* Update sim engine on timeout qc
* Remove rebuild method
2023-08-10 13:18:17 +02:00
d675585a0f
Test for partial message sending in simulation ( #294 )
...
* Test for partial message sending
* Test correctness, typos
* Fix the node capacity flushing
* Not process double timeout qcs in simulations
* Discard older view messages in simulations messages
* Refactor committed_blocks to latest_committed_blocks
* Remove tally default
* Fix condition to root committee parenting
* Bring back pruning
* Clippy happy
---------
Co-authored-by: danielsanchezq <sanchez.quiros.daniel@gmail.com>
2023-08-08 18:00:08 +03:00
4bdc3ed15a
Add update committees structure for overlay ( #295 )
...
* Move roundrobin to leadership module
* Use references in leader selection
* Add membership traits to overlay and create membership module
* Implement committee membership for random beacon state
* Update flat overlay
* Create updateable membership traits and impls
* Update tree overlay
* Update overlay on consensus service
* Update overlay on simulations nomos node
* Update types on tests and modules
* Use chacha for shuffling
* Change to mut slice instead of inner cloning
* Use fisher yates shuffle from scratch
* Stylish and clippy happy
2023-08-08 10:34:02 +02:00
30bf101576
Disable tree rebuild ( #293 )
2023-08-04 12:17:10 +02:00
c16b794517
fix large msg sending logic ( #274 )
...
* fix large msg sending logic
2023-08-03 20:05:43 +08:00
fa8e1025f5
Fix: genesis pruning in engine ( #289 )
...
* Do not prune genesis
* Fix retain condition
2023-08-03 12:25:33 +02:00
fac42cd31d
Prune blocks in consensus engine ( #275 )
...
* Add prune older blocks call
* Prune older blocks in simulation app
* Use latest safe view for failure case
* Pick parent view from high qc
* Add safety assertion on pruning method
2023-08-01 16:47:47 +02:00
ae59be5466
Refactor parent committee methods to return optional parent instead of empty one ( #282 )
2023-08-01 09:19:29 +02:00
8dd34f81b4
Simulation tally fix ( #268 )
...
* Do not remove entry cache when the threshold is reached
* Leader super majority change
* Update leader threshold test
2023-07-21 12:30:32 +03:00
94e384f609
Tree settings for sim overlay ( #257 )
...
* Tree settings for sim overlay
* WIP dyn node simulation
* Allow generic settings and states for dispatched nodes
* Update tests
2023-07-18 14:44:23 +03:00
c29a641a9f
Refactor NetworkAdapter ( #258 )
...
* Rework NetworkAdapter API
The NetworkAdapter API failed to isolate the internals by
providing a way to send a message to a user-provided channel while
the stream listeners expected specific formats.
Unify network messages under the same enum and simplify sending/
broadcasting messages.
* remove redundant inlines
* use committee.id()
* fmt
2023-07-12 16:12:25 +02:00
9467351c10
Finish `View` wrapper ( #254 )
...
* finish View wrapper
2023-07-12 21:30:22 +08:00
7a776af530
Finish `BlockId` wrapper ( #253 )
...
* finish BlockId wrapper
2023-07-12 19:15:29 +08:00
2135676606
Finish `NodeId` type wrapper ( #252 )
...
* add NodeId wrapper
2023-07-11 23:16:49 +08:00
a0cb738b9f
Finish `Committee` wrapper ( #251 )
...
* Committee type wrapper
---------
Co-authored-by: danielsanchezq <sanchez.quiros.daniel@gmail.com>
2023-07-07 16:08:19 +08:00
95bfd24f6d
Finish `CommitteeId` wrapper ( #249 )
...
* CommitteeId type wrapper
* rewrite committee id logic
* remove unused convert
* use correct way to get committe id by member id
* cleanup and add comment
* Simplify child committees method
---------
Co-authored-by: danielsanchezq <sanchez.quiros.daniel@gmail.com>
2023-07-07 15:38:28 +08:00
8d0360ab3c
fix(fuzz): fix precondition for receive_timeout_qc ( #248 )
2023-07-06 21:14:26 +09:00
d0c6df23fc
Implement Rust version tree overlay ( #238 )
...
* implement tree overlay in Rust
2023-07-05 17:28:22 +08:00
90cf29bf86
Better tests ( #232 )
...
* add basic integration tests
* add a way to configure overlay threshold
* Save logs to file in case of failure
* Increase number of test nodes to 10
* fix tests
* use fraction instead of tuple
* fmt
2023-07-04 11:39:28 +02:00
5199ee12e9
fix: add a guard on the view for LeaderProof validation ( #233 )
2023-07-04 18:31:33 +09:00
98aa138b87
feat: enforce TimeoutQc to be constructed only by new() ( #229 )
2023-06-28 23:37:27 +09:00
8956a3547b
fix: remove `Qc.parent_view()` that has a bug ( #223 )
2023-06-27 23:56:36 +09:00
3eceed5d9a
test(fuzz): more randomness for receive_timeout_qc ( #226 )
2023-06-27 08:34:16 +09:00
bdf31b77cb
docs(fuzz): update README ( #227 )
2023-06-26 23:46:24 +09:00
2c9d6b1401
test(fuzz): add more invariant checks ( #224 )
2023-06-26 20:34:12 +09:00
feb428bf18
fix(fuzz): apply `ReceiveSafeBlock` & `ReceiveUnsafeBlock` only when parent exists ( #222 )
2023-06-26 20:20:55 +09:00
deeb3eeba0
test(fuzz): add `approve_new_view` & `receive_safe_block_with_aggregated_qc` transition ( #208 )
2023-06-26 18:37:35 +09:00
ec3fb62baf
refactor(fuzz): modularize fuzz testing ( #211 )
2023-06-22 15:15:38 +09:00
f4194bc728
test(fuzz): add `local_timeout` & `receive_timeout_qc` transitions ( #207 )
2023-06-22 10:58:26 +09:00
371ba17922
test: consensus-engine fuzz testing (happy path) ( #186 )
2023-06-21 22:35:32 +09:00
f631d9b737
fix: prevent approving new views not bigger than the last timeout_qc ( #165 )
2023-06-15 18:31:52 +09:00
2c23bda702
Carnot info api ( #177 )
...
* add missing accessors
* add api to return info from consensus service
* move processing into its own function
* feat: add HTTP GET `/carnot/info` API (#178 )
---------
Co-authored-by: Youngjoon Lee <taxihighway@gmail.com>
2023-06-15 11:29:50 +02:00
27d9f72035
Simulation happy path ( #161 )
...
* finish subscriber manager
* optimize subscribe on SimulationRunnerHandle
* fix comment
* replace std locks to parking_lot locks
* move producer initialization out simulate fn
* WIP
* optimize run fn
* update condition
* fix CI
* collect run times
* Add happy-path consensus engine
* tmp
* Fit types from spec (#124 )
* Match types to spec
* Remove Output import
* Consensus engine rework (#126 )
* rework
* fix test
* clippy happy
---------
Co-authored-by: Giacomo Pasini <Zeegomo@users.noreply.github.com>
* Adapt carnot network adapter interfaces and implementations
* Fix errors
* Update network with engine types
* Fit types yet again
* Remove leadership and old overlay
Create carnot event builder
Added some adjustments
* Add view to vote
* Fix serde derive in consensus-engine
* Add serde feature for engine in core
* Use view in tally
* Move carnot tally to consensus service
* Add new view msg
* Fit engine types in adapter
* Missing serde feature in consensus service
* Implement carnot event builder
* Implement even builder run main tasks
* Fill up view resolver
* Fix errors on network adapter implementations
* Clippy happy
* Extract event handling to independent methods in View
* Fix test
* Refactor carnot event builder (#135 )
* refactor
* format
* Discriminate proposal messages (#136 )
* Derive block id from wire format (#139 )
* Derive block id from wire format
* Derive id on block creation
* Use compile time hash size
* Add leader role (#138 )
* add leadership stub
* fix gather_new_views
* fmt
* actually build qc
* remove redundant fields
* add flat overlay (#143 )
* add flat overlay
* fix
* sort imports
* fix tests
* Fix waku update
* rewrite data collection add different kind of subscribers
* fix fmt
* Unhappy tally (#137 )
* Refactor tally module
* Implement tally for new view messages
* Assess pr comments
* Fix rebase
* simplify tally
---------
Co-authored-by: Giacomo Pasini <g.pasini98@gmail.com>
* fix gather_new_views
* working node
* fix unhappy path
* remove leftover
* Kickstart event building in sim app
* finish event builder
* fix comment
* add Tally
* gather enough new views then construct ProposalBlock event
* Revert "gather enough new views then construct ProposalBlock event"
This reverts commit 87da2bdd0c#154 )
* Implement leader proposing
* fix fmt
---------
Co-authored-by: al8n <scygliu1@gmail.com>
* fix ser/deser bugs
* fix subscriber bugs
* Fix proposing genesis
* Fix genesis retrieval in consensus-engine
* Bring back general block proposal event
* Fix leaf voting
* fix init node bugs
* add more tracing
* fix empty qc
* fix data race
* fix all panics
* cleanup
* propose new blocks
* fix comment
* do not approve for the same block
* no panics
* fix some comments
* use serde_with
* Bring back genesis on 0
* Fix genesis retrieval
Replace output enum
Vote for genesis proposal
* Genesis methods
* fix StardardQc::genesis()
* fix genesis block bug
* fix PR comment
* fix PR comment
* fix PR comment
* fix PR comment
* fix PR comment
* Fix tally
Fix proposing
* Remove public block building
Added raw method
* Missing fmt
* clippy happy
* fix io stream downcast
* optional stream-type arg, by default we do not run any subscriber
* fmt
* cleanup
* Remove from header block constructor
* Fix duplicated approve (#180 )
* fix duplicated approve
* Success tally just on threshold
* Integrate random beacon on happy path
* Fix missing updating beacon
* Replicate consensus output
* Prune older non relevant messages from cache
* Remove view info just again
* Refactor Block deps
* Reverse wrong parent committee call in Consensus engine
* Remove useless event builder settings
* Remove blocks store from event builder
* Remove unnecessary carnot seed
* Remove duplicated proposals check
---------
Co-authored-by: Giacomo Pasini <g.pasini98@gmail.com>
Co-authored-by: Daniel Sanchez <sanchez.quiros.daniel@gmail.com>
Co-authored-by: Giacomo Pasini <Zeegomo@users.noreply.github.com>
2023-06-14 16:52:37 +02:00
9c81b72711
Random beacon ( #167 )
...
* move overlay to consensus engine
* Integrate random beacon in overlay
This commit integrates the random beacon as specified in the nomos
spec into the consensus engine library as part of the overlay.
In addition, it separates the overlay part responsible for leader
selection as an independent trait LeaderSelection, so as to share
the overall overlay structure among most constructions.
Furthermore, a leader proof has been added to a block to verify
that the proposer had valid rights to do so.
The current implementation hardcodes the leader selection update
in the consensus service, but we probably want to abstract it away
through something like the adapter pattern we use of other services
in the node.
* Move leader selection update to separate function
* Add generic support for leader selection in consensus service (#170 )
* Add generic support for leader selection in consensus service
* fix
* use settings struct instead of tuple
* fix tests
2023-06-12 15:14:49 +02:00
cfaa7cf772
fix: prevent skipping the current view when calculating latest_committed_block ( #163 )
2023-06-07 19:32:48 +09:00
6c64720e39
Add root timeout handling ( #153 )
...
* Add handling of root timeout
* Use StandardQc instead of Qc for high_qc
Since high_qc is guaranteed to be a standard qc, let's just use
StandardQc as a type in all messages that use it.
2023-05-22 15:23:30 +02:00
bc453b686f
Consensus orchestrator ( #128 )
...
* Add happy-path consensus engine
* tmp
* Fit types from spec (#124 )
* Match types to spec
* Remove Output import
* Consensus engine rework (#126 )
* rework
* fix test
* clippy happy
---------
Co-authored-by: Giacomo Pasini <Zeegomo@users.noreply.github.com>
* Adapt carnot network adapter interfaces and implementations
* Fix errors
* Update network with engine types
* Fit types yet again
* Remove leadership and old overlay
Create carnot event builder
Added some adjustments
* Add view to vote
* Fix serde derive in consensus-engine
* Add serde feature for engine in core
* Use view in tally
* Move carnot tally to consensus service
* Add new view msg
* Fit engine types in adapter
* Missing serde feature in consensus service
* Implement carnot event builder
* Implement even builder run main tasks
* Fill up view resolver
* Fix errors on network adapter implementations
* Clippy happy
* Extract event handling to independent methods in View
* Fix test
* Refactor carnot event builder (#135 )
* refactor
* format
* Discriminate proposal messages (#136 )
* Derive block id from wire format (#139 )
* Derive block id from wire format
* Derive id on block creation
* Use compile time hash size
* Add leader role (#138 )
* add leadership stub
* fix gather_new_views
* fmt
* actually build qc
* remove redundant fields
* add flat overlay (#143 )
* add flat overlay
* fix
* sort imports
* fix tests
* Unhappy tally (#137 )
* Refactor tally module
* Implement tally for new view messages
* Assess pr comments
* Fix rebase
* simplify tally
---------
Co-authored-by: Giacomo Pasini <g.pasini98@gmail.com>
* Working node (#149 )
* fix gather_new_views
* working node
* fix unhappy path
* remove leftover
* fix comments
* update waku (#146 )
* update waku
* Fix waku update
---------
Co-authored-by: danielsanchezq <sanchez.quiros.daniel@gmail.com>
* little fixes
* Consensus tasks cancellation (#147 )
* fix
* Create view cancel and cancel cache
* Attach cancellation to consensus tasks
* Fix view binds
---------
Co-authored-by: Giacomo Pasini <g.pasini98@gmail.com>
---------
Co-authored-by: danielsanchezq <sanchez.quiros.daniel@gmail.com>
* Remove clones on consts
---------
Co-authored-by: Al Liu <scygliu1@gmail.com>
Co-authored-by: Giacomo Pasini <g.pasini98@gmail.com>
Co-authored-by: Giacomo Pasini <Zeegomo@users.noreply.github.com>
2023-05-22 14:56:56 +02:00
26d10856ff
Adapt carnot network adapter interfaces and implementations to latest spec ( #121 )
...
* Add happy-path consensus engine
* tmp
* Fit types from spec (#124 )
* Match types to spec
* Remove Output import
* Consensus engine rework (#126 )
* rework
* fix test
* clippy happy
---------
Co-authored-by: Giacomo Pasini <Zeegomo@users.noreply.github.com>
* Adapt carnot network adapter interfaces and implementations
* Fix errors
* Update network with engine types
* Fit types yet again
---------
Co-authored-by: Al Liu <scygliu1@gmail.com>
Co-authored-by: Giacomo Pasini <g.pasini98@gmail.com>
Co-authored-by: Giacomo Pasini <Zeegomo@users.noreply.github.com>
2023-05-01 17:47:56 +02:00
f8617d7331
Consensus engine rework ( #127 )
...
---------
Co-authored-by: Giacomo Pasini <Zeegomo@users.noreply.github.com>
---------
Co-authored-by: Al Liu <scygliu1@gmail.com>
Co-authored-by: Daniel Sanchez <sanchez.quiros.daniel@gmail.com>
2023-04-27 19:18:24 +02:00
Al Liu
GoodDaisy
gusto
Daniel Sanchez
Giacomo Pasini
Youngjoon Lee