99d77f7e1c
Nomos node tree overlay ( #415 )
...
* Use tree overlay in nomos node
* Use tree overlay in tests module
* Handle unexpected consensus vote stream end in tally
* Spawn the next leader first (#425 )
* Report unhappy blocks in happy path test (#430 )
* report unhappy blocks in the happy path test
* Make threshold configurable for TreeOverlay (#426 )
* Modified test, so that all nodes don’t all connect to the first node only (#442 )
* merge fix
---------
Co-authored-by: Youngjoon Lee <taxihighway@gmail.com>
Co-authored-by: Al Liu <scygliu1@gmail.com>
2023-10-05 11:56:24 +03:00
285804e1d5
Remove unnecessary dependencies ( #433 )
2023-09-25 17:26:21 +09:00
04d07038dc
Remove unused fountain codes ( #407 )
...
* Remove unused fountain codes
* Refactor ProposalChunk to Proposal
2023-09-15 09:39:53 +02:00
e1e2b84921
Block building core ( #401 )
...
* Move core block to folder base module
* Added blob select trait
* Added tx select trait
* Implement blob and tx selection filling size
* Added blobs to block constructor
Added block builder module
* Implement block builder
* Remove unnnecesary phantoms
* Add default impl for selections
* Added example
* Fix typos
Co-authored-by: Youngjoon Lee <taxihighway@gmail.com>
* tx to item on generic function
Co-authored-by: Youngjoon Lee <taxihighway@gmail.com>
---------
Co-authored-by: Youngjoon Lee <taxihighway@gmail.com>
2023-09-15 09:39:16 +02:00
03b9f69a01
Simulations: make polars dependancy optional ( #385 )
...
* Remove explicit target
* Use default profile for windows build
* Use stable-gnu for tests
* Make polars optional feature in simulations
* Remove unnecessary ci debug steps
2023-09-11 14:29:54 +03:00
57746f5e76
Block refactor ( #368 )
...
* Add sidecars to block
* Use cl and bl prefixes
2023-09-05 13:43:36 +02:00
5391382928
Using CSV format by default for simulations ( #304 )
2023-08-21 12:02:10 +08:00
87f49c0fe9
Optimize serialization/deserialization for `[u8; 32]` ( #306 )
...
* optimize simulation app serialization
2023-08-17 23:46:07 +08:00
fe3d39071d
Simulation graceful shutdown fixes ( #305 )
...
* Handle finished simulation
* Flush remaining records to the sink
* Write runtime metadata last
2023-08-16 11:08:03 +03:00
2612c77306
Implement research overlay ( #291 )
...
* implement research overlay
* Add branch overlay
* Branch overlay config in simulation
* Remove research related extensions
* Use branch_depth and chunks
---------
Co-authored-by: Gusto <bacvinka@gmail.com>
2023-08-10 14:31:15 +03:00
44367a15a5
Sim update overlay ( #298 )
...
* Update sim engine on timeout qc
* Remove rebuild method
2023-08-10 13:18:17 +02:00
ef5935d572
Simulation prune tally ( #299 )
...
* Add pruning methods
* Call pruning on node step method
* Fix tally retain closure
2023-08-10 13:12:48 +02:00
da60d8fc95
Parallel node init ( #300 )
2023-08-10 12:56:02 +03:00
d675585a0f
Test for partial message sending in simulation ( #294 )
...
* Test for partial message sending
* Test correctness, typos
* Fix the node capacity flushing
* Not process double timeout qcs in simulations
* Discard older view messages in simulations messages
* Refactor committed_blocks to latest_committed_blocks
* Remove tally default
* Fix condition to root committee parenting
* Bring back pruning
* Clippy happy
---------
Co-authored-by: danielsanchezq <sanchez.quiros.daniel@gmail.com>
2023-08-08 18:00:08 +03:00
4bdc3ed15a
Add update committees structure for overlay ( #295 )
...
* Move roundrobin to leadership module
* Use references in leader selection
* Add membership traits to overlay and create membership module
* Implement committee membership for random beacon state
* Update flat overlay
* Create updateable membership traits and impls
* Update tree overlay
* Update overlay on consensus service
* Update overlay on simulations nomos node
* Update types on tests and modules
* Use chacha for shuffling
* Change to mut slice instead of inner cloning
* Use fisher yates shuffle from scratch
* Stylish and clippy happy
2023-08-08 10:34:02 +02:00
c16b794517
fix large msg sending logic ( #274 )
...
* fix large msg sending logic
2023-08-03 20:05:43 +08:00
fac42cd31d
Prune blocks in consensus engine ( #275 )
...
* Add prune older blocks call
* Prune older blocks in simulation app
* Use latest safe view for failure case
* Pick parent view from high qc
* Add safety assertion on pruning method
2023-08-01 16:47:47 +02:00
ae59be5466
Refactor parent committee methods to return optional parent instead of empty one ( #282 )
2023-08-01 09:19:29 +02:00
976b1f9577
Add node step time to report ( #271 )
...
* Skip step if previous node exeeded step time
* Fix clippy
* Passively keep track of step execution time in node
2023-07-26 10:21:05 +03:00
ef0b0701f3
Simulations nodeid report ( #272 )
...
* Add node id to the node state report
* Update config with record settings example
2023-07-25 17:17:26 +03:00
df97ea2543
Simulatios network throughput ( #270 )
...
* Use step time from configuration
* PayloadSize trait for inmemory network interface
* Per node network capacity
* Initial carnot message sizes
* Divide provided kbps by step time
* Use std::mem::size_of for msg sizes
2023-07-25 16:03:05 +03:00
7d64915dd7
Get current view directly from the engine ( #269 )
2023-07-21 12:30:52 +03:00
8dd34f81b4
Simulation tally fix ( #268 )
...
* Do not remove entry cache when the threshold is reached
* Leader super majority change
* Update leader threshold test
2023-07-21 12:30:32 +03:00
6dee12704d
Change simulation `Node` trait `current_view` method return `View` ( #259 )
2023-07-20 17:31:55 +08:00
0866dfc8af
Process single events ( #267 )
2023-07-20 10:08:09 +02:00
adf935830e
Do not remove entry cache when the threshold is reached ( #266 )
...
Co-authored-by: Gusto <bacvinka@gmail.com>
2023-07-19 18:04:08 +02:00
9f71dbb24c
Simulation network broadcast fix ( #262 )
...
* Replace network broadcast msg type to a dedicated channel
* Update tests with broadcast chan
* Replace threadrng with seedable smallrng
* Simplify the broadcast loop
2023-07-18 15:01:29 +03:00
94e384f609
Tree settings for sim overlay ( #257 )
...
* Tree settings for sim overlay
* WIP dyn node simulation
* Allow generic settings and states for dispatched nodes
* Update tests
2023-07-18 14:44:23 +03:00
9467351c10
Finish `View` wrapper ( #254 )
...
* finish View wrapper
2023-07-12 21:30:22 +08:00
2135676606
Finish `NodeId` type wrapper ( #252 )
...
* add NodeId wrapper
2023-07-11 23:16:49 +08:00
a0cb738b9f
Finish `Committee` wrapper ( #251 )
...
* Committee type wrapper
---------
Co-authored-by: danielsanchezq <sanchez.quiros.daniel@gmail.com>
2023-07-07 16:08:19 +08:00
90cf29bf86
Better tests ( #232 )
...
* add basic integration tests
* add a way to configure overlay threshold
* Save logs to file in case of failure
* Increase number of test nodes to 10
* fix tests
* use fraction instead of tuple
* fmt
2023-07-04 11:39:28 +02:00
98aa138b87
feat: enforce TimeoutQc to be constructed only by new() ( #229 )
2023-06-28 23:37:27 +09:00
224a3a53f5
fix tally_by condition ( #234 )
2023-06-28 17:56:16 +08:00
4a80690c1f
Add a cli arg to support redirect log ( #230 )
...
* add cli arg to support redirect log
2023-06-28 17:10:19 +08:00
55648e3151
Simulation network broadcast ( #225 )
...
* Sim network broadcast method
* Remove list of all nodes in carnot node instance
* Use network broadcast method within nodes
* Add helper method for network payload
2023-06-27 14:20:59 +03:00
18bbf63a3f
update simulation app configuration ( #217 )
2023-06-22 19:50:34 +08:00
4ccc19f5a3
Support configurable records ( #200 )
...
* support configurable records
2023-06-21 16:31:36 +08:00
bed0b9448d
Simulation unhappy path ( #193 )
...
* Use elapsed time
* Added timeout
* Extract tally
* Missing elapsed time
* Fix new view leader behaviour
* Fix tests
* Fix timeout double check
* Fix logs
* TimeoutHandler nitpicks
* Clippy happy
* Fix timeout sub
* Modify discard messages comment
2023-06-19 15:27:14 +02:00
faacd10172
Remove dummy node from simapp settings and app itself ( #196 )
2023-06-17 13:38:59 +03:00
40048fa47b
Fix #183 : support structured log for sim app. ( #184 )
2023-06-16 19:26:48 +08:00
9afd6c007c
Simulation conf polish ( #181 )
...
* polish Duration ser/deser and support step_time in conf file
2023-06-15 17:42:42 +08:00
b6789b994e
Fix #182 : gracefully shutdown ( #187 )
...
* finish signal and cleanup
2023-06-15 17:40:29 +08:00
e57cb7b3cf
remove RwLock from SimmulationRunnerInner ( #189 )
2023-06-15 11:16:28 +02:00
27d9f72035
Simulation happy path ( #161 )
...
* finish subscriber manager
* optimize subscribe on SimulationRunnerHandle
* fix comment
* replace std locks to parking_lot locks
* move producer initialization out simulate fn
* WIP
* optimize run fn
* update condition
* fix CI
* collect run times
* Add happy-path consensus engine
* tmp
* Fit types from spec (#124 )
* Match types to spec
* Remove Output import
* Consensus engine rework (#126 )
* rework
* fix test
* clippy happy
---------
Co-authored-by: Giacomo Pasini <Zeegomo@users.noreply.github.com>
* Adapt carnot network adapter interfaces and implementations
* Fix errors
* Update network with engine types
* Fit types yet again
* Remove leadership and old overlay
Create carnot event builder
Added some adjustments
* Add view to vote
* Fix serde derive in consensus-engine
* Add serde feature for engine in core
* Use view in tally
* Move carnot tally to consensus service
* Add new view msg
* Fit engine types in adapter
* Missing serde feature in consensus service
* Implement carnot event builder
* Implement even builder run main tasks
* Fill up view resolver
* Fix errors on network adapter implementations
* Clippy happy
* Extract event handling to independent methods in View
* Fix test
* Refactor carnot event builder (#135 )
* refactor
* format
* Discriminate proposal messages (#136 )
* Derive block id from wire format (#139 )
* Derive block id from wire format
* Derive id on block creation
* Use compile time hash size
* Add leader role (#138 )
* add leadership stub
* fix gather_new_views
* fmt
* actually build qc
* remove redundant fields
* add flat overlay (#143 )
* add flat overlay
* fix
* sort imports
* fix tests
* Fix waku update
* rewrite data collection add different kind of subscribers
* fix fmt
* Unhappy tally (#137 )
* Refactor tally module
* Implement tally for new view messages
* Assess pr comments
* Fix rebase
* simplify tally
---------
Co-authored-by: Giacomo Pasini <g.pasini98@gmail.com>
* fix gather_new_views
* working node
* fix unhappy path
* remove leftover
* Kickstart event building in sim app
* finish event builder
* fix comment
* add Tally
* gather enough new views then construct ProposalBlock event
* Revert "gather enough new views then construct ProposalBlock event"
This reverts commit 87da2bdd0c#154 )
* Implement leader proposing
* fix fmt
---------
Co-authored-by: al8n <scygliu1@gmail.com>
* fix ser/deser bugs
* fix subscriber bugs
* Fix proposing genesis
* Fix genesis retrieval in consensus-engine
* Bring back general block proposal event
* Fix leaf voting
* fix init node bugs
* add more tracing
* fix empty qc
* fix data race
* fix all panics
* cleanup
* propose new blocks
* fix comment
* do not approve for the same block
* no panics
* fix some comments
* use serde_with
* Bring back genesis on 0
* Fix genesis retrieval
Replace output enum
Vote for genesis proposal
* Genesis methods
* fix StardardQc::genesis()
* fix genesis block bug
* fix PR comment
* fix PR comment
* fix PR comment
* fix PR comment
* fix PR comment
* Fix tally
Fix proposing
* Remove public block building
Added raw method
* Missing fmt
* clippy happy
* fix io stream downcast
* optional stream-type arg, by default we do not run any subscriber
* fmt
* cleanup
* Remove from header block constructor
* Fix duplicated approve (#180 )
* fix duplicated approve
* Success tally just on threshold
* Integrate random beacon on happy path
* Fix missing updating beacon
* Replicate consensus output
* Prune older non relevant messages from cache
* Remove view info just again
* Refactor Block deps
* Reverse wrong parent committee call in Consensus engine
* Remove useless event builder settings
* Remove blocks store from event builder
* Remove unnecessary carnot seed
* Remove duplicated proposals check
---------
Co-authored-by: Giacomo Pasini <g.pasini98@gmail.com>
Co-authored-by: Daniel Sanchez <sanchez.quiros.daniel@gmail.com>
Co-authored-by: Giacomo Pasini <Zeegomo@users.noreply.github.com>
2023-06-14 16:52:37 +02:00
d864fecd07
chore: replace std locks to parking_lot locks in simulations ( #141 )
...
* replace std locks to parking_lot locks
2023-05-31 12:57:42 +08:00
4b880778ab
finish subscriber manager ( #140 )
...
* finish subscriber manager
2023-05-11 16:09:53 +08:00
0bf6e6d272
Simulation initialization ( #122 )
...
* Use enums for different settings types
* Enum for overlay settings
* Configurable simulation overlay
* Use duration type for network behaviour delays
* Configurable simulation nodes
* Runner for different node types
* Seedable rng
* Convert settings to required objects
* Implement IOStreamSettings deserialization
* Use common run method for different node types
* Configuration for simapp
* Testcase for region distribution
* Use unix time if seed is not provided
2023-05-08 13:06:39 +03:00
ea7896f06c
Simulation streaming and gracefully shutdown ( #119 )
...
* add stream supports
* add test case
* add polars stream, and force to use stream for the runner
* using arcswap instead RefCell for producers
* finish gracefully shutdown
* - add IOProducer and IOSubscriber
- fix deadlock in sync runner
- fix testcases
2023-04-25 19:14:30 +08:00
c2d1ee1291
add rustfmt.toml ( #118 )
2023-04-19 15:59:51 +08:00
gusto
Youngjoon Lee
Daniel Sanchez
Al Liu
Giacomo Pasini