44367a15a5
Sim update overlay ( #298 )
...
* Update sim engine on timeout qc
* Remove rebuild method
2023-08-10 13:18:17 +02:00
ef5935d572
Simulation prune tally ( #299 )
...
* Add pruning methods
* Call pruning on node step method
* Fix tally retain closure
2023-08-10 13:12:48 +02:00
da60d8fc95
Parallel node init ( #300 )
2023-08-10 12:56:02 +03:00
d675585a0f
Test for partial message sending in simulation ( #294 )
...
* Test for partial message sending
* Test correctness, typos
* Fix the node capacity flushing
* Not process double timeout qcs in simulations
* Discard older view messages in simulations messages
* Refactor committed_blocks to latest_committed_blocks
* Remove tally default
* Fix condition to root committee parenting
* Bring back pruning
* Clippy happy
---------
Co-authored-by: danielsanchezq <sanchez.quiros.daniel@gmail.com>
2023-08-08 18:00:08 +03:00
4bdc3ed15a
Add update committees structure for overlay ( #295 )
...
* Move roundrobin to leadership module
* Use references in leader selection
* Add membership traits to overlay and create membership module
* Implement committee membership for random beacon state
* Update flat overlay
* Create updateable membership traits and impls
* Update tree overlay
* Update overlay on consensus service
* Update overlay on simulations nomos node
* Update types on tests and modules
* Use chacha for shuffling
* Change to mut slice instead of inner cloning
* Use fisher yates shuffle from scratch
* Stylish and clippy happy
2023-08-08 10:34:02 +02:00
c16b794517
fix large msg sending logic ( #274 )
...
* fix large msg sending logic
2023-08-03 20:05:43 +08:00
fac42cd31d
Prune blocks in consensus engine ( #275 )
...
* Add prune older blocks call
* Prune older blocks in simulation app
* Use latest safe view for failure case
* Pick parent view from high qc
* Add safety assertion on pruning method
2023-08-01 16:47:47 +02:00
ae59be5466
Refactor parent committee methods to return optional parent instead of empty one ( #282 )
2023-08-01 09:19:29 +02:00
976b1f9577
Add node step time to report ( #271 )
...
* Skip step if previous node exeeded step time
* Fix clippy
* Passively keep track of step execution time in node
2023-07-26 10:21:05 +03:00
ef0b0701f3
Simulations nodeid report ( #272 )
...
* Add node id to the node state report
* Update config with record settings example
2023-07-25 17:17:26 +03:00
df97ea2543
Simulatios network throughput ( #270 )
...
* Use step time from configuration
* PayloadSize trait for inmemory network interface
* Per node network capacity
* Initial carnot message sizes
* Divide provided kbps by step time
* Use std::mem::size_of for msg sizes
2023-07-25 16:03:05 +03:00
7d64915dd7
Get current view directly from the engine ( #269 )
2023-07-21 12:30:52 +03:00
8dd34f81b4
Simulation tally fix ( #268 )
...
* Do not remove entry cache when the threshold is reached
* Leader super majority change
* Update leader threshold test
2023-07-21 12:30:32 +03:00
6dee12704d
Change simulation `Node` trait `current_view` method return `View` ( #259 )
2023-07-20 17:31:55 +08:00
0866dfc8af
Process single events ( #267 )
2023-07-20 10:08:09 +02:00
adf935830e
Do not remove entry cache when the threshold is reached ( #266 )
...
Co-authored-by: Gusto <bacvinka@gmail.com>
2023-07-19 18:04:08 +02:00
9f71dbb24c
Simulation network broadcast fix ( #262 )
...
* Replace network broadcast msg type to a dedicated channel
* Update tests with broadcast chan
* Replace threadrng with seedable smallrng
* Simplify the broadcast loop
2023-07-18 15:01:29 +03:00
94e384f609
Tree settings for sim overlay ( #257 )
...
* Tree settings for sim overlay
* WIP dyn node simulation
* Allow generic settings and states for dispatched nodes
* Update tests
2023-07-18 14:44:23 +03:00
9467351c10
Finish `View` wrapper ( #254 )
...
* finish View wrapper
2023-07-12 21:30:22 +08:00
2135676606
Finish `NodeId` type wrapper ( #252 )
...
* add NodeId wrapper
2023-07-11 23:16:49 +08:00
a0cb738b9f
Finish `Committee` wrapper ( #251 )
...
* Committee type wrapper
---------
Co-authored-by: danielsanchezq <sanchez.quiros.daniel@gmail.com>
2023-07-07 16:08:19 +08:00
90cf29bf86
Better tests ( #232 )
...
* add basic integration tests
* add a way to configure overlay threshold
* Save logs to file in case of failure
* Increase number of test nodes to 10
* fix tests
* use fraction instead of tuple
* fmt
2023-07-04 11:39:28 +02:00
98aa138b87
feat: enforce TimeoutQc to be constructed only by new() ( #229 )
2023-06-28 23:37:27 +09:00
224a3a53f5
fix tally_by condition ( #234 )
2023-06-28 17:56:16 +08:00
4a80690c1f
Add a cli arg to support redirect log ( #230 )
...
* add cli arg to support redirect log
2023-06-28 17:10:19 +08:00
55648e3151
Simulation network broadcast ( #225 )
...
* Sim network broadcast method
* Remove list of all nodes in carnot node instance
* Use network broadcast method within nodes
* Add helper method for network payload
2023-06-27 14:20:59 +03:00
18bbf63a3f
update simulation app configuration ( #217 )
2023-06-22 19:50:34 +08:00
4ccc19f5a3
Support configurable records ( #200 )
...
* support configurable records
2023-06-21 16:31:36 +08:00
bed0b9448d
Simulation unhappy path ( #193 )
...
* Use elapsed time
* Added timeout
* Extract tally
* Missing elapsed time
* Fix new view leader behaviour
* Fix tests
* Fix timeout double check
* Fix logs
* TimeoutHandler nitpicks
* Clippy happy
* Fix timeout sub
* Modify discard messages comment
2023-06-19 15:27:14 +02:00
faacd10172
Remove dummy node from simapp settings and app itself ( #196 )
2023-06-17 13:38:59 +03:00
40048fa47b
Fix #183 : support structured log for sim app. ( #184 )
2023-06-16 19:26:48 +08:00
9afd6c007c
Simulation conf polish ( #181 )
...
* polish Duration ser/deser and support step_time in conf file
2023-06-15 17:42:42 +08:00
b6789b994e
Fix #182 : gracefully shutdown ( #187 )
...
* finish signal and cleanup
2023-06-15 17:40:29 +08:00
e57cb7b3cf
remove RwLock from SimmulationRunnerInner ( #189 )
2023-06-15 11:16:28 +02:00
27d9f72035
Simulation happy path ( #161 )
...
* finish subscriber manager
* optimize subscribe on SimulationRunnerHandle
* fix comment
* replace std locks to parking_lot locks
* move producer initialization out simulate fn
* WIP
* optimize run fn
* update condition
* fix CI
* collect run times
* Add happy-path consensus engine
* tmp
* Fit types from spec (#124 )
* Match types to spec
* Remove Output import
* Consensus engine rework (#126 )
* rework
* fix test
* clippy happy
---------
Co-authored-by: Giacomo Pasini <Zeegomo@users.noreply.github.com>
* Adapt carnot network adapter interfaces and implementations
* Fix errors
* Update network with engine types
* Fit types yet again
* Remove leadership and old overlay
Create carnot event builder
Added some adjustments
* Add view to vote
* Fix serde derive in consensus-engine
* Add serde feature for engine in core
* Use view in tally
* Move carnot tally to consensus service
* Add new view msg
* Fit engine types in adapter
* Missing serde feature in consensus service
* Implement carnot event builder
* Implement even builder run main tasks
* Fill up view resolver
* Fix errors on network adapter implementations
* Clippy happy
* Extract event handling to independent methods in View
* Fix test
* Refactor carnot event builder (#135 )
* refactor
* format
* Discriminate proposal messages (#136 )
* Derive block id from wire format (#139 )
* Derive block id from wire format
* Derive id on block creation
* Use compile time hash size
* Add leader role (#138 )
* add leadership stub
* fix gather_new_views
* fmt
* actually build qc
* remove redundant fields
* add flat overlay (#143 )
* add flat overlay
* fix
* sort imports
* fix tests
* Fix waku update
* rewrite data collection add different kind of subscribers
* fix fmt
* Unhappy tally (#137 )
* Refactor tally module
* Implement tally for new view messages
* Assess pr comments
* Fix rebase
* simplify tally
---------
Co-authored-by: Giacomo Pasini <g.pasini98@gmail.com>
* fix gather_new_views
* working node
* fix unhappy path
* remove leftover
* Kickstart event building in sim app
* finish event builder
* fix comment
* add Tally
* gather enough new views then construct ProposalBlock event
* Revert "gather enough new views then construct ProposalBlock event"
This reverts commit 87da2bdd0c#154 )
* Implement leader proposing
* fix fmt
---------
Co-authored-by: al8n <scygliu1@gmail.com>
* fix ser/deser bugs
* fix subscriber bugs
* Fix proposing genesis
* Fix genesis retrieval in consensus-engine
* Bring back general block proposal event
* Fix leaf voting
* fix init node bugs
* add more tracing
* fix empty qc
* fix data race
* fix all panics
* cleanup
* propose new blocks
* fix comment
* do not approve for the same block
* no panics
* fix some comments
* use serde_with
* Bring back genesis on 0
* Fix genesis retrieval
Replace output enum
Vote for genesis proposal
* Genesis methods
* fix StardardQc::genesis()
* fix genesis block bug
* fix PR comment
* fix PR comment
* fix PR comment
* fix PR comment
* fix PR comment
* Fix tally
Fix proposing
* Remove public block building
Added raw method
* Missing fmt
* clippy happy
* fix io stream downcast
* optional stream-type arg, by default we do not run any subscriber
* fmt
* cleanup
* Remove from header block constructor
* Fix duplicated approve (#180 )
* fix duplicated approve
* Success tally just on threshold
* Integrate random beacon on happy path
* Fix missing updating beacon
* Replicate consensus output
* Prune older non relevant messages from cache
* Remove view info just again
* Refactor Block deps
* Reverse wrong parent committee call in Consensus engine
* Remove useless event builder settings
* Remove blocks store from event builder
* Remove unnecessary carnot seed
* Remove duplicated proposals check
---------
Co-authored-by: Giacomo Pasini <g.pasini98@gmail.com>
Co-authored-by: Daniel Sanchez <sanchez.quiros.daniel@gmail.com>
Co-authored-by: Giacomo Pasini <Zeegomo@users.noreply.github.com>
2023-06-14 16:52:37 +02:00
d864fecd07
chore: replace std locks to parking_lot locks in simulations ( #141 )
...
* replace std locks to parking_lot locks
2023-05-31 12:57:42 +08:00
4b880778ab
finish subscriber manager ( #140 )
...
* finish subscriber manager
2023-05-11 16:09:53 +08:00
0bf6e6d272
Simulation initialization ( #122 )
...
* Use enums for different settings types
* Enum for overlay settings
* Configurable simulation overlay
* Use duration type for network behaviour delays
* Configurable simulation nodes
* Runner for different node types
* Seedable rng
* Convert settings to required objects
* Implement IOStreamSettings deserialization
* Use common run method for different node types
* Configuration for simapp
* Testcase for region distribution
* Use unix time if seed is not provided
2023-05-08 13:06:39 +03:00
ea7896f06c
Simulation streaming and gracefully shutdown ( #119 )
...
* add stream supports
* add test case
* add polars stream, and force to use stream for the runner
* using arcswap instead RefCell for producers
* finish gracefully shutdown
* - add IOProducer and IOSubscriber
- fix deadlock in sync runner
- fix testcases
2023-04-25 19:14:30 +08:00
c2d1ee1291
add rustfmt.toml ( #118 )
2023-04-19 15:59:51 +08:00
2206d7a291
Use par_iter in network module ( #116 )
...
* Use par_iter in network module
* Pass rng from filter function
* Remove locking for network messages
2023-04-13 17:24:27 +03:00
a87418c908
Test voting steps with various node roles ( #114 )
...
* Single step send receive runner tests
* Rename network to overlay state
* Store every overlay that might happen
* Get related nodes for from node id
* Add leader role
* WIP dummy handle events
* Leader and root roles
* Add intent as helper to dummy vote
* Check if has enough votes
* Proposal to vote to new view tests
* Update sync runner tests
* Smallrng in send receive tests
* Send initial vote next view leaders in tests
* Local view struct for node
* Large node number tests
* Fix internal to internal case
2023-04-10 13:18:20 +03:00
a19278480d
Pipe output data ( #111 )
2023-04-04 20:26:57 +08:00
d07da7ba5a
add missing ward ( #112 )
2023-04-03 17:17:14 +08:00
901ebf4152
Message sending between simulation nodes ( #110 )
...
* Dummy node for simulations
* Shared network state for nodes
* Runner one step test
* Beginning of network interface
* Connect dummy node to network
* Network step tests
* Pop messages that are being sent
* Regions send receive tests
* Setup network in sync runner tests
* Dispatch and collect node messages during sim step
* Improve network interface receiver
2023-03-31 12:48:06 +03:00
c882a58286
Add StalledViewWard ( #109 )
2023-03-30 16:35:29 +08:00
61356f1116
Add minmax warding ( #108 )
...
* add minmax warding
2023-03-28 00:20:01 +08:00
f4b94c8267
Simple warding fix ( #107 )
...
* fix negated analyze
2023-03-27 19:16:48 +08:00
ae16e8583b
impl ward condition for max view ward ( #106 )
...
* impl ward condition for max view ward
2023-03-27 18:45:50 +08:00
92ef9e5a77
Simulation app runners ( #105 )
...
* Integrate new runners, fit types...make it compile
* Added missing runners
* Fix rebased changes
* Make tests pass
* Clippy happy
* More clippy happy
2023-03-24 13:21:10 -07:00
Daniel Sanchez
gusto
Al Liu
Giacomo Pasini
Youngjoon Lee