e57cb7b3cf
remove RwLock from SimmulationRunnerInner ( #189 )
2023-06-15 11:16:28 +02:00
27d9f72035
Simulation happy path ( #161 )
...
* finish subscriber manager
* optimize subscribe on SimulationRunnerHandle
* fix comment
* replace std locks to parking_lot locks
* move producer initialization out simulate fn
* WIP
* optimize run fn
* update condition
* fix CI
* collect run times
* Add happy-path consensus engine
* tmp
* Fit types from spec (#124 )
* Match types to spec
* Remove Output import
* Consensus engine rework (#126 )
* rework
* fix test
* clippy happy
---------
Co-authored-by: Giacomo Pasini <Zeegomo@users.noreply.github.com>
* Adapt carnot network adapter interfaces and implementations
* Fix errors
* Update network with engine types
* Fit types yet again
* Remove leadership and old overlay
Create carnot event builder
Added some adjustments
* Add view to vote
* Fix serde derive in consensus-engine
* Add serde feature for engine in core
* Use view in tally
* Move carnot tally to consensus service
* Add new view msg
* Fit engine types in adapter
* Missing serde feature in consensus service
* Implement carnot event builder
* Implement even builder run main tasks
* Fill up view resolver
* Fix errors on network adapter implementations
* Clippy happy
* Extract event handling to independent methods in View
* Fix test
* Refactor carnot event builder (#135 )
* refactor
* format
* Discriminate proposal messages (#136 )
* Derive block id from wire format (#139 )
* Derive block id from wire format
* Derive id on block creation
* Use compile time hash size
* Add leader role (#138 )
* add leadership stub
* fix gather_new_views
* fmt
* actually build qc
* remove redundant fields
* add flat overlay (#143 )
* add flat overlay
* fix
* sort imports
* fix tests
* Fix waku update
* rewrite data collection add different kind of subscribers
* fix fmt
* Unhappy tally (#137 )
* Refactor tally module
* Implement tally for new view messages
* Assess pr comments
* Fix rebase
* simplify tally
---------
Co-authored-by: Giacomo Pasini <g.pasini98@gmail.com>
* fix gather_new_views
* working node
* fix unhappy path
* remove leftover
* Kickstart event building in sim app
* finish event builder
* fix comment
* add Tally
* gather enough new views then construct ProposalBlock event
* Revert "gather enough new views then construct ProposalBlock event"
This reverts commit 87da2bdd0c#154 )
* Implement leader proposing
* fix fmt
---------
Co-authored-by: al8n <scygliu1@gmail.com>
* fix ser/deser bugs
* fix subscriber bugs
* Fix proposing genesis
* Fix genesis retrieval in consensus-engine
* Bring back general block proposal event
* Fix leaf voting
* fix init node bugs
* add more tracing
* fix empty qc
* fix data race
* fix all panics
* cleanup
* propose new blocks
* fix comment
* do not approve for the same block
* no panics
* fix some comments
* use serde_with
* Bring back genesis on 0
* Fix genesis retrieval
Replace output enum
Vote for genesis proposal
* Genesis methods
* fix StardardQc::genesis()
* fix genesis block bug
* fix PR comment
* fix PR comment
* fix PR comment
* fix PR comment
* fix PR comment
* Fix tally
Fix proposing
* Remove public block building
Added raw method
* Missing fmt
* clippy happy
* fix io stream downcast
* optional stream-type arg, by default we do not run any subscriber
* fmt
* cleanup
* Remove from header block constructor
* Fix duplicated approve (#180 )
* fix duplicated approve
* Success tally just on threshold
* Integrate random beacon on happy path
* Fix missing updating beacon
* Replicate consensus output
* Prune older non relevant messages from cache
* Remove view info just again
* Refactor Block deps
* Reverse wrong parent committee call in Consensus engine
* Remove useless event builder settings
* Remove blocks store from event builder
* Remove unnecessary carnot seed
* Remove duplicated proposals check
---------
Co-authored-by: Giacomo Pasini <g.pasini98@gmail.com>
Co-authored-by: Daniel Sanchez <sanchez.quiros.daniel@gmail.com>
Co-authored-by: Giacomo Pasini <Zeegomo@users.noreply.github.com>
2023-06-14 16:52:37 +02:00
75025e8cf0
Consensus service refactor ( #176 )
...
The consensus service was becoming a bit messy and difficult to maintain.
This PR moves handling invididual events to their own function and refactors the cancelabe task system in a separate struct.
2023-06-13 12:11:55 +02:00
Jakub Sokołowski
fe0361a5b8
ci: fix GOCACHE and GOPATH to avoid poisoning ( #125 )
...
We've had this issue before with some `go-waku` builds:
https://github.com/waku-org/go-waku/pull/512
Signed-off-by: Jakub Sokołowski <jakub@status.im>
2023-06-13 11:29:01 +03:00
9c81b72711
Random beacon ( #167 )
...
* move overlay to consensus engine
* Integrate random beacon in overlay
This commit integrates the random beacon as specified in the nomos
spec into the consensus engine library as part of the overlay.
In addition, it separates the overlay part responsible for leader
selection as an independent trait LeaderSelection, so as to share
the overall overlay structure among most constructions.
Furthermore, a leader proof has been added to a block to verify
that the proposer had valid rights to do so.
The current implementation hardcodes the leader selection update
in the consensus service, but we probably want to abstract it away
through something like the adapter pattern we use of other services
in the node.
* Move leader selection update to separate function
* Add generic support for leader selection in consensus service (#170 )
* Add generic support for leader selection in consensus service
* fix
* use settings struct instead of tuple
* fix tests
2023-06-12 15:14:49 +02:00
cfaa7cf772
fix: prevent skipping the current view when calculating latest_committed_block ( #163 )
2023-06-07 19:32:48 +09:00
a055c2524a
fix: prevent skipping one view when proposing a block in unhappy path ( #166 )
2023-06-07 09:56:21 +09:00
d864fecd07
chore: replace std locks to parking_lot locks in simulations ( #141 )
...
* replace std locks to parking_lot locks
2023-05-31 12:57:42 +08:00
2d60ce9921
fix: remove duplicate `tracing_subscriber` init for `nomos-node` bin ( #157 )
2023-05-29 22:40:22 +09:00
6229fc98bc
fix gather_timeout being pending ( #156 )
2023-05-27 09:32:39 +09:00
6c64720e39
Add root timeout handling ( #153 )
...
* Add handling of root timeout
* Use StandardQc instead of Qc for high_qc
Since high_qc is guaranteed to be a standard qc, let's just use
StandardQc as a type in all messages that use it.
2023-05-22 15:23:30 +02:00
bc453b686f
Consensus orchestrator ( #128 )
...
* Add happy-path consensus engine
* tmp
* Fit types from spec (#124 )
* Match types to spec
* Remove Output import
* Consensus engine rework (#126 )
* rework
* fix test
* clippy happy
---------
Co-authored-by: Giacomo Pasini <Zeegomo@users.noreply.github.com>
* Adapt carnot network adapter interfaces and implementations
* Fix errors
* Update network with engine types
* Fit types yet again
* Remove leadership and old overlay
Create carnot event builder
Added some adjustments
* Add view to vote
* Fix serde derive in consensus-engine
* Add serde feature for engine in core
* Use view in tally
* Move carnot tally to consensus service
* Add new view msg
* Fit engine types in adapter
* Missing serde feature in consensus service
* Implement carnot event builder
* Implement even builder run main tasks
* Fill up view resolver
* Fix errors on network adapter implementations
* Clippy happy
* Extract event handling to independent methods in View
* Fix test
* Refactor carnot event builder (#135 )
* refactor
* format
* Discriminate proposal messages (#136 )
* Derive block id from wire format (#139 )
* Derive block id from wire format
* Derive id on block creation
* Use compile time hash size
* Add leader role (#138 )
* add leadership stub
* fix gather_new_views
* fmt
* actually build qc
* remove redundant fields
* add flat overlay (#143 )
* add flat overlay
* fix
* sort imports
* fix tests
* Unhappy tally (#137 )
* Refactor tally module
* Implement tally for new view messages
* Assess pr comments
* Fix rebase
* simplify tally
---------
Co-authored-by: Giacomo Pasini <g.pasini98@gmail.com>
* Working node (#149 )
* fix gather_new_views
* working node
* fix unhappy path
* remove leftover
* fix comments
* update waku (#146 )
* update waku
* Fix waku update
---------
Co-authored-by: danielsanchezq <sanchez.quiros.daniel@gmail.com>
* little fixes
* Consensus tasks cancellation (#147 )
* fix
* Create view cancel and cancel cache
* Attach cancellation to consensus tasks
* Fix view binds
---------
Co-authored-by: Giacomo Pasini <g.pasini98@gmail.com>
---------
Co-authored-by: danielsanchezq <sanchez.quiros.daniel@gmail.com>
* Remove clones on consts
---------
Co-authored-by: Al Liu <scygliu1@gmail.com>
Co-authored-by: Giacomo Pasini <g.pasini98@gmail.com>
Co-authored-by: Giacomo Pasini <Zeegomo@users.noreply.github.com>
2023-05-22 14:56:56 +02:00
fdc22111d3
use nanosecond precision in waku timestamp ( #152 )
2023-05-18 15:19:36 +02:00
bbd313f70a
update waku ( #146 )
...
* update waku
* Fix waku update
---------
Co-authored-by: danielsanchezq <sanchez.quiros.daniel@gmail.com>
2023-05-15 16:21:49 +02:00
4b880778ab
finish subscriber manager ( #140 )
...
* finish subscriber manager
2023-05-11 16:09:53 +08:00
eb6f36782d
fix link flgs ( #144 )
2023-05-08 17:28:10 +02:00
0bf6e6d272
Simulation initialization ( #122 )
...
* Use enums for different settings types
* Enum for overlay settings
* Configurable simulation overlay
* Use duration type for network behaviour delays
* Configurable simulation nodes
* Runner for different node types
* Seedable rng
* Convert settings to required objects
* Implement IOStreamSettings deserialization
* Use common run method for different node types
* Configuration for simapp
* Testcase for region distribution
* Use unix time if seed is not provided
2023-05-08 13:06:39 +03:00
26d10856ff
Adapt carnot network adapter interfaces and implementations to latest spec ( #121 )
...
* Add happy-path consensus engine
* tmp
* Fit types from spec (#124 )
* Match types to spec
* Remove Output import
* Consensus engine rework (#126 )
* rework
* fix test
* clippy happy
---------
Co-authored-by: Giacomo Pasini <Zeegomo@users.noreply.github.com>
* Adapt carnot network adapter interfaces and implementations
* Fix errors
* Update network with engine types
* Fit types yet again
---------
Co-authored-by: Al Liu <scygliu1@gmail.com>
Co-authored-by: Giacomo Pasini <g.pasini98@gmail.com>
Co-authored-by: Giacomo Pasini <Zeegomo@users.noreply.github.com>
2023-05-01 17:47:56 +02:00
f8617d7331
Consensus engine rework ( #127 )
...
---------
Co-authored-by: Giacomo Pasini <Zeegomo@users.noreply.github.com>
---------
Co-authored-by: Al Liu <scygliu1@gmail.com>
Co-authored-by: Daniel Sanchez <sanchez.quiros.daniel@gmail.com>
2023-04-27 19:18:24 +02:00
ea7896f06c
Simulation streaming and gracefully shutdown ( #119 )
...
* add stream supports
* add test case
* add polars stream, and force to use stream for the runner
* using arcswap instead RefCell for producers
* finish gracefully shutdown
* - add IOProducer and IOSubscriber
- fix deadlock in sync runner
- fix testcases
2023-04-25 19:14:30 +08:00
b1381d727f
Update README.md ( #120 )
2023-04-24 11:05:00 +02:00
c2d1ee1291
add rustfmt.toml ( #118 )
2023-04-19 15:59:51 +08:00
2206d7a291
Use par_iter in network module ( #116 )
...
* Use par_iter in network module
* Pass rng from filter function
* Remove locking for network messages
2023-04-13 17:24:27 +03:00
a87418c908
Test voting steps with various node roles ( #114 )
...
* Single step send receive runner tests
* Rename network to overlay state
* Store every overlay that might happen
* Get related nodes for from node id
* Add leader role
* WIP dummy handle events
* Leader and root roles
* Add intent as helper to dummy vote
* Check if has enough votes
* Proposal to vote to new view tests
* Update sync runner tests
* Smallrng in send receive tests
* Send initial vote next view leaders in tests
* Local view struct for node
* Large node number tests
* Fix internal to internal case
2023-04-10 13:18:20 +03:00
a19278480d
Pipe output data ( #111 )
2023-04-04 20:26:57 +08:00
d07da7ba5a
add missing ward ( #112 )
2023-04-03 17:17:14 +08:00
901ebf4152
Message sending between simulation nodes ( #110 )
...
* Dummy node for simulations
* Shared network state for nodes
* Runner one step test
* Beginning of network interface
* Connect dummy node to network
* Network step tests
* Pop messages that are being sent
* Regions send receive tests
* Setup network in sync runner tests
* Dispatch and collect node messages during sim step
* Improve network interface receiver
2023-03-31 12:48:06 +03:00
c882a58286
Add StalledViewWard ( #109 )
2023-03-30 16:35:29 +08:00
61356f1116
Add minmax warding ( #108 )
...
* add minmax warding
2023-03-28 00:20:01 +08:00
f4b94c8267
Simple warding fix ( #107 )
...
* fix negated analyze
2023-03-27 19:16:48 +08:00
ae16e8583b
impl ward condition for max view ward ( #106 )
...
* impl ward condition for max view ward
2023-03-27 18:45:50 +08:00
92ef9e5a77
Simulation app runners ( #105 )
...
* Integrate new runners, fit types...make it compile
* Added missing runners
* Fix rebased changes
* Make tests pass
* Clippy happy
* More clippy happy
2023-03-24 13:21:10 -07:00
4a90ba6926
chore: types wrapper ( #104 )
...
* chore: types wrapper
2023-03-24 19:50:30 +08:00
ea589d018a
Simulation App: Make step cost configurable ( #102 )
...
* WIP: make step cost configurable
* fix fmt
2023-03-23 16:57:18 +08:00
2d8c9f1099
Tree overlay for simulation app ( #101 )
...
* Add simulations crate
* Firs working runner
* Cleanupt
* Extract overlay from runner
* Fix off id calls
* Fix testing values
* WIP: Tree overlay for simulation app
* Build tree overlay from depth and size
* Tree overlay generation tests
* WIP: Committee role in layout
* Use commtittee role when constructing node
* Pairprogramming role calculation
* WIP: Extract step runner to node
* Aggregate times from different overlay layers
* Runner for overlay with layers
* Use StepRng in runner tests
* Decouple role from node
* Define leader steps and behaviour
* Get possible node ids from the overlay
* Handle unknown state in flat overlay
* Use solver type for deserialization
* Remove refcell from nodes hashmap
* Create layout from the provided node ids
* Break runner loops into seperate functions
* Use for loop to collect node step times
* Revert leader times collection
* Add interegion latency tests
---------
Co-authored-by: danielsanchezq <sanchez.quiros.daniel@gmail.com>
2023-03-23 01:32:54 -07:00
8a1af8c234
Add configuration for simulation app ( #94 )
...
* make simulation app compile to wasm
* add configuration
* improve args parsing
* add steps in configuration
---------
Co-authored-by: Daniel Sanchez Quiros <sanchez.quiros.daniel@gmail.com>
2023-03-20 17:13:55 +08:00
1ec4231a7a
Add Qc to block ( #99 )
...
* Added Qc type to Tally trait
* Add generic Qc to block header
* Use blockid instead of unnecessary header
* Expand Qc over generics
* Build up block with proper qc header
2023-03-17 06:23:50 -07:00
91ce4e6fa1
Make a transaction trait ( #98 )
...
* Impl Transaction trait
* Impl Transaction for MockTransaction
* <ake carnot transaction a module
* Refactor consensus to use Transaction
* Fix tests
* Constrain Transaction::Hash
* Refactor redundant Carnot in CarnotTx
2023-03-16 22:42:56 -07:00
780276497f
Fix endless loop on mocktally
2023-03-15 10:02:29 +01:00
818d7f29cd
Add generic block ( #93 )
...
* add generic block
* fix PR comments
* Block uses tx hashes
* Refactor bounds and generics to accept block type
* remove Tx generics
* add generic for block
* Remove unnecessary bounds on leadership
* Impl from with ownership for mock tx and txid
* feature gate
---------
Co-authored-by: danielsanchezq <sanchez.quiros.daniel@gmail.com>
2023-03-14 09:55:08 -07:00
6ce9fdf553
[WIP] Simulation app barebones ( #90 )
...
* Add simulations crate
* Kickstart modules
* Add overlay
* Use node steps
* Add initial carnot node
* Implement Carnot node costs simulation
* Firs working runner
* Cleanup
* Extract overlay from runner
* Fix off id calls
* Fix testing values
* Clippy happy
* make simulation app compile to wasm
---------
Co-authored-by: al8n <scygliu1@gmail.com>
2023-03-14 07:15:45 -07:00
274e8d55fd
Voting core primitive ( #82 )
...
* Add tally trait
Implement mock tally
* Add tally to consensus
* Use tally in implemented overlays
* Clippy happy
* Scratch carnot voting as per specification
* Add missing derives
* Clippy happy
* Fix tests
* Add checks on valid votes
* Refactor ApprovalMsg to VoteMsg
* Remove no vote in MockVote and MockQC
* Remove no vote in MockVote and MockQC
* Remove timeout todo
* Fix tests
2023-03-14 03:32:36 -07:00
ae5c5b9d4c
Update rust docker image for CI ( #91 )
...
* Update rust docker image for CI
* Use macos aarch64 until x86 issues are solved
2023-03-07 21:19:38 +02:00
4981c724af
Http status codes ( #88 )
...
* Include status codes in http error responses
* Mockpool bridges error handling
* Last TX in milliseconds in mempool metrics
* u64 for last tx metrics
2023-03-06 15:19:27 +02:00
42ea8f9be3
Update waku-bindings to 0.1.0-rc.2 ( #86 )
2023-03-06 15:18:27 +02:00
c6bc35a931
Make carnot overlay generic ( #87 )
...
* Make carnot overlay generic
* support generic
* add back 'view for View fns
* add assertion on view number
* remove unused comments
* fix fmt
---------
Co-authored-by: al8n <scygliu1@gmail.com>
2023-03-03 00:23:51 +08:00
dfe17696e6
Fix Nix derivation so it works in a Darwin Nix environment (explicitly requiring clang and setting env variable) ( #84 )
2023-02-24 11:08:46 +02:00
55aece674b
Update to waku rc 1 ( #80 )
2023-02-22 15:55:59 +01:00
7f609db62a
Improve Mock network ( #78 )
...
* add MockTxId and send back transaction response messages
2023-02-21 16:10:26 +08:00
7f7a0db88a
Update rust and go versions for macos builds ( #77 )
...
* Add additional libs for macos nix env
* Add CoreFoundation and Security for macos
* Update rust 1.67.0 and go 1.19.5
2023-02-16 17:42:31 +02:00
Al Liu
Giacomo Pasini
Jakub Sokołowski
Youngjoon Lee
Daniel Sanchez
gusto
Álvaro Castro-Castilla