1
0
mirror of synced 2025-01-11 08:15:48 +00:00

19 Commits

Author SHA1 Message Date
gusto
d17afcbe4d
Replication behavior membership test (#805) 2024-10-03 20:37:13 +03:00
Daniel Sanchez
9ecd738d6e
DA: Executor network service (#795)
* Refactor common things in backends

* Further extract to common

* Extract dial_peers

* Pipe out executor events

* Add wrapper settings

* Dial up dispersal peers

* Clippy happy

* dial_peers -> dial_validator_subnetwork_peers

* Add members list to membership trait

* Implement peers selection and dialing up from specification

* Fix tests

* Fix tiny doc
2024-10-01 12:44:13 +00:00
Daniel Sanchez
02b6969967
DA: Executor behaviour + swarm (#768)
* Add executor behaviour

* Implement executor behaviour

* Implement executor swarm

* Fix comment

* Clippy happy

* Bubbled up utility methods on swarms
2024-09-26 09:58:49 +00:00
gusto
915fe3f098
DA: Replication integration (#764)
* Handle replication messages

* Multiple indexes for kzgrs verifier

* Add rocksdb storage to node config

* Configurable delay before sending blob info to mempool

* Dial peers in same subnet for replication

* Update da integration tests

* Update nomos node tests

* Load kzgrs config from file

* SLOW_ENV set true for macos CI

* Assert retrieved blobs in dissemination test

* Kzgrs params deserialize unchecked
2024-09-25 11:27:39 +03:00
gusto
9f4f139771
DA: Integration tests for sampling and dispersal (#733)
* Use common test node in da tests

* Pick ranged subnetids when sampling

* Da network settings for integration tests

* Predefined node keys and deterministic rng

* Disperse kzgrs encoded blob

* Cli swarm fixes

* Increase indexer integration test timeout

* Check dispersal responses in cli da adapter

* DA membership configuration in node tests

* Nomos Cli act as a node in tests

* Increase timeout for dispersal tests

* Different node configurations for da tests

* Collect unique ids for sampling
2024-09-18 16:22:36 +03:00
gusto
9835d09b8c
Column index in the sample requests (#731) 2024-09-06 18:39:50 +03:00
gusto
f8a73529cf
DA: Incoming sampling request handling (#730)
* WIP: Sample message to behaviour massage translation

* Sampling request handling

* Sampling storage in node

* Sampling rocksdb scaffold
2024-09-06 16:47:38 +03:00
holisticode
efff80de67
Sampling backend implementation (#709)
* implemented backend; correct start of monitoring thread needs to be finished

* Fix kzgrs backend (#710)

* abstract rng

* fix rng abstraction

* replaced subnets vector with HashSet, fixed bugs, added tests

* addressed PR comments

* fix clippy warnings

* Rename TrackingState -> SamplingState

* Short circuit failure on init error

---------

Co-authored-by: Daniel Sanchez <sanchez.quiros.daniel@gmail.com>
Co-authored-by: danielSanchezQ <3danimanimal@gmail.com>
2024-09-03 08:25:12 +00:00
holisticode
0cb039d806
DA: Sampling service (#705)
* initial

* first iteration, lots of lifetime and trait issues, does not compile

* Daniel/sampling service (#706)

* Implement sampling service

* Implement libp2p adapter listen_to_sampling_messages

* temporary empty backend mod; implement start_sampling

* addressed PR comments; moved random sampling subnet list generation to backend (todo impl)

* addressed PR nitpicks

* remove empty mod which was failing CI due to cargo fmt

* removed unused code

---------

Co-authored-by: holisticode <holistic.computing@gmail.com>
Co-authored-by: Daniel Sanchez <sanchez.quiros.daniel@gmail.com>
2024-08-27 13:42:52 -05:00
Daniel Sanchez
c611319622
Da: Network addresses (#703)
* Crate address book

* Add addressbook on necesary places
Incorporate to sampling test

* fmt happy
2024-08-27 10:15:45 +00:00
gusto
37c22d23ce
DA: Network service executor (#701)
* Nomos cli executor swarm

* Executor swarm and backend in nomos-cli

* Libp2p executor adapter in nomos cli

* Update nomos-cli/src/da/network/backend.rs

Co-authored-by: Daniel Sanchez <sanchez.quiros.daniel@gmail.com>

* Use FillFromNodeList with executor backend

* Integrate Executor backend into the cli

* Use subnetwork id where appropriate

* Common clone_deserialize_error

* Use thiserror

* Minimal dialing in cli swarm

* Send dispersal messages async

* Remove unused connection id

---------

Co-authored-by: Daniel Sanchez <sanchez.quiros.daniel@gmail.com>
2024-08-26 11:00:34 +03:00
Daniel Sanchez
66de571bfa
Da: network service backend (#695)
* Sketch up types and backend skeleton

* Pipe up sampling events

* Pipe up validation blobs events

* Added docs

* Cargo fmt

* Clippy happy

* Debug log events in validator swarm run method
2024-08-22 10:14:42 +00:00
gusto
2ca822e6ec
ColumnIndex in DaBlob (#693)
* ColumnIndex in DaBlob

* Use u16 for column index
2024-08-20 13:15:16 +03:00
Daniel Sanchez
2922e9ce74
Da: validator network behaviour & swarm (#691)
* Build up validator behaviour and swarm

* Refactor modules

* More barebones structure on validator swarm

* Refactor protocols into submodule

* Pipe up events stream into swarm impl

* Clippy happy on tests
2024-08-19 16:17:10 +00:00
Daniel Sanchez
d2c178f33d
Da: Network sampling (#689)
* Fix reading messages state machine

* Improve waker on behaviour

* Rename test

* First sketch

* Implement replication

* Refactor protocols names

* Clippy happy

* expose behaviour

* Return dummy on outbound for validator behaviour

* Implement behaviour, lacks streams handling

* Cleanup

* Implement stream handling

* Add some documentation and comments

* Clippy fixes after rebase

* Move all neighbours to test_utils module

* Executor only outgoing streams

* First assault at testing behaviours

* Add debug instead of traces

* Added more logs

* Bullshitting tests

* Fix outstream handling, offers hook to send new peer streams

* Fix blob dispersal handling

* Refactor test, use new api

* Imports cleanup

* Working test

* Add docs

* Non overlapping test port

* Added sampling protocol

* Sketch up sampling behaviour types

* Implement behaviour

* Added test

* Mid protocol, not working

* Reimplement using streams (just outgoing, incoming missing)

* Implemented with single streams

* Fix unreachable code reachable

* Fix block test

* Clippy happy

* Better docs
2024-08-16 11:12:34 +00:00
Daniel Sanchez
5f0707c276
Da: Network dispersal (#688)
* Fix reading messages state machine

* Improve waker on behaviour

* Rename test

* First sketch

* Implement replication

* Refactor protocols names

* Clippy happy

* expose behaviour

* Return dummy on outbound for validator behaviour

* Implement behaviour, lacks streams handling

* Cleanup

* Implement stream handling

* Add some documentation and comments

* Clippy fixes after rebase

* Move all neighbours to test_utils module

* Executor only outgoing streams

* First assault at testing behaviours

* Add debug instead of traces

* Added more logs

* Bullshitting tests

* Fix outstream handling, offers hook to send new peer streams

* Fix blob dispersal handling

* Refactor test, use new api

* Imports cleanup

* Working test

* Add docs

* Non overlapping test port
2024-08-12 09:12:08 +02:00
Daniel Sanchez
e380bf4117
Da: network replication fix connection (#685)
* Fix attempt 1 - nonworking

* Added missing open stream request

* Expand outbound lifecycle to openstream state

* Fix reading messages state machine

* Use tracing

* Remove unnecessary loop

* Use stream style for test

* Send 10 messages

* Use quic in test

* Sends and receives

* cargo update fixed the panic

* Refactor test

* reschedule polls

* Clippy cleanup

* Remove leftover features

* Cargo fmt

* Improve waker on behaviour

* Rename test
2024-08-02 16:21:38 +00:00
Daniel Sanchez
91d3558cb2
Create basic structure of mempool network adapter for da sampling (#683)
* Create basic structure of mempool network adapter for da sampling

* Return blob instead from sample method

* fmt
2024-07-26 08:49:24 +00:00
Daniel Sanchez
b9efe1f1ba
Da: network replication core (#680)
* Add core structure
Sketch first version of da network handler

* Added assgination module
Added basic assignation trait

* Added assgination module
Added basic assignation trait

* Refuse connection for non members

* Use proper protocol name

* Split in broadcast/dispersal/sampling modules
Move behaviour and handler to broadcast

* Remove unused broadcast behaviour attributes

* Add basic docs

* Remove unused pending errors on broadcast handler

* Clippy happy

* Refactor Broadcast -> Replication

* Pipe message sending

* Clippy happy

* Clippy fmt

* Refactor outgoing -> outgoing_events

* Pipe out replication

* Include subnetworks id in dispersal and replication messages

* Pipe replication messaging

* fmt

* Fix tests
2024-07-25 17:03:32 +00:00