1
0
mirror of synced 2025-02-19 11:17:08 +00:00

81 Commits

Author SHA1 Message Date
Roman
54e6939141
fix: have executors and validators in the same subnet_id = 0 2024-10-23 18:26:28 +08:00
Roman
3eeafc819c
fix: address to contain p2p address 2024-10-23 15:44:11 +08:00
Roman Zajic
52459f3b8e
Merge branch 'master' into chore-da-behavior-membership-tests 2024-10-23 07:57:58 +08:00
Daniel Sanchez
5a85281ec0
Use tokio 1 in deps (#842) 2024-10-22 16:59:42 +02:00
Giacomo Pasini
ad88d2bc0c
Add first tx defition (#835)
* Add first tx defition

* address review comments

* review comments
2024-10-22 15:51:54 +02:00
Roman Zajic
8e75709edb
Merge branch 'master' into chore-da-behavior-membership-tests 2024-10-22 20:09:10 +08:00
Roman
6861dc18a6
test: executor > validator auto connect 2024-10-22 20:08:36 +08:00
Deep Mondal
b0142d7173
Fix: membership comments to documentation (#838)
* rebased

* doc comment add

* fmt
2024-10-22 13:13:29 +02:00
Roman
1a7d01f032
test: send messages using swarms 2024-10-22 14:25:11 +08:00
Roman Zajic
6293837890
Merge branch 'master' into chore-da-behavior-membership-tests 2024-10-22 09:20:51 +08:00
gusto
328398ca68
DA: Use executor in tests (#818)
* Use executor in tests instead of nomos node

* Executor config from node config

* Bring generics to the testing game

* Fill in missing gaps in test

* Implement testnode wrapper

* Use sleep on dispersal service instead

* Fix cfgsync

* Clippy happy

* Clippy happy tests

* Mixnet config in tests for validator

* Tests: General config and multiple nodes (#832)

* Use executor in tests instead of nomos node

* Bring generics to the testing game

* Fill in missing gaps in test

* Clippy happy

* Mixnet config in tests for validator

* Derive different types of configs from general in tests

* Validator and executor in cfgsync

---------

Co-authored-by: danielSanchezQ <3danimanimal@gmail.com>

* Tests executor node mix config (#834)

* Merge branch 'master' into tests-executor-node-mix-config

* add mix configs

---------

Co-authored-by: danielSanchezQ <3danimanimal@gmail.com>
Co-authored-by: Youngjoon Lee <5462944+youngjoon-lee@users.noreply.github.com>
2024-10-21 15:38:56 +03:00
Roman Zajic
6f13b78335
Merge branch 'master' into chore-da-behavior-membership-tests 2024-10-21 13:12:49 +08:00
Roman
d18144d4af
fix: formatting 2024-10-21 12:33:44 +08:00
Roman
ac627a3885
test: create swarms 2024-10-21 11:23:06 +08:00
Youngjoon Lee
163cc2ce64
chore: clippy: remove irrefutable let...else patterns in DA (#830) 2024-10-20 01:49:48 +09:00
Roman Zajic
e0e98f6de6
Merge branch 'master' into chore-da-behavior-membership-tests 2024-10-18 14:06:04 +08:00
Roman
cb4681de42
test: plug in behaviours into swarms 2024-10-18 14:03:44 +08:00
Daniel Sanchez
41a9387b4b
DA: Executor dispersal auto stream (#826)
* Re-structure connection handling properly

* Handle event based connections

* Fix test

* Handling auto reconnection for dispersal behaviour

* Ignore cli test
2024-10-17 14:10:28 +00:00
Roman
81e571d434
test: extend behavior with channels 2024-10-17 20:05:54 +08:00
Roman
c0b3cb1b95
test: attempt to open stream between two peers without swarm 2024-10-17 15:36:43 +08:00
Roman
7d21571e55
test: simple basics to understand 2024-10-16 09:28:34 +08:00
Roman
a2c54231b3
fix: members for MockMembershipHandler 2024-10-14 14:57:29 +08:00
Roman
3995e4a4cc
test: validation behaviour establish_connection 2024-10-14 13:49:16 +08:00
Daniel Sanchez
11ddb23795
Ci: kzg bench update (#813)
* Update kzg benches

* Add bench check to CI
2024-10-08 16:39:38 +00:00
gusto
22dfb51eba
Executor: Dispersal service mempool adapter (#810)
* Pass metadata when dispersing

* Sketch of executor dispersal mempool adapter
2024-10-08 11:03:34 +03:00
Daniel Sanchez
a3cea9a09c
DA: Implement data reconstruction from original columns (#808)
* Implement data reconstruction from original columns

* Make reconstruction public
2024-10-08 07:15:41 +00:00
Daniel Sanchez
98e9cde66d
DA: Executor dispersal service (#806)
* Added basic dispersal structure

* Expand impl

* Pull events from network service

* Implement network adapter
Tie types together in service

* Fill up service processing

* Tie up types in kzgrs backend implementation

* Implement disperse

* Pipe encode and dispersal

* Create mempool adapter

* Add mempool adapter

* Tiny comment
2024-10-04 13:06:42 +00:00
gusto
d17afcbe4d
Replication behavior membership test (#805) 2024-10-03 20:37:13 +03:00
Caio
93de3afaf9
chose: fk20 benches clippy happy (#800) 2024-10-03 04:43:46 +00:00
Deep Mondal
563bf6f948
DA: kzg main testing parallelize proofs checks (#803) 2024-10-03 04:37:53 +00:00
Daniel Sanchez
9ecd738d6e
DA: Executor network service (#795)
* Refactor common things in backends

* Further extract to common

* Extract dial_peers

* Pipe out executor events

* Add wrapper settings

* Dial up dispersal peers

* Clippy happy

* dial_peers -> dial_validator_subnetwork_peers

* Add members list to membership trait

* Implement peers selection and dialing up from specification

* Fix tests

* Fix tiny doc
2024-10-01 12:44:13 +00:00
Daniel Sanchez
02b6969967
DA: Executor behaviour + swarm (#768)
* Add executor behaviour

* Implement executor behaviour

* Implement executor swarm

* Fix comment

* Clippy happy

* Bubbled up utility methods on swarms
2024-09-26 09:58:49 +00:00
gusto
915fe3f098
DA: Replication integration (#764)
* Handle replication messages

* Multiple indexes for kzgrs verifier

* Add rocksdb storage to node config

* Configurable delay before sending blob info to mempool

* Dial peers in same subnet for replication

* Update da integration tests

* Update nomos node tests

* Load kzgrs config from file

* SLOW_ENV set true for macos CI

* Assert retrieved blobs in dissemination test

* Kzgrs params deserialize unchecked
2024-09-25 11:27:39 +03:00
gusto
9f4f139771
DA: Integration tests for sampling and dispersal (#733)
* Use common test node in da tests

* Pick ranged subnetids when sampling

* Da network settings for integration tests

* Predefined node keys and deterministic rng

* Disperse kzgrs encoded blob

* Cli swarm fixes

* Increase indexer integration test timeout

* Check dispersal responses in cli da adapter

* DA membership configuration in node tests

* Nomos Cli act as a node in tests

* Increase timeout for dispersal tests

* Different node configurations for da tests

* Collect unique ids for sampling
2024-09-18 16:22:36 +03:00
Youngjoon Lee
8a1e705792
DA: Fix compliation errors in KZG backend bench (#737) 2024-09-17 18:32:11 +09:00
gusto
9835d09b8c
Column index in the sample requests (#731) 2024-09-06 18:39:50 +03:00
gusto
f8a73529cf
DA: Incoming sampling request handling (#730)
* WIP: Sample message to behaviour massage translation

* Sampling request handling

* Sampling storage in node

* Sampling rocksdb scaffold
2024-09-06 16:47:38 +03:00
Daniel Sanchez
a13f8611ee
DA: Consensus sampling (#708)
* Add sampling relay to consensus and massage all generics

* Pipe in sampling filtering of blob info

* Add mark in block

* Pipe validate block

* Refactor mark_in_block -> mark_complete

* Fix generics on tests

* Fix generics on tests

* Fix rebase

* Cargo fmt after rebase

* Sampling service configuration

* Sampling service config in indexer integration tests

---------

Co-authored-by: Gusto <bacvinka@gmail.com>
2024-09-03 15:02:49 +00:00
Roman Zajic
c4c5eba642
chore: Da kzgrs-backend unit tests update (#700)
* test: verify column error cases

* test: verify chunks error cases

* test: simplify vec modification

* test: check column_id getter

* fix: remove check column_id getter
- related function will be removed
2024-09-03 18:56:20 +08:00
holisticode
efff80de67
Sampling backend implementation (#709)
* implemented backend; correct start of monitoring thread needs to be finished

* Fix kzgrs backend (#710)

* abstract rng

* fix rng abstraction

* replaced subnets vector with HashSet, fixed bugs, added tests

* addressed PR comments

* fix clippy warnings

* Rename TrackingState -> SamplingState

* Short circuit failure on init error

---------

Co-authored-by: Daniel Sanchez <sanchez.quiros.daniel@gmail.com>
Co-authored-by: danielSanchezQ <3danimanimal@gmail.com>
2024-09-03 08:25:12 +00:00
holisticode
0cb039d806
DA: Sampling service (#705)
* initial

* first iteration, lots of lifetime and trait issues, does not compile

* Daniel/sampling service (#706)

* Implement sampling service

* Implement libp2p adapter listen_to_sampling_messages

* temporary empty backend mod; implement start_sampling

* addressed PR comments; moved random sampling subnet list generation to backend (todo impl)

* addressed PR nitpicks

* remove empty mod which was failing CI due to cargo fmt

* removed unused code

---------

Co-authored-by: holisticode <holistic.computing@gmail.com>
Co-authored-by: Daniel Sanchez <sanchez.quiros.daniel@gmail.com>
2024-08-27 13:42:52 -05:00
gusto
d5ceceff9e
DA Network in node (#704)
* DA Network in node

* Use ed25519 curve for peerIds in libp2p
2024-08-27 16:47:14 +03:00
Daniel Sanchez
c611319622
Da: Network addresses (#703)
* Crate address book

* Add addressbook on necesary places
Incorporate to sampling test

* fmt happy
2024-08-27 10:15:45 +00:00
Daniel Sanchez
2f6d265aa6
Implement filler with original side replication (#702) 2024-08-26 10:20:47 +02:00
gusto
37c22d23ce
DA: Network service executor (#701)
* Nomos cli executor swarm

* Executor swarm and backend in nomos-cli

* Libp2p executor adapter in nomos cli

* Update nomos-cli/src/da/network/backend.rs

Co-authored-by: Daniel Sanchez <sanchez.quiros.daniel@gmail.com>

* Use FillFromNodeList with executor backend

* Integrate Executor backend into the cli

* Use subnetwork id where appropriate

* Common clone_deserialize_error

* Use thiserror

* Minimal dialing in cli swarm

* Send dispersal messages async

* Remove unused connection id

---------

Co-authored-by: Daniel Sanchez <sanchez.quiros.daniel@gmail.com>
2024-08-26 11:00:34 +03:00
Daniel Sanchez
fcb903c14a
Implement filler v1 (#698) 2024-08-23 08:56:10 +00:00
Daniel Sanchez
66de571bfa
Da: network service backend (#695)
* Sketch up types and backend skeleton

* Pipe up sampling events

* Pipe up validation blobs events

* Added docs

* Cargo fmt

* Clippy happy

* Debug log events in validator swarm run method
2024-08-22 10:14:42 +00:00
gusto
3671376691
DA: Store and load multiple columns for the same blob in a single node (#694)
* ColumnIndex type in Blob trait

* Store blob id and column after verification

* Use blob_id+column_idx storage scheme in indexer

* Store and load multiple columns for the same blob

* Add comments
2024-08-21 16:23:36 +03:00
gusto
2ca822e6ec
ColumnIndex in DaBlob (#693)
* ColumnIndex in DaBlob

* Use u16 for column index
2024-08-20 13:15:16 +03:00
Daniel Sanchez
2922e9ce74
Da: validator network behaviour & swarm (#691)
* Build up validator behaviour and swarm

* Refactor modules

* More barebones structure on validator swarm

* Refactor protocols into submodule

* Pipe up events stream into swarm impl

* Clippy happy on tests
2024-08-19 16:17:10 +00:00