Commit Graph

36 Commits

Author SHA1 Message Date
gusto b341a11eb2
Metrics to nomos services (#623)
* Remove unused metrics crate from services

* Move prometheus metrics to nomos-services
2024-03-22 14:03:35 +02:00
gusto de6138fc27
Remove old http api implementation (#622) 2024-03-22 12:55:25 +02:00
Giacomo Pasini 1b925d9a3a
Add cryptarchia consensus service (#612)
* add cryptarchia consensus service

* fmt

* clippy happy

* review comments
2024-03-15 15:53:51 +01:00
Youngjoon Lee e7d591b7bc
Mixnet v1 (#569)
* base

* Remove mixnet client from libp2p network backend (#572)

* Mixnet v1: Remove all mixnet legacies: mixnet crate, mixnode binary, tests, and docker (#573)

* Mixnet v1: Skeleton (#570)

* Use QUIC for libp2p (#580)

* Add Poisson interval function for Mixnet (#575)

* Mixnet network backend skeleton (#586)

* Libp2p stream read/write (#587)

* Emitting packets from mixclient using libp2p stream (#588)

* Handle outputs from mixnode using libp2p stream/gossipsub (#589)

* Refactor poisson (#590)

* Mix client Poisson emission (#591)

* Mix node packet handling (#592)

* Mix Packet / Fragment logic (#593)

* Move FisherYates to `nomos-utils` (#594)

* Mixnet topology (#595)

* Mix client/node unit tests (#596)

* change multiaddr from tcp to udp with quic-v1 (#607)

---------

Co-authored-by: Al Liu <scygliu1@gmail.com>
2024-03-12 15:47:35 +09:00
Giacomo Pasini e51865fe33
move ledger into separate crate (#606)
* move ledger into separate crate

* clippy happy
2024-03-11 15:15:17 +01:00
Giacomo Pasini 2730c2f579
Cryptarchia engine (#583)
* add cryptarchia engine

* address comments

* split into separate files

* clarify comment
2024-02-29 10:51:25 +01:00
gusto 46d53479a2
Prometheus metrics service (#522)
* A wrapper crate for prometheus client

* Initial integration of metrics for mempool

* Merge mempool metrics imports

* Add cli flag to enable metrics

* Add nomos metrics service for serving metrics

* Use nomos prometheus metrics in the node

* Rename metrics to registry where applicable

* Expose metrics via http

* Featuregate the metrics service

* Style and fail on encode error

* Add metrics cargo feature for mempool
2024-01-12 16:15:12 +02:00
Daniel Sanchez c3b5dc98e4
Rename consensus service to carnot consensus (#556) 2024-01-09 11:08:24 +01:00
Daniel Sanchez 6287e554ae
consensus-engine to carnot-engine. (#551)
* Refactor: consensus-engine to carnot-engine.
Move under consensus/carnot-engine

* Fuzztests update with carnot-engine

* Update missing refactor in tests

---------

Co-authored-by: Gusto <bacvinka@gmail.com>
2024-01-04 15:09:43 +01:00
Daniel Sanchez cc6b6d1cc9
Refactor nomos api: (#521)
* Extracted common service into services
* Moved api implementation to nomos node crate
2023-11-10 09:28:10 +01:00
Al Liu 9b3c675b3a
Metrics api (#466)
* Add metrics API
2023-10-31 17:20:04 +08:00
Daniel Sanchez ef243b0ee5
Add Sigkill handling to node binary (#480)
* Implement ctrlc watcher service

* Add ctrlc service to nomos node bin

* Use 1 sized buffer
2023-10-30 09:36:07 +01:00
Al Liu 1553f29bd9
Humanize array ser/deser (#468)
* humanize array ser/deser

* split fns

* use `const-hex`

* fix fmt

* create `nomos-utils` crate

* Human serde committeeid (#478)

* Human readable serde for CommitteeId

* Deserialize bytes to string if human readable

* Don't allocate if possible in human serde bytes

---------

Co-authored-by: gusto <bacv@users.noreply.github.com>
2023-10-26 23:16:10 +08:00
Al Liu b43d268543
Finish the skeleton of the api for nomos-node (#451)
* finish the barebone of the api for nomos-node
2023-10-06 16:32:35 +08:00
Giacomo Pasini d672be3bb0
[nomos-cli] Add command to disseminate data (#400)
* Add nomos-cli command to disseminate data across the network

Add nomos-cli to hold various utilities for the node.
To start, this commit adds a command to disseminate some
data through the network and build a certificate of correct
dispersal for inclusion in a block

* fmt

* reorder loop instructions

* Send blobs concurrently
2023-09-18 15:35:20 +02:00
Youngjoon Lee 8449c81d0f
Mixnet PoC base branch (#316)
* Add `mixnode` and `mixnet-client` crate (#302)

* Add `mixnode` binary (#317)

* Integrate mixnet with libp2p network backend (#318)

* Fix #312: proper delays (#321)

* proper delays

* add missing duration param

* tiny fix: compilation error caused by `rand` 0.8 -> 0.7

* use `get_available_port()` for mixnet integration tests (#333)

* add missing comments

* Overwatch mixnet node (#339)

* Add mixnet service and overwatch app

* remove #[tokio::main]

---------

Co-authored-by: Youngjoon Lee <taxihighway@gmail.com>

* fix tests for the overwatch mixnode (#342)

* fix panic when corner case happen in RandomDelayIter (#335)

* Use `log` service for `mixnode` bin (#341)

* Use `wire` for MixnetMessage in libp2p (#347)

* Prevent tmixnet tests from running forever (#363)

* Use random delay when sending msgs to mixnet (#362)

* fix a minor compilation error caused by the latest master

* Fix run output fd (#343)

* add a connection pool

* Exp backoff (#332)

* move mixnet listening into separate task

* add exponential retry for insufficient peers in libp2p

* fix logging

* Fix MutexGuard across await (#373)

* Fix MutexGuard across await

Holding a MutexGuard across an await point is not a good idea.
Removing that solves the issues we had with the mixnet test

* Make mixnode handle bodies coming from the same source concurrently (#372)

---------

Co-authored-by: Youngjoon Lee <taxihighway@gmail.com>

* Move wait at network startup (#338)

We now wait after the call to 'subscribe' to give the network
the time to register peers in the mesh before starting to
publish messages

* Remove unused functions from mixnet connpool (#374)

* Mixnet benchmark (#375)

* merge fixes

* add `connection_pool_size` field to `config.yaml`

* Simplify mixnet topology (#393)

* Simplify bytes and duration range ser/de (#394)

* optimize bytes serde and duration serde

---------

Co-authored-by: Al Liu <scygliu1@gmail.com>
Co-authored-by: Daniel Sanchez <sanchez.quiros.daniel@gmail.com>
Co-authored-by: Giacomo Pasini <Zeegomo@users.noreply.github.com>
2023-09-14 17:38:47 +09:00
Giacomo Pasini d72e54f9be
Add full replication implementation for DA (#396)
Add an initial simple but functional implementation for a data
availability protocol.
Full replication simply encodes bytes in a single blob which is
replicated in all nodes.
2023-09-13 11:01:20 +02:00
Daniel Sanchez 8da13f7012
Da nomos core (#390)
* Refactor da modules
Include da core module in nomos-core

* Include attestation trait

* Added initial approach for Da protocol

* Added empty certificate trait

* Added certificate dispersal method

* Rename validate method to validate attestation

* Clippy happy

* Add validate certificate method
2023-09-11 17:44:26 +02:00
Daniel Sanchez bca27bd27f
Initial DA service sketch (#376)
* Add basic da module and traits

* Pipe new blobs and internal message handling

* Add and pipe send attestation method
2023-09-06 15:11:15 +02:00
Daniel Sanchez e5ec2b7739
Update resolver version in workspace (#337)
* Update resolver version in workspace

* Clippy happy
2023-08-29 15:45:48 +02:00
Daniel Sanchez 2bd8ea92b1
DA kzg initial approach (#309)
* Added kzg crate

* Added basic commitment and proof creation methods

* Added blob verification

* DA kzg custom blob sizes (#325)

* Added types module

* Added dynamic size methods

* Added compute and verify test

* Fix using wrong constant on compute_commitment

* Criterion setup and first benchmark

* Move constants to settings struct

* Add tiny docs

* Pad blob is data shorter than it should

---------

Co-authored-by: Gusto <bacvinka@gmail.com>

---------

Co-authored-by: Gusto <bacvinka@gmail.com>
2023-08-28 10:51:46 +02:00
Daniel Sanchez 61b587a4ce
RS encoding/decoding base methods (#303)
* Add rs lib, implement encoding

* Implement decoding
2023-08-17 09:05:52 +02:00
Youngjoon Lee 2b9769b5b7
feat: add `nomos-libp2p` crate (for nomos-network backend) (#237)
* feat: add libp2p network backend skeleton

* use tokio runtime managed by Overwatch

* feat: add nomos-libp2p crate

* remove gossipsub_message_id_fn

* clippy

* use next() instead of select_next_some()

* rename send_command to execute_command

* const timeout

* disable authn / msg signing to start from a clean slate

* rename CommandSender to CommandResultSender

* add comments

* move node machinery to networkbackend

* fmt

* logs more network events

---------

Co-authored-by: Giacomo Pasini <g.pasini98@gmail.com>
2023-07-11 10:33:57 +02:00
Giacomo Pasini 09370dcef8
Integrations tests (#221)
* make config struct fields public

* add basic integration tests

* Add libssl-dev dependency

* fix comments

---------

Co-authored-by: Gusto <bacvinka@gmail.com>
2023-06-27 13:05:09 +02:00
Giacomo Pasini 2fc10f94f8
Rename mockpool-node to nomos-node (#203) 2023-06-20 17:04:52 +02:00
Giacomo Pasini f8617d7331
Consensus engine rework (#127)
---------

Co-authored-by: Giacomo Pasini <Zeegomo@users.noreply.github.com>

---------

Co-authored-by: Al Liu <scygliu1@gmail.com>
Co-authored-by: Daniel Sanchez <sanchez.quiros.daniel@gmail.com>
2023-04-27 19:18:24 +02:00
Daniel Sanchez 6ce9fdf553
[WIP] Simulation app barebones (#90)
* Add simulations crate

* Kickstart modules

* Add overlay

* Use node steps

* Add initial carnot node

* Implement Carnot node costs simulation

* Firs working runner

* Cleanup

* Extract overlay from runner

* Fix off id calls

* Fix testing values

* Clippy happy

* make simulation app compile to wasm

---------

Co-authored-by: al8n <scygliu1@gmail.com>
2023-03-14 07:15:45 -07:00
Daniel Sanchez dbe36bba3c
Mockpool node (#53)
* Create nodes folder
Kickstart mockpool node

* Create nodes folder
Added bridges file

* Added metrics to mempool

* Remove metrics from node

* Added mempool metrics bridge

* Pipe in mempool_metric bridge

* Add wakuinfo to waku network service

* Add waku network info bridge

* Added waku info bridge to node

* Use mock Tx wrapper over a string

* Create add tx http bridge

* Add tx bridge to http config

* Use hash for Tx

* Remove tracing subscriber from binary

* Fix bridges routes

* Added mimimal configuration example

* Remove subscribing to default waku pubsub topic

* Use addtx payload for tx

* Remove pub serde mod from core transaction

* Clippy happy

* Id from &Tx instead of owned value

* Removed mempool metrics feature
2023-01-25 07:24:33 -08:00
gusto 915ec00b34
Http service api (#44)
* Cargo http-service folder

* WIP: http server

* Revert comments in network service

* Router service and axum implementation structure

* Move bin contents to examples dir

* Add http service and server traits

* HttpMsg definition

* WIP: axum backend

* fix example in Cargo.toml

* Shared axum router and router modification methods

* Http example with axum and metrics service

* make project compile and add Error associated type

* Axum backend shared router fixes

* Dummy service implementation for http example

* remove unused clone on mutex

* Cargo http-service folder

* WIP: http server

* Revert comments in network service

* Router service and axum implementation structure

* Move bin contents to examples dir

* Add http service and server traits

* HttpMsg definition

* WIP: axum backend

* fix example in Cargo.toml

* Shared axum router and router modification methods

* Http example with axum and metrics service

* make project compile and add Error associated type

* Axum backend shared router fixes

* Dummy service implementation for http example

* remove unused clone on mutex

* Fix typos and remove unused code

* Fix failing tests when feature flags are not set

* Use bytes as a type for payload and response in http service

* Refactored http crate layout into differential services files

* First stab at router service

* Fully piped http bridge system

* Start building bridge helper function

* Refactor bridge builder helper and update example

* impl serialization for metrics data

* Get updated copy of router when processing request

* remove unused code

* fix typo

* [POC]: Http service: support add graphql handler (#47)

* WIP: add graphql endpoint

* support add graphql handler

* remove generic

* fix clippy warnings

* Add post put and patch handlers that expect bytes as body

* Graphql example file

* WIP: Use http post method for graphql related queries

* Parse graphql requests in handler

* Simplify handlers for post and other data methods

* Revert "Simplify handlers for post and other data methods"

This reverts commit 96f2b1821e.

* add tracing and remove comments

* Pass response bytes without any modifications

* Use receive_batch_json for gql request parsing

* Readme for running examples

* fix conflicts

* add a general helper function for graphql

* remove unused function

* cleanup code

* move schema initialization to handle function

* adapt metrics to http service

* fix clippy warnings

* remove unused fn

* fix clippy

* optimize example

Co-authored-by: gusto <bacvinka@gmail.com>
Co-authored-by: Gusto Bacvinka <augustinas.bacvinka@gmail.com>

* Fix cargo build without features

* Simplify handlers for routes with data

Co-authored-by: al8n <scygliu1@gmail.com>
Co-authored-by: Daniel Sanchez Quiros <sanchez.quiros.daniel@gmail.com>
2023-01-19 16:51:30 +02:00
Giacomo Pasini cfcc664e9e
Add dummy binary (#5)
* add dummy binary

* add flags for macos

* add metrics service

* rename Services to Nomos
2023-01-12 12:57:02 +01:00
Daniel Sanchez 52708b8253
Metrics refactor (#20)
* metrics service

Co-authored-by: al8n <scygliu1@gmail.com>
2022-12-15 04:23:06 +13:00
Giacomo Pasini 539c986f69
Add mempool stub (#29)
* add mempool stub

* address review comments

* move base data types to nomos-core

* allow clippy warning
2022-12-14 15:30:45 +01:00
Daniel Sanchez fb3fd6f3b1
Nomos core (#28)
* Extract block to core crate

* Added linking flag for waku

* Cleanup imports

* Add missing core files

* Create more base mods
2022-12-13 15:35:11 +01:00
Giacomo Pasini 7cc9181574
Consensus backbone (#16)
* tmp

* add peers implem

* update return values

* Backport network consensus adapter (#26)

* Pipe network adapter with proper types over original implementation

* Hold relay on CarnotConsensus

* Scratch Network adapter methods

* Fix tests blocking CI

* Fix waku feature on network crate

* Fix waku_bindings refs

* Restructure consensus network

* Stream block chunk

* Pipe adapter creation with subscription

* Add placeholder proposal chunk and approval messages

* Implement waku backend

* Clippy happy

* Use full path for tokio oneshot and error types in message_subscriber_channel method

* Clean imports

* small fixes

Co-authored-by: Daniel Sanchez Quiros <sanchez.quiros.daniel@gmail.com>

Co-authored-by: Daniel Sanchez Quiros <sanchez.quiros.daniel@gmail.com>
2022-12-13 11:15:54 +01:00
Giacomo Pasini 90c7de6271
Split services into different crates (#13) 2022-11-23 14:43:22 +01:00
Giacomo Pasini 0fcfd92a55
Initial network service (#2)
* initial network service

* [style] unpack inbound relay
2022-11-03 06:28:37 -07:00