Commit Graph

83 Commits

Author SHA1 Message Date
Daniel Sanchez aebdd5c814 DA: KZG+RS core (#632)
* Removed old kzg rs modules

* Added new kzgrs core module

* Implemented bytes_to_polynomial and tests

* Use coefficient form

* Refactor evaluations into method

* Use domain elements instead of roots of unity in tests

* Fix encoding and test

* Clippy happy

* Add comments

* Implement polynomial commitment

* Implement proof generation

* Sketch fn signature for verification

* implement proof verification

* Implemented verification and tests

* Return evaluations from bytes_to_polynomial as well

* Use modular le bytes

* Implement rs encode/decode

* Implement decoding tests

* Implement decode using lagrange

* Cleanup imports
2024-04-18 16:24:56 +03:00
danielsanchezq b33dbc27e8 Base cleaning of da to new traits/structure
Added new da protocols and types
2024-04-18 16:24:54 +03:00
Daniel Sanchez e085b6bef4
Split mempool into tx and da mempool (#636)
* Split mempool into tx and da mempool

* Fix tests

* Differentiate between da and cl mempool msgs in consensus

* Common mempool msg type

---------

Co-authored-by: Gusto <bacvinka@gmail.com>
2024-04-18 15:07:28 +02:00
Giacomo Pasini 7e4d00cc78
Switch from Carnot to Cryptarchia (#624)
* Switch from Carnot to Cryptarchia

* Do not delete old coins

* Update tests/src/lib.rs

Co-authored-by: davidrusu <davidrusu.me@gmail.com>

* clippy happy

---------

Co-authored-by: davidrusu <davidrusu.me@gmail.com>
2024-03-28 12:56:04 +01:00
gusto b341a11eb2
Metrics to nomos services (#623)
* Remove unused metrics crate from services

* Move prometheus metrics to nomos-services
2024-03-22 14:03:35 +02:00
gusto de6138fc27
Remove old http api implementation (#622) 2024-03-22 12:55:25 +02:00
Al Liu fd8ea50dd1
Use rocksdb as storage backend (#585)
* finish rocksdb storage service

* use rocksdb

* create missing cf

* Update Cargo.toml
2024-03-21 12:23:26 +01:00
Youngjoon Lee d449114044
Replace mixnet backend with mixnet addon (#615) 2024-03-19 19:15:09 +09:00
Youngjoon Lee dbda061f04
Added `mixnet` compilation feature (#610) 2024-03-14 09:53:41 +09:00
Giacomo Pasini 50cff241fe
Refactor block (#609)
* Refactor Block/Header definition

Refactor block/header definition so that it's now responsibility
of the nomos-core crate. This removes definitions in ledger/consensus
crates since there's no need at that level to have an understanding
of the block format.

The new header format supports both carnot and cryptarchia.
2024-03-13 18:46:10 +01:00
Youngjoon Lee e7d591b7bc
Mixnet v1 (#569)
* base

* Remove mixnet client from libp2p network backend (#572)

* Mixnet v1: Remove all mixnet legacies: mixnet crate, mixnode binary, tests, and docker (#573)

* Mixnet v1: Skeleton (#570)

* Use QUIC for libp2p (#580)

* Add Poisson interval function for Mixnet (#575)

* Mixnet network backend skeleton (#586)

* Libp2p stream read/write (#587)

* Emitting packets from mixclient using libp2p stream (#588)

* Handle outputs from mixnode using libp2p stream/gossipsub (#589)

* Refactor poisson (#590)

* Mix client Poisson emission (#591)

* Mix node packet handling (#592)

* Mix Packet / Fragment logic (#593)

* Move FisherYates to `nomos-utils` (#594)

* Mixnet topology (#595)

* Mix client/node unit tests (#596)

* change multiaddr from tcp to udp with quic-v1 (#607)

---------

Co-authored-by: Al Liu <scygliu1@gmail.com>
2024-03-12 15:47:35 +09:00
Giacomo Pasini 06d225db20
Make block id generic in mempool (#605) 2024-03-11 11:07:58 +01:00
gusto aa06080baa
Docker compose metrics containers (#564)
* Prometheus container and configuration for testnet

* Add graphana related configuration and container

* Use metrics feature in testnet nodes

* Grafana configuration files and datasources
2024-01-23 18:16:33 +02:00
gusto 46d53479a2
Prometheus metrics service (#522)
* A wrapper crate for prometheus client

* Initial integration of metrics for mempool

* Merge mempool metrics imports

* Add cli flag to enable metrics

* Add nomos metrics service for serving metrics

* Use nomos prometheus metrics in the node

* Rename metrics to registry where applicable

* Expose metrics via http

* Featuregate the metrics service

* Style and fail on encode error

* Add metrics cargo feature for mempool
2024-01-12 16:15:12 +02:00
Daniel Sanchez c3b5dc98e4
Rename consensus service to carnot consensus (#556) 2024-01-09 11:08:24 +01:00
Daniel Sanchez 6287e554ae
consensus-engine to carnot-engine. (#551)
* Refactor: consensus-engine to carnot-engine.
Move under consensus/carnot-engine

* Fuzztests update with carnot-engine

* Update missing refactor in tests

---------

Co-authored-by: Gusto <bacvinka@gmail.com>
2024-01-04 15:09:43 +01:00
Daniel Sanchez cc6b6d1cc9
Refactor nomos api: (#521)
* Extracted common service into services
* Moved api implementation to nomos node crate
2023-11-10 09:28:10 +01:00
Al Liu c3422c196c
New http api to nomos-node integration (#490)
* Integrate new http api to nomos-node
2023-11-08 16:55:47 +08:00
gusto c14998bb9a
Da voter config for docker compose (#513)
* Readd docker build context

* Configurable da protocol voter

* Do not use private key naming for da voter
2023-11-07 13:52:27 +02:00
Giacomo Pasini 0730e05a6f
Add raw api module to nomos-cli (#511)
* Add raw api module to nomos-cli

Add a separate module which contains all raw api calls and make
them reuse the same reqwest client for increased efficiency

* Use json instead of wire format in mempool add api

Uniform all http api to use json as the encoding
2023-11-06 16:24:43 +01:00
Al Liu 196f395992
Add `block_info` api for new HTTP service (#509) 2023-11-06 22:14:16 +08:00
Giacomo Pasini 56e8506704
Split long consensus response in separate APIs (#502)
* Split long consensus response in separate APIs

Consensus info was returning the full list of blocks even though
that can get quite large with time. Instead, this commit change
that API to return a constant size message and adds a new one to
return a chain of blocks with user specified endings.

* Update nomos-services/consensus/src/lib.rs

Co-authored-by: Youngjoon Lee <taxihighway@gmail.com>

* Fix test

---------

Co-authored-by: Youngjoon Lee <taxihighway@gmail.com>
2023-11-06 12:43:48 +01:00
omahs e8310e3ea8
Fix typos (#503)
* fix typo

* fix typo

* fix typo

* fix typos

* fix typo
2023-11-03 09:29:40 +01:00
Youngjoon Lee c3478cf6a6
Set voter to DA attestation (#498) 2023-11-02 18:44:31 +09:00
Al Liu ba90ed1b55
Nomos http API (#476)
* Info api

* Da blob api (#487)

* Add storage api for new http backend (#488)

* Mempool add APIs (#489)
2023-11-01 17:14:58 +08:00
GoodDaisy ff8fc577a0
Fix typos (#481)
Co-authored-by: Daniel Sanchez <sanchez.quiros.daniel@gmail.com>
2023-10-31 14:38:42 +01:00
Al Liu 9b3c675b3a
Metrics api (#466)
* Add metrics API
2023-10-31 17:20:04 +08:00
gusto 7fcfe890be
Configure supermajority threshold via envvars (#471) 2023-10-30 10:38:57 +02:00
Daniel Sanchez ef243b0ee5
Add Sigkill handling to node binary (#480)
* Implement ctrlc watcher service

* Add ctrlc service to nomos node bin

* Use 1 sized buffer
2023-10-30 09:36:07 +01:00
Giacomo Pasini 2f9ebbd32f
Storage api (#473)
* Change impl of StorageReceiver to Option<Bytes>

Load and remove messages return Option<Bytes> and not Bytes, so
let's change the implementation to work around that.

* Add storage/block http api to retrieve blocks from storage

* add tests for storage/block api

* debug tests

* tweak test node online condition
2023-10-25 12:46:26 +02:00
Daniel Sanchez 75b36020c2
Lifecycle update and implementations (#457)
* Update deps

* Implement base lifecycle handling in network service

* Implement base lifecycle handling in storage service

* Use methods instead of functions

* Pipe lifecycle in metrics service

* Pipe lifecycle in mempool service

* Pipe lifecycle in log service

* Pipe lifecycle in da service

* Pipe lifecycle in consensus service

* Refactor handling of lifecycle message to should_stop_service

* Update overwatch version to fixed run_all one
2023-10-25 12:10:21 +02:00
Giacomo Pasini 89b8e27612
Add API to revieve DA blobs (#477)
* Use Vec instead of HashMap in DaMsg::Get

* Add http API to retrieve DA blobs
2023-10-24 15:38:33 +02:00
Gusto f14df0c479 Add libp2p feature flag for da dep 2023-10-17 14:46:41 +03:00
Giacomo Pasini f90704f1d7
Save safe blocks contents to storage (#464) 2023-10-17 11:04:47 +02:00
Daniel Sanchez f04c4a6492
Use specific revision on overwatch (#455) 2023-10-09 09:18:56 +02:00
gusto 309e5a29e9
Nomos node config (#423)
* Use tree overlay in nomos node

* Use tree overlay in tests module

* Handle unexpected consensus vote stream end in tally

* Add defaults for retry and delay in mixnet client

* Add missing entries in config.yaml

* Spawn the next leader first (#425)

---------

Co-authored-by: Youngjoon Lee <taxihighway@gmail.com>
Co-authored-by: Al Liu <scygliu1@gmail.com>
2023-10-05 15:16:40 +03:00
gusto 99d77f7e1c
Nomos node tree overlay (#415)
* Use tree overlay in nomos node

* Use tree overlay in tests module

* Handle unexpected consensus vote stream end in tally

* Spawn the next leader first (#425)

* Report unhappy blocks in happy path test (#430)

* report unhappy blocks in the happy path test

* Make threshold configurable for TreeOverlay (#426)

* Modified test, so that all nodes don’t all connect to the first node only (#442)

* merge fix

---------

Co-authored-by: Youngjoon Lee <taxihighway@gmail.com>
Co-authored-by: Al Liu <scygliu1@gmail.com>
2023-10-05 11:56:24 +03:00
Giacomo Pasini 58686b2a04
Send certificate to node after dissemination + tests (#450)
* Send DA certificate to node after dissemination

* rename mempool endpoints

* Check certificate inclusion in tests

* rename endpoint

* Rename addcert and addtx to add

* tweak test condition

* add option to save certificate to file

* move thread join

* remove fancy prints
2023-10-04 13:37:13 +02:00
Giacomo Pasini 29b963fa29
Add mempool status API (#449)
* Add status service API

* Add status API to node
2023-10-02 19:06:12 +02:00
Giacomo Pasini f9c446e48c
Add a canonical way to get an id for Attestation and Certificate (#448)
* add hash() method on Attestation and Certificate

* clippy
2023-10-02 15:59:57 +02:00
Al Liu 4cf57eee74
remove waku from the codebase (#446) 2023-10-02 16:41:08 +08:00
Giacomo Pasini d67d08e81d
Add a second mempool for DA certificates (#443)
* Use more descriptive names for generic parameters

We're starting to have tens on generic parameters, if we don't use
descriptive names it'll be pretty hard to understand what they do.
This commit changes the names of the mempool parameters in
preparation for the insertion of the da certificates mempool to
distinguish it from cl transactions.

* Add mempool for da certificates

* Add separate certificates mempool to binary

* ignore clippy lints
2023-10-02 10:38:05 +02:00
Giacomo Pasini 36b3ccc043
Add `send` method to mempool network adapter (#439)
* Add `send` method to mempool network adapter

Centralize responsabilities for mempool-network interface in the
adapter trait.

* Update nomos-services/mempool/src/lib.rs

Co-authored-by: Youngjoon Lee <taxihighway@gmail.com>

---------

Co-authored-by: Youngjoon Lee <taxihighway@gmail.com>
2023-09-27 11:58:42 +02:00
Giacomo Pasini dda4a1365c
Add discriminant type to allow multiple mempool services (#441)
Overwatch requires all services to have a different service id.
Unfortunately, such service id can't depend on generic parameters,
which means that we can't have two instances of the mempool
service even if they are instantiated with different types.
This commit circuments this limitation by adding another
type parameter.
2023-09-27 10:55:15 +02:00
Giacomo Pasini 03973cd422
Make mempool generic (#428)
* Make mempool item generic

Make the mempool generic with respect to the item and remove
mentions of specific transaction formats/traits. This will allow
us to reuse the same code for both coordination layer transactions
and certificates, or in general, whatever items need to be included
in a block.

* Add mempool network adapter settings

Allow for greater customization of the mempool network adapter by
adding a settings field.

* update node after mempool changes

* fix waku mempool adapter

* fmt

* fix tests

* fmt
2023-09-26 11:14:44 +02:00
Daniel Sanchez 95618c0a72
Use selection for blob certificates (#427)
* Use selection for blob certificates

* Fix bin imports

* Fix rebase

* Missing blobs -> certificates refactor

* Fix attestation and certificate as_bytes

* More naming refactors
2023-09-25 15:34:05 +02:00
Daniel Sanchez e71c359171
Remove waku support in nomos-node binary (#429) 2023-09-20 16:32:43 +02:00
Daniel Sanchez 500683b0da
Pipe propose block (#422)
* Add new method to tx selector

* Add new method to blob selector

* Docs typo

* Added missing txs and blobs methods to block builder

* Extend selector generics

* Pipe proposing selector types

* Remove leftover

* Added missing types on waku feature
2023-09-20 11:01:04 +02:00
Giacomo Pasini 5e194922c6
Add da service to nomos node (#404)
* Make  the data availability service work with multiple protocols

* Add a generic way to instantiate DaProtocol

Add settings type and a new `new(Self::Settings)` method to
build a new DaProtocol instance

* Add data availability service to node

* fix tests

* fix imports
2023-09-18 11:43:24 +02:00
gusto 8d9a1516be
ci: fix cargo clippy deprecated warnings (#412) 2023-09-18 11:24:49 +03:00