Commit Graph

27 Commits

Author SHA1 Message Date
Daniel Sanchez 7b9643f17e
DA Protocol V1 (#626)
* Base cleaning of da to new traits/structure
Added new da protocols and types

* DA: KZG+RS core (#632)

* Removed old kzg rs modules

* Added new kzgrs core module

* Implemented bytes_to_polynomial and tests

* Use coefficient form

* Refactor evaluations into method

* Use domain elements instead of roots of unity in tests

* Fix encoding and test

* Clippy happy

* Add comments

* Implement polynomial commitment

* Implement proof generation

* Sketch fn signature for verification

* implement proof verification

* Implemented verification and tests

* Return evaluations from bytes_to_polynomial as well

* Use modular le bytes

* Implement rs encode/decode

* Implement decoding tests

* Implement decode using lagrange

* Cleanup imports

* Da: v1 encoder (#633)

* Added new kzgrs core module

* Implemented bytes_to_polynomial and tests

* Use coefficient form

* Refactor evaluations into method

* Use domain elements instead of roots of unity in tests

* Fix encoding and test

* Clippy happy

* Add comments

* Implement polynomial commitment

* Implement proof generation

* Sketch fn signature for verification

* implement proof verification

* Implemented verification and tests

* Return evaluations from bytes_to_polynomial as well

* Use modular le bytes

* Implement rs encode/decode

* Implement decoding tests

* Implement decode using lagrange

* Added chunksmatrix

* Implement encoder with chunkify

* Added missing files

* Implement commit row commitments

* Implement compute elements (row) proofs

* Fix lib and types exposures

* Implement encoder main methods

* Implement encode method

* Implement chunkify test
Fix related bugs

* Implement compute row kzg commitments
Fix related bugs

* Implement rs encode rows test
Fix related bugs
Refactored API

* Implement row proofs tests
Fix fieldelement encoding/decoding bug

* Implement aggregated commitment test
Implement aggregated column proofs test

* Cleanup

* Fix deps

* Fix tests

* Fix chunk too big test

* Da: v1 verifier (#635)

* Fix encoding and test

* Implement commit row commitments

* Implemented dablob

* Implement verifier new
Implement verifier check column

* Clippy cleanup

* Implement verifier

* Implemented verify column test

* Implemented full verify test

* DA API Payload to Item in mempool  (#634)

* Base cleaning of da to new traits/structure
Added new da protocols and types

* DA: KZG+RS core (#632)

* Removed old kzg rs modules

* Added new kzgrs core module

* Implemented bytes_to_polynomial and tests

* Use coefficient form

* Refactor evaluations into method

* Use domain elements instead of roots of unity in tests

* Fix encoding and test

* Clippy happy

* Add comments

* Implement polynomial commitment

* Implement proof generation

* Sketch fn signature for verification

* implement proof verification

* Implemented verification and tests

* Return evaluations from bytes_to_polynomial as well

* Use modular le bytes

* Implement rs encode/decode

* Implement decoding tests

* Implement decode using lagrange

* Cleanup imports

* Reduce abstraction for certificate and vid metadata

* Allow payload to mempool as long as it converts into item

* Da Certificate verifier

* Add mock certificate for core tests

* Mempool payload verification

* Integrate mock verifiers for tx and certs

* Detach verification from cert and tx

* Seperate payload and item in mempools

* Mempools in integration tests

* Remove old cert verifier

* Network payload to item constraints in da mempool

* Update nomos-da/full-replication/src/lib.rs

Co-authored-by: Daniel Sanchez <sanchez.quiros.daniel@gmail.com>

* Sort attestations for cert signature

* Update nomos-da/full-replication/src/lib.rs

Co-authored-by: Daniel Sanchez <sanchez.quiros.daniel@gmail.com>

---------

Co-authored-by: danielsanchezq <sanchez.quiros.daniel@gmail.com>

* DA API Certificate verification  (#641)

* Redo certificate verification in mempool

* FullReplication verifier params provider

* Integrate da params provider into the node

* DA API Indexer service (#643)

* Base cleaning of da to new traits/structure
Added new da protocols and types

* Remove da availability crate completely

* Scaffold for da storage service

* Indexer service responsible for storage and blockchain subscription

* Handle index related ops only

* Acquire storage and consensus relays

* Indexer trait

* wip: storage adapter

* Use storage adapter instead of storage

* Add consensus adapter trait for block subscriptions

* Consensus block subscriber adapter

* Use metadata from da core in indexer

* Update nomos-services/data-availability/indexer/src/lib.rs

Co-authored-by: Daniel Sanchez <sanchez.quiros.daniel@gmail.com>

* Update nomos-services/data-availability/indexer/src/lib.rs

Co-authored-by: Daniel Sanchez <sanchez.quiros.daniel@gmail.com>

* Use std::ops::Range for da queries

* Return certificate metadata

* Da storage adapter methods

---------

Co-authored-by: danielsanchezq <sanchez.quiros.daniel@gmail.com>

* Reuse evaluations when computing proofs (#647)

* Reuse precomputed evaluations instead of evaluation polynomial for each proof

* kzgrs benchmarks

* Clippy happy

* DA API indexer implementation (#644)

* Rocksb adapter in da indexer

* Handle da service messages

* Remove indexer trait, use storage directly in the indexer service

* Return unordered indexes range

* Load blob by vid from file

* Use VID in consensus

* Change full replication index type to byte array

* Change payload to cert and item to vid where required

* Service integration tests for indexer

* Feature gate rocksdb backend

* Propagate range response send error

* FRIndex to Index

* VID to VidCertificate

* Pass blobs file dir via settings

* Da v1 multiple proofs bench (#648)

* Parallel proof generation bench

* Added bench comment

* Modify domain to fit exact sizes

* Fix domain in benches

* Force parallelization features in lib

* DA: Implement base structure for verifier service (#627)

* Base cleaning of da to new traits/structure
Added new da protocols and types

* Implement base structure for verifier service

* Added comments and todo!

* Cleanup imports

* Size of VidCert in full replication

* Nomos Da Verifier service crate

* Extension replaced with metadata

* Fix DaIndexer service name

* Storage adapter trait in verifier

* Manage lifecycle and messages in verifier

* Blob trait in core

* Common nomos da storage crate

* Use updated nomos da storage in indexer

* Verifier storage adapter

* Libp2p adaper for verifier

* Kzgrs backend in verifier service

* Fix fmt

* Clippy happy

---------

Co-authored-by: Gusto <bacvinka@gmail.com>

* DA Verifier service integration tests (#650)

* Base cleaning of da to new traits/structure
Added new da protocols and types

* Implement base structure for verifier service

* Added comments and todo!

* Cleanup imports

* Size of VidCert in full replication

* Nomos Da Verifier service crate

* Extension replaced with metadata

* Storage adapter trait in verifier

* Manage lifecycle and messages in verifier

* Common nomos da storage crate

* Use updated nomos da storage in indexer

* Verifier storage adapter

* Libp2p adaper for verifier

* Kzgrs backend in verifier service

* Fix fmt

* Data availability tests module

* Return attestation in service msg response

* Common definitions for da tests

* Serde for kzgrs proofs and commitments

* Da verifier integration test

* WIP nomos-core in kzgrs backend

* Kzgrs blob to common module

* Add client zone to verifier test and check if attestations are created

* Cleanup and comments

* Use libp2p only for verifier and indexer service tests

* Lint in da tests

* Simplify da blob serialization

* Remove attester from nomos-core attestation

* Verifier backend error

---------

Co-authored-by: danielsanchezq <sanchez.quiros.daniel@gmail.com>

* DA Kzgrs Backend Certificate implementation (#651)

* Kzgrs backend certificate definition

* Encoded data to certificate test

* Nomos da domain specific tag

* Handle errors in da certificate creation

* Add nomos core traits to da cert

* Derive ordering traits for da index

* Add failure test cases to kzgrs certificate

* Da v1 benches expand (#658)

* Update benches with more cases

* Expand benches

* Added parallel feature

* Fix test comment

Co-authored-by: Youngjoon Lee <5462944+youngjoon-lee@users.noreply.github.com>

* Remove outdated comment

---------

Co-authored-by: Gusto <bacvinka@gmail.com>
Co-authored-by: gusto <bacv@users.noreply.github.com>
Co-authored-by: Youngjoon Lee <5462944+youngjoon-lee@users.noreply.github.com>
2024-06-13 11:23:11 +03:00
Giacomo Pasini 7e4d00cc78
Switch from Carnot to Cryptarchia (#624)
* Switch from Carnot to Cryptarchia

* Do not delete old coins

* Update tests/src/lib.rs

Co-authored-by: davidrusu <davidrusu.me@gmail.com>

* clippy happy

---------

Co-authored-by: davidrusu <davidrusu.me@gmail.com>
2024-03-28 12:56:04 +01:00
Youngjoon Lee d449114044
Replace mixnet backend with mixnet addon (#615) 2024-03-19 19:15:09 +09:00
Youngjoon Lee dbda061f04
Added `mixnet` compilation feature (#610) 2024-03-14 09:53:41 +09:00
Giacomo Pasini 50cff241fe
Refactor block (#609)
* Refactor Block/Header definition

Refactor block/header definition so that it's now responsibility
of the nomos-core crate. This removes definitions in ledger/consensus
crates since there's no need at that level to have an understanding
of the block format.

The new header format supports both carnot and cryptarchia.
2024-03-13 18:46:10 +01:00
Youngjoon Lee e7d591b7bc
Mixnet v1 (#569)
* base

* Remove mixnet client from libp2p network backend (#572)

* Mixnet v1: Remove all mixnet legacies: mixnet crate, mixnode binary, tests, and docker (#573)

* Mixnet v1: Skeleton (#570)

* Use QUIC for libp2p (#580)

* Add Poisson interval function for Mixnet (#575)

* Mixnet network backend skeleton (#586)

* Libp2p stream read/write (#587)

* Emitting packets from mixclient using libp2p stream (#588)

* Handle outputs from mixnode using libp2p stream/gossipsub (#589)

* Refactor poisson (#590)

* Mix client Poisson emission (#591)

* Mix node packet handling (#592)

* Mix Packet / Fragment logic (#593)

* Move FisherYates to `nomos-utils` (#594)

* Mixnet topology (#595)

* Mix client/node unit tests (#596)

* change multiaddr from tcp to udp with quic-v1 (#607)

---------

Co-authored-by: Al Liu <scygliu1@gmail.com>
2024-03-12 15:47:35 +09:00
gusto f33c2613fb
Nomos chat app non interactive (#567)
* Add option to send chat message non iteractively via nomos cli

* Use clap to check if data or file is set

* Require author if message flag set
2024-01-29 09:43:18 +02:00
gusto f741590315
Disseminate app file input (#565)
* Accept file option in dissemination app

* File dissemination tests
2024-01-23 18:17:05 +02:00
Giacomo Pasini 16c97c07ec
Add test for big blob dissemination (#563)
* Add tests for dissemination of big blob

* Warn if a requested blob was not returned
2024-01-19 19:08:13 +01:00
Giacomo Pasini aeaf13fc88
Limit in-flight requests in chat app (#562) 2024-01-19 14:17:10 +01:00
Daniel Sanchez c3b5dc98e4
Rename consensus service to carnot consensus (#556) 2024-01-09 11:08:24 +01:00
Giacomo Pasini b7d1fd9256
Nomos chat fixes (#555)
* Make chat messages unique

Since DA will rightfully ignore duplicated messages, we need to add a nonce to make sure
every message is unique, so that a user can send the same message twice.
This nonce is randomly generated for simplicity.

* Clear in_flight status upon error on chat app
2024-01-08 15:40:19 +01:00
gusto 285300f365
Receive blocks blobs in parallel (#554)
* Receive blocks blobs in parallel

* Update nomos-cli config
2024-01-08 12:56:11 +02:00
Daniel Sanchez 6287e554ae
consensus-engine to carnot-engine. (#551)
* Refactor: consensus-engine to carnot-engine.
Move under consensus/carnot-engine

* Fuzztests update with carnot-engine

* Update missing refactor in tests

---------

Co-authored-by: Gusto <bacvinka@gmail.com>
2024-01-04 15:09:43 +01:00
Giacomo Pasini 6e718e7bba
Add chat demo for testnet (#495)
* Add chat demo for testnet

This commit adds a simple demo to showcase the capabilities of the
Nomos architecture. In particular, we want to leverage the DA
features and explore participants roles.
At the same time, we're not ready to commit to any speficic format
or decision regarding common ground yet.
For this reason, we chose to implement the demo at the Execution
Zone (EZ) level.
In contrast to the coordination layer, each execution
zone can decide on its own format, which allows us to experiment
without having to set a standard.

The application of choice for the demo is an (almost) instant
messaging app where the messages are broadcast to the public by
leveraging the full replication data availability protocol.
In this context, the cli app acts as a small EZ disseminating
blobs, promoting blob inclusion and updating its state (i.e. list
of exchanged messages) upon blob inclusion in the chain.



---------

Co-authored-by: danielsanchezq <sanchez.quiros.daniel@gmail.com>
2024-01-03 15:47:21 +01:00
Giacomo Pasini a524cc2dcf
use reqwest json() method (#525) 2023-11-11 16:49:18 +01:00
Al Liu c3422c196c
New http api to nomos-node integration (#490)
* Integrate new http api to nomos-node
2023-11-08 16:55:47 +08:00
Giacomo Pasini 1b4c0638fa
Do not exit the process during dissemination (#516)
* Do not exit the process during dissemination

This is now library code and we should not exit the process if an
error is encountered

* clippy happy
2023-11-07 15:24:31 +01:00
Giacomo Pasini 0730e05a6f
Add raw api module to nomos-cli (#511)
* Add raw api module to nomos-cli

Add a separate module which contains all raw api calls and make
them reuse the same reqwest client for increased efficiency

* Use json instead of wire format in mempool add api

Uniform all http api to use json as the encoding
2023-11-06 16:24:43 +01:00
Youngjoon Lee c3478cf6a6
Set voter to DA attestation (#498) 2023-11-02 18:44:31 +09:00
Giacomo Pasini a487d8c7a0
Add DA utilities to nomos-cli (#493)
* Move da disseminate command into separate module

* Add utilities to retrieve block contents

* fix tests

* fix typo
2023-10-30 16:19:25 +01:00
Daniel Sanchez 75b36020c2
Lifecycle update and implementations (#457)
* Update deps

* Implement base lifecycle handling in network service

* Implement base lifecycle handling in storage service

* Use methods instead of functions

* Pipe lifecycle in metrics service

* Pipe lifecycle in mempool service

* Pipe lifecycle in log service

* Pipe lifecycle in da service

* Pipe lifecycle in consensus service

* Refactor handling of lifecycle message to should_stop_service

* Update overwatch version to fixed run_all one
2023-10-25 12:10:21 +02:00
Giacomo Pasini d528ebd2ea
Extract nomos-cli da stuff into shared module (#461)
This will be useful for other commands that might want to interact
with the network to perform tasks related to DA. One example is
the upcoming testnet demo.
2023-10-23 11:34:20 +02:00
Daniel Sanchez f04c4a6492
Use specific revision on overwatch (#455) 2023-10-09 09:18:56 +02:00
Giacomo Pasini 58686b2a04
Send certificate to node after dissemination + tests (#450)
* Send DA certificate to node after dissemination

* rename mempool endpoints

* Check certificate inclusion in tests

* rename endpoint

* Rename addcert and addtx to add

* tweak test condition

* add option to save certificate to file

* move thread join

* remove fancy prints
2023-10-04 13:37:13 +02:00
Giacomo Pasini 3b64cd6b3a
Add tests for blob dissemination (#416)
* Make nomos-cli config public

Split nomos-cli into lib and bin and make config public
so that it can be reused in tests

* Add tests for nomos-cli disseminate

* fmt

* feature gate whole file instead of function

* Exit unsuccessfully when failing dissemination
2023-09-25 11:52:45 +02:00
Giacomo Pasini d672be3bb0
[nomos-cli] Add command to disseminate data (#400)
* Add nomos-cli command to disseminate data across the network

Add nomos-cli to hold various utilities for the node.
To start, this commit adds a command to disseminate some
data through the network and build a certificate of correct
dispersal for inclusion in a block

* fmt

* reorder loop instructions

* Send blobs concurrently
2023-09-18 15:35:20 +02:00