Commit Graph

154 Commits

Author SHA1 Message Date
Al Liu a4f403b545
explorer tests 2024-03-11 18:14:51 +08:00
Al Liu 2c79c8c77a
merge 2024-03-04 21:16:38 +08:00
Al Liu bd750f0ae6
try to fix blocking test 2024-03-04 21:11:42 +08:00
Al Liu fde0d29860
RocksDB Storage service (#584)
* finish rocksdb storage service
2024-02-29 12:32:45 +08:00
Al Liu ad47dcf228
Block explorer 2 (#566)
* support query block list from explorer
2024-01-31 16:30:48 +08:00
gusto 46d53479a2
Prometheus metrics service (#522)
* A wrapper crate for prometheus client

* Initial integration of metrics for mempool

* Merge mempool metrics imports

* Add cli flag to enable metrics

* Add nomos metrics service for serving metrics

* Use nomos prometheus metrics in the node

* Rename metrics to registry where applicable

* Expose metrics via http

* Featuregate the metrics service

* Style and fail on encode error

* Add metrics cargo feature for mempool
2024-01-12 16:15:12 +02:00
Daniel Sanchez c3b5dc98e4
Rename consensus service to carnot consensus (#556) 2024-01-09 11:08:24 +01:00
gusto 7077ef38fa
Limit number of committed blocks in info requests (#552)
* Limit number of committed blocks in info requests

* Make carnot info limit configurable
2024-01-05 11:21:59 +02:00
Daniel Sanchez 6287e554ae
consensus-engine to carnot-engine. (#551)
* Refactor: consensus-engine to carnot-engine.
Move under consensus/carnot-engine

* Fuzztests update with carnot-engine

* Update missing refactor in tests

---------

Co-authored-by: Gusto <bacvinka@gmail.com>
2024-01-04 15:09:43 +01:00
Giacomo Pasini 6e718e7bba
Add chat demo for testnet (#495)
* Add chat demo for testnet

This commit adds a simple demo to showcase the capabilities of the
Nomos architecture. In particular, we want to leverage the DA
features and explore participants roles.
At the same time, we're not ready to commit to any speficic format
or decision regarding common ground yet.
For this reason, we chose to implement the demo at the Execution
Zone (EZ) level.
In contrast to the coordination layer, each execution
zone can decide on its own format, which allows us to experiment
without having to set a standard.

The application of choice for the demo is an (almost) instant
messaging app where the messages are broadcast to the public by
leveraging the full replication data availability protocol.
In this context, the cli app acts as a small EZ disseminating
blobs, promoting blob inclusion and updating its state (i.e. list
of exchanged messages) upon blob inclusion in the chain.



---------

Co-authored-by: danielsanchezq <sanchez.quiros.daniel@gmail.com>
2024-01-03 15:47:21 +01:00
Youngjoon Lee 643bcef542
Reestablish conn with mixnode if err occurs (#501) 2023-11-28 18:38:48 +09:00
Daniel Sanchez cc6b6d1cc9
Refactor nomos api: (#521)
* Extracted common service into services
* Moved api implementation to nomos node crate
2023-11-10 09:28:10 +01:00
Giacomo Pasini 3e1a722bd2
Add option to not collect logs (#517)
There's currently no way for an app with a logger service to disable
log collection.
2023-11-08 15:47:07 +01:00
Giacomo Pasini ccc85904d9
do not use prefix in serialization (#514) 2023-11-07 15:24:49 +01:00
Giacomo Pasini 3028cdb5d1
Add option to provide custom writer to the logger backend (#518)
* Add option to provide custom writer to the logger backend

* fmt
2023-11-07 15:24:02 +01:00
Giacomo Pasini df683ad154
Limit GetBlocks response size (#515)
Put a hard limit of 512 blocks in the response returned by
GetBlocks to avoid slowing things down. This number was chosen
rather arbitrarily. We might want to do some more fine tuning.
2023-11-07 10:25:22 +01:00
Giacomo Pasini 56e8506704
Split long consensus response in separate APIs (#502)
* Split long consensus response in separate APIs

Consensus info was returning the full list of blocks even though
that can get quite large with time. Instead, this commit change
that API to return a constant size message and adds a new one to
return a chain of blocks with user specified endings.

* Update nomos-services/consensus/src/lib.rs

Co-authored-by: Youngjoon Lee <taxihighway@gmail.com>

* Fix test

---------

Co-authored-by: Youngjoon Lee <taxihighway@gmail.com>
2023-11-06 12:43:48 +01:00
Giacomo Pasini 350620b829
Do not skip Block::id during ser/de (#505)
Block.id is a necessary piece of information in the context
of the consensus engine since there it's not possible to recover
the id of the block since the contents are not available.
Instead, we should only skip that field when serializing/deserializing
a full block.
2023-11-03 10:14:26 +01:00
omahs e8310e3ea8
Fix typos (#503)
* fix typo

* fix typo

* fix typo

* fix typos

* fix typo
2023-11-03 09:29:40 +01:00
Al Liu ba90ed1b55
Nomos http API (#476)
* Info api

* Da blob api (#487)

* Add storage api for new http backend (#488)

* Mempool add APIs (#489)
2023-11-01 17:14:58 +08:00
Al Liu 9b3c675b3a
Metrics api (#466)
* Add metrics API
2023-10-31 17:20:04 +08:00
Giacomo Pasini e50561839d
Remove block contents from mempool (#485)
* Add Hash type param to Attestation and Certificate

* remove block contents from mempool
2023-10-30 12:38:04 +01:00
Daniel Sanchez ef243b0ee5
Add Sigkill handling to node binary (#480)
* Implement ctrlc watcher service

* Add ctrlc service to nomos node bin

* Use 1 sized buffer
2023-10-30 09:36:07 +01:00
Giacomo Pasini c84c3fb93b
Allow deprecated type in Swarm (#486)
We have to use the type of the upstream dependency
2023-10-27 17:47:53 +02:00
Al Liu 1553f29bd9
Humanize array ser/deser (#468)
* humanize array ser/deser

* split fns

* use `const-hex`

* fix fmt

* create `nomos-utils` crate

* Human serde committeeid (#478)

* Human readable serde for CommitteeId

* Deserialize bytes to string if human readable

* Don't allocate if possible in human serde bytes

---------

Co-authored-by: gusto <bacv@users.noreply.github.com>
2023-10-26 23:16:10 +08:00
Giacomo Pasini 2f9ebbd32f
Storage api (#473)
* Change impl of StorageReceiver to Option<Bytes>

Load and remove messages return Option<Bytes> and not Bytes, so
let's change the implementation to work around that.

* Add storage/block http api to retrieve blocks from storage

* add tests for storage/block api

* debug tests

* tweak test node online condition
2023-10-25 12:46:26 +02:00
Daniel Sanchez 75b36020c2
Lifecycle update and implementations (#457)
* Update deps

* Implement base lifecycle handling in network service

* Implement base lifecycle handling in storage service

* Use methods instead of functions

* Pipe lifecycle in metrics service

* Pipe lifecycle in mempool service

* Pipe lifecycle in log service

* Pipe lifecycle in da service

* Pipe lifecycle in consensus service

* Refactor handling of lifecycle message to should_stop_service

* Update overwatch version to fixed run_all one
2023-10-25 12:10:21 +02:00
Giacomo Pasini 89b8e27612
Add API to revieve DA blobs (#477)
* Use Vec instead of HashMap in DaMsg::Get

* Add http API to retrieve DA blobs
2023-10-24 15:38:33 +02:00
Giacomo Pasini 3ce8cacb30
Revert "Reestablish connection when mixclient fails (#445)" (#472)
This reverts commit aa69eeca00.
2023-10-23 16:53:20 +02:00
Al Liu aa69eeca00
Reestablish connection when mixclient fails (#445)
* Reestablish connection when mixclient fails

* Add delay and remove select
2023-10-20 14:58:58 +08:00
Giacomo Pasini f90704f1d7
Save safe blocks contents to storage (#464) 2023-10-17 11:04:47 +02:00
Giacomo Pasini 2016f75213
Add API to return DA blobs (#453)
* Add API to return DA blobs

* remove pending blobs API
2023-10-09 10:53:01 +02:00
Daniel Sanchez f04c4a6492
Use specific revision on overwatch (#455) 2023-10-09 09:18:56 +02:00
Al Liu dbac7d7597
fix clippy and check warnings (#452) 2023-10-06 15:08:52 +08:00
gusto 99d77f7e1c
Nomos node tree overlay (#415)
* Use tree overlay in nomos node

* Use tree overlay in tests module

* Handle unexpected consensus vote stream end in tally

* Spawn the next leader first (#425)

* Report unhappy blocks in happy path test (#430)

* report unhappy blocks in the happy path test

* Make threshold configurable for TreeOverlay (#426)

* Modified test, so that all nodes don’t all connect to the first node only (#442)

* merge fix

---------

Co-authored-by: Youngjoon Lee <taxihighway@gmail.com>
Co-authored-by: Al Liu <scygliu1@gmail.com>
2023-10-05 11:56:24 +03:00
Giacomo Pasini 58686b2a04
Send certificate to node after dissemination + tests (#450)
* Send DA certificate to node after dissemination

* rename mempool endpoints

* Check certificate inclusion in tests

* rename endpoint

* Rename addcert and addtx to add

* tweak test condition

* add option to save certificate to file

* move thread join

* remove fancy prints
2023-10-04 13:37:13 +02:00
Giacomo Pasini 29b963fa29
Add mempool status API (#449)
* Add status service API

* Add status API to node
2023-10-02 19:06:12 +02:00
Al Liu 4cf57eee74
remove waku from the codebase (#446) 2023-10-02 16:41:08 +08:00
Giacomo Pasini d67d08e81d
Add a second mempool for DA certificates (#443)
* Use more descriptive names for generic parameters

We're starting to have tens on generic parameters, if we don't use
descriptive names it'll be pretty hard to understand what they do.
This commit changes the names of the mempool parameters in
preparation for the insertion of the da certificates mempool to
distinguish it from cl transactions.

* Add mempool for da certificates

* Add separate certificates mempool to binary

* ignore clippy lints
2023-10-02 10:38:05 +02:00
Giacomo Pasini 36b3ccc043
Add `send` method to mempool network adapter (#439)
* Add `send` method to mempool network adapter

Centralize responsabilities for mempool-network interface in the
adapter trait.

* Update nomos-services/mempool/src/lib.rs

Co-authored-by: Youngjoon Lee <taxihighway@gmail.com>

---------

Co-authored-by: Youngjoon Lee <taxihighway@gmail.com>
2023-09-27 11:58:42 +02:00
Giacomo Pasini dda4a1365c
Add discriminant type to allow multiple mempool services (#441)
Overwatch requires all services to have a different service id.
Unfortunately, such service id can't depend on generic parameters,
which means that we can't have two instances of the mempool
service even if they are instantiated with different types.
This commit circuments this limitation by adding another
type parameter.
2023-09-27 10:55:15 +02:00
Giacomo Pasini 03973cd422
Make mempool generic (#428)
* Make mempool item generic

Make the mempool generic with respect to the item and remove
mentions of specific transaction formats/traits. This will allow
us to reuse the same code for both coordination layer transactions
and certificates, or in general, whatever items need to be included
in a block.

* Add mempool network adapter settings

Allow for greater customization of the mempool network adapter by
adding a settings field.

* update node after mempool changes

* fix waku mempool adapter

* fmt

* fix tests

* fmt
2023-09-26 11:14:44 +02:00
Daniel Sanchez 95618c0a72
Use selection for blob certificates (#427)
* Use selection for blob certificates

* Fix bin imports

* Fix rebase

* Missing blobs -> certificates refactor

* Fix attestation and certificate as_bytes

* More naming refactors
2023-09-25 15:34:05 +02:00
Youngjoon Lee 285804e1d5
Remove unnecessary dependencies (#433) 2023-09-25 17:26:21 +09:00
Daniel Sanchez 500683b0da
Pipe propose block (#422)
* Add new method to tx selector

* Add new method to blob selector

* Docs typo

* Added missing txs and blobs methods to block builder

* Extend selector generics

* Pipe proposing selector types

* Remove leftover

* Added missing types on waku feature
2023-09-20 11:01:04 +02:00
Youngjoon Lee e0cd35dcdc
Refactor libp2p network backend into submodules (#417) 2023-09-19 22:16:27 +09:00
Giacomo Pasini 5e194922c6
Add da service to nomos node (#404)
* Make  the data availability service work with multiple protocols

* Add a generic way to instantiate DaProtocol

Add settings type and a new `new(Self::Settings)` method to
build a new DaProtocol instance

* Add data availability service to node

* fix tests

* fix imports
2023-09-18 11:43:24 +02:00
gusto 8d9a1516be
ci: fix cargo clippy deprecated warnings (#412) 2023-09-18 11:24:49 +03:00
Youngjoon Lee 82946b8637
etry connecting with initial peers (#410) 2023-09-16 00:23:21 +09:00
Daniel Sanchez 04d07038dc
Remove unused fountain codes (#407)
* Remove unused fountain codes

* Refactor ProposalChunk to Proposal
2023-09-15 09:39:53 +02:00