Commit Graph

298 Commits

Author SHA1 Message Date
Al Liu 146001c9fe
Mock mempool integration test (#66)
* finish mock mempool integration test

* use Log service for test

* remove unused example

* use millis and merge log PR
2023-02-07 17:13:22 +08:00
Daniel Sanchez c5ac1db44c
Network transaction broadcasting (#63)
* Broadcast transaction when validated in mempool

* Clippy happy

* Use standard bincode config for tx message decoding

* add send_transaction for mock (#64)

* add send_transaction for mock

* Use wire instead of direct bincode

* Use wire instead of direct bincode on deserialization

---------

Co-authored-by: Al Liu <scygliu1@gmail.com>
2023-02-06 12:49:24 +01:00
Al Liu 4705cc213b
Add constructor for LogSettings (#67)
* add constructor for LogSettings
2023-02-06 19:23:25 +08:00
Daniel Sanchez 9a8400f4ce
Use blake 2b instead of 2s for mockpool node TxId (#62)
* Use blake 2b instead of 2s

* Clippy happy
2023-02-02 00:20:16 -08:00
Al Liu 3d3d2760ec
Implement #31: Mock network backend (#41)
* relax trait bounds

* mock mempool

* remove unused generic

* add mock network test case

* fix some PR comments

* simplify match branch

* finish mempool adapter example

* clippy happy

* mock consensus
2023-02-01 21:58:23 +08:00
Daniel Sanchez 567188c248
Fix blake2b usage on mockpool node tx (#61) 2023-02-01 04:10:57 -08:00
gusto 57991d6e02
Docker image for nomos node (#60)
* Docker container for nomos node

* Remove peers from sample config

* Use mockpool-node and update config
2023-02-01 10:37:15 +02:00
Daniel Sanchez 1e20c3b6cc
Use commitee and view information for consensus network adapter (#57)
* Add committee and view information to network adapter

* Use committee and view on waku adapter

* Add committee and view info to flat view implementation.

* Clippy happy

* Rename flat -> root

* Split broadcast block iterator into l/r childs

* Extract topics to builder functions
2023-01-31 03:03:45 -08:00
Giacomo Pasini f5175c74c0
Add flat overlay (#55) 2023-01-27 10:37:04 +01:00
gusto 8e2346c29e
Rust 1.67 cargo clippy fix (#56)
* Run clippy --fix for updated lint requirements

* Fmt after clippy
2023-01-27 11:04:42 +02:00
Daniel Sanchez e007539531
This update fixes the default waku subscription (#54) 2023-01-26 06:53:27 -08:00
Daniel Sanchez dbe36bba3c
Mockpool node (#53)
* Create nodes folder
Kickstart mockpool node

* Create nodes folder
Added bridges file

* Added metrics to mempool

* Remove metrics from node

* Added mempool metrics bridge

* Pipe in mempool_metric bridge

* Add wakuinfo to waku network service

* Add waku network info bridge

* Added waku info bridge to node

* Use mock Tx wrapper over a string

* Create add tx http bridge

* Add tx bridge to http config

* Use hash for Tx

* Remove tracing subscriber from binary

* Fix bridges routes

* Added mimimal configuration example

* Remove subscribing to default waku pubsub topic

* Use addtx payload for tx

* Remove pub serde mod from core transaction

* Clippy happy

* Id from &Tx instead of owned value

* Removed mempool metrics feature
2023-01-25 07:24:33 -08:00
gusto 915ec00b34
Http service api (#44)
* Cargo http-service folder

* WIP: http server

* Revert comments in network service

* Router service and axum implementation structure

* Move bin contents to examples dir

* Add http service and server traits

* HttpMsg definition

* WIP: axum backend

* fix example in Cargo.toml

* Shared axum router and router modification methods

* Http example with axum and metrics service

* make project compile and add Error associated type

* Axum backend shared router fixes

* Dummy service implementation for http example

* remove unused clone on mutex

* Cargo http-service folder

* WIP: http server

* Revert comments in network service

* Router service and axum implementation structure

* Move bin contents to examples dir

* Add http service and server traits

* HttpMsg definition

* WIP: axum backend

* fix example in Cargo.toml

* Shared axum router and router modification methods

* Http example with axum and metrics service

* make project compile and add Error associated type

* Axum backend shared router fixes

* Dummy service implementation for http example

* remove unused clone on mutex

* Fix typos and remove unused code

* Fix failing tests when feature flags are not set

* Use bytes as a type for payload and response in http service

* Refactored http crate layout into differential services files

* First stab at router service

* Fully piped http bridge system

* Start building bridge helper function

* Refactor bridge builder helper and update example

* impl serialization for metrics data

* Get updated copy of router when processing request

* remove unused code

* fix typo

* [POC]: Http service: support add graphql handler (#47)

* WIP: add graphql endpoint

* support add graphql handler

* remove generic

* fix clippy warnings

* Add post put and patch handlers that expect bytes as body

* Graphql example file

* WIP: Use http post method for graphql related queries

* Parse graphql requests in handler

* Simplify handlers for post and other data methods

* Revert "Simplify handlers for post and other data methods"

This reverts commit 96f2b1821e.

* add tracing and remove comments

* Pass response bytes without any modifications

* Use receive_batch_json for gql request parsing

* Readme for running examples

* fix conflicts

* add a general helper function for graphql

* remove unused function

* cleanup code

* move schema initialization to handle function

* adapt metrics to http service

* fix clippy warnings

* remove unused fn

* fix clippy

* optimize example

Co-authored-by: gusto <bacvinka@gmail.com>
Co-authored-by: Gusto Bacvinka <augustinas.bacvinka@gmail.com>

* Fix cargo build without features

* Simplify handlers for routes with data

Co-authored-by: al8n <scygliu1@gmail.com>
Co-authored-by: Daniel Sanchez Quiros <sanchez.quiros.daniel@gmail.com>
2023-01-19 16:51:30 +02:00
gusto 613d9ac1b2
Add a step to run cargo build w/o features (#52) 2023-01-19 16:34:35 +02:00
gusto dbef2906b5
Autoauthor action (#51) 2023-01-18 17:10:13 +02:00
Giacomo Pasini efb1690872
Add transaction type (#50)
* Add transaction type
2023-01-18 15:02:58 +01:00
Daniel Sanchez 677d4a245c
Mockpool (#43)
* Use ids slices on mempool trait

* Create a mock mempool
Added placeholders for block/blockheader/blockid relations

* Use linked hashmap

* Fix bounds

* Remove unnecessary bound
2023-01-18 03:11:03 -08:00
Giacomo Pasini f5a1dd5513
Relax type bounds (#49) 2023-01-18 11:30:37 +01:00
Daniel Sanchez f1412b112e
Implemented mock fountain code protocol (#45) 2023-01-17 01:08:00 -08:00
Giacomo Pasini cfcc664e9e
Add dummy binary (#5)
* add dummy binary

* add flags for macos

* add metrics service

* rename Services to Nomos
2023-01-12 12:57:02 +01:00
Daniel Sanchez 4d43c18846
Add missing send + sync bounds to network backend state (#46) 2023-01-12 11:33:57 +01:00
Daniel Sanchez 7ff63d4824
Add fountain codes to consensus (#38)
* Add basic encode/decode test

* Use Stream for trait instead of Iterator

* Removed unnecessary pin

* Add custom fountain error

* Add failing path to tests

* Added docs

* Normalized chunks to bytes

* Added settings initialization for fountain codes trait

* Pipe fountain code through consensus

* Implement broadcast block and block reconstruction for Member Overlay

* Fix failing raptor test

* Use seed for raptorq tests

* Use const for topics instead of static

* Clippy happy
2023-01-10 12:58:51 +01:00
Giacomo Pasini 15f97dcace
fix consensus merge (#42) 2023-01-10 12:26:36 +01:00
Giacomo Pasini 0229337414
Add leadership stub (#22)
* add leadership stub

* move types to core

* clippy happy
2023-01-10 11:33:08 +01:00
Daniel Sanchez 9721e6f5fa
Mempool network adapter (#30)
* Refactor mempool naming and added settings to backend trait

* Implement mempool networking traits and waku backend

* Transaction networking plumbing in mempool service

* Make TransactionMsg generic
Use bincode to deserialize tx messages

* Make wakuadapter generic over tx and tx-id

* Fix wrong backend type bound

* Adapt to waku beta2

* Thread tx and id together for adapter and pool

* Panic on subscribing error

* Prune unnecessary Id type bounds

* Remove transaction placeholder

* Remove Id bound from waku adapter

* Remove empty transactions module
2023-01-09 17:22:46 +01:00
Daniel Sanchez 664cb3ccd8
Update waku-bindings to beta2 (#40) 2023-01-09 12:20:14 +01:00
Giacomo Pasini 3a72cddb81
remove unneeded generic parameters (#39) 2023-01-09 12:15:21 +01:00
Daniel Sanchez fc75ad1732
Fountain codes (#37)
* Add fountain module and raptorq basic implementation

* Add basic encode/decode test

* Use Stream for trait instead of Iterator

* Removed unnecessary pin

* Add custom fountain error

* Add failing path to tests

* Added docs
2023-01-05 10:35:12 +01:00
Daniel Sanchez 52708b8253
Metrics refactor (#20)
* metrics service

Co-authored-by: al8n <scygliu1@gmail.com>
2022-12-15 04:23:06 +13:00
Giacomo Pasini 539c986f69
Add mempool stub (#29)
* add mempool stub

* address review comments

* move base data types to nomos-core

* allow clippy warning
2022-12-14 15:30:45 +01:00
Daniel Sanchez fb3fd6f3b1
Nomos core (#28)
* Extract block to core crate

* Added linking flag for waku

* Cleanup imports

* Add missing core files

* Create more base mods
2022-12-13 15:35:11 +01:00
gusto ae97db3e0f
Lint and check all features (#27)
Added flags for the "Check" stage for linux and macos PR targets.
2022-12-13 13:33:51 +02:00
Giacomo Pasini 7cc9181574
Consensus backbone (#16)
* tmp

* add peers implem

* update return values

* Backport network consensus adapter (#26)

* Pipe network adapter with proper types over original implementation

* Hold relay on CarnotConsensus

* Scratch Network adapter methods

* Fix tests blocking CI

* Fix waku feature on network crate

* Fix waku_bindings refs

* Restructure consensus network

* Stream block chunk

* Pipe adapter creation with subscription

* Add placeholder proposal chunk and approval messages

* Implement waku backend

* Clippy happy

* Use full path for tokio oneshot and error types in message_subscriber_channel method

* Clean imports

* small fixes

Co-authored-by: Daniel Sanchez Quiros <sanchez.quiros.daniel@gmail.com>

Co-authored-by: Daniel Sanchez Quiros <sanchez.quiros.daniel@gmail.com>
2022-12-13 11:15:54 +01:00
gusto 5484cb7079
ci: add Jenkins and Docker file for PR and doc builds (#23)
* ci: add Jenkins and Docker file for PR and doc builds

* Fixing waku-bindings dependacy definitions

* Add post cleanup and musl-dev to Jenkinsfile

* Add go to the Dockerfile for ci

* GOCACHE set to tmp for ci builds

* Use slim-bullseye docker image for ci builds

* Update ci/Jenkinsfile.docs

Co-authored-by: Jakub <i+github@always.fail>

* Add image version and env vars for docs build

* Remove duplicated environment section

* Add missing rust dependencies

* Split jenkins file into linux and macos targets

* Removing github actions for PR checks

* Add explicit versions to the shell.nix dependencies

* Add jenkins libs required for nix

* Use default rust-bin version that includes clippy and fmt

* Add readme to ci folder

Co-authored-by: Jakub <i+github@always.fail>
2022-12-12 21:29:23 +02:00
Al Liu 8c1a521440
adapt to overwatch error handle (#24) 2022-12-12 23:06:15 +13:00
gusto ee0085e873
Continue CI steps even when one of them fails (#15)
* Continue CI steps even when one of them fails
* Build all features in CI
* Test all features in CI
2022-12-01 18:46:27 +02:00
Giacomo Pasini 0db0544a77
fix tests (#14) 2022-11-23 15:49:43 +01:00
Giacomo Pasini 90c7de6271
Split services into different crates (#13) 2022-11-23 14:43:22 +01:00
gusto 26e5580443
Add CI tests, clippy and fmt checks (#12)
Using logos-co/Overwatch CI scripts for tests and linting in PRs.
The idea is to use this for now, and later gather all common action definitions into one dedicated repository.

* Copied CI test, clippy and fmt steps from Overwatch
* Added go 1.19 to an action setup (required for building waku-rust-bindigs dep)
2022-11-23 15:10:17 +02:00
Al Liu e656fb2d9d
Bugfix: Forget to change `overwatch` to `overwatch_rs` (#11)
* direct clone on Bytes instead of deep copy(Bytes::copy_from_slice)

* change overwatch to overwatch_rs
2022-11-22 06:16:07 -08:00
Al Liu c127c1d18d
Fix cheap clone Bytes and Overwatch crate name update 2022-11-22 04:44:17 -08:00
Giacomo Pasini 3a15a9b722
Add log service (#4)
* add log service

* add ser/de to log config

* add futures dep
2022-11-21 15:35:52 +01:00
Daniel Sanchez dce6678904
Sled backend (#7)
* Add sled module

* Remove sled for now

* Implement sled as a storage backend

* Remove manual panic

* Refactor sled to adap to latest StorageBackend traits changes

* Use option bytes for sled transaction output

* Cleanup imports

* Add tests

* Removed unused scopes
2022-11-21 06:23:45 -08:00
Daniel Sanchez ece4b90550
Add testing NoSerde operator (#9) 2022-11-21 06:15:28 -08:00
Giacomo Pasini 61de96a5d3
Add Waku network backend (#3)
* Add Waku network backend

Add Waku as the first supported network backend and rework API.
In particular, the network message now depends on the underlying
network backend so that it can properly reflects specificities
of the protocol.
Another choice could have been to hardwire domain-specific actions
in the network message type (e.g. send block, send message, ...)
but was discarded in favor of a more general network service.

* address review comments

* add debug impl

* add Serialize/Deserialize to network settings

* add waku functionalities

* add a little bit of documentation
2022-11-21 12:03:47 +01:00
Daniel Sanchez a343249d92
Storage service (#6)
* Added storage main files and trait

* Pipe skeleton for StorageService

* Implement running StorageService

* Add sled module

* Refactor error reporting

* Missing logging todo

* Implement mock storage

* Add channel for auto-converting types on channel replies

* Simplify StorageBackend trait

* Refactor mock backend and use HashMap

* Remove sled for now

* Refactor serialization scheme

* Add storage traits docs

* Make transaction a custom trait. This way we can return stuff from the transactions themselves if needed.
2022-11-18 09:46:54 +01:00
Giacomo Pasini 0fcfd92a55
Initial network service (#2)
* initial network service

* [style] unpack inbound relay
2022-11-03 06:28:37 -07:00
Daniel Sanchez e9857d979b
Initial commit 2022-10-25 13:40:24 +02:00