Commit Graph

405 Commits

Author SHA1 Message Date
gusto 7f7a0db88a
Update rust and go versions for macos builds (#77)
* Add additional libs for macos nix env

* Add CoreFoundation and Security for macos

* Update rust 1.67.0 and go 1.19.5
2023-02-16 17:42:31 +02:00
Daniel Sanchez 8cc37385b3
Waku cached streams consensus adapter (#70)
* Added waku archive message to waku network backend

* Use cached streams in consensus waku adapter

* Fix mock test

* Add missing import

* Join requests tasks

* Use waku-bindings beta4

* Get stream from archive query method

* Set store protocol active for waku backend

* Implement local query stream response

* Add missing linking flags for new waku-bindings version

* Cleanup unbounded sender fuse/unwrap

* Clippy happy
2023-02-15 16:49:49 +01:00
Daniel Sanchez 1d195959d8
Add services and project information to project README (#72)
* Add services and project information to project README

* Better phrased nomos services
2023-02-10 17:07:04 +01:00
Al Liu 6b45bf408e
add Debug and constructor for CarnotSettings (#71) 2023-02-10 19:04:56 +08:00
Daniel Sanchez 320755d19d
Detach tx broadcast from mempool (#69)
* Remove send tx method from mempool network adapter

* Add error reporting to add_tx operation in mempool
Delegate broadcasting to external caller
2023-02-08 11:07:09 +01:00
Giacomo Pasini bbb783e1da
Complete consensus (#65)
* complete consensus

* review comments
2023-02-08 10:23:55 +01:00
gusto 5f21a2734a
Add endpoint to make a new peer conn for mockpool node (#68)
* Add enpoint to make a new peer conn for mockpool node

* Pass multiple addresses in connection request

* Join async reqs to network service, cleanup

* Collect and join reqs
2023-02-07 17:50:38 +02:00
Al Liu 146001c9fe
Mock mempool integration test (#66)
* finish mock mempool integration test

* use Log service for test

* remove unused example

* use millis and merge log PR
2023-02-07 17:13:22 +08:00
Daniel Sanchez c5ac1db44c
Network transaction broadcasting (#63)
* Broadcast transaction when validated in mempool

* Clippy happy

* Use standard bincode config for tx message decoding

* add send_transaction for mock (#64)

* add send_transaction for mock

* Use wire instead of direct bincode

* Use wire instead of direct bincode on deserialization

---------

Co-authored-by: Al Liu <scygliu1@gmail.com>
2023-02-06 12:49:24 +01:00
Al Liu 4705cc213b
Add constructor for LogSettings (#67)
* add constructor for LogSettings
2023-02-06 19:23:25 +08:00
Daniel Sanchez 9a8400f4ce
Use blake 2b instead of 2s for mockpool node TxId (#62)
* Use blake 2b instead of 2s

* Clippy happy
2023-02-02 00:20:16 -08:00
Al Liu 3d3d2760ec
Implement #31: Mock network backend (#41)
* relax trait bounds

* mock mempool

* remove unused generic

* add mock network test case

* fix some PR comments

* simplify match branch

* finish mempool adapter example

* clippy happy

* mock consensus
2023-02-01 21:58:23 +08:00
Daniel Sanchez 567188c248
Fix blake2b usage on mockpool node tx (#61) 2023-02-01 04:10:57 -08:00
gusto 57991d6e02
Docker image for nomos node (#60)
* Docker container for nomos node

* Remove peers from sample config

* Use mockpool-node and update config
2023-02-01 10:37:15 +02:00
Daniel Sanchez 1e20c3b6cc
Use commitee and view information for consensus network adapter (#57)
* Add committee and view information to network adapter

* Use committee and view on waku adapter

* Add committee and view info to flat view implementation.

* Clippy happy

* Rename flat -> root

* Split broadcast block iterator into l/r childs

* Extract topics to builder functions
2023-01-31 03:03:45 -08:00
Giacomo Pasini f5175c74c0
Add flat overlay (#55) 2023-01-27 10:37:04 +01:00
gusto 8e2346c29e
Rust 1.67 cargo clippy fix (#56)
* Run clippy --fix for updated lint requirements

* Fmt after clippy
2023-01-27 11:04:42 +02:00
Daniel Sanchez e007539531
This update fixes the default waku subscription (#54) 2023-01-26 06:53:27 -08:00
Daniel Sanchez dbe36bba3c
Mockpool node (#53)
* Create nodes folder
Kickstart mockpool node

* Create nodes folder
Added bridges file

* Added metrics to mempool

* Remove metrics from node

* Added mempool metrics bridge

* Pipe in mempool_metric bridge

* Add wakuinfo to waku network service

* Add waku network info bridge

* Added waku info bridge to node

* Use mock Tx wrapper over a string

* Create add tx http bridge

* Add tx bridge to http config

* Use hash for Tx

* Remove tracing subscriber from binary

* Fix bridges routes

* Added mimimal configuration example

* Remove subscribing to default waku pubsub topic

* Use addtx payload for tx

* Remove pub serde mod from core transaction

* Clippy happy

* Id from &Tx instead of owned value

* Removed mempool metrics feature
2023-01-25 07:24:33 -08:00
gusto 915ec00b34
Http service api (#44)
* Cargo http-service folder

* WIP: http server

* Revert comments in network service

* Router service and axum implementation structure

* Move bin contents to examples dir

* Add http service and server traits

* HttpMsg definition

* WIP: axum backend

* fix example in Cargo.toml

* Shared axum router and router modification methods

* Http example with axum and metrics service

* make project compile and add Error associated type

* Axum backend shared router fixes

* Dummy service implementation for http example

* remove unused clone on mutex

* Cargo http-service folder

* WIP: http server

* Revert comments in network service

* Router service and axum implementation structure

* Move bin contents to examples dir

* Add http service and server traits

* HttpMsg definition

* WIP: axum backend

* fix example in Cargo.toml

* Shared axum router and router modification methods

* Http example with axum and metrics service

* make project compile and add Error associated type

* Axum backend shared router fixes

* Dummy service implementation for http example

* remove unused clone on mutex

* Fix typos and remove unused code

* Fix failing tests when feature flags are not set

* Use bytes as a type for payload and response in http service

* Refactored http crate layout into differential services files

* First stab at router service

* Fully piped http bridge system

* Start building bridge helper function

* Refactor bridge builder helper and update example

* impl serialization for metrics data

* Get updated copy of router when processing request

* remove unused code

* fix typo

* [POC]: Http service: support add graphql handler (#47)

* WIP: add graphql endpoint

* support add graphql handler

* remove generic

* fix clippy warnings

* Add post put and patch handlers that expect bytes as body

* Graphql example file

* WIP: Use http post method for graphql related queries

* Parse graphql requests in handler

* Simplify handlers for post and other data methods

* Revert "Simplify handlers for post and other data methods"

This reverts commit 96f2b1821e.

* add tracing and remove comments

* Pass response bytes without any modifications

* Use receive_batch_json for gql request parsing

* Readme for running examples

* fix conflicts

* add a general helper function for graphql

* remove unused function

* cleanup code

* move schema initialization to handle function

* adapt metrics to http service

* fix clippy warnings

* remove unused fn

* fix clippy

* optimize example

Co-authored-by: gusto <bacvinka@gmail.com>
Co-authored-by: Gusto Bacvinka <augustinas.bacvinka@gmail.com>

* Fix cargo build without features

* Simplify handlers for routes with data

Co-authored-by: al8n <scygliu1@gmail.com>
Co-authored-by: Daniel Sanchez Quiros <sanchez.quiros.daniel@gmail.com>
2023-01-19 16:51:30 +02:00
gusto 613d9ac1b2
Add a step to run cargo build w/o features (#52) 2023-01-19 16:34:35 +02:00
gusto dbef2906b5
Autoauthor action (#51) 2023-01-18 17:10:13 +02:00
Giacomo Pasini efb1690872
Add transaction type (#50)
* Add transaction type
2023-01-18 15:02:58 +01:00
Daniel Sanchez 677d4a245c
Mockpool (#43)
* Use ids slices on mempool trait

* Create a mock mempool
Added placeholders for block/blockheader/blockid relations

* Use linked hashmap

* Fix bounds

* Remove unnecessary bound
2023-01-18 03:11:03 -08:00
Giacomo Pasini f5a1dd5513
Relax type bounds (#49) 2023-01-18 11:30:37 +01:00
Daniel Sanchez f1412b112e
Implemented mock fountain code protocol (#45) 2023-01-17 01:08:00 -08:00
Giacomo Pasini cfcc664e9e
Add dummy binary (#5)
* add dummy binary

* add flags for macos

* add metrics service

* rename Services to Nomos
2023-01-12 12:57:02 +01:00
Daniel Sanchez 4d43c18846
Add missing send + sync bounds to network backend state (#46) 2023-01-12 11:33:57 +01:00
Daniel Sanchez 7ff63d4824
Add fountain codes to consensus (#38)
* Add basic encode/decode test

* Use Stream for trait instead of Iterator

* Removed unnecessary pin

* Add custom fountain error

* Add failing path to tests

* Added docs

* Normalized chunks to bytes

* Added settings initialization for fountain codes trait

* Pipe fountain code through consensus

* Implement broadcast block and block reconstruction for Member Overlay

* Fix failing raptor test

* Use seed for raptorq tests

* Use const for topics instead of static

* Clippy happy
2023-01-10 12:58:51 +01:00
Giacomo Pasini 15f97dcace
fix consensus merge (#42) 2023-01-10 12:26:36 +01:00
Giacomo Pasini 0229337414
Add leadership stub (#22)
* add leadership stub

* move types to core

* clippy happy
2023-01-10 11:33:08 +01:00
Daniel Sanchez 9721e6f5fa
Mempool network adapter (#30)
* Refactor mempool naming and added settings to backend trait

* Implement mempool networking traits and waku backend

* Transaction networking plumbing in mempool service

* Make TransactionMsg generic
Use bincode to deserialize tx messages

* Make wakuadapter generic over tx and tx-id

* Fix wrong backend type bound

* Adapt to waku beta2

* Thread tx and id together for adapter and pool

* Panic on subscribing error

* Prune unnecessary Id type bounds

* Remove transaction placeholder

* Remove Id bound from waku adapter

* Remove empty transactions module
2023-01-09 17:22:46 +01:00
Daniel Sanchez 664cb3ccd8
Update waku-bindings to beta2 (#40) 2023-01-09 12:20:14 +01:00
Giacomo Pasini 3a72cddb81
remove unneeded generic parameters (#39) 2023-01-09 12:15:21 +01:00
Daniel Sanchez fc75ad1732
Fountain codes (#37)
* Add fountain module and raptorq basic implementation

* Add basic encode/decode test

* Use Stream for trait instead of Iterator

* Removed unnecessary pin

* Add custom fountain error

* Add failing path to tests

* Added docs
2023-01-05 10:35:12 +01:00
Daniel Sanchez 52708b8253
Metrics refactor (#20)
* metrics service

Co-authored-by: al8n <scygliu1@gmail.com>
2022-12-15 04:23:06 +13:00
Giacomo Pasini 539c986f69
Add mempool stub (#29)
* add mempool stub

* address review comments

* move base data types to nomos-core

* allow clippy warning
2022-12-14 15:30:45 +01:00
Daniel Sanchez fb3fd6f3b1
Nomos core (#28)
* Extract block to core crate

* Added linking flag for waku

* Cleanup imports

* Add missing core files

* Create more base mods
2022-12-13 15:35:11 +01:00
gusto ae97db3e0f
Lint and check all features (#27)
Added flags for the "Check" stage for linux and macos PR targets.
2022-12-13 13:33:51 +02:00
Giacomo Pasini 7cc9181574
Consensus backbone (#16)
* tmp

* add peers implem

* update return values

* Backport network consensus adapter (#26)

* Pipe network adapter with proper types over original implementation

* Hold relay on CarnotConsensus

* Scratch Network adapter methods

* Fix tests blocking CI

* Fix waku feature on network crate

* Fix waku_bindings refs

* Restructure consensus network

* Stream block chunk

* Pipe adapter creation with subscription

* Add placeholder proposal chunk and approval messages

* Implement waku backend

* Clippy happy

* Use full path for tokio oneshot and error types in message_subscriber_channel method

* Clean imports

* small fixes

Co-authored-by: Daniel Sanchez Quiros <sanchez.quiros.daniel@gmail.com>

Co-authored-by: Daniel Sanchez Quiros <sanchez.quiros.daniel@gmail.com>
2022-12-13 11:15:54 +01:00
gusto 5484cb7079
ci: add Jenkins and Docker file for PR and doc builds (#23)
* ci: add Jenkins and Docker file for PR and doc builds

* Fixing waku-bindings dependacy definitions

* Add post cleanup and musl-dev to Jenkinsfile

* Add go to the Dockerfile for ci

* GOCACHE set to tmp for ci builds

* Use slim-bullseye docker image for ci builds

* Update ci/Jenkinsfile.docs

Co-authored-by: Jakub <i+github@always.fail>

* Add image version and env vars for docs build

* Remove duplicated environment section

* Add missing rust dependencies

* Split jenkins file into linux and macos targets

* Removing github actions for PR checks

* Add explicit versions to the shell.nix dependencies

* Add jenkins libs required for nix

* Use default rust-bin version that includes clippy and fmt

* Add readme to ci folder

Co-authored-by: Jakub <i+github@always.fail>
2022-12-12 21:29:23 +02:00
Al Liu 8c1a521440
adapt to overwatch error handle (#24) 2022-12-12 23:06:15 +13:00
gusto ee0085e873
Continue CI steps even when one of them fails (#15)
* Continue CI steps even when one of them fails
* Build all features in CI
* Test all features in CI
2022-12-01 18:46:27 +02:00
Giacomo Pasini 0db0544a77
fix tests (#14) 2022-11-23 15:49:43 +01:00
Giacomo Pasini 90c7de6271
Split services into different crates (#13) 2022-11-23 14:43:22 +01:00
gusto 26e5580443
Add CI tests, clippy and fmt checks (#12)
Using logos-co/Overwatch CI scripts for tests and linting in PRs.
The idea is to use this for now, and later gather all common action definitions into one dedicated repository.

* Copied CI test, clippy and fmt steps from Overwatch
* Added go 1.19 to an action setup (required for building waku-rust-bindigs dep)
2022-11-23 15:10:17 +02:00
Al Liu e656fb2d9d
Bugfix: Forget to change `overwatch` to `overwatch_rs` (#11)
* direct clone on Bytes instead of deep copy(Bytes::copy_from_slice)

* change overwatch to overwatch_rs
2022-11-22 06:16:07 -08:00
Al Liu c127c1d18d
Fix cheap clone Bytes and Overwatch crate name update 2022-11-22 04:44:17 -08:00
Giacomo Pasini 3a15a9b722
Add log service (#4)
* add log service

* add ser/de to log config

* add futures dep
2022-11-21 15:35:52 +01:00
Daniel Sanchez dce6678904
Sled backend (#7)
* Add sled module

* Remove sled for now

* Implement sled as a storage backend

* Remove manual panic

* Refactor sled to adap to latest StorageBackend traits changes

* Use option bytes for sled transaction output

* Cleanup imports

* Add tests

* Removed unused scopes
2022-11-21 06:23:45 -08:00