mirror of
https://github.com/logos-blockchain/logos-blockchain-testing.git
synced 2026-01-02 13:23:13 +00:00
docs(book): sync with repo and fix mdbook test
This commit is contained in:
andrussal
parent
e1f299157b
commit
9655dae6a1
70
book/COMPREHENSIVE_REPO_SYNC_REVIEW.md
Normal file
70
book/COMPREHENSIVE_REPO_SYNC_REVIEW.md
Normal file
@ -0,0 +1,70 @@
|
||||
# Book → Repo Sync Review (2025-12-20)
|
||||
|
||||
Reviewed against `git rev-parse HEAD` at the time of writing, plus local working tree changes.
|
||||
|
||||
## Checks Run
|
||||
|
||||
- `mdbook build book`
|
||||
- `mdbook test book`
|
||||
- `cargo build -p doc-snippets`
|
||||
- Verified `book/src/SUMMARY.md` covers all pages in `book/src/` (no orphaned pages)
|
||||
- Verified all `scripts/...` paths referenced from the book exist
|
||||
- Compared `NOMOS_*` environment variables used in `scripts/`, `testing-framework/`, and `examples/` vs. `book/src/environment-variables.md`
|
||||
|
||||
## Findings / Fixes Applied
|
||||
|
||||
- `book/src/environment-variables.md` was not a complete reference: it missed multiple `NOMOS_*` variables used by the repo (scripts + framework). Added the missing variables and corrected a misleading note about `RUST_LOG` vs node logging.
|
||||
- `book/src/running-examples.md` “Quick Smoke Matrix” section didn’t reflect current `scripts/run/run-test-matrix.sh` flags. Added the commonly used options and clarified the relationship to `NOMOS_SKIP_IMAGE_BUILD`.
|
||||
- `book/src/part-iv.md` existed but was not in `book/src/SUMMARY.md`. Removed it so the rendered book doesn’t silently diverge from the filesystem.
|
||||
- `mdbook test book` was failing because:
|
||||
- Many Rust examples were written as ` ```rust` (doctested by default) but depend on workspace crates; they aren’t standalone doctest snippets.
|
||||
- Several unlabeled code blocks (e.g. tree/log output) were treated as Rust by rustdoc.
|
||||
- Updated code fences to ` ```rust,ignore` for non-standalone Rust examples and to ` ```text` for non-Rust output blocks so `mdbook test book` succeeds.
|
||||
|
||||
## Pages Reviewed (No Skips)
|
||||
|
||||
All pages under `book/src/` currently included by `book/src/SUMMARY.md`:
|
||||
|
||||
- `annotated-tree.md`
|
||||
- `api-levels.md`
|
||||
- `architecture-overview.md`
|
||||
- `authoring-scenarios.md`
|
||||
- `best-practices.md`
|
||||
- `chaos.md`
|
||||
- `ci-integration.md`
|
||||
- `custom-workload-example.md`
|
||||
- `design-rationale.md`
|
||||
- `dsl-cheat-sheet.md`
|
||||
- `environment-variables.md`
|
||||
- `examples-advanced.md`
|
||||
- `examples.md`
|
||||
- `extending.md`
|
||||
- `faq.md`
|
||||
- `glossary.md`
|
||||
- `internal-crate-reference.md`
|
||||
- `introduction.md`
|
||||
- `logging-observability.md`
|
||||
- `node-control.md`
|
||||
- `operations-overview.md`
|
||||
- `part-i.md`
|
||||
- `part-ii.md`
|
||||
- `part-iii.md`
|
||||
- `part-v.md`
|
||||
- `part-vi.md`
|
||||
- `prerequisites.md`
|
||||
- `project-context-primer.md`
|
||||
- `quickstart.md`
|
||||
- `runners.md`
|
||||
- `running-examples.md`
|
||||
- `running-scenarios.md`
|
||||
- `scenario-builder-ext-patterns.md`
|
||||
- `scenario-lifecycle.md`
|
||||
- `scenario-model.md`
|
||||
- `testing-philosophy.md`
|
||||
- `topology-chaos.md`
|
||||
- `troubleshooting.md`
|
||||
- `usage-patterns.md`
|
||||
- `what-you-will-learn.md`
|
||||
- `workloads.md`
|
||||
- `workspace-layout.md`
|
||||
|
||||
@ -2,7 +2,7 @@
|
||||
|
||||
Directory structure with key paths annotated:
|
||||
|
||||
```
|
||||
```text
|
||||
logos-blockchain-testing/
|
||||
├─ testing-framework/ # Core library crates
|
||||
│ ├─ configs/ # Node config builders, topology generation, tracing/logging config
|
||||
|
||||
@ -11,7 +11,7 @@ Both styles produce the same runtime behavior because they ultimately call the s
|
||||
|
||||
The DSL is implemented as extension traits (primarily `testing_framework_workflows::ScenarioBuilderExt`) on the core scenario builder.
|
||||
|
||||
```rust
|
||||
```rust,ignore
|
||||
use std::time::Duration;
|
||||
|
||||
use testing_framework_core::scenario::ScenarioBuilder;
|
||||
@ -34,7 +34,7 @@ let plan = ScenarioBuilder::topology_with(|t| t.network_star().validators(3).exe
|
||||
|
||||
Direct instantiation gives you explicit control over the concrete types you attach:
|
||||
|
||||
```rust
|
||||
```rust,ignore
|
||||
use std::{
|
||||
num::{NonZeroU64, NonZeroUsize},
|
||||
time::Duration,
|
||||
@ -88,7 +88,7 @@ These bundled expectations are attached automatically whenever you call `.with_w
|
||||
|
||||
Mixing is common: use the DSL for built-ins, and direct instantiation for custom pieces.
|
||||
|
||||
```rust
|
||||
```rust,ignore
|
||||
use std::time::Duration;
|
||||
|
||||
use testing_framework_core::scenario::ScenarioBuilder;
|
||||
|
||||
@ -165,7 +165,7 @@ These binaries use the framework API (`ScenarioBuilder`) to construct and execut
|
||||
|
||||
Scenarios are defined using a fluent builder pattern:
|
||||
|
||||
```rust
|
||||
```rust,ignore
|
||||
use std::time::Duration;
|
||||
|
||||
use testing_framework_core::scenario::ScenarioBuilder;
|
||||
|
||||
@ -33,7 +33,7 @@ This page collects proven patterns for authoring, running, and maintaining test
|
||||
|
||||
**Example: Topology preset**
|
||||
|
||||
```rust
|
||||
```rust,ignore
|
||||
pub fn standard_da_topology() -> GeneratedTopology {
|
||||
TopologyBuilder::new()
|
||||
.network_star()
|
||||
@ -45,7 +45,7 @@ pub fn standard_da_topology() -> GeneratedTopology {
|
||||
|
||||
**Example: Shared constants**
|
||||
|
||||
```rust
|
||||
```rust,ignore
|
||||
pub const STANDARD_TX_RATE: f64 = 10.0;
|
||||
pub const STANDARD_DA_CHANNEL_RATE: f64 = 2.0;
|
||||
pub const SHORT_RUN_DURATION: Duration = Duration::from_secs(60);
|
||||
@ -140,7 +140,7 @@ POL_PROOF_DEV_MODE=true cargo run -p runner-examples --bin local_runner
|
||||
```
|
||||
|
||||
**DON'T: Use tiny durations**
|
||||
```rust
|
||||
```rust,ignore
|
||||
// BAD: Not enough time for blocks to propagate
|
||||
.with_run_duration(Duration::from_secs(5))
|
||||
|
||||
@ -149,7 +149,7 @@ POL_PROOF_DEV_MODE=true cargo run -p runner-examples --bin local_runner
|
||||
```
|
||||
|
||||
**DON'T: Ignore cleanup failures**
|
||||
```rust
|
||||
```rust,ignore
|
||||
// BAD: Next run inherits leaked state
|
||||
runner.run(&mut scenario).await?;
|
||||
// forgot to call cleanup or use CleanupGuard
|
||||
@ -160,7 +160,7 @@ runner.run(&mut scenario).await?;
|
||||
```
|
||||
|
||||
**DON'T: Mix concerns in one scenario**
|
||||
```rust
|
||||
```rust,ignore
|
||||
// BAD: Hard to debug when it fails
|
||||
.transactions_with(|tx| tx.rate(50).users(100)) // high load
|
||||
.chaos_with(|c| c.restart().min_delay(...)) // AND chaos
|
||||
@ -173,7 +173,7 @@ runner.run(&mut scenario).await?;
|
||||
```
|
||||
|
||||
**DON'T: Hardcode paths or ports**
|
||||
```rust
|
||||
```rust,ignore
|
||||
// BAD: Breaks on different machines
|
||||
let path = PathBuf::from("/home/user/circuits/kzgrs_test_params");
|
||||
let port = 9000; // might conflict
|
||||
|
||||
@ -20,7 +20,7 @@ recovery. The built-in restart workload lives in
|
||||
`NodeControlHandle` support (e.g., compose) for chaos workloads.
|
||||
|
||||
## Usage
|
||||
```rust
|
||||
```rust,ignore
|
||||
use std::time::Duration;
|
||||
|
||||
use testing_framework_core::scenario::ScenarioBuilder;
|
||||
|
||||
@ -12,7 +12,7 @@ Key ideas:
|
||||
- **init**: derive inputs from the generated topology (e.g., pick a target node).
|
||||
- **start**: drive async activity using the shared `RunContext`.
|
||||
|
||||
```rust
|
||||
```rust,ignore
|
||||
use async_trait::async_trait;
|
||||
use testing_framework_core::{
|
||||
scenario::{DynError, Expectation, RunContext, RunMetrics, Workload},
|
||||
@ -85,7 +85,7 @@ Key ideas:
|
||||
- **start_capture**: snapshot baseline if needed (not used here).
|
||||
- **evaluate**: assert the condition after workloads finish.
|
||||
|
||||
```rust
|
||||
```rust,ignore
|
||||
use async_trait::async_trait;
|
||||
use testing_framework_core::scenario::{DynError, Expectation, RunContext};
|
||||
|
||||
|
||||
@ -4,7 +4,7 @@ Quick reference for the scenario builder DSL. All methods are chainable.
|
||||
|
||||
## Imports
|
||||
|
||||
```rust
|
||||
```rust,ignore
|
||||
use std::time::Duration;
|
||||
|
||||
use testing_framework_core::scenario::{Deployer, ScenarioBuilder};
|
||||
@ -16,7 +16,7 @@ use testing_framework_workflows::{ChaosBuilderExt, ScenarioBuilderExt};
|
||||
|
||||
## Topology
|
||||
|
||||
```rust
|
||||
```rust,ignore
|
||||
use testing_framework_core::scenario::{Builder, ScenarioBuilder};
|
||||
|
||||
pub fn topology() -> Builder<()> {
|
||||
@ -30,7 +30,7 @@ pub fn topology() -> Builder<()> {
|
||||
|
||||
## Wallets
|
||||
|
||||
```rust
|
||||
```rust,ignore
|
||||
use testing_framework_core::scenario::ScenarioBuilder;
|
||||
use testing_framework_workflows::ScenarioBuilderExt;
|
||||
|
||||
@ -43,7 +43,7 @@ pub fn wallets_plan() -> testing_framework_core::scenario::Scenario<()> {
|
||||
|
||||
## Transaction Workload
|
||||
|
||||
```rust
|
||||
```rust,ignore
|
||||
use testing_framework_core::scenario::ScenarioBuilder;
|
||||
use testing_framework_workflows::ScenarioBuilderExt;
|
||||
|
||||
@ -60,7 +60,7 @@ pub fn transactions_plan() -> testing_framework_core::scenario::Scenario<()> {
|
||||
|
||||
## DA Workload
|
||||
|
||||
```rust
|
||||
```rust,ignore
|
||||
use testing_framework_core::scenario::ScenarioBuilder;
|
||||
use testing_framework_workflows::ScenarioBuilderExt;
|
||||
|
||||
@ -78,7 +78,7 @@ pub fn da_plan() -> testing_framework_core::scenario::Scenario<()> {
|
||||
|
||||
## Chaos Workload (Requires `enable_node_control()`)
|
||||
|
||||
```rust
|
||||
```rust,ignore
|
||||
use std::time::Duration;
|
||||
|
||||
use testing_framework_core::scenario::{NodeControlCapability, ScenarioBuilder};
|
||||
@ -100,7 +100,7 @@ pub fn chaos_plan() -> testing_framework_core::scenario::Scenario<NodeControlCap
|
||||
|
||||
## Expectations
|
||||
|
||||
```rust
|
||||
```rust,ignore
|
||||
use testing_framework_core::scenario::ScenarioBuilder;
|
||||
use testing_framework_workflows::ScenarioBuilderExt;
|
||||
|
||||
@ -113,7 +113,7 @@ pub fn expectations_plan() -> testing_framework_core::scenario::Scenario<()> {
|
||||
|
||||
## Run Duration
|
||||
|
||||
```rust
|
||||
```rust,ignore
|
||||
use std::time::Duration;
|
||||
|
||||
use testing_framework_core::scenario::ScenarioBuilder;
|
||||
@ -128,7 +128,7 @@ pub fn run_duration_plan() -> testing_framework_core::scenario::Scenario<()> {
|
||||
|
||||
## Build
|
||||
|
||||
```rust
|
||||
```rust,ignore
|
||||
use testing_framework_core::scenario::ScenarioBuilder;
|
||||
use testing_framework_workflows::ScenarioBuilderExt;
|
||||
|
||||
@ -139,7 +139,7 @@ pub fn build_plan() -> testing_framework_core::scenario::Scenario<()> {
|
||||
|
||||
## Deployers
|
||||
|
||||
```rust
|
||||
```rust,ignore
|
||||
use testing_framework_runner_compose::ComposeDeployer;
|
||||
use testing_framework_runner_k8s::K8sDeployer;
|
||||
use testing_framework_runner_local::LocalDeployer;
|
||||
@ -158,7 +158,7 @@ pub fn deployers() {
|
||||
|
||||
## Execution
|
||||
|
||||
```rust
|
||||
```rust,ignore
|
||||
use anyhow::Result;
|
||||
use testing_framework_core::scenario::{Deployer, ScenarioBuilder};
|
||||
use testing_framework_runner_local::LocalDeployer;
|
||||
@ -179,7 +179,7 @@ pub async fn execution() -> Result<()> {
|
||||
|
||||
## Complete Example
|
||||
|
||||
```rust
|
||||
```rust,ignore
|
||||
use std::time::Duration;
|
||||
|
||||
use anyhow::Result;
|
||||
|
||||
@ -77,8 +77,10 @@ Required for compose and k8s runners:
|
||||
| Variable | Required | Default | Effect |
|
||||
|----------|----------|---------|--------|
|
||||
| `NOMOS_TESTNET_IMAGE` | Yes (compose/k8s) | `logos-blockchain-testing:local` | Docker image tag for node containers |
|
||||
| `NOMOS_TESTNET_IMAGE_PULL_POLICY` | No | `IfNotPresent` (local) / `Always` (ECR) | K8s `imagePullPolicy` used by the runner |
|
||||
| `NOMOS_BINARIES_TAR` | No | — | Path to prebuilt bundle (`.tar.gz`) for image build |
|
||||
| `NOMOS_SKIP_IMAGE_BUILD` | No | 0 | Skip image rebuild (compose/k8s); assumes image already exists |
|
||||
| `NOMOS_FORCE_IMAGE_BUILD` | No | 0 | Force rebuilding the image even when the script would normally skip it (e.g. non-local k8s) |
|
||||
|
||||
**Example:**
|
||||
|
||||
@ -103,7 +105,18 @@ Circuit asset configuration for DA workloads:
|
||||
|----------|---------|--------|
|
||||
| `NOMOS_KZGRS_PARAMS_PATH` | `testing-framework/assets/stack/kzgrs_test_params/kzgrs_test_params` | Path to KZG proving key file |
|
||||
| `NOMOS_KZG_DIR_REL` | `testing-framework/assets/stack/kzgrs_test_params` | Directory containing KZG assets (relative to workspace root) |
|
||||
| `NOMOS_KZG_FILE` | `kzgrs_test_params` | Filename of the proving key within `NOMOS_KZG_DIR_REL` |
|
||||
| `NOMOS_KZG_CONTAINER_PATH` | `/kzgrs_test_params/kzgrs_test_params` | File path where the node expects KZG params inside containers |
|
||||
| `NOMOS_KZG_MODE` | Runner-specific | K8s only: `hostPath` (mount from host) or `inImage` (embed into image) |
|
||||
| `NOMOS_KZG_IN_IMAGE_PARAMS_PATH` | `/opt/nomos/kzg-params/kzgrs_test_params` | K8s `inImage` mode: where the proving key is stored inside the image |
|
||||
| `VERSION` | From `versions.env` | Circuit release tag (used by helper scripts) |
|
||||
| `NOMOS_CIRCUITS` | — | Directory containing fetched circuit bundles (set by `scripts/setup/setup-circuits-stack.sh`) |
|
||||
| `NOMOS_CIRCUITS_VERSION` | — | Legacy alias for `VERSION` (supported by some build scripts) |
|
||||
| `NOMOS_CIRCUITS_PLATFORM` | Auto-detected | Override circuits platform (e.g. `linux-x86_64`, `macos-aarch64`) |
|
||||
| `NOMOS_CIRCUITS_HOST_DIR_REL` | `.tmp/nomos-circuits-host` | Output dir for host circuits bundle (relative to repo root) |
|
||||
| `NOMOS_CIRCUITS_LINUX_DIR_REL` | `.tmp/nomos-circuits-linux` | Output dir for linux circuits bundle (relative to repo root) |
|
||||
| `NOMOS_CIRCUITS_NONINTERACTIVE` | 0 | Set to `1` to overwrite outputs without prompting in setup scripts |
|
||||
| `NOMOS_CIRCUITS_REBUILD_RAPIDSNARK` | 0 | Set to `1` to force rebuilding rapidsnark (host bundle only) |
|
||||
|
||||
**Example:**
|
||||
|
||||
@ -127,7 +140,7 @@ Control node log output (not framework runner logs):
|
||||
| `NOMOS_TESTS_KEEP_LOGS` | 0 | Keep per-run temporary directories (useful for debugging/CI artifacts) |
|
||||
| `NOMOS_TESTS_TRACING` | false | Enable debug tracing preset (combine with `NOMOS_LOG_DIR` unless external tracing backends configured) |
|
||||
|
||||
**Important:** Nodes ignore `RUST_LOG` and only respond to `NOMOS_*` variables.
|
||||
**Important:** Node logging ignores `RUST_LOG`; use `NOMOS_LOG_LEVEL` and `NOMOS_LOG_FILTER` for node logs.
|
||||
|
||||
**Example:**
|
||||
|
||||
@ -251,7 +264,9 @@ Platform-specific build configuration:
|
||||
| Variable | Default | Effect |
|
||||
|----------|---------|--------|
|
||||
| `NOMOS_BUNDLE_DOCKER_PLATFORM` | Host arch | Docker platform for bundle builds: `linux/arm64` or `linux/amd64` (macOS/Windows hosts) |
|
||||
| `NOMOS_BIN_PLATFORM` | — | Legacy alias for `NOMOS_BUNDLE_DOCKER_PLATFORM` |
|
||||
| `COMPOSE_CIRCUITS_PLATFORM` | Host arch | Circuits platform for image builds: `linux-aarch64` or `linux-x86_64` |
|
||||
| `NOMOS_EXTRA_FEATURES` | — | Extra cargo features to enable when building bundles (used by `scripts/build/build-bundle.sh`) |
|
||||
|
||||
**macOS / Apple Silicon:**
|
||||
|
||||
@ -273,6 +288,14 @@ Timeout and performance tuning:
|
||||
|----------|---------|--------|
|
||||
| `SLOW_TEST_ENV` | false | Doubles built-in readiness timeouts (useful in CI / constrained laptops) |
|
||||
| `TESTNET_PRINT_ENDPOINTS` | 0 | Print `TESTNET_ENDPOINTS` / `TESTNET_PPROF` lines during deploy (set automatically by `scripts/run/run-examples.sh`) |
|
||||
| `NOMOS_DISPERSAL_TIMEOUT_SECS` | 20 | DA dispersal timeout (seconds) |
|
||||
| `NOMOS_RETRY_COOLDOWN_SECS` | 3 | Cooldown between retries (seconds) |
|
||||
| `NOMOS_GRACE_PERIOD_SECS` | 1200 | Grace period before enforcing strict time-based expectations (seconds) |
|
||||
| `NOMOS_PRUNE_DURATION_SECS` | 30 | Prune step duration (seconds) |
|
||||
| `NOMOS_PRUNE_INTERVAL_SECS` | 5 | Interval between prune cycles (seconds) |
|
||||
| `NOMOS_SHARE_DURATION_SECS` | 5 | Share duration (seconds) |
|
||||
| `NOMOS_COMMITMENTS_WAIT_SECS` | 1 | Commitments wait duration (seconds) |
|
||||
| `NOMOS_SDP_TRIGGER_DELAY_SECS` | 5 | SDP trigger delay (seconds) |
|
||||
|
||||
**Example:**
|
||||
|
||||
@ -292,6 +315,9 @@ Node-level configuration passed through to nomos-node/nomos-executor:
|
||||
|----------|---------|--------|
|
||||
| `CONSENSUS_SLOT_TIME` | — | Consensus slot time (seconds) |
|
||||
| `CONSENSUS_ACTIVE_SLOT_COEFF` | — | Active slot coefficient (0.0-1.0) |
|
||||
| `NOMOS_USE_AUTONAT` | Unset | If set, use AutoNAT instead of a static loopback address for libp2p NAT settings |
|
||||
| `NOMOS_CFGSYNC_PORT` | 4400 | Port used for cfgsync service inside the stack |
|
||||
| `NOMOS_TIME_BACKEND` | `monotonic` | Select time backend (used by compose/k8s stack scripts and deployers) |
|
||||
|
||||
**Example:**
|
||||
|
||||
@ -334,6 +360,10 @@ Variables used by helper scripts (`scripts/run/run-examples.sh`, etc.):
|
||||
|----------|---------|--------|
|
||||
| `NOMOS_NODE_REV` | From `versions.env` | nomos-node git revision to build/fetch |
|
||||
| `NOMOS_BUNDLE_VERSION` | From `versions.env` | Bundle schema version |
|
||||
| `NOMOS_IMAGE_SELECTION` | — | Internal: image selection mode set by `run-examples.sh` (`local`/`ecr`/`auto`) |
|
||||
| `NOMOS_NODE_APPLY_PATCHES` | 1 | Set to `0` to disable applying local patches when building bundles |
|
||||
| `NOMOS_NODE_PATCH_DIR` | `patches/nomos-node` | Patch directory applied to nomos-node checkout during bundle builds |
|
||||
| `NOMOS_NODE_PATCH_LEVEL` | — | Patch application level (`all` or an integer) for bundle builds |
|
||||
|
||||
---
|
||||
|
||||
|
||||
@ -21,7 +21,7 @@ Realistic advanced scenarios demonstrating framework capabilities for production
|
||||
|
||||
Test consensus under progressively increasing transaction load:
|
||||
|
||||
```rust
|
||||
```rust,ignore
|
||||
use std::time::Duration;
|
||||
|
||||
use anyhow::Result;
|
||||
@ -56,7 +56,7 @@ pub async fn load_progression_test() -> Result<()> {
|
||||
|
||||
Run high transaction and DA load for extended duration:
|
||||
|
||||
```rust
|
||||
```rust,ignore
|
||||
use std::time::Duration;
|
||||
|
||||
use anyhow::Result;
|
||||
@ -87,7 +87,7 @@ pub async fn sustained_load_test() -> Result<()> {
|
||||
|
||||
Frequent node restarts with active traffic:
|
||||
|
||||
```rust
|
||||
```rust,ignore
|
||||
use std::time::Duration;
|
||||
|
||||
use anyhow::Result;
|
||||
|
||||
@ -25,7 +25,7 @@ and expectations.
|
||||
|
||||
Minimal test that validates basic block production:
|
||||
|
||||
```rust
|
||||
```rust,ignore
|
||||
use std::time::Duration;
|
||||
|
||||
use anyhow::Result;
|
||||
@ -53,7 +53,7 @@ pub async fn simple_consensus() -> Result<()> {
|
||||
|
||||
Test consensus under transaction load:
|
||||
|
||||
```rust
|
||||
```rust,ignore
|
||||
use std::time::Duration;
|
||||
|
||||
use anyhow::Result;
|
||||
@ -83,7 +83,7 @@ pub async fn transaction_workload() -> Result<()> {
|
||||
|
||||
Combined test stressing both transaction and DA layers:
|
||||
|
||||
```rust
|
||||
```rust,ignore
|
||||
use std::time::Duration;
|
||||
|
||||
use anyhow::Result;
|
||||
@ -114,7 +114,7 @@ pub async fn da_and_transactions() -> Result<()> {
|
||||
|
||||
Test system resilience under node restarts:
|
||||
|
||||
```rust
|
||||
```rust,ignore
|
||||
use std::time::Duration;
|
||||
|
||||
use anyhow::Result;
|
||||
|
||||
@ -13,7 +13,7 @@ This guide shows how to extend the framework with custom workloads, expectations
|
||||
|
||||
**Trait outline:**
|
||||
|
||||
```rust
|
||||
```rust,ignore
|
||||
use async_trait::async_trait;
|
||||
use testing_framework_core::scenario::{
|
||||
DynError, Expectation, RunContext, RunMetrics, Workload,
|
||||
@ -100,7 +100,7 @@ See [Example: New Workload & Expectation](custom-workload-example.md) for a comp
|
||||
|
||||
**Trait outline:**
|
||||
|
||||
```rust
|
||||
```rust,ignore
|
||||
use async_trait::async_trait;
|
||||
use testing_framework_core::scenario::{DynError, Expectation, RunContext};
|
||||
|
||||
@ -176,7 +176,7 @@ impl Expectation for MyExpectation {
|
||||
|
||||
**Trait outline:**
|
||||
|
||||
```rust
|
||||
```rust,ignore
|
||||
use async_trait::async_trait;
|
||||
use testing_framework_core::scenario::{
|
||||
CleanupGuard, Deployer, DynError, Metrics, NodeClients, RunContext, Runner, Scenario,
|
||||
@ -253,7 +253,7 @@ impl Deployer<()> for MyDeployer {
|
||||
|
||||
**Example:**
|
||||
|
||||
```rust
|
||||
```rust,ignore
|
||||
use testing_framework_core::topology::{
|
||||
config::TopologyBuilder,
|
||||
configs::network::Libp2pNetworkLayout,
|
||||
@ -279,7 +279,7 @@ impl TopologyBuilderExt for TopologyBuilder {
|
||||
|
||||
To expose your custom workload through the high-level DSL, add a trait extension:
|
||||
|
||||
```rust
|
||||
```rust,ignore
|
||||
use async_trait::async_trait;
|
||||
use testing_framework_core::scenario::{DynError, RunContext, ScenarioBuilder, Workload};
|
||||
|
||||
@ -344,7 +344,7 @@ impl MyWorkloadDsl for ScenarioBuilder {
|
||||
|
||||
Users can then call:
|
||||
|
||||
```rust
|
||||
```rust,ignore
|
||||
ScenarioBuilder::topology_with(|t| t.network_star().validators(1).executors(1))
|
||||
.my_workload_with(|w| {
|
||||
w.target_rate(10)
|
||||
|
||||
@ -30,7 +30,7 @@ High-level roles of the crates that make up the framework:
|
||||
### Adding a New Workload
|
||||
|
||||
1. **Define the workload** in `testing-framework/workflows/src/workloads/your_workload.rs`:
|
||||
```rust
|
||||
```rust,ignore
|
||||
use async_trait::async_trait;
|
||||
use testing_framework_core::scenario::{DynError, RunContext, Workload};
|
||||
|
||||
@ -50,7 +50,7 @@ impl Workload for YourWorkload {
|
||||
```
|
||||
|
||||
2. **Add builder extension** in `testing-framework/workflows/src/builder/mod.rs`:
|
||||
```rust
|
||||
```rust,ignore
|
||||
pub struct YourWorkloadBuilder;
|
||||
|
||||
impl YourWorkloadBuilder {
|
||||
@ -65,7 +65,7 @@ pub trait ScenarioBuilderExt: Sized {
|
||||
```
|
||||
|
||||
3. **Use in examples** in `examples/src/bin/your_scenario.rs`:
|
||||
```rust
|
||||
```rust,ignore
|
||||
use testing_framework_core::scenario::ScenarioBuilder;
|
||||
|
||||
pub struct YourWorkloadBuilder;
|
||||
@ -102,7 +102,7 @@ pub fn use_in_examples() {
|
||||
### Adding a New Expectation
|
||||
|
||||
1. **Define the expectation** in `testing-framework/workflows/src/expectations/your_expectation.rs`:
|
||||
```rust
|
||||
```rust,ignore
|
||||
use async_trait::async_trait;
|
||||
use testing_framework_core::scenario::{DynError, Expectation, RunContext};
|
||||
|
||||
@ -122,7 +122,7 @@ impl Expectation for YourExpectation {
|
||||
```
|
||||
|
||||
2. **Add builder extension** in `testing-framework/workflows/src/builder/mod.rs`:
|
||||
```rust
|
||||
```rust,ignore
|
||||
use testing_framework_core::scenario::ScenarioBuilder;
|
||||
|
||||
pub trait YourExpectationDslExt: Sized {
|
||||
@ -145,7 +145,7 @@ pub fn use_in_examples() {
|
||||
### Adding a New Deployer
|
||||
|
||||
1. **Implement `Deployer` trait** in `testing-framework/runners/your_runner/src/deployer.rs`:
|
||||
```rust
|
||||
```rust,ignore
|
||||
use async_trait::async_trait;
|
||||
use testing_framework_core::scenario::{Deployer, Runner, Scenario};
|
||||
|
||||
|
||||
@ -18,7 +18,7 @@ declarative, observable, and portable across environments.
|
||||
|
||||
Here's the conceptual shape of every test you'll write:
|
||||
|
||||
```rust
|
||||
```rust,ignore
|
||||
// 1. Define the cluster
|
||||
let scenario = ScenarioBuilder::topology_with(|t| {
|
||||
t.network_star()
|
||||
|
||||
@ -290,7 +290,7 @@ POL_PROOF_DEV_MODE=true scripts/run/run-examples.sh -t 60 -v 3 -e 1 compose
|
||||
|
||||
**Example usage in expectations:**
|
||||
|
||||
```rust
|
||||
```rust,ignore
|
||||
use testing_framework_core::scenario::{DynError, RunContext};
|
||||
|
||||
async fn evaluate(ctx: &RunContext) -> Result<(), DynError> {
|
||||
|
||||
@ -32,7 +32,7 @@ The `BlockFeed` is a broadcast stream of block observations that allows workload
|
||||
|
||||
BlockFeed is available through `RunContext`:
|
||||
|
||||
```rust
|
||||
```rust,ignore
|
||||
let block_feed = ctx.block_feed();
|
||||
```
|
||||
|
||||
@ -42,7 +42,7 @@ Expectations typically use BlockFeed to verify block production and inclusion of
|
||||
|
||||
**Example: Counting blocks during a run**
|
||||
|
||||
```rust
|
||||
```rust,ignore
|
||||
use std::sync::{
|
||||
Arc,
|
||||
atomic::{AtomicU64, Ordering},
|
||||
@ -112,7 +112,7 @@ impl Expectation for MinimumBlocksExpectation {
|
||||
|
||||
**Example: Inspecting block contents**
|
||||
|
||||
```rust
|
||||
```rust,ignore
|
||||
use testing_framework_core::scenario::{DynError, RunContext};
|
||||
|
||||
async fn start_capture(ctx: &RunContext) -> Result<(), DynError> {
|
||||
@ -152,7 +152,7 @@ Workloads can use BlockFeed to coordinate timing or wait for specific conditions
|
||||
|
||||
**Example: Wait for N blocks before starting**
|
||||
|
||||
```rust
|
||||
```rust,ignore
|
||||
use async_trait::async_trait;
|
||||
use testing_framework_core::scenario::{DynError, RunContext, Workload};
|
||||
|
||||
@ -196,7 +196,7 @@ impl Workload for DelayedWorkload {
|
||||
|
||||
**Example: Rate limiting based on block production**
|
||||
|
||||
```rust
|
||||
```rust,ignore
|
||||
use testing_framework_core::scenario::{DynError, RunContext};
|
||||
|
||||
async fn generate_request() -> Option<()> {
|
||||
@ -245,7 +245,7 @@ async fn start(ctx: &RunContext) -> Result<(), DynError> {
|
||||
|
||||
Example direct polling in expectations:
|
||||
|
||||
```rust
|
||||
```rust,ignore
|
||||
use testing_framework_core::scenario::{DynError, RunContext};
|
||||
|
||||
async fn evaluate(ctx: &RunContext) -> Result<(), DynError> {
|
||||
@ -264,7 +264,7 @@ async fn evaluate(ctx: &RunContext) -> Result<(), DynError> {
|
||||
|
||||
Access aggregated statistics without subscribing to the feed:
|
||||
|
||||
```rust
|
||||
```rust,ignore
|
||||
use testing_framework_core::scenario::{DynError, RunContext};
|
||||
|
||||
async fn evaluate(ctx: &RunContext, expected_min: u64) -> Result<(), DynError> {
|
||||
@ -341,7 +341,7 @@ which describes the proposed `block_peer`/`unblock_peer` API (not yet implemente
|
||||
|
||||
Check for control support and use it conditionally:
|
||||
|
||||
```rust
|
||||
```rust,ignore
|
||||
use async_trait::async_trait;
|
||||
use testing_framework_core::scenario::{DynError, RunContext, Workload};
|
||||
|
||||
@ -370,7 +370,7 @@ scenario builder and deploy with a runner that supports it.
|
||||
|
||||
The `NodeControlHandle` trait currently provides:
|
||||
|
||||
```rust
|
||||
```rust,ignore
|
||||
use async_trait::async_trait;
|
||||
use testing_framework_core::scenario::DynError;
|
||||
|
||||
|
||||
@ -1,4 +0,0 @@
|
||||
# Part IV — Appendix
|
||||
|
||||
Quick-reference material and supporting guidance to keep scenarios discoverable,
|
||||
debuggable, and consistent.
|
||||
@ -26,7 +26,7 @@ NOMOS_BUNDLE_VERSION=v1
|
||||
- CI workflows
|
||||
|
||||
**Error if missing:**
|
||||
```
|
||||
```text
|
||||
ERROR: versions.env not found at repository root
|
||||
This file is required and should define:
|
||||
VERSION=<circuit release tag>
|
||||
@ -140,7 +140,7 @@ cargo run -p runner-examples --bin local_runner
|
||||
|
||||
**Error without assets:**
|
||||
|
||||
```
|
||||
```text
|
||||
Error: Custom { kind: NotFound, error: "Circuit file not found at: testing-framework/assets/stack/kzgrs_test_params/kzgrs_test_params" }
|
||||
```
|
||||
|
||||
@ -283,4 +283,3 @@ These scripts:
|
||||
- [Running Examples](running-examples.md) — Learn how to run scenarios
|
||||
- [Environment Variables](environment-variables.md) — Full variable reference
|
||||
- [Troubleshooting](troubleshooting.md) — Common issues and fixes
|
||||
|
||||
|
||||
@ -69,7 +69,7 @@ flowchart LR
|
||||
|
||||
## Quick Example
|
||||
|
||||
```rust
|
||||
```rust,ignore
|
||||
use std::time::Duration;
|
||||
|
||||
use testing_framework_core::scenario::ScenarioBuilder;
|
||||
|
||||
@ -37,7 +37,7 @@ POL_PROOF_DEV_MODE=true cargo run -p runner-examples --bin local_runner
|
||||
|
||||
**Core API Pattern** (simplified example):
|
||||
|
||||
```rust
|
||||
```rust,ignore
|
||||
use std::time::Duration;
|
||||
|
||||
use anyhow::Result;
|
||||
@ -88,7 +88,7 @@ Let's unpack the code:
|
||||
|
||||
### 1. Topology Configuration
|
||||
|
||||
```rust
|
||||
```rust,ignore
|
||||
use testing_framework_core::scenario::ScenarioBuilder;
|
||||
|
||||
pub fn step_1_topology() -> testing_framework_core::scenario::Builder<()> {
|
||||
@ -104,7 +104,7 @@ This defines **what** your test network looks like.
|
||||
|
||||
### 2. Wallet Seeding
|
||||
|
||||
```rust
|
||||
```rust,ignore
|
||||
use testing_framework_core::scenario::ScenarioBuilder;
|
||||
use testing_framework_workflows::ScenarioBuilderExt;
|
||||
|
||||
@ -117,7 +117,7 @@ Provides funded accounts for transaction submission.
|
||||
|
||||
### 3. Workloads
|
||||
|
||||
```rust
|
||||
```rust,ignore
|
||||
use testing_framework_core::scenario::ScenarioBuilder;
|
||||
use testing_framework_workflows::ScenarioBuilderExt;
|
||||
|
||||
@ -140,7 +140,7 @@ Generates both transaction and DA traffic to stress both subsystems.
|
||||
|
||||
### 4. Expectation
|
||||
|
||||
```rust
|
||||
```rust,ignore
|
||||
use testing_framework_core::scenario::ScenarioBuilder;
|
||||
use testing_framework_workflows::ScenarioBuilderExt;
|
||||
|
||||
@ -153,7 +153,7 @@ This says **what success means**: blocks must be produced continuously.
|
||||
|
||||
### 5. Run Duration
|
||||
|
||||
```rust
|
||||
```rust,ignore
|
||||
use std::time::Duration;
|
||||
|
||||
use testing_framework_core::scenario::ScenarioBuilder;
|
||||
@ -167,7 +167,7 @@ Run for 60 seconds (~27 blocks with default 2s slots, 0.9 coefficient). Framewor
|
||||
|
||||
### 6. Deploy and Execute
|
||||
|
||||
```rust
|
||||
```rust,ignore
|
||||
use anyhow::Result;
|
||||
use testing_framework_core::scenario::{Deployer, ScenarioBuilder};
|
||||
use testing_framework_runner_local::LocalDeployer;
|
||||
@ -256,7 +256,7 @@ Then run your compose scenario as usual (the environment variables enable PromQL
|
||||
|
||||
**In code:** Just swap the deployer:
|
||||
|
||||
```rust
|
||||
```rust,ignore
|
||||
use anyhow::Result;
|
||||
use testing_framework_core::scenario::{Deployer, ScenarioBuilder};
|
||||
use testing_framework_runner_compose::ComposeDeployer;
|
||||
|
||||
@ -42,11 +42,19 @@ scripts/run/run-test-matrix.sh -t 120 -v 1 -e 1
|
||||
|
||||
This runs host, compose, and k8s modes with various image-build configurations. Useful after making runner/image/script changes. Forwards `--metrics-*` options through to `scripts/run/run-examples.sh`.
|
||||
|
||||
**Common options:**
|
||||
- `--modes host,compose,k8s` — Restrict which modes run
|
||||
- `--no-clean` — Skip `scripts/ops/clean.sh` step
|
||||
- `--no-bundles` — Skip `scripts/build/build-bundle.sh` (reuses existing `.tmp` tarballs)
|
||||
- `--no-image-build` — Skip the “rebuild image” variants in the matrix (compose/k8s)
|
||||
- `--allow-nonzero-progress` — Soft-pass expectation failures if logs show non-zero progress (local iteration only)
|
||||
- `--force-k8s-image-build` — Allow the k8s image-build variant even on non-docker-desktop clusters
|
||||
|
||||
**Environment overrides:**
|
||||
- `VERSION=v0.3.1` — Circuit version
|
||||
- `NOMOS_NODE_REV=<commit>` — nomos-node git revision
|
||||
- `NOMOS_BINARIES_TAR=path/to/bundle.tar.gz` — Use prebuilt bundle
|
||||
- `NOMOS_SKIP_IMAGE_BUILD=1` — Skip image rebuild (compose/k8s)
|
||||
- `NOMOS_SKIP_IMAGE_BUILD=1` — Skip image rebuild inside `run-examples.sh` (compose/k8s)
|
||||
- `NOMOS_BUNDLE_DOCKER_PLATFORM=linux/arm64|linux/amd64` — Docker platform for bundle builds (macOS/Windows)
|
||||
- `COMPOSE_CIRCUITS_PLATFORM=linux-aarch64|linux-x86_64` — Circuits platform for image builds
|
||||
- `SLOW_TEST_ENV=true` — Doubles built-in readiness timeouts (useful in CI / constrained laptops)
|
||||
@ -279,7 +287,7 @@ scripts/run/run-examples.sh -t 60 -v 3 -e 1 k8s \
|
||||
|
||||
### In Code (Optional)
|
||||
|
||||
```rust
|
||||
```rust,ignore
|
||||
use testing_framework_core::scenario::ScenarioBuilder;
|
||||
use testing_framework_workflows::ObservabilityBuilderExt as _;
|
||||
|
||||
@ -304,4 +312,3 @@ let plan = ScenarioBuilder::with_node_counts(1, 1)
|
||||
- [Environment Variables](environment-variables.md) — Full variable reference
|
||||
- [Logging & Observability](logging-observability.md) — Log collection and metrics
|
||||
- [Troubleshooting](troubleshooting.md) — Common issues and fixes
|
||||
|
||||
|
||||
@ -8,7 +8,7 @@ interpret results correctly.
|
||||
|
||||
Describe **what** you want to test, not **how** to orchestrate it:
|
||||
|
||||
```rust
|
||||
```rust,ignore
|
||||
use testing_framework_core::scenario::ScenarioBuilder;
|
||||
use testing_framework_workflows::ScenarioBuilderExt;
|
||||
|
||||
@ -39,7 +39,7 @@ Reason in **blocks** and **consensus intervals**, not wall-clock seconds.
|
||||
- Active slot coefficient: 0.9 (90% block probability per slot, configurable via `CONSENSUS_ACTIVE_SLOT_COEFF`)
|
||||
- Expected rate: ~27 blocks per minute
|
||||
|
||||
```rust
|
||||
```rust,ignore
|
||||
use std::time::Duration;
|
||||
|
||||
use testing_framework_core::scenario::ScenarioBuilder;
|
||||
@ -76,7 +76,7 @@ not "blocks produced in exact wall-clock seconds".
|
||||
- Deterministic checks
|
||||
|
||||
**Chaos is opt-in:**
|
||||
```rust
|
||||
```rust,ignore
|
||||
use std::time::Duration;
|
||||
|
||||
use testing_framework_core::scenario::ScenarioBuilder;
|
||||
@ -135,7 +135,7 @@ perspective.
|
||||
|
||||
Always run long enough for **meaningful block production**:
|
||||
|
||||
```rust
|
||||
```rust,ignore
|
||||
use std::time::Duration;
|
||||
|
||||
use testing_framework_core::scenario::ScenarioBuilder;
|
||||
|
||||
@ -38,7 +38,7 @@ This section shows what you'll actually see when common issues occur. Each examp
|
||||
|
||||
**What you'll see:**
|
||||
|
||||
```
|
||||
```text
|
||||
$ cargo run -p runner-examples --bin local_runner
|
||||
Finished dev [unoptimized + debuginfo] target(s) in 0.48s
|
||||
Running `target/debug/local_runner`
|
||||
@ -70,7 +70,7 @@ POL_PROOF_DEV_MODE=true cargo run -p runner-examples --bin local_runner
|
||||
|
||||
**What you'll see:**
|
||||
|
||||
```
|
||||
```text
|
||||
$ scripts/run/run-examples.sh -t 60 -v 1 -e 1 host
|
||||
ERROR: versions.env not found at repository root
|
||||
This file is required and should define:
|
||||
@ -102,7 +102,7 @@ cat versions.env
|
||||
|
||||
**What you'll see:**
|
||||
|
||||
```
|
||||
```text
|
||||
$ POL_PROOF_DEV_MODE=true cargo run -p runner-examples --bin local_runner
|
||||
[INFO testing_framework_runner_local] Starting DA workload
|
||||
[ERROR nomos_da_dispersal] Failed to load KZG parameters
|
||||
@ -144,7 +144,7 @@ ls -lh testing-framework/assets/stack/kzgrs_test_params/kzgrs_test_params
|
||||
|
||||
**What you'll see:**
|
||||
|
||||
```
|
||||
```text
|
||||
$ POL_PROOF_DEV_MODE=true cargo run -p runner-examples --bin local_runner
|
||||
[INFO testing_framework_runner_local] Spawning validator 0
|
||||
Error: Os { code: 2, kind: NotFound, message: "No such file or directory" }
|
||||
@ -187,7 +187,7 @@ POL_PROOF_DEV_MODE=true cargo run -p runner-examples --bin local_runner
|
||||
|
||||
**What you'll see:**
|
||||
|
||||
```
|
||||
```text
|
||||
$ scripts/run/run-examples.sh -t 60 -v 1 -e 1 compose
|
||||
[INFO runner_examples::compose_runner] Starting compose deployment
|
||||
Error: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
|
||||
@ -224,7 +224,7 @@ sudo usermod -aG docker $USER
|
||||
|
||||
**What you'll see:**
|
||||
|
||||
```
|
||||
```text
|
||||
$ POL_PROOF_DEV_MODE=true cargo run -p runner-examples --bin compose_runner
|
||||
[INFO testing_framework_runner_compose] Starting compose deployment
|
||||
Error: Failed to pull image 'logos-blockchain-testing:local': No such image
|
||||
@ -271,7 +271,7 @@ kind load docker-image logos-blockchain-testing:local
|
||||
|
||||
**What you'll see:**
|
||||
|
||||
```
|
||||
```text
|
||||
$ POL_PROOF_DEV_MODE=true cargo run -p runner-examples --bin local_runner
|
||||
[INFO testing_framework_runner_local] Launching validator 0 on port 18080
|
||||
Error: Os { code: 48, kind: AddrInUse, message: "Address already in use" }
|
||||
@ -322,7 +322,7 @@ vim scripts/observability/compose/docker-compose.yml
|
||||
|
||||
**What you'll see:**
|
||||
|
||||
```
|
||||
```text
|
||||
$ POL_PROOF_DEV_MODE=true cargo run -p runner-examples --bin local_runner
|
||||
[INFO testing_framework_workflows] Starting transaction workload with 10 users
|
||||
[ERROR testing_framework_workflows] Wallet seeding failed: requested 10 users but only 3 wallets available
|
||||
@ -333,7 +333,7 @@ thread 'main' panicked at 'workload init failed: insufficient wallets'
|
||||
|
||||
**Fix:**
|
||||
|
||||
```rust
|
||||
```rust,ignore
|
||||
use testing_framework_core::scenario::ScenarioBuilder;
|
||||
use testing_framework_workflows::ScenarioBuilderExt;
|
||||
|
||||
@ -358,7 +358,7 @@ let scenario = ScenarioBuilder::topology_with(|t| t.network_star().validators(3)
|
||||
|
||||
**What you'll see:**
|
||||
|
||||
```
|
||||
```text
|
||||
$ docker ps --filter "name=nomos-compose-"
|
||||
CONTAINER ID STATUS
|
||||
abc123def456 Restarting (137) 30 seconds ago # 137 = OOM killed
|
||||
@ -402,7 +402,7 @@ ulimit -n 4096
|
||||
|
||||
**What you'll see:**
|
||||
|
||||
```
|
||||
```text
|
||||
$ POL_PROOF_DEV_MODE=true cargo run -p runner-examples --bin local_runner
|
||||
[INFO runner_examples] Test complete, cleaning up
|
||||
[INFO testing_framework_runner_local] Removing temporary directories
|
||||
@ -439,7 +439,7 @@ ls /tmp/test-logs/
|
||||
|
||||
**What you'll see:**
|
||||
|
||||
```
|
||||
```text
|
||||
$ POL_PROOF_DEV_MODE=true cargo run -p runner-examples --bin local_runner
|
||||
[INFO testing_framework_core] Starting workloads
|
||||
[INFO testing_framework_core] Run window: 10 seconds
|
||||
@ -452,7 +452,7 @@ thread 'main' panicked at 'expectations failed'
|
||||
|
||||
**Fix:**
|
||||
|
||||
```rust
|
||||
```rust,ignore
|
||||
use std::time::Duration;
|
||||
|
||||
use testing_framework_core::scenario::ScenarioBuilder;
|
||||
@ -743,7 +743,7 @@ Run a minimal baseline test (e.g., 2 validators, consensus liveness only). If it
|
||||
|
||||
- **Cause**: Helper scripts (`run-examples.sh`, `build-bundle.sh`, `setup-circuits-stack.sh`) require `versions.env` file at repository root.
|
||||
- **Fix**: Ensure you're running from the repository root directory. The `versions.env` file should already exist and contains:
|
||||
```
|
||||
```text
|
||||
VERSION=<circuit release tag>
|
||||
NOMOS_NODE_REV=<nomos-node git revision>
|
||||
NOMOS_BUNDLE_VERSION=<bundle schema version>
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user