mirror of
https://github.com/logos-blockchain/logos-blockchain-testing.git
synced 2026-01-07 15:53:10 +00:00
docs: comprehensive documentation improvements
- Rename to 'Logos Blockchain Testing Framework Book' - Rebrand protocol references from Nomos to Logos - Add narrative improvements (Core Concept, learning paths, callouts) - Expand best-practices and what-you-will-learn pages - Add maintenance guide (README.md) with doc-snippets documentation - Add Notion documentation links - Fix code example imports and API signatures - Remove all icons/emojis
This commit is contained in:
parent
64dfe398e0
commit
f355ead47e
533
book/README.md
Normal file
533
book/README.md
Normal file
@ -0,0 +1,533 @@
|
|||||||
|
# Documentation Maintenance Guide
|
||||||
|
|
||||||
|
This guide helps maintainers keep the book synchronized with code changes. Use these checklists when modifying the framework.
|
||||||
|
|
||||||
|
**Key Tool:** The `examples/doc-snippets/` crate contains compilable versions of code examples from the book. Always run `cargo build -p doc-snippets` after API changes to catch broken examples early.
|
||||||
|
|
||||||
|
## Quick Reference: What to Update When
|
||||||
|
|
||||||
|
| Change Type | Pages to Check | Estimated Time |
|
||||||
|
|-------------|----------------|----------------|
|
||||||
|
| API method renamed/changed | [API Changes](#api-changes) | 1-2 hours |
|
||||||
|
| New workload/expectation added | [New Features](#new-features) | 30 minutes |
|
||||||
|
| Environment variable added/changed | [Environment Variables](#environment-variables) | 15 minutes |
|
||||||
|
| Script path/interface changed | [Scripts & Tools](#scripts--tools) | 30 minutes |
|
||||||
|
| New runner/deployer added | [New Runner](#new-runner) | 2-3 hours |
|
||||||
|
| Trait signature changed | [Trait Changes](#trait-changes) | 1-2 hours |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Detailed Checklists
|
||||||
|
|
||||||
|
### API Changes
|
||||||
|
|
||||||
|
**When:** Builder API methods, trait methods, or core types change
|
||||||
|
|
||||||
|
**Examples:**
|
||||||
|
- Rename: `.transactions_with()` → `.with_transactions()`
|
||||||
|
- New method: `.with_timeout()`
|
||||||
|
- Parameter change: `.validators(3)` → `.validators(count, config)`
|
||||||
|
|
||||||
|
**Update these pages:**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# 1. Search for affected API usage
|
||||||
|
rg "old_method_name" book/src/
|
||||||
|
|
||||||
|
# 2. Update these files:
|
||||||
|
- [ ] src/dsl-cheat-sheet.md # Builder API reference (highest priority)
|
||||||
|
- [ ] src/quickstart.md # First example users see
|
||||||
|
- [ ] src/examples.md # 4 complete scenarios
|
||||||
|
- [ ] src/examples-advanced.md # 3 advanced scenarios
|
||||||
|
- [ ] src/introduction.md # "A Scenario in 20 Lines" example
|
||||||
|
- [ ] src/project-context-primer.md # Quick example section
|
||||||
|
- [ ] src/authoring-scenarios.md # Scenario patterns
|
||||||
|
- [ ] src/best-practices.md # Code organization examples
|
||||||
|
- [ ] src/custom-workload-example.md # Complete implementation
|
||||||
|
- [ ] src/extending.md # Trait implementation examples
|
||||||
|
```
|
||||||
|
|
||||||
|
**Verification:**
|
||||||
|
```bash
|
||||||
|
# Compile doc-snippets to catch API breakage
|
||||||
|
cargo build -p doc-snippets
|
||||||
|
|
||||||
|
# Check if book links are valid
|
||||||
|
mdbook test
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### New Features
|
||||||
|
|
||||||
|
#### New Workload or Expectation
|
||||||
|
|
||||||
|
**When:** Adding a new traffic generator or success criterion
|
||||||
|
|
||||||
|
**Examples:**
|
||||||
|
- New workload: `MemoryPressureWorkload`
|
||||||
|
- New expectation: `ExpectZeroDroppedTransactions`
|
||||||
|
|
||||||
|
**Update these pages:**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
- [ ] src/workloads.md # Add to built-in workloads section
|
||||||
|
- [ ] src/dsl-cheat-sheet.md # Add DSL helper if provided
|
||||||
|
- [ ] src/examples-advanced.md # Consider adding example usage
|
||||||
|
- [ ] src/glossary.md # Add term definition
|
||||||
|
- [ ] src/internal-crate-reference.md # Document crate location
|
||||||
|
```
|
||||||
|
|
||||||
|
**Optional (if significant feature):**
|
||||||
|
```bash
|
||||||
|
- [ ] src/what-you-will-learn.md # Add to learning outcomes
|
||||||
|
- [ ] src/best-practices.md # Add usage guidance
|
||||||
|
```
|
||||||
|
|
||||||
|
#### New Runner/Deployer
|
||||||
|
|
||||||
|
**When:** Adding support for a new deployment target (e.g., AWS ECS)
|
||||||
|
|
||||||
|
**Update these pages:**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Core documentation
|
||||||
|
- [ ] src/runners.md # Add to comparison table and decision guide
|
||||||
|
- [ ] src/operations-overview.md # Update runner-agnostic matrix
|
||||||
|
- [ ] src/architecture-overview.md # Update deployer list and diagram
|
||||||
|
- [ ] src/running-examples.md # Add runner-specific section
|
||||||
|
|
||||||
|
# Reference pages
|
||||||
|
- [ ] src/dsl-cheat-sheet.md # Add deployer import/usage
|
||||||
|
- [ ] src/internal-crate-reference.md # Document new crate
|
||||||
|
- [ ] src/glossary.md # Add runner type definition
|
||||||
|
|
||||||
|
# Potentially affected
|
||||||
|
- [ ] src/ci-integration.md # Add CI example if applicable
|
||||||
|
- [ ] src/troubleshooting.md # Add common issues
|
||||||
|
- [ ] src/faq.md # Add FAQ entries
|
||||||
|
```
|
||||||
|
|
||||||
|
#### New Topology Helper
|
||||||
|
|
||||||
|
**When:** Adding topology generation helpers (e.g., `.network_mesh()`)
|
||||||
|
|
||||||
|
**Update these pages:**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
- [ ] src/dsl-cheat-sheet.md # Add to topology section
|
||||||
|
- [ ] src/authoring-scenarios.md # Add usage pattern
|
||||||
|
- [ ] src/topology-chaos.md # Add topology description
|
||||||
|
- [ ] src/examples.md # Consider adding example
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Trait Changes
|
||||||
|
|
||||||
|
**When:** Core trait signatures change (breaking changes)
|
||||||
|
|
||||||
|
**Examples:**
|
||||||
|
- `Workload::init()` adds new parameter
|
||||||
|
- `Expectation::evaluate()` changes return type
|
||||||
|
- `Deployer::deploy()` signature update
|
||||||
|
|
||||||
|
**Update these pages:**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Critical - these show full trait definitions
|
||||||
|
- [ ] src/extending.md # Complete trait outlines (6+ examples)
|
||||||
|
- [ ] src/custom-workload-example.md # Full implementation example
|
||||||
|
- [ ] src/scenario-model.md # Core model documentation
|
||||||
|
|
||||||
|
# Important - these reference traits
|
||||||
|
- [ ] src/api-levels.md # Trait usage patterns
|
||||||
|
- [ ] src/architecture-overview.md # Extension points diagram
|
||||||
|
- [ ] src/internal-crate-reference.md # Trait locations
|
||||||
|
```
|
||||||
|
|
||||||
|
**Verification:**
|
||||||
|
```bash
|
||||||
|
# Ensure trait examples would compile
|
||||||
|
cargo doc --no-deps --document-private-items
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Environment Variables
|
||||||
|
|
||||||
|
**When:** New environment variable added, changed, or removed
|
||||||
|
|
||||||
|
**Examples:**
|
||||||
|
- New: `NOMOS_NEW_FEATURE_ENABLED`
|
||||||
|
- Changed: `NOMOS_LOG_LEVEL` accepts new values
|
||||||
|
- Deprecated: `OLD_FEATURE_FLAG`
|
||||||
|
|
||||||
|
**Update these pages:**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Primary location (single source of truth)
|
||||||
|
- [ ] src/environment-variables.md # Add to appropriate category table
|
||||||
|
|
||||||
|
# Secondary mentions
|
||||||
|
- [ ] src/prerequisites.md # If affects setup
|
||||||
|
- [ ] src/running-examples.md # If affects runner usage
|
||||||
|
- [ ] src/troubleshooting.md # If commonly misconfigured
|
||||||
|
- [ ] src/glossary.md # If significant/commonly referenced
|
||||||
|
```
|
||||||
|
|
||||||
|
**Environment Variables Table Location:**
|
||||||
|
```
|
||||||
|
src/environment-variables.md
|
||||||
|
├─ Runner Configuration
|
||||||
|
├─ Node Binary & Paths
|
||||||
|
├─ Circuit Assets
|
||||||
|
├─ Logging & Tracing
|
||||||
|
├─ Observability & Metrics
|
||||||
|
├─ Proof System
|
||||||
|
├─ Docker & Images
|
||||||
|
├─ Testing Behavior
|
||||||
|
└─ CI/CD
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Scripts & Tools
|
||||||
|
|
||||||
|
**When:** Helper scripts move, rename, or change interface
|
||||||
|
|
||||||
|
**Examples:**
|
||||||
|
- Script moved: `scripts/run-examples.sh` → `scripts/run/run-examples.sh`
|
||||||
|
- New script: `scripts/clean-all.sh`
|
||||||
|
- Interface change: `run-examples.sh` adds new required flag
|
||||||
|
|
||||||
|
**Update these pages:**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# High impact
|
||||||
|
- [ ] src/quickstart.md # Uses run-examples.sh prominently
|
||||||
|
- [ ] src/running-examples.md # Documents all scripts
|
||||||
|
- [ ] src/prerequisites.md # References setup scripts
|
||||||
|
- [ ] src/examples.md # Script recommendations
|
||||||
|
- [ ] src/examples-advanced.md # Script recommendations
|
||||||
|
|
||||||
|
# Moderate impact
|
||||||
|
- [ ] src/ci-integration.md # May reference scripts in workflows
|
||||||
|
- [ ] src/troubleshooting.md # Cleanup scripts
|
||||||
|
- [ ] src/architecture-overview.md # Asset preparation scripts
|
||||||
|
```
|
||||||
|
|
||||||
|
**Find all script references:**
|
||||||
|
```bash
|
||||||
|
rg "scripts/" book/src/ --no-heading
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Operational Changes
|
||||||
|
|
||||||
|
#### Docker Image Changes
|
||||||
|
|
||||||
|
**When:** Image build process, tag names, or embedded assets change
|
||||||
|
|
||||||
|
**Update these pages:**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
- [ ] src/prerequisites.md # Image build instructions
|
||||||
|
- [ ] src/runners.md # Compose/K8s prerequisites
|
||||||
|
- [ ] src/environment-variables.md # NOMOS_TESTNET_IMAGE, NOMOS_BINARIES_TAR
|
||||||
|
- [ ] src/architecture-overview.md # Assets and Images section
|
||||||
|
```
|
||||||
|
|
||||||
|
#### Observability Stack Changes
|
||||||
|
|
||||||
|
**When:** Prometheus, Grafana, OTLP, or metrics configuration changes
|
||||||
|
|
||||||
|
**Update these pages:**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
- [ ] src/logging-observability.md # Primary documentation
|
||||||
|
- [ ] src/environment-variables.md # NOMOS_METRICS_*, NOMOS_OTLP_*
|
||||||
|
- [ ] src/architecture-overview.md # Observability section
|
||||||
|
- [ ] src/runners.md # Runner observability support
|
||||||
|
```
|
||||||
|
|
||||||
|
#### CI/CD Changes
|
||||||
|
|
||||||
|
**When:** CI workflow changes, new actions, or integration patterns
|
||||||
|
|
||||||
|
**Update these pages:**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
- [ ] src/ci-integration.md # Complete workflow examples
|
||||||
|
- [ ] src/best-practices.md # CI recommendations
|
||||||
|
- [ ] src/operations-overview.md # CI mentioned in runner matrix
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Node Protocol Changes
|
||||||
|
|
||||||
|
**When:** Changes to Logos blockchain protocol or node behavior
|
||||||
|
|
||||||
|
**Examples:**
|
||||||
|
- New consensus parameter
|
||||||
|
- DA protocol change
|
||||||
|
- Network layer update
|
||||||
|
|
||||||
|
**Update these pages:**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Context pages (high-level only)
|
||||||
|
- [ ] src/project-context-primer.md # Protocol overview
|
||||||
|
- [ ] src/glossary.md # Protocol terms
|
||||||
|
- [ ] src/faq.md # May need protocol updates
|
||||||
|
|
||||||
|
# Usually NOT affected (framework is protocol-agnostic)
|
||||||
|
- Testing framework abstracts protocol details
|
||||||
|
- Only update if change affects testing methodology
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
### Crate Structure Changes
|
||||||
|
|
||||||
|
**When:** Crate reorganization, renames, or new crates added
|
||||||
|
|
||||||
|
**Examples:**
|
||||||
|
- New crate: `testing-framework-metrics`
|
||||||
|
- Crate rename: `runner-examples` → `examples`
|
||||||
|
- Module moved: `core::scenario` → `core::model`
|
||||||
|
|
||||||
|
**Update these pages:**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Critical
|
||||||
|
- [ ] src/internal-crate-reference.md # Complete crate listing
|
||||||
|
- [ ] src/architecture-overview.md # Crate dependency diagram
|
||||||
|
- [ ] src/workspace-layout.md # Directory structure
|
||||||
|
- [ ] src/annotated-tree.md # File tree with annotations
|
||||||
|
|
||||||
|
# Code examples (update imports)
|
||||||
|
- [ ] src/dsl-cheat-sheet.md # Import statements
|
||||||
|
- [ ] src/extending.md # use statements in examples
|
||||||
|
- [ ] src/custom-workload-example.md # Full imports
|
||||||
|
```
|
||||||
|
|
||||||
|
**Find all import statements:**
|
||||||
|
```bash
|
||||||
|
rg "^use testing_framework" book/src/
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Testing Documentation Changes
|
||||||
|
|
||||||
|
### Build the Book
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd book
|
||||||
|
mdbook build
|
||||||
|
|
||||||
|
# Output: ../target/book/
|
||||||
|
```
|
||||||
|
|
||||||
|
### Test Documentation
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Check for broken links
|
||||||
|
mdbook test
|
||||||
|
|
||||||
|
# Preview locally
|
||||||
|
mdbook serve
|
||||||
|
# Open http://localhost:3000
|
||||||
|
```
|
||||||
|
|
||||||
|
### Test Code Examples (Doc Snippets)
|
||||||
|
|
||||||
|
**The `examples/doc-snippets/` crate contains compilable versions of code examples from the book.**
|
||||||
|
|
||||||
|
This ensures examples stay synchronized with the actual API and don't break when code changes.
|
||||||
|
|
||||||
|
**Why doc-snippets exist:**
|
||||||
|
- Code examples in the book (73 blocks across 18 files) can drift from reality
|
||||||
|
- Compilation failures catch API breakage immediately
|
||||||
|
- Single source of truth for code examples
|
||||||
|
|
||||||
|
**Current coverage:** 40+ snippet files corresponding to examples in:
|
||||||
|
- `quickstart.md` (7 snippets)
|
||||||
|
- `examples.md` (4 scenarios)
|
||||||
|
- `examples-advanced.md` (3 scenarios)
|
||||||
|
- `dsl-cheat-sheet.md` (11 DSL examples)
|
||||||
|
- `custom-workload-example.md` (2 trait implementations)
|
||||||
|
- `internal-crate-reference.md` (6 extension examples)
|
||||||
|
- And more...
|
||||||
|
|
||||||
|
**Testing snippets:**
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Compile all doc snippets
|
||||||
|
cargo build -p doc-snippets
|
||||||
|
|
||||||
|
# Run with full warnings
|
||||||
|
cargo build -p doc-snippets --all-features
|
||||||
|
|
||||||
|
# Check during CI
|
||||||
|
cargo check -p doc-snippets
|
||||||
|
```
|
||||||
|
|
||||||
|
**When to update snippets:**
|
||||||
|
|
||||||
|
1. **API method changed** → Update corresponding snippet file
|
||||||
|
```bash
|
||||||
|
# Example: If .transactions_with() signature changes
|
||||||
|
# Update: examples/doc-snippets/src/examples_transaction_workload.rs
|
||||||
|
```
|
||||||
|
|
||||||
|
2. **New code example added to book** → Create new snippet file
|
||||||
|
```bash
|
||||||
|
# Example: Adding new topology pattern
|
||||||
|
# Create: examples/doc-snippets/src/topology_mesh_example.rs
|
||||||
|
```
|
||||||
|
|
||||||
|
3. **Trait signature changed** → Update trait implementation snippets
|
||||||
|
```bash
|
||||||
|
# Update: custom_workload_example_*.rs
|
||||||
|
# Update: internal_crate_reference_add_*.rs
|
||||||
|
```
|
||||||
|
|
||||||
|
**Snippet naming convention:**
|
||||||
|
```
|
||||||
|
book/src/examples.md → examples_*.rs
|
||||||
|
book/src/quickstart.md → quickstart_*.rs
|
||||||
|
book/src/dsl-cheat-sheet.md → dsl_cheat_sheet_*.rs
|
||||||
|
```
|
||||||
|
|
||||||
|
**Best practice:**
|
||||||
|
When updating code examples in markdown, update the corresponding snippet file first, verify it compiles, then copy to the book. This ensures examples are always valid.
|
||||||
|
|
||||||
|
### Check for Common Issues
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Find outdated API references
|
||||||
|
rg "old_deprecated_api" src/
|
||||||
|
|
||||||
|
# Find broken GitHub links
|
||||||
|
rg "github.com.*404" src/
|
||||||
|
|
||||||
|
# Find TODO/FIXME markers
|
||||||
|
rg "(TODO|FIXME|XXX)" src/
|
||||||
|
|
||||||
|
# Check for inconsistent terminology
|
||||||
|
rg "(Nomos node|nomos blockchain)" src/ # Should be "Logos"
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Maintenance Schedule
|
||||||
|
|
||||||
|
### On Every PR
|
||||||
|
|
||||||
|
- [ ] Check if changes affect documented APIs
|
||||||
|
- [ ] Update relevant pages per checklist above
|
||||||
|
- [ ] Update corresponding doc-snippets if code examples changed
|
||||||
|
- [ ] Run `cargo build -p doc-snippets` to verify examples compile
|
||||||
|
- [ ] Build book to verify no broken links
|
||||||
|
- [ ] Verify code examples still make sense
|
||||||
|
|
||||||
|
### Monthly
|
||||||
|
|
||||||
|
- [ ] Review recent PRs for documentation impact
|
||||||
|
- [ ] Update environment variables table
|
||||||
|
- [ ] Check script references are current
|
||||||
|
- [ ] Verify GitHub source links are not 404
|
||||||
|
|
||||||
|
### Quarterly
|
||||||
|
|
||||||
|
- [ ] Full audit of code examples against latest API
|
||||||
|
- [ ] Verify all doc-snippets still compile with latest dependencies
|
||||||
|
- [ ] Check for code examples in book that don't have corresponding snippets
|
||||||
|
- [ ] Review troubleshooting for new patterns
|
||||||
|
- [ ] Update FAQ with common questions
|
||||||
|
- [ ] Check all Mermaid diagrams render correctly
|
||||||
|
|
||||||
|
### Major Release
|
||||||
|
|
||||||
|
- [ ] Complete review of all technical content
|
||||||
|
- [ ] Verify all version-specific references
|
||||||
|
- [ ] Update "What You Will Learn" outcomes
|
||||||
|
- [ ] Add release notes for documentation changes
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Content Organization
|
||||||
|
|
||||||
|
### Stability Tiers (Change Frequency)
|
||||||
|
|
||||||
|
**Stable (Rarely Change)**
|
||||||
|
- Part I — Foundations (philosophy, architecture, design rationale)
|
||||||
|
- Part VI — Appendix (glossary, FAQ, troubleshooting symptoms)
|
||||||
|
- Front matter (project context, introduction)
|
||||||
|
|
||||||
|
**Semi-Stable (Occasional Changes)**
|
||||||
|
- Part II — User Guide (usage patterns, best practices, examples)
|
||||||
|
- Part V — Operations (prerequisites, CI, logging)
|
||||||
|
|
||||||
|
**High Volatility (Frequent Changes)**
|
||||||
|
- API references (dsl-cheat-sheet.md, extending.md)
|
||||||
|
- Code examples (73 blocks across 18 files)
|
||||||
|
- Environment variables (50+ documented)
|
||||||
|
- Runner comparisons (features evolve)
|
||||||
|
|
||||||
|
### Page Dependency Map
|
||||||
|
|
||||||
|
**Core pages** (many other pages reference these):
|
||||||
|
- `dsl-cheat-sheet.md` ← Referenced by examples, quickstart, authoring
|
||||||
|
- `environment-variables.md` ← Referenced by operations, troubleshooting, runners
|
||||||
|
- `runners.md` ← Referenced by operations, quickstart, examples
|
||||||
|
- `glossary.md` ← Referenced throughout the book
|
||||||
|
|
||||||
|
**When updating core pages, check for broken cross-references.**
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Common Patterns
|
||||||
|
|
||||||
|
### Adding a Code Example
|
||||||
|
|
||||||
|
```markdown
|
||||||
|
# 1. Add the code block
|
||||||
|
```rust
|
||||||
|
use testing_framework_core::scenario::ScenarioBuilder;
|
||||||
|
// ... example code
|
||||||
|
```
|
||||||
|
|
||||||
|
# 2. Add context
|
||||||
|
**When to use:** [explain use case]
|
||||||
|
|
||||||
|
# 3. Link to complete source (if applicable)
|
||||||
|
[View in source](https://github.com/logos-blockchain/logos-blockchain-testing/blob/master/examples/src/bin/example.rs)
|
||||||
|
```
|
||||||
|
|
||||||
|
### Adding a Cross-Reference
|
||||||
|
|
||||||
|
```markdown
|
||||||
|
See [Environment Variables](environment-variables.md) for complete configuration reference.
|
||||||
|
```
|
||||||
|
|
||||||
|
### Adding a "When to Read" Callout
|
||||||
|
|
||||||
|
```markdown
|
||||||
|
> **When should I read this?** [guidance on when this content is relevant]
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Contact & Questions
|
||||||
|
|
||||||
|
When in doubt:
|
||||||
|
1. Check this README for guidance
|
||||||
|
2. Review recent similar changes in git history
|
||||||
|
3. Ask the team in technical documentation discussions
|
||||||
|
|
||||||
|
**Remember:** Documentation quality directly impacts framework adoption and user success. Taking time to update docs properly is an investment in the project's future.
|
||||||
@ -2,7 +2,7 @@
|
|||||||
authors = ["Nomos Testing"]
|
authors = ["Nomos Testing"]
|
||||||
language = "en"
|
language = "en"
|
||||||
src = "src"
|
src = "src"
|
||||||
title = "Nomos Testing Book"
|
title = "Logos Blockchain Testing Framework Book"
|
||||||
|
|
||||||
[build]
|
[build]
|
||||||
# Keep book output in target/ to avoid polluting the workspace root.
|
# Keep book output in target/ to avoid polluting the workspace root.
|
||||||
|
|||||||
@ -92,14 +92,15 @@ Mixing is common: use the DSL for built-ins, and direct instantiation for custom
|
|||||||
use std::time::Duration;
|
use std::time::Duration;
|
||||||
|
|
||||||
use testing_framework_core::scenario::ScenarioBuilder;
|
use testing_framework_core::scenario::ScenarioBuilder;
|
||||||
use testing_framework_workflows::ScenarioBuilderExt;
|
use testing_framework_workflows::{ScenarioBuilderExt, workloads::transaction};
|
||||||
|
|
||||||
let custom_workload = MyCustomWorkload::new(config);
|
let tx_workload = transaction::Workload::with_rate(5)
|
||||||
|
.expect("transaction rate must be non-zero");
|
||||||
|
|
||||||
let plan = ScenarioBuilder::topology_with(|t| t.network_star().validators(3).executors(2))
|
let plan = ScenarioBuilder::topology_with(|t| t.network_star().validators(3).executors(2))
|
||||||
.transactions_with(|txs| txs.rate(5).users(3)) // DSL
|
.wallets(5)
|
||||||
.with_workload(custom_workload) // direct
|
.with_workload(tx_workload) // direct instantiation
|
||||||
.expect_consensus_liveness() // DSL
|
.expect_consensus_liveness() // DSL
|
||||||
.with_run_duration(Duration::from_secs(60))
|
.with_run_duration(Duration::from_secs(60))
|
||||||
.build();
|
.build();
|
||||||
```
|
```
|
||||||
@ -108,15 +109,11 @@ let plan = ScenarioBuilder::topology_with(|t| t.network_star().validators(3).exe
|
|||||||
|
|
||||||
The DSL methods are thin wrappers. For example:
|
The DSL methods are thin wrappers. For example:
|
||||||
|
|
||||||
```rust
|
`builder.transactions_with(|txs| txs.rate(5).users(3))`
|
||||||
builder.transactions_with(|txs| txs.rate(5).users(3))
|
|
||||||
```
|
|
||||||
|
|
||||||
is roughly equivalent to:
|
is roughly equivalent to:
|
||||||
|
|
||||||
```rust
|
`builder.transactions().rate(5).users(3).apply()`
|
||||||
builder.transactions().rate(5).users(3).apply()
|
|
||||||
```
|
|
||||||
|
|
||||||
## Troubleshooting
|
## Troubleshooting
|
||||||
|
|
||||||
|
|||||||
@ -1,17 +1,248 @@
|
|||||||
# Best Practices
|
# Best Practices
|
||||||
|
|
||||||
- **State your intent**: document the goal of each scenario (throughput, DA
|
This page collects proven patterns for authoring, running, and maintaining test scenarios that are reliable, maintainable, and actionable.
|
||||||
validation, resilience) so expectation choices are obvious.
|
|
||||||
- **Keep runs meaningful**: choose durations that allow multiple blocks and make
|
## Scenario Design
|
||||||
timing-based assertions trustworthy.
|
|
||||||
- **Separate concerns**: start with deterministic workloads for functional
|
**State your intent**
|
||||||
checks; add chaos in dedicated resilience scenarios to avoid noisy failures.
|
- Document the goal of each scenario (throughput, DA validation, resilience) so expectation choices are obvious
|
||||||
- **Reuse patterns**: standardize on shared topology and workload presets so
|
- Use descriptive variable names that explain topology purpose (e.g., `star_topology_3val_2exec` vs `topology`)
|
||||||
results are comparable across environments and teams.
|
- Add comments explaining why specific rates or durations were chosen
|
||||||
- **Observe first, tune second**: rely on liveness and inclusion signals to
|
|
||||||
interpret outcomes before tweaking rates or topology.
|
**Keep runs meaningful**
|
||||||
- **Environment fit**: pick runners that match the feedback loop you need—local
|
- Choose durations that allow multiple blocks and make timing-based assertions trustworthy
|
||||||
for speed (including fast CI smoke tests), compose for reproducible stacks
|
- Use [FAQ: Run Duration Calculator](faq.md#how-long-should-a-scenario-run) to estimate minimum duration
|
||||||
(recommended for CI), k8s for cluster-grade fidelity.
|
- Avoid runs shorter than 30 seconds unless testing startup behavior specifically
|
||||||
- **Minimal surprises**: seed only necessary wallets and keep configuration
|
|
||||||
deltas explicit when moving between CI and developer machines.
|
**Separate concerns**
|
||||||
|
- Start with deterministic workloads for functional checks
|
||||||
|
- Add chaos in dedicated resilience scenarios to avoid noisy failures
|
||||||
|
- Don't mix high transaction load with aggressive chaos in the same test (hard to debug)
|
||||||
|
|
||||||
|
**Start small, scale up**
|
||||||
|
- Begin with minimal topology (1-2 validators) to validate scenario logic
|
||||||
|
- Gradually increase topology size and workload rates
|
||||||
|
- Use Host runner for fast iteration, then validate on Compose before production
|
||||||
|
|
||||||
|
## Code Organization
|
||||||
|
|
||||||
|
**Reuse patterns**
|
||||||
|
- Standardize on shared topology and workload presets so results are comparable across environments and teams
|
||||||
|
- Extract common topology builders into helper functions
|
||||||
|
- Create workspace-level constants for standard rates and durations
|
||||||
|
|
||||||
|
**Example: Topology preset**
|
||||||
|
|
||||||
|
```rust
|
||||||
|
pub fn standard_da_topology() -> GeneratedTopology {
|
||||||
|
TopologyBuilder::new()
|
||||||
|
.network_star()
|
||||||
|
.validators(3)
|
||||||
|
.executors(2)
|
||||||
|
.generate()
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**Example: Shared constants**
|
||||||
|
|
||||||
|
```rust
|
||||||
|
pub const STANDARD_TX_RATE: f64 = 10.0;
|
||||||
|
pub const STANDARD_DA_CHANNEL_RATE: f64 = 2.0;
|
||||||
|
pub const SHORT_RUN_DURATION: Duration = Duration::from_secs(60);
|
||||||
|
pub const LONG_RUN_DURATION: Duration = Duration::from_secs(300);
|
||||||
|
```
|
||||||
|
|
||||||
|
## Debugging & Observability
|
||||||
|
|
||||||
|
**Observe first, tune second**
|
||||||
|
- Rely on liveness and inclusion signals to interpret outcomes before tweaking rates or topology
|
||||||
|
- Enable detailed logging (`RUST_LOG=debug`, `NOMOS_LOG_LEVEL=debug`) only after initial failure
|
||||||
|
- Use `NOMOS_TESTS_KEEP_LOGS=1` to persist logs when debugging failures
|
||||||
|
|
||||||
|
**Use BlockFeed effectively**
|
||||||
|
- Subscribe to BlockFeed in expectations for real-time block monitoring
|
||||||
|
- Track block production rate to detect liveness issues early
|
||||||
|
- Use block statistics (`block_feed.stats().total_transactions()`) to verify inclusion
|
||||||
|
|
||||||
|
**Collect metrics**
|
||||||
|
- Set up Prometheus/Grafana via `scripts/observability/compose/up.sh` for visualizing node behavior
|
||||||
|
- Use metrics to identify bottlenecks before adding more load
|
||||||
|
- Monitor mempool size, block size, and consensus timing
|
||||||
|
|
||||||
|
## Environment & Runner Selection
|
||||||
|
|
||||||
|
**Environment fit**
|
||||||
|
- Pick runners that match the feedback loop you need:
|
||||||
|
- **Host**: Fast iteration during development, quick CI smoke tests
|
||||||
|
- **Compose**: Reproducible environments (recommended for CI), chaos testing
|
||||||
|
- **K8s**: Production-like fidelity, large topologies (10+ nodes)
|
||||||
|
|
||||||
|
**Runner-specific considerations**
|
||||||
|
|
||||||
|
| Runner | When to Use | When to Avoid |
|
||||||
|
|--------|-------------|---------------|
|
||||||
|
| Host | Development iteration, fast feedback | Chaos testing, container-specific issues |
|
||||||
|
| Compose | CI pipelines, chaos tests, reproducibility | Very large topologies (>10 nodes) |
|
||||||
|
| K8s | Production-like testing, cluster behaviors | Local development, fast iteration |
|
||||||
|
|
||||||
|
**Minimal surprises**
|
||||||
|
- Seed only necessary wallets and keep configuration deltas explicit when moving between CI and developer machines
|
||||||
|
- Use `versions.env` to pin node versions consistently across environments
|
||||||
|
- Document non-default environment variables in scenario comments or README
|
||||||
|
|
||||||
|
## CI/CD Integration
|
||||||
|
|
||||||
|
**Use matrix builds**
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
strategy:
|
||||||
|
matrix:
|
||||||
|
runner: [host, compose]
|
||||||
|
topology: [small, medium]
|
||||||
|
```
|
||||||
|
|
||||||
|
**Cache aggressively**
|
||||||
|
- Cache Rust build artifacts (`target/`)
|
||||||
|
- Cache circuit parameters (`assets/stack/kzgrs_test_params/`)
|
||||||
|
- Cache Docker layers (use BuildKit cache)
|
||||||
|
|
||||||
|
**Collect logs on failure**
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
- name: Collect logs on failure
|
||||||
|
if: failure()
|
||||||
|
run: |
|
||||||
|
mkdir -p test-logs
|
||||||
|
find /tmp -name "nomos-*.log" -exec cp {} test-logs/ \;
|
||||||
|
- uses: actions/upload-artifact@v3
|
||||||
|
if: failure()
|
||||||
|
with:
|
||||||
|
name: test-logs-${{ matrix.runner }}
|
||||||
|
path: test-logs/
|
||||||
|
```
|
||||||
|
|
||||||
|
**Time limits**
|
||||||
|
- Set job timeout to prevent hung runs: `timeout-minutes: 30`
|
||||||
|
- Use shorter durations in CI (60s) vs local testing (300s)
|
||||||
|
- Run expensive tests (k8s, large topologies) only on main branch or release tags
|
||||||
|
|
||||||
|
**See also:** [CI Integration](ci-integration.md) for complete workflow examples
|
||||||
|
|
||||||
|
## Anti-Patterns to Avoid
|
||||||
|
|
||||||
|
**DON'T: Run without POL_PROOF_DEV_MODE**
|
||||||
|
```bash
|
||||||
|
# BAD: Will hang/timeout on proof generation
|
||||||
|
cargo run -p runner-examples --bin local_runner
|
||||||
|
|
||||||
|
# GOOD: Fast mode for testing
|
||||||
|
POL_PROOF_DEV_MODE=true cargo run -p runner-examples --bin local_runner
|
||||||
|
```
|
||||||
|
|
||||||
|
**DON'T: Use tiny durations**
|
||||||
|
```rust
|
||||||
|
// BAD: Not enough time for blocks to propagate
|
||||||
|
.with_run_duration(Duration::from_secs(5))
|
||||||
|
|
||||||
|
// GOOD: Allow multiple consensus rounds
|
||||||
|
.with_run_duration(Duration::from_secs(60))
|
||||||
|
```
|
||||||
|
|
||||||
|
**DON'T: Ignore cleanup failures**
|
||||||
|
```rust
|
||||||
|
// BAD: Next run inherits leaked state
|
||||||
|
runner.run(&mut scenario).await?;
|
||||||
|
// forgot to call cleanup or use CleanupGuard
|
||||||
|
|
||||||
|
// GOOD: Cleanup via guard (automatic on panic)
|
||||||
|
let _cleanup = CleanupGuard::new(runner.clone());
|
||||||
|
runner.run(&mut scenario).await?;
|
||||||
|
```
|
||||||
|
|
||||||
|
**DON'T: Mix concerns in one scenario**
|
||||||
|
```rust
|
||||||
|
// BAD: Hard to debug when it fails
|
||||||
|
.transactions_with(|tx| tx.rate(50).users(100)) // high load
|
||||||
|
.chaos_with(|c| c.restart().min_delay(...)) // AND chaos
|
||||||
|
.da_with(|da| da.channel_rate(10).blob_rate(20)) // AND DA stress
|
||||||
|
|
||||||
|
// GOOD: Separate tests for each concern
|
||||||
|
// Test 1: High transaction load only
|
||||||
|
// Test 2: Chaos resilience only
|
||||||
|
// Test 3: DA stress only
|
||||||
|
```
|
||||||
|
|
||||||
|
**DON'T: Hardcode paths or ports**
|
||||||
|
```rust
|
||||||
|
// BAD: Breaks on different machines
|
||||||
|
let path = PathBuf::from("/home/user/circuits/kzgrs_test_params");
|
||||||
|
let port = 9000; // might conflict
|
||||||
|
|
||||||
|
// GOOD: Use env vars and dynamic allocation
|
||||||
|
let path = std::env::var("NOMOS_KZGRS_PARAMS_PATH")
|
||||||
|
.unwrap_or_else(|_| "assets/stack/kzgrs_test_params/kzgrs_test_params".to_string());
|
||||||
|
let port = get_available_tcp_port();
|
||||||
|
```
|
||||||
|
|
||||||
|
**DON'T: Ignore resource limits**
|
||||||
|
```bash
|
||||||
|
# BAD: Large topology without checking resources
|
||||||
|
scripts/run/run-examples.sh -v 20 -e 10 compose
|
||||||
|
# (might OOM or exhaust ulimits)
|
||||||
|
|
||||||
|
# GOOD: Scale gradually and monitor resources
|
||||||
|
scripts/run/run-examples.sh -v 3 -e 2 compose # start small
|
||||||
|
docker stats # monitor resource usage
|
||||||
|
# then increase if resources allow
|
||||||
|
```
|
||||||
|
|
||||||
|
## Scenario Design Heuristics
|
||||||
|
|
||||||
|
**Minimal viable topology**
|
||||||
|
- Consensus: 3 validators (minimum for Byzantine fault tolerance)
|
||||||
|
- DA: 2+ executors (test dispersal and sampling)
|
||||||
|
- Network: Star topology (simplest for debugging)
|
||||||
|
|
||||||
|
**Workload rate selection**
|
||||||
|
- Start with 1-5 tx/s per user, then increase
|
||||||
|
- DA: 1-2 channels, 1-3 blobs/channel initially
|
||||||
|
- Chaos: 30s+ intervals between restarts (allow recovery)
|
||||||
|
|
||||||
|
**Duration guidelines**
|
||||||
|
|
||||||
|
| Test Type | Minimum Duration | Typical Duration |
|
||||||
|
|-----------|------------------|------------------|
|
||||||
|
| Smoke test | 30s | 60s |
|
||||||
|
| Integration test | 60s | 120s |
|
||||||
|
| Load test | 120s | 300s |
|
||||||
|
| Resilience test | 120s | 300s |
|
||||||
|
| Soak test | 600s (10m) | 3600s (1h) |
|
||||||
|
|
||||||
|
**Expectation selection**
|
||||||
|
|
||||||
|
| Test Goal | Expectations |
|
||||||
|
|-----------|--------------|
|
||||||
|
| Basic functionality | `expect_consensus_liveness()` |
|
||||||
|
| Transaction handling | `expect_consensus_liveness()` + custom inclusion check |
|
||||||
|
| DA correctness | `expect_consensus_liveness()` + DA dispersal/sampling checks |
|
||||||
|
| Resilience | `expect_consensus_liveness()` + recovery time measurement |
|
||||||
|
|
||||||
|
## Testing the Tests
|
||||||
|
|
||||||
|
**Validate scenarios before committing**
|
||||||
|
1. Run on Host runner first (fast feedback)
|
||||||
|
2. Run on Compose runner (reproducibility check)
|
||||||
|
3. Check logs for warnings or errors
|
||||||
|
4. Verify cleanup (no leaked processes/containers)
|
||||||
|
5. Run 2-3 times to check for flakiness
|
||||||
|
|
||||||
|
**Handling flaky tests**
|
||||||
|
- Increase run duration (timing-sensitive assertions need longer runs)
|
||||||
|
- Reduce workload rates (might be saturating nodes)
|
||||||
|
- Check resource limits (CPU/RAM/ulimits)
|
||||||
|
- Add debugging output to identify race conditions
|
||||||
|
- Consider if test is over-specified (too strict expectations)
|
||||||
|
|
||||||
|
**See also:**
|
||||||
|
- [Troubleshooting](troubleshooting.md) for common failure patterns
|
||||||
|
- [FAQ](faq.md) for design decisions and gotchas
|
||||||
|
|||||||
@ -1,5 +1,7 @@
|
|||||||
# Chaos Workloads
|
# Chaos Workloads
|
||||||
|
|
||||||
|
> **When should I read this?** You don't need chaos testing to be productive with the framework. Focus on basic scenarios first—chaos is for resilience validation and operational readiness drills once your core tests are stable.
|
||||||
|
|
||||||
Chaos in the framework uses node control to introduce failures and validate
|
Chaos in the framework uses node control to introduce failures and validate
|
||||||
recovery. The built-in restart workload lives in
|
recovery. The built-in restart workload lives in
|
||||||
`testing_framework_workflows::workloads::chaos::RandomRestartWorkload`.
|
`testing_framework_workflows::workloads::chaos::RandomRestartWorkload`.
|
||||||
|
|||||||
@ -1,5 +1,7 @@
|
|||||||
# Advanced Examples
|
# Advanced Examples
|
||||||
|
|
||||||
|
> **When should I read this?** Skim now to see what's possible, revisit later when you need load testing, chaos scenarios, or custom extensions. Start with [basic examples](examples.md) first.
|
||||||
|
|
||||||
Realistic advanced scenarios demonstrating framework capabilities for production testing.
|
Realistic advanced scenarios demonstrating framework capabilities for production testing.
|
||||||
|
|
||||||
**Adapt from Complete Source:**
|
**Adapt from Complete Source:**
|
||||||
|
|||||||
@ -20,6 +20,19 @@ use testing_framework_core::scenario::{
|
|||||||
};
|
};
|
||||||
use testing_framework_core::topology::generation::GeneratedTopology;
|
use testing_framework_core::topology::generation::GeneratedTopology;
|
||||||
|
|
||||||
|
struct MyExpectation;
|
||||||
|
|
||||||
|
#[async_trait]
|
||||||
|
impl Expectation for MyExpectation {
|
||||||
|
fn name(&self) -> &str {
|
||||||
|
"my_expectation"
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn evaluate(&mut self, _ctx: &RunContext) -> Result<(), DynError> {
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
pub struct MyWorkload {
|
pub struct MyWorkload {
|
||||||
// Configuration fields
|
// Configuration fields
|
||||||
target_rate: u64,
|
target_rate: u64,
|
||||||
@ -39,7 +52,7 @@ impl Workload for MyWorkload {
|
|||||||
|
|
||||||
fn expectations(&self) -> Vec<Box<dyn Expectation>> {
|
fn expectations(&self) -> Vec<Box<dyn Expectation>> {
|
||||||
// Return bundled expectations that should run with this workload
|
// Return bundled expectations that should run with this workload
|
||||||
vec![Box::new(MyExpectation::new(self.target_rate))]
|
vec![Box::new(MyExpectation)]
|
||||||
}
|
}
|
||||||
|
|
||||||
fn init(
|
fn init(
|
||||||
@ -60,7 +73,7 @@ impl Workload for MyWorkload {
|
|||||||
|
|
||||||
for client in clients {
|
for client in clients {
|
||||||
let info = client.consensus_info().await?;
|
let info = client.consensus_info().await?;
|
||||||
tracing::info!(?info, "workload queried node");
|
tracing::info!(height = info.height, "workload queried node");
|
||||||
}
|
}
|
||||||
|
|
||||||
Ok(())
|
Ok(())
|
||||||
@ -117,7 +130,7 @@ impl Expectation for MyExpectation {
|
|||||||
.ok_or("no validators")?;
|
.ok_or("no validators")?;
|
||||||
|
|
||||||
let info = client.consensus_info().await?;
|
let info = client.consensus_info().await?;
|
||||||
self.captured_baseline = Some(info.current_block_id.slot);
|
self.captured_baseline = Some(info.height);
|
||||||
|
|
||||||
tracing::info!(baseline = self.captured_baseline, "captured baseline");
|
tracing::info!(baseline = self.captured_baseline, "captured baseline");
|
||||||
Ok(())
|
Ok(())
|
||||||
@ -129,10 +142,10 @@ impl Expectation for MyExpectation {
|
|||||||
.ok_or("no validators")?;
|
.ok_or("no validators")?;
|
||||||
|
|
||||||
let info = client.consensus_info().await?;
|
let info = client.consensus_info().await?;
|
||||||
let final_slot = info.current_block_id.slot;
|
let final_height = info.height;
|
||||||
|
|
||||||
let baseline = self.captured_baseline.unwrap_or(0);
|
let baseline = self.captured_baseline.unwrap_or(0);
|
||||||
let delta = final_slot.saturating_sub(baseline);
|
let delta = final_height.saturating_sub(baseline);
|
||||||
|
|
||||||
if delta < self.expected_value {
|
if delta < self.expected_value {
|
||||||
return Err(format!(
|
return Err(format!(
|
||||||
@ -198,7 +211,11 @@ impl Deployer<()> for MyDeployer {
|
|||||||
let topology: Option<Topology> = None; // Some(topology) if you spawned one
|
let topology: Option<Topology> = None; // Some(topology) if you spawned one
|
||||||
let node_clients = NodeClients::default(); // Or NodeClients::from_topology(...)
|
let node_clients = NodeClients::default(); // Or NodeClients::from_topology(...)
|
||||||
|
|
||||||
let (block_feed, block_feed_guard) = spawn_block_feed(&node_clients).await?;
|
let client = node_clients
|
||||||
|
.any_client()
|
||||||
|
.ok_or("no api clients available")?
|
||||||
|
.clone();
|
||||||
|
let (block_feed, block_feed_guard) = spawn_block_feed(client).await?;
|
||||||
|
|
||||||
let telemetry = Metrics::empty(); // or Metrics::from_prometheus(...)
|
let telemetry = Metrics::empty(); // or Metrics::from_prometheus(...)
|
||||||
let node_control = None; // or Some(Arc<dyn NodeControlHandle>)
|
let node_control = None; // or Some(Arc<dyn NodeControlHandle>)
|
||||||
@ -237,28 +254,18 @@ impl Deployer<()> for MyDeployer {
|
|||||||
**Example:**
|
**Example:**
|
||||||
|
|
||||||
```rust
|
```rust
|
||||||
use testing_framework_core::topology::config::TopologyBuilder;
|
use testing_framework_core::topology::{
|
||||||
|
config::TopologyBuilder,
|
||||||
|
configs::network::Libp2pNetworkLayout,
|
||||||
|
};
|
||||||
|
|
||||||
impl TopologyBuilder {
|
pub trait TopologyBuilderExt {
|
||||||
/// Creates a "ring" topology where each node connects to its neighbors
|
fn network_full(self) -> Self;
|
||||||
pub fn network_ring(&mut self) -> &mut Self {
|
}
|
||||||
// Configure peer connections in a ring layout
|
|
||||||
self.with_network_layout(|layout| {
|
|
||||||
// Implement ring connection logic
|
|
||||||
layout.ring_peers()
|
|
||||||
});
|
|
||||||
self
|
|
||||||
}
|
|
||||||
|
|
||||||
/// Preset for high-throughput DA configuration
|
impl TopologyBuilderExt for TopologyBuilder {
|
||||||
pub fn da_high_throughput(&mut self) -> &mut Self {
|
fn network_full(self) -> Self {
|
||||||
self.with_da_params(|params| {
|
self.with_network_layout(Libp2pNetworkLayout::Full)
|
||||||
params
|
|
||||||
.dispersal_factor(8)
|
|
||||||
.replication_factor(16)
|
|
||||||
.chunk_size(4096)
|
|
||||||
});
|
|
||||||
self
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
```
|
```
|
||||||
@ -273,7 +280,49 @@ impl TopologyBuilder {
|
|||||||
To expose your custom workload through the high-level DSL, add a trait extension:
|
To expose your custom workload through the high-level DSL, add a trait extension:
|
||||||
|
|
||||||
```rust
|
```rust
|
||||||
use testing_framework_core::scenario::Builder as ScenarioBuilder;
|
use async_trait::async_trait;
|
||||||
|
use testing_framework_core::scenario::{DynError, RunContext, ScenarioBuilder, Workload};
|
||||||
|
|
||||||
|
#[derive(Default)]
|
||||||
|
pub struct MyWorkloadBuilder {
|
||||||
|
target_rate: u64,
|
||||||
|
some_option: bool,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl MyWorkloadBuilder {
|
||||||
|
pub const fn target_rate(mut self, target_rate: u64) -> Self {
|
||||||
|
self.target_rate = target_rate;
|
||||||
|
self
|
||||||
|
}
|
||||||
|
|
||||||
|
pub const fn some_option(mut self, some_option: bool) -> Self {
|
||||||
|
self.some_option = some_option;
|
||||||
|
self
|
||||||
|
}
|
||||||
|
|
||||||
|
pub const fn build(self) -> MyWorkload {
|
||||||
|
MyWorkload {
|
||||||
|
target_rate: self.target_rate,
|
||||||
|
some_option: self.some_option,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
pub struct MyWorkload {
|
||||||
|
target_rate: u64,
|
||||||
|
some_option: bool,
|
||||||
|
}
|
||||||
|
|
||||||
|
#[async_trait]
|
||||||
|
impl Workload for MyWorkload {
|
||||||
|
fn name(&self) -> &str {
|
||||||
|
"my_workload"
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn start(&self, _ctx: &RunContext) -> Result<(), DynError> {
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
pub trait MyWorkloadDsl {
|
pub trait MyWorkloadDsl {
|
||||||
fn my_workload_with(
|
fn my_workload_with(
|
||||||
@ -282,7 +331,7 @@ pub trait MyWorkloadDsl {
|
|||||||
) -> Self;
|
) -> Self;
|
||||||
}
|
}
|
||||||
|
|
||||||
impl<Caps> MyWorkloadDsl for ScenarioBuilder<Caps> {
|
impl MyWorkloadDsl for ScenarioBuilder {
|
||||||
fn my_workload_with(
|
fn my_workload_with(
|
||||||
self,
|
self,
|
||||||
f: impl FnOnce(MyWorkloadBuilder) -> MyWorkloadBuilder,
|
f: impl FnOnce(MyWorkloadBuilder) -> MyWorkloadBuilder,
|
||||||
@ -296,7 +345,7 @@ impl<Caps> MyWorkloadDsl for ScenarioBuilder<Caps> {
|
|||||||
Users can then call:
|
Users can then call:
|
||||||
|
|
||||||
```rust
|
```rust
|
||||||
ScenarioBuilder::topology_with(|t| { /* ... */ })
|
ScenarioBuilder::topology_with(|t| t.network_star().validators(1).executors(1))
|
||||||
.my_workload_with(|w| {
|
.my_workload_with(|w| {
|
||||||
w.target_rate(10)
|
w.target_rate(10)
|
||||||
.some_option(true)
|
.some_option(true)
|
||||||
|
|||||||
@ -42,7 +42,7 @@
|
|||||||
- **State assertion**: expectation that verifies specific values in the system
|
- **State assertion**: expectation that verifies specific values in the system
|
||||||
state (e.g., wallet balances, UTXO sets) rather than just progress signals.
|
state (e.g., wallet balances, UTXO sets) rather than just progress signals.
|
||||||
Also called "correctness expectations."
|
Also called "correctness expectations."
|
||||||
- **Mantle transaction**: transaction type in Nomos that can contain UTXO transfers
|
- **Mantle transaction**: transaction type in Logos that can contain UTXO transfers
|
||||||
(LedgerTx) and operations (Op), including channel data (ChannelBlob).
|
(LedgerTx) and operations (Op), including channel data (ChannelBlob).
|
||||||
- **Channel**: logical grouping for DA blobs; each blob belongs to a channel and
|
- **Channel**: logical grouping for DA blobs; each blob belongs to a channel and
|
||||||
references a parent blob in the same channel, creating a chain of related data.
|
references a parent blob in the same channel, creating a chain of related data.
|
||||||
@ -50,3 +50,9 @@
|
|||||||
proof generation for leader election. **Required for all runners** (local, compose, k8s)
|
proof generation for leader election. **Required for all runners** (local, compose, k8s)
|
||||||
for practical testing—without it, proof generation causes timeouts. Should never be
|
for practical testing—without it, proof generation causes timeouts. Should never be
|
||||||
used in production environments.
|
used in production environments.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## External Resources
|
||||||
|
|
||||||
|
- **[Nomos Project Documentation](https://nomos-tech.notion.site/project)** — Protocol specifications, node internals, and architecture details
|
||||||
|
|||||||
@ -1,6 +1,6 @@
|
|||||||
# Introduction
|
# Introduction
|
||||||
|
|
||||||
The Nomos Testing Framework is a purpose-built toolkit for exercising Nomos in
|
The Nomos Testing Framework is a purpose-built toolkit for exercising Logos in
|
||||||
realistic, multi-node environments. It solves the gap between small, isolated
|
realistic, multi-node environments. It solves the gap between small, isolated
|
||||||
tests and full-system validation by letting teams describe a cluster layout,
|
tests and full-system validation by letting teams describe a cluster layout,
|
||||||
drive meaningful traffic, and assert the outcomes in one coherent plan.
|
drive meaningful traffic, and assert the outcomes in one coherent plan.
|
||||||
@ -9,7 +9,38 @@ It is for protocol engineers, infrastructure operators, and QA teams who need
|
|||||||
repeatable confidence that validators, executors, and data-availability
|
repeatable confidence that validators, executors, and data-availability
|
||||||
components work together under network and timing constraints.
|
components work together under network and timing constraints.
|
||||||
|
|
||||||
Multi-node integration testing is required because many Nomos behaviors—block
|
Multi-node integration testing is required because many Logos behaviors—block
|
||||||
progress, data availability, liveness under churn—only emerge when several
|
progress, data availability, liveness under churn—only emerge when several
|
||||||
roles interact over real networking and time. This framework makes those checks
|
roles interact over real networking and time. This framework makes those checks
|
||||||
declarative, observable, and portable across environments.
|
declarative, observable, and portable across environments.
|
||||||
|
|
||||||
|
## A Scenario in 20 Lines
|
||||||
|
|
||||||
|
Here's the conceptual shape of every test you'll write:
|
||||||
|
|
||||||
|
```rust
|
||||||
|
// 1. Define the cluster
|
||||||
|
let scenario = ScenarioBuilder::topology_with(|t| {
|
||||||
|
t.network_star()
|
||||||
|
.validators(3)
|
||||||
|
.executors(2)
|
||||||
|
})
|
||||||
|
// 2. Add workloads (traffic)
|
||||||
|
.transactions_with(|tx| tx.rate(10).users(5))
|
||||||
|
.da_with(|da| da.channel_rate(2).blob_rate(2))
|
||||||
|
|
||||||
|
// 3. Define success criteria
|
||||||
|
.expect_consensus_liveness()
|
||||||
|
|
||||||
|
// 4. Set experiment duration
|
||||||
|
.with_run_duration(Duration::from_secs(60))
|
||||||
|
.build();
|
||||||
|
|
||||||
|
// 5. Deploy and run
|
||||||
|
let runner = deployer.deploy(&scenario).await?;
|
||||||
|
runner.run(&mut scenario).await?;
|
||||||
|
```
|
||||||
|
|
||||||
|
This pattern—topology, workloads, expectations, duration—repeats across all scenarios in this book.
|
||||||
|
|
||||||
|
**Learn more:** For protocol-level documentation and node internals, see the [Nomos Project Documentation](https://nomos-tech.notion.site/project).
|
||||||
|
|||||||
@ -291,13 +291,14 @@ POL_PROOF_DEV_MODE=true scripts/run/run-examples.sh -t 60 -v 3 -e 1 compose
|
|||||||
**Example usage in expectations:**
|
**Example usage in expectations:**
|
||||||
|
|
||||||
```rust
|
```rust
|
||||||
async fn evaluate(&self, ctx: &RunContext) -> Result<(), DynError> {
|
use testing_framework_core::scenario::{DynError, RunContext};
|
||||||
let clients = ctx.node_clients().validator_clients();
|
|
||||||
let client = &clients[0];
|
async fn evaluate(ctx: &RunContext) -> Result<(), DynError> {
|
||||||
|
let client = &ctx.node_clients().validator_clients()[0];
|
||||||
|
|
||||||
let info = client.consensus_info().await?;
|
let info = client.consensus_info().await?;
|
||||||
tracing::info!(?info, "consensus info from validator 0");
|
tracing::info!(height = info.height, "consensus info from validator 0");
|
||||||
|
|
||||||
Ok(())
|
Ok(())
|
||||||
}
|
}
|
||||||
```
|
```
|
||||||
@ -362,4 +363,3 @@ kubectl logs -n nomos-debug -l nomos/logical-role=validator
|
|||||||
- [Troubleshooting](troubleshooting.md) — Log-related debugging (see "Where to Find Logs")
|
- [Troubleshooting](troubleshooting.md) — Log-related debugging (see "Where to Find Logs")
|
||||||
- [Running Examples](running-examples.md) — Runner-specific logging details
|
- [Running Examples](running-examples.md) — Runner-specific logging details
|
||||||
- [Prerequisites & Setup](prerequisites.md) — Setup before running
|
- [Prerequisites & Setup](prerequisites.md) — Setup before running
|
||||||
|
|
||||||
|
|||||||
@ -43,10 +43,13 @@ Expectations typically use BlockFeed to verify block production and inclusion of
|
|||||||
**Example: Counting blocks during a run**
|
**Example: Counting blocks during a run**
|
||||||
|
|
||||||
```rust
|
```rust
|
||||||
use std::sync::{Arc, atomic::{AtomicU64, Ordering}};
|
use std::sync::{
|
||||||
|
Arc,
|
||||||
|
atomic::{AtomicU64, Ordering},
|
||||||
|
};
|
||||||
|
|
||||||
use async_trait::async_trait;
|
use async_trait::async_trait;
|
||||||
use testing_framework_core::scenario::{DynError, Expectation, RunContext};
|
use testing_framework_core::scenario::{DynError, Expectation, RunContext};
|
||||||
use tokio::{spawn, time::sleep, select, pin};
|
|
||||||
|
|
||||||
struct MinimumBlocksExpectation {
|
struct MinimumBlocksExpectation {
|
||||||
min_blocks: u64,
|
min_blocks: u64,
|
||||||
@ -67,7 +70,7 @@ impl Expectation for MinimumBlocksExpectation {
|
|||||||
let mut receiver = ctx.block_feed().subscribe();
|
let mut receiver = ctx.block_feed().subscribe();
|
||||||
|
|
||||||
// Spawn a task to count blocks
|
// Spawn a task to count blocks
|
||||||
spawn(async move {
|
tokio::spawn(async move {
|
||||||
loop {
|
loop {
|
||||||
match receiver.recv().await {
|
match receiver.recv().await {
|
||||||
Ok(_record) => {
|
Ok(_record) => {
|
||||||
@ -110,12 +113,12 @@ impl Expectation for MinimumBlocksExpectation {
|
|||||||
**Example: Inspecting block contents**
|
**Example: Inspecting block contents**
|
||||||
|
|
||||||
```rust
|
```rust
|
||||||
use testing_framework_core::scenario::{BlockRecord, RunContext};
|
use testing_framework_core::scenario::{DynError, RunContext};
|
||||||
|
|
||||||
async fn start_capture(&mut self, ctx: &RunContext) -> Result<(), DynError> {
|
async fn start_capture(ctx: &RunContext) -> Result<(), DynError> {
|
||||||
let mut receiver = ctx.block_feed().subscribe();
|
let mut receiver = ctx.block_feed().subscribe();
|
||||||
|
|
||||||
spawn(async move {
|
tokio::spawn(async move {
|
||||||
loop {
|
loop {
|
||||||
match receiver.recv().await {
|
match receiver.recv().await {
|
||||||
Ok(record) => {
|
Ok(record) => {
|
||||||
@ -123,8 +126,7 @@ async fn start_capture(&mut self, ctx: &RunContext) -> Result<(), DynError> {
|
|||||||
let header_id = &record.header;
|
let header_id = &record.header;
|
||||||
|
|
||||||
// Access full block
|
// Access full block
|
||||||
let block = &record.block;
|
let tx_count = record.block.transactions().len();
|
||||||
let tx_count = block.transactions().len();
|
|
||||||
|
|
||||||
tracing::debug!(
|
tracing::debug!(
|
||||||
?header_id,
|
?header_id,
|
||||||
@ -195,26 +197,32 @@ impl Workload for DelayedWorkload {
|
|||||||
**Example: Rate limiting based on block production**
|
**Example: Rate limiting based on block production**
|
||||||
|
|
||||||
```rust
|
```rust
|
||||||
async fn start(&self, ctx: &RunContext) -> Result<(), DynError> {
|
use testing_framework_core::scenario::{DynError, RunContext};
|
||||||
|
|
||||||
|
async fn generate_request() -> Option<()> {
|
||||||
|
None
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn start(ctx: &RunContext) -> Result<(), DynError> {
|
||||||
let clients = ctx.node_clients().validator_clients();
|
let clients = ctx.node_clients().validator_clients();
|
||||||
let mut receiver = ctx.block_feed().subscribe();
|
let mut receiver = ctx.block_feed().subscribe();
|
||||||
let mut pending_txs = Vec::new();
|
let mut pending_requests: Vec<()> = Vec::new();
|
||||||
|
|
||||||
loop {
|
loop {
|
||||||
tokio::select! {
|
tokio::select! {
|
||||||
// Send batch on new block
|
// Issue a batch on each new block.
|
||||||
Ok(_record) = receiver.recv() => {
|
Ok(_record) = receiver.recv() => {
|
||||||
if !pending_txs.is_empty() {
|
if !pending_requests.is_empty() {
|
||||||
tracing::debug!(count = pending_txs.len(), "sending batch on new block");
|
tracing::debug!(count = pending_requests.len(), "issuing requests on new block");
|
||||||
for tx in pending_txs.drain(..) {
|
for _req in pending_requests.drain(..) {
|
||||||
clients[0].send_transaction(tx).await?;
|
let _info = clients[0].consensus_info().await?;
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// Generate transactions continuously
|
// Generate work continuously.
|
||||||
Some(tx) = generate_transaction() => {
|
Some(req) = generate_request() => {
|
||||||
pending_txs.push(tx);
|
pending_requests.push(req);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@ -238,7 +246,9 @@ async fn start(&self, ctx: &RunContext) -> Result<(), DynError> {
|
|||||||
Example direct polling in expectations:
|
Example direct polling in expectations:
|
||||||
|
|
||||||
```rust
|
```rust
|
||||||
async fn evaluate(&mut self, ctx: &RunContext) -> Result<(), DynError> {
|
use testing_framework_core::scenario::{DynError, RunContext};
|
||||||
|
|
||||||
|
async fn evaluate(ctx: &RunContext) -> Result<(), DynError> {
|
||||||
let client = &ctx.node_clients().validator_clients()[0];
|
let client = &ctx.node_clients().validator_clients()[0];
|
||||||
|
|
||||||
// Poll current height once
|
// Poll current height once
|
||||||
@ -255,16 +265,18 @@ async fn evaluate(&mut self, ctx: &RunContext) -> Result<(), DynError> {
|
|||||||
Access aggregated statistics without subscribing to the feed:
|
Access aggregated statistics without subscribing to the feed:
|
||||||
|
|
||||||
```rust
|
```rust
|
||||||
async fn evaluate(&mut self, ctx: &RunContext) -> Result<(), DynError> {
|
use testing_framework_core::scenario::{DynError, RunContext};
|
||||||
|
|
||||||
|
async fn evaluate(ctx: &RunContext, expected_min: u64) -> Result<(), DynError> {
|
||||||
let stats = ctx.block_feed().stats();
|
let stats = ctx.block_feed().stats();
|
||||||
let total_txs = stats.total_transactions();
|
let total_txs = stats.total_transactions();
|
||||||
|
|
||||||
tracing::info!(total_txs, "transactions observed across all blocks");
|
tracing::info!(total_txs, "transactions observed across all blocks");
|
||||||
|
|
||||||
if total_txs < self.expected_min {
|
if total_txs < expected_min {
|
||||||
return Err(format!(
|
return Err(format!(
|
||||||
"expected at least {} transactions, observed {}",
|
"expected at least {} transactions, observed {}",
|
||||||
self.expected_min, total_txs
|
expected_min, total_txs
|
||||||
).into());
|
).into());
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@ -32,6 +32,24 @@ Operational readiness focuses on prerequisites, environment fit, and clear signa
|
|||||||
- Health checks prevent premature workload execution
|
- Health checks prevent premature workload execution
|
||||||
- Consensus liveness expectations validate basic operation
|
- Consensus liveness expectations validate basic operation
|
||||||
|
|
||||||
|
## Runner-Agnostic Design
|
||||||
|
|
||||||
|
The framework is intentionally **runner-agnostic**: the same scenario plan runs across all deployment targets. Understanding which operational concerns apply to each runner helps you choose the right fit.
|
||||||
|
|
||||||
|
| Concern | Host | Compose | Kubernetes |
|
||||||
|
|---------|------|---------|------------|
|
||||||
|
| **Topology** | Full support | Full support | Full support |
|
||||||
|
| **Workloads** | All workloads | All workloads | All workloads |
|
||||||
|
| **Expectations** | All expectations | All expectations | All expectations |
|
||||||
|
| **Chaos / Node Control** | Not supported | Supported | Not yet |
|
||||||
|
| **Metrics / Observability** | Manual setup | External stack | Cluster-wide |
|
||||||
|
| **Log Collection** | Temp files | Container logs | Pod logs |
|
||||||
|
| **Isolation** | Process-level | Container | Pod + namespace |
|
||||||
|
| **Setup Time** | < 1 min | 2-5 min | 5-10 min |
|
||||||
|
| **CI Recommended?** | Smoke tests | Primary | Large-scale only |
|
||||||
|
|
||||||
|
**Key insight:** Operational concerns (prerequisites, environment variables) are largely **consistent** across runners, while deployment-specific concerns (isolation, chaos support) vary by backend.
|
||||||
|
|
||||||
## Operational Workflow
|
## Operational Workflow
|
||||||
|
|
||||||
```mermaid
|
```mermaid
|
||||||
|
|||||||
@ -8,6 +8,20 @@ The Nomos Testing Framework enables you to test consensus, data availability, an
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
## Core Concept
|
||||||
|
|
||||||
|
**Everything in this framework is a Scenario.**
|
||||||
|
|
||||||
|
A Scenario is a controlled experiment over time, composed of:
|
||||||
|
- **Topology** — The cluster shape (validators, executors, network layout)
|
||||||
|
- **Workloads** — Traffic and conditions that exercise the system (transactions, DA, chaos)
|
||||||
|
- **Expectations** — Success criteria verified after execution (liveness, inclusion, recovery)
|
||||||
|
- **Duration** — The time window for the experiment
|
||||||
|
|
||||||
|
This single abstraction makes tests declarative, portable, and composable.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
## How It Works
|
## How It Works
|
||||||
|
|
||||||
```mermaid
|
```mermaid
|
||||||
@ -56,7 +70,10 @@ flowchart LR
|
|||||||
## Quick Example
|
## Quick Example
|
||||||
|
|
||||||
```rust
|
```rust
|
||||||
|
use std::time::Duration;
|
||||||
|
|
||||||
use testing_framework_core::scenario::ScenarioBuilder;
|
use testing_framework_core::scenario::ScenarioBuilder;
|
||||||
|
use testing_framework_core::scenario::Deployer as _;
|
||||||
use testing_framework_runner_local::LocalDeployer;
|
use testing_framework_runner_local::LocalDeployer;
|
||||||
use testing_framework_workflows::ScenarioBuilderExt;
|
use testing_framework_workflows::ScenarioBuilderExt;
|
||||||
|
|
||||||
@ -67,7 +84,7 @@ async fn main() -> anyhow::Result<()> {
|
|||||||
.validators(3)
|
.validators(3)
|
||||||
.executors(1)
|
.executors(1)
|
||||||
})
|
})
|
||||||
.transactions_with(|tx| tx.rate(10.0).users(5))
|
.transactions_with(|tx| tx.rate(10).users(5))
|
||||||
.expect_consensus_liveness()
|
.expect_consensus_liveness()
|
||||||
.with_run_duration(Duration::from_secs(60))
|
.with_run_duration(Duration::from_secs(60))
|
||||||
.build();
|
.build();
|
||||||
@ -116,6 +133,8 @@ These roles interact tightly, which is why meaningful testing must be performed
|
|||||||
|
|
||||||
The Nomos Testing Framework provides the infrastructure to orchestrate these multi-node scenarios reliably across development, CI, and production-like environments.
|
The Nomos Testing Framework provides the infrastructure to orchestrate these multi-node scenarios reliably across development, CI, and production-like environments.
|
||||||
|
|
||||||
|
**Learn more about the protocol:** [Nomos Project Documentation](https://nomos-tech.notion.site/project)
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Documentation Structure
|
## Documentation Structure
|
||||||
|
|||||||
@ -1,5 +1,7 @@
|
|||||||
# Core Content: ScenarioBuilderExt Patterns
|
# Core Content: ScenarioBuilderExt Patterns
|
||||||
|
|
||||||
|
> **When should I read this?** After writing 2-3 scenarios. This page documents patterns that emerge from real usage—come back when you're refactoring or standardizing your test suite.
|
||||||
|
|
||||||
Patterns that keep scenarios readable and reusable:
|
Patterns that keep scenarios readable and reusable:
|
||||||
|
|
||||||
- **Topology-first**: start by shaping the cluster (counts, layout) so later
|
- **Topology-first**: start by shaping the cluster (counts, layout) so later
|
||||||
|
|||||||
@ -334,15 +334,15 @@ thread 'main' panicked at 'workload init failed: insufficient wallets'
|
|||||||
**Fix:**
|
**Fix:**
|
||||||
|
|
||||||
```rust
|
```rust
|
||||||
// In your scenario:
|
use testing_framework_core::scenario::ScenarioBuilder;
|
||||||
let scenario = Scenario::builder("my_test")
|
use testing_framework_workflows::ScenarioBuilderExt;
|
||||||
.topology(
|
|
||||||
Topology::preset_3v1e()
|
let scenario = ScenarioBuilder::topology_with(|t| t.network_star().validators(3).executors(1))
|
||||||
.wallets(20) // ← Increase wallet count
|
.wallets(20) // ← Increase wallet count
|
||||||
)
|
.transactions_with(|tx| {
|
||||||
.workload(TransactionWorkload::new()
|
tx.users(10) // ← Must be ≤ wallets(20)
|
||||||
.users(10) // ← Must be ≤ wallets(20)
|
.rate(5)
|
||||||
.rate(5.0))
|
})
|
||||||
.build();
|
.build();
|
||||||
```
|
```
|
||||||
|
|
||||||
@ -453,13 +453,15 @@ thread 'main' panicked at 'expectations failed'
|
|||||||
**Fix:**
|
**Fix:**
|
||||||
|
|
||||||
```rust
|
```rust
|
||||||
// Increase run duration to allow more blocks
|
use std::time::Duration;
|
||||||
let scenario = Scenario::builder("my_test")
|
|
||||||
.topology(Topology::preset_3v1e())
|
use testing_framework_core::scenario::ScenarioBuilder;
|
||||||
.with_run_duration(Duration::from_secs(120)) // ← Give more time
|
use testing_framework_workflows::ScenarioBuilderExt;
|
||||||
.expectation(ConsensusLiveness::new()
|
|
||||||
.min_blocks(5) // ← Adjust expectation to match duration
|
// Increase run duration to allow more blocks.
|
||||||
)
|
let scenario = ScenarioBuilder::topology_with(|t| t.network_star().validators(3).executors(1))
|
||||||
|
.expect_consensus_liveness()
|
||||||
|
.with_run_duration(Duration::from_secs(120)) // ← Give more time
|
||||||
.build();
|
.build();
|
||||||
```
|
```
|
||||||
|
|
||||||
|
|||||||
@ -1,6 +1,63 @@
|
|||||||
# What You Will Learn
|
# What You Will Learn
|
||||||
|
|
||||||
This book gives you a clear mental model for Nomos multi-node testing, shows how
|
This book gives you a clear mental model for Logos multi-node testing, shows how
|
||||||
to author scenarios that pair realistic workloads with explicit expectations,
|
to author scenarios that pair realistic workloads with explicit expectations,
|
||||||
and guides you to run them across local, containerized, and cluster environments
|
and guides you to run them across local, containerized, and cluster environments
|
||||||
without changing the plan.
|
without changing the plan.
|
||||||
|
|
||||||
|
## By the End of This Book, You Will Be Able To:
|
||||||
|
|
||||||
|
**Understand the Framework**
|
||||||
|
- Explain the six-phase scenario lifecycle (Build, Deploy, Capture, Execute, Evaluate, Cleanup)
|
||||||
|
- Describe how Deployers, Runners, Workloads, and Expectations work together
|
||||||
|
- Navigate the crate architecture and identify extension points
|
||||||
|
- Understand when to use each runner (Host, Compose, Kubernetes)
|
||||||
|
|
||||||
|
**Author and Run Scenarios**
|
||||||
|
- Define multi-node topologies with validators and executors
|
||||||
|
- Configure transaction and DA workloads with appropriate rates
|
||||||
|
- Add consensus liveness and inclusion expectations
|
||||||
|
- Run scenarios across all three deployment modes
|
||||||
|
- Use BlockFeed to monitor block production in real-time
|
||||||
|
- Implement chaos testing with node restarts
|
||||||
|
|
||||||
|
**Operate in Production**
|
||||||
|
- Set up prerequisites and dependencies correctly
|
||||||
|
- Configure environment variables for different runners
|
||||||
|
- Integrate tests into CI/CD pipelines (GitHub Actions)
|
||||||
|
- Troubleshoot common failure scenarios
|
||||||
|
- Collect and analyze logs from multi-node runs
|
||||||
|
- Optimize test durations and resource usage
|
||||||
|
|
||||||
|
**Extend the Framework**
|
||||||
|
- Implement custom Workload traits for new traffic patterns
|
||||||
|
- Create custom Expectation traits for domain-specific checks
|
||||||
|
- Add new Deployer implementations for different backends
|
||||||
|
- Contribute topology helpers and DSL extensions
|
||||||
|
|
||||||
|
## Learning Path
|
||||||
|
|
||||||
|
**Beginner** (0-2 hours)
|
||||||
|
- Read [Quickstart](quickstart.md) and run your first scenario
|
||||||
|
- Review [Examples](examples.md) to see common patterns
|
||||||
|
- Understand [Scenario Lifecycle](scenario-lifecycle.md) phases
|
||||||
|
|
||||||
|
**Intermediate** (2-8 hours)
|
||||||
|
- Study [Runners](runners.md) comparison and choose appropriate mode
|
||||||
|
- Learn [Workloads & Expectations](workloads.md) in depth
|
||||||
|
- Review [Prerequisites & Setup](prerequisites.md) for your environment
|
||||||
|
- Practice with [Advanced Examples](examples-advanced.md)
|
||||||
|
|
||||||
|
**Advanced** (8+ hours)
|
||||||
|
- Master [Environment Variables](environment-variables.md) configuration
|
||||||
|
- Implement [Custom Workloads](extending.md) for your use cases
|
||||||
|
- Set up [CI Integration](ci-integration.md) for automated testing
|
||||||
|
- Explore [Internal Crate Reference](internal-crate-reference.md) for deep dives
|
||||||
|
|
||||||
|
## What This Book Does NOT Cover
|
||||||
|
|
||||||
|
- **Nomos node internals** — This book focuses on testing infrastructure, not the blockchain protocol implementation. See the Nomos node repository for protocol documentation.
|
||||||
|
- **Consensus algorithm theory** — We assume familiarity with basic blockchain concepts (validators, blocks, transactions, data availability).
|
||||||
|
- **Rust language basics** — Examples use Rust, but we don't teach the language. See [The Rust Book](https://doc.rust-lang.org/book/) if you're new to Rust.
|
||||||
|
- **Kubernetes administration** — We show how to use the K8s runner, but don't cover cluster setup, networking, or operations.
|
||||||
|
- **Docker fundamentals** — We assume basic Docker/Compose knowledge for the Compose runner.
|
||||||
|
|||||||
Loading…
x
Reference in New Issue
Block a user