139 lines
5.7 KiB
Markdown
Raw Normal View History

# Testing Framework
System-level testing for networked applications from Rust.
The testing framework deploys and controls processes, containers, and clusters. Tests can run several nodes over network connections for a bounded period. Application-specific configuration and clients stay outside the framework, so the runtime can be used with a key-value store, a Raft cluster, a message queue, or a blockchain.
[**Get Started**](quickstart.md)
---
## Scenarios
A declarative test is represented by a `Scenario` containing:
- **Topology** — the system under test (a uniform cluster, a composed application stack, or attached external nodes)
- **Workloads** — traffic and conditions that exercise the system
- **Expectations** — success criteria verified after execution
- **Duration** — the time window for the experiment
```mermaid
flowchart LR
subgraph SC["Scenario"]
T["topology<br/><small>the system under test</small>"]:::cl
W["workloads<br/><small>drive traffic</small>"]:::sc
EX["expectations<br/><small>verify outcomes</small>"]:::sc
D["duration<br/><small>the run window</small>"]:::sc
end
SC --> RN["Runner<br/><small>deploy · run · evaluate · teardown</small>"]:::sc
classDef cl stroke:#4a90d9,stroke-width:2.5px;
classDef sc stroke:#9b6dd6,stroke-width:2.5px;
```
The scenario runtime executes these parts in the same order for each declarative entry pattern. The entry pattern determines how the system is supplied.
---
## Entry Patterns
```mermaid
flowchart LR
A[Uniform managed cluster]:::cl --> S[Scenario]
B[AppHost composed stack] --> S
C[Attached / external nodes]:::cl --> S
S:::sc --> R[Runner: workloads + expectations]:::sc
M[ManualCluster] --> I[Imperative orchestration]
classDef cl stroke:#4a90d9,stroke-width:2.5px;
classDef hd stroke:#4caf7d,stroke-width:2.5px;
classDef sc stroke:#9b6dd6,stroke-width:2.5px;
```
Three entry patterns use the scenario runtime:
1. **Uniform managed cluster** — the framework generates configs and launches N identical nodes from a topology. See [Part IV](part-iv.md).
2. **AppHost composed stack** — the app layer deploys heterogeneous components (processes, child clusters, in-process services) as one system and exposes typed handles to workloads. See [Part II](part-ii.md).
3. **Attached and external nodes** — the scenario targets clusters you already run, or plain URLs. See [Existing and External Clusters](external-clusters.md).
**[ManualCluster](manual-cluster.md)** is the imperative alternative. It provides direct start, stop, restart, and readiness operations without the scenario runner, including for step-driven BDD harnesses.
If you are not sure which to use, read [Choosing an Entry Pattern](entry-patterns.md).
---
## Provided APIs
**Declarative API**
- Express tests as topology + workloads + expectations
- Reuse the same definition across local, Compose, and Kubernetes deployers
- Compose stacks from reusable application deployments
**Application layer**
- Deploy heterogeneous systems as one root `AppDeployment`
- Typed, named handles connect workloads to components
- Deterministic cleanup, including on partial-deployment failure
**Runtime capabilities**
- Capability-gated node control: restart nodes from workloads, portably
- Continuous observation: snapshots, history, and event streams of application state
- Telemetry: metrics, logs, and tracing endpoints
**Operations**
- Binary providers resolve node binaries from paths, env vars, builds, or downloads
- Reproducible deployments via seeds
- Artifact preservation for post-mortem debugging
---
## Quick Example
```rust,ignore
use testing_framework_app::{AppHost, AppHostLocalDeployer, AppScenarioBuilderExt as _};
use testing_framework_core::scenario::Deployer as _;
let mut scenario = AppHost::scenario()
.with_app(KvLocalApp::nodes(3))
.with_workload(KvAppHostConvergence::new(3))
.build()?;
let runner = AppHostLocalDeployer::default().deploy(&scenario).await?;
runner.run(&mut scenario).await?;
```
This deploys a three-node key-value store cluster, runs a convergence workload against it (including a node restart), and tears everything down. The remaining chapters cover each part of this pattern in detail.
[View the example apps](running-examples.md)
---
## The Example Apps
The repository includes small applications under `examples/` that exercise the framework APIs:
| App | Demonstrates |
|-----|--------------|
| `kvstore` | Uniform clusters, app hosting, convergence testing, all three deployers |
| `openraft_kv` | Node control, failover, continuous observation |
| `multi_app` | Composing heterogeneous stacks with typed handles |
| `nats`, `redis_streams` | Testing third-party binaries you did not write |
| `pubsub`, `queue`, `metrics_counter` | Additional workload and expectation patterns |
Some chapters also link to adopter repositories. The examples listed in this table run from this workspace.
---
## Documentation Structure
| Section | Description |
|---------|-------------|
| **[Part I — Mental Model](part-i.md)** | The core abstractions and how to choose between entry patterns |
| **[Part II — Composing Applications](part-ii.md)** | The app layer: deployments, handles, teardown |
| **[Part III — Scenario Runtime](part-iii.md)** | Workloads, expectations, capabilities, observation |
| **[Part IV — Uniform Clusters](part-iv.md)** | Implementing `Application`, topology, config, manual control |
| **[Part V — Deployers and Sources](part-v.md)** | Local, Compose, Kubernetes, external clusters, binaries |
| **[Part VI — Extending](part-vi.md)** | Extension points, crate map, boundaries |
| **[Part VII — Operations](part-vii.md)** | Running examples, CI, diagnostics, troubleshooting |
---
Start with the **[Quickstart](quickstart.md)**.