docs(gitbook): align log/versions guidance and document keep-logs

book/src/operations.md

@@ -108,8 +108,9 @@ cargo run -p runner-examples --bin local_runner
 - `NOMOS_DEMO_EXECUTORS=2` — Number of executors (default: 1, or use legacy `LOCAL_DEMO_EXECUTORS`)
 - `NOMOS_DEMO_RUN_SECS=120` — Run duration in seconds (default: 60, or use legacy `LOCAL_DEMO_RUN_SECS`)
 - `NOMOS_NODE_BIN` / `NOMOS_EXECUTOR_BIN` — Paths to binaries (required for direct run)
-- `NOMOS_TESTS_TRACING=true` — Enable persistent file logging
-- `NOMOS_LOG_DIR=/tmp/logs` — Directory for per-node log files
+- `NOMOS_LOG_DIR=/tmp/logs` — Directory for per-node log files (works across runners)
+- `NOMOS_TESTS_KEEP_LOGS=1` — Keep per-run temporary directories (useful for debugging/CI artifacts)
+- `NOMOS_TESTS_TRACING=true` — Enable the debug tracing preset (optional; combine with `NOMOS_LOG_DIR` unless you have external tracing backends configured)
 - `NOMOS_LOG_LEVEL=debug` — Set log level (default: info)
 - `NOMOS_LOG_FILTER=consensus=trace,da=debug` — Fine-grained module filtering
 
@@ -317,7 +318,7 @@ RUST_LOG=debug NOMOS_LOG_LEVEL=debug cargo run -p runner-examples --bin local_ru
 | `NOMOS_LOG_DIR` | None (console only) | Directory for per-node log files. If unset, logs go to stdout/stderr. |
 | `NOMOS_LOG_LEVEL` | `info` | Global log level: `error`, `warn`, `info`, `debug`, `trace` |
 | `NOMOS_LOG_FILTER` | None | Fine-grained target filtering (e.g., `consensus=trace,da=debug`) |
-| `NOMOS_TESTS_TRACING` | `false` | Enable tracing subscriber for local runner file logging |
+| `NOMOS_TESTS_TRACING` | `false` | Enable the debug tracing preset (optional; combine with `NOMOS_LOG_DIR` unless you have external tracing backends configured) |
 | `NOMOS_OTLP_ENDPOINT` | None | OTLP trace endpoint (optional, disables OTLP noise if unset) |
 | `NOMOS_OTLP_METRICS_ENDPOINT` | None | OTLP metrics endpoint (optional) |
 
@@ -339,7 +340,7 @@ When `NOMOS_LOG_DIR` is set, each node writes logs to separate files:
 - **Validators**: Prefix `nomos-node-0`, `nomos-node-1`, etc. (may include timestamp suffix)
 - **Executors**: Prefix `nomos-executor-0`, `nomos-executor-1`, etc. (may include timestamp suffix)
 
-**Local runner caveat:** By default, the local runner writes logs to temporary directories in the working directory. These are automatically cleaned up after tests complete. To preserve logs, you MUST set both `NOMOS_TESTS_TRACING=true` AND `NOMOS_LOG_DIR=/path/to/logs`.
+**Local runner note:** The local runner uses per-run temporary directories under the current working directory and removes them after the run unless `NOMOS_TESTS_KEEP_LOGS=1`. Use `NOMOS_LOG_DIR=/path/to/logs` to write per-node log files to a stable location.
 
 ### Filter Target Names
 
@@ -374,7 +375,6 @@ POL_PROOF_DEV_MODE=true cargo run -p runner-examples --bin local_runner
 
 **Persistent file output:**
 ```bash
-NOMOS_TESTS_TRACING=true \
 NOMOS_LOG_DIR=/tmp/local-logs \
 POL_PROOF_DEV_MODE=true \
 cargo run -p runner-examples --bin local_runner
@@ -385,7 +385,7 @@ ls /tmp/local-logs/
 # May include timestamps in filename
 ```
 
-**Both flags required:** You MUST set both `NOMOS_TESTS_TRACING=true` (enables tracing file sink) AND `NOMOS_LOG_DIR` (specifies directory) to get persistent logs.
+**Tip:** Use `NOMOS_LOG_DIR` for persistent per-node log files, and `NOMOS_TESTS_KEEP_LOGS=1` if you want to keep the per-run temporary directories (configs/state) for post-mortem inspection.
 
 #### Compose Runner
 

book/src/quickstart.md

@@ -77,8 +77,8 @@ let _handle = runner.run(&mut plan).await?;
 - Nodes spawn as local processes
 - Consensus starts producing blocks
 - Scenario runs for the configured duration
-- Node logs written to temporary directories in working directory (auto-cleaned up after test)
-- To persist logs: set `NOMOS_TESTS_TRACING=true` and `NOMOS_LOG_DIR=/path/to/logs` (files will have prefix like `nomos-node-0*`, may include timestamps)
+- Node state/logs written under a temporary per-run directory in the current working directory (removed after the run unless `NOMOS_TESTS_KEEP_LOGS=1`)
+- To write per-node log files to a stable location: set `NOMOS_LOG_DIR=/path/to/logs` (files will have prefix like `nomos-node-0*`, may include timestamps)
 
 ## What Just Happened?
 

book/src/troubleshooting.md

@@ -30,12 +30,12 @@ Common symptoms and likely causes:
 
 | Runner | Default Output | With `NOMOS_LOG_DIR` + Flags | Access Command |
 |--------|---------------|------------------------------|----------------|
-| **Host** (local) | Temporary directories (cleaned up) | Per-node files with prefix `nomos-node-{index}` (requires `NOMOS_TESTS_TRACING=true`) | `cat $NOMOS_LOG_DIR/nomos-node-0*` |
+| **Host** (local) | Per-run temporary directories under the current working directory (removed unless `NOMOS_TESTS_KEEP_LOGS=1`) | Per-node files with prefix `nomos-node-{index}` (set `NOMOS_LOG_DIR`) | `cat $NOMOS_LOG_DIR/nomos-node-0*` |
 | **Compose** | Docker container stdout/stderr | Per-node files inside containers (if path is mounted) | `docker ps` then `docker logs <container-id>` |
-| **K8s** | Pod stdout/stderr | Per-node files inside pods (if path is mounted) | `kubectl logs -l app=nomos-validator` |
+| **K8s** | Pod stdout/stderr | Per-node files inside pods (if path is mounted) | `kubectl logs -l nomos/logical-role=validator` |
 
 **Important Notes:**
-- **Host runner** (local processes): Logs go to system temporary directories (NOT in working directory) by default and are automatically cleaned up after tests. To persist logs, you MUST set both `NOMOS_TESTS_TRACING=true` AND `NOMOS_LOG_DIR=/path/to/logs`.
+- **Host runner** (local processes): Per-run temporary directories are created under the current working directory and removed after the run unless `NOMOS_TESTS_KEEP_LOGS=1`. To write per-node log files to a stable location, set `NOMOS_LOG_DIR=/path/to/logs`.
 - **Compose/K8s**: Per-node log files only exist inside containers/pods if `NOMOS_LOG_DIR` is set AND the path is writable inside the container/pod. By default, rely on `docker logs` or `kubectl logs`.
 - **File naming**: Log files use prefix `nomos-node-{index}*` or `nomos-executor-{index}*` with timestamps, e.g., `nomos-node-0.2024-12-01T10-30-45.log` (NOT just `.log` suffix).
 - **Container names**: Compose containers include project UUID, e.g., `nomos-compose-<uuid>-validator-0-1` where `<uuid>` is randomly generated per run
@@ -51,7 +51,6 @@ POL_PROOF_DEV_MODE=true cargo run -p runner-examples --bin local_runner 2>&1 | t
 
 **Persistent file output:**
 ```bash
-NOMOS_TESTS_TRACING=true \
 NOMOS_LOG_DIR=/tmp/debug-logs \
 NOMOS_LOG_LEVEL=debug \
 POL_PROOF_DEV_MODE=true \
@@ -109,28 +108,28 @@ docker logs <container-id> > debug.log
 kubectl config view --minify | grep namespace
 
 # All validator pods (add -n <namespace> if not using default)
-kubectl logs -l app=nomos-validator -f
+kubectl logs -l nomos/logical-role=validator -f
 
 # All executor pods
-kubectl logs -l app=nomos-executor -f
+kubectl logs -l nomos/logical-role=executor -f
 
 # Specific pod by name (find exact name first)
-kubectl get pods -l app=nomos-validator  # Find the exact pod name
+kubectl get pods -l nomos/logical-role=validator  # Find the exact pod name
 kubectl logs -f <actual-pod-name>        # Then use it
 
 # With explicit namespace
-kubectl logs -n my-namespace -l app=nomos-validator -f
+kubectl logs -n my-namespace -l nomos/logical-role=validator -f
 ```
 
 **Download logs from crashed pods:**
 
 ```bash
 # Previous logs from crashed pod
-kubectl get pods -l app=nomos-validator  # Find crashed pod name first
+kubectl get pods -l nomos/logical-role=validator  # Find crashed pod name first
 kubectl logs --previous <actual-pod-name> > crashed-validator.log
 
 # Or use label selector for all crashed validators
-for pod in $(kubectl get pods -l app=nomos-validator -o name); do
+for pod in $(kubectl get pods -l nomos/logical-role=validator -o name); do
   kubectl logs --previous $pod > $(basename $pod)-previous.log 2>&1
 done
 ```
@@ -145,11 +144,11 @@ for pod in $(kubectl get pods -o name); do
 done > all-logs.txt
 
 # Or use label selectors (recommended)
-kubectl logs -l app=nomos-validator --tail=500 > validators.log
-kubectl logs -l app=nomos-executor --tail=500 > executors.log
+kubectl logs -l nomos/logical-role=validator --tail=500 > validators.log
+kubectl logs -l nomos/logical-role=executor --tail=500 > executors.log
 
 # With explicit namespace
-kubectl logs -n my-namespace -l app=nomos-validator --tail=500 > validators.log
+kubectl logs -n my-namespace -l nomos/logical-role=validator --tail=500 > validators.log
 ```
 
 ## Debugging Workflow
@@ -180,8 +179,8 @@ ps aux | grep nomos
 docker ps -a --filter "name=nomos-compose-"
 
 # K8s: check pod status (use label selectors, add -n <namespace> if needed)
-kubectl get pods -l app=nomos-validator
-kubectl get pods -l app=nomos-executor
+kubectl get pods -l nomos/logical-role=validator
+kubectl get pods -l nomos/logical-role=executor
 kubectl describe pod <actual-pod-name>  # Get name from above first
 ```
 
@@ -272,11 +271,11 @@ Run a minimal baseline test (e.g., 2 validators, consensus liveness only). If it
 - **Cause**: Helper scripts (`run-examples.sh`, `build-bundle.sh`, `setup-circuits-stack.sh`) require `versions.env` file at repository root.
 - **Fix**: Ensure you're running from the repository root directory. The `versions.env` file should already exist and contains:
   ```
-  VERSION=v0.3.1
-  NOMOS_NODE_REV=d2dd5a5084e1daef4032562c77d41de5e4d495f8
-  NOMOS_BUNDLE_VERSION=v4
+  VERSION=<circuit release tag>
+  NOMOS_NODE_REV=<nomos-node git revision>
+  NOMOS_BUNDLE_VERSION=<bundle schema version>
   ```
-  If the file is missing, restore it from version control or create it with the above content.
+  Use the checked-in `versions.env` at the repository root as the source of truth.
 
 ### "Port already in use"