From 2d388ba1312a47ebfd9393511c60c3dc7b7804af Mon Sep 17 00:00:00 2001 From: andrussal Date: Mon, 15 Dec 2025 20:26:07 +0100 Subject: [PATCH] Docs: environment flags and platform notes --- book/src/operations.md | 22 ++++++++++++++++++++++ book/src/quickstart.md | 1 + book/src/troubleshooting.md | 6 ++++++ 3 files changed, 29 insertions(+) diff --git a/book/src/operations.md b/book/src/operations.md index 1d562b2..0a0ca62 100644 --- a/book/src/operations.md +++ b/book/src/operations.md @@ -67,6 +67,9 @@ This script handles circuit setup, binary building/bundling, image building, and - `NOMOS_NODE_REV=` — nomos-node git revision - `NOMOS_BINARIES_TAR=path/to/bundle.tar.gz` — Use prebuilt bundle - `NOMOS_SKIP_IMAGE_BUILD=1` — Skip image rebuild (compose/k8s) +- `NOMOS_BUNDLE_DOCKER_PLATFORM=linux/arm64|linux/amd64` — Docker platform used when building a Linux bundle on non-Linux hosts (macOS/Windows) +- `COMPOSE_CIRCUITS_PLATFORM=linux-aarch64|linux-x86_64` — Circuits platform used when building the compose/k8s image (defaults based on host arch) +- `SLOW_TEST_ENV=true` — Doubles built-in readiness timeouts (useful in slower CI / constrained laptops) ### Host Runner (Direct Cargo Run) @@ -112,6 +115,10 @@ POL_PROOF_DEV_MODE=true \ cargo run -p runner-examples --bin compose_runner ``` +**Platform note (macOS / Apple silicon):** +- Docker Desktop runs a `linux/arm64` engine. If Linux bundle builds are slow/unstable when producing `.tmp/nomos-binaries-linux-*.tar.gz`, prefer `NOMOS_BUNDLE_DOCKER_PLATFORM=linux/arm64` for local compose/k8s runs. +- If you need amd64 images/binaries specifically (e.g., deploying to amd64-only environments), set `NOMOS_BUNDLE_DOCKER_PLATFORM=linux/amd64` and expect slower builds via emulation. + **Alternative:** Manual circuit/image setup (rebuilds during image build): ```bash @@ -394,6 +401,12 @@ cargo run -p runner-examples --bin compose_runner # Containers remain running after test—inspect with docker logs or docker exec ``` +**Compose networking/debug knobs:** +- `COMPOSE_RUNNER_HOST=127.0.0.1` — host used for readiness probes (override for remote Docker daemons / VM networking) +- `COMPOSE_RUNNER_HOST_GATEWAY=host.docker.internal:host-gateway` — controls the `extra_hosts` entry injected into compose (set to `disable` to omit) +- `TESTNET_RUNNER_PRESERVE=1` — alias for `COMPOSE_RUNNER_PRESERVE=1` +- `COMPOSE_GRAFANA_PORT=` — pin Grafana to a fixed host port instead of ephemeral assignment + **Note:** Container names follow pattern `nomos-compose-{uuid}-validator-{index}-1` where `{uuid}` changes per run. #### K8s Runner @@ -425,6 +438,15 @@ kubectl logs nomos-validator-0 > validator-0.log kubectl logs nomos-executor-1 > executor-1.log ``` +**K8s environment notes:** +- The k8s runner is optimized for local clusters (Docker Desktop Kubernetes / minikube / kind): + - The default image `logos-blockchain-testing:local` must be available on the cluster’s nodes (Docker Desktop shares the local daemon; kind/minikube often requires an explicit image load step). + - The Helm chart mounts KZG params via a `hostPath` to your workspace path; this typically won’t work on remote/managed clusters without replacing it with a PV/CSI volume or baking the params into an image. +- Debug helpers: + - `K8S_RUNNER_DEBUG=1` — logs Helm stdout/stderr for install commands. + - `K8S_RUNNER_PRESERVE=1` — keep the namespace/release after the run. + - `K8S_RUNNER_NODE_HOST=` — override NodePort host resolution for non-local clusters. + **Specify namespace (if not using default):** ```bash kubectl logs -n my-namespace -l app=nomos-validator -f diff --git a/book/src/quickstart.md b/book/src/quickstart.md index c7b09a1..e4179ab 100644 --- a/book/src/quickstart.md +++ b/book/src/quickstart.md @@ -8,6 +8,7 @@ Get a working example running quickly. - This repository cloned - Unix-like system (tested on Linux and macOS) - For Docker Compose examples: Docker daemon running +- For Docker Desktop on Apple silicon (compose/k8s): set `NOMOS_BUNDLE_DOCKER_PLATFORM=linux/arm64` to avoid slow/fragile amd64 emulation builds - **`versions.env` file** at repository root (defines VERSION, NOMOS_NODE_REV, NOMOS_BUNDLE_VERSION) **Note:** `nomos-node` binaries are built automatically on demand or can be provided via prebuilt bundles. diff --git a/book/src/troubleshooting.md b/book/src/troubleshooting.md index 4782842..30f043f 100644 --- a/book/src/troubleshooting.md +++ b/book/src/troubleshooting.md @@ -5,6 +5,11 @@ - **`POL_PROOF_DEV_MODE=true`** MUST be set for all runners (host, compose, k8s) to avoid expensive Groth16 proof generation that causes timeouts - **KZG circuit assets** must be present at `testing-framework/assets/stack/kzgrs_test_params/kzgrs_test_params` (note the repeated filename) for DA workloads +**Platform/Environment Notes:** +- **macOS + Docker Desktop (Apple silicon):** prefer `NOMOS_BUNDLE_DOCKER_PLATFORM=linux/arm64` for local compose/k8s runs to avoid slow/fragile amd64 emulation builds. +- **Disk space:** bundle/image builds are storage-heavy. If you see I/O errors or Docker build failures, check free space and prune old artifacts (`.tmp/`, `target/`, and Docker build cache) before retrying. +- **K8s runner scope:** the default Helm chart mounts KZG params via `hostPath` and uses a local image tag (`logos-blockchain-testing:local`). This is intended for local clusters (Docker Desktop / minikube / kind), not remote managed clusters without additional setup. + **Recommended:** Use `scripts/run-examples.sh` which handles all setup automatically. ## Quick Symptom Guide @@ -289,6 +294,7 @@ Run a minimal baseline test (e.g., 2 validators, consensus liveness only). If it 1. Build bundle: `scripts/build-bundle.sh --platform linux` 2. Set bundle path: `export NOMOS_BINARIES_TAR=.tmp/nomos-binaries-linux-v0.3.1.tar.gz` 3. Build image: `testing-framework/assets/stack/scripts/build_test_image.sh` + 4. **kind/minikube:** load the image into the cluster nodes (e.g. `kind load docker-image logos-blockchain-testing:local`, or `minikube image load ...`), or push to a registry and set `NOMOS_TESTNET_IMAGE` accordingly. ### "Failed to load KZG parameters" or "Circuit file not found"