mirror of
https://github.com/vacp2p/dst-libp2p-test-node.git
synced 2026-08-27 12:51:12 +00:00
Zhuolun LiandClaude Opus 4.7
ec1d9a9b62
* shadow: add k8s-based runner setup
Adds a containerized Shadow runner so the existing single-machine prototype
(shadow/{topogen.py,run.sh,*.awk}) can execute on a lab worker without SSH.
- shadow/Dockerfile: multi-stage build — nim test-node `main` (mirrors
nim-test-node/Dockerfile_amd64) + Ubuntu 22.04 runner with Shadow v3.3.0
built from source (rustup + cmake/glib deps).
- shadow/k8s-job.yaml: Job pinned to node-05 in zerotesting-shadow, with
seccomp Unconfined + SYS_PTRACE for Shadow's syscall interposition.
- shadow/kaniko-build-job.yaml: in-cluster image build via kaniko on node-04
(avoids 30+ min QEMU emulation on arm64 macs); pushes to
radiken/dst-shadow-nim.
- .dockerignore at repo root: keeps the buildx context lean for the new
root-context Shadow build; per-experiment builds under nim-test-node/ are
unaffected.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* shadow: fix kaniko git context branch case
GitHub branch refs are case-sensitive server-side. The branch landed as
Alan/shadow-k8s (capital A, matching team convention), but the manifest
referenced lowercase.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* shadow: add python3-yaml to the runner image
topogen.py imports yaml; without python3-yaml the sim aborts at
ModuleNotFoundError before generating shadow.yaml.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* shadow: build nim test node dynamically for Shadow runner
The k8s test-node build links the nim main fully static (-static -mmusl,
-static-libgcc, -static-libstdc++) for container portability. Shadow won't
accept a static binary because its syscall interposer is LD_PRELOAD-based,
which only hooks dynamically linked ELFs. Smoke test confirmed:
Failed to verify plugin path '/tmp/t2/main'
Caused by: not a dynamically linked ELF
Drop the static linker flags in the Shadow stage of the Dockerfile so the
nim main ends up dynamically linked against the runner image's glibc /
libssl3 / libstdc++. The k8s build (nim-test-node/Dockerfile_amd64) is
unchanged.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* shadow: split runner image into base + per-experiment test-node
Until now we baked the nim test-node main binary into the Shadow runner
image. That couples the generic Shadow runtime to one specific test node
and means rebuilding the whole image (Rust + Shadow) any time the nim
source changes.
Split into two images:
- radiken/dst-shadow-base: Ubuntu 22.04 + Shadow v3.3.0 + python deps.
No nim binary. Built rarely. shadow/Dockerfile.
- radiken/dst-test-node-shadow: dynamic-linked nim main, two-stage build
with debian:bookworm-slim final stage. nim-test-node/Dockerfile_amd64_shadow.
Per-run Job specs use an init container to copy /node/main out of the
test-node image into an emptyDir, then run shadow from the base image with
the binary + shadow.yaml mounted in. Decouples the Shadow runtime from the
per-experiment binary.
Two kaniko Job manifests (shadow/kaniko-build-job.yaml and
shadow/kaniko-build-test-node-job.yaml), both pinned to node-04.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* shadow: bake python3-requests into base for the publisher host
The Shadow runner pod runs traffic_sync.py (mounted via ConfigMap) as a
publisher host inside shadow.yaml. traffic_sync.py uses the requests
library, which isn't in the upstream python3 package. Bake it in once
rather than apt-installing at every run start.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* test-node: make storeMetrics scrape interval configurable
storeMetrics (used in Shadow mode to curl /metrics into metrics_pod-N.txt)
was hardcoded to a 5-minute interval. Shadow sims run for seconds-to-minutes
of simulated time, so a single scrape fires at t~=0 (before the mesh forms)
and the run ends before the next one. The captured libp2p_network_bytes
counter is therefore ~0 and useless for bandwidth comparison.
Add METRICS_INTERVAL_S env var (default 300 to preserve k8s behavior).
Shadow runs set it short (e.g. 15s) so the last scrape captures the
post-traffic cumulative byte counts.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* shadow: drop superseded k8s-job.yaml, fix stale image comment
k8s-job.yaml was the old single-image by-hand runner manifest pointing at the
retired radiken/dst-shadow-nim image; the runner Job is now generated by
10ksim's builders. Also fix a kaniko comment that still named dst-shadow-nim.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix comments
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
51 lines
2.2 KiB
Docker
51 lines
2.2 KiB
Docker
# Generic Shadow runner image.
|
|
#
|
|
# Ubuntu 22.04 + Shadow built from source + python deps + nothing else.
|
|
# No nim test-node binary baked in: per-experiment dynamic test-node images
|
|
# (built from nim-test-node/Dockerfile_amd64_shadow) ship the `main` binary
|
|
# separately. A Job spec uses an init container to copy `main` from the
|
|
# per-experiment image into an emptyDir, then runs Shadow from this image.
|
|
#
|
|
# Build via kaniko on a lab worker (Apple Silicon emulation is too slow):
|
|
# kubectl apply -f shadow/kaniko-build-job.yaml
|
|
# Pushes to radiken/dst-shadow-base:latest.
|
|
|
|
FROM ubuntu:22.04
|
|
|
|
ENV DEBIAN_FRONTEND=noninteractive
|
|
|
|
# Shadow build/runtime deps (https://shadow.github.io/docs/guide/install_dependencies.html)
|
|
# + python3-networkx + python3-yaml for any in-cluster topology generation.
|
|
RUN apt-get update && apt-get install -y --no-install-recommends \
|
|
cmake findutils libclang-dev libc-dbg \
|
|
libglib2.0-0 libglib2.0-dev make netbase \
|
|
python3 python3-networkx python3-yaml python3-requests xz-utils util-linux \
|
|
gcc g++ git ca-certificates curl gawk \
|
|
&& rm -rf /var/lib/apt/lists/* \
|
|
&& apt-get clean
|
|
|
|
# Shadow v3.x has a Rust workspace, so we need cargo on PATH before building.
|
|
ENV RUSTUP_HOME=/usr/local/rustup CARGO_HOME=/usr/local/cargo
|
|
ENV PATH="${CARGO_HOME}/bin:/root/.local/bin:${PATH}"
|
|
RUN curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs \
|
|
| sh -s -- -y --default-toolchain stable --profile minimal --no-modify-path \
|
|
&& chmod -R a+w ${RUSTUP_HOME} ${CARGO_HOME} \
|
|
&& rustc --version
|
|
|
|
# Build Shadow from source at a pinned tag for reproducibility.
|
|
# Bump SHADOW_REF deliberately when picking up a new version.
|
|
ARG SHADOW_REF=v3.3.0
|
|
RUN git clone --depth 1 --branch ${SHADOW_REF} https://github.com/shadow/shadow.git /tmp/shadow \
|
|
&& cd /tmp/shadow \
|
|
&& ./setup build --jobs $(nproc) \
|
|
&& ./setup install \
|
|
&& rm -rf /tmp/shadow
|
|
|
|
# Default working directory for sims; the Job mounts the test-node binary and
|
|
# the shadow.yaml here at runtime.
|
|
WORKDIR /sim
|
|
|
|
# No ENTRYPOINT — the Job manifest sets its own command (typically
|
|
# `shadow /sim/config/shadow.yaml`) so different experiments can pass
|
|
# different invocations without rebuilding the image.
|