mirror of
https://github.com/vacp2p/dst-libp2p-test-node.git
synced 2026-08-27 12:51:12 +00:00
* shadow: add k8s-based runner setup
Adds a containerized Shadow runner so the existing single-machine prototype
(shadow/{topogen.py,run.sh,*.awk}) can execute on a lab worker without SSH.
- shadow/Dockerfile: multi-stage build — nim test-node `main` (mirrors
nim-test-node/Dockerfile_amd64) + Ubuntu 22.04 runner with Shadow v3.3.0
built from source (rustup + cmake/glib deps).
- shadow/k8s-job.yaml: Job pinned to node-05 in zerotesting-shadow, with
seccomp Unconfined + SYS_PTRACE for Shadow's syscall interposition.
- shadow/kaniko-build-job.yaml: in-cluster image build via kaniko on node-04
(avoids 30+ min QEMU emulation on arm64 macs); pushes to
radiken/dst-shadow-nim.
- .dockerignore at repo root: keeps the buildx context lean for the new
root-context Shadow build; per-experiment builds under nim-test-node/ are
unaffected.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* shadow: fix kaniko git context branch case
GitHub branch refs are case-sensitive server-side. The branch landed as
Alan/shadow-k8s (capital A, matching team convention), but the manifest
referenced lowercase.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* shadow: add python3-yaml to the runner image
topogen.py imports yaml; without python3-yaml the sim aborts at
ModuleNotFoundError before generating shadow.yaml.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* shadow: build nim test node dynamically for Shadow runner
The k8s test-node build links the nim main fully static (-static -mmusl,
-static-libgcc, -static-libstdc++) for container portability. Shadow won't
accept a static binary because its syscall interposer is LD_PRELOAD-based,
which only hooks dynamically linked ELFs. Smoke test confirmed:
Failed to verify plugin path '/tmp/t2/main'
Caused by: not a dynamically linked ELF
Drop the static linker flags in the Shadow stage of the Dockerfile so the
nim main ends up dynamically linked against the runner image's glibc /
libssl3 / libstdc++. The k8s build (nim-test-node/Dockerfile_amd64) is
unchanged.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* shadow: split runner image into base + per-experiment test-node
Until now we baked the nim test-node main binary into the Shadow runner
image. That couples the generic Shadow runtime to one specific test node
and means rebuilding the whole image (Rust + Shadow) any time the nim
source changes.
Split into two images:
- radiken/dst-shadow-base: Ubuntu 22.04 + Shadow v3.3.0 + python deps.
No nim binary. Built rarely. shadow/Dockerfile.
- radiken/dst-test-node-shadow: dynamic-linked nim main, two-stage build
with debian:bookworm-slim final stage. nim-test-node/Dockerfile_amd64_shadow.
Per-run Job specs use an init container to copy /node/main out of the
test-node image into an emptyDir, then run shadow from the base image with
the binary + shadow.yaml mounted in. Decouples the Shadow runtime from the
per-experiment binary.
Two kaniko Job manifests (shadow/kaniko-build-job.yaml and
shadow/kaniko-build-test-node-job.yaml), both pinned to node-04.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* shadow: bake python3-requests into base for the publisher host
The Shadow runner pod runs traffic_sync.py (mounted via ConfigMap) as a
publisher host inside shadow.yaml. traffic_sync.py uses the requests
library, which isn't in the upstream python3 package. Bake it in once
rather than apt-installing at every run start.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* test-node: make storeMetrics scrape interval configurable
storeMetrics (used in Shadow mode to curl /metrics into metrics_pod-N.txt)
was hardcoded to a 5-minute interval. Shadow sims run for seconds-to-minutes
of simulated time, so a single scrape fires at t~=0 (before the mesh forms)
and the run ends before the next one. The captured libp2p_network_bytes
counter is therefore ~0 and useless for bandwidth comparison.
Add METRICS_INTERVAL_S env var (default 300 to preserve k8s behavior).
Shadow runs set it short (e.g. 15s) so the last scrape captures the
post-traffic cumulative byte counts.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* shadow: drop superseded k8s-job.yaml, fix stale image comment
k8s-job.yaml was the old single-image by-hand runner manifest pointing at the
retired radiken/dst-shadow-nim image; the runner Job is now generated by
10ksim's builders. Also fix a kaniko comment that still named dst-shadow-nim.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix comments
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
dst-libp2p-test-node for shadow simulator
This directory contains necessary files for running the dst-libp2p-test-nod using the shadow simulator. This can either be done by compiling the main.nim file directly or by extracting the executable from a Docker image. Currently we support nim-libp2p only. Support for other implementations will be added soon. This directory includes utilities for:
- Creating a
shadow.yamlfile for describing the network topology. - Running the shadow simulation with configurable parameters.
- Listing latency and bandwidth utilization.
- Collecting Prometheus metrics from simulated nodes
Getting the Executable Test Node
You have two options for obtaining the main executable:
Option 1: Extract from Docker Image (Recommended)
Extract the pre-built executable from the Docker image without building it locally.
# Pull the Docker image
docker pull <docker_image>
# Create a temporary container from the image
docker create --name temp-container <docker_image>
# Copy the 'main' binary from the container to this directory
docker cp temp-container:/node/main ./main
# Make it executable
chmod +x ./main
# Remove the temporary container
docker rm temp-container
Option 2: Build from Source
Build the main executable directly from the source.
#Navigate to the parent directory
cd ../nim-test-node/
#Compile with required flags
nim c -d:chronicles_colors=None --threads:on -d:metrics -d:libp2p_network_protocols_metrics -d:release main
#Move the executable to the shadow directory
mv ./nim-test-node/main ../shadow/
Running Simulations
Use the run.sh script to create and run simulations with custo parameters
Example
#usage parameters
./run.sh <runs> <nodes> <Message_size> <num_fragment> <num_publishers>
<min_bandwidth> <max_bandwidth> <min_latency> <max_latency>
<anchor_stages> <packet_loss> <publisher_id> <publisher_rotation> <inter_message_delay>
#example
./run.sh 1 2000 15000 1 10 50 150 40 130 5 0.0 6 1 1000
#This will output latency/bandwidth and store prometheus metrics (and peer data) in shadow.data directory
Parameters
| Parameter | Description | Example Value |
|---|---|---|
runs |
Number of times to run a simulation | 1 |
nodes |
Total number of nodes in the network | 2000 |
message_size |
Message size in bytes | 15000 |
num_fragments |
Number of message fragments (1-10). 1 means not fragmented | 1 |
num_publishers |
Number of messages to publish | 10 |
min_bandwidth |
Minimum peer bandwidth in Mbps | 50 |
max_bandwidth |
Maximum peer bandwidth in Mbps | 150 |
min_latency |
Minimum link latency in ms | 40 |
max_latency |
Maximum link latency in ms | 130 |
anchor_stages |
Bandwidth/latency variations | 5 |
packet_loss |
Packet loss rate (0.0-1.0) | 0.0 |
publisher_id |
Starting publisher node ID | 6 |
publisher_rotation |
Rotate publishers (0=no, 1=yes) | 1 |
inter_message_delay |
Delay between messages in ms | 1000 |
Supported Implementation
✅ = Available 🚧 = In progress
| Name | Node | Latest tested version |
|---|---|---|
| nim-libp2p | ✅ | v1.12.0 |
| go-libp2p | 🚧 | - |
| rust-libp2p | 🚧 | - |