feat(sim): testnet pipeline with seed bootstrap, slim mode, polish

Adds a SIM_NETWORK=testnet path to mix_lez_chat that:

- Skips local sequencer launch; reachability-checks the testnet RPC
  up front.
- Selects vendor/logos-lez-rln/testnet/ for wallet config + persistent
  state. Default flow runs run_setup against the deployed RLN program
  (is_initialized short-circuits to create_funded_user).
- Bootstraps from the submodule-shipped artifacts on first run:
  copies testnet/storage.json.seed -> testnet/storage.json and
  testnet/{supply_holding,payment_account}.txt ->
  ~/.logos-lez-rln/{supply_holding,payment_account}_<tree>.txt. Lets
  a fresh-clone dev reuse the canonical deployment.
- Adds SIM_SLIM=1 opt-in: skip run_setup entirely, use the shipped
  config_account + cached payment_account. Saves devs from building
  the lez-rln Rust toolchain (and transitively the lssa nested
  submodule).
- Raises timing floors for testnet block times (~60s vs local ~15s);
  adds KADEMLIA_HARD_CAP and SIM_PERSIST_LOCAL escape hatches.
- Polled-readiness loops replace fixed sleeps (kademlia mesh, mix
  peer counts, receiver join, delivery — each with bounded retry).
- Extracted seed_copy and wait_method_calls helpers; dropped a stale
  mix-peer kademlia gate (static mixNodes config seeds the pool via
  processBootNodes — kademlia counts are diagnostics only).

README updated with the testnet-flow + slim-mode + fresh-clone story.
This commit is contained in:
Arseniy Klempner 2026-05-17 18:59:31 -06:00
parent c6a993a209
commit 8977064671
No known key found for this signature in database
GPG Key ID: 51653F18863BD24B
2 changed files with 198 additions and 85 deletions

View File

@ -70,9 +70,11 @@ Override defaults via environment:
| `SIM_LOG_LEVEL` | `INFO` | Node log level (TRACE, DEBUG, INFO, WARN, ERROR) |
| `SIM_CHAT_RECV_PORT` | `60010` | Chat receiver TCP port |
| `SIM_CHAT_SEND_PORT` | `60011` | Chat sender TCP port |
| `SIM_KADEMLIA_MIN_WAIT` | `30` | Minimum seconds to wait for kademlia propagation |
| `SIM_RECEIVER_MIN_WAIT` | `15` | Minimum seconds to wait for receiver to join mix |
| `SIM_DELIVERY_TIMEOUT` | `120` | Max seconds to wait for message delivery |
| `SIM_KADEMLIA_MIN_WAIT` | `30` (local) / `120` (testnet) | Minimum seconds to wait for kademlia propagation |
| `SIM_RECEIVER_MIN_WAIT` | `15` (local) / `60` (testnet) | Minimum seconds to wait for receiver to join mix |
| `SIM_DELIVERY_TIMEOUT` | `120` (local) / `300` (testnet) | Max seconds to wait for message delivery |
| `SIM_NODE_STARTUP_SLEEP` | `10` (local) / `30` (testnet) | Seconds between launching each mix node |
| `SIM_NETWORK` | `local` | `local` runs against a sequencer on `127.0.0.1:3040`; `testnet` runs against `https://testnet.lez.logos.co/` |
Example — fast iteration with verbose logging:
@ -81,17 +83,52 @@ SIM_LOG_LEVEL=TRACE SIM_KADEMLIA_MIN_WAIT=10 SIM_RECEIVER_MIN_WAIT=5 \
bash simulations/mix_lez_chat/run_simulation.sh --fresh
```
## Running against the public testnet
```bash
SIM_NETWORK=testnet bash simulations/mix_lez_chat/run_simulation.sh --fresh
```
Effect of `SIM_NETWORK=testnet`:
- Phase 1 (local sequencer launch) is skipped; the script does a one-shot reachability check against `https://testnet.lez.logos.co/` and dies up front if unreachable.
- Wallet config is picked from `vendor/logos-lez-rln/testnet/` instead of `dev/`. The wallet's `storage.json` and the on-chain registration accounts persist across runs.
- `run_setup` deploys + initializes on first run, then short-circuits via `is_initialized` on every run after — see "Config account: …" in the setup output either way.
- Timing floors (`SIM_KADEMLIA_MIN_WAIT`, `SIM_RECEIVER_MIN_WAIT`, `SIM_DELIVERY_TIMEOUT`, `SIM_NODE_STARTUP_SLEEP`) and `LEZ_RLN_BLOCK_SEAL_SECS` default higher to match ~60s testnet block times.
Prerequisites:
- The gifter mix node's payment account must be funded on testnet. The first successful `SIM_NETWORK=testnet … --fresh` run populates `~/.logos-lez-rln/payment_account_<tree_id>.txt` automatically; subsequent runs reuse it.
- Expected wall-clock runtime: ~2025 minutes (first run) / ~15 minutes (subsequent runs), vs. ~3 minutes locally.
- Only one developer at a time — concurrent testnet sim runs share the gifter wallet and will collide.
### Reproducibility on a fresh clone
The canonical testnet deployment (RLN tree + minted supply) is shared across developers. On first run, the script seeds two artifacts from the submodule so `run_setup` can short-circuit to `create_funded_user`:
- `vendor/logos-lez-rln/testnet/storage.json.seed` → copied to `vendor/logos-lez-rln/testnet/storage.json` if absent. Contains only the supply holding account + its signing key.
- `vendor/logos-lez-rln/testnet/supply_holding.txt` → copied to `~/.logos-lez-rln/supply_holding_<tree_id>.txt` if absent. Contains the supply `AccountId`.
What stays shared vs. fresh:
| Artifact | Shared | Per-dev fresh |
|---|---|---|
| `TREE_ID`, sequencer URL, deployed program IDs (on-chain), gifter EIP-191 auth keys, mix node identity keys | ✓ | |
| Supply holding account + signing key (seeded from submodule) | ✓ | |
| Per-run payment account (`~/.logos-lez-rln/payment_account_<tree>.txt`) | | ✓ |
| Working-copy `testnet/storage.json` (gitignored; accumulates payment accounts) | | ✓ |
| Mix + chat RLN credentials (in `.sim_state/rln_keystore_*.json`) | | ✓ |
**Security:** the supply signing key being in the repo is acceptable only because testnet does not charge gas and the tokens are test tokens with no real value.
## `--fresh` behavior
When `--fresh` is passed:
- Kills all existing `logos_host` processes
- Cleans `/tmp/logos_*` Qt RemoteObjects sockets
- Removes `.sim_state/` directory
- Removes sequencer state (`rocksdb/`, `bedrock_signing_key`)
- Rebuilds and restarts the sequencer
- Redeploys RLN programs via `run_setup`
- On `SIM_NETWORK=local` (default): removes sequencer state (`rocksdb/`, `bedrock_signing_key`), rebuilds and restarts the sequencer, redeploys RLN programs via `run_setup`
- On `SIM_NETWORK=testnet`: leaves on-chain state and the persistent wallet under `vendor/logos-lez-rln/testnet/` intact; `run_setup` short-circuits to `create_funded_user`
Without `--fresh`, reuses existing sequencer if port 3040 is already bound.
Without `--fresh`, on `SIM_NETWORK=local` it reuses an existing sequencer if port 3040 is already bound.
## Troubleshooting

View File

@ -31,11 +31,25 @@ DELIVERY_DIR="$DELIVERY_MODULE_DIR/vendor/logos-delivery"
export RISC0_DEV_MODE=1
export TMPDIR=/tmp
export LOGOS_EVENT_STDERR=1 # Enable EVENT: stderr output for sim script event observation
export LOGOS_EVENT_STDERR=1 # Mirror EVENT: lines to stderr so the sim can grep them.
die() { echo " FATAL: $*" >&2; exit 1; }
log() { echo "[$(date '+%H:%M:%S')] $*"; }
# Poll a logoscore log until it shows >= $expected "Method call successful"
# lines, or until $timeout iterations (sleeping $sleep_sec each) have elapsed.
# Sets global $N to the last observed count so callers can branch on it.
wait_method_calls() {
local logfile="$1" expected="$2" timeout="$3" sleep_sec="${4:-1}"
local t
for t in $(seq 1 "$timeout"); do
N=$(grep -c '^Method call successful' "$logfile" 2>/dev/null || true); N=${N:-0}
[ "$N" -ge "$expected" ] && return 0
sleep "$sleep_sec"
done
return 1
}
# --- Node identity constants (4 mix nodes) ---
NODEKEYS=(
"f98e3fba96c32e8d1967d460f1b79457380e1a895f7971cecc8528abe733781a"
@ -72,9 +86,28 @@ TEST_MESSAGE_PREFIX="chatmixtest"
LOG_LEVEL=${SIM_LOG_LEVEL:-INFO}
CHAT_RECV_PORT=${SIM_CHAT_RECV_PORT:-60010}
CHAT_SEND_PORT=${SIM_CHAT_SEND_PORT:-60011}
KADEMLIA_MIN_WAIT=${SIM_KADEMLIA_MIN_WAIT:-30}
RECEIVER_MIN_WAIT=${SIM_RECEIVER_MIN_WAIT:-15}
DELIVERY_TIMEOUT=${SIM_DELIVERY_TIMEOUT:-120}
SIM_NETWORK=${SIM_NETWORK:-local}
case "$SIM_NETWORK" in
local|testnet) ;;
*) die "SIM_NETWORK must be 'local' or 'testnet', got: $SIM_NETWORK";;
esac
# Timing floors: local sequencer ~15s blocks vs testnet ~60s + more variance.
if [ "$SIM_NETWORK" = testnet ]; then
KADEMLIA_MIN_WAIT=${SIM_KADEMLIA_MIN_WAIT:-120}
KADEMLIA_HARD_CAP=${SIM_KADEMLIA_HARD_CAP:-600}
RECEIVER_MIN_WAIT=${SIM_RECEIVER_MIN_WAIT:-60}
DELIVERY_TIMEOUT=${SIM_DELIVERY_TIMEOUT:-300}
NODE_STARTUP_SLEEP=${SIM_NODE_STARTUP_SLEEP:-30}
export LEZ_RLN_BLOCK_SEAL_SECS="${LEZ_RLN_BLOCK_SEAL_SECS:-90}"
else
KADEMLIA_MIN_WAIT=${SIM_KADEMLIA_MIN_WAIT:-30}
KADEMLIA_HARD_CAP=${SIM_KADEMLIA_HARD_CAP:-180}
RECEIVER_MIN_WAIT=${SIM_RECEIVER_MIN_WAIT:-15}
DELIVERY_TIMEOUT=${SIM_DELIVERY_TIMEOUT:-120}
NODE_STARTUP_SLEEP=${SIM_NODE_STARTUP_SLEEP:-10}
fi
TESTNET_RPC_URL="https://testnet.lez.logos.co/"
case "$(uname -s)-$(uname -m)" in
Darwin-arm64) PLATFORM="darwin-arm64-dev"; EXT="dylib";;
@ -111,32 +144,43 @@ cleanup() {
trap cleanup EXIT
echo "=== Mix + LEZ RLN Chat Simulation ($NUM_NODES nodes) ==="
echo " Network: $SIM_NETWORK"
[ "$SIM_NETWORK" = testnet ] && echo " Testnet RPC: $TESTNET_RPC_URL"
echo " LEZ repo: $LEZ_RLN_DIR"
echo " Chat module: $CHAT_MODULE_DIR"
echo " Logos-chat: $LOGOS_CHAT_DIR"
echo ""
pkill -f 'logos_host' 2>/dev/null || true; sleep 1
# Clean stale QtRO LocalServer sockets from prior runs (can confuse capability_module lookups)
# Stale QtRO LocalServer sockets confuse capability_module lookups.
rm -f /tmp/logos_* 2>/dev/null || true
# ---------- Phase 1: Sequencer ----------
echo "[1/6] Sequencer..."
if nc -z 127.0.0.1 3040 2>/dev/null && [ "$FRESH" -eq 0 ]; then
if [ "$SIM_NETWORK" = testnet ]; then
# Fail fast if testnet RPC is unreachable before 10+ min of setup work.
BLOCK_RESP=$(curl -sS -m 10 -X POST -H 'Content-Type: application/json' \
--data '{"jsonrpc":"2.0","method":"getLastBlockId","params":[],"id":1}' \
"$TESTNET_RPC_URL" 2>&1)
case "$BLOCK_RESP" in
*'"result"'*) log " Testnet reachable: $(echo "$BLOCK_RESP" | grep -oE '"result":[0-9]+' | head -1)";;
*) die "Testnet unreachable at $TESTNET_RPC_URL: $BLOCK_RESP";;
esac
elif nc -z 127.0.0.1 3040 2>/dev/null && [ "$FRESH" -eq 0 ]; then
SEQUENCER_PID=$(lsof -ti tcp:3040 2>/dev/null || true)
echo " Already running (PID $SEQUENCER_PID)"
else
[ "$(nc -z 127.0.0.1 3040 2>/dev/null; echo $?)" = "0" ] && kill "$(lsof -ti tcp:3040 2>/dev/null)" 2>/dev/null || true; sleep 1
rm -rf "$LEZ_RLN_DIR/lssa/rocksdb" "$LEZ_RLN_DIR/lssa/sequencer/service/bedrock_signing_key"
# Skip sequencer build if binary already exists (pre-built in Docker image).
# Also skip lssa auto-sync (requires full git history, fails on shallow clones).
# Pre-built binary path skips both the cargo build and the lssa auto-sync
# (auto-sync needs full git history; shallow Docker clones break it).
if [ -x "$LEZ_RLN_DIR/lssa/target/debug/sequencer_service" ]; then
log " Using pre-built sequencer"
SEQ_BIN="./target/debug/sequencer_service"; SEQ_CFG="sequencer/service/configs/debug/sequencer_config.json"
else
# Sequencer guest binaries must match the lssa rev that lez-rln's host-side
# client pins. Mismatch → DeserializeUnexpectedEnd (wire format divergence).
# lssa rev must match what lez-rln's host-side client pins, else
# DeserializeUnexpectedEnd from wire-format divergence.
LSSA_REV=$(grep -oE '(rev|tag)\s*=\s*"[^"]+"' "$LEZ_RLN_DIR/lez-rln/Cargo.toml" | head -1 | sed 's/.*"\([^"]*\)"/\1/')
[ -z "$LSSA_REV" ] && die "Could not extract lssa rev from lez-rln/Cargo.toml"
if ! git -C "$LEZ_RLN_DIR/lssa" merge-base --is-ancestor "$LSSA_REV" HEAD 2>/dev/null; then
@ -160,27 +204,78 @@ fi
# ---------- Phase 2: Deploy programs ----------
echo "[2/6] Deploying programs..."
export NSSA_WALLET_HOME_DIR="$LEZ_RLN_DIR/dev"
# `local` -> dev/ (re-init each run); `testnet` -> testnet/ (persistent
# wallet + on-chain state). wallet_config.json under each picks the sequencer.
WALLET_HOME_SUBDIR=$([ "$SIM_NETWORK" = testnet ] && echo testnet || echo dev)
export NSSA_WALLET_HOME_DIR="$LEZ_RLN_DIR/$WALLET_HOME_SUBDIR"
export WALLET_CONFIG="$NSSA_WALLET_HOME_DIR/wallet_config.json"
export WALLET_STORAGE="$NSSA_WALLET_HOME_DIR/storage.json"
TREE_ID_HEX="000102030405060708090a0b0c0d0e0f10111213141516170000000000000000"
TREE_ID_HEX="000102030405060708090a0b0c0d0e0f10111213141516171a05100200000000"
GIFTER_ACCOUNT_FILE="$HOME/.logos-lez-rln/payment_account_${TREE_ID_HEX}.txt"
rm -f "$WALLET_CONFIG" "$WALLET_STORAGE"
# Use pre-built binary if available, otherwise cargo run
if [ -x "$LEZ_RLN_DIR/lez-rln/target/debug/run_setup" ]; then
SETUP_OUTPUT=$(cd "$LEZ_RLN_DIR/lez-rln" && ./target/debug/run_setup 2>&1) || die "run_setup failed"
else
SETUP_OUTPUT=$(cd "$LEZ_RLN_DIR/lez-rln" && cargo run --bin run_setup 2>&1) || die "run_setup failed"
# Local: clean wallet each run so run_setup re-deploys.
# Testnet: persist wallet + on-chain state; run_setup short-circuits via
# is_initialized() into create_funded_user.
if [ "$SIM_NETWORK" = local ] && [ "${SIM_PERSIST_LOCAL:-0}" != "1" ]; then
rm -f "$WALLET_CONFIG" "$WALLET_STORAGE"
fi
echo "$SETUP_OUTPUT" | tail -4
CONFIG_ACCOUNT=$(echo "$SETUP_OUTPUT" | grep -oE 'Config account:\s+\S+' | awk '{print $NF}' || true)
[ -z "$CONFIG_ACCOUNT" ] && die "Failed to parse config account"
GIFTER_ACCOUNT=$(cat "$GIFTER_ACCOUNT_FILE" 2>/dev/null || true)
[ -z "$GIFTER_ACCOUNT" ] && die "Gifter account not found at $GIFTER_ACCOUNT_FILE"
# EIP-191 auth fixtures: per-process secp256k1 keys and the allowlist installed
# on the gifter (mix node 0). Keys are committed test fixtures — not for prod.
# Testnet bootstrap: seed wallet + supply-holding sidecar from the
# submodule-shipped artifacts on first run. After that, create_funded_user
# draws fresh per-dev payment accounts from the shared supply.
if [ "$SIM_NETWORK" = testnet ]; then
# Copy shipped seed -> runtime location iff runtime is missing.
seed_copy() {
local label="$1" src="$2" dst="$3"
[ -f "$dst" ] && return 0
[ -f "$src" ] || return 0
log " Seeding $label -> $dst"
mkdir -p "$(dirname "$dst")"
cp "$src" "$dst"
}
seed_copy "testnet wallet" \
"$LEZ_RLN_DIR/testnet/storage.json.seed" "$WALLET_STORAGE"
seed_copy "supply holding sidecar" \
"$LEZ_RLN_DIR/testnet/supply_holding.txt" \
"$HOME/.logos-lez-rln/supply_holding_${TREE_ID_HEX}.txt"
seed_copy "payment account sidecar" \
"$LEZ_RLN_DIR/testnet/payment_account.txt" \
"$HOME/.logos-lez-rln/payment_account_${TREE_ID_HEX}.txt"
fi
# Slim mode (SIM_SLIM=1, testnet only): skip run_setup when the shipped
# config_account + cached payment_account are both present — lets fresh
# clones avoid building lez-rln/run_setup. The shared payment_account is
# signed in storage.json.seed and has enough RLNTOK for ~1M Register txs.
# Experimental; the default run_setup path is better-tested.
CONFIG_ACCOUNT_SEED="$LEZ_RLN_DIR/testnet/config_account.txt"
SLIM=0
if [ "$SIM_NETWORK" = testnet ] && [ "${SIM_SLIM:-0}" = "1" ] \
&& [ -f "$CONFIG_ACCOUNT_SEED" ] && [ -f "$GIFTER_ACCOUNT_FILE" ]; then
SLIM=1
fi
if [ "$SLIM" = "1" ]; then
log " Slim mode: skipping run_setup (using shipped config_account + cached payment_account)"
CONFIG_ACCOUNT=$(tr -d '\n\r' < "$CONFIG_ACCOUNT_SEED")
GIFTER_ACCOUNT=$(cat "$GIFTER_ACCOUNT_FILE")
else
if [ -x "$LEZ_RLN_DIR/lez-rln/target/debug/run_setup" ]; then
SETUP_OUTPUT=$(cd "$LEZ_RLN_DIR/lez-rln" && ./target/debug/run_setup 2>&1) || die "run_setup failed"
else
SETUP_OUTPUT=$(cd "$LEZ_RLN_DIR/lez-rln" && cargo run --bin run_setup 2>&1) || die "run_setup failed"
fi
echo "$SETUP_OUTPUT" | tail -4
# Both deploy + already-initialized branches print "Config account:".
CONFIG_ACCOUNT=$(echo "$SETUP_OUTPUT" | grep -oE 'Config account:\s+\S+' | awk '{print $NF}' || true)
[ -z "$CONFIG_ACCOUNT" ] && die "Failed to parse config account"
GIFTER_ACCOUNT=$(cat "$GIFTER_ACCOUNT_FILE" 2>/dev/null || true)
[ -z "$GIFTER_ACCOUNT" ] && die "Gifter account not found at $GIFTER_ACCOUNT_FILE"
fi
echo " CONFIG_ACCOUNT=$CONFIG_ACCOUNT"
echo " GIFTER_ACCOUNT=$GIFTER_ACCOUNT"
# EIP-191 auth fixtures: secp256k1 keys + gifter (mix node 0) allowlist.
# Committed test fixtures only — do NOT reuse in prod.
GIFTER_AUTH_DIR="$SCRIPT_DIR/fixtures/gifter_auth"
# shellcheck disable=SC1091
source "$GIFTER_AUTH_DIR/keys.env"
@ -202,8 +297,8 @@ else
DELIVERY_PLUGIN="$DELIVERY_MODULE_DIR/result/lib/delivery_module_plugin.$EXT"
fi
# Chat module plugin (for sender/receiver)
# Prefer locally-built liblogoschat over nix result (uses vendored Nim toolchain)
# Chat module plugin (sender/receiver). Prefer locally-built liblogoschat
# (uses vendored Nim toolchain) over the nix result.
CHAT_MODULE_RESULT="$CHAT_MODULE_DIR/result"
if [ -f "$LOGOS_CHAT_DIR/build/liblogoschat.$EXT" ]; then
CHAT_LIB="$LOGOS_CHAT_DIR/build/liblogoschat.$EXT"
@ -229,9 +324,8 @@ LOAD_ORDER="liblogos_execution_zone_wallet_module,liblogos_rln_module,delivery_m
WALLET_CALL="liblogos_execution_zone_wallet_module.open($WALLET_CONFIG,$WALLET_STORAGE)"
BOOTSTRAP_PEER="/ip4/127.0.0.1/tcp/$BASE_TCP_PORT/p2p/${PEER_IDS[0]}"
# NOTE: Off-chain credential generation (setup_credentials) and pre-registration
# (register_commitments) are not used. All nodes and chat clients register their
# RLN memberships at runtime via the gifter protocol on node 0.
# All RLN memberships are issued at runtime by the gifter on node 0 — no
# off-chain setup_credentials / pre-registration step.
for i in $(seq 0 $((NUM_NODES - 1))); do
TCP_PORT=$((BASE_TCP_PORT + i)); DISC_PORT=$((BASE_DISC_PORT + i))
@ -250,7 +344,7 @@ for i in $(seq 0 $((NUM_NODES - 1))); do
if [ "$i" -eq 0 ]; then
GIFTER_FIELDS="\"mixGifterService\": true, \"mixGifterWalletAccount\": \"$GIFTER_ACCOUNT\", \"mixGifterAllowlist\": \"$GIFTER_ALLOWLIST\","
else
# KEY_MIX1..3 align with non-gifter nodes 1..3.
# KEY_MIX{1..3} align with non-gifter nodes 1..3.
AUTH_KEY_VAR="KEY_MIX$i"
GIFTER_FIELDS="\"mixGifterNode\": \"$BOOTSTRAP_PEER\", \"mixGifterWalletAccount\": \"$GIFTER_ACCOUNT\", \"mixGifterAuthKey\": \"${!AUTH_KEY_VAR}\","
fi
@ -284,17 +378,14 @@ for i in $(seq 0 $((NUM_NODES - 1))); do
EOF
MDIR=$(mktemp -d); MODULES_DIRS+=("$MDIR")
# Stage wallet module
mkdir -p "$MDIR/liblogos_execution_zone_wallet_module"
cp -L "$WALLET_MODULE/liblogos_execution_zone_wallet_module.$EXT" "$MDIR/liblogos_execution_zone_wallet_module/"
[ -f "$WALLET_MODULE/libwallet_ffi.$EXT" ] && cp -L "$WALLET_MODULE/libwallet_ffi.$EXT" "$MDIR/liblogos_execution_zone_wallet_module/"
echo "{\"name\":\"liblogos_execution_zone_wallet_module\",\"version\":\"1.0.0\",\"type\":\"core\",\"main\":{\"$PLATFORM\":\"liblogos_execution_zone_wallet_module.$EXT\"},\"dependencies\":[],\"capabilities\":[]}" > "$MDIR/liblogos_execution_zone_wallet_module/manifest.json"
# Stage RLN module
mkdir -p "$MDIR/liblogos_rln_module"
cp -L "$RLN_MODULE/liblogos_rln_module.$EXT" "$MDIR/liblogos_rln_module/"
cp -L "$RLN_MODULE/liblez_rln_ffi.$EXT" "$MDIR/liblogos_rln_module/" 2>/dev/null || true
echo "{\"name\":\"liblogos_rln_module\",\"version\":\"1.0.0\",\"type\":\"core\",\"main\":{\"$PLATFORM\":\"liblogos_rln_module.$EXT\"},\"dependencies\":[\"liblogos_execution_zone_wallet_module\"],\"capabilities\":[]}" > "$MDIR/liblogos_rln_module/manifest.json"
# Stage delivery module
mkdir -p "$MDIR/delivery_module"
cp -L "$DELIVERY_PLUGIN" "$MDIR/delivery_module/"
if [ -f "$DELIVERY_DIR/build/liblogosdelivery.$EXT" ]; then
@ -315,63 +406,58 @@ EOF
</dev/null >"$LOG_FILE" 2>&1) &
EXPECTED_CALLS=5
NODE_PID=$!; INSTANCE_PIDS+=($NODE_PID)
WAIT_TIMEOUT=90
for t in $(seq 1 $WAIT_TIMEOUT); do
N=$(grep -c '^Method call successful' "$LOG_FILE" 2>/dev/null || true); N=${N:-0}
[ "$N" -ge "$EXPECTED_CALLS" ] && break; sleep 1
done
wait_method_calls "$LOG_FILE" "$EXPECTED_CALLS" 90 1 || true
if [ "${N:-0}" -ge "$EXPECTED_CALLS" ]; then
log " Node $i ready ($N/$EXPECTED_CALLS calls) PID: $NODE_PID"
else
echo " WARNING: Node $i: $N/$EXPECTED_CALLS calls"
fi
sleep 10
sleep "$NODE_STARTUP_SLEEP"
done
echo ""
# ---------- Phase 5: Chat module sender/receiver ----------
echo "[5/6] Starting chat module instances..."
# Wait for all nodes to be fully ready (gifter registrations finalize during startup)
# Wait for all nodes ready (gifter registrations finalize during startup).
for i in $(seq 0 $((NUM_NODES - 1))); do
LOG_FILE="$STATE_DIR/node${i}.log"
EC=5
for t in $(seq 1 120); do
N=$(grep -c '^Method call successful' "$LOG_FILE" 2>/dev/null || true); N=${N:-0}
[ "$N" -ge "$EC" ] && break; sleep 2
done
wait_method_calls "$STATE_DIR/node${i}.log" 5 120 2 || true
done
# Wait for mix mesh + RLN root convergence. Requires: 3 gifter registrations on
# node0, LEZ root polling events across all nodes, AND min 30s floor for mix
# protocol handshakes to complete (no single log line signals this cleanly).
# Wait for RLN root convergence. Mix pool is seeded from each node's
# mixNodes config (processBootNodes), so kademlia isn't required for routing —
# the kad-peer count is logged only for diagnostics.
echo " Waiting for kademlia propagation + RLN convergence..."
KADEMLIA_T0=$SECONDS
while true; do
ELAPSED=$((SECONDS - KADEMLIA_T0))
GR=$(sed 's/\x1b\[[0-9;]*m//g' "$STATE_DIR/node0.log" 2>/dev/null | grep -c "RLN gifter registration succeeded" || true); GR=${GR:-0}
LR=0
MIX_PEERS_PER_NODE=""
for i in $(seq 0 $((NUM_NODES - 1))); do
L=$(sed 's/\x1b\[[0-9;]*m//g' "$STATE_DIR/node${i}.log" 2>/dev/null | grep -c "Polled valid roots\|Fetched roots from\|valid_roots\|OnchainLEZGroupManager initialized\|Wired LEZ callbacks" || true)
LOG="$STATE_DIR/node${i}.log"
L=$(sed 's/\x1b\[[0-9;]*m//g' "$LOG" 2>/dev/null | grep -c "Polled valid roots\|Fetched roots from\|valid_roots\|OnchainLEZGroupManager initialized\|Wired LEZ callbacks" || true)
LR=$((LR + L))
MP=$(sed 's/\x1b\[[0-9;]*m//g' "$LOG" 2>/dev/null | grep -c "mix peer added via kademlia lookup" || true); MP=${MP:-0}
MIX_PEERS_PER_NODE="${MIX_PEERS_PER_NODE}n${i}=${MP} "
done
if [ "$ELAPSED" -ge "$KADEMLIA_MIN_WAIT" ] && [ "$GR" -ge 3 ] && [ "$LR" -ge 40 ]; then break; fi
[ "$ELAPSED" -ge 120 ] && break
[ "$ELAPSED" -ge "$KADEMLIA_HARD_CAP" ] && break
sleep 1
done
log " Kademlia ready after $((SECONDS - KADEMLIA_T0))s ($GR gifter regs, $LR LEZ root events)"
log " Kademlia ready after $((SECONDS - KADEMLIA_T0))s ($GR gifter regs, $LR LEZ root events, kad mix peers: $MIX_PEERS_PER_NODE)"
RECEIVER_LOG="$STATE_DIR/chat_receiver.log"
SENDER_LOG="$STATE_DIR/chat_sender.log"
# Build mix node list for chat config (multiaddr:mixPubKey format)
# Build mix node list (multiaddr:mixPubKey) for chat config.
MIXNODE_LIST=""
for j in $(seq 0 $((NUM_NODES - 1))); do
[ -n "$MIXNODE_LIST" ] && MIXNODE_LIST="$MIXNODE_LIST,"
MIXNODE_LIST="$MIXNODE_LIST\"/ip4/127.0.0.1/tcp/$((BASE_TCP_PORT + j))/p2p/${PEER_IDS[$j]}:${MIX_PUBKEYS[$j]}\""
done
# Helper: stage chat_module for a logoscore instance
# Stage wallet + RLN + chat modules for a logoscore instance.
stage_chat_module() {
local MDIR=$1
mkdir -p "$MDIR/liblogos_execution_zone_wallet_module"
@ -397,7 +483,7 @@ RECV_MDIR=$(mktemp -d); MODULES_DIRS+=("$RECV_MDIR")
stage_chat_module "$RECV_MDIR"
RECV_CONFIG="$STATE_DIR/chat_receiver_config.json"
# Build static peer list (ENR or multiaddr) for chat nodes to join relay mesh
# Static peers so chat nodes join the relay mesh.
CHAT_STATIC_PEERS=""
for j in $(seq 0 $((NUM_NODES - 1))); do
[ -n "$CHAT_STATIC_PEERS" ] && CHAT_STATIC_PEERS="$CHAT_STATIC_PEERS,"
@ -433,16 +519,11 @@ log " Starting receiver..."
RECEIVER_PID=$!; INSTANCE_PIDS+=($RECEIVER_PID)
log " Receiver PID: $RECEIVER_PID"
# Wait for receiver to complete all method calls (6 calls now including createIntroBundle)
RECV_EXPECTED=6
for t in $(seq 1 180); do
N=$(grep -c '^Method call successful' "$RECEIVER_LOG" 2>/dev/null || true); N=${N:-0}
[ "$N" -ge "$RECV_EXPECTED" ] && break; sleep 2
done
N=$(grep -c '^Method call successful' "$RECEIVER_LOG" 2>/dev/null || true); N=${N:-0}
wait_method_calls "$RECEIVER_LOG" "$RECV_EXPECTED" 180 2 || true
log " Receiver method calls: $N/$RECV_EXPECTED"
# Extract intro bundle from receiver log (event callback delivers it as chatCreateIntroBundleResult)
# Extract intro bundle (emitted as chatCreateIntroBundleResult).
INTRO_BUNDLE=""
for t in $(seq 1 30); do
INTRO_BUNDLE=$(grep -oE 'logos_chatintro_[A-Za-z0-9_-]+' "$RECEIVER_LOG" 2>/dev/null | head -1 || true)
@ -454,8 +535,8 @@ else
log " WARNING: Could not extract intro bundle from receiver log"
fi
# Wait for receiver's async startChat to finish (Waku client started) AND give
# filter subscription a 15s floor to propagate through the relay mesh.
# Wait for async startChat (Waku client started) + a floor for the filter
# subscription to propagate through the relay mesh.
echo " Waiting for receiver to join mix network..."
JOIN_T0=$SECONDS
while true; do
@ -488,13 +569,13 @@ cat > "$SEND_CONFIG" <<EOF
}
EOF
# Build sender -c calls: init, start, setRlnConfig, then if we have the intro bundle,
# create a private conversation with the test message
# Sender -c calls: init/start/setRlnConfig, then newPrivateConversation if
# we have the receiver's intro bundle.
SENDER_CALLS="-c \"$WALLET_CALL\""
SENDER_CALLS="$SENDER_CALLS -c \"chat_module.initChat(@$SEND_CONFIG)\""
SENDER_CALLS="$SENDER_CALLS -c \"chat_module.setEventCallback()\""
SENDER_CALLS="$SENDER_CALLS -c \"chat_module.startChat()\""
# RLN leaf indices: 0-3 = mix nodes, 4 = gifter-reserved, 5 = receiver, 6 = sender
# RLN leaf layout: 0-3 mix nodes, 4 gifter-reserved, 5 receiver, 6 sender.
SENDER_CALLS="$SENDER_CALLS -c \"chat_module.setRlnConfig($CONFIG_ACCOUNT,6)\""
if [ -n "$INTRO_BUNDLE" ]; then
SENDER_CALLS="$SENDER_CALLS -c \"chat_module.newPrivateConversation($INTRO_BUNDLE,$TEST_MSG_HEX)\""
@ -507,18 +588,13 @@ eval "(cd \"$STATE_DIR\" && TMPDIR=/tmp \"$LOGOSCORE\" -m \"$SEND_MDIR\" -l \"$C
SENDER_PID=$!; INSTANCE_PIDS+=($SENDER_PID)
log " Sender PID: $SENDER_PID"
# Wait for sender to complete method calls
SEND_EXPECTED=6
[ -n "$INTRO_BUNDLE" ] && SEND_EXPECTED=7
for t in $(seq 1 180); do
N=$(grep -c '^Method call successful' "$SENDER_LOG" 2>/dev/null || true); N=${N:-0}
[ "$N" -ge "$SEND_EXPECTED" ] && break; sleep 2
done
wait_method_calls "$SENDER_LOG" "$SEND_EXPECTED" 180 2 || true
# On slower systems (Docker/ARM), the sender's async gifter registration may not
# complete before newPrivateConversation runs. Wait for gifter + RLN readiness
# before checking for message delivery. If newPrivateConversation initially failed,
# the Nim async code will retry once credentials are valid.
# On slower systems (Docker/ARM) the sender's async gifter registration may
# trail newPrivateConversation. Wait for RLN readiness; the Nim async code
# retries newPrivateConversation once credentials land.
echo " Waiting for sender RLN readiness..."
SENDER_RLN_T0=$SECONDS
for t in $(seq 1 60); do
@ -530,7 +606,7 @@ log " Sender RLN ready after $((SECONDS - SENDER_RLN_T0))s"
N=$(grep -c '^Method call successful' "$SENDER_LOG" 2>/dev/null || true); N=${N:-0}
log " Sender method calls: $N/$SEND_EXPECTED"
# Poll receiver log for incoming message(s) instead of waiting a fixed 120s.
# Poll receiver log for delivery.
echo " Waiting for message delivery via mix..."
DELIVERY_T0=$SECONDS
for t in $(seq 1 $DELIVERY_TIMEOUT); do