diff --git a/reports/tsi/README.md b/reports/tsi/README.md
index 26f115b..fb9afd5 100644
--- a/reports/tsi/README.md
+++ b/reports/tsi/README.md
@@ -4,6 +4,8 @@
This report selects and justifies the TSI parameters for Cryptarchia from a per-node network simulation. It is split into four cohesive parts; section numbers (§1–§9, A–C) are stable identifiers preserved across the set.
+> **Uncle references.** The model analysed throughout is the **countable** one: counting-only references, deduplicated by slot, drawn from first-fork blocks only, within a window derived as `w_u = W_abs/f`. An **unrestricted** baseline — any orphan in the window at any fork depth — is measured alongside it for comparison. The two are indistinguishable in the design regime `ρ < 1` and diverge only under overload. See the model note at the top of [Part 1](tsi-report-1-overview-and-recommendations.md), the mechanism in [§2.1](tsi-report-2-accuracy-and-design.md#s2-1), the comparison in [§3.2](tsi-report-2-accuracy-and-design.md#s3-2)–[§3.2a](tsi-report-2-accuracy-and-design.md#s3-2a), and the reproduction notes in [§9](tsi-report-4-reproducibility-and-appendices.md#s9).
+
## Parts
1. **[Overview and recommendations](tsi-report-1-overview-and-recommendations.md)** — the executive summary, the per-knob parameter reference (§7), and the safest selection with residual risks and the recommendation-vs-spec deltas (§8).
diff --git a/reports/tsi/report-figures/fig30_countable_vs_old.png b/reports/tsi/report-figures/fig30_countable_vs_old.png
index c8bc698..b187b75 100644
Binary files a/reports/tsi/report-figures/fig30_countable_vs_old.png and b/reports/tsi/report-figures/fig30_countable_vs_old.png differ
diff --git a/reports/tsi/report-figures/fig33_absorption_window.png b/reports/tsi/report-figures/fig33_absorption_window.png
index 45c8165..025b572 100644
Binary files a/reports/tsi/report-figures/fig33_absorption_window.png and b/reports/tsi/report-figures/fig33_absorption_window.png differ
diff --git a/reports/tsi/report-figures/fig34_fine_delay_accuracy.png b/reports/tsi/report-figures/fig34_fine_delay_accuracy.png
new file mode 100644
index 0000000..f2dffbc
Binary files /dev/null and b/reports/tsi/report-figures/fig34_fine_delay_accuracy.png differ
diff --git a/reports/tsi/report-figures/fig35_fine_delay_gap.png b/reports/tsi/report-figures/fig35_fine_delay_gap.png
new file mode 100644
index 0000000..cc879f5
Binary files /dev/null and b/reports/tsi/report-figures/fig35_fine_delay_gap.png differ
diff --git a/reports/tsi/tsi-report-1-overview-and-recommendations.md b/reports/tsi/tsi-report-1-overview-and-recommendations.md
index a38fdb6..4491dfb 100644
--- a/reports/tsi/tsi-report-1-overview-and-recommendations.md
+++ b/reports/tsi/tsi-report-1-overview-and-recommendations.md
@@ -8,6 +8,8 @@
---
+> **The uncle-reference model analysed here.** Uncle references are *counting-only* — they never affect a block's validity ([§6.7](tsi-report-3-robustness-and-incentives.md#s6-7)–[§6.8](tsi-report-3-robustness-and-incentives.md#s6-8)) — and are deduplicated by **slot**, not by block ([§8.5](#s8-5)). The reference window is derived from a **window absorption parameter**, `w_u = W_abs/f` with `W_abs = 10` expected block-intervals and bound `W_abs ≤ 0.6·k`, which is the `W = 300 = 10/f` sizing rule of [§3.4](tsi-report-2-accuracy-and-design.md#s3-4). One structural constraint shapes the results: only the **first block of a fork** — one whose parent lies on the referencing chain — is countable, which is the price of verifying every reference from chain data alone. This report calls that the **countable** model, and measures it against an **unrestricted** baseline in which any orphan in the window is referenceable at any fork depth ([§2.1](tsi-report-2-accuracy-and-design.md#s2-1), [§3.2](tsi-report-2-accuracy-and-design.md#s3-2), [§9](tsi-report-4-reproducibility-and-appendices.md#s9)). The result: **in the design regime `ρ < 1` the two are practically identical.** A dedicated 40-replicate sweep of the operating band (`δ_max` 1–5, `ρ ≈ 0.25–0.45`) puts every `U ≥ 1` cell of both models in 0.998–1.001, resolves no difference in any individual cell (widest 95 % CI ±0.15 pp), and bounds the pooled first-fork cost below **0.15 %** — rising monotonically with delay and separating from zero only at the top of the band ([§3.2a](tsi-report-2-accuracy-and-design.md#s3-2a)). Differences appear only at `ρ ≥ 1`: the restriction trims ~1.4 % at `ρ ≈ 1`, and under overload (`ρ ≈ 1.8`) recovery saturates near 0.95 where the unrestricted model reaches ~1.00. Deep forks are unrecoverable by construction, so past `ρ ≈ 1` the ceiling is set by the fork *depth* profile and **no uncle cap buys it back** — a second, independent reason for the [§8](#s8) "keep `ρ < 1`" rule. The closed-form accuracy with the measured effective utilization, `log(1−f)/log(1−f/q_u)`, predicts every countable cell to within 0.2 %.
+
## 1. Executive summary
diff --git a/reports/tsi/tsi-report-2-accuracy-and-design.md b/reports/tsi/tsi-report-2-accuracy-and-design.md
index 0409b30..577038e 100644
--- a/reports/tsi/tsi-report-2-accuracy-and-design.md
+++ b/reports/tsi/tsi-report-2-accuracy-and-design.md
@@ -48,9 +48,10 @@ D̂_next = max(1, D̂ · (1 − β·(f_p − m/T)/f_p)) β = learning rate
Occupied slots more frequent than the target push the estimate up (the lottery then gets harder); less frequent pulls it down; `m/T = f_p` is the resting point. The two versions differ **only in which slots count**:
- **Before uncle references (spec v1.0):** `m` counts only slots occupied by the *canonical* chain. Under Blend delay roughly a third of honest blocks are orphaned, so their slots vanish from `m`, the resting point sits far below truth (`D̂/D ≈ 0.64–0.74`, deepening with network size), and the deflated estimate makes the network *chronically over-produce* blocks at up to ~2× the target rate ([§3.2](#s3-2), fig1). Pre-uncle TSI is not viable under Blend.
-- **After uncle references (spec v1.1):** `m` also counts the *referenced uncles* — orphans pointed at from the canonical chain — which puts the delay-orphaned slots back into the count and lifts the resting point up toward `D`. The spec as written deduplicates uncles by *block identity* (the same orphan referenced by two blocks counts once), recovering the bulk of the orphan loss. **This report recommends one refinement ([§8.5](tsi-report-1-overview-and-recommendations.md#s8-5)):** deduplicate by *slot*, not by block — a referenced uncle that shares a slot with a canonical block, or with another counted uncle, must add nothing. With slot-deduplication the orphaned slots return one-per-slot and the resting point lands at **exactly `D`**.
+- **With uncle references, counting whole blocks (the *unrestricted* model):** `m` also counts the *referenced uncles* — orphans pointed at from the canonical chain — which puts the delay-orphaned slots back into the count and lifts the resting point up toward `D`. Any orphan in the reference window is eligible regardless of how deep in a fork it sits, and references are deduplicated by *block identity*. This report analyses it as the comparison baseline throughout, and calls it the **unrestricted** model.
+- **With uncle references, counting slots under the first-fork restriction (the *countable* model — the rules analysed here):** two changes. First, deduplication is by **slot**, not by block: a referenced uncle sharing a slot with a canonical block, or with another counted uncle, adds nothing. Second, only the **first block of a fork** — one whose parent lies on the referencing chain — is countable, which is what makes every reference verifiable from chain data alone (deeper fork blocks would need the fork branch's ledger state). References never affect a block's validity; they are *counting-only*, the soft-rule property the incentive analysis requires ([§6.7](tsi-report-3-robustness-and-incentives.md#s6-7)–[§6.8](tsi-report-3-robustness-and-incentives.md#s6-8)). The reference window is *derived* from a **window absorption parameter**, `w_u = W_abs/f` with `W_abs = 10` expected block-intervals and bound `W_abs ≤ 0.6·k` — the `W = 10/f` sizing rule of [§3.4](#s3-4). The price of chain-only verifiability is that deep-fork orphans are unrecoverable by construction; [§3.2](#s3-2) measures what that costs, and the answer is nothing in the design regime and a bounded ceiling under overload.
-So the slot-deduplicated uncle rule recovers the orphan-loss the delay caused while preserving the "one count per slot" invariant, and the recovered equilibrium is exactly 1.0 — not a ceiling above it. Counting uncle *blocks* instead — as spec v1.1 is currently written, and as an earlier version of this simulator did — double-counts same-slot co-winners (one canonical, one referenced orphan) and inflates the equilibrium by the fixed multi-winner factor `c(f) ≈ 1.017`. Every result in this report uses the corrected slot count, and [§8.5](tsi-report-1-overview-and-recommendations.md#s8-5) carries the corresponding spec recommendation; the difference is a genuine ~1.7 % accuracy bias in the deployed rule, not merely a fixed simulation bug.
+Slot deduplication is what makes the recovered equilibrium exactly 1.0 rather than a ceiling above it. Counting uncle *blocks* instead double-counts same-slot co-winners (one canonical, one referenced orphan) and inflates the equilibrium by the fixed multi-winner factor `c(f) ≈ 1.017` — a genuine accuracy bias, not a simulation artefact. Every result in this report uses the slot count; where a figure or table compares the two referencing models, the unrestricted baseline is labelled as such ([§9](tsi-report-4-reproducibility-and-appendices.md#s9)).
### 2.2 The equilibrium is bounded by 1
@@ -111,10 +112,62 @@ The U = 0 under-count **deepens with N** (more nodes → more concurrent proposa
+**What the first-fork restriction costs.** Running the delay × U grid under both referencing models on the same grid (`configs/countable-vs-old.yaml`, N = 1 000, blend, 3 hops, 5 replicates; [§9](tsi-report-4-reproducibility-and-appendices.md#s9)) separates the regimes cleanly (`fig30`). Cells are replicate means ± SEM; `t` is the two-sample statistic for the countable−unrestricted gap, and cells with `t < 2` do **not** resolve a difference at this replicate count:
+
+| `δ_max` (s) | ρ | U | countable | unrestricted | gap (t) |
+|---|---|---|---|---|---|
+| 4 | ≈ 0.4 | 1 / 2 / 4 | 0.9979 ± 0.0015 / 0.9982 ± 0.0014 / 0.9995 ± 0.0012 | 0.9981 ± 0.0006 / 0.9985 ± 0.0012 / 0.9992 ± 0.0013 | *none resolved* (0.1–0.2) |
+| 8 | ≈ 0.6 | 1 / 2 / 4 | 0.9965 ± 0.0015 / 0.9955 ± 0.0017 / 0.9963 ± 0.0015 | 0.9986 ± 0.0017 / 0.9993 ± 0.0016 / 0.9989 ± 0.0017 | *none resolved* (1.0–1.6) |
+| 16 | ≈ 1.0 | 1 | 0.9647 ± 0.0065 | 0.9610 ± 0.0043 | *not resolved* (0.5) |
+| 16 | ≈ 1.0 | 2 / 4 | 0.9862 ± 0.0010 / 0.9911 ± 0.0032 | 0.9999 ± 0.0005 / 1.0013 ± 0.0011 | −0.014 (12.1) / −0.010 (3.0) |
+| 32 | ≈ 1.8 | 1 | 0.608 ± 0.069 | 0.570 ± 0.042 | *not resolved* (0.5) |
+| 32 | ≈ 1.8 | 2 / 4 | 0.9343 ± 0.0015 / 0.9517 ± 0.0007 | 0.9774 ± 0.0064 / 1.0020 ± 0.0010 | −0.043 (6.5) / −0.050 (41.0) |
+
+**The `U = 0` rows are a negative control.** With no uncles the two models are identical by construction — no reference is ever taken, so any measured gap is pure between-run noise (the two models draw independent RNG streams by design, so the comparison is unpaired). That control reads **+0.010 (t = 0.8) at `δ_max = 4`** but **−0.230 (t = 2.1) at `δ_max = 32`**, where single replicates range from 0.05 to 0.60. Read the table against that floor: at `δ_max = 32` a gap must clear ~0.2 to mean anything on a single-replicate basis, which is exactly why the `U = 1` cells there are reported as unresolved while the tightly-clustered `U = 2` and `U = 4` cells are not.
+
+So: **at the operating loads (`ρ < 1`) no difference between the models is detectable at all** — the first-fork restriction costs nothing measurable where forks are shallow, and both models sit at ≥ 0.995 for every `U ≥ 1`. ([§3.2a](#s3-2a) puts a much tighter bound on this region.) Differences appear only at `ρ ≥ 1` and only at `U ≥ 2`: at `ρ ≈ 1` the restriction trims ~1.4 %, and in overload the countable recovery **saturates** — raising `U` 2 → 4 buys only 0.934 → 0.952 while the unrestricted model reaches ~1.00. The ceiling is the measured recovery rate `r` (0.92 at `δ_max = 32`, `U = 4`), set by the fork-*depth* profile rather than by capacity, so no uncle cap buys it back. The closed form `D̂/D = log(1−f)/log(1−f/q_u)` with the measured `q_u = q + (1−q)·r` predicts every countable cell to within 0.2 % (`fig31`).
+
+`fig32` shows `r` and the residual `1−r` per delay, rising from 0.3 % to 7.8 % of the waste as `δ_max` goes 4 → 32 at `U = 4`. That residual is the *combined* first-fork and capacity loss — the simulator cannot separate the two, because the restriction acts at selection (a deep orphan is simply never referenced) rather than at counting. At `U = 4`, where capacity is not binding for `ρ ≤ 1.8`, most of it is attributable to fork depth; the split is not measured directly.
+
+
+
+
+
+
+
**Bootstrap is self-limiting — but only with uncles** (`fig1`; full scale, k = 2160, Blend, N = 1 000 and 5 000, genesis guesses 0.01×–2× the true stake; 0.1×–2× at N = 5 000). With uncles (U = 2, solid lines) the cold start is a non-event: whatever the guess, block production snaps back to the target `f` within ~2–2.5 epochs and the estimate lands on 1.0 by epoch 2 — identically at both network sizes. Without uncles (U = 0, dashed) the system also converges from any guess, **but to the wrong place**: the estimate settles below truth (mean ≈ 0.58× across guesses, ranging ≈0.48–0.85; ≈0.51× at N = 5 000) and the network then *chronically over-produces* blocks at ~1.85× the target rate (~2.0× at N = 5 000) — the [§6.2](tsi-report-3-robustness-and-incentives.md#s6-2) load feedback in the flesh (a low estimate makes the lottery easier, extra blocks orphan, the count stays low). So uncle references are load-bearing from the very first epochs: they are what makes the bootstrap end *at the truth* rather than at a permanently overheated equilibrium.
+
+### 3.2a The design band at high precision — `δ_max` 1–5
+
+**The question.** [§3.2](#s3-2) samples the mixing delay at 4/8/16/32. That resolves the overload regime, but it leaves the band the parameters are actually chosen in — the low-delay end, where every operating point sits — measured at four-fold spacing and five replicates, which is enough to say "no difference detected" and nothing more. How accurate is TSI across `δ_max` = 1–5, and how large a first-fork cost can be *excluded* there?
+
+**The finding.** With one uncle slot the estimate is exact across the whole band: every `U ≥ 1` cell under both referencing models lands in **0.998–1.001**, flat in delay. Without uncles the same band decays steeply, 0.810 → 0.640. Through `δ_max ≤ 4` the two referencing models are **indistinguishable**, and the first-fork restriction begins to cost something measurable only at the top of the band.
+
+`configs/fine-delay.yaml` spends replicates instead of range — `δ_max` ∈ {1,2,3,4,5}, `U` ∈ {0,1,2,4}, **40 replicates** per cell (8× [§3.2](#s3-2)), N = 1 000, blend, 3 hops — run under both models ([§9](tsi-report-4-reproducibility-and-appendices.md#s9)). The band spans `ρ ≈ 0.25` to `0.45`, entirely inside the design regime.
+
+| `δ_max` (s) | ρ | `U=0` countable / unrestricted | `U≥1` countable | `U≥1` unrestricted | pooled gap (t) |
+|---|---|---|---|---|---|
+| 1 | ≈ 0.25 | 0.810 / 0.806 | 0.9994 – 0.9999 | 0.9990 – 1.0010 | −0.0004 ± 0.0007 (1.1) |
+| 2 | ≈ 0.30 | 0.762 / 0.763 | 0.9996 – 1.0002 | 0.9996 – 1.0003 | +0.0000 ± 0.0008 (0.0) |
+| 3 | ≈ 0.35 | 0.716 / 0.723 | 0.9995 – 1.0002 | 0.9992 – 1.0004 | −0.0001 ± 0.0008 (0.2) |
+| 4 | ≈ 0.40 | 0.664 / 0.668 | 0.9988 – 0.9996 | 0.9995 – 0.9999 | −0.0005 ± 0.0008 (1.3) |
+| 5 | ≈ 0.45 | 0.640 / 0.623 | 0.9982 – 0.9988 | 0.9995 – 1.0003 | **−0.0014 ± 0.0007 (3.7)** |
+
+**No individual cell resolves a model difference.** Across the 15 `U ≥ 1` cells the widest 95 % CI half-width is ±0.0015, one cell clears `t = 2` (0.75 are expected to by chance), and its `t = 2.59` does not survive the Bonferroni threshold of 2.94 for 15 tests. Read cell by cell, the honest statement is that any difference is **smaller than ±0.15 pp**.
+
+**Pooled, a small delay-dependent cost appears.** The three uncle caps are independent measurements of the same underlying difference, so inverse-variance pooling across them buys ~√3 in precision. Pooled per delay, `δ_max` 1–4 stay unresolved (|t| ≤ 1.3) while **`δ_max = 5` resolves at −0.0014 ± 0.0007 (t = 3.7)** — surviving correction for the five delays tested. Over the whole band the pooled gap is −0.00048 ± 0.00033 (t = 2.8), and 11 of 15 cells are negative. So the first-fork restriction does cost something, in the direction theory predicts, and the cost is **monotone in delay and below 0.15 % everywhere in this band** — negligible against the ±0.9 % per-epoch sampling noise of [Appendix B](tsi-report-4-reproducibility-and-appendices.md#sB). (The pooling was chosen after inspecting the per-cell results; the per-delay trend, not the whole-band figure, is the defensible claim.)
+
+**The negative control passes.** The `U = 0` arms — identical models by construction — show |gap| ≤ 0.016 with max `t` = 1.26, i.e. no spurious signal, but a 95 % CI of ±0.025: **17× wider than the entire `U ≥ 1` range.** The unrecovered regime is intrinsically noisy, which is precisely why the model comparison has to be made where uncles are active.
+
+
+
+
+
+**What this settles.** One uncle slot is sufficient — not approximately, exactly — everywhere in the operating band, under either referencing model; the [§3.3](#s3-3) `U ≥ ⌈ρ⌉` rule has margin to spare at `ρ ≤ 0.45`. And the first-fork restriction, which [§3.2](#s3-2) shows costing 1.4 % at `ρ ≈ 1` and 5 % under overload, costs **less than 0.15 %** anywhere a deployment should be operating.
+
### 3.3 One uncle is not always enough — the load `ρ`
@@ -183,6 +236,12 @@ So the two levers separate cleanly: **`U` must cover the average load; `W` must
+**The derived window.** Sweeping the *derived* window `w_u = W_abs/f` directly (`configs/absorption-window.yaml`, U = 1, 5 replicates; `fig33`) confirms the `10/f` margin, with one correction to the shape. At the 8-s budget recovery runs 0.709 → 0.940 → **0.996** as `W_abs` goes 1 → 3 → 10; at the 16-s budget (`ρ ≈ 1`) `W_abs = 10` reaches 0.964; at the 32-s budget no window recovers what `U = 1` cannot drain. The window-miss probability `(1−f)^(W_abs/f)` falls below 10⁻⁴ by `W_abs = 10`, so past that point the window is no longer a loss channel — which is what makes `W_abs = 10` and the bound `W_abs ≤ 0.6·k` land on this section's `W = 10/f` rule.
+
+Two cautions on reading `fig33`. **Accuracy has not saturated at `W_abs ≈ 7`** — it is still climbing at every delay (8 s: 0.989 → 0.996; 16 s: 0.942 → 0.964 from 7 to 10), so the ≈ 7-block-interval queueing floor of the raw-slot sweep above is a floor on *usefulness*, not a point where widening stops paying. And the **32-s curve is noise, not a trend**: replicate SD reaches 0.22 there and the sampled points are non-monotonic (0.623 at `W_abs = 5`, 0.526 at 7, 0.617 at 10). Only its ceiling is meaningful, and that ceiling is set by `U = 1`, not by the window.
+
+
+
### 3.5 The joint (W, U) region — the levers are hierarchical
diff --git a/reports/tsi/tsi-report-4-reproducibility-and-appendices.md b/reports/tsi/tsi-report-4-reproducibility-and-appendices.md
index fa24591..473701b 100644
--- a/reports/tsi/tsi-report-4-reproducibility-and-appendices.md
+++ b/reports/tsi/tsi-report-4-reproducibility-and-appendices.md
@@ -45,10 +45,17 @@ Sweep studies are committed configs, run with `make ` (writes a dated `run
| soft uncle inclusion (reward share vs emergent `p_ref`) | `scripts/reward_mandate.py` | [§6.8](tsi-report-3-robustness-and-incentives.md#s6-8), [§6.9](tsi-report-3-robustness-and-incentives.md#s6-9) |
| U = 0 fluctuation series (zero delay, k ∈ {256, 1024, 2160}) | `scripts/appendix_fluct.py --run` (`runs/fluctuation_u0.parquet`) | [Appendix B](#sB) |
| CI smoke grid + analytic sanity checks | `configs/smoke.yaml`; `scripts/verify.py` (`make verify`) — validation only, no figures | — |
+| **countable vs unrestricted referencing** (accuracy over delay × U; measured `q_u`/recovery `r`) | `configs/countable-vs-old.yaml` run twice — default and with `--old`; figures + significance table by `scripts/plot_countable_vs_old.py` | [§2.1](tsi-report-2-accuracy-and-design.md#s2-1), [§3.2](tsi-report-2-accuracy-and-design.md#s3-2) |
+| **fine delay band** (δ_max 1–5 at 40 replicates; tight CI on the model gap in the design regime) | `configs/fine-delay.yaml` run twice — default and with `--old`; figure + table by `scripts/plot_fine_delay.py` | [§3.2a](tsi-report-2-accuracy-and-design.md#s3-2a) |
+| **window absorption sweep** (`W` in expected block-intervals, `w_u = W/f` derived) | `configs/absorption-window.yaml`; figure by `scripts/plot_countable_vs_old.py` | [§3.4](tsi-report-2-accuracy-and-design.md#s3-4) |
+
+**Uncle-model convention.** The simulator's default is the **countable** model — first-fork candidates only, derived window `w_u = W/f`, occupied-slot exclusion, per-reference counting rules ([§2.1](tsi-report-2-accuracy-and-design.md#s2-1)). The **unrestricted** baseline is preserved in the code and selected with `--old` on `tsi-sweep`/`tsi-verify`. Its RNG key is byte-identical to the pre-restriction key, so `--old` **bit-reproduces the earlier runs**: a `rho-boundary` cell (δ_max = 8, U = 2, k = 256, N = 1 000) re-run under `--old` matches the committed `2026-07-27_195627_rho-boundary` parquet with `max |Δ| = 0` on every epoch and every metric. Studies in the table above that predate the countable default were produced under the unrestricted model and reproduce exactly under `--old`; the comparison rows quantify where the two models differ, and in the design regime (`ρ < 1`) no difference is resolvable, so those findings carry over unchanged.
+
+Because the two models draw independent RNG streams, every countable-vs-unrestricted comparison is **unpaired**, and its resolution is set by the replicate spread rather than by the effect size. Each comparison sweep therefore includes a `U = 0` arm as a **negative control**: with no uncles the models are identical by construction, so the measured `U = 0` gap is a direct reading of the noise floor at that delay and replicate count. At `δ_max = 32` with 5 replicates that floor is ≈ 0.23 in `D̂/D` — larger than several real effects elsewhere in the grid — which is why [§3.2](tsi-report-2-accuracy-and-design.md#s3-2) reports a `t` statistic per cell and why the design regime is measured separately at 40 replicates ([§3.2a](tsi-report-2-accuracy-and-design.md#s3-2a)).
All studies were **re-run on 2026-07-23/24 with the corrected slot-counting mechanism** ([§2.1](tsi-report-2-accuracy-and-design.md#s2-1)) and the early-stop optimisation; the resilient batch is `scripts/run_all_reruns.sh` (per-step log in `runs/rerun_status.log`). Canonical run directories (latest): fullscale N=5000/10000 = `2026-07-24_094519_fullscale`; fullscale N=1000/2000 = `2026-07-23_171803_fullscale-small`; uncle-window = `2026-07-24_001456`; window-uncles = `2026-07-24_014240`; block-rate = `2026-07-24_043943`; blend-hops-delay = `2026-07-24_064052`; window-scale = `2026-07-24_085234`; latency-shape = `2026-07-24_090014_expdist`; stake-tail = `2026-07-24_090044_pareto133`; heterogeneous-start = `2026-07-24_090114_default`; N-scaling = the `nscaling-{a,b}` + `nscaling32-{a,b}` runs; adversary grids = `runs/adversary_grid/`; jitter = `runs/jitter_grid/`; bootstrap = `runs/bootstrap_fullscale/`; fluctuation = `runs/fluctuation_u0.parquet`; fork-rate = `runs/fork_rate_vs_delay.parquet`; ρ-boundary = `2026-07-27_195627_rho-boundary`.
-Figures are in `report-figures/` (`fig1`–`fig29`, plus [Appendix B](#sB)'s `figB1`–`figB2`; numbering is generation order, not order of appearance). Committed generators: `fig1` (bootstrap, k=2160) by `scripts/bootstrap_dynamics.py`; `fig2`,`fig4`,`fig5`,`fig17`–`fig22` by `scripts/regenerate_extra_figs.py` from the latest sweeps (`fig3` hops×delay×U grid by `scripts/hops_delay_grid.py`, `fig6` (block-rate `U_min` grid + ρ-collapse) rendered ad hoc from `runs/2026-07-24_043943_block-rate` with no committed generator; `fig26` deficit-vs-ρ by `scripts/rho_boundary_analysis.py`) (fullscale-derived `fig17`–`fig20` pool both sizes in that run, N = 5 000 and N = 10 000 — the generators filter on stake_dist/topology/degree/init_dest only, never on `n_nodes`); `fig8`,`fig9` by `scripts/adversary_figs.py` from `runs/adversary_grid/`; `fig10`–`fig12` by `scripts/dynamic_withhold.py`; `fig13`–`fig15` by `scripts/selfish_mining.py`/`selfish_rewards.py`/`reward_mandate.py`; `fig16` by `scripts/stake_vs_delay.py`; `fig23`–`fig24` by `scripts/nscaling_analysis.py`; `fig25` by `scripts/window_scale_analysis.py`; `fig27`–`fig28` by `scripts/reorg_depth.py` (fork rates via `--measure`; private-chain model `src/tsi_sim/reorg.py`); `fig29` by `scripts/churn.py`; `figB1`–`figB2` by `scripts/appendix_fluct.py`. `fig7` (feedback fixed-point) is an analytic overlay. Every figure type the per-node simulator generates appears in this report, and the fork-rate/reorg-depth study closes the previous reproducibility gap for the adversarial figures (`fig8`,`fig9` now have committed generators from `runs/adversary_grid/`).
+Figures are in `report-figures/` (`fig1`–`fig29`, plus [Appendix B](#sB)'s `figB1`–`figB2`; numbering is generation order, not order of appearance). Committed generators: `fig1` (bootstrap, k=2160) by `scripts/bootstrap_dynamics.py`; `fig2`,`fig4`,`fig5`,`fig17`–`fig22` by `scripts/regenerate_extra_figs.py` from the latest sweeps (`fig3` hops×delay×U grid by `scripts/hops_delay_grid.py`, `fig6` (block-rate `U_min` grid + ρ-collapse) rendered ad hoc from `runs/2026-07-24_043943_block-rate` with no committed generator; `fig26` deficit-vs-ρ by `scripts/rho_boundary_analysis.py`) (fullscale-derived `fig17`–`fig20` pool both sizes in that run, N = 5 000 and N = 10 000 — the generators filter on stake_dist/topology/degree/init_dest only, never on `n_nodes`); `fig8`,`fig9` by `scripts/adversary_figs.py` from `runs/adversary_grid/`; `fig10`–`fig12` by `scripts/dynamic_withhold.py`; `fig13`–`fig15` by `scripts/selfish_mining.py`/`selfish_rewards.py`/`reward_mandate.py`; `fig16` by `scripts/stake_vs_delay.py`; `fig23`–`fig24` by `scripts/nscaling_analysis.py`; `fig25` by `scripts/window_scale_analysis.py`; `fig27`–`fig28` by `scripts/reorg_depth.py` (fork rates via `--measure`; private-chain model `src/tsi_sim/reorg.py`); `fig29` by `scripts/churn.py`; `figB1`–`figB2` by `scripts/appendix_fluct.py`. `fig7` (feedback fixed-point) is an analytic overlay; `fig30`–`fig33` (countable-vs-unrestricted accuracy, `q_u`-prediction check, recovery rate, absorption-window sweep) by `scripts/plot_countable_vs_old.py` from the `cvo-countable`/`cvo-old`/`absorption-window` runs; `fig34`–`fig35` (design-regime accuracy and the model gap with 95 % CIs) by `scripts/plot_fine_delay.py` from the `fine-countable`/`fine-old` runs. Every figure type the per-node simulator generates appears in this report, and the fork-rate/reorg-depth study closes the previous reproducibility gap for the adversarial figures (`fig8`,`fig9` now have committed generators from `runs/adversary_grid/`).
---
diff --git a/tools/simulators/tsi/tsi-sim-pernode/README.md b/tools/simulators/tsi/tsi-sim-pernode/README.md
index 9acf057..fac1903 100644
--- a/tools/simulators/tsi/tsi-sim-pernode/README.md
+++ b/tools/simulators/tsi/tsi-sim-pernode/README.md
@@ -156,7 +156,9 @@ src/tsi_sim/ constants config rng stake lottery topology blocktree(+build_tree
theory verify plotting/{style, figures_pernode, make_figures}
configs/ smoke.yaml default.yaml fullscale.yaml
countable-vs-old.yaml absorption-window.yaml (countable-model studies)
+ fine-delay.yaml (delay 1-5 at 40 replicates: the design band, high precision)
tests/ test_{pernode,config,rng,lottery,blocktree,uncles,tsi_counting,stake,
theory,latency,theory_convergence,countable_counting,...}.py
-scripts/ plot_countable_vs_old.py (old-vs-countable comparison figures)
+scripts/ plot_countable_vs_old.py (countable-vs-unrestricted comparison figures)
+ plot_fine_delay.py (design-band accuracy + model gap with 95% CIs)
```
diff --git a/tools/simulators/tsi/tsi-sim-pernode/configs/countable-vs-old.yaml b/tools/simulators/tsi/tsi-sim-pernode/configs/countable-vs-old.yaml
index 3bdd5d6..ce4f71d 100644
--- a/tools/simulators/tsi/tsi-sim-pernode/configs/countable-vs-old.yaml
+++ b/tools/simulators/tsi/tsi-sim-pernode/configs/countable-vs-old.yaml
@@ -1,8 +1,12 @@
# Headline comparison for the countable uncle model (cryptarchia-v1-protocol.md counting
# rules) vs the old pre-redesign model: accuracy vs Blend mixing delay at U in {0,1,2,4}.
-# Run TWICE — default (countable) and with --old — same grid; the countable run also
-# yields deep_ref_share (the first-fork restriction's rejection rate) and q/q_eff for the
-# q_u = q + (1-q) r theory overlay. Latency is in SLOTS (1 slot = 1 s).
+# Run TWICE — default (countable) and with --old — same grid; the countable run yields
+# q/q_eff for the q_u = q + (1-q) r theory overlay. (deep_ref_share is a drift alarm that
+# reads 0 on every real run, NOT a measured rejection rate — see measure.py.) The U=0 rows
+# are the NEGATIVE CONTROL: with no uncles the two models are identical by construction, so
+# any U=0 gap is pure between-run RNG noise (the models draw different streams by design)
+# and bounds what a real countable-vs-old effect must exceed.
+# Latency is in SLOTS (1 slot = 1 s).
n_nodes: [1000] # network size
stake_dist: [pareto] # heavy-tailed (realistic) stake distribution
topology: [blend] # Blend mixnet (the multi-slot fork regime)
diff --git a/tools/simulators/tsi/tsi-sim-pernode/configs/fine-delay.yaml b/tools/simulators/tsi/tsi-sim-pernode/configs/fine-delay.yaml
new file mode 100644
index 0000000..e801234
--- /dev/null
+++ b/tools/simulators/tsi/tsi-sim-pernode/configs/fine-delay.yaml
@@ -0,0 +1,35 @@
+# High-precision sweep of the LOW mixing-delay region — the intended operating band.
+#
+# countable-vs-old.yaml samples delay at 4/8/16/32 with 5 replicates, which resolves the
+# overload regime but leaves the design regime under-measured: at delay <= 8 every
+# countable-vs-old gap there sits inside the replicate noise, so the report can only say
+# "no difference detected", not how large a difference is excluded. This sweep spends the
+# replicates instead of the range — delay 1..5 at 40 replicates — to put a tight CI on the
+# model difference exactly where the parameters are actually chosen.
+#
+# rho = f * D_vis with D_vis = hops*delta_max/2 + (hops+1)*ell_mean spans ~0.25 (delay 1)
+# to ~0.45 (delay 5), so the whole sweep sits well inside rho < 1 and U >= 1 should hold
+# the equilibrium at 1.0 under BOTH models. The question is the size of the residual gap.
+#
+# Run TWICE — default (countable) and with --old — same grid. U=0 is the negative control:
+# with no uncles the models are identical by construction, so the U=0 gap measures the
+# between-run RNG noise floor at this replicate count and bounds what a real effect must
+# exceed. Latency is in SLOTS (1 slot = 1 s).
+n_nodes: [1000] # network size
+stake_dist: [pareto] # heavy-tailed (realistic) stake distribution
+topology: [blend] # Blend mixnet (the multi-slot fork regime)
+degree: [6] # peering degree of the d-regular graph
+link_latency_mean: [0.5] # natural geographic transport (sub-slot)
+link_latency_dist: [geo] # real-world geographic band mixture
+blend_hops: [3] # fixed hop count; delay is the swept knob
+blend_delay_max: [1.0, 2.0, 3.0, 4.0, 5.0] # THE fine region (integer slots)
+max_uncles: [0, 1, 2, 4] # U: 0 = negative control, then the levers
+uncle_strategy: [oldest] # spec selection: oldest-first fill
+init_dest: [common] # per-node initial D_est from agreement
+replicates: 40 # 8x countable-vs-old.yaml: this is the point
+base: # per-run settings shared by every cell
+ k: 2160 # true security parameter
+ epochs: 20 # equilibrium within ~2 epochs; burn 50%
+ f: 0.03333333333333333 # slot activation coefficient (1/30)
+ genesis_d_factor: 0.5 # start near true stake (cheap epoch 0)
+ early_stop: true
diff --git a/tools/simulators/tsi/tsi-sim-pernode/scripts/bootstrap_dynamics.py b/tools/simulators/tsi/tsi-sim-pernode/scripts/bootstrap_dynamics.py
index 09455af..e35e308 100644
--- a/tools/simulators/tsi/tsi-sim-pernode/scripts/bootstrap_dynamics.py
+++ b/tools/simulators/tsi/tsi-sim-pernode/scripts/bootstrap_dynamics.py
@@ -54,7 +54,7 @@ def fig1(df: pd.DataFrame) -> None:
# Both panels are indexed by the estimate that DROVE each epoch's production: the
# start-of-epoch estimate `mean_ratio_in` (block rate depends on it, and at epoch 0 it
# IS the genesis guess, matching the legend). Plotting end-of-epoch `mean_ratio` here
- # would show the already-updated value at epoch 0 and offset the two panels by one epoch.
+ # would show the already-updated value at epoch 0 and offset the panels by one epoch.
s = (d[(d.gdf == gdf) & (d.u == u)]
.groupby("epoch").agg(rate=("n_blocks", "mean"), ratio=("mean_ratio_in", "mean")))
rate = s.rate / (10 * int(2160 / F)) # blocks per slot
diff --git a/tools/simulators/tsi/tsi-sim-pernode/scripts/plot_countable_vs_old.py b/tools/simulators/tsi/tsi-sim-pernode/scripts/plot_countable_vs_old.py
index 77b32c4..db3273a 100644
--- a/tools/simulators/tsi/tsi-sim-pernode/scripts/plot_countable_vs_old.py
+++ b/tools/simulators/tsi/tsi-sim-pernode/scripts/plot_countable_vs_old.py
@@ -12,8 +12,11 @@ and renders (into --out):
dashed = old, one Okabe-Ito hue per U (color follows U).
cvo_prediction_vs_sim predicted log(1-f)/log(1-f/q_u) from the MEASURED q_u vs the
simulated equilibrium — the q -> q_u reduction check.
- cvo_recovery_vs_delay measured recovery r = (q_eff - q)/(1 - q) and the first-fork
- rejection share (deep_ref_share) vs delay.
+ cvo_recovery_vs_delay measured recovery r = (q_eff - q)/(1 - q) and the non-recovered
+ waste share 1-r vs delay. (NOT deep_ref_share: under joint
+ countable selection+counting that is identically 0 — the
+ first-fork restriction acts at selection, so it shows up
+ inside 1-r alongside capacity losses.)
absorption_window equilibrium vs the window absorption parameter W per delay.
Usage:
@@ -31,8 +34,8 @@ import numpy as np
import pandas as pd
from tsi_sim.plotting import style
-from tsi_sim.plotting.figures_pernode import equilibrium
-from tsi_sim.theory import expected_ratio
+from tsi_sim.plotting.figures_pernode import equilibrium, recovery_rate, sem
+from tsi_sim.theory import expected_ratio, q_effective
DELAY = "blend_delay_max"
@@ -42,37 +45,56 @@ def _load(run_dir: str | Path) -> pd.DataFrame:
def _eq(df: pd.DataFrame, extra_cols: tuple[str, ...] = ()) -> pd.DataFrame:
- """Equilibrium (post-burn) means per (delay, U) cell, averaged over replicates."""
+ """Equilibrium (post-burn) per (delay, U) cell: replicate mean, SEM and count.
+
+ ``equilibrium`` returns one row per (config, replicate); the spread across those
+ replicates is the only uncertainty estimate available, and at high delay it is large
+ enough to swamp the model difference — so every consumer here carries it.
+ """
eq = equilibrium(df)
keys = [DELAY, "max_uncles", *extra_cols]
- agg = {"mean_ratio": "mean", "mean_q": "mean", "mean_q_eff": "mean"}
- if "deep_ref_share" in eq.columns:
- agg["deep_ref_share"] = "mean"
- return eq.groupby(keys, as_index=False).agg(agg)
+ agg = {"mean_ratio": ("mean_ratio", "mean"),
+ "sem_ratio": ("mean_ratio", sem),
+ "n_rep": ("mean_ratio", "size"),
+ "mean_q": ("mean_q", "mean"),
+ "mean_q_eff": ("mean_q_eff", "mean")}
+ return eq.groupby(keys, as_index=False).agg(**agg)
def fig_accuracy_vs_delay(cnt: pd.DataFrame, old: pd.DataFrame) -> plt.Figure:
- fig, ax = plt.subplots()
+ """Accuracy vs delay per U, with replicate SEM bars.
+
+ U=0 is the NEGATIVE CONTROL: with no uncles the two models are identical by
+ construction, so the visible countable-vs-old gap on that pair of curves is pure
+ between-run RNG noise and calibrates what a real difference has to beat.
+ """
+ fig, ax = plt.subplots(figsize=(6.4, 4.2))
us = sorted(cnt["max_uncles"].unique())
for i, u in enumerate(us):
c = style.color_for(i)
a = cnt[cnt.max_uncles == u].sort_values(DELAY)
b = old[old.max_uncles == u].sort_values(DELAY)
- ax.plot(a[DELAY], a.mean_ratio, "-o", color=c, label=f"U={u} countable", ms=4)
- ax.plot(b[DELAY], b.mean_ratio, "--s", color=c, label=f"U={u} old", ms=4,
- alpha=0.75)
+ ctl = " (control)" if u == 0 else ""
+ ax.errorbar(a[DELAY], a.mean_ratio, yerr=a.sem_ratio, fmt="-o", color=c,
+ label=f"U={u} countable{ctl}", ms=4, capsize=2, lw=1.2)
+ ax.errorbar(b[DELAY], b.mean_ratio, yerr=b.sem_ratio, fmt="--s", color=c,
+ label=f"U={u} unrestricted{ctl}", ms=4, capsize=2, alpha=0.75, lw=1.2)
ax.axhline(1.0, color="0.4", lw=0.8, ls=":")
ax.set_xlabel("max per-relay mixing delay (slots)")
ax.set_ylabel(r"equilibrium $\hat{D}/D_{true}$")
- ax.set_title("Accuracy vs delay: countable (solid) vs old (dashed) uncle model")
- ax.legend(ncol=2)
+ ax.set_title("Accuracy vs delay: countable (solid) vs unrestricted (dashed) referencing")
+ ax.legend(ncol=2, fontsize="x-small")
return fig
def fig_prediction_vs_sim(cnt: pd.DataFrame, f: float) -> plt.Figure:
fig, ax = plt.subplots(figsize=(4.6, 4.4))
sub = cnt[cnt.max_uncles > 0]
- pred = expected_ratio(f, sub.mean_q_eff.to_numpy())
+ # Reconstruct q_u through the theory identity the report quotes, q_u = q + (1-q) r,
+ # rather than reading mean_q_eff straight off the parquet: the figure then exercises
+ # the same closed form the text claims.
+ r = recovery_rate(sub.mean_q.to_numpy(), sub.mean_q_eff.to_numpy())
+ pred = expected_ratio(f, q_effective(sub.mean_q.to_numpy(), r))
us = sorted(sub["max_uncles"].unique())
for i, u in enumerate(us):
m = (sub.max_uncles == u).to_numpy()
@@ -96,8 +118,7 @@ def fig_recovery_vs_delay(cnt: pd.DataFrame) -> plt.Figure:
us = [u for u in sorted(cnt["max_uncles"].unique()) if u > 0]
for i, u in enumerate(us):
a = cnt[cnt.max_uncles == u].sort_values(DELAY)
- denom = np.maximum(1.0 - a.mean_q.to_numpy(), 1e-12)
- r = (a.mean_q_eff.to_numpy() - a.mean_q.to_numpy()) / denom
+ r = recovery_rate(a.mean_q.to_numpy(), a.mean_q_eff.to_numpy())
ax1.plot(a[DELAY], r, "-o", color=style.color_for(i), label=f"U={u}", ms=4)
ax2.semilogy(a[DELAY], np.maximum(1.0 - r, 1e-4), "-o",
color=style.color_for(i), label=f"U={u}", ms=4)
@@ -114,13 +135,20 @@ def fig_recovery_vs_delay(cnt: pd.DataFrame) -> plt.Figure:
def fig_absorption_window(absw: pd.DataFrame) -> plt.Figure:
- fig, ax = plt.subplots()
+ """Accuracy vs the derived window, with replicate SEM bars.
+
+ The bars matter here: at the 32-slot delay the run-to-run spread is the dominant
+ feature (replicate sd up to ~0.22), so that curve's shape is not interpretable as a
+ trend — only its ceiling is, and the ceiling is set by U=1, not by the window.
+ """
+ fig, ax = plt.subplots(figsize=(6.0, 4.0))
eq = equilibrium(absw)
- agg = eq.groupby([DELAY, "window_absorption"], as_index=False).mean_ratio.mean()
+ agg = eq.groupby([DELAY, "window_absorption"], as_index=False).agg(
+ mean_ratio=("mean_ratio", "mean"), sem_ratio=("mean_ratio", sem))
for i, d in enumerate(sorted(agg[DELAY].unique())):
a = agg[agg[DELAY] == d].sort_values("window_absorption")
- ax.plot(a.window_absorption, a.mean_ratio, "-o", color=style.color_for(i),
- label=f"delay={d:g}", ms=4)
+ ax.errorbar(a.window_absorption, a.mean_ratio, yerr=a.sem_ratio, fmt="-o",
+ color=style.color_for(i), label=f"delay={d:g}", ms=4, capsize=2, lw=1.2)
ax.axhline(1.0, color="0.4", lw=0.8, ls=":")
ax.set_xlabel("window absorption parameter W (expected block-intervals)")
ax.set_ylabel(r"equilibrium $\hat{D}/D_{true}$")
@@ -151,17 +179,32 @@ def main() -> None:
written += style.save(fig_recovery_vs_delay(cnt), out / "cvo_recovery_vs_delay", prov)
written += style.save(fig_absorption_window(_load(args.absorption)),
out / "absorption_window", prov)
- # headline numbers for the report / analysis doc
+ # Headline numbers for the report. Every countable-vs-old gap is printed with the
+ # two-sample t = |diff| / SE(diff) over replicates; |t| < 2 means the cell does NOT
+ # resolve a model difference at this replicate count and must not be read as one.
+ # The U=0 rows are the negative control (models identical by construction, so their
+ # t is a pure noise reading).
+ print(f"{'cell':<14} {'countable':>17} {'old':>17} {'diff':>9} {'t':>6} verdict")
for u in sorted(cnt["max_uncles"].unique()):
for _, row in cnt[cnt.max_uncles == u].sort_values(DELAY).iterrows():
q, qu = row.mean_q, row.mean_q_eff
- r = (qu - q) / max(1.0 - q, 1e-12)
+ r = float(recovery_rate(q, qu))
o = old[(old.max_uncles == u) & (old[DELAY] == row[DELAY])]
- old_ratio = float(o.mean_ratio.iloc[0]) if len(o) else float("nan")
- print(f"U={u} delay={row[DELAY]:>5g} countable={row.mean_ratio:.4f} "
- f"old={old_ratio:.4f} q={q:.4f} q_u={qu:.4f} r={r:.4f} "
- f"pred={float(expected_ratio(f, qu)):.4f} "
- f"deep={row.get('deep_ref_share', float('nan')):.4f}")
+ if not len(o):
+ continue
+ orow = o.iloc[0]
+ diff = row.mean_ratio - orow.mean_ratio
+ se = float(np.hypot(row.sem_ratio, orow.sem_ratio))
+ t = abs(diff) / se if se > 0 else float("inf")
+ verdict = ("CONTROL (must be 0)" if u == 0 else
+ "resolved" if t >= 2 else "NOT RESOLVED (noise)")
+ print(f"U={u} delay={row[DELAY]:>5g} "
+ f"{row.mean_ratio:.4f}+-{row.sem_ratio:.4f} "
+ f"{orow.mean_ratio:.4f}+-{orow.sem_ratio:.4f} "
+ f"{diff:+.4f} {t:6.2f} {verdict}")
+ print(f"{'':>14} q={q:.4f} q_u={qu:.4f} r={r:.4f} "
+ f"pred={float(expected_ratio(f, q_effective(q, r))):.4f} "
+ f"n_rep={int(row.n_rep)}")
print(f"wrote {len(written)} files -> {out}")
diff --git a/tools/simulators/tsi/tsi-sim-pernode/scripts/plot_fine_delay.py b/tools/simulators/tsi/tsi-sim-pernode/scripts/plot_fine_delay.py
new file mode 100644
index 0000000..5bfd32c
--- /dev/null
+++ b/tools/simulators/tsi/tsi-sim-pernode/scripts/plot_fine_delay.py
@@ -0,0 +1,200 @@
+"""High-precision figures for the LOW mixing-delay band (the design regime).
+
+countable-vs-old.yaml samples delay at 4/8/16/32 with 5 replicates. That resolves the
+overload regime but leaves the design regime under-measured: every countable-vs-unrestricted
+gap at delay <= 8 sits inside the replicate noise there, so the only honest statement is
+"no difference detected" — with no bound on how large an undetected difference could be.
+
+fine-delay.yaml spends replicates instead of range (delay 1..5, 40 replicates) to turn that
+into a real bound. Consumes:
+
+ tsi-sweep --config configs/fine-delay.yaml --label fine-countable
+ tsi-sweep --config configs/fine-delay.yaml --old --label fine-old
+
+and renders (into --out):
+
+ fine_accuracy_vs_delay equilibrium D/D_true vs delay 1..5, countable (solid) vs
+ unrestricted (dashed) per U, replicate-SEM bars.
+ fine_gap_vs_delay THE precision figure: the countable - unrestricted gap with 95%
+ CIs, against the U=0 negative-control band. A CI straddling zero
+ means no difference at this power; the band shows the floor.
+
+Usage:
+ python scripts/plot_fine_delay.py --countable RUNDIR --old RUNDIR \
+ --out figures/fine-delay
+"""
+
+from __future__ import annotations
+
+import argparse
+from pathlib import Path
+
+import matplotlib.pyplot as plt
+import numpy as np
+import pandas as pd
+
+from tsi_sim.plotting import style
+from tsi_sim.plotting.figures_pernode import equilibrium, sem
+
+DELAY = "blend_delay_max"
+# Normal approximation: with 40 replicates per arm the t-quantile is within ~2% of 1.96,
+# and the replicate spread itself is the dominant uncertainty, so 1.96 is precise enough.
+Z95 = 1.96
+
+
+def _load(run_dir: str | Path) -> pd.DataFrame:
+ return pd.read_parquet(Path(run_dir) / "results.parquet")
+
+
+def _cells(df: pd.DataFrame) -> pd.DataFrame:
+ """Per (delay, U): replicate mean, SEM and count of the equilibrium accuracy."""
+ return equilibrium(df).groupby([DELAY, "max_uncles"], as_index=False).agg(
+ mean_ratio=("mean_ratio", "mean"),
+ sem_ratio=("mean_ratio", sem),
+ n_rep=("mean_ratio", "size"),
+ mean_q=("mean_q", "mean"),
+ mean_q_eff=("mean_q_eff", "mean"))
+
+
+def gaps(cnt: pd.DataFrame, old: pd.DataFrame) -> pd.DataFrame:
+ """countable - unrestricted per cell, with the unpaired SE and 95% CI half-width."""
+ m = cnt.merge(old, on=[DELAY, "max_uncles"], suffixes=("_c", "_o"))
+ m["gap"] = m.mean_ratio_c - m.mean_ratio_o
+ m["se"] = np.hypot(m.sem_ratio_c, m.sem_ratio_o)
+ m["ci95"] = Z95 * m.se
+ m["t"] = np.where(m.se > 0, np.abs(m.gap) / m.se.replace(0, np.nan), np.inf)
+ return m.sort_values(["max_uncles", DELAY])
+
+
+def fig_accuracy(cnt: pd.DataFrame, old: pd.DataFrame) -> plt.Figure:
+ fig, ax = plt.subplots(figsize=(6.4, 4.2))
+ for i, u in enumerate(sorted(cnt["max_uncles"].unique())):
+ c = style.color_for(i)
+ a = cnt[cnt.max_uncles == u].sort_values(DELAY)
+ b = old[old.max_uncles == u].sort_values(DELAY)
+ ctl = " (control)" if u == 0 else ""
+ ax.errorbar(a[DELAY], a.mean_ratio, yerr=a.sem_ratio, fmt="-o", color=c,
+ label=f"U={u} countable{ctl}", ms=4, capsize=2, lw=1.2)
+ ax.errorbar(b[DELAY], b.mean_ratio, yerr=b.sem_ratio, fmt="--s", color=c,
+ label=f"U={u} unrestricted{ctl}", ms=4, capsize=2, alpha=0.75, lw=1.2)
+ ax.axhline(1.0, color="0.4", lw=0.8, ls=":")
+ ax.set_xlabel("max per-relay mixing delay (slots)")
+ ax.set_ylabel(r"equilibrium $\hat{D}/D_{true}$")
+ ax.set_title("Design-regime accuracy: countable vs unrestricted referencing")
+ ax.legend(ncol=2, fontsize="x-small")
+ return fig
+
+
+def pooled_by_delay(g: pd.DataFrame) -> pd.DataFrame:
+ """Inverse-variance pooled gap across the U >= 1 arms, per delay.
+
+ Individual cells are underpowered against a sub-0.1 % effect even at 40 replicates, but
+ the three uncle caps are independent measurements of the same underlying difference, so
+ pooling them buys back a factor of ~sqrt(3) and is what actually resolves the trend.
+ """
+ u = g[g.max_uncles > 0]
+ rows = []
+ for d, s in u.groupby(DELAY):
+ w = 1.0 / s.se.to_numpy() ** 2
+ p = float((s.gap.to_numpy() * w).sum() / w.sum())
+ e = float(np.sqrt(1.0 / w.sum()))
+ rows.append({DELAY: d, "gap": p, "se": e, "ci95": Z95 * e,
+ "t": abs(p) / e if e > 0 else np.inf})
+ return pd.DataFrame(rows).sort_values(DELAY)
+
+
+def fig_gap(g: pd.DataFrame) -> plt.Figure:
+ """The countable − unrestricted gap with 95% CIs, zoomed to the U >= 1 scale.
+
+ The U=0 negative control is NOT plotted as a band here: with no uncles the models are
+ identical by construction, but the unrecovered regime is so noisy that its CI (±0.025)
+ is ~17x the entire range of the U >= 1 gaps and would fill the axes. Its magnitude is
+ annotated instead — the point being that the control's noise floor lives far outside
+ anything the uncle arms show, so those arms are measuring signal, not spread.
+ """
+ fig, ax = plt.subplots(figsize=(6.6, 4.2))
+ pooled = pooled_by_delay(g)
+ for i, u in enumerate(sorted(g["max_uncles"].unique())):
+ if u == 0:
+ continue
+ a = g[g.max_uncles == u].sort_values(DELAY)
+ ax.errorbar(a[DELAY], a.gap, yerr=a.ci95, fmt="-o", color=style.color_for(i),
+ label=f"U={u}", ms=4, capsize=3, lw=1.0, alpha=0.75)
+ ax.errorbar(pooled[DELAY], pooled.gap, yerr=pooled.ci95, fmt="-D", color="0.15",
+ label="pooled over U≥1", ms=5, capsize=4, lw=1.8, zorder=5)
+ ax.axhline(0.0, color="0.3", lw=0.9, ls=":")
+ ax.set_xlabel("max per-relay mixing delay (slots)")
+ ax.set_ylabel(r"$\hat{D}/D$ gap: countable $-$ unrestricted")
+ ax.set_title("The first-fork cost across the design band (95% CI)")
+ ctl = g[g.max_uncles == 0]
+ if len(ctl):
+ band = float(ctl.ci95.max())
+ span = float(np.abs(np.r_[g[g.max_uncles > 0].gap + g[g.max_uncles > 0].ci95,
+ g[g.max_uncles > 0].gap - g[g.max_uncles > 0].ci95]).max())
+ ax.set_ylim(-1.35 * span, 1.35 * span)
+ ax.text(0.015, 0.03,
+ f"U=0 negative control (true gap = 0): 95% CI ±{band:.4f}, "
+ f"{band / span:.0f}× outside this range",
+ transform=ax.transAxes, fontsize=6.5, alpha=0.75)
+ ax.legend(fontsize="x-small", ncol=2)
+ return fig
+
+
+def main() -> None:
+ ap = argparse.ArgumentParser(description=__doc__.splitlines()[0])
+ ap.add_argument("--countable", required=True, help="run dir of fine-countable")
+ ap.add_argument("--old", required=True, help="run dir of fine-old")
+ ap.add_argument("--out", default="figures/fine-delay")
+ args = ap.parse_args()
+ style.apply_style()
+ out = Path(args.out)
+ out.mkdir(parents=True, exist_ok=True)
+
+ cnt, old = _cells(_load(args.countable)), _cells(_load(args.old))
+ g = gaps(cnt, old)
+ prov = "tsi-sim-pernode fine-delay.yaml (+--old)"
+
+ written = []
+ written += style.save(fig_accuracy(cnt, old), out / "fine_accuracy_vs_delay", prov)
+ written += style.save(fig_gap(g), out / "fine_gap_vs_delay", prov)
+
+ print(f"{'cell':<16} {'countable':>17} {'unrestricted':>17} "
+ f"{'gap':>9} {'95% CI':>9} {'t':>6} verdict")
+ for _, r in g.iterrows():
+ verdict = ("CONTROL (true gap = 0)" if r.max_uncles == 0 else
+ "resolved" if r.t >= 2 else "no difference resolved")
+ print(f"U={int(r.max_uncles)} delay={r[DELAY]:>5g} "
+ f"{r.mean_ratio_c:.4f}+-{r.sem_ratio_c:.4f} "
+ f"{r.mean_ratio_o:.4f}+-{r.sem_ratio_o:.4f} "
+ f"{r.gap:+.4f} +-{r.ci95:.4f} {r.t:6.2f} {verdict}")
+ worst = g[g.max_uncles > 0]
+ print(f"\nn_rep = {int(g.n_rep_c.min())}/{int(g.n_rep_o.min())} per arm")
+ print(f"widest 95% CI half-width at U>=1: +-{worst.ci95.max():.4f} "
+ f"({100 * worst.ci95.max():.2f} pp)")
+ # Per-cell significance must be read against the number of cells tested: with 15 cells,
+ # ~0.75 are expected to clear t=2 by chance alone, so quote the Bonferroni threshold.
+ bonf = 2.935 if len(worst) == 15 else float("nan")
+ print(f"per-cell: {int((worst.t >= 2).sum())}/{len(worst)} cells with t>=2 "
+ f"(expected by chance {0.05 * len(worst):.2f}); max t = {worst.t.max():.2f} "
+ f"vs Bonferroni threshold {bonf:.3f}")
+
+ print("\npooled over U>=1 (the three caps measure the same difference):")
+ pooled = pooled_by_delay(g)
+ for _, r in pooled.iterrows():
+ mark = " <-- resolved" if r.t >= 2 else ""
+ print(f" delay={r[DELAY]:>4g}: {r.gap:+.5f} +-{r.ci95:.5f} t={r.t:5.2f}{mark}")
+ w = 1.0 / worst.se.to_numpy() ** 2
+ allp = float((worst.gap.to_numpy() * w).sum() / w.sum())
+ alle = float(np.sqrt(1.0 / w.sum()))
+ print(f" whole band: {allp:+.5f} +-{Z95 * alle:.5f} t={abs(allp) / alle:.2f}")
+
+ ctl = g[g.max_uncles == 0]
+ if len(ctl):
+ print(f"\nU=0 negative control (true gap = 0): |gap| up to {ctl.gap.abs().max():.4f}, "
+ f"max t = {ctl.t.max():.2f}, 95% CI +-{ctl.ci95.max():.4f} "
+ f"-> control {'PASSES' if ctl.t.max() < 2 else 'FAILS'}")
+ print(f"wrote {len(written)} files -> {out}")
+
+
+if __name__ == "__main__":
+ main()
diff --git a/tools/simulators/tsi/tsi-sim-pernode/src/tsi_sim/measure.py b/tools/simulators/tsi/tsi-sim-pernode/src/tsi_sim/measure.py
index 80945c2..4bc5081 100644
--- a/tools/simulators/tsi/tsi-sim-pernode/src/tsi_sim/measure.py
+++ b/tools/simulators/tsi/tsi-sim-pernode/src/tsi_sim/measure.py
@@ -12,7 +12,13 @@ Counting models (``countable`` flag; CLI ``--old`` clears it):
(cryptarchia-v1-protocol.md): the reference must be within the window
(``0 < slot_B - slot_U <= w``), the uncle must not lie on the counting chain, and its
**parent must lie on the counting chain** (only the first block of a fork counts).
- References failing the parent rule are tallied as ``deep`` diagnostics.
+ References failing the parent rule are tallied as ``deep``. On a real run that tally is
+ always ZERO and is a defensive invariant, not a measured rate: countable *selection*
+ already refuses non-first-fork candidates, and for a chain block ``B`` the producer's
+ chain below ``B`` is the same ancestor path as the counting chain below ``B``, so the
+ two rules cannot disagree. A non-zero ``deep`` means selection and counting have drifted
+ (asserted end-to-end by test_countable_counting.py). Only hand-baked references — the
+ ones the unit tests construct — can make it fire.
* **old** — every baked reference in the measurement window counts (fork depth ignored),
reproducing the pre-redesign behaviour.
@@ -51,9 +57,11 @@ class Measurement:
q: np.ndarray # (N,) honest active-slot fraction
q_eff: np.ndarray # (N,) uncle-recovered fraction
orphan_rate: np.ndarray # (N,)
- ref_total: np.ndarray # (N,) distinct referenced uncles examined in the window
- ref_deep: np.ndarray # (N,) of those, rejected by the parent-on-chain (first-fork) rule;
- # always 0 under the old model (no rule to reject on)
+ ref_total: np.ndarray # (N,) distinct referenced uncles examined (slot in [0,T); the
+ # per-reference window check is applied after this tally)
+ ref_deep: np.ndarray # (N,) of those, rejected by the parent-on-chain (first-fork) rule.
+ # ALWAYS 0 on a real run under either model — a selection/
+ # counting drift alarm, not a rate (see the module docstring)
agreement_window: float
agreement_tip: float
diff --git a/tools/simulators/tsi/tsi-sim-pernode/src/tsi_sim/plotting/figures_pernode.py b/tools/simulators/tsi/tsi-sim-pernode/src/tsi_sim/plotting/figures_pernode.py
index e7c3164..dfad269 100644
--- a/tools/simulators/tsi/tsi-sim-pernode/src/tsi_sim/plotting/figures_pernode.py
+++ b/tools/simulators/tsi/tsi-sim-pernode/src/tsi_sim/plotting/figures_pernode.py
@@ -27,6 +27,27 @@ CONFIG_COLS = ["n_nodes", "stake_dist", "pareto_shape", "topology", "degree",
GRAPH_TOPOLOGIES = ("regular", "blend")
+def sem(x) -> float:
+ """Standard error of the mean over replicates (0 for a single replicate).
+
+ Replicate spread is the only uncertainty estimate these sweeps carry, and at high
+ mixing delay it is large enough to swamp the effects being compared — so any figure or
+ table quoting a cell mean should quote this alongside it.
+ """
+ x = np.asarray(x, dtype=float)
+ return float(x.std(ddof=1) / np.sqrt(len(x))) if len(x) > 1 else 0.0
+
+
+def recovery_rate(q, q_u):
+ """Invert ``theory.q_effective``: ``r = (q_u - q) / (1 - q)``.
+
+ The share of wasted active slots that a countable uncle reference puts back into the
+ count. Clamped denominator so a saturated ``q -> 1`` cell stays finite.
+ """
+ q, q_u = np.asarray(q, dtype=float), np.asarray(q_u, dtype=float)
+ return (q_u - q) / np.maximum(1.0 - q, 1e-12)
+
+
def _lat_axis(topo: str) -> tuple[str, str]:
"""(dataframe column, axis label) for the dominant latency knob of a graph topology."""
if topo == "blend":
diff --git a/tools/simulators/tsi/tsi-sim-pernode/src/tsi_sim/uncles.py b/tools/simulators/tsi/tsi-sim-pernode/src/tsi_sim/uncles.py
index 04cc065..ba11961 100644
--- a/tools/simulators/tsi/tsi-sim-pernode/src/tsi_sim/uncles.py
+++ b/tools/simulators/tsi/tsi-sim-pernode/src/tsi_sim/uncles.py
@@ -16,79 +16,21 @@ block id only (no slot exclusion).
Selected refs are baked at production and immutable once adopted, so density counting
stays view-independent under both models.
+
+``select_uncles_at_production`` is the ONLY selection entry point — blocktree.py calls it
+once per produced block. Tests drive it through the same signature (see
+tests/test_uncles.py ``annotate_via_production``) rather than through a parallel offline
+implementation, so there is no second copy of the rules to drift out of sync.
"""
from __future__ import annotations
import numpy as np
-from .blocktree import GENESIS, BlockTree
+from .blocktree import GENESIS
from .config import SimConfig
-def _orphans_sorted(tree: BlockTree, canonical_ids: list[int]) -> tuple[np.ndarray, np.ndarray]:
- """Return orphan block ids sorted by (slot, id) and their slots."""
- canonical = np.zeros(tree.n_blocks, dtype=bool)
- canonical[canonical_ids] = True
- all_real = np.arange(1, tree.n_blocks)
- orphan_ids = all_real[~canonical[1:]]
- orphan_slots = tree.slot[orphan_ids]
- order = np.lexsort((orphan_ids, orphan_slots)) # by slot, then id
- return orphan_ids[order], orphan_slots[order]
-
-
-def annotate_uncles(
- tree: BlockTree, canonical_ids: list[int], config: SimConfig, rng: np.random.Generator
-) -> None:
- """Fill ``tree.uncles[B]`` for every canonical block ``B`` per the selection rule (offline).
-
- Countable model: candidates are restricted to orphans whose parent is canonical (first
- fork blocks), slots already occupied on the chain are excluded, and at most one uncle
- per slot is picked. Old model: any orphan in the window, dedup by id only.
- """
- u_max = config.max_uncles
- if u_max <= 0:
- return
- w = config.effective_uncle_window
- orphan_ids, orphan_slots = _orphans_sorted(tree, canonical_ids)
- if orphan_ids.size == 0:
- return
-
- countable = config.uncle_model != "old"
- occupied: set[int] = set()
- if countable:
- canonical = set(canonical_ids)
- keep = [i for i in range(orphan_ids.size)
- if int(tree.parent[orphan_ids[i]]) == GENESIS
- or int(tree.parent[orphan_ids[i]]) in canonical]
- orphan_ids, orphan_slots = orphan_ids[keep], orphan_slots[keep]
- if orphan_ids.size == 0:
- return
- occupied = {int(tree.slot[b]) for b in canonical_ids}
-
- referenced: set[int] = set()
- # oldest canonical block first
- for b in reversed(canonical_ids):
- sb = int(tree.slot[b])
- lo = int(np.searchsorted(orphan_slots, sb - w, side="left")) # slot_U >= sb - w
- hi = int(np.searchsorted(orphan_slots, sb, side="left")) # slot_U < sb
- if hi <= lo:
- continue
- window_ids = orphan_ids[lo:hi] # already oldest-first
- if countable:
- window_ids = np.array(
- [x for x in window_ids.tolist() if int(tree.slot[x]) not in occupied],
- dtype=np.int64,
- )
- selected = _select(window_ids, referenced, config, rng,
- slot=tree.slot, one_per_slot=countable)
- if selected:
- tree.uncles[b] = tuple(selected)
- referenced.update(selected)
- if countable:
- occupied.update(int(tree.slot[u]) for u in selected)
-
-
def _select(
window_ids: np.ndarray,
referenced: set[int],
diff --git a/tools/simulators/tsi/tsi-sim-pernode/tests/test_countable_counting.py b/tools/simulators/tsi/tsi-sim-pernode/tests/test_countable_counting.py
index 748516a..b23a573 100644
--- a/tools/simulators/tsi/tsi-sim-pernode/tests/test_countable_counting.py
+++ b/tools/simulators/tsi/tsi-sim-pernode/tests/test_countable_counting.py
@@ -82,3 +82,28 @@ def test_window_recheck_rejects_stale_reference():
tree = _tree()
assert countable_refs(tree, CANONICAL, 2) == {5} # 7, 8 now out of window (d=3)
assert density_m(tree, CANONICAL, T, countable=True, w=2) == 5
+
+
+def test_deep_ref_share_is_zero_end_to_end():
+ """The counting-side parent-on-chain re-check must never fire on a real countable run.
+
+ Countable SELECTION already refuses to reference a non-first-fork block, and for a
+ block ``b`` on the counting chain the producer's chain below ``b`` IS the counting
+ chain below ``b`` (they are the same ancestor path). So ``ref_deep`` is a defensive
+ invariant, not a measured rate: any non-zero value means selection and counting have
+ drifted apart. The hand-built trees above are the only way to make it fire — they bake
+ references selection would never emit.
+ """
+ import pandas as pd
+
+ from tsi_sim.config import SimConfig
+ from tsi_sim.engine import run_trajectory
+
+ # Small but fork-rich: Blend delay spreads proposals over many slots.
+ cfg = SimConfig(n_nodes=60, topology="blend", blend_hops=2, blend_delay_max=8.0,
+ degree=4, max_uncles=2, k=32, epochs=3, f=0.1,
+ stake_dist="pareto", init_dest="common")
+ df = pd.DataFrame(run_trajectory(cfg))
+ assert df.n_blocks.sum() > 0 # the run actually produced blocks
+ assert (df.deep_ref_share == 0.0).all(), (
+ f"counting rejected references selection emitted: {df.deep_ref_share.tolist()}")
diff --git a/tools/simulators/tsi/tsi-sim-pernode/tests/test_theory.py b/tools/simulators/tsi/tsi-sim-pernode/tests/test_theory.py
index 8141e8b..b0f6f98 100644
--- a/tools/simulators/tsi/tsi-sim-pernode/tests/test_theory.py
+++ b/tools/simulators/tsi/tsi-sim-pernode/tests/test_theory.py
@@ -38,3 +38,46 @@ def test_fixed_point_bias_about_one_percent():
b = theory.fixed_point_bias(F)
assert abs(b - (F / (33 / 1000))) < 1e-12
assert 1.005 < b < 1.02
+
+
+def test_q_effective_interpolates_between_q_and_one():
+ # r = 0 -> no recovery (chain-only q); r = 1 -> every wasted slot recovered (q_u = 1).
+ for q in (0.3, 0.65, 0.9):
+ assert abs(float(theory.q_effective(q, 0.0)) - q) < 1e-15
+ assert abs(float(theory.q_effective(q, 1.0)) - 1.0) < 1e-15
+ # monotone and strictly between for partial recovery
+ half = float(theory.q_effective(q, 0.5))
+ assert q < half < 1.0
+
+
+def test_q_effective_round_trips_the_measured_recovery_rate():
+ # The identity the report quotes: given measured q and q_u, r = (q_u - q)/(1 - q)
+ # reconstructs q_u exactly. This is how plot_countable_vs_old.py derives its overlay.
+ for q, q_u in ((0.3129, 0.6147), (0.5380, 0.9874), (0.7485, 0.9992)):
+ r = (q_u - q) / (1.0 - q)
+ assert abs(float(theory.q_effective(q, r)) - q_u) < 1e-12
+
+
+def test_q_effective_recovers_unbiased_equilibrium():
+ # Full recovery must lift the equilibrium to exactly 1 for any starting q.
+ for q in (0.3, 0.65, 0.9):
+ assert abs(float(theory.expected_ratio(F, theory.q_effective(q, 1.0))) - 1.0) < 1e-12
+
+
+def test_window_miss_prob_decays_as_exp_minus_w():
+ # P(no canonical block in w_u = W/f slots) = (1-f)^(W/f) = exp(W * ln(1-f)/f).
+ # The docstring's "~ e^-W" is the f -> 0 limit: ln(1-f)/f = -(1 + f/2 + ...) = -1.0170
+ # at f = 1/30, so the true decay is slightly FASTER than e^-W, by a factor that grows
+ # with W (16% low by W = 10). Assert the exact form, and bracket the heuristic.
+ rate = np.log(1.0 - F) / F
+ assert -1.02 < rate < -1.0
+ for w_abs in (1.0, 3.0, 10.0):
+ p = float(theory.window_miss_prob(F, w_abs))
+ assert abs(p - (1.0 - F) ** (w_abs / F)) < 1e-15
+ assert abs(p - np.exp(rate * w_abs)) < 1e-15 # exact closed form
+ assert np.exp(-1.02 * w_abs) < p < np.exp(-w_abs) # brackets the e^-W heuristic
+ # the spec default W = 10 makes the window a negligible loss channel
+ assert float(theory.window_miss_prob(F, 10.0)) < 1e-4
+ # strictly decreasing in W
+ ps = [float(theory.window_miss_prob(F, w)) for w in (1, 2, 3, 5, 7, 10)]
+ assert all(a > b for a, b in zip(ps[:-1], ps[1:], strict=True))
diff --git a/tools/simulators/tsi/tsi-sim-pernode/tests/test_uncles.py b/tools/simulators/tsi/tsi-sim-pernode/tests/test_uncles.py
index 4240cc8..8807728 100644
--- a/tools/simulators/tsi/tsi-sim-pernode/tests/test_uncles.py
+++ b/tools/simulators/tsi/tsi-sim-pernode/tests/test_uncles.py
@@ -2,7 +2,7 @@ import numpy as np
from tsi_sim.blocktree import BlockTree
from tsi_sim.config import SimConfig
-from tsi_sim.uncles import annotate_uncles
+from tsi_sim.uncles import select_uncles_at_production
def make_tree(slots, parents, heights, leaders):
@@ -16,6 +16,23 @@ def make_tree(slots, parents, heights, leaders):
)
+def annotate_via_production(tree, canonical_ids, config, rng):
+ """Fill ``tree.uncles`` by replaying the PRODUCTION selection for each canonical block.
+
+ The simulator has exactly one selection implementation
+ (``select_uncles_at_production``, called from blocktree.py per produced block); this
+ replays it offline — oldest canonical block first, each block selecting as its producer
+ would have at ``t = slot[b]`` extending ``parent[b]`` — so the tests below exercise the
+ live code path rather than a parallel offline copy. Requires ``tree.slot`` ascending by
+ block id (true for every tree these tests build, and for real runs).
+ """
+ arrival = np.zeros(tree.n_blocks) # every block already in the producer's view
+ for b in reversed(canonical_ids): # oldest first, so earlier refs are visible
+ tree.uncles[b] = select_uncles_at_production(
+ tree.slot, tree.parent, tree.uncles, arrival, nb=tree.n_blocks,
+ parent_id=int(tree.parent[b]), t=int(tree.slot[b]), config=config, rng=rng)
+
+
def _canonical_and_orphan_tree():
# genesis(0); canonical chain 1(slot0)->3(slot3)->4(slot5); orphan 2(slot1)
tree = make_tree(
@@ -31,7 +48,7 @@ def _canonical_and_orphan_tree():
def test_oldest_selection_and_window():
tree, canonical = _canonical_and_orphan_tree()
cfg = SimConfig(max_uncles=1, uncle_window=300, uncle_strategy="oldest")
- annotate_uncles(tree, canonical, cfg, np.random.default_rng(0))
+ annotate_via_production(tree, canonical, cfg, np.random.default_rng(0))
# orphan 2 (slot1) is within window of block 3 (slot3) -> referenced there
referenced = {u for b in canonical for u in tree.uncles[b]}
assert referenced == {2}
@@ -40,7 +57,7 @@ def test_oldest_selection_and_window():
def test_no_uncles_when_u_zero():
tree, canonical = _canonical_and_orphan_tree()
cfg = SimConfig(max_uncles=0)
- annotate_uncles(tree, canonical, cfg, np.random.default_rng(0))
+ annotate_via_production(tree, canonical, cfg, np.random.default_rng(0))
assert all(tree.uncles[b] == () for b in canonical)
@@ -49,7 +66,7 @@ def test_window_excludes_out_of_range_orphan():
# and derives the window from window_absorption — see the countable twin below.)
tree, canonical = _canonical_and_orphan_tree()
cfg = SimConfig(max_uncles=1, uncle_window=1, uncle_strategy="oldest", uncle_model="old")
- annotate_uncles(tree, canonical, cfg, np.random.default_rng(0))
+ annotate_via_production(tree, canonical, cfg, np.random.default_rng(0))
# orphan 2 at slot1; nearest canonical after it is block3 at slot3 -> gap 2 > W=1
referenced = {u for b in canonical for u in tree.uncles[b]}
assert referenced == set()
@@ -61,14 +78,14 @@ def test_countable_window_is_derived_from_absorption():
tree, canonical = _canonical_and_orphan_tree()
cfg = SimConfig(max_uncles=1, f=0.5, window_absorption=1.0)
assert cfg.effective_uncle_window == 2
- annotate_uncles(tree, canonical, cfg, np.random.default_rng(0))
+ annotate_via_production(tree, canonical, cfg, np.random.default_rng(0))
referenced = {u for b in canonical for u in tree.uncles[b]}
assert referenced == {2} # block3 (slot3) still reaches it
# shrink f so the derived window rounds to 1 slot: gap 2 > 1 -> excluded
tree2, canonical2 = _canonical_and_orphan_tree()
cfg2 = SimConfig(max_uncles=1, f=0.9, window_absorption=1.0)
assert cfg2.effective_uncle_window == 1
- annotate_uncles(tree2, canonical2, cfg2, np.random.default_rng(0))
+ annotate_via_production(tree2, canonical2, cfg2, np.random.default_rng(0))
assert {u for b in canonical2 for u in tree2.uncles[b]} == set()
@@ -88,8 +105,8 @@ def test_random_strategy_deterministic_and_capped():
tree_a, canon = _wide_orphan_tree()
tree_b, _ = _wide_orphan_tree()
cfg = SimConfig(max_uncles=2, uncle_window=300, uncle_strategy="random", uncle_random_p=0.5)
- annotate_uncles(tree_a, canon, cfg, np.random.default_rng(7))
- annotate_uncles(tree_b, canon, cfg, np.random.default_rng(7))
+ annotate_via_production(tree_a, canon, cfg, np.random.default_rng(7))
+ annotate_via_production(tree_b, canon, cfg, np.random.default_rng(7))
assert tree_a.uncles == tree_b.uncles # same seed -> identical
total = sum(len(tree_a.uncles[b]) for b in canon)
assert total <= cfg.max_uncles # capped
@@ -99,18 +116,20 @@ def test_random_p_one_matches_oldest():
import numpy as np
tree_r, canon = _wide_orphan_tree()
tree_o, _ = _wide_orphan_tree()
- annotate_uncles(tree_r, canon, SimConfig(max_uncles=2, uncle_strategy="random",
- uncle_random_p=1.0), np.random.default_rng(1))
- annotate_uncles(tree_o, canon, SimConfig(max_uncles=2, uncle_strategy="oldest"),
- np.random.default_rng(1))
+ annotate_via_production(tree_r, canon, SimConfig(max_uncles=2, uncle_strategy="random",
+ uncle_random_p=1.0),
+ np.random.default_rng(1))
+ annotate_via_production(tree_o, canon, SimConfig(max_uncles=2, uncle_strategy="oldest"),
+ np.random.default_rng(1))
assert tree_r.uncles == tree_o.uncles # p=1 deterministically takes oldest-first
def test_random_p_zero_selects_nothing():
import numpy as np
tree, canon = _wide_orphan_tree()
- annotate_uncles(tree, canon, SimConfig(max_uncles=4, uncle_strategy="random",
- uncle_random_p=0.0), np.random.default_rng(1))
+ annotate_via_production(tree, canon, SimConfig(max_uncles=4, uncle_strategy="random",
+ uncle_random_p=0.0),
+ np.random.default_rng(1))
assert all(tree.uncles[b] == () for b in canon)
@@ -124,7 +143,7 @@ def test_dedup_across_ancestors():
)
canonical = [4, 3, 1]
cfg = SimConfig(max_uncles=4, uncle_window=300, uncle_strategy="oldest")
- annotate_uncles(tree, canonical, cfg, np.random.default_rng(0))
+ annotate_via_production(tree, canonical, cfg, np.random.default_rng(0))
counts = sum(len(tree.uncles[b]) for b in canonical)
assert counts == 1 # orphan 2 referenced exactly once despite two eligible blocks
@@ -147,7 +166,7 @@ def _deep_fork_tree():
def test_countable_excludes_deep_fork_blocks():
tree, canonical = _deep_fork_tree()
cfg = SimConfig(max_uncles=4)
- annotate_uncles(tree, canonical, cfg, np.random.default_rng(0))
+ annotate_via_production(tree, canonical, cfg, np.random.default_rng(0))
referenced = {u for b in canonical for u in tree.uncles[b]}
assert referenced == {2, 4} # deep block 3 (parent is an orphan) excluded
@@ -155,7 +174,7 @@ def test_countable_excludes_deep_fork_blocks():
def test_old_model_still_references_deep_fork_blocks():
tree, canonical = _deep_fork_tree()
cfg = SimConfig(max_uncles=4, uncle_model="old", uncle_window=300)
- annotate_uncles(tree, canonical, cfg, np.random.default_rng(0))
+ annotate_via_production(tree, canonical, cfg, np.random.default_rng(0))
referenced = {u for b in canonical for u in tree.uncles[b]}
assert referenced == {2, 3, 4} # --old: fork depth ignored
@@ -170,7 +189,7 @@ def test_countable_excludes_occupied_slots_and_dedups_per_slot():
)
canonical = [5, 1]
cfg = SimConfig(max_uncles=4)
- annotate_uncles(tree, canonical, cfg, np.random.default_rng(0))
+ annotate_via_production(tree, canonical, cfg, np.random.default_rng(0))
referenced = {u for b in canonical for u in tree.uncles[b]}
# slot0 is canonical-occupied -> orphan 2 excluded; slot2 pair -> exactly one picked
assert referenced == {3}
@@ -196,3 +215,53 @@ def test_production_selection_countable_rules():
slot, parent, uncles, arrival, nb=6, parent_id=1, t=5, config=old,
rng=np.random.default_rng(0))
assert sel_old == (2, 3, 4, 5) # --old: depth and slot-dedup ignored
+
+
+def test_production_parent_below_window_resolves_chain_membership():
+ """Candidate parents can sit BELOW the reference window (uncles.py `pmin`/`below` walk).
+
+ The window walk only establishes chain membership for ancestors with slot >= t-w, so a
+ candidate whose parent is older than that needs the extra walk down to `pmin`. With a
+ narrow window this is the branch that decides first-fork eligibility.
+
+ chain 1(s0) -> 2(s1) -> 6(s8); off-chain 7(s2, parent 3) where 3(s2) is NOT on the chain.
+ Candidates at t=9 with w=4 are the blocks in slots [5,9): 4(s5, parent 2 -> ON chain,
+ countable) and 5(s6, parent 3 -> OFF chain, deep). Both parents sit below the window,
+ so only the `below` walk can tell them apart.
+ """
+ from tsi_sim.uncles import select_uncles_at_production
+
+ # id: 0 1 2 3 4 5 6
+ slot = np.array([-1, 0, 1, 2, 5, 6, 8], np.int64)
+ parent = np.array([-1, 0, 1, 1, 2, 3, 2], np.int64)
+ uncles: list = [() for _ in range(7)]
+ arrival = np.zeros(7)
+ # w = round(1/0.25) = 4 slots; window at t=9 is [5, 9)
+ cfg = SimConfig(max_uncles=4, f=0.25, window_absorption=1.0)
+ assert cfg.effective_uncle_window == 4
+ sel = select_uncles_at_production(
+ slot, parent, uncles, arrival, nb=7, parent_id=6, t=9, config=cfg,
+ rng=np.random.default_rng(0))
+ assert sel == (4,) # 5 rejected: parent 3 is off-chain (deep) and below the window
+
+
+def test_production_excludes_slots_occupied_by_chain_and_prior_uncles():
+ """The occupied-slot exclusion is built from the producer's chain walk (uncles.py).
+
+ chain 1(s0) -> 2(s1) -> 5(s4); block 2 already references uncle 3(s2). Candidates at
+ t=5 are 3 (already referenced), 4(s2, same slot as the referenced 3 -> slot occupied)
+ and 6(s3, free slot). Only 6 survives.
+ """
+ from tsi_sim.uncles import select_uncles_at_production
+
+ # id: 0 1 2 3 4 5 6
+ slot = np.array([-1, 0, 1, 2, 2, 4, 3], np.int64)
+ parent = np.array([-1, 0, 1, 1, 1, 2, 1], np.int64)
+ uncles: list = [() for _ in range(7)]
+ uncles[2] = (3,) # block 2 already referenced uncle 3
+ arrival = np.zeros(7)
+ cfg = SimConfig(max_uncles=4) # w = 300, whole tree in window
+ sel = select_uncles_at_production(
+ slot, parent, uncles, arrival, nb=7, parent_id=5, t=5, config=cfg,
+ rng=np.random.default_rng(0))
+ assert sel == (6,) # 3 already referenced; 4 shares slot 2 with it; 6 is free