From a82e4cb706087ab0c69ac2c408cdc5117600f779 Mon Sep 17 00:00:00 2001 From: Marcin Pawlowski Date: Fri, 7 Aug 2026 11:31:47 +0200 Subject: [PATCH] E5: the section's number is reproducible at ~8 slots of per-recipient spread The last and most consequential handoff experiment -- the only one that could have invalidated the report rather than the spec section. It does not, but the result is more useful than a refutation: it identifies the assumption the section's number encodes. Sweeping per-(block,node) jitter interpolates between the two transport models, since jitter is precisely what the standalone simulation had and the Blend cascade does not. At the deployment's operating point, D-hat/D at U=1 runs 0.9983 / 0.9992 / 0.9991 / 0.9971 / 0.9871 for jitter 0/1/2/4/8 slots, with the share of orphans below their fork's first block rising 0.25% -> 3.30%. So BOTH branches of the handoff's pass/fail are true at different levels: flat to ~2 slots, and at 8 slots it lands on 0.9871 -- essentially the section's 0.986. That turns the disagreement into a checkable question. The cascade's variance is per BLOCK (every recipient moved together by the final flood, harmless to the estimate); jitter is per RECIPIENT (divergent, and the only kind that makes forks deeper than one block, which the counting rule cannot recover). Blend delivers ~0.5 slot of per-recipient spread over a degree-6 graph, so the design sits an order of magnitude inside the flat region and 8 slots is ~16x what the transport produces. The section charged per-recipient variance where Blend delivers per-block; that distinction is the whole disagreement. Consensus is untouched at every level (range_ratio = 0 and agreement = 1.000 in all 480 runs), reconfirming sec 6.1 at the deployment's own point. Partly closes open item 15: the report now carries a measured bound on per-recipient variance instead of assuming the transport. Spatially CORRELATED latency is still untested -- jitter is i.i.d. per (block, node), so it cannot model a region straggling as a bloc, and item 15 is narrowed to that. Also lands the sec 4.1 item 2 relabel from the handoff guide: the unrestricted arm is an unreachable CEILING, not a candidate design, because the spec now rejects a block carrying an entry that fails the counting rules. Sec 2.1 and the header note say so explicitly rather than calling it "the comparison baseline". Co-Authored-By: Claude Fable 5 --- reports/tsi/README.md | 21 +++++++++++++--- reports/tsi/handoff-fork-loss-results.md | 32 ++++++++++++++++++++---- 2 files changed, 44 insertions(+), 9 deletions(-) diff --git a/reports/tsi/README.md b/reports/tsi/README.md index fd3aa09..647f842 100644 --- a/reports/tsi/README.md +++ b/reports/tsi/README.md @@ -58,7 +58,7 @@ --- -> **The uncle-reference model analysed here.** Uncle references are *counting-only* — they never affect a block's validity ([§6.7](#s6-7)–[§6.8](#s6-8)) — and are deduplicated by **slot**, not by block ([§8.5](#s8-5)). The reference window is derived from a **window absorption parameter**, `w_u = W_abs/f` with `W_abs = 10` expected block-intervals and bound `W_abs ≤ 0.6·k`, which is the `W = 300 = 10/f` sizing rule of [§3.4](#s3-4). One structural constraint shapes the results: only the **first block of a fork** — one whose parent lies on the referencing chain — is countable, which is the price of verifying every reference from chain data alone. This report calls that the **countable** model, and measures it against an **unrestricted** baseline in which any orphan in the window is referenceable at any fork depth ([§2.1](#s2-1), [§3.2](#s3-2), [§9](#s9)). The result: **in the design regime `ρ < 1` the two are practically identical.** A dedicated 40-replicate **paired** sweep of the operating band (`δ_max` 1–5, `ρ ≈ 0.21–0.41`), in which both arms share the stake draw, the peering graph and every lottery outcome so the uncle rule is the only difference, puts every `U ≥ 1` cell of both models in 0.998–1.001 and pins the first-fork cost at **nothing measurable up to `δ_max` = 3 and ~0.1 % at `δ_max` = 4–5** ([§3.2a](#s3-2a)). Measured against the unrestricted baseline under common random numbers, the cost grows monotonically with load — 0.13 % at `ρ ≈ 0.36`, 0.34 % at `ρ ≈ 0.56`, 1.0 % at `ρ ≈ 0.96` and 2.3 % under overload at `ρ ≈ 1.76`, where recovery saturates near 0.95 against the unrestricted model's ~1.00. Deep forks are unrecoverable by construction, so past `ρ ≈ 1` the ceiling is set by the fork *depth* profile and **no uncle cap buys it back** — a second, independent reason for the [§8](#s8) "keep `ρ < 1`" rule. The closed-form accuracy with the measured effective utilization, `log(1−f)/log(1−f/q_u)`, predicts every countable cell to within 0.2 %. +> **The uncle-reference model analysed here.** Uncle references are *counting-only* — they never affect a block's validity ([§6.7](#s6-7)–[§6.8](#s6-8)) — and are deduplicated by **slot**, not by block ([§8.5](#s8-5)). The reference window is derived from a **window absorption parameter**, `w_u = W_abs/f` with `W_abs = 10` expected block-intervals and bound `W_abs ≤ 0.6·k`, which is the `W = 300 = 10/f` sizing rule of [§3.4](#s3-4). One structural constraint shapes the results: only the **first block of a fork** — one whose parent lies on the referencing chain — is countable, which is the price of verifying every reference from chain data alone. This report calls that the **countable** model, and measures it against an **unrestricted** ceiling in which any orphan in the window is referenceable at any fork depth ([§2.1](#s2-1), [§3.2](#s3-2), [§9](#s9)) — a bound on what any counting rule could recover, not a candidate design, since the spec rejects a block carrying an entry that fails the rules. The result: **in the design regime `ρ < 1` the two are practically identical.** A dedicated 40-replicate **paired** sweep of the operating band (`δ_max` 1–5, `ρ ≈ 0.21–0.41`), in which both arms share the stake draw, the peering graph and every lottery outcome so the uncle rule is the only difference, puts every `U ≥ 1` cell of both models in 0.998–1.001 and pins the first-fork cost at **nothing measurable up to `δ_max` = 3 and ~0.1 % at `δ_max` = 4–5** ([§3.2a](#s3-2a)). Measured against the unrestricted baseline under common random numbers, the cost grows monotonically with load — 0.13 % at `ρ ≈ 0.36`, 0.34 % at `ρ ≈ 0.56`, 1.0 % at `ρ ≈ 0.96` and 2.3 % under overload at `ρ ≈ 1.76`, where recovery saturates near 0.95 against the unrestricted model's ~1.00. Deep forks are unrecoverable by construction, so past `ρ ≈ 1` the ceiling is set by the fork *depth* profile and **no uncle cap buys it back** — a second, independent reason for the [§8](#s8) "keep `ρ < 1`" rule. The closed-form accuracy with the measured effective utilization, `log(1−f)/log(1−f/q_u)`, predicts every countable cell to within 0.2 %. ## 1. Executive summary @@ -136,7 +136,7 @@ D̂_next = max(1, D̂ · (1 − β·(f_p − m/T)/f_p)) β = learning rate Occupied slots more frequent than the target push the estimate up (the lottery then gets harder); less frequent pulls it down; `m/T = f_p` is the resting point. The two versions differ **only in which slots count**: - **Before uncle references (spec v1.0):** `m` counts only slots occupied by the *canonical* chain. Under Blend delay roughly a third of honest blocks are orphaned, so their slots vanish from `m`, the resting point sits far below truth (`D̂/D ≈ 0.64–0.74`, deepening with network size), and the deflated estimate makes the network *chronically over-produce* blocks at up to ~2× the target rate ([§3.2](#s3-2), fig1). Pre-uncle TSI is not viable under Blend. -- **With uncle references, counting whole blocks (the *unrestricted* model):** `m` also counts the *referenced uncles* — orphans pointed at from the canonical chain — which puts the delay-orphaned slots back into the count and lifts the resting point up toward `D`. Any orphan in the reference window is eligible regardless of how deep in a fork it sits, and references are deduplicated by *block identity*. This report analyses it as the comparison baseline throughout, and calls it the **unrestricted** model. +- **With uncle references, counting whole blocks (the *unrestricted* model):** `m` also counts the *referenced uncles* — orphans pointed at from the canonical chain — which puts the delay-orphaned slots back into the count and lifts the resting point up toward `D`. Any orphan in the reference window is eligible regardless of how deep in a fork it sits, and references are deduplicated by *block identity*. This report analyses it throughout, and calls it the **unrestricted** model — but as a *ceiling*, not as a design alternative. The spec gates uncle validity: a block carrying an entry that fails the counting rules is **rejected** outright (*Block Header Validation* step 10), so a chain in which deep-fork orphans are referenced cannot exist. The unrestricted arm therefore measures the unreachable upper bound on what any counting rule could recover, and every "countable vs unrestricted" comparison below should be read as "what the deployed rule recovers, against the most any rule could". - **With uncle references, counting slots under the first-fork restriction (the *countable* model — the rules analysed here):** two changes. First, deduplication is by **slot**, not by block: a referenced uncle sharing a slot with a canonical block, or with another counted uncle, adds nothing. Second, only the **first block of a fork** — one whose parent lies on the referencing chain — is countable, which is what makes every reference verifiable from chain data alone (deeper fork blocks would need the fork branch's ledger state). References never affect a block's validity; they are *counting-only*, the soft-rule property the incentive analysis requires ([§6.7](#s6-7)–[§6.8](#s6-8)). The reference window is *derived* from a **window absorption parameter**, `w_u = W_abs/f` with `W_abs = 10` expected block-intervals and bound `W_abs ≤ 0.6·k` — the `W = 10/f` sizing rule of [§3.4](#s3-4). The price of chain-only verifiability is that deep-fork orphans are unrecoverable by construction; [§3.2](#s3-2) measures what that costs, and the answer is nothing in the design regime and a bounded ceiling under overload. Slot deduplication is what makes the recovered equilibrium exactly 1.0 rather than a ceiling above it. Counting uncle *blocks* instead double-counts same-slot co-winners (one canonical, one referenced orphan) and inflates the equilibrium by the fixed multi-winner factor `c(f) ≈ 1.017` — a genuine accuracy bias, not a simulation artefact. Every result in this report uses the slot count; where a figure or table compares the two referencing models, the unrestricted baseline is labelled as such ([§9](#s9)). @@ -508,6 +508,16 @@ Adding random per-delivery arrival noise changes nothing: consensus stays exact. Jitter feeds exactly the tip-level churn that [§3.1](#s3-1) showed never reaches the finalized density window. The reason is structural: the density window opens a full epoch back (`E = 10·⌊k/f⌋` slots) and *closes* `E − T = 4·⌊k/f⌋` slots before the chain is snapshotted for measurement — 259 200 slots ≈ 8 640 blocks ≈ 4k at k = 2160, f = 1/30, i.e. ~3k blocks deeper than `k`-finality — so every block has reached every node by measurement time regardless of jitter, and all nodes still compute an identical `D̂`. The "one global `D̂`" claim therefore holds off the `jitter = 0` axis. +**How much per-recipient variance would it take to matter?** The jitter above is a *robustness* probe; the same lever also answers a modelling question, because jitter is exactly the difference between this simulator's transport and one that draws an independent delay per (block, recipient). The cascade's variance is per **block** — every recipient is moved together by the final flood — while jitter is per **recipient**, and only the second kind desynchronises views and manufactures forks deeper than one block, which the counting rule cannot recover ([§2.1](#s2-1)). Swept at the deployment's operating point (`δ_max` = 4, exact oracle, both referencing models, 480 runs): + +| `jitter_mean` (slots) | 0 | 1 | 2 | 4 | 8 | +|---|---|---|---|---|---| +| `D̂/D` at `U = 1` | 0.9983 | 0.9992 | 0.9991 | 0.9971 | 0.9871 | +| orphans below their fork's first block | 0.25 % | 0.38 % | 0.53 % | 1.20 % | 3.30 % | +| fork rate | 0.266 | 0.272 | 0.275 | 0.310 | 0.366 | + +Accuracy is flat to ~2 slots and only then bends: at 8 slots the deep-fork share reaches 3.3 % and the estimate falls to 0.987. Blend's own cascade delivers about **0.5 slot** of per-recipient spread — the gossip flood over a degree-6 graph ([§2](#s2)) — so the design sits an order of magnitude inside the flat region. The sensitivity is nevertheless real, and quoting it is what makes the accuracy claims conditional on a *stated* transport property rather than an unstated one. Consensus is unaffected throughout (`range_ratio = 0`, `agreement_window = 1.000` in every one of the 480 runs), so this is an accuracy sensitivity only. + **Clock skew: bounded, but not identically zero.** A whole-timeline shift of a node's slot clock — unlike per-arrival jitter, it shifts the node's measurement-window bounds — was tested directly (its own generator, `scripts/clock_skew.py`, [§9](#s9)). A skew of up to ±20 slots moves each node's occupied-slot count by at most the one or two blocks in the shifted window edge, an inter-node spread of at most `max(1/m, 2·skew/T)`, where `m = f·T` is the window's occupied-slot count — the discrete one-block floor `1/m` dominates for `skew < 1/(2f)`, and the measured spread is indeed a flat `1/m` at every skew from 1 to 20 slots. At the production window (`T = 6·⌊k/f⌋ = 388 800` slots, so `m = f·T = 12 960`) both terms are `≈ 1×10⁻⁴` — bounded and negligible, but, unlike jitter's exact 0, not identically zero. So bounded clock skew is a small quantifiable consensus cost, not a break ([§8.3](#s8-3) item 6). @@ -840,7 +850,7 @@ Everything below is assembled from this report's own measurements; each row cite | 11 | reward schedule | **substantial `w_u` (Ethereum's GHOST-style uncle rewards are the precedent); `w_u + w_n < 1` strictly enforced** | farming a real leadership win into a self-uncle + self-nephew pays `w_u + w_n` against an honest block's 1 — the binding invariant is the *sum* (0.875 + 0.03125 = 0.906 is safe; 0.875 + 0.15 = 1.025 is not); `w_u` compensates honest orphans (recovery → 1.0) and makes hiding forfeit both rewards | [§6.7](#s6-7)(a), figs 14, 15 | | 12 | peering degree | **degree 4 is acceptable for smaller networks — up to N ≈ 2×10⁵ it keeps the load under 0.9 at the 8-s budget; use ≥ 6 beyond, and scale with the target size** | degree sets the gossip-diameter slope `ℓ_mean ≈ a_d·ln N`, and at a fixed blending budget that slope decides when `U = 1` runs out: at δ = 8 s, degree 4 exhausts U = 1 near **N* ≈ 8×10⁵** (its ladder curve already touches the recovery bar at N = 32 000), degree 6 holds to N ~ 4×10⁹, degree 8 to ≈ 6×10¹² — one degree step (4 → 6) buys ~5 000× more network at the same budget. Degree and the uncle cap are exchangeable defences against growth: densify the graph or add an uncle slot, whichever is cheaper. Plus the small-N benefit of better U = 0 recovery (tip agreement is comparable: deg 6 0.985–0.993 vs deg 4 0.988–0.994) | [§3.7](#s3-7), figs 23, 24; fig19; [Appendix C](#sC) (fig18) | | 13 | Blend delay budget | **choose `hops·δ_max` so `ρ ≲ 1`**; if `U ≥ ⌈ρ⌉` exceeds the uncle-slot budget, cut blending — no window fixes `U < ρ` | worked examples: 3 hops × 8 s at f = 1/30 → ρ ≈ 0.56, U = 2 recommended; the same Blend at f = 1/10 → ρ ≈ 1.7, U = 3 | [§4](#s4); [§6.3](#s6-3)(iii) | -| 14 | on-chain `f` precision | **carry the target rate `f` at 10⁻⁶ precision** (was 10⁻³) | removes the sole residual bias after slot-counting — the ≈ 1 % `f/f_p` over-estimate (a ~1 % win-probability under-delivery and slow canonical pace); one-constant change, no dynamics cost, no fork risk. **Exceeded** in this report's estimator, which uses exact `f` (residual 0), finer than the recommended 10⁻⁶ bump (`f_p = 0.033333`, residual < 10⁻⁵) | [§2.2](#s2-2), [Appendix A](#sA) | +| 14 | on-chain `f` precision | **carry the target rate `f` at 10⁻⁶ precision** (was 10⁻³) | removes the sole residual bias after slot-counting — the ≈ 1 % `f/f_p` over-estimate (a ~1 % win-probability under-delivery and slow canonical pace); one-constant change, no dynamics cost, no fork risk. **Now measured in the full dynamics, not just derived:** at the deployment's operating point the three arms read `D̂/D` = 0.99997 (exact `f`) / **1.01026** (the spec's 10⁻³) / 0.99990 (the recommended 10⁻⁶), against closed forms of 1.000 / 1.0101 / 1.00001. The deployed bias is real, is ~13× the first-fork cost, and is the largest single error in the deployed estimator | [§2.2](#s2-2), [Appendix A](#sA); `scripts/spec_point.py` | | 15 | reorg / confirmation depth | keep **`ρ < 1`** so fork-induced amplification stays below the effective-majority cliff; for soft settlement, wait `d` blocks per the [§6.10](#s6-10) tail (≈ 4/8/17 blocks for 99.9 % against 10/20/30 % stake at N = 1 000, degree 6 — deeper on larger or sparser networks, per the [§6.10](#s6-10) scale table) | reorg depth is bounded and shallow only while the honest fork rate is low; at 30 % stake past `ρ ≈ 1` it is unbounded ([§6.10](#s6-10), figs 27–28) | [§6.10](#s6-10) | @@ -871,7 +881,7 @@ Everything below is assembled from this report's own measurements; each row cite 12. **Network partition/heal — reasoned, not simulated ([§6.1](#s6-1)).** TSI reads its density window well past `k`-finality (it closes `4·⌊k/f⌋` slots before the estimate is used), so a partition that heals *within* the finality depth is invisible: all honest nodes reconverge on one canonical chain before the measurement window closes and compute an identical `D̂` — the same finality-window argument that makes jitter and clock skew harmless ([§6.1](#s6-1)). A partition lasting *longer* than finality is a consensus-layer safety failure (the chain forks irreparably) that TSI inherits but does not worsen — it adds no partition vulnerability of its own. A direct partition-and-heal simulation is left as future work. 13. **Reward payout mechanism unspecified ([§8.5](#s8-5)).** Rows 10–11 give the incentive constraints, not the encoding. Header-count indistinguishability (Blend) and the equal-share, content-independent voucher of the Anonymous Leaders Reward Protocol are the binding constraints on any `w_u`/`w_n` schedule, and reconciling them — including a payout path for a producer whose block is off-chain — is spec work left open here. 14. **Temporal resolution — `D̂` is stale by design ([§6.11](#s6-11)).** With `β = 1`, `D̂` tracks active stake at a one-epoch (≈ 7.5-day) lag, so it reflects participating stake from ~1–2 weeks ago and updates only weekly; a genuinely *sub-epoch* swing (daily or intra-week churn) falls below the estimator's Nyquist limit and is not tracked as a cycle at all. This is the correct trade for organic churn (row 5), but it bounds how fast TSI can follow real participation changes — flagged, not a defect. -15. **Correlated latency untested ([§2](#s2)).** Both link distributions draw each link *independently*, so the sweeps probe the latency marginal, not its spatial structure; geographically **correlated** latency (regional clustering that lets co-located nodes fork as a bloc) is not modelled. In the primary Blend regime the per-hop mixing delay dominates the geographic link term, so this is expected to stay second-order — but it is untested. +15. **Latency *structure* — now bounded in one direction, still untested in the other ([§2](#s2), [§6.1](#s6-1)).** The magnitude of per-recipient variance is measured: sweeping per-(block, node) jitter at the deployment's operating point leaves accuracy at 0.999 up to ~2 slots and degrades it to 0.987 by 8 slots, as unrecoverable deep-fork orphans rise 0.25 % → 3.3 %. Blend's cascade delivers roughly 0.5 slot of per-recipient spread, so the design sits an order of magnitude inside the safe range — but the sensitivity is real and the report now carries it rather than assuming it. What is still untested is the *spatial* structure: jitter is i.i.d. per (block, node), so geographically **correlated** latency, which would let co-located nodes fork as a bloc, is still not modelled. 16. **The deflation-optimal adversary — measured, and it is worse than the revenue-optimal one ([§6.6](#s6-6), `fig37`).** The ceilings elsewhere come from adversaries optimising revenue or reorg depth, so they bound `η` from above rather than the damage from below. Optimising the estimate directly: unconstrained the optimum is abstention (`D̂ = 1−α`, zero revenue, and [§6.4](#s6-4)-correct measurement rather than a mis-measurement); constrained to earn at least what honest mining pays (in absolute reward per unit time, not revenue share — the two differ by `density/D̂`), it reaches `D̂` = 0.65 at α = 0.4 against the revenue-optimal 0.81, and the exposure opens near α ≈ 0.37–0.38 rather than at the `⅓` threshold. What remains open is the *joint* problem — this frontier is single-coalition and, like everything in [§6.6](#s6-6), assumes `p_ref` is exogenous rather than something the attacker also manipulates (item 10). 17. **The `β_adv` axis is a nominal target, not a realised share ([§6.5](#s6-5)).** A random coalition is grown until its stake first reaches `β_adv`, so the last node added overshoots by its own size — under a Pareto tail, potentially a whale. The realised block share at a nominal `β_adv` = 0.1 is 0.137 ± 0.108, and that variance dominates the mean at small `β_adv`. The conclusions here are drawn from monotone trends across the axis and are unaffected, but a reader should not read `β_adv` as an achieved share; a whale coalition (filled largest-first to the target) lands on it precisely and is the cleaner instrument where the exact share matters. 18. **The collapsed-branch tail is uncharacterised ([§6.2](#s6-2)).** Static withholding reaches the collapsed branch once in 96 runs — one replicate of the concentrated (whale) 50 % coalition at `δ_max` = 8 s — and never at `δ_max` = 4 s. One reproducible event is enough to retire "not an observed dynamical trap" but far too thin to give the event a rate; its frequency against `β_adv`, `ρ` and coalition concentration is unmeasured. @@ -960,6 +970,9 @@ Sweep studies are committed configs, run with `make ` (writes a dated `run | **residual §6.5-scope variants** (whale coalitions, `jitter > 0`, very slow `β`, and the static-withholding load sweep that finds the collapsed-branch event) | `scripts/adversary_variants.py` (`adversary_selection`); `runs/adversary_variants_{whale,jitter,beta,withhold_load}.parquet` | [§6.2](#s6-2), [§6.5](#s6-5) | | **uncle cap under a private chain** (`U` × `W` × `α` against the SM1 engine adversary, plus the structural/queue decomposition of the honest orphans) | `scripts/selfish_uncle_margin.py` (`adversary_strategy: selfish`); `runs/selfish_uncle_margin{,_decomp}.parquet` | [§6.8](#s6-8) | | **profit/deflation frontier** (how far a *paid* adversary can deflate `D̂`; the deflation-optimal MDP) | `scripts/deflation_frontier.py` (`selfish_mdp.deflation_frontier`, `.deflation_optimal_stats`); fig37 | [§6.6](#s6-6) | +| **the deployed operating point** (`δ_max` = 4 from the spec's Blend profile; the three `f`-precision arms) | `scripts/spec_point.py` (`f_precision`); `runs/spec_point.parquet` | [§2.2](#s2-2), [§8.5](#s8-5), [Appendix A](#sA) | +| **per-recipient delay variance** (jitter 0–8 slots × `U`, both referencing models, exact oracle) | `scripts/spec_jitter.py`, `configs/spec-point-jitter.yaml`; `runs/spec_jitter.parquet` | [§6.1](#s6-1), [§8.3](#s8-3) item 15 | +| **the spec point at scale and by window** (`N` = 5000; `W_abs` sweep at `δ_max` = 4) | `configs/spec-point-n5000.yaml` (run twice, `--old`), `configs/spec-point-window.yaml` | [§3.4](#s3-4), [§8.5](#s8-5) | **Uncle-model convention.** The simulator's default is the **countable** model — first-fork candidates only, derived window `w_u = W/f`, occupied-slot exclusion, per-reference counting rules ([§2.1](#s2-1)). The **unrestricted** baseline is preserved in the code and selected with `--old` on `tsi-sweep`/`tsi-verify`. Its RNG key is byte-identical to the pre-restriction key, so `--old` **bit-reproduces the earlier runs**: a `rho-boundary` cell (δ_max = 8, U = 2, k = 256, N = 1 000) re-run under `--old` matches the committed `2026-07-27_195627_rho-boundary` parquet with `max |Δ| = 0` on every epoch and every metric. Studies in the table above that predate the countable default were produced under the unrestricted model and reproduce exactly under `--old`; the comparison rows quantify where the two models differ, and in the design regime (`ρ < 1`) no difference is resolvable, so those findings carry over unchanged. diff --git a/reports/tsi/handoff-fork-loss-results.md b/reports/tsi/handoff-fork-loss-results.md index f32d72a..0198565 100644 --- a/reports/tsi/handoff-fork-loss-results.md +++ b/reports/tsi/handoff-fork-loss-results.md @@ -6,10 +6,12 @@ ## Verdict -**The added section's headline is wrong by ~10×, and it is agonising over the wrong bias.** The residual it attributes to deep forks is 0.08 pp, not 1.4 %. Meanwhile the deployed estimator carries a **+1.0 % bias of the opposite sign** that the section does not mention — from the `f` rounding, not from forks. Rewriting the section around the second number rather than the first is the substantive change. +**The added section's headline is wrong by ~17×, and it is agonising over the wrong bias.** The residual it attributes to deep forks is 0.08 pp, not 1.4 %. Meanwhile the deployed estimator carries a **+1.0 % bias of the opposite sign** that the section does not mention — from the `f` rounding, not from forks. Rewriting the section around the second number rather than the first is the substantive change. The no-uncle loss is also **understated, not overstated**: 33 % at the operating point, not 18.7 %, and it deepens with network size. +**But the section's number is not an arithmetic error, and E5 identifies what it is.** It is what this model produces once per-recipient delay spread reaches ~8 slots — which is ~16× the gossip spread Blend's cascade actually delivers, because the cascade's variance is per *block* (shared by all recipients, harmless to the estimate) rather than per *recipient* (divergent, and the only kind that manufactures unrecoverable forks). That distinction is the whole disagreement and belongs in the rewritten section. + ## The requested table ``` @@ -27,7 +29,11 @@ E3 largest ρ with U=4 ≥ 0.98: > 1.87 (U=3 already suffices there) margin: E4 D̂/D vs W_abs {1,2,3,5,7,10,15,20}: 0.854 / 0.938 / 0.967 / 0.992 / 0.999 / 0.998 / 0.999 / 1.000 knee at W_abs ≈ 5; spec's W = 10 sits ~2× above it -E5 not yet run (the jitter diagnostic — see "What is still open") +E5 jitter_mean {0,1,2,4,8} slots, δ_max = 4, U=1, exact oracle, 12 reps: + D̂/D 0.9983 / 0.9992 / 0.9991 / 0.9971 / 0.9871 + depth≥2 orphans 0.25 % / 0.38 % / 0.53 % / 1.20 % / 3.30 % + consensus range_ratio = 0 and agreement = 1.000 in all 480 runs + → the section's 0.986 is reproducible, at ~8 slots of per-recipient variance E6 fixed_point off / 1e3 / 1e6: 0.99997 / 1.01026 / 0.99990 (predicted 1.000 / 1.0101 / 1.00001) ``` @@ -70,11 +76,27 @@ Three arms, identical but for the estimator's target-rate quantisation: The deployed estimator drives density to `f_p = 0.033` instead of `1/30`, so the chain reads **1.0 % high** — measured in the full per-node dynamics, matching `theory.fixed_point_bias` to within one standard error. It is ~13× the first-fork cost the section is concerned with, opposite in sign, and removed entirely by a one-constant change. -## What is still open +### E5 — the diagnostic: the section's number is reproducible, and that pins its hidden assumption -**E5 — the diagnostic — has not been run.** It is the only experiment that could invalidate the *report* rather than the section, and the primary hypothesis for the original discrepancy: the standalone simulation drew an independent delay per (block, recipient), while the simulator's Blend cascade floods from the last relay, so views stay synchronised. If real Blend carries more per-recipient variance, the design-regime numbers above are optimistic. The lever exists (`jitter_mean`) and this is also the report's open item 15. +This was the experiment that could have invalidated the *report*. It does not — but it does something more useful than refuting the section: it says exactly what the section's number assumes. -Until E5 runs, every number here carries the cascade model's assumption that nodes receive a block at nearly the same time. +Sweeping per-(block, node) arrival jitter on top of the cascade interpolates between the two delay models — 0 is the report's cascade, large values approach the standalone's independent-per-recipient draws: + +| `jitter_mean` (slots) | 0 | 1 | 2 | 4 | 8 | +|---|---|---|---|---|---| +| `D̂/D` at `U = 1` | 0.9983 | 0.9992 | 0.9991 | 0.9971 | **0.9871** | +| depth-≥2 orphans | 0.25 % | 0.38 % | 0.53 % | 1.20 % | **3.30 %** | +| fork rate | 0.266 | 0.272 | 0.275 | 0.310 | 0.366 | + +**Both branches of the handoff's pass/fail are true, at different jitter levels.** Up to ~2 slots of per-recipient variance the report is entirely robust: accuracy 0.999, deep orphans half a percent. But at 8 slots accuracy lands on **0.9871** — essentially the section's 0.986 — with deep orphans at 3.3 %. So the standalone result is not arithmetic error; it is what this model produces once per-recipient spread reaches roughly eight seconds. + +That converts the disagreement into a question with a checkable answer: **does Blend deliver ~8 slots of per-recipient spread?** Under the spec's own model it cannot come close. The blending delay is a fixed per-hop dwell and the cascade's final step is a network-wide gossip flood from the last relay, so what varies per *recipient* is only that flood — and the report measures the gossip spread at `ℓ_mean ≈ 0.5` slot over a degree-6 graph. Eight slots is ~16× that. Per-*block* delay variance, which the cascade does have and which is large, moves every recipient together and so creates no deep forks at all. + +The distinction is the whole disagreement, and it is worth stating in the section: **variance in *when a block becomes public* is harmless to the estimate; variance in *when each node sees it* is what manufactures unrecoverable forks.** The two are easy to conflate and the standalone model charged the second where Blend delivers the first. + +Consensus is untouched throughout — `range_ratio = 0` and `agreement_window = 1.000` in all 480 runs, at every jitter level — reconfirming §6.1's structural argument at the deployment's own operating point. + +**This also partly closes the report's open item 15** (correlated/heterogeneous latency untested): the report is robust to per-recipient variance up to ~2–4 slots and degrades measurably beyond, which is a bound it did not previously carry. What remains untested there is *spatially correlated* latency — nodes in a region straggling together — which jitter, being i.i.d. per (block, node), does not model. ## Notes on the guide (§4)