logos-blockchain/research
1
0
Fork 0
mirror of https://github.com/logos-blockchain/research.git synced 2026-08-10 21:23:12 +00:00
Code Issues Packages Projects Releases Wiki Activity

Fix adversary coalition sizing; add rival coalitions and a lead cap

Browse Source 
Three engine defects, found while building the multi-coalition study §6.9 flags
as open. The first is the serious one.

1. COALITION SIZING (engine._adversary_mask, `random` selection — the default).
   The coalition was the smallest random prefix whose stake reached the target.
   Under a Pareto tail a whale straddling the cut carries it far past its label:
   over 60 replicates, a nominal adversary_frac of 0.4 realised a MAJORITY in
   ~10% of them and reached 0.97, and 0.2 reached 0.90. The median was always
   on-label, which is why it hid — it distorts the tail, not the centre.

   Both other places in the code that size a set by stake had already rejected
   this rule: the `whale` arm uses fit-then-close, and _churn_inactive_mask
   documents the identical failure ("a 30% label realising up to ~53%"). The
   `random` arm kept it. Now fit-then-close in random order, and a draw where
   the tail leaves no subset near the label warns instead of silently running a
   different attacker. Realised stake is now within 0.1% of its label.

   Re-ran the load-bearing studies. §8.4 capstone (2 of 8 replicates
   contaminated, one a 61% majority): spec rule 0.994 -> 0.995, p_ref 0.936 ->
   0.937. The parent-anchored variant is far more sensitive — 0.974 -> 0.990,
   p_ref 0.875 -> 0.923 — because a tighter window and a larger suppressing
   coalition compound, so §8.4's argument for the W = 12 pairing rested on
   0.021 of cost that is really 0.006. The pairing itself survives re-measurement
   and is now better supported: p_ref reaches parity at W = 12 too, not at 15.
   §6.8's uncle-margin sweep and §6.5's random-arm variants are flagged as
   needing re-measurement (§8.3 item 20), not silently carried.

2. SM1 NEVER TERMINATED under a forking honest network. Textbook SM1 waits while
   it leads, assuming the lead returns to zero. But honest blocks fork against
   each other, so the public chain's HEIGHT grows at ~(1-a)*f*(1-fork) while a
   coalition sharing one view extends privately at the full a*f; past a fork rate
   of ~1 - a/(1-a) the private chain outruns the public one and `wait` never
   fires. The lead ran to thousands and every block was stranded at the epoch
   boundary — 98% of adversarial blocks at alpha=0.4, delta_max=8 — scoring an
   attacker that WON the race as having earned nothing. selfish_lead_cap
   (default: the finality depth k) publishes a lead that can no longer be caught.
   Inert unless `wait` stops terminating; pinned paired.

3. RIVAL COALITIONS (adversary_coalitions = K) for the §6.9 study: K private
   chains, each invisible to the others by the same arrival sentinel that hides
   them from honest nodes, so they orphan each other as well as the honest chain.
   Stake-balanced partition (LPT), K=1 bit-identical to the single-coalition path.

Also corrects §8.4's closing paragraph, which still quoted a pre-countable
D-hat/D of 1.001 and fork rates that contradicted its own table.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
Marcin Pawlowski 2026-08-10 12:21:00 +02:00
parent 74ee06047c
commit 516783146d
No known key found for this signature in database
8 changed files with 715 additions and 45 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

46
reports/tsi/README.md
View File

@ -819,13 +819,15 @@ The floor rises by one to two block-intervals, which is what the ~one-block-inte
| `W` | uncle-anchored: `D̂/D` / `p_ref` | parent-anchored: `D̂/D` / `p_ref` |
|---|---|---|
| 8 | 0.9852 / 0.917 | 0.9724 / 0.877 |
| **10** (today) | **0.9916 / 0.933** | 0.9851 / 0.913 |
| **12** (proposed pair) | 0.9965 / 0.942 | **0.9908 / 0.924** |
| 15 | 0.9921 / 0.912 | 0.9946 / 0.943 |
| 20 | 0.9971 / 0.945 | 0.9958 / 0.944 |
| 8 | 0.9893 / 0.925 | 0.9777 / 0.888 |
| **10** (today) | **0.9932 / 0.937** | 0.9885 / 0.923 |
| **12** (proposed pair) | 0.9969 / 0.942 | **0.9941 / 0.937** |
| 15 | 0.9965 / 0.944 | 0.9958 / 0.944 |
| 20 | 0.9973 / 0.946 | 0.9960 / 0.944 |
Parent-anchored at `W` = 12 lands within **+0.0008** of today's uncle-anchored `W` = 10 — parity inside one standard error — so the anchor's cost under a suppressing adversary is bought back exactly by the two extra block-intervals the longer parent gap consumes. `p_ref` recovers more slowly (0.924 against 0.933, matching only at `W` = 15), but it is far above the `≳ 0.3` the soft rule needs either way, so it does not bind the recommendation. **Adopt the anchor change and `W = 12` together, or neither**: at `W` = 10 the anchor costs 0.0065 under this adversary, which is the one regime where the change is not free.
Parent-anchored at `W` = 12 lands within **+0.0009** of today's uncle-anchored `W` = 10 (0.9941 ± 0.0010 against 0.9932 ± 0.0008) — parity inside one standard error — so the anchor's cost under a suppressing adversary is bought back by the two extra block-intervals the longer parent gap consumes. `p_ref` recovers with it, to 0.9372 against 0.9374: parity on the reference rate too, at the same `W`. At `W` = 10 the anchor costs 0.0046 under this adversary (0.9885 against 0.9932), which is the one regime where the change is not free — so **pair the anchor change with `W` = 12, or take neither**.
*These are the re-measured values.* The first edition of this sweep put the same pairing at +0.0008 but had `p_ref` still short at `W` = 12 (0.924 against 0.933) and only matching at `W` = 15, and put the `W` = 10 cost at 0.0065. Two of its twelve replicates were running a larger adversary than their label ([§9](#s9)); with that fixed the recommendation is unchanged and its justification is stronger, because the reference rate now reaches parity at the same `W` as the estimate rather than three intervals later.
**One further observation, and it sharpens the case.** At `δ_max` = 16 the *honest* uncle-anchored arm already reaches **315 slots** — past its own `w_u` = 300. Under the current rule `w_u` is not a bound on how far back validation reaches even with no adversary present; under the parent rule the same cell is 296. The parameter only becomes a state-retention bound once it is anchored to the parent.
@ -946,6 +948,7 @@ Everything below is assembled from this report's own measurements; each row cite
17. **The `β_adv` axis is a nominal target, not a realised share ([§6.5](#s6-5)).** A random coalition is grown until its stake first reaches `β_adv`, so the last node added overshoots by its own size — under a Pareto tail, potentially a whale. The realised block share at a nominal `β_adv` = 0.1 is 0.137 ± 0.108, and that variance dominates the mean at small `β_adv`. The conclusions here are drawn from monotone trends across the axis and are unaffected, but a reader should not read `β_adv` as an achieved share; a whale coalition (filled largest-first to the target) lands on it precisely and is the cleaner instrument where the exact share matters.
18. **The collapsed-branch tail is uncharacterised ([§6.2](#s6-2)).** Static withholding reaches the collapsed branch once in 96 runs — one replicate of the concentrated (whale) 50 % coalition at `δ_max` = 8 s — and never at `δ_max` = 4 s. One reproducible event is enough to retire "not an observed dynamical trap" but far too thin to give the event a rate; its frequency against `β_adv`, `ρ` and coalition concentration is unmeasured.
19. **Structurally uncompensatable orphans ([§6.7](#s6-7)(1), [§6.8](#s6-8)).** The uncle reward refunds `w_u` per *referenced* orphan, but 20–40 % of the honest blocks a private-chain attack destroys are unreferenceable by construction — buried behind the first block of a discarded run. Those producers cannot be compensated at any `w_u`, so the fairness guarantee inherits the first-fork ceiling exactly as the density repair does. No reward schedule fixes this; it is a property of the counting rule. Whether that is acceptable — honest miners bearing an uninsurable loss precisely when attacked — is a protocol-design question this report raises rather than settles.
20. **Two adversary studies await re-measurement ([§9](#s9)).** The coalition-sizing defect — a Pareto whale straddling the prefix cut could carry the coalition past its label, up to a majority in ~10 % of replicates — was fixed, and the load-bearing studies ([§8.4](#s8-4) capstone under both anchors, [§6.12](#s6-12)'s `W` pairing) were re-run. Two were not: [§6.8](#s6-8)'s selfish uncle-margin sweep (3/8, 2/8 and 1/8 replicates contaminated at `α` = 0.2/0.3/0.4) and [§6.5](#s6-5)'s `random`-selection scope variants (2/12 and 1/12). Both report *directional* results — whether a larger `U` clears a bar, whether a variant overturns a conclusion — and a too-large adversary does not flip either direction, which is why they are not withdrawn. But their **levels and thresholds should not be quoted as measured** until re-run. The `whale` arms, every honest sweep, and the [§6.2](#s6-2)/[§6.5](#s6-5) collapse event (a whale coalition) were never affected.
<a id="s8-4"></a>
### 8.4 Capstone: the whole recipe, in one run
@ -954,17 +957,19 @@ Every finding above comes from a different sweep. As a final check that the reco
| metric | honest | 30 % suppression adversary |
|---|---|---|
| accuracy `D̂/D` | 0.996 | 0.994 |
| accuracy `D̂/D` | 0.996 | 0.995 |
| consensus `range_ratio` / `agreement` | 0 / 1.000 | 0 / 1.000 |
| fork rate | 0.328 ± 0.014 | 0.350 ± 0.004 |
| max reorg depth | 3 | 3 |
| emergent `p_ref` | 0.944 | 0.936 |
| emergent `p_ref` | 0.944 | 0.937 |
*These numbers are the **countable** model's. An earlier edition of this table reported 1.001 / 0.998 with `p_ref` 1.000 / 0.990, measured before the countable redesign and never re-run against it — the capstone was missed when the other studies were brought over ([§9](#s9)). The corrected values are lower and the reference rate is materially lower (0.944 against 1.000), because the first-fork restriction makes some orphans unreferenceable that the earlier model counted. Nothing in the recommendation changes: accuracy still sits within the ±0.9 % per-epoch noise floor of [Appendix B](#sB) below the hard ceiling of 1, consensus is still exact, and `p_ref` remains far above the `≳ 0.3` the soft rule needs.*
*These numbers are the **countable** model's. An earlier edition of this table reported 1.001 / 0.998 with `p_ref` 1.000 / 0.990, measured before the countable redesign and never re-run against it — the capstone was missed when the other studies were brought over ([§9](#s9)). The corrected values are lower and the reference rate is materially lower (0.944 against 1.000), because the first-fork restriction makes some orphans unreferenceable that the earlier model counted. Nothing in the recommendation changes: accuracy still sits within the ±0.9 % per-epoch noise floor of [Appendix B](#sB) below the hard ceiling of 1, consensus is still exact, and `p_ref` remains far above the `≳ 0.3` the soft rule needs. The adversarial column was re-measured a second time after the coalition-sizing defect of [§9](#s9) was found — 2 of its 8 replicates had been running a larger adversary than their label, one of them a 61 % majority — which moved `D̂/D` from 0.994 to 0.995 and `p_ref` from 0.936 to 0.937. The spec's own recipe is barely sensitive to that defect. The [§6.12](#s6-12) variant below is not, which is the whole reason to state both.*
**Under the [§6.12](#s6-12) parent-anchored window** the same run gives 0.996 honest — no change — but **0.974 under the 30 % suppression adversary**, with `p_ref` falling to 0.875. The two effects compound: a tighter effective window leaves an orphan fewer chances to be referenced before its *parent* ages out, and an adversary suppressing references on its own blocks removes some of those chances. Neither isolated sweep shows this, because [§6.12](#s6-12)'s honest arms hold the adversary at zero and its adversarial arms use a slack cap. It is the reason the capstone exists, and it is the strongest argument for pairing the anchor change with `W = 12` rather than adopting it at `W = 10`.
**Under the [§6.12](#s6-12) parent-anchored window** the same run gives 0.996 honest — no change — and **0.990 under the 30 % suppression adversary**, with `p_ref` at 0.923. The two effects still compound in the direction predicted: a tighter effective window leaves an orphan fewer chances to be referenced before its *parent* ages out, and an adversary suppressing references on its own blocks removes some of those chances. Neither isolated sweep shows the interaction, because [§6.12](#s6-12)'s honest arms hold the adversary at zero and its adversarial arms use a slack cap; that is the reason the capstone exists.
The whole recipe coheres: accurate to the true stake (the ~1 % rounding removed), unanimous, shallow-forking, and its soft-inclusion premise (`p_ref` high) confirmed — and all of it survives a 30 % adversary actively trying to deflate the estimate — the honest and adversarial fork rates are statistically indistinguishable (0.342 ± 0.009 vs 0.343 ± 0.005), i.e. the suppression adversary does not measurably change the fork structure it exploits. (The honest `D̂/D` = 1.001 sits within the ±0.9 % per-epoch noise floor of [Appendix B](#sB), consistent with the equilibrium's hard ceiling of 1.) This also resolves the finding-2 ([§3.2](#s3-2)) vs row-4 tension in the recommendation's favour: at this operating point `⌈ρ⌉ = 1`, and the recommended `U = 2` (the `+1` margin) holds every metric at target with room to spare.
*This is the one place the [§9](#s9) coalition-sizing defect changed a conclusion rather than a digit.* An earlier edition read **0.974** here with `p_ref` 0.875, and drew from it the strongest argument for pairing the anchor change with `W = 12`. Two of the eight adversarial replicates were running a bigger adversary than their label — one of them a 61 % majority — and the parent anchor is far more sensitive to that than the spec's rule is, because a tighter window and a larger suppressing coalition compound. Re-measured, the anchor's cost under the same 30 % adversary is **0.006** (0.9953 → 0.9896), not the 0.021 that argument rested on. The `W = 12` pairing is a smaller correction than it appeared, and [§6.12](#s6-12) re-derives it from the re-measured sweep rather than from this figure.
The whole recipe coheres: accurate to the true stake (the ~1 % rounding removed), unanimous, shallow-forking, and its soft-inclusion premise (`p_ref` high) confirmed — and all of it survives a 30 % adversary actively trying to deflate the estimate — the honest and adversarial fork rates are close (0.328 ± 0.014 vs 0.350 ± 0.004), i.e. the suppression adversary barely changes the fork structure it exploits. (The honest `D̂/D` = 0.996 sits within the ±0.9 % per-epoch noise floor of [Appendix B](#sB), consistent with the equilibrium's hard ceiling of 1.) This also resolves the finding-2 ([§3.2](#s3-2)) vs row-4 tension in the recommendation's favour: at this operating point `⌈ρ⌉ = 1`, and the recommended `U = 2` (the `+1` margin) holds every metric at target with room to spare.
<a id="s8-5"></a>
### 8.5 Recommendation vs the current spec
@ -1058,6 +1063,25 @@ Sweep studies are committed configs, run with `make <name>` (writes a dated `run
Because the two models draw independent RNG streams, every countable-vs-unrestricted comparison is **unpaired**, and its resolution is set by the replicate spread rather than by the effect size. Each comparison sweep therefore includes a `U = 0` arm as a **negative control**: with no uncles the models are identical by construction, so the measured `U = 0` gap is a direct reading of the noise floor at that delay and replicate count. At `δ_max = 32` with 5 replicates that floor is ≈ 0.23 in `D̂/D` — larger than several real effects elsewhere in the grid — which is why [§3.2](#s3-2) reports a `t` statistic per cell and why the design regime is measured separately at 40 replicates ([§3.2a](#s3-2a)).
**Coalition sizing: a tail defect in the adversary construction, found and fixed.** The adversarial coalition is the node set whose stake sums to `adversary_frac`. It was built by a **cumulative-prefix cut** — the smallest random prefix whose stake reaches the target — and under a Pareto tail that lets a single whale straddling the cutoff carry the coalition far past its label. The *median* was always on-label, which is why it survived review; the tail was not. Over 60 replicates of the default draw, a nominal `adversary_frac` of 0.4 realised a **majority** coalition in ~10 % of replicates and reached 0.97, and a nominal 0.2 reached 0.90. A sweep labelled "30 % adversary" was therefore, in a minority of its replicates, simulating a substantially larger attacker — occasionally one holding more stake than everyone else combined.
Two parts of the code were never affected, which is what makes this a miss rather than an oversight. The **`whale` selection arm** already used the fit-then-close rule below — it exists because "taking whales until the cumulative sum first exceeds the target would overshoot badly under a heavy tail" — and the **churn amplitude** helper documents the identical failure ("a 30 % label realising up to ~53 %") and fixes it the same way. The `random` arm, the default, kept the rule the other two had each rejected.
The construction is now fit-then-close in random order: take every node that still fits under the target, then close the remaining gap with the smallest node that can, and only when closing lands nearer than stopping short. Realised stake now matches its label to better than 0.1 % on every reachable draw. A draw where the tail leaves no subset near the label — one holder above the target, so the coalition is either far under it or nearly the whole network — raises a `RuntimeWarning` naming the realised fraction instead of silently running a different attacker. Pinned by `tests/test_multi_coalition.py::test_realised_adversary_stake_stays_on_its_label`, which asserts on the tail and on the warning: a mean-only check passes against the defect.
**What it reached.** Only studies with an adversary and the default `random` selection; the `whale` arms, every honest sweep, and the [§6.2](#s6-2)/[§6.5](#s6-5) collapse event (a *whale* 50 % coalition) are untouched. Contaminated replicates per study, and what re-measuring moved:
| study | contaminated | effect |
|---|---|---|
| [§8.4](#s8-4) capstone, spec rule | 2 of 8 | `D̂/D` 0.994 → 0.995, `p_ref` 0.936 → 0.937 |
| [§8.4](#s8-4)/[§6.12](#s6-12) capstone, parent anchor | 2 of 8 | `D̂/D` 0.974 → **0.990**, `p_ref` 0.875 → **0.923** — the one changed conclusion |
| [§6.12](#s6-12) `W` pairing under suppression | 2 of 12 | re-measured; see [§6.12](#s6-12) |
| [§6.8](#s6-8) selfish uncle margin | 3/8, 2/8, 1/8 at `α` = 0.2/0.3/0.4 | not re-run — see below |
| [§6.5](#s6-5) scope variants, `random` arm | 2 of 12 at `β_adv` = 0.3, 1 of 12 at 0.5 | not re-run — see below |
| [§6.12](#s6-12) `uncle_parent_window` | 0 of 10 | clean as run |
The two not re-run are stated as outstanding rather than quietly carried: both report *directional* results (does a larger `U` clear a bar; does a variant overturn a conclusion) whose direction a too-large adversary does not flip, but neither their levels nor their thresholds should be quoted as measured until they are re-run. That is [§8.3](#s8-3) item 20.
The capstone was re-run under the countable model (and both window anchors) after it was found still carrying pre-redesign numbers; its parquet now carries an `anchor` column. All other studies were **re-run on 2026-07-23/24 with the corrected slot-counting mechanism** ([§2.1](#s2-1)) and the early-stop optimisation; the resilient batch is `scripts/run_all_reruns.sh` (per-step log in `runs/rerun_status.log`). Canonical run directories (latest): fullscale N=5000/10000 = `2026-07-24_094519_fullscale`; fullscale N=1000/2000 = `2026-07-23_171803_fullscale-small`; uncle-window = `2026-07-24_001456`; window-uncles = `2026-07-24_014240`; block-rate = `2026-07-24_043943`; blend-hops-delay = `2026-07-24_064052`; window-scale = `2026-07-24_085234`; latency-shape = `2026-07-24_090014_expdist`; stake-tail = `2026-07-24_090044_pareto133`; heterogeneous-start = `2026-07-24_090114_default`; N-scaling = the `nscaling-{a,b}` + `nscaling32-{a,b}` runs; adversary grids = `runs/adversary_grid/`; jitter = `runs/jitter_grid/`; bootstrap = `runs/bootstrap_fullscale/`; fluctuation = `runs/fluctuation_u0.parquet`; fork-rate = `runs/fork_rate_vs_delay.parquet`; ρ-boundary = `2026-07-27_195627_rho-boundary`. Referencing-model studies: countable-vs-unrestricted (paired, primary) = `2026-08-05_120804_cvo-paired-countable` / `2026-08-05_123403_cvo-paired-old`; the superseded unpaired pair = `2026-08-04_103536_cvo-countable` / `2026-08-04_104010_cvo-old`; window absorption = `2026-08-04_104633_absorption-window`; fine delay band (unpaired) = `2026-08-04_191441_fine-countable` / `2026-08-04_195353_fine-old`; fine delay band (paired, primary) = `2026-08-05_111109_fine-paired-countable` / `2026-08-05_113218_fine-paired-old`. Adversarial-countability studies: uncle-selection deviation = `2026-08-05_152617_uncle-selection`; countable-selfish MDP = `runs/countable_selfish.parquet` + `runs/countable_selfish_reorg.parquet`; §6.5 scope variants and the withhold-load sweep = `runs/adversary_variants_{whale,jitter,beta,withhold_load}.parquet`.
Figures are embedded from [`report-figures/`](report-figures) and versioned here alongside the report; the simulator folder does **not** commit its own generated figures, so the copies checked in here are the figures of record. Figures are in `report-figures/` (`fig1`–`fig29`, plus [Appendix B](#sB)'s `figB1`–`figB2`; numbering is generation order, not order of appearance). Committed generators: `fig1` (bootstrap, k=2160) by `scripts/bootstrap_dynamics.py`; `fig2`,`fig4`,`fig5`,`fig17`–`fig22` by `scripts/regenerate_extra_figs.py` from the latest sweeps (`fig3` hops×delay×U grid by `scripts/hops_delay_grid.py`, `fig6` (block-rate `U_min` grid + ρ-collapse) rendered ad hoc from `runs/2026-07-24_043943_block-rate` with no committed generator; `fig26` deficit-vs-ρ by `scripts/rho_boundary_analysis.py`) (fullscale-derived `fig17`–`fig20` pool both sizes in that run, N = 5 000 and N = 10 000 — the generators filter on stake_dist/topology/degree/init_dest only, never on `n_nodes`); `fig8`,`fig9` by `scripts/adversary_figs.py` from `runs/adversary_grid/`; `fig10`–`fig12` by `scripts/dynamic_withhold.py`; `fig13`–`fig15` by `scripts/selfish_mining.py`/`selfish_rewards.py`/`reward_mandate.py`; `fig16` by `scripts/stake_vs_delay.py`; `fig23`–`fig24` by `scripts/nscaling_analysis.py`; `fig25` by `scripts/window_scale_analysis.py`; `fig27`–`fig28` by `scripts/reorg_depth.py` (fork rates via `--measure`; private-chain model `src/tsi_sim/reorg.py`); `fig29` by `scripts/churn.py`; `figB1`–`figB2` by `scripts/appendix_fluct.py`. `fig7` (feedback fixed-point) is an analytic overlay; `fig30`–`fig33` (countable-vs-unrestricted accuracy, `q_u`-prediction check, recovery rate, absorption-window sweep) by `scripts/plot_countable_vs_old.py` from the `cvo-countable`/`cvo-old`/`absorption-window` runs; `fig34`–`fig35` (design-regime accuracy and the model gap with 95 % CIs) by `scripts/plot_fine_delay.py` from the `fine-countable`/`fine-old` runs; `fig36` (the countable recovery ceiling under a selfish adversary) by `scripts/countable_selfish.py`, which solves the MDP directly and writes `runs/countable_selfish.parquet` + `runs/countable_selfish_reorg.parquet` (no sweep input); `fig37` (the profit/deflation frontier) by `scripts/deflation_frontier.py` from `runs/deflation_frontier.parquet`, likewise solver-only; `fig38` (the reference-window anchor) by `scripts/uncle_parent_window.py`. Every figure type the per-node simulator generates appears in this report, and the fork-rate/reorg-depth study closes the previous reproducibility gap for the adversarial figures (`fig8`,`fig9` now have committed generators from `runs/adversary_grid/`).

36
tools/simulators/tsi/tsi-sim-pernode/configs/pref-window-anchor.yaml
View File

@ -1,16 +1,23 @@
# Does widening W restore what the parent anchor costs under a suppression adversary? (§6.8, §6.12)
# Which W pairs the parent-anchored window back to today's recipe? (§6.8, §6.12)
#
# The capstone found the one place the anchor change actually bites: honest recovery is
# unaffected, but under a 30 % uncle-suppression coalition the parent-anchored rule drops to
# D_hat = 0.974 with p_ref = 0.875, against 0.994 / 0.936 for the spec's rule. The two effects
# compound — a tighter effective window leaves an orphan fewer chances to be referenced before
# its PARENT ages out, and a suppressing adversary removes some of those chances by declining to
# reference from its own blocks.
# The capstone found the one place the anchor change bites: honest recovery is unaffected, but
# under a 30 % uncle-suppression coalition the parent-anchored rule costs accuracy, because a
# tighter effective window and a suppressing adversary compound — an orphan gets fewer chances to
# be referenced before its PARENT ages out, and the adversary removes some of the chances it has.
# The report recommends pairing the anchor change with a wider W; this sweep is what fixes the
# value, so it has to resolve differences of ~0.001 in D_hat/D.
#
# The report now recommends pairing the anchor change with W = 12 to restore the margin. That
# recommendation has been asserted from the shape of the mechanism and not yet measured, which is
# what this config is for: sweep W across both anchors, with and without the adversary, and read
# off whether a wider window buys back p_ref and the estimate.
# Two things make it able to. First, W is swept at EVERY integer from 8 to 15 rather than at 4
# scattered points, so the crossing is located rather than bracketed. Second, `paired_streams`
# puts the whole grid on COMMON RANDOM NUMBERS: SimConfig._base_key() excludes both
# uncle_window_anchor and window_absorption, so every cell of this sweep draws the same stake
# vector, the same peering graph and the same lottery outcomes at a given replicate, and the only
# thing that differs between two cells is the window rule itself. Differences are then paired,
# which removes the between-replicate variance that dominated the earlier unpaired version (SEM
# ~0.0009 on a difference of ~0.0009 — i.e. unresolved).
#
# Supersedes the earlier 5-point, 12-replicate unpaired version of this file. Its numbers were
# also taken before the §9 coalition-sizing fix, so they are superseded on both counts.
#
# Geometry is the capstone's, so the answer lands on the recommended configuration rather than a
# probe point: delta_max = 8 (rho ~ 0.47), U = 2 = ceil(rho)+1, degree 6, k = 2160.
@ -23,12 +30,12 @@ link_latency_mean: [0.5] # natural geographic transport (sub
link_latency_dist: [geo] # real-world geographic band mixture
blend_hops: [3] # the spec's Blend cascade length
blend_delay_max: [8.0] # the capstone's operating point, rho ~ 0.47
window_absorption: [8, 10, 12, 15, 20] # W in block-intervals; 10 = today, 12 = proposed
window_absorption: [8, 9, 10, 11, 12, 13, 14, 15] # every integer across the crossing
uncle_window_anchor: [uncle, parent] # spec rule vs the §6.12 proposal
max_uncles: [2] # the recommended cap U = ceil(rho)+1
uncle_strategy: [oldest] # spec Uncle Selection
init_dest: [common] # per-node initial D_est from agreement
replicates: 12
replicates: 32 # with pairing, enough to resolve ~0.0005
base: # per-run settings shared by every cell
k: 2160 # true security parameter
epochs: 20 # equilibrium within ~2 epochs; burn 50%
@ -36,4 +43,5 @@ base: # per-run settings shared by every
genesis_d_factor: 0.5 # start near true stake (cheap epoch 0)
early_stop: true
adversary_frac: 0.3 # the capstone's suppression coalition
adversary_strategy: suppress # references no uncles from its own blocks
adversary_strategy: suppress # references no uncles from its own blocks
paired_streams: true # COMMON RANDOM NUMBERS across anchor and W

217
tools/simulators/tsi/tsi-sim-pernode/scripts/multi_coalition.py Normal file
View File

@ -0,0 +1,217 @@
"""Do K rival selfish coalitions deflate D-hat further than one coalition of the same size? (fig39)
§6.9 settles the *withholding* commons exactly — deflation depends on the total abstaining stake
and not on how it is partitioned — and then flags the selfish case as conjectural, on two counts
taken from the literature:
(a) total orphaning, hence raw D-hat deflation, *can exceed* the single-coalition value, so the
§6.6 figure at alpha = 0.4 is not a multi-coalition upper bound; and
(b) several individually sub-threshold coalitions may be *jointly* profitable, i.e. the 1/3
threshold is not a per-coalition safety argument.
Both are now testable. The per-node engine runs one private chain per coalition, and a rival's
unreleased blocks are invisible to every other coalition by the same arrival sentinel that hides
them from honest nodes — so the chains race each other as well as the public chain, which is the
whole mechanism the conjecture rests on. `adversary_coalitions = K` splits a fixed adversarial
stake into K near-equal rivals (`engine._coalition_ids`), holding beta constant so K is the only
thing that moves.
Two arms:
* **accuracy** — D-hat/D, fork rate and p_ref against (beta, K), which answers (a) directly;
* **profitability** — each coalition's share of the canonical chain against its OWN stake, which
answers (b). A coalition profits when its canonical share exceeds its stake share; the joint
question is whether that holds for every one of K rivals that are each below 1/3.
Note the realised beta overshoots its target under a Pareto tail (`_adversary_mask` takes the
smallest node set reaching the target, and the last node can be a whale), so both arms record the
REALISED stake shares and the report quotes those, not the nominal knob.
Run: python scripts/multi_coalition.py (writes runs/multi_coalition{,_split}.parquet + fig39)
"""
from __future__ import annotations
from pathlib import Path
import numpy as np
import pandas as pd
from joblib import Parallel, delayed
from tsi_sim import lottery, topology
from tsi_sim.blocktree import build_tree_pernode
from tsi_sim.config import SimConfig
from tsi_sim.engine import _adversary_mask, _coalition_ids, run_trajectory
from tsi_sim.epoch import _canonical_producer_split
from tsi_sim.plotting import style
from tsi_sim.rng import rng_for, seedseq_for
from tsi_sim.stake import make_stake
HERE = Path(__file__).resolve().parent.parent
RUNS = HERE / "runs"
FIGS = HERE / "report-figures"
RUNS.mkdir(exist_ok=True)
FIGS.mkdir(exist_ok=True)
REPS = 10
N_JOBS = 10
BETAS = [0.2, 0.3, 0.4]
KS = [1, 2, 3, 4]
# Full scan / no prune: the selfish path forces both anyway (a private chain's release reorders
# the fork-choice frontier), stated here so the geometry is explicit rather than implied.
BASE = dict(n_nodes=600, stake_dist="pareto", topology="blend", degree=6,
link_latency_mean=0.5, link_latency_dist="geo", blend_hops=3, blend_delay_max=8.0,
max_uncles=2, uncle_strategy="oldest", window_absorption=10.0,
k=256, epochs=10, genesis_d_factor=0.5, early_stop=False,
adversary_strategy="selfish", prune_arrival=False, windowed_fork_choice=False)
def _accuracy(beta: float, K: int, rep: int) -> dict:
"""What K rivals at a fixed total stake do to the estimator."""
cfg = SimConfig(**BASE, adversary_frac=beta, adversary_coalitions=K, replicate=rep)
t = pd.DataFrame(run_trajectory(cfg))
t = t[t.epoch >= t.epoch.max() // 2]
adv, hon = t.adv_blocks.sum(), t.honest_blocks.sum()
return dict(beta=beta, K=K, rep=rep, mean_ratio=float(t.mean_ratio.mean()),
fork_rate=float(t.fork_rate.mean()), p_ref=float(t.p_ref.mean()),
mean_orphan_rate=float(t.mean_orphan_rate.mean()),
joint_share=float(adv / (adv + hon)) if adv + hon else 0.0)
def _profit(beta: float, K: int, rep: int, ratio: float) -> list[dict]:
"""Each coalition's canonical share against its own stake, on one rebuilt epoch.
``ratio`` is the accuracy arm's measured ``D-hat/D`` for this cell, and it matters: the
lottery is driven by the estimate, so rebuilding at the GENESIS d_est would produce blocks at
roughly twice the equilibrium rate and measure profitability in a regime the chain never
occupies. Seeding at ``ratio * D_true`` puts the rebuild at the operating point the
trajectory actually converges to under this attack.
"""
cfg = SimConfig(**{**BASE, "epochs": 2}, adversary_frac=beta,
adversary_coalitions=K, replicate=rep)
kids = seedseq_for(cfg).spawn(cfg.epochs + 3)
stake = make_stake(cfg, rng_for(cfg))
mask = _adversary_mask(cfg, stake)
ids = _coalition_ids(cfg, stake, mask)
pl = topology.build_path_latency(cfg, np.random.default_rng(kids[1]))
d = np.full(cfg.n_nodes, max(ratio, 0.05) * float(stake.sum()))
ws, wn = lottery.sample_wins(lottery.win_probs(stake, d, cfg.f), cfg.epoch_len,
np.random.default_rng(kids[3]))
slots, groups = lottery.group_by_slot(ws, wn)
tree, A = build_tree_pernode(slots, groups, pl, cfg, np.random.default_rng(kids[4]),
adversary_mask=mask, coalition_ids=ids)
total = float(stake.sum())
# Blocks that reached nobody: private chains still hidden when the epoch ended. The lead cap
# should keep this near zero — a large value means `wait` stopped terminating and the arm is
# measuring the epoch boundary rather than the attack, so it is reported, not assumed away.
adv_blk = np.array([bool(mask[int(tree.leader[b])]) for b in range(1, tree.n_blocks)])
never = (A[:, 1:] > cfg.epoch_len).all(axis=0)
stranded = float((adv_blk & never).sum() / max(int(adv_blk.sum()), 1))
# K == 1 has no id vector (the single-coalition path is left untouched), so synthesise one.
groups_of = [mask] if ids is None else [ids == g for g in range(K)]
out = []
for g, gmask in enumerate(groups_of):
won, hon = _canonical_producer_split(tree, A, gmask, cfg.period_T, cfg.epoch_len)
out.append(dict(beta=beta, K=K, rep=rep, coalition=g, stranded=stranded,
stake_share=float(stake[gmask].sum() / total),
canonical_share=float(won / (won + hon)) if won + hon else 0.0))
return out
def sweep() -> tuple[pd.DataFrame, pd.DataFrame]:
jobs = [(b, k, r) for b in BETAS for k in KS for r in range(REPS)]
par = Parallel(n_jobs=N_JOBS, backend="loky", inner_max_num_threads=1)
acc = pd.DataFrame(par(delayed(_accuracy)(*j) for j in jobs))
# the profit arm rebuilds at each cell's MEASURED operating point, so accuracy runs first
at = {(r.beta, r.K, r.rep): r.mean_ratio for r in acc.itertuples()}
spl = pd.DataFrame([r for rows in par(delayed(_profit)(b, k, rp, at[(b, k, rp)])
for b, k, rp in jobs) for r in rows])
acc.to_parquet(RUNS / "multi_coalition.parquet", index=False)
spl.to_parquet(RUNS / "multi_coalition_split.parquet", index=False)
return acc, spl
def report(acc: pd.DataFrame, spl: pd.DataFrame) -> None:
print("\n=== (a) does splitting the SAME stake deflate D-hat further? ===")
# MEDIAN, not mean: the deflation feedback of §6.2 is bistable, so a cell that drops a
# replicate onto the collapsed branch has a bimodal sample and its mean sits between two
# branches, describing neither. `low` counts those replicates so the tail stays visible.
print(f"{'beta':>6} {'K':>3} | {'median D_hat/D':>15} {'IQR':>15} {'low':>4} "
f"{'fork':>6} {'p_ref':>7} {'joint sh':>9}")
for b in BETAS:
for k in KS:
g = acc[(acc.beta == b) & (acc.K == k)]
med = g.mean_ratio.median()
lo, hi = g.mean_ratio.quantile(0.25), g.mean_ratio.quantile(0.75)
n_low = int((g.mean_ratio < med - 0.15).sum())
print(f"{b:6.2f} {k:3d} | {med:15.4f} {f'[{lo:.3f}, {hi:.3f}]':>15} {n_low:4d} "
f"{g.fork_rate.median():6.3f} {g.p_ref.median():7.3f} "
f"{g.joint_share.median():9.4f}")
one = acc[(acc.beta == b) & (acc.K == 1)].mean_ratio.median()
by_k = acc[acc.beta == b].groupby("K").mean_ratio.median()
worst, got = by_k.idxmin(), by_k.min()
verdict = ("SPLITTING DEFLATES FURTHER" if got < one - 0.005
else "the single coalition bounds it")
print(f" -> worst K = {worst} at {got:.4f} vs K=1 {one:.4f}: {verdict}\n")
print("=== (b) are individually sub-threshold coalitions each profitable? ===")
print(f"{'beta':>6} {'K':>3} | {'own stake':>10} {'canonical':>10} {'ratio':>8} "
f"{'stranded':>9} verdict")
for b in BETAS:
for k in KS:
g = spl[(spl.beta == b) & (spl.K == k)]
st, cs = g.stake_share.median(), g.canonical_share.median()
strand = g.stranded.median()
sub = "sub-1/3" if st < 1 / 3 else "over-1/3"
pays = "PAYS" if cs > st * 1.005 else "does not pay"
# A stranded fraction above ~0.2 means private chains were still hidden at the epoch
# boundary, so the cell measures the boundary and not the attack — flag, do not hide.
flag = " <-- BOUNDARY-DOMINATED" if strand > 0.2 else ""
print(f"{b:6.2f} {k:3d} | {st:10.4f} {cs:10.4f} {cs / st:8.3f} {strand:9.3f} "
f"{sub}, {pays}{flag}")
print()
def fig39(acc: pd.DataFrame, spl: pd.DataFrame) -> None:
import matplotlib.pyplot as plt
style.apply_style()
fig, axes = plt.subplots(1, 2, figsize=(9.6, 3.8))
ax = axes[0]
for i, b in enumerate(BETAS):
g = (acc[acc.beta == b].groupby("K").mean_ratio.agg(["mean", "sem"]).reset_index())
ax.errorbar(g.K, g["mean"], yerr=g["sem"], marker="o", ms=4, capsize=2,
color=style.OKABE_ITO[i + 1], label=rf"$\beta$ = {b:.1f}")
ax.axhline(1.0, color="0.5", lw=0.9, ls="--")
ax.set_xticks(KS)
ax.set_xlabel("number of rival coalitions $K$ (total stake held fixed)")
ax.set_ylabel(r"$\hat D / D^*$")
ax.set_title("Accuracy vs how the same stake is split")
ax.legend(fontsize=7)
ax = axes[1]
for i, b in enumerate(BETAS):
g = spl[spl.beta == b].groupby("K")[["stake_share", "canonical_share"]].mean()
ax.plot(g.index, g.canonical_share / g.stake_share, marker="o", ms=4,
color=style.OKABE_ITO[i + 1], label=rf"$\beta$ = {b:.1f}")
ax.axhline(1.0, color="0.5", lw=0.9, ls="--")
ax.set_xticks(KS)
ax.set_xlabel("number of rival coalitions $K$")
ax.set_ylabel("canonical share / own stake\n(per coalition; > 1 = selfish mining pays)")
ax.set_title("Profitability of each rival")
ax.legend(fontsize=7)
style.save(fig, FIGS / "fig39_multi_coalition", provenance="scripts/multi_coalition.py")
plt.close(fig)
def main() -> None:
print("=== K rival selfish coalitions at fixed total stake (§6.9) ===")
acc, spl = sweep()
report(acc, spl)
fig39(acc, spl)
print(f"wrote {RUNS}/multi_coalition{{,_split}}.parquet + fig39")
if __name__ == "__main__":
main()

86
tools/simulators/tsi/tsi-sim-pernode/src/tsi_sim/blocktree.py
View File

@ -198,16 +198,19 @@ class _SelfishCoalition:
bound on the damage.
"""
def __init__(self, coal_idx: np.ndarray, n_blocks: int, E: int) -> None:
def __init__(self, coal_idx: np.ndarray, n_blocks: int, E: int,
lead_cap: int = 1 << 60) -> None:
self.coal = coal_idx
self.priv: list[int] = [] # unreleased private blocks, oldest first
self.fork_height = 0 # height of the block the private chain forked from
self.unreleased = np.zeros(n_blocks, dtype=bool)
self.coal_arr = np.full(n_blocks, float(E) + 1.0) # when the coalition learns of a block
self.coal_arr[0] = 0.0
self.lead_cap = lead_cap # publish once ahead by this much (see `decide`)
self.n_released = 0 # blocks made public by a release
self.n_abandoned = 0 # private blocks the coalition gave up on
self.n_override = 0 # releases that orphaned >=1 honest block
self.n_cashed_in = 0 # releases forced by the lead cap
def note_block(self, b: int, arrival_at_coalition: float) -> None:
self.coal_arr[b] = arrival_at_coalition
@ -241,6 +244,27 @@ class _SelfishCoalition:
if h >= 1:
self.n_override += 1
return out
if a - h >= self.lead_cap: # cash in: the lead is already irreversible
# Textbook SM1 has no such branch — it waits while it leads, on the assumption that
# the lead returns to zero and the chain is cashed in then. That assumption fails
# here. Honest blocks FORK against each other, so the public chain's HEIGHT grows at
# roughly (1-alpha)*f*(1-fork_rate) while a coalition sharing one view extends its
# private chain at the full alpha*f; past a fork rate of about 1 - alpha/(1-alpha)
# the private chain simply outruns the public one and `wait` never terminates. The
# lead then runs to thousands of blocks and every one of them is stranded at the
# epoch boundary (measured: 98% of adversarial blocks at alpha = 0.4, delta_max = 8),
# which scores an attacker that WON the race as having earned nothing.
#
# A lead of `lead_cap` = k is past the finality depth: no honest chain can ever catch
# it, so holding it gains nothing and risks everything. Publishing is what a rational
# coalition does, so that is what this does.
out = self.priv
self.priv = []
self.n_released += len(out)
self.n_cashed_in += 1
if h >= 1:
self.n_override += 1
return out
return [] # wait
@ -251,12 +275,16 @@ def build_tree_pernode(
config,
rng: np.random.Generator,
adversary_mask: np.ndarray | None = None,
coalition_ids: np.ndarray | None = None,
):
"""Build the global block tree AND the per-node arrival matrix.
``adversary_mask[v] == True`` marks a node that suppresses uncle references in its own blocks
(references none), to deflate the TSI density count (grinding). ``None`` = fully honest.
``coalition_ids`` optionally splits those nodes into rival selfish coalitions (``-1`` honest,
``0..K-1`` membership — see ``engine._coalition_ids``). ``None`` is the single-coalition case.
Each winner builds on the best tip *in its own arrival-filtered view*; uncle refs are
baked at production from the producer's view. Returns ``(BlockTree, A)`` where
``A[i, b]`` is the slot block ``b`` becomes usable at node ``i``.
@ -370,7 +398,24 @@ def build_tree_pernode(
gb_id = 0
fp_idx = 1 # frontier pointer over fully-propagated blocks
coalition = _SelfishCoalition(np.nonzero(adversary_mask)[0], n_blocks, E) if selfish else None
# One private chain per coalition. With K > 1 the groups are RIVALS: a coalition never learns
# of another's unreleased blocks (they reach no node until released, so the arrival sentinel
# keeps them out of every rival's `public_height`), so the private chains race each other as
# well as the honest chain, and a release that overrides the public chain also buries whatever
# a rival was hiding behind it.
if not selfish:
coalitions: list[_SelfishCoalition] = []
else:
cap = getattr(config, "selfish_lead_cap", 0)
cap = config.k if cap == 0 else (1 << 60 if cap < 0 else cap)
members = ([np.nonzero(adversary_mask)[0]] if coalition_ids is None else
[np.nonzero(coalition_ids == g)[0]
for g in range(int(coalition_ids.max()) + 1)])
coalitions = [_SelfishCoalition(m, n_blocks, E, lead_cap=cap) for m in members]
# Which coalition a producer belongs to; with K == 1 every adversary is in group 0.
coal_of = (coalition_ids if coalition_ids is not None
else (np.where(adversary_mask, 0, -1).astype(np.int8)
if selfish and adversary_mask is not None else None))
nb = 1
for si in range(active_slots.shape[0]):
@ -435,23 +480,25 @@ def build_tree_pernode(
if hide:
A[:, b] = float(E) + 1.0 # withheld: never arrives -> orphan
withheld[b] = True
elif coalition is not None and adv:
# Private: visible to the whole coalition at once, invisible to everyone else
# until released. Kept off the honest side by the same sentinel `withhold` uses.
elif coalitions and adv:
# Private: visible to the producer's OWN coalition at once, invisible to everyone
# else — honest nodes and rival coalitions alike — until released. Kept off the
# public side by the same sentinel `withhold` uses.
own = coalitions[int(coal_of[v])]
A[:, b] = float(E) + 1.0
A[coalition.coal, b] = max(float(t), float(A[v, p_id]))
A[own.coal, b] = max(float(t), float(A[v, p_id]))
withheld[b] = True # flipped back on release
coalition.add_private(b, t, int(height[p_id]))
own.add_private(b, t, int(height[p_id]))
else:
np.maximum(col, A[:, p_id], out=col)
A[:, b] = col
A[v, b] = max(float(t), float(A[v, p_id])) # producer sees own block at its slot
if coalition is not None:
coalition.note_block(b, float(A[coalition.coal, b].min()))
for c in coalitions:
c.note_block(b, float(A[c.coal, b].min()))
nb += 1
if coalition is not None:
for rb in coalition.decide(t, height, nb):
for c in coalitions:
for rb in c.decide(t, height, nb):
# Release by DIRECT gossip from the producer, bypassing the Blend cascade: the
# adversary has no privacy budget to respect and wants the race won, so this is
# its fastest legal publication. Oldest first, so each block's parent arrival is
@ -462,18 +509,25 @@ def build_tree_pernode(
np.minimum(rel, A[:, rb], out=rel) # coalition already had it privately
A[:, rb] = rel
withheld[rb] = False
coalition.unreleased[rb] = False
c.unreleased[rb] = False
# A release is public, so every RIVAL coalition learns of it now — this is the
# channel through which one coalition's override orphans another's private chain.
for other in coalitions:
if other is not c:
other.note_block(rb, float(A[other.coal, rb].min()))
if coalition is not None and coalition.priv:
for c in coalitions:
if not c.priv:
continue
# Private blocks still hidden when the epoch ends are abandoned: the race they were held
# for is over, so they can never be cashed in. Hide them from the coalition too, or the
# canonical-tip search (which takes the best tip ANY node holds) would crown a chain no
# honest node ever saw and credit it phantom blocks.
stranded = np.array(coalition.priv, dtype=np.int64)
stranded = np.array(c.priv, dtype=np.int64)
A[:, stranded] = float(E) + 1.0
withheld[stranded] = True
coalition.n_abandoned += len(coalition.priv)
coalition.priv.clear()
c.n_abandoned += len(c.priv)
c.priv.clear()
tree = BlockTree(slot=slot, parent=parent, height=height, leader=leader, uncles=uncles)
return tree, A

30
tools/simulators/tsi/tsi-sim-pernode/src/tsi_sim/config.py
View File

@ -166,6 +166,26 @@ class SimConfig:
# lever, and adversary_selection controls WHICH nodes are taken at that fixed stake.
adversary_frac: float = 0.0
adversary_selection: AdversarySelection = "random"
# How many INDEPENDENT coalitions the adversarial stake is split into (§6.9). 1 = the single
# coalition every other study assumes. K > 1 partitions the same adversary_frac into K groups
# of near-equal stake, each running its OWN private chain with its own view: a rival's
# unreleased blocks are invisible (they reach no node until released), so the coalitions
# orphan each other as well as the honest chain. Only meaningful for
# adversary_strategy == "selfish" — under "suppress"/"withhold" the deflation depends on the
# summed stake alone, which is exactly the structure-independence result of §6.9, so the
# partition provably cannot matter there and the engine ignores it.
adversary_coalitions: int = 1
# Lead at which a selfish coalition PUBLISHES its private chain instead of extending it.
# 0 (default) -> the finality depth k: a lead past k can never be caught, so holding it
# gains nothing and a rational coalition cashes in.
# > 0 -> that lead, explicitly.
# < 0 -> uncapped: textbook SM1, which waits for as long as it leads.
# Textbook SM1 assumes the lead returns to zero and the chain is cashed in then. That fails
# under a forking honest network: the public chain's HEIGHT grows at ~(1-alpha)*f*(1-fork)
# while a coalition sharing one view extends privately at the full alpha*f, so past a fork
# rate of ~1 - alpha/(1-alpha) the private chain outruns the public one and `wait` never
# terminates. See _SelfishCoalition.decide.
selfish_lead_cap: int = 0
# Dynamic (withhold-then-rejoin) schedule for the withholding lever (§6.5). The coalition is
# FIXED (identity from adversary_frac); this only gates whether it withholds in a given epoch.
# adversary_period == 0 -> STATIC: the coalition attacks (withholds) every epoch (the §6.4
@ -334,6 +354,7 @@ class SimConfig:
"adversary_frac": 0.0 <= self.adversary_frac < 1.0,
"adversary_period": self.adversary_period >= 0,
"adversary_withhold_epochs": self.adversary_withhold_epochs >= 0,
"adversary_coalitions": self.adversary_coalitions >= 1,
}
bad = [name for name, ok in checks.items() if not ok]
if bad:
@ -425,6 +446,15 @@ class SimConfig:
# byte-identical key, so no committed result is reseeded by adding the knob.
if self.f_precision != 1_000_000:
base = base + (self.f_precision,)
# Likewise: K == 1 is the single-coalition assumption every prior study was run under, so
# a K == 1 key must stay byte-identical to the historical one.
if self.adversary_coalitions != 1:
base = base + (self.adversary_coalitions,)
# Same discipline again. NOTE this knob changes RESULTS at the default value (it fixes the
# runaway described on the field), so it is the one case where an unchanged key does not
# imply an unchanged number — see report §9 on which selfish runs were re-measured.
if self.selfish_lead_cap != 0:
base = base + (self.selfish_lead_cap,)
return base if self.uncle_window_anchor == "uncle" else base + (self.uncle_window_anchor,)
def seed_key(self) -> tuple:

81
tools/simulators/tsi/tsi-sim-pernode/src/tsi_sim/engine.py
View File

@ -2,6 +2,7 @@
from __future__ import annotations
import warnings
from typing import Any
import numpy as np
@ -77,12 +78,84 @@ def _adversary_mask(config: SimConfig, stake: np.ndarray) -> np.ndarray | None:
mask[order[chosen]] = True
return mask
cum_r = np.cumsum(stake[rand_order])
take = int(np.searchsorted(cum_r, target, side="left")) + 1 # smallest coalition >= target
mask[rand_order[:take]] = True
# Fit-then-close, the whale branch's rule walked in random order instead of descending: take
# every node that still FITS under the target, then close whatever gap remains with the
# smallest node that can.
#
# This replaces a plain cumulative-prefix cut ("smallest prefix reaching the target"), which
# let a single whale straddling the cutoff carry the coalition far past its label — the exact
# failure ``_churn_inactive_mask`` documents for the churn amplitude, never applied here.
# Measured over 60 replicates of the Pareto default, an adversary_frac of 0.4 realised a
# MAJORITY coalition in ~10 % of replicates and up to 0.97, and 0.2 reached 0.90, so the
# knob's tail was simulating a different attacker than the one it names. The median was always
# on-label, which is why it hid: it distorted the tail, not the centre.
#
# Nearest-fill alone (the churn rule) is not enough: it accepts an early whale whenever
# including it lands closer than stopping short, which is locally right and still leaves a
# ~3 % tail of majority coalitions. Fit-then-close cannot overshoot by more than one closing
# node. Candidate membership and RNG draw are unchanged.
st = stake[rand_order]
chosen = np.zeros(rand_order.size, dtype=bool)
cum = 0.0
for j, s in enumerate(st):
if cum + s <= target:
chosen[j] = True
cum += float(s)
if cum < target:
cand = np.nonzero(~chosen & (st >= target - cum))[0]
if cand.size:
# Smallest node that closes the gap — but only if closing beats stopping short. When
# one holder has most of the stake, nothing fits under the target and the only
# candidate is that whale, so "close the gap" would hand over the whole network to
# reach a 40 % label. Undershooting is then the nearer answer, and the warning below
# reports the miss rather than letting it pass as an on-label run.
j = cand[np.argmin(st[cand])]
if float(st[j]) - (target - cum) < (target - cum):
chosen[j] = True
cum += float(st[j])
mask[rand_order[chosen]] = True
got = cum / float(stake.sum())
if abs(got - config.adversary_frac) > 0.2 * config.adversary_frac:
warnings.warn(
f"adversary_frac={config.adversary_frac} is not reachable on this stake draw "
f"(n_nodes={config.n_nodes}, {config.stake_dist}, replicate={config.replicate}): "
f"realised {got:.3f}. The heavy tail leaves no subset near the label; treat this "
f"replicate's adversary as {got:.3f}, not {config.adversary_frac}.",
RuntimeWarning, stacklevel=2,
)
return mask
def _coalition_ids(config: SimConfig, stake: np.ndarray,
mask: np.ndarray | None) -> np.ndarray | None:
"""Split the adversarial set into ``adversary_coalitions`` rival groups of near-equal stake.
Returns a per-node ``int8`` label: ``-1`` for honest nodes, ``0..K-1`` for coalition members.
``None`` when there is nothing to split (honest run, or the default single coalition), which
keeps the single-coalition path exactly as it was.
The split is by **stake**, not by node count: under a Pareto tail an even split of members
would hand one group most of the adversarial power, and the question §6.9 asks is what happens
when the same total stake is held by ``K`` *equal* rivals. Longest-processing-time first —
walk the members in descending stake and put each into the lightest group so far — which for
this input lands every group within one small holder of ``beta/K``.
Deterministic given the mask and the stake vector: it draws no randomness, so adding the knob
reseeds nothing, and ``K == 1`` returns ``None`` rather than an all-zero label for the same
reason.
"""
if mask is None or config.adversary_coalitions <= 1:
return None
members = np.nonzero(mask)[0]
ids = np.full(config.n_nodes, -1, dtype=np.int8)
loads = np.zeros(config.adversary_coalitions, dtype=float)
for v in members[np.argsort(-stake[members], kind="stable")]:
g = int(np.argmin(loads))
ids[v] = g
loads[g] += float(stake[v])
return ids
def _initial_d_est(config: SimConfig, d_true: float, rng: np.random.Generator) -> np.ndarray:
"""Per-node initial estimate: common genesis, or heterogeneous around it."""
base = config.genesis_d_factor * d_true
@ -151,6 +224,7 @@ def run_trajectory(config: SimConfig) -> list[dict[str, Any]]:
path_latency = topology.build_path_latency(config, np.random.default_rng(children[1]))
d_est = _initial_d_est(config, d_true, np.random.default_rng(children[2]))
adv_mask = _adversary_mask(config, stake)
coal_ids = _coalition_ids(config, stake, adv_mask)
# exact stake fraction of the (integer-rounded) coalition, for the active-stake bookkeeping
coalition_frac = float(stake[adv_mask].sum() / d_true) if adv_mask is not None else 0.0
withholding = adv_mask is not None and config.adversary_strategy == "withhold"
@ -190,6 +264,7 @@ def run_trajectory(config: SimConfig) -> list[dict[str, Any]]:
active_stake_frac = float(stake[~inactive_mask].sum() / d_true) * active_stake_frac
er = simulate_epoch(config, stake, d_est, path_latency, children[epoch + 3],
adversary_mask=behaviour_mask, coalition_mask=adv_mask,
coalition_ids=coal_ids,
inactive_mask=inactive_mask)
row = divergence_row(config, epoch, d_est, er, d_true)
row["adversary_withholding"] = bool(withholding and attacks)

5
tools/simulators/tsi/tsi-sim-pernode/src/tsi_sim/epoch.py
View File

@ -85,10 +85,12 @@ def simulate_epoch(
adversary_mask: np.ndarray | None = None,
coalition_mask: np.ndarray | None = None,
inactive_mask: np.ndarray | None = None,
coalition_ids: np.ndarray | None = None,
) -> EpochResult:
"""``adversary_mask`` drives BEHAVIOUR this epoch (None == honest); ``coalition_mask`` is the
fixed coalition identity used only for reward attribution (so a rejoin epoch, mask None, still
credits the coalition's honestly-produced blocks). Defaults to ``adversary_mask`` when unset.
``coalition_ids`` splits the adversary into rival selfish groups (``engine._coalition_ids``).
"""
f, T, E = config.f, config.period_T, config.epoch_len
lottery_ss, aux_ss = epoch_ss.spawn(2)
@ -102,7 +104,8 @@ def simulate_epoch(
active_slots, groups = lottery.group_by_slot(winner_slots, winner_nodes)
tree, A = build_tree_pernode(active_slots, groups, path_latency, config, aux_rng,
adversary_mask=adversary_mask)
adversary_mask=adversary_mask,
coalition_ids=coalition_ids)
# measurement: each node's own canonical chain, deduped by tip + numba-accelerated
ms = measure(tree, A, active_slots, T, cutoff=E,

259
tools/simulators/tsi/tsi-sim-pernode/tests/test_multi_coalition.py Normal file
View File

@ -0,0 +1,259 @@
"""Rival selfish coalitions: the partition, the isolation invariant, and K == 1 compatibility."""
from __future__ import annotations
import dataclasses
import numpy as np
import pandas as pd
import pytest
from tsi_sim import lottery, topology
from tsi_sim.blocktree import build_tree_pernode
from tsi_sim.config import SimConfig
from tsi_sim.engine import _adversary_mask, _coalition_ids, run_trajectory
from tsi_sim.rng import rng_for, seedseq_for
from tsi_sim.stake import make_stake
KW = dict(n_nodes=250, stake_dist="pareto", topology="blend", degree=6, link_latency_mean=0.5,
link_latency_dist="geo", blend_hops=3, blend_delay_max=8.0, max_uncles=2,
window_absorption=10.0, k=64, epochs=3, genesis_d_factor=0.5, early_stop=False,
adversary_frac=0.4, adversary_strategy="selfish", replicate=0,
prune_arrival=False, windowed_fork_choice=False)
def _tree(cfg):
"""Rebuild one epoch's tree with the coalition split applied."""
kids = seedseq_for(cfg).spawn(cfg.epochs + 3)
stake = make_stake(cfg, rng_for(cfg))
mask = _adversary_mask(cfg, stake)
ids = _coalition_ids(cfg, stake, mask)
pl = topology.build_path_latency(cfg, np.random.default_rng(kids[1]))
d = np.full(cfg.n_nodes, cfg.genesis_d_factor * float(stake.sum()))
ws, wn = lottery.sample_wins(lottery.win_probs(stake, d, cfg.f), cfg.epoch_len,
np.random.default_rng(kids[3]))
slots, groups = lottery.group_by_slot(ws, wn)
tree, A = build_tree_pernode(slots, groups, pl, cfg, np.random.default_rng(kids[4]),
adversary_mask=mask, coalition_ids=ids)
return tree, A, stake, mask, ids
def test_k1_is_bit_identical_to_the_single_coalition_path():
"""Adding the knob must not disturb any committed selfish result.
Both the key (so the RNG stream is untouched) and the trajectory itself.
"""
a = SimConfig(**KW)
b = SimConfig(**KW, adversary_coalitions=1)
assert a.key() == b.key()
assert pd.DataFrame(run_trajectory(a)).equals(pd.DataFrame(run_trajectory(b)))
# ...and K > 1 DOES get its own stream, or the two would silently share one.
assert SimConfig(**KW, adversary_coalitions=2).key() != a.key()
def test_partition_splits_stake_and_is_as_even_as_the_tail_allows():
"""Equal-*stake* rivals is the question §6.9 asks; equal-member-count would not be it.
Perfect evenness is not achievable and not claimed: under a Pareto tail a single adversarial
whale can exceed beta/K on its own, and then no partition is even. What LPT guarantees is that
the spread is at most one member's stake — if bin A ends heaviest, the last item x placed in it
went there because A was then the lightest, so load(A) - x <= load(B) for every B at that time
and loads only grow. That bound is what this pins, together with the partition being exact.
"""
K = 3
cfg = SimConfig(**{**KW, "adversary_coalitions": K})
stake = make_stake(cfg, rng_for(cfg))
mask = _adversary_mask(cfg, stake)
ids = _coalition_ids(cfg, stake, mask)
assert ids is not None
shares = np.array([stake[ids == g].sum() for g in range(K)]) / stake.sum()
beta = stake[mask].sum() / stake.sum()
biggest = stake[mask].max() / stake.sum()
assert shares.max() - shares.min() <= biggest + 1e-12 # the LPT bound
# an exact partition of the coalition, and nothing else
assert shares.sum() == pytest.approx(beta)
assert set(np.nonzero(ids >= 0)[0]) == set(np.nonzero(mask)[0])
assert (ids[~mask] == -1).all()
def test_k1_returns_no_labels():
"""K == 1 must return None, not an all-zero vector: that is what keeps the old path exact."""
cfg = SimConfig(**KW)
stake = make_stake(cfg, rng_for(cfg))
assert _coalition_ids(cfg, stake, _adversary_mask(cfg, stake)) is None
assert _coalition_ids(SimConfig(**{**KW, "adversary_frac": 0.0, "adversary_coalitions": 3}),
stake, None) is None
def test_rivals_cannot_see_each_others_private_blocks():
"""THE isolation invariant the whole study rests on, checked on the mechanism itself.
A coalition's unreleased block must be visible to its own members and to nobody else — not
honest nodes, and (the part K > 1 adds) not a rival coalition either. If it leaked, the rivals
would effectively be one coalition sharing a view and the experiment would measure nothing.
This cannot be read off the FINAL arrival matrix: by the end every private block has either
been released (public arrivals) or stranded (hidden from its own coalition too), so the private
state is gone. So drive the two coalition objects directly, which is where the rule lives.
"""
from tsi_sim.blocktree import _SelfishCoalition
nb, E = 8, 1000
left = _SelfishCoalition(np.array([0, 1]), nb, E)
right = _SelfishCoalition(np.array([2, 3]), nb, E)
height = np.zeros(nb, dtype=np.int64)
# a public block both sides see
height[1] = 1
left.note_block(1, 5.0)
right.note_block(1, 5.0)
# `left` mines two private blocks on top of it; `right` mines one
height[2], height[3] = 2, 3
left.add_private(2, 10, 1)
left.add_private(3, 11, 2)
height[4] = 2
right.add_private(4, 12, 1)
# neither side's private chain enters the other's view of the PUBLIC chain
assert left.public_height(20, height, 5) == 1, "a rival's hidden block raised left's public h"
assert right.public_height(20, height, 5) == 1, "left's hidden blocks raised right's public h"
# ...and each side does see its own
assert left.unreleased[2] and left.unreleased[3] and not left.unreleased[4]
assert right.unreleased[4] and not right.unreleased[2]
# a rival's block is invisible in time, not just excluded by the unreleased flag: that is what
# makes the isolation hold for blocks the engine has not flagged on this side at all.
assert left.coal_arr[4] > E and right.coal_arr[2] > E
# once `right` releases, `left` learns of it (the engine notes releases to every rival) and it
# counts toward left's public height — the channel by which one override buries another's chain
left.note_block(4, 13.0)
assert left.public_height(20, height, 5) == 2
def test_splitting_changes_the_fork_structure():
"""K > 1 must actually behave differently, or the knob is inert.
Rivals cut each other's leads short, so the private chains are far shallower than the single
coalition's. This pins the direction, not a value.
"""
def max_fork_depth(cfg):
tree, A, *_ = _tree(cfg)
seen = A.min(axis=0) <= cfg.epoch_len
h = np.where(seen, tree.height, -1)
chain, c = set(), int(np.argmax(h))
while c > 0:
chain.add(c)
c = int(tree.parent[c])
best = 0
for b in range(1, tree.n_blocks):
if b in chain:
continue
d, x = 0, b
while x > 0 and x not in chain:
x = int(tree.parent[x])
d += 1
best = max(best, d)
return best
deep_one = max_fork_depth(SimConfig(**KW))
deep_many = max_fork_depth(SimConfig(**{**KW, "adversary_coalitions": 4}))
assert deep_many < deep_one, (
f"4 rivals held a deeper private chain ({deep_many}) than one coalition ({deep_one})")
def test_partition_is_ignored_where_it_provably_cannot_matter():
"""Under suppression the deflation depends on summed stake alone (§6.9, exact by construction),
so the partition must leave that path untouched.
Compared at the TREE, holding every RNG input fixed and varying only ``coalition_ids``. A
trajectory comparison would not show this: K enters ``key()``, so two trajectories draw
different stake, graph and lottery, and at test scale that difference swamps the effect.
"""
cfg = SimConfig(**{**KW, "adversary_strategy": "suppress", "adversary_coalitions": 4})
kids = seedseq_for(cfg).spawn(cfg.epochs + 3)
stake = make_stake(cfg, rng_for(cfg))
mask = _adversary_mask(cfg, stake)
ids = _coalition_ids(cfg, stake, mask)
assert ids is not None and (ids >= 0).any()
pl = topology.build_path_latency(cfg, np.random.default_rng(kids[1]))
d = np.full(cfg.n_nodes, cfg.genesis_d_factor * float(stake.sum()))
ws, wn = lottery.sample_wins(lottery.win_probs(stake, d, cfg.f), cfg.epoch_len,
np.random.default_rng(kids[3]))
slots, groups = lottery.group_by_slot(ws, wn)
def build(coalition_ids):
tree, A = build_tree_pernode(slots, groups, pl, cfg, np.random.default_rng(kids[4]),
adversary_mask=mask, coalition_ids=coalition_ids)
return tree, A
t0, a0 = build(None)
t1, a1 = build(ids)
assert np.array_equal(t0.parent, t1.parent) and np.array_equal(t0.leader, t1.leader)
assert np.array_equal(t0.slot, t1.slot) and t0.uncles == t1.uncles
assert np.array_equal(a0, a1)
def test_realised_adversary_stake_stays_on_its_label():
"""The coalition must hold the stake the knob names — including in the tail.
A plain cumulative-prefix cut let one whale straddling the cutoff carry the coalition far past
its label: ~10% of Pareto replicates at adversary_frac = 0.4 realised a MAJORITY, up to 0.97.
The median was always on-label, so only a tail check catches it. This is the regression guard.
"""
import warnings as _w
for nominal in (0.2, 0.3, 0.4):
got, unreachable = [], 0
for rep in range(40):
cfg = SimConfig(n_nodes=800, stake_dist="pareto",
adversary_frac=nominal, replicate=rep)
stake = make_stake(cfg, rng_for(cfg))
with _w.catch_warnings(record=True) as caught:
_w.simplefilter("always")
mask = _adversary_mask(cfg, stake)
unreachable += any("not reachable" in str(c.message) for c in caught)
got.append(float(stake[mask].sum() / stake.sum()))
got = np.array(got)
on_label = got[np.abs(got - nominal) <= 0.2 * nominal]
# every reachable draw lands essentially exactly on the label...
assert np.abs(on_label - nominal).max() < 0.01 * nominal, f"{nominal}: {on_label.max()}"
# ...nothing silently becomes a majority attacker...
assert (got > 0.5).sum() == 0, f"{nominal} produced a majority coalition: {got.max()}"
# ...and any draw that misses the label is loud about it, not silent.
assert unreachable == len(got) - len(on_label)
def test_lead_cap_only_changes_runaway_private_chains():
"""The cap must be inert unless `wait` has stopped terminating.
Held paired (same RNG, cap the only difference) so this is the mechanism and not a reseed.
"""
cfg_kw = {**KW, "n_nodes": 400, "adversary_frac": 0.3}
ref = SimConfig(**cfg_kw)
kids = seedseq_for(ref).spawn(ref.epochs + 3)
stake = make_stake(ref, rng_for(ref))
mask = _adversary_mask(ref, stake)
pl = topology.build_path_latency(ref, np.random.default_rng(kids[1]))
d = np.full(ref.n_nodes, ref.genesis_d_factor * float(stake.sum()))
ws, wn = lottery.sample_wins(lottery.win_probs(stake, d, ref.f), ref.epoch_len,
np.random.default_rng(kids[3]))
slots, groups = lottery.group_by_slot(ws, wn)
def build(cap):
cfg = SimConfig(**{**cfg_kw, "selfish_lead_cap": cap})
return build_tree_pernode(slots, groups, pl, cfg, np.random.default_rng(kids[4]),
adversary_mask=mask)
t_unc, a_unc = build(-1) # textbook SM1, no cap
t_cap, a_cap = build(0) # default: cap at k
stranded = (a_unc > ref.epoch_len).all(axis=0)[1:]
adv = np.array([mask[int(t_unc.leader[b])] for b in range(1, t_unc.n_blocks)])
frac_stranded = int((adv & stranded).sum()) / max(int(adv.sum()), 1)
if frac_stranded < 0.5:
# `wait` terminated normally, so the cap never fired and the trees must be identical
assert np.array_equal(a_unc, a_cap) and np.array_equal(t_unc.parent, t_cap.parent)
else:
# it ran away: the cap must have rescued the chain from the epoch boundary
cap_stranded = (a_cap > ref.epoch_len).all(axis=0)[1:]
assert int((adv & cap_stranded).sum()) < int((adv & stranded).sum())