diff --git a/reports/tsi/README.md b/reports/tsi/README.md index 477ffff..fd3aa09 100644 --- a/reports/tsi/README.md +++ b/reports/tsi/README.md @@ -72,7 +72,7 @@ 1. **All nodes always agree.** In every run, at every network size and block rate tested (1 000–10 000 nodes at the production security parameter; to 32 000 in the size-scaling study, [§3.7](#s3-7); blocks every 10–30 s), all nodes computed *exactly* the same estimate — even though they constantly disagree about the newest blocks. The reason is structural: TSI measures a window buried deep in the finalized past, where every honest node sees identical history ([§3.1](#s3-1)). Two caveats keep this honest: agreement is not accuracy — nodes can unanimously agree on a wrong value (see next point) — and agreement is inherited from the common genesis value, not rebuilt: TSI maintains agreement but cannot repair a divergence that somehow arose ([Appendix C](#sC)). 2. **Network delay makes the estimate too low; uncle references fix it.** When blocks race, the losers ("orphans") drop off the chain and out of the count, so the network under-counts its own stake — by roughly a quarter to a third under Blend (the mix-network transport) delays, deepening with network size. The error is strictly one-sided: the equilibrium estimate is **bounded above by the true stake** — `D̂/D` cannot exceed 1 (no tested cell sits above 1 beyond sampling noise, which is why the accuracy plots are capped at 1; [§2.2](#s2-2)), so delay can only deflate the estimate, never inflate it. Letting each block also reference recent orphans ("uncles") puts them back into the count — counting *slots*, so a slot with two winners still counts once — and a single uncle reference per block restores the estimate to **exactly the true stake** (the recommended cap adds a one-uncle margin, [§8](#s8)) (the only residual is an optional ~1 % from on-chain rounding of the target rate, [§2.2](#s2-2)/[Appendix A](#sA)) ([§3.2](#s3-2)). 3. **Two sizing rules cover the deployments tested.** How many uncle slots a block needs (`U`) is dictated by the **load** — the number of blocks the whole network produces during the time one block needs to reach everyone (written `ρ`; formally `ρ = f·D_vis`, [§3.3](#s3-3)). The load counts the concurrent blocks each new block must be able to reference, and one uncle slot drains one of them per block — so `U` must be at least the load rounded up, plus one spare. How far back a block may reach for an orphan (`W`) is dictated by block spacing, not delay: about ten block intervals. The formal equations, their calibration, and worked examples are in [§4](#s4). -4. **Cheating doesn't pay.** Hiding blocks to deflate the estimate costs the attacker more than anyone else, heals within one epoch, and — for a full withdrawal — the lower estimate is simply the *correct* count of who is still participating ([§6.4](#s6-4)–[§6.5](#s6-5)). The one genuinely profitable attack is classic selfish mining, which needs roughly one-third of all stake and afflicts every Nakamoto-style chain, not TSI specifically; the damage it does to the estimate is *partly* repaired by the same uncle mechanism — about half of it against an optimally-played attack, because the counting rule can reach only the first block of each discarded fork ([§6.6](#s6-6)). One qualification matters for how that threshold is read: it bounds what an attacker *earns*, not what the estimate *loses*. An attacker content to earn its stake share rather than the selfish premium can push the estimate substantially further down at no cost to itself — so above the one-third threshold the estimate is exposed to a wider set of strategies than the profit analysis alone would suggest ([§6.6](#s6-6), `fig37`). Rewarding uncles compensates the honest victims of delay and makes hiding strictly worse — but the reward must remain a bonus (a *soft* rule, never a validity requirement, which an attacker could turn into a tool for censoring or forking the chain), and the uncle + nephew rewards (the nephew reward pays the block that *includes* the reference) must together stay below one block reward, or deliberately orphaning one's own blocks becomes a business ([§6.7](#s6-7)–[§6.8](#s6-8)). +4. **Cheating doesn't pay.** Hiding blocks to deflate the estimate costs the attacker more than anyone else, heals within one epoch, and — for a full withdrawal — the lower estimate is simply the *correct* count of who is still participating ([§6.4](#s6-4)–[§6.5](#s6-5)). The one genuinely profitable attack is classic selfish mining, which needs roughly one-third of all stake and afflicts every Nakamoto-style chain, not TSI specifically; the damage it does to the estimate is *partly* repaired by the same uncle mechanism — about half of it against an optimally-played attack, because the counting rule can reach only the first block of each discarded fork ([§6.6](#s6-6)). One qualification matters for how that threshold is read: it bounds what an attacker *earns*, not what the estimate *loses*. An attacker content to earn what honest mining would pay it, rather than the selfish premium, can push the estimate substantially further down at no cost to itself — so somewhat above the one-third threshold the estimate is exposed to a wider set of strategies than the profit analysis alone would suggest ([§6.6](#s6-6), `fig37`). Rewarding uncles compensates the honest victims of delay and makes hiding strictly worse — but the reward must remain a bonus (a *soft* rule, never a validity requirement, which an attacker could turn into a tool for censoring or forking the chain), and the uncle + nephew rewards (the nephew reward pays the block that *includes* the reference) must together stay below one block reward, or deliberately orphaning one's own blocks becomes a business ([§6.7](#s6-7)–[§6.8](#s6-8)). 5. **Reorganisations stay shallow if the load stays low.** Forks are the cost centre: a deep reorg discards confirmed blocks. Fork depth grows with delay, and uncles keep it shallow (without them the deflated estimate over-produces blocks, and honest reorgs reach ~17 deep; with them, ~4). Against a private-chain attacker holding 10–30 % of stake trying to force the deepest reorg, keeping the load below one (few honest forks) keeps reorgs bounded and shallow (single digits to low tens of blocks even at 30 % stake) — but past that, at 30 % stake and heavy delay, fork-induced orphaning tips the attacker over an effective majority and reorgs become unbounded. The same "keep `ρ < 1`" rule that keeps the estimate accurate keeps reorgs shallow ([§6.10](#s6-10)). *Method note: results come from the per-node simulator at the production security parameter k = 2160, with the robustness and size-scaling studies at a reduced k = 256 where noted; the sizing rules are semi-empirical (their form follows from the fork/orphan mechanism, their constants are fitted to the sweeps) and are derived under honest, deterministic-delay conditions — noise, attacks, and incentives are treated in [§6](#s6).* @@ -621,15 +621,18 @@ So **SM1 is a faithful proxy for selfish-mining revenue but not for TSI's estima **The revenue-optimal adversary is not the estimator's worst case (`fig37`).** Every ceiling above comes from an adversary optimising something *other* than the estimate, so they bound `η` from above without bounding the damage from below. Optimising the estimator directly closes that gap — and needs no ratio transform, because each transition consumes exactly one block-finding event, so minimising `D̂/D = (canonical + p_ref·countable uncles)/events` is a plain average-reward MDP over the same transition table. -Asked without constraint the question degenerates, informatively: the optimum is **pure abstention** — publish nothing, adopt when overtaken — landing `D̂` on exactly `1 − α` with zero revenue. That is [§6.4](#s6-4)'s withholding, which is *correct* measurement rather than mis-measurement, since a coalition that publishes nothing genuinely is not participating. So the question that bites is what an adversary can do **while still being paid**. Sweeping the mixed objective `λ·(adversary blocks) − (contribution to D̂)` enumerates candidate policies; the line of interest is where the revenue *share* reaches `α`, i.e. where the attack costs nothing against simply mining honestly (below it, the attack is self-punishing griefing already bounded by [§6.5](#s6-5)): +Asked without constraint the question degenerates, informatively: the optimum is **pure abstention** — publish nothing, adopt when overtaken — landing `D̂` on exactly `1 − α` with zero revenue. That is [§6.4](#s6-4)'s withholding, which is *correct* measurement rather than mis-measurement, since a coalition that publishes nothing genuinely is not participating. So the question that bites is what an adversary can do **while still being paid**. Sweeping the mixed objective `λ·(adversary blocks) − (contribution to D̂)` enumerates candidate policies; the line of interest is where the attacker's **absolute reward per unit time** matches honest mining, so that the attack costs it nothing (below that line it is self-punishing griefing, already bounded by [§6.5](#s6-5)). -| `α` | revenue-optimal: share / `D̂` | best break-even-paid: share / `D̂` | extra deflation | +*Pay, not share.* The two differ, and not negligibly. A revenue *share* is measured against the canonical block rate, but what an attacker is actually paid is set by the estimator: TSI drives the **counted** density — canonical blocks plus recovered uncles — to `f` per slot, so a run yields `f/D̂` block-finding events per slot and a deflating attack makes the lottery easier for everyone. Pay per slot is therefore `adv_rate·f/D̂` against an honest miner's `α·f`, and the correction on the share ratio is `density/D̂`, below 1 whenever any uncle is recovered. Scoring the frontier on share alone credits the attacker with pay it never receives. + +| `α` | revenue-optimal `D̂` | best *paying* policy: pay / `D̂` | extra deflation | |---|---|---|---| -| 0.36 | 0.392 / 0.845 | 0.372 / **0.763** | 0.082 | -| 0.40 | 0.489 / 0.811 | 0.400 / **0.642** | 0.170 | -| 0.45 | 0.667 / 0.664 | 0.453 / **0.561** | 0.103 | +| 0.36 | 0.845 | — nothing in the sweep both pays and deflates | — | +| 0.38 | 0.830 | 1.02× / **0.749** | 0.081 | +| 0.40 | 0.811 | 1.00× / **0.649** | 0.163 | +| 0.45 | 0.664 | 1.18× / **0.568** | 0.096 | -At `α` = 0.4 an adversary content to earn exactly its stake share — no worse than honest mining — drives the estimate to **0.64** where the revenue-maximiser reaches only 0.81. The 17 points are bought with the selfish premium alone, which is a price an attacker whose goal is the estimator will happily pay. Below the `⅓` threshold nothing profitable deflates at all, so the exposure begins exactly where selfish mining does. (The sweep parameter is deliberately not monotone in revenue: selfish mining takes a larger share of a *smaller* pie, so raw block rate is maximised by honesty and large `λ` returns there. It enumerates policies; it is not a path.) +At `α` = 0.4 an adversary earning exactly what honest mining would pay it drives the estimate to **0.65**, where the revenue-maximiser reaches only 0.81. Those 16 points are bought with the selfish premium alone — a price an attacker whose goal is the estimator will happily pay. The exposure does *not* open at the `⅓` selfish threshold, though: at `α` = 0.36 no policy in the sweep both pays and deflates, so it begins nearer 0.37–0.38, above the point where selfish mining itself turns profitable. (The sweep parameter is deliberately not monotone in revenue: selfish mining takes a larger share of a *smaller* pie, so raw block rate is maximised by honesty and large `λ` returns there — it enumerates policies, it is not a path. The pay conversion neglects the multi-winner factor `c(f) ≈ 1.017` between events and occupied slots ([§2.1](#s2-1)), small against a 16-point effect.) ![Fig 37 — the profit/deflation frontier: adversaries that give up the selfish premium deflate the estimate substantially further (left, ★ = revenue-optimal), so the revenue-optimal ceiling is not the worst case and the break-even-paid curve sits far below it, approaching the unpaid abstention bound (right).](report-figures/fig37_deflation_frontier.png) @@ -849,7 +852,7 @@ Everything below is assembled from this report's own measurements; each row cite **Why soft, not hard, inclusion.** "Owed" cannot be proven — no node can prove which forks a producer saw. An objective slot-gap encoding lets an attacker withhold-then-late-release an uncle to retroactively invalidate honest blocks (censorship); a visibility encoding is subjective and forks consensus with no adversary at all ([§6.8](#s6-8)). The soft rule concedes only a small residual (quantified in [§8.3](#s8-3) item 3): a bounded near-threshold backfire that vanishes as the emergent `p_ref` rises, and a per-α premium that stays ≤ +0.014 even at full inclusion. -**What TSI defends, and what it measures.** Abstention-style withholding is strictly unprofitable (reward/stake 0.95 → 0.31 for duty 0.10 → 0.75 at β_adv = 0.3), bounded as griefing (deflation = β_adv·ψ), non-persistent, and does not stack across coalitions ([§6.5](#s6-5), [§6.9](#s6-9)) — and a full withdrawal is *correctly measured*, not mis-measured: `D̂ → (1−β_adv)` is the right answer for the participating stake ([§6.4](#s6-4)). The one profitable lever is classic selfish mining above `(1−γ)/(3−2γ)` — a general Nakamoto-PoS result, not TSI-specific; TSI's added exposure (estimator deflation to 0.70 at α = 0.4) is *partly* repaired by uncle-counting (row 9 — to 0.81, not the 0.94 an unrestricted count would reach, because the deployed rule can reference only the first block of each discarded fork), and the profit premium is blunted, not eliminated, by the soft-rewarded references (row 10). The *profit* frontier is exactly optimal selfish mining, and no compounding lever raises it ([§6.7](#s6-7)) — but the *estimator* frontier is not the same policy: an adversary willing to settle for break-even pay deflates `D̂` to 0.64 at α = 0.4 where the revenue-maximiser reaches 0.81 ([§6.6](#s6-6), `fig37`). Profitability bounds what an attacker gains, not what the estimate loses. +**What TSI defends, and what it measures.** Abstention-style withholding is strictly unprofitable (reward/stake 0.95 → 0.31 for duty 0.10 → 0.75 at β_adv = 0.3), bounded as griefing (deflation = β_adv·ψ), non-persistent, and does not stack across coalitions ([§6.5](#s6-5), [§6.9](#s6-9)) — and a full withdrawal is *correctly measured*, not mis-measured: `D̂ → (1−β_adv)` is the right answer for the participating stake ([§6.4](#s6-4)). The one profitable lever is classic selfish mining above `(1−γ)/(3−2γ)` — a general Nakamoto-PoS result, not TSI-specific; TSI's added exposure (estimator deflation to 0.70 at α = 0.4) is *partly* repaired by uncle-counting (row 9 — to 0.81, not the 0.94 an unrestricted count would reach, because the deployed rule can reference only the first block of each discarded fork), and the profit premium is blunted, not eliminated, by the soft-rewarded references (row 10). The *profit* frontier is exactly optimal selfish mining, and no compounding lever raises it ([§6.7](#s6-7)) — but the *estimator* frontier is not the same policy: an adversary willing to settle for the pay honest mining would give it deflates `D̂` to 0.65 at α = 0.4, where the revenue-maximiser reaches 0.81 ([§6.6](#s6-6), `fig37`). Profitability bounds what an attacker gains, not what the estimate loses. ### 8.3 Residual risks and open items @@ -869,7 +872,7 @@ Everything below is assembled from this report's own measurements; each row cite 13. **Reward payout mechanism unspecified ([§8.5](#s8-5)).** Rows 10–11 give the incentive constraints, not the encoding. Header-count indistinguishability (Blend) and the equal-share, content-independent voucher of the Anonymous Leaders Reward Protocol are the binding constraints on any `w_u`/`w_n` schedule, and reconciling them — including a payout path for a producer whose block is off-chain — is spec work left open here. 14. **Temporal resolution — `D̂` is stale by design ([§6.11](#s6-11)).** With `β = 1`, `D̂` tracks active stake at a one-epoch (≈ 7.5-day) lag, so it reflects participating stake from ~1–2 weeks ago and updates only weekly; a genuinely *sub-epoch* swing (daily or intra-week churn) falls below the estimator's Nyquist limit and is not tracked as a cycle at all. This is the correct trade for organic churn (row 5), but it bounds how fast TSI can follow real participation changes — flagged, not a defect. 15. **Correlated latency untested ([§2](#s2)).** Both link distributions draw each link *independently*, so the sweeps probe the latency marginal, not its spatial structure; geographically **correlated** latency (regional clustering that lets co-located nodes fork as a bloc) is not modelled. In the primary Blend regime the per-hop mixing delay dominates the geographic link term, so this is expected to stay second-order — but it is untested. -16. **The deflation-optimal adversary — measured, and it is worse than the revenue-optimal one ([§6.6](#s6-6), `fig37`).** The ceilings elsewhere come from adversaries optimising revenue or reorg depth, so they bound `η` from above rather than the damage from below. Optimising the estimate directly: unconstrained the optimum is abstention (`D̂ = 1−α`, zero revenue, and [§6.4](#s6-4)-correct measurement rather than a mis-measurement); constrained to earn at least its stake share, it reaches `D̂` = 0.64 at α = 0.4 against the revenue-optimal 0.81. What remains open is the *joint* problem — this frontier is single-coalition and, like everything in [§6.6](#s6-6), assumes `p_ref` is exogenous rather than something the attacker also manipulates (item 10). +16. **The deflation-optimal adversary — measured, and it is worse than the revenue-optimal one ([§6.6](#s6-6), `fig37`).** The ceilings elsewhere come from adversaries optimising revenue or reorg depth, so they bound `η` from above rather than the damage from below. Optimising the estimate directly: unconstrained the optimum is abstention (`D̂ = 1−α`, zero revenue, and [§6.4](#s6-4)-correct measurement rather than a mis-measurement); constrained to earn at least what honest mining pays (in absolute reward per unit time, not revenue share — the two differ by `density/D̂`), it reaches `D̂` = 0.65 at α = 0.4 against the revenue-optimal 0.81, and the exposure opens near α ≈ 0.37–0.38 rather than at the `⅓` threshold. What remains open is the *joint* problem — this frontier is single-coalition and, like everything in [§6.6](#s6-6), assumes `p_ref` is exogenous rather than something the attacker also manipulates (item 10). 17. **The `β_adv` axis is a nominal target, not a realised share ([§6.5](#s6-5)).** A random coalition is grown until its stake first reaches `β_adv`, so the last node added overshoots by its own size — under a Pareto tail, potentially a whale. The realised block share at a nominal `β_adv` = 0.1 is 0.137 ± 0.108, and that variance dominates the mean at small `β_adv`. The conclusions here are drawn from monotone trends across the axis and are unaffected, but a reader should not read `β_adv` as an achieved share; a whale coalition (filled largest-first to the target) lands on it precisely and is the cleaner instrument where the exact share matters. 18. **The collapsed-branch tail is uncharacterised ([§6.2](#s6-2)).** Static withholding reaches the collapsed branch once in 96 runs — one replicate of the concentrated (whale) 50 % coalition at `δ_max` = 8 s — and never at `δ_max` = 4 s. One reproducible event is enough to retire "not an observed dynamical trap" but far too thin to give the event a rate; its frequency against `β_adv`, `ρ` and coalition concentration is unmeasured. 19. **Structurally uncompensatable orphans ([§6.7](#s6-7)(1), [§6.8](#s6-8)).** The uncle reward refunds `w_u` per *referenced* orphan, but 20–40 % of the honest blocks a private-chain attack destroys are unreferenceable by construction — buried behind the first block of a discarded run. Those producers cannot be compensated at any `w_u`, so the fairness guarantee inherits the first-fork ceiling exactly as the density repair does. No reward schedule fixes this; it is a property of the counting rule. Whether that is acceptable — honest miners bearing an uninsurable loss precisely when attacked — is a protocol-design question this report raises rather than settles. diff --git a/reports/tsi/report-figures/fig37_deflation_frontier.png b/reports/tsi/report-figures/fig37_deflation_frontier.png index 2e58f7a..dc65f05 100644 Binary files a/reports/tsi/report-figures/fig37_deflation_frontier.png and b/reports/tsi/report-figures/fig37_deflation_frontier.png differ diff --git a/tools/simulators/tsi/tsi-sim-pernode/scripts/deflation_frontier.py b/tools/simulators/tsi/tsi-sim-pernode/scripts/deflation_frontier.py index d77f673..b1013a4 100644 --- a/tools/simulators/tsi/tsi-sim-pernode/scripts/deflation_frontier.py +++ b/tools/simulators/tsi/tsi-sim-pernode/scripts/deflation_frontier.py @@ -68,10 +68,18 @@ def sweep() -> pd.DataFrame: def best_profitable(df: pd.DataFrame) -> pd.DataFrame: - """Per alpha, the frontier point minimising D_hat among those paying at least stake share.""" + """Per alpha, the frontier point minimising D_hat among those that actually pay. + + "Pays" means ABSOLUTE reward per unit time at least matching honest mining + (``pay_vs_honest >= 1``), not merely a revenue *share* at least matching stake. The two + differ by ``density / dhat``: the share's denominator is the canonical block rate, but the + pay rate is set by the estimator, which holds counted density at ``f`` per slot. Using the + share alone credits the attacker with pay it does not receive — at alpha = 0.36 it marks + points as break-even that are ~5 % short. + """ out = [] for _a, g in df.groupby("alpha"): - paid = g[g.reward_per_stake >= 1.0 - 1e-9] + paid = g[g.pay_vs_honest >= 1.0 - 1e-9] if paid.empty: continue out.append(paid.loc[paid.dhat_countable.idxmin()]) @@ -81,18 +89,20 @@ def best_profitable(df: pd.DataFrame) -> pd.DataFrame: def report(df: pd.DataFrame) -> None: best = best_profitable(df) print(f"{'alpha':>6} {'rev-opt rev':>12} {'rev-opt D':>10} | " - f"{'best paid rev':>14} {'xstake':>7} {'D':>7} {'extra deflation':>16}") + f"{'best paid rev':>14} {'pay/hon':>8} {'D':>7} {'extra deflation':>16}") for a, g in df.groupby("alpha"): ro_r, ro_d = g.revenue_optimal.iloc[0], g.dhat_revenue_optimal.iloc[0] b = best[best.alpha == a] if b.empty: - print(f"{a:6.3f} {ro_r:12.4f} {ro_d:10.4f} | (nothing profitable deflates)") + print(f"{a:6.3f} {ro_r:12.4f} {ro_d:10.4f} | (no policy in the sweep both pays " + f"and deflates)") continue b = b.iloc[0] print(f"{a:6.3f} {ro_r:12.4f} {ro_d:10.4f} | {b.revenue:14.4f} " - f"{b.reward_per_stake:7.3f} {b.dhat_countable:7.4f} {ro_d - b.dhat_countable:+16.4f}") - print("\n(extra deflation > 0 means the profitable-but-deflating policy beats the " - "revenue-optimal one at damaging the estimate, at no cost versus honest mining)") + f"{b.pay_vs_honest:8.3f} {b.dhat_countable:7.4f} {ro_d - b.dhat_countable:+16.4f}") + print("\n(extra deflation > 0 means the paying-but-deflating policy beats the revenue-optimal " + "one at damaging the estimate, at no cost versus honest mining; 'pay/hon' is ABSOLUTE " + "reward per unit time, not revenue share)") def fig37(df: pd.DataFrame) -> None: diff --git a/tools/simulators/tsi/tsi-sim-pernode/src/tsi_sim/selfish_mdp.py b/tools/simulators/tsi/tsi-sim-pernode/src/tsi_sim/selfish_mdp.py index f02530b..9a56133 100644 --- a/tools/simulators/tsi/tsi-sim-pernode/src/tsi_sim/selfish_mdp.py +++ b/tools/simulators/tsi/tsi-sim-pernode/src/tsi_sim/selfish_mdp.py @@ -364,12 +364,24 @@ def deflation_frontier(alpha: float, gamma: float, lam: float, p_ref: float = 1. canonical = rates["adv_rate"] + rates["hon_rate"] blocks, runs = rates["orphan_hon_blocks"], rates["orphan_hon_runs"] revenue = (rates["adv_rate"] / canonical) if canonical > 0 else 0.0 + dhat = canonical + float(p_ref) * runs + + # Absolute pay, not just share. Revenue *share* understates what an attack costs the attacker, + # because the share's denominator is the canonical rate while the pay rate is set by the + # ESTIMATOR: TSI drives counted density (canonical + recovered uncles) to `f` per slot, so a + # run yields `f / dhat` block-finding events per slot and a deflating attack makes the lottery + # easier. Pay per slot is therefore `adv_rate · f / dhat` against an honest miner's `alpha · f`, + # so the correction on the share ratio is `density / dhat` — below 1 whenever uncles are + # recovered. (Events vs occupied slots differ by the multi-winner factor c(f) ~ 1.017 at + # f = 1/30, §2.1, which is neglected here and is small against the effect.) + pay_vs_honest = (rates["adv_rate"] / (alpha * dhat)) if alpha and dhat > 0 else 0.0 return dict( alpha=alpha, gamma=gamma, lam=lam, revenue=revenue, reward_per_stake=(revenue / alpha) if alpha else 0.0, + pay_vs_honest=pay_vs_honest, density_fraction=canonical, - dhat_countable=canonical + float(p_ref) * runs, + dhat_countable=dhat, dhat_unrestricted=canonical + float(p_ref) * blocks, eta=(runs / blocks) if blocks > 0 else 1.0, orphan_hon_blocks=blocks,