mirror of
https://github.com/logos-blockchain/research.git
synced 2026-08-07 11:43:20 +00:00
Item 16: the revenue-optimal adversary is not the estimator's worst case
Both eta ceilings in sec 6.6 come from adversaries optimising something else (revenue, reorg depth), so they bound eta from above without bounding the damage from below. Optimising the estimate directly needs no ratio transform: each transition consumes exactly one block-finding event, so minimising D-hat = (canonical + p_ref * countable uncles)/events is a plain average-reward MDP over the transition table that already carries the orphan counts. One value-iteration pass, no bisection. Unconstrained, the answer degenerates -- and usefully. The optimum is pure abstention: publish nothing, adopt when overtaken, D-hat = 1 - alpha exactly, revenue zero. That is sec 6.4's withholding, which the report already shows is CORRECT measurement rather than mis-measurement, so the unconstrained objective asks the wrong question. The constrained one bites. Sweeping lam * (adversary blocks) - (contribution to D-hat) enumerates policies; the line of interest is where revenue SHARE reaches alpha, i.e. where attacking costs nothing versus mining honestly. At alpha=0.4 such a policy drives D-hat to 0.642 where the revenue-maximiser reaches 0.811 -- 17 points of extra deflation bought with the selfish premium alone. At 0.36 and 0.45 the gaps are 0.082 and 0.103. Below the 1/3 threshold nothing profitable deflates, so the exposure starts exactly where selfish mining does. This revises two claims that were about revenue but read as though they were about the adversary in general: sec 6.7's "the adversary frontier is exactly optimal selfish mining; no compounding lever remains" and sec 8.2's echo of it. Both now say the PROFIT frontier is bounded and the estimator frontier is not the same policy. Note the sweep parameter is deliberately non-monotone in revenue -- selfish mining takes a bigger share of a smaller pie, so raw block rate is maximised by honesty and large lam returns there; it enumerates policies rather than tracing a path. Also closes a fairness loop these findings opened. Sec 6.7(1) credits uncle rewards with compensating orphaned honest producers, computed on the SM1 race where every orphan is a first-fork block. Under a private chain 20-40% of the honest blocks destroyed are unreferenceable by construction, so those producers are uncompensatable at ANY w_u -- not underpaid because p_ref is low, but unreachable because no valid block may name them. The fairness guarantee inherits the same first-fork ceiling as the density repair. Logged as item 19, flagged as a protocol-design question rather than something a schedule fixes. _solve_mdp is refactored into _solve_reward/_greedy_policy/_stationary/ _policy_rates so both objectives share one implementation; optimal_policy_stats reproduces its committed figures exactly (eta 0.4413, D-hat 0.9447/0.8111 at alpha=0.4). 251 tests pass. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Marcin Pawlowski
parent
5a8cc4437a
commit
8cc682aa49
@ -72,7 +72,7 @@
|
||||
1. **All nodes always agree.** In every run, at every network size and block rate tested (1 000–10 000 nodes at the production security parameter; to 32 000 in the size-scaling study, [§3.7](#s3-7); blocks every 10–30 s), all nodes computed *exactly* the same estimate — even though they constantly disagree about the newest blocks. The reason is structural: TSI measures a window buried deep in the finalized past, where every honest node sees identical history ([§3.1](#s3-1)). Two caveats keep this honest: agreement is not accuracy — nodes can unanimously agree on a wrong value (see next point) — and agreement is inherited from the common genesis value, not rebuilt: TSI maintains agreement but cannot repair a divergence that somehow arose ([Appendix C](#sC)).
|
||||
2. **Network delay makes the estimate too low; uncle references fix it.** When blocks race, the losers ("orphans") drop off the chain and out of the count, so the network under-counts its own stake — by roughly a quarter to a third under Blend (the mix-network transport) delays, deepening with network size. The error is strictly one-sided: the equilibrium estimate is **bounded above by the true stake** — `D̂/D` cannot exceed 1 (no tested cell sits above 1 beyond sampling noise, which is why the accuracy plots are capped at 1; [§2.2](#s2-2)), so delay can only deflate the estimate, never inflate it. Letting each block also reference recent orphans ("uncles") puts them back into the count — counting *slots*, so a slot with two winners still counts once — and a single uncle reference per block restores the estimate to **exactly the true stake** (the recommended cap adds a one-uncle margin, [§8](#s8)) (the only residual is an optional ~1 % from on-chain rounding of the target rate, [§2.2](#s2-2)/[Appendix A](#sA)) ([§3.2](#s3-2)).
|
||||
3. **Two sizing rules cover the deployments tested.** How many uncle slots a block needs (`U`) is dictated by the **load** — the number of blocks the whole network produces during the time one block needs to reach everyone (written `ρ`; formally `ρ = f·D_vis`, [§3.3](#s3-3)). The load counts the concurrent blocks each new block must be able to reference, and one uncle slot drains one of them per block — so `U` must be at least the load rounded up, plus one spare. How far back a block may reach for an orphan (`W`) is dictated by block spacing, not delay: about ten block intervals. The formal equations, their calibration, and worked examples are in [§4](#s4).
|
||||
4. **Cheating doesn't pay.** Hiding blocks to deflate the estimate costs the attacker more than anyone else, heals within one epoch, and — for a full withdrawal — the lower estimate is simply the *correct* count of who is still participating ([§6.4](#s6-4)–[§6.5](#s6-5)). The one genuinely profitable attack is classic selfish mining, which needs roughly one-third of all stake and afflicts every Nakamoto-style chain, not TSI specifically; the damage it does to the estimate is *partly* repaired by the same uncle mechanism — about half of it against an optimally-played attack, because the counting rule can reach only the first block of each discarded fork ([§6.6](#s6-6)). Rewarding uncles compensates the honest victims of delay and makes hiding strictly worse — but the reward must remain a bonus (a *soft* rule, never a validity requirement, which an attacker could turn into a tool for censoring or forking the chain), and the uncle + nephew rewards (the nephew reward pays the block that *includes* the reference) must together stay below one block reward, or deliberately orphaning one's own blocks becomes a business ([§6.7](#s6-7)–[§6.8](#s6-8)).
|
||||
4. **Cheating doesn't pay.** Hiding blocks to deflate the estimate costs the attacker more than anyone else, heals within one epoch, and — for a full withdrawal — the lower estimate is simply the *correct* count of who is still participating ([§6.4](#s6-4)–[§6.5](#s6-5)). The one genuinely profitable attack is classic selfish mining, which needs roughly one-third of all stake and afflicts every Nakamoto-style chain, not TSI specifically; the damage it does to the estimate is *partly* repaired by the same uncle mechanism — about half of it against an optimally-played attack, because the counting rule can reach only the first block of each discarded fork ([§6.6](#s6-6)). One qualification matters for how that threshold is read: it bounds what an attacker *earns*, not what the estimate *loses*. An attacker content to earn its stake share rather than the selfish premium can push the estimate substantially further down at no cost to itself — so above the one-third threshold the estimate is exposed to a wider set of strategies than the profit analysis alone would suggest ([§6.6](#s6-6), `fig37`). Rewarding uncles compensates the honest victims of delay and makes hiding strictly worse — but the reward must remain a bonus (a *soft* rule, never a validity requirement, which an attacker could turn into a tool for censoring or forking the chain), and the uncle + nephew rewards (the nephew reward pays the block that *includes* the reference) must together stay below one block reward, or deliberately orphaning one's own blocks becomes a business ([§6.7](#s6-7)–[§6.8](#s6-8)).
|
||||
5. **Reorganisations stay shallow if the load stays low.** Forks are the cost centre: a deep reorg discards confirmed blocks. Fork depth grows with delay, and uncles keep it shallow (without them the deflated estimate over-produces blocks, and honest reorgs reach ~17 deep; with them, ~4). Against a private-chain attacker holding 10–30 % of stake trying to force the deepest reorg, keeping the load below one (few honest forks) keeps reorgs bounded and shallow (single digits to low tens of blocks even at 30 % stake) — but past that, at 30 % stake and heavy delay, fork-induced orphaning tips the attacker over an effective majority and reorgs become unbounded. The same "keep `ρ < 1`" rule that keeps the estimate accurate keeps reorgs shallow ([§6.10](#s6-10)).
|
||||
|
||||
*Method note: results come from the per-node simulator at the production security parameter k = 2160, with the robustness and size-scaling studies at a reduced k = 256 where noted; the sizing rules are semi-empirical (their form follows from the fork/orphan mechanism, their constants are fitted to the sweeps) and are derived under honest, deterministic-delay conditions — noise, attacks, and incentives are treated in [§6](#s6).*
|
||||
@ -619,6 +619,20 @@ The curve is **within `0.09` for `β = 0.5` and `β = 1.0`** at every duty — p
|
||||
|
||||
So **SM1 is a faithful proxy for selfish-mining revenue but not for TSI's estimator damage**: at `α = 0.4, γ = 0` its revenue share (0.484) sits within 1 % of the optimum's (0.489), while its recoverable-orphan fraction (1.00) is more than double (0.44). The repair is real but roughly **half** of what the unrestricted model shows — `0.81`, not `0.94`, at `α = 0.4` — and it moves the *wrong way* with `α`: the unrestricted recovery improves as the attacker grows (0.926 → 0.955) because a stronger attacker orphans a larger share of a shrinking honest chain, while the countable recovery **degrades** (0.856 → 0.619) because a stronger attacker waits longer and buries more blocks per fork. (Read the `α ≥ 0.42` cells as indicative: the orphan *shape* converges more slowly in the MDP's lead cap than the revenue does — `η` at `α = 0.45` still moves by ~3 % relative between cap 48 and 64, against ~0.1 % at `α = 0.4`.) The depth-maximising adversary of [§6.10](#s6-10) agrees on the magnitude from the other direction — `η = 0.52` at `α = 0.30` with the measured honest fork rate `o = 0.35` (`reorg.countable_recovery_from_depths`). Neither figure is a worst case: both adversaries optimise something else (revenue, depth), and one optimising *deflation* directly would trade revenue for fork depth. The attacker's own channel is capped by the same rule — its abandoned secret chain is also one chain, so it can self-uncle only the first block of it (`η_adv = 0.750` at `α = 0.4`), which narrows the [§6.7](#s6-7)(a) farming payoff below what the block count suggests.
|
||||
|
||||
**The revenue-optimal adversary is not the estimator's worst case (`fig37`).** Every ceiling above comes from an adversary optimising something *other* than the estimate, so they bound `η` from above without bounding the damage from below. Optimising the estimator directly closes that gap — and needs no ratio transform, because each transition consumes exactly one block-finding event, so minimising `D̂/D = (canonical + p_ref·countable uncles)/events` is a plain average-reward MDP over the same transition table.
|
||||
|
||||
Asked without constraint the question degenerates, informatively: the optimum is **pure abstention** — publish nothing, adopt when overtaken — landing `D̂` on exactly `1 − α` with zero revenue. That is [§6.4](#s6-4)'s withholding, which is *correct* measurement rather than mis-measurement, since a coalition that publishes nothing genuinely is not participating. So the question that bites is what an adversary can do **while still being paid**. Sweeping the mixed objective `λ·(adversary blocks) − (contribution to D̂)` enumerates candidate policies; the line of interest is where the revenue *share* reaches `α`, i.e. where the attack costs nothing against simply mining honestly (below it, the attack is self-punishing griefing already bounded by [§6.5](#s6-5)):
|
||||
|
||||
| `α` | revenue-optimal: share / `D̂` | best break-even-paid: share / `D̂` | extra deflation |
|
||||
|---|---|---|---|
|
||||
| 0.36 | 0.392 / 0.845 | 0.372 / **0.763** | 0.082 |
|
||||
| 0.40 | 0.489 / 0.811 | 0.400 / **0.642** | 0.170 |
|
||||
| 0.45 | 0.667 / 0.664 | 0.453 / **0.561** | 0.103 |
|
||||
|
||||
At `α` = 0.4 an adversary content to earn exactly its stake share — no worse than honest mining — drives the estimate to **0.64** where the revenue-maximiser reaches only 0.81. The 17 points are bought with the selfish premium alone, which is a price an attacker whose goal is the estimator will happily pay. Below the `⅓` threshold nothing profitable deflates at all, so the exposure begins exactly where selfish mining does. (The sweep parameter is deliberately not monotone in revenue: selfish mining takes a larger share of a *smaller* pie, so raw block rate is maximised by honesty and large `λ` returns there. It enumerates policies; it is not a path.)
|
||||
|
||||
|
||||
|
||||
**The per-node engine reaches the same ceiling independently.** Everything above is a stationary MDP: one global chain, no network, recovery inferred from the policy's orphan structure. Running a private-chain coalition inside the per-node simulator instead — real propagation delays, per-node views, an actual uncle queue draining against `U` and `W` — reproduces the mechanism from the other side: of the honest blocks a private-chain attack destroys, only 59–81 % are countable at all (against 97 % of ordinary latency orphans), and the shortfall is precisely the blocks buried behind the first of each discarded run ([§6.8](#s6-8)). Two models that share no code and make different idealisations agree on both the direction and the rough size of the loss, which is the strongest evidence available here that the first-fork ceiling is a property of the counting rule rather than of either model.
|
||||
|
||||
|
||||
@ -632,7 +646,9 @@ So **SM1 is a faithful proxy for selfish-mining revenue but not for TSI's estima
|
||||
|
||||
Everything above measures *blocks*. Whether an attack **pays** depends on the reward schedule, which is a design choice. We add one as a configurable option (`selfish.RewardParams`): a block reward (`= 1`), an **uncle reward** `w_u` (a reward weight — distinct from the uncle *window*, which this report calls `W`) paid to the *producer* of an orphaned block that is referenced as an uncle, and a **nephew reward** `w_n` paid to the block that *references* it. The model has five knobs in total: `w_u`, `w_n`, the honest-orphan reference rate `p_ref` (defined below), the attacker's self-uncling rate `p_ref_adv` (= 1 for a rational attacker), and `adv_nephew` — the fraction of referenced honest orphans whose *nephew* reward the attacker's own canonical blocks capture (0 without a forced-inclusion mandate, since honest blocks do the referencing). Three effects, all adversarially analysed.
|
||||
|
||||
**(1) It compensates orphaned honest producers (fairness).** A block-only schedule pays an orphaned honest block **nothing** — the same "off-chain, uncounted" fate that drives both the [§3.2](#s3-2) latency under-count and the losses a selfish attacker inflicts. Define the **reference rate** `p_ref` = the probability that an orphaned honest block is referenced as an uncle by some canonical block within `W`. An uncle reward `w_u` refunds the loss in proportion to `p_ref`: honest recovery of *mined* value rises with `w_u` toward `1.0` (e.g. in the α = 0.4 selfish race, `0.61 → 0.80 → 1.00` for `w_u = 0 → 0.5 → 1` at `p_ref = 1`; `fig15`, right, plots recovery vs `p_ref`). The same references that restore the TSI *density* ([§6.6](#s6-6), `D̂`) restore the *reward*. **This benefit is robust** because a genuine latency orphan is *published* and straddled by *honest* canonical blocks that reference it, so `p_ref ≈ 1` here — unlike the selfish case in (3), where the attacker owns the straddling blocks.
|
||||
**(1) It compensates orphaned honest producers (fairness).** A block-only schedule pays an orphaned honest block **nothing** — the same "off-chain, uncounted" fate that drives both the [§3.2](#s3-2) latency under-count and the losses a selfish attacker inflicts. Define the **reference rate** `p_ref` = the probability that an orphaned honest block is referenced as an uncle by some canonical block within `W`. An uncle reward `w_u` refunds the loss in proportion to `p_ref`: honest recovery of *mined* value rises with `w_u` toward `1.0` (e.g. in the α = 0.4 selfish race, `0.61 → 0.80 → 1.00` for `w_u = 0 → 0.5 → 1` at `p_ref = 1`; `fig15`, right, plots recovery vs `p_ref`). The same references that restore the TSI *density* ([§6.6](#s6-6), `D̂`) restore the *reward*. **This benefit is robust for the case it is designed for** — a genuine latency orphan is *published*, straddled by *honest* canonical blocks that reference it, and (crucially) the **first block of its fork**, so `p_ref ≈ 1` and the refund is available in full.
|
||||
|
||||
**But it cannot reach the orphans a private chain makes, at any `w_u`.** The recovery figures above are computed on the SM1 race, where the attacker acts the moment the honest branch reaches length 1 and so every orphan it makes is a first-fork block. A private-chain override discards a *chain*, and only its first block is referenceable ([§2.1](#s2-1)) — measured in the per-node engine, 20–40 % of the honest blocks such an attack destroys can never be referenced by anyone ([§6.8](#s6-8)). An uncle reward refunds `w_u` per *referenced* orphan, so those producers are uncompensatable by construction: not underpaid because `p_ref` is low and could be raised, but unreachable because no valid block may name them. The fairness ceiling is therefore the same first-fork ceiling that caps the density repair, and it binds hardest on exactly the honest miners an attacker targets. This is a limit of the *counting rule*, not of the reward schedule, so no choice of `w_u`/`w_n` addresses it; what a schedule can still do is avoid *compounding* it, which is what the [§6.7](#s6-7)(a) `w_u + w_n < 1` invariant does.
|
||||
|
||||
**(2) It disincentivises hiding.** A withheld block (the [§6.4](#s6-4)/[§6.5](#s6-5) lever) *never propagates*, so it can never be referenced as an uncle — the withholder forfeits **both** the block reward **and** any uncle reward. Publishing promptly is therefore strictly dominant for an honest-but-unlucky miner, and abstention is made even costlier than the already-negative [§6.5](#s6-5) result. The reward schedule turns "publish everything you win, immediately" into the incentive-compatible action.
|
||||
|
||||
@ -650,7 +666,7 @@ The backfire is *unconditional* below the `⅓` threshold and holds at higher `
|
||||
|
||||
*(c) The reference game is not fixed by a plain `w_n`.* Because the selfish attacker suppresses references on its *own* override blocks, a nephew reward it *also collects* does **not** by itself force inclusion — in the model, raising `w_n` slightly *lowers* `α*` (the stake threshold above which selfish mining is profitable — i.e. it marginally *helps* the attacker, which pockets a canonical-share slice of the nephew pot). What *does* keep the effective reference rate high is that the orphans are **published**, so the many *other* honest blocks within `W` reference them anyway ([§6.8](#s6-8)). A hard "invalid block if it omits an uncle" mandate would force this outright but **cannot be encoded fork-safely** (no node can prove what forks a producer saw — [§6.8](#s6-8)), so we use a **soft (reward-weighted) inclusion rule** instead. *(Consolidated protocol recommendation: end of [§6.8](#s6-8).)*
|
||||
|
||||
**Joint / adaptive adversary** (the adaptive-schedule case left open in [§6.5](#s6-5)'s scope note). The optimal *within-race* adversary is the [§6.6](#s6-6) MDP. Combining selfish release with the [§6.5](#s6-5) *abstention* lever is **dominated**: abstaining a fraction `ψ` of your wins lowers effective mining power to `α_abst = α(1−ψ)/(α(1−ψ)+1−α) < α`, and selfish revenue is monotone increasing in `α` (`fig14`), so withholding-to-abstain strictly reduces the take. Closed-loop timing on `D̂` adds nothing either: the selfish revenue *share* depends only on `(α, γ)`, not on `D̂` (the race is decided block-by-block, while `D̂` moves on the epoch scale). So the adversary frontier is exactly **optimal selfish mining, bounded by the stake threshold** — and, *under the [§6.8](#s6-8) soft inclusion rule that keeps the emergent `p_ref` high*, blunted but not eliminated by rewarded uncles. No compounding lever remains.
|
||||
**Joint / adaptive adversary** (the adaptive-schedule case left open in [§6.5](#s6-5)'s scope note). The optimal *within-race* adversary is the [§6.6](#s6-6) MDP. Combining selfish release with the [§6.5](#s6-5) *abstention* lever is **dominated**: abstaining a fraction `ψ` of your wins lowers effective mining power to `α_abst = α(1−ψ)/(α(1−ψ)+1−α) < α`, and selfish revenue is monotone increasing in `α` (`fig14`), so withholding-to-abstain strictly reduces the take. Closed-loop timing on `D̂` adds nothing either: the selfish revenue *share* depends only on `(α, γ)`, not on `D̂` (the race is decided block-by-block, while `D̂` moves on the epoch scale). So the **profit** frontier is exactly **optimal selfish mining, bounded by the stake threshold** — and, *under the [§6.8](#s6-8) soft inclusion rule that keeps the emergent `p_ref` high*, blunted but not eliminated by rewarded uncles. No compounding lever raises the *take*. This argument is about revenue throughout, and does not transfer to the estimate: an adversary indifferent between earning its stake share and earning the selfish premium is free to spend that premium on deflation instead, which is a strictly larger policy space than the one bounded here ([§6.6](#s6-6), `fig37`).
|
||||
|
||||
<a id="s6-8"></a>
|
||||
### 6.8 A *soft* inclusion rule delivers the benefit without a fork risk (`fig15`)
|
||||
@ -833,7 +849,7 @@ Everything below is assembled from this report's own measurements; each row cite
|
||||
|
||||
**Why soft, not hard, inclusion.** "Owed" cannot be proven — no node can prove which forks a producer saw. An objective slot-gap encoding lets an attacker withhold-then-late-release an uncle to retroactively invalidate honest blocks (censorship); a visibility encoding is subjective and forks consensus with no adversary at all ([§6.8](#s6-8)). The soft rule concedes only a small residual (quantified in [§8.3](#s8-3) item 3): a bounded near-threshold backfire that vanishes as the emergent `p_ref` rises, and a per-α premium that stays ≤ +0.014 even at full inclusion.
|
||||
|
||||
**What TSI defends, and what it measures.** Abstention-style withholding is strictly unprofitable (reward/stake 0.95 → 0.31 for duty 0.10 → 0.75 at β_adv = 0.3), bounded as griefing (deflation = β_adv·ψ), non-persistent, and does not stack across coalitions ([§6.5](#s6-5), [§6.9](#s6-9)) — and a full withdrawal is *correctly measured*, not mis-measured: `D̂ → (1−β_adv)` is the right answer for the participating stake ([§6.4](#s6-4)). The one profitable lever is classic selfish mining above `(1−γ)/(3−2γ)` — a general Nakamoto-PoS result, not TSI-specific; TSI's added exposure (estimator deflation to 0.70 at α = 0.4) is *partly* repaired by uncle-counting (row 9 — to 0.81, not the 0.94 an unrestricted count would reach, because the deployed rule can reference only the first block of each discarded fork), and the profit premium is blunted, not eliminated, by the soft-rewarded references (row 10). The adversary frontier is exactly optimal selfish mining; no compounding lever remains ([§6.7](#s6-7)).
|
||||
**What TSI defends, and what it measures.** Abstention-style withholding is strictly unprofitable (reward/stake 0.95 → 0.31 for duty 0.10 → 0.75 at β_adv = 0.3), bounded as griefing (deflation = β_adv·ψ), non-persistent, and does not stack across coalitions ([§6.5](#s6-5), [§6.9](#s6-9)) — and a full withdrawal is *correctly measured*, not mis-measured: `D̂ → (1−β_adv)` is the right answer for the participating stake ([§6.4](#s6-4)). The one profitable lever is classic selfish mining above `(1−γ)/(3−2γ)` — a general Nakamoto-PoS result, not TSI-specific; TSI's added exposure (estimator deflation to 0.70 at α = 0.4) is *partly* repaired by uncle-counting (row 9 — to 0.81, not the 0.94 an unrestricted count would reach, because the deployed rule can reference only the first block of each discarded fork), and the profit premium is blunted, not eliminated, by the soft-rewarded references (row 10). The *profit* frontier is exactly optimal selfish mining, and no compounding lever raises it ([§6.7](#s6-7)) — but the *estimator* frontier is not the same policy: an adversary willing to settle for break-even pay deflates `D̂` to 0.64 at α = 0.4 where the revenue-maximiser reaches 0.81 ([§6.6](#s6-6), `fig37`). Profitability bounds what an attacker gains, not what the estimate loses.
|
||||
|
||||
<a id="s8-3"></a>
|
||||
### 8.3 Residual risks and open items
|
||||
@ -853,9 +869,10 @@ Everything below is assembled from this report's own measurements; each row cite
|
||||
13. **Reward payout mechanism unspecified ([§8.5](#s8-5)).** Rows 10–11 give the incentive constraints, not the encoding. Header-count indistinguishability (Blend) and the equal-share, content-independent voucher of the Anonymous Leaders Reward Protocol are the binding constraints on any `w_u`/`w_n` schedule, and reconciling them — including a payout path for a producer whose block is off-chain — is spec work left open here.
|
||||
14. **Temporal resolution — `D̂` is stale by design ([§6.11](#s6-11)).** With `β = 1`, `D̂` tracks active stake at a one-epoch (≈ 7.5-day) lag, so it reflects participating stake from ~1–2 weeks ago and updates only weekly; a genuinely *sub-epoch* swing (daily or intra-week churn) falls below the estimator's Nyquist limit and is not tracked as a cycle at all. This is the correct trade for organic churn (row 5), but it bounds how fast TSI can follow real participation changes — flagged, not a defect.
|
||||
15. **Correlated latency untested ([§2](#s2)).** Both link distributions draw each link *independently*, so the sweeps probe the latency marginal, not its spatial structure; geographically **correlated** latency (regional clustering that lets co-located nodes fork as a bloc) is not modelled. In the primary Blend regime the per-hop mixing delay dominates the geographic link term, so this is expected to stay second-order — but it is untested.
|
||||
16. **The deflation-optimal adversary is unquantified ([§6.6](#s6-6)).** The countable recovery ceilings measured there come from adversaries optimising something *else* — revenue (the SSZ MDP: `η = 0.44` at α = 0.4) and reorg depth (`η = 0.52` at α = 0.30). An adversary optimising the *estimator* directly would trade revenue for fork depth, since every block buried behind a fork's first is permanently uncountable while a deeper fork costs it only the marginal revenue of waiting. Both measured ceilings are therefore upper bounds on `η` — best-case-for-the-defender — not lower bounds on the damage; sizing the deflation-optimal policy is future work.
|
||||
16. **The deflation-optimal adversary — measured, and it is worse than the revenue-optimal one ([§6.6](#s6-6), `fig37`).** The ceilings elsewhere come from adversaries optimising revenue or reorg depth, so they bound `η` from above rather than the damage from below. Optimising the estimate directly: unconstrained the optimum is abstention (`D̂ = 1−α`, zero revenue, and [§6.4](#s6-4)-correct measurement rather than a mis-measurement); constrained to earn at least its stake share, it reaches `D̂` = 0.64 at α = 0.4 against the revenue-optimal 0.81. What remains open is the *joint* problem — this frontier is single-coalition and, like everything in [§6.6](#s6-6), assumes `p_ref` is exogenous rather than something the attacker also manipulates (item 10).
|
||||
17. **The `β_adv` axis is a nominal target, not a realised share ([§6.5](#s6-5)).** A random coalition is grown until its stake first reaches `β_adv`, so the last node added overshoots by its own size — under a Pareto tail, potentially a whale. The realised block share at a nominal `β_adv` = 0.1 is 0.137 ± 0.108, and that variance dominates the mean at small `β_adv`. The conclusions here are drawn from monotone trends across the axis and are unaffected, but a reader should not read `β_adv` as an achieved share; a whale coalition (filled largest-first to the target) lands on it precisely and is the cleaner instrument where the exact share matters.
|
||||
18. **The collapsed-branch tail is uncharacterised ([§6.2](#s6-2)).** Static withholding reaches the collapsed branch once in 96 runs — one replicate of the concentrated (whale) 50 % coalition at `δ_max` = 8 s — and never at `δ_max` = 4 s. One reproducible event is enough to retire "not an observed dynamical trap" but far too thin to give the event a rate; its frequency against `β_adv`, `ρ` and coalition concentration is unmeasured.
|
||||
19. **Structurally uncompensatable orphans ([§6.7](#s6-7)(1), [§6.8](#s6-8)).** The uncle reward refunds `w_u` per *referenced* orphan, but 20–40 % of the honest blocks a private-chain attack destroys are unreferenceable by construction — buried behind the first block of a discarded run. Those producers cannot be compensated at any `w_u`, so the fairness guarantee inherits the first-fork ceiling exactly as the density repair does. No reward schedule fixes this; it is a property of the counting rule. Whether that is acceptable — honest miners bearing an uninsurable loss precisely when attacked — is a protocol-design question this report raises rather than settles.
|
||||
|
||||
<a id="s8-4"></a>
|
||||
### 8.4 Capstone: the whole recipe, in one run
|
||||
@ -939,6 +956,7 @@ Sweep studies are committed configs, run with `make <name>` (writes a dated `run
|
||||
| **uncle-selection deviation** (the spec's deterministic oldest-first vs a coin, over `W` × delay) | `configs/uncle-selection.yaml` (`uncle_strategy`, `uncle_random_p`) | [§3.4](#s3-4) |
|
||||
| **residual §6.5-scope variants** (whale coalitions, `jitter > 0`, very slow `β`, and the static-withholding load sweep that finds the collapsed-branch event) | `scripts/adversary_variants.py` (`adversary_selection`); `runs/adversary_variants_{whale,jitter,beta,withhold_load}.parquet` | [§6.2](#s6-2), [§6.5](#s6-5) |
|
||||
| **uncle cap under a private chain** (`U` × `W` × `α` against the SM1 engine adversary, plus the structural/queue decomposition of the honest orphans) | `scripts/selfish_uncle_margin.py` (`adversary_strategy: selfish`); `runs/selfish_uncle_margin{,_decomp}.parquet` | [§6.8](#s6-8) |
|
||||
| **profit/deflation frontier** (how far a *paid* adversary can deflate `D̂`; the deflation-optimal MDP) | `scripts/deflation_frontier.py` (`selfish_mdp.deflation_frontier`, `.deflation_optimal_stats`); fig37 | [§6.6](#s6-6) |
|
||||
|
||||
**Uncle-model convention.** The simulator's default is the **countable** model — first-fork candidates only, derived window `w_u = W/f`, occupied-slot exclusion, per-reference counting rules ([§2.1](#s2-1)). The **unrestricted** baseline is preserved in the code and selected with `--old` on `tsi-sweep`/`tsi-verify`. Its RNG key is byte-identical to the pre-restriction key, so `--old` **bit-reproduces the earlier runs**: a `rho-boundary` cell (δ_max = 8, U = 2, k = 256, N = 1 000) re-run under `--old` matches the committed `2026-07-27_195627_rho-boundary` parquet with `max |Δ| = 0` on every epoch and every metric. Studies in the table above that predate the countable default were produced under the unrestricted model and reproduce exactly under `--old`; the comparison rows quantify where the two models differ, and in the design regime (`ρ < 1`) no difference is resolvable, so those findings carry over unchanged.
|
||||
|
||||
@ -946,7 +964,7 @@ Because the two models draw independent RNG streams, every countable-vs-unrestri
|
||||
|
||||
All studies were **re-run on 2026-07-23/24 with the corrected slot-counting mechanism** ([§2.1](#s2-1)) and the early-stop optimisation; the resilient batch is `scripts/run_all_reruns.sh` (per-step log in `runs/rerun_status.log`). Canonical run directories (latest): fullscale N=5000/10000 = `2026-07-24_094519_fullscale`; fullscale N=1000/2000 = `2026-07-23_171803_fullscale-small`; uncle-window = `2026-07-24_001456`; window-uncles = `2026-07-24_014240`; block-rate = `2026-07-24_043943`; blend-hops-delay = `2026-07-24_064052`; window-scale = `2026-07-24_085234`; latency-shape = `2026-07-24_090014_expdist`; stake-tail = `2026-07-24_090044_pareto133`; heterogeneous-start = `2026-07-24_090114_default`; N-scaling = the `nscaling-{a,b}` + `nscaling32-{a,b}` runs; adversary grids = `runs/adversary_grid/`; jitter = `runs/jitter_grid/`; bootstrap = `runs/bootstrap_fullscale/`; fluctuation = `runs/fluctuation_u0.parquet`; fork-rate = `runs/fork_rate_vs_delay.parquet`; ρ-boundary = `2026-07-27_195627_rho-boundary`. Referencing-model studies: countable-vs-unrestricted (paired, primary) = `2026-08-05_120804_cvo-paired-countable` / `2026-08-05_123403_cvo-paired-old`; the superseded unpaired pair = `2026-08-04_103536_cvo-countable` / `2026-08-04_104010_cvo-old`; window absorption = `2026-08-04_104633_absorption-window`; fine delay band (unpaired) = `2026-08-04_191441_fine-countable` / `2026-08-04_195353_fine-old`; fine delay band (paired, primary) = `2026-08-05_111109_fine-paired-countable` / `2026-08-05_113218_fine-paired-old`. Adversarial-countability studies: uncle-selection deviation = `2026-08-05_152617_uncle-selection`; countable-selfish MDP = `runs/countable_selfish.parquet` + `runs/countable_selfish_reorg.parquet`; §6.5 scope variants and the withhold-load sweep = `runs/adversary_variants_{whale,jitter,beta,withhold_load}.parquet`.
|
||||
|
||||
Figures are embedded from [`report-figures/`](report-figures) and versioned here alongside the report; the simulator folder does **not** commit its own generated figures, so the copies checked in here are the figures of record. Figures are in `report-figures/` (`fig1`–`fig29`, plus [Appendix B](#sB)'s `figB1`–`figB2`; numbering is generation order, not order of appearance). Committed generators: `fig1` (bootstrap, k=2160) by `scripts/bootstrap_dynamics.py`; `fig2`,`fig4`,`fig5`,`fig17`–`fig22` by `scripts/regenerate_extra_figs.py` from the latest sweeps (`fig3` hops×delay×U grid by `scripts/hops_delay_grid.py`, `fig6` (block-rate `U_min` grid + ρ-collapse) rendered ad hoc from `runs/2026-07-24_043943_block-rate` with no committed generator; `fig26` deficit-vs-ρ by `scripts/rho_boundary_analysis.py`) (fullscale-derived `fig17`–`fig20` pool both sizes in that run, N = 5 000 and N = 10 000 — the generators filter on stake_dist/topology/degree/init_dest only, never on `n_nodes`); `fig8`,`fig9` by `scripts/adversary_figs.py` from `runs/adversary_grid/`; `fig10`–`fig12` by `scripts/dynamic_withhold.py`; `fig13`–`fig15` by `scripts/selfish_mining.py`/`selfish_rewards.py`/`reward_mandate.py`; `fig16` by `scripts/stake_vs_delay.py`; `fig23`–`fig24` by `scripts/nscaling_analysis.py`; `fig25` by `scripts/window_scale_analysis.py`; `fig27`–`fig28` by `scripts/reorg_depth.py` (fork rates via `--measure`; private-chain model `src/tsi_sim/reorg.py`); `fig29` by `scripts/churn.py`; `figB1`–`figB2` by `scripts/appendix_fluct.py`. `fig7` (feedback fixed-point) is an analytic overlay; `fig30`–`fig33` (countable-vs-unrestricted accuracy, `q_u`-prediction check, recovery rate, absorption-window sweep) by `scripts/plot_countable_vs_old.py` from the `cvo-countable`/`cvo-old`/`absorption-window` runs; `fig34`–`fig35` (design-regime accuracy and the model gap with 95 % CIs) by `scripts/plot_fine_delay.py` from the `fine-countable`/`fine-old` runs; `fig36` (the countable recovery ceiling under a selfish adversary) by `scripts/countable_selfish.py`, which solves the MDP directly and writes `runs/countable_selfish.parquet` + `runs/countable_selfish_reorg.parquet` (no sweep input). Every figure type the per-node simulator generates appears in this report, and the fork-rate/reorg-depth study closes the previous reproducibility gap for the adversarial figures (`fig8`,`fig9` now have committed generators from `runs/adversary_grid/`).
|
||||
Figures are embedded from [`report-figures/`](report-figures) and versioned here alongside the report; the simulator folder does **not** commit its own generated figures, so the copies checked in here are the figures of record. Figures are in `report-figures/` (`fig1`–`fig29`, plus [Appendix B](#sB)'s `figB1`–`figB2`; numbering is generation order, not order of appearance). Committed generators: `fig1` (bootstrap, k=2160) by `scripts/bootstrap_dynamics.py`; `fig2`,`fig4`,`fig5`,`fig17`–`fig22` by `scripts/regenerate_extra_figs.py` from the latest sweeps (`fig3` hops×delay×U grid by `scripts/hops_delay_grid.py`, `fig6` (block-rate `U_min` grid + ρ-collapse) rendered ad hoc from `runs/2026-07-24_043943_block-rate` with no committed generator; `fig26` deficit-vs-ρ by `scripts/rho_boundary_analysis.py`) (fullscale-derived `fig17`–`fig20` pool both sizes in that run, N = 5 000 and N = 10 000 — the generators filter on stake_dist/topology/degree/init_dest only, never on `n_nodes`); `fig8`,`fig9` by `scripts/adversary_figs.py` from `runs/adversary_grid/`; `fig10`–`fig12` by `scripts/dynamic_withhold.py`; `fig13`–`fig15` by `scripts/selfish_mining.py`/`selfish_rewards.py`/`reward_mandate.py`; `fig16` by `scripts/stake_vs_delay.py`; `fig23`–`fig24` by `scripts/nscaling_analysis.py`; `fig25` by `scripts/window_scale_analysis.py`; `fig27`–`fig28` by `scripts/reorg_depth.py` (fork rates via `--measure`; private-chain model `src/tsi_sim/reorg.py`); `fig29` by `scripts/churn.py`; `figB1`–`figB2` by `scripts/appendix_fluct.py`. `fig7` (feedback fixed-point) is an analytic overlay; `fig30`–`fig33` (countable-vs-unrestricted accuracy, `q_u`-prediction check, recovery rate, absorption-window sweep) by `scripts/plot_countable_vs_old.py` from the `cvo-countable`/`cvo-old`/`absorption-window` runs; `fig34`–`fig35` (design-regime accuracy and the model gap with 95 % CIs) by `scripts/plot_fine_delay.py` from the `fine-countable`/`fine-old` runs; `fig36` (the countable recovery ceiling under a selfish adversary) by `scripts/countable_selfish.py`, which solves the MDP directly and writes `runs/countable_selfish.parquet` + `runs/countable_selfish_reorg.parquet` (no sweep input); `fig37` (the profit/deflation frontier) by `scripts/deflation_frontier.py` from `runs/deflation_frontier.parquet`, likewise solver-only. Every figure type the per-node simulator generates appears in this report, and the fork-rate/reorg-depth study closes the previous reproducibility gap for the adversarial figures (`fig8`,`fig9` now have committed generators from `runs/adversary_grid/`).
|
||||
|
||||
---
|
||||
|
||||
|
||||
BIN
reports/tsi/report-figures/fig37_deflation_frontier.png
Normal file
BIN
reports/tsi/report-figures/fig37_deflation_frontier.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 231 KiB |
@ -0,0 +1,147 @@
|
||||
"""How far can a PROFITABLE adversary deflate the estimate? — REPORT §8.3 item 16 (fig37).
|
||||
|
||||
§6.6 measures the countable recovery ceiling under two adversaries, and both optimise something
|
||||
other than the estimator: revenue (the SSZ objective) and reorg depth. Item 16 recorded the
|
||||
consequence — those are upper bounds on `eta`, not lower bounds on the damage — and asked what an
|
||||
adversary optimising deflation directly would do.
|
||||
|
||||
Two answers, because the obvious question is the wrong one:
|
||||
|
||||
* **Unconstrained**, the deflation optimum is pure abstention: publish nothing, adopt when
|
||||
overtaken, drive `D_hat` to exactly `1 - alpha` and revenue to zero. §6.4 already covers this
|
||||
and shows it is *correct* measurement — a coalition that publishes nothing is not
|
||||
participating, so `1 - alpha` is the right answer for the stake that is.
|
||||
* **Constrained to stay paid**, the question bites. Sweeping the mixed objective
|
||||
`lam * (adversary blocks) - (contribution to D_hat)` traces the profit/deflation frontier;
|
||||
the point of interest is where the revenue *share* reaches `alpha`, i.e. where the adversary
|
||||
does no worse than mining honestly. Everything below that line is self-punishing griefing,
|
||||
already bounded by §6.5.
|
||||
|
||||
The headline: at `alpha = 0.4` an adversary content with break-even revenue pushes `D_hat` to
|
||||
~0.64, against the revenue-optimal policy's 0.81. The revenue-optimal adversary is not the
|
||||
estimator's worst case, and the gap is free — it costs only the selfish premium.
|
||||
|
||||
Note the sweep parameter is not monotone in revenue: selfish mining wins a larger share of a
|
||||
*smaller* pie, so raw adversary block rate is maximised by honest mining and large `lam` returns
|
||||
there. That is fine — the sweep is used to enumerate candidate policies and their measured
|
||||
(share, `D_hat`) pairs, not as a monotone path, and any point on it is a legitimate strategy.
|
||||
|
||||
Run: python scripts/deflation_frontier.py (writes runs/deflation_frontier.parquet + fig37)
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
import numpy as np
|
||||
import pandas as pd
|
||||
|
||||
from tsi_sim.plotting import style
|
||||
from tsi_sim.selfish_mdp import deflation_frontier, deflation_optimal_stats, optimal_policy_stats
|
||||
|
||||
HERE = Path(__file__).resolve().parent.parent
|
||||
RUNS = HERE / "runs"
|
||||
FIGS = HERE / "report-figures"
|
||||
RUNS.mkdir(exist_ok=True)
|
||||
FIGS.mkdir(exist_ok=True)
|
||||
|
||||
CAP = 64
|
||||
GAMMA = 0.0
|
||||
ALPHAS = [0.30, 1 / 3, 0.36, 0.38, 0.40, 0.42, 0.45]
|
||||
LAMS = [round(x, 2) for x in np.arange(0.0, 3.05, 0.1)]
|
||||
|
||||
|
||||
def sweep() -> pd.DataFrame:
|
||||
rows = []
|
||||
for a in ALPHAS:
|
||||
ro = optimal_policy_stats(a, GAMMA, cap=CAP)
|
||||
do = deflation_optimal_stats(a, GAMMA, cap=CAP)
|
||||
for lam in LAMS:
|
||||
r = deflation_frontier(a, GAMMA, lam, cap=CAP)
|
||||
r |= dict(revenue_optimal=ro.revenue,
|
||||
dhat_revenue_optimal=ro.dhat_ratio(1.0, True) if ro.deviates else 1.0,
|
||||
dhat_abstention=do.dhat_ratio(1.0, True))
|
||||
rows.append(r)
|
||||
df = pd.DataFrame(rows)
|
||||
df.to_parquet(RUNS / "deflation_frontier.parquet", index=False)
|
||||
return df
|
||||
|
||||
|
||||
def best_profitable(df: pd.DataFrame) -> pd.DataFrame:
|
||||
"""Per alpha, the frontier point minimising D_hat among those paying at least stake share."""
|
||||
out = []
|
||||
for _a, g in df.groupby("alpha"):
|
||||
paid = g[g.reward_per_stake >= 1.0 - 1e-9]
|
||||
if paid.empty:
|
||||
continue
|
||||
out.append(paid.loc[paid.dhat_countable.idxmin()])
|
||||
return pd.DataFrame(out)
|
||||
|
||||
|
||||
def report(df: pd.DataFrame) -> None:
|
||||
best = best_profitable(df)
|
||||
print(f"{'alpha':>6} {'rev-opt rev':>12} {'rev-opt D':>10} | "
|
||||
f"{'best paid rev':>14} {'xstake':>7} {'D':>7} {'extra deflation':>16}")
|
||||
for a, g in df.groupby("alpha"):
|
||||
ro_r, ro_d = g.revenue_optimal.iloc[0], g.dhat_revenue_optimal.iloc[0]
|
||||
b = best[best.alpha == a]
|
||||
if b.empty:
|
||||
print(f"{a:6.3f} {ro_r:12.4f} {ro_d:10.4f} | (nothing profitable deflates)")
|
||||
continue
|
||||
b = b.iloc[0]
|
||||
print(f"{a:6.3f} {ro_r:12.4f} {ro_d:10.4f} | {b.revenue:14.4f} "
|
||||
f"{b.reward_per_stake:7.3f} {b.dhat_countable:7.4f} {ro_d - b.dhat_countable:+16.4f}")
|
||||
print("\n(extra deflation > 0 means the profitable-but-deflating policy beats the "
|
||||
"revenue-optimal one at damaging the estimate, at no cost versus honest mining)")
|
||||
|
||||
|
||||
def fig37(df: pd.DataFrame) -> None:
|
||||
import matplotlib.pyplot as plt
|
||||
style.apply_style()
|
||||
fig, axes = plt.subplots(1, 2, figsize=(9.6, 3.8))
|
||||
|
||||
# LEFT: the frontier itself for a few alphas — revenue share against the estimate
|
||||
ax = axes[0]
|
||||
for i, a in enumerate([0.36, 0.40, 0.45]):
|
||||
g = df[np.isclose(df.alpha, a)].sort_values("lam")
|
||||
c = style.OKABE_ITO[i + 1]
|
||||
ax.plot(g.revenue, g.dhat_countable, "-o", ms=3, color=c, label=rf"$\alpha={a}$")
|
||||
ax.axvline(a, color=c, lw=0.7, ls=":")
|
||||
ro = g.dhat_revenue_optimal.iloc[0]
|
||||
ax.plot([g.revenue_optimal.iloc[0]], [ro], "*", ms=11, color=c)
|
||||
ax.set_xlabel("adversary revenue share (dotted line = its stake, i.e. break-even)")
|
||||
ax.set_ylabel(r"$\hat D / D^*$")
|
||||
ax.set_title("Profit/deflation frontier (★ = revenue-optimal)")
|
||||
ax.legend(fontsize=7, loc="lower right")
|
||||
|
||||
# RIGHT: how much worse the estimator gets once the adversary stops maximising revenue
|
||||
ax = axes[1]
|
||||
best = best_profitable(df).sort_values("alpha")
|
||||
ref = df.groupby("alpha").agg(d_ro=("dhat_revenue_optimal", "first"),
|
||||
d_ab=("dhat_abstention", "first")).reset_index()
|
||||
ax.plot(ref.alpha, ref.d_ro, "-*", ms=9, color=style.OKABE_ITO[2],
|
||||
label="revenue-optimal (§6.6)")
|
||||
ax.plot(best.alpha, best.dhat_countable, "-o", ms=4, color=style.OKABE_ITO[3],
|
||||
label="best deflation at break-even pay")
|
||||
ax.plot(ref.alpha, ref.d_ab, "--v", ms=4, color=style.OKABE_ITO[1],
|
||||
label=r"abstention ($1-\alpha$, unpaid)")
|
||||
ax.axhline(1.0, color="0.5", lw=0.9, ls="--")
|
||||
ax.set_xlabel(r"adversary stake $\alpha$")
|
||||
ax.set_ylabel(r"$\hat D / D^*$")
|
||||
ax.set_title("The revenue-optimal adversary is not the worst case")
|
||||
ax.legend(fontsize=7, loc="lower left")
|
||||
|
||||
style.save(fig, FIGS / "fig37_deflation_frontier", provenance="scripts/deflation_frontier.py")
|
||||
plt.close(fig)
|
||||
|
||||
|
||||
def main() -> None:
|
||||
print(f"=== profit/deflation frontier (MDP cap={CAP}, gamma={GAMMA}) ===")
|
||||
df = sweep()
|
||||
report(df)
|
||||
fig37(df)
|
||||
print("wrote fig37_deflation_frontier")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@ -124,16 +124,16 @@ def _precompute(alpha, gamma, states, index, cap):
|
||||
return probs, nxt, radv, rhon, ohon, oadv, legal
|
||||
|
||||
|
||||
def _solve_mdp(pc, rho, ref, iters, tol):
|
||||
"""Optimal average gain for the rho-parametrised reward, by *damped* relative value iteration.
|
||||
def _solve_reward(pc, reward, ref, iters, tol):
|
||||
"""Optimal average gain for an arbitrary per-branch reward, by *damped* relative value
|
||||
iteration. Returns ``(gain, V)``.
|
||||
|
||||
The chain is periodic, so undamped VI oscillates and a naive |Δgain| stop can false-trigger as
|
||||
the gain crosses zero. We damp (``V ← V + τ(TV − V)``) to break periodicity and stop on the
|
||||
textbook span criterion: at the average-reward fixed point ``TV − V = g·1`` (span → 0), and the
|
||||
gain ``g`` is that uniform increment. Returns the span-centre of the final Bellman increment.
|
||||
"""
|
||||
probs, nxt, radv, rhon, _ohon, _oadv, legal = pc
|
||||
reward = (1.0 - rho) * radv - rho * rhon # (4, n, K), constant across iterations
|
||||
probs, nxt, _radv, _rhon, _ohon, _oadv, legal = pc
|
||||
V = np.zeros(probs.shape[1])
|
||||
tau = 0.5
|
||||
d = np.zeros(1)
|
||||
@ -145,7 +145,61 @@ def _solve_mdp(pc, rho, ref, iters, tol):
|
||||
break
|
||||
V = V + tau * d
|
||||
V -= V[ref] # anchor to keep values bounded
|
||||
return 0.5 * (d.max() + d.min())
|
||||
return 0.5 * (d.max() + d.min()), V
|
||||
|
||||
|
||||
def _solve_mdp(pc, rho, ref, iters, tol):
|
||||
"""Optimal average gain for the rho-parametrised *revenue* reward (the ratio transform)."""
|
||||
_probs, _nxt, radv, rhon, _ohon, _oadv, _legal = pc
|
||||
return _solve_reward(pc, (1.0 - rho) * radv - rho * rhon, ref, iters, tol)[0]
|
||||
|
||||
|
||||
def _greedy_policy(pc, reward, V):
|
||||
"""Greedy action per state, tie-broken toward the lowest index (ADOPT first, i.e. the
|
||||
least-deviating action) so near-ties resolve deterministically rather than arbitrarily."""
|
||||
probs, nxt, _radv, _rhon, _ohon, _oadv, legal = pc
|
||||
q = (probs * (reward + V[nxt])).sum(axis=2)
|
||||
q[~legal] = -1e18
|
||||
best = q.max(axis=0)
|
||||
return np.where(q >= best[None, :] - 1e-9, np.arange(4)[:, None], 99).min(axis=0)
|
||||
|
||||
|
||||
def _stationary(pc, pol):
|
||||
"""Stationary distribution of the policy-induced chain. The chain is periodic (see
|
||||
:func:`_solve_reward`), so iterate the LAZY chain — same stationary vector, no oscillation."""
|
||||
probs, nxt, *_ = pc
|
||||
n = probs.shape[1]
|
||||
rows = np.arange(n)
|
||||
p_s, n_s = probs[pol, rows], nxt[pol, rows]
|
||||
pi = np.full(n, 1.0 / n)
|
||||
for _ in range(500_000):
|
||||
new = 0.5 * pi + 0.5 * np.bincount(n_s.ravel(), weights=(pi[:, None] * p_s).ravel(),
|
||||
minlength=n)
|
||||
new /= new.sum()
|
||||
if np.abs(new - pi).max() < 1e-15:
|
||||
return new
|
||||
pi = new
|
||||
return pi
|
||||
|
||||
|
||||
def _policy_rates(pc, pol, pi):
|
||||
"""Per-block-finding-event rates under a policy's stationary distribution.
|
||||
|
||||
Every MDP transition consumes exactly one block-finding event (the alpha/beta branch), so
|
||||
stationary per-step rates *are* per-event rates.
|
||||
"""
|
||||
probs, nxt, radv, rhon, ohon, oadv, _legal = pc
|
||||
rows = np.arange(probs.shape[1])
|
||||
w = pi[:, None] * probs[pol, rows]
|
||||
oh, oa = ohon[pol, rows], oadv[pol, rows]
|
||||
return dict(
|
||||
adv_rate=float((w * radv[pol, rows]).sum()),
|
||||
hon_rate=float((w * rhon[pol, rows]).sum()),
|
||||
orphan_hon_blocks=float((w * oh).sum()),
|
||||
orphan_hon_runs=float((w * (oh > 0)).sum()),
|
||||
orphan_adv_blocks=float((w * oa).sum()),
|
||||
orphan_adv_runs=float((w * (oa > 0)).sum()),
|
||||
)
|
||||
|
||||
|
||||
def optimal_selfish_revenue(alpha: float, gamma: float, cap: int = 60,
|
||||
@ -226,6 +280,102 @@ class OptimalPolicyStats:
|
||||
return self.density_fraction + float(np.clip(p_ref, 0.0, 1.0)) * rec
|
||||
|
||||
|
||||
def deflation_optimal_stats(alpha: float, gamma: float, p_ref: float = 1.0, cap: int = 64,
|
||||
iters: int = 4000, tol: float = 1e-10) -> OptimalPolicyStats:
|
||||
"""The policy that MINIMISES the estimate, rather than the one that maximises revenue.
|
||||
|
||||
Both ceilings in §6.6 come from adversaries optimising something else — revenue (the SSZ
|
||||
objective) and reorg depth — so they bound ``eta`` from above without bounding the damage
|
||||
from below. This closes that gap by optimising the estimator directly.
|
||||
|
||||
No ratio transform is needed, unlike the revenue objective. Every transition consumes exactly
|
||||
one block-finding event, and the estimate is
|
||||
|
||||
D̂/D = (canonical blocks + p_ref · countable uncles) / events
|
||||
|
||||
with one countable uncle per discarded honest *run* (§2.1). So the per-event contribution is
|
||||
``radv + rhon + p_ref·[orphaned honest run]``, and minimising its long-run average is a plain
|
||||
average-reward MDP — solved by maximising the negated reward in one value-iteration pass.
|
||||
|
||||
``p_ref`` is the share of countable orphans that honest referencers actually pick up; at the
|
||||
default 1 the adversary faces the most effective possible repair, so the resulting deflation
|
||||
is the worst case it can force against a fully-cooperative honest network.
|
||||
|
||||
**The unconstrained optimum is degenerate, and usefully so.** It is pure abstention: mine
|
||||
privately, publish nothing, adopt when overtaken. That drives ``D̂`` to exactly ``1 - alpha``
|
||||
and revenue to zero. But §6.4 already establishes that this is *correct* measurement rather
|
||||
than mis-measurement — a coalition that publishes nothing genuinely is not participating, and
|
||||
``1 - alpha`` is the right answer for the stake that is. So the unconstrained objective asks
|
||||
the wrong question; the one that matters is how far ``D̂`` can be pushed by an adversary that
|
||||
stays profitable, which :func:`deflation_frontier` traces.
|
||||
"""
|
||||
states, index = _build_states(cap)
|
||||
pc = _precompute(alpha, gamma, states, index, cap)
|
||||
probs, nxt, radv, rhon, ohon, oadv, legal = pc
|
||||
ref = index[(1, 0, IRRELEVANT)]
|
||||
|
||||
dhat_step = radv + rhon + float(p_ref) * (ohon > 0)
|
||||
gain, V = _solve_reward(pc, -dhat_step, ref, iters, tol)
|
||||
pol = _greedy_policy(pc, -dhat_step, V)
|
||||
pi = _stationary(pc, pol)
|
||||
rates = _policy_rates(pc, pol, pi)
|
||||
|
||||
# The gain IS the negated minimum estimate; cross-check it against the stationary rates so a
|
||||
# silent mismatch between the solver and the accounting cannot pass unnoticed.
|
||||
dhat = rates["adv_rate"] + rates["hon_rate"] + float(p_ref) * rates["orphan_hon_runs"]
|
||||
if abs(-gain - dhat) > 1e-6:
|
||||
raise AssertionError(f"deflation MDP gain {-gain:.9f} != stationary D-hat {dhat:.9f}")
|
||||
|
||||
revenue = (rates["adv_rate"] / (rates["adv_rate"] + rates["hon_rate"])
|
||||
if rates["adv_rate"] + rates["hon_rate"] > 0 else 0.0)
|
||||
return OptimalPolicyStats(alpha=alpha, gamma=gamma, revenue=revenue, deviates=True,
|
||||
density_fraction=rates["adv_rate"] + rates["hon_rate"],
|
||||
orphan_hon_blocks=rates["orphan_hon_blocks"],
|
||||
orphan_hon_runs=rates["orphan_hon_runs"],
|
||||
orphan_adv_blocks=rates["orphan_adv_blocks"],
|
||||
orphan_adv_runs=rates["orphan_adv_runs"])
|
||||
|
||||
|
||||
def deflation_frontier(alpha: float, gamma: float, lam: float, p_ref: float = 1.0,
|
||||
cap: int = 64, iters: int = 4000, tol: float = 1e-10) -> dict:
|
||||
"""One point on the profit/deflation trade-off: the policy optimal for a mixed objective.
|
||||
|
||||
Neither pure objective answers item 16. Maximising revenue ignores the estimator; minimising
|
||||
the estimate degenerates to abstention, which forfeits every block reward and is correctly
|
||||
measured anyway (:func:`deflation_optimal_stats`). What the report needs to know is how much
|
||||
deflation an adversary can force *while still being paid* — i.e. the Pareto frontier between
|
||||
the two.
|
||||
|
||||
Sweeping ``lam`` from 0 upward traces it: the per-event reward is
|
||||
``lam · (adversary blocks) − (contribution to D̂)``, so ``lam = 0`` is the deflation optimum
|
||||
and large ``lam`` approaches the revenue optimum. The point of interest is where the revenue
|
||||
*share* crosses ``alpha`` — an adversary doing at least as well as honest mining — because
|
||||
below that the attack is self-punishing griefing already bounded by §6.5.
|
||||
"""
|
||||
states, index = _build_states(cap)
|
||||
pc = _precompute(alpha, gamma, states, index, cap)
|
||||
_probs, _nxt, radv, rhon, ohon, _oadv, _legal = pc
|
||||
ref = index[(1, 0, IRRELEVANT)]
|
||||
|
||||
reward = float(lam) * radv - (radv + rhon + float(p_ref) * (ohon > 0))
|
||||
_gain, V = _solve_reward(pc, reward, ref, iters, tol)
|
||||
pol = _greedy_policy(pc, reward, V)
|
||||
rates = _policy_rates(pc, pol, _stationary(pc, pol))
|
||||
canonical = rates["adv_rate"] + rates["hon_rate"]
|
||||
blocks, runs = rates["orphan_hon_blocks"], rates["orphan_hon_runs"]
|
||||
revenue = (rates["adv_rate"] / canonical) if canonical > 0 else 0.0
|
||||
return dict(
|
||||
alpha=alpha, gamma=gamma, lam=lam,
|
||||
revenue=revenue,
|
||||
reward_per_stake=(revenue / alpha) if alpha else 0.0,
|
||||
density_fraction=canonical,
|
||||
dhat_countable=canonical + float(p_ref) * runs,
|
||||
dhat_unrestricted=canonical + float(p_ref) * blocks,
|
||||
eta=(runs / blocks) if blocks > 0 else 1.0,
|
||||
orphan_hon_blocks=blocks,
|
||||
)
|
||||
|
||||
|
||||
def optimal_policy_stats(alpha: float, gamma: float, cap: int = 64,
|
||||
iters: int = 4000, tol: float = 1e-10) -> OptimalPolicyStats:
|
||||
"""Orphan structure of the *optimal* selfish policy — the input the countable model needs.
|
||||
@ -252,46 +402,16 @@ def optimal_policy_stats(alpha: float, gamma: float, cap: int = 64,
|
||||
orphan_hon_runs=0.0, orphan_adv_blocks=0.0,
|
||||
orphan_adv_runs=0.0)
|
||||
|
||||
# Recover V at the optimal rho, then the greedy policy.
|
||||
# Recover V at the optimal rho, then read off the greedy policy and its stationary rates.
|
||||
reward = (1.0 - revenue) * radv - revenue * rhon
|
||||
n = probs.shape[1]
|
||||
V = np.zeros(n)
|
||||
for _ in range(iters):
|
||||
q = (probs * (reward + V[nxt])).sum(axis=2)
|
||||
q[~legal] = -1e18
|
||||
d = q.max(axis=0) - V
|
||||
if d.max() - d.min() < tol:
|
||||
break
|
||||
V = V + 0.5 * d
|
||||
V -= V[ref]
|
||||
q = (probs * (reward + V[nxt])).sum(axis=2)
|
||||
q[~legal] = -1e18
|
||||
# Deterministic tie-break toward the lowest action index (ADOPT < OVERRIDE < MATCH < WAIT) so
|
||||
# near-ties resolve to the least-deviating policy instead of an arbitrary argmax.
|
||||
best = q.max(axis=0)
|
||||
pol = np.where(q >= best[None, :] - 1e-9, np.arange(4)[:, None], 99).min(axis=0)
|
||||
|
||||
rows = np.arange(n)
|
||||
p_s, n_s = probs[pol, rows], nxt[pol, rows] # (n, K)
|
||||
# Stationary distribution. The policy chain is periodic (see _solve_mdp), so iterate the lazy
|
||||
# chain — same stationary vector, no oscillation.
|
||||
pi = np.full(n, 1.0 / n)
|
||||
for _ in range(500_000):
|
||||
new = 0.5 * pi + 0.5 * np.bincount(n_s.ravel(), weights=(pi[:, None] * p_s).ravel(),
|
||||
minlength=n)
|
||||
new /= new.sum()
|
||||
if np.abs(new - pi).max() < 1e-15:
|
||||
pi = new
|
||||
break
|
||||
pi = new
|
||||
|
||||
w = pi[:, None] * p_s # stationary branch flow
|
||||
ohon_s, oadv_s = ohon[pol, rows], oadv[pol, rows]
|
||||
_gain, V = _solve_reward(pc, reward, ref, iters, tol)
|
||||
pol = _greedy_policy(pc, reward, V)
|
||||
rates = _policy_rates(pc, pol, _stationary(pc, pol))
|
||||
return OptimalPolicyStats(
|
||||
alpha=alpha, gamma=gamma, revenue=revenue, deviates=True,
|
||||
density_fraction=float((w * (radv[pol, rows] + rhon[pol, rows])).sum()),
|
||||
orphan_hon_blocks=float((w * ohon_s).sum()),
|
||||
orphan_hon_runs=float((w * (ohon_s > 0)).sum()),
|
||||
orphan_adv_blocks=float((w * oadv_s).sum()),
|
||||
orphan_adv_runs=float((w * (oadv_s > 0)).sum()),
|
||||
density_fraction=rates["adv_rate"] + rates["hon_rate"],
|
||||
orphan_hon_blocks=rates["orphan_hon_blocks"],
|
||||
orphan_hon_runs=rates["orphan_hon_runs"],
|
||||
orphan_adv_blocks=rates["orphan_adv_blocks"],
|
||||
orphan_adv_runs=rates["orphan_adv_runs"],
|
||||
)
|
||||
|
||||
@ -76,6 +76,58 @@ def test_attacker_self_uncle_is_capped_too():
|
||||
assert 0.5 < s.countable_recovery_adv < 1.0
|
||||
|
||||
|
||||
def test_unconstrained_deflation_optimum_is_abstention():
|
||||
# Minimising the estimate with no constraint degenerates: publish nothing, and D-hat lands on
|
||||
# exactly 1 - alpha with zero revenue. §6.4 already covers that case and shows it is CORRECT
|
||||
# measurement rather than mis-measurement, which is why item 16 needs the paid frontier.
|
||||
from tsi_sim.selfish_mdp import deflation_optimal_stats
|
||||
|
||||
for alpha in (0.2, 0.4):
|
||||
s = deflation_optimal_stats(alpha, 0.0, cap=16)
|
||||
assert abs(s.dhat_ratio(1.0, True) - (1.0 - alpha)) < 1e-6
|
||||
assert s.revenue < 1e-9
|
||||
assert s.orphan_hon_blocks < 1e-9 # it orphans no honest work at all
|
||||
|
||||
|
||||
def test_deflation_solver_gain_matches_its_stationary_accounting():
|
||||
# deflation_optimal_stats raises if the MDP's average gain disagrees with the estimate
|
||||
# recomputed from the stationary distribution -- an independent check that the solver and the
|
||||
# orphan accounting describe the same policy. Exercise it across a spread of inputs.
|
||||
from tsi_sim.selfish_mdp import deflation_optimal_stats
|
||||
|
||||
for alpha in (0.25, 0.35, 0.45):
|
||||
for p_ref in (0.0, 0.85, 1.0):
|
||||
deflation_optimal_stats(alpha, 0.0, p_ref=p_ref, cap=16) # no AssertionError
|
||||
|
||||
|
||||
def test_frontier_endpoints_bracket_the_two_pure_objectives():
|
||||
from tsi_sim.selfish_mdp import deflation_frontier, deflation_optimal_stats
|
||||
|
||||
alpha = 0.4
|
||||
zero = deflation_frontier(alpha, 0.0, 0.0, cap=16)
|
||||
pure = deflation_optimal_stats(alpha, 0.0, cap=16)
|
||||
assert abs(zero["dhat_countable"] - pure.dhat_ratio(1.0, True)) < 1e-6 # lam=0 is that optimum
|
||||
# Selfish mining takes a bigger share of a SMALLER pie, so maximising raw adversary block
|
||||
# rate returns to honest mining -- the frontier is not monotone in revenue, by construction.
|
||||
far = deflation_frontier(alpha, 0.0, 50.0, cap=16)
|
||||
assert abs(far["revenue"] - alpha) < 1e-3
|
||||
assert abs(far["dhat_countable"] - 1.0) < 1e-3
|
||||
|
||||
|
||||
def test_a_paid_policy_deflates_further_than_the_revenue_optimum():
|
||||
# Item 16's answer: the revenue-optimal adversary is not the estimator's worst case. At
|
||||
# alpha = 0.4 a policy exists that pays at least as well as honest mining yet deflates
|
||||
# substantially further than the revenue optimum does.
|
||||
from tsi_sim.selfish_mdp import deflation_frontier, optimal_policy_stats
|
||||
|
||||
alpha, cap = 0.4, 32
|
||||
ro = optimal_policy_stats(alpha, 0.0, cap=cap)
|
||||
paid = [deflation_frontier(alpha, 0.0, lam, cap=cap) for lam in (0.4, 0.6, 0.8, 1.0)]
|
||||
paid = [p for p in paid if p["reward_per_stake"] >= 1.0 - 1e-9]
|
||||
assert paid, "expected at least one break-even-or-better frontier point"
|
||||
assert min(p["dhat_countable"] for p in paid) < ro.dhat_ratio(1.0, True) - 0.05
|
||||
|
||||
|
||||
def test_reorg_countable_recovery_from_depths():
|
||||
# A depth-d reorg discards one chain of d blocks -> 1 countable uncle: runs / blocks.
|
||||
from tsi_sim.reorg import countable_recovery_from_depths
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user