The study started as a peering-degree question and grew well past it: propagation,
adversary exposure, deanonymization and time-to-link, reliability under uniform
and correlated churn, messaging redundancy, and cover traffic. The pd name no
longer describes it.
tools/simulators/blend/pd/ -> tools/simulators/blend/, package src/pd -> src/blend,
and reports/blend/pd/ -> reports/blend/. Moved with git mv so history follows.
The text substitutions are deliberately narrow. pd is also the conventional pandas
alias, and pandas genuinely has a pd.plotting submodule, so a blanket pd. -> blend.
rewrite would have corrupted four files. Only package-unambiguous forms were
changed: from pd.X, -m pd.X, pd.<our module>, PD_BYTES_BUDGET, src/pd, and the
pyproject name. All four import pandas as pd lines are untouched and verified.
Both READMEs reframed: peering degree is now presented as the primary axis that
ties the others together rather than as the subject, and the relative links, which
lost a directory level in the move, are corrected.
Verified after the move: ruff clean, 101 tests, 45 verify anchors, make targets,
the script shims, an end-to-end smoke run, and data/report_numbers.py still
reproducing the report tables from the checked-in evidence.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Found by the cover-traffic sweep: the 99%-safe stake ceiling FELL as the cover
rate rose, while the mean bind rose to 0.38 -- backwards.
quota_exceedance_prob summed the Poisson CDF by hand starting from exp(-lam).
That underflows to zero past lam ~ 745, so the CDF collapsed to 0 and the function
reported every node as exceeding its quota, which drove the bisection in
max_alpha_for_confidence to a meaningless answer. The default rate is unaffected
(lam ~ 32), but raising the cover rate reaches the broken regime immediately,
because the quota and the tolerable block count grow in proportion.
Replaced with scipy poisson.sf. Exceedance at the mean bind is now ~0.5 at every
rate, as it must be, and the safe/mean ratio rises 0.65 -> 0.81 -> 0.90 -> 0.95 ->
0.98 across the swept rates: Poisson noise shrinks relative to a growing quota, so
less headroom is needed for the same confidence. Two regression tests pin both.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Check 9 -- cover traffic on a timeline. Blending follows rate*(2M+1)/3 (7.39 vs
7.41 predicted), the mean hold is the renewal residual (2M+1)/6, and mixing is
nil at the baseline rate: 0.007 concurrent holds, max 2. That last one is the
substantive finding rather than a sanity check -- at one message per second a
relay has nothing to mix, so the anonymity comes entirely from what it has seen.
Check 10 -- the emission quota. The measured breakpoint brackets the closed form
(band [0.127%, 0.159%] against a predicted 0.1475%), and deflating D_hat/D to
0.64 pushes more nodes over their quota, as the (D_hat/D)*alpha_max form requires.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
engine gains a cover_rates axis: each rate plays a timeline through the same
graph and pairs it with the epoch emission budget, which needs no graph and so is
computed alongside rather than inside the window. Seeds are separate streams
(traffic_seedseq for the timeline and clocks, stake_seedseq for the stake draw
and budget), so the stake distribution is independent of the topology and of the
message schedule.
sweep writes traffic.parquet only when a cover-traffic study actually ran, so
every existing config keeps producing exactly three tables. quota_summary reports
the measured ceiling beside the predicted one in the same row, so a run can be
checked against the closed form instead of asked to be believed.
Two figures: blending against cover rate and release delay with the
rate*(2M+1)/3 law overlaid, and the quota ceiling with the measured transition
band against the prediction.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The quota ceiling was closed-form only. This adds per-node stake so a run can
show nodes actually breaking it.
- assign_stake: uniform, or heavy-tailed zipf (s ~ 1/rank^a), which is what makes
the ceiling bite -- the head sits orders of magnitude above it, the tail far below;
- inferred_alpha: converts true relative stake to the sigma/D_hat the lottery
actually weighs, so a low estimate inflates every node alpha;
- simulate_epoch_emissions: measures the budget over a full epoch. Overrun happens
at epoch scale and needs no graph, so this is cheap: proposals are Binomial over
the epoch slots, a proposal cancels the next cover, and a node stays at exactly
its quota until its wins no longer fit -- at which point it emits more often than
everyone else, which is the signal cover traffic exists to suppress.
Measured against the closed form at N=20,000, zipf stake, over an epoch: the
predicted ceiling falls inside the transition band every time, and at D_hat/D = 1
the smallest overrunning node sits at 0.1468% against a predicted 0.1475%. The
D_hat/D normalisation is confirmed empirically -- deflating the estimate to 0.64
pulls the measured ceiling down with it, as the (D_hat/D)*alpha_max form requires.
With heavy-tailed stake 99.7% of nodes comply and only the head breaks; the
largest holder at 9.5% stake is some 65x over its allowance.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The key() coverage test enumerates every SimConfig field, so the five cover-
traffic knobs had to be given alternative values. Caught by the test itself
immediately after the previous commit.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
First half of the cover-traffic work: the two new modules and their tests.
quota.py -- the emission budget. Cover traffic gives every node the same number
of emissions per epoch, which only holds while a node block proposals fit inside
its quota. The bind is exact: alpha_max = ln(1-q)/ln(1-f), where alpha is stake
relative to the INFERRED total D_hat, since that is the denominator the lottery
threshold is derived from. In true stake the ceiling carries the estimator ratio,
s_max = (D_hat/D)*alpha_max, with D_hat/D an input rather than an assumption. The
familiar q/f is a small-q approximation that runs 1.7% high and so overstates the
tolerable stake. Sitting on the mean bind overruns the quota half the time, so
max_alpha_for_confidence gives the ceiling that holds with stated probability.
traffic.py -- the timeline. The rest of the simulator samples independent rounds
and draws each hold from the stationary residual, which has no notion of time and
so can never let two messages meet at a relay. Here every node owns one
free-running clock shared by all messages through it, extended lazily so only the
relays actually visited grow one. A clock sampled once still reproduces
mixclock.mix_wait, so single-message statistics are unchanged.
It separates two quantities that are easy to conflate: mixing (messages a relay
holds at once) and blending (messages it has SEEN between consecutive releases).
Blending is the anonymity set -- every broadcast reaches every node, so an
observer cannot tell which of them the relay forwarded. Gaps sampled at a release
are size-biased, so blending is rate*(2M+1)/3, twice the mean hold, not
rate*M/2 as a naive reading gives. Measured within 1-4% of that at M = 3, 10, 30
and linear in the cover rate.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Third review pass over the blend material.
Completeness:
- the simulator README documented neither frac_reached_live nor the three
correlated-churn columns (churn_mode, n_regions, region_locality) that every
run now writes, and its model section never described correlated outages at all;
- the knowledge graph had no pd nodes -- graphify update had never been run since
the simulator was added (2643 -> 2968 nodes).
Correctness/coherence:
- section 3.5 quotes coverage without saying which coverage, now that 3.9
distinguishes all-node from live-network. It is all-node; under uniform churn
the two agree to 0.001, so nothing in 3.5 turns on it. Said so explicitly;
- 3.9 named its groups AS/region without noting that link latency ignores them.
Regions are failure and peering domains, not latency domains -- real co-located
nodes would also be faster, so the clustered delays are if anything pessimistic.
Redundancy:
- style.band_plot was dead: never called by any figure. Removed, with the two
imports it alone needed;
- the units sentence appeared verbatim in the header note and again opening the
model section. Dropped the second.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Uncorrelated churn alone was incomplete: real outages take out a datacentre, AS
or region as a unit. Adds failure domains and a correlated churn mode, plus the
metric needed to tell the two apart.
- n_regions / region_locality: nodes belong to equal-sized failure domains, and
a configurable share of each node peers inside its own domain. Locality is what
makes a failure domain a connectivity domain -- with region-blind peering,
dropping whole regions removes a uniformly random set of nodes and is
indistinguishable from uniform churn. The locality matchings keep the graph
exactly d-regular (they change where peers are, never how many).
- churn_mode = uniform | regional, swept per topology so both modes are compared
on the same graph at an identical dead-node count.
- frac_reached_live: coverage of the *responsive* network, alongside coverage of
all nodes. The two move in opposite directions under correlated failure, so one
number could not express the result.
Measured (degree 4, 20 domains, 75% locality, half the network dead): clustered
failure leaves the survivors fully connected -- live coverage 1.000 and delivery
equal to the live-relay rate, i.e. nothing lost to routing -- where the same
number of scattered failures gives 0.857 live coverage and loses delivery to
broken routes. Correlated outages are gentler on the survivors than uniform
churn, while stranding the dead domains. Verify check 8 anchors this.
Also, per review of the caveats: exact d-regularity is a protocol requirement
rather than a modelling simplification, and the timing-correlation adversary is
deferred because it is only meaningful once the network emits cover traffic,
which this simulator does not yet do.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Correctness/completeness pass over the blend material only (TSI untouched).
- report Model section (2) was missing two of the six axes: messaging
redundancy (R cascades, first-arrival combination) and the emission/linking
model (30 s stake-proportional cadence, what counts as linked) were defined
only inline in the findings;
- method note still claimed 200 rounds x 8 topologies, contradicting the 1000
x 8 the tables now come from;
- design guidance carried two superseded numbers: worst-case observation as
"+0.15 absolute" (it saturates at 1.000 at degree 8, f_adv 0.2) and the
redundancy example (0.34 -> 0.72, measured 0.342 -> 0.713);
- figure references were incoherent: Figs 2 and 14 were cited in the text but
never shown, and Fig 8 was shown but never cited. All 15 embedded figures are
now cited and all citations resolve;
- simulator README listed two parquets for smoke (there are three) and omitted
redundancy from the propagation/deanon column lists.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
smoke.yaml never set redundancy > 1, so the R-cascade aggregation and the two
redundancy figures were only covered by unit tests, never by the end-to-end run.
Adding redundancy: [1, 2] takes smoke from 17 to 19 of the 21 figure builders
(only delay_vs_N and the churn-percolation figure need grids smoke does not have).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Extends the pd Blend simulator along two axes the deanonymization model
opened up, adds the reports/blend/pd report of record, and fixes three
correctness defects found while reviewing the result.
Linkability over time (pd.linkability):
- time to link an emitter ~ 30s*ln(1/(1-alpha))/(stake*q): inversely
proportional to stake, so a 5% staker is linked in ~2 days and a 0.001%
staker only after ~27 years;
- time to certify a node's stake >= theta from the count of attributable
observations (relative precision ~1/sqrt(N)): sizing a node costs 100-400x
more than identifying it, and sub-0.1% stake is practically unlearnable.
Both are closed forms over the exact deanonymization rates and a
stake-proportional 30 s emission cadence, checked against a Monte-Carlo of
the emission process in verify.
Messaging redundancy (R independent cascades per emission, R = 1..4):
- `redundancy` knob threaded through config/rng/propagation/engine/metrics/
sweep; a node receives from whichever cascade reaches it first, so arrival
times combine element-wise. Delivery and capture both follow 1-(1-x)^R, so
redundancy trades reliability against anonymity and divides time-to-link
by ~R. Measured: delivery 0.34 -> 0.81 at 30% churn for R = 1 -> 4, while a
1%-staker's time to link falls 10 d -> 2.5 d.
- Redundancy buys NO coverage: a cascade only delivers if the sender could
already route to its relay, so every delivered cascade floods the sender's
own component. Coverage is flat in R to four decimals at every degree.
- Near the percolation threshold the cascades fail together rather than
independently, so redundancy under-delivers against 1-(1-p1)^R there.
Churn percolation (configs/percolation.yaml, verify check 7):
- the flood only crosses responsive nodes, so it lives on the responsive
sub-graph -- site percolation on a d-regular graph. A network survives churn
only up to u_c = 1 - 1/(degree-1); measured collapse lands on the predicted
threshold for every degree (3 -> 0.50, 6 -> 0.80, 16 -> 0.93), which inverts
into the sizing rule degree > 1 + 1/(1-u).
Correctness fixes:
- redundancy delay used the fastest cascade's own full delay, which
over-states it (min-max vs max-min); now the element-wise earliest arrival,
reducing exactly to the single-cascade model at R = 1 (test);
- the "redundancy improves coverage" claim was false in both the report and
the simulator README -- removed and replaced with the measured result;
- per-hop latency is degree-dependent (1.5 s at degree 16 to 2.7 s at degree
3), not a flat 1.6 s; and the worst-case observation figure was averaged
over degrees -- at degree 8 and f_adv = 0.2 it is 0.83 -> 1.000.
Statistics: round counts raised for resolution rather than speed -- 8000
rounds per cell in the main sweep, 9600 in the redundancy study, 6400 in the
percolation study, giving SEM <= 0.009 on every delivery rate and <= 0.04 s
on every delay mean. The previous redundancy grid (144 rounds/cell) produced a
non-monotonic delivery curve; it is now monotonic and within 0.015 of theory.
Adversary and deanonymization metrics remain closed-form and exact.
reports/blend/pd: the report of record -- peering-degree trade-offs across
speed, observation, eclipse, deanonymization and reliability, plus the
time-to-link, stake-inference, redundancy and churn-threshold sections, with
21 figures of record and an explicit sampling-error statement.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Static-graph simulator quantifying how a node's peering degree trades off
propagation speed, adversary exposure, deanonymization, and reliability in the
Blend network. Scales to 1e6 nodes (sparse CSR + sampled Dijkstra); the
adversary and deanonymization metrics are exact at every N.
Model (ms): seeded d-regular peer graph (matching-union), Blend cascade
(sender -> blend_hops timed-release mix relays -> final flood), geographic link
base + exponential transport jitter, per-node processing lag, free-running
release-clock mixing.
Metrics:
- propagation: full-delay mean/p50/p90/p99, path/broadcast split, coverage times
- reliability: message success-delivery-rate ~ (1-unresponsive_frac)^blend_hops
and flood coverage, with unresponsive nodes modelled as routing holes
- adversary (exact): observed/eclipsed fractions, random + worst-case placement
- deanonymization (exact): P(whole blend path adversarial) ~ f_adv^blend_hops,
and full deanonymization (path adversarial AND honest sender peered with an
adversary) = deanon_rate * observed_frac
Deterministic blake2b seed streams, three parquet tables, joblib parallelism,
memguard, an analytic verify harness, 50 unit tests, and an auto-installing
Makefile.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Both eta ceilings in sec 6.6 come from adversaries optimising something else
(revenue, reorg depth), so they bound eta from above without bounding the
damage from below. Optimising the estimate directly needs no ratio transform:
each transition consumes exactly one block-finding event, so minimising
D-hat = (canonical + p_ref * countable uncles)/events is a plain average-reward
MDP over the transition table that already carries the orphan counts. One
value-iteration pass, no bisection.
Unconstrained, the answer degenerates -- and usefully. The optimum is pure
abstention: publish nothing, adopt when overtaken, D-hat = 1 - alpha exactly,
revenue zero. That is sec 6.4's withholding, which the report already shows is
CORRECT measurement rather than mis-measurement, so the unconstrained objective
asks the wrong question.
The constrained one bites. Sweeping lam * (adversary blocks) - (contribution to
D-hat) enumerates policies; the line of interest is where revenue SHARE reaches
alpha, i.e. where attacking costs nothing versus mining honestly. At alpha=0.4
such a policy drives D-hat to 0.642 where the revenue-maximiser reaches 0.811
-- 17 points of extra deflation bought with the selfish premium alone. At 0.36
and 0.45 the gaps are 0.082 and 0.103. Below the 1/3 threshold nothing
profitable deflates, so the exposure starts exactly where selfish mining does.
This revises two claims that were about revenue but read as though they were
about the adversary in general: sec 6.7's "the adversary frontier is exactly
optimal selfish mining; no compounding lever remains" and sec 8.2's echo of it.
Both now say the PROFIT frontier is bounded and the estimator frontier is not
the same policy. Note the sweep parameter is deliberately non-monotone in
revenue -- selfish mining takes a bigger share of a smaller pie, so raw block
rate is maximised by honesty and large lam returns there; it enumerates
policies rather than tracing a path.
Also closes a fairness loop these findings opened. Sec 6.7(1) credits uncle
rewards with compensating orphaned honest producers, computed on the SM1 race
where every orphan is a first-fork block. Under a private chain 20-40% of the
honest blocks destroyed are unreferenceable by construction, so those producers
are uncompensatable at ANY w_u -- not underpaid because p_ref is low, but
unreachable because no valid block may name them. The fairness guarantee
inherits the same first-fork ceiling as the density repair. Logged as item 19,
flagged as a protocol-design question rather than something a schedule fixes.
_solve_mdp is refactored into _solve_reward/_greedy_policy/_stationary/
_policy_rates so both objectives share one implementation; optimal_policy_stats
reproduces its committed figures exactly (eta 0.4413, D-hat 0.9447/0.8111 at
alpha=0.4). 251 tests pass.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
With the SM1 adversary in the engine, the question item 5 could not ask is now
a measurement. It asked whether the honest-load cap needs margin when the
attack inflates orphaning, on the theory that owed uncles would defer past W.
They do not. Sweeping U x W x alpha against the attack (512 runs, N=1000,
k=256, 8 reps): at the design point and alpha = 0.3, D-hat/D reads
0.729/0.755/0.738/0.758 for U = 1/2/3/4 -- flat within noise -- and no attacked
cell reaches the 0.98 bar at any cap or either window. The honest baseline in
the same sweep reproduces sec 3.4 exactly (U=1 clears at delta=8; delta=16
needs U=2 at W=10 or W=20 at U=1), which is a useful check that the engine
adversary has not disturbed the honest regime.
Splitting the honest orphans by WHY they went unreferenced explains it. Neither
existing metric separates the two causes -- p_ref mixes them, and
deep_ref_share is 0 by construction here because the proposer's candidate
filter drops deep-fork blocks before any reference to one is proposed -- so the
script walks the tree. Countable share (first block of its fork): 97% honest,
76-81% at alpha=0.2, 59-72% at alpha=0.3. Referenced OF those: 90-93% honest,
84-93% and 80-88% under attack. The queue drains at essentially the honest rate
whatever the cap; what collapses is eligibility. An override discards a CHAIN
and only its first block has a parent on the surviving chain, so 20-40% of the
honest work destroyed is unreferenceable by construction. U governs drain
capacity for candidates that exist; it cannot manufacture eligibility.
So U = ceil(rho) + 1 stands unchanged and needs no adversarial margin -- and
the one place the cap does matter is the honest-load reason it was sized for
(U=1 -> 2 lifts the referenced-of-countable rate from 84% to 93% at
alpha = 0.2, then U=4 adds nothing).
This is the fig36 first-fork ceiling reached from an independent direction: a
per-node network simulation with real delays and a real queue, versus a
stationary MDP. Two models sharing no code, agreeing on direction and rough
size, is the strongest available evidence that the ceiling is a property of the
counting rule rather than of either model. Recorded in sec 6.6 and sec 6.8, with
the sec 6.8 structural argument corrected: it holds for orphans that are
referenceable, but a private chain buries most of them out of reach.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sec 6.8 recorded that "the per-node engine has no private-chain strategy", which is
why every selfish result came from the global race model with uncle recovery as
a free knob eta -- and why open item 5 (does the uncle cap need margin under
attack-inflated orphaning?) could not be sized: a knob has no queue to overflow.
adversary_strategy="selfish" adds it. The coalition mines one shared private
chain and releases under the classic SM1 rules in (a, h) form: adopt when the
public chain wins, match at equal length, override at a one-block lead, else
wait. Only VISIBILITY is modelled -- the coalition's mining needs no special
case, because a member's fork choice already builds on the private tip whenever
it leads (that tip has the greatest height among blocks the member can see) and
falls back to the public chain exactly when the public chain overtakes, which
is the adopt branch. So the private chain forms, extends and is abandoned
emergently, and the code that had to be written is the arrival matrix.
Design notes worth keeping:
- Private blocks reuse the sentinel `withhold` already had (never-arrives), so
the existing exclusions from canonical-tip selection apply unchanged; release
flips it back and gossips DIRECTLY from the producer, bypassing Blend, since
an adversary has no privacy budget to respect and wants the race won.
- A private chain breaks the windowed horizon's premise (a hidden block is old
enough to look fully-propagated while no honest node has it, and it becomes
visible LATER, which the one-way frontier pointer cannot revisit), so selfish
forces the exact full scan and full matrix.
- Blocks still hidden at epoch end are abandoned and hidden from the coalition
too, or the canonical-tip search would crown a chain no honest node saw.
Validated against Eyal-Sirer at sub-slot latency: revenue share 0.0356 vs an
exact 0.0356 at alpha = 0.1, and above the closed form at higher alpha by just
the margin the alpha_eff fork-amplification correction predicts (0.498 vs 0.484
at alpha = 0.4, with fork rate 0.38).
Adds p_ref_honest: the reference rate over orphans produced OUTSIDE the
coalition. Under a private-chain attack this diverges sharply from p_ref, and
only the honest one measures the repair the report credits to uncle counting --
an attacker's own discarded blocks are its loss to bear.
test_fork unpacks fork_stats positionally, so its three call sites take the new
fifth value. 247 tests pass.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Correctness/completeness review of the report and simulator. Verified against
the committed parquets: the sec 6.6 countable-ceiling table (cap-64 MDP sweep),
sec 6.10 Result 4's depth ceilings, the sec 3.4 uncle-selection table, all
adversary-variant numbers, the rho-boundary row-4 quotes (0.976 at rho=0.91,
4-sigma shortfall at 0.96, max cell 1.0024), and the sec 8.4 capstone table.
Three defects found, all fixed:
1. The collapse event was not reproducible from the committed script. Study D
swept only the default (random) coalition, but the one observed collapse is
a whale cell; the "once in 144 runs" count came from an ad-hoc probe. The
committed sweep now carries the selection axis (96 runs) and reproduces the
event: 1/12 in the whale 50% cell at delta_max = 8, never at 4. All six
fold-related passages now quote the committed sweep, which also retires the
stale "the full dynamics never reach it" wording in the sec 6 arc, the
sec 6.2 intro, row 6 and item 1 -- text that contradicted item 18 since
yesterday's finding.
2. capstone.py's printout could not reproduce the report's sec 8.4 table. The
report's numbers are a per-replicate-tail aggregation (each replicate burns
in against its own early-stop length); the script cut the tail at the ARM's
max epoch, silently dropping any replicate that stopped earlier (7 of 8 in
the adversary arm) and landing one rounding step off on three cells. The
script now aggregates per replicate and prints the SEM; against the existing
parquet it reproduces the table exactly (1.001/0.998, 0.342+-0.009 /
0.343+-0.005, p_ref 1.000/0.990, 8 reps both arms). The report table was
right all along; sec 6.8's p_ref quote (0.989, the per-arm value) is aligned
to 0.990.
3. Small report fixes: slow-beta deflation rounded 0.765 -> "0.77" (now 0.76);
fig13's caption now points at the fig36 ceiling instead of implying free
recovery; row 5 cites the measured slow-beta standing deflation; the
canonical-data paragraph lists the new studies' artifacts; the simulator
README's layout block lists the new tests and scripts.
Adds a unit test for reorg.countable_recovery_from_depths (the one new
function that had none). 236 tests pass; the new-study parquets are copied to
the main checkout's runs/, where every other study's data of record lives.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Whale coalitions, jitter > 0 and very slow beta were the residual "untested
adversary variants" of open item 11. None moves a conclusion:
- Concentration does not change the deflation (suppression D-hat within noise
at every stake), and a whale coalition reproduces the sec 6.4 withholding law
D-hat -> (1-beta_adv) more cleanly than a random one: 0.9005/0.6997/0.5010
against a predicted 0.9/0.7/0.5. The "lumpier share statistic" worry points
the other way, and for a reason that is about coalition CONSTRUCTION rather
than concentration: a random coalition grows until its stake first reaches
the target, so the last node added overshoots by its own size -- a whale,
under a Pareto tail. Realised block share at a nominal beta_adv = 0.1 is
0.137 +- 0.108. Logged as item 17: the beta_adv axis is a nominal target.
- jitter up to 1 slot changes nothing under attack (notch 0.390 -> 0.410,
attacker share flat, range_ratio identically 0), as sec 6.1 found honestly.
- Slow beta shrinks the notch (0.415 -> 0.080 for beta 1 -> 0.1) at flat
attacker take, but sinks the MEAN estimate to 0.765 at beta = 0.05: the
estimator can no longer track back up during the honest half of the cycle.
Slowing beta buys the defender nothing on either axis.
Unplanned: study A blew past the memory guard, which turned out to be the
sec 6.2 fold being reached. The mechanism is sec 6.2's own -- rho_eff = rho/r,
and withholding deflates r by design, so a 50 % coalition doubles the load onto
rho_eff ~ 1.1 at the design point. Swept directly, the estimate collapses once
in 144 runs at delta_max = 8 (a concentrated 50 % coalition) and never at
delta_max = 4. That retires "not an observed dynamical trap" but is one event,
so the claim is stated as a rare tail and the rate is logged unmeasured as
item 18.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Open item 11 listed "a random (rather than oldest-first) uncle-selection draw"
as an untested spec sensitivity. The spec does not leave it open: Uncle
Selection in cryptarchia-v1-protocol.md has the proposer take the oldest
candidates first, deterministically, because an uncle expires w_u slots after
its own slot. That is exactly what every result in the report already uses, so
the item is a conformance match, not a gap -- and the simulator comment calling
uncle_random_p "the spec's unbiased coin" cites text the spec no longer has.
What is genuinely open is deviation FROM that rule: selection is proposer-local
and the uncles field is never validated. configs/uncle-selection.yaml measures
the cost. A proposer that includes each candidate on a fair coin instead loses
up to 0.10 in D-hat/D, and 0.063 at the recommended W = 10 once rho ~ 1
(0.902 vs 0.965, t = -8.6). At the design point the margin survives but is
spent: 0.980 vs 0.997 against a 0.98 bar. The loss does not close as W grows,
because a coin wastes opportunities rather than queue capacity and a well-sized
window is precisely what keeps the queue short enough for that to bite.
This matters for the sec 8.5 reward recommendation: the spec argues a proposer
has no incentive to deviate BECAUSE uncles grant no reward, and paying them
removes that argument.
Also adds adversary_selection=whale (the largest holders at matched stake, for
the untested concentration case). The marker is appended to key() only when
non-default so every historical run's seed stays byte-identical, guarded by a
test alongside the paired_streams one.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The countable model can reference only the first block of a fork, so a
discarded chain of h honest blocks yields one countable uncle, not h. Sec 6.6
reads the estimator repair off a free knob eta and quotes it at eta = 1 --
attainable under SM1, which acts the moment the honest branch reaches length 1
and so never buries a second block. The optimal SSZ policy waits and does bury
them, and there the deployed counting rules cap eta at 0.44 (alpha = 0.4,
gamma = 0), landing D-hat at 0.81 rather than the 0.94 an unrestricted count
gives -- and the ceiling degrades with alpha while the unrestricted value
improves. So SM1 is a faithful proxy for selfish-mining revenue (0.484 vs
0.489) but not for TSI's estimator damage.
selfish_mdp: carry per-branch orphan counts on the transition table so the
accounting cannot drift from the race logic; optimal_policy_stats solves the
policy's stationary distribution for per-event canonical/orphan rates. The
per-event rates sum to 1 (every block is canonical or orphaned), which the
tests assert as an independent check on the whole derivation.
reorg: the same ceiling for the depth-maximising adversary -- 0.52 at
alpha = 0.30 with the measured honest fork rate -- reached from the other
direction. Neither adversary optimises deflation directly, so both ceilings
are upper bounds on eta; that gap is logged as open item 16.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
§3.2 rested on 5 unpaired replicates while §3.2a used a paired design.
Re-running the delta_max 4/8/16/32 grid with common random numbers and
20 replicates (configs/countable-vs-old-paired.yaml) changes the answer
at the design end.
The cost is resolved at EVERY delay and grows monotonically with load:
delta_max=4 (rho 0.36) -0.0013 t= 4.0 (unpaired: not resolved)
delta_max=8 (rho 0.56) -0.0034 t= 9.8 (unpaired: not resolved)
delta_max=16 (rho 0.96) -0.0102 t=17.5
delta_max=32 (rho 1.76) -0.0228 t=22.4
So §3.2's claim that "at the operating loads (rho < 1) no difference
between the models is detectable at all" was an artefact of the weak
design, not a property of the system. There is a difference; it is just
small — 0.13% and 0.34% at the two sub-unit loads. The new delta_max=4
figure (-0.0013 at 20 reps) independently reproduces §3.2a's (-0.0011 at
40 reps) from a separate sweep.
11 of 12 U>=1 cells resolve individually; max t = 29.0 against a
Bonferroni threshold of 2.87 for twelve tests. The U=0 control is exact:
80/80 replicate pairs differ by 0.0.
New finding at U=1 under overload: the sign FLIPS and the countable rule
wins, +0.0127 (t = 7.6), positive in 19 of 20 pairs. Both models have
collapsed at rho ~ 1.76 with a single uncle slot, but when capacity is
the binding constraint the countable rule's occupied-slot exclusion
means its one reference always recovers a NEW slot, while the
unrestricted rule dedups by block id and can spend that reference on an
orphan whose slot is already counted. Measured recovery agrees:
q_u = 0.591 countable vs 0.579 unrestricted. The slot-vs-block
distinction of §2.1 is worth most exactly where references are scarcest.
Code: paired_gaps and pooled_by_delay move from scripts/plot_fine_delay
into figures_pernode so both plot scripts share one implementation;
plot_countable_vs_old now detects paired runs and uses the
per-replicate difference, falling back to the unpaired two-sample test
otherwise. Two hardcoded reporting values fixed — the Bonferroni
threshold was pinned to 2.935 and printed nan for any grid that was not
15 cells, and a per-cell comparison line had a hardcoded /15 denominator;
both now derive from the grid actually run.
Figures 30-32 regenerated from the paired grid. Tests: 214 passed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The report is now reports/tsi/README.md, so browsing to reports/tsi/
lands on the report itself rather than on an index that points at it.
The old index carried nothing the report lacked except the note that
report-figures/ holds the figures of record (the simulator folder does
not commit its own), which is folded into §9; its section table is
superseded by the report's own contents block. The "[Index]" self-link
in the header is replaced by the simulator link the index used to carry.
The contents block was inconsistent: §2 listed subsection titles, §3 and
§6 listed bare numbers with no titles at all, and the appendices were
crammed onto one line while their subsections went unlisted. Rebuilt
from the document's actual headings so every entry has a real title,
top-level entries carry a one-line gloss, and subsections sit indented
under their parent. It now covers all 47 anchors, including B.1-B.4 and
C.1-C.2 which were previously absent.
scripts/build_html.py follows the rename (DOCS is a single document) and
still renders clean: 47 anchors, 0 broken internal links, 0 unrewritten
.md links, 37 images.
Also adds configs/countable-vs-old-paired.yaml — the paired, 20-replicate
version of the overload grid. §3.2a now rests on a paired design while
§3.2 still rests on 5 unpaired replicates, which is why its U=1 cells at
delta_max 16 and 32 sit unresolved at t ~ 0.5 against a replicate sd of
0.15. That sweep is running; the report is not yet updated from it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The unpaired comparison could not answer the question it was asked. The
two uncle models draw independent RNG streams -- uncle_model is in the
config key, which is what makes --old bit-reproduce earlier runs -- so
the arms differed in stake draw, peering graph and every lottery
outcome, each comparison paid the between-run variance twice, and the
per-cell floor (+-0.0015) sat an order of magnitude above the effect.
Only delta_max = 5 resolved, and only after pooling.
Adds `paired_streams`: the RNG root is derived from the model-
independent part of the key, so a countable cell and its --old twin get
the SAME stake, graph and lottery draws and the uncle rule is the only
difference. Each replicate is then a matched pair and the shared
variance cancels. Trajectories still diverge after epoch 0 through the
genuine feedback (a different counted density changes the next epoch's
difficulty), which is the signal.
The flag is deliberately NOT in key(): it selects which key the seed is
derived from, so including it would perturb every historical seed.
Re-verified that --old still bit-reproduces the committed 2026-07-27
rho-boundary parquet, max |delta| = 0.
Results (configs/fine-delay-paired.yaml, 40 replicates per arm):
- Negative control becomes an IDENTITY check. With U = 0 no reference is
taken, so shared streams must give bit-identical trajectories. All 200
replicate pairs differ by exactly 0.0. Unpaired, the same control only
had to agree within +-0.025 and drifted by 0.016.
- Per-cell SE shrinks by a median 1.6x (1.2-2.1x); widest 95% CI goes
+-0.0015 -> +-0.0010. 5/15 cells resolve at |t| >= 2 (0.75 expected by
chance); the largest, U=2 at delta_max=4, is t = 4.32 and clears
Bonferroni for 15 tests.
- The cost is a STEP, not the ramp the unpaired data suggested:
delta_max 1-3 unresolved (t = 1.1, 1.8, 1.4), then delta_max 4 AND 5
both resolve at -0.0011 (t = 4.7) and -0.0009 (t = 3.7). Whole-band
pooled -0.00060 +- 0.00021, t = 5.7 -- where the unpaired estimate of
the same quantity (t = 2.8) had failed correction.
So the first-fork restriction costs nothing measurable up to
delta_max = 3 and about 0.1% at 4-5 -- an order of magnitude below the
+-0.9% per-epoch sampling noise.
Two bugs found while building this, both of which would have silently
produced a wrong answer:
- paired_streams was missing from metrics._CONFIG_FIELDS, so it never
reached the parquet; plot_fine_delay.py falls back to the unpaired
test when it cannot confirm pairing, so the sweep would have completed
and quietly reported the old result. Caught before the run finished;
the sweep was restarted and a test now pins the field.
- The U=0 control check reported FAILS on a PERFECT control: paired, the
gap is exactly 0 so its SE is 0 and t is 0/0. It now checks the gap
itself when the streams are shared, and falls back to the t-test only
when there is real spread.
§3.2a is rewritten around the paired measurement; the unpaired sweep is
retained in §9 as the power comparison that motivated it. Figures 34-35
regenerated, with the control annotation and provenance reflecting the
design actually used.
Tests: 214 passed (was 209). ruff clean.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The four-part split existed because the single report had grown dense
and heavily cross-referenced; splitting traded that for a different
cost, which the merged read makes visible. Section numbers (§1–§9,
Appendices A–C) were already the stable identifiers, so the parts were
a packaging choice, not a structural one.
reports/tsi/tsi-report.md is now the whole report. Parts are
interleaved back into section order — §1, §2–§5, §6, §7–§8, §9 +
appendices — which is NOT concatenation order: Part 1 carried §1, §7
and §8, so appending files in sequence would have put §7–§8 ahead of
§2. Every cross-file link collapses to an internal anchor; all 47
anchors resolve, all 37 figure embeds resolve, and no line of prose was
lost (verified by diffing normalised content lines with link targets
stripped — 0 lost, additions are the new header and table of contents).
Coherence fixes the merge exposed, all artefacts of the split:
- The roadmap paragraph described "four parts (see the index)" and is
now a section-order roadmap, with its circular self-link to §1
dropped.
- §7's figure-location note pointed readers at "the other parts". It
now names the actual sections, and it was also WRONG about three
figures: fig17/fig18/fig21 are in Appendix C and figB1/figB2 in
Appendix B, not §9. It had also never been updated for fig30–fig35.
- §9's "throughout this part" is now "throughout".
README.md becomes a proper index — a section table pointing into the
one document — rather than a list of four files.
scripts/split_report.py is deleted: a one-time migration that produced
the split, now both obsolete and pointing the wrong way.
scripts/build_html.py was already broken before this change — it still
read the report from tsi-sim-pernode/, where the files stopped living
when they moved to reports/tsi/. Retargeted at reports/tsi/ and the
single document; verified end-to-end (0 broken internal anchors, 0
unrewritten .md links, 37 images in the rendered HTML). Its output is
now gitignored, as its docstring always claimed it was.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Two findings from re-reviewing the fine-delay section.
1. The rho values I put in s3.2a were wrong. The report derives
rho = f*D_vis with D_vis = hops*delta_max/2 + (hops+1)*ell_mean from
a MEASURED ell_mean (1.211 slots at N=1000/degree=6), not from the
link_latency_mean parameter (0.5). Hand-substituting a guessed 1.5
inflated every value by ~0.04: the band is rho 0.21-0.41, not
0.25-0.45.
To stop that recurring, graph_ell_mean moves out of
rho_boundary_analysis.py into figures_pernode.py, joined by a new
rho_for() that both scripts and any future quotation go through;
plot_fine_delay.py now prints the derived rho per delay.
This also exposed an inconsistency in the existing s3.2 table, which
rounded delta_max=4 to "rho ~ 0.4" while s3.2a called the same cell
0.36 and prose elsewhere already used 0.56 for delta_max=8. The s3.2
column now carries the derived values (0.36/0.56/0.96/1.76).
2. Testing each cell against the exact target 1.0 -- the same question
the gap test asks, without reference to the other model --
corroborates the first-fork onset independently. Unrestricted: 1/15
cells below 1 (t=-2.09, chance). Countable: 4/15, and not scattered
-- delta_max=4 at U=1, and ALL THREE caps at delta_max=5 (-0.0012 to
-0.0019, t=-2.5..-3.7). A shortfall appearing at every cap at once,
only at the top of the band, only under the restricted model, is the
first-fork cost seen absolutely.
That makes "one uncle slot is sufficient -- not approximately,
exactly" too strong as I had written it. s3.2a now states the
residual (0.1-0.2% at the top of the band, zero below delta_max=3),
reconciles it with the s1 headline, and notes that since all three
caps show the same shortfall the residual is not a capacity limit.
The bound quoted in s1 moves from "below 0.15%" to "<= 0.2%".
Also adds the new run directories to s9's canonical list, which covered
every other study but not these.
Tests: 209 passed. ruff clean.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Acts on a correctness/completeness review of the countable uncle model
and its report material.
Correctness fixes in the report:
- s3.4 quoted 0.998 for W_abs=10 at the 8s budget; the run says 0.9963.
- s1 claimed both models >= 0.996 at U >= 1; countable U=2 delta=8 is
0.9955. Corrected to >= 0.995.
- The s3.2 table presented two cells (U=1 at delta 16 and 32) as model
differences. They are not resolvable: t = 0.46 and 0.47 over 5
replicates. The table now carries +-SEM and a t per cell.
- s3.4 claimed the ~7-block-interval floor "carries over unchanged".
Accuracy is still climbing past W=7 at every delay (8s: 0.989 ->
0.996), so the claim is dropped. The 32s curve is non-monotonic with
replicate SD up to 0.22 and is now flagged as noise, not a trend.
- 1-r was attributed to the first-fork restriction alone; it is the
combined first-fork and capacity loss, which this measurement cannot
separate. Hedged to match fig32's own axis label.
Completeness: the U=0 negative control was swept but never reported.
With no uncles the two models are identical by construction, yet they
differ by -0.23 at delta_max=32 (t=2.1) because they draw independent
RNG streams. That is the noise floor the rest of the grid must clear,
and it is now in s3.2, s9, fig30 and the config header.
New study (configs/fine-delay.yaml, scripts/plot_fine_delay.py, s3.2a,
fig34/fig35): the design band delta_max 1-5 at 40 replicates, both
models. Findings: every U >= 1 cell of both models lands in
0.998-1.001, flat in delay, while U=0 decays 0.810 -> 0.640. No
individual cell resolves a model difference (widest 95% CI +-0.15pp;
max t=2.59 vs Bonferroni 2.94 over 15 cells). Pooled across uncle caps
the first-fork cost is monotone in delay and separates from zero only
at delta_max=5 (-0.0014 +- 0.0007, t=3.7) -- below 0.15% everywhere in
the band, against +-0.9% per-epoch sampling noise.
Code:
- deep_ref_share is identically 0 on every real countable run: for a
chain block B the producer's chain below B is the counting chain
below B, so the counting-side parent-on-chain re-check cannot reject
what selection emitted. It is a drift alarm, not a rate. Documented
as such in measure.py, the plot docstring and the config header, and
pinned by a new end-to-end test.
- Removed annotate_uncles: a second countable implementation that
production never called, while carrying most of the selection test
coverage. Tests now drive select_uncles_at_production through an
annotate_via_production replay helper -- same assertions, live path.
- Added tests for the two previously uncovered branches of the live
selection: the pmin/below chain walk that resolves parent-on-chain
for candidates whose parent sits below the window, and the
occupied-slot exclusion built from the chain walk.
- theory.q_effective and theory.window_miss_prob were unused and
untested. Now used (the prediction figure reconstructs q_u through
the identity the report quotes) and tested. The window_miss_prob test
records that its "~ e^-W" docstring is the f->0 limit: the true decay
is e^-1.017W at f=1/30, 16% off by W=10.
- Shared sem()/recovery_rate() moved into figures_pernode.py; fig30 and
fig33 regenerated with SEM error bars and the U=0 control curve.
- Fixed the pre-existing E501 in bootstrap_dynamics.py; ruff clean.
Report prose reworked to read standalone: the countable model is
described as the rules under analysis and the former model as a
labelled "unrestricted" comparison baseline, with no dated banners and
no round-to-round narration.
Tests: 209 passed (was 202).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Implement the countable uncle model from the Cryptarchia spec's
counting-only reference rules, and make it the simulator default.
Counting rules (uncles.py, measure.py):
- Only the first block of a fork (parent on the producer's chain) is
referenceable and countable, which makes every reference verifiable
from chain data alone.
- The reference window is derived from a window-absorption parameter,
w_u = W_abs/f slots (W_abs in expected block-intervals, default 10,
bounded W_abs <= 0.6*k), replacing the free-standing uncle_window.
- Selection skips slots already occupied on the producer's chain and
takes at most one uncle per slot.
- The measurement pass re-checks every rule per reference and tallies
rejections as deep_ref_share.
The pre-redesign model is preserved behind --old on tsi-sweep and
tsi-verify. Its RNG key is byte-identical to the pre-uncle_model key,
so --old bit-reproduces the historical runs.
Supporting changes: uncle_model and window_absorption config surface
with validation (config.py, constants.py); accuracy closed form over
the effective q_u (theory.py); plumbing through tsi.py, epoch.py,
sweep.py, blocktree.py, metrics.py, verify.py, figures_pernode.py.
Studies and figures:
- configs/countable-vs-old.yaml -- delay x U grid, run under both
models on the same grid.
- configs/absorption-window.yaml -- accuracy vs W_abs at U=1.
- scripts/plot_countable_vs_old.py renders fig30-fig33 into
reports/tsi/report-figures/.
Tests: tests/test_countable_counting.py (7 cases) covering first-fork
eligibility, derived-window bounds, occupied-slot exclusion, and
per-reference re-checking; extensions to test_uncles.py,
test_config.py, test_slot_counting.py. Full fast suite: 202 passed.
Also adds CLAUDE.md (graphify project instructions) and ignores
editor/local-agent state plus the vendored Equi-X benchmark clone.
The reports/tsi/ prose describing this model is held back for a
separate editorial pass.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Applied the reconstructed round-4 review to the TSI parameter-selection report
set (reports/tsi) and executed the follow-ups.
Report (reports/tsi):
- Applied the must+should findings across README + parts 1-4: cross-part numeric
corrections, figure-caption fixes, spec reconciliation, and cross-file companions
(hops-degradation and notch/reward numbers, tip-agreement ordering, density-window
timing, VRF -> ZK Proof-of-Leadership, w_u window/reward gloss).
- Editorial pass for timeless voice (no "now adopted / merged / coin" narration) and
a gentle spec-safety framing (recommendations are thresholds; the protocol's
MAX_UNCLES=4 sits safely above them).
- Added the fork-rate-vs-scale table (6.10), defined "grinding gain", promoted the
clock-skew study to its own paragraph, added the correlated-latency caveat, and
moved fig27/fig28 beside their discussion.
- Documented the Blend cascade in 2: hops propagate over the shared gossip graph
(not direct links), the final broadcast comes from the last relay, relays are
blind forwarders.
Simulator (tools/simulators/tsi/tsi-sim-pernode):
- Docstring/dead-code fixes: theory.block_count_ceiling (legacy framing), measure,
reorg (catch-up reading), metrics (removed two dead helpers), config (fixed_point
10^-6; clock_skew_max/lottery_chunks documented inert), stake_vs_delay.
- Generator correctness + regenerated figures: figures_pernode.CONFIG_COLS now
exhaustive (f no longer pooled); rho_boundary_analysis SEM across replicates +
hollow floored markers + de-hardcoded ell_mean (measured from the run's graph);
appendix_fluct per-N sigma + ~18x title (figB2); bootstrap_dynamics driving
estimate so fig1 epoch-0 matches genesis.
- pytest: 186 passed; report links 528/0 dangling.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
