5 Commits

Author SHA1 Message Date
Marcin Pawlowski
f1433cbbb3
Anchor the uncle reference window to the parent, not the uncle
The spec bounds an uncle's own slot (0 < sl_A - sl_U <= w_u) but leaves its
PARENT unconstrained beyond lying on the referencing chain. So a block minted
NOW, built on a chain block from arbitrarily far back, is a legal first-fork
uncle: recent by its own slot, ancient by its parent's. Verifying it means
deriving the epoch state and ledger root as of that ancient parent, per
reference, and those are precisely the inputs the counting rules require -- so
the work cannot be amortised. It costs the adversary nothing beyond lottery
wins it already has; it just builds them somewhere useless.

Measured with a deep_parent coalition. At the deployed operating point a 30%
adversary moves the MEDIAN counted reference's reach from 54 slots back to
20,144, and the worst case to 76,778 -- the epoch boundary, ~21 hours of
history, ~256x the nominal window. It is not a tail effect.

The fix is a SUBSTITUTION, not an additional rule. A block strictly postdates
its parent and a referenced uncle strictly precedes its referencer, so
sl_A - sl_U < sl_A - sl_parent(U) <= w_u: bounding the parent bounds the uncle
for free, and a both-windows variant would be identical to the parent one. Both
invariants are pinned in a new test_slot_ordering.py rather than argued -- the
user asked to confirm sl_A > sl_U explicitly, and it turns out to be
load-bearing for the whole implication, so it is tested at three geometries
plus a hand-built counting case.

Under the parent anchor the same coalition reaches 292/300/300 slots at
delta_max 4/8/16 -- capped by construction. Honest recovery is unaffected:
0.9993 -> 0.9999, 0.9969 -> 0.9986, 0.9791 -> 0.9858, no loss anywhere within
one to two SEM, because a latency orphan's parent is recent by construction.

One finding that sharpens the case: at delta_max = 16 the HONEST uncle-anchored
arm already reaches 315 slots, past its own w_u = 300. Under the current rule
w_u is not a bound on validation reach even with no adversary present. It only
becomes a state-retention bound once anchored to the parent.

Recorded as sec 6.12 with fig38, a new row in the sec 8.5 spec deltas, both new
knobs in sec 7, and the study in sec 9. uncle_window_anchor and the deep_parent
strategy are appended to the RNG key only when non-default, so no committed run
is reseeded.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-07 12:22:50 +02:00
Marcin Pawlowski
88f31340ad
Uncle selection: the spec fixes oldest-first, so measure deviation from it
Open item 11 listed "a random (rather than oldest-first) uncle-selection draw"
as an untested spec sensitivity. The spec does not leave it open: Uncle
Selection in cryptarchia-v1-protocol.md has the proposer take the oldest
candidates first, deterministically, because an uncle expires w_u slots after
its own slot. That is exactly what every result in the report already uses, so
the item is a conformance match, not a gap -- and the simulator comment calling
uncle_random_p "the spec's unbiased coin" cites text the spec no longer has.

What is genuinely open is deviation FROM that rule: selection is proposer-local
and the uncles field is never validated. configs/uncle-selection.yaml measures
the cost. A proposer that includes each candidate on a fair coin instead loses
up to 0.10 in D-hat/D, and 0.063 at the recommended W = 10 once rho ~ 1
(0.902 vs 0.965, t = -8.6). At the design point the margin survives but is
spent: 0.980 vs 0.997 against a 0.98 bar. The loss does not close as W grows,
because a coin wastes opportunities rather than queue capacity and a well-sized
window is precisely what keeps the queue short enough for that to bite.

This matters for the sec 8.5 reward recommendation: the spec argues a proposer
has no incentive to deviate BECAUSE uncles grant no reward, and paying them
removes that argument.

Also adds adversary_selection=whale (the largest holders at matched stake, for
the untested concentration case). The marker is appended to key() only when
non-default so every historical run's seed stays byte-identical, guarded by a
test alongside the paired_streams one.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-05 15:42:16 +02:00
Marcin Pawlowski
4baccd8d4b
Paired design: resolve the design band with common random numbers
The unpaired comparison could not answer the question it was asked. The
two uncle models draw independent RNG streams -- uncle_model is in the
config key, which is what makes --old bit-reproduce earlier runs -- so
the arms differed in stake draw, peering graph and every lottery
outcome, each comparison paid the between-run variance twice, and the
per-cell floor (+-0.0015) sat an order of magnitude above the effect.
Only delta_max = 5 resolved, and only after pooling.

Adds `paired_streams`: the RNG root is derived from the model-
independent part of the key, so a countable cell and its --old twin get
the SAME stake, graph and lottery draws and the uncle rule is the only
difference. Each replicate is then a matched pair and the shared
variance cancels. Trajectories still diverge after epoch 0 through the
genuine feedback (a different counted density changes the next epoch's
difficulty), which is the signal.

The flag is deliberately NOT in key(): it selects which key the seed is
derived from, so including it would perturb every historical seed.
Re-verified that --old still bit-reproduces the committed 2026-07-27
rho-boundary parquet, max |delta| = 0.

Results (configs/fine-delay-paired.yaml, 40 replicates per arm):
- Negative control becomes an IDENTITY check. With U = 0 no reference is
  taken, so shared streams must give bit-identical trajectories. All 200
  replicate pairs differ by exactly 0.0. Unpaired, the same control only
  had to agree within +-0.025 and drifted by 0.016.
- Per-cell SE shrinks by a median 1.6x (1.2-2.1x); widest 95% CI goes
  +-0.0015 -> +-0.0010. 5/15 cells resolve at |t| >= 2 (0.75 expected by
  chance); the largest, U=2 at delta_max=4, is t = 4.32 and clears
  Bonferroni for 15 tests.
- The cost is a STEP, not the ramp the unpaired data suggested:
  delta_max 1-3 unresolved (t = 1.1, 1.8, 1.4), then delta_max 4 AND 5
  both resolve at -0.0011 (t = 4.7) and -0.0009 (t = 3.7). Whole-band
  pooled -0.00060 +- 0.00021, t = 5.7 -- where the unpaired estimate of
  the same quantity (t = 2.8) had failed correction.

So the first-fork restriction costs nothing measurable up to
delta_max = 3 and about 0.1% at 4-5 -- an order of magnitude below the
+-0.9% per-epoch sampling noise.

Two bugs found while building this, both of which would have silently
produced a wrong answer:
- paired_streams was missing from metrics._CONFIG_FIELDS, so it never
  reached the parquet; plot_fine_delay.py falls back to the unpaired
  test when it cannot confirm pairing, so the sweep would have completed
  and quietly reported the old result. Caught before the run finished;
  the sweep was restarted and a test now pins the field.
- The U=0 control check reported FAILS on a PERFECT control: paired, the
  gap is exactly 0 so its SE is 0 and t is 0/0. It now checks the gap
  itself when the streams are shared, and falls back to the t-test only
  when there is real spread.

§3.2a is rewritten around the paired measurement; the unpaired sweep is
retained in §9 as the power comparison that motivated it. Figures 34-35
regenerated, with the control annotation and provenance reflecting the
design actually used.

Tests: 214 passed (was 209). ruff clean.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-05 11:58:26 +02:00
Marcin Pawlowski
bd2ac7b7be
Countable uncle model: spec counting rules, sweeps, figures
Implement the countable uncle model from the Cryptarchia spec's
counting-only reference rules, and make it the simulator default.

Counting rules (uncles.py, measure.py):
- Only the first block of a fork (parent on the producer's chain) is
  referenceable and countable, which makes every reference verifiable
  from chain data alone.
- The reference window is derived from a window-absorption parameter,
  w_u = W_abs/f slots (W_abs in expected block-intervals, default 10,
  bounded W_abs <= 0.6*k), replacing the free-standing uncle_window.
- Selection skips slots already occupied on the producer's chain and
  takes at most one uncle per slot.
- The measurement pass re-checks every rule per reference and tallies
  rejections as deep_ref_share.

The pre-redesign model is preserved behind --old on tsi-sweep and
tsi-verify. Its RNG key is byte-identical to the pre-uncle_model key,
so --old bit-reproduces the historical runs.

Supporting changes: uncle_model and window_absorption config surface
with validation (config.py, constants.py); accuracy closed form over
the effective q_u (theory.py); plumbing through tsi.py, epoch.py,
sweep.py, blocktree.py, metrics.py, verify.py, figures_pernode.py.

Studies and figures:
- configs/countable-vs-old.yaml -- delay x U grid, run under both
  models on the same grid.
- configs/absorption-window.yaml -- accuracy vs W_abs at U=1.
- scripts/plot_countable_vs_old.py renders fig30-fig33 into
  reports/tsi/report-figures/.

Tests: tests/test_countable_counting.py (7 cases) covering first-fork
eligibility, derived-window bounds, occupied-slot exclusion, and
per-reference re-checking; extensions to test_uncles.py,
test_config.py, test_slot_counting.py. Full fast suite: 202 passed.

Also adds CLAUDE.md (graphify project instructions) and ignores
editor/local-agent state plus the vendored Equi-X benchmark clone.

The reports/tsi/ prose describing this model is held back for a
separate editorial pass.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-04 18:48:46 +02:00
Marcin Pawlowski
24da2fc8b3
Importing tsi-sim v3 2026-07-30 18:57:10 +02:00