3 Commits

Author SHA1 Message Date
Marcin Pawlowski
4baccd8d4b
Paired design: resolve the design band with common random numbers
The unpaired comparison could not answer the question it was asked. The
two uncle models draw independent RNG streams -- uncle_model is in the
config key, which is what makes --old bit-reproduce earlier runs -- so
the arms differed in stake draw, peering graph and every lottery
outcome, each comparison paid the between-run variance twice, and the
per-cell floor (+-0.0015) sat an order of magnitude above the effect.
Only delta_max = 5 resolved, and only after pooling.

Adds `paired_streams`: the RNG root is derived from the model-
independent part of the key, so a countable cell and its --old twin get
the SAME stake, graph and lottery draws and the uncle rule is the only
difference. Each replicate is then a matched pair and the shared
variance cancels. Trajectories still diverge after epoch 0 through the
genuine feedback (a different counted density changes the next epoch's
difficulty), which is the signal.

The flag is deliberately NOT in key(): it selects which key the seed is
derived from, so including it would perturb every historical seed.
Re-verified that --old still bit-reproduces the committed 2026-07-27
rho-boundary parquet, max |delta| = 0.

Results (configs/fine-delay-paired.yaml, 40 replicates per arm):
- Negative control becomes an IDENTITY check. With U = 0 no reference is
  taken, so shared streams must give bit-identical trajectories. All 200
  replicate pairs differ by exactly 0.0. Unpaired, the same control only
  had to agree within +-0.025 and drifted by 0.016.
- Per-cell SE shrinks by a median 1.6x (1.2-2.1x); widest 95% CI goes
  +-0.0015 -> +-0.0010. 5/15 cells resolve at |t| >= 2 (0.75 expected by
  chance); the largest, U=2 at delta_max=4, is t = 4.32 and clears
  Bonferroni for 15 tests.
- The cost is a STEP, not the ramp the unpaired data suggested:
  delta_max 1-3 unresolved (t = 1.1, 1.8, 1.4), then delta_max 4 AND 5
  both resolve at -0.0011 (t = 4.7) and -0.0009 (t = 3.7). Whole-band
  pooled -0.00060 +- 0.00021, t = 5.7 -- where the unpaired estimate of
  the same quantity (t = 2.8) had failed correction.

So the first-fork restriction costs nothing measurable up to
delta_max = 3 and about 0.1% at 4-5 -- an order of magnitude below the
+-0.9% per-epoch sampling noise.

Two bugs found while building this, both of which would have silently
produced a wrong answer:
- paired_streams was missing from metrics._CONFIG_FIELDS, so it never
  reached the parquet; plot_fine_delay.py falls back to the unpaired
  test when it cannot confirm pairing, so the sweep would have completed
  and quietly reported the old result. Caught before the run finished;
  the sweep was restarted and a test now pins the field.
- The U=0 control check reported FAILS on a PERFECT control: paired, the
  gap is exactly 0 so its SE is 0 and t is 0/0. It now checks the gap
  itself when the streams are shared, and falls back to the t-test only
  when there is real spread.

§3.2a is rewritten around the paired measurement; the unpaired sweep is
retained in §9 as the power comparison that motivated it. Figures 34-35
regenerated, with the control annotation and provenance reflecting the
design actually used.

Tests: 214 passed (was 209). ruff clean.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-05 11:58:26 +02:00
Marcin Pawlowski
bd2ac7b7be
Countable uncle model: spec counting rules, sweeps, figures
Implement the countable uncle model from the Cryptarchia spec's
counting-only reference rules, and make it the simulator default.

Counting rules (uncles.py, measure.py):
- Only the first block of a fork (parent on the producer's chain) is
  referenceable and countable, which makes every reference verifiable
  from chain data alone.
- The reference window is derived from a window-absorption parameter,
  w_u = W_abs/f slots (W_abs in expected block-intervals, default 10,
  bounded W_abs <= 0.6*k), replacing the free-standing uncle_window.
- Selection skips slots already occupied on the producer's chain and
  takes at most one uncle per slot.
- The measurement pass re-checks every rule per reference and tallies
  rejections as deep_ref_share.

The pre-redesign model is preserved behind --old on tsi-sweep and
tsi-verify. Its RNG key is byte-identical to the pre-uncle_model key,
so --old bit-reproduces the historical runs.

Supporting changes: uncle_model and window_absorption config surface
with validation (config.py, constants.py); accuracy closed form over
the effective q_u (theory.py); plumbing through tsi.py, epoch.py,
sweep.py, blocktree.py, metrics.py, verify.py, figures_pernode.py.

Studies and figures:
- configs/countable-vs-old.yaml -- delay x U grid, run under both
  models on the same grid.
- configs/absorption-window.yaml -- accuracy vs W_abs at U=1.
- scripts/plot_countable_vs_old.py renders fig30-fig33 into
  reports/tsi/report-figures/.

Tests: tests/test_countable_counting.py (7 cases) covering first-fork
eligibility, derived-window bounds, occupied-slot exclusion, and
per-reference re-checking; extensions to test_uncles.py,
test_config.py, test_slot_counting.py. Full fast suite: 202 passed.

Also adds CLAUDE.md (graphify project instructions) and ignores
editor/local-agent state plus the vendored Equi-X benchmark clone.

The reports/tsi/ prose describing this model is held back for a
separate editorial pass.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-04 18:48:46 +02:00
Marcin Pawlowski
24da2fc8b3
Importing tsi-sim v3 2026-07-30 18:57:10 +02:00