4 Commits

Author	SHA1	Message	Date
Marcin Pawlowski	4baccd8d4b	Paired design: resolve the design band with common random numbers ... The unpaired comparison could not answer the question it was asked. The two uncle models draw independent RNG streams -- uncle_model is in the config key, which is what makes --old bit-reproduce earlier runs -- so the arms differed in stake draw, peering graph and every lottery outcome, each comparison paid the between-run variance twice, and the per-cell floor (+-0.0015) sat an order of magnitude above the effect. Only delta_max = 5 resolved, and only after pooling. Adds `paired_streams`: the RNG root is derived from the model- independent part of the key, so a countable cell and its --old twin get the SAME stake, graph and lottery draws and the uncle rule is the only difference. Each replicate is then a matched pair and the shared variance cancels. Trajectories still diverge after epoch 0 through the genuine feedback (a different counted density changes the next epoch's difficulty), which is the signal. The flag is deliberately NOT in key(): it selects which key the seed is derived from, so including it would perturb every historical seed. Re-verified that --old still bit-reproduces the committed 2026-07-27 rho-boundary parquet, max |delta| = 0. Results (configs/fine-delay-paired.yaml, 40 replicates per arm): - Negative control becomes an IDENTITY check. With U = 0 no reference is taken, so shared streams must give bit-identical trajectories. All 200 replicate pairs differ by exactly 0.0. Unpaired, the same control only had to agree within +-0.025 and drifted by 0.016. - Per-cell SE shrinks by a median 1.6x (1.2-2.1x); widest 95% CI goes +-0.0015 -> +-0.0010. 5/15 cells resolve at |t| >= 2 (0.75 expected by chance); the largest, U=2 at delta_max=4, is t = 4.32 and clears Bonferroni for 15 tests. - The cost is a STEP, not the ramp the unpaired data suggested: delta_max 1-3 unresolved (t = 1.1, 1.8, 1.4), then delta_max 4 AND 5 both resolve at -0.0011 (t = 4.7) and -0.0009 (t = 3.7). Whole-band pooled -0.00060 +- 0.00021, t = 5.7 -- where the unpaired estimate of the same quantity (t = 2.8) had failed correction. So the first-fork restriction costs nothing measurable up to delta_max = 3 and about 0.1% at 4-5 -- an order of magnitude below the +-0.9% per-epoch sampling noise. Two bugs found while building this, both of which would have silently produced a wrong answer: - paired_streams was missing from metrics._CONFIG_FIELDS, so it never reached the parquet; plot_fine_delay.py falls back to the unpaired test when it cannot confirm pairing, so the sweep would have completed and quietly reported the old result. Caught before the run finished; the sweep was restarted and a test now pins the field. - The U=0 control check reported FAILS on a PERFECT control: paired, the gap is exactly 0 so its SE is 0 and t is 0/0. It now checks the gap itself when the streams are shared, and falls back to the t-test only when there is real spread. §3.2a is rewritten around the paired measurement; the unpaired sweep is retained in §9 as the power comparison that motivated it. Figures 34-35 regenerated, with the control annotation and provenance reflecting the design actually used. Tests: 214 passed (was 209). ruff clean. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-05 11:58:26 +02:00
Marcin Pawlowski	ac6a309e58	Review fixes + high-precision design-band delay study ... Acts on a correctness/completeness review of the countable uncle model and its report material. Correctness fixes in the report: - s3.4 quoted 0.998 for W_abs=10 at the 8s budget; the run says 0.9963. - s1 claimed both models >= 0.996 at U >= 1; countable U=2 delta=8 is 0.9955. Corrected to >= 0.995. - The s3.2 table presented two cells (U=1 at delta 16 and 32) as model differences. They are not resolvable: t = 0.46 and 0.47 over 5 replicates. The table now carries +-SEM and a t per cell. - s3.4 claimed the ~7-block-interval floor "carries over unchanged". Accuracy is still climbing past W=7 at every delay (8s: 0.989 -> 0.996), so the claim is dropped. The 32s curve is non-monotonic with replicate SD up to 0.22 and is now flagged as noise, not a trend. - 1-r was attributed to the first-fork restriction alone; it is the combined first-fork and capacity loss, which this measurement cannot separate. Hedged to match fig32's own axis label. Completeness: the U=0 negative control was swept but never reported. With no uncles the two models are identical by construction, yet they differ by -0.23 at delta_max=32 (t=2.1) because they draw independent RNG streams. That is the noise floor the rest of the grid must clear, and it is now in s3.2, s9, fig30 and the config header. New study (configs/fine-delay.yaml, scripts/plot_fine_delay.py, s3.2a, fig34/fig35): the design band delta_max 1-5 at 40 replicates, both models. Findings: every U >= 1 cell of both models lands in 0.998-1.001, flat in delay, while U=0 decays 0.810 -> 0.640. No individual cell resolves a model difference (widest 95% CI +-0.15pp; max t=2.59 vs Bonferroni 2.94 over 15 cells). Pooled across uncle caps the first-fork cost is monotone in delay and separates from zero only at delta_max=5 (-0.0014 +- 0.0007, t=3.7) -- below 0.15% everywhere in the band, against +-0.9% per-epoch sampling noise. Code: - deep_ref_share is identically 0 on every real countable run: for a chain block B the producer's chain below B is the counting chain below B, so the counting-side parent-on-chain re-check cannot reject what selection emitted. It is a drift alarm, not a rate. Documented as such in measure.py, the plot docstring and the config header, and pinned by a new end-to-end test. - Removed annotate_uncles: a second countable implementation that production never called, while carrying most of the selection test coverage. Tests now drive select_uncles_at_production through an annotate_via_production replay helper -- same assertions, live path. - Added tests for the two previously uncovered branches of the live selection: the pmin/below chain walk that resolves parent-on-chain for candidates whose parent sits below the window, and the occupied-slot exclusion built from the chain walk. - theory.q_effective and theory.window_miss_prob were unused and untested. Now used (the prediction figure reconstructs q_u through the identity the report quotes) and tested. The window_miss_prob test records that its "~ e^-W" docstring is the f->0 limit: the true decay is e^-1.017W at f=1/30, 16% off by W=10. - Shared sem()/recovery_rate() moved into figures_pernode.py; fig30 and fig33 regenerated with SEM error bars and the U=0 control curve. - Fixed the pre-existing E501 in bootstrap_dynamics.py; ruff clean. Report prose reworked to read standalone: the countable model is described as the rules under analysis and the former model as a labelled "unrestricted" comparison baseline, with no dated banners and no round-to-round narration. Tests: 209 passed (was 202). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-04 20:48:38 +02:00
Marcin Pawlowski	bd2ac7b7be	Countable uncle model: spec counting rules, sweeps, figures ... Implement the countable uncle model from the Cryptarchia spec's counting-only reference rules, and make it the simulator default. Counting rules (uncles.py, measure.py): - Only the first block of a fork (parent on the producer's chain) is referenceable and countable, which makes every reference verifiable from chain data alone. - The reference window is derived from a window-absorption parameter, w_u = W_abs/f slots (W_abs in expected block-intervals, default 10, bounded W_abs <= 0.6*k), replacing the free-standing uncle_window. - Selection skips slots already occupied on the producer's chain and takes at most one uncle per slot. - The measurement pass re-checks every rule per reference and tallies rejections as deep_ref_share. The pre-redesign model is preserved behind --old on tsi-sweep and tsi-verify. Its RNG key is byte-identical to the pre-uncle_model key, so --old bit-reproduces the historical runs. Supporting changes: uncle_model and window_absorption config surface with validation (config.py, constants.py); accuracy closed form over the effective q_u (theory.py); plumbing through tsi.py, epoch.py, sweep.py, blocktree.py, metrics.py, verify.py, figures_pernode.py. Studies and figures: - configs/countable-vs-old.yaml -- delay x U grid, run under both models on the same grid. - configs/absorption-window.yaml -- accuracy vs W_abs at U=1. - scripts/plot_countable_vs_old.py renders fig30-fig33 into reports/tsi/report-figures/. Tests: tests/test_countable_counting.py (7 cases) covering first-fork eligibility, derived-window bounds, occupied-slot exclusion, and per-reference re-checking; extensions to test_uncles.py, test_config.py, test_slot_counting.py. Full fast suite: 202 passed. Also adds CLAUDE.md (graphify project instructions) and ignores editor/local-agent state plus the vendored Equi-X benchmark clone. The reports/tsi/ prose describing this model is held back for a separate editorial pass. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-04 18:48:46 +02:00
Marcin Pawlowski	24da2fc8b3	Importing tsi-sim v3	2026-07-30 18:57:10 +02:00