65 lines
3.1 KiB
Python
Raw Normal View History

2026-07-30 18:57:10 +02:00
#!/usr/bin/env python
"""Capstone: the recommended configuration end-to-end at true k=2160 (report §8).
One config f=1/30, W=300, U=2, β=1, degree 6, Blend 3 hops × 8 s, Pareto stake run honest
and under a 30 % uncle-suppression adversary, confirming accuracy, consensus, fork rate, reorg
depth, and the emergent reference rate p_ref ALL hold together. Writes runs/capstone.parquet.
"""
from __future__ import annotations
import sys
from pathlib import Path
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "src"))
import pandas as pd
from joblib import Parallel, delayed
from tsi_sim.config import SimConfig
from tsi_sim.engine import run_trajectory
REC = dict(n_nodes=1000, stake_dist="pareto", topology="blend", degree=6,
link_latency_mean=0.5, link_latency_dist="geo", blend_hops=3, blend_delay_max=8.0,
max_uncles=2, uncle_window=300, uncle_strategy="oldest", k=2160, epochs=40,
genesis_d_factor=0.5, early_stop=True)
The sec 8.4 capstone was still reporting pre-countable numbers Re-running the capstone under both anchors turned up something worse than the anchor question. The uncle-anchored arm should have bit-reproduced the committed capstone, since its key is unchanged -- and it did not. Bisecting against the pre-session source showed my changes are clean (identical trajectory, epochs=21, tail 0.994358 either way); the committed parquet is simply stale. It is dated 24 July and has no uncle_model column at all, so it predates the countable redesign. sec 8.4's headline table has been reporting UNRESTRICTED-model numbers ever since that landed. sec 9's "all studies re-run" note covers the slot-counting fix of 23/24 July, not the countable redesign of 4 August, and the capstone was missed. Corrected, countable model, delta_max = 8, U = 2, W = 10, k = 2160, 8 reps: honest D-hat/D 0.996 (was 1.001) p_ref 0.944 (was 1.000) 30% suppress 0.994 (was 0.998) p_ref 0.936 (was 0.990) Lower, and p_ref materially lower, because the first-fork restriction makes some orphans unreferenceable that the earlier model counted. No recommendation changes: accuracy sits inside the +-0.9% per-epoch noise floor below the hard ceiling of 1, consensus is exact, p_ref stays far above the ~0.3 the soft rule needs. sec 6.8's p_ref quotes came from the same stale run and are corrected too. The anchor result the re-run was for: parent-anchored gives 0.996 honest -- unchanged -- but 0.974 under the 30% suppression adversary, with p_ref 0.875. The effects compound: a tighter effective window leaves an orphan fewer chances to be referenced before its PARENT ages out, and a suppressing adversary removes some of those chances. Neither isolated sweep shows it, because sec 6.12's honest arms hold the adversary at zero and its adversarial arms use a slack cap. That is what the capstone is for, and it is the strongest argument for pairing the anchor change with W = 12 rather than adopting it at W = 10. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-10 11:06:39 +02:00
def _one(adv: float, rep: int, anchor: str = "uncle") -> list[dict]:
cfg = SimConfig(**REC, adversary_frac=adv, adversary_strategy="suppress",
uncle_window_anchor=anchor, replicate=rep)
2026-07-30 18:57:10 +02:00
rows = run_trajectory(cfg)
for r in rows:
r["adv"] = adv
The sec 8.4 capstone was still reporting pre-countable numbers Re-running the capstone under both anchors turned up something worse than the anchor question. The uncle-anchored arm should have bit-reproduced the committed capstone, since its key is unchanged -- and it did not. Bisecting against the pre-session source showed my changes are clean (identical trajectory, epochs=21, tail 0.994358 either way); the committed parquet is simply stale. It is dated 24 July and has no uncle_model column at all, so it predates the countable redesign. sec 8.4's headline table has been reporting UNRESTRICTED-model numbers ever since that landed. sec 9's "all studies re-run" note covers the slot-counting fix of 23/24 July, not the countable redesign of 4 August, and the capstone was missed. Corrected, countable model, delta_max = 8, U = 2, W = 10, k = 2160, 8 reps: honest D-hat/D 0.996 (was 1.001) p_ref 0.944 (was 1.000) 30% suppress 0.994 (was 0.998) p_ref 0.936 (was 0.990) Lower, and p_ref materially lower, because the first-fork restriction makes some orphans unreferenceable that the earlier model counted. No recommendation changes: accuracy sits inside the +-0.9% per-epoch noise floor below the hard ceiling of 1, consensus is exact, p_ref stays far above the ~0.3 the soft rule needs. sec 6.8's p_ref quotes came from the same stale run and are corrected too. The anchor result the re-run was for: parent-anchored gives 0.996 honest -- unchanged -- but 0.974 under the 30% suppression adversary, with p_ref 0.875. The effects compound: a tighter effective window leaves an orphan fewer chances to be referenced before its PARENT ages out, and a suppressing adversary removes some of those chances. Neither isolated sweep shows it, because sec 6.12's honest arms hold the adversary at zero and its adversarial arms use a slack cap. That is what the capstone is for, and it is the strongest argument for pairing the anchor change with W = 12 rather than adopting it at W = 10. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-10 11:06:39 +02:00
r["anchor"] = anchor
2026-07-30 18:57:10 +02:00
return rows
def main() -> None:
out = Path(__file__).resolve().parents[1] / "runs"
The sec 8.4 capstone was still reporting pre-countable numbers Re-running the capstone under both anchors turned up something worse than the anchor question. The uncle-anchored arm should have bit-reproduced the committed capstone, since its key is unchanged -- and it did not. Bisecting against the pre-session source showed my changes are clean (identical trajectory, epochs=21, tail 0.994358 either way); the committed parquet is simply stale. It is dated 24 July and has no uncle_model column at all, so it predates the countable redesign. sec 8.4's headline table has been reporting UNRESTRICTED-model numbers ever since that landed. sec 9's "all studies re-run" note covers the slot-counting fix of 23/24 July, not the countable redesign of 4 August, and the capstone was missed. Corrected, countable model, delta_max = 8, U = 2, W = 10, k = 2160, 8 reps: honest D-hat/D 0.996 (was 1.001) p_ref 0.944 (was 1.000) 30% suppress 0.994 (was 0.998) p_ref 0.936 (was 0.990) Lower, and p_ref materially lower, because the first-fork restriction makes some orphans unreferenceable that the earlier model counted. No recommendation changes: accuracy sits inside the +-0.9% per-epoch noise floor below the hard ceiling of 1, consensus is exact, p_ref stays far above the ~0.3 the soft rule needs. sec 6.8's p_ref quotes came from the same stale run and are corrected too. The anchor result the re-run was for: parent-anchored gives 0.996 honest -- unchanged -- but 0.974 under the 30% suppression adversary, with p_ref 0.875. The effects compound: a tighter effective window leaves an orphan fewer chances to be referenced before its PARENT ages out, and a suppressing adversary removes some of those chances. Neither isolated sweep shows it, because sec 6.12's honest arms hold the adversary at zero and its adversarial arms use a slack cap. That is what the capstone is for, and it is the strongest argument for pairing the anchor change with W = 12 rather than adopting it at W = 10. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-10 11:06:39 +02:00
# Both window anchors: the spec's uncle-anchored rule, and the §6.12 proposal. The capstone
# is the "whole recipe together" check, so a change to any rule in the recipe has to be run
# through it rather than argued from the isolated sweeps.
jobs = [(a, r, w) for w in ("uncle", "parent") for a in (0.0, 0.3) for r in range(8)]
2026-07-30 18:57:10 +02:00
res = Parallel(n_jobs=4, backend="loky", inner_max_num_threads=1)(
The sec 8.4 capstone was still reporting pre-countable numbers Re-running the capstone under both anchors turned up something worse than the anchor question. The uncle-anchored arm should have bit-reproduced the committed capstone, since its key is unchanged -- and it did not. Bisecting against the pre-session source showed my changes are clean (identical trajectory, epochs=21, tail 0.994358 either way); the committed parquet is simply stale. It is dated 24 July and has no uncle_model column at all, so it predates the countable redesign. sec 8.4's headline table has been reporting UNRESTRICTED-model numbers ever since that landed. sec 9's "all studies re-run" note covers the slot-counting fix of 23/24 July, not the countable redesign of 4 August, and the capstone was missed. Corrected, countable model, delta_max = 8, U = 2, W = 10, k = 2160, 8 reps: honest D-hat/D 0.996 (was 1.001) p_ref 0.944 (was 1.000) 30% suppress 0.994 (was 0.998) p_ref 0.936 (was 0.990) Lower, and p_ref materially lower, because the first-fork restriction makes some orphans unreferenceable that the earlier model counted. No recommendation changes: accuracy sits inside the +-0.9% per-epoch noise floor below the hard ceiling of 1, consensus is exact, p_ref stays far above the ~0.3 the soft rule needs. sec 6.8's p_ref quotes came from the same stale run and are corrected too. The anchor result the re-run was for: parent-anchored gives 0.996 honest -- unchanged -- but 0.974 under the 30% suppression adversary, with p_ref 0.875. The effects compound: a tighter effective window leaves an orphan fewer chances to be referenced before its PARENT ages out, and a suppressing adversary removes some of those chances. Neither isolated sweep shows it, because sec 6.12's honest arms hold the adversary at zero and its adversarial arms use a slack cap. That is what the capstone is for, and it is the strongest argument for pairing the anchor change with W = 12 rather than adopting it at W = 10. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-10 11:06:39 +02:00
delayed(_one)(a, r, w) for a, r, w in jobs)
2026-07-30 18:57:10 +02:00
df = pd.DataFrame([row for traj in res for row in traj])
df.to_parquet(out / "capstone.parquet", index=False)
print("=== Capstone: recommended config, all metrics together (equilibrium tail) ===")
The sec 8.4 capstone was still reporting pre-countable numbers Re-running the capstone under both anchors turned up something worse than the anchor question. The uncle-anchored arm should have bit-reproduced the committed capstone, since its key is unchanged -- and it did not. Bisecting against the pre-session source showed my changes are clean (identical trajectory, epochs=21, tail 0.994358 either way); the committed parquet is simply stale. It is dated 24 July and has no uncle_model column at all, so it predates the countable redesign. sec 8.4's headline table has been reporting UNRESTRICTED-model numbers ever since that landed. sec 9's "all studies re-run" note covers the slot-counting fix of 23/24 July, not the countable redesign of 4 August, and the capstone was missed. Corrected, countable model, delta_max = 8, U = 2, W = 10, k = 2160, 8 reps: honest D-hat/D 0.996 (was 1.001) p_ref 0.944 (was 1.000) 30% suppress 0.994 (was 0.998) p_ref 0.936 (was 0.990) Lower, and p_ref materially lower, because the first-fork restriction makes some orphans unreferenceable that the earlier model counted. No recommendation changes: accuracy sits inside the +-0.9% per-epoch noise floor below the hard ceiling of 1, consensus is exact, p_ref stays far above the ~0.3 the soft rule needs. sec 6.8's p_ref quotes came from the same stale run and are corrected too. The anchor result the re-run was for: parent-anchored gives 0.996 honest -- unchanged -- but 0.974 under the 30% suppression adversary, with p_ref 0.875. The effects compound: a tighter effective window leaves an orphan fewer chances to be referenced before its PARENT ages out, and a suppressing adversary removes some of those chances. Neither isolated sweep shows it, because sec 6.12's honest arms hold the adversary at zero and its adversarial arms use a slack cap. That is what the capstone is for, and it is the strongest argument for pairing the anchor change with W = 12 rather than adopting it at W = 10. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-10 11:06:39 +02:00
for (anchor, adv), g in df.groupby(["anchor", "adv"]):
Review pass: reproduce every number from its data of record, fix what did not Correctness/completeness review of the report and simulator. Verified against the committed parquets: the sec 6.6 countable-ceiling table (cap-64 MDP sweep), sec 6.10 Result 4's depth ceilings, the sec 3.4 uncle-selection table, all adversary-variant numbers, the rho-boundary row-4 quotes (0.976 at rho=0.91, 4-sigma shortfall at 0.96, max cell 1.0024), and the sec 8.4 capstone table. Three defects found, all fixed: 1. The collapse event was not reproducible from the committed script. Study D swept only the default (random) coalition, but the one observed collapse is a whale cell; the "once in 144 runs" count came from an ad-hoc probe. The committed sweep now carries the selection axis (96 runs) and reproduces the event: 1/12 in the whale 50% cell at delta_max = 8, never at 4. All six fold-related passages now quote the committed sweep, which also retires the stale "the full dynamics never reach it" wording in the sec 6 arc, the sec 6.2 intro, row 6 and item 1 -- text that contradicted item 18 since yesterday's finding. 2. capstone.py's printout could not reproduce the report's sec 8.4 table. The report's numbers are a per-replicate-tail aggregation (each replicate burns in against its own early-stop length); the script cut the tail at the ARM's max epoch, silently dropping any replicate that stopped earlier (7 of 8 in the adversary arm) and landing one rounding step off on three cells. The script now aggregates per replicate and prints the SEM; against the existing parquet it reproduces the table exactly (1.001/0.998, 0.342+-0.009 / 0.343+-0.005, p_ref 1.000/0.990, 8 reps both arms). The report table was right all along; sec 6.8's p_ref quote (0.989, the per-arm value) is aligned to 0.990. 3. Small report fixes: slow-beta deflation rounded 0.765 -> "0.77" (now 0.76); fig13's caption now points at the fig36 ceiling instead of implying free recovery; row 5 cites the measured slow-beta standing deflation; the canonical-data paragraph lists the new studies' artifacts; the simulator README's layout block lists the new tests and scripts. Adds a unit test for reorg.countable_recovery_from_depths (the one new function that had none). 236 tests pass; the new-study parquets are copied to the main checkout's runs/, where every other study's data of record lives. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-06 12:29:13 +02:00
# Per-REPLICATE tail: early_stop ends replicates at different epochs, so a per-arm cut
# (epoch >= arm_max//2) would silently drop any replicate that stopped before the cut
# and skew the tail toward the slow-converging ones. The report's §8.4 numbers are the
# per-replicate aggregation; keep this printout matching them.
t = pd.concat([r[r.epoch >= r.epoch.max() // 2] for _, r in g.groupby("replicate")])
per_rep = t.groupby("replicate").fork_rate.mean()
sem = per_rep.std(ddof=1) / (len(per_rep) ** 0.5)
The sec 8.4 capstone was still reporting pre-countable numbers Re-running the capstone under both anchors turned up something worse than the anchor question. The uncle-anchored arm should have bit-reproduced the committed capstone, since its key is unchanged -- and it did not. Bisecting against the pre-session source showed my changes are clean (identical trajectory, epochs=21, tail 0.994358 either way); the committed parquet is simply stale. It is dated 24 July and has no uncle_model column at all, so it predates the countable redesign. sec 8.4's headline table has been reporting UNRESTRICTED-model numbers ever since that landed. sec 9's "all studies re-run" note covers the slot-counting fix of 23/24 July, not the countable redesign of 4 August, and the capstone was missed. Corrected, countable model, delta_max = 8, U = 2, W = 10, k = 2160, 8 reps: honest D-hat/D 0.996 (was 1.001) p_ref 0.944 (was 1.000) 30% suppress 0.994 (was 0.998) p_ref 0.936 (was 0.990) Lower, and p_ref materially lower, because the first-fork restriction makes some orphans unreferenceable that the earlier model counted. No recommendation changes: accuracy sits inside the +-0.9% per-epoch noise floor below the hard ceiling of 1, consensus is exact, p_ref stays far above the ~0.3 the soft rule needs. sec 6.8's p_ref quotes came from the same stale run and are corrected too. The anchor result the re-run was for: parent-anchored gives 0.996 honest -- unchanged -- but 0.974 under the 30% suppression adversary, with p_ref 0.875. The effects compound: a tighter effective window leaves an orphan fewer chances to be referenced before its PARENT ages out, and a suppressing adversary removes some of those chances. Neither isolated sweep shows it, because sec 6.12's honest arms hold the adversary at zero and its adversarial arms use a slack cap. That is what the capstone is for, and it is the strongest argument for pairing the anchor change with W = 12 rather than adopting it at W = 10. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-10 11:06:39 +02:00
print(f"{anchor:>6}-anchored, adversary {adv:.0%}: D̂/D {t.mean_ratio.mean():.4f} "
2026-07-30 18:57:10 +02:00
f"range_ratio {t.range_ratio.max():.4f} agreement {t.agreement_window.min():.4f} "
Review pass: reproduce every number from its data of record, fix what did not Correctness/completeness review of the report and simulator. Verified against the committed parquets: the sec 6.6 countable-ceiling table (cap-64 MDP sweep), sec 6.10 Result 4's depth ceilings, the sec 3.4 uncle-selection table, all adversary-variant numbers, the rho-boundary row-4 quotes (0.976 at rho=0.91, 4-sigma shortfall at 0.96, max cell 1.0024), and the sec 8.4 capstone table. Three defects found, all fixed: 1. The collapse event was not reproducible from the committed script. Study D swept only the default (random) coalition, but the one observed collapse is a whale cell; the "once in 144 runs" count came from an ad-hoc probe. The committed sweep now carries the selection axis (96 runs) and reproduces the event: 1/12 in the whale 50% cell at delta_max = 8, never at 4. All six fold-related passages now quote the committed sweep, which also retires the stale "the full dynamics never reach it" wording in the sec 6 arc, the sec 6.2 intro, row 6 and item 1 -- text that contradicted item 18 since yesterday's finding. 2. capstone.py's printout could not reproduce the report's sec 8.4 table. The report's numbers are a per-replicate-tail aggregation (each replicate burns in against its own early-stop length); the script cut the tail at the ARM's max epoch, silently dropping any replicate that stopped earlier (7 of 8 in the adversary arm) and landing one rounding step off on three cells. The script now aggregates per replicate and prints the SEM; against the existing parquet it reproduces the table exactly (1.001/0.998, 0.342+-0.009 / 0.343+-0.005, p_ref 1.000/0.990, 8 reps both arms). The report table was right all along; sec 6.8's p_ref quote (0.989, the per-arm value) is aligned to 0.990. 3. Small report fixes: slow-beta deflation rounded 0.765 -> "0.77" (now 0.76); fig13's caption now points at the fig36 ceiling instead of implying free recovery; row 5 cites the measured slow-beta standing deflation; the canonical-data paragraph lists the new studies' artifacts; the simulator README's layout block lists the new tests and scripts. Adds a unit test for reorg.countable_recovery_from_depths (the one new function that had none). 236 tests pass; the new-study parquets are copied to the main checkout's runs/, where every other study's data of record lives. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-08-06 12:29:13 +02:00
f"fork_rate {per_rep.mean():.3f}+-{sem:.3f}(SEM over {len(per_rep)} reps) "
f"max_reorg_depth {t.max_reorg_depth.max()} p_ref {t.p_ref.mean():.3f}")
2026-07-30 18:57:10 +02:00
print(f"wrote {out/'capstone.parquet'} ({len(df)} rows)")
if __name__ == "__main__":
main()