diff --git a/.gitignore b/.gitignore index 83972fa..bc74d7e 100644 --- a/.gitignore +++ b/.gitignore @@ -216,3 +216,14 @@ __marimo__/ # Streamlit .streamlit/secrets.toml + +# Editor / local agent state +.obsidian/ +.claude/settings.json +.claude/settings.local.json + +# External upstream clones vendored for benchmarking (own .git, not our history) +tools/benchmarks/original/ + +# Rendered report HTML — a build artifact of scripts/build_html.py; markdown is the source +reports/**/*.html diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 0000000..417efeb --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1,9 @@ +## graphify + +This project has a knowledge graph at graphify-out/ with god nodes, community structure, and cross-file relationships. + +Rules: +- For codebase questions, first run `graphify query ""` when graphify-out/graph.json exists. Use `graphify path "" ""` for relationships and `graphify explain ""` for focused concepts. These return a scoped subgraph, usually much smaller than GRAPH_REPORT.md or raw grep output. +- If graphify-out/wiki/index.md exists, use it for broad navigation instead of raw source browsing. +- Read graphify-out/GRAPH_REPORT.md only for broad architecture review or when query/path/explain do not surface enough context. +- After modifying code, run `graphify update .` to keep the graph current (AST-only, no API cost). diff --git a/reports/tsi/README.md b/reports/tsi/README.md index 9587fc6..477ffff 100644 --- a/reports/tsi/README.md +++ b/reports/tsi/README.md @@ -1,24 +1,1055 @@ # Total-Stake-Inference parameter selection -*Per-node network simulation of Cryptarchia Total Stake Inference (TSI). Simulator: [`tsi-sim-pernode`](../../tools/simulators/tsi/tsi-sim-pernode). All runs at the true security parameter **k = 2160** unless noted; latency is in slots and **1 slot = 1 s**.* +*Per-node network simulation of Cryptarchia Total Stake Inference (TSI). Simulator: `tsi-sim-pernode`. All runs at the true security parameter **k = 2160** unless noted; latency is in slots and **1 slot = 1 s**.* -This report selects and justifies the TSI parameters for Cryptarchia from a per-node network simulation. It is split into four cohesive parts; section numbers (§1–§9, A–C) are stable identifiers preserved across the set. +*Section numbers (§1–§9, Appendices A–C) are stable identifiers — §3.2a and the appendix letters are cited from the simulator and from spec discussion, so they are preserved verbatim. Simulation code, configs and run data: [`tools/simulators/tsi/tsi-sim-pernode`](../../tools/simulators/tsi/tsi-sim-pernode).* -## Parts +--- -1. **[Overview and recommendations](tsi-report-1-overview-and-recommendations.md)** — the executive summary, the per-knob parameter reference (§7), and the safest selection with residual risks and the recommendation-vs-spec deltas (§8). -2. **[Accuracy and design](tsi-report-2-accuracy-and-design.md)** — the model and counting rule (§2), the seven findings and their evidence (§3), and the design equations / selection algorithm (§4–§5). -3. **[Robustness and incentives](tsi-report-3-robustness-and-incentives.md)** — jitter, grinding, withholding, selfish mining, the reward design, fork/reorg depth, and organic churn (§6). -4. **[Reproducibility and appendices](tsi-report-4-reproducibility-and-appendices.md)** — how to re-run every study (§9), the residual f-rounding offset (App A), the per-epoch noise floor (App B), and consensus detail (App C). +## Contents -## Headline recommendation +- **[1. Executive summary](#s1)** — the problem, what was found, and the recommended configuration +- **[2. Model and method](#s2)** — what is simulated, and how accuracy is measured + - [2.1 The TSI algorithm — before and after uncle references](#s2-1) + - [2.2 The equilibrium is bounded by 1](#s2-2) +- **[3. Findings](#s3)** — the evidence, in the order a designer needs it + - [3.1 The per-node estimate is consensus-safe](#s3-1) + - [3.2 Latency biases TSI low; uncles recover it](#s3-2) + - [3.2a The design band at high precision — `δ_max` 1–5](#s3-2a) + - [3.3 When one uncle is not enough — the load `ρ`](#s3-3) + - [3.4 The uncle window is set by block spacing, not by delay](#s3-4) + - [3.5 The joint (W, U) region — the levers are hierarchical](#s3-5) + - [3.6 Block rate `f` moves every threshold predictably](#s3-6) + - [3.7 Network size erodes the one-uncle margin](#s3-7) +- **[4. Design equations and parameter-selection algorithm](#s4)** — the sizing rules, calibrated, with worked examples +- **[5. Discussion and caveats](#s5)** — the regime of validity, and what is not covered +- **[6. Robustness beyond the honest, deterministic regime](#s6)** — noise, attacks, and the incentive design + - [6.1 Consensus survives network jitter](#s6-1) + - [6.2 The estimator is a load-feedback loop](#s6-2) + - [6.3 Grinding by uncle suppression](#s6-3) + - [6.4 Block withholding](#s6-4) + - [6.5 Dynamic withhold-then-rejoin grinding](#s6-5) + - [6.6 Private-chain (selfish) withholding](#s6-6) + - [6.7 Block and uncle rewards](#s6-7) + - [6.8 A soft inclusion rule delivers the benefit without fork risk](#s6-8) + - [6.9 Multiple coalitions and bribery](#s6-9) + - [6.10 Fork rate and reorg depth](#s6-10) + - [6.11 Organic stake churn and the wall-clock cadence](#s6-11) +- **[7. Parameter reference — what each knob does](#s7)** + - [7.1 Protocol parameters](#s7-1) + - [7.2 Network / environment parameters](#s7-2) + - [7.3 Model and attack knobs (robustness probes)](#s7-3) +- **[8. Safest parameter selection and design](#s8)** — the recommendation and its rationale + - [8.1 The selection](#s8-1) + - [8.2 Design decisions and their rationale](#s8-2) + - [8.3 Residual risks and open items](#s8-3) + - [8.4 Capstone: the whole recipe, in one run](#s8-4) + - [8.5 Recommendation vs the current spec](#s8-5) +- **[9. Reproducibility](#s9)** — how to re-run every study +- **[Appendix A — the residual ~1 % offset: on-chain rounding of `f`](#sA)** +- **[Appendix B — the per-epoch sampling-noise floor](#sB)** + - [B.1 The mechanism](#sB-1) + - [B.2 Measured: the ±0.9 % floor, shrinking as 1/√T](#sB-2) + - [B.3 Delay converts the fluctuation into a one-sided under-count](#sB-3) + - [B.4 What precision is meaningful](#sB-4) +- **[Appendix C — consensus properties in detail](#sC)** + - [C.1 Per-epoch traces at the largest scale](#sC-1) + - [C.2 Consensus rests on common initialization](#sC-2) -Cryptarchia baseline f = 1/30: security `k = 2160`, uncle window `W = 300` slots, uncle cap `U = ⌈ρ⌉ + 1` (2 at the Blend target), learning rate `β = 1`, peering degree ≥ 6 at scale, soft uncle rewards with `w_u + w_n < 1`, and operate at load `ρ = f·D_vis < 1`. The full recommended-configuration table and rationale are in **[Part 1 →](tsi-report-1-overview-and-recommendations.md)**. +--- -## Figures +> **The uncle-reference model analysed here.** Uncle references are *counting-only* — they never affect a block's validity ([§6.7](#s6-7)–[§6.8](#s6-8)) — and are deduplicated by **slot**, not by block ([§8.5](#s8-5)). The reference window is derived from a **window absorption parameter**, `w_u = W_abs/f` with `W_abs = 10` expected block-intervals and bound `W_abs ≤ 0.6·k`, which is the `W = 300 = 10/f` sizing rule of [§3.4](#s3-4). One structural constraint shapes the results: only the **first block of a fork** — one whose parent lies on the referencing chain — is countable, which is the price of verifying every reference from chain data alone. This report calls that the **countable** model, and measures it against an **unrestricted** baseline in which any orphan in the window is referenceable at any fork depth ([§2.1](#s2-1), [§3.2](#s3-2), [§9](#s9)). The result: **in the design regime `ρ < 1` the two are practically identical.** A dedicated 40-replicate **paired** sweep of the operating band (`δ_max` 1–5, `ρ ≈ 0.21–0.41`), in which both arms share the stake draw, the peering graph and every lottery outcome so the uncle rule is the only difference, puts every `U ≥ 1` cell of both models in 0.998–1.001 and pins the first-fork cost at **nothing measurable up to `δ_max` = 3 and ~0.1 % at `δ_max` = 4–5** ([§3.2a](#s3-2a)). Measured against the unrestricted baseline under common random numbers, the cost grows monotonically with load — 0.13 % at `ρ ≈ 0.36`, 0.34 % at `ρ ≈ 0.56`, 1.0 % at `ρ ≈ 0.96` and 2.3 % under overload at `ρ ≈ 1.76`, where recovery saturates near 0.95 against the unrestricted model's ~1.00. Deep forks are unrecoverable by construction, so past `ρ ≈ 1` the ceiling is set by the fork *depth* profile and **no uncle cap buys it back** — a second, independent reason for the [§8](#s8) "keep `ρ < 1`" rule. The closed-form accuracy with the measured effective utilization, `log(1−f)/log(1−f/q_u)`, predicts every countable cell to within 0.2 %. -Figures are embedded from [`report-figures/`](report-figures) via relative links and are versioned here alongside the report. They are produced by the simulator's plotting scripts (`scripts/*.py` and `tsi_sim.plotting`) in [`tsi-sim-pernode`](../../tools/simulators/tsi/tsi-sim-pernode); that simulation folder does **not** commit its own generated figures — the copies checked in here are the report's figures of record. + +## 1. Executive summary -## Reproducing the results +**The problem.** A Cryptarchia node wins the right to produce a block through a stake-weighted lottery, and the lottery's difficulty must be tuned to the total stake that is actively online — a number nobody can read directly, because participants join, leave, and fail without notice. Total Stake Inference (TSI) is the protocol's answer: every node keeps its own estimate `D̂` of the active stake and continuously corrects it against the one thing it can observe — how often blocks actually appear on chain. -The simulation code, configs, and run data live in [`tools/simulators/tsi/tsi-sim-pernode`](../../tools/simulators/tsi/tsi-sim-pernode). Every study's exact command is listed in [Part 4 — Reproducibility (§9)](tsi-report-4-reproducibility-and-appendices.md). In short, from the simulator directory: `make install`, then `make ` to run a sweep (results land under `runs/_