chore(chain_state): update DESIGN.md

lez/chain_state/DESIGN.md

@@ -33,20 +33,16 @@ sequencer and indexer expose their exact current state.
 lee_core  ←  lee (owns V03State)  ←  common (owns Block, BedrockStatus, clock_invocation, recompute_hash)  ←  lez/chain_state  ←  { indexer/core, sequencer/core }
 ```
 
-- **Not** `lee_core`: that is the RISC0 guest crate; a stateful host machine has
-  no place in the zkVM circuit and does not know what a `Block` is.
 - **Not** `lee`: the apply logic needs `Block`/`BedrockStatus`/`clock_invocation`
   from `common`, and `common` depends on `lee` — putting it in `lee` inverts the
   layer.
 - `lez/chain_state` sits above `common`, depends on `common` + `lee`, and is
   consumed by both `indexer/core` and `sequencer/core`.
 
-**Persistence boundary (decision: A).** `chain_state` holds the in-memory state
-machine and the pure logic; it performs **no I/O**. Each consumer keeps its own
-`RocksDBIO` and drives the `scratch → put_block → commit` ordering, exactly as
-`accept_block` does today. This keeps the crate fully unit-testable without a DB.
-A storage-trait abstraction (option B) is a possible later follow-up, not part of
-this task.
+**Persistence boundary.** `chain_state` holds the in-memory state machine and the
+pure logic; it performs **no I/O**. Each consumer keeps its own `RocksDBIO` and
+drives the `scratch → put_block → commit` ordering, exactly as `accept_block` does
+today. This keeps the crate fully unit-testable without a DB.
 
 ## 3. The `apply_block` entry point
 
@@ -90,7 +86,8 @@ struct ChainState {
     final_tip:    Option<Tip>,
     head_state:   V03State,        // final_state + applied head blocks
     head_blocks:  Vec<HeadEntry>,  // ordered, above final_tip
-    stall:        Option<StallReason>,
+    final_stall:  Option<StallReason>,  // persisted to RocksDB — see §4a
+    head_stall:   Option<StallReason>,  // in-memory only — recomputed from the stream on restart
 }
 
 struct HeadEntry { this_msg: MsgId, block: Block }
@@ -105,8 +102,9 @@ for revert correlation.
 Operations:
 
 - `apply_adopted(inscription) -> AcceptOutcome` — dedup by `this_msg` against our
-  outbox, else `apply_block` on the head tip; on success push `HeadEntry`; on
-  failure record stall + park (head tip frozen).
+  outbox, else `apply_block` on the head tip; on success push `HeadEntry`. On
+  failure, set **`head_stall`** (in-memory `StallReason`) and freeze the head tip
+  at the last valid block — do **not** persist.
 - `apply_channel_update(orphaned, adopted)` — revert every `orphaned` by
   `this_msg`, re-derive `head_state` (clone `final_state`, replay survivors),
   then apply every `adopted` in order. Atomic per event.
@@ -114,14 +112,59 @@ Operations:
   `final_state` (already validated; a move, not a re-apply).
 - `apply_finalized(inscription)` — steady state: if present in head by
   `this_msg`, `finalize_up_to`; cold-start backfill (not in head):
-  `apply_block` directly to `final_state`, mirror into head.
-- `rollback_orphan(this_msg)` — drop from that entry forward, re-derive head.
-- `status() -> { final_height, head_height, stall }` — for RPC/UI.
+  `apply_block` directly to `final_state`, mirror into head. If a finalized block
+  fails to apply, set **`final_stall`** and persist it — this is the **only**
+  `StallReason` written to disk (see §4a).
+- `rollback_orphan(this_msg)` — drop from that entry forward, re-derive head;
+  clears `head_stall` if the re-derived head is clean.
+- `status() -> { final_height, head_height, head_stall, final_stall }` — for RPC/UI.
 
 For the **indexer** (finalized-only `next_messages` stream), `head_blocks` stays
 empty and `head == final`; it exercises only `apply_finalized`. The **sequencer**
 uses both tiers from day one.
 
+### 4a. Two stalls — persisted vs in-memory
+
+Both tiers carry a `StallReason` (`final_stall`, `head_stall`); they are equally
+informative. The difference is **persistence**, which follows from durability. L1
+finality is about inscription **canonicality**, not LEZ-block **content**
+validity, so an authorized sequencer can get a content-invalid block finalized —
+which is why both tiers can meet an invalid block in the first place.
+
+- **`head_stall` — in-memory only.** The head is reorg-able and re-derived from
+  every `channel_update`. An invalid adopted block is transient: it is either
+  orphaned (a competing valid block at the same height wins) or it finalizes. We
+  set `head_stall` for observability (RPC/UI can show "head blocked at `N`:
+  StateTransition at tx 3") and freeze the head tip at the last valid block, but
+  we do **not** write it to disk — on restart the head is rebuilt from the stream,
+  so a persisted head stall would be redundant and could go stale. Because the
+  next adopted block chains on the bad one, it fails validation too, so the head
+  stays frozen until a valid block (competing or post-reorg) is adopted.
+- **`final_stall` — persisted.** The final tier is irreversible. If an invalid
+  block *finalizes*, the node is durably stuck until a valid successor (built by
+  honest sequencers on the last valid parent) finalizes. This must survive
+  restart: the startup chain-consistency / anchor check reads it, it is what we
+  surface as `Stalled`, and it is the signal the committee acts on to evict a bad
+  sequencer.
+
+An invalid block **migrates the problem head→final** when it finalizes: `head_stall`
+is set at `N` first; once `N+1(bad)` finalizes, `apply_finalized` fails and records
+the persisted `final_stall`. Both tiers end up stuck at `N` consistently, and both
+recover when `N+1′` finalizes. The indexer (final tier only) never sets
+`head_stall`, so it behaves exactly as it does today.
+
+### 4b. Producer contract — write on turn, build on last valid
+
+The sequencer publishes **only on its own turn** (the SDK queues out-of-turn
+publishes). When it is our turn we build the next block on the **current head
+tip**, which is by construction the last validly-applied block. So if the head is
+frozen (`head_stall` set) on a peer's bad block, we build on that frozen valid
+tip — the same parent every honest sequencer chooses — and thereby skip the bad
+block rather than extend it. A parked node keeps following peers' valid blocks as
+they arrive; the moment it also gets a turn, it produces the next valid block on
+its last valid tip. Net: parking never stops us from producing correctly on our
+turn.
+
 ## 5. Event → tier mapping
 
 `Event::BlocksProcessed { checkpoint, channel_update: { orphaned, adopted }, finalized }`:
@@ -160,16 +203,20 @@ flowchart TD
     VAL --> OUT{"AcceptOutcome"}
     OUT -->|Applied| APP["append this_msg+block to head,<br/>advance head tip, clear stall"]
     OUT -->|AlreadyApplied| SKIP
-    OUT -->|"Parked(err)"| PARK["record StallReason,<br/>freeze head tip,<br/>mark processed — do NOT apply"]
+    OUT -->|"Parked(err)"| PARK["set head_stall (in-memory),<br/>freeze head tip — do NOT apply.<br/>Not persisted"]
     SKIP --> FIN
     APP --> FIN
     PARK --> FIN
     FIN{"finalized<br/>inscriptions?"}
     FIN -->|"already in head (steady state)"| MOVE["finalize_up_to:<br/>move head→final, trim head_blocks"]
-    FIN -->|"not in head (cold-start backfill)"| DIRECT["apply_block directly to final,<br/>mirror into head"]
+    FIN -->|"not in head (cold-start backfill)"| DIRECT["apply_block directly to final"]
+    DIRECT --> DOK{"applied?"}
+    DOK -->|yes| MIRROR["mirror into head"]
+    DOK -->|"no (invalid finalized)"| FSTALL["record StallReason on FINAL,<br/>freeze final tip"]
     FIN -->|none| CP
     MOVE --> CP
-    DIRECT --> CP
+    MIRROR --> CP
+    FSTALL --> CP
     CP["persist checkpoint atomically"]
 ```
 
@@ -180,15 +227,15 @@ stateDiagram-v2
     [*] --> Syncing
     Syncing --> CaughtUp: stream drained, no stall
     CaughtUp --> Syncing: new adopted / finalized arrives
-    Syncing --> Parked: apply_block returns Parked(err)
-    CaughtUp --> Parked: invalid block adopted
-    Parked --> Parked: further non-chaining blocks (orphans_since++)
-    Parked --> Syncing: valid successor chains on frozen tip → stall cleared
+    Syncing --> Parked: invalid block FINALIZED (apply_finalized fails)
+    Parked --> Parked: further non-chaining finalized blocks (orphans_since++)
+    Parked --> Syncing: valid successor finalizes on frozen final tip → stall cleared
     note right of Parked
-        head tip frozen at last valid block.
-        Producer (on our turn) builds on this
-        frozen tip, same as honest peers,
-        so the invalid block is skipped.
+        final_stall — persisted, survives restart.
+        Head-tier bad blocks do NOT enter this state:
+        they set head_stall (in-memory) and self-heal
+        via reorg/finalization. Producer (on our turn)
+        builds on the last valid tip either way.
     end note
 ```
 
@@ -211,11 +258,13 @@ stateDiagram-v2
 | 6   | Batch reorg: some `orphaned` + some `adopted` in one event | revert all orphaned, re-derive head, then apply all adopted in order                         | deterministic convergence                       |
 | 7   | Orphan chain (parent transitively off canonical)           | SDK surfaces all affected as `orphaned`; revert each, replay survivors                       | head_state matches new canonical branch         |
 
-**Invalid / bad block**
+**Invalid / bad block** — "stall" below means the **persisted `final_stall`**
+(§4a). A bad block seen only in `adopted` sets the in-memory `head_stall` (not
+persisted); it becomes a persisted `final_stall` only if it finalizes.
 
 | #   | Scenario                                                            | Handling                                                                                 | Expected                                        |
 | --- | ------------------------------------------------------------------- | ---------------------------------------------------------------------------------------- | ----------------------------------------------- |
-| 8   | Authorized sequencer posts a block with an invalid state transition | `apply_block` → `Parked(StateTransition)`; freeze head tip, record stall, mark processed | park-and-skip; no apply, no halt                |
+| 8   | Authorized sequencer posts a block with an invalid state transition | head: `apply_block` → `Parked`, set `head_stall` (in-memory), no persist. If it finalizes: persisted `final_stall` | park-and-skip; no apply, no halt                |
 | 9   | Broken chain link / hash mismatch / unexpected id in adopted        | `Parked(BrokenChainLink / HashMismatch / UnexpectedBlockId)`; same park                  | frozen tip; peers park identically              |
 | 10  | Undeserializable inscription payload                                | park with `Deserialize` (no header); processing advances                                 | recover when a valid block chains on frozen tip |
 | 11  | Valid successor after a park (recovery)                             | block chaining on frozen tip → `Applied` → clear stall                                   | head resumes automatically; no divergence       |