research/reports/EmPoWering/Equi-X/analysis/Equi-X-Vol-II-Deep-Analysis.md
2026-07-30 14:55:21 +02:00

505 lines
39 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# Equi-X — Deep Technical Comparison and Code Analysis
### Volume II: a rigorous C ↔ Rust correspondence, exact algorithmic mechanics, and design analysis
*Part of the Equi-X implementation series — Volume II of V (I: Walkthrough · III: Attacking & Accelerating · IV: PoW Landscape · V: Future Directions) · June 2026 · revised July 2026*
---
## 0. Scope
Volume I built the intuition: HashX manufactures a per-challenge hash function shaped like CPU-friendly code, and Equihash(60, 3) searches for eight inputs whose hashes sum to zero mod 2⁶⁰. This volume assumes that picture and goes underneath it. It is deliberately denser and more code-level, organized around four questions:
1. **Correspondence** — exactly how the C reference and the Rust crates map onto each other, structure by structure.
2. **Mechanics** — the parts Volume I summarized: the PRNG's compatibility contract, the superscalar scheduler, register-selection constraints, the JIT, the bucket solver's arithmetic.
3. **Analysis** — complexity and the birthday accounting; where time and memory actually go; why the solver is heuristic but the verifier is exact.
4. **Discipline** — determinism, bug-for-bug compatibility, and the security-relevant consequences of specific code choices.
Sources are the same four trees as Volume I: C `tevador/equix` + `tevador/hashx` (v1.0.0); Rust `equix` 0.6.1 + `hashx` 0.7.1 (later crate versions — 0.7.0/0.9.0, June 2026 — change packaging, not behavior). Snippets are trimmed (elisions marked `...`) but identifiers, constants, and arithmetic are verbatim; re-verified against the sources, July 2026.
---
## 1. Architectural correspondence
The two codebases are the *same algorithm* under two engineering philosophies: the C is a compact, macro-and-struct performance reference that **defines** the behavior; the Rust is a generic, trait-driven, memory-safe reimplementation that **reproduces** it. The single most useful artifact for reading them together is a map.
### 1.1 HashX (inner hash)
| Concept | C (`hashx/src`) | Rust (`hashx/src`) |
|---|---|---|
| Seed → key material | `hashx_make` → Blake2b(`HashX v1`) → `siphash_state keys[2]` | `SipState::pair_from_seed``(key0, key1)` |
| Decision PRNG | `siphash_rng` (`siphash_rng.c`) over SipHash1,3 | `SipRand` + `RngBuffer` (`rand.rs`) over SipHash1,3 |
| Program generator | `hashx_program_generate` (`program.c`) | `Generator::generate_program` (`generator.rs`) |
| Superscalar model | `generator_ctx.ports[]` + `schedule_instr` | `Scheduler` / `ExecSchedule` (`scheduler.rs`) |
| Constraints | inlined in `select_destination`/`select_template` | `Validator` (`constraints.rs`) |
| Instruction repr | `struct instruction {opcode,src,dst,imm32,op_par}` | `enum Instruction { ... }` (`program.rs`) |
| Interpreter | `hashx_program_execute` (`program_exec.c`) | `Program::interpret` (`program.rs`) |
| JIT | `compiler_x86.c`, `compiler_a64.c` (raw bytes) | `compiler/x86_64.rs`, `aarch64.rs` (`dynasmrt`) |
| Register file + digest | `r[8]` + finalize in `hashx_exec` | `RegisterFile` + `digest` (`register.rs`) |
| Failure signal | `hashx_make` returns `0` | `Error::ProgramConstraints` |
### 1.2 Equi-X (puzzle)
| Concept | C (`equix/src`) | Rust (`equix/src`) |
|---|---|---|
| Public puzzle ops | `equix_solve` / `equix_verify` (`equix.c`) | `EquiX::solve` / `verify`, free fns (`lib.rs`) |
| Solution definition + order/sum checks | `verify_order`, `verify_internal` (`equix.c`), `tree_cmp*` (`solver.h`) | `Solution`, `check_tree_order`, `check_tree_sums` (`solution.rs`) |
| Wagner solver | `solve_stage0..3` (`solver.c`) | `find_solutions` + `collision::search` (`solver.rs`, `collision.rs`) |
| Sorting memory | `struct solver_heap` + `union` (`solver_heap.h`) | `BucketArrayMemory` + `Overlay` union (`bucket_array/`) |
| Parent-pointer pack | `MAKE_ITEM(b,l,r)` | `PackedCollision<u32, 8, 9>` |
| Error taxonomy | `enum equix_result` | `enum Error` (`err.rs`) |
Read any Rust file with its C row open beside it and the design becomes legible from both directions: the Rust narrates *intent* in doc-comments, the C settles *ground truth* in bytes.
---
## 2. The HashX generator, exactly
The generator is the part most worth reading at the level of individual decisions, because *every* decision is part of the spec — a compatible third implementation must reproduce all of them bit for bit.
### 2.1 The PRNG and its compatibility contract
HashX draws generation decisions from SipHash1,3 in counter mode, but the *packaging* of that stream into `u8` and `u32` values is itself normative. Two rules matter:
**Rule 1 — big-endian extraction.** Each 64-bit SipHash output is sliced from the most-significant end. The C shifts down by a decreasing count:
```c
/* hashx/src/siphash_rng.c */
uint8_t hashx_siphash_rng_u8(siphash_rng* gen) {
if (gen->count8 == 0) {
gen->buffer8 = hashx_siphash13_ctr(gen->counter, &gen->keys);
gen->counter++;
gen->count8 = sizeof(gen->buffer8); /* = 8 */
}
gen->count8--;
return gen->buffer8 >> (gen->count8 * 8); /* high byte first */
}
```
The Rust reaches the same order by popping bytes off the end of a little-endian array:
```rust
// hashx/src/rand.rs
let bytes = self.inner.next_u64().to_le_bytes();
let (last, saved) = bytes.split_last()...; // returns MSB first, saves the rest
```
**Rule 2 — separate `u8` and `u32` buffers over one shared counter.** `u8` and `u32` each have their own refill buffer, but both pull from the *same* incrementing counter. So whether a given 64-bit draw becomes bytes or words depends on the **order** in which the generator asks for them — interleaving is part of the contract. The Rust spells this out:
> *"It's important for the u8 and u32 queues to share a common generator. The order of dequeueing u8 items vs u32 items intentionally modifies the assignment of particular u64 RngCore values to the two queues."* — `rand.rs`
Both projects guard this with **captured reference streams** as unit tests (`rng_vectors` in `rand.rs`; the Tor vectors in C). The practical upshot: you cannot refactor the order of `next_u8`/`next_u32` calls without breaking every downstream hash.
### 2.2 The 36-slot instruction template
The opcode attempted at each step is fixed by position in a repeating 36-sub-cycle layout, which is why every accepted program has the same instruction *mix* (notably exactly 192 multiplies and one branch site). The C stores it as an array; the Rust as a match. They are identical:
| sub-cycle mod 36 | C `program_layout[]` | Rust `choose_opcode_selector` | Opcode(s) |
|---|---|---|---|
| 1 | `item_target` | `n == 1` | `TARGET` |
| 19 | `item_branch` | `n == 19` | `BRANCH` |
| 12, 24 | `item_wide_mul` | `n == 12 \|\| 24` | `SMULH`/`UMULH` |
| 0,3,6,9,15,18,21,27,30,33 | `item_mul` | `n % 3 == 0` | `MUL` |
| all others | `item_any` | `Normal` / `ImmediateSrc` | ALU op from a table |
The "any" slot draws from an 8-entry table on the first pass and a 4-entry immediate-source-only table on retry:
```c
/* C: item_any.mask0 = 7 (8 ops), mask1 = 3 (4 ops, no src register) */
const static instr_template* instr_lookup[] =
{ &tpl_ror_c,&tpl_xor_c,&tpl_add_c,&tpl_add_c,&tpl_sub_r,&tpl_xor_r,&tpl_xor_c,&tpl_add_rs };
```
```rust
// Rust: identical tables
const NORMAL_OPS_TABLE: [Opcode;8] = [Rotate,XorConst,AddConst,AddConst,Sub,Xor,XorConst,AddShift];
const IMMEDIATE_SRC_OPS_TABLE:[Opcode;4] = [Rotate,XorConst,AddConst,AddConst];
```
A subtle shared rule: the `item_any` slot forbids repeating the *previous* op's **group** (`duplicates = false` in C; `disallow_opcode_pair` in Rust), and `Sub` and `AddShift` share one group, so they can't be adjacent either. This is checked at the selector level and costs only a re-roll, not a failed instruction.
### 2.3 The superscalar scheduler
The generator places each instruction on a simulated Ivy-Bridge-like core: **3 ports** (the C calls them P0/P1/P5; multiply lives only on P1), per-opcode **latencies** (ALU 1, `MUL` 3, wide-mul 4), and a per-register **retire-cycle scoreboard**. Two mechanisms deserve attention.
**Port allocation order is P5 → P0 → P1**, deliberately checking the multiply port *last* so general ALU ops don't starve multiplications:
```c
/* hashx/src/program.c — schedule_uop */
if ((uop & PORT_P5) && !ctx->ports[cycle][2]) { ...; return cycle; }
if ((uop & PORT_P0) && !ctx->ports[cycle][0]) { ...; return cycle; }
if ((uop & PORT_P1) && !ctx->ports[cycle][1]) { ...; return cycle; }
```
The Rust encodes the same priority in its bit numbering and iteration:
```rust
// scheduler.rs — P5 = 1<<0, P0 = 1<<1, P1 = 1<<2; iterate indices 0,1,2
for index in 0..NUM_EXECUTION_PORTS { if (ports.0 & (1<<index)) != 0 && !busy { return ... } }
```
**Two-µop instructions** (the wide multiplies, `TARGET`, `BRANCH`) are scheduled *conservatively*: the generator searches forward for the first cycle on which **both** µops can issue simultaneously. The C tries a non-committing probe of both, then commits only on a match:
```c
/* schedule_instr, 2-uop branch */
for (int cycle = ctx->cycle; cycle < PORT_MAP_SIZE; ++cycle) {
int c1 = schedule_uop(tpl->uop1, ctx, cycle, false); // probe
int c2 = schedule_uop(tpl->uop2, ctx, cycle, false);
if (c1 >= 0 && c1 == c2) { /* commit both */ return c1; }
}
```
The Rust `instruction_plan` does precisely this two-port "same cycle" search and returns an `InstructionPlan` carrying the issue cycle and the chosen ports. The scoreboard then records the destination register's **retire cycle** (`issue + latency`) so later instructions only pick operands that are ready.
### 2.4 Register selection and the constraint stack
Once an opcode and a timing plan exist, the generator must pick source/destination registers that (a) are ready by the issue cycle and (b) satisfy the anti-shortcut rules. The C builds the candidate set with a **branchless filter** that ANDs five conditions and accumulates without a branch:
```c
/* program.c — select_destination (conditions condensed) */
for (int i = 0; i < 8; ++i) {
bool available = ctx->registers[i].latency <= cycle; // ready in time
available &= ((!tpl->distinct_dst) | (i != instr->src)); // dst != src (some ops)
available &= (ctx->chain_mul | (tpl->group != INSTR_MUL_R)
| (ctx->registers[i].last_op != INSTR_MUL_R));// no back-to-back MUL
available &= ((ctx->registers[i].last_op != tpl->group)
| (ctx->registers[i].last_op_par != instr->op_par)); // no trivial repeat
available &= ((instr->opcode != INSTR_ADD_RS) | (i != 5)); // R5 not ADD_RS dst
available_regs[regs_count] = available ? i : 0;
regs_count += available;
}
```
The Rust expresses the identical five conditions through `RegisterSet::from_filter` plus a `Validator::dst_registers_allowed` checker — same logic, type-checked instead of bit-ANDed. The conditions exist to forbid *optimizable* sequences (`xor r,r`; `ror r,c1; ror r,c2`; runaway multiplication that floods a register with trailing zeros) so that an attacker can't simplify the function.
Two details that a reimplementation must copy and that the Rust flags explicitly:
- **The R5 / `ADD_RS` source short-circuit.** When only two registers are free for an `ADD_RS` and one is R5 (which can't be the *destination*), HashX deterministically forces R5 to be the *source* rather than rolling the dice — preventing a frequent dead-end. Present in both (`select_source` in C; `src_registers_allowed` in Rust).
- **The wide-multiply "vestigial" RNG draw.** `UMULH`/`SMULH` are `op_par_src = false, distinct_dst = false`, so the C falls into `op_par = hashx_siphash_rng_u32(gen)` — it consumes a `u32` whose *only* use is the "no trivial repeat" check above. The Rust keeps it as `RegisterWriter::UMulH(u32)` with a comment calling it a probable bug it cannot remove without breaking compatibility. It costs one PRNG draw per wide multiply and changes the stream alignment for everything after it.
### 2.5 Multi-pass generation, acceptance, and the 194/195 question
Each slot is attempted up to twice before time advances. The C uses an `attempt` counter and `MAX_RETRIES = 1`; on the retry it sets `chain_mul = true` (allowing back-to-back multiplies on a register if nothing else fits — the "prevents catastrophic failure" path). If both attempts fail it advances a full cycle (`sub_cycle += 3`) and resets. The Rust mirrors this as `Pass::Original → Pass::Retry → scheduler.stall()`.
Generation stops when an instruction would schedule at or beyond cycle 192 (`TARGET_CYCLE`), or the 512-slot buffer fills. A finished program is then **accepted only if it hits three exact targets**:
```c
/* program.c — the only thing that makes a program "real" */
return (program->code_size == 512)
& (ctx.mul_count == 192)
& (ctx.latency == REQUIREMENT_LATENCY - 1); /* 195 - 1; "cycles numbered from 0" */
```
```rust
// constraints.rs — same three gates
instructions.len() == 512
&& scheduler.overall_latency().as_usize() == 194
&& multiply_count == 192
```
This resolves a discrepancy a careful reader will hit: the C `#define REQUIREMENT_LATENCY 195` but tests `latency == 194`; the Rust hard-codes `194`. They agree — the final register write must retire on the cycle indexed 194 (the "195th"). The C source notes ~**1 seed in 10,000** fails these gates; the Rust says "once per several thousand." Callers must skip such seeds — this is the `EQUIX_CHALLENGE` / `Error::ProgramConstraints` path, distinct from the ~0.2% of seeds that *generate fine* but are collision-weak (handled by Layer 2's use of addition).
> **What the generator guarantees.** Every accepted instance is a 512-instruction, 192-multiply program whose dependency chain exactly fills a ~195-cycle window on a realistic 3-port core. That uniformity is the proof-of-work "shape": equal cost across instances, dominated by a long multiply chain a CPU executes natively and a GPU/FPGA cannot meaningfully shorten. Generation is itself cheap — O(512) instructions, each a bounded forward search over the ~196-entry port map.
---
## 3. HashX execution: interpreter and JIT
### 3.1 The exact hash pipeline
`H(input)` is fully determined by: initialize 8 registers from `key1` and the input via SipHash2,4 counter mode; run the program; finalize. The arithmetic is unsigned 64-bit **wrapping** throughout, constants are **sign-extended** from 32 bits, rotates are by a constant, and the wide multiplies keep the **upper** 64 bits of a 128-bit product. The finalization (identical in both; verified byte-for-byte) folds `key1` back in to cancel the multiply-induced bias toward zero:
```c
/* hashx.c */
r[0]+=keys.v0; r[1]+=keys.v1; r[6]+=keys.v2; r[7]+=keys.v3;
SIPROUND(r[0],r[1],r[2],r[3]); SIPROUND(r[4],r[5],r[6],r[7]);
/* out = r0^r4, r1^r5, r2^r6, r3^r7 ; Equi-X keeps only out[0..8) */
```
### 3.2 The one-shot branch in three forms
The branch is the same idea expressed three ways, and seeing all three removes any ambiguity about its semantics.
**Interpreter** (both languages identical in effect): a `branch_enable` fuse, blown the first time any branch is taken; the test is against the low 32 bits of the most recent wide-multiply result:
```c
/* program_exec.c */
case INSTR_BRANCH:
if (branch_enable && (result & instr->imm32) == 0) { i = target; branch_enable = false; }
break;
```
One micro-difference worth noting for bit-pedants: on a taken branch the **C interpreter resumes at `target + 1`** (the `for` loop's `++i` after `i = target`), whereas the **Rust interpreter re-executes the `Target` instruction** (a no-op that just re-marks the target). Identical results, since `Target` has no effect once the fuse is blown.
**Compiled** (C x86): the one-shot is implemented without a second test by exploiting the flags that caused the jump. The `Target` site is `test edi,edi; «label»: cmovz esi,edi`, and `Branch` is `or edx,esi; test edx,imm; jz label`. When `jz` is taken, the zero flag is still set on arrival at the label, so `cmovz` moves `edi`(= 1) into `esi`; thereafter `or edx,esi` forces the `test` non-zero and the branch can never be taken again. A genuinely elegant branchless disable, and exactly the sort of thing the interpreter exists to specify unambiguously.
### 3.3 JIT: raw bytes vs `dynasmrt`
Both compile the program to native code that takes a pointer to the register file; the difference is how bytes are produced.
The **C** emits pre-encoded machine code, one `EMIT` per instruction, mapping HashX R0R7 to native `r8``r15`:
```c
/* compiler_x86.c — e.g. MUL_R is a single 4-byte imul */
case INSTR_MUL_R:
EMIT_U32(pos, 0xc0af0f4d | (instr->dst << 27) | (instr->src << 24)); /* imul dst,src */
```
It brackets emission with a **W^X** transition — `hashx_vm_rw(code,…)` before writing, `hashx_vm_rx(code,…)` after — and sizes the page as `align(512·5 + 1024, 4096)` (`COMP_AVG_INSTR_SIZE = 5`). The ARM64 backend (`compiler_a64.c`) does the same with fixed 4-byte instructions (`ldr x7,[x0,#56]`, …).
The **Rust** uses the `dynasmrt` runtime assembler, so the backend reads like annotated assembly and the buffer is an `ExecutableBuffer` mmap managed by the crate:
```rust
// hashx/src/compiler/x86_64.rs
use dynasmrt::{DynasmApi, DynasmLabelApi, x64};
dynasm!(asm ; .arch x64 ; mov rcx, rdi ; ...);
```
Both default to **try-compile-then-fall-back-to-interpret** (`RuntimeOption::TryCompile`; in C, `hashx_alloc(HASHX_COMPILED)` else `HASHX_INTERPRETED`), and the compiler is only available on x86-64 and aarch64 — elsewhere the interpreter is mandatory. The crucial invariant across all four execution engines — C-interp, C-x86, C-arm, and the Rust trio — is **bit-identical output**, enforced by shared test vectors.
> **Determinism is a cross-product property.** {interpret, x86 JIT, arm JIT} × {C, Rust} must all agree, for every seed and input. That is why the generator's PRNG order, the wrapping/sign-extension rules, the digest, and even the "vestigial" draw are pinned: any of them differing would split the matrix.
---
## 4. The Equihash search, analyzed
With `H` fixed, the puzzle is a constrained subset-sum. This section states it formally, explains the solver's arithmetic precisely, and derives its complexity.
### 4.1 The puzzle, formally
A solution is eight 16-bit indices `i0..i7` such that, with `H(i)` the low 64 bits of the inner hash and all sums taken mod 2⁶⁴:
- **Full sum:** `Σ H(iₖ) ≡ 0 (mod 2⁶⁰)`.
- **Tree partial sums (Wagner structure):** each adjacent pair zeros the low 15 bits; each group of four zeros the low 30; all eight zero 60. (`N/(K+1) = 60/4 = 15` bits per level.)
- **Ordering (canonicalization):** at each tree node the left branch must be ≤ the right branch when the index groups are compared as little-endian byte strings (`tree_cmp1/2/4` over 2/4/8 bytes).
The ordering constraint is not cosmetic: without it the same eight hashes could be permuted into many distinct "solutions," so it is a genuine validity condition and is checked *first* (before any hashing) during verification. The Rust verifier is the cleanest statement of the sum tree — one recursion halving the bit-width at each level:
```rust
// equix/src/solution.rs
fn check_tree_sums(func, items, n_bits) -> Result<HashValue, ()> {
let sum = if items.len() == 2 {
item_hash(func, items[0]).wrapping_add(item_hash(func, items[1]))
} else {
let (l, r) = items.split_at(items.len()/2);
check_tree_sums(func,l,n_bits/2)?.wrapping_add(check_tree_sums(func,r,n_bits/2)?)
};
if (sum & ((1 << n_bits) - 1)) == 0 { Ok(sum) } else { Err(()) } // 60 → 30 → 15
}
```
### 4.2 Wagner as bucketed meet-in-the-middle, with the carry
Finding the sum directly is `2^128`. Wagner's algorithm finds it level by level: to zero 15 more bits of a sum, bucket every value by those 15 bits and only combine **complementary** buckets — bucket `b` with bucket `b mod 2¹⁵` — because only those can cancel. Each 15-bit cancellation is split into a **coarse** pass (8 bits, `value % 256`) and a **fine** pass (7 bits, `value % 128`); `8 + 7 = 15`.
The one piece of arithmetic that is easy to get wrong — and therefore a precise compatibility checkpoint — is the **carry correction**. When you pair coarse bucket `b` with bucket `256 b`, their coarse parts sum to exactly 256, producing a carry out of the 8-bit field. The C re-injects it before the fine pass with `value + CARRY`, where `CARRY = (bucket_idx != 0)`:
```c
/* solver.c — MAKE_PAIRS1 (per layer; same shape for 2 and 3) */
stage1_data_item value = STAGE1_DATA(bucket_idx, item_idx) + CARRY; /* +1 if b != 0 */
u32 fine_buck_idx = value % NUM_FINE_BUCKETS;
u32 fine_cpl_bucket = INVERT_SCRATCH(fine_buck_idx); /* -f mod 128 */
... stage1_data_item sum = value + cpl_value;
assert((sum % NUM_FINE_BUCKETS) == 0); /* fine bits cancel */
sum /= NUM_FINE_BUCKETS; /* shift off 7 bits */
```
The Rust performs the identical correction inside `collision::search`, iterating `first_bucket in 0..=(N/2)` and pairing with `first_bucket.wrapping_neg() % N`, then keeping sums whose `low_bits_are_zero(num_bits)`. Same buckets, same complement, same carry — by construction, so that both implementations *discard the same overflow solutions* and emit identical output.
### 4.3 Complexity and the birthday accounting
Let `S = 2¹⁶` be the index space. The four stages:
- **Stage 0** — evaluate `H` on all `S` indices; bucket by 8 bits into 256 coarse buckets. Mean occupancy `S/256 = 256` items/bucket (cap `COARSE_BUCKET_ITEMS = 336`). Cost: `S` HashX evaluations — *this dominates a solve.*
- **Stage 1** — within each complementary coarse-bucket pair (~512 items) sub-bucket by 7 fine bits (128 fine buckets, cap 12) and combine complements. Pairs with low 15 bits zero ≈ `C(S,2) / 2¹⁵ ≈ 2³¹ / 2¹⁵ = 2¹⁶`. So ~`S` items survive into layer 2.
- **Stage 2** — same on the 15-bit-zero items; ~`2¹⁶ / 2¹⁵·... ≈ 2¹⁶` survive with 30 bits zero.
- **Stage 3** — match the final 30 bits; expected solutions ≈ `2¹⁶ / 2¹⁵ ≈ 2`.
That last line is the origin of the famous **"~2 solutions per challenge on average."** Each layer keeps the population near `2¹⁶`, so total work is `Θ(2¹⁶)` operations with a small constant, and **time is dominated by the 65,536 stage-0 hash evaluations**, not the bucketing. Peak memory is the fixed solver heap, ≈ **1.81 MiB**.
**Why the solver is heuristic but the verifier is exact.** Bucket capacities (336 coarse, 12 fine) are finite, and occupancy is roughly Poisson. Coarse mean 256 vs cap 336 is ~5σ (σ ≈ 16), so overflow is rare but *possible*; when a bucket overflows, extra items are silently dropped (`if (item_idx >= CAP) continue;`). Therefore the solver may **miss** some valid solutions — it is a probabilistic search tuned to find *enough*, not *all*. The Rust documents this as intentional and matches the C's capacities precisely so the two miss the *same* solutions. The **verifier**, by contrast, does no bucketing and is exact: it recomputes the eight hashes and checks the sums directly. This asymmetry is fine for a client puzzle — a solver only needs one solution; a verifier must be sound.
### 4.4 Self-complementary buckets, dedup, and the solution cap
Two coarse buckets are their own complement: `0` (since `0 = 0`) and `128` (since `128 ≡ 128 mod 256`). Pairing a bucket with itself would double-count and produce duplicate pairs, so the solver special-cases `cpl_bucket == bucket_idx`: it interleaves matching into the scratch-building loop so each item only pairs with *earlier* items in the same bucket. The C carries a `nodup`/branch for this; the Rust's `collision::search` handles it by construction. The solver returns at most `EQUIX_MAX_SOLS = 8` solutions (the Rust `SolutionArray` is an `ArrayVec<Solution, 8>`), and dedups adjacent equal solutions before pushing.
### 4.5 Parent pointers and canonical reconstruction
Each combined node stores where its two parents live, packed into one integer with **8 bits of bucket + 9 + 9 bits of item index** — identical widths in both:
```c
#define MAKE_ITEM(bucket, left, right) ((left) << 17 | (right) << 8 | (bucket))
```
```rust
type Layer0Collision = PackedCollision<u32, 8, 9>; // BUCKET_BITS=8, ITEM_BITS=9
```
At the end the solver walks this tree back to the leaves to recover the eight indices, then puts them in canonical order. The C does it with explicit conditional swaps at each level (`tree_cmp1/2/4` deciding whether to swap pairs/quads/halves); the Rust collects the leaves and calls `Solution::sort_from_array`, whose `branches_are_sorted` predicate compares groups reversed (i.e., as little-endian values). Same canonical form, so the emitted 16-byte solutions match.
---
## 5. Solver memory: two designs for the same 1.81 MiB
Both implementations keep the entire search in one preallocated block sized to live in cache, and both **reuse** the same bytes across stages via a union. The implementations of that idea are a study in contrasts.
**C — one struct, one union, hand-counted bytes.** `solver_heap.h` lays out every table explicitly and overlaps the early-stage data with the late-stage tables:
```c
typedef struct solver_heap {
stage1_idx_hashtab stage1_indices; /* 172 544 B */
stage2_idx_hashtab stage2_indices; /* 344 576 B */
stage2_data_hashtab stage2_data; /* 688 128 B */
union { /* reuse: early vs late stages */
stage1_data_hashtab stage1_data; /* 688 128 B (stages 01) */
struct { stage3_idx_hashtab stage3_indices; /* 344 576 B */
stage3_data_hashtab stage3_data; }; /* 344 064 B (stage 3)*/
};
fine_hashtab scratch_ht; /* 3 200 B */
} solver_heap; /* TOTAL: 1 897 088 B */
```
Per-bucket counts live *inside* each table (`uint16_t counts[256]`), and the block is obtained with plain `malloc` — or `hashx_vm_alloc_huge` when `EQUIX_CTX_HUGEPAGES` is set, the one allocation knob the C exposes.
**Rust — `MaybeUninit` layout, checked union, counts outside.** The same overlap is a `union Overlay { first: OverlayFirst, second: OverlaySecond }`, but the whole structure is built from `BucketArrayMemory<N, M, T>([[ MaybeUninit<T>; M]; N])` and marked with an `unsafe` `Uninit` trait that promises the bytes are safe to leave uninitialized until written. Allocation goes straight to the heap (too large for the stack) and is reused across solves via `SolverMemory`:
```rust
// bucket_array/mem.rs — the safety-critical insert: count only ever rises after a real write
fn insert<F: FnMut(usize)>(&mut self, bucket: usize, mut writer: F) -> Result<(), ()> {
let n: usize = self.counts[bucket].into();
if n < CAP { writer(n); self.counts[bucket] = self.counts[bucket] + C::one(); Ok(()) }
else { Err(()) }
}
```
The safety argument is explicit and worth appreciating: reads use `assume_init`, which is only sound if `counts` accurately reflects which slots were written; the code guarantees this by *only* incrementing a count after the writer has unconditionally written, and by tying everything to a `&mut` whose lifetime begins with zeroed counts. Switching layouts is done by borrowing a different union field, so the borrow checker enforces that the two overlays are never live at once. Because the bucket *counters* live outside the overlaid block, the Rust's measured size is **1,895,424 B** vs the C's **1,897,088 B** — a ~1,664-byte difference of bookkeeping placement, not of algorithm.
| | C `solver_heap` | Rust `SolverMemory` |
|---|---|---|
| Total | 1,897,088 B | 1,895,424 B |
| Counters | inside each table | in separate `BucketState` |
| Reuse mechanism | `union` | checked `union Overlay` |
| Uninit handling | raw `malloc` (bytes undefined) | `MaybeUninit` + `unsafe Uninit` |
| Big pages | `EQUIX_CTX_HUGEPAGES` | (none) |
| Reuse across solves | caller keeps `ctx->heap` | `solve_with_memory(&mut SolverMemory)` |
| Bucket geometry | 256×336 / 128×12 | **identical** |
The identical geometry is the point: same capacities ⇒ same overflow-discard behavior ⇒ identical solution sets.
---
## 6. API surface and error models
The libraries diverge most visibly at their edges, and the differences are idiomatic rather than algorithmic.
**Lifecycle.** C uses an opaque context allocated once and reused (`equix_alloc(flags)` → many `equix_solve`/`equix_verify`), with `flags` selecting verify/solve, compiled/interpreted, and hugepages. Rust uses values and builders (`EquiX::new(challenge)`, `EquiXBuilder` for `RuntimeOption`), plus free functions (`equix::solve`, `verify`, `verify_bytes`) for the common path. Solver scratch is a caller-held `ctx->heap` in C and a `SolverMemory` you can thread through `solve_with_memory` in Rust.
**Errors.** The taxonomies line up one-to-one, but Rust splits the *order* check into the type system:
| C `equix_result` | Rust | When |
|---|---|---|
| `EQUIX_OK` | `Ok(())` | valid |
| `EQUIX_ORDER` | `Error::Order` | indices not canonical (checked first, no hashing) |
| `EQUIX_CHALLENGE` | `Error::Hash(ProgramConstraints)` | HashX won't build for this seed |
| `EQUIX_PARTIAL_SUM` / `EQUIX_FINAL_SUM` | `Error::HashSum` | tree/full sum check failed |
In Rust you cannot hold a `Solution` without having passed the order check (it's enforced in `try_from_bytes`/`try_from_array`), so `EquiX::verify` only ever needs to check sums — the same cheapest-first ordering as the C `equix_verify`, but encoded in types. The Rust additionally surfaces `Error::Hash(Compiler(...))` when `RuntimeOption::CompileOnly` is chosen and the JIT is unavailable, a state the C reaches via the `HASHX_NOTSUPP` sentinel at `alloc` time.
---
## 7. Determinism and compatibility as an engineering discipline
Equi-X has no prose specification; correctness *is* reproducing the reference's output. Reading the two codebases together, you can enumerate exactly what a third implementation must match. This is the checklist:
- **Endianness:** hash output read little-endian (`load64`); PRNG bytes/words extracted big-endian from each SipHash block.
- **PRNG consumption order:** separate `u8`/`u32` buffers over a shared counter; the *interleaving* of draws is significant (§2.1).
- **The vestigial wide-multiply `u32` draw** (§2.4) — must be consumed even though unused, or the stream desynchronizes.
- **Arithmetic semantics:** 64-bit wrapping add/sub/mul; 32-bit immediates sign-extended; wide multiply keeps the high 64 bits; rotate-right by constant.
- **Acceptance gates:** 512 instructions, 192 multiplies, retire at cycle 194 (§2.5).
- **Digest:** fold `key1` into `r0,r1,r6,r7`, one SipRound per half, XOR-combine (§3.1).
- **Solver geometry:** 256/128 buckets, 336/12 capacities, the `b ↔ b` complement, the `+CARRY` correction, the 8/9/9 parent packing, and the canonical ordering (§4) — so even *which* solutions are discarded matches.
The Rust enforces all of this with **captured reference vectors** at every layer (`rng_vectors`, `siphash24_ctr_vectors`, `hashx_vectors`, `tor_equix_vectors`), which is how it can be both idiomatic and bug-for-bug faithful. The few intentional internal differences are behaviorally invisible: the interpreter's `target + 1` vs re-running `Target` (§3.2), and where bucket counters are stored (§5). Everything an external observer can see — generated programs, hash outputs, the set of emitted solutions, verification verdicts — is identical.
---
## 8. Security-relevant code analysis
Reading the implementations surfaces several properties that bear on Equi-X's security posture. Stated precisely, with what the code does and does not defend:
- **No secret inputs.** HashX's control flow is input-dependent (the one-shot branch tests live data), so evaluation time can leak the input. The code is explicit that this is acceptable *only because the puzzle input is public*. Using HashX as a keyed hash over secret data would be a timing-side-channel mistake. (Volume I's "intuition" caveat, here as a hard rule.)
- **Weak instances are neutralized by addition, not avoided.** ~0.2% of seeds produce HashX functions with many internal collisions. Under XOR these would yield enormous numbers of trivial solutions; Equi-X's switch to **sum mod 2⁶⁰** means equal hashes no longer cancel, so weak instances don't become exploitable. The solver and verifier therefore never need a collision-resistance assumption on HashX — only preimage resistance.
- **Solver non-exhaustiveness is sound.** Bucket overflow silently discards candidates (§4.3), so the solver can miss solutions — but it can never *invent* one, because every emitted solution is rebuilt from real items and (in practice) re-checked by the exact verifier. Missing solutions only costs the solver attempts, never soundness.
- **JIT hardening.** The compiled path uses W^X page permissions (`vm_rw` to emit, then `vm_rx` to execute) so the code page is never simultaneously writable and executable. Emission is straight-line from a trusted program with no input-derived lengths, limiting the JIT's attack surface.
- **The memory-hardness/branch interplay is by design and acknowledged as imperfect.** HashX itself is register-only (no scratchpad), which is what makes its program shape so CPU-like; the Equihash layer adds the ~1.81 MiB cache-resident working set that blunts a GPU's bandwidth edge and gives a logic-only FPGA something it must store. The generator even computes a hypothetical **"ASIC latency"** (`asic_latencies`, assuming unlimited parallelism and 1-cycle ops) in its stats build — a window into the designer's own modeling of the parallel lower bound. None of this targets true ASIC-resistance, which Equi-X explicitly abandons.
- **Memory-safety boundary.** The C solver is classic manual indexing into a `malloc`'d block; correctness rests on the capacity checks (`if (idx >= CAP) continue;`). The Rust achieves the same layout but routes every read through `assume_init` guarded by a borrow-checked count, converting "don't read uninitialized memory" from a discipline into a compile-time-checked invariant. Same bytes, very different safety story.
---
## 9. Performance model
A coherent mental model of where the cost goes, assembled from the code structure (absolute figures are the designer's, on era-specific hardware, and approximate):
- **Solve ≈ 58 ms.** Decomposes as: one program generation (~0.05 ms, O(512) with bounded per-slot search), then `2¹⁶` HashX evaluations in stage 0 (the dominant term), then three `Θ(2¹⁶)` bucket passes (cheap relative to hashing). HashX overhead vs the search is well under 1%.
- **Verify ≈ 50 µs.** Eight HashX evaluations + a few additions + six integer comparisons, with the free ordering check rejecting malformed proofs before any hash runs.
- **The asymmetry, quantified.** A solver performs ~`2¹⁶` hash evaluations to a verifier's 8 — a factor of ~`2¹³ ≈ 8000` in hashing alone — and the wall-clock ratio is ~100×. The 16-byte proof keeps the *verifier's* input cost negligible, which is the property that makes Equi-X usable as a DoS gate rather than a self-inflicted one.
- **JIT vs interpret.** Compiled HashX is roughly an order of magnitude faster than interpreted and is the deployed path; both are required to agree bit-for-bit, so the interpreter doubles as the executable specification and the portable fallback.
- **C vs Rust.** The Rust crate's own tests note it is modestly slower than the C reference (its comments reference small solver/verifier deltas), the expected cost of generic, bounds-checked, memory-safe code; the algorithmic complexity and outputs are identical.
---
## 10. Synthesis
The C and Rust implementations are a near-perfect natural experiment: the same nontrivial algorithm written once for *speed and definition* and once for *safety and clarity*, constrained to produce identical output down to which solutions they discard. Reading them in parallel is the most efficient way to understand Equi-X, because each compensates for the other's weakness as a teaching text — the C answers "what exactly happens," the Rust answers "why, and what's safe."
The deeper lesson the code carries is that Equi-X's security rests less on any single clever primitive than on **disciplined uniformity**: a program generator that manufactures equal-cost, CPU-shaped functions; a combiner (addition) chosen so a weak hash can't be exploited; a solver engineered to stay in cache; and a verifier kept structurally cheap and exact. The implementations make those choices legible in a way the (nonexistent) prose spec never could.
---
## Appendix A — Constant and parameter correspondence
| Quantity | Value | C symbol | Rust symbol |
|---|---|---|---|
| Program size | 512 | `REQUIREMENT_SIZE` / `HASHX_PROGRAM_MAX_SIZE` | `NUM_INSTRUCTIONS` |
| Required multiplies | 192 | `REQUIREMENT_MUL_COUNT` | `REQUIRED_MULTIPLIES` |
| Scheduler target | 192 cycles | `TARGET_CYCLE` | `TARGET_CYCLES` |
| Required final retire | cycle 194 | `REQUIREMENT_LATENCY - 1` | `REQUIRED_OVERALL_RESULT_AT_CYCLE` |
| Ports | 3 (P5,P0,P1) | `NUM_PORTS` | `NUM_EXECUTION_PORTS` |
| Latencies | 1 / 3 / 4 | per `instr_template.latency` | `instruction_latency_cycles` |
| Branch mask weight | 4 (≈1/16) | `LOG2_BRANCH_PROB` | `BRANCH_MASK_BIT_WEIGHT` |
| Registers | 8; R5 special | `REGISTER_NEEDS_DISPLACEMENT = 5` | `register::R5` |
| Gen reject rate | ~1 / 10⁴ seeds | comment in `program.c` | doc in `program.rs` |
| Weak-seed rate | ~0.2% | (HashX README) | (HashX README) |
| Equihash N, K | 60, 3 | `EQUIX_*_MASK` widths | `EQUIHASH_N`, `EQUIHASH_K` |
| Items / solution | 8 → 16 B | `EQUIX_NUM_IDX` | `Solution::NUM_ITEMS` |
| Index space | 2¹⁶ | `INDEX_SPACE` | `u16::MIN..=u16::MAX` |
| Coarse / fine buckets | 256 / 128 | `NUM_COARSE/FINE_BUCKETS` | bucket array `N` params |
| Bucket capacities | 336 / 12 | `COARSE/FINE_BUCKET_ITEMS` | `CAP` params |
| Parent pack | 8 / 9 / 9 bits | `MAKE_ITEM` shifts | `PackedCollision<u32,8,9>` |
| Stage masks | 2¹⁵, 2³⁰, 2⁶⁰ 1 | `EQUIX_STAGE1/STAGE2/FULL_MASK` | `n_bits` recursion |
| Solver memory | ~1.81 MiB | 1,897,088 B (`solver_heap`) | 1,895,424 B (`SolverMemory::SIZE`) |
| Max solutions | 8 | `EQUIX_MAX_SOLS` | `SolutionArray` capacity |
## Appendix B — Instruction template fields (C `instr_template`)
Each C template carries the fields the generator and JIT consume; the Rust spreads the same data across `scheduler::model` (ports/latency), `generator::model` (selection), and `program::Instruction` (semantics).
| Opcode | latency | uop1 / uop2 | distinct dst? | has src? | immediate |
|---|---|---|---|---|---|
| `UMULH_R` / `SMULH_R` | 4 | P1 / P5 | no | yes | — (draws an unused `op_par` u32) |
| `MUL_R` | 3 | P1 / — | yes | yes | — |
| `SUB_R` | 1 | P015 / — | yes | yes | — |
| `XOR_R` | 1 | P015 / — | yes | yes | — |
| `ADD_RS` | 1 | P01 / — | yes | yes | 2-bit shift; dst ≠ R5 |
| `ROR_C` | 1 | P05 / — | yes | no | 6-bit rotate (nonzero) |
| `ADD_C` / `XOR_C` | 1 | P015 / — | yes | no | 32-bit (nonzero) |
| `TARGET` / `BRANCH` | 1 | P015 / P015 | — | no | branch: 4-bit-weight mask |
## Appendix C — Provenance
- **C:** `tevador/equix`, `tevador/hashx`, v1.0.0 (2020), LGPL-3.0. The normative reference; no separate written spec.
- **Rust:** `equix` 0.6.1, `hashx` 0.7.1 (crates.io), Tor Project / Arti, LGPL-3.0. Reproduces the C output exactly; ships shared Tor test vectors. Later versions (`equix` 0.7.0 / `hashx` 0.9.0, June 30, 2026) carry no algorithmic changes.
- Snippets trimmed for readability (`...`); identifiers, constants, and arithmetic verbatim from the cited files (re-verified July 2026). Read alongside Volume I (the Walkthrough) for the conceptual model this analysis assumes.