mirror of
https://github.com/logos-blockchain/research.git
synced 2026-08-07 11:43:20 +00:00
Importing analysis of Equi-X
This commit is contained in:
Marcin Pawlowski
parent
ae54acd83d
commit
3302f7b96b
61
reports/EmPoWering/Equi-X/analysis/Equi-X-README.md
Normal file
61
reports/EmPoWering/Equi-X/analysis/Equi-X-README.md
Normal file
@ -0,0 +1,61 @@
|
||||
# Equi-X — Documentation Set
|
||||
|
||||
Equi-X is the CPU-friendly, GPU/FPGA-resistant **asymmetric client puzzle** that Tor onion services use to defend against denial-of-service (shipped in Tor 0.4.8, Aug 2023). It is `Equihash(60, 3)` with two changes: the Blake2b inner hash is replaced by **HashX** (a per-challenge randomly-generated hash function), and XOR is replaced by **modular addition mod 2⁶⁰**.
|
||||
|
||||
This **Equi-X Documentation** folder holds the **5-volume Markdown series** that reads the actual reference code. Its DOCX companions (the Technical Reference and Research Survey) and the broader project surveys live in the **main project folder, one level up**. This file is the index to all of them.
|
||||
|
||||
---
|
||||
|
||||
## The implementation series (Markdown)
|
||||
|
||||
A five-volume set built by reading the real implementations — the C reference (`tevador/equix` + `tevador/hashx`, v1.0.0) and the Rust crates (`equix` 0.6.1 + `hashx` 0.7.1, from Tor's Arti). Those are the versions *read*; as of July 2026 the C is unchanged (v1.0.0 remains the only tagged release) and the crates have moved to `equix` 0.7.0 / `hashx` 0.9.0 (June 30, 2026) with no algorithmic changes — output compatibility is frozen by the shared test vectors. The volumes cross-reference each other; read in order for a full treatment, or jump to the one matching your question.
|
||||
|
||||
| # | File | What it answers |
|
||||
|---|------|-----------------|
|
||||
| I | [Equi-X-Vol-I-Walkthrough.md](Equi-X-Vol-I-Walkthrough.md) | **How it works.** Intuition-first, data-flow walkthrough of both layers (HashX → the Equihash search), C and Rust side by side. |
|
||||
| II | [Equi-X-Vol-II-Deep-Analysis.md](Equi-X-Vol-II-Deep-Analysis.md) | **How it's built.** Rigorous C↔Rust correspondence, the generator/scheduler internals, solver complexity & birthday math, memory, determinism, security. |
|
||||
| III | [Equi-X-Vol-III-Attacking-and-Accelerating.md](Equi-X-Vol-III-Attacking-and-Accelerating.md) | **How well it holds up.** Optimized solving, GPU/FPGA/ASIC feasibility grounded in the code, and an honest verdict on the CPU-friendliness claim. |
|
||||
| IV | [Equi-X-Vol-IV-PoW-Landscape.md](Equi-X-Vol-IV-PoW-Landscape.md) | **Where it fits.** Lineage and placement against RandomX, Equihash, Argon2/yespower, Cuckoo Cycle, ProgPoW, on the time/memory/bandwidth axes. |
|
||||
| V | [Equi-X-Vol-V-Future-Directions.md](Equi-X-Vol-V-Future-Directions.md) | **What's next.** HashWX (the successor primitive), Proposal 362 (the control-loop fix), and a prioritized open-research roadmap. |
|
||||
|
||||
### Where to start
|
||||
|
||||
- **New to Equi-X?** Volume I, then II.
|
||||
- **Implementing or porting it?** Volumes I–II (II §7 is the bit-exact compatibility checklist).
|
||||
- **Evaluating its security / hardware resistance?** Volume III, then V §3.
|
||||
- **Choosing a PoW / comparing designs?** Volume IV.
|
||||
- **Planning the next version?** Volume V.
|
||||
|
||||
---
|
||||
|
||||
## The book (single-source consolidation)
|
||||
|
||||
Everything below — the five volumes, the Technical Reference, the Research Survey, and a component-by-component C↔Rust implementation study — is consolidated, de-duplicated, and fact-checked into one book:
|
||||
|
||||
- **[Equi-X: The Complete Reference](../Books/Equi-X-Complete-Reference.pdf)** (`Books/Equi-X-Complete-Reference.pdf`; LaTeX source alongside it). If you read only one document, read this one.
|
||||
|
||||
---
|
||||
|
||||
## The reference documents (DOCX)
|
||||
|
||||
Specification- and survey-grade companions to the code series. The `-revised` copies are the current versions (originals retained).
|
||||
|
||||
- **[Equi-X-Technical-Reference-revised.docx](../Equi-X-Technical-Reference-revised.docx)** — implementation-grade specification of the algorithm and the Tor integration (the *what*, to the series' *how*).
|
||||
- **[Equi-X-Research-Survey-revised.docx](../Equi-X-Research-Survey-revised.docx)** — the literature/landscape survey: the design record, independent scrutiny (OnionFlation, Proposal 362), and HashWX.
|
||||
|
||||
---
|
||||
|
||||
## Broader project context
|
||||
|
||||
The volumes cross-reference these wider surveys in the main project folder (one level up), which place Equi-X within the larger proof-of-work space:
|
||||
|
||||
- **Acceleration-Resistant-PoW-Survey.docx** — the master taxonomy (Families A–F, the three parameter axes) that Volume IV builds on.
|
||||
- **RandomX-Research-Survey.docx** / **RandomX-Technical-Reference.docx** — Equi-X's direct ancestor (HashX descends from RandomX's SuperscalarHash).
|
||||
- **Memory-Latency-Bound-Functions.docx** — theory background on the egalitarian/CPU-friendly thesis.
|
||||
- **PoW-Design-Discussion-DoS-and-Mining.docx** — the DoS-vs-consensus framing.
|
||||
|
||||
---
|
||||
|
||||
## Source & provenance
|
||||
|
||||
The series is derived from primary sources: the reference C and Rust code, tevador's `equix` devlog, the HashWX design document, the Tor `hspow-spec` and Proposal 362, and the cited academic papers — with constants and code claims verified against the source, and 2020-era figures updated against 2026 evidence where they had aged. Compiled June 2026; revised July 2026 (all code snippets re-verified against the sources, crate-version currency noted, status lines re-checked: Proposal 362 still open, HashWX still unreleased, still no public GPU/FPGA solver).
|
||||
598
reports/EmPoWering/Equi-X/analysis/Equi-X-Vol-I-Walkthrough.md
Normal file
598
reports/EmPoWering/Equi-X/analysis/Equi-X-Vol-I-Walkthrough.md
Normal file
@ -0,0 +1,598 @@
|
||||
# Equi-X — An Implementation Walkthrough and Analysis
|
||||
|
||||
### Volume I: a unified, comparative reading of the C and Rust code, built to give you the *intuition* first
|
||||
|
||||
*Part of the Equi-X implementation series — Volume I of V (II: Deep Analysis · III: Attacking & Accelerating · IV: PoW Landscape · V: Future Directions). Companion to the Equi-X Technical Reference and Research Survey · June 2026 · revised July 2026*
|
||||
|
||||
---
|
||||
|
||||
## 0. What this document is
|
||||
|
||||
The Technical Reference in this project specifies *what* Equi-X computes. This document reads the *code that actually computes it* and tries to make the design click — why each piece exists, what it is really doing, and how the two real-world implementations express the same idea in very different styles.
|
||||
|
||||
It is **unified and comparative**: instead of two separate tours, we follow the data as it flows through the algorithm and, at each step, look at both implementations side by side. It is also **intuition-first** — code appears only where it illuminates an idea, and every snippet is preceded by the mental model it supports.
|
||||
|
||||
The two implementations:
|
||||
|
||||
| | Reference C | Rust (Arti) |
|
||||
|---|---|---|
|
||||
| Repos | [`tevador/equix`](https://github.com/tevador/equix) + [`tevador/hashx`](https://github.com/tevador/hashx) | `equix` + `hashx` crates (part of [Arti](https://gitlab.torproject.org/tpo/core/arti)) |
|
||||
| Version read here | v1.0.0 (2020) | `equix` 0.6.1 · `hashx` 0.7.1 |
|
||||
| Author / maintainer | tevador (designer) | The Tor Project |
|
||||
| Role | The normative, canonical behavior | A safe, idiomatic reimplementation, bug-for-bug compatible |
|
||||
| License | LGPL-3.0 | LGPL-3.0 |
|
||||
|
||||
A crucial fact that shapes everything below: **there is no separate written specification.** The C code *is* the specification, and the Rust crates are written to reproduce its output exactly — including quirks the Rust authors flag as probable bugs but deliberately preserve. Both ship the same published test vectors.
|
||||
|
||||
*(Version currency, July 2026: the C is unchanged — v1.0.0 is still the only tagged release — and the crates have moved on to `equix` 0.7.0 / `hashx` 0.9.0, published June 30, 2026. The newer crate versions carry no algorithmic changes; the test vectors freeze the behavior this volume describes.)*
|
||||
|
||||
---
|
||||
|
||||
## 1. The one-paragraph mental model
|
||||
|
||||
Equi-X is an **asymmetric client puzzle**: hard to solve, trivial to check. It is built from two layers that solve two different problems.
|
||||
|
||||
> **Layer 1 — HashX** turns a *challenge* into a brand-new, one-off **hash function**. Not a hash *value* — an actual function, a little randomly-generated program of integer math. This is the trick that makes the puzzle CPU-friendly: the program is shaped to saturate a real CPU pipeline, so specialized hardware can't do much better than your laptop.
|
||||
>
|
||||
> **Layer 2 — Equihash(60, 3)** poses a search puzzle *over that function*: find **eight 16-bit inputs whose hash outputs sum to zero in the low 60 bits** (modular addition, not XOR). Finding them needs a memory-hungry birthday search; checking them needs just eight hash evaluations and seven additions.
|
||||
|
||||
Everything else is detail in service of those two sentences. Hold onto this picture:
|
||||
|
||||
```text
|
||||
challenge bytes
|
||||
│
|
||||
▼
|
||||
┌─────────────────────────────────────────────┐
|
||||
│ LAYER 1: HashX │
|
||||
│ Blake2b("HashX v1") ── 512-bit seed │
|
||||
│ │ │
|
||||
│ ├── key0 ─▶ SipHash1,3 PRNG ─▶ generate a 512-instruction program
|
||||
│ │ (simulated superscalar CPU)
|
||||
│ └── key1 ─▶ register init + digest │
|
||||
│ │
|
||||
│ H(i) : i (u64) ─▶ [run program] ─▶ 64-bit digest
|
||||
└─────────────────────────────────────────────┘
|
||||
│ (H is now a fixed function of the challenge)
|
||||
▼
|
||||
┌─────────────────────────────────────────────┐
|
||||
│ LAYER 2: Equihash(60,3), SUM not XOR │
|
||||
│ find i0..i7 in [0, 2^16) with │
|
||||
│ H(i0)+...+H(i7) ≡ 0 (mod 2^60) │
|
||||
│ via Wagner's tree: │
|
||||
│ pairs cancel 15 bits ─▶ quads 30 ─▶ all-8 60
|
||||
└─────────────────────────────────────────────┘
|
||||
│
|
||||
▼
|
||||
solution = 8 × u16 = 16 bytes
|
||||
```
|
||||
|
||||
The rest of the document expands each box, then steps back to compare the two codebases and draw out what the implementation teaches about the design.
|
||||
|
||||
---
|
||||
|
||||
## 2. A map of the code
|
||||
|
||||
Both projects split cleanly along the two layers. The inner hash lives in its own library (`hashx`); the puzzle layer (`equix`) depends on it.
|
||||
|
||||
**Reference C** — small, macro-and-struct, performance-first:
|
||||
|
||||
```text
|
||||
hashx/ equix/
|
||||
include/hashx.h API include/equix.h API
|
||||
src/siphash.c PRNG core src/equix.c solve/verify entry + verifier
|
||||
src/siphash_rng.c PRNG stream src/solver.c Wagner's algorithm (stages 0–3)
|
||||
src/program.c generator src/solver_heap.h the 1.81 MiB memory layout
|
||||
src/program_exec.c interpreter src/context.c allocation
|
||||
src/compiler_x86.c JIT (x86-64)
|
||||
src/compiler_a64.c JIT (ARM64)
|
||||
src/hashx.c exec + digest
|
||||
```
|
||||
|
||||
**Rust (Arti)** — typed, generic, safety-annotated:
|
||||
|
||||
```text
|
||||
hashx/src/ equix/src/
|
||||
lib.rs HashX/Builder lib.rs EquiX/Builder + free fns
|
||||
siphash.rs PRNG core solution.rs puzzle definition + verify
|
||||
rand.rs PRNG stream solver.rs Wagner's algorithm (3 layers)
|
||||
generator.rs generator collision.rs the collision search primitive
|
||||
scheduler.rs superscalar model bucket_array/ generic sorting-bucket memory
|
||||
constraints.rs program rules
|
||||
program.rs instr set + interpreter
|
||||
compiler/ JIT (dynasmrt)
|
||||
register.rs register file + digest
|
||||
```
|
||||
|
||||
The public entry points line up almost one to one:
|
||||
|
||||
| Operation | C | Rust |
|
||||
|---|---|---|
|
||||
| Make the puzzle's hash | `hashx_make(ctx, seed, len)` | `HashX::new(seed)` |
|
||||
| Evaluate the hash | `hashx_exec(ctx, i, out)` | `hash.hash_to_u64(i)` |
|
||||
| Build an Equi-X instance | `equix_alloc` + first `equix_solve`/`verify` | `EquiX::new(challenge)` |
|
||||
| Solve | `equix_solve(ctx, ch, len, out[8])` | `equix::solve(challenge)` |
|
||||
| Verify | `equix_verify(ctx, ch, len, sol)` | `equix::verify(challenge, sol)` |
|
||||
|
||||
Notice the Rust API returns a *value* (`EquiX`, `Solution`) where C fills a caller-owned buffer and returns a status — the first of many "same algorithm, different idiom" contrasts.
|
||||
|
||||
---
|
||||
|
||||
## 3. Layer 1 — HashX, a hash function generated per challenge
|
||||
|
||||
This is the conceptually deepest part of Equi-X, so we spend the most time here. The promise of HashX: **given a seed, deterministically produce a unique one-way function** whose runtime cost is essentially fixed and whose shape resembles a tight loop of dependent integer instructions on a real CPU.
|
||||
|
||||
### 3.1 Seeding — from arbitrary bytes to two key blocks
|
||||
|
||||
Everything starts by stretching the seed (for Equi-X, the challenge string) into 512 pseudorandom bits with Blake2b, personalized with the literal string `"HashX v1"`. Those 512 bits are split into **two** 256-bit SipHash key blocks with different jobs:
|
||||
|
||||
- **key0** seeds the PRNG that *generates the program*.
|
||||
- **key1** seeds the *register file* at hash time and is folded back in during finalization.
|
||||
|
||||
The Rust says this almost declaratively:
|
||||
|
||||
```rust
|
||||
// hashx/src/siphash.rs
|
||||
pub fn pair_from_seed(seed: &[u8]) -> (SipState, SipState) {
|
||||
let mut core = Core::new_with_params(b"HashX v1", &[], 0, 64); // Blake2b, 64-byte out
|
||||
// ... hash the seed ...
|
||||
(Self::new_from_bytes(&digest[0..32]), // key0 → program generator
|
||||
Self::new_from_bytes(&digest[32..64])) // key1 → register init + digest
|
||||
}
|
||||
```
|
||||
|
||||
The C is the same computation inside `hashx_make`: `blake2b` the seed into `siphash_state keys[2]`, hand `keys[0]` to the generator and stash `keys[1]` for execution.
|
||||
|
||||
> **Intuition.** The seed never touches the math directly. It is laundered through Blake2b so that even a tiny challenge change produces a completely different program *and* different register initialization. key0 and key1 are kept separate so the program's *structure* and its *input mixing* are independent pseudorandom streams.
|
||||
|
||||
### 3.2 The PRNG — SipHash as a faucet of decision bits
|
||||
|
||||
Program generation is a long sequence of small random choices ("which opcode? which register? which rotation amount?"). HashX draws them from **SipHash1,3 in counter mode**: a fast, weak-but-sufficient stream, since this randomness only has to be unpredictable enough to resist shortcutting, not cryptographically strong.
|
||||
|
||||
```rust
|
||||
// hashx/src/rand.rs — the underlying 64-bit stream
|
||||
fn next_u64(&mut self) -> u64 {
|
||||
let value = siphash13_ctr(self.key, self.counter); // SipHash1,3(key0, counter)
|
||||
self.counter += 1;
|
||||
value
|
||||
}
|
||||
```
|
||||
|
||||
A subtlety worth its own note, because it is exactly the kind of thing a reimplementation must get bit-for-bit right: the generator consumes a mix of `u8` and `u32` values, and both are carved out of that shared `u64` stream through small queues. The Rust wraps this in an `RngBuffer` holding "up to one u32 and up to seven bytes," and the C does the equivalent in `siphash_rng.c`. The *order* in which u8s and u32s are pulled changes which bytes of each u64 go where — so the queueing rule is part of the spec, not an implementation detail. The Rust comments call this out explicitly and even ship a captured reference stream as a test.
|
||||
|
||||
### 3.3 What a "program" is
|
||||
|
||||
A HashX program is a fixed-length list of **512 instructions** over a tiny register-only virtual machine: **8 integer registers (R0–R7), no memory, no I/O.** The instruction set is deliberately small and CPU-flavored:
|
||||
|
||||
| Instruction | Meaning | Latency | Notes |
|
||||
|---|---|---|---|
|
||||
| `UMULH_R` / `SMULH_R` | high 64 bits of a 64×64 multiply (unsigned/signed) | 4 | sets the value tested by the next branch |
|
||||
| `MUL_R` | `dst *= src` (low 64 bits) | 3 | |
|
||||
| `ADD_RS` | `dst += src << s`, `s ∈ 0..3` | 1 | `dst ≠ R5` (an x86 encoding quirk) |
|
||||
| `SUB_R`, `XOR_R` | register–register | 1 | `dst ≠ src` |
|
||||
| `ADD_C`, `XOR_C` | with a sign-extended 32-bit constant | 1 | |
|
||||
| `ROR_C` | rotate right by a constant | 1 | |
|
||||
| `TARGET` / `BRANCH` | the one-shot conditional jump | 1 | see below |
|
||||
|
||||
The Rust models this as an `enum Instruction`; the C as a `struct instruction { opcode; src; dst; imm32; op_par; }`. Same eleven operations.
|
||||
|
||||
**The one-shot branch** is HashX's signature anti-GPU feature. `TARGET` marks a spot; `BRANCH` carries a 32-bit mask with exactly 4 bits set, and jumps back to the target *iff* the masked bits of the last wide-multiply result are zero — but **at most once per execution**. The interpreter shows the whole mechanism plainly:
|
||||
|
||||
```rust
|
||||
// hashx/src/program.rs (interpret)
|
||||
Instruction::Branch { mask } => {
|
||||
if allow_branch && (mask & mulh_result) == 0 {
|
||||
allow_branch = false; // fuse blows: never branch again
|
||||
branch_target
|
||||
.expect("generated programs always have a target before branch")
|
||||
} else { next_pc }
|
||||
}
|
||||
```
|
||||
|
||||
> **Intuition.** A 4-bit mask means the branch is taken with probability ≈ 1/16, and it depends on data the CPU can't know until mid-execution. That single unpredictable, input-dependent jump is cheap on a CPU's branch predictor but punishes the lock-step execution model of GPUs. Making it *one-shot* keeps every instance's worst-case runtime bounded and uniform — there is no loop to unroll.
|
||||
|
||||
### 3.4 The heart: generating a program by simulating a CPU
|
||||
|
||||
Here is the idea that makes HashX more than "a random sequence of ops." The generator builds the program **against a simulated superscalar CPU** (modeled on Intel Ivy Bridge: 3 execution ports, instruction latencies, in-order issue). It only emits an instruction if the simulated CPU could issue it *now* without stalling, and it keeps going until a fixed cycle budget is spent. A program is accepted only if it lands on exact targets.
|
||||
|
||||
The scheduler model is stated as constants — and the C and Rust agree to the number:
|
||||
|
||||
```rust
|
||||
// hashx/src/scheduler.rs
|
||||
const TARGET_CYCLES: usize = 192; // stop issuing once we reach this cycle
|
||||
const NUM_EXECUTION_PORTS: usize = 3; // P5, P0, P1 (multiply only on P1)
|
||||
// latency: ALU = 1, Mul = 3, wide-mul (UMulH/SMulH) = 4
|
||||
```
|
||||
|
||||
```c
|
||||
/* hashx/src/program.c */
|
||||
#define TARGET_CYCLE 192
|
||||
#define REQUIREMENT_SIZE 512
|
||||
#define REQUIREMENT_MUL_COUNT 192
|
||||
#define REQUIREMENT_LATENCY 195
|
||||
```
|
||||
|
||||
The generator threads three pieces of state (named the same in both languages):
|
||||
|
||||
1. **A scheduler** (`scheduler.rs` / the port map in `program.c`) — a scoreboard of which of the 3 ports are busy on which cycle, and when each register's last write retires. It answers "at the earliest cycle, on which port, could this op run, and which registers are ready by then?"
|
||||
2. **A validator / constraints** (`constraints.rs`) — the anti-shortcut rules: no two identical ops back to back, no back-to-back register add/sub, no `dst == src` for several ops, R5 may not be an `ADD_RS` destination, and others.
|
||||
3. **A selector pattern** — the opcode to attempt is chosen by position in a repeating **36-sub-cycle template**, so the *mix* of instructions (and the count of multiplies and branches) is constant across all instances.
|
||||
|
||||
The selector pattern is the clearest single piece of "why programs look alike":
|
||||
|
||||
```rust
|
||||
// hashx/src/generator.rs — choose_opcode_selector (paraphrased)
|
||||
let n = sub_cycle % 36;
|
||||
if n == 1 { Target }
|
||||
else if n == 19 { Branch }
|
||||
else if n == 12 || n == 24 { WideMul }
|
||||
else if n % 3 == 0 { Mul } // the (Mul, _, _) backbone
|
||||
else { Normal } // Add/Sub/Xor/Rotate/...
|
||||
```
|
||||
|
||||
Generation is **multi-pass and self-healing**. For each slot it tries an *original* pass; if register selection fails it tries a simplified *retry* pass (immediate-source ops only); if that also fails it **stalls** one cycle (advancing simulated time so more registers retire) and tries again. The Rust loop is the readable version of the same control flow in C's big `while` loop:
|
||||
|
||||
```rust
|
||||
// hashx/src/generator.rs
|
||||
fn generate_instruction(&mut self) -> Result<.., ()> {
|
||||
loop {
|
||||
if let Ok(r) = self.instruction_gen_attempt(Pass::Original) { return Ok(r); }
|
||||
if let Ok(r) = self.instruction_gen_attempt(Pass::Retry) { return Ok(r); }
|
||||
self.scheduler.stall()?; // ran out of time → stop
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Finally, **acceptance**. A finished program is kept only if it hits all three targets exactly:
|
||||
|
||||
```rust
|
||||
// hashx/src/constraints.rs — check_whole_program
|
||||
instructions.len() == 512
|
||||
&& scheduler.overall_latency() == 194 // last write retires here
|
||||
&& multiply_count == 192
|
||||
```
|
||||
|
||||
This is where the famous "≈195 cycles" and "192 multiplies" come from. Note the off-by-one that a careful reader will trip on: Rust checks `== 194` while C `#define`s `REQUIREMENT_LATENCY 195`. They agree — the C acceptance test is `latency == REQUIREMENT_LATENCY - 1` with the comment *"cycles are numbered from 0."* So both require the final register write to retire on the 195th cycle (index 194). About **1 seed in a few thousand** fails these checks; the caller must skip it (`Error::ProgramConstraints` / `hashx_make` returns 0).
|
||||
|
||||
> **Intuition.** The acceptance criteria are the real "proof of work shape." By forcing every accepted program to contain exactly 192 multiplications and to fill a ~195-cycle dependency chain on a realistic 3-port machine, HashX guarantees that (a) every instance costs the same, and (b) the cost is dominated by a long chain of multiplies that a CPU is *already optimized for* and that a GPU/FPGA cannot meaningfully accelerate. The program generator is, in effect, a tiny optimizing compiler run backwards: it manufactures code that is maximally friendly to a CPU and maximally boring to anything else.
|
||||
|
||||
### 3.5 Evaluating the hash
|
||||
|
||||
With a program in hand, computing `H(input)` is three steps, identical in both languages:
|
||||
|
||||
1. **Initialize 8 registers** from key1 and the input via SipHash2,4 counter mode (`siphash24_ctr(key1, input)` → `[u64; 8]`).
|
||||
2. **Run the program** (interpret or JIT — next section).
|
||||
3. **Finalize / digest** to remove a multiply-induced bias toward zero, then fold to the output width.
|
||||
|
||||
The digest is small and worth seeing because it pins down the output exactly — and because the C and Rust are byte-identical here:
|
||||
|
||||
```rust
|
||||
// hashx/src/register.rs — digest (Equi-X uses only the first returned word)
|
||||
let mut x = SipState { v0: r0+key.v0, v1: r1+key.v1, v2: r2, v3: r3 };
|
||||
let mut y = SipState { v0: r4, v1: r5, v2: r6+key.v2, v3: r7+key.v3 };
|
||||
x.sip_round(); y.sip_round();
|
||||
[x.v0 ^ y.v0, x.v1 ^ y.v1, x.v2 ^ y.v2, x.v3 ^ y.v3]
|
||||
```
|
||||
|
||||
```c
|
||||
/* hashx/src/hashx.c — same arithmetic */
|
||||
r[0]+=keys.v0; r[1]+=keys.v1; r[6]+=keys.v2; r[7]+=keys.v3;
|
||||
SIPROUND(r[0],r[1],r[2],r[3]); SIPROUND(r[4],r[5],r[6],r[7]);
|
||||
/* out: r[0]^r[4], r[1]^r[5], r[2]^r[6], r[3]^r[7] */
|
||||
```
|
||||
|
||||
Equi-X calls the inner hash with the 16-bit index as the input and keeps only the **first 64-bit word** of the digest (of which the low 60 bits matter). In Rust that is `hash_to_u64(item)`; in C it is `load64(hash)` of the first eight output bytes.
|
||||
|
||||
> **Intuition.** Initialization makes the input "spray" across all 8 registers before any program logic runs; finalization runs one SipHash round over each half and XOR-combines them so that the heavy multiplications (which statistically pull bits toward 0) don't bias the output. The result behaves like a good hash even though the middle is a random integer program.
|
||||
|
||||
### 3.6 Interpreter vs JIT — the same program, two engines
|
||||
|
||||
Both libraries can either **interpret** the instruction list or **JIT-compile** it to native code; both default to "try to compile, fall back to interpret." The compiled path is roughly an order of magnitude faster and is what real deployments use.
|
||||
|
||||
The difference is purely in *how* they emit machine code:
|
||||
|
||||
- **C** hand-emits raw opcode bytes. `compiler_x86.c` is full of `memcpy`'d byte arrays with assembly in the comments — e.g. the prologue loads R0–R7 into the native registers `r8`–`r15`:
|
||||
|
||||
```c
|
||||
static const uint8_t x86_prologue[] = {
|
||||
0x48,0x89,0xF9, /* mov rcx, rdi ; rcx = ®isters */
|
||||
...
|
||||
0x4C,0x8B,0x01, /* mov r8, [rcx+0] ; R0 */
|
||||
0x4C,0x8B,0x49,0x08, /* mov r9, [rcx+8] ; R1 */
|
||||
... }; /* one emitter each for x86-64 and ARM64 */
|
||||
```
|
||||
|
||||
- **Rust** uses the `dynasmrt` runtime-assembler crate and the `dynasm!` macro, so the backend reads like annotated assembly rather than hex:
|
||||
|
||||
```rust
|
||||
// hashx/src/compiler/x86_64.rs
|
||||
use dynasmrt::{DynasmApi, DynasmLabelApi, x64};
|
||||
dynasm!(asm ; .arch x64 ; mov rcx, rdi ; ...);
|
||||
```
|
||||
|
||||
Both map HashX's 8 virtual registers onto the same physical registers and produce a function that takes a pointer to the register file. The choice is an engineering trade — the C is dependency-free and tiny; the Rust is far more readable and memory-safe at the cost of a build-time assembler dependency. The mapping detail also explains a generation-time rule we already met: **R5 ↔ x86 `r13`**, whose addressing-mode encoding forces the "no `ADD_RS` into R5" constraint *even on ARM*, so that every backend produces the same program.
|
||||
|
||||
---
|
||||
|
||||
## 4. Layer 2 — Equihash(60, 3): the search puzzle
|
||||
|
||||
Now `H` is a fixed function. The puzzle is to find structure in its outputs. Equi-X uses Equihash with two changes the C README states up front:
|
||||
|
||||
> "It is based on Equihash(60, 3) with two major changes: (1) Blake2b is replaced with HashX; (2) **XOR is replaced with modular addition**."
|
||||
|
||||
### 4.1 The puzzle, precisely
|
||||
|
||||
A solution is **eight 16-bit indices** `i0..i7` such that, writing `H(i)` for the low 64 bits of the inner hash:
|
||||
|
||||
```text
|
||||
H(i0) + H(i1) + ... + H(i7) ≡ 0 (mod 2^60)
|
||||
```
|
||||
|
||||
plus a **partial-sum (tree) structure** proving the solution came from Wagner's algorithm, and an **ordering constraint** that makes each solution canonical (so you can't rearrange one solution into many).
|
||||
|
||||
The parameters fall out of the names `N = 60`, `K = 3`:
|
||||
|
||||
```rust
|
||||
// equix/src/solution.rs
|
||||
pub(crate) const EQUIHASH_N: usize = 60; // bits that must cancel
|
||||
pub(crate) const EQUIHASH_K: usize = 3; // tree depth → 2^K = 8 items
|
||||
pub const NUM_ITEMS: usize = 1 << EQUIHASH_K; // 8
|
||||
pub const NUM_BYTES: usize = NUM_ITEMS * 2; // 16-byte solution
|
||||
```
|
||||
|
||||
The "tree structure" is just the requirement that the cancellation happen **level by level**: each adjacent pair cancels the low 15 bits, each group of four cancels 30, and all eight cancel 60. The Rust verifier expresses this with a recursion that is the single clearest statement of the whole puzzle in either codebase:
|
||||
|
||||
```rust
|
||||
// equix/src/solution.rs — check_tree_sums
|
||||
fn check_tree_sums(func, items, n_bits) -> Result<HashValue, ()> {
|
||||
let sum = if items.len() == 2 {
|
||||
item_hash(func, items[0]).wrapping_add(item_hash(func, items[1]))
|
||||
} else {
|
||||
let (left, right) = items.split_at(items.len() / 2);
|
||||
check_tree_sums(func, left, n_bits / 2)? // 60 → 30 → 15
|
||||
.wrapping_add(check_tree_sums(func, right, n_bits / 2)?)
|
||||
};
|
||||
if (sum & ((1 << n_bits) - 1)) == 0 { Ok(sum) } else { Err(()) }
|
||||
}
|
||||
```
|
||||
|
||||
Called with `n_bits = 60` on 8 items, it recurses into 30-bit checks on each half and 15-bit checks on each pair — exactly the spec's `STAGE1/STAGE2/FULL` masks. The C verifier (`verify_internal`) is the same logic written flat, with the masks named:
|
||||
|
||||
```c
|
||||
/* equix/src/solver.h */
|
||||
#define EQUIX_STAGE1_MASK ((1ull << 15) - 1) /* pairs cancel 15 bits */
|
||||
#define EQUIX_STAGE2_MASK ((1ull << 30) - 1) /* quads cancel 30 bits */
|
||||
#define EQUIX_FULL_MASK ((1ull << 60) - 1) /* all eight cancel 60 */
|
||||
```
|
||||
|
||||
> **Why SUM instead of XOR — the decisive line.** Equihash's security assumes a *collision-resistant* inner hash, but HashX is only preimage-resistant: about 0.2% of instances have many internal collisions. Under XOR, two equal hashes cancel to zero, so one small multicollision explodes into billions of trivial "solutions." Switching the combiner to addition mod 2^60 means equal values no longer cancel, which neutralizes weak instances — and, as a bonus, modular adders with their carry chains cost custom hardware slightly more than XOR gates. This one substitution is what lets Equi-X safely build on a non-collision-resistant hash, and it is why the solver below works with carries.
|
||||
|
||||
### 4.2 Intuition for Wagner's algorithm
|
||||
|
||||
Brute force over eight indices is `(2^16)^8 = 2^128` — hopeless. Wagner's generalized-birthday algorithm trades memory for time and finds the eight-way sum in a tree of two-way steps:
|
||||
|
||||
1. Compute all `2^16` hashes (cheap — that's only 65,536 evaluations).
|
||||
2. **Find pairs** whose sum has the low 15 bits zero. There are ~65,536 hashes and ~2^15 possible low-15-bit values, so by the birthday principle you get ~2^16 such pairs.
|
||||
3. **Find pairs of pairs** whose (already-15-bit-zero) sums *also* zero the next 15 bits — now 30 bits are zero across four indices.
|
||||
4. **Find pairs of quads** that zero the final 30 bits — 60 bits zero across all eight. Those are solutions (~2 per challenge on average).
|
||||
|
||||
The engine for "find pairs that cancel the next 15 bits" is bucketing: drop every value into a bucket keyed by the bits that must cancel, then only compare values in **complementary** buckets (bucket `b` with bucket `−b`), because only those *can* sum to zero in those bits. That is the birthday meet-in-the-middle, made concrete.
|
||||
|
||||
### 4.3 The solver, walked
|
||||
|
||||
Both solvers implement exactly that tree, with **identical bucket geometry** (this is deliberate — it makes them discard the same overflow solutions and thus produce identical output):
|
||||
|
||||
```text
|
||||
INDEX_SPACE = 2^16 = 65536 coarse buckets = 256 (8 bits)
|
||||
COARSE_BUCKET_ITEMS = 336 fine buckets = 128 (7 bits)
|
||||
FINE_BUCKET_ITEMS = 12 8 + 7 = 15 bits cancelled per layer
|
||||
```
|
||||
|
||||
Each 15-bit cancellation is split into a **coarse** pass (8 bits, by `value % 256`) and a **fine** pass (7 bits, by `value % 128`) using a small scratch hash table — `8 + 7 = 15`. Stage 0 fills the first table; stages 1–3 each cancel 15 more bits:
|
||||
|
||||
```c
|
||||
/* equix/src/solver.c — stage 0: hash everything, bucket by 8 bits */
|
||||
for (u32 i = 0; i < INDEX_SPACE; ++i) {
|
||||
uint64_t value = hash_value(hash_func, i);
|
||||
u32 bucket_idx = value % NUM_COARSE_BUCKETS; /* low 8 bits */
|
||||
...
|
||||
STAGE1_DATA(bucket_idx, item_idx) = value / NUM_COARSE_BUCKETS; /* keep 52 bits */
|
||||
}
|
||||
```
|
||||
|
||||
The Rust expresses the same three layers declaratively, calling a single generic `collision::search` per layer instead of the C's three near-identical `MAKE_PAIRS` macros:
|
||||
|
||||
```rust
|
||||
// equix/src/solver.rs (structure)
|
||||
for item in u16::MIN..=u16::MAX { layer0.insert(item_hash(func, item), item); }
|
||||
collision::search(&layer0, temp, 15, |sum, loc| layer1.insert(sum, pack(loc)));
|
||||
collision::search(&layer1, temp, 30 - 15, |sum, loc| layer2.insert(sum, pack(loc)));
|
||||
collision::search(&layer2, temp, 60 - 30, |_, loc| { /* assemble 8 items */ });
|
||||
```
|
||||
|
||||
Inside `collision::search`, the complementary-bucket pairing is explicit, and it carries the same **carry-bit correction** the C uses (`value + CARRY`, where `CARRY = bucket_idx != 0`) because the coarse complement wraps mod 256:
|
||||
|
||||
```rust
|
||||
// equix/src/collision.rs
|
||||
for first_bucket in 0..=(NUM_BUCKETS / 2) {
|
||||
let second_bucket = first_bucket.wrapping_neg() % NUM_BUCKETS; // −b
|
||||
// index the first bucket by its key remainder, then for each item in the
|
||||
// complementary bucket, look up matches and keep those whose full sum
|
||||
// has the required low bits zero:
|
||||
if sum.low_bits_are_zero(num_bits) { predicate(sum >> num_bits, location); }
|
||||
}
|
||||
```
|
||||
|
||||
> **Intuition for the carry.** When you pair bucket `b` with bucket `256 − b`, their coarse parts sum to 256, i.e. they produce a carry *out* of the 8-bit coarse field into the bits above. The `+ CARRY` term re-injects that carry before the fine pass so the arithmetic stays exact. It is a tiny line that is easy to get wrong and is precisely the sort of thing the Rust had to copy faithfully.
|
||||
|
||||
### 4.4 Remembering *where* solutions came from
|
||||
|
||||
As the tree is built, each combined node must remember its two parents so the eight original indices can be reconstructed at the end. Both implementations pack `(parent bucket, left item index, right item index)` into one integer with the **same bit widths** — 8 bits of bucket, 9 + 9 bits of item index:
|
||||
|
||||
```c
|
||||
/* equix/src/solver.c */
|
||||
#define MAKE_ITEM(bucket, left, right) ((left) << 17 | (right) << 8 | (bucket))
|
||||
```
|
||||
|
||||
```rust
|
||||
// equix/src/solver.rs
|
||||
type Layer0Collision = PackedCollision<u32, 8, 9>; // BUCKET_BITS = 8, ITEM_BITS = 9
|
||||
```
|
||||
|
||||
At the end, the solver walks this little tree back down to the leaves to collect the eight `SolutionItem`s, then puts them in canonical order. The C does it with explicit swaps guided by the `tree_cmp1/2/4` comparators; the Rust collects then calls `Solution::sort_from_array`. Both enforce the same rule: compare index groups as little-endian byte strings and keep the smaller branch on the left.
|
||||
|
||||
```c
|
||||
/* equix/src/solver.c — build_solution (top level) */
|
||||
if (!tree_cmp4(&solution->idx[0], &solution->idx[4])) { /* 8-byte LE compare */
|
||||
SWAP_IDX(solution->idx[0], solution->idx[4]); ... } /* swap whole quads */
|
||||
```
|
||||
|
||||
```rust
|
||||
// equix/src/solution.rs — same ordering, as a predicate
|
||||
fn branches_are_sorted(left, right) -> bool {
|
||||
matches!(left.iter().rev().cmp(right.iter().rev()), Less | Equal)
|
||||
}
|
||||
```
|
||||
|
||||
This ordering is what makes a solution **canonical**: without it, the same eight indices could be emitted in many permutations, each passing the sum test — so the order rule is a genuine part of validity, checked first during verification.
|
||||
|
||||
### 4.5 The memory — engineered to live in cache
|
||||
|
||||
The whole point of the bucketed search is to fit in CPU cache, where a CPU's memory bandwidth is competitive with a GPU's. Both implementations therefore pack everything into one preallocated block of **~1.81 MiB** and even **reuse** the same bytes for different purposes across stages via a union.
|
||||
|
||||
The C declares the layout as a single struct, with a literal `union` overlapping the stage-1 data (no longer needed late) with the stage-3 tables:
|
||||
|
||||
```c
|
||||
/* equix/src/solver_heap.h */
|
||||
typedef struct solver_heap {
|
||||
stage1_idx_hashtab stage1_indices; /* 172 544 bytes */
|
||||
stage2_idx_hashtab stage2_indices; /* 344 576 */
|
||||
stage2_data_hashtab stage2_data; /* 688 128 */
|
||||
union { /* reuse the same memory: */
|
||||
stage1_data_hashtab stage1_data; /* early stages need this ... */
|
||||
struct { stage3_idx_hashtab stage3_indices;
|
||||
stage3_data_hashtab stage3_data; }; /* ... late stages this */
|
||||
};
|
||||
fine_hashtab scratch_ht; /* 3 200 */
|
||||
} solver_heap; /* TOTAL: 1 897 088 bytes */
|
||||
```
|
||||
|
||||
The Rust does the *same optimization* with a checked `union Overlay { first: OverlayFirst, second: OverlaySecond }`, but wraps it in a `MaybeUninit`-based abstraction (`bucket_array`) whose module doc explains the reasoning — a single static layout is "between 2% and 10%" faster than separate allocations, but too large for the stack, so it is built in uninitialized heap memory with access policed by the borrow checker. Its self-test even pins the size:
|
||||
|
||||
```rust
|
||||
// equix/src/solver.rs
|
||||
assert_eq!(SolverMemory::SIZE, 1_895_424); // ~1.81 MiB (counters live outside)
|
||||
```
|
||||
|
||||
> The ~1,664-byte difference from the C total is purely because the Rust keeps its per-bucket counters in separate storage. The *engineering intent* is identical: one cache-resident block, reused in place.
|
||||
|
||||
### 4.6 Verification — the cheap half
|
||||
|
||||
Verification is where "asymmetric" pays off. It needs **no search and no memory** — eight hash evaluations and a handful of additions — and both implementations check from cheapest to most expensive so a flood of junk proofs is rejected almost for free:
|
||||
|
||||
1. **Order** — pure integer comparisons on the raw indices, *before any hashing*. Fail → `EQUIX_ORDER` / `Error::Order`.
|
||||
2. **Challenge** — build the HashX instance; if program generation fails its constraints (the rare ~1-in-several-thousand seed) → `EQUIX_CHALLENGE`.
|
||||
3. **Sums** — recompute the eight hashes, check the pair/quad partial sums (`EQUIX_PARTIAL_SUM`) and the final 60-bit sum (`EQUIX_FINAL_SUM`). All pass → `EQUIX_OK`.
|
||||
|
||||
```c
|
||||
/* equix/src/equix.c — order first, then hash, then sums */
|
||||
equix_result equix_verify(ctx, challenge, len, solution) {
|
||||
if (!verify_order(solution)) return EQUIX_ORDER;
|
||||
if (!hashx_make(ctx->hash_func, challenge, len)) return EQUIX_CHALLENGE;
|
||||
return verify_internal(ctx->hash_func, solution); /* staged sums */
|
||||
}
|
||||
```
|
||||
|
||||
The Rust reaches the same order through its type system: you cannot hold a `Solution` without having passed the order check (it is enforced in `try_from_bytes`/`try_from_array`), so `EquiX::verify` only has to "check hash tree sums," exactly as its doc comment says.
|
||||
|
||||
> **Intuition.** A verifier does ~50 µs of work to check what cost a solver ~5 ms to find — and it does the free check (ordering) before spending anything on hashing. That asymmetry, plus the tiny 16-byte proof, is the entire reason Equi-X is usable as a DoS defense: an attacker can't make verification itself expensive.
|
||||
|
||||
---
|
||||
|
||||
## 5. The two implementations, compared
|
||||
|
||||
Same algorithm, same outputs, different philosophies. The contrasts are instructive in their own right.
|
||||
|
||||
| Dimension | Reference C | Rust (Arti) |
|
||||
|---|---|---|
|
||||
| Primary goal | Be the canonical definition; be fast and small | Be safe, auditable, and *exactly* compatible |
|
||||
| Program/solver code | Macros (`MAKE_PAIRS1/2/3`), structs, manual indexing | Generics + traits (`BucketArray`, `collision::search`, `PackedCollision`) |
|
||||
| JIT | Hand-emitted opcode bytes (`compiler_x86.c`, `compiler_a64.c`) | `dynasmrt` runtime assembler with `dynasm!` |
|
||||
| Memory model | One `struct` + `union`, allocated once | `MaybeUninit` block + checked `union`, borrow-policed |
|
||||
| Errors | Return codes / sentinels (`HASHX_NOTSUPP`, `equix_result`) | `Result<…, Error>` enums |
|
||||
| API shape | Caller-owned buffers, opaque `ctx` | Owned values, builders, free functions |
|
||||
| Randomness handling | Implicit in code order | Documented as spec, with captured reference streams as tests |
|
||||
|
||||
The most telling detail is **deliberate bug-for-bug compatibility.** During wide-multiply generation, HashX draws an extra 32-bit value from the PRNG and uses it only as a tie-breaker for a writer-collision check. The Rust authors believe this is vestigial — but they keep it, with a comment that doubles as design commentary:
|
||||
|
||||
```rust
|
||||
// hashx/src/constraints.rs
|
||||
// "As far as I can tell this is a bug in the original implementation but we
|
||||
// can't change the behavior without breaking compatibility. ... It seems like
|
||||
// this was a vestigial feature ... but I can't be sure."
|
||||
```
|
||||
|
||||
That single comment captures the relationship between the two codebases: **the C defines truth; the Rust reproduces it, warts and all, but explains itself.** For anyone trying to understand *why* a step exists, reading the two together is far more illuminating than either alone — the Rust narrates the intent, the C settles the ground truth.
|
||||
|
||||
---
|
||||
|
||||
## 6. End-to-end: following one solve and one verify
|
||||
|
||||
Pulling the layers together, here is the whole journey with the intuition attached.
|
||||
|
||||
**Solving** `solve(challenge)`:
|
||||
|
||||
1. `EquiX::new(challenge)` / `equix_solve` → `HashX::new` builds the per-challenge function: Blake2b the challenge, generate a 512-instruction program against the simulated CPU, JIT it. (≈0.05 ms; ~1/few-thousand challenges are rejected here.)
|
||||
2. Stage 0: evaluate `H` on all 65,536 indices into 256 coarse buckets. (This is the bulk of the ~5 ms; it's just a lot of hashing.)
|
||||
3. Stages 1–3: three rounds of complementary-bucket pairing, each cancelling 15 more bits (coarse 8 + fine 7), remembering parent pointers.
|
||||
4. For each all-60-bits-zero hit, walk the parent tree to recover 8 indices, sort them canonically, dedup, and emit. (~2 solutions per challenge on average; at most 8 returned.)
|
||||
|
||||
**Verifying** `verify(challenge, solution)`:
|
||||
|
||||
1. Check the six ordering comparisons on the raw 16 bytes — no hashing yet.
|
||||
2. Rebuild `H` from the challenge (fails closed on a bad seed).
|
||||
3. Evaluate `H` on the eight indices; check pair sums (15 bits), quad sums (30 bits), and the full sum (60 bits), cheapest first.
|
||||
|
||||
The asymmetry between these two lists — a memory-bound search over 65,536 hashes versus eight hash evaluations and a few adds — *is* Equi-X.
|
||||
|
||||
---
|
||||
|
||||
## 7. What the implementation teaches
|
||||
|
||||
Reading the code (rather than the spec) surfaces a few things worth stating plainly:
|
||||
|
||||
- **The cost is deliberately front-loaded into hashing, not searching.** Stage 0's 65,536 evaluations dominate a solve; the bucketed tree is comparatively cheap. So the puzzle's hardness rests almost entirely on HashX being genuinely CPU-shaped, which is why so much engineering goes into the *generator*, not the search.
|
||||
- **Determinism is a feature with teeth.** Interpreter and JIT must agree bit-for-bit; C and Rust must agree bit-for-bit; even a probable bug is preserved. The PRNG draw order, the carry bit, the 8/9/9 packing, the `194`-vs-`195` cycle convention — these are not incidental, and the Rust's habit of capturing reference streams as tests is how that discipline is enforced.
|
||||
- **Memory is treated as the real adversary.** Both keep the solver inside ~1.81 MiB and overlap buffers via a union so the working set stays cache-resident — the precondition for a CPU to compete with a GPU on bandwidth. The Rust shows you can keep that micro-optimization *and* memory safety with `MaybeUninit` and borrow-checked unions.
|
||||
- **Verification is structurally cheap, by construction.** Ordering is checked before any hash runs; sums are checked cheapest-first. The 16-byte proof and ~50 µs check are what make Equi-X viable as an anti-DoS gate rather than a curiosity.
|
||||
- **One honest caveat lives in the code.** HashX's branch makes control flow input-dependent, so secret values must never be hashed with it (timing would leak them). For a *public* client puzzle this is fine — and the source treats it as an explicit, accepted trade rather than an accident.
|
||||
|
||||
If you want to go deeper, the highest-leverage files to read in full are, in order: `hashx/src/generator.rs` (the idea), `hashx/src/scheduler.rs` (the CPU model), `equix/src/solution.rs` (the puzzle), and `equix/src/solver.c` (the search, most compactly stated). Read each Rust file with its C counterpart open beside it.
|
||||
|
||||
---
|
||||
|
||||
## Appendix A — Key constants (both implementations agree)
|
||||
|
||||
| Constant | Value | Where |
|
||||
|---|---|---|
|
||||
| HashX program size | 512 instructions | `NUM_INSTRUCTIONS` / `REQUIREMENT_SIZE` |
|
||||
| Required multiplies | 192 | `REQUIRED_MULTIPLIES` / `REQUIREMENT_MUL_COUNT` |
|
||||
| Scheduler target | 192 cycles | `TARGET_CYCLES` / `TARGET_CYCLE` |
|
||||
| Required final latency | retire at cycle 194 (= "195th", 0-indexed) | `REQUIRED_OVERALL_RESULT_AT_CYCLE` / `REQUIREMENT_LATENCY - 1` |
|
||||
| Registers | 8 (R5 special-cased) | `NUM_REGISTERS` / `x86_reg_map` |
|
||||
| Execution ports | 3 (P5, P0, P1; mul on P1) | `scheduler.rs` / `program.c` |
|
||||
| Branch probability | ≈ 1/16 (mask weight 4) | `BRANCH_MASK_BIT_WEIGHT` / `LOG2_BRANCH_PROB` |
|
||||
| Equihash N, K | 60, 3 | `EQUIHASH_N/K` |
|
||||
| Items per solution | 8 → 16 bytes | `NUM_ITEMS` / `EQUIX_NUM_IDX` |
|
||||
| Index space | 2^16 = 65536 | `INDEX_SPACE` |
|
||||
| Coarse / fine buckets | 256 / 128 (8 + 7 = 15 bits) | `NUM_COARSE/FINE_BUCKETS` |
|
||||
| Bucket capacities | 336 coarse / 12 fine | `COARSE/FINE_BUCKET_ITEMS` |
|
||||
| Stage masks | 2^15−1, 2^30−1, 2^60−1 | `EQUIX_STAGE1/STAGE2/FULL_MASK` |
|
||||
| Solver memory | ≈ 1.81 MiB (C 1,897,088 B; Rust 1,895,424 B) | `solver_heap` / `SolverMemory::SIZE` |
|
||||
| Max solutions returned | 8 | `EQUIX_MAX_SOLS` |
|
||||
|
||||
## Appendix B — Provenance
|
||||
|
||||
- **C:** `tevador/equix` and `tevador/hashx`, v1.0.0 (2020), LGPL-3.0. The normative reference; no separate written spec exists.
|
||||
- **Rust:** `equix` 0.6.1 and `hashx` 0.7.1 (crates.io), part of the Tor Project's Arti, LGPL-3.0. Written to reproduce the C output exactly; ships the shared Tor test vectors (`tests/tor_equix_vectors.rs`, `hashx_vectors.rs`). Later crate versions (`equix` 0.7.0 / `hashx` 0.9.0, June 2026) change packaging, not behavior.
|
||||
- Snippets above are lightly trimmed for readability (elided lines marked `...`); identifiers, constants, and control flow are verbatim from the sources named in each caption. Re-verified against the sources, July 2026.
|
||||
|
||||
@ -0,0 +1,504 @@
|
||||
# Equi-X — Deep Technical Comparison and Code Analysis
|
||||
|
||||
### Volume II: a rigorous C ↔ Rust correspondence, exact algorithmic mechanics, and design analysis
|
||||
|
||||
*Part of the Equi-X implementation series — Volume II of V (I: Walkthrough · III: Attacking & Accelerating · IV: PoW Landscape · V: Future Directions) · June 2026 · revised July 2026*
|
||||
|
||||
---
|
||||
|
||||
## 0. Scope
|
||||
|
||||
Volume I built the intuition: HashX manufactures a per-challenge hash function shaped like CPU-friendly code, and Equihash(60, 3) searches for eight inputs whose hashes sum to zero mod 2⁶⁰. This volume assumes that picture and goes underneath it. It is deliberately denser and more code-level, organized around four questions:
|
||||
|
||||
1. **Correspondence** — exactly how the C reference and the Rust crates map onto each other, structure by structure.
|
||||
2. **Mechanics** — the parts Volume I summarized: the PRNG's compatibility contract, the superscalar scheduler, register-selection constraints, the JIT, the bucket solver's arithmetic.
|
||||
3. **Analysis** — complexity and the birthday accounting; where time and memory actually go; why the solver is heuristic but the verifier is exact.
|
||||
4. **Discipline** — determinism, bug-for-bug compatibility, and the security-relevant consequences of specific code choices.
|
||||
|
||||
Sources are the same four trees as Volume I: C `tevador/equix` + `tevador/hashx` (v1.0.0); Rust `equix` 0.6.1 + `hashx` 0.7.1 (later crate versions — 0.7.0/0.9.0, June 2026 — change packaging, not behavior). Snippets are trimmed (elisions marked `...`) but identifiers, constants, and arithmetic are verbatim; re-verified against the sources, July 2026.
|
||||
|
||||
---
|
||||
|
||||
## 1. Architectural correspondence
|
||||
|
||||
The two codebases are the *same algorithm* under two engineering philosophies: the C is a compact, macro-and-struct performance reference that **defines** the behavior; the Rust is a generic, trait-driven, memory-safe reimplementation that **reproduces** it. The single most useful artifact for reading them together is a map.
|
||||
|
||||
### 1.1 HashX (inner hash)
|
||||
|
||||
| Concept | C (`hashx/src`) | Rust (`hashx/src`) |
|
||||
|---|---|---|
|
||||
| Seed → key material | `hashx_make` → Blake2b(`HashX v1`) → `siphash_state keys[2]` | `SipState::pair_from_seed` → `(key0, key1)` |
|
||||
| Decision PRNG | `siphash_rng` (`siphash_rng.c`) over SipHash1,3 | `SipRand` + `RngBuffer` (`rand.rs`) over SipHash1,3 |
|
||||
| Program generator | `hashx_program_generate` (`program.c`) | `Generator::generate_program` (`generator.rs`) |
|
||||
| Superscalar model | `generator_ctx.ports[]` + `schedule_instr` | `Scheduler` / `ExecSchedule` (`scheduler.rs`) |
|
||||
| Constraints | inlined in `select_destination`/`select_template` | `Validator` (`constraints.rs`) |
|
||||
| Instruction repr | `struct instruction {opcode,src,dst,imm32,op_par}` | `enum Instruction { ... }` (`program.rs`) |
|
||||
| Interpreter | `hashx_program_execute` (`program_exec.c`) | `Program::interpret` (`program.rs`) |
|
||||
| JIT | `compiler_x86.c`, `compiler_a64.c` (raw bytes) | `compiler/x86_64.rs`, `aarch64.rs` (`dynasmrt`) |
|
||||
| Register file + digest | `r[8]` + finalize in `hashx_exec` | `RegisterFile` + `digest` (`register.rs`) |
|
||||
| Failure signal | `hashx_make` returns `0` | `Error::ProgramConstraints` |
|
||||
|
||||
### 1.2 Equi-X (puzzle)
|
||||
|
||||
| Concept | C (`equix/src`) | Rust (`equix/src`) |
|
||||
|---|---|---|
|
||||
| Public puzzle ops | `equix_solve` / `equix_verify` (`equix.c`) | `EquiX::solve` / `verify`, free fns (`lib.rs`) |
|
||||
| Solution definition + order/sum checks | `verify_order`, `verify_internal` (`equix.c`), `tree_cmp*` (`solver.h`) | `Solution`, `check_tree_order`, `check_tree_sums` (`solution.rs`) |
|
||||
| Wagner solver | `solve_stage0..3` (`solver.c`) | `find_solutions` + `collision::search` (`solver.rs`, `collision.rs`) |
|
||||
| Sorting memory | `struct solver_heap` + `union` (`solver_heap.h`) | `BucketArrayMemory` + `Overlay` union (`bucket_array/`) |
|
||||
| Parent-pointer pack | `MAKE_ITEM(b,l,r)` | `PackedCollision<u32, 8, 9>` |
|
||||
| Error taxonomy | `enum equix_result` | `enum Error` (`err.rs`) |
|
||||
|
||||
Read any Rust file with its C row open beside it and the design becomes legible from both directions: the Rust narrates *intent* in doc-comments, the C settles *ground truth* in bytes.
|
||||
|
||||
---
|
||||
|
||||
## 2. The HashX generator, exactly
|
||||
|
||||
The generator is the part most worth reading at the level of individual decisions, because *every* decision is part of the spec — a compatible third implementation must reproduce all of them bit for bit.
|
||||
|
||||
### 2.1 The PRNG and its compatibility contract
|
||||
|
||||
HashX draws generation decisions from SipHash1,3 in counter mode, but the *packaging* of that stream into `u8` and `u32` values is itself normative. Two rules matter:
|
||||
|
||||
**Rule 1 — big-endian extraction.** Each 64-bit SipHash output is sliced from the most-significant end. The C shifts down by a decreasing count:
|
||||
|
||||
```c
|
||||
/* hashx/src/siphash_rng.c */
|
||||
uint8_t hashx_siphash_rng_u8(siphash_rng* gen) {
|
||||
if (gen->count8 == 0) {
|
||||
gen->buffer8 = hashx_siphash13_ctr(gen->counter, &gen->keys);
|
||||
gen->counter++;
|
||||
gen->count8 = sizeof(gen->buffer8); /* = 8 */
|
||||
}
|
||||
gen->count8--;
|
||||
return gen->buffer8 >> (gen->count8 * 8); /* high byte first */
|
||||
}
|
||||
```
|
||||
|
||||
The Rust reaches the same order by popping bytes off the end of a little-endian array:
|
||||
|
||||
```rust
|
||||
// hashx/src/rand.rs
|
||||
let bytes = self.inner.next_u64().to_le_bytes();
|
||||
let (last, saved) = bytes.split_last()...; // returns MSB first, saves the rest
|
||||
```
|
||||
|
||||
**Rule 2 — separate `u8` and `u32` buffers over one shared counter.** `u8` and `u32` each have their own refill buffer, but both pull from the *same* incrementing counter. So whether a given 64-bit draw becomes bytes or words depends on the **order** in which the generator asks for them — interleaving is part of the contract. The Rust spells this out:
|
||||
|
||||
> *"It's important for the u8 and u32 queues to share a common generator. The order of dequeueing u8 items vs u32 items intentionally modifies the assignment of particular u64 RngCore values to the two queues."* — `rand.rs`
|
||||
|
||||
Both projects guard this with **captured reference streams** as unit tests (`rng_vectors` in `rand.rs`; the Tor vectors in C). The practical upshot: you cannot refactor the order of `next_u8`/`next_u32` calls without breaking every downstream hash.
|
||||
|
||||
### 2.2 The 36-slot instruction template
|
||||
|
||||
The opcode attempted at each step is fixed by position in a repeating 36-sub-cycle layout, which is why every accepted program has the same instruction *mix* (notably exactly 192 multiplies and one branch site). The C stores it as an array; the Rust as a match. They are identical:
|
||||
|
||||
| sub-cycle mod 36 | C `program_layout[]` | Rust `choose_opcode_selector` | Opcode(s) |
|
||||
|---|---|---|---|
|
||||
| 1 | `item_target` | `n == 1` | `TARGET` |
|
||||
| 19 | `item_branch` | `n == 19` | `BRANCH` |
|
||||
| 12, 24 | `item_wide_mul` | `n == 12 \|\| 24` | `SMULH`/`UMULH` |
|
||||
| 0,3,6,9,15,18,21,27,30,33 | `item_mul` | `n % 3 == 0` | `MUL` |
|
||||
| all others | `item_any` | `Normal` / `ImmediateSrc` | ALU op from a table |
|
||||
|
||||
The "any" slot draws from an 8-entry table on the first pass and a 4-entry immediate-source-only table on retry:
|
||||
|
||||
```c
|
||||
/* C: item_any.mask0 = 7 (8 ops), mask1 = 3 (4 ops, no src register) */
|
||||
const static instr_template* instr_lookup[] =
|
||||
{ &tpl_ror_c,&tpl_xor_c,&tpl_add_c,&tpl_add_c,&tpl_sub_r,&tpl_xor_r,&tpl_xor_c,&tpl_add_rs };
|
||||
```
|
||||
|
||||
```rust
|
||||
// Rust: identical tables
|
||||
const NORMAL_OPS_TABLE: [Opcode;8] = [Rotate,XorConst,AddConst,AddConst,Sub,Xor,XorConst,AddShift];
|
||||
const IMMEDIATE_SRC_OPS_TABLE:[Opcode;4] = [Rotate,XorConst,AddConst,AddConst];
|
||||
```
|
||||
|
||||
A subtle shared rule: the `item_any` slot forbids repeating the *previous* op's **group** (`duplicates = false` in C; `disallow_opcode_pair` in Rust), and `Sub` and `AddShift` share one group, so they can't be adjacent either. This is checked at the selector level and costs only a re-roll, not a failed instruction.
|
||||
|
||||
### 2.3 The superscalar scheduler
|
||||
|
||||
The generator places each instruction on a simulated Ivy-Bridge-like core: **3 ports** (the C calls them P0/P1/P5; multiply lives only on P1), per-opcode **latencies** (ALU 1, `MUL` 3, wide-mul 4), and a per-register **retire-cycle scoreboard**. Two mechanisms deserve attention.
|
||||
|
||||
**Port allocation order is P5 → P0 → P1**, deliberately checking the multiply port *last* so general ALU ops don't starve multiplications:
|
||||
|
||||
```c
|
||||
/* hashx/src/program.c — schedule_uop */
|
||||
if ((uop & PORT_P5) && !ctx->ports[cycle][2]) { ...; return cycle; }
|
||||
if ((uop & PORT_P0) && !ctx->ports[cycle][0]) { ...; return cycle; }
|
||||
if ((uop & PORT_P1) && !ctx->ports[cycle][1]) { ...; return cycle; }
|
||||
```
|
||||
|
||||
The Rust encodes the same priority in its bit numbering and iteration:
|
||||
|
||||
```rust
|
||||
// scheduler.rs — P5 = 1<<0, P0 = 1<<1, P1 = 1<<2; iterate indices 0,1,2
|
||||
for index in 0..NUM_EXECUTION_PORTS { if (ports.0 & (1<<index)) != 0 && !busy { return ... } }
|
||||
```
|
||||
|
||||
**Two-µop instructions** (the wide multiplies, `TARGET`, `BRANCH`) are scheduled *conservatively*: the generator searches forward for the first cycle on which **both** µops can issue simultaneously. The C tries a non-committing probe of both, then commits only on a match:
|
||||
|
||||
```c
|
||||
/* schedule_instr, 2-uop branch */
|
||||
for (int cycle = ctx->cycle; cycle < PORT_MAP_SIZE; ++cycle) {
|
||||
int c1 = schedule_uop(tpl->uop1, ctx, cycle, false); // probe
|
||||
int c2 = schedule_uop(tpl->uop2, ctx, cycle, false);
|
||||
if (c1 >= 0 && c1 == c2) { /* commit both */ return c1; }
|
||||
}
|
||||
```
|
||||
|
||||
The Rust `instruction_plan` does precisely this two-port "same cycle" search and returns an `InstructionPlan` carrying the issue cycle and the chosen ports. The scoreboard then records the destination register's **retire cycle** (`issue + latency`) so later instructions only pick operands that are ready.
|
||||
|
||||
### 2.4 Register selection and the constraint stack
|
||||
|
||||
Once an opcode and a timing plan exist, the generator must pick source/destination registers that (a) are ready by the issue cycle and (b) satisfy the anti-shortcut rules. The C builds the candidate set with a **branchless filter** that ANDs five conditions and accumulates without a branch:
|
||||
|
||||
```c
|
||||
/* program.c — select_destination (conditions condensed) */
|
||||
for (int i = 0; i < 8; ++i) {
|
||||
bool available = ctx->registers[i].latency <= cycle; // ready in time
|
||||
available &= ((!tpl->distinct_dst) | (i != instr->src)); // dst != src (some ops)
|
||||
available &= (ctx->chain_mul | (tpl->group != INSTR_MUL_R)
|
||||
| (ctx->registers[i].last_op != INSTR_MUL_R));// no back-to-back MUL
|
||||
available &= ((ctx->registers[i].last_op != tpl->group)
|
||||
| (ctx->registers[i].last_op_par != instr->op_par)); // no trivial repeat
|
||||
available &= ((instr->opcode != INSTR_ADD_RS) | (i != 5)); // R5 not ADD_RS dst
|
||||
available_regs[regs_count] = available ? i : 0;
|
||||
regs_count += available;
|
||||
}
|
||||
```
|
||||
|
||||
The Rust expresses the identical five conditions through `RegisterSet::from_filter` plus a `Validator::dst_registers_allowed` checker — same logic, type-checked instead of bit-ANDed. The conditions exist to forbid *optimizable* sequences (`xor r,r`; `ror r,c1; ror r,c2`; runaway multiplication that floods a register with trailing zeros) so that an attacker can't simplify the function.
|
||||
|
||||
Two details that a reimplementation must copy and that the Rust flags explicitly:
|
||||
|
||||
- **The R5 / `ADD_RS` source short-circuit.** When only two registers are free for an `ADD_RS` and one is R5 (which can't be the *destination*), HashX deterministically forces R5 to be the *source* rather than rolling the dice — preventing a frequent dead-end. Present in both (`select_source` in C; `src_registers_allowed` in Rust).
|
||||
- **The wide-multiply "vestigial" RNG draw.** `UMULH`/`SMULH` are `op_par_src = false, distinct_dst = false`, so the C falls into `op_par = hashx_siphash_rng_u32(gen)` — it consumes a `u32` whose *only* use is the "no trivial repeat" check above. The Rust keeps it as `RegisterWriter::UMulH(u32)` with a comment calling it a probable bug it cannot remove without breaking compatibility. It costs one PRNG draw per wide multiply and changes the stream alignment for everything after it.
|
||||
|
||||
### 2.5 Multi-pass generation, acceptance, and the 194/195 question
|
||||
|
||||
Each slot is attempted up to twice before time advances. The C uses an `attempt` counter and `MAX_RETRIES = 1`; on the retry it sets `chain_mul = true` (allowing back-to-back multiplies on a register if nothing else fits — the "prevents catastrophic failure" path). If both attempts fail it advances a full cycle (`sub_cycle += 3`) and resets. The Rust mirrors this as `Pass::Original → Pass::Retry → scheduler.stall()`.
|
||||
|
||||
Generation stops when an instruction would schedule at or beyond cycle 192 (`TARGET_CYCLE`), or the 512-slot buffer fills. A finished program is then **accepted only if it hits three exact targets**:
|
||||
|
||||
```c
|
||||
/* program.c — the only thing that makes a program "real" */
|
||||
return (program->code_size == 512)
|
||||
& (ctx.mul_count == 192)
|
||||
& (ctx.latency == REQUIREMENT_LATENCY - 1); /* 195 - 1; "cycles numbered from 0" */
|
||||
```
|
||||
|
||||
```rust
|
||||
// constraints.rs — same three gates
|
||||
instructions.len() == 512
|
||||
&& scheduler.overall_latency().as_usize() == 194
|
||||
&& multiply_count == 192
|
||||
```
|
||||
|
||||
This resolves a discrepancy a careful reader will hit: the C `#define REQUIREMENT_LATENCY 195` but tests `latency == 194`; the Rust hard-codes `194`. They agree — the final register write must retire on the cycle indexed 194 (the "195th"). The C source notes ~**1 seed in 10,000** fails these gates; the Rust says "once per several thousand." Callers must skip such seeds — this is the `EQUIX_CHALLENGE` / `Error::ProgramConstraints` path, distinct from the ~0.2% of seeds that *generate fine* but are collision-weak (handled by Layer 2's use of addition).
|
||||
|
||||
> **What the generator guarantees.** Every accepted instance is a 512-instruction, 192-multiply program whose dependency chain exactly fills a ~195-cycle window on a realistic 3-port core. That uniformity is the proof-of-work "shape": equal cost across instances, dominated by a long multiply chain a CPU executes natively and a GPU/FPGA cannot meaningfully shorten. Generation is itself cheap — O(512) instructions, each a bounded forward search over the ~196-entry port map.
|
||||
|
||||
---
|
||||
|
||||
## 3. HashX execution: interpreter and JIT
|
||||
|
||||
### 3.1 The exact hash pipeline
|
||||
|
||||
`H(input)` is fully determined by: initialize 8 registers from `key1` and the input via SipHash2,4 counter mode; run the program; finalize. The arithmetic is unsigned 64-bit **wrapping** throughout, constants are **sign-extended** from 32 bits, rotates are by a constant, and the wide multiplies keep the **upper** 64 bits of a 128-bit product. The finalization (identical in both; verified byte-for-byte) folds `key1` back in to cancel the multiply-induced bias toward zero:
|
||||
|
||||
```c
|
||||
/* hashx.c */
|
||||
r[0]+=keys.v0; r[1]+=keys.v1; r[6]+=keys.v2; r[7]+=keys.v3;
|
||||
SIPROUND(r[0],r[1],r[2],r[3]); SIPROUND(r[4],r[5],r[6],r[7]);
|
||||
/* out = r0^r4, r1^r5, r2^r6, r3^r7 ; Equi-X keeps only out[0..8) */
|
||||
```
|
||||
|
||||
### 3.2 The one-shot branch in three forms
|
||||
|
||||
The branch is the same idea expressed three ways, and seeing all three removes any ambiguity about its semantics.
|
||||
|
||||
**Interpreter** (both languages identical in effect): a `branch_enable` fuse, blown the first time any branch is taken; the test is against the low 32 bits of the most recent wide-multiply result:
|
||||
|
||||
```c
|
||||
/* program_exec.c */
|
||||
case INSTR_BRANCH:
|
||||
if (branch_enable && (result & instr->imm32) == 0) { i = target; branch_enable = false; }
|
||||
break;
|
||||
```
|
||||
|
||||
One micro-difference worth noting for bit-pedants: on a taken branch the **C interpreter resumes at `target + 1`** (the `for` loop's `++i` after `i = target`), whereas the **Rust interpreter re-executes the `Target` instruction** (a no-op that just re-marks the target). Identical results, since `Target` has no effect once the fuse is blown.
|
||||
|
||||
**Compiled** (C x86): the one-shot is implemented without a second test by exploiting the flags that caused the jump. The `Target` site is `test edi,edi; «label»: cmovz esi,edi`, and `Branch` is `or edx,esi; test edx,imm; jz label`. When `jz` is taken, the zero flag is still set on arrival at the label, so `cmovz` moves `edi`(= −1) into `esi`; thereafter `or edx,esi` forces the `test` non-zero and the branch can never be taken again. A genuinely elegant branchless disable, and exactly the sort of thing the interpreter exists to specify unambiguously.
|
||||
|
||||
### 3.3 JIT: raw bytes vs `dynasmrt`
|
||||
|
||||
Both compile the program to native code that takes a pointer to the register file; the difference is how bytes are produced.
|
||||
|
||||
The **C** emits pre-encoded machine code, one `EMIT` per instruction, mapping HashX R0–R7 to native `r8`–`r15`:
|
||||
|
||||
```c
|
||||
/* compiler_x86.c — e.g. MUL_R is a single 4-byte imul */
|
||||
case INSTR_MUL_R:
|
||||
EMIT_U32(pos, 0xc0af0f4d | (instr->dst << 27) | (instr->src << 24)); /* imul dst,src */
|
||||
```
|
||||
|
||||
It brackets emission with a **W^X** transition — `hashx_vm_rw(code,…)` before writing, `hashx_vm_rx(code,…)` after — and sizes the page as `align(512·5 + 1024, 4096)` (`COMP_AVG_INSTR_SIZE = 5`). The ARM64 backend (`compiler_a64.c`) does the same with fixed 4-byte instructions (`ldr x7,[x0,#56]`, …).
|
||||
|
||||
The **Rust** uses the `dynasmrt` runtime assembler, so the backend reads like annotated assembly and the buffer is an `ExecutableBuffer` mmap managed by the crate:
|
||||
|
||||
```rust
|
||||
// hashx/src/compiler/x86_64.rs
|
||||
use dynasmrt::{DynasmApi, DynasmLabelApi, x64};
|
||||
dynasm!(asm ; .arch x64 ; mov rcx, rdi ; ...);
|
||||
```
|
||||
|
||||
Both default to **try-compile-then-fall-back-to-interpret** (`RuntimeOption::TryCompile`; in C, `hashx_alloc(HASHX_COMPILED)` else `HASHX_INTERPRETED`), and the compiler is only available on x86-64 and aarch64 — elsewhere the interpreter is mandatory. The crucial invariant across all four execution engines — C-interp, C-x86, C-arm, and the Rust trio — is **bit-identical output**, enforced by shared test vectors.
|
||||
|
||||
> **Determinism is a cross-product property.** {interpret, x86 JIT, arm JIT} × {C, Rust} must all agree, for every seed and input. That is why the generator's PRNG order, the wrapping/sign-extension rules, the digest, and even the "vestigial" draw are pinned: any of them differing would split the matrix.
|
||||
|
||||
---
|
||||
|
||||
## 4. The Equihash search, analyzed
|
||||
|
||||
With `H` fixed, the puzzle is a constrained subset-sum. This section states it formally, explains the solver's arithmetic precisely, and derives its complexity.
|
||||
|
||||
### 4.1 The puzzle, formally
|
||||
|
||||
A solution is eight 16-bit indices `i0..i7` such that, with `H(i)` the low 64 bits of the inner hash and all sums taken mod 2⁶⁴:
|
||||
|
||||
- **Full sum:** `Σ H(iₖ) ≡ 0 (mod 2⁶⁰)`.
|
||||
- **Tree partial sums (Wagner structure):** each adjacent pair zeros the low 15 bits; each group of four zeros the low 30; all eight zero 60. (`N/(K+1) = 60/4 = 15` bits per level.)
|
||||
- **Ordering (canonicalization):** at each tree node the left branch must be ≤ the right branch when the index groups are compared as little-endian byte strings (`tree_cmp1/2/4` over 2/4/8 bytes).
|
||||
|
||||
The ordering constraint is not cosmetic: without it the same eight hashes could be permuted into many distinct "solutions," so it is a genuine validity condition and is checked *first* (before any hashing) during verification. The Rust verifier is the cleanest statement of the sum tree — one recursion halving the bit-width at each level:
|
||||
|
||||
```rust
|
||||
// equix/src/solution.rs
|
||||
fn check_tree_sums(func, items, n_bits) -> Result<HashValue, ()> {
|
||||
let sum = if items.len() == 2 {
|
||||
item_hash(func, items[0]).wrapping_add(item_hash(func, items[1]))
|
||||
} else {
|
||||
let (l, r) = items.split_at(items.len()/2);
|
||||
check_tree_sums(func,l,n_bits/2)?.wrapping_add(check_tree_sums(func,r,n_bits/2)?)
|
||||
};
|
||||
if (sum & ((1 << n_bits) - 1)) == 0 { Ok(sum) } else { Err(()) } // 60 → 30 → 15
|
||||
}
|
||||
```
|
||||
|
||||
### 4.2 Wagner as bucketed meet-in-the-middle, with the carry
|
||||
|
||||
Finding the sum directly is `2^128`. Wagner's algorithm finds it level by level: to zero 15 more bits of a sum, bucket every value by those 15 bits and only combine **complementary** buckets — bucket `b` with bucket `−b mod 2¹⁵` — because only those can cancel. Each 15-bit cancellation is split into a **coarse** pass (8 bits, `value % 256`) and a **fine** pass (7 bits, `value % 128`); `8 + 7 = 15`.
|
||||
|
||||
The one piece of arithmetic that is easy to get wrong — and therefore a precise compatibility checkpoint — is the **carry correction**. When you pair coarse bucket `b` with bucket `256 − b`, their coarse parts sum to exactly 256, producing a carry out of the 8-bit field. The C re-injects it before the fine pass with `value + CARRY`, where `CARRY = (bucket_idx != 0)`:
|
||||
|
||||
```c
|
||||
/* solver.c — MAKE_PAIRS1 (per layer; same shape for 2 and 3) */
|
||||
stage1_data_item value = STAGE1_DATA(bucket_idx, item_idx) + CARRY; /* +1 if b != 0 */
|
||||
u32 fine_buck_idx = value % NUM_FINE_BUCKETS;
|
||||
u32 fine_cpl_bucket = INVERT_SCRATCH(fine_buck_idx); /* -f mod 128 */
|
||||
... stage1_data_item sum = value + cpl_value;
|
||||
assert((sum % NUM_FINE_BUCKETS) == 0); /* fine bits cancel */
|
||||
sum /= NUM_FINE_BUCKETS; /* shift off 7 bits */
|
||||
```
|
||||
|
||||
The Rust performs the identical correction inside `collision::search`, iterating `first_bucket in 0..=(N/2)` and pairing with `first_bucket.wrapping_neg() % N`, then keeping sums whose `low_bits_are_zero(num_bits)`. Same buckets, same complement, same carry — by construction, so that both implementations *discard the same overflow solutions* and emit identical output.
|
||||
|
||||
### 4.3 Complexity and the birthday accounting
|
||||
|
||||
Let `S = 2¹⁶` be the index space. The four stages:
|
||||
|
||||
- **Stage 0** — evaluate `H` on all `S` indices; bucket by 8 bits into 256 coarse buckets. Mean occupancy `S/256 = 256` items/bucket (cap `COARSE_BUCKET_ITEMS = 336`). Cost: `S` HashX evaluations — *this dominates a solve.*
|
||||
- **Stage 1** — within each complementary coarse-bucket pair (~512 items) sub-bucket by 7 fine bits (128 fine buckets, cap 12) and combine complements. Pairs with low 15 bits zero ≈ `C(S,2) / 2¹⁵ ≈ 2³¹ / 2¹⁵ = 2¹⁶`. So ~`S` items survive into layer 2.
|
||||
- **Stage 2** — same on the 15-bit-zero items; ~`2¹⁶ / 2¹⁵·... ≈ 2¹⁶` survive with 30 bits zero.
|
||||
- **Stage 3** — match the final 30 bits; expected solutions ≈ `2¹⁶ / 2¹⁵ ≈ 2`.
|
||||
|
||||
That last line is the origin of the famous **"~2 solutions per challenge on average."** Each layer keeps the population near `2¹⁶`, so total work is `Θ(2¹⁶)` operations with a small constant, and **time is dominated by the 65,536 stage-0 hash evaluations**, not the bucketing. Peak memory is the fixed solver heap, ≈ **1.81 MiB**.
|
||||
|
||||
**Why the solver is heuristic but the verifier is exact.** Bucket capacities (336 coarse, 12 fine) are finite, and occupancy is roughly Poisson. Coarse mean 256 vs cap 336 is ~5σ (σ ≈ 16), so overflow is rare but *possible*; when a bucket overflows, extra items are silently dropped (`if (item_idx >= CAP) continue;`). Therefore the solver may **miss** some valid solutions — it is a probabilistic search tuned to find *enough*, not *all*. The Rust documents this as intentional and matches the C's capacities precisely so the two miss the *same* solutions. The **verifier**, by contrast, does no bucketing and is exact: it recomputes the eight hashes and checks the sums directly. This asymmetry is fine for a client puzzle — a solver only needs one solution; a verifier must be sound.
|
||||
|
||||
### 4.4 Self-complementary buckets, dedup, and the solution cap
|
||||
|
||||
Two coarse buckets are their own complement: `0` (since `−0 = 0`) and `128` (since `−128 ≡ 128 mod 256`). Pairing a bucket with itself would double-count and produce duplicate pairs, so the solver special-cases `cpl_bucket == bucket_idx`: it interleaves matching into the scratch-building loop so each item only pairs with *earlier* items in the same bucket. The C carries a `nodup`/branch for this; the Rust's `collision::search` handles it by construction. The solver returns at most `EQUIX_MAX_SOLS = 8` solutions (the Rust `SolutionArray` is an `ArrayVec<Solution, 8>`), and dedups adjacent equal solutions before pushing.
|
||||
|
||||
### 4.5 Parent pointers and canonical reconstruction
|
||||
|
||||
Each combined node stores where its two parents live, packed into one integer with **8 bits of bucket + 9 + 9 bits of item index** — identical widths in both:
|
||||
|
||||
```c
|
||||
#define MAKE_ITEM(bucket, left, right) ((left) << 17 | (right) << 8 | (bucket))
|
||||
```
|
||||
|
||||
```rust
|
||||
type Layer0Collision = PackedCollision<u32, 8, 9>; // BUCKET_BITS=8, ITEM_BITS=9
|
||||
```
|
||||
|
||||
At the end the solver walks this tree back to the leaves to recover the eight indices, then puts them in canonical order. The C does it with explicit conditional swaps at each level (`tree_cmp1/2/4` deciding whether to swap pairs/quads/halves); the Rust collects the leaves and calls `Solution::sort_from_array`, whose `branches_are_sorted` predicate compares groups reversed (i.e., as little-endian values). Same canonical form, so the emitted 16-byte solutions match.
|
||||
|
||||
---
|
||||
|
||||
## 5. Solver memory: two designs for the same 1.81 MiB
|
||||
|
||||
Both implementations keep the entire search in one preallocated block sized to live in cache, and both **reuse** the same bytes across stages via a union. The implementations of that idea are a study in contrasts.
|
||||
|
||||
**C — one struct, one union, hand-counted bytes.** `solver_heap.h` lays out every table explicitly and overlaps the early-stage data with the late-stage tables:
|
||||
|
||||
```c
|
||||
typedef struct solver_heap {
|
||||
stage1_idx_hashtab stage1_indices; /* 172 544 B */
|
||||
stage2_idx_hashtab stage2_indices; /* 344 576 B */
|
||||
stage2_data_hashtab stage2_data; /* 688 128 B */
|
||||
union { /* reuse: early vs late stages */
|
||||
stage1_data_hashtab stage1_data; /* 688 128 B (stages 0–1) */
|
||||
struct { stage3_idx_hashtab stage3_indices; /* 344 576 B */
|
||||
stage3_data_hashtab stage3_data; }; /* 344 064 B (stage 3)*/
|
||||
};
|
||||
fine_hashtab scratch_ht; /* 3 200 B */
|
||||
} solver_heap; /* TOTAL: 1 897 088 B */
|
||||
```
|
||||
|
||||
Per-bucket counts live *inside* each table (`uint16_t counts[256]`), and the block is obtained with plain `malloc` — or `hashx_vm_alloc_huge` when `EQUIX_CTX_HUGEPAGES` is set, the one allocation knob the C exposes.
|
||||
|
||||
**Rust — `MaybeUninit` layout, checked union, counts outside.** The same overlap is a `union Overlay { first: OverlayFirst, second: OverlaySecond }`, but the whole structure is built from `BucketArrayMemory<N, M, T>([[ MaybeUninit<T>; M]; N])` and marked with an `unsafe` `Uninit` trait that promises the bytes are safe to leave uninitialized until written. Allocation goes straight to the heap (too large for the stack) and is reused across solves via `SolverMemory`:
|
||||
|
||||
```rust
|
||||
// bucket_array/mem.rs — the safety-critical insert: count only ever rises after a real write
|
||||
fn insert<F: FnMut(usize)>(&mut self, bucket: usize, mut writer: F) -> Result<(), ()> {
|
||||
let n: usize = self.counts[bucket].into();
|
||||
if n < CAP { writer(n); self.counts[bucket] = self.counts[bucket] + C::one(); Ok(()) }
|
||||
else { Err(()) }
|
||||
}
|
||||
```
|
||||
|
||||
The safety argument is explicit and worth appreciating: reads use `assume_init`, which is only sound if `counts` accurately reflects which slots were written; the code guarantees this by *only* incrementing a count after the writer has unconditionally written, and by tying everything to a `&mut` whose lifetime begins with zeroed counts. Switching layouts is done by borrowing a different union field, so the borrow checker enforces that the two overlays are never live at once. Because the bucket *counters* live outside the overlaid block, the Rust's measured size is **1,895,424 B** vs the C's **1,897,088 B** — a ~1,664-byte difference of bookkeeping placement, not of algorithm.
|
||||
|
||||
| | C `solver_heap` | Rust `SolverMemory` |
|
||||
|---|---|---|
|
||||
| Total | 1,897,088 B | 1,895,424 B |
|
||||
| Counters | inside each table | in separate `BucketState` |
|
||||
| Reuse mechanism | `union` | checked `union Overlay` |
|
||||
| Uninit handling | raw `malloc` (bytes undefined) | `MaybeUninit` + `unsafe Uninit` |
|
||||
| Big pages | `EQUIX_CTX_HUGEPAGES` | (none) |
|
||||
| Reuse across solves | caller keeps `ctx->heap` | `solve_with_memory(&mut SolverMemory)` |
|
||||
| Bucket geometry | 256×336 / 128×12 | **identical** |
|
||||
|
||||
The identical geometry is the point: same capacities ⇒ same overflow-discard behavior ⇒ identical solution sets.
|
||||
|
||||
---
|
||||
|
||||
## 6. API surface and error models
|
||||
|
||||
The libraries diverge most visibly at their edges, and the differences are idiomatic rather than algorithmic.
|
||||
|
||||
**Lifecycle.** C uses an opaque context allocated once and reused (`equix_alloc(flags)` → many `equix_solve`/`equix_verify`), with `flags` selecting verify/solve, compiled/interpreted, and hugepages. Rust uses values and builders (`EquiX::new(challenge)`, `EquiXBuilder` for `RuntimeOption`), plus free functions (`equix::solve`, `verify`, `verify_bytes`) for the common path. Solver scratch is a caller-held `ctx->heap` in C and a `SolverMemory` you can thread through `solve_with_memory` in Rust.
|
||||
|
||||
**Errors.** The taxonomies line up one-to-one, but Rust splits the *order* check into the type system:
|
||||
|
||||
| C `equix_result` | Rust | When |
|
||||
|---|---|---|
|
||||
| `EQUIX_OK` | `Ok(())` | valid |
|
||||
| `EQUIX_ORDER` | `Error::Order` | indices not canonical (checked first, no hashing) |
|
||||
| `EQUIX_CHALLENGE` | `Error::Hash(ProgramConstraints)` | HashX won't build for this seed |
|
||||
| `EQUIX_PARTIAL_SUM` / `EQUIX_FINAL_SUM` | `Error::HashSum` | tree/full sum check failed |
|
||||
|
||||
In Rust you cannot hold a `Solution` without having passed the order check (it's enforced in `try_from_bytes`/`try_from_array`), so `EquiX::verify` only ever needs to check sums — the same cheapest-first ordering as the C `equix_verify`, but encoded in types. The Rust additionally surfaces `Error::Hash(Compiler(...))` when `RuntimeOption::CompileOnly` is chosen and the JIT is unavailable, a state the C reaches via the `HASHX_NOTSUPP` sentinel at `alloc` time.
|
||||
|
||||
---
|
||||
|
||||
## 7. Determinism and compatibility as an engineering discipline
|
||||
|
||||
Equi-X has no prose specification; correctness *is* reproducing the reference's output. Reading the two codebases together, you can enumerate exactly what a third implementation must match. This is the checklist:
|
||||
|
||||
- **Endianness:** hash output read little-endian (`load64`); PRNG bytes/words extracted big-endian from each SipHash block.
|
||||
- **PRNG consumption order:** separate `u8`/`u32` buffers over a shared counter; the *interleaving* of draws is significant (§2.1).
|
||||
- **The vestigial wide-multiply `u32` draw** (§2.4) — must be consumed even though unused, or the stream desynchronizes.
|
||||
- **Arithmetic semantics:** 64-bit wrapping add/sub/mul; 32-bit immediates sign-extended; wide multiply keeps the high 64 bits; rotate-right by constant.
|
||||
- **Acceptance gates:** 512 instructions, 192 multiplies, retire at cycle 194 (§2.5).
|
||||
- **Digest:** fold `key1` into `r0,r1,r6,r7`, one SipRound per half, XOR-combine (§3.1).
|
||||
- **Solver geometry:** 256/128 buckets, 336/12 capacities, the `b ↔ −b` complement, the `+CARRY` correction, the 8/9/9 parent packing, and the canonical ordering (§4) — so even *which* solutions are discarded matches.
|
||||
|
||||
The Rust enforces all of this with **captured reference vectors** at every layer (`rng_vectors`, `siphash24_ctr_vectors`, `hashx_vectors`, `tor_equix_vectors`), which is how it can be both idiomatic and bug-for-bug faithful. The few intentional internal differences are behaviorally invisible: the interpreter's `target + 1` vs re-running `Target` (§3.2), and where bucket counters are stored (§5). Everything an external observer can see — generated programs, hash outputs, the set of emitted solutions, verification verdicts — is identical.
|
||||
|
||||
---
|
||||
|
||||
## 8. Security-relevant code analysis
|
||||
|
||||
Reading the implementations surfaces several properties that bear on Equi-X's security posture. Stated precisely, with what the code does and does not defend:
|
||||
|
||||
- **No secret inputs.** HashX's control flow is input-dependent (the one-shot branch tests live data), so evaluation time can leak the input. The code is explicit that this is acceptable *only because the puzzle input is public*. Using HashX as a keyed hash over secret data would be a timing-side-channel mistake. (Volume I's "intuition" caveat, here as a hard rule.)
|
||||
- **Weak instances are neutralized by addition, not avoided.** ~0.2% of seeds produce HashX functions with many internal collisions. Under XOR these would yield enormous numbers of trivial solutions; Equi-X's switch to **sum mod 2⁶⁰** means equal hashes no longer cancel, so weak instances don't become exploitable. The solver and verifier therefore never need a collision-resistance assumption on HashX — only preimage resistance.
|
||||
- **Solver non-exhaustiveness is sound.** Bucket overflow silently discards candidates (§4.3), so the solver can miss solutions — but it can never *invent* one, because every emitted solution is rebuilt from real items and (in practice) re-checked by the exact verifier. Missing solutions only costs the solver attempts, never soundness.
|
||||
- **JIT hardening.** The compiled path uses W^X page permissions (`vm_rw` to emit, then `vm_rx` to execute) so the code page is never simultaneously writable and executable. Emission is straight-line from a trusted program with no input-derived lengths, limiting the JIT's attack surface.
|
||||
- **The memory-hardness/branch interplay is by design and acknowledged as imperfect.** HashX itself is register-only (no scratchpad), which is what makes its program shape so CPU-like; the Equihash layer adds the ~1.81 MiB cache-resident working set that blunts a GPU's bandwidth edge and gives a logic-only FPGA something it must store. The generator even computes a hypothetical **"ASIC latency"** (`asic_latencies`, assuming unlimited parallelism and 1-cycle ops) in its stats build — a window into the designer's own modeling of the parallel lower bound. None of this targets true ASIC-resistance, which Equi-X explicitly abandons.
|
||||
- **Memory-safety boundary.** The C solver is classic manual indexing into a `malloc`'d block; correctness rests on the capacity checks (`if (idx >= CAP) continue;`). The Rust achieves the same layout but routes every read through `assume_init` guarded by a borrow-checked count, converting "don't read uninitialized memory" from a discipline into a compile-time-checked invariant. Same bytes, very different safety story.
|
||||
|
||||
---
|
||||
|
||||
## 9. Performance model
|
||||
|
||||
A coherent mental model of where the cost goes, assembled from the code structure (absolute figures are the designer's, on era-specific hardware, and approximate):
|
||||
|
||||
- **Solve ≈ 5–8 ms.** Decomposes as: one program generation (~0.05 ms, O(512) with bounded per-slot search), then `2¹⁶` HashX evaluations in stage 0 (the dominant term), then three `Θ(2¹⁶)` bucket passes (cheap relative to hashing). HashX overhead vs the search is well under 1%.
|
||||
- **Verify ≈ 50 µs.** Eight HashX evaluations + a few additions + six integer comparisons, with the free ordering check rejecting malformed proofs before any hash runs.
|
||||
- **The asymmetry, quantified.** A solver performs ~`2¹⁶` hash evaluations to a verifier's 8 — a factor of ~`2¹³ ≈ 8000` in hashing alone — and the wall-clock ratio is ~100×. The 16-byte proof keeps the *verifier's* input cost negligible, which is the property that makes Equi-X usable as a DoS gate rather than a self-inflicted one.
|
||||
- **JIT vs interpret.** Compiled HashX is roughly an order of magnitude faster than interpreted and is the deployed path; both are required to agree bit-for-bit, so the interpreter doubles as the executable specification and the portable fallback.
|
||||
- **C vs Rust.** The Rust crate's own tests note it is modestly slower than the C reference (its comments reference small solver/verifier deltas), the expected cost of generic, bounds-checked, memory-safe code; the algorithmic complexity and outputs are identical.
|
||||
|
||||
---
|
||||
|
||||
## 10. Synthesis
|
||||
|
||||
The C and Rust implementations are a near-perfect natural experiment: the same nontrivial algorithm written once for *speed and definition* and once for *safety and clarity*, constrained to produce identical output down to which solutions they discard. Reading them in parallel is the most efficient way to understand Equi-X, because each compensates for the other's weakness as a teaching text — the C answers "what exactly happens," the Rust answers "why, and what's safe."
|
||||
|
||||
The deeper lesson the code carries is that Equi-X's security rests less on any single clever primitive than on **disciplined uniformity**: a program generator that manufactures equal-cost, CPU-shaped functions; a combiner (addition) chosen so a weak hash can't be exploited; a solver engineered to stay in cache; and a verifier kept structurally cheap and exact. The implementations make those choices legible in a way the (nonexistent) prose spec never could.
|
||||
|
||||
---
|
||||
|
||||
## Appendix A — Constant and parameter correspondence
|
||||
|
||||
| Quantity | Value | C symbol | Rust symbol |
|
||||
|---|---|---|---|
|
||||
| Program size | 512 | `REQUIREMENT_SIZE` / `HASHX_PROGRAM_MAX_SIZE` | `NUM_INSTRUCTIONS` |
|
||||
| Required multiplies | 192 | `REQUIREMENT_MUL_COUNT` | `REQUIRED_MULTIPLIES` |
|
||||
| Scheduler target | 192 cycles | `TARGET_CYCLE` | `TARGET_CYCLES` |
|
||||
| Required final retire | cycle 194 | `REQUIREMENT_LATENCY - 1` | `REQUIRED_OVERALL_RESULT_AT_CYCLE` |
|
||||
| Ports | 3 (P5,P0,P1) | `NUM_PORTS` | `NUM_EXECUTION_PORTS` |
|
||||
| Latencies | 1 / 3 / 4 | per `instr_template.latency` | `instruction_latency_cycles` |
|
||||
| Branch mask weight | 4 (≈1/16) | `LOG2_BRANCH_PROB` | `BRANCH_MASK_BIT_WEIGHT` |
|
||||
| Registers | 8; R5 special | `REGISTER_NEEDS_DISPLACEMENT = 5` | `register::R5` |
|
||||
| Gen reject rate | ~1 / 10⁴ seeds | comment in `program.c` | doc in `program.rs` |
|
||||
| Weak-seed rate | ~0.2% | (HashX README) | (HashX README) |
|
||||
| Equihash N, K | 60, 3 | `EQUIX_*_MASK` widths | `EQUIHASH_N`, `EQUIHASH_K` |
|
||||
| Items / solution | 8 → 16 B | `EQUIX_NUM_IDX` | `Solution::NUM_ITEMS` |
|
||||
| Index space | 2¹⁶ | `INDEX_SPACE` | `u16::MIN..=u16::MAX` |
|
||||
| Coarse / fine buckets | 256 / 128 | `NUM_COARSE/FINE_BUCKETS` | bucket array `N` params |
|
||||
| Bucket capacities | 336 / 12 | `COARSE/FINE_BUCKET_ITEMS` | `CAP` params |
|
||||
| Parent pack | 8 / 9 / 9 bits | `MAKE_ITEM` shifts | `PackedCollision<u32,8,9>` |
|
||||
| Stage masks | 2¹⁵, 2³⁰, 2⁶⁰ −1 | `EQUIX_STAGE1/STAGE2/FULL_MASK` | `n_bits` recursion |
|
||||
| Solver memory | ~1.81 MiB | 1,897,088 B (`solver_heap`) | 1,895,424 B (`SolverMemory::SIZE`) |
|
||||
| Max solutions | 8 | `EQUIX_MAX_SOLS` | `SolutionArray` capacity |
|
||||
|
||||
## Appendix B — Instruction template fields (C `instr_template`)
|
||||
|
||||
Each C template carries the fields the generator and JIT consume; the Rust spreads the same data across `scheduler::model` (ports/latency), `generator::model` (selection), and `program::Instruction` (semantics).
|
||||
|
||||
| Opcode | latency | uop1 / uop2 | distinct dst? | has src? | immediate |
|
||||
|---|---|---|---|---|---|
|
||||
| `UMULH_R` / `SMULH_R` | 4 | P1 / P5 | no | yes | — (draws an unused `op_par` u32) |
|
||||
| `MUL_R` | 3 | P1 / — | yes | yes | — |
|
||||
| `SUB_R` | 1 | P015 / — | yes | yes | — |
|
||||
| `XOR_R` | 1 | P015 / — | yes | yes | — |
|
||||
| `ADD_RS` | 1 | P01 / — | yes | yes | 2-bit shift; dst ≠ R5 |
|
||||
| `ROR_C` | 1 | P05 / — | yes | no | 6-bit rotate (nonzero) |
|
||||
| `ADD_C` / `XOR_C` | 1 | P015 / — | yes | no | 32-bit (nonzero) |
|
||||
| `TARGET` / `BRANCH` | 1 | P015 / P015 | — | no | branch: 4-bit-weight mask |
|
||||
|
||||
## Appendix C — Provenance
|
||||
|
||||
- **C:** `tevador/equix`, `tevador/hashx`, v1.0.0 (2020), LGPL-3.0. The normative reference; no separate written spec.
|
||||
- **Rust:** `equix` 0.6.1, `hashx` 0.7.1 (crates.io), Tor Project / Arti, LGPL-3.0. Reproduces the C output exactly; ships shared Tor test vectors. Later versions (`equix` 0.7.0 / `hashx` 0.9.0, June 30, 2026) carry no algorithmic changes.
|
||||
- Snippets trimmed for readability (`...`); identifiers, constants, and arithmetic verbatim from the cited files (re-verified July 2026). Read alongside Volume I (the Walkthrough) for the conceptual model this analysis assumes.
|
||||
|
||||
|
||||
@ -0,0 +1,122 @@
|
||||
# Equi-X — Attacking and Accelerating
|
||||
|
||||
### Volume III: how hard is it *really*? Optimized solving, hardware feasibility, and an honest verdict on the CPU-friendliness claim
|
||||
|
||||
*Part of the Equi-X implementation series · companion to Volumes I (Walkthrough), II (Deep Analysis), IV (PoW Landscape), V (Future Directions) · June 2026 · revised July 2026*
|
||||
|
||||
---
|
||||
|
||||
## 0. Scope
|
||||
|
||||
Volumes I and II explained what Equi-X computes and how the C and Rust implementations build it. This volume turns adversarial: **assume you want to solve Equi-X as cheaply and as fast as possible — how far can you get, and where does the design stop you?** That question is the entire point of the algorithm (its third and "most important" requirement is that GPUs and FPGAs gain little advantage), and it is the open problem the project's Research Survey flags as highest-value.
|
||||
|
||||
The analysis is grounded in the code (Volumes I–II) and in tevador's own design log, which contains the only first-party hardware-resistance reasoning that exists. Where the devlog's 2020-era empirical claims have aged, this volume corrects them against 2026 evidence. Absolute performance figures are approximate and hardware-specific; the structural arguments are what matter.
|
||||
|
||||
A framing note up front: Equi-X is a **DoS puzzle, not a mining function**. The attacker's goal is not to win a block race but to *impersonate many legitimate clients cheaply* — to solve at volume for less than the defender assumes, or to turn the puzzle itself into a DoS vector. That changes which attacks matter, as Section 6 shows.
|
||||
|
||||
---
|
||||
|
||||
## 1. What "winning" means for an attacker
|
||||
|
||||
The reference design states three requirements and one explicit non-goal (devlog):
|
||||
|
||||
1. Proof < ~200 bytes — *Equi-X solutions are 16 bytes.*
|
||||
2. Verification must be fast — *~50 µs, exact, no memory.*
|
||||
3. **GPUs/FPGAs must not provide a large solving advantage** — "the most important one."
|
||||
4. **ASIC-resistance is explicitly NOT required** — a 28 nm ASIC costs >US$1M, and Tor can change the algorithm with a patch, unlike a consensus blockchain.
|
||||
|
||||
So an attacker "wins" by achieving any of:
|
||||
|
||||
- **(A) A large per-dollar solving speedup on commodity parallel hardware** (GPU/FPGA), letting them mint solutions far faster than a browser CPU. This is the threat the design targets.
|
||||
- **(B) Making verification expensive** (a junk-proof flood). The asymmetric verifier is the defense.
|
||||
- **(C) Attacking the protocol economics around the puzzle** rather than the puzzle itself — precomputation, or gaming the effort controller. In practice this is where the deployed system is actually weak (Section 6).
|
||||
|
||||
ASIC silicon-proofing is *not* on the attacker's win list from the defender's perspective, because the defender's answer to an ASIC is a one-line algorithm patch. We return to that asymmetry in Section 5.
|
||||
|
||||
---
|
||||
|
||||
## 2. Accelerating the CPU solver
|
||||
|
||||
Before reaching for exotic hardware, how much headroom does the *reference CPU solver* leave? First, where the time goes (Volume II, §9):
|
||||
|
||||
> A solve is **dominated by the 2¹⁶ = 65,536 HashX evaluations in stage 0.** Program generation is ~0.05 ms; the three bucket passes are `Θ(2¹⁶)` and cheap next to the hashing; HashX overhead is "under 1%" of an attempt (devlog). So *accelerating Equi-X ≈ accelerating HashX throughput*, with the bucket sort a distant second.
|
||||
|
||||
That single fact shapes every optimization:
|
||||
|
||||
**Batching and SIMD.** The reference evaluates indices one at a time. A throughput solver wants to run many hash evaluations in parallel lanes. But HashX resists data-parallelism at the instruction level: each hash is a **single dependent chain of ~512 integer ops filling ~195 cycles** (Volume II, §2.5), dominated by `MUL`/wide-multiply latency, with almost no instruction-level parallelism *within* one hash. You can run *different* indices in different SIMD lanes, but the chain is full of 64×64→128 multiplies and one-shot input-dependent branches, which vectorize poorly (no wide-lane high-multiply on common SIMD; divergent branches break lockstep). The practical CPU win is *thread-level* parallelism across indices (the reference already uses 16 threads for ~2400 Sol/s) plus the JIT — not lane-level SIMD inside a hash. The JIT already captures most of the win; the interpreter is ~10× slower and exists mainly as the portable spec.
|
||||
|
||||
**The bucket sort: a known memory–time knob.** The solver heap is 1.81 MiB "with negligible solution discarding," and the devlog notes it "could be reduced to a minimum of 1 MiB with perfect bit packing and around 25% of discarded solutions, although this is viable only for custom hardware." That is a real memory–time tradeoff, but it cuts the *cheap* part of the solve (the sort), not the dominant hashing, and it costs solutions — so it does nothing for a CPU attacker and only matters when squeezing a hardware datapath (Section 4–5).
|
||||
|
||||
**The 2025 Wagner memory–time tradeoffs, and why they barely touch Equi-X.** Recent cryptanalysis — ePrint 2025/2141, *"Memory Optimizations of Wagner's Algorithm with Applications to Equihash"* (Tang, Ding, Sun, Gong; TCHES 2026) — gives a near-linear tradeoff that **halves Wagner's peak memory (2nN → nN bits) for ~2× time across all Equihash parameters**, plus an "ASIC-friendly framework leveraging an external-memory caching mechanism." This is significant for *large* Equihash (Zcash's 144 MiB, BTG's 2.5 GiB), where peak memory is the binding constraint. For **Equi-X it is largely moot**: the working set is already ~1.8 MiB and *deliberately cache-resident*, so halving it buys nothing a CPU cares about, and the "external-memory caching" framing is precisely the ASIC/large-memory regime Equi-X engineered itself out of. The companion paper (ePrint 2025/1351, *Single or K Lists?*) is more pointed — it argues Equihash's **index-pointer technique weakens ASIC-resistance** and proposes *Requihash* — and it explicitly extends to the **k-SUM** variant Equi-X uses. But its target is ASIC-resistance, which Equi-X never claims; see Section 5.
|
||||
|
||||
**Bottom line for CPU:** the reference solver is already near the achievable envelope — JIT + threads + cache-resident sort. The interesting question is whether *other* hardware can beat a CPU at all.
|
||||
|
||||
---
|
||||
|
||||
## 3. GPU feasibility, grounded in the code
|
||||
|
||||
This is requirement #3, the one that matters. The structural reasons a GPU struggles with Equi-X are all visible in the implementation:
|
||||
|
||||
- **Per-instance code generation.** Every challenge produces a *different* 512-instruction program (Volume I, §3). A GPU can't bake Equi-X into a fixed kernel; it must JIT a new kernel (or interpret) for each challenge, and that cost is amortized over only **2¹⁶ hashes per instance** — small. (The successor HashWX slashes this to **463 attempts per instance**, deliberately, to make per-instance compilation dominate on a GPU.)
|
||||
- **Latency-bound hashes, not throughput-bound.** Each hash is a ~195-cycle dependent multiply chain with near-zero internal ILP (Section 2). GPUs win on throughput over thousands of independent threads, but each Equi-X hash is serial and multiply-heavy; you need enormous occupancy to hide the latency, and the integer-multiply units are the bottleneck.
|
||||
- **Warp divergence from the one-shot branch.** HashX's input-dependent branch (≈1/16) was *added specifically to hinder GPUs* (devlog). On a 32-thread warp, divergent branches serialize. (HashWX's design doc concedes 16 branches at 1/16 is *insufficient* divergence — which is exactly why HashWX moves to 64 sub-programs each looping with probability 1/2.)
|
||||
- **Cache-residency neutralizes the GPU's trump card.** A GPU's main edge is memory bandwidth. Equi-X's ~1.8 MiB sort fits in CPU L2/L3, "the only case when CPUs can compete with GPUs in memory bandwidth" (devlog). On a GPU the sort lands in higher-latency local/L2 memory.
|
||||
|
||||
**What the evidence actually says.** As of July 2026, **no public GPU (or FPGA or ASIC) solver for Equi-X or HashX exists** — confirmed by exhaustive search (re-checked July 2026) and by the author's own devlog footnote: *"No GPU implementation exists; upper bound based on RandomX performance."* The headline "<50% of CPU" is therefore an **extrapolation, not a measurement**, and the comparison table lists Equi-X's GPU figure as literally "?".
|
||||
|
||||
How good is that extrapolation? It rests on RandomX's GPU behavior via `SChernykh/RandomX_CUDA`. A crucial subtlety the devlog glosses: that project's oft-quoted ~144–153% figures are **GPU-vs-old-GPU-algorithm, not GPU-vs-CPU**. Measured GPU-vs-CPU for RandomX is roughly **6–10%** (a top GPU ~2000 H/s vs a top CPU ~28,000–31,000 H/s). So if HashX/Equi-X resemble RandomX on a GPU, **"<50%" is conservative** — the real figure could be far lower. That is the optimistic reading.
|
||||
|
||||
The honest, pessimistic reading is equally important: **the designer himself is no longer confident.** HashWX's design document lists HashX's GPU weaknesses plainly — too few branches at too low a rate, and *no memory at all*, so "GPUs can achieve good interpreter performance while storing VM registers in shared memory." HashWX exists *because* HashX's GPU resistance was softer than first implied. So the defensible verdict is: **probably GPU-resistant, plausibly by a wide margin, but unproven — and the primitive's own author has shipped a redesign to shore it up.**
|
||||
|
||||
---
|
||||
|
||||
## 4. FPGA feasibility
|
||||
|
||||
FPGAs are the reason the Equihash layer exists at all. The devlog: *"I still felt a bit uneasy about HashX using no memory. This means that logic-only FPGAs could be a viable option to run HashX."* Three code-level facts bound an FPGA attack:
|
||||
|
||||
- **HashX is essentially a tiny CPU.** A per-instance program means an FPGA can't hard-wire one datapath; it must either reconfigure per challenge (bitstream generation is far slower than a CPU JIT) or implement a *soft processor* that interprets HashX — and a soft-core on an FPGA fabric will not out-run a hardened CPU at integer multiply chains.
|
||||
- **The memory layer forces storage.** The ~1.8 MiB sort gives a logic-only FPGA something it must hold in block RAM, and the bandwidth-bound sort doesn't favor the fabric the way bespoke pipelines would.
|
||||
- **SUM, not XOR, taxes the adders.** Equi-X sums hashes mod 2⁶⁰ instead of XORing them. "XOR is much faster in custom hardware… an FPGA-based solver will have to use slightly more resources to calculate the modular additions" (devlog). The carry chains cost real LUTs/area at scale.
|
||||
|
||||
No public FPGA solver exists. By analogy, FPGA attempts at RandomX (a close relative) are non-competitive: low clock speed and memory-latency bottlenecks mean "only an ASIC can outperform" the CPU. Equi-X's FPGA story is plausibly stronger still, because the per-instance program defeats fixed pipelines. **Verdict: FPGA acceleration is unlikely to beat a CPU, though — like the GPU case — this is reasoned, not demonstrated.**
|
||||
|
||||
---
|
||||
|
||||
## 5. ASIC feasibility — and why it's deliberately off the table
|
||||
|
||||
Equi-X **drops ASIC-resistance by design**, for two stated reasons: a 28 nm ASIC exceeds ~US$1M (un-amortizable against one victim), and Tor can swap the algorithm with a patch. An ASIC attacker would build a small CPU-like core (the HashX VM: 8 registers, integer ALU with fast multiply, one branch unit) plus modular adders and ~1.8 MiB of on-die SRAM, replicated for parallel attempts.
|
||||
|
||||
How much would that win? The most instructive data point is **RandomX, which *does* target ASIC-resistance and got ASIC'd anyway**: Bitmain's Antminer **X5** (2023) and **X9** (late 2025; ~1 MH/s at ~2.5 kW) are real commercial RandomX ASICs. But their edge is only **~3× CPU energy-efficiency — right at tevador's own predicted single-die ceiling** — a far cry from the millions-fold gap of SHA-256 ASICs. The lesson generalizes: a program-execution PoW caps the ASIC advantage at "a better CPU," not "a different universe." A hypothetical Equi-X ASIC would likely land in the same low-single-digit multiple — meaningful for a consensus coin, **irrelevant against a target Tor can re-key with a patch and that costs an attacker far less to DDoS by other means.**
|
||||
|
||||
The 2025 cryptanalysis (ePrint 2025/2141's external-memory ASIC framework; 2025/1351's index-pointer critique and *Requihash*) sharpens the ASIC picture for the **Equihash family at large-n** — but its relevance to Equi-X is limited precisely by Equi-X's choices: tiny cache-resident n=60, modular SUM rather than XOR, and the explicit abandonment of ASIC-resistance as a goal. **Verdict: an Equi-X ASIC is buildable and would win a bounded ~few× margin — and that's fine, because silicon-proofing was never the defense. Patchability is.**
|
||||
|
||||
---
|
||||
|
||||
## 6. The real attack surface: economics, not silicon
|
||||
|
||||
For a DoS puzzle the decisive question isn't "can you build faster hardware" but "can you make the *defended system* cheap to overwhelm." Here the code and the deployment diverge, and the honest answer is that **the deployed weakness is the protocol around Equi-X, not Equi-X.**
|
||||
|
||||
- **Precomputation is bounded but real.** Seeds rotate every ~105 min–2 h, and the challenge binds the server seed, the blinded service identity, and a client nonce (Volume I). Within a seed window an attacker can pre-mine solutions, but each `(seed, nonce)` is replay-checked, and effort is a *linear bid* rather than a fixed target — so precomputation buys a burst at seed rollover, not a standing advantage.
|
||||
- **Verification flooding is well-defended.** The verifier is ~50 µs, checks the *ordering constraint first with no hashing*, and rebuilds the hash only if order passes (Volume II, §6). Junk proofs are rejected almost for free; the puzzle does not become a DoS vector on itself.
|
||||
- **The control loop is the soft underbelly.** The genuinely demonstrated attack on deployed Tor is **OnionFlation** (USENIX Security 2025): gaming the suggested-effort AIMD controller to inflate difficulty for *all* clients for ~$1.20 to trigger and ~$0.10/hour to sustain — with a proven impossibility result that no estimator resists both congestion and inflation at once. This attacks the *economics*, not the primitive; Tor's response is **Proposal 362** (open, unmerged as of July 2026). See the Research Survey and the Technical Reference §10.5; this is also Volume V's top priority.
|
||||
|
||||
In other words: an attacker who wants to take down an onion service does **not** build an Equi-X GPU solver — they game the control loop. The primitive is doing its job; the surrounding protocol is where the value is.
|
||||
|
||||
---
|
||||
|
||||
## 7. Verdict: does the CPU-friendliness claim hold?
|
||||
|
||||
A scorecard, by requirement:
|
||||
|
||||
| Property | Status | Basis |
|
||||
|---|---|---|
|
||||
| Small proof (<200 B) | **✓ proven** | 16 bytes, by construction |
|
||||
| Fast, cheap verify | **✓ proven** | ~50 µs, exact, memoryless, order-checked first |
|
||||
| GPU resistance | **✓ likely, UNPROVEN** | no public solver; "<50%" extrapolated from RandomX (conservative — real RandomX GPU is ~6–10% of CPU); but the author shipped HashWX to harden known gaps |
|
||||
| FPGA resistance | **✓ plausible, UNPROVEN** | register-only HashX + cache-resident SUM sort; per-instance program defeats fixed pipelines; no public solver |
|
||||
| ASIC resistance | **✗ by design** | abandoned deliberately; bounded ~few× (cf. RandomX's ~3× ASIC); defense is patchability + economics |
|
||||
| Resistance to economic attack | **✗ at the protocol layer** | OnionFlation games the effort loop; fix (Prop 362) not yet deployed |
|
||||
|
||||
The structural case for CPU-friendliness is strong and coherent — per-instance latency-bound programs, an anti-GPU branch, modular-SUM, and a cache-resident sort all push in the same direction, and a decade of the RandomX family shows program-execution PoW caps specialized-hardware gains at low multiples. But the **central claim rests on an extrapolation, not a demonstration**, and the designer's own successor signals the margin is thinner than the "<50%" headline suggests.
|
||||
|
||||
The single highest-value adversarial project, therefore, is the one nobody has published: **build a real GPU (and FPGA) Equi-X/HashX solver and measure it.** Until then, "GPUs gain little" is a well-argued, conservatively-extrapolated, but still *unverified* claim — and that gap, not any ASIC, is the most interesting thing an attacker (or an honest researcher) could close. Volume V puts this at the top of the open agenda.
|
||||
@ -0,0 +1,117 @@
|
||||
# Equi-X in the Proof-of-Work Landscape
|
||||
|
||||
### Volume IV: lineage, family placement, and where Equi-X sits on the time / memory / bandwidth axes
|
||||
|
||||
*Part of the Equi-X implementation series · companion to Volumes I–III and V · June 2026 · revised July 2026*
|
||||
|
||||
---
|
||||
|
||||
## 0. Scope
|
||||
|
||||
Volumes I–III looked inward at the algorithm and its implementations. This volume looks outward: **where does Equi-X fit among proof-of-work designs, what did it inherit from its ancestors, and why are its specific choices the right ones for its niche?**
|
||||
|
||||
It is written to dovetail with two existing project documents and uses their vocabulary deliberately: the **Acceleration-Resistant PoW Survey** (which classifies the field into Families A–F along three parameter axes) and the **RandomX Research Survey** (whose §8.2 traces the "HashX → Equi-X → HashWX" successor lineage). The thesis here is simple and, once seen, hard to unsee:
|
||||
|
||||
> **Equi-X is a deliberate hybrid of two PoW families** — it grafts the CPU-binding *program-execution* core of Family D (the RandomX lineage) onto the asymmetric, small-proof, fast-verify *birthday-search shell* of Family B (Equihash), and then tunes every parameter for **denial-of-service defense rather than consensus mining.** Almost everything distinctive about it follows from that one sentence.
|
||||
|
||||
---
|
||||
|
||||
## 1. The axes, and the niche
|
||||
|
||||
The Acceleration-Resistant Survey frames hardware resistance along **three parameter axes** — compute/time, memory/space, and bandwidth/energy — and distinguishes *absolute* resistance (no one can do better) from *economic* resistance (no one can do better cheaply enough to matter). Two more distinctions matter for placing Equi-X:
|
||||
|
||||
- **Consensus mining vs. DoS defense.** A mining PoW is run continuously by adversarial profit-maximizers and must resist amortization over millions of dollars of hardware for years. A DoS puzzle is run occasionally by ordinary clients and need only make *bulk* solving uneconomic for an attacker who could otherwise impersonate many clients — and, crucially, it can be **re-keyed with a software patch** if broken. Equi-X is squarely the second kind.
|
||||
- **Where Equi-X sits on the axes: tiny on all three.** ~1.8 MiB memory, ~6–8 ms solve, ~50 µs verify, 16-byte proof. It is not trying to be expensive; it is trying to be *flat* — equally cheap on a CPU and on an attacker's GPU. That is a different objective from almost every mining PoW in the landscape, and it explains why a direct "which is more ASIC-resistant" comparison often misses the point.
|
||||
|
||||
---
|
||||
|
||||
## 2. Parent #1 — Family D, the program-execution lineage
|
||||
|
||||
Equi-X's inner hash is the end of a clear genealogical line within Family D (CPU/VM program-execution functions):
|
||||
|
||||
```text
|
||||
CryptoNight ─► RandomX ─► SuperscalarHash ─► HashX ─► (HashWX)
|
||||
(2013-19) (Monero, (RandomX's DAG- (Equi-X's (successor,
|
||||
2019) init component) inner hash) Vol V)
|
||||
```
|
||||
|
||||
The decisive move, in tevador's own words (devlog): *"I remembered SuperscalarHash, which is a part of RandomX that's used only to initialize the DAG… a lightweight version of RandomX with only integer operations and no memory accesses."* HashX is a refactored, hardened SuperscalarHash: a faster generator (250 µs → 50 µs), reciprocal multiplication removed, a better JIT, and — the anti-GPU addition — input-dependent branches. The **per-instance random program** is the Family D signature: instead of a fixed compression function an attacker can bake into silicon, every challenge is a *new* function that must be (re)compiled, which is what binds the work to a general-purpose CPU.
|
||||
|
||||
What Equi-X inherits from this parent: CPU-friendliness, GPU/FPGA resistance via per-instance code + a dependent multiply chain + a divergent branch, and the JIT machinery. What it deliberately *sheds*: RandomX's heavy memory and slow verification (Section 5).
|
||||
|
||||
---
|
||||
|
||||
## 3. Parent #2 — Family B, the asymmetric birthday-search shell
|
||||
|
||||
The outer structure comes from Family B (asymmetric / proof-size PoW), whose lineage is:
|
||||
|
||||
```text
|
||||
Momentum (2013) ─► Equihash (2016) ─► Equi-X (Equihash with SUM, not XOR)
|
||||
Cuckoo Cycle (2014) ── asymmetric cousin (different problem)
|
||||
```
|
||||
|
||||
Family B's appeal for a *client puzzle* is exactly its asymmetry: a memory-assisted **generalized-birthday search** to find a solution, but a tiny proof and a near-instant check. Equihash gives "find 2^k hashes that combine to zero," with memory-hardness ∝ 2^(n/(k+1)) and verification of just 2^k hash evaluations. That is precisely the small-proof/fast-verify profile a DoS puzzle needs (requirements #1 and #2).
|
||||
|
||||
Equi-X's one substantive change to this parent is the combiner: **modular addition instead of XOR** (the "k-SUM" variant from Wagner's original paper). Volume II §4.1 and the Research Survey cover *why* (it neutralizes HashX's collision-weak instances and taxes hardware adders); here the point is genealogical — Equi-X is Equihash's structure with a different group operation and a different inner hash.
|
||||
|
||||
---
|
||||
|
||||
## 4. Equi-X as a hybrid — which property comes from which parent
|
||||
|
||||
| Property | Inherited from | Mechanism |
|
||||
|---|---|---|
|
||||
| CPU-binding / GPU-FPGA resistance | **Family D (HashX/RandomX)** | per-instance random integer program; dependent multiply chain; one-shot branch |
|
||||
| Small proof (16 B) + fast verify (50 µs) | **Family B (Equihash)** | asymmetric birthday search; verify = 8 hashes + 7 adds |
|
||||
| Memory-hardness (~1.8 MiB, cache-resident) | **Family B**, tuned | n=60/k=3 sized to fit CPU cache |
|
||||
| Weak-seed safety + hardware adder tax | **Equi-X's own tweak** | 2^k-SUM instead of 2^k-XOR |
|
||||
| ASIC-resistance | **neither — abandoned** | DoS use case + patchability (Section 6) |
|
||||
|
||||
No other deployed PoW occupies this exact intersection. RandomX is Family D without the asymmetric shell (heavy verify). Equihash is Family B without the program-execution core (GPU-friendly). Equi-X is the graft of the two, which is why it is essentially *sui generis* among shipped algorithms.
|
||||
|
||||
---
|
||||
|
||||
## 5. Head-to-head on the axes
|
||||
|
||||
The devlog's own comparison tables, updated against 2026 reality, place Equi-X against its neighbors. (Figures are tevador's, on a Ryzen 1700 / GTX 1660 Ti unless noted; "GPU %" is GPU speed relative to CPU — **lower is better for a client puzzle**.)
|
||||
|
||||
| Algorithm | Family | Memory | Verify | Proof | GPU vs CPU | Niche |
|
||||
|---|---|---|---|---|---|---|
|
||||
| **Equi-X** | B×D hybrid | 1.8 MiB | ~50 µs | 16 B | **<50%** (extrapolated; likely far lower) | DoS puzzle |
|
||||
| RandomX(-Tor) | D | >1 GiB | ~0.5–2 ms | 16 B | ~10% | consensus mining (Monero) |
|
||||
| Equihash (Zcash 200,9) | B | 144 MiB | >150 µs | 1344 B | ~10–13× (GPU **faster**) | consensus mining |
|
||||
| Equihash (BTG 144,5) | B | 2.5 GiB | ~10 µs | 100 B | GPU-mineable | consensus mining |
|
||||
| Argon2(id) | A (KDF) | tunable | slow | — | **~300%** (low-mem) | password hashing |
|
||||
| yespower/yescrypt | A (KDF) | ~2 MiB | slow | — | **~40%** (GPU-*un*friendly) | CPU-only coins |
|
||||
| Cuckaroo / Cuckatoo | B (cousin) | bandwidth-bound | fast | small | GPU / ASIC resp. | consensus mining (Grin) |
|
||||
| ProgPoW / KawPoW | C | DAG (GBs) | moderate | — | GPU-tuned (embraces) | GPU mining (Ravencoin) |
|
||||
|
||||
Reading the rows:
|
||||
|
||||
**vs RandomX (its ancestor).** This is the most illuminating comparison because Equi-X *is* a slimmed RandomX descendant. RandomX optimizes purely for CPU-friendliness and pays for it in verification: >2 GiB and ~2 ms (or 256 MiB and ~15 ms), "way too slow to be used as a client puzzle." RandomX-Tor trimmed it to ~0.5 ms / >1 GiB / ~2000 verif/s — still too heavy (two live seeds ⇒ >2 GiB on the service). Equi-X's leap was to keep the per-instance-program idea but wrap it in Equihash so the *memory and verify collapse to 1.8 MiB / 50 µs* while the GPU resistance is preserved. Track-record footnote: RandomX held CPU-dominance ~4 years before the **Bitmain X5 (2023)** RandomX ASIC appeared — at only ~3× efficiency — vindicating "ASIC-resistant, not ASIC-proof." Equi-X, by contrast, simply doesn't try to be ASIC-proof (Section 6).
|
||||
|
||||
**vs Equihash proper.** Vanilla Equihash is GPU-friendly — GPUs run it up to ~100× a CPU (devlog) — and at consensus scale it has been ASIC'd (Zcash's **Bitmain Antminer Z9**, 2018, ~10–20× a high-end GPU; Zcash *voted against* fighting it and still runs Equihash(200,9)). The cautionary cousin is **Bitcoin Gold's Equihash(144,5)/Zhash**: chosen for GPU-friendly ASIC-resistance, it kept hashrate low and rentable, and was **51%-attacked repeatedly** (~$18M in 2018; ~$70k in 2020). Equi-X inverts Equihash's GPU disadvantage by swapping Blake2b for HashX (so each leaf hash is itself CPU-bound) and shrinking n to 60 so the whole search is cache-resident. It keeps Equihash's *good* part (tiny proof, cheap verify) and removes its *bad* part (GPU dominance) — at the cost of being useless for consensus, which it never wanted to be.
|
||||
|
||||
**vs Argon2 / yespower (Family A KDFs).** tevador tested both and rejected them: both verify slowly, and — per the devlog — both "run faster on GPUs." That is half right by 2026 evidence. **Argon2** genuinely runs ~3× *faster* on a GPU in low-memory configurations (the working set fits GPU caches, so memory-hardness stops binding and core count wins) — a real disqualifier for a CPU puzzle. **yespower**, however, is actually GPU-*unfriendly* (it runs ~2–3× *slower* on GPU and powers CPU-only coins like Yenten and Sugarchain); the devlog's "~200%" appears to be a misattribution of Argon2's number. Either way the verification cost rules them out — KDFs are built to be slow to check, the opposite of what a DoS verifier needs.
|
||||
|
||||
**vs Cuckoo Cycle (Grin).** The asymmetric cousin: also small-proof and fast-verify, but its hardness is *graph-theoretic* (find a cycle) and *bandwidth-bound* rather than program-execution-bound. Grin's history is the instructive contrast in *philosophy*: it split into ASIC-resistant **Cuckaroo** and ASIC-friendly **Cuckatoo** and ran a scheduled 2-year migration from the former to the latter (completed Jan 2021), explicitly *giving up* on perpetual ASIC-resistance ("preventing single-chip ASICs no longer seems worthwhile or feasible"). Equi-X reaches the same conclusion — ASIC-resistance isn't worth chasing — but from the opposite direction: it can afford to ignore ASICs because, unlike a coin, **Tor can re-key with a patch.** Grin, now ASIC-mined and largely dormant, is a quiet warning about what happens when an asymmetric PoW's economic moat erodes.
|
||||
|
||||
**vs ProgPoW / KawPoW (Family C).** The polar opposite design goal: ProgPoW *embraces* GPUs and merely closes the GPU→ASIC gap. It was never activated on Ethereum (which went proof-of-stake at The Merge, Sept 2022) and lives on as Ravencoin's KawPoW. Listed here only to mark the far end of the spectrum: Family C wants GPU mining; Equi-X wants GPU *parity-or-worse*.
|
||||
|
||||
---
|
||||
|
||||
## 6. Why Equi-X's choices are the right ones for its niche
|
||||
|
||||
Read against the landscape, each design decision is a niche-specific optimum rather than a universal one:
|
||||
|
||||
- **n = 60, k = 3 (cache-resident).** Larger n (the rejected n=96 ⇒ ~2 GB) would reuse each HashX instance for 2^25 hashes — long enough for a GPU to compile an optimized per-instance kernel and to exploit its sorting bandwidth. n=60 keeps the working set under ~2 MiB so it lives in CPU cache, "the only case when CPUs can compete with GPUs in memory bandwidth." Small k keeps the proof tiny and verify cheap. This is the Family B knob tuned for the DoS axis.
|
||||
- **2^k-SUM, not XOR.** Neutralizes the collision-weak HashX instances that would otherwise explode into trivial multicollision "solutions," and incidentally taxes FPGA/ASIC adders. A Family-B structure adapted to tolerate a Family-D inner hash that is only preimage-resistant.
|
||||
- **Per-instance program (the RandomX inheritance).** The single most important anti-acceleration lever, and the thing no pure Family-B design has.
|
||||
- **Dropping ASIC-resistance.** Legitimate *only because* of the use case: a DoS target can be patched in hours, so a >$1M ASIC can't be amortized; a consensus coin enjoys no such escape hatch, which is why RandomX, Equihash, Cuckoo, and ProgPoW all had to take ASICs seriously and Equi-X does not.
|
||||
|
||||
---
|
||||
|
||||
## 7. Placement verdict
|
||||
|
||||
On the Acceleration-Resistant Survey's map, Equi-X is best described as **the asymmetric (Family B) shell of Equihash filled with a Family D program-execution core, sized for the DoS corner of the design space** rather than for mining. That corner has very few residents — most asymmetric PoWs are GPU-friendly mining functions, and most program-execution PoWs are heavy mining functions — which makes Equi-X close to unique among *deployed* algorithms: a small-proof, instantly-verifiable, CPU-egalitarian puzzle that explicitly trades away ASIC-resistance for patch-ability.
|
||||
|
||||
Its closest conceptual neighbors are its own family members — RandomX above it (heavier, consensus-grade) and HashWX ahead of it (the GPU-hardened successor, Volume V) — and its closest *functional* role is the one the Survey's DoS-suitability section identifies as the genuinely hard target: a client puzzle that must be **flat across hardware and cheap to verify at scale**, not merely expensive. Equi-X is the most fully-realized answer to that specific problem the field has shipped to date — with the caveat, from Volume III, that its central hardware-flatness claim remains argued rather than empirically proven.
|
||||
@ -0,0 +1,148 @@
|
||||
# Equi-X — Future Directions and Improvements
|
||||
|
||||
### Volume V: the successor primitive (HashWX), the control-loop fix, and the open research agenda
|
||||
|
||||
*Part of the Equi-X implementation series · companion to Volumes I–IV · June 2026 · revised July 2026*
|
||||
|
||||
---
|
||||
|
||||
## 0. Scope
|
||||
|
||||
The first four volumes described Equi-X as it exists: how it works (I), how it's built (II), how well it resists acceleration (III), and where it sits in the landscape (IV). This volume looks forward — **what is already designed to replace or fix parts of it, what is still unverified, and what the prioritized research agenda should be.**
|
||||
|
||||
It draws on first-party material where possible: tevador's HashWX repository and its design document for the successor primitive, the Tor protocol specs and Proposal 362 for the deployment layer, and the recent Equihash-family cryptanalysis. It is consistent with — and extends — the Research Survey's §8 ("Future Directions") and the revised Technical Reference's §10.5 (the control loop). Where a direction is speculative, it is labeled as such.
|
||||
|
||||
The headline: **the primitive is conservative and probably fine; the deployed risk is economic (the effort controller); the designed successor (HashWX) is ready on paper but adopted nowhere; and the most valuable missing work is empirical, not theoretical.**
|
||||
|
||||
---
|
||||
|
||||
## 1. The successor primitive: HashWX
|
||||
|
||||
The most concrete "future direction" already has a repository. **HashWX** is tevador's redesign of HashX, and the HashX README now formally declares itself "superseded by HashWX." It is the natural drop-in upgrade for Equi-X's inner-hash layer.
|
||||
|
||||
### 1.1 What HashWX fixes
|
||||
|
||||
Its design document names five HashX problems — each of which Volumes I–III touched — and addresses every one:
|
||||
|
||||
| # | HashX problem (per HashWX design.md) | Volume cross-ref |
|
||||
|---|---|---|
|
||||
| 1 | Repeated multiplies accumulate trailing zeros ⇒ ~0.2% "weak" instances | II §4.1, the SUM workaround |
|
||||
| 2 | 16 branches at rate 1/16 ⇒ **insufficient GPU divergence** | III §3 |
|
||||
| 3 | **No memory at all** ⇒ GPUs keep VM registers in shared memory | III §3 |
|
||||
| 4 | Instruction set too x86-centric ⇒ poor ARM/WASM performance | II §3.3 |
|
||||
| 5 | Program generation is complex and can *fail* (~1/10⁴ seeds) | II §2.5 |
|
||||
|
||||
### 1.2 How it fixes them
|
||||
|
||||
- **Healing multiplies (problem 1).** Every multiply is fused with an entropy-preserving op: `MULOR` (`(dst|imm)*src`), `MULXOR` (`(dst^imm)*src`), `MULADD` (`(dst+imm)*src`), with odd immediates from {1, 9, 33}. Empirically these cap trailing-zero accumulation at ~4 bits (MULXOR/MULADD) or ~1 (MULOR), versus HashX's runaway zeros. Two **read-only registers R8/R9** preserve input entropy even when R0–R7 degrade (and double as the PRNG's multiplier constants).
|
||||
- **Far more divergence (problem 2).** A HashWX instance is **64 sub-programs, each a loop that repeats with probability 1/2.** A CPU runs each ~2× on average (cheap, and pipeline bubbles fill by running 2 threads/core); a 32-thread GPU warp must run ~6× on average because of divergence. This is the direct answer to HashX's too-weak branch.
|
||||
- **Actual memory (problem 3).** A **2 KB scratchpad**, L1-resident on a CPU (3–4 cycle latency, hideable by scheduling) but forced into higher-latency local/L2 memory (~100 cycles) on a GPU, where it competes with the L1 carve-out.
|
||||
- **Portable instruction set (problem 4).** Only operations expressible in **WebAssembly 1.0** (64-bit mul/add/sub, XOR/OR, rotate/shift), 7-bit immediates for compact x86/ARM/RISC-V encoding, and an **MCG (Lehmer) generator** for branch randomness (one multiply + one rotate per output, constants 3 and 5 mod 8).
|
||||
- **Simpler, infallible generation (problem 5).** A **constant number** of random draws, **no backtracking**, destination registers chosen as a permutation (so every register is written), source-register rules replaced by precomputed permutation lists. Result: **~5× faster generation than HashX, and it never fails** — which removes the `EQUIX_CHALLENGE`/`ProgramConstraints` reject path entirely.
|
||||
|
||||
### 1.3 Status and what adoption would mean
|
||||
|
||||
HashWX is **design-complete but unshipped**: the repo carries reference C, JavaScript, and a **WebAssembly JIT** (`compiler_wasm.c`), a written `specification.md`, and benchmarks (~20,000 cycles to generate, <2,400 cycles/hash compiled, ~10× slower interpreted, on Zen 2). But there is **no tagged release, no audit, and no use in Tor** — still true as of July 2026 (latest repo activity: April 2026, a design-doc fix). WebAssembly reaches ~70% of native speed, and the recommended per-instance reuse drops to **463 attempts** (65,536 in-browser) — versus Equi-X's 2¹⁶ — explicitly to make per-instance compilation dominate on a GPU.
|
||||
|
||||
Folding HashWX into Equi-X would be a **"v2" puzzle**: genuine (and harder-to-doubt) GPU resistance, fixed weak seeds, native cross-platform/WASM execution, and faster verification. But it is not free:
|
||||
|
||||
- It requires a **new `hspow-spec` scheme version** and full re-benchmarking; no such Tor proposal exists.
|
||||
- HashWX is itself **unaudited**, and several of its choices (2 KB scratchpad sizing, the 64×½ branch structure) are *heuristic*, not proven — it trades HashX's known-soft GPU resistance for new, untested machinery.
|
||||
- The infallible generator is a real ergonomic win (no skipped seeds), but changes the Equi-X verifier's error model.
|
||||
|
||||
**Direction:** HashWX is the obvious primitive-level upgrade and the clearest signal that the designer considers HashX's GPU margin too thin. Adopting it should wait on (a) an audit and (b) the empirical hardware study of Section 3 — ideally measuring HashX *and* HashWX on the same GPU to quantify the actual improvement before committing Tor to a v2.
|
||||
|
||||
---
|
||||
|
||||
## 2. The deployment layer: fix the effort controller (highest priority)
|
||||
|
||||
The most important "future direction" is **not** about the primitive at all. Equi-X the function is doing its job; the demonstrated weakness in deployed Tor is the **economic control loop layered on top of it.**
|
||||
|
||||
The OnionFlation attacks (USENIX Security 2025) game Tor's suggested-effort AIMD controller to inflate difficulty for *all* clients at ~$1.20 to trigger and ~$0.10/hour to sustain, with a proven impossibility result (no update algorithm resists both congestion and inflation simultaneously). Tor's answer is **Proposal 362, "Updating the Proof-of-Work Control Loop"** (Aptekar-Cassels; torspec#329) — a time-independent controller that makes a request count the same regardless of when in the update period it arrives, caps both suggested and accepted effort, and adds consensus parameters. As of July 2026 it is **open and unmerged**, so deployed Tor (and Arti) still run the vulnerable loop.
|
||||
|
||||
This is the top of the roadmap for three reasons: it is the *only demonstrated* attack on the deployed system; it is **independent of the primitive** (no HashWX migration needed); and the fix is already drafted but needs to **land and then receive independent scrutiny** — the redesign itself has had none. See the Research Survey §4/§8 and the Technical Reference §10.5; Volume III §6 explains why this, not a GPU solver, is how an attacker actually takes down a service today.
|
||||
|
||||
---
|
||||
|
||||
## 3. Empirical hardware validation (the biggest unverified assumption)
|
||||
|
||||
Volume III's verdict: the CPU-friendliness claim is **argued, not measured.** No public GPU, FPGA, or ASIC solver for Equi-X or HashX exists; the headline "<50% of CPU on GPU" is an extrapolation from RandomX (conservative — real RandomX GPU performance is ~6–10% of CPU — but still an extrapolation), and the comparison table literally lists Equi-X's GPU figure as "?".
|
||||
|
||||
The single highest-value research project is therefore to **build adversarial GPU and FPGA solvers and benchmark them.** Concretely:
|
||||
|
||||
- A CUDA/OpenCL Equi-X solver that JITs a kernel per challenge and measures real GPU-vs-CPU throughput, including the warp-divergence cost of the one-shot branch and the bandwidth cost of the cache-vs-local-memory sort.
|
||||
- The same for HashX in isolation, and for **HashWX**, so the GPU-resistance *improvement* of the successor can be quantified before Tor commits to a v2 (Section 1.3).
|
||||
- An FPGA feasibility study: soft-core vs per-challenge reconfiguration, and the real LUT cost of the modular-SUM adders vs XOR.
|
||||
|
||||
This would convert the design's central claim from "well-reasoned" to "demonstrated" (or refute it) and is the prerequisite for every primitive-level decision below.
|
||||
|
||||
---
|
||||
|
||||
## 4. Independent cryptanalysis
|
||||
|
||||
HashX's security rests on (a) Equihash/Wagner birthday hardness and (b) HashX's *claimed* preimage resistance — and **(b) has never been independently cryptanalyzed.** Open theoretical questions:
|
||||
|
||||
- **HashX/HashWX preimage security and weak-seed density.** The ~0.2% weak-instance figure is the author's; the SUM-binding argument that makes a non-collision-resistant inner hash safe has never been refereed.
|
||||
- **How much Equihash-family cryptanalysis transfers.** Recent work is pointed but aimed elsewhere: Alcock–Ren (CCSW 2017) showed Equihash has *no proven tradeoff bound*; ePrint 2025/2141 gives a ~50%-memory / ~2×-time Wagner tradeoff plus an ASIC-friendly external-memory framework; ePrint 2025/1351 argues the **index-pointer technique weakens Equihash's ASIC-resistance** and proposes **Requihash**, explicitly extending to the **k-SUM** variant Equi-X uses. The honest assessment (Volumes III–IV): these mostly target large-n, XOR, ASIC-scale Equihash, and transfer *weakly* to Equi-X's tiny cache-resident SUM instance that abandons ASIC-resistance anyway — but the family's foundations are clearly under active, skeptical study, and **Requihash is worth evaluating as an alternative shell** if a v2 is ever opened.
|
||||
|
||||
**Direction:** solicit refereed analysis of HashX/HashWX specifically (not just the Equihash shell), and track the Tang–Sun–Gong line of work for any small-n / SUM result that *does* transfer.
|
||||
|
||||
---
|
||||
|
||||
## 5. Parameter retuning as hardware evolves
|
||||
|
||||
Equi-X's parameters are **patchable but not runtime-tunable**, and several are pinned to *today's* hardware:
|
||||
|
||||
- **n, k and the cache-resident working set.** The whole GPU-resistance argument depends on the ~1.8 MiB sort fitting in CPU L2/L3. As cache sizes grow and as **unified-memory architectures** (the Acceleration-Resistant Survey §12) erode the CPU-vs-GPU bandwidth gap, the "only case where CPUs compete on bandwidth" premise weakens, and n may need revisiting.
|
||||
- **Instance lifetime.** 2¹⁶ hashes per HashX instance (vs HashWX's 463) is a CPU-vs-GPU-amortization knob; wider SIMD (AVX-512) or cheaper GPU JIT could shift the optimum.
|
||||
- **Pipeline assumptions.** The generator models a ~12-year-old 3-port Ivy-Bridge core (Volume II §2.3); HashWX's design doc explicitly criticizes this as outdated. A retune toward modern issue widths is part of the HashWX rationale.
|
||||
|
||||
None of these are runtime parameters, so any change is a spec revision + re-benchmark — manageable for Tor (patchability again), but a reason to keep the empirical study (Section 3) running as hardware moves.
|
||||
|
||||
---
|
||||
|
||||
## 6. Quantum and longer-horizon questions
|
||||
|
||||
No Equi-X-specific quantum analysis exists. A first-principles read: Equi-X's binding cost is **memory bandwidth plus program execution, not hash inversion**, so Grover-style speedups on the inner hash are likely secondary; the more relevant question is quantum algorithms for the generalized-birthday / k-list problem, which is speculative and not obviously threatening at n=60. This is **low priority** for a patchable DoS puzzle, but worth a paragraph in any long-lived adopter's threat model — and a sharp reminder (Volume IV) that **any consensus system tempted to reuse Equi-X would inherit real ASIC risk**, since Equi-X drops ASIC-resistance by design and leans entirely on Tor's patch-ability assumption.
|
||||
|
||||
---
|
||||
|
||||
## 7. Broader adoption and the WASM/CAPTCHA bet
|
||||
|
||||
HashWX is explicitly aimed at a market Equi-X never entered: **browser CAPTCHA-style client puzzles**, via its WebAssembly build. The opportunity is real — today's PoW-CAPTCHA ecosystem (Anubis, ALTCHA, Cap, Friendly Captcha) still leans on Hashcash-style static SHA-256 or KDFs, all of which inherit the GPU-offload weakness HashX/HashWX were built to remove. Tellingly, **Anubis evaluated HashX/Equi-X and declined** — partly because the Rust `equix` crate runs interpreter-only under WebAssembly, handing native-compiled attackers an edge. **That is exactly the gap HashWX's WASM JIT closes**, which is why HashWX's CAPTCHA framing is the designer's bid to enter this space. As of July 2026 no CAPTCHA product has adopted it, and Equi-X adoption beyond Tor remains essentially nil.
|
||||
|
||||
**Direction:** a released, audited HashWX with a maintained WASM package is the precondition for any non-Tor adoption; the browser-CAPTCHA niche is the most plausible second home.
|
||||
|
||||
---
|
||||
|
||||
## 8. Concrete implementation improvements
|
||||
|
||||
Smaller, code-level items surfaced by Volumes I–II:
|
||||
|
||||
- **The bit-packed ~1 MiB solver heap.** The devlog notes the 1.81 MiB heap compresses to ~1 MiB with bit-packing at ~25% discarded solutions — "viable only for custom hardware." Not worth it for the CPU reference, but relevant to any hardware-resistance study (Section 3).
|
||||
- **Retire the vestigial wide-multiply PRNG draw.** Both implementations consume an otherwise-unused `u32` per wide multiply purely for stream-compatibility (Volume II §2.4); a clean v2 could remove it — but *only* in a compatibility-breaking revision, alongside HashWX.
|
||||
- **Shared, expanded conformance vectors.** The Rust crates already pin reference streams at every layer; a primitive-independent, cross-implementation conformance suite (the Volume II §7 checklist as runnable tests) would de-risk any third implementation and any v2 migration.
|
||||
- **SIMD-batched verification** for services checking many proofs, where throughput (not single-proof latency) matters.
|
||||
|
||||
---
|
||||
|
||||
## 9. A prioritized roadmap
|
||||
|
||||
| Priority | Direction | Layer | Status / dependency |
|
||||
|---|---|---|---|
|
||||
| **1** | Land **Proposal 362** and have the new control loop independently reviewed | Protocol | drafted, open, unmerged; the only *demonstrated* deployed weakness (§2) |
|
||||
| **2** | Build & benchmark a **GPU/FPGA solver** for Equi-X, HashX, HashWX | Empirical | none exists; gates every primitive decision (§3) |
|
||||
| **3** | **Independent cryptanalysis** of HashX/HashWX; track Requihash & k-SUM results | Theory | never refereed (§4) |
|
||||
| **4** | Audit HashWX, then design an **Equi-X v2** on it (new `hspow-spec`) | Primitive | depends on #2, #3; no proposal yet (§1) |
|
||||
| **5** | **Retune parameters** for modern caches / unified memory / wide SIMD | Maintenance | patch-only; ongoing as hardware moves (§5) |
|
||||
| **6** | **Quantum** threat-model note; explicit warning for consensus reuse | Theory | low priority (§6) |
|
||||
|
||||
The ordering reflects a single principle: **fix what is demonstrably broken (the control loop) first, measure what is merely assumed (hardware resistance) second, and only then change the primitive.**
|
||||
|
||||
---
|
||||
|
||||
## 10. Synthesis
|
||||
|
||||
Equi-X's future is unusually legible because so much of it is already written down. The deployed system's real risk is economic and has a drafted fix awaiting deployment and review (Proposal 362). The primitive's headline property — GPU/FPGA flatness — is well-argued but unmeasured, and the person best placed to judge it has already built a hardened successor (HashWX) that fixes HashX's acknowledged soft spots and targets the browser-CAPTCHA market HashX couldn't serve. The cryptographic foundations are quietly contested at the Equihash-family level, though the critiques transfer weakly to Equi-X's specific, ASIC-indifferent parameter choices.
|
||||
|
||||
The throughline across all five volumes is that **Equi-X's safety has always rested less on any single primitive being unbreakable than on a use case that lets it be patched.** That same property defines its future: every improvement here — a control-loop fix, a measured GPU verdict, a HashWX-based v2, a parameter retune — is something Tor can ship as an update, exactly the freedom a consensus blockchain never has. The work that remains is real, but none of it is existential, and most of it is already on someone's bench.
|
||||
Loading…
x
Reference in New Issue
Block a user