logos-delivery/docs/analysis/async_copy_analysis.md

Async deep-copy analysis — chronos, Option/Opt/Result, WakuMessage path

Environment: Nim 2.2.4, --mm:refc, chronos 4.2.2 (nimbledeps/pkgs2/chronos-4.2.2-*), nim-results 0.5.1, nim-libp2p 2.0.0. All findings verified against these sources.

Key refc facts driving everything below:

• Assignment of string/seq/object-with-seqs = deep copy (copyString/genericSeqAssign). No =sink hooks.
• system.move() degrades to copy + reset under refc (move semantics require ARC/ORC).
• lent returns work under refc (true borrow, no copy at return) — the copy materializes when the borrow is bound to a let/var or passed by value.

---

Table 1 — Where chronos generates deep copies

#	Copy point	Mechanism (evidence)	refc	ORC
1	Every {.async.} call: each seq/string/object param	async transform moves body into a closure iterator (chronos/internal/asyncmacro.nim:445-517); params are lambda-lifted into the env object → env.param = param = genericSeqAssign	1 deep copy per param per call	move/cursor — free
2	complete(future, val)	val: T is not sink (asyncfutures.nim:198-202); future.internalValue = val. Macro calls complete(fut, move(result)) (asyncmacro.nim:160) but move is a copy under refc	1–2 deep copies per completion	1 copy (non-sink param survives ORC)
3	Consuming the result: let x = await f(...)	Future.value() returns lent T (futures.nim:231-241) — free at return, but binding the borrow to x copies	1 deep copy per await	1 copy (can't move out of a borrow)
4	Callbacks / futureContinue	future passed by ref, callbacks moved (asyncfutures.nim:187)	none	none
5	{.async: (raw: true).} procs	no closure-iterator transform → no param capture	none	none

Net: let x = await f(a, b) where a, b, and the result are seq-heavy costs ~2 copies of each argument-side seq + 2–3 copies of the result under refc. Rows 2–3 are chronos API design (non-sink complete, lent read) and persist under ORC; row 1 is refc-specific.

libp2p is aware of row 1: pubsub.nim:415-427 uses {.async: (raw: true).} for handleData "without copying data into closure" — but every downstream TopicHandler is a normal {.async.} taking (topic: string, data: seq[byte]) by value, so the copy fires there, once per registered handler.

---

Table 2 — Option[T], Opt[T], Result[T, E] (results 0.5.1, resultsLent=true on Nim 2.2.4)

Accessor	Kind / returns	Copies under refc?	Evidence (results.nim)
ok(x) / err(x)	template, field-construct	copy of x into Result (move only if rvalue construction elides)	:451-454, :471-474
value(), get(), tryGet(), expect()	lent T (or template → lent)	none at return; deep copy when bound to let	:870-879, :1072, :1078, :952
value()/get() on var Result	var T borrow	none	:881-889, :1092
unsafeGet()/unsafeValue()	template, in-place field access	none in place; copy on bind	:902-905, :1084
get(otherwise)	func returning T by value	always copies (plus otherwise passed by value)	:1095
valueOr:	template: let s = (self) then yields field	copies whole Result if self is lvalue (rvalue temp is sunk); value deep-copied on the outer let bind	:1396-1432
isOkOr:	template: let s = (self)	copies Result (upstream # TODO avoid copy)	~:1355
? operator	template: let v = (self)	copies Result; value copied on bind	:1533-1544
std/options some(sink T)	proc, sink param	move if rvalue, copy if lvalue	options.nim:125
std/options get()	lent T	none at return; copy on bind	options.nim:191
std/options get(otherwise)	proc returning T	always copies	options.nim:206

Rule of thumb: the wrappers themselves are fine; the cost is one full deep copy each time a large value is bound out (let msg = decode(...).valueOr: ... = copy), and valueOr/isOkOr/? additionally copy the whole Result when applied to an lvalue.

---

WakuMessage path — hop-by-hop copy count

WakuMessage (message.nim:12-29) is a plain object with three heap seqs (payload, meta, proof) plus contentTopic: string — every deep copy touches ~4 heap allocations.

A. Inbound relay — the same proto bytes are decoded ≥4×

Hop	Site	Operation	Cost
A1	protocol.nim:543	WakuMessage.decode(message.data) in ordered validator	decode alloc + valueOr bind copy
A2	protocol.nim:271	decode again in onRecv observer	same
A3	protocol.nim:326	decode again in onValidated observer	same
A5	protocol.nim:603	decode again in subscribe topicHandler	same
(out)	protocol.nim:341	decode in onSend observer (outbound leg)	same

Each decode = fresh payload/meta/proof seqs (codec.nim:23) + ok() wrap + valueOr bind ≈ 2 avoidable full-message copies per decode site on top of the one unavoidable materialization.

B. Node dispatch — 6+ async closure captures of the decoded message

registerRelayHandler (subscription_manager.nim:29) builds uniqueTopicHandler (:72, {.async.}, msg by value → env capture copy #1), which sequentially awaits trace(:45), filter(:51), archive(:57), sync(:63), internal(:69), legacyApp(:82) handlers — each a {.async.} proc taking msg: WakuMessage by value → 6 more env-capture deep copies, plus MessageSeenEvent.emit(...msg) (:70) broker copy.

C–F. Downstream

Path	Sites	Copies
Archive	archive.nim:98,101	async capture + computeMessageHash (re-hash) + field copies into SQLite params; queue_driver retains a full WakuMessage copy in SortedSet[Index, WakuMessage]
Filter push	protocol.nim:243-264,213,216,170	async capture + re-hash ×2 + MessagePush(wakuMessage: message) wrap + one encode (good: not per-peer) + per-peer buffer env-capture copy in pushToPeer
Lightpush inbound	protocol.nim:84,67, callbacks.nim:16-24	decode + hash + validateMessage re-encodes + re-hashes + relay publish re-encodes
Outbound publish	relay/protocol.nim:674-690	explicit var message = wakuMessage copy (:680, for timestamp mutation) + encode + hash
FFI/JSON boundary	library/json_message_event.nim:72-95, relay_api.nim:115-121	explicit copyMem of payload/meta/proof + base64 encode/decode string copies (inherent to the JSON API)

G. computeMessageHash — recomputed 4–6× per message

Call sites for the same inbound message: relay :217/validator, archive :101, filter :246 + :195, store-sync reconciliation :78 — plus lightpush/API paths. The hash itself doesn't copy (params feed openArray into sha256), but it's ~4–6 redundant SHA-256 passes over the full payload per message.

Copy budget (one inbound relayed message, archive + filter + sync active, F filter peers)

Category	Full-payload copies
Redundant proto decodes (A)	~8 (4 sites × ~2)
Dispatch env captures (B)	7
Archive + filter + push wrap/encode	~4 + F (per filter peer)
Total	~19 + F, vs. theoretical minimum ≈ 3 (decode once, hash once, encode once)

Plus 4–6 redundant SHA-256 passes. For a 1 MB payload that is ~20 MB of allocation + memcpy + refc GC pressure per message.

---

Recommendations (prioritized)

Yes, this is worth fixing — the relay hot path does ~6× more copying than necessary, and it's all in per-message code.

1. P1 — Decode once in WakuRelay (waku_relay/protocol.nim). The validator (A1), both observers (A2/A3), and the topicHandler (A5) each decode independently. Decode in the ordered validator, cache (msgId → decoded msg + hash) in a small TimedCache (the gossipsub validator→handler API gives no side channel), or restructure so observers receive the already-decoded message. Removes ~6 full copies + 2–3 hash passes per message. No public API change.

2. P2 — Pass a ref envelope through internal dispatch. Introduce e.g. MessageEnvelope = ref object holding msg: WakuMessage, hash: WakuMessageHash, pubsubTopic, created once after validation; change WakuRelayHandler, subscription_manager handlers, archive.handleMessage, filter.handleMessage, and sync ingress to take the ref. Under refc a ref copy is pointer + refcount — kills all 7 env-capture copies (Table 1 row 1) and all downstream re-hashing (compute once, carry it). This is the single highest-leverage internal refactor. (Immutable-by-convention; document that handlers must not mutate.)

3. P3 — Filter pushToPeer: share the encoded buffer. The buffer is already encoded once, but the {.async.} by-value param copies it per peer (protocol.nim:170,216). Pass a ref seq[byte] (or make pushToPeer {.async: (raw: true).}).

4. P4 — Don't return large values from async procs. Each such return costs 2–3 copies (Table 1 rows 2–3) that even ORC won't remove. Return ref T, or fill a caller-provided buffer. Applies to store query results (seq[WakuMessage]) and codec-heavy client calls.

5. P5 — Result/Option idioms for large T. Avoid get(otherwise) (always copies both sides); prefer in-place access (.value.field, var-borrow get) where the value isn't stored anyway. Low priority once P2 makes messages refs — Opt/Result over ref or small types is free.

6. P6 — Upstream / longer term.

   • chronos: make complete() take sink T (removes Table 1 row 2 under ORC; today it copies under every mm).
   • ORC migration removes row 1 (env captures become moves) — the biggest class of copies — but rows 2–3 remain, so P2/P4 stay valuable regardless.
   • The explicit var message = wakuMessage copy at publish (protocol.nim:680) exists only to patch the timestamp — set the timestamp before entering publish and take the param immutably.

Gains and effort estimate

Baseline per inbound relayed message (archive + filter + sync active, F filter peers): ~19 + F full-payload copies, 4–6 SHA-256 passes.

Fix	Copies removed	Hash passes removed	Remaining after fix	Effort (dev-days)	Risk / blast radius
P1 decode-once in relay	~6 (3 redundant decode sites × 2)	2–3	~13 + F	2–3 (incl. tests)	Contained to waku_relay/protocol.nim; needs msgId→decoded cache or observer restructure
P2 ref envelope through dispatch	~9 (7 env captures + archive/filter captures)	1–2 (hash carried)	~4 + F	3–5	Internal API change: WakuRelayHandler, subscription_manager, archive, filter, sync + their tests
P3 shared filter buffer	F (per-peer buffer capture)	—	~4	0.5	One proc signature in waku_filter_v2
P4 no large async returns (store/query hot paths)	2–3 per large-value await	—	—	2–3 (hot paths only)	Store/query client APIs
P5 Result/Option idioms	marginal once P2 lands	—	—	opportunistic	none
P6 chronos sink complete / ORC	env-capture class (row 1)	—	—	weeks (upstream cycle) / months (ORC)	out of repo control

After P1+P2+P3: ~4 full-payload copies and 1–2 hash passes per message — a ~75–80 % reduction in per-message memcpy/alloc volume and ~70 % less hashing CPU. The remaining ~4 are near the floor: proto→WakuMessage materialization, envelope fill, outbound encode, DB row.

Concrete scale for the target payload profile (min 10 kB / avg 50 kB / max 150 kB). Assumptions: copies 20 → 4 per message (F≈1), hash passes 5 → 1.5; effective alloc+memcpy throughput under refc ≈ 1–2 GB/s per core (allocation-bound, not memcpy-bound); SHA-256 via nimcrypto (pure Nim, no SHA-NI) ≈ 300–500 MB/s per core.

Per-message volume:

Payload	Copied (baseline 20×)	Copied (after 4×)	Hashed (5×)	Hashed (1.5×)	Allocs/msg
10 kB	200 kB	40 kB	50 kB	15 kB	~60–80 → ~12–16
50 kB	1.0 MB	200 kB	250 kB	75 kB	"
150 kB	3.0 MB	600 kB	750 kB	225 kB	"

Sustained-rate CPU cost (share of one core spent purely on copy+hash overhead):

Scenario	Baseline	After P1–P3
50 kB @ 100 msg/s	~10–18 %	~3–5 %
50 kB @ 500 msg/s	~50–90 % (near saturation)	~13–22 %
150 kB @ 100 msg/s	~30–55 %	~8–14 %
10 kB @ 1000 msg/s	~20–37 %	~5–9 %

Single-core throughput ceiling from overhead alone (copy+hash time per message):

Payload	Baseline ceiling	After P1–P3	Headroom gain
10 kB	~2,700–5,000 msg/s	~11,000–20,000 msg/s	~4×
50 kB	~550–1,000 msg/s	~2,200–4,000 msg/s	~4×
150 kB	~180–330 msg/s	~740–1,300 msg/s	~4×

GC pressure: heap churn equals the copied volume — at 50 kB × 100 msg/s the baseline churns ~100 MB/s through the refc heap (that much garbage accumulates between collection cycles); after the fixes ~20 MB/s. Fewer, larger GC cycles → visibly lower tail latency. Per-message transient footprint at 150 kB drops from ~3 MB to ~0.6 MB.

These are arithmetic projections, not measurements — the day-one benchmark harness validates them (and nimcrypto's actual SHA-256 rate, which sets how much of the win comes from P2's hash-once).

Suggested order: benchmark harness first (0.5–1 day: fixed-seed message generator through uniqueTopicHandler, measure allocs via getOccupiedMem/instrumented computeMessageHash counter) → P3 → P1 → P2 → re-benchmark. Total ≈ 1.5–2 calendar weeks for P1–P3 including tests and verification; P4 as follow-up when touching store paths.

Memory-model note (refc vs ORC)

Copy class	refc	ORC	Fix
async param → closure env	deep copy	elided (move/cursor)	P2 (ref) now; ORC later
complete(fut, val)	1–2 copies (move degrades)	1 copy (non-sink)	P4 / upstream sink
await result bind	copy	copy (lent borrow)	P4
valueOr/?/bind-out of Result	copy	usually moved (rvalue)	P5