"token" was overloaded (auth tokens, cgo handles, lexer tokens) and didn't say
what it is — a per-call correlation id linking an outgoing {.ffiHost.} call to
the answer that arrives later (possibly from another thread). Renamed across the
runtime (ffi_host / ffi_context), the macro, the exported C ABI (FFIHostFn,
<lib>_host_complete), the Go trampoline, and the tests; regenerated bindings.
The unrelated request-path cgo.Handle result-slot (also informally called a
"token" in go.nim comments) is left as-is — different mechanism.
16 host unit tests + the examples/host_demo Go round-trip stay green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Increment 3: the {.ffiHost.} pragma. A bodyless
proc fetchToken(key: string): Future[Result[string, string]] {.ffiHost.}
expands into an async proc that resolves the thread-local host registry +
pending table, looks the fn up by snake_case wire name, allocates a token,
invokes the host with the raw request bytes, and awaits the answer.
This is the inverse of {.ffi.} and the first end-to-end use of the registry
(increment 1) + completion bridge (increment 2). First slice is deliberately
narrow — raw ABI, one string param, Future[Result[string, string]] — to prove
the round-trip with zero serialization; struct params/returns and the
{.ffiHost: cbor.} format arg are follow-ups.
The body reads two new threadvars (ffiCurrentHostRegistry / ffiCurrentPendingTable)
set by ffiThreadBody alongside ffiCurrentEventRegistry, so the user's signature
stays ctx-free. The host fn is invoked synchronously before the await, while the
string arg is still alive (honouring the "req valid only for the call" contract).
5 macro tests pass under orc+refc; host + ffi_context suites stay green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Increment 2: wires the host-call machinery into the running FFI thread so a
host answer (delivered from any thread) resolves the chronos Future an awaiting
handler is blocked on.
- FFICompletionQueue (ffi_host.nim): a GC-free intrusive queue. host_complete
pushes c_malloc'd nodes from any thread; the FFI thread drains, copies the
payload into GC memory, completes the future by token, and frees the node.
- FFIContext gains hostRegistry / pendingTable / completionQueue, init'd and
deinit'd alongside the event registry.
- completeHostCall parks the answer and fires the EXISTING reqSignal — no second
ThreadSignalPtr needed; the loop drains completions every iteration, on the
loop thread (chronos single-thread invariant).
- On shutdown the loop failAllPendings first, so a handler awaiting a host
answer that never arrives can't hang the allFutures(pending) drain.
4 new queue unit tests (10 total) pass under orc+refc; the 19 ffi_context
integration tests stay green.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* protect against mem leak in case of failures sending requests to ffi thread
* better cleanup if failures in createFFIContext
* avoid dangling cstring in handleRes under ARC/ORC
* better resource cleanup in destroyFFIContext
* invoke onNotResponding if failure in destroyFFIContext
* correct seq copy in alloc
* make sure the lock is init before cleanUpResources
* better possible exception handling in processReq
* guard allocSharedSeq if given seq is empty
* enhance error handling in ffi_context
* add new tests and some corrections
