# nim-ffi Expose a Nim library to C, C++ and Rust by annotating ordinary Nim procs. You write async Nim; `nim-ffi` provides the whole FFI runtime — a dedicated worker thread, a request channel, CBOR (de)serialization, an event queue and a context/handle registry — and generates the foreign-language bindings for you. No hand-written `.h` files, no manual request enums, no shared-memory plumbing. ## Install Add nim-ffi to your library's `.nimble`, then `import ffi`: ```nim requires "https://github.com/logos-messaging/nim-ffi >= 0.3.0" ``` ## Mental model - You **declare a library** once with `declareLibrary(name, LibType)`. - You **annotate procs and types** with pragmas (`{.ffi.}`, `{.ffiCtor.}`, `{.ffiDtor.}`, `{.ffiStatic.}`, `{.ffiEvent.}`). - You **call `genBindings()` last**, which emits the foreign bindings. By default, every request/response crosses the boundary as a single CBOR blob (the wire format is configurable per library or per annotation — see [ABI format](#abi-format)); the ctx handle returned by the constructor is the only pointer that crosses. Each `{.ffi.}` proc runs on the library's own chronos event loop, so bodies can `await` freely. ## Minimal example ```nim import ffi, chronos # 1. The library's main state. The FFI context owns one instance. type Counter = object value: int declareLibrary("counter", Counter) # 2. Request/response shapes. Any {.ffi.} object or enum type becomes a # first-class struct/class in the generated bindings and rides the wire in # the library's ABI format (CBOR by default). type BumpRequest {.ffi.} = object by: int type BumpResponse {.ffi.} = object newValue: int # 3. Constructor: returns Future[Result[LibType, string]]. proc counterNew*(): Future[Result[Counter, string]] {.ffiCtor.} = return ok(Counter(value: 0)) # 4. A method: first param is the library value, then any typed params. # Return type is always Future[Result[T, string]]. proc counterBump*( c: Counter, req: BumpRequest ): Future[Result[BumpResponse, string]] {.ffi.} = await sleepAsync(1.milliseconds) return ok(BumpResponse(newValue: c.value + req.by)) # 5. Destructor: exactly one param (the library value). proc counterDestroy*(c: Counter) {.ffiDtor.} = discard # 6. genBindings() must be the LAST FFI call in the file (see below). genBindings() ``` The generated C export names are the snake_case form of the proc names, e.g. `counterBump` → `counter_bump`. ## Pragma reference | Pragma | Applies to | Purpose | | --- | --- | --- | | `declareLibrary(name, LibType[, defaultABIFormat])` | call | Registers the library, its state type, and the default wire format. Must run before any annotation. | | `{.ffi.}` on a `type` | `object`, `enum` | Registers the type for binding generation; it serializes via the library's ABI format (CBOR by default). Enums are CBOR-only — see below. | | `{.ffi.}` on a `proc` | proc | Exposes a method. First param is the library value, then typed params; returns `Future[Result[T, string]]`. | | `{.ffiStatic.}` | proc | Exposes a context-independent proc: no library param, and its wrapper takes no ctx — see below. | | `{.ffiCtor.}` | proc | The constructor. Returns `Future[Result[LibType, string]]`; creates the FFI context. | | `{.ffiDtor.}` | proc | The destructor. Exactly one param `(x: LibType)`; tears the context down. | | `{.ffiEvent[: "wire_name"].}` | proc (empty body) | A library-initiated callback. Call the proc from any `{.ffi.}` handler to fire it. The wire name is optional — see below. | | `{.ffiHandle.}` | `ref object` | Marks a type as an opaque handle: it stays server-side and crosses the wire as a `uint64` id. | | `{.ffiConst.}` | `const` | Re-emits the value as a native constant in every generated binding — see below. | | `genBindings()` | call | Emits the bindings. Must be the **last** FFI call in the compilation root. | ### Enums `{.ffi.}` on an `enum` gives each language its own native enum: ```nim type Level {.ffi.} = enum lLow = "low" lHigh = "high" ``` ```c typedef enum { LEVEL_L_LOW = 0, LEVEL_L_HIGH = 1 } Level; /* C */ enum class Level { lLow = 0, lHigh = 1 }; // C++ pub enum Level { #[serde(rename = "low")] LLow, ... } // Rust ``` An enum crosses the wire as **text**, not as its ordinal — whatever `$value` yields, so the associated string if the enum declares one (`"low"` above) and the symbol name otherwise (`lLow`). That is what `cbor_serialization` writes on the Nim side, and the generated codecs map name ↔ value on the far side, so reordering or renumbering values doesn't break an already-deployed peer. Explicit ordinals are carried into the foreign enum so the two sides agree if you ever cast. Enums are supported on the CBOR wire only. Reaching one from an `abi = c` type or proc is a compile error naming the type — the `abi = c` `_CWire` structs have no enum form yet. ### Constants `{.ffiConst.}` copies a Nim `const` into the generated bindings, so callers don't hand-maintain a second copy of a limit, a default or a protocol string: ```nim const MaxPeers* {.ffiConst.} = 42 DefaultTimeoutMs* {.ffiConst.}: uint32 = 3 * 1000 Greeting* {.ffiConst.} = "hello" ``` ```c static const int64_t MAX_PEERS = 42LL; /* C */ static const uint32_t DEFAULT_TIMEOUT_MS = 3000; static const char* const GREETING = "hello"; ``` ```cpp constexpr int64_t MAX_PEERS = 42LL; // C++ ``` ```rust pub const MAX_PEERS: i64 = 42; // Rust ``` Integer, float, `bool` and `string` consts are supported; anything else is a compile error. The value is whatever the const evaluates to, so computed expressions arrive folded. Names are re-cased to `UPPER_SNAKE`, preserving acronyms (`httpTTL` → `HTTP_TTL`). A constant is a compile-time value in each language, not a symbol exported by the shared library — it never crosses the wire, so `{.ffiConst.}` is ABI-agnostic. ### Doc comments A `##` doc comment on an annotated proc is carried through to every generated binding, so the exported API is documented once, at the source: ```nim proc myTimerEcho*( timer: MyTimer, req: EchoRequest ): Future[Result[EchoResponse, string]] {.ffi.} = ## Sleeps `delayMs` then echoes the message back. ... ``` becomes ``/** Sleeps `delayMs` then echoes the message back. */`` above both the exported symbol and the `_ctx_echo` wrapper in the C header, `/// ...` on the C++ class method and the Rust `pub fn`, and a `;` comment in the CDDL schema. Multi-line doc comments are preserved as-is. Only `##` doc comments are propagated, and only on procs — a plain `#` comment above the proc, and any comment on a `{.ffi.}` type or its fields, is dropped by Nim's parser before the macro can see it. ### The return-type contract Every `{.ffi.}` / `{.ffiCtor.}` proc must have an explicit `Future[Result[T, string]]` return type — even for synchronous logic (just `return ok(...)` without awaiting). The `Result`'s error string is delivered to the foreign caller as the failure message. ### Context-independent procs A `{.ffi.}` proc is a method: it takes the library value, and its wrapper takes a `ctx`, so the host must construct the library to call it. A stateless utility shouldn't have to pay for that. Annotate it `{.ffiStatic.}` instead — drop the library param, and the ctx disappears from the generated wrapper: ```nim proc counterParse*(text: string): Future[Result[BumpRequest, string]] {.ffiStatic.} = return ok(BumpRequest(by: text.parseInt())) ``` ```c /* {.ffi.} method */ counter_ctx_bump(ctx, &req, on_reply, ud); /* {.ffiStatic.} — no ctx */ counter_static_parse(text, on_reply, ud); ``` The wrapper is `_static_`, not `_`, for the same reason a method's is `_ctx_`: `_` is the raw symbol the dylib exports. In C++ and Rust a static is an associated function on the ctx type (`EchoCtx::lib_version()`), taking the `timeout` a method reads from its ctx. The handler still needs an FFI thread, so it runs on the library's **static context**: created on the first `{.ffiStatic.}` call, then alive for the rest of the process — no ctx owns it, so nothing tears its thread pair down, and it holds one of the pool's slots. It has no `myLib`, which is why a static proc cannot take the library value. There is no foreign teardown for it. From Nim, `destroyStaticFFIContext(pool)` stops the thread pair and frees the slot; it is only sound once nothing will call a `{.ffiStatic.}` proc again, so it is meant for process shutdown and tests. The macro rejects an `{.ffiHandle.}` parameter or return: a handle is registered in the context that created it, which a static proc cannot reach. Under `abi = c` a static replies with a `string` or an `{.ffi.}` object type — a scalar return is wired only for an all-scalar `{.ffi.}` method, which rides the [CBOR-free fast path](#abi-format) through the ctx a static doesn't have. ### The result callback contract Each request carries a result callback. It receives one of these status codes (`ret` / `err_code`): | Code | Value | Terminal? | Meaning | | --- | --- | --- | --- | | `RET_OK` | 0 | yes | Success; the payload carries the encoded result. | | `RET_ERR` | 1 | yes | Failure; the payload carries the UTF-8 error string. | | `RET_MISSING_CALLBACK` | 2 | — | No callback was passed; the request path reports this itself. | | `RET_STALE_WARN` | 3 | **no** | Progress ping — the handler is still running. | **nim-ffi never times a handler out.** A slow request runs to its natural `RET_OK` / `RET_ERR`; it is never cancelled (a hard-cancel mid-call into the underlying library can leave it half-applied). Instead, while a handler is still in flight the callback receives a **non-terminal** `RET_STALE_WARN` every 5s (Android's ANR interval; override at build time with `-d:ffiStaleWarnIntervalMs=`), with the payload carrying the elapsed milliseconds as a decimal string. The dev decides what to do with a slow request — keep waiting, surface a spinner, tear the context down — nim-ffi does not decide for them. `RET_STALE_WARN` may fire any number of times and is **always** followed by exactly one terminal `RET_OK` / `RET_ERR`. A caller that only wants the final answer must ignore it (do not treat a non-zero code as an error without checking for `RET_STALE_WARN` first). The generated higher-level typed wrappers currently ignore it; the progress signal is delivered at the raw result-callback boundary. ### Events An event is a proc with an empty body annotated `{.ffiEvent.}`. You fire it by calling it with a typed payload from inside any `{.ffi.}` handler; the foreign side receives it through a registered callback. ```nim type PeerConnected {.ffi.} = object id: string proc onPeerConnected*(peer: PeerConnected) {.ffiEvent.} # wire name: "on_peer_connected" proc counterBump*(c: Counter, req: BumpRequest): Future[Result[BumpResponse, string]] {.ffi.} = onPeerConnected(PeerConnected(id: "p-1")) return ok(BumpResponse(newValue: c.value + req.by)) ``` The wire name is **optional**: when omitted it is derived from the proc name (`onPeerConnected` → `on_peer_connected`), matching how `{.ffi.}` derives its C export symbol. Pass a string literal (`{.ffiEvent: "custom_name".}`) only when you need a name that differs from the proc. ### ABI format The wire format is chosen **in code**, never by a compile flag. Override the library default with `declareLibrary("lib", Lib, defaultABIFormat = "c")`, or per annotation with an `"abi = ..."` spec, e.g. `{.ffi: "abi = c".}`. The `-d:targetLang` flag (below) picks which *language* the bindings are emitted for; it does not change the wire. `cbor` is the default and fully-supported format: every proc, ctor, dtor and event serializes through the generic CBOR path, and all binding generators emit working callers for it. `abi = c` is a newer, native C-struct wire (no CBOR round-trip). The single `c` generator (`-d:targetLang=c`) emits its callers, choosing the `abi = c` or CBOR header shape from the library's ABI format. It carries two honest limits today: - **Events are CBOR-only.** Applying `abi = c` to an `{.ffiEvent.}` proc is a hard compile error; declare events with `abi = cbor` (they ride CBOR internally regardless of the library default). - **Enums are CBOR-only.** A `{.ffi.}` enum in an `abi = c` library, or reached from an `abi = c` type or proc, is a hard compile error naming the type. - **All-scalar `abi = c` procs bind only in the `abi = c` C header.** A `{.ffi: "abi = c".}` method whose params and return are all scalars — ints, floats, bools; a `string` return is fine, a `string` param is not — takes a CBOR-free fast path, and its C wrapper passes the args inline instead of packing a request struct. Only the `abi = c` C header emits that shape. The CBOR C header and the `cpp`, `rust` and `cddl` targets have no scalar codegen and would silently omit the proc, so `genBindings()` fails and names it. Fix it by generating C bindings from an `abi = c` library, switching the proc to `abi = cbor`, giving it a non-scalar param, or passing `-d:ffiAllowScalarSkip` to accept the omission — the proc still works over the fast path, it's just absent from the bindings. ## Placement of `genBindings()` `genBindings()` reads the compile-time registries that the pragmas populate as the compiler expands them, so **it must come after every annotation**. Since Nim resolves `import`s before running the importing module's body, a multi-file library keeps its annotations in imported sub-modules and calls `genBindings()` once at the bottom of the top-level root file. An annotation that expands *after* `genBindings()` is now a **compile error** (previously it was silently dropped from the bindings). ## Building — the two-compile model A nim-ffi library ships from **two separate compiles of the same source**, because binding emission is gated behind `-d:ffiGenBindings`: without that define `genBindings()` is a no-op, so the normal build just produces the shared library and nothing else. **1. Build the shared library** (the artifact your host loads): ```sh nim c --app:lib --noMain --nimMainPrefix:libmylib mylib.nim ``` **2. Emit the foreign bindings** — add the binding defines and `--compileOnly`, which stops after codegen: the binding files are written during macro expansion, so there's no library to link (no `--app:lib`/`-o:/dev/null` needed). The generated files (for `targetLang=c`: the `.h` header your host includes, plus a `CMakeLists.txt`) land in `-d:ffiOutputDir`: ```sh nim c -d:ffiGenBindings -d:targetLang=rust,cpp,c --compileOnly mylib.nim ``` - `-d:targetLang` — which generator(s) run; pass a comma-separated list to emit several from one compile: - **Language bindings:** `rust` (default), `cpp`, `c`. The `c` target follows the library's ABI format — an `abi = c` C-struct header for `abi = c`, a CBOR header otherwise; `rust`/`cpp` speak CBOR. - **`cddl`** — a CDDL schema of the CBOR wire, not a language binding at all. - `-d:ffiOutputDir` — override where the generated files land. Defaults to `_bindings/` next to the compiled source. - `-d:ffiSrcPath` — override the Nim source path embedded in the generated build files. Defaults to the compiled source made relative to the output dir. ### The `--nimMainPrefix:lib` rule `--app:lib` builds a shared library; `--noMain` hands program entry to the foreign host rather than Nim's own `main`. To initialize the Nim runtime, `declareLibrary("", …)` emits an `initializeLibrary()` export that calls `libNimMain()` — the symbol Nim's `NimMain` is renamed to by `--nimMainPrefix`. So the prefix **must be exactly `lib` + the `declareLibrary` name**, on *both* compiles above, or the library fails to link. For `declareLibrary("my_timer", …)` that is `--nimMainPrefix:libmy_timer`. **Library-naming collisions.** When several nim-ffi libraries are loaded into one process (as the C++ end-to-end test does with `timer` + `echo`), each must use a **distinct** library name — and therefore a distinct `--nimMainPrefix` — so their exported `NimMain`, `initializeLibrary` and per-symbol names don't clash. The example libraries deliberately differ: `libmy_timer` vs `libecho`. ## Examples - `examples/timer` — a self-contained Nimble project covering the ctor, sync and async methods, multi-param methods, events, and C / C++ / Rust / CDDL bindings with runnable clients. Start here: ```sh cd examples/timer && nimble install -y ../.. && nimble build ``` - `examples/echo` — a second minimal library, loaded alongside `timer` in the C++ end-to-end test to prove two libraries coexist in one process.