nim-ffi/README.md

nim-ffi

Expose a Nim library to C, C++ and Rust by annotating ordinary Nim procs.

You write async Nim; nim-ffi provides the whole FFI runtime — a dedicated worker thread, a request channel, CBOR (de)serialization, an event queue and a context/handle registry — and generates the foreign-language bindings for you. No hand-written .h files, no manual request enums, no shared-memory plumbing.

Install

Add nim-ffi to your library's .nimble, then import ffi:

requires "https://github.com/logos-messaging/nim-ffi >= 0.3.0"

Mental model

• You declare a library once with declareLibrary(name, LibType).
• You annotate procs and types with pragmas ({.ffi.}, {.ffiCtor.}, {.ffiDtor.}, {.ffiStatic.}, {.ffiEvent.}).
• You call genBindings() last, which emits the foreign bindings.

By default, every request/response crosses the boundary as a single CBOR blob (the wire format is configurable per library or per annotation — see ABI format); the ctx handle returned by the constructor is the only pointer that crosses. Each {.ffi.} proc runs on the library's own chronos event loop, so bodies can await freely.

Minimal example

import ffi, chronos

# 1. The library's main state. The FFI context owns one instance.
type Counter = object
  value: int

declareLibrary("counter", Counter)

# 2. Request/response shapes. Any {.ffi.} object or enum type becomes a
#    first-class struct/class in the generated bindings and rides the wire in
#    the library's ABI format (CBOR by default).
type BumpRequest {.ffi.} = object
  by: int

type BumpResponse {.ffi.} = object
  newValue: int

# 3. Constructor: returns Future[Result[LibType, string]].
proc counterNew*(): Future[Result[Counter, string]] {.ffiCtor.} =
  return ok(Counter(value: 0))

# 4. A method: first param is the library value, then any typed params.
#    Return type is always Future[Result[T, string]].
proc counterBump*(
    c: Counter, req: BumpRequest
): Future[Result[BumpResponse, string]] {.ffi.} =
  await sleepAsync(1.milliseconds)
  return ok(BumpResponse(newValue: c.value + req.by))

# 5. Destructor: exactly one param (the library value).
proc counterDestroy*(c: Counter) {.ffiDtor.} =
  discard

# 6. genBindings() must be the LAST FFI call in the file (see below).
genBindings()

The generated C export names are the snake_case form of the proc names, e.g. counterBump → counter_bump.

Pragma reference

Pragma	Applies to	Purpose
declareLibrary(name, LibType[, defaultABIFormat])	call	Registers the library, its state type, and the default wire format. Must run before any annotation.
{.ffi.} on a type	object, enum	Registers the type for binding generation; it serializes via the library's ABI format (CBOR by default). Enums are CBOR-only — see below.
{.ffi.} on a proc	proc	Exposes a method. First param is the library value, then typed params; returns Future[Result[T, string]].
{.ffiStatic.}	proc	Exposes a context-independent proc: no library param, and its wrapper takes no ctx — see below.
{.ffiCtor.}	proc	The constructor. Returns Future[Result[LibType, string]]; creates the FFI context.
{.ffiDtor.}	proc	The destructor. Exactly one param (x: LibType); tears the context down.
{.ffiEvent[: "wire_name"].}	proc (empty body)	A library-initiated callback. Call the proc from any {.ffi.} handler to fire it. The wire name is optional — see below.
{.ffiHandle.}	ref object	Marks a type as an opaque handle: it stays server-side and crosses the wire as a uint64 id.
{.ffiConst.}	const	Re-emits the value as a native constant in every generated binding — see below.
genBindings()	call	Emits the bindings. Must be the last FFI call in the compilation root.

Enums

{.ffi.} on an enum gives each language its own native enum:

type Level {.ffi.} = enum
  lLow = "low"
  lHigh = "high"

typedef enum { LEVEL_L_LOW = 0, LEVEL_L_HIGH = 1 } Level;  /* C   */
enum class Level { lLow = 0, lHigh = 1 };                  // C++
pub enum Level { #[serde(rename = "low")] LLow, ... }      // Rust

An enum crosses the wire as text, not as its ordinal — whatever $value yields, so the associated string if the enum declares one ("low" above) and the symbol name otherwise (lLow). That is what cbor_serialization writes on the Nim side, and the generated codecs map name ↔ value on the far side, so reordering or renumbering values doesn't break an already-deployed peer. Explicit ordinals are carried into the foreign enum so the two sides agree if you ever cast.

Enums are supported on the CBOR wire only. Reaching one from an abi = c type or proc is a compile error naming the type — the abi = c _CWire structs have no enum form yet.

Constants

{.ffiConst.} copies a Nim const into the generated bindings, so callers don't hand-maintain a second copy of a limit, a default or a protocol string:

const
  MaxPeers* {.ffiConst.} = 42
  DefaultTimeoutMs* {.ffiConst.}: uint32 = 3 * 1000
  Greeting* {.ffiConst.} = "hello"

static const int64_t MAX_PEERS = 42LL;            /* C   */
static const uint32_t DEFAULT_TIMEOUT_MS = 3000;
static const char* const GREETING = "hello";

constexpr int64_t MAX_PEERS = 42LL;               // C++

pub const MAX_PEERS: i64 = 42;                    // Rust

Integer, float, bool and string consts are supported; anything else is a compile error. The value is whatever the const evaluates to, so computed expressions arrive folded. Names are re-cased to UPPER_SNAKE, preserving acronyms (httpTTL → HTTP_TTL). A constant is a compile-time value in each language, not a symbol exported by the shared library — it never crosses the wire, so {.ffiConst.} is ABI-agnostic.

Doc comments

A ## doc comment on an annotated proc is carried through to every generated binding, so the exported API is documented once, at the source:

proc myTimerEcho*(
    timer: MyTimer, req: EchoRequest
): Future[Result[EchoResponse, string]] {.ffi.} =
  ## Sleeps `delayMs` then echoes the message back.
  ...

becomes /** Sleeps `delayMs` then echoes the message back. */ above both the exported symbol and the <lib>_ctx_echo wrapper in the C header, /// ... on the C++ class method and the Rust pub fn, and a ; comment in the CDDL schema. Multi-line doc comments are preserved as-is.

Only ## doc comments are propagated, and only on procs — a plain # comment above the proc, and any comment on a {.ffi.} type or its fields, is dropped by Nim's parser before the macro can see it.

The return-type contract

Every {.ffi.} / {.ffiCtor.} proc must have an explicit Future[Result[T, string]] return type — even for synchronous logic (just return ok(...) without awaiting). The Result's error string is delivered to the foreign caller as the failure message.

Context-independent procs

A {.ffi.} proc is a method: it takes the library value, and its wrapper takes a ctx, so the host must construct the library to call it. A stateless utility shouldn't have to pay for that. Annotate it {.ffiStatic.} instead — drop the library param, and the ctx disappears from the generated wrapper:

proc counterParse*(text: string): Future[Result[BumpRequest, string]] {.ffiStatic.} =
  return ok(BumpRequest(by: text.parseInt()))

/* {.ffi.} method              */ counter_ctx_bump(ctx, &req, on_reply, ud);
/* {.ffiStatic.} — no ctx      */ counter_static_parse(text, on_reply, ud);

The wrapper is <lib>_static_<proc>, not <lib>_<proc>, for the same reason a method's is <lib>_ctx_<proc>: <lib>_<proc> is the raw symbol the dylib exports. In C++ and Rust a static is an associated function on the ctx type (EchoCtx::lib_version()), taking the timeout a method reads from its ctx.

The handler still needs an FFI thread, so it runs on the library's static context: created on the first {.ffiStatic.} call, then alive for the rest of the process — no ctx owns it, so nothing tears its thread pair down, and it holds one of the pool's slots. It has no myLib, which is why a static proc cannot take the library value.

There is no foreign teardown for it. From Nim, destroyStaticFFIContext(pool) stops the thread pair and frees the slot; it is only sound once nothing will call a {.ffiStatic.} proc again, so it is meant for process shutdown and tests.

The macro rejects an {.ffiHandle.} parameter or return: a handle is registered in the context that created it, which a static proc cannot reach. Under abi = c a static replies with a string or an {.ffi.} object type — a scalar return is wired only for an all-scalar {.ffi.} method, which rides the CBOR-free fast path through the ctx a static doesn't have.

The result callback contract

Each request carries a result callback. It receives one of these status codes (ret / err_code):

Code	Value	Terminal?	Meaning
RET_OK	0	yes	Success; the payload carries the encoded result.
RET_ERR	1	yes	Failure; the payload carries the UTF-8 error string.
RET_MISSING_CALLBACK	2	—	No callback was passed; the request path reports this itself.
RET_STALE_WARN	3	no	Progress ping — the handler is still running.

nim-ffi never times a handler out. A slow request runs to its natural RET_OK / RET_ERR; it is never cancelled (a hard-cancel mid-call into the underlying library can leave it half-applied). Instead, while a handler is still in flight the callback receives a non-terminal RET_STALE_WARN every 5s (Android's ANR interval; override at build time with -d:ffiStaleWarnIntervalMs=<ms>), with the payload carrying the elapsed milliseconds as a decimal string. The dev decides what to do with a slow request — keep waiting, surface a spinner, tear the context down — nim-ffi does not decide for them.

RET_STALE_WARN may fire any number of times and is always followed by exactly one terminal RET_OK / RET_ERR. A caller that only wants the final answer must ignore it (do not treat a non-zero code as an error without checking for RET_STALE_WARN first). The generated higher-level typed wrappers currently ignore it; the progress signal is delivered at the raw result-callback boundary.

Events

An event is a proc with an empty body annotated {.ffiEvent.}. You fire it by calling it with a typed payload from inside any {.ffi.} handler; the foreign side receives it through a registered callback.

type PeerConnected {.ffi.} = object
  id: string

proc onPeerConnected*(peer: PeerConnected) {.ffiEvent.}  # wire name: "on_peer_connected"

proc counterBump*(c: Counter, req: BumpRequest): Future[Result[BumpResponse, string]] {.ffi.} =
  onPeerConnected(PeerConnected(id: "p-1"))
  return ok(BumpResponse(newValue: c.value + req.by))

The wire name is optional: when omitted it is derived from the proc name (onPeerConnected → on_peer_connected), matching how {.ffi.} derives its C export symbol. Pass a string literal ({.ffiEvent: "custom_name".}) only when you need a name that differs from the proc.

ABI format

The wire format is chosen in code, never by a compile flag. Override the library default with declareLibrary("lib", Lib, defaultABIFormat = "c"), or per annotation with an "abi = ..." spec, e.g. {.ffi: "abi = c".}. The -d:targetLang flag (below) picks which language the bindings are emitted for; it does not change the wire.

cbor is the default and fully-supported format: every proc, ctor, dtor and event serializes through the generic CBOR path, and all binding generators emit working callers for it.

abi = c is a newer, native C-struct wire (no CBOR round-trip). The single c generator (-d:targetLang=c) emits its callers, choosing the abi = c or CBOR header shape from the library's ABI format. It carries two honest limits today:

• Events are CBOR-only. Applying abi = c to an {.ffiEvent.} proc is a hard compile error; declare events with abi = cbor (they ride CBOR internally regardless of the library default).
• Enums are CBOR-only. A {.ffi.} enum in an abi = c library, or reached from an abi = c type or proc, is a hard compile error naming the type.
• All-scalar abi = c procs bind only in the abi = c C header. A {.ffi: "abi = c".} method whose params and return are all scalars — ints, floats, bools; a string return is fine, a string param is not — takes a CBOR-free fast path, and its C wrapper passes the args inline instead of packing a request struct. Only the abi = c C header emits that shape. The CBOR C header and the cpp, rust and cddl targets have no scalar codegen and would silently omit the proc, so genBindings() fails and names it. Fix it by generating C bindings from an abi = c library, switching the proc to abi = cbor, giving it a non-scalar param, or passing -d:ffiAllowScalarSkip to accept the omission — the proc still works over the fast path, it's just absent from the bindings.

Placement of genBindings()

genBindings() reads the compile-time registries that the pragmas populate as the compiler expands them, so it must come after every annotation. Since Nim resolves imports before running the importing module's body, a multi-file library keeps its annotations in imported sub-modules and calls genBindings() once at the bottom of the top-level root file.

An annotation that expands after genBindings() is now a compile error (previously it was silently dropped from the bindings).

Building — the two-compile model

A nim-ffi library ships from two separate compiles of the same source, because binding emission is gated behind -d:ffiGenBindings: without that define genBindings() is a no-op, so the normal build just produces the shared library and nothing else.

1. Build the shared library (the artifact your host loads):

nim c --app:lib --noMain --nimMainPrefix:libmylib mylib.nim

2. Emit the foreign bindings — add the binding defines and --compileOnly, which stops after codegen: the binding files are written during macro expansion, so there's no library to link (no --app:lib/-o:/dev/null needed). The generated files (for targetLang=c: the <name>.h header your host includes, plus a CMakeLists.txt) land in -d:ffiOutputDir:

nim c -d:ffiGenBindings -d:targetLang=rust,cpp,c --compileOnly mylib.nim

• -d:targetLang — which generator(s) run; pass a comma-separated list to emit several from one compile:
  • Language bindings: rust (default), cpp, c. The c target follows the library's ABI format — an abi = c C-struct header for abi = c, a CBOR header otherwise; rust/cpp speak CBOR.
  • cddl — a CDDL schema of the CBOR wire, not a language binding at all.
• -d:ffiOutputDir — override where the generated files land. Defaults to <lang>_bindings/ next to the compiled source.
• -d:ffiSrcPath — override the Nim source path embedded in the generated build files. Defaults to the compiled source made relative to the output dir.

The --nimMainPrefix:lib<name> rule

--app:lib builds a shared library; --noMain hands program entry to the foreign host rather than Nim's own main. To initialize the Nim runtime, declareLibrary("<name>", …) emits an initializeLibrary() export that calls lib<name>NimMain() — the symbol Nim's NimMain is renamed to by --nimMainPrefix. So the prefix must be exactly lib + the declareLibrary name, on both compiles above, or the library fails to link. For declareLibrary("my_timer", …) that is --nimMainPrefix:libmy_timer.

Library-naming collisions. When several nim-ffi libraries are loaded into one process (as the C++ end-to-end test does with timer + echo), each must use a distinct library name — and therefore a distinct --nimMainPrefix — so their exported NimMain, initializeLibrary and per-symbol names don't clash. The example libraries deliberately differ: libmy_timer vs libecho.

Examples

• examples/timer — a self-contained Nimble project covering the ctor, sync and async methods, multi-param methods, events, and C / C++ / Rust / CDDL bindings with runnable clients. Start here:

  cd examples/timer && nimble install -y ../.. && nimble build
  

• examples/echo — a second minimal library, loaded alongside timer in the C++ end-to-end test to prove two libraries coexist in one process.