logos-cpp-sdk/cpp-generator/docs/spec.md

Logos Code Generator — Experimental

Overall Description

The experimental code generator extends logos-cpp-generator with three input modes: a lightweight Interface Definition Language (LIDL) for declaring module contracts, a C++ header parser that infers module interfaces from pure C++ implementation classes, and a C header parser that generates Qt plugin glue directly from plain C function declarations. All three paths produce the same output: Qt plugin glue code that bridges module implementations to the Logos runtime's Qt Remote Objects transport.

The goal is to decouple module business logic from the Qt framework. Module authors write their logic in any language (Rust, Go, Zig, C, C++), and the build system generates all Qt boilerplate (QObject, Q_PLUGIN_METADATA, QString conversions, method dispatch) automatically. The --from-c-header mode goes furthest: a module backed by a Rust static library requires zero hand-written C++.

Definitions & Acronyms

Term	Definition
LIDL	Logos Interface Definition Language — a lightweight DSL for declaring module interfaces
Universal Module	A module whose implementation is pure C++ (no Qt types) with all Qt glue generated at build time
C-FFI Module	A module whose implementation is any language (Rust, Go, Zig, C) exposing a C ABI; Qt glue is generated directly from the C header
Provider Glue	Generated code that wraps an implementation in a LogosProviderObject with callMethod() dispatch and getMethods() introspection
Client Stub	Generated type-safe C++ wrapper class that callers use to invoke a module's methods without string-based dispatch
Dispatch	The generated callMethod() function that maps string method names to typed method calls on the provider object
Impl Header	The pure C++ header file (_impl.h) that declares a module's public methods using standard C++ types
C Header	A plain C header (*.h) that declares exported functions sharing a common prefix; used in --from-c-header mode
Function Prefix	The shared prefix of all exported C functions in a C-FFI module (e.g. rust_example_). Auto-derived from module name.
TypeExpr	The AST node representing a type in the LIDL type system
ModuleDecl	The AST node representing a complete module declaration (name, version, methods, events, types, hasEmitEvent flag)

Domain Model

Three Paths to the Same Output

Path 1: LIDL file         Path 2: C++ impl header    Path 3: C header (c-ffi)
    │                            │                           │
    ▼                            ▼                           ▼
 lidlTokenize()          parseImplHeader()           parseCHeader()
    │                            │                           │
    ▼                            │                           │
 lidlParse()                     │                           │
    │                            │                           │
    ▼                            │                           │
 lidlValidate()                  │                           │
    │                            │                           │
    ▼                            ▼                           ▼
 ModuleDecl  ◄──────── same AST ──────────────────► ModuleDecl
    │                                                    │
    │                                                CHeaderParseResult
    │                                                (adds C function names
    │                                                 + string ownership info)
    │                                                    │
    ├──► lidlMakeProviderHeader()  ◄──────────────────────┤ (via lidlMakeProviderHeaderCFFI)
    │         → <name>_qt_glue.h                          │
    │                                                     │
    ├──► lidlMakeProviderDispatch() ◄────────────────────-┤ (via lidlMakeProviderDispatchCFFI)
    │         → <name>_dispatch.cpp                       │
    │
    ├──► lidlMakeHeader()
    │         → <name>_api.h
    │
    └──► lidlMakeSource()
              → <name>_api.cpp

All three paths converge at ModuleDecl. Path 3 carries additional per-method metadata in CHeaderParseResult (original C function name, heap string ownership) so the generator can emit direct C function calls instead of m_impl.method() calls.

Key difference between Path 2 and Path 3

	Path 2 (--from-header)	Path 3 (--from-c-header)
Input	C++ class with public methods	C functions with shared prefix
Generated calls	m_impl.method(args) (via C++ object)	c_function(args) (direct C call)
String handling	std::string ↔ QString	char*/const char* ↔ QString
Heap strings	N/A (std::string manages memory)	char* return → freed with {prefix}free_string
Hand-written C++	~80 lines (impl .h + .cpp)	Zero

LIDL Language

LIDL is a minimal interface definition language. A module declaration contains metadata, type definitions, method signatures, and event signatures:

module wallet_module {
    version "1.0.0"
    description "Wallet operations"
    category "finance"
    depends [crypto_module]

    type Account {
        address: tstr
        balance: uint
        ? label: tstr          ; optional field
    }

    method createAccount(passphrase: tstr) -> tstr
    method getBalance(address: tstr) -> uint
    method listAccounts() -> [tstr]
    method transfer(from: tstr, to: tstr, amount: uint) -> result

    event onTransfer(from: tstr, to: tstr, amount: uint)
}

Comments start with ; and run to end of line.

Type System

Built-in primitive types:

LIDL type	Meaning	Qt mapping	C++ std mapping
tstr	Text string	QString	std::string
bstr	Binary data	QByteArray	std::vector<uint8_t>
int	Signed 64-bit integer	int	int64_t
uint	Unsigned 64-bit integer	int	uint64_t
float64	Double precision float	double	double
bool	Boolean	bool	bool
result	Structured result (success/value/error)	LogosResult	LogosResult
any	Untyped value	QVariant	QVariant
void	No return value	void	void

Composite types:

• [T] — Array of T (e.g., [tstr] → QStringList / std::vector<std::string>)
• {K: V} — Map from K to V (e.g., {tstr: int} → QVariantMap)
• ?T — Optional T (→ QVariant)

Named types reference type definitions within the same module.

C++ Header Parsing

The --from-header mode parses a C++implementation header to extract public method signatures. It maps C++ types to LIDL types:

C++ type	LIDL type
std::string / const std::string&	tstr
bool	bool
int64_t	int
uint64_t	uint
double	float64
void	void
std::vector<std::string>	[tstr]
std::vector<uint8_t>	bstr
std::vector<int64_t>	[int]
std::vector<uint64_t>	[uint]
std::vector<double>	[float64]
std::vector<bool>	[bool]
LogosMap	{tstr: any} (Map) — nlohmann::json alias; sets jsonReturn flag
LogosList	[any] (Array) — nlohmann::json alias; sets jsonReturn flag
QVariantMap	{tstr: any} (Map) — legacy Qt type
QVariantList	[any] (Array) — legacy Qt type
QStringList	[tstr] (Array) — legacy Qt type
Anything else	any

LogosMap and LogosList are using aliases for nlohmann::json defined in logos_json.h (part of the SDK). They allow module implementations to remain completely Qt-free while returning rich structured data. The parser maps them to the same LIDL shapes as QVariantMap/QVariantList, but sets the jsonReturn flag on the method so the generator emits an nlohmannToQVariant() conversion in the glue layer.

The parser uses a state machine to find the target class, track access specifiers (public/private/protected), and extract method declarations. It skips constructors, destructors, typedefs, using declarations, std::function members, and non-method statements.

Module metadata (name, version, description, dependencies) comes from metadata.json, not from the header.

C Header Parsing (--from-c-header)

The --from-c-header mode parses a plain C header (no class syntax) to extract function declarations that share a common prefix. It maps C types to LIDL types:

C type	LIDL type	Qt type	Notes
int64_t, int32_t, int, long	int	int	Cast via static_cast<int64_t> for C call
uint64_t, uint32_t, unsigned int	uint	int
double, float	float64	double
bool, _Bool	bool	bool
const char*	tstr	QString	Static/borrowed — no free
char*	tstr	QString	Heap-allocated — freed with {prefix}free_string
void	void	void
anything else	any	QVariant

Prefix convention: All exported C functions must share a prefix. The prefix is auto-derived from the module name: strip _module suffix, append _. For "name": "rust_example_module" → prefix rust_example_. Override with --prefix CLI flag or "codegen": {"c_prefix": "..."} in metadata.json.

String ownership: The parser distinguishes char* (mutable, heap-allocated) from const char* (immutable, static/borrowed). For char* returns, the generator emits:

char* _result = c_function(args);
QString _ret = _result ? QString::fromUtf8(_result) : QString();
prefix_free_string(_result);   // auto-wired
return _ret;

For const char* returns: no free call. The {prefix}free_string(char*) function is detected automatically in the header and not exposed as a module method.

Reserved name renaming: {prefix}version → libVersion, {prefix}name → libName, {prefix}initLogos → libInitLogos.

The parser skips: #include, #ifdef/#endif, extern "C" blocks, block comments (/* ... */), blank lines, typedef/struct/union/enum declarations, and lines that don't start with the expected prefix.

Module metadata (name, version, description, dependencies) comes from metadata.json, same as Path 2.

Event Emission via Header Detection

Universal modules can emit named events to the host/runtime by declaring a public emitEvent callback in their impl header:

class MyModuleImpl {
public:
    std::function<void(const std::string& eventName, const std::string& data)> emitEvent;
    // ... methods ...
};

The parser detects this std::function member by name and sets ModuleDecl.hasEmitEvent = true. The generator then wires the callback in the provider constructor:

MyModuleProviderObject() {
    m_impl.emitEvent = [this](const std::string& name, const std::string& data) {
        QVariantList args;
        if (!data.empty()) args << QString::fromStdString(data);
        emitEvent(QString::fromStdString(name), args);
    };
}

This replaces the previous approach of declaring events in metadata.json. The events array in metadata.json is still supported for backward compatibility (e.g., LIDL-defined modules), but header detection is the preferred approach for universal modules since it keeps event information co-located with the implementation.

Generated Output

Provider Glue (<name>_qt_glue.h) — universal mode (--from-header)

Contains two classes:

1. ProviderObject — inherits LogosProviderBase, holds an instance of the impl class (m_impl). Each public method is wrapped with type conversion:

• Qt parameters → C++ std parameters (e.g., QString.toStdString())
• Call m_impl.method(...)
• C++ std return → Qt return (e.g., QString::fromStdString(result))
• For jsonReturn methods (returning LogosMap/LogosList), the glue calls a generated nlohmannToQVariant() recursive helper to convert nlohmann::json → QVariant/QVariantMap/QVariantList
• If the impl declares an emitEvent callback (hasEmitEvent), the constructor wires it to LogosProviderBase::emitEvent

2. Plugin — QObject subclass implementing PluginInterface and LogosProviderPlugin. Carries Q_PLUGIN_METADATA and Q_INTERFACES. Its createProviderObject() factory returns a new ProviderObject instance.

Provider Glue (<name>_qt_glue.h) — c-ffi mode (--from-c-header)

Same two-class structure, but the ProviderObject:

• Has no m_impl member — no C++ class is instantiated
• Includes the C header directly via extern "C" { #include "..." }
• Each method calls the C function directly with Qt ↔ C type conversions:
  • QString → QByteArray (kept alive) → const char* for input strings
  • char* C return → QString::fromUtf8() + {prefix}free_string() call
  • const char* C return → QString::fromUtf8(), no free
  • int64_t / uint64_t ↔ int via static_cast

Dispatch (<name>_dispatch.cpp)

Implements two methods on the ProviderObject:

1. **callMethod(methodName, args)** — string-based dispatch table. For each method, extracts args from QVariantList, calls the typed wrapper, returns result as QVariant. Void methods return QVariant(true).
2. **getMethods()** — returns QJsonArray of method metadata. Each entry has name, signature, returnType, isInvokable, and parameters[] (with type and name).

Client Stubs (<name>_api.h + <name>_api.cpp)

Generated from LIDL (not from --from-header). Provides:

• Typed sync methods that call invokeRemoteMethod() and convert the QVariant result
• Async overloads with callback + timeout
• Event subscription (on()) and emission (trigger())
• Umbrella logos_sdk.h / logos_sdk.cpp aggregating all module wrappers

Features & Requirements

LIDL Pipeline

1. Lexer (lidlTokenize) — tokenizes source into keywords, identifiers, string literals, symbols
2. Parser (lidlParse) — recursive descent parser producing a ModuleDecl AST
3. Validator (lidlValidate) — checks for duplicate names, unknown type references, builtin shadowing, duplicate parameters
4. Serializer (lidlSerialize) — pretty-prints a ModuleDecl back to LIDL text (useful for roundtrip testing)

Impl Header Pipeline

1. parseImplHeader — reads metadata.json + C++ header, produces a ModuleDecl
2. Same generation functions as LIDL path

C Header Pipeline

1. parseCHeader — reads metadata.json + C header, produces a CHeaderParseResult containing:
   • ModuleDecl (same AST as the other paths)
   • Per-method CHeaderMethod entries: original C function name + heap-string ownership flag
   • freeStringFunc — the detected {prefix}free_string function name (empty if none)
2. lidlMakeProviderHeaderCFFI — generates Qt glue with direct C function calls
3. lidlMakeProviderDispatchCFFI — generates callMethod/getMethods dispatch (delegates to lidlMakeProviderDispatch since the dispatch shape is identical)

Backwards Compatibility

• All existing generator modes (--provider-header, --metadata, plugin path) continue to work unchanged via legacy_main()
• The --from-header, --from-c-header, and --lidl modes are additive
• Generated plugins implement both PluginInterface (for lm introspection) and LogosProviderPlugin (for new-API provider creation)
• The runtime (logos-liblogos) already supports both old and new plugin types via qobject_cast detection

Conversion Helper Generation

Conversion helpers are only emitted when needed:

• String vector helpers (lidlToQStringList, lidlToStdStringVector) — emitted when the module uses [tstr] parameters or return types
• nlohmann→Qt helper (nlohmannToQVariant) — emitted when any method has jsonReturn = true (i.e., the impl returns LogosMap or LogosList). This recursive function converts nlohmann::json objects, arrays, strings, numbers, and booleans to their QVariant equivalents.

Choosing an Interface Mode

Situation	Recommended mode
Logic in Rust/Go/Zig/C with synchronous C API	c-ffi (--from-c-header)
Logic in C++ without Qt dependency	universal (--from-header)
Complex async/callback-driven C library	Hand-written plugin (or universal with manual impl class)
Formal cross-team contract needed	lidl (.lidl file)

When c-ffi is the right choice

Use c-ffi when the C library is synchronous and the return value is immediate:

Qt call → C function → immediate return value → Qt return

This covers pure computation (math, crypto primitives, string processing), simple config/storage APIs, and most wrappers around Rust or Zig libraries. Zero hand-written C++ required.

When c-ffi is NOT the right choice

c-ffi generates a direct call-and-return for every method. It cannot express:

1. Async / callback-driven APIs. If the C library takes a void (*callback)(int code, const char* msg, void* userData) and calls it later, there is nowhere in the generated code to wait for it. A real example is logos-storage-module, which wraps libstorage — every operation (init, start, upload, download) is asynchronous. The hand-written plugin uses a Qt mutex + QWaitCondition to turn callbacks into synchronous LogosResult returns, and a Qt signal system to propagate async events (storageConnect, uploadProgress, etc.) to the host.

2. Per-instance state. c-ffi generates no class instance — all state must live inside the C library itself (global or thread-local). If you need a context pointer (void* ctx) that is created on init and passed to every subsequent call, you need an impl class (or a hand-written plugin) to hold it as a member variable.

3. Asynchronous events. If the C library fires callbacks on its own schedule (connection events, progress notifications), those need to be converted into Logos events (emitEvent()). This requires a custom callback registration step and a way to route the callback back to the Qt object — not expressible in generated straight-line code.

4. Non-trivial Qt types as parameters. c-ffi supports int64_t, bool, double, char*, const char*, and void. Parameters like QUrl, QByteArray, QStringList, default argument values, or overloaded methods require hand-written conversion logic.

The rule of thumb: if wrapping the library requires more than type conversions in the generated glue, use universal or write the plugin by hand.