logos-cpp-sdk/cpp-generator/docs/spec.md

Logos Code Generator — Experimental

Overall Description

The experimental code generator extends logos-cpp-generator with two new capabilities: a lightweight Interface Definition Language (LIDL) for declaring module contracts, and a C++header parser that can infer module interfaces directly from pure C++ implementation classes. Both paths produce the same output: the Qt-free logos_module_* C-ABI provider glue that bridges pure C++ module implementations to the runtime. (It used to emit the Qt plugin glue directly; turning the C ABI into a Qt plugin is now a downstream step, logos-qt-host-generator --backend cdylib in logos-plugin-qt, and that seam is what lets the Rust and JS providers target the same ABI.)

The goal is to decouple module business logic from the Qt framework. Module authors write standard C++ using std::string, int64_t, std::vector<T>, and the build system generates all Qt boilerplate (QObject, Q_PLUGIN_METADATA, QString conversions, method dispatch) automatically.

Definitions & Acronyms

Term	Definition
LIDL	Logos Interface Definition Language — a lightweight DSL for declaring module interfaces
Universal Module	A module whose implementation is pure C++ (no Qt types) with all Qt glue generated at build time
Provider Glue	Generated code that wraps a pure C++ impl class in a LogosProviderObject with callMethod() dispatch and getMethods() introspection
Client Stub	Generated type-safe C++ wrapper class that callers use to invoke a module's methods without string-based dispatch
Dispatch	The generated callMethod() function that maps string method names to typed method calls on the provider object
Impl Header	The pure C++header file (_impl.h) that declares a module's public methods using standard C++ types
TypeExpr	The AST node representing a type in the LIDL type system
ModuleDecl	The AST node representing a complete module declaration (name, version, methods, events, types)

Domain Model

Two Paths to the Same Output

Path 1: LIDL file                    Path 2: C++ impl header
    │                                     │
    ▼                                     ▼
 lidlParse()                       parseImplHeader()
    │  (logos-lidl's lidl::parse,         │
    │   via lidl_compat.h)                │
    ▼                                     │
 lidlValidate()                           │
    │                                     │
    ▼                                     ▼
 ModuleDecl  ◄────── same AST ──────► ModuleDecl
    │                                     │
    ├──► lidlMakeTypesHeaderCdylib()      │
    │    lidlMakeModuleImplExports()      │
    │    lidlMakeEventsSourceCdylib()     │
    │         → logos_module_* C ABI       │
    │           (Qt packaging is a         │
    │            downstream step:          │
    │            logos-qt-host-generator)  │
    │                                     │
    ├──► lidlMakeHeader()                 │
    │         → <name>_api.h              │
    │                                     │
    └──► lidlMakeSource()                 │
              → <name>_api.cpp            │

(There is no lidlTokenize() step here any more: the lexer lives in logos-lidl with the rest of the frontend, behind lidl::parse.)

Both paths converge at ModuleDecl, the shared AST. From there, the same generation functions produce identical output regardless of the input format.

LIDL Language

LIDL is a minimal interface definition language. A module declaration contains metadata, type definitions, method signatures, and event signatures:

module wallet_module {
    version "1.0.0"
    description "Wallet operations"
    category "finance"
    depends [crypto_module]

    type Account {
        address: tstr
        balance: uint
        ? label: tstr          ; optional field
    }

    method createAccount(passphrase: tstr) -> tstr
    method getBalance(address: tstr) -> uint
    method listAccounts() -> [tstr]
    method transfer(from: tstr, to: tstr, amount: uint) -> result

    event onTransfer(from: tstr, to: tstr, amount: uint)
}

Comments start with ; and run to end of line.

Type System

Built-in primitive types:

LIDL type	Meaning	Qt mapping	C++ std mapping
tstr	Text string	QString	std::string
bstr	Binary data	QByteArray	std::vector<uint8_t>
int	Signed 64-bit integer	qlonglong	int64_t
uint	Unsigned 64-bit integer	qulonglong	uint64_t
float64	Double precision float	double	double
bool	Boolean	bool	bool
result	Structured result (success/value/error)	LogosResult	LogosResult
any	Untyped value	QVariant	QVariant
void	No return value	void	void

Composite types:

• [T] — Array of T (e.g., [tstr] → QStringList / std::vector<std::string>)
• {K: V} — Map from K to V (e.g., {tstr: int} → QVariantMap)
• ?T — Optional T (→ QVariant on the Qt surface, which loses the value type; std::optional<T> on the std surface — see Optionality in project.md)

Named types reference type definitions within the same module.

C++ Header Parsing

The --from-header mode parses a C++implementation header to extract public method signatures. It maps C++ types to LIDL types:

C++ type	LIDL type
std::string / const std::string&	tstr
bool	bool
int64_t	int
uint64_t	uint
double	float64
void	void
std::vector<std::string>	[tstr]
std::vector<uint8_t>	bstr
std::vector<int64_t>	[int]
std::vector<uint64_t>	[uint]
std::vector<double>	[float64]
std::vector<bool>	[bool]
LogosMap	{tstr: any} (Map) — nlohmann::json alias; sets jsonReturn flag
LogosList	[any] (Array) — nlohmann::json alias; sets jsonReturn flag
QVariantMap	{tstr: any} (Map) — legacy Qt type
QVariantList	[any] (Array) — legacy Qt type
QStringList	[tstr] (Array) — legacy Qt type
Anything else	any

LogosMap and LogosList are using aliases for nlohmann::json defined in logos_json.h (part of the SDK). They allow module implementations to remain completely Qt-free while returning rich structured data. The parser maps them to the same LIDL shapes as QVariantMap/QVariantList, but sets the jsonReturn flag on the method so the generator emits an nlohmannToQVariant() conversion in the glue layer.

The parser uses a state machine to find the target class, track access specifiers (public/private/protected), and extract method declarations. It skips constructors, destructors, typedefs, using declarations, std::function members, and non-method statements. While scanning, it also captures any doc comment immediately above a method declaration as that method's description (see Method documentation).

Module metadata (name, version, description, dependencies) comes from metadata.json, not from the header.

Method documentation

A doc comment written directly above a method's declaration in the impl header becomes that method's description, stored on MethodDecl.description in the shared AST and emitted into the description field of each getMethods() entry. Because getMethods() is what the framework's getPluginMethods() returns, the description flows — with no extra call — to lm methods, logoscore module-info, and Basecamp's Methods list.

Only doc comments are captured: /// line comments and /** … */ / /*! … */ block comments. Plain // and /* … */ comments are ignored, so section separators and incidental notes don't leak into the API. A multi-line doc comment is preserved with its line breaks (markers stripped, lines joined with \n; leading/trailing blank lines dropped, interior blank lines kept), and only comments immediately adjacent to the declaration (no blank line in between) attach.

class WalletModuleImpl : public LogosModuleContext {
public:
    /// Transfers `amount` from the active account to `toAddress`.
    /// Returns the resulting transaction hash.
    std::string transfer(const std::string& toAddress, int64_t amount);
};

→ the transfer entry in getMethods() gains "description": "Transfers amountfrom the active account totoAddress.\nReturns the resulting transaction hash." (the two lines preserved, joined with \n)

A method with no doc comment simply has no description field. Methods introspected purely via Qt's QMetaObject (legacy Q_INVOKABLE modules with no generated dispatch) carry no comments at runtime and therefore have no description.

Event documentation

Events are the subscribe-half of a module's API (methods are the call-half), and document the same way. A doc comment directly above an event declaration in the logos_events: section (see Event Emission below) becomes that event's description, stored on EventDecl.description in the shared AST and emitted into the description field of the event's entry in getMethods().

getMethods() returns the module's whole interface — methods and events — with each entry tagged by a "type" field ("method" or "event"). Events ride inside getMethods() deliberately: there is no separate getEvents() vtable method, so LogosProviderObject's vtable layout never shifts and old/new hosts and modules stay binary-compatible (see Why events live in getMethods() below). The framework then offers three filtered views over that one call — getPluginMethods() (entries that aren't events), getPluginEvents() (type == "event"), and getPluginInterface() (everything) — so the description flows, with no extra provider call, to lm events, logoscore module-info's Events section, and Basecamp's Interface screen.

The capture rules are identical to methods: only /// line comments and /** … */ / /*! … */ block comments are captured (plain // and /* … */ are ignored); multi-line comments preserve their line breaks (markers stripped, joined with \n, leading/trailing blanks dropped); only comments immediately adjacent to the declaration attach.

logos_events:
    /// Emitted once the user has authenticated.
    /// Carries the freshly issued session token.
    void userLoggedIn(const std::string& userId, const std::string& token);

→ the userLoggedIn entry in getMethods() gains "type": "event" and "description": "Emitted once the user has authenticated.\nCarries the freshly issued session token."

An event entry carries type: "event", name, signature, parameters[] (each with type and name), and — when documented — description. Unlike a method entry it has no returnType or isInvokable: events are void, fire-and-forget. Events are a universal (--from-header) concept. (An entry with no "type" is treated as a method, so a module built against a pre-events SDK simply reports zero events.)

An event's description may also be supplied out-of-band via an optional description field on the corresponding metadata.json events[] entry (the doc comment takes the same role for both sources).

Event Emission via logos_events:

Universal modules declare events in a Qt-signals:-style section parsed by the codegen. The same method name appears on both sides — declared in logos_events:, called directly to emit:

#include <logos_module_context.h>

class MyModuleImpl : public LogosModuleContext {
public:
    void doWork() {
        userLoggedIn("alice", 12345);            // typed emit, same name
    }

logos_events:                                     // expands to `public:`; recognised by impl_header_parser
    void userLoggedIn(const std::string& userId, int64_t timestamp);
    void messageReceived(const std::string& from, const std::string& body);
};

impl_header_parser.cpp recognises the raw logos_events: token (before preprocessing) and populates ModuleDecl.events with one EventDecl per prototype. Three artifacts get emitted from this:

1. <name>_events_cdylib.cpp — Qt-MOC-style definitions of each declared event method on the impl class. Bodies marshal typed args into an nlohmann::json array and call this->emitEventImpl_("<event>", &args), a protected helper on LogosModuleContext:

   void MyModuleImpl::userLoggedIn(const std::string& userId, int64_t timestamp) {
       nlohmann::json args = nlohmann::json::array();
       args.push_back(userId);
       args.push_back(timestamp);
       emitEventImpl_("userLoggedIn", &args);
   }
   

   (This used to be a <name>_events.cpp marshalling into a QVariantList, back when the emitter it fed was a Qt provider object. A universal module's impl side is Qt-free, so the payload is JSON and the file carries the _cdylib suffix.)

2. Emit-callback wiring — <name>_module_impl.cpp, the generated C-ABI export TU, installs the callback through _logos_codegen_::maybeSetEmitEvent alongside maybeSetModuleName / maybeSetContext / maybeSetLogosModules. The lambda casts the void* back to nlohmann::json, dumps it, and hands it to the logos_module_emit_cb the host registered via logos_module_set_emit_callback:

   _logos_codegen_::maybeSetEmitEvent(lidlImpl(),
       [](const std::string& name, void* args) {
           const nlohmann::json* payload = static_cast<const nlohmann::json*>(args);
           std::lock_guard<std::mutex> lock(g_emitMutex);
           if (g_emitCb)
               g_emitCb(name.c_str(), payload ? payload->dump().c_str() : "[]", g_emitUd);
       });
   

   (Was a <name>_qt_glue.h lambda forwarding to LogosProviderBase::emitEvent(QString, QVariantList); that glue is the retired shape described under Generated Output below.)

3. <name>.lidl sidecar — a serialised view of the module's declared events (using lidlSerialize, which since the frontend extraction is lidl::serialize in the logos-lidl library, re-exported by experimental/lidl_compat.h; the lidl_serializer.cpp that used to hold it is gone from this repo):

   module my_module {
     event userLoggedIn(userId: tstr, timestamp: int)
     event messageReceived(from: tstr, body: tstr)
   }
   

   buildPlugin.nix ships this at $out/share/logos/<name>.lidl. buildHeaders.nix passes it to the consumer-side codegen via --events-from, which adds typed on<EventName>(callback) accessors to the generated <Module> wrapper (one per declared event, callback-arg types respect --api-style).

Module metadata (name, version, description, dependencies) still comes from metadata.json, not from the header.

Generated Output

Historical. <name>_qt_glue.h / <name>_dispatch.cpp were emitted by lidl_gen_provider, which is deleted. A module now emits the logos_module_* C ABI (--backend cdylib) and logos-qt-host-generator turns that into a Qt plugin. The sections below describe the retired shape and are kept because the onInit wiring they document still applies to the cdylib glue.

Provider Glue (<name>_qt_glue.h)

Contains two classes:

1. ProviderObject — inherits LogosProviderBase, holds an instance of the impl class (m_impl). Each public method is wrapped with type conversion:

• Qt parameters → C++ std parameters (e.g., QString.toStdString())
• Call m_impl.method(...)
• C++ std return → Qt return (e.g., QString::fromStdString(result))
• For jsonReturn methods (returning LogosMap/LogosList), the glue calls a generated nlohmannToQVariant() recursive helper to convert nlohmann::json → QVariant/QVariantMap/QVariantList
• Always overrides onInit(LogosAPI*) to (a) copy the three runtime-injected properties (modulePath, instanceId, instancePersistencePath) into the impl when it inherits from LogosModuleContext, and (b) construct a per-module LogosModules (from generated_code/logos_sdk.h) owned by the provider, threading its pointer through the same context base. Both wire-ups go through SFINAE'd helpers in logos_module_context.h (_logos_codegen_::maybeSetContext / maybeSetLogosModules), so non-inheriting impls compile unchanged and the LogosAPI never escapes the provider.

2. Plugin — QObject subclass implementing PluginInterface and LogosProviderPlugin. Carries Q_PLUGIN_METADATA and Q_INTERFACES. Its createProviderObject() factory returns a new ProviderObject instance.

Dispatch (<name>_dispatch.cpp)

Implements two methods on the ProviderObject:

1. **callMethod(methodName, args)** — string-based dispatch table. For each method, extracts args from QVariantList, calls the typed wrapper, returns result as QVariant. Void methods return QVariant(true).

2. **getMethods()** — returns a QJsonArray describing the module's whole interface — both methods and events. Each entry carries a "type" of "method" or "event":

   • method entries have type: "method", name, signature, returnType, isInvokable, parameters[] (with type and name), and — when the declaration has a doc comment — description (see Method documentation).
   • event entries (one per logos_events: declaration) have type: "event", name, signature, parameters[], and an optional description (see Event documentation). They omit returnType/isInvokable — events are void.

   The framework slices this single array into getPluginMethods() (non-event entries), getPluginEvents() (type == "event"), and getPluginInterface() (everything), which is what surfaces in lm methods/lm events, logoscore module-info, and Basecamp's Interface screen.

Why events live in getMethods()

Folding events into getMethods() — rather than adding a sibling getEvents() virtual — is a deliberate ABI choice. LogosProviderObject is the in-process vtable contract between a host/runtime and a loaded module; inserting a new virtual would shift every later vtable slot and break any mix of old/new host and module binaries. Reusing the existing getMethods() slot keeps the vtable byte-for-byte stable: a new host reading an old module just sees no type: "event" entries (so zero events), and an old host reading a new module ignores the "type" field (events show up in its method list — cosmetic, never a crash). Legacy Qt modules declare no events, so their getMethods() is methods-only.

Client Stubs (<name>_api.h + <name>_api.cpp)

Generated from LIDL (not from --from-header). Each module gets one <Module> wrapper class whose signature shape is picked by the consumer's build via --api-style:

--api-style	Wrapper signatures
qt (default)	QString / QStringList / QVariantList / QVariantMap / int / LogosResult
lp	std::string / std::vectorstd::string / LogosMap / LogosList / int64_t / StdLogosResult, over the Qt-free logos-protocol C ABI

(std — the same signatures over a QVariant / LogosAPIClient body — was retired; --api-style=std is now an error.)

Both styles provide:

• Typed sync methods. The Qt style calls LogosAPIClient::invokeRemoteMethod() and converts the QVariant result; the lp style calls logos::LpClient::invoke() and converts the nlohmann::json result — no Qt anywhere in the call.
• Async overloads with callback + timeout.
• Event subscription. The Qt style exposes the generic on(eventName, callback) channel plus one typed on<EventName>(callback) adapter per declared event; the std style exposes the typed adapters over logos::LpClient::subscribe, holding each RAII LpSubscription for the wrapper's lifetime. (Both styles once also emitted setEventSource() / eventSource() / trigger() — a consumer-side emission surface. It is gone: test_lidl_gen_client.cpp asserts no trigger( is emitted. A module emits its own events through logos_events:, never through a dependency's wrapper.)

The lp wrappers marshal over the logos-protocol C ABI (lp_*) instead, so the calling translation unit needs zero Qt headers and links no qt-sdk. (The retired std style was the one that shared invokeRemoteMethod with the Qt path and generated a Qt<->std conversion inline in its .cpp.) Both styles emit the same filename (<name>_api.h / <name>_api.cpp) and the same class name (<Module>) — the two are mutually exclusive at build time. No _api_std.{h,cpp} files are ever produced.

Umbrella files (logos_sdk.h / logos_sdk.cpp) aggregate every dep into a flat LogosModules struct — one accessor per metadata.json#dependencies entry, nothing else:

struct LogosModules {
    LogosAPI*    api;
    SomeDep      some_dep;       // one per declared dependency
    // ...
};

Only the modules explicitly listed as dependencies appear. The runtime's core_manager is intentionally NOT exposed here — apps that need to manage the core (basecamp, logoscore) use liblogos' C API directly, not a typed RPC wrapper.

Features & Requirements

LIDL Pipeline

The whole frontend now lives in the standalone logos-lidl repo; this generator links it and reaches it through experimental/lidl_compat.h, which re-exports the three stages below under their historical lidl* names. There is no separately callable lexer entry point here any more — lidlTokenize was part of the embedded copy that was deleted.

1. Lexer — tokenizes source into keywords, identifiers, string literals, symbols (internal to lidl::parse)
2. Parser (lidlParse → lidl::parse) — recursive descent parser producing a ModuleDecl AST
3. Validator (lidlValidate → lidl::validate) — checks for duplicate names, unknown type references, builtin shadowing, duplicate parameters
4. Serializer (lidlSerialize → lidl::serialize) — pretty-prints a ModuleDecl back to LIDL text (useful for roundtrip testing)

Impl Header Pipeline

1. parseImplHeader — reads metadata.json + C++ header, produces a ModuleDecl
2. Same generation functions as LIDL path

Backwards Compatibility

• The remaining generator modes (--metadata, plugin path) continue to work unchanged via runPluginIntrospectMode() (plugin_introspect.cpp; was legacy/main.cpp's legacy_main()). --provider-header (the LOGOS_METHOD dispatch behind interface: "provider") was REMOVED — every provider now goes through the module-impl C ABI; the flag is refused with a message pointing at interface: "universal"
• The new --from-header and --lidl modes are additive
• Generated plugins implement both PluginInterface (for lm introspection) and LogosProviderPlugin (for new-API provider creation)
• The runtime (logos-liblogos) already supports both old and new plugin types via qobject_cast detection

Conversion Helper Generation

Conversion helpers are only emitted when needed:

• String vector helpers (lidlToQStringList, lidlToStdStringVector) — emitted when the module uses [tstr] parameters or return types
• nlohmann→Qt helper (nlohmannToQVariant) — emitted when any method has jsonReturn = true (i.e., the impl returns LogosMap or LogosList). This recursive function converts nlohmann::json objects, arrays, strings, numbers, and booleans to their QVariant equivalents.