mirror of
https://github.com/logos-co/logos-cpp-sdk.git
synced 2026-08-27 15:51:10 +00:00
Dario LipicarandClaude Opus 5
acea0d24e2
* feat(codegen): a lossless Qt type mapping — typed containers and optionals
`lidlTypeToQt` answered four different LIDL types with one Qt name. `[uint]`,
`[bstr]`, `[[uint]]` and `[any]` were all QVariantList; `{tstr: uint}` and
`{tstr: any}` were both QVariantMap; every `?T` was a bare QVariant. A Qt
consumer therefore lost, on the SAME contract, types that the std consumer next
door kept — it could not tell `?tstr` from `?uint`, and got no compile-time
check on any element.
The table is now recursive:
[T] QList<qtOf(T)> ([tstr] stays QStringList)
{tstr: V} QMap<QString, qtOf(V)>
?T std::optional<qtOf(T)> (through optionalValueType,
so ??T stays two-state)
any QVariant — KEPT, deliberately
`any` is the one row that must not widen: QVariant is the only Qt type that
holds bytes AND an exact uint64 AND arbitrary nesting at once, so every
narrower spelling would lose what it was chosen to carry. The rule is applied
at the LEAF, so anything whose element type bottoms out at `any` keeps the
QVariant-family spelling at every depth — `[any]` is QVariantList, `[[any]]`
still is, `{tstr: [any]}` is QVariantMap, `?any` is QVariant.
THE TRAP, and why this is not just a rename. A widened name must never reach
QVariant::fromValue / qvariant_cast / logos::qt::toWire as a WHOLE value.
logos-protocol's qvariantToNlohmann matches a CLOSED userType() set:
QList<qulonglong> is in none of it, so it serialises to JSON null. The decode
fails just as quietly — qvariant_cast<QList<qulonglong>> of a QVariantList
yields an EMPTY list. Neither direction warns. So every widened slot is encoded
and decoded by a generator-emitted ELEMENT LOOP, the shape the record cases
already used, and `lidlQtNeedsElementLoop` is the single predicate that decides
which slots need one.
The emitted loops take their source as a lambda PARAMETER, not a body-local
binding. They nest (`[[uint]]`), every level wants the same short names, and a
local — or a range-for over a name the loop itself declares — is then
self-referential: it compiles and reads uninitialised memory. Measured: three
round-trip tests died on SIGTRAP before the argument form.
THE STRING-KEYED EMITTER IS FROZEN, ON PURPOSE. generator_lib is keyed on flat
type NAMES (lidl_to_json flattens the contract before it gets there, because
that emitter also serves the metaobject-introspection path), so it cannot
derive the levels an element loop needs without parsing C++ type names back
into a tree. Every widened spelling is folded back to the name it produced
before (legacyQtBase), which keeps BOTH surfaces it feeds byte-for-byte
unchanged: the legacy Qt consumer, and the Qt-free lp one whose table is
DERIVED from it through mapParamTypeStd. Verified by generating a
28-method contract through both before and after: the diff is empty. The
widened types are spent in the TypeExpr-driven emitters instead
(lidl_gen_client.cpp here, lidl_gen_qt_consumer.cpp in logos-qt-sdk).
Also here, because both are consequences of the table becoming recursive:
* lidlTypeToQt gained a record-name HOOK. A wrapper nests its record structs
in the wrapper class, so a type written outside that scope must qualify
them — and the emitters used to do that by matching the three shapes that
could mention a record on the finished string. `?Point` and
`QList<QList<Point>>` are now spellable, so the qualification happens
during the walk, at the one place that knows a name is a record.
* lidlTypeToLidlText — the LIDL contract spelling of a type. Unused here; the
commit that follows puts getMethods() on it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* feat(cdylib): getMethods publishes the LIDL contract vocabulary, not Qt names
A module's published metadata — `returnType`, `parameters[].type`, `signature`
— answered in Qt type names. Two things wrong with that, and the second is the
one that matters:
* a cdylib module is Qt-FREE. It described itself in the types of a language
it does not use, to readers (`lm`, logoscore's method listing, basecamp's
module inspector) that are showing a human what the module offers.
* it was LOSSY. `[uint]`, `[bstr]` and `[any]` are three different LIDL types
and all three published as the single word QVariantList, so the listing
could not be read back as a contract. That is now `[uint]`, `[bstr]`,
`[any]`; `{tstr: uint}`; `? tstr`; and a record publishes its declared
name.
WHY THIS IS SAFE — checked, not assumed. The historical objection is recorded
in the function this replaces: these strings are read as METATYPES, and
emitting a record's struct name here once made the host SIGSEGV. Nothing in the
current runtime does that. logos-plugin-qt's QtProviderObject dispatches on
`method.returnMetaType()` / `parameterMetaType(i)` — the QMetaObject, never
this JSON — and every remaining reader treats these fields as opaque text:
logos-module's `lm` prints them, logoscore's output.cpp prints them, basecamp's
CoreModuleManager forwards the JSON to QML, and the plain wire's json_mapping
only round-trips them. Nothing anywhere builds a QMetaObject from this
metadata.
The spelling comes from lidlTypeToLidlText, which mirrors logos-lidl's
serializeTypeExpr. It is a COPY, because that function is file-local to
logos-lidl's serializer.cpp and the public headers expose no type printer —
so instead of hoping, the pairing is ASSERTED: the test round-trips each shape
through `lidl::serialize` and reads the type text back out of the emitted
`.lidl`. When logos-lidl exports a printer, delete the copy and call it.
Not fixed by this, and not attempted: the Rust SDK's provider generator has its
own `qt_type_name` writing the same JSON, so the two languages now disagree
about how a module describes itself. That is a cross-repo change.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* test(doctests): the generator round-trip pins the lossless Qt spellings
`cpp-sdk-generator-roundtrip.test.yaml` is a CI gate
(.github/workflows/doctests.yml), and two of its `expect_contains` were
pinned to the type names the Qt consumer produced BEFORE the lossless
mapping:
QStringList labels(const QVariantList& ids
QVariant nearest(const Point& p, QVariant limit
The generator now emits `QList<qulonglong>` and
`std::optional<Point>` / `std::optional<qulonglong>` for those slots, so both
assertions failed. The `nearest` step's `run` grep was pinned the same way
(`QVariant nearest`), so the line it was supposed to assert on was not even
in the output being searched.
Verified by running the spec's own steps against the generator built from
this commit: 10 run-steps, 0 failures. The `[uint]` -> QList<qulonglong> and
`?T` -> std::optional<T> lines were read out of the real
`consumer/sensor_module_api.h` and `geometry/geometry_module_api.h`, not
written from the mapping table.
Prose too, in three places that described the old table: the Flow-3 type
mapping ("other arrays -> QVariantList"), the composite-types intro
("optionals ... stay QVariantMap / QVariant"), and the composite-signature
step. They now say what the mapping actually is — one LIDL type, one C++
spelling, with `any` the single deliberate exception — and `nearest` is
called out as the one signature carrying both halves of the optional
mapping.
`doctests/outputs/cpp-sdk-generator-roundtrip.md` carries the same prose
corrections. That tree is hand-pinned and CI never diffs it, which is
exactly why it must be corrected by hand.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(generator): the consumer wrapper comes from the contract, not from getMethods
`logos-cpp-generator <plugin> --module-only` — the invocation
logos-plugin-qt's generate-module-headers.sh makes for every module's lp
wrapper — built that wrapper's whole type surface out of the plugin's
PUBLISHED `getMethods()` metadata. It now builds it out of the module's `.lidl`
contract, the file the same invocation already passes as `--events-from`.
WHY THIS IS A DEFECT AND NOT A PREFERENCE. generator_lib is keyed on flat type
NAMES, and mapParamType / mapReturnType fall back to QVariant for a name they
do not recognise (generator_lib.cpp:142 and :153). So the wrapper's types
depend on the VOCABULARY a module happens to publish its metadata in, and a
vocabulary this emitter has no row for degrades to QVariant — LogosMap on the
lp surface — with no diagnostic at any layer. It is a machine reader of a
listing that every other consumer treats as human-facing text, and it fails
silently.
It was measured, not theorised. 621772a made the cdylib backend publish the
LIDL contract vocabulary (`tstr`, `[uint]`, `result`, `? tstr`) in place of Qt
type names, because that listing is what `lm`, logoscore and basecamp show a
human and Qt names are the wrong answer for a Qt-free module. Every
`interface: "universal"` module's lp wrapper collapsed:
logos-test-modules' `checks.unit-tests-new-api` went PASS -> FAIL, and the
compiler said exactly why —
error: no viable conversion from 'LogosMap' to 'StdLogosResult'
StdLogosResult r = modules().test_basic_module.resultWithMap();
`result` is not a name mapReturnType knows, so it became QVariant, so it became
LogosMap. Bisected to exactly 621772a (5ffd90b passes, dd52d9d fails).
THE FIX IS TO STOP READING THAT VOCABULARY, not to learn a second one.
`int` means a 32-bit Qt int in one table and a 64-bit LIDL integer in the
other, and the reader cannot tell from the string which table it is holding —
a merged table would silently mistype every integer on every module. The
contract has no such ambiguity: it is a TypeExpr tree, and lidl_to_json is the
single place it is flattened. Taking methods from it makes this path emit the
same wrapper as `--general-only --dep <name>=<name>.lidl`, which is what
buildHeaders.nix already runs under cross-compilation and for the entire Qt
surface. Contract-first, on every platform, for every surface.
WHAT CHANGED, exactly:
* loadEventsFromLidl -> loadContractFromLidl. It already parsed the whole
contract and threw the methods away; it now returns them, after the same
lidlCheckRecords + lidlInjectIdentity + noteOptionalPositionalSlots that
main.cpp's --dep path applies. Identity is injected rather than read,
matching the provider side (main.cpp's --backend cdylib), so the two cannot
disagree about name() / version().
* A sidecar that is NAMED BUT MISSING is now refused (exit 2), and an
unreadable or malformed one is fatal (exit 4). Both used to be shrugged off
— which shipped a wrapper with no typed events, and would now ship one with
no typed methods, in the silently-empty shape generate-module-headers.sh
exists to refuse.
* The plugin is STILL LOADED. That load is the dlopen check this path
performs (exit 3 on an SDK/ABI skew) and it is unchanged; what the plugin
says about itself is now compared against the contract instead of believed,
and a divergence — a stale sidecar — is reported by name on stderr. Only
`isInvokable` entries are compared: a cdylib publishes its events into the
same array, tagged `"type": "event"`, and both emitters already skip those.
* A module with NO contract keeps introspection — a handcrafted Qt plugin's
QMetaObject is still the only description of its API that exists, and Qt
type names are the right vocabulary for it — but a listing spelled in the
LIDL vocabulary with no contract to go with it is now REFUSED (exit 7)
instead of silently producing the untyped wrapper. That combination is only
reachable by hand: buildHeaders.nix always passes the flag when the sidecar
exists, and it is the shape the developer guide used to suggest. The two
vocabularies are not distinguishable in general, which is the whole reason
this emitter must read only one — but they do not have to be: the words
they share (`int`, `bool`) are all in the known table and never reach the
fallback, so the check keys on the LIDL half Qt has no word for at all
(`tstr`, `bstr`, `uint`, `float64`, `result`, `any`, and anything starting
`[`, `{` or `?`). No Qt type is spelled that way, so it cannot false-fire;
a false negative is just the old behaviour.
THE ENUMERATION, because two previous ones missed this reader. Searching for
who greps `returnType` is what missed it; the question is what the data FLOWS
INTO. Every consumer of a published getMethods array in the workspace:
MACHINE (one, and it is this one)
logos-cpp-sdk cpp-generator/plugin_introspect.cpp, reached only through
logos-plugin-qt's generate-module-headers.sh / buildHeaders.nix.
HUMAN-READABLE OR OPAQUE PASSTHROUGH (all of them)
logos-module's `lm` (prints; --json re-emits verbatim), logoscore-cli's
client/output.cpp (prints) and core_service_dispatch.cpp (forwards),
logos-logoscore-tui (formats one line per method), logos-module-viewer
(reads the QMetaObject directly, not this JSON), basecamp's
CoreModuleManager / MainUIBackend (hands the JSON string to QML),
logos-protocol's json_mapping.cpp and qvariant_rpc_value.cpp (round-trip
the strings unread).
PRODUCERS, for completeness: lidl_gen_cdylib.cpp (LIDL vocabulary),
logos-plugin-qt's QtProviderObject (Qt names, from the QMetaObject) and
lidl_gen_cdylib_glue.cpp (forwards the cdylib's), logos-rust-sdk's
rustgen_provider.rs (still Qt names — the two languages disagree, as
621772a noted), and logos-protocol's ModuleProxy, which appends derived
name()/version() entries spelled `QString`. None of that reaches a type
decision any more, which is the point of the change.
Build-system paths checked and clear: `<plugin> --module-only` is invoked
from exactly one place in the workspace (generate-module-headers.sh:60);
LogosModule.cmake, buildPlugin.nix and mkLogosModuleTests.nix all use
`--general-only`, which is contract-driven already; the doctests' `--lidl
--module-only` is a different mode entirely.
VERIFIED.
`nix build path:./repos/logos-test-modules#checks.aarch64-darwin.unit-tests-new-api`
with this SDK overridden in (plus the logos-lidl overrides the branch needs at
the qt-sdk and plugin-qt nodes) — 32 passed, 0 failed. The same command against
this branch's HEAD fails to compile, as above. The build log shows the path
taken, per module:
Detected new-API plugin (LogosProviderPlugin), using getMethods() — 43 methods
Using the module's LIDL contract for the method surface — 41 methods
(the plugin's published listing is a description, not a type source)
The refusal, measured by hand against a real LIDL-publishing plugin
(test_basic_module, built from this branch) because no check exercises a
hand-run invocation:
no --events-from -> exit 7, nothing written, the message above naming
8 offending slots
with --events-from -> exit 0, 41 typed methods, 69 `std::string` in the
emitted lp header
a pre-621772a build of the SAME module (Qt-name listing), no --events-from
-> exit 0, still generates, still typed — the refusal does
not fire on the vocabulary this emitter can read
nix/tests-generator-cli.nix gains the two CLI-surface cases this adds: a
`--events-from` naming a file that does not exist is refused with that
sentence, and — the control that makes it mean something — the same command
with a READABLE contract gets past the flag and fails on the plugin instead. No
plugin is needed for either: the contract is loaded before the plugin is
opened.
logos-cpp-sdk's own checks (tests, generator-cli, module-impl-abi): 334 of 334.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
399 lines
28 KiB
Markdown
399 lines
28 KiB
Markdown
# Logos Code Generator — Project Description
|
|
|
|
## Project Structure
|
|
|
|
```
|
|
cpp-generator/
|
|
├── main.cpp # Entry point — `--umbrella`/`--general-only` mode, dispatch to the LIDL backends or plugin introspection
|
|
├── CMakeLists.txt # Build config
|
|
├── compile.sh # Standalone build script
|
|
├── metadata_dependencies.h # What a metadata.json `dependencies[]` array declares
|
|
├── generator_lib.h/cpp # Shared emitter library: type mapping, wrapper + umbrella emission
|
|
├── lidl_to_json.h/cpp # ModuleDecl → the JSON surface generator_lib consumes
|
|
├── plugin_introspect.h/cpp # runPluginIntrospectMode() — the QPluginLoader path
|
|
│ # (plugin/metadata modes). Was `legacy/`, which was
|
|
│ # never a library: one exported symbol, compiled into
|
|
│ # this binary and reached by fallthrough.
|
|
├── experimental/ # C++/Qt-specific generator backends
|
|
│ ├── lidl_compat.h # Bridges the backends onto logos-lidl's std AST
|
|
│ ├── lidl_emit_common.h/cpp # LIDL type → Qt/std type-name mapping
|
|
│ ├── lidl_gen_client.h/cpp # Typed client stub generation (+ Doxygen /// docs)
|
|
│ ├── lidl_gen_cdylib.h/cpp # cdylib module-impl C-ABI export generation
|
|
│ └── impl_header_parser.h/cpp # C++ impl header → lidl::ModuleDecl
|
|
│ # The lexer/parser/AST/serializer/validator now live in the standalone
|
|
│ # logos-lidl repo (linked via find_package(logos-lidl)); the Qt glue
|
|
│ # emitters live in logos-qt-sdk's logos-qt-generator.
|
|
└── docs/ # This documentation
|
|
```
|
|
|
|
## Components
|
|
|
|
### Entry Point (`main.cpp`)
|
|
|
|
Checks for `--from-header` or `--lidl` flags before creating `QCoreApplication`. If neither is present, falls through to `runPluginIntrospectMode()` in `plugin_introspect.cpp`.
|
|
|
|
### LIDL frontend — `logos-lidl` (consumed as a library)
|
|
|
|
The lexer, parser, AST, serializer, and validator are **no longer embedded here** — they live in the standalone **`logos-lidl`** repo, the language-neutral (Qt-free) common frontend every Logos SDK shares (C++ here, Rust in logos-rust-sdk, …). cpp-generator links it via `find_package(logos-lidl)` and reaches it through `experimental/lidl_compat.h`.
|
|
|
|
`logos-lidl` exposes (`namespace lidl`):
|
|
|
|
- `lidl::parse(std::string) → ParseResult` (`ModuleDecl` + error/line/column)
|
|
- `lidl::serialize(ModuleDecl) → std::string`
|
|
- `lidl::validate(ModuleDecl) → ValidationResult`
|
|
- the **AST**: `TypeExpr` (`Kind`: Primitive/Array/Map/Optional/Named, `name`, `elements`), `ParamDecl`, `FieldDecl`, `MethodDecl` (name, params, returnType, `description`, `jsonReturn`, `resultReturn`), `EventDecl` (name, params, `description`), `TypeDecl`, `ModuleDecl`. (logos-lidl also exposes an AST↔JSON bridge and a C ABI that the Rust SDK consumes over FFI — not used by this generator.)
|
|
|
|
The `.lidl` grammar (defined in logos-lidl):
|
|
|
|
```
|
|
module = "module" IDENT "{" body "}"
|
|
body = (metadata | type_def | method_def | event_def)*
|
|
metadata = "version" STRING | "description" STRING | "category" STRING
|
|
| "depends" "[" (IDENT ("," IDENT)*)? "]"
|
|
type_def = "type" IDENT "{" field* "}"
|
|
field = "?"? IDENT ":" type_expr
|
|
method_def = "method" IDENT "(" params ")" "->" type_expr ("description" STRING)?
|
|
event_def = "event" IDENT "(" params ")" ("description" STRING)?
|
|
params = (IDENT ":" type_expr ("," IDENT ":" type_expr)*)?
|
|
type_expr = IDENT | "[" type_expr "]" | "{" type_expr ":" type_expr "}"
|
|
| "?" type_expr
|
|
```
|
|
|
|
Validation (in logos-lidl) checks: empty module name, duplicate type/method/event names, builtin type shadowing, unknown named type references, duplicate parameter names. Serialization round-trips `ModuleDecl` back to `.lidl` text (incl. the trailing `description "…"` clause).
|
|
|
|
### Compat shim (`lidl_compat.h`)
|
|
|
|
Bridges the existing Qt-flavored backends onto logos-lidl's std AST so they compile unchanged:
|
|
|
|
- brings the `lidl::` AST types into the global scope the backends use (via `using`)
|
|
- `qs(std::string) → QString` plus a `QTextStream << std::string` overload, so emission of AST string fields just works
|
|
- name-compatible shims `lidlParse` / `lidlSerialize` / `lidlValidate` over `lidl::parse`/`serialize`/`validate`
|
|
|
|
### Type Mapping (`lidl_emit_common.h/cpp`)
|
|
|
|
- `lidlTypeToQt(TypeExpr)` / `lidlTypeToStd(TypeExpr)` — LIDL type → Qt / std type-name strings
|
|
- `lidlIsStdConvertible(TypeExpr)` — whether a type has a pure-C++ (Qt-free) representation
|
|
- `lidlToPascalCase(name)` — converts `snake_case` to `PascalCase`
|
|
|
|
### Optionality
|
|
|
|
`?T` is **two-state**: a value of `T`, or empty. Never three-state — "one LIDL type ↔ one
|
|
type per language" leaves nowhere for a third state, because every target has exactly one
|
|
empty inhabitant.
|
|
|
|
**Two spellings, one meaning.** A record field may be written `? name: T` (the flag) or
|
|
`name: ?T` (the type kind); the spec binds them to the same declaration, so they MUST emit
|
|
identical code. Backends never answer this themselves — logos-lidl's `fieldIsOptional(f)` /
|
|
`fieldValueType(f)` (re-exported by `lidl_compat.h`) are the one place the two are
|
|
reconciled. **Reading `f.optional` or `f.type.kind == Optional` on its own is a bug.**
|
|
|
|
**Wire rule.** Absent and explicit null are the *same* state on decode and *different* on
|
|
encode:
|
|
|
|
| | empty is spelled |
|
|
|---|---|
|
|
| decode, optional slot | absent **or** null → empty |
|
|
| decode, required slot | absent and null are both still errors |
|
|
| encode, **named** slot (a record field) | the key is **omitted** |
|
|
| encode, **positional** slot (argument, return, event param) | `null` — there is no key to omit, and arity must never change |
|
|
|
|
A round trip therefore **canonicalises**: a peer that sent `"f": null` gets the key back
|
|
omitted. A present-but-wrong-typed value is still an error — optional widens the domain by
|
|
exactly one inhabitant, it does not switch type checking off.
|
|
|
|
Per surface:
|
|
|
|
| Surface | `?T` | Notes |
|
|
|---|---|---|
|
|
| cdylib / std (`lidlTypeToStd`, `lidl_gen_cdylib`) | `std::optional<T>` | encoded by logos-protocol's `Codec<std::optional<T>>`; key omission is the record emitter's job (a codec never sees the slot) |
|
|
| `?any` / `?{tstr: any}` / `?[any]` | `LogosMap` / `LogosList` | collapses: `nlohmann::json` already carries `null`, so wrapping it would make the slot three-state |
|
|
| Qt (`lidlTypeToQt`, `lidl_gen_client`) | `QVariant` | two-state (an invalid QVariant is Qt's empty inhabitant) but **untyped** — Qt has no optional template |
|
|
| legacy consumer, record **field** (`lidl_to_json` + `generator_lib`) | `QVariant` (Qt) / `std::optional<T>` (Lp) | same answer as the client-stub and cdylib backends respectively; the alias collapse above applies on Lp |
|
|
| legacy consumer, **positional** slot (param, return, event param) | `QVariant` (Qt) / `LogosMap` (Lp) | still flattened — see Known Limitations |
|
|
| header-first (`impl_header_parser`) | `std::optional<T>` ↔ `?T` | `std::optional<std::optional<T>>` has no LIDL type; it collapses to `?T` and is reported on stderr |
|
|
|
|
### Client stubs (`lidl_gen_client.h/cpp`)
|
|
|
|
- `lidlMakeHeader(ModuleDecl)` / `lidlMakeSource(ModuleDecl)` — typed `<Module>` client wrapper; each method (and its `…Async` twin) carries a Doxygen `///` comment generated from the method's `description`
|
|
- `lidlGenerateMetadataJson(ModuleDecl)` — generates metadata.json content
|
|
|
|
### cdylib backend (`lidl_gen_cdylib.h/cpp`)
|
|
|
|
Emits the Qt-free half of a universal C++ cdylib module:
|
|
|
|
- `lidlCdylibSupported(ModuleDecl)` — gate to the std-convertible (Qt-free) type subset
|
|
- `lidlMakeModuleImplExports(...)` — the `logos_module_impl.h` C-ABI export wrapper around the universal impl class (compiled into the module's cdylib; dispatches via nlohmann::json)
|
|
- `lidlMakeEventsSourceCdylib(...)` — typed `logos_events:` bodies marshalling into nlohmann::json
|
|
|
|
### Per-build API-style choice (`generator_lib.{h,cpp}`)
|
|
|
|
The codegen exposes **one** wrapper class per module — `<Module>` — with signatures that match the API style picked at the consumer's build time. The two styles are mutually exclusive (no composite output):
|
|
|
|
| `--api-style` | Wrapper signatures |
|
|
|---|---|
|
|
| `qt` (default) | `QString` / `QStringList` / `QVariantList` / `QVariantMap` / `int` / `LogosResult` |
|
|
| `lp` | `std::string` / `std::vector<std::string>` / `LogosMap` / `LogosList` / `int64_t` / `StdLogosResult`, over the Qt-free logos-protocol C ABI |
|
|
|
|
(A third value, `std` — std signatures over a `QVariant` / `LogosAPIClient` body — was retired; the generator now rejects `--api-style=std` instead of aliasing it.)
|
|
|
|
Both styles emit:
|
|
|
|
- A `<Module>` client class with sync method shapes + matching `<method>Async(...)` overloads.
|
|
- The std variant additionally inlines Qt↔std conversion in its `.cpp` so the caller's translation unit needs zero Qt headers.
|
|
|
|
The umbrella `logos_sdk.h` is also generated per-build and aggregates every dep into a flat `LogosModules` struct — no nested view:
|
|
|
|
```cpp
|
|
struct LogosModules {
|
|
LogosAPI* api;
|
|
SomeDep some_dep; // one accessor per `metadata.json#dependencies` entry
|
|
// ...
|
|
};
|
|
```
|
|
|
|
Only the modules explicitly listed as dependencies are exposed. The runtime's `core_manager` is intentionally NOT in `LogosModules` — apps that need to manage the core do so via liblogos' C API, not via a typed RPC wrapper.
|
|
|
|
A `dependencies[]` element is either a bare name or an object carrying that name alongside the constraints an installer resolves it by — the two declare the same dependency and generate the same code:
|
|
|
|
```json
|
|
"dependencies": [
|
|
"dep_a",
|
|
{ "name": "dep_b", "version": "=1.2.3" },
|
|
{ "name": "dep_c", "version": "^2.0", "signer": "did:jwk:abc" }
|
|
]
|
|
```
|
|
|
|
Read the array through `dependencyNames()` (`metadata_dependencies.h`) rather than element by element. The umbrella is emitted by several passes over the same array — includes, constructor initialisers, members — and a pass that decides on its own what an element names can decide differently from its neighbours, yielding a member whose type was never included. That aggregate no longer compiles, and nothing catches it until a module builds against it. One reader, one answer.
|
|
|
|
`ApiStyle` enum + new helpers in `generator_lib`:
|
|
|
|
- `enum class ApiStyle { Qt, Lp }` — passed to every wrapper-emitting function.
|
|
- File-local `mapParamTypeStd` / `mapReturnTypeStd` — the std-side type-mapping table the `lp` surface exposes. Hidden from `generator_lib.h` (not part of the public surface).
|
|
- `makeHeader(moduleName, className, methods, apiStyle, events)` / `makeSource(moduleName, className, headerBaseName, methods, apiStyle, events)` — single entry points that branch on `apiStyle` internally to emit the right include block, signature shape, and conversion bridges. `methods`, `events` and `records` all come from the same `<name>.lidl` contract when the module ships one (loaded via `--events-from`); only a module with no contract is described by its plugin's `QMetaObject`. A non-empty `events` also gives the wrapper one typed `on<EventName>(callback)` adapter per declared event (callback arg types follow `apiStyle`).
|
|
- `makeUmbrellaHeaderFromDeps(deps, interfaceNames, apiStyle, originName, binding)` / `makeUmbrellaSourceFromDeps(deps, interfaceNames)` — the `logos_sdk.{h,cpp}` aggregate above. `binding` is the `UmbrellaBinding` from `--binding api|origin`: `FromApi` emits the `LogosModules(LogosAPI*)` constructor, `ExplicitOrigin` emits a default-constructible umbrella that names `originName` as the call origin and mentions no `LogosAPI` at all. They return the text; `main.cpp`'s `runUmbrellaMode` writes it. That split is what lets the aggregate be asserted on directly, without a filesystem.
|
|
|
|
Flag plumbing:
|
|
|
|
1. `metadata.json#interface == "universal"` (or `"cdylib"`) → `mkLogosModule.nix` adds `-DLOGOS_API_STYLE=lp` to `extraCmakeFlags`. Anything else (`"legacy"`, absent — and `"universal"` with `type: "ui_qml"`, which is packaged as a Qt plugin) leaves the default `qt`. The only other value that was ever accepted, `"provider"`, was removed: `logos-module-builder` now throws on it rather than silently generating no glue. `metadata.json#codegen.consumer_api_style` can override the derived answer in one direction only — a cdylib-packaged module may ask for `"qt"`; a Qt-plugin module asking for `"lp"` is refused.
|
|
2. `LogosModule.cmake` reads `${LOGOS_API_STYLE}` (default `qt`) and forwards `--api-style=${LOGOS_API_STYLE}` to the `logos-cpp-generator --general-only` invocation that writes the umbrella. Each module's Nix build emits **two** header derivations (`<name>.headers-qt` and `<name>.headers-lp`) via `buildHeaders.nix` — one `logos-cpp-generator --api-style=…` run per style, at the dep's build time. A consumer's `buildPlugin.nix` picks `dep.headers-${apiStyle}` and copies its `include/` straight into the build sandbox; no codegen runs at consume time. Nix's laziness means only the variant a downstream actually depends on is realised.
|
|
3. `parseApiStyleFlag()` in `generator_lib` parses `--api-style` once (rejecting the retired `std`); `main.cpp`'s `runUmbrellaMode` threads the resulting `ApiStyle` into `makeUmbrella*FromDeps`, and `plugin_introspect.cpp` threads it through `generateFromPlugin` (the QPluginLoader path). The directory-scraping `writeUmbrellaHeader`/`writeUmbrellaSource` pair that used to sit beside it is deleted — `makeUmbrella*FromDeps` is the only umbrella emitter now, so the two cannot drift. No per-style filenames are ever emitted; each module gets a single `<name>_api.h` + `<name>_api.cpp` pair regardless of style.
|
|
|
|
### Provider Generation — REMOVED
|
|
|
|
> The Qt provider glue emitter (`lidl_gen_provider.{h,cpp}` in logos-qt-sdk) is **deleted**. It
|
|
> wrapped a plain impl directly in a Qt provider object, skipping the language-neutral seam.
|
|
>
|
|
> A module is a plain shared library. Turning one into a Qt plugin is a downstream HOSTING step,
|
|
> and the two halves meet only at `logos_module_impl.h`:
|
|
>
|
|
> ```
|
|
> plain std impl
|
|
> --> logos-cpp-generator --backend cdylib -> logos_module_* C ABI exports
|
|
> --> logos-qt-host-generator --backend cdylib -> <name>CdylibProvider : LogosProviderBase
|
|
> (logos-plugin-qt)
|
|
> ```
|
|
>
|
|
> That seam is what lets the Rust and JS providers target the same ABI. `logos-qt-generator` still
|
|
> owns `--backend consumer` (Qt-typed dependency wrappers) and `--backend ui` (view plugins) — and
|
|
> nothing else: **both** `--backend qt` and `--backend cdylib` were removed from it and are refused
|
|
> with a message naming the replacement. `cdylib` is the one that moved rather than died: the
|
|
> HOSTING half now lives with the host, as `logos-qt-host-generator --backend cdylib` in
|
|
> logos-plugin-qt.
|
|
|
|
### Impl Header Parser (`impl_header_parser.h/cpp`)
|
|
|
|
- `parseImplHeader(headerPath, className, metadataPath, err)` — parses C++ header + metadata.json into ModuleDecl
|
|
- State machine: `LookingForClass` → `InClass` → `InPublic`/`InPrivate`/`InLogosEvents`
|
|
- The literal `logos_events:` token (defined in `logos_module_context.h` as `#define logos_events public`) opens an events section; bare prototypes inside become `EventDecl{name, params, description}` entries appended to `ModuleDecl.events` (the `description` is the doc comment immediately above the declaration, captured via `joinDocLines` exactly as for methods)
|
|
- Skips: constructors, destructors, typedefs, using, friend, enum, struct, `std::function` declarations
|
|
- Recognizes `LogosMap` and `LogosList` return types (nlohmann::json aliases) and sets `MethodDecl.jsonReturn = true`
|
|
- Recognizes `std::optional<T>` → `?T` (see Optionality). Anything it does *not* recognize still falls back to the opaque `any`, silently — that fallback is why an optional was unexpressible header-first until it was named explicitly
|
|
- Template-aware parameter splitting (handles `std::vector<std::string>` correctly)
|
|
|
|
## CLI Usage
|
|
|
|
### From C++ impl header (primary use case for universal modules)
|
|
|
|
```bash
|
|
logos-cpp-generator --from-header src/my_module_impl.h \
|
|
--backend cdylib \
|
|
--metadata metadata.json \
|
|
--output-dir ./generated_code
|
|
```
|
|
|
|
Generates the module-impl C ABI exports. Qt-plugin packaging is a separate step
|
|
(`logos-qt-host-generator --backend cdylib`).
|
|
|
|
### From LIDL file — cdylib glue
|
|
|
|
```bash
|
|
logos-cpp-generator --lidl my_module.lidl \
|
|
--backend cdylib \
|
|
--output-dir ./generated_code
|
|
```
|
|
|
|
### From LIDL file — client stubs
|
|
|
|
```bash
|
|
logos-cpp-generator --lidl my_module.lidl \
|
|
--output-dir ./generated_code \
|
|
--module-only
|
|
```
|
|
|
|
### Plugin-introspection and umbrella modes
|
|
|
|
```bash
|
|
logos-cpp-generator /path/to/plugin.so --output-dir ./generated
|
|
logos-cpp-generator --metadata metadata.json --umbrella --output-dir ./generated
|
|
```
|
|
|
|
Only the first line is legacy: it is the QPluginLoader path in
|
|
`plugin_introspect.cpp`. The umbrella is not — `LogosModuleContext::modules()`
|
|
returns `LogosModules&`, so every `interface: "universal"` module that calls a
|
|
declared dependency goes through it, and `LogosModule.cmake` runs it for every
|
|
module build. `--general-only` is an exact alias for `--umbrella` (it is what
|
|
`LogosModule.cmake`, `buildPlugin.nix` and `buildHeaders.nix` pass today), and
|
|
both route to the one implementation in `main.cpp`.
|
|
|
|
### Consumer wrapper from the module's contract
|
|
|
|
The `--events-from <path>` flag points the `<plugin>.dylib` plugin-introspection codegen at the LIDL sidecar shipped alongside the dep's pre-built headers. The flag keeps its historical name, but the file it names is the module's whole **contract**, and everything the wrapper is generated from comes out of it: the typed methods, the typed `on<EventName>(callback)` accessors, and the record structs. Callback and signature types match `--api-style`.
|
|
|
|
```bash
|
|
logos-cpp-generator /path/to/plugin.dylib \
|
|
--module-only --api-style lp \
|
|
--events-from /path/to/dep/share/logos/my_module.lidl \
|
|
--output-dir ./generated
|
|
```
|
|
|
|
**Contract-first, exactly like the Qt surface.** A module that ships a contract is described by it; only a module that ships none (a handcrafted Qt plugin) is described by its compiled plugin's `QMetaObject`. Both paths end in the same `makeHeader` / `makeSource`, and with a contract this path emits the same wrapper as `--general-only --dep <name>=<name>.lidl` — the path `buildHeaders.nix` already takes under cross-compilation.
|
|
|
|
The methods used to come from the plugin's published `getMethods()`, and that was a defect rather than a simplification. `generator_lib` is keyed on flat type NAMES with a QVariant fallback (`mapParamType` / `mapReturnType`), so a module whose metadata is spelled in a vocabulary this emitter does not recognise silently produced a wrapper of `QVariant` / `LogosMap` with no diagnostic anywhere. It was measured: when the cdylib backend began publishing the LIDL contract vocabulary (`tstr`, `[uint]`, `? tstr`) instead of Qt type names, every `interface: "universal"` module's lp wrapper collapsed to `LogosMap`. Teaching the reader a second vocabulary is not a fix — `int` is a 32-bit Qt int in one table and a 64-bit LIDL integer in the other, so a merged table mistypes every integer and the reader cannot tell from the string which one it is holding.
|
|
|
|
Two consequences worth knowing:
|
|
|
|
- **A named-but-missing sidecar is refused** (exit 2), as is an unreadable or malformed one (exit 4). Falling back to introspection would emit a wrapper that compiles and is wrong in a way nothing downstream can see.
|
|
- **A LIDL-spelled listing with no contract is refused** (exit 7). Only a hand-run invocation can reach that combination — `buildHeaders.nix` always passes the flag when the sidecar exists — and it is the shape this section used to suggest. The check keys on the LIDL primitives Qt has no word for (`tstr`, `bstr`, `uint`, `float64`, `result`, `any`) plus anything starting `[`, `{` or `?`, so it cannot false-fire on a Qt name; the words the two vocabularies share (`int`, `bool`) are in the known table and never reach the fallback.
|
|
- **The plugin is still loaded**, so the dlopen check (exit 3 on an SDK/ABI skew) is unchanged, and the two method NAME sets are compared. A divergence — a stale sidecar — is reported on stderr as a `Note:`; the wrapper follows the contract. Only `isInvokable` entries are compared, because a cdylib publishes its events into the same array.
|
|
|
|
In Nix builds this is wired automatically: `buildHeaders.nix` looks for `<pluginLib>/share/logos/<name>.lidl` (which `buildPlugin.nix`'s installPhase placed there) and threads it through.
|
|
|
|
## Building
|
|
|
|
The generator is built as part of logos-cpp-sdk:
|
|
|
|
```bash
|
|
ws build logos-cpp-sdk # builds everything including the generator
|
|
```
|
|
|
|
The generator binary is available as `logos-cpp-generator` in module build environments (provided by logos-module-builder's `nativeBuildInputs`).
|
|
|
|
## Testing
|
|
|
|
The LIDL backends are tested in `tests/experimental/`, the shared `generator_lib` emitters in `tests/generator/`:
|
|
|
|
```bash
|
|
ws test logos-cpp-sdk # runs all tests including experimental
|
|
```
|
|
|
|
The frontend tests (lexer/parser/validator/serializer) moved to the **logos-lidl** repo along with the code; only the C++/Qt-specific backends are tested here:
|
|
|
|
| Test file | What it tests |
|
|
|-----------|---------------|
|
|
| `test_lidl_type_mapping.cpp` | `lidlTypeToQt`, `lidlTypeToStd`, `lidlIsStdConvertible`, `lidlToPascalCase`, optionality on both surfaces |
|
|
| `test_lidl_gen_client.cpp` | Client stub generation: sync/async methods, events, metadata JSON, edge cases, both optional spellings agreeing |
|
|
| `test_lidl_gen_cdylib.cpp` | cdylib eligibility + emission: bytes at depth, records, typed maps, optionality (key omission, arity, `?any` collapse) |
|
|
| `test_impl_header_parser.cpp` | Header parsing: type mapping, access specifiers, skipping private/protected, error cases, `std::optional<T>` |
|
|
|
|
(The lexer/parser/AST/serializer/validator round-trip + description tests live in logos-lidl's own `tests/test_lidl.cpp`.)
|
|
|
|
In `tests/generator/`, alongside the wrapper-emitter tests:
|
|
|
|
| Test file | What it tests |
|
|
|-----------|---------------|
|
|
| `test_make_umbrella.cpp` | The `LogosModules` aggregate: both dependency forms on both API styles, that every member's type is included, dropped nameless entries, empty deps |
|
|
|
|
Fixture files in `tests/experimental/fixtures/`:
|
|
- `sample_impl.h` — module with all supported type variations
|
|
- `sample_metadata.json` — metadata with dependencies
|
|
- `object_deps_metadata.json` — dependencies declared in both forms, with resolution constraints
|
|
- `complex_impl.h` — module with multiple access specifier sections
|
|
- `empty_class_impl.h` — class with no public methods
|
|
- `empty_metadata.json` — minimal metadata
|
|
- `optional_impl.h` / `optional_metadata.json` — `std::optional<T>` header-first, incl. an optional over a declared record
|
|
|
|
## Known Limitations
|
|
|
|
- The impl header parser is lightweight (regex + state machine). It does not handle:
|
|
- Multi-line method declarations
|
|
- Default parameter values
|
|
- Method definitions in the header (only declarations ending with `;`)
|
|
- Nested classes
|
|
- Template methods
|
|
- `std::function` members are silently skipped (never treated as methods)
|
|
- LIDL does not support generic/parameterized types or inheritance
|
|
- `--from-header` emits the **cdylib** backend here (the `qt` glue backend moved to logos-plugin-qt's `logos-qt-host-generator --backend cdylib`, NOT to logos-qt-generator, which refuses both `qt` and `cdylib`); the **Rust** backend lives in logos-rust-sdk's `lidl-gen`, generating over logos-lidl's C ABI
|
|
- Client stub generation (`lidlMakeHeader`/`lidlMakeSource`) is only available from LIDL files, not from `--from-header`
|
|
- **Optionality is still untyped in *positional* slots on the legacy consumer path.** The
|
|
consumer wrappers real modules get come from `main.cpp` →
|
|
`generateInterfaceWrappers` → `lidl_to_json` → `generator_lib`, and that JSON boundary
|
|
carries a single Qt **type-name string** per slot. Record *fields* now carry an
|
|
`optional` flag alongside the value type, so both LIDL spellings emit identical, typed
|
|
code (`QVariant` on Qt, `std::optional<T>` on Lp). A **method parameter, a return type
|
|
and an event parameter still do not**: they arrive as `QVariant` (Qt) / `LogosMap` (Lp)
|
|
— the right *shape* (an invalid QVariant / a JSON `null` is the empty inhabitant) with
|
|
no *type*, so a caller gets no compile-time check and cannot tell `?tstr` from `?uint`
|
|
or recover a `?Record`'s struct. There is no spelling divergence there — a positional
|
|
slot has no name to hang a flag on, so it only ever had the type-kind form — and closing
|
|
it changes generated method **signatures**, i.e. a source break for every existing call
|
|
site. Until then the generator prints a `Note:` naming every still-flattened slot, so an
|
|
affected build is never silent. `OptionalSpellings.PositionalSlotsAreStillFlattened`
|
|
pins the current behaviour so closing the gap is a deliberate change.
|
|
- **Nesting, map key types and descriptions still do not cross that boundary either** —
|
|
the flag added for optionality is per-field, not a general widening.
|
|
- `lidlRecordCollidesWithBytesTag` reads *through* an optional (via `fieldValueType`), so a
|
|
single-`_bytes`-field record is refused under both spellings. It used to read `f.type`,
|
|
which refused `? _bytes: tstr` and let `_bytes: ?tstr` through — the same declaration,
|
|
two answers. `?bstr` is unaffected either way: the tag lives in the value, not the slot.
|
|
- **A provider REJECTION reaches `…Async`'s callback only as a log line** (but
|
|
`…AsyncResult`'s callback gets it properly). A provider that refuses a call answers the
|
|
canonical `{"code":…, "message":…, "origin":…}` object as its RESULT, not
|
|
as a transport error, and the Qt return table would convert it like any other value —
|
|
erasing it (`_result.toList()` on that map is `[]`). The Qt consumer emitter therefore
|
|
detects it (`logosDispatchRejection`, emitted once per wrapper) and folds it into the
|
|
error channel of every surface that HAS one:
|
|
- **sync** — the `logos::CallError*` out-parameter, so `mod.echoUintList(v, &err)` can
|
|
tell a rejection from an empty return;
|
|
- **`…AsyncResult`** — `logos::AsyncResult<T>::error`, so `r.ok()` is false and
|
|
`r.error.code` carries the provider's code exactly as on the sync path.
|
|
|
|
`code` is matched against a **closed set** — `kRejectionCodes` in `generator_lib.cpp`,
|
|
the single source of truth both emitters build their condition from:
|
|
`dispatch_failed` (the provider refused the argument VALUES), `invalid_args` (wrong
|
|
argument COUNT) and `unknown_method`. It was the single literal `dispatch_failed` until
|
|
the arity code was found to be live and undetected — `experimental/lidl_gen_cdylib.cpp`
|
|
and logos-rust-sdk's `args::invalid_args` have both emitted `invalid_args` all along,
|
|
so a missing argument reached a typed consumer as a *successful* call returning a
|
|
three-key map. `unknown_method` is in the set before any provider emits it: widening a
|
|
detector is backwards-compatible on its own, whereas a new provider code shipped against
|
|
narrow detectors would arrive as data. The set stays CLOSED — a method may legitimately
|
|
return a three-string map, so matching the shape alone would let user data impersonate a
|
|
refusal.
|
|
|
|
The historical **`…Async`** overload is the one exception: its callback is
|
|
`std::function<void(T)>`, and adding an error parameter would change a generated public
|
|
surface (which logos-qt-sdk's `qt-generator --backend consumer` veneer mirrors 1:1). It
|
|
is left untouched, so there an async rejection is reported with `qWarning` and the
|
|
callback still receives the default-converted value. `…AsyncResult` exists precisely
|
|
because giving async an error channel was an API addition rather than a code-generation
|
|
fix — a caller that needs to SEE the rejection uses it.
|
|
|
|
The **Qt-free (`lp`) emitter** folds the same rejection through a `nlohmann::json` twin
|
|
of the detector (`logosDispatchRejectionJson`, under its own guard macro so both can
|
|
share a translation unit), into the same two surfaces: the sync `logos::CallError*`
|
|
out-parameter and `…AsyncResult`. Two differences from the Qt twin, both deliberate:
|
|
its sync path has no `qWarning` fallback for a caller that passed no `err` (a Qt-free
|
|
wrapper pulling in `<iostream>` to say so would cost every generated TU for a
|
|
diagnostic nobody reads), and lp `…Async` is left alone for the same reason the Qt one
|
|
is — its callback takes the value alone.
|