From 3d322bd3159075443f4976ebb35040a70f7ff3df Mon Sep 17 00:00:00 2001 From: Dario Lipicar Date: Tue, 28 Jul 2026 17:24:49 -0300 Subject: [PATCH] fix(codegen): LIDL int/uint are 64-bit in the Qt spelling too (#113) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * fix(codegen): LIDL int/uint are 64-bit in the Qt spelling too lidlTypeToQt mapped BOTH `int` and `uint` to plain `int`. Everywhere else in the stack a LIDL int/uint is 64-bit — int64_t/uint64_t in C++ impls, i64/u64 in the Rust SDK — so the Qt spelling broke the one-type-per-LIDL-type rule and lost data: a Qt consumer reading a `uint` return got a SIGNED 32-bit value, so anything above 2^31 came back wrong and anything above 2^63 was never expressible. int -> qlonglong, uint -> qulonglong, and returnConversion() gains the matching accessors (toLongLong / toULongLong instead of toInt). qlonglong/qulonglong rather than qint64/quint64 so the generated introspection JSON uses the same names Qt's own metaobject normalisation produces — otherwise a cdylib module's generated `signature` and a legacy module's QMetaObject-derived one would disagree for the same LIDL type. Nothing looks these strings up: the only QMetaType::fromName call in the stack is for "LogosResult". This changes two generated surfaces: the Qt consumer wrapper signatures and the introspection JSON. Passing an int argument still converts implicitly, so callers keep compiling; code that assigns a wrapper's return into an `int` narrows and may warn, which is the bug being surfaced rather than a regression. Tests: 168/168, with the type-mapping and client-emitter expectations updated to the 64-bit spelling. Co-Authored-By: Claude Opus 5 * feat(records): typed C++ structs for Qt consumers Completes the Qt half of the type mapping this branch started. A `type Foo { … }` in a contract now generates a real struct in the client header, so a consumer writes `Status s = client.makeStatus();` instead of digging fields out of a QVariantMap. One LIDL type, one type per language. lidlTypeToQt - Named -> the record's struct (was QVariant) - [Record] -> QList, {tstr: Record} -> QMap. QVariantList CANNOT hold a record without Q_DECLARE_METATYPE, and a typed list is the point. client emitter - struct + inline ToVariant/FromVariant per record, emitted before the class; conversions come after all structs so records may reference each other. Recursive, so a field may itself be [Status] or {tstr: bstr}. - records pass by const&, decode on return, and convert at the call site (sync and async) bstr fields are QByteArray on purpose: logos-protocol's QVariant<->JSON conversion already materialises the canonical {"_bytes": base64url} form as a QByteArray and back, so the record conversions stay pure field mapping and binary survives at any depth with no record-specific bytes handling. Verified by COMPILING and RUNNING the generated code, not just asserting on text — the string tests would not have caught either bug this found: [Record] first mapped to QVariantList (appending a Status to it does not compile) and the decode lambdas shadowed their accumulator. Extracted the emitted record block for a contract with a nested record and a bytes field, compiled it against Qt6Core, and round-tripped Batch -> QVariant -> Batch asserting items[0].port, the QByteArray blob and the label all survive. Exit 0. The LidlTypeToQt.NamedType expectation flips from "QVariant" to the struct name, which is the behaviour change. Tests: 169/169. Co-Authored-By: Claude Opus 5 * fix(codegen): teach the lp/std consumer wrappers the 64-bit spellings Caught while checking whether the SDKs are ready for the `any` migration, and it is a regression THIS BRANCH would otherwise have shipped. legacy/generator_lib.cpp generates the lp/std consumer wrappers a universal module uses to call its dependencies. It matches type names against an allow-list and falls back to QVariant for anything else: static const QSet known = { "void","bool","int","double","float","QString", … }; if (known.contains(base)) return base; return QString("QVariant"); Once lidlTypeToQt reports `qlonglong`/`qulonglong`, every LIDL int/uint method misses that list — so a typed `int` parameter would have silently become an opaque QVariant in those wrappers. Worse than the truncation this branch set out to fix, and invisible until someone read the generated header. Adds the two spellings to both allow-lists, plus the conversions they imply: QVariant->Qt (toLongLong / toULongLong), the std spellings (int64_t / uint64_t), the QVariant->std return path, the Qt-style return, and the default-value case. The existing `int` entries stay for legacy Qt plugins, whose QMetaObject still reports `int` for a 32-bit parameter. Tests: 171/171, with the allow-list pinned in both mapping test files — including that an unknown spelling still falls back to QVariant, so the fallback itself is not what regressed. Co-Authored-By: Claude Opus 5 * feat(records): typed structs for C++ dependency wrappers Closes the second record gap. The wrapper every C++ module actually gets for its dependencies comes from the LEGACY generator (`--dep =`), not the client-stub backend the previous commit taught about records — and there a contract's `type Status { ... }` reached the consumer as an untyped bag: QVariant on the Qt surface, LogosMap on the std/lp one. Worse than inconvenient for a `bstr` field: the caller received the canonical `{"_bytes": "..."}` envelope and had to know to unwrap it, while Rust and the stub backend handed back real bytes. Now, on all three api styles: struct Status { uint64_t port{}; std::vector blob{}; }; Status getStatus(logos::CallError* err = nullptr); std::string describeStatus(const Status& s, ...); std::vector listStatuses(...); The struct is NESTED in the wrapper class (`InfoModule::Status`) because a module consuming two deps that each declare a `Status` includes both wrappers into one translation unit. Conversions are file-local statics in the generated .cpp, so a Qt-free module's own TUs still never see QVariant or nlohmann. Records reach parameters, returns, event callbacks, `[Record]` and `{tstr: Record}` — at any depth, with bytes tagged throughout. Same commit, the legacy path's half of the 64-bit fix: `lidlTypeExprToQtTypeName` mapped BOTH int and uint to `int` ("wire-as-int for now"), so a `uint` method on a dep reached a Qt consumer as a signed 32-bit value and a std/lp consumer as a signed int64_t. Now qlonglong/qulonglong, matching the spelling the other half of this PR gave the stub backend. `lpFromJsonExpr` grew the uint64_t branch it needed — without it a mistyped payload THREW out of nlohmann's implicit conversion instead of defaulting like every other scalar. Verified by generating a contract with a record, a record-of-records, a `uint` above 2^32 and a high-byte `bstr`, then COMPILING the output for qt/std/lp and round-tripping the emitted conversions: - `{"_bytes":"gAH_"}` at every depth, decoding back to the same bytes - 4294967296 intact through both directions - garbage/missing fields default rather than throw That compile is what caught the one real bug here: the container decode lambdas declared `__m`/`__j`, shadowing the record decoder's own locals, so a map-of-records field read from its own uninitialized local — it compiled with nothing but a -Wuninitialized warning. Locals are `__acc`/`__src` now, pinned by a test. Also: 8 generator tests (one asserting an empty record set leaves every byte of the output as it was), 179 total green; logos-test-modules builds and tests green against this generator. Co-Authored-By: Claude Opus 5 * fix(records): async record returns decoded a default-constructed struct Two correctness holes in the record work, both silent, found while scoping the cdylib provider side. 1. The async consumer overload emitted `qvariant_cast(v)` while the sync one emitted `StatusFromVariant(_result)`. The wire delivers a QVariantMap and no Q_DECLARE_METATYPE is emitted for the struct, so the cast does not fail — it returns a DEFAULT-CONSTRUCTED Status and the caller sees empty fields with no diagnostic. The sync path being correct is what makes it bad: the same call is right or wrong depending only on which overload the caller reached for. Async now decodes field by field through the same conversion. (The legacy dependency-wrapper generator already did this correctly — this was the experimental client-stub backend only.) 2. A record whose ONLY field is a tstr named `_bytes` is wire-identical to a canonical tagged byte string: `isTaggedBytes()` is checked before `is_object()` in both logos_codec.h and logos_json_convert.cpp, so such a record decodes as bytes and the struct silently disappears. The ambiguity is inherent to the tagged form — the codec's own comment says not to name a map key `_bytes` — but the generator can refuse to emit the one shape guaranteed to misdecode instead of leaving it to be found at runtime. Both front doors (the .lidl client-stub path and the --dep/--interface path) now reject it with a message naming the type and the fix. A second field disambiguates it (isTaggedBytes requires exactly one key), so that shape still generates — verified, not assumed. 181 tests, +2. Co-Authored-By: Claude Opus 5 * feat(cdylib): records, [bstr], typed maps and nested composites on the C++ provider The C++ cdylib backend could express scalars, `[scalar]`, `any` and an untyped map. Everything else it rejected BY NAME — `[bstr]`, `[[int]]`, `{tstr: int}`, and records, which the impl-header parser could not even declare because it skipped `struct` outright. That is why the ext contract had to be Rust-only. Four changes, in the order they matter: typeSupported() recurses instead of whitelisting element names, which admits [bstr], [[int]], [Record] and [{tstr: T}] in one rule; a declared record is admitted; a map now REQUIRES a tstr key (it used to `return true` for any map and then silently flatten {int: tstr} to an untyped LogosMap, losing the key type). lidlTypeToStdCdylib() became total. Its `lidlTypeToStd` fallback answers QVariantList / QVariantMap — Qt names in a Qt-FREE translation unit — and only failed to appear because the gate rejected everything that reached it. Widening the gate made that fallback a live leak, so composites now recurse and never reach it. _types.h new: the generated codec, recursive, with a FULL specialization for std::vector that wins over the generic vector rule — which is what keeps a bstr tagged at any depth instead of becoming a plain array of numbers. One Codec specialization per declared record, field by field, with the field path in the error. impl_header_parser learned `struct` (two passes, because a record field may name another record and the type mapper only answers Named() for an already-registered name — one pass silently typed `Blob inner;` as `any`), std::map, and recursion into vector elements so std::vector is [Blob] rather than falling through to `any`. Records are only names the contract DECLARES: `void` is not a LIDL builtin, so `-> void` arrives as Named("void"), and treating every Named as a record is the exact trap that made the Rust generator emit `-> Void`. Two things the interface JSON got wrong, both found by running it: - it spelled a record `Blob` and a `[Record]` `QList`. Those are the CONSUMER's names, correct in a generated wrapper where the struct exists — but this JSON is the module's getMethods(), read by the host to marshal a QVariant, and there is no metatype called `Blob`. The host SIGSEGV'd on the first call to any record method. A record IS a variant map at that boundary; lidlTypeToQtWire() says so. - the types header emitted the structs. Header-first, the author owns them and the contract was derived from those very declarations, so it was a redefinition. It emits forward declarations and the codec. Also: `jsonReturn` is set by the front end for any map return, which no longer implies the C++ type IS nlohmann::json now that a typed map is std::map — checking the flag before the spelling emitted `result.dump()` on a std::map. The spelling decides. Scalars keep their nlohmann accessor verbatim rather than routing through the codec: `.get()` TRUNCATES a float instead of throwing, and the conformance matrix pins that leniency (hostile/int/fractional expects 3 from 3.7). Changing it would silently move behaviour something depends on. The pinned-rejection test for [bstr] is INVERTED rather than deleted — the cell it pinned still matters, only its answer changed — plus new tests for the non-tstr map key rejection and for declared-vs-undeclared records. 183 tests. Every existing module still builds; test_fullapi_cpp is unchanged. Co-Authored-By: Claude Opus 5 * fix(records): only structs the API mentions become contract types Teaching the impl-header parser to read `struct` (previous commit) published EVERY struct in a header as a contract `type`. Two production modules already carry private helpers: openmetrics-module struct ModuleSource (namespace scope, internal) logos-package-manager struct PendingAction (PRIVATE, inside the class) Both were being published — a module's interface changing as a side effect of an internal refactor, which is not something deriving a contract from a header may do. PendingAction was published WRONG as well: its fields carry trailing `// comments`, the field regex requires a line ending in ';', and the unmatched fields were silently dropped. A record with a partial field list is worse than no record, because it looks like a contract. A struct now earns its place by appearing in a method or event signature — transitively, since a published record's own fields may name others. Verified on the real headers: package-manager and openmetrics publish zero types again, while the ext provider keeps both Blob and Wrapper (Wrapper is reachable only through Blob's use in a signature). Trailing comments are stripped before the field match, so no field is dropped. Two tests over a fixture carrying both an internal namespace-scope struct and a private in-class one; 185 tests. test-modules and openmetrics both rebuild. Co-Authored-By: Claude Opus 5 * docs(doctests): the generator round-trip now shows 64-bit ints and typed records The two failing assertions in cpp-sdk-generator-roundtrip were documentation asserting the OLD behaviour, and both changes are the point of this branch: int record(int id, …) -> qlonglong record(qulonglong id, …) QVariant translate(QVariant p) -> Point translate(const Point& p, …) `record`'s `id` is a `uint64_t` in the impl header, so the old signature was handing a caller a SIGNED 32-bit value for a `uint` — the doc showed the bug. The surrounding prose was wrong too, not just the expectations, so both blocks are rewritten rather than patched: * Flow 3 now states the mapping as int->qlonglong / uint->qulonglong and says why (LIDL int/uint are int64_t/uint64_t in every other binding), pointing at `record` as the worked example. * The composite section claimed "records and optionals surface as QVariant". Records now generate a struct, `[Point]` a QList and `{tstr: Point}` a QMap; maps of `any`, optionals and bare `any` still cross untyped and stay QVariant/QVariantMap — a record has a declared shape, those do not. The new text draws that line explicitly. Expectations added for `struct Point` and `Point bounds(const QList&…)` so the record path is pinned in the doc, not just described. Verified the way CI runs it — `--release-for logos-cpp-sdk=feat/qt-64bit-numerics`, which is what makes `{release}` resolve to this branch instead of master: 10 passed, 0 failed. (A plain local run builds master and is not representative — that is why it still showed the old signatures.) outputs/ regenerated; the diff also picks up unrelated pre-existing drift where the committed Markdown had fallen behind the spec. Co-Authored-By: Claude Opus 5 --------- Co-authored-by: Claude Opus 5 --- .../experimental/impl_header_parser.cpp | 150 ++++++ cpp-generator/experimental/lidl_compat.h | 33 ++ .../experimental/lidl_emit_common.cpp | 23 +- .../experimental/lidl_gen_cdylib.cpp | 436 ++++++++++++----- cpp-generator/experimental/lidl_gen_cdylib.h | 5 + .../experimental/lidl_gen_client.cpp | 176 ++++++- cpp-generator/legacy/generator_lib.cpp | 452 +++++++++++++++--- cpp-generator/legacy/generator_lib.h | 18 +- cpp-generator/legacy/main.cpp | 74 ++- cpp-generator/main.cpp | 5 + .../cpp-sdk-generator-roundtrip.test.yaml | 47 +- .../outputs/cpp-sdk-generator-roundtrip.md | 56 ++- tests/experimental/fixtures/records_impl.h | 43 ++ .../fixtures/records_metadata.json | 10 + .../experimental/test_impl_header_parser.cpp | 59 +++ tests/experimental/test_lidl_gen_cdylib.cpp | 168 +++++-- tests/experimental/test_lidl_gen_client.cpp | 121 ++++- tests/experimental/test_lidl_type_mapping.cpp | 14 +- tests/generator/CMakeLists.txt | 1 + tests/generator/test_map_param_type.cpp | 13 + tests/generator/test_map_return_type.cpp | 13 + tests/generator/test_records.cpp | 216 +++++++++ 22 files changed, 1851 insertions(+), 282 deletions(-) create mode 100644 tests/experimental/fixtures/records_impl.h create mode 100644 tests/experimental/fixtures/records_metadata.json create mode 100644 tests/generator/test_records.cpp diff --git a/cpp-generator/experimental/impl_header_parser.cpp b/cpp-generator/experimental/impl_header_parser.cpp index 9f961e9..d129016 100644 --- a/cpp-generator/experimental/impl_header_parser.cpp +++ b/cpp-generator/experimental/impl_header_parser.cpp @@ -6,6 +6,10 @@ #include #include #include +#include + +#include +#include #include // --------------------------------------------------------------------------- @@ -34,6 +38,12 @@ static QString stripDeclarationSpecifiers(QString string) // C++ type string → LIDL TypeExpr // --------------------------------------------------------------------------- +// The records the header declares, discovered by scanForRecords() before any +// method is parsed. A bare `Blob` in a signature is only a record if the header +// actually declared `struct Blob { ... };` — otherwise it stays the opaque +// `any` it always was. +static QSet g_recordNames; + static TypeExpr cppTypeToLidl(const QString& raw) { // Normalize: strip const, &, leading/trailing whitespace @@ -91,6 +101,12 @@ static TypeExpr cppTypeToLidl(const QString& raw) TypeExpr elem = { TypeExpr::Primitive, "bool", {} }; return { TypeExpr::Array, "", { elem } }; } + // Anything else: recurse. That is what makes `std::vector` a + // [Blob] and `std::vector>` a + // [{tstr: int}]. Without it the element list above was exhaustive and + // every other vector fell all the way through to the opaque `any`, + // which then encoded a record as a LogosMap. + return { TypeExpr::Array, "", { cppTypeToLidl(inner) } }; } // Qt collection types — pass through directly (non-std-convertible) @@ -113,10 +129,130 @@ static TypeExpr cppTypeToLidl(const QString& raw) if (t == "StdLogosResult") return { TypeExpr::Primitive, "result", {} }; + // std::map -> {tstr: T}. Absent before, so a typed map was + // unspellable header-first and fell through to `any`. + static QRegularExpression mapRe("^std::map\\s*<\\s*std::string\\s*,\\s*(.+)\\s*>$"); + QRegularExpressionMatch mm = mapRe.match(t); + if (mm.hasMatch()) { + TypeExpr val = cppTypeToLidl(mm.captured(1).trimmed()); + return { TypeExpr::Map, "", { {TypeExpr::Primitive, "tstr", {}}, val } }; + } + + // A record the header declared. Checked LAST so it can never shadow a + // builtin spelling, and gated on the declared set so an unknown type keeps + // the historical `any` fallback rather than naming a struct nobody emits. + if (g_recordNames.contains(t)) + return { TypeExpr::Named, t.toStdString(), {} }; + // Fallback: treat as opaque return { TypeExpr::Primitive, "any", {} }; } +// Find `struct Name { Type field; ... };` blocks and turn them into `type` +// declarations. +// +// The parser used to SKIP any line starting with `struct`, which meant a record +// could not be declared header-first at all — the only way to get one was a +// hand-written .lidl. Worse, a method mentioning the struct still parsed: its +// type fell through to the opaque `any`, so the contract silently disagreed +// with the header. +static std::vector scanForRecords(const QStringList& lines) +{ + static QRegularExpression openRe("^struct\\s+(\\w+)\\s*\\{\\s*$"); + static QRegularExpression fieldRe("^([\\w:<>,\\s\\*]+?)\\s+(\\w+)\\s*(=[^;]*)?;$"); + + // TWO passes. A record field may name another record (`Blob inner;` inside + // Wrapper), and cppTypeToLidl only answers Named() for a name already in + // g_recordNames — so every struct name has to be registered before any + // field is typed. One pass silently typed such a field as `any`, and the + // generated codec then tried to encode a Blob as a LogosMap. + for (int i = 0; i < lines.size(); ++i) { + QRegularExpressionMatch om = openRe.match(lines.at(i).trimmed()); + if (om.hasMatch()) + g_recordNames.insert(om.captured(1)); + } + + std::vector out; + for (int i = 0; i < lines.size(); ++i) { + const QString line = lines.at(i).trimmed(); + QRegularExpressionMatch om = openRe.match(line); + if (!om.hasMatch()) continue; + + TypeDecl td; + td.name = om.captured(1).toStdString(); + for (int j = i + 1; j < lines.size(); ++j) { + const QString body = lines.at(j).trimmed(); + if (body.startsWith("};")) break; + if (body.isEmpty() || body.startsWith("//")) continue; + // Strip a trailing line comment before matching: a field written + // `std::string name; // what it is` does not end in ';' and was + // silently DROPPED, publishing a record with a partial field list — + // the worst kind of wrong, because it looks like a contract. + QString field = body; + const int comment = field.indexOf("//"); + if (comment >= 0) field = field.left(comment).trimmed(); + if (field.isEmpty()) continue; + QRegularExpressionMatch fm = fieldRe.match(field); + if (!fm.hasMatch()) continue; + FieldDecl fd; + fd.name = fm.captured(2).toStdString(); + fd.type = cppTypeToLidl(fm.captured(1).trimmed()); + td.fields.push_back(fd); + } + if (!td.fields.empty()) out.push_back(td); + } + return out; +} + +// Keep only the records the module's API actually mentions. +// +// An impl header routinely declares helper structs that are none of a +// consumer's business — `struct PendingAction` inside the class, a +// `struct ModuleSource` next to it. Publishing every struct as a contract +// `type` changes the module's PUBLISHED interface as a side effect of an +// internal refactor, which is not something deriving a contract from a header +// is allowed to do. A struct earns its place in the contract by appearing in a +// method or event signature — transitively, since a published record's own +// fields may name others. +static void keepOnlyReferencedRecords(ModuleDecl& module) +{ + auto mention = [](const TypeExpr& te, std::set& out) { + std::function walk = [&](const TypeExpr& t) { + if (t.kind == TypeExpr::Named) out.insert(t.name); + for (const TypeExpr& e : t.elements) walk(e); + }; + walk(te); + }; + + std::set referenced; + for (const MethodDecl& md : module.methods) { + mention(md.returnType, referenced); + for (const ParamDecl& pd : md.params) mention(pd.type, referenced); + } + for (const EventDecl& ed : module.events) + for (const ParamDecl& pd : ed.params) mention(pd.type, referenced); + + // Transitive closure: a referenced record's fields may name more records. + bool grew = true; + while (grew) { + grew = false; + for (const TypeDecl& td : module.types) { + if (!referenced.count(td.name)) continue; + for (const FieldDecl& fd : td.fields) { + std::set here; + mention(fd.type, here); + for (const std::string& n : here) + if (referenced.insert(n).second) grew = true; + } + } + } + + std::vector kept; + for (const TypeDecl& td : module.types) + if (referenced.count(td.name)) kept.push_back(td); + module.types = std::move(kept); +} + // --------------------------------------------------------------------------- // Parse a single method declaration line // --------------------------------------------------------------------------- @@ -289,6 +425,8 @@ ImplParseResult parseImplHeader(const QString& headerPath, QString source = QString::fromUtf8(hf.readAll()); hf.close(); + // Records first: cppTypeToLidl consults the declared set, so the structs + // have to be known before a single signature is looked at. // Split into physical lines, then merge any whose parentheses are still // open into one logical line. The scanner below is line-based — it only // accepts a method when a single trimmed line ends in ';' and @@ -340,6 +478,13 @@ ImplParseResult parseImplHeader(const QString& headerPath, lines.append(acc); } + // Records, before any signature is examined: cppTypeToLidl() consults the + // declared set, so a `Blob` parameter only becomes Named("Blob") once the + // struct has been seen. Reset per parse — the set is file-static and a + // single process generates for more than one module. + g_recordNames.clear(); + result.module.types = scanForRecords(lines); + // State machine: find "class ", then collect declarations. // `InLogosEvents` is entered by the literal `logos_events:` token // (mirrors Qt's `signals:`) — methods declared there are parsed as @@ -551,6 +696,11 @@ ImplParseResult parseImplHeader(const QString& headerPath, } done: + // Now that every signature is known, drop the structs the API never + // mentions — a header's internal helpers must not become published + // contract types. + keepOnlyReferencedRecords(result.module); + if (result.module.methods.empty()) { err << "Warning: no public methods found in class " << className << " in " << headerPath << "\n"; diff --git a/cpp-generator/experimental/lidl_compat.h b/cpp-generator/experimental/lidl_compat.h index c556c9a..b91c523 100644 --- a/cpp-generator/experimental/lidl_compat.h +++ b/cpp-generator/experimental/lidl_compat.h @@ -54,4 +54,37 @@ inline lidl::ValidationResult lidlValidate(const ModuleDecl& module) return lidl::validate(module); } +// A record whose ONLY field is a `tstr` named `_bytes` is indistinguishable on +// the wire from a canonical tagged byte string: `isTaggedBytes()` is checked +// BEFORE `is_object()` in both logos_codec.h and logos_json_convert.cpp, so +// such a record silently decodes as a byte string and the struct is gone. The +// ambiguity is inherent to the tagged form — the codec's own comment says not +// to name a map key `_bytes` — but a generator can at least refuse to emit the +// one shape that is guaranteed to misdecode, instead of leaving it to be +// discovered at runtime. +inline bool lidlRecordCollidesWithBytesTag(const TypeDecl& t) +{ + return t.fields.size() == 1 + && t.fields[0].name == "_bytes" + && t.fields[0].type.kind == TypeExpr::Primitive + && t.fields[0].type.name == "tstr"; +} + +// Returns false and fills `error` when any declared record cannot round-trip. +inline bool lidlCheckRecords(const ModuleDecl& m, QString* error) +{ + for (const TypeDecl& t : m.types) { + if (lidlRecordCollidesWithBytesTag(t)) { + if (error) + *error = QString("type '%1': a record whose only field is a tstr named " + "'_bytes' is wire-identical to a tagged byte string and " + "would decode as bytes, not as the record. Rename the " + "field or give the record another field.") + .arg(qs(t.name)); + return false; + } + } + return true; +} + #endif // LIDL_COMPAT_H diff --git a/cpp-generator/experimental/lidl_emit_common.cpp b/cpp-generator/experimental/lidl_emit_common.cpp index 83af2a1..77aa918 100644 --- a/cpp-generator/experimental/lidl_emit_common.cpp +++ b/cpp-generator/experimental/lidl_emit_common.cpp @@ -20,26 +20,41 @@ QString lidlTypeToQt(const TypeExpr& te) if (te.name == "void") return "void"; if (te.name == "tstr") return "QString"; if (te.name == "bstr") return "QByteArray"; - if (te.name == "int") return "int"; - if (te.name == "uint") return "int"; + // 64-bit, and unsigned stays unsigned. LIDL int/uint are int64_t/uint64_t + // everywhere else (C++ impls, Rust's i64/u64), so spelling them `int` + // here broke the 1-1 mapping and truncated: a Qt consumer reading a + // `uint` return got a SIGNED 32-bit value. qlonglong/qulonglong rather + // than qint64/quint64 so the generated introspection matches the names + // Qt's own metaobject normalisation produces. + if (te.name == "int") return "qlonglong"; + if (te.name == "uint") return "qulonglong"; if (te.name == "float64") return "double"; if (te.name == "bool") return "bool"; if (te.name == "result") return "LogosResult"; if (te.name == "any") return "QVariant"; return "QVariant"; + case TypeExpr::Named: + // A record declared by the contract: its generated struct. One LIDL + // type, one type per language — a record is not a QVariant blob. + return QString::fromStdString(te.name); case TypeExpr::Array: if (te.elements.size() == 1 && te.elements[0].kind == TypeExpr::Primitive && te.elements[0].name == "tstr") { return "QStringList"; } + // A list of records is a typed list: QVariantList could not hold a + // record without Q_DECLARE_METATYPE, and the point of a record is that + // the consumer gets the struct. + if (te.elements.size() == 1 && te.elements[0].kind == TypeExpr::Named) + return "QList<" + QString::fromStdString(te.elements[0].name) + ">"; return "QVariantList"; case TypeExpr::Map: + if (te.elements.size() == 2 && te.elements[1].kind == TypeExpr::Named) + return "QMap"; return "QVariantMap"; case TypeExpr::Optional: return "QVariant"; - case TypeExpr::Named: - return "QVariant"; } return "QVariant"; } diff --git a/cpp-generator/experimental/lidl_gen_cdylib.cpp b/cpp-generator/experimental/lidl_gen_cdylib.cpp index fe85431..62c3d7a 100644 --- a/cpp-generator/experimental/lidl_gen_cdylib.cpp +++ b/cpp-generator/experimental/lidl_gen_cdylib.cpp @@ -3,6 +3,9 @@ #include +#include +#include + QString lidlToPascalCase(const QString& name); QString lidlTypeToQt(const TypeExpr& te); bool lidlIsStdConvertible(const TypeExpr& te); @@ -12,7 +15,23 @@ namespace { // The cdylib-supported subset: std-convertible LIDL types only — the same // Qt-free set the std apiStyle handled, so any universal module that built // under std also builds as a header-first cdylib. -bool typeSupported(const TypeExpr& te, bool isReturn) +// The records a contract DECLARES. A `Named` type is a record only if it is in +// here: `void` is not a LIDL builtin, so `-> void` arrives as Named("void") and +// treating every Named as a record is how the Rust generator once emitted +// `-> Void`. Same trap, same guard. +std::set recordNames(const ModuleDecl& module) +{ + std::set out; + for (const TypeDecl& t : module.types) out.insert(t.name); + return out; +} + +bool isRecord(const TypeExpr& te, const std::set& recs) +{ + return te.kind == TypeExpr::Named && recs.count(te.name) > 0; +} + +bool typeSupported(const TypeExpr& te, bool isReturn, const std::set& recs) { if (te.kind == TypeExpr::Primitive) { if (te.name == "tstr" || te.name == "bstr" || te.name == "int" @@ -27,60 +46,44 @@ bool typeSupported(const TypeExpr& te, bool isReturn) return true; return false; } - if (te.kind == TypeExpr::Array && te.elements.size() == 1) { - const TypeExpr& e = te.elements[0]; - // `bstr` elements are the one array kind that cannot ride nlohmann's - // blanket get<>(): each element is the tagged {"_bytes": base64url} - // OBJECT, not a number array, so it needs the emitted per-element - // codec (lidlBytesListFromJson / lidlBytesListToJson) below. - return e.kind == TypeExpr::Primitive - && (e.name == "tstr" || e.name == "int" || e.name == "uint" - || e.name == "float64" || e.name == "bool" || e.name == "any" - || e.name == "bstr"); - } - // Maps ({k: v}, i.e. LogosMap) round-trip through nlohmann too. - if (te.kind == TypeExpr::Map) + // A declared record is a generated struct with a generated codec. + if (isRecord(te, recs)) return true; - return false; -} - -// `[bstr]` — an array of byte strings (std::vector>). -// Every place scalar `bstr` needs the tagged-bytes codec, this needs the list -// form of it, so the check lives in one place. -bool isBytesArray(const TypeExpr& te) -{ - return te.kind == TypeExpr::Array && te.elements.size() == 1 - && te.elements[0].kind == TypeExpr::Primitive - && te.elements[0].name == "bstr"; -} - -// True when the module mentions `[bstr]` anywhere the emitted TU has to encode -// or decode it — method params, method returns, or event payloads. Gates the -// list codec so modules that never use it don't carry an unused static function -// (the same reason hasBytesEventParam() gates the scalar encoder). -bool usesBytesArray(const ModuleDecl& module) -{ - for (const MethodDecl& md : module.methods) { - if (isBytesArray(md.returnType)) - return true; - for (const ParamDecl& pd : md.params) - if (isBytesArray(pd.type)) - return true; + // Recurse rather than whitelisting element names: that admits [bstr], + // [[int]], [Record] and [{tstr: T}] in one rule, and keeps the gate and + // the spelling function agreeing about what is expressible. + if (te.kind == TypeExpr::Array && te.elements.size() == 1) + return typeSupported(te.elements[0], false, recs); + // Only tstr keys: the generated codec spells a map as + // std::map, so a non-tstr key has no C++ spelling. This + // used to `return true` for ANY map, which admitted `{int: tstr}` and then + // silently produced a LogosMap that lost the key type. + if (te.kind == TypeExpr::Map) { + if (te.elements.size() != 2) return false; + const TypeExpr& k = te.elements[0]; + if (!(k.kind == TypeExpr::Primitive && k.name == "tstr")) return false; + return typeSupported(te.elements[1], false, recs); } - for (const EventDecl& ed : module.events) - for (const ParamDecl& pd : ed.params) - if (isBytesArray(pd.type)) - return true; return false; } // Qt-free spelling of a LIDL type (defined below). Forward-declared so the // method-param decoder can spell composite `any` containers as their nlohmann // aliases instead of Qt containers in this Qt-free TU. -QString lidlTypeToStdCdylib(const TypeExpr& te); +QString lidlTypeToStdCdylib(const TypeExpr& te, const std::set& recs); // json arg expression -> std-typed C++ expression -QString jsonArgToStd(const TypeExpr& te, const QString& expr) +// A method argument, decoded into the author's C++ type. +// +// Everything that is not a plain scalar goes through the generated codec, which +// recurses — so a bstr keeps its canonical tag at ANY depth and a record +// decodes field by field with a path in the error. The scalars keep their +// nlohmann accessor verbatim: `.get()` TRUNCATES a float rather than +// throwing, and that leniency is pinned by the conformance matrix +// (`hostile/int/fractional` expects 3 from 3.7 on this provider). Routing them +// through the codec would silently change behaviour that something depends on. +QString jsonArgToStd(const TypeExpr& te, const QString& expr, const QString& path, + const std::set& recs) { if (te.kind == TypeExpr::Primitive) { if (te.name == "tstr") return expr + ".get()"; @@ -89,26 +92,17 @@ QString jsonArgToStd(const TypeExpr& te, const QString& expr) if (te.name == "uint") return expr + ".get()"; if (te.name == "float64") return expr + ".get()"; if (te.name == "bool") return expr + ".get()"; + if (te.name == "any") return expr; } - if (te.kind == TypeExpr::Array && te.elements.size() == 1) { - // `[bstr]` cannot go through get>>(): - // its elements arrive as tagged {"_bytes": …} objects, which nlohmann - // would refuse (and a raw number-array element would silently bypass - // the base64 decode). Route it through the per-element codec instead. - if (isBytesArray(te)) - return "lidlBytesListFromJson(" + expr + ")"; - // Qt-free spelling: `[any]` must decode as LogosList, NOT QVariantList - // (lidlTypeToStd's Qt fallback), which is undeclared in this TU. Typed - // scalar arrays ([tstr]/[int]/…) are unaffected — Cdylib defers to the - // same std spelling for them. - const QString inner = lidlTypeToStdCdylib(te); - return expr + ".get<" + inner + ">()"; - } - return expr; + const QString cpp = lidlTypeToStdCdylib(te, recs); + if (cpp == "LogosMap" || cpp == "LogosList") + return expr; // untyped JSON passes through, as it always has + return "logos_gen::Codec<" + cpp + ">::from(" + expr + ", \"" + path + "\")"; } // std-typed return variable -> json expression -QString stdReturnToJson(const MethodDecl& md, const QString& var) +QString stdReturnToJson(const MethodDecl& md, const QString& var, + const std::set& recs) { const TypeExpr& te = md.returnType; if (md.resultReturn) { @@ -116,18 +110,24 @@ QString stdReturnToJson(const MethodDecl& md, const QString& var) // (same shape logos_json_convert emits for Qt LogosResult). return "lidlResultToJson(" + var + ")"; } - if (md.jsonReturn) { + // `jsonReturn` is set by the front end for any map/list return, but that no + // longer implies the C++ type IS nlohmann::json: a TYPED map now spells + // std::map. Checking the flag before the spelling emitted + // `result.dump()` on a std::map. The spelling decides. + const QString cppRet = lidlTypeToStdCdylib(te, recs); + if (md.jsonReturn && (cppRet == "LogosMap" || cppRet == "LogosList")) { return var; // LogosMap / LogosList are nlohmann::json already } if (te.kind == TypeExpr::Primitive) { if (te.name == "bstr") return "lidlBytesToJson(" + var + ")"; + if (te.name == "any") return var; return "nlohmann::json(" + var + ")"; } - // `[bstr]`: nlohmann::json(std::vector>) would emit - // nested number arrays, which no consumer decodes as bytes. Tag each element. - if (isBytesArray(te)) - return "lidlBytesListToJson(" + var + ")"; - return "nlohmann::json(" + var + ")"; + if (cppRet == "LogosMap" || cppRet == "LogosList") + return var; + // `nlohmann::json(v)` would serialize a vector as a plain number + // array and a record not at all; the codec keeps bytes tagged at depth. + return "logos_gen::Codec<" + cppRet + ">::to(" + var + ")"; } // Qt-free spelling of a LIDL type. lidlTypeToStd() falls back to Qt containers @@ -136,16 +136,35 @@ QString stdReturnToJson(const MethodDecl& md, const QString& var) // spell those as their nlohmann aliases (LogosMap / LogosList) instead. Without // this the events sidecar emits a bare `QVariant` parameter and does not // compile. -QString lidlTypeToStdCdylib(const TypeExpr& te) +QString lidlTypeToStdCdylib(const TypeExpr& te, const std::set& recs) { if (te.kind == TypeExpr::Primitive && te.name == "any") return "LogosMap"; - if (te.kind == TypeExpr::Map) + // `{tstr: any}` and `[any]` keep their nlohmann aliases: every existing + // universal module spells them that way, and narrowing them would be a + // source break for no gain (they ARE untyped JSON). + if (te.kind == TypeExpr::Map && te.elements.size() == 2 + && te.elements[1].kind == TypeExpr::Primitive && te.elements[1].name == "any") return "LogosMap"; if (te.kind == TypeExpr::Array && te.elements.size() == 1 && te.elements[0].kind == TypeExpr::Primitive && te.elements[0].name == "any") return "LogosList"; + + // A declared record is its generated struct. + if (isRecord(te, recs)) + return qs(te.name); + // Recurse, so [bstr] is std::vector> and {tstr: Blob} + // is std::map. lidlTypeToStd() would answer QVariantList + // / QVariantMap here — a Qt name in a Qt-FREE translation unit, which only + // failed to appear because the gate used to reject these types. Widening + // the gate makes that fallback a live leak, so composites must never reach + // it. + if (te.kind == TypeExpr::Array && te.elements.size() == 1) + return "std::vector<" + lidlTypeToStdCdylib(te.elements[0], recs) + ">"; + if (te.kind == TypeExpr::Map && te.elements.size() == 2) + return "std::map"; + return lidlTypeToStd(te); } @@ -153,44 +172,181 @@ QString lidlTypeToStdCdylib(const TypeExpr& te) // reason the events sidecar needs the bytes encoder. Emitting it unconditionally // leaves an unused static function (a -Wunused-function warning) in every module // whose events carry no binary data. +// ── The generated codec ───────────────────────────────────────────────────── +// +// Emitted into the module's types header so the author's impl class and the +// generated dispatch share one definition of how a value crosses the wire. +// +// This is deliberately the same SHAPE as logos-protocol's logos_codec.h — and +// it exists as generated code only because that header cannot currently be +// included here: logos_json.h (which every universal module pulls in for +// LogosMap) and logos_codec.h both define logos::b64UrlEncode / +// b64UrlDecode / bytesToJson as inline, so including both in one translation +// unit is a redefinition error. Unify when that is resolved; the emitted +// specializations would then be the only generated part. +// +// The primary template is intentionally left UNDEFINED: an unsupported T is a +// compile error naming the type, never a silent default-constructed value. +void emitGeneratedCodec(QTextStream& s, const ModuleDecl& module, + const std::set& recs) +{ + s << "namespace logos_gen {\n\n"; + s << "// Codec::to / ::from — the one place a value's wire form is decided.\n"; + s << "// The primary template is undefined on purpose: an unsupported T is a\n"; + s << "// compile error naming the type, not a silent default.\n"; + s << "template struct Codec;\n\n"; + + s << "[[noreturn]] inline void lidlTypeError(const char* want, const std::string& path,\n"; + s << " const nlohmann::json& got)\n{\n"; + s << " throw std::runtime_error(std::string(\"expected \") + want + \" at \" + path\n"; + s << " + \", got \" + std::string(got.type_name()));\n}\n\n"; + + // Scalars. Their leniency matches what the dispatch did before the codec + // existed, so behaviour for already-working modules is unchanged. + struct Scalar { const char* cpp; const char* want; const char* check; const char* get; }; + const Scalar scalars[] = { + {"std::string", "string", "is_string()", "get()"}, + {"int64_t", "integer", "is_number()", "get()"}, + {"uint64_t", "integer", "is_number()", "get()"}, + {"double", "number", "is_number()", "get()"}, + {"bool", "boolean", "is_boolean()", "get()"}, + }; + for (const Scalar& sc : scalars) { + s << "template <> struct Codec<" << sc.cpp << "> {\n"; + s << " static nlohmann::json to(const " << sc.cpp << "& v) { return nlohmann::json(v); }\n"; + s << " static " << sc.cpp << " from(const nlohmann::json& j, const std::string& path) {\n"; + s << " if (!j." << sc.check << ") lidlTypeError(\"" << sc.want << "\", path, j);\n"; + s << " return j." << sc.get << ";\n }\n};\n\n"; + } + + // bstr. The FULL specialization wins over the generic vector rule below, + // which is what keeps bytes tagged at every depth instead of being + // serialized as a plain array of numbers. + s << "template <> struct Codec> {\n"; + s << " static nlohmann::json to(const std::vector& v) { return logos::bytesToJson(v); }\n"; + s << " static std::vector from(const nlohmann::json& j, const std::string& path) {\n"; + s << " if (j.is_object() && j.size() == 1 && j.contains(\"_bytes\")\n"; + s << " && j.at(\"_bytes\").is_string())\n"; + s << " return logos::jsonToBytes(j);\n"; + s << " lidlTypeError(\"bytes\", path, j);\n }\n};\n\n"; + + // Untyped JSON passes through unchanged — `any`, and the LogosMap/LogosList + // aliases, are all nlohmann::json. + s << "template <> struct Codec {\n"; + s << " static nlohmann::json to(const nlohmann::json& v) { return v; }\n"; + s << " static nlohmann::json from(const nlohmann::json& j, const std::string&) { return j; }\n"; + s << "};\n\n"; + + s << "template struct Codec> {\n"; + s << " static nlohmann::json to(const std::vector& v) {\n"; + s << " nlohmann::json out = nlohmann::json::array();\n"; + s << " for (const T& e : v) out.push_back(Codec::to(e));\n"; + s << " return out;\n }\n"; + s << " static std::vector from(const nlohmann::json& j, const std::string& path) {\n"; + s << " if (!j.is_array()) lidlTypeError(\"array\", path, j);\n"; + s << " std::vector out;\n out.reserve(j.size());\n"; + s << " for (size_t i = 0; i < j.size(); ++i)\n"; + s << " out.push_back(Codec::from(j.at(i), path + \"[\" + std::to_string(i) + \"]\"));\n"; + s << " return out;\n }\n};\n\n"; + + s << "template struct Codec> {\n"; + s << " static nlohmann::json to(const std::map& v) {\n"; + s << " nlohmann::json out = nlohmann::json::object();\n"; + s << " for (const auto& kv : v) out[kv.first] = Codec::to(kv.second);\n"; + s << " return out;\n }\n"; + s << " static std::map from(const nlohmann::json& j, const std::string& path) {\n"; + s << " if (!j.is_object()) lidlTypeError(\"object\", path, j);\n"; + s << " std::map out;\n"; + s << " for (auto it = j.begin(); it != j.end(); ++it)\n"; + s << " out.emplace(it.key(), Codec::from(it.value(), path + \".\" + it.key()));\n"; + s << " return out;\n }\n};\n\n"; + + // One specialization per declared record. Field order follows the contract. + for (const TypeDecl& t : module.types) { + const QString name = qs(t.name); + s << "template <> struct Codec<" << name << "> {\n"; + s << " static nlohmann::json to(const " << name << "& v) {\n"; + s << " nlohmann::json out = nlohmann::json::object();\n"; + for (const FieldDecl& f : t.fields) { + const QString ft = lidlTypeToStdCdylib(f.type, recs); + s << " out[\"" << qs(f.name) << "\"] = Codec<" << ft << ">::to(v." + << qs(f.name) << ");\n"; + } + s << " return out;\n }\n"; + s << " static " << name << " from(const nlohmann::json& j, const std::string& path) {\n"; + s << " if (!j.is_object()) lidlTypeError(\"object\", path, j);\n"; + s << " " << name << " out;\n"; + for (const FieldDecl& f : t.fields) { + const QString ft = lidlTypeToStdCdylib(f.type, recs); + const QString fn = qs(f.name); + // A missing field is reported at its own path rather than + // default-constructed: a record that silently loses a field is the + // failure mode this whole layer exists to prevent. + s << " out." << fn << " = Codec<" << ft << ">::from(\n"; + s << " j.contains(\"" << fn << "\") ? j.at(\"" << fn + << "\") : nlohmann::json(),\n"; + s << " path + \"." << fn << "\");\n"; + } + s << " return out;\n }\n};\n\n"; + } + s << "} // namespace logos_gen\n\n"; +} + bool hasBytesEventParam(const ModuleDecl& module) { for (const EventDecl& ed : module.events) for (const ParamDecl& pd : ed.params) - if ((pd.type.kind == TypeExpr::Primitive && pd.type.name == "bstr") - || isBytesArray(pd.type)) + if (pd.type.kind == TypeExpr::Primitive && pd.type.name == "bstr") return true; return false; } -// True when an EVENT carries `[bstr]`, so the sidecar needs the list encoder on -// top of the scalar one. Method params/returns are handled in the impl TU. -bool hasBytesArrayEventParam(const ModuleDecl& module) +// The Qt spelling of what actually crosses the Qt boundary. +// +// NOT lidlTypeToQt: that answers the CONSUMER's question ("what type does the +// caller hold?") and since records became real structs it answers `Blob` / +// `QList`. Those names are correct in a generated consumer wrapper, where +// the struct exists — but this JSON is the module's getMethods(), read by the +// host to marshal a QVariant across the plugin boundary, and there is no +// metatype called `Blob`. Emitting it made the host SIGSEGV on the first call +// to any record method. +// +// A record IS a variant map at that boundary; the struct only exists inside the +// cdylib. +QString lidlTypeToQtWire(const TypeExpr& te, const std::set& recs) { - for (const EventDecl& ed : module.events) - for (const ParamDecl& pd : ed.params) - if (isBytesArray(pd.type)) - return true; - return false; + if (isRecord(te, recs)) + return "QVariantMap"; + if (te.kind == TypeExpr::Array && te.elements.size() == 1 + && isRecord(te.elements[0], recs)) + return "QVariantList"; + if (te.kind == TypeExpr::Map && te.elements.size() == 2 + && isRecord(te.elements[1], recs)) + return "QVariantMap"; + return lidlTypeToQt(te); } // True when any event parameter is spelled LogosMap / LogosList, so the sidecar // needs for those aliases. bool hasJsonEventParam(const ModuleDecl& module) { + const std::set recs = recordNames(module); for (const EventDecl& ed : module.events) for (const ParamDecl& pd : ed.params) { - const QString t = lidlTypeToStdCdylib(pd.type); + const QString t = lidlTypeToStdCdylib(pd.type, recs); if (t == "LogosMap" || t == "LogosList") return true; } return false; } -// `withList` additionally emits lidlBytesListToJson for `[bstr]`. Off by -// default so a module that never carries a byte-string array does not gain an -// unused static function. -void emitBytesEncodeHelpers(QTextStream& s, bool withList = false) +// The SCALAR tagged-bytes helpers. A `[bstr]` (and bytes at any deeper +// nesting) rides logos_gen::Codec instead: its full specialization for +// std::vector beats the generic vector rule, so one mechanism covers +// [bstr], [[bstr]] and {tstr: [bstr]} alike. #111 emitted a dedicated depth-1 +// list codec here; the generic one subsumes it, and keeping both left an +// unused static in every module that mentioned [bstr]. +void emitBytesEncodeHelpers(QTextStream& s) { s << "// Canonical tagged bytes form {\"_bytes\": base64url} (see logos_protocol.h)\n"; s << "std::string lidlB64UrlEncode(const std::vector& bytes)\n{\n"; @@ -212,17 +368,11 @@ void emitBytesEncodeHelpers(QTextStream& s, bool withList = false) s << "nlohmann::json lidlBytesToJson(const std::vector& bytes)\n{\n"; s << " return nlohmann::json{{\"_bytes\", lidlB64UrlEncode(bytes)}};\n}\n\n"; - if (withList) { - s << "nlohmann::json lidlBytesListToJson(const std::vector>& list)\n{\n"; - s << " nlohmann::json out = nlohmann::json::array();\n"; - s << " for (const std::vector& bytes : list)\n"; - s << " out.push_back(lidlBytesToJson(bytes));\n"; - s << " return out;\n}\n\n"; - } } void emitInterfaceJson(QTextStream& s, const ModuleDecl& module) { + const std::set recs = recordNames(module); s << "static nlohmann::json lidlInterfaceJson()\n{\n"; s << " nlohmann::json methods = nlohmann::json::array();\n"; for (const MethodDecl& md : module.methods) { @@ -235,17 +385,17 @@ void emitInterfaceJson(QTextStream& s, const ModuleDecl& module) } QString sig = qs(md.name) + "("; for (int i = 0; i < md.params.size(); ++i) { - sig += lidlTypeToQt(md.params[i].type); + sig += lidlTypeToQtWire(md.params[i].type, recs); if (i + 1 < md.params.size()) sig += ","; } sig += ")"; s << " obj[\"signature\"] = \"" << sig << "\";\n"; - s << " obj[\"returnType\"] = \"" << lidlTypeToQt(md.returnType) << "\";\n"; + s << " obj[\"returnType\"] = \"" << lidlTypeToQtWire(md.returnType, recs) << "\";\n"; s << " obj[\"isInvokable\"] = true;\n"; if (!md.params.empty()) { s << " nlohmann::json params = nlohmann::json::array();\n"; for (const ParamDecl& pd : md.params) { - s << " params.push_back({{\"type\", \"" << lidlTypeToQt(pd.type) + s << " params.push_back({{\"type\", \"" << lidlTypeToQtWire(pd.type, recs) << "\"}, {\"name\", \"" << pd.name << "\"}});\n"; } s << " obj[\"parameters\"] = params;\n"; @@ -263,7 +413,7 @@ void emitInterfaceJson(QTextStream& s, const ModuleDecl& module) } QString sig = qs(ed.name) + "("; for (int i = 0; i < ed.params.size(); ++i) { - sig += lidlTypeToQt(ed.params[i].type); + sig += lidlTypeToQtWire(ed.params[i].type, recs); if (i + 1 < ed.params.size()) sig += ","; } sig += ")"; @@ -271,7 +421,7 @@ void emitInterfaceJson(QTextStream& s, const ModuleDecl& module) if (!ed.params.empty()) { s << " nlohmann::json params = nlohmann::json::array();\n"; for (const ParamDecl& pd : ed.params) { - s << " params.push_back({{\"type\", \"" << lidlTypeToQt(pd.type) + s << " params.push_back({{\"type\", \"" << lidlTypeToQtWire(pd.type, recs) << "\"}, {\"name\", \"" << pd.name << "\"}});\n"; } s << " obj[\"parameters\"] = params;\n"; @@ -285,9 +435,10 @@ void emitInterfaceJson(QTextStream& s, const ModuleDecl& module) bool lidlCdylibSupported(const ModuleDecl& module, QString* error) { + const std::set recs = recordNames(module); for (const MethodDecl& md : module.methods) { for (const ParamDecl& pd : md.params) { - if (!typeSupported(pd.type, /*isReturn=*/false)) { + if (!typeSupported(pd.type, /*isReturn=*/false, recs)) { if (error) *error = QString("method '%1': parameter '%2' has a type outside the " "cdylib-supported (Qt-free) subset") @@ -302,7 +453,7 @@ bool lidlCdylibSupported(const ModuleDecl& module, QString* error) md.returnType.name == "void" || (md.returnType.kind == TypeExpr::Primitive && md.returnType.name.empty()); if (!voidReturn && !md.jsonReturn && !md.resultReturn - && !typeSupported(md.returnType, /*isReturn=*/true)) { + && !typeSupported(md.returnType, /*isReturn=*/true, recs)) { if (error) *error = QString("method '%1': return type outside the cdylib-supported " "(Qt-free) subset").arg(qs(md.name)); @@ -311,7 +462,7 @@ bool lidlCdylibSupported(const ModuleDecl& module, QString* error) } for (const EventDecl& ed : module.events) { for (const ParamDecl& pd : ed.params) { - if (!typeSupported(pd.type, /*isReturn=*/false)) { + if (!typeSupported(pd.type, /*isReturn=*/false, recs)) { if (error) *error = QString("event '%1': parameter '%2' has a type outside the " "cdylib-supported (Qt-free) subset") @@ -323,10 +474,47 @@ bool lidlCdylibSupported(const ModuleDecl& module, QString* error) return true; } +QString lidlMakeTypesHeaderCdylib(const ModuleDecl& module) +{ + const std::set recs = recordNames(module); + QString c; + QTextStream s(&c); + s << "// AUTO-GENERATED by logos-cpp-generator --backend cdylib -- do not edit\n"; + s << "//\n"; + s << "// The record types `" << module.name << "` declares, plus the codec that moves\n"; + s << "// them across the wire. Qt-FREE. The author's impl header includes this and\n"; + s << "// writes the structs directly:\n"; + s << "//\n"; + s << "// Blob echoBlob(const Blob& v);\n"; + s << "//\n"; + s << "// rather than picking fields out of a LogosMap.\n"; + s << "#pragma once\n"; + s << "#include \n"; + s << "#include \n"; + s << "#include \n"; + s << "#include \n"; + s << "#include \n"; + s << "#include \n\n"; + + // The structs themselves are the AUTHOR's: this file is included after the + // impl header, and the contract was derived from those very declarations, + // so emitting them again is a redefinition error. Only forward + // declarations, so the codec below can name them in any order. + if (!module.types.empty()) { + for (const TypeDecl& t : module.types) + s << "struct " << qs(t.name) << ";\n"; + s << "\n"; + } + + emitGeneratedCodec(s, module, recs); + return c; +} + QString lidlMakeModuleImplExports(const ModuleDecl& module, const QString& implClass, const QString& implHeader) { + const std::set recs = recordNames(module); QString c; QTextStream s(&c); @@ -337,6 +525,7 @@ QString lidlMakeModuleImplExports(const ModuleDecl& module, s << "// module's cdylib; the uniform Qt-plugin glue (or a future no-Qt host)\n"; s << "// drives it exclusively through these symbols.\n"; s << "#include \"" << implHeader << "\"\n"; + s << "#include \"" << module.name << "_types.h\"\n"; s << "#include \"logos_module_impl.h\"\n"; s << "#include \"logos_protocol.h\"\n"; s << "#include \"logos_module_context.h\"\n"; @@ -373,8 +562,7 @@ QString lidlMakeModuleImplExports(const ModuleDecl& module, s << " if (out) std::memcpy(out, str.data(), str.size() + 1);\n"; s << " return out;\n}\n\n"; - const bool bytesList = usesBytesArray(module); - emitBytesEncodeHelpers(s, bytesList); + emitBytesEncodeHelpers(s); s << "int lidlB64Idx(char ch)\n{\n"; s << " if (ch >= 'A' && ch <= 'Z') return ch - 'A';\n"; @@ -431,20 +619,6 @@ QString lidlMakeModuleImplExports(const ModuleDecl& module, s << " out.push_back((n >> 8) & 0xff);\n"; s << " }\n }\n return out;\n}\n\n"; - if (bytesList) { - s << "std::vector> lidlBytesListFromJson(const nlohmann::json& j)\n{\n"; - s << " std::vector> out;\n"; - s << " // Each element runs through the lenient scalar decode above, so a\n"; - s << " // caller may send tagged {\"_bytes\": base64url} objects, plain\n"; - s << " // strings or number arrays — element by element. A non-array arg\n"; - s << " // yields an empty list rather than throwing, matching the scalar\n"; - s << " // decoder's behaviour on an unexpected shape.\n"; - s << " if (!j.is_array()) return out;\n"; - s << " out.reserve(j.size());\n"; - s << " for (const auto& e : j)\n"; - s << " out.push_back(lidlBytesFromJson(e));\n"; - s << " return out;\n}\n\n"; - } s << "nlohmann::json lidlResultToJson(const StdLogosResult& r)\n{\n"; s << " nlohmann::json obj;\n"; @@ -534,7 +708,8 @@ QString lidlMakeModuleImplExports(const ModuleDecl& module, QString call = "lidlImpl()." + qs(md.name) + "("; for (int i = 0; i < md.params.size(); ++i) { call += jsonArgToStd(md.params[i].type, - QString("args.at(%1)").arg(i)); + QString("args.at(%1)").arg(i), + QString("arg%1").arg(i), recs); if (i + 1 < md.params.size()) call += ", "; } call += ")"; @@ -549,7 +724,7 @@ QString lidlMakeModuleImplExports(const ModuleDecl& module, s << " return lidlStrdup(\"true\");\n"; } else { s << " auto result = " << call << ";\n"; - s << " return lidlStrdup(" << stdReturnToJson(md, "result") << ".dump());\n"; + s << " return lidlStrdup(" << stdReturnToJson(md, "result", recs) << ".dump());\n"; } s << " }\n"; } @@ -617,9 +792,12 @@ QString lidlMakeEventsSourceCdylib(const ModuleDecl& module, s << "// Typed `logos_events:` bodies, cdylib flavor: marshal into\n"; s << "// nlohmann::json and route through LogosModuleContext::emitEventImpl_\n"; s << "// (the export wrapper forwards to the host's emit callback).\n"; + const std::set recsEv = recordNames(module); s << "#include \"" << implHeader << "\"\n"; + s << "#include \"" << module.name << "_types.h\"\n"; s << "#include \n\n"; s << "#include \n"; + s << "#include \n"; s << "#include \n"; s << "#include \n"; // LogosMap / LogosList (nlohmann aliases) appear in the emitted signatures @@ -632,17 +810,24 @@ QString lidlMakeEventsSourceCdylib(const ModuleDecl& module, // encoder; emitting it everywhere would leave it unused (and warned about). if (hasBytesEventParam(module)) { s << "namespace {\n\n"; - emitBytesEncodeHelpers(s, hasBytesArrayEventParam(module)); + emitBytesEncodeHelpers(s); s << "} // namespace\n\n"; } for (const EventDecl& ed : module.events) { s << "void " << implClass << "::" << ed.name << "("; for (int i = 0; i < ed.params.size(); ++i) { - const QString stdType = lidlTypeToStdCdylib(ed.params[i].type); + const QString stdType = lidlTypeToStdCdylib(ed.params[i].type, recsEv); // Must match the author's declaration in the `logos_events:` block: // the non-scalar types are conventionally taken by const-ref there. + // Records and std::map belong in that set too — they are structs and + // containers, and emitting them BY VALUE makes the generated + // definition not match the author's declaration, which is a compile + // error naming a parameter type mismatch rather than anything + // helpful. if (stdType == "std::string" || stdType.startsWith("std::vector") + || stdType.startsWith("std::map") + || isRecord(ed.params[i].type, recsEv) || stdType == "LogosMap" || stdType == "LogosList") s << "const " << stdType << "& " << ed.params[i].name; else @@ -652,10 +837,19 @@ QString lidlMakeEventsSourceCdylib(const ModuleDecl& module, s << ")\n{\n"; s << " nlohmann::json args = nlohmann::json::array();\n"; for (const ParamDecl& pd : ed.params) { + const QString evStd = lidlTypeToStdCdylib(pd.type, recsEv); + // A record or a composite carrying bytes rides the generated codec, + // exactly like a method return — otherwise an event payload would be + // the one place a bstr silently loses its tag. + if (evStd != "LogosMap" && evStd != "LogosList" + && (isRecord(pd.type, recsEv) + || pd.type.kind == TypeExpr::Array || pd.type.kind == TypeExpr::Map)) { + s << " args.push_back(logos_gen::Codec<" << evStd << ">::to(" + << pd.name << "));\n"; + continue; + } if (pd.type.kind == TypeExpr::Primitive && pd.type.name == "bstr") s << " args.push_back(lidlBytesToJson(" << pd.name << "));\n"; - else if (isBytesArray(pd.type)) - s << " args.push_back(lidlBytesListToJson(" << pd.name << "));\n"; else s << " args.push_back(" << pd.name << ");\n"; } diff --git a/cpp-generator/experimental/lidl_gen_cdylib.h b/cpp-generator/experimental/lidl_gen_cdylib.h index 0fd4f42..78f772c 100644 --- a/cpp-generator/experimental/lidl_gen_cdylib.h +++ b/cpp-generator/experimental/lidl_gen_cdylib.h @@ -29,6 +29,11 @@ // cdylib-supported subset. bool lidlCdylibSupported(const ModuleDecl& module, QString* error); +// The record structs a contract declares, plus their codec — a Qt-free header +// the author's impl class includes so it can name the structs directly. +// Empty of types (but still valid) when the contract declares no records. +QString lidlMakeTypesHeaderCdylib(const ModuleDecl& module); + QString lidlMakeModuleImplExports(const ModuleDecl& module, const QString& implClass, const QString& implHeader); diff --git a/cpp-generator/experimental/lidl_gen_client.cpp b/cpp-generator/experimental/lidl_gen_client.cpp index 492fdf5..5a256c2 100644 --- a/cpp-generator/experimental/lidl_gen_client.cpp +++ b/cpp-generator/experimental/lidl_gen_client.cpp @@ -17,8 +17,14 @@ static bool isRefType(const QString& qt) { - return qt == "QString" || qt == "QStringList" || qt == "QJsonArray" - || qt == "QVariantList" || qt == "QVariantMap" || qt == "QByteArray"; + if (qt == "QString" || qt == "QStringList" || qt == "QJsonArray" + || qt == "QVariantList" || qt == "QVariantMap" || qt == "QByteArray") + return true; + // A record is a struct: pass it by const& too. Anything that is not a known + // Qt scalar/handle spelling is a generated record type. + return !(qt == "bool" || qt == "int" || qt == "double" || qt == "float" + || qt == "void" || qt == "qlonglong" || qt == "qulonglong" + || qt == "QVariant" || qt == "LogosResult"); } static void emitParam(QTextStream& s, const QString& qtType, const std::string& name) @@ -29,10 +35,18 @@ static void emitParam(QTextStream& s, const QString& qtType, const std::string& s << qtType << " " << name; } +static bool lidlIsRecord(const TypeExpr& te); +static QString qtToVariantExpr(const TypeExpr& te, const QString& expr); +static QString qtFromVariantExpr(const TypeExpr& te, const QString& expr); +static QString returnConversionFor(const TypeExpr& te, const QString& qt); + static QString returnConversion(const QString& qt) { if (qt == "bool") return "return _result.toBool();"; - if (qt == "int") return "return _result.toInt();"; + // 64-bit, matching lidlTypeToQt: toInt() truncated a LIDL int/uint, and for + // uint it also read the value as signed. + if (qt == "qlonglong") return "return _result.toLongLong();"; + if (qt == "qulonglong") return "return _result.toULongLong();"; if (qt == "double") return "return _result.toDouble();"; if (qt == "float") return "return _result.toFloat();"; if (qt == "QString") return "return _result.toString();"; @@ -44,6 +58,39 @@ static QString returnConversion(const QString& qt) return "return _result;"; } +// Records (and containers holding them) decode through the generated +// conversions; everything else keeps the historical QVariant accessor. +static QString returnConversionFor(const TypeExpr& te, const QString& qt) +{ + const bool holdsRecord = + lidlIsRecord(te) + || (te.kind == TypeExpr::Array && te.elements.size() == 1 && lidlIsRecord(te.elements[0])) + || (te.kind == TypeExpr::Map && te.elements.size() == 2 && lidlIsRecord(te.elements[1])); + if (holdsRecord) + return "return " + qtFromVariantExpr(te, "_result") + ";"; + return returnConversion(qt); +} + +// The async twin of returnConversionFor: `v` is the wire QVariant. +// +// A record-bearing return MUST decode field by field here too. The wire carries +// a QVariantMap and no Q_DECLARE_METATYPE is emitted for the struct, so +// `qvariant_cast(v)` does not fail — it silently returns a +// DEFAULT-CONSTRUCTED Status, and the caller sees empty fields with no +// diagnostic. That is the worst failure mode available: the sync path is +// correct, so the same call is right or wrong depending only on which overload +// the caller reached for. +static QString asyncReturnConversionFor(const TypeExpr& te, const QString& qt) +{ + const bool holdsRecord = + lidlIsRecord(te) + || (te.kind == TypeExpr::Array && te.elements.size() == 1 && lidlIsRecord(te.elements[0])) + || (te.kind == TypeExpr::Map && te.elements.size() == 2 && lidlIsRecord(te.elements[1])); + if (holdsRecord) + return qtFromVariantExpr(te, "v"); + return "qvariant_cast<" + qt + ">(v)"; +} + static QString asyncDefaultVal(const QString& qt) { if (qt == "bool") return "false"; @@ -56,6 +103,114 @@ static QString asyncDefaultVal(const QString& qt) return qt + "{}"; } + +// --------------------------------------------------------------------------- +// Records +// +// A `type Foo { … }` in the contract becomes a real C++ struct plus two inline +// conversions, so a Qt consumer says `Status s = client.makeStatus();` instead +// of digging fields out of a QVariantMap. One LIDL type, one type per language. +// +// bstr fields are QByteArray on purpose: logos-protocol's QVariant<->JSON +// conversion already materialises the canonical {"_bytes": base64url} form as a +// QByteArray and back (logos_json_convert.cpp), so the record conversions stay +// pure field mapping and binary survives at any depth for free. +// --------------------------------------------------------------------------- + +static bool lidlIsRecord(const TypeExpr& te) +{ + return te.kind == TypeExpr::Named && !te.name.empty(); +} + +// value expression of the Qt type -> QVariant +static QString qtToVariantExpr(const TypeExpr& te, const QString& expr) +{ + if (lidlIsRecord(te)) + return qs(te.name) + "ToVariant(" + expr + ")"; + if (te.kind == TypeExpr::Array && te.elements.size() == 1 + && (lidlIsRecord(te.elements[0]) || te.elements[0].kind != TypeExpr::Primitive)) { + return "[&]{ QVariantList __l; for (const auto& __e : " + expr + ") __l.append(" + + qtToVariantExpr(te.elements[0], "__e") + "); return QVariant(__l); }()"; + } + if (te.kind == TypeExpr::Map && te.elements.size() == 2 + && (lidlIsRecord(te.elements[1]) || te.elements[1].kind != TypeExpr::Primitive)) { + return "[&]{ QVariantMap __m; for (auto __it = " + expr + ".begin(); __it != " + expr + + ".end(); ++__it) __m.insert(__it.key(), " + + qtToVariantExpr(te.elements[1], "__it.value()") + "); return QVariant(__m); }()"; + } + return "QVariant::fromValue(" + expr + ")"; +} + +// QVariant expression -> value of the Qt type +static QString qtFromVariantExpr(const TypeExpr& te, const QString& expr) +{ + if (lidlIsRecord(te)) + return qs(te.name) + "FromVariant(" + expr + ")"; + if (te.kind == TypeExpr::Primitive) { + const QString n = qs(te.name); + if (n == "tstr") return expr + ".toString()"; + if (n == "bstr") return expr + ".toByteArray()"; + if (n == "int") return expr + ".toLongLong()"; + if (n == "uint") return expr + ".toULongLong()"; + if (n == "float64") return expr + ".toDouble()"; + if (n == "bool") return expr + ".toBool()"; + } + if (te.kind == TypeExpr::Array && te.elements.size() == 1) { + const TypeExpr& e = te.elements[0]; + return "[&]{ " + lidlTypeToQt(te) + " __acc; for (const QVariant& __e : " + expr + + ".toList()) __acc.append(" + qtFromVariantExpr(e, "__e") + "); return __acc; }()"; + } + if (te.kind == TypeExpr::Map && te.elements.size() == 2) { + const TypeExpr& v = te.elements[1]; + return "[&]{ " + lidlTypeToQt(te) + " __acc; const QVariantMap __mm = " + expr + + ".toMap(); for (auto __it = __mm.begin(); __it != __mm.end(); ++__it) __acc.insert(" + + "__it.key(), " + qtFromVariantExpr(v, "__it.value()") + "); return __acc; }()"; + } + return expr; +} + +// A method argument as passed to packVariantList: records convert, everything +// else goes through unchanged (packVariantList wraps with QVariant::fromValue). +static QString qtArgExpr(const TypeExpr& te, const QString& name) +{ + const bool holdsRecord = + lidlIsRecord(te) + || (te.kind == TypeExpr::Array && te.elements.size() == 1 && lidlIsRecord(te.elements[0])) + || (te.kind == TypeExpr::Map && te.elements.size() == 2 && lidlIsRecord(te.elements[1])); + return holdsRecord ? qtToVariantExpr(te, name) : name; +} + +static void emitRecords(QTextStream& s, const ModuleDecl& module) +{ + if (module.types.empty()) return; + for (const TypeDecl& t : module.types) { + const QString n = qs(t.name); + s << "/// `" << n << "` — a record declared by the `" << qs(module.name) << "` contract.\n"; + s << "struct " << n << " {\n"; + for (const FieldDecl& f : t.fields) + s << " " << lidlTypeToQt(f.type) << " " << qs(f.name) << "{};\n"; + s << "};\n\n"; + } + // Conversions come after ALL structs so records may reference each other. + for (const TypeDecl& t : module.types) { + const QString n = qs(t.name); + s << "inline QVariant " << n << "ToVariant(const " << n << "& v)\n{\n"; + s << " QVariantMap __m;\n"; + for (const FieldDecl& f : t.fields) + s << " __m.insert(\"" << qs(f.name) << "\", " + << qtToVariantExpr(f.type, "v." + qs(f.name)) << ");\n"; + s << " return QVariant(__m);\n}\n\n"; + + s << "inline " << n << " " << n << "FromVariant(const QVariant& value)\n{\n"; + s << " const QVariantMap __m = value.toMap();\n"; + s << " " << n << " __out;\n"; + for (const FieldDecl& f : t.fields) + s << " __out." << qs(f.name) << " = " + << qtFromVariantExpr(f.type, "__m.value(\"" + qs(f.name) + "\")") << ";\n"; + s << " return __out;\n}\n\n"; + } +} + // --------------------------------------------------------------------------- // Header generation // --------------------------------------------------------------------------- @@ -81,6 +236,8 @@ QString lidlMakeHeader(const ModuleDecl& module, BindMode bindMode) s << "#include \"logos_call_error.h\"\n"; s << "#include \"logos_object.h\"\n\n"; + emitRecords(s, module); + s << "class " << className << " {\n"; s << "public:\n"; if (bindMode == BindMode::Bound) @@ -240,7 +397,7 @@ QString lidlMakeSource(const ModuleDecl& module, BindMode bindMode) // one — the historical "typed arrays empty over the Qt path" bug. s << "m_client->invokeRemoteMethod(" << targetExpr << ", \"" << md.name << "\", packVariantList("; for (int i = 0; i < nParams; ++i) { - s << md.params[i].name; + s << qtArgExpr(md.params[i].type, qs(md.params[i].name)); if (i + 1 < nParams) s << ", "; } s << "), Timeout(), &_err);\n"; @@ -249,7 +406,7 @@ QString lidlMakeSource(const ModuleDecl& module, BindMode bindMode) << ": remote call failed:\" << QString::fromStdString(_err.message);\n"; if (ret != "void") - s << " " << returnConversion(ret) << "\n"; + s << " " << returnConversionFor(md.returnType, ret) << "\n"; s << "}\n\n"; s << "void " << className << "::" << md.name << "Async("; @@ -264,7 +421,7 @@ QString lidlMakeSource(const ModuleDecl& module, BindMode bindMode) // QVariantList-typed arg must not be spread across the args list. s << " m_client->invokeRemoteMethodAsync(" << targetExpr << ", \"" << md.name << "\", packVariantList("; for (int i = 0; i < nParams; ++i) { - s << md.params[i].name; + s << qtArgExpr(md.params[i].type, qs(md.params[i].name)); if (i + 1 < nParams) s << ", "; } s << ")"; @@ -274,7 +431,8 @@ QString lidlMakeSource(const ModuleDecl& module, BindMode bindMode) } else if (ret == "QVariant") { s << " callback(v);\n"; } else { - s << " callback(v.isValid() ? qvariant_cast<" << ret << ">(v) : " << asyncDefaultVal(ret) << ");\n"; + s << " callback(v.isValid() ? " << asyncReturnConversionFor(md.returnType, ret) + << " : " << asyncDefaultVal(ret) << ");\n"; } s << " }, timeout);\n"; s << "}\n\n"; @@ -323,6 +481,10 @@ int lidlGenerateClientStubs(const QString& lidlPath, const QString& outputDir, LidlValidationResult vr = lidlValidate(pr.module); if (vr.hasErrors()) { for (const std::string& e : vr.errors) err << lidlPath << ": " << e << "\n"; return 5; } + { + QString recErr; + if (!lidlCheckRecords(pr.module, &recErr)) { err << lidlPath << ": " << recErr << "\n"; return 5; } + } const ModuleDecl& mod = pr.module; QString genDirPath = outputDir.isEmpty() ? QDir::current().filePath("logos-cpp-sdk/cpp/generated") : outputDir; diff --git a/cpp-generator/legacy/generator_lib.cpp b/cpp-generator/legacy/generator_lib.cpp index 518f197..78154b6 100644 --- a/cpp-generator/legacy/generator_lib.cpp +++ b/cpp-generator/legacy/generator_lib.cpp @@ -34,7 +34,7 @@ QString mapParamType(const QString& qtType) { const QString base = normalizeType(qtType); static const QSet known = { - "void","bool","int","double","float","QString","QStringList","QByteArray","QJsonArray","QVariantList","QVariantMap","QVariant" + "void","bool","int","qlonglong","qulonglong","double","float","QString","QStringList","QByteArray","QJsonArray","QVariantList","QVariantMap","QVariant" }; if (known.contains(base)) return base; // Fallback to QVariant for unknown types @@ -46,7 +46,7 @@ QString mapReturnType(const QString& qtType) const QString base = normalizeType(qtType); if (base.isEmpty() || base == "void") return QString("void"); static const QSet known = { - "bool","int","double","float","QString","QStringList","QByteArray","QJsonArray","QVariantList","QVariantMap","QVariant","LogosResult" + "bool","int","qlonglong","qulonglong","double","float","QString","QStringList","QByteArray","QJsonArray","QVariantList","QVariantMap","QVariant","LogosResult" }; if (known.contains(base)) return base; return QString("QVariant"); @@ -55,6 +55,9 @@ QString mapReturnType(const QString& qtType) QString toQVariantConversion(const QString& type, const QString& argExpr) { if (type == "int") return argExpr + ".toInt()"; + // LIDL int/uint are 64-bit; toInt() would truncate and re-sign them. + if (type == "qlonglong") return argExpr + ".toLongLong()"; + if (type == "qulonglong") return argExpr + ".toULongLong()"; if (type == "bool") return argExpr + ".toBool()"; if (type == "double") return argExpr + ".toDouble()"; if (type == "float") return argExpr + ".toFloat()"; @@ -91,6 +94,8 @@ static QString mapParamTypeStd(const QString& qtType) if (base == "QVariantMap") return "LogosMap"; if (base == "QVariant") return "LogosMap"; if (base == "int") return "int64_t"; + if (base == "qlonglong") return "int64_t"; + if (base == "qulonglong") return "uint64_t"; return base; } @@ -107,6 +112,8 @@ static QString mapReturnTypeStd(const QString& qtType) if (base == "QVariant") return "LogosMap"; if (base == "LogosResult") return "StdLogosResult"; if (base == "int") return "int64_t"; + if (base == "qlonglong") return "int64_t"; + if (base == "qulonglong") return "uint64_t"; return base; } @@ -155,6 +162,10 @@ static QString qVariantToStdReturn(const QString& qtType, const QString& varExpr return varExpr + ".toBool()"; if (base == "int") return "static_cast(" + varExpr + ".toInt())"; + if (base == "qlonglong") + return varExpr + ".toLongLong()"; + if (base == "qulonglong") + return varExpr + ".toULongLong()"; if (base == "double" || base == "float") return varExpr + ".toDouble()"; if (base == "QString") @@ -181,10 +192,171 @@ static QString qVariantToStdReturn(const QString& qtType, const QString& varExpr return varExpr + ".toString().toStdString()"; } +// ─── Records ───────────────────────────────────────────────────────────── +// +// A contract's `type Status { port: uint }` is a REAL C++ struct on the +// consumer side, not a QVariant / LogosMap the caller picks apart by string +// key. Without this a `bstr` field is the worst case: the caller receives the +// canonical `{"_bytes": "..."}` envelope and has to know to unwrap it, while +// every other language's consumer hands back plain bytes. +// +// main.cpp passes the declarations alongside the methods: +// [ { "name": "Status", "fields": [ { "name": "port", "type": "qulonglong" } ] } ] +// spelled with the same Qt type names methods use, so a field can name another +// record ("Status"), a list of them ("QList") or a map of them +// ("QMap"). The struct is nested in the wrapper class — +// `InfoModule::Status` — because one module consuming two deps that each +// declare `Status` includes both wrappers into the same translation unit. +// +// An empty record set leaves every emission path byte-for-byte as it was. + +struct RecordField { QString name; QString type; }; +struct RecordDef { QString name; QVector fields; }; +using RecordSet = QVector; + +// Forward declarations: the Lp (Qt-free) conversion helpers live further down +// with the rest of the Lp backend, but the record helpers below dispatch to +// them for non-record field types. +static QString lpPushExpr(const QString& qtType, const QString& argName); +static QString lpFromJsonExpr(const QString& qtType, const QString& jv); + +static RecordSet parseRecords(const QJsonArray& records) +{ + RecordSet out; + for (const QJsonValue& rv : records) { + const QJsonObject ro = rv.toObject(); + RecordDef def; + def.name = ro.value("name").toString(); + if (def.name.isEmpty()) continue; + for (const QJsonValue& fv : ro.value("fields").toArray()) { + const QJsonObject fo = fv.toObject(); + RecordField f; + f.name = fo.value("name").toString(); + f.type = fo.value("type").toString(); + if (f.name.isEmpty()) continue; + def.fields.append(f); + } + out.append(def); + } + return out; +} + +static bool isRecordName(const RecordSet& rs, const QString& name) +{ + for (const RecordDef& d : rs) if (d.name == name) return true; + return false; +} + +// How a type name mentions a record, if at all. +enum class RecordShape { None, Scalar, List, Map }; + +static RecordShape recordShape(const RecordSet& rs, const QString& t, QString* elem) +{ + if (rs.isEmpty()) return RecordShape::None; + if (isRecordName(rs, t)) { if (elem) *elem = t; return RecordShape::Scalar; } + if (t.startsWith("QList<") && t.endsWith(">")) { + const QString e = t.mid(6, t.size() - 7).trimmed(); + if (isRecordName(rs, e)) { if (elem) *elem = e; return RecordShape::List; } + } + if (t.startsWith("QMap")) { + const QString e = t.mid(13, t.size() - 14).trimmed(); + if (isRecordName(rs, e)) { if (elem) *elem = e; return RecordShape::Map; } + } + return RecordShape::None; +} + +// The C++ spelling of a record-bearing type, or empty when `t` names none. +// `qual` qualifies the nested struct ("InfoModule::") where class scope does +// not already apply — i.e. a return type written before the `Class::` in a +// definition. +static QString recordCppType(const RecordSet& rs, const QString& t, ApiStyle style, const QString& qual) +{ + QString elem; + const RecordShape shape = recordShape(rs, t, &elem); + const QString q = qual + elem; + switch (shape) { + case RecordShape::None: return QString(); + case RecordShape::Scalar: return q; + case RecordShape::List: + return style == ApiStyle::Qt ? "QList<" + q + ">" : "std::vector<" + q + ">"; + case RecordShape::Map: + return style == ApiStyle::Qt ? "QMap" + : "std::map"; + } + return QString(); +} + +// File-local conversion helpers emitted into the generated .cpp — never into +// the header, so a std/lp consumer's own translation units stay free of the +// wire type (QVariant / nlohmann::json) the conversion is written in. +static QString recToWireFn(const QString& record) { return "recToWire_" + record; } +static QString recFromWireFn(const QString& record) { return "recFromWire_" + record; } + +// Record value -> wire value, and back. Empty when `t` names no record. +static QString recordToWireExpr(const RecordSet& rs, const QString& t, ApiStyle style, const QString& expr) +{ + QString elem; + const RecordShape shape = recordShape(rs, t, &elem); + if (shape == RecordShape::None) return QString(); + const QString conv = recToWireFn(elem); + if (shape == RecordShape::Scalar) return conv + "(" + expr + ")"; + // Locals are named apart from the record encoder/decoder's own `__m` / `__j` + // / `__out`: these lambdas are emitted INSIDE those functions when a record + // has a container-of-record field, and a shadowing local silently reads + // itself (caught by -Wuninitialized, not by any assertion on the text). + if (style == ApiStyle::Lp) { + if (shape == RecordShape::List) + return "[&]{ nlohmann::json __acc = nlohmann::json::array(); for (const auto& __e : " + + expr + ") __acc.push_back(" + conv + "(__e)); return __acc; }()"; + return "[&]{ nlohmann::json __acc = nlohmann::json::object(); for (const auto& __kv : " + + expr + ") __acc[__kv.first] = " + conv + "(__kv.second); return __acc; }()"; + } + if (shape == RecordShape::List) + return "[&]{ QVariantList __acc; for (const auto& __e : " + expr + + ") __acc.append(" + conv + "(__e)); return __acc; }()"; + // Qt keys are QString, std keys std::string. + if (style == ApiStyle::Qt) + return "[&]{ QVariantMap __acc; for (auto __i = " + expr + ".cbegin(); __i != " + expr + + ".cend(); ++__i) __acc.insert(__i.key(), " + conv + "(__i.value())); return __acc; }()"; + return "[&]{ QVariantMap __acc; for (const auto& __kv : " + expr + + ") __acc.insert(QString::fromStdString(__kv.first), " + conv + "(__kv.second)); return __acc; }()"; +} + +static QString recordFromWireExpr(const RecordSet& rs, const QString& t, ApiStyle style, + const QString& wire, const QString& qual) +{ + QString elem; + const RecordShape shape = recordShape(rs, t, &elem); + if (shape == RecordShape::None) return QString(); + const QString conv = recFromWireFn(elem); + const QString cpp = recordCppType(rs, t, style, qual); + if (shape == RecordShape::Scalar) return conv + "(" + wire + ")"; + if (style == ApiStyle::Lp) { + if (shape == RecordShape::List) + return "[&]{ " + cpp + " __acc; const nlohmann::json& __src = " + wire + + "; if (__src.is_array()) for (const auto& __e : __src) __acc.push_back(" + conv + + "(__e)); return __acc; }()"; + return "[&]{ " + cpp + " __acc; const nlohmann::json& __src = " + wire + + "; if (__src.is_object()) for (auto __i = __src.begin(); __i != __src.end(); ++__i) " + "__acc[__i.key()] = " + conv + "(__i.value()); return __acc; }()"; + } + if (shape == RecordShape::List) + return "[&]{ " + cpp + " __acc; for (const QVariant& __e : (" + wire + + ").toList()) __acc.push_back(" + conv + "(__e)); return __acc; }()"; + if (style == ApiStyle::Qt) + return "[&]{ " + cpp + " __acc; const QVariantMap __src = (" + wire + + ").toMap(); for (auto __i = __src.cbegin(); __i != __src.cend(); ++__i) " + "__acc.insert(__i.key(), " + conv + "(__i.value())); return __acc; }()"; + return "[&]{ " + cpp + " __acc; const QVariantMap __src = (" + wire + + ").toMap(); for (auto __i = __src.cbegin(); __i != __src.cend(); ++__i) " + "__acc[__i.key().toStdString()] = " + conv + "(__i.value()); return __acc; }()"; +} + // Param-type predicate: passed by const-ref? static bool isStdRefType(const QString& t) { return t == "std::string" || t.startsWith("std::vector") + || t == "std::map" || t.startsWith("std::map") || t == "LogosMap" || t == "LogosList"; } @@ -203,10 +375,144 @@ static bool isQtRefType(const QString& t) || t == "QJsonArray" || t == "QVariantList" || t == "QVariantMap"; } -QString makeHeader(const QString& moduleName, const QString& className, const QJsonArray& methods, ApiStyle apiStyle, const QJsonArray& events, BindMode bindMode) +// ─── Record-aware type / conversion dispatch ───────────────────────────── +// +// The one entry point every emission site goes through. A record-bearing type +// takes the record path; everything else falls through to the pre-existing +// mapping tables unchanged, so an empty record set is a no-op. + +static QString paramTypeFor(const QString& qtType, ApiStyle style, const RecordSet& rs, + const QString& qual = QString()) +{ + const QString rec = recordCppType(rs, qtType, style, qual); + if (!rec.isEmpty()) return rec; + return (style == ApiStyle::Qt) ? mapParamType(qtType) : mapParamTypeStd(qtType); +} + +static QString returnTypeFor(const QString& qtType, ApiStyle style, const RecordSet& rs, + const QString& qual = QString()) +{ + const QString rec = recordCppType(rs, qtType, style, qual); + if (!rec.isEmpty()) return rec; + return (style == ApiStyle::Qt) ? mapReturnType(qtType) : mapReturnTypeStd(qtType); +} + +// Records are structs — always by const-ref, never copied into a call. +static bool byRefFor(const QString& qtType, const QString& cppType, ApiStyle style, const RecordSet& rs) +{ + if (recordShape(rs, qtType, nullptr) != RecordShape::None) return true; + return (style == ApiStyle::Qt) ? isQtRefType(cppType) : isStdRefType(cppType); +} + +// Typed value -> wire value (QVariant for Qt/Std, nlohmann::json for Lp). +static QString toWireFor(const QString& qtType, ApiStyle style, const RecordSet& rs, const QString& expr) +{ + const QString rec = recordToWireExpr(rs, qtType, style, expr); + if (!rec.isEmpty()) return rec; + if (style == ApiStyle::Lp) return lpPushExpr(qtType, expr); + if (style == ApiStyle::Std) return stdParamToQVariant(qtType, expr); + return expr; // Qt: the wrapper's own surface already IS the wire type +} + +// Wire value -> typed value. +static QString fromWireFor(const QString& qtType, ApiStyle style, const RecordSet& rs, + const QString& wire, const QString& qual = QString()) +{ + const QString rec = recordFromWireExpr(rs, qtType, style, wire, qual); + if (!rec.isEmpty()) return rec; + if (style == ApiStyle::Lp) return lpFromJsonExpr(qtType, wire); + if (style == ApiStyle::Std) return qVariantToStdReturn(qtType, wire); + return toQVariantConversion(mapParamType(qtType), wire); +} + +// The struct declarations, emitted inside the wrapper class. +static void emitRecordStructs(QTextStream& s, const RecordSet& rs, ApiStyle style) +{ + if (rs.isEmpty()) return; + s << " // Record types declared by the contract.\n"; + for (const RecordDef& d : rs) { + s << " struct " << d.name << " {\n"; + for (const RecordField& f : d.fields) + s << " " << paramTypeFor(f.type, style, rs) << " " << f.name << "{};\n"; + s << " };\n"; + } + s << "\n"; +} + +// The struct <-> wire conversions, emitted as file-local statics in the +// generated .cpp. Declared up front so records can reference each other (and +// themselves, through a list field) regardless of declaration order. +static void emitRecordConversions(QTextStream& s, const RecordSet& rs, ApiStyle style, + const QString& className) +{ + if (rs.isEmpty()) return; + const QString wire = (style == ApiStyle::Lp) ? "nlohmann::json" : "QVariant"; + const QString qual = className + "::"; + + for (const RecordDef& d : rs) { + s << "static " << wire << " " << recToWireFn(d.name) + << "(const " << qual << d.name << "& v);\n"; + s << "static " << qual << d.name << " " << recFromWireFn(d.name) + << "(const " << wire << "& w);\n"; + } + s << "\n"; + + for (const RecordDef& d : rs) { + // Encode. + s << "static " << wire << " " << recToWireFn(d.name) + << "(const " << qual << d.name << "& v) {\n"; + if (style == ApiStyle::Lp) { + s << " nlohmann::json __j = nlohmann::json::object();\n"; + for (const RecordField& f : d.fields) + s << " __j[\"" << f.name << "\"] = " + << toWireFor(f.type, style, rs, "v." + f.name) << ";\n"; + s << " return __j;\n"; + } else { + s << " QVariantMap __m;\n"; + for (const RecordField& f : d.fields) { + // Qt's surface type IS the wire type for non-record fields, so + // fromValue is what puts it in the map; records/containers + // already produce a QVariant-compatible value. + const QString v = toWireFor(f.type, style, rs, "v." + f.name); + const bool isRec = recordShape(rs, f.type, nullptr) != RecordShape::None; + s << " __m.insert(QStringLiteral(\"" << f.name << "\"), " + << (isRec || style == ApiStyle::Std ? v : "QVariant::fromValue(" + v + ")") + << ");\n"; + } + s << " return __m;\n"; + } + s << "}\n\n"; + + // Decode. A missing / mistyped field keeps its default rather than + // failing the whole call — same leniency the scalar paths use. + s << "static " << qual << d.name << " " << recFromWireFn(d.name) + << "(const " << wire << "& w) {\n"; + s << " " << qual << d.name << " __out;\n"; + if (style == ApiStyle::Lp) { + s << " if (!w.is_object()) return __out;\n"; + for (const RecordField& f : d.fields) { + const QString acc = "w.at(\"" + f.name + "\")"; + s << " if (w.contains(\"" << f.name << "\")) __out." << f.name << " = " + << fromWireFor(f.type, style, rs, acc, qual) << ";\n"; + } + } else { + s << " const QVariantMap __m = w.toMap();\n"; + for (const RecordField& f : d.fields) { + const QString acc = "__m.value(QStringLiteral(\"" + f.name + "\"))"; + s << " __out." << f.name << " = " + << fromWireFor(f.type, style, rs, acc, qual) << ";\n"; + } + } + s << " return __out;\n"; + s << "}\n\n"; + } +} + +QString makeHeader(const QString& moduleName, const QString& className, const QJsonArray& methods, ApiStyle apiStyle, const QJsonArray& events, BindMode bindMode, const QJsonArray& records) { if (apiStyle == ApiStyle::Lp) - return makeHeaderLp(moduleName, className, methods, events, bindMode); + return makeHeaderLp(moduleName, className, methods, events, bindMode, records); + const RecordSet rs = parseRecords(records); QString h; QTextStream s(&h); s << "#pragma once\n"; @@ -229,6 +535,8 @@ QString makeHeader(const QString& moduleName, const QString& className, const QJ // any events. Cheap to include unconditionally — keeps the // header symmetric with the Qt-style branch. if (!events.isEmpty()) s << "#include \"logos_object.h\"\n"; + // Record maps are std::map on the std surface. + if (!rs.isEmpty()) s << "#include \n"; s << "\n"; } else { s << "#include \n"; @@ -247,6 +555,7 @@ QString makeHeader(const QString& moduleName, const QString& className, const QJ } s << "class " << className << " {\n"; s << "public:\n"; + emitRecordStructs(s, rs, apiStyle); if (bindMode == BindMode::Bound) { // Interface wrapper: the module to talk to is chosen at runtime. s << " explicit " << className << "(LogosAPI* api, const QString& moduleName);\n\n"; @@ -297,9 +606,8 @@ QString makeHeader(const QString& moduleName, const QString& className, const QJ for (int i = 0; i < evParams.size(); ++i) { const QJsonObject p = evParams.at(i).toObject(); QString qtPt = p.value("type").toString(); - QString pt = (apiStyle == ApiStyle::Std) - ? mapParamTypeStd(qtPt) : mapParamType(qtPt); - bool byRef = (apiStyle == ApiStyle::Std) ? isStdRefType(pt) : isQtRefType(pt); + QString pt = paramTypeFor(qtPt, apiStyle, rs); + bool byRef = byRefFor(qtPt, pt, apiStyle, rs); if (byRef) cbParams += "const " + pt + "& "; else cbParams += pt + " "; cbParams += p.value("name").toString(); @@ -316,17 +624,15 @@ QString makeHeader(const QString& moduleName, const QString& className, const QJ if (!invokable) continue; const QString name = o.value("name").toString(); const QString qtRet = o.value("returnType").toString(); - const QString ret = (apiStyle == ApiStyle::Std) - ? mapReturnTypeStd(qtRet) : mapReturnType(qtRet); + const QString ret = returnTypeFor(qtRet, apiStyle, rs); s << " " << ret << " " << name << "("; QJsonArray params = o.value("parameters").toArray(); for (int i = 0; i < params.size(); ++i) { QJsonObject p = params.at(i).toObject(); QString qtPt = p.value("type").toString(); - QString pt = (apiStyle == ApiStyle::Std) - ? mapParamTypeStd(qtPt) : mapParamType(qtPt); + QString pt = paramTypeFor(qtPt, apiStyle, rs); QString pn = p.value("name").toString(); - bool byRef = (apiStyle == ApiStyle::Std) ? isStdRefType(pt) : isQtRefType(pt); + bool byRef = byRefFor(qtPt, pt, apiStyle, rs); if (byRef) s << "const " << pt << "& " << pn; else s << pt << " " << pn; if (i + 1 < params.size()) s << ", "; @@ -344,10 +650,9 @@ QString makeHeader(const QString& moduleName, const QString& className, const QJ for (int i = 0; i < params.size(); ++i) { QJsonObject p = params.at(i).toObject(); QString qtPt = p.value("type").toString(); - QString pt = (apiStyle == ApiStyle::Std) - ? mapParamTypeStd(qtPt) : mapParamType(qtPt); + QString pt = paramTypeFor(qtPt, apiStyle, rs); QString pn = p.value("name").toString(); - bool byRef = (apiStyle == ApiStyle::Std) ? isStdRefType(pt) : isQtRefType(pt); + bool byRef = byRefFor(qtPt, pt, apiStyle, rs); if (byRef) s << "const " << pt << "& " << pn; else s << pt << " " << pn; if (i + 1 < params.size()) s << ", "; @@ -388,10 +693,11 @@ QString makeHeader(const QString& moduleName, const QString& className, const QJ return h; } -QString makeSource(const QString& moduleName, const QString& className, const QString& headerBaseName, const QJsonArray& methods, ApiStyle apiStyle, const QJsonArray& events, BindMode bindMode) +QString makeSource(const QString& moduleName, const QString& className, const QString& headerBaseName, const QJsonArray& methods, ApiStyle apiStyle, const QJsonArray& events, BindMode bindMode, const QJsonArray& records) { if (apiStyle == ApiStyle::Lp) - return makeSourceLp(moduleName, className, headerBaseName, methods, events, bindMode); + return makeSourceLp(moduleName, className, headerBaseName, methods, events, bindMode, records); + const RecordSet rs = parseRecords(records); QString c; QTextStream s(&c); s << "#include \"" << headerBaseName << "\"\n\n"; @@ -412,7 +718,12 @@ QString makeSource(const QString& moduleName, const QString& className, const QS // std mode when events are present. if (!events.isEmpty()) s << "#include \"logos_object.h\"\n"; } + if (apiStyle == ApiStyle::Qt && !rs.isEmpty()) { + // Record conversions build QVariantMaps regardless of api style. + s << "#include \n"; + } s << "\n"; + emitRecordConversions(s, rs, apiStyle, className); // The expression every remote call uses to name its target module. // Static: the baked string literal "" (unchanged // behaviour). Bound: the m_moduleName member set from the runtime ctor @@ -524,9 +835,8 @@ QString makeSource(const QString& moduleName, const QString& className, const QS for (int i = 0; i < evParams.size(); ++i) { const QJsonObject p = evParams.at(i).toObject(); QString qtPt = p.value("type").toString(); - QString pt = (apiStyle == ApiStyle::Std) - ? mapParamTypeStd(qtPt) : mapParamType(qtPt); - bool byRef = (apiStyle == ApiStyle::Std) ? isStdRefType(pt) : isQtRefType(pt); + QString pt = paramTypeFor(qtPt, apiStyle, rs); + bool byRef = byRefFor(qtPt, pt, apiStyle, rs); if (byRef) cbParams += "const " + pt + "& "; else cbParams += pt + " "; cbParams += p.value("name").toString(); @@ -550,13 +860,7 @@ QString makeSource(const QString& moduleName, const QString& className, const QS QString qtPt = p.value("type").toString(); // Build the QVariant → typed-arg conversion expression. const QString argExpr = QString("_args.at(%1)").arg(i); - QString conv; - if (apiStyle == ApiStyle::Std) { - conv = qVariantToStdReturn(qtPt, argExpr); - } else { - conv = toQVariantConversion(mapParamType(qtPt), argExpr); - } - s << conv; + s << fromWireFor(qtPt, apiStyle, rs, argExpr); if (i + 1 < evParams.size()) s << ", "; } s << ");\n"; @@ -571,8 +875,11 @@ QString makeSource(const QString& moduleName, const QString& className, const QS if (!invokable) continue; const QString name = o.value("name").toString(); const QString qtRet = o.value("returnType").toString(); - const QString ret = (apiStyle == ApiStyle::Std) - ? mapReturnTypeStd(qtRet) : mapReturnType(qtRet); + // Inside the class's own scope (parameter lists, bodies) a nested + // record needs no qualification; a return type written before the + // `Class::` in a definition does. + const QString ret = returnTypeFor(qtRet, apiStyle, rs); + const QString retQual = returnTypeFor(qtRet, apiStyle, rs, className + "::"); QJsonArray params = o.value("parameters").toArray(); // Helper closures kept inline so the two branches don't get @@ -580,21 +887,20 @@ QString makeSource(const QString& moduleName, const QString& className, const QS // the only thing that varies between Qt and Std modes. auto emitParam = [&](const QJsonObject& p, bool& byRefOut) { QString qtPt = p.value("type").toString(); - QString pt = (apiStyle == ApiStyle::Std) - ? mapParamTypeStd(qtPt) : mapParamType(qtPt); + QString pt = paramTypeFor(qtPt, apiStyle, rs); QString pn = p.value("name").toString(); - byRefOut = (apiStyle == ApiStyle::Std) ? isStdRefType(pt) : isQtRefType(pt); + byRefOut = byRefFor(qtPt, pt, apiStyle, rs); if (byRefOut) s << "const " << pt << "& " << pn; else s << pt << " " << pn; }; auto wireArg = [&](const QJsonObject& p) -> QString { QString qtPt = p.value("type").toString(); QString pn = p.value("name").toString(); - return (apiStyle == ApiStyle::Std) ? stdParamToQVariant(qtPt, pn) : pn; + return toWireFor(qtPt, apiStyle, rs, pn); }; // Signature - s << ret << " " << className << "::" << name << "("; + s << retQual << " " << className << "::" << name << "("; for (int i = 0; i < params.size(); ++i) { bool byRef; emitParam(params.at(i).toObject(), byRef); @@ -630,12 +936,19 @@ QString makeSource(const QString& moduleName, const QString& className, const QS << ": remote call failed:\" << QString::fromStdString(_err.message);\n"; // Return conversion + const bool retIsRecord = recordShape(rs, qtRet, nullptr) != RecordShape::None; if (ret == "void") { // nothing + } else if (retIsRecord) { + s << " return " << fromWireFor(qtRet, apiStyle, rs, "_result") << ";\n"; } else if (apiStyle == ApiStyle::Std) { s << " return " << qVariantToStdReturn(qtRet, "_result") << ";\n"; } else if (ret == "bool") { s << " return _result.toBool();\n"; + } else if (ret == "qlonglong") { + s << " return _result.toLongLong();\n"; + } else if (ret == "qulonglong") { + s << " return _result.toULongLong();\n"; } else if (ret == "int") { s << " return _result.toInt();\n"; } else if (ret == "double") { @@ -690,6 +1003,10 @@ QString makeSource(const QString& moduleName, const QString& className, const QS s << ", [callback](QVariant v) {\n"; if (ret == "void") { s << " (void)v; callback();\n"; + } else if (retIsRecord) { + // A record decodes field by field; an invalid QVariant yields a + // default-constructed struct, matching the scalar paths. + s << " callback(" << fromWireFor(qtRet, apiStyle, rs, "v", className + "::") << ");\n"; } else if (apiStyle == ApiStyle::Std) { // Default-construct on dispatch failure, matching the // existing Qt code path which falls back to a zero / empty @@ -709,7 +1026,8 @@ QString makeSource(const QString& moduleName, const QString& className, const QS } else { QString defaultVal; if (ret == "bool") defaultVal = "false"; - else if (ret == "int" || ret == "double" || ret == "float") defaultVal = "0"; + else if (ret == "int" || ret == "qlonglong" || ret == "qulonglong" + || ret == "double" || ret == "float") defaultVal = "0"; else if (ret == "QString") defaultVal = "QString()"; else if (ret == "QStringList") defaultVal = "QStringList()"; else if (ret == "QJsonArray") defaultVal = "QJsonArray()"; @@ -871,6 +1189,7 @@ static QString lpFromJsonExpr(const QString& qtType, const QString& jv) if (t == "void") return QString(); if (t == "std::string") return "(" + jv + ".is_string() ? " + jv + ".get() : std::string())"; if (t == "int64_t") return "(" + jv + ".is_number_integer() ? " + jv + ".get() : (" + jv + ".is_number() ? static_cast(" + jv + ".get()) : (int64_t)0))"; + if (t == "uint64_t") return "(" + jv + ".is_number_integer() ? " + jv + ".get() : (" + jv + ".is_number() ? static_cast(" + jv + ".get()) : (uint64_t)0))"; if (t == "double") return "(" + jv + ".is_number() ? " + jv + ".get() : 0.0)"; if (t == "bool") return "(" + jv + ".is_boolean() ? " + jv + ".get() : false)"; if (t == "std::vector") return "logos::jsonToStringVec(" + jv + ")"; @@ -889,14 +1208,15 @@ static QString lpFromJsonExpr(const QString& qtType, const QString& jv) // Build the callback parameter list (std types, by-ref where appropriate) for // a typed event accessor `on`. -static QString lpEventCbParams(const QJsonArray& evParams) +static QString lpEventCbParams(const QJsonArray& evParams, const RecordSet& rs) { QString cbParams; for (int i = 0; i < evParams.size(); ++i) { const QJsonObject p = evParams.at(i).toObject(); - const QString pt = mapParamTypeStd(p.value("type").toString()); - if (isStdRefType(pt)) cbParams += "const " + pt + "& "; - else cbParams += pt + " "; + const QString qtPt = p.value("type").toString(); + const QString pt = paramTypeFor(qtPt, ApiStyle::Lp, rs); + if (byRefFor(qtPt, pt, ApiStyle::Lp, rs)) cbParams += "const " + pt + "& "; + else cbParams += pt + " "; cbParams += p.value("name").toString(); if (i + 1 < evParams.size()) cbParams += ", "; } @@ -910,9 +1230,10 @@ static QString lpEventAccessorName(const QString& evName) return QString("on") + cap; } -QString makeHeaderLp(const QString& moduleName, const QString& className, const QJsonArray& methods, const QJsonArray& events, BindMode bindMode) +QString makeHeaderLp(const QString& moduleName, const QString& className, const QJsonArray& methods, const QJsonArray& events, BindMode bindMode, const QJsonArray& records) { (void)moduleName; + const RecordSet rs = parseRecords(records); QString h; QTextStream s(&h); s << "#pragma once\n"; @@ -924,10 +1245,14 @@ QString makeHeaderLp(const QString& moduleName, const QString& className, const s << "#include \"logos_json.h\"\n"; s << "#include \"logos_result.h\"\n"; s << "#include \"logos_call_error.h\"\n"; - s << "#include \"logos_lp_client.h\"\n\n"; + s << "#include \"logos_lp_client.h\"\n"; + // Record maps are std::map on the Qt-free surface. + if (!rs.isEmpty()) s << "#include \n"; + s << "\n"; s << "class " << className << " {\n"; s << "public:\n"; + emitRecordStructs(s, rs, ApiStyle::Lp); if (bindMode == BindMode::Bound) { // Bound (interface) wrappers are THIN, copyable handles over // umbrella-owned persistent State, so a transient @@ -953,7 +1278,7 @@ QString makeHeaderLp(const QString& moduleName, const QString& className, const const QString evName = eo.value("name").toString(); if (evName.isEmpty()) continue; s << " bool " << lpEventAccessorName(evName) - << "(std::function callback);\n"; + << "(std::function callback);\n"; } if (!events.isEmpty()) s << "\n"; @@ -962,15 +1287,16 @@ QString makeHeaderLp(const QString& moduleName, const QString& className, const const QJsonObject o = v.toObject(); if (!o.value("isInvokable").toBool()) continue; const QString name = o.value("name").toString(); - const QString ret = mapReturnTypeStd(o.value("returnType").toString()); + const QString ret = returnTypeFor(o.value("returnType").toString(), ApiStyle::Lp, rs); const QJsonArray params = o.value("parameters").toArray(); s << " " << ret << " " << name << "("; for (int i = 0; i < params.size(); ++i) { const QJsonObject p = params.at(i).toObject(); - const QString pt = mapParamTypeStd(p.value("type").toString()); - if (isStdRefType(pt)) s << "const " << pt << "& " << p.value("name").toString(); - else s << pt << " " << p.value("name").toString(); + const QString qtPt = p.value("type").toString(); + const QString pt = paramTypeFor(qtPt, ApiStyle::Lp, rs); + if (byRefFor(qtPt, pt, ApiStyle::Lp, rs)) s << "const " << pt << "& " << p.value("name").toString(); + else s << pt << " " << p.value("name").toString(); if (i + 1 < params.size()) s << ", "; } if (!params.isEmpty()) s << ", "; @@ -982,9 +1308,10 @@ QString makeHeaderLp(const QString& moduleName, const QString& className, const s << " void " << name << "Async("; for (int i = 0; i < params.size(); ++i) { const QJsonObject p = params.at(i).toObject(); - const QString pt = mapParamTypeStd(p.value("type").toString()); - if (isStdRefType(pt)) s << "const " << pt << "& " << p.value("name").toString(); - else s << pt << " " << p.value("name").toString(); + const QString qtPt = p.value("type").toString(); + const QString pt = paramTypeFor(qtPt, ApiStyle::Lp, rs); + if (byRefFor(qtPt, pt, ApiStyle::Lp, rs)) s << "const " << pt << "& " << p.value("name").toString(); + else s << pt << " " << p.value("name").toString(); if (i + 1 < params.size()) s << ", "; } if (!params.isEmpty()) s << ", "; @@ -1002,12 +1329,14 @@ QString makeHeaderLp(const QString& moduleName, const QString& className, const return h; } -QString makeSourceLp(const QString& moduleName, const QString& className, const QString& headerBaseName, const QJsonArray& methods, const QJsonArray& events, BindMode bindMode) +QString makeSourceLp(const QString& moduleName, const QString& className, const QString& headerBaseName, const QJsonArray& methods, const QJsonArray& events, BindMode bindMode, const QJsonArray& records) { + const RecordSet rs = parseRecords(records); QString c; QTextStream s(&c); s << "#include \"" << headerBaseName << "\"\n"; s << "#include \n\n"; + emitRecordConversions(s, rs, ApiStyle::Lp, className); // How the wrapper reaches its persistent LpClient + subscription store. // Static (concrete dep): owns them by value — the wrapper itself is a @@ -1031,14 +1360,15 @@ QString makeSourceLp(const QString& moduleName, const QString& className, const if (evName.isEmpty()) continue; const QJsonArray evParams = eo.value("params").toArray(); s << "bool " << className << "::" << lpEventAccessorName(evName) - << "(std::function callback) {\n"; + << "(std::function callback) {\n"; s << " if (!callback) return false;\n"; s << " auto _sub = " << clientExpr << ".subscribe(\"" << evName << "\", [callback](nlohmann::json _a) {\n"; s << " if (!_a.is_array() || _a.size() < " << evParams.size() << ") return;\n"; s << " callback("; for (int i = 0; i < evParams.size(); ++i) { const QJsonObject p = evParams.at(i).toObject(); - s << lpFromJsonExpr(p.value("type").toString(), QString("_a.at(%1)").arg(i)); + s << fromWireFor(p.value("type").toString(), ApiStyle::Lp, rs, + QString("_a.at(%1)").arg(i), className + "::"); if (i + 1 < evParams.size()) s << ", "; } s << ");\n"; @@ -1055,15 +1385,17 @@ QString makeSourceLp(const QString& moduleName, const QString& className, const if (!o.value("isInvokable").toBool()) continue; const QString name = o.value("name").toString(); const QString qtRet = o.value("returnType").toString(); - const QString ret = mapReturnTypeStd(qtRet); + const QString ret = returnTypeFor(qtRet, ApiStyle::Lp, rs); + const QString retQual = returnTypeFor(qtRet, ApiStyle::Lp, rs, className + "::"); const QJsonArray params = o.value("parameters").toArray(); auto emitParams = [&]() { for (int i = 0; i < params.size(); ++i) { const QJsonObject p = params.at(i).toObject(); - const QString pt = mapParamTypeStd(p.value("type").toString()); - if (isStdRefType(pt)) s << "const " << pt << "& " << p.value("name").toString(); - else s << pt << " " << p.value("name").toString(); + const QString qtPt = p.value("type").toString(); + const QString pt = paramTypeFor(qtPt, ApiStyle::Lp, rs); + if (byRefFor(qtPt, pt, ApiStyle::Lp, rs)) s << "const " << pt << "& " << p.value("name").toString(); + else s << pt << " " << p.value("name").toString(); if (i + 1 < params.size()) s << ", "; } }; @@ -1071,12 +1403,12 @@ QString makeSourceLp(const QString& moduleName, const QString& className, const s << " nlohmann::json _args = nlohmann::json::array();\n"; for (const QJsonValue& pv : params) { const QJsonObject p = pv.toObject(); - s << " _args.push_back(" << lpPushExpr(p.value("type").toString(), p.value("name").toString()) << ");\n"; + s << " _args.push_back(" << toWireFor(p.value("type").toString(), ApiStyle::Lp, rs, p.value("name").toString()) << ");\n"; } }; // Sync - s << ret << " " << className << "::" << name << "("; + s << retQual << " " << className << "::" << name << "("; emitParams(); if (!params.isEmpty()) s << ", "; s << "logos::CallError* err) {\n"; @@ -1085,7 +1417,7 @@ QString makeSourceLp(const QString& moduleName, const QString& className, const s << " " << clientExpr << ".invoke(\"" << name << "\", _args, err);\n"; } else { s << " nlohmann::json _r = " << clientExpr << ".invoke(\"" << name << "\", _args, err);\n"; - s << " return " << lpFromJsonExpr(qtRet, "_r") << ";\n"; + s << " return " << fromWireFor(qtRet, ApiStyle::Lp, rs, "_r", className + "::") << ";\n"; } s << "}\n\n"; @@ -1103,7 +1435,7 @@ QString makeSourceLp(const QString& moduleName, const QString& className, const if (ret == "void") { s << " (void)_r; callback();\n"; } else { - s << " callback(" << lpFromJsonExpr(qtRet, "_r") << ");\n"; + s << " callback(" << fromWireFor(qtRet, ApiStyle::Lp, rs, "_r", className + "::") << ");\n"; } s << " });\n"; s << "}\n\n"; diff --git a/cpp-generator/legacy/generator_lib.h b/cpp-generator/legacy/generator_lib.h index 6feb656..50166cd 100644 --- a/cpp-generator/legacy/generator_lib.h +++ b/cpp-generator/legacy/generator_lib.h @@ -72,15 +72,25 @@ QString toQVariantConversion(const QString& type, const QString& argExpr); // runtime-bound interface wrapper (Bound) — see BindMode above. In Bound // mode `moduleName` is used only for the class/file naming the caller // already decided; the emitted code never bakes it into a call. -QString makeHeader(const QString& moduleName, const QString& className, const QJsonArray& methods, ApiStyle apiStyle = ApiStyle::Qt, const QJsonArray& events = {}, BindMode bindMode = BindMode::Static); -QString makeSource(const QString& moduleName, const QString& className, const QString& headerBaseName, const QJsonArray& methods, ApiStyle apiStyle = ApiStyle::Qt, const QJsonArray& events = {}, BindMode bindMode = BindMode::Static); +// +// `records` carries the contract's `type Foo { ... }` declarations, as +// [ { "name": "Foo", "fields": [ { "name": "...", "type": "" } ] } ] +// Each becomes a struct NESTED in the wrapper class (`::Foo`, so two +// deps may both declare a `Status`), and every method / event that mentions +// one is typed with it instead of falling back to QVariant / LogosMap. Field +// types use the same Qt type-name spelling as methods, so a field can name +// another record, `QList`, or `QMap`. Empty (the +// default, and what the metaobject-introspection path passes) leaves the +// generated output exactly as it was. +QString makeHeader(const QString& moduleName, const QString& className, const QJsonArray& methods, ApiStyle apiStyle = ApiStyle::Qt, const QJsonArray& events = {}, BindMode bindMode = BindMode::Static, const QJsonArray& records = {}); +QString makeSource(const QString& moduleName, const QString& className, const QString& headerBaseName, const QJsonArray& methods, ApiStyle apiStyle = ApiStyle::Qt, const QJsonArray& events = {}, BindMode bindMode = BindMode::Static, const QJsonArray& records = {}); // Qt-free (ApiStyle::Lp) wrapper emission. Same std-typed surface as the Std // flavor, but the generated body calls the logos-protocol C ABI through // logos::LpClient instead of LogosAPIClient — no Qt in the wrapper's TU. // makeHeader/makeSource dispatch here when apiStyle == ApiStyle::Lp. -QString makeHeaderLp(const QString& moduleName, const QString& className, const QJsonArray& methods, const QJsonArray& events = {}, BindMode bindMode = BindMode::Static); -QString makeSourceLp(const QString& moduleName, const QString& className, const QString& headerBaseName, const QJsonArray& methods, const QJsonArray& events = {}, BindMode bindMode = BindMode::Static); +QString makeHeaderLp(const QString& moduleName, const QString& className, const QJsonArray& methods, const QJsonArray& events = {}, BindMode bindMode = BindMode::Static, const QJsonArray& records = {}); +QString makeSourceLp(const QString& moduleName, const QString& className, const QString& headerBaseName, const QJsonArray& methods, const QJsonArray& events = {}, BindMode bindMode = BindMode::Static, const QJsonArray& records = {}); QVector parseProviderHeader(const QString& headerPath, QTextStream& err); #endif // GENERATOR_LIB_H diff --git a/cpp-generator/legacy/main.cpp b/cpp-generator/legacy/main.cpp index 3f84783..0784597 100644 --- a/cpp-generator/legacy/main.cpp +++ b/cpp-generator/legacy/main.cpp @@ -37,8 +37,12 @@ static QString lidlTypeExprToQtTypeName(const TypeExpr& te) if (te.kind == TypeExpr::Primitive) { if (te.name == "tstr") return "QString"; if (te.name == "bstr") return "QByteArray"; - if (te.name == "int") return "int"; - if (te.name == "uint") return "int"; // wire-as-int for now + // 64-bit, and unsigned stays unsigned. LIDL int/uint are int64_t / + // uint64_t in every other binding; spelling them `int` here handed a + // Qt-style consumer a signed 32-bit value (silent truncation above + // 2^31) and a std/lp consumer a SIGNED int64_t for a `uint`. + if (te.name == "int") return "qlonglong"; + if (te.name == "uint") return "qulonglong"; if (te.name == "float64") return "double"; if (te.name == "bool") return "bool"; if (te.name == "result") return "LogosResult"; @@ -49,20 +53,33 @@ static QString lidlTypeExprToQtTypeName(const TypeExpr& te) const TypeExpr& elem = te.elements[0]; if (elem.kind == TypeExpr::Primitive && elem.name == "tstr") return "QStringList"; + // A list of records keeps its element type — the wrapper emits a + // typed container, not a bag of QVariants. + if (elem.kind == TypeExpr::Named) + return "QList<" + qs(elem.name) + ">"; return "QVariantList"; } - if (te.kind == TypeExpr::Map) return "QVariantMap"; + if (te.kind == TypeExpr::Map) { + if (te.elements.size() == 2 && te.elements[1].kind == TypeExpr::Named) + return "QMap"; + return "QVariantMap"; + } if (te.kind == TypeExpr::Optional) return "QVariant"; - if (te.kind == TypeExpr::Named) return "QVariant"; + // A record declared by the contract: generator_lib emits the struct and + // types every mention of it with the struct. + if (te.kind == TypeExpr::Named) return qs(te.name); return "QVariant"; } +static QJsonArray moduleRecordsToJson(const ModuleDecl& mod); + // Load events from a `.lidl` sidecar shipped alongside a module's // pre-built headers. Returns a JSON array of // { name, params: [ { name, type } ] } // using Qt-typed type names — same shape generator_lib's makeHeader / // makeSource already consume for methods. -static QJsonArray loadEventsFromLidl(const QString& lidlPath, QTextStream& err) +static QJsonArray loadEventsFromLidl(const QString& lidlPath, QTextStream& err, + QJsonArray* outRecords = nullptr) { QJsonArray result; QFile f(lidlPath); @@ -80,6 +97,7 @@ static QJsonArray loadEventsFromLidl(const QString& lidlPath, QTextStream& err) return result; } + if (outRecords) *outRecords = moduleRecordsToJson(pr.module); for (const EventDecl& ed : pr.module.events) { QJsonObject obj; obj["name"] = qs(ed.name); @@ -171,6 +189,28 @@ static QJsonArray moduleMethodsToJson(const ModuleDecl& mod) return arr; } +// Build the records QJsonArray ({ name, fields:[{name,type}] }) from a parsed +// ModuleDecl — the contract's `type Foo { ... }` declarations, which +// generator_lib turns into structs nested in the wrapper class. +static QJsonArray moduleRecordsToJson(const ModuleDecl& mod) +{ + QJsonArray arr; + for (const TypeDecl& td : mod.types) { + QJsonObject o; + o["name"] = qs(td.name); + QJsonArray fields; + for (const FieldDecl& fd : td.fields) { + QJsonObject f; + f["name"] = qs(fd.name); + f["type"] = lidlTypeExprToQtTypeName(fd.type); + fields.append(f); + } + o["fields"] = fields; + arr.append(o); + } + return arr; +} + // Build the events QJsonArray ({ name, params:[{name,type}] }) — same shape // loadEventsFromLidl produces — from a parsed ModuleDecl. static QJsonArray moduleEventsToJson(const ModuleDecl& mod) @@ -273,14 +313,23 @@ static bool generateInterfaceWrappers(const QVector& ifaces, ModuleDecl mod; if (!parseInterfaceFile(spec, genDirPath, mod, err)) return false; + { + QString recErr; + if (!lidlCheckRecords(mod, &recErr)) { + err << spec.path << ": " << recErr << "\n"; + return false; + } + } + const QString className = toPascalCase(spec.name); const QJsonArray methods = moduleMethodsToJson(mod); const QJsonArray events = moduleEventsToJson(mod); + const QJsonArray records = moduleRecordsToJson(mod); const QString headerRel = spec.name + "_api.h"; const QString sourceRel = spec.name + "_api.cpp"; - const QString header = makeHeader(spec.name, className, methods, apiStyle, events, bindMode); - const QString source = makeSource(spec.name, className, headerRel, methods, apiStyle, events, bindMode); + const QString header = makeHeader(spec.name, className, methods, apiStyle, events, bindMode, records); + const QString source = makeSource(spec.name, className, headerRel, methods, apiStyle, events, bindMode, records); { QFile f(QDir(genDirPath).filePath(headerRel)); @@ -773,7 +822,7 @@ static int generateProviderDispatch(const QString& headerPath, const QString& ou return 0; } -static int generateFromPlugin(const QString& pluginInputPath, const QString& outputDir, bool moduleOnly, ApiStyle apiStyle, const QJsonArray& events, QTextStream& out, QTextStream& err) +static int generateFromPlugin(const QString& pluginInputPath, const QString& outputDir, bool moduleOnly, ApiStyle apiStyle, const QJsonArray& events, QTextStream& out, QTextStream& err, const QJsonArray& records = {}) { QFileInfo fi(pluginInputPath); if (!fi.exists()) { @@ -841,8 +890,8 @@ static int generateFromPlugin(const QString& pluginInputPath, const QString& out // doesn't need to know which style was picked. `events` (loaded // from a sibling `.lidl` sidecar via --events-from) adds typed // `on(callback)` accessors next to the existing methods. - QString header = makeHeader(moduleName, className, methods, apiStyle, events); - QString source = makeSource(moduleName, className, headerRel, methods, apiStyle, events); + QString header = makeHeader(moduleName, className, methods, apiStyle, events, BindMode::Static, records); + QString source = makeSource(moduleName, className, headerRel, methods, apiStyle, events, BindMode::Static, records); { QFile f(headerAbs); @@ -1195,6 +1244,7 @@ int legacy_main(int argc, char* argv[]) // `on(callback)` accessors next to the existing // generic `onEvent(name, callback)` channel. QJsonArray eventsFromSidecar; + QJsonArray recordsFromSidecar; { const int evIdx = args.indexOf("--events-from"); QString evPath; @@ -1209,10 +1259,10 @@ int legacy_main(int argc, char* argv[]) } } if (!evPath.isEmpty() && QFileInfo(evPath).exists()) { - eventsFromSidecar = loadEventsFromLidl(evPath, err); + eventsFromSidecar = loadEventsFromLidl(evPath, err, &recordsFromSidecar); } } QString argPath = args.at(1); - return generateFromPlugin(argPath, outputDir, moduleOnly, apiStyle, eventsFromSidecar, out, err); + return generateFromPlugin(argPath, outputDir, moduleOnly, apiStyle, eventsFromSidecar, out, err, recordsFromSidecar); } diff --git a/cpp-generator/main.cpp b/cpp-generator/main.cpp index df6e3d7..4acbb38 100644 --- a/cpp-generator/main.cpp +++ b/cpp-generator/main.cpp @@ -172,6 +172,10 @@ int main(int argc, char* argv[]) } struct Out { QString file; QString content; }; QList outs; + // Always emitted, records or not: the exports TU and the events + // sidecar both reference the generated codec, and a module with + // no `type` decls still has containers to encode. + outs.append({qs(mod.name) + "_types.h", lidlMakeTypesHeaderCdylib(mod)}); outs.append({qs(mod.name) + "_module_impl.cpp", lidlMakeModuleImplExports(mod, implClass, implHeader)}); if (!mod.events.empty()) @@ -274,6 +278,7 @@ int main(int argc, char* argv[]) implHeader = args.at(implHeaderIdx + 1); else implHeader = qs(mod.name) + "_impl.h"; + outs.append({qs(mod.name) + "_types.h", lidlMakeTypesHeaderCdylib(mod)}); outs.append({qs(mod.name) + "_module_impl.cpp", lidlMakeModuleImplExports(mod, implClass, implHeader)}); if (!mod.events.empty()) diff --git a/doctests/cpp-sdk-generator-roundtrip.test.yaml b/doctests/cpp-sdk-generator-roundtrip.test.yaml index b954ed0..9b3168e 100644 --- a/doctests/cpp-sdk-generator-roundtrip.test.yaml +++ b/doctests/cpp-sdk-generator-roundtrip.test.yaml @@ -271,9 +271,14 @@ sections: wrapper a *consumer* compiles against to call `sensor_module`. Each LIDL `method` becomes a synchronous caller plus an `…Async` variant, and each `event` an `on(...)` subscription. The type mapping is the Qt caller style: - `float64`→`double`, `tstr`→`QString`, `int`/`uint`→`int`, `bstr`→ - `QByteArray`, `[tstr]`→`QStringList`, other arrays→`QVariantList`, and - `result`→`LogosResult`. + `float64`→`double`, `tstr`→`QString`, `int`→`qlonglong`, `uint`→ + `qulonglong`, `bstr`→`QByteArray`, `[tstr]`→`QStringList`, other + arrays→`QVariantList`, and `result`→`LogosResult`. + + The integer spellings are 64-bit, and unsigned stays unsigned: LIDL + `int`/`uint` are `int64_t`/`uint64_t` in every other binding, so spelling + them `int` here truncated above 2^31 and read a `uint` back as *signed*. + `record` below shows it — its `id` is a `uint64_t` in the impl header. steps: - title: "Generate the consumer header" run: "./result/bin/logos-cpp-generator --lidl extracted/sensor_module.lidl --output-dir consumer --module-only" @@ -285,13 +290,13 @@ sections: - "sensor_module_api.h" check_file: "consumer/sensor_module_api.h" - title: "Inspect the consumer API" - run: "grep -E 'class SensorModule|double temperature|int record|QByteArray firmware|QStringList labels|LogosResult reset|bool on\\(' consumer/sensor_module_api.h" + run: "grep -E 'class SensorModule|double temperature|qlonglong record|QByteArray firmware|QStringList labels|LogosResult reset|bool on\\(' consumer/sensor_module_api.h" code_block: | grep -E 'class|temperature|record|firmware|labels|reset|on\(' consumer/sensor_module_api.h expect_contains: - "class SensorModule" - "double temperature(" - - "int record(int id, double value, const QString& note, bool valid" + - "qlonglong record(qulonglong id, double value, const QString& note, bool valid" - "QByteArray firmware(const QByteArray& image" - "QStringList labels(const QVariantList& ids" - "LogosResult reset(const QString& id" @@ -301,12 +306,16 @@ sections: text: | Beyond the round-trippable core above, LIDL also has **composite** types: named record types (`type`), maps (`{K: V}`), and optionals (`?T`), plus the - untyped escape hatch `any`. These cross the wire as untyped JSON, so the - generated wrappers carry them as `QVariant` / `QVariantMap` / `QVariantList` - (record/map/optional shapes live in the authored contract; a - `--header-to-lidl` extraction recovers the std-friendly subset shown - earlier). Here is a contract that uses all of them, taken straight to a - consumer header. + untyped escape hatch `any`. + + A **record becomes a real C++ struct**: a `type Point { … }` in the contract + generates `struct Point` plus the conversions, so a caller writes + `Point p = client.translate(q, 1, 2)` instead of digging fields out of a + `QVariantMap`. `[Point]` is a `QList` and `{tstr: Point}` a + `QMap`. Maps of `any`, optionals (`?T`) and `any` itself + still cross as untyped JSON and stay `QVariantMap` / `QVariant` — a record + has a declared shape, those do not. Here is a contract that uses all of + them, taken straight to a consumer header. steps: - title: "geometry_module.lidl" file: @@ -341,14 +350,18 @@ sections: check_file: "geometry/geometry_module_api.h" - title: "Inspect the composite signatures" text: | - Records and optionals surface as `QVariant`, an array-of-records as - `QVariantList`, and a map as `QVariantMap` — the untyped carriers for the - JSON that crosses the process boundary. - run: "grep -E 'class GeometryModule|QVariant translate|QVariantList|QVariantMap attributes|QVariant nearest' geometry/geometry_module_api.h" + `Point` is generated as a struct, so a record parameter is taken by + const-ref and a record return comes back typed. An array-of-records is a + `QList`. A map of `any`, an optional and a bare `any` stay + `QVariantMap` / `QVariant` — the untyped carriers for JSON whose shape + the contract does not declare. + run: "grep -E 'class GeometryModule|struct Point|Point translate|QList|QVariantMap attributes|QVariant nearest' geometry/geometry_module_api.h" code_block: | grep -E 'class|translate|bounds|attributes|nearest' geometry/geometry_module_api.h expect_contains: - "class GeometryModule" - - "QVariant translate(QVariant p, double dx, double dy" + - "struct Point" + - "Point translate(const Point& p, double dx, double dy" + - "Point bounds(const QList& points" - "QVariantMap attributes(const QVariantMap& tags" - - "QVariant nearest(QVariant p, QVariant limit" + - "QVariant nearest(const Point& p, QVariant limit" diff --git a/doctests/outputs/cpp-sdk-generator-roundtrip.md b/doctests/outputs/cpp-sdk-generator-roundtrip.md index fbe5851..8b1bf7b 100644 --- a/doctests/outputs/cpp-sdk-generator-roundtrip.md +++ b/doctests/outputs/cpp-sdk-generator-roundtrip.md @@ -163,6 +163,11 @@ logos_events: /// Fires when a channel faults. /// Carries an error code, a message, and whether the fault is fatal. void fault(int64_t code, const std::string& message, bool fatal); + + /// Fires with a raw capture buffer — an event carrying a byte + /// string, which the generator must encode rather than pass + /// through as text. + void capture(uint64_t id, const std::vector& frame); }; ``` @@ -230,6 +235,22 @@ host's emit callback. grep -E 'extern \"C\"|logos_module_|SensorModuleImpl' provider/sensor_module_module_impl.cpp ``` +### 4.3 How each event marshals its payload + +The events file defines the body of every `logos_events:` declaration: +each argument is pushed into a JSON array and handed to the host's emit +callback. Scalars and strings go in as they are — but a byte string +cannot, because JSON has no binary type and a NUL would truncate it. +`frame` is therefore encoded into the canonical tagged form +`{"_bytes": ""}`, the same representation the transport and +the consumer both understand. A generator that pushed the raw value — +or dropped it — would leave every subscriber with an empty payload +([#99](https://github.com/logos-co/logos-cpp-sdk/issues/99)). + +```bash +cat provider/sensor_module_events_cdylib.cpp +``` + --- ## Step 5: Flow 3 — LIDL → consumer header @@ -238,9 +259,14 @@ The consumer side. From the same contract, `--module-only` emits the typed wrapper a *consumer* compiles against to call `sensor_module`. Each LIDL `method` becomes a synchronous caller plus an `…Async` variant, and each `event` an `on(...)` subscription. The type mapping is the Qt caller style: -`float64`→`double`, `tstr`→`QString`, `int`/`uint`→`int`, `bstr`→ -`QByteArray`, `[tstr]`→`QStringList`, other arrays→`QVariantList`, and -`result`→`LogosResult`. +`float64`→`double`, `tstr`→`QString`, `int`→`qlonglong`, `uint`→ +`qulonglong`, `bstr`→`QByteArray`, `[tstr]`→`QStringList`, other +arrays→`QVariantList`, and `result`→`LogosResult`. + +The integer spellings are 64-bit, and unsigned stays unsigned: LIDL +`int`/`uint` are `int64_t`/`uint64_t` in every other binding, so spelling +them `int` here truncated above 2^31 and read a `uint` back as *signed*. +`record` below shows it — its `id` is a `uint64_t` in the impl header. ### 5.1 Generate the consumer header @@ -261,12 +287,16 @@ grep -E 'class|temperature|record|firmware|labels|reset|on\(' consumer/sensor_mo Beyond the round-trippable core above, LIDL also has **composite** types: named record types (`type`), maps (`{K: V}`), and optionals (`?T`), plus the -untyped escape hatch `any`. These cross the wire as untyped JSON, so the -generated wrappers carry them as `QVariant` / `QVariantMap` / `QVariantList` -(record/map/optional shapes live in the authored contract; a -`--header-to-lidl` extraction recovers the std-friendly subset shown -earlier). Here is a contract that uses all of them, taken straight to a -consumer header. +untyped escape hatch `any`. + +A **record becomes a real C++ struct**: a `type Point { … }` in the contract +generates `struct Point` plus the conversions, so a caller writes +`Point p = client.translate(q, 1, 2)` instead of digging fields out of a +`QVariantMap`. `[Point]` is a `QList` and `{tstr: Point}` a +`QMap`. Maps of `any`, optionals (`?T`) and `any` itself +still cross as untyped JSON and stay `QVariantMap` / `QVariant` — a record +has a declared shape, those do not. Here is a contract that uses all of +them, taken straight to a consumer header. ### 6.1 geometry_module.lidl @@ -299,9 +329,11 @@ logos-cpp-generator --lidl geometry_module.lidl \ ### 6.3 Inspect the composite signatures -Records and optionals surface as `QVariant`, an array-of-records as -`QVariantList`, and a map as `QVariantMap` — the untyped carriers for the -JSON that crosses the process boundary. +`Point` is generated as a struct, so a record parameter is taken by +const-ref and a record return comes back typed. An array-of-records is a +`QList`. A map of `any`, an optional and a bare `any` stay +`QVariantMap` / `QVariant` — the untyped carriers for JSON whose shape +the contract does not declare. ```bash grep -E 'class|translate|bounds|attributes|nearest' geometry/geometry_module_api.h diff --git a/tests/experimental/fixtures/records_impl.h b/tests/experimental/fixtures/records_impl.h new file mode 100644 index 0000000..e7c23a1 --- /dev/null +++ b/tests/experimental/fixtures/records_impl.h @@ -0,0 +1,43 @@ +#pragma once +// Fixture: records DERIVED from an impl header. +// +// Blob and Wrapper are part of the API (they appear in signatures). Internal +// and Helper are not — they are the kind of private helper struct a real +// module carries (openmetrics has `struct ModuleSource`, the package manager +// has `struct PendingAction` inside the class), and publishing them as +// contract types would change a module's interface as a side effect of an +// internal refactor. +#include +#include +#include +#include +#include + +struct Blob { + std::string id; + uint64_t n; + std::vector payload; // trailing comment: must NOT drop the field +}; + +struct Wrapper { + Blob inner; + std::vector blobs; +}; + +// Never named by any signature. +struct Internal { + std::string secret; +}; + +class RecordsImpl : public LogosModuleContext { +public: + Blob echoBlob(const Blob& v); + Wrapper echoWrapper(const Wrapper& v); + std::map echoIntMap(const std::map& v); + +private: + // A private helper INSIDE the class — also never published. + struct Helper { + int64_t count; + }; +}; diff --git a/tests/experimental/fixtures/records_metadata.json b/tests/experimental/fixtures/records_metadata.json new file mode 100644 index 0000000..36f5ca8 --- /dev/null +++ b/tests/experimental/fixtures/records_metadata.json @@ -0,0 +1,10 @@ +{ + "name": "sample_module", + "version": "1.2.3", + "description": "A sample module for testing", + "author": "Test", + "type": "core", + "category": "testing", + "main": "sample_module_plugin", + "dependencies": ["dep_a", "dep_b"] +} diff --git a/tests/experimental/test_impl_header_parser.cpp b/tests/experimental/test_impl_header_parser.cpp index 88ee404..bdfca02 100644 --- a/tests/experimental/test_impl_header_parser.cpp +++ b/tests/experimental/test_impl_header_parser.cpp @@ -1,5 +1,6 @@ #include #include +#include #include "impl_header_parser.h" #include #include @@ -206,6 +207,64 @@ TEST_F(ImplHeaderParserTest, ComplexAccessSpecifiers) // Error cases // --------------------------------------------------------------------------- +// A struct in an impl header becomes a contract `type` — but ONLY if the API +// mentions it. A header routinely declares private helpers, and publishing +// those would change the module's interface as a side effect of an internal +// refactor. Verified against two real modules: openmetrics' `ModuleSource` and +// the package manager's in-class `PendingAction` were both being published. +TEST_F(ImplHeaderParserTest, OnlyApiReferencedStructsBecomeRecords) +{ + auto r = parseImplHeader( + fixturesDir() + "/records_impl.h", + "RecordsImpl", + fixturesDir() + "/records_metadata.json", + err); + ASSERT_FALSE(r.hasError()) << r.error.toStdString(); + + std::vector names; + for (const auto& t : r.module.types) names.push_back(t.name); + std::sort(names.begin(), names.end()); + + // Blob is named directly; Wrapper too. Internal and Helper are not. + ASSERT_EQ(names.size(), 2u) << "published: " << [&]{ + std::string j; for (const auto& n : names) j += n + " "; return j; }(); + EXPECT_EQ(names[0], "Blob"); + EXPECT_EQ(names[1], "Wrapper"); + + // A field with a trailing comment must NOT be silently dropped: a record + // published with a partial field list looks like a contract and is not one. + for (const auto& t : r.module.types) { + if (t.name != "Blob") continue; + ASSERT_EQ(t.fields.size(), 3u); + EXPECT_EQ(t.fields[2].name, "payload"); + EXPECT_EQ(t.fields[2].type.name, "bstr"); + } +} + +// The closure is transitive: a record reaches the contract because something +// the API names refers to it, however indirectly. +TEST_F(ImplHeaderParserTest, RecordsReachableOnlyThroughAnotherRecordAreKept) +{ + auto r = parseImplHeader( + fixturesDir() + "/records_impl.h", + "RecordsImpl", + fixturesDir() + "/records_metadata.json", + err); + ASSERT_FALSE(r.hasError()) << r.error.toStdString(); + + // Wrapper's own fields name Blob; both survive even though a signature + // could have named only one of them. + bool sawWrapper = false; + for (const auto& t : r.module.types) { + if (t.name != "Wrapper") continue; + sawWrapper = true; + ASSERT_EQ(t.fields.size(), 2u); + EXPECT_EQ(t.fields[0].type.name, "Blob"); + EXPECT_EQ(t.fields[1].type.elements.at(0).name, "Blob"); + } + EXPECT_TRUE(sawWrapper); +} + TEST_F(ImplHeaderParserTest, MissingHeaderFile) { auto r = parseImplHeader( diff --git a/tests/experimental/test_lidl_gen_cdylib.cpp b/tests/experimental/test_lidl_gen_cdylib.cpp index 07fa290..6fb13dc 100644 --- a/tests/experimental/test_lidl_gen_cdylib.cpp +++ b/tests/experimental/test_lidl_gen_cdylib.cpp @@ -141,9 +141,14 @@ TEST(LidlGenCdylib, JsonEventPayloadIsQtFree) EXPECT_FALSE(source.contains("QVariant")); } -// `[bstr]` is in the supported subset: each element carries the canonical tagged -// form, so a module can take or return a list of blobs (e.g. a program plus its -// dependency ELFs) instead of hand-encoding them as hex strings. +// `[bstr]` is in the supported subset: each element carries the canonical +// tagged form, so a module can take or return a list of blobs (e.g. a program +// plus its dependency ELFs) instead of hand-encoding them as hex strings. +// +// #111 reached this with a dedicated depth-1 list codec; the gate now RECURSES +// and the generated Codec's full specialization for std::vector beats +// its generic vector rule, so the same mechanism covers [bstr], [[bstr]] and +// {tstr: [bstr]}. The assertions moved to that mechanism; what they pin did not. TEST(LidlGenCdylib, ArrayOfBytesEventParamIsEligibleAndTagsEachElement) { const ModuleDecl m = moduleWithEvent("batchReceived", { @@ -154,23 +159,22 @@ TEST(LidlGenCdylib, ArrayOfBytesEventParamIsEligibleAndTagsEachElement) EXPECT_TRUE(lidlCdylibSupported(m, &error)) << error.toStdString(); const QString source = eventsSourceFor(m); - - // Each element is tagged, not emitted as a nested number array — which is - // what nlohmann::json(std::vector>) would have produced - // and no consumer decodes as bytes. - EXPECT_TRUE(source.contains("args.push_back(lidlBytesListToJson(payloads));")); - EXPECT_TRUE(source.contains("nlohmann::json lidlBytesListToJson")); - EXPECT_TRUE(source.contains("out.push_back(lidlBytesToJson(bytes));")); - - // Qt-free, and taken by const-ref like the other composite payloads. - EXPECT_TRUE(source.contains("const std::vector>& payloads")); - EXPECT_FALSE(source.contains("QVariant")); + // Spelled Qt-free and encoded through the codec, so each element keeps its + // canonical tag instead of becoming a plain array of numbers. + EXPECT_TRUE(source.contains("std::vector>")) << source.toStdString(); + EXPECT_TRUE(source.contains("logos_gen::Codec>>::to(payloads)")) + << source.toStdString(); + // From #111, still exactly right: Qt-free, and taken by const-ref like the + // other composite payloads. + EXPECT_TRUE(source.contains("const std::vector>& payloads")) + << source.toStdString(); + EXPECT_FALSE(source.contains("QVariant")) << source.toStdString(); } -// The method path: a `[bstr]` parameter must be DECODED per element, never via -// nlohmann's blanket get<>(). get>>() throws on -// the tagged {"_bytes": …} object form, and would silently skip the base64 -// decode for a number-array element. +// Ported from #111. Its assertions named that PR's depth-1 helpers +// (lidlBytesListFromJson / lidlBytesListToJson); the generated Codec subsumes +// them, so the assertions moved to the codec while what they pin — per-element +// tagging, and never nlohmann's blanket container conversion — did not. TEST(LidlGenCdylib, ArrayOfBytesMethodParamDecodesPerElement) { const ModuleDecl m = moduleWithMethod(method("send", prim("tstr"), { @@ -183,42 +187,130 @@ TEST(LidlGenCdylib, ArrayOfBytesMethodParamDecodesPerElement) const QString source = implSourceFor(m); - EXPECT_TRUE(source.contains("lidlBytesListFromJson(")); - EXPECT_TRUE(source.contains("std::vector> lidlBytesListFromJson")); - EXPECT_TRUE(source.contains("out.push_back(lidlBytesFromJson(e));")); - // The scalar param still uses the scalar decoder. - EXPECT_TRUE(source.contains("lidlBytesFromJson(")); - // The blanket container decode must not be used for this type. - EXPECT_FALSE(source.contains(".get>>()")); + EXPECT_TRUE(source.contains("logos_gen::Codec>>::from(")) + << source.toStdString(); + // The scalar param still uses the scalar decoder — deliberately NOT routed + // through the codec, so its documented leniency is unchanged. + EXPECT_TRUE(source.contains("lidlBytesFromJson(")) << source.toStdString(); + // nlohmann's blanket container decode must not be used for this type: it + // refuses a tagged object and would silently accept a raw number array, + // skipping the base64 decode entirely. + EXPECT_FALSE(source.contains(".get>>()")) + << source.toStdString(); } -// The return path: nlohmann::json(std::vector>) would emit -// nested number arrays, which no consumer decodes as bytes. +// Ported from #111: a `[bstr]` RETURN tags each element. +// nlohmann::json(std::vector>) would emit nested number +// arrays, which no consumer decodes as bytes. TEST(LidlGenCdylib, ArrayOfBytesReturnTagsEachElement) { const ModuleDecl m = moduleWithMethod( - method("dependencies", TypeExpr{TypeExpr::Array, "", {prim("bstr")}}, {})); + method("fetchAll", TypeExpr{TypeExpr::Array, "", {prim("bstr")}}, {})); QString error; ASSERT_TRUE(lidlCdylibSupported(m, &error)) << error.toStdString(); const QString source = implSourceFor(m); - EXPECT_TRUE(source.contains("lidlBytesListToJson(")); - EXPECT_TRUE(source.contains("nlohmann::json lidlBytesListToJson")); + EXPECT_TRUE(source.contains("logos_gen::Codec>>::to(")) + << source.toStdString(); + EXPECT_FALSE(source.contains("nlohmann::json(result)")) << source.toStdString(); } -// The list encoder is gated the same way the scalar one is: a module whose -// events carry only a single blob must not gain an unused static function. -TEST(LidlGenCdylib, BytesListEncoderOmittedWhenNoEventCarriesAnArray) +// #111 gated its list encoder so a module that never carries `[bstr]` did not +// gain an unused static function. The generic codec is a TEMPLATE — it only +// instantiates where used — so that hazard is gone and there is no dedicated +// list encoder to omit. What still needs gating is the SCALAR encoder, and it +// still is; this pins both halves so neither regresses. +TEST(LidlGenCdylib, NoDedicatedListEncoderAndTheScalarOneStaysGated) { - const ModuleDecl m = moduleWithEvent("messageReceived", { - param("payload", prim("bstr")), + const ModuleDecl noBytes = moduleWithEvent("fault", { + param("code", prim("int")), + param("message", prim("tstr")), }); + const QString plain = eventsSourceFor(noBytes); + EXPECT_FALSE(plain.contains("lidlBytesToJson")) << plain.toStdString(); + EXPECT_FALSE(plain.contains("lidlBytesListToJson")) << plain.toStdString(); - const QString source = eventsSourceFor(m); + const ModuleDecl withList = moduleWithEvent("batchReceived", { + param("payloads", TypeExpr{TypeExpr::Array, "", {prim("bstr")}}), + }); + const QString listed = eventsSourceFor(withList); + // The list rides the codec; no bespoke list encoder is emitted at all. + EXPECT_FALSE(listed.contains("lidlBytesListToJson")) << listed.toStdString(); + EXPECT_TRUE(listed.contains("logos_gen::Codec>>::to(")) + << listed.toStdString(); +} + +// The gate recurses, so what it refuses is now a property of the leaf. A map +// with a non-tstr key has no C++ spelling (the codec spells a map as +// std::map) and must still be refused BY NAME — it used to be +// admitted by a blanket `return true` for any map and then silently flattened +// to an untyped LogosMap, losing the key type. +TEST(LidlGenCdylib, NonStringMapKeyIsRejected) +{ + ModuleDecl m; + m.name = "k_module"; + MethodDecl md; + md.name = "takeOddMap"; + md.returnType = prim("tstr"); + ParamDecl p; + p.name = "m"; + p.type = TypeExpr{TypeExpr::Map, "", {prim("int"), prim("tstr")}}; + md.params.push_back(p); + m.methods.push_back(md); + + QString error; + EXPECT_FALSE(lidlCdylibSupported(m, &error)); + EXPECT_TRUE(error.contains("takeOddMap")) << error.toStdString(); +} + +// A record the contract declares is admitted and spelled as its struct; an +// UNDECLARED Named type is not. `void` is the reason that distinction has to +// exist — it is not a LIDL builtin, so `-> void` arrives as Named("void"). +TEST(LidlGenCdylib, OnlyDeclaredRecordsAreRecords) +{ + ModuleDecl m; + m.name = "r_module"; + + TypeDecl rec; + rec.name = "Blob"; + FieldDecl f; + f.name = "payload"; + f.type = prim("bstr"); + rec.fields = {f}; + m.types.push_back(rec); + + MethodDecl good; + good.name = "echoBlob"; + good.returnType = TypeExpr{TypeExpr::Named, "Blob", {}}; + ParamDecl gp; gp.name = "v"; gp.type = TypeExpr{TypeExpr::Named, "Blob", {}}; + good.params.push_back(gp); + m.methods.push_back(good); + + QString error; + EXPECT_TRUE(lidlCdylibSupported(m, &error)) << error.toStdString(); + + // The struct and its codec specialization are emitted. + const QString types = lidlMakeTypesHeaderCdylib(m); + // Forward-declared, not defined: the struct is the author's (the contract + // was derived from that very declaration), so emitting it again would be a + // redefinition. + EXPECT_TRUE(types.contains("struct Blob;")) << types.toStdString(); + EXPECT_FALSE(types.contains("struct Blob {")) << types.toStdString(); + EXPECT_TRUE(types.contains("template <> struct Codec")) << types.toStdString(); + // The bstr field goes through the bytes codec, not nlohmann's array-of-numbers. + EXPECT_TRUE(types.contains("Codec>::to(v.payload)")) << types.toStdString(); + + // An undeclared Named type is NOT a record and stays refused. + MethodDecl bad; + bad.name = "takeGhost"; + bad.returnType = prim("tstr"); + ParamDecl bp; bp.name = "g"; bp.type = TypeExpr{TypeExpr::Named, "Ghost", {}}; + bad.params.push_back(bp); + m.methods.push_back(bad); + EXPECT_FALSE(lidlCdylibSupported(m, &error)); + EXPECT_TRUE(error.contains("takeGhost")) << error.toStdString(); - EXPECT_TRUE(source.contains("nlohmann::json lidlBytesToJson")); - EXPECT_FALSE(source.contains("lidlBytesListToJson")); } // The supported scalar / bytes payloads stay eligible. diff --git a/tests/experimental/test_lidl_gen_client.cpp b/tests/experimental/test_lidl_gen_client.cpp index 4acceb0..fdbd8ed 100644 --- a/tests/experimental/test_lidl_gen_client.cpp +++ b/tests/experimental/test_lidl_gen_client.cpp @@ -66,7 +66,7 @@ TEST(LidlGenClient, HeaderHasSyncMethods) auto m = makeTestModule(); QString h = lidlMakeHeader(m); EXPECT_TRUE(h.contains("QString createAccount(")); - EXPECT_TRUE(h.contains("int getBalance(")); + EXPECT_TRUE(h.contains("qulonglong getBalance(")); EXPECT_TRUE(h.contains("QStringList listAccounts(")); } @@ -149,8 +149,9 @@ TEST(LidlGenClient, SourceHasReturnConversion) QString s = lidlMakeSource(m); // createAccount returns tstr → QString, should use .toString() EXPECT_TRUE(s.contains("_result.toString()")); - // getBalance returns uint → int, should use .toInt() - EXPECT_TRUE(s.contains("_result.toInt()")); + // getBalance returns uint → qulonglong, so the accessor must be the 64-bit + // unsigned one; toInt() truncated and re-signed it. + EXPECT_TRUE(s.contains("_result.toULongLong()")); // listAccounts returns [tstr] → QStringList, should use .toStringList() EXPECT_TRUE(s.contains("_result.toStringList()")); } @@ -249,3 +250,117 @@ TEST(LidlGenClient, VoidReturnMethod) // The source should just call the method without capturing return EXPECT_FALSE(s.contains("QVariant _result = m_client->invokeRemoteMethod(\"test\", \"doStuff\"")); } + +// Records: a `type` decl becomes a real C++ struct in the generated header, so a +// Qt consumer says `Status s = client.makeStatus();` rather than digging fields +// out of a QVariantMap. Additive — nothing generated records before this. +static ModuleDecl makeRecordModule() +{ + ModuleDecl m; + m.name = "info_module"; + m.version = "1.0.0"; + + TypeDecl rec; + rec.name = "Status"; + FieldDecl a; a.name = "port"; a.type = { TypeExpr::Primitive, "uint", {} }; + FieldDecl b; b.name = "blob"; b.type = { TypeExpr::Primitive, "bstr", {} }; + rec.fields = {a, b}; + m.types.push_back(rec); + + { + MethodDecl md; + md.name = "describeStatus"; + md.returnType = { TypeExpr::Primitive, "tstr", {} }; + ParamDecl p; p.name = "s"; p.type = { TypeExpr::Named, "Status", {} }; + md.params.push_back(p); + m.methods.push_back(md); + } + { + MethodDecl md; + md.name = "makeStatuses"; + TypeExpr elem = { TypeExpr::Named, "Status", {} }; + md.returnType = { TypeExpr::Array, "", { elem } }; + m.methods.push_back(md); + } + return m; +} + +// `isTaggedBytes()` is checked BEFORE `is_object()` in both the codec and the +// QVariant bridge, so a record whose only field is a tstr named `_bytes` is +// wire-identical to a tagged byte string and decodes as bytes — the struct +// silently disappears. The ambiguity is inherent to the tagged form; refusing +// to emit the one shape guaranteed to misdecode is what a generator can do +// about it. +TEST(LidlGenClient, RecordThatCollidesWithTheBytesTagIsRefused) +{ + ModuleDecl m; + m.name = "c_module"; + TypeDecl bad; + bad.name = "Sneaky"; + FieldDecl f; f.name = "_bytes"; f.type = { TypeExpr::Primitive, "tstr", {} }; + bad.fields = {f}; + m.types.push_back(bad); + + QString err; + EXPECT_FALSE(lidlCheckRecords(m, &err)); + EXPECT_TRUE(err.contains("Sneaky")) << err.toStdString(); + EXPECT_TRUE(err.contains("_bytes")) << err.toStdString(); + + // A SECOND field disambiguates it — isTaggedBytes requires exactly one key, + // so this shape round-trips and must still be allowed. + FieldDecl g; g.name = "other"; g.type = { TypeExpr::Primitive, "int", {} }; + m.types[0].fields.push_back(g); + EXPECT_TRUE(lidlCheckRecords(m, nullptr)); + + // A `_bytes` field that is not the only one, and a differently-named sole + // field, are both fine. + ModuleDecl ok; + ok.name = "ok_module"; + TypeDecl t; t.name = "Fine"; + FieldDecl h; h.name = "payload"; h.type = { TypeExpr::Primitive, "tstr", {} }; + t.fields = {h}; + ok.types.push_back(t); + EXPECT_TRUE(lidlCheckRecords(ok, nullptr)); +} + +// The ASYNC overload must decode a record the same way the sync one does. +// +// `qvariant_cast(v)` does not fail on the wire's QVariantMap: no +// Q_DECLARE_METATYPE is emitted for the struct, so the cast silently yields a +// DEFAULT-CONSTRUCTED Status and the caller sees empty fields with no +// diagnostic. The sync path was already correct, which makes it worse — the +// same call would be right or wrong depending only on which overload the +// caller reached for. +TEST(LidlGenClient, AsyncRecordReturnsDecodeFieldByField) +{ + const QString c = lidlMakeSource(makeRecordModule(), BindMode::Bound); + + // A [Record] return, in the async callback. + EXPECT_TRUE(c.contains("StatusFromVariant")) << c.toStdString(); + EXPECT_FALSE(c.contains("qvariant_cast>")) << c.toStdString(); + EXPECT_FALSE(c.contains("qvariant_cast")) << c.toStdString(); +} + +TEST(LidlGenClient, RecordsBecomeStructsWithConversions) +{ + const QString h = lidlMakeHeader(makeRecordModule(), BindMode::Bound); + + // The struct, at the 1-1 Qt spellings: 64-bit unsigned, QByteArray for bytes. + EXPECT_TRUE(h.contains("struct Status {")) << h.toStdString(); + EXPECT_TRUE(h.contains("qulonglong port{};")) << h.toStdString(); + EXPECT_TRUE(h.contains("QByteArray blob{};")) << h.toStdString(); + + // Conversions both ways. + EXPECT_TRUE(h.contains("inline QVariant StatusToVariant(const Status& v)")) << h.toStdString(); + EXPECT_TRUE(h.contains("inline Status StatusFromVariant(const QVariant& value)")) << h.toStdString(); + // A bstr field is a QByteArray: logos-protocol's QVariant<->JSON conversion + // already materialises the tagged {"_bytes":…} form as QByteArray, so binary + // survives without record-specific bytes handling. + EXPECT_TRUE(h.contains("__out.blob = __m.value(\"blob\").toByteArray();")) << h.toStdString(); + + // Methods speak the record: by const& in, typed list out. A QVariantList + // could not hold a Status without Q_DECLARE_METATYPE. + EXPECT_TRUE(h.contains("describeStatus(const Status& s")) << h.toStdString(); + EXPECT_TRUE(h.contains("QList makeStatuses(")) << h.toStdString(); +} + diff --git a/tests/experimental/test_lidl_type_mapping.cpp b/tests/experimental/test_lidl_type_mapping.cpp index 45ca949..43053db 100644 --- a/tests/experimental/test_lidl_type_mapping.cpp +++ b/tests/experimental/test_lidl_type_mapping.cpp @@ -18,16 +18,19 @@ TEST(LidlTypeToQt, Bstr) EXPECT_EQ(lidlTypeToQt(te), "QByteArray"); } +// LIDL int/uint are 64-bit everywhere (int64_t/uint64_t in C++ impls, i64/u64 in +// Rust), so the Qt spelling has to be 64-bit too — one LIDL type, one type per +// language. `int` truncated, and for `uint` it also flipped the signedness. TEST(LidlTypeToQt, Int) { TypeExpr te = { TypeExpr::Primitive, "int", {} }; - EXPECT_EQ(lidlTypeToQt(te), "int"); + EXPECT_EQ(lidlTypeToQt(te), "qlonglong"); } TEST(LidlTypeToQt, Uint) { TypeExpr te = { TypeExpr::Primitive, "uint", {} }; - EXPECT_EQ(lidlTypeToQt(te), "int"); + EXPECT_EQ(lidlTypeToQt(te), "qulonglong"); } TEST(LidlTypeToQt, Float64) @@ -83,10 +86,13 @@ TEST(LidlTypeToQt, OptionalType) EXPECT_EQ(lidlTypeToQt(te), "QVariant"); } -TEST(LidlTypeToQt, NamedType) +// A Named type is a RECORD declared by the contract, and the client generator +// emits a struct of that name — so the Qt spelling is the struct, not an opaque +// QVariant. One LIDL type, one type per language. +TEST(LidlTypeToQt, NamedTypeIsItsRecordStruct) { TypeExpr te = { TypeExpr::Named, "MyStruct", {} }; - EXPECT_EQ(lidlTypeToQt(te), "QVariant"); + EXPECT_EQ(lidlTypeToQt(te), "MyStruct"); } // --------------------------------------------------------------------------- diff --git a/tests/generator/CMakeLists.txt b/tests/generator/CMakeLists.txt index 7c15561..0075af2 100644 --- a/tests/generator/CMakeLists.txt +++ b/tests/generator/CMakeLists.txt @@ -8,6 +8,7 @@ add_executable(generator_tests test_make_header.cpp test_make_source.cpp test_parse_provider_header.cpp + test_records.cpp ) target_include_directories(generator_tests PRIVATE diff --git a/tests/generator/test_map_param_type.cpp b/tests/generator/test_map_param_type.cpp index e981286..b3c7705 100644 --- a/tests/generator/test_map_param_type.cpp +++ b/tests/generator/test_map_param_type.cpp @@ -33,3 +33,16 @@ TEST(MapParamTypeTest, LogosResultFallsBack) // LogosResult is NOT in the param known set EXPECT_EQ(mapParamType("LogosResult"), "QVariant"); } + +// LIDL int/uint are 64-bit, and since the Qt spelling became qlonglong/qulonglong +// they have to be in the known set — otherwise every int/uint method silently +// degrades to an opaque QVariant in the generated lp/std consumer wrappers. +TEST(MapParamType, SixtyFourBitIntegersAreKnown) +{ + EXPECT_EQ(mapParamType("qlonglong"), "qlonglong"); + EXPECT_EQ(mapParamType("qulonglong"), "qulonglong"); + EXPECT_EQ(mapParamType("const qlonglong&"), "qlonglong"); + // An unknown spelling still falls back. + EXPECT_EQ(mapParamType("SomeRecord"), "QVariant"); +} + diff --git a/tests/generator/test_map_return_type.cpp b/tests/generator/test_map_return_type.cpp index c9d2e54..cbe6df0 100644 --- a/tests/generator/test_map_return_type.cpp +++ b/tests/generator/test_map_return_type.cpp @@ -31,3 +31,16 @@ TEST(MapReturnTypeTest, UnknownType) { EXPECT_EQ(mapReturnType("MyCustomType"), "QVariant"); } + +// LIDL int/uint are 64-bit, and since the Qt spelling became qlonglong/qulonglong +// they have to be in the known set — otherwise every int/uint method silently +// degrades to an opaque QVariant in the generated lp/std consumer wrappers. +TEST(MapReturnType, SixtyFourBitIntegersAreKnown) +{ + EXPECT_EQ(mapReturnType("qlonglong"), "qlonglong"); + EXPECT_EQ(mapReturnType("qulonglong"), "qulonglong"); + EXPECT_EQ(mapReturnType("const qlonglong&"), "qlonglong"); + // An unknown spelling still falls back. + EXPECT_EQ(mapReturnType("SomeRecord"), "QVariant"); +} + diff --git a/tests/generator/test_records.cpp b/tests/generator/test_records.cpp new file mode 100644 index 0000000..09c869a --- /dev/null +++ b/tests/generator/test_records.cpp @@ -0,0 +1,216 @@ +// Records on the consumer side of the LEGACY generator — the wrapper every +// C++ module actually gets for its dependencies (`--dep =`). +// +// A contract's `type Status { ... }` used to reach every C++ consumer as an +// untyped bag: QVariant on the Qt surface, LogosMap on the std/lp one. The +// caller then had to know the field names AND, for a `bstr` field, that the +// value arrives as the canonical `{"_bytes": "..."}` envelope it must unwrap +// itself — while Rust and the client-stub backend hand back a real struct. +// +// These assert on generated source text. That the emitted conversions compile +// and round-trip (bytes tagged at every depth, uint64 above 2^32 intact) is +// covered by generating a wrapper and building it — see the PR description. + +#include +#include +#include +#include "generator_lib.h" + +namespace { + +QJsonObject field(const char* name, const char* type) +{ + QJsonObject f; + f["name"] = name; + f["type"] = type; + return f; +} + +QJsonObject param(const char* name, const char* type) +{ + QJsonObject p; + p["name"] = name; + p["type"] = type; + return p; +} + +QJsonObject method(const char* name, const char* returnType, const QJsonArray& params = {}) +{ + QJsonObject m; + m["name"] = name; + m["returnType"] = returnType; + m["isInvokable"] = true; + m["parameters"] = params; + return m; +} + +// `type Status { port: uint, blob: bstr }` plus a record that nests it in +// both container shapes. +QJsonArray statusRecords() +{ + QJsonObject status; + status["name"] = "Status"; + status["fields"] = QJsonArray{field("port", "qulonglong"), field("blob", "QByteArray")}; + + QJsonObject batch; + batch["name"] = "Batch"; + batch["fields"] = QJsonArray{field("label", "QString"), + field("items", "QList"), + field("tags", "QMap")}; + return QJsonArray{status, batch}; +} + +QJsonArray statusMethods() +{ + return QJsonArray{ + method("getStatus", "Status"), + method("describeStatus", "QString", QJsonArray{param("s", "Status")}), + method("listStatuses", "QList"), + method("getBatch", "Batch"), + }; +} + +} // namespace + +// The Qt surface: a struct nested in the wrapper class, typed accessors, and +// no QVariant anywhere a record is named. +TEST(Records, QtWrapperExposesTheStruct) +{ + const QString h = makeHeader("info_module", "InfoModule", statusMethods(), + ApiStyle::Qt, {}, BindMode::Static, statusRecords()); + + // Nested, so two deps may each declare a `Status` in one consumer. + EXPECT_TRUE(h.contains(" struct Status {")); + EXPECT_TRUE(h.contains(" qulonglong port{};")); + EXPECT_TRUE(h.contains(" QByteArray blob{};")); + + // Containers of records keep their element type. + EXPECT_TRUE(h.contains(" QList items{};")); + EXPECT_TRUE(h.contains(" QMap tags{};")); + + EXPECT_TRUE(h.contains("Status getStatus(logos::CallError* err = nullptr);")); + EXPECT_TRUE(h.contains("QString describeStatus(const Status& s,")); + EXPECT_TRUE(h.contains("QList listStatuses(")); + + // The old fallback is gone. + EXPECT_FALSE(h.contains("QVariant getStatus(")); + EXPECT_FALSE(h.contains("describeStatus(QVariant")); +} + +// The std / lp surfaces spell the same records in std types — a universal +// (Qt-free) module never sees a Qt name. +TEST(Records, StdAndLpWrappersUseStdFieldTypes) +{ + for (ApiStyle style : {ApiStyle::Std, ApiStyle::Lp}) { + const QString h = makeHeader("info_module", "InfoModule", statusMethods(), + style, {}, BindMode::Static, statusRecords()); + EXPECT_TRUE(h.contains(" uint64_t port{};")) << h.toStdString(); + EXPECT_TRUE(h.contains(" std::vector blob{};")); + EXPECT_TRUE(h.contains(" std::vector items{};")); + EXPECT_TRUE(h.contains(" std::map tags{};")); + EXPECT_TRUE(h.contains("std::vector listStatuses(")); + // No LogosMap stand-in for a record. + EXPECT_FALSE(h.contains("LogosMap getStatus(")); + } + + // std::map needs its header on the std/lp surfaces. + const QString lp = makeHeader("info_module", "InfoModule", statusMethods(), + ApiStyle::Lp, {}, BindMode::Static, statusRecords()); + EXPECT_TRUE(lp.contains("#include ")); +} + +// A `bstr` field must ride the canonical tagged form at any depth — the +// defect class that made a record-as-LogosMap actively wrong rather than +// merely inconvenient. +TEST(Records, BytesFieldsUseTheCanonicalEncoding) +{ + const QString lp = makeSource("info_module", "InfoModule", "info_module_api.h", + statusMethods(), ApiStyle::Lp, {}, BindMode::Static, + statusRecords()); + EXPECT_TRUE(lp.contains("__j[\"blob\"] = logos::bytesToJson(v.blob);")); + EXPECT_TRUE(lp.contains("__out.blob = logos::jsonToBytes(w.at(\"blob\"));")); + + const QString qt = makeSource("info_module", "InfoModule", "info_module_api.h", + statusMethods(), ApiStyle::Qt, {}, BindMode::Static, + statusRecords()); + EXPECT_TRUE(qt.contains("__out.blob = __m.value(QStringLiteral(\"blob\")).toByteArray();")); +} + +// The conversions are file-local statics in the .cpp: a std/lp consumer's own +// translation units must not need the wire type to include the header. +TEST(Records, ConversionsStayOutOfTheHeader) +{ + const QString h = makeHeader("info_module", "InfoModule", statusMethods(), + ApiStyle::Lp, {}, BindMode::Static, statusRecords()); + EXPECT_FALSE(h.contains("recToWire_Status")); + + const QString c = makeSource("info_module", "InfoModule", "info_module_api.h", + statusMethods(), ApiStyle::Lp, {}, BindMode::Static, + statusRecords()); + EXPECT_TRUE(c.contains("static nlohmann::json recToWire_Status(const InfoModule::Status& v);")); + EXPECT_TRUE(c.contains("static InfoModule::Status recFromWire_Status(const nlohmann::json& w);")); + // Declared before defined, so records may reference each other in any order. + EXPECT_LT(c.indexOf("static nlohmann::json recToWire_Batch(const InfoModule::Batch& v);"), + c.indexOf("static nlohmann::json recToWire_Status(const InfoModule::Status& v) {")); +} + +// A return type written before the `Class::` of a definition is outside class +// scope and must be qualified; a parameter is inside it and must not be +// (an unqualified return type simply does not compile). +TEST(Records, ReturnTypesAreQualifiedInTheDefinition) +{ + const QString c = makeSource("info_module", "InfoModule", "info_module_api.h", + statusMethods(), ApiStyle::Qt, {}, BindMode::Static, + statusRecords()); + EXPECT_TRUE(c.contains("InfoModule::Status InfoModule::getStatus(")); + EXPECT_TRUE(c.contains("QList InfoModule::listStatuses(")); + EXPECT_TRUE(c.contains("QString InfoModule::describeStatus(const Status& s,")); +} + +// Container decode lambdas are emitted INSIDE the record decoder, which has +// its own `__m` / `__j`. Reusing those names made a map-of-records field read +// from its own uninitialized local — it compiled, with only a warning. +TEST(Records, ContainerLambdasDoNotShadowTheDecoderLocals) +{ + for (ApiStyle style : {ApiStyle::Qt, ApiStyle::Std, ApiStyle::Lp}) { + const QString c = makeSource("info_module", "InfoModule", "info_module_api.h", + statusMethods(), style, {}, BindMode::Static, + statusRecords()); + // The decoder's own map is `__m` (Qt/Std); a nested lambda must not + // declare another one. + EXPECT_FALSE(c.contains("const QVariantMap __m = (__m.value")) << c.toStdString(); + EXPECT_FALSE(c.contains("const nlohmann::json& __j = w.at")) << c.toStdString(); + } +} + +// The record path is additive: with no records declared, every byte of the +// generated output is what it was before. +TEST(Records, EmptyRecordSetChangesNothing) +{ + const QJsonArray methods{method("ping", "QString", QJsonArray{param("msg", "QString")})}; + for (ApiStyle style : {ApiStyle::Qt, ApiStyle::Std, ApiStyle::Lp}) { + EXPECT_EQ(makeHeader("m", "M", methods, style, {}, BindMode::Static, {}), + makeHeader("m", "M", methods, style, {}, BindMode::Static)); + EXPECT_EQ(makeSource("m", "M", "m_api.h", methods, style, {}, BindMode::Static, {}), + makeSource("m", "M", "m_api.h", methods, style, {}, BindMode::Static)); + } +} + +// Records reach event callbacks too — an event payload is as typed as a +// return value. +TEST(Records, TypedEventCallbacksTakeTheStruct) +{ + QJsonObject ev; + ev["name"] = "statusChanged"; + ev["params"] = QJsonArray{param("s", "Status"), param("at", "qlonglong")}; + const QJsonArray events{ev}; + + const QString h = makeHeader("info_module", "InfoModule", statusMethods(), + ApiStyle::Qt, events, BindMode::Static, statusRecords()); + EXPECT_TRUE(h.contains("bool onStatusChanged(std::function callback);")); + + const QString c = makeSource("info_module", "InfoModule", "info_module_api.h", + statusMethods(), ApiStyle::Qt, events, BindMode::Static, + statusRecords()); + EXPECT_TRUE(c.contains("recFromWire_Status(_args.at(0))")); +}