Files
logos-cpp-sdk/cpp-generator/experimental/impl_header_parser.cpp
T
Dario LipicarandClaude Opus 5 3d322bd315 fix(codegen): LIDL int/uint are 64-bit in the Qt spelling too (#113)
* fix(codegen): LIDL int/uint are 64-bit in the Qt spelling too

lidlTypeToQt mapped BOTH `int` and `uint` to plain `int`. Everywhere else in the
stack a LIDL int/uint is 64-bit — int64_t/uint64_t in C++ impls, i64/u64 in the
Rust SDK — so the Qt spelling broke the one-type-per-LIDL-type rule and lost
data: a Qt consumer reading a `uint` return got a SIGNED 32-bit value, so
anything above 2^31 came back wrong and anything above 2^63 was never
expressible.

int -> qlonglong, uint -> qulonglong, and returnConversion() gains the matching
accessors (toLongLong / toULongLong instead of toInt).

qlonglong/qulonglong rather than qint64/quint64 so the generated introspection
JSON uses the same names Qt's own metaobject normalisation produces — otherwise
a cdylib module's generated `signature` and a legacy module's
QMetaObject-derived one would disagree for the same LIDL type. Nothing looks
these strings up: the only QMetaType::fromName call in the stack is for
"LogosResult".

This changes two generated surfaces: the Qt consumer wrapper signatures and the
introspection JSON. Passing an int argument still converts implicitly, so
callers keep compiling; code that assigns a wrapper's return into an `int`
narrows and may warn, which is the bug being surfaced rather than a regression.

Tests: 168/168, with the type-mapping and client-emitter expectations updated to
the 64-bit spelling.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* feat(records): typed C++ structs for Qt consumers

Completes the Qt half of the type mapping this branch started. A `type Foo { … }`
in a contract now generates a real struct in the client header, so a consumer
writes `Status s = client.makeStatus();` instead of digging fields out of a
QVariantMap. One LIDL type, one type per language.

  lidlTypeToQt   - Named -> the record's struct (was QVariant)
                 - [Record] -> QList<Record>, {tstr: Record} -> QMap<QString,
                   Record>. QVariantList CANNOT hold a record without
                   Q_DECLARE_METATYPE, and a typed list is the point.
  client emitter - struct + inline ToVariant/FromVariant per record, emitted
                   before the class; conversions come after all structs so
                   records may reference each other. Recursive, so a field may
                   itself be [Status] or {tstr: bstr}.
                 - records pass by const&, decode on return, and convert at the
                   call site (sync and async)

bstr fields are QByteArray on purpose: logos-protocol's QVariant<->JSON
conversion already materialises the canonical {"_bytes": base64url} form as a
QByteArray and back, so the record conversions stay pure field mapping and binary
survives at any depth with no record-specific bytes handling.

Verified by COMPILING and RUNNING the generated code, not just asserting on text
— the string tests would not have caught either bug this found: [Record] first
mapped to QVariantList (appending a Status to it does not compile) and the decode
lambdas shadowed their accumulator. Extracted the emitted record block for a
contract with a nested record and a bytes field, compiled it against Qt6Core, and
round-tripped Batch -> QVariant -> Batch asserting items[0].port, the QByteArray
blob and the label all survive. Exit 0.

The LidlTypeToQt.NamedType expectation flips from "QVariant" to the struct name,
which is the behaviour change.

Tests: 169/169.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(codegen): teach the lp/std consumer wrappers the 64-bit spellings

Caught while checking whether the SDKs are ready for the `any` migration, and it
is a regression THIS BRANCH would otherwise have shipped.

legacy/generator_lib.cpp generates the lp/std consumer wrappers a universal
module uses to call its dependencies. It matches type names against an
allow-list and falls back to QVariant for anything else:

    static const QSet<QString> known = {
        "void","bool","int","double","float","QString", … };
    if (known.contains(base)) return base;
    return QString("QVariant");

Once lidlTypeToQt reports `qlonglong`/`qulonglong`, every LIDL int/uint method
misses that list — so a typed `int` parameter would have silently become an
opaque QVariant in those wrappers. Worse than the truncation this branch set out
to fix, and invisible until someone read the generated header.

Adds the two spellings to both allow-lists, plus the conversions they imply:
QVariant->Qt (toLongLong / toULongLong), the std spellings (int64_t / uint64_t),
the QVariant->std return path, the Qt-style return, and the default-value case.
The existing `int` entries stay for legacy Qt plugins, whose QMetaObject still
reports `int` for a 32-bit parameter.

Tests: 171/171, with the allow-list pinned in both mapping test files —
including that an unknown spelling still falls back to QVariant, so the fallback
itself is not what regressed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* feat(records): typed structs for C++ dependency wrappers

Closes the second record gap. The wrapper every C++ module actually gets for
its dependencies comes from the LEGACY generator (`--dep <name>=<lidl>`), not
the client-stub backend the previous commit taught about records — and there a
contract's `type Status { ... }` reached the consumer as an untyped bag:
QVariant on the Qt surface, LogosMap on the std/lp one. Worse than
inconvenient for a `bstr` field: the caller received the canonical
`{"_bytes": "..."}` envelope and had to know to unwrap it, while Rust and the
stub backend handed back real bytes.

Now, on all three api styles:

    struct Status { uint64_t port{}; std::vector<uint8_t> blob{}; };
    Status getStatus(logos::CallError* err = nullptr);
    std::string describeStatus(const Status& s, ...);
    std::vector<Status> listStatuses(...);

The struct is NESTED in the wrapper class (`InfoModule::Status`) because a
module consuming two deps that each declare a `Status` includes both wrappers
into one translation unit. Conversions are file-local statics in the generated
.cpp, so a Qt-free module's own TUs still never see QVariant or nlohmann.
Records reach parameters, returns, event callbacks, `[Record]` and
`{tstr: Record}` — at any depth, with bytes tagged throughout.

Same commit, the legacy path's half of the 64-bit fix: `lidlTypeExprToQtTypeName`
mapped BOTH int and uint to `int` ("wire-as-int for now"), so a `uint` method on
a dep reached a Qt consumer as a signed 32-bit value and a std/lp consumer as a
signed int64_t. Now qlonglong/qulonglong, matching the spelling the other half
of this PR gave the stub backend. `lpFromJsonExpr` grew the uint64_t branch it
needed — without it a mistyped payload THREW out of nlohmann's implicit
conversion instead of defaulting like every other scalar.

Verified by generating a contract with a record, a record-of-records, a `uint`
above 2^32 and a high-byte `bstr`, then COMPILING the output for qt/std/lp and
round-tripping the emitted conversions:
  - `{"_bytes":"gAH_"}` at every depth, decoding back to the same bytes
  - 4294967296 intact through both directions
  - garbage/missing fields default rather than throw
That compile is what caught the one real bug here: the container decode lambdas
declared `__m`/`__j`, shadowing the record decoder's own locals, so a
map-of-records field read from its own uninitialized local — it compiled with
nothing but a -Wuninitialized warning. Locals are `__acc`/`__src` now, pinned
by a test.

Also: 8 generator tests (one asserting an empty record set leaves every byte
of the output as it was), 179 total green; logos-test-modules builds and tests
green against this generator.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(records): async record returns decoded a default-constructed struct

Two correctness holes in the record work, both silent, found while scoping the
cdylib provider side.

1. The async consumer overload emitted `qvariant_cast<Status>(v)` while the sync
   one emitted `StatusFromVariant(_result)`. The wire delivers a QVariantMap and
   no Q_DECLARE_METATYPE is emitted for the struct, so the cast does not fail —
   it returns a DEFAULT-CONSTRUCTED Status and the caller sees empty fields with
   no diagnostic. The sync path being correct is what makes it bad: the same
   call is right or wrong depending only on which overload the caller reached
   for. Async now decodes field by field through the same conversion.

   (The legacy dependency-wrapper generator already did this correctly — this
   was the experimental client-stub backend only.)

2. A record whose ONLY field is a tstr named `_bytes` is wire-identical to a
   canonical tagged byte string: `isTaggedBytes()` is checked before
   `is_object()` in both logos_codec.h and logos_json_convert.cpp, so such a
   record decodes as bytes and the struct silently disappears. The ambiguity is
   inherent to the tagged form — the codec's own comment says not to name a map
   key `_bytes` — but the generator can refuse to emit the one shape guaranteed
   to misdecode instead of leaving it to be found at runtime. Both front doors
   (the .lidl client-stub path and the --dep/--interface path) now reject it
   with a message naming the type and the fix.

   A second field disambiguates it (isTaggedBytes requires exactly one key), so
   that shape still generates — verified, not assumed.

181 tests, +2.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* feat(cdylib): records, [bstr], typed maps and nested composites on the C++ provider

The C++ cdylib backend could express scalars, `[scalar]`, `any` and an untyped
map. Everything else it rejected BY NAME — `[bstr]`, `[[int]]`, `{tstr: int}`,
and records, which the impl-header parser could not even declare because it
skipped `struct` outright. That is why the ext contract had to be Rust-only.

Four changes, in the order they matter:

  typeSupported()  recurses instead of whitelisting element names, which admits
                   [bstr], [[int]], [Record] and [{tstr: T}] in one rule; a
                   declared record is admitted; a map now REQUIRES a tstr key
                   (it used to `return true` for any map and then silently
                   flatten {int: tstr} to an untyped LogosMap, losing the key
                   type).
  lidlTypeToStdCdylib()  became total. Its `lidlTypeToStd` fallback answers
                   QVariantList / QVariantMap — Qt names in a Qt-FREE
                   translation unit — and only failed to appear because the
                   gate rejected everything that reached it. Widening the gate
                   made that fallback a live leak, so composites now recurse and
                   never reach it.
  <name>_types.h   new: the generated codec, recursive, with a FULL
                   specialization for std::vector<uint8_t> that wins over the
                   generic vector rule — which is what keeps a bstr tagged at
                   any depth instead of becoming a plain array of numbers. One
                   Codec specialization per declared record, field by field,
                   with the field path in the error.
  impl_header_parser  learned `struct` (two passes, because a record field may
                   name another record and the type mapper only answers Named()
                   for an already-registered name — one pass silently typed
                   `Blob inner;` as `any`), std::map<std::string, T>, and
                   recursion into vector elements so std::vector<Blob> is
                   [Blob] rather than falling through to `any`.

Records are only names the contract DECLARES: `void` is not a LIDL builtin, so
`-> void` arrives as Named("void"), and treating every Named as a record is the
exact trap that made the Rust generator emit `-> Void`.

Two things the interface JSON got wrong, both found by running it:

  - it spelled a record `Blob` and a `[Record]` `QList<Blob>`. Those are the
    CONSUMER's names, correct in a generated wrapper where the struct exists —
    but this JSON is the module's getMethods(), read by the host to marshal a
    QVariant, and there is no metatype called `Blob`. The host SIGSEGV'd on the
    first call to any record method. A record IS a variant map at that boundary;
    lidlTypeToQtWire() says so.
  - the types header emitted the structs. Header-first, the author owns them and
    the contract was derived from those very declarations, so it was a
    redefinition. It emits forward declarations and the codec.

Also: `jsonReturn` is set by the front end for any map return, which no longer
implies the C++ type IS nlohmann::json now that a typed map is
std::map<std::string, T> — checking the flag before the spelling emitted
`result.dump()` on a std::map. The spelling decides.

Scalars keep their nlohmann accessor verbatim rather than routing through the
codec: `.get<int64_t>()` TRUNCATES a float instead of throwing, and the
conformance matrix pins that leniency (hostile/int/fractional expects 3 from
3.7). Changing it would silently move behaviour something depends on.

The pinned-rejection test for [bstr] is INVERTED rather than deleted — the cell
it pinned still matters, only its answer changed — plus new tests for the
non-tstr map key rejection and for declared-vs-undeclared records. 183 tests.
Every existing module still builds; test_fullapi_cpp is unchanged.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(records): only structs the API mentions become contract types

Teaching the impl-header parser to read `struct` (previous commit) published
EVERY struct in a header as a contract `type`. Two production modules already
carry private helpers:

  openmetrics-module      struct ModuleSource   (namespace scope, internal)
  logos-package-manager   struct PendingAction  (PRIVATE, inside the class)

Both were being published — a module's interface changing as a side effect of
an internal refactor, which is not something deriving a contract from a header
may do. PendingAction was published WRONG as well: its fields carry trailing
`// comments`, the field regex requires a line ending in ';', and the
unmatched fields were silently dropped. A record with a partial field list is
worse than no record, because it looks like a contract.

A struct now earns its place by appearing in a method or event signature —
transitively, since a published record's own fields may name others. Verified
on the real headers: package-manager and openmetrics publish zero types again,
while the ext provider keeps both Blob and Wrapper (Wrapper is reachable only
through Blob's use in a signature). Trailing comments are stripped before the
field match, so no field is dropped.

Two tests over a fixture carrying both an internal namespace-scope struct and a
private in-class one; 185 tests. test-modules and openmetrics both rebuild.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(doctests): the generator round-trip now shows 64-bit ints and typed records

The two failing assertions in cpp-sdk-generator-roundtrip were documentation
asserting the OLD behaviour, and both changes are the point of this branch:

  int record(int id, …)          ->  qlonglong record(qulonglong id, …)
  QVariant translate(QVariant p) ->  Point translate(const Point& p, …)

`record`'s `id` is a `uint64_t` in the impl header, so the old signature was
handing a caller a SIGNED 32-bit value for a `uint` — the doc showed the bug.

The surrounding prose was wrong too, not just the expectations, so both blocks
are rewritten rather than patched:

  * Flow 3 now states the mapping as int->qlonglong / uint->qulonglong and says
    why (LIDL int/uint are int64_t/uint64_t in every other binding), pointing at
    `record` as the worked example.
  * The composite section claimed "records and optionals surface as QVariant".
    Records now generate a struct, `[Point]` a QList<Point> and `{tstr: Point}`
    a QMap<QString, Point>; maps of `any`, optionals and bare `any` still cross
    untyped and stay QVariant/QVariantMap — a record has a declared shape, those
    do not. The new text draws that line explicitly.

Expectations added for `struct Point` and `Point bounds(const QList<Point>&…)`
so the record path is pinned in the doc, not just described.

Verified the way CI runs it — `--release-for logos-cpp-sdk=feat/qt-64bit-numerics`,
which is what makes `{release}` resolve to this branch instead of master:
10 passed, 0 failed. (A plain local run builds master and is not
representative — that is why it still showed the old signatures.)

outputs/ regenerated; the diff also picks up unrelated pre-existing drift where
the committed Markdown had fallen behind the spec.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-07-28 17:24:49 -03:00

711 lines
30 KiB
C++

#include "impl_header_parser.h"
#include <QFile>
#include <QFileInfo>
#include <QJsonDocument>
#include <QJsonObject>
#include <QJsonArray>
#include <QRegularExpression>
#include <QSet>
#include <functional>
#include <set>
#include <QStringList>
// ---------------------------------------------------------------------------
// Strip leading declaration specifiers / attributes from a return-type string.
// ---------------------------------------------------------------------------
static QString stripDeclarationSpecifiers(QString string)
{
static const QRegularExpression attributeRe("\\[\\[[^\\]]*\\]\\]");
static const QRegularExpression specifierRe(
"^(static|virtual|inline|explicit|constexpr|consteval|friend)\\s+");
string.remove(attributeRe);
string = string.trimmed();
QRegularExpressionMatch specifierMatch = specifierRe.match(string);
while (specifierMatch.hasMatch()) {
string = string.mid(specifierMatch.capturedLength()).trimmed();
specifierMatch = specifierRe.match(string);
}
return string;
}
// ---------------------------------------------------------------------------
// C++ type string → LIDL TypeExpr
// ---------------------------------------------------------------------------
// The records the header declares, discovered by scanForRecords() before any
// method is parsed. A bare `Blob` in a signature is only a record if the header
// actually declared `struct Blob { ... };` — otherwise it stays the opaque
// `any` it always was.
static QSet<QString> g_recordNames;
static TypeExpr cppTypeToLidl(const QString& raw)
{
// Normalize: strip const, &, leading/trailing whitespace
QString t = raw.trimmed();
t.remove(QRegularExpression("^const\\s+"));
t.remove(QRegularExpression("\\s*&$"));
t = t.trimmed();
// Primitives
if (t == "bool") return { TypeExpr::Primitive, "bool", {} };
if (t == "int64_t") return { TypeExpr::Primitive, "int", {} };
if (t == "uint64_t") return { TypeExpr::Primitive, "uint", {} };
if (t == "double") return { TypeExpr::Primitive, "float64", {} };
if (t == "void") return { TypeExpr::Primitive, "void", {} };
// std::string
if (t == "std::string")
return { TypeExpr::Primitive, "tstr", {} };
// std::vector<T>
static QRegularExpression vecRe("^std::vector\\s*<\\s*(.+)\\s*>$");
QRegularExpressionMatch m = vecRe.match(t);
if (m.hasMatch()) {
QString inner = m.captured(1).trimmed();
if (inner == "std::string") {
TypeExpr elem = { TypeExpr::Primitive, "tstr", {} };
return { TypeExpr::Array, "", { elem } };
}
if (inner == "uint8_t") {
return { TypeExpr::Primitive, "bstr", {} };
}
// std::vector<std::vector<uint8_t>> — an array of byte strings. Spelled
// out so it lands on `[bstr]` rather than the opaque `any` fallback
// below, which would emit a bare QVariant into the Qt-free TU. As
// `[bstr]` it goes through the cdylib list codec
// (lidlBytesListFromJson / lidlBytesListToJson), so each element keeps
// the canonical tagged {"_bytes": base64url} form on the wire.
if (inner == "std::vector<uint8_t>") {
TypeExpr elem = { TypeExpr::Primitive, "bstr", {} };
return { TypeExpr::Array, "", { elem } };
}
if (inner == "int64_t") {
TypeExpr elem = { TypeExpr::Primitive, "int", {} };
return { TypeExpr::Array, "", { elem } };
}
if (inner == "uint64_t") {
TypeExpr elem = { TypeExpr::Primitive, "uint", {} };
return { TypeExpr::Array, "", { elem } };
}
if (inner == "double") {
TypeExpr elem = { TypeExpr::Primitive, "float64", {} };
return { TypeExpr::Array, "", { elem } };
}
if (inner == "bool") {
TypeExpr elem = { TypeExpr::Primitive, "bool", {} };
return { TypeExpr::Array, "", { elem } };
}
// Anything else: recurse. That is what makes `std::vector<Blob>` a
// [Blob] and `std::vector<std::map<std::string, int64_t>>` a
// [{tstr: int}]. Without it the element list above was exhaustive and
// every other vector fell all the way through to the opaque `any`,
// which then encoded a record as a LogosMap.
return { TypeExpr::Array, "", { cppTypeToLidl(inner) } };
}
// Qt collection types — pass through directly (non-std-convertible)
if (t == "QVariantMap")
return { TypeExpr::Map, "", { {TypeExpr::Primitive, "tstr", {}}, {TypeExpr::Primitive, "any", {}} } };
if (t == "QVariantList")
return { TypeExpr::Array, "", { {TypeExpr::Primitive, "any", {}} } };
if (t == "QStringList")
return { TypeExpr::Array, "", { {TypeExpr::Primitive, "tstr", {}} } };
// LogosMap / LogosList — nlohmann::json aliases; same LIDL shape as the Qt types
// but flagged so the generator emits an nlohmann→Qt conversion in the glue.
if (t == "LogosMap")
return { TypeExpr::Map, "", { {TypeExpr::Primitive, "tstr", {}}, {TypeExpr::Primitive, "any", {}} } };
if (t == "LogosList")
return { TypeExpr::Array, "", { {TypeExpr::Primitive, "any", {}} } };
// StdLogosResult — pure C++ result type for universal impls. The generator
// emits a StdLogosResult→Qt LogosResult conversion in the glue layer.
if (t == "StdLogosResult")
return { TypeExpr::Primitive, "result", {} };
// std::map<std::string, T> -> {tstr: T}. Absent before, so a typed map was
// unspellable header-first and fell through to `any`.
static QRegularExpression mapRe("^std::map\\s*<\\s*std::string\\s*,\\s*(.+)\\s*>$");
QRegularExpressionMatch mm = mapRe.match(t);
if (mm.hasMatch()) {
TypeExpr val = cppTypeToLidl(mm.captured(1).trimmed());
return { TypeExpr::Map, "", { {TypeExpr::Primitive, "tstr", {}}, val } };
}
// A record the header declared. Checked LAST so it can never shadow a
// builtin spelling, and gated on the declared set so an unknown type keeps
// the historical `any` fallback rather than naming a struct nobody emits.
if (g_recordNames.contains(t))
return { TypeExpr::Named, t.toStdString(), {} };
// Fallback: treat as opaque
return { TypeExpr::Primitive, "any", {} };
}
// Find `struct Name { Type field; ... };` blocks and turn them into `type`
// declarations.
//
// The parser used to SKIP any line starting with `struct`, which meant a record
// could not be declared header-first at all — the only way to get one was a
// hand-written .lidl. Worse, a method mentioning the struct still parsed: its
// type fell through to the opaque `any`, so the contract silently disagreed
// with the header.
static std::vector<TypeDecl> scanForRecords(const QStringList& lines)
{
static QRegularExpression openRe("^struct\\s+(\\w+)\\s*\\{\\s*$");
static QRegularExpression fieldRe("^([\\w:<>,\\s\\*]+?)\\s+(\\w+)\\s*(=[^;]*)?;$");
// TWO passes. A record field may name another record (`Blob inner;` inside
// Wrapper), and cppTypeToLidl only answers Named() for a name already in
// g_recordNames — so every struct name has to be registered before any
// field is typed. One pass silently typed such a field as `any`, and the
// generated codec then tried to encode a Blob as a LogosMap.
for (int i = 0; i < lines.size(); ++i) {
QRegularExpressionMatch om = openRe.match(lines.at(i).trimmed());
if (om.hasMatch())
g_recordNames.insert(om.captured(1));
}
std::vector<TypeDecl> out;
for (int i = 0; i < lines.size(); ++i) {
const QString line = lines.at(i).trimmed();
QRegularExpressionMatch om = openRe.match(line);
if (!om.hasMatch()) continue;
TypeDecl td;
td.name = om.captured(1).toStdString();
for (int j = i + 1; j < lines.size(); ++j) {
const QString body = lines.at(j).trimmed();
if (body.startsWith("};")) break;
if (body.isEmpty() || body.startsWith("//")) continue;
// Strip a trailing line comment before matching: a field written
// `std::string name; // what it is` does not end in ';' and was
// silently DROPPED, publishing a record with a partial field list —
// the worst kind of wrong, because it looks like a contract.
QString field = body;
const int comment = field.indexOf("//");
if (comment >= 0) field = field.left(comment).trimmed();
if (field.isEmpty()) continue;
QRegularExpressionMatch fm = fieldRe.match(field);
if (!fm.hasMatch()) continue;
FieldDecl fd;
fd.name = fm.captured(2).toStdString();
fd.type = cppTypeToLidl(fm.captured(1).trimmed());
td.fields.push_back(fd);
}
if (!td.fields.empty()) out.push_back(td);
}
return out;
}
// Keep only the records the module's API actually mentions.
//
// An impl header routinely declares helper structs that are none of a
// consumer's business — `struct PendingAction` inside the class, a
// `struct ModuleSource` next to it. Publishing every struct as a contract
// `type` changes the module's PUBLISHED interface as a side effect of an
// internal refactor, which is not something deriving a contract from a header
// is allowed to do. A struct earns its place in the contract by appearing in a
// method or event signature — transitively, since a published record's own
// fields may name others.
static void keepOnlyReferencedRecords(ModuleDecl& module)
{
auto mention = [](const TypeExpr& te, std::set<std::string>& out) {
std::function<void(const TypeExpr&)> walk = [&](const TypeExpr& t) {
if (t.kind == TypeExpr::Named) out.insert(t.name);
for (const TypeExpr& e : t.elements) walk(e);
};
walk(te);
};
std::set<std::string> referenced;
for (const MethodDecl& md : module.methods) {
mention(md.returnType, referenced);
for (const ParamDecl& pd : md.params) mention(pd.type, referenced);
}
for (const EventDecl& ed : module.events)
for (const ParamDecl& pd : ed.params) mention(pd.type, referenced);
// Transitive closure: a referenced record's fields may name more records.
bool grew = true;
while (grew) {
grew = false;
for (const TypeDecl& td : module.types) {
if (!referenced.count(td.name)) continue;
for (const FieldDecl& fd : td.fields) {
std::set<std::string> here;
mention(fd.type, here);
for (const std::string& n : here)
if (referenced.insert(n).second) grew = true;
}
}
}
std::vector<TypeDecl> kept;
for (const TypeDecl& td : module.types)
if (referenced.count(td.name)) kept.push_back(td);
module.types = std::move(kept);
}
// ---------------------------------------------------------------------------
// Parse a single method declaration line
// ---------------------------------------------------------------------------
static bool parseMethodLine(const QString& line, MethodDecl& out)
{
// Find the parameter list: everything between the last '(' and ')'
int parenOpen = -1;
int parenClose = -1;
int depth = 0;
for (int i = line.size() - 1; i >= 0; --i) {
if (line[i] == ')') {
if (parenClose < 0) parenClose = i;
depth++;
} else if (line[i] == '(') {
depth--;
if (depth == 0) {
parenOpen = i;
break;
}
}
}
if (parenOpen < 0 || parenClose < 0)
return false;
QString paramStr = line.mid(parenOpen + 1, parenClose - parenOpen - 1).trimmed();
// Everything before '(' is "returnType methodName"
QString prefix = line.left(parenOpen).trimmed();
// The method name is the last identifier token in prefix
int nameEnd = prefix.size();
while (nameEnd > 0 && prefix[nameEnd - 1].isSpace())
nameEnd--;
int nameStart = nameEnd;
while (nameStart > 0 && (prefix[nameStart - 1].isLetterOrNumber() || prefix[nameStart - 1] == '_'))
nameStart--;
if (nameStart >= nameEnd)
return false;
const QString methodName = prefix.mid(nameStart, nameEnd - nameStart);
// Reject if the extracted name is a C++ keyword — this filters out
// member variable declarations like "std::function<void(...)> onEvent"
// where the parser would mistakenly extract "void" as the method name.
static const QSet<QString> cppKeywords = {
"void", "int", "bool", "char", "short", "long", "double", "float",
"auto", "return", "if", "else", "for", "while", "do", "switch",
"case", "break", "continue", "const", "static", "inline", "virtual"
};
if (cppKeywords.contains(methodName))
return false;
out.name = methodName.toStdString();
QString retTypeStr = stripDeclarationSpecifiers(prefix.left(nameStart).trimmed());
out.returnType = cppTypeToLidl(retTypeStr);
// Flag methods whose impl returns LogosMap / LogosList so the generator
// can emit nlohmann→Qt conversion code in the glue layer.
out.jsonReturn = (retTypeStr == "LogosMap" || retTypeStr == "LogosList");
// Flag methods whose impl returns StdLogosResult so the generator can
// emit a StdLogosResult→Qt LogosResult conversion in the glue layer.
out.resultReturn = (retTypeStr == "StdLogosResult");
// Parse parameters
out.params.clear();
if (!paramStr.isEmpty()) {
// Split by comma, respecting template depth
QStringList parts;
int start = 0;
int tdepth = 0;
for (int i = 0; i < paramStr.size(); ++i) {
if (paramStr[i] == '<') tdepth++;
else if (paramStr[i] == '>') tdepth--;
else if (paramStr[i] == ',' && tdepth == 0) {
parts.append(paramStr.mid(start, i - start).trimmed());
start = i + 1;
}
}
parts.append(paramStr.mid(start).trimmed());
for (const QString& part : parts) {
if (part.isEmpty()) continue;
QString p = part.trimmed();
int pNameEnd = p.size();
while (pNameEnd > 0 && p[pNameEnd - 1].isSpace())
pNameEnd--;
int pNameStart = pNameEnd;
while (pNameStart > 0 && (p[pNameStart - 1].isLetterOrNumber() || p[pNameStart - 1] == '_'))
pNameStart--;
if (pNameStart >= pNameEnd) continue;
ParamDecl pd;
pd.name = p.mid(pNameStart, pNameEnd - pNameStart).toStdString();
pd.type = cppTypeToLidl(p.left(pNameStart));
out.params.push_back(pd);
}
}
return true;
}
// ---------------------------------------------------------------------------
// Main entry point
// ---------------------------------------------------------------------------
// Join doc-comment lines preserving line breaks (drop leading/trailing blanks).
static QString joinDocLines(QStringList lines)
{
while (!lines.isEmpty() && lines.first().trimmed().isEmpty()) lines.removeFirst();
while (!lines.isEmpty() && lines.last().trimmed().isEmpty()) lines.removeLast();
return lines.join('\n');
}
ImplParseResult parseImplHeader(const QString& headerPath,
const QString& className,
const QString& metadataPath,
QTextStream& err)
{
ImplParseResult result;
// --- Read metadata.json ---
{
QFile mf(metadataPath);
if (!mf.open(QIODevice::ReadOnly | QIODevice::Text)) {
result.error = "Failed to open metadata file: " + metadataPath;
return result;
}
QJsonParseError pe;
QJsonDocument doc = QJsonDocument::fromJson(mf.readAll(), &pe);
if (pe.error != QJsonParseError::NoError) {
result.error = "Failed to parse metadata JSON: " + pe.errorString();
return result;
}
QJsonObject obj = doc.object();
result.module.name = obj.value("name").toString().toStdString();
result.module.version = obj.value("version").toString().toStdString();
result.module.description = obj.value("description").toString().toStdString();
result.module.category = obj.value("category").toString().toStdString();
QJsonArray deps = obj.value("dependencies").toArray();
for (const QJsonValue& v : deps)
result.module.depends.push_back(v.toString().toStdString());
// Read events declared in metadata.json
QJsonArray events = obj.value("events").toArray();
for (const QJsonValue& ev : events) {
QJsonObject evObj = ev.toObject();
EventDecl ed;
ed.name = evObj.value("name").toString().toStdString();
ed.description = evObj.value("description").toString().toStdString();
QJsonArray params = evObj.value("params").toArray();
for (const QJsonValue& pv : params) {
QJsonObject po = pv.toObject();
ParamDecl pd;
pd.name = po.value("name").toString().toStdString();
pd.type = cppTypeToLidl(po.value("type").toString());
ed.params.push_back(pd);
}
if (!ed.name.empty())
result.module.events.push_back(ed);
}
}
// --- Read and parse header ---
QFile hf(headerPath);
if (!hf.open(QIODevice::ReadOnly | QIODevice::Text)) {
result.error = "Failed to open header file: " + headerPath;
return result;
}
QString source = QString::fromUtf8(hf.readAll());
hf.close();
// Records first: cppTypeToLidl consults the declared set, so the structs
// have to be known before a single signature is looked at.
// Split into physical lines, then merge any whose parentheses are still
// open into one logical line. The scanner below is line-based — it only
// accepts a method when a single trimmed line ends in ';' and
// parseMethodLine finds a balanced '(...)' on it — so without this a method
// signature wrapped across several physical lines is silently dropped.
// Parens inside comments / string / char literals are ignored.
QStringList lines;
{
const QStringList physical = source.split('\n');
QString acc;
int parenDepth = 0;
bool inBlockComment = false;
for (const QString& phys : physical) {
bool inStr = false;
bool inChr = false;
for (int i = 0; i < phys.size(); ++i) {
const QChar c = phys[i];
const QChar n = (i + 1 < phys.size()) ? phys[i + 1] : QChar();
if (inBlockComment) {
if (c == '*' && n == '/') { inBlockComment = false; ++i; }
} else if (inStr) {
if (c == '\\') ++i; else if (c == '"') inStr = false;
} else if (inChr) {
if (c == '\\') ++i; else if (c == '\'') inChr = false;
} else if (c == '/' && n == '*') {
inBlockComment = true; ++i;
} else if (c == '/' && n == '/') {
break;
} else if (c == '"') {
inStr = true;
} else if (c == '\'') {
inChr = true;
} else if (c == '(') {
++parenDepth;
} else if (c == ')') {
if (parenDepth > 0) --parenDepth;
}
}
if (acc.isEmpty())
acc = phys;
else
acc += ' ' + phys.trimmed();
if (parenDepth <= 0) {
lines.append(acc);
acc.clear();
}
}
if (!acc.isEmpty())
lines.append(acc);
}
// Records, before any signature is examined: cppTypeToLidl() consults the
// declared set, so a `Blob` parameter only becomes Named("Blob") once the
// struct has been seen. Reset per parse — the set is file-static and a
// single process generates for more than one module.
g_recordNames.clear();
result.module.types = scanForRecords(lines);
// State machine: find "class <className>", then collect declarations.
// `InLogosEvents` is entered by the literal `logos_events:` token
// (mirrors Qt's `signals:`) — methods declared there are parsed as
// EventDecls and appended to ModuleDecl.events instead of .methods.
enum State { LookingForClass, InClass, InPublic, InPrivate, InLogosEvents };
State state = LookingForClass;
int braceDepth = 0;
// Accumulates doc-comment lines adjacent to a method so the doc comment
// becomes the method's description. Reset on any blank / non-comment line.
QStringList pendingDoc;
bool inBlockComment = false;
QRegularExpression classRe("\\bclass\\s+" + QRegularExpression::escape(className) + "\\b");
QRegularExpression accessRe("^\\s*(public|private|protected)\\s*:");
QRegularExpression eventsRe("^\\s*logos_events\\s*:");
QRegularExpression ctorDtorRe("^\\s*~?" + QRegularExpression::escape(className) + "\\s*\\(");
for (const QString& rawLine : lines) {
QString line = rawLine.trimmed();
switch (state) {
case LookingForClass:
if (classRe.match(line).hasMatch()) {
state = InClass;
for (QChar c : line) {
if (c == '{') braceDepth++;
else if (c == '}') braceDepth--;
}
}
break;
case InClass:
case InPublic:
case InPrivate:
case InLogosEvents:
// Inside a multi-line /** ... */ doc-comment block: capture its
// text (skip brace counting — comments don't affect scope).
if (inBlockComment) {
QString t = line;
int end = t.indexOf("*/");
if (end >= 0) { t = t.left(end); inBlockComment = false; }
t.remove(QRegularExpression(R"(^\*+\s?)"));
t = t.trimmed();
pendingDoc.append(t);
break;
}
// Count braces only on real code lines. Braces inside a doc/line
// comment (e.g. `/// returns { "k": v }`) must not affect scope
// tracking, or an unbalanced brace in a comment would make the
// parser think the class ended early and drop later declarations.
if (!(line.startsWith("//") || line.startsWith("/*") || line.startsWith("*"))) {
for (QChar c : line) {
if (c == '{') braceDepth++;
else if (c == '}') braceDepth--;
}
if (braceDepth <= 0) {
state = LookingForClass;
goto done;
}
}
// A section specifier may be followed by a declaration on the
// *same* physical line — e.g. clang-format / prettier collapse
// logos_events:
// void versionReady(const std::string& version);
// into `logos_events : void versionReady(const std::string& version);`.
// Strip any leading specifiers, updating the section state, and
// let whatever remains fall through to the declaration parser
// below — otherwise everything after the colon is discarded and
// the same valid C++ is parsed differently based on formatting.
//
// `logos_events:` takes precedence over the standard access
// specifiers: it's a separate section that the codegen pulls
// event prototypes from. (At preprocess time, `logos_events`
// expands to `public`, but the raw source still carries the
// token we recognise here.)
bool specifierStripped = false;
while (true) {
QRegularExpressionMatch em = eventsRe.match(line);
if (em.hasMatch()) {
state = InLogosEvents;
line = line.mid(em.capturedEnd()).trimmed();
specifierStripped = true;
continue;
}
QRegularExpressionMatch am = accessRe.match(line);
if (am.hasMatch()) {
QString spec = am.captured(1);
if (spec == "public") state = InPublic;
else state = InPrivate;
line = line.mid(am.capturedEnd()).trimmed();
specifierStripped = true;
continue;
}
break;
}
// A *bare* specifier (nothing after the colon) is a section
// boundary and resets any pending doc-comment, mirroring Qt's
// `signals:`. But when a declaration shares the line, the doc
// comment preceding the whole line must still attach to that
// declaration — otherwise documentation, like the declaration
// itself (#76), would become formatting-dependent. So only clear
// here for the bare form; the same-line form keeps pendingDoc and
// attaches it in the declaration parser below.
if (specifierStripped && line.isEmpty())
pendingDoc.clear();
// Only doc comments (/// or /** ... */ / /*! ... */) accumulate as
// the pending description for the next method. Plain // and /*
// comments are ignored but leave pending doc intact; blank /
// preprocessor lines reset it so only *adjacent* comments attach.
if (line.startsWith("///")) {
QString text = line.mid(3);
if (text.startsWith('<')) text = text.mid(1); // ///< trailing form
text = text.trimmed();
pendingDoc.append(text);
break;
}
if (line.startsWith("/**") || line.startsWith("/*!")) {
QString text = line.mid(3);
int end = text.indexOf("*/");
if (end >= 0) text = text.left(end);
else inBlockComment = true;
text.remove(QRegularExpression(R"(^\*+\s?)"));
text = text.trimmed();
pendingDoc.append(text);
break;
}
if (line.startsWith("//") || line.startsWith("/*") || line.startsWith("*")) {
break; // non-doc comment: ignore, keep pending doc
}
if (line.isEmpty() || line.startsWith("#")) {
pendingDoc.clear();
break;
}
if (ctorDtorRe.match(line).hasMatch()) {
pendingDoc.clear();
break;
}
if (line.startsWith("typedef") || line.startsWith("using")
|| line.startsWith("friend") || line.startsWith("enum")
|| line.startsWith("struct")) {
pendingDoc.clear();
break;
}
if (state == InLogosEvents) {
// Inside `logos_events:` — every bare prototype is an event.
// Events are always void-returning by definition, so we
// re-use parseMethodLine to extract name + params and
// discard the return type.
if (line.endsWith(';')) {
QString decl = line.left(line.size() - 1).trimmed();
MethodDecl md;
if (parseMethodLine(decl, md)) {
EventDecl ed;
ed.name = md.name;
ed.params = md.params;
ed.description = joinDocLines(pendingDoc).toStdString();
result.module.events.push_back(ed);
}
}
pendingDoc.clear();
break;
}
if (state != InPublic) { pendingDoc.clear(); break; }
if (line.contains("std::function<")) {
// A std::function member is not a method — skip it so the
// `parseMethodLine` path below doesn't choke on the nested
// parens in its type. (Events are declared in a typed
// `logos_events:` section, parsed above — there is no longer
// any special `std::function emitEvent` member to detect.)
pendingDoc.clear();
break;
}
if (line.endsWith(';')) {
QString decl = line.left(line.size() - 1).trimmed();
MethodDecl md;
if (parseMethodLine(decl, md)) {
// LogosModuleContext lifecycle hooks / context accessors are
// framework plumbing, not part of the module's API contract.
// An impl commonly overrides `onContextReady()` (and could
// re-declare an accessor) in its own public section, so the
// header parser would otherwise emit them into the derived
// LIDL — breaking cdylib eligibility (e.g. the inherited
// accessors' Qt-free-subset check) and exposing non-API
// methods. Skip the reserved names regardless of access.
static const QSet<QString> reserved = {
"onContextReady", "modules", "modulePath",
"instanceId", "instancePersistencePath"
};
if (!reserved.contains(qs(md.name))) {
md.description = joinDocLines(pendingDoc).toStdString();
result.module.methods.push_back(md);
}
}
}
pendingDoc.clear();
break;
}
}
done:
// Now that every signature is known, drop the structs the API never
// mentions — a header's internal helpers must not become published
// contract types.
keepOnlyReferencedRecords(result.module);
if (result.module.methods.empty()) {
err << "Warning: no public methods found in class " << className
<< " in " << headerPath << "\n";
}
return result;
}