mirror of
https://github.com/logos-co/logos-cpp-sdk.git
synced 2026-08-30 09:11:13 +00:00
* feat(optional): ?T is two-state, and the generators finally read it
No generator in any language read the optional flag — it had never been
implemented. `?T` was a HARD REJECT on the cdylib backend ("module not
cdylib-eligible"), `std::optional<T>` in an impl header fell through to the
opaque `any` with no diagnostic, and a `? name: T` field was emitted as a
required `T`. Three real contracts in the workspace already declare optionals
and were silently getting one of those three answers.
`?T` is TWO-state: a value of T, or empty. Never three — "one LIDL type <-> one
type per language" leaves nowhere for a third state, because every target has
exactly one empty inhabitant.
ONE MEANING, TWO SPELLINGS. `? name: T` (the field flag) and `name: ?T` (the
type kind) are the same declaration. Backends no longer answer that themselves:
logos-lidl's fieldIsOptional/fieldValueType are re-exported from lidl_compat.h
and every site THIS COMMIT TOUCHES reads them, so the two spellings emit
byte-identical code on the cdylib and client backends. That
caught a live drift on the way in — lidlRecordCollidesWithBytesTag read `f.type`
and so refused `? _bytes: tstr` while letting `_bytes: ?tstr` straight through,
one declaration with two answers.
THE WIRE RULE DEPENDS ON THE SLOT. Absent and explicit null are the SAME state
on decode and DIFFERENT on encode:
- decode is liberal, by exactly one inhabitant: in an optional slot absent and
null both mean empty; in a required slot both stay errors. A present value
goes through the decoder a required T would get, so a wrong type still fails
at the same path — optional widens the domain, it does not switch checking
off. `?bstr` therefore keeps the LENIENT bytes decode a bare `bstr` gets,
rather than silently becoming stricter in the optional slot.
- encode has one canonical form: empty OMITS the key where the slot is NAMED
(a record field) and is spelled null where it is POSITIONAL (an argument, a
return, an event parameter — no key to omit, and arity must not change). Key
omission lives in the record emitter because a Codec only ever sees a value,
never the slot it sits in. A round trip therefore canonicalises.
- `?any` collapses onto `any`: nlohmann::json already carries null, so
std::optional<LogosMap> would give the slot two spellings of empty.
The dispatch gate now admits a missing trailing optional argument and
materialises it as null, exactly the way a missing record field already was. A
method with no optional parameter emits the byte-identical gate it always did.
Header-first: `std::optional<T>` <-> `?T`, composing with records and
containers. `std::optional<std::optional<T>>` has NO LIDL type (three C++ states
over a two-state wire), so it maps down to `?T` — which makes the author's own
declaration stop compiling against the generated codec, deliberately — and says
so at derivation time instead of leaving a conversion error in generated code.
The Qt/Lp consumer surface is NOT fixed and does not pretend to be. The wrappers
real modules get come from legacy/main.cpp, where the AST is flattened to a
single Qt type-name string per slot before optionality could be seen; Qt has no
optional metatype, so `?T` lands on QVariant — the right shape (an invalid
QVariant is Qt's empty inhabitant) with no type. The generator now prints a Note
naming every flattened slot so an affected build is never silent, and
docs/project.md records exactly what a Qt consumer will still do with an
optional field.
Verified by output equivalence, not by a green build: the generator was built
before and after and run over every .lidl in the workspace plus the impl-header
fixtures, in cdylib, consumer-qt, consumer-lp, client and header-first modes.
428 of 465 artefacts are byte-identical; all 37 that differ belong to one of the
four contracts that declare an optional (the 38th path is the manifest). The
harness's sensitivity is pinned by a negative control: qt vs lp output differs
in 45 files. The emitted codec was additionally compiled under -Wall -Wextra and
run against the rules above — omission, absent==null, required-still-rejects,
present-but-wrong-still-fails, and canonicalising round trip.
Tests: 199 pass, 0 fail (180 before, 19 new).
Requires logos-lidl's optionality accessors and logos-protocol's
Codec<std::optional<T>>.
NOT FIXED, AND IT IS THE PATH THAT MATTERS MOST. The legacy interface-wrapper
path is untouched, and it is the one every real module builds through
(buildPlugin.nix:145 -> logos-cpp-generator --general-only). There the two
spellings still diverge:
? maybe: tstr -> QString maybe{}; __m.value("maybe").toString()
maybe: ?tstr -> QVariant maybe{}; __m.value("maybe")
and --api-style lp diverges too, neither side being std::optional. So R3 holds
on the backends below and NOT on the Qt consumer a shipping module actually
gets. logos-chat-module -- the contract that prompted this work -- uses the
field-flag spelling, so it lands on the branch that silently defaults.
The cause is upstream of codegen: legacy/main.cpp's moduleRecordsToJson and
moduleMethodsToJson flatten every TypeExpr to a single Qt TYPE-NAME STRING, so
optionality (along with nesting, map key types and descriptions) is gone before
generator_lib.cpp sees it. Widening that interface is a larger change and is
deliberately not attempted here. The only R3 test on a Qt surface covers
lidl_gen_client.cpp, which is on no live build path.
* chore: re-pin logos-lidl to master for the optionality accessors
lidl_compat.h re-exports typeIsOptional / optionalValueType / fieldIsOptional /
fieldValueType / paramIsOptional / paramValueType, which landed in
logos-lidl#7. The pinned lidl predated it, so CI failed to compile.
logos-lidl 8c95d4f -> 35f33d8. Tests: 199 pass, 0 fail.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* chore: re-pin logos-protocol to master for Codec<std::optional<T>>
The generated record codecs emit Codec<std::optional<T>> for an optional
field; that specialisation landed in logos-protocol#37 and the pinned
protocol predated it.
Note this repo's own tests would NOT have caught the omission -- the
generator tests string-assert emitted text rather than compiling it, so a
missing codec specialisation only surfaces when a real module compiles
generated optional code (logos-test-modules' ext provider).
logos-protocol 4359557 -> 72754ab. Tests: 199 pass, 0 fail.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* test(doctests): override logos-lidl alongside every logos-cpp-sdk override
The doc-tests build downstream repos (logoscore-cli, capability_module,
accounts_module) with --override-input logos-cpp-sdk. Nix does not carry the
overridden input's OWN lock, so those builds got this branch's cpp-sdk source
while still resolving logos-lidl from their own, older locks. The shipped
share/lidl-frontend/lidl_compat.h then calls accessors that lidl does not
have:
lidl_compat.h:46: error: 'paramValueType' has not been declared in 'lidl'
lidl_compat.h:92: error: 'fieldValueType' was not declared in this scope
Every --override-input logos-cpp-sdk now has a matching
--override-input <same-path>/logos-cpp-sdk/logos-lidl.
This is specific to the override path. A normal consumer running
'nix flake update logos-cpp-sdk' inherits cpp-sdk's own lock, which pins the
lidl carrying these accessors, and is unaffected.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* test(doctests): move logos-lidl at the qt-sdk nodes, not under logos-cpp-sdk
The doc-tests failed to build logos-qt-generator:
share/lidl-frontend/lidl_compat.h:46: error: 'paramValueType' has not been
declared in 'lidl'
MECHANISM. This SDK installs cpp-generator/experimental/lidl_compat.h into
$out/share/lidl-frontend/, and logos-qt-sdk's logos-qt-generator *compiles*
that installed header against qt-sdk's OWN logos-lidl input. Under
logos-qt-sdk, logos-lidl is a SIBLING of logos-cpp-sdk, not a descendant:
logos-qt-sdk
|-- logos-cpp-sdk <- --override-input moves this to the commit under test
`-- logos-lidl <- stays on qt-sdk's lock (8c95d4f), lacks the accessors
logos-logoscore-cli and logos-module-builder both declare
`logos-qt-sdk.inputs.logos-cpp-sdk.follows = "logos-cpp-sdk"` but no lidl
follows, so overriding the SDK hands qt-sdk a new lidl_compat.h next to its
old lidl. The failing derivation is logos-qt-generator — not anything in
logos-cpp-sdk, which is why the previous attempt aimed at the wrong node.
THE FIX is one `<path-to-logos-qt-sdk>/logos-lidl` override per qt-sdk node
that ends up on the SDK under test. A tree-walk over the resolved lock found
four in logoscore-cli's closure and two per module build; with the overrides
applied the walk reports zero remaining.
WHAT WAS REMOVED, and why it was doing nothing:
* The `.../logos-cpp-sdk/logos-lidl` overrides added in bef3ef5 were no-ops.
With only `--override-input logos-cpp-sdk <sha>`, that node's logos-lidl
already resolves to 35f33d87 out of cpp-sdk's own lock — nix >= 2.26
carries an overridden input's lock, and CI runs Determinate Nix. Verified
by resolving the lock with and without them: byte-identical.
* The `logos-module-client/...` overrides never matched anything. Nix says so
out loud ("does not match any input"): logoscore-cli has no such root
input; module-client only appears under logos-test-modules/, outside the
runtime closure. The prose claiming it pins the SDK is corrected too.
cpp-sdk-concurrent-dispatch is fixed here as well — it failed the same way and
carried no lidl overrides at all.
VERIFIED locally against bef3ef5, the exact commit CI failed on:
* accounts .lgx -> exit 0, logos-accounts_module-module-lib.lgx (5,939,898 B)
* logoscore CLI -> exit 0, ./logos/bin/logoscore reports
"logos-cpp-sdk bef3ef57d3f489073672e70a786c550df7edd003"
* negative control (same command minus the single qt-sdk lidl flag) fails
with CI's exact derivation,
/nix/store/pf96n2ldvhy6sq39ygkh5zdqx7dcn4df-logos-qt-generator-0.1.0.drv
* no "does not match any input" warnings remain on any command
The durable fix is a one-line bump of logos-qt-sdk's own flake.lock logos-lidl
to master (logos-lidl#7 is purely additive: six new inline helpers, nothing
removed or renamed). Once qt-sdk carries it, every override added here can go.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
765 lines
33 KiB
C++
765 lines
33 KiB
C++
#include "impl_header_parser.h"
|
|
|
|
#include "metadata_dependencies.h"
|
|
|
|
#include <QFile>
|
|
#include <QFileInfo>
|
|
#include <QJsonDocument>
|
|
#include <QJsonObject>
|
|
#include <QJsonArray>
|
|
#include <QRegularExpression>
|
|
#include <QSet>
|
|
|
|
#include <functional>
|
|
#include <set>
|
|
#include <QStringList>
|
|
|
|
// ---------------------------------------------------------------------------
|
|
// Strip leading declaration specifiers / attributes from a return-type string.
|
|
// ---------------------------------------------------------------------------
|
|
|
|
static QString stripDeclarationSpecifiers(QString string)
|
|
{
|
|
static const QRegularExpression attributeRe("\\[\\[[^\\]]*\\]\\]");
|
|
static const QRegularExpression specifierRe(
|
|
"^(static|virtual|inline|explicit|constexpr|consteval|friend)\\s+");
|
|
|
|
string.remove(attributeRe);
|
|
string = string.trimmed();
|
|
|
|
QRegularExpressionMatch specifierMatch = specifierRe.match(string);
|
|
while (specifierMatch.hasMatch()) {
|
|
string = string.mid(specifierMatch.capturedLength()).trimmed();
|
|
specifierMatch = specifierRe.match(string);
|
|
}
|
|
|
|
return string;
|
|
}
|
|
|
|
// ---------------------------------------------------------------------------
|
|
// C++ type string → LIDL TypeExpr
|
|
// ---------------------------------------------------------------------------
|
|
|
|
// The records the header declares, discovered by scanForRecords() before any
|
|
// method is parsed. A bare `Blob` in a signature is only a record if the header
|
|
// actually declared `struct Blob { ... };` — otherwise it stays the opaque
|
|
// `any` it always was.
|
|
static QSet<QString> g_recordNames;
|
|
|
|
// C++ spellings seen that have NO LIDL type, and were mapped onto the nearest
|
|
// one that does. Collected here because cppTypeToLidl has no diagnostic channel
|
|
// (same reason g_recordNames is file-static); parseImplHeader drains it.
|
|
static QStringList g_unmappableSpellings;
|
|
|
|
static TypeExpr cppTypeToLidl(const QString& raw)
|
|
{
|
|
// Normalize: strip const, &, leading/trailing whitespace
|
|
QString t = raw.trimmed();
|
|
t.remove(QRegularExpression("^const\\s+"));
|
|
t.remove(QRegularExpression("\\s*&$"));
|
|
t = t.trimmed();
|
|
|
|
// Primitives
|
|
if (t == "bool") return { TypeExpr::Primitive, "bool", {} };
|
|
if (t == "int64_t") return { TypeExpr::Primitive, "int", {} };
|
|
if (t == "uint64_t") return { TypeExpr::Primitive, "uint", {} };
|
|
if (t == "double") return { TypeExpr::Primitive, "float64", {} };
|
|
if (t == "void") return { TypeExpr::Primitive, "void", {} };
|
|
|
|
// std::string
|
|
if (t == "std::string")
|
|
return { TypeExpr::Primitive, "tstr", {} };
|
|
|
|
// std::vector<T>
|
|
static QRegularExpression vecRe("^std::vector\\s*<\\s*(.+)\\s*>$");
|
|
QRegularExpressionMatch m = vecRe.match(t);
|
|
if (m.hasMatch()) {
|
|
QString inner = m.captured(1).trimmed();
|
|
if (inner == "std::string") {
|
|
TypeExpr elem = { TypeExpr::Primitive, "tstr", {} };
|
|
return { TypeExpr::Array, "", { elem } };
|
|
}
|
|
if (inner == "uint8_t") {
|
|
return { TypeExpr::Primitive, "bstr", {} };
|
|
}
|
|
// std::vector<std::vector<uint8_t>> — an array of byte strings. Spelled
|
|
// out so it lands on `[bstr]` rather than the opaque `any` fallback
|
|
// below, which would emit a bare QVariant into the Qt-free TU. As
|
|
// `[bstr]` it goes through the cdylib list codec
|
|
// (lidlBytesListFromJson / lidlBytesListToJson), so each element keeps
|
|
// the canonical tagged {"_bytes": base64url} form on the wire.
|
|
if (inner == "std::vector<uint8_t>") {
|
|
TypeExpr elem = { TypeExpr::Primitive, "bstr", {} };
|
|
return { TypeExpr::Array, "", { elem } };
|
|
}
|
|
if (inner == "int64_t") {
|
|
TypeExpr elem = { TypeExpr::Primitive, "int", {} };
|
|
return { TypeExpr::Array, "", { elem } };
|
|
}
|
|
if (inner == "uint64_t") {
|
|
TypeExpr elem = { TypeExpr::Primitive, "uint", {} };
|
|
return { TypeExpr::Array, "", { elem } };
|
|
}
|
|
if (inner == "double") {
|
|
TypeExpr elem = { TypeExpr::Primitive, "float64", {} };
|
|
return { TypeExpr::Array, "", { elem } };
|
|
}
|
|
if (inner == "bool") {
|
|
TypeExpr elem = { TypeExpr::Primitive, "bool", {} };
|
|
return { TypeExpr::Array, "", { elem } };
|
|
}
|
|
// Anything else: recurse. That is what makes `std::vector<Blob>` a
|
|
// [Blob] and `std::vector<std::map<std::string, int64_t>>` a
|
|
// [{tstr: int}]. Without it the element list above was exhaustive and
|
|
// every other vector fell all the way through to the opaque `any`,
|
|
// which then encoded a record as a LogosMap.
|
|
return { TypeExpr::Array, "", { cppTypeToLidl(inner) } };
|
|
}
|
|
|
|
// Qt collection types — pass through directly (non-std-convertible)
|
|
if (t == "QVariantMap")
|
|
return { TypeExpr::Map, "", { {TypeExpr::Primitive, "tstr", {}}, {TypeExpr::Primitive, "any", {}} } };
|
|
if (t == "QVariantList")
|
|
return { TypeExpr::Array, "", { {TypeExpr::Primitive, "any", {}} } };
|
|
if (t == "QStringList")
|
|
return { TypeExpr::Array, "", { {TypeExpr::Primitive, "tstr", {}} } };
|
|
|
|
// LogosMap / LogosList — nlohmann::json aliases; same LIDL shape as the Qt types
|
|
// but flagged so the generator emits an nlohmann→Qt conversion in the glue.
|
|
if (t == "LogosMap")
|
|
return { TypeExpr::Map, "", { {TypeExpr::Primitive, "tstr", {}}, {TypeExpr::Primitive, "any", {}} } };
|
|
if (t == "LogosList")
|
|
return { TypeExpr::Array, "", { {TypeExpr::Primitive, "any", {}} } };
|
|
|
|
// StdLogosResult — pure C++ result type for universal impls. The generator
|
|
// emits a StdLogosResult→Qt LogosResult conversion in the glue layer.
|
|
if (t == "StdLogosResult")
|
|
return { TypeExpr::Primitive, "result", {} };
|
|
|
|
// std::map<std::string, T> -> {tstr: T}. Absent before, so a typed map was
|
|
// unspellable header-first and fell through to `any`.
|
|
static QRegularExpression mapRe("^std::map\\s*<\\s*std::string\\s*,\\s*(.+)\\s*>$");
|
|
QRegularExpressionMatch mm = mapRe.match(t);
|
|
if (mm.hasMatch()) {
|
|
TypeExpr val = cppTypeToLidl(mm.captured(1).trimmed());
|
|
return { TypeExpr::Map, "", { {TypeExpr::Primitive, "tstr", {}}, val } };
|
|
}
|
|
|
|
// std::optional<T> -> ?T. Absent before, and the failure was silent: the
|
|
// fallback at the bottom of this function maps ANY unrecognised spelling to
|
|
// the opaque `any`, so a header-first C++ provider could not express
|
|
// optionality at all — it declared `std::optional<std::string>` and
|
|
// published a contract saying `any`, with no diagnostic.
|
|
//
|
|
// The derived contract uses the type-kind spelling (`name: ?T`). C++ has
|
|
// only one spelling, LIDL has two, and they are bound to the same meaning —
|
|
// so which one is emitted is a serialization choice, not a semantic one.
|
|
static QRegularExpression optRe("^std::optional\\s*<\\s*(.+)\\s*>$");
|
|
QRegularExpressionMatch om = optRe.match(t);
|
|
if (om.hasMatch()) {
|
|
TypeExpr inner = cppTypeToLidl(om.captured(1).trimmed());
|
|
// std::optional<std::optional<T>> has NO LIDL type.
|
|
//
|
|
// `?T` is two-state, and optionality is idempotent under that rule — so
|
|
// the nearest contract type is plain `?T`, and that is what gets
|
|
// published. But the author's C++ has THREE states (nullopt / an engaged
|
|
// outer holding nullopt / a value), and the wire has two: accepting the
|
|
// declaration as-is would make `optional(nullopt)` and `nullopt` encode
|
|
// to the same null and decode back as one of them, silently. So it maps
|
|
// down, the generated codec is written for std::optional<T>, and the
|
|
// author's own declaration stops compiling against it — deliberately.
|
|
// Say why here, where the reason is known, rather than leaving a
|
|
// conversion error in generated code the author never wrote.
|
|
if (inner.kind == TypeExpr::Optional) {
|
|
g_unmappableSpellings << t;
|
|
return inner;
|
|
}
|
|
return { TypeExpr::Optional, "", { inner } };
|
|
}
|
|
|
|
// A record the header declared. Checked LAST so it can never shadow a
|
|
// builtin spelling, and gated on the declared set so an unknown type keeps
|
|
// the historical `any` fallback rather than naming a struct nobody emits.
|
|
if (g_recordNames.contains(t))
|
|
return { TypeExpr::Named, t.toStdString(), {} };
|
|
|
|
// Fallback: treat as opaque
|
|
return { TypeExpr::Primitive, "any", {} };
|
|
}
|
|
|
|
// Find `struct Name { Type field; ... };` blocks and turn them into `type`
|
|
// declarations.
|
|
//
|
|
// The parser used to SKIP any line starting with `struct`, which meant a record
|
|
// could not be declared header-first at all — the only way to get one was a
|
|
// hand-written .lidl. Worse, a method mentioning the struct still parsed: its
|
|
// type fell through to the opaque `any`, so the contract silently disagreed
|
|
// with the header.
|
|
static std::vector<TypeDecl> scanForRecords(const QStringList& lines)
|
|
{
|
|
static QRegularExpression openRe("^struct\\s+(\\w+)\\s*\\{\\s*$");
|
|
static QRegularExpression fieldRe("^([\\w:<>,\\s\\*]+?)\\s+(\\w+)\\s*(=[^;]*)?;$");
|
|
|
|
// TWO passes. A record field may name another record (`Blob inner;` inside
|
|
// Wrapper), and cppTypeToLidl only answers Named() for a name already in
|
|
// g_recordNames — so every struct name has to be registered before any
|
|
// field is typed. One pass silently typed such a field as `any`, and the
|
|
// generated codec then tried to encode a Blob as a LogosMap.
|
|
for (int i = 0; i < lines.size(); ++i) {
|
|
QRegularExpressionMatch om = openRe.match(lines.at(i).trimmed());
|
|
if (om.hasMatch())
|
|
g_recordNames.insert(om.captured(1));
|
|
}
|
|
|
|
std::vector<TypeDecl> out;
|
|
for (int i = 0; i < lines.size(); ++i) {
|
|
const QString line = lines.at(i).trimmed();
|
|
QRegularExpressionMatch om = openRe.match(line);
|
|
if (!om.hasMatch()) continue;
|
|
|
|
TypeDecl td;
|
|
td.name = om.captured(1).toStdString();
|
|
for (int j = i + 1; j < lines.size(); ++j) {
|
|
const QString body = lines.at(j).trimmed();
|
|
if (body.startsWith("};")) break;
|
|
if (body.isEmpty() || body.startsWith("//")) continue;
|
|
// Strip a trailing line comment before matching: a field written
|
|
// `std::string name; // what it is` does not end in ';' and was
|
|
// silently DROPPED, publishing a record with a partial field list —
|
|
// the worst kind of wrong, because it looks like a contract.
|
|
QString field = body;
|
|
const int comment = field.indexOf("//");
|
|
if (comment >= 0) field = field.left(comment).trimmed();
|
|
if (field.isEmpty()) continue;
|
|
QRegularExpressionMatch fm = fieldRe.match(field);
|
|
if (!fm.hasMatch()) continue;
|
|
FieldDecl fd;
|
|
fd.name = fm.captured(2).toStdString();
|
|
fd.type = cppTypeToLidl(fm.captured(1).trimmed());
|
|
td.fields.push_back(fd);
|
|
}
|
|
if (!td.fields.empty()) out.push_back(td);
|
|
}
|
|
return out;
|
|
}
|
|
|
|
// Keep only the records the module's API actually mentions.
|
|
//
|
|
// An impl header routinely declares helper structs that are none of a
|
|
// consumer's business — `struct PendingAction` inside the class, a
|
|
// `struct ModuleSource` next to it. Publishing every struct as a contract
|
|
// `type` changes the module's PUBLISHED interface as a side effect of an
|
|
// internal refactor, which is not something deriving a contract from a header
|
|
// is allowed to do. A struct earns its place in the contract by appearing in a
|
|
// method or event signature — transitively, since a published record's own
|
|
// fields may name others.
|
|
static void keepOnlyReferencedRecords(ModuleDecl& module)
|
|
{
|
|
auto mention = [](const TypeExpr& te, std::set<std::string>& out) {
|
|
std::function<void(const TypeExpr&)> walk = [&](const TypeExpr& t) {
|
|
if (t.kind == TypeExpr::Named) out.insert(t.name);
|
|
for (const TypeExpr& e : t.elements) walk(e);
|
|
};
|
|
walk(te);
|
|
};
|
|
|
|
std::set<std::string> referenced;
|
|
for (const MethodDecl& md : module.methods) {
|
|
mention(md.returnType, referenced);
|
|
for (const ParamDecl& pd : md.params) mention(pd.type, referenced);
|
|
}
|
|
for (const EventDecl& ed : module.events)
|
|
for (const ParamDecl& pd : ed.params) mention(pd.type, referenced);
|
|
|
|
// Transitive closure: a referenced record's fields may name more records.
|
|
bool grew = true;
|
|
while (grew) {
|
|
grew = false;
|
|
for (const TypeDecl& td : module.types) {
|
|
if (!referenced.count(td.name)) continue;
|
|
for (const FieldDecl& fd : td.fields) {
|
|
std::set<std::string> here;
|
|
mention(fd.type, here);
|
|
for (const std::string& n : here)
|
|
if (referenced.insert(n).second) grew = true;
|
|
}
|
|
}
|
|
}
|
|
|
|
std::vector<TypeDecl> kept;
|
|
for (const TypeDecl& td : module.types)
|
|
if (referenced.count(td.name)) kept.push_back(td);
|
|
module.types = std::move(kept);
|
|
}
|
|
|
|
// ---------------------------------------------------------------------------
|
|
// Parse a single method declaration line
|
|
// ---------------------------------------------------------------------------
|
|
|
|
static bool parseMethodLine(const QString& line, MethodDecl& out)
|
|
{
|
|
// Find the parameter list: everything between the last '(' and ')'
|
|
int parenOpen = -1;
|
|
int parenClose = -1;
|
|
int depth = 0;
|
|
for (int i = line.size() - 1; i >= 0; --i) {
|
|
if (line[i] == ')') {
|
|
if (parenClose < 0) parenClose = i;
|
|
depth++;
|
|
} else if (line[i] == '(') {
|
|
depth--;
|
|
if (depth == 0) {
|
|
parenOpen = i;
|
|
break;
|
|
}
|
|
}
|
|
}
|
|
if (parenOpen < 0 || parenClose < 0)
|
|
return false;
|
|
|
|
QString paramStr = line.mid(parenOpen + 1, parenClose - parenOpen - 1).trimmed();
|
|
|
|
// Everything before '(' is "returnType methodName"
|
|
QString prefix = line.left(parenOpen).trimmed();
|
|
|
|
// The method name is the last identifier token in prefix
|
|
int nameEnd = prefix.size();
|
|
while (nameEnd > 0 && prefix[nameEnd - 1].isSpace())
|
|
nameEnd--;
|
|
int nameStart = nameEnd;
|
|
while (nameStart > 0 && (prefix[nameStart - 1].isLetterOrNumber() || prefix[nameStart - 1] == '_'))
|
|
nameStart--;
|
|
|
|
if (nameStart >= nameEnd)
|
|
return false;
|
|
|
|
const QString methodName = prefix.mid(nameStart, nameEnd - nameStart);
|
|
|
|
// Reject if the extracted name is a C++ keyword — this filters out
|
|
// member variable declarations like "std::function<void(...)> onEvent"
|
|
// where the parser would mistakenly extract "void" as the method name.
|
|
static const QSet<QString> cppKeywords = {
|
|
"void", "int", "bool", "char", "short", "long", "double", "float",
|
|
"auto", "return", "if", "else", "for", "while", "do", "switch",
|
|
"case", "break", "continue", "const", "static", "inline", "virtual"
|
|
};
|
|
if (cppKeywords.contains(methodName))
|
|
return false;
|
|
out.name = methodName.toStdString();
|
|
QString retTypeStr = stripDeclarationSpecifiers(prefix.left(nameStart).trimmed());
|
|
out.returnType = cppTypeToLidl(retTypeStr);
|
|
// Flag methods whose impl returns LogosMap / LogosList so the generator
|
|
// can emit nlohmann→Qt conversion code in the glue layer.
|
|
out.jsonReturn = (retTypeStr == "LogosMap" || retTypeStr == "LogosList");
|
|
// Flag methods whose impl returns StdLogosResult so the generator can
|
|
// emit a StdLogosResult→Qt LogosResult conversion in the glue layer.
|
|
out.resultReturn = (retTypeStr == "StdLogosResult");
|
|
|
|
// Parse parameters
|
|
out.params.clear();
|
|
if (!paramStr.isEmpty()) {
|
|
// Split by comma, respecting template depth
|
|
QStringList parts;
|
|
int start = 0;
|
|
int tdepth = 0;
|
|
for (int i = 0; i < paramStr.size(); ++i) {
|
|
if (paramStr[i] == '<') tdepth++;
|
|
else if (paramStr[i] == '>') tdepth--;
|
|
else if (paramStr[i] == ',' && tdepth == 0) {
|
|
parts.append(paramStr.mid(start, i - start).trimmed());
|
|
start = i + 1;
|
|
}
|
|
}
|
|
parts.append(paramStr.mid(start).trimmed());
|
|
|
|
for (const QString& part : parts) {
|
|
if (part.isEmpty()) continue;
|
|
QString p = part.trimmed();
|
|
int pNameEnd = p.size();
|
|
while (pNameEnd > 0 && p[pNameEnd - 1].isSpace())
|
|
pNameEnd--;
|
|
int pNameStart = pNameEnd;
|
|
while (pNameStart > 0 && (p[pNameStart - 1].isLetterOrNumber() || p[pNameStart - 1] == '_'))
|
|
pNameStart--;
|
|
|
|
if (pNameStart >= pNameEnd) continue;
|
|
|
|
ParamDecl pd;
|
|
pd.name = p.mid(pNameStart, pNameEnd - pNameStart).toStdString();
|
|
pd.type = cppTypeToLidl(p.left(pNameStart));
|
|
out.params.push_back(pd);
|
|
}
|
|
}
|
|
|
|
return true;
|
|
}
|
|
|
|
// ---------------------------------------------------------------------------
|
|
// Main entry point
|
|
// ---------------------------------------------------------------------------
|
|
|
|
// Join doc-comment lines preserving line breaks (drop leading/trailing blanks).
|
|
static QString joinDocLines(QStringList lines)
|
|
{
|
|
while (!lines.isEmpty() && lines.first().trimmed().isEmpty()) lines.removeFirst();
|
|
while (!lines.isEmpty() && lines.last().trimmed().isEmpty()) lines.removeLast();
|
|
return lines.join('\n');
|
|
}
|
|
|
|
ImplParseResult parseImplHeader(const QString& headerPath,
|
|
const QString& className,
|
|
const QString& metadataPath,
|
|
QTextStream& err)
|
|
{
|
|
ImplParseResult result;
|
|
|
|
// Both file-statics are per-parse state: one process generates for more than
|
|
// one module. Cleared here rather than next to their first use because the
|
|
// metadata's event params are typed before the header is even read.
|
|
g_unmappableSpellings.clear();
|
|
|
|
// --- Read metadata.json ---
|
|
{
|
|
QFile mf(metadataPath);
|
|
if (!mf.open(QIODevice::ReadOnly | QIODevice::Text)) {
|
|
result.error = "Failed to open metadata file: " + metadataPath;
|
|
return result;
|
|
}
|
|
QJsonParseError pe;
|
|
QJsonDocument doc = QJsonDocument::fromJson(mf.readAll(), &pe);
|
|
if (pe.error != QJsonParseError::NoError) {
|
|
result.error = "Failed to parse metadata JSON: " + pe.errorString();
|
|
return result;
|
|
}
|
|
QJsonObject obj = doc.object();
|
|
result.module.name = obj.value("name").toString().toStdString();
|
|
result.module.version = obj.value("version").toString().toStdString();
|
|
result.module.description = obj.value("description").toString().toStdString();
|
|
result.module.category = obj.value("category").toString().toStdString();
|
|
const QJsonArray deps = obj.value("dependencies").toArray();
|
|
for (const QString& depName : dependencyNames(deps))
|
|
result.module.depends.push_back(depName.toStdString());
|
|
|
|
// Read events declared in metadata.json
|
|
QJsonArray events = obj.value("events").toArray();
|
|
for (const QJsonValue& ev : events) {
|
|
QJsonObject evObj = ev.toObject();
|
|
EventDecl ed;
|
|
ed.name = evObj.value("name").toString().toStdString();
|
|
ed.description = evObj.value("description").toString().toStdString();
|
|
QJsonArray params = evObj.value("params").toArray();
|
|
for (const QJsonValue& pv : params) {
|
|
QJsonObject po = pv.toObject();
|
|
ParamDecl pd;
|
|
pd.name = po.value("name").toString().toStdString();
|
|
pd.type = cppTypeToLidl(po.value("type").toString());
|
|
ed.params.push_back(pd);
|
|
}
|
|
if (!ed.name.empty())
|
|
result.module.events.push_back(ed);
|
|
}
|
|
}
|
|
|
|
// --- Read and parse header ---
|
|
QFile hf(headerPath);
|
|
if (!hf.open(QIODevice::ReadOnly | QIODevice::Text)) {
|
|
result.error = "Failed to open header file: " + headerPath;
|
|
return result;
|
|
}
|
|
QString source = QString::fromUtf8(hf.readAll());
|
|
hf.close();
|
|
|
|
// Records first: cppTypeToLidl consults the declared set, so the structs
|
|
// have to be known before a single signature is looked at.
|
|
// Split into physical lines, then merge any whose parentheses are still
|
|
// open into one logical line. The scanner below is line-based — it only
|
|
// accepts a method when a single trimmed line ends in ';' and
|
|
// parseMethodLine finds a balanced '(...)' on it — so without this a method
|
|
// signature wrapped across several physical lines is silently dropped.
|
|
// Parens inside comments / string / char literals are ignored.
|
|
QStringList lines;
|
|
{
|
|
const QStringList physical = source.split('\n');
|
|
QString acc;
|
|
int parenDepth = 0;
|
|
bool inBlockComment = false;
|
|
for (const QString& phys : physical) {
|
|
bool inStr = false;
|
|
bool inChr = false;
|
|
for (int i = 0; i < phys.size(); ++i) {
|
|
const QChar c = phys[i];
|
|
const QChar n = (i + 1 < phys.size()) ? phys[i + 1] : QChar();
|
|
if (inBlockComment) {
|
|
if (c == '*' && n == '/') { inBlockComment = false; ++i; }
|
|
} else if (inStr) {
|
|
if (c == '\\') ++i; else if (c == '"') inStr = false;
|
|
} else if (inChr) {
|
|
if (c == '\\') ++i; else if (c == '\'') inChr = false;
|
|
} else if (c == '/' && n == '*') {
|
|
inBlockComment = true; ++i;
|
|
} else if (c == '/' && n == '/') {
|
|
break;
|
|
} else if (c == '"') {
|
|
inStr = true;
|
|
} else if (c == '\'') {
|
|
inChr = true;
|
|
} else if (c == '(') {
|
|
++parenDepth;
|
|
} else if (c == ')') {
|
|
if (parenDepth > 0) --parenDepth;
|
|
}
|
|
}
|
|
if (acc.isEmpty())
|
|
acc = phys;
|
|
else
|
|
acc += ' ' + phys.trimmed();
|
|
if (parenDepth <= 0) {
|
|
lines.append(acc);
|
|
acc.clear();
|
|
}
|
|
}
|
|
if (!acc.isEmpty())
|
|
lines.append(acc);
|
|
}
|
|
|
|
// Records, before any signature is examined: cppTypeToLidl() consults the
|
|
// declared set, so a `Blob` parameter only becomes Named("Blob") once the
|
|
// struct has been seen. Reset per parse — the set is file-static and a
|
|
// single process generates for more than one module.
|
|
g_recordNames.clear();
|
|
result.module.types = scanForRecords(lines);
|
|
|
|
// State machine: find "class <className>", then collect declarations.
|
|
// `InLogosEvents` is entered by the literal `logos_events:` token
|
|
// (mirrors Qt's `signals:`) — methods declared there are parsed as
|
|
// EventDecls and appended to ModuleDecl.events instead of .methods.
|
|
enum State { LookingForClass, InClass, InPublic, InPrivate, InLogosEvents };
|
|
State state = LookingForClass;
|
|
int braceDepth = 0;
|
|
|
|
// Accumulates doc-comment lines adjacent to a method so the doc comment
|
|
// becomes the method's description. Reset on any blank / non-comment line.
|
|
QStringList pendingDoc;
|
|
bool inBlockComment = false;
|
|
|
|
QRegularExpression classRe("\\bclass\\s+" + QRegularExpression::escape(className) + "\\b");
|
|
QRegularExpression accessRe("^\\s*(public|private|protected)\\s*:");
|
|
QRegularExpression eventsRe("^\\s*logos_events\\s*:");
|
|
QRegularExpression ctorDtorRe("^\\s*~?" + QRegularExpression::escape(className) + "\\s*\\(");
|
|
|
|
for (const QString& rawLine : lines) {
|
|
QString line = rawLine.trimmed();
|
|
|
|
switch (state) {
|
|
case LookingForClass:
|
|
if (classRe.match(line).hasMatch()) {
|
|
state = InClass;
|
|
for (QChar c : line) {
|
|
if (c == '{') braceDepth++;
|
|
else if (c == '}') braceDepth--;
|
|
}
|
|
}
|
|
break;
|
|
|
|
case InClass:
|
|
case InPublic:
|
|
case InPrivate:
|
|
case InLogosEvents:
|
|
// Inside a multi-line /** ... */ doc-comment block: capture its
|
|
// text (skip brace counting — comments don't affect scope).
|
|
if (inBlockComment) {
|
|
QString t = line;
|
|
int end = t.indexOf("*/");
|
|
if (end >= 0) { t = t.left(end); inBlockComment = false; }
|
|
t.remove(QRegularExpression(R"(^\*+\s?)"));
|
|
t = t.trimmed();
|
|
pendingDoc.append(t);
|
|
break;
|
|
}
|
|
|
|
// Count braces only on real code lines. Braces inside a doc/line
|
|
// comment (e.g. `/// returns { "k": v }`) must not affect scope
|
|
// tracking, or an unbalanced brace in a comment would make the
|
|
// parser think the class ended early and drop later declarations.
|
|
if (!(line.startsWith("//") || line.startsWith("/*") || line.startsWith("*"))) {
|
|
for (QChar c : line) {
|
|
if (c == '{') braceDepth++;
|
|
else if (c == '}') braceDepth--;
|
|
}
|
|
|
|
if (braceDepth <= 0) {
|
|
state = LookingForClass;
|
|
goto done;
|
|
}
|
|
}
|
|
|
|
// A section specifier may be followed by a declaration on the
|
|
// *same* physical line — e.g. clang-format / prettier collapse
|
|
// logos_events:
|
|
// void versionReady(const std::string& version);
|
|
// into `logos_events : void versionReady(const std::string& version);`.
|
|
// Strip any leading specifiers, updating the section state, and
|
|
// let whatever remains fall through to the declaration parser
|
|
// below — otherwise everything after the colon is discarded and
|
|
// the same valid C++ is parsed differently based on formatting.
|
|
//
|
|
// `logos_events:` takes precedence over the standard access
|
|
// specifiers: it's a separate section that the codegen pulls
|
|
// event prototypes from. (At preprocess time, `logos_events`
|
|
// expands to `public`, but the raw source still carries the
|
|
// token we recognise here.)
|
|
bool specifierStripped = false;
|
|
while (true) {
|
|
QRegularExpressionMatch em = eventsRe.match(line);
|
|
if (em.hasMatch()) {
|
|
state = InLogosEvents;
|
|
line = line.mid(em.capturedEnd()).trimmed();
|
|
specifierStripped = true;
|
|
continue;
|
|
}
|
|
QRegularExpressionMatch am = accessRe.match(line);
|
|
if (am.hasMatch()) {
|
|
QString spec = am.captured(1);
|
|
if (spec == "public") state = InPublic;
|
|
else state = InPrivate;
|
|
line = line.mid(am.capturedEnd()).trimmed();
|
|
specifierStripped = true;
|
|
continue;
|
|
}
|
|
break;
|
|
}
|
|
// A *bare* specifier (nothing after the colon) is a section
|
|
// boundary and resets any pending doc-comment, mirroring Qt's
|
|
// `signals:`. But when a declaration shares the line, the doc
|
|
// comment preceding the whole line must still attach to that
|
|
// declaration — otherwise documentation, like the declaration
|
|
// itself (#76), would become formatting-dependent. So only clear
|
|
// here for the bare form; the same-line form keeps pendingDoc and
|
|
// attaches it in the declaration parser below.
|
|
if (specifierStripped && line.isEmpty())
|
|
pendingDoc.clear();
|
|
|
|
// Only doc comments (/// or /** ... */ / /*! ... */) accumulate as
|
|
// the pending description for the next method. Plain // and /*
|
|
// comments are ignored but leave pending doc intact; blank /
|
|
// preprocessor lines reset it so only *adjacent* comments attach.
|
|
if (line.startsWith("///")) {
|
|
QString text = line.mid(3);
|
|
if (text.startsWith('<')) text = text.mid(1); // ///< trailing form
|
|
text = text.trimmed();
|
|
pendingDoc.append(text);
|
|
break;
|
|
}
|
|
if (line.startsWith("/**") || line.startsWith("/*!")) {
|
|
QString text = line.mid(3);
|
|
int end = text.indexOf("*/");
|
|
if (end >= 0) text = text.left(end);
|
|
else inBlockComment = true;
|
|
text.remove(QRegularExpression(R"(^\*+\s?)"));
|
|
text = text.trimmed();
|
|
pendingDoc.append(text);
|
|
break;
|
|
}
|
|
if (line.startsWith("//") || line.startsWith("/*") || line.startsWith("*")) {
|
|
break; // non-doc comment: ignore, keep pending doc
|
|
}
|
|
if (line.isEmpty() || line.startsWith("#")) {
|
|
pendingDoc.clear();
|
|
break;
|
|
}
|
|
|
|
if (ctorDtorRe.match(line).hasMatch()) {
|
|
pendingDoc.clear();
|
|
break;
|
|
}
|
|
|
|
if (line.startsWith("typedef") || line.startsWith("using")
|
|
|| line.startsWith("friend") || line.startsWith("enum")
|
|
|| line.startsWith("struct")) {
|
|
pendingDoc.clear();
|
|
break;
|
|
}
|
|
|
|
if (state == InLogosEvents) {
|
|
// Inside `logos_events:` — every bare prototype is an event.
|
|
// Events are always void-returning by definition, so we
|
|
// re-use parseMethodLine to extract name + params and
|
|
// discard the return type.
|
|
if (line.endsWith(';')) {
|
|
QString decl = line.left(line.size() - 1).trimmed();
|
|
MethodDecl md;
|
|
if (parseMethodLine(decl, md)) {
|
|
EventDecl ed;
|
|
ed.name = md.name;
|
|
ed.params = md.params;
|
|
ed.description = joinDocLines(pendingDoc).toStdString();
|
|
result.module.events.push_back(ed);
|
|
}
|
|
}
|
|
pendingDoc.clear();
|
|
break;
|
|
}
|
|
|
|
if (state != InPublic) { pendingDoc.clear(); break; }
|
|
|
|
if (line.contains("std::function<")) {
|
|
// A std::function member is not a method — skip it so the
|
|
// `parseMethodLine` path below doesn't choke on the nested
|
|
// parens in its type. (Events are declared in a typed
|
|
// `logos_events:` section, parsed above — there is no longer
|
|
// any special `std::function emitEvent` member to detect.)
|
|
pendingDoc.clear();
|
|
break;
|
|
}
|
|
|
|
if (line.endsWith(';')) {
|
|
QString decl = line.left(line.size() - 1).trimmed();
|
|
MethodDecl md;
|
|
if (parseMethodLine(decl, md)) {
|
|
// LogosModuleContext lifecycle hooks / context accessors are
|
|
// framework plumbing, not part of the module's API contract.
|
|
// An impl commonly overrides `onContextReady()` (and could
|
|
// re-declare an accessor) in its own public section, so the
|
|
// header parser would otherwise emit them into the derived
|
|
// LIDL — breaking cdylib eligibility (e.g. the inherited
|
|
// accessors' Qt-free-subset check) and exposing non-API
|
|
// methods. Skip the reserved names regardless of access.
|
|
static const QSet<QString> reserved = {
|
|
"onContextReady", "modules", "modulePath",
|
|
"instanceId", "instancePersistencePath"
|
|
};
|
|
if (!reserved.contains(qs(md.name))) {
|
|
md.description = joinDocLines(pendingDoc).toStdString();
|
|
result.module.methods.push_back(md);
|
|
}
|
|
}
|
|
}
|
|
pendingDoc.clear();
|
|
break;
|
|
}
|
|
}
|
|
|
|
done:
|
|
// Now that every signature is known, drop the structs the API never
|
|
// mentions — a header's internal helpers must not become published
|
|
// contract types.
|
|
keepOnlyReferencedRecords(result.module);
|
|
|
|
if (!g_unmappableSpellings.isEmpty()) {
|
|
g_unmappableSpellings.removeDuplicates();
|
|
err << "Warning: " << headerPath << ": " << g_unmappableSpellings.join(", ")
|
|
<< " has no LIDL type. `?T` is TWO-state — a value or empty — so a "
|
|
"nested optional cannot denote a third state; the contract "
|
|
"publishes the collapsed `?T`, and the generated codec is "
|
|
"written for std::optional<T>. Declare it that way, or the "
|
|
"generated code will not compile against this header.\n";
|
|
}
|
|
|
|
if (result.module.methods.empty()) {
|
|
err << "Warning: no public methods found in class " << className
|
|
<< " in " << headerPath << "\n";
|
|
}
|
|
|
|
return result;
|
|
}
|