mirror of
https://github.com/logos-co/logos-cpp-sdk.git
synced 2026-08-31 09:41:06 +00:00
feat/inbound-door-cpp-backend
4
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Dario LipicarandClaude Opus 5
|
acea0d24e2 |
feat(codegen): a lossless Qt type mapping, and LIDL types in getMethods (#149)
* feat(codegen): a lossless Qt type mapping — typed containers and optionals
`lidlTypeToQt` answered four different LIDL types with one Qt name. `[uint]`,
`[bstr]`, `[[uint]]` and `[any]` were all QVariantList; `{tstr: uint}` and
`{tstr: any}` were both QVariantMap; every `?T` was a bare QVariant. A Qt
consumer therefore lost, on the SAME contract, types that the std consumer next
door kept — it could not tell `?tstr` from `?uint`, and got no compile-time
check on any element.
The table is now recursive:
[T] QList<qtOf(T)> ([tstr] stays QStringList)
{tstr: V} QMap<QString, qtOf(V)>
?T std::optional<qtOf(T)> (through optionalValueType,
so ??T stays two-state)
any QVariant — KEPT, deliberately
`any` is the one row that must not widen: QVariant is the only Qt type that
holds bytes AND an exact uint64 AND arbitrary nesting at once, so every
narrower spelling would lose what it was chosen to carry. The rule is applied
at the LEAF, so anything whose element type bottoms out at `any` keeps the
QVariant-family spelling at every depth — `[any]` is QVariantList, `[[any]]`
still is, `{tstr: [any]}` is QVariantMap, `?any` is QVariant.
THE TRAP, and why this is not just a rename. A widened name must never reach
QVariant::fromValue / qvariant_cast / logos::qt::toWire as a WHOLE value.
logos-protocol's qvariantToNlohmann matches a CLOSED userType() set:
QList<qulonglong> is in none of it, so it serialises to JSON null. The decode
fails just as quietly — qvariant_cast<QList<qulonglong>> of a QVariantList
yields an EMPTY list. Neither direction warns. So every widened slot is encoded
and decoded by a generator-emitted ELEMENT LOOP, the shape the record cases
already used, and `lidlQtNeedsElementLoop` is the single predicate that decides
which slots need one.
The emitted loops take their source as a lambda PARAMETER, not a body-local
binding. They nest (`[[uint]]`), every level wants the same short names, and a
local — or a range-for over a name the loop itself declares — is then
self-referential: it compiles and reads uninitialised memory. Measured: three
round-trip tests died on SIGTRAP before the argument form.
THE STRING-KEYED EMITTER IS FROZEN, ON PURPOSE. generator_lib is keyed on flat
type NAMES (lidl_to_json flattens the contract before it gets there, because
that emitter also serves the metaobject-introspection path), so it cannot
derive the levels an element loop needs without parsing C++ type names back
into a tree. Every widened spelling is folded back to the name it produced
before (legacyQtBase), which keeps BOTH surfaces it feeds byte-for-byte
unchanged: the legacy Qt consumer, and the Qt-free lp one whose table is
DERIVED from it through mapParamTypeStd. Verified by generating a
28-method contract through both before and after: the diff is empty. The
widened types are spent in the TypeExpr-driven emitters instead
(lidl_gen_client.cpp here, lidl_gen_qt_consumer.cpp in logos-qt-sdk).
Also here, because both are consequences of the table becoming recursive:
* lidlTypeToQt gained a record-name HOOK. A wrapper nests its record structs
in the wrapper class, so a type written outside that scope must qualify
them — and the emitters used to do that by matching the three shapes that
could mention a record on the finished string. `?Point` and
`QList<QList<Point>>` are now spellable, so the qualification happens
during the walk, at the one place that knows a name is a record.
* lidlTypeToLidlText — the LIDL contract spelling of a type. Unused here; the
commit that follows puts getMethods() on it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* feat(cdylib): getMethods publishes the LIDL contract vocabulary, not Qt names
A module's published metadata — `returnType`, `parameters[].type`, `signature`
— answered in Qt type names. Two things wrong with that, and the second is the
one that matters:
* a cdylib module is Qt-FREE. It described itself in the types of a language
it does not use, to readers (`lm`, logoscore's method listing, basecamp's
module inspector) that are showing a human what the module offers.
* it was LOSSY. `[uint]`, `[bstr]` and `[any]` are three different LIDL types
and all three published as the single word QVariantList, so the listing
could not be read back as a contract. That is now `[uint]`, `[bstr]`,
`[any]`; `{tstr: uint}`; `? tstr`; and a record publishes its declared
name.
WHY THIS IS SAFE — checked, not assumed. The historical objection is recorded
in the function this replaces: these strings are read as METATYPES, and
emitting a record's struct name here once made the host SIGSEGV. Nothing in the
current runtime does that. logos-plugin-qt's QtProviderObject dispatches on
`method.returnMetaType()` / `parameterMetaType(i)` — the QMetaObject, never
this JSON — and every remaining reader treats these fields as opaque text:
logos-module's `lm` prints them, logoscore's output.cpp prints them, basecamp's
CoreModuleManager forwards the JSON to QML, and the plain wire's json_mapping
only round-trips them. Nothing anywhere builds a QMetaObject from this
metadata.
The spelling comes from lidlTypeToLidlText, which mirrors logos-lidl's
serializeTypeExpr. It is a COPY, because that function is file-local to
logos-lidl's serializer.cpp and the public headers expose no type printer —
so instead of hoping, the pairing is ASSERTED: the test round-trips each shape
through `lidl::serialize` and reads the type text back out of the emitted
`.lidl`. When logos-lidl exports a printer, delete the copy and call it.
Not fixed by this, and not attempted: the Rust SDK's provider generator has its
own `qt_type_name` writing the same JSON, so the two languages now disagree
about how a module describes itself. That is a cross-repo change.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* test(doctests): the generator round-trip pins the lossless Qt spellings
`cpp-sdk-generator-roundtrip.test.yaml` is a CI gate
(.github/workflows/doctests.yml), and two of its `expect_contains` were
pinned to the type names the Qt consumer produced BEFORE the lossless
mapping:
QStringList labels(const QVariantList& ids
QVariant nearest(const Point& p, QVariant limit
The generator now emits `QList<qulonglong>` and
`std::optional<Point>` / `std::optional<qulonglong>` for those slots, so both
assertions failed. The `nearest` step's `run` grep was pinned the same way
(`QVariant nearest`), so the line it was supposed to assert on was not even
in the output being searched.
Verified by running the spec's own steps against the generator built from
this commit: 10 run-steps, 0 failures. The `[uint]` -> QList<qulonglong> and
`?T` -> std::optional<T> lines were read out of the real
`consumer/sensor_module_api.h` and `geometry/geometry_module_api.h`, not
written from the mapping table.
Prose too, in three places that described the old table: the Flow-3 type
mapping ("other arrays -> QVariantList"), the composite-types intro
("optionals ... stay QVariantMap / QVariant"), and the composite-signature
step. They now say what the mapping actually is — one LIDL type, one C++
spelling, with `any` the single deliberate exception — and `nearest` is
called out as the one signature carrying both halves of the optional
mapping.
`doctests/outputs/cpp-sdk-generator-roundtrip.md` carries the same prose
corrections. That tree is hand-pinned and CI never diffs it, which is
exactly why it must be corrected by hand.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(generator): the consumer wrapper comes from the contract, not from getMethods
`logos-cpp-generator <plugin> --module-only` — the invocation
logos-plugin-qt's generate-module-headers.sh makes for every module's lp
wrapper — built that wrapper's whole type surface out of the plugin's
PUBLISHED `getMethods()` metadata. It now builds it out of the module's `.lidl`
contract, the file the same invocation already passes as `--events-from`.
WHY THIS IS A DEFECT AND NOT A PREFERENCE. generator_lib is keyed on flat type
NAMES, and mapParamType / mapReturnType fall back to QVariant for a name they
do not recognise (generator_lib.cpp:142 and :153). So the wrapper's types
depend on the VOCABULARY a module happens to publish its metadata in, and a
vocabulary this emitter has no row for degrades to QVariant — LogosMap on the
lp surface — with no diagnostic at any layer. It is a machine reader of a
listing that every other consumer treats as human-facing text, and it fails
silently.
It was measured, not theorised. 621772a made the cdylib backend publish the
LIDL contract vocabulary (`tstr`, `[uint]`, `result`, `? tstr`) in place of Qt
type names, because that listing is what `lm`, logoscore and basecamp show a
human and Qt names are the wrong answer for a Qt-free module. Every
`interface: "universal"` module's lp wrapper collapsed:
logos-test-modules' `checks.unit-tests-new-api` went PASS -> FAIL, and the
compiler said exactly why —
error: no viable conversion from 'LogosMap' to 'StdLogosResult'
StdLogosResult r = modules().test_basic_module.resultWithMap();
`result` is not a name mapReturnType knows, so it became QVariant, so it became
LogosMap. Bisected to exactly 621772a (5ffd90b passes, dd52d9d fails).
THE FIX IS TO STOP READING THAT VOCABULARY, not to learn a second one.
`int` means a 32-bit Qt int in one table and a 64-bit LIDL integer in the
other, and the reader cannot tell from the string which table it is holding —
a merged table would silently mistype every integer on every module. The
contract has no such ambiguity: it is a TypeExpr tree, and lidl_to_json is the
single place it is flattened. Taking methods from it makes this path emit the
same wrapper as `--general-only --dep <name>=<name>.lidl`, which is what
buildHeaders.nix already runs under cross-compilation and for the entire Qt
surface. Contract-first, on every platform, for every surface.
WHAT CHANGED, exactly:
* loadEventsFromLidl -> loadContractFromLidl. It already parsed the whole
contract and threw the methods away; it now returns them, after the same
lidlCheckRecords + lidlInjectIdentity + noteOptionalPositionalSlots that
main.cpp's --dep path applies. Identity is injected rather than read,
matching the provider side (main.cpp's --backend cdylib), so the two cannot
disagree about name() / version().
* A sidecar that is NAMED BUT MISSING is now refused (exit 2), and an
unreadable or malformed one is fatal (exit 4). Both used to be shrugged off
— which shipped a wrapper with no typed events, and would now ship one with
no typed methods, in the silently-empty shape generate-module-headers.sh
exists to refuse.
* The plugin is STILL LOADED. That load is the dlopen check this path
performs (exit 3 on an SDK/ABI skew) and it is unchanged; what the plugin
says about itself is now compared against the contract instead of believed,
and a divergence — a stale sidecar — is reported by name on stderr. Only
`isInvokable` entries are compared: a cdylib publishes its events into the
same array, tagged `"type": "event"`, and both emitters already skip those.
* A module with NO contract keeps introspection — a handcrafted Qt plugin's
QMetaObject is still the only description of its API that exists, and Qt
type names are the right vocabulary for it — but a listing spelled in the
LIDL vocabulary with no contract to go with it is now REFUSED (exit 7)
instead of silently producing the untyped wrapper. That combination is only
reachable by hand: buildHeaders.nix always passes the flag when the sidecar
exists, and it is the shape the developer guide used to suggest. The two
vocabularies are not distinguishable in general, which is the whole reason
this emitter must read only one — but they do not have to be: the words
they share (`int`, `bool`) are all in the known table and never reach the
fallback, so the check keys on the LIDL half Qt has no word for at all
(`tstr`, `bstr`, `uint`, `float64`, `result`, `any`, and anything starting
`[`, `{` or `?`). No Qt type is spelled that way, so it cannot false-fire;
a false negative is just the old behaviour.
THE ENUMERATION, because two previous ones missed this reader. Searching for
who greps `returnType` is what missed it; the question is what the data FLOWS
INTO. Every consumer of a published getMethods array in the workspace:
MACHINE (one, and it is this one)
logos-cpp-sdk cpp-generator/plugin_introspect.cpp, reached only through
logos-plugin-qt's generate-module-headers.sh / buildHeaders.nix.
HUMAN-READABLE OR OPAQUE PASSTHROUGH (all of them)
logos-module's `lm` (prints; --json re-emits verbatim), logoscore-cli's
client/output.cpp (prints) and core_service_dispatch.cpp (forwards),
logos-logoscore-tui (formats one line per method), logos-module-viewer
(reads the QMetaObject directly, not this JSON), basecamp's
CoreModuleManager / MainUIBackend (hands the JSON string to QML),
logos-protocol's json_mapping.cpp and qvariant_rpc_value.cpp (round-trip
the strings unread).
PRODUCERS, for completeness: lidl_gen_cdylib.cpp (LIDL vocabulary),
logos-plugin-qt's QtProviderObject (Qt names, from the QMetaObject) and
lidl_gen_cdylib_glue.cpp (forwards the cdylib's), logos-rust-sdk's
rustgen_provider.rs (still Qt names — the two languages disagree, as
621772a noted), and logos-protocol's ModuleProxy, which appends derived
name()/version() entries spelled `QString`. None of that reaches a type
decision any more, which is the point of the change.
Build-system paths checked and clear: `<plugin> --module-only` is invoked
from exactly one place in the workspace (generate-module-headers.sh:60);
LogosModule.cmake, buildPlugin.nix and mkLogosModuleTests.nix all use
`--general-only`, which is contract-driven already; the doctests' `--lidl
--module-only` is a different mode entirely.
VERIFIED.
`nix build path:./repos/logos-test-modules#checks.aarch64-darwin.unit-tests-new-api`
with this SDK overridden in (plus the logos-lidl overrides the branch needs at
the qt-sdk and plugin-qt nodes) — 32 passed, 0 failed. The same command against
this branch's HEAD fails to compile, as above. The build log shows the path
taken, per module:
Detected new-API plugin (LogosProviderPlugin), using getMethods() — 43 methods
Using the module's LIDL contract for the method surface — 41 methods
(the plugin's published listing is a description, not a type source)
The refusal, measured by hand against a real LIDL-publishing plugin
(test_basic_module, built from this branch) because no check exercises a
hand-run invocation:
no --events-from -> exit 7, nothing written, the message above naming
8 offending slots
with --events-from -> exit 0, 41 typed methods, 69 `std::string` in the
emitted lp header
a pre-621772a build of the SAME module (Qt-name listing), no --events-from
-> exit 0, still generates, still typed — the refusal does
not fire on the vocabulary this emitter can read
nix/tests-generator-cli.nix gains the two CLI-surface cases this adds: a
`--events-from` naming a file that does not exist is refused with that
sentence, and — the control that makes it mean something — the same command
with a READABLE contract gets past the flag and fails on the plugin instead. No
plugin is needed for either: the contract is loaded before the plugin is
opened.
logos-cpp-sdk's own checks (tests, generator-cli, module-impl-abi): 334 of 334.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
|
||
|
Dario LipicarandClaude Opus 5
|
198f0317ca |
feat(optional): ?T is two-state, and the generators finally read it (#125)
* feat(optional): ?T is two-state, and the generators finally read it
No generator in any language read the optional flag — it had never been
implemented. `?T` was a HARD REJECT on the cdylib backend ("module not
cdylib-eligible"), `std::optional<T>` in an impl header fell through to the
opaque `any` with no diagnostic, and a `? name: T` field was emitted as a
required `T`. Three real contracts in the workspace already declare optionals
and were silently getting one of those three answers.
`?T` is TWO-state: a value of T, or empty. Never three — "one LIDL type <-> one
type per language" leaves nowhere for a third state, because every target has
exactly one empty inhabitant.
ONE MEANING, TWO SPELLINGS. `? name: T` (the field flag) and `name: ?T` (the
type kind) are the same declaration. Backends no longer answer that themselves:
logos-lidl's fieldIsOptional/fieldValueType are re-exported from lidl_compat.h
and every site THIS COMMIT TOUCHES reads them, so the two spellings emit
byte-identical code on the cdylib and client backends. That
caught a live drift on the way in — lidlRecordCollidesWithBytesTag read `f.type`
and so refused `? _bytes: tstr` while letting `_bytes: ?tstr` straight through,
one declaration with two answers.
THE WIRE RULE DEPENDS ON THE SLOT. Absent and explicit null are the SAME state
on decode and DIFFERENT on encode:
- decode is liberal, by exactly one inhabitant: in an optional slot absent and
null both mean empty; in a required slot both stay errors. A present value
goes through the decoder a required T would get, so a wrong type still fails
at the same path — optional widens the domain, it does not switch checking
off. `?bstr` therefore keeps the LENIENT bytes decode a bare `bstr` gets,
rather than silently becoming stricter in the optional slot.
- encode has one canonical form: empty OMITS the key where the slot is NAMED
(a record field) and is spelled null where it is POSITIONAL (an argument, a
return, an event parameter — no key to omit, and arity must not change). Key
omission lives in the record emitter because a Codec only ever sees a value,
never the slot it sits in. A round trip therefore canonicalises.
- `?any` collapses onto `any`: nlohmann::json already carries null, so
std::optional<LogosMap> would give the slot two spellings of empty.
The dispatch gate now admits a missing trailing optional argument and
materialises it as null, exactly the way a missing record field already was. A
method with no optional parameter emits the byte-identical gate it always did.
Header-first: `std::optional<T>` <-> `?T`, composing with records and
containers. `std::optional<std::optional<T>>` has NO LIDL type (three C++ states
over a two-state wire), so it maps down to `?T` — which makes the author's own
declaration stop compiling against the generated codec, deliberately — and says
so at derivation time instead of leaving a conversion error in generated code.
The Qt/Lp consumer surface is NOT fixed and does not pretend to be. The wrappers
real modules get come from legacy/main.cpp, where the AST is flattened to a
single Qt type-name string per slot before optionality could be seen; Qt has no
optional metatype, so `?T` lands on QVariant — the right shape (an invalid
QVariant is Qt's empty inhabitant) with no type. The generator now prints a Note
naming every flattened slot so an affected build is never silent, and
docs/project.md records exactly what a Qt consumer will still do with an
optional field.
Verified by output equivalence, not by a green build: the generator was built
before and after and run over every .lidl in the workspace plus the impl-header
fixtures, in cdylib, consumer-qt, consumer-lp, client and header-first modes.
428 of 465 artefacts are byte-identical; all 37 that differ belong to one of the
four contracts that declare an optional (the 38th path is the manifest). The
harness's sensitivity is pinned by a negative control: qt vs lp output differs
in 45 files. The emitted codec was additionally compiled under -Wall -Wextra and
run against the rules above — omission, absent==null, required-still-rejects,
present-but-wrong-still-fails, and canonicalising round trip.
Tests: 199 pass, 0 fail (180 before, 19 new).
Requires logos-lidl's optionality accessors and logos-protocol's
Codec<std::optional<T>>.
NOT FIXED, AND IT IS THE PATH THAT MATTERS MOST. The legacy interface-wrapper
path is untouched, and it is the one every real module builds through
(buildPlugin.nix:145 -> logos-cpp-generator --general-only). There the two
spellings still diverge:
? maybe: tstr -> QString maybe{}; __m.value("maybe").toString()
maybe: ?tstr -> QVariant maybe{}; __m.value("maybe")
and --api-style lp diverges too, neither side being std::optional. So R3 holds
on the backends below and NOT on the Qt consumer a shipping module actually
gets. logos-chat-module -- the contract that prompted this work -- uses the
field-flag spelling, so it lands on the branch that silently defaults.
The cause is upstream of codegen: legacy/main.cpp's moduleRecordsToJson and
moduleMethodsToJson flatten every TypeExpr to a single Qt TYPE-NAME STRING, so
optionality (along with nesting, map key types and descriptions) is gone before
generator_lib.cpp sees it. Widening that interface is a larger change and is
deliberately not attempted here. The only R3 test on a Qt surface covers
lidl_gen_client.cpp, which is on no live build path.
* chore: re-pin logos-lidl to master for the optionality accessors
lidl_compat.h re-exports typeIsOptional / optionalValueType / fieldIsOptional /
fieldValueType / paramIsOptional / paramValueType, which landed in
logos-lidl#7. The pinned lidl predated it, so CI failed to compile.
logos-lidl 8c95d4f -> 35f33d8. Tests: 199 pass, 0 fail.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* chore: re-pin logos-protocol to master for Codec<std::optional<T>>
The generated record codecs emit Codec<std::optional<T>> for an optional
field; that specialisation landed in logos-protocol#37 and the pinned
protocol predated it.
Note this repo's own tests would NOT have caught the omission -- the
generator tests string-assert emitted text rather than compiling it, so a
missing codec specialisation only surfaces when a real module compiles
generated optional code (logos-test-modules' ext provider).
logos-protocol 4359557 -> 72754ab. Tests: 199 pass, 0 fail.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* test(doctests): override logos-lidl alongside every logos-cpp-sdk override
The doc-tests build downstream repos (logoscore-cli, capability_module,
accounts_module) with --override-input logos-cpp-sdk. Nix does not carry the
overridden input's OWN lock, so those builds got this branch's cpp-sdk source
while still resolving logos-lidl from their own, older locks. The shipped
share/lidl-frontend/lidl_compat.h then calls accessors that lidl does not
have:
lidl_compat.h:46: error: 'paramValueType' has not been declared in 'lidl'
lidl_compat.h:92: error: 'fieldValueType' was not declared in this scope
Every --override-input logos-cpp-sdk now has a matching
--override-input <same-path>/logos-cpp-sdk/logos-lidl.
This is specific to the override path. A normal consumer running
'nix flake update logos-cpp-sdk' inherits cpp-sdk's own lock, which pins the
lidl carrying these accessors, and is unaffected.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* test(doctests): move logos-lidl at the qt-sdk nodes, not under logos-cpp-sdk
The doc-tests failed to build logos-qt-generator:
share/lidl-frontend/lidl_compat.h:46: error: 'paramValueType' has not been
declared in 'lidl'
MECHANISM. This SDK installs cpp-generator/experimental/lidl_compat.h into
$out/share/lidl-frontend/, and logos-qt-sdk's logos-qt-generator *compiles*
that installed header against qt-sdk's OWN logos-lidl input. Under
logos-qt-sdk, logos-lidl is a SIBLING of logos-cpp-sdk, not a descendant:
logos-qt-sdk
|-- logos-cpp-sdk <- --override-input moves this to the commit under test
`-- logos-lidl <- stays on qt-sdk's lock (8c95d4f), lacks the accessors
logos-logoscore-cli and logos-module-builder both declare
`logos-qt-sdk.inputs.logos-cpp-sdk.follows = "logos-cpp-sdk"` but no lidl
follows, so overriding the SDK hands qt-sdk a new lidl_compat.h next to its
old lidl. The failing derivation is logos-qt-generator — not anything in
logos-cpp-sdk, which is why the previous attempt aimed at the wrong node.
THE FIX is one `<path-to-logos-qt-sdk>/logos-lidl` override per qt-sdk node
that ends up on the SDK under test. A tree-walk over the resolved lock found
four in logoscore-cli's closure and two per module build; with the overrides
applied the walk reports zero remaining.
WHAT WAS REMOVED, and why it was doing nothing:
* The `.../logos-cpp-sdk/logos-lidl` overrides added in
|
||
|
Dario LipicarandClaude Opus 5
|
3d322bd315 |
fix(codegen): LIDL int/uint are 64-bit in the Qt spelling too (#113)
* fix(codegen): LIDL int/uint are 64-bit in the Qt spelling too lidlTypeToQt mapped BOTH `int` and `uint` to plain `int`. Everywhere else in the stack a LIDL int/uint is 64-bit — int64_t/uint64_t in C++ impls, i64/u64 in the Rust SDK — so the Qt spelling broke the one-type-per-LIDL-type rule and lost data: a Qt consumer reading a `uint` return got a SIGNED 32-bit value, so anything above 2^31 came back wrong and anything above 2^63 was never expressible. int -> qlonglong, uint -> qulonglong, and returnConversion() gains the matching accessors (toLongLong / toULongLong instead of toInt). qlonglong/qulonglong rather than qint64/quint64 so the generated introspection JSON uses the same names Qt's own metaobject normalisation produces — otherwise a cdylib module's generated `signature` and a legacy module's QMetaObject-derived one would disagree for the same LIDL type. Nothing looks these strings up: the only QMetaType::fromName call in the stack is for "LogosResult". This changes two generated surfaces: the Qt consumer wrapper signatures and the introspection JSON. Passing an int argument still converts implicitly, so callers keep compiling; code that assigns a wrapper's return into an `int` narrows and may warn, which is the bug being surfaced rather than a regression. Tests: 168/168, with the type-mapping and client-emitter expectations updated to the 64-bit spelling. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * feat(records): typed C++ structs for Qt consumers Completes the Qt half of the type mapping this branch started. A `type Foo { … }` in a contract now generates a real struct in the client header, so a consumer writes `Status s = client.makeStatus();` instead of digging fields out of a QVariantMap. One LIDL type, one type per language. lidlTypeToQt - Named -> the record's struct (was QVariant) - [Record] -> QList<Record>, {tstr: Record} -> QMap<QString, Record>. QVariantList CANNOT hold a record without Q_DECLARE_METATYPE, and a typed list is the point. client emitter - struct + inline ToVariant/FromVariant per record, emitted before the class; conversions come after all structs so records may reference each other. Recursive, so a field may itself be [Status] or {tstr: bstr}. - records pass by const&, decode on return, and convert at the call site (sync and async) bstr fields are QByteArray on purpose: logos-protocol's QVariant<->JSON conversion already materialises the canonical {"_bytes": base64url} form as a QByteArray and back, so the record conversions stay pure field mapping and binary survives at any depth with no record-specific bytes handling. Verified by COMPILING and RUNNING the generated code, not just asserting on text — the string tests would not have caught either bug this found: [Record] first mapped to QVariantList (appending a Status to it does not compile) and the decode lambdas shadowed their accumulator. Extracted the emitted record block for a contract with a nested record and a bytes field, compiled it against Qt6Core, and round-tripped Batch -> QVariant -> Batch asserting items[0].port, the QByteArray blob and the label all survive. Exit 0. The LidlTypeToQt.NamedType expectation flips from "QVariant" to the struct name, which is the behaviour change. Tests: 169/169. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(codegen): teach the lp/std consumer wrappers the 64-bit spellings Caught while checking whether the SDKs are ready for the `any` migration, and it is a regression THIS BRANCH would otherwise have shipped. legacy/generator_lib.cpp generates the lp/std consumer wrappers a universal module uses to call its dependencies. It matches type names against an allow-list and falls back to QVariant for anything else: static const QSet<QString> known = { "void","bool","int","double","float","QString", … }; if (known.contains(base)) return base; return QString("QVariant"); Once lidlTypeToQt reports `qlonglong`/`qulonglong`, every LIDL int/uint method misses that list — so a typed `int` parameter would have silently become an opaque QVariant in those wrappers. Worse than the truncation this branch set out to fix, and invisible until someone read the generated header. Adds the two spellings to both allow-lists, plus the conversions they imply: QVariant->Qt (toLongLong / toULongLong), the std spellings (int64_t / uint64_t), the QVariant->std return path, the Qt-style return, and the default-value case. The existing `int` entries stay for legacy Qt plugins, whose QMetaObject still reports `int` for a 32-bit parameter. Tests: 171/171, with the allow-list pinned in both mapping test files — including that an unknown spelling still falls back to QVariant, so the fallback itself is not what regressed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * feat(records): typed structs for C++ dependency wrappers Closes the second record gap. The wrapper every C++ module actually gets for its dependencies comes from the LEGACY generator (`--dep <name>=<lidl>`), not the client-stub backend the previous commit taught about records — and there a contract's `type Status { ... }` reached the consumer as an untyped bag: QVariant on the Qt surface, LogosMap on the std/lp one. Worse than inconvenient for a `bstr` field: the caller received the canonical `{"_bytes": "..."}` envelope and had to know to unwrap it, while Rust and the stub backend handed back real bytes. Now, on all three api styles: struct Status { uint64_t port{}; std::vector<uint8_t> blob{}; }; Status getStatus(logos::CallError* err = nullptr); std::string describeStatus(const Status& s, ...); std::vector<Status> listStatuses(...); The struct is NESTED in the wrapper class (`InfoModule::Status`) because a module consuming two deps that each declare a `Status` includes both wrappers into one translation unit. Conversions are file-local statics in the generated .cpp, so a Qt-free module's own TUs still never see QVariant or nlohmann. Records reach parameters, returns, event callbacks, `[Record]` and `{tstr: Record}` — at any depth, with bytes tagged throughout. Same commit, the legacy path's half of the 64-bit fix: `lidlTypeExprToQtTypeName` mapped BOTH int and uint to `int` ("wire-as-int for now"), so a `uint` method on a dep reached a Qt consumer as a signed 32-bit value and a std/lp consumer as a signed int64_t. Now qlonglong/qulonglong, matching the spelling the other half of this PR gave the stub backend. `lpFromJsonExpr` grew the uint64_t branch it needed — without it a mistyped payload THREW out of nlohmann's implicit conversion instead of defaulting like every other scalar. Verified by generating a contract with a record, a record-of-records, a `uint` above 2^32 and a high-byte `bstr`, then COMPILING the output for qt/std/lp and round-tripping the emitted conversions: - `{"_bytes":"gAH_"}` at every depth, decoding back to the same bytes - 4294967296 intact through both directions - garbage/missing fields default rather than throw That compile is what caught the one real bug here: the container decode lambdas declared `__m`/`__j`, shadowing the record decoder's own locals, so a map-of-records field read from its own uninitialized local — it compiled with nothing but a -Wuninitialized warning. Locals are `__acc`/`__src` now, pinned by a test. Also: 8 generator tests (one asserting an empty record set leaves every byte of the output as it was), 179 total green; logos-test-modules builds and tests green against this generator. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(records): async record returns decoded a default-constructed struct Two correctness holes in the record work, both silent, found while scoping the cdylib provider side. 1. The async consumer overload emitted `qvariant_cast<Status>(v)` while the sync one emitted `StatusFromVariant(_result)`. The wire delivers a QVariantMap and no Q_DECLARE_METATYPE is emitted for the struct, so the cast does not fail — it returns a DEFAULT-CONSTRUCTED Status and the caller sees empty fields with no diagnostic. The sync path being correct is what makes it bad: the same call is right or wrong depending only on which overload the caller reached for. Async now decodes field by field through the same conversion. (The legacy dependency-wrapper generator already did this correctly — this was the experimental client-stub backend only.) 2. A record whose ONLY field is a tstr named `_bytes` is wire-identical to a canonical tagged byte string: `isTaggedBytes()` is checked before `is_object()` in both logos_codec.h and logos_json_convert.cpp, so such a record decodes as bytes and the struct silently disappears. The ambiguity is inherent to the tagged form — the codec's own comment says not to name a map key `_bytes` — but the generator can refuse to emit the one shape guaranteed to misdecode instead of leaving it to be found at runtime. Both front doors (the .lidl client-stub path and the --dep/--interface path) now reject it with a message naming the type and the fix. A second field disambiguates it (isTaggedBytes requires exactly one key), so that shape still generates — verified, not assumed. 181 tests, +2. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * feat(cdylib): records, [bstr], typed maps and nested composites on the C++ provider The C++ cdylib backend could express scalars, `[scalar]`, `any` and an untyped map. Everything else it rejected BY NAME — `[bstr]`, `[[int]]`, `{tstr: int}`, and records, which the impl-header parser could not even declare because it skipped `struct` outright. That is why the ext contract had to be Rust-only. Four changes, in the order they matter: typeSupported() recurses instead of whitelisting element names, which admits [bstr], [[int]], [Record] and [{tstr: T}] in one rule; a declared record is admitted; a map now REQUIRES a tstr key (it used to `return true` for any map and then silently flatten {int: tstr} to an untyped LogosMap, losing the key type). lidlTypeToStdCdylib() became total. Its `lidlTypeToStd` fallback answers QVariantList / QVariantMap — Qt names in a Qt-FREE translation unit — and only failed to appear because the gate rejected everything that reached it. Widening the gate made that fallback a live leak, so composites now recurse and never reach it. <name>_types.h new: the generated codec, recursive, with a FULL specialization for std::vector<uint8_t> that wins over the generic vector rule — which is what keeps a bstr tagged at any depth instead of becoming a plain array of numbers. One Codec specialization per declared record, field by field, with the field path in the error. impl_header_parser learned `struct` (two passes, because a record field may name another record and the type mapper only answers Named() for an already-registered name — one pass silently typed `Blob inner;` as `any`), std::map<std::string, T>, and recursion into vector elements so std::vector<Blob> is [Blob] rather than falling through to `any`. Records are only names the contract DECLARES: `void` is not a LIDL builtin, so `-> void` arrives as Named("void"), and treating every Named as a record is the exact trap that made the Rust generator emit `-> Void`. Two things the interface JSON got wrong, both found by running it: - it spelled a record `Blob` and a `[Record]` `QList<Blob>`. Those are the CONSUMER's names, correct in a generated wrapper where the struct exists — but this JSON is the module's getMethods(), read by the host to marshal a QVariant, and there is no metatype called `Blob`. The host SIGSEGV'd on the first call to any record method. A record IS a variant map at that boundary; lidlTypeToQtWire() says so. - the types header emitted the structs. Header-first, the author owns them and the contract was derived from those very declarations, so it was a redefinition. It emits forward declarations and the codec. Also: `jsonReturn` is set by the front end for any map return, which no longer implies the C++ type IS nlohmann::json now that a typed map is std::map<std::string, T> — checking the flag before the spelling emitted `result.dump()` on a std::map. The spelling decides. Scalars keep their nlohmann accessor verbatim rather than routing through the codec: `.get<int64_t>()` TRUNCATES a float instead of throwing, and the conformance matrix pins that leniency (hostile/int/fractional expects 3 from 3.7). Changing it would silently move behaviour something depends on. The pinned-rejection test for [bstr] is INVERTED rather than deleted — the cell it pinned still matters, only its answer changed — plus new tests for the non-tstr map key rejection and for declared-vs-undeclared records. 183 tests. Every existing module still builds; test_fullapi_cpp is unchanged. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(records): only structs the API mentions become contract types Teaching the impl-header parser to read `struct` (previous commit) published EVERY struct in a header as a contract `type`. Two production modules already carry private helpers: openmetrics-module struct ModuleSource (namespace scope, internal) logos-package-manager struct PendingAction (PRIVATE, inside the class) Both were being published — a module's interface changing as a side effect of an internal refactor, which is not something deriving a contract from a header may do. PendingAction was published WRONG as well: its fields carry trailing `// comments`, the field regex requires a line ending in ';', and the unmatched fields were silently dropped. A record with a partial field list is worse than no record, because it looks like a contract. A struct now earns its place by appearing in a method or event signature — transitively, since a published record's own fields may name others. Verified on the real headers: package-manager and openmetrics publish zero types again, while the ext provider keeps both Blob and Wrapper (Wrapper is reachable only through Blob's use in a signature). Trailing comments are stripped before the field match, so no field is dropped. Two tests over a fixture carrying both an internal namespace-scope struct and a private in-class one; 185 tests. test-modules and openmetrics both rebuild. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(doctests): the generator round-trip now shows 64-bit ints and typed records The two failing assertions in cpp-sdk-generator-roundtrip were documentation asserting the OLD behaviour, and both changes are the point of this branch: int record(int id, …) -> qlonglong record(qulonglong id, …) QVariant translate(QVariant p) -> Point translate(const Point& p, …) `record`'s `id` is a `uint64_t` in the impl header, so the old signature was handing a caller a SIGNED 32-bit value for a `uint` — the doc showed the bug. The surrounding prose was wrong too, not just the expectations, so both blocks are rewritten rather than patched: * Flow 3 now states the mapping as int->qlonglong / uint->qulonglong and says why (LIDL int/uint are int64_t/uint64_t in every other binding), pointing at `record` as the worked example. * The composite section claimed "records and optionals surface as QVariant". Records now generate a struct, `[Point]` a QList<Point> and `{tstr: Point}` a QMap<QString, Point>; maps of `any`, optionals and bare `any` still cross untyped and stay QVariant/QVariantMap — a record has a declared shape, those do not. The new text draws that line explicitly. Expectations added for `struct Point` and `Point bounds(const QList<Point>&…)` so the record path is pinned in the doc, not just described. Verified the way CI runs it — `--release-for logos-cpp-sdk=feat/qt-64bit-numerics`, which is what makes `{release}` resolve to this branch instead of master: 10 passed, 0 failed. (A plain local run builds master and is not representative — that is why it still showed the old signatures.) outputs/ regenerated; the diff also picks up unrelated pre-existing drift where the committed Markdown had fallen behind the spec. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com> |
||
|
Dario LipicarandClaude Opus 4.8
|
f0fe8cbfeb |
Make the base SDK Qt-free: Qt developer layer moves to logos-qt-sdk (#83)
* Extract the protocol layer into logos-protocol; consume it as a flake input The transport/token/IPC layer (transports incl. QRO + plain TCP/TLS, consumer core LogosAPIClient/LogosAPIConsumer with the capability auto-requestModule flow, ModuleProxy, token manager, QVariant<->JSON conversion, the abstract LogosProviderObject interface) now lives in the logos-protocol repo behind the versioned lp_* C ABI. This SDK keeps the typed C++ developer layer (LogosAPI, provider base classes + Qt provider glue, module context, code generator) and still compiles the protocol sources INTO liblogos_sdk.a from the flake input, so the installed artifact (archive symbols, include/ + include/cpp layouts, cmake config) stays byte-compatible: existing consumers need no changes. Public headers are unchanged; logos_provider_object.h keeps its name and now re-exports the abstract interface from logos_provider_interface.h. Transport/protocol component tests moved to logos-protocol with the code; the remaining sdk/generator/experimental suites are unchanged (432/432 green against the local protocol checkout). * lock: add logos-protocol input * Make the base SDK Qt-free: move the Qt developer layer to logos-qt-sdk LogosAPI, LogosAPIProvider, LogosProviderBase/LOGOS_PROVIDER macros, the QObject provider glue (QtProviderObject) and the legacy PluginInterface (core/interface.h) move to the new logos-qt-sdk repo. The protocol sources are no longer compiled into a monolithic archive — consumers link logos-qt-sdk (which layers on logos-protocol) instead. What remains here is header-only std C++: logos_module_context.h, logos_result.h (StdLogosResult), logos_json.h — exported as the CMake INTERFACE target logos-cpp-sdk::logos_headers — plus the code generator (a build-time tool; its introspection mode now includes logos_provider_interface.h from logos-protocol, where LogosProviderPlugin moved). Mechanically verified Qt-free: the logos-cpp-lib / logos-cpp-include closures contain only nlohmann_json. Tests: 245/245 (module-context std suite + generator + experimental). * fix: accept the installed source-export layout in the protocol-root check The fail-fast only tested <root>/cpp/logos_protocol.h, but the LP_SRC selection right below (and the error message itself) support the installed export layout <root>/include/cpp as well. Pointing LOGOS_PROTOCOL_ROOT at an installed export tripped the FATAL_ERROR before that fallback could apply. Caught by Copilot review on #82. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * lock: pin logos-protocol to the qt-free-split branch head The Qt-free SDK (and the cdylib backend stacked on it) reference LogosProviderPlugin from protocol's logos_provider_interface.h, which lands on feat/qt-free-split — the P1-branch pin no longer compiles standalone. Temporary — drop when the chain PRs merge. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * doctest: pin the logoscore runtime via its {release} placeholder The spec built logoscore-cli at bare master with only the cpp-sdk inputs overridden — master's stack cannot compile against the qt-free SDK, so the suite failed on the chain branches. With the placeholder, CI's --release-for pins expand it to the workspace's logoscore commit (and local runs without a pin still fall back to master, unchanged). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * doctest: override the nested module builders to {release} too capability_module (via logoscore's lock) and the cloned accounts module resolve module-builder from their own locks — pre-split revs whose LogosModule.cmake still detects the SDK by logos_api.h, which the qt-free SDK no longer ships ('logos-cpp-sdk not found'). Overriding the builder itself to the workspace-pinned chain rev (keeping the nested cpp-sdk override) builds both modules with the split-aware builder. Verified end-to-end locally with the exact doctest command. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * doctest: apply the {release} + nested-builder overrides to all three specs The runtime spec got the treatment in 210eea1; the composition and worker-thread specs have the same logoscore/module build commands and failed identically (pre-split builders from the modules' own locks). All executed run: blocks now pin logoscore-cli{release} and override the nested module builders to logos-module-builder{release}; the displayed code_block: variants stay in their generic master form. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * codegen: typed wrappers throw on call failure; dispatch catches escapes Generated sync client wrappers call the new err-out invokeRemoteMethod overload and throw logos::LogosCallError when the call fails (e.g. the bound module is missing) — previously the empty QVariant silently degraded to the return type's default and a caller could not tell failure from a legitimate 0 / "". Both generators (legacy + LIDL), both API styles. Async paths unchanged. Generated provider dispatch (universal qt glue + LOGOS_PROVIDER) wraps the method body in a catch-all that logs and returns an invalid QVariant — an escaped exception becomes an ordinary METHOD_FAILED instead of unwinding through Qt event dispatch and killing the module process. * codegen: CallError out-param instead of throwing wrappers Per review, the generated sync wrappers expose the error channel as an optional trailing parameter — add(a, b, &err) — rather than throwing: explicit, stateless, works on temporaries, and existing call sites compile unchanged (they keep default-on-failure, now with a qWarning so failures are visible in the module log). The dispatch catch-all from the previous commit stays: it contains author exceptions, it doesn't introduce any. * glue: fire onContextReady AFTER modules()/event wiring The generated onInit set the context (which fires the impl's onContextReady hook) before constructing the LogosModules aggregate and wiring typed event emission — so an impl doing its documented one-time setup there (typed dependency calls, event subscriptions) dereferenced a null aggregate and crashed the module process (signal 11). Found by the first module to subscribe to a dependency's typed event from onContextReady. Context now goes last. * ci: run workflows on stacked PRs + workflow_dispatch Both workflows filtered pull_request to master-based PRs, so stacked PRs (feat/qt-free-sdk -> feat/extract-logos-protocol, feat/cdylib-authoring -> feat/qt-free-sdk) ran NO checks at all. Drop the base-branch filter for pull_request and add workflow_dispatch for manual runs. Same fix as logos-module-builder 232b8a2. * lock: protocol at the typed-requestModule port (3de5398) * ci: chain pins for the doc-tests (drop at merge) In repo CI only cpp-sdk's {release} is the commit under test — logoscore-cli and module-builder expanded to master, which doesn't link against the chain SDK the specs override in ('Build the CLI with the SDK override' failed on every run since the stacked-PR triggers were enabled). Pin both to the extraction-chain heads; the workspace pipeline is unaffected (it pins every repo itself). * generator: distribute the LIDL frontend for external generators First step of moving ALL Qt glue emission out of this repo into logos-qt-sdk's logos-qt-generator (cpp-sdk's generator keeps only the Qt-free outputs: std typed wrappers, logos_sdk umbrella, cdylib impl-exports, LIDL derivation). - Shared emit helpers (lidlToPascalCase, lidlTypeToQt, lidlTypeToStd, lidlIsStdConvertible) move to a new lidl_emit_common.{h,cpp} unit, used by both generators. - The frontend set (AST, lexer, parser, serializer, validator, impl-header parser, emit-common) is installed under share/lidl-frontend/ — the qt generator compiles these sources in directly, so the two tools share one frontend without a binary ABI. * lock: protocol#3 merged — pin advances to protocol master --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |