mirror of
https://github.com/logos-co/logos-test-modules.git
synced 2026-08-27 10:11:12 +00:00
Dario LipicarandClaude Opus 5
966323d1b3
* fix(conformance): the bstr leniency divergence is closed Re-pins onto the deduped codec and flips the one cell it moves. logos-module-builder -> 72eb720 (#172 — protocol 4ee85b2, cpp-sdk 5f63af6) [bstr]/lenient-plain-string: both providers now ACCEPT a plain string where the contract says [bstr]. Before the unification the C++ side refused, because a [bstr] ELEMENT went through the strict codec the cdylib backend emitted while a SCALAR bstr went through a lenient one — so echoBytes("hi") succeeded and echoBytesList(["hi"]) threw, inside one module, for the same type. One function now. Still pinned per provider, because the ECHOED FORM differs and that is a different thing: C++ decodes to real bytes and re-emits the canonical tag, while Rust binds [bstr] as an untyped Value and echoes the input verbatim. A representation difference from an untyped binding, not a disagreement about what the value is. Recording it as such rather than collapsing it to a single expectation that would hide the binding asymmetry. Measured on merged pins with no overrides: exactly the one cell predicted when the baseline landed (#31) moved, and nothing else. ext 44/44 differential 20; full_api 170 pass / 2 xfail differential 80. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * probe(qt-consumer): throwaway module proving interface_dependencies works on a Qt api-style module test-qtbind-probe-module is type: core with NO `interface` key, so the backend picks apiStyle=qt. It declares interface_dependencies on full_api (a .lidl copy of the shared contract) plus concrete dependencies on both providers, forwards echoInt, and re-emits intEvent. Phase-1 throwaway: delete once the real Qt-typed proxy lands. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * feat(conformance): a QT-TYPED consumer of full_api — the matrix's third surface The matrix had two providers and one consumer coordinate. The existing proxies do not add another: `interface: "universal"` selects apiStyle=lp and `interface: "cdylib"` selects the Rust client, so both bypass the Qt generated wrappers. Replaying the whole case table through them moved 2 cells of 86. test_fullapi_qtproxy is `type: core` with NO `interface` key, which selects apiStyle=qt (mkLogosModule.nix picks lp only for universal non-ui_qml), so `modules().bind_full_api(name)` hands back a Qt-typed wrapper — QString / QByteArray / qlonglong / qulonglong / QVariantList / QVariantMap / LogosResult. It forwards all 33 full_api methods and re-emits all 15 events, so the matrix replays cases.json through it unchanged. Two things only this surface reaches, both now measured end-to-end under logoscore against BOTH providers: * M3. A one-key `_bytes` map is reinterpreted as bytes in logos_json_convert.cpp on the way to a Qt consumer. At a `{tstr: any}` slot that is total loss: `echoMap {"_bytes":"AID_"}` answers `{}`, where the same call at the provider round-trips and looks green. * The generated ASYNC return table converts with `qvariant_cast<T>(v)` where the SYNC one uses `_result.toT()`. syncProbe() and probeAsync() / getAsyncProbe() render the same 18 calls through both tables in one format so they are diffable; today they agree, and now a change to either is visible instead of silent. Also: * check_contract_copies.py grows from 5 to 8 tracked copies and learns a third parse kind. The Qt copy is compared by COMPATIBILITY, not equality, because the Qt style is not 1:1 with LIDL — `QVariantList` is the one type behind `[any]`, `[int]`, `[uint]`, `[float64]` and `[bool]`. Verified live against four injected drifts (missing method, wrong type, wrong arity, unknown spelling); a Qt provider has no declared event block, so its events stay compiler-enforced rather than silently "checked". * Subscriptions are per-provider-once and drop deliveries from a provider that is no longer bound. Without this, re-binding stacked callbacks and every event arrived three times — measured, and it would have multiplied every event-position cell. * Drops test-qtbind-probe-module, the phase-1 throwaway; every finding it established is reproduced by this module. It stays in history at432812a. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * feat(conformance): sync/async as an axis, and what the Qt consumer actually changes The Qt proxy forwarded every method through the SYNC generated wrapper only. That left the async table — which converts with `qvariant_cast<T>(v)` and substitutes a default on an invalid QVariant, where sync uses `_result.toT()` — driven by nothing but an 18-call fixed probe. `useCallMode sync|async` makes it an axis: all 33 forwarded methods route through whichever table is selected, so the whole case table replays twice. `lastCallStatus()` reports `ok-sync`/`ok-async` per call, because a mode switch that silently no-ops would make the async half a duplicate of the sync half and 86 green cells would prove nothing. Async waits by polling a mutex-guarded slot and pumping, never a cross-thread QEventLoop::quit(); a completion is delivered on whatever thread the transport uses. Events keep no mode — a subscription is a callback either way, and the header says so rather than leaving it a gap. Measured over both providers: * sync vs async: 0 deltas. The difference is real in the generated source and is now driven; it does not currently change an answer. * M3 is MEASURABLE and moves from `unmeasurable` to an ordinary xfail. The collision is symmetric, so an `any` echo is green no matter what happens in between — the observation is a TYPED slot: echoMap({"_bytes":"aGk"}) arrives {}. Confirmed head-on by the proxy's type-tagged event rendering, which says QByteArray where the sender put a map. * Q1 is new and was not suspected: the generated Qt dispatch coerces a hostile argument into the declared parameter type before the method body runs, so echoUint(-1) answers 18446744073709551615 where every other surface answers dispatch_failed. Nine cases. Argument validation is not something a Qt-typed module can rely on. Both attributed by replaying the same table through the lp proxy — a hop of the same shape on a non-Qt client — which rejects all nine and preserves the map. So it is the api style, not the extra hop. Registry: `consumers` is now REQUIRED on every xfail entry, with no default. An entry that does not say which surface it was measured on manufactures failures on the surfaces it was not, and that already happened once on the provider axis. M4-residual gains its two Qt consumers only after measuring them. `skip[]` was declared and never read. Reading it found its own first bug: the pattern `uint/boundary` matched nothing, because the case had been split in two and the entry was never updated. A registry that is not executed decays into a comment. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * probe(uiqml): the smallest universal ui_qml module, to settle which dispatch it gets test-uiqml-probe-module is `type: ui_qml` + `interface: universal`, forwarding the eight methods behind the nine hostile-argument cases to test_fullapi_cpp. Built and run to answer, empirically, whether the Qt argument-coercion defect survives the migration to `interface: universal` for a UI backend. Throwaway: delete once the question is settled. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * probe(qt-consumer): throwaway module proving interface_dependencies works on a Qt api-style module test-qtbind-probe-module is type: core with NO `interface` key, so the backend picks apiStyle=qt. It declares interface_dependencies on full_api (a .lidl copy of the shared contract) plus concrete dependencies on both providers, forwards echoInt, and re-emits intEvent. Phase-1 throwaway: delete once the real Qt-typed proxy lands. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * feat(conformance): a QT-TYPED consumer of full_api — the matrix's third surface The matrix had two providers and one consumer coordinate. The existing proxies do not add another: `interface: "universal"` selects apiStyle=lp and `interface: "cdylib"` selects the Rust client, so both bypass the Qt generated wrappers. Replaying the whole case table through them moved 2 cells of 86. test_fullapi_qtproxy is `type: core` with NO `interface` key, which selects apiStyle=qt (mkLogosModule.nix picks lp only for universal non-ui_qml), so `modules().bind_full_api(name)` hands back a Qt-typed wrapper — QString / QByteArray / qlonglong / qulonglong / QVariantList / QVariantMap / LogosResult. It forwards all 33 full_api methods and re-emits all 15 events, so the matrix replays cases.json through it unchanged. Two things only this surface reaches, both now measured end-to-end under logoscore against BOTH providers: * M3. A one-key `_bytes` map is reinterpreted as bytes in logos_json_convert.cpp on the way to a Qt consumer. At a `{tstr: any}` slot that is total loss: `echoMap {"_bytes":"AID_"}` answers `{}`, where the same call at the provider round-trips and looks green. * The generated ASYNC return table converts with `qvariant_cast<T>(v)` where the SYNC one uses `_result.toT()`. syncProbe() and probeAsync() / getAsyncProbe() render the same 18 calls through both tables in one format so they are diffable; today they agree, and now a change to either is visible instead of silent. Also: * check_contract_copies.py grows from 5 to 8 tracked copies and learns a third parse kind. The Qt copy is compared by COMPATIBILITY, not equality, because the Qt style is not 1:1 with LIDL — `QVariantList` is the one type behind `[any]`, `[int]`, `[uint]`, `[float64]` and `[bool]`. Verified live against four injected drifts (missing method, wrong type, wrong arity, unknown spelling); a Qt provider has no declared event block, so its events stay compiler-enforced rather than silently "checked". * Subscriptions are per-provider-once and drop deliveries from a provider that is no longer bound. Without this, re-binding stacked callbacks and every event arrived three times — measured, and it would have multiplied every event-position cell. * Drops test-qtbind-probe-module, the phase-1 throwaway; every finding it established is reproduced by this module. It stays in history at432812a. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * feat(conformance): sync/async as an axis, and what the Qt consumer actually changes The Qt proxy forwarded every method through the SYNC generated wrapper only. That left the async table — which converts with `qvariant_cast<T>(v)` and substitutes a default on an invalid QVariant, where sync uses `_result.toT()` — driven by nothing but an 18-call fixed probe. `useCallMode sync|async` makes it an axis: all 33 forwarded methods route through whichever table is selected, so the whole case table replays twice. `lastCallStatus()` reports `ok-sync`/`ok-async` per call, because a mode switch that silently no-ops would make the async half a duplicate of the sync half and 86 green cells would prove nothing. Async waits by polling a mutex-guarded slot and pumping, never a cross-thread QEventLoop::quit(); a completion is delivered on whatever thread the transport uses. Events keep no mode — a subscription is a callback either way, and the header says so rather than leaving it a gap. Measured over both providers: * sync vs async: 0 deltas. The difference is real in the generated source and is now driven; it does not currently change an answer. * M3 is MEASURABLE and moves from `unmeasurable` to an ordinary xfail. The collision is symmetric, so an `any` echo is green no matter what happens in between — the observation is a TYPED slot: echoMap({"_bytes":"aGk"}) arrives {}. Confirmed head-on by the proxy's type-tagged event rendering, which says QByteArray where the sender put a map. * Q1 is new and was not suspected: the generated Qt dispatch coerces a hostile argument into the declared parameter type before the method body runs, so echoUint(-1) answers 18446744073709551615 where every other surface answers dispatch_failed. Nine cases. Argument validation is not something a Qt-typed module can rely on. Both attributed by replaying the same table through the lp proxy — a hop of the same shape on a non-Qt client — which rejects all nine and preserves the map. So it is the api style, not the extra hop. Registry: `consumers` is now REQUIRED on every xfail entry, with no default. An entry that does not say which surface it was measured on manufactures failures on the surfaces it was not, and that already happened once on the provider axis. M4-residual gains its two Qt consumers only after measuring them. `skip[]` was declared and never read. Reading it found its own first bug: the pattern `uint/boundary` matched nothing, because the case had been split in two and the entry was never updated. A registry that is not executed decays into a comment. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * spike(qtproxy): the Qt full_api surface as a veneer over the lp wrapper Phase-1 spike for "make the Qt consumer a skin over the lp one". Adds, beside the generated Qt `FullApi`, a `FullApiVeneer` with the SAME 33 sync + 33 async + 15 typed-event surface whose every body is convert-args / delegate / convert- return against `FullApiLp` (verbatim `--api-style lp` generator output, class renamed). `useWrapper("generated"|"veneer")` routes the whole forwarded case table through either one against the same provider in the same process, so the two are diffed rather than reasoned about. The constraint under test is that the veneer writes NO conversion of its own. It does not: the entire adapter is four function templates over logos::qvariantToNlohmann / nlohmannToQVariant (logos-protocol logos_json_convert.cpp:38,129) and logos::toJson / fromJson (logos_codec.h), plus two specialisations for `result` — the one LIDL type whose canonical converters do not exist and would have to be added to logos-protocol (json -> LogosResult) and logos-cpp-sdk (StdLogosResult -> json). Measured, both providers, sync and async: identical on every cell except `result`, where the std hop turns a null error into "". `makeResultNoStdHop` renders the same call without the std intermediate and matches the generated wrapper exactly, so the divergence is the hop, not the veneer. `tokenProbe` records the prerequisite the spike hit first: a Qt-style plugin has TWO TokenManager singletons (qtTM=100889ec0 lpTM=102fb58a0 same=NO qtCap=yes lpCap=no), because LogosAPI is constructed in the host image while lp_client_create is compiled into the plugin. Every lp call from a Qt plugin is therefore unauthorized and returns a default value. The cdylib backend seeds its copy via logos_module_accept_token -> lp_token_save; the Qt backend has no such hook. veneerTarget() does that seeding inline so the type matrix could run. Spike only — not a proposal to merge. * test(qtproxy): A/B the two implementations of one Qt surface, in one process The hand-written spike veneer is replaced by verbatim `logos-qt-generator --backend consumer` output (class FullApiVeneer, bound mode). `useWrapper` still swaps every forwarded call between the legacy generated Qt wrapper and the veneer, against the same provider in the same process, so the two are diffed rather than reasoned about. Result: identical on every cell — 40 method/probe cells and 15 events, sync and async, against both test_fullapi_cpp and test_fullapi_rust. Including uint64 max, int64 min, an int outside double's exact range, bytes carrying 0x00/0x80/0xFF, and `any` as string/number/object. The one cell that diverged in the spike is closed. `makeResult(true)` rendered `e=s:` through a veneer that hopped via StdLogosResult (whose error is a std::string and so cannot be absent) and `e=-` through the generated wrapper. This wrapper converts Qt <-> canonical JSON and calls the lp client directly — no std intermediate — and now renders `e=-` both ways. Everything the spike did by hand in veneerTarget() (owning the LpClient and its subscriptions per provider, seeding the plugin-side TokenManager) is gone: it lives in logos-qt-sdk's LpBridge, which is where generated code can reach it. The veneer is now constructed exactly like target() — same two arguments. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * test(qtproxy): the same consumer module, built against each Qt implementation Phase 2 A/B'd the two Qt consumer implementations by compiling BOTH into the module: the build's generated `FullApi` plus a committed copy of the veneer emitter's output under a second class name. That proves the two agree; it does not prove the veneer is reachable through the BUILD, and every call site had to be made generic over the wrapper type to switch between them. `mkQtProxy` / `mkFullapiUi` take `qtConsumerCodegen`, so each module is built TWICE from one `src`, differing only in which generator emitted full_api_api.{h,cpp}: legacy logos-cpp-generator --general-only --api-style qt veneer logos-qt-generator --backend consumer (over the legacy output) Not one line under src/ differs between the two builds — the premise of the whole change is that a Qt call site cannot tell. The generated trees differ in exactly the three full_api_api files and nothing else, and both plugins link. (The phase-2 in-process copy stays for now as a control: `useWrapper veneer` still reaches it, and it agrees with the build's wrapper in the veneer build.) Measured under logoscore against both providers, every call routed through `modules().bind_full_api(provider)`: * 212 cells (2 providers x sync/async x 53 type cells incl. every event) identical between the two builds. * 36 failure-path cells (bound to a module that does not implement the contract, so the invalid-QVariant branch runs) identical between the two builds AND between sync and async in each. * Against a DIRECT call on the provider, 35 of 36 cells agree in both builds. The one that does not is registry entry M3 and it is unchanged by this work: `echoMap({"_bytes":"aGk"})` is `{}` through a Qt consumer either way. The runtime proof that `bind_full_api` really is the lp path in the veneer build is the token probe. After one warm-up call: legacy qtCap=yes lpCap=no qtProv=yes Qt client minted a target token veneer qtCap=yes lpCap=yes qtProv=no LpBridge mirrored the capability token; no Qt target token exists test_fullapi_ui gets the same treatment: a second, independently written Qt consumer, binding a `.h` interface rather than a .lidl contract. Build only — it is a UI plugin and needs the app to drive. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * spike(uiqml): the .rep slot type as the author's declared choice of transport Each covered type (int, uint, [uint], {tstr:any}, any) now has TWIN slots that forward to the SAME test_fullapi_cpp method: a native one taking the Qt type the LIDL type maps to, and a `*Json` one taking a QString of canonical JSON text. bstr has only the QString form — QML has no byte-array literal, so there is nothing for a native slot to receive. The native bodies forward whatever QML already coerced. The QString bodies run one decode — nlohmann parse, logos::fromJson<T>, logos::toJson<T>, logos::nlohmannToQVariant — so the value handed to the wrapper is produced only by logos-protocol's canonical converters and an out-of-domain input becomes a visible REJECTED answer instead of a repaired one. test_fullapi_cpp gains an env-gated (LOGOS_FULLAPI_TRACE) stderr dump of what its own typed parameter holds. A return value cannot answer "which value crossed" — a coerced value echoes back perfectly — and ModuleProxy deliberately never logs arguments, so the observation has to happen at the provider. Throwaway: delete once the transport question is settled. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * spike(uiqml): measure both transports against a non-QML reference Phase 2 of the transport-choice spike: drive every twin and report what the PROVIDER'S IMPLEMENTATION received, against the same canonical JSON delivered by a consumer with no QML in the path. Harness changes that the measurement needed: * answers carry a monotonic sequence number, so the view advances on "my case was answered" instead of a guessed settling delay, and two consecutive identical answers still move the property (a guarded setter would otherwise suppress the second and the view would re-report the first); * a caseGroup property from LOGOS_UIQML_CASES, because one case kills the process and everything after it goes unmeasured — every composite `any` case now runs in a group of its own, so each is measured rather than inferred from the case before the crash; * doEchoBytesNative, so the design table's empty "native bstr" cell is measured instead of asserted; * every echo in test_fullapi_cpp is traced, not just the five the first pass used — an uninstrumented method makes a successful call look like a rejected one. Findings, in the log and reproducible: * the QString path is exact for int, uint, [uint] and bstr — all 256 byte values, verified by decoding what the provider received with an independent base64 implementation rather than by string equality; * native `any` is not lossy, it is fatal: every composite (array, object, object with a big uint, object shaped like a bytes tag) SIGSEGVs in QtRemoteObjects' isSequentialGadgetType. Scalars pass; * a native [uint] whose element is out of domain comes back as an EMPTY list with no error — the provider's dispatch did reject it, but the generated Qt wrapper's return type cannot carry that refusal; * QML CAN build a QByteArray from a JS array, exactly, for 0..255 — but truncates 256 to 0, -1 to 255, 1.5 to 1 and "a" to 0, silently; * a one-key map named `_bytes` survives on neither path (registry M3). Throwaway: delete once the transport question is settled. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * test(basic): argument decoding, and the registry entry it closes The "Type coercion" block asserted the defect: addInts(3.7, 1.2) expecting "Result: 5", with a comment saying QtProviderObject "coerces them correctly". Qt rounded both arguments before the method body ran, so the module added numbers nobody sent. That row now asserts the refusal, and six more shapes join it, each naming the value a coercion used to invent. byteArraySize(42) and urlToString(...) are kept as NON-refusals on purpose: bstr keeps the codec's documented lenient form, and a type with no LIDL counterpart keeps Qt's own conversion. Both would break if the rule were "reject anything inexact" — as would the four whole-valued-float rows above them, which is why the rule is the codec's. conformance: Q1 loses its six scalar/container cases and keeps the three typed-numeric-array ones, whose element type a C++ signature cannot express. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * conformance: Q1b — the two cells that inverted instead of closing The Q1 fix was recorded as closing six of the nine. Measured against both providers, four closed outright; the other two — echoList("notalist") and echoMap(5) — closed against the Rust provider only. The C++ cdylib provider ACCEPTS a scalar in a [any]/{tstr:any} slot (cases.json pins it with expect_by_provider), and the Qt proxy can no longer reproduce that answer because its own dispatch refuses the argument before forwarding it. So the divergence inverted rather than closing: the Qt side used to be the lenient one and is now the strict one, and the outlier is the C++ provider. Retiring those two case-ids wholesale turned four green cells red — the matrix reported `fail` on (hostile/[any]/scalar, test_fullapi_cpp, qtproxy-sync|async) and the same for {tstr:any}, plus their py-vs-qtproxy differentials. Q1b registers exactly that, scoped to the C++ provider, so the Rust half stays unregistered and reports xpass if it ever regresses. The fix named is the cdylib container decode — widening the Qt rule back out to match the lenient provider would re-open the cells Q1 just closed. Measured, four runs of the same table on the same daemons: pre-fix code + pre-fix registry 470 pass / 86 xfail / 0 fail (green) post-fix code + pre-fix registry 470 pass / 46 xfail / 20 xpass / 0 fail post-fix code + retired registry 490 pass / 22 xfail / 4 FAIL post-fix code + this registry 490 pass / 26 xfail / 0 fail (green) Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * chore: re-pin logos-module-builder (brings the Qt argument-validation fix) This PR's new hostile-argument assertions expect dispatch_failed, but its pins predated the fix that produces it: test-modules consumes both SDKs only through logos-module-builder, which pinned qt-sdk a5874fe and cpp-sdk 5f63af6 -- both older than logos-qt-sdk#22 and logos-cpp-sdk#121. logos-module-builder -> 43c1c01 (module-builder#176), which carries qt-sdk 7832345 and cpp-sdk 5d0a99a. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * test: drop the pure-Qt-provider refusal assertions; they are out of scope Seven assertions expected dispatch_failed from test_basic_module, a pure Qt provider, where the Qt dispatch coerces the argument instead. They are removed, not fixed: hardening a Qt-typed PROVIDER is no longer a goal -- the Qt surface we intend to be correct is the CONSUMER side. Leaving them red would misreport a deliberate scope decision as a defect; registering them as xfail would imply a fix is coming. The coercion itself is unchanged and is still asserted by the neighbouring rows, which describe it rather than refuse it. Registry, correspondingly -- both stay in `xfail` because the driver reads only that section and moving them would turn the cells red, but the rationale is corrected from "not yet fixed" to what is actually true: Q1 SPLIT. The provider half (a typed-numeric-array element is not validated) is now accepted. The CONSUMER half is a live defect and is why the entry stays: the Qt consumer flattens the provider's rejection envelope into an empty list on the return, so a caller sees [] instead of dispatch_failed. Losing the error is a consumer bug whatever the provider did. Q1b OUT OF SCOPE, won't fix. The cell exists because the Qt proxy's own provider dispatch refuses an argument before forwarding it. Kept registered because the cells still diverge and an unregistered divergence fails the run. Integration check: 0 failures. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com>