Dario LipicarandClaude Opus 5
a5d4790809
* conformance(ext): the consumer axis, and what it found
The ext table ran 44 cases x 2 providers x ONE consumer. Its own comment said
so. Everything the lossless Qt mapping widened lives in THIS contract and was
measured by nobody: `full_api` has no record, no typed container and no
optional in it, so `FullApiExt::Blob`, `QList<Blob>`, `QMap<QString, Blob>`,
`QList<QByteArray>`, `QList<QList<qlonglong>>`,
`QMap<QString, QList<QByteArray>>` and `std::optional<QString>` are reachable
from `full_api_ext` and from nowhere else — and each of them is emitted as an
element LOOP rather than handed to the codec whole, i.e. a body of generated
code the two existing qtproxy points cannot execute.
test-fullapi-ext-qtproxy-module is the same two-key shape as
test-fullapi-qtproxy-module against that contract: `interface: universal` for
the provider half (a header-first cdylib whose records are C++ structs in the
impl header) and `codegen.consumer_api_style: "qt"` for the consumer half, so
`modules().bind_full_api_ext(name)` is the Qt-typed, origin-bound wrapper. All
15 contract methods and the one event are forwarded, in both call modes.
WHAT IT FOUND. 44 x 2 x 3 = 264 cells: 217 pass, 47 xfail, 0 fail.
* every widened spelling ROUND-TRIPS. Blob, Wrapper, Opt, [Blob], [Opt],
{tstr: Blob}, {tstr: bstr}, {tstr: int}, {tstr: tstr}, {tstr: [bstr]},
[[int]] and [bstr] are green on all six provider x consumer surfaces.
* `hostile/Optional/scalar/wrong-type` on test_fullapi_ext_cpp is the sharp
one, and it is new: the provider's class-E REJECTION arrives through the Qt
wrapper as a successful call returning
{"success": false, "value": null, "error": null} — the same value its three
well-formed inputs produce. Registered as
ext-optional-return-shape-through-the-qt-proxy, with the cause (the two
providers still disagree about echoOptional's return) and the two halves of
the fix kept apart.
* the table's one DECLARED divergence stops diverging through a proxy: the hop
canonicalises `["hi"]` to tagged bytes before the Rust provider is offered
it, so its leniency is never exercised. Registered rather than smoothed with
a per-consumer expectation, which would pin an accident as a contract.
* pre-99-null-is-not-an-error and B-arity-overflow reproduce IDENTICALLY on
both new consumers, which is the expected result and is why they were
widened rather than given entries of their own. B-arity-overflow gains a
fifth module and a third contract: the proxy's own derived dispatch drops
the extra argument too.
Requires the logos-qt-sdk fix described in the accompanying report: the
generated record codecs (`recToWire_Blob` / `recFromWire_Blob`) are unqualified
file-scope statics, and the umbrella amalgamates every `<name>_api.cpp` into one
translation unit — so a module holding three wrappers for one record-bearing
contract does not compile. A proxy is exactly that shape.
* conformance(ext): name the generator prerequisite where it will be hit
A reader who builds this module against logos-qt-sdk 4ab78a1 gets forty
'ambiguating new declaration' errors inside generated code, with nothing in
this repo to point at. Say what the collision is, why this module is the shape
that triggers it, and why it cannot be worked around here.
* conformance(ext): replace two claims about the proxy consumers with measurements
Two of the things this table now says about its new consumer axis were argued
rather than measured, and one of them is false.
FALSE: pre-99-null-is-not-an-error claimed failure/A answers METHOD_FAILED
"IDENTICAL on extqtproxy-sync and extqtproxy-async, on both providers". It does
not. Run three times at this fixture rev — the nix check, then the same driver
standalone twice — exactly ONE of the six proxy coordinates answered RPC_FAILED
each time, and a different one each time: extqtproxy-async/cpp, then
extqtproxy-async/rust, then extqtproxy-sync/rust. Never on py (0 of 6 samples
against 3 of 12 proxy samples). The entry now names the flake and says why the
gate is unaffected: an xfail is keyed by (case, provider, consumer), both shapes
are the same class-A defect, and neither is the `object_unavailable` that would
retire the entry, so there is no xpass risk either. failure/C is separated out
because it IS deterministic. A registry that claims a determinism it does not
have is worse than one that names the flake.
ARGUED: the two new entries ruled out cross-case pollution from call_status and
the sync/async agreement. That is suggestive, not decisive — the last round of
this work nearly registered four phantom cells that one `isolate: true` made
behave. So it was measured as an A/B with one variable: the same driver, modules
and registry, the table re-run with `isolate: true` added to the four Optional
cases and to [bstr]/lenient-plain-string and nothing else changed. All 50 cells
of those five cases are identical between the runs — same status, same value, on
all three consumers and both providers.
No case, expectation or registry KEY moves; this is the `measured` and `note`
text catching up with what was actually run.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* conformance(ext): a case for the nested int range hole, found by probing the widened path
The consumer axis landed green, so the nested widened containers were probed by
hand instead — {tstr:[bstr]} and [[int]] are the least-exercised shapes in the
system and a green table is not evidence about the inputs it does not contain.
FOUND, and now a case: `echoNestedInts([[18446744073709551615]])`.
test_fullapi_ext_cpp rejects it and names the coordinate — `expected signed
integer in range at arg0[0][0], got number`. test_fullapi_ext_rust ACCEPTS it
and echoes the exact value back, so the decode never went through an i64.
`int` is i64, this table already pins the u64 end separately in a `uint` field,
and the neighbouring positive control ([[-9007199254740993, 9007199254740993]],
both edges outside double precision, at depth 2) round-trips on all six
surfaces — so the hole is the RANGE check, not the depth.
Both proxy consumers PASS it: the proxy decodes into std::vector<std::vector<
int64_t>> before calling out, so the Rust provider is never offered the value.
The cell therefore prints as py disagreeing with both proxy points, which is the
consumer axis earning its keep in the other direction — the hop HIDES a provider
defect, and only a table carrying both surfaces can say so. Spelled as a plain
`expect`, not expect_by_provider: one contract, one of the two is wrong.
RECORDED BUT NOT GIVEN CASES, on the entry that owns [bstr] leniency: three more
shapes of the same provider disagreement, one container deeper than the case
that is registered. `{"k": ["hi"]}` and `{"k": [42]}` into {tstr:[bstr]}, and
`[[1,2]]` into [bstr]. The middle one is not leniency but a coercion — the
number 42 comes back as the two ASCII bytes of its decimal spelling — and
whether a bstr slot should coerce, reject or stay lenient is a contract decision
the ext table's owner has not made. Writing an expectation would make it by
accident. The positive control passes everywhere: 0x00/0x80/0xFF and an empty
bstr, two containers deep, byte-identical on all six surfaces.
Measured: conformance-matrix-ext at 45 x 2 x 3 — 222 pass / 48 xfail / 0 fail /
0 xpass, exit 0.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* conformance(ext): replace the falsified determinism claim with the samples
The entry said the class-A flake hit 'exactly ONE of the six proxy
coordinates each time' and 'never on py'. Across 42 samples (7 runs x 6
coordinates) both are false: 3 of 7 runs had zero, and 1 of 14 py samples
hit it.
The py sample also refutes the explanation the claim rested on — it was
attributed to a daemon holding the proxy plus both providers, and the py
daemon holds no proxy. What is established is that the ~20s acquire can
outrun the core_service RPC deadline under load, on any coordinate; the
entry no longer claims a mechanism beyond that.
Two consecutive drafts made this error, in an entry whose own last sentence
argues that a registry claiming a determinism it does not have is worse
than one that names the flake.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* chore(deps): relock module-builder for the record-codec qualification
Picks up module-builder 236a409, which pins logos-qt-sdk 3ab7421 (#43).
The ext qtproxy fixture cannot compile without it — a proxy over a
record-bearing contract collides on every record.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
13 KiB
logos-test-modules
Test modules for the Logos platform. These modules exercise every API type and
combination exposed by logos-cpp-sdk, organised into complementary modules
and a standalone thread-safety test suite:
| Module | Purpose |
|---|---|
| test_basic_module | Standalone module (no external libs, no IPC). Covers every supported parameter type, return type, argument count (0–5), LogosResult patterns, and events. |
| test_extlib_module | Wraps an external C library (libstrutil). Validates the external-library build pipeline. |
| test_ipc_new_api_module | Calls the two modules above. Validates inter-module communication, generated type-safe wrappers (sync and async), cross-module chaining, and events — all from an interface: "universal" module with no Qt in its own translation units. |
| test_dummy_module | Minimal interface: "universal" module (noop() only) — its impl derives from LogosModuleContext (it was a hand-written LogosProviderBase Qt plugin before the universal migration). Used as a binary template for the thread-safety tests — patched at the binary level to generate unique module copies. |
SDK coverage matrix
Parameter types
Spelled below as the Qt types a Qt consumer sees. The impl headers declare the
std equivalents (std::string, int64_t, bool, std::vector<std::string>,
std::vector<uint8_t>); an inbound argument is decoded by the generated cdylib
glue through logos::fromJson<T>, which has the declared element type. (It used
to arrive via toScopedQArgs, the QMetaObject dispatch helper — that path still
exists in logos-plugin-qt but only serves interface: "legacy" Q_INVOKABLE
providers, and this repo no longer has one.)
| Type | Tested in |
|---|---|
QString |
test_basic_module, test_extlib_module, test_ipc_new_api_module |
qlonglong |
test_basic_module |
bool |
test_basic_module |
QStringList |
test_basic_module |
QByteArray |
test_basic_module |
Return types (callRemoteMethod)
| Type | Tested in |
|---|---|
void |
test_basic_module |
bool |
test_basic_module |
qlonglong |
test_basic_module, test_extlib_module |
QString |
test_basic_module, test_extlib_module, test_ipc_new_api_module |
LogosResult |
test_basic_module, test_ipc_new_api_module |
QVariant |
test_basic_module |
QVariantList |
test_basic_module |
QStringList |
test_basic_module |
Universal migration — the Qt types that moved
test_basic_module and test_extlib_module used to be hand-written Qt plugins.
Both now declare interface: "universal": the impl class is plain C++ with no Qt
in it, that class IS the contract. logos-cpp-generator --header-to-lidl derives
the LIDL from its header, logos-cpp-generator --lidl ... --backend cdylib emits
the Qt-free C-ABI exports, and logos-plugin-qt's logos-qt-host-generator --lidl ... --backend cdylib emits the Qt plugin glue. (There is no LOGOS_METHOD
marker any more — a module's plain public methods ARE its API — and no
logos_provider_dispatch.cpp.)
A Qt-free header has no way to ask for a Qt-specific type, so three declared types could not be carried across. The values on the wire are unchanged — only the declared types move:
| Was (Qt plugin) | Is (universal) | Why |
|---|---|---|
int |
qlonglong |
LIDL numbers are 64-bit; the only widening in the table |
QJsonArray (returnJsonArray, makeJsonArray) |
QVariantList |
LogosList → LIDL [any]; still the same JSON array on the wire |
QUrl param (urlToString) |
QString param |
LIDL tstr; callers were serialising the URL to its string form anyway |
QUrl no longer appears anywhere in the suite, so the SDK's QUrl parameter
conversion is no longer covered by these fixtures. urlToString keeps its
behaviour — it returns the URL with the case-insensitive parts (scheme, host)
lowercased — but it now does that normalisation itself instead of borrowing
QUrl's.
Strings are UTF-8, and lengths are in characters
Every string these fixtures take or return is UTF-8, and every method that counts, indexes or reorders "characters" means one Unicode code point:
| Method | Answer |
|---|---|
test_basic_module.stringLength("héllo") |
5 |
test_basic_module.validateInput("héllo").value.length |
5 |
test_extlib_module.countChars("héllo") |
5 |
test_extlib_module.countChar("héllo", "é") |
1 |
test_extlib_module.reverseString("héllo") |
"olléh" |
A character above the BMP (an emoji, say) counts as 1. Neither of the two
answers this suite used to get is a length: the Qt modules answered
QString::length(), the number of UTF-16 units (2 for that emoji), and the
first universal port answered std::string::size(), the number of UTF-8 bytes
(4 for it, and 6 for "héllo"). Counting bytes is also why reverseString used
to produce a byte sequence that was not even valid UTF-8: in the universal port
the call failed outright at serialisation, and in the Qt module before it the
broken bytes were decoded into replacement characters. Every row above has a
regression assertion in the basic or extlib group of tests/run_tests.sh;
before those were added, every assertion touching these methods was ASCII, where
characters, UTF-16 units and bytes all agree.
uppercaseString / lowercaseString are the exception, and deliberately so:
they are ASCII-only case mappings performed by the external C library, and bytes
≥ 0x80 pass through untouched ("héllo" → "HéLLO").
Argument counts (0–5)
| Count | Method |
|---|---|
| 0 | noArgs() |
| 1 | oneArg(QString) |
| 2 | twoArgs(QString, qlonglong) |
| 3 | threeArgs(QString, qlonglong, bool) |
| 4 | fourArgs(QString, qlonglong, bool, QString) |
| 5 | fiveArgs(QString, qlonglong, bool, QString, qlonglong) |
Inter-module communication
| Pattern | Tested in |
|---|---|
Generated modules().<dep> wrappers (sync) |
test_ipc_new_api_module |
Generated <name>Async wrappers over lp_invoke_async |
test_ipc_new_api_module |
Event subscription — generated on<Event>(callback) accessors |
test_fullapi_proxy, test_fullapi_proxy_rust, test_fullapi_qtproxy, test_fullapi_ext_qtproxy |
Event emission — typed logos_events: |
test_basic_module, test_basic_module_cpp, test_ipc_new_api_module |
| Cross-module chaining | test_ipc_new_api_module |
Qt-typed consumer (consumer_api_style: "qt") |
test_fullapi_qtproxy |
Qt-typed consumer of RECORDS and typed containers (FullApiExt::Blob, QList<QByteArray>, QMap<QString, QList<QByteArray>>, QList<QList<qlonglong>>, std::optional<QString>) |
test_fullapi_ext_qtproxy |
Running tests
The integration test suite exercises the core modules (test_basic_module,
test_basic_module_cpp, test_context_module_cpp, test_extlib_module,
test_ipc_new_api_module, and the full_api provider/proxy chain) against a
long-lived logoscore daemon:
# From logos-test-modules
nix build .#tests -L
# From the workspace root
ws test logos-test-modules
Running specific test groups
Use TEST_GROUPS to run a subset of tests. Available groups: basic,
basic-cpp, context-cpp, extlib, fullapi, ipc-new-api, multi,
errors, unit-new-api.
(The ipc, async and unit groups were removed in af567c1 together with
test_ipc_module, the last interface: "legacy" provider in the repo — its
surface is mirrored 1:1 by test_ipc_new_api_module, and the async assertions
moved to the ipc-new-api group before the module was deleted.)
Not every group has a dedicated flake check; the ones that do are
ipc-new-api-tests, fullapi-tests, unit-tests-new-api,
thread-safety-tests and qml-modules. tests runs every group.
# IPC new-API tests — the Qt-free `interface: "universal"` consumer
nix build .#checks.aarch64-darwin.ipc-new-api-tests -L # macOS ARM
nix build .#checks.x86_64-linux.ipc-new-api-tests -L # Linux
# full_api provider + proxy chain
nix build .#checks.aarch64-darwin.fullapi-tests -L # macOS ARM
nix build .#checks.x86_64-linux.fullapi-tests -L # Linux
Unit tests (mock-based)
Unit tests use the SDK's mock transport layer — no real IPC or logoscore needed.
They verify that module methods call the expected inter-module APIs with the correct
arguments and handle return values properly.
There is exactly one such check now, unit-tests-new-api — see the next section
for how to run it. (A second one, unit-tests, built test_ipc_module's Qt-typed
binary; the module and the check were both removed in af567c1, so
.#checks.<system>.unit-tests no longer exists.)
Temporary note — running from the workspace with local
logos-cpp-sdkchanges:nix build 'path:./repos/logos-test-modules#checks.aarch64-darwin.unit-tests-new-api' -L \ --override-input logos-module-builder/logos-cpp-sdk path:./repos/logos-cpp-sdk(Only
logos-module-builder/logos-cpp-sdkis needed — there is no directlogos-cpp-sdkinput.)
Unit tests — test_ipc_new_api_module (mock-based)
Unit tests for test_ipc_new_api_module, an interface: "universal" consumer:
its impl is a plain C++ class deriving from LogosModuleContext and the contract
is derived from its header by the generator. (It was written against
LogosProviderBase + the LOGOS_METHOD marker; both that base class as an
authoring surface and the marker are gone — a module's plain public methods are
now its API.) Same mock transport as above — no real IPC or logoscore needed.
# Standalone (from the logos-test-modules repo)
nix build .#checks.x86_64-linux.unit-tests-new-api -L # Linux
nix build .#checks.aarch64-darwin.unit-tests-new-api -L # macOS ARM
From the workspace root:
# Via workspace flake (propagates local overrides)
nix build ".#checks.aarch64-darwin.logos-test-modules--unit-tests-new-api" \
--override-input logos-cpp-sdk path:./repos/logos-cpp-sdk \
--override-input logos-liblogos path:./repos/logos-liblogos \
--override-input logos-module-builder path:./repos/logos-module-builder \
--override-input logos-test-modules path:./repos/logos-test-modules -L
Thread-safety tests
Exercises logos_core under concurrent load using GTest.
The test_dummy_module binary is used as a template: 100 copies are generated by
patching the embedded plugin name at the binary level, giving each thread a distinct
real Qt plugin to work with.
All operations go through the public logos_core C API. Tests cover:
- Process —
logos_core_process_moduleconcurrently on disjoint and shared module sets - Query under writes —
logos_core_get_known_modules/logos_core_get_loaded_modulescalled by reader threads while writers are processing or loading - Load —
logos_core_load_module(name, with_dependencies)on disjoint and shared sets, with the flag both ways, including unknown-name fast-failure paths (there used to be a separatelogos_core_load_module_with_dependencies; liblogos#130 merged it into the bool) - Unload —
logos_core_unload_module(name, with_dependents)interleaved with concurrent load threads on a shared small module set
# Standalone
nix build .#checks.aarch64-darwin.thread-safety-tests -L # macOS ARM
nix build .#checks.x86_64-linux.thread-safety-tests -L # Linux
# From workspace root
ws test logos-test-modules --auto-local
Building
# All modules
nix build
# Individual modules
nix build .#test_basic_module
nix build .#test_extlib_module
nix build .#test_ipc_new_api_module
nix build .#test_dummy_module
# `nix flake show` lists the rest — the basic-cpp / context / interface
# fixtures, the full_api provider-proxy-UI chain, and the QML modules.
Manual testing with logoscore
logoscore is daemon + client only: the inline mode (-l <mods> -c "<module>.<method>(args)") was removed in logos-logoscore-cli#41, so start a
daemon over a modules directory and drive it with the call client. This is
what tests/run_tests.sh does — its dcall_inline helper exists purely to
translate the old inline spelling into these subcommands so the pre-existing
substring assertions still match.
# Build and test a single module
nix build .#test_basic_module -o result-basic
CFG=$(mktemp -d)
logoscore -D --config-dir "$CFG" -m ./result-basic/lib &
logoscore --config-dir "$CFG" load-module test_basic_module
logoscore --config-dir "$CFG" call test_basic_module echo hello
logoscore --config-dir "$CFG" call test_basic_module addInts 3 4
logoscore --config-dir "$CFG" stop
# Test extlib module
nix build .#test_extlib_module -o result-extlib
CFG=$(mktemp -d)
logoscore -D --config-dir "$CFG" -m ./result-extlib/lib &
logoscore --config-dir "$CFG" load-module test_extlib_module
logoscore --config-dir "$CFG" call test_extlib_module reverseString hello
logoscore --config-dir "$CFG" call test_extlib_module uppercaseString hello
logoscore --config-dir "$CFG" stop