mirror of
https://github.com/logos-co/logos-test-modules.git
synced 2026-08-27 10:11:12 +00:00
Dario LipicarandClaude Opus 5
fbd9d161b3
An audit of every factual claim in the conformance corpus against a fresh
green run of both flake checks. 29 stale sites, 9 files, no declaration
changed — the registry layer is correct and stays exactly as it was.
Almost all of it is two landings that were never swept:
* OPT1 + the C++ ext provider (logos-cpp-sdk#125). The ext table is
described throughout as Rust-only, single-provider, with no
differential, an Optional family "red on purpose", and two module
sources that announce "DOES NOT COMPILE TODAY" — while the check
measures 39 cases x 2 providers, 76 pass, differential-provider-pass
37, and compiles both modules to get there.
* the Qt consumer axis + Q1b. Two cases.json paragraphs say "this
matrix has no driver for that surface" and point at known.json M3
under `unmeasurable`; the driver runs three consumer points and M3
moved to `xfail` long enough ago that `unmeasurable` now holds only
the note explaining it moved.
The worst of it is the README, which is the front door: its hand-kept
known-broken table had drifted to eleven rows against a registry of four,
listing M1/M1b, M2, M4, M5/E1, E2, E3 and OPT1 as current defects after
each had moved to `fixed[]`, and omitting M4-residual, the live entry with
the most cells. That table is now the GENERATED one — run_matrix.py --md
already writes it and the flake check drops it at $out/known-broken.md —
pasted verbatim, with the regeneration command next to it. A table that
cannot be hand-edited cannot drift again.
Also corrected there: the documented command line omitted the two
--proxy-consumer flags that are the whole consumer axis, and the exit
status was listed as three statuses when the driver checks six. The three
missing ones — dead-skip, skip-passes, setup-failed — are each a way for a
run to look green while measuring less than it claims, which is the
opposite of a detail.
`expect_error` (cases.json schema comment) documented an expectation key
no driver ever implemented. A case written the way that paragraph
described takes have_want=False and is filed `skip`, which is not in the
failing-status set: present in the table, counted as coverage, asserting
nothing. The paragraph now documents `{"__error__": "<code>"}`, which is
what exists; logos-logoscore-py grows the guard that makes the mistake
impossible rather than merely undocumented.
Verified: both flake checks pass against this tree, and the full-contract
report diff against the pre-sweep run is ONLY the two known.json blocks
the report reproduces verbatim — every count, cell and coordinate
identical. The ext modules were recompiled, which is what proves the
comment edits did not touch code.
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>