* fix(plain): give every handle its own event subscription on the shared connection
RpcConnection::m_eventCallbacks was keyed by (object, eventName) and ASSIGNED.
One RpcConnection is shared by every PlainLogosObject a PlainTransportConnection
hands out, and requestObject() mints a fresh handle per acquire, so the second
handle to subscribe to the same event on the same module silently took the first
one's channel — including the deferred ("multi") completion channel every handle
subscribes to on its first call.
No concurrency is needed to reach it. Measured on cf1b9b0: handle A's deferred
call answers 8/8 in 0 ms alone, and 0/8 in 1505 ms (timeout, INVALID) once a
second handle merely exists. Identical on master.
THE WIRE DOES NOT MOVE. Subscribe/Unsubscribe still carry (object, event) and
nothing else. The host keeps ONE sink per (object, event, connection) — the right
model, since every sink for a connection is the same "write this frame back down
that socket" — and the CONSUMER, the only end that knows how many of its own
handles want an event, does the demultiplexing: a list of registrations per key,
a subscription id to withdraw one of them, and an Unsubscribe frame only when the
last local registration for the pair is gone, because that frame is
connection-wide.
Also here, because the consumer-side fan-out makes it observable as a doubled
delivery: PlainTransportHost::fanOutEvent sent a connection subscribed both by
name and by wildcard two copies of the same event. One copy per connection now.
And the parked-completion staging area is gated. It exists for one ordering — a
"multi" worker that finishes before the sentinel it answers has been written, so
the completion event overtakes its own Result — and a completion arriving that
early is not attributable to a handle at all. Every handle now sees every
completion on the object, so parking is allowed only while this handle has a call
outstanding, the map is emptied the instant none is, and it is capped at 512 with
oldest-first eviction. Pre-fix the same leak existed with one victim instead of N:
the handle that had stolen the channel parked every other handle's completions and
never claimed one.
Detectors: nine tests in test_plain_event_sub_sharing.cpp, all validated RED on
cf1b9b0 and on master and green here, including both mixed-version directions (a
new consumer against a verbatim pre-fix host, and an old consumer's frame sequence
against the new host over a raw socket).
── REBASED ONTO #50, THE FIRST REPLAY IN THIS STACK WITH A REAL PRODUCTION
CONFLICT (and then onto #51, which lands cleanly) ──────────────────────────
the object (a live-reference count with deferred destruction) and to give async
callbacks somewhere to land in a Qt-free process (a never-destroyed
DeliveryService). Both of those and both of this commit's changes are kept; the
resolution picks no side anywhere.
* plain_logos_object.cpp, hunk 1 — #50 replaced postToQtEventLoop's comment
header with DeliveryService; this commit inserted SyncCallScope immediately
above it. The insert point survives, the replaced header does not:
SyncCallScope now sits above #50's DeliveryService block.
* plain_logos_object.cpp, hunk 2 — #50 split disconnectEvents() into a guarded
entry point plus an unguarded disconnectEventsImpl() (release() and the
destructor must not take a reference to what they are destroying); this
commit changed that body's local from (name, callback) pairs to subscription
ids. Kept as #50's split with this commit's body. onEvent() auto-merged the
same way: #50's EntryGuard declared FIRST, then the m_mu-across-the-subscribe
body from here.
* test_sync_call_release_race.cpp — not a text conflict and not visible to git:
#50's StalledConnection double implements RpcConnectionBase, and this commit
changes that interface, so it became abstract and the file stopped compiling.
Its stub now returns a DISTINCT id per subscribe, so it cannot hide a bug
that withdraws the wrong registration.
* tests/protocol/CMakeLists.txt — both detector lists are additive and both are
in. The prose is NOT concatenated: an earlier replay on this stack had
grafted a duplicated fragment ("per-path test as evidence the gate is there.
Nor is") into the middle of a paragraph, and the sentence it belonged to is
now where it was meant to go, at the end of the "what is NOT on that list"
paragraph. #50's "the CAS is TWO gates" correction is untouched. #51's own
additions to the same file merged without a conflict on the second replay.
gives the five LiveHost fixtures a teardown that destroys the host on the thread
that emits into it, and the host in test_plain_event_sub_sharing.cpp is not that
shape — its ModuleProxy stays on the test thread, so there is no worker to race.
RE-VERIFIED BY RUNNING, on macOS arm64, Debug, all after the rebase:
* THIS FIX still fixes the bug on the NEW master. The ten
PlainEventSubSharingTest cases compile unmodified on 5be3a84 and nine go RED
there: handle A alone 8/8 answered at 0 ms avg, handle A once B exists 0/8 at
1503 ms avg with 8 timeouts, and 0/4 with 4 timeouts through the shipping
host. On this commit: 8/8 and 4/4, 0 ms avg, 0 timeouts. So neither #50 nor
#51 caused or masked this defect, and this branch is not a no-op.
* #50's FIX still works through this change. SyncCallReleaseRaceTest 8/8 green
(release() returns in 0 ms with a call parked, destroyed=1 only after the
caller leaves; 400 release-wake races, one destruction each), the whole no-Qt
binary 7/7 plus the after-main() probe (delivered=1, off the issuing thread,
exit 0), and its mechanisms are still in the source: m_liveRefs with the
EntryGuard reference-first/reference-last ordering, release() dropping the
owner's reference instead of `delete this`, DeliveryService `new`-ed with
`~DeliveryService() = delete` and a detached thread, and NO m_conn.reset() in
release().
* EXACTLY-ONCE, in the release-race shape rather than the per-path pins. Qt
vehicle: 20 rounds x 500 calls released mid-burst, answered-by-reply=1034,
cancelled-by-teardown=8966, 0 doubles, 0 dropped. No-Qt twin: 1130 / 8870, 0
and 0. Both resolvers live in both. Re-validated as a DETECTOR on this merged
tree by removing BOTH gates (claim()'s CAS and takeCallback()'s swap) in a
throwaway build: 19 doubles Qt, 13 doubles no-Qt, both FAIL, while the
per-path exactly-once tests stay green — which is the difference the
CMakeLists note describes.
* NO USE-AFTER-FREE. Guard Malloc clean over SyncCallReleaseRace, IoFold,
PlainObjectTeardown, PlainCompletionSubLifetime, PlainCancelPendingRace,
PlainWaiterReaping, PlainHostEventTeardown, PlainEventSubSharing and
PlainParkedCompletionGate (52/52), and over the whole no-Qt binary (7/7,
exit 0).
* TEARDOWN AND DEADLINES. release() with 32 calls in flight: 0 ms. release()
after 100 reaped calls: 0 ms. A 200 ms deadline fires at 200 ms while an
onEvent handler holds the io thread for 2001 ms; a 250 ms deadline fires at
250 ms with the io thread blocked forever; 12 idle 200 ms deadlines
min=200 median=200 max=202 ms.
* NO USER CALLBACK INLINE ON AN IO THREAD: no-Qt replies 300/300 with
on-caller-thread=0 and delivery-thread!=io-thread=1; cancellations 20/20 with
inside-release=0 and on-releasing-thread=0.
* Full suite 433/433 (was 421 on master; this adds 12), and `nix build .#tests`
433/433.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* test(plain): make the subscription detectors barrier-driven instead of timed
Every "wait a bit then count" in the new tests is now an in-band round trip: a
Methods request travels the same socket and is dispatched on the same strand as
everything written before it, so its reply proves the earlier Subscribe /
Unsubscribe frames have been applied AND that the Event frames the peer wrote
before answering have been dispatched. The audits are exact counts rather than
polls with a timeout, which is what they have to be on a loaded CI runner — a
short sleep there fails the test rather than skipping it.
Still 9 of 10 RED on cf1b9b0 and on master, with the teardown pin green; the set
now runs in 9s instead of 110s.
* test(plain): drop the poll helper the barrier replaced
---------
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
* fix: make isConnected() mean connected, and stop the log claiming it
QRemoteObjectNode::connectToNode() returns false only when the URL
SCHEME is unregistered -- it never contacts the peer. Our registry URLs
are COMPUTED rather than discovered (logos_instance.h:
local:logos_<module>_<instanceId>), so they are identical whether or not
the module exists. Latching m_connected from that return therefore made
isConnected() answer "yes" for modules that were never loaded, which made
every `if (!client->isConnected()) return;` guard in the codebase DEAD
CODE.
Callers then paid a 20 s waitForSource per call, twice over, because the
token handshake tries capability_module first. Measured in Basecamp with
package_manager absent: ~417 s of blocked GUI thread on macOS and 361 s
on Linux before the window appeared, and over 900 s under load. Not a
Windows bug -- the Windows port merely exposed it.
isConnected() now also requires a listener at the endpoint. For `local:`
that is a direct socket / named-pipe probe, which costs microseconds
precisely in the case that used to cost 20 seconds; any other scheme
keeps its previous behaviour.
Two logging changes, because the diagnostics cost more than the defect:
"Successfully connected to registry" asserted a connection that often did
not exist and sent three separate investigations to the wrong place -- it
now says a connect attempt started and makes no claim about the peer.
And requestObject warns BEFORE a doomed wait instead of going silent for
20 s and then reporting failure.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix: let event subscriptions survive a module that is not reachable yet
requestObject() answers "is the module there RIGHT NOW", and every
subscriber in this codebase asks at the one moment the answer is no: a
module's init(), a UI backend's onContextReady(), a QML view's
Component.onCompleted. All of those run while the dependency's host
process has been spawned but has not called listen() yet. The subscriber
then gave up permanently -- lp_subscribe returned nullptr with no log at
all, and callers turned that into a `false` the documented example
discards. Method calls kept working through the same window because
acquireCachedObject() reaches the replica by a path that never asks, so
the symptom was "events are broken", not "the subscription never
happened".
1238316 (isConnected() means connected) is what made this deterministic
rather than lucky, and it must not be reverted -- it removed ~417 s
(macOS) / 361 s (Linux) of blocked GUI thread at Basecamp startup. So
the subscription becomes deferrable instead.
- LogosTransportAsyncAcquire: a sibling interface (dynamic_cast, like
LogosObjectErrorChannel) so LogosTransportConnection's installed
vtable is unchanged. requestObjectWhenAvailable() registers interest
and returns; it never blocks and never spins a nested event loop.
- qt_remote implements it by acquiring a dynamic replica before the
peer exists -- legal, free, and armed by the node's existing 250 ms
reconnect loop, so it adds no polling. Delivery is deferred one
event-loop turn because stateChanged fires from inside onClientRead
(the refresh_balances re-entrancy SIGSEGV).
- LogosAPIConsumer::onEventWhenAvailable() holds the pending
subscriptions, arms them when the object appears, shares ONE handle
per object (separate from the call cache, so a call re-acquiring a
stale handle cannot silently kill a live subscription), and re-arms
them after reconnect(). Unbounded in time on purpose -- a module can
be installed mid-session -- but bounded in noise: one warning at 3 s,
one at 60 s, a log line when it arms, and a loud abandon when the
transport proves it impossible.
- lp_subscribe routes through it, which fixes the same defect for
every C++/Nim/Rust module and UI backend without touching qt-sdk or
any generated code.
tests/protocol/test_deferred_subscription.cpp pins all three layers,
each with a published-first control so a red case cannot be a mis-wired
fixture.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix: close the remaining silent-failure holes in deferred event subscriptions
The deferred-subscription registry from the previous commit fixed the reported
defect, but review found six ways it could still lose a subscription without
saying so — five in the registry itself, one in the plain transport's host — and
every one of them lived in a cell with no test. All of its tests ran in Remote
mode; three of the four transports had none at all.
Registry (cpp/logos_api_consumer.cpp):
* An already-present module was deferred to the first 250 ms tick on every
transport without a deferred acquire, and every event emitted in that window
was dropped. lp_subscribe used to attach synchronously and deliver them, so
this relocated the silent event loss rather than removing it. startAcquire()
now reports which of three answers the transport gave, and only an
Unsupported answer takes the one synchronous requestObject() — which is also
what keeps that call structurally away from qt_remote, whose requestObject()
enters waitForSource()'s nested event loop even at timeout 0. Previously that
invariant lived in a comment, and tick() could reach it whenever
acquireDynamic() returned null.
* reconnected() put every armed subscription back in the pending set but never
restarted the timer, which takeMatching() had stopped when they armed. Since
tick() is the sole driver of both the retry and the watchdog, a reconnect left
the subscription dead AND silent — quieter than the "not connected" warning it
replaced.
* armAgainst() released a stale handle while entries were still attached to its
event helper. Those entries stayed in m_armed, never fired again, and reported
as healthy. They are now revived and re-armed against the new handle.
* The retry timer ran forever at the 5 s cap with nothing to do. It now stops
once every pending entry has an acquire in flight and has said everything it
will say, and restarts when that changes.
* A cancelled subscription had no way to leave the registry, so lp_unsubscribe
left it holding the timer up and warning about a subscription nobody wanted.
onEventWhenAvailable() now returns an id; cancelEventSubscription() and
eventSubscriptionState() are its counterparts, and lp_unsubscribe uses them.
Plain transport (cpp/implementations/plain/plain_transport_host.cpp):
* onSubscribe() dropped a Subscribe for an object that was not published YET —
which is exactly when consumers subscribe — and the consumer could not know,
because requestObject() had already succeeded. Publishing also overwrote the
sink table wholesale, so a republish took every subscriber down with it. The
sinks now live in a table keyed independently of publication.
Also adds lp_pending_subscriptions() to the C ABI. The Qt consumer has had this
visibility all along and the C ABI had none, which is why a subscription that
silently never armed was undetectable from Rust, Nim or a universal C++ module.
tests/protocol/test_event_delivery_matrix.cpp pins the product rather than a
sample of it: 3 transports x 2 provider kinds (Qt-native and universal/std, which
reach the wire by different conversions) x 2 consumer paths (onEventWhenAvailable
and lp_subscribe) x 6 timings, plus mock and the non-blocking guard. Every
delivery case has a control that is green independently of these fixes.
One thing that is NOT fixed and is now stated in the contract: arming is not
retroactive and no transport buffers, so a module that emits a one-shot "ready"
event synchronously inside its own init() can still be missed. That window is
inherent to the transport — the blocking requestObject() this replaced had it
too — but "subscriptions survive a late module" is not "no event can be missed".
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* docs: name the QtRO invariant the stale-handle revive rests on
* test(events): state what the non-blocking guard can and cannot catch
The acquireCount assertion catches a retry that polls qt_remote's blocking
requestObject() in the ordinary case. It cannot reach the narrow one -- the
poll is only reachable when the transport declines a deferred acquire while
still reporting connected, which needs acquireDynamic() to return null and is
not forcible from outside. That case is held shut by control flow instead, and
saying so is better than leaving a reader to assume the test covers it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix: make the async-acquire contract and lp_subscribe's return honest
Both from review on #47, both real.
The LogosTransportAsyncAcquire contract promised that a true return means
onReady "WILL be invoked exactly once". It will not: RemoteTransportConnection
parents every in-flight PendingAcquire to m_pendingAcquires, which is reset at
the top of the destructor and rebuilt on reconnect, so an accepted request is
cancelled silently with no callback whenever the connection it belongs to goes
away. The contract now says AT MOST once, names both cancellation triggers, and
states what a caller has to do about them — re-issue after a reconnect, or carry
its own deadline. It also records that the layer above already does the first,
which is why a subscription made through onEventWhenAvailable() survives
something the raw transport call does not. That asymmetry is the reason to
prefer the consumer API, and it was previously implicit.
lp_subscribe returned a non-null lp_subscription even when onEventWhenAvailable
refused and returned 0, leaving the caller with a handle that can never fire
while the ABI documents NULL as the one signal that the arguments were refused.
It now checks sub->id and returns nullptr.
That second one is defensive rather than a live bug, and the code says so: the
guard at the top of lp_subscribe already rejects an empty event name and a null
callback, and lp_client_create rejects an empty target, so the three inputs that
make onEventWhenAvailable() return 0 cannot all arrive there today. No test
drives it. The two contracts simply have to agree, and one of them changing is
how they would stop agreeing.
374/374 green.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix: stop lp_unsubscribe deadlocking, without dereferencing a freed client
lp_unsubscribe took ownerGuard->mutex and, while holding it, called
cancelEventSubscription(), which marshals to the owner thread with a BLOCKING
queued connection. The delivery callback lp_subscribe installs runs ON that
thread and takes subGuard->mutex then clientGuard->mutex — and clientGuard IS
ownerGuard, both assigned from client->guard. Lock-order inversion. It also hung
outright once the owner's event loop had stopped, which is exactly when a
language binding drops its subscription handle.
The first attempt at this dropped the guard entirely and checked `alive` inside
the posted lambda. That was a use-after-free: QMetaObject::invokeMethod
dereferences the target (it reads object->thread()) before the lambda can run,
and lp_client_destroy sets alive=false and deletes the client synchronously —
so the check was unreachable on the exact ordering lp_subscription's own comment
documents as supported. Proven rather than argued: with MallocScribble=1, a test
that destroys the client before unsubscribing segfaulted 6/6 with the guard
removed and passed 6/6 with it restored.
So the guard is held across the POST and not across the cancel. Both halves are
load-bearing, and the distinction is the whole fix: posting never waits on the
owner thread, so holding the mutex across it cannot invert; only the blocking
marshal ever had to move.
Consequence, now stated in the ABI header: un-registration is EVENTUAL. The
callback-will-not-fire guarantee stays synchronous and unconditional, but
lp_pending_subscriptions() may still list a just-cancelled subscription until the
owner thread runs, and if the client is destroyed first the cancellation never
runs at all — correct, since the registry died with it. The matrix test now
pumps for the drain instead of asserting it happened synchronously.
374/374 green.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix: arm a subscription immediately when the module is already reachable
Deferral introduced a narrower version of the loss it removed. The common
consumer shape is a call followed by a subscription in the same function --
wallet-ui's backend calls get_chains() and subscribes on the next line, the
tutorial's C++ UI backend does the same. Before deferral the generated Qt
wrapper acquired synchronously, so the subscription was live before on()
returned and an event emitted straight after was delivered. Holding it until the
next event-loop turn silently drops that event.
Measured on the generated-wrapper harness: 1/1 delivered pre-migration, 0/1
after, over 3 runs.
LogosTransportAsyncAcquire gains tryAcquireNow(): hand back a handle ONLY if
that costs nothing -- for qt_remote, a replica that is already Valid, which is
exactly the state a prior call leaves behind since QtRO shares one replica
implementation per object name on a node. It must never block, never spin a
nested event loop and never wait on a peer; "not immediately available" is an
answer and the caller falls back to the deferred path. Default returns nullptr,
so a transport that cannot answer cheaply simply does not.
Delivering inline here is safe for the reason the never-synchronous rule exists:
that rule protects against re-entering the transport's READ stack from a
stateChanged callback. tryAcquireNow runs on the subscriber's own stack.
The new matrix case fires ONCE, synchronously, with no pumping in between --
re-firing would hide the exact gap under test -- and states the transport
difference rather than papering over it. Subscription registration is local on
qt_remote (attach to a held replica) and qt_local (connect an in-process
signal), so delivery there must be instant. On plain it is a wire frame to the
host, so instant delivery was never on offer and never was before this change
either; that leg asserts it still arms and delivers.
Also de-flaked EventDeliveryNonBlocking: its heartbeat COUNT over a fixed
wall-clock window measures the machine, not the code. The gap assertion is the
one that means something; the count is now only a floor.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix: stop tryAcquireNow leaving a dangling facade in QtRO's connect list
e9f82ac introduced a use-after-free. tryAcquireNow() acquired a dynamic replica
and, when it was not already Valid, deleted it. That is not safe: QtRO shares one
replica IMPLEMENTATION per object name per node, and while that implementation is
still waiting for the source's metaobject it records every facade built on it as a
RAW pointer in QConnectedReplicaImplementation::m_parentsNeedingConnect.
~QRemoteObjectReplica is an empty body, so destroying a facade never deregisters
it, and the implementation dereferences the whole list when the class definition
arrives.
So each probe of an unreachable module left one dangling pointer behind.
WHY IT HID. The first probe owns the only implementation and takes it down with
itself, so a single subscription is harmless. It needs a second subscription whose
implementation is pinned by an in-flight PendingAcquire before a freed facade can
outlive its implementation. A consumer subscribing once sees nothing; the QML
plugin shape -- a view registering every event it cares about up front -- dies.
REPRODUCED, 4 runs of 4, serially as well as in parallel, in
logos-view-module-runtime's existing suite (unchanged from master, and green there
against this same protocol checkout):
LogosQmlBridge: subscription accepted for "echo_module" :: "ev13"
Received signal 10 (SIGBUS), code 1, for address 0x5a
SIGBUS code 1 is BUS_ADRALN -- a misaligned atomic access on a garbage base read
out of a recycled heap block, in the event loop rather than at the call site,
which is why it reads as a mystery crash rather than as a subscription bug.
PROVEN, before writing this fix, by commenting out that single `delete replica`:
the same suite went 4 failures -> 6/6 with no other change. With this fix: 6/6.
THE FIX IS TO PARK, NOT TO FREE. One probe per object name, parented to
m_pendingAcquires -- which both the destructor and reconnect() already destroy
BEFORE the node, so the implementations die in the same breath and freeing them
there is safe. Ownership transfers out only when the replica reaches Valid, by
which point the implementation is configured and is no longer holding the facade.
It costs one idle replica per name until it goes Valid or the connection dies.
AND REMOVE THE MULTIPLIER: beginAcquire() probed on EVERY add(), ahead of
startAcquire() and therefore ahead of the m_acquiring one-acquire-per-object
guard. tick() already applies that filter; beginAcquire() was the one caller that
did not, which is what turned one probe per module into one per subscription.
While an acquire is in flight its PendingAcquire already holds a replica and will
arm every waiting entry at once, so the probe buys nothing there.
Not QML-specific: lp_subscribe reaches the same entry point.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
* fix(lp): lp_invoke_async can finally report a failure
lp_result_cb has always been documented as carrying an outcome —
"ok != 0 -> `json` is the result JSON value; ok == 0 -> `json` is the
canonical error object" — and the synchronous twin lp_invoke has always
honoured it (LP_ERR_UNAVAILABLE + out_error_json). lp_invoke_async did
not: it subscribed with the VALUE-ONLY invokeRemoteMethodAsync overload
and called back `cb(1, json, user_data)` with ok hard-coded to 1, so a
call to a module that cannot be acquired reached the callback as a
SUCCESS carrying a default-constructed value.
The fix is to pass a TWO-argument lambda, which is invocable only as
LogosAPIClient::AsyncResultErrorCallback and so binds to the
CallError-aware overload that already exists next to the value-only one.
The failure is then rendered with the same makeErrorJson the sync path
uses, so both entry points report the same event in the same shape.
The ABI is unchanged. lp_result_cb's (ok, json, user_data) signature
already reserves ok == 0 for exactly this; nothing had to grow a new
entry point, and every in-tree consumer already branches on `ok`
(logos-rust-sdk's async_call_trampoline even parses `message` out of the
canonical error object — code written against a contract the
implementation never kept).
Regression test: a matched pair over a REAL transport (plain TCP), not
the mock.
FAILING async call -> ok=0 {"code":"object_unavailable", ...}
SUCCEEDING async call -> ok=1 7
The first fails on the unfixed tree (ok=1, json "null"); the second
passes on both, so an over-eager "report failure everywhere" fix cannot
sneak through.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(protocol): report the failures that happen AFTER acquire, on both twins
#40 made lp_invoke_async able to report a failure, but only for the two
conditions produced ABOVE the transport: acquire failure and the unauthorized
sentinel. Everything the transport learns while the call is in flight was still
discarded — PlainLogosObject answered a bare QVariant() for a timeout and for
`ResultMessage.ok == false` alike, and LogosAPIConsumer hard-coded an empty
CallError next to it.
Two ordinary failures therefore still reported success on both entry points:
a TIMEOUT, and MODULE NOT LOADED against a host that is up (which is not an
acquire failure on the plain wire — requestObject hands back a handle for any
name over an open connection).
The information already exists: ResultMessage carries err/errCode, the futures
know they expired, QtRO knows its pending call never finished. It had nowhere to
go because LogosObject's callMethod returns a lone QVariant and its
callMethodAsync callback takes a lone QVariant.
Widening those virtuals would append a vtable slot to an installed, subclassed
interface, so instead this adds LogosObjectErrorChannel — a SIBLING interface
reached by dynamic_cast. LogosObject's size, layout and vtable are unchanged
(verified: a subclass compiled against the old and new headers emits the same
14-entry vtable with identical slot indices), and a transport that does not
implement it keeps today's behaviour.
logos_protocol.cpp needs no change: lp_invoke and lp_invoke_async already render
this CallError, so both twins gain the coverage together.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(protocol): stop the macOS flake that was sinking #41
Three real races the new CallErrorAfterAcquire suite exposed (and that
Copilot flagged on the QtRO half):
1. ~PlainTransportHost stopped the acceptor but did not quiesce the shared
Asio I/O thread. Server-side RpcConnections hold a raw IncomingCallHandler*
back to the host; a fail()/onConnectionClosed racing teardown freed the
handler mid-call. That is the macOS CI SIGSEGV in
AsyncSuccessStillReportsTheValue — it fires with no output of its own
because the previous live-host test's destructor left the heap corrupted.
Restore the I/O barrier that landed on the qtfree branches but never on
master (proven: 80/80 clean on the CI crash sequence that was ~2/50 before).
2. PlainLogosObject::callMethodAsync detached its per-call waiter while
capturing `this`. release()/delete this could then race the waiter.
Join waiters in the destructor/release, and register the thread under the
lock before it can outrun teardown.
3. QtRO async could deliver the user callback twice when the timeout timer
and the pending-call watcher finished around the same moment, violating
the exactly-once contract. Gate both paths (and the deferred-completion
arm) on one atomic.
Also drain queued onCall invokes after host.reset() in the #40 live-target
control, matching LiveHost's teardown discipline.
Co-authored-by: Cursor <cursoragent@cursor.com>
* fix(protocol): the drain barrier must not dangle on its own timeout
Two defects in the barrier added by 1e9c934, both on the path it takes when
it fails:
std::promise<void> drained; // stack local
boost::asio::post(ioc, [&drained] { drained.set_value(); }); // by REFERENCE
fut.wait_for(std::chrono::seconds(5)); // result dropped
1. The wait is bounded, so on timeout this frame returns while the posted task
is still queued -- and the task holds a pointer to a destroyed stack object.
set_value() then writes to freed stack memory. The bound that stops a wedged
I/O thread hanging teardown introduced the exact class of use-after-free the
barrier exists to prevent. The promise is now a shared_ptr captured BY VALUE,
so the task keeps it alive whether or not anyone is still waiting.
2. The wait_for result was discarded. A timeout means the barrier did NOT hold
and we are about to free an IncomingCallHandler that a live connection may
still call back into -- the original crash, minus any way to know it
happened. It now warns, naming the consequence.
Neither is reachable while the I/O thread drains promptly, which is why the
suite is green either way; both matter precisely when it does not, which is
the only situation the barrier is for.
Tests: 270/270.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(protocol): teardown must not wait out the call it is abandoning
Joining the per-call waiters (rather than detaching them) closed a real
use-after-free: the waiter captures `this`, and release() used to `delete this`
underneath it. But joinWaiters() could only join. It had no way to ASK a waiter
to stop, so destroying a PlainLogosObject with a call in flight blocked for the
remainder of that call's timeout — up to 20s on the protocol default. A module
unloading mid-call stalled the unload for that long, on the releasing thread.
Measured, 8s call timeout, provider parked:
release() before after
future wait (site 1) 7804 ms 11 ms
deferred completion (site 2) 7703 ms 0 ms
Both blocking sites are now interruptible, and they need different treatment:
* the std::future wait cannot be interrupted at all, so it is SLICED: one
deadline computed up front, waited in 25ms increments, stop flag checked
between them. Teardown latency is one slice; the timeout the caller asked
for is unchanged, because the last slice ends exactly on the deadline. 25ms
is under two frames (so a module unload stays imperceptible) and costs 40
wakeups/second per in-flight call, which is nothing beside the Qt event loop
these threads already sit next to.
* awaitCompletion's condition_variable is interruptible by construction:
widen the predicate, notify_all. No latency floor at all — hence 0 ms. The
flag is published under m_completionMu so a waiter cannot evaluate the
predicate, decide to sleep, and then miss the notify.
A CANCELLED CALL STILL DELIVERS, EXACTLY ONCE. This is the part a naive fix
breaks: callMethodAsyncWithError and lp_invoke_async promise the callback fires
exactly once, so a waiter that simply returns on stop trades a bounded stall for
an unbounded hang in every caller awaiting it. Proven by building that naive
variant: it passes the latency test and fails three exactly-once tests with the
callback never arriving.
The code is "transport_error", from the existing vocabulary rather than a new
one, since these codes are the wire contract. logos_call_error.h defines it as
"the connection failed or was torn down mid-call", which is precisely what
happened — the consumer tore its own end down. The alternatives all misattribute
it: "object_unavailable" says the module is absent (it is not, and callers
re-acquire on that code), "call_failed" blames the peer for a dispatch it
performed fine, and "timeout" — what this used to report, after waiting the
deadline out — claims a deadline elapsed that did not. It is also already what
the wire produces for the same event seen from the other side (callErrorFromWire
maps TRANSPORT_CLOSED to transport_error).
Delivering during teardown is safe because postToQtEventLoop touches nothing
owned by the object: it is a free function taking the callback, value and error
BY VALUE, and the waiter copies objectName/method up front. That was already
true and is now load-bearing, so it is documented at the function. The queued
lambda runs after the object may be gone; everything the waiter reaches through
`this` runs before the join returns, which is why the join must stay.
Also closes the registration window it opens: a call arriving after the stop
would push a thread onto an m_waiters that teardown has already swapped out, so
it would never be joined. It is answered as cancelled instead.
The UAF is verified still closed under macOS Guard Malloc rather than ASan —
libclang_rt livelocks in its own initializer before main on this toolchain, for
both ASan and TSan, on a hello-world. Under Guard Malloc the race test is clean
across 5 runs and SIGSEGVs immediately when the join is turned back into a
detach, so the check is a real detector and not a vacuous pass.
Tests: 277/277 (was 270; 7 new). CallErrorAfterAcquireTest hammered 40x, 0
failures — it was ~2/50 flaky before this branch's earlier fixes.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(protocol): a call that FINISHED must not park its thread for the object's life
The per-call waiters are joinable rather than detached, which is what closed the
use-after-free where release() deleted the object under a still-running waiter
(4f9d824), and they are interruptible, so teardown no longer waits out the call's
timeout (731e579). Both stay. What neither did was retire a waiter that had
FINISHED: m_waiters was only ever swap()ped, in stopAndJoinWaiters(), so an
exited-but-unjoined std::thread — whose stack and pthread struct are not
reclaimed until somebody joins it — stayed parked for the lifetime of the handle.
Measured against a live PlainTransportHost over TCP, every call completing
normally, one handle held throughout, before:
10000 calls m_waiters 300 -> 10300 rss +156.56 MiB 16417 B/call
30000 calls m_waiters 300 -> 30300 rss +469.28 MiB 16403 B/call
and the same through the production C ABI — one lp_client, N lp_invoke_async —
at +156.53 MiB. That path is why this matters: LogosAPIConsumer caches ONE
handle per module and reuses it for every async call, releasing it only on
eviction or teardown (cpp/logos_api_consumer.cpp:129 and :207), so a
long-lived module leaks per lp_invoke_async. The ~16KB constant is one page on
this 16KiB-page arm64 and will be smaller elsewhere; the UNBOUNDEDNESS is the
platform-independent part, and follows from m_waiters.size() rising 1:1 with
completed calls and only ever falling in teardown. Attribution: the retention
arrived with the join in 4f9d824, not with 731e579 — but 731e579 is what makes
the join permanent.
The registry is now KEYED, because a thread cannot join itself and so a waiter
can never retire its own entry. Each waiter publishes its id as its FINAL act (a
scope guard declared first, so it destructs last, covering all four exit paths),
and the next spawn — plus teardown — joins those ids and erases them. Joining a
thread that has already returned is a couple of syscalls. Same probe, same
workload, after:
10000 calls m_waiters 15 -> 16 rss +0.08 MiB 8 B/call
30000 calls m_waiters 16 -> 16 rss +0.06 MiB 2 B/call
10000 calls via lp_invoke_async rss +0.09 MiB 10 B/call
Retention is now bounded by the waiters that finish after the LAST spawn, i.e.
by peak in-flight concurrency — 16 at the in-flight window above, and exactly 1
when calls are issued sequentially — instead of by call count.
THE DEADLOCK THIS SHAPE INVITES is a reaper that joins while holding m_waiterMu,
against a waiter blocked on m_waiterMu trying to publish. It is avoided by
construction rather than by argument: nothing is joined with a lock held, in the
reaper or in teardown, whatever a waiter does on its way out. Proven by building
the naive variant that does join under the lock — the new hammer wedges it, with
the main thread in reapFinishedWaiters -> pthread_join and a waiter in
publishFinishedWaiter -> mutex wait, and the test's watchdog names the cause
instead of letting CI hang.
Teardown's guarantee is restated rather than weakened. It is not "every waiter
has been joined by the time stopAndJoinWaiters() returns" — a waiter a
concurrent reaper is mid-join on is no longer in the map — but the thing that
guarantee was ever for: NO WAITER TOUCHES THE OBJECT AFTER IT RETURNS. An entry
leaves m_waiters only once its thread has published, and publishing is that
thread's last access.
The TODO above the waiter still stands: the real fix is to fold the wait into
the shared Asio io_context and have no thread per pending RPC at all. This makes
the interim honest; it does not replace that.
Two more things review turned up, folded in here:
* The two wait sites resolved stop-vs-result in OPPOSITE directions.
waitForResult tested the stop flag BEFORE polling, so an already-ready
future was still reported as transport_error, while awaitCompletion
deliberately preferred a completion that had landed — and both were
commented as intentional. One rule now, applied to both: AN ANSWER ALREADY
IN HAND BEATS A CONCURRENT STOP, and the stop only decides what happens when
there is nothing to hand over. The callback fires either way
(postToQtEventLoop copies everything it delivers), so the only thing a stop
can change is what the callback SAYS — and manufacturing transport_error
while the true answer sits in the future reports a failure that did not
happen, to callers that re-acquire, retry and log on that code. Preferring
the answer costs nothing, since it is already there: the flag is still
checked before every sleep, so the teardown-latency bound is unchanged.
* CORRECTION to 731e579's message, which claimed it "closes the registration
window" where a call arriving after the stop would never be joined. That
branch is unreachable in defined behaviour: m_stopping is raised only by
teardown, so any thread that can read it inside callMethodAsyncWithError is
already calling a method on an object whose destructor is running — the load
is itself the use-after-free, reproduced as a SIGSEGV on that commit and on
its parent alike, and nothing inside that function can repair it. The guard
is harmless and stays (one predictable branch, and it fails safe with one
callback), but its comment now says what it is instead of claiming a fix it
does not make.
Verified by running, with every check first shown to FAIL on unfixed code:
* Retention: the probe above, plus a committed regression test that reads
m_waiters out of the live object through the explicit-instantiation access
hole ([temp.spec] does not check access on an explicit instantiation's
template arguments) — so the code under test keeps its private state, with
no friend, no test-only accessor and no `#define private public`. 200
sequential completed calls keep 1 waiter; without pruning they keep 200.
* Exactly-once on all four paths — normal completion, timeout, cancellation
and the deferred-completion (pending-sentinel) arm — counted PER CALL so a
dropped one and a doubled one cannot cancel out, plus the 60-round
release-during-call race. Shown to catch a cancelled path that returns
silently (3 failures) rather than delivering.
* Teardown latency unchanged from 731e579: 10-17ms with an in-flight 8000ms
call and 0-1ms mid-defer, against 15ms / 1ms on that commit.
* The UAF stays closed: 11 teardown + reaping tests clean under macOS Guard
Malloc (ASan/TSan remain unusable on this toolchain).
* Full suite 281/281 twice, `nix build .#tests` green (281/281 in the
sandbox), CallErrorAfterAcquireTest hammered 40x clean.
* fix(protocol): a burst that goes quiet must not wait for a call that never comes
378d889 retired finished waiters, but from ONE site: the async-call spawn path.
So whatever finishes after the LAST spawn is never reaped, and a module that
bursts and then goes idle parks it all until the handle dies. Measured on
378d889, one handle, 2000 concurrent calls, every one delivered:
after 2000 completed calls, IDLE: m_waiters=1428 rss=+24.17 MiB
after ONE further call: m_waiters=1 rss=+ 1.92 MiB
The unbounded-per-call class was gone; this is what it left behind, and the
second line is the whole diagnosis — the corpses go the instant anything calls
again, so the reaper works and simply never runs. LogosAPIConsumer caches one
handle per module and never releases it between calls, so "bursts, then quiet"
is not a corner case: it is a UI that fans out on a refresh and then waits for
the user.
A finishing waiter now reaps the OTHER finished waiters before publishing
itself, so a burst drains as it completes. Same probe, same workload:
after 2000 completed calls, IDLE: m_waiters=1 rss=+ 1.88 MiB
THE BOUND IS ONE, NOT ZERO, and by construction rather than by luck: a waiter
can only reap OTHERS (a thread cannot join itself), so the last one to finish
has nobody behind it to collect it. Anything that publishes after the final
reap survives too, which is why 12 runs of the probe gave 1 eleven times and 2
once. Those go on the next call, or in teardown. Retention now tracks neither
call count nor peak concurrency — the sequential and in-flight-16 numbers move
from "15 -> 16 waiters" to "1 -> 1" — and the memory figures are unchanged
against 378d889 where they were already flat: 10k sequential +0.00 MiB, 10k at
16 in flight +0.06 MiB, 30k +0.09 MiB, and 10k through the production C ABI
(one lp_client, N lp_invoke_async) +0.09 MiB / 10 B per call, the same as
378d889 reported.
THE ORDER IS THE SAFETY ARGUMENT. Reap first, publish last, never the reverse:
* Publishing is what makes a waiter joinable BY ANOTHER WAITER. Reaping first
keeps that relation one-way — unpublished threads join published ones,
published ones join nobody — so it has no cycles. Inverted, two waiters
publishing in the same instant can each take the other's thread out of
m_waiters and then join it; both are already out of the registry, so
teardown does not even wait for them. Built that variant: pthread_join
detects the cycle and throws, the half-drained thread vector then destroys
a still-joinable thread, and the process aborts — the EXISTING hammer
(ReapingRacesPublishingWithoutDeadlocking) catches it 5 runs out of 5, with
the stack showing two waiters inside FinishOnExit joining each other.
* While a waiter is unpublished it is still in m_waiters, so a concurrent
teardown joins it and the object cannot be destroyed under the reap. Once
published, a reaper may take its thread out of the map and release() may
`delete this` — and a reaper on the CALLER's thread (the spawn path) is one
teardown neither knows about nor waits for, so a post-publish touch of
m_waiterMu is a use-after-free on a member mutex. That path needs a caller
still issuing calls while another thread releases, which this class already
treats as caller-side UB, so it is stated as an argument; the cycle above is
what the tests actually demonstrate.
Two corrections to 378d889, which this change makes load-bearing rather than
cosmetic. NOT amended into it — it is pushed, and a commit that misstates its
own reasoning is better read alongside the correction than rewritten.
* plain_logos_object.h:107-109 said reapFinishedWaiters() is "called on every
async spawn ... and from stopAndJoinWaiters()". It is not, and never was,
called from stopAndJoinWaiters(): teardown does its own id-independent
brute-force join, which is precisely why it needs no cooperation from the
reaper. Harmless behaviourally, wrong in a mechanism whose entire argument
is who joins what and when. The comment now names the two real callers —
the spawn path and, as of this commit, every waiter on its way out.
* 378d889's message presented "the join is outside the lock" as THE property
that prevents the reaper deadlock, "proven by construction" by its hammer.
That is overstated, in a way that would let the guarantee be refactored
away with the suite still green. TWO independent properties each suffice:
joining only PUBLISHED ids (a published waiter never needs m_waiterMu
again, so it cannot be the thread being shut out), and joining outside the
lock. The hammer only wedges when BOTH are gone. Measured, on top of this
change: the variant that joins under the lock but KEEPS the published-only
filter passes ReapingRacesPublishingWithoutDeadlocking in 293/297/290ms
across three runs and the whole reaping suite besides, while the variant
that joins everything under the lock trips the watchdog at 60s. So a later
"simplification" that moves the join inside the lock would ship green. Both
properties are kept, and the comment now says which one the test is
actually testing.
The TODO above the waiter still stands: the real fix is to fold the wait into
the shared Asio io_context and have no thread per pending RPC at all. This
makes the interim honest; it does not replace it.
Verified by running, each check first shown to FAIL on unfixed code:
* Retention: the burst probe above, plus a committed regression test that
reads m_waiters out of the live object through the explicit-instantiation
access hole. 800 concurrent completed calls, then IDLE with NO further
call: 1 waiter left, 20 runs out of 20. On 378d889 the same test leaves
610 of 800 and fails. The pre-existing sequential and in-flight tests are
unchanged and still pass.
* The UAF stays closed — the check that matters most here, because this adds
an object access late in the waiter's life. 9 reaping/teardown-race tests
plus the 7-test teardown suite clean under macOS Guard Malloc (ASan is
unusable on this box: it hangs in its own initializer). DETECTOR VALIDATED
both ways: turning teardown's join back into a detach SIGSEGVs under Guard
Malloc on the release-during-call hammer (exit 139), and the specific
inversion this change risks — reaping AFTER publishing — aborts as
described above.
* No deadlock: reap-vs-publish hammered 20x (1600 calls in 40 overlapping
bursts each), plus 60 rounds of teardown landing from another thread while
the tail of a burst retires itself, plus 6x600-call bursts checking that
LIVE OS threads (task_threads, which counts wedges and not corpses) come
back to baseline every round. Clean; the watchdog names the cause if it
ever is not.
* Exactly-once on all four paths — normal, timeout, cancellation, deferred
sentinel — counted per call. Each detector validated with a broken build:
dropping the cancelled callback fails 4 tests, dropping the timeout one
fails its test, and double-delivering the normal/deferred arm fails those.
* Teardown latency unchanged from 378d889: 1-25ms with an in-flight 8000ms
call and 0ms mid-defer across 5 runs, against 2-21ms / 0ms on that commit —
the same one-wait-slice (25ms) bound, since a waiter's extra work happens
after it has stopped waiting.
* Full suite 282/282 three times, `nix build .#tests` green,
CallErrorAfterAcquireTest hammered 40x clean.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* test(protocol): pin publishing as a waiter's LAST access to the object
PlainLogosObject's waiters are joinable, interruptible and reaped, and all
three rest on one ordering rule that nothing in the suite could see:
~FinishOnExit() {
self->reapFinishedWaiters(); // others, never itself
self->publishFinishedWaiter(id); // strictly last
}
reapFinishedWaiters() erases published entries from m_waiters under m_waiterMu
and joins those threads OUTSIDE it. stopAndJoinWaiters() swaps m_waiters under
the same lock and brute-force joins whatever it got. So a waiter that a
concurrent reaper is mid-join on is NOT in teardown's map, and teardown can
return — with release() going straight on to `delete this` — while that waiter
is still unwinding. stopAndJoinWaiters() already says this in as many words:
the guarantee is not "everything is joined when this returns" but "no waiter
touches this object after this returns". Publishing being last is the entire
reason the second sentence is true, so one member access below it is a
use-after-free, and moving the publish above the reap is a join cycle.
THE DEFECT SHIPS GREEN. Rebuild plain_logos_object.cpp with a single object
read after the publish and the whole of PlainObjectTeardownTest and
PlainWaiterReapingTest passes, 10 runs out of 10, cleanly under Guard Malloc.
That is not a hole in those suites. No SUPPORTED caller can provoke it: under
calls-in-flight-plus-release, every waiter is still joined transitively,
because a waiter leaves m_waiters only via teardown (which joins it) or a
reaper, and a reaper is either another waiter — itself in m_waiters until after
its join returns — or the async-spawn path, whose join completes before the
call returns. The one uncovered reaper is the spawn path racing a concurrent
release(), and calling a method on an object another thread is releasing is
caller-side UB that faults on correct code too. A test built on that race would
be red on green code, so it is not a usable detector.
SO STOP RACING AND OBSERVE. tests/protocol/test_plain_waiter_publish_is_last.cpp
drives a real PlainLogosObject through a scripted RpcConnectionBase — no socket,
no host, no event-loop timing, and the test decides exactly when the call's
future is satisfied — and watches the accesses in two halves.
* THE STATE. The object is placement-newed into an mmap'd two-page arena, put
down so a page boundary lands at m_waiterMu: the members teardown
coordinates on go on the second page, everything else on the first. The
first page is mprotect(PROT_NONE)'d for exactly as long as a waiter runs,
and a SIGSEGV/SIGBUS handler RECORDS each access — address, thread, and how
many ids were published at that instant — then unprotects so the access
proceeds. Nothing crashes; the access is evidence. A correct waiter touches
that page zero times: objectName and method are copied into the closure
precisely so it needs nothing from the object. Four rounds, one per exit
path out of the lambda (answered, rejected, timed out, cancelled), since all
four end in the same guard.
* THE REGISTRY, which that page cannot cover because publishing has to reach
it. Caught with bait, using the reaper's own shape: reapFinishedWaiters()
joins outside m_waiterMu, so a waiter that has picked up somebody else's
finished thread sits in that join holding nothing — a window the test holds
open as long as it likes, because the thread being joined is one the test
planted and keeps parked. Plant bait 1; let the call finish; the exit guard
reaps, takes it, parks. Plant bait 2 at leisure. Release bait 1; the waiter
finishes its reap and publishes. Bait 2 must still be registered. Bait 1
doubles as a check that the reap really does join with the lock free.
Neither half is probabilistic. A third test proves the detector can fire at all,
so the two "this counter stayed at zero" assertions are not vacuous.
MEASURED, rebuilding the file under test with each defect (caught/runs):
defect below publishFinishedWaiter() new teardown+reaping
------------------------------------ --- ----------------
read m_objectName 40/40 0/10
read m_conn 10/10 0/10
read m_completions 10/10 0/10
read m_completionSubscribed 10/10 0/10
lock m_mu 10/10 0/10
call reapFinishedWaiters() again 20/20 2/2
read m_stopping 0/10 0/10
(publish moved ABOVE the reap) 0/5 12/15
no defect — 8f0c60f 0/40 0/10
The one gap is m_stopping, the single member sharing the registry's page, which
cannot be guarded without guarding the publish. The inverted order is left to
the reaping suite's hammer, which has it covered. Runtime 0.9-1.0s for all
three tests; clean 40/40 on 8f0c60f, and clean 3/3 under Guard Malloc
(MALLOC_PROTECT_BEFORE=1, banner confirmed) — the test never touches freed
memory itself, which is the other half of not being built on UB. No Guard
Malloc needed to detect anything: mprotect and the bait are the detectors.
Also: nix build .#tests 100% (285/285), the full binary 285/285, and
CallErrorAfterAcquireTest 40/40.
CORRECTIONS to measurements claimed earlier on this branch. All three were
overstated in the same direction — a single sample read as a constant:
* "ReapingRacesPublishingWithoutDeadlocking aborts the process, 5 runs out of
5" (plain_logos_object.cpp, and 378d889's message) is 12 runs in 15, ~80%.
It is a race detector, so one green run of it proves nothing — which is
exactly the argument for the deterministic suite added here. Corrected in
the comment.
* C-ABI retention was reported as "+0.09 MiB / 10 B per call" for 10k
lp_invoke_async on one client (8f0c60f's message). ~6 B/call. Same
conclusion — flat — different arithmetic.
* The burst retention figures 1428 (2000 calls, idle) and 610 of 800 came
back as 1421 and 599 on re-measure of the same build. Race-dependent, same
magnitude, which is why the tests assert a bound and not a value. Noted in
test_plain_waiter_reaping.cpp so the next reader does not treat them as
reproducible constants.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* test(protocol): make the publish-is-last probe fail loudly, and state the rule it actually checks
Three defects in cb015f5's regression test, plus the corrections that commit's
own CORRECTIONS section still owed. No behaviour change: the diff against
8f0c60f under cpp/ is comment lines only, verified by filtering the diff.
(a) THE BAIL-OUT PATH HUNG INSTEAD OF FAILING, which is the one that can stall
CI. PublishedWaiterDoesNotTouchTheRegistryAgain plants parked "bait" threads
behind gates and registers them in m_waiters. An ASSERT that fires before
gate1.open() — ASSERT_TRUE(tookBait1) is the obvious one — returns from the
function, and then ~PlainLogosObject blocks forever joining a thread nobody
will release. Reproduced by removing the reap from the exit guard: the
assertion PRINTS and the run still ends as a timeout kill, exit 124, with no
test result at all.
The gates are now opened by a scope guard on every exit path, and declared
BEFORE the GuardedObject so they outlive the teardown that joins the threads
parked on them. Same break, after: the same assertion, exit 1, 10.0s — which
is the probe's own tryWithRegistry budget and not a hang.
This is the shape a future refactor trips, not a hypothetical: the TODO
above the waiter (fold the wait into the shared Asio io_context) moves where
reaping happens, which is exactly the edit that makes tookBait1 false.
Both tests also stopped capturing their delivery counter by reference. On a
bail-out the cancelled call's callback is delivered on a later event-loop
iteration, i.e. after the frame is gone — a real use-after-free on the way
out of a failing test in a file about use-after-free. Owned by the callback
now.
(b) THE STATE ASSERTION WAS STRICTER THAN THE INVARIANT. It asserted
accessCount() == 0; the rule is only "no access AFTER the publish", and the
fault handler already stamps each access with how many ids were published at
that instant, so it can tell them apart.
Not hypothetical either. On the deferred/"multi" path a CORRECT waiter calls
awaitCompletion() (plain_logos_object.cpp:338), which locks m_completionMu
and reads m_completions and m_objectName — all on the guarded page, all
before it publishes. cb015f5 was green only because none of its four rounds
returned a pending sentinel, and the header's claim that "a correct waiter
touches that page ZERO times, before the publish or after" was true only of
the non-deferred rounds.
So: a fifth round drives the pending-sentinel path (ScriptedConn now answers
with the sentinel; nothing pushes the completion, so awaitCompletion runs out
its deadline), and the assertion narrowed to accesses stamped published >= 1.
PROVEN BOTH WAYS on this tree — with the old accessCount() == 0 predicate the
new round fails on correct code, naming offset 144 with "0 waiter id(s)
already published"; with the narrowed one the suite is 30/30 clean.
The round cannot pass vacuously: it REQUIRES at least one recorded access, so
a machine slow enough to turn it into a plain timeout fails it instead of
quietly proving nothing. Each round also now asserts m_finishedWaiters is
empty before arming, which is what makes "published >= 1" mean "after THIS
waiter's publish".
Two things guard against the narrowing being a quiet disarm:
* the handler now re-arms. It could not before (the faulting instruction
re-runs immediately), so the observing thread does it — it polls the
registry anyway and never touches the guarded page. Without it the first
legitimate access disarms the detector for the whole round.
* the catch rates were re-measured, not assumed. They are unchanged.
(c) TWO OVERSTATED NUMBERS, in the section whose whole point was to stop
overstating. Fixed where they live; cb015f5 is pushed and is not rewritten.
* "C-ABI retention ~6 B/call, not 10" replaced one sample with another.
10k lp_invoke_async on one lp_client, run ten times: 0, 5, 5, 5, 7, 7, 8,
10, 13, 10 bytes/call (mean 7.0). Ten more, run here: 3, 11, 8, 5, 8, 10,
13, 8, 3, 10 (mean 7.9). One distribution, range 0-13; both 6 and 10 sit
inside it and 8f0c60f's arithmetic (0.09 MiB / 10k) was not wrong.
THE HONEST STATEMENT IS THAT IT IS FLAT: indistinguishable from zero, RSS
noise and not a per-call rate. Recorded in test_plain_waiter_reaping.cpp
beside the other retention figures, where the next person to quote one
will see it.
* the table cell "reapFinishedWaiters() again ... 2/2" for the older
teardown+reaping suites was a two-run sample printed beside 10-40 run
samples. Re-measured over 30 runs: 9/30 here, 12/30 on another 30-run
sample — roughly one run in three, matching what reapFinishedWaiters'
own comment already said ("about one run in four"). The cell now reads
9/30, and the table says to read that column as rates and the left-hand
one as deterministic.
ALSO STATED PLAINLY, because it was overstated in review: the window where
"teardown returns while a reaped waiter is still unwinding" is NOT reachable by
a supported caller. A waiter leaves m_waiters only via teardown (which joins it)
or via a reaper, and that reaper is either another waiter — still registered
itself, since it reaps before it publishes, so teardown joins it and therefore
waits out the join it is in — or the async-spawn path, whose join completes
before the call returns. The only uncovered reaper is the spawn path racing a
concurrent release(), which is caller-side UB on any version of this class.
So publish-is-last is an invariant the design rests on and documents, not a
lurking use-after-free. This suite pins it against future edits; it does not
close an open hole. The file header, both failure messages and the comment in
plain_logos_object.cpp now say that instead of implying otherwise.
MEASURED AFTER THE CHANGE, rebuilding plain_logos_object.cpp with each defect
below publishFinishedWaiter() and running the suite (caught/runs), beside the
numbers from before it:
defect before after
------ ------ -----
read m_objectName 25/25 25/25
lock m_mu 25/25 25/25
write m_completions under m_completionMu 25/25 25/25
call reapFinishedWaiters() again (bait) 20/20 20/20
read m_conn - 10/10
read m_completions - 10/10
read m_completionSubscribed - 10/10
read m_stopping (declared blind spot) 0/10 0/10
publish moved ABOVE the reap (delegated) 0/5 0/5
no defect 0/30 0/30
Nothing moved, including the two declared blind spots — a narrowing that had
started catching or stopped catching something would show here. cb015f5 reported
40/40 for m_objectName from a longer run; 25/25 is this run, not a regression.
WHERE THE DEFERRED ROUND IS WEAKER, said here rather than left to be found: on
that one round the post-publish half is best-effort. A legitimate access opens
the page, the re-arm is a syscall behind, and a defect firing a microsecond
later slips through — measured with every round forced to run, the other four
catch a post-publish m_objectName read 5/5 and the deferred round 0/5, and a
variant that spins on the re-arm instead of polling records 4-24 accesses per
round and still catches it 0/5. It costs nothing: FinishOnExit is ONE piece of
code shared by all five exit paths, so the same defect is the same defect on
every round and the other four catch it deterministically. The deferred round is
there to keep the assertion honest about correct code, not to add a fifth copy
of the same detection.
Verified: PlainWaiterPublishIsLastTest 30/30 clean, the three waiter suites
15/15, the full binary 285/285, `nix build .#tests` 100% (285/285),
CallErrorAfterAcquireTest 40/40. Suite runtime 1.3-1.5s for the three tests
(0.9-1.0s before — the deferred round waits out a 400ms completion deadline).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
* feat: group-shareable local sockets, stale-socket reaper, bind-failure detection
The QtRO local transport binds each module's unix socket at 0777 & ~umask
(0755) with no way for a second OS user to reach it, discards the listen
result so a failed bind surfaces only as clients hanging, and never cleans up
the socket file — a hard-killed logos_host leaks it forever.
Add a Qt-free helper (logos_socket_paths.{h,cpp}) usable from both the qt_remote
and plain transport paths:
- applySocketPerms(path): chgrp + chmod a bound socket per LOGOS_SOCKET_GROUP /
LOGOS_SOCKET_MODE (chgrp-then-chmod so a half-applied policy is only ever
too strict). No-op when unset, so default behaviour is unchanged. Connecting
to an AF_UNIX socket needs write permission, so 0660 is what lets a group
member in.
- isSocketDead(path): S_ISSOCK && owned-by-us && non-blocking connect returns
ECONNREFUSED/ENOENT. Fails closed on any other outcome, so it never reports
a live socket or a regular file dead.
- reapStaleSockets(dir, prefix): unlink only the dead sockets, never a regular
file that shares the prefix (e.g. a *.lgx build artefact).
Wire it into RemoteTransportHost::publishObject and QtRemoteRegistry:
- construct QRemoteObjectRegistryHost empty and listen via setRegistryUrl() so
a bind failure is observed and logged (with lastError() + the socket path)
instead of leaving a silently-broken host;
- apply the socket-access policy to the freshly-bound local: socket.
The env-driven policy means every process in a node's tree (daemon, logos_host
subprocesses, their children) applies the same rule to every socket it binds
without threading config through each layer — the daemon exports the vars once.
Adds test_socket_paths.cpp (8 gtests): mode/group application, no-op default,
bad-mode rejection, live/dead/regular-file classification, and the reaper
keeping live sockets and regular files while removing only dead ones.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* review: harden socket helpers (gid overflow, socket-owner check, empty-prefix guard, dedup path)
Addressing automated review feedback on the socket helpers:
- resolveGid(): validate strtoul() errno/range so an out-of-range numeric
LOGOS_SOCKET_GROUP is rejected instead of silently truncating to a wrong gid.
- applySocketPerms(): when a policy is requested, stat the path first and refuse
unless it's a socket we own (S_ISSOCK + st_uid == geteuid()), so a malformed
URL can never chmod/chown a stray file. No-op fast path when the env is unset.
- reapStaleSockets(): refuse an empty prefix (would make every dead socket the
process owns a deletion candidate).
- Extract the duplicated `localSocketFilePath()` (Qt QLocalServer name->path
rule) into a shared qt_remote/qt_socket_path.h so RemoteTransportHost and
QtRemoteRegistry can't drift.
Adds tests: non-socket path refused (mode unchanged), empty-prefix reaper no-op.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* feat: transport-aware token validator hook on ModuleProxy (#22)
* feat: transport-aware token validator hook on ModuleProxy
Adds an injectable authorizer so a host (the logoscore daemon) can accept tokens
the built-in issued-token scan doesn't know — specifically operator-issued named
tokens validated against a persistent store — with per-token expiry and
local_only enforced against the transport the call arrived on.
- ModuleProxy::setTokenValidator(std::function<bool(token, transportProtocol)>).
isAuthorized() consults it ONLY after the existing m_tokens + TokenManager
scan fails, so installing a validator is purely additive: it can grant, never
revoke, access the built-in path already allows. Empty (default) = today's
behaviour exactly.
- callRemoteMethod() gains a defaulted `transportProtocol` ("local"). The QtRO
local path (RemoteTransportHost) uses the default; PlainTransportHost::onCall
passes the real wire ("tcp" | "tcp_ssl", fail-closed to non-local on an
unexpected protocol) so a local_only token presented over the network is
rejected. One ModuleProxy is shared across a provider's transports, so the
transport can't be inferred — it must be threaded per call, which the defaulted
arg does without changing the QtRO replica's 3-arg call.
The daemon backs the validator with TokenStore::lookupByToken; other modules
keep the default (no validator) and are unaffected.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
* review: split callRemoteMethod into explicit 3-arg + 4-arg overloads; include <utility>
Addressing review feedback:
- Replace the defaulted transportProtocol argument with two explicit Q_INVOKABLE
overloads. The Qt meta-object system matches methods by their full parameter
list and doesn't apply C++ default arguments, so the QtRO/local 3-arg call
must remain a real 3-arg method rather than relying on moc's reduced-arity
generation. The 3-arg form forwards to the transport-aware 4-arg form with
"local"; PlainTransportHost keeps calling the 4-arg form with the real wire.
- Include <utility> explicitly in module_proxy.h for std::move rather than
relying on an indirect include.
Full protocol suite green (160/160).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
* Extract the Logos protocol layer from logos-cpp-sdk
Transports (plain TCP/TLS, qt_local, qt_remote/QRO, mock), token manager,
consumer core (LogosAPIClient/LogosAPIConsumer incl. the capability
auto-requestModule flow), ModuleProxy, the abstract LogosProviderObject
interface, and the canonical QVariant<->JSON conversion — now behind the
language-neutral lp_* C ABI (logos_protocol.h) carrying the protocol
semver (LOGOS_PROTOCOL_VERSION_*, lp_protocol_version()).
Bytes crossing the ABI use the lossless {"_bytes": base64url} tagging
(NUL-safe), matching the plain wire encoding.
Provider lp_* surface is compiled groundwork; serving lands with module
authoring.
* consumer: typed requestModule for the capability flow
Port of logos-cpp-sdk master f5a127dd ('use updated capability module',
cpp-sdk#85, Iuri Matias) — the touched files (logos_api_client.cpp,
logos_api_consumer.{h,cpp}) moved into this repo in the P1 extraction.
The capability auto-requestModule path now calls a typed std::string
helper on the consumer (which acquires the capability object directly)
instead of a stringly invokeRemoteMethod round-trip. 111/111 tests.
