Dario LipicarandClaude Opus 5
84564f0f6a
* feat(core): observe module lifecycle as sequenced facts, not log lines
Until now load, unload and crash were spdlog::info lines and registry
membership changes were silent, so every consumer polled — basecamp runs a
2s QTimer and infers module state from package-install events.
ModuleStateObserver turns those into structured, sequenced transitions and
hands them to a SINK. It reports only: it does not talk to any module and
does not know modules_state exists. Wiring a sink that pushes to that module
is the next stage. With no sink installed record() early-outs before it
allocates, so a host consuming nothing pays nothing — and buffering with no
consumer would be an unbounded leak in the normal case.
TWO RULES, both load-bearing:
1. Never dispatch under loadMutex(). record() buffers; flush() dispatches,
and every entry point declares ScopedModuleStateFlush BEFORE its lock
guard so it is destroyed AFTER it. A sink doing an RPC from inside the
load path while holding that lock is the shape of two failures already
paid for here: the ui-host startup token deadlock, and the ~417s basecamp
stall from a synchronous call to an absent module.
2. One seq counter, for deltas and snapshots alike. Consumers apply a
transition only when its seq beats what they hold for that module, and
keep a seq tombstone for a departed one. A second counter makes that
tombstone either unreachably high (a real later delta dropped forever) or
trivially low (a stale delta resurrecting a pruned module).
Seams: unloaded->loading at load start, loading->loaded on success carrying
instanceId and pid, loading->error on all three failure paths,
loaded->stopping->unloaded on unload, and the membership edges
absent->unloaded / unloaded->absent in discovery and prune. processModule()
gets the discovery edge too — it is a second way a module enters the
registry, and a consumer that only saw scan edges would be surprised by a
`unloaded -> loading` for a module it had never heard of.
Two bugs found while wiring it, both of which would have made the feed lie:
* onTerminated fires for BOTH an orderly unload and a module that died,
and cannot tell them apart from its arguments. Teardown now announces
intent before terminate(); the callback consumes it, once, so an
unload/reload/crash still reads as a crash.
* terminateAll() and clear() tear down every loaded module at once, so a
CLEAN HOST SHUTDOWN would have reported the entire fleet as crashed —
`loaded -> error`, "module exited without being asked to", once per
module. They now announce every loaded module first.
9 tests, and they are proven to bite: making record() dispatch inline (the
rule-1 violation) fails RecordDoesNotDispatch and reddens the check. Full
suite 194/194.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* feat(core): feed module lifecycle to modules_state
The consumer end of ModuleStateObserver. The observer produces sequenced
transitions and knows nothing about any module; this turns them into calls on
`modules_state`, so the state liblogos has always had stops being spdlog lines
and becomes queryable and subscribable.
Modelled on the capability_module push already in this file -- one long-lived
"core" LogosAPI, per-module transport honoured -- with three differences, all
forced by where it runs:
1. ASYNC deltas. registerRestrictionRpc is synchronous and gets away with it
because it is rare and short. This runs on EVERY load, unload and crash,
from the observer's flush, on whichever thread did the work. A synchronous
RPC there would put a 20 s worst case on the load path.
2. CHEAP NO-OP WHEN ABSENT, checked before the client is even fetched. A
synchronous dial to a module that is not there cost Basecamp ~417 s of
blocked GUI thread once already.
3. THE SINK IS UNINSTALLED when modules_state unloads, so the observer goes
back to buffering nothing rather than buffering into a sink that drops.
SNAPSHOT ON AVAILABILITY, NOT ON LOAD. modules_state loads after other modules,
so deltas alone give it a permanently short list -- which is what `partial`
exists to signal. The snapshot clears it. It is armed with whenObjectAvailable()
rather than fired at load, because `load-module` returns when the plugin is IN
and the module PUBLISHES later -- measured elsewhere at ~390 ms on a cold start
-- and whenObjectAvailable is the primitive that waits without failing fast or
burning the acquire timeout on the calling thread.
ONE SEQ COUNTER. Every record seq and the listing seq come from the observer's
counter, listing drawn LAST so it is >= every record in it. modules_state
tombstones a pruned record at the LISTING's seq, so a second counter would make
that tombstone either unreachably high (a real later delta dropped forever) or
trivially low (a stale delta resurrecting a pruned module).
partial:false is a claim, and it is defensible: `partial` means the host's scan
SKIPPED something, and discoverInstalledModules drops a module it cannot read
before it ever enters the registry. Anything missing is not something the host
knows and is withholding; it is something the host does not know.
PROVEN END TO END against a live daemon, with NO test door open -- so the facts
arriving also prove the module's structural gate admitted core as kind=host:
snapshot: "Pushed module snapshot to modules_state", and list_modules then
reports both modules with real paths and load timestamps,
partial:false, seqs 1/2 under listing seq 3.
delta: unload capability_module -> its record goes loaded(seq 2) ->
unloaded(seq 5) through the ASYNC path, which is separate code from
the snapshot's synchronous one and needed proving separately.
refusals: 0.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* refactor: cut the comment walls
Same pass as the module side. The prose had grown past the point where it helps.
module_state_observer.h 174 -> 127 lines, 103 -> 56 comment
module_manager.cpp 285 -> ~215 comment
module_registry.cpp / observer.cpp / tests / README trimmed to match
Kept the non-obvious why: the two rules (never dispatch under loadMutex, one seq
counter for deltas and snapshots), orderly-teardown vs death, why a clean
shutdown would otherwise report the fleet as crashed, and why partial:false is a
defensible claim rather than an assumption.
The worst offender was mechanical: the four-line "declare the flusher before the
lock guard" explanation was pasted at all SEVEN call sites. It is now one line
pointing at the rule in the header, where the explanation lives once.
194/194.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* feat(core): emit loaded -> ready when a module publishes
`loaded` means the host owns the process: markLoaded runs once the child is
spawned and its token is written, which is strictly before the module publishes
its object (~390ms cold for modules_state; 826ms measured for a module with an
empty initializer). Nothing emitted that gap, so consumers had to treat "loaded"
as "callable" and be wrong for the width of the window.
Adds a second edge rather than moving the first. `loaded` stays an ownership
fact -- total across loaders, and the thing unload and the double-spawn guard
need. `ready` is the readiness fact, observed asynchronously.
armReadinessWatch arms a one-shot whenObjectAvailable and returns: it never
waits, so rule 1 (no dispatch under loadMutex) holds, and the callback lands
with no lock held. Only armed when a sink is installed -- without a consumer
each watch would hold a client and a replica for nothing, and it keeps the
FakeModuleLoader tests, which publish nothing, arming nothing.
The callback carries the loadEpoch it was armed under and markPublished drops
it if that no longer matches, so a fast unload/reload cannot let a stale watch
mark the new instance ready. Epoch rather than loadedAt: the latter is whole
seconds and collides.
modules_info gains `published` / `published_at`. published is null, not false,
when no watch is armed -- "nobody looked" and "not ready" are different answers.
Also folds capabilityModuleClient and modulesStateClient into one
moduleClient(name); they were identical apart from the name.
checks.tests green: 194 tests.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
13 KiB
logos-liblogos
The core runtime library for the Logos modular application platform. Provides liblogos_core (a C-API shared library) and logos_host (the module subprocess host binary).
logos-liblogos is a library. It is consumed by two frontends:
- logos-basecamp — the desktop GUI application shell
- logos-logoscore-cli — the headless CLI runtime (
logoscore)
Composed from
The runtime pulls its module-loading behaviour from a few separate repos so each piece can be swapped independently:
- logos-capability-module — issues the auth tokens that gate inter-module calls.
- logos-container — the container interface (where/how a module runs); current default is the subprocess implementation: logos-container-subprocess (one OS process per module).
- logos-module-loader — the format-loader interface (what kind of module); current default is : logos-module-loader-qt.
How to Build
The project uses a Nix flake for reproducible builds with a modular structure:
Build Complete Library (Binaries + Libraries + Headers)
# Build everything (default)
nix build
# Or explicitly
nix build '.#logos-liblogos'
nix build '.#default'
The result will include:
/bin/- Host binary (logos_host)/lib/- Core library (liblogos_core)/include/- Headers (logos_core.h, interface.h)
Build Individual Components
# Build only the binaries (outputs to /bin)
nix build '.#logos-liblogos-bin'
# Build only the libraries (outputs to /lib)
nix build '.#logos-liblogos-lib'
# Build only the headers (outputs to /include)
nix build '.#logos-liblogos-include'
# Build and run tests
nix build '.#logos-liblogos-tests'
# Build portable variant (selects portable LGX variants instead of dev)
nix build '.#portable'
Running Tests
# Build and run tests (tests run automatically during build)
nix build '.#logos-liblogos-tests'
# To run tests manually after building:
./result/bin/logos_core_tests
# Run specific tests
./result/bin/logos_core_tests --gtest_filter=AppLifecycleTest.*
# List all available tests
./result/bin/logos_core_tests --gtest_list_tests
Development Shell
# Enter development shell with all dependencies
nix develop
Note: In zsh, you need to quote targets with # to prevent glob expansion.
If you don't have flakes enabled globally, add experimental flags:
nix build '.#logos-liblogos' --extra-experimental-features 'nix-command flakes'
The compiled artifacts can be found at result/
Modular Architecture
The nix build system is organized into modular files in the /nix directory:
nix/default.nix- Common configuration (dependencies, flags, metadata)nix/build.nix- Shared build that compiles everything oncenix/bin.nix- Extracts binaries (logos_host, includes libraries for runtime linking)nix/lib.nix- Extracts libraries onlynix/include.nix- Header installationnix/tests.nix- Test suite build and execution
Note: The logos-liblogos-bin package includes both the logos_host binary and its required libraries to ensure proper runtime linking.
Local Development
To build against a local checkout of a dependency, override its input. For example the SDK:
nix build --override-input logos-cpp-sdk path:../logos-cpp-sdk
The container and format-loader abstractions live in their own repos, consumed
as inputs the same way process-stats is. To build against local checkouts:
nix build \
--override-input logos-container path:../logos-container \
--override-input logos-module-loader path:../logos-module-loader \
--override-input default-container path:../logos-container-subprocess \
--override-input default-module-loader path:../logos-module-loader-qt
(default-container / default-module-loader are the input slots for the
built-in default implementations; they point at the subprocess / qt-plugin repos.)
Library API
logos-liblogos exposes a C API via logos_core.h:
// Lifecycle
void logos_core_init(int argc, char *argv[]);
void logos_core_start();
void logos_core_cleanup();
// Module directory management
void logos_core_add_modules_dir(const char* dir);
// Instance persistence
void logos_core_set_persistence_base_path(const char* path);
// Per-module transport configuration (forwarded to the module's
// child subprocess so its LogosAPIProvider binds every listener
// instead of only the global default LocalSocket). Must be called
// before the module is loaded.
void logos_core_set_module_transports(const char* name, const char* transport_set_json);
// Inter-module access policy (per-target allowed-caller allowlists).
// Core parses it and registers the per-target restrictions with
// capability_module, which then denies token issuance for disallowed
// (caller, target) pairs. Call before logos_core_start(); NULL/"" clears.
void logos_core_set_access_policy(const char* policy_json);
// Module management
int logos_core_load_module(const char* name, bool with_dependencies);
int logos_core_unload_module(const char* name, bool with_dependents);
char* logos_core_process_module(const char* path);
void logos_core_refresh_modules();
// Dependency graph queries (forward + reverse edges; recursive walks BFS)
char** logos_core_get_module_dependencies(const char* name, bool recursive);
char** logos_core_get_module_dependents(const char* name, bool recursive);
// Module queries
char** logos_core_get_loaded_modules();
char** logos_core_get_known_modules();
// Module stats and tokens
char* logos_core_get_module_stats();
char* logos_core_get_token(const char* key);
See src/logos_core/logos_core.h for the full API.
Inter-module access enforcement (off by default)
By default a loaded module may call any other loaded module. Enforcement is
opt-in, and mode in the access policy is the switch:
{"version": 1, "mode": "enforce"}
Installing that document (via logos_core_set_access_policy, before
logos_core_start()) turns on deny-by-default: for every loaded target,
core derives the allowed callers from the declared dependency graph — the
target's loaded dependents, plus the trusted core / core_service — and
registers them with capability_module. A module that never declared the target
as a dependency is refused a token, so its call can never proceed, and
capability_module logs the refusal with both names:
[capability_module] access policy denies 'caller_module' -> 'target_module'
Anything other than mode: "enforce" — no policy, NULL, "", unparseable
JSON, a different mode — leaves enforcement off, which is the pre-existing
behaviour. Core says which side it landed on at startup, so a mistyped mode is
visible rather than silently permissive:
Inter-module access enforcement is ON (mode=enforce): deny-by-default — ...
Inter-module access enforcement is OFF (no access policy set): ...
A restrictions entry overrides the derived list for that target verbatim,
which is the escape hatch for callers that legitimately cannot declare their
target (out-of-process ui_qml plugins are not tracked as dependents, so they
need an explicit entry):
{"version": 1, "mode": "enforce",
"restrictions": {"accounts_module": {"allowedCallers": ["accounts_ui"]}}}
capability_module, core and core_service are never restricted as targets.
Hosts expose this as --access-policy — see the logoscore CLI and Basecamp
READMEs.
Thread safety
Module load/unload operations (logos_core_load_module, logos_core_unload_module) are serialised internally by a single mutex. It is safe to call them concurrently from multiple threads, including rapid and repeated load/unload cycles on the same module — each call waits for its turn and the process management layer handles teardown cleanly before the next launch. logos_core_unload_module with with_dependents=true in particular holds the lock for its entire leaves-first teardown so a late-arriving load can't interleave between tearing down a dependent and its parent.
logos_core_refresh_modules is synchronised through the module registry's reader-writer lock — it is safe to call concurrently with other registry accesses, but it is not serialised against load/unload by the same mutex as above.
Read-only accessors (logos_core_get_known_modules, logos_core_get_loaded_modules) use that shared reader-writer lock and are safe to call concurrently with each other and with logos_core_refresh_modules.
Module lifecycle observer
src/logos_core/module_state_observer.h turns lifecycle changes into
structured, sequenced facts. Until it existed, load/unload/crash were
spdlog::info lines and registry membership changes were silent, so every
consumer polled — logos-basecamp runs a 2s QTimer and infers module state
from package-install events.
It reports; it does not drive anything. Transitions go to a sink, and with
none installed record() early-outs before it allocates, so a host that
consumes nothing pays nothing. The sink that pushes to modules_state is in
module_manager.cpp.
States: unloaded, loading, loaded, stopping, error, plus the
event-only absent, which names the two membership edges (absent -> unloaded
on discovery, unloaded -> absent on prune).
Two rules govern every call site, and both are load-bearing:
-
Never dispatch under
loadMutex().record()buffers;flush()dispatches, and entry points declareScopedModuleStateFlushbefore their lock guard so it is destroyed after it. A sink doing an RPC from inside the load path while holding that lock is the shape of two failures already paid for here: the ui-host startup token deadlock, and a ~417s Basecamp stall from a synchronous call to an absent module. -
One
seqcounter, for deltas and snapshots alike. Consumers apply a transition only when itsseqbeats what they hold, and tombstone a departed record at a seq. A second counter makes that tombstone unreachably high (a real later delta dropped forever) or trivially low (a stale delta resurrecting a pruned module).
onTerminated fires for both an orderly unload and a module that died, so
teardown announces intent before terminate() and the callback consumes it.
Host shutdown announces every loaded module first — without that, a clean exit
reports the whole fleet as crashed.
Dev vs Portable Builds
The library supports two build modes controlled by the LOGOS_PORTABLE_BUILD CMake flag:
- Dev build (default): Module loading looks for LGX variants with
-devsuffix (e.g.,linux-amd64-dev). Used in Nix/development environments. - Portable build (
-DLOGOS_PORTABLE_BUILD=ON): Looks for portable variants without suffix (e.g.,linux-amd64). Used in self-contained distributed applications.
Build the portable variant with nix build '.#portable'.
Supported Platforms
- macOS (aarch64-darwin, x86_64-darwin)
- Linux (aarch64-linux, x86_64-linux)
Building with a different container or module loader
The container and format-loader are selected by which package liblogos links — not by its C++ or CMake. To use your own, build a package that:
- implements the contract interface (
LogosCore::ModuleContainerfromlogos-container, orLogosCore::ModuleFormatLoaderfromlogos-module-loader), - defines the factory symbol (
LogosCore::makeContainer()/LogosCore::makeFormatLoader()), and - ships the generic CMake config (
LogosContainerImpl/LogosFormatLoaderImpl, exposing the…::impltarget).
(Copy the structure from logos-container-subprocess / logos-module-loader-qt.)
Then point liblogos at it by overriding the input slot:
# different container
nix build '.#logos-liblogos' --override-input default-container <flake-ref>
# different format loader
nix build '.#logos-liblogos' --override-input default-module-loader <flake-ref>
<flake-ref> is e.g. github:you/your-impl or path:../your-impl; it must expose
packages.<system>.default carrying that config + factory symbol. For a permanent
default, add it as an input and set containerImpl / formatLoaderImpl in
flake.nix. Container and loader are independent — swap either or both.
Note: the format-loader package also provides the logos_host binary that
bin.nix re-exports, so a replacement loader must ship its own host binary.
Disclaimer
This repository is part of an experimental development environment. Components are under active development and may be incomplete, unstable, modified or discontinued at any time.
The software is provided for development and testing purposes only and is not intended for production use.
The code and related materials are made available on an open-source, “as-is” basis without warranties or guarantees of any kind, express or implied, including warranties of correctness, security, performance or fitness for a particular purpose. Use at your own risk.