diff --git a/cfgsync/README.md b/cfgsync/README.md new file mode 100644 index 0000000..28adeee --- /dev/null +++ b/cfgsync/README.md @@ -0,0 +1,293 @@ +# cfgsync + +`cfgsync` is a small library stack for node registration and config artifact delivery. + +It is designed for distributed test and bootstrap flows where nodes need to: +- register themselves with a config service +- wait until config is ready +- fetch one artifact payload containing the files they need +- write those files locally and continue startup + +The important boundary is: +- `cfgsync` owns transport, registration storage, polling, and artifact serving +- the application adapter owns readiness policy and artifact generation + +That keeps `cfgsync` generic while still supporting app-specific bootstrap logic. + +## Crates + +### `cfgsync-artifacts` +Data types for delivered files. + +Primary types: +- `ArtifactFile` +- `ArtifactSet` + +Use this crate when you only need to talk about files and file collections. + +### `cfgsync-core` +Protocol and server/client building blocks. + +Primary types: +- `NodeRegistration` +- `RegistrationPayload` +- `NodeArtifactsPayload` +- `CfgsyncClient` +- `NodeConfigSource` +- `StaticConfigSource` +- `BundleConfigSource` +- `CfgsyncServerState` +- `build_cfgsync_router(...)` +- `serve_cfgsync(...)` + +This crate defines the generic HTTP contract: +- `POST /register` +- `POST /node` + +Typical flow: +1. client registers a node +2. client requests its artifacts +3. server returns either: + - `Ready` payload + - `NotReady` + - `Missing` + +### `cfgsync-adapter` +Adapter-facing materialization layer. + +Primary types: +- `NodeArtifacts` +- `NodeArtifactsCatalog` +- `RegistrationSnapshot` +- `NodeArtifactsMaterializer` +- `RegistrationSnapshotMaterializer` +- `CachedSnapshotMaterializer` +- `MaterializingConfigSource` +- `SnapshotConfigSource` +- `DeploymentAdapter` + +This crate is where app-specific bootstrap logic plugs in. + +Two useful patterns exist: +- single-node materialization + - `NodeArtifactsMaterializer` +- whole-snapshot materialization + - `RegistrationSnapshotMaterializer` + +Use snapshot materialization when readiness depends on the full registered set. + +### `cfgsync-runtime` +Small runtime helpers and binaries. + +Primary exports: +- `ArtifactOutputMap` +- `register_and_fetch_artifacts(...)` +- `fetch_and_write_artifacts(...)` +- `run_cfgsync_client_from_env()` +- `CfgsyncServerConfig` +- `CfgsyncServingMode` +- `serve_cfgsync_from_config(...)` +- `serve_snapshot_cfgsync(...)` + +This crate is for operational wiring, not for app-specific logic. + +## Design + +There are two serving models. + +### 1. Static bundle serving +Config is precomputed up front. + +Use: +- `NodeArtifactsBundle` +- `BundleConfigSource` +- `CfgsyncServingMode::Bundle` + +This is the simplest path when the full artifact set is already known. + +### 2. Registration-backed serving +Config is produced from node registrations. + +Use: +- `RegistrationSnapshotMaterializer` +- `CachedSnapshotMaterializer` +- `SnapshotConfigSource` +- `serve_snapshot_cfgsync(...)` + +This is the right model when config readiness depends on the current registered set. + +## Public API shape + +### Register a node + +Nodes register with: +- stable identifier +- IPv4 address +- optional typed application metadata + +Application metadata is carried as an opaque serialized payload: +- generic in `cfgsync` +- interpreted only by the adapter + +Example: + +```rust +use cfgsync_core::NodeRegistration; + +#[derive(serde::Serialize)] +struct MyNodeMetadata { + network_port: u16, + api_port: u16, +} + +let registration = NodeRegistration::new("node-1", "127.0.0.1".parse().unwrap()) + .with_metadata(&MyNodeMetadata { + network_port: 3000, + api_port: 18080, + })?; +``` + +### Materialize from the registration snapshot + +```rust +use cfgsync_adapter::{ + DynCfgsyncError, NodeArtifacts, NodeArtifactsCatalog, RegistrationSnapshot, + RegistrationSnapshotMaterializer, +}; +use cfgsync_artifacts::ArtifactFile; + +struct MyMaterializer; + +impl RegistrationSnapshotMaterializer for MyMaterializer { + fn materialize_snapshot( + &self, + registrations: &RegistrationSnapshot, + ) -> Result, DynCfgsyncError> { + if registrations.len() < 2 { + return Ok(None); + } + + let nodes = registrations + .iter() + .map(|registration| NodeArtifacts { + identifier: registration.identifier.clone(), + files: vec![ArtifactFile::new( + "/config.yaml", + format!("id: {}\n", registration.identifier), + )], + }) + .collect(); + + Ok(Some(NodeArtifactsCatalog::new(nodes))) + } +} +``` + +### Serve registration-backed cfgsync + +```rust +use cfgsync_runtime::serve_snapshot_cfgsync; + +# async fn run() -> anyhow::Result<()> { +serve_snapshot_cfgsync(4400, MyMaterializer).await?; +# Ok(()) +# } +``` + +### Fetch from a client + +```rust +use cfgsync_core::CfgsyncClient; + +# async fn run(registration: cfgsync_core::NodeRegistration) -> anyhow::Result<()> { +let client = CfgsyncClient::new("http://127.0.0.1:4400"); +client.register_node(®istration).await?; +let payload = client.fetch_node_config(®istration).await?; +# Ok(()) +# } +``` + +### Fetch and write artifacts with runtime helpers + +```rust +use cfgsync_runtime::{ArtifactOutputMap, fetch_and_write_artifacts}; + +# async fn run(registration: cfgsync_core::NodeRegistration) -> anyhow::Result<()> { +let outputs = ArtifactOutputMap::new() + .route("/config.yaml", "/node-data/node-1/config.yaml") + .route("deployment-settings.yaml", "/node-data/shared/deployment-settings.yaml"); + +fetch_and_write_artifacts(®istration, "http://127.0.0.1:4400", &outputs).await?; +# Ok(()) +# } +``` + +## What belongs in the adapter + +Put these in your app adapter: +- registration payload type +- readiness rule +- conversion from registration snapshot to artifacts +- shared file generation if your app needs shared files + +Examples: +- wait for `n` initial nodes +- derive peer lists from registrations +- build node-local config files +- include shared deployment/config files in every node payload + +## What does **not** belong in `cfgsync-core` + +Do not put these into generic cfgsync: +- app-specific topology rules +- domain-specific genesis/deployment generation +- app-specific command/state-machine logic +- service-specific semantics for what a node means + +Those belong in the adapter or the consuming application. + +## Recommended integration model + +If you are integrating a new app, start here: + +1. define a typed registration payload +2. implement `RegistrationSnapshotMaterializer` +3. return one artifact payload per node +4. include shared files inside that payload if your app needs them +5. serve with `serve_snapshot_cfgsync(...)` +6. use `CfgsyncClient` on the node side +7. use runtime helpers if you want generic client-side file writing instead of custom dispatch code + +This model keeps the generic library small and keeps application semantics where they belong. + +## Compatibility + +The primary surface is the one reexported from crate roots. + +There are hidden compatibility aliases in some crates to keep older internal consumers building, but they are not the recommended API for new integrations. + +## Runtime config files + +`serve_cfgsync_from_config(...)` is for runtime-config-driven serving. + +Today it supports: +- static bundle serving +- registration serving from a prebuilt artifact catalog + +If your app has a real registration-backed materializer, prefer the direct runtime API: +- `serve_snapshot_cfgsync(...)` + +That keeps application behavior in adapter code instead of trying to encode it into YAML. + +## Current status + +`cfgsync` is suitable for: +- internal reuse across multiple apps +- registration-backed bootstrap flows +- static precomputed artifact serving + +It is not intended to be: +- a generic orchestration framework +- a topology engine +- a secret-management system +- an app-specific bootstrap policy layer