# External Libraries Guide

How to wrap external C/C++ libraries in Logos modules.

> **Step-by-step tutorials.** Complete, runnable walkthroughs of each approach
> below live as executable doc-tests in
> [`doctests/`](https://github.com/logos-co/logos-module-builder/tree/master/doctests)
> (`wrap-external-lib-1-source`, `-2-prebuilt-binaries`, `-3-external-source`,
> `-4-nix-flake`). Each scaffolds a real module, builds it, loads it in
> `logoscore`, and calls it — run/published in CI via
> [logos-doctest](https://github.com/logos-co/logos-doctest).

## Overview

Logos modules can wrap external C/C++ libraries to expose their functionality to the Logos ecosystem. There are four approaches:

1. **Vendor/Pre-built** — Library already compiled, in the `lib/` directory (simplest)
2. **Flake Input (build from source)** — Library source as a flake input, built by `mkExternalLib` during nix build
3. **Flake Input (Nix package)** — Library provided by a flake that has its own Nix build. Detected automatically with `lib.isDerivation`; there is no `built_nix` flag to set
4. **Vendor Submodule** — Source in a git submodule, built by a `build_script`

## Approach 1: Vendor / Pre-built Library

Best for: Pre-built proprietary libraries or binaries you already have compiled.

### Setup

1. Place the pre-built library in `lib/` and **git-track it** (Nix only sees tracked files):
```bash
cp /path/to/libmylib.dylib lib/
git add lib/libmylib.dylib lib/libmylib.h
```

2. Configure `metadata.json`:
```json
{
  "nix": {
    "external_libraries": [
      { "name": "mylib", "vendor_path": "lib" }
    ],
    "cmake": {
      "extra_include_dirs": ["lib"]
    }
  }
}
```

3. `flake.nix` stays simple — no extra inputs needed:
```nix
{
  inputs = {
    logos-module-builder.url = "github:logos-co/logos-module-builder";
  };

  outputs = inputs@{ logos-module-builder, ... }:
    logos-module-builder.lib.mkLogosModule {
      src = ./.;
      configFile = ./metadata.json;
      flakeInputs = inputs;
    };
}
```

### Multiple platforms (per-platform binaries)

A flat `lib/libmylib.so` only works for the platform it was built for, and you
cannot keep both a `linux x86_64` and a `linux aarch64` build in `lib/` — they
share the name `libmylib.so`. To ship several platforms, put each binary in a
subdirectory named by its **Nix system string**; the build selects the one
matching the platform it is building for:

```
lib/
├── mylib.h                    # shared header, stays flat
├── x86_64-linux/libmylib.so
├── aarch64-linux/libmylib.so
├── x86_64-darwin/libmylib.dylib
└── aarch64-darwin/libmylib.dylib
```

The valid subdirectory names are `x86_64-linux`, `aarch64-linux`,
`x86_64-darwin`, `aarch64-darwin`. `vendor_path` and `metadata.json` are
unchanged (`{ "name": "mylib", "vendor_path": "lib" }`); commit only the
platforms you support. Don't mix layouts — use per-platform subdirs *or* a flat
binary for a given library, not both.

Notes:
- These are **Nix** system strings, distinct from the `.lgx` packaging variant
  labels (`linux-amd64`, `darwin-arm64`).
- Give shared libraries a SONAME (`libmylib.so`) / `install_name`
  (`@rpath/libmylib.dylib`) so the plugin records a relocatable dependency.
- Selection happens during the Nix build's staging step; raw `nix develop` +
  cmake does not descend into the subdirs.

Full walkthrough: the [`wrap-external-lib-2-prebuilt-binaries`](https://github.com/logos-co/logos-module-builder/tree/master/doctests/wrap-external-lib-2-prebuilt-binaries.test.yaml) doc-test.

## Approach 2: Flake Input (Build from Source)

Best for: Libraries with clean build systems (make, cmake, etc.) whose source you want pinned as a flake input.

### Configuration

`flake.nix`:
```nix
{
  inputs = {
    logos-module-builder.url = "github:logos-co/logos-module-builder";

    my-lib-src = {
      url = "github:org/my-lib/v1.0.0";
      flake = false;
    };
  };

  outputs = inputs@{ logos-module-builder, ... }:
    logos-module-builder.lib.mkLogosModule {
      src = ./.;
      configFile = ./metadata.json;
      flakeInputs = inputs;
      externalLibInputs = {
        mylib = inputs.my-lib-src;
      };
    };
}
```

`metadata.json`:
```json
{
  "nix": {
    "external_libraries": [
      {
        "name": "mylib",
        "build_command": "make shared",
        "output_pattern": "build/libmylib.*"
      }
    ],
    "cmake": {
      "extra_include_dirs": ["lib"]
    }
  }
}
```

### Build Command Options

```json
{ "build_command": "make" }
{ "build_command": "make shared-library" }
{ "build_command": "mkdir build && cd build && cmake .. && make" }
{ "build_command": "./build.sh" }
```

### Go Libraries

For Go libraries that produce C shared libraries:

```json
{
  "nix": {
    "external_libraries": [
      {
        "name": "gowalletsdk",
        "build_command": "make shared-library",
        "go_build": true
      }
    ]
  }
}
```

The `go_build: true` flag sets up `GOCACHE`, `GOPATH`, `CGO_ENABLED=1`, and the Go toolchain in the build environment.

## Approach 3: Flake Input (Nix Package)

Best for: Libraries that already have their own `flake.nix` producing a Nix derivation with `lib/` and `include/` outputs. The module builder detects this automatically — if the resolved input is a Nix derivation, it's used directly; no extra flags needed in `metadata.json`.

### How detection works

The module builder calls `lib.isDerivation` on the resolved input:
- **Derivation** (a specific package output) → used directly, no build step
- **Raw source** (non-flake input, `flake = false`) → built with `make` / custom command (Approach 2)

When you point `externalLibInputs` at a specific package output (or use the structured format with `packages`), the resolved value is always a derivation, so it's used as-is.

### Configuration

`flake.nix`:
```nix
{
  inputs = {
    logos-module-builder.url = "github:logos-co/logos-module-builder";
    my-lib.url = "github:org/my-lib";
  };

  outputs = inputs@{ logos-module-builder, ... }:
    logos-module-builder.lib.mkLogosModule {
      src = ./.;
      configFile = ./metadata.json;
      flakeInputs = inputs;
      externalLibInputs = {
        mylib = inputs.my-lib;
      };
    };
}
```

`metadata.json` — only the name is needed:
```json
{
  "nix": {
    "external_libraries": [
      { "name": "mylib" }
    ],
    "cmake": {
      "extra_include_dirs": ["lib"]
    }
  }
}
```

If `my-lib` has `packages.${system}.default`, the module builder resolves to that derivation and uses it directly. If it's a non-flake source repo, it falls back to building with `make`.

### Per-variant packages

If the flake input provides multiple package outputs (e.g. a dev build and a portable build), use the structured `externalLibInputs` format:

```nix
externalLibInputs = {
  mylib = {
    input = inputs.my-lib;
    packages = {
      default = "lib";           # used for nix build .#lib
      portable = "lib-portable"; # used for nix build .#lib-portable
    };
  };
};
```

## Approach 4: Vendor Submodule (Build from Source in Repo)

Best for: Libraries requiring custom build scripts, where source lives in a git submodule.

### Setup

1. Add library as git submodule:
```bash
git submodule add https://github.com/org/my-lib vendor/my-lib
```

2. Create build script:
```bash
# scripts/build-mylib.sh
#!/bin/bash
cd vendor/my-lib
make clean
make shared
cp build/libmylib.* ../../lib/
```

### Custom Build Scripts

Build scripts receive no arguments and should:
1. Build the library
2. Copy outputs to `lib/` directory

Example for nwaku/libwaku:
```bash
#!/bin/bash
set -e

cd vendor/nwaku

# Build libwaku
make libwaku

# Copy to lib/
mkdir -p ../../lib
cp build/libwaku.* ../../lib/
cp library/libwaku.h ../../lib/
```

3. Configure `metadata.json`:
```json
{
  "nix": {
    "external_libraries": [
      {
        "name": "mylib",
        "vendor_path": "vendor/my-lib",
        "build_script": "scripts/build-mylib.sh"
      }
    ]
  }
}
```

## CMake Integration

### Basic Linking

In `CMakeLists.txt`:
```cmake
logos_module(
    NAME my_module
    SOURCES ...
    EXTERNAL_LIBS
        mylib
)
```

This will:
1. Search for library in `lib/`
2. Add `lib/` to include directories
3. Link the library
4. Copy library to output directory

### Manual Linking

For more control:
```cmake
# After logos_module()
find_library(EXTRA_LIB extralib PATHS ${CMAKE_CURRENT_SOURCE_DIR}/lib)
target_link_libraries(my_module_module_plugin PRIVATE ${EXTRA_LIB})
```

## Module Implementation

These snippets use the universal authoring model: you write only
`src/my_module_impl.{h,cpp}`, its public methods are the module's API, and the
code is **Qt-free** (`std::string`, not `QString`). The snippets here used to
show a hand-written `MyModulePlugin` with `Q_OBJECT` / `eventResponse`; that
class is generated now, and a core module that declares no `interface` is
refused at evaluation.

### Including Headers

```cpp
// In my_module_impl.h
#include "lib/libmylib.h"  // Include the C header
```

### Using the Library

```cpp
// In my_module_impl.cpp
#include "my_module_impl.h"
#include "lib/libmylib.h"

bool MyModuleImpl::initLibrary() {
    m_handle = mylib_init();
    return m_handle != nullptr;
}

void MyModuleImpl::cleanup() {
    if (m_handle) {
        mylib_cleanup(m_handle);
        m_handle = nullptr;
    }
}
```

### Memory Management

C libraries often return allocated memory. Always free it:

```cpp
std::string MyModuleImpl::getData() {
    char* result = mylib_get_data(m_handle);
    std::string output = result ? result : "";
    mylib_free_string(result);  // Don't forget!
    return output;
}
```

### Callbacks

For C callbacks, use static methods and route the payload into a typed event.
Declaring a method under `logos_events:` is all it takes — the generator writes
the body, and calling it fans the payload out to every subscriber.

```cpp
// src/my_module_impl.h
class MyModuleImpl : public LogosModuleContext {
public:
    void subscribe();

logos_events:
    /// Emitted for every callback the library delivers.
    void libraryEvent(int64_t code, const std::string& message);

private:
    static void callback(int code, const char* msg, void* user_data);
    void* m_handle = nullptr;
};

// src/my_module_impl.cpp
void MyModuleImpl::callback(int code, const char* msg, void* user_data) {
    auto* self = static_cast<MyModuleImpl*>(user_data);
    self->libraryEvent(code, msg ? msg : "");
}

void MyModuleImpl::subscribe() {
    mylib_subscribe(m_handle, callback, this);  // Pass 'this' as user_data
}
```

## Platform Considerations

### macOS

Libraries need correct install names. The builder automatically runs:
```bash
install_name_tool -id "@rpath/libmylib.dylib" libmylib.dylib
```

For the plugin:
```bash
install_name_tool -change "/old/path/libmylib.dylib" "@rpath/libmylib.dylib" my_module_plugin.dylib
```

### Linux

Libraries are found via `$ORIGIN` RPATH:
```bash
patchelf --set-rpath '$ORIGIN' my_module_plugin.so
```

### Troubleshooting

**Library not found at runtime:**
```bash
# Check RPATH on macOS
otool -L my_module_plugin.dylib

# Check RPATH on Linux
readelf -d my_module_plugin.so | grep RPATH
ldd my_module_plugin.so
```

**Symbol not found:**
```bash
# List symbols in library
nm -gU libmylib.dylib

# Check if symbol is referenced
nm -u my_module_plugin.dylib | grep mylib
```

**Library not copied to result/lib:**

For vendor libraries: ensure the `.dylib`/`.so` is git-tracked:
```bash
git add lib/libmylib.dylib
```

## Complete Example: Wallet Module

Here's how the wallet module wraps go-wallet-sdk:

`flake.nix`:
```nix
{
  inputs = {
    logos-module-builder.url = "github:logos-co/logos-module-builder";
    go-wallet-sdk = {
      url = "github:status-im/go-wallet-sdk/v1.0.0";
      flake = false;
    };
  };

  outputs = inputs@{ logos-module-builder, ... }:
    logos-module-builder.lib.mkLogosModule {
      src = ./.;
      configFile = ./metadata.json;
      flakeInputs = inputs;
      externalLibInputs = {
        gowalletsdk = inputs.go-wallet-sdk;
      };
    };
}
```

`metadata.json`:
```json
{
  "name": "wallet_module",
  "version": "1.0.0",
  "type": "core",
  "interface": "universal",
  "category": "wallet",
  "main": "wallet_module_plugin",
  "dependencies": [],
  "nix": {
    "packages": { "build": ["gnumake", "go"], "runtime": [] },
    "external_libraries": [
      {
        "name": "gowalletsdk",
        "build_command": "make shared-library",
        "go_build": true
      }
    ],
    "cmake": { "extra_include_dirs": ["lib"] }
  }
}
```

`src/wallet_module_impl.cpp`:
```cpp
#include "wallet_module_impl.h"
#include "lib/libgowalletsdk.h"

bool WalletModuleImpl::initWallet(const std::string& rpcUrl) {
    char* err = nullptr;
    m_handle = GoWSK_ethclient_NewClient(rpcUrl.c_str(), &err);
    if (err) {
        GoWSK_FreeCString(err);
        return false;
    }
    return true;
}
```