mirror of
https://github.com/logos-messaging/logos-messaging-nim.git
synced 2026-01-02 05:53:11 +00:00
Add text file for coding LLMs with high-level nwaku info and style guide advice (#3624)
* add CLAUDE.md first version * extract style guide advice * use AGENTS.md instead of CLAUDE.md for neutrality * chore: update AGENTS.md w.r.t. master developments * Apply suggestions from code review Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com> * remove project tree from AGENTS.md; minor editx * Apply suggestions from code review Co-authored-by: NagyZoltanPeter <113987313+NagyZoltanPeter@users.noreply.github.com> --------- Co-authored-by: Ivan FB <128452529+Ivansete-status@users.noreply.github.com> Co-authored-by: NagyZoltanPeter <113987313+NagyZoltanPeter@users.noreply.github.com>
This commit is contained in:
Sergei Tikhomirov
GitHub
parent
7920368a36
commit
12952d070f
509
AGENTS.md
Normal file
509
AGENTS.md
Normal file
@ -0,0 +1,509 @@
|
||||
# AGENTS.md - AI Coding Context
|
||||
|
||||
This file provides essential context for LLMs assisting with Logos Messaging development.
|
||||
|
||||
## Project Identity
|
||||
|
||||
Logos Messaging is designed as a shared public network for generalized messaging, not application-specific infrastructure.
|
||||
|
||||
This project is a Nim implementation of a libp2p protocol suite for private, censorship-resistant P2P messaging. It targets resource-restricted devices and privacy-preserving communication.
|
||||
|
||||
Logos Messaging was formerly known as Waku. Waku-related terminology remains within the codebase for historical reasons.
|
||||
|
||||
### Design Philosophy
|
||||
|
||||
Key architectural decisions:
|
||||
|
||||
Resource-restricted first: Protocols differentiate between full nodes (relay) and light clients (filter, lightpush, store). Light clients can participate without maintaining full message history or relay capabilities. This explains the client/server split in protocol implementations.
|
||||
|
||||
Privacy through unlinkability: RLN (Rate Limiting Nullifier) provides DoS protection while preserving sender anonymity. Messages are routed through pubsub topics with automatic sharding across 8 shards. Code prioritizes metadata privacy alongside content encryption.
|
||||
|
||||
Scalability via sharding: The network uses automatic content-topic-based sharding to distribute traffic. This is why you'll see sharding logic throughout the codebase and why pubsub topic selection is protocol-level, not application-level.
|
||||
|
||||
See [documentation](https://docs.waku.org/learn/) for architectural details.
|
||||
|
||||
### Core Protocols
|
||||
- Relay: Pub/sub message routing using GossipSub
|
||||
- Store: Historical message retrieval and persistence
|
||||
- Filter: Lightweight message filtering for resource-restricted clients
|
||||
- Lightpush: Lightweight message publishing for clients
|
||||
- Peer Exchange: Peer discovery mechanism
|
||||
- RLN Relay: Rate limiting nullifier for spam protection
|
||||
- Metadata: Cluster and shard metadata exchange between peers
|
||||
- Mix: Mixnet protocol for enhanced privacy through onion routing
|
||||
- Rendezvous: Alternative peer discovery mechanism
|
||||
|
||||
### Key Terminology
|
||||
- ENR (Ethereum Node Record): Node identity and capability advertisement
|
||||
- Multiaddr: libp2p addressing format (e.g., `/ip4/127.0.0.1/tcp/60000/p2p/16Uiu2...`)
|
||||
- PubsubTopic: Gossipsub topic for message routing (e.g., `/waku/2/default-waku/proto`)
|
||||
- ContentTopic: Application-level message categorization (e.g., `/my-app/1/chat/proto`)
|
||||
- Sharding: Partitioning network traffic across topics (static or auto-sharding)
|
||||
- RLN (Rate Limiting Nullifier): Zero-knowledge proof system for spam prevention
|
||||
|
||||
### Specifications
|
||||
All specs are at [rfc.vac.dev/waku](https://rfc.vac.dev/waku). RFCs use `WAKU2-XXX` format (not legacy `WAKU-XXX`).
|
||||
|
||||
## Architecture
|
||||
|
||||
### Protocol Module Pattern
|
||||
Each protocol typically follows this structure:
|
||||
```
|
||||
waku_<protocol>/
|
||||
├── protocol.nim # Main protocol type and handler logic
|
||||
├── client.nim # Client-side API
|
||||
├── rpc.nim # RPC message types
|
||||
├── rpc_codec.nim # Protobuf encoding/decoding
|
||||
├── common.nim # Shared types and constants
|
||||
└── protocol_metrics.nim # Prometheus metrics
|
||||
```
|
||||
|
||||
### WakuNode Architecture
|
||||
- WakuNode (`waku/node/waku_node.nim`) is the central orchestrator
|
||||
- Protocols are "mounted" onto the node's switch (libp2p component)
|
||||
- PeerManager handles peer selection and connection management
|
||||
- Switch provides libp2p transport, security, and multiplexing
|
||||
|
||||
Example protocol type definition:
|
||||
```nim
|
||||
type WakuFilter* = ref object of LPProtocol
|
||||
subscriptions*: FilterSubscriptions
|
||||
peerManager: PeerManager
|
||||
messageCache: TimedCache[string]
|
||||
```
|
||||
|
||||
## Development Essentials
|
||||
|
||||
### Build Requirements
|
||||
- Nim 2.x (check `waku.nimble` for minimum version)
|
||||
- Rust toolchain (required for RLN dependencies)
|
||||
- Build system: Make with nimbus-build-system
|
||||
|
||||
### Build System
|
||||
The project uses Makefile with nimbus-build-system (Status's Nim build framework):
|
||||
```bash
|
||||
# Initial build (updates submodules)
|
||||
make wakunode2
|
||||
|
||||
# After git pull, update submodules
|
||||
make update
|
||||
|
||||
# Build with custom flags
|
||||
make wakunode2 NIMFLAGS="-d:chronicles_log_level=DEBUG"
|
||||
```
|
||||
|
||||
Note: The build system uses `--mm:refc` memory management (automatically enforced). Only relevant if compiling outside the standard build system.
|
||||
|
||||
### Common Make Targets
|
||||
```bash
|
||||
make wakunode2 # Build main node binary
|
||||
make test # Run all tests
|
||||
make testcommon # Run common tests only
|
||||
make libwakuStatic # Build static C library
|
||||
make chat2 # Build chat example
|
||||
make install-nph # Install git hook for auto-formatting
|
||||
```
|
||||
|
||||
### Testing
|
||||
```bash
|
||||
# Run all tests
|
||||
make test
|
||||
|
||||
# Run specific test file
|
||||
make test tests/test_waku_enr.nim
|
||||
|
||||
# Run specific test case from file
|
||||
make test tests/test_waku_enr.nim "check capabilities support"
|
||||
|
||||
# Build and run test separately (for development iteration)
|
||||
make test tests/test_waku_enr.nim
|
||||
```
|
||||
|
||||
Test structure uses `testutils/unittests`:
|
||||
```nim
|
||||
import testutils/unittests
|
||||
|
||||
suite "Waku ENR - Capabilities":
|
||||
test "check capabilities support":
|
||||
## Given
|
||||
let bitfield: CapabilitiesBitfield = 0b0000_1101u8
|
||||
|
||||
## Then
|
||||
check:
|
||||
bitfield.supportsCapability(Capabilities.Relay)
|
||||
not bitfield.supportsCapability(Capabilities.Store)
|
||||
```
|
||||
|
||||
### Code Formatting
|
||||
Mandatory: All code must be formatted with `nph` (vendored in `vendor/nph`)
|
||||
```bash
|
||||
# Format specific file
|
||||
make nph/waku/waku_core.nim
|
||||
|
||||
# Install git pre-commit hook (auto-formats on commit)
|
||||
make install-nph
|
||||
```
|
||||
The nph formatter handles all formatting details automatically, especially with the pre-commit hook installed. Focus on semantic correctness.
|
||||
|
||||
### Logging
|
||||
Uses `chronicles` library with compile-time configuration:
|
||||
```nim
|
||||
import chronicles
|
||||
|
||||
logScope:
|
||||
topics = "waku lightpush"
|
||||
|
||||
info "handling request", peerId = peerId, topic = pubsubTopic
|
||||
error "request failed", error = msg
|
||||
```
|
||||
|
||||
Compile with log level:
|
||||
```bash
|
||||
nim c -d:chronicles_log_level=TRACE myfile.nim
|
||||
```
|
||||
|
||||
|
||||
## Code Conventions
|
||||
|
||||
Common pitfalls:
|
||||
- Always handle Result types explicitly
|
||||
- Avoid global mutable state: Pass state through parameters
|
||||
- Keep functions focused: Under 50 lines when possible
|
||||
- Prefer compile-time checks (`static assert`) over runtime checks
|
||||
|
||||
### Naming
|
||||
- Files/Directories: `snake_case` (e.g., `waku_lightpush`, `peer_manager`)
|
||||
- Procedures: `camelCase` (e.g., `handleRequest`, `pushMessage`)
|
||||
- Types: `PascalCase` (e.g., `WakuFilter`, `PubsubTopic`)
|
||||
- Constants: `PascalCase` (e.g., `MaxContentTopicsPerRequest`)
|
||||
- Constructors: `func init(T: type Xxx, params): T`
|
||||
- For ref types: `func new(T: type Xxx, params): ref T`
|
||||
- Exceptions: `XxxError` for CatchableError, `XxxDefect` for Defect
|
||||
- ref object types: `XxxRef` suffix
|
||||
|
||||
### Imports Organization
|
||||
Group imports: stdlib, external libs, internal modules:
|
||||
```nim
|
||||
import
|
||||
std/[options, sequtils], # stdlib
|
||||
results, chronicles, chronos, # external
|
||||
libp2p/peerid
|
||||
import
|
||||
../node/peer_manager, # internal (separate import block)
|
||||
../waku_core,
|
||||
./common
|
||||
```
|
||||
|
||||
### Async Programming
|
||||
Uses chronos, not stdlib `asyncdispatch`:
|
||||
```nim
|
||||
proc handleRequest(
|
||||
wl: WakuLightPush, peerId: PeerId
|
||||
): Future[WakuLightPushResult] {.async.} =
|
||||
let res = await wl.pushHandler(peerId, pubsubTopic, message)
|
||||
return res
|
||||
```
|
||||
|
||||
### Error Handling
|
||||
The project uses both Result types and exceptions:
|
||||
|
||||
Result types from nim-results are used for protocol and API-level errors:
|
||||
```nim
|
||||
proc subscribe(
|
||||
wf: WakuFilter, peerId: PeerID
|
||||
): Future[FilterSubscribeResult] {.async.} =
|
||||
if contentTopics.len > MaxContentTopicsPerRequest:
|
||||
return err(FilterSubscribeError.badRequest("exceeds maximum"))
|
||||
|
||||
# Handle Result with isOkOr
|
||||
(await wf.subscriptions.addSubscription(peerId, criteria)).isOkOr:
|
||||
return err(FilterSubscribeError.serviceUnavailable(error))
|
||||
|
||||
ok()
|
||||
```
|
||||
|
||||
Exceptions still used for:
|
||||
- chronos async failures (CancelledError, etc.)
|
||||
- Database/system errors
|
||||
- Library interop
|
||||
|
||||
Most files start with `{.push raises: [].}` to disable exception tracking, then use try/catch blocks where needed.
|
||||
|
||||
### Pragma Usage
|
||||
```nim
|
||||
{.push raises: [].} # Disable default exception tracking (at file top)
|
||||
|
||||
proc myProc(): Result[T, E] {.async.} = # Async proc
|
||||
```
|
||||
|
||||
### Protocol Inheritance
|
||||
Protocols inherit from libp2p's `LPProtocol`:
|
||||
```nim
|
||||
type WakuLightPush* = ref object of LPProtocol
|
||||
rng*: ref rand.HmacDrbgContext
|
||||
peerManager*: PeerManager
|
||||
pushHandler*: PushMessageHandler
|
||||
```
|
||||
|
||||
### Type Visibility
|
||||
- Public exports use `*` suffix: `type WakuFilter* = ...`
|
||||
- Fields without `*` are module-private
|
||||
|
||||
## Style Guide Essentials
|
||||
|
||||
This section summarizes key Nim style guidelines relevant to this project. Full guide: https://status-im.github.io/nim-style-guide/
|
||||
|
||||
### Language Features
|
||||
|
||||
Import and Export
|
||||
- Use explicit import paths with std/ prefix for stdlib
|
||||
- Group imports: stdlib, external, internal (separate blocks)
|
||||
- Export modules whose types appear in public API
|
||||
- Avoid include
|
||||
|
||||
Macros and Templates
|
||||
- Avoid macros and templates - prefer simple constructs
|
||||
- Avoid generating public API with macros
|
||||
- Put logic in templates, use macros only for glue code
|
||||
|
||||
Object Construction
|
||||
- Prefer Type(field: value) syntax
|
||||
- Use Type.init(params) convention for constructors
|
||||
- Default zero-initialization should be valid state
|
||||
- Avoid using result variable for construction
|
||||
|
||||
ref object Types
|
||||
- Avoid ref object unless needed for:
|
||||
- Resource handles requiring reference semantics
|
||||
- Shared ownership
|
||||
- Reference-based data structures (trees, lists)
|
||||
- Stable pointer for FFI
|
||||
- Use explicit ref MyType where possible
|
||||
- Name ref object types with Ref suffix: XxxRef
|
||||
|
||||
Memory Management
|
||||
- Prefer stack-based and statically sized types in core code
|
||||
- Use heap allocation in glue layers
|
||||
- Avoid alloca
|
||||
- For FFI: use create/dealloc or createShared/deallocShared
|
||||
|
||||
Variable Usage
|
||||
- Use most restrictive of const, let, var (prefer const over let over var)
|
||||
- Prefer expressions for initialization over var then assignment
|
||||
- Avoid result variable - use explicit return or expression-based returns
|
||||
|
||||
Functions
|
||||
- Prefer func over proc
|
||||
- Avoid public (*) symbols not part of intended API
|
||||
- Prefer openArray over seq for function parameters
|
||||
|
||||
Methods (runtime polymorphism)
|
||||
- Avoid method keyword for dynamic dispatch
|
||||
- Prefer manual vtable with proc closures for polymorphism
|
||||
- Methods lack support for generics
|
||||
|
||||
Miscellaneous
|
||||
- Annotate callback proc types with {.raises: [], gcsafe.}
|
||||
- Avoid explicit {.inline.} pragma
|
||||
- Avoid converters
|
||||
- Avoid finalizers
|
||||
|
||||
Type Guidelines
|
||||
|
||||
Binary Data
|
||||
- Use byte for binary data
|
||||
- Use seq[byte] for dynamic arrays
|
||||
- Convert string to seq[byte] early if stdlib returns binary as string
|
||||
|
||||
Integers
|
||||
- Prefer signed (int, int64) for counting, lengths, indexing
|
||||
- Use unsigned with explicit size (uint8, uint64) for binary data, bit ops
|
||||
- Avoid Natural
|
||||
- Check ranges before converting to int
|
||||
- Avoid casting pointers to int
|
||||
- Avoid range types
|
||||
|
||||
Strings
|
||||
- Use string for text
|
||||
- Use seq[byte] for binary data instead of string
|
||||
|
||||
### Error Handling
|
||||
|
||||
Philosophy
|
||||
- Prefer Result, Opt for explicit error handling
|
||||
- Use Exceptions only for legacy code compatibility
|
||||
|
||||
Result Types
|
||||
- Use Result[T, E] for operations that can fail
|
||||
- Use cstring for simple error messages: Result[T, cstring]
|
||||
- Use enum for errors needing differentiation: Result[T, SomeErrorEnum]
|
||||
- Use Opt[T] for simple optional values
|
||||
- Annotate all modules: {.push raises: [].} at top
|
||||
|
||||
Exceptions (when unavoidable)
|
||||
- Inherit from CatchableError, name XxxError
|
||||
- Use Defect for panics/logic errors, name XxxDefect
|
||||
- Annotate functions explicitly: {.raises: [SpecificError].}
|
||||
- Catch specific error types, avoid catching CatchableError
|
||||
- Use expression-based try blocks
|
||||
- Isolate legacy exception code with try/except, convert to Result
|
||||
|
||||
Common Defect Sources
|
||||
- Overflow in signed arithmetic
|
||||
- Array/seq indexing with []
|
||||
- Implicit range type conversions
|
||||
|
||||
Status Codes
|
||||
- Avoid status code pattern
|
||||
- Use Result instead
|
||||
|
||||
### Library Usage
|
||||
|
||||
Standard Library
|
||||
- Use judiciously, prefer focused packages
|
||||
- Prefer these replacements:
|
||||
- async: chronos
|
||||
- bitops: stew/bitops2
|
||||
- endians: stew/endians2
|
||||
- exceptions: results
|
||||
- io: stew/io2
|
||||
|
||||
Results Library
|
||||
- Use cstring errors for diagnostics without differentiation
|
||||
- Use enum errors when caller needs to act on specific errors
|
||||
- Use complex types when additional error context needed
|
||||
- Use isOkOr pattern for chaining
|
||||
|
||||
Wrappers (C/FFI)
|
||||
- Prefer native Nim when available
|
||||
- For C libraries: use {.compile.} to build from source
|
||||
- Create xxx_abi.nim for raw ABI wrapper
|
||||
- Avoid C++ libraries
|
||||
|
||||
Miscellaneous
|
||||
- Print hex output in lowercase, accept both cases
|
||||
|
||||
### Common Pitfalls
|
||||
|
||||
- Defects lack tracking by {.raises.}
|
||||
- nil ref causes runtime crashes
|
||||
- result variable disables branch checking
|
||||
- Exception hierarchy unclear between Nim versions
|
||||
- Range types have compiler bugs
|
||||
- Finalizers infect all instances of type
|
||||
|
||||
## Common Workflows
|
||||
|
||||
### Adding a New Protocol
|
||||
1. Create directory: `waku/waku_myprotocol/`
|
||||
2. Define core files:
|
||||
- `rpc.nim` - Message types
|
||||
- `rpc_codec.nim` - Protobuf encoding
|
||||
- `protocol.nim` - Protocol handler
|
||||
- `client.nim` - Client API
|
||||
- `common.nim` - Shared types
|
||||
3. Define protocol type in `protocol.nim`:
|
||||
```nim
|
||||
type WakuMyProtocol* = ref object of LPProtocol
|
||||
peerManager: PeerManager
|
||||
# ... fields
|
||||
```
|
||||
4. Implement request handler
|
||||
5. Mount in WakuNode (`waku/node/waku_node.nim`)
|
||||
6. Add tests in `tests/waku_myprotocol/`
|
||||
7. Export module via `waku/waku_myprotocol.nim`
|
||||
|
||||
### Adding a REST API Endpoint
|
||||
1. Define handler in `waku/rest_api/endpoint/myprotocol/`
|
||||
2. Implement endpoint following pattern:
|
||||
```nim
|
||||
proc installMyProtocolApiHandlers*(
|
||||
router: var RestRouter, node: WakuNode
|
||||
) =
|
||||
router.api(MethodGet, "/waku/v2/myprotocol/endpoint") do () -> RestApiResponse:
|
||||
# Implementation
|
||||
return RestApiResponse.jsonResponse(data, status = Http200)
|
||||
```
|
||||
3. Register in `waku/rest_api/handlers.nim`
|
||||
|
||||
### Adding Database Migration
|
||||
For message_store (SQLite):
|
||||
1. Create `migrations/message_store/NNNNN_description.up.sql`
|
||||
2. Create corresponding `.down.sql` for rollback
|
||||
3. Increment version number sequentially
|
||||
4. Test migration locally before committing
|
||||
|
||||
For PostgreSQL: add in `migrations/message_store_postgres/`
|
||||
|
||||
### Running Single Test During Development
|
||||
```bash
|
||||
# Build test binary
|
||||
make test tests/waku_filter_v2/test_waku_client.nim
|
||||
|
||||
# Binary location
|
||||
./build/tests/waku_filter_v2/test_waku_client.nim.bin
|
||||
|
||||
# Or combine
|
||||
make test tests/waku_filter_v2/test_waku_client.nim "specific test name"
|
||||
```
|
||||
|
||||
### Debugging with Chronicles
|
||||
Set log level and filter topics:
|
||||
```bash
|
||||
nim c -r \
|
||||
-d:chronicles_log_level=TRACE \
|
||||
-d:chronicles_disabled_topics="eth,dnsdisc" \
|
||||
tests/mytest.nim
|
||||
```
|
||||
|
||||
## Key Constraints
|
||||
|
||||
### Vendor Directory
|
||||
- Never edit files directly in vendor - it is auto-generated from git submodules
|
||||
- Always run `make update` after pulling changes
|
||||
- Managed by `nimbus-build-system`
|
||||
|
||||
### Chronicles Performance
|
||||
- Log levels are configured at compile time for performance
|
||||
- Runtime filtering is available but should be used sparingly: `-d:chronicles_runtime_filtering=on`
|
||||
- Default sinks are optimized for production
|
||||
|
||||
### Memory Management
|
||||
- Uses `refc` (reference counting with cycle collection)
|
||||
- Automatically enforced by the build system (hardcoded in `waku.nimble`)
|
||||
- Do not override unless absolutely necessary, as it breaks compatibility
|
||||
|
||||
### RLN Dependencies
|
||||
- RLN code requires a Rust toolchain, which explains Rust imports in some modules
|
||||
- Pre-built `librln` libraries are checked into the repository
|
||||
|
||||
## Quick Reference
|
||||
|
||||
Language: Nim 2.x | License: MIT or Apache 2.0
|
||||
|
||||
### Important Files
|
||||
- `Makefile` - Primary build interface
|
||||
- `waku.nimble` - Package definition and build tasks (called via nimbus-build-system)
|
||||
- `vendor/nimbus-build-system/` - Status's build framework
|
||||
- `waku/node/waku_node.nim` - Core node implementation
|
||||
- `apps/wakunode2/wakunode2.nim` - Main CLI application
|
||||
- `waku/factory/waku_conf.nim` - Configuration types
|
||||
- `library/libwaku.nim` - C bindings entry point
|
||||
|
||||
### Testing Entry Points
|
||||
- `tests/all_tests_waku.nim` - All Waku protocol tests
|
||||
- `tests/all_tests_wakunode2.nim` - Node application tests
|
||||
- `tests/all_tests_common.nim` - Common utilities tests
|
||||
|
||||
### Key Dependencies
|
||||
- `chronos` - Async framework
|
||||
- `nim-results` - Result type for error handling
|
||||
- `chronicles` - Logging
|
||||
- `libp2p` - P2P networking
|
||||
- `confutils` - CLI argument parsing
|
||||
- `presto` - REST server
|
||||
- `nimcrypto` - Cryptographic primitives
|
||||
|
||||
Note: For specific version requirements, check `waku.nimble`.
|
||||
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user