diff --git a/serde/cbor/README.md b/serde/cbor/README.md index f546c4d..1478de1 100644 --- a/serde/cbor/README.md +++ b/serde/cbor/README.md @@ -1,15 +1,18 @@ # nim-serde CBOR -This README details the usage of CBOR serialization and deserialization features offered by nim-serde, in compliance with [RFC 8949](https://datatracker.ietf.org/doc/html/rfc8949). +The CBOR module in nim-serde provides serialization and deserialization for Nim values following [RFC 8949](https://www.rfc-editor.org/rfc/rfc8949.html). Unlike the JSON implementation, CBOR offers a stream-based API that enables direct binary serialization without intermediate representations. + +> Note: While the JSON implementation supports serde modes via pragmas, the current CBOR implementation does not support the `serialize` and `deserialize` pragmas. The library will raise an assertion error if you try to use these pragmas with CBOR serialization. + ## Table of Contents - [nim-serde CBOR](#nim-serde-cbor) - [Table of Contents](#table-of-contents) - [Serialization API](#serialization-api) - - [Basic Serialization with Stream API](#basic-serialization-with-stream-api) - - [Object Serialization](#object-serialization) - - [Custom Type Serialization](#custom-type-serialization) - - [Converting to CBOR with `toCbor`](#converting-to-cbor-with-tocbor) + - [Stream API: Primitive Type and Sequence Serialization](#stream-api-primitive-type-and-sequence-serialization) + - [Stream API: Object Serialization](#stream-api-object-serialization) + - [Stream API: Custom Type Serialization](#stream-api-custom-type-serialization) + - [Serialization without Stream API: `toCbor`](#serialization-without-stream-api-tocbor) - [Working with CborNode](#working-with-cbornode) - [Convenience Functions for CborNode](#convenience-functions-for-cbornode) - [Deserialization API](#deserialization-api) @@ -24,7 +27,7 @@ This README details the usage of CBOR serialization and deserialization features The nim-serde CBOR serialization API provides several ways to convert Nim values to CBOR. -### Basic Serialization with Stream API +### Stream API: Primitive Type and Sequence Serialization The `writeCbor` function writes Nim values to a stream in CBOR format: @@ -50,7 +53,7 @@ discard stream.writeCbor(@[1, 2, 3]) # Sequence let cborData = stream.data ``` -### Object Serialization +### Stream API: Object Serialization Objects can be serialized to CBOR format using the stream API: @@ -78,7 +81,7 @@ discard stream.writeCbor(person) let cborData = stream.data ``` -### Custom Type Serialization +### Stream API: Custom Type Serialization You can extend nim-serde to support custom types by defining your own `writeCbor` procs: @@ -114,7 +117,7 @@ discard userStream.writeCbor(user) let userCborData = userStream.data ``` -### Converting to CBOR with `toCbor` +### Serialization without Stream API: `toCbor` The `toCbor` function can be used to directly convert a Nim value to CBOR binary data: @@ -363,7 +366,7 @@ For deserialization, the library parses CBOR data into a `CborNode` representati ### Current Limitations -While the JSON implementation supports serde modes via pragmas, the current CBOR implementation does not support the `serialize` and `deserialize` pragmas. The library will raise an assertion error if you try to use these pragmas with CBOR serialization. +This implementation does not support the `serialize` and `deserialize` pragmas. The library will raise an assertion error if you try to use these pragmas with CBOR serialization. ```nim import pkg/serde/cbor diff --git a/serde/cbor/helpers.nim b/serde/cbor/helpers.nim index dc26430..5dc6176 100644 --- a/serde/cbor/helpers.nim +++ b/serde/cbor/helpers.nim @@ -36,10 +36,6 @@ template assertNoPragma*(value, pragma, msg) = when value.hasCustomPragma(pragma): raiseAssert(msg) -macro dot*(obj: object, fld: string): untyped = - ## Turn ``obj.dot("fld")`` into ``obj.fld``. - newDotExpr(obj, newIdentNode(fld.strVal)) - func floatSingle*(half: uint16): float32 = ## Convert a 16-bit float to 32-bits. func ldexp( diff --git a/serde/json/README.md b/serde/json/README.md index 9aa0af9..cdc036c 100644 --- a/serde/json/README.md +++ b/serde/json/README.md @@ -1,17 +1,22 @@ # nim-serde JSON -Explore JSON serialization and deserialization using nim-serde, an improved alternative to `std/json`. +The JSON module in nim-serde provides serialization and deserialization for Nim values, offering an improved alternative to the standard `std/json` library. Unlike the standard library, nim-serde JSON implements a flexible system of serialization/deserialization modes that give developers precise control over how Nim objects are converted to and from JSON. ## Table of Contents - [nim-serde JSON](#nim-serde-json) - [Table of Contents](#table-of-contents) + - [Serde Modes](#serde-modes) + - [Modes Overview](#modes-overview) + - [Default Modes](#default-modes) + - [Field Options](#field-options) - [Serialization API](#serialization-api) - [Basic Serialization with `%` operator](#basic-serialization-with--operator) - [Object Serialization](#object-serialization) - - [Serialization with `%*`](#serialization-with-) + - [Inlining JSON Directly in Code with `%*`](#inlining-json-directly-in-code-with-) - [Converting to JSON String with `toJson`](#converting-to-json-string-with-tojson) - [Serialization Modes](#serialization-modes) - [Field Customization for Serialization](#field-customization-for-serialization) + - [Custom Type Serialization](#custom-type-serialization) - [Deserialization API](#deserialization-api) - [Basic Deserialization with `fromJson`](#basic-deserialization-with-fromjson) - [Error Handling](#error-handling) @@ -19,9 +24,111 @@ Explore JSON serialization and deserialization using nim-serde, an improved alte - [Deserialization Modes](#deserialization-modes) - [Field Customization for Deserialization](#field-customization-for-deserialization) - [Using as a Drop-in Replacement for std/json](#using-as-a-drop-in-replacement-for-stdjson) - - [Custom Type Serialization](#custom-type-serialization) - [Implementation Details](#implementation-details) + +## Serde Modes +This implementation supports three different modes to control de/serialization: + +```nim +OptIn +OptOut +Strict +``` + +Modes can be set in the `{.serialize.}` and/or `{.deserialize.}` pragmas on type +definitions. Each mode has a different meaning depending on if the type is being +serialized or deserialized. Modes can be set by setting `mode` in the `serialize` or +`deserialize` pragma annotation, eg: + +```nim +type MyType {.serialize(mode=Strict).} = object + field1: bool + field2: bool +``` + +### Modes Overview + +| Mode | Serialize | Deserialize | +|:-----|:----------|:------------| +| `OptOut` | All object fields will be serialized, except fields marked with `{.serialize(ignore=true).}`. | All JSON keys will be deserialized, except fields marked with `{.deserialize(ignore=true).}`. No error if extra JSON fields exist. | +| `OptIn` | Only fields marked with `{.serialize.}` will be serialized. Fields marked with `{.serialize(ignore=true).}` will not be serialized. | Only fields marked with `{.deserialize.}` will be deserialized. Fields marked with `{.deserialize(ignore=true).}` will not be deserialized. A `SerdeError` is raised if the field is missing in JSON. | +| `Strict` | All object fields will be serialized, regardless if the field is marked with `{.serialize(ignore=true).}`. | Object fields and JSON fields must match exactly, otherwise a `SerdeError` is raised. | + +### Default Modes + +Types can be serialized and deserialized even without explicit annotations, using default modes. Without any pragmas, types are serialized in OptIn mode and deserialized in OptOut mode. When types have pragmas but no specific mode is set, OptOut mode is used for both serialization and deserialization. + + +| Context | Serialize | Deserialize | +|:--------|:----------|:------------| +| Default (no pragma) | `OptIn` | `OptOut` | +| Default (pragma, but no mode) | `OptOut` | `OptOut` | + +```nim +# Type is not annotated +# A default mode of OptIn (for serialize) and OptOut (for deserialize) is assumed. +type MyObj1 = object + field1: bool + field2: bool + +# Type is annotated, but mode not specified +# A default mode of OptOut is assumed for both serialize and deserialize. +type MyObj2 {.serialize, deserialize.} = object + field1: bool + field2: bool +``` + +### Field Options + +Individual fields can be customized using the `{.serialize.}` and `{.deserialize.}` pragmas with additional options that control how each field is processed during serialization and deserialization + + +| | serialize | deserialize | +|:---------|:-----------------------------------------------------------------------------------------------------------|:-----------------------------------------------------------------------------------------------------------------| +| `key` | aliases the field name in json | deserializes the field if json contains `key` | +| `ignore` |
  • **OptOut:** field not serialized
  • **OptIn:** field not serialized
  • **Strict:** field serialized
    • |
  • **OptOut:** field not deserialized
  • **OptIn:** field not deserialized
  • **Strict:** field deserialized
    • | + + +Example with field options: + +```nim +import pkg/serde/json + +type + Person {.serialize(mode=OptOut), deserialize(mode=OptIn).} = object + id {.serialize(ignore=true), deserialize(key="personid").}: int + name: string + birthYear: int + address: string + phone: string + +let person = Person( + name: "Lloyd Christmas", + birthYear: 1970, + address: "123 Sesame Street, Providence, Rhode Island 12345", + phone: "555-905-justgivemethedamnnumber!⛽️🔥") + +let createRequest = """{ + "name": "Lloyd Christmas", + "birthYear": 1970, + "address": "123 Sesame Street, Providence, Rhode Island 12345", + "phone": "555-905-justgivemethedamnnumber!⛽️🔥" +}""" +assert person.toJson(pretty=true) == createRequest + +let createResponse = """{ + "personid": 1, + "name": "Lloyd Christmas", + "birthYear": 1970, + "address": "123 Sesame Street, Providence, Rhode Island 12345", + "phone": "555-905-justgivemethedamnnumber!⛽️🔥" +}""" +assert !Person.fromJson(createResponse) == Person(id: 1) +``` + +More examples can be found in [Serialization Modes](#serialization-modes) and [Deserialization Modes](#deserialization-modes). + ## Serialization API The nim-serde JSON serialization API provides several ways to convert Nim values to JSON. @@ -67,16 +174,19 @@ assert jsonNode["age"].getInt == 30 assert "address" notin jsonNode ``` -### Serialization with `%*` +### Inlining JSON Directly in Code with `%*` The `%*` macro provides a more convenient way to create JSON objects: ```nim import pkg/serde/json -let jsonObj = %*{ - "name": "John", - "age": 30, +let + name = "John" + age = 30 + jsonObj = %*{ + "name": name, + "age": age, "hobbies": ["reading", "coding"], "address": { "street": "123 Main St", @@ -85,8 +195,8 @@ let jsonObj = %*{ } assert jsonObj.kind == JObject -assert jsonObj["name"].getStr == "John" -assert jsonObj["age"].getInt == 30 +assert jsonObj["name"].getStr == name +assert jsonObj["age"].getInt == age assert jsonObj["hobbies"].kind == JArray assert jsonObj["hobbies"][0].getStr == "reading" assert jsonObj["address"]["street"].getStr == "123 Main St" @@ -171,6 +281,72 @@ let jsonStr = toJson(person) assert jsonStr == """{"first_name":"John","last_name":"Doe","age":30}""" ``` +## Custom Type Serialization + +You can extend nim-serde to support custom types by defining your own `%` operator overloads and `fromJson` procs: + +```nim +import pkg/serde/json +import pkg/serde/utils/errors +import pkg/questionable/results +import std/strutils + +# Define a custom type +type + UserId = distinct int + +# Custom serialization for UserId +proc `%`*(id: UserId): JsonNode = + %("user-" & $int(id)) + +# Custom deserialization for UserId +proc fromJson*(_: type UserId, json: JsonNode): ?!UserId = + if json.kind != JString: + return failure(newSerdeError("Expected string for UserId, got " & $json.kind)) + + let str = json.getStr() + if str.startsWith("user-"): + let idStr = str[5..^1] + try: + let id = parseInt(idStr) + success(UserId(id)) + except ValueError: + failure(newSerdeError("Invalid UserId format: " & str)) + else: + failure(newSerdeError("UserId must start with 'user-' prefix")) + +# Test serialization +let userId = UserId(42) +let jsonNode = %userId +assert jsonNode.kind == JString +assert jsonNode.getStr() == "user-42" + +# Test deserialization +let jsonStr = "\"user-42\"" +let parsedJson = !JsonNode.parse(jsonStr) +let result = UserId.fromJson(parsedJson) +assert result.isSuccess +assert int(!result) == 42 + +# Test in object context +type User {.serialize(mode = OptOut).} = object + id: UserId + name: string + +let user = User(id: UserId(123), name: "John") +let userJson = %user +assert userJson.kind == JObject +assert userJson["id"].getStr() == "user-123" +assert userJson["name"].getStr() == "John" + +# Test deserialization of object with custom type +let userJsonStr = """{"id":"user-123","name":"John"}""" +let userResult = User.fromJson(userJsonStr) +assert userResult.isSuccess +assert int((!userResult).id) == 123 +assert (!userResult).name == "John" +``` + ## Deserialization API nim-serde provides a type-safe way to convert JSON data back into Nim types. @@ -342,72 +518,6 @@ assert parsedNode["name"].getStr == "John" let prettyJson = pretty(jsonNode) ``` -## Custom Type Serialization - -You can extend nim-serde to support custom types by defining your own `%` operator overloads and `fromJson` procs: - -```nim -import pkg/serde/json -import pkg/serde/utils/errors -import pkg/questionable/results -import std/strutils - -# Define a custom type -type - UserId = distinct int - -# Custom serialization for UserId -proc `%`*(id: UserId): JsonNode = - %("user-" & $int(id)) - -# Custom deserialization for UserId -proc fromJson*(_: type UserId, json: JsonNode): ?!UserId = - if json.kind != JString: - return failure(newSerdeError("Expected string for UserId, got " & $json.kind)) - - let str = json.getStr() - if str.startsWith("user-"): - let idStr = str[5..^1] - try: - let id = parseInt(idStr) - success(UserId(id)) - except ValueError: - failure(newSerdeError("Invalid UserId format: " & str)) - else: - failure(newSerdeError("UserId must start with 'user-' prefix")) - -# Test serialization -let userId = UserId(42) -let jsonNode = %userId -assert jsonNode.kind == JString -assert jsonNode.getStr() == "user-42" - -# Test deserialization -let jsonStr = "\"user-42\"" -let parsedJson = !JsonNode.parse(jsonStr) -let result = UserId.fromJson(parsedJson) -assert result.isSuccess -assert int(!result) == 42 - -# Test in object context -type User {.serialize(mode = OptOut).} = object - id: UserId - name: string - -let user = User(id: UserId(123), name: "John") -let userJson = %user -assert userJson.kind == JObject -assert userJson["id"].getStr() == "user-123" -assert userJson["name"].getStr() == "John" - -# Test deserialization of object with custom type -let userJsonStr = """{"id":"user-123","name":"John"}""" -let userResult = User.fromJson(userJsonStr) -assert userResult.isSuccess -assert int((!userResult).id) == 123 -assert (!userResult).name == "John" -``` - ## Implementation Details The JSON serialization in nim-serde is based on the `%` operator pattern: