Add features documentation
This commit is contained in:
jangko
parent
1996de769b
commit
9720b33f99
230
README.md
230
README.md
|
|
@ -5,7 +5,234 @@
|
|||
|
||||
|
||||
|
||||
Flexible JSON serialization not relying on run-time type information.
|
||||
Flexible JSON serialization does not rely on run-time type information.
|
||||
|
||||
## Overview
|
||||
nim-json-serialization offers rich features on top of [nim-serialization](https://github.com/status-im/nim-serialization)
|
||||
framework. The following is available but not an exhaustive list of features:
|
||||
|
||||
- Decode into Nim data types efficiently without an intermediate token.
|
||||
- Able to parse full spec of JSON including the notorious JSON number.
|
||||
- Support stdlib/JsonNode out of the box.
|
||||
- While stdlib/JsonNode does not support the full spec of the Json number, we offer an alternative `JsonValueRef`.
|
||||
- Skipping Json value is an efficient process, no token is generated at all and at the same time, the grammar is checked.
|
||||
- Skipping is also free from custom serializer interference.
|
||||
- An entire Json value can be parsed into a valid Json document string. This string document can be parsed again without losing any information.
|
||||
- Custom serialization is easy and safe to implement with the help of many built-in parsers.
|
||||
- Nonstandard features are put behind flags. You can choose which features to switch on or off.
|
||||
- Because the intended usage of this library will be in a security-demanding application, we make sure malicious inputs will not crash
|
||||
this library through fuzz tests.
|
||||
- The user also can tweak certain limits of the lexer/parser behavior using the configuration object.
|
||||
- `createJsonFlavor` is a powerful way to prevent cross contamination between different subsystem using different custom serializar on the same type.
|
||||
|
||||
## Spec compliance
|
||||
nim-json-serialization implements [RFC8259](https://datatracker.ietf.org/doc/html/rfc8259)
|
||||
JSON spec and pass these test suites:
|
||||
|
||||
- [JSONTestSuite](https://github.com/nst/JSONTestSuite)
|
||||
|
||||
## Switchable features
|
||||
Many of these switchable features are widely used features in various projects but are not standard JSON features.
|
||||
But you can access them using the flags:
|
||||
|
||||
- **allowUnknownFields[=off]**: enable unknown fields to be skipped instead of throwing an error.
|
||||
- **requireAllFields[=off]**: if one of the required fields is missing, the serializer will throw an error.
|
||||
- **escapeHex[=off]**: JSON doesn't support `\xHH` escape sequence, but it is a common thing in many languages.
|
||||
- **relaxedEscape[=off]**: only '0x00'..'0x1F' can be prepended by escape char `\\`, turn this on and you can escape any char.
|
||||
- **portableInt[=off]**: set the limit of integer to `-2**53 + 1` and `+2**53 - 1`.
|
||||
- **trailingComma[=on]**: allow the presence of a trailing comma after the last object member or array element.
|
||||
- **allowComments[=on]**: JSOn standard doesn't mention about comments. Turn this on to parse both C style comments of `//..EOL` and `/* .. */`.
|
||||
- **leadingFraction[=on]**: something like `.123` is not a valid JSON number, but its widespread usage sometimes creeps into Json documents.
|
||||
- **integerPositiveSign[=on]**: `+123` is also not a valid JSON number, but since `-123` is a valid JSON number, why not parse it safely?
|
||||
|
||||
## Safety features
|
||||
You can modify these default configurations to suit your needs.
|
||||
|
||||
- **nestedDepthLimit: 512**: maximum depth of the nested structure, they are a combination of objects and arrays depth(0=disable).
|
||||
- **arrayElementsLimit: 0**: maximum number of allowed array elements(0=disable).
|
||||
- **objectMembersLimit: 0**: maximum number of key-value pairs in an object(0=disable).
|
||||
- **integerDigitsLimit: 128**: limit the maximum digits of the integer part of JSON number.
|
||||
- **fractionDigitsLimit: 128**: limit the maximum digits of faction part of JSON number.
|
||||
- **exponentDigitsLimit: 32**: limit the maximum digits of the exponent part of JSON number.
|
||||
- **stringLengthLimit: 0**: limit the maximum bytes of string(0=disable).
|
||||
|
||||
## Special types
|
||||
|
||||
- **JsonString**: Use this type if you want to parse a Json value to a valid Json document contained in a string.
|
||||
- **JsonVoid**: Use this type to skip a valid Json value.
|
||||
- **JsonNumber**: Use this to parse a valid Json number including the fraction and exponent part.
|
||||
- Please note that this type is a generic, it support `uint64` and `string` as generic param.
|
||||
- The generic param will define the integer and exponent part as `uint64` or `string`.
|
||||
- If the generic param is `uint64`, overflow can happen, or max digit limit will apply.
|
||||
- If the generic param is `string`, the max digit limit will apply.
|
||||
- The fraction part is always a string to keep the leading zero of the fractional number.
|
||||
- **JsonValueRef**: Use this type to parse any valid Json value into something like stdlib/JsonNode.
|
||||
- `JsonValueRef` is using `JsonNumber` instead of `int` or `float` like stdlib/JsonNode.
|
||||
|
||||
## Flavor
|
||||
|
||||
While flags and limits are runtime configuration, flavor is a powerful compile time mechanism to prevent
|
||||
cross contamination between different custom serializer operated the same type. For example,
|
||||
`json-rpc` subsystem dan `json-rest` subsystem maybe have different custom serializer for the same `UInt256`.
|
||||
|
||||
Json-Flavor will make sure, the compiler picks the right serializer for the right subsystem.
|
||||
You can use `useDefaultSerializationIn` to add serializers of a flavor to a specific type.
|
||||
|
||||
```Nim
|
||||
# These are the parameters you can pass to `createJsonFlavor` to create a new flavor.
|
||||
|
||||
FlavorName: untyped
|
||||
mimeTypeValue = "application/json"
|
||||
automaticObjectSerialization = false
|
||||
requireAllFields = true
|
||||
omitOptionalFields = true
|
||||
allowUnknownFields = true
|
||||
```
|
||||
|
||||
```Nim
|
||||
type
|
||||
OptionalFields = object
|
||||
one: Opt[string]
|
||||
two: Option[int]
|
||||
|
||||
createJsonFlavor OptJson
|
||||
OptionalFields.useDefaultSerializationIn OptJson
|
||||
```
|
||||
|
||||
## Decoder example
|
||||
```nim
|
||||
type
|
||||
NimServer = object
|
||||
name: string
|
||||
port: int
|
||||
|
||||
MixedServer = object
|
||||
name: JsonValueRef
|
||||
port: int
|
||||
|
||||
StringServer = object
|
||||
name: JsonString
|
||||
port: JsonString
|
||||
|
||||
# decode into native Nim
|
||||
var nim_native = Json.decode(rawJson, NimServer)
|
||||
|
||||
# decode into mixed Nim + JsonValueRef
|
||||
var nim_mixed = Json.decode(rawJson, MixedServer)
|
||||
|
||||
# decode any value into string
|
||||
var nim_string = Json.decode(rawJson, StringServer)
|
||||
|
||||
# decode any valid JSON
|
||||
var json_value = Json.decode(rawJson, JsonValueRef)
|
||||
```
|
||||
|
||||
## Load and save
|
||||
```Nim
|
||||
var server = Json.loadFile("filename.json", Server)
|
||||
var server_string = Json.loadFile("filename.json", JsonString)
|
||||
|
||||
Json.saveFile("filename.json", server)
|
||||
```
|
||||
|
||||
## Objects
|
||||
Decoding an object can be achieved via the `parseObject` template.
|
||||
To parse the value, you can use one of the helper functions or use `readValue`.
|
||||
`readObject` and `readObjectFields` iterators are also handy when creating a custom object parser.
|
||||
|
||||
```Nim
|
||||
proc readValue*(r: var JsonReader, table: var Table[string, int]) =
|
||||
parseObject(r, key):
|
||||
table[key] = r.parseInt(int)
|
||||
```
|
||||
|
||||
## Sets and list-like
|
||||
Similar to `Object`, sets and list or array-like data structures can be parsed using
|
||||
`parseArray` template. It comes in two variations, indexed and non-indexed.
|
||||
|
||||
Built-in `readValue` for regular `seq` and `array` is implemented for you.
|
||||
No built-in `readValue` for `set` or `set-like` is provided, you must overload it yourself depending on your need.
|
||||
|
||||
```nim
|
||||
type
|
||||
HoldArray = object
|
||||
data: array[3, int]
|
||||
|
||||
HoldSeq = object
|
||||
data: seq[int]
|
||||
|
||||
WelderFlag = enum
|
||||
TIG
|
||||
MIG
|
||||
MMA
|
||||
|
||||
Welder = object
|
||||
flags: set[WelderFlag]
|
||||
|
||||
proc readValue*(r: var JsonReader, value: var HoldArray) =
|
||||
# parseArray with index, `i` can be any valid identifier
|
||||
r.parseArray(i):
|
||||
value.data[i] = r.parseInt(int)
|
||||
|
||||
proc readValue*(r: var JsonReader, value: var HoldSeq) =
|
||||
# parseArray without index
|
||||
r.parseArray:
|
||||
let lastPos = value.data.len
|
||||
value.data.setLen(lastPos + 1)
|
||||
readValue(r, value.data[lastPos])
|
||||
|
||||
proc readValue*(r: var JsonReader, value: var Welder) =
|
||||
# populating set also okay
|
||||
r.parseArray:
|
||||
value.flags.incl r.parseInt(int).WelderFlag
|
||||
```
|
||||
|
||||
## Custom iterators
|
||||
Using these custom iterators, you can have access to sub-token elements.
|
||||
|
||||
```Nim
|
||||
customIntValueIt(r: var JsonReader; body: untyped)
|
||||
customNumberValueIt(r: var JsonReader; body: untyped)
|
||||
customStringValueIt(r: var JsonReader; limit: untyped; body: untyped)
|
||||
customStringValueIt(r: var JsonReader; body: untyped)
|
||||
```
|
||||
## Convenience iterators
|
||||
|
||||
```Nim
|
||||
readArray(r: var JsonReader, ElemType: typedesc): ElemType
|
||||
readObjectFields(r: var JsonReader, KeyType: type): KeyType
|
||||
readObjectFields(r: var JsonReader): string
|
||||
readObject(r: var JsonReader, KeyType: type, ValueType: type): (KeyType, ValueType)
|
||||
```
|
||||
|
||||
## Helper procs
|
||||
When crafting a custom serializer, use these parsers, they are safe and intuitive.
|
||||
Avoid using the lexer directly.
|
||||
|
||||
```Nim
|
||||
tokKind(r: var JsonReader): JsonValueKind
|
||||
parseString(r: var JsonReader, limit: int): string
|
||||
parseString(r: var JsonReader): string
|
||||
parseBool(r: var JsonReader): bool
|
||||
parseNull(r: var JsonReader)
|
||||
parseNumber(r: var JsonReader, T: type): JsonNumber[T: string or uint64]
|
||||
parseNumber(r: var JsonReader, val: var JsonNumber)
|
||||
toInt(r: var JsonReader, val: JsonNumber, T: type SomeInteger, portable: bool): T
|
||||
parseInt(r: var JsonReader, T: type SomeInteger, portable: bool = false): T
|
||||
toFloat(r: var JsonReader, val: JsonNumber, T: type SomeFloat): T
|
||||
parseFloat(r: var JsonReader, T: type SomeFloat): T
|
||||
parseAsString(r: var JsonReader, val: var string)
|
||||
parseAsString(r: var JsonReader): JsonString
|
||||
parseValue(r: var JsonReader, T: type): JsonValueRef[T: string or uint64]
|
||||
parseValue(r: var JsonReader, val: var JsonValueRef)
|
||||
parseArray(r: var JsonReader; body: untyped)
|
||||
parseArray(r: var JsonReader; idx: untyped; body: untyped)
|
||||
parseObject(r: var JsonReader, key: untyped, body: untyped)
|
||||
parseObjectCustomKey(r: var JsonReader, keyAction: untyped, body: untyped)
|
||||
parseJsonNode(r: var JsonReader): JsonNode
|
||||
skipSingleJsValue(r: var JsonReader)
|
||||
readRecordValue[T](r: var JsonReader, value: var T)
|
||||
```
|
||||
|
||||
## License
|
||||
|
||||
|
|
@ -18,4 +245,3 @@ or
|
|||
* Apache License, Version 2.0, ([LICENSE-APACHEv2](LICENSE-APACHEv2) or http://www.apache.org/licenses/LICENSE-2.0)
|
||||
|
||||
at your option. These files may not be copied, modified, or distributed except according to those terms.
|
||||
|
||||
|
|
|
|||
Loading…
Reference in New Issue