Better doc index. YamlStream now an object.

* Also various fixes to serialization and presentation
This commit is contained in:
Felix Krause 2016-02-12 19:53:25 +01:00
parent cb18c5cb9c
commit c6c13eb044
11 changed files with 452 additions and 299 deletions

View File

@ -2,18 +2,146 @@
NimYAML NimYAML
======= =======
Overview Introduction
======== ============
**NimYAML** is a pure YAML implementation for Nim. It is able to read from and **NimYAML** is a pure YAML implementation for Nim. It is able to read from and
write to YAML character streams, and to serialize from and construct to native write to YAML character streams, and to serialize from and construct to native
Nim types. It exclusively supports Nim types. It exclusively supports
`YAML 1.2 <#http://www.yaml.org/spec/1.2/spec.html>`_. NimYAML exposes all `YAML 1.2 <#http://www.yaml.org/spec/1.2/spec.html>`_.
processing steps defined by the YAML specification to the user, so it is
possible to customize the processing pipeline. The following diagram gives an Quickstart
overview of NimYAML's features based on the YAML processing pipeline. The items ----------
and terminology YAML defines is shown in black, NimYAML's implementation is
shown in red. Data used in quickstart code may not be accurate and is solely used to showcase
NimYAML's features.
.. code-block:: nim
import yaml
type
GenderKind* = enum
male, female, other
Person* = object
name*: string
gender*: GenderKind
age*: int32
spouse*: ref Person
offspring*: seq[ref Person]
let input = newStringStream("""
%YAML 1.2
---
- &a
name: Luke Skywalker
gender: male
age: 19
spouse: ~
offspring: []
- &b
name: Han Solo
gender: male
age: 35
spouse: &c
name: Leia Organa
gender: female
age: 19
spouse: *b
offspring: []
offspring: []
- *c
-
name: Anakin Skywalker
gender: male
age: 42
spouse: ~
offspring: [*a, *c]
""")
var persons: seq[ref Person]
load(input, persons)
for person in persons:
echo person.name, "\nage ", person.age
if person.spouse != nil:
echo "spouse: ", person.spouse.name
for child in person.offspring:
case child.gender
of male: echo "son: ", child.name
of female: echo "daughter: ", child.name
of other: echo "child: ", child.name
echo "------------------------"
dump(persons, newFileStream(stdout))
API Overview
============
NimYAML advocates parsing YAML input into native Nim types. Basic library types
like integers, floats and strings, as well as all tuples, enums and objects
without private fields are supported out-of-the-box. Reference types are also
supported, and NimYAML is able to detect if a reference occurs more than once
and will serialize it accordingly. This means that NimYAML is able to dump and
load potentially cyclic objects.
While loading into and dumping from native Nim types is the preferred way to use
NimYAML, it also gives you complete control over each processing step, so that
you can for example only use the parser and process its event stream yourself.
The following diagram gives an overview of NimYAML's features based on the YAML
processing pipeline. The items and terminology YAML defines is shown in
*italic*, NimYAML's implementation name is shown in **bold**.
.. image:: processing.svg .. image:: processing.svg
:align: center
Intermediate Representation
---------------------------
The base of all YAML processing with NimYAML is the
`YamlStream <yaml.html#YamlStream>`_. This is basically an iterator over
`YamlStreamEvent <yaml.html#YamlStreamEvent>`_ objects. Every proc that
represents a single stage of the loading or dumping process will either take a
``YamlStream`` as input or return a ``YamlStream``. Procs that implement the
whole process in one step hide the ``YamlStream`` from the user. Every proc that
returns a ``YamlStream`` guarantees that this stream is well-formed according to
the YAML specification.
This stream-oriented API can efficiently be used to parse large amounts of data.
The drawback is that errors in the input are only discovered while processing
the ``YamlStream``. If the ``YamlStream`` encounters an exception while
producing the next event, it will throw a ``YamlStreamError`` which contains the
original exception as ``parent``. The caller should know which exceptions are
possible as parents of ``YamlStream`` because they know the source of the
``YamlStream`` they provided.
Loading YAML
------------
For parsing, a `YamlParser <yaml.html#YamlParser>`_ object is needed. This
object stores some state while parsing that may be useful for error reporting to
the user. The
`parse <yaml.html#present,YamlStream,Stream,TagLibrary,PresentationStyle,int>`_
proc implements the YAML processing step of the same name. All syntax errors in
the input character stream are processed by ``parse``, which will raise a
``YamlParserError`` if it encounters a syntax error.
Transforming a ``YamlStream`` to a native YAML object is done via
``construct``. It skips the ``compose`` step for efficiency reasons. As Nim is
statically typed, you have to know the target type when you write your loading
code. This is different from YAML APIs of dynamically typed languages. If you
cannot know the type of your YAML input at compile time, you have to manually
process the ``YamlStream`` to serve your needs.
If you want to load YAML character data directly into a native Nim variable, you
can use `load <yaml.html#load,Stream,K>`_.
Dumping YAML
------------
Dumping YAML is straightforward: You transform a variable into a ``YamlStream``
with `represent <yaml.html#represent,T,TagStyle,AnchorStyle>`_ and then write
that to a stream using
`present <yaml.html#present,YamlStream,Stream,TagLibrary,PresentationStyle,int>`_.
If you want to execute both steps at once, you can use
`dump <yaml.html#dump,K,Stream,PresentationStyle,TagStyle,AnchorStyle,int>`_.
The ``present`` step allows you to specify how you want the output YAML
character stream to be formatted. Amongst other options, it is possible to
output pure JSON, but only if the stream does not contain any constructs that
cannot be presented in JSON.

View File

@ -91,3 +91,9 @@ dt:before {
display: inline-block; display: inline-block;
margin-right: 20px; margin-right: 20px;
} }
object {
margin-left: auto;
margin-right: auto;
display: block;
}

View File

@ -986,7 +986,7 @@ template blockScalar(lexer: BaseLexer, content: var string,
discard discard
proc parse*(p: YamlParser, s: Stream): YamlStream = proc parse*(p: YamlParser, s: Stream): YamlStream =
result = iterator(): YamlStreamEvent = var backend = iterator(): YamlStreamEvent =
var var
state = fpInitial state = fpInitial
shorthands: Table[string, string] shorthands: Table[string, string]
@ -1600,3 +1600,6 @@ proc parse*(p: YamlParser, s: Stream): YamlStream =
else: else:
startToken() startToken()
parserError("Unexpected content (expected flow indicator)") parserError("Unexpected content (expected flow indicator)")
try:
result = initYamlStream(backend)
except Exception: assert(false) # compiler error

View File

@ -78,26 +78,13 @@ proc jsonFromScalar(content: string, tag: TagId): JsonNode
e.parent = getCurrentException() e.parent = getCurrentException()
raise e raise e
proc constructJson*(s: YamlStream): seq[JsonNode] = proc constructJson*(s: var YamlStream): seq[JsonNode] =
newSeq(result, 0) newSeq(result, 0)
var var
levels = newSeq[Level]() levels = newSeq[Level]()
anchors = initTable[AnchorId, JsonNode]() anchors = initTable[AnchorId, JsonNode]()
safeIter = iterator(): YamlStreamEvent for event in s:
{.raises: [YamlConstructionStreamError].} =
while true:
var item: YamlStreamEvent
try:
item = s()
if finished(s): break
except AssertionError: raise
except Exception:
var e = newException(YamlConstructionStreamError, "")
e.parent = getCurrentException()
raise e
yield item
for event in safeIter():
case event.kind case event.kind
of yamlStartDocument: of yamlStartDocument:
# we don't need to do anything here; root node will be created # we don't need to do anything here; root node will be created
@ -205,7 +192,7 @@ proc loadToJson*(s: Stream): seq[JsonNode] =
e.column = parser.getColNumber() e.column = parser.getColNumber()
e.lineContent = parser.getLineContent() e.lineContent = parser.getLineContent()
raise e raise e
except YamlConstructionStreamError: except YamlStreamError:
let e = getCurrentException() let e = getCurrentException()
if e.parent of IOError: if e.parent of IOError:
raise cast[ref IOError](e.parent) raise cast[ref IOError](e.parent)

View File

@ -160,28 +160,16 @@ proc writeTagAndAnchor(target: Stream, tag: TagId, tagLib: TagLibrary,
e.parent = getCurrentException() e.parent = getCurrentException()
raise e raise e
proc present*(s: YamlStream, target: Stream, tagLib: TagLibrary, proc present*(s: var YamlStream, target: Stream, tagLib: TagLibrary,
style: PresentationStyle = psDefault, style: PresentationStyle = psDefault,
indentationStep: int = 2) = indentationStep: int = 2) =
var var
cached = initQueue[YamlStreamEvent]()
cacheIterator = iterator(): YamlStreamEvent =
while true:
while cached.len > 0:
yield cached.dequeue()
try:
let item = s()
if finished(s):
break
cached.enqueue(item)
except Exception:
var e = newException(YamlPresenterStreamError, "")
e.parent = getCurrentException()
raise e
indentation = 0 indentation = 0
levels = newSeq[DumperState]() levels = newSeq[DumperState]()
cached = initQueue[YamlStreamEvent]()
for item in cacheIterator(): while cached.len > 0 or not s.finished():
let item = if cached.len > 0: cached.dequeue else: s.next()
case item.kind case item.kind
of yamlStartDocument: of yamlStartDocument:
if style != psJson: if style != psJson:
@ -253,9 +241,8 @@ proc present*(s: YamlStream, target: Stream, tagLib: TagLibrary,
of psDefault: of psDefault:
var length = 0 var length = 0
while true: while true:
try: assert(not(s.finished()))
let next = s() let next = s.next()
assert (not finished(s))
cached.enqueue(next) cached.enqueue(next)
case next.kind case next.kind
of yamlScalar: of yamlScalar:
@ -267,10 +254,6 @@ proc present*(s: YamlStream, target: Stream, tagLib: TagLibrary,
else: else:
length = high(int) length = high(int)
break break
except Exception:
var e = newException(YamlPresenterStreamError, "")
e.parent = getCurrentException()
raise e
nextState = if length <= 60: dFlowSequenceStart else: nextState = if length <= 60: dFlowSequenceStart else:
dBlockSequenceItem dBlockSequenceItem
of psJson: of psJson:
@ -318,11 +301,7 @@ proc present*(s: YamlStream, target: Stream, tagLib: TagLibrary,
mpInitial, mpKey, mpValue, mpNeedBlock mpInitial, mpKey, mpValue, mpNeedBlock
var mps = mpInitial var mps = mpInitial
while mps != mpNeedBlock: while mps != mpNeedBlock:
try: case s.peek().kind
let next = s()
assert (not finished(s))
cached.enqueue(next)
case next.kind
of yamlScalar, yamlAlias: of yamlScalar, yamlAlias:
case mps case mps
of mpInitial: mps = mpKey of mpInitial: mps = mpKey
@ -332,10 +311,6 @@ proc present*(s: YamlStream, target: Stream, tagLib: TagLibrary,
break break
else: else:
mps = mpNeedBlock mps = mpNeedBlock
except Exception:
var e = newException(YamlPresenterStreamError, "")
e.parent = getCurrentException()
raise e
nextState = if mps == mpNeedBlock: dBlockMapValue else: nextState = if mps == mpNeedBlock: dBlockMapValue else:
dBlockInlineMap dBlockInlineMap
of psMinimal: of psMinimal:
@ -363,11 +338,11 @@ proc present*(s: YamlStream, target: Stream, tagLib: TagLibrary,
indentation += indentationStep indentation += indentationStep
else: else:
if nextState in [dBlockMapValue, dBlockImplicitMapKey]: if nextState in [dBlockMapValue, dBlockImplicitMapKey]:
startItem(target, style, indentation, levels[levels.high],
true)
if style != psJson: if style != psJson:
writeTagAndAnchor(target, writeTagAndAnchor(target,
item.mapTag, tagLib, item.mapAnchor) item.mapTag, tagLib, item.mapAnchor)
startItem(target, style, indentation, levels[levels.high],
true)
else: else:
startItem(target, style, indentation, levels[levels.high], startItem(target, style, indentation, levels[levels.high],
true) true)
@ -453,15 +428,7 @@ proc present*(s: YamlStream, target: Stream, tagLib: TagLibrary,
assert false assert false
indentation -= indentationStep indentation -= indentationStep
of yamlEndDocument: of yamlEndDocument:
try: if finished(s): break
let next = s()
if finished(s):
break
cached.enqueue(next)
except Exception:
var e = newException(YamlPresenterStreamError, "")
e.parent = getCurrentException()
raise e
safeWrite("...\x0A") safeWrite("...\x0A")
proc transform*(input: Stream, output: Stream, style: PresentationStyle, proc transform*(input: Stream, output: Stream, style: PresentationStyle,
@ -473,7 +440,7 @@ proc transform*(input: Stream, output: Stream, style: PresentationStyle,
try: try:
if style == psCanonical: if style == psCanonical:
var specificTagEvents = iterator(): YamlStreamEvent = var specificTagEvents = iterator(): YamlStreamEvent =
for e in events(): for e in events:
var event = e var event = e
case event.kind case event.kind
of yamlStartDocument, yamlEndDocument, yamlEndMap, of yamlStartDocument, yamlEndDocument, yamlEndMap,
@ -503,12 +470,14 @@ proc transform*(input: Stream, output: Stream, style: PresentationStyle,
elif event.scalarTag == yTagExclamationMark: elif event.scalarTag == yTagExclamationMark:
event.scalarTag = yTagString event.scalarTag = yTagString
yield event yield event
present(specificTagEvents, output, tagLib, style, var s = initYamlStream(specificTagEvents)
present(s, output, tagLib, style,
indentationStep) indentationStep)
else: else:
present(events, output, tagLib, style, indentationStep) present(events, output, tagLib, style, indentationStep)
except YamlPresenterStreamError: except YamlStreamError:
let e = getCurrentException() var e = getCurrentException()
while e.parent of YamlStreamError: e = e.parent
if e.parent of IOError: if e.parent of IOError:
raise cast[ref IOError](e.parent) raise cast[ref IOError](e.parent)
elif e.parent of YamlParserError: elif e.parent of YamlParserError:

55
private/streams.nim Normal file
View File

@ -0,0 +1,55 @@
proc initYamlStream*(backend: iterator(): YamlStreamEvent): YamlStream =
result.peeked = false
result.backend = backend
proc next*(s: var YamlStream): YamlStreamEvent =
if s.peeked:
s.peeked = false
result = s.cached
else:
try:
result = s.backend()
assert(not finished(s.backend))
except AssertionError: raise
except YamlStreamError:
let cur = getCurrentException()
var e = newException(YamlStreamError, cur.msg)
e.parent = cur.parent
raise e
except Exception:
let cur = getCurrentException()
var e = newException(YamlStreamError, cur.msg)
e.parent = cur
raise e
proc peek*(s: var YamlStream): YamlStreamEvent =
if not s.peeked:
s.cached = s.next()
s.peeked = true
result = s.cached
proc `peek=`*(s: var YamlStream, value: YamlStreamEvent) =
s.cached = value
s.peeked = true
proc finished*(s: var YamlStream): bool =
if s.peeked:
result = false
else:
try:
s.cached = s.backend()
if finished(s.backend): result = true
else:
s.peeked = true
result = false
except AssertionError: raise
except YamlStreamError:
let cur = getCurrentException()
var e = newException(YamlStreamError, cur.msg)
e.parent = cur.parent
raise e
except Exception:
let cur = getCurrentException()
var e = newException(YamlStreamError, cur.msg)
e.parent = cur
raise e

View File

@ -8,7 +8,8 @@ proc wc(line, column: int, lineContent: string, message: string) =
proc ensureEqual(yamlIn, jsonIn: string) = proc ensureEqual(yamlIn, jsonIn: string) =
var var
parser = newYamlParser(initCoreTagLibrary(), wc) parser = newYamlParser(initCoreTagLibrary(), wc)
yamlResult = constructJson(parser.parse(newStringStream(yamlIn))) s = parser.parse(newStringStream(yamlIn))
yamlResult = constructJson(s)
jsonResult = parseJson(jsonIn) jsonResult = parseJson(jsonIn)
assert yamlResult.len == 1 assert yamlResult.len == 1
assert(jsonResult == yamlResult[0]) assert(jsonResult == yamlResult[0])

View File

@ -56,7 +56,7 @@ template ensure(input: string, expected: varargs[YamlStreamEvent]) {.dirty.} =
parser = newYamlParser(tagLib) parser = newYamlParser(tagLib)
events = parser.parse(newStringStream(input)) events = parser.parse(newStringStream(input))
try: try:
for token in events(): for token in events:
if i >= expected.len: if i >= expected.len:
echo "received more tokens than expected (next token = ", echo "received more tokens than expected (next token = ",
token.kind, ")" token.kind, ")"

View File

@ -223,7 +223,14 @@ next:
result: seq[ref Node] result: seq[ref Node]
parser = newYamlParser(tagLib) parser = newYamlParser(tagLib)
events = parser.parse(input) events = parser.parse(input)
try:
construct(events, result) construct(events, result)
except YamlConstructionError:
let ex = (ref YamlConstructionError)(getCurrentException())
echo "line ", parser.getLineNumber, ", column ",
parser.getColNumber, ": ", ex.msg
echo parser.getLineContent
assert(result.len == 3) assert(result.len == 3)
assert(result[0].value == "a") assert(result[0].value == "a")
assert(result[1].value == "b") assert(result[1].value == "b")

View File

@ -100,8 +100,9 @@ type
of yamlAlias: of yamlAlias:
aliasTarget* : AnchorId aliasTarget* : AnchorId
YamlStream* = iterator(): YamlStreamEvent ## \ YamlStream* = object ## \
## A ``YamlStream`` is an iterator that yields a well-formed stream of ## A ``YamlStream`` is an iterator-like object that yields a
## well-formed stream of
## ``YamlStreamEvents``. Well-formed means that every ``yamlStartMap`` ## ``YamlStreamEvents``. Well-formed means that every ``yamlStartMap``
## is terminated by a ``yamlEndMap``, every ``yamlStartSequence`` is ## is terminated by a ``yamlEndMap``, every ``yamlStartSequence`` is
## terminated by a ``yamlEndSequence`` and every ``yamlStartDocument`` ## terminated by a ``yamlEndSequence`` and every ``yamlStartDocument``
@ -113,6 +114,11 @@ type
## and is not required to check for it. The procs in this module will ## and is not required to check for it. The procs in this module will
## always yield a well-formed ``YamlStream`` and expect it to be ## always yield a well-formed ``YamlStream`` and expect it to be
## well-formed if it's an input. ## well-formed if it's an input.
##
##
backend: iterator(): YamlStreamEvent
peeked: bool
cached: YamlStreamEvent
TagLibrary* = ref object TagLibrary* = ref object
## A ``TagLibrary`` maps tag URIs to ``TagId`` s. ## A ``TagLibrary`` maps tag URIs to ``TagId`` s.
@ -228,10 +234,9 @@ type
## writing character data to the output stream raises any exception. ## writing character data to the output stream raises any exception.
## The error that has occurred is available from ``parent``. ## The error that has occurred is available from ``parent``.
YamlPresenterStreamError* = object of Exception YamlStreamError* = object of Exception
## Exception that may be raised by the YAML presenter. This occurs if ## Exception that may be raised by a ``YamlStream`` when the underlying
## an exception is raised while retrieving the next item from a ## backend raises an exception. The error that has occurred is
## `YamlStream <#YamlStream>`_. The error that has occurred is
## available from ``parent``. ## available from ``parent``.
YamlConstructionError* = object of YamlLoadingError YamlConstructionError* = object of YamlLoadingError
@ -241,10 +246,6 @@ type
## parsing, because otherwise this information is not available to the ## parsing, because otherwise this information is not available to the
## costruction proc. ## costruction proc.
YamlConstructionStreamError* = object of YamlLoadingError
## Exception that may be raised by a constructor if the input
## `YamlStream <#YamlStream>`_ raises an error. The error that has
## occurred is available from ``parent``.
const const
# failsafe schema # failsafe schema
@ -327,6 +328,33 @@ proc `==`*(left, right: AnchorId): bool {.borrow.}
proc `$`*(id: AnchorId): string {.borrow.} proc `$`*(id: AnchorId): string {.borrow.}
proc hash*(id: AnchorId): Hash {.borrow.} proc hash*(id: AnchorId): Hash {.borrow.}
proc initYamlStream*(backend: iterator(): YamlStreamEvent):
YamlStream {.raises: [].}
proc next*(s: var YamlStream): YamlStreamEvent {.raises: [YamlStreamError].}
proc peek*(s: var YamlStream): YamlStreamEvent {.raises: [YamlStreamError].}
proc `peek=`*(s: var YamlStream, value: YamlStreamEvent) {.raises: [].}
proc finished*(s: var YamlStream): bool {.raises: [YamlStreamError].}
iterator items*(s: var YamlStream): YamlStreamEvent {.raises: [YamlStreamError].} =
if s.peeked:
s.peeked = false
yield s.cached
while true:
var event: YamlStreamEvent
try:
event = s.backend()
if finished(s.backend): break
except AssertionError: raise
except YamlStreamError:
let cur = getCurrentException()
var e = newException(YamlStreamError, cur.msg)
e.parent = cur.parent
raise e
except Exception:
var e = newException(YamlStreamError, getCurrentExceptionMsg())
e.parent = getCurrentException()
raise e
yield event
proc initTagLibrary*(): TagLibrary {.raises: [].} proc initTagLibrary*(): TagLibrary {.raises: [].}
## initializes the ``tags`` table and sets ``nextCustomTagId`` to ## initializes the ``tags`` table and sets ``nextCustomTagId`` to
## ``yFirstCustomTagId``. ## ``yFirstCustomTagId``.
@ -386,12 +414,11 @@ proc getLineContent*(p: YamlParser, marker: bool = true): string {.raises: [].}
## be returned containing a ``^`` at the position of the recent parser ## be returned containing a ``^`` at the position of the recent parser
## token. ## token.
proc parse*(p: YamlParser, s: Stream): proc parse*(p: YamlParser, s: Stream): YamlStream {.raises: [].}
YamlStream {.raises: [IOError, YamlParserError].}
## Parse the given stream as YAML character stream. ## Parse the given stream as YAML character stream.
proc constructJson*(s: YamlStream): seq[JsonNode] proc constructJson*(s: var YamlStream): seq[JsonNode]
{.raises: [YamlConstructionError, YamlConstructionStreamError].} {.raises: [YamlConstructionError, YamlStreamError].}
## Construct an in-memory JSON tree from a YAML event stream. The stream may ## Construct an in-memory JSON tree from a YAML event stream. The stream may
## not contain any tags apart from those in ``coreTagLibrary``. Anchors and ## not contain any tags apart from those in ``coreTagLibrary``. Anchors and
## aliases will be resolved. Maps in the input must not contain ## aliases will be resolved. Maps in the input must not contain
@ -406,22 +433,24 @@ proc constructJson*(s: YamlStream): seq[JsonNode]
## of these values into a JSON character stream. ## of these values into a JSON character stream.
proc loadToJson*(s: Stream): seq[JsonNode] proc loadToJson*(s: Stream): seq[JsonNode]
{.raises: [IOError, YamlParserError, YamlConstructionError].} {.raises: [IOError, YamlParserError, YamlConstructionError,
OutOfMemError].}
## Uses `YamlParser <#YamlParser>`_ and ## Uses `YamlParser <#YamlParser>`_ and
## `constructJson <#constructJson>`_ to construct an in-memory JSON tree ## `constructJson <#constructJson>`_ to construct an in-memory JSON tree
## from a YAML character stream. ## from a YAML character stream.
proc present*(s: YamlStream, target: Stream, tagLib: TagLibrary, proc present*(s: var YamlStream, target: Stream, tagLib: TagLibrary,
style: PresentationStyle = psDefault, style: PresentationStyle = psDefault,
indentationStep: int = 2) {.raises: [YamlPresenterJsonError, indentationStep: int = 2) {.raises: [YamlPresenterJsonError,
YamlPresenterOutputError, YamlPresenterOutputError,
YamlPresenterStreamError].} YamlStreamError].}
## Convert ``s`` to a YAML character stream and write it to ``target``. ## Convert ``s`` to a YAML character stream and write it to ``target``.
proc transform*(input: Stream, output: Stream, style: PresentationStyle, proc transform*(input: Stream, output: Stream, style: PresentationStyle,
indentationStep: int = 2) {.raises: [IOError, YamlParserError, indentationStep: int = 2) {.raises: [IOError, YamlParserError,
YamlPresenterJsonError, YamlPresenterJsonError,
YamlPresenterOutputError].} YamlPresenterOutputError,
OutOfMemError].}
## Parser ``input`` as YAML character stream and then dump it to ``output`` ## Parser ``input`` as YAML character stream and then dump it to ``output``
## while resolving non-specific tags to the ones in the YAML core tag ## while resolving non-specific tags to the ones in the YAML core tag
## library. ## library.
@ -434,3 +463,4 @@ include private.json
include private.presenter include private.presenter
include private.hints include private.hints
include private.fastparse include private.fastparse
include private.streams

View File

@ -21,6 +21,8 @@ type
refsList: seq[RefNodeData] refsList: seq[RefNodeData]
style: AnchorStyle style: AnchorStyle
RawYamlStream* = iterator(): YamlStreamEvent {.raises: [].}
const const
yTagNimInt8* = 100.TagId yTagNimInt8* = 100.TagId
yTagNimInt16* = 101.TagId yTagNimInt16* = 101.TagId
@ -175,7 +177,7 @@ macro serializable*(types: stmt): stmt =
newNimNode(nnkExprColonExpr).add(newIdentNode("raises"), newNimNode(nnkExprColonExpr).add(newIdentNode("raises"),
newNimNode(nnkBracket).add( newNimNode(nnkBracket).add(
newIdentNode("YamlConstructionError"), newIdentNode("YamlConstructionError"),
newIdentNode("YamlConstructionStreamError")))) newIdentNode("YamlStreamError"))))
impl = quote do: impl = quote do:
var event = s() var event = s()
if finished(s) or event.kind != yamlStartMap: if finished(s) or event.kind != yamlStartMap:
@ -213,7 +215,7 @@ macro serializable*(types: stmt): stmt =
# representObject() # representObject()
var representProc = newProc(newIdentNode("representObject"), [ var representProc = newProc(newIdentNode("representObject"), [
newIdentNode("YamlStream"), newIdentNode("RawYamlStream"),
newIdentDefs(newIdentNode("value"), tIdent), newIdentDefs(newIdentNode("value"), tIdent),
newIdentDefs(newIdentNode("ts"), newIdentDefs(newIdentNode("ts"),
newIdentNode("TagStyle")), newIdentNode("TagStyle")),
@ -278,12 +280,6 @@ macro serializable*(types: stmt): stmt =
representProc[6] = impl representProc[6] = impl
result.add(representProc) result.add(representProc)
proc prepend(event: YamlStreamEvent, s: YamlStream): YamlStream {.raises: [].} =
result = iterator(): YamlStreamEvent =
yield event
for e in s():
yield e
proc safeTagUri*(id: TagId): string {.raises: [].} = proc safeTagUri*(id: TagId): string {.raises: [].} =
try: try:
let uri = serializationTagLibrary.uri(id) let uri = serializationTagLibrary.uri(id)
@ -295,17 +291,10 @@ proc safeTagUri*(id: TagId): string {.raises: [].} =
# cannot happen (theoretically, you known) # cannot happen (theoretically, you known)
assert(false) assert(false)
template constructScalarItem(item: YamlStreamEvent, name: string, t: TagId, template constructScalarItem(bs: var YamlStream, item: YamlStreamEvent,
content: stmt) = name: string, t: TagId, content: stmt) =
try: item = bs.next()
item = s() if item.kind != yamlScalar:
except YamlConstructionStreamError, AssertionError:
raise
except Exception:
var e = newException(YamlConstructionStreamError, "")
e.parent = getCurrentException()
raise e
if finished(s) or item.kind != yamlScalar:
raise newException(YamlConstructionError, "Expected scalar") raise newException(YamlConstructionError, "Expected scalar")
if item.scalarTag notin [yTagQuestionMark, yTagExclamationMark, t]: if item.scalarTag notin [yTagQuestionMark, yTagExclamationMark, t]:
raise newException(YamlConstructionError, "Wrong tag for " & name) raise newException(YamlConstructionError, "Wrong tag for " & name)
@ -319,26 +308,17 @@ template constructScalarItem(item: YamlStreamEvent, name: string, t: TagId,
e.parent = getCurrentException() e.parent = getCurrentException()
raise e raise e
template safeNextEvent(e: YamlStreamEvent, s: YamlStream) =
try:
e = s()
except YamlConstructionStreamError, AssertionError:
raise
except Exception:
var ex = newException(YamlConstructionStreamError, "")
ex.parent = getCurrentException()
raise ex
proc yamlTag*(T: typedesc[string]): TagId {.inline, raises: [].} = yTagString proc yamlTag*(T: typedesc[string]): TagId {.inline, raises: [].} = yTagString
proc constructObject*(s: YamlStream, c: ConstructionContext, result: var string) proc constructObject*(s: var YamlStream, c: ConstructionContext,
{.raises: [YamlConstructionError, YamlConstructionStreamError].} = result: var string)
{.raises: [YamlConstructionError, YamlStreamError].} =
var item: YamlStreamEvent var item: YamlStreamEvent
constructScalarItem(item, "string", yTagString): constructScalarItem(s, item, "string", yTagString):
result = item.scalarContent result = item.scalarContent
proc representObject*(value: string, ts: TagStyle = tsNone, proc representObject*(value: string, ts: TagStyle = tsNone,
c: SerializationContext): YamlStream {.raises: [].} = c: SerializationContext): RawYamlStream {.raises: [].} =
result = iterator(): YamlStreamEvent = result = iterator(): YamlStreamEvent =
yield scalarEvent(value, presentTag(string, ts), yAnchorNone) yield scalarEvent(value, presentTag(string, ts), yAnchorNone)
@ -348,25 +328,25 @@ proc yamlTag*(T: typedesc[int32]): TagId {.inline, raises: [].} = yTagNimInt32
proc yamlTag*(T: typedesc[int64]): TagId {.inline, raises: [].} = yTagNimInt64 proc yamlTag*(T: typedesc[int64]): TagId {.inline, raises: [].} = yTagNimInt64
proc constructObject*[T: int8|int16|int32|int64]( proc constructObject*[T: int8|int16|int32|int64](
s: YamlStream, c: ConstructionContext, result: var T) s: var YamlStream, c: ConstructionContext, result: var T)
{.raises: [YamlConstructionError, YamlConstructionStreamError].} = {.raises: [YamlConstructionError, YamlStreamError].} =
var item: YamlStreamEvent var item: YamlStreamEvent
constructScalarItem(item, name(T), yamlTag(T)): constructScalarItem(s, item, name(T), yamlTag(T)):
result = T(parseBiggestInt(item.scalarContent)) result = T(parseBiggestInt(item.scalarContent))
template constructObject*(s: YamlStream, c: ConstructionContext, template constructObject*(s: var YamlStream, c: ConstructionContext,
result: var int) = result: var int) =
{.fatal: "The length of `int` is platform dependent. Use int[8|16|32|64].".} {.fatal: "The length of `int` is platform dependent. Use int[8|16|32|64].".}
discard discard
proc representObject*[T: int8|int16|int32|int64]( proc representObject*[T: int8|int16|int32|int64](
value: T, ts: TagStyle = tsNone, c: SerializationContext): value: T, ts: TagStyle = tsNone, c: SerializationContext):
YamlStream {.raises: [].} = RawYamlStream {.raises: [].} =
result = iterator(): YamlStreamEvent = result = iterator(): YamlStreamEvent =
yield scalarEvent($value, presentTag(T, ts), yAnchorNone) yield scalarEvent($value, presentTag(T, ts), yAnchorNone)
template representObject*(value: int, tagStyle: TagStyle, template representObject*(value: int, tagStyle: TagStyle,
c: SerializationContext) = c: SerializationContext): RawYamlStream =
{.fatal: "The length of `int` is platform dependent. Use int[8|16|32|64].".} {.fatal: "The length of `int` is platform dependent. Use int[8|16|32|64].".}
discard discard
@ -388,13 +368,13 @@ proc parseBiggestUInt(s: string): uint64 =
{.pop.} {.pop.}
proc constructObject*[T: uint8|uint16|uint32|uint64]( proc constructObject*[T: uint8|uint16|uint32|uint64](
s: YamlStream, c: ConstructionContext, result: var T) s: var YamlStream, c: ConstructionContext, result: var T)
{.raises: [YamlConstructionError, YamlConstructionStreamError].} = {.raises: [YamlConstructionError, YamlStreamError].} =
var item: YamlStreamEvent var item: YamlStreamEvent
constructScalarItem(item, name[T], yamlTag(T)): constructScalarItem(s, item, name[T], yamlTag(T)):
result = T(parseBiggestUInt(item.scalarContent)) result = T(parseBiggestUInt(item.scalarContent))
template constructObject*(s: YamlStream, c: ConstructionContext, template constructObject*(s: var YamlStream, c: ConstructionContext,
result: var uint) = result: var uint) =
{.fatal: {.fatal:
"The length of `uint` is platform dependent. Use uint[8|16|32|64].".} "The length of `uint` is platform dependent. Use uint[8|16|32|64].".}
@ -402,11 +382,12 @@ template constructObject*(s: YamlStream, c: ConstructionContext,
proc representObject*[T: uint8|uint16|uint32|uint64]( proc representObject*[T: uint8|uint16|uint32|uint64](
value: T, ts: TagStyle, c: SerializationContext): value: T, ts: TagStyle, c: SerializationContext):
YamlStream {.raises: [].} = RawYamlStream {.raises: [].} =
result = iterator(): YamlStreamEvent = result = iterator(): YamlStreamEvent =
yield scalarEvent($value, presentTag(T, ts), yAnchorNone) yield scalarEvent($value, presentTag(T, ts), yAnchorNone)
template representObject*(value: uint, ts: TagStyle, c: SerializationContext) = template representObject*(value: uint, ts: TagStyle, c: SerializationContext):
RawYamlStream =
{.fatal: {.fatal:
"The length of `uint` is platform dependent. Use uint[8|16|32|64].".} "The length of `uint` is platform dependent. Use uint[8|16|32|64].".}
discard discard
@ -417,10 +398,10 @@ proc yamlTag*(T: typedesc[float64]): TagId {.inline, raises: [].} =
yTagNimFloat64 yTagNimFloat64
proc constructObject*[T: float32|float64]( proc constructObject*[T: float32|float64](
s: YamlStream, c: ConstructionContext, result: var T) s: var YamlStream, c: ConstructionContext, result: var T)
{.raises: [YamlConstructionError, YamlConstructionStreamError].} = {.raises: [YamlConstructionError, YamlStreamError].} =
var item: YamlStreamEvent var item: YamlStreamEvent
constructScalarItem(item, name(T), yamlTag(T)): constructScalarItem(s, item, name(T), yamlTag(T)):
let hint = guessType(item.scalarContent) let hint = guessType(item.scalarContent)
case hint case hint
of yTypeFloat: of yTypeFloat:
@ -436,13 +417,13 @@ proc constructObject*[T: float32|float64](
raise newException(YamlConstructionError, raise newException(YamlConstructionError,
"Cannot construct to float: " & item.scalarContent) "Cannot construct to float: " & item.scalarContent)
template constructObject*(s: YamlStream, c: ConstructionContext, template constructObject*(s: var YamlStream, c: ConstructionContext,
result: var float) = result: var float) =
{.fatal: "The length of `float` is platform dependent. Use float[32|64].".} {.fatal: "The length of `float` is platform dependent. Use float[32|64].".}
proc representObject*[T: float32|float64](value: T, ts: TagStyle, proc representObject*[T: float32|float64](value: T, ts: TagStyle,
c: SerializationContext): c: SerializationContext):
YamlStream {.raises: [].} = RawYamlStream {.raises: [].} =
result = iterator(): YamlStreamEvent = result = iterator(): YamlStreamEvent =
var var
asString: string asString: string
@ -458,15 +439,16 @@ proc representObject*[T: float32|float64](value: T, ts: TagStyle,
yield scalarEvent(asString, presentTag(T, ts), yAnchorNone) yield scalarEvent(asString, presentTag(T, ts), yAnchorNone)
template representObject*(value: float, tagStyle: TagStyle, template representObject*(value: float, tagStyle: TagStyle,
c: SerializationContext) = c: SerializationContext): RawYamlStream =
{.fatal: "The length of `float` is platform dependent. Use float[32|64].".} {.fatal: "The length of `float` is platform dependent. Use float[32|64].".}
proc yamlTag*(T: typedesc[bool]): TagId {.inline, raises: [].} = yTagBoolean proc yamlTag*(T: typedesc[bool]): TagId {.inline, raises: [].} = yTagBoolean
proc constructObject*(s: YamlStream, c: ConstructionContext, result: var bool) proc constructObject*(s: var YamlStream, c: ConstructionContext,
{.raises: [YamlConstructionError, YamlConstructionStreamError].} = result: var bool)
{.raises: [YamlConstructionError, YamlStreamError].} =
var item: YamlStreamEvent var item: YamlStreamEvent
constructScalarItem(item, "bool", yTagBoolean): constructScalarItem(s, item, "bool", yTagBoolean):
case guessType(item.scalarContent) case guessType(item.scalarContent)
of yTypeBoolTrue: of yTypeBoolTrue:
result = true result = true
@ -477,17 +459,18 @@ proc constructObject*(s: YamlStream, c: ConstructionContext, result: var bool)
"Cannot construct to bool: " & item.scalarContent) "Cannot construct to bool: " & item.scalarContent)
proc representObject*(value: bool, ts: TagStyle, proc representObject*(value: bool, ts: TagStyle,
c: SerializationContext): YamlStream {.raises: [].} = c: SerializationContext): RawYamlStream {.raises: [].} =
result = iterator(): YamlStreamEvent = result = iterator(): YamlStreamEvent =
yield scalarEvent(if value: "y" else: "n", presentTag(bool, ts), yield scalarEvent(if value: "y" else: "n", presentTag(bool, ts),
yAnchorNone) yAnchorNone)
proc yamlTag*(T: typedesc[char]): TagId {.inline, raises: [].} = yTagNimChar proc yamlTag*(T: typedesc[char]): TagId {.inline, raises: [].} = yTagNimChar
proc constructObject*(s: YamlStream, c: ConstructionContext, result: var char) proc constructObject*(s: var YamlStream, c: ConstructionContext,
{.raises: [YamlConstructionError, YamlConstructionStreamError].} = result: var char)
{.raises: [YamlConstructionError, YamlStreamError].} =
var item: YamlStreamEvent var item: YamlStreamEvent
constructScalarItem(item, "char", yTagNimChar): constructScalarItem(s, item, "char", yTagNimChar):
if item.scalarContent.len != 1: if item.scalarContent.len != 1:
raise newException(YamlConstructionError, raise newException(YamlConstructionError,
"Cannot construct to char (length != 1): " & "Cannot construct to char (length != 1): " &
@ -496,7 +479,7 @@ proc constructObject*(s: YamlStream, c: ConstructionContext, result: var char)
result = item.scalarContent[0] result = item.scalarContent[0]
proc representObject*(value: char, ts: TagStyle, proc representObject*(value: char, ts: TagStyle,
c: SerializationContext): YamlStream {.raises: [].} = c: SerializationContext): RawYamlStream {.raises: [].} =
result = iterator(): YamlStreamEvent = result = iterator(): YamlStreamEvent =
yield scalarEvent("" & value, presentTag(char, ts), yAnchorNone) yield scalarEvent("" & value, presentTag(char, ts), yAnchorNone)
@ -504,35 +487,28 @@ proc yamlTag*[I](T: typedesc[seq[I]]): TagId {.inline, raises: [].} =
let uri = "!nim:system:seq(" & safeTagUri(yamlTag(I)) & ")" let uri = "!nim:system:seq(" & safeTagUri(yamlTag(I)) & ")"
result = lazyLoadTag(uri) result = lazyLoadTag(uri)
proc constructObject*[T](s: YamlStream, c: ConstructionContext, proc constructObject*[T](s: var YamlStream, c: ConstructionContext,
result: var seq[T]) result: var seq[T])
{.raises: [YamlConstructionError, YamlConstructionStreamError].} = {.raises: [YamlConstructionError, YamlStreamError].} =
var event: YamlStreamEvent let event = s.next()
safeNextEvent(event, s) if event.kind != yamlStartSequence:
if finished(s) or event.kind != yamlStartSequence:
raise newException(YamlConstructionError, "Expected sequence start") raise newException(YamlConstructionError, "Expected sequence start")
if event.seqTag notin [yTagQuestionMark, yamlTag(seq[T])]: if event.seqTag notin [yTagQuestionMark, yamlTag(seq[T])]:
raise newException(YamlConstructionError, "Wrong tag for seq[T]") raise newException(YamlConstructionError, "Wrong tag for seq[T]")
result = newSeq[T]() result = newSeq[T]()
safeNextEvent(event, s) while s.peek().kind != yamlEndSequence:
assert(not finished(s)) var item: T
while event.kind != yamlEndSequence: try: constructObject(s, c, item)
var
item: T
events = prepend(event, s)
try:
constructObject(events, c, item)
except AssertionError, YamlConstructionError, except AssertionError, YamlConstructionError,
YamlConstructionStreamError: raise YamlStreamError: raise
except: except:
# compiler bug: https://github.com/nim-lang/Nim/issues/3772 # compiler bug: https://github.com/nim-lang/Nim/issues/3772
assert(false) assert(false)
result.add(item) result.add(item)
safeNextEvent(event, s) discard s.next()
assert(not finished(s))
proc representObject*[T](value: seq[T], ts: TagStyle, proc representObject*[T](value: seq[T], ts: TagStyle,
c: SerializationContext): YamlStream {.raises: [].} = c: SerializationContext): RawYamlStream {.raises: [].} =
result = iterator(): YamlStreamEvent = result = iterator(): YamlStreamEvent =
let childTagStyle = if ts == tsRootOnly: tsNone else: ts let childTagStyle = if ts == tsRootOnly: tsNone else: ts
yield YamlStreamEvent(kind: yamlStartSequence, yield YamlStreamEvent(kind: yamlStartSequence,
@ -540,7 +516,9 @@ proc representObject*[T](value: seq[T], ts: TagStyle,
seqAnchor: yAnchorNone) seqAnchor: yAnchorNone)
for item in value: for item in value:
var events = representObject(item, childTagStyle, c) var events = representObject(item, childTagStyle, c)
for event in events(): while true:
let event = events()
if finished(events): break
yield event yield event
yield YamlStreamEvent(kind: yamlEndSequence) yield YamlStreamEvent(kind: yamlEndSequence)
@ -559,38 +537,32 @@ proc yamlTag*[K, V](T: typedesc[Table[K, V]]): TagId {.inline, raises: [].} =
# cannot happen (theoretically, you known) # cannot happen (theoretically, you known)
assert(false) assert(false)
proc constructObject*[K, V](s: YamlStream, c: ConstructionContext, proc constructObject*[K, V](s: var YamlStream, c: ConstructionContext,
result: var Table[K, V]) result: var Table[K, V])
{.raises: [YamlConstructionError, YamlConstructionStreamError].} = {.raises: [YamlConstructionError, YamlStreamError].} =
var event: YamlStreamEvent let event = s.next()
safeNextEvent(event, s)
assert(not finished(s))
if event.kind != yamlStartMap: if event.kind != yamlStartMap:
raise newException(YamlConstructionError, "Expected map start, got " & raise newException(YamlConstructionError, "Expected map start, got " &
$event.kind) $event.kind)
if event.mapTag notin [yTagQuestionMark, yamlTag(Table[K, V])]: if event.mapTag notin [yTagQuestionMark, yamlTag(Table[K, V])]:
raise newException(YamlConstructionError, "Wrong tag for Table[K, V]") raise newException(YamlConstructionError, "Wrong tag for Table[K, V]")
result = initTable[K, V]() result = initTable[K, V]()
safeNextEvent(event, s) while s.peek.kind != yamlEndMap:
assert(not finished(s))
while event.kind != yamlEndMap:
var var
key: K key: K
value: V value: V
events = prepend(event, s)
try: try:
constructObject(events, c, key) constructObject(s, c, key)
constructObject(s, c, value) constructObject(s, c, value)
except AssertionError: raise except AssertionError: raise
except Exception: except Exception:
# compiler bug: https://github.com/nim-lang/Nim/issues/3772 # compiler bug: https://github.com/nim-lang/Nim/issues/3772
assert(false) assert(false)
result[key] = value result[key] = value
safeNextEvent(event, s) discard s.next()
assert(not finished(s))
proc representObject*[K, V](value: Table[K, V], ts: TagStyle, proc representObject*[K, V](value: Table[K, V], ts: TagStyle,
c: SerializationContext): YamlStream {.raises:[].} = c: SerializationContext): RawYamlStream {.raises:[].} =
result = iterator(): YamlStreamEvent = result = iterator(): YamlStreamEvent =
let childTagStyle = if ts == tsRootOnly: tsNone else: ts let childTagStyle = if ts == tsRootOnly: tsNone else: ts
yield YamlStreamEvent(kind: yamlStartMap, yield YamlStreamEvent(kind: yamlStartMap,
@ -598,16 +570,20 @@ proc representObject*[K, V](value: Table[K, V], ts: TagStyle,
mapAnchor: yAnchorNone) mapAnchor: yAnchorNone)
for key, value in value.pairs: for key, value in value.pairs:
var events = representObject(key, childTagStyle, c) var events = representObject(key, childTagStyle, c)
for event in events(): while true:
let event = events()
if finished(events): break
yield event yield event
events = representObject(value, childTagStyle, c) events = representObject(value, childTagStyle, c)
for event in events(): while true:
let event = events()
if finished(events): break
yield event yield event
yield YamlStreamEvent(kind: yamlEndMap) yield YamlStreamEvent(kind: yamlEndMap)
template yamlTag*(T: typedesc[object|enum]): expr = template yamlTag*(T: typedesc[object|enum]): expr =
var uri = when compiles(yamlTagId(T)): yamlTagId(T) else: var uri = when compiles(yamlTagId(T)): yamlTagId(T) else:
"!nim:custom:" & T.name "!nim:custom:" & (typetraits.name(type(T)))
try: try:
serializationTagLibrary.tags[uri] serializationTagLibrary.tags[uri]
except KeyError: except KeyError:
@ -626,32 +602,34 @@ template yamlTag*(T: typedesc[tuple]): expr =
try: serializationTagLibrary.tags[uri] try: serializationTagLibrary.tags[uri]
except KeyError: serializationTagLibrary.registerUri(uri) except KeyError: serializationTagLibrary.registerUri(uri)
proc constructObject*[O: object|tuple](s: YamlStream, c: ConstructionContext, proc constructObject*[O](s: var YamlStream, c: ConstructionContext,
result: var ref O)
{.raises: [YamlConstructionError, YamlStreamError].}
proc constructObject*[O: object|tuple](s: var YamlStream,
c: ConstructionContext,
result: var O) result: var O)
{.raises: [YamlConstructionError, YamlConstructionStreamError].} = {.raises: [YamlConstructionError, YamlStreamError].} =
var e = s() let e = s.next()
assert(not finished(s))
if e.kind != yamlStartMap: if e.kind != yamlStartMap:
raise newException(YamlConstructionError, "Expected map start, got " & raise newException(YamlConstructionError, "Expected map start, got " &
$e.kind) $e.kind)
if e.mapAnchor != yAnchorNone: if e.mapAnchor != yAnchorNone:
raise newException(YamlConstructionError, "Anchor on a non-ref type") raise newException(YamlConstructionError, "Anchor on a non-ref type")
e = s() while s.peek.kind != yamlEndMap:
assert(not finished(s)) let e = s.next()
while e.kind != yamlEndMap:
if e.kind != yamlScalar: if e.kind != yamlScalar:
raise newException(YamlConstructionError, "Expected field name") raise newException(YamlConstructionError,
"Expected field name, got " & $e.kind)
let name = e.scalarContent let name = e.scalarContent
for fname, value in fieldPairs(result): for fname, value in fieldPairs(result):
if fname == name: if fname == name:
constructObject(s, c, value) constructObject(s, c, value)
break break
e = s() discard s.next()
assert(not finished(s))
proc representObject*[O: object|tuple](value: O, ts: TagStyle, proc representObject*[O: object|tuple](value: O, ts: TagStyle,
c: SerializationContext): c: SerializationContext): RawYamlStream {.raises: [].} =
YamlStream {.raises: [].} =
result = iterator(): YamlStreamEvent = result = iterator(): YamlStreamEvent =
let childTagStyle = if ts == tsRootOnly: tsNone else: ts let childTagStyle = if ts == tsRootOnly: tsNone else: ts
yield startMapEvent(presentTag(O, ts), yAnchorNone) yield startMapEvent(presentTag(O, ts), yAnchorNone)
@ -659,15 +637,16 @@ proc representObject*[O: object|tuple](value: O, ts: TagStyle,
yield scalarEvent(name, presentTag(string, childTagStyle), yield scalarEvent(name, presentTag(string, childTagStyle),
yAnchorNone) yAnchorNone)
var events = representObject(value, childTagStyle, c) var events = representObject(value, childTagStyle, c)
for event in events(): while true:
let event = events()
if finished(events): break
yield event yield event
yield endMapEvent() yield endMapEvent()
proc constructObject*[O: enum](s: YamlStream, c: ConstructionContext, proc constructObject*[O: enum](s: var YamlStream, c: ConstructionContext,
result: var O) result: var O)
{.raises: [YamlConstructionError, YamlConstructionStreamError].} = {.raises: [YamlConstructionError, YamlStreamError].} =
let e = s() let e = s.next()
assert(not finished(s))
if e.kind != yamlScalar: if e.kind != yamlScalar:
raise newException(YamlConstructionError, "Expected scalar, got " & raise newException(YamlConstructionError, "Expected scalar, got " &
$e.kind) $e.kind)
@ -679,55 +658,57 @@ proc constructObject*[O: enum](s: YamlStream, c: ConstructionContext,
raise ex raise ex
proc representObject*[O: enum](value: O, ts: TagStyle, proc representObject*[O: enum](value: O, ts: TagStyle,
c: SerializationContext): c: SerializationContext): RawYamlStream {.raises: [].} =
YamlStream {.raises: [].} =
result = iterator(): YamlStreamEvent = result = iterator(): YamlStreamEvent =
yield scalarEvent($value, presentTag(O, ts), yAnchorNone) yield scalarEvent($value, presentTag(O, ts), yAnchorNone)
proc yamlTag*[O](T: typedesc[ref O]): TagId {.inline, raises: [].} = yamlTag(O) proc yamlTag*[O](T: typedesc[ref O]): TagId {.inline, raises: [].} = yamlTag(O)
proc constructObject*[O](s: YamlStream, c: ConstructionContext, proc constructObject*[O](s: var YamlStream, c: ConstructionContext,
result: var ref O) result: var ref O) =
{.raises: [YamlConstructionError, YamlConstructionStreamError].} = var e = s.peek()
var e = s()
assert(not finished(s))
if e.kind == yamlScalar: if e.kind == yamlScalar:
if e.scalarTag == yTagNull or ( if e.scalarTag == yTagNull or (
e.scalarTag in [yTagQuestionMark, yTagExclamationMark] and e.scalarTag in [yTagQuestionMark, yTagExclamationMark] and
guessType(e.scalarContent) == yTypeNull): guessType(e.scalarContent) == yTypeNull):
result = nil result = nil
discard s.next()
return return
elif e.kind == yamlAlias: elif e.kind == yamlAlias:
try: try:
result = cast[ref O](c.refs[e.aliasTarget]) result = cast[ref O](c.refs[e.aliasTarget])
discard s.next()
return return
except KeyError: except KeyError:
assert(false) assert(false)
new(result) new(result)
var a: ptr AnchorId template removeAnchor(anchor: var AnchorId) {.dirty.} =
if anchor != yAnchorNone:
assert(not c.refs.hasKey(anchor))
c.refs[anchor] = cast[pointer](result)
anchor = yAnchorNone
case e.kind case e.kind
of yamlScalar: a = addr(e.scalarAnchor) of yamlScalar: removeAnchor(e.scalarAnchor)
of yamlStartMap: a = addr(e.mapAnchor) of yamlStartMap: removeAnchor(e.mapAnchor)
of yamlStartSequence: a = addr(e.seqAnchor) of yamlStartSequence: removeAnchor(e.seqAnchor)
else: assert(false) else: assert(false)
if a[] != yAnchorNone: s.peek = e
assert(not c.refs.hasKey(a[]))
c.refs[a[]] = cast[pointer](result)
a[] = yAnchorNone
try: try:
constructObject(prepend(e, s), c, result[]) constructObject(s, c, result[])
except YamlConstructionError, YamlConstructionStreamError, AssertionError: except YamlConstructionError, YamlStreamError, AssertionError:
raise raise
except Exception: except Exception:
var e = newException(YamlConstructionStreamError, var e = newException(YamlStreamError,
getCurrentExceptionMsg()) getCurrentExceptionMsg())
e.parent = getCurrentException() e.parent = getCurrentException()
raise e raise e
proc representObject*[O](value: ref O, ts: TagStyle, c: SerializationContext): proc representObject*[O](value: ref O, ts: TagStyle, c: SerializationContext):
YamlStream {.raises: [].} = RawYamlStream {.raises: [].} =
if value == nil: if value == nil:
result = iterator(): YamlStreamEvent = yield scalarEvent("~", yTagNull) result = iterator(): YamlStreamEvent =
yield scalarEvent("~", yTagNull)
elif c.style == asNone: elif c.style == asNone:
result = representObject(value[], ts, c) result = representObject(value[], ts, c)
else: else:
@ -743,11 +724,10 @@ proc representObject*[O](value: ref O, ts: TagStyle, c: SerializationContext):
c.refsList.add(initRefNodeData(p)) c.refsList.add(initRefNodeData(p))
let a = if c.style == asAlways: AnchorId(c.refsList.high) else: let a = if c.style == asAlways: AnchorId(c.refsList.high) else:
cast[AnchorId](p) cast[AnchorId](p)
try: result = iterator(): YamlStreamEvent =
var var child = representObject(value[], ts, c)
objStream = representObject(value[], ts, c) var first = child()
first = objStream() assert(not finished(child))
assert(not finished(objStream))
case first.kind case first.kind
of yamlStartMap: of yamlStartMap:
first.mapAnchor = a first.mapAnchor = a
@ -755,27 +735,29 @@ proc representObject*[O](value: ref O, ts: TagStyle, c: SerializationContext):
first.seqAnchor = a first.seqAnchor = a
of yamlScalar: of yamlScalar:
first.scalarAnchor = a first.scalarAnchor = a
else: else: discard
assert(false) yield first
result = prepend(first, objStream) while true:
except Exception: let event = child()
assert(false) if finished(child): break
yield event
proc construct*[T](s: YamlStream, target: var T) proc construct*[T](s: var YamlStream, target: var T)
{.raises: [YamlConstructionError, YamlConstructionStreamError].} = {.raises: [YamlConstructionError, YamlStreamError].} =
var context = newConstructionContext() var
context = newConstructionContext()
try: try:
var e = s() var e = s.next()
assert((not finished(s)) and e.kind == yamlStartDocument) assert(e.kind == yamlStartDocument)
constructObject(s, context, target) constructObject(s, context, target)
e = s() e = s.next()
assert((not finished(s)) and e.kind == yamlEndDocument) assert(e.kind == yamlEndDocument)
except YamlConstructionError, YamlConstructionStreamError, AssertionError: except YamlConstructionError, YamlStreamError, AssertionError:
raise raise
except Exception: except Exception:
# may occur while calling s() # may occur while calling s()
var ex = newException(YamlConstructionStreamError, "") var ex = newException(YamlStreamError, "")
ex.parent = getCurrentException() ex.parent = getCurrentException()
raise ex raise ex
@ -788,14 +770,13 @@ proc load*[K](input: Stream, target: var K)
construct(events, target) construct(events, target)
except YamlConstructionError, AssertionError: except YamlConstructionError, AssertionError:
raise raise
except YamlConstructionStreamError: except YamlStreamError:
let e = (ref YamlConstructionStreamError)(getCurrentException()) let e = (ref YamlStreamError)(getCurrentException())
if e.parent of IOError: if e.parent of IOError:
raise (ref IOError)(e.parent) raise (ref IOError)(e.parent)
elif e.parent of YamlParserError: elif e.parent of YamlParserError:
raise (ref YamlParserError)(e.parent) raise (ref YamlParserError)(e.parent)
else: else:
echo e.parent.repr
assert(false) assert(false)
except Exception: except Exception:
# compiler bug: https://github.com/nim-lang/Nim/issues/3772 # compiler bug: https://github.com/nim-lang/Nim/issues/3772
@ -826,31 +807,18 @@ proc setAliasAnchor(a: var AnchorId, q: var seq[RefNodeData]) {.inline.} =
return return
assert(false) assert(false)
proc insideDoc(s: YamlStream): YamlStream {.raises: [].} =
result = iterator(): YamlStreamEvent =
yield YamlStreamEvent(kind: yamlStartDocument)
while true:
var event: YamlStreamEvent
try:
event = s()
if finished(s): break
except AssertionError: raise
except Exception:
# serializing object does not raise any errors, so we can
# ignore this
assert(false)
yield event
yield YamlStreamEvent(kind: yamlEndDocument)
proc represent*[T](value: T, ts: TagStyle = tsRootOnly, proc represent*[T](value: T, ts: TagStyle = tsRootOnly,
a: AnchorStyle = asTidy): YamlStream {.raises: [].} = a: AnchorStyle = asTidy): YamlStream {.raises: [].} =
var var
context = newSerializationContext(a) context = newSerializationContext(a)
objStream: YamlStream objStream = iterator(): YamlStreamEvent =
try: yield YamlStreamEvent(kind: yamlStartDocument)
objStream = insideDoc(representObject(value, ts, context)) var events = representObject(value, ts, context)
except Exception: while true:
assert(false) let e = events()
if finished(events): break
yield e
yield YamlStreamEvent(kind: yamlEndDocument)
if a == asTidy: if a == asTidy:
var objQueue = newSeq[YamlStreamEvent]() var objQueue = newSeq[YamlStreamEvent]()
try: try:
@ -859,7 +827,7 @@ proc represent*[T](value: T, ts: TagStyle = tsRootOnly,
except Exception: except Exception:
assert(false) assert(false)
var next = 0.AnchorId var next = 0.AnchorId
result = iterator(): YamlStreamEvent = var backend = iterator(): YamlStreamEvent =
for i in countup(0, objQueue.len - 1): for i in countup(0, objQueue.len - 1):
var event = objQueue[i] var event = objQueue[i]
case event.kind case event.kind
@ -874,8 +842,9 @@ proc represent*[T](value: T, ts: TagStyle = tsRootOnly,
else: else:
discard discard
yield event yield event
result = initYamlStream(backend)
else: else:
result = objStream result = initYamlStream(objStream)
proc dump*[K](value: K, target: Stream, style: PresentationStyle = psDefault, proc dump*[K](value: K, target: Stream, style: PresentationStyle = psDefault,
tagStyle: TagStyle = tsRootOnly, tagStyle: TagStyle = tsRootOnly,
@ -885,13 +854,11 @@ proc dump*[K](value: K, target: Stream, style: PresentationStyle = psDefault,
if style == psJson: asNone else: anchorStyle) if style == psJson: asNone else: anchorStyle)
try: try:
present(events, target, serializationTagLibrary, style, indentationStep) present(events, target, serializationTagLibrary, style, indentationStep)
except YamlPresenterStreamError: except YamlStreamError:
# serializing object does not raise any errors, so we can ignore this # serializing object does not raise any errors, so we can ignore this
var e = getCurrentException() var e = getCurrentException()
echo e.msg
echo e.parent.repr
assert(false) assert(false)
except YamlPresenterJsonError, YamlPresenterOutputError, AssertionError: except YamlPresenterJsonError, YamlPresenterOutputError, AssertionError, FieldError:
raise raise
except Exception: except Exception:
# cannot occur as represent() doesn't raise any errors # cannot occur as represent() doesn't raise any errors