status-im
/
NimYAML
mirror of https://github.com/status-im/NimYAML.git
2
0
Fork 0
Code Issues Projects Releases Wiki Activity

Improved docs. Fixed serialization problem. 

* Added serialization.txt to doc
 * Fixed rendering problem in api.txt
 * Use explicit tag when ref type renders to a scalar that can be
   parsed to !!null
 * Added test cases for this ref type fix
This commit is contained in:
Felix Krause 2016-02-15 22:54:05 +01:00
parent d2fbcb9b70
commit 0a7f87a539
7 changed files with 192 additions and 14 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
config.nims
View File

@ -23,6 +23,7 @@ task documentation, "Generate documentation":
exec r"nim doc2 -o:docout/yaml.html --docSeeSrcUrl:https://github.com/flyx/NimYAML/blob/`git log -n 1 --format=%H` yaml"
exec r"nim rst2html -o:docout/index.html doc/index.txt"
exec r"nim rst2html -o:docout/api.html doc/api.txt"
exec r"nim rst2html -o:docout/serialization.html doc/serialization.txt"
exec "cp doc/docutils.css doc/style.css doc/testing.html doc/processing.svg docout"
setCommand "nop"

20
doc/api.txt
View File

@ -1,9 +1,9 @@
===
API
===
============
API Overview
============
Overview
========
Introduction
============
NimYAML advocates parsing YAML input into native Nim types. Basic Nim library
types like integers, floats and strings, as well as all tuples, enums and
@ -51,10 +51,10 @@ way to load YAML data. The following paragraphs will explain the steps involved.
For parsing, a `YamlParser <yaml.html#YamlParser>`_ object is needed. This
object stores some state while parsing that may be useful for error reporting to
the user. The
`parse <yaml.html#present,YamlStream,Stream,TagLibrary,PresentationStyle,int>`_
proc implements the YAML processing step of the same name. All syntax errors in
the input character stream are processed by ``parse``, which will raise a
``YamlParserError`` if it encounters a syntax error.
`parse <yaml.html#parse,YamlParser,Stream>`_ proc implements the YAML processing
step of the same name. All syntax errors in the input character stream are
processed by ``parse``, which will raise a ``YamlParserError`` if it encounters
a syntax error.
Transforming a ``YamlStream`` to a native YAML object is done via
``construct``. It skips the ``compose`` step for efficiency reasons. As Nim is
@ -81,6 +81,6 @@ that if you use ``asNone``, the value you serialize might not round-trip.
Transforming a ``YamlStream`` into YAML character data is done with
`present <yaml.html#present,YamlStream,Stream,TagLibrary,PresentationStyle,int>`_.
You can choose from multiple presentation styles. ``psJson`` is not able to
process some features of ``YamlStream``s, the other styles support all features
process some features of ``YamlStream`` s, the other styles support all features
and are guaranteed to round-trip to the same ``YamlStream`` if you parse the
generated YAML character stream again.

139
doc/serialization.txt Normal file
View File

@ -0,0 +1,139 @@
======================
Serialization Overview
======================
Introduction
============
NimYAML tries hard to make transforming YAML characters streams to native Nim
types and vice versa as easy as possible. In simple scenarios, you might not
need anything else than the two procs
`dump <yaml.html#dump,K,Stream,PresentationStyle,TagStyle,AnchorStyle,int>`_ and
`load <yaml.html#load,Stream,K>`_. On the other side, the process should be as
customizable as possible to allow the user to tightly control how the generated
YAML character stream will look and how a YAML character stream is interpreted.
An important thing to remember in NimYAML is that unlike in interpreted
languages like Ruby, Nim cannot load a YAML character stream without knowing the
resulting type beforehand. For example, if you want to load this piece of YAML:
.. code-block:: yaml
%YAML 1.2
--- !nim:system:seq(nim:system:int8)
- 1
- 2
- 3
You would need to know that it will load a ``seq[int8]`` *at compile time*. This
is not really a problem because without knowing which type you will load, you
cannot do anything useful with the result afterwards in the code. But it may be
unfamiliar for programmers who are used to the YAML libraries of Python or Ruby.
Supported Types
===============
NimYAML supports a growing number of types of Nim's ``system`` module and
standard library, and it also supports user-defined object, tuple and enum types
out of the box.
**Important**: NimYAML currently does not support polymorphism or variant
object types. This may be added in the future.
This also means that NimYAML is generally able to work with object, tuple and
enum types defined in the standard library or a third-party library without
further configuration.
Scalar Types
------------
The following integer types are supported by NimYAML: ``int8``, ``int16``,
``int32``, ``int64``, ``uint8``, ``uint16``, ``uint32``, ``uint64``. Note that
the ``int`` type is *not* supported, since its sice varies based on the target
operation system. This makes it unfit for usage within NimYAML, because a value
might not round-trip between a binary for a 32-bit OS and another binary for a
64-bit OS. The same goes for ``uint``.
The floating point types ``float32`` and ``float64`` are also supported, but
``float`` is not, for the same reason.
``string`` is supported and one of the few Nim types which directly map to a
standard YAML type. ``char`` is also supported.
To support new scalar types, you must implement the ``constructObject()`` and
``representObject()`` procs on that type (see below).
Collection Types
----------------
NimYAML supports Nim's ``seq`` and ``Table`` types out of the box. Unlike the
native YAML types ``!!seq`` and ``!!map``, ``seq`` and ``Table`` define the type
of all their items (or keys and values). So YAML objects with heterogeneous
types in them cannot be loaded to Nim collection types. For example, this
sequence:
.. code-block:: yaml
%YAML 1.2
--- !!seq
- !!int 1
- !!string foo
Cannot be loaded to a Nim ``seq``. For this reason, you cannot load YAML's
native ``!!map`` and ``!!seq`` types directly into Nim types.
Reference Types
---------------
A reference to any supported non-reference type (including user defined types,
see below) is supported by NimYAML. A reference type will be treated like its
base type, but NimYAML is able to detect multiple references to the same object
and dump the structure properly with anchors and aliases in place. It is
possible to dump and load cyclic data structures without further configuration.
It is possible for reference types to hold a ``nil`` value, which will be mapped
to the ``!!null`` YAML scalar type.
Pointer types are not supported because it seems dangerous to automatically
allocate memory which the user must then manually deallocate.
User Defined Types
------------------
For an object or tuple type to be directly usable with NimYAML, the following
conditions must be met:
- Every type contained in the object/tuple must be supported
- All fields of an object type must be accessible from the code position where
you call NimYAML. If an object has non-public member fields, it can only be
processed in the module where it is defined.
- The object may not contain a ``case`` clause.
- The object may not have a generic parameter
NimYAML will present enum types as YAML scalars, and tuple and object types as
YAML maps. Some of the conditions above may be loosened in future releases.
Tags
====
NimYAML uses local tags to represent Nim types that do not map directly to a
YAML type. For example, ``int8`` is presented with the tag ``!nim:system:int8``.
Tags are mostly unnecessary when loading YAML data because the user already
defines the target Nim type which usually defines all types of the structure.
However, there is one case where a tag is necessary: A reference type with the
value ``nil`` is represented in YAML as a ``!!null`` scalar. This will be
automatically detected by type guessing, but if it is for example a reference to
a string with the value ``"~"``, it must be tagged with ``!!string``, because
otherwise, it would be loaded as ``nil``.
As you might have noticed in the example above, the YAML tag of a ``seq``
depends on its generic type parameter. The same applies to ``Table``. So, a
table that maps ``int8`` to string sequences would be presented with the tag
``!nim:tables:Table(nim:system:int8,nim:system:seq(tag:yaml.org,2002:string))``.
These tags are generated on the fly based on the types you instantiate
``Table`` or ``seq`` with.
You may customize the tags used for your types by using the template
`setTagUriForType <yaml.html#setTagUriForType.t,typedesc,string>`_. It may not
be applied to scalar and collection types implemented by NimYAML, but you can
for example use it on a certain ``seq`` type:
.. code-block:: nim
setTagUriForType(seq[string], "!nim:my:seq")

3
doc/testing.html
View File

@ -15,8 +15,9 @@
<a class="pagetitle" href="/">NimYAML</a>
<a href="index.html">Home</a>
<a href="testing.html">Testing Ground</a>
<span>API:</span>
<span>Docs:</span>
<a href="api.html">Overview</a>
<a href="serialization.html">Serialization</a>
<a href="yaml.html">Module yaml</a>
</header>
<article id="documentId">

3
nimdoc.cfg
View File

@ -107,8 +107,9 @@ doc.file = """
<a class="pagetitle" href="/">NimYAML</a>
<a href="index.html">Home</a>
<a href="testing.html">Testing Ground</a>
<span>API:</span>
<span>Docs:</span>
<a href="api.html">Overview</a>
<a href="serialization.html">Serialization</a>
<a href="yaml.html">Module yaml</a>
</header>
<article id="documentId">

10
private/serialization.nim
View File

@ -691,19 +691,25 @@ proc representObject*[O](value: ref O, ts: TagStyle, c: SerializationContext):
cast[AnchorId](p))
return
c.refsList.add(initRefNodeData(p))
let a = if c.style == asAlways: AnchorId(c.refsList.high) else:
let
a = if c.style == asAlways: AnchorId(c.refsList.high) else:
cast[AnchorId](p)
childTagStyle = if ts == tsAll: tsAll else: tsRootOnly
result = iterator(): YamlStreamEvent =
var child = representObject(value[], ts, c)
var child = representObject(value[], childTagStyle, c)
var first = child()
assert(not finished(child))
case first.kind
of yamlStartMap:
first.mapAnchor = a
if ts == tsNone: first.mapTag = yTagQuestionMark
of yamlStartSequence:
first.seqAnchor = a
if ts == tsNone: first.seqTag = yTagQuestionMark
of yamlScalar:
first.scalarAnchor = a
if ts == tsNone and guessType(first.scalarContent) != yTypeNull:
first.scalarTag = yTagQuestionMark
else: discard
yield first
while true:

30
test/serializing.nim
View File

@ -234,6 +234,7 @@ next:
echo "line ", parser.getLineNumber, ", column ",
parser.getColNumber, ": ", ex.msg
echo parser.getLineContent
raise ex
assert(result.len == 3)
assert(result[0].value == "a")
@ -242,4 +243,33 @@ next:
assert(result[0].next == result[1])
assert(result[1].next == result[2])
assert(result[2].next == result[0])
test "Serialization: Load nil values":
let input = newStringStream("- ~\n- !!str ~")
var
result: seq[ref string]
parser = newYamlParser(tagLib)
events = parser.parse(input)
try:
construct(events, result)
except YamlConstructionError:
let ex = (ref YamlConstructionError)(getCurrentException())
echo "line ", parser.getLineNumber, ", column ",
parser.getColNumber, ": ", ex.msg
echo parser.getLineContent
raise ex
assert(result.len == 2)
assert(result[0] == nil)
assert(result[1][] == "~")
test "Serialization: Represent nil values":
var input = newSeq[ref string]()
input.add(nil)
input.add(new string)
input[1][] = "~"
var output = newStringStream()
dump(input, output, psBlockOnly, tsRootOnly)
assertStringEqual "%YAML 1.2\n--- !nim:system:seq(tag:yaml.org,2002:str) \n- !!null ~\n- !!str ~",
output.data