diff --git a/config.nims b/config.nims index 5e34367..da29fc0 100644 --- a/config.nims +++ b/config.nims @@ -48,7 +48,8 @@ task documentation, "Generate documentation": exec r"nim rst2html -o:../docout/api.html tmp.rst" exec r"./rstPreproc -o:tmp.rst serialization.txt" exec r"nim rst2html -o:../docout/serialization.html tmp.rst" - exec r"nim rst2html -o:../docout/testing.html testing.txt" + exec r"nim rst2html -o:../docout/testing.html testing.rst" + exec r"nim rst2html -o:../docout/schema.html schema.rst" exec "cp docutils.css style.css processing.svg ../docout" exec r"nim doc2 -o:docout/yaml.html --docSeeSrcUrl:https://github.com/flyx/NimYAML/blob/`git log -n 1 --format=%H` yaml" for file in listFiles("yaml"): diff --git a/doc/schema.rst b/doc/schema.rst new file mode 100644 index 0000000..b838c78 --- /dev/null +++ b/doc/schema.rst @@ -0,0 +1,139 @@ +==================== +Serialization Schema +==================== + +This document details the existing mappings in NimYAML from Nim types to YAML +tags. Throughout this document, there are two *tag shorthands* being used: + +========= ========================= +Shorthand Expansion +========= ========================= +``!!`` ``tag:yaml.org,2002:`` +``!n!`` ``tag:nimyaml.org,2016:`` +========= ========================= + +The first one is defined by the YAML specification and is used for types from +the YAML failsafe, JSON or core schema. The second one is defined by NimYAML and +is used for types from the Nim standard library. + +The YAML tag system has no understanding of generics. This means that NimYAML +must map every generic type instance to a YAML tag that describes that exact +type instance. For example, a ``seq[string]`` is mapped to the tag +``!n!system:seq(tag:yaml.org;2002:string)``. + +As you can see, the expanded tag handle of the generic type parameter is added +to the tag of the generic type. To be compliant with the YAML spec, the +following modifications are made: + +* Any exclamation marks are removed from the expanded tag. An exclamation mark + may only occur at the beginning of the tag as defined by the YAML spec. +* Any commas are replaces by semicolons, because they may not occur in a tag + apart from within the tag handle expansion. + +If a type takes multiple generic parameters, the tag handles are separated by +semicolons within the parentheses. Note that this does not guarantee unique tag +handles for every type, but it is currently seen as good enough. + +Note that user-defined generic types are currently not officially supported by +NimYAML. Only the generic collection types explicitly listed here use this +mechanism for crafting YAML tags. + +Scalar Types +============ + +The following table defines all non-composed, atomar types that are mapped to +YAML types by NimYAML. + +========= =========================================================== +Nim type YAML tag +========= =========================================================== +char ``!n!system:char`` +string ``!!string`` (or ``!n!nil:string`` if nil) +int ``!n!system:int32`` (independent on target architecture) +int8 ``!n!system:int8`` +int16 ``!n!system:int16`` +int32 ``!n!system:int32`` +int64 ``!n!system:int64`` +uint ``!n!system:uint32`` (independent from target architecture) +uint8 ``!n!system:uint8`` +uint16 ``!n!system:uint16`` +uint32 ``!n!system:uint32`` +uint64 ``!n!system:uint64`` +float ``!n!system:float64`` +float32 ``!n!system:float32`` +float64 ``!n!system:float64`` +bool ``!!bool`` +========= =========================================================== + +Apart from these standard library types, NimYAML also supports all enum types +as scalar types. They will be serialized to their string representation. + +Apart from the types listed here and enum tyes, no atomar types are supported. + +Collection Types +================ + +Collection types in Nim are typically generic. As such, they take their +contained types as parameters inside parentheses as explained above. The +following types are supported: + +============ ============================================================ ================================ +Nim type YAML tag YAML structure +============ ============================================================ ================================ +array ``!n!system:array(?;?)`` (first parameter like ``0..5``) sequence +seq ``!n!system:seq(?)`` (or ``!n!nil:seq`` if nil) sequence +set ``!n!system:set(?)`` sequence +Table ``!n!tables:Table(?;?)`` mapping +OrderedTable ``!n!tables:OrderedTable(?;?)`` sequence of single-pair mappings +============ ============================================================ ================================ + +Standard YAML Types +=================== + +NimYAML does not support all types defined in the YAML specification, **not even +those of the failsafe schema**. The reason is that the failsafe schema is +designed for dynamic type systems where a sequence can contain arbitrarily typed +values. This is not fully translatable into a static type system. NimYAML does +support some mechanisms to make working with heterogeneous collection structures +easier, see `Serialization Overview `_. + +Note that because the specification only defines that an implementation *should* +implement the failsafe schema, NimYAML is still compliant; it has valid reasons +not to implement the schema. + +This is a full list of all types defined in the YAML specification or the +`YAML type registry `_. It gives an overview of which +types are supported by NimYAML, which may be supported in the future and which +will never be supported. + +=============== ============================================ +YAML type Status +=============== ============================================ +``!!map`` Cannot be supported +``!!omap`` Cannot be supported +``!!pairs`` Cannot be supported +``!!set`` Cannot be supported +``!!seq`` Cannot be supported +``!!binary`` Currently not supported +``!!bool`` Maps to Nim's ``bool`` type +``!!float`` Not supported (user can choose) +``!!int`` Not supported (user can choose) +``!!merge`` Not supported and unlikely to be implemented +``!!null`` Used for reference types that are ``nil`` +``!!str`` Maps to Nim's ``string`` type +``!!timestamp`` Currently not supported +``!!value`` Not supported and unlikely to be implemented +``!!yaml`` Not supported and unlikely to be implemented +=============== ============================================ + +``!!int`` and ``!!float`` are not supported out of the box to let the user +choose where to map them (for example, ``!!int`` may map to ``int32`` or +``int64``, or the the generic ``int`` whose size is platform-dependent). If one +wants to use ``!!int``or ``!!float``, the process is to create a ``distinct`` +type derived from the desired base type and then set its tag using +``setTagUri``. + +``!!merge`` and ``!!value`` are not supported because the semantics of these +types would make a multi-pass loading process necessary and if one takes the +tag system seriously, ``!!merge`` can only be used with YAML's collection types, +which, as explained above, cannot be supported. \ No newline at end of file diff --git a/doc/serialization.txt b/doc/serialization.txt index f19ba54..6c36916 100644 --- a/doc/serialization.txt +++ b/doc/serialization.txt @@ -36,7 +36,8 @@ Supported Types NimYAML supports a growing number of types of Nim's ``system`` module and standard library, and it also supports user-defined object, tuple and enum types -out of the box. +out of the box. A complete list of explicitly supported types is available in +`Schema `_. **Important**: NimYAML currently does not support polymorphism. This may be added in the future. @@ -105,7 +106,7 @@ possible to dump and load cyclic data structures without further configuration. It is possible for reference types to hold a ``nil`` value, which will be mapped to the ``!!null`` YAML scalar type. -Pointer types are not supported because it seems dangerous to automatically +``ptr`` types are not supported because it seems dangerous to automatically allocate memory which the user must then manually deallocate. User Defined Types @@ -234,7 +235,7 @@ otherwise, it would be loaded as ``nil``. As you might have noticed in the example above, the YAML tag of a ``seq`` depends on its generic type parameter. The same applies to ``Table``. So, a table that maps ``int8`` to string sequences would be presented with the tag -``!nim:tables:Table(nim:system:int8,nim:system:seq(tag:yaml.org,2002:string))``. +``!n!tables:Table(tag:nimyaml.org,2016:int8,tag:nimyaml.org,2016:system:seq(tag:yaml.org,2002:string))``. These tags are generated on the fly based on the types you instantiate ``Table`` or ``seq`` with. @@ -274,6 +275,24 @@ Example: markAsTransient(MyObject, temporary) +Default Values +-------------- + +When you load YAML that has been written by a human, you might want to allow the +user to omit certain fields, which should then be filled with a default value. +You can do that like this: + +.. code-block:: nim + + type MyObject: object + required: string + optional: string + + setDefaultValue(MyObject, optional, "default value") + +Whenever ``MyObject`` now is loaded and the input stream does not contain the +field ``optional``, that field will be set to the value ``"default value"``. + Customize Serialization ======================= diff --git a/doc/style.css b/doc/style.css index ec7d977..513492f 100644 --- a/doc/style.css +++ b/doc/style.css @@ -54,12 +54,15 @@ header span:hover > ul { } header span ul a { - font-size: smaller; - font-family: "Source Code Pro", Menlo, "Courier New", Courier, monospace; padding: 0 10px; line-height: 40px; } +header span ul.monospace a { + font-size: smaller; + font-family: "Source Code Pro", Menlo, "Courier New", Courier, monospace; +} + header a:link, header a:visited { background: inherit; diff --git a/doc/testing.txt b/doc/testing.rst similarity index 100% rename from doc/testing.txt rename to doc/testing.rst diff --git a/nimdoc.cfg b/nimdoc.cfg index fcea78a..0ab14c9 100644 --- a/nimdoc.cfg +++ b/nimdoc.cfg @@ -110,10 +110,16 @@ doc.file = """ Testing Ground Docs: Overview - Serialization + + Serialization + + Modules -