Documentation overhaul.

* submodule docs are now located at /api/yaml/<name>.html instead of
   /api/<name>.html.
 * can now simply link to symbols in other modules.
 * fixed some links
 * fixed quickstart example list
 * added quickstart item displaying the new style api
 * fixed a bug in presenter that doubled a line in literal/flow style

doc/api.txt

@@ -2,6 +2,12 @@
 API Overview
 ============
 
+.. importdoc::
+  api/yaml/loading.nim, api/yaml/dumping.nim, api/yaml/annotations.nim,
+  api/yaml/taglib.nim, api/yaml/style.nim, api/yaml/dom.nim, api/yaml/tojson.nim,
+  api/yaml/parser.nim, api/yaml/presenter.nim, api/yaml/data.nim,
+  api/yaml/stream.nim  
+
 Introduction
 ============
 
@@ -24,19 +30,18 @@ processing pipeline. The items and terminology YAML defines is shown in
 Intermediate Representation
 ===========================
 
-The base of all YAML processing with NimYAML is the
-`YamlStream <api/stream.html#YamlStream>`_. This is basically an iterator over
-`Event <api/data.html#Event>`_ objects. Every proc that
-represents a single stage of the loading or dumping process will either take a
-``YamlStream`` as input or return a ``YamlStream``. Procs that implement the
-whole process in one step hide the ``YamlStream`` from the user. Every proc that
-returns a ``YamlStream`` guarantees that this stream is well-formed according to
-the YAML specification.
+The base of all YAML processing with NimYAML is the `YamlStream`_. This is an
+iterator over `Event`_ objects. Every proc that represents a single stage of
+the loading or dumping process will either take a ``YamlStream`` as input or
+return a ``YamlStream``. Procs that implement the whole process in one step
+hide the ``YamlStream`` from the caller. Every proc that returns a
+``YamlStream`` guarantees that this stream is well-formed according to the
+YAML specification.
 
 This stream-oriented API can efficiently be used to parse large amounts of data.
 The drawback is that errors in the input are only discovered while processing
 the ``YamlStream``. If the ``YamlStream`` encounters an exception while
-producing the next event, it will throw a ``YamlStreamError`` which contains the
+producing the next event, it will throw a `YamlStreamError`_ which contains the
 original exception as ``parent``. The caller should know which exceptions are
 possible as parents of ``YamlStream`` because they know the source of the
 ``YamlStream`` they provided.
@@ -44,21 +49,19 @@ possible as parents of ``YamlStream`` because they know the source of the
 Loading YAML
 ============
 
-If you want to load YAML character data directly into a native Nim variable, you
-can use `load <api/loading.html#load%2C%2CK>`_. This is the easiest and
-recommended way to load YAML data. This section gives an overview about how
-``load`` is implemented. It is absolutely possible to reimplement the loading
-step using the low-level API.
+If you want to load YAML character data directly into a native Nim variable,
+you can use `load`_. This is the easiest and recommended way to load YAML
+data. This section gives an overview about how ``load`` is implemented. It is
+absolutely possible to reimplement the loading step using the low-level API.
 
-For parsing, a `YamlParser <api/parser.html#YamlParser>`_ object is needed.
-This object stores some state while parsing that may be useful for error
-reporting to the user. The `parse <api/parser.html#parse,YamlParser,Stream>`_
+For parsing, a `YamlParser`_ object is needed. This object stores some state
+while parsing that may be useful for error reporting to the user. The `parse`_
 proc implements the YAML processing step of the same name. All syntax errors in
 the input character stream are processed by ``parse``, which will raise a
-``YamlParserError`` if it encounters a syntax error.
+`YamlParserError`_ if it encounters a syntax error.
 
 Transforming a ``YamlStream`` to a native YAML object is done via
-``construct``. It skips the ``compose`` step for efficiency reasons. As Nim is
+`construct`_. It skips the **compose** step for efficiency reasons. As Nim is
 statically typed, you have to know the target type when you write your loading
 code. This is different from YAML APIs of dynamically typed languages. If you
 cannot know the type of your YAML input at compile time, you have to manually
@@ -67,43 +70,40 @@ process the ``YamlStream`` to serve your needs.
 Dumping YAML
 ============
 
-Dumping is preferredly done with
-`dump <api/dumping.html#dump%2CDumper%2CK>`_,
-which serializes a native Nim value to a character stream. As with ``load``,
-the following paragraph describes how ``dump`` is implemented using the
-low-level API.
+Dumping is preferably done with `dump`_, which serializes a native Nim value
+to a character stream. As with ``load``, this section describes how
+``dump`` is implemented using the low-level API.
 
-A Nim value is transformed into a ``YamlStream`` with
-`represent <api/native.html#represent%2CT%2CSerializationOptions>`_.
-Depending on the ``AnchorStyle`` you specify in the given ``SerializationOptions``, this will transform ``ref``
-variables with multiple instances into anchored elements and aliases (for
-``asTidy`` and ``asAlways``) or write the same element into all places it
-occurs (for ``asNone``). Be aware that if you use ``asNone``, the value you
-serialize might not round-trip.
+A Nim value is transformed into a ``YamlStream`` with `represent`_.
+Depending on the `AnchorStyle`_ you specify in the `SerializationOptions`_ of
+your `Dumper`_, this will transform ``ref`` variables with multiple instances
+into anchored elements and aliases (for ``asTidy`` and ``asAlways``) or write
+the same element into all places it occurs (for ``asNone``). Be aware that if
+you use ``asNone``, the value you serialize might not round-trip.
 
 Transforming a ``YamlStream`` into YAML character data is done with
-`present <api/presenter.html#present%2CYamlStream%2CStream%2CPresentationOptions>`_.
-You can choose from multiple presentation styles. ``psJson`` is not able to
-process some features of ``YamlStream`` s, the other styles support all features
-and are guaranteed to round-trip to the same ``YamlStream`` if you parse the
-generated YAML character stream again.
+`present`_ which is customized by your Dumper's `PresentationOptions`_. The
+Dumper provides multiple presets, for example the `jsonDumper`_ dumps your
+value in JSON style (which is also valid YAML since YAML is a superset of
+JSON).
 
 The Document Object Model
 =========================
 
 Unlike XML, YAML does not define an official *document object model*. However,
 if you cannot or do not want to load a YAML input stream to native Nim types,
-you can load it into the predefined type `YamlNode <api/dom.html#YamlNode>`_.
-You can also use this type inside your native types to deserialize parts of the
-YAML input into it. Likewise, you can serialize a ``YamlNode`` into YAML. You
-can use this to preserve parts of YAML data you do not wish to or cannot fully
-deserialize.
+you can load it into the predefined type `YamlNode`_. You can also use this
+type inside your native types to deserialize parts of the YAML input into it.
+Likewise, you can serialize a ``YamlNode`` into YAML. You can use this to
+preserve parts of YAML data you do not wish to or cannot fully deserialize.
 
-A ``YamlNode`` preserves its given tag and the tags of any child nodes. However,
-anchors will be resolved during loading and re-added during serialization. It is
-allowed for a ``YamlNode`` to occur multiple times within source/target root
-object, in which case it will be serialized once and referred to afterwards via
-aliases.
+A ``YamlNode`` preserves its given tag and the tags of any child nodes, and
+also its style (which means, unless you override style with Dumper options,
+the node will be serialized with the same style it had originally).
+However, anchors will be resolved during loading and re-added during
+serialization. It is possible for a ``YamlNode`` to occur multiple times within
+source/target root object, in which case it will be serialized once and
+referred to afterwards via aliases.
 
 ``YamlNode`` is allocated on the heap and using it will be slower and consume
 more memory than deserializing into native types.

doc/index.txt

@@ -16,6 +16,9 @@ install it with `Nimble <https://github.com/nim-lang/nimble>`_:
 .. code-block:: bash
   nimble install yaml
 
+You can find a conceptual overview of the library `here <api.html>`_ and an 
+overview over the library's API `here <api/yaml.html>`_.
+
 NimYAML 2.x
 ===========
 

doc/rstPreproc.nim

@@ -17,7 +17,7 @@
 ## available as source files for automatic testing. This way, we can make sure
 ## that the code in the docs actually works.
 
-import parseopt, streams, tables, strutils, os, options, algorithm
+import parseopt, streams, tables, strutils, unicode, os, options, algorithm
 
 var
   infile = ""
@@ -62,7 +62,7 @@ var tmpOut = newFileStream(path.get(), fmWrite)
 proc append(s: string) =
   tmpOut.writeLine(s)
 
-const headingChars = ['=', '-', '`', ':', '\'']
+const headingChars = ['=', '-', '^', ':', '\'']
 
 proc outputExamples(curPath: string, level: int = 0) =
   let titlePath = curPath / "title"

doc/snippets/quickstart/02/00/00-code.nim

@@ -0,0 +1,27 @@
+import yaml, yaml/style, streams
+
+type
+  Strings = object
+    first {.scalar: ssSingleQuoted.}: string
+    second {.scalar: ssLiteral.}: string
+    third {.scalar: ssDoubleQuoted.}: string
+  
+  Numbers {.collection: csFlow.} = object
+    start, stop: int32
+  
+  Root = object
+    strings: Strings
+    numbers: Numbers
+    blockNumbers {.collection: csBlock.}: Numbers
+
+var root = Root(
+  strings: Strings(
+    first: "foo", second: "bar\n", third: "baz"
+  ),
+  numbers: Numbers(start: 0, stop: 23),
+  blockNumbers: Numbers(start: 23, stop: 42)
+)
+
+var s = newFileStream("out.yaml", fmWrite)
+Dumper().dump(root, s)
+s.close()

doc/snippets/quickstart/02/00/01-out.yaml

@@ -0,0 +1,12 @@
+strings:
+  first: 'foo'
+  second: |
+    bar
+  third: "baz"
+numbers: {
+    start: 0,
+    stop: 23
+  }
+blockNumbers:
+  start: 23
+  stop: 42

doc/snippets/quickstart/02/00/title

@@ -0,0 +1 @@
+… via style pragmas

doc/snippets/quickstart/02/01/title

@@ -0,0 +1 @@
+… by customizing the Dumper

doc/snippets/quickstart/08/00/title

@@ -1 +1 @@
-… With variant objects
+… with variant objects

doc/snippets/quickstart/08/01/title

@@ -1 +1 @@
-… With the Sequential API
+… with the YamlStream API

flake.nix

@@ -54,6 +54,11 @@
               };
               configurePhase = ''
                 mkdir -p docout/api
+                for srcFile in yaml/*.nim; do
+                  echo "generate index for $srcFile"
+                  ${pkgs.nim2}/bin/nim doc --index:only --outdir:docout/api/yaml $srcFile
+                done
+
                 (
                   cd doc
                   for rstFile in *.rst; do
@@ -62,16 +67,19 @@
                   ${pkgs.nim2}/bin/nim c --nimcache:.cache rstPreproc
                   for txtFile in *.txt; do
                     ./rstPreproc -o:tmp.rst $txtFile
-                    ${pkgs.nim2}/bin/nim rst2html -o:../docout/''${txtFile%.txt}.html tmp.rst
+                    fn=$(basename -- "$txtFile")
+                    ${pkgs.nim2}/bin/nim rst2html -o:../docout/''${fn%.txt}.html tmp.rst
+                    rm tmp.rst
                   done
                   cp docutils.css style.css processing.svg github-mark-white.svg ../docout
                 )
-                ${pkgs.nim2}/bin/nim doc2 -o:docout/api/yaml.html --docSeeSrcUrl:https://github.com/flyx/NimYAML/blob/${
+
+                ${pkgs.nim2}/bin/nim doc -o:docout/api/yaml.html --docSeeSrcUrl:https://github.com/flyx/NimYAML/blob/${
                   self.rev or "master"
                 } yaml
                 for srcFile in yaml/*.nim; do
                   bn=''${srcFile#yaml/}
-                  ${pkgs.nim2}/bin/nim doc2 -o:docout/api/''${bn%.nim}.html --docSeeSrcUrl:https://github.com/flyx/NimYAML/blob/yaml/${
+                  ${pkgs.nim2}/bin/nim doc -o:docout/api/yaml/''${bn%.nim}.html --docSeeSrcUrl:https://github.com/flyx/NimYAML/blob/yaml/${
                     self.rev or "master"
                   } $srcFile
                 done

nimdoc.cfg

@@ -120,19 +120,19 @@ doc.file = """
     <a href="/api/yaml.html">Modules</a>
     <ul class="monospace">
       <li><a href="/api/yaml.html">yaml</a></li>
-      <li><a href="/api/annotations.html">yaml/annotations</a></li>
-      <li><a href="/api/data.html">yaml/data</a></li>
-      <li><a href="/api/dom.html">yaml/dom</a></li>
-      <li><a href="/api/dumping.html">yaml/dumping</a></li>
-      <li><a href="/api/hints.html">yaml/hints</a></li>
-      <li><a href="/api/loading.html">yaml/loading</a></li>
-      <li><a href="/api/native.html">yaml/native</a></li>
-      <li><a href="/api/parser.html">yaml/parser</a></li>
-      <li><a href="/api/presenter.html">yaml/presenter</a></li>
-      <li><a href="/api/stream.html">yaml/stream</a></li>
-      <li><a href="/api/style.html">yaml/style</a></li>
-      <li><a href="/api/taglib.html">yaml/taglib</a></li>
-      <li><a href="/api/tojson.html">yaml/tojson</a></li>
+      <li><a href="/api/yaml/annotations.html">yaml/annotations</a></li>
+      <li><a href="/api/yaml/data.html">yaml/data</a></li>
+      <li><a href="/api/yaml/dom.html">yaml/dom</a></li>
+      <li><a href="/api/yaml/dumping.html">yaml/dumping</a></li>
+      <li><a href="/api/yaml/hints.html">yaml/hints</a></li>
+      <li><a href="/api/yaml/loading.html">yaml/loading</a></li>
+      <li><a href="/api/yaml/native.html">yaml/native</a></li>
+      <li><a href="/api/yaml/parser.html">yaml/parser</a></li>
+      <li><a href="/api/yaml/presenter.html">yaml/presenter</a></li>
+      <li><a href="/api/yaml/stream.html">yaml/stream</a></li>
+      <li><a href="/api/yaml/style.html">yaml/style</a></li>
+      <li><a href="/api/yaml/taglib.html">yaml/taglib</a></li>
+      <li><a href="/api/yaml/tojson.html">yaml/tojson</a></li>
     </ul>
   </span>
   <span>

yaml.nim

@@ -4,12 +4,17 @@
 #    See the file "copying.txt", included in this
 #    distribution, for details about the copyright.
 
-## This is the parent module of NimYAML, a package that provides facilities to
+## .. importdoc::
+##   yaml/loading.nim, yaml/dumping.nim, yaml/annotations.nim, yaml/taglib.nim,
+##   yaml/style.nim, yaml/dom.nim, yaml/tojson.nim,
+##   yaml/parser.nim, yaml/presenter.nim, yaml/data.nim, yaml/stream.nim  
+##
+## This is the root module of NimYAML, a package that provides facilities to
 ## generate and interpret `YAML <http://yaml.org>`_ character streams. Importing
 ## this package will import NimYAML's high level loading & dumping API.
 ## Additional APIs must be imported explicitly.
 ##
-## There is no code in this package, all functionality is available from the
+## There is no code in this package, all functionality is available via the
 ## exported sub-packages. You can import parts of the API by importing
 ## certain sub-packages only.
 ##
@@ -20,26 +25,38 @@
 ##
 ##   import yaml
 ##   # or alternatively:
-##   import yaml / [loading, dumping, annotations]
+##   import yaml / [loading, dumping, annotations, taglib, dom]
 ##
 ## Enables you to load YAML data directly into native Nim types and reversely
 ## dump native Nim types into YAML documents. This API corresponds to the full
 ## **Load** / **Dump** process as defined in the
 ## `YAML specification <https://yaml.org/spec/1.2.2/#31-processes>`_.
 ##
+## The module `module yaml/loading`_ provides the `load`_ and `loadAs`_ procs
+## which load a single YAML document into a native Nim value.
+##
+## The `module yaml/dumping`_ provides the `Dumper`_ object together with its
+## `dump`_ methods that serialize a given Nim value into YAML.
+##
+## The `module yaml/annotations`_ provides various pragmas that allow you to
+## define how certain aspects of your types are to be serialized, e.g. whether
+## ``Optional`` fields may be omitted.
+##
+## The `module yaml/taglib`_ provides facilities that customize the YAML tags
+## that are generated for your types. The primary usage for tags in the context
+## of NimYAML is to define the type of a value in a heterogeneous collection node.
+##
 ## The following additional APIs extend the basic high-level API:
 ##
 ## DOM API
 ## -------
 ##
-## .. code-block::
+## *Also exported by default, no import necessary*
 ##
-##   import yaml / [loading, dumping, dom]
-##
-## Enables you to load YAML into ``YamlNode`` objects and dump those back into
-## YAML. This gives you a structured view of your YAML stream. The DOM API
-## provides the types and their handling, which can then be used via the
-## loading & dumping API.
+## The `module yaml/dom`_ enables you to load YAML into `YamlNode`_ objects and
+## dump those back into YAML. This gives you a structured view of your YAML
+## stream. The DOM API provides the types and their handling, which can then
+## be used via the loading & dumping API.
 ##
 ## You can use ``YamlNode`` objects inside other objects to hold subtrees of
 ## the input YAML, or you can load the whole YAML into a ``YamlNode``.
@@ -48,36 +65,40 @@
 ## defined in the
 ## `YAML specification <https://yaml.org/spec/1.2.2/#31-processes>`_.
 ##
+## Style API
+## ---------
+##
+## .. code-block::
+##
+##   # needs explicit import to use:
+##   import yaml/style
+##
+## The `module yaml/style`_ lets you define the preferred YAML node style of
+## your objects and fields, giving you a greater control over how your
+## generated YAML looks.
+##
 ## JSON API
 ## --------
 ##
 ## .. code-block::
 ##
+##   # needs explicit import to use:
 ##   import yaml/tojson
 ##
-## Enables you to load YAML input into the stdlib's ``JsonNode`` structure.
-## This can be useful for other libraries that expect JSON input. Note that
-## the loading & dumping API is able to read & write JSON files, you don't need
-## the JSON API for that.
+## The `module yaml/tojson`_ enables you to load YAML input into the stdlib's
+## ``JsonNode`` structure. This can be useful for other libraries that expect
+## JSON input. Mind that the loading & dumping API is able to read & write
+## JSON files (since YAML  is a superset of JSON), you don't need the JSON
+## API for that.
 ##
-## Taglib API
-## ----------
-##
-## .. code-block::
-##
-##   import yaml/taglib
-##
-## This API allows you to customize the YAML tags used for the Nim types you're
-## serializing. The primary usage for tags in the context of NimYAML is to
-## define the type of a value in a heterogeneous collection node.
 ##
 ## Low Level Event Handling
 ## ========================
 ##
 ## NimYAML exposes lower-level APIs that allow you to access the different
 ## steps used for YAML loading & dumping. These APIs have at their core a
-## ``YamlStream`` which is an object that supplies ``Event``s. This corresponds
-## to the **Serialization (Event Tree)** stage defined in the
+## `YamlStream`_ which is an object that supplies a stream of `Event`_.
+## This corresponds to the **Serialization (Event Tree)** stage defined in the
 ## `YAML specification <https://yaml.org/spec/1.2.2/#31-processes>`_.
 ##
 ## Parsing & Presenting API
@@ -85,22 +106,24 @@
 ##
 ## .. code-block::
 ##
+##   # needs explicit import to use:
 ##   import yaml / [parser, presenter, stream, data]
 ##
-## Provides ``parse``, a proc that feeds a ``YamlStream`` from YAML input,
-## and ``present``, which consumes a ``YamlStream`` and writes out YAML. 
-## You can use a ``BufferYamlStream`` to supply manually generated events.
+## Provides `parse`_, a proc that feeds a ``YamlStream`` from YAML input,
+## and `present`_, which consumes a ``YamlStream`` and writes out YAML. 
+## You can use a `BufferYamlStream`_ to supply manually generated events.
 ##
 ## Native API
 ## ----------
 ##
 ## .. code-block::
 ##
+##   # needs explicit import to use:
 ##   import yaml/native
 ##
 ## This part of the API takes care of generating Nim values from a
-## ``YamlStream`` via ``construct``, and transforming them back into a
-## ``YamlStream`` via ``represent``. This complements the Event API.
+## ``YamlStream`` via `construct`_, and transforming them back into a
+## ``YamlStream`` via `represent`_. This complements the Event API.
 ##
 ## Typically, you'd only access this API when defining custom constructors
 ## and representers.
@@ -111,6 +134,7 @@
 ##
 ## .. code-block::
 ##
+##   # needs explicit import to use:
 ##   import yaml/hints
 ##
 ## Provides type guessing, i.e. figuring out which type would be appropriate

yaml/presenter.nim

@@ -188,6 +188,8 @@ proc inspect(
   ## For example, the presenter is currently unable to emit multi-line plain
   ## scalars, therefore multi-line string will never yield ssPlain. Similarly,
   ## ssFolded will never be returned if there are more-indented lines.
+  lines.setLen(0)
+  words.setLen(0)
   var
     inLine = false
     inWord = false