# Notes on automatic reference page generation

Main takeaway point: ref page generation from the spec sources is ready to
go. It can be tweaked in lots of small ways, but as it stands it's a huge
improvement over the current ref pages and should be adopted ASAP.

## Approach

This is done with some Python scripts that rely on the rigid structure
we've enforced on command and structure definitions in the spec. Everything
from the paragraph *preceding* the protos/structs include:: through the
validity:: include is pulled out and then split up into ref page sections.
In some cases, it isn't immediately obvious how to reorder to get all the
content relevant to a single command or struct blocked together in this
fashion, and some simplistic semantic tagging has been added for those cases
where the structural organization isn't appropriate.

The changes to the spec are straightforward, though they appear large in
some places. What's happening is:

  * Validity includes are moved below all the relevant text.
  * Structure definitions are moved after the validity includes for
    commands, so they don't overlap or imbed. This has been done for
    enumerant definitions in some cases, but those are harder to unwind.
  * Blank lines are being removed from parameter / member definition bullet
    lists, so they don't confuse the extraction script.
  * Annotations in the form of asciidoc comments were added where
    they were needed to assist the extraction scripts. These look
    like
    <br>// refBegin vkSomething short description of command for ref page title
    <br>// refEnd vkSomething [list of related ref pages]
  * A bunch of small consistency edits were done to match the style guide.
    These (mostly) aren't directly related to the extraction process, but it
    is an opportune time to do them.
  * Pages that don't have extractable descriptive text in the specification,
    such as Vk*Flags and some Vk*FlagBits types, get automatically generated
    ref pages. These just define the interface and link to other pages using
    that type.

## Things to be done

These are (more or less) in decreasing priority order. Few of them are
difficult.

  * Something is messed up with the individual page generation targets in
    the Makefile, so the genRef.py action isn't a dependency of
    $(MANSOURCES) yet. See
      https://www.cmcrossroads.com/article/rules-multiple-outputs-gnu-make?page=0%2C2
  * Cross-page macro links are broken in the individual ref pages, due to
    using the wrong macros, but are OK in the single-page apispec.html.
    Should be easily fixed.
  * manpdf (single-page PDF ref document) target is currently broken.
  * Turn handle references from sname: to slink: so they link to the right
    place. Consider using an hlink/hname macro instead. Similarly for
    basetype: (which doesn't link, but could now with blink:/btext:). In
    general, most of the ?name: macros can and should turn into ?link:,
    aside from those directly in the section describing that name.
  * Need to tweak asciidoc macros for man pages (manpages.conf).
  * "See Also" links are automatically generated based on the type
    dependencies between pages. This works well, but doesn't include logical
    groupings (all clear commands, all drawing commands, etc.). These can be
    added to the explicit crosslinks in // refEnd statements if desired.
  * Remove autogenerated files from git and enhance the Makefiles to do all
    the scripting work required to generate the ref page sources, vkapi.py,
    etc. at build time.
  * Some problems with a2x not finding NOTE icons from the asciidoc
    distribution (worked around by copying into the man/images directory,
    for now). It would sure be nice if the build were smart enough to use
    the icons distributed with asciidoc, instead of needing separate copies.
  * I'm not sure how to best deal with KHR extension ref pages. We could
    generate them separately in their branches and combine with core pages,
    or generate everything in the wsi_extensions branch - but that will
    leave us with core ref pages including WSI extension material, without
    any obvious indication of why or where. Changes for the KHR extension
    branches will be small once the 1.0 changes are merged in, and I'll
    undertake to do those once the core changes are accepted.
  * Cross-page asciidoc links are broken. This is hard to fix. Probably the
    best thing to do is strip \<\<\>\> links from the source, or perhaps
    generate an HTML link to the live spec.
  * VkResult page needs a better "See Also" section - but not the full list
    of functions returning VkResult.
  * Short descriptions are currently embedded in the spec source, and are
    missing in some cases. Would it make more sense to add them to vk.xml?