Vulkan-Docs/scripts/validitygenerator.py
Jon Leech 5abf83f95d Change log for April 16, 2019 Vulkan 1.1.107 spec update:
* Update release number to 107.

Public Issues:

  * Fix revision date for the `<<VK_AMD_gpu_shader_half_float>>` appendix
    (public issue 617).
  * Make <<synchronization-pipeline-barriers-subpass-self-dependencies,
    subpass self-dependencies>> less restrictive (public issue 777).
  * Fix the `<<VK_EXT_full_screen_exclusive>>` dependency on
    `<<VK_KHR_win32_surface>>` in `vk.xml` (public pull request 849).
  * Remove single-page (`apispec.html`) refpage sub-targets from the
    Makefile `allman` target and the build instructions. The target is still
    present in the Makefile, but we have not been actively maintaining the
    single-page document and do not promise it will work. The full
    Specification and the individual API reference pages are what we support
    and publish at present (public issue 949).

Internal Issues:

  * De-duplicate common valid usage statements shared by multiple commands
    or structures by using asciidoctor includes and dynamically assigning
    part of the valid usage ID based on which command or structure they're
    being applied to (internal issue 779).
  * Add reference pages for constructs not part of the formal API, such as
    platform calling convention macros, and script changes supporting them
    This required suppressing some check_spec_links warning classes in order
    to pass CI, until a more sophisticated fix can be done (internal issue
    888).
  * Change math notation for the elink:VkPrimitiveTopology descriptions to
    use short forms `v` and `p` instead of `vertex` and `primitive`,
    increasing legibility (internal issue 1611).
  * Rewrite generated file includes relative to a globally specified path,
    fixing some issues with refpage generation (internal issue 1630).
  * Update contributor list for `<<VK_EXT_calibrated_timestamps>>`.
  * Fix use of pathlin in `scripts/generator.py` so the script will work on
    Windows under Python 3.5 (internal merge request 3107).
  * Add missing conditionals around the
    <<descriptorsets-accelerationstructure, Acceleration Structure>>
    section (internal merge request 3108).
  * More script synchronization with OpenXR spec repository (internal merge
    request 3109).
  * Mark the `<<VK_AMD_gpu_shader_half_float>>` and
    `<<VK_AMD_gpu_shader_int16>>` extensions as deprecated in `vk.xml` and
    the corresponding extension appendices (internal merge request 3112).

New Extensions:

  * `<<VK_EXT_headless_surface>>`
2019-04-16 05:19:43 -07:00

1474 lines
60 KiB
Python

#!/usr/bin/python3 -i
#
# Copyright (c) 2013-2019 The Khronos Group Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import re
import sys
from collections import OrderedDict, namedtuple
from functools import reduce
from generator import OutputGenerator, write
class UnhandledCaseError(RuntimeError):
def __init__(self):
super().__init__('Got a case in the validity generator that we have not explicitly handled.')
def _genericIterateIntersection(a, b):
"""Iterate through all elements in a that are also in b.
Somewhat like a set's intersection(),
but not type-specific so it can work with OrderedDicts, etc.
It also returns a generator instead of a set,
so you can pick what container type you'd like,
if any.
"""
return (x for x in a if x in b)
def _make_ordered_dict(gen):
"""Make an ordered dict (with None as the values) from a generator."""
return OrderedDict(((x, None) for x in gen))
def _orderedDictIntersection(a, b):
return _make_ordered_dict(_genericIterateIntersection(a, b))
def _genericIsDisjoint(a, b):
"""Return true if nothing in a is also in b.
Like a set's is_disjoint(),
but not type-specific so it can work with OrderedDicts, etc.
"""
for _ in _genericIterateIntersection(a, b):
return False
# if we never enter the loop...
return True
class LengthEntry:
"""An entry in a (comma-separated) len attribute"""
def __init__(self, val):
self.other_param_name = None
self.null_terminated = False
self.number = None
if val == 'null-terminated':
self.null_terminated = True
return
if val.isdigit():
self.number = int(val)
return
# Must be another param name.
self.other_param_name = val
@staticmethod
def parse_len_from_param(param):
"""Get a list of LengthEntry."""
return [LengthEntry(elt) for elt in param.get('len').split(',')]
def _getElemName(elem, default=None):
"""Get the name associated with an element, either a name child or name attribute."""
name_elem = elem.find('name')
if name_elem is not None:
return name_elem.text
# Fallback if there is no child.
return elem.get('name', default)
def _getElemType(elem, default=None):
"""Get the type associated with an element, either a type child or type attribute."""
type_elem = elem.find('type')
if type_elem is not None:
return type_elem.text
# Fallback if there is no child.
return elem.get('type', default)
def _findNamedElem(elems, name):
"""Traverse a collection of elements with 'name' nodes or attributes, looking for and returning one with the right name.
NOTE: Many places where this is used might be better served by changing to a dictionary.
"""
for elem in elems:
if _getElemName(elem) == name:
return elem
return None
def _findNamedObject(collection, name):
"""Traverse a collection of elements with 'name' attributes, looking for and returning one with the right name.
NOTE: Many places where this is used might be better served by changing to a dictionary.
"""
for elt in collection:
if elt.name == name:
return elt
return None
class ValidityCollection:
"""Combines validity for a single entity."""
def __init__(self, entity, conventions):
self.entity = entity
self.conventions = conventions
self.lines = []
def possiblyAddExtensionRequirement(self, entity_data, entity_preface):
"""Add an extension-related validity statement if required.
entity_preface is a string that goes between "must be enabled prior to "
and the name of the entity, and normally ends in a macro.
For instance, might be "calling flink:" for a function.
"""
if entity_data.extensionname:
msg = 'The {} extension must: be enabled prior to {}{}'.format(
self.conventions.formatExtension(entity_data.extensionname), entity_preface, self.entity)
self.addValidityEntry(msg, 'extension', 'notenabled')
def addValidityEntry(self, msg, *args):
"""Add a validity entry, optionally with a VUID anchor.
If any trailing arguments are supplied,
an anchor is generated by concatenating them with dashes
at the end of the VUID anchor name.
"""
parts = ['*']
if args:
parts.append('[[{}]]'.format(
'-'.join(['VUID', self.entity] + list(args))))
parts.append(msg)
self.lines.append(' '.join(parts))
def addText(self, msg):
"""Add already formatted validity text."""
if not msg:
return
msg = msg.rstrip()
if not msg:
return
self.lines.append(msg)
@property
def text(self):
"""Access validity statements as a single string."""
if not self.lines:
return None
return '\n'.join(self.lines) + '\n'
class ValidityOutputGenerator(OutputGenerator):
"""ValidityOutputGenerator - subclass of OutputGenerator.
Generates AsciiDoc includes of valid usage information, for reference
pages and the Vulkan specification. Similar to DocOutputGenerator.
---- methods ----
ValidityOutputGenerator(errFile, warnFile, diagFile) - args as for
OutputGenerator. Defines additional internal state.
---- methods overriding base class ----
beginFile(genOpts)
endFile()
beginFeature(interface, emit)
endFeature()
genCmd(cmdinfo)
"""
def __init__(self,
errFile = sys.stderr,
warnFile = sys.stderr,
diagFile = sys.stdout):
OutputGenerator.__init__(self, errFile, warnFile, diagFile)
@property
def null(self):
"""Preferred spelling of NULL.
Delegates to the object implementing ConventionsBase.
"""
return self.conventions.null
@property
def structtype_member_name(self):
"""Return name of the structure type member.
Delegates to the object implementing ConventionsBase.
"""
return self.conventions.structtype_member_name
@property
def nextpointer_member_name(self):
"""Return name of the structure pointer chain member.
Delegates to the object implementing ConventionsBase.
"""
return self.conventions.nextpointer_member_name
def makeProseList(self, elements, connective='and'):
"""Make a (comma-separated) list for use in prose.
Adds a connective (by default, 'and')
before the last element if there are more than 1.
Delegates to the object implementing ConventionsBase.
"""
return self.conventions.makeProseList(elements, connective)
def makeValidityCollection(self, entity_name):
"""Create a ValidityCollection object, passing along our Conventions."""
return ValidityCollection(entity_name, self.conventions)
def beginFile(self, genOpts):
if not genOpts.conventions:
raise RuntimeError('Must specify conventions object to generator options')
self.conventions = genOpts.conventions
# Vulkan says 'must: be a valid pointer' a lot, OpenXR just says
# 'must: be a pointer'.
self.valid_pointer_text = ' '.join((self.conventions.valid_pointer_prefix, 'pointer'))
OutputGenerator.beginFile(self, genOpts)
def endFile(self):
OutputGenerator.endFile(self)
def beginFeature(self, interface, emit):
# Start processing in superclass
OutputGenerator.beginFeature(self, interface, emit)
self.currentExtension = interface.get('name')
def endFeature(self):
# Finish processing in superclass
OutputGenerator.endFeature(self)
@property
def struct_macro(self):
"""Get the appropriate format macro for a structure."""
# delegate to conventions
return self.conventions.struct_macro
def makeStructName(self, name):
"""Prepend the appropriate format macro for a structure to a structure type name."""
# delegate to conventions
return self.conventions.makeStructName(name)
def makeParameterName(self, name):
"""Prepend the appropriate format macro for a parameter/member to a parameter name."""
return 'pname:' + name
def makeBaseTypeName(self, name):
"""Prepend the appropriate format macro for a 'base type' to a type name."""
return 'basetype:' + name
def makeEnumerationName(self, name):
"""Prepend the appropriate format macro for an enumeration type to a enum type name."""
return 'elink:' + name
def makeFuncPointerName(self, name):
"""Prepend the appropriate format macro for a function pointer type to a type name."""
return 'tlink:' + name
def makeExternalTypeName(self, name):
"""Prepend the appropriate format macro for an external type like uint32_t to a type name."""
# delegate to conventions
return self.conventions.makeExternalTypeName(name)
def makeEnumerantName(self, name):
"""Prepend the appropriate format macro for an enumerate (value) to a enum value name."""
return 'ename:' + name
def makeAnchor(self, blockname, pname, category):
"""Create a unique namespaced Valid Usage anchor name.
blockname - command or structure
pname - parameter or member (may be None)
category - distinct implicit VU type
"""
# For debugging
# return '* '
if pname is not None:
return '* [[VUID-%s-%s-%s]] ' % (blockname, pname, category)
return '* [[VUID-%s-%s]] ' % (blockname, category)
def writeInclude(self, directory, basename, validity, threadsafety, commandpropertiesentry, successcodes, errorcodes):
"""Generate an include file.
directory - subdirectory to put file in (relative pathname)
basename - base name of the file
contents - contents of the file (Asciidoc boilerplate aside)
"""
# Create subdirectory, if needed
directory = self.genOpts.directory + '/' + directory
self.makeDir(directory)
# Create validity file
filename = directory + '/' + basename + '.txt'
self.logMsg('diag', '# Generating include file:', filename)
fp = open(filename, 'w', encoding='utf-8')
# Asciidoc anchor
write(self.conventions.warning_comment, file=fp)
# Valid Usage
if validity:
write('.Valid Usage (Implicit)', file=fp)
write('****', file=fp)
write(validity, file=fp, end='')
write('****', file=fp)
write('', file=fp)
# Host Synchronization
if threadsafety:
write('.Host Synchronization', file=fp)
write('****', file=fp)
write(threadsafety, file=fp, end='')
write('****', file=fp)
write('', file=fp)
# Command Properties - contained within a block, to avoid table numbering
if commandpropertiesentry:
write('.Command Properties', file=fp)
write('****', file=fp)
write('[options="header", width="100%"]', file=fp)
write('|====', file=fp)
write('|<<VkCommandBufferLevel,Command Buffer Levels>>|<<vkCmdBeginRenderPass,Render Pass Scope>>|<<VkQueueFlagBits,Supported Queue Types>>|<<synchronization-pipeline-stages-types,Pipeline Type>>', file=fp)
write(commandpropertiesentry, file=fp)
write('|====', file=fp)
write('****', file=fp)
write('', file=fp)
# Success Codes - contained within a block, to avoid table numbering
has_success = (successcodes is not None and len(successcodes) > 0)
has_errors = (errorcodes is not None and len(errorcodes) > 0)
if has_success or has_errors:
write('.Return Codes', file=fp)
write('****', file=fp)
if has_success:
write('ifndef::doctype-manpage[]', file=fp)
write('<<fundamentals-successcodes,Success>>::', file=fp)
write('endif::doctype-manpage[]', file=fp)
write('ifdef::doctype-manpage[]', file=fp)
write('On success, this command returns::', file=fp)
write('endif::doctype-manpage[]', file=fp)
write(successcodes, file=fp)
if has_errors:
write('ifndef::doctype-manpage[]', file=fp)
write('<<fundamentals-errorcodes,Failure>>::', file=fp)
write('endif::doctype-manpage[]', file=fp)
write('ifdef::doctype-manpage[]', file=fp)
write('On failure, this command returns::', file=fp)
write('endif::doctype-manpage[]', file=fp)
write(errorcodes, file=fp)
write('****', file=fp)
write('', file=fp)
fp.close()
def paramIsPointer(self, param):
"""Check if the parameter passed in is a pointer."""
tail = param.find('type').tail
return tail is not None and '*' in tail
def paramIsStaticArray(self, param):
"""Check if the parameter passed in is a static array."""
tail = param.find('name').tail
return tail and tail[0] == '['
def staticArrayLength(self, param):
"""Get the length of a parameter that's been identified as a static array."""
paramenumsize = param.find('enum')
if paramenumsize is not None:
return paramenumsize.text
return param.find('name').tail[1:-1]
def paramIsArray(self, param):
"""Check if the parameter passed in is a pointer to an array."""
return param.get('len') is not None
def getHandleParent(self, typename):
"""Get the parent of a handle object."""
types = self.registry.tree.findall("types/type")
elem = _findNamedElem(types, typename)
if elem:
return elem.get('parent')
return None
def iterateHandleAncestors(self, typename):
"""Iterate through the ancestors of a handle type."""
current = self.getHandleParent(typename)
while current is not None:
yield current
current = self.getHandleParent(current)
def getHandleAncestors(self, typename):
"""Get the ancestors of a handle object."""
ancestors = []
current = typename
while True:
current = self.getHandleParent(current)
if current is None:
return ancestors
ancestors.append(current)
def getHandleDispatchableAncestors(self, typename):
"""Get the ancestors of a handle object."""
ancestors = []
current = typename
while True:
current = self.getHandleParent(current)
if current is None:
return ancestors
if self.isHandleTypeDispatchable(current):
ancestors.append(current)
def isHandleTypeDispatchable(self, handlename):
"""Check if a parent object is dispatchable or not."""
handle = self.registry.tree.find("types/type/[name='" + handlename + "'][@category='handle']")
if handle is not None and _getElemType(handle) == 'VK_DEFINE_HANDLE':
return True
else:
return False
def isHandleOptional(self, param, params):
# Simple, if it's optional, return true
if param.get('optional') is not None:
return True
# If no validity is being generated, it usually means that validity is complex and not absolute, so let's say yes.
if param.get('noautovalidity') is not None:
return True
# If the parameter is an array and we haven't already returned, find out if any of the len parameters are optional
if self.paramIsArray(param):
for length in LengthEntry.parse_len_from_param(param):
if not length.other_param_name:
# don't care about constants or "null-terminated"
continue
other_param = _findNamedElem(params, length.other_param_name)
if other_param is None:
self.logMsg('warn', length.other_param_name, 'is listed as a length for parameter', param, 'but no such parameter exists')
if other_param and other_param.get('optional'):
return True
return False
def getTypeCategory(self, typename):
"""Get the category of a type."""
types = self.registry.tree.findall("types/type")
elem = _findNamedElem(types, typename)
if elem is not None:
return elem.get('category')
return None
def makeAsciiDocPreChunk(self, blockname, param, params):
"""Make a chunk of text for the end of a parameter if it is an array."""
param_name = _getElemName(param)
paramtype = _getElemType(param)
# General pre-amble. Check optionality and add stuff.
asciidoc = self.makeAnchor(blockname, param_name, 'parameter')
optionallengths = []
if self.paramIsStaticArray(param):
asciidoc += 'Any given element of '
elif self.paramIsArray(param):
# Find all the parameters that are called out as optional, so we can document that they might be zero, and the array may be ignored
lengths = LengthEntry.parse_len_from_param(param)
for length in lengths:
if not length.other_param_name:
# don't care about constants or "null-terminated"
continue
other_param = _findNamedElem(params, length.other_param_name)
if other_param is not None and other_param.get('optional') is not None:
if self.paramIsPointer(other_param):
optionallengths.append(
'the value referenced by ' + self.makeParameterName(length.other_param_name))
else:
optionallengths.append(
self.makeParameterName(length.other_param_name))
# Document that these arrays may be ignored if any of the length values are 0
if optionallengths or param.get('optional') is not None:
asciidoc += 'If '
if optionallengths:
# TODO switch to commented line, but this introduces cosmetic changes.
#asciidoc += self.makeProseList(optionallengths, 'or')
asciidoc += ', or '.join(optionallengths)
if len(optionallengths) == 1:
asciidoc += ' is '
else:
asciidoc += ' are '
asciidoc += 'not `0`, '
if optionallengths and param.get('optional') is not None:
asciidoc += 'and '
if param.get('optional') is not None:
asciidoc += self.makeParameterName(param_name)
asciidoc += ' is not `NULL`, '
elif param.get('optional'):
# Don't generate this stub for bitflags
if self.getTypeCategory(paramtype) != 'bitmask':
if param.get('optional').split(',')[0] == 'true':
asciidoc += 'If '
asciidoc += self.makeParameterName(param_name)
asciidoc += ' is not '
if self.paramIsArray(param) or self.paramIsPointer(param) or self.isHandleTypeDispatchable(paramtype):
asciidoc += self.null
elif self.getTypeCategory(paramtype) == 'handle':
asciidoc += 'dlink:' + self.conventions.api_prefix + 'NULL_HANDLE'
else:
asciidoc += '`0`'
asciidoc += ', '
return asciidoc
def createValidationLineForParameterIntroChunk(self, blockname, param, params, typetext):
"""Make the generic asciidoc line chunk portion used for all parameters.
May return an empty string if nothing to validate.
"""
asciidoc = ''
param_name = _getElemName(param)
paramtype = _getElemType(param)
asciidoc += self.makeAsciiDocPreChunk(blockname, param, params)
asciidoc += self.makeParameterName(param_name)
asciidoc += ' must: be '
if self.paramIsArray(param):
# Arrays. These are hard to get right, apparently
lengths = param.get('len').split(',')
if lengths[0] == 'null-terminated':
asciidoc += 'a null-terminated '
elif lengths[0] == '1':
asciidoc += 'a ' + self.valid_pointer_text + ' to '
else:
asciidoc += 'a ' + self.valid_pointer_text + ' to an array of '
# Handle equations, which are currently denoted with latex
if 'latexmath:' in lengths[0]:
asciidoc += lengths[0]
else:
asciidoc += self.makeParameterName(lengths[0])
asciidoc += ' '
for length in lengths[1:]:
if length == 'null-terminated':
# This should always be the last thing.
# If it ever isn't for some bizarre reason, then this will need some massaging.
asciidoc += 'null-terminated '
elif length == '1':
asciidoc += self.valid_pointer_text + 's to '
else:
asciidoc += self.valid_pointer_text + 's to arrays of '
# Handle equations, which are currently denoted with latex
if 'latexmath:' in length:
asciidoc += length
else:
asciidoc += self.makeParameterName(length)
asciidoc += ' '
# Void pointers don't actually point at anything - remove the word "to"
if paramtype == 'void':
if lengths[-1] == '1':
if len(lengths) > 1:
asciidoc = asciidoc[:-5] # Take care of the extra s added by the post array chunk function. #HACK#
else:
asciidoc = asciidoc[:-4]
else:
# An array of void values is a byte array.
asciidoc += 'byte'
elif paramtype == 'char':
# A null terminated array of chars is a string
if lengths[-1] == 'null-terminated':
asciidoc += 'UTF-8 string'
else:
# Else it's just a bunch of chars
asciidoc += 'char value'
elif param.text is not None:
# If a value is "const" that means it won't get modified, so it must be valid going into the function.
if 'const' in param.text:
typecategory = self.getTypeCategory(paramtype)
if (typecategory not in ('struct', 'union', 'basetype') and typecategory is not None) \
or not self.isStructAlwaysValid(blockname, paramtype):
asciidoc += 'valid '
asciidoc += typetext
# pluralize
if len(lengths) > 1 or (lengths[0] != '1' and lengths[0] != 'null-terminated'):
asciidoc += 's'
elif self.paramIsPointer(param):
# Handle pointers - which are really special case arrays (i.e. they don't have a length)
#TODO should do something here if someone ever uses some intricate comma-separated `optional`
pointercount = param.find('type').tail.count('*')
# Treat void* as an int
if paramtype == 'void':
pointercount -= 1
# Could be multi-level pointers (e.g. ppData - pointer to a pointer). Handle that.
asciidoc += 'a '
asciidoc += (self.valid_pointer_text + ' to a ') * pointercount
# Handle void* and pointers to it
if paramtype == 'void':
# If there is only void*, it is just optional int - we don't need any language.
if pointercount == 0 and param.get('optional') is not None:
return '' # early return
if param.get('optional') is None or param.get('optional').split(',')[pointercount]:
# The last void* is just optional int (e.g. to be filled by the impl.)
typetext = 'pointer value'
# If a value is "const" that means it won't get modified, so it must be valid going into the function.
if param.text is not None and paramtype != 'void':
if 'const' in param.text:
asciidoc += 'valid '
asciidoc += typetext
else:
# Non-pointer, non-optional things must be valid
asciidoc += 'a valid '
asciidoc += typetext
if asciidoc != '':
asciidoc += '\n'
# Add additional line for non-optional bitmasks
isOutputParam = self.paramIsPointer(param) and not (param.text and 'const' in param.text)
if self.getTypeCategory(paramtype) == 'bitmask' and not isOutputParam:
isMandatory = param.get('optional') is None #TODO does not really handle if someone tries something like optional="true,false"
if isMandatory:
asciidoc += self.makeAnchor(blockname, param_name, 'requiredbitmask')
if self.paramIsArray(param):
asciidoc += 'Each element of '
asciidoc += 'pname:'
asciidoc += param_name
asciidoc += ' must: not be `0`'
asciidoc += '\n'
return asciidoc
def makeAsciiDocLineForParameter(self, blockname, param, params, typetext):
if param.get('noautovalidity') is not None:
return ''
asciidoc = self.createValidationLineForParameterIntroChunk(blockname, param, params, typetext)
return asciidoc
def isStructAlwaysValid(self, blockname, structname):
"""Try to do check if a structure is always considered valid (i.e. there's no rules to its acceptance)."""
struct = self.registry.tree.find("types/type[@name='" + structname + "']")
if struct.get('returnedonly'):
return True
params = struct.findall('member')
for param in params:
param_name = _getElemName(param)
paramtype = _getElemType(param)
typecategory = self.getTypeCategory(paramtype)
if param_name in (self.structtype_member_name, self.nextpointer_member_name):
return False
if param.get('noautovalidity'):
return False
if paramtype in ('void', 'char') or self.paramIsArray(param) or self.paramIsPointer(param):
if self.makeAsciiDocLineForParameter(blockname, param, params, '') != '':
return False
elif typecategory in ('handle', 'enum', 'bitmask'):
return False
elif typecategory in ('struct', 'union'):
if self.isStructAlwaysValid(blockname, paramtype) is False:
return False
return True
def fix_an(self, text):
"""Fix 'a' vs 'an' in a string"""
return re.sub(r' a ([a-z]+:)?([aAeEiIoOxX]\w+\b)(?!:)', r' an \1\2', text)
def createValidationLineForParameter(self, blockname, param, params, typecategory):
"""Make an entire asciidoc line for a given parameter."""
asciidoc = ''
param_name = _getElemName(param)
paramtype = _getElemType(param)
type_name = paramtype
is_array = self.paramIsArray(param)
is_pointer = self.paramIsPointer(param)
needs_recursive_validity = (is_array or
is_pointer or
not self.isStructAlwaysValid(blockname, type_name))
typetext = None
if type_name in ('void', 'char'):
# Chars and void are special cases - needs care inside the generator functions
# A null-terminated char array is a string, else it's chars.
# An array of void values is a byte array, a void pointer is just a pointer to nothing in particular
typetext = ''
elif typecategory == 'bitmask':
bitsname = type_name.replace('Flags', 'FlagBits')
if self.registry.tree.find("enums[@name='" + bitsname + "']") is None:
# Empty bit mask: presumably just a placeholder (or only in an extension not enabled for this build)
asciidoc += self.makeAnchor(blockname,
param_name, 'zerobitmask')
asciidoc += self.makeParameterName(param_name)
asciidoc += ' must: be `0`'
asciidoc += '\n'
else:
const_in_text = param.text is not None and 'const' in param.text
if is_array:
if const_in_text:
# input an array of bitmask values
template = 'combinations of {bitsname} value'
else:
template = '{paramtype} value'
elif is_pointer:
if const_in_text:
template = 'combination of {bitsname} values'
else:
template = '{paramtype} value'
else:
template = 'combination of {bitsname} values'
# The above few cases all use makeEnumerationName, just with different context.
typetext = template.format(bitsname=self.makeEnumerationName(bitsname), paramtype=self.makeEnumerationName(type_name))
elif typecategory == 'handle':
typetext = '{} handle'.format(self.makeStructName(type_name))
elif typecategory == 'enum':
typetext = '{} value'.format(self.makeEnumerationName(type_name))
elif typecategory == 'funcpointer':
typetext = '{} value'.format(self.makeFuncPointerName(type_name))
elif typecategory == 'struct':
if needs_recursive_validity:
typetext = '{} structure'.format(
self.makeStructName(type_name))
else:
# TODO right now just skipping generating a line here.
# I think this was intentional in general, but maybe not in that particular case.
#raise UnhandledCaseError()
pass
elif typecategory == 'union':
if needs_recursive_validity:
typetext = '{} union'.format(self.makeStructName(type_name))
else:
# TODO right now just skipping generating a line here.
# I think this was intentional in general:
# we do hit this in Vulkan.
pass
elif self.paramIsArray(param) or self.paramIsPointer(param):
typetext = '{} value'.format(self.makeBaseTypeName(paramtype))
elif typecategory is None:
# "a valid uint32_t value" doesn't make much sense.
pass
# If any of the above conditions matched and set typetext,
# we call using it.
if typetext is not None:
asciidoc += self.makeAsciiDocLineForParameter(
blockname, param, params, typetext)
return self.fix_an(asciidoc)
def makeAsciiDocHandleParent(self, blockname, param, params):
"""Make an asciidoc validity entry for a handle's parent object.
Creates 'parent' VUID.
"""
asciidoc = ''
param_name = _getElemName(param)
paramtype = _getElemType(param)
# Deal with handle parents
handleparent = self.getHandleParent(paramtype)
if handleparent is None:
return asciidoc
otherparam = None
for p in params:
if _getElemType(p) == handleparent:
otherparam = p
break
if otherparam is None:
return asciidoc
parentreference = _getElemName(otherparam)
asciidoc += self.makeAnchor(blockname, param_name, 'parent')
if self.isHandleOptional(param, params):
if self.paramIsArray(param):
asciidoc += 'Each element of '
asciidoc += self.makeParameterName(param_name)
asciidoc += ' that is a valid handle'
else:
asciidoc += 'If '
asciidoc += self.makeParameterName(param_name)
asciidoc += ' is a valid handle, it'
else:
if self.paramIsArray(param):
asciidoc += 'Each element of '
asciidoc += self.makeParameterName(param_name)
asciidoc += ' must: have been created, allocated, or retrieved from '
asciidoc += self.makeParameterName(parentreference)
asciidoc += '\n'
return asciidoc
def makeAsciiDocHandlesCommonAncestor(self, blockname, handles, params):
"""Make an asciidoc validity entry for a common ancestors between handles.
Only handles parent validity for signatures taking multiple handles
any ancestors also being supplied to this function.
(e.g. "Each of x, y, and z must: come from the same slink:ParentHandle")
See self.makeAsciiDocHandleParent() for instances where the parent
handle is named and also passed.
Creates 'commonparent' VUID.
"""
asciidoc = ''
if len(handles) > 1:
ancestormap = {}
anyoptional = False
# Find all the ancestors
for param in handles:
paramtype = _getElemType(param)
if not self.paramIsPointer(param) or (param.text and 'const' in param.text):
ancestors = self.getHandleDispatchableAncestors(paramtype)
ancestormap[param] = ancestors
anyoptional |= self.isHandleOptional(param, params)
# Remove redundant ancestor lists
for param in handles:
paramtype = _getElemType(param)
removals = []
for ancestors in ancestormap.items():
if paramtype in ancestors[1]:
removals.append(ancestors[0])
if removals != []:
for removal in removals:
del(ancestormap[removal])
# Intersect
if len(ancestormap.values()) > 1:
current = list(ancestormap.values())[0]
for ancestors in list(ancestormap.values())[1:]:
current = [val for val in current if val in ancestors]
if len(current) > 0:
commonancestor = current[0]
if len(ancestormap.keys()) > 1:
asciidoc += self.makeAnchor(blockname, None, 'commonparent')
parametertexts = []
for param in ancestormap.keys():
param_name = _getElemName(param)
parametertext = self.makeParameterName(param_name)
if self.paramIsArray(param):
parametertext = 'the elements of ' + parametertext
parametertexts.append(parametertext)
parametertexts.sort()
if len(parametertexts) > 2:
asciidoc += 'Each of '
else:
asciidoc += 'Both of '
asciidoc += ", ".join(parametertexts[:-1])
asciidoc += ', and '
asciidoc += parametertexts[-1]
if anyoptional is True:
asciidoc += ' that are valid handles'
asciidoc += ' must: have been created, allocated, or retrieved from the same '
asciidoc += self.makeStructName(commonancestor)
asciidoc += '\n'
return asciidoc
def makeStructureTypeFromName(self, structname):
"""Create text for a structure type name, like ename:VK_STRUCTURE_TYPE_CREATE_INSTANCE_INFO"""
return self.makeEnumerantName(self.conventions.generate_structure_type_from_name(structname))
def makeStructureType(self, structname, param):
"""Generate an asciidoc validity line for the type value of a struct."""
param_name = _getElemName(param)
asciidoc = self.makeAnchor(structname, param_name, self.structtype_member_name)
asciidoc += self.makeParameterName(param_name)
asciidoc += ' must: be '
values = param.get('values')
if values:
# Extract each enumerant value. They could be validated in the
# same fashion as validextensionstructs in
# makeStructureExtensionPointer, although that's not relevant in
# the current extension struct model.
asciidoc += self.makeProseList(( self.makeEnumerantName(v) for v in values.split(',')), 'or')
elif 'Base' in structname:
# This type doesn't even have any values for its type,
# and it seems like it might be a base struct that we'd expect to lack its own type,
# so omit the entire statement
return ''
else:
self.logMsg('warn', 'No values were marked-up for the structure type member of', structname, 'so making one up!')
asciidoc += self.makeStructureTypeFromName(structname)
asciidoc += '\n'
return asciidoc
def makeStructureExtensionPointer(self, blockname, param):
"""Generate an asciidoc validity line for the pointer chain member value of a struct."""
param_name = _getElemName(param)
if param.get('validextensionstructs') is not None:
self.logMsg('warn', blockname, 'validextensionstructs is deprecated/removed', '\n')
asciidoc = self.makeAnchor(blockname, param_name, self.nextpointer_member_name)
validextensionstructs = self.registry.validextensionstructs.get(blockname)
extensionstructs = []
if validextensionstructs is not None:
# Check each structure name and skip it if not required by the
# generator. This allows tagging extension structs in the XML
# that are only included in validity when needed for the spec
# being targeted.
for struct in validextensionstructs:
# Unpleasantly breaks encapsulation. Should be a method in the registry class
type = self.registry.lookupElementInfo(struct, self.registry.typedict)
if type is None:
self.logMsg('warn', 'makeStructureExtensionPointer: struct', struct, 'is in a validextensionstructs= attribute but is not in the registry')
elif type.required:
extensionstructs.append('slink:' + struct)
else:
self.logMsg('diag', 'makeStructureExtensionPointer: struct', struct, 'IS NOT required')
if not extensionstructs:
asciidoc += self.makeParameterName(param_name)
asciidoc += ' must: be '
asciidoc += self.null
elif len(extensionstructs) == 1:
asciidoc += self.makeParameterName(param_name)
asciidoc += ' must: be '
asciidoc += self.null
asciidoc +=' or a pointer to a valid instance of '
asciidoc += extensionstructs[0]
else:
asciidoc += 'Each '
asciidoc += self.makeParameterName(param_name)
asciidoc += ' member of any structure (including this one) in the '
asciidoc += 'pname:' + self.nextpointer_member_name
asciidoc += ' chain must: be either '
asciidoc += self.null
asciidoc += ' or a pointer to a valid instance of '
asciidoc += self.makeProseList(extensionstructs, 'or')
asciidoc += '\n'
asciidoc += self.makeAnchor(blockname, self.structtype_member_name, 'unique')
asciidoc += 'Each pname:' + self.structtype_member_name + ' member in the '
asciidoc += 'pname:' + self.nextpointer_member_name
asciidoc += ' chain must: be unique'
asciidoc += '\n'
return asciidoc
def makeValidUsageStatementsReturnedOnly(self, cmd, blockname, params):
"""Generate all the valid usage information for a given struct
that's only ever filled out by the implementation other than the
structure type and pointer chain members.
"""
# Start the asciidoc block for this
asciidoc = ''
for param in params:
param_name = _getElemName(param)
paramtype = _getElemType(param)
# Valid usage ID tags (VUID) are generated for various
# conditions based on the name of the block (structure or
# command), name of the element (member or parameter), and type
# of VU statement.
if param.get('noautovalidity') is None:
# Generate language to independently validate a parameter
if self.conventions.is_structure_type_member(paramtype, param_name):
asciidoc += self.makeStructureType(blockname, param)
elif self.conventions.is_nextpointer_member(paramtype, param_name) and cmd.get('structextends') is None:
asciidoc += self.makeStructureExtensionPointer(blockname, param)
# In case there's nothing to report, return None
if asciidoc == '':
return None
return asciidoc
def makeValidUsageStatements(self, cmd, blockname, params):
"""Generate all the valid usage information for a given struct or command."""
# Start the asciidoc block for this
asciidoc = ''
handles = []
arraylengths = set()
for param in params:
param_name = _getElemName(param)
paramtype = _getElemType(param)
# Valid usage ID tags (VUID) are generated for various
# conditions based on the name of the block (structure or
# command), name of the element (member or parameter), and type
# of VU statement.
# Get the type's category
typecategory = self.getTypeCategory(paramtype)
if param.get('noautovalidity') is None:
# Generate language to independently validate a parameter
if self.conventions.is_structure_type_member(paramtype, param_name):
asciidoc += self.makeStructureType(blockname, param)
elif self.conventions.is_nextpointer_member(paramtype, param_name):
if cmd.get('structextends') is None:
asciidoc += self.makeStructureExtensionPointer(blockname, param)
else:
asciidoc += self.createValidationLineForParameter(blockname, param, params, typecategory)
# Ensure that any parenting is properly validated, and list that a handle was found
if typecategory == 'handle':
handles.append(param)
# Get the array length for this parameter
arraylength = param.get('len')
if arraylength is not None:
arraylengths.update(set(arraylength.split(',')))
# For any vkQueue* functions, there might be queue type data
if 'vkQueue' in blockname:
# The queue type must be valid
queuetypes = cmd.get('queues')
if queuetypes:
queuebits = []
for queuetype in re.findall(r'([^,]+)', queuetypes):
queuebits.append(queuetype.replace('_',' '))
asciidoc += self.makeAnchor(blockname, None, 'queuetype')
asciidoc += 'The pname:queue must: support '
if len(queuebits) == 1:
asciidoc += queuebits[0]
else:
asciidoc += (', ').join(queuebits[:-1])
asciidoc += ', or '
asciidoc += queuebits[-1]
asciidoc += ' operations'
asciidoc += '\n'
if 'vkCmd' in blockname:
# The commandBuffer parameter must be being recorded
asciidoc += self.makeAnchor(blockname, 'commandBuffer', 'recording')
asciidoc += 'pname:commandBuffer must: be in the <<commandbuffers-lifecycle, recording state>>'
asciidoc += '\n'
#
# Start of valid queue type validation - command pool must have been
# allocated against a queue with at least one of the valid queue types
asciidoc += self.makeAnchor(blockname, 'commandBuffer', 'cmdpool')
#
# This test for vkCmdFillBuffer is a hack, since we have no path
# to conditionally have queues enabled or disabled by an extension.
# As the VU stuff is all moving out (hopefully soon), this hack solves the issue for now
if blockname == 'vkCmdFillBuffer':
if 'VK_KHR_maintenance1' in self.registry.requiredextensions:
asciidoc += 'The sname:VkCommandPool that pname:commandBuffer was allocated from must: support transfer, graphics or compute operations\n'
else:
asciidoc += 'The sname:VkCommandPool that pname:commandBuffer was allocated from must: support graphics or compute operations\n'
else:
# The queue type must be valid
queuetypes = cmd.get('queues')
queuebits = []
for queuetype in re.findall(r'([^,]+)', queuetypes):
queuebits.append(queuetype.replace('_',' '))
asciidoc += 'The sname:VkCommandPool that pname:commandBuffer was allocated from must: support '
if len(queuebits) == 1:
asciidoc += queuebits[0]
else:
asciidoc += (', ').join(queuebits[:-1])
asciidoc += ', or '
asciidoc += queuebits[-1]
asciidoc += ' operations'
asciidoc += '\n'
# Must be called inside/outside a renderpass appropriately
renderpass = cmd.get('renderpass')
if renderpass != 'both':
asciidoc += self.makeAnchor(blockname, None, 'renderpass')
asciidoc += 'This command must: only be called '
asciidoc += renderpass
asciidoc += ' of a render pass instance'
asciidoc += '\n'
# Must be in the right level command buffer
cmdbufferlevel = cmd.get('cmdbufferlevel')
if cmdbufferlevel != 'primary,secondary':
asciidoc += self.makeAnchor(blockname, None, 'bufferlevel')
asciidoc += 'pname:commandBuffer must: be a '
asciidoc += cmdbufferlevel
asciidoc += ' sname:VkCommandBuffer'
asciidoc += '\n'
# Any non-optional arraylengths should specify they must be greater than 0
non_optional_array_lengths = ((param, _getElemName(param))
for param in params
if _getElemName(param) in arraylengths and not param.get('optional'))
for param, param_name in non_optional_array_lengths:
# Get all the array dependencies
arrays = cmd.findall(
"param/[@len='{}'][@optional='true']".format(param_name))
# Get all the optional array dependencies, including those not generating validity for some reason
optionalarrays = arrays + \
cmd.findall(
"param/[@len='{}'][@noautovalidity='true']".format(param_name))
# If arraylength can ever be not a legal part of an
# asciidoc anchor name, this will need to be altered.
asciidoc += self.makeAnchor(blockname, param_name, 'arraylength')
# Allow lengths to be arbitrary if all their dependents are optional
if optionalarrays and len(optionalarrays) == len(arrays):
asciidoc += 'If '
if len(optionalarrays) > 1:
asciidoc += 'any of '
# TODO uncomment following statement once cosmetic changes OK
# asciidoc += self.makeProseList((self.makeParameterName(_getElemName(array))
# for array in optionalarrays),
# 'or')
if len(optionalarrays) > 1:
# TODO remove following statement once cosmetic changes OK
asciidoc += self.makeProseList((self.makeParameterName(_getElemName(array))
for array in optionalarrays),
'or')
asciidoc += ' are '
else:
# TODO remove following statement once cosmetic changes OK
asciidoc += ', '.join((self.makeParameterName(_getElemName(array))
for array in optionalarrays))
asciidoc += ' is '
asciidoc += 'not ' + self.null + ', '
if self.paramIsPointer(param):
asciidoc += 'the value referenced by '
elif self.paramIsPointer(param):
asciidoc += 'The value referenced by '
asciidoc += self.makeParameterName(param_name)
asciidoc += ' must: be greater than `0`'
asciidoc += '\n'
# Find the parents of all objects referenced in this command
for param in handles:
paramtype = _getElemType(param)
# Don't detect a parent for return values!
if not self.paramIsPointer(param) or (param.text is not None and 'const' in param.text):
parent = self.getHandleParent(paramtype)
if parent is not None:
asciidoc += self.makeAsciiDocHandleParent(blockname, param, params)
# Find the common ancestor of all objects referenced in this command
asciidoc += self.makeAsciiDocHandlesCommonAncestor(blockname, handles, params)
# In case there's nothing to report, return None
if asciidoc == '':
return None
# Delimit the asciidoc block
return asciidoc
def makeThreadSafetyBlock(self, cmd, paramtext):
"""Generate C function pointer typedef for <command> Element"""
paramdecl = ''
# Find and add any parameters that are thread unsafe
explicitexternsyncparams = cmd.findall(paramtext + "[@externsync]")
if explicitexternsyncparams is not None:
for param in explicitexternsyncparams:
externsyncattribs = param.get('externsync')
param_name = _getElemName(param)
for externsyncattrib in externsyncattribs.split(','):
paramdecl += '* '
paramdecl += 'Host access to '
if externsyncattrib == 'true':
if self.paramIsArray(param):
paramdecl += 'each member of ' + \
self.makeParameterName(param_name)
elif self.paramIsPointer(param):
paramdecl += 'the object referenced by ' + \
self.makeParameterName(param_name)
else:
paramdecl += self.makeParameterName(param_name)
else:
paramdecl += 'pname:'
paramdecl += externsyncattrib
paramdecl += ' must: be externally synchronized\n'
# Vulkan-specific
# For any vkCmd* functions, the command pool is externally synchronized
if cmd.find('proto/name') is not None and 'vkCmd' in cmd.find('proto/name').text:
paramdecl += '* '
paramdecl += 'Host access to the sname:VkCommandPool that pname:commandBuffer was allocated from must: be externally synchronized'
paramdecl += '\n'
# Find and add any "implicit" parameters that are thread unsafe
implicitexternsyncparams = cmd.find('implicitexternsyncparams')
if implicitexternsyncparams is not None:
for elem in implicitexternsyncparams:
paramdecl += '* '
paramdecl += 'Host access to '
paramdecl += elem.text
paramdecl += ' must: be externally synchronized\n'
if not paramdecl:
return None
return paramdecl
def makeCommandPropertiesTableEntry(self, cmd, name):
if 'vkCmd' in name:
# Must be called inside/outside a renderpass appropriately
cmdbufferlevel = cmd.get('cmdbufferlevel')
cmdbufferlevel = (' + \n').join(cmdbufferlevel.title().split(','))
renderpass = cmd.get('renderpass')
renderpass = renderpass.capitalize()
#
# This test for vkCmdFillBuffer is a hack, since we have no path
# to conditionally have queues enabled or disabled by an extension.
# As the VU stuff is all moving out (hopefully soon), this hack solves the issue for now
if name == 'vkCmdFillBuffer':
if 'VK_KHR_maintenance1' in self.registry.requiredextensions:
queues = 'Transfer + \nGraphics + \nCompute'
else:
queues = 'Graphics + \nCompute'
else:
queues = cmd.get('queues')
queues = (' + \n').join(queues.title().split(','))
pipeline = cmd.get('pipeline')
if pipeline:
pipeline = pipeline.capitalize()
else:
pipeline = ''
return '|' + cmdbufferlevel + '|' + renderpass + '|' + queues + '|' + pipeline
elif 'vkQueue' in name:
# Must be called inside/outside a renderpass appropriately
queues = cmd.get('queues')
if queues is None:
queues = 'Any'
else:
queues = (' + \n').join(queues.upper().split(','))
return '|-|-|' + queues + '|-'
return None
# Check each enumerant name in the enums list and remove it if not
# required by the generator. This allows specifying success and error
# codes for extensions that are only included in validity when needed
# for the spec being targeted.
def findRequiredEnums(self, enums):
out = []
for enum in enums:
# Unpleasantly breaks encapsulation. Should be a method in the registry class
ei = self.registry.lookupElementInfo(enum, self.registry.enumdict)
if ei is None:
self.logMsg('warn', 'findRequiredEnums: enum', enum,
'is in an attribute list but is not in the registry')
elif ei.required:
out.append(enum)
else:
self.logMsg('diag', 'findRequiredEnums: enum', enum, 'IS NOT required, skipping')
return out
def makeReturnCodeList(self, attrib, cmd, name):
"""Return a list of possible return codes for a function.
attrib is either 'successcodes' or 'errorcodes'.
"""
return_lines = []
RETURN_CODE_FORMAT = '* ename:{}'
codes_attr = cmd.get(attrib)
if codes_attr:
codes = self.findRequiredEnums(codes_attr.split(','))
if codes:
return_lines.extend((RETURN_CODE_FORMAT.format(code)
for code in codes))
applicable_ext_codes = (ext_code
for ext_code in self.registry.commandextensionsuccesses
if ext_code.command == name)
for ext_code in applicable_ext_codes:
line = RETURN_CODE_FORMAT.format(ext_code.value)
if ext_code.extension:
line += ' [only if {} is enabled]'.format(
self.conventions.formatExtension(ext_code.extension))
return_lines.append(line)
if return_lines:
return '\n'.join(return_lines)
return None
def makeSuccessCodes(self, cmd, name):
return self.makeReturnCodeList('successcodes', cmd, name)
def makeErrorCodes(self, cmd, name):
return self.makeReturnCodeList('errorcodes', cmd, name)
def genCmd(self, cmdinfo, name, alias):
"""Command generation."""
OutputGenerator.genCmd(self, cmdinfo, name, alias)
# @@@ (Jon) something needs to be done here to handle aliases, probably
# Get all the parameters
params = cmdinfo.elem.findall('param')
validity = self.makeValidUsageStatements(cmdinfo.elem, name, params)
threadsafety = self.makeThreadSafetyBlock(cmdinfo.elem, 'param')
# Vulkan-specific
commandpropertiesentry = self.makeCommandPropertiesTableEntry(cmdinfo.elem, name)
successcodes = self.makeSuccessCodes(cmdinfo.elem, name)
errorcodes = self.makeErrorCodes(cmdinfo.elem, name)
# OpenXR-specific
# beginsstate = self.makeBeginState(cmdinfo.elem, name)
# endsstate = self.makeEndState(cmdinfo.elem, name)
# checksstatedata = self.makeCheckState(cmdinfo.elem, name)
self.writeInclude('protos', name, validity, threadsafety, commandpropertiesentry, successcodes, errorcodes)
def genStruct(self, typeinfo, typeName, alias):
"""Struct Generation."""
OutputGenerator.genStruct(self, typeinfo, typeName, alias)
# @@@ (Jon) something needs to be done here to handle aliases, probably
# Anything that's only ever returned can't be set by the user, so shouldn't have any validity information.
params = []
validity = ''
threadsafety = []
if typeinfo.elem.get('returnedonly') is None:
params = typeinfo.elem.findall('member')
generalvalidity = self.makeValidUsageStatements(typeinfo.elem, typeName, params)
if generalvalidity:
validity += generalvalidity
threadsafety = self.makeThreadSafetyBlock(typeinfo.elem, 'member')
self.writeInclude('structs', typeName, validity, threadsafety, None, None, None)
else:
# Need to generate structure type and next pointer chain member validation
params = typeinfo.elem.findall('member')
retvalidity = self.makeValidUsageStatementsReturnedOnly(typeinfo.elem, typeName, params)
if retvalidity:
validity += retvalidity
self.writeInclude('structs', typeName, validity, None, None, None, None)
def genGroup(self, groupinfo, groupName, alias):
"""Group (e.g. C "enum" type) generation.
For the validity generator, this just tags individual enumerants
as required or not.
"""
OutputGenerator.genGroup(self, groupinfo, groupName, alias)
# @@@ (Jon) something needs to be done here to handle aliases, probably
groupElem = groupinfo.elem
# Loop over the nested 'enum' tags. Keep track of the minimum and
# maximum numeric values, if they can be determined; but only for
# core API enumerants, not extension enumerants. This is inferred
# by looking for 'extends' attributes.
for elem in groupElem.findall('enum'):
name = elem.get('name')
ei = self.registry.lookupElementInfo(name, self.registry.enumdict)
# Tag enumerant as required or not
ei.required = self.isEnumRequired(elem)
def genType(self, typeinfo, name, alias):
"""Type Generation."""
OutputGenerator.genType(self, typeinfo, name, alias)
# @@@ (Jon) something needs to be done here to handle aliases, probably
category = typeinfo.elem.get('category')
if category in ('struct', 'union'):
self.genStruct(typeinfo, name, alias)