include::meta/VK_NV_shading_rate_image.txt[] *Last Modified Date*:: 2018-09-13 *Contributors*:: - Pat Brown, NVIDIA - Carsten Rohde, NVIDIA - Jeff Bolz, NVIDIA - Daniel Koch, NVIDIA - Mathias Schott, NVIDIA - Matthew Netsch, Qualcomm Technologies, Inc. This extension allows applications to use a variable shading rate when processing fragments of rasterized primitives. By default, Vulkan will spawn one fragment shader for each pixel covered by a primitive. In this extension, applications can bind a _shading rate image_ that can be used to vary the number of fragment shader invocations across the framebuffer. Some portions of the screen may be configured to spawn up to 16 fragment shaders for each pixel, while other portions may use a single fragment shader invocation for a 4x4 block of pixels. This can be useful for use cases like eye tracking, where the portion of the framebuffer that the user is looking at directly can be processed at high frequency, while distant corners of the image can be processed at lower frequency. Each texel in the shading rate image represents a fixed-size rectangle in the framebuffer, covering 16x16 pixels in the initial implementation of this extension. When rasterizing a primitive covering one of these rectangles, the Vulkan implementation reads a texel in the bound shading rate image and looks up the fetched value in a palette to determine a base shading rate. In addition to the API support controlling rasterization, this extension also adds Vulkan support for the +SPV_NV_shading_rate+ extension to SPIR-V. That extension provides two fragment shader variable decorations that allow fragment shaders to determine the shading rate used for processing the fragment: * code:FragmentSizeNV, which indicates the width and height of the set of pixels processed by the fragment shader. * code:InvocationsPerPixel, which indicates the maximum number of fragment shader invocations that could be spawned for the pixel(s) covered by the fragment. When using SPIR-V in conjunction with the OpenGL Shading Language (GLSL), the fragment shader capabilities are provided by the +GL_NV_shading_rate_image+ language extension and correspond to the built-in variables code:gl_FragmentSizeNV and code:gl_InvocationsPerPixelNV, respectively. === New Object Types None. === New Enum Constants * Extending elink:VkStructureType: ** ename:VK_STRUCTURE_TYPE_PIPELINE_VIEWPORT_SHADING_RATE_IMAGE_STATE_CREATE_INFO_NV ** ename:VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_SHADING_RATE_IMAGE_FEATURES_NV ** ename:VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_SHADING_RATE_IMAGE_PROPERTIES_NV * Extending elink:VkImageLayout: ** ename:VK_IMAGE_LAYOUT_SHADING_RATE_OPTIMAL_NV * Extending elink:VkDynamicState: ** ename:VK_DYNAMIC_STATE_VIEWPORT_SHADING_RATE_PALETTE_NV * Extending elink:VkAccessFlagBits: ** ename:VK_ACCESS_SHADING_RATE_IMAGE_READ_BIT_NV * Extending elink:VkImageUsageFlagBits: ** ename:VK_IMAGE_USAGE_SHADING_RATE_IMAGE_BIT_NV * Extending elink:VkPipelineStageFlagBits ** ename:VK_PIPELINE_STAGE_SHADING_RATE_IMAGE_BIT_NV === New Enums * elink:VkShadingRatePaletteEntryNV, containing the following constants: ** ename:VK_SHADING_RATE_PALETTE_ENTRY_NO_INVOCATIONS_NV ** ename:VK_SHADING_RATE_PALETTE_ENTRY_16_INVOCATIONS_PER_PIXEL_NV ** ename:VK_SHADING_RATE_PALETTE_ENTRY_8_INVOCATIONS_PER_PIXEL_NV ** ename:VK_SHADING_RATE_PALETTE_ENTRY_4_INVOCATIONS_PER_PIXEL_NV ** ename:VK_SHADING_RATE_PALETTE_ENTRY_2_INVOCATIONS_PER_PIXEL_NV ** ename:VK_SHADING_RATE_PALETTE_ENTRY_1_INVOCATION_PER_PIXEL_NV ** ename:VK_SHADING_RATE_PALETTE_ENTRY_1_INVOCATION_PER_2X1_PIXELS_NV ** ename:VK_SHADING_RATE_PALETTE_ENTRY_1_INVOCATION_PER_1X2_PIXELS_NV ** ename:VK_SHADING_RATE_PALETTE_ENTRY_1_INVOCATION_PER_2X2_PIXELS_NV ** ename:VK_SHADING_RATE_PALETTE_ENTRY_1_INVOCATION_PER_4X2_PIXELS_NV ** ename:VK_SHADING_RATE_PALETTE_ENTRY_1_INVOCATION_PER_2X4_PIXELS_NV ** ename:VK_SHADING_RATE_PALETTE_ENTRY_1_INVOCATION_PER_4X4_PIXELS_NV === New Structures * slink:VkShadingRatePaletteNV * slink:VkPipelineViewportShadingRateImageStateCreateInfoNV * slink:VkPhysicalDeviceShadingRateImageFeaturesNV * slink:VkPhysicalDeviceShadingRateImagePropertiesNV === New Functions * flink:vkCmdBindShadingRateImageNV * flink:vkCmdSetViewportShadingRatePaletteNV === Issues (1) When using shading rates that specify "coarse" fragments covering multiple pixels, we will generate a combined coverage mask that combines the coverage masks of all pixels covered by the fragment. By default, these masks are combined in an implementation-dependent order. Should we provide a mechanism allowing applications to query or specify an exact order? *RESOLVED*: Yes, this feature is useful for cases where most of the fragment shader can be evaluated once for an entire coarse fragment, but where some per-pixel computations are also required. For example, a per-pixel alpha test may want to kill all the samples for some pixels in a coarse fragment. This sort of test can be implemented using an output sample mask, but such a shader would need to know which bit in the mask corresponds to each sample in the coarse fragment. We are including a mechanism to allow aplications to specify the orders of coverage samples for each shading rate and sample count, either as static pipeline state or dynamically via a command buffer. This portion of the extension has its own feature bit. We will not be providing a query to determine the implementation-dependent default ordering. The thinking here is that if an application cares enough about the coarse fragment sample ordering to perform such a query, it could instead just set its own order, also using custom per-pixel sample locations if required. (2) For the pipeline stage ename:VK_PIPELINE_STAGE_SHADING_RATE_IMAGE_BIT_NV, should we specify a precise location in the pipeline the shading rate image is accessed (after geometry shading, but before the early fragment tests) or leave it under-specified in case there are other implementations that access the image in a different pipeline location? *RESOLVED* We are specifying the pipeline stage to be between the final stage used for vertex processing (ename:VK_PIPELINE_STAGE_GEOMETRY_SHADER_BIT) and before the first stage used for fragment processing (ename:VK_PIPELINE_STAGE_EARLY_FRAGMENT_TESTS_BIT), which seems to be the natural place to access the shading rate image. (3) How do centroid-sampled variables work with fragments larger than one pixel? *RESOLVED* For single-pixel fragments, fragment shader inputs decorated with ename:Centroid are sampled at an implementation-dependent location in the intersection of the area of the primitive being rasterized and the area of the pixel that corresponds to the fragment. With multi-pixel fragments, we follow a similar pattern, using the intersection of the primitive and the *set* of pixels corresponding to the fragment. One important thing to keep in mind when using such "coarse" shading rates is that fragment attributes are sampled at the center of the fragment by default, regardless of the set of pixels/samples covered by the fragment. For fragments with a size of 4x4 pixels, this center location will be more than two pixels (1.5 * sqrt(2)) away from the center of the pixels at the corners of the fragment. When rendering a primitive that covers only a small part of a coarse fragment, sampling a color outside the primitive can produce overly bright or dark color values if the color values have a large gradient. To deal with this, an application can use centroid sampling on attributes where "extrapolation" artifacts can lead to overly bright or dark pixels. Note that this same problem also exists for multisampling with single-pixel fragments, but is less severe because it only affects certain samples of a pixel and such bright/dark samples may be averaged with other samples that don't have a similar problem. === Version History * Revision 2, 2018-09-13 (Pat Brown) - Miscellaneous edits preparing the specification for publication. * Revision 1, 2018-08-08 (Pat Brown) - Internal revisions