VK_QCOM_image_processing(3)

# VK_QCOM_image_processing(3) Manual Page

## Name

VK_QCOM_image_processing - device extension

Loading… please wait.

## Registered Extension Number

441

## Revision

1

## Extension and Version Dependencies

- Requires support for Vulkan 1.0

- Requires `VK_KHR_format_feature_flags2` to be enabled for any device-level functionality

## Contact

- Jeff Leger jackohound

## Extension Proposal

VK_QCOM_image_processing

## Other Extension Metadata

Last Modified Date

  2022-07-08

Interactions and External Dependencies

  - This extension requires `SPV_QCOM_image_processing`

  - This extension provides API support for `GL_QCOM_image_processing`

Contributors

  - Jeff Leger, Qualcomm Technologies, Inc.

  - Ruihao Zhang, Qualcomm Technologies, Inc.

## Description

GPUs are commonly used to process images for various applications from 3D graphics to UI and from composition to compute applications. Simple scaling and filtering can be done with bilinear filtering, which comes for free during texture sampling. However, as screen sizes get larger and more use-cases rely on GPU such as camera and video post-processing needs, there is increasing demand for GPU to support higher order filtering and other advanced image processing.

This extension introduces a new set of SPIR-V built-in functions for image processing. It exposes the following new imaging operations

- The `OpImageSampleWeightedQCOM` instruction takes 3 operands: sampled image, weight image, and texture coordinates. The instruction computes a weighted average of an MxN region of texels in the sampled image, using a set of MxN weights in the weight image.

- The `OpImageBoxFilterQCOM` instruction takes 3 operands: sampled image, box size, and texture coordinates. Note that box size specifies a floating point width and height in texels. The instruction computes a weighted average of all texels in the sampled image that are covered (either partially or fully) by a box with the specified size and centered at the specified texture coordinates.

- The `OpImageBlockMatchSADQCOM` and `OpImageBlockMatchSSDQCOM` instructions each takes 5 operands: target image, target coordinates, reference image, reference coordinates, and block size. Each instruction computes an error metric, that describes whether a block of texels in the target image matches a corresponding block of texels in the reference image. The error metric is computed per-component. `OpImageBlockMatchSADQCOM` computes "Sum Of Absolute Difference" and `OpImageBlockMatchSSDQCOM` computes "Sum of Squared Difference".

Each of the image processing instructions operate only on 2D images. The instructions do not-support sampling of mipmap, multi-plane, multi-layer, multi-sampled, or depth/stencil images. The instructions can be used in any shader stage.

Implementations of this this extension should support these operations natively at the HW instruction level, offering potential performance gains as well as ease of development.

## New Structures

- Extending VkImageViewCreateInfo:

  - VkImageViewSampleWeightCreateInfoQCOM

- Extending VkPhysicalDeviceFeatures2, VkDeviceCreateInfo:

  - VkPhysicalDeviceImageProcessingFeaturesQCOM

- Extending VkPhysicalDeviceProperties2:

  - VkPhysicalDeviceImageProcessingPropertiesQCOM

## New Enum Constants

- `VK_QCOM_IMAGE_PROCESSING_EXTENSION_NAME`

- `VK_QCOM_IMAGE_PROCESSING_SPEC_VERSION`

- Extending VkDescriptorType:

  - `VK_DESCRIPTOR_TYPE_BLOCK_MATCH_IMAGE_QCOM`

  - `VK_DESCRIPTOR_TYPE_SAMPLE_WEIGHT_IMAGE_QCOM`

- Extending VkImageUsageFlagBits:

  - `VK_IMAGE_USAGE_SAMPLE_BLOCK_MATCH_BIT_QCOM`

  - `VK_IMAGE_USAGE_SAMPLE_WEIGHT_BIT_QCOM`

- Extending VkSamplerCreateFlagBits:

  - `VK_SAMPLER_CREATE_IMAGE_PROCESSING_BIT_QCOM`

- Extending VkStructureType:

  - `VK_STRUCTURE_TYPE_IMAGE_VIEW_SAMPLE_WEIGHT_CREATE_INFO_QCOM`

  - `VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_IMAGE_PROCESSING_FEATURES_QCOM`

  - `VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_IMAGE_PROCESSING_PROPERTIES_QCOM`

If VK_KHR_format_feature_flags2 is supported:

- Extending VkFormatFeatureFlagBits2:

  - `VK_FORMAT_FEATURE_2_BLOCK_MATCHING_BIT_QCOM`

  - `VK_FORMAT_FEATURE_2_BOX_FILTER_SAMPLED_BIT_QCOM`

  - `VK_FORMAT_FEATURE_2_WEIGHT_IMAGE_BIT_QCOM`

  - `VK_FORMAT_FEATURE_2_WEIGHT_SAMPLED_IMAGE_BIT_QCOM`

## Version History

- Revision 1, 2022-07-08 (Jeff Leger)

## See Also

VkImageViewSampleWeightCreateInfoQCOM, VkPhysicalDeviceImageProcessingFeaturesQCOM, VkPhysicalDeviceImageProcessingPropertiesQCOM

## Document Notes

For more information, see the Vulkan Specification

This page is a generated document. Fixes and changes should be made to the generator scripts, not directly.

## Copyright

Copyright 2014-2022 The Khronos Group Inc.

SPDX-License-Identifier: CC-BY-4.0