VK_NV_external_memory_rdma(3)

# VK_NV_external_memory_rdma(3) Manual Page

## Name

VK_NV_external_memory_rdma - device extension

Loading… please wait.

## Registered Extension Number

372

## Revision

1

## Ratification Status

Not ratified

## Extension and Version Dependencies

VK_KHR_external_memory

## Contact

- Carsten Rohde crohde

## Other Extension Metadata

Last Modified Date

  2021-04-19

IP Status

  No known IP claims.

Contributors

  - Carsten Rohde, NVIDIA

## Description

This extension adds support for allocating memory which can be used for remote direct memory access (RDMA) from other devices.

## New Base Types

- VkRemoteAddressNV

## New Commands

- vkGetMemoryRemoteAddressNV

## New Structures

- VkMemoryGetRemoteAddressInfoNV

- Extending VkPhysicalDeviceFeatures2, VkDeviceCreateInfo:

  - VkPhysicalDeviceExternalMemoryRDMAFeaturesNV

## New Enum Constants

- `VK_NV_EXTERNAL_MEMORY_RDMA_EXTENSION_NAME`

- `VK_NV_EXTERNAL_MEMORY_RDMA_SPEC_VERSION`

- Extending VkExternalMemoryHandleTypeFlagBits:

  - `VK_EXTERNAL_MEMORY_HANDLE_TYPE_RDMA_ADDRESS_BIT_NV`

- Extending VkMemoryPropertyFlagBits:

  - `VK_MEMORY_PROPERTY_RDMA_CAPABLE_BIT_NV`

- Extending VkStructureType:

  - `VK_STRUCTURE_TYPE_MEMORY_GET_REMOTE_ADDRESS_INFO_NV`

  - `VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_EXTERNAL_MEMORY_RDMA_FEATURES_NV`

## Issues

## Examples

    VkPhysicalDeviceMemoryBudgetPropertiesEXT memoryBudgetProperties = { VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_MEMORY_BUDGET_PROPERTIES_EXT };
    VkPhysicalDeviceMemoryProperties2 memoryProperties2 = { VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_MEMORY_PROPERTIES_2, &memoryBudgetProperties };
    vkGetPhysicalDeviceMemoryProperties2(physicalDevice, &memoryProperties2);
    uint32_t heapIndex = (uint32_t)-1;
    for (uint32_t memoryType = 0; memoryType < memoryProperties2.memoryProperties.memoryTypeCount; memoryType++) {
        if (memoryProperties2.memoryProperties.memoryTypes[memoryType].propertyFlags & VK_MEMORY_PROPERTY_RDMA_CAPABLE_BIT_NV) {
            heapIndex = memoryProperties2.memoryProperties.memoryTypes[memoryType].heapIndex;
            break;
        }
    }
    if ((heapIndex == (uint32_t)-1) ||
        (memoryBudgetProperties.heapBudget[heapIndex] < size)) {
        return;
    }

    VkPhysicalDeviceExternalBufferInfo externalBufferInfo = { VK_STRUCTURE_TYPE_PHYSICAL_DEVICE_EXTERNAL_BUFFER_INFO };
    externalBufferInfo.usage = VK_BUFFER_USAGE_TRANSFER_SRC_BIT | VK_BUFFER_USAGE_TRANSFER_DST_BIT;
    externalBufferInfo.handleType = VK_EXTERNAL_MEMORY_HANDLE_TYPE_RDMA_ADDRESS_BIT_NV;

    VkExternalBufferProperties externalBufferProperties = { VK_STRUCTURE_TYPE_EXTERNAL_BUFFER_PROPERTIES };
    vkGetPhysicalDeviceExternalBufferProperties(physicalDevice, &externalBufferInfo, &externalBufferProperties);

    if (!(externalBufferProperties.externalMemoryProperties.externalMemoryFeatures & VK_EXTERNAL_MEMORY_FEATURE_EXPORTABLE_BIT)) {
        return;
    }

    VkExternalMemoryBufferCreateInfo externalMemoryBufferCreateInfo = { VK_STRUCTURE_TYPE_EXTERNAL_MEMORY_BUFFER_CREATE_INFO };
    externalMemoryBufferCreateInfo.handleTypes = VK_EXTERNAL_MEMORY_HANDLE_TYPE_RDMA_ADDRESS_BIT_NV;

    VkBufferCreateInfo bufferCreateInfo = { VK_STRUCTURE_TYPE_BUFFER_CREATE_INFO, &externalMemoryBufferCreateInfo };
    bufferCreateInfo.size = size;
    bufferCreateInfo.usage = VK_BUFFER_USAGE_TRANSFER_SRC_BIT | VK_BUFFER_USAGE_TRANSFER_DST_BIT;

    VkMemoryRequirements mem_reqs;
    vkCreateBuffer(device, &bufferCreateInfo, NULL, &buffer);
    vkGetBufferMemoryRequirements(device, buffer, &mem_reqs);

    VkExportMemoryAllocateInfo exportMemoryAllocateInfo = { VK_STRUCTURE_TYPE_EXPORT_MEMORY_ALLOCATE_INFO };
    exportMemoryAllocateInfo.handleTypes = VK_EXTERNAL_MEMORY_HANDLE_TYPE_RDMA_ADDRESS_BIT_NV;

    // Find memory type index
    uint32_t i = 0;
    for (; i < VK_MAX_MEMORY_TYPES; i++) {
        if ((mem_reqs.memoryTypeBits & (1 << i)) &&
            (memoryProperties.memoryTypes[i].propertyFlags & VK_MEMORY_PROPERTY_RDMA_CAPABLE_BIT_NV)) {
            break;
        }
    }

    VkMemoryAllocateInfo memAllocInfo = { VK_STRUCTURE_TYPE_MEMORY_ALLOCATE_INFO, &exportMemoryAllocateInfo };
    memAllocInfo.allocationSize = mem_reqs.size;
    memAllocInfo.memoryTypeIndex = i;

    vkAllocateMemory(device, &memAllocInfo, NULL, &mem);
    vkBindBufferMemory(device, buffer, mem, 0);

    VkMemoryGetRemoteAddressInfoNV getMemoryRemoteAddressInfo = { VK_STRUCTURE_TYPE_MEMORY_GET_REMOTE_ADDRESS_INFO_NV };
    getMemoryRemoteAddressInfo.memory = mem;
    getMemoryRemoteAddressInfo.handleType = VK_EXTERNAL_MEMORY_HANDLE_TYPE_RDMA_ADDRESS_BIT_NV;

    VkRemoteAddressNV rdmaAddress;
    vkGetMemoryRemoteAddressNV(device, &getMemoryRemoteAddressInfo, &rdmaAddress);
    // address returned in 'rdmaAddress' can be used by external devices to initiate RDMA transfers

## Version History

- Revision 1, 2020-12-15 (Carsten Rohde)

  - Internal revisions

## See Also

VkMemoryGetRemoteAddressInfoNV, VkPhysicalDeviceExternalMemoryRDMAFeaturesNV, VkRemoteAddressNV, vkGetMemoryRemoteAddressNV

## Document Notes

For more information, see the Vulkan Specification

This page is a generated document. Fixes and changes should be made to the generator scripts, not directly.

## Copyright

Copyright 2014-2023 The Khronos Group Inc.

SPDX-License-Identifier: CC-BY-4.0