GPU-assisted validation in Vulkan: catching descriptor and OOB errors with GPU-AV

CPU-side validation can’t see what the GPU does: a descriptor bound to an out-of-date buffer, an index buffer read past its end, a binding = -1 that only fires in one branch — all silent corruption or device loss on the real driver. GPU-AV (GPU-assisted validation, part of VK_LAYER_KHRONOS_validation) injects instrumented SPIR-V into your pipelines at create time and makes the GPU itself check every access.

Enabling it

// Instance layer + the GPU-AV validation feature:
const char* layers[] = {"VK_LAYER_KHRONOS_validation"};

vk::ValidationFeatureEnableEXT enables[] = {
    vk::ValidationFeatureEnableEXT::eGpuAssisted,               // instrument
    vk::ValidationFeatureEnableEXT::eGpuAssistedReserveBindingSlot,
        // reserves the last descriptor-set slot for GPU-AV's own
        // error-report buffer — needed if your app uses all slots
};
vk::ValidationFeaturesEXT vf{};
vf.setEnabledValidationFeatures(enables);
ici.pNext = &vf;   // chain into vkCreateInstance

GPU-AV ≠ debug printf. eDebugPrintf prints shader printf; eGpuAssisted rewrites your SPIR-V to emit errors into a GPU buffer the layer reads back — they are mutually exclusive ValidationFeature bits.

What it catches

descriptor OOB      texel buffer / SSBO / UBO / image index out of range
index buffer OOB    vkCmdDrawIndexed* reading past index buffer
descriptor indexing VK_EXT_descriptor_indexing + bindless arrays
                    (post-validate: gpuav_post_validate_descriptor_indexing)
ray queries         invalid accel structure / descriptor in RT shaders
device address      buffer-device-address OOB (gpuav_buffer_address_oob)

On error you get a validation message with shaderObject/pipeline name, the faulting descriptor binding + index, and the accessed vs. allowed range — something like VUID-VkDescriptorBufferInfo-range with binding=3 index=7 read 32 B past 128 B.

Tuning — layer settings

Fine-grained control via VkLayerSettingsCreateInfoEXT (chained into vkCreateInstance) or vk_layer_settings.txt / env vars:

const char* keys[] = {
    "gpuav_descriptor_checks",          // on: descriptor OOB
    "gpuav_buffer_address_oob",         // on: BDA OOB
    "gpuav_select_instrumented_shaders",// limit to listed shaders
    "gpuav_reserve_binding_slot",       // force slot reservation
    "gpuav_vma_linear_output",          // VMA-style linear allocs
    "gpuav_debug_dump_on_error",        // dump instrumented SPIR-V
};
// e.g. enable everything, instrument only the suspect pipeline:
// gpuav_select_instrumented_shaders = "my_shader.spv"

The most-used knobs:

• gpuav_descriptor_checks — full descriptor range validation (default on with eGpuAssisted).
• gpuav_buffer_address_oob — VK_KHR_buffer_device_address OOB — catches “pointer walks off the buffer” that nothing else sees.
• gpuav_select_instrumented_shaders — comma-separated shader module names/SPV checksums; instruments only those — halves the overhead on a big engine.

Cost and limits

- every access gets bounds-check + error-write code injected:
  ~1.5-3× shader execution time, more register pressure
- extra memory: one report buffer + per-pipeline instrumented module
- requires descriptor indexing on the app side (bindless path)
- only shaders that pass spirv-val get instrumented —
  broken SPIR-V falls back to CPU-side only

Practical rules

• Run GPU-AV on a debug build, one frame at a time — it’s too slow for interactive, but fatal for the bugs it finds.
• Reserve the binding slot (eGpuAssistedReserveBindingSlot) — without it GPU-AV steals the last set and can hide your descriptor bugs.
• gpuav_debug_dump_on_error writes the instrumented SPIR-V — disasm it with spirv-dis to see exactly where the check fired.
• If GPU-AV finds nothing but the device still dies → the bug is below shader visibility (fd/DMA-Buf lifetime, driver); switch to VK_EXT_device_fault + breadcrumbs (see the DEVICE_LOST post).
• Pair with spirv-val on your output shaders first — instrumenting a broken shader wastes a run.