How to use a dedicated async compute queue in Vulkan (measured overlap)

Post 63. Discrete GPUs ship separate hardware engines for graphics and compute — but Vulkan only uses them if you submit to a different queue family. Get it right and your physics/culling runs while the frame renders. Get it wrong and everything serializes on one queue.

example.sh

$ ./app
graphics qfam=0  compute qfam=1
graphics: 811.6 us   compute: 688.2 us   overlap: +525.6 us
compute tagged 65536/65536 buffer entries
OK: true overlap

Example: VulkanExamples/p63_asynccompute

Both workloads carry vkCmdWriteTimestamp brackets — the overlap number is measured, not assumed.

Pick two queue families

main.cpp (device setup)

auto qprops = phys.getQueueFamilyProperties();
uint32_t gfxFam = /* first family with eGraphics */;
uint32_t compFam = /* first family with eCompute AND != gfxFam */;
// this GPU: qfam0 flags=0xf (gfx|comp|xfer|sparse)
//           qfam1 flags=0xe (comp|xfer|sparse) — dedicated, ×4

std::vector<vk::DeviceQueueCreateInfo> qis;
for (uint32_t fam : {gfxFam, compFam})
    qis.push_back({{}, fam, 1, &prio});
vk::DeviceCreateInfo di{};
di.setQueueCreateInfos(qis);  // BOTH families at device creation

Queues must be declared at vkCreateDevice time — you can’t get a queue from a family you didn’t ask for. Then each command pool is bound to its family:

example.cpp

auto gpool = device->createCommandPoolUnique({flags, gfxFam});
auto cpool = device->createCommandPoolUnique({flags, compFam});
auto gqueue = device->getQueue(gfxFam, 0);
auto cqueue = device->getQueue(compFam, 0);

Submit both, order with a timeline semaphore

main.cpp

// timeline semaphore: value N means "everything up to N done"
auto sem = device->createSemaphoreUnique(
    vk::SemaphoreCreateInfo{}.setPNext(
        vk::SemaphoreTypeCreateInfo{}
            .setSemaphoreType(vk::SemaphoreType::eTimeline)
            .setInitialValue(0)));

// compute queue: signal value 1
csi.setSignalSemaphores(sem.get())
   .setPNext(vk::TimelineSemaphoreSubmitInfo{}
                 .setSignalSemaphoreValues(uint64_t(1)));
cqueue.submit(csi);

// graphics queue: signal value 2 — submitted right after, GPU runs
// them in PARALLEL on different engines
gsi.setSignalSemaphores(sem.get()).setPNext(...values={2});
gqueue.submit(gsi);

The overlap measurement

Each command buffer wraps its work in vkCmdWriteTimestamp (eTopOfPipe/eBottomOfPipe). With both intervals on the same device clock:

example.cpp

int64_t overlap = std::min(tsGfxEnd, tsCompEnd)
                - std::max(tsGfxStart, tsCompStart);
// >0  → true parallel execution
// <=0 → they serialized (submission latency or single engine)

Gotcha: submit order doesn’t force overlap. If compute is shorter than graphics’ submission latency, it finishes before the graphics work even reaches the GPU — -28us gap in one of my runs. For meaningful benchmarks, make both chunks at least as long as the submit overhead and synchronize the start if you need precision (release the compute side on a semaphore the graphics side signals early).

When async compute actually pays

Post-processing chain on a separate engine — bloom/mips while the next frame’s geometry runs.
Simulation (particles, physics, cloth) overlapped with raster.
Culling/compaction feeding post 61’s indirect draws.
Not free: cross-queue sync costs a semaphore round-trip (~µs) and on some GPUs the “compute queue” shares SMs anyway — measure (as this example does) instead of assuming.