How to sync GPU and CPU clocks in Vulkan with VK_KHR_calibrated_timestamps

vkCmdWriteTimestamp gives you a number in the device clock domain. clock_gettime(CLOCK_MONOTONIC) gives you CPU time. Comparing them naively is meaningless — the device clock may drift, run at a different rate, or be unanchored. VK_KHR_calibrated_timestamps (was VK_EXT_calibrated_timestamps) asks the driver for both clocks at essentially the same instant, so you can translate.

Why

why.txt

GPU timestamp:  "frame done at device_t=98237462"  ← unknown epoch
CPU timestamp:  "at 14:03:22.417123"               ← CLOCK_MONOTONIC
calibrated pair: (device_t, monotonic_t) captured together → correlate

Without it: you can only diff GPU↔GPU or CPU↔CPU, never cross.

Discovering the domains

domains.cpp

uint32_t n = 0;
phys.getCalibrateableTimeDomainsKHR(&n, nullptr);
std::vector<vk::TimeDomainKHR> doms(n);
phys.getCalibrateableTimeDomainsKHR(&n, doms.data());
// eDevice            — vkCmdWriteTimestamp's domain
// eClockMonotonic    — Linux CLOCK_MONOTONIC
// eClockMonotonicRaw — unslept monotonic
// eQueryPerformanceCounter — Windows QPC
// ePresentStageLocalExt / eSwapchainLocalExt — present_timing domains

Taking a calibrated sample

calibrate.cpp

vk::CalibratedTimestampInfoKHR infos[] = {
    {vk::TimeDomainKHR::eDevice},
    {vk::TimeDomainKHR::eClockMonotonic},
};
uint64_t ts[2];  uint64_t maxDev;
dev.getCalibratedTimestampsKHR(2, infos, ts, &maxDev);

// ts[0] = device-clock now, ts[1] = CLOCK_MONOTONIC now,
// both captured "as close together as the driver can manage".
// maxDev = worst-case skew between the two reads (ns).

Now any future vkCmdWriteTimestamp result T maps to wall time: wall ≈ ts[1] + (T - ts[0]) — assuming the device clock doesn’t drift measurably between calls (take a fresh pair each frame).

Completing the triangle

VK_EXT_present_timing’s VkPresentStageTimeEXT.time uses a swapchain-local domain (eSwapchainLocalExt) — you query its mapping to timeDomainId via vkGetSwapchainTimeDomainPropertiesEXT, then calibrate that domain too:

loop.txt

submit ──► vkCmdWriteTimestamp      ──►  eDevice domain
        └► vkGetPastPresentationTiming ─► eSwapchainLocalExt
        └► wp_presentation / CLOCK_MONOTONIC
vkGetCalibratedTimestampsKHR on all three → one timeline, real latency

Practical rules

Take a fresh calibrated pair per frame if you’re correlating to present-timing — clock drift over seconds is visible.
maxDeviation is your error bar — if it’s larger than your measurement accuracy needs, the correlation is unreliable on that driver.
eDevice and eClockMonotonic is the minimal useful pair; add eSwapchainLocalExt when combining with VK_EXT_present_timing.
Not all devices expose all domains — query, don’t assume.
For pure “how long did the GPU take”, stay in eDevice — calibration is only needed when crossing to CPU/display clocks.