How to use timeline semaphores in Vulkan for value-based GPU synchronization

Example: VulkanExamples/p17-timeline

Post 17 in the series (8: frames in flight, 16: multi-pass). The frames-in-flight post needed one fence per frame slot plus acquire/render-done semaphore bookkeeping per swapchain image. Timeline semaphores (core since Vulkan 1.2) replace all of that with a single object holding a monotonically increasing 64-bit counter: submits signal a value, waits target a value — on the GPU and on the host.

Binary vs timeline

binary semaphoretimeline semaphore
statesignaled / unsignaledcounter, only grows
reusemust be waited before reusereuse freely — wait on a higher value next time
host waitno (that’s what fences are for)yes — vkWaitSemaphores on a value
“wait for submits 1..N”N fence waitsone wait on value N

The four API points

  1. Feature: VkPhysicalDeviceTimelineSemaphoreFeatures:: timelineSemaphore at device creation.
  2. Creation: VkSemaphoreTypeCreateInfo{semaphoreType = eTimeline, initialValue = 0} chained to VkSemaphoreCreateInfo.
  3. Submit: VkTimelineSemaphoreSubmitInfo (chained to VkSubmitInfo) carries waitSemaphoreValues / signalSemaphoreValues — one value per semaphore in the corresponding arrays.
  4. Host wait: vkWaitSemaphores({semaphore, value}) — the host blocks until the counter reaches the value. One call replaces the entire fence machinery; vkGetSemaphoreCounterValue polls non-blockingly.
example.cpp
vk::TimelineSemaphoreSubmitInfo tl{};
tl.setWaitSemaphoreValues(i).setSignalSemaphoreValues(i + 1);
vk::SubmitInfo si{};
si.setPNext(&tl)
    .setWaitSemaphores(sem)      // waits for counter >= i
    .setSignalSemaphores(sem);   // bumps counter to i+1
vk.queue.submit(si);             // no fence!

The demo submits a chain of four batches — batch i waits for value i-1 (its predecessor’s signal) before its vkCmdFillBuffer may run. Ordering is enforced entirely on the GPU; the host doesn’t round-trip between submits. Then one host wait on value 4 replaces four fence waits.

Build & run

build.sh
g++ -std=c++23 -O2 main.cpp -o app -lvulkan -lglfw -lshaderc
./app           # submits the chain, waits on value 4, prints readback
python3 check.py

The check

Deterministic: each batch fills its 4-byte slot with a known pattern. If the chain executed out of order or the host waited wrongly, the readback or the final counter value would differ.

check.py
#!/usr/bin/env python3
# SPDX-License-Identifier: CC0-1.0
# check.py — run the timeline-semaphore demo and verify:
#   1. exit 0
#   2. all 4 slots written in order (the chained waits worked)
#   3. the semaphore counter reached 4
import re
import subprocess
import sys


def main():
    exe = sys.argv[1] if len(sys.argv) > 1 else "./app"
    r = subprocess.run([exe], capture_output=True, text=True,
                       timeout=60)
    out = r.stdout + r.stderr
    print(out, end="")
    assert r.returncode == 0, f"exit {r.returncode}"

    m = re.search(r"readback: ([0-9a-f]+) ([0-9a-f]+) ([0-9a-f]+) "
                  r"([0-9a-f]+)", out)
    assert m, "no readback line"
    vals = [int(g, 16) for g in m.groups()]
    assert vals == [0x11111111, 0x22222222, 0x33333333, 0x44444444], \
        f"slots {vals}"

    m2 = re.search(r"counter: (\d+)", out)
    assert m2 and int(m2.group(1)) == 4, "counter != 4"
    print("OK: 4 chained submits ordered + completed via "
          "timeline values")


if __name__ == "__main__":
    main()
output.txt
$ ./app && python3 check.py
device: AMD Radeon RX 7800 XT (RADV NAVI32)
readback: 11111111 22222222 33333333 44444444
counter: 4 (expected 4)
OK: 4 chained submits ordered + completed via timeline values

When to keep binary semaphores anyway

Swapchain acquire/present must use binary semaphores — WSI doesn’t accept timeline values there. The common production pattern: binary semaphores at the WSI boundary, timeline semaphores for everything internal (frames-in-flight tracking, queue-to-queue, compute→graphics handoff).

Full source

main.cpp
// SPDX-License-Identifier: CC0-1.0
//
// Post 17: timeline semaphores. A binary semaphore is a one-shot
// signal; a TIMELINE semaphore is a monotonically increasing
// counter. Any submit can signal a specific value; any wait (device
// or HOST) can wait for a specific value. That replaces the
// fence-per-frame juggling from post 8 and enables submit chains
// that order themselves on the GPU.
//
// The demo submits 4 chained batches — batch i waits for value i-1
// and signals value i — each doing a vkCmdFillBuffer on its own
// 4-byte slot. The HOST then waits for value 4 (one call, no fence
// at all) and reads back the buffer.
//
//   ./app     prints the readback; check.py asserts it
//
// Build:  g++ -std=c++23 -O2 main.cpp -o app -lvulkan -lglfw -lshaderc

#include "vkmini.hpp"

#include <GLFW/glfw3.h>

int main() {
    vkmini::Vk vk;
    vk.createInstance({});
    vk.pickPhysicalDevice();

    // ---- device with the timelineSemaphore feature ---------------
    auto props = vk.phys.getQueueFamilyProperties();
    for (uint32_t i = 0; i < props.size(); ++i)
        if (props[i].queueFlags & vk::QueueFlagBits::eGraphics)
            vk.qfam = i;
    float prio = 1.0f;
    vk::DeviceQueueCreateInfo qi{};
    qi.setQueueFamilyIndex(vk.qfam)
        .setQueueCount(1)
        .setPQueuePriorities(&prio);
    vk::PhysicalDeviceTimelineSemaphoreFeatures tlFeat{};
    tlFeat.setTimelineSemaphore(true);
    vk::PhysicalDeviceDynamicRenderingFeatures dyn{};
    dyn.setDynamicRendering(true)
        .setPNext(&tlFeat);
    vk::DeviceCreateInfo di{};
    di.setQueueCreateInfos(qi).setPNext(&dyn);
    vk.device = vk.phys.createDeviceUnique(di);
    vk.queue = vk.device->getQueue(vk.qfam, 0);
    vk.pool = vk.device->createCommandPoolUnique(
        {vk::CommandPoolCreateFlagBits::eResetCommandBuffer,
         vk.qfam});

    // ---- the timeline semaphore: a counter, not a flag ------------
    vk::SemaphoreTypeCreateInfo typeInfo{};
    typeInfo.setSemaphoreType(vk::SemaphoreType::eTimeline)
        .setInitialValue(0);
    auto sem = vk.device->createSemaphoreUnique({{}, &typeInfo});

    // ---- output buffer: 4 slots, one per batch --------------------
    auto buf = vk.createBuffer(
        16, vk::BufferUsageFlagBits::eTransferDst |
                vk::BufferUsageFlagBits::eTransferSrc,
        vk::MemoryPropertyFlagBits::eHostVisible |
            vk::MemoryPropertyFlagBits::eHostCoherent,
        true);

    auto cmds = vk.device->allocateCommandBuffersUnique(
        {vk.pool.get(), vk::CommandBufferLevel::ePrimary, 4});
    for (int i = 0; i < 4; ++i) {
        cmds[i]->begin(
            {vk::CommandBufferUsageFlagBits::eOneTimeSubmit});
        // write (i+1)*0x11111111 into slot i
        cmds[i]->fillBuffer(buf.buf.get(), vk::DeviceSize(i) * 4, 4,
                            uint32_t(i + 1) * 0x11111111u);
        cmds[i]->end();
    }

    // ---- submit the chain ------------------------------------------
    // Timeline semaphore values ride along in VkTimelineSemaphore-
    // SubmitInfo, chained to VkSubmitInfo via pNext. Batch i waits
    // for the semaphore to reach i (i.e. batch i-1's signal) before
    // it may start — GPU-side ordering, zero CPU round-trips.
    for (uint64_t i = 0; i < 4; ++i) {
        uint64_t waitVal = i;      // wait for previous batch
        uint64_t sigVal = i + 1;   // then bump the counter
        vk::TimelineSemaphoreSubmitInfo tl{};
        if (i > 0) {
            tl.setWaitSemaphoreValues(waitVal);
        }
        tl.setSignalSemaphoreValues(sigVal);
        vk::SubmitInfo si{};
        si.setPNext(&tl);
        vk::PipelineStageFlags ws =
            vk::PipelineStageFlagBits::eTransfer;
        if (i > 0) {
            si.setWaitSemaphores(sem.get())
                .setWaitDstStageMask(ws);
        }
        si.setCommandBuffers(cmds[i].get())
            .setSignalSemaphores(sem.get());
        vk.queue.submit(si); // NOTE: no fence!
    }

    // ---- host-side wait on the VALUE, not on a fence --------------
    // One call waits for "all submits that signal <= 4" — equivalent
    // to waiting on 4 separate fences.
    uint64_t target = 4;
    vk::SemaphoreWaitInfo wi{};
    wi.setSemaphores(sem.get()).setValues(target);
    auto res = vk.device->waitSemaphores(wi, UINT64_MAX);
    if (res != vk::Result::eSuccess)
        throw std::runtime_error("timeline wait failed");

    auto* d = static_cast<const uint32_t*>(buf.mapped);
    printf("readback: %08x %08x %08x %08x\n", d[0], d[1], d[2], d[3]);
    printf("counter: %lu (expected 4)\n",
           vk.device->getSemaphoreCounterValue(sem.get()));
    return 0;
}

All source code in this series is released under CC0 1.0 Universal (CC0-1.0) — public domain dedication, no attribution required.

Next (final post): validation layers and debug labels — the tooling that makes all of the above debuggable.


Check out similar posts by category: C++ Vulkan