Atomics in Vulkan shaders: OpAtomic*, ordering, scopes and coherent memory

Every shader invocation runs in parallel — two a = a + 1 on the same counter lose an update. Atomics fix that, but in Vulkan the op is only half the story: the scope says who else can see it, and the semantics say what order the surrounding reads/writes stay in.

The GLSL surface

layout(set=0,binding=0) coherent buffer C { uint v[]; } c;

atomicAdd(c.v[i], 1);        // OpAtomicIAdd
atomicAnd(c.v[i], mask);     // OpAtomicAnd  …Or/Xor/Min/Max/Exchange
atomicCompSwap(c.v[i], 0,1); // OpAtomicCompareExchange
// memoryBarrierBuffer() for ordering; barrier() for workgroup exec

VK_EXT_shader_atomic_float adds atomicAdd(float); VK_KHR_shader_atomic_int64 adds 64-bit ops — both need their feature bit enabled.

Scope — who sees it

Invocation   — only me (rarely useful)
Subgroup     — my wave/warp
Workgroup    — my dispatch block  (shared mem / workgroup counters)
QueueFamily  — whole queue family
Device       — every invocation on the device
Instance/System — host-visible devices (e.g. external memory)

Default for atomicAdd on a buffer is Device — correct but the widest scope; narrow to Workgroup for block-local counters and you skip the cross-SCU sync.

Semantics — the ordering half

Relaxed          — atomic itself only; no ordering (counter use)
Acquire          — reads after this see the writer's released writes
Release          — my prior writes are visible before this op
AcquireRelease   — both
SequentiallyConsistent — total order (overkill, expensive)

SPIR-V adds MakeAvailable/MakeVisible — the Vulkan-specific cache-flush that actually moves data between L2 and the read port; without them a “release” doesn’t flush, a plain “acquire” doesn’t invalidate.

// producer: write payload, then publish the flag
c.data[0] = payload;
memoryBarrier();               // release semantics
atomicExchange(c.ready, 1);    // MakeAvailable: data flushed

// consumer: spin on flag, then read
while (atomicCompSwap(c.ready, 1, 1) != 1);  // Acquire
memoryBarrier();               // MakeVisible: see the payload
payload = c.data[0];

The traps

• coherent qualifier ≠ synchronization — it forces the access to go to memory (no L1 caching) but gives you no ordering; a coherent non-atomic read+write still races.
• barrier() is execution order, not memory order — it syncs threads in a workgroup; memory visibility still needs memoryBarrier*() (which map to OpMemoryBarrier).
• shared vs buffer atomics are different scopes — shared is workgroup-scope and cheaper; a buffer atomic pays device scope.
• Atomics on images (imageAtomicAdd) are device-scope by spec — they’re for sparse/global counters, not per-workgroup scratch.

histogram/counter in a workgroup:  shared + Workgroup scope
global draw counter / allocator:   buffer + Device scope + Relaxed
flag/producer-consumer:            Release/Acquire + MakeAvailable/
                                   MakeVisible + coherent

Practical rules

• atomicCompSwap is the lock primitive — everything else (add/min/ max) is a hardware fast path; CAS is general but slower.
• 32-bit atomics are universal; 16-bit (VK_EXT_shader_atomic_float16/ shader_atomic_int16) and 64-bit (int64) are feature-gated.
• For profiling use VK_KHR_shader_clock timestamps instead of atomic counters — atomics serialize, clocks don’t.
• Debugging: a relaxed counter that’s right is often enough — don’t over-order; the memory-model post covers when acquire/release are actually needed.