Post 80. The decoder counterpart to post 51’s GPU JPEG encoder: a complete baseline JPEG decoder in three compute passes. The CPU only walks the JFIF markers (DQT/DHT/SOF/DRI/SOS) and uploads tables — entropy decode, dequant+IDCT, chroma upsampling and YCbCr→RGB all run on the GPU.
$ ./app --verify photo.jpg
coefs: 0/6144000 mismatch
rgb: max |diff| 0 mean 0.0000
Bit-exact vs libjpeg-turbo on a 192-file corpus (gray, 4:4:4,
4:2:2, 4:2:0, restart intervals, odd dimensions, quality 5–100,
optimized Huffman tables, real-world photos) — same integer islow
IDCT, same “fancy” upsampling, same fixed-point color conversion.
Example: VulkanExamples/p80_gpujpegdec
Structure
JPEG → host: marker walk → DQT/DHT/sof tables, scan span,
restart-interval boundaries
dispatch 1: entropy — one invocation per restart interval
VLC bitreader → 16-bit fused Huffman LUT → quantized coefs
(2048-MCU chunks, bitstate+DC-pred handoff between generations)
dispatch 2: recon — one invocation per 8x8 block
dequant + libjpeg islow integer IDCT → component planes
dispatch 3: upsample — one invocation per pixel
fancy h2v1/h1v2/h2v2 triangle upsampling + fixed-point YCbCr→RGBRestart intervals are the parallelism
The encoder post manufactured parallelism by emitting restart
markers every MCU. The decoder inherits whatever the file says —
and it turns out the same RSTm markers are the only legal sync
points in the stream: at a restart, the bitstream re-aligns to a byte
boundary and the DC predictor resets to 0. So the work item is the
interval, not the MCU: an interval table (built by the host from
the DRI and the SOS scan) maps each invocation to a byte offset and
an MCU range. A 3000×2000 photo with DRI=1 has 23500 independent
intervals — dispatched in a single wave, the whole 4 MB entropy
stream decodes in ~14 ms (~430 MP/s, >10× libjpeg-turbo).
DRI=0 files are the opposite extreme: one interval, one serial
bitstream. There a single invocation walks ~2M symbols — latency-bound
at ~1.9 MP/s on a Renoir iGPU. (Production GPU decoders handle this
with speculative boundary resync — decode chunks at guessed offsets,
keep the chains that tile the stream exactly. That’s the known next
step; every other stage is already parallel either way.)
The watchdog problem — chunked generations
A serial invocation decoding 115K MCUs takes seconds — long enough
for the driver to kill the submission mid-flight, leaving an
untouched zero tail (the failure is non-deterministic: the cutoff
moves between runs). The fix: split each interval’s MCU range into
2048-MCU chunks forming an (interval × chunk) grid, and hand off the
bit-reader state (pos, buf, cnt, end) plus the four DC
predictors through an SSBO — each generation of chunks is a
separate queue submission, so no single submission runs long.
if (w.w == 0u) { // first chunk of an interval
b.pos = w.x; b.buf = 0u; b.cnt = 0; b.end = false;
pred = ivec4(0);
} else { // resume predecessor's state
uint s = (idx - nIv) * 8u;
b.pos = st[s]; b.buf = st[s+1]; b.cnt = int(st[s+2]);
b.end = st[s+3] != 0u;
pred = ivec4(int(st[s+4]), int(st[s+5]), int(st[s+6]), int(st[s+7]));
}One load per symbol: the fused 16-bit LUT
Classic Huffman decode walks mincode/maxcode/valptr length by
length — several dependent loads per symbol, fatal on a latency-bound
serial path. Instead the host precomputes a flat 64K-entry LUT per
table: index it with the next 16 bits of the stream, get everything
back in one u32:
- bit 31 set → fused:
[4:0]total bit length,[8:5]run,[12:9]size,[28:13]the already sign-extended coefficient value — valid wheneverhuffLen + extLen ≤ 16, which covers the overwhelming majority of symbols; - bit 31 clear → plain:
[4:0]code length,[15:8]symbol — rare long codes still work via a secondrext()peek.
int e = flut[uint(at) * 65536u + peekN(b, 16u)];
if (e < 0) { // fused: one load, everything resolved
eat(b, uint(e & 0x1F));
rr = (e >> 5) & 0xF; ss = (e >> 9) & 0xF;
val = ((e >> 13) & 0xFFFF) - 32768;
} else { // long code: symbol + explicit extension
int len = e & 0x1F;
if (len == 0) break;
eat(b, uint(len));
int sym = (e >> 8) & 0xFF;
rr = sym >> 4; ss = sym & 15;
val = (ss > 0) ? rext(b, ss) : 0;
}Coef writeback is similarly latency-aware: the buffer is pre-zeroed
with vkCmdFillBuffer, and each block accumulates one pending u32 in
registers — one plain store per word, no read-modify-write.
Bit-exactness is the hard part, not the shaders
The IDCT/upsample/color stages reproduce libjpeg-turbo’s arithmetic
exactly: the islow integer IDCT (CONST_BITS=13, PASS1_BITS=2),
the “fancy” triangle upsampler with its alternating +1/+2 /
+7/+8 rounding biases and edge replication, and the 16-bit
fixed-point YCbCr→RGB tables. Any deviation is instantly visible in
the verifier, which compares against jpeg_read_coefficients
(entropy stage) and jpeg_start_decompress (final pixels):
$ ./check.py
ok corpus/cj_rst1.jpg 97280 coefs
ok corpus/photo_1023x767_s2q75.jpg 1179648 coefs
...
192 pass, 0 fail (137 generated + 40 real + 15 reject)
ALL BIT-EXACT vs libjpeg
Comparing coefficients has one gotcha: libjpeg’s virt_barray dims
are ceil(downsampled/8) — not MCU-padded — while the GPU grid is
MCU-padded. The verifier maps each reference block into the padded
layout; read past libjpeg’s array and you’re diffing heap garbage.
Support matrix & honest limits
Baseline SOF0/SOF1, 8-bit precision, 1 or 3 components, all sampling factors, restart markers, fill bytes, optimized Huffman tables. Progressive, multi-scan, arithmetic-coded, >3-component and non-8-bit inputs are rejected cleanly by the marker walk — no out-of-bounds GPU access on malformed input.
- restart-heavy files: ~430 MP/s — intervals dispatch in parallel
- DRI=0: ~1.9 MP/s — serial VLC, memory-latency bound
- libjpeg-turbo (CPU): ~33 MP/s regardless of restart markers
For an image pipeline that controls its encoder (like post 51 or
VulkanoCam’s captures), emitting restart markers makes GPU decode
13× faster than libjpeg. For arbitrary camera files it’s a mixed bag:
many cameras do write restart intervals, but DRI=0 needs the
speculative-resync scheme to shine — a different article.