When you try to transcode an old or exotic input codec directly with h264_vaapi on an AMD Radeon, FFmpeg may try to use the GPU to decode it too. If the VAAPI driver does not support the input codec, the filter graph fails and nothing is written. This post shows how to keep decoding on the CPU while still using the GPU for H.264 encoding.
The problem
A command like this fails for an MPEG-4 input:
LIBVA_DRIVER_NAME=radeonsi ffmpeg \
-hwaccel vaapi -hwaccel_device /dev/dri/renderD129 -hwaccel_output_format vaapi \
-i input.avi \
-c:v h264_vaapi -rc_mode CQP -global_quality:v 24 \
-c:a copy \
output-h264.mp4The error output looks like this:
[mpeg4 @ 0x5a0c496f8a00] No support for codec mpeg4 profile -99.
[mpeg4 @ 0x5a0c496f8a00] Failed setup for format vaapi: hwaccel initialisation returned error.
[mpeg4 @ 0x5a0c496fb800] Video uses a non-standard and wasteful way to store B-frames ('packed B-frames'). Consider using the mpeg4_unpack_bframes bitstream filter without encoding but stream copy to fix it.
Impossible to convert between the formats supported by the filter 'Parsed_null_0' and the filter 'auto_scale_0'
Link 'Parsed_null_0.default' -> 'auto_scale_0.default':
src: yuv420p
dst: yuv420p
[vf#0:0 @ 0x5a0c494a8780] Error reinitializing filters!
[vf#0:0 @ 0x5a0c494a8780] Task finished with error code: -38 (Function not implemented)
[vf#0:0 @ 0x5a0c494a8780] Terminating thread with return code -38 (Function not implemented)
[vost#0:0/h264_vaapi @ 0x5a0c494a7f80] [enc:h264_vaapi @ 0x5a0c494a8340] Could not open encoder before EOF
[vost#0:0/h264_vaapi @ 0x5a0c494a7f80] Task finished with error code: -22 (Invalid argument)
[vost#0:0/h264_vaapi @ 0x5a0c494a7f80] Terminating thread with return code -22 (Invalid argument)
[out#0/mp4 @ 0x5a0c49504640] Nothing was written into output file, because at least one of its streams received no packets.The giveaway is No support for codec mpeg4 profile ... and Failed setup for format vaapi: hwaccel initialisation returned error. The decoder cannot be hardware-accelerated, so the entire pipeline collapses.
The solution: software decode, then upload to VAAPI
Remove -hwaccel, create the VAAPI device explicitly, decode in software, convert the frames to nv12 and upload them to the GPU before h264_vaapi takes over:
export RENDER_DEVICE=/dev/dri/renderD128
LIBVA_DRIVER_NAME=radeonsi ffmpeg \
-init_hw_device vaapi=va:"$RENDER_DEVICE" \
-filter_hw_device va \
-i input.avi \
-vf 'format=nv12,hwupload' \
-c:v h264_vaapi -rc_mode CQP -global_quality:v 15 \
-c:a copy \
output-h264.mp4What the flags do
-init_hw_device vaapi=va:/dev/dri/renderD128creates a named VAAPI device calledvaon the Radeon render node.-filter_hw_device vatells the filter graph to use that device for hardware filters and the encoder.-i input.aviis now decoded by the CPU, because no-hwaccelis set.-vf 'format=nv12,hwupload'converts the softwareyuv420pframes to the VAAPI-compatiblenv12format and uploads them into GPU memory.-c:v h264_vaapiencodes the uploaded frames in hardware.-rc_mode CQP -global_quality:v 15uses CQP and a low quality value becauseh264_vaapialready has to carry the upload overhead; for this use case the result is usually high-quality.-c:a copycopies the audio without re-encoding.
Wrapper script
Save this as transcode-h264-software-decode-radeon.sh and run it with any input. It writes output-h264.mp4 in the current directory:
#!/bin/bash
set -euo pipefail
RENDER_DEVICE="${RENDER_DEVICE:-/dev/dri/renderD128}"
if [ $# -ne 2 ]; then
echo "Usage: $0 <input video> <output-h264.mp4>" >&2
exit 1
fi
input="$1"
output="$2"
LIBVA_DRIVER_NAME=radeonsi ffmpeg \
-init_hw_device vaapi=va:"$RENDER_DEVICE" \
-filter_hw_device va \
-i "$input" \
-vf 'format=nv12,hwupload' \
-c:v h264_vaapi -rc_mode CQP -global_quality:v 15 \
-c:a copy \
"$output"Example:
chmod +x transcode-h264-software-decode-radeon.sh
RENDER_DEVICE=/dev/dri/renderD129 ./transcode-h264-software-decode-radeon.sh input.avi output-h264.mp4Verification on this computer
The failing command for an MPEG-4 AVI produced:
[mpeg4 @ ...] No support for codec mpeg4 profile 0.
[mpeg4 @ ...] Failed setup for format vaapi: hwaccel initialisation returned error.
...
[out#0/mp4 @ ...] Nothing was written into output file, because at least one of its streams received no packets.
Conversion failed!The software-decode / hardware-encode command completed successfully:
Stream mapping:
Stream #0:0 -> #0:0 (mpeg4 (native) -> h264 (h264_vaapi))
Stream #0:1 -> #0:1 (copy)
Output #0, mp4, to 'testsrc_mpeg4_h264.mp4':
Stream #0:0: Video: h264 (High) (avc1 / 0x31637661), vaapi(tv, progressive), 1920x1080, q=2-31, 30 fps
Stream #0:1: Audio: aac (LC), 44100 Hz, mono, fltp, 69 kb/s
[out#0/mp4] video:590KiB audio:43KiB ... bitrate= 1046.8kbits/s speed=7.75xffprobe on the output confirmed codec_name=h264 and profile=High, produced by h264_vaapi.
On this particular PC the Radeon card is actually /dev/dri/renderD129 because /dev/dri/renderD128 belongs to the integrated Intel GPU. Always use the device that vainfo reports as your Radeon card.
