ffmpeg and VA-API: How to handle YUV 4:2:2 MJPG?

[LigH]

The question was originally posted in the German doom9/Gleitz board, but we seem to lack of experts, so I will forward it here (because the original poster is not fluent in English and also has to wait for 5 days after registration):

Hi,

my system:

Kernel: 4.16.11-1-ARCH
Resolution: 2560x1440
Terminal: xfce4-terminal
CPU: Intel i7-6700K (8) @ 4.200GHz
GPU: Intel HD Graphics 530
GPU: NVIDIA GeForce GTX 750 Ti
Memory: 31094MiB



vainfo
libva info: VA-API version 1.1.0
libva info: va_getDriverName() returns 0
libva info: Trying to open /usr/lib/dri/i965_drv_video.so
libva info: Found init function __vaDriverInit_1_1
libva info: va_openDriver() returns 0
vainfo: VA-API version: 1.1 (libva 2.1.0)
vainfo: Driver version: Intel i965 driver for Intel(R) Skylake - 2.1.0
vainfo: Supported profile and entrypoints
VAProfileMPEG2Simple : VAEntrypointVLD
VAProfileMPEG2Simple : VAEntrypointEncSlice
VAProfileMPEG2Main : VAEntrypointVLD
VAProfileMPEG2Main : VAEntrypointEncSlice
VAProfileH264ConstrainedBaseline: VAEntrypointVLD
VAProfileH264ConstrainedBaseline: VAEntrypointEncSlice
VAProfileH264ConstrainedBaseline: VAEntrypointEncSliceLP
VAProfileH264ConstrainedBaseline: VAEntrypointFEI
VAProfileH264ConstrainedBaseline: VAEntrypointStats
VAProfileH264Main : VAEntrypointVLD
VAProfileH264Main : VAEntrypointEncSlice
VAProfileH264Main : VAEntrypointEncSliceLP
VAProfileH264Main : VAEntrypointFEI
VAProfileH264Main : VAEntrypointStats
VAProfileH264High : VAEntrypointVLD
VAProfileH264High : VAEntrypointEncSlice
VAProfileH264High : VAEntrypointEncSliceLP
VAProfileH264High : VAEntrypointFEI
VAProfileH264High : VAEntrypointStats
VAProfileH264MultiviewHigh : VAEntrypointVLD
VAProfileH264MultiviewHigh : VAEntrypointEncSlice
VAProfileH264StereoHigh : VAEntrypointVLD
VAProfileH264StereoHigh : VAEntrypointEncSlice
VAProfileVC1Simple : VAEntrypointVLD
VAProfileVC1Main : VAEntrypointVLD
VAProfileVC1Advanced : VAEntrypointVLD
VAProfileNone : VAEntrypointVideoProc
VAProfileJPEGBaseline : VAEntrypointVLD
VAProfileJPEGBaseline : VAEntrypointEncPicture
VAProfileVP8Version0_3 : VAEntrypointVLD
VAProfileVP8Version0_3 : VAEntrypointEncSlice
VAProfileHEVCMain : VAEntrypointVLD
VAProfileHEVCMain : VAEntrypointEncSlice


Why is it not possible to decode this AVI file (https://drive.google.com/file/d/1PHjHzTn7GoBCGP9aaPj3UTkDX6hDKOD4/view?usp=sharing)?


General
Complete name : /media/a5_Lwork/va/test_01_.avi
Format : AVI
Format/Info : Audio Video Interleave
File size : 30.6 MiB
Duration : 4 s 120 ms
Overall bit rate : 62.3 Mb/s

Video
ID : 0
Format : JPEG
Codec ID : MJPG
Duration : 4 s 120 ms
Bit rate : 59.2 Mb/s
Width : 1 920 pixels
Height : 1 080 pixels
Display aspect ratio : 16:9
Frame rate : 25.000 FPS
Color space : YUV
Chroma subsampling : 4:2:2
Bit depth : 8 bits
Scan type : Progressive
Compression mode : Lossy
Bits/(Pixel*Frame) : 1.142
Stream size : 29.1 MiB (95%)

Audio
ID : 1
Format : PCM
Format profile : Float
Codec ID : 00000003-0000-0010-8000-00AA00389B71
Codec ID/Hint : IEEE
Duration : 4 s 120 ms
Bit rate mode : Constant
Bit rate : 3 072 kb/s
Channel(s) : 2 channels
Channel positions : Front: L R
Sampling rate : 48.0 kHz
Bit depth : 32 bits
Stream size : 1.51 MiB (5%)
Alignment : Aligned on interleaves
Interleave, duration : 45 ms (1.12 video frame)
Interleave, preload duration : 500 ms



$ ffmpeg -hwaccel vaapi -hwaccel_device /dev/dri/renderD129 -hwaccel_output_format vaapi -i test_01_.avi -f null -
ffmpeg version 4.0 Copyright (c) 2000-2018 the FFmpeg developers
built with gcc 8.1.0 (GCC)
configuration: --prefix=/usr --disable-debug --disable-static --disable-stripping --enable-avresample --enable-fontconfig --enable-gmp --enable-gnutls --enable-gpl --enable-ladspa
--enable-libass --enable-libbluray --enable-libdrm --enable-libfreetype --enable-libfribidi --enable-libgsm --enable-libiec61883 --enable-libmodplug --enable-libmp3lame
--enable-libopencore_amrnb --enable-libopencore_amrwb --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora
--enable-libv4l2 --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxcb --enable-libxml2 --enable-libxvid
--enable-nvenc --enable-omx --enable-shared --enable-version3
libavutil 56. 14.100 / 56. 14.100
libavcodec 58. 18.100 / 58. 18.100
libavformat 58. 12.100 / 58. 12.100
libavdevice 58. 3.100 / 58. 3.100
libavfilter 7. 16.100 / 7. 16.100
libavresample 4. 0. 0 / 4. 0. 0
libswscale 5. 1.100 / 5. 1.100
libswresample 3. 1.100 / 3. 1.100
libpostproc 55. 1.100 / 55. 1.100
Input #0, avi, from 'test_01_.avi':
Duration: 00:00:04.12, start: 0.000000, bitrate: 62304 kb/s
Stream #0:0: Video: mjpeg (MJPG / 0x47504A4D), yuvj422p(pc, bt470bg/unknown/unknown), 1920x1080, 59783 kb/s, 25 fps, 25 tbr, 25 tbn, 25 tbc
Stream #0:1: Audio: pcm_f32le ([3][0][0][0] / 0x0003), 48000 Hz, stereo, flt, 3072 kb/s
Stream mapping:
Stream #0:0 -> #0:0 (mjpeg (native) -> wrapped_avframe (native))
Stream #0:1 -> #0:1 (pcm_f32le (native) -> pcm_s16le (native))
Press [q] to stop, [?] for help
ffmpeg: gen8_mfd.c:2602: gen8_mfd_jpeg_bsd_object: Assertion `0' failed.
Abort (memory dump created)


bye ggrub

Avoiding further chroma subsampling to 4:2:0 would be preferred; but first of all, decoding this material is already an issue in certain combinations of ffmpeg parameters, resulting in green/purple garbage.

[SeeMoreDigital]

Shame the original poster is not using VirtualDub2, as it can open the mjpg.avi container file fine ;)

[LigH]

May work under Linux. But that's not his workflow... and VDub2 won't use VA-API. In the meantime ... he even wonders if it may be a bug in intel's VA-API part of the graphic driver.

[SeeMoreDigital]

If it helps... I'm currently using a very cheap Asus laptop with Intel(R) HD graphics, which can play and display the 'test 01' sample just fine....

[Rumbah]

I would guess that the GPU simply cannot decode 4:2:2 JPEGs.

I'm not even sure that it supports decoding MJPEG at all as I don't see a MJPEG entry point. Some MJPEGs have different headers for the JPEGs so that it's not even a valid JPEG on its own.

In addition to that I guess that the GPU just gets fed the direct video bitstream via the VA API. Even if the MJPEG only contains baseline JPEGs (the only ones supported) I doubt that a standard JPEG decoder could decode a MJPEG bitstream.


I'd just use software decode as I don't think that a GPU decode would be noticable faster/efficient for JPEGs on this CPU.

[LigH]

Indeed, before he knows whether the GPU is able or the driver lacks support ... software decoding may be the simpler solution, and possibly faster without the GPU transfer overhead.

[nevcairiel]

From a performance standpoint, (M)JPEG hardware decoding is usually not worth it anyway, unless you plan to do full zero-copy on GPU processing from decode to encode, and even then it will depend on the decoding speed of your CPU if its any faster.