AVS classic YUY2 vs YV16 handling bug?

[poisondeathray]

Unexpected difference between YUY2 and YV16. Converting back to YUY2 is equivalent.

Affects x86 2.6 vanilla, 2.6MT Set, 2.61 alpha (Did not test x64)

Does not affect AVS+ x64 (did not test AVS+ x86)



Colorbars(pixel_type="YUY2")
a=last

Colorbars(pixel_type="YUY2")
ConvertToYV16()
#ConvertToYUY2()
b=last

Overlay(a,b, mode="Difference", pc_range=true)

[raffriff42]

Classic Overlay converts everything to YV24 as an intermediate format, whereas AVS+ (I THINK) tries to avoid that conversion if possible - as in your example, where both sources are 4:2:2. So you would see some chroma resampling artifacts in classic AviSynth.

The wiki page might be wrong here, need to test...http://avisynth.nl/index.php/Overlay
bool use444 = true [WRONG?]
AVS+ If set to false, Overlay uses conversionless mode where possible instead of going through YUV 4:4:4.
However, for Luma and Chroma modes, RGB must be converted to YUV 4:4:4.

[poisondeathray]

Classic Overlay converts everything to YV24 as an intermediate format, whereas AVS+ (I THINK) tries to avoid that conversion if possible - as in your example, where both sources are 4:2:2. So you would see some chroma resampling artifacts in classic AviSynth.

The wiki page might be wrong here, need to test...

Sure, but why wouldn't 4:2:2 be processed to the 4:4:4 intermediate using the same method or algorithm ? Shouldn't it be identical ? ie Why would 4:2:2 => 4:4:4 in overlay be done differently for a vs. b ? You're starting with 4:2:2 in both cases. YV16 vs. YUY2 is just planar vs. interleaved. Functionally you would expect no difference, both are 4:2:2

Also, you can "see" differences without overlay . For example , load a and b in different tabs in avspmod and switch back & forth . Maybe it converts YUY2 vs. YV16 to RGB differently for the preview ? Even if you control the RGB preview conversion with avisynth, they are still different

Or are you saying only one is converted to the YV24 intermediate in the overlay operation ??

[poisondeathray]

The docs say all inputs are converted
http://avisynth.nl/index.php/Overlay

Input clips (base, overlay and mask) are converted to YV24 internally. The output is re-converted to the input colorspace (or to the output colorspace, if specified).



Ok, but if you manually ConvertToYV24(), you get no differences, as expected. So why is overlay converting to YV24 differently for one?

colorbars(pixel_type="YUY2")
converttoyv24()
a=last

colorbars(pixel_type="YUY2")
converttoyv16()
converttoyv24()
b=last

Overlay(a,b, mode="Difference", pc_range=true)

[pinterf]

Classic Overlay converts everything to YV24 as an intermediate format, whereas AVS+ (I THINK) tries to avoid that conversion if possible - as in your example, where both sources are 4:2:2. So you would see some chroma resampling artifacts in classic AviSynth.

The wiki page might be wrong here, need to test...

Avisynth+ is trying to avoid 444 conversion only for 'Blend', 'Luma', 'Chroma' (except for RGB where 444 conversion occurs at 'Luma' and 'Chroma'). For the other modes I've not yet done the necessary modifications.

444: conversionless of course
RGB 'luma' and 'chroma': converted to 444 (and use444=false is not allowed)
RGB 'blend': no 444 (no conversion)
RGB other modes: convert to 444
Y, 420, 422, 444 with 'Blend', 'Luma', 'Chroma': no 444 (no conversion)
All other cases: converted to 444

[pinterf]

Strange, looking at the source, the YV16 case is not handled at all.
This link is for the old non-MT Avisynth+ (at that time it was not differing from classic Avisynth):
https://github.com/pinterf/AviSynthPlus/blob/master/avs_core/filters/overlay/overlay.cpp#L336

EDIT: the main difference comes from the input side:
https://github.com/pinterf/AviSynthPlus/blob/master/avs_core/filters/overlay/overlay.cpp#L91
https://github.com/pinterf/AviSynthPlus/blob/master/avs_core/filters/overlay/overlay.cpp#L406
YV16 is converted to YV24 through ConvertToYV24.
YUY2 is converted to internal 4:4:4 by a fast internal converter, which simply duplicates the chroma pixels.
(YV12 is also uses a fast internal converter, which is different from a ConvertToYV24 function, because upsampling is a simple pixel quadrupling to have the 444. Then converting 444 chroma back to YV12 is a simple 2x2 pixel averaging)