MRecalculate() fails after conversion from-to YV12

[fvisagie]

Hi All,

When converting from YV12 to RGB32 for a VirtualDub filter and back to YV12, MRecalculate() fails with "wrong pixel type". Without the conversions, everything works perfectly.

Even without the VirtualDub filter and just the conversions, this short extract fails similarly:

a = last
MSuper(hpad=16, vpad=16, pel=2, rfilter=4)
bvec1 = MAnalyse(isb=true, delta=1, blksize=16, overlap=8, temporal=true, lambda=500, pnew=25)
fvec1 = MAnalyse(isb=false, delta=1, blksize=16, overlap=8, temporal=true, lambda=500, pnew=25)
# ... other stuff
a.ConvertToRGB32().ConvertToYV12()
# ... other stuff
MSuper(hpad=16, vpad=16, pel=2, rfilter=4, levels=1)
bvec1 = MRecalculate(bvec1, thSAD=400, blksize=8, overlap=4, lambda=500, pnew=25)
fvec1 = MRecalculate(fvec1, thSAD=400, blksize=8, overlap=4, lambda=500, pnew=25)
# ...


How do these conversions cause this failure, and how do I get around it?

Many thanks,
Francois

[Gavino]

When converting from YV12 to RGB32 for a VirtualDub filter and back to YV12, MRecalculate() fails with "wrong pixel type".
Are you sure your starting clip is really YV12 and not YUY2?

[fvisagie]

No problem here. What is the version of your AviSynth and MVTools2?


Avisynth 2.58 and mvtools-v2.5.11.3. Also FFmpegSource2 2.17, see below.

@Gavino, IsYV12() does say so?

With the ColorBars() source I do not get the problem either. Also with DirectShowSource() there's no problem.

With FFMS2 there's no problem either, provided I do not do the conversions! So the problem seems to be the combination of FFMS2 AND the conversions? Could you please try

source = "Wide-angle lens blurring sample.wmv"
FFIndex(source=source, cachefile=source+".ffindex", indexmask=-1, dumpmask=0, errorhandling=3, overwrite=false)
Audio = FFAudioSource(source=source, track=-1, cache=true, cachefile=source+".ffindex", adjustdelay=-1)
Video = FFVideoSource(source=source, track=-1, cache=true, cachefile=source+".ffindex", seekmode=1, rffmode=0, width=-1, height=-1, colorspace="YV12")
AudioDub(Video, Audio)
#~ ColorBars(pixel_type="YV12")
a = last
Assert(IsYV12(a), "Source is not YV12")
MSuper(hpad=16, vpad=16, pel=2, rfilter=4)
bvec1 = MAnalyse(isb=true, delta=1, blksize=16, overlap=8, temporal=true, lambda=500, pnew=25)
fvec1 = MAnalyse(isb=false, delta=1, blksize=16, overlap=8, temporal=true, lambda=500, pnew=25)
a.ConvertToRGB32().ConvertToYV12()
MSuper(hpad=16, vpad=16, pel=2, rfilter=4, levels=1)
bvec1 = MRecalculate(bvec1, thSAD=400, blksize=8, overlap=4, lambda=500, pnew=25)
fvec1 = MRecalculate(fvec1, thSAD=400, blksize=8, overlap=4, lambda=500, pnew=25)
MDegrain1(a, last, bvec1, fvec1)


on this clip (http://www.mediafire.com/file/0lym5r657vplaa7/Wide-angle_lens_blurring_sample.wmv)?

At this point, what would you suggest?

Many thanks.

[Gavino]

The root of the problem is that FFMS2 delivers YV12 frames in the internal format CS_I420 (y-u-v, 4:2:0 planar), while most Avisynth filters (including the built-in ones, and specifically ConvertToYV12()) use the format CS_YV12 (y-v-u, 4:2:0 planar).

For most purposes, the distinction is immaterial and both will return true for IsYV12().
However, MRecalculate checks the actual (internal) pixel types and fails if they are not identical for the supplied super-clip and vector clip. (I suspect this is unnecessarily strict and it should simply check that they both satisfy IsYV12()).

A workaround is to use the following code after FFVideoSource to reformat the frames to the CS_YV12 format:
BlankClip(last, pixel_type="YV12").MergeLuma(last).MergeChroma(last)
This is lossless and pretty fast.

[fvisagie]

Thanks for the thorough explanation and excellent work-around, Gavino. Another superb and really helpful contribution from you.

[fvisagie]

A workaround is to use the following code after FFVideoSource to reformat the frames to the CS_YV12 format:
BlankClip(last, pixel_type="YV12").MergeLuma(last).MergeChroma(last)
This is lossless and pretty fast.

How does the fidelity - from a theoretical point of view - of this approach compare to that of

ConvertToRGB32().ConvertToYV12()?

[Gavino]

How does the fidelity - from a theoretical point of view - of this approach compare to that of

ConvertToRGB32().ConvertToYV12()?
Conversion YV12->RGB32->YV12 is not lossless for two reasons:
- rounding errors in YUV<->RGB conversions
- chroma blurring from resampling

See this thread.

[fvisagie]

Thanks, Gavino!

[IanB]

A zero cost solution would be to use Merge(1.0) instead of MergeLuma().MergeChroma() which cost 1 luma blit plus 2 chroma blits, i.e :-BlankClip(last, pixel_type="YV12").Merge(last, 1.0)


Merge(1.0) optimises into a pass through call to the merge clip :-PVideoFrame __stdcall MergeAll::GetFrame(int n, IScriptEnvironment* env)
{
if (weight<0.0039f) return child->GetFrame(n, env);
if (weight>0.9961f) return clip->GetFrame(n, env);

[Gavino]

A zero cost solution would be to use Merge(1.0) instead of MergeLuma().MergeChroma() which cost 1 luma blit plus 2 chroma blits, i.e :-BlankClip(last, pixel_type="YV12").Merge(last, 1.0)
Neat solution, Ian. :)

Interesting that the pixel_type gets changed from CS_I420 to CS_YV12, but the actual frame contents are completely unchanged.
Which neatly illustrates that inside Avisynth the two formats really are completely interchangeable.

[fvisagie]

Neat solution, Ian. :)


Ditto, thanks Ian.

[Gavino]

For anyone wanting some background details on the I420/YV12 distinction, see this wiki page:
http://avisynth.org/mediawiki/I420