YUV "Banding"

[Xesdeeni]

In the interest of speed, I am trying to maintain YUV all the way through my script. But I'm seeing a weird banding effect, that looks like over-quantization or too few bits/pixel when frameserving YUV data from AVISynth. The really weird thing is that I see this in TMPGEnc and MediaPlayer, but not in VirtualDub. If I add ConvertToRGB(), the export looks fine. Any idea what's going on here?

Xesdeeni

[Didée]

A picture, please ...

Of course, you don´t have your desktop at 16bit?

[sh0dan]

Could you possibly post screenshots that show the difference.

Mediaplayer may look different, because it uses your hardware for drawing overlay - in that case it is a driver or hardware issue, but it's easier to see if we get a picture.

[WarpEnterprises]

This is because TMPG uses the default YUY2-codec which can be unable to convert the avisynth output correctly (as my matrox codec).
(In my case it looks like vertical bands in the colour channel).
Use ConvertToRGB (won't harm as TMPG needs to convert anyway)

[dividee]

http://forum.doom9.org/showthread.php?s=&threadid=27932

[Xesdeeni]

dividee - Thanks! That's probably what I'm looking for (although I'm running Windows ME not Windows XP at home). If not, I'll post some images from home.

WarpEnterprises - I've seen the note that TMPGEnc converts YUV to RGB in a couple of places. But I think it would be quite ridiculous if TMPGEnc did such a conversion on the front end, because it would just have to convert it back in order to do the actual encoding, since MPEG files are YUV. The YUV->RGB conversion would only be necessary for display of the frames during encoding, and I typically have it show only I pictures so that it should only have to do that conversion once every 18 frames. Of course, the 4:2:2 input would still have to be converted to 4:2:0 for the actual encoding, but that should be easier and less time-consuming than converting RGB->YUV 4:4:4->YUV 4:2:0.

Xesdeeni

[ErMaC]

It may seem silly that TMPGEnc does this, but it does. All VFAPI plugins, including the internal ones, report back RGB video. So opening an AVI file, or avs file, or mpg file, or whatever in TMPGEnc will report RGB colordata to TMPGEnc's engine, which then encodes to MPEG.

Yea it's stupid, but it's free so who's to complain.

[Xesdeeni]

OK, I'll bite. How do I get Windows ME to accept my change of this driver in the registry?

I've tried modifying the entry myself in RegEdit. I've tried creating a .inf file to modify it for me. I've deleted it; I've changed it; I've changed it and rebooted; I've deleted it, rebooted, and changed it....and more.

But obviously I haven't hit on the right combination, because even once it appears correct in RegEdit, once I load TMPGEnc up and view the image, it is still bad. And then when I refresh the registry entry, it's back to msyuv.dll!

Xesdeeni

[Koepi]

Use the installer i built (it's on that thread, and in doom9's download section,... and i could email it to you - but, why that if you can d/l it in the thread mentioned above?).

It'll setup everything for you as you need it.

Regards,
Koepi

[WarpEnterprises]

But once again: what's the sense of NOT using ConvertToRGB when feeding to TMPG?

[Koepi]

Saving CPU cycles maybe. Or just saving yourself that line in every AVS you create.

Plenty of reasons.

Regards,
Koepi

[Xesdeeni]

Koepi:Use the installer i builtSorry, I missed that one the first time around. So that will work for WinME as well?

WarpEnterprises:But once again: what's the sense of NOT using ConvertToRGB when feeding to TMPG?TMPGEnc may prefer RGB, but what about CCE? MainConcept? Must this conversion be in place for every encoder?

Xesdeeni

[WarpEnterprises]

AFAIK if the encoder calls for YUY2 there is no external codec involved and therefore no error (as with VD).
Do you have other experience?

[Koepi]

Warp,

you're mistaken.

Open an AVS in tmpegenc and you'll see (_without_ YUVtoRGB() before of course).

You think we would even write installers against a non-existing problem?

Regards,
Koepi

[Xesdeeni]

Well...****! The installer runs, and now it looks like the registry correctly links YUY2 to the huffman YUV codec, but I still get screwed up video in YUV mode. I did a test, and adding ConvertToRGB() didn't seem to measurably increase the overhead of my script, but it's really frustrating not to be able to beat this problem.

Xesdeeni