Bug in XviD ColorSpace Conversion? RGB->YUV

[PowerK6]

hi,guys,I have found a problem about XviD's ColorSpace Conversion
I have a Source clip of Uncompressed RGB,it has no artifacts.But When I encoded it with XviD,i found some 2x2 pixels red sawtooth(?) on the edge of the racket not only in the input Preview window of VirtualDubMOD but also in the playback window of media player classic 6.4.8.2.

Is's interesting that when i force the XviD Dshow output as YV12 colorspace,is has different results on the Geforce2 MX 400 and Geforce 4 MX4000(ATI 9600 Pro)

Geforce2 MX 400:
Appications |sawtooth status

VirtualDubMOD preview window(VFW) |Yes
MPC Playback window(XviD Dshow force YV12 output)|Yes
MPC Playback window(XviD Dshow no force ) |Yes


Geforce4 MX 4000(ATI 9600 Pro):
Appications |sawtooth status

VirtualDubMOD preview window(VFW) |Yes
MPC Playback window(XviD Dshow force YV12 output)|No
MPC Playback window(XviD Dshow no force ) |Yes


the Uncompressed RGB Source Snapshot
http://test.sugang.com.cn/pot0.png

the encoded clip(XviD)
http://test.sugang.com.cn/pot1.png

if needed,you can download the Uncompressed RGB Source clip from here (http://test.sugang.com.cn/rgb.avi) 36.04 MB

P.S My XviD Encoder Seetings
Single-Pass,Quantizer 2;
no B-VOPs;
MPEG-Custom: CG Animation Matrix;
Motion Search Precision: 6-UltraHigh,VHQ Mode:4;
VHQ for Bframe off ,Chroma motion off;
Maximum I-Frame interval:200
Cartoon Mode off;Chroma optimizer disable;
Min-Max I,P quantizer is 1-3

So,any suggestion?

And sorry for my poor English


-------Edit---------
BTW: I have tested some other codec, Nero AVC¡¢WMV9¡¢VP6¡¢SysKin's SexH264,RealVideo 10,they are all have the issus except the VP6.

VP6.4.2.0

Appications |sawtooth status
VirtualDubMOD preview window |Yes
MPC Playback window |No


I have turned off the Deinterlacing of PostProcessing£¨By Default VP6 Decoder Settings,the Deinterlacing of PP is on £©.

My XviD Version : 1.1.0-beta1 Build by Koepi
VirtualDubMod Version : 1.5.10.1

A Friend of mine tell me that it may be a chroma upsample bug in some nvidia drivers, I have tested the 4x.xx(forgotten),52.16,61.76
it seems no help.

So I think,is's the decode's issue not the encode's ?

[Soulhunter]

Old problem... (http://forum.doom9.org/showthread.php?s=&threadid=63736) ;)

Simply use VMR9 in MPC to fix it!


Bye

[PowerK6]

Originally posted by Soulhunter
Old problem... (http://forum.doom9.org/showthread.php?s=&threadid=63736) ;)

Simply use VMR9 in MPC to fix it !!!

Bye

VMR7 or VMR9 (including windowed and renderless) seems no use.

thanks anyway.

[Ark]

It's the colorspace itself (YV12) that has this "problem" because chroma has 1/2 information than luma, so it creates blocks expecially on red or other strong colors.

You can avoid it by encoder side enambling chroma optimizer in the zone settings, it's intended just for this kind of issues.

[Soulhunter]

Originally posted by Ark

You can avoid it by encoder side enambling chroma optimizer in the zone settings, it's intended just for this kind of issues.

Erm, afaik XviD's chroma optimizer wont change the colorspace... :rolleyes:

Its not the encoder or decoder, its the bad chroma upsampling of the renderer!


Originally posted by PowerK6

VMR7 or VMR9 (including windowed and renderless) seems no use.


Hrm, for me it woks...

Overlay = bad chroma upsampling

VMR9 = proper chroma upsampling

Have you closed/restarted the player after changing the render mode?


Bye

[Soulhunter]

Did a small test... ;)


XviD -> Overlay = Bad

XviD -> VMR9 renderless = Bad

XviD -> VMR9 windowed = Good

XviD + Chroma optimizer -> Overlay = Bad

XviD + Chroma optimizer -> VMR9 renderless = Bad

XviD + Chroma optimizer -> VMR9 windowed = Good


RGB Source
http://img30.exs.cx/img30/9310/source3nd.png

RGB Source -> XviD -> Overlay
http://img70.exs.cx/img70/856/xvidoverlay6ru.png

RGB Source -> XviD -> VMR9 windowed
http://img70.exs.cx/img70/5455/xvidvmr9windowed1uv.png


Bye

[PowerK6]

Originally posted by Soulhunter
Did a small test... ;)


XviD -> Overlay = Bad

XviD -> VMR9 renderless = Bad

XviD -> VMR9 windowed = Good

XviD + Chroma optimizer -> Overlay = Bad

XviD + Chroma optimizer -> VMR9 renderless = Bad

XviD + Chroma optimizer -> VMR9 windowed = Good



My result is all bad ,faint!

[Soulhunter]

Originally posted by PowerK6

My result is all bad ,faint!

You use XviD's decoder, and I use ffdshow...

Install ffdshow and see if it solves ya problem!


Bye

[Soulhunter]

From irc://irc.freenode.com/xvid

<Soulhunter> PowerK6: still here?
<PowerK6> yes
<Soulhunter> K!
<Soulhunter> Just noticed ya name...
<Soulhunter> Afaik you cant get correct chroma upsampling in VDub...
<Soulhunter> Coz afaik VDub uses Overlay mode!
<Soulhunter> But in overlay mode you get b0ked chroma upsampling...
<Soulhunter> In MPC VMR9 should fix it...
<Soulhunter> Well, if you gfx supports it!
<Soulhunter> Maybe its 2 old!?
<Soulhunter> GF2MX, no?
<PowerK6> GF2
<PowerK6> MX 400
<Soulhunter> I also had a GF2MX...
<Soulhunter> With the same probelms...
<Soulhunter> After upgrading to a Radeon9200 all probs were gone!
<Soulhunter> Now, after getting a GF6600...
<Soulhunter> I have the same problem again... >.<
<PowerK6> ffdshow's libavcodec decoder has the same result on GF2MX
<PowerK6> Geforce 4 MX 4000 fix it
<Soulhunter> So...
<PowerK6> and also ATI Radeon 9600 Pro
<Soulhunter> Only @ the GF2MX its b0rked?
<Soulhunter> GF2MX = b0rked overlay mode... ^^
<Soulhunter> Hrm, I could try the XviD decoder as well!
<Soulhunter> Wait...
<PowerK6> yes GF4MX is OK on YV12
<PowerK6> ok
<Soulhunter> Yap!
<Soulhunter> Same result here...
<Soulhunter> With xvids decoder it looks b0rked!
<PowerK6> the same to me
<Soulhunter> But using "force YV12" helps!
<Soulhunter> Hrm... :\
<PowerK6> it's no use on my PC
<PowerK6> maybe it does matter with the version of the driver?
<Soulhunter> Ah...
<Soulhunter> ffdshow outputs YV12 @ default!
<Soulhunter> Force RGB in ffdshow...
<Soulhunter> its also b0rked!
<PowerK6> right
<Soulhunter> Hrm...
<Soulhunter> Conclusion?
<Soulhunter> VMR9 converts -> RGB...
<Soulhunter> So...
<Soulhunter> If you feed it with RGB...
<Soulhunter> it wont convert it!
<Soulhunter> So...
<Soulhunter> it wont upsample the chroma!
<Soulhunter> Means...
<Soulhunter> you have to feed VMR9 with YV12 to get correct upsampling!
<Soulhunter> CASE CLOSED... ^^
<PowerK6> how to?
<Soulhunter> eh?
<PowerK6> force dshow ouput @YV12
<PowerK6> and use VMR9 on MPC?
<Soulhunter> Yeah...
<Soulhunter> you have to force ALL decoders to output YV12 and use VMR9!
<PowerK6> wait....
<Soulhunter> ALL decoders means -> all decoders that're decoding YV12 content!
<Soulhunter> MPEG1/2/4...
<Soulhunter> etc.
<PowerK6> i force the XviD Dshow to output @ YV12...
<PowerK6> and use VMR9 windowed Render on MPC,while playing the encoded clip...
<PowerK6> it seems the VMR9 is not enable
<PowerK6> i mean ,the MPC--> Filters --> Video Render (not VMR9 windowed)
<Soulhunter> Wich card?
<Soulhunter> Have you restarted MPC?
<PowerK6> ? GF2 MX 400
<PowerK6> certainly
<Soulhunter> Maybe a card must have support for DX9 to use VMR9... :\
<Soulhunter> OLD GFX = NO DX9 = NO VMR9 !!!
<PowerK6> and your card?
<PowerK6> radeon 9200?
<Soulhunter> Hrm...
<Soulhunter> Cant remember...
<Soulhunter> B4 the GF6600 i also had a R9800XT... ^^
<Soulhunter> Soulhunter upgrades very often!
<PowerK6> hehe
<Soulhunter> Btw, i think we should post this stuff into the thread, no?
<PowerK6> ok,you do?
<Soulhunter> Yeah, K!

[stephanV]

the picture with VMR9 windowed is much dimmer than the others... the black is really awful (well... its not black anymore)

[Soulhunter]

Originally posted by stephanV

the picture with VMR9 windowed is much dimmer than the others...

Yeah, you have good eyes... ;)

But its easy to correct this behavior via ffdshow!


Bye

[Didée]

You make it almost sound as if using HW overlay, chroma upsampling was generally broken. That's of course not true. It's only crappy video drivers that are broken. And Nvidea leads the ranking of broken-ness ... they care for maximum 3D-fps, less for (2D-) quality ...

[stephanV]

Originally posted by Soulhunter
But its easy to correct this behavior via ffdshow !!!


I shouldnt have to :p

[Soulhunter]

Originally posted by Didée

You make it almost sound as if using HW overlay, chroma upsampling was generally broken. That's of course not true. It's only crappy video drivers that are broken. And Nvidea leads the ranking of broken-ness ... they care for maximum 3D-fps, less for (2D-) quality ...

Nah, read my posts again plz... ;)

<Soulhunter> I also had a GF2MX...
<Soulhunter> With the same problems...
<Soulhunter> After upgrading to a Radeon9200 all probs were gone!


Bye

[Soulhunter]

Originally posted by stephanV

I shouldnt have to :p

Hey, thats my line while deinterlacing NTSC/PAL mess... :D


Bye

[Ark]

Originally posted by Soulhunter
Erm, afaik XviD's chroma optimizer wont change the colorspace... :rolleyes:

I know, i meant that it can eliminate *blockiness* on some problematic areas (very strong red) ;)

...or it can reduce *almost* all blockiness, that's my result from various test... and with the same renderer used for playback...

(the videocard is a Matrox G550)

[Didée]

Originally posted by Soulhunter
Nah, read my posts again plz... ;)
And you do the same with mine :)

I wasn't saying "you say that overlay is always broken", but "you make it almost sound as if ... ". That's a difference, isn't it.

But then, the clarification on your side was "only" in that quote from IRC. The only things you stated *directly* were

Overlay = bad chroma upsampling

VMR9 = proper chroma upsamplingandXviD -> Overlay = Bad
XviD -> VMR9 renderless = Bad
XviD -> VMR9 windowed = Good
XviD + Chroma optimizer -> Overlay = Bad
XviD + Chroma optimizer -> VMR9 renderless = Bad
XviD + Chroma optimizer -> VMR9 windowed = Good
Sorry for being penetrant. :D

[Soulhunter]

Originally posted by Didée

But then, the clarification on your side was "only" in that quote from IRC...

Well, thats the reason why I've posted the IRC stuff here!

Originally posted by Didée

That's a difference, isn't it.


Erm, almost... ^^


Bye

[Soulhunter]

Originally posted by Ark

I know, i meant that it can eliminate *blockiness* on some problematic areas [...]

Ok, that are the usual compression artifacts (DCT blocks) then...

But the problem above would also occur with lossless YV12 compression!


Bye

[PowerK6]

I have done an interesting test as following:
the Source is the very uncompressed RGB clip that i was referred to
Codec :XviD 1.1.0-beta

GF4 MX4000
Driver Version 67.66 beta
XviD(no force)+MPC(overlay/VMR9) = Bad
XviD(force YV12)+MPC(overlay/VMR9) = Good
ffdshow(libavcodec)+MPC(overlay/VMR9) = Good
ffdshow(XviD 1)+MPC(overlay/VMR9) = Good

GF4 MX4000
Driver Version 61.76
the same to 67.66 beta

GF4 MX4000
Driver Version 56.56
the same to 67.66 beta


GF2 MX400
Driver Version 67.66 beta
XviD(no force)+MPC(overlay/VMR9) = Bad
XviD(force YV12)+MPC(overlay/VMR9) = Bad
ffdshow(libavcodec)+MPC(VMR9) = Good
ffdshow(XviD 1)+MPC(VMR9) = Good
ffdshow(libavcodec)+MPC(overlay) = Bad
ffdshow(XviD 1)+MPC(overlay) = Bad

GF2 MX400
Driver Version 61.76
the same to 67.66 beta

GF2 MX400
Driver Version 56.56
the same to 67.66 beta


And I asked a friend to help me to do this test(His Graphics Card is GF2 MX400 too)
the only difference is at the version 67.66 beta

his result:
GF2 MX400
Driver Version 67.66 beta
XviD(no force)+MPC(overlay/VMR9) = Bad
XviD(force YV12)+MPC(overlay/VMR9) = Bad
ffdshow(libavcodec)+MPC(VMR9) = Good
ffdshow(XviD 1)+MPC(VMR9) = Good
ffdshow(libavcodec)+MPC(overlay) = Good
ffdshow(XviD 1)+MPC(overlay) = Good

-------------------Edit----------------------------------
we have found the reason , see my post here (http://forum.doom9.org/showthread.php?s=&postid=599016#post599016)

[Ark]

Originally posted by Soulhunter
Ok, that are the usual compression artifacts (DCT blocks) then...

But the problem above would also occur with lossless YV12 compression !!!


Bye

Yes, even in .vob files there is this issue, as they're YV12 like XviD, but i remember some samples with/without chroma optimizer and the one with it is like your third "A" sample some reply up...:confused:

[RadicalEd]

It must be a common misconception that XviD's chroma optimizer deals with upsampling. All it actually does is zero the chroma in 100% or 0% luma areas, which is just a common sense thing really

[PowerK6]

I have found something new.

When I was loading the subtitle file(VSFilter) with the encoded clip at the same time,I found it OK!!
That is because the VSFilter output @ YUY2 by default, however, ffdshow output @ YV12! So i used the VSFilter->Colors->dynamic change to force YV12 output, it's Bad!

and more , I disabled the YV12 output @ ffdshow(force YUY2) , the result is
ffdshow(libavcodec)+MPC(VMR9) = Good
ffdshow(XviD 1)+MPC(VMR9) = Good
ffdshow(libavcodec)+MPC(overlay) = Good
ffdshow(XviD 1)+MPC(overlay) = Good

XviD(Force YUY2)+MPC(overlay/VMR9) = Bad


i was confused.

it seems that the xvid.ax has different colorspace-conversion than ffdshow.

[Soulhunter]

Some more tests...

XviD -> XviD decoder + YV12 + VMR9 Windowed = Ok
XviD -> XviD decoder + YUY2 + VMR9 Windowed = Bad
XviD -> XviD decoder + YV12 + VMR9 Renderless = Bad
XviD -> XviD decoder + YUY2 + VMR9 Renderless = Bad

XviD -> ffdshow + XviD1 + YV12 + Overlay mixer = Bad
XviD -> ffdshow + XviD1 + YV12 + VMR9 Windowed = Ok
XviD -> ffdshow + XviD1 + YUY2 + VMR9 Windowed = Bad

XviD -> ffdshow + libavcodec + YV12 + Overlay mixer = Bad
XviD -> ffdshow + libavcodec + YUY2 + Overlay mixer = Bad
XviD -> ffdshow + libavcodec + YV12 + VMR9 Windowed = Ok
XviD -> ffdshow + libavcodec + YUY2 + VMR9 Windowed = Bad


EDIT:

I have a GeForce 6600GT + DX 9.0c + latest beta drivers!


Bye

[PowerK6]

Originally posted by Soulhunter
Some more tests...

......
......

I have a GeForce 6600GT + DX 9.0c + latest beta drivers !!!


Bye

My Source is not the "Red Letter A " but my encoded(XviD) clip of Prince.Of.Tennis.

[Soulhunter]

Originally posted by PowerK6

My Source is not the "Red Letter A " but my encoded(XviD) clip of Prince.Of.Tennis.

Well, then do the test again with a red letter... :D

Thats the only way to get sure that its bad chroma-upsampling!


Bye

[Soulhunter]

Btw, I discovered something strange with VMR9... >.<

If I enable anti-aliasing @ my gfx settings, the pic flickers!

Its mainly visible in scenes with white text on black background!


Bye

[Koepi]

Nice hardware acceleration ;)

I seriously want to blame the vga/drivers here. That doesn't feel like xvid being the evil thing[tm], but it seems it shows up some bad behavior of the vga/drivers. [You know, like with the crashing computers which get maxed out in temperature...]

Cheers
Koepi

[Soulhunter]

Originally posted by Koepi
Nice hardware acceleration ;)

I seriously want to blame the vga/drivers here. That doesn't feel like xvid being the evil thing[tm], but it seems it shows up some bad behavior of the vga/drivers. [You know, like with the crashing computers which get maxed out in temperature...]

Cheers
Koepi

Right, this "anti-aliasing + VMR9 = flicker" bug has absolutely nothing to do with XviD!

Just thought I should post this here in the "Soulhunter says: Use VMR9" thread... :D


Bye