Lumi masking + greyscale => chroma shifting at high quants with last dev builds

[Manao]

I made some tests with a black and white movie, with 14/11/02 Koepi's build, with and without lumi masking, greyscale and bframes. Each times, I did a First pass - quality mode @ 67.

It appeared that when lumi and greyscale are used together, green and purple show up in the movie. It's not a playback issue ( I tried with nic's dshow, ffdshow with several IDCT routines, and virtual dub ).

I also did a test with bframes (2,200), lumi and greyscale, but at quality 85, and the chroma problem disappeared ( or at least was almost invisible ). So it seems the problem only occurs when high quants are used.

Finally, I also tried lumi masking, bframes, and quality 40, and the green slightly shows up on some places, but I think here it's normal ( very high quants are used )

The same problem occured with 04/11/02 Koepi's build, with a regular encode ( same source, two pass, linear scaling, bframes, lumi and greyscale, fstpass size 1,1 Go ( if I remember well ), sndpass size 650000 ). I didn't try older builds.

[Manao]

I tried again a black and white movie, with Koepi's 16/11/02 build, with bframes 2,200, no lumi masking and no greyscale.

The green shows up when playbacked with BSPlayer ( whatever the filter is : ffdshow or nic's ), but not with Zoom Player or MPC, nor when opened with Vdub ( in fact, there is a _very slight_ chroma shifting, but not as strong as with BSPlayer).

Here again, the quants must be high ( first pass size 2.6 Go, downsized to 640 Mo - I didn't know movie would be such incompressible, even with c3d and a resolution of 576*240 )

Moreover, Vdub says that during the first 2000 frames, there is no I-frames ( whereas there were set to max interval 300 ). This part of the movie is beginning credits, without scrolling, but with different texts, all in white over black. I didn't set credits range when encoding.

Also, the first two frames when opened with Vdub are shown green. Worse, there is a yellow line at the left of the Vdub output, where the gray should be fair ( ie : when it should be dark, there is no yellow, it is dark ), and it's present on all the encodes which use bframes. I can post a screenshot if you want.

Finally, I must add that when I check 'Xvid' + 'use Xvid' in ffdshow, it doesn't work ( choppy playback, but not during all the movie, with every movie I made ). I don't know if this as a role to play in my problem, it is just to mention it.

This is what I tried to get rid of this problem :

* Reinstall Nic's Decoder, try to decode with it
* Install an old, bug free ( I hope ) version of ffdshow : 14/10/02
* Play with IDCT routine, box checking ( use xvid or not )
* Change the fourCC code to DivX, but the movie then doesn't playback

I read somewhere that BSPlayer doesn't use the same filter as the other player ( ie it uses xvid.dll whereas the others use xvid.ax, or the contrary I don't know ), so I think there is a problem with one of these two files ( EDIT : I don't know how it works with ffdshow, I don't find a ffdshow.dll ).

Of course, in my previous post, as I only tried Vdub on one test, perhaps all I said in it is wrong ( and guess what, I have no more the results files, but I will do it again ). But I'm sure that at least one time, I got green with Vdub.

[-h]

I'd have to put this under the astonishingly-weird category - if all chroma information is discarded (as it is in greyscale mode) there is nowhere for iDCT inaccuracies to creep in and cause artifacting. Having it being quantizer-sensitive is even stranger, since no chroma information is ever dequantized (all chroma blocks are skipped, so only transfer is done, no dequant).

I'll try to look into it soon, but I'm having some job problems at the moment (i.e. there aren't any jobs inside my travel radius).

-h