Artifacts with XviD grayscale encoding

[helge79]

Hi!

I am getting some strange artifacts while encoding a black-and-white video. They persist even for high quality settings (the sample below has been encoded with an average quantizer 3.17 and maximal quantizer 5; I've been using XviD grayscale encoding mode). I am not sure what the technical term is, but some portions of the clip develop what looks like a pulsating cloud of haze hovering in front of the image (it is particularly obvious if you enlarge it to the full screen during playback). There is also quite a bit of posterization.

I hope some of you XviD gurus will be able to tell me what's going on. Thanks a lot in advance. Here are the corresponding samples of the original and the XviD output:

http://s31.yousendit.com/d.aspx?id=2U04R4K8DXZWK0WPAJMHODVNVF
http://s31.yousendit.com/d.aspx?id=0FAJK951O5J8Y3CEX1LX7BBA97

PS: You may notice that the brightness and contrast have been lightly adjusted before the encoding (with the VirtualDub curves filter), but I don't think it has anything to do with the problem...

[Koepi]

I didn't look at the samples (need to go to work now), but using virtualdub filters is the worst thing you could do. You get colour space conversions (YV12 -> RGB -> YV12) which could lead to artefacts, for example some material will look milky after such conversions.

Please head over to the avisynth forum to read about the YV12 process chain.

Cheers
Koepi

[helge79]

Well, I've never seen a curve filter for AviSynth (one that would be able to import a PhotoShop luminance curve), so I've used the VirtualDub filter that can do it (by Alexander Nagiller). I do not see any artifacts when the filter is applied in VirtualDub, they only appear after the XviD encoding. (Also, I am not sure if color space conversion could do anything to a black-and-white clip.)

I guess I should try to encode my sample without any filtering and see if the artifact persist, but I strongly suspect that it would... So, I am still hoping some of the experts will be able to download the samples and make a few comments.

Thanks for your input!


PS: The AviSynth script I've used is completely trivial:
LoadPlugin("C:\Program Files\GordianKnot\AviSynthPlugins\DGDecode.dll")
mpeg2source("koshikei.d2v")
Telecide(order=1,guide=1)
Decimate(5)
trim(0,85077)
crop(6,62,708,356)
BicubicResize(640,352,0.5)I've been loading it into VirtualDub, adding the curves filter and encoding with XviD-1.0.3

[Teegedeck]

You might find that the following AviSynth filters are more than a good replacement for what you've been using in VDub:

http://www.avisynth.org/Levels
http://www.avisynth.org/Tweak

And temporarily using that filter (remove it before encoding) helps a lot, too:

http://www.avisynth.org/Histogram

If you want to make sure that the clip is B/W add the line
Greyscale()

Edit: Though I see that this has nothing to do with your problem. The problem is terrible noise. The resulting 'swimming'-effect is quite a typical MPEG-4 phenomenon.

Basically, you have two options as far as I see it: Either you denoise the source with a powerful denoiser (PixieDust, RemoveDirt) or you use a high-bitrate matrix at a very low quantizer. And perhaps not using a higher VHQ mode than '1' would be a good idea. To skip resizing (only crop to a mod-16-resolution) could also help a bit, I guess.

[Wilbert]

Well, I've never seen a curve filter for AviSynth (one that would be able to import a PhotoShop luminance curve)
GiCoCu: C:\Program Files\AviSynth25\Docs\english\externalfilters\gicocu.htm

but that works also in RGB.

[helge79]

Edit: Though I see that this has nothing to do with your problem. The problem is terrible noise. The resulting 'swimming'-effect is quite a typical MPEG-4 phenomenon.

Basically, you have two options as far as I see it: Either you denoise the source with a powerful denoiser (PixieDust, RemoveDirt) or you use a high-bitrate matrix at a very low quantizer. And perhaps not using a higher VHQ mode than '1' would be a good idea. To skip resizing (only crop to a mod-16-resolution) could also help a bit, I guess.
Thank you very much, Teegedeck. This is precisely the kind of explanation I wanted the hear. In fact, I've suspected something along these lines, and I am very happy to see it spelled out so clearly.

However, I still have a few questions regarding your suggestions about what to do. So, if you (or someone else) have time, I would appreciate a few words about the following:

1) If I choose to denoise, would you anticipate anything about the relative role of spatial and temporal smoothing as far as getting rid of the "swimming" artifact is concerned. My clip is fairly low motion, so temporal filtering would be preferred (like I could use the temporal component of PixieDust, which is called FaeryDust, I think...)

2) If I choose to go with a high bitrate, my understanding is that you've suggested using a custom quantization matrix. Is there any specific matrix that you would recommend?

3) I never knew that choosing a lower VHQ mode may improve quality. Are there general criteria for when the higher VHQ modes should be avoided?

Thanks! :)


PS@Wilbert: Thanks a lot for pointing out the AviSynth filter!