What is this weird artifacting?

[saint-francis]

Before:

http://i182.photobucket.com/albums/x270/saint-francis-photos/snapshot20080310182125.jpg

After:

http://i182.photobucket.com/albums/x270/saint-francis-photos/snapshot20080310182107.jpg

Script:

DirectShowSource("D:\undersiege\undersiege.mkv",fps=23.9759856527702,audio=false)
crop( 6, 0, -2, 0)
Spline36Resize(1280,720)
MT ("FFT3DFILTER(sigma=1.5, ncpu=2, bw=14, bh=14, ow=7, oh=7, bt=3, hr=1,sharpen=.4,dehalo=.7).TTempSmooth (maxr=6, lthresh=4, cthresh=5, lmdiff=2, cmdiff=3, strength=2, scthresh=12.0, fp=true, vis_blur=0, debug=false, interlaced=false)",3)


What is the... I don't know what to call it... rings... in the top left corner in the black? I'm assuming that it's related to the sharpening in FFT3DFilter but I'm not sure. Any ideas?

[Tack]

It's called banding. Try a search on that term here and you should find a lot of options.

[saint-francis]

I have scoured the board and there is a great deal about banding but I still can't make heads or tails of this issue. I have tried the same script with VAQ and no VAQ and no sharpening in FFT3DFilter at all and it still has the banding. I would like to add that with VAQ the file is roughly %50 larger. I tried it using Undot only with and without VAQ as a denoiser and there is no banding so the issue is somewhere in FFT3DFilter, TTempSmooth or both. The file made using Undot is 3X larger and is full of the nastiest grain so this isn't an option for me. does anyone have any suggestions as to why my denoising is causing this banding?

[setarip_old]

Hi!

Perhaps unrelated but, it appears to my tired old eyes that the second image is somewhat washed out vis-a-vis the first. Perhaps a reduction of brightness might "hide" the problem...

[saint-francis]

How would I reduce the brightness? Nothing on it on the AVS wiki.
But I still don't think that would take care of it personally. It doesn't seem to be much brighter to me if at all.

[[P]ako]

The filter to use is called Tweak:

http://avisynth.org/mediawiki/Tweak

[Sharktooth]

you can try adding some debanding filter (gradfun? (http://forum.doom9.org/showthread.php?t=108681))

[saint-francis]

you can try adding some debanding filter (gradfun? (http://forum.doom9.org/showthread.php?t=108681))

Damn! I had been fearing the day when I would need to use gradfun. I have a pretty decent processor but the encoding still takes hours. This particular movie encoded at 4.7 FPS! And that is with reduced x264 settings. OK. That will be my next experiment.

:thanks:

[check]

gradfun is pretty fast. Be aware it can increase the bitrate required rather amazingly if you want to actually have it useful. This is just because it turns the smooth bands of colour into semi random dither (a similar principle to how people turned 16bit JPEGs into 256colour GIFs).
You might prefer running it on playback only - it's in ffdshow.

[Comatose]

Note that x264 doesn't usually maintain dithering, which is what gradfun2db adds to "remove" banding.

To give x264 a hand, you'll need to enable Adaptive Quantization.
To do this, add --aq-strength 0.5 and --aq-sensitivity 13 to the commandline.

These seem to be the default values... to find the options, I had to cross reference Wikipedia, some Virtualdub-like program's settings and another site which I can't remember, so I'm not sure.

A good way to test would be to encode 10-20 seconds where there's banding, trying it without AQ, with, and playing with the settings until you get a pleasing result.


If you don't know, what AQ does is allocate more bitrate to parts of the image. Like check said, dithering can increase the bitrate required a lot.

AQ allows you (the codec) to allocate more bitrate to the dithered areas, instead of just dividing it evenly (?).


Also, you might want to try replacing fft3dfilter with something else. I've noticed that this filter in specific causes a lot of banding.

[Sharktooth]

AQ is enabled by default. so if you dont set any AQ options it will be always enabled (btw your options are correct).

[saint-francis]

To give x264 a hand, you'll need to enable Adaptive Quantization.
To do this, add --aq-strength 0.5 and --aq-sensitivity 13 to the commandline.
A good way to test would be to encode 10-20 seconds where there's banding, trying it without AQ, with, and playing with the settings until you get a pleasing result.


I tried it with and without AQ. See post #3

[saint-francis]

I couldn't get gradfun to remove the banding. I'm not sure it helped too much at all.

My script:

DirectShowSource("D:\underseige\underseige.mkv",fps=23.9759856527702,audio=false)
crop( 6, 0, -2, 0)
Spline36Resize(1280,720)
MT ("FFT3DFILTER(sigma=1.5, ncpu=2, bw=14, bh=14, ow=7, oh=7, bt=3, hr=1,sharpen=.4,dehalo=.7).TTempSmooth (maxr=6, lthresh=4, cthresh=5, lmdiff=2, cmdiff=3, strength=2, scthresh=12.0, fp=true, vis_blur=0, debug=false, interlaced=false)",3)
gradfun2db (1.7)

I would like to remove the issue at the source. Can anyone see where there is a problem in the denoising that is causing this?

[Sharktooth]

the problem is in the colorspace. YV12 is a limited colorspace, it's not so good for "gradients" and smoothing (dithering) helps a lot.
you removed the noise and the grain with your denoisers so there's no longer dithering. as a consequence, banding appears.
if gradfun doesnt help, the only thing you can do is to NOT denoise or use a much lighter denoiser.

[setarip_old]

Have you as yet tried the previously suggested "Tweak" filter?

[saint-francis]

Have you as yet tried the previously suggested "Tweak" filter?

Yes I did but without too much luck.

Here's an interesting new development though. I tried playing the .avs and there is significantly less banding than in the encoded file.

From .avs playback:

http://i182.photobucket.com/albums/x270/saint-francis-photos/avsplayback.jpg

From file encoded using the same .avs:

http://i182.photobucket.com/albums/x270/saint-francis-photos/snapshot20080311115725.jpg

Could there be something inherently wrong with my x264 script?

--crf 18.0 --keyint 240 --min-keyint 24 --ref 10 --mixed-refs --no-fast-pskip --bframes 16 --b-pyramid --b-rdo --bime --weightb --direct auto --filter -2,-1 --subme 6 --trellis 1 --analyse p8x8,b8x8,i4x4,i8x8 --8x8dct --me umh --threads auto --thread-input --progress --no-dct-decimate --no-psnr --no-ssim

I think I'm just going to give up on trying to denoise this. I'll just have to suffer through the grain. :(

[Sharktooth]

ah! the banding is AFTER encoding...
i thought it was on .avs playback.
however i can see banding/blocking in the .avs screenshot as well.
well, x264 tends to accentuate a bit those defects, so there's not much you can do except avoid denoising

OR

use a gradfun/deband during playback

OR

add noise during playback

maybe you can try try playing with blockbuster (using noise method) just after your denising filters.

[gav1577]

Are you using megui ? if so i have a profile i find works well for reducing banding without the use of any filters if u want to give it a try let me know

[Sharktooth]

i have my serious doubts a "profile" can fix banding... expecially if it's already in the .avs source...

[saint-francis]

You mean an x264 profile? How could it work? You can post it here if you want.
I am now looking using convolution3d and ttempsmooth and it works decently but doesn't denoise as well as FFT3DFilter with TTempSmooth. It also produces about 45% larger files then FFT3DFilter does alone (which is the pits).

[Sharktooth]

look at the source image, try to imagine to "flat" out the noise and you'll see the banding is even in the source...

[gav1577]

You mean an x264 profile? How could it work? You can post it here if you want.
I am now looking using convolution3d and ttempsmooth and it works decently but doesn't denoise as well as FFT3DFilter with TTempSmooth. It also produces about 45% larger files then FFT3DFilter does alone (which is the pits).

here it is use the prestige matrix or sharktooths matrix or even flat the choice is yours also try without any filters first let us know how you got on http://www.mediafire.com/?zgwy5lh2jp9

[saint-francis]

here it is use the prestige matrix or sharktooths matrix or even flat the choice is yours also try without any filters first let us know how you got on http://www.mediafire.com/?zgwy5lh2jp9

Correct me if I'm wrong but isn't the use of the CQM's for retaining grain and the like? :eek:

[gav1577]

Correct me if I'm wrong but isn't the use of the CQM's for retaining grain and the like? :eek:

yes that is correct the prestige cqm is for retaining grain. removing all the noise or grain increases banding and loss of detail my tests have shown. if you want to remove all noise and grain and banding while keeping detail without reintroducing grain or noise let me know how to do this because i spent hours upon hours trying to reduce banding and in the end the only way for me was to add grain or noise. i suppose altering colour levels may help im not sure i havent tried :)