Color banding and noise removal

[cretindesalpes]

Indeed, this looks correct. The filter order doesn’t matter much regarding the 8/16-bit pipeline. It’s more important to focus on the actual processing. Moreover, strengths and thresholds depends heavily on the filter order. Of course it’s easier to insert the 8-bit filters before the 16-bit ones. Debanding generally comes at the end because it may have to repair damages done by previous filters.

[LouieChuckyMerry]

Thanks for your answer, cretindesalpes, I appreciate it. I've been slowly learning a bit here and a bit there and wanted to make sure I was putting the pieces together correctly. Another question, if I may. I updated Dither a few days ago, from v1.25.1 to v1.27.1 (I'm a bit slow) and in the process unwittingly added the avstp.dll to my plugins folder. I typically use the basic above script with SEt's MT AviSynth 2.6 like so:

SetMemoryMax(2048)
SetMTMode(3,7)
LoadPlugin("F:\[0]StandAloneApps\MeGUI-2500(core)2443(data)0.3.5(libs)[Portable]\tools\lsmash\LSMASHSource.dll")
LWLibavVideoSource("D:\Temp\[0000]ReEncTemp\TheUnbearableLightnessOfBeing[S]{1988}[720p]\OriginalVideo,OriginalAudio[ViGi].mkv")
SetMTMode(2)
SMDegrain(tr=3,thSAD=400,RefineMotion=True,Plane=0,Chroma=False,Lsb=True,Lsb_Out=True,PreFilter=2)
F=DitherPost(Mode=-1)
S=F.FastLineDarkenMod()
D=MT_MakeDiff(S,F).Dither_Convert_8_To_16()
Dither_Add16(Last,D,Dif=True,U=2,V=2)
GradFun3(Lsb_In=True,Lsb=True)
# DitherPost
Dither_Out()

This has always given me good results (except for the occasional corrupt source), but with the avstp.dll in my plugins folder the output video had nasty artifacts every few seconds. Once I figured out that the avstp.dll was the cause of the glitches I found my way to your thread (http://forum.doom9.org/showpost.php?p=1564516&postcount=1) and, after reading the "Short user manuel" I don't understand why the avstp.dll would cause so much trouble. To my obviously wrong understanding it should help. Or is the problem that SMDegrain and FastLineDarkenMod aren't AVSTP enabled?

[Groucho2004]

To my obviously wrong understanding it should help. Or is the problem that SMDegrain and FastLineDarkenMod aren't AVSTP enabled?
avstp certainly helps when you don't use AVS MT. Before trying to figure out what the problem may be, what speeds do you get when you don't use MT? Make sure that you use mvtools2.dll from this post (http://forum.doom9.org/showthread.php?p=1386559#post1386559). Most of the plugins in your script (mvtools, dither) can actually run multi-threaded through avstp.dll, so that should be worth a try. Multi-threading Mvtools and Dither through avstp is very efficient, CPU usage/speed scale almost linear.

Edit: I did a couple of tests with your script, SMDegrain is quite the CPU hog. You may be better off using AVS MT and disabling avstp, for example using Boulder's method below.


My guess regarding the artefacts you're seeing - AVS MT sometimes doesn't play well with internally multi-threaded plugins. In case of dither.dll (and mvtools 2.6.0.5), the presence of avstp.dll enables the internal multi-threading.

[Boulder]

You can use the line "avstp_set_threads(x)" (without the quotes) to control the amount of internal threads being used with filters that utilize avstp.dll. I have "avstp_set_threads(1)" in my plugins folder as avstp.avsi so that is always the default, and then I can change it to something else in my script if needed. Whenever I switch off MT, I set the avstp threads higher to compensate. With QTGMC that has given a nice boost without the stability issues.

[cretindesalpes]

Check the result with avstp multithreading but without avisynth MT. MT is not reliable when scripts are complex and heavy. Try to increase the memory available for avisynth too.
EDIT: Do you know which plug-in or script part causes the glitches?

[Groucho2004]

Fixed in dither 1.27.1 (http://forum.doom9.org/showthread.php?p=1386559#post1386559):
Fixed a wrong code path in Dither_limit_dif16 causing a crash on CPU without AVX2 instruction set. Thanks to real.finder and Groucho2004 for their report.
Documentation typos fixed thanks to Colours.

Still crashing sometimes with mpc-be(hc) when closing the player (AltF4). Error 0xC0000005 (access violation).

[cretindesalpes]

Groucho2004: avs script + input video format?

[Groucho2004]

Groucho2004: avs script + input video format?
Input is 1080p AVC clip cut from BD. This happens with every source format/filter, by the way.

The only way I can get it to crash is MPC (HC or BE). With VDub, AVSMeter, AVSPMod it does not crash whatever I try.


For the 8 bit filter I used "sharpen()" for simplicity. I usually would use LSF().

LoadPlugin("E:\Apps\VideoTools\DGDecNV\DGDecodeNV.dll")
DGSource("F:\Aliens1\aliens1.dgi")
w = 1280
h = round((last.height() / 1.5) / 2.0) * 2
Dither_convert_8_to_16()
S16 = last
DitherPost(mode = -1)
sharpen(0.5)
Dither_convert_8_to_16()
s16.Dither_limit_dif16(last, thr = 1.0, elast = 2.0)
Dither_resize16(w, h, kernel = "Spline64")
DitherPost()

[LouieChuckyMerry]

avstp certainly helps when you don't use AVS MT. Before trying to figure out what the problem may be, what speeds do you get when you don't use MT? Make sure that you use mvtools2.dll from this post (http://forum.doom9.org/showthread.php?p=1386559#post1386559). Most of the plugins in your script (mvtools, dither) can actually run multi-threaded through avstp.dll, so that should be worth a try. Multi-threading Mvtools and Dither through avstp is very efficient, CPU usage/speed scale almost linear.

Edit: I did a couple of tests with your script, SMDegrain is quite the CPU hog. You may be better off using AVS MT and disabling avstp, for example using Boulder's method below.

My guess regarding the artefacts you're seeing - AVS MT sometimes doesn't play well with internally multi-threaded plugins. In case of dither.dll (and mvtools 2.6.0.5), the presence of avstp.dll enables the internal multi-threading.

I'll try to run some with-without MT tests this weekend, and I'm using the version of mvtools that you linked, thanks. And given that the artifacts occur only when the avstp.dll is in my plugins folder I'd reckon you're right about the not playing well.


You can use the line "avstp_set_threads(x)" (without the quotes) to control the amount of internal threads being used with filters that utilize avstp.dll. I have "avstp_set_threads(1)" in my plugins folder as avstp.avsi so that is always the default, and then I can change it to something else in my script if needed. Whenever I switch off MT, I set the avstp threads higher to compensate. With QTGMC that has given a nice boost without the stability issues.

Thanks for that, I'll check it out this weekend. Is it literally:

avstp_set_threads(1)

as an .avsi file? Also, if I used "avstp_set_threads(0)" would that "deactivate" it?


Check the result with avstp multithreading but without avisynth MT. MT is not reliable when scripts are complex and heavy. Try to increase the memory available for avisynth too.
EDIT: Do you know which plug-in or script part causes the glitches?

That's the thing I don't understand. Without the avstp.dll in my plugins folder the above script (Post #905) reliably outputs glitch-free video (dozens of movies, and I've watched most of them) using SEt's MT AviSynth 2.6, provided the source isn't somehow corrupt. It's only when I decided to ignorantly place the avstp.dll in my plugins folder a couple days ago, when updating SMDegrain and Dither, that I discovered this issue. It took me a bit of testing to figure it out, but it's definitely the avstp.dll that's causing the glitches, and that's obvious, major artifacts every 2-3 seconds.

This weekend I'll run some tests with the avstp.dll and without MT AviSynth and see what happens. Thanks for your help.

[cretindesalpes]

LouieChuckyMerry: this is my experience using MT. I had stability issues on complex scripts (before writing avstp).
And when there is a problem with avstp, it generally does not comes from avstp itself, but from the plug-in using avstp for multi-threading its algorithm. So knowing which plug-in creates the artefacts in conjunction with avstp is important for debugging. Dogway, who wrote the improved SMDegrain version, reported me strange artefacts using avstp (most likely in the MVTools) but unfortunately I never could reproduce them.

Groucho2004: Thanks. I tried your script but I couldn’t reproduce the crash (using MPC-HC from CCCP 2015-02-22 Beta with MadVR 0.88.14). Is it avs MT or a standard version?

[Groucho2004]

Groucho2004: Thanks. I tried your script but I couldn’t reproduce the crash (using MPC-HC from CCCP 2015-02-22 Beta with MadVR 0.88.14). Is it avs MT or a standard version?
I'm using the release MPC-HC v1.7.9. With AVS 2.6, and 2.6 MT, opening and closing (Alt-F4) the script several times will produce the crash.
With AVS+ r1576, things are slightly different. No crash, no error message, but mpc-hc stays in memory after closing.

I just tried with a simpler script:
w = 1280
h = round((last.height() / 1.5) / 2.0) * 2
Dither_convert_8_to_16()
Dither_resize16(w, h, kernel = "Spline64")
DitherPost()

Still crashes but possibly less often.

The best way to test this seems to be with AVS+ r1576 since it causes mpchc to crash (i.e stay in memory) more reliably.
As I mentioned, all this does not happen with the previous Dither tools.

In case you need this info, the CPU is a Sandy Bridge i5 2500K, OS is XPSP3.

[Boulder]

Thanks for that, I'll check it out this weekend. Is it literally:

avstp_set_threads(1)

as an .avsi file? Also, if I used "avstp_set_threads(0)" would that "deactivate" it?Yes, it's just that inside the .avsi file. Setting threads to 1 kind of disables internal multithreading, at least I've not had any glitches with that setting and I've used it for quite some time now.

Unfortunately MVTools is quite unstable, I'd say 95% of my crashes have pointed to mvtools2.dll. And Avisynth MT is a yet another story..

[Dogway]

I thought the avstp issues were resolved (see last post (https://web.archive.org/web/20130521103336/http://doom10.org/index.php?topic=2178.msg10388#msg10388))

I have been using it for a while now, latest SEt's MT on Win7 with avstp and no issues... I recommend you to try to nail it down. Take all the plugins out and test with a very simple script using avstp and mvtools2, something along those lines. Try also to use avisource, instead of LSMASHSource just in case, and encode to lossless instead of x264 (trying different things while still using avstp).

[LouieChuckyMerry]

LouieChuckyMerry: this is my experience using MT. I had stability issues on complex scripts (before writing avstp).
And when there is a problem with avstp, it generally does not comes from avstp itself, but from the plug-in using avstp for multi-threading its algorithm. So knowing which plug-in creates the artefacts in conjunction with avstp is important for debugging. Dogway, who wrote the improved SMDegrain version, reported me strange artefacts using avstp (most likely in the MVTools) but unfortunately I never could reproduce them.

I guess it's good that you couldn't reproduce them, ha ha. If I use the MinimumPlugins+AVSTP&SEtMTAviSynt (http://www.mediafire.com/download/ynuj3srin9xo9ft/MinimumPlugins+AVSTP&SEtMTAviSynth.7z) to encode this TestClip (http://www.mediafire.com/download/mz6r8ema5ck12zd/TestClipShawshank2.7z) (which is just one example of many) using the following script with either FFMS2 or L-SMASH Works:

SetMemoryMax(2048)
SetMTMode(3,7)
"SourcePath"
SetMTMode(2)
SMDegrain(TR=3,ThSAD=400,RefineMotion=True,Plane=0,Chroma=False,Lsb=True,Lsb_Out=True)
F=DitherPost(Mode=-1)
S=F.FastLineDarkenMod()
D=MT_MakeDiff(S,F).Dither_Convert_8_To_16()
Dither_Add16(Last,D,Dif=True,U=2,V=2)
### Add "SMode=1" to GradFun3, Someday ###
GradFun3(Radius=16,Lsb_In=True,Lsb=True)
### Preview Source OR Send 16-bit Output To x264 10-bit ###
# Trim()
# DitherPost()
Dither_Out()

with my i7 3820QM, 12 GB RAM rig, the output video has artifacts (more motion means more artifacts) every time. However, if I remove the AVSTP.dll then the output video has no artifacts.

I know this is probably a silly question (and most likely beyond my understanding), but what how do you use AVSTP to output video without artifacts? As Groucho2004 and Boulder suggested above?


Boulder: thanks for the explanation. Now to bed...

[cretindesalpes]

Dogway: Huhuhu my memory must be failing, I remember that we had this issue unresolved for a very long time but I forgot the end of the story…

LouieChuckyMerry: Thanks, I’ll try to reproduce it with your material.

[Dogway]

@cretindesalpes: I haven't seen this for a while but I got to reproduce the issue here with LouieChuckyMerry's source, still trying to figure out the culprit. Looks like the more complex the script is most likely are the artifacts to arise.

[Groucho2004]

with my i7 3820QM, 12 GB RAM rig, the output video has artifacts (more motion means more artifacts) every time. However, if I remove the AVSTP.dll then the output video has no artifacts.

I know this is probably a silly question (and most likely beyond my understanding), but what how do you use AVSTP to output video without artifacts? As Groucho2004 and Boulder suggested above?
With your set of DLLs and script, I can also reproduce the artefacts.
Solutions (tested):

Don't use avstp if you're using "SetMTMode(()" and vice versa. Besides, there is no advantage in using both anyway.
Don't use SMDegrain.


I guess (2) is not an option so just stick with (1).

[Dogway]

Groucho2004: Saying SMDegrain is a hog, is mostly saying mvtools2 is a hog... I don't think it's wise to tell people to not use one of the best temporal denoisers out there. In SMDegrain most of the code is for YUY2 and LSB support, as well as chroma switches to speed up the script, because of this chroma optimization code I have discovered some mvtools2 bugs here (http://forum.doom9.org/showthread.php?p=1551305#post1551305), as is possibly the case here too.

As you will see below there's not much going on inside the script.
SetMemoryMax(1024)
SetMTMode(5,6)
ffvideosource("TestClipShawshank2.mkv")
SetMTMode(2,6)

#SMDegrain(tr=3,thSAD=400,RefineMotion=True,Plane=0,Chroma=False,Lsb=True,Lsb_Out=True,PreFilter=2)
#the same as above
super_search = MinBlur(2,1).Dither_Luma_Rebuild(S0=1.0,c=0.0625,lsb=true,uv=1).MSuper(rfilter=4,pel=1,hpad=16,vpad=16,chroma=false)

bv3 = super_search.MAnalyse(isb = true, delta = 3, overlap=8, blksize=16, truemotion=false,chroma=false)
bv3 = super_search.MRecalculate( bv3, thSAD=200, overlap=4, blksize=8, truemotion=false,chroma=false)
bv2 = super_search.MAnalyse(isb = true, delta = 2, overlap=8, blksize=16, truemotion=false,chroma=false)
bv2 = super_search.MRecalculate( bv2, thSAD=200, overlap=4, blksize=8, truemotion=false,chroma=false)
bv1 = super_search.MAnalyse(isb = true, delta = 1, overlap=8, blksize=16, truemotion=false,chroma=false)
bv1 = super_search.MRecalculate( bv1, thSAD=200, overlap=4, blksize=8, truemotion=false,chroma=false)
fv1 = super_search.MAnalyse(isb = false, delta = 1, overlap=8, blksize=16, truemotion=false,chroma=false)
fv1 = super_search.MRecalculate( fv1, thSAD=200, overlap=4, blksize=8, truemotion=false,chroma=false)
fv2 = super_search.MAnalyse(isb = false, delta = 2, overlap=8, blksize=16, truemotion=false,chroma=false)
fv2 = super_search.MRecalculate( fv2, thSAD=200, overlap=4, blksize=8, truemotion=false,chroma=false)
fv3 = super_search.MAnalyse(isb = false, delta = 3, overlap=8, blksize=16, truemotion=false,chroma=false)
fv3 = super_search.MRecalculate( fv3, thSAD=200, overlap=4, blksize=8, truemotion=false,chroma=false)

MDegrain3(MSuper(levels=1,pel=1,hpad=16,vpad=16,chroma=false), bv1, fv1, bv2, fv2, bv3, fv3, plane=0, thSAD=400, lsb=true)

F=DitherPost(Mode=-1)
S=F.FastLineDarkenMod()
D=MT_MakeDiff(S,F).Dither_Convert_8_To_16()
Dither_Add16(Last,D,Dif=True,U=2,V=2)

GradFun3(Radius=16,Lsb_In=True,Lsb=True)

DitherPost(mode=6)


I normally process chroma so maybe there it goes why I never encountered this issue.

In this case (not processing chroma), if I turn Dither_Luma_Rebuild() "uv" parameter to 3 (process chroma) I don't get artifacts. If after MDegrain3() you just add the DitherPost() line you have some chances of not getting artifacts but in a real scenario you normally do some stuff after lsb_out... looks like some chroma-memory related issue in mvtools2.

[Groucho2004]

Groucho2004: Saying SMDegrain is a hog, is mostly saying mvtools2 is a hog... I don't think it's wise to tell people to not use one of the best temporal denoisers out there.
1. Your logic is flawed.
SMDegrain is a hog because it uses mvtools excessively. I can put 1000 PointResize() one after another and it will be very slow even though a single Pointresize() is very fast. If you use mvtools for a simple mdegrain with a delta of 1, it will also be fast.

2. At no point did I tell people to not use SMDegrain. I simply stated that it is slow. If you have trouble understanding my last post, let me put it in other words:
Either use SetMTMode() or avstp. If you use both, SMDegrain will cause artefacts.

[LouieChuckyMerry]

cretindesalpes: sorry for the trouble, I wish that I was capable of actually helping.

With your set of DLLs and script, I can also reproduce the artefacts.
Solutions (tested):

Don't use avstp if you're using "SetMTMode(()" and vice versa. Besides, there is no advantage in using both anyway.
Don't use SMDegrain.


I guess (2) is not an option so just stick with (1).

Thanks for the help, and you're right: two isn't an option (don't use SMDegrain? Never!), so I'll remove the AVSTP.dll from my plugins folder for now, then when able I'll try your and Boulder's earlier suggestions and try to learn more.

[feisty2]

switch to vaporsynth
solution once for all
:)

[foxyshadis]

If your GPU is reasonably powerful, you can swap SMDegrain with KNLMeansCL. SMDegrain (especially with dfttest) is great but it is slooooowwww, and KNLMeansCL is nearly as good.

It supports 16bit (and float in VS) natively (except not 16bit RGB64 in Avisynth, only YUV). Give it a shot, maybe it'll relieve the bottleneck.

[Groucho2004]

If your GPU is reasonably powerful, you can swap SMDegrain with KNLMeansCL. SMDegrain (especially with dfttest) is great but it is slooooowwww, and KNLMeansCL is nearly as good.
Indeed, this has been suggested before in a similar scenario (http://forum.doom9.org/showthread.php?p=1728514#post1728514).

[cretindesalpes]

LouieChuckyMerry: I think I nailed the problem. The culprit was DitherPost. It was a frame pointer released a bit too early, before the corresponding frame buffer was fully read. OK in a single-threaded Avisynth but definitely wrong in MT. Please test this dither.dll (http://ldesoras.free.fr/src/avs/dither-test.zip) and tell me if it works better on your side.

Groucho2004, maybe it addresses your issue too.

[Dogway]

it fixed it for me, thank you.

[Groucho2004]

Groucho2004, maybe it addresses your issue too.
No, still the same.

[LouieChuckyMerry]

LouieChuckyMerry: I think I nailed the problem. The culprit was DitherPost. It was a frame pointer released a bit too early, before the corresponding frame buffer was fully read. OK in a single-threaded Avisynth but definitely wrong in MT. Please test this dither.dll (http://ldesoras.free.fr/src/avs/dither-test.zip) and tell me if it works better on your side.

Thank you for that :) . I've presently a several-day encode running, but I'll do some testing as soon as it's finished (should be Saturday noonish) then post the results.

[LouieChuckyMerry]

Happy Saturday! I ran multiple tests with the new Dither.dll and the AVSTP.dll and there were no artifacts, only shiny shiny video. Thank you very much, cretindesalpes.

[real.finder]

Happy Saturday! I ran multiple tests with the new Dither.dll and the AVSTP.dll and there were no artifacts, only shiny shiny video. Thank you very much, cretindesalpes.

so, in this case there is no need for "the slice parameter to disable multi-threaded slicing for error-diffusion dithering (DitherPost and related functions)."

"Set it to false to disable plane slicing. Slicing is used in multithreading and can create artefacts at slice boundaries with error-diffusion algorithms. Disabling slicing reduces the threading preformances."

?

[Groucho2004]

Happy Saturday! I ran multiple tests with the new Dither.dll and the AVSTP.dll and there were no artifacts, only shiny shiny video. Thank you very much, cretindesalpes.
So, you're not using AVS MT any more?

:Edit
Just in case you're still using both, SetMTMode() and avstp:

I used your script (more or less) from post #917 and ran 2 tests with and without avstp:

Script:
SetMemoryMax(1024)
SetMTMode(5)
LoadPlugin("E:\Apps\VideoTools\DGDec\DGDecode.dll")
MPEG2Source("F:\Test\test.d2v", idct = 4)
SetMTMode(2)
SMDegrain(TR=3,ThSAD=400,RefineMotion=True,Plane=0,Chroma=False,Lsb=True,Lsb_Out=True)
F=DitherPost(Mode=-1)
S=F.FastLineDarkenMod()
D=MT_MakeDiff(S,F).Dither_Convert_8_To_16()
Dither_Add16(Last,D,Dif=True,U=2,V=2)
GradFun3(Radius=16,Lsb_In=True,Lsb=True)
DitherPost()

Result with avstp.dll present:
[Runtime info]
Frames processed: 2000 (0 - 1999)
FPS (min | max | average): 4.767 | 238636 | 22.04
Memory usage (phys | virt): 1076 | 1172 MB
Thread count: 12
CPU usage (average): 98%
Time (elapsed): 00:01:30.737


Result with avstp.dll removed:
[Runtime info]
Frames processed: 2000 (0 - 1999)
FPS (min | max | average): 4.947 | 238636 | 25.27
Memory usage (phys | virt): 1063 | 1171 MB
Thread count: 9
CPU usage (average): 95%
Time (elapsed): 00:01:19.130


These results make of course sense since using just SetMTMode() removes the complexity of two multi-threading "layers".

[cretindesalpes]

Thanks everyone for your reports. I’ll release a new official version later.

real.finder: these are not the same kind of artefacts. They’re almost invisible, unless you dramatically amplify the contrast to see the dithering patterns on large, flat areas (here (http://screenshotcomparison.com/comparison/136301), amplified 128x). They are caused by the error not diffusing between simultaneously processed areas.

[real.finder]

Thanks everyone for your reports. I’ll release a new official version later.

real.finder: these are not the same kind of artefacts. They’re almost invisible, unless you dramatically amplify the contrast to see the dithering patterns on large, flat areas (here (http://screenshotcomparison.com/comparison/136301), amplified 128x). They are caused by the error not diffusing between simultaneously processed areas.

I see, how about make slice=false by default in DitherPost mode 6 and up?

[cretindesalpes]

It disables avstp multithreading within the planes so it’s much less efficient in general. The visual quality gain is insignificant and mostly restricted to corner cases. Another solution is to add a very subtle noise, something like ampn=0.02, which keeps you away from most of the artefacts related to error diffusion algorithms.

[LouieChuckyMerry]

so, in this case there is no need for "the slice parameter to disable multi-threaded slicing for error-diffusion dithering (DitherPost and related functions)."

"Set it to false to disable plane slicing. Slicing is used in multithreading and can create artefacts at slice boundaries with error-diffusion algorithms. Disabling slicing reduces the threading preformances."

?

I'm really sorry, but I don't understand your question :o . I was referring to my issue from Post #917, that with that script, the AVSTP.dll, and the new Dither.dll there are no longer artifacts. Sorry for the confusion.


So, you're not using AVS MT any more?

:Edit
Just in case you're still using both, SetMTMode() and avstp:

I used your script (more or less) from post #917 and ran 2 tests with and without avstp:

See Above

These results make of course sense since using just SetMTMode() removes the complexity of two multi-threading "layers".

Thanks for the tests, the results are good to know. I'm still using SEt's MT AviSynth 2.6, without the AVSTP.dll in the Plugins folder, since my typical usage is the script from Post #917 and I never had the AVSTP.dll in my Plugins folder until I put it there, well, just because when updating Dither some days ago. I want to try the things you and Boulder suggested a few posts ago but there's not enough time in a day; every time I learn something new about AviSynth that leads to two more things, which leads to four more things... AviSynth³ ;) .

[Motenai Yoda]

@cretindesalpes

IIRC converting from YUV to RGB with Dither_convert_yuv_to_rgb() will clip into sRGB/bt.709 colorspace, there is a way to get an arbitrary colorspace (ie using bt.601 to get a bt.601's RGB, the same for bt.2020), or better in CIE 1976 (L*, u*, v*)?


ps how much Dither_min_dif16() is faster than ?


quant= int(pow(((2*radius)+1),2) - 1)

dark = Dither_median16(rx=radius, ry=radius, rt=0,ql=0,qh=0, y=3, u=1, v=1)
bright = Dither_median16(rx=radius, ry=radius, rt=0,ql=quant,qh=quant, y=3, u=1, v=1)

darkdiff = Dither_sub16(dark, sharp, wrap=False, y=3, u=1, v=1)
brightdiff = Dither_sub16(sharp, bright, wrap=False, y=3, u=1, v=1)

darkclamp = Dither_add16(sharp, darkdiff, wrap=False, y=3, u=1, v=1)
brightclamp = Dither_sub16(darkclamp, brightdiff, wrap=False, y=3, u=1, v=1)
clamp = (radius==1) ? Dither_Repair16(sharp, last, 1, -1) : brightclamp

[cretindesalpes]

Motenai Yoda: For BT.2020 (non-constant luminance version), it’s already possible, but assuming your RGB data is already in the BT.2020 gamut. The conversion to Luv is not linear so it’s not possible without a huge modification of the conversion function.

Dither_min_dif16 is fast and should be faster than the piece of code you quoted (I haven’t check what it does so I can’t confirm both are equivalent). But the best way to know is to check yourself.

[LouieChuckyMerry]

Hello :) . I'm encoding SD animation (early seasons Simpsons) and was wondering if there might be any noticeable benefit to upping the default Radius=12 to Radius=16? I've run tests and can't see a difference on my 14" 1600x800 laptop screen, but I'm hoping to someday own the world's biggest, er, a large screen TV, so I thought I'd ask before doing 200+ episodes in case I'd see the benefit on a much bigger panel. Thanks for your experienced opinion(s).

[foxyshadis]

If you can't see it at full size (bigger than your screen), you can rely on your future TV to handle it well. The only way it could be worse is if it's the cheapest 6-bit TN screen ever, which are getting hard to find, and even then the dithering from debanding will help.

Just make sure you double-check after the first episode that the compression didn't wipe out the debanding and make it ugly again. (If you're encoding to 8-bit.) That's why debanding on playback is useful.

[LouieChuckyMerry]

If you can't see it at full size (bigger than your screen), you can rely on your future TV to handle it well. The only way it could be worse is if it's the cheapest 6-bit TN screen ever, which are getting hard to find, and even then the dithering from debanding will help.

I'm really sorry, foxyshadis, but I'm not certain I understand the above. Do you mean that, although I'm currently unable to check the output on a larger screen, any reasonable large screen TV I might possibly own in the future will most likely benefit from raising the GradFun3 radius from 12 to 16 even though the source is SD? I just want to be sure ;) .


Just make sure you double-check after the first episode that the compression didn't wipe out the debanding and make it ugly again. (If you're encoding to 8-bit.) That's why debanding on playback is useful.

I'm encoding the DVD's with 10-bit x264 after denoising, sharpening, and debanding in a 16-bit pipeline (all hail Dither!). Debanding on playback sounds like a good future project...

[Groucho2004]

GradFun3
Alternatively, you might give "flash3kyuu_deband (http://forum.doom9.org/showthread.php?t=161411)" a try. It supports 16 bit stacked format, is very fast and in my opinion works very well.

[LouieChuckyMerry]

Alternatively, you might give "flash3kyuu_deband (http://forum.doom9.org/showthread.php?t=161411)" a try. It supports 16 bit stacked format, is very fast and in my opinion works very well.

But that would entail more testing! "Can't read you. Can't read you. Can't read you..."

[Groucho2004]

But that would entail more testing!
That's part of the fun, isn't it?

[LouieChuckyMerry]

Dang, you're too fast. Seriously though, what settings would you recommend that would be comparable to GradFun3(ThR=0.55,SMode=2,StaticNoise=True,Lsb_In=True,Lsb=True)? Also, so I can start simultaneous encodes before bed, do you think I'd see an improvement on a future 50" TV upping the GradFun3 radius from 12 to 16 with an SD source?

[Groucho2004]

Dang, you're too fast. Seriously though, what settings would you recommend that would be comparable to GradFun3(ThR=0.55,SMode=2,StaticNoise=True,Lsb_In=True,Lsb=True)? Also, so I can start simultaneous encodes before bed, do you think I'd see an improvement on a future 50" TV upping the GradFun3 radius from 12 to 16 with an SD source?
Never used GradFun3, so I can't recommend settings.

[cretindesalpes]

Increasing the radius helps with very progressive gradients, when the distance between the bands is large. It helps making the gradient really smooth without flat-gradient-flat-gradient staircases (bumps in the derivative function) which can be visible sometimes.

If you don’t get this kind of artefacts, there is no need to increase the radius.

[LouieChuckyMerry]

Never used GradFun3, so I can't recommend settings.

Ahhh, then given your experience with Flash3KYUUDeband could you recommend settings for SD animation (early seasons Simpsons) that you'd consider a good starting point for testing?


Increasing the radius helps with very progressive gradients, when the distance between the bands is large. It helps making the gradient really smooth without flat-gradient-flat-gradient staircases (bumps in the derivative function) which can be visible sometimes.

If you don’t get this kind of artefacts, there is no need to increase the radius.

So the increased radius is good for things like early morning-late evening skies and such in film, but wouldn't really do much for big blocks of solid colors found in animation? Thanks for the answer :) .

[Groucho2004]

Ahhh, then given your experience with Flash3KYUUDeband could you recommend settings for SD animation (early seasons Simpsons) that you'd consider a good starting point for testing?
For 16 bit processing, I usually use something like this:
f3kdb(sample_mode = 2, keep_tv_range = true, input_mode = 1, output_mode = 1)

Pretty much the defaults, does exactly what I want. I found the clips I linked to here (http://forum.doom9.org/showthread.php?t=172321) very useful for testing de-banding filters.

[Keiyakusha]

f3kdb at defaults adds quite a bit of grain on top of debanding. Personally I can't imagine using it without setting grainY=0, grainC=0. I think defaults were tweaked for 8bit video, but with 10bit video there are no problems retaining f3kdb's deband without any grain. At least when it comes to real-world content.

[foxyshadis]

I'm really sorry, foxyshadis, but I'm not certain I understand the above. Do you mean that, although I'm currently unable to check the output on a larger screen, any reasonable large screen TV I might possibly own in the future will most likely benefit from raising the GradFun3 radius from 12 to 16 even though the source is SD? I just want to be sure ;) .

I'm encoding the DVD's with 10-bit x264 after denoising, sharpening, and debanding in a 16-bit pipeline (all hail Dither!). Debanding on playback sounds like a good future project...

I mean, view it at 100% so there's absolutely no scaling or processing being applied. Sure, you can't fit it all on the screen at once, but you can scroll -- but if you're already encoding to 10-bit after debanding then I doubt you could possibly do better. Expanding the radius further just slows encoding down for a difference you might never notice even side-by-side.

Since your TV will likely never play AVC 10-bit, you'll have to play it from PC anyway, where you can apply further debanding or light noise. If you're encoding to HEVC 10-bit, a future TV will probably be able to play it, so just double-check the encode anyway and what you see is what you'll get, unless the TV has some kind of grain-generation.

[LouieChuckyMerry]

For 16 bit processing, I usually use something like this:
f3kdb(sample_mode = 2, keep_tv_range = true, input_mode = 1, output_mode = 1)

Pretty much the defaults, does exactly what I want. I found the clips I linked to here (http://forum.doom9.org/showthread.php?t=172321) very useful for testing de-banding filters.

Finally ran some tests. I prefer the results with GradFun3 but Flash3KYUUDeband is definitely faster and doesn't look at all bad. It's, to simplify, as if GradFun3 has relatively large areas of "movement" (blocking?) and Flash3KYUUDeband has relatively small areas of "movement".


f3kdb at defaults adds quite a bit of grain on top of debanding. Personally I can't imagine using it without setting grainY=0, grainC=0. I think defaults were tweaked for 8bit video, but with 10bit video there are no problems retaining f3kdb's deband without any grain. At least when it comes to real-world content.

Thanks! Now I have to run more tests ;) . This might explain my technical observation of large vs. small areas of "movement", too.

Edit: I ran some tests and much prefer the results with "GrainY=0,GrainC=0" added to the F3KDB script; the results are quite similar to those of GradFun3 (and still a bit faster, too). Thanks for the suggestion, Keiyakusha.


I mean, view it at 100% so there's absolutely no scaling or processing being applied. Sure, you can't fit it all on the screen at once, but you can scroll -- but if you're already encoding to 10-bit after debanding then I doubt you could possibly do better. Expanding the radius further just slows encoding down for a difference you might never notice even side-by-side.

Since your TV will likely never play AVC 10-bit, you'll have to play it from PC anyway, where you can apply further debanding or light noise. If you're encoding to HEVC 10-bit, a future TV will probably be able to play it, so just double-check the encode anyway and what you see is what you'll get, unless the TV has some kind of grain-generation.

For fear of being labeled more ignorant, how would I "view it at 100%"? I understand what you typed, but I'm not sure how to "upscale" the video.