Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Usage

Reply
 
Thread Tools Search this Thread Display Modes
Old 25th March 2010, 20:40   #1  |  Link
osgZach
Registered User
 
Join Date: Feb 2009
Location: Waterloo, WI - USA
Posts: 650
Color banding and noise removal

Is it possible to deband a source and also denoise it?

Should I denoise first and then deband?

Using GradFun2DB for instance, it won't get rid of it but it will take the banding down a notch, however if I were to there apply something like LSFMod, or in general some other noise reducers / it will undo the bebanding( as it seems it just adds grain to break it up), and then the banding will come back with even more clarity and well defined borders.

So it ends up looking like an ugly gradient, for lack of a better term, with those wavey line borders and all that fun stuff.

What also puzzles me, is that even though I am doing no color mode conversions, etc to even create this banding (its present on the DVD source) when I watch either the DVD or a straight unfiltered encode of the same episode, you can almost not even tell it is there on the TV screen.

Why is that?

Any suggestions on some strong, but safe settings for GradFun2DB, or other x64 compiled filters for this (degrainmedian, etc?) would be preferred. I'll provide a source section once a test encode I am running is done, if its needed. But this was more of a general question on effective settings for some filters versus a plea for someone to do the work for me.
osgZach is offline   Reply With Quote
Old 27th March 2010, 00:18   #2  |  Link
OvejaNegra
ekTOMBE STUDIOS
 
OvejaNegra's Avatar
 
Join Date: Dec 2005
Location: Cuba
Posts: 252
After ALOT of battle agains grain VS Banding i can tell you this for sure:
You eat the grain, you ll get banding.
Some sources are encoded without dithering and have banding.
Some sources have no grain but are encoded with dithering to avoid banding (dithering in the end is noise like grain for you)

My actual choices (after MAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAANY teest with animes and grainy sources)

Opt A: Do a strong denoising using FFT3d or moderate denoising using vaguedenoiser or anything like that -> good compression (not too high bit rate needed, not too big file) -> banding -> enable deband filter on FFDSHOW duting playback and it works excellent.

Opt B: Calm the grain using some temporal grain compressor like TTEMPSMOTH (my favorite) or this:
http://forum.doom9.org/showthread.php?t=139766
or this:
http://forum.doom9.org/showthread.ph...rain+optimizer
that will give you a more stable grain
then -> encode using high bit rate (1800 kbps or more) using grain retention settings on X264 or qpel + MPEG matrix on XVID -> Bigger file -> no banding -> looks very nice to me (for anime i'm ok sometimes with the anime preset on x264, i don need the grain preset)


IF your source have banding -> subtle denoiser (there is ALWAYS some noise on DVDs) -> encode normally -> enable deband on ffdshow during playback.

Or

Encode using gradfun or THIS: GradFun2DBmod.v1.3.avsi (google) Your source will look grainy again but it wont have banding. Of course you need high bit rate for this.


But i can tell you this for sure >
DVD sources have grain or dithering (i cant speak for HD yet, ask other person) so Degrain or denoise + no banding is impossible.

Yes on the TV (a common TV) you can see the banding even if is an encode or the original DVD with banding.
When you are on your monitor maybe bacause a bigger luma range you can see the banding. If you have a flat display it will look worst.

It's up to you, keep the grain or remove it and leave the banding. If your DVD has banding, well use some of the solutions i tested.

Good luck


EDIT:
Read this:
http://forum.doom9.org/showthread.ph...rain+optimizer
__________________
So, it works or not???
OvejaNegra is offline   Reply With Quote
Old 27th March 2010, 16:03   #3  |  Link
cretindesalpes
͡҉҉ ̵̡̢̛̗̘̙̜̝̞̟̠͇̊̋̌̍̎̏̿̿
 
cretindesalpes's Avatar
 
Join Date: Feb 2009
Location: No support in PM
Posts: 610
(Original reply replaced by this Dither presentation.)


Download
Masktools 2 a48 or above is required. Previous Masktools versions will silently fail and cause the greenish screen of death. Please update your Masktools before installing the Dither script.

All these plug-ins include the -Vit- threading fixes.


What is it?

Dither is a package including several scripts and avisynth plug-ins. Some of them are modification of existing ones (dfttest and MVTools 2). I primarily wrote the Dither tools to fight color banding during denoising in an unusual way. But Dither can perform various operations :
  • Recover the high bitdepth data possibly contained in a noisy clip. This can be seen as an "inverse-dithering" operation. In computer graphics, people usually use a gaussian filter to turn halftones into shades of grey. We'll use denoisers here.
  • Dither a high bitdepth picture into a standard YV12. The picture may have been obtained with the inverse-dithering operation, the combination could be called "transdithering".
  • Smooth fine gradients to remove colorbanding, resulting in a high bitdepth picture which can be dithered later.
  • Performing basic operations (masking, curves...) on high bitdepth pictures, as they cannot be manipulated safely with conventional avisynth filters.


Why should I use it?

Because denoising and lossy compression can easily make colorbanding appear. By transdithering, it is possible to recover original gradients better than using a filter + GradFun2DB(mod). Moreover, classic error-diffusion dithering algorithms require a high bitrate during encoding, otherwise color banding appears again. Dither offers an ordered dithering (Bayer matrix), which is more resilient to data compression.


How can I use it?

The simplest way to transdither a relatively clean source, using the modified dfttest:
Code:
dfttest (sigma=2, lsb=true)
DitherPost ()
You can add a SmoothGrad() call between both statements to remove colorbanding present in the source. And because dfttest is a spatio-temporal filter, it would be better to also use motion compensation to increase filter accuracy and prevent ghosting.

For an already denoised source, use
Code:
GradFun3 ()
GradFun3 is intended to replace GradFun2DB(mod). Actually it works on the same principle and does almost the same things, but it uses ordered dithering by default, which is better for compressibility.
__________________
dither 1.27.2 for AviSynth | avstp 1.0.3 for AviSynth development | fmtconv r19 for Vapoursynth | trimx264opt segmented encoding

Last edited by cretindesalpes; 30th December 2015 at 23:51. Reason: Package update (v1.27.2)
cretindesalpes is offline   Reply With Quote
Old 9th May 2012, 03:18   #4  |  Link
06_taro
soy sauce buyer
 
Join Date: Mar 2010
Location: United Kingdom
Posts: 164
cretindesalpes, the BT.709 coefficients matrix used in dither package is:
kr = 0.2125
kg = 0.7154
kb = 0.0721

That is the outdated version in early draft of ITU.709, and the latest one in ITU.709-5 is:
kr = 0.2126
kg = 0.7152
kb = 0.0722

Hope to correct it in the next release.
06_taro is offline   Reply With Quote
Old 9th May 2012, 07:48   #5  |  Link
cretindesalpes
͡҉҉ ̵̡̢̛̗̘̙̜̝̞̟̠͇̊̋̌̍̎̏̿̿
 
cretindesalpes's Avatar
 
Join Date: Feb 2009
Location: No support in PM
Posts: 610
Thanks. I've taken the coefficients from the Avisynth documentation, without checking if they were right. It's fixed now, it will be available in the next release. I also corrected the Avisynth page.
__________________
dither 1.27.2 for AviSynth | avstp 1.0.3 for AviSynth development | fmtconv r19 for Vapoursynth | trimx264opt segmented encoding
cretindesalpes is offline   Reply With Quote
Old 3rd June 2012, 19:55   #6  |  Link
Revgen
Registered User
 
Join Date: Sep 2004
Location: Near LA, California, USA
Posts: 1,545
Quote:
Originally Posted by cretindesalpes View Post
The link isn't working for me. AVSTP link isn't working either.

EDIT

Link works now. Thank You.
__________________
Pirate: Now how would you like to die? Would you like to have your head chopped off or be burned at the stake?

Curly: Burned at the stake!

Moe: Why?

Curly: A hot steak is always better than a cold chop.

Last edited by Revgen; 4th June 2012 at 13:17.
Revgen is offline   Reply With Quote
Old 5th July 2013, 16:20   #7  |  Link
spawnbsd
Registered User
 
Join Date: Jun 2006
Posts: 32
cretindesalpes, now that x264 supports BT2020 colorimetry, any chance we can get an update to dither tools with proper BT2020 support ? =)
spawnbsd is offline   Reply With Quote
Old 7th July 2013, 17:43   #8  |  Link
cretindesalpes
͡҉҉ ̵̡̢̛̗̘̙̜̝̞̟̠͇̊̋̌̍̎̏̿̿
 
cretindesalpes's Avatar
 
Join Date: Feb 2009
Location: No support in PM
Posts: 610
Yes people already asked me about BT.2020. Sorry for my slow replies, I’m not visiting the forum regularly these days and didn't code much for the Dither tools either. I want to finish optimizing fmtconv for Vapoursynth first, then backport the changes to Dither and finally add new features. I may release a quickfix in the meantime for various problems reported with the latest Avisynth 2.6a4.
__________________
dither 1.27.2 for AviSynth | avstp 1.0.3 for AviSynth development | fmtconv r19 for Vapoursynth | trimx264opt segmented encoding
cretindesalpes is offline   Reply With Quote
Old 9th June 2014, 08:39   #9  |  Link
Sparktank
47.952fps@71.928Hz
 
Sparktank's Avatar
 
Join Date: Mar 2011
Posts: 853
Quote:
Originally Posted by cretindesalpes View Post
Dither 1.26.0
Quote:
Originally Posted by cretindesalpes View Post
dither-1.26.1
<strike>Is the update a minor thing? I hadn't even noticed it changed versions until I had to update my laptop (low priority udpates).</strike>

EDIT: Ah, the small change doesn't affect my normal work environment.

Here's what the changelog says in the doc for those who are interested:
Quote:
v1.26.1, 2014.06.05

Fixed a wrong saturation in Dither_sub16 on the last pixels of a clip whose width is not mod-8.
__________________
Win10 (x64) build 17134 | GPU Caps Viewer v1.39.0.0
NVIDIA GeForce GT 640 (GK107) 2047MB/DDR3 | (R398.11)
NTSC | DVD: R1 | BD: A

Last edited by Sparktank; 9th June 2014 at 08:43. Reason: found changelog
Sparktank is offline   Reply With Quote
Old 27th August 2014, 08:00   #10  |  Link
asarian
Registered User
 
Join Date: May 2005
Posts: 1,170
Quote:
Originally Posted by cretindesalpes View Post
(Original reply replaced by this Dither presentation.)


Download
Is it just me, or has the Dither download site disappeared?!
__________________
Gorgeous, delicious, deculture!
asarian is offline   Reply With Quote
Old 27th August 2014, 08:13   #11  |  Link
Sparktank
47.952fps@71.928Hz
 
Sparktank's Avatar
 
Join Date: Mar 2011
Posts: 853
Quote:
Originally Posted by asarian View Post
Is it just me, or has the Dither download site disappeared?!
Yes, it seems down (for now).
http://www.downforeveryoneorjustme.c...esoras.free.fr

Site Status reports a specific error: 500 (timeout)
http://www.sitestatus.net/

I'm sure it's temporary for now.

EDIT: teh actual domain "free.fr" is up.
http://www.isitdownrightnow.com/free.fr.html
__________________
Win10 (x64) build 17134 | GPU Caps Viewer v1.39.0.0
NVIDIA GeForce GT 640 (GK107) 2047MB/DDR3 | (R398.11)
NTSC | DVD: R1 | BD: A

Last edited by Sparktank; 27th August 2014 at 08:17.
Sparktank is offline   Reply With Quote
Old 27th August 2014, 08:17   #12  |  Link
asarian
Registered User
 
Join Date: May 2005
Posts: 1,170
Quote:
Originally Posted by Sparktank View Post
Yes, it seems down (for now).
http://www.downforeveryoneorjustme.c...esoras.free.fr

Site Status reports a specific error: 500 (timeout)
http://www.sitestatus.net/

I'm sure it's temporary for now.

EDIT: teh actual domain "free.fr" is up.
http://www.isitdownrightnow.com/free.fr.html
Thx.
__________________
Gorgeous, delicious, deculture!
asarian is offline   Reply With Quote
Old 27th August 2014, 09:31   #13  |  Link
asarian
Registered User
 
Join Date: May 2005
Posts: 1,170
Quote:
Originally Posted by cretindesalpes View Post
(Original reply replaced by this Dither presentation.)


Download
Masktools 2 a48 or above is required. Previous Masktools versions will silently fail and cause the greenish screen of death. Please update your Masktools before installing the Dither script.

Hmm, if I'm to believe my hover-text, my MaskTools are still at 1.15.10 (I could have sworn they were at 2 a48). They're on the same French site, though, which is still down. There used to be an AviSynth filter repository, iirc.
__________________
Gorgeous, delicious, deculture!
asarian is offline   Reply With Quote
Old 25th September 2015, 11:46   #14  |  Link
tormento
Acid fr0g
 
tormento's Avatar
 
Join Date: May 2002
Location: Italy
Posts: 926
Quote:
Originally Posted by cretindesalpes View Post
Any chance to get a x64 compile of them?

At least of modified MVTools.

Please. =P
__________________
@turment on Telegram

Last edited by tormento; 25th September 2015 at 11:48.
tormento is offline   Reply With Quote
Old 25th September 2015, 18:28   #15  |  Link
Reel.Deel
Registered User
 
Join Date: Mar 2012
Location: Texas
Posts: 1,085
Quote:
Originally Posted by tormento View Post
Any chance to get a x64 compile of them?

At least of modified MVTools.

Please. =P
Both x64 plugins can be found here: http://avisynth.nl/index.php/AviSynt...2B_x64_plugins
Reel.Deel is offline   Reply With Quote
Old 25th September 2015, 20:34   #16  |  Link
tormento
Acid fr0g
 
tormento's Avatar
 
Join Date: May 2002
Location: Italy
Posts: 926
Quote:
Originally Posted by Reel.Deel View Post
Both x64 plugins can be found here: http://avisynth.nl/index.php/AviSynt...2B_x64_plugins


It's a pity AviSynth+ has a problem of image corruption.
__________________
@turment on Telegram
tormento is offline   Reply With Quote
Old 26th February 2017, 11:42   #17  |  Link
tormento
Acid fr0g
 
tormento's Avatar
 
Join Date: May 2002
Location: Italy
Posts: 926
Quote:
Originally Posted by cretindesalpes View Post
Could you give it a bump to adapt to latest AVS+ with high bit support too?
__________________
@turment on Telegram
tormento is offline   Reply With Quote
Old 27th March 2010, 18:53   #18  |  Link
osgZach
Registered User
 
Join Date: Feb 2009
Location: Waterloo, WI - USA
Posts: 650
Interesting solution you have there.

As for this particular source, I decided to just let it be as it was too much confusion on my part. Its worth noting I don't watch much on my PC, I encode for playback on my WDTV Live on a large screen (sadly I don't know its size haha) HDTV. Compressibility isn't much of a concern as x264 under CRF gives close to DVD quality results in the upper range of sub-1000kbps (on the TV). But I'll probably come back to this for future encodes I know will be pretty noisy as they were poor transfers to DVD.
osgZach is offline   Reply With Quote
Old 29th March 2010, 09:43   #19  |  Link
Terka
Registered User
 
Join Date: Jan 2005
Location: cz
Posts: 676
could you please post same samples?
Terka is offline   Reply With Quote
Old 29th March 2010, 17:32   #20  |  Link
osgZach
Registered User
 
Join Date: Feb 2009
Location: Waterloo, WI - USA
Posts: 650
Clip #1 This is not an example clip, I was wondering however if this is an example of Halo artifacts? (the edges around the students portable game thing)

Clip #2 - This is the color banding (I hope) clip. It's 26MB and spans several scenes which each have varying degrees of banding. I have never been able to successfully filter them, or reduce noise without greatly enhancing the visibility of the bands.

These come from Death Note, Episode 1, R1 DVD
osgZach is offline   Reply With Quote
Reply

Tags
color banding, deblocking, noise reduction

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 07:06.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2018, vBulletin Solutions Inc.