Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Usage

Reply
 
Thread Tools Search this Thread Display Modes
Old 19th May 2020, 21:18   #1  |  Link
samf88
Registered User
 
Join Date: Oct 2006
Posts: 19
Nasty noise on a bad DVD transfer

Hi, hope someone out there can help me out.

I've ripped a DVD which looks shockingly bad and I was hoping there was something I could do to clean it a little. I fear it's too far gone and I'd end up decimating what little detail there is, but maybe there's a magic filter chain out there that's perfect for making this situation slightly more bearable? I think this will be one of those shows that was edited on video so a lot of the damage may be permanent.

I've been dabbling with MDegrain3 which does clear up small parts of the picture on default settings, but everything else (such as TBilateral, VagueDenoiser, RemoveGrain) has to be ramped up to insane settings to even touch the sides of this awful noise, probably because each speck is massive. Does it have a name? I thought originally it was chroma noise, but now I'm not so sure.

This is a particularly bad frame, just using MPEG2Source and QTGMC's Very Slow preset.


Here's a video excerpt cut from the VOB, incase it's easier to identify frame-by-frame: https://drive.google.com/open?id=1Sr...fF7Rj8qShxhu3h

If anyone can shed some light - or even recommend a filter/chain that may help in some way - I'd really appreciate it. This show has a very special place in my heart and I wish it was treated better for its (now out of print) DVD release.

I use AVISynth+ 64 bit, if that matters.
samf88 is offline   Reply With Quote
Old 20th May 2020, 11:38   #2  |  Link
tebasuna51
Moderator
 
tebasuna51's Avatar
 
Join Date: Feb 2005
Location: Spain
Posts: 7,019
For what do you use QTGMC?
It is not interlaced, and you can use the QTGMC denoisers directly to see what is your prefered.
__________________
BeHappy, AviSynth audio transcoder.
tebasuna51 is offline   Reply With Quote
Old 20th May 2020, 13:06   #3  |  Link
samf88
Registered User
 
Join Date: Oct 2006
Posts: 19
Hi - thanks for replying!

QTGMC is being used to deinterlace the TFF footage which contains visible combing. I did try with EZDenoise early on in my quest but unfortunately it didn't really affect this type of noise on sensible values, and of course high values just turn it to a blur. More advanced settings - such as switching to dfttest - also don't seem to touch it.

I'm guessing I would need to combine a filter with some kind of edge mask to try and retain some detail, but I highly doubt I'll be able to get good results from this.

Last edited by samf88; 20th May 2020 at 13:33.
samf88 is offline   Reply With Quote
Old 20th May 2020, 16:21   #4  |  Link
hello_hello
Registered User
 
Join Date: Mar 2011
Posts: 4,860
Just throwing some scripts at it, I preferred the TemporalDegrains.

TemporalDegrain removes more noise than TemporalDegrain2, but it blurs more. You'd probably have to encode a bit to judge how it looks when the pictures are moving. MAA is from the AnimeIVTC script, or there's MAA2. It's one of real.finder's functions, as is DeHalo_alpha_mt
https://forum.doom9.org/showthread.php?t=174121

Some SMDegrain experts might come along with some clever denoising settings using it instead.

MAA()
TemporalDeGrain()
DeHalo_alpha_mt()

Source


TemporalDegrain


TemporalDegrain2

Last edited by hello_hello; 20th May 2020 at 16:25.
hello_hello is offline   Reply With Quote
Old 23rd May 2020, 15:31   #5  |  Link
VoodooFX
Banana User
 
VoodooFX's Avatar
 
Join Date: Sep 2008
Posts: 997
What about something like:

MCDegrainSharp( frames=1, bblur=0.2, csharp=0.0 )
QTGMC( Preset="Medium", InputType=1, Sharpness=0.6 )
Santiag(1,1, nns=4)

QTGMC "Medium" wouldn't blur fast motion areas much like other presets, but sometimes in can create some stabilization issues, when "Slower" stabilization is more precise/better.

Quote:
Originally Posted by hello_hello View Post
I preferred the TemporalDegrains.
Don't you get some strange "effects-bugs" with TemporalDegrain2? Every time I tried it I noticed something strange going on besides noise removal.
VoodooFX is offline   Reply With Quote
Old 24th May 2020, 13:19   #6  |  Link
samf88
Registered User
 
Join Date: Oct 2006
Posts: 19
Quote:
Originally Posted by VoodooFX View Post
What about something like:

MCDegrainSharp( frames=1, bblur=0.2, csharp=0.0 )
QTGMC( Preset="Medium", InputType=1, Sharpness=0.6 )
Santiag(1,1, nns=4)

QTGMC "Medium" wouldn't blur fast motion areas much like other presets, but sometimes in can create some stabilization issues, when "Slower" stabilization is more precise/better.
Many thanks for your contribution.

I'm seeing aliasing on the black/white stripey top with InputType=1, but InputType=3 seems to solve that. http://www.framecompare.com/image-co...rison/999F1NNU

I think I might have to treat the end credits differently to the rest of the script though. There does seem to be 50fps worth of unique frames there in the background graphic, although given that it's only end credits I might just leave it. http://www.framecompare.com/image-co...rison/999FMNNU InputType=1 does seem to look best for the text, but InputType=3 looks best for the graphics. (Link if anyone wants to investigate: https://drive.google.com/file/d/1ohK...ew?usp=sharing)
samf88 is offline   Reply With Quote
Old 24th May 2020, 19:03   #7  |  Link
hello_hello
Registered User
 
Join Date: Mar 2011
Posts: 4,860
Quote:
Originally Posted by VoodooFX View Post
Don't you get some strange "effects-bugs" with TemporalDegrain2? Every time I tried it I noticed something strange going on besides noise removal.
I haven't used it much, but I didn't see anything nasty. The main problem I have with it, is I don't seem to be able to adjust how much noise it removes to any great extent. When there's just the right amount of noise it works, but considering the denoising logic came from QTGMC you'd think it'd have something like the EzDenoise option where you can simply dial in how much noise removal you want.

One day I might pull QTGMC apart to see if I can do better. Probably not, but maybe one day I'll try.
hello_hello is offline   Reply With Quote
Old 20th May 2020, 16:28   #8  |  Link
samf88
Registered User
 
Join Date: Oct 2006
Posts: 19
Thanks! That's wonderful, gives me something to build on. I really appreciate your help, thank you. :-)

If anyone else has any alternative suggestions I'd still be happy to hear them.

Last edited by samf88; 20th May 2020 at 17:43.
samf88 is offline   Reply With Quote
Old 21st May 2020, 17:36   #9  |  Link
hello_hello
Registered User
 
Join Date: Mar 2011
Posts: 4,860
I had a silly idea. I can't say I've done it before myself (it'd be dog slow), and I don't know if there'd be a better way to go about it, but what if you only denoised the chroma, then denoised the chroma and luma? You could probably get away with less noise removal the second time around because it'd look more like grain. It's probably mainly an improvement for flat areas of color, such as the sky, and not so much everywhere else.

A = MAA()
MergeChroma(A, A.TemporalDegrain())
TemporalDegrain2()
DeHalo_alpha_mt()



You could do something like this, just to double denoise where it's needed.

A = MAA()
B = A.TemporalDeGrain2()
C = MergeChroma(A, A.TemporalDegrain()).TemporalDegrain2()

B.Trim(0,478) ++ C.Trim(479,683) ++ B.Trim(684,0)
DeHalo_alpha_mt()

Except the logic for running two instances of TemporalDeGrain2 in a script doesn't seem right. To make that work you need to change line 156 in the script to this:

ReplaceGlobals = true

Last edited by hello_hello; 21st May 2020 at 18:40.
hello_hello is offline   Reply With Quote
Old 21st May 2020, 19:33   #10  |  Link
Mounir
Registered User
 
Join Date: Nov 2006
Posts: 776
That should be more than enough to me:
Quote:
SetMemoryMax(1200)
SetMTMode(3,4)
MPEG2Source("tgft.d2v", cpu=0, info=3)
# Step 1: Remove chroma noise:
assumetff()
ConvertToYV16(interlaced=true)
orig=last
ev=orig.assumetff().separatefields().selecteven()
od=orig.assumetff().separatefields().selectodd()
ev
ue_chroma = UToY(ev).blur(0,1.5).binomialblur(5).ttempsmooth(maxr=2,lthresh=150, strength=1)
ve_chroma = VToY(ev).blur(0,1.5).binomialblur(5).ttempsmooth(maxr=2,lthresh=150, strength=1)
YToUV(ue_chroma, ve_chroma)
MergeLuma(ev)
ev_filtered=last
od
uo_chroma = UToY(od).blur(0,1.5).binomialblur(5).ttempsmooth(maxr=2,lthresh=150, strength=1)
vo_chroma = VToY(od).blur(0,1.5).binomialblur(5).ttempsmooth(maxr=2,lthresh=150, strength=1)
YToUV(uo_chroma, vo_chroma)
MergeLuma(od)
od_filtered=last
interleave(ev_filtered,od_filtered)
assumefieldbased().assumetff().weave()
# Step2: color correction
ConvertToRGB32(interlaced=true)
########
LoadVirtualDubPlugin("C:\Program Files (x86)\VirtualDub\plugins\CMYK_rj.vdf", "CMYK", 0)
CMYK(255,0,266,10,257,-24,255,0,0,42) # Modifying Yellow high tones (-24%) and magenta lows (4%)


https://www.hostpic.org/view.php?fil...0130260109.jpg

Last edited by Mounir; 21st May 2020 at 21:00.
Mounir is offline   Reply With Quote
Old 21st May 2020, 19:37   #11  |  Link
Boulder
Pig on the wing
 
Boulder's Avatar
 
Join Date: Mar 2002
Location: Finland
Posts: 5,770
QTGMC(preset="very slow", inputtype=2) should already help nicely as it will press down the sharp noise and remove some aliasing.
It's hard to remove all the noise without removing all the detail. If you like, you could hit it with something TemporalDegrain2 and apply some light artificial noise with GrainFactory to fake detail.
__________________
And if the band you're in starts playing different tunes
I'll see you on the dark side of the Moon...
Boulder is offline   Reply With Quote
Old 21st May 2020, 22:16   #12  |  Link
samf88
Registered User
 
Join Date: Oct 2006
Posts: 19
Quote:
Originally Posted by hello_hello View Post
I had a silly idea. I can't say I've done it before myself (it'd be dog slow), and I don't know if there'd be a better way to go about it, but what if you only denoised the chroma, then denoised the chroma and luma? You could probably get away with less noise removal the second time around because it'd look more like grain. It's probably mainly an improvement for flat areas of color, such as the sky, and not so much everywhere else.
Thank you so much, I do really like those results. Thanks so much for your help

Quote:
Originally Posted by Mounir View Post
That should be more than enough to me:
Thanks so much for that. I've had to modify it slightly for 64 bit, but I do like the results. Still trying to find a 64 bit version of the VDub filter, but I tried it manually in VDub itself to see the results. Thanks for taking the time to work on that, I really appreciate it.

Quote:
Originally Posted by Boulder View Post
QTGMC(preset="very slow", inputtype=2) should already help nicely as it will press down the sharp noise and remove some aliasing.
It's hard to remove all the noise without removing all the detail. If you like, you could hit it with something TemporalDegrain2 and apply some light artificial noise with GrainFactory to fake detail.
Thanks - definitely thinking I will do pre-treatment with QTGMC before anything else. I will look in to GrainFactory - that's a good idea to reintroduce some "grain" to disguise other things.

Much appreciated everyone!

EDIT: Here's what I've been able to achieve so far, using the help provided here: http://www.framecompare.com/screensh...rison/99J2JNNU. I'm much happier with this than what I was getting a few days ago.

Last edited by samf88; 22nd May 2020 at 11:01. Reason: Adding current progress
samf88 is offline   Reply With Quote
Old 22nd May 2020, 10:50   #13  |  Link
Katie Boundary
Registered User
 
Katie Boundary's Avatar
 
Join Date: Jan 2015
Posts: 1,072
I swear by TNLmeans: http://avisynth.nl/index.php/TNLMeans

There's also a KNLmeans but I think it requires an NVIDIA GPU or something.
__________________
I ask unusual questions but always give proper thanks to those who give correct and useful answers.
Katie Boundary is offline   Reply With Quote
Old 22nd May 2020, 11:03   #14  |  Link
samf88
Registered User
 
Join Date: Oct 2006
Posts: 19
Thanks! I actually use KNLMeansCL quite often but on its own it wasn't touching the main problem. My current filter chain (with results shown in the link I edited in my post above) does include it as part of a denoise mask.
samf88 is offline   Reply With Quote
Old 22nd May 2020, 11:57   #15  |  Link
hello_hello
Registered User
 
Join Date: Mar 2011
Posts: 4,860
Quote:
Originally Posted by samf88 View Post
Thanks! I actually use KNLMeansCL quite often but on its own it wasn't touching the main problem. My current filter chain (with results shown in the link I edited in my post above) does include it as part of a denoise mask.
So don't keep us in suspense. What's the script you're using?
hello_hello is offline   Reply With Quote
Old 22nd May 2020, 12:42   #16  |  Link
samf88
Registered User
 
Join Date: Oct 2006
Posts: 19
Below is what I've got at the moment. Hopefully it all makes sense and I haven't made any silly mistakes. I'm using AvsPmod to preview/make changes which seems to crash every time I change a parameter, so it wasn't an easy task to adjust things. I haven't yet found an alternative filter to adjust the CMYK levels as suggested by Mounir. I'm also not confident in my grain settings and keep adjusting them. It's possible I've used the masks wrong too - I never was confident using those. I may have switched colour space more than necessary too. I've also taken out all the SetFilterMTMode/Prefetch stuff for the sake of posting here.

Code:
MPEG2Source("movie1.d2v", cpu=0)

# Mounir's Chroma Noise Removal
assumetff()
ConvertToYV16(interlaced=true)
orig=last
ev=orig.assumetff().separatefields().selecteven()
od=orig.assumetff().separatefields().selectodd()
ev
ue_chroma = UToY(ev).blur(0,1.5).binomialblur(5).ttempsmooth(maxr=2,lthresh=150, strength=1)
ve_chroma = VToY(ev).blur(0,1.5).binomialblur(5).ttempsmooth(maxr=2,lthresh=150, strength=1)
YToUV(ue_chroma, ve_chroma)
MergeLuma(ev)
ev_filtered=last
od
uo_chroma = UToY(od).blur(0,1.5).binomialblur(5).ttempsmooth(maxr=2,lthresh=150, strength=1)
vo_chroma = VToY(od).blur(0,1.5).binomialblur(5).ttempsmooth(maxr=2,lthresh=150, strength=1)
YToUV(uo_chroma, vo_chroma)
MergeLuma(od)
od_filtered=last
interleave(ev_filtered,od_filtered)
assumefieldbased().assumetff().weave()

## Mounir's Colour Correction
#ConvertToRGB32(interlaced=true)
#LoadVirtualDubPlugin("VirtualDub\x86\plugins32\CMYK_rj.VDF", "CMYK", 0)
#CMYK(255,0,266,10,257,-24,255,0,0,42) # Modifying Yellow high tones (-24%) and magenta lows (4%)

ConvertToYV12()

LoadPlugin("DeGrainMedian64\DeGrainMedian.dll")
Import("kirsch.avsi")
aux=DeGrainMedian(mode=0, limity=7, interlaced=true)
denoised = KNLMeansCL(d=4,a=4,s=6,h=2.0,channels="Y",rclip=aux)
edgemask1 = ConvertToRGB32(interlaced=true).Kirsch().ConvertToYV12(interlaced=true,matrix="PC.601")
edgemask2 = edgemask1.mt_binarize(120).Blur(1.0).Blur(1.0)
mt_merge(last,denoised,edgemask2)

# Deinterlace
QTGMC(preset="Very Slow", FPSDivisor=2)

# End filtering
DeHalo_alpha_mt()
Grainfactory3mod(g1str=2,g2str=2,g3str=2)

# Shifting image up 1px and cropping (I should probably do the shift and crop all in RGB32 colour space actually)
ConvertToRGB32(matrix="Rec601", interlaced=false)
Crop(0,1,0,0)
AddBorders(0,0,0,1)
ConvertToYV12()
Crop(10, 2, -18, -2)
FillMargins(0,1,0,1) # I like to keep to mod4 resolutions, so this just makes the extra 2px less obvious
samf88 is offline   Reply With Quote
Old 22nd May 2020, 12:58   #17  |  Link
Boulder
Pig on the wing
 
Boulder's Avatar
 
Join Date: Mar 2002
Location: Finland
Posts: 5,770
Your QTGMC call is incorrect. The sample was not interlaced like tebasuna51 mentioned. Set inputtype=2 instead of FPSDivisor if you want to see how it works on deinterlaced (as that's what your sample looks like) content.
__________________
And if the band you're in starts playing different tunes
I'll see you on the dark side of the Moon...
Boulder is offline   Reply With Quote
Old 22nd May 2020, 13:07   #18  |  Link
samf88
Registered User
 
Join Date: Oct 2006
Posts: 19
Hi,

I've adjusted it now, thanks.
http://www.framecompare.com/image-co...rison/77DPWNNX
samf88 is offline   Reply With Quote
Old 22nd May 2020, 13:49   #19  |  Link
Boulder
Pig on the wing
 
Boulder's Avatar
 
Join Date: Mar 2002
Location: Finland
Posts: 5,770
Quote:
Originally Posted by samf88 View Post
You could also try inputtype=3, I couldn't tell which one looks better but type 2 seems worse in that sample screenshot.
__________________
And if the band you're in starts playing different tunes
I'll see you on the dark side of the Moon...
Boulder is offline   Reply With Quote
Old 22nd May 2020, 14:18   #20  |  Link
samf88
Registered User
 
Join Date: Oct 2006
Posts: 19
Just gave it a go with inputtype=3 (http://www.framecompare.com/image-co...rison/99J21NNU). Judging by the pattern on the red blouse alone I'd say it looks best treating it as interlaced (inputtype=0 and discarding the dupe frames using FPSDivisor). The stripey top and pants definitely look better with inputtype=3 though.
samf88 is offline   Reply With Quote
Reply

Tags
bad quality, dvd, noise

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 18:51.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.