Huge noise

[san_vez]

Hello, im having very big problems encoding a dvd (vhs recorded to a dvd) wich has a lot of noise, i think this is signal noise.
I have been reading a lot of posts of denoising but noneone helped me.
Please, help me with this (im relative newbie in encoding)
Here are some captures from the VOB.
http://img493.imageshack.us/img493/8817/bscap0015ye.th.jpg (http://img493.imageshack.us/my.php?image=bscap0015ye.jpg)
http://img153.imageshack.us/img153/1883/bscap0039rz.th.jpg (http://img153.imageshack.us/my.php?image=bscap0039rz.jpg)
http://img438.imageshack.us/img438/7616/bscap0024cb.th.jpg (http://img438.imageshack.us/my.php?image=bscap0024cb.jpg)

[t1dUs]

for what I have seen from the first and 3rd captures you haven't deinterlaced the source, I suggest you do that 1st
And what I saw in the 2nd capture is to remove grain.
What I would do 1st is work with the vhs capture pass it to avi, then deinterlace, then apply denoising filters to remove grain and other stuff, and then pass it to dvd.

There might be other ppl more capable to help you since I'm almost new to this stuff, been only 2 months since I started :D

by the way: "Arriba Alonso!!!!" :D:D:D

[Chainmax]

A vob sample would give us a better chance to help you. You can cut a vob sample with programs like ChopperXP (http://www.inchwest.com/digital-miner.com/chopperxp.html) and upload it to free hosting sites like RapidShare (http://www.rapidshare.de/).

I concur with t1dUs, you haven't decombed your source, it might be telecined or interlaced. I strongly recommend you to read the capture guide (http://www.geocities.com/wilbertdijkhof/ACG41.zip), the IVTC tutorial (http://www.doom9.org/ivtc-tut.htm) and the Decomb package's readmes, tutorial and reference guides (http://www.neuron2.net/decomb/decombnew.html). They have a lot of extremely useful info on the subject.

If you want high quality deinterlacing, download TDeint.dll, EEDI2.dll and use either of the following lines:

For TFF:

interp = separatefields().eedi2(field=3)
tdeint(mode=1,order=1,edeint=interp)


For BFF:

interp = separatefields().eedi2(field=2)
tdeint(mode=1,order=0,edeint=interp)


For faster encodes with good quality, use LeakKernelDeint with a threshold of 8 or 6.


For high quality IVTC, download TIVTC.dll, TDeint.dll, EEDI2.dll and use either of the following lines:

For TFF clips:

AssumeTFF()
Interp = SeparateFields().SelectEven().EEDI2(field=1)
Deinted=TDeint(order=1,field=1,edeint=Interp)
TFM(mode=6,order=1,PP=7,slow=2,mChroma=true,Clip2=Deinted)
TDecimate(mode=1)


For BFF clips:

Interp = SeparateFields().SelectEven().EEDI2(field=0)
Deinted = TDeint(order=0,field=0,edeint=Interp)
TFM(mode=6,order=0,PP=7,slow=2,mChroma=true,Clip2=Deinted)
TDecimate(mode=1)

For faster encodes with good quality, use the Decomb package.



As for the noise, this seems like a good candidate for LTSMC or whatever it is Didée is cooking up these days. This will probably need some sharpening after resizing as well.

[krieger2005]

I work now on a source which is variable grainy (variable over time... some frames are clean, others are very grainy). For me it works to use LRemoveDustMC (search here in the forum) with a limit-value about 4. This does not remove all the noise, so i use then FFT3DFilter on the unfiltered parts of the frames.

The frame ist very clean and not blurry at the end.

I don't know if this work for you, becauce i can't test this on a single picture... Maybe only FFT3DFilter works well. But since this seems to be a movie, where is much movements i think you can't use a temporal filter with too high denoise-values.

here my script:

c=LRemoveDustMC(4)
strL="x y - abs 1 > 255 0 ?"
strC="x y - abs 2 > 255 0 ?"
dRemDust=YV12LutXY(last,c,strL,strC,strC,Y=3,U=3,V=3)
MaskedMerge(last,c,dRemDust,Y=3,V=3,U=3)
s1=last

c1=Repair(FFT3DFIlter(sigma=3).FFT3DFilter(sigma=1.5, plane=3),last,1,1)
c2=FFT3DFIlter(sigma=3, sharpen=1, bw=16, bh=16).FFT3DFilter(sigma=1.5, plane=3).hqdn3d(3,2,0,0)
strL="x y - abs 3 > 255 0 ?"
strC="x y - abs 2 > 255 0 ?"
d=YV12LutXY(last,c1,strL,strC,strC,Y=3,U=3,V=3)
e=FineEdge(16).Levels(25,1,70,0,255,false).inflate().FitY2UV()
str="y 10 > 0 x ?"
r=YV12LutXY(d,e,str,str,str,Y=3,U=3,V=3)
MaskedMerge(last,c2,r,Y=3,V=3,U=3)
s2a=last

# Part 2
f=FFT3DFilter(sigma=5)
d=YV12LutXY(last,f,"x y - abs 4 > 255 0 ?")
m=YV12LutXY(e,d,"x 10 > y 0 ?")
dull=BilinearResize(704,528).FFT3DFilter(sigma=5)
sharp=TUnSharp(dull,strength=100)
c=SootheX(sharp,dull,20).LanczosResize(width,height)
MaskedMerge(last,c,m)
s2b=last

str1="x y - abs 0 = 255 0 ?"
str2="y 0 > 0 x ?"
dRest=YV12LutXY(start,last ,str1,str1,str1,Y=3,U=3,v=3)
dRest=YV12LutXY(dRest,dRemDust,str2,str2,str2,Y=3,U=3,V=3)
dRest=YV12LutXY(dRest,r ,str2,str2,str2,Y=3,U=3,V=3)
dRest=YV12LutXY(dRest,m ,str2,str2,str2,Y=3,U=3,V=3)
MaskedMerge(last,RemoveGrain(4),dRest,Y=3,U=3,V=3)


EDIT: Modified a littlebit

[communist]

Just to clear up - there is no need to deinterlace this. Process it interlaced (sepfields/filter/weave or bob/filter/reinterlace) and encode interlaced, after all DVD allows interlaced material. It'll look really jerky when deinterlaced because its mostly fast motion here.

To the original poster: Did you 'capture' with a DVD recorder and then rip that DVD to the computer? You might get better quality when you capture it with an analog capturing card (uncompressed / lossless compressed) and then filter it. If thats too much hassle look if you can increase bitrate when recording to DVD because either its recorded with lots of macro blocks or the source has them (or the jpg compression added them).
You can deblock it when you feed the source to Avisynth with DGDecode (cpu parameter).

[zilog jones]

I'm not sure exactly what san_vez is doing - are you in the process of converting from VHS to DVD, or are you converting from a DVD (already made from a VHS recording) to some other video format?

If you are making a DVD from this, communist is right - there is no need to deinterlace. If it is being converted to DivX, XviD, etc. for viewing on a PC you will probably want to deinterlace it - I'd recommend trying TDeint or LeakKernelDeint.

@chainmax & krieger2005:
You need some practice with your video recognition skills! ^_^ For starters the resolution of those videos are full PAL D1 and the channel (in the corner) is TVE2 - a Spanish channel - so it's definately PAL, therefore requirement of IVTC is out of the question. Also, it's an F1 race, probably from a live transmission (or a re-broadcasting of it), as those graphics overlays are used internationally on live broadcasts of the F1 - so it is definately proper interlaced PAL.

[Chainmax]

I noticed the PAL frame size, but there is a sort-of IVTC for PAL that consists only of field matching without the decimation. I wouldn't have guessed anything from the graphic overlays though :).

[LocalH]

If you are making a DVD from this, communist is right - there is no need to deinterlace. If it is being converted to DivX, XviD, etc. for viewing on a PC you will probably want to deinterlace it - I'd recommend trying TDeint or LeakKernelDeint.
In my opinion, even if the intended destination is PC viewing, it's advisable to keep the true interlaced nature of the video and use a bobber on playback - it might tax the codec a bit more, but your final result will be heaps closer to the original video, and motion will be smooth like the original source. The only other way to do this is to use a smart bob and double the framerate - if anything, both of these methods are a toss-up for which one will give a better result. I've not tried bobbing before encoding, so I can't vouch for it.

Note that this only applies to a piece of footage that has predominantly true interlaced material (with maybe a small percentage of telecined material). Telecined film should be IVTC'd, and 25/30p footage can just be encoded as-is (barring any field swaps by capture hardware), since it's already progressive.

[zilog jones]

I noticed the PAL frame size, but there is a sort-of IVTC for PAL that consists only of field matching without the decimation. I wouldn't have guessed anything from the graphic overlays though .

But it's F1 racing, though - anyone who broadcasts something like that deinterlaced should be shot! Generally, all live-broadcast sports are interlaced - it wouldn't make sense to do it any other way, unless it's HD of course.

[zilog jones]

In my opinion, even if the intended destination is PC viewing, it's advisable to keep the true interlaced nature of the video and use a bobber on playback - it might tax the codec a bit more, but your final result will be heaps closer to the original video, and motion will be smooth like the original source. The only other way to do this is to use a smart bob and double the framerate - if anything, both of these methods are a toss-up for which one will give a better result. I've not tried bobbing before encoding, so I can't vouch for it.
Yeah, I agree in a way. I personally would keep any 50i/60i interlaced, but I'd just watch it on a TV (usually with my standalone DVD/MPEG-4 player) where possible. Nothing beats watching interlaced material than on a real interlaced monitor ;)
Having to set up a postprocessing bobber on a PC isn't exatly the easiest thing to do, especially for those inexperienced with such stuff, and in certain situations (e.g. doing a presentation somewhere using a PC where you have no privilages to install software) it may not be possible to do at all. Plus ffdshow and my PC *do not* get along well...

[LocalH]

Well, true, it all depends on the individual person. From my perspective, I just use ffdshow and haven't had any problems with enabling deinterlacing (I use ffdshow's kernel bob with the threshold at 0). It all depends on the individual situation, I just advocate what's worked for me so far.

In my opinion, if you have to deinterlace a true interlaced source for whatever reason, you should at least ensure that your original source is preserved for later use, if possible.

[san_vez]

Hi, thanks you very much for your fast replys.
The captures that i posted were without any filter, so i didnt deinterlaced.
Im going to try with your scripts.
But i have a question, what does TFF clips or BFF clips mean?
Here i have uploaded a 1 min VOB sample, for if you can help me more to denoise it
http://www.megaupload.com/?d=DTI59Y8V

Yeah, i am converting from a DVD (already made from a VHS recording
using my new DVD recorder) to AVI.
I want to make a DVD with 2 Grand prix (1:45 h each one) and 2 qualify (45 min each one) in AVI, so the bitrate will be very high.
I will see the new DVD on PC and TV
Yes, the source is PAL


For t1dUs:
Im spaniard, and im very surprised that in USA are Alonso fans :)

[zilog jones]

TFF = Top Field First
BFF = Bottom Field First
These are the two types of interlaced video you will come across, and it's important that you find out which type your video is. You can find out with a simple AviSynth script like this:

DirectShowSource("F:\video\1.vob")
AssumeTFF()
bob()

The video plays correctly, so it is TFF. If you change it to "AssumeBFF", the video plays with every 2 frames the wrong way round - it looks like a mess! You need to know which one it is when you go to encode the video - if you set the field order wrong it will look wrong like that video does when you set it to BFF in the example above. I suggest you read the Capture Guide (http://www.doom9.org/capture/start.html) for more info on stuff like that - unfortunately a lot of the stuff in that guide referring to filters/denoising is quite out-of-date.

[Didée]

Sorry for being slightly OT, but I just had my laugh of the day!

For me it works to use LRemoveDustMC (search here in the forum)
"Heya" I thought, "now what's that? Sounds interesting!", did search, found this (http://forum.doom9.org/showthread.php?p=623812#post623812) ...

If anyone knows some good Ginko products I could try ... ;)


@ krieger:

Interesting post-denoising, must be tried out. (BTW, your "speaking" names for variables are almost as horrendous as mine.)
Lately I did some things in a similar direction ... but before deciding on post-denoising, mc-denoising should be pushed to its limits.
Any experiences with MCNR_simple so far?


To put at least something on topic:

If any sort of motion-compensated denoiser should be tried in this case, be it Pixie/GoldDust or an MVTools application, do make sure to work on a *bobbed* clip only! None of the currently available MC-denoisers respect interlaced input...

In this light, FFT3DFilter shines even more. Definitely try it on these sources, with interlaced=true.

[DeepDVD]

Any experiences with MCNR_simple so far?


Hey,

my experiences are very good on light-noised input. The main problem is the speed. :( (especially with removedirt=true)

[Mug Funky]

lately i've been thinking fft3dfilter's kalman filter could be combined with forward motion compensation (internally i guess) to give some pretty strong and relatively transparent noise reduction.

bt=0 works wonders on 16mm HD stuff, especially when scaling down to SDTV (with HDV there's no choice - in HD size it's blocky as hell with all that grain). you've just got to keep sigma around the 1.0 mark or it'll smear the balls off everything. nobody likes a video that's been belt-sanded :)

[Dyolfknip]

Hello guys!

I'm looking for a good deinterlacer and I tried filter that Chainmax suggested. I copied his example script and the files needed and it gave me an error saying that "edeint=Interp" is not valid argument. Am I doing something wrong (And yes I loaded the plugins in my script)?

[foxyshadis]

Not wrong, you just need the latest version of TDeint (http://bengal.missouri.edu/~kes25c/) (and update the rest of the filters for good measure).

[Dyolfknip]

Damed! I Should have known. I thought that i had the lastest files... now it works!

[san_vez]

Hi, have someone tried to denoise the vob that i uploaded ?
I dont like my results.

[paviko]

@krieger2005
Hi! How can I use your posted script, is it complete? What is a, b ? Why LRemoveDustMC doesn't have _grey and _smooth parameters ?

Thank you.

[krieger2005]

I use a modified version of LRemoveDustMC. I modify the most functions for my own purpose... But however, the modifications here in LRemoveDustMC are not that big. You can use this (http://forum.doom9.org/showthread.php?p=662180#post662180) version (if you like).

The variables a and b are not really usefull. I use for temporalcleaning an aproach like this one:
a=trim(3,20).reverse()
b=last
c=reverse().trim(3,20)

a+b+c

So to get the real part you must trim this way:
trim(a.framecount,a.framecount+b.framecount-1)

But however... This is just as information. The Function "SootheX" is the old one from "Didée". I renamed it, because in his "MCNR_simple2" there is also a function with this name. Here the link (http://forum.doom9.org/showthread.php?p=708217#post708217), from where you can get it.

TUnSharp is a filter by Trictical (i guess). One can use LimitedSharpen or any other (for your movie) good sharpener.
FineEdge is also a Function by "Didée".

I think a have it...

[paviko]

Thank you krieger2005. It's working. The results are very good. But it's extremely slow on my system ;)

Regards

[krieger2005]

i saw also, that some detail get lost with this script... i think i must accept this, since its difficult to indicate "detail" and "noise"...

On my system it is also slow :). But Didée said before something, that he working on something... And you know, Didée scripts are mostly faster, so lets wait and hope