TempGaussMC beta2

[rkalwaitis]

Sorry I have a couple of questions concerning TGMC.

What is the latestest version?

Can I increase its denoising capabilities? How?

Lastly if I had a film that is interlaced and its tivtc, what would be the correct way to use TGMC to deinterlace it? Megui does analysis and gives me this command for its script. tfm(order=1).tdecimate()

tfm(order=1).tdecimate()

[Didée]

What is the latestest version?
Technically, beta2 is the latest. There are "..u" and "..x" and "..z" modifications floating around ... some have taken NNEDI7 into the mix, others have EEDI815, ... nothing serious. ome incorpo

Can I increase its denoising capabilities?
Yes.
How?
Better don't.

if I had a film that is interlaced and its tivtc, what would be the correct way to use TGMC to deinterlace it?
If the source is hard telecined, the correct way of using TGMC is to NOT USE IT. TGMC is made to process pure video-type interlacing. Nothing else.

[rkalwaitis]

Thanks Didee, are there any other little tools which let a person know what type of interlacing a film has?

[18fps]

If the source is hard telecined, the correct way of using TGMC is to NOT USE IT. TGMC is made to process pure video-type interlacing. Nothing else.

What about using it as a bobber (with lossless option) for srestore? I liked it better than using yadifmod with needi.

[rkalwaitis]

18fps, how did you do that? This interests me very much, as Im not so certain that the tool that automatically test a source for me is always giving me the best solution. Unfortunately I am not experienced enough as many people are to simply look at it and see how something is interlaced. However the results that are provided by the program determining how something is interlaced sometimes leave me wondering if it was the right answer after looking at the product. I had such an issue and tried TGMC in conjunction with the deinterlacer and it seemed better, but the detail was much better. But I was not sure if TGMC was just cleaning up the mediocre attempt at deinterlacing the source.? I hope that made sense.

Thanks

[txporter]

You can try following the FAQ on neuron2's site to determine whether your source is interlaced, progressive, or telecined.

http://neuron2.net/faq.html

[rkalwaitis]

Thanks txporter this is a good place for me to start, which brings me to my original saying its much easier identified by eye. Perhaps why I sometimes get a quirky response when I use the analsye tool.

[rkalwaitis]

I followed Neron's FAQ page on figuring out how your source is interlaced. My results are --a a a b b c c c d d-- Bottom Field First(BFF) If I seperate the fields then everything doubles as such aaabbbbccccccdddd. When the source in not seperated you can definately see the interlaced venitian blind look, when seperated it looks melted together. My analysis tool gave me this to use to deinterlace the source --tfm(order=1).tdecimate()-- however Im not liking it. tfm(order=1)--means top field first. Is this right? Or do I have this backwards? However, If I followed Neuron's instructions properly my source is BBF because when I stepped forward frame by frame the video images did no go forward and then back--to the right in one frame and back to the left a bit in the next. So I changed tfm(order=1) to tfm(order=0).tdecimate() and things seem much better. Is their another deinterlacer that I can try that you guru's may know of? Is tdecimate necessary? Im not sure I understand it. What I read about it is that never use it with PAL and you may or may not with NTSC. Neuron's FAQ page was very educational, Im glad txporter turned me on to it.
Thanks

[18fps]

Is tdecimate necessary?

Yes, it is, as tfm just matches fields to recreate the whole frame, replacing the combed frames with good ones, but leaving repeated frames (it doesn't changes the frame count). tdecimate will try to delete repeated frames, if you give it the right ratio (delete x frames every y frames).

srestore should be used only when, even after a SeparateFields(), you can see two images blended on the same frame.

[xv]

I tried to deinterlace a quite noisy DV source. Unfortunately the temporal noise partly turns to static noise (all options default using nnedi3, also tried tr2=3 and tr0=tr1=1).
What is the recommend way to remove the noise using TempGaussMC or how to use TempGaussMC with an additional denoiser? Or is TempGaussMC not recommend at all with a noisy source?

[Gser]

I tried to deinterlace a quite noisy DV source. Unfortunately the temporal noise partly turns to static noise (all options default using nnedi3, also tried tr2=3 and tr0=tr1=1).
What is the recommend way to remove the noise using TempGaussMC or how to use TempGaussMC with an additional denoiser? Or is TempGaussMC not recommend at all with a noisy source?

Feel free to denoise before it.

[xv]

What denoiser is recommend? I tried MCTemporalDenoise, but it doesn´t work correctly with interlaced sources (I set interlaced=true), the picture jumps up and down every frame one line.
I tried it the other way round but result is rather poor (that´s what I expected), I also tried MCBob instead of TempGaussMC before denoising, the result is ok, but it is really really slow (playback ~0,2fps vs. 2fps with the first method)

[LoRd_MuldeR]

You can use something like:

function TempGaussMC_WithNoiseBypass(clip Source)
{
Denoised = Source.FFT3DFilter(sigma=3, bt=3, interlaced=true)
NoiseOnly = mt_makediff(Source, Denoised, U=3, V=3)
TGMC = Denoised.TempGaussMC_beta2u(edimode="nnedi3")
return mt_adddiff(TGMC, NoiseOnly.DoubleWeave(), U=3, V=3)
}

(Based on a former suggestion by Didée)

[xv]

If I understand that correctly, that means I deinterlace the video and keeping the noise, so that TempGaussMC works similar to MCBob.
That´s not my intention, I wan´t to remove the noise, adding a denoiser after deinterlacing makes it awfully slow (~0.2fps) and the additional FFT3DFilter will reduce speed even more (that means processing 1 hour video will take about 2 weeks).

I think the denoiser needs to be a) before deinterlacing or b) inside TempGaussMC. Because there is no stronger denoiser inside TempGaussMC, a) seems to be the only option, so what denoiser works good before TempGaussMC.
MCTemporalDenoise gives a great denoised result, but it doesn´t work correctly.

[Terka]

tgmc is a _strong_ denoiser! tr2=3 or you can use j. wells mvtools modification and use mdegrain5 or whatever in tgmc to be even stronger. but tr2=3 is enough. or use
http://forum.doom9.org/showthread.php?p=1411610#post1411610

[Didée]

Additional denoising "inside" of TGMC ain't gonna happen, most probably.
a) Too many different kinds of noise ask for too many different kinds of denoisers.
b) There's no particular reason (i.e.: no explicitely well-suited place) to make such internal denoising.

If TGMC's denoising isn't sufficient for your case, just use normal post-denoising like you would do with any other bob/deinterlacer.
And if so, it could really be beneficial to use the noise-bypass method.


So ... feel free to use


preNR = source.FFT3DFilter(sigma=8,bt=5,interlaced=true,plane=4)
preNRD = mt_makediff(source,preNR,U=3,V=3)
TGMC = preNR.TempGaussMC_beta2u(2,2,3,4,4,4,edimode="nnedi3",SLrad=2)
TGMC.mt_adddiff(preNRD.DoubleWeave(),U=3,V=3) # or "preNRD.bob()
MCTemporalDenoise(setting="very high",radius=6,pel=4,twopass=true,useTTmpSmth=true,deblock=true,useQED=true)


You might get a nice result ... around christmas time. :p

However, most likely not this year's christmas. :D

[Terka]

preNR = source.FFT3DFilter(sigma=8,bt=5,interlaced=true,plane=4)
preNRD = mt_makediff(source,preNR,U=3,V=3)
TGMC = preNR.TempGaussMC_beta2u(2,2,3,4,4,4,edimode="nnedi3",SLrad=2)
TGMC.mt_adddiff(preNRD.DoubleWeave(),U=3,V=3) # or "preNRD.bob()
MCTemporalDenoise(setting="very high",radius=6,pel=4,twopass=true,useTTmpSmth=true,deblock=true,useQED=true)Didee, you could advice also TNLmeans to rkalwaitis..

[Terka]

Didee,
could you give me some comments to
http://forum.doom9.org/showthread.php?p=1411610#post1411610
?

[Didée]

When you say that it works for you, then I assume that ... it works for you.

Interesting move to use global compensation for supposely-static areas. (Adjusting saturation of a B/W source?)


No time to look into that, I'm highly busy with renovating - the house looks like a battleground, and I absolutely need to finish before the wife returns from regimen. :devil: :D

[Terka]

When you say that it works for you, then I assume that ... it works for you.ya

Interesting move to use global compensation for supposely-static areas. (Adjusting saturation of a B/W source?)yayyayyay!

No time to look into that, I'm highly busy with renovating - the house looks like a battleground, and I absolutely need to finish before the wife returns from regimen. yaaaaaaaay!!!

[bloodem]

I'm gonna ask here, to avoid starting a new thread. So basically the problem is this: Until now, I used VirtualDub + AviSynth + TempGaussMC + Lagarith lossless codec, to deinterlace my AVCHD 1080i content. On my Core2 Duo E7300 the speed was somewhere around 0.58 FPS. I always thought that with a more powerful machine I could get better results... but now, when trying the same settings on a new machine with two Xeon CPUs (the new ones, based on Core i7 architecture) with a total of 8 physical cores + 8 virtual cores (HyperThreading enabled) I get THE EXACT same results. So it looks like this is the maximum speed that can be achieved with TempGaussMC.

I have TempGaussMC_beta1u2. I tried the latest version but the quality was a lot worse and the speed was practically the same. Am I missing something? Is there some way I can speed things up to at least 1 - 2 FPS?

[TheSkiller]

Do you use SetMTmode() at all (in other words, multithreaded version of AviSynth)? If not I'm not surprised.

But the problem could also be caused by something else, maybe your hard drive can't write the lossless Legarith video faster than that because of the insane amount of data, I'm mean we're talking about 50 or 59.94fps of lossless full HD. Please do the following: open your Legarith video with VirtualDub, go to File - File Information and tell us what is says at "Average bitrate".

Edit: These are (very) fast settings for TempGaussMC:

TempGaussMC(1,1,0, 0,0,0, edimode="bob",SVthin=0.0,pelsearch=1)

[LoRd_MuldeR]

Did you try to multi-thread your script? Like this:

MT("TempGaussMC_beta2()", overlap=4)

[um3k]

Yeah, if you want a significant improvement, you're going to have to download a multithreaded build of AviSynth.

Also, I recommend SetMTMode over MT for TGMC. MT causes problems for TGMC's motion compensation.

[bloodem]

Thanks for the quick replies guys!
I definitely wasn't using SetMTMode or something similar. I have AviSynth 2.58 installed and it looks like I probably need another version or a plugin in order to use this function, cause it doesn't seem to work. I tried this:

DirectShowSource("D:\00006.MTS")
AssumeTFF()
TempGaussMC_beta1u(tr2=0, EdiMode="nnedi2")
SetMTMode()

Error "there is no such function"

A HDD speed related problem is out of the question... I have two 128GB SSDs in RAID0. They are capable of handling a lot more than a 425149 kbps bitrate (~ 52 MB /s )

[TheSkiller]

It works with 2.5.8. see this (http://forum.doom9.org/showthread.php?t=148117) thread.

Put the avisynth.dll into your \WINDOWS\system32 folder (backup the old one!) and the stuff in plugins into your plugins folder. Reboot.
Then use SetMTmode(2) before TempGaussMC.

[LoRd_MuldeR]

I definitely wasn't using SetMTMode or something similar. I have AviSynth 2.58 installed and it looks like I probably need another version or a plugin in order to use this function, cause it doesn't seem to work.

See:
http://avisynth.org/mediawiki/MT

[um3k]

I use SEt's Avisynth 2.5.8 MT: http://forum.doom9.org/showthread.php?t=148117

[bloodem]

Thanks a lot guys. It works pretty well. I'll get back with further details!

[txporter]

In my experience, 1080i material + TGMC + setmtmode results in a crash. Didee reported it in some thread that MVtools doesn't play well with setmtmode and that is my experience as well.

Once you do get 2.5.8MT avisynth working, your script should look something like this to use multi-threading:
setmtmode(5,0)
loadCplugin("path\to\yadif.dll")
mpeg2source("blah.d2v")
setmtmode(2)
TempGaussMC_beta2(1,1,0,4,0,0,edimode="Yadif",Smode=1, SLmode=1, Sbb=0,SVthin=0.0,pelsearch=1)

My experience is that setmtmode(2) before TGMC shows no improvement on SD material and causes crashes with 1080i.

[Didée]

In my experience, 1080i material + TGMC + setmtmode results in a crash. Didee reported it in some thread that MVtools doesn't play well with setmtmode and that is my experience as well.
Incidentially, in some recent tests I didn't experience those instabilities anymore. Don't dare to say, but it could be I intermittently fiddled with a BIOS voltage and forgot to restore the provenly stable preset. :o
For now, everything seems fine, however I didn't do very much the past days.

My experience is that setmtmode(2) before TGMC shows no improvement on SD material and causes crashes with 1080i.
Beg your pardon, but what you mean with "no improvement on SD material" by setmtmode(2)? Not faster than singlethreaded? I see numbers like
TGMC(1,1,0,4,4,0,"--",SVthin=0.0)
- 1 thread -> 9.9 fps / (64bit: 21.5 fps)
- 4 threads -> 23.9 fps / (64bit: 54.4 fps)
- 8 threads -> 41.5 fps / (64bit: 72.3fps)

That's for PAL 576i, on 4 cores + HT. Didn't ever test 1080i at all, but that surely needs to use much more memory. Maybe too much?

[Forteen88]

@Didée: Doesn't the TGMC-parameter lossless=3 save more details and grain?!

[Didée]

That's to be expected, yes. Your question sounds like it would not be so?

[txporter]

Incidentially, in some recent tests I didn't experience those instabilities anymore. Don't dare to say, but it could be I intermittently fiddled with a BIOS voltage and forgot to restore the provenly stable preset. :o
For now, everything seems fine, however I didn't do very much the past days.


Beg your pardon, but what you mean with "no improvement on SD material" by setmtmode(2)? Not faster than singlethreaded? I see numbers like
TGMC(1,1,0,4,4,0,"--",SVthin=0.0)
- 1 thread -> 9.9 fps / (64bit: 21.5 fps)
- 4 threads -> 23.9 fps / (64bit: 54.4 fps)
- 8 threads -> 41.5 fps / (64bit: 72.3fps)

That's for PAL 576i, on 4 cores + HT. Didn't ever test 1080i at all, but that surely needs to use much more memory. Maybe too much?

I will try retesting it. I am fairly certain that I was seeing no fps improvement with and without setmtmode on 480i material. And 1080i material would go 2+ gb memory and crash with setmtmode. Otherwise it would stay around 900mb (without setmtmode), if I remember correctly.

update:
mpeg2source.TGMC(1,1,0,4,0,0,edimode="Yadif",Smode=1,SLmode=1,Sbb=1,SVthin=0.0)
Q6600, 4gb, 64bit avisynth, 64bit x264
- 1 thread -> 13.58 fps (64bit) <-- no setmtmode used not setting threads to 1
- 2 threads -> 23.36 fps (64bit)
- 3 threads -> 35.27 fps (64bit)
- 4 threads -> 42.08 fps (64bit) # same setting 4 or 0 (verified to make sure) - initially was choking with x264 preset

Looks like I was incorrect. Perhap it was my use of Srestore earlier? Will check with 1080i material to see if I can avoid crashes.

update2 (1080i numbers):
- 1 thread -> 2.08 fps (64bit) # ran at 617mb memory & 35% cpu
- 2 threads -> 1.11 fps (64bit) # shot up over 2.7gb & 45% cpu / encoding didn't finish
- 4 threads -> didn't test

Looks to be a problem with 1080i material. Too much data to process with multi-threading?

update3: Srestore seems to be fine with setmtmode. Don't know where my data points came from before.

mpeg2source, bob=dclip, tgmc.srestore
- 4 threads + mt srestore - 13.96fps (64bit)
- 4 threads + no mt srestore - 7.8fps (64bit)

Memory usage with/without setmtmode + various number of threads varied only slightly from 582mb to 600mb. CPU usage improves from 30% to 95%. Seems perfectly stable on 480i material. Not sure why I have issues with 1080i sources.

[Forteen88]

That's to be expected, yes. Your question sounds like it would not be so?I was confused because you of you conversation:
"-Can I increase its denoising capabilities?"
"-Yes."
"-How?"
"-Better don't."

so I obviously mistakenly thought you meant it's not good to set a grain-keeping parameter such as lossless=3 :)
Anyway thanks a lot for TGMC, Didée.

[bloodem]

Well, back with details regarding SetMTMode(2)... It's unstable as hell ! After a couple of minutes, VirtualDub goes berzerk... Anybody know how I can fix it?

[TheSkiller]

Try SetMTmode(3) and SetMTmode(4). They are more compatible, but also slower.
Also adding a SetMemoryMax(2/3 of your total RAM) as the very first line of your script is usually a good idea. And don't use SetMTmode(2) before AviSource, AviSource works good with SetMTmode(3).

[bloodem]

This is my script:

SetMTMode(2)
DirectShowSource("D:\00006.MTS")
AssumeTFF()
TempGaussMC_beta1u(tr2=0, EdiMode="nnedi2")


I'll try mode 3 and 4... see what happens.

[Didée]

The general rule-of-thumb is to let any kind of source filter work only single-threaded:

SetMTmode(5,threads) # set only 1 thread active for the source filter ...
XYZsource("source")
SetMTmode(2) # ... and multi-thread only after that
Do().More().Things()

That's the most failsafe way. Use SourceFilter-MT only when you specifically know it works for a certain filter.

[bloodem]

Well, tried the following and it's slow, as if I'm not using SetMTmode... :\


SetMTmode(5,1)
DirectShowSource("D:\00006.MTS")
AssumeTFF()
SetMTmode(2)
TempGaussMC_beta1u(tr2=0, EdiMode="nnedi2")

[Didée]

No, no no.

The very first SetMTmode statement makes a fixed setting of how much threads will be used. This number-of-threads can not be changed later in the script again.

You want

SetMTmode(5,4) # set the script to use 4 threads, but only 1 active thread for the mode=5 section
DirectShowSource("D:\00006.MTS")
AssumeTFF()
SetMTmode(2) # now activate all of the 4 initiated threads to work in mode=2
TempGaussMC_beta1u(tr2=0, EdiMode="nnedi2")

[bloodem]

Thanks for the help Didee, but it's useless. I used your script example and it's very very strange... It starts out rather fast, at about 1.8 FPS... and then it goes down to 0.3 FPS (without multithreading I had 0.6 FPS). I'm starting to think that it just doesn't work well with TempGaussMC...

[um3k]

Try adding a SetMemoryMax to the beginning of the script. Set it to a fairly high number, with my 3 gigs total ram, I usually set it to 512.

[bloodem]

Already tried it... Nothing changes.

[Didée]

Simple fact: 1080i (or p, for that matter) is not suited to be processed in such an exhaustive way. Or, if you prefer, today's typical home PC is not up to such a task.

Anyway. When you're going to TGMC some 1080i footage with SetMTmode, then 512MB is by far too little. With 4 MT threads, the RAM usage of TGMC is in range 1GB to 1.5 GB, depending on TGMC settings.

Try with SetMemoryMax(1200) or (1500), and see how it goes.

Also, it's (probably) better to use Avisynth 2.5.8-MT, not the older 2.5.7-MT. (Memory managment of 2.5.7-MT, albeit somehow working, is somehow broken.)

[Didée]

Small test. System: i7-860 8GB RAM, Win7 64bit.

SetMTmode(5,4)
SetMemoryMax(1536)
mpeg2source("shuttle.d2v") # NASA "Shuttle" sequence, 1080i 29.97, mpeg-2, 4746 input frames
SetMTmode(2)
TGMC_b2( 1,1,0, EdiMode=XX, SVthin=0 )

Avisynth 2.5.8 MT x86
XX="bicubic" : ca. 4.7 fps
XX="nnedi2" : ca. 3.7 fps

Avisynth 2.5.8 MT x64
XX="bicubic" : ca. 9.7 fps
XX="nnedi2" : still no nnedi2.dll for x64 available....

In all cases, processing speed stayed at a constant level from start til end. No speed breakdown.
_____

Trying something more insane under x64.

SetMemoryMax(1536)
TGMC_b2( 2,2,1, EdiMode="eedi2", SVthin=0 )

Starts good, but after some minutes broke down to 0.5 fps. VeeDub64 was using >3.2GB at that point, despite of SMM(1536).


SetMemoryMax(3072)
TGMC_b2( 2,2,1, EdiMode="eedi2", SVthin=0 )

3.3~4.3 fps, stable from start til end. Veedub64 stayed below 3 GB for at least the first 1800 output frames. Somewhen later, it had raised to 3.2 GB.


Seems the memory handling is not perfectly mature, and the speed surely is not big fun.
But overall, processing 1080i is technically possible, in reasonable time, if you're not too short with hardware ressources.

[bloodem]

Thanks for trying to help Didee, but the problem still doesn't go away. Just to make things clear: I have a powerful HP workstation, with 2 Xeon quad-core processors based on Nehalem architecture and 72 GB RAM. I have Windows 7 Ultimate x64, I'm using Avisynth 2.5.8 MT and my script is the following:

SetMTmode(5,4)
SetMemoryMax(1536)
DirectShowSource("D:\00004.MTS")
AssumeTFF()
SetMTmode(2)
TempGaussMC_beta1u(tr2=0, EdiMode="nnedi2")

This script results in a crash in VirtualDub 1.9.9 right after starting the encoding process. (An exception occurred in module KERNELBASE)

If I change SetMTmode(5,4) to SetMTmode(5,3) and SetMemoryMax(1536) to SetMemoryMax(512), The encode seems to work, but it starts at 1.5 FPS and after one minute it goes down to about 0.3 FPS. If I change again SetMemoryMax(512) to anything higher it results in a crash after 20 - 30 seconds.

PS: I'm starting to think that the problem comes from Haali Media Splitter. But if I uninstall Haali Media Splitter, avisynth can't import the video. If I install FFDshow, I still can't open the AVCHD file. It looks like I can only open it if I use Haali Media Splitter + CoreAVC.

[Emulgator]

I guess this may be unrelated to TGMC, I have a similar issue crashing VDub 1.9.9.
Looks like increased memory use when watching Taskmanager...

Every F2 reload of a >700line script using
AnimeIVTC (although bbob=3, not in TGMC mode bbob=4)

AnimeIVTC(mode=2, aa=4, precision=3, killcomb=0, cache=15, normconv=true, bbob=3, omode=1, dark=0.2, thin=10, sharp=120)

On a 4GB RAM WinXP32SP3 DELL System:

F2 adds +1 new instance of ffdshow in taskbar
crashes when more than 2-3 instances are open.
ReduceflickerSSE3.dll seems to eat memory outside.

On a 8GBRAM Win7U64 CLEVO System:

F2 adds + 1 new instance of ffdshow + 2 instances of Haali in taskbar
crashing when more than 6 + 12 instances are open.

Crashreports to come...when it crashes again.
Right now I am content with closing and opening VDub new for every pass.

TGMC alone works nicely here on both engines since 2009...

[bloodem]

I uninstalled CoreAVC and Haali, tried DGAVCDec, the problem persists (VirtualDub crash) using scripts such as this one (with memory variations, thread variations, etc):

SetMTmode(5,4)
SetMemoryMax(1536)
LoadPlugin("c:\dga\DGAVCDecode.dll")
AVCSource("D:\00003.dga"
DirectShowSource("D:\00004.MTS")
AssumeTFF()
SetMTmode(2)
TempGaussMC_beta1u(tr2=0, EdiMode="nnedi2")

[bloodem]

Yep, the problem is TempGaussMC. I removed the line and it works without a glitch.
I guess I should try the new TempGaussMC, but like I said, I don't really like the quality :(

[bloodem]

Well, it looks like I found a satisfying setting that works. What I did: I downloaded nnedi3 instead of nnedi2 and changed the TempGaussMC line to TempGaussMC_beta1u(tr2=0, EdiMode="nnedi3"). I also downloaded the latest versions of MVTools2, MaskTools, etc

With SetMTmode(5,3) and SetMemoryMax(768) it works flawlessly (2.4 FPS). If I increase the memory size or the number of threads, VirtualDub crashes.


Later edit: Well, it looks like I spoke too soon. It's not always stable. Sometimes it crashes right at 90% :( There must be something I'm missing.
Didee, could you please give me a link to the latest version of TGMC? I can't seem to find it right now...

[Didée]

Here you go: TGMC beta-2 (http://forum.doom9.org/showthread.php?p=1378526#post1378526). That's the original beta2 that I had posted ... the modification to support NNEDI3 you have to put in by yourself. But that's not a problem, methinks.

@Emulgator: It could be that ReduceFlicker doesn't play nicely. It uses "recursion slots", for recursive usage of already-processed clips. Perhaps that plays a role, those slots are handled privately by the filter, not by Avisynth's cache.

However from the past, VirtualDub is known (to me) as sometimes having problems with script reloading, not flushing memory completely. That was a problem many years ago, and it seems the problem still isn't gone?

[bloodem]

Thanks Didee! Well, I don't really like nnedi3...
Here is a comparison: http://img443.imageshack.us/img443/9610/70801924.png (nnedi2)
http://img819.imageshack.us/img819/6656/91224323.png (nnedi3)

So it's back to nnedi2 for me...

Later edit: If I use nnedi2, the crashes return. So I guess the only solution for me is single-threaded. Sucks :(

[um3k]

It looks to me like the "NNEDI3" picture actually used dumb bob, which is the expected outcome of using EdiMode="nnedi3" in a version of TGMC that hasn't been modified to allow NNEDI3.

[bloodem]

And isn't there a modified version, that allows NNEDI3?

[LoRd_MuldeR]

And isn't there a modified version, that allows NNEDI3?

If you can't do this trivial change yourself, get "TempGaussMC_beta2u.avsi" here:
http://forum.doom9.org/showpost.php?p=1400371&postcount=438

[Didée]

You can use any bob-deinterlacer via the "edeint" parameter.

my_wannahave_bob = NNEDI45(field=-2, parameters=to_your_liking)

TGMC_b2(...... , edeint=my_wannahave_bob, .....)

[bloodem]

No need for sarcasm, LoRd_MuldeR. I'm a network security engineer, not a video software developer. I only need a specific set of software, that will help me process my personal video footage. That's it ! I don't need/want to know anything else.
Anyway, thanks for the link, but I already found the modified version of TempGaussMC_beta2, and the problems still won't go away. When I open the script I get this error: http://img46.imageshack.us/img46/1571/errorfe.jpg

These are my files in the Avisynth Plugins Folder (as far as I can tell, they're all up to date): http://img143.imageshack.us/img143/8969/pluginsz.png

[Didée]

Obviously you're using a too old version of RemoveGrain. ("invalid mode 20").

Oh, and ... be sure to not have *both* masktools-25.dll and masktools-26.dll in the plugins directory. In particular, you only need -25.dll. (When running Avisynth 2.5.x)
Remove the 26 one. Remove it right now. It is only for Avisynth 2.6.x.


And while you're at it, use RemoveGrainSSE2 instead of SSE3. Itsa long story whether-or-not the SSE3 version is stable ... you don't lose anything by using the SSE2 one, anyway. Once you have everything running stable, *then* it is time to try SSE3. (To get about 0.0001% more speed out of TGMC:p)

[Didée]

Admittedly, the version dilemma of RemoveGrain is not trivial. To make it easier, this here (http://forum.doom9.org/showthread.php?p=976573#post976573) is the version of RemoveGrain that I am using personally. From that, use the SSE2 versions.
(The link states "Version 1.0", but in fact, it is "pre-release#1 of v1.0". Opposed to pre-release#2 (which breaks backwards compatibility), and opposed to final 1.0 (which breaks backwards compatibility like #2, adds a bunch of new modes, but doesn't have any documentation at all...)

[bloodem]

Thanks Didee. It was listed as being a beta, that's why I was using the "stable" version: RemoveGrain v0.9 (as I can see, all versions are ancient, from 2005, specifically).
Well, it all works now and the quality seems to be just as good as before, if not better. I stole your script line: "TGMC_b2(1,1,0, EdiMode="nnedi3", SVthin=0)
I have no idea what all those parameters mean, but they work for me. :-)

Later edit: Surprise, surprise! Multithreading works like a charm now !

[Usedocne]

A settings question. Say I'm denoising a mixed source "progressive+interlaced", and I'm using MC_Spuds(mode="medium",prefilter=0,strength=2,thSCD1=100) for the progressive part(s). Is there a way to config TGMC to denoise at the exact same strength as the MC_Spuds settings for the interlaced part(s)?

[Didée]

Is there a way to config TGMC to denoise at the exact same strength as the MC_Spuds settings for the interlaced part(s)?

No. Not possible.

[Usedocne]

No. Not possible.

:thanks: for the answer and highlighted words Didée. Guess I should have typed similar/very similar strength instead of exact... So asking the same question again but substituting similar/very similar for exact. Could it be done and if so, what settings should I be looking at tweaking?

[Undead Sega]

woah woah woah... there's a TempgaussMC_Beta2??? I always thought TempgaussMC_Beta1 was the latest release of this filter.

[Terranigma]

woah woah woah... there's a TempgaussMC_Beta2??? I always thought TempgaussMC_Beta1 was the latest release of this filter.

Yes, It was updated by Didée earlier of this year, and -Vit- and myself has posted variations of the latest beta. -Vit-'s version is faster and based off of the variation I posted with additional support for EEDI3, and the one I posted is pretty much the same as Didée's with support for nnedi3.

Didée (Official):
TempGaussMC, version beta-2. (http://forum.doom9.org/showthread.php?p=1378526#post1378526)

Terranigma (Unofficial):
TempGaussMC_beta2u (http://forum.doom9.org/showthread.php?p=1400371#post1400371)

-Vit- (Unofficial):
QTGMC (http://forum.doom9.org/showthread.php?t=156028)


Pick your poison :p

[Undead Sega]

ohhh right, okay then. Damn looks like im gonna have to change a few things again. I must say, TempgaussMC is like the best deinterlacer out there :D I use it for everything now.

In the plugins folder of avisynth, Do i just place the filters/.dll's in there directly? Or can it be in folders and it can still be recognized?

[Terranigma]

In the plugins folder of avisynth, Do i just place the filters/.dll's in there directly? Or can it be in folders and it can still be recognized?

Well, if you want them to auto-load, you'd need to put them in avisynth's "plugins" directory. The only way to load them from folders outside of "plugins" would be to import each plugins/scripts independently.

[Mounir]

Where can i download TempGaussMC beta2 i don't find any OFFICIAL topic

[bin_ch]

Where can i download TempGaussMC beta2 i don't find any OFFICIAL topic

It's just a few posts above.

[Redsandro]

Also speed issues with TempGaussMC_beta2.

Testing on a 15 second clip gives me 2 fps.
Using the same script on the original (same codec) 50 minute footage from which the 15 second test clip was taken, gives me 0.2 fps. :eek:

SetMTMode had minimal (micromal) speed improvement but with too many crashes not usable anyway for 50 minute footage at 0.2 FPS.

Don't get why the length of the clip has drastic FPS influence.

[Taurus]

Also speed issues with TempGaussMC_beta2.

Testing on a 15 second clip gives me 2 fps.
Using the same script on the original (same codec) 50 minute footage from which the 15 second test clip was taken, gives me 0.2 fps.

SetMTMode had minimal (micromal) speed improvement but with too many crashes not usable anyway for 50 minute footage at 0.2 FPS.

Don't get why the length of the clip has drastic FPS influence.
You did not tell us which hardware, which OS, which avisynth version, which plugins you are using.
I'm using TempGaussMC_beta2 and/or QTGMC on an almost daily basis.
And getting about 5 - 8fps with heavy pre and past processing.
This is on SD material.
And yes everything's MT'd without any glitch.
Have you read the first post of Vit in the QTGMC thread?
http://forum.doom9.org/showthread.php?t=156028

[Redsandro]

Indeed I neglected to tell you this, because on ANY given combination of hard- and software, the amount of time given to process a single frame should not increase when the total number of frames increases from a lot to rediculously many.

But for your entertainment, I use AviSynth 2.5.8-mt (32 bit) (unofficial download linked from the AviSynth download page) in Windows 7 Pro 64 with VirtualDub 1.9.10 (32 bit) on an Intel Core 2 Quad Q6700 @ 2.66GHz with 8GB RAM and an NVIDIA GeForce GTS 450

The unabridged script I am using is:
## RED DGDecode Template
## Decode from HV20 709, cropping bad supers
## www.Redsandro.com

# Keep it down, dunno if it helps, I don't notice swapping if set to 1 and I don't know optimal value for HD content.
#SetMemoryMax(8)

#SetMTMode() random crash

# Default - Open for export (quality - slow)
AviSource("Festifal PAL50i.avi")
# DirectShowSource("Festifal PAL50i.avi")


Loadplugin("D:\Program Files (x86)\AviSynth 2.5\plugins\MT.dll")
#SetMTmode(2) # no
ColorMatrix(dest=2,hints=false,interlaced=true)

LoadPlugin("D:\Program Files (x86)\AviSynth 2.5\plugins\TDeint\TMM.dll")
LoadPlugin("D:\Program Files (x86)\AviSynth 2.5\plugins\TDeint\TDeint.dll")
LoadPlugin("D:\Program Files (x86)\AviSynth 2.5\plugins\nnedi2\nnedi2.dll")
LoadPlugin("D:\Program Files (x86)\AviSynth 2.5\plugins\mvtools\mvtools2.dll")
LoadPlugin("D:\Program Files (x86)\AviSynth 2.5\plugins\VerticalCleaner\VerticalCleaner.dll")

AssumeTFF()
TempGaussMC_beta2(2,2,3,EdiMode="NNEDI2",qual=3)

# "pic-000000.png", "pic-000001.png", etc.
#ImageWriter(file="ST_SD", type="png")

#nnedi2_rpow2(rfactor=2,cshift="spline36resize",fwidth=1440,fheight=1280)

#ImageWriter(file="ST_1440", type="png")

#SRestore(FRate=25)

# 709 RGB Conversion
#ConvertToRGB(matrix="PC.709", interlaced=false)
# HV20 based levels
#Levels(16, 1.07, 252, 0, 255, coring=false)
# Lineair rescaling
#Levels(16, 1, 252, 0, 255, coring=false)

# Resize to full-HD
#Lanczos4Resize(1920, 1080)
#BicubicResize(1920, 1080)


Thanks for the link though, turns out I was using MT wrong. But that's not really the issue here.

[-Vit-]

Don't get why the length of the clip has drastic FPS influence.
It doesn't normally. However, I wonder if you're running out of memory (only 2Gb available to Avisynth regardless of your physical RAM). That *might* have effects that worsen as the clip progresses.

First I guess that the source is high resolution given those processing fps and machine spec (otherwise you're in real trouble).

Now are you encoding directly to x264? If so then both x264 and Avisynth are sharing the available 2Gb. That wouldn't normally be a problem single-threaded, except for your setting choices. You've set maximum temporal radiuses using lots of memory (as well as smoothing the output). And qual=3 may use more memory in NNEDI2 as well, and it certainly slows things down. (can't imagine it is necessary at HD, the docs only recommend qual>1 for single image enlargement).

If this is fairly HD input then you need to:
a) Use faster settings (ultra settings really not needed for HD and qual=3 not even needed for SD), or
b) Render in two passes, first avisynth to lossless, then lossless to x264 (need 100s GB free space for a HD lossless file though), or
c) both

You could also try piping the output of avisynth to x264 to use more of your 8GB using avs2pipe or avs2yuv

[Redsandro]

I think I'm in real trouble. The HD comments are there because I used my default import template as a base for making this script. In this case it's a PAL (SD) footage file I am making progressive. The first 10000 frames I encoded to images, then I cancelled and now I am encoding to Lagarith lossless .avi. I thought maybe the images were somehow the problem, but they're not. I encode to lossless because this is all an intermediate step in my processing adventure, and very much so the slowest.

I always encode to X264 in a final separate step without any extra processing, and usually piping to 64 bit x264.

My current encode is estimated to last another 47 hours. Would you dare propose some settings that give a near-similar result so I can cancel this operation and continue in a faster pace, without a notable change of quality halfway through the footage?

-update-

By the way, right now, VirtualDub has 313 MB private bytes with a 302 MB working set. AviSynth is loaded through VirtualDub at the moment. I don't know if the working set is part of the private bytes, but even together, sounds like there is room for more. :)

[-Vit-]

Your workflow sounds OK (maybe try other lossless codecs? or encode to a different drive than the source?). Dunno why you're getting the strange slower fps on longer clip problem. However, that CPU is a little underpowered, so reducing the settings is a good idea and may help.

Definitely get rid of qual=3, you will not see the difference at playback.

Do you have to use TempGaussMC_beta2? Because QTGMC will be faster and virtually equivalent. And you can experiment with the presets to find one that is fast enough for your machine, and similar enough to your current encode.
So just try:
QTGMC("Slower") # or "Slow", "Medium", "Fast", "Faster, "Very Fast", "Super Fast"...

If you need a bit more smoothing, then add TR2=2 or 3 but these will slow it down, e.g.
QTGMC("Medium", TR2=2)

And/or you may wish to tweak sharpness:
QTGMC("Fast", Sharpness=0.75) # Default is 1.0

Suggest you also attempt to follow the multithreading tips in the QTGMC thread you were previously referred to. I strongly recommend SEt's 2.6 MT Avisynth for that.

[Redsandro]

Yes I tried images and huffy for a short while, no change really. It's not the codecs anyway, because I notice the same difference when I just open the scripts in VirtualDub and press play.

I actually tried QTGMC(Preset="Slow") before I started my new deinterlacing adventure (in the past I just used tdeint with nnedi2), and in comparison with TempGaussMC_beta2(2,2,3,EdiMode="NNEDI2",qual=3), I noticed TGMC was actually a lot nicer to my eye for high frequency data.

QTGMC showed more interpolated 2x2 'like' pixels where TGMC had more 1x1 pixels, as if the former did more interpolation and the latter put some actual data there.

Here some fast moving example, you have to look closely, drag images to new tabs and switch back and forth or something, to notice the difference. Look at legs and darker stuff, the QTGMC way seems to provide interpolated like pixels.

http://stuff.rednet.nl/rommel/trash/avs_deint-q.png

http://stuff.rednet.nl/rommel/trash/avs_deint-t.png

http://stuff.rednet.nl/rommel/trash/avs_deint-d.png

If I get to be really picky (and I do :D), qual=1 also leaves out a tad bit too much, although it's only just visible to the eye, but qual=2 could be acceptable. It's just that I'm treating high frequency data as holy because I need to use some SD sources in HD projects, so I like to be as ridiculous as possible with this.

It's just that I don't get why there is a 1000% speed improvement per frame in a clip that is 2% of the original size.

Should I be clever and fanatic enough, I would write a script that cuts the footage in 30 second pieces, TGMC these pieces, and glue them back together. The final result would be 10 times faster. There is probably a logical explanation but it's eating me for not having one.

[-Vit-]

Perhaps you don't realize the underlying equivalence of TGMC and QTGMC. QTGMC "Slow" is a less precise setting. "Slower" is the default. The QTGMC (almost) equivalent settings to yours are:
QTGMC( "Slower", TR0=2, TR1=2, TR2=3, EdiMode="NNEDI2", EdiQual=3 )
[BTW your setting TR2=3 is a temporal smooth - which will reduce high frequency data. Also note that NNEDI2 rarely outperforms NNEDI3, the [faster] QTGMC default]

QTGMC has SourceMatch and Noise bypass modes which explicitly attempt to retain high frequency data. Try this (although on your rig you won't enjoy it...):
QTGMC( "Slower", TR2=3, EZKeepGrain=0.6, SourceMatch=2, Lossless=2, Sharpness=0.2 ) # Adjust EZKeepGrain and Sharpness to taste. Try dropping the TR2=3

[2Bdecided]

I noticed TGMC was actually a lot nicer to my eye for high frequency data.In the pictures you posted, there doesn't seem to be much high frequency data, but QTGMC is retaining a little more noise/grain, which helps the picture look a little less artificial. Though you'll probably add a little noise/grain in HD to help with that.

I had the same slowdown issue you mention, but when encoding straight to X264 via MeGUI from the AVIsynth script itself. Never figured it out.

FWIW I still prefer TGMC alpha3 for some of my footage (it's software, but so is my footage, and there are fewer artefacts), but given the speed of QTGMC I usually use that.

Whatever version of (Q)TGMC you use, a lot of what you see on screen is artificial/interpolated and (with some sources) plastic-looking (it's still by far my favourite deinterlacer though). Noise bypass helps a lot, but whether it's "better" is entirely subjective.

Cheers,
David.

[Redsandro]

Perhaps you don't realize the underlying equivalence of TGMC and QTGMC. QTGMC "Slow" is a less precise setting. "Slower" is the default. The QTGMC (almost) equivalent settings to yours are:
QTGMC( "Slower", TR0=2, TR1=2, TR2=3, EdiMode="NNEDI2", EdiQual=3 )
(..)

QTGMC has SourceMatch and Noise bypass modes which explicitly attempt to retain high frequency data. Try this (although on your rig you won't enjoy it...):
QTGMC( "Slower", TR2=3, EZKeepGrain=0.6, SourceMatch=2, Lossless=2, Sharpness=0.2 ) # Adjust EZKeepGrain and Sharpness to taste. Try dropping the TR2=3
Thanks, I will try that in my next case. I secretly thought I had a pretty descent machine but time flies by so fast. :mad:
Thanks for the settings. Noted for the next batch. :D

Still, the biggest issue is the speed drop, because it COULD work, had I chopped up the footage in pieces, which is strange.

I had the same slowdown issue you mention, but when encoding straight to X264 via MeGUI from the AVIsynth script itself. Never figured it out.
It might be less strange in that situation, -Vit- explained how bottleneck cascading could be the issue in that case as both the script and x264 share limited resource and can be quite hoggy.

Noise bypass helps a lot, but whether it's "better" is entirely subjective.
True, because as for your first perfectly legal statement:
QTGMC is retaining a little more noise/grain, which helps the picture look a little less artificial. Though you'll probably add a little noise/grain in HD to help with that.I disagree. There appears to be more noise, but it's rough noise like I explained, 2x2 'like' oldskool interpolation artefact noise. The TGMC one is definitely better with these settings, you can photoshop with a 16x magnification and see how TGMC is covered more with high frequency data and QTGMC is covered more with 2x2 blobs.

But as -Vit- said that might go away entirely with different settings, I will try them out soon.

Whatever version of (Q)TGMC you use, a lot of what you see on screen is artificial/interpolated and (with some sources) plastic-looking (it's still by far my favourite deinterlacer though).
There's also multiple versions of the QTGMC one?

-update-

The QTGMC (almost) equivalent settings to yours are:
QTGMC( "Slower", TR0=2, TR1=2, TR2=3, EdiMode="NNEDI2", EdiQual=3 )
[BTW your setting TR2=3 is a temporal smooth - which will reduce high frequency data. Also note that NNEDI2 rarely outperforms NNEDI3, the [faster] QTGMC default]
I didn't realize there was such a difference. This looks pretty good. It's making a lot of 'different' decisions according to my friend 16x zoom, but since I cannot decide if it's better or worse, it's a good alternative. Will try your other settings too.

-edit-

Still about the (ongoing) TGMC process, interesting to see how the non-MC version is still using every core. Usually a typical singlethread process involves only one core.
(No I'm not doing anything else at the same time except for a little browsing)
http://stuff.rednet.nl/rommel/trash/avs-deint-cpu.png

-edit-

QTGMC has SourceMatch and Noise bypass modes which explicitly attempt to retain high frequency data. Try this (although on your rig you won't enjoy it...):
QTGMC( "Slower", TR2=3, EZKeepGrain=0.6, SourceMatch=2, Lossless=2, Sharpness=0.2 ) # Adjust EZKeepGrain and Sharpness to taste. Try dropping the TR2=3
Nice indeed! It's just fine for my current source. It's about twice as slow, but still 1,5 fps on a short clip. If it has the same unexplainable speed problem, it will be 0,15 fps on the full clip.

I should try multithreading this, but 2.5.8 mt crashed too much on me. Can I just insert the 2.6.0 DLL without reinstalling, like 2.5.7 and 2.5.8 are interchangable?

[Didée]

Avisynth 32bit, mpeg2 source 576i, i7-860

setmtmode(5,6)
DGsource("source.d2v")
setmtmode(2)
TGMC_b2(1,1,1,edimode="nnedi3")

renders at approx. 33 fps for me

[Redsandro]

i7, what a difference.

2.5.8-MT @ Q6700

SetMTMode(5, 8)
AviSource("test50i.avi")
AssumeTFF()
SetMTMode(2)
QTGMC("Slower", TR0=2, TR1=2, TR2=2, EdiMode="NNEDI3", EdiQual=2, EdiThreads=2)

Renders 3.5 fps for 20 seconds and then crashes VirtualDub.

-edit-

2.5.8-MT @ Q6700

SetMTMode(5, 6)
AviSource("test50i.avi")
AssumeTFF()
SetMTMode(2)
QTGMC("Slower", TR0=2, TR1=2, TR2=2, EdiMode="NNEDI3", EdiQual=2, EdiThreads=2)

5.5 fps, crash after 10 seconds.

-edit-

2.6.0-MT (http://forum.doom9.org/showthread.php?t=148782) as above:
Wildly varying around 4 fps. Crash after [to be edited] seconds.

Wow no crash yet for 10 minutes! But this really hogs my computer. Ofcourse I prefer this speed, but would be nice to have 2% of resources left to move my mouse cursor.

With the fluctuating fps, it's hard to test what number of threads and stuff is the optimal setting for my computer. But if 2.6.0 reaches the finish without crashing, this is waay better. Finish in 4 hours in stead of 2 days.

[Redsandro]

-edit-

Once every few hundred frames or so when the sun don't shine and the moon don't glow, something will break the flow. There's like a loooong delay (tens of minutes) that will up the ETA from 4 hours to 22 hours.
This was the same with 2.5.8 and without MT. At this point vdub is back to using 1 core. Partially. Again, speed would improve if I cancelled everytime this occured and just started from where I cancelled. But I should not have to do this.

The second avs fail. :/

-edit-
..and my first edit fail. Meant to edit my previous post. :P

-edit-
meh
http://stuff.rednet.nl/rommel/trash/avs-deint-vdub.png
This didn't happen on 2.5.8.

-edit-

Using MT, the occasional frame is totally wrong. Imagine every frame has a number, here's an example:
11050, 11051, 11052, 950, 11053, 1104

Makes no sense part 3.

[2Bdecided]

...you can photoshop with a 16x magnification and see...You can, but that is the route to insanity IMO! (and IME!!!) ;)

Try TGMC alpha3 with default settings on those motion-blurred frames. It'll be softer (+ less noise), but may retain real high frequency detail on shallow angle lines better (IME, YMMV!).

Cheers,
David.

[Redsandro]

You can, but that is the route to insanity IMO! (and IME!!!) ;)
Yes, you are right. That's actually not how I compare. My previously stated opinion was based on a naked eye test. The 16x zoom is so you could see for yourself what I was talking about. :)

-Vit- just convinced me to use QTGMC, it's the latest chapter in the TGMC saga, pretty fast, and I like the results. :) It's good for intermediate rendering step for use inside HD projects. But if the final target is SD DVD, those high frequencies are mostly gonna be discarded by the quantizer anyway.