MCTemporalDenoise [v1.4.20 - Update 2010/07/02] [Archive] - Page 9

asarian

29th June 2010, 20:21

LaTo, I may not be doing it entirely right yet. :) I'm using this avs script on a 1-minute clip:

LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\FFMS2.dll")
Loadplugin("C:\Program Files\AviSynth 2.5\plugins\FFT3DFilter.dll")
Loadplugin("C:\Program Files\AviSynth 2.5\plugins\mt_masktools-25.dll")
Loadplugin("C:\Program Files\AviSynth 2.5\plugins\mvtools2.dll")
Loadplugin("C:\Program Files\AviSynth 2.5\plugins\RemoveGrainSSE3.dll")
Loadplugin("C:\Program Files\AviSynth 2.5\plugins\TTempSmooth.dll")
Loadplugin("C:\Program Files\AviSynth 2.5\plugins\RepairSSE3.dll")
Loadplugin("C:\Program Files\AviSynth 2.5\plugins\deblock.dll")
Loadplugin("C:\Program Files\AviSynth 2.5\plugins\DctFilter.dll")
Loadplugin("C:\Program Files\AviSynth 2.5\plugins\EEDI2.dll")
Loadplugin("C:\Program Files\AviSynth 2.5\plugins\SangNom.dll")
Loadplugin("C:\Program Files\AviSynth 2.5\plugins\gradfun2db.dll")
Loadplugin("C:\Program Files\AviSynth 2.5\plugins\AddGrainC.dll")

video=FFVideoSource("M:\jobs\2010test.mkv").ConvertToYV12()
video=MCTemporalDenoise(video, settings="very high")
video=Crop(video, 0, 142, 0, -142)

return video

Nothing happens for a long while, though. First I get a Windoze error, saying it can't use "mt_masktools-25-x64.dll" (of course it can't: I'm on 32bit; and I didn't ask for it to be run, either). Then x264 keeps hovering around 25% CPU, steadily increasing in memory size (to around 1.4GB). And finally it tells me that 1 minute clip, at 0.00fps, (sic!) will take over 300 hours! Why, that seems a mite extreme. :)

x264 remains at ca. 25% CPU. So, I guess I'm doing something wrong. Any hints?
Anyone? Anyone? Bueller? Bueller? :)

Seriously; I may not have the fastest machine (Q6700 @ 3.6 Ghz); but 300 hours for a 1 minute clip??! That's looking at a ~4 year encode!!

I could probably crop a little more efficiently; like first do 400, -400, so at to still be div 16 (one of the filters was complaining about that), and only then do MCTemporalDenoise, and only then crop an extra 2, -2 afterwards:

video=FFVideoSource("M:\jobs\2010test.mkv").ConvertToYV12()
video=Crop(video, 0, 140, 0, -140)
video=MCTemporalDenoise(video, settings="very high")
video=Crop(video, 0, 2, 0, -2)

But still, these encoding times seem too long (and unworkable; I mean, who can keep his machine on for 4 years? Hoping it won't reset in the meantime).

Gavino

29th June 2010, 22:36

I get a Windoze error, saying it can't use "mt_masktools-25-x64.dll" (of course it can't: I'm on 32bit; and I didn't ask for it to be run, either).
Is mt_masktools-25-x64.dll in your plugins folder (C:\Program Files\AviSynth 2.5\plugins)? If so, it will be loaded automatically by Avisynth, so you want to remove it from there.

For the same reason, all your LoadPlugin commands are unnecessary.

asarian

30th June 2010, 09:51

Is mt_masktools-25-x64.dll in your plugins folder (C:\Program Files\AviSynth 2.5\plugins)? If so, it will be loaded automatically by Avisynth, so you want to remove it from there.

For the same reason, all your LoadPlugin commands are unnecessary.
Yeah, I put mt_masktools-25-x64.dll there recently, as part of upgrading all required tools for MCTemporalDenoise. Didn't realize they all autoload. :)

P.S. Any idea why x264 constantly only takes about 25% CPU for MCTemporalDenoise? That doesn't exactly help encoding times either. :(

LaTo

30th June 2010, 09:54

Yeah, I put mt_masktools-25-x64.dll there recently, as part of upgrading all required tools for MCTemporalDenoise. Didn't realize they all autoload. :)

P.S. Any idea why x264 constantly only takes about 25% CPU for MCTemporalDenoise? That doesn't exactly help encoding times either. :(

x264 wait a long time after MCTD... Because avisynth isn't multithreaded, MCTD use only one core of your quad (25%).

asarian

30th June 2010, 14:27

x264 wait a long time after MCTD... Because avisynth isn't multithreaded, MCTD use only one core of your quad (25%).
LaTo, you're my new hero!! :)

I just tried my sample with 'settings=high', and instead of taking 300 hours (for 'very high'), it was done in 30 minutes! :) Which is very reasonable. And the result (http://little-albatross.net/2010test.rar)? Not only does MCTemporalDenoise pass the SC test (whereas TemporalDegrain, on the defaults, failed), but the resulant video is spendid! Beyond belief, so good actually!!

I can't get 'GPU=true' to work yet (GTX 260), though:

avs [info]: 1920x796p 1:1 @ 24000/1001 fps (cfr)
x264 [info]: using SAR=1/1
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 Cache64
x264 [info]: profile High, level 4.1
x264 [error]: malloc of size 6856704 failed/s, eta 19:40:54
x264 [error]: x264_encoder_encode failed

Not sure what that is all about, but, regardless, I'm already happy. :)

LaTo

2nd July 2010, 11:48

v1.4.20 :
tweaked some settings
updated QED_Deblock version

:)

Boulder

2nd July 2010, 11:55

Thanks :)

Are you by any chance planning on doing a quick-and-dirty fix for the missing scenechange detection?

LaTo

2nd July 2010, 13:16

Thanks :)

Are you by any chance planning on doing a quick-and-dirty fix for the missing scenechange detection?

Yes it's on my todo list!

asarian

5th July 2010, 05:56

When I run MCTemporalDenoise with 'enhance=true' ("medium"), suddenly a GHOST PORTRAIT (http://www.mediafire.com/file/idmtow3dokh/2rachels.jpg) appears in one of the frames in my Blade Runner encode. Oops! :) When I add a GradFun2DBmod command after MCTemporalDenoise, the issue is resolved again.

Maybe this needs a little looking into?

LaTo

5th July 2010, 08:01

asarian

5th July 2010, 08:07

It's not a problem with enhance=true because it's only spatial.
Do you use DirectShowSource() or GPU=true in your script?
No, I always use FFVideoSource for frame consistency.

I did use 'GPU=true', though. How would GPU usage cause just a temporal displacement, though?

EDIT: I *do* stream using avs2yuv now, though, because otherwise x264 would run out of memory. Could using avs2yuv to stream make things frame inconsistent again?

LaTo

5th July 2010, 08:14

No, I always use FFVideoSource for frame consistency.

I did use 'GPU=true', though. How would GPU usage cause just a temporal displacement, though?

EDIT: I *do* stream using avs2yuv now, though, because otherwise x264 would run out of memory. Coould using avs2yuv to stream make things frame inconsistent again?

GPU=true causes ghosting when seeking, moreover I have somes issues with HD-1080p.
So GPU=false is the default :D

For avs2yuv, I don't know... Maybe, I never use it.

asarian

5th July 2010, 08:32

GPU=true causes ghosting when seeking, moreover I have somes issues with HD-1080p.
So GPU=false is the default :D

For avs2yuv, I don't know... Maybe, I never use it.
Well, that's it for "GPU=true" then. :) Wasn't getting a speed improvement out of it anyway; just a memory benefit.

The avs2yuv frame consistency matter has got me worried a bit (if it is, indeed, an issue). Maybe someone else knows?

asarian

6th July 2010, 02:11

Okay, some very weird stuff going on. I used this following line for x264 on this Total Recall TEST SCENE (http://www.mediafire.com/file/jmmziew4f3n/trecalltest.mkv):

"C:\x264\avs2yuv.exe" "M:\jobs\trecalltest.avs" -o - | "C:\x264\x264.exe" - --demuxer y4m --crf 14 --sar 1:1 --aud --nal-hrd none --level 4.1 --preset placebo --vbv-bufsize 70000 --vbv-maxrate 60000 --bframes 8 --merange 32 --aq-mode 2 --ref 4 --tune grain --output "C:\video\trecalltest.264"

And the AVS scripts is as follows:

FFVideoSource("M:\jobs\trecalltest.mkv").ConvertToYV12()
Crop(0, 4, 0, -4)
MCTemporalDenoise(settings="medium", enhance=true)
Crop(0, 20, 0, -20)

The ghosting/misplaced frames in the RESULT (http://www.mediafire.com/file/lzgjdfnfkwx/trecalltest1410grain2.m2ts) are numerous. :( What the heck is going on!? The weird part is, that I can get this to work properly when I vary CRF and qpmin some. For instance, CRF = 10 + QPMIN = 10 gives me a flawless encode. This faulting file, however (CRF = 14 + QPMIN = 10) totally flips out. You'd think that makes it a x264 problem; but not so! If I run just the AVS script alone, and watch the output in GOM Player (very slowly, of course), I get some frame mess-ups too.

So, it seems to be a timing problem of some sort rather (in which x264 settings, only indirectly, play a role). Could someone test the above clean sample with the same x24 settings I used, please? Very mysterious things going on. :)

asarian

6th July 2010, 08:44

It's not a problem with enhance=true because it's only spatial.

LaTo, I'm really thinking it may be 'enhance=true' after all. When I remove the enhance option from the MCTemporalDenoise line, and export it simply as a second post-filter, everything goes well again!

FFVideoSource("M:\jobs\trecalltest.mkv").ConvertToYV12()
Crop(0, 4, 0, -4)
MCTemporalDenoise(settings="medium")
GradFun2DBmod(thr=1.4,thrC=1.0,mode=0,str=1.2,strC=0.0,temp=50,adapt=64,mask=false,show=false)
Crop(0, 20, 0, -20)

This script, contrary to the other, yields a perfect encode, under any conditions! So, maybe the 'in-house' enhance feature from MCTemporalDenoise corrupts something internally on occasion (like a buffer accidentally being overwritten? Or some variables maybe?). Whatever the case may be, doing GradFun2DBmod seperately definitely solves the issue, on every test I've run so far.

LaTo

6th July 2010, 09:21

And with temp=-1 ..?

Which version of avisynth do you use?

asarian

6th July 2010, 10:33

And with temp=-1 ..?
No, that makes no difference: same messed-up output.

Which version of avisynth do you use?
I'm using regular AviSynth version 2.5.8 (non-MT!). Why? You think it's a threads issue somehow?

Anyway, thanks for looking into it.

LaTo

6th July 2010, 10:35

Really strange.... I will try with your sample+script.

asarian

6th July 2010, 10:42

Really strange.... I will try with your sample+script.
Thanks, appreciate it. :)

In the meantime at least I can start a 'sane' encode again, using GradFun2DBmod on a separate line.

LaTo

6th July 2010, 12:47

All is working, no problem on my side.

Seriously, it looks like a "source" problem... Replace FFVideoSource by something else.

asarian

6th July 2010, 16:28

All is working, no problem on my side.

Seriously, it looks like a "source" problem... Replace FFVideoSource by something else.
Thanks for your time, LaTo.

FFVideoSource, eh? Hmm, if that's true, then I'm having a problem, as I replace DirectShowSource with FFVideoSource calls, these days, because the former is frame-consistent (and DirectShowSource is not!).

So, is there anything else out there, besides FFVideoSource, which is frame consistent? With a heavy script like MCTemporalDenoise, I really need frame-consistency. :(

LaTo

6th July 2010, 16:32

I haven't FFVideoSource on my PC, so for testing I have replaced this one by DirectShowSource in your script.
No ghosting at all, maybe a bug in your version of FFVideoSource?

Try with DGAVCIndex or DGNVTools, the problem should be solved.

asarian

7th July 2010, 12:32

asarian

29th July 2010, 05:37

I wonder, if I have a really grainy source, could I just double the MCTemporalDenoise line? Like so:

MCTemporalDenoise(settings="medium")
MCTemporalDenoise(settings="medium")
GradFun2DBmod(thr=1.8,thrC=1.0,mode=0,str=2.4,strC=0.0,temp=50,adapt=64,mask=false,show=false)

Instead of using 1x MCTemporalDenoise(settings="high").

Or does that mess up the lookahead functionality of MCTemporalDenoise?

Reason I ask, is that MCTemporalDenoise(settings="high") takes incredibly long on HD material (several hundred hours on my i7 980x). So maybe just doubling is the way to go.

LaTo

29th July 2010, 15:24

asarian

30th July 2010, 09:48

Try with MCTemporalDenoise(settings="high",stabilize=false,bwbh=32,owoh=16).
It should be a good speed boost ;)

Thanks. :) Good tips.

I guess the real issue is still that I can't get MCTemporalDenoise to utilize the CPU properly. At 'high' CPU usage drops tp about 8% (over all 12 logical cores). And an MT"" call, or using any of the SetMTMode calls, makes MCTemporalDenoise crash; at least on my Highlander Blu-Ray.

P.S. Callimg MCTemporalDenoise twice in a row, btw, doesn't seem to work: it uses CPU, but x264 never starts up; so I guess it just hangs at initializong things, or gets into a loop of some sort; or twice 'medium' is simply even slower than 1x 'high'. :)

tormento

30th July 2010, 10:02

I guess the real issue is still that I can't get MCTemporalDenoise to utilize the CPU properly.
AFAIK horizontal/threads and vertical/threads must be multiple of 4. Check it out.

skaleton

22nd August 2010, 02:18

Hello,
Where can I download MCTemporalDenoise?
The link in avisynth.org does not work anymore.

poisondeathray

22nd August 2010, 05:15

Hello,
Where can I download MCTemporalDenoise?
The link in avisynth.org does not work anymore.

Look on the very 1st page of this thread

skaleton

22nd August 2010, 14:41

Thanks poisondeathray
I saw the link but yesterday I didn't know what to do with it. I expected a zip or something like most other filters.
Thanks to another post of yoursmost .dlls are autoloading if put into the avisynth\plugins directory
.avsi are autoloading scripts if put into the plugins directory
.avs are manually loaded - use Import("path\script.avs") . You can rename extension to .avsi to autoloadI just saved the avsi file.

What is not clear to me is what do you do about the required filters?
Do you need to load the 15 filters before loading the avsi? Should you make a script so that they are all loaded in one go?

poisondeathray

22nd August 2010, 14:59

The prerequisite filters are listed at the top of the script. You have to get them (not included)

If the prereq's are .dll's most are autoloading if placed in the plugins folder. Otherwise use LoadPlugin() , or you can make your own custom .avsi file . But beware the more autoloading filters you have, the slower and more memory consumption . Once you start accumulating a bunch of filters, it can make a difference

The reason why a static .zip file is not provded, is the filters are always being updated . You should get them from the original source, with original documentation. People sometimes modify filters. This way no mixups or filter version errors occur when debugging

You don't need all the pre-req's for this one . Only if you use some of the PP functions. It will tell you what one you are missing if you try to use it.

skaleton

23rd August 2010, 00:05

Thank you poisondeathray

So if I understand correctly,
Only the most frequently used .dll's and .avsi's should remain in the plugins folder.
When in another folder, should a .avsi be loaded with "loadplugin" or with "import" like a .avs or should it be renamed .avs?

For this script, I don't need to load all pre-req filters,
the MCTemporalDenoise script will tell me which one I am missing and then I can incorporate it in my .avs
and although it may be easier, as I was asking before, to make a .avs that would contain all the loadplugin/import of the pre-req filters
The first solution would be more efficient in the long run.

Am I correct?

poisondeathray

23rd August 2010, 03:08

When in another folder, should a .avsi be loaded with "loadplugin" or with "import" like a .avs or should it be renamed .avs?

you can rename it .avs and leave it in the plugins folder and it won't load until you use Import()

For this script, I don't need to load all pre-req filters,
the MCTemporalDenoise script will tell me which one I am missing and then I can incorporate it in my .avs
and although it may be easier, as I was asking before, to make a .avs that would contain all the loadplugin/import of the pre-req filters
The first solution would be more efficient in the long run.

Am I correct?

you can organize it however you like :)

some people load everything with loadplugin or import at the top of each and every script

you can make your own avsi that autoloads everything if you want

aand

25th August 2010, 21:46

Any chance of this getting a bit of multithreading? Using GPU is a big boost (0.9 to 1.5 fps). Two instances don't work.

Also I think post-TTempSmooth is broken (i added vis_blur=1 inside the scrip but nothing happened).

asarian

18th September 2010, 10:59

I'm having trouble with MCTemporalDenoise 'hanging' at times. Well, it doesn't really hang, in the sense of not responding any more; it just doesn't output anything any more, like being caught in a loop somehow. Here's a good example of a video fragment (5 MB) causing this, from the movie 'Push':

Push fragment (http://www.mediafire.com/?b66rvl2ig9fql6x)

Processed as follows:

DSS2("c:\video\push_test.m2ts", fps=24).ConvertToYV12()

Crop(0, 132, 0, -132)

MCTemporalDenoise(settings="low")
GradFun2DBmod(thr=1.4,thrC=1.0,mode=0,str=1.2,strC=0.0,temp=50,adapt=64,mask=false,show=false)

It happens with more movies (Stargate); but this sample is nice, as it occurs directly from the start.

I would really like to know what's causing this; and how to fix it, of course.

Thanks.

7ekno

18th September 2010, 12:35

DSS2("c:\video\push_test.m2ts", fps=24).ConvertToYV12()

Use a different source filter, DSS2 is pretty bad at direct M2TS ...

Index it first for a much better chance at success (I know indexing is stable, I tried a few source filters on BD M2TS files and DSS2 was the least likely to complete the encode ... I use DGDecNV to consistently run 30GB M2TS's thru MCTemporalDenoise() for the compression boost) ...

7ek

asarian

18th September 2010, 14:10

Use a different source filter, DSS2 is pretty bad at direct M2TS ...

Index it first for a much better chance at success (I know indexing is stable, I tried a few source filters on BD M2TS files and DSS2 was the least likely to complete the encode ... I use DGDecNV to consistently run 30GB M2TS's thru MCTemporalDenoise() for the compression boost) ...

7ek
Indexing is not really the issue here, I think. I tried it with FFVideoSource too (which I didn't use in this case, as I need to set the fps; FFVideoSource just assumes 23.976).

Mind you, I use MCTemporalDenoise a lot (~300GB a week, as I'm reprocessing my now vast Blu-Ray collection). Mostly it goes very well. There's just a few films where the process gets stuck somehow, like with 'Push'.

EDIT: I'll be d*amned. :) Finally bought a license for DGDecNV (about time; no point in spending $1,000 on a i7 980x CPU, only to skimp out on this cheap decoder license), and now it actually works. :) Guess FFVideoSource isn't at good at indexing as it thinks. I should have done this much earlier!

P.S. DGDecNV is also the only decoder, so far, that gets the framerate right without external help.

Usedocne

18th September 2010, 14:57

kypec

26th September 2010, 09:32

Is there anybody out there who successfully runs this script in MT mode?
I mean, though I've an Intel i7-920 (quad core+HT = 8 threads) I can't make it to use more than 13% of CPU (which effectively means that only 1 thread is always running).
Here's my AVS script:# Initialize variable with path to my A/V tools
global avdir="C:\Program Files (x86)\AVTools\"
SetMTMode(2,2)
# Load all the plugins that will be used in the script
LoadPlugin(avdir + "DGDecNV\DGDecodeNV.dll")
Loadplugin(avdir + "plugins\mvtools2.dll")
Loadplugin(avdir + "plugins\mt_masktools-25.dll")
Loadplugin(avdir + "plugins\FFT3DFilter.dll")
Loadplugin(avdir + "plugins\TTempSmooth.dll")
Loadplugin(avdir + "plugins\RemoveGrainSSE2.dll")
Loadplugin(avdir + "plugins\RepairSSE2.dll")
Loadplugin(avdir + "plugins\deblock.dll")
Loadplugin(avdir + "plugins\DctFilter.dll")
Loadplugin(avdir + "plugins\EEDI2.dll")
Loadplugin(avdir + "plugins\SangNom.dll")
Loadplugin(avdir + "plugins\gradfun2db.dll")
Loadplugin(avdir + "plugins\AddGrainC.dll")

Import(avdir + "plugins\LSFmod.v1.9.avsi")
Import(avdir + "plugins\Deblock_QED.avsi")
Import(avdir + "plugins\GradFun2DBmod.v1.5.avsi")
Import(avdir + "plugins\MCTemporalDenoise.v1.4.20.avsi")

SetMTMode(5)
# Open source video clip
DGSource("300_raw_bd.dgi")

SetMTMode(2)
MCTemporalDenoise(settings="medium")

SelectRangeEvery(FrameCount/12, Round(FrameRate*5), 2400)

I can't select more than 2 threads in initial SetMTMode(2,2) because I'm using only 32-bit Avisynth and any more threads would just crash (tested).
Any suggestions of proven solutions are welcome, quality of LaTo's script is outstanding but not being able to utilize the full power of my CPU efficiently is just so frustrating :(
Maybe 64-bit MT Avisynth build is needed? But are there all required plugins available in 64-bit then?

levi

26th September 2010, 19:35

I mean, though I've an Intel i7-920 (quad core+HT = 8 threads) I can't make it to use more than 13% of CPU

Try disabling HT in the BIOS, I know it sucks but I always get better performance without it. 64 bit is an option, all requirements are available. However improvement in speed are about 10-15% in fps in my experience & does not help with maximizing CPU.

asarian

26th September 2010, 19:46

I can't select more than 2 threads in initial SetMTMode(2,2) because I'm using only 32-bit Avisynth and any more threads would just crash (tested).
Any suggestions of proven solutions are welcome, quality of LaTo's script is outstanding but not being able to utilize the full power of my CPU efficiently is just so frustrating :(
Maybe 64-bit MT Avisynth build is needed? But are there all required plugins available in 64-bit then?

I know SetMTMode is supposed to be the better way; but I'm having bad experiences with it myself (very slow, and not very compatible, regardless of mode). I currently encode just the old MT() call; like:

MT("""MCTemporalDenoise(settings="medium")""", threads=5, overlap=8)

Guaranteed to cause a 100% CPU load.

asarian

26th September 2010, 20:03

Try disabling HT in the BIOS, I know it sucks but I always get better performance without it. 64 bit is an option, all requirements are available.
Disabling HT is not a very good option. Sure, single-threaded processes may run marginally faster; but best is just to properly thread the MCTemporalDenoise call (see also above).

And 64-bit? Since when are all prerequisites for MCTemporalDenoise available in 64-bit?

Didée

26th September 2010, 20:22

Somwhen past, I compared plain source->x264 encoding with and without HT. The maximum performance with HT was 20% faster.
___

OTOH, it seems people just don't understand how very complex & ressource-hungry MCTemporalDenoise actually is, even more so with full-HD input.

With the (blataneously simple) assumption that many not-specifically-optimized processes have a complexity of O(x^2), the (same blataneously-simple) estimation is:

full-HD has 5 times the pixels of SD, requiring 5^2 = 25 times more work to do for processing filters.

levi

26th September 2010, 20:39

Disabling HT is not a very good option. Sure, single-threaded processes may run marginally faster; but best is just to properly thread the MCTemporalDenoise call (see also above).

And 64-bit? Since when are all prerequisites for MCTemporalDenoise available in 64-bit?

I'll definitely try your suggestion. you can find all 64 needs here (http://forum.doom9.org/showthread.php?t=152800).

asarian

27th September 2010, 04:24

I'll definitely try your suggestion. you can find all 64 needs here (http://forum.doom9.org/showthread.php?t=152800).

All my x64 needs? Looking at that threads, it seems SangNom, for instance, doesn't exist for 64-bit yet (needed for MCTemporalDenoise).

kypec

27th September 2010, 20:04

I know SetMTMode is supposed to be the better way; but I'm having bad experiences with it myself (very slow, and not very compatible, regardless of mode). I currently encode just the old MT() call; like:

MT("""MCTemporalDenoise(settings="medium")""", threads=5, overlap=8)

:thanks: a heap for your recommendation! Yeah, I was also a bit sceptical in the beginning about MT(...) due to its spatial splitting nature but eventually after reading your post I thought: "What the heck, MCTD works in temporal domain anyway so why should it be affected in negative way by spatial multithreading?"
Few sample runs seem fairly promising
MT("""MCTemporalDenoise(settings="medium")""", threads=4, overlap=8)
speeded up my encoding performance from 0.95 fps -> 2.33 fps with about same target bitrate as running regular non-MT MCTD. I encode with x264 at CRF=18, Preset=Slow and CPU utilization boosted ~90% all the time. Increasing number of threads to 8 didn't really help, the performance even dropped to 2.14 fps so I think 4 threads are sweet spot for me. :)

asarian

28th September 2010, 06:54

:thanks: a heap for your recommendation! Yeah, I was also a bit sceptical in the beginning about MT(...) due to its spatial splitting nature but eventually after reading your post I thought: "What the heck, MCTD works in temporal domain anyway so why should it be affected in negative way by spatial multithreading?"

It wouldn't. Except perhaps around the edges/borders of the individually processed segments, where you might see some if not artifacts, then visible distortions (so that the whole wouldn't look seamless any more). Fortunately, the overlap takes care of those. :)

Increasing number of threads to 8 didn't really help, the performance even dropped to 2.14 fps so I think 4 threads are sweet spot for me. :)

It depens on your CPU, for one (on how much physical threads it has to give; like my i7 980x has 12). And if you crop HD sources (like a typical 140px off of both top and bottom), you'll find that not all number of threads work: some settings will simply crash on the spot (has to do with internal divisibility).

Sometimes the MT() call will crash regardless. Like if you're 16 hours underway, at 56% or something. Most annoying. Then, when you simply rerun the process, you'll see it get past the previous crash-point without a problem. Can't be helped. Still, the speed increase is simply worth it. And going it 'safe' and always run single-threaded is not a real solution, either, as memory requirements are so high that some HD sources I had simply became stuck somewhere, and only doing the MT() thing solved it.

Again, it's all worth it, though. :) MCTemporalDenoise is the best script ever!

kypec

28th September 2010, 09:44

It depens on your CPU, for one (on how much physical threads it has to give; like my i7 980x has 12). And if you crop HD sources (like a typical 140px off of both top and bottom), you'll find that not all number of threads work: some settings will simply crash on the spot (has to do with internal divisibility).
Yep, setting threads=6 kept failing in my case (1920x800) though 4 or 8 worked fine.

Sometimes the MT() call will crash regardless. Like if you're 16 hours underway, at 56% or something. Most annoying. Then, when you simply rerun the process, you'll see it get past the previous crash-point without a problem. Can't be helped.:eek: Now you have scared me a little, hopefully my full run on 2-hour movie will finish OK at first attempt.
And going it 'safe' and always run single-threaded is not a real solution, either, as memory requirements are so high that some HD sources I had simply became stuck somewhere, and only doing the MT() thing solved it.
Again, it's all worth it, though. :) MCTemporalDenoise is the best script ever!If only all required plugins were available in 64-bit too, that would solve big memory demands this script has on HD material. My rig has 6GB DDR3 that deserve to be (ab)used up to the top now when CPU is on its job! :goodpost: