Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion. Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules. |
29th June 2015, 12:43 | #1 | Link |
Registered User
Join Date: Feb 2015
Posts: 34
|
Speeding up my script
I'm using AviSynth+ on a 64-bit machine with i7 (8 threads) and 8GB of RAM. All plugins are 32-bit and so is the x264 build. I couldn't get 64-bit to work because not all plugins are available in 64-bit. Source is 1080p animation.
It takes a lot of time before it even begins encoding, and then it stays at about 0.01fps and completion is ~170 hours away. The output looks perfect (all noticable problems are gone). The main issue is probably MCTemporalDenoise, because if I remove that filter, it takes only a minute or two before it begins encoding, and at a more acceptable 0.4fps (still takes 30 hours), but the output looks bad. When encoding without MCTemporalDenoise, it uses 1.4-1.8GB of memory and 40-60% CPU. AviSynth+ code: Code:
SetFilterMTMode("DEFAULT_MT_MODE", 2) SetFilterMTMode("FFVideoSource", 3) ffVideoSource("input.mkv") MosquitoNR(strength=21) SeparateFields() SelectEven() nnedi3(dh=true, nns=4) Santiag() Santiag() Santiag() Dehalo_alpha() MergeChroma(aWarpSharp(depth=5), aWarpSharp(depth=10)) ChromaShift(c=2) MCTemporalDenoise(settings="medium") GradFun3(radius=16,smode=2) LSFMod(defaults="slow") Prefetch(8) Code:
x264_x86.exe --profile high --level 4.0 --preset veryslow --crf 16 --tune animation --output output.mkv script.avs |
29th June 2015, 17:31 | #3 | Link |
Join Date: Mar 2006
Location: Barcelona
Posts: 5,034
|
Why do you de-interlace when the clip is 1080p?
Anyway, I did a little single threaded test with your script (except MCTemporaldenoise) with a 1080i clip. I used DGDecNV as the source filter with AVS+ r1576. CPU is a i5-2500K Sandy Bridge, 4 cores @ 4GHz. Code:
[General info] Log file created with: AVSMeter 2.0.5 (x86) Script file: F:\Test\1080i50.avs Command line switches: -log Avisynth version: AviSynth+ 0.1 (r1576, x86) (2.6.0.5) Avisynth.dll linker/compiler version: 11.0 (VC 2012) Avisynth DLL location: E:\Apps\Div\AviSynth.dll AutoLoadDir 1 (x86): E:\Apps\VideoTools\AVSPlugins\AutoLoad [Clip info] Number of frames: 4817 Length (hh:mm:ss.ms): 00:03:12.680 Frame width: 1920 Frame height: 1080 Framerate: 25.000 (25/1) Colorspace: YV12 Audio channels: n/a Audio bits/sample: n/a Audio sample rate: n/a Audio samples: n/a [Runtime info] Frames processed: 187 (0 - 186) FPS (min | max | average): 0.942 | 2.777 | 2.698 Memory usage (phys | virt): 1025 | 1176 MB Thread count: 63 CPU usage (average): 55% Time (elapsed): 00:01:09.465 [Script] LoadPlugin("E:\Apps\VideoTools\DGDecNV\DGDecodeNV.dll") DGSource("F:\Test\1080i50.dgi") MosquitoNR(strength=21) SeparateFields() SelectEven() nnedi3(dh=true, nns=4) Santiag() Santiag() Santiag() Dehalo_alpha() MergeChroma(aWarpSharp(depth=5), aWarpSharp(depth=10)) ChromaShift(c=2) GradFun3(radius=16,smode=2) LSFMod(defaults="slow") You probably don't have to multi-thread that script. For example, NNEDI3, which is internally multi-threaded, runs in at least 4(!) instances in your script and is the main CPU hog.
__________________
Groucho's Avisynth Stuff Last edited by Groucho2004; 29th June 2015 at 18:21. |
29th June 2015, 18:54 | #4 | Link | ||
Registered User
Join Date: Feb 2015
Posts: 34
|
Thanks for the replies
Large frames? If you mean the resolution of the frames, it's 1920x1080. If you mean the size, source bitrate is about 10000kbit/s. Framerate is 25, duration about 23¾ minutes. Quote:
To get rid of some horrible aliasing that was in the video. It was progressive and got deinterlaced, and that's the video I'm trying to fix with this script. Quote:
Code:
E:\avs\>avsmeter.exe script.avs -log AVSMeter 2.0.5 (x86) AviSynth+ 0.1 (r1689, x86) (2.6.0.5) Number of frames: 550 Length (hh:mm:ss.ms): 00:00:22.000 Frame width: 1920 Frame height: 1080 Framerate: 25.000 (25/1) Colorspace: YV12 Frame (current | last): 15 | 549 FPS (cur | min | max | avg): 0.050 | 0.003 | 0.086 | 0.016 Memory usage (phys | virt): 2546 | 2795 MB Thread count: 257 CPU usage (current | average): 25% | 29% Time (elapsed | estimated): 00:16:41.745 | 09:33:54.984 |
||
29th June 2015, 21:32 | #6 | Link |
Join Date: Mar 2006
Location: Barcelona
Posts: 5,034
|
Highly unlikely with 257 threads.
Edit: Is this with or without MCTemporalDenoise?
__________________
Groucho's Avisynth Stuff Last edited by Groucho2004; 29th June 2015 at 21:40. |
29th June 2015, 21:58 | #7 | Link |
Join Date: Mar 2006
Location: Barcelona
Posts: 5,034
|
Made another test with MCTemporalDenoise included:
Code:
[General info] Log file created with: AVSMeter 2.0.5 (x86) Script file: F:\Test\1080i50.avs Command line switches: -log Avisynth version: AviSynth+ 0.1 (r1576, x86) (2.6.0.5) Avisynth.dll linker/compiler version: 11.0 (VC 2012) Avisynth DLL location: E:\Apps\Div\AviSynth.dll AutoLoadDir 1 (x86): E:\Apps\VideoTools\AVSPlugins\AutoLoad [Runtime info] Frames processed: 71 (0 - 70) FPS (min | max | average): 0.205 | 1.269 | 1.120 Memory usage (phys | virt): 3246 | 3458 MB Thread count: 63 CPU usage (average): 47% Time (elapsed): 00:01:03.386 [Script] setmemorymax(2500) LoadPlugin("E:\Apps\VideoTools\DGDecNV\DGDecodeNV.dll") DGSource("F:\Test\1080i50.dgi") MosquitoNR(strength=21) SeparateFields() SelectEven() nnedi3(dh=true, nns=4) Santiag() Santiag() Santiag() Dehalo_alpha() MergeChroma(aWarpSharp(depth=5), aWarpSharp(depth=10)) ChromaShift(c=2) MCTemporalDenoise(settings="medium") GradFun3(radius=16,smode=2) LSFMod(defaults="slow") As you can see, the memory used is almost at the limit for AVS32 (4GB) so you would have to pipe this script to x264 (64 bit) with avs2yuv or similar. You might also consider a less resource-chewing denoiser. Edit: Which i7 CPU exactly do you have? There are several generations with vastly different performance.
__________________
Groucho's Avisynth Stuff Last edited by Groucho2004; 29th June 2015 at 22:01. |
29th June 2015, 23:03 | #8 | Link | |||
Registered User
Join Date: Feb 2015
Posts: 34
|
Quote:
See the attachments (script.zip) Quote:
Quote:
This is with |
|||
29th June 2015, 23:06 | #9 | Link | |
Registered User
Join Date: Feb 2015
Posts: 34
|
Quote:
Old one, i7-2600CPU@3.40GHz |
|
29th June 2015, 23:19 | #10 | Link |
Join Date: Mar 2006
Location: Barcelona
Posts: 5,034
|
If you attach something here it can take days to be approved. It's just a script, post it here and use [code] tags, you did it in your first post.
Odd. That would mean that you didn't install Avisynth properly. Use the installer from avs-plus.net.
__________________
Groucho's Avisynth Stuff |
29th June 2015, 23:27 | #11 | Link |
Join Date: Mar 2006
Location: Barcelona
Posts: 5,034
|
It's simple.
Here is avs2yuv which you need. The command line would be (using the same parameters from your first post): Code:
avs2yuv.exe "source.avs" -o - | x264.exe - --profile high --level 4.0 --preset veryslow --crf 16 --tune animation --output "target.mkv" --demuxer y4m
__________________
Groucho's Avisynth Stuff |
30th June 2015, 08:17 | #12 | Link | |
Registered User
Join Date: Feb 2015
Posts: 34
|
Code:
LoadPlugin("C:\Program Files (x86)\AviSynth+\pluginsx32\ffms2.dll") LoadPlugin("C:\Program Files (x86)\AviSynth+\pluginsx32\MosquitoNR.dll") LoadPlugin("C:\Program Files (x86)\AviSynth+\pluginsx32\nnedi3.dll") LoadPlugin("C:\Program Files (x86)\AviSynth+\pluginsx32\gradfun2db.dll") LoadPlugin("C:\Program Files (x86)\AviSynth+\pluginsx32\aWarpSharp.dll") LoadPlugin("C:\Program Files (x86)\AviSynth+\pluginsx32\masktools2-25.dll") LoadPlugin("C:\Program Files (x86)\AviSynth+\pluginsx32\FFT3DFilter.dll") LoadPlugin("C:\Program Files (x86)\AviSynth+\pluginsx32\ChromaShift.dll") LoadPlugin("C:\Program Files (x86)\AviSynth+\pluginsx32\mvtools2.dll") LoadPlugin("C:\Program Files (x86)\AviSynth+\pluginsx32\RemoveGrainSSE3.dll") LoadPlugin("C:\Program Files (x86)\AviSynth+\pluginsx32\RepairSSE3.dll") LoadPlugin("C:\Program Files (x86)\AviSynth+\pluginsx32\dither.dll") Import("C:\Program Files (x86)\AviSynth+\pluginsx32\dither.avsi") Import("C:\Program Files (x86)\AviSynth+\pluginsx32\LSFmod.v1.9.avsi") Import("C:\Program Files (x86)\AviSynth+\pluginsx32\santiag.avsi") Import("C:\Program Files (x86)\AviSynth+\pluginsx32\MCTemporalDenoise.v1.4.20.avsi") Import("C:\Program Files (x86)\AviSynth+\pluginsx32\Dehalo_alpha_MT2.avsi") Import("C:\Program Files (x86)\AviSynth+\pluginsx32\mt_xxpand_multi.avsi") #SetFilterMTMode("DEFAULT_MT_MODE", 2) #SetFilterMTMode("FFVideoSource", 3) ffVideoSource("input.mkv") MosquitoNR(strength=21) SeparateFields() SelectEven() nnedi3(dh=true, nns=4) Santiag() Santiag() Santiag() Dehalo_alpha() MergeChroma(aWarpSharp(depth=5), aWarpSharp(depth=10)) ChromaShift(c=2) MCTemporalDenoise(settings="medium") GradFun3(radius=16,smode=2) LSFMod(defaults="slow") #Prefetch(8) Quote:
|
|
30th June 2015, 09:17 | #13 | Link | |
Join Date: Mar 2006
Location: Barcelona
Posts: 5,034
|
Quote:
I see that you're using some outdated plugins (masktools25, removegrain), I suggest that you use this plugin package. It will also make things faster in single threaded mode. I ran a few (longer) test with your script and it turns out that it will never work with 32 Bit Avisynth. MCTemporaldenoise uses too much memory. You either hit the 4GB ceiling or it starts slowing down after a few minutes. You have to use a different denoiser.
__________________
Groucho's Avisynth Stuff |
|
30th June 2015, 12:23 | #14 | Link | ||
Registered User
Join Date: Feb 2015
Posts: 34
|
Quote:
Quote:
|
||
30th June 2015, 12:26 | #15 | Link | |
Join Date: Mar 2006
Location: Barcelona
Posts: 5,034
|
Quote:
Edit: I guess KNLMeansCL is not quite what you're looking for. How about MDegrain? It features motion compensation.
__________________
Groucho's Avisynth Stuff Last edited by Groucho2004; 30th June 2015 at 13:07. |
|
30th June 2015, 13:46 | #16 | Link | |
Registered User
Join Date: Feb 2015
Posts: 34
|
Quote:
Code:
error: Script error: Invalid arguments to function 'MDeGrain2'. |
|
30th June 2015, 14:12 | #17 | Link |
Join Date: Mar 2006
Location: Barcelona
Posts: 5,034
|
The MVTools version I posted is from this post. Parameters have changed. Download it and check the included documentation.
__________________
Groucho's Avisynth Stuff |
30th June 2015, 14:46 | #19 | Link | |
Join Date: Mar 2006
Location: Barcelona
Posts: 5,034
|
Quote:
Code:
super = MSuper(pel = 2, sharp = 1) backward_vec2 = MAnalyse(super, isb = true, delta = 2, overlap=4) backward_vec1 = MAnalyse(super, isb = true, delta = 1, overlap=4) forward_vec1 = MAnalyse(super, isb = false, delta = 1, overlap=4) forward_vec2 = MAnalyse(super, isb = false, delta = 2, overlap=4) MDegrain2(super, backward_vec1,forward_vec1,backward_vec2,forward_vec2,thSAD=400)
__________________
Groucho's Avisynth Stuff Last edited by Groucho2004; 30th June 2015 at 14:49. |
|
30th June 2015, 22:45 | #20 | Link |
ангел смерти
Join Date: Nov 2004
Location: Lost
Posts: 9,558
|
If you're going to use MDegrain, stick with SMDegrain, it's so much simpler and more flexible. Also, Vapoursynth makes the whole 32-bit ceiling go away and it's quite clean once you get used to it, eliminating the swapping that's killing your speed, but that's a personal decision.
If you upload an chunk of the raw source, it might help in removing some of the other pain points. |
Thread Tools | Search this Thread |
Display Modes | |
|
|