Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Usage

Reply
 
Thread Tools Search this Thread Display Modes
Old 12th May 2013, 14:19   #1  |  Link
Max147
Registered User
 
Join Date: May 2013
Posts: 12
mvtools2 is very slow

Hello,

I'm new to this forum, I searched before I ask my question, but I have not found.
I'm trying to encode a bluray with megui, but I think it has a lot of grain, it is an old film that has been remastered quickly.
So I use the "mvtools2" I find effective filter, but the problem is that the encoding is very very slow, 0.92 fps per second. My processor is being used than 15%, this is a I7 970.
Does anyone have an idea to improve the encoding time?
Thank you in advance for your answers.

I attached a copy of my avisynth script.

LoadPlugin("C:\Program Files (x86)\MeGUI_0.3.5_x86_WithoutInstaller_OfflinePackage\tools\avisynth_plugin\mvtools2.dll")
LoadPlugin("C:\Program Files (x86)\MeGUI_0.3.5_x86_WithoutInstaller_OfflinePackage\tools\dgavcindex\DGAVCDecode.dll")
AVCSource("J:\_RIP\beverlly hills cop 2\00051.dga")
#deinterlace
crop(0, 132, 0, -132)
#resize
#denoise
super = MSuper(pel=2, sharp=1)
backward_vec3 = MAnalyse(super, isb = true, delta = 3, overlap=4)
backward_vec2 = MAnalyse(super, isb = true, delta = 2, overlap=4)
backward_vec1 = MAnalyse(super, isb = true, delta = 1, overlap=4)
forward_vec1 = MAnalyse(super, isb = false, delta = 1, overlap=4)
forward_vec2 = MAnalyse(super, isb = false, delta = 2, overlap=4)
forward_vec3 = MAnalyse(super, isb = false, delta = 3, overlap=4)
MDegrain3(super, backward_vec1,forward_vec1,backward_vec2,forward_vec2,backward_vec3,forward_vec3,thSAD=1200)
Max147 is offline   Reply With Quote
Old 13th May 2013, 09:18   #2  |  Link
Selur
Registered User
 
Selur's Avatar
 
Join Date: Oct 2001
Location: Germany
Posts: 5,674
Quote:
Does anyone have an idea to improve the encoding time?
You might want to read up on Avisynth MT.
__________________
Hybrid here in the forum, homepage
Selur is offline   Reply With Quote
Old 13th May 2013, 18:32   #3  |  Link
Max147
Registered User
 
Join Date: May 2013
Posts: 12
Hello,

thank you for answering,

* I looked for a good time on avisynth mt, so I downloaded the MT filter 07, for installation I have copied the avisynth.dll in system32, and other files in the plugins folder megui.
* So I tried with this script, but the software can not read the first line of course.

is there something basic that I do not understand?

SetMTMode (2)
LoadPlugin ("C: \ Program Files (x86) \ MeGUI_2341_x86 \ tools \ avisynth_plugin \ mvtools2.dll")
LoadPlugin ("C: \ Program Files (x86) \ MeGUI_2341_x86 \ tools \ dgavcindex \ DGAVCDecode.dll")
AVCSource ("C: \ Users \ Geo \ Desktop \ testttttt \ 00051.dga")
# deinterlace
crop (0, 132, 0, -132)
LanczosResize (1904.816) # Lanczos (Sharp)
# denoise
super = MSuper (pel = 2, sharp = 1)
backward_vec3 = MAnalyse (super, isb = true, delta = 3, overlap = 4)
backward_vec2 = MAnalyse (super, isb = true, delta = 2, overlap = 4)
backward_vec1 = MAnalyse (super, isb = true, delta = 1, overlap = 4)
forward_vec1 = MAnalyse (super, isb = false, delta = 1, overlap = 4)
forward_vec2 = MAnalyse (super, isb = false, delta = 2, overlap = 4)
forward_vec3 = MAnalyse (super, isb = false, delta = 3, overlap = 4)
MDegrain3 (super backward_vec1, forward_vec1, backward_vec2, forward_vec2, backward_vec3, forward_vec3, thSAD = 1200)
Max147 is offline   Reply With Quote
Old 14th May 2013, 02:09   #4  |  Link
johnmeyer
Registered User
 
Join Date: Feb 2002
Location: California
Posts: 2,064
I would suggest the following changes. You have used SetMTMode incorrectly (see correct usage in code sample). Also, I don't think you need to use MDegrain3 (and its three motion vectors). If performance is important, I'd also suggest using a lower overlap, although you should do some side-by-side comparisons with overlap=2 and overlap=4. In my experience, the improvement using overlap=4 is sometimes pretty subtle, although it depends strongly on your source material.

So, try the following script instead, and see if you get better performance. Then, look carefully at the results and start tweaking to get whatever tradeoff between performance and quality suits your taste.

Code:
threads=8
SetMTMode(5,threads)
AVCSource ("C: \ Users \ Geo \ Desktop \ testttttt \ 00051.dga")
SetMTMode(2)
# deinterlace
crop (0, 132, 0, -132)
# denoise
super = MSuper (pel = 2, sharp = 1)
backward_vec2 = MAnalyse (super, isb = true, delta = 2, overlap = 2)
backward_vec1 = MAnalyse (super, isb = true, delta = 1, overlap = 2)
forward_vec1 = MAnalyse (super, isb = false, delta = 1, overlap = 2)
forward_vec2 = MAnalyse (super, isb = false, delta = 2, overlap = 2)
MDegrain2 (super, backward_vec1, forward_vec1, backward_vec2, forward_vec2, thSAD = 1200)
Reduce the number of threads if you have stability problems, or if you don't have that number of cores. Also, you can increase the overlap back to 4. However, you should do a careful check to see if this improves quality. It probably will, but it may or may not be noticeable, depending on your source material.

With 1920x1080 60p source material, I got 11 fps with my 3-year-old i7, using this script. Without the SetMTMode statements, I got 2 fps. (Oh yes, if I use all of your original settings, and don't use SetMTMode, I get 0.9 fps, the same as you).

More benchmarks: If I use your original settings (overlap=4 and MDegrain3 instead of MDegrain2), I get 4 fps with the SetMTMode statement and threads=8. If I use overlap=4 and MDegrain2, I get 5.5 fps.

For even faster performance, you might search this forum for posts about the MRecalculate. To use this, you do a first MAnalyse pass with large block size (e.g., 16), and then do a second analysis using MRecalculate and a block size of 1/2 or 1/4 what you use with MAnalyze. See the MVTools2 documentation.

Finally, make sure you have set the thSAD correctly. This won't affect performance, but it very definitely can cause the denoising to fail if set wrong.
johnmeyer is offline   Reply With Quote
Old 14th May 2013, 17:49   #5  |  Link
Max147
Registered User
 
Join Date: May 2013
Posts: 12
Thank you for your reply,

I think I must have a problem when I test your script, I can not load it, I get the message : "Script error: there is no function named" SetMTMode "(c, \ users \ geo \ desktop \ testtt \ 00051 - copie.avs, line 2).

and the other thing I do not understand, you have not taken the line (load plugin .... mvtools2.dll and gdavcdecode.dll).

I'm a little lost.
Max147 is offline   Reply With Quote
Old 14th May 2013, 18:02   #6  |  Link
johnmeyer
Registered User
 
Join Date: Feb 2002
Location: California
Posts: 2,064
Quote:
Originally Posted by Max147 View Post
I think I must have a problem when I test your script, I can not load it, I get the message : "Script error: there is no function named" SetMTMode "(c, \ users \ geo \ desktop \ testtt \ 00051 - copie.avs, line 2).
I just copy/pasted from my post back into Notepad and ran the script. It runs OK. Therefore, you either have installed a version of AVISynth that doesn't contain the multi-threading code, or you have a simple syntax error in your code.

Quote:
Originally Posted by Max147 View Post
and the other thing I do not understand, you have not taken the line (load plugin .... mvtools2.dll and gdavcdecode.dll).
If you put your plugins in the default AVISynth plugin folder, you don't need to explicitly load them. I actually recommend that you DO explicitly load them, and I usually do this. However, since the path to my plugins don't match yours, I didn't want to introduce additional confusion by including those two lines in the code I posted. Go ahead an put them back in.
johnmeyer is offline   Reply With Quote
Old 14th May 2013, 20:05   #7  |  Link
Max147
Registered User
 
Join Date: May 2013
Posts: 12
Well, there is progress,

I actually have a 64-bit Windows, so you had to copy the file in syswow64 avisynth.dll and not in system 32.

I can now load your script with VirtualDub and encode but I can not load it with megui, j'' gets the same error as before.

I do not understand why it works with VirtualDub and not with megui.

I tested the preview with VirtualDub, we see immediately the difference is much faster.

But why I can not run it with megui?

Last edited by Max147; 14th May 2013 at 20:10.
Max147 is offline   Reply With Quote
Old 14th May 2013, 20:28   #8  |  Link
StainlessS
HeartlessS Usurer
 
StainlessS's Avatar
 
Join Date: Dec 2009
Location: Over the rainbow
Posts: 6,053
MeGUI has Avisynth.dll in the MeGUI/Tools/avs directory (version given as v2.5.8.5)
Perhaps you need to update with your version avisynth.dll.
__________________
I sometimes post sober.
StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace

"Some infinities are bigger than other infinities", but how many of them are infinitely bigger ???
StainlessS is offline   Reply With Quote
Old 14th May 2013, 20:29   #9  |  Link
Asmodian
Registered User
 
Join Date: Feb 2002
Location: San Jose, California
Posts: 3,270
I think megui might be using its own avisynth.dll? Look in its folders for one to replace.

Maybe: "C:\Program Files (x86)\MeGUI\tools\avisynth\avisynth.dll"
Asmodian is offline   Reply With Quote
Old 14th May 2013, 21:00   #10  |  Link
Max147
Registered User
 
Join Date: May 2013
Posts: 12
Great, thank you guys, it works now!

He had to replace the dll in the avs folder.

I can rest easy now.

Thank you again.
Max147 is offline   Reply With Quote
Old 19th August 2013, 09:22   #11  |  Link
Max147
Registered User
 
Join Date: May 2013
Posts: 12
Hello,

I again a little problem with my avisynth script, I already encoded carefree ten movie with this script but then after an hour encoding plant during the first pass.

Here is a copy of my script:

threads=12
SetMTMode(5,threads)
LoadPlugin("C:\Program Files (x86)\MeGUI_2341_x86\tools\ffms\ffms2.dll")
FFVideoSource("J:\_RIP\Die hard 5\T1_Video - .mkv", threads=1)
SetMTMode(2)
# deinterlace
crop(0, 20, 0, -20)
# denoise
super = MSuper(pel=2, sharp=1)
backward_vec3 = MAnalyse(super, isb = true, delta = 3, overlap=4)
backward_vec2 = MAnalyse(super, isb = true, delta = 2, overlap=4)
backward_vec1 = MAnalyse(super, isb = true, delta = 1, overlap=4)
forward_vec1 = MAnalyse(super, isb = false, delta = 1, overlap=4)
forward_vec2 = MAnalyse(super, isb = false, delta = 2, overlap=4)
forward_vec3 = MAnalyse(super, isb = false, delta = 3, overlap=4)
MDegrain3(super, backward_vec1,forward_vec1,backward_vec2,forward_vec2,backward_vec3,forward_vec3,thSAD=1200)

does anyone have an idea ?
Max147 is offline   Reply With Quote
Old 19th August 2013, 12:49   #12  |  Link
Guest
Guest
 
Join Date: Jan 2002
Posts: 21,923
Your word "plant" is not clear. What does it mean?

Also, where did you get the source MKV file?
Guest is offline   Reply With Quote
Old 19th August 2013, 14:26   #13  |  Link
Max147
Registered User
 
Join Date: May 2013
Posts: 12
In fact I'm trying to encode a bluray I bought, for that I used to use megui with two pass. So I extracted video track m2ts file into a mkv container and I use ffms2 to index. But the encoding stops after an hour and I have status "error" in the megui software for the first pass.

I tried to encode the easiest way and there it works without problem.

LoadPlugin("C:\Program Files (x86)\MeGUI_2341_x86\tools\ffms\ffms2.dll")
FFVideoSource("J:\_RIP\Die hard 5\T1_Video - .mkv", threads=1)
#deinterlace
crop(0, 20, 0, -20)
#resize
#denoise

Last edited by Max147; 19th August 2013 at 14:29.
Max147 is offline   Reply With Quote
Old 19th August 2013, 17:06   #14  |  Link
johnmeyer
Registered User
 
Join Date: Feb 2002
Location: California
Posts: 2,064
Two things to try when the script stops with an error message after running without problems for at least a few minutes.

1. Add a SetMemoryMax() statement to the beginning of the script. I usually start with SetMemoryMax(768) which restricts how much memory can be allocated for the multi-threading.

2. Reduce the number of threads. Try threads=4 or threads=6. This may make the script run more slowly, but it also may avoid the crashes. If the script works without crashing, you can try increasing "threads" to a number halfway between the number that works (no crashes) and your original value of 12.
johnmeyer is offline   Reply With Quote
Old 19th August 2013, 23:03   #15  |  Link
Max147
Registered User
 
Join Date: May 2013
Posts: 12
Thank you for answering!

I'll try that tonight
Max147 is offline   Reply With Quote
Old 20th August 2013, 11:36   #16  |  Link
chainik_svp
Registered User
 
Join Date: Mar 2012
Location: Saint-Petersburg
Posts: 221
Alternatively you can try Avisynth build from SVP. It has an option to cut off unneeded internal cache and sigificantly reduce memory consumption, so your script could be run in ~20 threads (for FHD source).
Code:
SetMemoryMax(1024)		#this value doesn't actually changes anything but it has to be defined
global svp_scheduler=true	#variable needed to turn on "special features" of SVP's build
global threads=7		#number of processing threads
global svp_cache_fwd=threads+2	#specifies the exact size of internal caches
SetMTMode(3,threads)		#turn on MT

<your code>
also it's a good idia to patch x264.exe from MeGUI distro with "4Gb patch"
__________________
SVPflow motion interpolation

Last edited by chainik_svp; 20th August 2013 at 11:40.
chainik_svp is offline   Reply With Quote
Old 20th August 2013, 22:35   #17  |  Link
Max147
Registered User
 
Join Date: May 2013
Posts: 12
First I tried the method of johnmeyer putting only 6 threads and the first pass was successful. I'll start the second pass that night to see if it is good.

But if it works I will also try your method chainik_svp, I'll keep you posted.

Thank you for your answers
Max147 is offline   Reply With Quote
Old 21st August 2013, 06:05   #18  |  Link
Seedmanc
Registered User
 
Join Date: Sep 2010
Location: Russia
Posts: 81
chainik_svp, are there any potential disadvantages of using SVP build of Avisynth over the usual MT one? Because if it has such a useful option to save memory I'd expect people to switch to it completely (at least I'd like to).

Speaking about motion analysis, I'm no expert, but I believe the script we have here has too strong settings, which results in slow encoding. I think motion compensation doesn't need extremely precise motion detection (as opposed to when it is used for reconstructing new frames in framerate upsampling), so I'd suggesting using these options:
Code:
super = MSuper(pel=1, sharp=1) # One-pixel precision might be enough
backward_vectors =  MAnalyse(super, isb = true, truemotion=false, blksize=32, overlap=16) # I think blocksize 32 should be enough for 1080p, default 8 is an overkill for FullHD.
forward_vectors =  MAnalyse(super, isb = false, truemotion=false, blksize=32, overlap=16)  # Overlap 16 is the maximum for blocksize of 32, but you might want to reduce it to 8.
# Truemotion=false might be enough for motion-compensated denoising, but you might remove this option so that default "true" will be used.
All this should also result in decreased memory usage, I believe.
Again, I'm not sure so it'd be nice if experts checked what I suggested here, or you can try these settings and see the quality vs speed difference.
Seedmanc is offline   Reply With Quote
Old 21st August 2013, 13:05   #19  |  Link
Selur
Registered User
 
Selur's Avatar
 
Join Date: Oct 2001
Location: Germany
Posts: 5,674
Quote:
chainik_svp, are there any potential disadvantages of using SVP build of Avisynth over the usual MT one?
looking over at http://www.svp-team.com/wiki/Download#libs the SVP edition of Avisynth is based on Avisynth 2.5.8 not Avisynth 2.6 so you will loose every functionality Avisynth 2.6 offers over 2.5.8.
-> @chainik_svp: any plans to create a 2.6 SVP edition?
__________________
Hybrid here in the forum, homepage
Selur is offline   Reply With Quote
Old 21st August 2013, 14:19   #20  |  Link
chainik_svp
Registered User
 
Join Date: Mar 2012
Location: Saint-Petersburg
Posts: 221
any plans to create a 2.6 SVP edition?

I don't want to do this until 2.6 release. It's "still in alfa stage" (c), right?
From my point of view 2.6 is better in just one case: (some of) its built-in resizing functions are significantly faster. May be I've missed someting?

are there any potential disadvantages of using SVP build of Avisynth over the usual MT one?

Well, it should lead to cache misses (which is equal to "reduced performance") in "non-linear" scripts. But it's great for linear processing like "classic" MSuper/MAnalyse/MFlowFps chain.
__________________
SVPflow motion interpolation
chainik_svp is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 03:55.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2018, vBulletin Solutions Inc.