Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Usage

Closed Thread
 
Thread Tools Search this Thread Display Modes
Old 5th August 2010, 03:47   #1  |  Link
-Vit-
Registered User
 
Join Date: Jul 2010
Posts: 448
QTGMC Deinterlacing Script (v3.32)

QTGMC is a very high quality deinterlacer for YV12 or YUY2 input, originally developed from TempGaussMC_beta2. It has a range of new features for the highest quality output and a convenient presets system to select processing speed. As well as interlaced input, it can also process progressive material for removal of interlacing or other shimmering artefacts. Extensive instructions on how to use all the settings are provided with the script.

Main Downloads
Thanks to SubJunk for filehost-free mirrors!

QTGMC v3.32 + English Instructions (mirror1 | mirror2)
Latest: v3.30:Speed-ups for many presets and a bug-fix in MVTools. v3.31: A minor bug fix. v3.32: A couple more minor tweaks and fixes. Used Windows line breaks.

Plugins Package (mirror1 | mirror2)
All the required plugin files for the script - follow the instructions inside. Individual plugin / source code links below.

Modded Plugins Package (mirror1 | mirror2)
Same as above, but these plugins contain threading fixes that can improve stability under MT.

There is also a new plugin pack using the updated Avisynth 2.6 interface available for testing.

Plugin Notes
For YUY2 input you must use mt_masktools-25.dll. Although not strictly 'correct' mt_masktools-25.dll can be used with SEt's Avisynth 2.6MT for mulithreaded usage.
If you get this error: "There is no function named RemoveGrain" then replace your RemoveGrainSSE2.dll with the version in this zip file
If you use EEDI3 you may need the Visual C++ 2010 Redistributable



Other Downloads

Spanish instructions (v3.32) (mirror)
Thanks Overdrive80!

Presets Detail (v3.32): Compared to "Faster" | Compared to "Medium" | Noise Presets
Thanks nhope!

Previous Version of QTGMC (v3.25)

________

Basic Usage

Install AviSynth and copy the plugins from the plugin package to the correct locations. Then to use QTGMC, write a script like this:
Code:
YourSource("yourfile")    # DGDecode_mpeg2source, FFVideoSource, AviSource, whatever your source requires
QTGMC( Preset="Slow" )
SelectEven()              # Add this line to keep original frame rate, leave it out for smoother doubled frame rate
Save this script with an ".avs" extension. You can now use it as an AVI source for encoding.

The "Preset" used selects sensible settings for a given encoding speed. Choose a preset from:
"Placebo", "Very Slow", "Slower", "Slow", "Medium", "Fast", "Faster", "Very Fast", "Super Fast", "Ultra Fast" & "Draft"
The default preset is "Slower", which is very close to the TempGaussMC_beta2 defaults

Don't be obsessed with running at the slowest possible settings. HD content especially doesn't need that precision and it will be very slow. More modest settings will give good speed and you may not even be able to tell the quality difference.

There are many settings to tweak the behavior of the script, full details are provided in the documentation that comes with the script. For example, to speed up processing and reduce sharpness a little:
Code:
QTGMC( Preset="Very Fast", Sharpness=0.75 )
________

Multi-Threaded Usage

QTGMC is very demanding on your system so for best performance try using it multi-threaded:
- Download SEt's latest 2.6MT version of avisynth. Do not use other MT versions as they have bugs.
- Copy the avisynth.dll from SEt's package over the one that is already in your SysWOW64 / System32 folder.
- You must also download either the modded plugin pack or the updated-interface plugin pack from the top of this post (*carefully* follow the instructions inside).

You need to tweak multi-threaded scripts to suit your system. Here is a basic template - read and follow the comments carefully and provide the values noted (you cannot use this code just as it is):
Code:
SetMemoryMax(M)  # Optional line. See below for value M
SetMTMode(3, X)  # See below for value X, could try 5 instead of 3 for non-standard source-filter/avisynth combinations
YourSource("yourfile")  # DGDecode_mpeg2source, FFVideoSource, AviSource, whatever your source requires
SetMTMode(2)
QTGMC( Preset="Slow", EdiThreads=Y ) # Choose preset based on overall speed/quality you want. See below for value Y
Distributor() # This line may or may not be necessary, try removing it and see if you get more speed
Setting X
- Start at the number of logical cores in your machine (note: with HyperThreading enabled, logical cores = 2x physical cores)
- If it crashes, decrease 1 at a time
- Otherwise increase 1 at a time until CPU usage just reaches 100%, don't go too far or it will slow down.
- On very slow settings you may not be able to reach 100% CPU before running out of memory and getting crashes. You can get round this by splitting your video and running several simultaneous encoding processes each with several threads, you're on your own with that though...

Setting Y
- Start with 1 for stability. In some cases increasing this value can give some speed up - try increasing it gradually. However, this setting has much less effect than X.

Setting M
- Running out of memory is the main problem with multithreaded avisynth so this line is important, especially for HD processing.
- For SD try leaving the line out first as it might just work.
- Then try using the SetMemoryMax line with values of M from 400-1200 or so. The default value is usually 512, so start around 700-900 and experiment from there.
- Don't just set the largest value because you're actually setting a memory balance (between avisynth cache and filter requirements)
- You can use *much* larger values (up to 2000 or more) if your encoding application is "Large Address Aware". So you are strongly advised to try the next step along with larger values of M
- Download this tool to check/set your encoding application as "Large Address Aware". You must point the tool at the exe file of your encoder, e.g. vdub.exe. Be careful since some encoding GUIs call out to other exe files for the actual encoding, for example MeGUI uses x264.exe or ffmpeg.exe from inside its "tools" folder. Some common examples: VirtualDub (vdub.exe) is not large address aware, "ffmpeg.exe" is often not large address aware, "x264.exe" is already large address aware.
- With a large address aware encoder, I often set M in the 1200-1600 range.
- Finally, you can reduce the memory needs of the encode by running 2 passes: in the first pass use QTGMC to encode to a temporary lossless file (I suggest huffyuv), then in the second pass encode your lossless file to your final format (e.g. h264). The temporary file will be huge, 100-200GB per hour for 1080p.

Notes
QTGMC is fine with SetMTMode(...), but the older MT(...) function will cause visual problems so don't use that.
Multi-threading crashes can be frustrating - they're usually due to memory problems (each thread uses more memory). However, most people can find a repeatedly stable memory balance for their system, and then the speed increases can be considerable. So tweak carefully starting with moderate settings.

Speed Guidance
On my i7 930@3.7Ghz I get from 24fps to 210+fps at 720x480 YV12 across the presets ("Placebo" to "Ultra Fast") using tweaked multi-threaded scripts as described below. It's about 7 times slower at 1920x1080. Obviously adding source-match, noise-bypass, motion-blur or other extras will make things slower. If you get poor speeds by comparison then check the instructions above, try tweaking the values. Make sure you're using correct plugins, try a different source filter, or a different encoding GUI. Don't use overkill settings. Upgrade your system.

________

Full Plugin List

This is a full list of all the plugins use by QTGMC. For convenience, you can get all of these in the Plugin Package at the top of this post. The source code for several of the plugins can be found in these links.

Core plugins
- MVTools2 (2.5.11.2 or above)
- MaskTools v2 (recommend 2.0a48 or above)
- NNEDI3 (recommend 0.9.4 or above for speed)
- RemoveGrain + Repair (extract only the SSE2 versions, some SSE3 versions have a bug and give no real benefit)
- SSE2Tools for YUY2 support (take only SSE2Tools.dll from this zip - do not take the SSE3 version nor the RemoveGrain or Repair dlls)

Additional plugins depending on settings
- [NNEDI2, NNEDI, EEDI3, EEDI2, TDeInt] - if selected directly or via a source-match preset
- Yadif - for Preset="Ultra Fast" or if selected directly (this plugin cannot be autoloaded, it must be loaded in the calling script)
- VerticalCleaner - for SVThin or lossless modes
- AddGrainC (or here) - for Preset="Placebo" or NoiseDeint="Generate" selected for noise processing
- FFT3DFilter - if selected for noise processing
- dfttest - if selected for noise processing
** For FFT3DFilter and ddftest you also need the FFTW3 library (main page / windows zips). On Windows the file needed for both is libfftw3f-3.dll. However for FFT3DFilter, it needs to be called FFTW3.dll, so you will need two copies and rename one. On Windows put the files in your System32 or SysWow64 folder (which depends on your OS and the version of AviSynth that you're running)

________

Notes

Comparison with TGMC
The default behaviour of QTGMC is almost identical to TempGaussMC_beta2 with particular settings. All the important defaults are effectively identical (including sharpness). However, some lesser settings have different defaults and some settings operate a little differently. There is a dedicated section on the comparison with TGMCb2 in the instructions.

Finally
Thanks go to Didée for the script from where this began, and to all the plugin scripters/coders involved too...

-Vit-

[The link at the top of the post will work if the attachment below is pending]
Attached Files
File Type: zip QTGMC-3.32.zip (37.0 KB, 3330 views)

Last edited by -Vit-; 7th April 2012 at 12:29. Reason: Mirrors, link to new plugin pack, large address aware, MT advice
-Vit- is offline  
Old 5th August 2010, 06:18   #2  |  Link
Tommy B.
in love with x264
 
Join Date: Jun 2004
Location: Germany
Posts: 89
Outstanding!

Thank you very much!
Tommy B. is offline  
Old 5th August 2010, 06:28   #3  |  Link
Revgen
Registered User
 
Join Date: Sep 2004
Location: Near LA, California, USA
Posts: 1,544
Thank You.
__________________
Pirate: Now how would you like to die? Would you like to have your head chopped off or be burned at the stake?

Curly: Burned at the stake!

Moe: Why?

Curly: A hot steak is always better than a cold chop.
Revgen is offline  
Old 6th August 2010, 05:26   #4  |  Link
bizz & buzz
Registered User
 
Join Date: Jun 2008
Posts: 95
Impressive, thank you very much for that
bizz & buzz is offline  
Old 6th August 2010, 05:31   #5  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 3,083
Thanks , nice first post !
poisondeathray is offline  
Old 6th August 2010, 07:23   #6  |  Link
Blue_MiSfit
Derek Prestegard IRL
 
Blue_MiSfit's Avatar
 
Join Date: Nov 2003
Location: Los Angeles
Posts: 5,301
Srsly, well done

I'd imagine this was done with some help from Didee?
Blue_MiSfit is offline  
Old 6th August 2010, 09:59   #7  |  Link
Didée
Registered User
 
Join Date: Apr 2002
Location: Germany
Posts: 5,390
No, I wasn't involved in any way. I'm as surprised as everyone else.

Some comments to come, but first I need to find time.
__________________
- We´re at the beginning of the end of mankind´s childhood -

My little flickr gallery. (Yes indeed, I do have hobbies other than digital video!)
Didée is offline  
Old 6th August 2010, 16:38   #8  |  Link
Terranigma
*Space Reserved*
 
Terranigma's Avatar
 
Join Date: May 2006
Posts: 953
Quote:
Originally Posted by -Vit- View Post
It provides all the same functionality

-Vit-
I haven't downloaded this script yet, but I was curious if anyone knew if this works with Non-MT avisynth versions.
Terranigma is offline  
Old 6th August 2010, 17:29   #9  |  Link
-Vit-
Registered User
 
Join Date: Jul 2010
Posts: 448
Quote:
Originally Posted by Terranigma View Post
I haven't downloaded this script yet, but I was curious if anyone knew if this works with Non-MT avisynth versions.
Yes, I've used it non-MT with 2.57 and 2.58 straight from the avisynth wiki page.
-Vit- is offline  
Old 6th August 2010, 17:37   #10  |  Link
Terranigma
*Space Reserved*
 
Terranigma's Avatar
 
Join Date: May 2006
Posts: 953
Quote:
Originally Posted by -Vit- View Post
Yes, I've used it non-MT with 2.57 and 2.58 straight from the avisynth wiki page.
OK ok, that clears that up. I'll check it out in a bit.
Terranigma is offline  
Old 8th August 2010, 04:46   #11  |  Link
Undead Sega
Registered User
 
Join Date: Oct 2007
Posts: 707
I'm interested to what you said about using TGMC as a stabilizer???
Undead Sega is offline  
Old 8th August 2010, 11:38   #12  |  Link
-Vit-
Registered User
 
Join Date: Jul 2010
Posts: 448
Quote:
Originally Posted by Undead Sega View Post
I'm interested to what you said about using TGMC as a stabilizer???
I don't mean as a deshaker of course. I use the word stable throughout the script comments to mean an output without shimmer or flicker. I've updated the OP for clarity...
-Vit- is offline  
Old 8th August 2010, 11:17   #13  |  Link
nibus
Telewhining
 
Join Date: Mar 2010
Posts: 275
so awesome, thank you!
nibus is offline  
Old 9th August 2010, 13:36   #14  |  Link
GDAce
Registered User
 
Join Date: Aug 2009
Posts: 13


i get this error when use this QTGMC( Preset="Very Fast", Sharpness=0.75 )...i have all filters in place
GDAce is offline  
Old 9th August 2010, 14:29   #15  |  Link
Didée
Registered User
 
Join Date: Apr 2002
Location: Germany
Posts: 5,390
You're probably using a too outdated version of MVTools2. Update to a recent version.

(The changelog isn't quite clear about the introduction/functionality of "big" blocksizes. But at least v2.5.8 - almost one year old - already did support 32x32. Most recent is v2.5.10.1.)
__________________
- We´re at the beginning of the end of mankind´s childhood -

My little flickr gallery. (Yes indeed, I do have hobbies other than digital video!)
Didée is offline  
Old 9th August 2010, 15:20   #16  |  Link
-Vit-
Registered User
 
Join Date: Jul 2010
Posts: 448
Quote:
Originally Posted by GDAce View Post
i get this error when use this QTGMC( Preset="Very Fast", Sharpness=0.75 )...i have all filters in place
Yes, you need a more recent MVTools2. You also need up to date NNEDI3 and EEDI3 (if you use them) as their parameters changed recently. Best to get the latest versions of all plugins because although it's had a few versions, the script is only six weeks old. I've clarified the requirements in the OP...

The larger blocksizes have been used since the beginning as a speed/quality tradeoff.

Last edited by -Vit-; 9th August 2010 at 16:02.
-Vit- is offline  
Old 10th August 2010, 06:31   #17  |  Link
GDAce
Registered User
 
Join Date: Aug 2009
Posts: 13
i had mvtools2 latest version in plugins folder...but i had to rename mvtools2.dll to mvtools.dll. it wud be nice if this can use mvtools2.dll as it is(a noob opinion).
GDAce is offline  
Old 10th August 2010, 11:31   #18  |  Link
-Vit-
Registered User
 
Join Date: Jul 2010
Posts: 448
Quote:
Originally Posted by GDAce View Post
i had mvtools2 latest version in plugins folder...but i had to rename mvtools2.dll to mvtools.dll. it wud be nice if this can use mvtools2.dll as it is(a noob opinion).
The filename of the DLLs used is not important to any script. I can rename mvtools2.dll to xxyyzz.dll and everything will still work fine because AviSynth looks "inside" the DLLs to find their content.

Problems occur when your DLLs are not in the correct plugins folder or, more subtly, when you have multiple DLLs with the same content. E.g. If you had an old MVTools2 called mvtools.dll and a new one called mvtools2.dll - that could cause problems when AviSynth comes to load the plugin, but you might not notice until a script used a new MVTools2 feature.

So I imagine there was something else you changed that made things work. Perhaps you removed a previous mvtools.dll to do the rename? Or moved some DLLs between folders? In any case, this script does nothing unusual in its use of MVTools2.
-Vit- is offline  
Old 13th August 2010, 18:23   #19  |  Link
txporter
Registered User
 
Join Date: Nov 2009
Posts: 110
Haven't had a chance to try out your script yet, but have browsed through it looking at your comments. I see a note about updating the yadif path in order to use some of the faster presets. Is it possible to define a path in the script but also call add a standard loadcplugin call within an avs script? The reason that I ask is that I do some encoding with 32-bit avisynth and some with 64-bit avisynth, so I need to make calls to both plugin directories (plugins and plugins64). Or maybe define pathway for both 32-bit and 64-bit avisynth?
txporter is offline  
Old 13th August 2010, 22:02   #20  |  Link
Didée
Registered User
 
Join Date: Apr 2002
Location: Germany
Posts: 5,390
Yeah, hard-loading a plugin in the script is not nice. For what reason? Because it's a LoadCPlugin/stdcall_plugin? Cmon. Yadif is used a lot, people supposely know how to load it. Kick that out, put Yadif in the list of "required plugins", and good. Buisness as usual.
__________________
- We´re at the beginning of the end of mankind´s childhood -

My little flickr gallery. (Yes indeed, I do have hobbies other than digital video!)
Didée is offline  
Closed Thread

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 22:02.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2017, vBulletin Solutions Inc.