QTGMC Deinterlacing Script (v3.32)

[-Vit-]

QTGMC is a very high quality deinterlacer for YV12 or YUY2 input, originally developed from TempGaussMC_beta2. It has a range of new features for the highest quality output and a convenient presets system to select processing speed. As well as interlaced input, it can also process progressive material for removal of interlacing or other shimmering artefacts. Extensive instructions on how to use all the settings are provided with the script.

Main Downloads
Thanks to SubJunk for filehost-free mirrors!

QTGMC v3.32 + English Instructions (http://www.mediafire.com/?su7l5jtcobabksk) (mirror1 (http://hotfile.com/dl/152000213/d36acd9/QTGMC-3.32.zip.html) | mirror2 (http://www.spirton.com/uploads/QTGMC/QTGMC-3.32.zip))
Latest: v3.30:Speed-ups (http://forum.doom9.org/showthread.php?p=1485933#post1485933) for many presets and a bug-fix in MVTools. v3.31: A minor bug fix. v3.32: A couple more minor tweaks and fixes (http://forum.doom9.org/showthread.php?p=1498428#post1498428). Used Windows line breaks.

Plugins Package (http://hotfile.com/dl/151999753/8fc2003/QTGMC_32-bit_Plugins.zip.html) (mirror1 (http://www.mediafire.com/?nm188lh2oam487m) | mirror2 (http://www.spirton.com/uploads/QTGMC/QTGMC%2032-bit%20Plugins.zip))
All the required plugin files for the script - follow the instructions inside. Individual plugin / source code links below.

Modded Plugins Package (http://hotfile.com/dl/151999944/9f5eee1/QTGMC_32-bit_Plugins_Vit-Mod.zip.html) (mirror1 (http://www.mediafire.com/?mfs7bp2rprbhp22) | mirror2 (http://www.spirton.com/uploads/QTGMC/QTGMC%2032-bit%20Plugins%20%5BVit-Mod%5D.zip))
Same as above, but these plugins contain threading fixes (http://forum.doom9.org/showpost.php?p=1503034&postcount=691) that can improve stability under MT.

There is also a new plugin pack using the updated Avisynth 2.6 interface (http://forum.doom9.org/showpost.php?p=1568142&postcount=1126) available for testing.

Plugin Notes
For YUY2 input you must use mt_masktools-25.dll. Although not strictly 'correct' mt_masktools-25.dll can be used with SEt's Avisynth 2.6MT for mulithreaded usage.
If you get this error: "There is no function named RemoveGrain" then replace your RemoveGrainSSE2.dll with the version in this zip file (http://home.arcor.de/kassandro/prerelease/RemoveGrain-1.0.rar)
If you use EEDI3 you may need the Visual C++ 2010 Redistributable (http://www.microsoft.com/download/en/details.aspx?id=5555)


Other Downloads

Spanish instructions (v3.32) (http://www.mediafire.com/?1h1r5tf2t9t8rq1) (mirror (http://www.spirton.com/uploads/QTGMC/QTGMC-3.32-Spanish.rar))
Thanks Overdrive80!

Presets Detail (v3.32): Compared to "Faster" (https://spreadsheets.google.com/spreadsheet/ccc?key=0AsftVT5A1FqQdHRKTUdlUXVUaDlxUE43c0NsQ3dFR3c) | Compared to "Medium" (https://spreadsheets.google.com/spreadsheet/ccc?key=0AsftVT5A1FqQdGJ5Rlo3TlJqXzR1Vm1xRV9XMkhNWXc) | Noise Presets (https://spreadsheets.google.com/spreadsheet/ccc?key=0AsftVT5A1FqQdHRnNWhESU92VExfcUpUTnNvUlRXc2c)
Thanks nhope!

Previous Version of QTGMC (v3.25) (http://www.mediafire.com/download.php?1td5o1y2wh4jbch)

________

Basic Usage

Install AviSynth and copy the plugins from the plugin package to the correct locations. Then to use QTGMC, write a script like this:

YourSource("yourfile") # DGDecode_mpeg2source, FFVideoSource, AviSource, whatever your source requires
QTGMC( Preset="Slow" )
SelectEven() # Add this line to keep original frame rate, leave it out for smoother doubled frame rate

Save this script with an ".avs" extension. You can now use it as an AVI source for encoding.

The "Preset" used selects sensible settings for a given encoding speed. Choose a preset from:
"Placebo", "Very Slow", "Slower", "Slow", "Medium", "Fast", "Faster", "Very Fast", "Super Fast", "Ultra Fast" & "Draft"
The default preset is "Slower", which is very close to the TempGaussMC_beta2 defaults

Don't be obsessed with running at the slowest possible settings. HD content especially doesn't need that precision and it will be very slow. More modest settings will give good speed and you may not even be able to tell the quality difference.

There are many settings to tweak the behavior of the script, full details are provided in the documentation that comes with the script. For example, to speed up processing and reduce sharpness a little:
QTGMC( Preset="Very Fast", Sharpness=0.75 )

________

Multi-Threaded Usage

QTGMC is very demanding on your system so for best performance try using it multi-threaded:
- Download SEt's latest 2.6MT (http://forum.doom9.org/showthread.php?t=148782) version of avisynth. Do not use other MT versions as they have bugs.
- Copy the avisynth.dll from SEt's package over the one that is already in your SysWOW64 / System32 folder.
- You must also download either the modded plugin pack or the updated-interface plugin pack from the top of this post (*carefully* follow the instructions inside).

You need to tweak multi-threaded scripts to suit your system. Here is a basic template - read and follow the comments carefully and provide the values noted (you cannot use this code just as it is):

SetMemoryMax(M) # Optional line. See below for value M
SetMTMode(3, X) # See below for value X, could try 5 instead of 3 for non-standard source-filter/avisynth combinations
YourSource("yourfile") # DGDecode_mpeg2source, FFVideoSource, AviSource, whatever your source requires
SetMTMode(2)
QTGMC( Preset="Slow", EdiThreads=Y ) # Choose preset based on overall speed/quality you want. See below for value Y
Distributor() # This line may or may not be necessary, try removing it and see if you get more speed

Setting X
- Start at the number of logical cores in your machine (note: with HyperThreading enabled, logical cores = 2x physical cores)
- If it crashes, decrease 1 at a time
- Otherwise increase 1 at a time until CPU usage just reaches 100%, don't go too far or it will slow down.
- On very slow settings you may not be able to reach 100% CPU before running out of memory and getting crashes. You can get round this by splitting your video and running several simultaneous encoding processes each with several threads, you're on your own with that though...

Setting Y
- Start with 1 for stability. In some cases increasing this value can give some speed up - try increasing it gradually. However, this setting has much less effect than X.

Setting M
- Running out of memory is the main problem with multithreaded avisynth so this line is important, especially for HD processing.
- For SD try leaving the line out first as it might just work.
- Then try using the SetMemoryMax line with values of M from 400-1200 or so. The default value is usually 512, so start around 700-900 and experiment from there.
- Don't just set the largest value because you're actually setting a memory balance (between avisynth cache and filter requirements)
- You can use *much* larger values (up to 2000 or more) if your encoding application is "Large Address Aware". So you are strongly advised to try the next step along with larger values of M
- Download this tool (http://www.techpowerup.com/forums/showthread.php?t=112556) to check/set your encoding application as "Large Address Aware". You must point the tool at the exe file of your encoder, e.g. vdub.exe. Be careful since some encoding GUIs call out to other exe files for the actual encoding, for example MeGUI uses x264.exe or ffmpeg.exe from inside its "tools" folder. Some common examples: VirtualDub (vdub.exe) is not large address aware, "ffmpeg.exe" is often not large address aware, "x264.exe" is already large address aware.
- With a large address aware encoder, I often set M in the 1200-1600 range.
- Finally, you can reduce the memory needs of the encode by running 2 passes: in the first pass use QTGMC to encode to a temporary lossless file (I suggest huffyuv), then in the second pass encode your lossless file to your final format (e.g. h264). The temporary file will be huge, 100-200GB per hour for 1080p.

Notes
QTGMC is fine with SetMTMode(...), but the older MT(...) function will cause visual problems so don't use that.
Multi-threading crashes can be frustrating - they're usually due to memory problems (each thread uses more memory). However, most people can find a repeatedly stable memory balance for their system, and then the speed increases can be considerable. So tweak carefully starting with moderate settings.

Speed Guidance
On my i7 930@3.7Ghz I get from 24fps to 210+fps at 720x480 YV12 across the presets ("Placebo" to "Ultra Fast") using tweaked multi-threaded scripts as described below. It's about 7 times slower at 1920x1080. Obviously adding source-match, noise-bypass, motion-blur or other extras will make things slower. If you get poor speeds by comparison then check the instructions above, try tweaking the values. Make sure you're using correct plugins, try a different source filter, or a different encoding GUI. Don't use overkill settings. Upgrade your system.

________

Full Plugin List

This is a full list of all the plugins use by QTGMC. For convenience, you can get all of these in the Plugin Package at the top of this post. The source code for several of the plugins can be found in these links.

Core plugins
- MVTools2 (http://avisynth.org.ru/mvtools/mvtools2.html#download) (2.5.11.2 or above)
- MaskTools v2 (http://forum.doom9.org/showthread.php?p=701539#post701539) (recommend 2.0a48 or above)
- NNEDI3 (http://web.missouri.edu/~kes25c/) (recommend 0.9.4 or above for speed)
- RemoveGrain + Repair (http://home.arcor.de/kassandro/RemoveGrain/RemoveGrain.rar) (extract only the SSE2 versions, some SSE3 versions have a bug and give no real benefit)
- SSE2Tools (http://home.pages.at/kassandro/RemoveGrain/RemoveGrain.zip) for YUY2 support (take only SSE2Tools.dll from this zip - do not take the SSE3 version nor the RemoveGrain or Repair dlls)

Additional plugins depending on settings
- [NNEDI2, NNEDI, EEDI3, EEDI2, TDeInt] (http://web.missouri.edu/~kes25c/) - if selected directly or via a source-match preset
- Yadif (http://avisynth.org.ru/yadif/yadif.html) - for Preset="Ultra Fast" or if selected directly (this plugin cannot be autoloaded, it must be loaded in the calling script)
- VerticalCleaner (http://home.arcor.de/kassandro/prerelease/VerticalCleaner.rar) - for SVThin or lossless modes
- AddGrainC (http://ldesoras.free.fr/src/AddGrainC-1.5.1.7z) (or here (http://www.megaupload.com/?d=9ZWS9HDF)) - for Preset="Placebo" or NoiseDeint="Generate" selected for noise processing
- FFT3DFilter (http://avisynth.org.ru/fft3dfilter/fft3dfilter.html) - if selected for noise processing
- dfttest (http://web.missouri.edu/~kes25c) - if selected for noise processing
** For FFT3DFilter and ddftest you also need the FFTW3 library (main page (http://www.fftw.org/) / windows zips (http://www.fftw.org/install/windows.html)). On Windows the file needed for both is libfftw3f-3.dll. However for FFT3DFilter, it needs to be called FFTW3.dll, so you will need two copies and rename one. On Windows put the files in your System32 or SysWow64 folder (which depends on your OS and the version of AviSynth that you're running)

________

Notes

Comparison with TGMC
The default behaviour of QTGMC is almost identical to TempGaussMC_beta2 with particular settings. All the important defaults are effectively identical (including sharpness). However, some lesser settings have different defaults and some settings operate a little differently. There is a dedicated section on the comparison with TGMCb2 in the instructions.

Finally
Thanks go to Didée for the script from where this began, and to all the plugin scripters/coders involved too...

-Vit-

[The link at the top of the post will work if the attachment below is pending]

[Tommy B.]

Outstanding!

Thank you very much!

[Revgen]

Thank You.

[bizz & buzz]

Impressive, thank you very much for that :)

[poisondeathray]

Thanks , nice first post ! :D

[Blue_MiSfit]

Srsly, well done :)

I'd imagine this was done with some help from Didee?

[Didée]

No, I wasn't involved in any way. I'm as surprised as everyone else. :)

Some comments to come, but first I need to find time.

[Terranigma]

It provides all the same functionality

-Vit-

I haven't downloaded this script yet, but I was curious if anyone knew if this works with Non-MT avisynth versions.

[-Vit-]

I haven't downloaded this script yet, but I was curious if anyone knew if this works with Non-MT avisynth versions.
Yes, I've used it non-MT with 2.57 and 2.58 straight from the avisynth wiki page.

[Terranigma]

Yes, I've used it non-MT with 2.57 and 2.58 straight from the avisynth wiki page.
OK ok, that clears that up. I'll check it out in a bit. :)

[Undead Sega]

I'm interested to what you said about using TGMC as a stabilizer???

[nibus]

so awesome, thank you!

[-Vit-]

I'm interested to what you said about using TGMC as a stabilizer???
I don't mean as a deshaker of course. I use the word stable throughout the script comments to mean an output without shimmer or flicker. I've updated the OP for clarity...

[GDAce]

http://lookpic.com/d2/i2/2971/oeasbfFj.png

i get this error when use this QTGMC( Preset="Very Fast", Sharpness=0.75 )...i have all filters in place

[Didée]

You're probably using a too outdated version of MVTools2. Update to a recent version (http://avisynth.org.ru/mvtools/mvtools2.html#download).

(The changelog isn't quite clear about the introduction/functionality of "big" blocksizes. But at least v2.5.8 - almost one year old - already did support 32x32. Most recent is v2.5.10.1.)

[-Vit-]

i get this error when use this QTGMC( Preset="Very Fast", Sharpness=0.75 )...i have all filters in place

Yes, you need a more recent MVTools2 ("http://avisynth.org.ru/mvtools/mvtools2.html#download). You also need up to date NNEDI3 and EEDI3 (if you use them) as their parameters changed recently. Best to get the latest versions of all plugins because although it's had a few versions, the script is only six weeks old. I've clarified the requirements in the OP...

The larger blocksizes have been used since the beginning as a speed/quality tradeoff.

[GDAce]

i had mvtools2 latest version in plugins folder...but i had to rename mvtools2.dll to mvtools.dll. it wud be nice if this can use mvtools2.dll as it is(a noob opinion).

[-Vit-]

i had mvtools2 latest version in plugins folder...but i had to rename mvtools2.dll to mvtools.dll. it wud be nice if this can use mvtools2.dll as it is(a noob opinion).
The filename of the DLLs used is not important to any script. I can rename mvtools2.dll to xxyyzz.dll and everything will still work fine because AviSynth looks "inside" the DLLs to find their content.

Problems occur when your DLLs are not in the correct plugins folder or, more subtly, when you have multiple DLLs with the same content. E.g. If you had an old MVTools2 called mvtools.dll and a new one called mvtools2.dll - that could cause problems when AviSynth comes to load the plugin, but you might not notice until a script used a new MVTools2 feature.

So I imagine there was something else you changed that made things work. Perhaps you removed a previous mvtools.dll to do the rename? Or moved some DLLs between folders? In any case, this script does nothing unusual in its use of MVTools2.

[txporter]

Haven't had a chance to try out your script yet, but have browsed through it looking at your comments. I see a note about updating the yadif path in order to use some of the faster presets. Is it possible to define a path in the script but also call add a standard loadcplugin call within an avs script? The reason that I ask is that I do some encoding with 32-bit avisynth and some with 64-bit avisynth, so I need to make calls to both plugin directories (plugins and plugins64). Or maybe define pathway for both 32-bit and 64-bit avisynth?

[Didée]

Yeah, hard-loading a plugin in the script is not nice. For what reason? Because it's a LoadCPlugin/stdcall_plugin? Cmon. Yadif is used a lot, people supposely know how to load it. Kick that out, put Yadif in the list of "required plugins", and good. Buisness as usual.

[-Vit-]

Yeah, hard-loading a plugin in the script is not nice.
Fair point. I wanted to avoid requirements on the calling script, but I see that leads to other problems. Happy to get rid of that bit of ugly really.
I've put an updated version in the OP that doesn't explicitly load Yadif.

[-Vit-]

Found a reproducible MT instability when using Super Fast and Ultra Fast presets, with mt_average throwing an exception. Couldn't say if the problem originated in mt_average or not because the particular setup was complex and unusual. However, replacing mt_average with merge remedied the situation. A quick test seems to show merge to be faster in any case. Updated version in the OP.

[Bi11]

Great work! :goodpost:

Just a few suggestions:


Supply sclip to eedi3 for more accurate interpolation.
For example,
(EdiMode == "EEDI3") ? ediInput.EEDI3(sclip=ediInput.NNEDI3(field=-2, nsize=NNSize, nns=NNeurons, qual=EdiQual), field=-2, mdis=EdiMaxD) : \

Super-sampled eedi3 after QTGMC removes most residual aliasing.
For example,
QTGMC(EdiMode="eedi3", Blocksize=8, Overlap=4, Search=5, SearchParam=4).SelectEven()

w=width h=height
lanczosresize(w*2,h*2,taps=8).eedi3().lanczosresize(w,h,taps=8)

RestoreLossless is called with only some parameters in the script. Why not pass all parameters it expects?


IMHO the ShowSettings output could be more spaced apart, like
addNoise.Subtitle( "tr0=" + string(tr0) + " | tr1=" + string(tr1) + " | tr2=" + string(tr2) + " | rep0=" + string(rep0) + " | rep1=" + string(rep1) + \
" | rep2=" + string(rep2) + "\nEdiMode='" + EdiMode + "' | NNSize=" + string(NNSize) + " | NNeurons=" + string(NNeurons) + " | EdiQual=" + \
string(EdiQual) + " | EdiMaxD=" + string(EdiMaxD) + "\nSharpness=" + string(Sharpness, "%.2f") + " | SMode=" + string(SMode) + " | SLMode=" + \
string(SLMode) + " | SLRad=" + string(SLRad) + " | SOvs=" + string(SOvs) + " | SVThin=" + string(SVThin, "%.2f") + " | Sbb=" + string(Sbb) + \
"\nSrchClipPP=" + string(SrchClipPP) + " | SubPel=" + string(SubPel) + " | SubPelInterp=" + string(SubPelInterp) + " | BlockSize=" + string(BlockSize) + \
" | Overlap=" + string(Overlap) + "\nSearch=" + string(Search) + " | SearchParam=" + string(SearchParam) + " | PelSearch=" + string(PelSearch) + \
" | TrueMotion=" + string(TrueMotion) + "\nLambda=" + string(Lambda) + " | LSAD=" + string(LSAD) + " | PNew=" + string(PNew) + " | PLevel=" + string(PLevel) + \
" | GlobalMotion=" + string(GlobalMotion) + " | DCT=" + string(DCT) + "\nLossless=" + string(Lossless) + " | LosslessTR=" + string(LosslessTR) + \
" | LosslessEdi='" + LosslessEdi + "'\nNoiseBypass=" + string(NoiseBypass) + " | NoiseRemove=" + string(NoiseRemove, "%.2f") + " | NoiseRestore=" + \
string(NoiseRestore, "%.2f") + "\nNoiseDeint='" + NoiseDeint + "' | Sigma=" + string(Sigma, "%.2f") + " | BT=" + string(BT) + "\nBorder=" + \
string(Border) + " | Precise=" + string(Precise) + " | SafeMode=" + string(SafeMode) + "\nPreset='" + Preset + "' | Tuning='" + Tuning + "'", \
font="Lucida Console", size=11, lsp=12 )

[-Vit-]

Great work!
Just a few suggestions:

Thank you, and thanks for the feedback.


Supply sclip to eedi3 for more accurate interpolation

The EdiExt parameter allows you to use any interpolator:
clip.QTGMC( Preset="Slow", EdiExt=clip.EEDI3(sclip=clip.NNEDI3(field=-2), field=-2) )
I work with naturalistic sources, so I almost exclusively use NNEDI2/3. So when it came to adding EEDI3, I just followed the style that TempGaussMC had used for EEDI2. I didn't think to use the new sclip parameter. The EdiMode strings are just shortcuts for the most common use cases. Bearing that in mind, do you think I should update EdiMode="EEDI3" to use "sclip" or add a new EdiMode="EEDI3x"...

____

Super-sampled eedi3 after QTGMC removes most residual aliasing

I assume this is an observation rather than a suggestion?

____

RestoreLossless is called with only some parameters in the script. Why not pass all parameters it expects?

The lossless code is a second simpler deinterlacing pass on the difference between the initial TGMC output and the source. As such it has some similar parameters to the main deinterlacing script. However, I'm not necessarily convinced that the same parameter values would be the most effective. I'm not unconvinced either, I just haven't had the time to test in detail.

The difference image is quite different in nature to the original source. It's largely gray (the origin, i.e. no difference, is at 128), with the edges made prominent (due to changes from the blur->resharpen tgmc process) and areas of noise (that removed by tgmc). One observation is that interpolator choice is slightly different here. NNEDI2 works OK, but even for naturalistic sources I found EEDI3 to work well, probably because of the prominence of edges in this difference image. Then again changes made to the lossless deinterlacing parameters are much less noticable than in the main script as it is working with small values.

I added the parameters to the lossless function out of good practice / coding habit. However, I don't think I want to add so many more parameters to the script itself as I'm reaching the limit. So if necessary I may add a boolean "LosslessDefaults" or similar, which determines whether to use the main deinterlacing parameters for the lossless step too, or whether to use the default lossless settings instead (as it does now). One reason to favor the defaults would be to speed up lossless scripts using high quality settings, which would be overkill if passed to the lossless step. Memory use is currently high when using lossless (especially MT), another reason to use simpler settings on that step.

____

IMHO the ShowSettings output could be more spaced apart, like...
I like it, thanks. I will update to something like this in the next revision.

[Bi11]

The EdiExt parameter allows you to use any interpolator
Yes, I've noticed.

Because my live sources are usually crappy quality, I usually use QTGMC like so:
edi=eedi3(sclip=nnedi3(field=-2,nsize=3,qual=2),field=-2,vcheck=3).raa()
QTGMC(ediext=edi,blocksize=8,overlap=4,search=5,searchparam=4).selecteven()

# Residual Anti-Aliasing
function RAA (clip input) {
w=input.width()
h=input.height()

input.spline64resize(w*2,h*2).eedi3(vcheck=3).spline64resize(w,h)
}
It's fairly aggressive at connecting straight lines, giving smooth lines and curves with little or no flickering.

I haven't tried eedi3/nnedi2, but from my observations eedi3/nnedi3 always gives more accurate interpolation and less artifacts than eedi3 alone when used for deinterlacing. So IMO sclip=nnedi3 should be the default.
I use vcheck=3 instead of the default vcheck=2 because it gives a bit less artifacts imo.

I use blocksize=8 and overlap=4 because from my observations it gives noticeably better quality than the defaults 16/8.
search=5 and searchparam=4 gives slightly more accurate motion vector search.

raa() is the super-sampled eedi3, which I use just after deinterlacing to smoothen any remaining jagged edges. raa() could be an optional feature, which could be included in some of the slower presets.

With regard to the lossless parameters, I understand now that higher quality settings may not be worthwhile. If anything, you could have a LosslessPreset option, which takes one of the preset values already defined, and uses the parameters it requires from that preset. But is there really a big difference in overall speed if lossless settings are the same as the preset settings, considering it's a difference clip that's being interpolated?

On a related note, I would like for a possible way to keep the EdiMode and LosslessEdi values in sync. Currently, if I specify LosslessEdi="nnedi3" then the script still tries to use nnedi2, which is not in my plugin directory.
One possible solution is to have LosslessEdi default to EdiMode, and a different LosslessEdi value can be explicitly specified if needed.

[yup]

-Vit-!
Which version I could use wit SET 2.6 Avisynth mt_masktools-25.dll or mt_masktools-26.dll.
I get constantly crash.
yup.

[henryho_hk]

Can you add an int parameter "NNEDIthreads" to control the number of threads of nnedi2/3?

[yup]

I remove
Lossless=1
and now work O'k both version masktools 2.5 and 2.6 with Set Avisynth 2.6 MT build.
yup.

[Blue_MiSfit]

This script is insanely awesome. I'm so thoroughly impressed with the quality / speed ratio. Wow.

[ajp_anton]

In the defaults, I think Overlap is supposed to be Blocksize/2 and not bs/s.

[-Vit-]

In the defaults, I think Overlap is supposed to be Blocksize/2 and not bs/s.
Well spotted, thanks. If you provided an explicit blocksize but left the default overlap then it would mistakenly base the overlap on the default blocksize. I've upped a fixed version in the first post.
____

I've been a bit busy of late but I have noted the suggestions that people have been making. I will be upping a new version relatively soon - just need to find the time to test. I mainly want to update the use of EEDI3 / NNEDI3 (particularly now NNEDI3 is faster).

[Hagbard23]

This script is insanely awesome. I'm so thoroughly impressed with the quality / speed ratio. Wow.

so true...

I've tried it with my old futurama PAL Discs. This Plugin is so f***ing good - it blasts me off the chair. OK Speed is no issue here... ;). I used it successfully with needi3 and preset "slower" and the result was perfect "placebo" pure progressive output.

Didee and ViT: Please keep up the good work - we strongly needed those two (tgmc / qtgmc) plugins.

[Hagbard23]

Nah...after tweaking for hours i found a "bug" now...:(:

I can't run any filters (i've tried warpsharp & fft3dgpu) after QTGMC? ViT do you know anything about it? :confused:

[aegisofrime]

Nah...after tweaking for hours i found a "bug" now...:(:

I can't run any filters (i've tried warpsharp & fft3dgpu) after QTGMC? ViT do you know anything about it? :confused:

FFT3DGPU works fine here. You have to SetMTMode(5), but that's not specific to QTGMC since you have to do that in any multithreaded script.

[Didée]

What's the actual problem? Does the additional filter simply have no effect, or does the script crash? If so, what's the error message?

:script:

[Hagbard23]

dgsource("E:\blah\VTS_01_1c.dgi")
assumetff()
QTGMC(EdiMode="NNEDI3")
awarpsharp2()
FFT3dGPU()
crop(12,2,-10,-0)
spline36resize(720,544)

That was the script..

(meanwhile i returned to TGMCBeta2, (w. NNEDI2). This worked just fine.)

With that script Virtual Dub crashed with some AccessViolation Error - sometimes i got "AviSynth Failure - Unknown Exception" when i seeked around in the video...

@aegisofrime
SetMTMode(5)
I worked without multithreading... ;)

[-Vit-]

I can't run any filters (i've tried warpsharp & fft3dgpu) after QTGMC? ViT do you know anything about it? :confused:
I've used many different filters both before and after QTGMC without problem. Both single and multi-threaded. Unfortunately, the error message isn't very specific. Do you get the same effect if you use "NNEDI2" in QTGMC? Do you get errors in anything other than VirtualDub?

Only thing that I can think of is memory use. The script shouldn't use more memory than tgmc-b2 when using the "Slower" preset, but I don't know about NNEDI3's memory usage - it's a long shot but maybe that just tipped things over the edge. Try adding a SetMemoryMax at the start of the script...

[Shon]

-Vit-
What should I do to work only deinterlace, but the noise was not removed?

[-Vit-]

-Vit-
What should I do to work only deinterlace, but the noise was not removed?
I'm not sure I understand. Are you asking how to keep the noise from the source? Or how to remove it? Here are both solutions...

Keep noise:

QTGMC( Preset="Slower", NoiseBypass=1, NoiseRestore=1.0 ) # Mode 1: Remove noise before, add back after. Medium effect

#or

QTGMC( Preset="Slower", NoiseBypass=2, NoiseRestore=0.4 ) # Mode 2: Get noise at start but don't remove it, add back after
# Stronger effect. Increase NoiseRestore for an even stronger
# effect (>1 for a very strong effect)

#or

QTGMC( Preset="Slower", Lossless=1, LosslessTR=3 ) # Lossless modes also retain noise. Combine with noise bypass if you want


Remove noise:

QTGMC( Preset="Slower", NoiseBypass=1, NoiseRestore=0.0 )

#or

QTGMC( Preset="Slower", Sigma=2.5, NoiseBypass=1, NoiseRestore=0.0 ) # Increase Sigma for noisier source (default is 2.0)

[Shon]

-Vit- I needed a first option. Thank you!

[-Vit-]

-Vit- I needed a first option. Thank you!
You're welcome.

BTW, for the best quality noise use the NoiseDeint="Generate" option, for example:
QTGMC( Preset="Slower", NoiseBypass=2, NoiseRestore=0.4, NoiseDeint="Generate" )

This option improves how the noise itself is "deinterlaced". It's only a small improvement, but it might be helpful for you.

[-Vit-]

I've uploaded a new QTGMC version 2.51 in the first post (http://forum.doom9.org/showthread.php?t=156028)

There are a number of changes:

The default interpolator for almost all presets is now NNEDI3 rather than NNEDI2, so make sure you have it. This improves the quality and speed of most presets, provided you use version 0.9.2 or above of NNEDI3 (http://web.missouri.edu/~kes25c/).
To accommodate this change many presets have had their settings tweaked. Two presets are particularly different:
The "Super Fast" preset now uses NNEDI3 with the fastest possible settings rather than a Yadif variant as before. This improves quality considerably, but it's a tiny bit slower than before
The "Ultra Fast" preset also gives higher quality output. Before it wasn't using any repair mode, which created much motion blur. Now it uses tr2=3. However, this means a ~6% loss in speed. Set tr2=0 for the old faster behavior
There's a new setting "EdiThreads", which allows you to specify how many threads EEDI3 and NNEDI2/3 will use. By default they will use one thread for each (logical) processor in your system. You may get better speeds on some scripts with more or less threads (experiment). Using less threads will also use less memory, which may help some complex/HD scripts, especially when also using MT or SetMTMode. Example:
QTGMC( Preset="Very Slow", EdiThreads=6 )
EdiMode can be set to "EEDI3+NNEDI3", which will use EEDI3 with its sclip parameter created with NNEDI3 - this should give a slightly improved EEDI3 result (but slower). This uses more memory - so be careful not to use too many threads if you're multithreading (EdiThreads or MT). E.g.
QTGMC( Preset="Slower", EdiMode="EEDI3+NNEDI3" )
Another new setting is "LosslessPreset". This is only used in lossless mode to drive the second lossless pass - it works exactly like the Preset setting except the default is "Fast". The quality difference between the lossless presets is minor so you can move towards faster settings here. Note: the LosslessEdi setting overrides the LosslessPreset edi mode setting.
QTGMC( Preset="Very Slow", Lossless=1, LosslessPreset="Faster" )
The output from ShowSettings has been made clearer and a minor bug fixed in that code
Fixed a serious bug when using lossless modes on the higher speed presets
I've tweaked the loading of Yadif again. The default behaviour hasn't changed - the calling script is required to load "Yadif" as normal if using settings that require it. However, now there is a global variable "YadifPath". It's initially empty, but if it is set to point to yadif.dll then it will be "autoloaded"


Thanks to Bi11 and henryho_hk for the suggestions that prompted some of these changes.

[dansrfe]

I love this function. Thanks for the update -Vit-. Brilliant script and of course kudos to Didée for coming up with TGMC in order for all this to happen! :D

[dansrfe]

Mediafire link to QTGMC v2.51: http://www.mediafire.com/?qxqp5q9dq2f5iqb

[-Vit-]

Mediafire link to QTGMC v2.51: http://www.mediafire.com/?qxqp5q9dq2f5iqb
Thanks for the mirror, the link in the OP is now direct to the zip file attachment. I just put it on a filehost temporarily whilst waiting for the attachment to be approved.

[Boulotaur2024]

Great "wrapper" I've been using it on almost all my projects lately. It's funny because the last changes you brought correspond more or less to what I was doing manually by using the ad-hoc parameters. I still think that maybe the default sharpening settings are a bit high but that may be just me since I usually resize x2 the QTGMC ouput which needs not to be "too sharp".

Thank you again for this function, very well documented too.

[dansrfe]

How can I use QTGMC to bob & interpolate at decent speeds on an i7 using 2.6 MT?

[-Vit-]

How can I use QTGMC to bob & interpolate at decent speeds on an i7 using 2.6 MT?
Use a script like this:

SetMTMode(5,8) # 8 threads for this script (the default for most i7s)
# Insert your source command here
SetMTMode(2)
QTGMC( Preset="Slow", EdiThreads=8 ) # 8 threads for NNEDI3 (the default for most i7s)
Distributor()


Change the QTGMC preset as necessary. Also tweak your encoder for the speed you require (e.g. x264 preset).

You can try increasing / decreasing the two threads values for best speed. I find having slightly fewer EdiThreads is best for my machine.

If it's a HD source then you can easily run out of memory with so many threads on a 32-bit system. So go 64-bit (I believe all the appropriate plugins exist now), or reduce the threads considerably and use a SetMemoryMax(xxx) at the start (experiment for best xxx value). Also encode in multiple passes, 1st pass QTGMC to lossless, 2nd pass lossless to x264 or whatever. MeGui pre-render job does exactly that. Will need lots of disk space for the lossless step. QTGMC is very forgiving with HD and you can get away with the quickest presets if you wish.

OC that i7 if you haven't already

[dansrfe]

Is encoding to lagarith and then sending that to x264 going to work the same way as well?

[-Vit-]

Is encoding to lagarith and then sending that to x264 going to work the same way as well?
I haven't used lagarith, but it's the same principle. Isn't encoding to huffyuv faster though? I know huffyuv will create larger files, but if you're going to encode to x264 afterwards then it doesn't really matter does it?

[dansrfe]

That's true^. btw what purpose does the "lossless" variable have? Since I'm encoding to a lossless format, be it megui pre-render, lagarith, or even huffyuv what does QTGMC's lossless do?

[-Vit-]

That's true^. btw what purpose does the "lossless" variable have? Since I'm encoding to a lossless format, be it megui pre-render, lagarith, or even huffyuv what does QTGMC's lossless do?
Yeah, I suppose I didn't explain it clearly anywhere. I'll add this info to the OP:

(Q)TGMC separates the interlaced source fields, then "fills in" the missing lines (with the help of an interpolator) to create a shimmer-free progressive output. However, by default, the original field lines do not remain unchanged in the output. Put simply, the input pixels are not in the output. This is not ideal - we are not rescaling, just filling in the "missing" lines so it would seem best to keep the original fields unchanged. But that is hard to do whilst removing the shimmer from the interlacing.

QTGMC's lossless mode will do additional processing to keep the original fields exactly as they were whilst still retaining as shimmer-free an output as possible. This mode gives output that is truer to the source, but it can contain some minor combing or bob-shimmer. It also retains any noise from the source (QTGMC denoises somewhat by default)

This mode has nothing to do with lossless encoding.

BTW. Megui pre-renders are encoded to huffyuv

[dansrfe]

Ok :D That makes things clearer. Thanks!

[yup]

dansrfe!
Try other lossless codec for example Ut. On my Core2 QUAD work faster. Lagarith use only 2 threads. I use Lagarith only for archive purpose, for intermediate using Ut.
yup.

[jpsdr]

@dansrfe
I also strongly suggest you tu use UT Video codec. I've made some tests, and confirmed they work faster. Problem of huffyuv is that you've not YV12 support, thing you have with UT Video. File with UT Video (YV12 codec) are a little bigger than lagarith, even in best compression, but it's realy a little, less than 10%.

[dansrfe]

Can you guys direct me to the link for UT Video?

[jpsdr]

Google : UT Video

[Revgen]

huffyuv does have yv12 support if you use the ffdshow version. I use it all the time to capture HDTV.

[moozooh]

Much thanks for taking over for Didee and his awesome script!

I have a question. I'm processing a considerably clean (very little noise, mostly MPEG2 artifacts if any) 60i game video for further encoding to x264 at around the crf of 22..27. Will it make sense for me to retain/generate the noise when using the slowest QTGMC presets? It seems like it would only complicate encoding since at that crf x264 will smear the noise invariably whilst spending more bits than otherwise in an attempt to preserve it.

Thanks in advance.

(Vit, out of curiosity… are you Russian?)

[-Vit-]

I have a question. I'm processing a considerably clean (very little noise, mostly MPEG2 artifacts if any) 60i game video for further encoding to x264 at around the crf of 22..27. Will it make sense for me to retain/generate the noise when using the slowest QTGMC presets? It seems like it would only complicate encoding since at that crf x264 will smear the noise invariably whilst spending more bits than otherwise in an attempt to preserve it.
The two slowest presets, "Very Slow" and "Placebo" use the wider temporal radii in the TGMC algorithm. This denoises strongly creating a very clean, but rather artificial look, also losing some detail. To counteract this, those presets also use some noise bypass to retain some of the detail/grain that would otherwise be lost. Obviously you can do this manually in the other settings if there's some grain you particularly want to keep.

Depending on the game content, there may well be grain in the textures/backgrounds (rather than over the entire image) that might be lost at the highest settings. And, assuming that this is a 720p+ capture, then a CRF of 22 might be enough to have caught some of it. But it will be a lesser effect than on the naturalistic sources for which the presets were designed. So, yes it may well be OK to reduce this processing to save on time & filesize (and memory used during the encode - important if you intend to multithread).

However, this is just speculation - I have only worked with a little game footage on the default "Slower" setting - and I didn't try noise bypass. Others will have more experience than I do for that kind of source. Why not just try encoding a few thousand frames each way and compare?

(Vit, out of curiosity… are you Russian?)
;)

[William.Lemos.BR]

Please, be pacient with my english, I'm from Brazil...

First of all, I'd like to say your filter is simply wonderful!! I've been testing other deinterlace methods (for years) and what you achieved here is very close to perfet!
Actually I'm not used to AVISynth but the reason I started to try it was exactly how impressed I was by the QTGMC's results. So, after some strugle to learn I managed to run it under VirtualDUb.

I became excited when I saw the new version here (specially for the speed tunnings). So I downloaded it and tried to test, but got the messege:
"Avisynth open failure:
The script's return value was not a video clip"

The strange is that I'm using the new QTGMC exactly the same way, and the previous version works fine! I double-checked if I have all the needed dlls (and last versions) but everything looks ok. I'm lost here.

Could you help me somehow?

Since now I thank you for the attention and, specially, for the EXCELENT work you've done!!

[-Vit-]

So I downloaded it and tried to test, but got the messege:
"Avisynth open failure:
The script's return value was not a video clip"
Please could you post the exact script you used to get this error message.

[William.Lemos.BR]

Please could you post the exact script you used to get this error message.

Thanks, -Vit-, for the reply!

As a matter of fact, my script is quite simple. I've just replaced the previous code by the new one. By the way, my OS is Windows 7 64 bits (but I'm using the 32 bits dlls and VirtualDUb). I'm quite sure the problem has nothing to do with this, 'cause everything runs fine, but I'm not an expert so... just in case.

PS: Yeah, I'm a noobie, but I'm not trying to open a .txt in VirtualDub. I added the .txt extention so I could upload the file, :sly:

There it is...

[poisondeathray]

Instead of waiting for a mod to approve your attachment, you can copy & paste the avs script here in text

Also, if you had screenshots or files you can use a free image host e.g. mediafire.com, sendspace.com

[Didée]

Are you using the latest version, v2.51? The previous versions "2.4r" and "2.47r" seemed to have a mistake in the showsettings section - according to the syntax highlighting of my editor, there was a " (quotation mark) too much (or one missing), which might cause the script to blow. (Though I didn't try).
The most recent version v2.51 seems to be correct. (Again, only judging by the syntax highlighting).

[William.Lemos.BR]

Instead of waiting for a mod to approve your attachment, you can copy & paste the avs script here in text

Also, if you had screenshots or files you can use a free image host e.g. mediafire.com, sendspace.com

Thanks for the suggestion, Poisondeathray! There it is the link with the .avs (I didn't paste the hole script because it is too big):

http://www.mediafire.com/?31dllf6310x3ds3

Are you using the latest version, v2.51?

Hi, Didée!

Yeah, I'm using the v2.51 that was downloaded in this very thread.

[-Vit-]

You've pasted the entire QTGMC script at the end of your script. This is unnecessary and causes a problem because of the global variable in QTGMC. Do this instead:
- Save QTGMC in a file on its own called "QTGMC2.51.avsi".
- Put that file in your Avisynth/plugins folder. Delete any other QTGMC files in there.
- Then QTGMC will "autoload" and your script is just this:

AviSource ("d:\VIDEO_720x480_1MIN_uncompressed.avi")

ConvertToYV12(interlaced=true)

QTGMC( Preset="Slower", EdiMode="NNEDI3" )

Or you can use the Import statement if you don't want to use autoload for some reason.

____

[The syntax was off in the earlier versions around the ShowSettings part but the Avisynth parser seemed to cope anyway.]

[William.Lemos.BR]

:stupid:

My God!!! How stupid I was!!!! hehehehehe

I'm going to fix it right now. Thank you VERY MUCH, for your help and pacience!

I'm still at the beggining of my learning but who knows someday I can retribute your help... :thanks:

EDITED:

I've just tested it and WORKS LIKE A CHARM!! Thank you, again!
Now I'm going to have fun all night long...

[Gavino]

You've pasted the entire QTGMC script at the end of your script. This is unnecessary and causes a problem because of the global variable in QTGMC.
Is there a good reason for the variable (Yadifpath) being global? It could just as easily be declared inside the QTGMC function (eg on the first line). Then such problems would not occur.

[-Vit-]

Is there a good reason for the variable (Yadifpath) being global?
Maybe there was a reason once, but I don't see it now... like much in life.
I'll shift it in the next release.

[SilaSurfer]

Hello -Vit-. First of all great work. I was wondering something is your script meant for pure progressive sources like Dvd's, if so probably for "denoising and sharpening/keeping detail" purposes wright?

[-Vit-]

Hello -Vit-. First of all great work. I was wondering something is your script meant for pure progressive sources like Dvd's, if so probably for "denoising and sharpening/keeping detail" purposes wright?
QTGMC is a deinterlacer: it's for converting interlaced content into progressive content.
DVDs are usually interlaced, not progressive, so QTGMC is an appropriate tool.

QTGMC can accept pure progressive content with the InputType setting:
QTGMC( Preset="Slow", InputType=1 )
It also has settings for retouching badly-deinterlaced material (progressive, but with strong bob-shimmer / interlacing artefacts remaining):
QTGMC( Preset="Slow", InputType=2 ) # or 3

Regardless of input type, QTGMC strongly denoises and sharpens by default. Although you can adjust how much as necessary.
However, if you only want to denoise or sharpen some progressive content, you're better using scripts/plugins designed for that. QTGMC is really only helpful if you have interlaced content or progressive content with some shimmer you want to remove as well.

[Didée]

DVDs are usually interlaced, not progressive, so QTGMC is an appropriate tool.
Depends on content, and where on the blue ball you're located. From the plentyness of my (PAL) DVDs, the vast majority has progressive content. Only very few have interleced content.

QTGMC can accept pure progressive content with the InputType setting
BTW, did you adjust the temporal averaging for this? For progressiv input, the whole "gaussian" thingy does not make all that much sense anymore.
(And BTW, the commented explanations about "gaussian" averaging are not good, they're almost wrong. At least, they are misleading and don't clarify the technical reason for using gaussian weights.) *)

The bigger deal would be to come up with a temporal straightening that works correctly for both interlaced and progressive sequences, i.e. for sources that contain both types of sequences.

Somehow, the real improvements hardly ever happen in modded spinoffs ...

- - - - - - -

*) Why gaussian averaging?

TGMC doesn't care for static sections. It assumes everything is moving. (Or: everything is static, due to MVTools.)

Imagine a perfectly-static thin line that is dumb-bobbed:

input pixel sequence (temporal): .... - 30 - 120 - 30 - 120 - 30 - 120 - 30 - .....


Now do a normal temporal average with 1-1-1 weighting:

resulting sequence: .... - 90 - 60 - 90 - 60 - 90 - 60 - 90 - ...

Result: not only the sequence is still flickering after the averaging. Worse, the flicker has been inversed! Up-down-up has become down-up-down!


Now, do a gaussian temporal average with 1-2-1 weighting:

resulting sequence: .... - 75 - 75 - 75 - 75 - 75 - 75 - 75 - ....

Result: stable. period.


The reason for using temporal gaussian is simply this:

Traditional 1-1-1 averaging will push the strobing pattern into inversion.
Gaussian 1-2-1 averaging is the [only] mathematically correct weighting to cancel-out the bob strobing.

Not more, not less, end of story. A simple, but well grounded mathematical reasoning.


- - - - - - -

This one is simply wrong:
# Spatially gaussian blur (and tweak) motion search clip for more stable motion vectors - removing noise makes features easier to track
This part is not about noise, and not about stable motion vectors. These issues already have been addressed previously, by the temporal filter.

What this section does is: reduction of the amplitude of the "signal peaks" of all prominent detail.

MVTools uses safe-guarding by SAD (or SSD). Big SAD -> do nothing. In traditional denoising, this avoids obvious artifacts where the motionsearch failed.

In particular, SAD is generally big around prominent detail. For general denoising this means that the very edges usually get almost no filtering. (!)
But in TGMC, the very edges are the main target of all filtering. For TGMC it is not acceptable that MDegrain "goes NOP" on edges. But, simply using insanely-high thSAD is not good either: this would lose the ability to reckognize bad motion estimation.
Therefore, TGMC does it the other way round. Instead of practically disabling the SAD concept, it "smallifies" the edges so that they don't produce such big SAD values anymore. This way, the error-reckognition-by-SAD stays at least partly intact.

And that's a completely different story, compared to "removing noise makes features easier to track" ... ;)


A (good) explanation surely is better than not explaining.

But - not explaining surely is better than giving a wrong explanation.

[-Vit-]

Ah! Finally some comment, Didee! Appreciated.

As I noted from the start, I commented for myself not for the wider world. I wanted to mod the script but it offered very limited explanations so it was a reverse engineering task and commentary helped me wade through it. The explanations evolved from deduction and experimentation. Inevitable that would lead to some misleading descriptions. I didn't originally intend to release this script so I wasn't concerned.

I toyed with the idea of deleting the comments before release but I disagree with your premise: I believe misleading comments are much better than none when the author is around to correct them. How else to get commented TGMC? :p

Your explanation regarding the two Gaussian averages makes perfect sense and is very welcome, thanks. I will update the commentary on those sections for the next release. If you notice anything else, please advise.

I didn't adjust the temporal work for progressive - I'll update the first temporal average code as appropriate. I'm not really interested in dealing with mixed sources.

[cretindesalpes]

The word "gaussian" is probably misleading too. If I understood correctly the concept, the real strategy for the coefficient choice is :

kill the component at the Nyquist frequency (bob flickering),
Preserve as much content as possible, especially the constant component,
and give more weight to the center frames, given their higher probability of validity after MC in the general case (oh statistics... now this may be a reason to wave the "gaussian" word !)

[-Vit-]

The word "gaussian" is probably misleading too. If I understood correctly the concept, the real strategy for the coefficient choice is :

kill the component at the Nyquist frequency (bob flickering),
Preserve as much content as possible, especially the constant component,
and give more weight to the center frames, given their higher probability of validity after MC in the general case (oh statistics... now this may be a reason to wave the "gaussian" word !)
I only identified the last point in the original commentary. I had queried Didee to see if that was at least part of the rationale - thanks for the confirmation that I didn't completely miss the point there :o

What I did miss was the fact that the temporal coefficients sum to give equal weight overall to even and odd frames (removing the Nyquist frequency). Leaving just one possible kernel for temporal radius 1 (1:2:1), and a constrained set of possibilities for radius 2 (e.g. 2:3:2:3:2, 1:3:4:3:1 or 1:4:6:4:1) from which a kernel is chosen that favors the central frame.

[aegisofrime]

Ok, so for the less technically inclined among us (myself especially):

Is it merely the comments that are wrong, or is the progressive input algorithm wrong as well? Because I have been finding that progressive input seems to have little effect on the results :/

[SilaSurfer]

Thanks -Vit- for the info. I will use QTGMC if I encounter any interlaced DVDs or videos in the future.

Didée
Just wanted to say Thank You for the past (IIP, LimitedSharpen, SeeSaw, 6of9 CQM...) and future if you have some plans. You're like Albert Einstein in video encoding world. ;)

[Didée]

aegisofrime: Yepp, I was only nagging about some comments giving rather poor explanation. (And I have one more, below.) *)
No technical bugs involved.

Regarding the "progressive mode(s)" ... there's nothing wrong about them. Just that they are supposed to help only very little.
(If 50% of the original input data is already missing, you can't use them to improve the remaining 50% that you have ... or to even *judge* the remaining half. Hence, consequentially, there's not that awfully much improvement to expect.)

Regarding different kernels ... well, the 1:3:4:3:1 one I had tried already, but there was hardly any difference.
Arguments:
a) if the input data fits the theoretical model, none of those kernels is better than the gaussian kernel.
b) if the input data does not fit the theoretical model (e.g.: bad motion matches), then the gaussian kernel does the least harm. The other kernels would include more potentially-bad data.
_______

*) Nagging ... :D ... about mt_inflate -vs- TGMC_inflate

First off, TGMC_in/deflate shouldn't be necessary anymore. This workaround was much faster than the C-routines of previous masktools. Now that Manao has asm'ed the XXflate filter, it's slightly slower. (thumb's values: old: 79 fps / TGMC_xxflate: 510 (750) fps / new masktools: 950 fps)
Now, one could put the original filters back in, since they are even faster. However ... the difference of the original workaround was noticeable. But if the even-faster original are put back in, you might sleep better, but you won't notice any difference.

In any case ... -Vit- had written that TGMC_xxflate would not do the same operation as the original masktools filters. That's mostly incorrect, it is the same operation. (Except a tiny difference.)

The first thing being wrong is ... the description in MaskTools' documentation. We read: mt_xxflate : It computes a local average by taking into account only the neighbourgh whose value is higher/lower than the pixel.
Short, grippy, and wrong. If this was the case, then xxflate would be almost the same as xxpand, in many cases.

To my knowledge, xxflate does this: compute the average of all eight neighbor pixels, then use that value only if it is darker than the original pixel (deflate), resp. if it is brighter than the original pixel (inflate).

And that's exactly what TGMC_xxflate is doing. Only difference: it uses RemoveGrain(20), i.e. computes the average of all nine pixels. To exactly mimick the masktools filter, it should use RemoveGrain(19). (which is also faster again, see thumb's values above).

Put in Removegrain(19), build a 10-fold inflate by both mt_inflate and TGMC_inflate, then look at the difference between them: There will be almost none ... ;)
(If any, it is because RemoveGrain and masktools use different code implementations for averaging. I.e., it's only rounding differences.)

[-Vit-]

-Vit- had written that TGMC_xxflate would not do the same operation as the original masktools filters. That's mostly incorrect, it is the same operation. (Except a tiny difference.)
That nag is a little unfair :(. My comment merely asserts that TGMC_xxflate is not "equivalent" to mt_xxflate, which is correct: TGMC_xxflate has a less strong effect. However, it is interesting to learn that the underlying operation is so similar.

@aegisofrime: Progressive input works, but has limited use - it is not a panacea. Maybe there are tweaks to be made, but any improvements will be tiny.

And on that point, I'm not seeing why the existing binomial smoothing kernels (QTBMC...?) are inappropriate for a progressive pre-filter. I was under the impression that a gaussian was at least a fair pre-filter. It seems convenient mathematical fate that a binomial kernel is both a good gauss approximation and also reduces Nyquist to 0 with its odd/even properties. But perhaps it's more complex than that...

[der_groschi]

Hey Vit, thanks for your great work on this awesome script.
@aegisofrime: Progressive input works, but has limited use - it is not a panacea. Maybe there are tweaks to be made, but any improvements will be tiny.
Well, recently "inputtype=2" did a great job for me on two problematic sources. The first one was a hard telecined DVD source which obviously came from an analog video source. It definitely wasn't fieldblended, but still something strange was going on with the original fields so the result of "TFM().TDecimate()" still had some combing and aliasing in it. QTGMC fixed it perfectly.
The second one was a heavily aliassed 30 FPS progressive DVD Source, which seemed to be the result of exactly the bad deinterlacing you say "inputtype=2" is designed for.
I haven't tested it yet, but i'd expect it to work just as good as a postprocessor for TDeint+NNEDI'ed and then srestore'd material.

Now, what do you suggest about that? Recently i've always feeded fieldblended material through good ol' TempGaussMC and then deblended it with srestore. The results were pretty satisfying, but it always seemed a bit unreasonable to me, first from a speed standpoint, beacuse TGMC had to process many frames (more then half) that get thrown away later anyway, and also because TGMC cannot develop all its power on such sources anyway because (if i understand this correctly) its motion compensated parts are pretty useless on non-pure interlaced sources.
What do you think about that? Is using QTGMC afterwards in progressive mode more reasonable or ar there still any benefits from the "oldschool"-aproach?

[-Vit-]

I don't work with fieldblended material or problem sources generally so others may be able to make better comment from experience. Your idea is to deinterlace in such a way to retain more detail (but also with more shimmering/aliasing), then remove the fieldblending, then QTGMC away the shimmer. The idea being that this will involve less processing as well as avoid using motion-compensation when the field blending is still there.

Progressive QTGMC is certainly effective to remove deinterlacing residue (http://forum.doom9.org/showthread.php?p=1455667) from earlier processing so your workflow seems to be within spec for QTGMC. Whether it's the best approach for this kind of material I cannot say having not tried it. I play down progressive QTGMC effectiveness in the general case because it's only tailored for sources with thin horizontal shimmer.

[pokazene_maslo]

Hello. How to set up QTGMC for stronger denoising? I have an interlaced DVD with lots of noise. So far I'm using this command: "QTGMC( Preset="Very Slow", Edimode="EEDI2", Sigma=2.2 , NoiseBypass=0)". AFAIK sigma parameter is useless if NoiseBypass=0. Thanks.

[henryho_hk]

Try putting degrainmedian(mode=2, interlaced=true) or even degrainmedian(mode=1, interlaced=true) before QTGMC().

[-Vit-]

The NoiseBypass setting is designed to retain noise that TGMC would normally lose, but it can be used to reduce noise too. However, you're switching off all noise processing by setting NoiseBypass to 0 and you're just seeing the denoising that inevitably comes with any TGMC processing.

There are three related parameters, NoiseBypass, NoiseRemove and NoiseRestore.
You should use NoiseBypass=1: this will remove noise before processing with the intent of restoring it afterwards. However, in your case you're not going to restore any.
NoiseRemove is a value 0.0 to 1.0, to indicate how much noise (at the selected Sigma level) you wish to remove before TGMC processing. Clearly you want 1.0 or close to it.
NoiseRestore is a value 0.0 to 1.0 (or more) to indicate how much noise to restore after processing. You should use 0.0 or a small value.
You should ensure you specify all these settings, because otherwise you'll get the defaults for "Very Slow", which include some noise retention.

So a initial attempt might be:
QTGMC(Preset="Very Slow", EdiMode="EEDI2", NoiseBypass=1, NoiseRemove=1.0, NoiseRestore=0.0, Sigma=2.2 )
Obviously, tweak for preference. You could try setting tr2=3 to get more denoising out of TGMC too.

As henryho_hk has indicated, for your purposes you could just denoise separately instead.

[henryho_hk]

When there is a large amount of grain, degrainmedian() helps mdegrain3() greatly, being more cost-effective than mdegrain4/5/6 mods in many cases.

[pokazene_maslo]

Thanks for very explanatory answer -Vit-. Finally a filter that provides results I wanted! Also thanks to henryho_hk, but I don't like the idea to run two separate denoisers when one properly set up can do the job.

[henryho_hk]

By adding NoiseRemove=1.0, we are activating FFT3DFilter(), which is quite strong. If we don't apply NoiseRestore in combination, the output may look too smooth to be realistic.

[-Vit-]

henryho_hk makes a good point. Should consider the effect of using NoiseBypass on details:

QTGMC-NoiseBypass uses FFT3DFilter, which is a spatio-temporal denoiser. However, at the core of QTGMC is MDegrain, which is a motion-compensated temporal denoiser (used regardless of NoiseBypass). MDegrain has a better chance of removing noise without losing detail on moving images since it is motion compensated. So there's good reason to consider lowering NoiseRemove from 1.0 - allowing the MDegrain to do some denoising, and hopefully retaining some more detail. Similarly, you may also wish to increase NoiseRestore from 0, to get back the impression of any lost detail, without making the restored noise too prominent. Depends on how terminally noisy your source is though...

A related point - QTGMC resharpens quite strongly, which will enhance any remaining noise. You may wish to consider reducing sharpness. I actually think Sharpness=0.3 or so is closer to the source.

Although not relevant for noise removal, for completeness I should note the purpose of NoiseBypass=2. This mode measures the noise at the start and restores at the end using the same settings as mode 1. But it only measures the noise at the start - it doesn't actually remove any. This mode expects the TGMC process to do all the denoising, then adds some noise back at the end. As such it tends to give stronger grain retention, and even a sharpening effect unless applied lightly. Suggested NoiseRestore value is around 0.4 for this mode.

[Boulotaur2024]

I actually think Sharpness=0.3 or so is closer to the source.... How about making it the default value then :] ?

[-Vit-]

... How about making it the default value then :] ?
I wanted QTGMC to be a drop-in replacement for TempGaussMC_beta2. I wanted to encourage uptake. So I kept all the important defaults the same when I first released it.
Many people used the defaults for TGMC_b2 and are continuing to do so on QTGMC. They're used to the additional sharpness. Didn't want them to think that QTGMC was "blurry"!

However, QTGMC is well used now so I'll probably tweak the default in the next version (with a big warning notice!).

[pokazene_maslo]

QTGMC-NoiseBypass uses FFT3DFilter, which is a spatio-temporal denoiser. However, at the core of QTGMC is MDegrain, which is a motion-compensated temporal denoiser (used regardless of NoiseBypass).
Hmm. What about adding motion compensation to FFT3DFilter?

[Didée]

You're not forced to like the default values. But they have been chosen for a technical reason.

The decision factor for TGMC's sharpness defaults was: the case of perfectly-static sections. (Keep in mind that TGMC does not use the "no motion -> weave fields" method). In perfectly motionless areas, the temporal processing is reducing the signal amplitude significantly (in borderline cases: 50% reduction). The default sharpness values of TGMC have been chosen to restore most of this loss. You can use lower sharpness settings, but then static sections (subtitles, logos, static backgrounds) will come out more blurry.

Example (http://www.mediafire.com/?mo4bljcmz24b3ez) (650 kB)

I agree the sharpening is a bit of a weak point - another aspect is areas of bad motion matching. there's danger that artifacts from bad matching may get additionally enhanced.

=> Full sharpening for static areas, somewhat less sharpening for motion areas, and no or only little sharpening for badly matched motion areas. (Sounds good? Sounds ... slow!)

Looking closer, there's quite a few more aspects that influence the required sharpening strength: the "Lmode" (spatial or temporal), which kind of interpolator is used, which kind of sharpening mode, thinning, a.s.o.

_____

Unrelated to sharpening:
A few days ago, on the German board we found a case (http://forum.gleitz.info/showthread.php?42935-DeRainbow-Skriptalternative-zu-DeRainbow-sh0dan) where TGMC turned out to be pretty effective in removing rainbowing ... but only when the chroma repair section was disabled. The bands of chroma fringes often are too broad to get through the repair section. OTOH, completely switching off the repair often is not such a good idea.
Suggestion: separate switch whether the repair values (NonBobDiff) shall work on luma+chroma, or only on luma.

[William.Lemos.BR]

Hi, everyone!

Since my last post I've been doing some tests and experiences with QTGMC. I say with gladness that I've achieved some REALLY IMPRESSIVE results. Thanks, Dideé, for your astonishing script and congratulations, Vit, for the improvements in QTGMC.

My goal is to use QTGMC for converting old VHS tapes into Blu-ray. Playing with the QTGMC's parameters and using MT I've managed to get a good balance between process speed / quality (in my old PentiumD 3.20GHz, running Win7 64). Now I can do 1 hour convertion in 24 hours with this code:

MT(""" QTGMC( Preset="Placebo", EdiMode="NNEDI3", EdiQual=1, NoiseBypass=2, Lossless=1, LosslessTR=3, LosslessPreset="Placebo", SubPel=4)""", 4,4)

I've choosed SubPel=4 'cause noticed some improovement in edge details (specially after I upscale to 720p using VideoEnhancer). Then I use NeatVideo to do some filtering in HD (my tests shows that this sequence gives the best results).

Just one thing that may be helpfull: when I set "EdiQual=3" VirtualDub crashes. Actually I'm satisfied with current settings and probably won't change them (just tried it 'cause QTGMC's help text gives this possible parameter), but NNEDI seems to alow EdiQual to be 1 or 2 (the error message said so).

Anyway, I'm going to use QTGMC extensively from now on and expect to help you somehow in retribution. Thanks a LOT!

[Didée]

VideoEnhancer
NeatVideo
Herewith I declare that you are forbidden to use TGMC or QTGMC. :D

/endofjoke

Could you show examples where pel=4 gives a visible advantage over pel=2? From my past experiences it's like 98% wastoftime + 2% improvement ...

Also, probably I'd rather go for upscaling with NNEDI, instead of VideoEnhancer. (It has been discussed before - my opinion is that VideoEnhancer can not hold its claims when it receives smoothely sampled input: it is impossible for simple technical reasons.)
Ideally would by upsampling internally in TGMC (i.e. combining the temporal processing with the upscaling), rather than using it in a post-upsampling style. But that's not trivial - it won't integrate easily, because of the "repair" section of Q/TGMC. (The spatial radii won't fit anymore when the upscale is made inbetween. )

[Boulotaur2024]

The decision factor for TGMC's sharpness defaults was: the case of perfectly-static sections. Ah sorry I thought QTGMC defaults were higher than the original TGMC (pretty sure I had read that somewhere... oh well)... Thanks for the explanation anyway, they're always valuable coming from you :]

Then I use NeatVideo to do some filtering in HD (my tests shows that this sequence gives the best results).Sorry for not being ontopic, but "best results" compared to what ? (just out of curiosity)

[-Vit-]

When I set "EdiQual=3" VirtualDub crashes. Actually I'm satisfied with current settings and probably won't change them (just tried it 'cause QTGMC's help text gives this possible parameter), but NNEDI seems to alow EdiQual to be 1 or 2 (the error message said so).

You're correct, thanks. NNEDI3 only allows quality setting = 1 or 2, whereas NNEDI2 allows 1,2 or 3. I'll correct the code/comment in the next version.

[wohoo]

Can some one kindly upload the AddGrainC 1.5, as the link for it on the first page doesn't work.

[-Vit-]

I've updated the links for AddGrain in the OP. There was a slightly newer version (http://ldesoras.free.fr/src/AddGrainC-1.5.1.7z) that I had missed.

[William.Lemos.BR]

Herewith I declare that you are forbidden to use TGMC or QTGMC. :D


My god! I'm feeling as if I had fallen (with a parachute) into the middle of a gunfight, not knowing who is the bad or the good guys. But now, after 24 hours of researching and testing I can say with ABSOLUT certainty: VIDEO ENHANCER IS A LIEWARE!

It doesn't use super resolution, as it claims. The comparison shows the same results of using nnedi3, with a tiny sharpening. I think the only time I was as shoked as now was when I discovered that Santa Claus doesn't exists! Thanks, Dideé, for helping me to see that this world is cruel, and that I've spent 46 bucks for a GARBAGE like this. :stupid:

Anyway, I'm a noobie, but I learn fast. This only confirmed to me that we may get better results with personal effort and research, instead of buying a comercial program from a software swindler.

Could you show examples where pel=4 gives a visible advantage over pel=2? From my past experiences it's like 98% wastoftime + 2% improvement

I think you're right. What I noticed was a little improvement in the "pixelation" of the image, but now I'm feeling like that philosopher who said "I know that I know nothing", thanks to you that ruined all my illusions... :(

But I'm going to make some more tests after recovering from the shock...

Ideally would by upsampling internally in TGMC


Totally agree with you, just don't have any idea of how to do it! The same about NeatVideo (probably I'm going to find a better tool within Avisynth). Need to study a lot... who knows in 20 years I become a Didée v.0.1? :D

But the idea behind super resolution is really interesting, do you agree? Going to give a try. In the end at least I'm going to learn usefull stuff.

Thanks Didée, for the enlightenment and pacience! :thanks:

[William.Lemos.BR]

Sorry for not being ontopic, but "best results" compared to what ? (just out of curiosity)

Actually is kind of obvious it's better to filter the video after upscaling it and, basically, that was what I confirmed. Indeed, the noise profile generated in Neat Video has more quality as big as the video is.

Regarding Neat Video: I've tested a lot of other softwares and plugins and conclueded this was the best (of the worst) way of filtering without loosing details (since you manage how to set it's parameters appropriately). What I noticed is that all filtering aways kills the details. But I'm new to Avisynth, and I'm pretty sure that with more knowlege I'll find a better way of doing it.

[Didée]

after 24 hours of researching and testing I can say with ABSOLUT certainty: VIDEO ENHANCER IS A LIEWARE!

It doesn't use super resolution, as it caims.
No, that's overboard again. It surely uses SR. The problem is that for this kind of SR to work out, you need input with special properties.

Take some random video.

a) Downscale with bicubicresize(halfwidth,halfheight,0.5,0.25). Upscale with VideoEnhancer. It will achieve "almost nothing" compared to a simple re-scaling.

b) Downscale with gaussresize(halfwidth,halfheight,p=100). Upscale with VideoEnhancer. It will achieve a good improvement over a simple re-scaling.

TSR can work pretty good when the input is of the "decimated resolution" kind. The only pity is that the vast majority of all sources doesn't fall into this category. :)


But the idea behind super resolution is really interesting, do you agree? Going to give a try. In the end at least I'm going to learn usefull stuff.
Sure it's an interesting idea. But as noted above, in most everyday cases it is almost impossible to perform.

Do a forum search on the term "super resolution" / "superresolution". The topic has been discussed a few times already. Usual partipiciants have been: the naive optimists stating "THIS IS IT!", and the pessimistic naysayers stating "Y've been rickrolled". Oh, and some others that were throwing whitepapers about different sensational breakthrough algorithms.

The current status is:

We have lots of paper, but no functioning general-purpose superresolution. ;)

[dansrfe]

We have lots of paper, but no functioning superresolution

Well said.

[William.Lemos.BR]

No, that's overboard again. It surely uses SR. The problem is that for this kind of SR to work out, you need input with special properties.

Actually I know somehow it uses SR, but not the way it promisses, 'cause it doesn't ADD new information to the resulting image by analysing each frame in a sequence (for instance: a small blurry face will continue blurred, but bigger; you're not going to see eyes mouth and nose, even blurred, as I imagine a real SR process would do). For me this is not much diferent or revolutionary compared to other enlargement methods.

We have lots of paper, but no functioning general-purpose superresolution. ;)

That's it, you're right. What a pity! :(

[Didée]

However, let me point out the term "general purpose".

It's a different story when talking about "specialised conditions". For example, the topic of this very thread is a script that truly performs temporal superresolution. :)

It is just limited by two conditions:

a) the input has to be interlaced. (Interlacing is a native form of "decimated resolution")

b) the more lowpass filtering was used to create the original interlacing, the less effective the TSR part will be. (Lowpassing effectively changes "decimation" into "blurring")

[plonk420]

is this supposed to be a 30fps-oriented deinterlacer? it seems to have half the framerate (or at least notable motion) that mcbob gives me... on even frames, the image seems to be just slightly changed from the odd ones...

love the speed, tho!

[aegisofrime]

is this supposed to be a 30fps-oriented deinterlacer? it seems to have half the framerate (or at least notable motion) that mcbob gives me... on even frames, the image seems to be just slightly changed from the odd ones...

love the speed, tho!

It's a bobbing deinterlacer. It does have a mode for processing progressive content that outputs same framerate, might you have used that by mistake? The parameters for that mode is

QuickTGMC(InputType=1)#or 2, or 3

[dansrfe]

yeah you still have to use srestore after this to remove all the extra dupes and blended frames.

[kypec]

Well, this is my first experience with any lossless video codec. I want to use it for intermediate files that are the product of heavy processing in Avisynth. Later I plan to encode those lossless files with x264 2-pass method.
So far I was able to create AVI files Video
ID : 0
Format : ULY0
Codec ID : ULY0
Duration : 1mn 0s
Bit rate : 50.7 Mbps
Width : 720 pixels
Height : 576 pixels
Display aspect ratio : 5:4
Frame rate : 25.000 fps
Standard : PAL
Bits/(Pixel*Frame) : 4.893
Stream size : 363 MiB (100%) via this command:avs2avi.exe myscript.avs -c ULY0

However, x264 can't recognize my AVI at all :(
What is the recommended way to feed UtVideo file into x264 encoder then?
:thanks:

[yup]

kypec!
try use simple script as input for x264
AVISource("Your file.avi")
I think this different approach for 420 coding.
yup.

[7ekno]

Totally agree with you, just don't have any idea of how to do it!

I would start with SimpleSlugUpscale (http://forum.doom9.org/showthread.php?t=153860), it works quite well on de-interlacing and upscaling ;) Sure it's not done within QTGMC, but it a nice "all in one" !

And it's a nice place to find all the filters needed for SSU & QTGMC in the one place (http://www.gyroshot.com/simpleslug.htm) (thanks Robert!) ;)

Also for those with little Avisynth experience, the walk thru (http://www.gyroshot.com/upscale2.htm) Robert has on his website (to get his filter & QTGMC) working is also pretty helpful !!

Regarding Neat Video: I've tested a lot of other softwares and plugins and conclueded this was the best

Um, yeah :P If I were you I would check out MCTemporalDenoise (http://forum.doom9.org/showthread.php?t=139766) and/or DFTtest (http://forum.doom9.org/showthread.php?t=132194) ;)

A simple command like MCTemporalDenoise(settings="low", gpu=true) will beat most things for denoising and speed !!! But of course it's probably far more customisable than NEATVideo !!!

7ek

[Desmodeus]

Comment Removed

[Mounir]

Is YUY2 /RGB support planned for QTMC ? I'm not a big fan of YV12

[-Vit-]

Is YUY2 /RGB support planned for QTMC ? I'm not a big fan of YV12
Adding other colorspaces wasn't on my to-do list, but I guess YUY2 should be fairly straightforward. It won't make it into the next update (due soon), perhaps the one after.
I don't think MVTools2 supports RGB input and MVTools2 is at the core of the script.

[Didée]

If it has to be by all means, you can do RGB, too. Take RGB in, clone it to YV12 to do the motion search. Then separate the three RGB channels to three greyscale YV12 clips, perform basically 3 times TGMC on that bunch, and join back to RGB.

"Mischief managed."

[palwan]

Any idea, why I get slower speed using QTGMC(preset="v.fast") ? what I mean I get faster speed using the options < "v.fast".

[Gavino]

Any idea, why I get slower speed using QTGMC(preset="v.fast") ? what I mean I get faster speed using the options < "v.fast".
It should be preset="Very Fast".
"v.fast" is not recognised, so it reverts to the default of "Slower".

[dansrfe]

Any idea, why I get slower speed using QTGMC(preset="v.fast") ? what I mean I get faster speed using the options < "v.fast".

With that logic I should be getting extreme or super duper fast speeds using QTGMC(preset="e.fast") or QTGMC(preset="s.d.fast") :rolleyes:

[palwan]

as it as been showed in script V.Fast,S.Fast,U.Fast, so I went according to that, but now I will try it with "Very Fast" and see the result.

[Gavino]

as it as been showed in script V.Fast,S.Fast,U.Fast, so I went according to that
You were looking at a comment in the code that uses abbreviations to fit on the line for the table below it.
For the actual values, see the comments in the Presets/Tuning section.

[palwan]

Yes, you were right Gavino :-) Now it works fine, thanks alot for pointing me out.

[szabi]

AVISource("C:\my_dv_vhs_video.avi,fourCC="cdvc").ConvertToYV12(interlaced=true).QTGMC( Preset="Slow", EdiMode="NNEDI3" ).SelectEven().FFT3DFilter (sigma=2, sharpen=0.4)

I am using this for deinterlacing and sharpening my video.
I tried preset slower but it was too slow that is why I changed preset slow, but the encoding speed looks like the same as slower.

Are there any way to speed up it?
Multithread support or anything else which can help to keep the quality with better speed.

bye
szabi

[yup]

szabi!
I am using QTGMC 2.51 wit Set Avisunth 2.6 build for multi-threading and masktools 2.6. For quad core speed up 2.5-3 times comparing to one thread.
For speed up encoding, first make lossless AVI (I am prefer Ut codec) using avisynth script and after make encoding.
yup.

[szabi]

I have DV-AVI (720x576@25fps). Should i transform it to an other avi, before encoding?
I think an unnecessary encoding just decrease the quality.
That is the last avisynth version I know: Avisynth 2.58 (http://sourceforge.net/projects/avisynth2/files/AviSynth%202.5/AviSynth%202.5.8/).
I have to check what masktool is.
It is the first time I hear about.

[Gavino]

I have DV-AVI (720x576@25fps). Should i transform it to an other avi, before encoding?
I think an unnecessary encoding just decrease the quality.
That's why yup said use a lossless codec for the intermediate. The idea is that a 2-pass encoder (for your final result) can be sped up by using the lossless intermediate as input rather than processing the (slow) script twice.

[Mystery Keeper]

I have DV-AVI (720x576@25fps). Should i transform it to an other avi, before encoding?
I think an unnecessary encoding just decrease the quality.
That is the last avisynth version I know: Avisynth 2.58 (http://sourceforge.net/projects/avisynth2/files/AviSynth%202.5/AviSynth%202.5.8/).
I have to check what masktool is.
It is the first time I hear about.

Your avi is encoded (not sure with what codec, but "avi" is just a container). When you process it with AviSynth, you get uncompressed frames, which you can either store as is or compress with encoder. If you save your video uncompressed, it takes a LOT of space to store and memory to play, so you better compress it. When it comes to using heavy scripts in AviSynth, it is oftenly good to save the output uncompressed or compressed with lossless codec, then compress it with the codec of your choice. That way AviSynth is not loading your CPU at actual compressing and doesn't run twice during 2 pass encoding.

[szabi]

As I wrote my avi is 720x576@25fps (FourCC:dsvd[sony digital video]).
But it is hard to work with it, so in script you see I change it to "cdvc"

Do i have to transfer it to uncompressed format or any kind of script modification is enough?

szabi

[ajp_anton]

I'm having trouble with a small timer that changes every frame. Instead of just upscaling the changing digit (MCBob does this), two frames are blended together.
Right now I'm using tr0=1,tr2=0,rep2=1 to improve this, but it reduces the quality of the rest of the video. What else can be done?

[-Vit-]

I'm having trouble with a small timer that changes every frame. Instead of just upscaling the changing digit (MCBob does this), two frames are blended together.
Right now I'm using tr0=1,tr2=0,rep2=1 to improve this, but it reduces the quality of the rest of the video. What else can be done?
You could set tr1=1 also to reduce the first temporal radius used for the output.

You could reduce blocksize so it is less likely to blend the digit as it finds it a poor block match. Try Blocksize=8. Too small a block size can cause poor motion matches (need enough detail in a block to make a good match)

A similar idea would be to reduce the SADs used for the temporal passes. However, I removed those as arguments in this version (can't remember why - probably cutting down argument count). You can find them in the script - search for _thSAD1 (& 2). The first has the stronger effect on the output. It defaults to 10*8*8, try reducing it to 5*8*8 or similar. This will make it less willing to blend blocks if they match poorly. It may reduce the effectiveness of the deshimmering though. If you don't like editing the script for a single vid, then use TempGaussMC_beta2, which has these values as arguments.

I may reinstate the SAD values as arguments next version.

[kolak]

Is there a way to use QTGMC with YUY2 in lossless mode?
When I tried it says not supported- converting to YV12 solves problem, but I would like to keep source as YUY2.

Thx

[Didée]

interlaced_yuy2_video

ConvertToYV12(interlaced=true).QTGMC().ConvertToYUY2().MergeChroma(last.Yadif(mode=1)) :p


It would be possible to make a "proper" YUY2 processing mode (it has been done before for TGMC). It only requires a bazillion of ConvertInterleavedToPlanar()/ConvertPlanarToInterleaved() Back-and-Forth-conversions in the script.

[kolak]

interlaced_yuy2_video

ConvertToYV12(interlaced=true).QTGMC().ConvertToYUY2().MergeChroma(last.Yadif(mode=1)) :p


It would be possible to make a "proper" YUY2 processing mode (it has been done before for TGMC). It only requires a bazillion of ConvertInterleavedToPlanar()/ConvertPlanarToInterleaved() Back-and-Forth-conversions in the script.

What is the quality loss?
What Yadif is doing there?

Thx

[Didée]

As they are, Q/TGMC can not process YUY2 video. However, the most obvious benefits of Q/TGMC is what they achieve on the luma plain. Therefore, that scriptline suggests to use Q/TGMC for luma (which is identical for YUY2 & YV12 anyways), and one of the "simple" deinterlacers for chroma.

[kolak]

Sounds good and I assume processing speed will increase a bit also.


Andrew

[Didée]

No. It will be a little slower. Q/TGMC is still doing "all" of the work, including chroma. A 2nd deinterlacer is used additionally, and its result replaces that of Q/TGMC (on chroma channels). More overall work => slower processing.

Such a "cheat mode" definetly could be integrated in the function, and then it would be possible to avoid a major amount of the overhead. (Not all, but most).

[-Vit-]

Thanks for the YUY2 version. It's good you didn't add it to the lossless code, because that part has been almost completely removed in the next version - it's been superseded by something quite new. I'll merge the changes into the version after next, when I'll also be looking at chroma optimizations.

[kolak]

Thanks- I was one of them, but interested in lossless part- hehehe :)


Andrew

[yup]

Stephen R. Savage! :thanks:
My source is analog capture in YUY2 colorspace. Now we can forget about wrong conversion for YUY2 to YV12 for interlaced source.
You use 2.58 Avisynth ot MT 2.57 build? Set 2.6 do not support planar video.
yup.

[pokazene_maslo]

Hello.
I have 3 questions about denoising.

fft3dfilter can be used to denoise interlaced and progressive video. If I use QTGMC like thisQTGMC( Preset="Very Slow", Sigma=2.2 , NoiseBypass=1, NoiseRemove=1.0, NoiseRestore=0.0 )

fft3dfilter is used before deinterlacing. However suggested order of filters is "deinterlace->crop->resize->denoise".
My question is which provides better quality? Denoise first and then deinterlace or deinterlace first and then denoise?

After answering question number 1, how to make fft3dfilter motion compensated? I'm using 64 bit avisynth and don't know how to do that.

Is it OK to replace fft3dfilter with fft3dgpu?

Thanks in advance!

[2Bdecided]

I just wanted to post to say that I love QTGMC. The overall speed increase on a full encode seems to be much greater than it appears to be during the first few frames. I don't know if that's right/wrong, but the output is fine so I'm very pleased.

Cheers,
David.

[-Vit-]

Hello.
I have 3 questions about denoising.

fft3dfilter can be used to denoise interlaced and progressive video. If I use QTGMC like thisQTGMC( Preset="Very Slow", Sigma=2.2 , NoiseBypass=1, NoiseRemove=1.0, NoiseRestore=0.0 )

fft3dfilter is used before deinterlacing. However suggested order of filters is "deinterlace->crop->resize->denoise".
My question is which provides better quality? Denoise first and then deinterlace or deinterlace first and then denoise?

After answering question number 1, how to make fft3dfilter motion compensated? I'm using 64 bit avisynth and don't know how to do that.

Is it OK to replace fft3dfilter with fft3dgpu?


The basic TGMC algorithm strongly deinoises, so the NoiseBypass setting is there to help keep grain and fine detail. But yes, with those settings it works for controlled denoising too. And since you're going to get some kind of denoising with TGMC whatever you do, then what you're doing is not unreasonable. However, which method provides best quality only you can decide. I use a hybrid approach, denoising some in QTGMC, some afterwards.

According to the docs, FFT3DFilter performs "partial motion compensation" internally. The temporal window is controlled with the BT setting in QTGMC. You can also get somewhat stronger motion-compensated denoising by increasing tr2 (max=3).

I haven't tried using fft3dgpu in QTGMC yet. I could be wrong, but doesn't fft3dgpu prevent use of SetMTMode? I couldn't imagine running (Q)TGMC single threaded, but I guess some people are...?

There are new features coming in the next couple of days concerning noise processing. I'll detail them at release time.


I just wanted to post to say that I love QTGMC. The overall speed increase on a full encode seems to be much greater than it appears to be during the first few frames. I don't know if that's right/wrong, but the output is fine so I'm very pleased.

Thanks. I use MeGUI for processing and I've noticed a similar effect. Maybe the fps calculation is taking account of initialization time?

[2Bdecided]

I use MeGUI for processing and I've noticed a similar effect. Maybe the fps calculation is taking account of initialization time?MeGUI here too - don't know - the frames themselves count by slowly at first, then far quicker after a while!

Cheers,
David.

[Gavino]

MeGUI here too - don't know - the frames themselves count by slowly at first, then far quicker after a while!
Probably because the later frames get more benefit from Avisynth's cache?

[-Vit-]

Probably because the later frames get more benefit from Avisynth's cache?
There are no huge temporal windows, so wouldn't the caching settle relatively quickly after only a few frames? The effect I see in MeGUI lasts for minutes. :confused:

[Didée]

If the effect is not present during e.g. screen previev in Vdub, but is present during encoding (probably to x264) - - - what does this tell you? :)

Maybe related (or maybe not) - see the speed oddity that Zep has reported here (http://forum.doom9.org/showthread.php?p=1445970#post1445970).

My vague guess is that it's related to "looking into the future". Keep in mind, with preset fast|medium|slow|slower, x264 is requesting 30|40|50|60 frames ahead of the current frame, due to MB-tree. That's not trivial! Quite possible that Avisynth needs some time to adapt its caching strategy to this situation.

[ajp_anton]

My vague guess is that it's related to "looking into the future". Keep in mind, with preset fast|medium|slow|slower, x264 is requesting 30|40|50|60 frames ahead of the current frame, due to MB-tree. That's not trivial! Quite possible that Avisynth needs some time to adapt its caching strategy to this situation.
I don't think Avisynth has any idea what x264 is doing. It should just read frames linearly no matter what the lookahead is.
x264's progress only reports when the frames are actually processed, which means that for the first ~40 frames of Avisynth processing, x264's speed is 0. Because the speed reported is the average, it will speed up during the encode. And finally, for the last ~40 frames, Avisynth will finish and x264 can process without any input bottlenecks.

[-Vit-]

I was encoding to lossless, so x264 is not the issue I'm seeing. Checking the MeGUI source, it is because MeGUI starts timing before the first frames are processed (or are reported as processed). The initialization time (several seconds on a complex script like this) skews the average for a while. I didn't check but there may also be a timing lag due to the threading of the status updates too (which sleep for a second at a time). I tried a quick tweak so it only starts the timer once frame 0 has come in and the fps converges quite quickly. MeGUI displays an overall average fps, a rolling average like vdub would be less prone to this.

[kypec]

I haven't tried using fft3dgpu in QTGMC yet. I could be wrong, but doesn't fft3dgpu prevent use of SetMTMode? I couldn't imagine running (Q)TGMC single threaded, but I guess some people are...?

Hey -Vit-, could you share an example script demonstrating how is QTGMC run multithreaded? I mean do you prefer SetMTMode(...) {temporal distribution} or MT(...) {spatial distribution} in your workflow?
:thanks: in advance!

[AnXioZ]

Hey guys, I'm having some trouble deinterlacing 1080i60 (60 fields per second) source. I want to keep the motion resolution so I am looking at 1080p60 deinterlaced result. When using QTGMC with any of the presets I get very weird results, such as repeated past frames so the video looks quite jerky. I am previewing the video in Avisynth. Adding "SelectEven()" or "SelectOdd()" fixes the issue but of course the motion resolution is cut in half. Any advice is appreciated. Thanks.

[aegisofrime]

Hey -Vit-, could you share an example script demonstrating how is QTGMC run multithreaded? I mean do you prefer SetMTMode(...) {temporal distribution} or MT(...) {spatial distribution} in your workflow?
:thanks: in advance!

Do it like this:

SetMTMode(5,4)
DGDecode_mpeg2source("C:\Source.d2v")
SetMTMode(2,4)
QuickTGMC(Preset="Slower")

Distributor()

Do not MT() QTGMC. It causes artifacts.

Hey guys, I'm having some trouble deinterlacing 1080i60 (60 fields per second) source. I want to keep the motion resolution so I am looking at 1080p60 deinterlaced result. When using QTGMC with any of the presets I get very weird results, such as repeated past frames so the video looks quite jerky. I am previewing the video in Avisynth. Adding "SelectEven()" or "SelectOdd()" fixes the issue but of course the motion resolution is cut in half. Any advice is appreciated. Thanks.

It sounds like you might have your field order wrong. Try adding either AssumeTFF() or AssumeBFF() before calling QTGMC. Like so:


SetMTMode(5,4)
DGDecode_mpeg2source("C:\Source.d2v")
SetMTMode(2,4)
AssumeTFF() # or AssumeBFF()
QuickTGMC(Preset="Slower")

Distributor()

@Vit: Any release date for the new version? :)

[Gavino]

Do it like this:
...
Distributor()
Whether or not to call Distributor() depends on how the client (encoder, player, etc) calls Avisynth. If it uses the VfW interface or if is MT-aware, you will end up calling it twice, creating twice as many threads.

[kypec]

Whether or not to call Distributor() depends on how the client (encoder, player, etc) calls Avisynth. If it uses the VfW interface or if is MT-aware, you will end up calling it twice, creating twice as many threads.
Sorry, didn't quite understand what you're trying to say with this:confused:
I'm using this Avisynth script either through avs2avi.exe (as intermediate file) or directly fed into x264.exe (perhaps piped over avs2yuv.exe with 64-bit x264).
Should I or should I not include Distributor() statement then?

[Gavino]

I don't know - it's not something I've had to deal with in practice as I have only ageing single-core machines and don't actually use MT. :(

What I do know is that Distributor() (described here (http://avisynth.org/mediawiki/MT_modes_explained)) is the function that creates the threads and controls their interaction, and is automatically added to the filter chain when Avisynth is used through VfW.
This will not happen with an application that invokes Avisynth directly, but the application may be clever enough to detect a multi-threaded Avisynth and call Distributor() itself.

If Distributor() is not called at all, you won't get any multithreading.
OTOH, if it is called twice, you will get twice as many threads, with unnecessary additional overhead.

So it's important to know what your Avisynth client already does in this respect.
Unfortunately, I don't know what the answer is for either avs2yuv, avs2avi, or x264. :(

[aegisofrime]

I think the simple solution will be to just try 2 scripts, one calling Distributor and one without, and see which one is faster for you. I tested this myself a long time ago and using Distributor was much faster than without. I use MeGUI to encode to x264 btw.

[AnXioZ]

It sounds like you might have your field order wrong. Try adding either AssumeTFF() or AssumeBFF() before calling QTGMC.
Yep, the field order was wrong... You are awesome! Thank you!

[-Vit-]

@Vit: Any release date for the new version? :)
It's finished, just one very minor bug to fix. Tomorrow.

[-Vit-]

Just posted the latest version 3.0 (http://forum.doom9.org/showthread.php?p=1423459). There are many new features and the script has been significantly overhauled. Please advise of any issues I've introduced.

Highlights
- A new algorithmic step called "source-match" which gives higher fidelity output and is more accurate to the source
- Simplified and improved lossless output (related to above)
- Improved progressive modes (InputType=2,3), which now use a motion mask to help recover more detail (progressive also supports source-match)
- Motion-compensated noise bypass. Effectively providing an alternate deinterlacing path for detail not amenable to interpolation
- Major documentation improvements - more user-friendly categorized information with many examples of use
- See version history in script for more...

The original algorithm is unchanged apart from a couple of minor bug fixes so existing scripts should work the same. No speed increases in this version, the changes all focus on quality.

For even more detail than this post look at the documentation in the script itself. I've split the algorithm into categories (e.g. Sharpness, Progressive Input). For each category I've provided a summary, full description, settings/defaults list and usage examples. Hopefully that will allow people to get the most out of the script without guesswork.

Impatient? tl:dr? This is the essence of the new features, higher quality than the previous version:
QTGMC( Preset="Slow", MatchPreset="Very Fast", SourceMatch=3, Lossless=2, NoiseBypass=2, NoiseDeint="Generate" )
________

Source-Match
The binomial (née Gaussian) smooth used at the core of the (Q)TGMC algorithm does a great job of removing bob-shimmer, but leaves the image blurred. The original algorithm resharpens to recover, but that process is far from ideal: sometimes it oversharpens and it's unable to recover lost detail in any case.

Source-match is an alternative to full resharpening that captures more source detail, noticeably improving perceived vertical "resolution". It also reduces the potential for oversharpening / haloing. There are three stages of source-match, each refining the last, which take the smoothed image and restore it to match the source without introducing shimmer. Once matched, further sharpening then becomes somewhat of an optional extra. Clearly these modes involve extra processing (see tech detail below). I'm going to cover this feature in depth, but please be aware that this is a precise and subtle effect for those requiring exacting detail only.

***Examples***
Here are some samples to show the difference. This is a 720p sample (from the "in_to_tree" video at Xiph.org (http://media.xiph.org/video/derf/)) thinned to 720i and then re-bobbed. This is a crop with a 3x zoom. All lossless x264 in MKV. The QTGMC settings used are later in this post.

First a comparison of placebo settings with source-match and placebo without. Look for the additional "resolution" with source match - this is genuine extra detail, not sharpness.
Placebo_Comparison.mkv (http://www.mediafire.com/?5szm5xfsn92o8kp) (300k)

Next a comparison of the 720p source and two source-match examples. This is a comparison of true 720p and bobbed 720i, I find this impressive given such a detailed source.
Note that source-match improves detail, but does not fix interpolation errors (as shown by the lines on the roof, which NNEDI3 joined up incorrectly)
Source_Comparison.mkv (http://www.mediafire.com/?6kcahljmqizr9mj) (500k)

A comparison at default settings with and without source-match ("Slower" preset).
Standard_Comparison.mkv (http://www.mediafire.com/?8h2l9eul8dl9yjm) (300k)

Full sample movies with and without source match. Although there is a demonstrable increase in detail in every frame, the differences may only be apparent on a large display or upscale (I did say it was a precise effect!). The source-match is frame-by-frame very close to the raw source. I should probably calculate some metrics, maybe later...
Source_Match.mkv (http://hotfile.com/dl/99271509/771258f/Source_Match.mkv.html) (720p 237Mb)
No_Source_Match.mkv (http://hotfile.com/dl/99279643/aabd97a/No_Source_Match.mkv.html) (720p 222Mb)

Using Source-Match
The templates below increase in both quality and slowness.

# Choose preset as required (very fastest presets are not suitable though)
# Start with Sharpness=0.2 and adjust as necessary. Use TR2=1 for a clean source, use TR2=2
# or TR2=3 (slower) if more denoising is required, or alternatively use Noise Bypass
QTGMC( Preset=XXXX, SourceMatch=1, Sharpness=XXX, TR2=X ) # Basic mode, fastest
QTGMC( Preset=XXXX, SourceMatch=1, Lossless=2, Sharpness=XXX, TR2=X ) # Good quality, efficient, minor residual combing
QTGMC( Preset=XXXX, SourceMatch=3, Sharpness=XXX, TR2=X ) # Good quality, slower, no combing
QTGMC( Preset=XXXX, SourceMatch=3, Lossless=2, Sharpness=XXX, TR2=X ) # Best quality, slowest, very minor residual combing

Caveats
Because of its very nature, source-matching will faithfully recover artefacts in the source. Consider this if you have a poor source. Similarly, it's sensitive to source noise. Increase TR2 to 2 or 3 for extra denoising or use noise bypass. Source match modes 2 and 3 occasionally introduce minor aliasing, relatively rare though. They occasionally repair aliasing too.

Source-matching requires additional interpolations / temporal smooths depending on mode. Expect slower processing. However, you can use somewhat faster presets and still get results better than no source-matching. See the algorithm section below or the source-match category in the script to find out about source-match presets for speed tweaking.

At this point I've barely tested source-match with the very high speed modes (so much to test on this version), but I don't recommend it since the improvements will be insignificant against those lower quality modes.
________

Lossless
Most of the lossless settings have been removed, superseded by source-match. Only one setting remains: Lossless=1 = true lossless. Lossless=2 = "fake" lossless. Both lossless modes will add a little extra detail to the output. But both lossless modes can also create combing where they don't match the other field lines. So they both work much better with SourceMatch=3 as this will ensure the best possible match (the combing will be very minor compared to previous versions). Lossless mode 1 can still create a shimmery incongruity between alternate lines though. Lossless mode 2 occurs prior to sharpening and the final smooth, which allows some measure of tweaking. That means lossless mode 2 isn't exactly lossless - this mode is better seen as an optional extra for source-match (can be applied alongside any source-match mode).

Examples above.
________

Noise Bypass
I consider noise bypass to be an alternate method to deinterlace components of the image which cannot be interpolated effectively - noise, fine texture, very subtle detail. Things that the binomial smooths might also blur away. The idea is to split the input into clean-image and noise/fine-detail and send each part down it's respective route to be reassembled at the end. There are two changes for this:

Firstly the addition of DetailRestore, which works together with NoiseRestore. It controls how much noise is restored prior to the final temporal smooth. This seems counter-intuitive, restoring noise prior to denoising, but the final denoise can be fairly light and will allow through detail extracted as noise. This detail may not have survived the two stronger temporal smooths that it bypassed.

Secondly, the NoiseDeint="Generate" setting is now motion-compensated. The noise, extracted from an interlaced source, is "deinterlaced" to create a full height frame of noise to add back at the end. The "Generate" setting uses AddGrain to generate new noise for the missing fields and now does something like an "MGrain" with the neighboring frame - kind of the opposite of MDeGrain - it doesn't smooth the noise but uses the motion compensated neighbor frame as a guide to help build the noise in the current frame. This creates a sequence of noise that is fairly bob-shimmer free (but otherwise "noisy").

Here are a couple of examples:

QTGMC( Preset="Slower", NoiseBypass=2, NoiseDeint="Generate" ) # Note that "Generate" mode is rather slower than the alternatives
QTGMC( Preset="Slower", NoiseBypass=2, NoiseRemove=1.0, DetailRestore=0.4, NoiseRestore=0.2, Sigma=1.8, NoiseDeint="Generate" )
For the lighter NoiseBypass=1, start with the sum of Noise+DetailRestore = 1.0 and adjust to taste. For the stronger NoiseBypass=2, start with the sum = 0.4.

You can visualize the noise by setting ShowNoise=X, where X is a value around about 4.0 to 8.0. This will display the extracted noise as output (display contrast determined by X). Can use this to see the "Generate" mode deinterlacing the noise in your video. Or more usefully it helps to select a good Sigma value (which determines how much noise is extracted).
________

Progressive Input
InputType=2 or 3 are designed for repairing progressive material with severe deinterlacing artefacts (or other horizontal artefacts). They effectively drop half the "fields" and reweave, so the less aggressive InputType=1 should be preferred when it works effectively. But for those sources where InputType=1 is not effective enough, there is now a setting ProgSADMask that helps modes 2 and 3 recover more detail in stable areas.

All InputTypes support source-match as described above. This particularly helps InputTypes 2 & 3 recover detail, it's less useful for InputType=1. Using ProgSADMask and SourceMatch can produce surprisingly good results for some sources.
________

Scripts used in Samples

# All QTGMC scripts began with:
RawSource("in_to_tree_420_720p.y4m")
SeparateFields().SelectEvery(4,0,3).Weave()

# Placebo_Comparison
a = QTGMC( Preset="Placebo", MatchPreset="Medium", SourceMatch=3, Lossless=2, MatchEnhance=0.7, Sharpness=0.1, tr2=1, \
NoiseBypass=2, NoiseRemove=1.0, DetailRestore=0.5, NoiseRestore=0.1, Sigma=3, NoiseDeint="Generate" )
# vs
b = QTGMC_2_51( Preset="Placebo", tr2=1, Sharpness=0.8, NoiseBypass=2, NoiseRemove=1.0, NoiseRestore=0.3, Sigma=3, NoiseDeint="Generate" )

# Source_Comparison
a = RawSource("in_to_tree_420_720p.y4m")
# vs
b = QTGMC( Preset="Placebo", MatchPreset="Medium", SourceMatch=3, Lossless=2, MatchEnhance=0.7, Sharpness=0.1, tr2=1, \
NoiseBypass=2, NoiseRemove=1.0, DetailRestore=0.5, NoiseRestore=0.1, Sigma=3, NoiseDeint="Generate" ) # Retain grain
# vs
c = QTGMC( Preset="Placebo", MatchPreset="Medium", SourceMatch=3, Lossless=2, MatchEnhance=0.7, Sharpness=0.15, tr2=3, \
NoiseBypass=2, NoiseRemove=1.0, DetailRestore=0.3, NoiseRestore=0.0, Sigma=3, NoiseDeint="Generate" ) # Denoise

# Standard Comparison
a = QTGMC( Preset="Slower", SourceMatch=3, Lossless=2 )
# vs
b = QTGMC_2_51( Preset="Slower", Sharpness=0.8 )


Continued...

[-Vit-]

Continued from post above, go there for basics...

Source Match Algorithm
Here's the algorithmic detail on this new addition to the core algorithm. There are three stages, basic, refined and twice-refined:

Basic Source-Match
Intuitively, a deinterlacer should just inject new fields into the source leaving the original pixels untouched. Unfortunately, the binomial smooth used to remove bob-shimmer also changes the original fields - the output no longer matches the source. So find the difference between the source and the equivalent fields in the output. Then a trick: using this difference, calculate a revised source that, when binomially smoothed, would result in an output that does match our original source. We then interpolate and binomially smooth this new source. Having gone through a binomial smooth the result will be shimmer free and, by definition, it will match the original source.

The formula used for this correction is:
P0' = P0 + (P0 - P1)/(k + S(1-k)), where:
P0 is original source
P1 is the output after the first attempt at interpolation/smoothing
P0' is the revised source for second attempt at interpolation/smoothing
k is the weighting given to the current frame in the binomial smooth
S is a factor indicating "temporal similarity" of the error correction required from frame to frame,
i.e. S = average over all pixels of (neighbor frame error / current frame error)
Decreasing S will make the result sharper, sensible range is about -0.25 to 1.0. Empirically, S=0.5 is generally effective.

The result is a better source match, which is shimmer-free and has some measure of sharpening. The match isn't perfect due to the approximated nature of the factor S [a deeper analysis might be worthwhile] and a little further sharpening is still needed. It causes few artefacts but it rarely repairs interpolation errors (e.g. when NNEDI3 mistakenly joins up lines). The process requires a second round of interpolation (e.g. NNEDI3) plus a second binomial smooth. In practice, you can use faster settings for the initial interpolation, and your desired settings for this basic source match pass.

Fine control of source match settings is left entirely to presets or there would be hundreds of settings. Basic source-match is enabled like this:

QTGMC( Preset="Slower", SourceMatch=1 )
QTGMC( Preset="Slower", SourceMatch=1, MatchPreset="Very Fast") # Tweak source-match interpolation settings (e.g. for NNEDI3)
# Default is 3 settings quicker than "Preset"

Refined Source-Match
You could chain together several iterations of the basic source-match method above in the hope that it would progressively refine to an ideal solution. However, subsequent passes make little headway. Instead, a slightly different approach is taken. Refined source-match takes the basic source-match output and again finds the difference between the original source fields and the updated fields. This time it interpolates the difference itself, then binomially smooths it, before adding it to the full-height output. This makes a small, but distinct step closer to the source. This method requires another interpolation step, but in practice a simple bob can be surprisingly effective. The better interpolaters are not really designed for difference images so it helps to pre-sharpen before this step to give them a more defined image to work on.

Twice Refined Source-Match
The refined stage above used another binomial smooth to avoid adding shimmer into the output. The second refinement uses the same formula as the basic source-match step to further correct the previous adjustment. This gets a tiny bit closer again to the source and also reduces the slight sensitivity to noise of the last step. This stage doesn't require an interpolation and is a worthwhile addition if using refined matching at all.

The refined source-match stages are enabled with SourceMatch = 2 or 3. There is another preset (MatchPreset2) that control their combined speed

QTGMC( Preset="Slower", SourceMatch=3 ) # Default basic preset is 3 settings quicker than "Preset", Refined preset is 2 settings faster again
QTGMC( Preset="Slower", SourceMatch=3, MatchPreset="Slow", MatchPreset2="Medium" ) # Force slower presets for source-match steps


Source-Match and Lossless
At this point we are very close to lossless: half the fields in the output are almost the same as the fields in the original source. The lossless steps simply copy the source fields directly into the output, where they now make a good match to the wholly generated fields. Where they don't quite match there will be residual combing. There is a little vertical cleaning (from original TGMCb2) that helps to clean that away. Artefacts are much rarer and less pronounced than previous versions. However, a different kind of shimmer problem can occur, since now half the lines are binomially smoothed and the other half aren't. The alternative lossless mode allows the injection of the original fields before the final temporal smooth to help mitigate this, at the expense of not being exactly lossless.

[pokazene_maslo]

Hello, first of all thank you for your great work!
I have read your explanations and I'm not sure if I understand them correctly.
I have 2 questions and one suggestion.

How to find out the correct sigma value? By using this QTGMC( Preset="Slow", ShowNoise=6.0, sigma=X) and adjusting sigma value until what is observed?
Is this the command to retain as much detail as possible while removing as much noise as possible?
QTGMC( Preset="Slower", NoiseBypass=1, NoiseRemove=1.0, NoiseRestore=0.0, Sigma=1.5 )
Suggestion: It is hard for an ordinary person to set up all parameters which control noise to get what he needs. What about introducing a "denoise=X" parameter, while X is a float value from interval (for example) <-1, 1> which would adjust TR2, NoiseBypass, NoiseRemove, DetailRestore, NoiseRestore, etc. at once while considering Preset for speed decisions.
So denoise=0 would mean output is as noisy as input, 1 would mean the strongest denoising, -1 would mean strongest noise amplification.
So the end user will get the best results by controlling only 2 parameters: denoise and sigma.

Once again, thanks for a great script!

[-Vit-]

I agree that there are probably too many noise controls now. I know the delicate effect of each, and will use all of them in different situations. But clearly that isn't easy for everyone else.

One simple piece of advice, NoiseRemove can almost always be 1.0. Because you can control the amount of noise removed with Sigma. One less setting to think about.

As to the other settings, there is a complex interaction between the different parts of the algorithm that determines their use. The defaults are a good starting point (use ShowSettings). Yet maybe I can simplify things with presets. I was careful to use presets with the new source-match feature. Let me think about that for noise in a later version.
___

ShowNoise: Adjust Sigma so you see noisy content, but not clean structural elements (edges). We want to make sure NNEDI3 gets everything that it can properly deal with, and that noise bypass gets anything that might be destroyed by the big temporal smooths. It's not an exact science though since NNEDI3 will have a go at any data you send it (and has probably seen noisy training data...), and the temporal smooths are not always destructive. Which is why the NoiseBypass=2 setting is there - send the noise down both routes...
___

Remove maximum noise, but keep maximum detail. An easy task then! Try:
QTGMC( Preset="Slower", NoiseBypass=1, NoiseRemove=1.0, DetailRestore=x, NoiseRestore=0, Sigma=1.5 )
Adjust DetailRestore=x downwards from 1.0 (maybe all the way to 0.0), until you're happy with the noise in the output. I simply cannot give you a value for x because that depends how sensitive you are to noise, which will be different to my sensitivity.

You could also try the same process, but set NoiseBypass=2, and start DetailRestore at 0.5. This might get you more detail.

[Didée]

Wow. That surely was lots of work. Respect!

Any thoughts about making the searchclip/processing better suited for mixed progressive/interlaced sources? :):)

* * * * *

BTW, now that the script is over 1000 lines and comes closer to 100 kB script size, I can't resist to post this - just for fun:

i = last
b = i.bob(.0,.5)
t = b.temporalsoften(1,255,255,25,2).merge(b,0.25)
s1 = t.blur(1.4).blur(1.4).merge(t,0.25).msuper()
s2 = b.msuper(levels=1)
b.mdegrain1(s2,s1.manalyse(isb=true),s1.manalyse(isb=false)).merge(b,0.25)
sharpen(.2,.8).repair(b,1)

The hard core of TGMC. Isn't it cute? ;)

[-Vit-]

Wow. That surely was lots of work.
This version only started as a little experiment a few months ago, but then it kinda snowballed... 1000 lines and half of them are comments! And still to add YUY2 and chroma speed-ups :eek:

Any thoughts about making the searchclip/processing better suited for mixed progressive/interlaced sources?
This does interest me as I have sources like that. Now I assume you mean without using some IsCombed variant, otherwise it's trivial. But then for each frame how do you know whether to bob-interpolate or not? There are [effectively] two frame rates in such sources. There may be a trick to it - I'll give it some thought.

The hard core of TGMC. Isn't it cute? ;)
Shall I add it...? Preset="Cute" :)
____

BTW: Just noticed that I've broken the "Draft" preset. That's not too important so I'll wait to see if there are any other issues before posting a fix.

[Boulder]

It would also be interesting to know whether there are good settings to use with fieldblended sources to get the good frames (output from SRestore, that is) as detailed as possible. The detection clip most likely needs to be treated in a different way from QTGMC.

Thanks for the new version :)

[elguaxo]

it would also be interesting to know whether there are good settings to use with fieldblended sources to get the good frames (output from srestore, that is) as detailed as possible. The detection clip most likely needs to be treated in a different way from qtgmc.

+1 :)

[-Vit-]

It would also be interesting to know whether there are good settings to use with fieldblended sources to get the good frames (output from SRestore, that is) as detailed as possible. The detection clip most likely needs to be treated in a different way from QTGMC.
I've had similar questions by PM. Unfortunately I don't work with fieldblended sources, and have barely used SRestore so I haven't looked at this issue. Maybe I'll get a chance to look at this, but I'm afraid it isn't a priority at the moment.

____

From some questions coming in I realize that despite that mass of detail earlier, I didn't properly address the practical issues of speed and memory. The new features require more memory and CPU time. They require you know how to run demanding scripts in avisynth. The defaults are tweaked for a particular balance of performance and quality, but you can tweak in either direction

Speed
Source-match performs 1 or 2 extra interpolations. So there are up to three interpolations per frame. The interpolators are determined by:
EdiMode="XXX" : the main interpolation, always occurs - nothing new here
MatchEdi="XXX" : Basic source match - for all source-match modes. Defaults to same as main interpolation, change if using a slow choice like "EEDI3+NNEDI3" as your main
MatchEdi2="XXX" : Refined source match - SourceMatch=2,3. Defaults to high speed NNEDI3. Can use "Bob" here with little penalty

Using "Bob" with SourceMatch=2,3 is a good performance boost for those modes:
QTGMC( Preset="Slower", SourceMatch=3, MatchEdi2="Bob" )

When using NNEDIx or EEDIx for source-match the interpolation speed can be tweaked. These default to much faster than the main preset:
MatchPreset="XXX" - typically "Fast" or quicker still
MatchPreset2="XXX" - typically "Very Fast" or quicker
____

Memory + Multi-Threading
Multiple NNEDI3s and complex QTGMC settings like noise bypass use memory. If you run out of memory you often get a crash. Use SetMemoryMax(XXX) at the start of the script to help. You might need higher values, or maybe lower, it depends on the source resolution, settings, SetMTMode use etc. - try 400,600,700,800,1000,1200...

Using multi-threading (SetMTMode) will help speed things up, but if you're on 32-bit avisynth you have a good chance of running out of memory if you go for very high settings, even on SD material, certainly on HD material. Again use SetMemoryMax, limit the number of threads in SetMTMode, and also use the EdiThreads setting of QTGMC - this last point is very important for source-match. 64-bit avisynth makes things easier - you still need to consider the same points, but you will have more latitude (depending on your machine).

An example multi-threading tweak to limit threads and allow a bit more memory, this is just an example, each case is different:

SetMemoryMax(700)
SetMTMode(5,4) # 4 threads for avisynth
WhateverSource(...)
SetMTMode(2)

# Three interpolations when using SourceMatch=2,3. Two defaulting to NNEDI3 here, plus a Bob. Set 2 threads for *each* NNEDI3
QTGMC( Preset="Slower", SourceMatch=3, Lossless=2, MatchEdi2="Bob", EdiThreads=2 )

[pokazene_maslo]

I'm trying to figure out how to use "ShowNoise". Whatever parameters I use, I get only plain grey picture, no detail, no noise. What am I doing wrong?
# Set DAR in encoder to 31 : 17. The following line is for automatic signalling
global MeGUI_darx = 31
global MeGUI_dary = 17
LoadPlugin("D:\Install\meGUIx64\tools\dgindex\DGDecode.dll")
DGDecode_mpeg2source("D:\Download\VTS_01_1.d2v", info=3)
LoadPlugin("D:\Install\meGUIx64\tools\avisynth_plugin\ColorMatrix.dll")
ColorMatrix(hints=true, interlaced=true, threads=0)

QTGMC( Preset="Slow", ShowNoise=8.0, sigma=2)

[Boulder]

I've had similar questions by PM. Unfortunately I don't work with fieldblended sources, and have barely used SRestore so I haven't looked at this issue. Maybe I'll get a chance to look at this, but I'm afraid it isn't a priority at the moment.Would you like to have a sample to look at? The problem with fieldblended material IMO is that the motion is not "real motion". The underlying source is basically always progressive.

[-Vit-]

I'm trying to figure out how to use "ShowNoise".
You need to enable noise bypass for this to work:
# I suggest you explicitly set NoiseDeint - it defaults to "DoubleWeave", "Generate" is best, you can also set "Bob"
QTGMC( Preset="Slow", NoiseBypass=1, ShowNoise=8.0, sigma=2, NoiseDeint="Generate" )
I suppose it should get switched on automatically if you set ShowNoise. I'll add that to the 3.01 version, which I'm using to fix all the inevitable minor issues like this.

____
Would you like to have a sample to look at?
It would be good to have a typical example of the problem from someone who deals with it. Still I don't expect to be able to look at this issue for some time...

[pokazene_maslo]

Heh :) I was assuming that it can set up all needed parameters automatically. Thanks.

Now I have another problem. I was trying to find appropriate sigma but I have found that I need to denoise also chroma planes to get good results. How it is with "plane=4" support? Are there needed any other changes in your script to get "plane=4" support?

[-Vit-]

Heh :) I was assuming that it can set up all needed parameters automatically. Thanks.

Now I have another problem. I was trying to find appropriate sigma but I have found that I need to denoise also chroma planes to get good results. How it is with "plane=4" support? Are there needed any other changes in your script to get "plane=4" support?
Sorry, it currently only denoises luma. That's temporary. There are quite a few changes needed to add chroma denoising, but it's the next version that will be focusing on the chroma planes. Maybe you should denoise as a separate step.

[dansrfe]

n00b question. Ok so I'm slightly confused. Does QTGMC work with fieldblended PAL material (with sresotore traling it) and if not then what other sources can it potentially fix?

[txporter]

n00b question. Ok so I'm slightly confused. Does QTGMC work with fieldblended PAL material (with sresotore traling it) and if not then what other sources can it potentially fix?

It can be used on fieldblended PAL material, but srestore needs to have another clip to detect the blended frames.

I use something like this for converting NTSC Torchwood/Dr Who/Robin Hood/etc DVDs:
setmtmode(5,0)
loadcplugin("C:\program files\avisynth 2.5\plugins\yadif.dll")
o = MPEG2Source("Torchwood.S1E01-clip.d2v",cpu=3)
setmtmode(2)
y = o.yadif(mode=1,order=1)
qtgmc = o.QTGMC(Preset="Ultra Fast",rep2=4,SLmode=1,SLrad=1)
qtgmc.Srestore(frate=25,dclip=y)

[kutjong]

I'm currently trying to repair some material that was sloppily converted from interlaced to progressive using what seems to be weave deinterlacing (lots of combing artifacts).

So far, I'm using this:

QTGMC( Preset="Slower", InputType=2, ProgSADMask = 10.0, SourceMatch=3, Sharpness=0.4, TR2=3 )

This makes substantial repairs, but there are still some combing artifacts in few frames.

Are there any other variables that I should experiment with?
I've read the documentation but embarrasingly most of it is just way over my head. :)

[-Vit-]

I'm currently trying to repair some material that was sloppily converted from interlaced to progressive using what seems to be weave deinterlacing (lots of combing artifacts).

So far, I'm using this:

QTGMC( Preset="Slower", InputType=2, ProgSADMask = 10.0, SourceMatch=3, Sharpness=0.4, TR2=3 )

This makes substantial repairs, but there are still some combing artifacts in few frames.

Are there any other variables that I should experiment with?
I've read the documentation but embarrasingly most of it is just way over my head. :)

That's a good start. Simplest thing to do is lower ProgSADMask, potentially to 0.0.

InputType=2 tries to rebuild a shimmer-free result from the even lines (fields), InputType=3 does the same using the odd lines. Each loses detail from the other lines. The following script runs both modes. It means some redundant processing, but it does improve detail and reduce artefacts:

c = WhateverSource("Your.src")

# Two versions of fixed source (use TR2 and ProgSADMask also if you wish)
t = c.QTGMC( Preset="Slower", InputType=2, SourceMatch=3, Sharpness=0.4 )
b = c.QTGMC( Preset="Slower", InputType=3, SourceMatch=3, Sharpness=0.4 )

# Now merge the results in some way. Choose ONLY ONE of the lines below and delete the other
Merge( t, b ) # Smooth result, may need more sharpening
t.Repair( Merge( t, b ), 1 ) # Sharper result, could try 2 at end of this line
There are almost certainly even better ways to merge the results, this was just a quick idea...

One of the minor fixes I'm sitting on at the moment in version 3.01 is allowing Lossless modes for InputType=2,3. That helps with detail, but I guess that's not your problem.
____

You can do some serious script abuse by chaining together progressive QTGMCs. Generally it's not a good idea because the repeated motion compensations are both redundant and drift in accuracy which causes smearing. You also get gradual over-sharpening. [Although sensible uses like the one above makes me think a toolbox approach might be beneficial]

So another approach for your situation, that I don't recommend but you might want to try, is to run a QTGMC( InputType=1 ) on the result of your QTGMC( InputType=2 ). It will certainly clean up the artefacts, but it may cause smearing / oversharpening:

QTGMC( Preset="Slower", InputType=2, SourceMatch=3, Sharpness=0.4 ) # Plus any other settings
QTGMC( Preset="Slower", InputType=1, Sharpness=0.1 ) # Don't use source-match here

[AnXioZ]

Excuse me for the n00b question but, do I loose any vertical resolution if I decide to keep the framerate of the interlaced source? In example, I have a 1080i60 (60 fields/s | 59.94 frames/s playback) bluray source, and I want to encode it to 1080p60 (59.94 frames/s). In my experiments I haven't seen any noticeable decrease in quality after deinterlacing, but I just wanted to know of the side-effects.
Thank you in advance.

[osgZach]

one field is thrown away, and the data from the other is used /interpolated to keep the same resolution, if I recall correctly. That's how it usually works. What you see is what you get basically... So if you are previewing and choose same rate deinterlacing, and you see the same image size as the source, then that is what you get ;)

[-Vit-]

No, you won't really lose any resolution. Deinterlacing effectively increases vertical resolution.

Put simply: your 1080i60 source is really a 60fps stream of 540 pixel high images (fields). Each pair of fields is interlaced (weaved) into a single 1080 pixel high frame, resulting in a 30fps stream of interlaced frames. Deinterlacing separates the fields again and doubles their height to give a 60fps stream of true 1080 pixel high frames. Compare with the starting point: the vertical resolution is increased.

The new pixels are created by intelligent interpolation of other pixels in the same field and pixels in neighboring fields. No fields are thrown away during this process. If you decide to drop half the frames after deinterlacing to keep the original 30fps rate, that's fine - no quality is lost on those frames you keep.

The original fields in your source are sometimes altered a little in the deinterlaced output to ensure that the result is smooth and shimmer-free. But there should be very little detail loss (especially with the latest features). In fact, the newest developments mean the doubled height output can make a good estimate of lines that are not actually stored in the source. That's what the Source_Comparison example (http://www.mediafire.com/?6kcahljmqizr9mj) was showing in my earlier post (http://forum.doom9.org/showpost.php?p=1474061&postcount=159). This compares a source recorded with 720 pixels of vertical resolution against a QTGMC deinterlacing of 360 pixel high fields (i.e. I deliberately threw away half the pixels and got QTGMC to try and work out what they were)

[AnXioZ]

Thank you for the input guys. Much appreciated.

No, you won't really lose any resolution. Deinterlacing effectively increases vertical resolution.

Put simply: your 1080i60 source is really a 60fps stream of 540 pixel high images (fields). Each pair of fields is interlaced (weaved) into a single 1080 pixel high frame, resulting in a 30fps stream of interlaced frames. Deinterlacing separates the fields again and doubles their height to give a 60fps stream of true 1080 pixel high frames. Compare with the starting point: the vertical resolution is increased.

The new pixels are created by intelligent interpolation of other pixels in the same field and pixels in neighboring fields. No fields are thrown away during this process. If you decide to drop half the frames after deinterlacing to keep the original 30fps rate, that's fine - no quality is lost on those frames you keep.

The original fields in your source are sometimes altered a little in the deinterlaced output to ensure that the result is smooth and shimmer-free. But there should be very little detail loss (especially with the latest features). In fact, the newest developments mean the doubled height output can make a good estimate of lines that are not actually stored in the source. That's what the Source_Comparison example (http://www.mediafire.com/?6kcahljmqizr9mj) was showing in my earlier post (http://forum.doom9.org/showpost.php?p=1474061&postcount=159). This compares a source recorded with 720 pixels of vertical resolution against a QTGMC deinterlacing of 360 pixel high fields (i.e. I deliberately threw away half the pixels and got QTGMC to try and work out what they were)

Thank you for the great expiation -Vit-, and thank you for the great work!

[AnXioZ]

Hey -Vit-, I just installed the new 3.0 version. I am still playing around with it, but one thing I noticed was that when I set SourceMatch=3 the CPU utilization drops significantly. Without it I'm getting ~98% utilization, with SourceMatch I'm getting ~70%. Am I doing something wrong or is that expected with the new enchantments?

[-Vit-]

CPU under-utilization often means that there are cores idle on a multi-core machine. E.g. One core is busy running a key step in the script, another core is trying to run a task that needs the result - if there are no other tasks to do then the second core idles (it's rather more intricate than that, but you get the idea). Run more threads, give your cores more to do. Run multi-threaded using SetMTMode, or increase the thread count if you're using it already. I posted some tips on that a few posts ago in this thread. Also NNEDI3 runs two or three times with source-match and NNEDI3 defaults to running multi-threaded. You can tweak its thread usage with EdiThreads=x. Try tweaking x, it might help (or not).

[osgZach]

Silly question if you don't mind. If I feed it a progressive clip, is it going to make a "this is combed" decision and skip frames, or is it going to process every single frame in the clip?

There is a lot of material to read through, so I'm sorry if this was asked/answered before. I'm generally interested in using it to clean up IVTC'd material where the stream result is progressive obviously, but sporadic combing will occur. I could see using this for Clip2 operations, but if it doesn't do "needless" processing where there is no combing, it may be easier to just throw it on at the end of the script in some cases.


edit: ok, its runs about 15 fps using "QTGMC( Preset="Medium", InputType=1 )" which is realistically fine I guess.. It seems to be pretty much identical to the source. So ignoring the fact I am deliberately slowing my script down more than I -need- to. If I wanted to use TFM and set PP to 0 (don't hint, don't attempt to deinterlace) and then just tack QTGMC on at the end, to catch and fix any combing (surprisingly I couldn't find any real combing but I used the most intensive match mode so thats probably why.) It would be a valid and correct usage, right? I just hate tweaking thresholds and all that stuff so I'd take a time increase vs having to tweak per file all the time.

I'm sure I could adjust a few things to make it go faster and not ruin the quality, too.

[-Vit-]

The script makes no attempt to detect combing. It runs a temporal binomial smooth on every frame to remove bob-shimmer when deinterlacing. This same process is also effective in remove deinterlacing residue from progressive video. But it isn't selective on which frames to process. I've seen others run a QTGMC in your situation, but it's not a workflow I need so I can't really comment further.

[osgZach]

Fair enough. If anyone else has been using it for this purpose, I'd be interested to hear their take as well.

[Didée]

What kind of "residual" combing? Low-value combing (from mpeg compression, because the matched fields come from different coded frames)? Or fully interlaced frames because of lone fields that couldn't be matched?

Because - when I hear (or see) "residual combing", my first thought is "vinverse". Not "QTGMC". ...

[osgZach]

Oh jeezus Didee, you scared me for a second. I thought I said "residual combing" somewhere and used the wrong term.. Bad Didee!

To Clarify: I am simply talking about combing left in frames either from doing no post-processing whatsoever (TFM PP=0), or combing that was not picked up after field matching for whatever reason (Mic below threshold, etc).. I'm not a "tweaker" at heart, I prefer "run it once and its done" operations, I guess. So I was just inquiring if it were a proper use or not. McBob/TGMC are just too slow for me, but QTGMC seems to be fast enough I don't mind waiting as much.

IIRC AnimeIVTC uses Vinverse doesn't it? I have always been curious about it but its options have always been a little intimidating (no offense). And seems like "more tweaking" for this "non-tweaker at heart" :p

[maki]

What settings should I use to decomb and deinterlace this (http://www.megaupload.com/?d=O8YWB61Q) progressive content. I also want to remove noise/dirt artifacts that appear around the lips, that's pretty much the only place where they appear.

I noticed that deinterlace/jaggy lines and noise only appear when I downsize the video to 720p.

I am new to the whole decoming and deinterlace in a video. And I don't really understand the difference between jagginess and deinterlace. Can someone explain it to me. I've read about it but I don't really get it.

Also I applied this plugin to the video and it seems that coming appears in every frame:

http://img15.imageshack.us/i/piczb.png/

AVCSource("C:\00007.track_4113.dga")
FieldDeinterlace(full=true, map=true)
LanczosResize(1280,720) # Lanczos (Sharp)
Trim(29560, 31846)
#deinterlace
#crop
#denoise
ConvertToYV12()

Can someone give me some nice settings to fix all of the above.

[Fullmetal Encoder]

Has anyone had any success getting the Very Slow preset, with or without any variations, to work with any speed/stability using 3.0 on a Core i7+ system? At this point I've run 44 different tests in which I've altered SetMTMode and SetMemoryMax and every test produces either less than 1fps output (usually much less) and/or crashes. Varying EdiThreads doesn't seem to have any effect. The basic script I am using is:


SetMemoryMax(700)
SetMTMode(5,4)
Import("C:\QTGMC3.0.avsi")
LoadCPlugin("F:\ffmpegsource-2.14-mt-avs64\ffms2.dll")
FFVideoSource("D:\X.mkv")
SetMTMode(2)
Trim(1753, 3542)
QTGMC(Preset="Very Slow", MatchPreset="Medium", SourceMatch=3, Lossless=2, MatchEnhance=0.6, Sharpness=0.25, NoiseBypass=2, NoiseRemove=1.0, DetailRestore=0.5, NoiseRestore=0.1, Sigma=3, NoiseDeint="Generate")

The MKV I'm using as a source is an interlaced 1080P clip. I realize the QTGMC settings I am using are rather aggressive but they are an example that I believe -Vit- posted previously. What I am trying to accomplish is to determine a set of very high quality, nearly lossless settings which are still functional, to then tweak the Avisynth settings to see how high I can push output performance, and then to do some quality comparison tests on various clips to determine whether it's worth it to stick with those settings or reduce quality in exchange for speed. I am using JoshyD's (04/16/2010) Avisynth on a Core i7 860 with Windows 7 Professional 64-bit and 8GB of RAM.

Using similar QTGMC (slow, very slow) settings which Avisynth settings give you the best performance? What kind of system are you running? So far, all I can conclude is that it's either impossible to run the above QTGMC settings on any modern system or that there is something seriously wrong with my set-up.

[Didée]

Way too complex for 1080 video. Are you even halfway aware of the amount & kind of operations that QTGMC is doing with these settings?

Running single-threaded, SetMemoryMax(1200) gives me 0.25 fps. SetMemoryMax(1400) gives 1.1 fps, however Vdub now has peaked at 1.85 GB. SetMemoryMax(1440) loads, but crashes upon seeking. Of course, such memory load doesn't allow for e.g. direct x264 encoding. I don't get the script to run using 4 Threads via SetMTmode(2), even setmmax(500) throws kernelbase after a few frames.

Way too complex for 1080 video. And breaking a fly on a wheel, anyway.

[-Vit-]

I designed the settings to make a steady gradient from "as fast as possible" to "unnecessarily accurate". The slowest settings are all about ultimate precision and completely ignore speed as a factor. Also, none of the presets take any account of resolution, so you need to. Settings like yours are visually great for big upscales on SD, but unless you have a huge display, you simply won't see the difference at the very slowest settings at HD resolutions.

If you must get it to run, a 64-bit workflow is required (check) and with regards threading work upwards from 1 main thread + 1 EdiThread. I find having EdiThreads lower than main threads helps on my system (I use a 12Gb i7 930 and a similar 950). Render lossless, do your x264 in a second pass.

Regarding tweaks other than the presets, you can get a speed boost from MatchEdi2="Bob". Also NoiseDeint="DoubleWeave" will be faster and might make little difference, depending on your source. Make sure your SetMemoryMax is tweaked - wrong setting there can slow things down. Check the presets table in the script for other ideas.

From my experience, the stability problems come from three points:
(a) Memory issues: 32-bit avisynth running out of memory (overeager / buggy caching?), the black art of SetMemoryMax, leaky plugins - thankfully these issues are less noticable on 64-bit
(b) Buggy plugins: many plugins inherit threading issues from avisynth (see below), but some have other bugs too. At this very moment I'm rewriting AddGrainC from scratch because I think it is causing crashes for me, I can almost guarantee a crash if I try very complex HD scripts with NoiseDeint="Generate" (uses AddGrain), and I don't get them without. This may be your problem. Try setting NoiseDeint="DoubleWeave" or NoiseBypass=0.
(c) Avisynth multithreading. In early experiments with QTGMC I had serious problems with crashes on 1080p even at medium settings. Traced the problem to the avisynth<->plugin threading (synchronization problems with ref counts usually). SEt has fixed some of these in his MT avisynth builds, but they affect "avisynth.h" which needs to be compiled into all the QTGMC plugins. I've done that with all the 32-bit versions and also added a hideous hack of my own, and it helped a lot. However, I still need to resolve a few issues with the 64-bit versions and test a bit more before I could post them (all VS2010 and IC2011 - some conversion issues need checking).

So even if you can get a speed you can live with, it's certain that as your resolution and script complexity increases, so does your chance of avisynth instability. I have had to tone down scripts from the level I wanted because of this in the past.

[Fullmetal Encoder]

Way too complex for 1080 video. Are you even halfway aware of the amount & kind of operations that QTGMC is doing with these settings?

Running single-threaded, SetMemoryMax(1200) gives me 0.25 fps. SetMemoryMax(1400) gives 1.1 fps, however Vdub now has peaked at 1.85 GB. SetMemoryMax(1440) loads, but crashes upon seeking. Of course, such memory load doesn't allow for e.g. direct x264 encoding. I don't get the script to run using 4 Threads via SetMTmode(2), even setmmax(500) throws kernelbase after a few frames.

Way too complex for 1080 video. And breaking a fly on a wheel, anyway.

Ok, thanks for the feedback. Yes, I know it is quite complicated and while I understand some of the basics of what it's doing I am not aware of its intricacies, especially when it comes to its use of mvtools and masktools, so probably, no :) Just wanted to get some input from others before proceeding with any other testing in case it would end up being a waste of time. And throughout my testing I have started to get the impression that what I was attempting to do was really to kill a mosquito with a bazooka.

The more I read up on this plugin though the more impressed I am with what it's able to do so thanks to -Vit- for his work on it and to you Didée for the underlying TGMC.

[Didée]

The "most important" application IMHO is on SD content, anyway. When SD content is cheaply bob/deinterlaced and viewed on a big screen, then you'll see the downsides from bad deinterlacing in all glory, because the upscale makes everything more objectable.
On 1080i, this is so much less of concern, because no upscale is taking place. Even the "simple" deinterlacers result in an acceptable viewing experience. If I personally use Q/TGMC on 1080i, then it surely is with pretty fast settings. Even the most simplistic settings give like, say, 80% of what Q/TGMC can achieve. And those 80% are already way ahead of what other deinterlacers can achieve. See this older sample (http://www.mediafire.com/?s23960xpv9wz318) from 1 year back: old TGMC with most-fast and most-basic settings, versus YadifMod with NNEDI2 interpolation. TGMC was much faster, still it let Yadifmod/NNEDI2 stand with egg in their face.

[TheSkiller]

Thanks for the new version, just tried it - looks great with the new SourceMatch on my MiniDV stuff.
But with another type of source that requires rep0=0 there is something odd I noticed: whenever I use any of the lossless modes, especially without SourceMatch (SourceMatch=0) I pretty much always have combing left in the output. With version 2.51 this was not the case (version 2.51's fake lossless "Lossless=1" was free of remaining combs for me).

I also find the fake lossless mode to have a lot more remaining combing than the true lossless mode Lossless=1 (which doesn't make sense), even with the default rep0=4.


Edit: Sorry, forgot to add: I'm using rep0=0 because otherwise static parts (Text) laying on top of the motion would have an ugly up and down bobbing occasionally on this source (is there a better way to fix this?). The described problem with combing never occurs with rep>0, and never with QTGMC 2.51.

Edit: cleaned my post, got messed up somehow.

QTGMC(2,2,1, 0,0,4, Lossless=2, SourceMatch=0) #lots of combs (but steady text)

[STC-Fan]

Tried this out today with one of my PAL VHS captures - really impressed with it. Using QTGMC with just the "Slow" preset has produced a massive improvement when compared with my current filter choice in AviSynth, which is MDegrain2i2 + Cnr2k + Yadif.

My only problem now is that I get a Visual C++ runtime error along the lines of "This application has requested the Runtime to terminate it in an unusual way", which happens after a short while when I try to do a 2-pass encode with x264 in MeGUI, which I am guessing is down to the program running out of memory.

I've only tried SetMemoryMax() with a value of 512 so far, and that hasn't resolved the problem. Presumably, I will need to increase the value to 1024 or more, and close Firefox when the encode is running along with any other programs.

Before it crashed, I was getting around 5.6 FPS on the first pass, which is fairly reasonable. I'm a bit surprised that even with SetMTMode(2) that I can't get more on a Core 2 Quad Q6600 - I suppose I could try a faster mode in QTGMC, or get myself a computer with a bit more power; I'm thinking along the lines of something from Cray, to be honest :P

I put up some comparison screenshots and sample scripts - the best thing is the reduction of jagged edges on the graphics, which the old trio of filters couldn't really do much about:

Original frame (http://stc.hacking-cult.org/videos/avisynth/bbc1_advert_original.png) | AviSynth script (http://stc.hacking-cult.org/videos/avisynth/bbc1_advert_original.avs)

MDegrain2i2 + Cnr2k + Yadif (http://stc.hacking-cult.org/videos/avisynth/bbc1_advert_mdegrain2i2+cnr2k+yadif.png) | AviSynth script (http://stc.hacking-cult.org/videos/avisynth/bbc1_advert_mdegrain2i2+cnr2k+yadif.avs)

QGTMC 3.0, Slow preset (http://stc.hacking-cult.org/videos/avisynth/bbc1_advert_qtgmc3-slow.png) | AviSynth script (http://stc.hacking-cult.org/videos/avisynth/bbc1_advert_qtgmc3-slow.avs)

Overall, though, it's a very good script, this - I think that QTGMC3 could even be better than Cillit Bang / Easy-Off BAM :)

[-Vit-]

...there is something odd I noticed: whenever I use any of the lossless modes, especially without SourceMatch (SourceMatch=0) I pretty much always have combing left in the output.
Some of what you say doesn't surprise me - it's a very intricate script now and I may have overlooked some combinations of settings. However, I don't get more combing with "fake" lossless than "true", the only difference between those modes is when they occur in the script, before or after the final smooth. Still, I'll have a look at the rep modes and see what I find...

Visual C++ runtime error
I listed the range of things that causes stability issues a couple of posts above. I also posted some notes on performance and MT use (http://forum.doom9.org/showthread.php?p=1474231#post1474231). One other important thing I note in the OP is using the right version of AviSynth. I can only get extended stability with SEt's 2.6MT (http://forum.doom9.org/showthread.php?t=148782). Make sure all your other plugins are up to date too. A SetMemoryMax of 512 should be OK for Preset="Slow" alone (provided your overall RAM is not low), but maybe tweaking it might help. Sometimes it needs to be lowered.

If you do get it stable, try tweaking the thread-count as described in that overview I mentioned. You may get better speeds with more or less threads.

[matfra]

I have a basic question for you.
MT("""
QTGMC(tr0=0,tr1=1,tr2=0,EdiMode="nnedi3",rep0=4,rep1=0,rep2=4,sharpness=1.00)
""",threads=4, overlap=4)

Is it the best way to run this script with MT ?
Also what is SetMemoryMax(). Does it really speed up the encoding ?
Merci !

[matfra]

Im wondering if its better to apply some denoising and sharpening in with QTGMC, or if its better to apply a simple Deinterlaced, then denoise and sharpen after.

What do you recommand guys ?

[Boulder]

I listed the range of things that causes stability issues a couple of posts above. I also posted some notes on performance and MT use (http://forum.doom9.org/showthread.php?p=1474231#post1474231). One other important thing I note in the OP is using the right version of AviSynth. I can only get extended stability with SEt's 2.6MT (http://forum.doom9.org/showthread.php?t=148782). Make sure all your other plugins are up to date too.For some reason, I've had much better stability using tsp's last MT build of Avisynth 2.5.7. For two weeks I played around with SEt's two builds but was never able to finish an x264 encode or encode to lossless intemediate file without Avisynth crashing. (For what it's worth, I had a script with Deblock_QED + MCTemporalDenoise in it).

[-Vit-]

@matfra

You should not use MT("...") with QTGMC, you should use SetMTMode(...). SetMemoryMax is not a speed-up generally - it sets a balance of memory for your script to work - you might not need it.
Here is a template script, adjust it as you need:

SetMemoryMax(700) # This line may not be necessary - depends on your settings, number of threads and available memory. Try 400,600,700,800,1000,1200...
SetMTMode( 5, 4 ) # Use 4 threads, adjust to suit your machine
WhateverSource(...) # FFVideoSource, DGDecode_mpeg2source, AviSource, whatever your source requires
SetMTMode(2)

QTGMC( ... , EdiThreads=2 ) # Set number of threads for NNEDI3 (more or less than setting above), add any other settings you require


QTGMC provides some sharpening and denoising control, but only you can decide if it is good enough, or if other processing is needed. In other words, try it both ways and decide for yourself which is better.

[ajp_anton]

Is it a good idea to merge this with MCTemporalDenoise into one single giant function so that MCTD can re-use the motion vectors from QTGMC?

[STC-Fan]

I listed the range of things that causes stability issues a couple of posts above. I also posted some notes on performance and MT use (http://forum.doom9.org/showthread.php?p=1474231#post1474231). One other important thing I note in the OP is using the right version of AviSynth. I can only get extended stability with SEt's 2.6MT (http://forum.doom9.org/showthread.php?t=148782)

Thanks for that - I've actually been able to resolve the biggest issue (slow speed encoding leading to a crash) by reverting back to using MT mode 5 before AVISource, then using MT mode 2 for QTGMC and other plugins. In fact, I'm now getting nearly 12 FPS for first and second passes - not 16 or 20 like with MDegrain2i2, but the payoff in restored detail and better correction of interlacing artifacts etc. is well worth it :)

As for the memory running out - on the last script I ran for a 6-minute video, I increased SetMemoryMax to 768 and this has eliminated crashes for the time being. However, I have recently had BSOD crashes with the MEMORY_MANAGEMENT error on Vista (SP2), so I might need to run the memory checker and possibly replace the RAM I have.

The other filter I sometimes use is fizick's DeFreq for removing RF (diagonal line) noise, so I'll check and see if that will work harmoniously with QTGMC.

EDIT: poisondeathray - my source video is all TFF, I have tested your suggestion and the 50 FPS output from QTGMC is now nice and smooth, thanks for the tip! The equivalent settings I used for Yadif were mode=1 (double framerate) and order=1 (top field first).

I've also tested the output on some more noisy VHS clips which need DeFreq (placed before QTGMC) and they looks very nice as well. With those clips (thankfully only a few) there is still a small amount of background noise crawl and feint horizontal lines, although in the final output from x264 much of this isn't really visible anyway. Probably I could mess around a bit with QTGMC's parameters in detail, beyond the standard Slow preset, but I'm more than satisfied with the results at the moment :)

UPDATE: I've had a look at some stronger denoising to go with the detail restoration / deinterlacing from QTGMC. I've found that for my VHS content, I can get better results by using the MDegrain2i2 function before QTGMC(Preset="Slow") - normally I would just use one or the other. With both of these in one script, they work extremely well together, though it does hit encoding time to x264 somewhat (down to about 6.5 FPS instead of 12 FPS with just QTGMC @ Slow). Despite this, the results are definitely worth it. I have uploaded some standard size screenshots, 4x magnified sections of those screenshots and my most recent AviSynth scripts here:

QTGMC + MDegrain2i2, QTGMC alone and MDegrain2i2 alone (http://stc.hacking-cult.org/videos/avisynth/AviSynth-QTGMC_vs_MDegrain2i2_06-11-2011.7z)

[poisondeathray]

One other question - I have used QTGMC to output in standard PAL (25) and doubled (50) frame rates, but in the 50 FPS mode the output appears to be choppy, as compared with the smoother 50 FPS output produced by Yadif. I will try and upload some sample video later on to demonstrate this.

Do you have the correct field order? What settings did you use for yadif ?

Try AssumeTFF() preceding QTGMC() if the field order is TFF

[matfra]

Thank you -Vit-.
I tried this
SetMTMode( 2,4 )
But with this CPU is at 20%. When I do SetMTMode( 2,32)
The CPU cap 100%. Is this possible or im a total noobie.

[-Vit-]

@matfra: Yes you can increase that number, 4 was just an example. I increase that value until my CPU reaches about 97-99%. Going too high can slow things down again.

____

@ajp_anton: I have also been considering how to remove redundant motion analysis from more complex scripts. I would favor a method where different sub-scripts remained separate but were able to share any available data, as this would allow more flexibility (e.g. sharing settings). But there are no good language constructs for that (lots of globals, I suppose...). Merging monster scripts sounds unpleasant, although it might not be so bad in practice.

[Fullmetal Encoder]

The "most important" application IMHO is on SD content, anyway. When SD content is cheaply bob/deinterlaced and viewed on a big screen, then you'll see the downsides from bad deinterlacing in all glory, because the upscale makes everything more objectable.

Yes, I was beginning to reach this conclusion on my own. After all, most of the artifacts in a video should come from what we've done to the video (deinterlacing, resizing, manipulation to address compression artifacts) and shouldn't be in the source, especially since I'm dealing with a Blu-ray. At the encoding stage I'm going to targeting about 1GB per each 22m episode anyway so I should have plenty of bits to throw at x264. Since my last post I've been putting QTGMC through the ropes on some truly nasty SD material with which it's doing wonders. Incidentally, after playing with the settings, I've been able to achieve an average of over 27fps using the "Very Slow" preset on this DVD using this script:


SetMTMode(2,10)
SetMemoryMax(700)
Import("QTGMC3.0.avsi")
LoadCPlugin("\ffms2-mt-r430-avs64-2.15\ffms2.dll")
FFVideoSource("VTS_01_2 - 1 - MPEG2, 480i60.m2v")
Trim(4034, 7893)
QTGMC(Preset="Very Slow")

This is on a Core i7 860 with 8GB of RAM if anyone is interested.

-Vit-:
I am currently trying to learn how to use FFT3DFilter more effectively to remove artifacts. There are many, many options for FFT3DFilter which are not exposed or explicitly handled by QTGMC, at least as far as my reading of the documentation has revealed. The plugin seems to be called at only 1 line in the QTGMC script. Can I manually edit the script and input those options for testing purposes where FFT3DFilter is called? Or will that in some other way disrupt QTGMC's output? I would like to bend FFT3DFilter to my will as it is run by QTGMC instead of calling it in a separate, second pass after QTGMC has finished with the video. Also, am I correct in thinking that "plane=4" does not at this time allow for any processing in the chroma planes?

[-Vit-]

I am currently trying to learn how to use FFT3DFilter more effectively to remove artifacts. There are many, many options for FFT3DFilter which are not exposed or explicitly handled by QTGMC, at least as far as my reading of the documentation has revealed. The plugin seems to be called at only 1 line in the QTGMC script. Can I manually edit the script and input those options for testing purposes where FFT3DFilter is called? Or will that in some other way disrupt QTGMC's output? I would like to bend FFT3DFilter to my will as it is run by QTGMC instead of calling it in a separate, second pass after QTGMC has finished with the video. Also, am I correct in thinking that "plane=4" does not at this time allow for any processing in the chroma planes?

Yes, abuse the script as you wish. As you say there is only one line that calls FFT3DFilter and it's not going to affect any other features. Any noise you remove is going to be added back in unless you set the xxxRestore settings to 0 though.

It is only dealing with luma noise at the moment - that's temporary and will change in a later version. It would require a number of changes throughout the script to bypass the chroma noise, but you could remove it by tweaking the FFT3DFilter line.

I toyed with adding more of the FFT3DFilter settings, but QTGMC was not originally supposed to be a denoiser and there are already an excess of settings. However, I now see the importance of treating "noise" correctly when recovering exact detail so I am likely to rework the noise related-settings.

[matfra]

I would like to thank you for this script. Its the more useful and managable script ever. I have a last question regarding the SetMTMode(2,4). I assume that QTGMC is working great with SetMTMode(2). I would like to apply a strong denoiser after it. Im looking for a good denoiser that can be run alors in SetMTMode(2). Do you have any suggestion ? Ill post you a script that im using. Can you suggest me something else that can be faster or better or maybe a other way to do it.
Thank you in advance.

AVISource("C:\Users\7\Documents\AvsPmod 2.0.5\VirtualDub-1.9.10\Dub\Templates\1.vdr.avi")
import("C:\Program Files (x86)\AviSynth 2.5\plugins\AA.avs")
import("C:\Program Files (x86)\AviSynth 2.5\plugins\MDegrain5.avs")
import("C:\Program Files (x86)\AviSynth 2.5\plugins\SeeSaw.v0.3e.avs")
ConverttoYV12()
MT("""QTGMC(Preset="fast",EdiMode="nnedi3",tr0=0,tr1=1,tr2=0,rep0=4,rep1=0,rep2=4,sharpness=1.00)""",threads=4, overlap=8)
AssumeTFF()
SelectEven()
MT("""
HDRAGC(avg_lum=0,max_gain=3.0,min_gain=1.2,coef_gain=1.0,max_sat=9.0,min_sat=0.0,coef_sat=1.3,shift=0,shift_u=-10,shift_v=6,protect=0,passes=4,corrector=0.0,reducer=0.0,shadows=false,avg_window=1,response=100,debug=0,freezer=-1)
""",threads=4, overlap=8)
MT("""
source1 = last
sharp1 = source1.Removegrain(2,2).SeeSaw(sLimit=100).mergechroma(source1)
super1 = source1.MSuper(pel=2,sharp=2)
super2 = sharp1.MSuper(pel=2,sharp=2,levels=1)
bvec1 = super1.MAnalyse(isb=true, delta=1,blksize=16, overlap=8)
fvec1 = super1.MAnalyse(isb=false,delta=1,blksize=16, overlap=8)
bvec2 = super1.MAnalyse(isb=true, delta=2,blksize=16, overlap=8)
fvec2 = super1.MAnalyse(isb=false,delta=2,blksize=16, overlap=8)
source1.MDegrain2(super2,bvec1,fvec1,bvec2,fvec2,thSAD=300)
""",threads=4, overlap=8)
MT("""
source2 = last
sharp2 = source2.SeeSaw(sLimit=100).mergechroma(source2).AA()
super3 = source2.MSuper(pel=2,sharp=2)
super4 = sharp2.MSuper(pel=2,sharp=2,levels=1)
bvec3 = super3.MAnalyse(isb=true, delta=1,blksize=16, overlap=8)
fvec3 = super3.MAnalyse(isb=false,delta=1,blksize=16, overlap=8)
bvec4 = super3.MAnalyse(isb=true, delta=2,blksize=16, overlap=8)
fvec4 = super3.MAnalyse(isb=false,delta=2,blksize=16, overlap=8)
source2.MDegrain2(super4,bvec3,fvec3,bvec4,fvec4,thSAD=300)
""",threads=4, overlap=8)

[-Vit-]

- You say you are using SetMTMode, but the script you have posted is using MT(...). MT shouldn't be used with QTGMC or MVTools2.
- The settings TR0=0, TR1=1, TR2=0 will be very fast, but will potentially give a very poor motion analysis. What are you trying to achieve with those settings?
- I'll let others comment on alternative denoisers, but you could try setting "TR2=2 "or "TR2=3" in QTGMC. Or instead you could add "NoiseBypass=1, NoiseRestore=0". Or do both.

[matfra]

I know in this script I used MT(). This was my previous script. Im using SetMTMode now. Im terms of QTGMC settings, I was trying to use the fastest as possible, and apply denoising and sharpening after. Maybe you can suggest me other settings I could use instead of this.
QTGMC(Preset="fast",EdiMode="nnedi3",tr0=0,tr1=1,tr2=0,rep0=4,rep1=0,rep2=4,sharpness=1.00)
Also, what other denoising fonction I could use in SetMTMode 2 that is doing a good work. I tried to chain 2x MDegrain2.
Thank you

[Didée]

As far as the "inherent" denoising of the TGMC-method is concerned ... keep in mind that the repair-settings (rep1, and evenmore rep2) are counteracting with the denoising. They will mostly allow denoising of high-frequencies, but will mostly prohibit denoising of low spatial frequencies. Hence, if you actually want Q/TGMC to (strongly) denoise, set tr2=2 or 3, and set rep2=0.

Then, even when going for "fast" settings, tr0 should always be "1" at least, but never "0". When you set tr0=0, then the whole Q/TGMC thingy starts making no sense anymore. The bare minimum for Q/TGMC to work as intended is tr0=1,tr1=1.

Regarding the last script posted ... seeing you're using SeeSaw+MDegrain, and even two instances of that, I'm not quite sure what is the main preference? The sharpening, or the denoising? In any case, if the SeeSaw-sharpening plays a significant role, then ... rather try to call it with "sootheT=0". Temporal soothing makes not much sense when you combine SeeSaw with mocomp-denoising.

[matfra]

Hi guys. I did lot of test with your suggestion. I manage to get a very nice output. But I can only run the script without MT. As soon I use MT command, there is strong Ghosting in the clip. I tried many setting but the Ghosting is always there with MT. I get only 4FPS without MT and about 24FPS with MT. Can you inspect my code and tell me if I do something Wrong.
Thank you

SetMTMode(5,0)
SetMemoryMax(400)
AVISource("C:\Users\7\Documents\AvsPmod 2.0.5\VirtualDub-1.9.10\Dub\Templates\1.vdr.avi")
import("C:\Program Files (x86)\AviSynth 2.5\plugins\MDegrain5.avs")
import("C:\Program Files (x86)\AviSynth 2.5\plugins\SeeSaw.v0.3e.avs")
ConverttoYV12()
SetMTMode(2,6)
QTGMC(SourceMatch=2,EdiMode="nnedi3",tr0=2,tr1=0,tr2=3,rep0=0,rep1=0,rep2=0,Sharpness=1.00,MatchEnhance=1.0,NoiseBypass=0, NoiseRemove=1.0, DetailRestore=0.0, NoiseRestore=0.0, Sigma=3, NoiseDeint="generate")
AssumeTFF()
SelectEven()

[Boulder]

If you use SetMTMode(5,0), your whole script will utilize the same amount of threads as there are logical or physical CPUs detected. So if you want to use 6 threads, you have to use SetMTMode(5,6).

[-Vit-]

As you have switched off all the rep modes you will likely get unwanted motion effects (your 'ghosting').

[ajp_anton]

Line 522:
MatchEdi = EdiMode
This destroys MatchEdi since it's not used before this.

[-Vit-]

Bug

Thanks, fixed - it just needed a 'default'. I've taken this opportunity to update the OP to version 3.01, which is a few tweaks and bug-fixes. The changelog:
# v3.01: Support for lossless modes on InputType=2,3 - improves detail retention
# Enabling ShowNoise automatically switches on NoiseBypass
# Bug fixes: "Draft" mode, noise extraction with InputType=1, MatchEdi setting


Only feature addition is support for lossless modes on InputType 2,3. Usual caveats about lossless modes and potential artefacts, but as these are repair modes that can lose detail it actually helps to use lossless (especially Lossless=2 and especially in conjunction with source-match).

Haven't had time to check on the reports of issues with switching off rep modes whilst using lossless.

[ajp_anton]

Line 541:
[...] default( MatchEdi, ((mpNum1 < 9) ? EdiMode : "Yadif") )
MatchEdi is always defined at this point (the old "line 522", now 527), so the mpNum1 thing is never checked.

[-Vit-]

Ack... that's where the 'default' was... Note to self: never do insta-fixes

Well, it won't cause much harm. I'll fix it in the next revision.

[matfra]

If you use SetMTMode(5,0), your whole script will utilize the same amount of threads as there are logical or physical CPUs detected. So if you want to use 6 threads, you have to use SetMTMode(5,6).

Thank for replying -Vit-

What rep setting should I instead of rep0=0,rep1=0,rep2=0 to prevent the ghosting in multithread script.
Thanks

[matfra]

I tried all kind of settings. I change the sourcematch to 1,2,3. I tried adding some rep. rep4=0,rep1=0,rep2=4. I even tried exagerated settings rep100=0,rep1=100,rep2=100. There is always some ghosting in the motion as soon its on MT. Any Idea what I can try

[dragon_warrior]

Hi
I have a problem: the fps increases dramatically after I applied QTGMC with the clip loaded using DirectShowSource : 29.97 -> 2774.69
Anyway, if I replace DirectShowSource with AVCSource, it doubled, from 29.97 -> 59.94

Any suggestions?

*edit* DGAVCIndex says the frame structure is MBAFF, not TFF or BFF. What script can I put right before QTGMC(..)?

[Emulgator]

dragon_warrior:
If your source is AVC with MBAFF, then you might use DGAVCDecDI (now in version 2025) (15$) + DiAVC Decoder (now in version 1.2.2) (9,99$).
DiAVC 1.2.2 can handle MBAFF properly,
The sourcefilter call is like: DGSource("C:\source.dgi")

Or if you happen to have a recent nVidia Graphics card, then you may use DGDecNV (now in version 2038) (the same 15$ as above)

DGAVCIndex 1.0.9 (now depreciated) relies on an older libavcodec and can not decode MBAFF properly.

[-Vit-]

@matfra: Your source filter might be the issue. Try using FFVideoSource (or DirectShowSource or DSS2). Also just use QTGMC(Preset="Super Fast") for testing.

[STC-Fan]

After getting over the crash problems, I have gone through several of my VHS captures again with no issues at all. However, in one of the captured videos, QTGMC seems to be adding in diagonal line noise not present in the source video, which happens if I use any of the presets below the "Faster" setting (I am currently using the "Slow" preset for all my videos). For the "Faster" preset and above, the diagonal lines are still present, but only appear in a few frames of the output video.

My script is also making use of MDegrain2i2 before QTGMC is applied, however commenting out MDegrain2i2 still results in QTGMC adding in diagonal lines.

I have uploaded a 50-frame clip of my source material (Lagarith), and two clips compressed to x264, one of which was deinterlaced with QTGMC(preset="Slow"), and one which was not deinterlaced at all - oh, and some still images:

QTGMC line noise (still images, 2 MB) (http://stc.hacking-cult.org/videos/avisynth/QTGMC_diag_line_noise_images.zip)
QTGMC line noise (video clips, 16 MB) (http://stc.hacking-cult.org/videos/avisynth/QTGMC_diag_line_noise_video.zip)

For reference, I am working in MeGUI build 1934 and have AviSynth 2.5.8.5 (SEt's MT build) installed.

[marcopolo80]

Hi,I'm trying to re-encode some personal footage shoot in HDV with an Canon HV10 camera.My intention is to keep them in HQ HD and I have 2 choice:
1.keep 1440x1080-60i @12Mbit-encoding speed is a litle slower-big space amount(`1.5h for an DVD-9)
2.go to 1280x720p@6-8Mbit-speed is better space the same.
So I was trying to encode with few software but I have an error when I try to use QTGMC(hear to be the best deinterlacer).

11998

I Can use Yadif in Bob mode without any error but I want to choose QTGMC because it is used also by other scripts(like SimpleSlugUpscale-for some old DV to upscale to 720 with best deinterlace).

My plug-in director look like this:
11999

My script is:

LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\DGDecode.dll")
MPEG2Source("C:\test5min.d2v")
AssumeTFF()
ConvertToYV12(interlaced=true)
Import("C:\Program Files\AviSynth 2.5\plugins\QTGMC3.0.avsi")
QTGMC( Preset="Slow" )
ConvertToYV12()
spline36resize(1280,720)


Can someone tell me what to do?Thanks
LE
The same error I receive also with 3.01 version

[STC-Fan]

marcopolo80: you don't need to have ConvertToYV12() after QTGMC as you have already converted it in the script. QTGMC's output will be in YV12 so there's no need for the ConvertToYV12 again.

EDIT: the error message you are receiving may be due to having an old version of MaskTools which does not include that function. Please ensure you have the latest version, download it here (http://manao4.free.fr/masktools-v2.0a48.zip)

[marcopolo80]

The error message said something like:there is no function named mt_makediff line 633 in qtgmc 3.0.avsi script

[marcopolo80]

It works now,thanks, but very slow: about 1fps on AMD Phenom IIx6 1055T@3.2Ghz/4GB DDR3.With yadif I get around 17fps

[-Vit-]

QTGMC really wants multi-threading. For HD it's a necessity. Get a multi-threaded version of AviSynth and use this script:

SetMemoryMax(900) # May need to tweak this value, try 700,800,1000. Depends on source resolution and preset used

SetMTMode(5,4) # Number of threads=4. Tweak this value (leave the 5 alone). If the script crashes, lower it.
# Otherwise increase it 1 at a time until you get no further speed increase

LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\DGDecode.dll")
MPEG2Source("C:\test5min.d2v")
SetMTMode(2)
AssumeTFF()
ConvertToYV12(interlaced=true)
Import("C:\Program Files\AviSynth 2.5\plugins\QTGMC3.0.avsi")

spline36resize(1280,1080) # Do horizontal resize before your deinterlacer for a speed-up (works for *all* deinterlacers)

QTGMC( Preset="Fast", EdiThreads=2 ) # Use faster preset for HD, all presets are better than Yadif (so try "Super Fast" or "Ultra Fast" too)
# Can tweak EdiThreads for speed as well, best to start low and work up.
spline36resize(1280,720)

Distributor() # You may not need this last line - depends on your encoder. Try removing it - it may be faster (or it may not work)

Try different versions of MT AviSynth if you get stability issues: Standard 2.57MT (http://avisynth.org/mediawiki/MT), SEt's 2.58MT (http://forum.doom9.org/showthread.php?t=148117) or SEt's 2.6MT (http://forum.doom9.org/showthread.php?t=148782)
Or if you've converted to a completely 64-bit workflow then use the x64-2.58MT (http://forum.doom9.org/showthread.php?t=152800)

[marcopolo80]

:thanks:
It work well(I get 9-10fps on first pass with x264,that means perhaps 5fps on second).I saw that the number of frames is double(I supose it is normal),that means the time for encoding is double because oh this reason.
I will try later this week in an x64 Windows 7 to see if there are speed increases.Al the plugins will work ok in 64 bit environment?

[-Vit-]

Yes QTGMC will give a doubled framerate by default. You can add the line SelectEven() just after the QTGMC line to get the original framerate instead (and speed the script up).

Edit: The 32-bit avisynth & plugins will work on a 64-bit system. But to take advantage of 64-bits you need to get the 64-bit versions of the plugins. There's a good list on the x64-2.58MT (http://forum.doom9.org/showthread.php?t=152800) post.

[henryho_hk]

QTGMC seems to be adding in diagonal line noise not present in the source video

The noise is there in the source video. Just separatefields() and u can see them.

[marcopolo80]

Yes QTGMC will give a doubled framerate by default. You can add the line SelectEven() just after the QTGMC line to get the original framerate instead (and speed the script up).

There will be no problems with moving people in movie?I mean,if my source is 60i the result with default QTGMC will be 60p?The moving person in movie looks ok but it will take much time to encode double amount of frames.If I put SelectEven() the result will be 30p?

[LoRd_MuldeR]

There will be no problems with moving people in movie?I mean,if my source is 60i the result with default QTGMC will be 60p?The moving person in movie looks ok but it will take much time to encode double amount of frames.If I put SelectEven() the result will be 30p?

Well, your "60i" source contains 30 interlaced frames pro second, each of which contains two separate fields (snapshots). That makes 60 fields per second.

Consequently the temporal resolution of that source is 60 "snapshots" per second, although each of these "snapshots" only contains half of the lines of a full frame.

De-interlacing to "30p" means that you'll get 30 full frames per second, so you are discarding half of temporal resolution of your source! Motion won't look "fluid" anymore :eek:

In order to retain the full temporal resolution of the source you'll have to deinterlace to "60p", which means you'll get 60 full frames per second...

(And yes, if you apply either SelectEven() or SelectOdd() on your 60p video then the result will be 30p. There may be smarter ways to downsample 60p to 30p though)

[kypec]

There may be smarter ways to downsample 60p to 30p though)
Please be so kind and elaborate on that smarter methods, :thanks: I'm very intrigued...

[matfra]

STC-Fan ,
Can you explain me the difference between MDegrain2 and you Mdegrain2i2 ? Its it for interlaced source ?

[LoRd_MuldeR]

Please be so kind and elaborate on that smarter methods, :thanks: I'm very intrigued...

Sorry, I can't name you a specific method or script. But I would assume that you could do something like using the additional "intermediate" images (the ones that you are going to discard) for a more detailed motion analysis and then use this information to add a bit of "motion blur" to the remaining images - in order to make the motion appear more fluid. Motion blur is the reason why motion looks halfway "smooth" in cinema movies at 24 fps, but at the same frame rate 3D games look jerky. Well, at least those games that don't simulate a motion blurring effect ^^

[Robert Martens]

... I would assume that you could do something like using the additional "intermediate" images (the ones that you are going to discard) for a more detailed motion analysis and then use this information to add a bit of "motion blur" to the remaining images ...

I offer a cheap, half-assed imitation of this idea in SimpleSlugUpscale; after deinterlacing with QTGMC, the double-rate video is in a variable named 'scaled'. By default I just use scaled.SelectEven, but if you turn on the 'shtrhack' boolean I use Merge():

Merge(scaled.SelectEven(),scaled.SelectOdd())

Replace "scaled" with "last" if you want to try the idea in your own script. It's hardly an intelligent solution, but it's reasonably effective for the effort required.

I imagine one could do something better by way of MFlowBlur() from MVTools, but effective use of the MVTools collection is well beyond me at the moment. That and I want SSU to provide this shutter speed effect with progressive input without requiring any external plugins.

[-Vit-]

QTGMC seems to be adding in diagonal line noise not present in the source video
This video isn't interlaced, there aren't two different fields in each frame. It's more like there's an offset between the odd and even lines in places giving combing-like effects. Run bob() on it and you see two things:
- after bobbing every pair of frames are just shifted duplicates, indicating that it's not really interlaced
- the noise deinterlaces into roughly the diagonal lines you saw (and which QTGMC recovered).

I think you should be treating this as progressive with residual combing-like artefacts. In which case QTGMC is not the only tool available. For example, VInverse() gets rid of the combing, although it's rather coarse and retains the noise.

If you do use QTGMC, then flag it as progressive input, e.g:
QTGMC( Preset="xxx", InputType=1, TR2=3 ) # TR2=1,2,3 for some temporal denoising (mdegrain-based)
VInverse() # This helps too
____

Edit: double-rate to single-rate using motion blur: there may be motion blur in the source already. You're suggesting it's necessary to increase motion blur if using a lower (effective) fps?
Edit2: Also MFlowFPS can make a mess where there's complex motion. Merge is a fairly blunt tool that could reduce static detail. This sounds difficult to do well.

[henryho_hk]

Merge(scaled.SelectEven(),scaled.SelectOdd())


Will the clip look as if blend-deinterlaced?

[Robert Martens]

Edit: double-rate to single-rate using motion blur: there may be motion blur in the source already. You're suggesting it's necessary to increase motion blur if using a lower (effective) fps?
Edit2: Also MFlowFPS can make a mess where there's complex motion. Merge is a fairly blunt tool that could reduce static detail. This sounds difficult to do well.

I've seen the disaster you can get with MFlowFPS on most clips, but I'm referencing MFlowBlur, and only in passing; the little I've toyed with it suggests it would produce potentially satisfactory results, if only I were more experienced with its use. If it's actually useless, forget I said anything.

As for Merge, yes, it most certainly is a clumsy, ham fisted approach, and is precisely why the effect in my own script is both termed 'shtrhack' (changed from 'shtrfix' in early versions) and disabled by default.

To be honest I'd have ignored the whole concept if someone hadn't asked me for it. The idea is that interlaced footage is usually shot with a shutter speed that matches the field duration; 1/60th for NTSC, 1/50th for PAL. Simply bob deinterlacing, and then selecting every other frame, will give you 30 images per second with a 1/60 shutter speed in NTSC, or 25 images per second with a 1/50 shutter speed for PAL.

I'm not bothered by the resulting effect, but the complaint was that it's too "choppy" for some people, and that they instead want the effect of blending two fields together. In other words, they want to get as close as possible to the look of having shot their footage at 30p with a 1/30th shutter, or 25p with a 1/25th.

Comparing the result of my Merge() approach with the output of other programs' "blend" deinterlacing, the effect is close enough to my eye to seem useful to that subset of people who want this. Nonetheless, I don't care for it, so it's given what I feel is an appropriately derisive name and disabled unless the user explicitly requests it.

Will the clip look as if blend-deinterlaced?

I believe so, yes, but Merge() is built into Avisynth, so anyone is free to try it themselves and correct my thinking if not.

Each field becomes a frame with the double rate deinterlacing, then each pair of those frames (starting with 0 and 1) is blended together. I think masktools has functions that could be used instead of Merge, that would let you apply a weight to the blend, and as I say there's always MFlowBlur for a more sophisticated simulation of motion blur, but as a quick, dirty, no-plugins-required approach this seems to get the job done.

[-Vit-]

I've seen the disaster you can get with MFlowFPS on most clips, but I'm referencing MFlowBlur
Ah, my bad... Wow, I never noticed that function before. Though I'm still rather doubtful: there are many times when MVTools gets its motion predictions very wrong (try it on a rotary fan - even more fun if it's behind a wire mesh). When it goes wrong MFlowBlur will blur lots of little errors, which might make a passable mush in motion but it could look bad as stills. (Q)TGMC removes the worst of such problems with the rep modes, but in this case you would need to keep everything that MVTools gave you. Still, I haven't tried it, and I do see the concept....

Really this is just a knee-jerk reaction from me - I just added new features to pull out the finest detail... now let's blur it! :eek:

[henryho_hk]

I think the simple merge() is out of consideration as we can use a simple and speedy blend-deinterlacer at the first place, no need for the sophisticated QTGMC() script.

MFlowBlur() from 60p to 30p is a better option. If we are to simulate (say, NTSC) 29.97 fps shutter speed from 59.94-bobbed footage, what should be the value of the "blur=" parameter?

And I am thinking whether the existing motion vectors in QTGMC() can be re-used for motion blur in case of single-rate deinterlacing (new QTGMC feature? :devil:)

[kypec]

MFlowBlur() from 60p to 30p is a better option. If we are to simulate (say, NTSC) 29.97 fps shutter speed from 59.94-bobbed footage, what should be the value of the "blur=" parameter?

And I am thinking whether the existing motion vectors in QTGMC() can be re-used for motion blur in case of single-rate deinterlacing (new QTGMC feature? :devil:)
I'd love to see this feature added into QTGMC, just for the sake of seeing the effect when compared to simple SelectEven() approach.
I'm mostly dealing with 50i PAL comedy TV series so there is not much fast motion scenes anyway but still...

[sadie]

Hello, Semi-newb here with a few comments & Qs. On a test clip with a BFF source including an up-pan reversed sequence followed by a standard speed down-pan through jungle foliage coming to rest on a background waterfall, QTGMC is the only deinterlacer (Yadif and TDeint were the other contenders) that could not only knock out that shimmer, but also retain the detail in the moving areas. Brilliant. On the other hand, I've found in more static areas, it resolves less detail than the others. Truth or illusion on my part? And the degradation is slightly worse using any number of combinations of the noise bypass feature. Other than upping the ante with extra sharpening is there any other single feature in the program that might counter this effect?

Is the improvement in resolved (moving) detail due all or in part to the Nnedi3 interpolation component in the plugin. If so, I wonder if configuring Tdeint with Nnedi3 (rather than using the out-of-the box settings in Avidemux) would come close to getting nearly as good results, yet with a dramatic savings in time. Overall, I prefer Tdeint over Yadif, as it can much better handle vertical movement and seems to hold more detail. On the other hand Yadif is tops to completely removing edge jaggies, especially in capture stills. Neither is perfect. Would Tdeint + Nnedi3 be significantly inferior to QGTMC?

Is deinterlacing now at the state of the art where QGTMC might be the preferred method of getting an SD NLE project to a HD flat progressive living room screen? Previously, the classic interlaced DVD was a no-brainer for a simple player on a CRT. But nowadays, it seems you've only had two options: 1 - invest in a top of the line DVD upscaler / Bluray player (which I can't afford) or watch the movie from the hallway outside of your apt (which is uncomfortable). So the question is: Would a SD DV to progressive produced DVD using 'Doom9 technology' produce a quality comparable to what a hardware Oppo or Faroudja would be capable of with a classic interlaced mpeg2 disc?

I wonder if Didee's Stockholm test clip isn't a bit misleading in that the apparently disastrous look of the Yadif sample is only so because the source material is in slow motion. I've never seen anything nearly so bad on a standard speed pan with Yadif. That said, slow motion has always been a royal pain in any of the NLE's i've used. Which leads me to ask the following perhaps dumb Q. Would using QGTMC at the earliest step of the NLE process (even at the pure DV smart render stage) significantly improve on the output of such filters as slow/fast motion, reverse, spot and dust removal, stabilization, etc? In that light, I might ask Didee if his clip was deinterlaced before he applied the slow motion or afterwards?

Anyway, I finish my first full blown encode in a couple of hours, so either I may have answered most of my Qs, or simply be led to come back with more. Thanks for any answers and kudos on a brilliant initiative.

[henryho_hk]

QTGMC/TGMC's output is much more stable than TDeint+NNEDI3+MDegrain3.

[henryho_hk]

I just added new features to pull out the finest detail... now let's blur it! :eek:

Coz "movie" and "motion" share the same prefix "mo". :p

[matfra]

You said that Tr2=3 is base on MDegrain 1,2,3. Is there a way to Change the thSAD of this Mdegrain with a parameter to get a stronger denoising. What can I do i Tr2=3 is not strong enough.

[Didée]

@matfra: did you already try "rep2=0"? (But don't whine when you discover artifacts afterwards ... it's for a reason that it's activated by default. "Denoising" is not what TGMC was made for.)

Though, your problem could be buried one lever deeper, at the stage of motion estimation. You know. (http://forum.doom9.org/showthread.php?p=1476830#post1476830)


@ sadie:

On the other hand, I've found in more static areas, it resolves less detail than the others. Truth or illusion on my part? And the degradation is slightly worse using any number of combinations of the noise bypass feature. Other than upping the ante with extra sharpening is there any other single feature in the program that might counter this effect?
It is true that TGMC doesn't resolve "100% perfectly" in fully-static areas. You might give a go on the "lossless" option - with this addition, detail recovery in static areas will (should) become better; however results in moving areas probably will become slightly worse.

Key point is that Q/TGMC doesn't care or watch out for "static areas". This property doesn't exist in the world of TGMC. This has upsides and downsides ... the downside you have just discovered. The upside is: when you don't try to guess static static areas, then it's 100% guaranteed that you won't guess wrong. ;)


Is the improvement in resolved (moving) detail due all or in part to the Nnedi3 interpolation component in the plugin. If so, I wonder if configuring Tdeint with Nnedi3 (rather than using the out-of-the box settings in Avidemux) would come close to getting nearly as good results, yet with a dramatic savings in time. Overall, I prefer Tdeint over Yadif, as it can much better handle vertical movement and seems to hold more detail. On the other hand Yadif is tops to completely removing edge jaggies, especially in capture stills. Neither is perfect. Would Tdeint + Nnedi3 be significantly inferior to QGTMC?

Simply try it!
interlaced_clip
tdeint(mode=1,edeint=NNEDI3(field=-2))
You tell if it's "nearly as good". :)

BTW, i once did Stockholm also with TDeint+NNEDI (http://www.mediafire.com/?mzdzmbyjnmg). (Old version of TGMC, and only NNEDI instead of NNEDI3 back then - but that doesn't impact the technical principles.)


I wonder if Didee's Stockholm test clip isn't a bit misleading in that the apparently disastrous look of the Yadif sample is only so because the source material is in slow motion. I've never seen anything nearly so bad on a standard speed pan with Yadif. That said, slow motion has always been a royal pain in any of the NLE's i've used. Which leads me to ask the following perhaps dumb Q. Would using QGTMC at the earliest step of the NLE process (even at the pure DV smart render stage) significantly improve on the output of such filters as slow/fast motion, reverse, spot and dust removal, stabilization, etc? In that light, I might ask Didee if his clip was deinterlaced before he applied the slow motion or afterwards?

"Misleading" ... maybe a little. It's a "worst case example". With realworld sources, usually you won't see other deinterlacers fail so blataneously obvious. But: technically, the sample is perfectly valid. The shortcomings shown there are exactly the same shortcomings you'll get with any other clip. The problems might be less obvious, but in essence the're the same.

The slow motion has nothing to do with this at all. The clip was bob-deinterlaced from 25i to 50p. At the end I did "AssumeFPS(12.5)" to force slower playback speed. In technical regards to deinterlacing, this is completely irrelevant.

[matfra]

Thks Didée for you reply.
I admit before I was using Rep2=0 in past. What other parameter I can use to get a stronger denoising.
Im using this now.

QTGMC(SourceMatch=3,EdiMode="nnedi3",tr0=2,tr1=0,tr2=3,rep4=0,rep1=0,rep2=4,Sharpness=1.00,MatchEnhance=1.0,NoiseBypass=0, NoiseRemove=1.0, DetailRestore=0.0, NoiseRestore=0.0, Sigma=3, NoiseDeint="generate")

[matfra]

What you think about this idea. Is it better to apply a denoising before the deinterlaced ? Explample run Mdegrain3, then QTGMC. Or may run QTGMC with sone denoising parameter then clean up the rest with a Mdegrain2.

[Didée]

Many ways lead to Rome.

However, I can't tell which road is most suited for your footwear.

[henryho_hk]

Sometimes heavy grain kinda confuses (Q)TGMC.... and I find degrainmedian(mode=2, interlaced=true) helps. I also prefer "tr2=3"

[-Vit-]

Single-rate motion-blur: would slot-in nicely at the end of the script, and would be able to reuse the motion vectors (they might need a reanalyze). I'll try it at some point, but I'm looking at other things at the moment. Not so hard to try yourself - do it in the "Post-Processing" section on the clip "addNoise2" - can see all the mo-co settings a few lines further up where there are some MDegrains.

Static areas: It's not right to say that QTGMC doesn't care for static areas. QTGMC with source match is damn good on static areas. It's much better than the standard TGMC algorithm.
QTGMC( SourceMatch=3, Lossless=2, Sharpness=0.05 ) is near perfect in most cases. QTGMC( SourceMatch=1, Lossless=2, Sharpness=0.05 ) is almost identical, but faster. You can drop the Lossless=2 part for a bit more speed, a bit less accuracy and to avoid worries about artefacts with lossless processing. Tweak the sharpness in each case, may need more if you switch Lossless off - the default for source-match is 0.2

@matfra: I tend to use QTGMC + noise bypass to remove the worst of the noise, but not all of it as that can reduce the quality of the QTMGC processing (the mo-co needs a bit of fine detail to "grip" on to, if your "denoising" starts to become "blurring" then that's a problem). I follow up the QTGMC with a finer denoise step (mdegrain, another FFT3DFilter with a lower Sigma, whatever). Edit: And yes, I may well expose the thSAD settings again.

I hadn't anticipated so much interest in the denoising aspect of QTGMC. It was never intended to be a denoiser, I originally wanted to retain grain in some vids. Recently my interest was caught when I realized you could nicely "deinterlace" noise with motion compensation. However, that's about retaining noise too (or more precisely, detail that has "noise-like" properties). This part of the script is going to undergo some change: (a) to finalize my mo-co noise-bypass technique and deal with chroma noise, (b) simplify the settings, so it's more obvious what the settings actually do, and (c) reluctantly, I'll put some more control over the straight denoising aspects as obviously some people want a one-stop solution for their sources [but QTGMC is never going to be a replacement for a fully featured denoiser].

[Didée]

Perhaps my wording was not good.

What I meant is: neither Q nor TGMC do a "detection" of static areas. Deinterlacers like e.g. tdeint try to detect static areas, then perform "static->weave, motion->interpolate". Which is okay when the detection is okay, but problematic when the detection is wrong. See tdeint's plentitude of parameters that are dealing with / related to artifact protection.

And yeah, QTGMC's source match delivers good results. Though, I think something should be done about "lossless=2". It's not really convenient to first have a "lossy" deinterlacer, then offer a "lossless" option, but still produce a "lossy" result in the end. Lossy is lossy, lossless is lossless.

Nothing against the operations as such, they are fine. It's just that they should be structured differently, somehow.
(I like to drink milk as well as beer. Both are fine. But when I order "beer", then I want to get beer, not milk.)

[-Vit-]

I don't like the name of the Lossless=2 setting either, and considered alternatives. From a scripting point of view, it really is exactly the same process as Lossless=1 (real lossless), just done earlier. But from a user point of view it looks like a source-match extension - yet it's completely independent of source-match. The "logical" alternative is to make Lossless a boolean (true, false) and add a new setting, say "SourceMatchBoost" or something, also a boolean. Before I was sensitive to adding even more settings because I thought that you couldn't have more than 60 of them, but now I think that's a limitation from an earlier version. So maybe I will make a change like that...

[Robert Martens]

I think the simple merge() is out of consideration as we can use a simple and speedy blend-deinterlacer at the first place, no need for the sophisticated QTGMC() script.

I agree that it's probably the way to go for those who want things done as fast as possible, but I'd still recommend people at least give Merge() a shot; it's been a while since I've done any real investigation, but I've never seen a blend deinterlacer (or any deinterlacer, for that matter, save MCBob) that comes close to (Q)TGMC as far as producing clean curves and diagonal lines. Using Merge() gives blend-style results while retaining the neat, smooth edges TempGauss and its variants manage to create.

I don't mean to suggest that there's a perfect solution, but if you're as easily irritated by aliased diagonals as I am, you may find Merge helpful for same-rate output.

[sadie]

Well after a full 24 hours of encoding I've obtained 3 full 1-hour encodes using the triad of Yadif, Tdeint and QTGMC. To be frank, much of what I'd commented on earlier this morning turn out to be unfounded. A 1-minute source using single pass settings with non-geometrical subject matter cannot be the litmus test for deinterlacing which excels at straight lines and curves. In fact it was a pure pleasure to go through scene after scene, the first such joy I've felt in some time now in this domain. Finally, a real wheel that works.

To be short, Yadif can match QTGMC in some areas but fails in others. Tdeint can match QTGMC in other areas but fails in turn elsewhere. But none of them can match OTGMC all the time in all areas, much less surpass it. And surpass QTGMC does most of the time. As for detail in static areas, I again spoke to soon. What I mistook for 'sharp details' in Tdeint were in fact artefacts, that because they were in a natural milieu (a stone here, a water droplet there) were not readily apparent as 'errors'. In fact, even with my still captures edges with your program were as good if not not a tad better than even Yadif.

The noise present in my source (for better or worse) was reproduced perfectly. Like Vit, I'm no enemy to noise as such except for certain classic areas such as fade to blacks over bright blue skies or darkish monochromatic walls. It's been my bane for years and perhaps I'll take a stab at some of your suggestions above, or again just throw a whole lot of 'bitwidth' at it (or is that bandwidth). I think Henry Ho will remember a few years back we had huge go around on this board to try and fix an earlier problematic clip.

It took me nearly 6 hours per pass using the 'fast' preset (no MT). I take it if I dared to take the source match route we'd be talking at least double time, no? Which brings me back to Didee and his partial answer to my question. The 'fast' preset did not eradicate all errors. In fact, in those rendered slow motion sequences where the rate was .5x or less QTGMC cleaned it up. But on two static but unstabilized examples where the slow down went as low as .2x it had no effect. So if I'd delaced before using the filter, I'd probably not have suffered the consequences. Or not? Which brings me also to ask whether there is a version in the works of this avisynth script to be of use in a major commercial NLE. Is there a practical workaround (other, that is, than that of delacing the entire body of source material from the outset)?

I may have noticed a slight anomaly. The QTGMC encode (via virtualdub x264 fast recompress) appears slightly darker/contrastier/more saturated than any either the Yadif or Tdeint encodes (using Avidemux x264) or the straight to DVD route with Procoder v3. The latter three are identical luma & chroma-wise? On my screen the effect is actually an improvement under mplayerhc using either VMR 7 windowed or Haali renderers. Any explanation? And I'd still be interested if anyone could compare their experience with Oppo/Faroudja.

For the anecdote, I was so ticked off last week about the whole interlace to progressive problematic that I decided to kiss it all good-bye and gave my daughter my Panasonic GS400. Trouble is she's a tough cookie and I fear there is now no getting it back. Well, maybe I'll just go and write a detective novel.

[STC-Fan]

STC-Fan ,
Can you explain me the difference between MDegrain2 and you Mdegrain2i2 ? Its it for interlaced source ?
MDegrain2i2 is for interlaced content only. The code is straight from fizick's MVTools2 page but with values from Cnr2 passed to it, which was added by someone on the VideoHelp forums.

This video isn't interlaced, there aren't two different fields in each frame. It's more like there's an offset between the odd and even lines in places giving combing-like effects.

Yes, I am seeing them now - and as henryho_uk pointed out just doing a SeparateFields() operation shows it up as well. I'll certainly make use of this a bit more in future as I suspect a lot of the other files I have have come out in this way, and are not really interlaced at all. I haven't really given it much thought before, aside from noticing the smaller and less frequent nature of combing artifacts, but I think only the content I have from the 1990s / 1980s is truly interlaced, so I probably need to treat more of the post-2000 content in the same way as for this Castrol video.

If you do use QTGMC, then flag it as progressive input, e.g:
QTGMC( Preset="xxx", InputType=1, TR2=3 ) # TR2=1,2,3 for some temporal denoising (mdegrain-based)
VInverse() # This helps too

EDIT: I've dropped in those settings (including Vinverse) as above, commented out the separate MDegrain2i2 and the results are very good - I hadn't realised MDegrain was actually used in QTGMC but looking at the preview in MeGUI and the final output encoded to x264, using QTGMC with TR2=3 removes all the noise that MDegrain2i2 was removing before, but is leaving a sharper finish. And the encoding speed hit nearly 10 FPS so that's a big improvement on the previous settings :)

[-Vit-]

I take it if I dared to take the source match route we'd be talking at least double time, no?
Not so much as that: I just did a quick single-threaded test and it's around 15% slower with SourceMatch=1, Lossless=2 when using Preset="Fast". Nice detail improvement too. A slower system might suffer more though...

The QTGMC encode [...] appears slightly darker/contrastier/more saturated [...]. Any explanation?
Try "Precise=true". To get a tiny bit more speed, QTGMC removes a couple of minor adjustments by default. I haven't looked at that setting for a while, it may or may not be relevant. I'll be looking carefully at the chroma side of the deinterlace for the next significant release - I'll take the time to do some similar comparisons.

[Didée]

The QTGMC encode [...] appears slightly darker/contrastier/more saturated [...]. Any explanation?
I reckon it's a playback issue, and/or the video got flagged differently by the different applications, or sth like that. When I compare bob/yadif/tdeint/qtgmc with levels(), the histograms are virtually identical. No darkening / lightening / contrast spreading is taking place. Everything's like it should be.

[sadie]

Thanks Vit, for your encouragement on the source match info. Doesn't sound as bad as all that. I'll give it a try next.

As for the eventual luma/chroma shift, yes, Didee that's what I originally thought as well. I've experienced it often in the past where it seems to be related to one of two issues, perhaps related. One is somehow connected with overlay. If I open any mplayerhc file under Xpsp2 system default (VMR7 windowed) I'll be presented with an image that is to my eyes flat, over-ex, washed out. If I open a second instance (or more) of that same file the video will then manifest the richer deeper color associated with my edit. Windows is apparently using a different color space the 2nd time around. Yet in the past the simple action of switching to Haali renderer corrected this anomaly. Haali would open all instances of the video with same color space/balance (whatever you want to call it). Except in this case. Here, Haali too reproduces the same aberrant behavior as VMR7. Which is why it lit up a bulb in me. Also, some time back I'd had a similar issue when dealing with Coreavc decoder and the luma variance turned out to be associated with TV vs Computer levels. Now, I haven't a clue what I might have done in virtualdub to possibly force/expand 601? levels artificially. It remains a mystery.

[sadie]

A little update. I tested (Preset="Fast", SourceMatch=1, Sharpness=0.30) and found it hardly any slower. Fine. But the visual quality is inconsistent. In an amorphous natural setting, detail in foliage seems indeed a bit crisper. Yet in an urban more geometric context, I find the contrary. Building details seem slightly softer. Flat areas even more. Is this merely an issue with the sharpness setting? In which case, I'd be hard pressed at this point to distinguish between resurgent hidden detail and newly fabricated noise. Anyway, I don't wish to risk too much haloing as I'd like to attack a 720p upscale at some point. Secondly, I've found that the doubling/bobbing of the image merely reinforces that 'video effect' so I may stick to the SelectEven setting and suffer the occasional judder. Besides aren't the 100/120 hz TV functions one of the few options that actually work well on a set top? Also, in order to limit processing time I'm leaning toward just using a CQM mode (I've got 4000kb/sec to throw at it). What is the incidence of using single vs dual pass with QTGMC?

I've found a possible small bug (or one that is encoder related). SourceMatch mode in Avidemux will cause it to crash when a screen saver kicks in. This doesn't happen with any other of the modes I've tried. And it doesn't happen in VirtualDub using SourceMatch. Any ideas?

The only systematic problem areas I have are those which involve NLE interference: slow motion & still captures. Improvement I suppose might marginally help by deinterlacing before operating those filters. To ask my previously posed Q differently, have they invented a way to import an avisynth function on a specific clip on a timeline and not the whole project? Whatever happened to Avisynth Studio development? I'm still googling for that one...

[matfra]

Can someone tell me if its possible to add a pre-filtered clip for QTGMC that Tr2=3 can use. I would like to strengh the denoising. Any Idea ? Here's my current script.
Thanks

###############################
# I7 950 4.3 MHz @ 100% CPU #
# 0:25s @ 33,80FPS #
###############################
import("C:\Program Files (x86)\AviSynth 2.5\plugins\MDegrain5.avs")
import("C:\Program Files (x86)\AviSynth 2.5\plugins\SeeSaw.v0.3e.avs")
SetMTMode(5)
SetMemoryMax(768)
DirectshowSource("C:\1.avi")
SetMTMode(2)
QTGMC(SourceMatch=3,Lossless=2,EdiMode="nnedi3",tr0=2,tr1=0,tr2=3,rep0=4,rep1=0,rep2=4,Sharpness=1.00,MatchEnhance=1.0,NoiseBypass=2, NoiseRemove=1.0, DetailRestore=0.0, NoiseRestore=0.0, Sigma=1, NoiseDeint="generate")
AssumeTFF()
SelectEven()
Seesaw(nrlimit=0,nrlimit2=0,sstr=1.5,sootheT=0,sootheS=0,Slimit=3,Sdamplo=29,Spower=3,SdampHi=35)

[-Vit-]

Edit: @matfra: No, you can't provide a pre-filtered clip for TR2 - such a clip would have to be smart-deinterlaced already. Possible within the algorithm I suppose but I haven't tried.
____

But the visual quality is inconsistent. In an amorphous natural setting, detail in foliage seems indeed a bit crisper. Yet in an urban more geometric context, I find the contrary. Building details seem slightly softer. Flat areas even more.
What you are seeing is what your source is like.

I had almost the exact same feeling when I first experimented source match - better on foliage, but softer on some edges/detail. I realized that I'd simply got used to the results of standard TGMC, which tends to strongly sharpen "real-world" edges/detail (as opposed to static logos, text etc), but in doing so it loses fine detail like foliage making it look plastic-like. DoubleWeave, Bob and NNEDI3 your source and compare stills with standard (Q)TGMC and source match QTGMC. But don't do it side-by-side, do it by switching between full-screen images. Believe me I've done this in exhaustive detail and with a huge range of sources (I should calculate some metrics really). With source-match, what you see is what you've got, although SourceMatch=1 is actually only half-way there...

Source-match is about accuracy, which is not quite the same thing as quality. Unlike standard (Q)TGMC source-match does not seek to denoise nor increase compressibility. It is about giving you the most representative shimmer-free deinterlace of your source without adjusting it in any way (so if you do find an example where source-match is actually further from the source than the standard algorithm I'd be very interested). In practice, I tweaked the application of the algorithm, so you can still do some denoising or sharpening, but both of these work against what source-match tries to do (not necessarily a bad thing - balancing requirements).

And sure the differences are fairly minor - I have been careful to highlight that. The best aspect is an increase in vertical detail - losing the very "interpolated" look of TGMC, especially with SourceMatch=2,3. This increased detail is unequivocal, but whether you can actual notice it or not is a different matter. If you can't then don't use source match. It really is for those who use a big screen/upscale and for pedants like myself. Makes it most appropriate for SD too.
_____

Crashes: I've noted earlier the various reasons for instability under MT which are to do with avisynth multithreading and caching - both of which become more problematic as scripts become complex. I am currently looking at some workarounds.
_____

"Double-rate "video" look": I have implemented some new settings to simulate a different shutter angle in the output (just the motion-blur implications) - for single or double-rate. It uses MFlowBlur, as discussed earlier. It does help with single rate output on my sources. As expected it suffers whenever MVTools struggles to analyze the motion, but for less chaotic sources it's not too bad. Main problem is that MVTools operates on blocks not exact shapes, so the motion blur "bleeds" into areas that are actually fairly static. I'm going to look if there are any ways to repair this bleeding then I'll post this version (later today or tomorrow)

[Boulder]

Crashes: I've noted earlier the various reasons for instability under MT which are to do with avisynth multithreading and caching - both of which become more problematic as scripts become complex. I am currently looking at some workarounds.Whenever you have something to test, I'm available. It seems that me going back to tsp's MT build only works in some cases. I usually have to lower the number of threads quite low and get much less than optimal performance.

[Didée]

Regarding "MT crashes", I've also fiddled with various things. Some things "improved", but when you start with, say, a 90~95% crash probability, and manage to come to a, say, 35-40% crash probability, then this might be an improvement, but it's still not usable.

Though ... almost all crashes disappear magically when I switch to SEt's latest Avisynth 2.6 build. Therefore I assume that all 2.5.x MT-builds are not stable. As simple as that.
Question is - what sorts of "MT-related fixes" have been done to 2.6, that 2.5.8.5 is missing? :devil:

[Boulder]

With the current video in the pipeline, I've been experimenting with all three known MT builds but with bad results. It usually takes 10-15 minutes to crash no matter what I do :( I usually get better results by encoding to lossless ffdshow HuffYUV first but this time even that doesn't work.

Mind you, my dual core was much more stable but that was probably simply because of less threads utilized ;)

[sadie]

Well, Vit, I finally hit at least one nail on the head. "Accuracy vs quality". You know there's the spark of a major philosopical debate in there somewhere. You suggest that SourceMatch is for "those who use a big screen/upscale". Is that to say that my above concern about haloing or bordering would be mitigated by sticking to SourceMatch if I were to go 720P? All in all, in real time viewing the 'foliage' improvement outweighs the 'non-resharpened' scenes so maybe I'll stick with it. I'll give SourceMatch 2 a try as well. Did I mention that avidemux crash was in single-thread mode?

[matfra]

Edit: @matfra: No, you can't provide a pre-filtered clip for TR2 - such a clip would have to be smart-deinterlaced already. Possible within the algorithm I suppose but I haven't tried.
I hope you could add this new feature in your next build of QTGMC. I would be nice to specify a Pre-sourceNR to the Tr2. I hope my cry will be listen by a master !
Thankx

[Didée]

I would be nice to specify a Pre-sourceNR to the Tr2.
Anyone explain to me what this whould be supposed to do? I don't see it.

Though, my guess is you're looking at the wrong place for a solution. (Solutions are good, but they need to fit to the problem.)
You're working with a VHS capture?

[-Vit-]

I figured matfra might be looking for one of those situations where you use a different source and super clip to enhance/reduce something based on the motion match. But my sentence is rather vague because I too have no real idea what is being suggested. Matfra: QTGMC cannot solve all problems.
____

Crashes: I have seen plenty of examples (from numerous systems) indicating that no MT version of avisynth is stable, neither 2.58 nor 2.6. Crashes vary by script, plugins, avisynth version, software, OS and hardware.

The caching in avisynth is overeager. It's a real problem in MT, but could affect non-MT too, especially HD. By default avisynth caches every intermediate filter result, which number in the hundreds per-frame in QTGMC. It also caches every output frame, which is redundant when encoding. However, the system is smart and will attempt to drop unneeded frames from the cache. Unfortunately it can think that unneeded intermediate results are actually important and will continue to eat more and more memory. Until it runs out of memory and crashes (and if you're using a 32-bit workflow, you have a memory cap of 2Gb per process regardless of physical memory).
____

Haloing & Source Match: Haloing in (Q)TGMC is caused by the Sharpness setting, which recovers the source sharpness after the initial shimmer-removing temporal blur. The standard algorithm has a high default Sharpness intended to keep static areas accurate, but potentially producing significant halos elsewhere. The source-match approach recovers most of the source sharpness without resharpening, so the Sharpness setting is just 0.2 producing minimal haloing. You can set it even lower with the better source match modes + lossless (about 0.05), giving no haloing at all (other than what is in the source).

However, if you want to increase sharpness from the source... well, increase the Sharpness delicately with source-match as there's no limiting by default (because sharpness limiting also limits the accuracy of source-match).

[Didée]

The standard algorithm has a high default Sharpness intended to keep static areas accurate, but potentially producing significant halos elsewhere.
In theory it shouldn't. The sharpened intermediate clip is limited by the (compensated) neighbor frames. Therefore the sharpening shouldn't introduce new halos on its own ... the limiting makes processed pixels stay in range produced by original source pixels.

Practice is a little different. First, even such limiting can make already-present halos more prominent. (As known from LSF - even with overshoot=0, source halos are more obvious after LSF).
Another point is the pel subsampling ... pel=2 is sharp, and to that extend, it is acting quite similar to a "sharp" resizer like e.g. lanczos4resize. Means, pel=2 will also introduce some ringing, and will amplify already-existing halos.

I don't claim that the "pel2hr" option in TGMC is particularly good. But it is there for a reason. Partly to help with the problem ... and partly to indicate that the script creator was aware of the problem from the very start. ;)

[-Vit-]

No criticism intended Didée, just communicating practicalities. I, more than anyone perhaps, appreciate the thought and detail that has gone into the TGMC algorithm.

[-Vit-]

I've posted the latest version v3.05 in the OP (http://forum.doom9.org/showthread.php?t=156028).

This version can simulate the motion blur of different shutter speeds using MFlowBlur as discussed earlier. In fact I decided to use shutter angle for units to avoid involving the frame-rate in the settings (didn't want to use MVTools non-standard percent either).

You need to choose an output frame rate and shutter angle, and also provide (or estimate) the effective shutter angle of your source. It's all switched off by default, so it will not affect any other scripts. Here's an example (edited: I missed the vital MotionBlur setting when I first posted...):

# Output every 2nd frame (i.e. single-rate), source filmed with high speed shutter, standard "film-like" shutter angle for output
QuickTGMC( Preset="Slower", MotionBlur=1, FPSDivisor=2, ShutterAngleSrc=45, ShutterAngleOut=180)

FPSDivisor=1 means select every frame (double-rate - the default). FPSDivisor=2 means select one out of every 2 frames (single-rate output - SelectEven() isn't needed). FPSDivisor > 2 is allowed. If this sounds backwards, it's simply the same concept as SelectEvery for those of you who script. I will leave detailed discussions of shutter angle to other experts. In summary, higher shutter angles mean more motion blur. You can estimate shutter angle on a source by looking at the motion blur on a moving object [on a still]. Perfectly crisp is 0, full blur from previous frame to next frame is 360. 180 is considered the norm for a film-like look, many non-film sources I see are rather sharper than that. Artificial output angles > 360 are possible in limited circumstances (extra blurry).

Things to watch out for: the motion blur can bleed into static areas due to the block-based nature of MVTools, there are settings to help repair that (read the "Motion Blur" section in the script docs). These repairs are slightly switched on by default. If the motion analysis makes bad decisions then there's little can be done - there's no effective way of detecting analysis errors that also doesn't detect most of the motion blur as errors...

I did a quick test with progressive input and it worked too. Note that you would usually set FPSDivisor as 1 since progressive input comes out as same-rate so you want to keep every frame.

Results are better than I expected for non-chaotic input. I would certainly consider it for single rate output as it smooths out the stutter I get (the above example suits my sources). However, it's not going to work well on some sources, and in any case consider this feature experimental.

[Robert Martens]

My god, you work fast! A quick test on my own material suggests you've succeeded in your aim. I'd love to comment in more detail, but the relationship between framerate and shutter speed, the difference between mechanical and electronic shutters, and the difference between exposure time, angle, and percent are all way over my head. I managed to wrap my mind around the basic idea, but beyond blending two fields together I'm out of my league.

I've just started working on a small update to my own script, and with any luck I can incorporate the new QTGMC options in there somehow. Looks like I may have occasion to use the 'val' data type for one of my parameters after all.

Thanks for the effort, this sounds like it'll come in handy!

[sadie]

Great, Vit, I'll have to give it a whirl myself. At the risk of exposing my naivete on this I'll dare ask why the need for FPS=1 at all. I mean isn't the 'video effect' a two-fold phenomenon? Either you have single-rate output with 'pseudo-motion blur' and judder or you go the double-rate route and suffer the full impact of zero temporal/motion blur while correcting the judder. So inasmuch as FPS=2 all at once alleviates the judder by emulating a true motion blur doesn't that obviate the need for going to double-rate (FPS=1) in the first place? There's no longer any judder for the doubling to correct. Or is it strictly meant for progressive sources? Be so kind as to tell me what simple principle I'm not connecting with. Thanks.

SourceMatch=2, Sharpness=0.30 on two samples gave me images that oddly closer resembled both the default TGMC test AND the source. SourceMatch=1 even transformed a woman's open mouth into the shape of an old black & white TV. Uncute. Yet SM=2 meant twice the encoding time. I'm wondering again what I was seeing in my foliage stuff. Oh well, more testing. And for something that in real time viewing won't be discernible for 99% of the population. Still, great product of yours (and Didee's).

[matfra]

I figured matfra might be looking for one of those situations where you use a different source and super clip to enhance/reduce something based on the motion match.

You understand right -Vit-. I would like to know if it possible to add to the script athe option to use a different source and super clip to enhance/reduce something based on the motion match.
:)

[-Vit-]

sadie: Double-rate material can and does contain motion blur, how much depends on the shutter speed used when filming it. But interlaced content shot with a shutter speed suitable for double-rate may not have enough motion blur for single rate, making it judder when you drop half the frames. And that's the primary use for these settings. But that's not their only use, they can smooth material shot with a very high shutter speed, even whilst keeping double rate. They can be used to give exaggerated blur, or to blur something that would normally be perfectly sharp (e.g. game footage). As a scripter, I prefer not to care about the why, and just provide for everything that's possible. The formula used is general, and supports all FPSDivisor & shutter angle values with a single piece of script.

Also, FPSDivisor=1 doesn't exactly mean double-rate, it means use every frame (field). In the case of progressive input, you would need FPSDivisor=1, or you would drop half the frames. [personally I don't like the moniker double-rate, though I understand why it has come about. If a video contains 60 images per second then that's its rate as I see it. Interlacing is just a form of compression scheme; if in decompressing you drop or merge half the images then that's half-rate, not single rate]

Standard TGMC will smooth the source, which you might prefer. I spent some time wondering why a line in one shot was aliased with source match but not without, until I realized the source itself was aliased on that shot. Whilst source match occasionally does introduce errors, often it is the source itself that you are unsatisfied with, and the stronger temporal smoothing of the standard algorithm is what you want. [I knew that adding a feature billed as "precise" would cause problems]

matfra: You're joking, right? That sentence was deliberately too vague to be useful.

[sadie]

Before my last fine tune-up and final output and then moving on, I thought I'd pass on a last rather serious quirk I discovered yesterday. I don't think it's related to QTGMC, but since it occurred with me for the first time in this context...well, who knows?

Yesterday, I did a half a dozen consecutive CRF 18-26 range tests (avidemux 2.5.3 medium preset) with regular QGTMC fast preset and all came out with finger-width horizontal smears across the center of the screen, occasional smears elsewhere and even some flickering in those areas. Static areas were much more affected than scenes with movement in the subject or camera.

The problem is I've since tried to reproduce the effect and simply cannot. Mediainfo confirms exact same settings for identical encodes. The only possible explanation I can imagine for this is that I had a few firefox windows open and was doing moderate surfing, but memory use remained well under 2gb (4gb ram installed). I'm not especially reassured that it hasn't resurfaced. Anyway, to whom it may concern...

[-Vit-]

sadie - non-reproducible problem: I believe all the plugins used by QTGMC are deterministic (i.e. produce the same output every time given the same input/settings). The script itself is deterministic. The bugs in avisynth may not be, the crashes certainly show up randomly. As you're running single-threaded it may be a case of memory corruption if you can't reproduce the error. I've processed hundreds of hours of footage with QTGMC and the only time I've ever seen non-reproducible effects was when I had a bad memory stick.

[henryho_hk]

Any sample snapshot, sadie?

[henryho_hk]

I know inputtype=2 is meant for fixing badly deinterlaced footages. But it will appear to me that the original TGMC is doing a much better job, especially for those ill-brained field-duplicated deinterlacers. What parameters I should be tuning for these sources?

[-Vit-]

Have you tried InputType=1? That's should be the same as what original TGMC does but without the frame duplication (which interferes with the motion analysis). It can be the appropriate method for less degraded sources.

InputType=2,3 drop half the fields - which is pretty extreme, but necessary when the source is a real mess.

There's also this approach from a previous post:

s = WhateverSource("Your.src")

# Make versions from upper fields and lower fields (use TR2 and ProgSADMask also if you wish)
t = s.QTGMC( InputType=2, ... ) # These lines work well with "SourceMatch=1, Lossless=2" (need v3.01+)
b = s.QTGMC( InputType=3, ... )

# Now merge the results in some way. Choose ONLY ONE of the lines below and delete the other
Merge( t, b ) # Smooth result, may need more sharpening
t.Repair( Merge( t, b ), 1 ) # Sharper result, could try 2 at end of this line


I find that latter approach works very well on some sources, less so on others. It's slower than it could be of course...

[@MeGui]

Vit, nice one mate...:)))
I have i7 core, so can u tell me how to use max of my cores?
I have Avisynth 2.5.8 MT installed.
What would be "lines" only for pure de-interlacing?

Tnx in advance...:)

[-Vit-]

Straight deinterlacing using multi-threading:

SetMemoryMax(M) # Optional line. Leave it out at first. See below for value M
SetMTMode(5, X) # See below for value X
WhateverSource("Your.src")
SetMTMode(2)
QTGMC( Preset="Slow", EdiThreads=Y ) # Choose preset based on overall speed/quality you want. See below for value Y
Distributor() # This line may or may not be necessary, try it without - it may be faster


Setting X
- Start at the number of cores in your machine
- If it crashes, decrease 1 at a time
- Otherwise increase 1 at a time until CPU usage is very, very close to 100%, don't go too far or it will slow down

Setting Y
- Start at about half number of cores and tweak upwards or downwards for best speed. I find Y=1 often works best when using SetMTMode.
- Obviously this is a balance with X

Setting M
- Using the SetMemoryMax line and choosing a good value for M can allow you to use more threads and so get more speed. Particularly important for slower settings
- Try 400,600,800,1000 etc.

[I should add this to the OP / docs...]

[@MeGui]

hmmm, using Virtualdub 1.9.11 and no meather what, crashed...

[-Vit-]

With Virtualdub I don't think you need the Distributor() line.

Try different versions of MT AviSynth if you get stability issues: Standard 2.57MT (http://avisynth.org/mediawiki/MT), SEt's 2.58MT (http://forum.doom9.org/showthread.php?t=148117) or SEt's 2.6MT (http://forum.doom9.org/showthread.php?t=148782)

[@MeGui]

I think is not Avisynth issue...this "MT" running good...

http://pastebin.com/EAmLksev

[-Vit-]

I don't know what that paste was about.

But in any case, MT avisynth is buggy. It might work with one script but not another. You should try the other versions. SEt's versions appear to be most reliable, but each system is different. There are many posts on the subject in this thread.

[kolak]

hmmm, using Virtualdub 1.9.11 and no meather what, crashed...

I have constant crashes also- my source is 1920x1080 50i and new QTGMC is almost unusable with MT.

Works fine when using simple command:
QTGMC(Preset="Fast")- any setting above crashes.


Andrew

[@MeGui]

Yes, indeed ...but, i can still get only 1.14 rendering...not more :(

[-Vit-]

New QTGMC is almost unusable with MT
I would be interested to know which version of QTGMC you last had stable...

I've made very few changes to the core algorithm for several versions. There's nothing new in version 3 - the presets on their own should result in the pretty much same filter graph as always. I have been adding extra features, but they're all off by default so it should run just as before when you just provide a basic preset. However, I used to have more explicit conditions to remove unused filters, I now rely on avisynth to do that job when it builds the filter graph - I wonder if that is causing a problem. Maybe I should revert to doing things manually again...
____

EDIT: A quick look at the avisynth parsing code and I think if you don't explicitly put a conditional around an unused filter then its constructor is called anyway [correct me if I'm wrong devs]. That's unfortunate, it could cause unnecessary memory use at the least.

In practical scripting terms it appears to mean that this:

input = last
processed = (setting == 1) ? input.ComplexFilter() : input
return processed

is better than this:

input = last
processed = input.ComplexFilter()
return (setting == 1) ? processed : input

These two scripts are equivalent. If setting = 2 then the ComplexFilter plugin isn't going to get used. However, in the second example it will be constructed when the script is parsed at the beginning - this will use up memory unnecessarily. The first example guards it with a condition, the plugin will neither be constructed nor called. [At least, that's what I surmise from a quick glance over the avisynth source code]. I had assumed the second method was OK as I had seen it in a number of other scripts and examples.

I'll revert back to my original method and see if there's a stability difference... stay tuned...

[Gavino]

A quick look at the avisynth parsing code and I think if you don't explicitly put a conditional around an unused filter then its constructor is called anyway [correct me if I'm wrong devs]. That's unfortunate, it could cause unnecessary memory use at the least.
That's correct. And with multi-threaded Avisynth, you will get an instance for each thread, using more memory.

How much impact this has will of course depend on how much memory the filter allocates on construction - many simple filters have nothing beyond a few simple variables, but something that keeps its own frame buffer (for example) will obviously use more.

[sadie]

...and I thought I was out of here. Well, I managed to reproduce my bug/error albeit in a still not completely conclusive way. And I'm also at the stage where in passing I'd like to apply a couple of old human adages to this technology: A: Honeymoons never last B: Life is a crap shoot.

That said, and after two successive crashes I managed to get a successful 9 hr encode (fast preset) for my l hr 10 min video. I also managed to custom start three threads for my modest core duo T8100. That should have given it some elbow room. Again, the encode and subsequent tests were avisynthproxy in avidemux and an x264 medium quality preset.

On playback without even looking too hard I noticed some soft softs which are discernable in the following captures which reflect the order with which I tested various things. Areas to look for are bldg to far right, lst floor window 2nd from left and 2nd fl window lst on left. (1) shows the smoothing/obliteration equal to my full encode. I re-reproduced it using different crt and abr settings with identical results. (2) here, I rebooted avisynthproxy in SourceMatch 2 25p mode. The anomaly disappeared. (3) further reboot back into default QTGMC and here the smoothing went bananas, equivalent to what I experienced on Saturday. (4) booted avs script with default QTGMC in 25p (selecteven) mode...the problem disappeared. (5) finally back to original avs in 50p and once again the error was gone. I should add that I think the source sample is not virginal. It might have undergone some slight slow motion filtering.

Vit, you mentioned 'memory corruption'. Whatever that's supposed to mean it doesn't get me very far. From my amateurish experience with computers over the years, in a case like this I'd always thought a hardware problem generally either translate into something that works or breaks. Be it the hardware itself or the task it was trying achieve. But it doesn't 'bend'. And this random smoothing strikes me as being just that: bending.

The problem isn't systematic except for the fact it's only ever occurred in 50p mode, never with 25p. Is there something to be learned here? Maybe I should play it safe and stick with that. (SM=2 by the way was a decided upgrade from SM=1). Further, is there any reason NOT to do a quality archive copy @7000-8000 kbs using sourcematch just in 25p and either a) count on my Sony flatscreen's 100mhz 'motion flow' to iron out the judder or b) later on down the line when I'd like to play around with some 720p upscaling go about doubling the frames then and hopefully not suffer a quality hit. As I always back up the finished DV render on mini-DV tapes that shouldn't be a such a huge risk.

You say, Vit, you've never had a 'failure' in hundreds of cases of QTGMC. But maybe you're on systems using quad-cores or far better. Which brings up the 'minimal configuration' issue. Is there one for this avisynth plugin?

I just had a last thought as I write these words. All the tests I've done were through to an x264 encode. Wouldn't the truly rigorous method be to output to another DV avi file? How much might the encoder end of these operations be interfering in performance/image quality? As I recall x264 loop filter settings bear on perceived sharpness. Doesn't this complicate matters? Thanks for your continuing help.

[sadie]

Oops, my attachments disappeared.

EDIT: Sorry for error. I'd thought attachments would remain in a link form with info file name. Didn't want to bloat the thread. I've removed pics 4 & 5. The middle image represents what the correct encode should like anyway.

[marcopolo80]

For this kind of source file (known as 1080-60i):

Video
ID : 2064 (0x810)
Menu ID : 100 (0x64)
Format : MPEG Video
Format_Commercial_IfAny : HDV 1080i
Format version : Version 2
Format profile : Main@High 1440
Format settings, BVOP : Yes
Format settings, Matrix : Default
Format settings, GOP : M=3, N=15
Codec ID : 2
Bit rate mode : Constant
Bit rate : 25.0 Mbps
Width : 1 440 pixels
Height : 1 080 pixels
Display aspect ratio : 16:9
Frame rate : 29.970 fps
Standard : Component
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Interlaced
Scan order : Top Field First
Compression mode : Lossy
Bits/(Pixel*Frame) : 0.536

and thinking that almost all the time the camera will shoot in natural and good light-so the footage will look OK-what is the best: to stay interlaced and encode to x264(or MainConcept h264 AVC Broadcast) at 8-12 Mbps(not sure if 12 is not a waste of space) or to go progressive(720p) also encoded in x264 or MC-264avc at 6-8Mbps?
For the second choice which settings for fast encoding speed?And keep 59.94 or to make a pulldown to 23,97?
My script looks now (obvious with the help of some people from this forum):


SetMemoryMax(900)
SetMTMode(5,6)
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\DGDecode.dll")
MPEG2Source("C:\test5min.d2v")
SetMTMode(2,6)
AssumeTFF()
ConvertToYV12(interlaced=true)
Import("C:\Program Files\AviSynth 2.5\plugins\QTGMC3.05.avsi")
spline36resize(1280,720)
QTGMC( Preset="Fast", EdiThreads=4 )


The CPU is AMD x6 1055T@3.3Ghz with 4GB DDR3 1366 and x86 OS

Thanks

[-Vit-]

Here's a revised version of QTGMC (v3.10) which reverts to the approach of v2.x. This will call far fewer filter constructors, using less memory and maybe prevent the crashes some people have had. If you couldn't run the recent versions, please give this one a quick test and report back. I haven't updated the OP yet in case this doesn't help. I tested this multi-threaded in VirtualDub (1.9.10) on some 1920x1080x60i material at preset "Slow" - it was fine:

QTGMC v3.10 (http://www.mediafire.com/download.php?4rgxf8s2iermiii)

____

sadie: I've never seen smearing like that out of QTGMC. Might it be x264? That can cause smearing, especially given that it's a fairly complex scene and you said you were going down to CRF 26? And it's strange that it is not consistently reproducible. These are deterministic algorithms, there isn't scope for variation. x264 does have a non-deterministic multi-threaded mode (the order of the threading calls is unpredictable and they get some tiny speed advantages by not worrying about that), but I don't think that mode is a default.

However, x264 is getting in the way with these examples. I always encode lossless 1st pass, then x264 the result. You should try to reproduce the error in a lossless format. Better still you should post that source snippet, so I can try to reproduce the issue.

____

marcopolo80: That seems like a decent start, but you've made a mistake with the resizing - you shouldn't vertically resize before the deinterlacing. The last part of the script should be this:

#...
spline36resize(1280,1080)
QTGMC( Preset="Fast", EdiThreads=4 )
spline36resize(1280,720)

You should tweak the threading settings to get best speed. About your other questions - there is no "best" choice, you must try each of your options and decide for yourself.

[marcopolo80]

Thanks,I'll try those settings.

[yup]

-Vit-!
Amazing!!!
Now my HD source work SetMTMode(2,4) and placebo preset without crash (before work only SetMTMode(2,2)). I am also note speedup 20...40%.
Waiting from Stephen R. Savage YUY2 release.
Good work.

[-Vit-]

Thanks yup. I'll take that as confirmation that I've at least fixed something. I've updated the OP to v3.10

Avisynth only calls what it needs when frame-serving, but it initializes everything that isn't explicitly guarded with a condition. QTGMC contains a huge number of filters, many of which are not used unless you choose a particular setting. However, almost all of them were being initialized, which must have been using a great deal of memory - it was hard to tell because the avisynth cache uses a huge amount of memory anyway. Other scripters with conditional / optional processing should note the example above (http://forum.doom9.org/showthread.php?p=1477993#post1477993).

[@MeGui]

In My case, calling "mt_masktools-26.dll" in script, did the trick...

LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\mt_masktools-26.dll")
SetMemoryMax(1024)
SetMTMode(5,8)
DirectShowSource("C:\Documents and Settings\XyZ\Desktop\Svadba2.avi")
SetMTMode(2)
ConvertToYV12(interlaced=true)
Import("C:\Program Files\AviSynth 2.5\plugins\QTGMC3.10.avsi")
spline36resize(720,576)
QTGMC( Preset="Slow", EdiThreads=8 )
spline36resize(720,432)


I'm getting 34.10 fps with "Slow" preset on i7's

Tnx Vit again :))

p.s. Vit, this script, is it correct for 720x576 ( DV-PAL-interlaced 25.000)-->720x432?

[-Vit-]

Of course, you're supposed to use mt_masktools-25.dll for 2.58MT. But I've heard this story before, the other way round - needed to use mt_xx-25.dll when using 2.6MT. There's only one rule to get best MT avisynth stability - try every version of everything.

I assume you've tweaked the thread counts. I often find > 8 is best for SetMTMode at "Slow" (I use as high as 14 in some cases), and recently I've noticed better results with EdiThreads lower (4 or less). However, these values will likely vary widely with different systems.

The output resolution is a little unusual (5:3), but I can't tell you what is correct though - depends on the source and how you intend to encode it / what you are using it for.

[@MeGui]

The output resolution is a little unusual (5:3), but I can't tell you what is correct though - depends on the source and how you intend to encode it / what you are using it for.

Then what res. should i use to get nice 16:9 as final PAL video?

This is my input:

Video
ID : 0
Format : DV
Format_Commercial_IfAny : DVCPRO
Codec ID : dvsd
Codec ID/Hint : Sony
Duration : 46s 320ms
Bit rate mode : Constant
Bit rate : 24.4 Mbps
Width : 720 pixels
Height : 576 pixels
Display aspect ratio : 4:3
Frame rate mode : Constant
Frame rate : 25.000 fps
Standard : PAL
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Interlaced
Compression mode : Lossy
Bits/(Pixel*Frame) : 2.357
Stream size : 159 MiB (95%)
Encoding settings : wb mode= / white balance= / fcm=auto focus

Should i add SelectEven()?

p.s. i'll keep current treads..works nice so far :)))

Tnx again mate :)

[pokazene_maslo]

Do you have any ETA when will be chroma planes denoising supported?

[sadie]

I've uploaded that miserable little 3 second clip.
http://rapidshare.com/files/448099289/Arg.avi

Yesterday I got the error with a sourcematch 25p test, so there goes that theory down the drain. I will try your new update tonight along with newly installed setMT. SetMTmode (5,4)>DirectShowSource>SetMtmode (2,4) sound OK for my core duo? Or perhaps down a notch. Since x264 has its own multi-thread settings I've wondered if there might be some interference going on there? Or should I even just go back to standard two-pass mode and forget about CRF. Lossless will have to wait for the next project and new hard disk.

I also want to reiterate that when my bug rears its head, it can just as easily disappear in the next sequence if there's adequate movement going on. I believe that Didee even mentioned straight out somewhere on this thread that TGMC wasn't ever made for static scenes? Or is that to be nuanced?

[-Vit-]

@@Megui: I'm sure there are many threads here about resizing. I work with NTSC material, SD at 720x480. I don't resize, instead encode as anamorphic 16:9 and let the player do the resizing. This keeps the original detail and doesn't bloat the final encode size, but relies on the player's resizers. I imagine a similar process would would work for you, but you'll have to find out for yourself. If you really want to explicitly resize, then 1024x576 would lose no vertical detail, but the encode will be somewhat larger.

Use SelectEven if you want a 25fps result. It's close to film rate and some people prefer that look. It can make for a smaller and quicker encode than 50fps. However, a 25fps result can look juddery depending on the source. You could use the new motion blur feature to help with that (in which case you don't use SelectEven, you set FPSDivisor=2 in QTGMC, along with the motion blur settings). Motion blur will slow the encode, but might reduce filesize a little.
____

@pokazene_maslo: Chroma/pixel format features are next on my list (as long as I don't go off on a whim as I just did with the motion blur). But currently I'm looking at Avisynth itself, in particular the caching, which is starting to become a serious stability problem with more complex settings. I'm also busy with several other projects. So I don't currently have an ETA, although I am quite active with QTGMC at the moment...

[-Vit-]

@sadie: DirectShowSource: Can be problematic, could explain any number of ills. I've seen strange ghosting with it sometimes. Try FFVideoSource?

EDIT: This might be a source filter issue. Comparing DirectShowSource to FFVideoSource on this vid, no other processing. FFVideoSource shows some horizontal lines - under the fountain and red mark in the first 4 or 5 frames, DirectShow has some flickering in that bright area but nothing so problematic. However, that was all. [DSS2 matches DirectShowSource, but it got the parity wrong and I had to use ConvertToYV12 - is there a DSS2 setting to set a preferred pixel format?]

Encoded both directly to MKV and via lossless, no other visual issues apart from that odd difference despite running several versions testing source accuracy. As expected, the most accurate to the source was:
QuickTGMC( Preset="Slower", SourceMatch=3, Lossless=2, Sharpness=0.1 )
EDIT2: Here's the result from the above line (http://www.mediafire.com/download.php?yy9fjty5i2esrrb) (x264 in MKV).

[I've reupped a cleaner version done with new v3.11]. I went for source accuracy. Everything was done with 10 main threads and 4 EdiThreads.

[wOxxOm]

I wonder how do I set qtgmc parameters to avoid ugly artefacts in static fades from/to black like these: qtgmc (http://i15.fastpic.ru/big/2011/0216/73/ac3acbc6e2556ac04b5ef88829475973.png) vs daa (http://i15.fastpic.ru/big/2011/0216/92/7b3014c14bff35fc69499f8650b86692.png) - changing Preset or SourceMatch=1/2/3 do not help...

[Didée]

Post your script. It's not clear what exactly you are doing there. In particular, since it seems you're actually doing IVTC ....

Fades generally are problematic. As far as MVTools are concerned, try different DCT modes - they can help to get better matches. (For some fun, try blocksize 16 together with DCT 1. No, nothing is broken ... it *IS* *THAT* slow. Really.)
But even then, it's still a fade ...

[wOxxOm]

IVTC(TFM) is a nonfactor here, it was used for debugging purposes, the script is plain simple:
DGDecode_mpeg2source("dvd1.d2v",info=3)
QTGMC( preset="very slow", SourceMatch=3 ).selectEven()

I've tried blocksize=16 & dct=1 (it was sloooooow) - slightly better (picture was kinda posterized/banded but without previously seen ghosting/blending artefacts), anyway it is far from being as clean as daa(), then avisynth hanged when I added sourcematch=3. Seems like I have to find manually every little fade throughout the whole movie.

[Didée]

Darn fades. Well ... if it was about TGMC, you could set low SAD thresholds like thSAD1=64,thSAD2=32, which probably would solve the problem. In QTGMC, those aren't exposed anymore.

May I ask what you're trying to achive, at all? For most Anime-content scenarios, Q/TGMC isn't the most-suited-filter-of-all, anyway.

[DVDBob]

Which interlace method, can this filter deinterlace??

[-Vit-]

Oops... I introduced a nasty bug in v3.05. The final couple of core algorithm stages were being skipped over including the final smooth, which left the output noisy. Fixed in the OP (http://forum.doom9.org/showthread.php?p=1423459#post1423459)(v3.11). If you have version 3.05 or 3.10, you really need to update to v3.11 as it was a serious bug. Sorry for the inconvenience.

Dunno if that was partly causing wOxxOm's problem (although Didée's points still hold regardless). Don't think it caused sadie's smears, the bug caused noise.
____

Given the above posts, I have allowed control over thSAD1, thSAD2, thSCD1 and thSCD2 (which are the same as TempGaussMC_beta2):
- ThSAD1/2 control how willing the temporal smooths are to blend with the neighbor frames. Increasing these values will increase smoothness, but can also can cause blurring or smearing. Conversely, reducing these values can reduce ghosting, or the blurring of high-speed temporal changes, but may leave shimmer or noise. In any case, they should be used rarely, and ThSAD2 is safer to use than ThSAD1.
- ThSCD1/2 control scene change detection. Again only change them if you must (read the MVTools documentation for details).

QTGMC( Preset="Slower", TR2=2, ThSAD2=400 ) # Smoother but might smear...
Defaults are: ThSAD1=640, ThSAD2=256, ThSCD1=180, ThSCD2=98
____

DVDBob: I'm not sure I understand, what other interlacing methods are there?

[SubJunk]

Hi -Vit-, thanks so much for this script! It is really great.
Also thanks for keeping it updated.
I have what is hopefully a quick question :)

I have an interlaced video at 706x480 which I want to resize to 706x528.
I've noticed that doing the resize before calling QTGMC produces better results than afterwards (it is a cartoon, so some of the lines connect up better this way) but then there are fat horizontal lines every 5 frames which aren't there if I resize after QTGMC (I attached a screenshot to show it)
I did of course expect that something wrong would happen if I resized it beforehand but I was curious if the interpolation would like the extra resolution - which it did.

So my question is: Is there somewhere I can put the resize line in your script so it will work properly?

The script I used for that screenshot is (with irrelevant things removed for clarity):


PointResize(706,528)
QTGMC()

[kypec]

I have an interlaced video at 706x480 which I want to resize to 706x528.

PointResize(706,528)
QTGMC()
Changing vertical resolution before the frame(s) were deinterlaced is plainly wrong. No deinterlacer can reliably re-construct the frame that had its fields resized vertically. Period.

[wOxxOm]

For most Anime-content scenarios, Q/TGMC isn't the most-suited-filter-of-all, anyway.I always use it as clip2 for tfm with pp=4, and sometimes for scrolling credits without tfm.

[SubJunk]

Changing vertical resolution before the frame(s) were deinterlaced is plainly wrong. No deinterlacer can reliably re-construct the frame that had its fields resized vertically. Period.Maybe you should read the whole post before replying :)

I did of course expect that something wrong would happen if I resized it beforehand [...] Is there somewhere I can put the resize line in your script so it will work properly?

[Gavino]

Any vertical resizing must be done after deinterlacing.
Why are you using PointResize? That is almost bound to introduce artifacts.

[SubJunk]

I was using Spline36Resize but I changed it in the post to the most simple one in order to avoid comments about which resizing filter is better
I obviously worded my first post in a confusing way, I'll try again. Hopefully this is clearer:


Hi -Vit-, thanks so much for this script! It is really great.
Also thanks for keeping it updated.
I have what is hopefully a quick question :)

I have an interlaced video at 706x480 which I want to resize to 706x528.
I've noticed that doing the resize before calling QTGMC produces better results than afterwards (it is a cartoon, so some of the lines connect up better this way) but then there are fat horizontal lines every 5 frames which aren't there if I resize after QTGMC (screenshot here (http://forum.doom9.org/attachment.php?attachmentid=12032&d=1297841165))
I did of course expect that something wrong would happen if I resized it before the deinterlacing but I was curious if the interpolation would like the extra resolution - which it did.

So my question is:
Is there somewhere I can add a resize line to your script - after deinterlacing but before the interpolation and postproc - so it will work properly?

The script I used for that screenshot is (with irrelevant things removed for clarity):


Spline36Resize(706,528)
QTGMC()

[Gavino]

Are you sure your source is really interlaced?
A glitch every 5 frames suggests that it is telecine material and should be IVTC'd rather than deinterlaced.

[DVDBob]

Oops... I introduced a nasty bug in v3.05. The final couple of core algorithm stages were being skipped over including the final smooth, which left the output noisy. Fixed in the OP (http://forum.doom9.org/showthread.php?p=1423459#post1423459)(v3.11). If you have version 3.05 or 3.10, you really need to update to v3.11 as it was a serious bug. Sorry for the inconvenience.

Dunno if that was partly causing wOxxOm's problem (although Didée's points still hold regardless). Don't think it caused sadie's smears, the bug caused noise.
____

Given the above posts, I have allowed control over thSAD1, thSAD2, thSCD1 and thSCD2 (which are the same as TempGaussMC_beta2):
- ThSAD1/2 control how willing the temporal smooths are to blend with the neighbor frames. Increasing these values will increase smoothness, but can also can cause blurring or smearing. Conversely, reducing these values can reduce ghosting, or the blurring of high-speed temporal changes, but may leave shimmer or noise. In any case, they should be used rarely, and ThSAD2 is safer to use than ThSAD1.
- ThSCD1/2 control scene change detection. Again only change them if you must (read the MVTools documentation for details).

QTGMC( Preset="Slower", TR2=2, ThSAD2=400 ) # Smoother but might smear...
Defaults are: ThSAD1=640, ThSAD2=256, ThSCD1=180, ThSCD2=98
____

DVDBob: I'm not sure I understand, what other interlacing methods are there?

What I mean is that there are some movies that are just interlaced, other films are partially interlaced, hybrid interlaced etc.

[Didée]

Regarding "what interlacing types can be processed" and "can I use for it IVTC" ...

What needs to be understood is: the consequences of the temporal blur used before the motionsearch.

This temporal blur is ...

a) ... ultimately necessary, it's THE key feature. Without it, Q/TGMC would not be what it is.

b) ... only well suited for "true" interlaced input, where each field represents a unique point in time. This is NOT the case for progressive input, nor for telecined input, nor for fieldblended input.

The problem with input that isn't of the "true" interlacing type is that the distribution of "temporal motion states" is not evenly distributed. In effect, the preprocessed searchclip will exhibit a certain kind of left-right-"wobble", which in turn will fool the motionsearch. (And if the motionsearch is bad, then everything is bad.)


Summed up: The method generally is made up only for "true" or "video-type" interlacing. With all other interlacing types, there be dragons. It might work out, or it might fail.


Clinic example: a grey square (with some letters & a vertical line) moving at constant speed.

http://img684.imageshack.us/img684/1797/tgmcsearchclip2inputtyp.th.png (http://img684.imageshack.us/i/tgmcsearchclip2inputtyp.png/)

Only in the case of "true" interlacing all frames look "the same". With the other types, the frames are changing their shape in a way that MVTools can not reasonably deal with.

(For clarity, this example is without the "rep0" feature. Using rep0 will lessen the problem to some extend, but it will not eliminate it.)

[sadie]

Ok, well 3.10 did succeed in holding things together and I managed to get a full crf 20 encode with SM=2, Sh=.3 settings. Then I looked and thought, boy, these are awfully sharp, rather...noisy sharp in fact. Then I go on-line and read about version 3.11. Grrr. Vit, ever heard of Murphy's law in Russia?

So back to square A. Maybe not in vain, as I had a few surprises.

http://rapidshare.com/files/448263181/A_Nervous_fingers.avi
http://rapidshare.com/files/448263182/B_Nervous_plastic.avi
http://rapidshare.com/files/448263180/C_Missing_frame.avi

Clips A & B are still captures that as you'll see are bit quite a bit more animated than your average noise. In a way kind of comical. Clip C missing frame syndrome (er I should have named it 'extra frame' is found in 3 places in the video). In fact, I say 'missing' because in a frame-by-frame search in Premiere, Pinnacle Studio and Virtualdub it simply cannot be found. Only Avidemux spotted it and of course when I chopped them out and direct stream saved the file they came right back (the ol' keyframe limitation). Also, neither my interlaced dvd copy nor an earlier 25p encode manifest it either. It's a purely full-rate 50p issue. I mean in three weeks of editing never saw it. As the French say, a true 'piege a con'. That said, all three had one trait in common, the sequences had been pre-rendered in reverse motion. Any thoughts or workaround? Another 12 hour encode for three lousy frames seem a bit much.

So before discovering your 'bug' I'd thought to do a follow-up 25p encode with the default sharpness=0.20 to correct my supposed error, but I'm not sure now. Is there that much of a difference .3 vs .2 or would your goof be the over-riding factor for that hardened image I got?

Is DirectShowSource really such a drag? In the past with mpeg4-asp I alternated with avisource indifferently. No probs.
Only here virtualdub wouldn't let me open as such, even if I believe I've got the DVCPro DV decoder by Sony. HenryHo recommended Cenocida as a solution. Why not, though I can't see how that would effect such things as smearing. Oh well, onwards.

[-Vit-]

Yup, QTGMC likes "real" predictable motion with true interlacing. I never work with anime or unpleasant hybrid sources (I looked at YATTA once... I had to go lie down) so QTGMC is never tested on such material. I occasionally work with game footage - the same issues can apply where the motion is jumpy or artificial. Source match will not help in these cases, it will probably make things worse as it does double or triple the temporal work

As an adjunct to Didee's comments though - progressive input will work in QTGMC if you use the InputType setting. The problem Didee illustrated is what happens if you don't set InputType and treat progressive material as though it were interlaced - the algorithm will (inappropriately) try to separate fields and interpolate the result. With InputType=1, those steps are skipped, leaving a correct result more like the interlaced case. With InputType=2,3 the source is reweaved into a true interlaced stream, also correct for TGMC processing (loses half the "fields" in the process though). However, despite these adjustments it's only appropriate to use progressive QTGMC if you're trying to remove some kind of shimmering post-deinterlacing artefacts. Q/TGMC is designed to interpolate and remove bob-shimmer. Progressive doesn't need interpolation, so if you don't have bob-like-shimmer then there's no reason left to use QTGMC - likely you just want a temporal smooth of some kind.
____

sadie: The bug in v3.05 and v3.10 created a noisier, sharper result, probably greater than the difference between 0.2 and 0.3 sharpness. The bug was worse if TR2 was higher (=slower presets).

Regarding source filters, I know no of no rationale for when they decide not to work, nor any workaround other than try every possible filter. Or pre-render to lossless in any package that can successfully deal with the vid. Rendering via lossless would have helped you - it's then easy (if rather manual) to reprocess the few problem section, splice the result in another script then encode to x264.

[sadie]

Yeah, I gotcha. Finally, to do this stuff with this level of sophistication I'm really going to have to get out of the mindset that pre-processing and encoding are a one-stop all inclusive operation. They really should be treated as two entirely separate stages in the workflow.

[SubJunk]

Are you sure your source is really interlaced?
A glitch every 5 frames suggests that it is telecine material and should be IVTC'd rather than deinterlaced.You're right, it turns out that I had improperly IVTC'd the clip. Thanks :)

[osgZach]

Is specifying progressive input appropriate enough to get the smoothing/resharpening effects without getting other funky artifacts on IVTC'd material?

I suppose that what I should have asked earlier back when I jumped into this thread. It is really helpful on sources that are slightly blurry, and also have some aliasing present.
I haven't really been able to duplicate the result I get from QTGMC with other traditional sharpening filters.

[fusebyte]

HY,only a little question.
Can I use QTGMC to de-interlace a MPEG-TS HDTV 1080i file to 1080p MKV file?

Thanks

[-Vit-]

Yes, you can use QTGMC to deinterlace 1080i to a 1080p MKV. As it can be a demanding script you're recommended to use a more modest preset depending on the power of your system (it will still look good at HD). Using multi-threaded Avisynth will help the speed (see the OP for instructions), but you should start low with the number of threads

[dragon_warrior]

I'm using QTGMC to deinterlace 2 videos, one is MPEG-TS and the other is H264-TS then joining them into one. But when the encoding goes for about 6% I keep getting a disturbing crash along with an error, which tells "x264.exe has stopped working...fault module name: avisynth.dll"
What can I do to get rid of it? Thanks.

[fusebyte]

Yes, you can use QTGMC to deinterlace 1080i to a 1080p MKV. As it can be a demanding script you're recommended to use a more modest preset depending on the power of your system (it will still look good at HD). Using multi-threaded Avisynth will help the speed (see the OP for instructions), but you should start low with the number of threads

Thanks for answer.
I don't understand the best script setting for deinterlace 1080i to 1080p.
Maybe...
DirectShowSource("MyInput1080i.ts")
QTGMC( Preset="Slow", SourceMatch=3, Lossless=2 ,EdiThreads=4 ) # I have a Quad core 6600 G0
...need some other parameters?

and more important, How can i start my AVS script?
Which GUI can i use?
I tried MEGUI but give me error at line 2 ( QTGMC( Preset="Slow", Sourc....)

All needed DLLs are downloaded and stored in AVISYNTH folder.

Thanks in advance for Help.

[TheSkiller]

and more important, How can i start my AVS script?
Which GUI can i use?

Try VirtualDub (http://www.virtualdub.org/).


All needed DLLs are downloaded and stored in AVISYNTH folder.

Put them into the /AviSynth/plugins folder.

Also, you can gain a lot of speed if you switch to a multithreaded AviSynth and add SetMTmode right before QTGMC.

[fusebyte]

Yes,pardon I mean Avisynt/plugin folder.


Do U mean like this?
DirectShowSource("MyInput1080i.ts")
SetMTmode QTGMC( Preset="Slow", SourceMatch=3, Lossless=2 ,EdiThreads=4 ) # I have a Quad core 6600 G0

Can I also set the .MKV output file?

I read the instructions carefully but I think I need an AVS script sample to better understand conversion from 1080i 29,970 fps to 1080p fps 23,976
and allow the same resolution and video bitrate.
TS file is "true interlaced" not Telecined.

Tk's

[-Vit-]

There is info on basic usage and multi-threaded usage near the top of the first post of this thread (http://forum.doom9.org/showthread.php?p=1423459#post1423459) and near the top of the script itself. There are example scripts in there. Please read that information first. If you still don't understand then you need to learn about AviSynth (http://avisynth.org/mediawiki/Main_Page).

I would advise against source-match/lossless on 1080i material. It will make a difference, but a very small difference at that resolution. In any case, on your system the encoding speed will be very slow with those settings. You need a top end system and a lot of MT encoding experience to do that. I suggest you start with Preset="Fast", EdiThreads=1 and work up from there. You can do a test encode by adding a trim on the end of your script (e.g. "trim(0,999)" for 1000 frames).

I would recommend MeGUI (http://sourceforge.net/projects/megui/) instead of VirtualDub, it seems to cope better with the most demanding scripts. If you have the disk space, then a good approach is to check the "Add pre-rendering job" button, which will do the QTGMC to a lossless avi first, then x264/mkv the result in a second pass. You will need at least 200Gb for an hour of 1080p footage though. If you have to do the QTGMC and encoding in one pass then you may need to reduce the number of threads.

QTGMC will give you a 59.94fps output by default. If you add "SelectEven()" at the end of your script you will get 29.97fps. There are methods to convert to 24p but (a) I never do that so I'm not going to advise and (b) you need to have a good reason to do it because the results might not be so good.
____

dragon_warrior: Sounds like the usual avisynth-doesn't-like-mt crash to me. Try adding a SetMemoryMax (see the multithreading section in the first post). Try different versions of MT avisynth (linked on the AviSynth wiki). Make sure your plugins are up to date. Reduce the number of threads or preset. Set EdiThreads to 1. Try a different machine.

[dragon_warrior]

Yes,pardon I mean Avisynt/plugin folder.


Do U mean like this?


Can I also set the .MKV output file?

I read the instructions carefully but I think I need an AVS script sample to better understand conversion from 1080i 29,970 fps to 1080p fps 23,976
and allow the same resolution and video bitrate.
TS file is "true interlaced" not Telecined.

Tk's

Personally I wouldn't prefer DirectShowSource, unless it is a last resort. If you have to use this, try adding 'audio=false' to prevent unnecessary audio loading. Also, AssumeFPS() should be called to avoid an outta-sync output, in your case, that would be AssumeFPS(29.97)
Anyway, my choice for TS is ffms. I'm also encoding 1080i materials to 720p. Unfortunately, TS container is not really friendly with ffms so you need to remux your TS to MKV before putting it into your script. Last but not least, don't forget to state the fps, which is "fpsnum=30000, fpsden=1001" (30000/1001 ~ 29.97)
Hope this help

[fusebyte]

Tk's Vit.

so you need to remux your TS to MKV before putting it into your script
I do it with DSMUX ( input = TS , output = MKV ) but the MKV remains interlaced like normal as is, it's only a container transcoded.
It's not a real problem except for some things.
One of these, for example, can't add SRT Forced track.
I know I can use PGS or m2ts+srt m2ts's same name but i like x264,Mkv and SRT . I find them are more flexibles for my project.
Unfortunately, TS container is not really friendly
...right! sorry if I repeat myself,but I would wish to find better deinterlacer from 1080i 29,970 fps to 1080p fps 23,976 and allow the same resolution and video bitrate.
About FFMS U mean like this?
ffmpegsource("film.mkv",vtrack=-1,atrack=-1)
Thanks so much.

[dragon_warrior]

I do it with DSMUX ( input = TS , output = MKV ) but the MKV remains interlaced like normal as is, it's only a container transcoded.
It's not a real problem except for some things.
One of these, for example, can't add SRT Forced track.
I know I can use PGS or m2ts+srt m2ts's same name but i like x264,Mkv and SRT . I find them are more flexibles for my project.

Is SRT the subtitles? Your remuxed MKV should include only the video stream with no audio and no subtitles. After it's processed, you can mux it again.

ffmpegsource("film.mkv",vtrack=-1,atrack=-1)
Yep, that's it

...right! sorry if I repeat myself,but I would wish to find better deinterlacer from 1080i 29,970 fps to 1080p fps 23,976 and allow the same resolution and video bitrate.

As -Vit- already stated, if your source is pure interlaced, QTGMC will do a good job, which result in a 59.94 FPS (or 29.97 FPS) output. Don't force it to be 23.976
Otherwise, if your source is telecined, then IVTC is the right option. It's to remove 1 frame every 5 frames. Let's do a little calculation, 29.97 - 29.97*1/5 = 23.976, and that's the FPS you get after IVTC'd.

Do analyse your source properly before deciding what to use


-------------
@-Vit-: It seems ffms 2.15 mt is the cause, I try switching back to the former version and surprisongly it works for me. Thanks

[fusebyte]

Is SRT the subtitles? Your remuxed MKV should include only the video stream with no audio and no subtitles. After it's processed, you can mux it again.
It's no possible.If I remux DsMux's output mkv file with i.e. MkvMerge to add Srt sub or another audio track, video is playing slowed down, and 'cause of this reason, audio goes desyncrinized.
Why video remuxed with MkvMerge or another GUI plays so bad? It's very very slowed.I can't touch video track anyway after DSMUX coonversion, if I do it,video
will always have playback problem.
If I could work on the file after being processed by DSMUX not have more problems.

Do U know a solution?


Do analyse your source properly before deciding what to use

I use Mediainfo.
Can read if prog. or Interl.,say me if YUV 4.2.0 or other, as U know.
But I ask U about interlaced file best way to analize right frame sequence, if 3:2 or i.e. ibrid 2i-4p-3i-4p or other sequence and how set output parameters.
(i.e =
if 3:2 telecined remove pulldown using telecide,
if "true interlaced" only change container,
if some sequence ibrid like example ??? )

thanks in advance.

[aegisofrime]

It's no possible.If I remux DsMux's output mkv file with i.e. MkvMerge to add Srt sub or another audio track, video is playing slowed down, and 'cause of this reason, audio gone desyncrinized.
Why video remuxed with MkvMerge or another GUI plays so bad? It's very very slowed.I can't touch video track anyway after DSMUX coonversion, if I do it,video
will always have playback problem.
If I could work on the file after being processed by DSMUX not have more problems.

Do U know a solution?



I use Mediainfo.
Can read if prog. ot Interl.,say me if YUV 4.2.0 or other, as U know.
But I ask U about interlaced file best way to analize right frame sequence, if 3:2 or i.e. ibrid 2i-4p-3i-4p or other sequence and how set output parameters.
(i.e =
if 3:2 telecined remove pulldown using telecide,
if "true interlaced" only change container,
if some sequence ibrid like example ??? )

thanks in advance.

Mediainfo isn't accurate. Not is DGIndex. I have came across DVD VOBs that were marked as interlaced in both Mediainfo and DGIndex, yet upon QTGMC deinterlacing and visual inspection it's clear that it was actually progressive.

[Didée]

MediaInfo is accurate. As is DGIndex. Such Tools report how the stream was encoded, not what the actual content is like. It is quite common that progressive content is encoded in interlaced mode.

[aegisofrime]

MediaInfo is accurate. As is DGIndex. Such Tools report how the stream was encoded, not what the actual content is like. It is quite common that progressive content is encoded in interlaced mode.

Yes, that's what I meant. Thanks for the clarification :)

[fusebyte]

To better understand,forgive my less experience,
this is Mediainfo analyzed content of my MPEG-TS 1080i.TS but don't see info about Frame sequence.


Generale
ID : 1 (0x1)
Complete name : D:\Documents and Settings\Administrator\Desktop\Working\MyHDTV_1080i.ts
Format : MPEG-TS
File size : 16,0 GiB
Duration : 2h 3min
Overall bit rate : 18,6 Mbps
Maximum Overall bit rate : 35,5 Mbps

Video
ID : 4113 (0x1011)
Menu ID : 1 (0x1)
Format : MPEG Video
Format version : Version 2
Format profile : Main@High
Format settings, BVOP : Si
Format settings, Matrix : Predefinited
Format settings, GOP : M=3, N=15
Codec ID : 2
Duration : 2h 3min
Bit rate mode : Variable
Bit rate : 17,1 Mbps
Nominal bit rate : 80,0 Mbps
Width : 1 920 pixel
Height : 1 080 pixel
Display aspect ratio : 16:9
Frame rate : 29,970 fps
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Interlaced
Scan order : Top Field First
Bits/(Pixel*Frame) : 0.275
Stream size : 14,8 GiB (92%)

[firebo14]

http://bigpichost.com/files/error_6ufxjhh2.png
http://bigpichost.com/files/error__0y35luzl.png

I cant seek the video (vid doesnt show) nor save the video. Am i missing plugins or wrong script?

Someone help please... i really want to deinterlace the vid with double fps.

Here is my .avs script:
FFVideoSource("C:\Documents and Settings\Extra\Desktop\New Folder\Gantz.2011.1080i.DD.5.1\Gantz.2011.1080i.DD.5.1.ts")

QTGMC( Preset="Slower" )


My video interlace type is:
Scan type : MBAFF

Is it possible to do triple fps or more? If yes, please tell me... I love the smoothness of high fps.

Thank you for the deinterlacing script.. i really want to use it... :]

http://tehparadox.com/forum/customavatars/avatar1997040_1.gif

[Didée]

1) "You shall not have multiple versions of the same plugin in the autoload folder."

- Keep RemoveGrainSSE2.dll. Delete all other RemoveGrain(XXX).dll files.
- same for Repair.dll
- same for Rsharpen.dll
- similar for mt_masktools-26.dll

2) Remux *.ts to *.mkv. FFmpegSource can have problems with *.ts container. (not necessarily, but it can happen)

3) Browse through the last few pages. -Vit- has given lots of useful pointers about stability. :)

[firebo14]

Thanks for the quick reply... i will try what you said.. now...
:]

Edit 1:
Thanks alot!.. it seems to be working, but i have to wait until i see if it works...

http://bigpichost.com/files/test_tj59_ogl7t580.png


Wow, 0.07 fps. This is going to take a while.. T-T

[Mini-Me]

Thanks for the quick reply... i will try what you said.. now...
:]

Edit 1:
Thanks alot!.. it seems to be working, but i have to wait until i see if it works...

http://bigpichost.com/files/test_tj59_ogl7t580.png


Wow, 0.07 fps. This is going to take a while.. T-T

OMFG. Is that 0.07 FPS of input or 0.07 FPS of output? If we're talking 0.07 FPS of 60 FPS output (which is probably the case), that would take 23 years for all 235 hours of my footage. I mean, not that 11.5 years is much better, but...wow. I'd better get started NOW. :eek:

[Boulder]

The higher the resolution, the more likely you could get away with using "lower quality" presets which are much faster. This issue has also been discussed in this thread a few times.

[Mini-Me]

The higher the resolution, the more likely you could get away with using "lower quality" presets which are much faster. This issue has also been discussed in this thread a few times.

Qualität über alles! What's a couple decades, anyway? I can wait.

Seriously though, I hope that firebro14 is testing this on an Atom netbook or something. He may be deinterlacing 1080i, but that 0.07 FPS isn't even using the very slow or placebo presets. :eek: firebro, would you mind posting your specs?

Also, does anyone know the time complexity of the filter with respect to resolution? Is it linear in terms of source or output pixels or lines, or is it higher order? (This might have been mentioned in the discussion you refer to, but just in case anyone knows off the top of their head...)

[-Vit-]

Most operations are per-pixel, but there is the motion search and NNEDI3, the complexity of which depends on your settings. It's a bit tricky to give a practical complexity scaling because it depends on hardware, choice of settings, number of threads (even if you single-thread QTGMC, NNEDI3 uses threads) and whether you're simultaneously trying to x264 the result or encoding to lossless. CPU underutilization can cause low encoding fps on faster machines for HD, which would make fps comparisons with SD unrepresentative. Many people's fps will scale worse than ideal because they haven't tweaked for all these factors.

However, on preset="Slower", with properly tweaked multi-threaded scripts using 100% CPU, I get ~42fps SD and ~6fps HD (i7 930@3.7Ghz). So the scaling there seems to be linear per-pixel, about 1.2x. Edit: That was in MeGUI, encoding to lossless (via mencoder, they've recently changed to ffmpeg, which is slower). So you would need to add an x264 (or whatever) encoding pass, but the speed of that would be pretty much independent of QTGMC.

[Mini-Me]

Most operations are per-pixel, but there is the motion search and NNEDI3, the complexity of which depends on your settings. It's a bit tricky to give a practical complexity scaling because it depends on hardware, choice of settings, number of threads (even if you single-thread QTGMC, NNEDI3 uses threads) and whether you're simultaneously trying to x264 the result or encoding to lossless. CPU underutilization can cause low encoding fps on faster machines for HD, which would make fps comparisons with SD unrepresentative. Many people's fps will scale worse than ideal because they haven't tweaked for all these factors.

However, on preset="Slower", with properly tweaked multi-threaded scripts using 100% CPU, I get ~42fps SD and ~6fps HD (i7 930@3.7Ghz). So the scaling there seems to be linear per-pixel, about 1.2x. Edit: That was in MeGUI, encoding to lossless (via mencoder, they've recently changed to ffmpeg, which is slower). So you would need to add an x264 (or whatever) encoding pass, but the speed of that would be pretty much independent of QTGMC.

Wow, that's a night and day difference compared to firebro14's setup with the same preset. It's almost 100x faster, so I imagine he must have been directly encoding the result with e.g. x264, using a weak computer with single-threading and a very unoptimized...something. I was kind of shocked when I saw his FPS, and I thought maybe the recent versions with source match (which I don't have yet) might have taken heavy hits at slower presets or something.

Anyway, thanks for your perspective on the scaling. :)

[Didée]

Try any "slow" preset with "blocksize=16,search=5,searchparam=8,DCT=1".

0.0xx fps are almost guaranteed, even with SD content. ;)

[fusebyte]

I tried with EAC3TO. Very good output and remuxable with MKVMERGE to add SRT sub track without any playback problem but I would wish try with QTGMC encoder
and don't understand how start avs script with cmd QTGMC inside.
I always obtain errors, I'm sure that's just my fault.

[dragon_warrior]

I tried with EAC3TO. Very good output and remuxable with MKVMERGE to add SRT sub track without any playback problem but I would wish try with QTGMC encoder
and don't understand how start avs script with cmd QTGMC inside.
I always obtain errors, I'm sure that's just my fault.
:script:
and
:logfile:

[Lyris]

First time user here.

That is magnificent. Thank you.

[Aktan]

First off, great job on this script. It does wonders. Now I do see it as being very very slow. I'm curious to know if it would be faster if it was combined into one plugin instead. I would think some of the work is redundant because you are using different plugins and they aren't talking to each other.

Minor comment bug report:

for MatchPreset2 table, the Very is on top of Placebo, making it look like "Very Placebo" instead of "Very Slow"

[dirk362]

I get a DLL error with only certain Presets ???

The example AVS below works fine
LoadPlugin("E:\Tools\Applications\DGMPGDec\DGDecode.dll")
SetMemoryMax(1024)
SetMTMode(5,6)
MPEG2Source("D:\temp\inputfile temp files\inputfile.d2v", CPU=6)
SetMTMode(2)
AssumeTFF()
ConvertToYV12(interlaced=true)
Import("C:\Program Files (x86)\AviSynth 2.5\plugins\QTGMC3.11.avsi")
QTGMC( Preset="Slower", EdiThreads=3 )
SelectEven()
However, if I change to
QTGMC( Preset="Very Slow", EdiThreads=3 )
I get the error message of
FFT3DFilter: Can not load FFTW3.DLL !
(C:\Program Files (x86)\AviSynth 2.5\plugins\QTGMC3.11.avsi, line 632)
(D:\temp\inputfile temp files\inputfile.avs, line 9)
This occurs for Placebo and Very Slow only. All other presets work fine.
I'm fairly confident I've not messed up any of the pre-reqs. And the file does exist in both the plugins directory for AviSynth and also in the path.

Any ideas anyone ?

Thanks in advance

[dirk362]

I get a DLL error with only certain Presets ???

FFT3DFilter: Can not load FFTW3.DLL !
(C:\Program Files (x86)\AviSynth 2.5\plugins\QTGMC3.11.avsi, line 632)
(D:\temp\inputfile temp files\inputfile.avs, line 9)

OK - I've been a bit thick for which I apologise. The issue was that as this is 64-bit Windows I need to have FFTW3.DLL in the C:\Windows\SysWOW64 directory (even though that's not in the path as an environment, the OS sorts this out by recognising it's a 32-bit app).

So thought I'd post for those on 64-bit OS but still using 32-bit AviSynth that you need to think about SysWOW64 directory as well. Doh !

[-Vit-]

Atkan: it's only as slow as your preset and computer. Seriously though, regarding plugins "talking to each other", there's not much redundancy in the script - that's not really the problem. It's the enormous number of operations that makes it slow. Some ideas I have for speed are:
- Using the GPU to do the simpler per-pixel tasks
- Using simpler algorithms for the chroma deinterlacing
- Reducing the cache load, to improve performance on very complex settings
I'm working on all three of those ideas.

Writing a QTGMC-custom plugin might bring some benefits, most notably being able to combine some of the simpler operations. That is partly what I'm intending to do with the GPU point above...

EDIT: Well spotted on that typo! Hmmm, "Very Placebo", I'm sure there are some people who would like that option...
____

dirk362: easy mistake to make - I've updated the requirements in the OP to briefly note the issue

[kypec]

EDIT: Well spotted on that typo! Hmmm, "Very Placebo", I'm sure there are some people who would like that option...

Sorry for being such numb but what exactly is affected by this bug (seeing that you did not mention any bug-fix released yet)?
If there is preset "Very Slow" used in the script, does it work as intended or not?:stupid:

[-Vit-]

It was just a mistake in a comment. No effect on the script.

[Vitaliy Gorbatenko]

Bad way to connect their modules through the autoload and :\windows\* directory
It is better to copy its modules to the selected directory, for example:
x:\Knot\AviSynthPluginsx64\
and connect them in a script:
SetWorkingDir ("x:\Knot\AviSynthPluginsx64\")
LoadPlugin ("average.dll")
...
and so on.
And with the re installation of the system problems will not arise and can be accurately select Plug-ins, and syntax is reduced.

PS: but this is my personal opinion.

[-Vit-]

I assume that post refers to the FFTW3.dll issue above... FFTW3.dll is not itself an avisynth plugin. It's a module used by FFT3DFilter. You can't use LoadPlugin on it. Using SetWorkingDir alone will work (as long as no plugin or other script changes the working directory... :eek:)

[fusebyte]

My wish?
Some expert makes a GUI for QTGMC.
I honestly am not able use it... just my fault.

greetings

[Dogway]

@fusebyte: be a bit more thankful by turning the complain into a petition. Its a hard work of one guy for free, but GUI? this is avisynth...

[henryho_hk]

I just notice (or re-discover) a pattern that QTGMC() fails but TGMC() works.

Pick a true intelaced footage (say, NTSC) that has a solid black 1-pixel line at row 478 on every frame. After bobbing, QTGMC() will have the black removed on even frames, and a 2-pixel-high black line on odd frames. TGMC() keeps the 1-pixel black line on every frame.

[-Vit-]

You didn't note your settings [or did you literally use QTGMC()?].
No time to check right now, but maybe Border=true or SVThin=1.0 will restore same behavior. They are defaults changed from original TGMC
[Why? My material never needs Border=true, it can cause occasional artefacts of its own and is a *tiny* bit slower. SVThin - I just don't like the results]

[henryho_hk]

It's TGMC(tr2=3,EDIMode="NNEDI3") vs QTGMC(Preset="Slower",tr2=3). I also notice similar behaviour in sadie's DV sample. His DV seems to leave the topmost line black in some scenes, and after QTGMC(), the black line will disappear and then duplicate on alternate frames.

[-Vit-]

I looked at sadie's example - I don't see a topmost black line - I see a darker line on the second line. And, yes it dances on QTGMC defaults - and the difference from TGMC is that you need Border=true.

The QTGMC docs say of Border: "set true you see flickering on the very top or bottom line of the output" (yep I guess that needs to read 'lines')
There's section at the end of the docs comparing with TGMC which also answers this question.

You set edimode="nnedi3" on TGMC, only modded versions of beta2 support nnedi3, in the original version that will just do a bob. So let's be exact:

TempGaussMC_beta2() is the same as:
TempGaussMC_beta2( 2,2,1, 4,0,4, "eedi2" ) is the same as:
QTGMC( preset="Slower", edimode="eedi2", border=true, svthin=1.0, lsad=1600, precise=true )
[Corrected - thanks Taurus]

The other default differences: Precise switches on/off barely visible steps for a small speed-up. lsad is different because the MVTools documentation is especially confusing on whether that setting should be scaled by blocksize - I had to read the MVTools source code to find that it shouldn't be.

[nhope]

I made a quick reference table of how the presets affect the settings. I'm deinterlacing 1080i so I've used "faster" as the benchmark.

http://dl.dropbox.com/u/21489814/QTGMC-presets.png

[Taurus]

TempGaussMC_beta2() is the same as:
TempGaussMC_beta2( 2,2,1, 4,0,4, "nnedi2" ) is the same as:
QTGMC( preset="Slower", edimode="nnedi2", border=true, svthin=1.0, lsad=1600, precise=true, nneurons=2 )

@Vit
Are you sure TempGaussMC_beta2() uses "nnedi2" as default?
From the TempGaussMC_beta2.avsi:
# Defaults for EDI interpolation
EdiMode = default( EdiMode,"EEDI2")
Same for TempGaussMC_beta2u.avsi mod (nnedi3 support).

[-Vit-]

Taurus: You're right! I changed my version a long time ago to default to nnedi2, and forgot. I've edited the post above. The equivalence is still there, QTGMC supports EEDI2 also.

nhope: Thanks for that. The presets gradually change. The upcoming version has some noise-related changes because I've brought in [motion-compensated] dfttest.

[kypec]

@ nhope & -Vit-
Would be nice if you could upload that preset table and made it available on GoogleDocs (not necessarily editable by others)
I'm sure it could be helpful for further development/discussion/changes tracking of this wonderful QTGMC script, thanks!

[kolak]

3.11 works much more stable.
About 30fps for 60i SD (slow preset) on 2x X5450. 8 cores at 90%.

Used on 60i footage, which had real 60i, badly deinterlaced 60i, 30p and 23.976+pulldown- hahhaha. Final result 29.97p upscaled to HD plays smooth :)

Amazing piece of script :)
Thanks Vit.

Andrew

[aegisofrime]

3.11 works much more stable.
About 30fps for 60i SD (slow preset) on 2x X5450. 8 cores at 90%.

Used on 60i footage, which had real 60i, badly deinterlaced 60i, 30p and 23.976+pulldown- hahhaha. Final result 29.97p upscaled to HD plays smooth :)

Amazing piece of script :)
Thanks Vit.

Andrew

Wait a minute, 30fps for 60i SD for dual X5450s? I know from a report of a fellow QTGMC user that a i7-2600K gets 57fps on slow preset. Granted, that is with encoding to lossless, so perhaps you are using very slow x264 settings? Or perhaps you have not multi-threaded your script properly?

[nhope]

@ -Vit- - I'll try to remember to go back and update it when your new versions come out.

@ kypec - Version with "faster" as the benchmark (https://spreadsheets.google.com/ccc?key=0AtvvpQjptJLFdGxFaEZwNDNXdGliTi1KdElEUEVtTHc&hl=en) & version with "medium" as the benchmark (https://spreadsheets.google.com/ccc?key=0AtvvpQjptJLFdGVGQzFUa0ozWTZJYXItLWFFTURBMXc&hl=en). I just changed the colours to make it easier to visualise the effect of changing the presets, depending on your preset starting point.

[kypec]

@ nhope: Many thanks for your contribution.
@ -Vit-: please copy that spreadsheets links to your first post in this thread for convenience, thank you.

[nhope]

In my script I am following QTGMC with a SelectEven() and a LanczosResize(1280,720). For multi-threading, should I put Distributor() before or after these lines?

[kolak]

Wait a minute, 30fps for 60i SD for dual X5450s? I know from a report of a fellow QTGMC user that a i7-2600K gets 57fps on slow preset. Granted, that is with encoding to lossless, so perhaps you are using very slow x264 settings? Or perhaps you have not multi-threaded your script properly?

It's from Canopus Lossless to Canopus Lossless- no encoding :(

setmtmode(mode=5, threads=8)
avisource("R:\SD.avi",audio=false)
converttoyv12(interlaced=true)
setmtmode(mode=2)
QTGMC( Preset="slow", EdiThreads=4)
converttoyuy2()
selecteven()


EdiThreads=4- tried different values- nothing helps.

Any idea?

X5450 are quite old- way slower than i7.

I was using avisynth 2.5.7.5 MT dll. Updated to 2.5.8 and now it's crashing much more :(

Andrew

[-Vit-]

nhope: Firstly you can do the horizontal resize (to 1280) before the QTGMC call for a good speed up without loss, just do the vertical resize afterwards.

Distributor() goes at the end of the script. You may not need it - it depends what encoding GUI you use. Try it without.

[nhope]

nhope: Firstly you can do the horizontal resize (to 1280) before the QTGMC call for a good speed up without loss, just do the vertical resize afterwards.
Thanks for the tip. I had wondered about that. Sorry to stray off topic but if I am resizing in one direction at a time as you suggest, is Lanczos3 still a sensible algorithm to use? I am downscaling to 1280x720 from 1920x1080 or 1440x1080 anamorphic.

Distributor() goes at the end of the script. You may not need it - it depends what encoding GUI you use. Try it without.
I am using x264 in MeGUI. Do you know if that normally requires Distributor()? I am only using a Core 2 Duo here in 32-bit XP and when I tested multi-threading without Distributor I had a slight speed drop over ST when I included Distributor and a crash when I didn't. I guess it's not worth attempting MT with a Core 2 Duo but I'm writing a QTGMC/MeGUI tutorial that will be read by a lot of quadcore+ users so I want to guide them the right way.

[-Vit-]

All avisynths resizers operate in two passes anyway (horizontal and vertical). So you don't need to do anything different if you split it up like this and there's no performance penalty from resizing twice in this particular way.

MeGUI requires Distributor(). The MT docs in the OP and in the script apply best to those with 4+ cores. With less cores there is less scope for MT tweaking, especially at HD, but if you ever see your CPU is not close to 100%, then you should see if you can get more out of MT (unless you have stability issues).

kolak: Try all versions of MT avisynth, everyone seems to have different stability results. Some find 2.6MT works well.

[jpsdr]

@kolak : Try eventualy xhmikosr builds (if it's not already the case) here (http://xhmikosr.1f0.de/).

[Didée]

Additional oddities stem from the fact that (all!) plugins *should* be compiled with the corresponding libraries of the targetted Avisynth version. A plugin for vanilla 2.5.8 should be compiled differently for 2.5.8.MT should be comiled differently for 2.5.7.MT should be compiled differently for 2.6 should be compiled differently for 2.6.MT.

If not, be prepared to experience problems.

Has it already been noted that Avisynth MT is an unutterable, giant mess?

[dirk362]

This statement about resize horizontal first for speed-up intrigues me ... Can I ask a couple of stupid questions in terms of sequence then ?

1. Should you perform the horizontal resize before QTGMC ?
2. For point 1 should it be new horizontal x cropped height ?

I appreciate these are not totally specific to QTGMC, but it seems people are talking about speed-ups and I'm just not seeing this when I test it. Of course I can perform lots of different tests to come up with the answer although I'm guessing given the posts in this thread so far there is a "known" way to achieve this...

As an example if I have source of 544x576 and have crop of 2,2,-2,-4 this gives me an effective image size of 538x572. If I'm working to 640xwhatever (accepting this is upscaling), should I therefore have something like:-

Crop(2,2,-4,-2)
Spline36Resize(640,572)
QTGMC( Preset="Faster", EdiThreads=4 ).SelectEven()
Spline36Resize(640,352)

[-Vit-]

Ah, no... that trick is only a speed-up if you're downscaling. QTGMC's speed is directly proportional to the number of pixels in the material it is given. So give it the lowest possible resolution. If you're downscaling, doing the horizontal resize first gives QTGMC less to work on. You can't do the vertical rescale before QTGMC because you'd destroy the interlacing.

However, if you're upscaling you should QTGMC the original smaller resolution.

[dirk362]

Ah, no... that trick is only a speed-up if you're downscaling. QTGMC's speed is directly proportional to the number of pixels in the material it is given. So give it the lowest possible resolution. If you're downscaling, doing the horizontal resize first gives QTGMC less to work on. You can't do the vertical rescale before QTGMC because you'd destroy the interlacing.

However, if you're upscaling you should QTGMC the original smaller resolution.

Thanks. That makes perfect sense and I guess if I'd engaged my brain just a little more I'd have realised part of that question was just plain daft as veritcal is what you're effectively working on via the deinterlace.
But it now makes sense, so thanks for imparting knowledge. If we didn't ask we wouldn't learn :)

[nhope]

I have written a tutorial for preparing video for the web using the following workflow:

http://dl.dropbox.com/u/21489814/workflow.png

I was going to use TDeint+nnedi3 as the recommended deinterlacing method until I discovered QTGMC. -Vit- and Didée (not to mention tritical et al), thanks for this brilliant script.

The tutorial is here (http://www.bubblevision.com/underwater-video/Vegas-YouTube-Vimeo.htm) and we're discussing it on the Sony Vegas Pro forum here (http://www.sonycreativesoftware.com/forums/ShowMessage.asp?MessageID=752833). If anyone has any feedback, please let me know.

John Meyer tried it an made some interesting findings regarding multi-threading QTGMC, which can be read about 30 posts down that thread. I've asked him to report his findings here too.

[Didée]

John Meyer tried it an made some interesting findings regarding multi-threading QTGMC, which can be read about 30 posts down that thread. I've asked him to report his findings here too.
Well, speking "generally", his finding is simply wrong. Both QTGMC & TGMC can happily use more than 4 threads. Right now, I've a (TGMC) script running with all 8 Threads on my i7-860 fully utilized. CPU usage oscillates in range 95%~100%.

The key is resolution. With SD input, getting full CPU load is no problem at all. However, with 1080i it's a problem indeed. CPU usage is much lower then. For the exact reasons - don't ask me. It must be related to all the memory shuffling that's going on under the hood. Somewhere there is a borderline, and 1080i is definetly located on the other side. :(

[-Vit-]

Ideal thread counts differ by hardware, by settings and by source resolution. Also by what you're encoding with/to. On default settings I run SD at 13 threads for best speed, full HD at 8 threads. I have seen people indicate anywhere from 2 to 14 threads as their ideal. With HD material 4 threads is a good start, and it can be tricky to get stability and balance memory with more. But it is certainly not a hard limit - don't make the mistake of thinking that threads are tied to physical cores (it's possible for a developer to specify this, but it's not the norm).

[-Vit-]

Regarding multithreading HD material, you need to work SetMemoryMax properly. But I haven't seen it explained in detail anywhere, so here goes:

SetMemoryMax effectively sets two memory limits:
- The number you provide is the maximum memory used for the avisynth cache*
- (2Gb - the number you provide) is the maximum memory reserved for everything else used by your script (e.g. avisynth itself, NNEDI3, MVTools) - that's on a 32-bit workflow
- With 64-bit avisynth, it's more like (available physical memory - number you provide) reserved for plugins etc.

That means as you increase SetMemoryMax, you are reducing the amount of memory available for the plugins like NNEDI3. Plugins working on HD or complex settings will need considerable memory. Even more so when multi-threading because each thread needs plugin memory. So if you set memory max too high -> script failure.

The nature of a cache means that it always will fill up whatever space it is given (except in trivial examples). But it only actually needs to cache those frames that will be reused. Once those are in the cache, making it larger will have no positive effect at all, but it will reduce memory available for plugins.

That means there's a sweet spot, where you've reserved just enough to cache the needed frames and no more. So you [B]do not simply set the value larger and larger for higher resolutions. Go above the sweet spot and you may exhaust memory with no benefit. Go below the sweet spot and it will slow down because frames are not found in the cache (or more usually it will crash - there must be bugs in the avisynth caching). I find the sweet spot for HD around 700-800 but YMMV.

At some point you hit a limit, where: (memory needed for the avisynth cache) + threads x (memory needed for plugins) > your available memory. The formula depends on the factors I noted in the last post - it sets a limit on the number of threads you can run. It might not be enough threads to max out your CPU.

* The memory max figure is actually just a target. Internally, if avisynth desperately wants to cache something it can exceed the value you set. That's one reason why the behaviour is so hard to predict.

[-Vit-]

The latest version, v3.20, is in the first post (http://forum.doom9.org/showthread.php?t=156028). There are three areas of improvement:

Documentation
Most documentation has been stripped from the script itself and put in a separate HTML file. In Avisynth livery. All the new features below are explained in detail in there.

Noise processing / Denoising
I've improved the noise processing in several ways and made it simpler to use:
- Supports chroma noise processing / denoising (set ChromaNoise=true, it defaults to false)
- Supports dfttest as an alternative to fft3dfilter. I added this when I saw what strange results fft3dfilter produces on chroma denoising. I might post about that experience elsewhere. dfttest seems to identify noise vs detail more accurately
- Can provide a motion compensated clip to the denoisers, which further improves noise vs detail detection
- New "EZ" modes of operation which allow denoising or grain retention with just two settings. You set a value to indicate the strength of effect and a preset to determine quality of processing (presets from "Slower" to "Faster" only - used to make choices for the other settings: denoiser, motion compensation etc.). Examples:

# Automatic denoising, value is denoising strength, preset is quality/speed tradeoff
QTGMC( Preset="Slower", EZDenoise=2.5, NoisePreset="Slow" )

# Automatic grain retention - set amount of grain to retain, can be > 1.0
QTGMC( Preset="Slower", EZKeepGrain=1.0, NoisePreset="Faster" )


Access to QTGMC Motion Vectors
QTGMC now sets various uniquely named globals to give the calling script access to its motion vectors and more. Saves time recalculating vectors. Refer to the documentation (the "External Linkage" section) for complete details. Here are some examples:

# Use QTGMC motion vectors for our own smoothing
QTGMC( Preset="Medium", ForceTR=2, SubPel=2 ) # Ensure we get temporal radius of 2 and match Pel setting with next line
super = MSuper( pel=2, levels=1 )
MDegrain2( super, QTGMC_bVec1,QTGMC_fVec1, QTGMC_bVec2,QTGMC_fVec2 ) # Use QTGMC-created motion vectors

# QTGMC can reuse its own vectors. Here repairing progressive material with serious deinterlacing artefacts
t = QTGMC( Preset="Slower", InputType=2 )
b = QTGMC( Preset="Slower", InputType=3, PrevGlobals="Reuse" ) # Reuse motion vectors from first call for a good speed-up
Repair( t, b, 1 )

I'll post about the techniques I'm using for this later. I see this as the first step towards breaking QTGMC up into a set of independently usable components.
____

Full Version Changelog:
* Motion vectors made available to calling script (see External Linkage), added related GlobalNames, PrevGlobals and ForceTR settings
* Added simplified noise processing settings: EZDenoise, EZKeepGrain and NoisePreset
* Support for dfttest as denoiser for noise processing - added settings Denoiser and DftThreads
* Added ChromaNoise setting to optionally enable chroma noise processing
* Added DenoiseMC setting to use motion-compensated clip for denoising, improves detection of noise vs detail
* Replaced fft3dfilter-specific BT setting with NoiseTR setting to set temporal radius for denoiser.
* Added StabilizeNoise setting to "stabilize" & "deshimmer" noise restored in noise bypass (was part of "Generate" mode)
* Dropped NoiseRemove setting, now always removes all noise specified by given Sigma (enough controls already!)
* MotionBlur renamed to ShutterBlur, MBlurLimit renamed to SBlurLimit, DetailRestore renamed to GrainRestore (old names gave wrong impression)
* Bug fixes: stabilization of "Generate" noise mode, manually enabling SLMode with SourceMatch
* Documentation converted to HTML

[SubJunk]

Thanks for the update!

[nhope]

Preset tables updated for 3.20:

Compared to "faster" (https://spreadsheets.google.com/ccc?key=0AtvvpQjptJLFdGxFaEZwNDNXdGliTi1KdElEUEVtTHc&hl=en&pli=1#gid=0)
Compared to "medium" (https://spreadsheets.google.com/ccc?key=0AtvvpQjptJLFdGVGQzFUa0ozWTZJYXItLWFFTURBMXc&hl=en#gid=0)
Noise presets (https://spreadsheets.google.com/ccc?key=tjcarn1zieM-8yvT5wENEYA#gid=0)

[kypec]

Many :thanks: for big improvements and mostly simplified usage of your script!

[aegisofrime]

Many thanks -Vit- for the continual development of this awesome script!

I have been wondering one thing...

AFAIK, there isn't a 64-bit version of Dfttest right? Yet when I modified my 64-bit script to use Dfttest as a denoiser, to my surprise it worked. So I was wondering, might there be a bug in the script that isn't calling Dfttest properly?

Here's my QTGMC settings, for reference.

QTGMC(Preset="Slower",ChromaNoise=True,denoiser="dfttest",DenoiseMC=True)

[henryho_hk]

If anyone has any feedback, please let me know.

1) do not resize vertically before deinterlace
2) you may need "interlaced=true" for ConvertToYV12()

[pokazene_maslo]

AFAIK, there isn't a 64-bit version of Dfttest right?
Here is dfttest x64: http://forum.doom9.org/showthread.php?t=152800 but only an older version

:thanks: for chroma denoising support.
I have couple questions:

NoiseDeint parameter: is it used when GrainRestore and NoiseRestore are 0?
fft3dfilter is missing interlaced=true parameter, is that intentional?

Once again thanks!

[-Vit-]

nhope: Thanks for the updates. I've added them to the OP

aegisofrime: Are you sure you don't actually have the 64-bit dfttest? Those settings will certainly select it. To test if dfttest is really being used, temporarily remove its dll from your plugins folder and try to run your script. If it comes up with a "can't find dfttest" error then you really were using dfttest.

pokazene_maslo:
1. NoiseDeint is only relevant if you are have either GrainRestore > 0 or NoiseRestore > 0, i.e. if you retaining grain rather than denoising. It is set automatically by NoisePreset. [This reminds me that I didn't mention the obvious: EZDenoise and EZKeepGrain are mutually exclusive]
2. Yes, that is intentional. Firstly, dfttest doesn't support interlaced input. Secondly, it's impractical to motion-compensate interlaced input for the denoisers (DenoiseMC=true). So I use a point resize on the fields to get something I can feed to the denoisers that matches the motion vectors. The point resize will not blur the noise and the denoisers seem to be able to handle the line doubling. After the denoise I drop the doubled lines and reweave the result to get noise to match the original interlace. Or put more simply, I implement "interlaced=true" manually in the script.

[aegisofrime]

In fact, I was unaware that there was a 64-bit dfttest before pokazene_maslo told me about it. I remembered checking a few times to make sure whether there was a dfttest.dll in my plugins64 folder. There wasn't.

[-Vit-]

aegisofrime: Just looked again at your QTGMC line. Although you're selecting dfttest you're not actually doing any denoising, that's why it isn't showing any error. I think I should make the documentation clearer and maybe rename the NoiseBypass setting to NoiseProcess

[aegisofrime]

aegisofrime: Just looked again at your QTGMC line. Although you're selecting dfttest you're not actually doing any denoising, that's why it isn't showing any error. I think I should make the documentation clearer and maybe rename the NoiseBypass setting to NoiseProcess

Ah, I see. I needed NoiseByPass to be "1" or "2". Yes, I would say that the name of the setting wasn't exactly intuitive :p

[-Vit-]

Well QTGMC was never supposed to be a replacement for a denoiser. Noise bypass was where it started - to get round the fact that the core algorithm temporally smooths and you might not want that. Which explains the setting names. But I noted in the docs somewhere that you could use it to denoise and... well, lots of people leapt at that idea. So the new settings were born and now it is trying to be a denoiser...
____

Just adding YUY2 support now. Someone made a version that supported YUY2, but I can't find it. Don't even know who made it. I looked to find it in the thread but any posts from 6th January to 18th January are missing (post 135-136 (http://forum.doom9.org/showthread.php?p=1468899#post1468899) in this thread) - which is when it was posted...

[yup]

[QUOTE=
Just adding YUY2 support now. Someone made a version that supported YUY2, but I can't find it. Don't even know who made it. I looked to find it in the thread but any posts from 6th January to 18th January are missing (post 135-136 (http://forum.doom9.org/showthread.php?p=1468899#post1468899) in this thread) - which is when it was posted...[/QUOTE]
It is Steven R. Savage version. I am try it, work fine.
yup.

[nhope]

1) do not resize vertically before deinterlace
2) you may need "interlaced=true" for ConvertToYV12()

Thanks for that. Actually the script is not resizing vertically before deinterlace. It resizes horizontally before deinterlace and vertically after. You are right about "interlaced=true". I had omitted that, and the levels match slightly more precisely now that I've added it. The new base script in my tutorial (http://www.bubblevision.com/underwater-video/Vegas-YouTube-Vimeo.htm) is now:

# Open frameserved source.
# Change path and file name as appropriate.
AviSource("d:\fs.avi")

# Convert to YV12 so filters will work.
# Use interlaced layout for conversion.
# Change to "true" to false" for progressive source.
# Use Rec.709 coefficients, keep full range [0,255].
ConvertToYV12(interlaced=true, matrix="PC.709")

# Assume footage is top field first.
# If it's not then use AssumeBFF.
# Leave it out for progressive source.
AssumeTFF

# Lanczos3 resize to 1280 wide.
# Leave it out if horizontal resolution is already 1280.
LanczosResize(1280,height)

# Deinterlace with QTGMC script.
# Available presets are placebo, very slow, slower,
# slow, medium, fast, faster, very fast, super fast,
# ultra fast, draft. Default is medium.
# Slower presets unnecessary for HD.
# ultra fast requires Yadif.
# Available noise presets (since version 3.20) are
# slower, slow, medium, fast, faster. Default is fast.
# Leave line out for progressive source.
QTGMC( Preset="faster", NoisePreset="Fast" )

# Add this line to keep original frame rate.
# Leave it out for smoother doubled frame rate.
# Leave it out for progressive source.
SelectEven()

# Lanczos3 resize to 720 lines.
# Leave it out if vertical resolution is already 720.
LanczosResize(width,720)

-Vit-, YUY2 support will be welcome. Thanks for your hard work and brilliance with this script, and for your help in this thread.

[-Vit-]

yup: Thanks for that (and thanks to Stephen R Savage too). It will be a handy cross reference.

nhope: It doesn't affect anything, but you don't need to set NoisePreset unless you're doing some noise processing (i.e. using one of the "EZ" modes or setting NoiseBypass). There is a little grain retention in the "Very Slow" and "Placebo" settings but from reading your thread over at Sony I think you're targeting the faster settings. In any case NoisePreset="Fast" is the default.

[nhope]

OK so I took the NoisePreset parts out of the tutorial again to keep it simple. Those wanting more can find it in your help file, which is now really easy to view in its new html form.

[Gavino]

Just adding YUY2 support now. Someone made a version that supported YUY2, but I can't find it. Don't even know who made it. I looked to find it in the thread but any posts from 6th January to 18th January are missing (post 135-136 (http://forum.doom9.org/showthread.php?p=1468899#post1468899) in this thread) - which is when it was posted...
It is Steven R. Savage version. I am try it, work fine.
Does this explain the missing posts?
For some reason known only to himself, Steven has a history of deleting his posts (which is unfortunate as his contributions are often useful).

[kolak]

Just adding YUY2 support now. Someone made a version that supported YUY2, ...


Great- YUY2 is nice to have for people, who have 4:2:2 sources.


Andrew

[2Bdecided]

The new base script in my tutorial (http://www.bubblevision.com/underwater-video/Vegas-YouTube-Vimeo.htm) is now:FWIW, unless the only copy you ever want to make of that Vegas timeline is a Vimeo upload, I think it makes sense to render to HDV (if the project can mostly be rendered with smart rendering, and you're starting from HDV of course), or some near-lossless or lossless format from Vegas - and then to work from that. Using the debug frame server means you never actually create a master copy for future use.

One trick: sometimes almost everything would smart render if only it weren't for the levels. If the required level correction is constant throughout, do it in AVIsynth - then you can smart render. Though it does annoy me to see the extra quantisation noise appear on the sections that don't smart render! It's not visible on any subsequent encode you create from this (unless using insane settings), and QTCMG will wipe it all out by default.

It's a rare bit of content where you can see any difference between Yadif and QTGMC by the time it's on YouTube or Vimeo at 720p. Near-stills at 1080p might be a different matter, but I don't know many people who can play YouTube 1080p to check this.

Very nice guide though! It's takes a lot to explain all the steps properly like this. Great job.

Cheers,
David.

EDIT: many of the MeGUI pre-sets are fine for Vimeo and YouTube upload IME. The sites are quite happy with unconstrained balanced (ABR or const. quality, whichever you prefer).

[@MeGui]

nhope...someth is wrong with MT script?

[henryho_hk]

SelectEven()

If you are doing single-rate deinterlacing, why not try the new motion blur feature?

[nhope]

@ 2Bdecided - I'll reply to your non-QTGMC points on a dedicated thread for this tutorial when it's refined.

@ henryho_hk - Good idea. I'll check it out.

nhope...someth is wrong with MT script?

Not sure because I can't test it here. The brief testing I did was actually slower than ST because I'm just on a Core 2 Duo laptop. What problem are you seeing? Here is the base MT script with the non-MT comments removed (full version here (http://www.bubblevision.com/underwater-video/Vegas-YouTube-Vimeo.htm#MultiThreaded)):

# Set maximum memory.
# Setting M:
# - First try without the SetMemoryMax line
# - However, using the SetMemoryMax line and a good value for
# M might allow more threads and so give more speed.
# Particularly important for slower settings
# - Try values 400,600,800,1000 etc.
# SetMemoryMax(M)

# Set multi-threaded mode.
# Setting X:
# - Start at the number of cores in your machine.
# - If it crashes, decrease 1 at a time.
# - Otherwise increase 1 at a time until CPU usage is very,
# very close to 100%, don't go too far or it will slow down .
SetMTMode(5, X)

AviSource("d:\fs.avi")
ConvertToYV12(interlaced=true, matrix="PC.709")
AssumeTFF

# Set multi-threaded mode.
SetMTMode(2)

LanczosResize(1280,height)

# Setting Y:
# - Start at about half number of cores and tweak upwards
# or downwards for best speed. Y=1 often works well.
# - Balance this setting with X (i.e. if you increase X,
# you might need to decrease Y and vice versa).
QTGMC( Preset="faster", EdiThreads=Y )

SelectEven()
LanczosResize(width,720)
ColorYUV(levels="PC->TV")

# MeGUI requires this line for multi-threaded operation.
Distributor()

[-Vit-]

The latest version, v3.25, is in the first post (http://forum.doom9.org/showthread.php?t=156028). The changes:

# v3.25
# - Added YUY2 support
# - NoiseBypass renamed to NoiseProcess
# - PrevGlobals now defaults to "Replace" for simplicity
# - Tweaked "Very Fast" preset for quality and consistency with other preset speeds


Dealing with the smaller points first. The NoiseProcess name change is just to avoid confusion - I've also clarified the docs. The PrevGlobals change means that multiple calls to QTGMC in a single script does not require special attention. The previous script was too defensive - now only those doing unfeasibly complex scripts or wishing to take advantage of the globals system need to worry about it. I found the "Very Fast" preset was slower than "Faster" in single threaded mode so I tweaked it.
____

YUY2 support is the main change here. All settings and features are supported and the speed seems OK. I get 20% to 30% slower depending on preset and settings, which is decent given that there's 33% more data in YUY2. I have only a very limited amount of interlaced YUY2 material so could those who work with YUY2 please confirm that everything is OK? In particular check the extra chroma data. If you're not so familiar with YUY2, please don't report problems until you're sure that your entire workflow is correctly passing through the YUY2 data all the way to your encode/player. Try substituting "Bob()" for "QTGMC()" before blaming it on me!

I minimized interleaved to planar conversions to keep the speed, there are just seven in default "Slower" settings, five at "Slow". Almost all the YUY2 processing is done in the faster planar format. To achieve this I've taken some liberties by passing the non-standard planar YUY2 format into several internal avisynth functions. Each has been tested to operate correctly on this form, although I didn't go through and check the source code of each. The functions I 'm using with the planar YUY2 form are:
Merge, Bob, Blur(0,x), Crop(0,x,0,x), TemporalSoften, SeparateFields, Weave, Interleave, SelectEvery

[nhope]

If I shot HDV 1080-50i with a shutter of 1/50 (or 1080-60i with a shutter of 1/60), and I want to restore the motion blur after deinterlacing to 25p (or 30p) with QTGMC, what values should I use for ShutterAngleSrc and ShutterAngleOut?

Also does FPSDivisor=2 mean I can drop SelectEven() ?

[-Vit-]

Yes with FPSDivisor=2 you can drop the SelectEven (I will make that clearer in the docs)

The answer to the first question is either both to 360 or both to 180. I would tend to suggest the latter, which is the default.

That question is one of the main reasons why I didn't use shutter speeds for the settings. In researching the shutter motion blur I found some confusion about shutter speeds when shooting interlaced material. Some suggest that a shutter speed matching the field rate (e.g. 1/50 shutter for 50i, also called "shutter off") implies a 360 degree shutter angle. However, I also found comment that the frame rate is relevant, in which case a 1/50 shutter speed on 50i footage implies a shutter angle of 180 degrees (film-like). There even seemed to be differences in how different cameras behaved with the same apparent settings. I have no opinion, this is not my area of expertise. So I went for shutter angle which is at least consistent if not the most widely known.

I would appreciate if anyone else can shed light on this one. If you shoot footage in this format, is the shutter effectively held open for 50% (180 degree) or 100% (360 degree) of the time between fields?

[SubJunk]

Thanks a lot for the update!

[Rhade3000]

im having some issues with qtgmc im not sure if its my system or the script, first some background

win7
6 core amd 3.2 gighz, 8 gig ram
2.5.8mt x64

script

SetMTMode(5,6)
DGDecode_mpeg2source("VTS_01_1.d2v", info=3)
crop( 2, 0, -2, 0)
setMTMode(2)
eedi3()
QTGMC( Preset="placebo", inputtype=2 )
spline64Resize(640,480)

im processings curb your enthus which is already 30p on the dvd

ive used this script for all of season 1 with no issue, but the last few versians of qtgmc i get this error


http://img257.imagevenue.com/loc479/th_007016739_Untitled_122_479lo.jpg (http://img257.imagevenue.com/img.php?image=007016739_Untitled_122_479lo.jpg)

the script is pretty cpu intensive, but its the only way i can seem to be abale to remove the horrible jagged edges in every scene

when i load the script in megui it loads fine and i see a preview pic, then i click add pre render job and enque and it throws up that error

if i do just a straight nnedi3() call with the qtgmc lined #' out it works, but it doesnt give as good of a result

[-Vit-]

As the error says, the problem is with loading NNEDI3. Unless others are getting the problem then it is likely to do with the setup of your plugins. Is the folder name in the error message correct? Is the file actually there? Do you have the latest version of NNEDI3? Download it again just in case there's a problem with the file, etc.

Have you tried running QTGMC using EEDI3 like this:
QTGMC( Preset=whatever, EdiMode="EEDI3" )
It might solve your problem (although it wouldn't explain the cause)

[nhope]

Preset tables updated for changes to "very fast" preset in 3.25:

Compared to "faster" (https://spreadsheets.google.com/ccc?key=0AtvvpQjptJLFdGxFaEZwNDNXdGliTi1KdElEUEVtTHc&hl=en&pli=1#gid=0)
Compared to "medium" (https://spreadsheets.google.com/ccc?key=0AtvvpQjptJLFdGVGQzFUa0ozWTZJYXItLWFFTURBMXc&hl=en#gid=0)

-Vit- - Please check! Sometimes I make clumsy errors.

[Taurus]

@Rhade3000
Take a look at this.
http://forum.doom9.org/showthread.php?p=1423128#post1423128
I've got this before.
Dont know about a x64 setup.
and have a look at the eventviewer.
If eedi3 is showing bad behaviour there, you can be sure a Visual C Redistributable is missing.

[Rhade3000]

@taurus and @vit

neither of those worked, but removing the "setmtmode(2)" line worked, and didnt slow down my cpu at all, i could have sworn that adding that line before saved me hours on the encode, but it didnt change processing time :P

thanks for the help :P

[aegisofrime]

-Vit-: Just to clarify, ChromaNoise works without needing NoiseProcess to be enabled?

[-Vit-]

-Vit-: Just to clarify, ChromaNoise works without needing NoiseProcess to be enabled?
ChromaNoise is only relevant if NoiseProcess is non-zero. I'll clarify the docs again... :sigh:

[nhope]

Yes with FPSDivisor=2 you can drop the SelectEven (I will make that clearer in the docs)

The answer to the first question is either both to 360 or both to 180. I would tend to suggest the latter, which is the default.

That question is one of the main reasons why I didn't use shutter speeds for the settings. In researching the shutter motion blur I found some confusion about shutter speeds when shooting interlaced material. Some suggest that a shutter speed matching the field rate (e.g. 1/50 shutter for 50i, also called "shutter off") implies a 360 degree shutter angle. However, I also found comment that the frame rate is relevant, in which case a 1/50 shutter speed on 50i footage implies a shutter angle of 180 degrees (film-like). There even seemed to be differences in how different cameras behaved with the same apparent settings. I have no opinion, this is not my area of expertise. So I went for shutter angle which is at least consistent if not the most widely known.

I would appreciate if anyone else can shed light on this one. If you shoot footage in this format, is the shutter effectively held open for 50% (180 degree) or 100% (360 degree) of the time between fields?

I asked the folk on the Sony Vegas forum (http://www.sonycreativesoftware.com/forums/ShowMessage.asp?MessageID=754540&Replies=10) and the general consensus is that 50i @ 1/50 or 60i @ 1/60 equates to a 180 degree shutter angle.

I've been doing some testing with it and it seems to be right. All just done single-threaded on my laptop. One thing I noticed was that the ShutterBlur setting didn't seem to make much difference to speed at all, so I left it at 3 for most tests. I didn't see much difference between using SBlurLimit or not on this particular clip. The table more or less speaks for itself:

http://dl.dropbox.com/u/21489814/QTGMC-motion-blur.png

I zipped up the 3 x 10 second tests I've highlighted in yellow and put them here (http://www.mediafire.com/?s0erw12ek7irikb) if anyone is interested. Footage used with permission from Stringer (Gregory) on the Vegas forum. I would be interested to hear which people prefer.

By the way Vit, are you making a general recommendation for FPSDivisor=2 over SelectEven (which is currently in your "Getting Started" section)? I just want to be consistent with you in my guide.

[aegisofrime]

ChromaNoise is only relevant if NoiseProcess is non-zero. I'll clarify the docs again... :sigh:

Sorry! I have read and re-read the Noise part of the docs and nowhere does it say that NoiseProcess has to be non-zero for ChromaNoise to work. If it's supposed to be obvious due to the nature of Chroma noise... Well it's lost on us newbie types who don't really understand the finer points of video :p

[-Vit-]

nhope: Thanks for the clarification that 180 degrees is the norm for shooting that kind of footage. Reading your thread over at Sony just confirms that there can be some confusion over shutter speeds. I'm going to stick with using shutter angle for the settings as it makes people think and/or ask rather than use the settings blindly...

The default setting for SBlurLimit should be OK for most sources, it's just an extra tweakable in case difficult motion is causing unwanted distortion/smearing. Similar with the higher shutter blur settings (2,3).
Edit: As noted in the docs, increasing those settings will likely reduce the level of blur a little from where it should be for the requested shutter angle. Looking at your motion blur samples - the 1/30s shutter version seems to have a little less blur than I would expect. I would stick to the (faster) ShutterBlur mode 1 and a lower SBlurLimit value (default is good). It doesn't harm to change ShutterAngleOut to get a bit more or less blur as you wish [The ideal would be to make some actual footage with known shutters and compare]. However, that particular footage is quite soft already and the motion is smooth, it doesn't benefit much from extra blur. I found motion blur works better on higher contrast video, with swift or sudden motion. It's also excellent for footage shot with a fast shutter, which really suffers when dropping to single rate.

You must use FPSDivisor if you're using shutter motion blur whilst cutting to single rate - it needs to know that information along with the shutter angles to calculate the correct amount to blur. If you're not using shutter motion blur it makes little difference whether you use SelectEven or FPSDivisor - whichever makes your tutorial easier.

[BTW: Ouch on those fps... Time to upgrade...]

____

Sorry! I have read and re-read the Noise part of the docs and nowhere does it say that NoiseProcess has to be non-zero for ChromaNoise to work. If it's supposed to be obvious due to the nature of Chroma noise... Well it's lost on us newbie types who don't really understand the finer points of video :p

I'm not sure which bit you were re-reading. Did you miss this bit?
Important: You must use a non-zero value in one of EZDenoise, EZKeepGrain or NoiseProcess or no noise processing will occur

[aegisofrime]

I'm not sure which bit you were re-reading. Did you miss this bit?

Ouch, I missed that. My fault :p

[Overdrive80]

Hi, thanks for your update.

This version throw an exception:

http://www.imagengratis.org/images/capturagr2gg.png

With version 3.20 doesnt pass me.

I do two scripts and run both, with rendering pass(lagarith).

1st script

DirectShowSource("E:\Pruebas Ghost Hound\16\F8_T2_Video - .mkv")

AssumeFPS(30000,1001)

QTGMC(preset="very fast" )

Selecteven()

threads=2

x=(framecount()/threads)

A=trim(0,x)
B=trim(x+1,0)



return A

2nd script

DirectShowSource("E:\Pruebas Ghost Hound\16\F8_T2_Video - .mkv")

AssumeFPS(30000,1001)

QTGMC(preset="very fast" )

Selecteven()

threads=2

x=(framecount()/threads)

A=trim(0,x)
B=trim(x+1,0)



return B

Any bug?

[-Vit-]

I have not seen an error like this before. Does it happen with preset "Faster" or "Ultra Fast"?

You could try this:
QTGMC( preset="very fast", SafeMode=true )

Or try making a second copy of your video to use in the second script.

[Overdrive80]

I have not seen an error like this before. Does it happen with preset "Faster" or "Ultra Fast"?

You could try this:
QTGMC( preset="very fast", SafeMode=true )

Or try making a second copy of your video to use in the second script.

In safemode run fine but decrease speed process, around of 2 frames/second. Now, my process is of 2.30 hours before was of 1.50 hours, more or less.

[-Vit-]

The bug is in MVTools2. Make sure you have the latest version of MVTools2 (http://avisynth.org.ru/mvtools/mvtools2.html#download) and try again without SafeMode.

If it still crashes then this line will restore the behavior of the last version (3.2):
QTGMC( Preset="Very Fast", Overlap=16, SLMode=1, SrchClipPP=0 )

This is a problem in MVAnalyse when the Overlap is not 1/2 the Blocksize. It crashes sometimes. I added the SafeMode setting just for this problem, but I haven't actually seen it for a while. It had assumed it was fixed in more recent versions...

[Overdrive80]

I already installed last version MVtools. In fact, i don know that new features in this version, but in my case, deinterlaced bad in the same mode. Snapshots:

Version 3.20, preset very fast

http://www.imagengratis.org/thumbs/comprimirnm3zn.jpg (http://www.imagengratis.org/?v=comprimirnm3zn.jpg)

New version, preset very fast

http://www.imagengratis.org/thumbs/comprimirce3cc.jpg (http://www.imagengratis.org/?v=comprimirce3cc.jpg)

[-Vit-]

Final Edit (I hope): I can reproduce the MVTools crash, I'll look into that later.

However, I cannot reproduce the major ghosting shown in your images. Could you upload part of your source please?

[henryho_hk]

I also found some problem bobbing a 1080i source. The field order is correct as confirmed by separatefields(). But the motion jumps "backward" in some scenes. I am using Preset="Faster" and tr2=2. Setting EdiMode="Yadif" solves the problem.

[nhope]

This version throw an exception:
...
I do two scripts and run both, with rendering pass(lagarith).

You didn't change your Lagarith version by any chance? There were recent updates (http://lags.leetcode.net/Changes.html) to it that caused some of us some problems (http://www.sonycreativesoftware.com/forums/ShowMessage.asp?ForumID=4&MessageID=754229).

[-Vit-]

henryho_hk: Try different source filters - I have often found that strange temporal effects are source filter related. Otherwise upload some source footage for me to test.

[henryho_hk]

It's ffvideosource() 2.15 over a MKV muxed from m2ts. I will try dss2() (i have ffdshow h264 decoder only, though).

[Overdrive80]

You didn't change your Lagarith version by any chance? There were recent updates (http://lags.leetcode.net/Changes.html) to it that caused some of us some problems (http://www.sonycreativesoftware.com/forums/ShowMessage.asp?ForumID=4&MessageID=754229).

Yes, I already installed lagarith 1.3.23. I dont think that could be for this reason.

@Vit. My source is 1080i@29.97. I upload a part of interlacing credits, compress in x264 @ 5000 Kb/s. If you need that upload in other format, only say me. ^^

http://www.megaupload.com/?d=ZJSXS36E

[-Vit-]

Overdrive80: The field parity needs to be reversed. This worked fine for me:

FFVideoSource( "E:\Video\Test\Original.mp4" )
ComplementParity()
QTGMC( Preset="Very Fast" )

Try "ComplementParity()" whenever you see "back & forward" jumping effects like this.

[Overdrive80]

Overdrive80: The field parity needs to be reversed. This worked fine for me:

FFVideoSource( "E:\Video\Test\Original.mp4" )
ComplementParity()
QTGMC( Preset="Very Fast" )

Try "ComplementParity()" whenever you see "back & forward" jumping effects like this.

With this code, the ghost effect is gone.

My question is for curiosity... because is necesary add a filter for autodetecting dominant field but not with previous version?

[johnmeyer]

I am using a script based on Nick Hope's work. However, I have made it multithreaded. Everything works with the older version of QTGMC, but only with Very Fast. When I use Fast, I get problems. Here's the script:
#Script to deinterlace video based on QTGMC deinterlacing script
#Designed to produce 1280x720 progressive video

loadplugin("C:\Program Files\AviSynth 2.5\plugins\MVTools\mvtools2.dll")

#---------------------------
#Import("E:\Documents\My Videos\AVISynth Scripts\QTGMC3.20.avsi") #older script
Import("E:\Documents\My Videos\AVISynth Scripts\QTGMC3.25.avsi")

#Serve out from Vegas in RGB24
SetMemoryMax(768)
SetMTMode(5,4)
AVISource("e:\frameserver.avi").assumetff().ConvertToYV12(interlaced=true,matrix="PC.709") #
SetMTMode(2)

LanczosResize(1280,height)

#---------------------------
QTGMC( Preset="Very Fast", FPSDivisor=2, EdiThreads=0, ShowSettings=false )
#QTGMC( Preset="Faster", FPSDivisor=2, EdiThreads=0, ShowSettings=false )

LanczosResize(width,720)
ColorYUV(levels="PC->TV") #Scale levels from [0,255] to [16,235]. Compensates for Flash Player scaling [16,235] to [0,255].


I have tried four combinations in the above script (by changing which lines are commented out):Script Version Script Preset
QTGMC3.20.avsi QTGMC( Preset="Very Fast" ... )
QTGMC3.20.avsi QTGMC( Preset="Faster" ... )
QTGMC3.25.avsi QTGMC( Preset="Very Fast" ... )
QTGMC3.25.avsi QTGMC( Preset="Faster" ... )

The first combination is the only one that works without crashing, when tested using a 1440x1080 1,000 frame input file. I test by reading the script into VirtualDub and then doing a video analysis pass. This eliminates any small interaction that might be happening with an output codec.

I have tried changing the SetMemoryMax number and also tried changing the number of cores in the first SetMTMode, but all that does is change how long it takes before the crash happens. It usually takes about 200-500 frames before the crash begins (the fps counter drops for about a dozen frames prior to the crash).

For the third test listed above (your newest script with the Very Fast setting), here is the relevant line (I think) from the crashinfo.txt log:

Crash reason: Access Violation

Crash context:
An out-of-bounds memory access (access violation) occurred in module 'mvtools2'...

...reading address 247EF1D2.
I read your earlier post about needing to update to the latest MVTools2, so I download the latest release (2.5.11.1), but still had the problems.

Oh, and prior to the beginning of the crash, the newer script runs considerably slower, probably about 30% slower, with each of the settings.

Let me know if you need any other information.

[-Vit-]

Overdrive80: No, I haven't changed anything that should affect the field parity. I also tested your clip on v3.20 and I needed the ComplementParity() line in that version too. Sometimes the source filters get the field parity wrong, I've seen it several times before.

johnmeyer: See post 430 above (http://forum.doom9.org/showthread.php?p=1485196#post1485196) for the settings to restore "Very Fast" in v3.25 to v3.20 behavior and speed. That might help for your "Very Fast" setting. I am nearly finished on some optimizations for all the faster settings, so I'm not going to worry about the speed issue just now.

Regarding crashes, the 'mvtools2' note in the error message may or may not be indicative. Try adding the setting "SafeMode=true" to your QTGMC line (it will be slower). If that fixes it then MVTools is really to blame. I can reproduce errors with MVTools, so I will debug it at some point. However, often multithreading or memory bugs due to AviSynth itself occur whilst running the plugin modules, so the problem actually might be with AviSynth. Let's hope it's a memory issue, because they're easier to fix. Absolute priority when running HD multithreaded is to render to a lossless intermediate file - don't try to use x264 at the same time as HD QTGMC. For MeGUI that means you must select "Add Pre-rendering Job". Setting EdiThreads=1 might help with memory too.

If you're crashes are problems with avisynth multithreading then try using different versions of MT avisynth. Find them linked on the AviSynth Wiki main page (http://avisynth.org/mediawiki/Main_Page)

[Overdrive80]

@Vit. I translate QTGMC doc to spanish, I hope that dont disturb you.

http://www.megaupload.com/?d=3XKOAFTN

[johnmeyer]

Regarding crashes, the 'mvtools2' note in the error message may or may not be indicative. Try adding the setting "SafeMode=true" to your QTGMC line (it will be slower). If that fixes it then MVTools is really to blame. I tried all four permutations of new/old script and Faster/Very Fast with "SafeMode=true" added to my QTGMC line.

With the old (v.20) QTGMC script, I was able to get my 1,000 frame test video to run through the video analysis pass without crashing, although the fps indicator started to fluctuate at about the 900 frame mark, which is usually a sign of impending crash. Thus, it might still crash if fed a longer test video. With the new (v25) QTGMC script and the more stable Very Fast setting, I was able to get almost to the end of the analysis pass before the script crashed, whereas without the SafeMode statement, it crashed almost immediately.

Hope that helps.

[johnmeyer]

Quick addendum (I didn't want to further edit my previous post):

I kept trying different things with my four permutations (described above) and found that I got more stability with both versions of the script if I changed EdiThreads=0 to EdiThreads=1. I still got crashes with the new script but they happened after a larger number of frames had been processed. So, following the guidelines in the QTGMC doc, I played around with the MT settings. SetMemoryMax didn't make much difference, but increasing the number of cores from 4 to 8 (my computer is a 8-core i7), allowed me to get all the way through my 1,000 frame test video with the "toughest" combination of the new v25 script and the "Faster" QTGMC setting. I haven't tried this on a really long video so I don't know if it is totally stable. Also, I did these most recent tests without the SafeMode statement.

Again, I hope this helps.

[edit] I just re-ran the script using the newest v25 script with these settings, but using Very Fast. It crashed. Bummer. So, these settings let me get through the test script with the settings that previously crashed, but they cause crashes on the settings that previously worked. I tried about half a dozen settings for the "Very Fast" setting, and was unable to get ANY that would work with the new script, even if I completely disabled all multi-threading.

So, I guess I haven't yet found settings that work for more than a few settings. It's still very "touchy."

[-Vit-]

Overdrive80: Thanks very much for that, I'm sure it will be appreciated. [Edit: I couldn't download it before, but now I have it. I will link to it on the first post.]

johnmeyer: There are two things here. Firstly there is certainly a problem with QTGMC<->MVTools on faster presets, which sometimes causes crashes before even a single frame is processed. I will look at that. But I don't think that's all that you're experiencing. Almost everyone who does multi-threading with complex scripts finds that avisynth will randomly crash with particular settings/scripts/hardware etc. The reason is that there are bugs with the multi-threading in avisynth. Sadly fixing those bugs would involve considerable effort, especially for those of us who aren't familiar with the avisynth code base.

I have had numerous detailed descriptions of problems, and they all follow the same pattern: encoding crashes after X-thousand frames; tweaking settings makes things slightly better, or slightly worse; problem is more pronounced with complex settings or HD material. But despite the similarity, there is no one solution. The first thing you need to try is different MT versions of AviSynth. For example, I can barely get a few hundred frames out with the standard 2.57MT, I only get limited stability with SEt's 2.58MT, but I have encoded tens of millions of frames with SEt's 2.6MT. For others it's the same story but a different version that works...

Edit: BTW, EdiThreads=0 means use 1 thread per logical core. So on your system it means EdiThreads=8. That's why setting it to 1 is more stable. Also it can help to use SetMemoryMax at a lower value than you might expect. I sometimes use 400 for HD material...

[johnmeyer]

-Vit-,

Yes, I agree with what you are saying about the less-than-robust behavior of the MT versions of AVISynth. Fortunately I think I have now managed to tweak things so that I have a stable workflow now.

One thing I didn't realize is that there are competing MT versions of AVISynth. When I do a version call, I get: "AVISynth 2.58 tsp version 5(mod seraphy), build: Aug 16, 2009 ..." I'll see if I can find a version "2.6MT." I don't think I've seen that before.

[-Vit-]

Here are the various versions: Standard 2.57MT (http://avisynth.org/mediawiki/MT), SEt's 2.58MT (http://forum.doom9.org/showthread.php?t=148117) or SEt's 2.6MT (http://forum.doom9.org/showthread.php?t=148782)
Or if you've converted to a completely 64-bit workflow the x64-2.58MT (http://forum.doom9.org/showthread.php?t=152800)

[SubJunk]

SET's 2.6 wins :)

[johnmeyer]

Thanks for the links. I tried set's 2.6, but apparently the direct support for reading RGB24 (at least from debugmode's frameserver) has been deleted. As soon as I try to open my script in VirtualDub, I get the message: "AviSource: Could not open video stream in any supported format." If I change the Debugmode Frameserver (which is how I serve out of Sony Vegas) from RGB24 to either RGB32 or YUY2, then I can open the script. RGB32 keeps the colorspace the same as my previous workflow, but cuts performance in half. Using YUY2 keeps performance the same, but introduces all sorts of color shift and levels issues. Nick Hope (who wrote the long tutorial on how to do this) has spent several months of his life working these things out, so I have no hope of trying to get the same levels. So, at least for the moment, I'm back to 2.5.8.

I like the idea, however, of having better stability with multi-threading (for ALL of my scripts), so I'll look into what else I can do here ...

[edit] I just did some more work ... if I use YUY2 out of the Debugmode frameserver, and strip all the colorspace corrections, I end up with the same levels. As for color space, all colors appear to be the same as those I got using Nick Hope's original workflow, except that pure red colors are truer to the original. With his workflow, it appears to me (now that I'm looking more critically) that there is a shift towards orange in the final render.

So, bottom line, I think I'll be able to use 2.6MT, and that all the changes you have suggested and that I've found on my own make this stable. I'm now getting over 15 fps doing video analysis in VirtualDub, and almost the same when rendering from MeGUI. All 8 cores are showing nearly 100%.

[Didée]

@ johnmeyer: Maybe I'm lacking imagination, but ... if you have to deinterlace a video, then why has it to come out of Vegas in the first place? I could more easily imagine the other way round: have some interlaced video, then deinterlace it with Avisynth/QTGMC, then do additional stuff in Vegas.
Basically this doesn't matter, of course. But since multithreaded QTGMC on Full-HD video is not a prime example of "stability" in the first place, I'm not sure if Vegas' frameserver does exactly help on the matter.

[johnmeyer]

@ johnmeyer: ... if you have to deinterlace a video, then why has it to come out of Vegas in the first place? I could more easily imagine the other way round:Didée,

For many situations, what you propose may make more sense. However, there are many cases where deinterlacing first does not make sense. For instance, I often need to produce several versions of the same video: one down-res'd to DVD; one for Blu-Ray; one for upload the web; and one to be archived in the original HDV or AVCHD format. So, in some cases I want to preserve interlacing, and other cases do not. Thus, for this workflow it makes more sense to maintain interlacing until the last step. Also, it has always been my mantra: don't deinterlace unless you absolutely must do this. Deinterlacing always degrades the video, both temporally and spatially, and therefore I believe it is always best to only take this step when sending the video to an unknown client (web browsers) where I cannot be certain that the display can either show the interlaced video natively, or can do deinterlacing on its own.

[aegisofrime]

Funny, johnmeyer, after I met Q/TGMC I always deinterlace before doing anything. Yup, it's that good to me.

Of course it would be best if your camera capture in 60p in the first place. :devil:

[johnmeyer]

Of course it would be best if your camera capture in 60p in the first place.It's hard to argue against 60p.

[Didée]

Well, I knew nothing about your workflow, and therefore I didn't mean to critizise it.:) - I simply was wondering if the mere fact of frameserving from Vegas could increase the overall instability. I'm not even sure about the memory model - but if no piping happens and Vegas' frameserver shares the same 2 GB memory block with Avisynth and its host application, then I could very well imagine that a successful start-to-end encoding of a FullHD source is nothing but a dicegame. :)

Just to serve the curiosity, you might try to save Vegas' output to a losslessly compressed AVI, and use QTGMC on that AVI. Obviously you don't want to do that for your everyday's workflow. However, lets assume you undertake this experiment ... and find out that the notorious crashes disappear, and all full-length processings/encodings all of a sudden finish successful. What is then? :)

Also, fact is that with interlaced video, all operations with a "spatial" filtering aspect have to be done in a suboptimal way. In particular for change of resolution, but also for denoising, or blur/sharpen filters, it will be better to have QTGMC at the start of the chain, not at the end.
"Degrading by deinterlacing" is a valid point. However, "degrading because of NOT deinterlacing" is also a valid point. It's like it always is: "it depends". ;)

[johnmeyer]

Didée,

Rendering first to uncompressed is a great idea for troubleshooting. I'll do that. The only issue I have ever had with this workflow is field reversal, and I think that was only with PAL sources.

And you may be correct about the DebubMode Frameserver, although I have no way of knowing about its internals. I've used it almost everyday for eight years in a variety of workflows, and not had problems, but I've only been using it with the multi-threaded version of AVISynth for about eighteen months, and I have definitely seen some issues. Until now, I never thought it could be part of the problem.

[-Vit-]

I've posted the latest version v3.30 in the first post (http://forum.doom9.org/showthread.php?t=156028).

This version includes some speed-ups and bug fixes:

# v3.30
# - Added ChromaMotion setting - can ignore chroma when analyzing motion (a speed-up with few ill-effects)
# - Added ChromaEdi setting - allow faster interpolation for chroma than luma (mainly useful for EEDI2/3)
# - New fast pre-filter (SrchClipPP) mode 1, previous modes 1 & 2 are now modes 2 & 3. Presets all updated with new modes.
# - EdiMode "TDIYadif" changed to "RepYadif" = Yadif+Repair giving less artefacts & more speed. Is default mode for "Ultra Fast"
# - Preset is now the first setting, allowing a simpler form: QuickTGMC( "Slow" ). Old TGMC style will no longer work
# - Workaround for padding bug in MVTools that crashed faster modes. Removes need for SafeMode setting, which has been removed


Speed-ups
Setting ChromaMotion=false makes QTGMC ignore chroma when analyzing motion. This provides a nice speed-up, and for the vast majority of the time the motion analysis comes out the same. [Edited:] There can be situations where moving items differ by color, but not by luma and in such a case the motion analysis could have used the chroma. I've seen this effect with a rapidly moving brightly colored object passing in in front of a plainly colored complex background: there was a tiny bit more ghosting with this speed-up enabled. This speed-up is used by default on all Presets "Slow" to "Ultra Fast". If you use "Slower" (default), "Very Slow" or "Placebo", set ChromaMotion=false to use this optimization

From a suggestion by Didée earlier, you can now select a different interpolation path for chroma. The EdiMode setting determines the main interpolation (NNEDI3 at various speed levels by default). Set ChromaEdi to use a different interpolation for Chroma, choose from "Bob", "Yadif" or "NNEDI3" (it uses the highest speed NNEDI3 settings). The idea is that it might be faster to process the chroma with a simpler interpolator and you might not notice the difference. However, it turns out that the default interpolator, NNEDI3 is already very efficient. By all means try this setting, but I found it was only a speed improvement if using an EEDIx variant as the main interpolator, e.g. QTGMC( EdiMode="EEDI3", ChromaEdi="Bob" )

I've added a simpler pre-filter mode for the fastest modes ("Super Fast", "Ultra Fast"). These presets were not pre-filtering before so this provides a quality improvement. Yet with the other changes above I still get an overall speed-up on these settings.

All presets from "Slow" and quicker are now faster on my machine and setting ChromaMotion=false can optionally speed up the slower settings. I would be interested to hear about other peoples speed experiences.

Other
I've worked around a bug in MVTools (http://forum.doom9.org/showthread.php?p=1485926#post1485926) that was causing crashes in the faster modes ("Very Fast", "Super Fast", "Ultra Fast"). If you had crashes in these fast modes but not the slower ones, then you may find this version fixes that. This means the SafeMode setting has been removed.

You can now call QTGMC without writing "Preset=". Like this:
QTGMC( "Faster" )
You must put your preset first if you use other settings though:
QTGMC( "Slow", Sharpness=1.2 )
You can no longer use the old TGMC format for settings (e.g. QTGMC( 2,1,1, 4,0,4 ) )

[SubJunk]

Very nice :)

[nhope]

Preset tables updated for 3.30:

Compared to "faster" (https://spreadsheets.google.com/ccc?key=0AtvvpQjptJLFdGxFaEZwNDNXdGliTi1KdElEUEVtTHc&hl=en&pli=1#gid=0)
Compared to "medium" (https://spreadsheets.google.com/ccc?key=0AtvvpQjptJLFdGVGQzFUa0ozWTZJYXItLWFFTURBMXc&hl=en#gid=0)

I can hardly keep up!

[nhope]

Can I please have clarification on which of the dll's from RemoveGrain-1.0.rar (http://home.arcor.de/kassandro/prerelease/RemoveGrain-1.0.rar) are now required? I have so far just had RemoveGrainSSE2.dll, Repair.dll and RSharpen.dll in my AviSynth plugins folder, but I'm trying YUY2 and it's throwing an error "No function named Interleaved2Planar". Is Interleaved2Planar a separate download or is it in the RemoveGrain package?

[nhope]

OK so I found SSETools (which provides the Interleaved2Planar function) in the older version of RemoveGrain (http://home.pages.at/kassandro/RemoveGrain/). I am ignorant about SSE2/SSE3 etc., but reading this (http://en.wikipedia.org/wiki/SSE3) makes me think that the majority of users should be using the SSE3 version of the filters, right?

Anyway, with SSE3Tools.dll in place, the error has now jumped forward to line 394, "mt_lutxy: unsupported colorspace. masktools only support planar YUV colorspaces (YV12, YV16, YV24)".

[nhope]

I went back to YV12 but version 3.30 is giving me a new crash that I didn't get with 3.11, 3.20 or 3.25. MeGUI version 1989 crashes if I have "Open Preview after AviSynth script selection" checked. Error message is:

AppName: megui.exe AppVer: 1.0.1989.0 ModName: unknown
ModVer: 0.0.0.0 Offset: 05537432

My script is:

AviSource("d:\fs.avi")
ConvertToYV12(interlaced=true)
AssumeTFF
QTGMC( Preset="faster", FPSDivisor=2 )

I'm on Core 2 Duo, XP SP3, everything in ST mode.

I can let you have the full MeGUI error report if it helps.

[jpsdr]

SET's 2.6 wins :)

Just out of curiosity, have you tried the version here (http://xhmikosr.1f0.de/index.php) ?

[Taurus]

@nhope
I've just tried yor simple script with MeGui. Standard Dev installation. All updates.
QTGMC 3.30
Core 2 Duo, XP SP3
Avisyth 2.58 Standard
No MT
Input: Lagarith avi yv12 ok
Output: mkv x264 ok
Preview: ok
No crashes.
Tested with Lagarith 1320 and 1321.
Ver. 1323 is much faster but gives disturbed/corrupted frames on two machines.
Each WinXP 32bit SP3
Core2duo and AMD XP ?????

[nhope]

I'm not quite sure what you mean by "Core2duo and AMD XP ????? "

[-Vit-]

nhope: You are correct that I forgot to add SSE3Tools as a requirement for YUY2 support. It's in the OP now, I will add it to the documentation for the next version.

On RemoveGrain versions:
- The 0.9 zip has SSE3Tools but doesn't have mode 20, which is used by QTGMC
- The 1.0 zips do have mode 20 (although no SSE3Tools), but there is a bug in some SSE3 versions. There are some fixed versions around, but there is no notable SSE3 code in there anyway so just stick with the SSE2 versions.

The YUY2 problem you got surprises me (error in mt_lutxy). That part of QTGMC looks correct and I tested your script with my own YUY2 material and it works fine. Could you perhaps post a small snippet of your source so I could test. I also get no problems in MeGui preview. Perhaps seeing the source might help with that too.

Thanks for the spreadsheet updates. I don't have any major changes planned at the moment, so maybe you'll get a break now... or maybe not...

[Taurus]

I'm not quite sure what you mean by "Core2duo and AMD XP ????? "
Two different machines
One Intel
Second AMD
:confused::p

[Overdrive80]

Update to 3.30, spanish translation´s doc. http://www.megaupload.com/?d=IB7YAZVV

[nhope]

The YUY2 problem you got surprises me (error in mt_lutxy). That part of QTGMC looks correct and I tested your script with my own YUY2 material and it works fine. Could you perhaps post a small snippet of your source so I could test.

I'm using Debugmode Frameserver to frameserve from Vegas Pro in YUY2, same as johnmeyer. As this improves his MT stability, I'm testing if we can keep YUY2 right through to MeGUI without converting to YV12. MediaInfo reports this on the frameserved signpost .avi file:

General
Complete name : D:\fs.avi
Format : AVI
Format/Info : Audio Video Interleave
File size : 2.36 KiB
Duration : 501ms
Overall bit rate : 38.5 Kbps

Video
ID : 0
Format : VFW
Codec ID : DFSC
Codec ID/Info : DebugMode FrameServer VFW
Duration : 501ms
Bit rate : 1 918 bps
Width : 1 440 pixels
Height : 1 080 pixels
Display aspect ratio : 4:3
Frame rate : 29.970 fps
Bits/(Pixel*Frame) : 0.000
Stream size : 120 Bytes (5%)

Audio
ID : 1
Format : FrameServer
Codec ID : DFAC
Codec ID/Info : DebugMode SonicFoundry Vegas FrameServer ACM Codec
Duration : 500ms
Bit rate : 6 400 bps
Channel(s) : 2 channels
Sampling rate : 48.0 KHz
Stream size : 400 Bytes (17%)
Interleave, duration : 10 ms (0.30 video frame)
Interleave, preload duration : 500 ms

I opened the frameserved signpost .avi file in virtualdub and rendered uncompressed .avi but MediaInfo reports it as RGB. What VirtualDub codec/settings would allow me to encode it to a YUY2 file that would let you debug my error? Huffyuv? Something else?

I also get no problems in MeGui preview. Perhaps seeing the source might help with that too.

Here (https://s3.amazonaws.com/bubblevision/somosomo-test.m2t) is 4 seconds of HDV that gives my repeatable MeGUI preview crash with QTGMC 3.30, but not with 3.20 or 3.25. As it's anamorphic HDV I'm also resizing the width.

AviSource("d:\fs.avi")
ConvertToYV12(interlaced=true)
AssumeTFF
LanczosResize(1920,height)
QTGMC( Preset="faster", FPSDivisor=2 )

I should note that I can open the MeGUI preview with "Reopen Video Preview" once the AviSynth script has loaded. It only crashes when I have MeGUI set to open the preview automatically after script selection.

Two different machines
One Intel
Second AMD
:confused::p

Who has the AMD machine? We both have Intel Core2Duo don't we? :confused: too

[Taurus]

Who has the AMD machine? We both have Intel Core2Duo don't we? :confused: too
Ah, damn language barriers..:mad:
I just wanted to say that I've tested your script on different machines.
Yes, one Core2Duo and one AMD.
Sorry for writing unclear...
I will test your snippet later on.
Stay tuned.

[-Vit-]

Overdrive80: Thanks again, the OP is updated.

nhope: What exactly are you dragging into Megui to cause the crash? That script doesn't refer to the file you've uploaded. In any case the snippet you sent (it's 4:2:0) works perfectly. I can drag the m2t file directly into the Avisynth Script box and a preview comes up no problem. If I adjust your script to accept it as a source, I can drag that in too. QTGMC happily processes it. This is all on a fully patched dev version of MeGUI.

To get YUY2 out of VDub, you can use ffdshow/HuffyUV and set the Colorspace to YUY2 in ( Compression...>>Configure>>Encoder>>Generic )

[Taurus]

Yes, I can confirm it.
Loaded your file with an avisynth script which contains embedded QTGMC.
Indexed the *.m2t first, even converted to lagarith staying inYUV color space.
All three attempts were doing fine. Preview worked, no crashes.
Yes, and verified with a process viewer that all plugins were loaded correctly.
I'm out.
And as -Vit- mentioned:
Your script does't belong to the file you've uploaded

[SubJunk]

Just out of curiosity, have you tried the version here (http://xhmikosr.1f0.de/index.php) ?Is that version of 2.6 multithreaded?
I tried his build of MVTools2 before and found it was slower than the official build (results here (http://www.avsforum.com/avs-vb/showpost.php?p=20041489&postcount=933)) so I don't have great confidence in his builds but maybe they have improved since then (last month).
Have you tried them?

[nhope]

-Vit-: "d:\fs.avi" is the avi signpost file from the Debugmode Frameserver which frameserves out of Vegas. So the HDV snippet I sent is sitting on the Vegas timeline and gets frameserved out as to AviSynth as RGB24 (or RGB32 or YUY2). Look 40% of the way down the guide in my signature to see screenshots.

So to get the frameserving out of the loop I tried accessing it direct with this script:

DirectShowSource("d:\somosomo-test.m2t")
ConvertToYV12(interlaced=true)
AssumeTFF
LanczosResize(1920,height)
QTGMC( Preset="faster", FPSDivisor=2 )

With QTGMC 3.11, 3.20 and 3.25 I have no problem, but with 3.30 I still get the MeGUI preview crash. Sometimes it bombs straight out, sometimes it shows me the middle preview frame and then as soon as I click on another time point it crashes. I have MeGUI version 1989. It's the stable version but it's pretty recent and I can't find a newer development version. MeGUI and all the plugins were updated a few days ago. I also tried fast and very fast QTGMC presets and I get the same crash.

As for the YUY2 issue, I'll get onto that tomorrow. Thanks.

[nhope]

Taurus: Thanks for trying that out. What MeGUI version do you have?

By the way (until my tower is fixed) I'm using a Dell Precision M6300 laptop, Core 2 Duo T7800 @ 2600MHz, 4GB RAM, XP SP3

[jmac698]

http://avisynth.org/mediawiki/External_filters#Deinterlacing
http://avisynth.org/mediawiki/QTGMC

[mr.duck]

First time avisynth user here.

I just got QTGMC working here to deinterlace standard definition TV in ffdshow. I used the preset "Super Fast" but it slows to a crawl and cannot get anywhere near 50 FPS. I have a Athlon X2 2.1GHz CPU. Is QTGMC not suitable for what I am trying to do?

[-Vit-]

jmac698: Thanks for that.

nhope. Your perseverance has paid off. I reproduced your problem in the MeGUI preview and fixed the cause, there's a new version v3.31 in the OP (http://forum.doom9.org/showthread.php?t=156028). There's no need to update documentation or spreadsheets, it's only this bug-fix.

mr.duck: Are you trying to use QTGMC in realtime? It's not really designed for that, it's designed to make high quality results to encode into another file (i.e. to make an mp4 or mkv). However, if you have a decent machine and use the multi-threading tips in the documentation you can get realtime SD playback for the quicker settings. Not sure how well your machine would cope with that.

[mr.duck]

Yes it's realtime.

I was searching for good avisynth deinterlacers that are better than yadif which comes with ffdshow. yadif doesn't need much CPU power at all and yet gives excellent results.

I couldn't really find any info on realtime avisynth deinterlacing. If anyone knows the best way to do it, I'm all ears...

[Didée]

yadif doesn't need much CPU power at all and yet gives excellent results.
If you judge Yadif's result as "exzellent", then what are you looking for, at all? Just use Yadif and enjoy its excellent desintegrat... erh, deinterlacing!

With some luck, this here (http://forum.doom9.org/showpost.php?p=1474120&postcount=161) might run for you in realtime, if you include blksize=16 into MAnalyse.
But frankly, a 2 GHz Dual Core is not well suited to perform MVTools-motion-compensated deinterlacing in realtime ....

[mr.duck]

If you judge Yadif's result as "exzellent", then what are you looking for, at all? Just use Yadif and enjoy its excellent desintegrat... erh, deinterlacing!

I tried software deinterlacing quite a while ago and found it to be quite poor compared to hardware deinterlacing. But installed a recent build of ffdshow and I couldn't believe how good yadif was. I could not tell it apart from hardware vector adaptive deinterlacing!

I will enjoy it's excellent result... but if I can get better than yadif then I will enjoy it even more ;)

[Taurus]

Taurus: Thanks for trying that out. What MeGUI version do you have?

Latest MeGui dev version. You can change to the update server within MeGui/Options.
Well, I wonder how -Vit- managed to crash MeGui.
I could scroll in the timeline, expand the preview window, stop, reload the video to my heart out.
No crashes.
But if it' fixed, it's fixed:p.

@mr.duck.
But installed a recent build of ffdshow and I couldn't believe how good yadif was
So what are you waiting/looking for?
If it fullfills your needs, use it.
If you want to see some real magic, try QTGMC or it's mother/father TGMC.
I never was satisfied with deinterlacers until I found Didée's and -Vit-'s scripts.
Thank you to both of you.

[nhope]

nhope. Your perseverance has paid off. I reproduced your problem in the MeGUI preview and fixed the cause, there's a new version v3.31 in the OP (http://forum.doom9.org/showthread.php?t=156028). There's no need to update documentation or spreadsheets, it's only this bug-fix.

Thanks very much for a such a fast fix. I see it was just one value in the new ChromaMotion code. I updated the version numbers on the 3 spreadsheets anyway to 3.31.

Latest MeGui dev version. You can change to the update server within MeGui/Options.

At the moment the development server isn't offering me anything that the stable one isn't, since there was a big update a few days ago. As so many variables are involved here, I think I prefer to stay with the stable update server. I'm really trying to put a "stake in the ground" regarding version numbers of programs and filters that, in most cases, work according to my guide. Once I've done that and spread the word a bit, that page is going to get a lot of traffic. Apologies in advance to -Vit- for all the people that are likely to come here saying "This isn't working on my 64-bit PC. I'm using Debugmode Frameserver blah blah...".

Well, I wonder how -Vit- managed to crash MeGui.

Maybe he dug out his old Commodore Amiga.

[Taurus]

@nhope
Great that it works now.
Nice guide you wrote!

[Delerue]

I posted this on x64 Avisynth thread, but maybe it's better here:

I've tried to run QTGMC with all 64-bit dependencies, but no success. It crashes instantly with MPC-HC x64 1.5.2.2972 + FFDShow x64 3760. I tried with avisynth tab inside FFDShow (checking 'Add ffdshow video source') using only this line of code:

QTGMC( Preset="Medium" )

Here's the list of the dependencies I have:

nnedi3_64.dll (0.9.2.0)
mt_masktools-25-x64.dll (2.0.48.0)
mvtools2.dll (last 'official' version of Joshua Duncan from the first page of this very thread)
avisynth.dll (same as above)
Repair.dll and Removegrain.dll (from here: http://code.google.com/p/avisynth64/wiki/PluginLinks)

Any idea?

Also, no success on running multi-thread with x86 version (although single-thread works flawlessly); instantly crash when using 'distributor()'. Without it I have no crash, but there's no multi-thread either. BTW, I'm using 'setMTMode(2,12)' because I have an i7 860. The problem doesn't seem to be related to MT version of 'avisynth.dll', as I can run MVTools2 with multi-thread.

Thanks

[jpsdr]

Is that version of 2.6 multithreaded?
No, it's standard 2.6 and MT 2.5.8.


Have you tried them?

Actualy, all my work are not MT script with x64 bit version.
If i'm not using MT in my script it's because all the slowestfilter/things i'm using are already multi-threaded themselve, and i'm not using QTGMC for now.

I was just asking out of curiosity, because i've noticed that he was doing updates of 2.6 and 2.5.8MT build regularly. So, i assumed (but maybe wrong), that maybe in these build there was regularly update/bug fixes, things you don't find in others MT avisynth versions, because these others builds begin to be "a little old", and don't seem to be updated, contrary to xkomisar's builds.

[Didée]

... couldn't believe how good yadif was. I could not tell it apart from hardware vector adaptive deinterlacing!
It seems that Yadif indeed is similar to HW deinterlacing. Or vice versa.

You might want to have a look at this little comparison (http://forum.doom9.org/showpost.php?p=1433847&postcount=2589).

Beware - this is the RED pill to take. :)

[nhope]

I nailed my problem with YUY2. I was using mt_masktools-26.dll instead of mt_masktools-25.dll. Presumably the version I was using was intended for AviSynth 2.6.

-Vit-, I think it would be a good idea in the documentation to specifically state exactly which files are required from the "Requirements". This will help not-so-avisynth-savvy users and keep problems like mine off your plate. i.e.:

RemoveGrainSSE2.dll
RepairSSE2.dll
RSharpenSSE2.dll
mt_masktools-25.dll (for 32-bit use in AviSynth 2.5)
SSE3Tools.dll (for YUY2)

Now YUY2 is working, I'll do a few speed trials and check out the colours/levels.

[Boulder]

-Vit-,

do you use some other program than Notepad to save your script? It seems that in the latest versions (at least v3.30 and v3.31), opening in Notepad shows a bit messy script. It looks to me that the script has been saved in Unix format instead of DOS.

[nhope]

Also, no success on running multi-thread with x86 version (although single-thread works flawlessly); instantly crash when using 'distributor()'. Without it I have no crash, but there's no multi-thread either. BTW, I'm using 'setMTMode(2,12)' because I have an i7 860. The problem doesn't seem to be related to MT version of 'avisynth.dll', as I can run MVTools2 with multi-thread.

I haven't tested this personally but some are getting more success with SEt's version of AviSynth 2.6 (http://forum.doom9.org/showthread.php?t=148782). If you use that you might want to use mt_masktools-26.dll, not mt_masktools-25.dll.

[-Vit-]

Boulder: Somehow the line breaks were converted to UNIX-style recently. I've updated the OP (http://forum.doom9.org/showthread.php?t=156028) to use Windows style line breaks again so it should be more readable now (no version number change). I use Notepad++, Visual Studio or AvsPmod to edit, but those editors will accept any line break style so I didn't notice the problem.

Delerue: You cannot always run 1 thread per core because you might run out of memory. On 32-bit avisynth you have 2Gb memory regardless of your machine. Multi-threaded complex HD scripts need more memory (which is why we need 64-bit Avisynth). Cut down the number of threads. Also always try different versions of MT avisynth. Just because MVTools works for you multi-threaded, doesn't mean that you are using the most stable MT version for your machine.

nhope: I'm going to add plugin packages for 2.5x, 2.6x and x64-2.5x to the OP to ease the plugin confusion. I'm not in my lair at the moment so I can't test anything properly - it will have to wait.

[Delerue]

I haven't tested this personally but some are getting more success with SEt's version of AviSynth 2.6 (http://forum.doom9.org/showthread.php?t=148782). If you use that you might want to use mt_masktools-26.dll, not mt_masktools-25.dll.

Thanks, but it didn't work either. Only single-thread.

Delerue: You cannot always run 1 thread per core because you might run out of memory.

I've managed to get x64 version to work using another version of Avisynth. But no success with multi-thread, again. I'm aware of the problem regarded to huge memory usage when working with lot of threads, but I believe that the problem here has nothing to do with memory as it doesn't matter how many threads I set, it always crash instantly as soon as I put 'distributor()' line. I even tried something like this with no success:

setMTMode(2,3)
QTGMC( Preset="Fast", EdiThreads=3 )
SelectEven()
distributor()

[Didée]

For the love of Manitu, make your first x64 tests with a VfW application like VirtualDub. Avery offers an x64 version "VeeDub".

If it works there (and there are good chances it will, I can personally confirm "success" with it) ... then it's time to go asking why it doesn't work in other scenarios.

After all, Avisynth support in ffdshow is somewhat fishy. Even in 32bit country. It works, but several things don't react like expected. And I suspect in 64bit country it's rather worse than better, if anything.

[Delerue]

Thanks, Didée. I managed to make it work with x64 AND MT using this:

setMTMode(2,4)
ffdshow_source()
QTGMC( Preset="Medium", EdiThreads=4 )
SelectEven()
distributor()

Apparently, unchecking 'Add ffdshow video source' and using 'ffdshow_source()' line fixed the crash.

Unfortunately, without 'SelectEven()' the motion is choppy/weird to the point that sometimes frame 2, for example, can be motion positioned before frame 1, and frame 3 is after frame 1; a completely mess. I tested with 640x480@30 FPS video.

[mr.duck]

You might want to have a look at this little comparison (http://forum.doom9.org/showpost.php?p=1433847&postcount=2589).

Beware - this is the RED pill to take. :)

hmmmmmm... I may be back after upgrading to a 8 core CPU. It will be a while :p

[nhope]

I'm going to add plugin packages for 2.5x, 2.6x and x64-2.5x to the OP to ease the plugin confusion.

That sounds like an excellent idea, especially as one or two of the required filters' servers seem to have off days.

YUY2 support seems to be fine. It's the same speed as YV12 and the result is identical on the scopes.

Also I'm seeing a 5-8% speed increase in 3.31 over 3.25 using "faster" preset (single-threaded), and negligible difference in the result.

[-Vit-]

Unfortunately, without 'SelectEven()' the motion is choppy/weird
Add "ComplementParity()" after your source line?

[SubJunk]

No, it's standard 2.6 and MT 2.5.8.



Actualy, all my work are not MT script with x64 bit version.
If i'm not using MT in my script it's because all the slowestfilter/things i'm using are already multi-threaded themselve, and i'm not using QTGMC for now.

I was just asking out of curiosity, because i've noticed that he was doing updates of 2.6 and 2.5.8MT build regularly. So, i assumed (but maybe wrong), that maybe in these build there was regularly update/bug fixes, things you don't find in others MT avisynth versions, because these others builds begin to be "a little old", and don't seem to be updated, contrary to xkomisar's builds.Ah I see. Well regardless of bugfixes I need MT. I have a quad core CPU so to only use one core would be to severely limit the speed :)

[Delerue]

Add "ComplementParity()" after your source line?

Perfect! Now I really can see the quality of this deinterlacer. Thanks a lot. ;)

[jpsdr]

Ah I see. Well regardless of bugfixes I need MT. I have a quad core CPU so to only use one core would be to severely limit the speed :)

As i said, the 2.5.8 is MT, and regularly updated. These are the facts. After, i don't realy know what there is in the update. I simply assumed it's bugfixes. So, between an old 2.6MT and a regularly updated 2.5.8MT, i've choosen the regularly updated. Now, is my choice correct ? I don't know. This is why i've asked if you have tested this version.

[SubJunk]

No, I haven't tried it. I will :)

[*.mp4 guy]

It seems that Yadif indeed is similar to HW deinterlacing. Or vice versa.

You might want to have a look at this little comparison (http://forum.doom9.org/showpost.php?p=1433847&postcount=2589).

Beware - this is the RED pill to take. :)

...I've never seen TGMC used to remove dotcrawl before, but clearly it works pretty well.

[Didée]

Well, that's a side-effect of the temporal processing. (There often is a positive effect on dotcrawl, but it's not a specialised feature --> "YMMV").
The main point of that small comparison is instability & flickering of the actual detail.

Just yesterday I played with a DVB cap of a 50i TV show, and compared TGMC with HW-deinterlacing via DGDecNV. No need to mention that TGMC was stable and NV-Bobbing was flickering. But another point was the encoded filesizes:

x264 @ CRF 18:

TGMC-bob: ~5100 kbps
NV-bob: ~9500 kbps

That is, HW deinterlacing required almost double the bitrate to encode something with noticeable lower quality.

[*.mp4 guy]

But another point was the encoded filesizes:

x264 @ CRF 18:

TGMC-bob: ~5100 kbps
NV-bob: ~9500 kbps

That is, HW deinterlacing required almost double the bitrate to encode something with noticeable lower quality.

Based on what it did to the dotcrawl, which can be quite expensive to encode and the massive boost to over-all temporal stability from TGMC, I would be much more surprised if there wasn't a large difference in file size.

A more apples to apples comparison might be denoising the other bobbers with mvtools and then checking their file sizes, though that isn't a very realistic usage scenario, pretty much everyone would just use TGMC in such a situation (or rather, they should). I suppose for speed purposes actual usage might look like hardwarebob->fft3dGPU, which is no longer directly analogous to TGMC qualitatively (almost purely spatial vs almost purely temporal processing, everyone knows how that ends).

[Didée]

Well, it'll never be apples to apples. Because of the different technics, there will always be some flavour of peas and bananas.

For reviewing, a 15sec sample ...

x264 --crf 18 --tune filme

CUDA-bob, direct: 8530 kb/s +86%

CUDA-bob + MDegrain2: 6422 kb/s +40%

TGMC: 4585 kb/s [ref]

All three samples: clickme (http://www.mediafire.com/?f98alr9ojnp6lt5) (MediaFire, ~35 MB)

- Plain CUDA bob is not good at deinterlacing, forces very big bitrate, but at least it is fast.

- CUDA with MDegrain2 is same bad at deinterlacing, still requires much bitrate, and it is not fast anymore.

- TGMC is TGMC.

All in all, this is too much noise about something very simple: HW deinterlacing can be an option for live viewing, but for encoding it is not a good choice. (Well okay, maybe for "burst encodings" ... in any case it's beyond me how someone with functionable eyes could rate this as "good quality" ...)

[*.mp4 guy]

All in all, this is too much noise about something very simple: HW deinterlacing can be an option for live viewing, but for encoding it is not a good choice. (Well okay, maybe for "burst encodings" ... in any case it's beyond me how someone with functionable eyes could rate this as "good quality" ...)

I think I may have been unclear, my intention wasn't to indicate that your conclusions were wrong. TBH I have always completely agreed with your conclusions regarding hardware bobbers, I just felt there was a bit of sloppiness in demonstrating it, so I played a bit of devils advocate. No harm meant.

[FlaShow]

Hello,

I want to run this code:

AVISource("Kids_16.avi")

QTGMC(Preset="Slower",ChromaNoise=True,denoiser="dfttest",DenoiseMC=True)
dfttest()
deen("a3d",4,8,9)
aWarpSharp(depth=12,blurlevel=4,thresh=0.2,cm=1)
nnedi3_rpow2(2)
LanczosResize(1024,600)


But it said there is no function named "QTGMC". I have download the latest version of QTGMC plugin and all the requirements but not work.
What should i do ?!!

[-Vit-]

Make sure you have installed avisynth properly. You must put the QTGMC.avsi script in your Avisynth plugins folder.

I have added a zip of all the required plugins in the first post (http://forum.doom9.org/showthread.php?t=156028). It contains instructions where to put everything.

[johnmeyer]

You have to use the "import" statement to use the script:

Import("E:\Documents\My Videos\AVISynth Scripts\QTGMC3.11.avsi")

[-Vit-]

Using 'import' is an option that some might prefer. However, Avisynth will autoload any '.avsi' files in the plugins folder, which is often more convenient. Downside being that you get problems if you end up with multiple versions of the same plugin in the autoload folder.

Edit @FlaShow: those QTGMC settings will not do any noise processing. Please read the manual:
Important: You must use a non-zero value in one of EZDenoise, EZKeepGrain or NoiseProcess or no noise processing will occur

[FlaShow]

thanks guys but it did not work :(

[Taurus]

thanks guys but it did not work :(
What did not work?

[Fizick]

I have added a zip of all the required plugins in the first post (http://forum.doom9.org/showthread.php?t=156028). It contains instructions where to put everything.

You forget GPL license terms - do not distribute binaries without source codes.

[pio_fawad]

hi,
thanx 4 script can u please tell if i leave script as

QTGMC( Preset="Slow" )
SelectEven()

should this remove any artefacts or grain automatically, if source is normal and i want to compress it???

THanx please tell

[Robert Martens]

Nevermind this! Everyone reading this, please see the next few posts before bothering with the files I mention here.

Just a quick note, I see in the OP that you're offering an older release of RemoveGrain 1.0, with the SSE3 issues; there's actually a newer version available (binaries (http://home.arcor.de/kassandro/prerelease/RemoveGrain-1.0.rar) and source (http://home.arcor.de/kassandro/prerelease/RemoveGrain-src-1.0.rar)), as per the final post in this thread (http://videoprocessing.11.forumer.com/viewtopic.php?t=97) on kassandro's forum. Apparently it was a problem with the Intel compiler he had used at the time, but the bugs have been addressed with these 2007 releases.

I've been using the SSE3 libraries from those archives for some time now, and have had no problems beyond what one expects when trying to multithread (Q)TGMC.

[-Vit-]

Fizick: The plugin package only contains unmodified binaries, so I only need link-to / offer the original source. I'll add a text file to the zip.

Robert Martens: Thanks, I'll check that out. I knew about the SSE3 problems - the plugin package contains the SSE2 version. But this might make things simpler. Although I recall that there's very, very little SSE3 code in it in any case...

pio_fawad: I assume you are deinterlacing, and want to remove some grain and other artefacts at the same time? Well QTGMC with those settings will help, you could add "TR2=2" or "TR2=3" for more smoothing/degrain. Or "EZDenoise=X" to remove noise, where X is a value that depends on how noisy your source is (anywhere from 0.5 to 40). However, there are many other specialized plugins that might be more effective at the clean-up.

[pio_fawad]

hi,
thanx 4 script can u please tell if i leave script as



should this remove any artefacts or grain automatically, if source is normal and i want to compress it???

THanx please tell

please @Vit Answer waiting thanx

[-Vit-]

pio_fawad: I have already edited the post above

[Didée]

The situation with RemoveGrain package is tricky.

Basically, I agree that the "official" v1.0 version should be preferred over the older pre-v1.0 versions. But then, how it comes that e.g. a certain Mr. Didée is regularly recommending the old pre-v1 version?

- v1.0 final is only backwards-compatible with pre-v1 in regards to the spatial filters.

- v1.0 final is backwards-incompatible in regards to the temporal filters. Namely, clense().

In effect, when v1.0 is getting used, then 99% of all scripts/functions that are using clense will not work anymore.

Of course the older scripts can be adapted accordingly. But it'll be a long way to go, with dozens or hundreds of mini-scriptlets out there that people might pick up from here, there, or elsewhere.


And, the final showstopper:

RemoveGrain v1.0 final is not documented. There are substantial changes that cause backwards incompatibility, and there is no documentation how to use v1.0 correctly.

That's a problem.


Staying with the old version might be somewhat unsatisfactory, but the situation is so that it avoids lots of problems.
The other way round, there's actually not much to "gain" from switching to the v1.0.final version. People crying for the SSE3 filters simply are clueless. The potential speed gain is practically irrelevant.

[Robert Martens]

I must admit to being a bit confused with the various RemoveGrain packages; I was under the impression the only difference between the version Vit has been offering in the first post and the one I just linked to was that the newer release fixed a problem with SSE3 instructions.

I'm not terribly concerned about using said instruction set, and can easily imagine there's minimal performance gain, but if it's a choice between recommending and distributing a broken release and one that's been fixed, I thought the latter would be preferred.

Of course, as you explain, that's apparently not all that's changed from one version to the next.

If I could bug you to clarify something, however, what do you mean by "1.0 final"? The file I mentioned is listed as "1.0 prerelease" on the wiki (http://avisynth.org/mediawiki/Removegrain). Do you recommend using what that page labels "1.0b" instead? That is to say, Vit's got nothing to change in his recommendations and I will instead need to modify mine?

[-Vit-]

Robert: I thought you were referring to a version of RemoveGrain I hadn't seen (I haven't really explored Kassandro's forum), but this is one that I deliberately avoided using after reading Didée's points above in an earlier thread.

There are three versions:
- v0.9 is the only one to contain SSE3Tools, which I need for YUY2 support. But it doesn't contain some newer modes which QTGMC uses. So I link to that one in the OP but indicate to take only SSE3Tools
- v1.0b (beta) contains all the modes, has source code and documentation, but doesn't contain SSE3Tools. It also has an SSE3 bug, but no real SSE3 code. I link to that in the OP and indicate to take the SSE2 versions
- v1.0 (pre-release) contains the SSE3 fix, but also some other undocumented changes and no source code. I cannot recommend that one.

I have noticed that the html documentation needs to be updated on these points though.

[Robert Martens]

Beautiful, thanks for the rundown! I'll update myself accordingly.

[-Vit-]

The plugins package in the OP now adheres to the terms of the GNU GPL v2. Please advise of any issues with respect to that.

[Fizick]

"written offer" 2b) clause ? :confused:
-Vit-, do you really want to send CD with source codes?
Why not upload the (separate) full source code package?
http://www.gnu.org/licenses/old-licenses/gpl-2.0-faq.html#UnchangedJustBinary

and please do not forget GPL inself (with binaries too).
http://www.gnu.org/licenses/old-licenses/gpl-2.0-faq.html#WhyMustIInclude

[Boulder]

Is using "noiseprocess=0", "tr2=0" and "smode=0" the way to minimize both the denoising and sharpening effect of QTGMC? I'm looking to output high quality smart bobbed video and then do the denoising and sharpening using different tools on the result.

[-Vit-]

Fizick: The final paragraph of clause 3 clearly sets out that there only needs to be equivalency of distribution channel between binary and source. So whilst a product distributed physically would require a physical source distribution, in this case I only need an offer of equivalent access, i.e. internet distribution. [The v3 FAQ alludes to this distinction (http://www.gnu.org/licenses/gpl-faq.html#DistributeWithSourceOnInternet) for v2 - note the reference to distribution on physical media]. Furthermore, as long as the OP has a link to the source code that is valid and synchronized then I am offering access to the dll and source from the same place, which essentially meets the stipulations of clause 3 anyway. The text in the zip is just for good measure. I have added the GPL text to the zip.

Boulder: I do this all the time. You want SourceMatch and EZKeepGrain for that. Something like this is near transparent. It will be slow:
QTGMC( "Slow", SourceMatch=2, Lossless=2, EZKeepGrain=0.4, Sharpness=0.1 )

[Boulder]

Thanks, I will try those settings out :)

[hydra3333]

Using MT 2.5.7 and x264.exe crashes randomly with this script (a tad disconcerting 6 hours into an encode)...
SetMTmode(mode=5,threads=4) # start with mode=5 forAVIsource http://forum.doom9.org/showthread.php?p=1067216#post1067216
SetMemoryMax(384)
LoadPlugin("C:\SOFTWARE\DGindex\DGDecodeNV.dll")
LoadPlugin("C:\SOFTWARE\DGindex\DGDecode.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins-zzz\Yadifmod.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins-zzz\DePan.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins-zzz\AGC.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins-zzz\Cnr2.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins-zzz\dctfilter.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins-zzz\fft3dfilter.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins-zzz\degrainmedian.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins-zzz\Convolution3d.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins-zzz\despot.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins-zzz\WarpSharp.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins-zzz\aWarpSharp.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins-zzz\mt_masktools-25-broke-backward-compatibility.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins-zzz\mvtools2.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins-zzz\Unfilter.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins-zzz\AddgrainC.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins-zzz\hqdn3d.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins-zzz\Colormatrix.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins-zzz\NNEDI2.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins-zzz\NNEDI3.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins-zzz\EEDI2.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins-zzz\EEDI3.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins-zzz\VerticalCleanerSSE3.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins-zzz\dfttest.dll")
# -------------------------------------------------------------------------------------
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins-zzz\VariableBlur.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins-zzz\FFTQuiver.dll")
# -------------------------------------------------------------------------------------
#LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins-zzz\RemoveGrain.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins-zzz\RemoveGrainSSE2.dll")
#LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins-zzz\Repair.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins-zzz\RepairSSE2.dll")
#LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins-zzz\RemoveDirt.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins-zzz\RemoveDirtSSE2.dll")
Import("C:\Program Files (x86)\AviSynth 2.5\LimitedSharpenFaster.avs")
Import("C:\Program Files (x86)\AviSynth 2.5\Deblock_QED_MT2.avs")
Import("C:\Program Files (x86)\AviSynth 2.5\QTGMC-3.31.avsi")
MPEG2Source("G:\HDTV\the-file.d2v",info=0,ipp=true,cpu=4)
AssumeTFF()
Assumefps(25)
changefps(last,last,true) # http://forum.doom9.org/showthread.php?p=1473445#post1473445
SetMTmode(mode=2,threads=4)
QTGMC(Preset="Fast",EdiThreads=2,Sharpness=1.2,SLMode=1) # result is double framerate progressive, so re-interlace it later
Lanczos4resize(720,576)
# Re-Interlace and halve framerate
AssumeTFF() #choose the SAME field order like before deinterlacing
SeparateFields().SelectEvery(4,0,3).Weave() #reinterlace.
#
ColorMatrix(interlaced=true,mode="Rec.709->Rec.601")
AssumeTFF() #choose the SAME field order like before deinterlacing
AssumeFPS(25)
SetPlanarLegacyAlignment(True)
Distributor()
Remove the MT components and it works fine.
Suggestions appreciated.

[Gavino]

Remove the MT components and it works fine.
Suggestions appreciated.
Looks like you have the solution right there.
:) only half-joking

[-Vit-]

I could never get stability from 2.57MT. I have only heard a couple of reports from those who could. Try one of SEt's builds, 2.58MT or 2.6MT.

[hydra3333]

OK ... now I'm off to locate and research "SEt's builds, 2.58MT or 2.6MT" (for known bugs, limitations) ... I recall looking at them some months ago and thought they appeared too fragile then; hope things have improved.

edit: Seems to me like "choose your poison" as all versions crash ...

Not so good for 2.5.8 MT -
a) Where TGMC + SetMTmode(2,4~6) would crash occasionally in Avisynth 2.5.8.5.MT, it seems rock stable in Avisynth 2.6 (SEt's 09-19 build).
b) in Avisynth 2.5.8.5.MT, QTGMC + SetMTmode(2,4~6) crashes more often for me than TGMC does. Can't tell why, it just is like that.

Not so good for 2.6 MT either -
Well, I can now happily report that I have completely resolved my bizarre KERNELBASE and unknown exception crashes with Avisynth 2.6. It occurred to me to go back and check to make sure that there weren't any memory access issues since this is 32-bit softwares. I found Avisynth crashing with these errors after running up to 1.6GB of RAM used. At first I didn't get it because previously I had tested my script with SetMemoryMax and had gotten the same errors. But I thought I'd test again with a very low limit at 1.2GB and when I did it worked flawlessly, zero crashes. I tested different values and got it up as far as 1500MB without crashing. I took a look at the script with which I had previously tested SetMemoryMax (I kept a record of all the scripts I had been testing) and found that the lowest I had gone in setting the memory limit was 1600MB.

I find that YToUV() with YUY2 do not work for Set 2.6 build. VirtualDub try show half horizontal size frame and crash. This reason why do not work nnedi3_rpow2().
SEt's avisynth.dll 32 MT Build 2.6.0 2009 09 19 in sysWOW64 on a Win7U64.
CPU i7920XM, 8GB RAM, SetMemoryMax=1536
...
Renders 50 frames in VirtualDub 1.9.10, then raises unknown exception.
No matter if script uses SetMTMode or not. Lowering SetMemoryMax to 768 did not help.
...
The same script on same engine, same system
with straight avisynth.dll 2.5.8.5 non-MT in sysWOW64 :
Both MemoryMax values run properly.
VirtualDub uses ~600MB.

[jpsdr]

@hydra3333

Just out of curiosity, have you tried the 2.5.8MT you can find here (http://xhmikosr.1f0.de/index.php) ?
I'm interested on what could the results be. These versions are updated regularly, so, maybe they could have fixes and could be more stable.

[-Vit-]

That all versions of Avisynth crash is a given, the Avisynth multithreading code is buggy and could do with a rewrite. However, SEt did fix some bugs in his version, which makes them slightly less prone. However, each system is different in which version suits best. You cannot search for the answer to this dilemma, you must practically try each version and see which works best for you.

There are no code fixes in xhmikosr's builds, in fact I see no code changes over SEt's 2.58MT. The fact that it's a clean VS2010 build might help a little, so worth a try.

[Boulder]

It looks like Avisynth 2.6 with MT support is not going to be here anytime soon.. IanB has said that he's not going to go the SetMTMode way so that means that MT code will basically have to be written from scratch. And as Avisynth 2.6.0 hasn't seen an official alpha release in over a year and a half, things are probably not going to process too fast :( Too bad since MT-capable CPUs have become very common items.

-Vit-, did you ever notice any improvements regarding stability by using customized plugins you mentioned earlier in this thread (the correct header files)?

[hydra3333]

On another tack, I am attempting to do a "high quality" size reduction from PAL 1440x1088i "HDTV" to 720x576i to burn a DVD.

The capture is a fast-movement sports action clip. Here is a 3Mb sample from the HDTV capture. http://www.mediafire.com/?awxorf6fo8q3520

I must be doing something wrong. I'd thought QTGMC would give me a very high quality "smooth movement" double-framerate deinterlaced result without shimmers etc which I could re-interlace to preserve "fluidity" of fast movement.

However, the result with this script ... well, look at the "50m arc" white line it jaggies and shimmers.
http://imagengratis.org/images/clipboardre4qr.jpg

Maybe nothing can be done, it just "is" ? Suggestions appreciated.
SetMemoryMax(384)
LoadPlugin("C:\SOFTWARE\DGindex\DGDecode.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins-zzz\Yadifmod.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins-zzz\DePan.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins-zzz\AGC.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins-zzz\Cnr2.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins-zzz\dctfilter.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins-zzz\fft3dfilter.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins-zzz\degrainmedian.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins-zzz\Convolution3d.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins-zzz\despot.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins-zzz\WarpSharp.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins-zzz\aWarpSharp.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins-zzz\mt_masktools-25-broke-backward-compatibility.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins-zzz\mvtools2.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins-zzz\Unfilter.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins-zzz\AddgrainC.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins-zzz\hqdn3d.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins-zzz\Colormatrix.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins-zzz\NNEDI2.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins-zzz\NNEDI3.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins-zzz\EEDI2.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins-zzz\EEDI3.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins-zzz\VerticalCleanerSSE3.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins-zzz\dfttest.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins-zzz\VariableBlur.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins-zzz\FFTQuiver.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins-zzz\RemoveGrainSSE2.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins-zzz\RepairSSE2.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\plugins-zzz\RemoveDirtSSE2.dll")
Import("C:\Program Files (x86)\AviSynth 2.5\LimitedSharpenFaster.avs")
Import("C:\Program Files (x86)\AviSynth 2.5\Deblock_QED_MT2.avs")
Import("C:\Program Files (x86)\AviSynth 2.5\QTGMC-3.31.avsi")
MPEG2Source("G:\HDTV\AFL\MP4\2\test.d2v",info=0,ipp=true,cpu=4)
#
QTGMC(Preset="Slow",EdiThreads=2,Sharpness=1.2,SLMode=1) # result is double framerate progressive, so re-interlate it later
Lanczos4resize(720,576)
AssumeTFF() #choose the SAME field order like before deinterlacing
SeparateFields().SelectEvery(4,0,3).Weave() #reinterlace.
ColorMatrix(interlaced=true,mode="Rec.709->Rec.601")
AssumeTFF() #choose the SAME field order like before deinterlacing
AssumeFPS(25)
SetPlanarLegacyAlignment(True)

[Didée]

Try SLmode=2 instead of SLmode=1.

Can't predict if that's the culprit here, but generally, the (spatial) limiting of SLmode=1|3 can cause shimmering on sharp/thin detail.

[hydra3333]

SLmode=2 yields almost identical result. I guess it must be a function of (re)interlacing. The larger-scale interlaced source appears to have very minor similar issues. The double-framerate progressive before re-interlacing looks very reasonable.

I do wish the PAL DVD spec included 576p50 :)

[Didée]

Ah, so you're talking about playback of the re-interlaced 576i stream.

You're probably missing the vertical lowpass. Are you watching this on a PC, or on a LCD TV set, or on an old-school tube TV?

Lanczos4resize(720,576)
AssumeTFF() #choose the SAME field order like before deinterlacing
Blur(0,1).SeparateFields().SelectEvery(4,0,3).Weave() #reinterlace.

[-Vit-]

Boulder: I always use my own mods of all the plugins which include SEt's fixes and a nasty hack of my own. They do help, but I haven't had a chance to thoroughly test in order to release. As they have all been converted to VS2010/Intel builds so there is scope for problems - I already get enough support queries on QTGMC...

hydra3333: Just run the QTGMC line from your script and the result is clean and shimmer free. It's the rest of the script that is the problem - <snip> ah... already other answers coming in...

[Didée]

Yes, seems that hydra3333 is fighting the basic technical shortcomings of interlacing.

Have a B/W photo (mpeg source), color it up with a nifty colorizer (QTGMC), then print it out on a B/W laser printer (reinterlacing).

Hey, it's only grey! Why are the final colors bad again?

[hydra3333]

Ah, so you're talking about playback of the re-interlaced 576i stream.

You're probably missing the vertical lowpass. Are you watching this on a PC, or on a LCD TV set, or on an old-school tube TV?

Lanczos4resize(720,576)
AssumeTFF() #choose the SAME field order like before deinterlacing
Blur(0,1).SeparateFields().SelectEvery(4,0,3).Weave() #reinterlace.

LCD TV and PC LCD. Appears on all. Yes it seems I do not understand the technical limitations of interlacing ... I had thought the issues would have been less visible. Blur(0,1) coming up.

edit: as an aside, the reduced-size 576p50 in a .mp4 from x264 looks very nice indeed from a WDTVlive on a 32"LCD TV... almost better than the original 1440x1080i ! No good for a DVD though :(

[Boulder]

Boulder: I always use my own mods of all the plugins which include SEt's fixes and a nasty hack of my own. They do help, but I haven't had a chance to thoroughly test in order to release. As they have all been converted to VS2010/Intel builds so there is scope for problems - I already get enough support queries on QTGMC...

Any chance you could release masktools-25.dll or -26.dll for testing? It seems that most of my crashes in VDub point to that dll.

[Gavino]

Lanczos4resize(720,576)
AssumeTFF() #choose the SAME field order like before deinterlacing
Blur(0,1).SeparateFields().SelectEvery(4,0,3).Weave() #reinterlace.
Would there be any advantage (or disadvantage) in doing the Blur before the resize?

Given you have to vertically blur the results, is there in fact any point in using (Q)TGMC for deinterlacing just in order to do an re-interlaced downsize? Would an 'inferior' but faster deinterlacer give equally good results, or do you still gain something from the temporal processing of (Q)TGMC?

Any chance you could release masktools-25.dll or -26.dll for testing? It seems that most of my crashes in VDub point to that dll.
I suspect that is simply because most of the time in QTGMC is spent inside masktools, so that's usually where it is when it runs out of memory or gets a bad pointer from the Avisynth cache. However, it is important to use the right version (25 or 26) to match your Avisynth.

[-Vit-]

This is a modded MaskTools 2.0a46 for Avisynth 2.5x which might be more stable with multithreading:

VS2010 (http://www.mediafire.com/download.php?o1apu584422a3ka) | Intel C++ 2011 (http://www.mediafire.com/download.php?vp9xpjg8p0whsdi) | Source (http://www.mediafire.com/download.php?0zkh6ueityoqmsk)

I believe the VS2010 version is more stable (hard to tell really). It is a bit slower though. [each Visual Studio compiler since around VS2005 generates slower code than the one before - what's that about? Profile-guided optimization doesn't help...]

All plugins started from a clean build and had all known bug fixes added (SEt's). The nasty hack: I removed the exception caused when the plugin finds that the frame it was about to work on is already in use. Instead the thread sleeps for a while and tries again ... it tries several times until the frame is free... and if that fails it just processes the frame anyway. Very ugly, I know. I can give you no guarantees at all that this will help you. You will still get out of memory conditions if you run too many threads or overburden the Avisynth cache.

[Boulder]

Thanks, I'll test the file as soon as my next crash occurs. I've been trying to bob Queen Live at the Wembley for at least fifteen times this evening, and having a long video with lots of motion doesn't help at all..

[hydra3333]

1. Would there be any advantage (or disadvantage) in doing the Blur before the resize?

2. Given you have to vertically blur the results, is there in fact any point in using (Q)TGMC for deinterlacing just in order to do an re-interlaced downsize? Would an 'inferior' but faster deinterlacer give equally good results, or do you still gain something from the temporal processing of (Q)TGMC?
I'd be interested in knowing 'expert' the answers to these questions too. I suspect with 1, it might be a lot slower but I'm not sure of the gain.

Also interestingly, this much faster Brute-Force approach doesn't suffer from the QTGMC jaggies and looks "reasonable".
I really should compare the QTGMC-blur with this brute-force, side by side...
# BRUTE-FORCE PURE VERSION PER http://forum.doom9.org/showthread.php?p=1185790#post1185790
Global NewHeight=576
Global NewWidth=720
SeparateFields()
Shift = (GetParity() ? -0.25 : 0.25) * (Height()/Float(NewHeight/2)-1.0)
E = SelectEven().Spline36resize(NewWidth, NewHeight/2, 0, Shift)
O = SelectOdd( ).Spline36resize(NewWidth, NewHeight/2, 0, -Shift)
Ec = SelectEven().Spline36Resize(NewWidth, NewHeight/2, 0, 2*Shift)
Oc = SelectOdd( ).Spline36Resize(NewWidth, NewHeight/2, 0, -2*shift)
Interleave(E, O)
IsYV12() ? MergeChroma(Interleave(Ec, Oc)) : Last
Weave()
AssumeTFF()
edit: Compared with stackhorizontal and frame-by-frame - the QTGMC-noblur looks nicest and sharpest in places with no motion however in places of motion with lines it is striking in how the jaggies stand out. The QTGMC-blur and Brute-Force appear almost identical to most intents and purposes (relatively clean 1440x1080i source). So... in this instance with this type of source with the intent of an interlaced size reduction from HDi to SDi, I can attain circa 20x the speed by abandoning QTGMC and going brute-force. Obvious I suppose, but as a dummy I'd thought "yeah, I want quality, just knock it about with QTGMC to get a great deinterlaced image for size-reduction and reinterlacing".

[sadie]

Just thought I'd pop in and let you know that after opening v. 3.31 for the first time I'm getting a 'script error: there is no function name "interleaved2planar" ' referencing line 351. Both in Vdub and avidemux. I'm still using DirectShowsource. Thanks

[Fizick]

I only have the dll and can't provide source code for the changes at the moment, but thankfully masktools is license-free.


You are not correct with GPL again.

See readme.txt in masktools-v2.0a48.zip :
The source code is released as gpl.
gpl.txt (v2) is included too.

[-Vit-]

sadie: You must have YUY2 input, which is why Interleaved2Planar is required. Looks like you are missing SSE3Tools. Look in the OP and either download the plugins package from the top of the post, or get the individual SSE3Tools plugin from the list further down.

hydra3333: "QTGMC jaggies", grrrr... The problem was with your original reinterlacing script. The second script you're using does a better job of downscaled reinterlacing. However, it assumes an interlaced source. If you want to start from a progressive source (e.g. including one that you've bobbed) just replace the "SeparateFields()" line with "SeparateFields().SelectEvery(4,0,3)"

I.e. If you want to deinterlace first (for some temporal processing perhaps?) then your second script starts thus:

QTGMC(...settings...) #Or whatever bobber
SeparateFields().SelectEvery(4,0,3)
# rest of that second script

Probably there's a simplification to be made to the remainder of the script in the progressive case, but it will do...

Fizick: :sigh: Done.
I did look, but missed it - that reference is tucked away, and omitted from many places where you would expect it (the source code doesn't mention it). That would affect enforceability for certain cases.

[Fizick]

-Vit-, I ask you to distribute plugins archives "as is", with source codes and documentation (it is useful for end user and contains copyright).

[sadie]

Thanks, Vit, but alright, now that's done I merely get a "mt-makediff: unsupported color space, masktools only supports planar YUV color spaces..." error. Evidently, YUY has finally crept in after all these years. Mediainfo though still tells me I've got an indeterminate 'YUV'. Oddly, the same error now occurs with my earlier project of several weeks ago that like all the projects I've done since the beginning of time were discreetly processing at YV12. We're still talking about a simple DV import by the way. It shouldn't be YUY AT ALL, should it? OK, since my last project I'd removed my older NLE for an upgrade of the same brand, and have also replaced an earlier version of ffdshow tryouts for the latest official ffdshow (in order to make a stab at their little makeavis.exe... which didn't work either of course). I think a new DV codec filter has slipped into MP Player classic as well. I know this problem is at the edge of this thread's topic but if someone has a generalist's intuition as to what might have entropied my configuration I'd be mighty appreciative.

[Boulder]

Thanks, I'll test the file as soon as my next crash occurs. I've been trying to bob Queen Live at the Wembley for at least fifteen times this evening, and having a long video with lots of motion doesn't help at all..It seems that the dll has a positive effect on stability. At least I was able to finish one encode in one go and the second one (the Queen b*****d) crashed once quite early on and after lowering the number of threads by one, has now been going on for over 3.5 hrs. Let's see if it finishes OK, it has 8 hours to go.

[-Vit-]

-Vit-, I ask you to distribute plugins archives "as is", with source codes and documentation (it is useful for end user and contains copyright).
Yes certainly, you only needed to ask.

sadie: I don't know how YUY has crept into your workflow, but your error is strange. You should have only got that Interleaved2Planar error if your source is YUY2 (or at least Avisynth thinks it is). And once you got the missing plugin, your clip should have been converted to the special planar YUY2 format that masktools has accepted since Alpha 34. Which begs the question: do you have an up to date masktools2? If you do then something odd is afoot and I'd like to see a sample of that clip...

[Gavino]

hydra3333: "QTGMC jaggies", grrrr... The problem was with your original reinterlacing script. The second script you're using does a better job of downscaled reinterlacing. However, it assumes an interlaced source. If you want to start from a progressive source (e.g. including one that you've bobbed) just replace the "SeparateFields()" line with "SeparateFields().SelectEvery(4,0,3)"

I.e. If you want to deinterlace first (for some temporal processing perhaps?) then your second script starts thus:

QTGMC(...settings...) #Or whatever bobber
SeparateFields().SelectEvery(4,0,3)
# rest of that second script

Isn't QTGMC(...).SeparateFields().SelectEvery(4,0,3) the same as (but much slower than) a simple SeparateFields()? (and with some quality loss, given the latter is lossless.)

Or are you saying it only makes sense to do this if you had some temporal processing between QTGMC and SeparateFields()?

What are your thoughts on my earlier question:
is there in fact any point in using (Q)TGMC for deinterlacing just in order to do an re-interlaced downsize? Would an 'inferior' but faster deinterlacer give equally good results, or do you still gain something from the temporal processing of (Q)TGMC?

[-Vit-]

Isn't QTGMC(...).SeparateFields().SelectEvery(4,0,3) the same as (but much slower than) a simple SeparateFields()? (and with some quality loss, given the latter is lossless.)
Er... yes... scratch that thought. Well, not quite, you do get temporal effects in the original fields as well as deliberate adjustments to help fit the interpolated fields (it's not quite "quality loss"). But that's only very marginally helpful.

I don't have time (nor, it would seem, the ability) to think about this deeply right now, but I had the gut feeling that something like that sub-pixel shifted rescale in the script above could be done on a progressive source. Then separate fields/weave on the reduced size image to give a more usable result. Kinda what I was hinting at at the end of that other post.

And I completely agree that there doesn't seem to be too much point doing elaborate bobbing. Maybe you might avoid certain aliasing in the result if you interpolate properly at the higher resolution?

[sadie]

Vit, your indulgence as always is appreciated. My problem was indeed linked to an update of ffdshow. Quite simply their internal DV decoder had become unchecked causing Windows to default to an unidentifiable (MS?) 3rd party codec (with its config menu in Chinese no less, always a good sign).Also, I've got Pinnacle, Canopus and DVCPro dv decoding filters in MPlayerC, but oddly can get none of them to kick in. Curious to know what decoder/encoder combo you show in MPlayer when opening the clip I posted a few weeks back. Anyway, now it's back to trying to get a Makeavis dummy file to open in Premiere/Pinnacle.

[yup]

Hi all!

I am from YUY2 world.
SeT 2.6 build + mt_masktools_26.dll do not work with YUY2 colorspace, error from masktools function mt_makedif, mt_masktools_25.dll and Vit modded dll work without problem.
SSE3Tools more unstable comparing to SSE2tools (VirtualDub crash when push Start button at job list, SSE2Tools work before 1.5 hour SD material), using save AVI work but for small clip.


yup.

[hydra3333]

hydra3333: "QTGMC jaggies", grrrr... The problem was with your original reinterlacing script. The second script you're using does a better job of downscaled reinterlacing. However, it assumes an interlaced source. If you want to start from a progressive source (e.g. including one that you've bobbed) just replace the "SeparateFields()" line with "SeparateFields().SelectEvery(4,0,3)"

I.e. If you want to deinterlace first (for some temporal processing perhaps?) then your second script starts thus:

QTGMC(...settings...) #Or whatever bobber
SeparateFields().SelectEvery(4,0,3)
# rest of that second script

Probably there's a simplification to be made to the remainder of the script in the progressive case, but it will do...

Huh ? It is an interlaced source. What was wrong with the 1st script to make you go grrrr ? This thread http://forum.doom9.org/showthread.php?p=1110741#post1110741 talks about re-interlacing from a double-framerate deinterlaced clip and I gathered SeparateFields().SelectEvery(4,0,3).Weave() was the way to reinterlace the double-framerate progressive clip which QTGMC produced. I also recall reading recommendations that resizing an interlaced clip is best done by deinterlacing it first and then reinterlacing after the resize.... which I did with QTGMC /resize/reinterlace.

[Didée]

@hydra - did you see my comment about colorizing a B/W picture, then printing it to a B/W printer? That was not a mere joke. That was an analogy.

For purposes of downscaling,
xdest = 720
ydest = 576 # or 480
Spline36resize(xdest,last.height)
Bob(newheight=ydest).SelectEvery(4,0,3).Weave()
does a pretty fair job. Anything more exhaustive will yield only marginal improvement.

For the (very rare) case of interlaced upscaling, like 576i -> 1080i, it's a completely different story.

Main point#1 is that an interlaced source has 50% of data missing (not considering lowpassing). QTGMC is a sophisticated method to interpolate those missing 50% as good as possible.
At the moment you do re-interlacing, you are throwing away 50% of the data again. Poof!

Main point #2 is the kind of de-interlacing used during playback of an interlaced stream. Most realtime deinterlacers are using rather "simple" methods, and therefore can't cope well with "sharp" interlaced sources. That's why lowpassing usually is done before re-interlacing. This makes the job for the deinterlacer much more easy, so even the most simple ones can deliver an acceptable result.

[kolak]

I still can't use QTGMC for HD sources. Got my new 12 core machine, but can't use it with QTGMC, because it's crashing all the time. Tried to move to 64bit, but couldn't make it working at all- Vdub could not understand incoming video.

Last thing which I did not try is avisynth 2.6- is it much more likely to work or not?

Thanks

[-Vit-]

yup: Thanks for that detail, that might partly explain sadie's problem. I'll investigate to see if there is anything I can do in the YUY2/avisynth 2.6 domain. The SSE2Tools info is useful, I didn't experience any issues but I haven't tested much YUY2. It makes sense since RemoveGrainSSE3 had problems around that time too.

kolak: Yes, you should try every Avisynth version. For HD input, if can be a good idea to start at 4 threads, 1 EdiThread and work up. Try my modded MaskTools above, see if it helps. I have a whole set of those modded plugins if people are finding them useful.

hydra3333: Just meaning to say that QTGMC doesn't create the jaggies in your original script, the subsequent statements do. If you started with a sharp progressive source instead of QTGMC you would get the same effect. Didee's script can work with any bobber but it doesn't work out of the box, this is the intent
[EDIT: This isn't correct - see later posts for correct script]:

xdest = 720
ydest = 576 # or 480
Spline36resize(xdest,last.height)
Bob(height=ydest/2).AssumeFieldBased().Weave()

and with QTGMC:

xdest = 720
ydest = 576 # or 480
Spline36resize(xdest,last.height)
QTGMC(...settings...).Spline36Resize(xdest,ydest/2).AssumeFieldBased().Weave()

Little difference if ydest is much smaller than the original height, and almost the same as any other working variant.

[Gavino]

Didee's script can work with any bobber but it doesn't work out of the box, this is the intent:
...
Bob(height=ydest/2).AssumeFieldBased().Weave()

That's not right, as it will produce incorrect field alignment.
The problem with Didee's script (apart from the incorrect parameter name newheight) is that it was missing a SeparateFields() - it should be:
Bob(height=ydest).SeparateFields().SelectEvery(4,0,3).Weave()

For the same reason, your edited code with QTGMC will also produce incorrect field alignment. Ironically, I was about to post to say that I agreed with the more complex code you first posted, until I found you had edited your post and deleted it.

[-Vit-]

Ah, yes I see what you mean. I was fooled by single stepping through the output of my "correction" (breaking) of Didee's script - the result had no aliasing and appeared to be the same as my complex version. So figured this solution was overkill. Didn't notice the field misalignment. Your correct correction of Didee's script leaves aliasing.

So this more complex script performs cleaner interlacing for progressive (or bobbed) input:

# Progressive input here

NewWidth=720
NewHeight=576

Shift = -0.5 * Height() / Float(NewHeight) # Change to +0.5 if you want BFF output
Spline36Resize( NewWidth,Height() )
E = Spline36Resize( NewWidth,NewHeight/2, 0, Shift )
O = Spline36Resize( NewWidth,NewHeight/2, 0,-Shift )
Ec = IsYV12() ? Spline36Resize( NewWidth,NewHeight/2, 0, 2*Shift ) : NOP()
Oc = IsYV12() ? Spline36Resize( NewWidth,NewHeight/2, 0,-2*shift ) : NOP()

Interleave( E, O )
IsYV12() ? MergeChroma(Interleave( Ec, Oc )) : last

AssumeFieldBased().AssumeTFF() # Or BFF
SelectEvery( 4, 0,3 ).Weave()

[Edited to include Gavino's tweak and my fix from the posts below]

You're still at the mercy of your playback deinterlacer on the final result though. QTGMC will happily deinterlace the output of this script and produce a clean result, but a hardware deinterlacer might not.

[Gavino]

So this more complex script performs a cleaner downscaled reinterlace for progressive or bobbed input
That looks right.
A minor speed improvement would be to factor out the horizontal resizing to be done once only, by adding Spline36Resize(NewWidth, Height()) before doing the others.

It's interesting that if you set NewHeight=Height(), the process reduces to pure interlacing without a resize.
In this case, your script replaces the conventional SeparateFields().SelectEvery(4,0,3).Weave().
At first sight, this seems to be wrong, or at best a terrible waste of time, but arguably it is more theoretically correct since by using a downsizer to construct the fields, you automatically include the necessary lowpass filtering.

[hydra3333]

@hydra - did you see my comment about colorizing a B/W picture, then printing it to a B/W printer? That was not a mere joke. That was an analogy. ...

Main point#1 is that an interlaced source has 50% of data missing (not considering lowpassing). QTGMC is a sophisticated method to interpolate those missing 50% as good as possible.
At the moment you do re-interlacing, you are throwing away 50% of the data again. Poof!

Main point #2 is the kind of de-interlacing used during playback of an interlaced stream. Most realtime deinterlacers are using rather "simple" methods, and therefore can't cope well with "sharp" interlaced sources. That's why lowpassing usually is done before re-interlacing. This makes the job for the deinterlacer much more easy, so even the most simple ones can deliver an acceptable result.

Thanks. and to -Vit- and Gavino. I think I'll move to the script here http://forum.doom9.org/showthread.php?p=1488542#post1488542 with Gavino's width adjustment.

I see there's generally no benefit to a fancy deinterlacer when downsizing to a re-interlaced result.

[-Vit-]

I believe the "Shift" calculation in the above is incorrect, it should simply be:
Shift = -0.5 * Height() / Float(NewHeight)
I took the original formula from the script posted earlier, but it is inappropriate for this case .

In fact the script above is really just a rewrite of SeparateFields() so it is better presented that way. This is a new function SeparateFieldsMod():

# Separates frame-based material into fields using a resizer rather than simply dropping alternate lines.
# The resize effectively performs a vertical low-pass to produce result that will interpolate more easily
# May output with different dimensions. Note that NewHeight refers to the frame height, not the field height
# E.g. given progressive HD input, interlace to SD like so: SeparateFieldsMod(720,480).SelectEvery(4,0,3).Weave()

function SeparateFieldsMod( clip c, int "NewWidth", int "NewHeight" )
{
Assert( c.IsFrameBased(), \
"SeparateFieldsMod: SeparateFieldsMod should be applied on frame-based material: use AssumeFrameBased() beforehand" )
Assert( !c.IsYV12() || c.Height()%4 == 0, "SeparateFieldsMod: YV12 height must be multiple of 4" )
Assert( c.Height()%2 == 0, "SeparateFieldsMod: height must be even" )

NewWidth = default( NewWidth, c.Width() )
NewHeight = default( NewHeight, c.Height() )

shift = (c.GetParity() ? -0.5 : 0.5) * c.Height() / Float(NewHeight)
c.Spline36Resize( NewWidth,c.Height() )
e = Spline36Resize( NewWidth,NewHeight/2, 0, shift )
o = Spline36Resize( NewWidth,NewHeight/2, 0,-shift )
ec = c.IsYV12() ? Spline36Resize( NewWidth,NewHeight/2, 0, 2*shift ) : NOP()
oc = c.IsYV12() ? Spline36Resize( NewWidth,NewHeight/2, 0,-2*shift ) : NOP()

Interleave( e, o )
c.IsYV12() ? MergeChroma(Interleave( ec, oc )) : last

AssumeFieldBased()
return c.GetParity() ? AssumeTFF() : AssumeBFF()
}


SeparateFieldsMod() does the same as SeparateFields(), but with a low-pass to allow easier interpolation when deinterlacing.
Can resize the output fields by passing new frame size: SeparateFieldsMod( 720, 480 )
Rescaling progressive to interlaced takes the expected form: SeparateFieldsMod( 720,480 ).SelectEvery(4,0,3).Weave()

[Gavino]

I believe the "Shift" calculation in the above is incorrect, it should simply be:
Shift = -0.5 * Height() / Float(NewHeight)
I took the original formula from the script posted earlier, but it is inappropriate for this case.
Yes, well spotted. Here you are starting from progressive and resizing a framebased clip with different offsets to extract top and bottom fields. The earlier script (in whose origins I had a hand) starts from an interlaced clip, separates it and resizes as fieldbased, which requires a different pair of offsets.

I didn't think of that when I looked at your earlier version. I used the case of NewHeight=Height as a sanity check, which worked. For that case, both the wrong formula and the right one produce the same result, so I didn't notice the error.

[SubJunk]

Just out of curiosity, have you tried the version here (http://xhmikosr.1f0.de/index.php) ?I finally got around to comparing that version with SET's 2.6.

XhmikosR's AviSynth_258_MT_MSVC2010 (27 March 2011 build): 41.414fps
SET's 2.6: 41.878fps

Those are averages from 5 tests per build.
In other words there seems to be no advantage in using XhmikosR's builds, and over 1% speed increase from using SET's.
This is in keeping with my previous tests of XhmikosR's builds of things, generally they perform slower than any other builds for me. But I am on a modern system so maybe his are more optimised for old computers, I'm not sure.

[hydra3333]

Um, wasn't there a discussion over a few pages, in 2008, on this stuff here http://forum.doom9.org/showthread.php?p=1185790#post1185790 ?

[-Vit-]

That was regarding resizing fields. The above is about resizing progressive frames into fields. Same principle, but require different sampling offsets.

[Gavino]

Um, wasn't there a discussion over a few pages, in 2008, on this stuff here http://forum.doom9.org/showthread.php?p=1185790#post1185790 ?
Ah, that thread brings back memories. :)
I was a fairly naive newbie then and learned a hell of a lot about both interlacing and resampling, which has served as the basis of my understanding ever since (still learning, of course).

[egrimisu]

should i use qtgmc for ivtc or it will not work?

[henryho_hk]

Use a proper ivtc function for telecine materials.

[2Bdecided]

There really is no point using anything beyond bob() for interlaced HD > interlaced SD resizing. As Didée said, you need more blur, not more sharp! ;)

e.g...

blur(0.0,1.0)
sharpen(0.0,0.5)

...before re-interlacing.

Cheers,
David.

[-Vit-]

This script (http://forum.doom9.org/showthread.php?p=1185790#post1185790) (the basis for the related discussion above) is better than bob/blur for interlaced HD > interlaced SD resizing

[2Bdecided]

This script (http://forum.doom9.org/showthread.php?p=1185790#post1185790) (the basis for the related discussion above) is better than bob/blur for interlaced HD > interlaced SD resizingHi Vit,

I'm well aware of that script (http://forum.doom9.org/showthread.php?p=1161524#post1161524) ;) (an even earlier version).

I'm not about to discuss what's best ;) but it seem to me that wiping away all the detail above half-height (and some below that, since no resizer is perfect) makes interlacing utterly pointless. You might as well just have 240p60 or 288p50; these options would have been trivial in 1936 (first interlaced broadcasts), 1941 (NTSC) and later (PAL), but interlacing was used instead.

What looks nicest to you is entirely subjective, but I've tried most possible solutions for putting my HDV footage onto PAL DVD and watching via a CRT, and...
http://forum.doom9.org/showthread.php?p=1171877#post1171877
...YMMV!

My current experience is that it matters even less than I thought at that time because the final interlaced SD MPEG-2 encoding of typical home movies (i.e. lots of movemen) trashes the quality so much.

Cheers,
David.

[-Vit-]

I'm well aware of that script (http://forum.doom9.org/showthread.php?p=1161524#post1161524) ;) (an even earlier version).
It's tough being a novice here (yes, I am that). So many old threads discussing almost everything, hard to know/find it all...

I'm not about to discuss what's best ;)
Point taken. So more correctly put, that script is less likely to produce aliasing but will give a softer result. I much prefer it for the sample posted in this thread earlier, but I otherwise rarely deliberately interlace (yuk!) so I don't really have a strong preference...

___

On a totally different point, has anyone an opinion on whether that masktools variant (http://forum.doom9.org/showpost.php?p=1488378&postcount=543) I posted earlier gave better MT stability? Boulder made some encouraging noises, but has not come back yet...

[Boulder]

Based on around 10 encodes of variable length and variable filters (all scripts utilizing MaskTools in some way), the stability is somewhat better on my system. I get occasional crashes but not as often as I used to, and MaskTools is rarely pointed out as the one which has been doing something when the crash occurs. Though it's more like the buggy caching/memory handling causing the actual crash.

[johnmeyer]

I spent quite a bit of time yesterday with the script that -Vit- linked to, and compared resizing 1440x1080 interlaced footage from my FX1 camcorder down to NTSC 720x480 interlaced. I agree with what -Vit- says in the post immediately above, namely that it produces less aliasing. However, in all other respects it fails to preserve detail that it should be able to hang onto. I spent several hours tweaking, and creating other scripts on my own. In the end, I got much better results using the resizing built into my editing program, Sony Vegas. Using this tool, I created (and have been doing so for almost five years) NTSC interlaced DVDs that look great on both my old Sony WEGA CRT monitor and on my new Samsung LED LCD 55" display.

As someone who has written a lot of software that deals with interlaced material, I understand very well all the issues involved. If handled properly, interlaced material looks great on all my equipment and, compared to anything except for 60p, provides the best temporal/spatial combination of anything I can create. I have the option, via ESPN3, to watch sports in 24p via my Xbox 360, but I'd much rather watch on my SD satellite connection which broadcasts 480i. This is personal preference, obviously, but the point is that I have never seen any artifacts in a properly handled interlaced video, whereas I see all sorts of artifacts with temporally degraded video, especially when watching sports, which is non-stop movement (unless you watch a lot of golf ...).

As to whether halving the vertical resolution and providing a progressive display would have been a better move when TV was first invented, I'm afraid that would not have produced a better result. First, it is based on the incorrect assumption that you don't get any benefit from the odd (or even) scan lines when the signal is interlaced. Maybe that is not what is being said, so I apologize if I didn't understand correctly. When you look at the analog video signal (which is the only correct way to look at it, since digital video didn't happen for another half century), the 468 lines that make up the picture each contribute to the details.

Finally, people who are a lot smarter than me -- and smarter than anyone in this forum -- worked together during the late 1980s to invent HD television, and then again in the early 1990s when they re-defined it to make it digital (the original HD spec was analog). These very smart people not only included interlaced video in the spec, but made it the pinnacle of the quality ladder (1080p was added later). They did this partly because TVs in the early 1990s were still based on scanning CRT technology, but also because interlaced TV looks darned good. Yes, I know that some of you are going to disagree with that (my assertion that interlaced video looks good), but I'll bet that if you took 100 people who never edited one second of video in their life, and put them in front of a TV that could display interlaced video natively, not one of them would complain about video quality.

Yes, I know that most modern sets no longer display interlaced video natively (although there is no reason that cannot change), but all of them have the ability to handle interlaced video and if you've got a decent set, the results look great.

[Gavino]

In the end, I got much better results using the resizing built into my editing program, Sony Vegas.
Interesting - do you know what resizing algorithm it uses?
Does it have a specific interlaced resize operation, or do you have to deinterlace first and re-interlace after resizing?

[-Vit-]

Based on around 10 encodes of variable length and variable filters (all scripts utilizing MaskTools in some way), the stability is somewhat better on my system. I get occasional crashes but not as often as I used to, and MaskTools is rarely pointed out as the one which has been doing something when the crash occurs. Though it's more like the buggy caching/memory handling causing the actual crash.
Thanks Boulder. Each plugin you use needs that treatment, so you may get further benefit from other tweaked versions. I'll provide a set later.

[johnmeyer]

Interesting - do you know what resizing algorithm it uses?
Does it have a specific interlaced resize operation, or do you have to deinterlace first and re-interlace after resizing?Vegas Pro has two re-sizing options: bilinear and bicubic. I used bicubic for my tests.

The deinterlacing/re-interlacing required for proper re-sizing of interlaced material is not explicitly exposed to the user in Vegas Pro, but it obviously is being done "under the covers."

BTW, I have three versions of Vegas Pro installed (7.0d, 8.0c, and 10.0a). I used 8.0c for my tests.

Some posters in the Sony Creative Software forum have complained about the quality of Vegas' SD (for DVD) MPEG-2 interlaced video created from HD interlaced sources. As usual, some of those posters are just using the wrong settings. However, the two legitimate complaints are too much staircasing on diagonal lines, and video that looks soft compared to video shot natively in SD (480i) format. In my extensive tests the last few days, I did resizing using various deinterlace -> resize -> reinterlace scripts (including one that deinterlaced and frame doubled) and then looked at the result one field at a time. The Vegas fields definitely had more jaggies than the AVISynth scripts produced. However, these jaggies were on field lines, and therefore when viewed on moving video rather than still frames (and I always have to remind myself that looking at still images of interlaced material is always an invitation to draw incorrect conclusions), they are going to look OK.

The complaint about softness has more to do, I think, with expectations set by years of watching over-sharpened SD video.

What was much more interesting was the amount of detail lost by the AVISynth approaches. I posted pics over in the Sony forum. Here they are. The first is a very small portion of one field from the 720x480 video that resulted from scaling a 1440x1080 interlaced video from my Sony FX1:

http://i177.photobucket.com/albums/w208/johnmeyer/VegasHDtoSD.png

Here is the same frame, scaled using the AVISynth script, shown below (your code I believe :)). This is a ballerina in the middle of a fast pirouette:

http://i177.photobucket.com/albums/w208/johnmeyer/AVISynthHDtoSD.png

As you can see (by the blue arrows), there is a lot more detail preserved by the Sony resize. These images are small on my browser, so you may have to right-click and save them to your computer so you can blow them up and see for yourself.

The following is the code I used for the re-size tests, although I also used the much simpler version without the somewhat broken (in 2.6MT) ConvertToYV12() conversion and without the chroma shift code. I also tried using QTGMC deinterlacing, using the frame doubling option, and then re-sized and re-interlaced. The results were virtually identical to what I got with the various other options.

#Re-size function
#ConvertToYV12 function has problems depending on version of AVISynth used.
source=AVISource("e:\frameserver.avi").converttoyv12(interlaced=true).assumetff()
IResize(source,720,480).swapfields().assumebff().separatefields()

function IResize(clip Clip, int NewWidth, int NewHeight) {
Clip
SeparateFields()
Shift=(GetParity() ? -0.25 : 0.25) * (Height()/Float(NewHeight/2)-1.0)
E = SelectEven().Spline36resize(NewWidth, NewHeight/2, 0, Shift)
O = SelectOdd( ).Spline36resize(NewWidth, NewHeight/2, 0, -Shift)
Ec = SelectEven().Spline36Resize(NewWidth, NewHeight/2, 0, 2*Shift)
Oc = SelectOdd( ).Spline36Resize(NewWidth, NewHeight/2, 0, -2*shift)
Interleave(E, O)
IsYV12() ? MergeChroma(Interleave(Ec, Oc)) : Last
Weave()
}

[nhope]

The deinterlacing/re-interlacing required for proper re-sizing of interlaced material is not explicitly exposed to the user in Vegas Pro, but it obviously is being done "under the covers."

This brief article (http://www.custcenter.com/app/answers/detail/a_id/334) says that for the "Full resolution rendering quality" setting in Vegas Pro, "Best" uses bicubic scaling with integration, while "Good" uses bilinear without integration.

Does anyone know what "integration" might mean in this context? It may be significant.

[Didée]

@ johnmeyer: small sample? Without knowing where it started from, it's hard to draw conclusions. Moreover, it's doesn't seem feasable that with QTGMC+reinterlacing you got the same detail loss as from the plain fieldbased resize...


Dropping a thought about it ... if you're interested in producing an interlaced result, and want to minimize the downsides of lowpassing, two basic approaches are obvious:

a) be motion adaptive: only moving areas are the main target for lowpassing

b) make an spatio-temporal alias: produce a lowpass, and produce a temporal average. From the lowpass, use only that change that is "confirmed" by the difference caused by the temporal filter.


Clear? It's not so that everything has to be lowpassed. There are certain structures that might need lowpassing. And I could swear to have suggested such an adaptive approch years ago. (But my interest never lied in producing interlaced results, hence nothing was fleshed out.):)

[johnmeyer]

@ johnmeyer: ... it's doesn't seem feasable that with QTGMC+reinterlacing you got the same detail loss as from the plain fieldbased resize...Thank you, thank you, thank you Didée for posting. Because I respect your opinion so much, it forced me to go back and re-do the QTGMC testing from yesterday. This time, I broke it down into smaller steps and found two errors: one with the re-interlacing (simple, but I blew it anyway) and one with the codec choice.

As you expected, the code below does provide better results than Vegas, and actually provides really fantastic results, if your goal is to take interlaced HD and produce great looking interlaced SD. I compared the uncompressed results with what I get with various codecs and found that the quality of the result really shows ALL the flaws in each codec. Obviously this is nothing surprising to readers on this forum, but my point is that this provides about the highest level of detail these SD codecs (like DV codecs) ever have to deal with.

So, be careful what codec you use, or you can throw away some of the detail that you worked to hard to preserve.

#Script to re-size from HD to SD using deinterlace/framedouble - resize - followed by re-interlace
loadplugin("C:\Program Files\AviSynth 2.5\plugins\MVTools\mvtools2.dll")
loadplugin("C:\Program Files\AviSynth 2.5\plugins\RemoveGrainSSE2.dll")
loadplugin("C:\Program Files\AviSynth 2.5\plugins\SSE3Tools.dll")
loadplugin("C:\Program Files\AviSynth 2.5\plugins\mt_masktools-25.dll")
Import("E:\Documents\My Videos\AVISynth Scripts\QTGMC-3.31.avsi") #

AVISource("e:\frameserver.avi").assumetff()

QTGMC( Preset="Very Fast", FPSDivisor=1, EdiThreads=0, ShowSettings=false ) #use for 60p output

LanczosResize(720,480)
separateFields()
#SelectEvery(4,0,3) #BFF
SelectEvery(4,1,2) #TFF
Weave()

[nhope]

As a test, I'm trying to run this file (http://www.jazzythedog.com/testing/videos/AVCHD.zip) (not mine) through a single-threaded variant of johnmeyer's script for subsequent encoding to MPEG-2 with CCE Basic for DVD. Here's my script:

DirectShowSource("D:\My Videos\Amendegw-AVCHD.MTS")
ConvertToYV12(interlaced=true)
AssumeTFF
LanczosResize(720,height)
QTGMC( Preset="faster" )
LanczosResize(width,480)
separateFields()
SelectEvery(4,1,2) #TFF
Weave()

However I'm getting errors. CCE tells me "Frame size 852 x 56 is not supported". If I drag the script into MeGUi it tells me "Source image is too small for this resize method. Width=1 Support=3". I get no such trouble with a 1440x1080 60i HDV file. Anyone know what I'd have to do to get it working in AviSynth? I messed around with tsMuxeR a bit but no joy, and I also tried adding "audio=false". I have Haali Splitter installed.

Mediainfo on the file:

General
ID : 0 (0x0)
Complete name : D:\My Videos\Amendegw-AVCHD.MTS
Format : BDAV
Format/Info : Blu-ray Video
File size : 24.7 MiB
Duration : 13s 500ms
Overall bit rate : 15.4 Mbps
Maximum Overall bit rate : 18.0 Mbps

Video
ID : 4113 (0x1011)
Menu ID : 1 (0x1)
Format : AVC
Format/Info : Advanced Video Codec
Format profile : High@L4.0
Format settings, CABAC : Yes
Format settings, ReFrames : 2 frames
Codec ID : 27
Duration : 13s 430ms
Bit rate mode : Variable
Bit rate : 14.5 Mbps
Maximum bit rate : 16.0 Mbps
Width : 1 920 pixels
Height : 1 080 pixels
Display aspect ratio : 16:9
Frame rate : 29.970 fps
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Interlaced
Scan order : Top Field First
Bits/(Pixel*Frame) : 0.233
Stream size : 23.2 MiB (94%)

Audio
ID : 4352 (0x1100)
Menu ID : 1 (0x1)
Format : AC-3
Format/Info : Audio Coding 3
Mode extension : CM (complete main)
Codec ID : 129
Duration : 13s 536ms
Bit rate mode : Constant
Bit rate : 256 Kbps
Channel(s) : 2 channels
Channel positions : Front: L R
Sampling rate : 48.0 KHz
Bit depth : 16 bits
Compression mode : Lossy
Delay relative to video : -67ms
Stream size : 423 KiB (2%)

[Didée]

"Frame size 852 x 56" suggests that Avisynth is returning an error message clip. Something is not in order, maybe you're missing a plugin, or something.

Open your script in VirtualDub, there you can see the error message.

BTW - ouch, that clip is evil.:scared: :D

[2Bdecided]

And I could swear to have suggested such an adaptive approch years ago. (But my interest never lied in producing interlaced results, hence nothing was fleshed out.):)You did - but letting high frequencies through when nothing moves is still going to allow horrible twitter on CRTs. It'll be OK on progressive displays with adaptive deinterlacers (assuming they work quickly).

Lots of things already work together to create a still=sharp moving=blurred effect; this is already objectionable (IMO) on many modern displays and low bitrate encodes. I think adding another intentional stage with this effect is just too much!

However, it could effectively combat the aliasing that sometimes occurs when something moves. Adapting like this might be the lesser of two evils for some people.

Cheers,
David.

[2Bdecided]

@johnmeyer

This code you suggest (http://forum.doom9.org/showthread.php?p=1492046#post1492046) has no filtering at all, and will flicker like mad on a CRT. It'll bob quite a lot during any motion on most progressive displays too.

I know - I've tried it. It's unbearable (IMO). And, as you say, very difficult to encode (lots of high frequencies and aliasing).

Cheers,
David.

P.S. IMO(!) that is not a good way to switch between TFF and BFF when re-interlacing - you should always use selectevery(4,0,3), but put assumetff() or assumebff() before it. Otherwise the output depends entirely on what field order AVIsynth thinks exists at that point - which will change with different functions, sources, etc, making the whole thing very easily broken - leading you to a big headache!

[nhope]

Thanks Didée. Same error in VirtualDub as in MeGUi. "AviSynth Open Failure: Resize: Source image is too small for this resize method. Width=1, Support=3". And then it refers to the first LanczosResize line. I have AviSynth 2.5.8 ST. I would be interested in knowing if anyone can get that file running sucessfully through a script such as mine.

Edit: By the way, if anyone is interested in the origin of that file and test, it's here (http://www.sonycreativesoftware.com/forums/ShowMessage.asp?MessageID=738210).

[2Bdecided]

Finally, people who are a lot smarter than me -- and smarter than anyone in this forum -- worked together during the late 1980s to invent HD television, and then again in the early 1990s when they re-defined it to make it digital (the original HD spec was analog). These very smart people not only included interlaced video in the spec, but made it the pinnacle of the quality ladder (1080p was added later). They did this partly because TVs in the early 1990s were still based on scanning CRT technology, but also because interlaced TV looks darned good.They did it for the same reason it was introduced in the 1930s - to increase the resolution without increasing the raw or encoded bandwidth to match.

Some of the original analogue HD formats weren't interlaced in the traditional sense - they had something far more evil (sometimes called dot interlacing).

Interlacing doesn't make any sense in the context of progressive displays or efficient codecs - never mind both. But we are where we are - and whether codecs are really so efficient as to make interlacing redundant is still up for debate (according to some).

Cheers,
David.

[Didée]

@ nhope: Can't tell what's up with that file, but something is not quite right with it. Slightly older versions of ffmpegsource crash right away with it. The most recent version can decode it, but the field/frame order is somehow not correct: everything is choppy, with some forward/backward/jump issues, ...
However, I have no problems with DGDecodeNV, this one decodes the file without problem. (And on my system, it is also correctly decoded by directshowsource ... but DSS is whacky, as you have experienced. Each system is unique in regard to DirectShowSource....)

In any case, the file is flagged with wrong field order. All tools report it to be TFF, but I need ComplementParity() to switch it over to BFF. Only then the field/frame sequence after bobbing is correct.

Funky file to make a challenge with. Seems the challenge is not so much how to get a good result ... the real challenge is to get the darn input file opened correctly in the first place! :rolleyes: :D

[nhope]

Thanks for looking into it. It's not that important. If I do an AviSynth-based test with it I'll probably put it in Vegas and frameserve out and accept the couple of conversion to RGB and back along the way. By the way, the owner tells me it's straight from his Canon HG21 recording in 1920x1080 60i, FXP (17Mbps) mode. Apologies for rather derailing the thread.

[johnmeyer]

They did it for the same reason it was introduced in the 1930s - to increase the resolution without increasing the raw or encoded bandwidth to match. ...Yes, that is exactly the reason.

Interlacing doesn't make any sense in the context of progressive displays or efficient codecs - never mind both. But we are where we are - and whether codecs are really so efficient as to make interlacing redundant is still up for debate (according to some).Once again, I agree completely.

However, the second sentence is the reason I responded. If you think back to the early 1990s, we were still using 486 computers, and "DSPs" were the only way to handle even the most rudimentary calculations related to video. Now, almost exactly twenty years later, with multiple cores and processor architecture that includes native instructions for handling media, we can finally deal with 1080p, although real-time performance for displaying this level of video has only happened in the past few years, and editing and encoding is usually far from real time (obviously depends on ... fill in the blank).

I will be the first to cheer the final demise of interlacing, but since we have so much legacy material that will be with us forever (I still play my 78 rpm records), I wish we could buy an LCD display that could handle interlaced material natively. For the life of me I do not understand why a line-addressable LCD display could not be built. This would provide a partial solution for displaying interlaced material. Some have said that this couldn't work because LCD persistence is different than CRT phosphors, but that I think that could be simulated using the same technology used in my "local dimming" Samsung LED LCD.

But, as I said, I agree with everything you said above.

[amendegw]

@ nhope: Can't tell what's up with that file, but something is not quite right with it. Slightly older versions of ffmpegsource crash right away with it. The most recent version can decode it, but the field/frame order is somehow not correct: everything is choppy, with some forward/backward/jump issues, ...
However, I have no problems with DGDecodeNV, this one decodes the file without problem. (And on my system, it is also correctly decoded by directshowsource ... but DSS is whacky, as you have experienced. Each system is unique in regard to DirectShowSource....)

In any case, the file is flagged with wrong field order. All tools report it to be TFF, but I need ComplementParity() to switch it over to BFF. Only then the field/frame sequence after bobbing is correct.

Funky file to make a challenge with. Seems the challenge is not so much how to get a good result ... the real challenge is to get the darn input file opened correctly in the first place! :rolleyes: :D

I guess I should register present here... that's my file & my camcorder. Canon HG21 1920x1080 60i FXP mode. The clip is straight out of the camera. The only change I made was to rename it from 00003.mts to AVCHD.mts.

Is there anything else I can do to resolve the problem? Maybe someone else with a HG20/21 (or similar a similar vintage Canon) can shoot a clip in the FXP mode and see if it displays the same funkiness.

...Jerry

[johnmeyer]

@ nhope: Can't tell what's up with that file, but something is not quite right with it ... the file is flagged with wrong field order. All tools report it to be TFF, but I need ComplementParity() to switch it over to BFF. Only then the field/frame sequence after bobbing is correct.Interesting that you too had problems with field order. I spent some time this morning on this test file, and posted the results in the Sony forum. I didn't bother to mention it there, but I've seldom had so much trouble with field issues. Very strange.

As for results, I tried the QTGMC deinterlace/frame double --> resize --> reinterlace approach that I posted earlier and that Nick is trying to duplicate. With this pathological test case (it really is a devilishly brutal test clip), the QTGMC approach totally failed: the resulting 720x480 file was nothing but obnoxious moiré patterns. By contrast, the resizing script (http://forum.doom9.org/showthread.php?p=1185790#post1185790) that was the result of a long discussion about resizing YV12 content a few years ago did a remarkable job of resizing this content. I created an interlaced 720x480 widescreen DVD MPEG-2 file from the result, created a DVD, and then displayed this DVD on my 30" Sony WEGA CRT, and it displayed with no appreciable artifacts whatsoever.

[Didée]

I guess I should register present here...
Hello! So it took only one and a half year until you made your 1st post? Not bad!;)

Frankly, I've only little clue about the "particularities" of Camcorder files/formats. It's just when you have a standard like AVC, it's a bit baffling when a usually well-working tool suddenly falls apart into pieces. Makes you wonder. :confused:


... the QTGMC approach totally failed: the resulting 720x480 file was nothing but obnoxious moiré patterns.
A good tool doesn't hinder you to use it badly. :)

1080i60 > TGMC > 480p60: http://www.mediafire.com/?djjcd5c88e2alqg

Lowpass-reinterlaced to 480i60: http://www.mediafire.com/?7d5fun3wiyjmep0

Ooops, sorry, I forgot to set the AR flag. Please set your player manually to 16:9. :D


Script ...
SetMTmode(5,4)
DGSource("X:\AVCHD.dgi")
changefps(last,last,true)
complementparity()

setmtmode(2)
tempgaussmc_beta2a(1,1,1,nsize=5,sharp=1,sharpness=0.0,svthin=0.0,sbb=0)

# -- >here< is a good point to save an intermediate Huffy or Laggy or FLV-y -- ;-)

bicubicresize(1440,960)
bicubicresize(720,960,-.8,.6)

p1 = bicubicresize(720,480,-.8,.6)

p2 = p1.QTGMC(1,1,2,inputtype=1)

p2.blur(0,1).sharpen(0,.51).blur(0,1).sharpen(0,.85)

i1 = assumetff().separatefields().selectevery(4,0,3).weave()

# p1 # straight to 60p
# p2 # plus more calmed
i1 # re-interlaced p2

return(last)

[sadie]

Speaking of strange source clips, I've come across a Japanese ntsc music video that is hardly catholic. https://rapidshare.com/files/457285734/KibSok_cut.avi for a 1 min segment.

It looks to be an ntsc tv capture and an attempted progressive divx encode but where someone forgot to turn off the interlace switch. The 4700 kbps is a giveaway it was handled by someone even more amateurish than myself. The trouble is that when QTGMC gets its mitts around it various sequences will show 'morphing' or stretching of facial features and such. As if the temporal smoothing is really having to work overtime (?) Very disagreeable.

I'm trying to overlay this into a timeline with standard Pal DV material so I've upscaled it and converted the FPS along the way, but even without, the deinterlacing is the stage that is either creating or emphasizing this phenomenon. If I import the video untouched into the NLE (Pinnacle 15), the resulting export (DV or Mpeg2) surprisingly reduces the issue to a minimum but the result is rather less detailed (as well as interlaced) than what QTGMC would give. I've tried throwing the job at Yadif, but no banana there either.

In short, is there a genial fix or is that cute little kawaii singer fit for the trash bin? Thanks.

[johnmeyer]

A good tool doesn't hinder you to use it badly. :)Das ist richtig.

Earlier in this thread, I showed the script I used:

QTGMC Deinterlace/FrameDouble-->Resize-->Reinterlace (http://forum.doom9.org/showthread.php?p=1492046#post1492046)

I tried various other QTGMC presets, but still did not achieve satisfactory results. This is an excellent tool, but even by AVISynth standards, has an unusually large number of optional parameters. Would you perhaps be willing to share what settings you used for this particular clip? Over at the Sony Vegas forum, about a year ago, several dozen people tried to resize this clip, and your result is certainly one of the best results of those I have seen.

[amendegw]

Hello! So it took only one and a half year until you made your 1st post? Not bad!;)


Das ist, weil mein Nachname ist "am Ende". :) Actually, I've been a "lurker".

...Jerry (who's almost always "at the end") :)

Oh, and one more thing, if this is helpful. I just captured another clip. This time using a Panny TM700. It's here if anyone is interested: http://www.jazzythedog.com/testing/videos/TM700-60i.zip

And, I compared the MediaInfo stats on the Canon & Panasonic:

http://dl.dropbox.com/u/20447760/MediaInfoCompares.png

[sadie]

A quick follow-up to myself. I've tried to run the clip through Assume TFF/BFF().SeparateFields() with indifferent results (normal since this is after all a 'progressive clip'.) But both do show the distinctive signs of a field order problem (forward & back then forward again). The question remains: how to clean this up if possible?

[Didée]

It looks to be an ntsc tv capture and an attempted progressive divx encode but where someone forgot to turn off the interlace switch. The 4700 kbps is a giveaway it was handled by someone even more amateurish than myself. The trouble is that when QTGMC gets its mitts around it various sequences will show 'morphing' or stretching of facial features and such. As if the temporal smoothing is really having to work overtime (?) Very disagreeable.
The "trouble" is not related to QTGMC, you'll see exactly the same issue when using a simple Bob(). The clip simply is broken. Someone did change the vertical framesize with a "progressive" resizing method, thereby destroying the interlaced field structure. As a result, each top field contains alternating bands of top- and bottom fields, and same for the bottom field.

Here's a simple simulation of what has happened:
BlankClip(width=576,height=480)

interleave(BlankClip(width=576,height=480,color=$006000),
\ BlankClip(width=576,height=480,color=$600000))

AssumeTFF().ShowFrameNumber()

i1 = SeparateFields().SelectEvery(4,0,3).Weave()
i2 = i1.bicubicresize(576,464,0,0.5) .addborders(0,8,0,8,color=$200000)

stackhorizontal(i1.addborders(0,0,8,0,color=$002000),i2)

bob(0,0.5)
subtitle("bobbed interlaced source",x=160,y=32)
subtitle("""interlaced source resized 'progressive', then bobbed""",x=576+88,y=32)
return(last)

[nhope]

I'm an idiot. I had H264 decoding disabled in ffshow. I've enabled it now and DirectShowSource will open both .MTS fine. Incidentally the TM700 file had been giving exactly the same error as the HG21 file before I enabled the H264 decoding.

[nhope]

deleted

[nhope]

Didée, 3 questions about the script you used for the doll, which gave excellent results...

1. Have you tweaked that script specifically to cater for the horrific, moiré-inducing nature of the background, or is it a script that you might use generically for HD to 480i conversion?

2. Since QTGMC is a development of tempgaussmc, could tempgaussmc_beta2a(1,1,1,nsize=5,sharp=1,sharpness=0.0,svthin=0.0,sbb=0)
be done in QTGMC, to minimise the number of scripts involved?

3. Where would the "missing" AR 16:9 aspect ratio flag go?

[Didée]

1. Yes / maybe
2. Sure, just need to use parameter names
3. x264: "--sar width:height"

Generally, the usage of QTGMC (or even the 'double' usage) in this case here is ever-so-slightly over the edge.:D - With such a big downsize as in 1080 -> 480, a quite similar result probably could be achived with a much simpler bob (even yadif), or even with a direct fieldbased downsize.
I just had to go the TGMC route, since johnmeyer concluded (http://www.sonycreativesoftware.com/forums/ShowMessage.asp?ForumID=4&MessageID=758762)
I tried your torture test using the QTGMC-based re-size script and the moiré patterns were horrible. I tried some of the slow presets, but that didn't help much. So, as much as I liked the results on my ballet footage, for this clip, I think it fails, and it fails quite spectacularly.
but as can be seen from my two samples, QTGMC of course can deal with this torture clip. One just needs to be a bit careful with the sharpness settings in this case. And of course, with so much high frequencies present, you should NOT do the final re-interlacing without lowpassing. "With best regards, Nyquist." ;)

[-Vit-]

Didée's script in QTGMC 3.31 form. Untested, just a settings conversion. I don't know what tempgaussmc_beta2a is, I only have beta2/mod (maybe a version that supports NNEDI3?).

# For more speed add Blocksize=32 and/or drop NNeurons and/or switch to "Faster"
QTGMC( "Fast", TR0=1,TR1=1,TR2=1, NNeurons=1, SubPelInterp=1, Sharpness=0 )

-- >here< is a good point to save an intermediate Huffy or Laggy or FLV-y -- ;-)

bicubicresize(1440,960)
bicubicresize(720,960,-.8,.6)

p1 = bicubicresize(720,480,-.8,.6)

p2 = p1.QTGMC( "Slower", TR0=1,TR1=1,TR2=2, InputType=1 )

p2.blur(0,1).sharpen(0,.51).blur(0,1).sharpen(0,.85)

i1 = assumetff().separatefields().selectevery(4,0,3).weave()

# p1 # straight to 60p
# p2 # plus more calmed
i1 # re-interlaced p2

return(last)


If you're serving the script through MeGUI then you can set the AR by adding this to your script:

global MeGUI_darx = 16
global MeGUI_dary = 9

[sadie]

@didee:

Nice analysis. I understood but half of it. The upshot I take it is that 'clip is broken' = 'clip cannot be reglued'. Again, I wonder if there is a strategy within qtgmc that might somehow minimize that goofy stretching yet still clean out the combing (SelectEven?). And of course I can always just keep it interlaced and live with the blur. It's only to be used at a low resolution anyway. Thanks for the help.

[johnmeyer]

A quick follow-up to myself. I've tried to run the clip through Assume TFF/BFF().SeparateFields() with indifferent results (normal since this is after all a 'progressive clip'.) But both do show the distinctive signs of a field order problem (forward & back then forward again). The question remains: how to clean this up if possible?I downloaded your clip. I have seen this sort of thing before and it is a perfect demonstration of why I always recommend that people NOT deinterlace video and just leave things alone. The problem is not that deinterlacing isn't necessary in some limited circumstances, but rather that so many people screw it up and make things far, far worse.

I have been given many clips like this to clean up. First of all, the idiot(s) that screwed around with it before you got it managed to encode black bars into the video. Obviously you want to crop those out.

Second, the way I have dealt with identical videos in the past is to separatefields() to end up with half-height double-fps video.

I then carefully look at the odd and even fields. In the case of your video, it is quite clear that the odd fields were created from some sort of horrible estimation algorithm. You can see people's faces warp and distort. Yuchh!

The approach then is to decimate (delete) the odd fields, crop out the black bars, resize the remaining fields to the original frame size (minus the crop), and then use MVTools2 to estimate the the missing frames/fields. Obviously you are losing the "resolution" from the decimated fields, but they contain nothing but garbage, so it really doesn't matter.

The code below is extracted from a script I used when I restored all fifty-seven hour-long episodes of the Johnny Cash show for a collector several years ago. He got them from a collector in the UK who got them from Australia who got them from source here in the US. You get the idea. Too many hands. In my case, I was trying to get them to look like the original video broadcasts, and so I converted back to 60i. In your case, you might be happy just to convert them to 30p which is what I have done.

Here's a low bitrate encode of the resulting video:

KibSok_cut (fixed).avi (http://dl.dropbox.com/u/1561578/KibSok_cut%20%28fixed%29.avi)

If you encode at higher bitrate, it should look pretty clean.

loadplugin("C:\Program Files\AviSynth 2.5\plugins\MVTools\mvtools2.dll")
top_bottom_crop=40

AVISource("e:\frameserver.avi").ConvertToYV12.killaudio()

crop(0,top_bottom_crop,-0,-top_bottom_crop)

#Throw away bad fields. Video is now half size vertically.
separatefields().selecteven()

source=LanczosResize(640,464-top_bottom_crop)

super2=MSuper(source,pel=2)
backward_vec = MAnalyse(super2,blksize=16, overlap=4, isb = true, search=3 )
forward_vec = MAnalyse(super2,blksize=16, overlap=4, isb = false, search=3 )

MFlowFps(source,super2,backward_vec, forward_vec, num=30, den=1, ml=100)

[Didée]

Sorry John, but the technical part of your analysis of the problem is completely wrong.

At 'mastering' stage, nobody did try to deinterlace this video. The odd fields are not made by some horrible estimation algorithm, and the even fields share the very same issue like the odd fields.

What happened was that interlaced 720x480 was resized to 640x464 with a resizer in progressive mode. Exactly that, nothing more, nothing less.

Edit: proof:
http://img59.imageshack.us/img59/3882/interlacinghell.th.jpg (http://img59.imageshack.us/i/interlacinghell.jpg/)


Your solution is good - since it's close to impossible to (sufficiently) revert the interlace-damage, keeping just one field & discarding the other is the next best thing you can only do. But the problem analysis definetly needed to be corrected. ;)



Edit: <*deleted*> (issue purged)

[johnmeyer]

... if you post things about me in public that are not true, it should be corrected in public. I am very sorry you are offended, and I apologize.

You ask me to correct untrue things, and do so in public, so in good faith, let me attempt to do that.

First, since you did not quote the entire post from the Sony forum let me clarify something. My post in the Sony forum was VERY long, so I agree that you should not have quoted it all, but since others may not read my post in that forum, there might be a mistaken impression that I somehow went on a rant against you. So, let me state for those reading here who don't visit that forum: I did not identify you in any way in that post, and no one reading that post would know that it was you.

Second, I did not see the script you posted. I just looked at your post and see that it was edited approximately an hour and a half after the original post (the original post was at 14:55 and was last edited at 16:26). I am certain that I read your post just after it initially posted, and in fact you can see that my next post was at 15:36, which is about halfway between your initial post and your final revision.

So, I don't know if you added or edited the script after your initial post. However, I can say for certain that the script I see there now is quite complete and exactly what I was looking for. Therefore, I can understand that you would be extremely upset with me for saying that you hadn't been responsive when in fact you have now provided exactly what I was asking for.

I should mention that in one of my posts earlier in this thread I said:Thank you, thank you, thank you Didée for posting. Because I respect your opinion so much, it forced me to go back and re-do the QTGMC testing from yesterday. I think you can agree that this is not something a person would post who is trying to pick a fight with you.

Finally, I just now went to the Sony forum and I removed the sentence you quoted above from my post.

So, I have apologized, have described how the situation arose, and have removed the offending quote. I hope that settles this.

I had been working on a response to your tutorial on how improper resizing may have contributed to the bad things happening on the "KibSok_cut.avi" file, but given the circumstances, I think I'll lay low for awhile.

[nhope]

Didée & -Vit-, thanks for the info.

Can anyone tell me how to correctly change the following script to output BFF instead of TFF? (I know this isn't directly QTGMC-related but I want to compare the result of this script with some others including some QTGMC versions and some BFF versions done in other ways)

source=DirectShowSource("D:\My Videos\720x480-for-NTSC-DVD\challenge-Amendegw-AVCHD.MTS").assumetff()
IResize(source,720,480)
function IResize(clip Clip, int NewWidth, int NewHeight) {
Clip
SeparateFields()
Shift=(GetParity() ? -0.25 : 0.25) * (Height()/Float(NewHeight/2)-1.0)
E = SelectEven().Spline36resize(NewWidth, NewHeight/2, 0, Shift)
O = SelectOdd( ).Spline36resize(NewWidth, NewHeight/2, 0, -Shift)
Ec = SelectEven().Spline36Resize(NewWidth, NewHeight/2, 0, 2*Shift)
Oc = SelectOdd( ).Spline36Resize(NewWidth, NewHeight/2, 0, -2*shift)
Interleave(E, O)
IsYV12() ? MergeChroma(Interleave(Ec, Oc)) : Last
Weave()
}

[sadie]

Thanks, John, for your effort of digging into the matter. I'm going to try and play with it next week when I'm off in the country. Still, I'm not entirely convinced. Even if your test sample is very low bit rate, I'm still not sure it only sort of resembles what I get using SelectEven settings with qtgmc. And i respectfully disagree that the odd fields are remarkably different from the even ones. The problem seems to be that they are just consecutively disordered. I should have uploaded a sample of my test using the simple avs:

DirectShowSource ("E:\KibSok.avi")
QTGMC (preset="fast")
Lanczosresize (720,576)
convertfps(50)

So here, it's true, the warping is more finely accentuated in the major moving parts of my sample than with yours, but so is the detail in the unaffected more 'static' parts. And the latter comprise roughly 2/3 of the video. So I suppose it all boils down to a global psycho-visual appreciation of the subject matter. In fact if I just closed my eyes once every three seconds I'd have no problems at all :rolleyes: But, I'll give your script an honest whirl.

More importantly, I want to know how come you guys don't seem knocked off your feet by the gal in this video.

[Gavino]

Can anyone tell me how to correctly change the following script to output BFF instead of TFF?
source=DirectShowSource("D:\My Videos\720x480-for-NTSC-DVD\challenge-Amendegw-AVCHD.MTS").assumetff()
IResize(source,720,480)

The simple answer would be to change 'assumetff' to 'assumebff'.
Since it works by resizing the individual fields directly, the IResize function preserves the field order of the input clip, so if you feed it BFF in, you will get BFF out.

However, that won't help if the input really is TFF and you want to change it to BFF. In that case you would need to reverse the field dominance before calling IResize. Various methods exist for doing this (eg stickboy (http://avisynth.org/stickboy/)'s JDL_ReverseFieldDominance), but a quick and dirty way (which loses the very first field) is
SeparateFields.Trim(1,0).Weave()

[nhope]

Thanks Gavino.

Going back to Didée's script to deal with that horror clip, I can see that the idea is to get a reasonable bobbed, 60p, full res clip and then do some fancy stuff to it, including bicubically downsizing it sequentially in stages and running QTGMC over it. Here's the resizing/filtering stage again:

bicubicresize(1440,960)
bicubicresize(720,960,-.8,.6)

p1 = bicubicresize(720,480,-.8,.6)

p2 = p1.QTGMC( "Slower", TR0=1,TR1=1,TR2=2, InputType=1 )

p2.blur(0,1).sharpen(0,.51).blur(0,1).sharpen(0,.85)

i1 = assumetff().separatefields().selectevery(4,0,3).weave()

# p1 # straight to 60p
# p2 # plus more calmed
i1 # re-interlaced p2

return(last)

I'm embarking on a round of tests for downscaling AVCHD/HDV to 720x480i for widescreen DVD. I have a minute of footage that includes that moiré-inducing horror clip but also more normal footage such as the HDV that can be seen on my YouTube channel (http://www.youtube.com/user/bubblevision). My list of tests at the moment:

1. Resizing in Vegas Pro
2. IResize "pure" script
3. QTGMC bob / Lanczos3 / no filtering > reinterlace
4. Simple bob > Didéeesque resize/filtering > reinterlace
5. Better bob (Yadif or TDeint) > Didéeesque resize/filtering > reinterlace
6. QTGMC bob > Didéeesque resize/filtering > reinterlace (=slow/ott)

Each will be encoded to MPEG-2 in CCE Basic.

It seems to me that there's a lot of filtering in Didée's script that may be excessive for most footage. Would anyone care to propose a "toned down" version that is more suitable for generic footage? Perhaps something between test 3 and tests 4/5/6? Resize in fewer stages perhaps? Use a faster preset for QTGMC perhaps? Or should I just go with it as is?

[johnmeyer]

Can anyone tell me how to correctly change the following script to output BFF instead of TFF? Just use ComplementParity() before the Weave or SwapFields() after. I think you get the same result either way. I tested, using your script, and it works.

[Gavino]

Just use ComplementParity() before the Weave or SwapFields() after. I think you get the same result either way. I tested, using your script, and it works.
That depends on what you're trying to achieve - cf my post above.

Yes, it will change TFF to BFF, but the resulting fields will have incorrect spatial alignment (if they were initially correct).

[johnmeyer]

Yes, it will change TFF to BFF, but the resulting fields will have incorrect spatial alignment (if they were initially correct).Are you saying that I'll get two top fields in a row and then two bottom fields?

I added an assumebff().separatefields() temporarily to the end of the script, and the temporal order was definitely correct (the initial video was tff), and I thought I saw the usual up/down movement between fields, but I guess I wasn't looking closely. I'll go back and look again.

The only issue I was aware of when changing parity is that you can lose the first and/or last field in the video. But, I'm still learning ...

[sadie]

John, before running off tonight I did finally manage to do a couple of encodes with your script and it pretty much confirms that it's at best a toss-up compared with my qtgmc approach. I incorporated your crop formula to good effect. There is a slight problem though in that your formula gives me an aspect ratio that is noticeably squished horizontally. I adjusted your script for PAL usage (num=25). Perhaps something went askew there. Anyway, thanks for the old college try.

On a quite separate note regarding your 'apology' to Didee, inasmuch as the offended party has re-edited his post for the second time and removed the accusation, maybe you ought to remove your apology as well. As it stands, it actually makes you out to look like the good guy:). To be more pointed, your altercation was another example of what unfortunately can happen when people abuse the prerogative to edit their own posts. I can't tell you how many times I've been thrown into a loop when people pulled such stunts with me. IMO an edit should be limited to the odd misspelling or rephrasal or such. But otherwise posts ought to remain in the spirit of what a forum really is supposed to be about: an honest running conversation between two or more parties. And if this means living with errors, omissions and delayed follow-ups so be it. I mean how often to do you go back and reread what what was posted an hour or day ago just to be sure there haven't been any 'updates'? And although it may have had the best intentions, Didee's edit should instead have been a separate follow-up. But as it stood, it made you appear not only to be presumptuous (which you were) but also a liar. I hope he at least sent you a humble little PM owning up to this. In fact his now excised 'accusation' (which by the way broke forum rule No. 9 big time) shouldn't even have appeared in this thread as it concerned a matter in a third-party forum. It should have been handled entirely by PM (or on the Sony site).

With that out of the way, I wish one and all a happy Easter week. Cheers.

[Gavino]

I added an assumebff().separatefields() temporarily to the end of the script, and the temporal order was definitely correct (the initial video was tff), and I thought I saw the usual up/down movement between fields, but I guess I wasn't looking closely.
The temporal order is correct, but top field pixels have become bottom field ones and vice versa, so will be displayed in the wrong place in the frame. Look at a still sequence and you will see immediately.

[johnmeyer]

The temporal order is correct, but top field pixels have become bottom field ones and vice versa, so will be displayed in the wrong place in the frame.I'll have to look at that when I get back to my main computer. I haven't actually used the script fragment I wrote above: I simply read the AVISynth doc, copied the code, and did a quick test which revealed that the temporal order was correct. I didn't see the spatial reversal. If that is happening, then Complementparity is of no use in this situation. I wonder when I would use it ...

In looking at scripts I wrote in the past, when I ended up having to reverse fields one time a few years ago, I wrote the code below, so I must have tried the simpler approach and realized it didn't work. I thought I was solving a tougher problem back then, but I guess it was this same thing.

source=AssumeBFF()

even=source.trim(1,0).separatefields().selecteven
odd=source.separatefields().selectodd
interleave(odd,even)
fixed=weave()

#In the line below, use the OPPOSITE of what you chose above
#because the fields have now been reversed.

AssumeTFF(fixed)

[Gavino]

In looking at scripts I wrote in the past, when I ended up having to reverse fields one time a few years ago, I wrote the code below, so I must have tried the simpler approach and realized it didn't work. I thought I was solving a tougher problem back then, but I guess it was this same thing.
It's even simpler than you think.
That entire code can be replaced by
SeparateFields().Trim(1,0).Weave()
which is what I suggested in post #614 above.

This works to reverse field dominance no matter whether you start with TFF or BFF.

[johnmeyer]

On a quite separate note regarding your 'apology' to Didee, inasmuch as the offended party has re-edited his post for the second time and removed the accusation, maybe you ought to remove your apology as well. I am a little shy about further editing my post unless Dideé or the forum admin requests it. As you state above, edits should usually be limited to spelling mistakes and similar things. However, if the forum moderator contacts me by PM, or if Dideé does the same, and if either asks me to edit the post, I will certainly do so. I have never intended to do anything except help people and do so in a way that is neither threatening, belittling, or difficult. Hopefully my collective posts in various forums back that up.

You mention forum rules, and one of the rules is to not have a discussion about the rules, so I think I will leave it at that. This matter is finished, and I suggest that none of us post further about it.

[johnmeyer]

SeparateFields().Trim(1,0).Weave()
Got it!

[boondoggle]

I tried out the example settings from the manual "QTGMC( Preset="Slower", NoiseProcess=1, NoiseRestore=0.0, Denoiser="dfttest", DenoiseMC=true, NoiseTR=2, Sigma=4.0 )" (the manual still says "NoiseBypass" instead of the new name "NoiseProcess" btw) but an error with "no function named "dfttest"" occurs. It doesn't seem to be gone in the script though? :confused:

[-Vit-]

Get the plugins package from the OP, or download dfttest directly from the link at the bottom of the OP. Also make sure you get the fftw3 library (again see the OP).

I suggest you use EZDenoise and NoisePreset as described in the manual rather than that more manual setup. Together they will create similar settings to yours and suffice for many straightforward denoising requirements. Someone else pointed out the error in the manual too - I've already updated it for the next version.

[Asterra]

I got a pretty basic question that probably has a pretty easy answer.

Following the guide in the OP, and using Avisynth 2.6 MT from SEt, I get this:

script error: there is no function named "SetMTMode"

This function is supposedly part of mt.dll, which does not come with 2.6. Lacking options, I snagged mt.dll from an earlier version but, unsurprisingly, that fixes nothing.

Not sure what step I missed (or is missing).

[SubJunk]

This isn't the right thread for that question, but the answer is you need to replace the avisynth.dll file with another one. You can fix the problem by following Step 1 of this guide (http://www.spirton.com/convert-videos-to-60fps/#Instructions), and hopefully a moderator will put our 2 posts into a new or relevant thread

[pachti]

hello,
sorry for my poor english, i try to do best i can do to be the most readable...

i try tonight to use QTGMC but i can't load my video in megui or vdub
megui close trying open the video in preview
vdub can load 2 frame then tell: "CAVIStreamSynth: system exception - -access violation at 0x10012380, reading from 0x580"

that's my best try to load... before i got many error about removegrain in line 394...

my script is:
FFVideoSource("D:\video.mkv")
QTGMC( Preset="Faster" )
SelectEven()
crop( 24, 148, -18, -138)
LanczosResize(1296,544)


thx

edit
well i think i got a bug with ffvideosource...
i try with DirectShowSource then it works!!
i didn't search enough before posting.... (since this mornig...:( )

edit2:
~4fps encoding in x264 bd5 profile... not so fast...
did i got another bug?
i have a i3... under seven64

[-Vit-]

Use an MT version of avisynth, then you can do something like this:

SetMemoryMax(400) # ...or 300 or 700 or whatever works without crashing
SetMTMode(5,6) # ...or (5,4) or (5,8) or whatever goes fast without crashing

XXXSource("D:\video.mkv") # Whatever source filter works for you
SetMTMode(2)

crop( 24, 148, -18, -138) # May need to tweak crop values to keep QTGMC happy
LanczosResize(1296,Height()) # Horizontal resize before QTGMC is faster (***as long as you are starting > 1296. I assume it is 1920 content?)

QTGMC( Preset="Faster" ) # Or "Very Fast" or "Super Fast" for HD content...

LanczosResize(1296,544)

SelectEven()
Distributor() # Use this line for MeGUI, remove this line for VDub

[pachti]

thx for your help
i tried avisynth mt but i never could load my video. neither directshowsource or ffvideosource
each time, megui close or i got " any fonction setMTmode exist" (message in french, i translate it ;) )

i will try this filtre later because I do not stay on a failure and i don't have time for the moment....
i'll be back ;)

PS my source is a french hdtv rip 1440x1080i
i miss my 1st resize it was 1080x544 no 1296...
the crop need 2more hight for qtgmc

[-Vit-]

The error "Function SetMTMode doesn't exist" suggests that you don't have MT avisynth. Go to the first post (http://forum.doom9.org/showthread.php?t=156028) - you will find some Avisynth MT links under "Multi-Threaded Usage".

About the crops and resizes - do whatever your source needs - just do not resize vertically before QTGMC, and keep the height a multiple of 4 (for YV12).

[matfra]

Hello,
Would it be possible to integrate SeeSaw script into QTGMC? I dont like the sharpening function into QTGMC. It give a greenish look to the video. I tried adding SeeSaw() after the QTGMC but it look too strong.

[-Vit-]

The "Sharpness" setting in (Q)TGMC is not a good substitute for a stand-alone sharpener. It's relatively simple and gives a little more attention to the vertical sharpness (as that's where the interpolation occurs in deinterlacing).

There might be some logic to adding more sharpening flexibility to QTGMC, given that it has to resharpen anyway. But it wouldn't have any more features/quality than a specialist sharpening script. In any case I'm not focused on QTGMC development at the moment and I will have other priorities when I return to it.

Try setting a lower sharpness value in QTGMC, say 0.4, and allow your post-script to do more of the sharpening. Or if you have the time & processor power, then setting SourceMatch=2 will allow you to almost switch off QTGMC resharpening (say Sharpness=0.1). The result will be very close to the source, which might allow you to do a better job with a standalone sharpener afterwards. Not appropriate for HD, too slow.

[ttatcs]

Thanks for this beautiful script ! Such amazing magic when I showed the result to others .

One thing I can not figure out :
I have read and followed the instructions the manual says but still can not use yuy2 video as source . Is there special match to achive this ? I mean avisynth version , mt avisynth , masktool ?
What is the must to make yuy2 video as the source ?

again , Thanks for your work!

[matfra]

In any case I'm not focused on QTGMC development at the moment and I will have other priorities when I return to it.

I hope you gonna return to it soon master :)

[-Vit-]

I have read and followed the instructions the manual says but still can not use yuy2 video as source . Is there special match to achive this ? I mean avisynth version , mt avisynth , masktool ?
What is the must to make yuy2 video as the source ?
What is the error that occurs when you try to use YUY2 as a source?

yup has reported that YUY2 doesn't work with 2.6 versions of avisynth - I haven't confirmed that but it sounds plausible. For similar reasons you should use mt_masktools-25.dll not mt_masktools-26.dll

[yup]

ttatcs and -Vit-!
Now for me YUY2 colorspace work. SEt 2.6 build, mt_masktool 25 not 26. Kasandro plugins only with SSE2 support.
Example
Import("QTGMC-3.31.avsi")
SetMTMode(2,2)
DirectShowSource("cap1.avi")# YUY2 source
SetMTMode(2,2)
AssumeTFF()
ConvertToYUY2(interlaced=true)
Crop(16,12,-16,-12)
ColorYUV(autogain=true, autowhite=true)
QTGMC(Preset="Placebo",ChromaNoise=true)
Real working script for 50 minutes video.
yup.

[nhope]

...Kasandro plugins only with SSE2 support.
Which plugins do you mean exactly?

[ttatcs]

What is the error that occurs when you try to use YUY2 as a source?

yup has reported that YUY2 doesn't work with 2.6 versions of avisynth - I haven't confirmed that but it sounds plausible. For similar reasons you should use mt_masktools-25.dll not mt_masktools-26.dll

my script:
Avisynth 2.5.8 tsp MT version

---------------------------------
SetMemoryMax(600)
SetMTMode(5, 6)
AviSource("c:\temp\jj.avi" ,audio=false ) # lagarith lossless yuy2 video
#Converttoyv12()
AssumeTFF()
SetMTMode(2)
QTGMC( Preset="Very Slow", EdiThreads=3)
------------------------------------------

load to virtualdub 1.9.11
error :
"Avisynth open failure: mt-makediff : unsupported colorspace. masktool only support planar yuv color space(yv12, yv16, yv24) "

when I add "converttoyv12()" then everything ok .

[-Vit-]

Is there a line number with that error message?

Edit: I've tested a Lagarith YUY2 interlaced AVI and it was fine. So...
- That is mt_masktools25.dll you are using?
- You don't have both masktools25 and masktools26 in your plugins do you?
- Could you upload a small piece of that source...

[Didée]

Is it definite that Lagarith is decoding to YUY2? Perhaps it decodes to RGB ...

---------------------------------
SetMemoryMax(600)
SetMTMode(5, 6)
AviSource("c:\temp\jj.avi" ,audio=false ) # lagarith lossless yuy2 video
return( info() )
#Converttoyv12()
AssumeTFF()
SetMTMode(2)
QTGMC( Preset="Very Slow", EdiThreads=3)
------------------------------------------

[yup]

@ttatcs
Please list for avisynth/pugin directory. May be problem with masktools 26?
yup.

[Gavino]

Now for me YUY2 colorspace work. SEt 2.6 build, mt_masktool 25 not 26. Kasandro plugins only with SSE2 support.
Are you saying that mt_masktools-25.dll works with SEt's 2.6 build and mt_masktools-26.dll does not? That seems contradictory to previous understanding.

[yup]

@Gavino.
Before using YUY2 colorspace inside QTGMC no problem I can use both version 26 and 25, but after transfer to YUY2 properly work only 25 version.
yup.

[yup]

@ttatcs
I am using following version
31.12.2010 18:13 966144 mt_masktools-25.dll
24.10.2007 10:41 33280 RemoveGrainSSE2.dll
24.10.2007 10:41 24576 RepairSSE2.dll
11.04.2005 12:16 30720 SSE2Tools.dll
28.02.2008 08:03 11776 VerticalCleanerSSE2.dll

@Gavino when I only replacing to 26 version masktools I get error message
---------------------------
VirtualDub Error
---------------------------
Avisynth open failure:
mt_makediff : unsupported colorspace. masktools only support planar YUV colorspaces (YV12, YV16, YV24)
(QTGMC-3.31.avsi, line 775)
(QTGMC-3.31.avsi, line 387)
(E:\Video\Mother\new\2002\deint2002.avs, line 9)
---------------------------
ОК
---------------------------

yup.

[ttatcs]

Is there a line number with that error message?

Edit: I've tested a Lagarith YUY2 interlaced AVI and it was fine. So...
- That is mt_masktools25.dll you are using?
- You don't have both masktools25 and masktools26 in your plugins do you?
- Could you upload a small piece of that source...

Thanks Vit , Yup and Didée . Your suggestions are priceless .
I kept both "mt_masktools-25.dll" and "mt_masktools-26.dll
" in the plugin directory .

After I put "mt_masktools-26.dll" out of that directory then the error was gone.

Really appreciate your work .

[Dogway]

Im getting heavy ghosting using QTGMC for deshimmer in an IVTC'ed anime source.
I use this line, is this normal?
QTGMC(Preset="Slower", InputType=1,sharpness=0.0)

[SubJunk]

Try different values for InputType (0-3)

[-Vit-]

Although I know a number of people use it with success, QTGMC is not really designed for anime. QTGMC relies on identifying the motion from frame to frame. With a film source the frame rate is fairly high and movements between frames are usually small; the detail of film means good matches can often be found. Anime will run at a lower rate, may contain duplicate frames, have stylized/artificial movement, and be hard to track if there is a lack of fine detail. So motion matches on anime can be poor, which will lead to ghosting on the temporal steps of the algorithm (and possible blurring / halo strengthening). The same problems can apply to game captures to a lesser extent.

Tweaking parameters might help a little [for example, reduce the temporal radiuses - QTGMC( "Slower", TR0=1,TR1=1,TR2=0) ], but if you're getting heavy ghosting then it may simply be that QTGMC is not going to work for this task.

[nhope]

For anyone who is interested, I did some 1080-60i to 480i DVD test renders and converted them to a ready-to-burn DVD project for comparison. One of them uses the "IResize" script referred to a couple of pages back, and another includes a QTGMC pass as suggested by Didée. See here (http://forum.doom9.org/showthread.php?t=160950).

[yup]

To all interested about support YUY2.
For all Avisynth build (2.58, 2.58 MT, IanB 2.6, SEt 2.6) will be properly work only mt_masktools-25***.dll, because mt_masktools-26***.dll need YV16 planar colorspace (not from Interlevead2Planar).
see http://forum.doom9.org/showthread.php?p=1497883#post1497883
If we want use 26 masktools we could work with 3 types colorspace YUY2, hacked Kassandro (Interleaved2Planar) and native YV16 it is ridiculous.
yup.

[Heaud]

I've been getting errors running QTGMC 3.31 when using qtgmc("slow", shutterblur=2, fpsdivisor=2) or any other faster preset. The error displays as "super clip does not contain needed color data". Could this be due to that slight speedup in a previous version?

[-Vit-]

I've been getting errors running QTGMC 3.31 when using qtgmc("slow", shutterblur=2, fpsdivisor=2) or any other faster preset. The error displays as "super clip does not contain needed color data". Could this be due to that slight speedup in a previous version?
Well spotted and the correct reason, thanks. I've updated the first post (http://forum.doom9.org/showthread.php?t=156028) with a fix (v3.32). There are some other very minor changes in there as well.

[K1CS1K]

Hy All!

I'd like you to check some of my footages and give me some advice. :)
I use QTGMC mainly for deinterlacing video game materials and I'd be glad if you could help me in some ways.

I've uploaded a .zip file here (http://www.megaupload.com/?d=QVAW11IQ) including the following files:

-3 .avs files (details below)
-source file (150 frames), UtVideo codec (in this case ULY0 due to QTGMC's obvious colorspace requirements)
-a test conversion with my script I normally use (x86 edition, with my PC it required 38m52s)
-a test conversion (x64) with the same specs I use in the .avs for x86 (12m1s but because of the quality loss and huge 'artifact' problem or something, I don't know what to call this deterioration of the frames)
-a test file (x64) with modified specs from my x86 .avs (35m35s ; see 'x64 without Blocksize & Overlap.avs' below)
-last test material (x64) with VirtualDub x86 (23m46s & exact filesize as x86's ; somehow x86 & x64 avisynths were installed at the same time but independently, Windows\System32 contained only 32-bit files for the process)

For all normal 64-bit conversions I used Veedub64 and in every case the output codec was ULY2 (in my opinion it's the fastest and makes the best possible results after the 'almighty' raw uncompressed; after a few identical tests it was waaay much faster than using the uncompressed output format in this case of ). All the 64-bit and 32-bit plugins needed for QTGMC are the latest, I can list them with the version numbers if you would like to. When I first met QTGMC, I read literally all the intructions in the QTGMC.avsi (now we know it's separated into 2 files) and after that I started to use these following avisynth scripts I found the toughest :

x86.avs:

SetMTMode(5,0)

AVISource("test.avi")

QTGMC( Preset="Placebo", NNeurons=4, EdiQual=2, Sharpness=0.25, SubPel=4, Blocksize=32, Overlap=16, SourceMatch=3, MatchPreset="Placebo", MatchPreset2="Slower", MatchEnhance=0.6, Lossless=2, Sigma=3, GrainRestore=0.5, FPSDivisor=2, ChromaMotion=false )

--------------------

x64.avs (the same as the x86 edition above but contains redirection to 64-bit plugins after installing avisynth64):

SetMTMode(5,0)

AVISource("test.avi")

SetWorkingDir("c:\Program Files (x86)\AviSynth 2.5\plugins64")

QTGMC( Preset="Placebo", NNeurons=4, EdiQual=2, Sharpness=0.25, SubPel=4, Blocksize=32, Overlap=16, SourceMatch=3, MatchPreset="Placebo", MatchPreset2="Slower", MatchEnhance=0.6, Lossless=2, Sigma=3, GrainRestore=0.5, FPSDivisor=2, ChromaMotion=false )

--------------------

x64 without Blocksize & Overlap.avs (after removing these specs from the script, the image became as clean as it should be):

SetMTMode(5,0)

AVISource("test.avi")

SetWorkingDir("c:\Program Files (x86)\AviSynth 2.5\plugins64")

QTGMC( Preset="Placebo", NNeurons=4, EdiQual=2, Sharpness=0.25, SubPel=4, SourceMatch=3, MatchPreset="Placebo", MatchPreset2="Slower", MatchEnhance=0.6, Lossless=2, Sigma=3, GrainRestore=0.5, FPSDivisor=2, ChromaMotion=false )

--------------------

Usually for normal length gameplay videos (4-5 mins) the average conversion speed is 0,25fps with my PC & the .avs files above.

Please help me in improving my script and make the best out of the source material. ;)

Thanks in advance,
K1CS1K

Our YouTube channel (http://www.youtube.com/K1CS1K)

[-Vit-]

That source is not interlaced. It's progressive that has been weaved strangely. One field of each frame exactly matches the other field of the next frame. A capture issue maybe? Anyway, no deinterlacing is needed:

AviSource( "YourPathTo\test_150_frames.avi" )
ComplementParity() # Might not always need this line
SeparateFields().Trim(1,0).Weave()


The source is sampled strangely horizontally, as though every 7th and 8th column have been swapped. That is either my decoder (I rarely use Ut), or it might be your emulator, capture process, encoder or something... You would need to fix the problem component or write a custom script if the problem is at your end.

You might prefer the look of your encodes because the QTGMC temporal processing smooths things a little. However, QTGMC is not the right tool for this job. [Those QTGMC settings were overkill for any source. MatchPreset="Placebo" is never worth it.]

[nhope]

-Vit-, I re-labelled the preset tables "3.32". You can all sleep soundly again now.

[egrimisu]

maybe it's out topic but shall i compress with qtgmc using selecteven if i want to deinterlace a pal dv source. I want to transfer it to dvd but i'm not sure if dvd players are compatible with pal 50fps progresive. Hope you cand help.

[Boulder]

I think you might want to experiment with the single-rate output options in QTGMC instead of using QTGMC+SelectEven. You could get a more film-like appearance to the result.

[egrimisu]

so basicly that means that i can't use 50fps in a pal dvd. i haven't found any single-rate option in qtgmc, plese guide me.

[Boulder]

You cannot use 50fps streams for DVD. See the "Shutter Speed Motion Blur / Frame Rate" section in the QTGMC documentation, that's where the magic is.

[egrimisu]

You cannot use 50fps streams for DVD. See the "Shutter Speed Motion Blur / Frame Rate" section in the QTGMC documentation, that's where the magic is.

does the hardware dvd player know how to decode 25fps interlaced into 50fps full temporal resolution? i mean a lot of anime dvd are compressed like that, but i don't have a hardware player to test it, pc software player just blend frames.

[kypec]

I assume every standalone DVD player knows how to deinterlace 50i -> 25p if the source is in MPEG2 alias original DVD-Video format. If you are going to use MPEG4 ASP (Xvid, DivX) then your best bet is to deinterlace with QTGMC and encode as 25p -> please see the QTGMC documentation that Boulder had suggested.

[egrimisu]

I assume every standalone DVD player knows how to deinterlace 50i -> 25p if the source is in MPEG2 alias original DVD-Video format. If you are going to use MPEG4 ASP (Xvid, DivX) then your best bet is to deinterlace with QTGMC and encode as 25p -> please see the QTGMC documentation that Boulder had suggested.

Hi and thanks for the answer, but my question was if the dvd player can convert 50i to 50p (like qtgmc does). Thanks

[Boulder]

If you encode PAL content for viewing on the DVD player, you have to have the framerate at 25fps but you can either encode as interlaced or progressive. If the material is flagged as interlaced, the player will deinterlace on the fly (EDIT: and output at 50fps). That's why I would go for the QTGMC route, unless you want to keep the fluid motion of truly interlaced content.

[Didée]

A DVD player does not need to deinterlace anything. Decode picture, send picture to display, job done. Okay, the player needs to be able to do split the fields and to re-combine them (to obey RFF flags), but the player does not need [to be able] to perform deinterlacing. That's the job of the display device.

Of course, nowadays HW DVD players often do offer lots of additional functionalities, but that's all optional and varies between different devices. Per DVD standard, the requirements are little. Keep in mind back when the DVD standard was made, all display devices were interlaced CRTs. :)

[egrimisu]

If you encode PAL content for viewing on the DVD player, you have to have the framerate at 25fps but you can either encode as interlaced or progressive. If the material is flagged as interlaced, the player will deinterlace on the fly (EDIT: and output at 50fps). That's why I would go for the QTGMC route, unless you want to keep the fluid motion of truly interlaced content.

Now thing are pretty clear now, thanks, so any pal device output 50fps(i believe because the displays are at 50hz) ( if the source is 25fps progressive every frame will be doubled.
if my source is 25fps avi usig dvsd codec, video is interlaced but when i compress it to mpeg 2 i shall mark it as interlaced and then the hardware player will do the rest.

[dcrht]

There is no SSE3Tools for 64 bit avisynth so I couldn't import YUY2 videos. :(

[smok3]

any recommendations of the type of virtual machine to run such scripts in? is virtualbox working well with MT versions? should i find a copy of 64bit windows? ect.. (curently i'am strugling a bit with parallels, host is OSX)

does the hardware dvd player know how to decode 25fps interlaced into 50fps full temporal resolution? i mean a lot of anime dvd are compressed like that, but i don't have a hardware player to test it, pc software player just blend frames.

anime at 50fps? i thought they are more like 2fps...

[Anakunda]

Hello, how do I implement QTGMC with StaxRip, is it enough to copy all support plugins + QTGMC-3.32.avsi into Avisynth\plugins dir, then replace YaDiff() with
QTGMC( Preset="Slow" )
SelectEven()
or is it more complicated?
Do I have to #include QTGMC-3.32.avsi first somehow, or is it autoloaded by AviSynth?
Thanks!

If I omit SelectEven(), then the result framerate is doubled, ie. for 25fp=>50fps? 50fps comes me quite overkill for an human eye, please explain.

[SubJunk]

50fps comes me quite overkill for an human eye, please explain.24-25fps is the past (http://www.spirton.com/james-cameron-george-lucas-and-peter-jackson-want-higher-framerates/), another link (http://www.digitaltrends.com/entertainment/director-peter-jackson-the-hobbit-is-shot-at-twice-the-normal-frame-rate/)

[Anakunda]

Ok let's move to high framerates. Do twice the framerate necessary mean doubled bitrate?
now I struggle with the speed. At Preset="slower" the encoding speed is really slow and I see CPU is not at 100% utilization. WHere do I get multithreaded version of AviSynth, and do I need also all plugins in multithreaded version?

[SubJunk]

Ok let's move to high framerates. Do twice the framerate necessary mean doubled bitrate?
now I struggle with the speed. At Preset="slower" the encoding speed is really slow and I see CPU is not at 100% utilization. WHere do I get multithreaded version of AviSynth, and do I need also all plugins in multithreaded version?Twice the framerate means roughly 1.4x the bitrate for x264, but it depends on the type of video. for drama tv shows the bitrate will be almost the same but for action movies much higher

for avisynth mt i recommend first installing 2.6 (http://www.videohelp.com/tools/Avisynth) then copy this file (http://www.spirton.com/avisynth-2-6mt-with-version-info/) over the one in System32/SysWOW64

[Anakunda]

for avisynth mt i recommend first installing 2.6 (http://www.videohelp.com/tools/Avisynth) then copy this file (http://www.spirton.com/avisynth-2-6mt-with-version-info/) over the one in System32/SysWOW64

I fell like having problem setting QTGMC MT usage. Copied your avisynth.dll over old one into SysWOW64
This is my basic script playable:

LoadPlugin("D:\media\StaxRip\Applications\DGMPGDec\DGDecode.dll")
SetMTMode(5, 2)
MPEG2Source("H:\video\(pending)\Simpsons VIII (12) temp files\Simpsons VIII (12).d2v")
Crop(0,0, -Width % 8,-Height % 8)
ConvertToYV12()
QTGMC( Preset="slow", EdiThreads=2 )
SelectEven()
Crop(6,0,-10,-0)

If I add the SetMTMode(2) command after MPEG2Source(), Staxrip crashes on preview, what's wrong?

/edit
SetMTMode(2) after MPEG2Source() doesnot invoke crash if I remove the first SetMTMode(..), does it matter?
Moreover StaxRip inhibits me from inserting anything before MPEG2Source on encode, how to elude this obstruction?

[SubJunk]

I'm not sure, I have never used StaxRip

[Didée]

Seeing the source is 'The Simpsons', it's questionable if using QTGMC is indicated at all. Simpsons might come along as telecined, or field blended (normconversion), or whatever. But it surely isn't "natural interlacing". Probably it's a case for either TFM(+TDecimate), or bob+Srestore. But not QTGMC.

[Anakunda]

Hello there, thanks for your support. This is finally the script that doesnot crash on my machine:

SetMTMode(5,1)
LoadPlugin("D:\media\StaxRip\Applications\DGMPGDec\DGDecode.dll")
MPEG2Source("test.d2v")
Crop(0,0, -Width % 8,-Height % 8)
ConvertToYV12()
Crop(40,0,-40,-0)
SetMTMode(2)
QTGMC(Preset="Slow", EdiThreads=2)
#Distributor()

I have commented the Distributor() and it doesnot crash, is it better to leave it off if not crashing?
The only initial SetMTMode() values that do not crash are (5,1). The problem is that the utilization of both cores is still not 100%, though it seems to be higher than when running in ST mode. The problem is that if I increase the number of threads to SetMTMode(5,2), I get x264 crash.
Script invoked with command line
x264 --preset veryslow --tune film --crf 20 --sar 16:11 --output test.h264 script.avs
Please help.

Seeing the source is 'The Simpsons', it's questionable if using QTGMC is indicated at all. Simpsons might come along as telecined, or field blended (normconversion), or whatever. But it surely isn't "natural interlacing". Probably it's a case for either TFM(+TDecimate), or bob+Srestore. But not QTGMC.

This was a DVB-T stream so probably whatever it was originally, it is normal interlacing now, I see the stripes

[TheSkiller]

Yes, of course there is interlacing visible but that does not imply it is "natual interlacing" as Didée discribed it.
(Q)TGMC is working properly only with footage where every field represents a different step in time. With a Cartoon source this is pretty much never the case. With normconverted footage you don't have a fully natual type of motion anymore either (there is blending).
Sure, you can still use (Q)TGMC but there are more effective ways to deal with this type of footage.

[Anakunda]

Yes, of course there is interlacing visible but that does not imply it is "natual interlacing" as Didée discribed it.

Do you think about nonstandard interlacing causing these crashes? I can use different source then with natural interlacing, the goal is to find a working setup for QTGMC no matter what video source I use.
Thanks for notes for cartoon movies. Is TFM/TDecimate always better for animated sources?

[TheSkiller]

Do you think about nonstandard interlacing causing these crashes?
No, it certainly never depends on the "type" of interlacing or content of the picture whether or not a script crashes. Don't get me wrong, your interlacing is not "non-standard", it's just not the right type interlacing for (Q)TGMC.

The core technique that (Q)TGMC uses to deinterlace can produce bad results with sources where the motion is not natual, that's all.

the goal is to find a working setup for QTGMC no matter what video source I use.Forget it. There is no all-round script for every video source you could imagine. It starts with the issue we just pointed out with (Q)TGMC not being very suitable for cartoon stuff.
But of course it also depends a lot on your expectations, so in the end it's you who has to decide what's good.

TFM/TDecimate are THE tools to use for any telecined sources which includes non-CGI cartoons if it has not been normconverted somewhere in-between. In case of The Simpsons and if you're a PAL user you can be sure that it is normconverted if interlacing is visible. In that case only SRestore is an option (besides plain 50fps bob-deinterlacing or keeping it interlaced of course).

[atl]

Moving from 32bit to 64bit Avisynth version got increase from 48 to 61 FPS on i7-2600.
Using older MVtools
I still can't find MVtoos 2.5.11.2 or above compiled for 64bit, can somebody point me where can download it. Many thanks

[Mounir]

I have a problem with QTGMC i'm trying the "best settings" taken from the manual for instance this:
QTGMC( Preset="Slower", SourceMatch=3, Lossless=2, Sharpness=1.0, TR2=1 ) and it return an error there is no function VerticalCleaner not sure what this is about. My video is YUY2, huffyuv

[-Vit-]

I have a problem with QTGMC i'm trying the "best settings" taken from the manual for instance this:
QTGMC( Preset="Slower", SourceMatch=3, Lossless=2, Sharpness=1.0, TR2=1 ) and it return an error there is no function VerticalCleaner not sure what this is about. My video is YUY2, huffyuv
Those settings work fine for me on a YUY2 raw interlaced source. Have you got the VerticalCleaner plugin (http://home.arcor.de/kassandro/prerelease/VerticalCleaner.rar)...?

Must update the examples in the manual. There is, of course, no such thing as "best settings". However, this is fairly transparent, keeping the source relatively untouched:

QTGMC( "Slower", TR2=1, SourceMatch=2, Lossless=2, Sharpness=0.2, EZKeepGrain=0.6 )

Vary EZKeepGrain depending on source noise/grain (assuming you want to keep it - this is transparent remember). Adjust Sharpness if you feel it's necessary (faster motion videos generally need a bit more). Add and adjust NoisePreset if you wish (it defaults to "Fast") for some very slight improvements to fine detail/grain at the cost of some speed [very pedantic].

I don't really recommend SourceMatch=3 now, rarely makes any noticeable effect.

[Mounir]

You're right i didn't have this plugin, still i have an error window (not a script error) with AVSp the frames are not processed as i have to cancel this window all the time:

Traceback (most recent call last):
File "AvsP.pyo", line 5819, in OnMenuVideoToggle
File "AvsP.pyo", line 8925, in ShowVideoFrame
File "AvsP.pyo", line 9474, in PaintAVIFrame
File "pyavs.pyo", line 322, in DrawFrame
File "pyavs.pyo", line 301, in _GetFrame
File "avisynth.pyo", line 277, in GetFrame
WindowsError: exception: access violation reading 0x3EF69000
Traceback (most recent call last):
File "AvsP.pyo", line 7123, in OnPaintVideoWindow
File "AvsP.pyo", line 9474, in PaintAVIFrame
File "pyavs.pyo", line 322, in DrawFrame
File "pyavs.pyo", line 301, in _GetFrame
File "avisynth.pyo", line 277, in GetFrame
WindowsError: exception: access violation reading 0x3EF69000

etc...

[-Vit-]

I don't know what that is. Nothing QTGMC in there.

Could be out of memory I suppose, if you're running with many threads. If so cut down SetMTMode threads. And EdiThreads when using SourceMatch. Tweak the MemoryMax.
The usual multi-threaded juggle...

[Mounir]

This multithread thing is complicated i have followed the instruction, installed avisynth.dll in sys32, mt.dll in the plugin folder but no luck (i'm forced to close Avsp) plus i can't find infos on MemoryMax

my script so far:
Avisource("H:\MyVideo.avi", fourCC="HFYU", pixel_type="YUY2").trim(2500,2900)
MT(QTGMC( "Slower", EdiThreads=2, TR2=1, SourceMatch=2, Lossless=2, Sharpness=0.2, EZKeepGrain=0.6 ))

The maximum threads i can have is 2 on this cpu

Edit:
I have also tried (i found out for setmemorymax)

SetMemoryMax(512) #
SetMTMode(2,2) #
Avisource("H:\MyVideo.avi", fourCC="HFYU", pixel_type="YUY2").trim(2500,2900)
Setmtmode(2,2) #
AssumeTFF
QTGMC( "Slower", EdiThreads=2, TR2=1, SourceMatch=2, Lossless=2, Sharpness=0.2, EZKeepGrain=0.6 )
selecteven()
Distributor() # for multithreading

AvSp bug and i'm forced to close it, i think the problem is with masktools-v2.0a36 which i use for QTGMC

[Didée]

First script, corrected:

Avisource("H:\MyVideo.avi", fourCC="HFYU", pixel_type="YUY2").trim(2500,2900)
MT("""QTGMC( "Slower", EdiThreads=2, TR2=1, SourceMatch=2, Lossless=2, Sharpness=0.2, EZKeepGrain=0.6 )""", threads=2,overlap=16)
(When the filter call within MT(...) contains "quotes", then you need """triple quotes""" for the outer MT() call.)


Second script, corrected:

SetMemoryMax(512) #
SetMTMode(5,2) # Not using multithreading for the source filter is more safe
Avisource("H:\MyVideo.avi", fourCC="HFYU", pixel_type="YUY2").trim(2500,2900)
Setmtmode(2,2) #
AssumeTFF
QTGMC( "Slower", EdiThreads=2, TR2=1, SourceMatch=2, Lossless=2, Sharpness=0.2, EZKeepGrain=0.6 )
selecteven()
# Distributor() # don't use Distributor for apps using VfW API. (AvSP does, or not?)

With only 2 cores / 2 threads, the MT(""" ... """) script probably is faster for you.
(With complex scripts, the overhead for SetMTmode often eats a good part of the theoretical benefit when only 2 threads can be used. For QTGMC, SetMTmode starts to shine with 4 threads and above. With a quadcore, o'course. DualCore/2threads is too much a toy for this.)

[Boulder]

I think you could use SetMTMode(2) with HuffYUV since it's all keyframes.

[Didée]

Possible. But it doesn't help anyway - in comparison to calculating QTGMC's processing chain, decoding Huffy is totally peanuts.

Hence, better to avoid SetMTmode for the source filter completely. Best case=it doesn't help anything, worst case=it crashes. So, what to recommend to a user that is fighting with script crashing? :)

[Boulder]

Since dealing with SetMTMode can be compared to walking on the edge of a cliff, I usually have to try different things to get through a long encode. Changing SetMTMode for the source filter has helped sometimes, don't ask me why ;)

OT: I built a "new" second-hand computer last weekend. After a few crashes, I noticed that it's once again time to try to find the working MT-Avisynth version..grrrr.

[aegisofrime]

I have no idea why you guys say that SetMTMode is unstable. My script here:


SetMTMode(5,4)
DGDecode_mpeg2source("C:\Raws\Work.d2v",idct=5)
SetMTMode(2,4)
QTGMC(Preset="Slower")

Distributor()


Is perfectly stable. I use it for all my DVD encodes and it has never crashed before.

[-Vit-]

In the first post I've added a modded plugins package. All the key plugins have been recompiled using SEt's threading fixes and an extra tweak that prevents the main crash that I see when multi-threading. I can't guarantee they will help, but they might be worth a try. They won't prevent running out of memory if you use complex settings/many threads.

Read the instructions in the zip before asking what to do with the contents.

Instructions for stability and getting the best out of SetMemoryMax & SetMTMode are also in the first post (and the QTGMC instructions).
Also covers the EdiThreads setting, which is best set to 1 if you're getting problems.
Some people's computers simply don't like SetMTMode. Nothing to be done except fix AviSynth.

I don't recommend MT("") unless absolutely necessary, because splitting up the image will affect motion analysis (you always get problems at the edges with MVTools, with MT("") you have edges in the middle of your image...)

[Mounir]

Thanks Didée, MT work and setMtmode doesn't (error with mt_masktools-25.dll apparently) on this computer but i think i will avoid to use it as it split the image

[Mini-Me]

I went digging through the script to see how SourceMatch and the lossless modes interact with noise removal/retention modes. First, I want to congratulate -Vit- on writing the most beautifully commented script I've seen yet. :) Second, I want to ask if I properly understand the interaction of NoiseRestore and GrainRestore with Lossless mode 1.

In case it helps anyone (or helps -Vit- point out if I went wrong somewhere), here is the rough order of major QTGMC steps:
Obtain motion vectors: Bob, temporally smooth to fight bob shimmer, and perform motion analysis.
Optionally remove/bypass noise: Optionally denoise the original input clip and/or save extracted noise and fine detail (optionally stabilizing it) for later recovery. The [optionally] denoised clip is now known as the "inner" clip, and it is used in place of the original source clip for the rest of the filter (unless and until the extracted noise is readded).
Primary bob deinterlace: Perform a high-quality upscale/interpolation of the fields in the "inner" clip to attain full frame height. Temporally smooth the result with motion-compensated neighbors (using previous motion analysis) to fight bob shimmer, then repair to fight blurring (and artifacts from bad motion compensation?). (Side question: Is the temporal smooth also meant to absorb real detail and structure from motion-compensated neighbors, or does that get removed anyway by the repair step?)
SourceMatch: Optionally redo the previous step (I think) with input refined by the previous result, such that the new result will have original (non-generated) lines better matching the actual lines from the source. (Refine this up to two more times, though not exactly in the same manner). As far as I can tell, this is meant to recover detail lost in the previous temporal smoothing, and the degree to which the "original lines" match the original source also indicates how well the generated lines probably match the "original lines" of neighboring fields.
Lossless mode 2: If the lossless mode is 2, drop in the original fields of the inner clip here.
Resharpen: ...with a complicated series of steps involving limiting and back-blending.
GrainRestore: Optionally readd noise prior to the final smooth, proportional to GrainRestore.
Clean the result: Do a final [light, motion-compensated] temporal smooth, then repair. (Also, finish sharpness limiting if necessary.)
Lossless mode 1: If the lossless mode is 1, drop in the original fields of the inner clip.
NoiseRestore: Optionally readd noise after the final smooth, proportional to NoiseRestore.

What I've learned is that SourceMatch and Lossless mode 2 work perfectly and intuitively with denoising, NoiseRestore, and GrainRestore. The reason is, SourceMatch and Lossless actually match to the [optionally denoised] source rather than the original source. This makes perfect sense, and it's actually "easier" and more robust than I expected:
If you want to denoise, using the original fields (before denoising) for SourceMatch and Lossless would contradict that.
If you denoised with the intention of adding it back later, your intent was likely to retain the original noise "as-is." Matching to the inner clip with SourceMatch and Lossless accomplishes this, whereas using the true original fields and then adding noise back in would result in amplified noise.
If you didn't denoise at all, there's no difference between the original and inner clips anyway.

However, Lossless mode 1 does not seem to work well with GrainRestore > 0.0: Only the generated lines will have grain restored, and lossless mode 1 will subsequently replace the original lines with the [possibly denoised] lines from the inner clip.

That said, assuming GrainRestore is 0.0, Lossless mode 1 seems to work fine:
If you denoised the clip and NoiseRestore is 0.0, you'll end up with a denoised clip with lossless denoised lines from the original.
If you denoised the clip and NoiseRestore is 1.0, you'll end up with a noise-bypassed clip, and the original lines should presumably be the same as the ones from the source (because the exact difference between the source and the "inner" clip is being readded to the original lines...I think ;)).
If you didn't denoise the clip, you'll want to use 0.0 for NoiseRestore, but using higher values still produces the expected result of amplifying noise.
If you're using a Lossless mode of 1, you're probably looking for one of those results anyway.

Am I correct? If I'm wrong, let me know what I got wrong. If I'm right, it might be a good idea to add a few details to the instructions. The instructions say that Lossless mode 1 will use the exact original pixels from the source lines if NoiseRestore is 0.0, but isn't this only true if you did NOT denoise the clip? (If you did denoise the clip, then a NoiseRestore of 1.0 would actually give the original lines, correct?) Similarly, the directions mention nothing about how GrainRestore and Lossless mode 1 appear to interact poorly.

Thanks :)

[-Vit-]

Your description is correct in most every detail. Including the interaction between Lossless mode 1 and noise processing. Yes, I should note that GrainRestore (and by implication EZKeepGrain) are not effective with that mode. I do need to look more carefully at Lossless mode 1, the "true" lossless mode (I will also tweak these setting names for clarity). It hasn't been a recent priority since you can get very close to lossless with source-match and Lossless mode 2, without introducing the artefacts that true lossless mode 1 will bring.

I'm glad you spotted the detail behind the inner clip, especially the interaction of source match and noise processing. I don't think I explicitly note that. One other point to note is that resharpening occurs on the denoised clip, with the noise being added back on after sharpening. That means it won't enhance noise so much (but may miss some minor detail to be sharpened)...

Regarding your side question: Didee may have more to comment as this step inherits from original TGMC. The main temporal smooth (TR1) has a variety of effects depending on the quality of the motion analysis. It's main intent is to remove bob-shimmer. If the motion analysis is correct then it will slightly pull in detail from neighbor frames, but may also slightly weaken detail in the current. If the motion analysis is off, then it can blur or ghost or other nasty things - depends on the exact circumstances. The repair step helps to remove some of that.

The noise motion compensation step, designed to "stabilize" the noise, will also pull in detail from neighbor frames. It only looks at one neighbor frame, but I have a (very slow) variant that looks at several sucking in as much extra detail as possible from neighbors. Creates a kinda "enhancing" effect; not sure I like it. Can get the effect by setting EZKeepGrain high with a slow NoisePreset.

[Mounir]

I'm trying my script again right now with MT i'm not sure why VIt called the best settings "lossless" because it's not lossless for me , take a look:

Original:
http://img195.imageshack.us/img195/5138/originalrn.jpg
QTGMC:
http://img802.imageshack.us/img802/8199/qtgmc.jpg

settings used:
MT("""QTGMC( "Slower", EdiThreads=1, TR2=1, SourceMatch=2, Lossless=2, Sharpness=0.2, EZKeepGrain=0.6 )""", threads=2,overlap=16)

[-Vit-]

You're expecting too much. Horizontal detail like that is the Achilles heel of of all deinterlacers. You might find a different deinterlacer will do a better job for that particular shot, or maybe not, hard to tell from a single image.

[Mounir]

Well msu seems to get it right with their deinterlacer (http://www.compression.ru/video/deinterlacing/index_en.html), what you think ? I think it's neater maybe i'ts just me i don't know

[-Vit-]

I think it's neater maybe i'ts just me i don't know
They are different pictures to yours. No problem cases there. But if their deinterlacer works for you then use it.

[Mounir]

I wish i could but it's not free it appears. Anyways, i truly salute the efforts made for this deinterlacer and hope you make it even better.

[Didée]

Well msu seems to get it right with their deinterlacer (http://www.compression.ru/video/deinterlacing/index_en.html), what you think ? I think it's neater maybe i'ts just me i don't know

You believe the marketing fairytales? Those are cherry-picked spots to convince the audience.

If we were going to produce cherry-picked spots of QTGMC, they would look at least as impressive as those. :)

Also, you should post a small sample of your problem source. That screenshot is zoomed, and it's impossible to tell of what kind the source is at all. It could be a case of using QTGMC wrongly, or a case of filter-not-suited-for-the source (if it's telecined or fieldblending-interlaced).

[Mini-Me]

I'm trying my script again right now with MT i'm not sure why VIt called the best settings "lossless" because it's not lossless for me , take a look:

Original:
http://img195.imageshack.us/img195/5138/originalrn.jpg
QTGMC:
http://img802.imageshack.us/img802/8199/qtgmc.jpg

settings used:
MT("""QTGMC( "Slower", EdiThreads=1, TR2=1, SourceMatch=2, Lossless=2, Sharpness=0.2, EZKeepGrain=0.6 )""", threads=2,overlap=16)

As -Vit- and Didee said, a sample would be helpful to see what's going on here. I'm not even sure what kind of image comparison I'm looking at; it could be any one of these four:
Woven frame and final frame: Assuming this is a pure interlaced source and QTGMC is being used correctly, there's a problem here. I don't think this is the case though.
Bobbed field and final frame: If this is the case, it would be helpful to see the neighboring bobbed fields from the source too (or better yet, the source clip).
Original separated field and supposedly (but obviously not) lossless counterpart from the final frame: Assuming this is a pure interlaced source and QTGMC is being used correctly, the last smoothing is doing more damage than I'd expect.
Original separated field and generated pair from the final frame: This kind of comparison wouldn't really make any sense; it would be better to see the whole final frame.

A thought just came to me though: The temporal smooths in QTGMC might GREATLY benefit from "gamma-corrected" smoothing, when it comes to thin horizontal details. See here (http://forum.doom9.org/showthread.php?t=160038) to get an idea of what I'm talking about. The basic idea is that at ordinary gammas, any kind of smoothing or averaging ends up reducing the overall luminosity of areas (the softened grill of the car has been darkened excessively). To correct for this, you would reduce the gamma, perform your operations, and increase the gamma again.

As a quick test to see if it might help, you could try sandwiching your QTGMC call between with two calls to levels or LaTo's Smoothlevels (the first reducing gamma and the second increasing it again). The optimal solution requires 10+ bit processing in Avisynth, but if the Smoothlevels solution makes the picture look better in the meantime, why not? :) If time is no object, you could even repair very subtle detail (least significant bits) lost in the gamma shifts by doing the following:
Run QTGMC normally and save the result as A.
Reduce source gamma, run QTGMC on darkened source, and increase the gamma of the result, and save the result as B.
Run some kind of repair filter that replaces pixels from B with pixels from A if they're in the same brightness neighborhood (within 1 or 2 luma or so). I could probably think of some hacky way to do this off the top of my head, but I don't want to embarrass myself in the presence of my betters. ;)

[Didée]

Regarding 'gamma correction' -- generally a good idea. But a simple un-gamma before QTGMC + re-gamma afterwards probably is no good idea. Besides of the 8bit problem, I assume that MVTools' motion search will not react very gracefully if ME is performed on a clip with linear gamma. If at all, then the motion search should be done on the normal input, and the gamma operations be done only in the parallel processing path ('super' clip & separate base clip for rendering).

[Mini-Me]

Regarding 'gamma correction' -- generally a good idea. But a simple un-gamma before QTGMC + re-gamma afterwards probably is no good idea. Besides of the 8bit problem, I assume that MVTools' motion search will not react very gracefully if ME is performed on a clip with linear gamma. If at all, then the motion search should be done on the normal input, and the gamma operations be done only in the parallel processing path ('super' clip & separate base clip for rendering).

I never even thought about linear gamma making motion estimation's job harder, but by compressing the lower-mid range compared to usual, I guess it might. In that case, I may have finally found a use for supplying my own motion clip to QTGMC! ;)

Actually, by the same token, would motion search possibly be better if performed on a clip with higher-than-usual gamma?

[javlak]

Hi. According to this (http://avisynth.org/mediawiki/Removegrain), the RemoveGrainSSE3.dll bug has been fixed. What is the official QTGMC author's stance on this? Can we revert to RemoveGrainSSE3 or should we still continue to use RemoveGrainSSE2.dll?

And I do not wish to step on any toes or insult anyone, but I have downloaded pretty much every free filter/plugin from MSU. Even though Mounir talks about a payware filter, if their freeware is any measure, I can safely say that their quality cannot match anything the high-performance plugins avisynth can throw at them. The demo pictures they show on their website are nothing more than demo pictures, not real-life applications. Believe me, I've tried Boolet, FieldsKit, Topaz Enhance et al, and QTGMC leaves them far behind. Your particular case is, as others have already mentioned quite difficult for any deinterlacer.

[-Vit-]

The fixed version of RemoveGrain has some changed and undocumented functionality. It comes with no source code. So it can't be recommended. In the last version with source code there is exactly one line of difference between SSE2 and SSE3 versions. It won't make a difference.

I tend to agree regarding the MSU plugins. Whilst the guys seem to know what they're doing, the plugins they do release are not very special. Their Video Quality Measurement tool is quite handy though.

[aegisofrime]

Hi. According to this (http://avisynth.org/mediawiki/Removegrain), the RemoveGrainSSE3.dll bug has been fixed. What is the official QTGMC author's stance on this? Can we revert to RemoveGrainSSE3 or should we still continue to use RemoveGrainSSE2.dll?

And I do not wish to step on any toes or insult anyone, but I have downloaded pretty much every free filter/plugin from MSU. Even though Mounir talks about a payware filter, if their freeware is any measure, I can safely say that their quality cannot match anything the high-performance plugins avisynth can throw at them. The demo pictures they show on their website are nothing more than demo pictures, not real-life applications. Believe me, I've tried Boolet, FieldsKit, Topaz Enhance et al, and QTGMC leaves them far behind. Your particular case is, as others have already mentioned quite difficult for any deinterlacer.

Q/TGMC and x264 were the two pieces of software that woke me up to the idea that freeware can be better than payware :)

[Mounir]

You believe the marketing fairytales?
Hmm yeah i believe the hype, why don't you ?

[Overdrive80]

Update doc plugin in spanish. Link: http://www.mediafire.com/?1h1r5tf2t9t8rq1

[CruNcher]

Hmm yeah i believe the hype, why don't you ?

Hehe because without hard facts it's hard and these samples are known once schumacher and flag :) people here on doom9 are confronted with situations most test sequences shown their don't reproduce :)
And if you want to test the base of their algorithm you can do that http://www.yuvsoft.com/download/deinterlacing/index.html
And well it's Didée (nothing to add) :D

[upyzl]

First I'm new to QTGMC here...
A question maybe stupid for others but confused for me

Could QTGMC use FFT3Dgpu instead of FFT3Dfilter? How?

[-Vit-]

Could QTGMC use FFT3Dgpu instead of FFT3Dfilter? How?
I guess you could replace the single call to FFT3DFilter in the script.
I have never done so because FFT3Dgpu is unstable for me. I believe it is also unstable generally with SetMTMode, which many people use with QTGMC.

I don't think it would give a great speed up because denoising is only a small part of the processing (and not enabled by default anyway).

[smok3]

thanks!

p.s. Getting around 1.7 fps in virtual machine with non-MT version (on quad core machine).

[upyzl]

@ -Vit-

Thanks for the reply!

[The Showstopper]

Does QTGMC works on interlaced credits (30i) scrolling on top of a telecined background??

[Heaud]

It needs to be done with a script such as the one in this link (http://forum.doom9.org/showthread.php?p=1466105#post1466105).

[Lighto]

Thanks guys for the awesome script.
I am having trouble getting MT to work without crashing, would it make sense for me to run parallel non-MT instead to fully make use of my CPU?

Thanks. :)

[-Vit-]

Would it make sense for me to run parallel non-MT instead to fully make use of my CPU?
It will certainly works and can be effective sometimes - depends on the source and settings. It's not as fast as using SetMTMode - having your CPU under full load does not necessarily mean it's working efficiently.

Have you tried the various ideas from the first post? Particularly different versions of Avisynth MT and my modded plugins? Also try running MT at less than full load, that can help whilst still being faster.

____

On a related point, I recommend updating to nnedi3 0.94 (http://forum.doom9.org/showthread.php?t=147695), it gives a nice speed-up (thanks tritical!).

[kolak]

I tried new plugins, all versions of avisynth and on HD sources and my 24core machine it crashes until I limit it to <4 cores. Even this sometimes crashes- only not using mt gives stable results, but 2fps is way to slow :(

It's a bit hopeless :(
Quality is better than anything I've seen including big K$ solutions.


Andrew

[-Vit-]

Well, you can safely use a faster preset with HD material, e.g.
QTGMC( "Faster" )


Also, you should usually output to a lossless intermediate file when using QTGMC on HD. Then encode that lossless file to x264 or whatever. Otherwise Avisynth will run out of memory (it's still a 32-bit app, so limited to 2Gb).

[kolak]

Well, you can safely use a faster preset with HD material, e.g.
QTGMC( "Faster" )


Also, you should usually output to a lossless intermediate file when using QTGMC on HD. Then encode that lossless file to x264 or whatever. Otherwise Avisynth will run out of memory (it's still a 32-bit app, so limited to 2Gb).

I use one of Canopus Lossless/HQX/HQ (or uncompressed).

I have to try again 64bit, but last time I had no luck. Couldn't make it working at all :(

Andrew

[Chikuzen]

I use one of Canopus Lossless/HQX/HQ (or uncompressed).

I have to try again 64bit, but last time I had no luck. Couldn't make it working at all :(

Andrew

http://forum.doom9.org/showpost.php?p=1507717&postcount=17

[smooth_hands]

Well, you can safely use a faster preset with HD material, e.g.
QTGMC( "Faster" )


Also, you should usually output to a lossless intermediate file when using QTGMC on HD. Then encode that lossless file to x264 or whatever. Otherwise Avisynth will run out of memory (it's still a 32-bit app, so limited to 2Gb).

Vit, would you mind sharing the methods/command line calls you use for the lossless intermediate file step? By looking at the command line parameters in MeGUI, I have figured out how to use mencoder to do this, but the intermediate files it produces are massive (like 250+GB for a normal length movie if I recall). Is this unavoidable?

[-Vit-]

In MeGUI just check "Add pre-rendering job". That will first run the avisynth script to output a lossless huffyuv file, then encode that file in a second pass. The intermediate file will be around 200Gb per hour at HD. It will be deleted after processing if you have "Delete intermediate files" checked in Options.

If you don't have the temporary space:
- Don't use a pre-rendering job - reduce settings/threads so it can cope
- Use avs2yuv (or avs2pipe?) to pipe the avisynth output to a separate x264 process to do the encoding. Each process will get it's own memory space. I haven't tried this, I saw the idea here, though I'm sure others use this method for the same reasons.

[kolak]

http://forum.doom9.org/showpost.php?p=1507717&postcount=17

Yes- this is some solution, but if you have few files, than it takes a bit of additional work. If we could make some batch file, which would do all automatically, than that would be more useful:)

I'll try and check how much my PC can do and how much faster it's compared to single thread.

I have 24GB of RAM, so this should not be a problem.

Andrew

[smooth_hands]

In MeGUI just check "Add pre-rendering job". That will first run the avisynth script to output a lossless huffyuv file, then encode that file in a second pass. The intermediate file will be around 200Gb per hour at HD. It will be deleted after processing if you have "Delete intermediate files" checked in Options.

If you don't have the temporary space:
- Don't use a pre-rendering job - reduce settings/threads so it can cope
- Use avs2yuv (or avs2pipe?) to pipe the avisynth output to a separate x264 process to do the encoding. Each process will get it's own memory space. I haven't tried this, I saw the idea here, though I'm sure others use this method for the same reasons.

Thanks for the response. The MeGUI option is the same as what I was describing; I just take the same parameters and enter them at the command line. I want to not bring MeGUI into the equation because I have a batch file I use to do all my processing, and MeGUI adds a non-automated step right in the middle of the chain.

I have tried avs2yuv but it always seemed to go slower AND be less stable for me. I also tried reducing settings as per your guide but making it through anything longer than about 15-30min (real-time) of HD video is a crapshoot.

I just wanted to make sure there wasn't some alternative that was as stable as, but faster than, simply running QTGMC in non-multithreaded mode. I don't seem to ever get crashes that way, and it's quicker than doing 2 passes (lossless via mencoder->lossy via x264), but obviously significantly slower than 1 pass running QTGMC in multithreaded mode.

How much memory should one have to have a good chance of avoiding these crashes, anyway? I have 4GB (Windows 7 x64) with no other programs running besides basically bare minimum Windows processes and Microsoft Security Essentials.

I just love this filter so much for interlaced content that I want to do everything in my power to get it into my workflow whenever feasible :)

[-Vit-]

You need 2Gb spare memory for 32-bit avisynth. 4Gb and a low system load sounds OK.

The alternative posted above, splitting an encode into chunks (http://forum.doom9.org/showthread.php?p=1507717#post1507717), is the other way to get more speed whilst retaining stability. It's not the speedup of SetMTMode but it can help. However, to take best advantage you may need more memory - that approach relies on creating several processes rather than several threads. Each process can have its own 2Gb.

One other thing: mencoder was dropped from MeGUI quite a while ago, in favor of ffmpeg for lossless output. I use the dev builds, dunno if that change has made it to the stable builds yet.

[dfan]

hi
how i could make a timecodes' file correctly with tdecimate?

if i use qtgmc+selecteven+tdecimate(+ovr) then there are some zones to 24fps but that really are to 30fps (tfm+tdecimate+ovr detect them correctly ._.). Any idea for fix this?

thx

[nhope]

I've started testing the following script for PAL DV > NTSC-for-DVD conversion, as an improvement over my previous method of blending frames in Vegas Pro. Initial results are promising, although of course it's slow. I would welcome any comments or suggestions for improvement. Thanks

#Frameserve from Vegas Pro in YUY2.

AviSource("d:\fs.avi")
ColorYUV(levels="TV->PC") #Reverses frameserving levels squeeze
QTGMC()
Spline36Resize(720,480)

#Could save lossless intermediate here

super = MSuper(pel=2)
backward_vec = MAnalyse(super, isb = true)
forward_vec = MAnalyse(super, isb = false)
MFlowFps(super, backward_vec, forward_vec, num=60000, den=1001)

AssumeBFF()
SeparateFields()
SelectEvery(4,0,3)
weave()

[johnmeyer]

Nick,

Is your source interlaced? If so, does this script give correct results?

[edit] Forget this post. I didn't see the QTGMC line.

[-Vit-]

Can use QTGMC's own motion vectors, which it exposes for later use. This will be faster and likely give better vectors. But then you must resize after the flow

#...
QTGMC( SubPel=2 ) # SubPel must match pel in MSuper below
super = MSuper(levels=1, pel=2) # Only need multiple levels for MAnalyse
MFlowFps(super, QTGMC_bVec1, QTGMC_fVec1, num=60000, den=1001)
Spline36Resize(720,480)
#...

[Edit: If you use faster presets on QTGMC then you will need to set hpad/vpad to 16 on the MSuper line]

This process is detailed in the QTGMC docs under the heading "Accessing QTGMC Motion Vectors"

Or you could try Interframe (http://forum.doom9.org/showthread.php?t=160226) to do the 50->60fps. The script is still essentially MFlowFPS, but it has been tweaked heavily.

[nhope]

Thanks very much -Vit-. I hadn't realised that Interframe could do framerate conversions other than doubling.

So now I want to try your 2 suggestions; The MFlowFps script using QTGMC vectors and the following script using InterFrame:

#Frameserve from Vegas Pro in YUY2

AviSource("d:\fs.avi")
ColorYUV(levels="TV->PC") #Reverses frameserving levels squeeze
QTGMC()
Spline36Resize(720,480)

InterFrame(FlowPath="C:\Program Files\AviSynth 2.5\plugins\")

AssumeBFF()
SeparateFields()
SelectEvery(4,0,3)
weave()

One of the Interframe dependencies is a modified mvtools2.dll from here (http://www.svp-team.com/wiki/Download). How would you recommend managing 2 different versions of mvtools2.dll (one for QTGMC and one for InterFrame)? Or could the same version be used for both? Would I have to split the script and load plugins from different folders?

Also, RemoveGrainSSE3.dll is in the InterFrame dependencies folder, but I'm running RemoveGrainSSE2.dll with QTGMC. I guess I should go for one or the other, but which?

[-Vit-]

I skimmed the SVP modded MVTools code a while back and the changes were fairly limited in scope, so I imagine it will still work for normal use [that's just a guess though, I don't actually use it myself]. Just note it's based on 2.5.11.1. The latest version is 2.5.11.2, which fixes a crash bug that affected QTGMC with certain resolutions/blocksizes. But that shouldn't affect your use case though.

Use RemoveGrainSSE2. There was a bug in RemoveGrainSSE3 until the final version, but that one has some changed and undocumented functionality so it shouldn't be relied on. There is no noticeable difference between the two anyway.

[SubJunk]

How would you recommend managing 2 different versions of mvtools2.dll (one for QTGMC and one for InterFrame)? Or could the same version be used for both?Yeah I don't think you will have any problems using the same one for both. Most of the SVP changes have been done to a new function they created called MSmoothFPS, so MFlowBlur and MSuper (which QTGMC uses) still work as they did before AFAIK.

Also, RemoveGrainSSE3.dll is in the InterFrame dependencies folder, but I'm running RemoveGrainSSE2.dll with QTGMC. I guess I should go for one or the other, but which?The one with the highest number :D ;) But seriously it won't make much difference, you can use either one

[nhope]

Thanks -Vit- and SubJunk. I ran the InterFrame script with the original RemoveGrainSSE2.dll and mvtools2.dll supplied in -Vit-'s required plugins pack and I had no problems.

I found little difference in the quality of the result, but in my single-threaded environment the MFlowFps method was about 7-8% faster than the InterFrame method. Also InterFrame doesn't support YUY2, and I prefer not to use YV12 in this case because it gives me this chroma shift error (http://forum.doom9.org/showthread.php?t=161011) when I take interlaced YV12 into CCE Basic for MPEG-2 encoding.

So anyway, the following script gives a stunningly good PAL>NTSC conversion compared to anything I've tried before (which is A LOT of software methods). So good in fact that I think I can start just offering NTSC DVDs instead of both PAL and NTSC depending on territory (my original footage is PAL, which won't play worldwide). Thank you!

#Frameserve from Vegas Pro in YUY2

AviSource("d:\fs.avi")
ColorYUV(levels="TV->PC") #Reverses frameserving levels squeeze

QTGMC( SubPel=2 ) # SubPel must match pel in MSuper below
super = MSuper(levels=1, pel=2) # Only need multiple levels for MAnalyse
#super = MSuper(levels=1, pel=2, hpad=16, vpad=16) # For faster QTGMC presets
MFlowFps(super, QTGMC_bVec1, QTGMC_fVec1, num=60000, den=1001)
Spline36Resize(720,480)

AssumeBFF()
SeparateFields()
SelectEvery(4,0,3)
weave()

Just one thing. There are still jaggies in evidence such as around the diver's mask in this screengrab. I suspect they are unavoidable, but any suggestions to reduce them?

http://dl.dropbox.com/u/21489814/jaggies.jpg

[Didée]

There are still jaggies in evidence such as around the diver's mask in this screengrab. I suspect they are unavoidable, but any suggestions to reduce them?
That can't be judged from one screenshot, because you're re-interlacing. Deactivate the Separate/Select/Weave sequence, and check if there still are jaggies then.

(Though, straight from the guts, those jaggies don't really look like they're re-interlacing related, but rather like something's going seriously wrong. It pretty much looks like aliasing from a simple standard deinterlacer!)


Oh, BTW ... if the frameserved output really has a PC-->TV levels squeeze, then that's not good at all ... in particular for underwater shots. (During level squeeze information is getting lost, and you won't get that loss back by expanding again....)

[nhope]

That can't be judged from one screenshot, because you're re-interlacing. Deactivate the Separate/Select/Weave sequence, and check if there still are jaggies then.
Sorry. that screenshot was misleading. It was actually of a single field doubled in height after re-interlacing :stupid: ... I did as you suggested and the pre-re-interlaced 59.94p frames are as smooth as a baby's bottom. The jaggies/combing were showing up on my 5-year-old LG DVD player + Samsung CRT TV. Perhaps it's normal and I'm looking too hard. Or perhaps, even though they are "multi-system", they are from PAL-land and don't behave the same as a TV from NTSC-land. It's definitely no show-stopper anyway.

Oh, BTW ... if the frameserved output really has a PC-->TV levels squeeze, then that's not good at all ... in particular for underwater shots. (During level squeeze information is getting lost, and you won't get that loss back by expanding again....)
Vegas works internally in RGB. The Debugmode frameserver will serve RGB or YUY2. When frameserved out in RGB there is no levels squeeze. When the frameserver does an RGB>YUY2 conversion it uses Rec601 and so I need to expand the levels out again in AviSynth. Perhaps I'd be better serving in RGB and converting to YUY2 with matrix="PC.601" in AviSynth?

[nhope]

This is where I have ended up and I'm really happy with the results, but didn't test the NTSC>PAL script much since I don't have original SD NTSC footage.

(Note that Vegas works internally in RGB, so I'm using the path of least conversions)

PAL to NTSC

#Frameserve PAL source from Vegas in RGB24 format

AviSource("d:\fs.avi")
AssumeBFF()
ConvertToYUY2(interlaced=true, matrix="PC.601")

QTGMC( SubPel=2 ) #SubPel must match pel in MSuper below
super = MSuper(levels=1, pel=2) #Only need multiple levels for MAnalyse
#super = MSuper(levels=1, pel=2, hpad=16, vpad=16) # For very fast and quicker QTGMC presets
MFlowFps(super, QTGMC_bVec1, QTGMC_fVec1, num=60000, den=1001)
Spline36Resize(720,480)

SeparateFields()
SelectEvery(4,0,3)
weave()

NTSC to PAL

#Frameserve NTSC source from Vegas in RGB24 format

AviSource("d:\fs.avi")
AssumeBFF()
ConvertToYUY2(interlaced=true, matrix="PC.601")

QTGMC( SubPel=2 ) # SubPel must match pel in MSuper below
super = MSuper(levels=1, pel=2) # Only need multiple levels for MAnalyse
#super = MSuper(levels=1, pel=2, hpad=16, vpad=16) # For very fast and quicker QTGMC presets
MFlowFps(super, QTGMC_bVec1, QTGMC_fVec1, num=50, den=1)
Spline36Resize(720,576)

SeparateFields()
SelectEvery(4,0,3)
weave()

1080-50i to SD NTSC

#Frameserve 1080-50i source from Vegas in RGB24 format

AviSource("d:\fs.avi")
AssumeTFF()
ConvertToYUY2(interlaced=true, matrix="PC.601")

Spline36Resize(720,height)
QTGMC("faster", sharpness=0, SubPel=2) #SubPel must match pel in MSuper below
super = MSuper(levels=1, pel=2) # Only need multiple levels for MAnalyse
#super = MSuper(levels=1, pel=2, hpad=16, vpad=16) # For very fast and quicker QTGMC presets
MFlowFps(super, QTGMC_bVec1, QTGMC_fVec1, num=60000, den=1001)
Spline36Resize(width,480)

blur(0.0,1.0)
sharpen(0.0,0.75) #Adjust 2nd value between 0.5 and 1.0 to taste

SeparateFields()
SelectEvery(4,0,3)
Weave()

One question. For which exact QTGMC presets would hpad=16 and vpad=16 be required in these 3 cases?

[-Vit-]

One question. For which exact QTGMC presets would hpad=16 and vpad=16 be required in these 3 cases?
"Very Fast" and quicker presets for all those 3 scripts.

This v/hpad requirement is only there because QTGMC still contains a workaround for a bug in MVTools 2.5.11.1 and earlier, fixed in the latest version 2.5.11.2. I kept the workaround in case people didn't upgrade their MVTools. The next QTGMC version will drop the workaround and will use the default v/hpad in general (and will require the latest version of MVTools).

[kypec]

The next QTGMC version will drop the workaround and will use the default v/hpad in general (and will require the latest version of MVTools).
Mind to tell what new exciting improvements can we look forward to in this next version? :)

[aegisofrime]

Mind to tell what new exciting improvements can we look forward to in this next version? :)

I would like to know that as well. :)

Still, if the next version of QTGMC requires the latest MVTools, I do hope somebody can compile a 64-bit version of that. 64-bit has consistently been more stable and faster than 32-bit for me, although the newest version of NNEDI3 has somewhat closed the gap considerably.

Also, on a slightly OT note, if somebody can post a guide on how we can do our own 64-bit compilations that would be great.

[Dogway]

I observed chroma smears too much. In a similar way mdegrain does when plane=4 with high SAD. Can I disable chroma in some way?

edit: it was MY BAD, cropping before QTGMC, just to save a few 0.xx fps

[-Vit-]

Mind to tell what new exciting improvements can we look forward to in this next version? :)

I didn't mean to suggest a new version was imminent. I have been very busy on a different project, but that has reached a stable point now, so I will be turning my attention back to QTGMC soon.

I have been working on a number of additions. In no particular order:
- Various motion vector tweaks for speed up or quality improvements: use of vectors derived from downsized clip (faster for HD); refinement of vectors through MRecalculate and a new plugin of mine; some vector mask trickery to help with areas of bad match
- Change output frame rate, i.e. use MFlowFPS with the refined vectors noted above. Allows 50i->60p or 60i->50p or whatever.
- Resizing of output, including an attempt to find extra detail from neighbors for upscales.
- Some denoising improvements (related to the above)
- Standalone functions (no deinterlacing): QDenoise, QMotionBlur, QVectors
- Some defaults automatically adjust based on input resolution
- Bug fixes (as ever)...

Most of these are still in progress and I dunno in what order they will appear. Also now would be a good time for requests.
____

...the latest MVTools...
QTGMC will still work with MVTools 2.5.11.1. It's just that version's MCompensate is unstable when there is edge motion on certain resolutions. Doesn't affect most typically used resolutions at all, so it may not affect you.

____

I observed chroma smears too much. In a similar way mdegrain does when plane=4 with high SAD. Can I disable chroma in some way?

src = whatever
luma = src.QTGMC()
chroma = src.YourFavoriteChromaDeinterlacer()
MergeChroma(luma, chroma)

Can't switch off chroma processing entirely in QTGMC, but can speed it up by setting RepChroma=false and ChromaEdi="Bob"

[nhope]

Also now would be a good time for requests.

Well, if you are going to build in options for frame-rate changes (which I think is a great idea), maybe you could build in options for rescaling? Under the covers it would do something like a width-before/height-after Spline36Resize. Or maybe there is other stuff you can do in the script to optimize rescaling quality.

[nhope]

Further to that you could put in shortcut options to effectively make it a format-converter. e.g. "PAL2NTSC", "NTSC2PAL"

[nhope]

Also re-interlacing of double-rate video with choice of field dominance handled as options. e.g. reint="TFF", reint="BFF".

[-Vit-]

Lol, anything else?

PAL<->NTSC:
- I already mentioned that rescaling will be added. My primary interest is high quality upscaling, rather than rescaling for PAL<->NTSC conversion, but I guess any rescaling will be supported.
- Frame-rate change (flow) has always been an obvious add-on since QTGMC already calculates motion vectors (also slow-mo / speeded-up output).
- Reinterlacing is the step that gets no benefit from being inside the QTGMC script, but I guess I'll add it for convenience. I'll provide some alternative approaches.

So although it wasn't explicitly my intent, I expect a future QTGMC to be able to do PAL<->NTSC conversions, and yes I'll make some easy settings for it. It will do InterFrame-like work too. Both these will probably support progressive or interlaced input or output.

Don't ask about ETA though, as I said my main interest is some upscaling experiments, which are only tangentially related to QTGMC. Conversion support will just be a beneficial side effect.

[Robert Martens]

Well, if you are going to build in options for frame-rate changes (which I think is a great idea), maybe you could build in options for rescaling? Under the covers it would do something like a width-before/height-after Spline36Resize. Or maybe there is other stuff you can do in the script to optimize rescaling quality.

Also re-interlacing of double-rate video with choice of field dominance handled as options. e.g. reint="TFF", reint="BFF".

You may have some fun playing with this thing (http://forum.doom9.org/showthread.php?t=153860), a script I've been revising since early last year. No framerate conversions, as I don't feel confident in my ability to do that well, but the width-before/height-after rescaling is already present (for interlaced input that's being downscaled; there's no sense resizing horizontally before deinterlacing if you're upscaling), as is re-interlacing with choice of field order. There's a variety of presets that should make usage a bit easier, taking proper pixel aspect into account to crop the input as necessary for different shaped output, pillar/letter/windowboxing, and some other features that may prove useful for some applications.

Mind you, with no background in math, signal processing, computer science, or any related field (and insufficient intellect to pick up the subjects in less than a lifetime), I've long since reached my limit with regard to quality improvements, so I certainly look forward to whatever Vit cooks up for rescaling.

[nhope]

@-Vit- & Robert Martens - Thanks a lot. I have a ton of interlaced 4:3 PAL DV to upscale to 960x720-25p for YouTube. Was going to just use QTGMC + Lanczos4 on it but I'll check out SimpleSlugUpscale and eagerly await QTGMC upscaling developments. If you want any PAL DV test footage, just say so.

[Yellow_]

If now is a good time for feature requests would there be any benefit in considering gamma aware resizing and 16bit msb/lsb stacked?

[-Vit-]

Adding resizing at all is rather pushing the feature envelope of QTGMC and gamma-aware resizing has never really caught my interest, but I'll make a note to have a look.
16-bit stacked does interest me and is similar in some ways to what I am looking at for resizing. I will look into it, but I expect speed / script complexity problems.

[aegisofrime]

For my part, any quality or/and speed improvement would be welcome. Are they possible though, or have you pushed the coding envelope as far as possible? :)

[kolak]

Done some testing- can run 8 instances until my CPU goes to 100% for real 12 cores. It looks like more optimal is 6 instances- combined speed is bigger than with 8. With 6 instances memory usage is 8.5GB.

Combined speed about 14 fps for 50i HD wit slow presset.

Also tried 64bit- managed to make it working but speed was crap and also it was crashing. Memory usage was jumping to 12GB, but even if I have 24GB it was crashing.


Andrew

[Overdrive80]

@Vit, would be possible use QTMGC for restoring framerate of video PAL to original rate NTSC. This case is for a lot PAL DVDs that I got. Thanks.

With this script obtein PAL video progressive:

QTGMC(preset="fast",FPSDivisor=2)

EDITO: With this script obtein effect ghosting...XS

[Revgen]

^Use SRestore for that.

[Boulder]

Also now would be a good time for requests.Well, a bobbing method for field-blended sources to feed to SRestore (for both detection clip and final result)..I don't know if there is a proper way to bob such sources but there's at least one soft bobber script by Didée which can be used for the detection clip. (some discussion here: http://forum.doom9.org/showthread.php?p=1413769#post1413769) I promised to get you a sample clip but it seems that I forgot that. I'll try to come up with a few good ones to see if there's anything interesting to you.

It would be nice to hear your thoughts regarding MRecalculate. It seems that it is not widely used (and the documentation really doesn't help much, at least didn't do that for me ;)) and I am not sure whether there is anything to gain from utilizing it. The hypothesis is interesting though.

[-Vit-]

MRecalculate is already used for motion blur refinement. In the next version it is an option for the main motion analysis. It can help to more precisely track around the edges of moving objects and hence reduce the blurring that can occur there. However, MRecalulate needs to be used with care as it can easily break good matches elsewhere, especially where there is faster motion. Part of my work to do is to determine best settings and write some custom masking/selection of MRecalculated vectors vs the original vectors.

I recall the discussions on deinterlacing field blended sources. Not sure if the appropriate approach will fit naturally into QTGMC - motion analysis of blended frames is not so effective; but I could have a look.

[bob6779]

Hi everyone. Could somebody help me with this clip. It's a PAL to NTSC transfer which I want to bring back to PAL.

http://www.mediafire.com/?w8axzsq55em8joo

It's my first try with encoding and avisynth and I've managed to clear up some of the shimmering with qtgmc and was able to get good motion playback with Didee's bob script, but I'm unable to remove the strange strobing seen on the roof tiles. Here's the script I'm using.


t = crop( 0, 44, 0, -44)
tgmc = t.QTGMC( Preset="placebo", SourceMatch=3, Lossless=2, Sharpness=0.5, blocksize=16, SLMode=1,SVthin=1.25,truemotion=true, MatchEnhance=0.75,TR2=3,NoiseProcess=2, GrainRestore=1.0, NoiseRestore=0.2, Sigma=2.2, NoiseDeint="Generate", StabilizeNoise=true )


bob1 = t.bob(0,0.5)
bob2 = bob1.clense(reduceflicker=false).merge(bob1,0.5)
bob3 = bob2.mt_adddiff(mt_makediff(bob2,bob2.clense(reduceflicker=false),U=3,V=3),U=3,V=3)

tgmc.vinverse.Srestore(frate=25.0,dclip=bob3)

Are there anymore parameters that I can use to retain more detail and grain or is this the best that I can achieve with this source?

Thanks in advance.

[Didée]

Instead of SLmode=1 (spatial), try SLMode=2 (temporal) with SLrad=3.

It (might) bring up ghosting problems again, but potentially should help with the roof strobings.

[Overdrive80]

Hi everyone. Could somebody help me with this clip. It's a PAL to NTSC transfer which I want to bring back to PAL.


Not recommended to crop after the deinterlacing?

[Mini-Me]

-Vit-, I'm currently experiencing crashes trying to reuse motion vectors from QTGMC. I'm using Avisynth 2.6 Alpha 3, QTGMC 3.32, and MVTools 2.5.11.2 beta.

Here is a pretty simple script that exhibits the behavior. If I load it up and scroll through a few dozen frames, it crashes pretty predictably for me (Virtualdub mentions an Access Violation, and AvsPMod gets a rapidly scrolling error window). If I calculate the motion vectors again myself instead of reusing the ones from QTGMC, no such crash occurs. Note that I'm transcribing my script by eye from another computer, so it might have typos.

orig = Avisource("whatever.avi").AssumeTFF()
deint = orig.QTGMC(preset = "super fast")
deintmcsoft = deint.MTemporalSoften1(QTGMC_bvec1, QTGMC_fvec1)
return deintmcsoft

function MTemporalSoften1(clip base, clip mvbw, clip mvfw)
{
super = base.MSuper(pel = 1)
backward = base.MCompensate(super, mvbw)
forward = base.MCompensate(super, mvfw)
interleaved = Interleave(forward, base, backward)
return interleaved.TemporalSoften(1, 255, 255, 28, 2).SelectEvery(3, 1)
}

(In case anyone's wondering why I'd use MTemporalSoften1 over MDegrain1 for anything, the actual use is to remove heavy flicker from parts of a helper clip in a script I'm writing. :p)

[-Vit-]

Overdrive: As Revgen has said, there are other scripts used for undoing such conversions (together with deinterlacing). I really don't want QTGMC to become some kind of general purpose conversion script, sorry.
____

bob6779: Very complex settings. Probably too complex. There are some interactions there that you likely don't want. For example sharpness limiting (especially temporal) will tend to remove the rather subtle results of source match. SourceMatch=3 is probably overkill. The GrainRestore value looks very high (for NoiseProcess=2) and is probably strengthening grain, although the TR2=3 setting is counteracting that to some degree. Have you tried the more straightforward EZKeepGrain setting together with NoisePreset instead of all those noise settings? That together with source match and sharpness tweaking usually gets most detail.
____

Mini-Me: Your script worked for me on a random video. It might be the hpad/vpad issue. QTGMC sets hpad/vpad in the super clip to work around a bug in MVTools 2.5.11.1 and earlier. The values set depend on resolution and may not be 8, the default. Same values must be used in your subsequent MSuper call. This requirement will be lifted in the next version. You can calculate and use the same values (see QTGMC script at line noted below), or since you are using the latest MVTools you could edit QTGMC itself (don't want to release a new version just for this):
- Around line 340, find 4 lines dealing with hpad/vpad
- Delete them and replace with:

hpad=8
vpad=8


Maybe that will fix your problem. Also watch the pel value used for the super clip, must also match what QTGMC used. It's safer to explicitly set Subpel=X in the QTGMC call to match what you use later since it varies by preset

[Mini-Me]

Mini-Me: Your script worked for me on a random video. It might be the hpad/vpad issue. QTGMC sets hpad/vpad in the super clip to work around a bug in MVTools 2.5.11.1 and earlier. The values set depend on resolution and may not be 8, the default. Same values must be used in your subsequent MSuper call. This requirement will be lifted in the next version. You can calculate and use the same values (see QTGMC script at line noted below), or since you are using the latest MVTools you could edit QTGMC itself (don't want to release a new version just for this):
- Around line 340, find 4 lines dealing with hpad/vpad
- Delete them and replace with:

hpad=8
vpad=8


Maybe that will fix your problem. Also watch the pel value used for the super clip, must also match what QTGMC used. It's safer to explicitly set Subpel=X in the QTGMC call to match what you use later since it varies by preset

Updating the lines in QTGMC did indeed fix the problem, so the crashes must have come from the mismatch between my MSuper and the one used to create the motion vectors. Thank you!

[bob6779]

Instead of SLmode=1 (spatial), try SLMode=2 (temporal) with SLrad=3.

It (might) bring up ghosting problems again, but potentially should help with the roof strobings.

Tried, but it didn't seem to make much of an impact.

Not recommended to crop after the deinterlacing?

Thanks for the tip :).

bob6779: Very complex settings. Probably too complex. There are some interactions there that you likely don't want. For example sharpness limiting (especially temporal) will tend to remove the rather subtle results of source match. SourceMatch=3 is probably overkill. The GrainRestore value looks very high (for NoiseProcess=2) and is probably strengthening grain, although the TR2=3 setting is counteracting that to some degree. Have you tried the more straightforward EZKeepGrain setting together with NoisePreset instead of all those noise settings? That together with source match and sharpness tweaking usually gets most detail.

Thanks for the hints. I'm trying ezkeepgrain and noisepreset now, but it's going too slow. Will get back once it's done.

[rtjnyoface]

Having some difficulty with the speed of QTGMC. I've used QTGMC for awhile now but recently something happened with megui/avisynth or something that affected the two. Anyways, I had to reinstall avisynth and megui and start all over from scratch. This wouldn't be a problem except that QTGMC isn't as fast as it was prior to my problems. I used to get very close to real time (a frame or two less) on the first pass and just slightly over half the fps using QTGMC (an i7 920 helps haha). ALL of my settings are exactly the same now that I've had to reinstall avisynth/megui but I'm getting NOWHERE close to what I was getting. I had to change my current settings to FASTER to get just 15 fps now. So, I'm wondering, am I using an MT version that isn't the same as my last? Is there something else it could be? My directories haven't changed, hardware hasn't changed. I can't think of anything else that's changed and could be making such a significant difference in speed other than my avisynth/wrong plugin versions used for QTGMC . Any help is appreciated :)

Win7
Megui 2028
QTGMC script 3.32
Downloaded and used the modified plugins package via the qtgmc thread
Fully interlaced material kept original fr rate
EXACT same settings (both qtgmc AND x264). I know because I've used them so much and made my own personal changes to the settings. QTGMC settings are basically default settings.
Avisynth 258 MT
My CPU usage seems to fluctuate a bit more using this setup. It used to stay pretty much at 99%. It's odd now though. If I use "FASTER" I get full CPU usage as I used to but if I use "MEDIUM" it hovers around 90. Don't know if that's useful but it seems to me it could be.


edit: would reverting to 3.25 possibly be one/a solution? Also, looking at my program files I do somewhat remember having both 2.5 and 2.6 avs installed.

[Groucho2004]

So, I'm wondering, am I using an MT version that isn't the same as my last?
...
...
Also, looking at my program files I do somewhat remember having both 2.5 and 2.6 avs installed.

We are not psychic so we don't know which version(s) you have installed or which DLL is actually being used.

Just check the file properties of your avisynth.dll in the system32 directory (or wherever it's located on Win7).

Edit: Even better - Create a small script with just this line:
Version()
and run it in your media player and tell us what it displays.

[rtjnyoface]

Alright. I followed the instructions in the modified plugins zip.

Changes:
- I started out getting 16 fps with full cpu usage and then for some reason it gradually went down...and down....and down to now less than 4 and still dropping (is it going to get to 0?). CPU usage at ~15% with a max of 3 threads being used.

What in the world is going on here? Attached is a picture of megui's status window, my cpu usage, and a picture of my trash can to show which files were deleted after following the instructions in the zip file. I just replaced what was there (even if it was the same file)


Whew....hope we can clear this up.

[rtjnyoface]

I've already listed the dll version. 2585

Ran the script. 2.58 tsp MT version 5 (mod seraphy)

We are not psychic so we don't know which version(s) you have installed or which DLL is actually being used.
Of course you don't know. I just thought that since there are so many diff versions that one worked better with qtgmc and I didn't know...


http://img69.imageshack.us/img69/9583/activityw.jpg (http://imageshack.us/photo/my-images/69/activityw.jpg/)

Uploaded with ImageShack.us (http://imageshack.us)

[Groucho2004]

Attachments take days to be approved here. Upload your screen shots somewhere else.

[Groucho2004]

I started out getting 16 fps with full cpu usage and then for some reason it gradually went down...and down...

Ha, that's what I experienced with MT versions. Some people have no problems with MT Avisynth, others (like me) never get it to work properly.

Anyway, it would help if you posted your script and the encoder settings.

[rtjnyoface]

# Set DAR in encoder to 4 : 3. The following line is for automatic signalling
global MeGUI_darx = 4
global MeGUI_dary = 3
SetMTMode(5,8)
LoadPlugin("C:\Program Files\Megui\tools\dgindex\DGDecode.dll")
DGDecode_mpeg2source("D:\FILE LOCATION\VTS_01_1.d2v", info=3)
LoadPlugin("C:\Program Files\Megui\tools\avisynth_plugin\ColorMatrix.dll")
ColorMatrix(hints=true, threads=0)
SetMTMode(2)
QTGMC( Preset="Medium" )
SelectEven()
#deinterlace
crop( 8, 0, -6, 0)
Lanczos4Resize(704,480) # Lanczos4 (Sharp)
#denoise


Um...I don't know how to access ALL of the x264 settings in one small place (unless you can tell me) so I copied the short megui window.

program --preset slower --pass 2 --bitrate 1390 --stats ".stats" --subme 8 --output "output" "input"

I actually use subme 9. Just changed it for the testing I was doing. Ha, that basically give you nothing lol. Sorry

[Groucho2004]

The script looks OK but are you sure that you are using the correct DLLs for QTGMC?

I see in your screen shot a plugin directory called "2.6x Plugins" which is most likely a leftover from your 2.6 installation.

Also, Avisynth 2.5x needs a different version of masktools than 2.6x.

Just make sure that the correct plugins are located in the correct plugin directory.

Edit - It looks like you have about 4 different locations with plugins. Very messy.

[rtjnyoface]

are using the correct DLLs for QTGMC? Yes, I am now lol ;). I accidentally added that folder from the qtgmc zip. I just deleted it as you saw.

Also, Avisynth 2.5x needs a different version of masktools than 2.6x. mt_masktools-25.dll is in the plugin directory.

Just make sure that the correct plugins are located in the correct plugin directory. Tis why I cleaned out the folder and started over with the qtgmc zip.


Ha, that's what I experienced with MT versions. Some people have no problems with MT Avisynth, others (like me) never get it to work properly. And what's got me is that I had it working just the way I wanted it to.


4 Different locations? How do you figure that? The picture was from my trash can. All the plugins and such are in the avs folder and the system 32 folder (32b here). If I've got it wrong please let me know. Be the first thing to correct...

[Groucho2004]

mt_masktools-25.dll is in the plugin directory.
Which one?


4 Different locations? How do you figure that?

OK, at least three:
C:\Program Files\Megui\tools\dgindex
C:\Program Files\Megui\tools\avisynth_plugin
<your actual Avisynth plugin folder as referenced in the registry> -> [HKEY_LOCAL_MACHINE\SOFTWARE\AviSynth]

[rtjnyoface]

Which one? 2.0.48

OK, at least three: Sorry, I thought megui just referenced the current avs installed in the system. If I remember correctly it didn't used to come with avs and would say so upon installation. I'll uninstall the 2.5 and just go with megui's. Lastly, what's wrong with dgindex? That's the only location of dgindex afaik. Just checked again. Yep, only location.


edit: man, that megui avs directory was a horrible overlook on my part. Geez...

[Groucho2004]

2.0.48
I meant which plugin directory.

Sorry, I thought megui just referenced the current avs installed in the system. If I remember correctly it didn't used to come with avs and would say so upon installation. I'll uninstall the 2.5 and just go with megui's. Lastly, what's wrong with dgindex? That's the only location of dgindex afaik. Just checked again. Yep, only location.

I don't know what megui does. I do know that it doesn't install Avisynth so you have to do that yourself.
I was just pointing out that source filters and plugins are loaded from various locations in your script which can be error-prone.

You either load plugins by specifying the path or you use the autoload function. In your case it seems a bit blurry as to what is loaded from where.

[Boulder]

Is MeGUI one of those applications in which you have to add Distributor() as the last line in the script to get MT working?

[Groucho2004]

Is MeGUI one of those applications in which you have to add Distributor() as the last line in the script to get MT working?

x264 should take care of that.

[Boulder]

I'm not so sure about that, it's more about the way MeGUI and Avisynth discuss with each other. If MeGUI uses the VfW interface, then Distributor() is not needed because it is automatically called. If it talks to Avisynth directly, the call is needed.

[Groucho2004]

I'm not so sure about that, it's more about the way MeGUI and Avisynth discuss with each other. If MeGUI uses the VfW interface, then Distributor() is not needed because it is automatically called. If it talks to Avisynth directly, the call is needed.

I'm quite sure that the script is fed directly into x264. Therefore, x264 invokes the "Distributor" call if necessary:

if( !strcasecmp( filename_ext, "avs" ) )
{
res = h->func.avs_invoke( h->env, "Import", arg, NULL );
FAIL_IF_ERROR( avs_is_error( res ), "%s\n", avs_as_string( res ) )
/* check if the user is using a multi-threaded script and apply distributor if necessary.
adapted from avisynth's vfw interface */
AVS_Value mt_test = h->func.avs_invoke( h->env, "GetMTMode", avs_new_value_bool( 0 ), NULL );
int mt_mode = avs_is_int( mt_test ) ? avs_as_int( mt_test ) : 0;
h->func.avs_release_value( mt_test );
if( mt_mode > 0 && mt_mode < 5 )
{
AVS_Value temp = h->func.avs_invoke( h->env, "Distributor", res, NULL );
h->func.avs_release_value( res );
res = temp;
}
}

[Boulder]

Hehe, that's clever :)

[-Vit-]

Although rtjnyoface mentioned a "first pass". It's not clear, but if he is using a pre-render job in MeGUI then the avisynth script is fed to ffmpeg or Mencoder (depending how old a version of MeGUI) for a lossless intermediate output, not into x264. So perhaps the question is do ffmpeg/Mencoder need a Distributor call...?

[Undead Sega]

Hey guys, I know this may seem abit off topic, but I really enjoy using QTGMC for the quality is gives as a result but it's just the speed that bothers abit, thus I was wondering, how do you use AviSynth MT? and in conjuction with QTGMC? I'm really confused how to use it or where to begin with it.

I hope you don't mind me asking about this, but QTGMC really does help on alot of things (probably better than some hardware deinterlacers for broadcast?) but I just wish to get things done qucker, I'm sure al of you do too :)

[poisondeathray]

Hey guys, I know this may seem abit off topic, but I really enjoy using QTGMC for the quality is gives as a result but it's just the speed that bothers abit, thus I was wondering, how do you use AviSynth MT? and in conjuction with QTGMC? I'm really confused how to use it or where to begin with it.

I hope you don't mind me asking about this, but QTGMC really does help on alot of things (probably better than some hardware deinterlacers for broadcast?) but I just wish to get things done qucker, I'm sure al of you do too :)

Did you look at the MT instructions in the 1st post :)

[rtjnyoface]

Woo hoo... now I get the "no function named mt_makediff" error. Replaced masktools with all different kinds of versions hoping to fix this error (this error was mentioned in THIS (http://forum.doom9.org/archive/index.php/t-128993.html) post but the solution provided there didn't fix my issue. Nothing worked. At least I had QTGMC working last night lol.

edit: fixed this by placing mt_masktools-25.dll DIRECTLY in the plugins folder instead of the 2.5 folder

edit 2: ho-ly s@%#. For all the changes and all the crap I've gone through I'm pretty sure I've got it working now. Thanks for the help folks. I don't know for sure if it were you or me but it seems to be "fixed" for the time being. I'm back to doing 30 fps on "medium".

[Groucho2004]

I don't know for sure if it were you or me but it seems to be "fixed" for the time being.

Neither. Have you never heard of the Folder Fairy that creeps around in your hard drives when you are sleeping and sorts out the mess you (or evil software) left during the day? :)

Anyway, glad you got it sorted.

[bob6779]

bob6779: Very complex settings. Probably too complex. There are some interactions there that you likely don't want. For example sharpness limiting (especially temporal) will tend to remove the rather subtle results of source match. SourceMatch=3 is probably overkill. The GrainRestore value looks very high (for NoiseProcess=2) and is probably strengthening grain, although the TR2=3 setting is counteracting that to some degree. Have you tried the more straightforward EZKeepGrain setting together with NoisePreset instead of all those noise settings? That together with source match and sharpness tweaking usually gets most detail.


The simplified presets helped, but I've still not been able to get rid of the strobing :(.

Also another thing that I have noted is that when I use MT any horizontal panning results in stuttering, but smooth when I remove MT.



SetMemoryMax(512)
SetMTMode(5, 4)
Import("F:\Program Files\AviSynth 2.5\plugins\R_pack\srestore.avsi")
Import("F:\Program Files\AviSynth 2.5\plugins\QTGMC-3.32.avsi")
Import("F:\Program Files\AviSynth 2.5\plugins\GrainFactory3.avsi")

LoadPlugin("F:\Program Files\AviSynth 2.5\plugins\mt_masktools-25.dll")

LoadPlugin("F:\Program Files\AviSynth 2.5\plugins\mvtools2.dll")


s=DGDecode_mpeg2source("F:\Documents and Settings\Raapakal DTS_Track1strobe.d2v", cpu=0,info=0)
SetMTMode(2)
tgmc = s.QTGMC( Preset="placebo", Lossless=2, EZKeepGrain=1, NoisePreset="faster", SourceMatch=2, Sharpness=0.3, EdiThreads=1 )
bob1 = s.bob(0,0.5)
bob2 = bob1.clense(reduceflicker=false).merge(bob1,0.5)
bob3 = bob2.mt_adddiff(mt_makediff(bob2,bob2.clense(reduceflicker=false),U=3,V=3),U=3,V=3)
tgmc.vinverse.Srestore(frate=25.0,dclip=bob3)
crop( 0, 44, 0, -44)
Grainfactory3(2,2,2)

[Undead Sega]

Did you look at the MT instructions in the 1st post :)

Indeed I have, however I just dont know how to use AvisYnth MT, I mean I would download it and I wouldnt know exactly what to do afterwards.

Even with that said, I wouldnt know how to use all my current filters in MT :(

[aegisofrime]

Indeed I have, however I just dont know how to use AvisYnth MT, I mean I would download it and I wouldnt know exactly what to do afterwards.

Even with that said, I wouldnt know how to use all my current filters in MT :(

MT mode can be toggled on and off. Installing the MT version of Avisynth does not mean that the behavior of your filters will change, if stability is what you are concerned about.

[Undead Sega]

The stability isnt wat I was referring to, I'm pretty sure it would be stable throughout and that the sped would improve dramatically?

I was merely talking about if I ever knew how to install or use AviSynth MT, I wouldnt know how to use all my current filters in it. I hope you get what I mean this time :)

[-Vit-]

It is very simple. Just copy the MT version of the avisynth.dll over your existing avisynth.dll. Then you can put SetMTMode, GetMTMode and Distributor in your scripts, as described in the first post of this thread. You don't have to use these multithreading features, in which case avisynth just behaves as normal.

You can also get the MT.dll if you want to use the MT("") filter, but that's a different matter and not very useful for QTGMC.

[Lighto]

It will certainly works and can be effective sometimes - depends on the source and settings. It's not as fast as using SetMTMode - having your CPU under full load does not necessarily mean it's working efficiently.

Have you tried the various ideas from the first post? Particularly different versions of Avisynth MT and my modded plugins? Also try running MT at less than full load, that can help whilst still being faster.

____

On a related point, I recommend updating to nnedi3 0.94 (http://forum.doom9.org/showthread.php?t=147695), it gives a nice speed-up (thanks tritical!).

Thanks for the tips.
Anyway, I got MT to run stable by setting fixed voltage to my CPU via my BIOS settings.

[Gavino]

I got MT to run stable by setting fixed voltage to my CPU via my BIOS settings.
That's a new one.
Perhaps -Vit- should add it to the MT tips in the QTGMC docs. :)

[Didée]

Note: Windows booting successfully is not a sufficient proof that your overclocking setup is running stable. :p

[Groucho2004]

That's a new one.
Perhaps -Vit- should add it to the MT tips in the QTGMC docs. :)

I agree. There should be a 'masochist' section for people who use Avisynth MT with an overclocked computer (and a very complex script with tons of plugins). Just imagine the number of possibilities for failure. :rolleyes:

[Lighto]

That's a new one.
Perhaps -Vit- should add it to the MT tips in the QTGMC docs. :)

New "tip", get Avisynth 2.6 (SEt Version 2011.07.14) with Standard 2.57MT's MT.dll.
Less crashing for me.

May be placebo effect.

[Mini-Me]

Note: Windows booting successfully is not a sufficient proof that your overclocking setup is running stable. :p

If it posts, I'd say it's good enough for anything up to and including life support systems. Then again, video processing is a bit more precious than mundane use cases like that, so...maybe a bit of Prime95 is called for in this case. ;)

__________

Seriously though, I actually do have a question for anyone who knows Avisynth better than I do (most of you):

If I use Quantile (from Kassandro's RemoveGrainHD package) on a clip that has gone through QTGMC, I often get unexpected/garbage results, like black frames, white frames, frames that are black/white in the right half but correct on the left, etc. The problem affects "random" frames, but it appears to be deterministic (just triggered by unknown factors).

My best guess is that Quantile has a bug in it somewhere, but I should ask: Is it possible that QTGMC's component filters are outputting frames that are visibly correct but badly formatted (thereby screwing up Quantile)? That seems unlikely to me, but something about the output - or maybe the complexity and processing time - seems to be making Quantile go haywire. This occurs in both Avisynth 2.5.8.5 and the 2.6 Alpha 3.

Depending on the source, I can get screwy results from a test script like:
orig = Avisource("HomeMovie.avi", pixel_type = "YUY2")
deint = orig.ConvertToYV12(interlaced = True).QTGMC(preset = "super fast")
return deint.Quantile(radius = 3, rank = 1)

[-Vit-]

Have you tried outputing the QTGMC result to a lossless file then running Quantile on that? That should determine if it is Quantile or the complexity of processing. [This is single-threaded, right?]

[henryho_hk]

If it posts, I'd say it's good enough for anything up to and including life support systems.

From my experience, multi-QTGMC() stress a system far better than multi-Prime95. :devil:

[Mini-Me]

Have you tried outputing the QTGMC result to a lossless file then running Quantile on that? That should determine if it is Quantile or the complexity of processing. [This is single-threaded, right?]

Wow, I can't believe I didn't think of that. Anyway, even if I deinterlace, output to file, and then load it back in, Quantile still barfs. That definitely rules out processing issues...and since any encoded value for the image should be a valid one, it looks like Quantile is indeed the problem. Thanks :) I posted a thread on Kassandro's board, but I don't think he checks it very often...

[Undead Sega]

It is very simple. Just copy the MT version of the avisynth.dll over your existing avisynth.dll. Then you can put SetMTMode, GetMTMode and Distributor in your scripts, as described in the first post of this thread. You don't have to use these multithreading features, in which case avisynth just behaves as normal.

You can also get the MT.dll if you want to use the MT("") filter, but that's a different matter and not very useful for QTGMC.

Oh right, that's great, I'll give that a go. Which version should I go for?

Also, is it recommended to use the 64bit version of Avisynth if one's PC can accomodate it?

[-Vit-]

I'll give that a go. Which version should I go for?
I would suggest SEt's 2.6MT (http://forum.doom9.org/showthread.php?t=148782) as SEt has recently updated it with a multithreading fix. It has been reported as more stable, certainly works fine for me. If you get stability problems try the other versions.

[SubJunk]

Wow, I hadn't noticed he updated it. I never thought I would see the day *tear*

[Undead Sega]

I would suggest SEt's 2.6MT (http://forum.doom9.org/showthread.php?t=148782) as SEt has recently updated it with a multithreading fix. It has been reported as more stable, certainly works fine for me. If you get stability problems try the other versions.

Oh okay then, so is his download links all Avisynth MT?

Should I also upgrade my current Avisynth to 2.6 from 2.5.8?

[-Vit-]

Get only the latest dated version. You are effectively upgrading by copying SEt's avisynth.dll.

[Undead Sega]

So I would upgrade my current Avisynth by just copying SEt's avisynth.dll? Which is at the same time an MT version? :D

[-Vit-]

You learn quickly

[Undead Sega]

Excellent.

Would I probably see a huge difference in speed? especially in regards to the hardware/specs I have of my PC?

[Br4twurscht]

Hi,

I have problem with a video. I hadn't interlaced source not very often yet.

I'm not sure how I can deinterlace the following source:
edit: entfernt

It's a clip of the docu How the Universe Works (http://en.wikipedia.org/wiki/How_the_Universe_Works). I'm from germany so my version is in german ("Das Universum - Eine Reise durch Raum und Zeit").

But thats everything I can safely say about this clip. MediaInfo says it is 25 fps, but I think its 50 fps. I tried both (assumefps).
I use StaxRip with ffVideoSource. If i look then at the raw video-stream i can see that the frames are jumping.
I tried to solve the problem with ivtc, so (if I merge all avisynth-plugin entries) this is my script:

FFVideoSource("%source_file%", cachefile="%working_dir%%source_name%.ffindex")
AssumeFPS(50.000)
Crop(0,0, -Width % 8,-Height % 8)
ConvertToYV12()
AssumeTFF()
Telecide(guide=2, post=0)
Decimate(cycle=2)
QTGMC( Preset="Fast" )
SelectEven()
Crop(%crop_left%,%crop_top%,-%crop_right%,-%crop_bottom%)
LanczosResize(%target_width%,%target_height%)

The problem is, that the result is not smooth and it seems that there are some standing frames (maybe blending???).
Another strange thing is, that MediaInfo says it would be NTSC. NTSC with 25fps??? Maybe a standard conversion of a stupid encoding guy?!?

Till now I helped myself with using DirectShowSource. So the Source is reading with ffdshow. Then the frames are right and i can deinterlace it simply with QTGMC.
But i think the quality would be better if would use ffvideosource. Is it right? And how i can solve my problem?

regards
Br4twurscht

[Didée]

It's a fieldblended normconversion. HDTV/BluRay does not safe you from crappy sources. (And noone has been 'stupid, it's the "standard" result when a conversion box changes from NTSC to PAL.) Your script is bad anyway - first TIVTC then bob-deinterlacing does not make sense. ;)

Look out for the "Srestore" script, then use sth like

bob(0,0.5) # or yadi, or tdeint, ...
SRestore()

or

QTGMC(preset="fast")
Srestore(dclip=bob())

[Br4twurscht]

Thank you, I will try it soon :)

[Undead Sega]

Okey dokey, I have copied over the dll thus replacing the original, now I was reading on the MT usage of this script but I couldnt quite work out how to use SetMemoryMax and SetMTMode, like what parameters to put in. So, if I have a Q9550 (4 Core) processor and 8GB of DDR2-800 Ram, how will I make use of that exactly?

[Groucho2004]

Okey dokey, I have copied over the dll thus replacing the original, now I was reading on the MT usage of this script but I couldnt quite work out how to use SetMemoryMax and SetMTMode, like what parameters to put in. So, if I have a Q9550 (4 Core) processor and 8GB of DDR2-800 Ram, how will I make use of that exactly?

A simple search for these terms ("SetMemoryMax" "SetMTMode") would result in a huge number of posts where other users report their experiences with Avisynth MT and its usage. There are also instructions in the very first post of the thread you are reading right now.

After 4 years and more than 650 posts one would think that you figured out how things work here. I can only assume that you're too lazy to search and prefer things to be spoon-fed to you.

[johnmeyer]

... I couldn't quite work out how to use SetMemoryMax and SetMTMode, like what parameters to put in. So, if I have a Q9550 (4 Core) processor and 8GB of DDR2-800 Ram, how will I make use of that exactly?As already mentioned your question is easily answered by reading the SetMTMode section of the documentation. That documentation gives very good instructions on how to "tune" the multi-threading settings for your situation.

I found that the version of AVISynth used makes a BIG difference in the stability of multi-threading with QTGMC. Also, you need to make sure you get the correct versions of all the plugins. Finally, if you are serving the results into something else -- in my case, this is often MeGUI -- then you may find that you have to adjust downwards the number of threads allocated to AVISynth multithreading. Here is the relevant MT section of a script I often use when using QTGMC and then opening the AVS file with MeGUI (I have a four-core/eight thread i7 computer):

SetMTMode(5,4)
AVISource("e:\frameserver.avi")
SetMTMode(2)

[Xaurus]

Hello, is there a guide or tutorial somewhere how to use this with media player classic home cinema?

[SubJunk]

It would be too slow. This isn't for realtime processing.

[-Vit-]

It can be done if you have a reasonably recent machine. QTGMC isn't designed for realtime viewing, but the faster presets will go realtime easily on SD material with multithreading.

Regarding MPC-HC, I can't get recent builds to work with avisynth scripts, though that might be a problem at my end - I haven't looked at the issue much. The old 1.3.1249.0 version works fine for me. Just drag'n'drop an avs file in there and it plays. So to answer the question directly: learn to write basic avisynth scripts, then make a multithreaded QTGMC script as described in the first post of this thread, drag it into MPC-HC (maybe an older build). Start with preset "Super Fast" and see if your machine can handle it...

[aegisofrime]

It would be too slow. This isn't for realtime processing.

It can be done if you have a reasonably recent machine. QTGMC isn't designed for realtime viewing, but the faster presets will go realtime easily on SD material with multithreading.

Regarding MPC-HC, I can't get recent builds to work with avisynth scripts, though that might be a problem at my end - I haven't looked at the issue much. The old 1.3.1249.0 version works fine for me. Just drag'n'drop an avs file in there and it plays. So to answer the question directly: learn to write basic avisynth scripts, then make a multithreaded QTGMC script as described in the first post of this thread, drag it into MPC-HC (maybe an older build). Start with preset "Super Fast" and see if your machine can handle it...

On my undervolted, stock clock i7-2600K preset Fast runs perfectly fine on DVD content. So I won't make a sweeping statement like that SubJunk. It depends on the content, your PC specs and preset used, like -Vit- said.

-Vit-, drag and dropping the .avs file works on my end. Build 1.5.2.3329 supplied as part of K-Lite Mega Codec Pack.

[Xaurus]

Well, I would assume a 6-core 4.0 Ghz. Intel 980x would suffice?

In any case, I have some issues with proper deinterlacing some 1080i videos.

I thought that this might do the trick, but it's so complex that
I'll skip it. I tried the YADIF implementation in FFDSHOW but
that gave me other problems, like dropped frames etc.

Thanks anyway. :thanks:

[SubJunk]

I sit corrected :)

[Undead Sega]

A simple search for these terms ("SetMemoryMax" "SetMTMode") would result in a huge number of posts where other users report their experiences with Avisynth MT and its usage. There are also instructions in the very first post of the thread you are reading right now.

After 4 years and more than 650 posts one would think that you figured out how things work here. I can only assume that you're too lazy to search and prefer things to be spoon-fed to you.

Oh no, absolutely not 'lazy' at all, I have done stuff that I've figured out myself but its just sometimes I have a hard time reading and understanding what is being said, thus not very clear to me which is why I just had to ask here, and that I dont know how some of the parameters I would have to enter corrospond to my system or if I'm doing it right :(

As already mentioned your question is easily answered by reading the SetMTMode section of the documentation. That documentation gives very good instructions on how to "tune" the multi-threading settings for your situation.

I found that the version of AVISynth used makes a BIG difference in the stability of multi-threading with QTGMC. Also, you need to make sure you get the correct versions of all the plugins. Finally, if you are serving the results into something else -- in my case, this is often MeGUI -- then you may find that you have to adjust downwards the number of threads allocated to AVISynth multithreading. Here is the relevant MT section of a script I often use when using QTGMC and then opening the AVS file with MeGUI (I have a four-core/eight thread i7 computer):

SetMTMode(5,4)
AVISource("e:\frameserver.avi")
SetMTMode(2)

I see, well at the moment I'#m just taking stuff to VirtualDub and save them as a lossless avi which then I take into MeGUI, I find that its more convenient for me this way. Before seeing your post, I was using:

SetMemoryMax(800)
SetMTmode(5,4)
mpeg2Source("D:\xxx.d2v")
AssumeTFF()
QTGMC(Preset="Placebo", TR2=3, EdiMode="NNEDI3", ChromaEdi="NNEDI3", NNeurons=4, EdiQual=2)

...on a SD source, but I found (speed-wise) it hasnt really improved at all :( Maybe I'm doing something wrong here?

[Boulder]

You don't have the SetMTMode(2) call after the MPEG2Source line so it's single threaded (except for the internal multithreading that QTGMC uses for NNEDI3).

[Undead Sega]

Alrighty, I gave it a quick shot and it seems alittle faster when previewing it, now I'm doing the encoding and seems alittle faster as well, what does SetMTMode(2) do exactly? Can this (and my SetMTMode() script) be applied to other filters as well?

[johnmeyer]

Before seeing your post, I was using:

SetMemoryMax(800)
SetMTmode(5,4)
mpeg2Source("D:\xxx.d2v")
AssumeTFF()
QTGMC(Preset="Placebo", TR2=3, EdiMode="NNEDI3", ChromaEdi="NNEDI3", NNeurons=4, EdiQual=2)

...on a SD source, but I found (speed-wise) it hasnt really improved at all :( Maybe I'm doing something wrong here?Where's the second SetMTMode? Have you read the documentation?

Multithreading functions (http://avisynth.org/mediawiki/Internal_functions/Multithreading_functions)

[Didée]

SetMemoryMax(800)
SetMTmode(5,4)
mpeg2Source("D:\xxx.d2v")
AssumeTFF()
SetMTmode(2)
QTGMC(Preset="Placebo", TR2=3, EdiMode="NNEDI3", ChromaEdi="NNEDI3", NNeurons=4, EdiQual=2)
Like so.

(However, to get into the swing of things, I wouldn't use 'placebo' of all presets ...)

[aegisofrime]

SetMemoryMax(800)
SetMTmode(5,4)
mpeg2Source("D:\xxx.d2v")
AssumeTFF()
SetMTmode(2)
QTGMC(Preset="Placebo", TR2=3, EdiMode="NNEDI3", ChromaEdi="NNEDI3", NNeurons=4, EdiQual=2)
Like so.

(However, to get into the swing of things, I wouldn't use 'placebo' of all presets ...)

Why not?

[-Vit-]

On speed: Placebo settings are less effective multi-threaded. Often with more extreme scripts you get <100% CPU usage, which affects processing speed more than expected. Some kind of bottleneck in the processing. Presets "Slower" and "Slow" are very well tuned for multi-threading and give great results.
____

On placebo: I would like to tweak the Placebo and Very Slow settings. They're perfectly usable (not as placebo as they could be really), but some of the choices are... particular, suiting some sources more than others. Most notably those presets set TR2 > 1, whereas that setting should better be tweaked on a per-source basis. So if you must use "Very Slow" or "Placebo" I suggest you at least tweak TR2 to taste.

I have considered adding source-match settings to "Very Slow" & "Placebo", since they create a result measurably more accurate to the source. However, the original rationale for TGMC was to aim for stability and compressibility (i.e no bob-shimmer & temporally smoothed), not absolute accuracy. Source-match doesn't introduce shimmer but it does recover more source detail including any noise/artefacts present. This affects compressibility, and in the case of a lower quality source recovering source artefacts might not be considered an improvement. Also default (Q)TGMC enhances/sharpens the image in its effort to counteract the temporal smoothing. It can overdo this in some circumstances leading to a slightly sharper/enhanced output. Source match will not oversharpen and is truer to the source sharpness, but regardless of accuracy, less sharpness might not be considered an improvement.

So there is a fundamental question about what constitutes an "improvement" in settings, and consequently what should be used for Placebo. Some things are obvious, such as wider, more precise searches for nnedi & MVTools, but these make less of a difference. Other settings that undoubtedly improve accuracy, such as noise-bypass and source-match, may not actually suit people's personal taste / expectations...

I'm of the opinion that the so-called "best" settings should strive for absolute transparency - a shimmer-free, combing-free deinterlace that is otherwise as exact to the source as possible. The nearest you can get to the theoretical progressive source. But that's not what people are used to from (Q)TGMC, they expect smoothing and enhancement too.

[Boulder]

Maybe you should create two basic scenarios, one for the original purpose of TGMC and the other for the enhanced accuracy. Then both of them could have their own presets, and presets make the function much easier to use (due to the huge amount of parameters to adjust).

[aegisofrime]

So in other words, its not for quality reasons that Didee says not to use Placebo, but rather they don't multi-thread effectively?

[Mini-Me]

Maybe you should create two basic scenarios, one for the original purpose of TGMC and the other for the enhanced accuracy. Then both of them could have their own presets, and presets make the function much easier to use (due to the huge amount of parameters to adjust).

Going with what Boulder said, you could borrow x264's tune system: Create different tunings (e.g. "transparency" vs. "enhancement") that modify the settings. Your tuning system would probably be a lot more complex than x264's, since the tuning value would change different things based on the preset...but it's an option. I'm not sure how many more parameters QTGMC can take before causing cosmic rifts though. ;)

So in other words, its not for quality reasons that Didee says not to use Placebo, but rather they don't multi-thread effectively?

The "problem" with placebo is not quality-related; the output is beautiful, especially with SourceMatch, etc. However, the output of most other presets is pretty beautiful too. Placebo is extremely slow (and not just because of multithreading effectiveness), and you might be surprised at how good the faster presets look. Even the super fast preset produces far better results than most deinterlacers. I think Didee was just suggesting that you try a faster preset while you're first testing out QTGMC, and then you can compare settings and see if placebo is really worth the time investment for you.

[aegisofrime]

The "problem" with placebo is not quality-related; the output is beautiful, especially with SourceMatch, etc. However, it's extremely slow, and you might be surprised at how good the faster presets look. I think Didee was just suggesting that you try a faster preset while you're first testing out QTGMC, and then you can compare settings and see if placebo is really worth the time investment for you.

I have been using QTGMC and it's predecessor TGMC for about two years now. I agree that the slower presets don't really bring about a huge improvement in quality. It's a case of diminishing returns I guess. However, for some important videos that I like I tend to just "to hell with it" and go Placebo. My thinking is that might as well do it once and do it good. Other than that, yes Slower does a great job most of the time. I was wondering from Didee's somewhat cryptic comment if he had found a bug with Placebo or something. :p

[Mini-Me]

I have been using QTGMC and it's predecessor TGMC for about two years now. I agree that the slower presets don't really bring about a huge improvement in quality. It's a case of diminishing returns I guess. However, for some important videos that I like I tend to just "to hell with it" and go Placebo. My thinking is that might as well do it once and do it good. Other than that, yes Slower does a great job most of the time. I was wondering from Didee's somewhat cryptic comment if he had found a bug with Placebo or something. :p

Ah, my bad. I should have read more closely. :o I'm the same way with important videos, too...sometimes, I feel I'd regret giving them anything less than the best.

[-Vit-]

I always assumed that a Placebo preset was so-named because it uses maximal settings that make you feel better but don't really doing anything much more than normal settings. It's supposed to be an exemplar of the law of diminishing returns. I suspect Didée was merely suggesting to start at a more standard setting since it would almost certainly be good enough. Although it's possible he intended to critique specific settings choices, as I did above.

Actually, sometimes there is very good reason to use settings beyond the defaults: when you have a low resolution but high quality source, which you expect to upscale considerably. For example, a DVD that has been letter-boxed to 16:9, giving an effective source resolution of 720x360 say. If you upscale that to a big screen, you're going to see the pixels in even more detail than usual. Source match and/or other slower settings are more readily visible and justifiable in this situation. My experience is that [given good source quality] resolution is the most important factor for selecting appropriate preset/settings for QTGMC (and some encoding settings too). Which works the other way of course, for HD material the default "Slower" is overkill. The current development version (yes, I am working on it, just very, very slowly...) uses the input resolution to determine some internal settings. But I am also very tempted to set a faster default preset for higher resolution sources...

And yes, the slower presets/settings create improvements that the ordinary viewer just won't notice. Yet I think such settings are important for script/plugin development: a single improvement of say 1% quality is negligible, but accumulate a sequence of such improvements and we get real progression over the long term. However, for day-to-day usage there's little practical purpose in squeezing the final 1% quality out of today's scripts if it means doubling your processing time. Instead keep the original, and re-encode in a couple of years when the scripts are [might be] 10% better.

[kolak]

It's very true about resolution. I have many good quality sources (sometimes also very bad) and when you upscale PAL to HD you can see big difference if deinterlacing was done properly. When you start with HD source there's almost no difference. It would matter if you would upscale to 4K, but when going down it almost makes no difference (which makes perfect sense).

I had to upscale some PAL interviews recently- just quite good, uncompressed source. I've noticed that there is difference even between medium and slow preset (in the same time YADIF looked way worse). To make it more interesting our Kuro plasma does (for my eyes) even better job. All Pioneer plasmas have very good processing.

I'm still struggling to get decent speed with QTGMC thought :(


Andrew

[Mini-Me]

With my 480i sources, I can flip between interleaved clips and see an immense increase in vertical detail with slower presets, SourceMatch, etc. Between the slowest presets and "super fast", the difference in detail looks similar to a 2x vertical resolution increase (and even "super fast" totally slaughters deinterlacers like Yadif in overall quality). It's definitely there, and the more I magnify the image, the more I see the difference. I can only imagine the difference with a projection setup.

My ideal settings so far are (preset = "placebo", SourceMatch = 3, MatchPreset = "placebo", MatchPreset2 = "placebo", lossless = 1, NoiseProcess = 1, NoisePreset = "slower", DenoiseMC = True, NoiseDeint = "Generate", StabilizeNoise = False, GrainRestore = 0.0, NoiseRestore = 1.0). StabilizeNoise = True looks even better, but that breaks losslessness. ;) Those settings aren't exactly feasible for my setup today, but as you say, that's why keeping the originals is so important...I can still get excellent results today with faster presets and then upgrade the processing when I have a couple orders of magnitude more horsepower. ;)

[zerowalker]

i have tried finding it myself with no success, so i will try to ask here;D
is there a way for QTGMC to work on 64 bit (without failing of course), with mt and etc, for good speed?
if so, can you help me get it to work, i have found every plugin except NNEDI3 (which i found, but it doesn´t work i think), maybe that isn´t the problem, but still.

PS: Can you please tell me wich Avisynth i should use also, as there are so many versions:)

[aegisofrime]

i have tried finding it myself with no success, so i will try to ask here;D
is there a way for QTGMC to work on 64 bit (without failing of course), with mt and etc, for good speed?
if so, can you help me get it to work, i have found every plugin except NNEDI3 (which i found, but it doesn´t work i think), maybe that isn´t the problem, but still.

PS: Can you please tell me wich Avisynth i should use also, as there are so many versions:)

You need to use NNEDI3 0.9.2, as for 0.9.4 there isn't a 64-bit build yet. Although I hope there will be, since 0.9.4 seems to bring about enough performance improvements to make QTGMC 32-bit almost as fast as 64-bit (although still less stable than 64-bit)

Link here: http://bengal.missouri.edu/~kes25c/nnedi3_64.dll

[zerowalker]

what tool do you use for AVS to AVI?;O
Thanks for the link:D!

EDIT: What settings in the script will make it more stable?
i want to use placebo, and it´s a very long video, 4 hours, maybe you can set it to clip video every 1 hour or something;D?

EDIT 2: Got Quad core btw, 8gb ram

[SubPixie]

I recently discovered the motion blur capabilities of QTGMC and thought the output was amazing calling the script this way :

QTGMC( Preset="faster", FPSDivisor=2, ShutterBlur=3, ShutterAngleSrc=180, ShutterAngleOut=360, SBlurLimit=8 )

The problem is that by doing so it runs through all the QTGMC processing and modifies sharpness (amongst other things I guess....) where I'm only interested in the motion blur part. In fact I'd need a standalone motion blur script out of QTGMC :D

Maybe I could do that more easily by running a simpler MFlowBlur script ? (But I really like the output of QTGMC)

[-Vit-]

All QTGMC processing is optional including the deinterlacing. To get a standalone motion blur do this:

QTGMC( "Faster", ShutterBlur=3, ShutterAngleSrc=180, ShutterAngleOut=360, SBlurLimit=8, \
InputType=1, TR0=0, TR1=0, TR2=0, Rep0=0, Rep2=0, SMode=0, SLMode=0, Sbb=0, NoiseProcess=0 )


In this case the preset simply selects the accuracy of the motion analysis. The end part of the line sets progressive input and switches of all the deinterlacing processes.
Despite QTGMC being a huge script there's no extra processing overhead to this except the initial parsing (a fraction of a second I guess). All unused processes are entirely bypassed, not even initialized.

However, you say you really like the output of QTGMC, which suggests you actually want some kind of temporal processing. So maybe you want the progressive repair mode combined with motion blur:
QTGMC( "Faster", InputType=1, Sharpness=0.4, ShutterBlur=3, ShutterAngleSrc=180, ShutterAngleOut=360, SBlurLimit=8 ) # Tweak sharpness to taste
This will remove any shimmer residue, temporally denoise a little and enhance a little depending on sharpness.

[kolak]

Vit, what cmd should I use for deinterlacing only. I don't want any processing, just deinterlace, so the file is as close as possible to the source (but deinterlaced :))


Thanks,
Andrew

[Boulder]

You might want to try the settings -Vit- mentions in this answer to my question: http://forum.doom9.org/showthread.php?p=1488143#post1488143

[Didée]

Vit, what cmd should I use for deinterlacing only. I don't want any processing, just deinterlace, so the file is as close as possible to the source (but deinterlaced :))

Sorry I'm not Vit, but that is easy to answer. If you want *strictly* only-deinterlacing, with *strictly* no additional processing, here you go:

NNEDI3(field=-2)


*All* code of TGMC/QTGMC is about "additional processing to improve the result of a given basic deinterlacer". Without any additional processing, you're left back with your raw basal deinterlacer of choice.

And no, there isn't really anything like a "bare minimum" of processing. Or perhaps like this: the bare minimum you wouldn't accept quality-wise.

[kolak]

I'm happy with any answer, which is informative:)

How does NNEDI3 compare to YADIF?
I've tried YADIF, but it's no near QTGMC.

Other way- what is the cmd line for good deinterlacing and keeping all noise/grain.
Sometimes QTGMC output looks digital- has some "special" look.

Thanks,
Andrew

[kolak]

You might want to try the settings -Vit- mentions in this answer to my question: http://forum.doom9.org/showthread.php?p=1488143#post1488143

Tanks.

Sounds good- bit different than I expected :) EZKeepgrain=1 -I thought this keeps all grain :)


Andrew

[aegisofrime]

Vit, what cmd should I use for deinterlacing only. I don't want any processing, just deinterlace, so the file is as close as possible to the source (but deinterlaced :))


Thanks,
Andrew

If you have read the documentation, that is what SourceMatch claims to try to do, hence the name. I would give that a shot if I were you.

Although personally I like the denoised sharpened look that non-SourceMatch gives :p

[kolak]

I went through readme, but it's not that clear- quite few settings, which work together (or not) to achieve some result.

Andrew

[-Vit-]

My personal favorite settings remain the same as Boulder's reference above, slight variations on this:
QTGMC( "Slower", SourceMatch=2, Lossless=2, EZKeepGrain=0.5, Sharpness=0.3 )
Vary EZKeepGrain depending on the noise/grain in the source and how much you want to keep it (warning: subtle effect) and adjust sharpness to taste (from around 0.05 to about 0.5).
This gives a result accurate to the source including source grain/noise/artefacts (so it can be rough if your source is not high quality). It won't shimmer like NNEDI3 or Yadif. It reduces that "digitally enhanced" look of the defaults.

[SubPixie]

All QTGMC processing is optional including the deinterlacing. To get a standalone motion blur do this:

QTGMC( "Faster", ShutterBlur=3, ShutterAngleSrc=180, ShutterAngleOut=360, SBlurLimit=8, \
InputType=1, TR0=0, TR1=0, TR2=0, Rep0=0, Rep2=0, SMode=0, SLMode=0, Sbb=0, NoiseProcess=0 )


In this case the preset simply selects the accuracy of the motion analysis. The end part of the line sets progressive input and switches of all the deinterlacing processes.
Despite QTGMC being a huge script there's no extra processing overhead to this except the initial parsing (a fraction of a second I guess). All unused processes are entirely bypassed, not even initialized.Thanks for you answer Vit, that's exactly what I needed. By the way, do you know if there's a reason to set the preset to anything higher than "Faster" in order to get... better quality motion blur ? (I don't think so but just asking for the sake of completeness)

Oh and thanks for all of your work you and Didée, this is... simply amazing quality-wise.

[-Vit-]

By the way, do you know if there's a reason to set the preset to anything higher than "Faster" in order to get... better quality motion blur ?Only difference is accuracy of motion analysis, but as you're blurring that's of rather less importance than in the normal case. Slower presets might occasionally find the correct motion when the faster ones miss it (which would make the blur go the wrong way or look odd). Depends on the speed/complexity of motion in your source.
Slower presets may slightly increase sharpness of slow moving areas (although SBlurLimit helps more there) and they may also slightly help the blur "bleeding" reduction of the higher SBlurLimit values. Both of those effects relatively minor.

[SubPixie]

You're right I did notice bad motion analysis (artefacts) using the "Faster" preset (bad scene change detection, you know, with objects abruptly and unnaturally zooming in...). But I noticed I have more or less the same problem with Interframe no matter what quality preset I use, so I guess the culprit here is MVTools... (?)

[Mini-Me]

Vit, what cmd should I use for deinterlacing only. I don't want any processing, just deinterlace, so the file is as close as possible to the source (but deinterlaced :))


Thanks,
Andrew

As Didee suggested, you could simply use NNEDI3 to fill in the missing lines based on spatial data from the same field. (Heck, how about original.SeparateFields()? ;)) Technically speaking, the result will be "close to the source," since you haven't done much with it...but it's FAR lower quality than the original camera source was before dropping half of the lines. Even and odd frames will contain different features, because the even and odd fields each had lines at different positions, and no effort was made to cross-reference them.

If you want to faithfully deinterlace your source and make it look as much as possible like the original camera footage (before half the fields were dropped), you need to do a LOT of processing. That involves estimates, guesstimates, coverups of wrong guesstimates, etc. ;)

TGMC's primary goal - which QTGMC adopted - is to create a smooth enough deinterlaced clip that there's no bob flicker. This is a good thing, because the original camera footage wouldn't have had bob flicker. ;) However, the processing that it takes to get this result tends to remove grain and noise from the picture, as well as fine detail...so by default, TGMC/QTGMC resharpens to strengthen some of the detail that was lost in the process of smooth deinterlacing.

A lot of QTGMC's "extra features" are not about changing the source but restoring what was lost in the process of removing bob flicker. For instance:
SourceMatch uses several clever methods to reduce the impact of temporal smoothing. It's so effective that you can significantly cut back on sharpening, which is otherwise necessary to recover faded detail.
Noise processing is not just about denoising your source. Instead, you can use it to prevent denoising: Since the main algorithm removes noise and fine detail as a side effect, you can remove it up front and add it back after the main algorithm is over with. Of course, that's also tricky, because you have to guess what kind of noise should have been in the new fields you're creating...and some guesses are better (and require more work) than others.
By default, QTGMC alters the original lines of each field; you can use lossless modes to reinsert them at particular time periods. Lossless mode 2 is only "mostly" lossless. Lossless mode 1 keeps the original fields entirely unmodified under two conditions: If you use NoiseProcessing = 1, you must set GrainRestore to 0.0, NoiseRestore to 1.0, and StabilizeNoise to False. If you use NoiseProcessing = 2, you must set GrainRestore and NoiseRestore to 0.0. Important: If you want the original fields to be totally lossless, do not use EZKeepGrain; it sets GrainRestore and NoiseRestore to different values.


Now that I've blabbed on forever, I'll throw out some settings. By "as close as possible to the source," I'm assuming you mean:
Keep the original lines of each field exactly as they are, completely unmodified.
Fill in the missing lines, and carry over as much detail as possible from nearby fields, so they look as close as possible to what the camera would have originally captured (before it dropped them).
Keep the noise characteristics the same as the source: Source lines are exactly the same, and new lines should match them as much as possible.
Keep the sharpness characteristics the same as the source: Source lines are exactly the same, and new lines should match them as much as possible.
Do not deliberately add motion blur or drop any fields.

In that case, the "ultimate" with the current version of QTGMC would be the ridiculous settings I posted earlier:
deint = source.QTGMC(preset = "placebo", SourceMatch = 3, MatchPreset = "placebo", \
MatchPreset2 = "placebo", lossless = 1, NoiseProcess = 1, NoisePreset = "slower", DenoiseMC = True, \
NoiseDeint = "Generate", StabilizeNoise = False, GrainRestore = 0.0, NoiseRestore = 1.0)

If you switched StabilizeNoise to True, the generated fields should look even closer to what the camera would have originally captured. Unfortunately, that alters the noise in the original fields too, and it prevents them from being 100% lossless. You might even want to switch NoiseDeint to "DoubleWeave"; the faster settings below actually seem to be retaining noise better in the generated fields, and the difference is due to that setting.

If you want usable settings that don't take a million years, I'd suggest this:
deint = source.QTGMC(preset = "slower", SourceMatch = 3, lossless = 1, NoiseProcess = 1, \
StabilizeNoise = False, GrainRestore = 0.0, NoiseRestore = 1.0)

The preset is lower here, but it's still a very solid setting. The match presets and noise presets are implicitly lower, and I removed some other noise processing settings to let the default preset (fast) override them. These settings will let you finish rendering the video sometime during the next month, but they'll still keep the original fields lossless and ensure the generated fields fit well with them. There's some subtle aliasing and a slight loss of subtle detail, but you'd never notice without flipping back and forth between the two outputs.

[kolak]

Thanks Vit and Mini-Me.

Will try and see how it compares to my simple cmd lines.

Mini-Me- I work with HD mainly so Placebo presets are out of option :) I need something which is at least eg. 8fps. MT is not working well with QTGMC, so I can't use properly my 12 cores machine. As far as I understand if I scale output from QTGMC than I don't need to keep original fields. If I'm deinterlacing eg, HD 50i to HD 50p than it would be better to keep original fields, so it plays properly on TVs.

Your post is great and in the same time makes me even more confused :)


Andrew

[Mini-Me]

Thanks Vit and Mini-Me.

Will try and see how it compares to my simple cmd lines.

Mini-Me- I work with HD mainly so Placebo presets are out of option :) I need something which is at least eg. 8fps. MT is not working well with QTGMC, so I can't use properly my 12 cores machine.
For HD material, you're going to be suffering with any of the slower presets. :eek: Thankfully, the slower presets don't help as much for HD, and the SourceMatch settings aren't quite as important either.

Try:
deint = orig.QTGMC(preset = "medium", SourceMatch = 1, Lossless = 1, NoiseProcess = 1, \
StabilizeNoise = False, GrainRestore = 0.0, NoiseRestore = 1.0)

I focused so much on keeping half the lines lossless because you indicated you wanted the output to remain faithful to the source. If lossless isn't important to you, you can trim things down to:
deint = orig.QTGMC(preset = "medium", SourceMatch = 1, EZDenoise = 1.0)
See how that works out. If you can afford to go slower, just bump the presets down one at a time.

As far as I understand if I scale output from QTGMC than I don't need to keep original fields. If I'm deinterlacing eg, HD 50i to HD 50p than it would be better to keep original fields, so it plays properly on TVs.

Your post is great and in the same time makes me even more confused :)

Andrew

I think you might be misunderstanding what I mean by lossless and keeping the original fields though. It has nothing to do with interlaced vs. deinterlaced: The output of QTGMC will be deinterlaced either way, and you can scale it all you want.

What I mean is this: Half of the lines from every deinterlaced frame will be newly generated, and half will be lines that were taken from the original interlaced fields. By default, QTGMC processes (smooths and sharpens) the latter lines instead of keeping them exactly as they were. The point of keeping them lossless is simply about keeping the output as faithful to the source as possible, with less smoothing and sharpening. (It also allows you to separate fields again and redo deinterlacing without any ill effect.)

[kolak]

I think you might be misunderstanding what I mean by lossless and keeping the original fields though. It has nothing to do with interlaced vs. deinterlaced: The output of QTGMC will be deinterlaced either way, and you can scale it all you want.

What I mean is this: Half of the lines from every deinterlaced frame will be newly generated, and half will be lines that were taken from the original interlaced fields. By default, QTGMC processes (smooths and sharpens) the latter lines instead of keeping them exactly as they were. The point of keeping them lossless is simply about keeping the output as faithful to the source as possible, with less smoothing and sharpening. (It also allows you to separate fields again and redo deinterlacing without any ill effect.)

Understand this perfectly :)

And you also confirmed my points saying this:

The point of keeping them lossless is simply about keeping the output as faithful to the source as possible, with less smoothing and sharpening. (It also allows you to separate fields again and redo deinterlacing without any ill effect.)

If you downscale than whole image changes a lot- you create new standard eg. PAL. If you just deinterlace (eg, 50i to 50p) than it's "less changing". In the 1st case you don't have to care that much about matching the source.

I don't always use QTGMC for downscaling to interlaced output- more often resizing by field, because it produces more stable output- bit softer, but not flickering.
There is quite often problem with to sharp image when going from eg 50i HD to PAL through QTGMC. When you start with interlaced and finish with progressive than QTGMC is unmatched- even some expensive pro solutions are not outperforming QTGMC:) Just wish for stable MT conversions.

Fields are annoying- I hope will be gone soon :)
Thanks for your comments- very useful :)



Andrew

[-Vit-]

Some awesomely accurate descriptions there Mini-Me.
I assume you'll be taking over development from here... :p

[Mini-Me]

If you downscale than whole image changes a lot- you create new standard eg. PAL. If you just deinterlace (eg, 50i to 50p) than it's "less changing". In the 1st case you don't have to care that much about matching the source.
I see what you mean now. :) If you're going to significantly alter the output anyway, it makes sense that you wouldn't care about absolute faithfulness to the source. Come to think of it, if you're going to downscale, you could do away with SourceMatch and NoiseProcess, since most of the extra detail retained would probably be lost in the resize anyway. Without SourceMatch and NoiseProcess, you can pick up quite a bit of speed too.

I don't always use QTGMC for downscaling to interlaced output- more often resizing by field, because it produces more stable output- bit softer, but not flickering.
There is quite often problem with to sharp image when going from eg 50i HD to PAL through QTGMC. When you start with interlaced and finish with progressive than QTGMC is unmatched- even some expensive pro solutions are not outperforming QTGMC:) Just wish for stable MT conversions.
Have you tried using a vertical low-pass filter (gentle vertical blurring) before dropping half the fields for PAL? It might sound like wanton destruction, but it should help with the sharpness problem. I've read that a lot of broadcast interlaced footage was created this way (by low-passing, then dropping half the lines per field).

Fields are annoying- I hope will be gone soon :)
Thanks for your comments- very useful :)

Andrew

You're welcome! I'm glad I could help. :)

Some awesomely accurate descriptions there Mini-Me.
I assume you'll be taking over development from here... :p

Thank you. :)

I know you're joking, but I wouldn't dare trying to improve something like QTGMC at a time like this...you and Didee set the bar extremely high for deinterlacing, and there are some more neglected filter categories that have lower-hanging fruit! :D

[zerowalker]

How do you remove these artifacts, i don´t know what it´s called, but i can trust you guys;D
http://imageshack.us/photo/my-images/51/artifacts.png/

[kolak]

Have you tried using a vertical low-pass filter (gentle vertical blurring) before dropping half the fields for PAL? It might sound like wanton destruction, but it should help with the sharpness problem. I've read that a lot of broadcast interlaced footage was created this way (by low-passing, then dropping half the lines per field).



I can easily match or outperform SD output from eg. HDCAM-SR. It's so heavy processed- low pass filtered, sharpened- looks bad, but most broadcast people are very happy with it :)
I prefer avisynth- if not one method than another :)

I add a bit blur on the HD source and than after resizing. Small blur at HD stage helps to keep smoother diagonals- they are less stepy after resizing. It also depends on the source nature- some source downscales nicely, some is very difficult.
If you shoot something with prosumer camera than quite often you have lots of flickering on HD source and after resizing it looks terrible- in the same time source from good cameras scale much better.


Andrew

[Dogway]

I was getting some local blending in this source (the flag at the end) when using QTGMC. I thought at first it was some field blending problem in my source but using a plain bob() shows it has to be in the way QTGMC works. Probably in the motion estimation/ compensation stage(?), any help on this?

this is the line Im testing with:
QTGMC(tr0=3,preset="very Slow",Lossless=2,sourcematch=3)

65Mb
http://www.mediafire.com/?a7yw57ft1i65p52

edit: no problem, I guess your explanation in this post (http://forum.doom9.org/showthread.php?p=1496803#post1496803) stands true for this task too. This being my second anime I try to deinterlace with no success I guess that qtgmc + anime = nono.
Im gonna test other deinterlacers, what is recommended out there for animation? (I found Terranigma's MCbob mod (http://forum.doom9.org/showthread.php?p=1396802#post1396802), looks fine)

[suloku]

Hi, I'm getting a lot of corrupted frames when using the mt version (2.6) with this script:

#SetMemoryMax(1000) # Optional line. Leave it out at first. See below for value M
SetMTMode(5, 4) # See below for value X
Avisource("e:\cinta.avi") # DGDecode_mpeg2source, FFVideoSource, AviSource, whatever your source requires
SetMTMode(2)
QTGMC( Preset="fast", EdiThreads=2 ) # Choose preset based on overall speed/quality you want. See below for value Y
#Distributor() # This line may or may not be necessary, try removing it and see if you get more speed
SelectEven() # Add this line to keep original frame rate, leave it out for smoother doubled frame rate

This is an example:
http://img708.imageshack.us/img708/2529/vlcsnap2011081710h22m51.png (http://imageshack.us/photo/my-images/708/vlcsnap2011081710h22m51.png/)

I've tried different values as specified in the first post but I still get the corrupted frames. Everything's fine without the mt version.

Also, if I disable SelectEven() the outcome is 50 fps, but the video gets jumpy. The source is a 25 fps 720x576 interlaced video in huffyuv (previously captured with virtualdub from a pal vhs source). This happens regardless of using MT or not.

Any ideas?

Thanks in advance.

[yup]

suloku!
Try DirectShowSource("e:\cinta.avi",pixel_type="YUY2") instead AVISource.
yup.

[suloku]

suloku!
Try DirectShowSource("e:\cinta.avi",pixel_type="YUY2") instead AVISource.
yup.

Thanks, that seems to have solved the corrupted frames, I'm enconding the full movie now to make sure, but there wasn't one in 1000 frames (before there were every 100 frames or so).

Also, I had to use "DirectShowSource("e:\cinta.avi", pixel_type="YV12")" as YUY2 isn't supported by mvtools (or at least that was the error virtualdub spit out).

About the jittering problem, I seem to have solved it adding AsumeTFF() before calling qtgmc.

I'll edit later with the results, thanks!

EDIT: All processed OK now, finally I can convert my vhs tapes, and I think that the result looks even better than the original tape!

EDIT2: I didn't notice earlier, but the audio gets unsyched after processing with qtgmc even though the video has the same lenght as the original. I've tried exchanging the audio of the converted clip with the one from the original capture, but still out of synch, so my guess is that the converted video plays faster than the original.

The original interlaced video is captured at 25 fps with virtualdub, so after qtgmc processing audio should still be in synch even if the framerate is doubled to 50 fps, but that's not what I get.

I'm using this script, I added the audio=false and assume 25 fps as I saw in another post in order to see if this helps.

SetMTMode(5, 4) # See below for value X
DirectShowSource("e:\cinta3.avi", pixel_type="YV12", audio=false).AssumeFPS(25)
SetMTMode(2)
AssumeTFF()
QTGMC( Preset="fast", EdiThreads=2 ) # Choose preset based on overall speed/quality you want. See below for value Y
Crop(12, 6, -12, -12) # left, top, right, bottom, 768x576 crop
Lanczos4Resize(640,480)


Any other ideas?

[Weirdo]

Thanks again for the great work, and for putting everything in an easy to understand context, for us avisynth noobs. Sorry for this slightly OT question: will the SetMT settings you mention work with the recommended avisynth MT versions, but without using QTGMC (for progressive sources)? I seem to get the same speed. (Intel Q9650)
edit: now that I look at it, I've disabled EdiThreads as well... :o

[-Vit-]

The template given in the first post can be used for non-QTGMC processing. Replace the QTGMC line with whatever other processing you need to do - the same instructions still apply. However, MT mainly benefits complex filters, it will have little or no impact on simpler processing.

I can't read your attachment yet (just write the script directly in your post in this format: SetMTMode(5,8) ...). However, if all you're doing is removing QTGMC as you say then you're doing no processing at all. There's nothing to speed up. Except the source filter, but that's not affected by/suitable for MT in this form.

[Weirdo]

The template given in the first post can be used for non-QTGMC processing. Replace the QTGMC line with whatever other processing you need to do - the same instructions still apply. However, MT mainly benefits complex filters, it will have little or no impact on simpler processing. However, if all you're doing is removing QTGMC as you say then you're doing no processing at all. There's nothing to speed up. Except the source filter, but that's not affected by/suitable for MT in this form.Thanks a lot - that's the perfect answer. In my case, there's no need for MT then. Here's the script btw.
SetMTMode(5, 5)
MPEG2Source("E:\my source.d2v",cpu=0)
SetMTMode(2)
#QTGMC( Preset="Slow", EdiThreads=2 )
Crop(12,0,-12,0)

[suloku]

I'm still with my wird problems, I've tried both mt and non mt versions of avisynth 2.6, but the results are always the same.

-Avisource produces a correct output, but there are corrupted frames.
-DirectShowSource produces no corrupted frames, but the video ends before he audio, it's like its playing faster.
-FFVideoSource: I can't use it because my source video is in yuy2, virtualdub crashes if I open the script with ConvertToYV12(), "An out-of-bounds memory access (access violation) occurred in module 'ffms2'..."

What can I do?

EDIT: I've tried many things, converting the clip to yv12 prior to processing with qtgmc (I mean in different scripts and processings) seemed to work, but I finally managed to convert it straight forward by splitting the avifile in < 2GB chunks and loading them via AviFileSource(clip1)+AviFileSource(clip2)... and so on.
AviSegmentedSource produced the same errors as with AviSource (corrupetd frames).

As I'm capturing with virtualdub I'll just capture directly in segments and hopefully everything will go alright from now on with my processing.

[Malow]

first, tnks for great work guys.

second, what setting most affect luma "fades"?

cause when there's a jump in the video brightness (iris adjustment) the details go away, then go back (a blurring during the transition)

tried various settings with TR and REP settings with no luck... :(

can post a video source if helps.

[sikboy1029]

Can someone help me out with a setting so I don't see ghosting from the previous frame when I process this clip? I have another thread open about the Futurama DVDs but I figured it couldn't hurt to get a proper script for this plugin as well.

http://www.mediafire.com/?n57n2bgskk1k7n3

[wOxxOm]

sikboy1029, I haven't checked your clip - beware it could have blends/ghosts due to bad authoring, but regarding qtgmc's ghosting try sourcematch=3 and btw post your qtgmc settings.

[-Vit-]

sikboy1029. That clip is not interlaced. It has some horizontal artefacting that might normally be suitable for QTGMC progressive mode (set InputType=1). However, see this post regarding QTGMC, animation and ghosting (http://forum.doom9.org/showpost.php?p=1496803&postcount=649).
____

Malow: The motion analysis may be incorrect at fades. When the motion analysis fails then blurring or ghosting can occur. The DCT modes of MVTools are intended to make it more robust to luma changes, but I haven't experimented much with them. QTGMC accepts a DCT=X setting for this. Read the MVTools documentation for the values you choose for X. I know that DCT=1 can be very, very slow. Mode 5 can be OK. Modes 2, 3 & 4 sound useful, don't know about speed, maybe modes 6 to 10 also (slower). Perhaps someone else has experience of using the MVTools DCT setting for this purpose?

[SubJunk]

I did a lot of testing with DCT and found that 5 produces the least warping with luma changes. 8 is also good, but 5 seems to be the best.

[sikboy1029]

Thus far I had only tried using the presets like in the first post, I tried all of the speeds. The worst ghosting is right at the end of the clip when Morgan turns, not a huge deal but annoying when you see a few instances per episode. I tryed a few different source match setting and it's still there. This is the last script I tried, do I have it setup incorrectly?

DGDecode_Mpeg2Source("fr.d2v")
QTGMC(sourcematch=8)
SelectEven()

[-Vit-]

QTGMC is primarily a deinterlacer, the video you posted is not interlaced (but was it originally?). You could try it's progressive "repair" mode: something like QTGMC(InputType=1,TR0=1,TR1=1,TR2=0). However, QTGMC is not well suited to such simple animation, read the link I posted above to understand why.
SourceMatch=8 is not a valid setting, read the docs.

[Dogway]

I am getting very jumpy processing load, ranging from 65% to 95%. I emptied my plugins folder, and now Im using all your mod versions. But it's still the same. Do I need .net 4, or something special? My source is ntsc DVD, not very demanding, I have a nice laptop, core 2 duo 2.53ghz 1066fsb 6mb l2, geforce 9600mgt, 4gb ddr3 533fsb.XP SP3.
Im on avisynth 2.58 MT set (build: jan 8 2010) and encoding to ffv1 through ffdshow vfw in virtualdub 1.10

ColorMatrix.dll
colors_rgb.avsi
deblock.dll
DGDecode.dll
MT.dll
mt_masktools-25.dll
mvtools2.dll
nnedi3.dll
QTGMC-3.32.avsi
RemoveGrainSSE2.dll
RepairSSE2.dll
SSE2Tools.dll
VerticalCleanerSSE2.dll

setmtmode(5,2)
MPEG2Source("source.d2v", cpu=0)
setmtmode(2,2)
QTGMC(tr2=3,preset="slower",Lossless=2,sourcematch=3,sharpness=0.3,MatchEnhance=0.3,MatchPreset="Slow", MatchPreset2="medium",EdiThreads=2)

this is the source Im working with (9Mb):
http://www.mediafire.com/?mfci31cc4uwzenj

[-Vit-]

...not very demanding...
QTGMC(tr2=3,preset="slower",Lossless=2,sourcematch=3,sharpness=0.3,MatchEnhance=0.3,MatchPreset="Slow", MatchPreset2="medium",EdiThreads=2)
Those settings are insanely demanding even for SD. Even more so given such a modest CPU. Extreme settings and/or HD sources cause some kind of processing bottleneck within avisynth. Not limited to QTGMC, many complex (esp. temporal) scripts will do it. The reasons... ?

[Dogway]

For the test I used your modded nnedi3 but commonly I use nnedi3 0.9.4. It doesn't make any difference right? Just by looking at the called settings.

Also I encourage you to add lsb=true for the next versions, for dfttest and mdegrain. I added them by myself in the script, but maybe youd want to add them "officially".

I don't consider my laptop underpowered, is a bit old (2 yrs) but is very nice, besides I keep my system clean of background processes, etc. I was suprised because I'm just calling the QTGMC line, no other processing is done, and I normally work at <1fps rates, with tnlmeans under dither() (x4 processing time), etc and several heavy power consumption filters, but even at 0.xfps they go steady at 100% load. Here I'm not even using "very slow" or "placebo" presets, same for matchpresets. And last night changed sourcematch to 2, but processing load keeps unstable. It's fishy, so Id like if someone can test with my source above so I can be 100% sure is my system. Anyway I'm going to test now with friendly settings, but Im not confident...

Probably the error comes from vdub, when encoding to lossless. Im still not sure how to tweak parameters in vdub for very slow, demanding scripts...

edit: I tested with only the MPEG2Source() line, and it goes at 55% average. Maybe this is the cause¿?
Also tested with next line which is considerably less demanding:
QTGMC(tr2=3,preset="slow",subpel=2,Lossless=2,sourcematch=2,sharpness=0.5,MatchEnhance=0.3,EdiThreads=2)

[-Vit-]

The modded version of NNEDI3 is 0.92. I believe 0.94 is faster, so that will make a difference. Good reminder, I'll update the modded plugins today.

It is my intent to add optional 16-bit processing into the script. However, I'm too busy to make major changes for a couple of months at least.

Re: CPU, QTGMC really wants 4 cores and 4+ threads at SD. The settings you've chosen are much, much slower than plain "Placebo". It's mainly because of source-match, which I haven't put into the "Placebo" setting because it may not be what people want (had a discussion about that a few pages back). Source match is very accurate, but slow. Even at placebo, NNEDI3 is just called once per frame, with SourceMatch=3 it's called three times per frame (plus three more MDegrains + other things...). Any increase in source match preset is effectively a placebo setting - making those additional NNEDI3 calls even slower. Setting tr2=3 is matching a placebo setting too.

[Dogway]

Well I guess I can't do any more, but thanks for the answers. Sourcematch is important in this case because its a videogame source, and I want to keep the pixel nature of it. I will test on a friend's PC (multicore) if I have the chance.

[-Vit-]

I've updated the plugin packages (standard and modded) in the first post (http://forum.doom9.org/showthread.php?p=1423459) to include the latest NNEDI3 0.9.4. Worth updating if you haven't already.
A reminder that the modded plugins include SEt's threading fixes plus an extra hack to help prevent crashes under Avisynth MT
____

dogway: Your updated settings seem more reasonable. Source match works well with moderately faster presets (but not the fastest), and produces a result almost as good but with better speeds, which seems a good balance for your machine.

[chrishallowell]

I'm getting the below error using the modded plugins and Avisynth SEt's 2.6MT (32bit). It's the Lossless parameter that is causing the problems. I've tried every combo I could think of but still not luck. I've also tried the below MT but it tells me the function MT doesn't exist. Any ideas on what to try next?
Thanks,
Chris

MT("""QTGMC( "Slower", EdiThreads=1, TR2=1, SourceMatch=2, Lossless=2, Sharpness=0.2, EZKeepGrain=0.6 )""", threads=2,overlap=16)

Traceback (most recent call last):
File "AvsP.pyo", line 7380, in OnMenuVideoRefresh
File "AvsP.pyo", line 11075, in ShowVideoFrame
File "AvsP.pyo", line 11662, in PaintAVIFrame
File "pyavs.pyo", line 343, in DrawFrame
File "pyavs.pyo", line 320, in _GetFrame
File "avisynth.pyo", line 277, in GetFrame
WindowsError: exception: access violation reading 0x2B1D9000


#SetMemoryMax(512) # Tried with and without this, also tried 1024 and 2048
SetMTMode(5, 2) # Tried 5,2 and 2,2
AviSource("G:\HomeVideo_RAW_Unedited\SD\001 - Vol01.avi", audio=true, pixel_type="YUY2", fourCC="LAGS").Trim(1,500)
SetMTMode(5) #Tried 2 and 5
AssumeBFF()
Crop(0,4,0,-8)
AddBorders(0,4,0,8)
#QTGMC( "Slower", TR2=1, SourceMatch=2, EZKeepGrain=0.6, Sharpness=0.2, EdiThreads=1) Works
QTGMC( "Slower", TR2=1, SourceMatch=2, Lossless=2, EZKeepGrain=0.6, Sharpness=0.2, EdiThreads=1) #Doesn't work

[yup]

chrishallowell!
At last SEt 2.6 build MT not implemented, only SetMTMode. Also problem maybe AVISource, replace to DirectShowSource.
yup.

[chrishallowell]

Thanks Yup. I'll give that a try when I get home.

[Gavino]

At last SEt 2.6 build MT not implemented, only SetMTMode.
The MT() function has always been in the separate MT.dll which you have to load explicitly if not in the plugins folder.
Or are you saying that this dll does not work with SEt's build?

[Taurus]

@yup
Just some cosmetics:
Is it possible to format the QTGMC-3.32.avsi in a more readable way?
I know this has happened in the past too :p.
3.25 is formated in the standard way.

[aegisofrime]

@yup
Just some cosmetics:
Is it possible to format the QTGMC-3.32.avsi in a more readable way?
I know this has happened in the past too :p.
3.25 is formated in the standard way.

I think you are supposed to read the .html file included now. Or else you can try opening with Wordpad instead of notepad, it's readable that way.

[Taurus]

Or else you can try opening with Wordpad instead of notepad, it's readable that way.
Thanks a lot!

[-Vit-]

Is it possible to format the QTGMC-3.32.avsi in a more readable way?
I've updated the script in the first post to use Windows line breaks again. No version number change.
Not sure what I'm doing that keeps changing them, but I use editors that handle any kind of line break so I don't notice when it happens.

[Taurus]

Thanks:thanks:

[chrishallowell]

Tried DirectShowSource and still no luck.
It works when I convert the video to YV12. The error seems to be caused by mt_masktools-25.dll. (I don't have 26 in the plugin directory). Since I'm using set Avisynth 2.6, I also tried 26 in the plugin directory and removed 25, but the same error happens unless I add the YV12 conversion.
SetMTMode(2, 2)
AviSource("G:\HomeVideo_RAW_Unedited\SD\001 - Vol01.avi", audio=true, pixel_type="YUY2", fourCC="LAGS").Trim(1,500)
SetMTMode(2)
AssumeBFF()
ConvertToYV12(interlaced=true) #Needs this to work because masktools only support YUV colorspaces (YV12, YV16, YV24)
QTGMC( "Slower", SourceMatch=2, Lossless=2, EZKeepGrain=0.6, Sharpness=0.2, EdiThreads=3)

[AzraelNewtype]

Tried DirectShowSource and still no luck.
It works when I convert the video to YV12. The error seems to be caused by mt_masktools-25.dll. (I don't have 26 in the plugin directory). Since I'm using set Avisynth 2.6, I also tried 26 in the plugin directory and removed 25, but the same error happens unless I add the YV12 conversion.

If your video is really YUY2, you will absolutely have to convert to YV12 to run it. This is not a bug. Also you absolutely need masktools-26 if you're running avisynth 2.6. Also your first SetMTMode should be (5,2), changing to 2 where you have it.

[yup]

If your video is really YUY2, you will absolutely have to convert to YV12
No!
QTGMC support YUY2 colorspace. In this case you need use masktools-25 not 26 see
http://forum.doom9.org/showthread.php?p=1497883#post1497883. (http://forum.doom9.org/showthread.php?p=1497883#post1497883)
yup.

[jmac698]

http://avisynth.org/mediawiki/QTGMC
Updated. Vit could you update this too, please.

[nhope]

No!
QTGMC support YUY2 colorspace. In this case you need use masktools-25 not 26 see
http://forum.doom9.org/showthread.php?p=1497883#post1497883. (http://forum.doom9.org/showthread.php?p=1497883#post1497883)
yup.

Is this true even if one is running QTGMC in SEt's 2011.09.11 build of AviSynth 2.6 (http://forum.doom9.org/showthread.php?p=1312666#post1312666) with plugins from Vit's latest modded plugins package? I'm guessing that combo currently gives the best chance of MT stability in the majority of cases.

[yup]

@nhope see
http://forum.doom9.org/showthread.php?p=1498328#post1498328
I am try last SEt 2.6 build with standard plugin for YUY2 colorspace without problem.
yup.

[Fullmetal Encoder]

I have tried searching around for some information regarding this and have failed to find anything so I apologize if it's already been discussed at length.

Is it possible to get QTGMC to do its work with 16-bit precision? If dfttest were being used for denoising then shouldn't it be possible to use the modified version of dfttest that produces a 16-bit output created for cretindesalpes's dither tools? If so, couldn't I simply invoke the lsb=true option in the QTGMC avsi?

Also, would any other changes need to be made to the QTGMC script or the other plugins it calls?

What I would like to do is achieve a pure 16-bit processing workflow that incorporates QTGMC for deinterlacing and denoising, SAPikachu's flash3kyuu_deband (once he gets around to coding it for 16-bit input) for gradients and finally x264-10bit.

[Dogway]

@Fullmetal Encoder: I doubt it can be done, because there is no 16 bit MaskTools2, what QTGMC is mostly based on. cretindesalpes ported some functions to 16 bit but it is not enough. What can be done is introduce 16 bit modified dfttest and mvtools(denoising) but not deinterlacing. I did the first, added lsb=true for dfttest, and also deleted the custom inflate deflate functions, so it should go faster, now even more as Vit added official nnedi3 v.0.9.4.
If you want try with my tweaked script (http://www.mediafire.com/?c4wxzhhsixslu3i).

I also wanted to note that QTGMC creates some ugly artifacts very occasionally, and normally in the first frames of source, or trimmings. It happens when there's a big memory load. I can't reproduce the bug, but just wanted to let it know.

[jmac698]

It's easy to simulate a 16 bit masktools. This is exactly what I did in my v210 reader. You need to write just some extra lines to process the other bits. Just think of it as two 8bit videos processed separately, but one more line to link them together (the carry) when adding or multplying.

[Fullmetal Encoder]

@Fullmetal Encoder: I doubt it can be done, because there is no 16 bit MaskTools2, what QTGMC is mostly based on. cretindesalpes ported some functions to 16 bit but it is not enough. What can be done is introduce 16 bit modified dfttest and mvtools(denoising) but not deinterlacing. I did the first, added lsb=true for dfttest, and also deleted the custom inflate deflate functions, so it should go faster, now even more as Vit added official nnedi3 v.0.9.4.
If you want try with my tweaked script (http://www.mediafire.com/?c4wxzhhsixslu3i).

Interesting, so I take it that everything else i.e., nnedi3, would also have to be modified for 16-bit precision? What exactly is involved in making these modifications? From a layman's standpoint I wouldn't expect it to be that difficult as I would think all of the calculations and functions should be the same; you're just adding length or memory to the values (although as I write this I can imagine the programmers out there getting a good chuckle out of it). From your response I take it that modifying masktools for 16-bit would be a massive undertaking. I would be willing to work on it myself if it were feasible. I don't have knowledge of programming languages or experience coding but I have done a fair bit of scripting in PowerShell 2.0 and am fairly good at learning on my own. Or would I require an in-depth understanding of C++ to attempt something like this?

I wanted to resolve these issues before beginning to do a lot of encoding work with x264-10bit. I will be doing processing with a deinterlacer and with denoising and debanding at the very least and from what I understand the more processing that's done on the video in 8-bit before the final encoding process the more those rounding errors are magnified in the final product. I'm really impressed with how much better the visual quality is in encoding into 10-bit even just from 8-bit sources.

I also wanted to note that QTGMC creates some ugly artifacts very occasionally, and normally in the first frames of source, or trimmings. It happens when there's a big memory load. I can't reproduce the bug, but just wanted to let it know.

This is interesting, is this something you've spotted only on trimmed video wherein you've spliced a video segment out of a larger clip? In that case then I would hope that it doesn't affect my encoding since I don't do any trimmings.

[Gavino]

From your response I take it that modifying masktools for 16-bit would be a massive undertaking.
Functions like mt_lut, etc, work by pre-calculating the results for all possible input values and storing them in lookup tables (LUTs).
Changing input from 8-bit to 16-bit would mean a huge increase in LUT sizes, from 256 entries to 65536 for mt_lut, and from 2^16 to 2^32 for mt_lutxy. mt_lutxyz would require 2^48 entries, totally impractical under the current design.

[-Vit-]

Dogway: I'm not sure if you script edit does what you want. QTGMC denoises the interlaced source, because it attempts to retain noise/grain that would be destroyed by the deinterlacing. You're denoising to 16-bit, then immediately adding dither at this still-interlaced stage. So QTGMC has to deinterlace your dither..?! That would definitely make a mess of ordered dither, but as you're using Floyd-Steinberg then it won't be quite so bad. However, asking NNEDI3 to interpolate every other line of a Floyd-Steinberg dither is rather odd...

You're not getting 16-bit QTGMC processing, you're just adding dither to your 8-bit source at an early stage. Wouldn't you be better just 16-bit dithering after QTGMC? Using NoiseBypass (EZKeepGrain) to retain the noise as best as possible.

____

There may be scope to add 16-bit processing to QTGMC, but it would be a major overhaul and likely *very* slow. You could use an early dfttest to get 16-bit data (or some other method - I actually don't like having to denoise to get 16-bit data - I often don't want to denoise. Maybe something with DitherPre, but I've had little success with that so far...). But then most other filters in the script would have to operate on 16-bit data. Many of the filters used don't have 16-bit versions so would need manual script workarounds (in particular, 16-bit NNEDI3 using script only...? I have ideas, but...). Or add native 16-bit support to the plugins, yet more work. Anyway, I do think it can be done but it's gonna be a huge and slow script (and that's coming from the author of QTGMC...).

And will it be worth it? This isn't real source 16-bit data, it's a smoothing (denoising) and/or expansion of your source 8-bit data. Will doing this at an earlier stage in your filter chain provide a benefit worth the massive slow down? Will you be able to tell the difference from just adding a 16-bit dither near the end of your script?

[Dogway]

@Fullmetal Encoder: Im not a programmer so I lack the knowledge to do what jmac698 suggests. But my best bet is if something so useful as masktools isn't already ported to 16 bit, is because it's not a so easy task (I once asked cretindesalpes to make Dither_lutxy16).
I just tweaked the script to process dfttest in 16 bit, and then right after dither down. Its not 16 bit chain, and doesn't output stacked 16 bit. Besides that I removed the inflate and deflate custom function Vit said was probably going to remove in next versions. nnedi3 is 8 bit, I also asked cretindesalpes about it, he said it wasn't easy but maybe by just doing what jmac698 said in the above post it would be feasible.

Im not 100% sure but the artifacts could be from QTGMC, or Dither. Im not sure, but I use Dither on a daily basis and never had this kind of issue. Its some colored (purple/green) squares, I will post a sample and debug next time I find it, but its not very common.

edit: @Vit: sorry I posted too late, I don't consider dither anything special, or something you can't scale or modify. Dither is what should be done to every single source in 16 bit or higher when going back to 8. It's not grain. There's nothing wrong in using an improved denoiser (just look how bad banding dfttest creates in 8 bit). By using 16 bit you don't smooth, you rather get more precise results (or less rounding errors), but at a cost, so it depends on the filter if its worth or not. Seeing how slow QTGMC and the probably small benefit one would get in 16 bit with it, I consider not worth it, but doing 16 bit processing of some core filters would be beneficial, like dfttest, mdegrain, and hopefully nnedi3 if it ever gets modified. About DitherPre from experience just use DitherPre1, I had some problems once with DitherPre2.

[Fullmetal Encoder]

And will it be worth it? This isn't real source 16-bit data, it's some kind of smoothing from your source 8-bit data. Will introducing this smoothing at an earlier stage in your filter chain provide a benefit worth the massive slow down? Will you be able to tell the difference from just adding a 16-bit smooth / dither near the end of your script?

Well, every comparison of 8-bit visa vi 10-bit output that I have seen even from 8-bit sources has been very very impressive but as I have never processed in full 10-bit/16-bit pipeline I can't say if it would be worth the time invested. At this point I would have to say not so much but it's a moot question since, as you and Dogway have suggested, if it were easy to implement then it would have been done already. My thinking was that creating a processing chain as thoroughly 16-bit as one could make it would somewhat future-proof encoded files. Some visual gains could be had immediately across the board but obtaining a full 10-bit output hardware chain down the line would reveal even higher quality from files encoded in 10-bit from a 16-bit process. I do plan on buying a 10-bit capable graphics card and monitor in the future.

Issues like these always leave me wishing I had hardcore programming skills so that I could create something that might solve these kinds of problems but short of obtaining a 4-year computer science degree I don't believe it's within the realm of possibility.

I wonder though what the next step will be. It feels to me that with the advent of widespread availability of 10-bit encoding/decoding and the increasing interest in HD that Avisynth and it's associated tools are becoming increasingly obsolete. I also wonder if the entire video production and broadcast industry will end up moving to 10-bit before that next step comes along or vice versa.

[-Vit-]

Dogway: Maybe you missed my comment about your QTGMC edit. You have added dither to an arbitrary intermediate result in the middle of QTGMC. That dither is not getting out of there alive. It's gonna be attacked by NNEDI3, a few MDegrains and a dozen other filters. None of which will treat it properly to it as either 16-bit color or dither. Your Floyd-Steinberg will make a beautiful 8-bit error diffusion pattern, which will then be mangled by everything else that happens in there. Unless I'm missing something important I don't see why you would want to do that there. Why not just denoise/dither after QTGMC?

I do understand the potential benefits of higher bit-depth processing, especially in conjunction with 10-bit output. I guess my issue is that I don't work with sources that suffer much noise or banding, so the recent interest in dither tools has only been of peripheral interest to me. But considering higher bit-depth processing in a more general sense I suppose improved interpolation accuracy may well help during deinterlacing, albeit primarily in a vertical direction (remember a pure deinterlacer is supposed to be lossless, 50% of the output is supposed to remain untouched, regardless of bit-depth). Still the cost in terms of processing time will be hideous from a purely scripted solution. It's the plugin authors you need to be convincing...

[jmac698]

You're saying that nneedi3 is holding you back? Here's a little math. With a unknown function, you can get 16 bits anytime, as long as f(a*m,b*m)=m*f(a,b). Only the author can tell us that. So the idea is to feed f() different bits of the picture and combine them. Imagine f(a,b)=(a+b)/2, a=12.3, b=15.6. Use m=2 and take only the 8 bits part of a,b, (12*2+15*2)/2=27. Now divide down to m: high=13, low=.5. Now use the low bits: (.3+.6)/2=.5 (keeping 1 decimal place here..) Now add low1+low2=0 and msb=+1, so the final answer, msb=14, lsb=0.
Yes it's a lot of steps and slow but you can extract 16bit from *any* plugin if it meets simple criteria. Does needi3 apply? I'd have to do a simple test and can get you the answer.

So in general, there's a very good chance that any 8bit plugin can be manipulated to get 16 bits. I can write a generic function to make 16bits out of any plugin, and with eval, i can be *very* easy to modify an existing plugin.. so I should give a shot at QTGMC. In fact in the best case, I can feed the original code as a string and make *everything* automatically 16 bit! Gonna take a bit of work though.. (I've already written a parser in plain avisynth, btw.. tedious as anything :)

As far as lutxyz needing a lot of memory, you can probably break up the tables and end up being a bit slower... I think there's identities even for trig that if you're smart, you can use a few lookups. Things like the double angle identity can be used to get more bits.

As for speed, it can be up to 3 times slower but remember, for something complex like QTGMC, only a few parts will be that slow so it depends what part is taking the most time, because some of the plugins are 16 bit.. and as plugins move towards 16bit, the whole thing gets faster...

[cretindesalpes]

With a unknown function, you can get 16 bits anytime, as long as f(a*m,b*m)=m*f(a,b).
This is basically what Dither1Pre and Dither2Pre do, but with a very limited success. And most plug-ins are clearly not linear functions.

[jmac698]

Ok, for f(x)=x^n, f(x*m)=f(x)*m^n so any power function can be automatically compensated and accounted for. That means any polynomial function can be made 16bit. Most of the time nnedi3 is bicubic so most pixels will look good, but then your losing the advantage of nnedi3 in the first place. You could try a small interpolation, though. Actually I know how to make a special case for nnedi3...

Also, there is a 16bit masktools: SmoothCustom16

[Dogway]

-Vit- Dither doesn't add any artificial pixels, is not like gradfundb (it doesn't deband, it prevents banding!). Ordered dither is only useful at the end ready for encoding in this context. Prior to that you can keep a 16 bit chain, or use 16 bit filters dithering down to floyd steinberg each time, still this is way better than a 8 bit chain. What I only did was to add better precision to dfttest, and had to dither down for consequent 8 bit processing, thus what you see as noise (which is error diffusion) is just useful source pixel information finding its place in the 8 bit realm. The same useful "noise" you find in every picture (in order to avoid bands/color stepping in 8 bit images), not random thrown noise like addgrain or gradfundb, thats why its not called noise, but dither. I know I didn't do anything wrong in the script, but if you are still insecure ask the gurus. I can delete the script if you think I'm misleading...

jmac698: good one! can I load two clips in smoothcustom for lutxy operations?

About the artifacts, I correct myself, its not QTGMC. I think it can probably be ffdshow FFV1, so Im gonna update that and see if it has been solved.

[jmac698]

@Dog
I'm afraid it only takes a single argument :(

[Nexin]

Any alternative site these files are hosted needing the dll plugin package. From a site that is not Mediafire ?

[Gser]

I was wondering if you can use RepYadif outside of this script?

[-Vit-]

I was wondering if you can use RepYadif outside of this script?
Yes, it's just this:

Repair( Yadif(mode=3), Bob(0,0.5), 2, 0 ) # Try 1,2,3 instead of 2 at the end

[Gser]

Yes, it's just this:

Repair( Yadif(mode=3), Bob(0,0.5), 2, 0 ) # Try 1,2,3 instead of 2 at the end


It does something freaky to the motion though.

[-Vit-]

Hence (Q)TGMC. Edit: Oh, I know what you mean...

[Gser]

Hence (Q)TGMC

Looks good if i remove every other frame.

[-Vit-]

Yes, should have said. It can't really be used stand-alone for double rate. It uses Bob so it introduces bob-flicker. That's what the other parts of the QTGMC algorithm remove.

[Zan7]

Hi.
Is it possible to use QTGMC in a 64-bit workflow?
I don't think I've seen a 64-bit version of NNEDI3 yet.

[jpsdr]

There is a 64 bits version of NNEDI3 (v0.9.2 or v0.9.3), but not of the last version (v0.9.4). I think you can find it in the nnedi thread.
It is possible to use QTGMC in a 64 bit workflow, but using avisynth64_4-16-10, not avisynth64_8-29-10 (crash). Maybe avisynth-x64-20110906 work, i have not tested yet.

[ajp_anton]

What do we lose by not having the latest NNEDI3 in 64-bit?

[aegisofrime]

What do we lose by not having the latest NNEDI3 in 64-bit?

It's supposed to be faster. Hopefully somebody compiles it for 64-bit.

[jpsdr]

Not the most important point. In the latest NNEDI3, author used a different convergence method to build his neural network, wich provide better result than previous versions.

[Zan7]

What do we lose by not having the latest NNEDI3 in 64-bit?

My problem is that I don't know how to mix the 32bit version with all the other 64bit plugins. I tried a few things but Megui always tells me that it can't find nnedi3.

I tried the version jpsdr referred to and it does work.
Don't really know what I would gain if it were the latest nnedi3.

[jpsdr]

Don't really know what I would gain if it were the latest nnedi3.
Try searching in the nnedi thread, what exactly the author has done.

[SamKook]

I started to mess around with some settings and I noticed that the NoiseRestore value is always set to 1.0 according to ShowSettings, even If I change it manually in my script(tried 0,0.2,0.6,2) or change the preset to placebo which should set it to 0.1 according to the doc.
Edit: Slight correction, that only happens if ShowNoise is set to something other than 0 so I guess it doesn't really matter.

The fansub quality checker in me also found a few mistakes in the doc since I finally had time to read it:
1. In "additional plugins", for yadif, you used "beloaded" instead of "be loaded".
2. In "Motion Analysis", for the Overlap setting, you used "compenstion" instead of "compensation".
3. In "Source Match / Lossless", in the third paragraph after the quickstart examples, you used "the the accuracy" instead of "the accuracy".
4. Still in "Source Match / Lossless", in the fourth paragraph after the quickstart examples, you used "liitle" instead of "little".

[-Vit-]

NoiseRestore value is always set to 1.0 according to ShowSettings [...] that only happens if ShowNoise is set to something other than 0
Yes, that's deliberate to make sure there's something to show

The fansub quality checker in me also found a few mistakes in the doc
Thanks, fixed for the next version.

[asarian]

Vit, I've been using your QTGMC for two DVD projects now, and I gotta say, it's brilliant! And I thought: "If only I could use it on non-interlaced material too." Turns out you can, according to the start post. :)

So, can I just let QTGMC do the same correctional stuff on a progressive source, without extra parameters?

[TheSkiller]

So, can I just let QTGMC do the same correctional stuff on a progressive source, without extra parameters?No, you have to specify InputType for progressive input (no deinterlacing). For example InputType=1. Read the included html documentation. ;)

[asarian]

No, you have to specify InputType for progressive input (no deinterlacing). For example InputType=1. Read the included html documentation. ;)

Found it! Thanks. :)

[Amdh]

Hi There,
Thanks for this nice deinterlacing filter.

I'm actually using these settings to deinterlace some SD content :
Preset="Medium",TR2=1,NoiseProcess=0,Sigma=2.0,NoiseRestore=0.3,GrainRestore=0.8,Sharpness=1.3,SLMode=3,SMode=2,EdiThreads=4

The result is pretty good, however I've noticed some cartoon/mosaic like smooth effect on the output. It's really slight and hard to distingush, but it makes the video look less sharp then what it is supposed to be. Could you please suggest some adjustments to the settings to get sharper content while keeping the same preset ?

Thanks in advance :)

[-Vit-]

NoiseProcess=0 is disabling noise processing, so your other noise settings are doing nothing. Looks like you're trying some noise bypass, which does help retain a little detail. With your other settings NoiseProcess=1 would be OK. Or remove all the noise settings and experiment with EZKeepGrain - somewhere between 0 and 1.0 depending on grain in the source.

The temporal smoothing and sharpening of QTGMC can give a very slightly cartoon/plastic/smooth look in some circumstances. You're increasing sharpness, but that might make things worse as the effect is more due to the loss of fine detail coupled with the enhancement of strong features. Getting noise bypass working will help a little. Enabling SourceMatch will help more, but will be slow. SourceMatch changes the default Sharpness to 0.2, increase it carefully.

Increasing slowness
QTGMC( "Medium", SourceMatch=1 )
or
QTGMC( "Medium", SourceMatch=2, Lossless=2 )
or
QTGMC( "Medium", EZKeepGrain=0.5, Sharpness=0.3, SourceMatch=2, Lossless=2 )

[Amdh]

NoiseProcess=0 is disabling noise processing, so your other noise settings are doing nothing. Looks like you're trying some noise bypass, which does help retain a little detail. With your other settings NoiseProcess=1 would be OK. Or remove all the noise settings and experiment with EZKeepGrain - somewhere between 0 and 1.0 depending on grain in the source.

The temporal smoothing and sharpening of QTGMC can give a very slightly cartoon/plastic/smooth look in some circumstances. You're increasing sharpness, but that might make things worse as the effect is more due to the loss of fine detail coupled with the enhancement of strong features. Getting noise bypass working will help a little. Enabling SourceMatch will help more, but will be slow. SourceMatch changes the default Sharpness to 0.2, increase it carefully.

Increasing slowness
QTGMC( "Medium", SourceMatch=1 )
or
QTGMC( "Medium", SourceMatch=2, Lossless=2 )
or
QTGMC( "Medium", EZKeepGrain=0.5, Sharpness=0.3, SourceMatch=2, Lossless=2 )

Thanks for your answer friend .. that was realy helpful :)

[Stereodude]

Has anyone put together a x64 download package for QTGMC?

I looked for the various plugins it uses in x64 versions, but didn't find some of them though I will admit that I didn't spend hours scouring every corner of the internet trying to find them.

[fbs]

While trying to use qtgmc with 2.6MT and modded plugins....

VirtualDub crash report -- build 32842 (release)
--------------------------------------

Disassembly:
025233c0: c9 leave
025233c1: c21000 ret 0010
025233c4: 55 push ebp
025233c5: 8bec mov ebp, esp
025233c7: 83ec4c sub esp, 4ch
025233ca: 33d2 xor edx, edx
025233cc: 2155f8 and [ebp-08h], edx
025233cf: 56 push esi
025233d0: c745fc80000000 mov dword ptr [ebp-04h], 00000080
025233d7: c745d401000000 mov dword ptr [ebp-2ch], 00000001
025233de: c745d802000000 mov dword ptr [ebp-28h], 00000002
025233e5: c745dc03000000 mov dword ptr [ebp-24h], 00000003
025233ec: c745e005000000 mov dword ptr [ebp-20h], 00000005
025233f3: c745e408000000 mov dword ptr [ebp-1ch], 00000008
025233fa: c745e80d000000 mov dword ptr [ebp-18h], 0000000d
02523401: c745ec15000000 mov dword ptr [ebp-14h], 00000015
02523408: c745f022000000 mov dword ptr [ebp-10h], 00000022
0252340f: 57 push edi
02523410: 33c0 xor eax, eax
02523412: 2145f4 and [ebp-0ch], eax
02523415: 33f6 xor esi, esi
02523417: 8d4db4 lea ecx, [ebp-4ch]
0252341a: 8d7dd4 lea edi, [ebp-2ch]
0252341d: 837df400 cmp dword ptr [ebp-0ch], 00h
02523421: 7404 jz 02523427
02523423: 85c0 test eax, eax
02523425: 752c jnz 02523453
02523427: 8945f4 mov [ebp-0ch], eax
0252342a: 8b4508 mov eax, [ebp+08h]
0252342d: 0fb60402 movzx eax, byte ptr [edx+eax]
02523431: 2345fc and eax, [ebp-04h]
02523434: d17dfc sar dword ptr [ebp-04h], 1
02523437: 7508 jnz 02523441
02523439: 42 inc edx
0252343a: c745fc80000000 mov dword ptr [ebp-04h], 00000080
02523441: 85c0 test eax, eax
02523443: 7408 jz 0252344d
02523445: 837df400 cmp dword ptr [ebp-0ch], 00h
02523449: 7502 jnz 0252344d
0252344b: 0337 add esi, [edi]
0252344d: 41 inc ecx
0252344e: 83c704 add edi, 04h
02523451: ebca jmp 0252341d
02523453: 33c9 xor ecx, ecx
02523455: 4e dec esi
02523456: 41 inc ecx
02523457: 85f6 test esi, esi
02523459: 7e21 jle 0252347c
0252345b: 8b4508 mov eax, [ebp+08h]
0252345e: 0fb60402 movzx eax, byte ptr [edx+eax] <-- FAULT
02523462: 2345fc and eax, [ebp-04h]
02523465: d17dfc sar dword ptr [ebp-04h], 1
02523468: 7508 jnz 02523472
0252346a: 42 inc edx
0252346b: c745fc80000000 mov dword ptr [ebp-04h], 00000080
02523472: 03c9 add ecx, ecx
02523474: 85c0 test eax, eax
02523476: 7401 jz 02523479
02523478: 41 inc ecx
02523479: 4e dec esi
0252347a: 75df jnz 0252345b
0252347c: 8b75f8 mov esi, [ebp-08h]
0252347f: 8b450c mov eax, [ebp+0ch]
02523482: 49 dec ecx
02523483: ff45f8 inc dword ptr [ebp-08h]
02523486: 817df800010000 cmp dword ptr [ebp-08h], 00000100
0252348d: 890cb0 mov [eax+esi*4], ecx
02523490: 0f8c7affffff jl 02523410
02523496: 5f pop edi
02523497: 8d4201 lea eax, [edx+01h]
0252349a: 5e pop esi
0252349b: c9 leave
0252349c: c20800 ret 0008
0252349f: 55 push ebp
025234a0: 8bec mov ebp, esp
025234a2: 83ec10 sub esp, 10h
025234a5: ff4508 inc dword ptr [ebp+08h]
025234a8: 8b4508 mov eax, [ebp+08h]
025234ab: ff4508 inc dword ptr [ebp+08h]
025234ae: 53 push ebx
025234af: 0fb618 movzx ebx, byte ptr [eax]
025234b2: 8b450c mov eax, [ebp+0ch]
025234b5: 03c8 add ecx, eax
025234b7: 895df8 mov [ebp-08h], ebx
025234ba: d1eb shr ebx, 1
025234bc: c7 db 0c7h
025234bd: 45 inc ebp
025234be: fc cld
025234bf: 80 db 80h

Built on Aegis on Fri Dec 24 13:18:44 2010 using compiler version 1400

Windows 6.1 (Windows Vista x64 build 7601) [Service Pack 1]

EAX = 19d4c8b8
EBX = 19d4dcf0
ECX = 00000000
EDX = 00003748
EBP = 19d4c898
ESI = 1e44a25e
EDI = 19d4c8b8
ESP = 19d4c844
EIP = 0252345e
EFLAGS = 00010202
FPUCW = 027f
FPUTW = ffff

Crash reason: Access Violation

Crash context:
An out-of-bounds memory access (access violation) occurred in module 'lagarith'...

...reading address 19D50000.

Pointer dumps:

EAX 19d4c8b8: 0007480e 052ddf20 00000080 00000000 00000000 0000d11c 00000001 0001878f
EBX 19d4dcf0: 00000000 fefefefe fefefefe fefefefe fefefefe 7712e36c 6afd4184 00000001
ESI 1e44a25a: 7e7e7e7e 7f7f7f7f 7d7d7d7e 7d7e7d7d 7d7d7d7d 7d7d7d7d 7c7c7c7d 7d7c7c7c
EDI 19d4c8b8: 0007480e 052ddf20 00000080 00000000 00000000 0000d11c 00000001 0001878f
ESP 19d4c840: 00000008 4ab9de11 04580030 00043a3b 0000000a 000447d1 0000000b 0004516a
19d4c860: 0000000c 0004585e 0000000d 00000001 00000002 00000003 00000005 00000008
19d4c880: 0000000d 00000015 00000022 00000080 00000002 00000080 19d4dcd4 0252379c
19d4c8a0: 19d4c8b8 19d4d8b8 4ab9de11 04580030 04717fa0 00000007 0007480e 052ddf20
EBP 19d4c898: 19d4dcd4 0252379c 19d4c8b8 19d4d8b8 4ab9de11 04580030 04717fa0 00000007
19d4c8b8: 0007480e 052ddf20 00000080 00000000 00000000 0000d11c 00000001 0001878f
19d4c8d8: 00000002 00022937 00000003 0002a722 00000004 000305ad 00000005 00034a89
19d4c8f8: 00000006 00037c4c 00000007 0003a13c 00000008 0003bd99 00000009 0003d44c

Thread call stack:
0252345e: lagarith!0002345e
0252379c: lagarith!0002379c
02523756: lagarith!00023756
025238d8: lagarith!000238d8
7712e36c: ntdll!RtlInitUnicodeString [77100000+2e208+164]
030a7355: mt_masktools-26!00017355
030ba3ea: mt_masktools-26!_AvisynthPluginInit2@4 [03090000+202b0+a13a]
7712e003: ntdll!RtlFreeHeap [77100000+2df85+7e]
7712e36c: ntdll!RtlInitUnicodeString [77100000+2e208+164]
03137497: mt_masktools-26!_AvisynthPluginInit2@4 [03090000+202b0+871e7]
030b2654: mt_masktools-26!_AvisynthPluginInit2@4 [03090000+202b0+23a4]
030a7355: mt_masktools-26!00017355
75ba38ce: kernel32!GetThreadLocale [75b90000+1359f+32f]
75ba38db: kernel32!GetThreadLocale [75b90000+1359f+33c]
75ba14dd: kernel32!HeapFree [75b90000+114c9+14]
7617247a: KERNELBASE!CompareStringW [76150000+22442+38]
76170cc4: KERNELBASE!CompareStringA [76150000+20b87+13d]
76170d43: KERNELBASE!CompareStringA [76150000+20b87+1bc]
030ba437: mt_masktools-26!_AvisynthPluginInit2@4 [03090000+202b0+a187]
030af707: mt_masktools-26!0001f707
030b7f7d: mt_masktools-26!_AvisynthPluginInit2@4 [03090000+202b0+7ccd]
7711f8f5: ntdll!NtReadFile [77100000+1f8e0+15]
7615d348: KERNELBASE!ReadFile [76150000+d230+118]
7615d39a: KERNELBASE!ReadFile [76150000+d230+16a]
7615d39a: KERNELBASE!ReadFile [76150000+d230+16a]
75ba3ef7: kernel32!ReadFile [75b90000+13ea3+54]
02520365: lagarith!00020365
0252063d: lagarith!0002063d
02524132: lagarith!DriverProc [02500000+23fe0+152]
6d841759: MSVFW32!ICSendMessage [6d840000+1728+31]
6d844e27: MSVFW32!ICDecompress [6d840000+4dea+3d]
6be06551: AviSynth!DllCanUnloadNow [6bd50000+d420+a9131]
6be07681: AviSynth!DllCanUnloadNow [6bd50000+d420+aa261]
6bd51206: AviSynth!00001206
6bd53647: AviSynth!00003647
6bd5c69f: AviSynth!avs_delete_script_environment [6bd50000+8030+466f]
6bd5ac54: AviSynth!avs_delete_script_environment [6bd50000+8030+2c24]
6bd5a7e8: AviSynth!avs_delete_script_environment [6bd50000+8030+27b8]
6bd5be7a: AviSynth!avs_delete_script_environment [6bd50000+8030+3e4a]
6bd5ac54: AviSynth!avs_delete_script_environment [6bd50000+8030+2c24]
6bdebe20: AviSynth!DllCanUnloadNow [6bd50000+d420+8ea00]
7712e003: ntdll!RtlFreeHeap [77100000+2df85+7e]
6bd5be7a: AviSynth!avs_delete_script_environment [6bd50000+8030+3e4a]
6bd5ac54: AviSynth!avs_delete_script_environment [6bd50000+8030+2c24]
0254146e: ChromaShift!0000146e
030ba437: mt_masktools-26!_AvisynthPluginInit2@4 [03090000+202b0+a187]
030af707: mt_masktools-26!0001f707
030b7f7d: mt_masktools-26!_AvisynthPluginInit2@4 [03090000+202b0+7ccd]
030a7355: mt_masktools-26!00017355
030ba3ea: mt_masktools-26!_AvisynthPluginInit2@4 [03090000+202b0+a13a]
7712e003: ntdll!RtlFreeHeap [77100000+2df85+7e]
75ba14dd: kernel32!HeapFree [75b90000+114c9+14]
03137497: mt_masktools-26!_AvisynthPluginInit2@4 [03090000+202b0+871e7]
030b2654: mt_masktools-26!_AvisynthPluginInit2@4 [03090000+202b0+23a4]
030ba437: mt_masktools-26!_AvisynthPluginInit2@4 [03090000+202b0+a187]
030af707: mt_masktools-26!0001f707
030b7f7d: mt_masktools-26!_AvisynthPluginInit2@4 [03090000+202b0+7ccd]
030c2093: mt_masktools-26!_AvisynthPluginInit2@4 [03090000+202b0+11de3]
0309e93c: mt_masktools-26!0000e93c
6bd5be7a: AviSynth!avs_delete_script_environment [6bd50000+8030+3e4a]
6bd5ac54: AviSynth!avs_delete_script_environment [6bd50000+8030+2c24]
03137497: mt_masktools-26!_AvisynthPluginInit2@4 [03090000+202b0+871e7]
6bd8f272: AviSynth!DllCanUnloadNow [6bd50000+d420+31e52]
6bdda522: AviSynth!DllCanUnloadNow [6bd50000+d420+7d102]
6bd905af: AviSynth!DllCanUnloadNow [6bd50000+d420+3318f]
6bd5a7e8: AviSynth!avs_delete_script_environment [6bd50000+8030+27b8]
6bd5be7a: AviSynth!avs_delete_script_environment [6bd50000+8030+3e4a]
6bd5ac54: AviSynth!avs_delete_script_environment [6bd50000+8030+2c24]
6bd93c0f: AviSynth!DllCanUnloadNow [6bd50000+d420+367ef]
6bd5be7a: AviSynth!avs_delete_script_environment [6bd50000+8030+3e4a]
6bd5ac54: AviSynth!avs_delete_script_environment [6bd50000+8030+2c24]
6bdce4ca: AviSynth!DllCanUnloadNow [6bd50000+d420+710aa]
6bd5be7a: AviSynth!avs_delete_script_environment [6bd50000+8030+3e4a]
6bd5ac54: AviSynth!avs_delete_script_environment [6bd50000+8030+2c24]
6bdce480: AviSynth!DllCanUnloadNow [6bd50000+d420+71060]
6bd5be7a: AviSynth!avs_delete_script_environment [6bd50000+8030+3e4a]
6bd5ac54: AviSynth!avs_delete_script_environment [6bd50000+8030+2c24]
030b67dc: mt_masktools-26!_AvisynthPluginInit2@4 [03090000+202b0+652c]
030ba3ea: mt_masktools-26!_AvisynthPluginInit2@4 [03090000+202b0+a13a]
7712e003: ntdll!RtlFreeHeap [77100000+2df85+7e]
75ba14dd: kernel32!HeapFree [75b90000+114c9+14]
03137497: mt_masktools-26!_AvisynthPluginInit2@4 [03090000+202b0+871e7]
6bd5be7a: AviSynth!avs_delete_script_environment [6bd50000+8030+3e4a]
6bd5a85a: AviSynth!avs_delete_script_environment [6bd50000+8030+282a]
6d206b93: RemoveGrainSSE2!00006b93
6bd5be7a: AviSynth!avs_delete_script_environment [6bd50000+8030+3e4a]
6bd5ac54: AviSynth!avs_delete_script_environment [6bd50000+8030+2c24]
6bdd986d: AviSynth!DllCanUnloadNow [6bd50000+d420+7c44d]
7712e36c: ntdll!RtlInitUnicodeString [77100000+2e208+164]
6bdda522: AviSynth!DllCanUnloadNow [6bd50000+d420+7d102]
6bd5be7a: AviSynth!avs_delete_script_environment [6bd50000+8030+3e4a]
6bd5ac54: AviSynth!avs_delete_script_environment [6bd50000+8030+2c24]
6bdce480: AviSynth!DllCanUnloadNow [6bd50000+d420+71060]
6bd5be7a: AviSynth!avs_delete_script_environment [6bd50000+8030+3e4a]
6bd5ac54: AviSynth!avs_delete_script_environment [6bd50000+8030+2c24]
6acef5f4: mvtools2!_AvisynthPluginInit2@4 [6acb0000+6660+38f94]
6bd5be7a: AviSynth!avs_delete_script_environment [6bd50000+8030+3e4a]
6bd5ac54: AviSynth!avs_delete_script_environment [6bd50000+8030+2c24]
6acbb1e8: mvtools2!_AvisynthPluginInit2@4 [6acb0000+6660+4b88]
7712e003: ntdll!RtlFreeHeap [77100000+2df85+7e]

-- End of report


setmtmode(5, 7)

AVISource("d:\ds000.avi", true)

setmtmode(2)

AssumeTFF()
crop(4, 0, -8, -12)
ChromaShift(C=-4,L=-2)

qtgmc(preset="fast",TR0=2,TR1=2,TR2=2,NoisePreset="slow", noiserestore=0,chromanoise=true,GrainRestore=0,noiseprocess=1, ShowNoise=0,sigma=1,RepChroma=true,stabilizenoise=false, Sharpness=2)

converttorgb32()
distributor()

Why it always points lagarith.dll as the culprit? ~~ without MT it doesn't crash

[nhope]

Has anyone put together a x64 download package for QTGMC?
Links to some of them are here (http://www.bubblevision.com/underwater-video/Vegas-YouTube-Vimeo.htm#64bit).

[TheRyuu]

Why it always points lagarith.dll as the culprit? ~~ without MT it doesn't crash

Isn't the reason for a crash almost always because your ran out of memory when it comes to SetMT?

[Didée]

Do not put "Distributor()" in the script when you load in a VFW application like Vdub. (It's added automatically for VfW - if you set it manually nonetheless, the number of Avisynth threads gets squared - 7*7 = 49 threads for Avisynth's filterchain. o-o)

Maybe Lagarith doesn't like to be used in an MT script either, but I'm not sure.

[-Vit-]

Yes, try removing Distributor().
Add EdiThreads=1 to the QTGMC line
Try setmtmode(5, 4), or even setmtmode(5, 2)
Try something other than Lagarith - suggest huffyuv

[fbs]

Thanks Didée and TheRyuu.. I'll remove distributor() but I would like to know what exactly is going wrong.. how can I know if it's running out of memory or if lagarith "doesn't like MT" ? Sometimes it works for like 20(!!) hours and then it crashes..

[fbs]

-Vit-, I had problems with huffyuv in win7-x64 with vegas 10 x64.. lagarith is the only codec which works here, apart from this mt issue :(

with edithreads=1, it's incredibly slow.. vdub says it will take 3 days and 10 hours to complete.. cpu usage is ~30%. 25 threads and 930MB ram usage

I'm not in a hurry but I'll have like 10 more videos like this to process... they're VHS-rips..
I'd like to come to a *stable and not too slow solution* then I'll use it to every video..

[TheRyuu]

I don't really think it has anything to do with the codec used.

[fbs]

I don't really think it has anything to do with the codec used.

I don't think so also...

[-Vit-]

What resolution is the material you are processing?

The settings you are using are fairly extreme. You've chosen the "Fast" preset, but then overridden settings to get something nearer the "Very Slow" setting. Then adding "Slow" noise processing which gives you a motion-compensated dfttest, which is very slow.
Your settings are perfectly workable on SD, but on the edge of unusable for 1080p due to speed and memory considerations.

[Stereodude]

Links to some of them are here (http://www.bubblevision.com/underwater-video/Vegas-YouTube-Vimeo.htm#64bit).Thanks. What do you do about SSE2tools though? That's the one I can't find in a x64 version. Or if you stick with YV12 is it not needed?

[fbs]

720x480x30 YV12
NOW I'm using:
qtgmc(preset="very slow",EdiThreads=12,NoisePreset="slow",ChromaMotion=false,
noiserestore=0,chromanoise=true,GrainRestore=0,noiseprocess=1, sigma=1,RepChroma=false,sbb=0,Sharpness=2)

I put EdiThreads=12 (instead of 1 as before) because I disabled MT completely.. it seems stable.. but it's still too slow: 2 days to complete.

My aim is to get the image more stable (and deinterlaced of course) to feed DeShaker in virtualdub..

[-Vit-]

QTGMC stabilizes in the sense of removing bob-flicker, not camera shake. All but the fastest presets will remove bob-flicker very well.

NoisePreset="Slow" uses motion compensated dfttest. That is slow. You probably don't need such intense denoising with a sigma of just 1.0. I suggest you drop the NoisePreset and go with the defaults. Add DenoiseMC=true if you really need the motion-compensation, but of course that will be slower again.
However, dfttest is internally threaded, the default number of threads comes from the EdiThreads setting (can be overridden with the DftThreads setting). One reason you may have been tempted to increase EdiThreads is because you can fill up your CPU load by giving more threads to dfttest. Nevertheless, it will still be slower than the default fft3dfilter.

On an i7 running at 3.6GHz I get 8fps with your settings single threaded, 12fps dropping the NoisePreset, then 34fps with 10 threads and NoisePreset dropped (all using Lagarith in Vdub). I don't know the typical performance of a system like yours, but a processing time in days (even single-threaded) indicates either your system is too slow to run these settings, or there's problem with it, or that your workflow out of Vegas is an issue.

Suggestions:
- Reduce settings, certainly that NoisePreset, probably the main preset too (e.g. use a faster preset, but add TR2=2 to get the smoothing you appear to be after).
- Maybe export to Lagarith without processing in a first pass, then in a second pass deinterlace that output - removing Vegas from the equation [this may improve stability and/or speed?].
- For MT you really should reduce EdiThreads to 3 or less for that system. EdiThreads will eat memory fast on MT, especially with high main thread counts. CPU usage can be sub-optimal on MT with the heaviest processing (especially temporal), no one has yet suggested a compelling reason why. Only way round that is to run multiple MT processes, but as you're struggling with one process then you're better selecting more modest settings that will cap out your CPU.

Thanks. What do you do about SSE2tools though? That's the one I can't find in a x64 version. Or if you stick with YV12 is it not needed?
Yes, that's correct SSE2Tools is only used for YUY2

[fbs]

I'm not trying to reduce camera shake with qtgmc, for camera shake I'm using DeShaker on vdub.. it is kinda cpu consuming but not as much as qtgmc.
It's first pass was done days ago, deinterlacing using vdub's yadif instead of qtgmc, as it just need to calculate motion vectors.. it was very fast that way.

What I'm doing now is opening the original 3h35m .avi, deinterlacing+denoising with qtgmc, and feeding vdub for it's DeShaker plugin (the 2nd pass)..
After that, the resulting denoised+deshaked .avi will be processed by vegas for color correction, edits, and re-encoding to MPEG2 (DVD)..

These settings I'm using are not blind, I compared frame-by-frame with interleave(orig,settings1,settings2).. of course I could reduce them to get speed, but I won't feel good about it.. :( :P This would be a last resort..

Once I get a "stable version" of my avs script, I will use it with more 10 videos... all of them being 2-3h long..

Without MT, things get so slow here because the cpu keeps simply IDLE.. Even raising edithreads to 12 or whatever won't help.. avs-mt is the only way of getting acceptable speed ~~ I just wish to know WHY it crashes so I could try to fix it..

Right now I'm running it again with 8 threads and 3 edithreads.. ~3.5fps, 90% cpu usage, 950mb memory usage, 67 threads on virtualdub.exe. I think I'm far away from any memory limits.. Maybe I should try 2.58MT or find someone who could understand the crashinfo.txt file better than me? I used to debug on unix, but windows is another world

[SamKook]

I don't have that much experience, but when avisynth crashes after a few hours(I often use placebo for qtmgc with 7-8 threads), I just add "SetMemoryMax(1024)" at the beginning of the script and it will prevent it from crashing.

[fbs]

Another day, another crash after 5h.. now trying again with:

SetMemoryMax(800)
SetMTMode(3, 8)
AviSource(whatever)
SetMTMode(2)
QTGMC(preset="Medium", TR2=2, EdiThreads=3, NoisePreset="slow", NoiseRestore=0, ChromaNoise=true, GrainRestore=0, NoiseProcess=1, Sigma=1.7,RepChroma=false, Sharpness=2)

Now virtualdub.exe is using 1.2gb of ram instead of 900. Before, the filter limit was 512 as I didn't specify a SetMemoryMax, right? Now there are much memory for its filters but less total memory for vdub as it seems to exist a 1.5gb limit for it.. 69 threads, 90-100% cpu usage ~~ Estimated time: exactly 1 day.. ~4.3fps

[Stereodude]

Maybe your overclocked PC isn't stable after 5 hours?

[-Vit-]

It's possible, I have had a previous report of exactly that problem, cured by reducing the OC. MT QTGMC is a *very* tough stress test - you are taxing your system heavily running it for 10s of hours.

Or it could be a yet-to-be-solved thread-safety issue, I don't get reports of them so often these days, but probably some left. SEt's 2.6MT thread is the one to watch there.

fbs, have you tried the two pass approach I noted above?

BTW: I assume it's a posting typo above, but your SetMemoryMax and SetMTMode should come first in the script...

[fbs]

Maybe your overclocked PC isn't stable after 5 hours?

My pc is NOT overclocked, who said that? Prime95 runs fine for ~8 hours (then I quit).. No hardware errors ever.. No blue screens ever.. CPU temperature is ok also. Rock-solid machine, baby!

-Vit-, I'm not yet using Vegas.. I'm just running VD, opening the .avs with it, setting some filters in it and saving to another .avi (with the same codec I always use) with it.

Now it's been running for 12 hours.. 50% of it.. I'll sleep with fingers crossed.. \o/

Oh, and the low ~4.3fps is because DeShaker is also a cpu-hungry VD-filter.. but I guarantee it is stable.. have used it a lot.

[-Vit-]

Can be better to avoid other CPU hungry processes at the same time as MT QTGMC. You could try two-passing QTGMC, then Deshaker. It may improve stability and/or overall speed.

[fbs]

Good news.. It worked! It looks like I needed to raise the default memory limit.. but also, I didn't use mode 5 as recommended (I read somewhere it is overkill): I used mode 3 for avisource() and 2 for the rest.
It would be awesome if avisynth could show a message "out of memory" or something like that instead of crashing.. but... :)

Thanks everyone who cared, specially the incredible -Vit- :D

Oh and just for the record, with medium/slow I get 10.7fps with just QTGMC.

[-Vit-]

Glad you got it working.
I've tweaked the MT instructions in the OP to better describe the correct MT setup. I now suggest mode 3 for the source filter (5 as a fallback). And I only recommend SEt's 2.6MT + modded plugins - removed links to other MT versions.

[fbs]

Cool! After a 3:30h long video I can say this combination (SEt's 2.6MT + modded plugins) IS stable as long as you change setmemorymax based on the number of threads you choose.. In my case it was 800mb for 8 threads, with SD material (720x480).. (using Medium preset.. I don't know if slowes presets would use more memory...)

Oh, and you can't just use a super high value for SetMemoryMax because it seems to exists a 1.5gb limit for 32 bit process. Sources: http://www.virtualdub.org/blog/pivot/entry.php?id=346 and http://forums.taleworlds.com/index.php?topic=113029.0

And it seems mode 3 uses much less memory than 5, also.

You can say these in the OP too to help other ppl.. ~~ I bet many just give up after 2-3 crashes, but as we say here, brazilians NEVER give up.. :D

[SamKook]

Oh, and you can't just use a super high value for SetMemoryMax because it seems to exists a 1.5gb limit for 32 bit process.

It depends on the software you use and your OS. If you have a 64bit OS and the software is made for it, you can go to up to 4GB per 32 bit process. But the most likely limit is around 2GB.

[Stereodude]

FWIW, QTGMC isn't usable for me with a x64 flow (AVIsynth 2.5.8 x64). I got it installed and operational (sort of). I can load an .avs file in Veedub64.exe. It displays an image (as expected), but after advancing a frames at a time forward 43 times (so frame 43) VirtualDub stops responding for several minutes before display the 43rd frame. Advancing to the 44th frame hangs VD again for several minutes. x264 just sits and spins before even encoding a single frame. I gave up after about 10 minutes and killed it.

Are there some tricks to making it work better?

I put the 4 plugins (mvtools2.dll, nnedi3_64.dll, RemoveGrain.dll, & mt_masktools-25-x64.dll) in C:\Program Files (x86)\AviSynth 2.5\plugins64

My .avs is as follows
LoadPlugin("C:\HDTV Tools\DGdecode\DGDecode.dll")
Import("C:\HDTV Tools\QTGMC\QTGMC-3.32.avsi")

MPEG2Source("JLS.d2v", idct=5, moderate_h=30, moderate_v=60, cpu2="xxoooo").trim(108,8524+49)
QTGMC( Preset="Slow" ).crop(2, 0, -2, -4).Spline36Resize(1280,720).FadeIn(60).FadeOut(60)

The MPEG-2 source is 1920x1080i.

[fbs]

It depends on the software you use and your OS. If you have a 64bit OS and the software is made for it, you can go to up to 4GB per 32 bit process. But the most likely limit is around 2GB.

Sure.. but Avisynth 2.6MT is 32 bits, so.... :)

[SamKook]

Even if it's 32 bit, there's a difference if your OS is 64bit or 32bit. A 64bit OS can allocate more memory to a single 32 bit process than a 32bit OS can.

Here's a source: http://msdn.microsoft.com/en-us/library/aa366778(v=vs.85).aspx

[fbs]

On x64 windows it WOULD be 4gb if the app in question was compiled with "IMAGE_FILE_LARGE_ADDRESS_AWARE" (/LARGEADDRESSAWARE linker option). The table on that page clearly shows it.

If think this could be tested by an avisynth script which keeps allocating dummy memory till it crash.. when it crashes you look at the task manager to see how much memory its using. I've never seen any app using more than 1.5gb EVER. Maybe nothing is compiled with /largeaddressaware?

[SamKook]

I've never seen an app using more than 2GB, but I've seen some using more than 1.5. Here's my latest script memory usage a minute or two after starting it(first time I had a crash using setmemorymax to 1024):
encoding with x264(1782MB used) http://imgur.com/4iod8
and with vdub in xvid(1271MB used) http://imgur.com/ScMNP

x264 reserved about 2GB for itself and vdub only 1.5GB so depending on the app you're using, the memory available can differ.

I assume there's a drawback to using IMAGE_FILE_LARGE_ADDRESS_AWARE or that it's a new thing or else I don't see why people aren't using it on every software. We could definitely benefit from it when we're using QTGMC.

[SEt]

IMAGE_FILE_LARGE_ADDRESS_AWARE is just one bit flag in exe header. It can be trivially enabled with hex editor or editbin. Example of enabled by default program: x264. I don't see any drawbacks of it other than letting buggy program accidently consume 4GB instead of 2GB, and no it's not a new thing at all - just no one bothers with reading and understanding linker options.

[SamKook]

Good to know, I didn't think about ignorance.

But I've never seen x264 use more than 2GB, is it avisynth that's limiting it or are the crashes due to too much memory used(seems to be the popular belief since limiting the memory in avisynth help prevent the crashes) only occur because avisynth MT is buggy?

[SEt]

First of all it depends on your OS: you'll see 4GB limit only on x64 one and I had experience when 3GB memory usage was exactly what my script required. Avisynth can't limit it - it depends only on exe, not on dlls. Also Avisynth MT is not buggy (applies only to 32-bit latest version, bugginess of plugins is a different question), but efficiency of its caching definitely can be improved.

[sneaker_ger]

First of all it depends on your OS: you'll see 4GB limit only on x64 one and I had experience when 3GB memory usage was exactly what my script required.

I guess it depends on whether you are using a pipe or not. If you are using 32bit AviSynth and directly opening the script with 32bit x264 you are limited to the combined 2 GB limit. With a pipe you can get to 2 GB + 2GB and with x64 you are practically not limited at all.

[TheRyuu]

I guess it depends on whether you are using a pipe or not. If you are using 32bit AviSynth and directly opening the script with 32bit x264 you are limited to the combined 2 GB limit. With a pipe you can get to 2 GB + 2GB and with x64 you are practically not limited at all.

You still get 4GB without the pipe if you're on a 64-bit OS and x264 was compiled with -Wl,--large-address-aware (which is the default in more recent versions). If you do use a pipe then you'll get 4GB + 4GB although x264 generally doesn't need anywhere near that much.

[StainlessS]

On a 32 bit system, you have 4GB of address space, the upper 2GB for the OS, and the lower
2GB for applications (flat memory model).
However some systems have Physical Address Extension (PAE) which extends that:-
http://en.wikipedia.org/wiki/Physical_Address_Extension

Extended or not, Microsoft might want to limit you unless you cough up more wonga (cash).

Also I think my (32bit) has something called "ET64" which is I think also to do with the same type of
thing (I could not find a good reference for it).

I would assume that if you are using a pipe, it is more related to file pointer rather than address space
(although it could of course be a mixture of the two), I have no idea on current limitations of NTFS.

EDIT: I think that even on 64bit MS servers, the current memory limitation is 64GB, irrespective of machine capability
that would not constrain eg Linux.

[fbs]

As I said before, "It would be awesome if avisynth could show a message "out of memory" or something like that instead of crashing.. but..." :P

[sneaker_ger]

You still get 4GB without the pipe if you're on a 64-bit OS and x264 was compiled with -Wl,--large-address-aware (which is the default in more recent versions). If you do use a pipe then you'll get 4GB + 4GB although x264 generally doesn't need anywhere near that much.

I wasn't aware of that. So "4GB + 4GB" means that AviSynth also supports it or was that a typo?

[TheRyuu]

I wasn't aware of that. So "4GB + 4GB" means that AviSynth also supports it or was that a typo?

It has nothing to do with avisynth as that's a DLL. It has everything to do with the executable file which is using it and whether or not it has been compiled with /LARGEADDRESSAWARE (-Wl,--large-address-aware for mingw, furthermore this can be done on any executable with editbin and doesn't have to be done at link/compile time).

I was simply using your example avs2yuv -> x264. Both processes will get 4GB to work with provided they have the proper flag set (and are running on a 64-bit Windows operating system).

[sneaker_ger]

I didn't mention any specific piping tool, but ok, it depends on the tool opening the script, not on how AviSynth itself was compiled.

[yup]

-Vit-!
Please advice combined use QTGMC and MVExtras for denoising and upsizing.
SetMemoryMax(768)
Import("QTGMC-3.32.avsi")
SetmtMode(3,4)
DirectShowSource("tv.avi",pixel_type="YUY2")# PAL VHS capture
SetmtMode(2,4)
AssumeTFF()
Crop(16,12,-16,-12)
#ConvertToYV12(interlaced=true)
QTGMC(Preset="Slower", EdiMode="NNEDI3",NNSize=0,NNeurons=0,EdiQual=1, ForceTR=2,Edithreads=1)

nnedi3_rpow2(2,nsize=4,nns=0,cshift="Spline64Resize", threads=1)
super = MSuper( pel=2, hpad=16,vpad=16)


bVec1 = QTGMC_bVec1.MScaleVectors( 2 )
fVec1 = QTGMC_fVec1.MScaleVectors( 2 )
bVec2 = QTGMC_bVec2.MScaleVectors( 2 )
fVec2 = QTGMC_fVec2.MScaleVectors( 2 )

MDegrain2( super, bVec1,fVec1, bVec2,fVec2 )
Spline64Resize(960,720)
ConvertToYV12(interlaced=false)
AddBorders(160,0,160,0)#BD compliant
Script work, but please confirm.
yup.

[-Vit-]

That looks fine. Reusing QTGMC's vectors, doubling their size to denoise at 2x resolution. I assume that gives you a result you prefer from MDegrain2?

[Dogway]

I found that running QTGMC alone goes well, but when something else is added afterwards CPU drops to around 60%, I tested using srestore(), or mdegrain reusing vectors. It only keeps 100% when very light processing is done like crop, trims, and resizing etc

My parameters are quite light this time:
QTGMC(tr2=3,preset="medium",subpel=2,chromamotion=false,edithreads=2)


Also how can I make QTGMC to implicit denoise using mdegrain? is it used by default? or do I need to set EZDenoise on? I mean if it is used by default it would be a bit redundant to denoise afterwards with smdegrain or MCTemporalDegrain for instance. Otherwise I could turn it off, reuse vectors and use smdegrain.

edit:
Also it seems that when reusing vectors you can't set hpad vpad to 0 in the new msuper.

[-Vit-]

I don't know what is responsible for stalling complex avisynth scripts. Doesn't just affect QTGMC, seems to be related to temporal processing and I suspect the avisynth cache. To ensure 100% add more threads and/or processes.

One of the final steps with default QTGMC settings is an MDegrainX, where X is the QTGMC setting TR2, and thSAD for the degrain comes from QTGMC setting ThSAD2.
However, there are some processing steps that follow:
- Most importantly the Rep2 step, which attempts to ensure that only bob-shimmer artefacts are altered by the temporal smoothing/re-sharpening. That removes some of the effects of this final degrain. You could set Rep1=4 and Rep2=0, then the final MDegrain is not constrained (although there are some other implications regarding some inner processing - try it and see).
- Also noise bypass, true lossless, a rarely used sharpness limiting mode and shutter blur all occur after the final degrain. None of that affects default settings.

Regarding hpad,vpad and vector reuse: Those padding values have been troublesome for QTGMC, having been buggy in MVTools earlier, and now relevant for vector reuse. They will usually be 8, 8 for most resolutions. Work on that basis. They will be promoted to QTGMC settings, you could do it yourself easily - they are set on line 342.

[Dogway]

So superclip padding must match the padding used to make the vectors...(?) Makes sense. My intention wasn't really to use 0 padding in QTGMC as I know it could hit accuracy, but to gain some speed after the huge CPU drop in the new msuper. As for the code I just defaulted them to blocksize.

Regarding perfomance I made some more tests:
smdegrain
srestore(23.976)
This also makes my CPU jump from 50% most of the time to 80% eventually. So you are right about the avisynth cache. Despite each of these filters, QTGMC, smdegrain or srestore run 100% when used alone. Nothing we can do I guess, I only have 2 threads but anyone can test to chain 2 temporal filters to see if they can achieve 100% on more threads.
edit: out of curiosity I tried to take srestore out of the equation. Did QTGMC+smdegrain and CPU was 100% so that points to srestore, I don't know if it's worth commenting this on its thread...

I will try your suggestions for mdegrain denoising. Thanks for the help.

[Gser]

I was wondering what exactly does Tuning's settings "None", "DV-SD", "DV-HD" set?

[-Vit-]

Just blocksize. Larger blocksize for HD material on faster settings. I think there's much more scope for tunings, but I have never had time to develop/test them. Next version auto-detects resolution to select settings defaults, so those tunings will be removed.

[kypec]

Next version auto-detects resolution to select settings defaults, so those tunings will be removed.
Wow, I just can't wait when that new version will be published at last! :D :thanks:

[Slasher]

I don't know if this issue has been discussed previousy in this thread (didn't find anything with search):
I have an HD partially interlaced source. I initially used tdeint but I discovered QTGMC and it's way better for the interlaced scenes. But I'm having some issues for the progressive scenes, tdeint is identical with the source but QTGMC has a slight distortion. I've read the documentation extensively trying to find a switch to fix this but I couldn't find what I needed; I tried different presets, sharpness, noise and sourcematch switches. To illustrate the problem, please check these screens:

http://thumbnails21.imagebam.com/16071/53acd6160703614.jpg (http://www.imagebam.com/image/53acd6160703614)http://thumbnails50.imagebam.com/16071/756636160703617.jpg (http://www.imagebam.com/image/756636160703617)http://thumbnails63.imagebam.com/16071/a8f3ea160703620.jpg (http://www.imagebam.com/image/a8f3ea160703620)

http://thumbnails64.imagebam.com/16071/9bd79e160703624.jpg (http://www.imagebam.com/image/9bd79e160703624)http://thumbnails65.imagebam.com/16071/cbfa18160703629.jpg (http://www.imagebam.com/image/cbfa18160703629)http://thumbnails45.imagebam.com/16071/092cb6160703634.jpg (http://www.imagebam.com/image/092cb6160703634)

The skier's details (clothing, skis, ski poles etc) are distorted from the original source. Best observed at 200% zoom.

[-Vit-]

QTGMC has a progressive mode, but that is to repair shimmer in progressive material.
Its normal deinterlacing modes are not intended for use on progressive material.

kypec: Sorry, the changes I have mentioned in QTGMC were made months ago and are WIP. I am busy atm so development is stalled, but not for much longer I hope...

[Slasher]

I discovered the switches, distortion 95% gone: TR0=2, TR1=0, TR2=0

[-Vit-]

By setting TR1=0 you're basically switching off QTGMC and getting something more like NNEDI3(field=-2) plus some sharpening.

[Slasher]

Yes, I've read what those TRx switches do but the result looks very good to me, almost no visible distortion for progressive scenes and very good deinterlacing, artifact free. I will definitely use the full power of QTGMC on fully interlaced, lower quality sources that will benefit from the advanced denoising and sharpening. Would simple NNEDI3 be faster in MT?

[-Vit-]

NNEDI3 on its own will certainly be faster, since QTGMC uses NNEDI3 as well as considerable other processing. However, that "other processing" is primarily the removal of bob-shimmer - by setting TR1/2 = 0, or using NNEDI3 alone you will get bob-shimmer in your result. Each individual frame will look fine, but there will be a rapid alternation/shaking between odd and even frames because NNEDI3 interpolates odd lines one frame, even lines the next. You may not notice it because the shimmer is rapid and at HD it's quite fine - try playing your result in slow-motion.

Using QTGMC or NNEDI3(field=-2) or any other kind of deinterlacing on your progressive scenes serves no purpose. I assume you are only doing this because you don't want to trim up your source...

[Slasher]

It would take a lot of work to start identifying the interlaced scenes, they are too many for the time that I have. I prefer to set the deinterlacer for the whole movie, something like "set-and-forget". Btw, this is how some interlaced scenes look like, just to give you an idea (source vs. qtgmc my settings):

http://thumbnails47.imagebam.com/16079/33100f160787660.jpg (http://www.imagebam.com/image/33100f160787660)http://thumbnails66.imagebam.com/16079/cc0dc0160787666.jpg (http://www.imagebam.com/image/cc0dc0160787666) http://thumbnails61.imagebam.com/16079/0f3616160787669.jpg (http://www.imagebam.com/image/0f3616160787669)http://thumbnails62.imagebam.com/16079/39047a160787673.jpg (http://www.imagebam.com/image/39047a160787673) http://thumbnails64.imagebam.com/16079/9ddbf9160787678.jpg (http://www.imagebam.com/image/9ddbf9160787678)http://thumbnails47.imagebam.com/16079/e558c6160787684.jpg (http://www.imagebam.com/image/e558c6160787684) http://thumbnails64.imagebam.com/16079/a5fec9160787689.jpg (http://www.imagebam.com/image/a5fec9160787689)http://thumbnails61.imagebam.com/16079/827db8160787696.jpg (http://www.imagebam.com/image/827db8160787696)

[nothingSoStrange]

I've a setup/tech question about QTGMC: I tryed to get it running using the packages linked in the first post, but it didn't worked as the script kept complaining about missing functions here and there :\

So I tryed to trace back which files were "broken" and ended up mixing a lot, taking here and there, trying, substituting and so on.. I've never seen a filter chain with so many variables.. take the file from 0.9, nope, 1.0pre, open the archive but just take this and so on.. :D

Ok, to the question. Actually I got it running, and I'm planning some test to see which setting over which video yelds better result.

I'm not so sure about the files I got, I wanted to ask you if the files I ended up are the "correct" versions (whatever the meaning is):

_ SSEToolS.dll 68K, 2005
_ RepairSSE3.dll 24K, 2007
_ RemoveGrainSSE3.dll, 33K 2007
_ VerticalCleanerSSE2.dll, 12K 2008
_ haven't played with the others

Trust me, until now, this is the only config (over many-many combinations, this is the first one working) which is actually loading the file with:

AviSource("foreman_i.avi").ConvertToYUY2(interlaced=true)
QTGMC(Preset="Slower", Sharpness=0.8)
SelectEven()

Using the original files in the package, AviSynth keeps complaining about missing functions.. I assume wrong dll versions.. or not? can you help me out on this?

The PC is a P4 3.0Ghz Prescott.

Tnx.

[-Vit-]

The files in the plugin package should be fine - your files are not all the recommended versions.
All those four files you mention are related plugins - they are no longer under active development so we have to put up with their confusing versioning.

You are using a YUY2 source, so you need to use masktools-2.5 regardless of your avisynth version. This is because masktools-2.6 does not support the YUY2 variant that is needed by other plugins. That may have been the cause of your error, though you really need to state exactly what the error was. I guess I should make a note of that YUY2 requirement in the OP and instructions...

[nothingSoStrange]

The files in the plugin package should be fine - your files are not all the recommended versions.
All those four files you mention are related plugins - they are no longer under active development so we have to put up with their confusing versioning.

You are using a YUY2 source, so you need to use masktools-2.5 regardless of your avisynth version. This is because masktools-2.6 does not support the YUY2 variant that is needed by other plugins. That may have been the cause of your error, though you really need to state exactly what the error was. I guess I should make a note of that YUY2 requirement in the OP and instructions...

Sorry, you are right I forgot to mention the name of the error messages. I'm using AviSynth 2.5, could this be a (starting) fault?

Yes, I'm using YUY2, now, I've read someting about, I don't want to ask here the obvious question though.. I felt the right thing to do was not to flat out videos in YUV12, rather trying to keep colors as much as possible. I'm deadly wrong, ain't I? :(

I'll revert all files tothe originals, and write down the exact error messages asap.

[Taurus]

@nothingSoStrange.
-Vit- 's plugin bundles should work right out of the box :p
Did you use the genuine versions of the plugins or the modded ones, courtesy of -Vit- ?
The readme included shows clearly what to look for and you should find the culprit by following the advices.

For a Pentium 4 the modded/multithreaded versions should not do any harm, but maybe better to use the genuine ones.
The modded plugins are optimized foe Avisysnth 2.6 MT (Set's) as far as I know.

And make sure there are no double (older) masktools, mvtooles,removegrain, remove.....etc. in your plugins folder.

For me QTGMC is running on "modern" CPUs (multithreaded)
and on ancient P4, AthlonXP, etc. (singlethreaded)

Or maybe there are some other plugins in your path which bite the dust?

[nothingSoStrange]

I'm back, so:

About my P4: I'm not interested in multithreading or modded version, I'm fine with the plain clean one; It's not for work, just for the occasional deinterlace of camera movie.

AviSynth was a clean install, plugin folder is empty (just a couple of input filter, see below), downloaded the plugin package from first post, currently using AviSynth 2.5 and I copyed all the plugins to the right dirs.

Using this script:

AviSource("original_L.avi").ConvertToYV12(interlaced=true)
QTGMC(Preset="Slower", Sharpness=0.8)
SelectEven()

Dropped over VirtualDub (or mplayerc.exe) I get:

http://img192.imageshack.us/img192/3002/boxrn.png

and this is AviSynth plugin listing:

http://img833.imageshack.us/img833/2332/lisths.png

What's my next move? :(

P.S. I've added convertyv12 for color space maximum compatibility.

[-Vit-]

I wonder if it is your avisynth / plugins folder setup that is incorrect. That RemoveGrain is the very first plugin that is used. Maybe it can't find it at all. Try this test script to replace yours:

AviSource("original_L.avi").ConvertToYV12(interlaced=true)
RemoveGrain(1)
NNEDI3(field=-2)
SelectEven()

If it still complains about RemoveGrain, then try removing that line and see if it will run the NNEDI3 script...

[yup]

nothingSoStrange!
Try replace removegrain to latest version
http://home.arcor.de/kassandro/prerelease/RemoveGrain-1.0.rar
yup.

[Taurus]

@nothingSoStrange
your avs is three liner...
and the error message shows the trouble at line 6 (basic.avs).
What's going on?
And what Avisynth version do you use?
2.5 or 2.58? :o

[nothingSoStrange]

I wonder if it is your avisynth / plugins folder setup that is incorrect. That RemoveGrain is the very first plugin that is used. Maybe it can't find it at all. Try this test script to replace yours:

AviSource("original_L.avi").ConvertToYV12(interlaced=true)
RemoveGrain(1)
NNEDI3(field=-2)
SelectEven()

If it still complains about RemoveGrain, then try removing that line and see if it will run the NNEDI3 script...

Nope, "there is no function named "RemoveGrain"....

NNEDI3 only script is loading the video. (#RemoveGrain(1))

[nothingSoStrange]

@nothingSoStrange
your avs is three liner...
and the error message shows the trouble at line 6 (basic.avs).
What's going on?
And what Avisynth version do you use?
2.5 or 2.58? :o

avisynth.dll 2.5.8.5
Filesize: 351.744
Date: 21 dic 2008

[nothingSoStrange]

nothingSoStrange!
Try replace removegrain to latest version
http://home.arcor.de/kassandro/prerelease/RemoveGrain-1.0.rar
yup.

Hooold!

from the above archive, I used:

RemoveGrainSSE2.dll
Size: 33.280
Date: 24 oct 2007

in place of:

RemoveGrainSSE2.dll
Size: 33.792
Date: 24 aug 2005

And it loads now! :D

I wonder if there are other good updates to the original, older version, first archive :\

Thanks yup! I was reading that exists multiple versions of these filter, 0.9, pre-1.0 and 1.0 - why they are not fully synced, or better, why thesere are on the wild multiple version of the same plugin (especially in the official package)? (just want to understand why.. for what I know (actually I tryed to search the forums:)) and I might be the only one having problems..)

[Didée]

But now you cannot use any scripts that happen to use the "Clense()" filter. (Not included in the version posted by yup.)

Clense isn't required for QTGMC, but if you get a Clense problem with other scripts, then you know why. ;)

(I never noticed that the 2005 SSE2 dll would not auto-load - because I never use autoloading anyway. LoadPlugin(...), that is.)

[nothingSoStrange]

But now you cannot use any scripts that happen to use the "Clense()" filter. (Not included in the version posted by yup.)

Clense isn't required for QTGMC, but if you get a Clense problem with other scripts, then you know why. ;)

(I never noticed that the 2005 SSE2 dll would not auto-load - because I never use autoloading anyway. LoadPlugin(...), that is.)

<noob>Can you explain in detail?</noob>

About loadplugin: shouldn't be the same putting filters inside avisynth directory or loading them externally? I mean, what is good/what is bad.. :o

What should I do?

--UPDATE
Reading here.. http://forum.doom9.org/showthread.php?p=1172271#post1172271

I'd better restate my last question: so, what's the big picture here? :D

[-Vit-]

I never noticed that the 2005 SSE2 dll would not auto-load - because I never use autoloading anyway. LoadPlugin(...), that is.
I can auto-load that 2005 version fine. Suspect many others can too or this issue would have come up much earlier.
Strange...

Have added a note to the OP to suggest that alternative version upon "there is no function named RemoveGrain" errors.
Agreed, the versioning for that filter family is dire but I'm not aware of any "big picture" (?), just seems to be how the author released things...

[AzraelNewtype]

<noob>Can you explain in detail?</noob>


Clense (why did he have to spell this function name wrong? Keep getting errors when I type it correctly and bash my head until I find the "problem") isn't in the final 1.0. As such, everyone gets to have the joys of keeping around two copies of the plugins and switch between then based on what scripts we're using. It owns. This, incidentally, is one of the arguments against dumping everything into the default directory and letting autoload do its thing.