QTGMC Deinterlacing Script (v3.32)

[-Vit-]

Well QTGMC was never supposed to be a replacement for a denoiser. Noise bypass was where it started - to get round the fact that the core algorithm temporally smooths and you might not want that. Which explains the setting names. But I noted in the docs somewhere that you could use it to denoise and... well, lots of people leapt at that idea. So the new settings were born and now it is trying to be a denoiser...
____

Just adding YUY2 support now. Someone made a version that supported YUY2, but I can't find it. Don't even know who made it. I looked to find it in the thread but any posts from 6th January to 18th January are missing (post 135-136 (http://forum.doom9.org/showthread.php?p=1468899#post1468899) in this thread) - which is when it was posted...

[yup]

[QUOTE=
Just adding YUY2 support now. Someone made a version that supported YUY2, but I can't find it. Don't even know who made it. I looked to find it in the thread but any posts from 6th January to 18th January are missing (post 135-136 (http://forum.doom9.org/showthread.php?p=1468899#post1468899) in this thread) - which is when it was posted...[/QUOTE]
It is Steven R. Savage version. I am try it, work fine.
yup.

[nhope]

1) do not resize vertically before deinterlace
2) you may need "interlaced=true" for ConvertToYV12()

Thanks for that. Actually the script is not resizing vertically before deinterlace. It resizes horizontally before deinterlace and vertically after. You are right about "interlaced=true". I had omitted that, and the levels match slightly more precisely now that I've added it. The new base script in my tutorial (http://www.bubblevision.com/underwater-video/Vegas-YouTube-Vimeo.htm) is now:

# Open frameserved source.
# Change path and file name as appropriate.
AviSource("d:\fs.avi")

# Convert to YV12 so filters will work.
# Use interlaced layout for conversion.
# Change to "true" to false" for progressive source.
# Use Rec.709 coefficients, keep full range [0,255].
ConvertToYV12(interlaced=true, matrix="PC.709")

# Assume footage is top field first.
# If it's not then use AssumeBFF.
# Leave it out for progressive source.
AssumeTFF

# Lanczos3 resize to 1280 wide.
# Leave it out if horizontal resolution is already 1280.
LanczosResize(1280,height)

# Deinterlace with QTGMC script.
# Available presets are placebo, very slow, slower,
# slow, medium, fast, faster, very fast, super fast,
# ultra fast, draft. Default is medium.
# Slower presets unnecessary for HD.
# ultra fast requires Yadif.
# Available noise presets (since version 3.20) are
# slower, slow, medium, fast, faster. Default is fast.
# Leave line out for progressive source.
QTGMC( Preset="faster", NoisePreset="Fast" )

# Add this line to keep original frame rate.
# Leave it out for smoother doubled frame rate.
# Leave it out for progressive source.
SelectEven()

# Lanczos3 resize to 720 lines.
# Leave it out if vertical resolution is already 720.
LanczosResize(width,720)

-Vit-, YUY2 support will be welcome. Thanks for your hard work and brilliance with this script, and for your help in this thread.

[-Vit-]

yup: Thanks for that (and thanks to Stephen R Savage too). It will be a handy cross reference.

nhope: It doesn't affect anything, but you don't need to set NoisePreset unless you're doing some noise processing (i.e. using one of the "EZ" modes or setting NoiseBypass). There is a little grain retention in the "Very Slow" and "Placebo" settings but from reading your thread over at Sony I think you're targeting the faster settings. In any case NoisePreset="Fast" is the default.

[nhope]

OK so I took the NoisePreset parts out of the tutorial again to keep it simple. Those wanting more can find it in your help file, which is now really easy to view in its new html form.

[Gavino]

Just adding YUY2 support now. Someone made a version that supported YUY2, but I can't find it. Don't even know who made it. I looked to find it in the thread but any posts from 6th January to 18th January are missing (post 135-136 (http://forum.doom9.org/showthread.php?p=1468899#post1468899) in this thread) - which is when it was posted...
It is Steven R. Savage version. I am try it, work fine.
Does this explain the missing posts?
For some reason known only to himself, Steven has a history of deleting his posts (which is unfortunate as his contributions are often useful).

[kolak]

Just adding YUY2 support now. Someone made a version that supported YUY2, ...


Great- YUY2 is nice to have for people, who have 4:2:2 sources.


Andrew

[2Bdecided]

The new base script in my tutorial (http://www.bubblevision.com/underwater-video/Vegas-YouTube-Vimeo.htm) is now:FWIW, unless the only copy you ever want to make of that Vegas timeline is a Vimeo upload, I think it makes sense to render to HDV (if the project can mostly be rendered with smart rendering, and you're starting from HDV of course), or some near-lossless or lossless format from Vegas - and then to work from that. Using the debug frame server means you never actually create a master copy for future use.

One trick: sometimes almost everything would smart render if only it weren't for the levels. If the required level correction is constant throughout, do it in AVIsynth - then you can smart render. Though it does annoy me to see the extra quantisation noise appear on the sections that don't smart render! It's not visible on any subsequent encode you create from this (unless using insane settings), and QTCMG will wipe it all out by default.

It's a rare bit of content where you can see any difference between Yadif and QTGMC by the time it's on YouTube or Vimeo at 720p. Near-stills at 1080p might be a different matter, but I don't know many people who can play YouTube 1080p to check this.

Very nice guide though! It's takes a lot to explain all the steps properly like this. Great job.

Cheers,
David.

EDIT: many of the MeGUI pre-sets are fine for Vimeo and YouTube upload IME. The sites are quite happy with unconstrained balanced (ABR or const. quality, whichever you prefer).

[@MeGui]

nhope...someth is wrong with MT script?

[henryho_hk]

SelectEven()

If you are doing single-rate deinterlacing, why not try the new motion blur feature?

[nhope]

@ 2Bdecided - I'll reply to your non-QTGMC points on a dedicated thread for this tutorial when it's refined.

@ henryho_hk - Good idea. I'll check it out.

nhope...someth is wrong with MT script?

Not sure because I can't test it here. The brief testing I did was actually slower than ST because I'm just on a Core 2 Duo laptop. What problem are you seeing? Here is the base MT script with the non-MT comments removed (full version here (http://www.bubblevision.com/underwater-video/Vegas-YouTube-Vimeo.htm#MultiThreaded)):

# Set maximum memory.
# Setting M:
# - First try without the SetMemoryMax line
# - However, using the SetMemoryMax line and a good value for
# M might allow more threads and so give more speed.
# Particularly important for slower settings
# - Try values 400,600,800,1000 etc.
# SetMemoryMax(M)

# Set multi-threaded mode.
# Setting X:
# - Start at the number of cores in your machine.
# - If it crashes, decrease 1 at a time.
# - Otherwise increase 1 at a time until CPU usage is very,
# very close to 100%, don't go too far or it will slow down .
SetMTMode(5, X)

AviSource("d:\fs.avi")
ConvertToYV12(interlaced=true, matrix="PC.709")
AssumeTFF

# Set multi-threaded mode.
SetMTMode(2)

LanczosResize(1280,height)

# Setting Y:
# - Start at about half number of cores and tweak upwards
# or downwards for best speed. Y=1 often works well.
# - Balance this setting with X (i.e. if you increase X,
# you might need to decrease Y and vice versa).
QTGMC( Preset="faster", EdiThreads=Y )

SelectEven()
LanczosResize(width,720)
ColorYUV(levels="PC->TV")

# MeGUI requires this line for multi-threaded operation.
Distributor()

[-Vit-]

The latest version, v3.25, is in the first post (http://forum.doom9.org/showthread.php?t=156028). The changes:

# v3.25
# - Added YUY2 support
# - NoiseBypass renamed to NoiseProcess
# - PrevGlobals now defaults to "Replace" for simplicity
# - Tweaked "Very Fast" preset for quality and consistency with other preset speeds


Dealing with the smaller points first. The NoiseProcess name change is just to avoid confusion - I've also clarified the docs. The PrevGlobals change means that multiple calls to QTGMC in a single script does not require special attention. The previous script was too defensive - now only those doing unfeasibly complex scripts or wishing to take advantage of the globals system need to worry about it. I found the "Very Fast" preset was slower than "Faster" in single threaded mode so I tweaked it.
____

YUY2 support is the main change here. All settings and features are supported and the speed seems OK. I get 20% to 30% slower depending on preset and settings, which is decent given that there's 33% more data in YUY2. I have only a very limited amount of interlaced YUY2 material so could those who work with YUY2 please confirm that everything is OK? In particular check the extra chroma data. If you're not so familiar with YUY2, please don't report problems until you're sure that your entire workflow is correctly passing through the YUY2 data all the way to your encode/player. Try substituting "Bob()" for "QTGMC()" before blaming it on me!

I minimized interleaved to planar conversions to keep the speed, there are just seven in default "Slower" settings, five at "Slow". Almost all the YUY2 processing is done in the faster planar format. To achieve this I've taken some liberties by passing the non-standard planar YUY2 format into several internal avisynth functions. Each has been tested to operate correctly on this form, although I didn't go through and check the source code of each. The functions I 'm using with the planar YUY2 form are:
Merge, Bob, Blur(0,x), Crop(0,x,0,x), TemporalSoften, SeparateFields, Weave, Interleave, SelectEvery

[nhope]

If I shot HDV 1080-50i with a shutter of 1/50 (or 1080-60i with a shutter of 1/60), and I want to restore the motion blur after deinterlacing to 25p (or 30p) with QTGMC, what values should I use for ShutterAngleSrc and ShutterAngleOut?

Also does FPSDivisor=2 mean I can drop SelectEven() ?

[-Vit-]

Yes with FPSDivisor=2 you can drop the SelectEven (I will make that clearer in the docs)

The answer to the first question is either both to 360 or both to 180. I would tend to suggest the latter, which is the default.

That question is one of the main reasons why I didn't use shutter speeds for the settings. In researching the shutter motion blur I found some confusion about shutter speeds when shooting interlaced material. Some suggest that a shutter speed matching the field rate (e.g. 1/50 shutter for 50i, also called "shutter off") implies a 360 degree shutter angle. However, I also found comment that the frame rate is relevant, in which case a 1/50 shutter speed on 50i footage implies a shutter angle of 180 degrees (film-like). There even seemed to be differences in how different cameras behaved with the same apparent settings. I have no opinion, this is not my area of expertise. So I went for shutter angle which is at least consistent if not the most widely known.

I would appreciate if anyone else can shed light on this one. If you shoot footage in this format, is the shutter effectively held open for 50% (180 degree) or 100% (360 degree) of the time between fields?

[SubJunk]

Thanks a lot for the update!

[Rhade3000]

im having some issues with qtgmc im not sure if its my system or the script, first some background

win7
6 core amd 3.2 gighz, 8 gig ram
2.5.8mt x64

script

SetMTMode(5,6)
DGDecode_mpeg2source("VTS_01_1.d2v", info=3)
crop( 2, 0, -2, 0)
setMTMode(2)
eedi3()
QTGMC( Preset="placebo", inputtype=2 )
spline64Resize(640,480)

im processings curb your enthus which is already 30p on the dvd

ive used this script for all of season 1 with no issue, but the last few versians of qtgmc i get this error


http://img257.imagevenue.com/loc479/th_007016739_Untitled_122_479lo.jpg (http://img257.imagevenue.com/img.php?image=007016739_Untitled_122_479lo.jpg)

the script is pretty cpu intensive, but its the only way i can seem to be abale to remove the horrible jagged edges in every scene

when i load the script in megui it loads fine and i see a preview pic, then i click add pre render job and enque and it throws up that error

if i do just a straight nnedi3() call with the qtgmc lined #' out it works, but it doesnt give as good of a result

[-Vit-]

As the error says, the problem is with loading NNEDI3. Unless others are getting the problem then it is likely to do with the setup of your plugins. Is the folder name in the error message correct? Is the file actually there? Do you have the latest version of NNEDI3? Download it again just in case there's a problem with the file, etc.

Have you tried running QTGMC using EEDI3 like this:
QTGMC( Preset=whatever, EdiMode="EEDI3" )
It might solve your problem (although it wouldn't explain the cause)

[nhope]

Preset tables updated for changes to "very fast" preset in 3.25:

Compared to "faster" (https://spreadsheets.google.com/ccc?key=0AtvvpQjptJLFdGxFaEZwNDNXdGliTi1KdElEUEVtTHc&hl=en&pli=1#gid=0)
Compared to "medium" (https://spreadsheets.google.com/ccc?key=0AtvvpQjptJLFdGVGQzFUa0ozWTZJYXItLWFFTURBMXc&hl=en#gid=0)

-Vit- - Please check! Sometimes I make clumsy errors.

[Taurus]

@Rhade3000
Take a look at this.
http://forum.doom9.org/showthread.php?p=1423128#post1423128
I've got this before.
Dont know about a x64 setup.
and have a look at the eventviewer.
If eedi3 is showing bad behaviour there, you can be sure a Visual C Redistributable is missing.

[Rhade3000]

@taurus and @vit

neither of those worked, but removing the "setmtmode(2)" line worked, and didnt slow down my cpu at all, i could have sworn that adding that line before saved me hours on the encode, but it didnt change processing time :P

thanks for the help :P

[aegisofrime]

-Vit-: Just to clarify, ChromaNoise works without needing NoiseProcess to be enabled?

[-Vit-]

-Vit-: Just to clarify, ChromaNoise works without needing NoiseProcess to be enabled?
ChromaNoise is only relevant if NoiseProcess is non-zero. I'll clarify the docs again... :sigh:

[nhope]

Yes with FPSDivisor=2 you can drop the SelectEven (I will make that clearer in the docs)

The answer to the first question is either both to 360 or both to 180. I would tend to suggest the latter, which is the default.

That question is one of the main reasons why I didn't use shutter speeds for the settings. In researching the shutter motion blur I found some confusion about shutter speeds when shooting interlaced material. Some suggest that a shutter speed matching the field rate (e.g. 1/50 shutter for 50i, also called "shutter off") implies a 360 degree shutter angle. However, I also found comment that the frame rate is relevant, in which case a 1/50 shutter speed on 50i footage implies a shutter angle of 180 degrees (film-like). There even seemed to be differences in how different cameras behaved with the same apparent settings. I have no opinion, this is not my area of expertise. So I went for shutter angle which is at least consistent if not the most widely known.

I would appreciate if anyone else can shed light on this one. If you shoot footage in this format, is the shutter effectively held open for 50% (180 degree) or 100% (360 degree) of the time between fields?

I asked the folk on the Sony Vegas forum (http://www.sonycreativesoftware.com/forums/ShowMessage.asp?MessageID=754540&Replies=10) and the general consensus is that 50i @ 1/50 or 60i @ 1/60 equates to a 180 degree shutter angle.

I've been doing some testing with it and it seems to be right. All just done single-threaded on my laptop. One thing I noticed was that the ShutterBlur setting didn't seem to make much difference to speed at all, so I left it at 3 for most tests. I didn't see much difference between using SBlurLimit or not on this particular clip. The table more or less speaks for itself:

http://dl.dropbox.com/u/21489814/QTGMC-motion-blur.png

I zipped up the 3 x 10 second tests I've highlighted in yellow and put them here (http://www.mediafire.com/?s0erw12ek7irikb) if anyone is interested. Footage used with permission from Stringer (Gregory) on the Vegas forum. I would be interested to hear which people prefer.

By the way Vit, are you making a general recommendation for FPSDivisor=2 over SelectEven (which is currently in your "Getting Started" section)? I just want to be consistent with you in my guide.

[aegisofrime]

ChromaNoise is only relevant if NoiseProcess is non-zero. I'll clarify the docs again... :sigh:

Sorry! I have read and re-read the Noise part of the docs and nowhere does it say that NoiseProcess has to be non-zero for ChromaNoise to work. If it's supposed to be obvious due to the nature of Chroma noise... Well it's lost on us newbie types who don't really understand the finer points of video :p

[-Vit-]

nhope: Thanks for the clarification that 180 degrees is the norm for shooting that kind of footage. Reading your thread over at Sony just confirms that there can be some confusion over shutter speeds. I'm going to stick with using shutter angle for the settings as it makes people think and/or ask rather than use the settings blindly...

The default setting for SBlurLimit should be OK for most sources, it's just an extra tweakable in case difficult motion is causing unwanted distortion/smearing. Similar with the higher shutter blur settings (2,3).
Edit: As noted in the docs, increasing those settings will likely reduce the level of blur a little from where it should be for the requested shutter angle. Looking at your motion blur samples - the 1/30s shutter version seems to have a little less blur than I would expect. I would stick to the (faster) ShutterBlur mode 1 and a lower SBlurLimit value (default is good). It doesn't harm to change ShutterAngleOut to get a bit more or less blur as you wish [The ideal would be to make some actual footage with known shutters and compare]. However, that particular footage is quite soft already and the motion is smooth, it doesn't benefit much from extra blur. I found motion blur works better on higher contrast video, with swift or sudden motion. It's also excellent for footage shot with a fast shutter, which really suffers when dropping to single rate.

You must use FPSDivisor if you're using shutter motion blur whilst cutting to single rate - it needs to know that information along with the shutter angles to calculate the correct amount to blur. If you're not using shutter motion blur it makes little difference whether you use SelectEven or FPSDivisor - whichever makes your tutorial easier.

[BTW: Ouch on those fps... Time to upgrade...]

____

Sorry! I have read and re-read the Noise part of the docs and nowhere does it say that NoiseProcess has to be non-zero for ChromaNoise to work. If it's supposed to be obvious due to the nature of Chroma noise... Well it's lost on us newbie types who don't really understand the finer points of video :p

I'm not sure which bit you were re-reading. Did you miss this bit?
Important: You must use a non-zero value in one of EZDenoise, EZKeepGrain or NoiseProcess or no noise processing will occur

[aegisofrime]

I'm not sure which bit you were re-reading. Did you miss this bit?

Ouch, I missed that. My fault :p

[Overdrive80]

Hi, thanks for your update.

This version throw an exception:

http://www.imagengratis.org/images/capturagr2gg.png

With version 3.20 doesnt pass me.

I do two scripts and run both, with rendering pass(lagarith).

1st script

DirectShowSource("E:\Pruebas Ghost Hound\16\F8_T2_Video - .mkv")

AssumeFPS(30000,1001)

QTGMC(preset="very fast" )

Selecteven()

threads=2

x=(framecount()/threads)

A=trim(0,x)
B=trim(x+1,0)



return A

2nd script

DirectShowSource("E:\Pruebas Ghost Hound\16\F8_T2_Video - .mkv")

AssumeFPS(30000,1001)

QTGMC(preset="very fast" )

Selecteven()

threads=2

x=(framecount()/threads)

A=trim(0,x)
B=trim(x+1,0)



return B

Any bug?

[-Vit-]

I have not seen an error like this before. Does it happen with preset "Faster" or "Ultra Fast"?

You could try this:
QTGMC( preset="very fast", SafeMode=true )

Or try making a second copy of your video to use in the second script.

[Overdrive80]

I have not seen an error like this before. Does it happen with preset "Faster" or "Ultra Fast"?

You could try this:
QTGMC( preset="very fast", SafeMode=true )

Or try making a second copy of your video to use in the second script.

In safemode run fine but decrease speed process, around of 2 frames/second. Now, my process is of 2.30 hours before was of 1.50 hours, more or less.

[-Vit-]

The bug is in MVTools2. Make sure you have the latest version of MVTools2 (http://avisynth.org.ru/mvtools/mvtools2.html#download) and try again without SafeMode.

If it still crashes then this line will restore the behavior of the last version (3.2):
QTGMC( Preset="Very Fast", Overlap=16, SLMode=1, SrchClipPP=0 )

This is a problem in MVAnalyse when the Overlap is not 1/2 the Blocksize. It crashes sometimes. I added the SafeMode setting just for this problem, but I haven't actually seen it for a while. It had assumed it was fixed in more recent versions...

[Overdrive80]

I already installed last version MVtools. In fact, i don know that new features in this version, but in my case, deinterlaced bad in the same mode. Snapshots:

Version 3.20, preset very fast

http://www.imagengratis.org/thumbs/comprimirnm3zn.jpg (http://www.imagengratis.org/?v=comprimirnm3zn.jpg)

New version, preset very fast

http://www.imagengratis.org/thumbs/comprimirce3cc.jpg (http://www.imagengratis.org/?v=comprimirce3cc.jpg)

[-Vit-]

Final Edit (I hope): I can reproduce the MVTools crash, I'll look into that later.

However, I cannot reproduce the major ghosting shown in your images. Could you upload part of your source please?

[henryho_hk]

I also found some problem bobbing a 1080i source. The field order is correct as confirmed by separatefields(). But the motion jumps "backward" in some scenes. I am using Preset="Faster" and tr2=2. Setting EdiMode="Yadif" solves the problem.

[nhope]

This version throw an exception:
...
I do two scripts and run both, with rendering pass(lagarith).

You didn't change your Lagarith version by any chance? There were recent updates (http://lags.leetcode.net/Changes.html) to it that caused some of us some problems (http://www.sonycreativesoftware.com/forums/ShowMessage.asp?ForumID=4&MessageID=754229).

[-Vit-]

henryho_hk: Try different source filters - I have often found that strange temporal effects are source filter related. Otherwise upload some source footage for me to test.

[henryho_hk]

It's ffvideosource() 2.15 over a MKV muxed from m2ts. I will try dss2() (i have ffdshow h264 decoder only, though).

[Overdrive80]

You didn't change your Lagarith version by any chance? There were recent updates (http://lags.leetcode.net/Changes.html) to it that caused some of us some problems (http://www.sonycreativesoftware.com/forums/ShowMessage.asp?ForumID=4&MessageID=754229).

Yes, I already installed lagarith 1.3.23. I dont think that could be for this reason.

@Vit. My source is 1080i@29.97. I upload a part of interlacing credits, compress in x264 @ 5000 Kb/s. If you need that upload in other format, only say me. ^^

http://www.megaupload.com/?d=ZJSXS36E

[-Vit-]

Overdrive80: The field parity needs to be reversed. This worked fine for me:

FFVideoSource( "E:\Video\Test\Original.mp4" )
ComplementParity()
QTGMC( Preset="Very Fast" )

Try "ComplementParity()" whenever you see "back & forward" jumping effects like this.

[Overdrive80]

Overdrive80: The field parity needs to be reversed. This worked fine for me:

FFVideoSource( "E:\Video\Test\Original.mp4" )
ComplementParity()
QTGMC( Preset="Very Fast" )

Try "ComplementParity()" whenever you see "back & forward" jumping effects like this.

With this code, the ghost effect is gone.

My question is for curiosity... because is necesary add a filter for autodetecting dominant field but not with previous version?

[johnmeyer]

I am using a script based on Nick Hope's work. However, I have made it multithreaded. Everything works with the older version of QTGMC, but only with Very Fast. When I use Fast, I get problems. Here's the script:
#Script to deinterlace video based on QTGMC deinterlacing script
#Designed to produce 1280x720 progressive video

loadplugin("C:\Program Files\AviSynth 2.5\plugins\MVTools\mvtools2.dll")

#---------------------------
#Import("E:\Documents\My Videos\AVISynth Scripts\QTGMC3.20.avsi") #older script
Import("E:\Documents\My Videos\AVISynth Scripts\QTGMC3.25.avsi")

#Serve out from Vegas in RGB24
SetMemoryMax(768)
SetMTMode(5,4)
AVISource("e:\frameserver.avi").assumetff().ConvertToYV12(interlaced=true,matrix="PC.709") #
SetMTMode(2)

LanczosResize(1280,height)

#---------------------------
QTGMC( Preset="Very Fast", FPSDivisor=2, EdiThreads=0, ShowSettings=false )
#QTGMC( Preset="Faster", FPSDivisor=2, EdiThreads=0, ShowSettings=false )

LanczosResize(width,720)
ColorYUV(levels="PC->TV") #Scale levels from [0,255] to [16,235]. Compensates for Flash Player scaling [16,235] to [0,255].


I have tried four combinations in the above script (by changing which lines are commented out):Script Version Script Preset
QTGMC3.20.avsi QTGMC( Preset="Very Fast" ... )
QTGMC3.20.avsi QTGMC( Preset="Faster" ... )
QTGMC3.25.avsi QTGMC( Preset="Very Fast" ... )
QTGMC3.25.avsi QTGMC( Preset="Faster" ... )

The first combination is the only one that works without crashing, when tested using a 1440x1080 1,000 frame input file. I test by reading the script into VirtualDub and then doing a video analysis pass. This eliminates any small interaction that might be happening with an output codec.

I have tried changing the SetMemoryMax number and also tried changing the number of cores in the first SetMTMode, but all that does is change how long it takes before the crash happens. It usually takes about 200-500 frames before the crash begins (the fps counter drops for about a dozen frames prior to the crash).

For the third test listed above (your newest script with the Very Fast setting), here is the relevant line (I think) from the crashinfo.txt log:

Crash reason: Access Violation

Crash context:
An out-of-bounds memory access (access violation) occurred in module 'mvtools2'...

...reading address 247EF1D2.
I read your earlier post about needing to update to the latest MVTools2, so I download the latest release (2.5.11.1), but still had the problems.

Oh, and prior to the beginning of the crash, the newer script runs considerably slower, probably about 30% slower, with each of the settings.

Let me know if you need any other information.

[-Vit-]

Overdrive80: No, I haven't changed anything that should affect the field parity. I also tested your clip on v3.20 and I needed the ComplementParity() line in that version too. Sometimes the source filters get the field parity wrong, I've seen it several times before.

johnmeyer: See post 430 above (http://forum.doom9.org/showthread.php?p=1485196#post1485196) for the settings to restore "Very Fast" in v3.25 to v3.20 behavior and speed. That might help for your "Very Fast" setting. I am nearly finished on some optimizations for all the faster settings, so I'm not going to worry about the speed issue just now.

Regarding crashes, the 'mvtools2' note in the error message may or may not be indicative. Try adding the setting "SafeMode=true" to your QTGMC line (it will be slower). If that fixes it then MVTools is really to blame. I can reproduce errors with MVTools, so I will debug it at some point. However, often multithreading or memory bugs due to AviSynth itself occur whilst running the plugin modules, so the problem actually might be with AviSynth. Let's hope it's a memory issue, because they're easier to fix. Absolute priority when running HD multithreaded is to render to a lossless intermediate file - don't try to use x264 at the same time as HD QTGMC. For MeGUI that means you must select "Add Pre-rendering Job". Setting EdiThreads=1 might help with memory too.

If you're crashes are problems with avisynth multithreading then try using different versions of MT avisynth. Find them linked on the AviSynth Wiki main page (http://avisynth.org/mediawiki/Main_Page)

[Overdrive80]

@Vit. I translate QTGMC doc to spanish, I hope that dont disturb you.

http://www.megaupload.com/?d=3XKOAFTN

[johnmeyer]

Regarding crashes, the 'mvtools2' note in the error message may or may not be indicative. Try adding the setting "SafeMode=true" to your QTGMC line (it will be slower). If that fixes it then MVTools is really to blame. I tried all four permutations of new/old script and Faster/Very Fast with "SafeMode=true" added to my QTGMC line.

With the old (v.20) QTGMC script, I was able to get my 1,000 frame test video to run through the video analysis pass without crashing, although the fps indicator started to fluctuate at about the 900 frame mark, which is usually a sign of impending crash. Thus, it might still crash if fed a longer test video. With the new (v25) QTGMC script and the more stable Very Fast setting, I was able to get almost to the end of the analysis pass before the script crashed, whereas without the SafeMode statement, it crashed almost immediately.

Hope that helps.

[johnmeyer]

Quick addendum (I didn't want to further edit my previous post):

I kept trying different things with my four permutations (described above) and found that I got more stability with both versions of the script if I changed EdiThreads=0 to EdiThreads=1. I still got crashes with the new script but they happened after a larger number of frames had been processed. So, following the guidelines in the QTGMC doc, I played around with the MT settings. SetMemoryMax didn't make much difference, but increasing the number of cores from 4 to 8 (my computer is a 8-core i7), allowed me to get all the way through my 1,000 frame test video with the "toughest" combination of the new v25 script and the "Faster" QTGMC setting. I haven't tried this on a really long video so I don't know if it is totally stable. Also, I did these most recent tests without the SafeMode statement.

Again, I hope this helps.

[edit] I just re-ran the script using the newest v25 script with these settings, but using Very Fast. It crashed. Bummer. So, these settings let me get through the test script with the settings that previously crashed, but they cause crashes on the settings that previously worked. I tried about half a dozen settings for the "Very Fast" setting, and was unable to get ANY that would work with the new script, even if I completely disabled all multi-threading.

So, I guess I haven't yet found settings that work for more than a few settings. It's still very "touchy."

[-Vit-]

Overdrive80: Thanks very much for that, I'm sure it will be appreciated. [Edit: I couldn't download it before, but now I have it. I will link to it on the first post.]

johnmeyer: There are two things here. Firstly there is certainly a problem with QTGMC<->MVTools on faster presets, which sometimes causes crashes before even a single frame is processed. I will look at that. But I don't think that's all that you're experiencing. Almost everyone who does multi-threading with complex scripts finds that avisynth will randomly crash with particular settings/scripts/hardware etc. The reason is that there are bugs with the multi-threading in avisynth. Sadly fixing those bugs would involve considerable effort, especially for those of us who aren't familiar with the avisynth code base.

I have had numerous detailed descriptions of problems, and they all follow the same pattern: encoding crashes after X-thousand frames; tweaking settings makes things slightly better, or slightly worse; problem is more pronounced with complex settings or HD material. But despite the similarity, there is no one solution. The first thing you need to try is different MT versions of AviSynth. For example, I can barely get a few hundred frames out with the standard 2.57MT, I only get limited stability with SEt's 2.58MT, but I have encoded tens of millions of frames with SEt's 2.6MT. For others it's the same story but a different version that works...

Edit: BTW, EdiThreads=0 means use 1 thread per logical core. So on your system it means EdiThreads=8. That's why setting it to 1 is more stable. Also it can help to use SetMemoryMax at a lower value than you might expect. I sometimes use 400 for HD material...

[johnmeyer]

-Vit-,

Yes, I agree with what you are saying about the less-than-robust behavior of the MT versions of AVISynth. Fortunately I think I have now managed to tweak things so that I have a stable workflow now.

One thing I didn't realize is that there are competing MT versions of AVISynth. When I do a version call, I get: "AVISynth 2.58 tsp version 5(mod seraphy), build: Aug 16, 2009 ..." I'll see if I can find a version "2.6MT." I don't think I've seen that before.

[-Vit-]

Here are the various versions: Standard 2.57MT (http://avisynth.org/mediawiki/MT), SEt's 2.58MT (http://forum.doom9.org/showthread.php?t=148117) or SEt's 2.6MT (http://forum.doom9.org/showthread.php?t=148782)
Or if you've converted to a completely 64-bit workflow the x64-2.58MT (http://forum.doom9.org/showthread.php?t=152800)

[SubJunk]

SET's 2.6 wins :)

[johnmeyer]

Thanks for the links. I tried set's 2.6, but apparently the direct support for reading RGB24 (at least from debugmode's frameserver) has been deleted. As soon as I try to open my script in VirtualDub, I get the message: "AviSource: Could not open video stream in any supported format." If I change the Debugmode Frameserver (which is how I serve out of Sony Vegas) from RGB24 to either RGB32 or YUY2, then I can open the script. RGB32 keeps the colorspace the same as my previous workflow, but cuts performance in half. Using YUY2 keeps performance the same, but introduces all sorts of color shift and levels issues. Nick Hope (who wrote the long tutorial on how to do this) has spent several months of his life working these things out, so I have no hope of trying to get the same levels. So, at least for the moment, I'm back to 2.5.8.

I like the idea, however, of having better stability with multi-threading (for ALL of my scripts), so I'll look into what else I can do here ...

[edit] I just did some more work ... if I use YUY2 out of the Debugmode frameserver, and strip all the colorspace corrections, I end up with the same levels. As for color space, all colors appear to be the same as those I got using Nick Hope's original workflow, except that pure red colors are truer to the original. With his workflow, it appears to me (now that I'm looking more critically) that there is a shift towards orange in the final render.

So, bottom line, I think I'll be able to use 2.6MT, and that all the changes you have suggested and that I've found on my own make this stable. I'm now getting over 15 fps doing video analysis in VirtualDub, and almost the same when rendering from MeGUI. All 8 cores are showing nearly 100%.

[Didée]

@ johnmeyer: Maybe I'm lacking imagination, but ... if you have to deinterlace a video, then why has it to come out of Vegas in the first place? I could more easily imagine the other way round: have some interlaced video, then deinterlace it with Avisynth/QTGMC, then do additional stuff in Vegas.
Basically this doesn't matter, of course. But since multithreaded QTGMC on Full-HD video is not a prime example of "stability" in the first place, I'm not sure if Vegas' frameserver does exactly help on the matter.