Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Usage

Closed Thread
 
Thread Tools Search this Thread Display Modes
Old 23rd May 2011, 01:54   #681  |  Link
Mounir
Registered User
 
Join Date: Nov 2006
Posts: 773
I have a problem with QTGMC i'm trying the "best settings" taken from the manual for instance this:
QTGMC( Preset="Slower", SourceMatch=3, Lossless=2, Sharpness=1.0, TR2=1 ) and it return an error there is no function VerticalCleaner not sure what this is about. My video is YUY2, huffyuv
Mounir is offline  
Old 23rd May 2011, 02:27   #682  |  Link
-Vit-
Registered User
 
Join Date: Jul 2010
Posts: 448
Quote:
Originally Posted by Mounir View Post
I have a problem with QTGMC i'm trying the "best settings" taken from the manual for instance this:
QTGMC( Preset="Slower", SourceMatch=3, Lossless=2, Sharpness=1.0, TR2=1 ) and it return an error there is no function VerticalCleaner not sure what this is about. My video is YUY2, huffyuv
Those settings work fine for me on a YUY2 raw interlaced source. Have you got the VerticalCleaner plugin...?

Must update the examples in the manual. There is, of course, no such thing as "best settings". However, this is fairly transparent, keeping the source relatively untouched:

QTGMC( "Slower", TR2=1, SourceMatch=2, Lossless=2, Sharpness=0.2, EZKeepGrain=0.6 )

Vary EZKeepGrain depending on source noise/grain (assuming you want to keep it - this is transparent remember). Adjust Sharpness if you feel it's necessary (faster motion videos generally need a bit more). Add and adjust NoisePreset if you wish (it defaults to "Fast") for some very slight improvements to fine detail/grain at the cost of some speed [very pedantic].

I don't really recommend SourceMatch=3 now, rarely makes any noticeable effect.
-Vit- is offline  
Old 23rd May 2011, 02:58   #683  |  Link
Mounir
Registered User
 
Join Date: Nov 2006
Posts: 773
You're right i didn't have this plugin, still i have an error window (not a script error) with AVSp the frames are not processed as i have to cancel this window all the time:

Traceback (most recent call last):
File "AvsP.pyo", line 5819, in OnMenuVideoToggle
File "AvsP.pyo", line 8925, in ShowVideoFrame
File "AvsP.pyo", line 9474, in PaintAVIFrame
File "pyavs.pyo", line 322, in DrawFrame
File "pyavs.pyo", line 301, in _GetFrame
File "avisynth.pyo", line 277, in GetFrame
WindowsError: exception: access violation reading 0x3EF69000
Traceback (most recent call last):
File "AvsP.pyo", line 7123, in OnPaintVideoWindow
File "AvsP.pyo", line 9474, in PaintAVIFrame
File "pyavs.pyo", line 322, in DrawFrame
File "pyavs.pyo", line 301, in _GetFrame
File "avisynth.pyo", line 277, in GetFrame
WindowsError: exception: access violation reading 0x3EF69000

etc...
Mounir is offline  
Old 23rd May 2011, 03:57   #684  |  Link
-Vit-
Registered User
 
Join Date: Jul 2010
Posts: 448
I don't know what that is. Nothing QTGMC in there.

Could be out of memory I suppose, if you're running with many threads. If so cut down SetMTMode threads. And EdiThreads when using SourceMatch. Tweak the MemoryMax.
The usual multi-threaded juggle...
-Vit- is offline  
Old 23rd May 2011, 04:48   #685  |  Link
Mounir
Registered User
 
Join Date: Nov 2006
Posts: 773
This multithread thing is complicated i have followed the instruction, installed avisynth.dll in sys32, mt.dll in the plugin folder but no luck (i'm forced to close Avsp) plus i can't find infos on MemoryMax

my script so far:
Avisource("H:\MyVideo.avi", fourCC="HFYU", pixel_type="YUY2").trim(2500,2900)
MT(QTGMC( "Slower", EdiThreads=2, TR2=1, SourceMatch=2, Lossless=2, Sharpness=0.2, EZKeepGrain=0.6 ))

The maximum threads i can have is 2 on this cpu

Edit:
I have also tried (i found out for setmemorymax)

SetMemoryMax(512) #
SetMTMode(2,2) #
Avisource("H:\MyVideo.avi", fourCC="HFYU", pixel_type="YUY2").trim(2500,2900)
Setmtmode(2,2) #
AssumeTFF
QTGMC( "Slower", EdiThreads=2, TR2=1, SourceMatch=2, Lossless=2, Sharpness=0.2, EZKeepGrain=0.6 )
selecteven()
Distributor() # for multithreading

AvSp bug and i'm forced to close it, i think the problem is with masktools-v2.0a36 which i use for QTGMC

Last edited by Mounir; 23rd May 2011 at 05:19.
Mounir is offline  
Old 23rd May 2011, 07:24   #686  |  Link
Didée
Registered User
 
Join Date: Apr 2002
Location: Germany
Posts: 5,389
First script, corrected:

Code:
Avisource("H:\MyVideo.avi", fourCC="HFYU", pixel_type="YUY2").trim(2500,2900)
MT("""QTGMC( "Slower", EdiThreads=2, TR2=1, SourceMatch=2, Lossless=2, Sharpness=0.2, EZKeepGrain=0.6 )""", threads=2,overlap=16)
(When the filter call within MT(...) contains "quotes", then you need """triple quotes""" for the outer MT() call.)


Second script, corrected:

Code:
SetMemoryMax(512) #
SetMTMode(5,2) # Not using multithreading for the source filter is more safe
Avisource("H:\MyVideo.avi", fourCC="HFYU", pixel_type="YUY2").trim(2500,2900)
Setmtmode(2,2) #
AssumeTFF
QTGMC( "Slower", EdiThreads=2, TR2=1, SourceMatch=2, Lossless=2, Sharpness=0.2, EZKeepGrain=0.6 )
selecteven()
# Distributor() # don't use Distributor for apps using VfW API. (AvSP does, or not?)
With only 2 cores / 2 threads, the MT(""" ... """) script probably is faster for you.
(With complex scripts, the overhead for SetMTmode often eats a good part of the theoretical benefit when only 2 threads can be used. For QTGMC, SetMTmode starts to shine with 4 threads and above. With a quadcore, o'course. DualCore/2threads is too much a toy for this.)
__________________
- We´re at the beginning of the end of mankind´s childhood -

My little flickr gallery. (Yes indeed, I do have hobbies other than digital video!)
Didée is offline  
Old 23rd May 2011, 08:12   #687  |  Link
Boulder
Pig on the wing
 
Boulder's Avatar
 
Join Date: Mar 2002
Location: Finland
Posts: 5,718
I think you could use SetMTMode(2) with HuffYUV since it's all keyframes.
__________________
And if the band you're in starts playing different tunes
I'll see you on the dark side of the Moon...
Boulder is offline  
Old 23rd May 2011, 08:20   #688  |  Link
Didée
Registered User
 
Join Date: Apr 2002
Location: Germany
Posts: 5,389
Possible. But it doesn't help anyway - in comparison to calculating QTGMC's processing chain, decoding Huffy is totally peanuts.

Hence, better to avoid SetMTmode for the source filter completely. Best case=it doesn't help anything, worst case=it crashes. So, what to recommend to a user that is fighting with script crashing?
__________________
- We´re at the beginning of the end of mankind´s childhood -

My little flickr gallery. (Yes indeed, I do have hobbies other than digital video!)
Didée is offline  
Old 23rd May 2011, 08:28   #689  |  Link
Boulder
Pig on the wing
 
Boulder's Avatar
 
Join Date: Mar 2002
Location: Finland
Posts: 5,718
Since dealing with SetMTMode can be compared to walking on the edge of a cliff, I usually have to try different things to get through a long encode. Changing SetMTMode for the source filter has helped sometimes, don't ask me why

OT: I built a "new" second-hand computer last weekend. After a few crashes, I noticed that it's once again time to try to find the working MT-Avisynth version..grrrr.
__________________
And if the band you're in starts playing different tunes
I'll see you on the dark side of the Moon...
Boulder is offline  
Old 23rd May 2011, 13:17   #690  |  Link
aegisofrime
Registered User
 
Join Date: Apr 2009
Posts: 478
I have no idea why you guys say that SetMTMode is unstable. My script here:

Code:
SetMTMode(5,4)
DGDecode_mpeg2source("C:\Raws\Work.d2v",idct=5)
SetMTMode(2,4)
QTGMC(Preset="Slower")

Distributor()
Is perfectly stable. I use it for all my DVD encodes and it has never crashed before.
aegisofrime is offline  
Old 23rd May 2011, 14:19   #691  |  Link
-Vit-
Registered User
 
Join Date: Jul 2010
Posts: 448
In the first post I've added a modded plugins package. All the key plugins have been recompiled using SEt's threading fixes and an extra tweak that prevents the main crash that I see when multi-threading. I can't guarantee they will help, but they might be worth a try. They won't prevent running out of memory if you use complex settings/many threads.

Read the instructions in the zip before asking what to do with the contents.

Instructions for stability and getting the best out of SetMemoryMax & SetMTMode are also in the first post (and the QTGMC instructions).
Also covers the EdiThreads setting, which is best set to 1 if you're getting problems.
Some people's computers simply don't like SetMTMode. Nothing to be done except fix AviSynth.

I don't recommend MT("") unless absolutely necessary, because splitting up the image will affect motion analysis (you always get problems at the edges with MVTools, with MT("") you have edges in the middle of your image...)

Last edited by -Vit-; 16th September 2011 at 14:48. Reason: bring up to date
-Vit- is offline  
Old 23rd May 2011, 17:44   #692  |  Link
Mounir
Registered User
 
Join Date: Nov 2006
Posts: 773
Thanks Didée, MT work and setMtmode doesn't (error with mt_masktools-25.dll apparently) on this computer but i think i will avoid to use it as it split the image

Last edited by Mounir; 23rd May 2011 at 17:48.
Mounir is offline  
Old 24th May 2011, 20:48   #693  |  Link
Mini-Me
Registered User
 
Join Date: Jan 2011
Posts: 121
I went digging through the script to see how SourceMatch and the lossless modes interact with noise removal/retention modes. First, I want to congratulate -Vit- on writing the most beautifully commented script I've seen yet. Second, I want to ask if I properly understand the interaction of NoiseRestore and GrainRestore with Lossless mode 1.

In case it helps anyone (or helps -Vit- point out if I went wrong somewhere), here is the rough order of major QTGMC steps:
  1. Obtain motion vectors: Bob, temporally smooth to fight bob shimmer, and perform motion analysis.
  2. Optionally remove/bypass noise: Optionally denoise the original input clip and/or save extracted noise and fine detail (optionally stabilizing it) for later recovery. The [optionally] denoised clip is now known as the "inner" clip, and it is used in place of the original source clip for the rest of the filter (unless and until the extracted noise is readded).
  3. Primary bob deinterlace: Perform a high-quality upscale/interpolation of the fields in the "inner" clip to attain full frame height. Temporally smooth the result with motion-compensated neighbors (using previous motion analysis) to fight bob shimmer, then repair to fight blurring (and artifacts from bad motion compensation?). (Side question: Is the temporal smooth also meant to absorb real detail and structure from motion-compensated neighbors, or does that get removed anyway by the repair step?)
  4. SourceMatch: Optionally redo the previous step (I think) with input refined by the previous result, such that the new result will have original (non-generated) lines better matching the actual lines from the source. (Refine this up to two more times, though not exactly in the same manner). As far as I can tell, this is meant to recover detail lost in the previous temporal smoothing, and the degree to which the "original lines" match the original source also indicates how well the generated lines probably match the "original lines" of neighboring fields.
  5. Lossless mode 2: If the lossless mode is 2, drop in the original fields of the inner clip here.
  6. Resharpen: ...with a complicated series of steps involving limiting and back-blending.
  7. GrainRestore: Optionally readd noise prior to the final smooth, proportional to GrainRestore.
  8. Clean the result: Do a final [light, motion-compensated] temporal smooth, then repair. (Also, finish sharpness limiting if necessary.)
  9. Lossless mode 1: If the lossless mode is 1, drop in the original fields of the inner clip.
  10. NoiseRestore: Optionally readd noise after the final smooth, proportional to NoiseRestore.

What I've learned is that SourceMatch and Lossless mode 2 work perfectly and intuitively with denoising, NoiseRestore, and GrainRestore. The reason is, SourceMatch and Lossless actually match to the [optionally denoised] source rather than the original source. This makes perfect sense, and it's actually "easier" and more robust than I expected:
  • If you want to denoise, using the original fields (before denoising) for SourceMatch and Lossless would contradict that.
  • If you denoised with the intention of adding it back later, your intent was likely to retain the original noise "as-is." Matching to the inner clip with SourceMatch and Lossless accomplishes this, whereas using the true original fields and then adding noise back in would result in amplified noise.
  • If you didn't denoise at all, there's no difference between the original and inner clips anyway.

However, Lossless mode 1 does not seem to work well with GrainRestore > 0.0: Only the generated lines will have grain restored, and lossless mode 1 will subsequently replace the original lines with the [possibly denoised] lines from the inner clip.

That said, assuming GrainRestore is 0.0, Lossless mode 1 seems to work fine:
  • If you denoised the clip and NoiseRestore is 0.0, you'll end up with a denoised clip with lossless denoised lines from the original.
  • If you denoised the clip and NoiseRestore is 1.0, you'll end up with a noise-bypassed clip, and the original lines should presumably be the same as the ones from the source (because the exact difference between the source and the "inner" clip is being readded to the original lines...I think ).
  • If you didn't denoise the clip, you'll want to use 0.0 for NoiseRestore, but using higher values still produces the expected result of amplifying noise.
If you're using a Lossless mode of 1, you're probably looking for one of those results anyway.

Am I correct? If I'm wrong, let me know what I got wrong. If I'm right, it might be a good idea to add a few details to the instructions. The instructions say that Lossless mode 1 will use the exact original pixels from the source lines if NoiseRestore is 0.0, but isn't this only true if you did NOT denoise the clip? (If you did denoise the clip, then a NoiseRestore of 1.0 would actually give the original lines, correct?) Similarly, the directions mention nothing about how GrainRestore and Lossless mode 1 appear to interact poorly.

Thanks

Last edited by Mini-Me; 24th May 2011 at 20:51.
Mini-Me is offline  
Old 25th May 2011, 00:16   #694  |  Link
-Vit-
Registered User
 
Join Date: Jul 2010
Posts: 448
Your description is correct in most every detail. Including the interaction between Lossless mode 1 and noise processing. Yes, I should note that GrainRestore (and by implication EZKeepGrain) are not effective with that mode. I do need to look more carefully at Lossless mode 1, the "true" lossless mode (I will also tweak these setting names for clarity). It hasn't been a recent priority since you can get very close to lossless with source-match and Lossless mode 2, without introducing the artefacts that true lossless mode 1 will bring.

I'm glad you spotted the detail behind the inner clip, especially the interaction of source match and noise processing. I don't think I explicitly note that. One other point to note is that resharpening occurs on the denoised clip, with the noise being added back on after sharpening. That means it won't enhance noise so much (but may miss some minor detail to be sharpened)...

Regarding your side question: Didee may have more to comment as this step inherits from original TGMC. The main temporal smooth (TR1) has a variety of effects depending on the quality of the motion analysis. It's main intent is to remove bob-shimmer. If the motion analysis is correct then it will slightly pull in detail from neighbor frames, but may also slightly weaken detail in the current. If the motion analysis is off, then it can blur or ghost or other nasty things - depends on the exact circumstances. The repair step helps to remove some of that.

The noise motion compensation step, designed to "stabilize" the noise, will also pull in detail from neighbor frames. It only looks at one neighbor frame, but I have a (very slow) variant that looks at several sucking in as much extra detail as possible from neighbors. Creates a kinda "enhancing" effect; not sure I like it. Can get the effect by setting EZKeepGrain high with a slow NoisePreset.

Last edited by -Vit-; 25th May 2011 at 00:23.
-Vit- is offline  
Old 25th May 2011, 02:55   #695  |  Link
Mounir
Registered User
 
Join Date: Nov 2006
Posts: 773
I'm trying my script again right now with MT i'm not sure why VIt called the best settings "lossless" because it's not lossless for me , take a look:

Original:
http://img195.imageshack.us/img195/5138/originalrn.jpg
QTGMC:
http://img802.imageshack.us/img802/8199/qtgmc.jpg

settings used:
MT("""QTGMC( "Slower", EdiThreads=1, TR2=1, SourceMatch=2, Lossless=2, Sharpness=0.2, EZKeepGrain=0.6 )""", threads=2,overlap=16)
Mounir is offline  
Old 25th May 2011, 03:46   #696  |  Link
-Vit-
Registered User
 
Join Date: Jul 2010
Posts: 448
You're expecting too much. Horizontal detail like that is the Achilles heel of of all deinterlacers. You might find a different deinterlacer will do a better job for that particular shot, or maybe not, hard to tell from a single image.
-Vit- is offline  
Old 25th May 2011, 04:48   #697  |  Link
Mounir
Registered User
 
Join Date: Nov 2006
Posts: 773
Well msu seems to get it right with their deinterlacer, what you think ? I think it's neater maybe i'ts just me i don't know
Mounir is offline  
Old 25th May 2011, 05:07   #698  |  Link
-Vit-
Registered User
 
Join Date: Jul 2010
Posts: 448
Quote:
Originally Posted by Mounir View Post
I think it's neater maybe i'ts just me i don't know
They are different pictures to yours. No problem cases there. But if their deinterlacer works for you then use it.

Last edited by -Vit-; 25th May 2011 at 05:09.
-Vit- is offline  
Old 25th May 2011, 05:34   #699  |  Link
Mounir
Registered User
 
Join Date: Nov 2006
Posts: 773
I wish i could but it's not free it appears. Anyways, i truly salute the efforts made for this deinterlacer and hope you make it even better.
Mounir is offline  
Old 25th May 2011, 07:57   #700  |  Link
Didée
Registered User
 
Join Date: Apr 2002
Location: Germany
Posts: 5,389
Quote:
Originally Posted by Mounir View Post
Well msu seems to get it right with their deinterlacer, what you think ? I think it's neater maybe i'ts just me i don't know
You believe the marketing fairytales? Those are cherry-picked spots to convince the audience.

If we were going to produce cherry-picked spots of QTGMC, they would look at least as impressive as those.

Also, you should post a small sample of your problem source. That screenshot is zoomed, and it's impossible to tell of what kind the source is at all. It could be a case of using QTGMC wrongly, or a case of filter-not-suited-for-the source (if it's telecined or fieldblending-interlaced).
__________________
- We´re at the beginning of the end of mankind´s childhood -

My little flickr gallery. (Yes indeed, I do have hobbies other than digital video!)
Didée is offline  
Closed Thread

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 14:28.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.