Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion. Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules. |
27th January 2011, 18:22 | #181 | Link |
Registered User
Join Date: Feb 2009
Location: USA
Posts: 676
|
Silly question if you don't mind. If I feed it a progressive clip, is it going to make a "this is combed" decision and skip frames, or is it going to process every single frame in the clip?
There is a lot of material to read through, so I'm sorry if this was asked/answered before. I'm generally interested in using it to clean up IVTC'd material where the stream result is progressive obviously, but sporadic combing will occur. I could see using this for Clip2 operations, but if it doesn't do "needless" processing where there is no combing, it may be easier to just throw it on at the end of the script in some cases. edit: ok, its runs about 15 fps using "QTGMC( Preset="Medium", InputType=1 )" which is realistically fine I guess.. It seems to be pretty much identical to the source. So ignoring the fact I am deliberately slowing my script down more than I -need- to. If I wanted to use TFM and set PP to 0 (don't hint, don't attempt to deinterlace) and then just tack QTGMC on at the end, to catch and fix any combing (surprisingly I couldn't find any real combing but I used the most intensive match mode so thats probably why.) It would be a valid and correct usage, right? I just hate tweaking thresholds and all that stuff so I'd take a time increase vs having to tweak per file all the time. I'm sure I could adjust a few things to make it go faster and not ruin the quality, too. Last edited by osgZach; 27th January 2011 at 20:10. |
27th January 2011, 21:22 | #182 | Link |
Registered User
Join Date: Jul 2010
Posts: 448
|
The script makes no attempt to detect combing. It runs a temporal binomial smooth on every frame to remove bob-shimmer when deinterlacing. This same process is also effective in remove deinterlacing residue from progressive video. But it isn't selective on which frames to process. I've seen others run a QTGMC in your situation, but it's not a workflow I need so I can't really comment further.
|
27th January 2011, 23:02 | #184 | Link |
Registered User
Join Date: Apr 2002
Location: Germany
Posts: 5,391
|
What kind of "residual" combing? Low-value combing (from mpeg compression, because the matched fields come from different coded frames)? Or fully interlaced frames because of lone fields that couldn't be matched?
Because - when I hear (or see) "residual combing", my first thought is "vinverse". Not "QTGMC". ...
__________________
- We´re at the beginning of the end of mankind´s childhood - My little flickr gallery. (Yes indeed, I do have hobbies other than digital video!) |
27th January 2011, 23:45 | #185 | Link |
Registered User
Join Date: Feb 2009
Location: USA
Posts: 676
|
Oh jeezus Didee, you scared me for a second. I thought I said "residual combing" somewhere and used the wrong term.. Bad Didee!
To Clarify: I am simply talking about combing left in frames either from doing no post-processing whatsoever (TFM PP=0), or combing that was not picked up after field matching for whatever reason (Mic below threshold, etc).. I'm not a "tweaker" at heart, I prefer "run it once and its done" operations, I guess. So I was just inquiring if it were a proper use or not. McBob/TGMC are just too slow for me, but QTGMC seems to be fast enough I don't mind waiting as much. IIRC AnimeIVTC uses Vinverse doesn't it? I have always been curious about it but its options have always been a little intimidating (no offense). And seems like "more tweaking" for this "non-tweaker at heart" |
28th January 2011, 19:16 | #186 | Link | |
Registered User
Join Date: Nov 2009
Posts: 12
|
What settings should I use to decomb and deinterlace this progressive content. I also want to remove noise/dirt artifacts that appear around the lips, that's pretty much the only place where they appear.
I noticed that deinterlace/jaggy lines and noise only appear when I downsize the video to 720p. I am new to the whole decoming and deinterlace in a video. And I don't really understand the difference between jagginess and deinterlace. Can someone explain it to me. I've read about it but I don't really get it. Also I applied this plugin to the video and it seems that coming appears in every frame: Quote:
|
|
28th January 2011, 23:40 | #187 | Link |
Registered User
Join Date: Jan 2011
Posts: 107
|
Has anyone had any success getting the Very Slow preset, with or without any variations, to work with any speed/stability using 3.0 on a Core i7+ system? At this point I've run 44 different tests in which I've altered SetMTMode and SetMemoryMax and every test produces either less than 1fps output (usually much less) and/or crashes. Varying EdiThreads doesn't seem to have any effect. The basic script I am using is:
Code:
SetMemoryMax(700) SetMTMode(5,4) Import("C:\QTGMC3.0.avsi") LoadCPlugin("F:\ffmpegsource-2.14-mt-avs64\ffms2.dll") FFVideoSource("D:\X.mkv") SetMTMode(2) Trim(1753, 3542) QTGMC(Preset="Very Slow", MatchPreset="Medium", SourceMatch=3, Lossless=2, MatchEnhance=0.6, Sharpness=0.25, NoiseBypass=2, NoiseRemove=1.0, DetailRestore=0.5, NoiseRestore=0.1, Sigma=3, NoiseDeint="Generate") Using similar QTGMC (slow, very slow) settings which Avisynth settings give you the best performance? What kind of system are you running? So far, all I can conclude is that it's either impossible to run the above QTGMC settings on any modern system or that there is something seriously wrong with my set-up. |
29th January 2011, 00:45 | #188 | Link |
Registered User
Join Date: Apr 2002
Location: Germany
Posts: 5,391
|
Way too complex for 1080 video. Are you even halfway aware of the amount & kind of operations that QTGMC is doing with these settings?
Running single-threaded, SetMemoryMax(1200) gives me 0.25 fps. SetMemoryMax(1400) gives 1.1 fps, however Vdub now has peaked at 1.85 GB. SetMemoryMax(1440) loads, but crashes upon seeking. Of course, such memory load doesn't allow for e.g. direct x264 encoding. I don't get the script to run using 4 Threads via SetMTmode(2), even setmmax(500) throws kernelbase after a few frames. Way too complex for 1080 video. And breaking a fly on a wheel, anyway.
__________________
- We´re at the beginning of the end of mankind´s childhood - My little flickr gallery. (Yes indeed, I do have hobbies other than digital video!) |
29th January 2011, 01:41 | #189 | Link |
Registered User
Join Date: Jul 2010
Posts: 448
|
I designed the settings to make a steady gradient from "as fast as possible" to "unnecessarily accurate". The slowest settings are all about ultimate precision and completely ignore speed as a factor. Also, none of the presets take any account of resolution, so you need to. Settings like yours are visually great for big upscales on SD, but unless you have a huge display, you simply won't see the difference at the very slowest settings at HD resolutions.
If you must get it to run, a 64-bit workflow is required (check) and with regards threading work upwards from 1 main thread + 1 EdiThread. I find having EdiThreads lower than main threads helps on my system (I use a 12Gb i7 930 and a similar 950). Render lossless, do your x264 in a second pass. Regarding tweaks other than the presets, you can get a speed boost from MatchEdi2="Bob". Also NoiseDeint="DoubleWeave" will be faster and might make little difference, depending on your source. Make sure your SetMemoryMax is tweaked - wrong setting there can slow things down. Check the presets table in the script for other ideas. From my experience, the stability problems come from three points: (a) Memory issues: 32-bit avisynth running out of memory (overeager / buggy caching?), the black art of SetMemoryMax, leaky plugins - thankfully these issues are less noticable on 64-bit (b) Buggy plugins: many plugins inherit threading issues from avisynth (see below), but some have other bugs too. At this very moment I'm rewriting AddGrainC from scratch because I think it is causing crashes for me, I can almost guarantee a crash if I try very complex HD scripts with NoiseDeint="Generate" (uses AddGrain), and I don't get them without. This may be your problem. Try setting NoiseDeint="DoubleWeave" or NoiseBypass=0. (c) Avisynth multithreading. In early experiments with QTGMC I had serious problems with crashes on 1080p even at medium settings. Traced the problem to the avisynth<->plugin threading (synchronization problems with ref counts usually). SEt has fixed some of these in his MT avisynth builds, but they affect "avisynth.h" which needs to be compiled into all the QTGMC plugins. I've done that with all the 32-bit versions and also added a hideous hack of my own, and it helped a lot. However, I still need to resolve a few issues with the 64-bit versions and test a bit more before I could post them (all VS2010 and IC2011 - some conversion issues need checking). So even if you can get a speed you can live with, it's certain that as your resolution and script complexity increases, so does your chance of avisynth instability. I have had to tone down scripts from the level I wanted because of this in the past. Last edited by -Vit-; 29th January 2011 at 02:08. |
29th January 2011, 01:56 | #190 | Link | |
Registered User
Join Date: Jan 2011
Posts: 107
|
Quote:
The more I read up on this plugin though the more impressed I am with what it's able to do so thanks to -Vit- for his work on it and to you Didée for the underlying TGMC. |
|
29th January 2011, 02:15 | #191 | Link |
Registered User
Join Date: Apr 2002
Location: Germany
Posts: 5,391
|
The "most important" application IMHO is on SD content, anyway. When SD content is cheaply bob/deinterlaced and viewed on a big screen, then you'll see the downsides from bad deinterlacing in all glory, because the upscale makes everything more objectable.
On 1080i, this is so much less of concern, because no upscale is taking place. Even the "simple" deinterlacers result in an acceptable viewing experience. If I personally use Q/TGMC on 1080i, then it surely is with pretty fast settings. Even the most simplistic settings give like, say, 80% of what Q/TGMC can achieve. And those 80% are already way ahead of what other deinterlacers can achieve. See this older sample from 1 year back: old TGMC with most-fast and most-basic settings, versus YadifMod with NNEDI2 interpolation. TGMC was much faster, still it let Yadifmod/NNEDI2 stand with egg in their face.
__________________
- We´re at the beginning of the end of mankind´s childhood - My little flickr gallery. (Yes indeed, I do have hobbies other than digital video!) |
29th January 2011, 16:08 | #192 | Link |
Registered User
Join Date: Dec 2007
Location: Germany
Posts: 632
|
Thanks for the new version, just tried it - looks great with the new SourceMatch on my MiniDV stuff.
But with another type of source that requires rep0=0 there is something odd I noticed: whenever I use any of the lossless modes, especially without SourceMatch (SourceMatch=0) I pretty much always have combing left in the output. With version 2.51 this was not the case (version 2.51's fake lossless "Lossless=1" was free of remaining combs for me). I also find the fake lossless mode to have a lot more remaining combing than the true lossless mode Lossless=1 (which doesn't make sense), even with the default rep0=4. Edit: Sorry, forgot to add: I'm using rep0=0 because otherwise static parts (Text) laying on top of the motion would have an ugly up and down bobbing occasionally on this source (is there a better way to fix this?). The described problem with combing never occurs with rep>0, and never with QTGMC 2.51. Edit: cleaned my post, got messed up somehow. Code:
QTGMC(2,2,1, 0,0,4, Lossless=2, SourceMatch=0) #lots of combs (but steady text) Last edited by TheSkiller; 29th January 2011 at 19:07. |
29th January 2011, 19:31 | #193 | Link |
MeGUI / AviSynth user
Join Date: Jul 2004
Location: England
Posts: 31
|
Excellent results
Tried this out today with one of my PAL VHS captures - really impressed with it. Using QTGMC with just the "Slow" preset has produced a massive improvement when compared with my current filter choice in AviSynth, which is MDegrain2i2 + Cnr2k + Yadif.
My only problem now is that I get a Visual C++ runtime error along the lines of "This application has requested the Runtime to terminate it in an unusual way", which happens after a short while when I try to do a 2-pass encode with x264 in MeGUI, which I am guessing is down to the program running out of memory. I've only tried SetMemoryMax() with a value of 512 so far, and that hasn't resolved the problem. Presumably, I will need to increase the value to 1024 or more, and close Firefox when the encode is running along with any other programs. Before it crashed, I was getting around 5.6 FPS on the first pass, which is fairly reasonable. I'm a bit surprised that even with SetMTMode(2) that I can't get more on a Core 2 Quad Q6600 - I suppose I could try a faster mode in QTGMC, or get myself a computer with a bit more power; I'm thinking along the lines of something from Cray, to be honest :P I put up some comparison screenshots and sample scripts - the best thing is the reduction of jagged edges on the graphics, which the old trio of filters couldn't really do much about: Original frame | AviSynth script MDegrain2i2 + Cnr2k + Yadif | AviSynth script QGTMC 3.0, Slow preset | AviSynth script Overall, though, it's a very good script, this - I think that QTGMC3 could even be better than Cillit Bang / Easy-Off BAM
__________________
When all else fails... "DOT CRAWL. RAINBOWS. CHROMA BLEEDING NOISE. They're a challenge for some household video cleaners - but not for CILLIT BANG. BANG! - and the VHS head switching noise is gone!" |
29th January 2011, 20:02 | #194 | Link | |
Registered User
Join Date: Jul 2010
Posts: 448
|
Quote:
I listed the range of things that causes stability issues a couple of posts above. I also posted some notes on performance and MT use. One other important thing I note in the OP is using the right version of AviSynth. I can only get extended stability with SEt's 2.6MT. Make sure all your other plugins are up to date too. A SetMemoryMax of 512 should be OK for Preset="Slow" alone (provided your overall RAM is not low), but maybe tweaking it might help. Sometimes it needs to be lowered. If you do get it stable, try tweaking the thread-count as described in that overview I mentioned. You may get better speeds with more or less threads. |
|
3rd February 2011, 13:57 | #195 | Link |
Registered User
Join Date: Jul 2009
Posts: 111
|
I have a basic question for you.
MT(""" QTGMC(tr0=0,tr1=1,tr2=0,EdiMode="nnedi3",rep0=4,rep1=0,rep2=4,sharpness=1.00) """,threads=4, overlap=4) Is it the best way to run this script with MT ? Also what is SetMemoryMax(). Does it really speed up the encoding ? Merci ! |
3rd February 2011, 14:03 | #197 | Link | |
Pig on the wing
Join Date: Mar 2002
Location: Finland
Posts: 5,733
|
Quote:
__________________
And if the band you're in starts playing different tunes I'll see you on the dark side of the Moon... |
|
3rd February 2011, 18:00 | #198 | Link |
Registered User
Join Date: Jul 2010
Posts: 448
|
@matfra
You should not use MT("...") with QTGMC, you should use SetMTMode(...). SetMemoryMax is not a speed-up generally - it sets a balance of memory for your script to work - you might not need it. Here is a template script, adjust it as you need: Code:
SetMemoryMax(700) # This line may not be necessary - depends on your settings, number of threads and available memory. Try 400,600,700,800,1000,1200... SetMTMode( 5, 4 ) # Use 4 threads, adjust to suit your machine WhateverSource(...) # FFVideoSource, DGDecode_mpeg2source, AviSource, whatever your source requires SetMTMode(2) QTGMC( ... , EdiThreads=2 ) # Set number of threads for NNEDI3 (more or less than setting above), add any other settings you require |
3rd February 2011, 18:31 | #200 | Link | |
MeGUI / AviSynth user
Join Date: Jul 2004
Location: England
Posts: 31
|
Quote:
As for the memory running out - on the last script I ran for a 6-minute video, I increased SetMemoryMax to 768 and this has eliminated crashes for the time being. However, I have recently had BSOD crashes with the MEMORY_MANAGEMENT error on Vista (SP2), so I might need to run the memory checker and possibly replace the RAM I have. The other filter I sometimes use is fizick's DeFreq for removing RF (diagonal line) noise, so I'll check and see if that will work harmoniously with QTGMC. EDIT: poisondeathray - my source video is all TFF, I have tested your suggestion and the 50 FPS output from QTGMC is now nice and smooth, thanks for the tip! The equivalent settings I used for Yadif were mode=1 (double framerate) and order=1 (top field first). I've also tested the output on some more noisy VHS clips which need DeFreq (placed before QTGMC) and they looks very nice as well. With those clips (thankfully only a few) there is still a small amount of background noise crawl and feint horizontal lines, although in the final output from x264 much of this isn't really visible anyway. Probably I could mess around a bit with QTGMC's parameters in detail, beyond the standard Slow preset, but I'm more than satisfied with the results at the moment UPDATE: I've had a look at some stronger denoising to go with the detail restoration / deinterlacing from QTGMC. I've found that for my VHS content, I can get better results by using the MDegrain2i2 function before QTGMC(Preset="Slow") - normally I would just use one or the other. With both of these in one script, they work extremely well together, though it does hit encoding time to x264 somewhat (down to about 6.5 FPS instead of 12 FPS with just QTGMC @ Slow). Despite this, the results are definitely worth it. I have uploaded some standard size screenshots, 4x magnified sections of those screenshots and my most recent AviSynth scripts here: QTGMC + MDegrain2i2, QTGMC alone and MDegrain2i2 alone
__________________
When all else fails... "DOT CRAWL. RAINBOWS. CHROMA BLEEDING NOISE. They're a challenge for some household video cleaners - but not for CILLIT BANG. BANG! - and the VHS head switching noise is gone!" Last edited by STC-Fan; 6th February 2011 at 17:34. Reason: Tested suggestion from poisondeathray, other filter testing, fixed URL |
|
|
|