QTGMC Deinterlacing Script (v3.32)

[sadie]

A little update. I tested (Preset="Fast", SourceMatch=1, Sharpness=0.30) and found it hardly any slower. Fine. But the visual quality is inconsistent. In an amorphous natural setting, detail in foliage seems indeed a bit crisper. Yet in an urban more geometric context, I find the contrary. Building details seem slightly softer. Flat areas even more. Is this merely an issue with the sharpness setting? In which case, I'd be hard pressed at this point to distinguish between resurgent hidden detail and newly fabricated noise. Anyway, I don't wish to risk too much haloing as I'd like to attack a 720p upscale at some point. Secondly, I've found that the doubling/bobbing of the image merely reinforces that 'video effect' so I may stick to the SelectEven setting and suffer the occasional judder. Besides aren't the 100/120 hz TV functions one of the few options that actually work well on a set top? Also, in order to limit processing time I'm leaning toward just using a CQM mode (I've got 4000kb/sec to throw at it). What is the incidence of using single vs dual pass with QTGMC?

I've found a possible small bug (or one that is encoder related). SourceMatch mode in Avidemux will cause it to crash when a screen saver kicks in. This doesn't happen with any other of the modes I've tried. And it doesn't happen in VirtualDub using SourceMatch. Any ideas?

The only systematic problem areas I have are those which involve NLE interference: slow motion & still captures. Improvement I suppose might marginally help by deinterlacing before operating those filters. To ask my previously posed Q differently, have they invented a way to import an avisynth function on a specific clip on a timeline and not the whole project? Whatever happened to Avisynth Studio development? I'm still googling for that one...

[matfra]

Can someone tell me if its possible to add a pre-filtered clip for QTGMC that Tr2=3 can use. I would like to strengh the denoising. Any Idea ? Here's my current script.
Thanks

###############################
# I7 950 4.3 MHz @ 100% CPU #
# 0:25s @ 33,80FPS #
###############################
import("C:\Program Files (x86)\AviSynth 2.5\plugins\MDegrain5.avs")
import("C:\Program Files (x86)\AviSynth 2.5\plugins\SeeSaw.v0.3e.avs")
SetMTMode(5)
SetMemoryMax(768)
DirectshowSource("C:\1.avi")
SetMTMode(2)
QTGMC(SourceMatch=3,Lossless=2,EdiMode="nnedi3",tr0=2,tr1=0,tr2=3,rep0=4,rep1=0,rep2=4,Sharpness=1.00,MatchEnhance=1.0,NoiseBypass=2, NoiseRemove=1.0, DetailRestore=0.0, NoiseRestore=0.0, Sigma=1, NoiseDeint="generate")
AssumeTFF()
SelectEven()
Seesaw(nrlimit=0,nrlimit2=0,sstr=1.5,sootheT=0,sootheS=0,Slimit=3,Sdamplo=29,Spower=3,SdampHi=35)

[-Vit-]

Edit: @matfra: No, you can't provide a pre-filtered clip for TR2 - such a clip would have to be smart-deinterlaced already. Possible within the algorithm I suppose but I haven't tried.
____

Originally Posted by sadie

But the visual quality is inconsistent. In an amorphous natural setting, detail in foliage seems indeed a bit crisper. Yet in an urban more geometric context, I find the contrary. Building details seem slightly softer. Flat areas even more.

What you are seeing is what your source is like.

I had almost the exact same feeling when I first experimented source match - better on foliage, but softer on some edges/detail. I realized that I'd simply got used to the results of standard TGMC, which tends to strongly sharpen "real-world" edges/detail (as opposed to static logos, text etc), but in doing so it loses fine detail like foliage making it look plastic-like. DoubleWeave, Bob and NNEDI3 your source and compare stills with standard (Q)TGMC and source match QTGMC. But don't do it side-by-side, do it by switching between full-screen images. Believe me I've done this in exhaustive detail and with a huge range of sources (I should calculate some metrics really). With source-match, what you see is what you've got, although SourceMatch=1 is actually only half-way there...

Source-match is about accuracy, which is not quite the same thing as quality. Unlike standard (Q)TGMC source-match does not seek to denoise nor increase compressibility. It is about giving you the most representative shimmer-free deinterlace of your source without adjusting it in any way (so if you do find an example where source-match is actually further from the source than the standard algorithm I'd be very interested). In practice, I tweaked the application of the algorithm, so you can still do some denoising or sharpening, but both of these work against what source-match tries to do (not necessarily a bad thing - balancing requirements).

And sure the differences are fairly minor - I have been careful to highlight that. The best aspect is an increase in vertical detail - losing the very "interpolated" look of TGMC, especially with SourceMatch=2,3. This increased detail is unequivocal, but whether you can actual notice it or not is a different matter. If you can't then don't use source match. It really is for those who use a big screen/upscale and for pedants like myself. Makes it most appropriate for SD too.
_____

Crashes: I've noted earlier the various reasons for instability under MT which are to do with avisynth multithreading and caching - both of which become more problematic as scripts become complex. I am currently looking at some workarounds.
_____

"Double-rate "video" look": I have implemented some new settings to simulate a different shutter angle in the output (just the motion-blur implications) - for single or double-rate. It uses MFlowBlur, as discussed earlier. It does help with single rate output on my sources. As expected it suffers whenever MVTools struggles to analyze the motion, but for less chaotic sources it's not too bad. Main problem is that MVTools operates on blocks not exact shapes, so the motion blur "bleeds" into areas that are actually fairly static. I'm going to look if there are any ways to repair this bleeding then I'll post this version (later today or tomorrow)

[Boulder]

Originally Posted by -Vit-

Crashes: I've noted earlier the various reasons for instability under MT which are to do with avisynth multithreading and caching - both of which become more problematic as scripts become complex. I am currently looking at some workarounds.

Whenever you have something to test, I'm available. It seems that me going back to tsp's MT build only works in some cases. I usually have to lower the number of threads quite low and get much less than optimal performance.

[Didée]

Regarding "MT crashes", I've also fiddled with various things. Some things "improved", but when you start with, say, a 90~95% crash probability, and manage to come to a, say, 35-40% crash probability, then this might be an improvement, but it's still not usable.

Though ... almost all crashes disappear magically when I switch to SEt's latest Avisynth 2.6 build. Therefore I assume that all 2.5.x MT-builds are not stable. As simple as that.

Originally Posted by Didée

Question is - what sorts of "MT-related fixes" have been done to 2.6, that 2.5.8.5 is missing?

[Boulder]

With the current video in the pipeline, I've been experimenting with all three known MT builds but with bad results. It usually takes 10-15 minutes to crash no matter what I do I usually get better results by encoding to lossless ffdshow HuffYUV first but this time even that doesn't work.

Mind you, my dual core was much more stable but that was probably simply because of less threads utilized

[sadie]

Well, Vit, I finally hit at least one nail on the head. "Accuracy vs quality". You know there's the spark of a major philosopical debate in there somewhere. You suggest that SourceMatch is for "those who use a big screen/upscale". Is that to say that my above concern about haloing or bordering would be mitigated by sticking to SourceMatch if I were to go 720P? All in all, in real time viewing the 'foliage' improvement outweighs the 'non-resharpened' scenes so maybe I'll stick with it. I'll give SourceMatch 2 a try as well. Did I mention that avidemux crash was in single-thread mode?

[matfra]

Originally Posted by -Vit-

Edit: @matfra: No, you can't provide a pre-filtered clip for TR2 - such a clip would have to be smart-deinterlaced already. Possible within the algorithm I suppose but I haven't tried.

I hope you could add this new feature in your next build of QTGMC. I would be nice to specify a Pre-sourceNR to the Tr2. I hope my cry will be listen by a master !
Thankx

[Didée]

Originally Posted by matfra

I would be nice to specify a Pre-sourceNR to the Tr2.

Anyone explain to me what this whould be supposed to do? I don't see it.

Though, my guess is you're looking at the wrong place for a solution. (Solutions are good, but they need to fit to the problem.)
You're working with a VHS capture?

[-Vit-]

I figured matfra might be looking for one of those situations where you use a different source and super clip to enhance/reduce something based on the motion match. But my sentence is rather vague because I too have no real idea what is being suggested. Matfra: QTGMC cannot solve all problems.
____

Crashes: I have seen plenty of examples (from numerous systems) indicating that no MT version of avisynth is stable, neither 2.58 nor 2.6. Crashes vary by script, plugins, avisynth version, software, OS and hardware.

The caching in avisynth is overeager. It's a real problem in MT, but could affect non-MT too, especially HD. By default avisynth caches every intermediate filter result, which number in the hundreds per-frame in QTGMC. It also caches every output frame, which is redundant when encoding. However, the system is smart and will attempt to drop unneeded frames from the cache. Unfortunately it can think that unneeded intermediate results are actually important and will continue to eat more and more memory. Until it runs out of memory and crashes (and if you're using a 32-bit workflow, you have a memory cap of 2Gb per process regardless of physical memory).
____

Haloing & Source Match: Haloing in (Q)TGMC is caused by the Sharpness setting, which recovers the source sharpness after the initial shimmer-removing temporal blur. The standard algorithm has a high default Sharpness intended to keep static areas accurate, but potentially producing significant halos elsewhere. The source-match approach recovers most of the source sharpness without resharpening, so the Sharpness setting is just 0.2 producing minimal haloing. You can set it even lower with the better source match modes + lossless (about 0.05), giving no haloing at all (other than what is in the source).

However, if you want to increase sharpness from the source... well, increase the Sharpness delicately with source-match as there's no limiting by default (because sharpness limiting also limits the accuracy of source-match).

[Didée]

The standard algorithm has a high default Sharpness intended to keep static areas accurate, but potentially producing significant halos elsewhere.

In theory it shouldn't. The sharpened intermediate clip is limited by the (compensated) neighbor frames. Therefore the sharpening shouldn't introduce new halos on its own ... the limiting makes processed pixels stay in range produced by original source pixels.

Practice is a little different. First, even such limiting can make already-present halos more prominent. (As known from LSF - even with overshoot=0, source halos are more obvious after LSF).
Another point is the pel subsampling ... pel=2 is sharp, and to that extend, it is acting quite similar to a "sharp" resizer like e.g. lanczos4resize. Means, pel=2 will also introduce some ringing, and will amplify already-existing halos.

I don't claim that the "pel2hr" option in TGMC is particularly good. But it is there for a reason. Partly to help with the problem ... and partly to indicate that the script creator was aware of the problem from the very start.

[-Vit-]

No criticism intended Didée, just communicating practicalities. I, more than anyone perhaps, appreciate the thought and detail that has gone into the TGMC algorithm.

[-Vit-]

I've posted the latest version v3.05 in the OP.

This version can simulate the motion blur of different shutter speeds using MFlowBlur as discussed earlier. In fact I decided to use shutter angle for units to avoid involving the frame-rate in the settings (didn't want to use MVTools non-standard percent either).

You need to choose an output frame rate and shutter angle, and also provide (or estimate) the effective shutter angle of your source. It's all switched off by default, so it will not affect any other scripts. Here's an example (edited: I missed the vital MotionBlur setting when I first posted...):

# Output every 2nd frame (i.e. single-rate), source filmed with high speed shutter, standard "film-like" shutter angle for output
QuickTGMC( Preset="Slower", MotionBlur=1, FPSDivisor=2, ShutterAngleSrc=45, ShutterAngleOut=180)

FPSDivisor=1 means select every frame (double-rate - the default). FPSDivisor=2 means select one out of every 2 frames (single-rate output - SelectEven() isn't needed). FPSDivisor > 2 is allowed. If this sounds backwards, it's simply the same concept as SelectEvery for those of you who script. I will leave detailed discussions of shutter angle to other experts. In summary, higher shutter angles mean more motion blur. You can estimate shutter angle on a source by looking at the motion blur on a moving object [on a still]. Perfectly crisp is 0, full blur from previous frame to next frame is 360. 180 is considered the norm for a film-like look, many non-film sources I see are rather sharper than that. Artificial output angles > 360 are possible in limited circumstances (extra blurry).

Things to watch out for: the motion blur can bleed into static areas due to the block-based nature of MVTools, there are settings to help repair that (read the "Motion Blur" section in the script docs). These repairs are slightly switched on by default. If the motion analysis makes bad decisions then there's little can be done - there's no effective way of detecting analysis errors that also doesn't detect most of the motion blur as errors...

I did a quick test with progressive input and it worked too. Note that you would usually set FPSDivisor as 1 since progressive input comes out as same-rate so you want to keep every frame.

Results are better than I expected for non-chaotic input. I would certainly consider it for single rate output as it smooths out the stutter I get (the above example suits my sources). However, it's not going to work well on some sources, and in any case consider this feature experimental.

[Robert Martens]

My god, you work fast! A quick test on my own material suggests you've succeeded in your aim. I'd love to comment in more detail, but the relationship between framerate and shutter speed, the difference between mechanical and electronic shutters, and the difference between exposure time, angle, and percent are all way over my head. I managed to wrap my mind around the basic idea, but beyond blending two fields together I'm out of my league.

I've just started working on a small update to my own script, and with any luck I can incorporate the new QTGMC options in there somehow. Looks like I may have occasion to use the 'val' data type for one of my parameters after all.

Thanks for the effort, this sounds like it'll come in handy!

[sadie]

Great, Vit, I'll have to give it a whirl myself. At the risk of exposing my naivete on this I'll dare ask why the need for FPS=1 at all. I mean isn't the 'video effect' a two-fold phenomenon? Either you have single-rate output with 'pseudo-motion blur' and judder or you go the double-rate route and suffer the full impact of zero temporal/motion blur while correcting the judder. So inasmuch as FPS=2 all at once alleviates the judder by emulating a true motion blur doesn't that obviate the need for going to double-rate (FPS=1) in the first place? There's no longer any judder for the doubling to correct. Or is it strictly meant for progressive sources? Be so kind as to tell me what simple principle I'm not connecting with. Thanks.

SourceMatch=2, Sharpness=0.30 on two samples gave me images that oddly closer resembled both the default TGMC test AND the source. SourceMatch=1 even transformed a woman's open mouth into the shape of an old black & white TV. Uncute. Yet SM=2 meant twice the encoding time. I'm wondering again what I was seeing in my foliage stuff. Oh well, more testing. And for something that in real time viewing won't be discernible for 99% of the population. Still, great product of yours (and Didee's).

[matfra]

Originally Posted by -Vit-

I figured matfra might be looking for one of those situations where you use a different source and super clip to enhance/reduce something based on the motion match.

You understand right -Vit-. I would like to know if it possible to add to the script athe option to use a different source and super clip to enhance/reduce something based on the motion match.

[-Vit-]

sadie: Double-rate material can and does contain motion blur, how much depends on the shutter speed used when filming it. But interlaced content shot with a shutter speed suitable for double-rate may not have enough motion blur for single rate, making it judder when you drop half the frames. And that's the primary use for these settings. But that's not their only use, they can smooth material shot with a very high shutter speed, even whilst keeping double rate. They can be used to give exaggerated blur, or to blur something that would normally be perfectly sharp (e.g. game footage). As a scripter, I prefer not to care about the why, and just provide for everything that's possible. The formula used is general, and supports all FPSDivisor & shutter angle values with a single piece of script.

Also, FPSDivisor=1 doesn't exactly mean double-rate, it means use every frame (field). In the case of progressive input, you would need FPSDivisor=1, or you would drop half the frames. [personally I don't like the moniker double-rate, though I understand why it has come about. If a video contains 60 images per second then that's its rate as I see it. Interlacing is just a form of compression scheme; if in decompressing you drop or merge half the images then that's half-rate, not single rate]

Standard TGMC will smooth the source, which you might prefer. I spent some time wondering why a line in one shot was aliased with source match but not without, until I realized the source itself was aliased on that shot. Whilst source match occasionally does introduce errors, often it is the source itself that you are unsatisfied with, and the stronger temporal smoothing of the standard algorithm is what you want. [I knew that adding a feature billed as "precise" would cause problems]

matfra: You're joking, right? That sentence was deliberately too vague to be useful.

[sadie]

Before my last fine tune-up and final output and then moving on, I thought I'd pass on a last rather serious quirk I discovered yesterday. I don't think it's related to QTGMC, but since it occurred with me for the first time in this context...well, who knows?

Yesterday, I did a half a dozen consecutive CRF 18-26 range tests (avidemux 2.5.3 medium preset) with regular QGTMC fast preset and all came out with finger-width horizontal smears across the center of the screen, occasional smears elsewhere and even some flickering in those areas. Static areas were much more affected than scenes with movement in the subject or camera.

The problem is I've since tried to reproduce the effect and simply cannot. Mediainfo confirms exact same settings for identical encodes. The only possible explanation I can imagine for this is that I had a few firefox windows open and was doing moderate surfing, but memory use remained well under 2gb (4gb ram installed). I'm not especially reassured that it hasn't resurfaced. Anyway, to whom it may concern...

[-Vit-]

sadie - non-reproducible problem: I believe all the plugins used by QTGMC are deterministic (i.e. produce the same output every time given the same input/settings). The script itself is deterministic. The bugs in avisynth may not be, the crashes certainly show up randomly. As you're running single-threaded it may be a case of memory corruption if you can't reproduce the error. I've processed hundreds of hours of footage with QTGMC and the only time I've ever seen non-reproducible effects was when I had a bad memory stick.

[henryho_hk]

Any sample snapshot, sadie?